Skip to content

Friedland Chapter 6 and half of Chapter 7#798

Open
henrydingliu wants to merge 103 commits into
mainfrom
experimental
Open

Friedland Chapter 6 and half of Chapter 7#798
henrydingliu wants to merge 103 commits into
mainfrom
experimental

Conversation

@henrydingliu
Copy link
Copy Markdown
Collaborator

@henrydingliu henrydingliu commented May 16, 2026

Summary of Changes

looking to push the work so far on friedland before the pr becomes toooo large.

adding an index for friedland
adding ch6 and half of 7
adding all to the toc (sitting between user guide and example gallery)

Related GitHub Issue(s)

#588

Additional Context for Reviewers

please visit the experimental version on rtd review content for accuracy and legibility

not checking the box below because i didn't develop the rst locally. but tests are passing on the branch

  • I passed tests locally for both code (uv run pytest) and documentation changes (uv run jb build docs --builder=custom --custom-builder=doctest)

Note

Low Risk
Low risk: changes are documentation-only (new friedland section and TOC wiring) plus minor formatting tweaks, with no impact to runtime code paths.

Overview
Introduces a new documentation section docs/friedland/index.rst and adds two new RST chapters (chapter_6.rst, chapter_7.rst) that recreate Friedland exhibits via chainladder doctest examples.

Updates docs/_toc.yml to include the new Friedland section between the user guide and gallery, and makes small formatting-only tweaks to .github/CODEOWNERS and the docs extras list in pyproject.toml.

Reviewed by Cursor Bugbot for commit 3901057. Bugbot is set up for automated code reviews on this repo. Configure here.

henrydingliu and others added 15 commits May 9, 2026 02:16
* Update chapter_7.rst

* Update chapter_7.rst

* Create chapter_6

* Update chapter_7.rst

* Rename chapter_6 to chapter_6.rst

* Update _toc.yml

* Update chapter_6.rst

* Update chapter_7.rst

* Update chapter_6.rst

* Update chapter_7.rst

* Update chapter_7.rst

* Update chapter_6.rst

* Update chapter_7.rst

* Update chapter_7.rst

* Update chapter_7.rst

* Update chapter_6.rst

* Update chapter_7.rst

* Update chapter_6.rst

* Update chapter_6.rst

* Update chapter_7.rst

* Update chapter_6.rst

* Update chapter_6.rst

* Update chapter_6.rst
* REFACTOR: Reorganize type hierarchy of sparse.py. Move array-level functions to sparse.COO, and keep module-level functions to sp.

Replaces xp() with xp.COO() to create sparse arrays when xp used to be callable and was used as a constructor.

* FIX: Apply bugbot fixes.

* FIX: Apply bugbot fixes.

* FIX: Apply bugbot fixes.

* TEST: Add unit tests to chainladder.utils.sparse to cover missing lines.

* Expanded apriori docstring

* Improved docstring and added examples

* added code for doc tests

* Added a new checkbox to remind PR to run tests

* better command and surpressed some warnings

* Added example, removed reference to chainladder

* doctstrings with examples

* Added covergence to the n_iters parameter.

* docstring improvement and examples

* Added expectedloss method

* Improved docstrings and examples

* added load_sample

* Build(deps): Bump mistune from 3.1.4 to 3.2.1

Bumps [mistune](https://github.com/lepture/mistune) from 3.1.4 to 3.2.1.
- [Release notes](https://github.com/lepture/mistune/releases)
- [Changelog](https://github.com/lepture/mistune/blob/main/docs/changes.rst)
- [Commits](lepture/mistune@v3.1.4...v3.2.1)

---
updated-dependencies:
- dependency-name: mistune
  dependency-version: 3.2.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Improved docstring

* Clarification

* Update pyproject.toml

* Updated the example in fit

* v.0.9.2 release notes

* Added a last minute PR

* Added more releaes notes from older versions that were not included in the doc site

* Bumping pandas to 2.3.3 in preparation of pandas 3.0

* Small typo, thanks AF!

* One more approved PR (docstrings improvement)

* bumping pandas

* Fix Adjustments API page linking to chainladder.workflow (#762)

Closes #757

The Adjustments section header in docs/library/api.md was a copy-paste
of the Workflow section. The :mod: link and .. automodule:: directive
both pointed at chainladder.workflow, so the rendered RTD page for
Adjustments hyperlinked to the Workflow module page.

The four classes listed under Adjustments (BootstrapODPSample,
BerquistSherman, Trend, ParallelogramOLF) all live in
chainladder.adjustments, which is the correct target.

Co-authored-by: Nick Kinney <nkinney06@gmail.com>

* Annotate matplotlib dependency as required for TriangleDisplay.heatmap() (#761)

Per maintainer feedback on #758/#761: matplotlib is needed at runtime
for TriangleDisplay.heatmap() even though it isn't imported directly,
so it must remain a core dependency. Add an inline comment so future
contributors don't try to move it again.

Co-authored-by: Nick Kinney <nkinney06@gmail.com>

* Build(deps): Bump urllib3 from 2.6.3 to 2.7.0

Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.3 to 2.7.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@2.6.3...2.7.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.7.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updated date and some final merged PRs

* Add more installation options

* Added uv run in the pytest

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Gene Dan <genedan@gmail.com>
Co-authored-by: Kenneth Hsu <kennethshsu@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: wendy-w2029z <wj78919@gmail.com>
Co-authored-by: Kevin <74339271+SaguaroDev@users.noreply.github.com>
Co-authored-by: Nick Kinney <nkinney06@gmail.com>
* REFACTOR: Reorganize type hierarchy of sparse.py. Move array-level functions to sparse.COO, and keep module-level functions to sp.

Replaces xp() with xp.COO() to create sparse arrays when xp used to be callable and was used as a constructor.

* FIX: Apply bugbot fixes.

* FIX: Apply bugbot fixes.

* FIX: Apply bugbot fixes.

* TEST: Add unit tests to chainladder.utils.sparse to cover missing lines.

* Expanded apriori docstring

* Improved docstring and added examples

* added code for doc tests

* Added a new checkbox to remind PR to run tests

* better command and surpressed some warnings

* Added example, removed reference to chainladder

* doctstrings with examples

* Added covergence to the n_iters parameter.

* docstring improvement and examples

* Added expectedloss method

* Improved docstrings and examples

* New clrd from CAS

* added load_sample

* Build(deps): Bump mistune from 3.1.4 to 3.2.1

Bumps [mistune](https://github.com/lepture/mistune) from 3.1.4 to 3.2.1.
- [Release notes](https://github.com/lepture/mistune/releases)
- [Changelog](https://github.com/lepture/mistune/blob/main/docs/changes.rst)
- [Commits](lepture/mistune@v3.1.4...v3.2.1)

---
updated-dependencies:
- dependency-name: mistune
  dependency-version: 3.2.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Improved docstring

* Clarification

* Update pyproject.toml

* Updated the example in fit

* v.0.9.2 release notes

* Added a last minute PR

* Added more releaes notes from older versions that were not included in the doc site

* Bumping pandas to 2.3.3 in preparation of pandas 3.0

* Small typo, thanks AF!

* One more approved PR (docstrings improvement)

* bumping pandas

* Fix Adjustments API page linking to chainladder.workflow (#762)

Closes #757

The Adjustments section header in docs/library/api.md was a copy-paste
of the Workflow section. The :mod: link and .. automodule:: directive
both pointed at chainladder.workflow, so the rendered RTD page for
Adjustments hyperlinked to the Workflow module page.

The four classes listed under Adjustments (BootstrapODPSample,
BerquistSherman, Trend, ParallelogramOLF) all live in
chainladder.adjustments, which is the correct target.

Co-authored-by: Nick Kinney <nkinney06@gmail.com>

* Annotate matplotlib dependency as required for TriangleDisplay.heatmap() (#761)

Per maintainer feedback on #758/#761: matplotlib is needed at runtime
for TriangleDisplay.heatmap() even though it isn't imported directly,
so it must remain a core dependency. Add an inline comment so future
contributors don't try to move it again.

Co-authored-by: Nick Kinney <nkinney06@gmail.com>

* Build(deps): Bump urllib3 from 2.6.3 to 2.7.0

Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.3 to 2.7.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@2.6.3...2.7.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.7.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updated date and some final merged PRs

* Add more installation options

* Added uv run in the pytest

* feat(data): wire clrd2025 into load_sample, docs, and tests (#745)

Adds the clrd2025 branch in load_sample mirroring the existing clrd
config, but using the modernized CAS Schedule P column names
(IncurredLosses rather than IncurLoss). Updates the docstring's
complete dataset list and the sample-data documentation page. Adds a
targeted test asserting the six LOBs, modern column names, and origin
starting at 1998.

The underlying clrd2025.csv was added by @kennethshsu on branch #745.

Closes part of #745.

* Added the new clrd2025

* test(development): assert exclusions-ignored UserWarning in drop tests (#765) (#777)

* test(development): assert exclusions-ignored UserWarning in drop tests (#765)

Per #765, ~75 UserWarnings firing from chainladder/development/base.py
during tests that deliberately exercise the drop/exclusion path were
not being asserted. Wrapping each warning-producing call in
`pytest.warns(UserWarning, match=...)` makes the contract explicit:
the test fails if the warning stops firing OR if its message changes.

Per @kennethshsu's preference on the issue thread, used the
`pytest.warns` idiom uniformly (no blanket `@pytest.mark.filterwarnings`),
so each individual call site that should warn does so observably.

Affected tests (chainladder/development/tests/):
  - test_development.py: test_drophighlow (5 of 7 calls), test_new_drop_5,
    test_new_drop_5a (both _set_weight_func calls), test_new_drop_7,
    test_pipeline (both fits)
  - test_incremental.py: test_pipeline (both fits); added `import pytest`

The two `test_drophighlow` calls that don't warn (`drop_high=0` and
`drop_high=[T,F,T,F]`) were left untouched — wrapping them would
incorrectly require a warning that doesn't fire.

Verified locally: 671 passed, 12 xfailed (pre-existing). Targeted
6-test 'exclusions ignored' warning count drops from 10 to 4 in the
pytest summary (the remaining 4 are pytest's source-line accounting
from inside the `pytest.warns` blocks, not leaked warnings).

Refs #765.

* Annotate exclusions-ignored warning as test-asserted

Add an in-source comment above each warnings.warn(...) call for the
'Some exclusions have been ignored...' UserWarning, noting that the
message is matched by pytest.warns(..., match=...) in the development
test suite. Future edits to the warning text must update the
corresponding test matchers.

Addresses review feedback on #777.

* Fixed readme failed tests, improved badges and words #768 (#776)

* uncomment the test runs on push

* badges, rearrangement, words

* #686 (#772)

* Implemented policy_length

* undo the refactor

* added tests for policy_length

* Fixed bug to round D to int

* Fixed an issue on the tests

* Removed blank line

* Formatted with black

* Added more tests

* clarification

* Moved the parallelogram tests from utilities to parallelogram file

* Refactored leap and nonleap

* Improved docstring with assumption

* Refactored, all bugs resolved

* Cleaned up debugger

* Clean up, removed debugger

* Added a guard to prevent averaging nothing

* Added another simple example

* Added Henry's tests

* Fixed bugbot errors

* Added 2 new tests

* Removed the fstring

* Improved docstring and added two examples (#788)

* Improved docstring and added two examples

* Added a new test iat exceptions - thanks HL

* Allowing project to drop a threshold of 0.1%

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Gene Dan <genedan@gmail.com>
Co-authored-by: Kenneth Hsu <kennethshsu@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: wendy-w2029z <wj78919@gmail.com>
Co-authored-by: Kevin <74339271+SaguaroDev@users.noreply.github.com>
Co-authored-by: Nick Kinney <nkinney06@gmail.com>
Co-authored-by: Nick Kinney <Nick.Kinney4@gmail.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented May 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.19%. Comparing base (3a8c5d5) to head (3901057).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #798   +/-   ##
=======================================
  Coverage   86.19%   86.19%           
=======================================
  Files          86       86           
  Lines        4934     4934           
  Branches      642      642           
=======================================
  Hits         4253     4253           
  Misses        484      484           
  Partials      197      197           
Flag Coverage Δ
unittests 86.19% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@henrydingliu
Copy link
Copy Markdown
Collaborator Author

will squash and merge so it doesnt flood the entire commit history

@henrydingliu
Copy link
Copy Markdown
Collaborator Author

open to any and all feedback on the overall flow (some manual coding in the beginning; then more scalable functions towards the end)

also open to any and all wordsmithing on the actual texts in the rst.

@genedan
Copy link
Copy Markdown
Collaborator

genedan commented May 20, 2026

Just letting you know I started reading. I'll be reading along between work breaks so it might take a while for full feedback. But what really pops out is the amount of typing you had to do to get the exhibits formed, which you'd mentioned before.

You've revealed that there's still a big gap between getting the numerical results and having them being presentable, which means we really need to work on improving the I/O of the package to get people to use it. I think anyone, including myself would have to do at least as much work as you did to make those summaries.

This is a good candidate for having a summary method that displays the link ratios vertically alongside the column values and ultimate. Something like IBNR might be hard to incorporate due to different companies' field definitions, but I think even a bare bones combo of [origin, link ratio, triangle columns, ultimate] and then having the user have the option to insert their own additional columns/ibnr formula would go a long way.

So I'd vote +1 on an additional summary method and I think we can use these examples as a starting point.

@kennethshsu
Copy link
Copy Markdown
Collaborator

I agree that the amount of "words" is very distracting. I don't want to erase any of the work, and we can probably save the clean up for later, but I'd be perfectly happy with something simple like this.

tri = cl.load_sample("friedland_us_industry_auto")["Reported Claims"] #page 106, part I, triangle data
tri
tri.link_ratio #page 106, part II, age-to-age factors
cl.Development(average="simple", n_periods = 5).fit(tri).ldf_ #page 106, part III, simple average latest 5

I don't think we need to do any intentional rounding to match the text; however, we should perform intermediate rounding to match the text if the rounded numbers are used in a subsequent action.

Honestly, I think this is perfect for a first pass. Let's keep going and see what other methods we are missing so we can get those in. (Still working #272...)

@genedan
Copy link
Copy Markdown
Collaborator

genedan commented May 20, 2026

One suggestion for the title. How about "Friedland Walkthrough"? Friedland text sounds like we might be referring to the actual paper, but something along the lines of walkthrough would make it sound more clear that it's a CL application of the text. I'm open to different names too.

@kennethshsu
Copy link
Copy Markdown
Collaborator

"Reproducing Friedland text"?

@kennethshsu
Copy link
Copy Markdown
Collaborator

"Friedland in chainladder-python"?

@henrydingliu
Copy link
Copy Markdown
Collaborator Author

I agree that the amount of "words" is very distracting. I don't want to erase any of the work, and we can probably save the clean up for later

I'm completely open to other presentation styles. I'm hoping people can take a chapter or a section of a chapter and just crank out something in their style and we can vote on the most beginner friendly.

I don't think we need to do any intentional rounding to match the text; however, we should perform intermediate rounding to match the text if the rounded numbers are used in a subsequent action.

yes, I'm only doing rounding when I need the test output to match the text.

One suggestion for the title. How about "Friedland Walkthrough"?

What about 'Learning Reserving (Friedland)`

Since geomean is so close, I'd like to hold off on merging till that's ready.

@kennethshsu
Copy link
Copy Markdown
Collaborator

"Replicating Friedland's Text"?

@kennethshsu
Copy link
Copy Markdown
Collaborator

Oh oops already have something similar above haha

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants