fix(errors): the 6.0 warnings deduped per filing, not per call site (07lk.10) - #1037
Merged
Conversation
…07lk.10) warn_will_raise's docstring promised "a loop over ten thousand filings warns once rather than ten thousand times". It did not. Python suppresses a repeat only when the rendered text matches exactly, and the warning was built from str(error) — whose message interpolates the accession number, and whose suggestions list the filing's own items. Every filing produced a distinct string, so every filing produced a warning. Measured at the report[item] site: 500 filings, 500 warnings. Nobody saw it because none of this has been released, and because the property is invisible from a single call — it only appears at corpus scale, which is where the bulk and RAG users live. Those are exactly the users a FutureWarning campaign cannot afford to read as noise. The fix splits the two audiences of the message. EdgarError grows a warning_summary, stable for a given call site, and that is what the warning renders; the detailed per-filing message stays on the error, which is what strict mode raises today and what 6.0 will raise. The detail is not dropped, it moves to the copy the user actually debugs against. Four sites, each deduping on the bounded thing rather than the unbounded one: report[item] report class + item name, not the accession and item list filing.obj() form type, not the accession find() fixed text, not the caller's input TenK.document exception type, not the accession and parser message TenK.document is the one that could not simply drop its detail: a parse failure is an anomaly worth investigating, unlike a routine missing item. It keeps a bare raise under strict (so the parser's own diagnosis survives), and its accession and parser message now go to edgar.core.log while the warning dedups on the exception type. A parser regression across a form-year gives a handful of warnings and a complete log trail rather than thousands of stderr lines. Three regression tests pin the property, all of which fail against the code they replace: 500 filings must warn once at each of the two per-filing sites, and the accession must appear in str(error) while staying out of warning_summary — so a future dedup cannot be bought by losing the detail. Fast suite green in both modes: 4,950 passed lenient and strict. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
warn_will_raise's docstring promised:It did not. Python suppresses a repeat only when the rendered text matches exactly, and the warning was built from
str(error)— whose message interpolates the accession number, and whosesuggestionslist the filing's own items. Every filing produced a distinct string, so every filing produced a warning.Measured at the
report[item]site, same call site, default filters:Why it matters
The property is invisible from a single call — it only appears at corpus scale, which is where the bulk and RAG users live. A user looping a few thousand 10-Ks pulling
Item 1Awould get a few thousand stderr lines, and the flood scales with corpus size, so the people hit hardest are the ones we most want to keep. That reads as a broken library rather than a considerate one, which is a bad first impression for a warning campaign whose whole purpose is to be helpful.Nothing here has been released, so this is fixable at zero user cost — but only before 5.48.0.
The fix
Split the two audiences of the message.
EdgarErrorgrows awarning_summary— stable for a given call site — and that is what the warning renders. The detailed per-filing message stays on the error, which is what strict mode raises today and what 6.0 will raise. The detail isn't dropped; it moves to the copy the user actually debugs against.report[item]missfiling.obj()no XMLfind()malformedTenK.documentparse failThe one judgement call
TenK.documentcould not simply drop its detail: a parse failure is an anomaly worth investigating, unlike a routine missing item. It keeps its bareraiseunder strict (so the parser's own diagnosis survives — it deliberately does not route throughwarn_will_raise), and its accession and parser message now go toedgar.core.logwhile the warning dedups on the exception type, a bounded set. A parser regression across a form-year now gives a handful of warnings plus a complete log trail, instead of thousands of stderr lines.Verification
Three regression tests pin the property, all of which fail against the code they replace:
report[item]— reverting the one-line fix gives500 filings produced 500 warnings, with an assertion message naming the curefiling.obj()str(error)but not inwarning_summary— so a future dedup cannot be bought by silently losing the detailFast suite green in both modes: 4,950 passed lenient and strict.
Docs: the CHANGELOG entry and the error-handling guide were both silent on warning volume, which is precisely what the affected audience needs to know. Both now state it, and the guide gains an "If you process filings in bulk" section.
🤖 Generated with Claude Code