0.4.0 Tooling API: legaldown.grammar, legaldown.syntax, and the answers tools re-derive today

[dvejsada]

Downstream tools (PactTrack, legaldown-render) import about 45 names from private modules, and re-derive what the validator already decides. This gives them a documented public surface for 0.4.0 that is covered by semver. Where the library already knows the answer, it now hands it over, so tools need fewer helpers.

No public behaviour changes. Every deep import path PactTrack and legaldown-render use keeps working. The new modules re-export the validator's own objects (grammar.X is validator.patterns.X), and the top-level legaldown.__all__ does not grow. Three internal names were renamed or dropped. Neither downstream repo uses them, and none of them was ever exported:

• the old internal validator.templates.Blank accumulator is now _BlankState;
• templates._DRAFTING_MARKER is now the public DRAFTING_MARKER;
• parser.iter_directives and parser.text_fragments were incidental imports and are gone.

Closes #31, closes #32, closes #33, closes #34, closes #88, closes #93. Covers the block and item lines of #27; Diagnostic.line already shipped in #85.

New public modules

legaldown.grammar: the language's constants and rules that need no document.

• Constants that were hard-coded or private before: PARTY_TYPES, MAX_SECTION_LEVEL, DRAFTING_MARKER, FINAL_CHECK_RULES. The validator now uses these constants itself, and its messages are unchanged.
• The directive, placeholder, question, duration and currency vocabularies, the identifier format, and the parser's depth limits.
• slugify_identifier(text, *, fallback="section") (Small public API: identifier fallback, final-check rules, one-directive choose check, quote body, grammar constants #34): fallback="" tells "nothing usable" apart from real text.
• choose_problem(directive, questions) (Small public API: identifier fallback, final-check rules, one-directive choose check, quote body, grammar constants #34): the first choose-invalid message, sharing one implementation with check_choose. It raises ValueError for a directive that is not a well-formed {{choose:}}.
• Value checks and conditions: parse_condition, condition_problem, exclusive, Presence, ALWAYS, and the is_valid_* helpers.

legaldown.syntax: reading source the way the validator does.

• The lexer: lex, Directive, Lexed, format_value.
  • Directive.positional_span and param_spans (Source spans for directive arguments, and the regions lex() blanks #32) give where each value is written, quotes included.
  • Lexed.literals (Source spans for directive arguments, and the regions lex() blanks #32) lists the comments and code spans the lexer skipped.
• iter_document_directives(document) → DirectiveLocation(section, block, fragment, directive) (Iterate a document's directives with their locations #33).
  • It covers every directive in the body, including the {{ref:}}, {{term:}} and {{def:}} the parser lifts into block fields; for those, fragment is None.
  • collect_source_directives is now built on it.
• Markers: find_markers(document) (the lexer argument is now optional), FoundMarker, parse_marker, format_marker, MARKER_RE.
• Fragments: block_fragments, list_fragments, text_fragments.
• Quotes: Quote, block_quotes, is_drafting_note.
  • quote_blocks(block) and drafting_note_blocks(block) (Expose a drafting note's content blocks without the [!DRAFTING] marker #88, Public API follow-up: quote content, fence helpers, answers reader, block lines #93): a quote's blocks as copies, and a drafting note's blocks without its [!DRAFTING] marker.
• code_content(block) → CodeContent(info, text, fenced) (Public API follow-up: quote content, fence helpers, answers reader, block lines #93): a code block read the CommonMark way.
• Layout:
  • SourceLayout, SectionSpan, HeadingSpan, BlockSpan, ItemSpan.
  • body_layout(text) lays out bare body text, such as an editor field, where a leading --- is a thematic break.
• The Markdown helpers: fences, dedent, indent_width, FRONTMATTER_RE, LINE_ENDING_RE, HTML_COMMENT_RE.

Answers in the result and the model

• result.index.blanks (Expose what validate_document computes and discards: section alternatives and blanks #31): Blank(id, type, fixed, in_frontmatter, consistent) for each placeholder id, for every document.
  • type is None when no occurrence has a usable type.
  • consistent is false when the occurrences have different types or fix two currencies or units, and fixed is only set when it is true.
• SectionIndexEntry.alternative (Expose what validate_document computes and discards: section alternatives and blanks #31): true when a section is an alternative to its preceding sibling with the same identifier, and so shares its number.
• Document.layout() (Public API follow-up: quote content, fence helpers, answers reader, block lines #93, Source positions on sections, blocks, and diagnostics #27): file lines of the frontmatter, headings, blocks and list items.
• Document.line_of(section, block=None, item=None): one of those lines. These are the lines diagnostics report. Both return None for a document built in code or changed since it was parsed.

Docs

The README gains a Tooling API section with both modules, examples, and a migration table from each private path to its public home. The old renderer-helpers table now points to that section. The result table documents blanks and alternative.

Verification

• 2961 tests pass with the spec fixtures corpus, 2210 without it. ruff is clean.

• tests/test_tooling_surface.py pins both __all__ lists, checks that every name is the implementation's own object, and maps each private import PactTrack and legaldown-render use today to its public home.

• Differential check against main: all 160 spec fixture documents, validated in normal and final mode, give byte-identical results. That covers every diagnostic (rule, level, message, line), the section index, the placed markers and collect_source_directives.

• An independent review fuzzed the new code. All of these held:

  • 125k directives: a span always replaced exactly that value;
  • 100k inputs for Lexed.literals;
  • code_content against a CommonMark parser;
  • line_of on 3000 random nested-list documents;
  • drafting_note_blocks against legaldown-render's current behaviour.

  Its should-fix findings are addressed in d42dc01.

🤖 Generated with Claude Code

https://claude.ai/code/session_013WmBAc5T7UCKVdpUmg9qxz