Skip to content

Implement the md-lines hook: one sentence per line in Markdown - #1

Merged
CodyCBakerPhD merged 10 commits into
mainfrom
claude/initial-implementation
Oct 8, 2026
Merged

CodyCBakerPhD merged 10 commits into
mainfrom
claude/initial-implementation

Conversation

@CodyCBakerPhD

@CodyCBakerPhD CodyCBakerPhD commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

What this is

A general pre-commit hook grown out of scripts/check_sentence_per_line.py in nwb-benchmarks-paper. Consumers add it with repo: + rev:; it also installs as the md-lines command. Python 3.13 or newer; consumers whose pre-commit runs under an older interpreter add language_version: python3.13 to the hook entry.

- repo: https://github.com/CodyCBakerPhD/md-lines-pre-commit
  rev: v0.1.0   # tag to create once this merges
  hooks:
    - id: md-lines

How it differs from the original script

  • Block structure comes from a CommonMark parser (markdown-it-py, plus tables and strikethrough). Only paragraph lines are reflowed, including paragraphs inside list items and block quotes. Headings, code, tables, HTML, link reference definitions, thematic breaks and YAML/TOML front matter are never touched. This fixes the two things the script got wrong on the runner README: a wrapped line starting with a Markdown link was never joined, and clauses after ;/: were left as fragments.
  • Prefix handling: new lines inside block quotes get the > marker, inside list items the item's indentation (also for nested lists and loose items). Lazy block quote continuations are normalized.
  • Sentence detection ignores punctuation in inline code, link destinations, autolinks, inline HTML, bare URLs and $math$; respects abbreviations (built-in list, --abbreviations, --abbreviations-file, --no-default-abbreviations), initials, dotted abbreviations, enumerators, bold lead-ins (**Note.**) and unbalanced bold markers, brackets or parentheses.
  • Options: --check, --diff, --split-lists, --keep-breaks-after (default .!?; add :; to keep the original script's behavior of breaking after colons and semicolons), -q.
  • Fidelity: CRLF, BOM and a missing final newline are preserved; the output is a fixed point (running again changes nothing). Exit codes 0 / 1 / 2.

Layout

  • src/md_lines/schema/md_lines.yaml: the data model (Options, Problem, Result, Paragraph) as a LinkML schema.
  • src/md_lines/models.py: generated from the schema with gen-python --no-metadata; a test fails when the two drift apart. Excluded from the formatters and from coverage.
  • src/md_lines/globals.py: the abbreviation list, the parser, the regular expressions and DEFAULT_OPTIONS.
  • src/md_lines/reflow.py: functions only. src/md_lines/cli.py: the command. src/md_lines/_version.py: reads the version from the installed metadata, so pyproject.toml is the only place it is written.
  • AGENTS.md ported from historia and applied: relative internal imports, whole-module external imports, keyword-only arguments on multi-input functions, ai_generated test marker, CHANGELOG.md.

Repository checks

pre-commit after historia: pre-commit-hooks, check-github-workflows, black, ruff (lint plus isort rules, --fix), codespell with the en-GB_to_en-US dictionary so the text stays in American English, and this repository's own md-lines hook on its Markdown. pre-commit.ci runs the hooks on every pull request, except the md-lines hook, which is language: system and skipped there. The Continuous Integration workflow runs the tests on the newest Python with coverage uploaded to Codecov (using the CODECOV_TOKEN repository secret), runs that one hook, and installs the hook the way a consumer does with pre-commit try-repo.

Validation

  • 52 tests (pytest), covering joins, splits, lists (kept, split, nested, loose, lazy), block quotes, every protected block type, front matter, hard breaks, inline masking, abbreviations, bold labels, CRLF/BOM, CLI modes and exit codes, the console entry point, and the schema/model agreement. 97% line coverage.
  • Compared against the original script on the runner repo's README: identical output except for the fragments the script left behind, which md-lines joins.
  • pre-commit run --all-files and pre-commit try-repo . md-lines --files README.md pass locally on Python 3.13.

After merging

Create the v0.1.0 tag so consumers can pin rev: v0.1.0. No LICENSE file is included; that choice is yours.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8

CodyCBakerPhD and others added 2 commits October 7, 2026 23:39
A general pre-commit hook grown out of the sentence-per-line script in
NeurodataWithoutBorders/nwb-benchmarks-paper.

- Block structure comes from markdown-it-py (CommonMark plus tables and
  strikethrough): only paragraph lines are reflowed, including paragraphs
  inside list items and block quotes; headings, code, tables, HTML, link
  reference definitions, thematic breaks and YAML/TOML front matter are
  never touched. A wrapped line that starts with a link or emphasis is
  therefore joined correctly, which the line-prefix heuristics of the
  original script got wrong.
- Within a paragraph, wrapped sentences are joined and lines holding
  several sentences are split. List items are joined but not split
  unless --split-lists is given; new lines reproduce block quote markers
  and list indentation. Hard line breaks are kept.
- Sentence ends are ignored inside inline code, link destinations,
  autolinks, inline HTML, bare URLs and $math$, after abbreviations
  (built-in list, --abbreviations, --abbreviations-file,
  --no-default-abbreviations), initials, dotted abbreviations and line
  enumerators, after a bold lead-in such as **Note.**, and inside
  unbalanced bold markers, brackets or parentheses.
- --check and --diff modes, --keep-breaks-after for authors who break
  after colons or semicolons, exit codes 0/1/2. CRLF, BOM and a missing
  final newline are preserved; the output is a fixed point.
- Packaged with a console script (md-lines) and a .pre-commit-hooks.yaml
  manifest so consumers use `repo:` + `rev:`; 48 tests; CI runs the
  tests on Python 3.9, 3.11 and 3.13, this repository's own pre-commit
  hooks (ruff, hygiene, md-lines on README.md) and a consumer-style
  `pre-commit try-repo`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8
@CodyCBakerPhD CodyCBakerPhD self-assigned this Oct 7, 2026
Comment thread src/md_lines/__init__.py Outdated
Comment thread src/md_lines/__main__.py Outdated
Comment thread src/md_lines/cli.py Outdated
Review feedback on #1: the version string was duplicated in
pyproject.toml and __init__.py. pyproject.toml is now the only place it
is written; __init__.py reads it through importlib.metadata, which is
what carries the pyproject version once the package is installed (also
for `pip install -e .`), and falls back to "0+unknown" in a bare
checkout. A test asserts the two agree.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8
Comment thread src/md_lines/cli.py Outdated
Review feedback on #1:

- requires-python >= 3.13, ruff targets py313, the `from __future__
  import annotations` shims are gone, and CI tests on the latest stable
  Python ("3.x") instead of a 3.9/3.11/3.13 matrix. The README notes
  that an older Python running pre-commit needs `language_version` on
  the hook entry.
- src/md_lines/__main__.py is removed: pre-commit runs the `md-lines`
  console script named in .pre-commit-hooks.yaml, so `python -m
  md_lines` was an extra entry point nothing here needs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8
Comment thread .github/workflows/ci.yml Outdated
CodyCBakerPhD and others added 3 commits October 7, 2026 20:47
Signed-off-by: Cody Baker <51133164+CodyCBakerPhD@users.noreply.github.com>
- src/md_lines/schema/md_lines.yaml defines Options, Problem, Result and
  Paragraph; src/md_lines/models.py is generated from it with
  `gen-python --no-metadata`, and a test fails when the two drift apart
- src/md_lines/globals.py holds the abbreviation list, the parser, the
  regular expressions and DEFAULT_OPTIONS; reflow.py keeps only functions
- Options.abbreviations is a list (multivalued slot) and Problem has no
  __str__, so the CLI formats line and message itself
- pre-commit after CodyCBakerPhD/historia: black, ruff (with its isort
  rules), codespell (with the en-GB_to_en-US dictionary, so the text
  stays in American English), check-github-workflows
- tests: the console entry point only, since `python -m md_lines` went
  away with __main__.py; CI installs the dev extra so the schema test runs
- README: Python 3.13 or newer, `language_version` for older pre-commit
  interpreters, and how the model is generated

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8
Review feedback on #1.

- AGENTS.md ported from CodyCBakerPhD/historia, with sections for this
  repository's data model (LinkML schema, generated models.py, globals)
  and the American English and sentence-per-line rules; CHANGELOG.md
  started with an Upcoming entry for this PR
- Internal imports are relative; external modules are imported whole
  (markdown_it, collections.abc, importlib.metadata); multi-input
  functions take keyword-only arguments, single-input ones positional-only
- __init__.py holds imports and __all__ only; the version lookup moves
  to _version.py
- Tests carry the ai_generated marker, registered in pyproject.toml
- .github/workflows/ci.yml renamed to continuous_integration.yml with
  name "Continuous Integration"
- pre-commit.ci skips the md-lines hook, which is `language: system` and
  therefore cannot run there; the GitHub Actions workflow runs it

Co-Authored-By: Claude Code 2.1.293 / Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8
@CodyCBakerPhD
CodyCBakerPhD marked this pull request as ready for review October 8, 2026 01:00
The tests job runs pytest with coverage and uploads coverage.xml with
codecov/codecov-action, failing when the upload fails, as in
CodyCBakerPhD/historia. pytest-cov joins the dev extra and the generated
models.py is omitted from coverage. actions/checkout and
actions/setup-python move to v7. docs/development.md names the
CODECOV_TOKEN secret the upload needs.

Co-Authored-By: Claude Code 2.1.293 / Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8

Copy link
Copy Markdown
Owner Author

The tests check is red on 2cc5030 only because of the new Codecov upload step: the tests themselves passed, then codecov-action logged "Token length: 0" and "Token required - not valid tokenless upload". The upload is set to fail CI on error, as in historia.

Nothing in the branch can fix this. The repository needs a CODECOV_TOKEN secret (Settings, Secrets and variables, Actions) holding this repository's upload token from Codecov. Re-running the job without it would fail the same way, so I have not re-run it. Once the secret exists, the next push or a manual re-run goes green.


Generated by Claude Code

Comment thread .github/workflows/continuous_integration.yml
Review feedback on #1. pre-commit.ci runs the hooks on every pull
request, so the Actions job no longer repeats them. It keeps the two
things pre-commit.ci does not do: this repository's own md-lines hook
(language: system, skipped there) and the try-repo installation test.

Co-Authored-By: Claude Code 2.1.293 / Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToiYPiSdU8NM9h1UmLgsq8
@codecov-commenter

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

Signed-off-by: Cody Baker <51133164+CodyCBakerPhD@users.noreply.github.com>
@CodyCBakerPhD
CodyCBakerPhD merged commit 9cb84d3 into main Oct 8, 2026
3 checks passed
@CodyCBakerPhD
CodyCBakerPhD deleted the claude/initial-implementation branch October 8, 2026 02:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants