Repository navigation
html_block: lowercase declarations and Unicode case-folding in tag names diverge from CommonMark 0.31.2 #437
Description
Describe the bug
Two HTML-block start conditions in markdown_it/rules_block/html_block.py differ from CommonMark 0.31.2. They also differ from markdown-it 15.0.2 (JS), and markdown-it-py is a port of it. Both reproduce on 4.2.0 with MarkdownIt("commonmark").
1. A declaration opens an HTML block only on an uppercase letter. Spec 0.31.2, HTML blocks, start condition 4: "line begins with the string <! followed by an ASCII letter." Line 23 is re.compile(r"^<![A-Z]"), which is 0.30's uppercase-only rule. The inline rule (common/html_re.py:20, declaration = "<![A-Za-z][^>]*>") already accepts both cases.
para
<!doctype html>
| output | |
|---|---|
| commonmark.js 0.31.2 | <p>para</p> + <!doctype html> (an HTML block) |
| markdown-it 15.0.2 | same as commonmark.js |
| markdown-it-py 4.2.0 | <p>para\n<!doctype html></p> |
2. Tag names fold Unicode. Conditions 1 and 6 compile str patterns with re.IGNORECASE (lines 17, 18 and 26), so Python matches ſ (U+017F) as s, K (U+212A, Kelvin) as k, and ı/İ as i. The spec says: "A tag name consists of an ASCII letter followed by zero or more ASCII letters, digits, or hyphens."
para
<ſcript>
text
| output | |
|---|---|
| commonmark.js 0.31.2 | <p>para\n<ſcript>\ntext</p> |
| markdown-it 15.0.2 | same as commonmark.js |
| markdown-it-py 4.2.0 | <p>para</p> + <ſcript>\ntext (a type-1 block that runs to the end of the input) |
Possible fix
- Line 23:
^<![A-Za-z]. - Lines 17, 18 and 26:
re.IGNORECASE | re.ASCII.
Neither case is among the spec's 652 examples, which may be why #351 / #362 did not catch them.
Environment
markdown-it-py 4.2.0 (also 3.0.0), CPython 3.14, compared with commonmark.js 0.31.2 and markdown-it 15.0.2.
Activity