Skip to content

Escape all Typst markup characters in typst output - #122

Open
soodoku wants to merge 1 commit into
py-econometrics:mainfrom
finite-sample:fix/typst-escape
Open

soodoku wants to merge 1 commit into
py-econometrics:mainfrom
finite-sample:fix/typst-escape

Conversation

@soodoku

@soodoku soodoku commented Nov 8, 2026

Copy link
Copy Markdown

What problem do you want to solve?

make(type="typst") escapes \ [ ] # * < > _, but not the rest of Typst's markup-mode syntax (https://typst.app/docs/reference/syntax/). Ordinary text in a label, cell, caption or note therefore either stops the document from compiling or is rendered as markup:

import pandas as pd
import maketables as mt

df = pd.DataFrame({"Income ($)": ["1"]}, index=["contact a@b.org"])
src = mt.MTable(df, notes="Costs in US$ 2020").make(type="typst")
open("t.typ", "w").write(src)
# $ typst compile t.typ
# error: unclosed delimiter

Observed with typst 0.13 for each text slot (cell, row label, column header, spanner, row-group name, notes, caption):

text result on main
Costs in US$ 2020 compile error: unclosed delimiter (math)
contact a@b compile error: label <b> does not exist (reference)
code `x compile error: unclosed delimiter (raw text)
a // b compile error: the comment swallows the cell's closing ]
a /* b compile error in headers, captions and row-group names
Model*, *bold compile error in headers, captions and row-group names (unclosed strong)
a \* b compile error everywhere: the \ is doubled first, leaving a bare *
~0.5 renders as 0.5 (~ is a non-breaking space)
- sub, + ctrl, = base, / term: x render as a bullet, 1. ctrl, a heading, a term list

Expected: the text prints as typed. (The docs' own Annual Salary ($) label in UsingLabels.ipynb hits the first case if rendered to Typst.)

Fix

All in _escape_typst, through which every text slot already passes:

  • escape $, `, @ and ~ like #, <, > and _;
  • escape a / that starts // or /*, except after :, so URLs (https://...) are still lexed as links;
  • escape a leading =, -, + or / followed by a space, at the start of each line. A bare - or -0.12 is left alone, so Typst still typesets negative numbers with a minus sign;
  • where asterisks are passed through for *bold* (headers, spanners, row-group names, caption), escape them only when their count is odd, i.e. when one would be left unclosed;
  • protect a user-written \* before backslashes are doubled, so it renders as *, which is what the old comment said the lookbehind was for.

Left as is: smart quotes, --/--- dashes and .... Typst typesets these like LaTeX does, and they compile.

Tests

  • test_typst_markup_characters_render_literally: 12 texts × 7 slots. Each one compiles with the typst CLI and checks that the text appears unchanged in the PDF text (pdfplumber).
  • test_typst_escaped_asterisk_renders_as_asterisk (cell, header).
  • test_typst_url_in_notes_stays_a_link: a guard that a URL is left unescaped and still prints.

The tests use the existing needs_typst skip marker and compile_typst helper in tests/api/test_output_formats.py. 77 of the 87 new cases fail on main and all 87 pass with the fix. The other 10 already pass on main and guard against regressions: a /* b, Model* and *bold in slots where asterisks were already escaped, plus the URL case. Full suite: 585 passed, and all 135 snapshots are unchanged. pre-commit and ty are clean.

Open question

This is a proposal that typst text is always literal, which is how \, _, # and < were already handled. The alternative is to let users write Typst math such as $beta$ in labels, as tex output lets them write $\beta$. That would need an opt-out, because $ in plain text (currency) is common and currently breaks compilation.

Not changed here: tab_label is inserted as a raw <label>, and typst_style values are inserted verbatim.

🤖 Generated with Claude Code

Problem: Typst output fails to compile when user text contains "$"
("Costs in US$ 2020": unclosed delimiter), "@" ("a@b": label does not
exist), a backtick, "//" or "/*" (comment swallows the cell's "]"), a
user-escaped "\*", or an unpaired "*" in a header/caption. "~" renders
as a space, and a leading "- ", "+ ", "= " or "/ " turns the cell into a
list, numbered item, heading or term.

Cause: _escape_typst only escaped \ [ ] # * < > _, not the rest of the
markup-mode syntax; "\*" was preserved by a lookbehind that runs after
backslashes have already been doubled.

Fix: escape $ ` @ ~ as well, comment openers (except after ":" so URLs
stay links), leading block markers, and unpaired asterisks where
asterisks are otherwise passed through for bold; protect a user "\*"
before backslash escaping. Every text slot (cells, row labels, headers,
spanners, row-group names, notes, caption) already goes through this
function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant