Support OpTeX

[Witiko]

The Markdown package currently supports plain TeX, LaTeX, and ConTeXt. We should add support for further formats such as Petr @olsak's OpTeX.

[Witiko]

@olsak I can handle tasks 2–4, but I would appreciate help with task 1 (Define renderers for OpTeX).

In the user manual, there is a list of all renderers in the Markdown package. We have some definitions for plain TeX (see markdown.tex) that will be inherited by OpTeX but anything more complicated (citations, tables, headings, …) needs to be defined for each format separately (see definitions for LaTeX in markdown.sty and ConTeXt in t-markdown.tex).

If I can get a list of definitions such as the following, I can take it from there.

\def\markdownRendererHeadingOnePrototype#1{\tit #1^^M}%
\def\markdownRendererHeadingTwoPrototype#1{\chap #1^^M}%
\def\markdownRendererHeadingThreePrototype#1{\sec #1^^M}%
% ...

With correct definitions, we should be able to typeset the document example.md in OpTeX as follows:

\input markdown

% Options
\def\markdownOptionContentBlocks{true}%
\def\markdownOptionDebugExtensions{true}%
\def\markdownOptionDefinitionLists{true}%
\def\markdownOptionFancyLists{true}%
\def\markdownOptionHashEnumerators{true}%
\def\markdownOptionInlineNotes{true}%
\def\markdownOptionNotes{true}%
\def\markdownOptionPipeTables{true}%
\def\markdownOptionRawAttributes{true}%
\def\markdownOptionSmartEllipses{true}%
\def\markdownOptionStrikeThrough{true}%
\def\markdownOptionSubscripts{true}%
\def\markdownOptionSuperscripts{true}%
\def\markdownOptionTableCaptions{true}%
\def\markdownOptionTaskLists{true}%

% Renderer prototypes
\def\markdownRendererHeadingOnePrototype#1{\tit #1^^M}%
\def\markdownRendererHeadingTwoPrototype#1{\chap #1^^M}%
\def\markdownRendererHeadingThreePrototype#1{\sec #1^^M}%
% ...

% Example document
\fontfam[LMfonts]
\markdownInput{example.md}
\bye

[olsak]

I mean that \def\markdownRendererHeadingTwoPrototype#1{\chap #1^^M} will not work because ^^M is TeX-unfriendly character. OpTeX scans the line after \chap, \sec etc to its end in verbatim mode, i.e things like "verbatim in title" are read as they are and the parameter is tokenized later when it is needed. So, user can write something like

\sec We talk about the `{` character

and it will work in titles, outlines, tables of contents etc. (Something similar is impossible in LaTeX.). But we can throw away this feature of scanning to the end of line and use internal OpTeX macros for markdown package, i.e.

\def\markdownRendererHeadingOnePrototype#1{\_printtit{#1}}
\def\markdownRendererHeadingTwoPrototype#1{\_inchap{#1}}
\def\markdownRendererHeadingThreePrototype#1{\_insec{#1}}
\def\markdownRendererHeadingFourPrototype#1{\_insecc{#1}}

I'll take look to another prototypes...

[olsak]

I tried to \input markdown in OpTeX but there are problems. It does \input expl3-generic and this macro file cannot be loaded in OpTeX without additional tricks. I hope that we need not to load this macro file in OpTeX, but I can mention the tricks.

1. expl3-generic loads a lua code which expects luatexbase.new_luafunction is defined. But OpTeX has its own luatexbase and tis function isn't defined in it. So, we need

\directlua{
   function luatexbase.new_luafunction(name)
      return \string#lua.get_functions_table() + 1
   end }

2. \foo_ macros are inaccessible in OpTeX by default, you need run \mathsboff before \input expl3-generic
3. expl3-generic expects that \newread and \newwrite are defined as in plain TeX but it is not true. We need

\def\newread  {\_newread}
\def\newwrite {\_newwrite}

4. expl3-generic needs the control sequence \e@alloc@ccodetable@count.

\slet{e@alloc@ccodetable@count}{_catcodetablealloc}

Sumarry: the following code must be inserted before \input markdown:

\directlua{
   function luatexbase.new_luafunction(name)
      return \string#lua.get_functions_table() + 1
   end }
\mathsboff
\def\newread  {\_newread}
\def\newwrite {\_newwrite}
\slet{e@alloc@ccodetable@count}{_catcodetablealloc}

\input markdown

\markdownBegin
Hello *world*!
\markdownEnd

\bye

[Witiko]

I hope that we need not to load this macro file in OpTeX, but I can mention the tricks.

The Markdown package exposes a Lua module, so it is definitely possible to make the OpTeX layer sit directly on top of the Lua layer unlike the ConTeXt and LaTeX layers, which sit on top of the Plain TeX + expl3 layer:

This would honor the spirit of the OpTeX format (minimalistic, fast, no-nonsense). It would reduce code reuse, but that may be a good thing, because it allows us to reimagine e.g. the option-passing from TeX to Lua. Here is how you could use the Markdown package via Lua in OpTeX:

\directlua{
  local ran_ok, kpse = pcall(require, "kpse")
  if ran_ok then kpse.set_program_name("luatex") end
  local convert = require("markdown").new()

  function markdown(input)
    local output = convert(input)
    tex.print(output)
  end
}

\let\markdownRendererDocumentBegin\relax
\def\markdownRendererEmphasis#1{{\em #1}}
\let\markdownRendererDocumentEnd\relax

\directlua{ markdown"Hello *world*!" }

\bye