Support email file formats: `.eml` and `.msg`

[walsha2]

Ability to parse .eml and .msg files would be great.

docling supports this:

MIME (.eml) and Outlook (.msg) email messages; attachment names can optionally be listed via EmailBackendOptions

[itsaaronngan]

I have this working on a fork. Would you like a PR?

Shape

Hand-rolled MIME on std + encoding_rs, no new dependencies. Follows csv.rs: parse(bytes) -> Result<Document, ConvertError>, produces the document model, does not touch the Markdown writer.

• RFC 5322 headers, continuation lines unfolded
• multipart traversal, depth-capped, prefers text/plain
• quoted-printable and base64 transfer-decoding
• RFC 2047 encoded-words, both B and Q
• charset via encoding_rs, mirroring csv.rs::decode
• Subject as H1, From/To/Cc/Date as a metadata paragraph
• extension-only detection, no detect.rs entry, per the CSV precedent for signature-less text formats

Registered in the node, wasm and python enums and in node/cli.js FORMATS.

A body that is text/html with no text/plain alternative returns Unsupported rather than emitting markup as prose. That case wants #52 or #53 landing first; I did not want to guess at HTML here.

Validation

142 real .eml files, Outlook/Exchange via Mimecast: 141 convert, median 4.0 ms.

The one failure is a calendar invite whose text/plain part is 2 bytes and whose content is text/calendar. It returns Unsupported rather than emitting the 2 bytes. Correct as far as I can tell, but it is a judgement call.

• cargo test --locked (307 passed, 1 ignored; 13 new unit tests)
• cargo clippy --workspace --all-targets --all-features -- -D warnings
• cargo clippy -p anydoc-wasm --target wasm32-unknown-unknown -- -D warnings
• cargo fmt --all --check

That corpus is private, so the fixtures are synthetic: seven handmade files under tests/fixtures/eml/, plus the fixtures_detect_from_content entry. Writing them turned up four bugs I had missed against real mail, the worst being that a whitespace-only line (one space, not empty) did not break a paragraph, so it rendered as a hard break with nothing after it. 3267 occurrences across the corpus, now zero.

Out of scope for this

• .msg, which is OLE-based and wants a separate frontend against cfb
• attachments; currently ignored, not listed
• fuzz/fuzz_targets/eml.rs, happy to add it if you want it in the same PR

Questions

1. Do you want this as a PR, or would you rather implement it yourself? Either is fine.
2. Headers as a metadata paragraph, or something else? A table reads as tabular data, which these are not.
3. Quoted reply chains: I preserve them verbatim. Stripping needs client-specific heuristics, which reads against what detect.rs says about heuristics.
4. text/calendar: leave as Unsupported, or extract?

Branch: https://github.com/itsaaronngan/anydoc/tree/main (three commits on top of v0.2.3)

[vaibhavdabas16]

@itsaaronngan I have opened #160 for the .msg half only — deliberately not touching .eml, since you have that written and validated against a real corpus and duplicating it would help nobody. Your comment scoped .msg out as wanting a separate frontend against cfb, so that is what #160 is. The two are independent frontends and can land in either order.

A few things I took from your write-up rather than rediscovering:

• Envelope as a metadata paragraph rather than a table, so the two frontends agree on shape if both land.
• The whitespace-only line bug. A "blank" line padded with a single space ended the paragraph with a hard break onto nothing. Handled from the start here because you flagged it.
• Unsupported for a body that is only HTML or RTF, rather than emitting markup as prose.

No new dependency either — anydoc already opens compound files with cfb for legacy .doc.

One overlap worth naming for whoever reviews: if both land, .eml and .msg will each have their own copy of the envelope-and-body rendering. That is a small shared helper once the second one is in, and I am happy to do that follow-up in whichever order you two prefer.

[itsaaronngan]

@vaibhavdabas16 Thank you for this, genuinely. I am fairly new to contributing to other people's codebases, so having someone read the write-up carefully and split the work cleanly rather than race me to it is more encouraging than you probably intended it to be. I mostly want this to end up useful to people other than me.

That split is right, and I am glad the whitespace-only line bug saved you some time. It only shows up against real mail, where a "blank" line padded with a single space is common enough to be everywhere and invisible until you diff the output.

.eml is now up as #164, rebased onto v0.2.4.

On the shared envelope helper: yes, and I think you should take it. You will have written the second frontend by then, which is the better vantage point for judging what actually generalises versus what merely looks similar. If you would rather not, I am happy to pick it up. Either way it wants both landed first, so nothing blocks here.

One difference between the two worth flagging for whoever reviews, since it is a judgement call rather than an oversight: I left .eml on extension-only detection with no detect.rs entry, reading the README's CSV rule as the precedent for signature-less text formats. .msg is a different case, since OLE has a real signature to match on, so I do not think we actually disagree. If the maintainers would rather .eml sniffed too, it is a small addition and I will make it.