Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ npx @firecrawl/anydoc - --format csv < data.csv # read stdin
npx @firecrawl/anydoc scan.pdf --ocr hosted # scanned pages via Firecrawl Parse
```

`npx` downloads the prebuilt binary for your platform on first run. For a permanent `anydoc` command, install globally with `npm install -g @firecrawl/anydoc`. Run `anydoc --help` for all options.
`npx` downloads the prebuilt binary for your platform on first run. For a permanent `anydoc` command, install globally with `npm install -g @firecrawl/anydoc`, or build the same CLI from source with `cargo install anydoc`. It also ships in the `firecrawl-anydoc` Python package, so `uvx --from firecrawl-anydoc anydoc report.docx` runs it with no Node and `uv tool install firecrawl-anydoc` makes it permanent. Run `anydoc --help` for all options.

### Node.js

Expand Down
12 changes: 12 additions & 0 deletions python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,18 @@ anydoc converts locally and does not do OCR, so a PDF with scanned or image-only
markdown = anydoc.to_markdown("scan.pdf", ocr="hosted")
```

## Command line

The package installs an `anydoc` command, so the conversion is one step away with no Node:

```bash
uvx --from firecrawl-anydoc anydoc report.docx # Markdown to stdout
uvx --from firecrawl-anydoc anydoc slides.pptx -o slides.md
uvx --from firecrawl-anydoc anydoc - --format csv < data.csv
```

`uv tool install firecrawl-anydoc` (or `pipx install firecrawl-anydoc`) puts `anydoc` on the `PATH` for good; `python -m anydoc` works too. It is the same command line the `cargo install anydoc` binary provides: same options and the same exit codes (`0` success, `1` conversion error, `2` usage error). Run `anydoc --help` for the full list.

## Errors

A conversion raises only when no complete Markdown could come out of the file. The exception type names what went wrong:
Expand Down
20 changes: 20 additions & 0 deletions python/anydoc/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
"""The ``anydoc`` command line: convert one document to Markdown.

Runs as ``anydoc`` (a console script) and as ``python -m anydoc``. It hands
straight off to the same Rust command the ``cargo install anydoc`` binary
runs, so every front end takes the same options and returns the same exit
codes (``0`` success, ``1`` conversion error, ``2`` usage error).
"""

import sys

from anydoc._anydoc import _cli


def main() -> int:
"""Run the command line and return its exit code."""
return _cli(sys.argv[1:])


if __name__ == "__main__":
sys.exit(main())
5 changes: 5 additions & 0 deletions python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,10 @@ Homepage = "https://github.com/firecrawl/anydoc#readme"
Repository = "https://github.com/firecrawl/anydoc"
Issues = "https://github.com/firecrawl/anydoc/issues"

# Console script, so `uvx --from firecrawl-anydoc anydoc <file>` and
# `uv tool install firecrawl-anydoc` both give the Node-free CLI.
[project.scripts]
anydoc = "anydoc.__main__:main"

[tool.maturin]
module-name = "anydoc._anydoc"
13 changes: 13 additions & 0 deletions python/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -195,9 +195,22 @@ fn to_document(
document::document(py, parsed)
}

/// Run the `anydoc` command line and return its process exit code. `argv` is
/// the argument list without the program name. Output goes to this process's
/// own stdout/stderr, so this is only useful from the package's console entry
/// point (`anydoc`, `python -m anydoc`); it is the same command the
/// `cargo install anydoc` binary runs. `OsString` keeps non-UTF-8 filenames
/// intact, so they reach the converter (and fail as I/O) just as they do
/// through the binary, rather than raising here.
#[pyfunction]
fn _cli(py: Python<'_>, argv: Vec<std::ffi::OsString>) -> i32 {
py.detach(|| anydoc::cli::run(argv))
}

/// Convert documents to GitHub-Flavored Markdown.
#[pymodule]
fn _anydoc(m: &Bound<'_, PyModule>) -> PyResult<()> {
m.add_function(wrap_pyfunction!(_cli, m)?)?;
m.add_function(wrap_pyfunction!(format_from_bytes, m)?)?;
m.add_function(wrap_pyfunction!(format_from_extension, m)?)?;
m.add_function(wrap_pyfunction!(format_from_path, m)?)?;
Expand Down
47 changes: 47 additions & 0 deletions python/tests/test_anydoc.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
import io
import json
import os
import subprocess
import sys
import threading
import unittest
import zipfile
Expand Down Expand Up @@ -152,6 +154,51 @@ def test_unreadable_files_and_bad_arguments_raise_the_python_exception(self):
with self.assertRaisesRegex(ValueError, "unknown format"):
anydoc.to_markdown_bytes(b"", "wat")

def test_the_command_line_converts_a_document_and_reports_its_exit_codes(self):
def anydoc_cli(*args, stdin=None):
return subprocess.run(
[sys.executable, "-m", "anydoc", *args],
capture_output=True,
text=True,
input=stdin,
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
timeout=30,
)

converted = anydoc_cli(str(OUTLINE))
self.assertEqual(converted.returncode, 0, converted.stderr)
self.assertRegex(converted.stdout, r"(?m)^# ")

# stdin has no extension, so signature-less CSV has to be named.
piped = anydoc_cli("-", "--format", "csv", stdin=CSV.read_text())
self.assertEqual(piped.returncode, 0, piped.stderr)
self.assertIn("| --- |", piped.stdout)

# No input at all is a usage error; an unreadable file is a conversion
# error; a scanned page needs OCR (exit 3). Each names itself on stderr.
usage = anydoc_cli()
self.assertEqual(usage.returncode, 2)
self.assertIn("missing input", usage.stderr)

unreadable = anydoc_cli("no-such-file.docx")
self.assertEqual(unreadable.returncode, 1)
self.assertIn("no-such-file.docx", unreadable.stderr)

needs_ocr = anydoc_cli(str(MIXED))
self.assertEqual(needs_ocr.returncode, 3, needs_ocr.stderr)

@unittest.skipIf(sys.platform == "win32", "POSIX-only non-UTF-8 filename")
def test_the_command_line_survives_a_non_utf8_filename(self):
# The byte reaches sys.argv as a surrogate; it must pass through to the
# converter and fail as a normal read error, not raise UnicodeError.
result = subprocess.run(
[sys.executable, "-m", "anydoc", b"no-such-\xff.docx"],
capture_output=True,
timeout=30,
)
self.assertEqual(result.returncode, 1)
self.assertTrue(result.stderr.startswith(b"anydoc: "), result.stderr)
self.assertNotIn(b"UnicodeError", result.stderr)

def test_the_stubs_cover_the_module(self):
stub = Path(anydoc.__file__).with_name("_anydoc.pyi")
stubbed = {
Expand Down
10 changes: 9 additions & 1 deletion skills/convert-documents-to-markdown/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,22 @@ metadata:

# Convert documents to Markdown

Run the anydoc CLI. It needs Node 20+ and no install:
Run the anydoc CLI. No install step. With Node 20+:

```bash
npx -y @firecrawl/anydoc <file> # Markdown to stdout
npx -y @firecrawl/anydoc <file> -o out.md # write to a file
npx -y @firecrawl/anydoc - --format csv < f # read stdin
```

Or, with Python instead of Node, the same CLI ships in `firecrawl-anydoc`:

```bash
uvx --from firecrawl-anydoc anydoc <file> # same flags, no Node
uv tool install firecrawl-anydoc # or install it for good
pipx install firecrawl-anydoc # same, without uv
```

Rules:

1. Supported inputs: `.doc`, `.docx`, `.docm`, `.odt`, `.rtf`, `.epub`, `.pdf`, `.ppt`, `.pps`, `.pot`, `.pptx`, `.pptm`, `.ppsx`, `.ppsm`, `.odp`, `.xls`, `.xlsx`, `.xlsm`, `.xlsb`, `.ods`, `.csv`.
Expand Down
12 changes: 12 additions & 0 deletions src/bin/anydoc.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//! The `anydoc` CLI: convert one document to GitHub-Flavored Markdown.
//!
//! The command itself lives in [`anydoc::cli`] so this `cargo install anydoc`
//! binary and the `firecrawl-anydoc` Python package's console script share one
//! implementation and stay interchangeable: same options, same help, same
//! exit codes as the npm CLI (`node/cli.js`).

use std::process::exit;

fn main() {
exit(anydoc::cli::run(std::env::args_os().skip(1)));
}
Loading