[Bug] Hybrid auto mode: TableFormer (hardcoded ACCURATE) collapses table rows on dense datasheets — orphan-cell fallback corrupts structure (cf. #627)

[acute1110]

Summary

Hybrid mode (docling-fast backend) corrupts table structure on dense,
multi-column engineering datasheets: logical rows are collapsed, cells are
misaligned, and one table's dimensions changed from 1x4 (local) to 10x4
(hybrid). The server log shows the direct mechanism — dozens of
Orphan pdf_cell ... recovered to row=N by nearest-row fallback warnings
from docling_ibm_models/tableformer/.../matching_post_processor.py, i.e.
cells that matched neither a predicted row band nor column band being
force-snapped to the nearest one.

This is the same class of bug as #627 (which reports full mode on a
financial PDF), but two new data points:

1. It also happens in auto mode (the default triage path), not only full.
2. Switching TableFormer to FAST mode does NOT help — the corruption is
   identical, so it is not a mode-selection issue; the predicted grid itself
   misses the real cells on this document class.

Environment

• opendataloader-pdf: 2.5.7 (pip, opendataloader-pdf[hybrid])
• docling: 2.126.0, docling-core as shipped with it
• Python 3.12, Java OpenJDK 11.0.32
• Backend: opendataloader-pdf-hybrid --port 5002 --device cpu (CPU-only)
• Client: default triage (--hybrid docling-fast, hybrid_mode=auto)
• OS: Ubuntu 24.04, 22 cores / 44 threads

Reproduction

1. pip install "opendataloader-pdf[hybrid]"
2. Start backend: opendataloader-pdf-hybrid --port 5002 --device cpu
3. Parse a dense multi-column datasheet (attached: Xphor 16x200GFC PIC
   datasheet, 11 pages, 10 tables, 2-column layout with many small spec
   tables):

import opendataloader_pdf
opendataloader_pdf.convert(
    input_path=["Xphor_16x200GFC_datasheet_rev0p2.pdf"],
    output_dir="out_hybrid",
    format="markdown,json",
    hybrid="docling-fast",
)

4. Repeat with --hybrid off (pure local Java mode) into out_local and
   compare table dimensions per page.

Observed behavior — table dimensions, same document

#	page	local-only	hybrid (ACCURATE)	hybrid (FAST, patched)
1	1	17x5	17x5	17x5
2	5	10x6	9x6	9x6
3	5	17x6	16x5	16x5
4	5	4x5	3x4	3x4
5	6	6x5	6x5	6x5
6	7	7x5	6x4	8x4
7	7	13x4	12x4	12x4
8	8	1x4	10x4	10x4
9	8	11x4	3x5	3x5
10	11	3x5	— (missing)	— (missing)

• Local: 10 tables, 89 rows, 447 cells.
• Hybrid ACCURATE: 9 tables, 82 rows, 388 cells; table chore(license): [OSS Spread] Reflect legal review #45 #8 went from a 1x4
  note-row to a 10x4 block (rows pulled in from elsewhere), table [AI-Safety] Hidden Off-page AI Injection Prompts in Documents #9 lost
  8 of 11 rows.
• Hybrid FAST (after locally patching hybrid_server.py line ~494 from
  TableFormerMode.ACCURATE to TableFormerMode.FAST): 9 tables, 82 rows,
  393 cells — same corruption pattern.
• Speed: local 4.4 s vs hybrid 72–76 s for this 11-page document (CPU).

Server log evidence (ACCURATE run)

WARNING: Orphan pdf_cell 51 recovered to row=2 by nearest-row fallback (col=1, y=553.5, dist=62.4)
WARNING: Orphan pdf_cell 52 recovered to row=2 by nearest-row fallback (col=1, y=553.5, dist=62.4)
...
WARNING: Orphan pdf_cell 158 recovered to row=9 by nearest-row fallback (col=1, y=1146.3, dist=46.8)

22 such warnings in the ACCURATE run, 18 in the FAST run. The recovery path
in matching_post_processor.py exists so text is not silently dropped, but
snapping cells to the wrong row/column is exactly what produces the
collapsed/misaligned tables above — content survives, structure does not.

Root-cause observation

opendataloader_pdf/hybrid_server.py hardcodes:

"table_structure_options": TableStructureOptions(mode=TableFormerMode.ACCURATE),

with no CLI flag or environment variable to override it. On documents whose
real table grid differs from the model's predicted bands (dense spec tables,
merged cells, small fonts), every unmatched cell takes the nearest-row/col
fallback and the table is corrupted in a way that is hard to detect without
cross-checking against the rendered page.

Suggestion

1. Expose the TableFormer mode (and ideally do_table_structure) as a
   backend/client option so users can fall back per document class.
2. Consider triaging tables with strong border/ruling structure to a
   deterministic path instead of the neural grid, or at least emit a
   machine-readable warning when orphan-fallback recovery touches more than
   N cells in one table, so RAG pipelines can flag low-confidence tables.

Attachments

• Xphor_16x200GFC_datasheet_rev0p2.pdf (reproducer)
• out_local / out_hybrid JSON + Markdown outputs
• hybrid-server.log (ACCURATE and FAST runs)