Skip to content

convert.md

Latest commit

 

History

History
115 lines (88 loc) · 5.13 KB

convert.md

File metadata and controls

115 lines (88 loc) · 5.13 KB
Raw

fdoc convert

Convert documents to Markdown, XHTML, JSON, DoCO JSON-LD or plain text.

fdoc convert <FILE|DIR|->... [options]

Reads PDF, Markdown, HTML, DOCX, PPTX, XLSX, WebVTT and SubRip transcripts, and email (.eml, .msg). PDF structure is inferred from layout; the others declare theirs and carry no geometry. See Input formats.

fdoc convert report.pdf                       # Markdown to stdout
fdoc convert report.pdf -f doco -o out.jsonld
fdoc convert ./corpus/ --out-dir ./out -j 8
cat report.pdf | fdoc convert -
fdoc convert call.vtt -f doco                 # a transcript: one paragraph per turn
fdoc convert reply.eml --attachments ./att    # an email, its attachments saved

Options

Option Description
-f, --format <FMT> md (default), xhtml, json, doco, text. See Output formats
-o, --output <FILE> write to a file; single input only. Conflicts with --out-dir
--out-dir <DIR> write one output file per input into this directory
--pages <RANGES> restrict to 1-based pages: 3, 1-5, 1,4,9-12
-j, --jobs <N> parallel workers for batch (0 = one per core)
--base-iri <IRI> what every node IRI minted by -f doco starts with (<IRI>-element-3). Default: --doc-iri, else urn:fluree-doc-parse:<stem>-<first 12 hex digits of the SHA-256>. One input only
--doc-iri <IRI> stamp -f doco nodes with doc:sourceDocument, the tag a re-extraction retracts by. One input only
--source-name <NAME> what -f doco records as the input's name, doc:sourceName. Default: the input's file name; none from stdin. One input only
--attachments <DIR> save each email's attachments under DIR/<email name>/, to convert on their own. See Email
--layout-boxes <DIR> layout-detector sidecars. Env: FDOC_TITLE_BOXES
--tier-results <DIR> model-tier readings to splice. Env: FDOC_TIER_RESULTS
--structure-results <DIR> table-structure readings. Env: FDOC_STRUCTURE_RESULTS
--emit-anchors emit [[VLM:…]] tokens where escalated crops belong. Env: FDOC_VLM_ANCHORS
--escalate read escalated pages with the configured model in this run
--no-escalate never call a model, whatever the config says

The last four wire the escalation tiers; without them you get tier 1.

Escalation

With a provider configured, convert escalates the pages that ask for it and splices the readings back — the whole loop, in this command. See fdoc config for the setup and the on/off rules.

fdoc convert report.pdf -f doco          # escalates once a provider is set
fdoc convert report.pdf --no-escalate    # deterministic for this run
fdoc convert report.pdf --escalate       # force it; warns if nothing is set up

With nothing configured this never happens and no connection is opened. --tier-results takes precedence: it supplies readings you already have, so producing them again would be surprising.

--pages narrows what is read, not only what is printed, so inspecting one page of a long document costs one crop rather than all of them.

Inputs

Files, directories, or - for stdin. Stdin is read as a PDF, an email or a transcript, which are told apart by their content; the other readers identify the format by extension.

Directories are scanned for every extension convert reads, including Word, PowerPoint and Excel's macro-enabled, template and show variants (.docm, .dotx, .potx, .ppsx, .xltx, …). Multiple inputs are allowed; with more than one you need --out-dir rather than --output.

Output naming

With --out-dir, each output is the input stem plus the format's extension (.md, .xhtml, .json, .jsonld, .txt). No two inputs write one file: where inputs share a stem, the source extension joins it (report.pdf.md, report.docx.md), and where that still repeats — a/report.md and b/report.md — later ones are numbered (report.md (2).md). Names are compared without case, as a case-insensitive file system would. --attachments names each email's directory the same way.

Batch and parallelism

fdoc convert ./corpus/ --out-dir ./out -j 8
fdoc convert ./corpus/ --out-dir ./out -j 0    # one worker per core

A document that fails does not stop the batch: the rest are written and the exit code is non-zero. Use -v to see which failed and how long each took.

Page ranges

fdoc convert report.pdf --pages 1-5
fdoc convert report.pdf --pages 1,4,9-12

Ranges are 1-based — matching what a PDF viewer shows — while the page field in the output is 0-based. The whole document is still parsed and analyzed; the filter applies to the emitted elements, so cross-page structure (sections spanning a boundary) is resolved before the cut.

Exit codes

0 on success, non-zero if any input failed.

Compatibility forms

fdoc md, fdoc json and fdoc xhtml are hidden single-file equivalents of convert <file> --format <fmt>, kept because benchmark adapters shell them. Prefer convert.