Incremental document to Markdown sync with local OCR, no AVX2 required
3.9K
Keeps a folder of documents mirrored as Markdown, so that LLM agents and RAG tooling can read them. Scanned documents get a text layer through local OCR first. Runs incrementally, cleans up after deleted documents and never overwrites a Markdown file you edited by hand. See the supported formats.
No machine learning runtime inside — no numpy, no onnxruntime, no torch —
so it runs on x86-64 CPUs without AVX2, where markitdown, docling and most
of the AI-oriented converters abort with Illegal instruction (SIGILL). It
also keeps the image small, the output deterministic, and the documents on your
machine.
documents/ documents/_md/
├── 2026/ ├── 2026/
│ ├── invoice-4417.pdf ──► │ ├── invoice-4417.pdf.md
│ ├── scan-receipt.pdf ──► │ ├── scan-receipt.pdf.md (via _ocr/)
│ └── assets.xlsx ──► │ └── assets.xlsx.md
convsync | entry point: sync, ocr, convert |
| ocrmypdf + Tesseract | text layer for scanned PDFs, output as PDF/A-2b |
pdftotext -layout (poppler) | PDF text extraction, keeping labels and values on the same line |
| pikepdf | PDF comments (annotations), which pdftotext does not read |
| openpyxl | XLSX sheets as Markdown tables |
| xlrd | XLS sheets as Markdown tables |
| python-docx | DOCX documents as Markdown |
| odfdo | ODT documents and ODS sheets as Markdown |
| Ghostscript, unpaper | ocrmypdf runtime dependencies |
Tesseract selects its SIMD code path at runtime and falls back to SSE on CPUs without AVX2. Every CI build fails if numpy, onnxruntime, torch or magika end up installed.
| Extension | Conversion |
|---|---|
.pdf | text layer via pdftotext -layout, plus comments; image-only scans are OCRed first |
.xlsx, .xls, .ods | one Markdown table per sheet, with the cached values |
.docx | body text with headings, lists and tables |
.odt | headings, lists, tables and emphasis |
Every other file is ignored. Plain text formats (CSV, TXT, JSON, HTML) need no conversion: agents read them as they are. More formats are tracked in #5.
.doc filesWord 97–2003 .doc files are a binary format (OLE2), not a renamed .docx:
renaming them only makes the conversion fail. Convert them to .docx first,
for example with LibreOffice in a throwaway container, run from the folder
that contains documents/:
docker run --rm -v "$PWD/documents:/w" -w /w debian:bookworm-slim sh -c '
apt-get update -qq &&
apt-get install -y -qq --no-install-recommends libreoffice-writer-nogui >/dev/null &&
owner="$(stat -c %u:%g .)" &&
find . \( -path ./_md -o -path ./_ocr \) -prune -o \
-type f -iname "*.doc" ! -name "~\$*" -print |
while IFS= read -r f; do
HOME=/tmp soffice --headless --convert-to docx --outdir "$(dirname "$f")" "$f" &&
chown "$owner" "${f%.*}.docx"
done'
Each .docx lands next to its original, overwriting any .docx of the same
name; the .doc files stay and are ignored. Some .doc files exported by
older software are actually RTF or HTML: file name.doc tells which.
| Tag | Content |
|---|---|
latest | last build of the main branch |
main | the same, to refer to the branch explicitly |
1, 1.2, 1.2.3 | semantic releases |
Platforms: linux/amd64, linux/arm64.
docker run --rm --network none \
-v "$PWD/documents:/data" \
frugan/convsync:1
The Markdown lands in documents/_md/, the OCR copies of scanned PDFs in
documents/_ocr/. Run it again whenever documents change: only what is new or
modified is processed.
--network none is not required, but nothing in the pipeline needs the
network: with it, the documents provably stay on the machine.
The container runs as root by default. With a Docker daemon configured for
userns-remap, container uid 0 maps to your host user and the generated files
are yours. Without it, pass your ids:
docker run --rm --network none --user "$(id -u):$(id -g)" \
-v "$PWD/documents:/data" frugan/convsync:1
The typical setup: the documents live in a gitignored folder of the project, and the agents get only the Markdown.
services:
convsync:
image: frugan/convsync:1
profiles: ["tools"] # never started by `docker compose up`
network_mode: none
volumes:
- ./agents/data:/data
docker compose run --rm convsync
Mount only _md/ into the containers the agents run in, read-only
(./agents/data/_md, with :ro): the originals and the OCR copies in _ocr/
then stay out of their reach. An agent running directly on the host can read
the whole folder regardless.
convsync [sync] [--force] # OCR the scanned PDFs, then convert everything
convsync ocr [--force] # only add the text layer to scanned PDFs
convsync convert [--force] # only convert to Markdown
convsync bash # anything else is executed as is
--force rebuilds everything instead of what changed. With sync it applies
to both steps, which means re-OCRing every scan: use convert --force when only
the Markdown has to be regenerated.
Every run ends with a summary, and the exit code is non-zero if any document failed:
ocr: 1 | skipped: 12 | native: 30 | removed: 0 | failed: 0
converted: 2 | skipped: 41 | removed: 1 | conflicts: 0 | failed: 0
OCR only where it is needed. A PDF that yields fewer than 100 characters of
text is treated as an image-only scan and goes through ocrmypdf into
_ocr/<path>.ocr.pdf. So is a PDF whose every page is a full-page image, such
as a filled-in form flattened back to PDF, whose text layer holds only the
field values. Born digital PDFs are left alone: rasterizing them would only
degrade the text. Originals are never modified.
Mirrored output. _md/ reproduces the source tree, keeping the original
extension in the name, so report.pdf and report.xlsx in the same folder do
not collide.
Incremental. A document is converted again only when it (or its OCR copy) is newer than its Markdown.
Orphans. When a document is deleted, its Markdown and its OCR copy go with it, and so do the folders left empty.
Hand edits are safe. Each generated file starts with a provenance stamp carrying the SHA-256 of its body:
<!-- generated from 2026/invoice-4417.pdf sha256:3f1a… -->
If the body no longer matches the stamp, someone edited it: the rebuild is
written next to it as <name>.md.new and reported as a CONFLICT, instead of
overwriting the edit. Merge by hand and delete the .new file; it also goes
away on its own as soon as the edit and the converter agree again.
Output shape. PDF text is wrapped in a fenced code block, because the
alignment produced by pdftotext -layout is meaningful on forms, invoices and
tax documents, and Markdown would collapse it. Comments (sticky notes,
highlights and other annotations with text) follow in a ## Comments section,
each with its page, the text it refers to, its author and whether it is a
reply. XLSX, XLS and ODS sheets become
one table per sheet, with the values cached by Excel or LibreOffice rather than
the formulas.
DOCX and ODT documents become plain Markdown: headings, lists and tables.
| Variable | Default | |
|---|---|---|
CONVSYNC_ROOT | /data | tree to process |
OCR_LANGS | ita+eng | Tesseract languages, joined with + |
Language packs are chosen at build time: the published image carries English and Italian. For other languages, build your own:
docker build --build-arg TESSERACT_LANGS="deu fra" -t convsync:custom .
docker run --rm -e OCR_LANGS=deu+fra+eng -v "$PWD/documents:/data" convsync:custom
Most converters are libraries that turn one file into Markdown, often with far better layout reconstruction than this image. What they do not do is keep a folder in sync, and the AI-based ones carry a runtime that rules out older CPUs. The table reflects the state of each project at the time of writing: check upstream before relying on it.
| Tool | OCR for scans | ML runtime | CPU without AVX2 | Folder sync | Formats |
|---|---|---|---|---|---|
| ConvSync | yes, Tesseract, local | none | yes | yes: incremental, orphans, hand-edit guard | PDF, XLSX, XLS, ODS, DOCX, ODT |
| MarkItDown | via plugins or LLM | onnxruntime (magika, core dependency) | no, SIGILL | no | many |
| Docling | yes | torch / onnxruntime | not guaranteed | no | many |
| Marker | yes | torch | not guaranteed | no | many |
| MinerU | yes | torch, VLM models | not guaranteed | no | many |
| PyMuPDF4LLM | yes, Tesseract | onnxruntime + numpy (PyMuPDF Layout, installed by default) | not guaranteed | no | PDF and MuPDF formats |
| Anydoc | hosted only (Firecrawl Parse) | none | likely, untested | no | Office, ODF, RTF, EPUB, CSV, PDF |
| Pandoc | no | none | yes | no | many, but no PDF or XLSX input |
When to pick something else:
pdftotext.The build: section of docker-compose.yml is active. image is both the
pull reference and the build tag, so set it in both commands: building with
IMAGE unset overwrites frugan/convsync:latest in the local cache.
cp .env.dist .env
IMAGE=convsync:local docker compose build
IMAGE=convsync:local docker compose run --rm convsync
The documents go in ./data, which is gitignored. The end-to-end test needs
no documents of yours:
docker run --rm --network none --entrypoint /opt/convsync/tests/smoke.sh convsync:local
List Number style.Pictures/...), which do not resolve; in a DOCX they are dropped./Rotate get wrong
matches. A scanned PDF with comments but no OCR copy yet no longer reports
EMPTY.For your contributions please use:
See CONTRIBUTING for detailed guidelines.
Copyright (c) 2026 Frugan. MIT, see the LICENSE file.
The bundled tools keep their own licenses: notably ocrmypdf (MPL-2.0), Tesseract (Apache-2.0), Ghostscript (AGPL-3.0) and poppler (GPL).
Content type
Image
Digest
sha256:4210b0f04…
Size
213.3 MB
Last updated
4 days ago
docker pull frugan/convsync