Skip to content
Paperless StarfruitPaperless Starfruit

How processing works

Every tagged document follows the same path.

  1. Picked up. The poller lists documents carrying the psf-process tag and queues a job for each, with at most one active job per document, so a slow run never piles up duplicates. A document already waiting in review, or one that has terminally failed, is skipped.

  2. Extraction page gate (outer). If the document has more pages than your extraction cap, it gets neither OCR nor extraction: paperless keeps whatever it already recognised, the trigger tag is dropped, and the run is recorded as skipped.

  3. OCR (inner gate). If OCR is on, a model is selected, and the document is under the OCR cap, the original is re-read by Mistral OCR (a dedicated page-billed endpoint) or a vision LLM, and the text is written back to paperless. Otherwise paperless’s existing text is reused, for free.

  4. Skip-if-identical check. A content_hash is computed from the extraction-input text plus a fingerprint of the active config (the OCR and language provider/model, whether extraction is enabled, and how much of the original the extraction model gets to see). If a previously completed job for this document has the same hash, the result would be byte-for-byte identical, so steps 5–7 are skipped, the tag is dropped, and the run finishes having spent nothing on extraction.

  5. Extract. One language-model call returns { title, tags, correspondent, date }, validated against a strict schema. Invalid output triggers a single repair retry before it’s trusted. The call carries more than text: when the language model is vision-capable, the original document is attached as visual evidence: letterheads, stamps and signatures often carry the correspondent and date the text layer garbles. Files over the OCR page cap attach only their first and last pages (the full text still covers the middle); models that can’t take files fall back to text-only. With extraction turned off (OCR-only mode) the run completes here instead: a single PATCH writes the OCR text back and drops the trigger tag, metadata is left untouched, and the run is recorded as ocr-only.

  6. Reconcile. Suggested tag and correspondent names are mapped to paperless IDs. New ones are created only when you’ve allowed it (and never for a blacklisted correspondent).

  7. Apply or queue. With auto-apply on, a single PATCH writes the metadata and removes the trigger tag. With it off, the suggestion lands in the review queue and the tag stays until you decide. Either way the run is recorded in the audit log.

Reprocessing

Section titled “Reprocessing”

Re-tagging a document is the explicit “process me again” signal.

  • If nothing relevant changed (same recognised text, same models), the skip-if-identical check (step 4) means a re-run costs no extraction call.
  • Change the OCR or language model, or re-OCR with a different provider, and the fingerprint (and usually the text) differ, so it genuinely reprocesses.

When a job fails

Section titled “When a job fails”

Errors use a plain retry counter, with no exponential backoff. A job is retried until it hits max attempts (default 3), then marked failed and surfaced in the dashboard. A permanently-failed document is not re-polled (so it can’t loop forever); use the Retry action on the dashboard to clear the failed rows and enqueue a fresh run.

A transient “paperless hasn’t OCR’d this yet” situation is treated as a deferral, not a failure: the job is dropped and simply re-created on the next poll, without consuming an attempt.

Worker model

Section titled “Worker model”

One worker, one job at a time, one SQLite file. At single-user scale (~50 docs/week) throughput is a non-issue, so concurrency is fixed at 1 by design.