Pipeline anatomy
A pipeline is an ordered set of stages. Each stage owns exactly one question, and configuration lives on the stage that owns it — never in two places.
The stages
A pipeline runs a document through an ordered set of stages. Each answers one question and owns its own configuration.
The pipeline builder. The left rail is the stage order for this pipeline; the right panel configures whichever stage is selected. A stage that is bypassed is skipped for every document.
The Parse, Split, Classify, and Extract stages each carry their own sub-stages. The pages below go one level deeper.
Source
Source answers where documents come from and nothing else. A pipeline can have several sources at once; results are unioned. Each source carries an editable source tag that is stamped onto every job it produces, so you can filter by origin in Jobs and Insights later.
- API — direct
POST /v1/jobssubmissions. - Email — a generated inbox address, optionally with attachment-only rules.
- Storage — S3, Azure Blob, GCS or SFTP, polled or event-driven.
- Manual upload — drag-and-drop from inside the app.
A source binds a connector instance by id. The same connector can serve many pipelines, and rotating its credentials updates every binding at once.
Initialization
Initialization owns what we accept and what we do to it before anything is decided: allowed file types, maximum size and page count, the on-invalid behaviour, deskew and OCR normalization. Antivirus scanning is always on and cannot be disabled.
Parse
Parse reads the document's structure before anything is split or extracted: layout detection, language detection, and OCR, plus the output format every later stage works from. See The Parse phase.
Split / Classify
One uploaded file is often many documents. Split cuts a package into document boundaries; Classify assigns each resulting document a class from your taxonomy. Both sub-stages can run independently — split-only, classify-only, or both.
Classification drives routing: each class maps to a schema, and a document that matches no class falls through to generic extraction.
Extract
Extract reads the fields defined by the schema bound to the document's class, returning a value and a confidence per field. With no schema bound it returns free-form JSON. Extract is the stage that produces the confidence signal the review loop keys off.
Post processing
Validation rules run after extraction: format checks, arithmetic checks (line items sum to total), and lookups against your master data. Auto-fill can populate a field from a lookup rather than from the page. A failed rule raises a validation_failed exception.
Destination
Destinations receive the finished record — object storage, a folder, a webhook, or a system of record. A pipeline can deliver to several destinations. Output format is set per destination, not globally.
Versions
Pipelines are versioned. Editing a published pipeline creates draft edits against the current version; publishing mints the next version and leaves in-flight jobs on the version they started with. Every job records the exact version it ran, which is what makes a result reproducible six months later.