Schemas & extraction
How field definitions are authored, typed, versioned, and bound to document classes — and how their rules shape what Extract returns.
A schema is the field-level contract for a document class: the named values IDPForge extracts, the type of each, and the rules that decide what passes. A taxonomy's leaf classes bind schemas; the Extract stage reads a document against the schema its class resolved to.
Field types and the type system
A schema is an ordered set of fields. Each field has a type that determines which extraction sub-stage reads it and how its value is validated.
| Type | Holds |
|---|---|
text | A short single-line value. |
long text | A multi-line or paragraph value. |
number | A numeric value without a currency. |
currency | A monetary amount, with its currency and decimals. |
date | A calendar date, normalized to a formatting instruction. |
boolean | A true/false value. |
enum | One value from a fixed set the field defines. |
checkbox | A tick, cross, or filled-mark selection. |
radio | One selection from a mutually exclusive group. |
table | Repeating rows, extracted column by column. |
group | A nested set of related fields. |
Beyond its type, a field carries a description, an extraction instruction (where and how to read the value), a formatting instruction (how to shape it), a required flag, a confidence threshold, and optional validation rules. A schema context prompt sits above the fields and applies to every one of them — the document type, language, currency, and tie-breaking guidance the model reads before any single field.
The field inspector exposes the whole per-field contract in one place: type, group, the two instructions, the required flag, and the confidence threshold. The context prompt above the list is the guidance every field inherits.
Tables and repeating groups
Two types hold structure rather than a single value. A table field extracts repeating rows — the line items on an invoice — column by column, so each row is a record rather than a run of loose values. A group field nests related fields that belong together, such as the parts of an address, without repetition. Both keep structured content structured instead of flattening it into one long field list.
Field-level confidence and what drives it
Every extracted value carries a confidence score. Each field defines a confidence threshold — 80% by default — and a value that scores below its threshold is flagged for review. The threshold is advisory: it routes a weak extraction to a reviewer, it does not fail the job. Confidence reflects how cleanly the value could be read — legibility of the source, how directly the value maps to the field, and how much ambiguity the model had to resolve.
Versioning a schema without breaking live pipelines
A schema exists as a draft and as a series of published versions. A draft is editable and is not usable by a pipeline. Publishing snapshots the current fields into an immutable, numbered version carrying a required release note; the first publish mints v1.
A pipeline binds a specific published version. Editing a published schema opens a new working draft and leaves the published version untouched, so live jobs keep running the version they are pinned to until the next version is published and rebound. This is the mechanism that lets a schema evolve without disturbing work in flight.
Inferring a schema from sample documents
Given a representative sample, IDPForge proposes a schema: it reads the document layout, detects fields and tables, and offers a field — with an inferred type and a confidence — for every value it can extract.
Inference is a proposal, not a commitment. Each proposed field shows its confidence and type against the sample; the author keeps what belongs in the contract and drops the rest, in bulk by confidence or one at a time. The result is a draft like any other, versioned from v1 on publish.