Financial documents to source-linked training data

Financial data preparation starts before the first question is generated. A table cell can be perfectly extracted and still be interpreted incorrectly because its scale, period or footnote was lost. “193.5” in an index table is not the same fact as “193.5 million dollars” in a revenue table.

The data pipeline therefore preserves the document’s structure while producing a smaller, typed representation for training and evaluation. A useful training row can always be traced back to the source, the extraction version and the transformation that created it.

Preserve meaning during extraction

The first artifact is a document envelope: source identifier, acquisition time, content hash, licence or access classification, publication date and parser version. The next artifact is a collection of passages and tables with page-level location information. Only then do we derive financial facts.

A normalized fact combines the value with its context. The following record is an authored example of the schema, not a claim about a public company.

{
  "fact_id": "example-revenue-fy25",
  "document_id": "example-report-2025",
  "source_family": "example-issuer-annual-report",
  "page": 42,
  "table_id": "income-statement",
  "row_label": "Revenue",
  "column_label": "FY2025",
  "raw_text": "1,240",
  "value": "1240000000",
  "currency": "INR",
  "display_scale": "millions",
  "period_type": "financial_year",
  "evidence_id": "example-report-2025:p42:t1:r3:c2"
}

Keep raw_text as well as the normalized value. It lets a reviewer distinguish a parser error from a scaling error. Parentheses, negative signs, dash placeholders and “not meaningful” are separate cases. A dash should not become zero unless the document’s convention explicitly defines it that way.

Tables need their row and column headers carried into any generated passage. A text chunk containing only “1,240 1,105 980” is nearly useless. A chunk containing the metric, periods and unit is suitable for retrieval and for a supervised example. Retain nearby footnotes where they qualify comparability or restatement.

Build a source registry before mixing datasets

The public reference kit pins source revisions and stores download hashes. The following counts were read from the pinned FinQA JSON files and the referenced dataset cards. They describe public resources, not the size of the private training corpus.

FinQA’s repository carries an MIT licence; the kit includes its licence alongside the selected annotations. FinanceBench’s dataset card specifies CC BY-NC 4.0, so it is not included as a default commercial training source. IndiaFinBench lists CC BY 4.0. A source registry records these distinctions before a pipeline decides what to download or train on.

Dataset purpose matters as much as the file’s split name. A public evaluation set uploaded under a split called train does not automatically become suitable fine-tuning material. Keep the intended benchmark role in your own manifest.

Eight real records, with the numerical target made visible

These are selected arithmetic annotations from FinQA’s development set at revision 0f16e2867befa6840783e58be38c9efb9229d742. The full records remain in the source repository. The table exposes the record identifier, arithmetic operation and the difference between an execution target and a displayed answer.

The percentage examples demonstrate why a scorer needs a unit contract. An execution target of 0.935 and a display answer of 93.5% can describe the same ratio. Conversely, a program that already multiplies by 100 returns percentage points directly. Multiplying every result by 100 produces wrong answers.

The eight records are a small, selected arithmetic teaching set. They establish that the reference interpreter executes these operations; they do not establish a model’s financial reasoning score. The FinQA paper explains the broader task of reasoning over financial text and tables.

Make acquisition reproducible

The download command in the kit obtains one explicitly selected split, verifies its SHA-256 and writes it to a destination you provide. It does not silently fetch evaluation material when training starts.

python fetch_finqa.py --split dev --output /tmp/finqa-dev.json

The essential verification step is short:

import hashlib

def verify_bytes(payload, expected_sha256):
    actual = hashlib.sha256(payload).hexdigest()
    if actual != expected_sha256:
        raise ValueError("dataset digest does not match the manifest")
    return payload

Record the source revision independently of the file hash. A revision explains where the artifact came from; the hash identifies the exact bytes consumed. An evaluation report should carry both.

The registry also records the transformation code version. Two teams can download identical JSON and still evaluate different tasks if one includes gold evidence in the prompt and the other requires retrieval over the whole report.

Split by evidence family, then generate variants

Randomly splitting final QA rows is a poor fit when several rows share a source passage. An English question, its Hindi translation and a shortened paraphrase can otherwise appear in three different partitions. The split looks large while the underlying evidence is repeated.

Assign a source_family before generating questions. Group document versions, derived passages, translations and near-duplicate evidence. Then assign the group deterministically.

import hashlib

def family_split(source_family):
    digest = hashlib.sha256(source_family.encode()).hexdigest()
    bucket = int(digest[:8], 16) % 100
    if bucket < 80:
        return "train"
    if bucket < 90:
        return "validation"
    return "test"

This function is a stable assignment mechanism, not a duplicate detector. The grouping step must identify related sources first. With a small number of families, inspect the resulting task and language balance rather than expecting an exact 80/10/10 row ratio.

Keep a temporal holdout when the deployment must handle newly published material. A company-year split asks a different question from a company-level split: the former tests new periods with potentially familiar issuers; the latter tests unseen issuers. Name the protocol in the report.

Keep gold evidence out of candidate inputs

A benchmark record often contains the answer, an execution program, selected supporting sentences and other annotation fields. Those are useful to the evaluator. They are not all legitimate input to the candidate.

For a full-context FinQA task, build the candidate request from the question, table, pre-table text and post-table text. Keep the answer and gold program in a separate object. If the experiment supplies a gold evidence excerpt, call it a supplied-evidence experiment. Do not compare that result directly with a system that had to retrieve the evidence itself.

The same principle applies to a private snapshot. The candidate receives the authorised facts it needs, not a hidden expected answer placed in a metadata field. In Part 11, the runner reads a candidate-only file while the scorer reads the gold contracts.

The finished data product is more than JSONL. It contains the rows, a source registry, a split manifest, transformation identifiers, acceptance rules and a small set of human-readable examples. That combination makes training repeatable and makes a bad answer traceable to the layer that produced it.