Tests

Validate workflow block outputs with reusable input snapshots and assertions

What are Tests?

Tests are reusable checks for individual workflow blocks. They let you freeze a real or hand-written set of block inputs, run that block again against the current workflow draft, and verify the output with an assertion.

Use tests when you want to answer questions like:

• Does this Extract block still return the expected invoice total?
• Does this Function block still compute the validation flag correctly?
• Does this Split block still assign pages to the right subdocuments?
• Does this Classifier block still route a file to the expected category?

Tests are block-scoped, not full-workflow test suites. Running a test replays the selected block with saved inputs instead of replaying the entire workflow. This makes tests fast to iterate on while you adjust schemas, prompts, function code, categories, or split definitions.

Testable Blocks

Tests are supported for:

Block	What you usually assert
Extract	Extracted JSON fields
Function	Returned JSON fields from the function output schema
Split	Split manifest quality or produced subdocument file handles
Classifier	Produced category file handles

Other block types, such as input blocks, notes, API calls, formulas, conditionals, review gate state, and loops are not currently testable through workflow tests.

What a Test Stores

A workflow test definition stores three things:

1. The block to test - the block id and block type.
2. The block inputs - JSON and file handle inputs captured from a workflow run or entered explicitly.
3. One assertion - the expected condition for one declared output handle.

When inputs come from a previous workflow run, Retab copies the block’s handle_inputs from that run into the test. If the inputs include files, Retab materializes durable file references for the test so future runs do not depend on the original browser session.

You can also provide explicit inputs manually. JSON handles store raw JSON data, and file handles store a document reference.

Assertions

An assertion targets a declared output handle and, optionally, a path inside that handle’s JSON payload.

Common targets include:

Block	Output target examples
Extract	output-json-0.total, output-json-0.vendor.name
Function	output-json-0.is_valid, output-json-0.error_message
Split	output-json-splits or a subdocument file handle
Classifier	A category file handle

Available assertion operators depend on the target value type. The product supports basic existence checks, equality checks, string and number comparisons, object and array containment, JSON Schema validation, nested array checks, similarity thresholds, LLM-judged rubric checks, and Split IOU checks for split manifests.

For example:

• output-json-0.total is equal to 1234.56
• output-json-0.is_valid is true
• output-json-0.vendor.name contains "Acme"
• output-json-splits has Split IOU greater than or equal to 0.95
• output-json-0.summary matches an LLM rubric

Running Tests

You can run one test from its detail page, run tests for a block, or run all tests in a workflow from the Tests page.

When a test runs, Retab:

1. Loads the current workflow draft and the current block configuration.
2. Rebuilds the saved handle inputs into normal workflow runtime inputs.
3. Executes only the selected block.
4. Collects the raw output, declared handle outputs, routing decision, warnings, and duration.
5. Resolves the assertion target from the block outputs.
6. Evaluates the assertion and records a run result.

Test runs are asynchronous. The dashboard streams live progress and stores a parent run plus one result per saved test. API clients inspect the parent run by run_id, then read results under /v1/workflows/tests/results?run_id={run_id}.

Results

A parent test run uses the shared run lifecycle:

Status	Meaning
pending	The run has been scheduled but has not started yet.
running	One or more test results are still executing.
completed	The run finished; inspect result verdicts for pass/fail detail.
error	The parent run failed before all results could be recorded.
cancelled	The run was cancelled.

Each child result has its own verdict:

Status	Meaning
passed	The block ran successfully and the assertion passed.
failed	The block ran successfully, but the assertion did not pass.
blocked	The assertion could not be evaluated against the selected target.
error	The block replay failed or another execution error occurred.

Each result includes the saved inputs used for that execution, the block output, handle outputs, assertion result, warnings, duration, and execution fingerprints. Retab also tracks the latest passing and failing summaries on the test definition.

If a block replay fails before an assertion can run, the assertion is marked as blocked and the run is recorded as an error.

Staleness and Schema Drift

Tests are tied to the block inputs and output schema that existed when the test was created or last updated. If the workflow draft changes, Retab compares the saved fingerprints and schema dependencies against the current block.

In the dashboard, the staleness badge tells you whether a test is:

Badge	Meaning
not run	The test has not produced a run record yet.
up to date	The latest run still matches the current block schema.
stale	The block schema or relevant assertion dependency changed. Run the test again or update the assertion.
unknown	Retab could not determine staleness for this test.

Staleness does not automatically mean the workflow is broken. It means the saved test definition should be rechecked against the current draft before you rely on its latest result.

Recommended Workflow

1. Run the workflow on a representative document.
2. Open the Tests page and create a workflow test from that completed run.
3. Pick the block output field or handle you want to protect.
4. Define one assertion for the expected behavior.
5. Run the test after changing schemas, prompts, code, categories, or split definitions.
6. Use stale tests as a review queue before publishing workflow changes.

Tests work best when each test protects one behavior. Prefer multiple small tests over one broad assertion so failures point directly to the changed output.