Skip to main content

Run results

Once a run is terminal its page becomes a record: what the tests said, what the run produced, and how it differs from the last good one.

Test results are parsed, not reported​

A finished run showing a stage timeline, a test summary reading 5 passed 1 failed, an expanded failing suite with its assertion message, and an artifacts table.A finished run showing a stage timeline, a test summary reading 5 passed 1 failed, an expanded failing suite with its assertion message, and an artifacts table.
Tests and artifacts on a finished run. Both sections disappear entirely when there is nothing to show.

There is no test API. The server looks through the run's collected artifacts for JUnit XML and parses what it finds, every time you open the page. A file qualifies when all three are true:

  • its name ends .xml
  • it is 8 MiB or smaller
  • its contents contain a <testsuite or <testcase element

Anything else is skipped in silence — no warning, no partial result. A report that doesn't appear is nearly always one that missed one of those three, and an oversized report is the usual culprit: it doesn't fail, it simply isn't there.

Each suite is attributed to the stage whose artifact it came from, which is why a monorepo that writes one report per package shows them separately.

To get a report here, declare it as an artifact of the stage that writes it. That's the authoring side, on Artifacts, cache and stash.

Artifacts​

Artifacts are declared by the pipeline, never discovered. A stage's outputs are gathered after it finishes and stored with their size and a SHA-256 checksum, so what you download is verifiably what the run produced.

Three properties matter in practice:

  • Collection never fails a run. If it can't be gathered, the run's own result stands.
  • It is skipped entirely when the workspace is over its artifact quota, and existing artifacts are kept. A green run with nothing attached is the symptom; see Workspaces.
  • A run collects at most 2000 files, and directories are flattened into individual entries.

You can download one artifact or the whole set as a single archive. A run with no artifacts has no archive to give you, and says so rather than handing back an empty one.

Local runs don't collect anything

Collection is a server-side step. flow run on your laptop executes the same stages and writes the same files, but nothing gathers them — there's no run to attach them to.

Comparing against a baseline​

The run comparison screen: two runs side by side with their status, duration and per-stage differences.The run comparison screen: two runs side by side with their status, duration and per-stage differences.
Comparing a run against its baseline — the most recent earlier success of the same project.

Compare answers "what changed since this last worked". By default the baseline is the most recent earlier successful run of the same project; you can also name a specific run.

A project whose first run just failed has no baseline, and the page says so. That is a normal outcome, not an error.

The comparison is stage by stage, matched by identity rather than position, so a stage added or removed since the baseline is called out as such rather than shifting everything below it. For duration it reports a stage as slower or faster only when the change clears both 100 ms and 10% — small enough runs jitter more than that, and a comparison that flagged every stage every time would be worthless.

How long results last​

Runs and their artifacts are pruned by retention, which is off by default and configured per instance, with a per-workspace override. Once a run is pruned its logs, artifacts and test results go with it. See Operations.