Notes  /  byte-identical-logs-are-a-test-tool2026-08-03

Byte-identical logs are a test tool

When the same scenario produces the same CSV down to the byte, diff becomes a regression harness — no framework required.

NOKTRA SIL Runtime makes a promise that sounds modest and is not: run the same scenario twice, and the two log files are identical. Not similar, not equal-within-tolerance — byte for byte. LF endings on every platform, UTF-8 without BOM, invariant number formatting, and no wall-clock timestamp written into the file.

The last item is the tell. A timestamp inside a log is harmless until the day you want to compare two logs, at which point every line differs and the comparison needs a parser, a tolerance, and a meeting. Leave the timestamp out and the comparison collapses into the oldest tool on the machine: diff. Empty output, or a finding.

This turns reproducibility from a philosophical stance into a daily instrument. Refactored the integrator? Run the golden scenario, diff against the stored log. Changed a compiler flag? Diff. Upgraded the framework? Diff. Anyone can run this check, it takes seconds, and its verdict cannot be argued with — which is the property that matters in review.

Getting there costs real design decisions, and they are worth naming. Simulation time is always step-index times dt, never accumulated, so 10,000 cycles at 1 kHz land on exactly 10.0 seconds with no float drift. Wall-clock pacing decides when a cycle runs, never what it computes, so a paced run for watching and a fast run for CI produce the same numbers. Display buffers live outside the deterministic path, so dropping a UI sample can never change a result. Even the compiled C models are built with FP contraction off, because a fused multiply-add is more accurate and differently wrong.

There is a general lesson here that extends past simulation. Any tool that writes results — reports, exports, generated code — quietly chooses whether its output can be diffed. Timestamps, map iteration order, locale formatting, float printing: each one is a small decision that either preserves diffability or spends it. Most software spends it without noticing.

The offline angle closes the loop. On an air-gapped bench there is no dashboard service to compare runs for you. There is the file, and there is diff. A tool designed for that room had better make those two things sufficient.