Engine Room Generated Projection Drift Gate

Generated files committed beside their sources can silently go stale, so given changed paths this gate reports which declared surfaces may now be out of date and whether each one's freshness check still passes. A repository that commits generated files next to the sources they were built from carries a standing risk: a generated file can quietly fall out of step with its source, and nothing fails until a reader trusts a stale copy. This component checks for that. Given a set of changed paths, it answers which declared generated surfaces might now be out of date, and whether each surface's own freshness check still passes.

Module is source-faithful

The module is a source-faithful refactor of two internal builders that emits one result record per surface: the fingerprints taken, whether a clean record was reused or the check ran, and any drift reasons. The public module is a source-faithful refactor of two internal builders, kept together in src/microcosm_core/engine_room/generated_projection_drift_gate.py. It emits one result record per checked surface: the fingerprints it took, whether it reused a prior clean record or ran the check, and, if the surface drifted, the exact reasons why.

Owner routing is the design choice worth naming: each generated surface is a row that names its own check, so a changed path selects only the rows it could affect rather than triggering one global diff. The gate does not diff every generated file in one global sweep. Each surface is a row that names its own check, and a changed path selects only the rows it could affect.

Purpose

Why a global snapshot fails

A single global-snapshot alternative fails in two directions: it reruns everything on every edit, and it cannot say who owns a difference or how to fix it. The loose alternative is a single global snapshot: hash the whole generated tree, compare against a stored snapshot, flag any difference. That fails in two directions. It reruns everything on every edit, and it cannot say who owns a difference or how to fix it.

Splitting the tree into owners

Each surface becomes a ProjectionOwner naming its artifacts, sources, and no-write check command, and that command, not the gate, decides freshness while the gate routes and records evidence. This component splits the tree into owners instead. A ProjectionOwner names its generated artifacts, its source authorities, and a no-write check command that is treated as the freshness authority for that surface. A changed path is matched against those patterns, so a small edit selects only the owners it could plausibly touch. The check command, not this gate, decides whether a surface is fresh. The gate routes to the right owner and records the evidence.

How it works

Runtime is pipeline

The runtime is a pipeline. The runtime is a pipeline. It runs in this order for one check.

Scope selects owners by path

select_projection_owners keeps only owners whose artifact or source patterns match a changed path, returning the selected rows and a selection mode of scoped_paths, owner_filter, or all_owners. select_projection_owners takes the full owner list plus optional owner ids and changed paths. When changed paths are given, it keeps only owners whose artifact or source patterns match one of those paths, through owner_matches_path and projection_pattern_matches_path (exact, prefix, or glob match). It returns the selected rows and a selection record whose mode is scoped_paths, owner_filter, or all_owners.

Fingerprint hashes sources and artifacts

_owner_fingerprint content-addresses every source and artifact file with sha256, recording a combined source_hash, artifact_hash, path counts, and any pattern that matches no file as missing. For each selected owner, _owner_fingerprint content-addresses every source file and every artifact file with sha256. It records a combined source_hash, a combined artifact_hash, path counts, and any missing patterns. A pattern that matches no file on disk is counted as missing.

Cache reuse is deliberately strict

_source_hash_cache_hit reuses a prior clean record only when its status, source_hash, artifact_hash, and check command all match and no artifacts are missing, otherwise the check runs. _source_hash_cache_hit decides whether a prior clean record can be reused. Reuse is deliberately strict. It requires a prior record with status clean, the same source_hash, the same artifact_hash, the same check command, and zero missing artifacts. If any of those differ, the cache misses and the check runs.

Run the check or reuse cache

A cache hit returns a synthetic source_hash_cache record, while a miss runs the owner's check command and captures its return code plus hashed, length-counted stdout and stderr without storing raw text. On a cache hit, _cached_clean_result returns a synthetic record marked source_hash_cache, returncode zero. On a miss, _run_command runs the owner's check command and captures its return code plus hashed, length-counted stdout and stderr (the raw text is not stored). The command runs through _run_builtin_command first, which handles the fixture-only builtin:pass, builtin:fail, and builtin:assert-file-equals verbs before falling through to a real subprocess.

Classify drift reasons per owner

_check_owner assembles drift reasons from a failing command, a missing artifact, or an incomplete lineage record, and any reason marks the owner drift while none marks it clean. _check_owner assembles the drift reasons. A nonzero return code adds check_command_failed. A missing artifact adds artifact_missing, independent of the command result, so an absent generated file cannot be laundered by a green command. A required-but-incomplete lineage record adds fact_authority_lineage_missing_required or fact_authority_lineage_invalid. Any reason makes the owner status drift; none makes it clean.

Aggregate the per-owner results

check_projection_drift runs the per-owner check over the selected rows and rolls them into an overall status, drift_owner_count, and source_hash_cache hit and miss counts. This is the record a caller reads. The residual explanation remains available so the reader can inspect the concrete boundary without reconstructing omitted context.

The two CLI subcommands

The check subcommand runs the gate against a live tree and exits nonzero on drift, while evaluate-fixtures materialises each bundled case in a temporary tree and checks observed against expected status. The check subcommand runs the gate against a live tree and exits nonzero when any owner drifted. The evaluate-fixtures subcommand runs the bundled cases; evaluate_case writes each case's declared files into a temporary tree, builds owners from the case JSON, and records whether the observed status matched the expected one.

flowchart TD A["Changed path or owner id"] --> B["select_projection_owners"] B --> C["_owner_fingerprint: sha256 sources and artifacts"] C --> D{"_source_hash_cache_hit matches prior clean record?"} D -- "yes" --> E["reuse cached clean record"] D -- "no" --> F["_run_command runs the owner check"] E --> G["_check_owner assembles drift reasons"] F --> G G --> H{"any reason?"} H -- "yes" --> I["status drift with reasons"] H -- "no" --> J["status clean"]

Negative cases

Two positive and three drift cases

Two positive fixtures and three drift fixtures cover a planted byte, a deleted artifact, and a missing lineage record, and a run over all five returns status: pass with every case passed. The fixture set has two positive cases, clean_owner and scoped_changed_path, and three that must be caught as drift. planted_byte_detected changes one byte in a generated file so the owner's builtin:assert-file-equals check returns nonzero, giving check_command_failed. missing_artifact deletes a declared artifact so the fingerprint reports a missing path, giving artifact_missing even though the check would otherwise pass. fact_authority_lineage_required_missing declares an owner that requires a lineage record but omits its guard, derivation, and residual fields, giving fact_authority_lineage_missing_required. A run over all five cases returns status: pass, case_count: 5, passed_case_count: 5.

Prior Art Grounding

Reproducible-build and regression-test lineage

The gate follows reproducible-build and regression-test practice: declare source inputs, produce artifacts, compare content hashes, and rerun the owner check when source or artifact identity changes. The gate follows reproducible-build and regression-test practice: declare source inputs, produce generated artifacts, compare content hashes, and rerun the owner check when either source or artifact identity changes. Two anchors are relevant.

• Bazel hermeticity, for its emphasis on declared inputs, source identity, repeatable actions, and cache validity.
• pytest-regtest snapshot testing, where recorded outputs are compared against reference outputs to catch unexpected change.

Borrowed discipline, added owner routing

The borrowed part is declared-input and artifact-fingerprint discipline; the added part is routing each check through its owner rather than treating all generated files as one snapshot. The borrowed part is declared-input and artifact-fingerprint discipline. The added part is routing each check through its owner rather than treating all generated files as one snapshot.

Validation Result record Path

PYTHONPATH=src ./repo-pytest tests/test_engine_room_generated_projection_drift_gate.py -q --basetemp /tmp/microcosm-generated-projection-drift-gate
cd microcosm-substrate && PYTHONPATH=src python3 scripts/build_doctrine_projection.py --check-paper-module-corpus

The public exercise runs the same fixtures directly:

PYTHONPATH=src python3 -m microcosm_core.engine_room.generated_projection_drift_gate evaluate-fixtures --input fixtures/first_wave/engine_room_generated_projection_drift_gate/input --json

What a pass means here

A pass means the fixture behavior and the bundle-backed JSON projection are reproducible; it runs no repair commands and excludes launch. A pass means the fixture behavior and the bundle-backed JSON projection are reproducible. It does not run repair commands and it excludes launch.