Engine Room Public Projection Leak Gate

Before rendered tree

Before a rendered tree reaches a public reader, one narrow question stands: does it carry account secret shapes, a non-public paths, a transport symbol, or an escaping symlink that should not leave the workspace? Before a rendered tree of files is exposed to a public reader, someone has to answer one narrow question: does this tree contain anything that should not leave the workspace? account secret-shaped strings, a private home path, a browser transport symbol, or a symlink pointing outside the tree are all ways for private material to ride along with an otherwise public projection.

Component answers question

This component answers that question over a directory of rendered files with a green or red verdict: a data-loss-prevention gate for one tree, not a general scanner. This component answers that one question over a directory of rendered files and returns a green or red verdict. It is a data-loss-prevention gate for one tree, not a general scanner.

Design point is

The design point is what it does with a find: each match is recorded by category, path, line, and a SHA-256 hash while the matched value is dropped, so the report never becomes a second copy of the leak. The design point is what it does with what it finds. A secret scanner that prints the secret into its own report has made a second copy of the leak. This gate never does that. Every match is recorded by category, path, line number, and a SHA-256 hash of the matched text, and the matched value itself is dropped. A reviewer can confirm that a leak was found and where, without the result record becoming the thing that leaks.

Purpose

The quiet co-travel failure mode

The failure a loose gate allows is quiet co-travel: a rendered file carries a key or non-public paths nobody noticed, and a naive report copies that value into a second guarded artifact. The failure mode a loose gate allows is quiet co-travel. A publish step renders a directory, a human skims it, and one file carries a key or a non-public paths that nobody noticed. The problem is not that leaks are undetectable. The problem is that ad hoc review misses them and that a naive report copies the sensitive value out of the file and into a second artifact that then also has to be guarded.

Fixing the position of the check

This gate reads rendered files and path names against fixed regular expressions, treats an escaping symlink as a hard blocker, folds in an optional gitleaks run, and emits a hash-only publishable record. This gate fixes the position of that check. It reads the rendered files and their path names against a fixed set of regular expressions, treats a symlink that escapes the root as a hard blocker, and folds an optional gitleaks run into the same result. The verdict is deterministic for the same input, the counts are honest, and the result record is safe to publish because it holds hashes and categories, not the matched text.

How it works

scan_projection resolves and walks the root

scan_projection resolves the supplied root, raises ValueError if it is missing or not a directory, normalises policy-exception paths, and walks the tree in sorted path order. The entry point is scan_projection(root, ...). It resolves the supplied root and raises ValueError if the root is missing or is not a directory. It then normalises the caller's policy-exception paths against DEFAULT_POLICY_EXCEPTION_PATHS through _normalise_policy_paths, and walks the tree in sorted path order.

Three per-entry checks in order

Each entry passes three checks in order: skip-and-count for named directories and .pyc files, record a symlink whose target escapes the root, and scan every other path two ways. For each entry the walk applies three checks in order. Directories named in SCAN_SKIP_DIR_NAMES (node_modules, .git, .vite, dist, __pycache__, .pytest_cache) and files with a .pyc suffix are skipped and counted. A symlink whose resolved target does not sit under the root is recorded as a symlink escape, keeping only the relative path and a hash of the escaped target. Every other path is scanned two ways.

Path and content pattern scans

_scan_path runs the relative path against PATH_PATTERNS and _scan_file runs UTF-8 contents against CONTENT_PATTERNS for account secrets, non-public paths, and host-bound transport, skipping any file it cannot read as text. _scan_path runs the relative path string against PATH_PATTERNS: private history and voice file shapes, a private notes tree shape, and browser transport file names. _scan_file reads each readable file as UTF-8 and runs its contents against CONTENT_PATTERNS: private key blocks and vendor key shapes under account secrets (for example openai_key_shape, github_token_shape, aws_access_key_shape), private home and profile paths under private_path, and browser transport symbols and debug ports under host_bound_transport. A file that cannot be read as text is skipped, not failed.

_hit stores a hash, never the value

Every match becomes a _hit record carrying category, pattern name, path, optional line, source kind, policy-exception flag, and match_sha256, and this is the single place the leak-safety property is enforced. Every match becomes a record through _hit. That record carries the category, the pattern name, the relative path, an optional line number, the source kind (content or path), whether the path is a policy exception, and match_sha256. The matched value is never stored. This is the single place the leak-safety property is enforced.

Splitting hits and computing the verdict

scan_projection retains policy-exception hits as non-blocking evidence, treats every other hit as blocking, folds in an optional redacted gitleaks run, and sets the verdict red on any blocker. scan_projection then splits the hits. Any hit whose path is in the policy exception set is retained as evidence but does not block. Every other hit is a blocking hit. If the caller asked for it, run_gitleaks runs the external scanner over the root with --redact and records only its status, finding count, return code, and hashed stdout and stderr. _overall_status computes the verdict: red if there is any blocking hit or symlink escape, or if the gitleaks status is red, error, or unavailable_fail_closed; otherwise green. The result sets public_release_allowed_by_scan to true only when the status is green.

Fixture-driven evaluation

evaluate_case materialises one JSON case, runs scan_projection, and checks the observed status, while evaluate_fixture_dir passes only when every case meets its expectation. Two functions drive the check from fixtures. evaluate_case materialises one JSON case into a temporary root, runs scan_projection, and compares the observed status to the case's expected status. evaluate_fixture_dir runs every case in a directory and returns status: pass only when all cases meet their expectation. main exposes both a scan and an evaluate-fixtures subcommand.

flowchart TD Root["Rendered root walk in sorted order"] Root --> Skip{"Skip dir or .pyc?"} Skip -- "yes" --> Count["Counted, not scanned"] Skip -- "no" --> Link{"Symlink escapes root?"} Link -- "yes" --> Escape["Symlink escape target hash only"] Link -- "no" --> Scan["_scan_path and _scan_file _hit stores hash only"] Scan --> Policy{"Path in policy exception set?"} Policy -- "yes" --> Kept["Retained, non-blocking"] Policy -- "no" --> Block["Blocking hit"] Escape --> Verdict{"Blocking hit, escape, or gitleaks red?"} Block --> Verdict Verdict -- "yes" --> Red["red verdict"] Verdict -- "no" --> Green["green verdict"]

Negative cases

Two green cases, three red

Five cases split two green and three red, and policy_exception_hash_only plants a boundary-document example inside a policy-exception path and still expects green with the hit retained but non-blocking. The fixture set has five cases: two that must stay green and three that must go red. clean_projection and policy_exception_hash_only are the positive cases; the second plants a boundary-document example inside a policy-exception path and expects a green verdict with the hit retained but non-blocking.

Three blockers, each hash-only

The three negative cases force a non-public paths, a vendor key shape, and a path-name pattern block, and the tests assert none of those values survive into the emitted record. The three negative cases each force a distinct blocker. planted_private_path places a private home path in a file body and expects red with the match stored as a hash. planted_key_shape plants a vendor key shape and expects red without the raw key appearing anywhere in the result. path_pattern_blocked carries a non-public paths shape in a file name so the block comes from _scan_path, not file contents. The focused tests confirm each of these, and they assert directly that the non-public paths, the key shape, and the symlink target do not survive into the emitted record.

Prior Art Grounding

Data-loss-prevention and secret-scanning lineage

The component follows data-loss-prevention and secret-scanning practice, anchored by NIST DLP, GitHub secret scanning, and Gitleaks, contributing the hash-only result record and policy-exception split over one projection tree. The component follows data-loss-prevention and secret-scanning practice: scan artifacts before public sharing, detect account secret-shaped strings and non-public paths markers, keep enough evidence for triage, and never copy the sensitive payload into the report. Relevant anchors are NIST's Data Loss Prevention work, GitHub secret scanning, and Gitleaks, which this gate can call as an optional second pass. The contribution here is the hash-only result record and the policy-exception split, narrowed to one rendered projection tree.

Validation Result record Path

Reader-verifiable result record is

The reader-verifiable result record is the focused test suite plus the corpus parity check. The reader-verifiable result record is the focused test suite plus the corpus parity check:

PYTHONPATH=src ./repo-pytest tests/test_engine_room_public_projection_leak_gate.py -q --basetemp /tmp/microcosm-public-projection-leak-gate
cd microcosm-substrate && PYTHONPATH=src ../repo-python scripts/build_doctrine_projection.py --check-paper-module-corpus

The public exercise runs the same fixture matrix directly:

PYTHONPATH=src python3 -m microcosm_core.engine_room.public_projection_leak_gate evaluate-fixtures --input fixtures/first_wave/engine_room_public_projection_leak_gate/input --json

What a pass means here

A pass means the five fixture cases behave as expected and the hash-only handling holds; it approves no launch and proves no whole-system safety. A pass means the five fixture cases behave as expected and the hash-only handling holds. It does not approve launch or prove whole-system safety.