Research & science

Abstract

research_replication_rubric_artifact_replay is a public Microcosm component that turns "an agent replicated a paper" into a replayable evidence contract. It does not rerun a real paper, use external model services, certify benchmark performance, or grant publishing-scope decision. It checks whether a public replay bundle exposes the objects a replication claim must cite before its authority can rise: contribution decomposition refs, rubric-tree refs, allowed input refs, scratch-scaffold refs, experiment-DAG refs, metric-script refs, declared artifact hashes, grader reports, runtime budgets, ablation diffs, failure taxonomies, cold-rerun refs, public execution-trace spans, and source-module digests.

The technical result is an R3 local artifact replay: one public metric script is executed over one allowed public input table, the produced output is compared with a declared output artifact, and the declared hash file is checked against that artifact. A successful run says the replay packet is structurally accountable, digest-bound, redaction-aware, and negative-case tested. It does not say that a real paper was independently replicated.

Purpose

The single question this component answers is narrow: before an agent is allowed to say it replicated a paper, can the claim be forced into a bundle that a cold runtime can check without trusting any prose? The interesting move is that the component refuses to treat "replicated" as one fact. It pulls the claim apart into the objects a real replication would have left behind, a contribution decomposition, a grading rubric tree, the allowed public inputs, an experiment DAG, metric scripts, declared artifact hashes, a grader report, a runtime budget, an ablation diff, a failure taxonomy, and a cold-rerun result record, and it asks for each one by name.

What keeps this from being a checklist linter is the small executable core. The exported bundle does not just assert that an artifact hash exists. The runtime reads one public metric script, runs it over one allowed public input table, produces an output, and then checks that output against both the declared output artifact and the declared hash file. A replay row can name all the right refs and still fail here if the numbers do not reproduce. The negative-case fixtures attack exactly the gap a plausible fake would exploit: report-only success, benchmark-performance language, final-answer-only grading, undeclared hashes, and reuse of the original author's code.

The deliberately modest part is the subject matter. The two paper bundles are public synthetic examples, and the metric is a single sum over a small table. The component's value is the boundary, not the science. It does not run a real paper, call a provider, search compute without bound, or grant any launch or publishing-scope decision. It only makes a replication claim accountable enough that an independent reader can see where the evidence stops.

Telos

Research-agent demos often collapse four objects into one sentence: the paper, the runnable artifact, the grading rubric, and the evidence that an independent rerun happened. This component keeps those objects separate. A replay is admissible only when it names each evidence object and when the local runtime can check the public artifact replay without touching private paper bodies, non-public data bodies, hidden rubrics, model-output data, original-author code bodies, or launch/publishing-scope decision.

The central bet is modest and technical: before any replication claim is made, the system can force the claim into a falsifiable bundle with declared hashes, bounded metric execution, metadata-only result records, and explicit scope boundaries.

Mechanism

The mechanism row is mechanism.research_replication_rubric_artifact_replay.validates_public_research_replication_replay. It runs in src/microcosm_core/organs/research_replication_rubric_artifact_replay.py and is backed by the functions run, run_replication_bundle, validate_source_module_imports, validate_projection_protocol, validate_replication_policy, validate_research_replays, _build_result, _freshness_basis, and the constants EXPECTED_NEGATIVE_CASES, AUTHORITY_CEILING, SOURCE_MODULE_MANIFEST_REF, BUNDLE_RESULT_NAME, and CARD_SCHEMA_VERSION.

The runtime has two modes:

• Fixture mode reads fixtures/first_wave/research_replication_rubric_artifact_replay/input, includes positive replay rows plus eight negative-case fixtures, and writes first-wave result, board, validation, and sign-off result records.
• Exported-bundle mode reads examples/research_replication_rubric_artifact_replay/exported_research_replication_bundle, validates the public runtime example, checks the source-module manifest, and writes receipts/runtime_shell/demo_project/organs/research_replication_rubric_artifact_replay/exported_research_replication_bundle_validation_result.json.

The proof object is the tuple:

1. replication_policy.json, which states required replay fields, rubric axes, and forbidden claims.
2. research_replays.json, which supplies two synthetic paper bundles that cite public inputs, metrics, artifact hashes, grader reports, budgets, failures, and cold-rerun result records.
3. execution_artifacts/execution_artifact_manifest.json, which authorizes the replayable artifact relation.
4. source_module_manifest.json, which names copied source bodies and digest obligations.
5. Runtime result records, which expose refs, counts, digests, trace spans, and scope boundaries without embedding private bodies.

Metric-Script and Artifact Evidence

The exported bundle includes a small but real artifact-replay loop:

run_replication_bundle reads execution_artifacts/execution_artifact_manifest.json, executes the public_sum_metric over the allowed public input, compares the produced payload with execution_artifacts/artifacts/result_table.json, and verifies the declared hash in execution_artifacts/artifacts/result_table.sha256.json. The focused tests mutate each side of that relation, so the pass is not just a field-presence check.

Pipeline

flowchart TD bundle["JSON bundle authority paper_module.research_replication_rubric_artifact_replay"] policy["Replication policy required fields + rubric axes + forbidden claims"] replay["Research replay rows 2 synthetic paper bundles"] artifacts["Execution artifacts allowed input + metric spec + declared hash"] metric["Local metric replay public_sum_metric over allowed input"] source_manifest["Source-module manifest 3 source pattern slices + 1 exact-copy component body"] trace["Public execution trace 2 metadata-only spans"] negatives["Negative fixtures 8 overclaim cases"] result records["metadata-only result records counts, refs, digests, scope boundaries"] ceiling["Scope limit no replication-success or publishing-scope decision"] bundle --> policy policy --> replay replay --> artifacts artifacts --> metric source_manifest --> result records metric --> result records trace --> result records negatives --> result records result records --> ceiling

Evidence Contract

The policy file requires fourteen replay fields: paper_id, contribution_decomposition_ref, rubric_tree_ref, allowed_public_input_refs, scratch_repo_scaffold_ref, experiment_dag_ref, metric_script_refs, artifact_hash_refs, declared_artifact_hash_refs, grader_report_ref, cost_runtime_budget_ref, ablation_diff_ref, failure_taxonomy_ref, and cold_rerun_receipt_ref.

The policy also requires eight rubric axes: contribution decomposition, artifact replay, experiment DAG, metric script, grader alignment, budget boundary, failure taxonomy, and cold rerun. A replay row can therefore pass only as a structured evidence packet, not as a final answer or narrative report.

The exported runtime result record currently records the following evidence floor: two synthetic paper bundles, two replay rows, two artifact replay rows, two cold-rerun refs, two public execution-trace spans, four copied source modules, no findings, no error codes, source-module status pass, and input_mode: exported_research_replication_bundle. The fixture result record records all eight negative cases as observed.

Failure Modes and Guardrails

The expected negative cases are:

• original-author code reuse
• hidden-rubric leakage
• report-only success
• benchmark-performance overclaim
• private paper or data body leakage
• unbounded compute search
• final-answer-only grading
• undeclared artifact hash refs

The tests also cover source-module digest mismatch, local bundle body tamper, rehashing a swapped source module, wrong execution-artifact hashes, wrong artifact refs with matching hashes, report-only exported replays, metric perturbation, replay metric-script ref tamper, input perturbation, output body tamper, baked output swaps, and self-consistent input/output/hash rewrites. These cases make the component stronger than a field-presence linter: it rejects common ways to produce plausible but unaccountable replication prose.

Test Matrix

The focused regression file tests/test_research_replication_rubric_artifact_replay.py carries the source proof for this module.

Realness Rungs

This module's realness is intentionally runged:

1. Synthetic replay subjects. The two paper bundles are public synthetic examples, one ML-method replay and one computational-science replay.
2. Real schema pressure. The required fields, rubric axes, declared hash roster, source-module manifest, and non-public-state exclusions are enforced by runtime code and focused tests.
3. Local artifact replay. The exported bundle executes a local metric over allowed public input and compares produced output against declared artifact hashes.
4. Source-open provenance. Three public source pattern bodies and one exact Python internal control body are copied into the bundle and digest-checked.
5. metadata-only public result records. Result records carry counts, refs, digests, verdicts, trace spans, and scope boundaries while excluding private/live/provider material.

The rung contract matters: the component is more than generic documentation polish, but it is still not paper-replication authority.

Relation to Concepts, Principles, and Axioms

The JSON bundle binds the module to concept.research_and_science_replay_evidence_bundle. That concept is instantiated by the mechanism above and abides by AX-1, AX-6, AX-8, and AX-12 at the concept layer. The bundle's direct axiom refs are AX-1, AX-2, AX-5, and AX-7.

The bundle's principle refs are P-1, P-2, P-3, P-6, P-8, and P-15. For this component, the important principle pressure is:

• Evidence must be structured and replayable before authority rises.
• Result records and scope boundaries are part of the artifact, not commentary after it.
• Projections stay below source authority; a readable paper module does not outrank the JSON bundle, mechanism row, runtime code, source-module manifest, or result records.
• Typed refusal is part of the mechanism: benchmark, provider, public sharing, private-body, original-code, and unbounded-compute claims remain false unless another authority surface actually grants them.

The module depends on paper_module.agent_benchmark_integrity_anti_gaming_replay. Benchmark performance overclaim controls stay routed through that sibling instead of being reinvented here.

Reader Evidence Routing

Open evidence in this order:

1. core/paper_module_capsules.json#paper_module.research_replication_rubric_artifact_replay for the source-authority bundle, scope limit, doctrine refs, generated projection statuses, and code loci.
2. core/mechanism_sources.json#mechanism.research_replication_rubric_artifact_replay.validates_public_research_replication_replay for the validator command, exported-bundle validator command, focused regression, guardrails, input refs, result record refs, and upstream mechanisms.
3. standards/std_microcosm_research_replication_rubric_artifact_replay.json for the first-wave standard, public/private boundary, source-body floor, and hard launch/public sharing/provider/source-file changes flags.
4. examples/research_replication_rubric_artifact_replay/exported_research_replication_bundle/source_module_manifest.json for source-open body-floor counts and digest obligations.
5. receipts/runtime_shell/demo_project/organs/research_replication_rubric_artifact_replay/exported_research_replication_bundle_validation_result.json for the current exported-bundle validation result.
6. tests/test_research_replication_rubric_artifact_replay.py for negative cases, digest tamper tests, metric replay tests, public-relative result record tests, command-card economy, and source-body exclusion.

Prior Art Grounding

This replay scores a research artifact against a replication rubric. It follows artifact-evaluation practice from systems and machine-learning venues (ACM Artifact Review and Badging), which separates 'available' from 'functional' from 'reproduced'. Microcosm borrows the rubric-over-artifact shape; the result is fixture-bound replay evidence, not a reproducibility guarantee or a peer-review verdict.

Validation Result record Path

Focused runtime validation:

./repo-pytest tests/test_research_replication_rubric_artifact_replay.py -q --basetemp=/tmp/microcosm_research_replication_rubric_artifact_replay_pytest

Paper-module corpus validation:

./repo-python scripts/build_doctrine_projection.py --check-paper-module-corpus

The runtime commands behind the result records are:

Purpose

Spatial world-model demos are unusually easy to oversell. A plausible-looking video, or a row that simply asserts "the model predicted the next state correctly", can pass for understanding without anything having been checked. This component exists to answer one narrow question: does a declared spatial counterfactual row actually bind a source state, an event, and a predicted outcome that survive an independent recomputation, or is it just a shape that looks right?

The approach is the unusual part. The predicted actor count, transition delta, event label, and spawn cells are derived from the inputs (sensor-packet refs, consistency budget, topology), so a stale or hand-edited prediction no longer matches and the row blocks. The point is not a good simulator. The point is that a spatial-AI claim cannot pass on appearance alone: it has to agree with a recomputation a reader can audit in one screen.

Abstract

spatial_world_model_counterfactual_simulation_replay is a Microcosm component for checking spatial world-model counterfactual claims as metadata transitions, not as generated video, robotics control, AV simulation, geographic truth, or benchmark authority. The component validates six synthetic scene-state rows, six counterfactual replay rows, six predicted transition rows, eight forbidden-claim negative cases, and an exported source-module bundle whose result record stays metadata-only.

The technical claim is deliberately small: for each replay row, the runtime recomputes a deterministic toy gridworld next state from the declared scene state, counterfactual event, sensor-packet refs, consistency budget, topology ref, and limitation labels; it then compares that actual transition against the declared predicted state, transition diff, and oracle check. A green run proves the public replay rows are internally consistent and bounded by their scope limit. It does not establish real-world spatial accuracy, trained simulator quality, generated-video correctness, robot or AV operation, provider behavior, hosting, public sharing, launch-scope decision, or whole-system correctness.

Telos

World-model demos are easy to overstate because visual plausibility can hide whether any state transition was checked. This component makes the proof surface inspectable: a reader can see the scene-state ref, action trace, predicted-state ref, transition-diff ref, oracle-check ref, fidelity limit, limitation labels, negative cases, and source-module digest evidence before accepting any spatial counterfactual claim.

The useful result is not a better simulator. The useful result is an evidence spine that refuses to let a spatial-AI claim advance unless the public row binds input state, counterfactual event, predicted output, actual recomputation, and scope boundary boundary in one result record.

Mechanism

The positive fixture has six scene states and six matching replay rows: warehouse occlusion, crosswalk emergence, drone-corridor gust recovery, mobile robot reflective-floor detour, loading-dock pallet shift, and unprotected-turn late yield. Each row declares a source scene-state ref, action-trace ref, counterfactual event, predicted-state ref, transition-diff ref, oracle-state-check ref, two public sensor-packet refs, a rare-event label, a fidelity-limit label, limitation labels, and explicit false values for private video, raw sensor export, live operation, geography, simulator-product, generated-video-only, benchmark, and launch claims.

Runtime transition checking happens in _state_transition_analysis:

1. The component resolves each replay to exactly one state-transition row.
2. It builds an 8 x 8 toy gridworld from the source scene's actor count and topology ref.
3. It maps the counterfactual event to a deterministic event action such as new_dynamic_actor.
4. It recomputes the actual next state and transition diff from the input row.
5. It compares predicted actor count, transition delta, event label, spawn cell or cells, predicted-state ref, diff ref, oracle-check ref, and metadata-only result record status.

The input-driven part matters. Actor-count delta is not copied from the expected fixture. It is recomputed as:

min(
  base_event_actor_count_delta
  + max(0, sensor_packet_count - max_timestep_lag - base_event_actor_count_delta),
  4,
  free_cell_count
)

Spawn cells are also input-derived: the runtime hashes the event, replay id, scene-state ref, topology ref, sensor-packet refs, consistency budget, limitation labels, and source actor count, then walks the bounded grid from the declared event cell. This makes the row sensitive to real input changes while remaining small enough to audit.

Transition Evidence

The current fixture proves a narrow but useful invariant: all six declared predicted states match the runtime's actual toy-gridworld step. The focused test expects:

• scene_state_count == 6
• replay_count == 6
• state_transition_count == 6
• predicted_state_body_count == 6
• deterministic_simulation_pass_count == 6
• gridworld_step_count == 6
• predicted_actual_match_count == 6
• transition_diff_count == 6
• oracle_state_check_count == 6
• sensor_packet_ref_count == 12

Those counts are technical evidence only because the runtime recomputes the state transition before accepting them. The result record cannot be read as a learned world-model score; it is a public replay consistency check over synthetic metadata and copied source-module digests.

Real-Bad Mutation Contract

The regression suite includes deliberately bad mutations that show the proof is not just shape validation:

• If a transition row changes actor_count_delta from the recomputed value, run_simulation_bundle blocks with SPATIAL_STATE_TRANSITION_SIMULATION_MISMATCH.
• If the predicted state misses the gridworld step, the transition row records predicted_state_actor_count_mismatch while the recomputed actual state still shows the expected gridworld execution.
• If a replay gains an extra sensor-packet ref, the recomputed actor delta moves from 1 to 2. The stale expected transition blocks until the predicted actor count, actor delta, and spawn cells are updated to match the new actual transition.
• If the source scene actor count and topology ref change, the recomputed source and spawn-cell state moves. The stale predicted state blocks until the transition row is updated.
• If a source-module manifest tries to place copied body text inside a result record, the source-module summary blocks with SPATIAL_SOURCE_BODY_TEXT_IN_RECEIPT_FORBIDDEN and SPATIAL_SOURCE_MODULE_BODY_TEXT_IN_RECEIPT_FORBIDDEN.

The negative payload cases are similarly typed: private video export, raw sensor export, live robot or AV operation, real-world location claims, simulator-product claims, generated-video-only authority, geographic accuracy claims, and benchmark-score claims without state-diff result records all have explicit forbidden-code coverage.

Shape

flowchart TD Scene["Scene-state row actor count + topology"] --> Replay["Counterfactual replay row event + sensor refs + budget"] Replay --> Step["Deterministic toy gridworld step 8x8 bounded recomputation"] Step --> Actual["Actual next state actor delta + spawn cells"] Replay --> Expected["Declared predicted state transition diff + oracle check"] Actual --> Compare{"Actual matches declared transition?"} Expected --> Compare Compare -->|yes| Result record["metadata-only pass result record counts + refs + digests"] Compare -->|no| Finding["Typed mismatch finding blocked status"] Replay --> Boundary{"Forbidden payload or claim?"} Boundary -->|no| Result record Boundary -->|yes| Finding

This diagram is a reader map for the runtime proof. The generated doctrine lattice Mermaid remains the bundle-derived edge proof.

Reader Evidence Routing

Read this page from source authority outward:

1. Open core/paper_module_capsules.json::paper_modules[53:paper_module.spatial_world_model_counterfactual_simulation_replay] for the JSON bundle and scope limit.
2. Open paper_modules/spatial_world_model_counterfactual_simulation_replay.json for generated relationship edges, Mermaid status, Atlas status, and source_authority: json_capsule.
3. Inspect src/microcosm_core/organs/spatial_world_model_counterfactual_simulation_replay.py, especially _state_transition_analysis, _gridworld_step, _gridworld_actor_count_delta, _gridworld_spawn_cells, _replay_policy_findings, and _source_module_manifest_result.
4. Inspect fixture inputs under fixtures/first_wave/spatial_world_model_counterfactual_simulation_replay/input and exported-bundle inputs under examples/spatial_world_model_counterfactual_simulation_replay/exported_spatial_world_model_simulation_bundle.
5. Inspect tests/test_spatial_world_model_counterfactual_simulation_replay.py for the positive replay, public-relative result record, source-module import, body-text rejection, transition-delta mutation, predicted-state mutation, input-perturbation, scene-perturbation, and fresh-card reuse contracts.

Runtime Command

microcosm spatial-world-model-counterfactual-simulation-replay run-simulation-bundle --input examples/spatial_world_model_counterfactual_simulation_replay/exported_spatial_world_model_simulation_bundle --out receipts/runtime_shell/demo_project/organs/spatial_world_model_counterfactual_simulation_replay

The runtime shell also exposes the compressed lens at:

microcosm spatial-simulation

Prior Art Grounding

This replay exercises a spatial world model under counterfactual interventions. It is grounded in the world-models line of work (Ha and Schmidhuber, World Models), where an agent learns a compressed model of its environment it can roll forward under hypothetical actions. Microcosm borrows the counterfactual-rollout shape over synthetic metadata; the result is fixture-bound replay evidence, not robot or AV operation, real-world geography, or a calibrated simulator.

Validation Result record Path

Run from microcosm-substrate:

The expected bundle projection is Mermaid available_from_capsule_edges, Atlas linked_from_capsule_edges, and 20 generated relationship edges. These checks prove the public synthetic replay and source-module import boundary only; they do not validate real geography, robot or AV operation, simulator-product claims, benchmark claims, public sharing, hosting, or launch.

Purpose

"Closed-loop materials lab" is one of the easier phrases to overclaim. A fixture can look like an autonomous discovery loop while carrying nothing that should be spoken aloud: wetlab steps, reagent quantities, a controlled or bioactive target, robot commands, or a flat assertion that some material was discovered. This component exists to sit in front of that language and answer one question: is a closed-loop-lab-shaped fixture safe and grounded enough to be talked about at all, in a simulator-only frame, before any lab claim is allowed?

Its real name inside the runtime is the materials_chemistry_artifact_safety_refusal_validator. The public-promise name "closed-loop replay" was deliberately reframed because nothing here executes a wetlab loop or commands a robot. The unusual part is that the component does not trust the fixture's own conclusion. A normal replay would read a declared "selected candidate" label and report it. This validator instead recomputes the winner from public numbers, weighting an assay proxy, an active-learning score, and a safety gate, then treats a mismatch between that recomputed pick and the declared label as a failure rather than a footnote. A stale or flattering label cannot pass.

The second discipline is refusal as a first-class result. Eight categories of dangerous or overclaiming content each have a named forbidden code, and a fixture that smuggles one in is expected to be refused, not quietly accepted. The verdict is computed from public simulator rows, safety fields, source-module manifests, replay-graph status, negative-case coverage, and a sentinel scan, and it stays inside a simulator-only ceiling. It is a safety and refusal check, not a laboratory.

Abstract

materials_chemistry_closed_loop_lab_safety_replay is a public, simulator-only replay validator for materials-lab language. It does not claim a material discovery, a wetlab protocol, a robot loop, or a benchmark. It checks whether a closed-loop-lab shaped public fixture has enough evidence to be talked about at all: candidate material refs, safety-screen refs, simulator-only assay rows, active-learning decisions, a Lab/Evolve replay graph, source-module manifest digests, negative-case refusals, metadata-only result records, and an explicit scope limit.

The technical claim is a numeric verdict proof boundary. A passing run must recompute the selected candidate from score-backed fixture rows rather than trusting a declared label. The baseline fixture contains four candidates and selects mat_polymer_membrane_001 with score 0.917; perturbation tests prove that stale labels, missing score rows, out-of-range scores, and safety-gate failures block the verdict.

Mechanism

The runtime locus is src/microcosm_core/organs/materials_chemistry_closed_loop_lab_safety_replay.py. The relevant entrypoints are run for first-wave fixture validation and run_lab_bundle for exported-bundle validation. The validator loads a replay policy, candidate rows, experiment DAG rows, simulator assays, active-learning decisions, optional source-module manifests, and eight forbidden negative-case fixtures.

The sign-off rule is deliberately small:

1. Positive rows must link candidates, experiments, assays, safety screens, active-learning decisions, failure taxonomy refs, and cold replay refs.
2. Negative cases must be observed and refused.
3. Numeric replay must recompute the selected candidate from public numbers.
4. Source-module imports must verify copied bodies without putting bodies into result records.
5. The safety verdict must remain inside the simulator-only scope limit.

flowchart TD policy["replay_policy.json numeric policy + expected label"] candidates["candidate_materials.json 4 candidate refs + safety gates"] assays["simulator_assays.json 4 public assay proxy values"] decisions["active_learning_decisions.json 4 active-learning scores"] numeric["numeric replay weighted recompute of the winner"] labelcheck{"recomputed pick == declared label? safety gate >= 0.70?"} negatives["negative-case fixtures 8 forbidden lab classes"] refuse{"any forbidden MATERIALS_*_FORBIDDEN observed?"} manifest["source_module_manifest.json 4 copied public body modules"] replay["Lab/Evolve replay graph replay cases"] verdict["safety verdict"] accepted["public_safe_simulator_replay_accepted"] blocked["blocked_public_safety_boundary"] result record["metadata-only result records counts, digests, findings"] ceiling["scope limit no wetlab / no discovery / no launch"] policy --> numeric candidates --> numeric assays --> numeric decisions --> numeric numeric --> labelcheck labelcheck -->|stale label or gate fail| blocked labelcheck -->|match| verdict negatives --> refuse refuse -->|yes| blocked refuse -->|no| verdict manifest --> replay replay --> verdict verdict --> accepted accepted --> result record blocked --> result record result record --> ceiling

Numeric Assay And Verdict Evidence

The replay policy declares:

• selection rule: max_weighted_public_assay_active_learning_and_safety_gate_score
• minimum safety gate: 0.70
• expected selected candidate: mat_polymer_membrane_001
• weighted score: 0.45 * public_assay_proxy_value + 0.35 * public_active_learning_score + 0.20 * public_safety_gate_score

The source fixture binds four score-backed rows:

The focused regression test_materials_chemistry_numeric_replay_recomputes_verdict_from_fixture_numbers proves the pass case: status pass, verified_numeric_row_count == 4, selected candidate mat_polymer_membrane_001, selected decision decision_membrane_001, selected next action simulate_assay, score 0.917, realness rung R3, and verdict basis recomputed_from_public_assay_active_learning_and_safety_gate_fixture_numbers.

The verifier does not use expected labels for selection. Expected labels are checked only after the selected row is recomputed from candidate, assay, and decision content.

Test Matrix

These cases are source/test-backed by tests/test_materials_chemistry_closed_loop_lab_safety_replay.py. Fresh local first-wave result record output is the authority for current numeric replay; older archived first-wave result records and the checked-in exported bundle predate the numeric replay rows and should not be read as the numeric proof. The exported bundle still needs refreshed numeric rows before it is a full exported-bundle pass.

Evidence Routes

• JSON bundle: core/paper_module_capsules.json::paper_module.materials_chemistry_closed_loop_lab_safety_replay
• Generated JSON instance: paper_modules/materials_chemistry_closed_loop_lab_safety_replay.json
• Mechanism source: core/mechanism_sources.json::mechanism.materials_chemistry_closed_loop_lab_safety_replay.validates_public_materials_lab_safety_replay
• Runtime: src/microcosm_core/organs/materials_chemistry_closed_loop_lab_safety_replay.py
• Domain standard: standards/std_microcosm_materials_chemistry_closed_loop_lab_safety_replay.json
• Paper-module standard: standards/std_microcosm_paper_module.json
• Fixture input: fixtures/first_wave/materials_chemistry_closed_loop_lab_safety_replay/input
• Exported bundle: examples/materials_chemistry_closed_loop_lab_safety_replay/exported_materials_lab_safety_bundle
• Focused tests: tests/test_materials_chemistry_closed_loop_lab_safety_replay.py

Prior Art Grounding

This replay exercises a closed-loop materials and chemistry lab controller with a safety gate over synthetic experiments. It is grounded in the self-driving laboratory literature, where a propose-run-measure loop is paired with safety interlocks that can refuse an unsafe experiment. Microcosm borrows the loop-plus-safety-gate shape on a simulator; the result is metadata-only simulator evidence, not a real laboratory controller, chemical-safety authority, or launch.

Validation Result record Path

Run the current runtime proof from the Microcosm root:

Inspect the exported source-body bundle. Until the exported fixture is refreshed with score-backed numeric rows, this command may return a blocked numeric verdict while still proving the manifest/body-floor boundary:

cd microcosm-substrate
PYTHONPATH=src ../repo-python -m microcosm_core.organs.materials_chemistry_closed_loop_lab_safety_replay run-lab-bundle --input examples/materials_chemistry_closed_loop_lab_safety_replay/exported_materials_lab_safety_bundle --out /tmp/microcosm_materials_chemistry_lab_safety_bundle

Run the focused regression suite:

cd microcosm-substrate
PYTHONPATH=src ../repo-pytest tests/test_materials_chemistry_closed_loop_lab_safety_replay.py -q

cd microcosm-substrate
PYTHONPATH=src ../repo-python scripts/build_doctrine_projection.py --check-paper-module-corpus

This lane intentionally does not run scripts/build_doctrine_projection.py --write; generated projections, atlas cards, and shared bundle surfaces belong to their owner lanes.

Purpose

Interpretability writing is unusually easy to overstate. A named feature can read like understanding, a graph picture can read like a discovered circuit, and a small local script can read like access to a real model. This component exists to hold one kind of claim to a smaller, checkable size. It answers a single question: before Microcosm lets a circuit-attribution story stand as public evidence, does the story survive a deterministic replay rather than being taken on trust?

The part worth noticing is how narrow the proof is, and how that narrowness is the point. The component does not attempt to interpret a trained model. It carries a tiny two-layer toy transformer with weights declared in the fixture, recomputes its forward pass, gradient attribution, and per-feature ablation, and then compares the recomputed top feature against the feature the fixture claims. A row passes only when the declared winner still matches after recomputation. Perturb the toy weights and leave the old claim in place and the row is rejected, because the recomputed answer has moved while the prose has not. That is the failure mode the component is built to catch: an interpretability statement that was once true of its inputs but no longer is.

Around that recomputation sit three further gates. Graph evidence must be machine-readable and traversable from declared sparse features to public error nodes, so a screenshot cannot stand in for a circuit. Transparency language needs a causal-intervention reference and faithfulness language needs an explicit limit, so the strongest words carry the strongest evidence requirements. Private weights, raw activations, proprietary prompts, and hidden reasoning are kept out of every result record. What the component produces is an accounting result record for a public fixture, not a transparency tool for any real model.

Abstract

mechanistic_interpretability_circuit_attribution_replay is a public Microcosm component that validates whether circuit-attribution claims are safe to represent as result record evidence. It is not a model-transparency product and does not inspect a live provider model. The component checks a fixture and exported bundle for machine-readable feature graph rows, causal-intervention references, faithfulness limits, source-module digest evidence, negative cases, and a small input-coupled toy-transformer replay.

The technical proof is deliberately modest. A replay passes only when its declared circuit-attribution story agrees with recomputed toy-transformer forward, gradient, and ablation winners; when graph evidence is traversable from public sparse features to public error nodes; when public result records omit private or raw bodies; and when the source-open body floor is backed by copied, source source modules with matching digests. A stale declared top feature is disconfirmed by perturbing the input fixture while leaving the old claim in place.

Problem Statement

Interpretability prose is easy to overclaim: a feature name can sound like transparency, a graph screenshot can sound like a circuit, and a local fixture can sound like model access. This module makes the public claim smaller and more testable. It asks: before Microcosm lets a circuit-attribution story become public evidence, can the story survive a deterministic replay membrane that checks structure, causality refs, source provenance, and explicit scope boundaries?

The answer is local and result record-scoped. Microcosm may claim public circuit-attribution replay accounting for this fixture and exported bundle. It may not claim live model internals, private weights, raw activations, proprietary prompts, hidden reasoning, provider behavior, benchmark claims, publishing-scope decision, hosting, launch-scope decision, or whole-system interpretability correctness.

The technical contribution is therefore an accounting membrane, not a new interpretability algorithm. The membrane turns an interpretability-shaped fixture into a pass/fail public result record by requiring all claim-bearing rows to cross four gates:

Technical Mechanism

flowchart TD Bundle["JSON bundle paper_module.mechanistic_interpretability_circuit_attribution_replay"] Fixture["Fixture / exported bundle feature catalog, replay rows, toy-transformer spec"] Policy["Policy gates required fields, forbidden private/raw exports, faithfulness limits"] Graph["Graph analyzer feature ids -> edges -> public error nodes"] Toy["Toy-transformer replay forward + gradient + ablation recomputation"] Source["Source-open body floor copied source bodies + digest checks"] Result records["metadata-only result records refs, digests, counts, verdicts"] Ceiling["Scope limit public replay accounting only"] Bundle --> Fixture Fixture --> Policy Fixture --> Graph Fixture --> Toy Fixture --> Source Policy --> Result records Graph --> Result records Toy --> Result records Source --> Result records Result records --> Ceiling

The component has four coupled checks:

1. Replay policy validation: each positive row must carry toy prompt refs, sparse feature ids, machine-readable graph nodes and edges, replacement-model approximation scores, causal inhibition and injection refs, causal-intervention result record refs, sufficiency labels, faithfulness limits, contradiction-case refs, cold-replay refs, target refs, and body_in_receipt: false.
2. Graph analysis: _graph_analysis_for_replay verifies that graph edges resolve to declared nodes and that at least one path exists from the row's sparse feature ids to a public error node. _weight_sequence_analysis rejects simple decorative arithmetic edge-weight sequences across replay rows.
3. Toy-transformer replay: _toy_transformer_attribution_runtime recomputes a pure-Python two-layer toy transformer from fixture-provided token_ids, embeddings, layer1, layer2, and target_logit_index, then compares the recomputed top attribution and ablation features against declared winners.
4. Source/body boundary: _source_module_manifest_result, _source_open_body_import_summary, scan_paths, _write_receipts, and result_card verify copied source bodies while keeping result record payloads metadata-only and public-safe.

Implementation Contract

Toy-Transformer Attribution Mechanism

The toy-transformer runtime is intentionally small enough to audit. The fixture in fixtures/first_wave/mechanistic_interpretability_circuit_attribution_replay/input/attribution_replays.json declares:

• token_ids: [0, 1, 2]
• a three-row embedding table over two dimensions
• a two-by-three first layer
• a three-by-two second layer
• target_logit_index: 1
• expected top feature by attribution and ablation: toy_hidden_feature_1

The runtime computes token embeddings, averages them into a context vector, applies the first layer, applies a tanh hidden activation, applies the second layer, and reads the target logit. It then computes activation-gradient scores for each hidden feature, using the analytic tanh derivative 1 - h^2 so the attribution score is grounded in the same forward pass rather than a separate estimate. It also ablates each hidden feature in turn, zeroing it and re-reading the target logit, to measure the output delta that feature is responsible for. The fixture currently produces target logit 0.044176; both the gradient attribution and the ablation delta select toy_hidden_feature_1, and the row passes only because those two independent paths agree with each other and with the fixture's declaration.

The important point is not that this is a serious transformer. It is a deterministic proof harness for the public replay claim. The result record can say the declared top feature agrees with recomputation only because the verifier recomputes from input fields and compares the result. The result record also records a weight digest so cached or exported bundle cards can prove which fixture basis they are coupled to.

Discriminating Tests

The proof is strongest where it distinguishes a real coupling from a plausible but stale story. The focused tests exercise those distinctions directly:

Evidence Contract

Reader Evidence Routing

The proof consumer for this reader slice is the focused interpretability replay suite plus the paper-module corpus parity check. The table below is the route a rank/projection reader should follow before trusting any claim in this module:

Failure Modes And Limitations

• Missing required replay fields block with INTERPRETABILITY_REPLAY_FIELD_REQUIRED.
• Feature names without catalog-backed ids block with INTERPRETABILITY_FEATURE_NAME_UNVERIFIABLE.
• Graph screenshots or disconnected graph rows block because machine-readable edges and traversable paths are required.
• Transparency language without a causal-intervention result record blocks with INTERPRETABILITY_INTERVENTION_RECEIPT_REQUIRED.
• Faithfulness language without explicit limits blocks with INTERPRETABILITY_FAITHFULNESS_REQUIRES_LIMITS.
• Private model weights, raw activation dumps, proprietary prompt exports, hidden chain-of-thought exports, model-output data bodies, and launch-scope decision are forbidden public outputs.
• Decorative graph-weight sequences block as suspected fabrication.
• Stale declared toy-transformer winners block when recomputation selects a different top feature.
• The proof is fixture-local. It verifies a public replay membrane and copied source evidence; it does not certify real-world model faithfulness.

Relation To Interpretability Literature

The module borrows its accounting shape from the transformer-circuits and mechanistic-interpretability tradition: circuits should be graph-structured, features should be identifiable, causal language should be backed by interventions, and faithfulness language should be bounded. Useful prior-art anchors include Anthropic's transformer-circuits framing, causal scrubbing, and SAE/sparse-feature circuit work.

Microcosm does not reproduce those methods. The local contribution is a public replay boundary around an interpretability-shaped claim: machine-readable edges instead of screenshots, causal-intervention refs instead of bare transparency language, fixture recomputation instead of stale row trust, and explicit scope boundaries before a claim becomes public evidence.

Relation To Microcosm Concepts, Mechanisms, And Principles

The bundle binds this module to:

• concept.research_and_science_replay_evidence_bundle
• mechanism.mechanistic_interpretability_circuit_attribution_replay.validates_public_mechanistic_interpretability_circuit_attribution_replay
• principles P-2, P-4, P-8, and P-9
• axioms AX-3, AX-5, AX-7, and AX-8

The practical reading is:

• P-2: claim language stays below the strength of the checker.
• P-4: public proof routes through result records and explicit evidence refs.
• P-8: failed preconditions are typed refusals, not vague warnings.
• P-9: provenance crosses from fixture, source source, and result record without upgrading authority.
• AX-3: dereferenced proof and policy refs matter more than prose labels.
• AX-5: status fails closed across all required parts.
• AX-7: partial computation returns a typed refusal.
• AX-8: public fixture and copied-source labels propagate without becoming private model access.

Named Proof Consumers

Run from microcosm-substrate:

This consumes the first-wave fixture, negative cases, source-module mirror, secret scan, toy-transformer replay, and result record writer.

PYTHONPATH=src ../repo-python -m microcosm_core.organs.mechanistic_interpretability_circuit_attribution_replay run-attribution-bundle \
  --input examples/mechanistic_interpretability_circuit_attribution_replay/exported_circuit_attribution_bundle \
  --out /tmp/microcosm-mechanistic-interpretability-circuit-attribution-replay/bundle \
  --card

This consumes the exported circuit-attribution bundle, copied body floor, digest checks, metadata-only result records, command-card omission contract, and runtime-shell validation shape.

PYTHONPATH=src ../repo-python -m pytest -p no:cacheprovider tests/test_mechanistic_interpretability_circuit_attribution_replay.py -q
PYTHONPATH=src ../repo-python scripts/build_doctrine_projection.py --check-paper-module-corpus

The focused regression pins recomputation, stale-row rejection, graph and source-body gates, card result record reuse, and body-text exclusions.

Reader Route

A cold reader should inspect in this order:

1. core/paper_module_capsules.json row 52 for authority and projection binding.
2. paper_modules/mechanistic_interpretability_circuit_attribution_replay.json for generated relationship edges.
3. src/microcosm_core/organs/mechanistic_interpretability_circuit_attribution_replay.py for runtime logic.
4. tests/test_mechanistic_interpretability_circuit_attribution_replay.py for the stale-row, perturbation, graph, source-body, and result record-boundary proof.
5. fixtures/first_wave/mechanistic_interpretability_circuit_attribution_replay/input for the fixture.
6. examples/mechanistic_interpretability_circuit_attribution_replay/exported_circuit_attribution_bundle for the public bundle.
7. receipts/first_wave/mechanistic_interpretability_circuit_attribution_replay and receipts/runtime_shell/public_mechanistic_interpretability_circuit_attribution_replay_lens.json for metadata-only public result record evidence.

Prior Art Grounding

This replay exercises a circuit-attribution pass that traces which internal components account for a behaviour. It is grounded in mechanistic interpretability, the study of the internal circuits of neural networks (Anthropic, Transformer Circuits). Microcosm borrows the attribution-replay shape over synthetic fixtures; the result is fixture-bound runtime evidence, not live model access, a transparency product, or a correctness claim about any real model.

Validation Result record Path

Reader-verifiable commands, run from the microcosm-substrate/ public root:

PYTHONPATH=src python3 -m pytest tests/test_mechanistic_interpretability_circuit_attribution_replay.py -q
PYTHONPATH=src python3 scripts/build_doctrine_projection.py --check-paper-module-corpus

These are reader-verifiable evidence only and do not include launch operations, external model access, source-file changes, or whole-system correctness.

prediction_oracle_reconciliation is a source-available runtime fixture component for the prediction-engine slice. It compresses the source pattern group around CP1 bifurcation resolution, CP2 valid target universes, oracle grounding firewalls, diff grading, and dossier mutation into a synthetic packet a cold reader can run.

It is deliberately not a market product. The component has no live data, no external model access, no trading authority, no financial or investment-related actions authority, no publishing-scope decision, and no launch-scope decision. Its job is to make the reasoning shape inspectable without making performance or action claims. The result record contract is source-open by default: public fixture packets, exported bundle refs, source refs, and runtime result records carry the evidence, while secret_exclusion_scan blocks only live market feeds, model-output data bodies, account or browser material, private dossiers, and account secret-equivalent access.

Purpose

A forecast that gets the direction right can still be badly wrong about the number, and a forecast can look accurate only because it quietly used evidence that arrived after the outcome it was meant to predict. This component exists to make those two failures visible on a synthetic packet, before any reasoning is dressed up as a track record. The single question it answers is narrow: does this prediction packet keep its evidence honest and its grading recomputable, or does it cut a corner?

The unusual choice is that the component does not trust the numbers the packet reports. For every numeric row it recomputes the absolute error, the percent error, and the direction hit from the snapshot, predicted, and realized prices, then rejects any claimed value that contradicts the recompute. It also surfaces a direction hit that is still a large numeric miss rather than letting the correct arrow hide the size of the error. Evidence is split at the prediction time: a reference that points past the target window is refused, not silently scored.

None of this is forecasting. There is no live market data, no external model access, no trading or investment-related actions, and no performance claim. The packet, its target universe, and its realized values are invented fixtures. A direction hit or a numeric miss inside a result record is a statement about the fixture and the grading mechanics, nothing more.

Public Contract

The input packet names:

• source_pattern_ids for the source pattern family being projected.
• valid_prediction_targets and target_universe for the CP2 gate.
• cp1_branches with selected side, rationale refs, and opposite-side invalidation refs.
• cp2_predictions with pre-target evidence refs and grounding ids.
• oracle_diff rows that grade synthetic realized direction against prediction.
• dossier_mutations constrained to fixture deltas.
• public_runtime_refs for the public fixture, exported bundle, and paper module system refs.
• authority_ceiling values that explicitly keep trading, advice, provider, live-market, public sharing, launch, and secret-export authority false.

How it works

validate_reconciliation_packet runs five checks over the packet and folds the findings into one status. Each check guards a specific way a forecast can flatter itself.

CP1 resolution. Every cp1_branches row must name the side it chose, carry rationale refs, and keep an opposite_side_invalidation_ref, the record of why the losing side lost. A branch that asserts a winner without retaining the discarded alternative is rejected as an unresolved bifurcation. Equity or market-lane branches additionally need an explicit confirmation bit before they count.

CP2 universe and pre-target evidence. Predictions must name a target_id inside the declared valid_prediction_targets, so the set of things being predicted is fixed before the outcome rather than chosen afterwards. Evidence refs must be pre-target: a ref is accepted only if it carries the T- time prefix, and a reference that points past the target window raises PREDICTION_ORACLE_POST_T_EVIDENCE_FORBIDDEN. This is the gate that stops a packet from grading itself with hindsight.

Recomputed numeric grading. This is the part that does real arithmetic. For each graded row the component takes the snapshot, predicted, and realized prices and recomputes the absolute delta, the percent delta against the snapshot, and the direction hit. If the row also reports its own abs_error, pred_error_pct, or direction_hit, the claimed value must match the recompute or the row is rejected. Two further rules matter. A row whose direction is correct but whose error clears the floor (ten in absolute terms, or five percent) is surfaced as a large miss, so a right arrow cannot conceal a large numeric error. A row with no realized price is not fabricated into a graded row, a row marked degraded is gated out of grading rather than scored, and the STOCK and ETF asset classes are kept as separate counts rather than blended.

Oracle diff and bounded mutation. The oracle_diff rows grade synthetic realized direction against each prediction, and dossier_mutations may only add a contradiction, revise a confidence band, or retire a claim. A high-severity mutation needs two evidence refs and an explicit public-delta allowlist before it is allowed.

A run passes only when at least two CP1 branches, two CP2 predictions, two graded numeric rows across both asset classes, and one bounded mutation are present, the recompute and evidence gates raise no findings, the source-module digests match, and the secret scan is clean. The result record records counts, verdicts, and authority booleans; the packet body, claimed numbers, and source bodies stay out of it.

Shape

flowchart TD Packet["Synthetic prediction packet target universe, CP1 branches, CP2 predictions, oracle diff, numeric rows, dossier mutations"] CP1["CP1 resolution chosen side + rationale + why the opposite side lost; equity lane needs confirmation"] CP2["CP2 universe + evidence target inside declared universe; evidence must be pre-target (T-)"] Numeric["Recomputed numeric grading abs error, percent error, direction hit recomputed; claimed values must match"] Oracle["Oracle diff + mutation realized vs predicted direction; bounded dossier deltas"] LargeMiss["Direction-right, numeric-miss surfaced, not hidden"] Gated["Degraded / missing-truth rows gated, not fabricated"] Result records["metadata-only result records result, board, validation, sign-off; counts and verdicts"] Ceiling["Scope limit synthetic fixture only; no trading, advice, provider, live market, publish, launch"] Packet --> CP1 Packet --> CP2 Packet --> Numeric Packet --> Oracle Numeric --> LargeMiss Numeric --> Gated CP1 --> Result records CP2 --> Result records LargeMiss --> Result records Gated --> Result records Oracle --> Result records Result records --> Ceiling

Evidence/accounting:

• Bundle authority: core/paper_module_capsules.json::paper_modules[54:paper_module.prediction_oracle_reconciliation] sets source_authority: json_capsule, binds the component, binds mechanism.prediction_oracle_reconciliation.validates_public_prediction_oracle_reconciliation, and resolves src/microcosm_core/organs/prediction_oracle_reconciliation.py.
• Generated instance: paper_modules/prediction_oracle_reconciliation.json reports paper_module_payload.source_authority: json_capsule, Mermaid available_from_capsule_edges, Atlas linked_from_capsule_edges, 15 relationship edges, and no unpopulated selective relations.
• Runtime and fixture floor: src/microcosm_core/organs/prediction_oracle_reconciliation.py exposes run, run_prediction_bundle, validate_source_module_imports, validate_reconciliation_packet, _source_open_body_import_summary, write_receipts, EXPECTED_NEGATIVE_CASES, and AUTHORITY_CEILING. fixtures/first_wave/prediction_oracle_reconciliation/input/reconciliation_packet.json carries the synthetic CP1/CP2, oracle-diff, target-universe, and dossier-mutation evidence shape.
• Exported bundle and result records: examples/prediction_oracle_reconciliation/exported_prediction_oracle_bundle/source_module_manifest.json and the exported source artifacts provide source-open replay evidence. receipts/first_wave/prediction_oracle_reconciliation/prediction_oracle_reconciliation_result.json, prediction_oracle_validation_receipt.json, and result records/sign-off/first_wave/prediction_oracle_reconciliation_fixture_acceptance.json keep the result record metadata-only and fixture-bounded.
• Test and claim boundary: tests/test_prediction_oracle_reconciliation.py checks invalid target universes, unresolved CP1 branches, post-target evidence, unsafe dossier mutation, live-market/trading/advice overclaims, exported-bundle validation, and source-module digest gates. The structured source record scope limit excludes forecasting correctness, financial decisions, trading authority, live market data, external model access, prediction public sharing, performance track record, non-public data import, launch-scope decision, publishing-scope decision, and whole-system correctness.

Reader Evidence Routing

Open this module as a reader map, not as prediction evidence. Use the runtime fixture input for packet shape, the exported bundle for source-open replay, the structured source record for relationship edges, and the test file for the negative cases that enforce the scope limit.

Route evidence in this order:

1. Read the structured lattice bindings section to confirm the source record path and subject edges.
2. Inspect the fixture input for declared target universes, CP1 branches, CP2 prediction evidence, oracle-diff rows, and fixture-bounded dossier mutations.
3. Run the fixture and exported-bundle commands to produce metadata-only result records.
4. Check tests/test_prediction_oracle_reconciliation.py for the negative cases that reject target-universe escapes, unresolved CP1 branches, post-target evidence, live-market overclaims, and authority overclaims.
5. Use paper_modules/prediction_oracle_reconciliation.json as the generated relationship graph for this module.

Negative Cases

The fixture rejects:

• a CP2 prediction outside the target universe;
• an unresolved CP1 bifurcation;
• post-target evidence used as prediction evidence;
• unconfirmed equity or market-lane claims;
• unsafe high-severity dossier mutation;
• trading, advice, live-provider, public sharing, launch, or secret-export authority overclaims.

Prior Art Grounding

This component is grounded in probabilistic forecast evaluation and prediction market infrastructure. The Brier score is an early probability-forecast verification anchor, proper-scoring-rule work such as Gneiting and Raftery motivates incentive-compatible forecast scoring, and Hanson's logarithmic market scoring rule grounds the prediction-market idea that forecasts can be updated and evaluated through explicit scoring mechanisms. Forecasting tournament work around tracking and calibration also motivates separating prediction evidence from post-outcome explanation.

Microcosm borrows the reconciliation pattern: declare the target universe before the outcome, keep pre-target evidence separate from post-target evidence, grade against a synthetic oracle diff, and constrain dossier mutation to declared fixture deltas. It does not trade, advise, publish predictions, or claim forecast performance.

Commands

PYTHONPATH=src python3 -m microcosm_core.organs.prediction_oracle_reconciliation run \
  --input fixtures/first_wave/prediction_oracle_reconciliation/input \
  --out receipts/first_wave/prediction_oracle_reconciliation

PYTHONPATH=src python3 -m microcosm_core.organs.prediction_oracle_reconciliation run-prediction-bundle \
  --input examples/prediction_oracle_reconciliation/exported_prediction_oracle_bundle \
  --out receipts/runtime_shell/demo_project/organs/prediction_oracle_reconciliation

Validation Result record Path

Run from microcosm-substrate:

PYTHONPATH=src ../repo-python -m microcosm_core.organs.prediction_oracle_reconciliation run \
  --input fixtures/first_wave/prediction_oracle_reconciliation/input \
  --out /tmp/microcosm-prediction-oracle-reconciliation/fixture \
  --card
PYTHONPATH=src ../repo-python -m microcosm_core.organs.prediction_oracle_reconciliation run-prediction-bundle \
  --input examples/prediction_oracle_reconciliation/exported_prediction_oracle_bundle \
  --out /tmp/microcosm-prediction-oracle-reconciliation/bundle \
  --card
PYTHONPATH=src ../repo-python -m pytest -p no:cacheprovider tests/test_prediction_oracle_reconciliation.py -q
PYTHONPATH=src ../repo-python scripts/build_doctrine_projection.py --check-paper-module-corpus

A passing run proves only synthetic target-universe reconciliation, CP1/CP2 accounting, oracle-diff grading, and fixture-bounded dossier mutation; it does not establish forecasting performance, financial decisions, trading authority, live market access, public sharing, or launch.

finance_forecast_evaluation_spine is a Crown Jewel import component with real runnable system and a strict public scope limit. It consumes synthetic public fixtures, copied source source bodies, and source manifests that verify sha256 digests, line counts, required anchors, secret-exclusion status, and result record body omission.

Purpose

Comparing two forecasting models is harder than it looks. A lower average loss does not establish that one model genuinely predicts better, because losses are autocorrelated, samples are short, and a careless split can let a model peek at the answer. This component exists to carry the statistical machinery that economists use to answer that question carefully, and to do so without ever claiming the machinery has been pointed at a real market.

The single question it answers is narrow: given two paired loss series over a synthetic fixture, can the difference in predictive accuracy be called significant under an admissible test, or must the test refuse? It computes the Diebold-Mariano loss-differential statistic with a Bartlett HAC long-run variance, the Harvey-Leybourne-Newbold small-sample correction, Hansen's test for superior predictive ability with recentering, a model confidence set, and a Politis-Romano stationary bootstrap.

Failure is handled explicitly. The Harvey-Leybourne-Newbold correction returns its computed statistic, but when SciPy is absent it refuses the p-value with a typed reason rather than fabricating one. The same discipline rejects a horizon that reaches the sample length, a sample too small to estimate anything, a time split that lets the evaluation date sit at or after the event window, and any policy flag that smuggles in advice or a track-record claim. A refusal is recorded as a first-class validator outcome, not an error: "we declined to answer" is itself a valid result.

The guards run before the statistics. If a boundary policy or a leakage check fails, the result record is blocked before any statistics subprocess starts, so an inadmissible request never produces a number that could be misread as a result.

What it proves: synthetic fixture forecast-evaluation statistics only; no investment-related actions, live market data, track record, or performance claim.

How to run it:

microcosm finance-forecast-evaluation-spine run --input fixtures/first_wave/finance_forecast_evaluation_spine/input --out receipts/first_wave/finance_forecast_evaluation_spine

Runtime bundle route:

python -m microcosm_core.organs.finance_forecast_evaluation_spine run-finance-forecast-bundle --input examples/finance_forecast_evaluation_spine/exported_finance_eval_bundle --out receipts/runtime_shell/demo_project/organs/finance_forecast_evaluation_spine

Negative cases covered by the fixture manifest: finance_hln_dependency_refusal, finance_leakage_lookahead_split, finance_no_advice_overclaim.

Source provenance is anchored by examples/finance_forecast_evaluation_spine/exported_finance_eval_bundle/source_module_manifest.json and result records carry refs, digests, counts, verdicts, and scope boundaries only.

Shape

flowchart TD Fixture["Synthetic fixture inputs family_loss_matrix, paired_loss_series, finance_boundary_policy, projection_protocol"] Source["Copied finance modules plus source manifest digests"] Runner["finance_forecast_evaluation_spine.run"] Guards["Guards run first policy no-advice flags, lookahead-split leakage check"] Blocked["Blocked result record statistics subprocess never starts"] Branch{"Admissible and exported bundle?"} Subprocess["Statistics subprocess DM/HAC, Hansen SPA, MCS, stationary bootstrap, HLN refusal"] Standalone["Standalone statistics contract no live source-root subprocess"] Result record["Result records refs, hashes, counts, verdicts, scope boundaries; body_in_receipt false"] Fixture --> Runner Source --> Runner Runner --> Guards Guards -->|"boundary fails"| Blocked Guards -->|"boundary passes"| Branch Branch -->|"first-wave fixture"| Subprocess Branch -->|"exported bundle"| Standalone Subprocess --> Result record Standalone --> Result record Blocked --> Result record

Technical Mechanism

The module is a deterministic forecast-evaluation harness around CrownJewelSpec, not a finance product. The spec fixes four required fixture inputs (family_loss_matrix.json, paired_loss_series.json, finance_boundary_policy.json, and projection_protocol.json), names the three required negative cases, binds the source manifest, and restricts the source-open import to required anchors in model_selection_stats.py, spa_statistics.py, loss_differentials.py, and family_loss_matrix.py.

At runtime, run delegates to run_crown_jewel_organ with evaluate and evaluate_negative_case. evaluate loads the synthetic loss matrix, paired loss series, and boundary policy, then calls _evaluate_payloads. That function first enforces the policy and lookahead-split guards; if either boundary fails, it returns a blocked result record before any statistics subprocess can run. Only after those guards pass does it run the copied statistics modules or, for the exported bundle path, use _standalone_exported_statistics_contract so the standalone public bundle does not depend on a live source-root subprocess.

The statistical witness is therefore deliberately narrow: Reality Check, Hansen-SPA, MCS, Diebold-Mariano/HAC, stationary bootstrap, and the HLN refusal are result record fields over the synthetic fixture. The same mechanism treats finance_hln_dependency_refusal as a typed negative case when SciPy support is absent, treats policy overclaims as FINANCE_NO_ADVICE_OVERCLAIM, treats temporal leakage as FINANCE_LOOKAHEAD_SPLIT_FORBIDDEN, and keeps copied source bodies out of result records with body_in_receipt: false.

Reader Evidence Routing

Read the positive fixture as a small statistical witness, not as a market result. The current result record has status: pass, sample_size: 40, candidate_count: 3, reality_check.status: computed_bootstrap, spa.status: computed_bootstrap, mcs.implemented: true, paired_loss.diebold_mariano.status: computed_hac_normal_approximation, and a five-replicate stationary-bootstrap witness. Those fields show that the component can exercise the copied forecast evaluation code paths on public synthetic data.

Read the negative floor as equal evidence. The observed negative cases are finance_hln_dependency_refusal, finance_leakage_lookahead_split, and finance_no_advice_overclaim, with stable error codes FINANCE_HLN_TYPED_REFUSAL_REQUIRED, FINANCE_LOOKAHEAD_SPLIT_FORBIDDEN, and FINANCE_NO_ADVICE_OVERCLAIM. The HLN case refuses because SciPy is unavailable for the t-distribution; that is the intended scope limit, not a missing p-value to fill in by hand.

Read source-open evidence through the manifest, not through result records. The source bundle carries 13 copied finance modules; result records carry references, hashes, counts, verdicts, and scope boundaries, and keep body_in_receipt: false. The local claim therefore stays at "synthetic fixture forecast-evaluation statistics and typed refusals." It does not become investment-related actions, live-market data, a track record, performance proof, optimizer authorization, or launch-scope decision.

Forecast-Evaluation Discipline

This component is evidence that the Microcosm can carry professional forecast evaluation logic without pretending to carry market authority. The admissible statistics include Diebold-Mariano loss-differential testing, the Harvey-Leybourne-Newbold small-sample correction, Hansen's SPA test, a Politis-Romano stationary bootstrap, Bartlett HAC long-run variance, and purged/embargoed cross-validation in the Lopez de Prado style.

The important doctrine is refusal discipline. Horizons greater than or equal to sample length, samples too small to estimate a statistic, leakage-prone splits, missing SciPy support, and advice-shaped claims must return typed refusals instead of crashes or meaningless numbers. Hansen-style recentering of poor or irrelevant alternatives is part of the SPA contract because it is the boundary between a useful superior-predictive-ability test and White Reality Check style over-penalization.

Result records should therefore distinguish "computed statistic" from "refused because inadmissible." Both are successful validator outcomes when the fixture asked for that behavior.

Named Proof Consumers

• Runtime fixture consumer: finance_forecast_evaluation_spine.run over fixtures/first_wave/finance_forecast_evaluation_spine/input must produce status: pass, the three observed semantic negative cases, false advice/live-data/performance authority flags, and metadata-only source-manifest result record material.
• Exported-bundle consumer: run-finance-forecast-bundle over examples/finance_forecast_evaluation_spine/exported_finance_eval_bundle must validate the 13 copied finance modules by digest and use the standalone statistics contract rather than a live source subprocess.
• Focused pytest consumer: tests/test_finance_forecast_evaluation_spine.py must keep the positive statistical fixture, no-advice overclaim, live-market overclaim, lookahead split, semantic-negative-case, standalone-bundle, and digest-mismatch tests green.
• Corpus consumer: scripts/build_doctrine_projection.py --check-paper-module-corpus must keep the 98-module Microcosm paper-module corpus valid without hand-editing the generated JSON instance.
• Scope limit consumer: any public or dissemination copy must preserve the local ceiling that this is synthetic fixture forecast-evaluation evidence, not investment-related actions, live data, performance proof, optimizer authorization, or launch-scope decision.

Prior Art Grounding

This component is grounded in forecast-evaluation statistics rather than trading systems. The core anchors are the Diebold-Mariano test for comparing predictive accuracy, the Harvey-Leybourne-Newbold small-sample correction for prediction-error tests (DOI reference), Hansen's test for superior predictive ability, and proper-scoring-rule work such as Gneiting and Raftery. The purged/embargoed split discipline also follows the financial ML concern that temporal leakage can make backtests look stronger than they are.

Microcosm borrows the professional evaluation posture: compute admissible statistics when the fixture supports them, return typed refusals when it does not, and keep evaluation separate from advice, live market data, or performance claims.

Validation Result record Path

PYTHONPATH=src ./repo-pytest tests/test_finance_forecast_evaluation_spine.py -q --basetemp=/tmp/microcosm_finance_forecast_evaluation_spine_pytest
./repo-python scripts/build_doctrine_projection.py --check-paper-module-corpus

Purpose

The underlying source module compiles a generated market-situation graph into a backend read model: a trust strip, a ranked situation queue, a detail index, a graph slice, facets, drilldowns, and an API contract. The read model is the shape a dashboard consumes. It runs the copied read-model helpers over small synthetic fixtures and asks one question: does the read-model layer hold its own claim boundary, or does it quietly become a market-truth or advice surface?

The interesting part is what the validator refuses rather than what it accepts. A presentation layer is the easy place for an overclaim to leak in: a label like "strong buy", an auto_apply_allowed flag left true, a freshness state that reports green from a stale or missing artifact. The copied validate_market_dashboard_read_model scans for trading and action-claim language, requires oracle_evolve.auto_apply_allowed to be false and review_gated to be true, requires no_advice_mode to be enabled, and requires the silent-omission count to be zero. The bundle drives those checks with fixtures designed to trip each one, then records whether the source actually flagged them.

The other two mechanisms guard the read path itself. A feed-freshness overlay classifies the current run into a small set of honest states so historical green proof cannot stand in for live-feed capability, and a related-situations scorer groups situations by shared entities or matching type without inventing links. Everything is fixture-bound: there is no live market data, no external model access, and no investment-related actions anywhere in scope.

Mechanisms

• validate_market_dashboard_read_model
• _runtime_feed_freshness_overlay
• _related_situations

What the checks do

validate_market_dashboard_read_model is the structural and overclaim gate. It first checks the read model is well formed: the schema version matches, every situation in the queue resolves to a detail entry, every graph-slice edge points at a node that exists, and each drilldown source-ref returns metadata only with no arbitrary file read and no .. traversal in its route. It then enforces the claim boundary. auto_apply_allowed must be false, review_gated must be true, no_advice_mode must be enabled, the silent-omission count must be zero, and any copied source text is scanned for trading or action-claim language (buy, sell, short, price target, stop loss, and similar). The bundle feeds it five negative fixtures, one per failure shape, and confirms the source emits the matching error string for each. A read model that passed these checks but stayed silent on a planted overclaim would be the real failure, so the bundle treats a missing error as a finding.

_runtime_feed_freshness_overlay reads a per-run readiness summary and reports one of three honest states. fresh_green_feed requires the run to be ready, all targets met, no blockers, and same-day generation. stale_green_feed is artifact-backed but no longer same-day. blocked_missing_artifact covers the run that is missing its readiness file, falls short on targets, or carries blockers. The point is that a stale or absent run never reports green: historical proof cannot stand in for live-feed capability, and the state carries a plain truth-statement saying so. The bundle writes synthetic readiness files for each case and checks the classifier returns the expected state.

_related_situations builds the "see also" cohort for a situation. It collects other situations that either share an entity or match the situation type, ranks them, excludes the focus situation itself, and caps the list at six. The bundle checks one boundary case in particular: a situation with no entity overlap and a different type produces an empty cohort rather than a spurious link.

Shape

flowchart TD A["Synthetic dashboard, freshness, related fixtures"] --> B["Copied read-model helpers (market_dashboard_read_model.py)"] B --> C["validate_market_dashboard_read_model"] C --> C1["Structure: schema, queue-to-detail, graph edges, drilldown route safety"] C --> C2["Scope limit: no auto-apply, review-gated, no-advice, no trading language, zero silent omissions"] B --> D["_runtime_feed_freshness_overlay"] D --> D1["fresh_green_feed"] D --> D2["stale_green_feed"] D --> D3["blocked_missing_artifact"] B --> E["_related_situations"] E --> E1["Entity overlap or type match; self-excluded, capped at six; no overlap means empty"] C1 --> F["metadata-only result record and card (refs, digests, counts, verdicts)"] C2 --> F D1 --> F D2 --> F D3 --> F E1 --> F

Reader Evidence Routing

Start with paper_modules/batch12_market_dashboard_read_model_capsule.json for bundle-derived source authority, then read this Markdown as the explanatory projection. Use examples/batch12_market_dashboard_read_model_capsule/exported_batch12_market_dashboard_read_model_capsule_bundle/source_module_manifest.json to inspect copied-source digest status before opening copied source modules. Use tests/test_batch12_market_dashboard_read_model_capsule.py to verify the fixture and bundle expectations.

The useful evidence is dashboard read-model accounting over synthetic public fixtures: validation rows, freshness overlays, related-situation joins, negative cases, metadata-only result records, and scope limit fields.

Prior Art Grounding

The component is grounded in CQRS/read-model and dashboard-observability patterns: derive presentation-ready projections from source data, make freshness visible, and keep the read surface separate from mutation authority. Useful anchors include:

• Microsoft's CQRS pattern, where read models are optimized for queries and presentation rather than command handling.
• Grafana dashboards, which query and transform data sources into operational panels.

Microcosm borrows the read-model shape for dashboard validation, runtime feed freshness overlays, and related-situation joins. The result is fixture-bound mechanism evidence; it does not become market-level conclusions, external model access, investment-related actions, or launch-scope decision.

Validation Result record Path

Reader-verifiable commands, run from the microcosm-substrate/ public root:

The fixture command writes the dashboard read-model result record and sign-off JSON. The bundle command validates copied source system, manifest digests, freshness overlay rows, related-situation joins, negative cases, and metadata-only result record posture. The focused test checks fixture validation, bundle validation, digest/anchor coverage, and scope limits.

This result record path is reader-verifiable evidence only. It excludes launch, external model access, private-system equivalence, market-level conclusions, investment-related actions, or whole-system correctness.

Purpose

Market and source dashboards have a recurring failure: a row looks like a fact when it is really a guess. A duplicate listing inflates a volume figure, an unmatched market slug grows a fabricated identity, a feed reports zero rows but the board shows it as healthy, and a "change since last time" number appears even when there is no prior baseline to compare against. The single question this component answers is whether the copied presentation-mart logic keeps those distinctions honest when run over public synthetic inputs.

It does that by importing the real quant_presentation_mart helper body and running it against fixtures that are built to expose each trap, then asserting the exact diagnostic the body should produce. The interesting choice is that the board never asserts what a market price means. It computes accounting about the data: which event a market belongs to, whether its identity was actually matched, how providers drifted, where rows went missing, and whether a vintage date is genuinely present. Aggregation is deliberately conservative. A missing value stays missing rather than defaulting to a confident zero, and an unmatched slug is reported as missing_from_feed_artifact instead of being given a synthetic event id.

The result is fixture-bound evidence, not a forecast. The board is a diagnostic surface over public synthetic rows. It does not read live markets, use external model services, or claim that any number is tradeable.

Mechanisms

• _prediction_market_board
• _polymarket_identity_by_slug
• _provider_drift_monitor
• _missingness_board
• _delta_since_previous_green
• _macro_lifecycle_by_slug
• _macro_regime_board

How it works

The bundle loads three fixtures, runs the copied helpers, and checks eight named invariants. Each check targets a specific way a board can quietly mislead.

The event-join engine (_prediction_market_board with _polymarket_identity_by_slug) groups raw market rows into events using the Polymarket identity snapshot. Identity is matched by market_slug. When two rows share the same slug and outcome, only the higher-volume one is kept, so a duplicate listing cannot double a market count or inflate an aggregate. A slug with no identity match is not dropped and is not given a made-up event id. Its event_identity_status becomes missing_from_feed_artifact and its max_liquidity stays at 0.0. The fixture proves all three: the duplicate fold (top volume 900000 with one surviving market), the orphan with a null event id, and the deduped aggregate.

The provider-drift monitor (_provider_drift_monitor) reads each feed's diagnostics and raises typed flags rather than a single health score. Generic transport problems (provider_fallback_used, html_response_seen, fetch_failures) are kept distinct from FRED-specific ones (fred_invalid_series, fred_network_warning). The fixture checks that the stock feed surfaces the generic set, the news feed stays clean, and the source feed surfaces the FRED set. Keeping the families apart means a source data-source fault is not laundered into a generic warning.

The missingness board (_missingness_board) lists only feeds that are not both non-empty and ok. A feed with zero rows is labelled zero_rows; a populated but low-quality feed is labelled quality_degraded; a healthy feed is omitted entirely. The fixture confirms the healthy feed is absent and the two failing lanes carry the correct reason, so an empty feed cannot read as present.

The prior-green delta (_delta_since_previous_green) only computes a "change since last run" when a previous green run actually exists. With no baseline it returns status: unavailable and an empty row_deltas_by_lane, which the fixture asserts directly. This is the guard against a delta number that has nothing to compare against.

The source lifecycle enrichment (_macro_lifecycle_by_slug feeding _macro_regime_board) buckets source series, then binds each bucket's vintage_status and release_calendar_status to whether the lifecycle structured source record genuinely carries that metadata. The fixture proves a series with a present vintage reads available with the expected observation date, while a series whose lifecycle row is absent reads missing_from_feed_artifact. A vintage date is shown only when it is really there.

Shape

flowchart TD Rows["Synthetic market rows"] --> Join["Event join + identity match _prediction_market_board"] Identity["Polymarket identity snapshot"] --> Join Helpers["Quant-mart helper fixtures"] --> Drift["Provider drift monitor generic vs FRED flags"] Helpers --> Miss["Missingness board zero_rows vs quality_degraded"] Helpers --> Delta["Prior-green delta unavailable with no baseline"] Helpers --> Source["Source regime board vintage status bound to structured source record"] Join --> Dedup{"Slug + outcome seen before?"} Dedup -->|yes| Keep["Keep higher-volume market"] Dedup -->|no, unmatched| Orphan["missing_from_feed_artifact no fabricated event id"] Dedup -->|no, matched| Append["Append to event aggregate"] Keep --> Result record["metadata-only result record and card diagnostic rows, negative cases, scope limit"] Orphan --> Result record Append --> Result record Drift --> Result record Miss --> Result record Delta --> Result record Source --> Result record

Reader Evidence Routing

Start with paper_modules/batch12_prediction_market_board_capsule.json for bundle-derived source authority, then read this Markdown as the explanatory projection. Use examples/batch12_prediction_market_board_capsule/exported_batch12_prediction_market_board_capsule_bundle/source_module_manifest.json to inspect copied-source digest status before opening copied source modules. Use tests/test_batch12_prediction_market_board_capsule.py to verify the fixture and bundle expectations.

The useful evidence is diagnostic accounting over synthetic public fixtures: provider identity matching, drift rows, missingness boards, prior-green deltas, lifecycle/vintage rows, source-regime enrichment, negative cases, metadata-only result records, and scope limit fields.

Prior Art Grounding

The component borrows from prediction-market information aggregation and public market-data integration practice: event contracts expose market prices and settlement states, while dashboards must keep provider identity, missingness, and vintage drift visible. Relevant anchors include:

• Robin Hanson's information markets framing, where markets are used to aggregate dispersed information about uncertain events.
• The CFTC's prediction markets explainer, which frames event contracts, market prices, and consumer cautions.
• Polymarket API documentation, as a concrete public API family for market, event, tag, series, and profile data.

Microcosm borrows the information-aggregation and provider-join shape, then keeps the board explicitly diagnostic: identity matching, provider drift, missingness, prior-green deltas, lifecycle vintage, and source-regime enrichment are tested over public synthetic fixtures. It is not market-level conclusions, provider truth, investment-related actions, or launch-scope decision.

Validation Result record Path

Reader-verifiable commands, run from the microcosm-substrate/ public root:

The fixture command writes the prediction-market board result record and sign-off JSON. The bundle command validates copied source system, manifest digests, provider identity and drift diagnostics, missingness rows, lifecycle rows, negative cases, and metadata-only result record posture. The focused test checks fixture validation, bundle validation, digest/anchor coverage, and scope limits.

This result record path is reader-verifiable evidence only. It excludes launch, external model access, private-system equivalence, market-level conclusions, provider truth, investment-related actions, or whole-system correctness.