# Intransic Validation Mechanical checks that the Foundation, evidence ledger and Trust Wall remain consistent with each other and with the repository. --- ## What these checks can and cannot establish **They CAN establish:** - Nothing is published that the evidence ledger does not support. - Every claim resolves to a ledger entry, and every cited file exists. - That no claim is marked `verified` on existence-only evidence. - No contradicted, planned or unknown claim has leaked into public assets. - No secret-shaped values, internal hostnames or restricted promotional terms appear in public files. - That the specific structural properties you declared as `R-*` rules still hold. **They CANNOT establish:** - That the system is secure, correct, private or compliant. - That a cited file actually contains what its explanation claims. - That a cited test passes, or asserts what its explanation says. `E-11` reads the ledger, not your test runner — it checks that behavioural evidence was *cited*, not that it was *green*. - That deployed configuration matches committed configuration. - That a secret has not leaked — the config scan reads the working tree only. **A secret removed from `HEAD` remains in git history and this check will not see it.** A fully passing run means *"nothing is claimed that the evidence does not support"* — nothing more. Do not treat a green run as an assurance result. --- ## Setup Python 3.9+ with two dev-only packages. These are **not** application dependencies and must never be added to the application's own manifest. ```bash python3 -m venv .venv . .venv/bin/activate pip install -r intransic/validation/requirements.txt ``` --- ## Running ```bash python3 intransic/validation/validate_evidence.py # ledger consistency (E-01..E-12) python3 intransic/validation/validate_foundation.py # foundation integrity (F-01..F-11) python3 intransic/validation/validate_trust_wall.py # publication safety + conformance (T-*, R-*) ``` Run all three: ```bash for v in evidence foundation trust_wall; do python3 intransic/validation/validate_$v.py || echo "FAILED: $v" done ``` **Exit codes:** `0` no errors · `1` errors found · `2` missing dependency or required file. Run order matters when diagnosing: the ledger is the source of truth, so fix `validate_evidence` findings first — Foundation and Trust Wall errors are often downstream of a ledger problem. The validators locate the repository root by walking up from their own location to find `intransic/validation/validation-rules.yaml`. Set `INTRANSIC_ROOT` to override, which is occasionally needed in CI images that stage the tree elsewhere. --- ## Severities | Severity | Meaning | |---|---| | **error** | Blocks publication or merge. Fix before proceeding. | | **warning** | Requires human judgement. Staleness and violated invariants surface here. | | **info** | Advisory (e.g. a line-number citation without a symbol). | --- ## The rules Defined in [`validation-rules.yaml`](validation-rules.yaml), grouped as: - **F-01..F-09** — Foundation integrity: sections present, statuses valid, references resolve, every ledger ID classified exactly once, version agreement. - **F-10..F-11** — Data governance: classification field paths exist, processor entries are complete and evidenced. See [Field-level PII and the processor register](#field-level-pii-and-the-processor-register). - **E-01..E-10** — Evidence integrity: unique IDs, verified claims carry evidence, cited paths exist, contradicted claims are never publishable, staleness. Note that `E-03` asks only whether evidence is *present*; `E-11` asks whether it is *strong enough*. - **E-11..E-12** — Evidence strength: `verified` needs behavioural evidence or stronger; documentation and git are never sole support. See [Evidence strength](#evidence-strength). - **T-01..T-14** — Trust Wall integrity: claims map to evidence, published wording matches the ledger exactly, no non-publishable statuses, no secret or internal leakage, no restricted terms, limitations page present. - **R-\*** — Repository conformance: structural properties of your codebase. **These are yours to write** — the rules file ships with worked examples for a .NET web API and an empty list. See the block comment above `repository_conformance:` for the four available rule kinds. F, E and T are stack-independent and should be carried across repositories unchanged. --- ## Why `T-04` exists Public wording lives in the ledger beside its evidence, and `trust/claims.json` is **generated** from it. `T-04` fails if published text no longer matches its approved wording — which is what happens when someone edits the Trust Wall by hand. Regenerate rather than edit. ## Evidence strength `E-03` asks whether a `verified` claim carries evidence *at all*. That was never enough on its own: it counts items without asking what any of them establishes, so a single citation of a file that merely exists satisfied it. The ladder, weakest first, with the default applied when an item does not declare `strength:`: | Strength | Establishes | Default for | |---|---|---| | `existence` | a file, symbol or interface is present | `documentation`, `git`, `schema` | | `implementation` | an active code path, or committed configuration | `implementation`, `configuration`, `infrastructure` | | `behavioural` | an executed test asserts the behaviour | `test` | | `operational` | the running system was directly observed | `observation` | | `deployed` | production configuration or telemetry | — | | `independent` | external audit or certification | — | **`E-11`** — `verified` requires `behavioural` or better. Below that the supportable status is `implemented_unverified`, which the vocabulary already provides for exactly this case. **`E-12`** — no publishable status (`verified`, `implemented_unverified`, `partial`) may rest *entirely* on `documentation` or `git` evidence. Documentation is a claim to be verified; a commit shows a change was made, not that it survived. Both also appear as conditional subschemas in `evidence.schema.json`, so a ledger fails `E-01` on the same grounds even where these checks are ported to another language. ### Declaring a weaker strength The default maps from the evidence *type*, and a type cannot distinguish a live code path from a dormant one. Where the citation is weaker than its type implies, say so: ```yaml - type: implementation path: src/Ports/IEvidenceStore.cs symbol: IEvidenceStore strength: existence # interface only; nothing binds an implementation explanation: … ``` There is no mechanism preventing an assessor declaring a strength the citation does not deserve — the ledger is written by hand, and this check reads the ledger. What it removes is the *silent* case: `verified` can no longer be reached by accident, only by an assertion someone made on the record. ## Why the `R-*` rules exist at all The first repository this ran against had a verified authorisation control silently reverted by a merge. No test asserted the property, so nothing failed, and the control was gone for a day before anyone noticed. An `R-*` rule was written to detect that exact shape. It is a backstop, not a substitute: **the durable fix is a regression test asserting the behaviour.** A static check can spot a known-bad shape; only a test can prove a behaviour. If you find yourself writing an `R-*` rule where a test would do, write the test. --- ## Field-level PII and the processor register Two ways an evidence-backed Foundation can still drift specifically around personal data, and the two checks that close them: **A classification section that lists categories but not fields.** *"Personal identifiers: name, email"* describes a shape, not the actual columns. A new field can land — a government health identifier, a date of birth — and nothing fails, because nothing was checking. `data.classifications[].fields` (see `foundation.schema.json`) traces each class to concrete `path` + `symbol` evidence, the same way every other Foundation claim is traced. `F-10` checks those paths resolve. The `pii_coverage` conformance kind goes the other direction: it scans your source for field names that *look* like PII and warns when one isn't named anywhere in the data section — a check the Foundation cannot run on itself, since it only knows what it was told. Both directions are needed. `F-10` catches a stale reference in the Foundation; `pii_coverage` catches a Foundation that was never updated for a change to the code. Neither proves correctness — `pii_coverage` is a name match, and a badly named field defeats it as easily as a well-named one satisfies it. Treat a hit as "go classify or exclude this", not as a verdict. **An outbound data flow to a third party with no record of it.** `privacy.processors` is a register: name, category, what data leaves, why the sharing is appropriate (`basis`), and evidence. `F-11` requires every entry to be complete and its evidence to resolve — the same discipline applied to every other claim. This is Foundation-level bookkeeping, not an audit: it does not by itself demonstrate a lawful basis exists, only that one has been stated and can be checked against its evidence. Cloud hosting, payment processing, an LLM provider and an identity provider are all processors; list them even though only some read as "integrations" in the architectural sense. --- ## Infrastructure the Foundation never mentions Every other rule here works by falsification: a claim is made, and evidence either carries it or contradicts it. That leaves one thing structurally invisible. A Foundation cannot contradict a claim nobody made, so a network posture that was never written down passes every check by never being asserted. Silence is not a status in the vocabulary, and it is the state most security findings actually live in. `infra_posture` inverts the direction for that case. It reads a captured infrastructure snapshot — the committed output of `az`, `gcloud`, `kubectl`, `terraform show -json` or equivalent — and for each posture found in it, asks whether the Foundation's `boundaries` or `security` sections mention it at all. An undeclared posture becomes a finding. A declared one does not, whatever the Foundation says about it: "we accept this until Phase 2" is an account, and accounting for a posture is all this rule asks for. It reads a snapshot rather than calling a cloud CLI on purpose. A validator that shells out needs credentials, a network and a particular cloud, and stops being reproducible — the same run against the same commit could disagree with itself. The snapshot is evidence: captured once, committed, reviewable in a diff, and subject to the same staleness questions as any other evidence. What a finding establishes is narrow. The snapshot contains something; the Foundation does not mention it. Not that the posture is wrong — a database open to a cloud's whole service range can be a considered decision — and not that a mentioned posture is adequately controlled. It says nobody wrote it down. Refresh the snapshot on the same cadence as the Foundation. A stale one reports the posture of a system that no longer exists, which is worse than no rule at all, because it reads as a pass. ## A rule whose target is missing is not a pass If a rule's `files:` globs match nothing, the validator emits a **warning saying the check was skipped**, not silence. A renamed entry point should not quietly turn a security check into a no-op. ## Adding a rule 1. Add it to `validation-rules.yaml` with an ID, severity, and an explicit `establishes:` statement saying what it does *not* prove. 2. Pick a `kind:` — `text`, `config_secrets`, `mock_binding`, `pii_coverage` or `infra_posture`. Most rules are `text`. 3. Prefer warnings for anything needing human judgement — a noisy error trains people to ignore the tool. You should not need to edit Python to add a rule. If you do, the kind you need is probably worth adding to the skill rather than to one repository.