V3 Journey Release Gate and Evidence Packet¶

OpenMed v3 release readiness is a signed, fail-closed decision. It is not inferred from issue closure, a single aggregate score, or a successful package build. The gate binds a tagged commit to frozen inputs and requires current, typed evidence across behavior, clinical NLP, privacy, interoperability, performance, recovery, security, and licensing.

This is technical release evidence. It is not clinical validation, certification, a medical-device determination, or permission for autonomous clinical action.

Public contracts¶

The release surface has two strict JSON contracts:

• journey_release_manifest.schema.json describes the operator-supplied, aggregate evidence manifest.
• journey_release_packet.schema.json describes the signed decision packet.

New reports and packets use schema version 1.1.0 and compatibility policy same_major. Version 1.0.0 manifests, reports and signed packets remain readable. Re-evaluating a legacy privacy report yields NOT_READY with explicit missing:* consistency blockers; missing measurements are never filled with zeros. Signature verification authenticates a historical packet and does not re-evaluate it against the newer release criteria. Unknown future minor or major versions require a deliberate reader update. Unknown fields, duplicate JSON keys, non-finite numbers, unbounded free text, absolute paths, parent traversal, symlinked evidence, and incomplete gate metrics are rejected.

The packet retains all seven non-empty result states:

success, partial, unknown, conflict, unsupported, denied, and failure.

Only success can satisfy a mandatory release lane. A different state remains visible in the packet and makes the release NOT_READY; it is never converted to an apparent success.

Mandatory evidence lanes¶

All ten lanes must be present exactly once. Each report records its generation time. The manifest policy sets a bounded maximum artifact age, up to one year; future-dated evidence is also rejected.

Patient-keyed privacy consistency¶

openmed.eval.golden_journey.verify_journey_privacy_consistency accepts exactly one protected text surface for each format, in this order: text, fhir_r4, hl7v2, csv, dicom_sr. For DICOM SR this is already extracted document text. It does not cover binary headers, images, pixels, or burned-in text. verify_golden_journey_privacy projects the supplied frozen scenario onto these surfaces. The verifier does not add dates or identifiers.

The verifier runs caller-supplied, trusted local processors. Each receives one JourneyPrivacySource and the same JourneyPrivacyContext: a confirmed patient key, the existing date-shift secret and bound, and a fresh memory-only SurrogateVault. A processor returns JourneyPrivacyTransformation containing actual transformed text, an explicit patient_keyed assertion, and half-open JourneyPrivacySpan offsets into both original and transformed text. Identifier witnesses must be confirmed patient aliases with PERSON or ID_NUM labels; bind them using context.vault.resolve_subject(context.patient_key, aliases=...). This boundary performs no patient matching or PII detection. Processors remain responsible for format parsing, detection, complete redaction and private logs; injection is not isolation of untrusted code. The verifier takes a bounded snapshot of at most five unique processor bindings and drops upstream exception contexts at both public privacy entry points.

The verifier reads the witnessed surfaces itself. It checks every observed calendar-day shift against stable_offset_for, checks preserved intervals, and compares emitted aliases with the one subject surrogate established before processing. It accepts complete ASCII YYYY-MM-DD and YYYYMMDD dates only; partial dates, timestamps and unrecognized temporal surfaces are unsupported. It does not change shift algorithms, surrogate generation or supplied sources.

Absent processors, an explicit lack of patient-keyed support, missing witnesses, or unsupported date precision receive unsupported, with no consistency credit. Malformed offsets, overlapping witnesses, invalid output or processor errors receive failure; source-bearing exception messages are discarded. An inconsistent format receives failure and nonzero mismatch counts, even if a caller labels the surrounding lane success.

The candidate's five-source synthetic fixture carries the same two calendar dates and patient alias in each payload. With the annotated fixture processors, the positive verifies all five formats; a different CSV offset blocks the signed privacy gate. The golden output is regenerated from the executed Journey runner and its source digests bind the updated fixture. These are synthetic witness tests, not native FHIR, HL7 or DICOM conformance certification.

The historical fixture remains available at the immutable v3.0.0 tag. Its missing witnesses remain unsupported under the current verifier. Missing processors also yield five unsupported formats, even for the witnessed fixture. Zero observed inconsistencies alone never establish coverage. In particular, the existing FHIR resource de-identification helper leaves temporal fields unchanged; wrapping it does not prove patient-keyed date coverage.

Keep date-shift and evidence-proof keys private and separate (at least 32 bytes each). Protected text, patient keys and witnesses remain in memory. Their representations omit source and replacement surfaces. result.to_dict() contains controlled format/state names, counts and domain-separated HMAC-SHA256 digests. Keyed proofs protect dates, small intervals and surrogate sets against the simple dictionary attacks possible with unkeyed hashes. Digests are comparable only within the same private key and patient scope; they prove witness consistency, not completeness of redaction or clinical validity.

Merge result.lane_metrics() into a version 1.1.0 privacy report and retain result.state. The four additional required metrics are:

• date_shift_inconsistency_count: formats with a wrong shift or interval;
• surrogate_inconsistency_count: formats with an inconsistent emitted alias;
• consistency_checked_format_count: formats with complete valid witnesses;
• consistency_unsupported_format_count: formats without usable proof, including failed processors.

Both inconsistency counts and unsupported coverage are zero-blockers. Exactly five checked formats are required. Keep the separate critical-leakage, raw-value and evaluated-case metrics; this verifier does not supply them. Freeze the counts-and-digests consistency report as an aggregate input to bind it into the signed release packet.

Performance evidence¶

A performance report is incomplete unless it states all of the following:

• normalized hardware identifier and runtime environment;
• dataset SHA-256 rather than source records;
• quantization format;
• concurrency, sample count, and warm-up count;
• p50, p95, and p99 latency with p50 <= p95 <= p99;
• throughput per second;
• peak memory in MiB;
• SLO-breach count; and
• artifact age computed against the manifest's pinned evaluation time.

The environment and measurement fields identify the benchmark conditions. They do not imply performance on another device, dataset, concurrency level, or model format.

Frozen inputs and reproducibility¶

Every input path must be relative to the checked-out repository. The gate rejects paths that escape the root or traverse a symlink. It recomputes each file digest and copies only the following into the output packet:

• a bounded input identifier;
• provenance class (synthetic, aggregate, configuration, or software);
• SHA-256 digest;
• byte count; and
• verification result.

The packet never copies the file path or the file contents. At least one input must assert synthetic provenance. A typical v3 packet binds the five-source scenario and its frozen golden result as synthetic inputs.

The release version must be semantic version text, and git_tag must equal v plus that version. The gate verifies both the current checkout and the tag target against the full 40-character commit in the manifest. A missing tag, an untagged checkout, or a mismatched tag yields NOT_READY.

The manifest also pins evaluated_at and the identical source_date_epoch. Given the same tagged checkout, manifest, frozen inputs, and signing key, the canonical packet digest and signature are reproducible.

Signing and verification¶

The packet uses HMAC-SHA256 and records a bounded key identifier, never the key. Supply at least 32 bytes through one of these boundaries:

export OPENMED_JOURNEY_RELEASE_KEY="$(security find-generic-password \
  -s openmed-journey-release -w)"

python scripts/release/journey_release_gate.py \
  --manifest journey-release-manifest.json \
  --output journey-release-packet.json

For offline CI, use a permission-restricted key file:

python scripts/release/journey_release_gate.py \
  --manifest journey-release-manifest.json \
  --signing-key-file /run/secrets/openmed-journey-release-key \
  --key-id production-release-2026 \
  --output journey-release-packet.json

The command refuses to replace an existing packet unless --overwrite is explicit. It writes through a temporary file, flushes it, and atomically replaces the destination. Exit status is:

• 0: a signed READY packet was written;
• 1: a signed NOT_READY packet was written; or
• 2: the manifest, key, repository, or output contract was invalid, so no release decision could be produced safely.

Independent verification uses the public Python contract:

import json
from pathlib import Path

from openmed.eval.journey_release import JourneyReleasePacket

payload = json.loads(Path("journey-release-packet.json").read_text())
packet = JourneyReleasePacket.from_dict(payload)
assert packet.verify(signing_key)
assert packet.decision == "READY"

License and distribution gate¶

The manifest catalogs every model, dataset, vocabulary, and software asset using code-like metadata only: asset ID, type, license ID, role, redistributability, and distribution mode. The catalog itself is hashed into the packet.

Non-redistributable assets may be declared for user-supplied evaluation or an out-of-process boundary. They cannot be bundled, and they cannot be declared as training inputs for a distributable artifact. Restricted vocabularies, credentialed corpora, and data-use-agreement datasets remain user supplied. The gate also cross-checks catalog counts against the licensing report so an asset cannot disappear from the aggregate total.

Exceptions¶

Exceptions are coded, time-bounded, and tied to one gate. Free-form notes do not enter the packet. An accepted low- or medium-severity exception may be recorded, but it does not change a failing gate into a pass. Pending, rejected, or expired exceptions block release. High- and critical-severity exceptions are non-waivable in this gate.

Critical privacy leakage, broken provenance, invalid spans or evidence, non-idempotent processing, migration failures, unreviewed high-risk output, and unresolved critical security findings therefore cannot be bypassed by an exception record.

Supported claims¶

A ready packet requires these precise claims:

• local-first operation;
• offline operation after explicitly requested assets are present;
• synthetic release evidence;
• human review for high-risk output; and
• typed failure states.

The packet also requires these limitations:

• not clinically validated;
• not a medical device;
• no autonomous clinical action;
• local validation is required; and
• restricted assets remain user supplied.

Public material must not turn technical conformance, synthetic fixtures, or a signed packet into a claim of clinical efficacy, regulatory certification, patient-outcome improvement, or autonomous diagnosis or treatment.

Deployment boundary¶

The gate is local and read-only with respect to release inputs. It does not download models or data, call a hosted verifier, publish a package, change a model pointer, modify a database, or promote a deployment. Promotion remains a separate action that may consume a verified READY packet.

For production deployment:

1. Produce lane reports on the intended artifact and hardware.
2. Freeze aggregate reports, the synthetic journey, schemas, policies, and license catalog in the tagged checkout.
3. Run the gate in that checkout with network access disabled.
4. Verify the packet signature in a separate process.
5. Archive the manifest and packet under the release retention policy.
6. Require a separate authorized promotion step.

Standards boundaries¶

The interoperability lane can record FHIR, HL7 v2, DICOM SR, OMOP, and table conformance evidence, but the packet does not certify conformance for every profile, implementation guide, terminology release, message variant, or downstream system. Declare the exact versions and fixtures in the hashed input artifacts. Loss that is explicit and typed may remain inspectable; undisclosed round-trip loss blocks release.

Related guidance¶

• Five-source Golden Journey
• V3 Journey Threat Model
• Migrate from 2.5 to 3.0
• Release Streams and Channels