Skip to content

V4.0 OpenMedKit Release Gate and Evidence Packet

The portable-kit milestone first planned as v3.5 was folded into v4.0. Its release gate is therefore named for v4.0. The gate makes one fail-closed, signed decision for the portable kits. It binds one exact candidate commit to:

  • the kit artifacts (Swift package source, AAR, npm, Flutter, Tauri, Electron and Python distributions where present);
  • the frozen support matrix;
  • the device qualification report;
  • model bundle manifests, parity and golden fixtures, the license inventory and the kit threat model;
  • test-lane results; and
  • P1 and P2 work moved out of the release, with a rationale, a destination and the claims it narrows.

The packet reuses the v3 Journey release gate canonical JSON, caller-keyed HMAC-SHA256 signature and atomic writer. It is technical release evidence. It is not clinical validation, certification, a medical-device determination or permission to publish. A READY packet is input to a separate, authorized promotion step. The gate does not run devices, tag, publish or promote anything.

The code is openmed.eval.kit_release. The command-line entry point is scripts/release/kit_release_gate.py.

Public contracts

Two strict JSON Schemas ship in openmed/core/schemas/json/:

  • kit_release_manifest.schema.json: the operator-supplied manifest.
  • kit_release_packet.schema.json: the signed decision packet.

Both use schema version 1.0.0 and compatibility policy same_major. An unknown schema version is refused with unknown_schema. Unknown fields, duplicate JSON keys, non-finite numbers, absolute paths, parent traversal and symlinked evidence are rejected. Errors carry value-free codes only, for example evidence_path_invalid. They never repeat a submitted value or path.

Manifest

Key Contents
release version (semantic version), matrix_release (v<major>.<minor> of the version), candidate_commit (full 40-character commit), evaluated_at and the matching source_date_epoch.
policy max_evidence_age_seconds, at most one year.
evidence One entry per evidence file: id, kind, path, sha256, candidate_commit and generated_at. Kit artifacts also name their surface and format.
moved_items issue, priority, destination, rationale and narrowed_claims.

Paths are relative to the candidate checkout. Evidence files do not have to be committed: build outputs such as an AAR can sit in an ignored directory of the same checkout. The gate reads each file without following symlinks, digests it and never copies the path or the content into the packet.

evaluated_at is pinned in the manifest, so the gate reads no wall clock. Given the same checkout, manifest, evidence and key, the packet digest and signature are reproducible.

Evidence kinds

Every kind is required for a release. A kind without an entry is reported as not_present and the decision is NOT_READY.

Kind Entries Default loader Checks
kit_artifact one or more digest only surface and format must be compatible; every surface with a supported cell needs one.
support_matrix exactly one typed Validates with the matrix validator; must be frozen and name release.matrix_release.
qualification_report exactly one typed Self-digest, matrix release, matrix digest, no rejected records, real-device qualification of supported cells.
bundle_manifest one or more digest only Digest binding.
parity_fixture one or more digest only Digest binding (parity vectors and golden fixtures).
license_inventory exactly one digest only Digest binding.
threat_model exactly one digest only Digest binding of the kit threat-model document.
test_lane one or more typed Lane state, failures, test count and the lane's own candidate commit.

Kit artifact formats and the surfaces they can ship:

Format Surfaces
swiftpm-source swift, document-intake, key-store
aar android, document-intake, key-store
npm-tarball web, react-native, electron
flutter-package flutter
tauri-crate tauri
electron-package electron
python-wheel, python-sdist python

A test-lane record (openmed.kit_release.test_lane.v1) is a closed object:

{
  "schema": "openmed.kit_release.test_lane.v1",
  "lane": "ci-swift-macos",
  "candidate_commit": "<40-character commit>",
  "state": "success",
  "test_count": 120,
  "failed_count": 0,
  "skipped_count": 3
}

state uses the Journey result states. Only success with no failures and at least one test passes. The lane id is the matrix evidence id of the CI lane it reports.

Pluggable loaders

Bundle manifests, parity vectors, the license gate report and the kit threat model are produced by separate work. Until a typed loader exists for them, the default loader binds only their digest and the packet records "loader": "digest_only". Callers can inject a typed loader for any kind:

from openmed.eval.kit_release import (
    KitEvidenceResult,
    KitEvidenceSource,
    evaluate_kit_release,
)


def load_license_report(source: KitEvidenceSource) -> KitEvidenceResult:
    report = source.read_json()
    blocked = report.get("decision") != "pass"
    return KitEvidenceResult(
        blocking_codes=("license_gate_failed",) if blocked else (),
        summary={"package_count": len(report.get("packages", []))},
    )


packet = evaluate_kit_release(
    manifest,
    repo_root=".",
    signing_key=key,
    loaders={"license_inventory": load_license_report},
)

A loader receives the digested bytes, never the path. It returns bare codes; the gate appends :<evidence id>. Codes and summary values that are not bounded lowercase codes, counts, booleans or digests are dropped and reported as loader_code_invalid or loader_summary_invalid. A loader that raises fails closed with evidence_loader_failed; its message is discarded.

Release checks

Candidate binding

The checkout's HEAD must equal release.candidate_commit (checkout_commit_mismatch, or checkout_unresolved when no commit can be read). Each evidence entry names the commit it was produced from. An entry from another commit fails with wrong_candidate:<id>, and so does a lane record whose own candidate_commit differs.

Evidence integrity and age

Code Meaning
evidence_not_present:<kind> No entry for a required kind.
evidence_kind_repeated:<kind> More than one entry for a single-entry kind.
evidence_missing:<id> The file is absent, not a regular file, a symlink, or outside the checkout.
stale_digest:<id> The file no longer matches the pinned digest. The loader does not run.
evidence_expired:<id>, evidence_from_future:<id> generated_at is older than the policy allows, or later than evaluated_at.
unknown_schema:<id> A typed evidence document has an unknown schema.
evidence_unreadable:<id> A typed document is not strict JSON.
evidence_too_large:<id> A typed document is larger than 16 MiB; it is digested but not parsed.
matrix_invalid:<id>, report_invalid:<id>, lane_invalid:<id> A typed document fails its own validation.
lane_not_successful:<id>, lane_failed:<id>, lane_empty:<id> A lane record is not success, has failures, or ran no tests.
evidence_loader_failed:<id> A loader raised or returned the wrong type.
stale_matrix_digest:<id> The qualification report was collected against a different matrix.
report_digest_mismatch:<id> The report's self-digest does not match its content.
qualification_records_rejected:<id> The report rejected at least one evidence record.
matrix_not_frozen:<id>, matrix_release_mismatch:<id> The matrix is a draft, or names another release.

Supported cells need real-device evidence

Every supported inference cell in the frozen matrix must be qualified in the qualification report. The gate does not trust the report's flag alone: it recomputes the result from the report's targets, and only passing physical-device records count.

Code Meaning
supported_cell_without_evidence:<cell> The cell is unplanned or has no records.
simulator_only_evidence:<cell> Only simulator or emulator runs passed, or the cell's own device class is a simulator.
fake_runtime_only_evidence:<cell> Only fake-runtime runs passed.
supported_cell_not_qualified:<cell> A device run failed, was stale or was not run.
qualification_report_inconsistent:<cell> The cell flag disagrees with its targets.
lane_result_missing:<lane>:<cell>, lane_not_passing:<lane>:<cell> A CI lane the cell names as evidence has no passing lane record.

Key-store and OCR cells have no model runtime. They are out_of_scope for qualification and still need their CI lanes. Every surface with a supported cell needs a kit artifact (kit_artifact_missing:<surface>). Surfaces that only have experimental cells may ship artifacts; the packet counts them.

Moved P1 and P2 work

Every item moved out of the release needs a coded rationale and a destination: a later milestone such as v4.1, or a follow-up issue such as #4455.

Code Meaning
moved_item_rationale_missing:<issue> No rationale.
moved_item_rationale_unknown:<issue> The rationale is not one of blocked_by_dependency, evidence_incomplete, hardware_unavailable, out_of_scope, risk_not_accepted, toolchain_unavailable or upstream_unreleased.
moved_item_destination_missing:<issue> No destination.
moved_item_destination_invalid:<issue> Not a milestone or issue reference, or the release itself.
moved_item_p0:<issue> A P0 item cannot be moved out.
narrowed_claim_unknown:<issue> A narrowed claim is not a matrix surface or cell.
narrowed_claim_still_supported:<claim>:<issue> The matrix still claims support for what the item narrows.

Unknown rationale or destination text is never copied into the packet; the packet records null instead.

Packet

The packet carries:

  • release and repository_binding (the resolved HEAD);
  • evidence: one row per entry in canonical order (kind order, then id), with its digest, size, age, candidate commit, loader, value-free summary and codes;
  • evidence_kinds: every kind with present, not_present or blocked and its entry count;
  • supported_cells, surfaces and moved_items;
  • blocking_codes, the sorted union of every code;
  • fixed limitations: no_autonomous_clinical_action, no_publication_authorization, not_a_medical_device, not_clinically_validated and requires_local_validation; and
  • manifest_digest, packet_digest and the signature (key_id, algorithm and HMAC value; never the key).

The decision is READY only when blocking_codes is empty.

Building and verifying

Supply a key of at least 32 bytes through OPENMED_KIT_RELEASE_KEY or a permission-restricted --signing-key-file. Do not put the key in a command, shell history or repository file.

python scripts/release/kit_release_gate.py \
  --manifest kit-release-manifest.json \
  --signing-key-file /run/secrets/openmed-kit-release-key \
  --key-id kit-release-2026 \
  --output kit-release-packet.json

python scripts/release/kit_release_gate.py \
  --verify kit-release-packet.json \
  --signing-key-file /run/secrets/openmed-kit-release-key \
  --expected-candidate "$(git rev-parse HEAD)" \
  --require-ready

The build refuses to replace an existing packet unless --overwrite is given, and writes atomically. Exit status for a build is 0 for a signed READY packet, 1 for a signed NOT_READY packet and 2 when no safe decision can be produced. For --verify it is 0 when the packet is valid, 1 with reason codes and 2 when the packet or key is unusable.

Offline verification recomputes the digest and HMAC and checks the packet's own structure:

Code Meaning
unknown_schema Unsupported schema version or compatibility policy.
packet_malformed A field is missing or has the wrong type.
signature_missing The packet is unsigned.
signature_invalid The HMAC does not match (changed content or wrong key).
packet_digest_mismatch The canonical digest does not match.
evidence_order_invalid Evidence or kind rows are not in canonical order.
evidence_kind_missing:<kind> A required kind row was removed.
evidence_count_mismatch:<kind> A kind row disagrees with the evidence rows.
wrong_candidate:<id> An evidence row is bound to another commit.
candidate_mismatch The packet is not for --expected-candidate.
decision_inconsistent READY with blocking codes, or NOT_READY without.
decision_not_ready --require-ready and the decision is NOT_READY.

From Python:

import json
from pathlib import Path

from openmed.eval.kit_release import verify_kit_release_packet

payload = json.loads(Path("kit-release-packet.json").read_text())
result = verify_kit_release_packet(
    payload, signing_key, expected_candidate=commit, require_ready=True
)
assert result.valid, result.reason_codes

The test suite signs fixture packets with a committed key named TEST_ONLY_SIGNING_KEY in tests/fixtures/kit_release.py. That key is public. Never use it for release evidence.

Privacy boundary

The packet holds controlled codes, counts, matrix identifiers, commits and digests only. It never holds evidence paths, file contents, device models or serials, free text, credentials or key material. Tests plant sentinel names, identifiers, paths and tokens in evidence files, file names and moved-item text, and check that none of them reach the packet or the command output.

What this gate does not do

  • It does not run kits on devices. Device evidence comes from the qualification harness, which does not attest that a record came from the hardware it names.
  • It does not parse bundle manifests, parity vectors, license reports or the threat model until a typed loader for them is supplied. Those producers are separate work; with the default loaders their digests are bound but their content is not evaluated.
  • It does not define the support matrix, tag a release, publish packages or authorize promotion. Keys are caller-supplied; the gate stores none.