Raspberry Pi and Jetson Edge Benchmark¶

OpenMed provides a minimal CPU-oriented install profile and a reproducible, socket-blocked benchmark for 64-bit ARM single-board computers. The benchmark records cold start, steady-state whitespace tokens per second, per-note latency, peak resident memory, and the complete isolated install size.

The default workload and ONNX graph are synthetic smoke fixtures. Their numbers validate packaging, architecture, offline execution, result serialization, and footprint enforcement; they are not clinical-model throughput claims or measurements from the named target boards. Run the same command with a local INT8 OpenMed artifact on the real device before making deployment decisions.

Minimal aarch64 profile¶

From a source checkout, create a locked installation containing the OpenMed core, NumPy, ONNX Runtime, and the tokenizer runtime:

UV_PROJECT_ENVIRONMENT=.venv-edge \
  uv sync --frozen --extra edge-sbc --no-dev --no-editable --python 3.11

For a released package, the equivalent profile is:

python3.11 -m venv .venv-edge
.venv-edge/bin/python -m pip install --no-compile "openmed[edge-sbc]"

edge-sbc deliberately excludes the ONNX graph-building package, Torch, and Transformers. It is for already-downloaded local artifacts and CPU execution. Use openmed[onnx-runtime] on a connected staging machine when the application also needs the Hub download helper, then transfer the pinned artifact directory to the SBC. The tokenizer wheel can carry transitive packaging or Hub utilities; the benchmark does not invoke them and blocks outbound sockets for the complete runtime-load and inference window.

Raspberry Pi OS or Ubuntu must be 64-bit and report aarch64. A 32-bit armv7l userspace is unsupported. Jetson uses CPUExecutionProvider; CUDA and TensorRT engines are intentionally outside this benchmark.

Run the reproducible smoke benchmark¶

Resolve the isolated environment's installation directory, then run one target profile:

EDGE_PYTHON=.venv-edge/bin/python
EDGE_SITE_PACKAGES="$($EDGE_PYTHON -c \
  'import sysconfig; print(sysconfig.get_paths()["purelib"])')"

OPENMED_OFFLINE=1 "$EDGE_PYTHON" -m openmed.eval.edge_benchmark \
  --profile raspberry-pi-5 \
  --repeat 20 \
  --install-path "$EDGE_SITE_PACKAGES" \
  --require-aarch64 \
  --output edge-raspberry-pi-5.json

"$EDGE_PYTHON" -m openmed.eval.footprint_gate \
  --profile raspberry-pi-5 \
  --report edge-raspberry-pi-5.json \
  --output footprint-raspberry-pi-5.json

Replace the profile with jetson-nano for its tighter resident-memory budget. Both commands exit non-zero on invalid input. The footprint gate also exits non-zero when either required measurement is missing or exceeds its inclusive maximum.

The benchmark loads edge_identity.onnx.b64 through the real aarch64 ONNX Runtime CPU provider and runs the committed two-note synthetic corpus. Cold start includes runtime/session construction and the first inference. Throughput and p50/p95/p99 latency cover the subsequent repeated corpus passes. Install size is the sum of regular files in --install-path, excluding symlinks and interpreter-generated bytecode caches; peak RSS uses the process high-water mark. Tokens are Unicode strings separated by whitespace, so the metric is reproducible without retaining tokenizer output.

Benchmark a local INT8 model¶

On a connected staging machine, pin and download the model revision. Copy the complete local artifact directory to the device and disconnect networking before measuring. Then add --model:

OPENMED_OFFLINE=1 "$EDGE_PYTHON" -m openmed.eval.edge_benchmark \
  --profile raspberry-pi-5 \
  --model /opt/openmed/model-int8 \
  --variant int8 \
  --repeat 20 \
  --install-path "$EDGE_SITE_PACKAGES" \
  --require-aarch64 \
  --output edge-local-int8.json

The path must already exist. OpenMed passes local_files_only=True, selects CPUExecutionProvider, and keeps the socket guard active while the model and tokenizer load. The result stores the ONNX artifact SHA-256 digest but never the local path, note text, document identifiers, predicted spans, or detected text.

Footprint budgets¶

The source of truth is openmed/eval/budgets/edge_sbc.json:

The install ceiling applies to the isolated edge-sbc environment, excluding the Python interpreter and model weights. Peak RSS includes the interpreter, OpenMed, ONNX Runtime, and the loaded benchmark/model process. A real model can therefore pass the install gate and fail the RAM gate independently.

Archived synthetic proxy results¶

The initial records below were captured on 2026-08-18 with a frozen OpenMed 2.2.0 source install, Python 3.11.10, ONNX Runtime 1.23.2, and the committed identity graph on an Apple ARM64 development host. The profile column selects the target budget only; the embedded machine object is authoritative, and these rows must not be described as Raspberry Pi or Jetson hardware results.

The aggregate source records are archived as 2026-08-18-raspberry-pi-5-arm64-proxy.json and 2026-08-18-jetson-nano-arm64-proxy.json. Short synthetic runs can vary with process startup and scheduler state; compare regressions through the hard byte budgets, not these informational rates.

CI and privacy guarantees¶

.github/workflows/edge-benchmark.yml runs both profiles on GitHub's native ubuntu-24.04-arm runner, verifies uname -m is aarch64, creates the exact frozen edge-sbc environment, executes the benchmark with OPENMED_OFFLINE=1, applies both footprint limits, and retains the aggregate JSON evidence for 90 days.

Only the committed fabricated notes are accepted by the default runner. Every document must declare metadata.source as synthetic; other inputs fail before runtime loading. Do not put real patient text, restricted corpora, model paths, or secrets in benchmark metadata or filenames.