gRPC Service¶
OpenMed ships a schema-first gRPC service for polyglot integrations that need typed contracts and server-streaming de-identification. The gRPC service uses the same ServiceRuntime and warm-pool model loader as the REST service, so preloaded models, resident-model limits, and keep-alive behavior stay consistent across transports.
Install¶
Install the service extra to get the gRPC runtime dependencies:
Developers regenerating protobuf stubs need dev dependencies:
Contract¶
The OpenMed service contract is committed at openmed/service/proto/openmed.proto. The client-only KServe V2 wire subset used by the remote inference backend is committed at openmed/service/proto/kserve_v2.proto. Generated Python modules for both are committed under openmed/service/proto/generated/.
The OpenMedService exposes:
Analyze: unary equivalent ofPOST /analyzeExtract: unary equivalent ofPOST /pii/extractDeidentify: unary equivalent ofPOST /pii/deidentifyStreamDeidentify: unary request with server-streamed redacted fragments
Response messages include typed entity records and the canonical OpenMedSpan message shape for callers that consume schema-versioned span contracts.
Start A Server¶
from openmed.service.grpc_server import serve
server = serve() # plaintext, loopback only: 127.0.0.1:50051
server.wait_for_termination()
serve() validates the listener before it creates the server, binds it, and only then starts serving. It raises GrpcStartupError instead of returning a server when the configuration is refused or the address cannot be bound, so a returned server is always listening.
Transport Security¶
The default listener is 127.0.0.1:50051. Plaintext is accepted only on loopback addresses (127.0.0.0/8, [::1], localhost) and on local unix: / unix-abstract: sockets. Wildcard (0.0.0.0, [::]), routable and named hosts require server TLS credentials:
from openmed.service.grpc_security import GrpcTLSConfig
from openmed.service.grpc_server import serve
server = serve(
"0.0.0.0:50051",
tls=GrpcTLSConfig(
certificate_chain_file="/etc/openmed/grpc/server-chain.pem",
private_key_file="/etc/openmed/grpc/server-key.pem",
),
)
Add client_ca_file to require mutual TLS. Clients must then present a certificate issued by one of the CAs in that bundle; clients without a certificate, or with one from another CA, fail the handshake:
tls = GrpcTLSConfig(
certificate_chain_file="/etc/openmed/grpc/server-chain.pem",
private_key_file="/etc/openmed/grpc/server-key.pem",
client_ca_file="/etc/openmed/grpc/client-ca.pem",
)
A mutual-TLS client trusts the server CA and presents its own certificate:
from pathlib import Path
import grpc
credentials = grpc.ssl_channel_credentials(
root_certificates=Path("/etc/openmed/grpc/server-ca.pem").read_bytes(),
private_key=Path("/etc/openmed/grpc/client-key.pem").read_bytes(),
certificate_chain=Path("/etc/openmed/grpc/client-chain.pem").read_bytes(),
)
channel = grpc.secure_channel("openmed-grpc.internal:50051", credentials)
require_client_cert defaults to whether a client CA is configured. Setting it to True without a client CA, or to False with one, is refused rather than silently ignored.
Credential material is loaded and checked before anything is bound. Startup is refused when a file is missing or unreadable, is not valid PEM, is empty or larger than 1 MiB, when the private key is encrypted or does not match the leaf certificate, or when the leaf certificate is expired or not yet valid. The certificate chain file must list the leaf first, followed by any intermediates. The private key must be unencrypted PEM, so protect it with file permissions or a secret mount.
Development override¶
For a disposable lab that needs plaintext on a non-loopback address, pass allow_insecure_remote_dev=True. The override is accepted only when the OPENMED_PROFILE (or the supplied runtime's profile) is dev or test, and it logs grpc_insecure_dev_override_active. Under any other profile, including prod and an unset OPENMED_PROFILE, setting the override fails startup with grpc_insecure_dev_override_rejected, even for a loopback address.
Startup reason codes¶
GrpcStartupError exposes reason, an optional category (certificate_chain, private_key, client_ca, client_auth, listener, or override) and, when a certificate was parsed, fingerprint_sha256. Messages never include file paths, addresses, PEM text or key bytes.
| Reason | Meaning |
|---|---|
grpc_address_invalid | The bind address is not host:port, [ipv6]:port, unix: or unix-abstract: |
grpc_insecure_remote_bind | Plaintext was requested on a non-loopback address |
grpc_insecure_dev_override_rejected | The development override was set outside dev/test |
grpc_tls_incomplete | A certificate, key, or required client CA is not configured |
grpc_tls_client_auth_ambiguous | A client CA was configured with require_client_cert=False |
grpc_tls_unreadable | A credential file could not be read |
grpc_tls_malformed | A credential file is empty, oversized, encrypted or not valid PEM |
grpc_tls_key_mismatch | The private key does not match the leaf certificate |
grpc_tls_certificate_expired | The leaf certificate is past its notAfter |
grpc_tls_certificate_not_yet_valid | The leaf certificate is before its notBefore |
grpc_bind_failed | grpcio raised or reported that nothing was bound |
grpc_auth_required | A non-loopback listener has neither service authentication nor mutual TLS |
Custom listeners¶
create_grpc_server() returns an unbound server and accepts interceptors, policy and listener. Use resolve_grpc_listener() and bind_grpc_listener() to apply the same policy to a server you construct yourself, and pass the listener to create_grpc_server(listener=...) so the authentication check and mutual-TLS principals apply. GrpcListener.report() returns a value-free summary (loopback, transport, override use, certificate and client CA fingerprints, and certificate expiry) for deployment self-checks.
Certificate rotation¶
A GrpcTLSConfig passed straight to serve() is read once at startup. To rotate without a restart, wrap it in a GrpcCredentialReloader. grpcio then asks the reloader for credentials on every new TLS handshake, so new connections use the newest validated generation while established connections and their in-flight unary and streaming RPCs keep the generation they were accepted under.
from openmed.service.grpc_security import GrpcCredentialReloader, GrpcTLSConfig
from openmed.service.grpc_server import serve
from openmed.service.tls_reload import install_reload_signal_handler
reloader = GrpcCredentialReloader(
GrpcTLSConfig(
certificate_chain_file="/etc/openmed/grpc/server-chain.pem",
private_key_file="/etc/openmed/grpc/server-key.pem",
client_ca_file="/etc/openmed/grpc/client-ca.pem",
),
trust_overlap_seconds=300, # old client CA accepted for 5 minutes
max_connection_age_seconds=3600, # old connections retire within an hour
connection_grace_seconds=300, # plus up to 5 minutes for running RPCs
)
server = serve("0.0.0.0:50051", tls=reloader)
install_reload_signal_handler(reloader) # `kill -HUP <pid>` reloads
server.wait_for_termination()
Reloads are local only. Call reloader.reload(), send SIGHUP after install_reload_signal_handler() (POSIX), or run a TLSCredentialWatcher(reloader, interval_seconds=...) that polls file metadata and reloads when it changes. reload() may also be given a new GrpcTLSConfig when each generation is staged in its own directory.
Every reload validates a complete candidate before anything changes: the same checks as startup, plus that the chain links from leaf to intermediates, that the leaf stays valid for at least min_remaining_validity_seconds (5 minutes by default), that every client CA is a currently valid CA certificate, and that the files did not change while they were read. A candidate that would change whether client certificates are required is refused, because grpcio fixes that when the listener is created. Any refusal keeps the previous generation serving; there is never a plaintext or unauthenticated interval. Write new material atomically (write a temporary file, then rename it, or use a secret volume) so a reload never sees half a generation. A restart loads the configured files again and fails closed if they are incomplete.
When the client CA bundle changes, clients issued by the outgoing CA can keep opening connections until the trust overlap ends. After that only the new CA is accepted for new connections. Connections opened under the old generation are closed once they reach max_connection_age_seconds; RPCs still running after connection_grace_seconds end with UNAVAILABLE, which clients can retry on a fresh connection. Pass max_connection_age_seconds=None to leave connection age unbounded.
reload() returns a TLSReloadResult with the new and previous generation IDs, validation_state, activation_state, the overlap deadline and a typed outcome (activated, unchanged or rejected) plus a reason code. reloader.readiness() reports whether the active certificate and client trust are inside their validity windows, and reloader.report() returns generation IDs, fingerprints, counters and states. None of these include file paths, PEM text, subjects or SANs.
| Reload reason | Meaning |
|---|---|
tls_activated / tls_unchanged | The candidate was activated, or matched the active generation |
tls_incomplete | A required file is not configured |
tls_unreadable / tls_malformed | A file could not be read, or is empty, oversized or not valid PEM |
tls_key_mismatch | The key does not match the leaf, for example after a partial write |
tls_certificate_expired / tls_certificate_not_yet_valid | A certificate is outside its validity window |
tls_certificate_expiring | The leaf, or the latest client CA, expires within the minimum remaining validity |
tls_chain_untrusted | The chain does not link leaf to intermediates, or does not lead to the optional TLSCredentialSource.server_ca_file |
tls_client_ca_invalid | The client CA bundle contains a certificate that is not a CA |
tls_policy_incompatible | The candidate would change the client-certificate requirement |
tls_candidate_changed | A file changed while the candidate was being read |
tls_reload_in_progress | Another reload was running; concurrent reloads are rejected, not queued |
tls_activation_failed | grpcio refused the validated material |
Authentication And Authorization¶
Every server built by serve() or create_grpc_server() runs GrpcSecurityInterceptor before the servicer and before any interceptors you add. It reads the same settings as the REST service, so one environment configures both transports:
OPENMED_SERVICE_AUTH_ENABLED,OPENMED_SERVICE_AUTH_API_KEYS, the JWKS, issuer, audience and lifetime settings,OPENMED_SERVICE_AUTH_ROUTE_SCOPES,OPENMED_SERVICE_AUTH_DENY_BY_DEFAULTand the failed-authentication rate limit, described in REST Service Authentication.OPENMED_SERVICE_MTLS_PRINCIPALS, the certificate identity mappings described in Mutual TLS.
Callers send exactly one credential in call metadata: x-api-key: <key>, authorization: Bearer <jwt> or authorization: ApiKey <key>. Duplicate entries, both kinds at once, -bin variants, empty values and non-printable values are refused as malformed. When the listener requires client certificates and no metadata credential is sent, the peer certificate is verified again against the listener's client CA bundle and mapped to a principal through the same mappings REST uses. Unmapped certificates get a fingerprint-derived subject with no scopes. Client certificates must be issued directly by a CA in that bundle, because grpcio exposes only the leaf.
Each RPC requires the scopes of the REST route it mirrors, including any route scope override:
| RPC | REST route | Default scope |
|---|---|---|
Analyze | POST /analyze | analyze:write |
Extract | POST /pii/extract | pii:read |
Deidentify | POST /pii/deidentify | pii:write |
StreamDeidentify | POST /pii/deidentify/stream | pii:write |
All of these checks finish before any request text is parsed or any model runs. With neither service authentication nor a mutual-TLS listener, calls are anonymous. That loopback development mode is still available, but serve() refuses a non-loopback listener in that state with grpc_auth_required. The only exception is the dev/test plaintext override.
metadata = (("authorization", f"Bearer {token}"),)
response = stub.Deidentify(request, metadata=metadata, timeout=30)
Resource Budgets¶
GrpcLimits bounds every call. GrpcLimits.from_env() shares the REST budgets where one exists:
| Limit | Default | Source |
|---|---|---|
max_receive_message_bytes | 16 MiB | OPENMED_SERVICE_MAX_REQUEST_BYTES |
max_send_message_bytes | 64 MiB | |
max_text_chars (also the total of all stream chunks) | 1,000,000 | OPENMED_SERVICE_MAX_TEXT_LENGTH |
max_stream_chunks | 4,096 | |
max_stream_chunk_bytes (UTF-8) | 128 KiB | |
default_stream_buffer_chars | 4,096 | |
max_stream_buffer_chars | 65,536 | |
max_rpc_seconds | the runtime config.timeout |
Message sizes are enforced by grpcio before the request reaches the application. Those rejections are RESOURCE_EXHAUSTED without an openmed-reason trailer. Text, chunk and buffer budgets are checked before chunks are copied or joined. A caller max_buffer above the cap is refused rather than trusted, and an unset chunk_size splits long text into chunks that always fit the per-chunk byte budget.
Rate and concurrency limits use the REST settings (OPENMED_SERVICE_RATE_LIMIT_RPS, OPENMED_SERVICE_RATE_LIMIT_BURST, OPENMED_SERVICE_RATE_LIMIT_MAX_CONCURRENCY, OPENMED_SERVICE_CONCURRENCY_WAIT_SECONDS and OPENMED_SERVICE_THROTTLE_KEY). The peer key uses the gRPC peer host. The x-forwarded-for key reads that metadata entry.
The effective deadline is the smaller of the client deadline and max_rpc_seconds. A concurrency permit is released exactly once, when the call's work settles:
- a unary handler returns or raises;
- a stream is exhausted, fails, is cancelled or passes its deadline.
A cancelled stream stops at the next fragment boundary, and its model bookkeeping is released on the worker thread that ran it. A unary model call that is already running cannot be pre-empted. If it overruns the server budget, its result is discarded with DEADLINE_EXCEEDED.
Failure Reason Codes¶
Refusals use canonical gRPC status codes. Each carries a stable reason in the openmed-reason trailing metadata. Status details are fixed strings. Metadata, tokens, request text, chunks, certificate contents and rejected values are never included.
| Reason | Status |
|---|---|
authentication_required | UNAUTHENTICATED |
invalid_credentials | UNAUTHENTICATED |
credentials_malformed | UNAUTHENTICATED |
insufficient_scope | PERMISSION_DENIED |
auth_rate_limited | RESOURCE_EXHAUSTED |
rate_limited | RESOURCE_EXHAUSTED |
concurrency_limited | RESOURCE_EXHAUSTED |
text_too_long | RESOURCE_EXHAUSTED |
stream_too_many_chunks | RESOURCE_EXHAUSTED |
stream_chunk_too_large | RESOURCE_EXHAUSTED |
stream_buffer_too_large | RESOURCE_EXHAUSTED |
invalid_request | INVALID_ARGUMENT |
model_not_served | INVALID_ARGUMENT |
deadline_exceeded | DEADLINE_EXCEEDED |
internal_error | INTERNAL |
try:
stub.Deidentify(request, metadata=metadata, timeout=30)
except grpc.RpcError as error:
reason = dict(error.trailing_metadata() or ()).get("openmed-reason")
Client Example¶
import grpc
from openmed.service.proto.generated import openmed_pb2, openmed_pb2_grpc
channel = grpc.insecure_channel("127.0.0.1:50051")
stub = openmed_pb2_grpc.OpenMedServiceStub(channel)
response = stub.Deidentify(
openmed_pb2.DeidentifyRequest(
text="Patient Maria Garcia can be reached at maria@example.test.",
method="mask",
lang="en",
)
)
print(response.deidentified_text)
for entity in response.pii_entities:
print(entity.label, entity.start, entity.end)
Streaming De-Identification¶
StreamDeidentify accepts either explicit chunks or a single request.text with an optional chunk_size. The stream yields redacted text fragments as they become safe to emit and ends with a final event that carries aggregate stream metadata and final spans.
request = openmed_pb2.DeidentifyStreamRequest(
request=openmed_pb2.DeidentifyRequest(
text="Long synthetic clinical note...",
method="mask",
),
chunk_size=4096,
max_buffer=8192,
)
for event in stub.StreamDeidentify(request):
if event.redacted_text:
print(event.redacted_text, end="")
Regeneration Gate¶
Run this before committing proto changes:
The unit test suite also runs the same drift check, so CI fails when openmed.proto and the committed generated stubs disagree. Stub freshness does not judge compatibility: the API compatibility gate compiles openmed.proto to a descriptor set and fails on removed, renumbered or retyped fields, reused reserved numbers or names, and removed messages, services or RPCs relative to docs/api/baselines/openmed-proto-descriptor.json.
Standard health probes¶
The service extra pins grpcio-health-checking==1.81.1, the Apache-2.0 upstream implementation of grpc.health.v1.Health. This matches the committed OpenMed generated stubs' grpcio version; the package requires grpcio 1.81.1 or newer and protobuf 6.33.5 through the 6.x series. No reflection service is enabled.
Check and Watch expose only the standard status enum for the overall server (empty service name) and openmed.service.v1.OpenMedService. Both start NOT_SERVING. server.preload_runtime() invokes the configured runtime's actual preload and changes them to SERVING only after success. serve() performs this preload after starting the listener. Failed preload exposes only grpc_preload_failed and serve() stops its listener.
Like the REST /health, /livez and /readyz probes, the two health RPCs never need credentials, even when service authentication is enabled with deny-by-default, so orchestrator probes keep working. They bypass admission control as well. Every OpenMedService RPC on the same listener still goes through GrpcSecurityInterceptor.
For a custom listener, use create_grpc_server(), configure the listener, call start(), then preload_runtime(). The wrapper forwards add_secure_port credentials unchanged, so health and OpenMed RPCs share the same TLS listener. Call begin_drain() before the host stops accepting work or drains existing requests; stop(grace) also publishes terminal NOT_SERVING before stopping. A preload finishing after drain cannot restore readiness. This notification seam does not itself implement coordinated admission or runtime lease draining (#3576). Listener binding and credential policy remain owned by #3546.
For Compose, an image that explicitly includes the separately maintained grpc_health_probe executable can use an exec healthcheck:
healthcheck:
test: ["CMD", "/bin/grpc_health_probe", "-addr=127.0.0.1:50051", "-service=openmed.service.v1.OpenMedService"]
interval: 10s
timeout: 3s
The same container-local exec probe supports a loopback-bound Helm deployment:
readinessProbe:
exec:
command: ["/bin/grpc_health_probe", "-addr=127.0.0.1:50051", "-service=openmed.service.v1.OpenMedService"]
periodSeconds: 10
timeoutSeconds: 3
These examples require a plaintext local listener and the executable in the image; OpenMed does not install it. A TLS listener requires that probe's TLS flags and trusted certificate configuration, rather than an additional insecure health port. Use the same health protocol over your configured TLS channel. The examples are configuration guidance; the focused tests exercise actual ephemeral loopback and TLS servers, not a deployed Compose or Helm cluster.