Synthetic training data¶
openmed.training.synthetic holds the deterministic, offline generators that OpenMed uses to build augmentation data for de-identification, document structure, and social-determinant models. The examples here use repository-owned templates and synthetic seeds without network access or restricted clinical records. Caller-supplied seeds and translators retain their own provenance and runtime requirements.
Synthetic outputs are augmentation data, not clinical ground truth. The spans, boxes, and section labels are correct by construction for the synthetic text they came from. They say nothing about how a model will behave on real notes, and they must never be used to report clinical accuracy. Keep them clearly separated from held-out evaluation data; the generators that touch evaluation fixtures include a leakage guard for exactly that reason.
Generators¶
| Page | Public symbols | Produces |
|---|---|---|
| Locale PHI notes | LocalePhiGenerator, generate_locale_phi_examples, LocalePhiExample, SyntheticPhiSpan, SUPPORTED_LOCALE_PHI_LANGUAGES, LOCALE_PHI_LABELS | One-paragraph notes in the declared supported languages with aligned PHI spans |
| Burned-in PHI images | BurnedInGenerator, generate_burned_in_example, BurnedInExample, BurnedInTextBox, BACKGROUND_MODES, BURNED_IN_LABELS, DEFAULT_CANVAS_SIZE, DEFAULT_FONT_NAMES, DEFAULT_FONT_SIZES | Images with rendered PHI text plus pixel boxes and text spans |
| Social-history sections | generate_social_history_examples, SyntheticSocialHistory, SDOHEvent, SOCIAL_HISTORY_CATEGORIES, SYNTHETIC_SOCIAL_HISTORY_SOURCE | Social History sections with SDOH gold events |
| Section-label datasets | build_section_dataset, load_section_dataset, load_section_manifest, validate_section_labels, validate_section_record, assert_no_eval_overlap, SectionDatasetBuildResult, SectionDatasetLeakageError, and the section constants | JSONL notes with BIO section tags and a hash manifest |
| Span offset projection | SpanAnnotation, SpanProjectionError, normalize_span_annotations, validate_span_integrity, span_corruption_count, realign_translated_spans | Checked span offsets, used by translation augmentation |
| Translation augmentation | augment_span_annotated_examples, DictionaryTranslator, ModelBackedTranslator, TranslationAugmentedExample, load_span_jsonl, write_augmented_jsonl, DEFAULT_TARGET_LANGUAGES, SYNTHETIC_SOURCE | Translated and back-translated span-annotated examples |
| SPDX license identifiers | normalize_spdx_identifier, SpdxIdentifier, SpdxIdentifierStatus, PERMISSIVE_SPDX_IDENTIFIERS, SPDX_IDENTIFIER_SCHEMA_VERSION | Normalized license identifiers for lineage manifests |
Conventions shared by every page¶
- Offline. Each example on these pages runs without network access. Pillow is only needed for the burned-in generator (
pip install "openmed[multimodal]"); the other generators need nothing beyond the core install. - Synthetic values only. Names, identifiers, phone numbers, and addresses in the examples come from the generators themselves. They are not real people and must not be copied into issues or logs as if they were.
- Seeded determinism. The same seed gives the same output within one environment. Values that come from Faker can change when the Faker version changes, so pin it, and record the seed and package versions next to any dataset you keep.
- Provenance flags. Preserve the source and synthetic metadata described on each generator page. Their keys differ: social-history records use
metadata.syntheticandmetadata.source, while locale PHI training items also exposeis_synthetic,synthetic_source, andcontains_real_phi. Keep these declarations when composing datasets.
Additional public exports¶
The package also re-exports these APIs added since the original generator set. The linked guide or schema section owns their contract.
- HL7 v2 message generation and leakage:
DEFAULT_HL7V2_TIMEZONE_OFFSETS,HL7V2_ADT_EVENTS,HL7V2_SYNTHETIC_MESSAGE_TYPES,HL7V2_SYNTHETIC_SCHEMA_VERSION,HL7V2_TIMESTAMP_PRECISIONS,MAX_HL7V2_MESSAGES,MAX_HL7V2_NTE_SEGMENTS,MAX_HL7V2_OBX_SEGMENTS,MAX_HL7V2_REPETITIONS,SYNTHETIC_HL7V2_SOURCE,HL7V2LeakageReport,HL7V2MessageGenerator,HL7V2ResidualPosition,HL7V2TruthValue,SyntheticHL7V2Message,HL7V2_SURROGATE_PROVIDERS,HL7V2LeakageError,HL7V2SynthesisConfig,HL7V2SynthesisError,evaluate_hl7v2_leakage,generate_hl7v2_messages. - Synthetic license and lineage validation:
DEFAULT_LICENSE_LINEAGE_POLICY,DEFAULT_MAX_DEPENDENCIES,LICENSE_LINEAGE_FIELDS,LICENSE_LINEAGE_SCHEMA_VERSION,LICENSE_LINEAGE_SOURCE,MAX_LICENSE_EXPRESSION_LENGTH,RESTRICTED_LICENSE_IDENTIFIERS,LicenseExpressionAssessment,LicenseExpressionStatus,LicenseLineageFinding,LicenseLineageReasonCode,LicenseLineageReport,LicenseLineageVerdict,SyntheticLicenseLineageError,SyntheticModelProvenance,load_license_lineage_manifest,MAX_DEPENDENCIES,LicenseLineageSourceClass,RedistributionDecision,SyntheticDatasetDependency,SyntheticDatasetLicenseManifest,SyntheticGeneratorProvenance,SyntheticLicenseLineagePolicy,assess_license_expression,assert_redistribution_allowed,manifest_digest,validate_license_lineage. - Synthetic record schemas:
SYNTHETIC_RECORD_SCHEMA_IDS,SYNTHETIC_RECORD_SCHEMA_NAMES,SYNTHETIC_RECORD_SCHEMA_VERSION,SyntheticRecordSchemaError,export_record_schema,export_record_schema_fingerprints,export_record_schema_json,export_record_schemas,export_record_schemas_json,record_schema_fingerprint,record_schema_id,validate_record,write_record_schemas.
Record schemas¶
The schema helpers describe the synthetic records already emitted by the package. validate_record checks the declared schema and span/box invariants; it does not certify clinical correctness or prove that arbitrary input is synthetic. SyntheticRecordSchemaError reports a rejected record.
from openmed.training.synthetic import (
LocalePhiGenerator, export_record_schema, record_schema_id, validate_record,
)
name = "locale_phi_example"
schema = export_record_schema(name)
assert schema["$id"] == record_schema_id(name)
validate_record(name, LocalePhiGenerator(seed=7).generate("en").to_training_item())
SYNTHETIC_RECORD_SCHEMA_NAMES, SYNTHETIC_RECORD_SCHEMA_IDS and SYNTHETIC_RECORD_SCHEMA_VERSION describe the available schema contracts. The JSON and fingerprint export helpers return canonical content; write_record_schemas(directory) writes them only to the directory you choose.