Synthetic Medical Documents Collection

TL;DR

If you don’t feel like creating a set of over 7,700 synthetic but realistic files for testing document recognition or masking, no worries, I’ve got you: https://huggingface.co/datasets/morzel85/synthetic-medical-document-recognition-benchmark

Intro

I’ve recently worked on building an AI document pipeline for a lightweight local service to mask PHI (Protected Health Information) in uploaded document scans (aka de-identification). I had to assess the performance of various OCR engines (Tesseract, RapidOCR, PaddleOCR, PaddleOCR-VL…) and NER classifiers (GLiNER, StanfordAIMI/stanford-deidentifier-v2…) running on ONNX with a sprinkle of old-school computer vision and regex tricks.

There are a lot of moving parts in such a setup, so it required a lot of testing and tuning. For that, a robust set of test documents was vital. I needed something realistic yet fully synthetic and marked as such, so the generated files could be safely shared and used for demos.

Fortunately, the good folks behind Synthea built a system that generates sensible patient medical histories with the ability to export data in FHIR format. I’ve used Codex (sorry AI haters, a modern LLM harness is great for that) to turn synthetic records into a set of 9 document types, each in 3 variants expressed in HTML and PDF. The documents are also available as rasters (PNG files) in a clean conversion and with simulated degradations (photocopy, stains, skewing, flipped orientation, clipping, etc.). Alongside the page images, there’s also a PDF version of each document that has scanned pages (rasters). JSON files with FHIR data are also included in case you want to compare recognition results with the ground truth…

README.md

Synthetic Medical Document Recognition Benchmark

This dataset contains synthetic, English-language medical records rendered as documents for evaluating automated data extraction and de-identification systems. Each synthetic patient has a longitudinal FHIR R4 record and multiple visual representations derived from that record.

Every rendered document is clearly marked as synthetic. This makes the dataset suitable for manual testing, product demonstrations, and workflows that benefit from realistic-looking medical documents without exposing real patient data.

The dataset is designed as an evaluation benchmark, not as a training corpus. It is too small for representative model training, and its repeated elements, synthetic-data notices, and watermarks could provide shortcuts for a trained model.

All people, clinical histories, organizations, and practitioners in this dataset are synthetic. The dataset is intended for software development and benchmarking, not for clinical use.

Bundle contents

The published bundle contains five synthetic patients:

• Adam Hirthe (1ce276e5)
• Andres O’Hara (348deeaf)
• Carolann Pfeffer (ebd5380d)
• Marion Cormier (2a636504)
• Rebecka Aufderhar (8e9abf1d)

For each patient, the bundle provides 27 document layouts across several document categories and three visual template variants. The document categories include clinical timelines, contact information and contact forms, diagnostic documents, notes, observations, reports, surveys, and vaccination histories.

The bundle contains:

Supporting FHIR resources are stored once even when they are referenced by multiple patients. The patient bundles contain their clinical resources and resolve all internal urn:uuid: references. Practitioner, organization, and location references that live outside a patient bundle can be resolved using the companion files included under data/fhir/.

Synthea run metadata and the source-generation project are intentionally not included. They are not required to consume the rendered documents or their FHIR companions.

Document categories and typical length

Each patient has one document for every document-category × template-variant combination: nine document categories and three template variants, for 27 source documents per patient. The average lengths below were measured from the clear 300-DPI renderings in this release. Each average covers 15 documents (five patients × three template variants); the range shows the shortest and longest document in that category.

The degraded PNG and image-only PDF variants preserve the pagination of their corresponding clear document.

Three vendor and style variants

Every document category has type_1, type_2, and type_3. These are three independent visual presentations of the same category of patient information, intended to simulate documents produced by different vendors and systems. Depending on the document category, the variants differ in branding, typography, spacing, table or card structure, information density, handwriting treatment, and patient-facing versus clinician-facing presentation.

The variants also exercise different date conventions where dates are displayed:

• type_1 primarily uses US numeric dates such as 08/24/2026;
• type_2 primarily uses abbreviated month names such as Aug 24, 2026;
• type_3 primarily uses ISO-style dates such as 2026-08-24.

Rendered document variants

The PNG collection contains a pristine rendering of every page and simulated capture degradations suitable for document-recognition testing. These include:

• clear 300-DPI renders;
• photocopies with straight, misoriented, rotated, clipped, and combined variants;
• photographs with stains, perspective skew, page bend, misorientation, clipping, and combined variants.

Every degraded page is derived from its corresponding clear page. Image-only PDFs combine the pages of each PNG variant in their original page order. Searchable PDFs preserve the text and vector content of the HTML source.

File naming

Files belonging to the same patient share a stable prefix:

synthetic_<Given>_<Family>_<patient-id>

For example:

synthetic_Adam_Hirthe_1ce276e5_report_type_2_ver_1.html
synthetic_Adam_Hirthe_1ce276e5_report_type_2_ver_1.pdf
synthetic_Adam_Hirthe_1ce276e5_report_type_2_ver_1_p_2_of_3_dpi_300_photo_ske.png
synthetic_Adam_Hirthe_1ce276e5.json

Common filename fields are:

• type_<n>: visual template variant;
• ver_<n>: document-content version;
• p_<n>_of_<n>: page number and total page count;
• dpi_300: raster resolution;
• clear, photo_*, or photocopy_*: capture/degradation variant;
• _img.pdf: a PDF assembled from raster page images rather than a searchable PDF rendered directly from HTML.

The degradation tokens can be read from left to right:

Combined tokens mean that all named effects are present. For example, photo_sta_ske is a stained, skewed photograph; photo_mis_ske_cli is a misoriented, skewed, clipped photograph; and photocopy_mis_rot_cli is a misoriented, rotated, clipped photocopy.

The shared patient prefix is the join key between HTML, PDF, PNG, and patient FHIR files. A patient FHIR bundle represents the complete longitudinal record; it is therefore a companion to all documents for that patient, rather than a ground-truth file for only one rendered document.

Downloading selected files with Python

The full repository is large, so consumers can use snapshot_download from huggingface_hub with glob-style allow_patterns and ignore_patterns. Install the client first:

python -m pip install -U huggingface_hub

With uv, add it to a project with:

uv add huggingface_hub

Or install it into the current environment without changing project dependencies:

uv pip install -U huggingface_hub

The following downloads every image-only PDF while skipping searchable PDFs, HTML, PNG, and FHIR files:

from huggingface_hub import snapshot_download

repo_id = "morzel85/synthetic-medical-document-recognition-benchmark"

snapshot_download(
    repo_id=repo_id,
    repo_type="dataset",
    allow_patterns="data/pdf/*_img.pdf",
    local_dir="benchmark",
)

Download only clear image-only PDFs:

snapshot_download(
    repo_id=repo_id,
    repo_type="dataset",
    allow_patterns="data/pdf/*_clear*_img.pdf",
    local_dir="benchmark-clear-pdfs",
)

The wildcard between clear and _img.pdf also matches the optional page count in names such as _clear_3p_img.pdf.

Download all image-only PDFs that simulate stained photographs, including stained-and-skewed and other combined variants:

snapshot_download(
    repo_id=repo_id,
    repo_type="dataset",
    allow_patterns="data/pdf/*_photo_sta*_img.pdf",
    local_dir="benchmark-stained-photo-pdfs",
)

Download only the searchable PDFs rendered directly from HTML:

snapshot_download(
    repo_id=repo_id,
    repo_type="dataset",
    allow_patterns="data/pdf/*.pdf",
    ignore_patterns="data/pdf/*_img.pdf",
    local_dir="benchmark-searchable-pdfs",
)

Patterns can also select a patient, document family, or condition. For example, this downloads Adam Hirthe’s clear report PDFs:

snapshot_download(
    repo_id=repo_id,
    repo_type="dataset",
    allow_patterns=(
        "data/pdf/synthetic_Adam_Hirthe_1ce276e5_report_*_clear*_img.pdf"
    ),
    local_dir="adam-clear-reports",
)

The same approach works for data/png/, data/html/, and data/fhir/.

Simulated handwriting

Handwritten-looking fields are rendered using CSS and handwriting fonts licensed under the SIL Open Font License, with per-glyph variation added during rendering. They are not produced using a handwriting model trained on the IAM Handwriting Database. Such a model might produce more realistic results, but it could introduce licensing restrictions because the database’s official terms limit its use to non-commercial research. This dataset is intended to support both commercial and non-commercial use under the CC BY 4.0 license.

Author and attribution

Created by Miłosz Orzeł.

The underlying synthetic health records were generated with Synthea.