update prd

docs/vigilcare-records-gap-analysis.md

@@ -905,7 +905,7 @@ For each fix, add or extend tests in `VigilCareRecordsAPI.Tests/`:
 ## Out of scope (unless explicitly requested)
 
 - HL7v2 ADT message support (FHIR inbound only in VigilCareClinical)
-- OCR or automated field extraction (explicitly excluded in PRD v1)
+- OCR bypassing human review (Phase 13 provides optional pre-fill only; verification and approval gates unchanged)
 - Multi-facility federated identity (single-tenant per deployment in v1)
 - Full EMR functionality (billing, pharmacy inventory, scheduling)
 - SMART on FHIR authorization (OAuth2 scopes for EHR launch context)

docs/vigilcare-records-prd.md

@@ -2,7 +2,7 @@
 
 ## Implementation Status
 
-**Phases 1–8 are complete.** Phase 9 is partially complete.
+**Phases 1–13 are complete.** Post-phase hardening (health checks, user management, auth rate limiting, document access audit, batch cancellation, list sorting, unified promotion retry, normalized patient deduplication, assignment-time `IN_ENTRY` transitions, API-proxied document streaming, CORS) is also done. See the [gap analysis](vigilcare-records-gap-analysis.md) summary matrix for remaining open items.
 
 | Phase | Scope | Status |
 |---|---|---|
@@ -14,13 +14,15 @@
 | 6 | Track B live capture with clinician attestation | Done |
 | 7 | Digitization workstation UI (`vigilcare-records-web`) | Done |
 | 8 | Prometheus metrics, supervisor overview, batch events API, promotion retry | Done |
-| 9 | E2E verification script, clinical scenario docs, extended seed data | Partial |
+| 9 | Extended seed data, E2E verification script, clinical scenario docs | Done |
+| 10 | Barcode/QR cover sheets for high-volume intake | Done |
+| 11 | HL7 FHIR R4 read-only API and FHIR Explorer UI | Done |
+| 12 | Backend-driven batch-type field requirements (`fieldRequirements` metadata) | Done |
+| 13 | Optional OCR-assisted draft pre-fill (Azure or Tesseract; disabled by default) | Done |
 
-**Delivered in Phase 9 so far:** `scripts/run-vigilcare-records-verification-p9.sh` (full workflow + metrics smoke test), [digitization-workstation-guide.md](digitization-workstation-guide.md) (backfill, live capture, corrections).
+**Verification scripts:** `./scripts/run-vigilcare-records-verification-p9.sh` (full workflow + metrics), `./scripts/run-vigilcare-records-phase-10-verification.sh`, `./scripts/run-vigilcare-records-phase-11-verification.sh`, `./scripts/run-vigilcare-records-phase-13-verification.sh`.
 
-**Remaining in Phase 9:** Extended `DataSeeder` with demo patients and batches across all statuses/types/tracks for dashboard demos without manual data entry.
+See [README.md](../README.md) for API reference, quick start, and [digitization-workstation-guide.md](digitization-workstation-guide.md) for operator workflows.
-
-See [README.md](../README.md) for API reference, quick start, and verification scripts.
 
 ---
 
@@ -28,17 +30,19 @@ See [README.md](../README.md) for API reference, quick start, and verification s
 
 A clinical records intake system that converts paper-based patient charts into structured, human-verified digital records before they enter the VigilCareClinical alerting pipeline. Designed for small hospitals, district clinics, and island health systems where the majority of historical and day-to-day records still exist on paper.
 
-The workflow is deliberately manual at every extraction step:
+The workflow is deliberately human-governed at every quality gate:
 
 **Scan / upload → Human data entry → Human verification → Approved patient record**
 
-There is **no OCR** in scope. Every structured field is typed by a human who reads the scan. A second human compares the entry against the original image before the record becomes clinically authoritative. Unapproved drafts never trigger alerts, scoring, or surveillance.
+Every structured field is ultimately verified by a human who reads the scan. A second human compares the entry against the original image before the record becomes clinically authoritative. Unapproved drafts never trigger alerts, scoring, or surveillance.
+
+**Optional OCR assist (Phase 13):** When `Ocr:Enabled` is true, a background service may pre-fill draft fields from the scan after upload. The entry clerk still reviews every value against the image, corrects errors, and submits for verification — OCR converts "type everything" into "review and correct" but does not bypass entry or verification gates. Disabled by default.
 
 VigilCareClinical ([vigilcare-clinical-api-prd.md](vigilcare-clinical-api-prd.md)) remains the downstream intelligence layer: threshold alerting, NEWS2, sepsis detection, ward dashboard, and long-term archival. VigilCare Records is the **precursor** that creates the fuel — patient identity, encounters, and observations — in a governed, auditable way.
 
 This project maps to `sd-mid-009` (Outbox Pattern for approval promotion events), `sd-mid-013` (CQRS — draft vs live read models), `sd-junior-004` (RBAC), and senior trade-off conversations around **data quality gates vs time-to-value** in resource-constrained health systems.
 
-**Stack:** .NET 8 Web API, PostgreSQL 16, MinIO (scanned document storage), Redis 7 (batch assignment locks and live-capture threshold cache), Serilog → Seq, Prometheus → Grafana, xUnit, Docker Compose. Vue 3 digitization workstation UI in `vigilcare-records-web/` (Vite, Pinia, Tailwind CSS).
+**Stack:** .NET 8 Web API, PostgreSQL 16, MinIO (scanned document storage), Redis 7 (batch assignment locks and live-capture threshold cache), Serilog → Seq, Prometheus → Grafana, xUnit, Docker Compose. Vue 3 digitization workstation UI in `vigilcare-records-web/` (Vite, Pinia, Tailwind CSS). Optional OCR via Azure Document Intelligence or Tesseract (`Ocr:Enabled`). Read-only FHIR R4 API at `/fhir` (Firely SDK).
 
 **Prerequisite / companion:** VigilCareClinicalAPI Phases 1–2 minimum (patient, encounter, observation ingest). Full VigilCare value unlocks as approved observations flow into the existing Kafka alert pipeline.
 
@@ -46,7 +50,7 @@ This project maps to `sd-mid-009` (Outbox Pattern for approval promotion events)
 
 ## Goals
 
-- Provide a complete scan-to-approved workflow for paper chart conversion without OCR or machine extraction
+- Provide a complete scan-to-approved workflow for paper chart conversion with human-verified data quality at every gate (optional OCR pre-fill to accelerate entry, never to replace it)
 - Enforce **separation of duties**: the person who enters data cannot verify their own entry
 - Promote approved structured data into VigilCareClinical's live domain model (Patient, Encounter, Observation) atomically
 - Maintain a full audit trail linking every approved field to its source scan, entry clerk, verifier, and approver
@@ -55,8 +59,8 @@ This project maps to `sd-mid-009` (Outbox Pattern for approval promotion events)
 
 ## Non-Goals
 
-- **OCR or automated field extraction** — explicitly out of scope for v1; may be evaluated in a future phase after human-verified baseline quality is established
+- **OCR bypassing human review** — OCR may pre-fill draft fields (Phase 13) but never skips entry, verification, or approval; fully automated extraction without human review remains out of scope
-- HL7/FHIR compliance or LIS instrument integration
+- **Full HL7v2 or FHIR write compliance** — Phase 11 delivers read-only FHIR R4; HL7v2 ADT, FHIR write, SMART on FHIR, and LIS instrument integration remain out of scope
 - Full EMR functionality (billing, pharmacy inventory, scheduling beyond basic encounter open)
 - Replacing VigilCareClinical's alert engine, scoring, or ward dashboard
 - HIPAA-certified or jurisdiction-specific medical device registration (model the patterns; certification scoped per deployment)
@@ -329,10 +333,10 @@ This keeps digitization history, patient coverage stats, and Prometheus batch me
 **Description:** Intake clerk scans or uploads a paper record. System stores the file in MinIO, computes SHA-256, creates a batch in `uploaded` state, and writes a `DigitizationEvent`.
 
 **Endpoints:**
-- `POST /api/v1/digitization-batches` — multipart upload: `file` (PDF, JPEG, PNG; max 25 MB), `batchType`, optional `patientId`, optional `track` (default `backfill`)
+- `POST /api/v1/digitization-batches` — multipart upload: `file` (PDF, JPEG, PNG; max 25 MB), `batchType`, optional `patientId`, optional `track` (default `backfill`), optional `coverSheetCode` (Phase 10 — auto-populates type/track/patient from barcode cover sheet)
 - `GET /api/v1/digitization-batches/:id` — batch detail with document presigned URL (15-minute expiry)
 - `GET /api/v1/digitization-batches` — filter by `status`, `batchType`, `assignedTo`, `track`; paginated
-- `PATCH /api/v1/digitization-batches/:id/assign` — assign to entry clerk (Redis lock prevents double-assignment)
+- `PATCH /api/v1/digitization-batches/:id/assign` — assign to entry clerk (Redis lock prevents double-assignment; transitions `uploaded → in_entry` immediately)
 
 **Validation:**
 - Accepted MIME types: `application/pdf`, `image/jpeg`, `image/png`
@@ -346,7 +350,7 @@ This keeps digitization history, patient coverage stats, and Prometheus batch me
 
 ### 2. Draft Data Entry — *implemented (Phase 2)*
 
-**Description:** Entry clerk views the scan alongside structured form fields. Saves draft patient updates, encounter context, and observations. Batch moves to `in_entry` on first save.
+**Description:** Entry clerk views the scan alongside structured form fields. Saves draft patient updates, encounter context, and observations. Batch moves to `in_entry` on assignment or first save. Form sections (allergies, medications, encounter summary, observations) are driven by backend `fieldRequirements` metadata (Phase 12). When OCR is enabled, pre-filled fields include confidence indicators (Phase 13).
 
 **Endpoints:**
 - `GET /api/v1/digitization-batches/:id/draft` — full draft payload: patient fields, encounter, observations[]
@@ -492,36 +496,95 @@ Docker Compose exposes Prometheus on port **9095** and Grafana on **3013**.
 
 ---
 
-### 9. Authentication and Audit — *implemented (Phases 1, 8)*
+### 9. Authentication and Audit — *implemented (Phases 1, 8, post-phase hardening)*
 
-**Description:** JWT auth with role claims. Every state transition writes a `DigitizationEvent`. Document access logged.
+**Description:** JWT auth with role claims. Every state transition writes a `DigitizationEvent`. Document access logged. Auth endpoints rate-limited (10 requests per 5 minutes). Administrators manage users via CRUD endpoints.
 
 **Endpoints:**
 - `POST /api/v1/auth/login` — returns access token (15 min), refresh token (7 days), and user profile
 - `POST /api/v1/auth/refresh` — rotates refresh token and issues new access token
 - `POST /api/v1/auth/logout` — revokes refresh token server-side
 - `GET /api/v1/auth/me`
+- `POST /api/v1/users` — create user (administrator)
+- `PATCH /api/v1/users/:id` — update user (administrator)
+- `POST /api/v1/users/:id/reset-password` — reset password (administrator)
+- `POST /api/v1/users/me/change-password` — self-service password change
 
 **Audit requirements:**
 - Auth events (`USER_LOGOUT`, `TOKEN_REFRESHED`) persisted in `auth_audit_events`
-- Who viewed a scan and when
+- Who viewed a scan and when (`document_accessed` events on batch detail and document download)
 - Who changed which draft field (field-level diff in event metadata on save)
 - Who approved promotion and which live record IDs were created
 
+**Health probes (post-phase hardening):** `GET /health/live`, `GET /health/ready` (PostgreSQL, Redis, MinIO), `GET /health/startup`.
+
 ---
 
-## Digitization Workstation UI — *implemented (Phase 7)*
+### 10. Cover Sheet System — *implemented (Phase 10)*
 
-Vue 3 SPA at `vigilcare-records-web/` (dev server port **3028**, proxies `/api` → API on **5217**). Four primary views:
+**Description:** Printable barcode/QR cover sheets encode batch type, track, optional patient, and optional entry-clerk pre-assignment. Intake clerks attach cover sheets to paper chart sections before bulk scanning; the workstation reads the code and auto-creates batches without manual classification.
+
+**Endpoints:**
+- `POST /api/v1/cover-sheets/generate` — create 1–100 cover sheets with unique `VCR-CS-{hex}` codes
+- `GET /api/v1/cover-sheets/lookup/{code}` — resolve a cover sheet for intake auto-fill
+- `GET /api/v1/cover-sheets` — list cover sheets with `isUsed` and `patientId` filters
+- `POST /api/v1/cover-sheets/{id}/pdf` — printable single cover sheet PDF with QR code
+- `POST /api/v1/cover-sheets/batch-pdf` — multi-page PDF for batch printing
+
+Cover sheets are single-use; redeemed on batch creation (`409 COVER_SHEET_ALREADY_USED` on reuse).
+
+---
+
+### 11. HL7 FHIR R4 Read API — *implemented (Phase 11)*
+
+**Description:** Read-only FHIR R4 endpoints expose promoted clinical data (`Patient`, `Encounter`, `Observation`) for external EHR and interoperability consumers. Maps VigilCare observation codes to LOINC; search bundles include pagination links.
+
+**Endpoints:**
+- `GET /fhir/metadata` — anonymous `CapabilityStatement`
+- `GET /fhir/Patient/{id}`, `GET /fhir/Patient?name=`, `GET /fhir/Patient?identifier=` (MRN)
+- `GET /fhir/Encounter/{id}`, `GET /fhir/Encounter?patient=`
+- `GET /fhir/Observation/{id}`, `GET /fhir/Observation?patient=&code=&category=&date=`
+- `GET /fhir/Patient/{id}/$everything` — composite bundle
+
+SMART on FHIR authorization and FHIR write operations remain out of scope.
+
+---
+
+### 12. Batch-Type Field Requirements — *implemented (Phase 12)*
+
+**Description:** `fieldRequirements` metadata on `GET /digitization-batches/:id` and `GET /digitization-batches/:id/draft` tells the workstation which form sections to render per batch type (patient demographics, encounter context, encounter summary fields, observations, allergies, medications). Entry and verification forms consume this metadata instead of hardcoding batch-type rules. Backend `ValidateCompleteness()` remains the enforcement layer.
+
+---
+
+### 13. Optional OCR-Assisted Pre-Fill — *implemented (Phase 13; disabled by default)*
+
+**Description:** When `Ocr:Enabled` is true, `OcrProcessingService` polls uploaded batches, extracts text via Azure Document Intelligence or self-hosted Tesseract, and pre-fills draft fields with per-field confidence scores. Entry clerks review and correct; verification and approval workflows are unchanged.
+
+**Configuration:** `Ocr:Enabled` (default `false`), `Ocr:Provider` (`azure` or `tesseract`), `Ocr:ConfidenceThreshold`, `Ocr:PollIntervalSeconds`.
+
+**Privacy:** Cloud provider requires a signed BAA; local Tesseract keeps PHI on-network.
+
+**Draft response:** `ocrConfidence` map on draft payload; UI shows confidence borders and an "OCR Pre-filled" banner when OCR was used.
+
+---
+
+## Digitization Workstation UI — *implemented (Phases 7, 10, 11, 12, 13)*
+
+Vue 3 SPA at `vigilcare-records-web/` (dev server port **3028**, proxies `/api` and `/fhir` → API on **5217**). Role-based routing with JWT refresh.
 
 | View | User | Purpose |
 |---|---|---|
-| **Intake** | Intake clerk | Upload, assign patient, assign entry clerk |
+| **Intake** | Intake clerk | Upload, barcode cover sheet lookup, assign patient, assign entry clerk |
-| **Entry** | Data entry clerk | Side-by-side scan + form with auto-save |
+| **Cover sheets** | Intake clerk, administrator | Generate, list, and print QR cover sheets |
+| **Entry** | Data entry clerk | Side-by-side scan + form with auto-save, backend-driven field visibility, optional OCR confidence indicators |
 | **Verification** | Verifier | Side-by-side with field checkboxes, approve/reject |
+| **Clinical approval** | Clinical approver | Scan review, approve/reject, retroactive alert toggle |
+| **Live capture** | Clinician | Bedside vitals with attestation and password confirm |
+| **Patient history** | All roles | Digitization timeline with correction chain and audit trail |
 | **Queue dashboard** | Administrator | Backlog metrics from work-queue overview |
+| **FHIR Explorer** | Administrator | Browse and search FHIR resources, inspect JSON, load Patient `$everything` |
 
-Role-based routing and JWT refresh are implemented. Not a full EMR UI — clinical alerting views remain in VigilCareClinical's ward dashboard.
+Scan viewer loads documents via authenticated `GET /digitization-batches/:id/document` blob URLs (avoids cross-origin MinIO iframe issues). Not a full EMR UI — clinical alerting views remain in VigilCareClinical's ward dashboard.
 
 See [digitization-workstation-guide.md](digitization-workstation-guide.md) for operator workflows and clinical scenarios.
 
@@ -576,7 +639,13 @@ If VigilCareClinical is unreachable in split deployment, batch remains `approved
 | Supervisor metrics | Work-queue overview and Prometheus gauges reflect live batch counts | Done |
 | Promotion retry | Transient promotion failure defers to `APPROVED` with automatic retry | Done |
 | E2E verification script | `./scripts/run-vigilcare-records-verification-p9.sh` passes against running API | Done |
-| Extended demo seed data | Startup seed includes patients/batches across all statuses for dashboard demos | Pending |
+| Extended demo seed data | Startup seed includes patients/batches across all statuses for dashboard demos | Done |
+| Cover sheet intake | Barcode cover sheet auto-creates batch with correct type/track/patient | Done |
+| FHIR read API | `GET /fhir/Patient`, `/Encounter`, `/Observation` return valid FHIR R4 JSON | Done |
+| Field requirements metadata | Entry/verification forms render sections from `fieldRequirements` on batch/draft responses | Done |
+| OCR pre-fill (when enabled) | Uploaded batch gets draft pre-fill with confidence map; disabled by default | Done |
+| Health checks | `/health/ready` reports PostgreSQL, Redis, MinIO status | Done |
+| Batch cancellation | Administrator can cancel batches in `uploaded`, `in_entry`, or `rejected` | Done |
 
 ---
 
@@ -592,7 +661,11 @@ If VigilCareClinical is unreachable in split deployment, batch remains `approved
 | 6 | Track B live capture with clinician attestation | Done |
 | 7 | Digitization workstation UI (entry + verification side-by-side) | Done |
 | 8 | Prometheus metrics, supervisor dashboard, promotion retry job | Done |
-| 9 | Extended seed data, E2E verification script, clinical scenario documentation | Partial |
+| 9 | Extended seed data, E2E verification script, clinical scenario documentation | Done |
+| 10 | Barcode/QR cover sheet system for high-volume intake | Done |
+| 11 | HL7 FHIR R4 read-only API and FHIR Explorer UI | Done |
+| 12 | Backend-driven batch-type field requirements | Done |
+| 13 | Optional OCR-assisted draft pre-fill | Done |
 
 ---
 
@@ -677,9 +750,49 @@ This phase connects Records to Clinical. Run against a VigilCareClinical Phase 2
 
 **Phase 8 (done):** Prometheus metrics, `GET /work-queue/overview`, cursor-paginated `GET /digitization-batches/:id/events`, promotion deferral (202) and `PromotionRetryService`, Docker Prometheus/Grafana stack. Verification: `./scripts/run-vigilcare-records-phase-8-verification.sh`.
 
-**Phase 9 (partial):**
+**Phase 9 (done):** Extended `DataSeeder.cs` with 10 demo batches across all statuses/types/tracks, `docs/digitization-workstation-guide.md`, `./scripts/run-vigilcare-records-verification-p9.sh`, project README.
-- Done: `docs/digitization-workstation-guide.md`, `./scripts/run-vigilcare-records-verification-p9.sh`, project README
+
-- Remaining: extend `DataSeeder.cs` with demo patients and batches across all statuses, types, and tracks
+---
+
+### Phase 10 — Cover Sheet System
+
+**What to do:**
+1. Add `CoverSheet` entity and cover sheet CRUD/generate/lookup endpoints.
+2. Wire `coverSheetCode` on batch upload to auto-populate type, track, patient, and redeem the sheet.
+3. Generate printable PDFs with QR codes; add `/cover-sheets` and barcode-assisted intake in the Vue UI.
+
+**Verification:** `./scripts/run-vigilcare-records-phase-10-verification.sh`
+
+---
+
+### Phase 11 — FHIR R4 Read API
+
+**What to do:**
+1. Install Firely SDK; map promoted clinical data to FHIR Patient, Encounter, Observation resources.
+2. Implement read and search endpoints with LOINC mapping and pagination bundles.
+3. Add administrator FHIR Explorer view at `/fhir-explorer`.
+
+**Verification:** `./scripts/run-vigilcare-records-phase-11-verification.sh`
+
+---
+
+### Phase 12 — Batch-Type Field Requirements
+
+**What to do:**
+1. Add `BatchTypeFieldRequirements` record with static mapping per batch type.
+2. Include `fieldRequirements` on `DraftPayloadResponse` and `BatchDetailResponse`.
+3. Update `EntryForm.vue` and `VerificationForm.vue` to consume metadata instead of hardcoded batch-type switches.
+
+---
+
+### Phase 13 — Optional OCR Pre-Fill
+
+**What to do:**
+1. Add `IOcrService` with Azure Document Intelligence and Tesseract providers behind `Ocr:Enabled` config.
+2. Implement `OcrProcessingService` background polling and `OcrDraftPreFiller`.
+3. Return `ocrConfidence` on draft payload; show confidence indicators in entry and verification forms.
+
+**Verification:** `./scripts/run-vigilcare-records-phase-13-verification.sh` (automated); manual Azure/Tesseract end-to-end when credentials are configured.
 
 ---
 
@@ -725,7 +838,8 @@ This phase connects Records to Clinical. Run against a VigilCareClinical Phase 2
 
 - [README.md](../README.md) — API reference, quick start, verification scripts, data models
 - [digitization-workstation-guide.md](digitization-workstation-guide.md) — clinical scenarios (backfill, live capture, corrections)
-- [plans/](plans/) — phase-by-phase implementation guides (Phases 1–9)
+- [plans/](plans/) — phase-by-phase implementation guides (Phases 1–13)
+- [vigilcare-records-gap-analysis.md](vigilcare-records-gap-analysis.md) — post-phase hardening tracker and remaining open items
 - [vigilcare-clinical-api-prd.md](vigilcare-clinical-api-prd.md) — downstream alerting and observation ingest
 - [Completed/VigilCareClinicalAPI/VigilCare-Partner-Brief.md](Completed/VigilCareClinicalAPI/VigilCare-Partner-Brief.md) — clinical positioning and scope boundaries
 - [Completed/national-digital-services-architecture.md](Completed/national-digital-services-architecture.md) — scan-and-verify pattern for paper-to-digital government services