Trent/vigilcare-records
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
voltsrage
2026-06-26 04:20:20 +08:00
commit 869006e5e7
64 changed files with 4632 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/vigilcare-records-prd.md
+680
View File
@@ -0,0 +1,680 @@
# PRD: VigilCare Records — Paper Chart Digitization & Approval Platform
## Overview
A clinical records intake system that converts paper-based patient charts into structured, human-verified digital records before they enter the VigilCareClinical alerting pipeline. Designed for small hospitals, district clinics, and island health systems where the majority of historical and day-to-day records still exist on paper.
The workflow is deliberately manual at every extraction step:
**Scan / upload → Human data entry → Human verification → Approved patient record**
There is **no OCR** in scope. Every structured field is typed by a human who reads the scan. A second human compares the entry against the original image before the record becomes clinically authoritative. Unapproved drafts never trigger alerts, scoring, or surveillance.
VigilCareClinical ([vigilcare-clinical-api-prd.md](vigilcare-clinical-api-prd.md)) remains the downstream intelligence layer: threshold alerting, NEWS2, sepsis detection, ward dashboard, and long-term archival. VigilCare Records is the **precursor** that creates the fuel — patient identity, encounters, and observations — in a governed, auditable way.
This project maps to `sd-mid-009` (Outbox Pattern for approval promotion events), `sd-mid-013` (CQRS — draft vs live read models), `sd-junior-004` (RBAC), and senior trade-off conversations around **data quality gates vs time-to-value** in resource-constrained health systems.
**Stack:** .NET 8 Web API, PostgreSQL, MinIO (scanned document storage), Redis (work-queue assignment locks), Serilog → Seq, Prometheus → Grafana, xUnit, Docker Compose. Vue 3 digitization workstation UI (separate repo or `VigilCare.Records.Web` project).
**Prerequisite / companion:** VigilCareClinicalAPI Phases 1–2 minimum (patient, encounter, observation ingest). Full VigilCare value unlocks as approved observations flow into the existing Kafka alert pipeline.
---
## Goals
- Provide a complete scan-to-approved workflow for paper chart conversion without OCR or machine extraction
- Enforce **separation of duties**: the person who enters data cannot verify their own entry
- Promote approved structured data into VigilCareClinical's live domain model (Patient, Encounter, Observation) atomically
- Maintain a full audit trail linking every approved field to its source scan, entry clerk, verifier, and approver
- Support two operational modes: **Track A** (historical backfill with full dual-human gate) and **Track B** (live bedside capture with clinician attestation, lighter gate)
- Produce a deployable precursor that makes VigilCareClinical credible in paper-only facilities — not as a standalone EMR replacement
## Non-Goals
- **OCR or automated field extraction** — explicitly out of scope for v1; may be evaluated in a future phase after human-verified baseline quality is established
- HL7/FHIR compliance or LIS instrument integration
- Full EMR functionality (billing, pharmacy inventory, scheduling beyond basic encounter open)
- Replacing VigilCareClinical's alert engine, scoring, or ward dashboard
- HIPAA-certified or jurisdiction-specific medical device registration (model the patterns; certification scoped per deployment)
- Multi-facility federated identity across islands (single-tenant deployment per site in v1)
---
## Problem Statement
VigilCareClinical assumes structured, timestamped clinical data already exists. In paper-based facilities:
1. Patient identity lives in folders, ward books, or duplicate index cards — no stable MRN workflow
2. Vitals and lab results are handwritten — illegible, untimestamped at minute precision, or lost between visits
3. There is no encounter boundary — "the patient" is not the same as "this admission" or "this clinic visit"
4. Clinicians cannot trust machine-generated alerts if the underlying values were guessed from poor handwriting
Without a digitization and approval layer, VigilCare has nothing to observe. With it, even a 20-bed district hospital can convert charts incrementally and activate real-time alerting as live capture replaces paper forms.
---
## Relationship to VigilCareClinical
```
┌─────────────────────────────────────────────────────────────────────┐
│ VigilCare Records (this PRD) │
│ │
│ Scan → Entry → Verify → Approve │
│ ↓ │
│ Draft tables (never alert) │
│ ↓ on approval │
│ Promotion service ──────────────────────────────────────────────┐ │
└──────────────────────────────────────────────────────────────────│──┘
│
┌──────────────────────────────────────────────────────────────────▼──┐
│ VigilCareClinicalAPI │
│ │
│ Patient → Encounter → Observation → Outbox → Kafka → Alerts │
└─────────────────────────────────────────────────────────────────────┘
```
**Invariant:** Draft observations are invisible to VigilCareClinical's alert engine, Elasticsearch ward projection, and scoring consumers. Only batches that have passed approval are promoted to live tables. Promotion is idempotent — re-running an approval job for the same batch produces no duplicate live rows.
**Track A (backfill):** Full pipeline — scan, entry, verification, approval. Historical vitals and labs enter as observations with `recordedAt` taken from the chart (not scan time). Alerts on backfilled critical values are **suppressed by default** unless the facility explicitly opts in per batch (`enableRetroactiveAlerts: false` default).
**Track B (live capture):** Credentialed clinician enters vitals at bedside. Skips verification queue; requires `clinicianAttestation: true` on submit. Observations promote immediately to VigilCareClinical with `source: live_capture`. Every live-capture submission still creates a `DigitizationBatch` (audit unit) with draft observations, `DigitizationEvent` entries, and live rows in a single transaction — see [Track B audit model](#track-b-audit-model).
**Deployment model (v1):** Integrated PostgreSQL — Records draft tables and VigilCareClinical live tables (`patients`, `encounters`, `observations`, `outbox_events`) share one database instance (separate schemas). Promotion runs in a single local transaction. Split deployment with HTTP + saga retry is documented as a future deployment option, not the portfolio default.
---
## API Conventions
Same response envelope as VigilCareClinical and other portfolio projects. Prefix: `/api/v1`.
**Success:**
```json
{
"success": true,
"statusCode": 200,
"data": {},
"error": null
}
```
**Error:**
```json
{
"success": false,
"statusCode": 422,
"data": null,
"error": {
"message": "Verifier cannot approve a batch they entered.",
"code": "SEPARATION_OF_DUTIES_VIOLATION"
}
}
```
**Pagination:** Work queues and batch lists use offset pagination (`?page=1&pageSize=20`). Audit event history uses cursor pagination on `(occurred_at DESC, id DESC)`.
**Idempotency:** `POST /api/v1/digitization-batches/:id/approve` accepts an `Idempotency-Key` header. Duplicate approval requests return the same promotion result without creating duplicate live observations in VigilCareClinical.
---
## Roles and Permissions
| Role | Permissions |
|---|---|
| **Intake clerk** | Upload scans, create batches, assign patient (existing MRN or new registration draft) |
| **Data entry clerk** | Edit draft fields on batches in `uploaded` or `rejected` state; submit for verification |
| **Verifier** | Review batches in `pending_verification`; field-level pass/fail; reject with reason; cannot verify batches they entered |
| **Clinical approver** | Final promotion trigger via `POST .../approve` for batches in `verified` or `awaiting_clinical_approval`; cannot approve batches they entered |
| **Clinician** | Track B live capture; attestation on own entries (attestation satisfies verify + approve for that batch only) |
| **Administrator** | User management, batch type config, retroactive alert policy, work-queue reassignment |
### Verification vs approval (Track A)
Track A uses **two distinct human gates** before promotion:
1. **Verifier** — compares structured draft fields against the scan (`POST .../verify` or `POST .../reject`). This is the dual-human data-quality check.
2. **Clinical approver** — authorizes promotion (`POST .../approve`), which creates live clinical records. Every Track A batch requires this step regardless of batch type.
**Optional third gate:** For high-stakes `batchType` values, site configuration routes verify-pass batches to `awaiting_clinical_approval` instead of `verified`. The clinical approver then reviews after the verifier before promotion. See [Site configuration](#site-configuration) below.
| After verifier pass | Next status | Who calls `approve` |
|---|---|---|
| Site config: clinical sign-off **not** required for this `batchType` | `verified` | Clinical approver (or administrator) |
| Site config: clinical sign-off **required** for this `batchType` | `awaiting_clinical_approval` | Clinical approver (or administrator) |
Separation of duties is enforced at the service layer, not only in the UI. `enteredByUserId === currentUserId` blocks verify and approve actions with `409 SEPARATION_OF_DUTIES_VIOLATION`. Verifiers never call `approve`; clinical approvers never call `verify`.
### Site configuration
Per-site `ClinicalApprovalRequired` maps `batchType` to whether verify-pass routes to `awaiting_clinical_approval` (physician queue) vs `verified` (ready for immediate approver sign-off). Default portfolio configuration:
| batchType | Clinical sign-off after verify? |
|---|---|
| `patient_registration` | No |
| `allergy_update` | No |
| `encounter_summary` | Yes |
| `vitals_sheet` | Yes |
| `lab_results` | Yes |
| `medication_list` | Yes |
| `mixed` | Yes |
Administrators may override this map per deployment in `appsettings.json` (`SiteConfig.ClinicalApprovalRequired`).
---
## Domain Model
### DigitizationBatch
The unit of work for one digitization effort — typically one scanned document or one logical chart section (vitals sheet, lab report, admission face sheet).
| Field | Description |
|---|---|
| `id` | UUID |
| `status` | See state machine below |
| `batchType` | `patient_registration`, `encounter_summary`, `vitals_sheet`, `lab_results`, `medication_list`, `allergy_update`, `mixed` |
| `patientId` | Nullable until linked; may reference draft or live patient |
| `encounterDraftId` | Nullable; encounter context for vitals/labs |
| `documentRef` | MinIO object key for the scanned PDF/image |
| `documentSha256` | Content hash for integrity verification |
| `track` | `backfill` (Track A) or `live_capture` (Track B) |
| `enableRetroactiveAlerts` | Default `false`; if `true` on approval, promoted observations participate in alerting |
| `enteredByUserId` | Set on first draft save |
| `verifiedByUserId` | Set on verification pass |
| `approvedByUserId` | Set on final approval |
| `rejectionReason` | Nullable; required when status → `rejected` |
| `promotedAt` | Nullable; timestamp when live records created |
| `promotionEncounterId` | VigilCareClinical encounter ID after promotion |
| `supersedesBatchId` | Nullable; links a correction batch to the batch it replaces |
| `clinicianAttestation` | `true` for Track B batches attested at bedside |
### DraftPatient / DraftPatientUpdate
Structured patient fields extracted from paper — demographics, allergies, blood type, emergency contact, medications (for `medication_list` batches). On approval of a `patient_registration` or `allergy_update` batch, merges into VigilCareClinical `Patient` (create or patch). `medicationsJson` is stored on the digitization audit record in v1; medication rows are not promoted to a pharmacy module (out of scope).
### DraftEncounter
A clinical episode extracted from the chart: admission date, department, room/bed, admission reason, discharge diagnosis (if applicable). Promotes to VigilCareClinical `Encounter`.
### DraftObservation
A single measurable value: observation code, numeric value, unit, `recordedAt` (from chart, required), optional note. Subject to the same plausibility ranges as VigilCareClinical ingest. Never written to live `observations` until batch approval.
### DigitizationEvent
Append-only audit log entry for every state transition and field-level correction.
```json
{
"id": "uuid",
"batchId": "uuid",
"eventType": "uploaded | entry_started | submitted_for_verification | rejected | verified | awaiting_clinical_approval | approved | promoted | live_capture_attested | correction_requested",
"actorUserId": "uuid",
"occurredAt": "2026-06-22T14:30:00Z",
"metadata": {
"rejectionReason": "SpO2 value unclear on scan — decimal ambiguous",
"fieldsChanged": ["observations[2].value"]
}
}
```
### ScannedDocument
Stored in MinIO. Original paper is the legal source; the scan is the working reference for entry and verification. Retention: minimum 7 years (configurable per jurisdiction). Scanned documents are **never deleted** when a batch is rejected — only the draft is returned for correction.
---
## Batch Status State Machine
```
┌──────────────┐
│ uploaded │
└──────┬───────┘
│ assign / first save
▼
┌──────────────┐
┌──────────│ in_entry │◄─────────┐
│ └──────┬───────┘ │
│ │ submit │ reject
│ ▼ │
│ ┌──────────────┐ │
│ │ pending │─────────┘
│ │ verification │
│ └──────┬───────┘
│ │
│ verify fail │ verify pass
│ ─────────┤
│ │
│ site config │ site config
│ = false │ = true
│ ┌────────┴────────┐
│ ▼ ▼
│ ┌──────────────┐ ┌──────────────────────┐
│ │ verified │ │ awaiting_clinical │
│ └──────┬───────┘ │ _approval │
│ │ └──────────┬─────────────┘
│ └──────────┬──────────┘
│ │ clinical approver: approve
│ ▼
│ ┌──────────────┐
└────────────►│ approved │
└──────┬───────┘
│ promotion (sync or retry job)
▼
┌──────────────┐
│ promoted │ (terminal — live records exist)
└──────────────┘
Track B shortcut: live_capture creates a batch already in promoted — see Track B audit model.
```
**Allowed transitions (Track A):**
| From | To |
|---|---|
| `uploaded` | `in_entry` |
| `in_entry` | `pending_verification` |
| `pending_verification` | `verified`, `awaiting_clinical_approval`, `rejected` |
| `rejected` | `in_entry` |
| `verified` | `approved` |
| `awaiting_clinical_approval` | `approved`, `rejected` |
| `approved` | `promoted` |
| `promoted` | *(none — terminal)* |
Illegal transitions return `409` with a stable error code. A `promoted` batch cannot return to any earlier state. Corrections require a **new** batch referencing `supersedesBatchId`.
### Track B audit model
Track B is **not** a bypass of the batch model — it is a shortcut through the **workflow states**, not the audit unit. Every live-capture submission:
1. Creates a `DigitizationBatch` with `track: live_capture`, `batchType: vitals_sheet` (or appropriate type), `documentRef: "live-capture"`, and a synthetic `documentSha256` derived from clinician + encounter + timestamp (no scan file).
2. Writes `DraftObservation` rows for every entered value (audit trail).
3. Sets `enteredByUserId`, `verifiedByUserId`, and `approvedByUserId` to the attesting clinician (attestation replaces the dual-human gate for that batch only).
4. Inserts live `Observation` rows and outbox events in the **same transaction**, leaving the batch in `promoted` immediately.
5. Emits `DigitizationEvent` entries: `live_capture_attested`, then `promoted`.
This keeps digitization history, patient coverage stats, and Prometheus batch metrics consistent across both tracks.
---
## Features
---
### 1. Document Upload and Batch Creation
**Description:** Intake clerk scans or uploads a paper record. System stores the file in MinIO, computes SHA-256, creates a batch in `uploaded` state, and writes a `DigitizationEvent`.
**Endpoints:**
- `POST /api/v1/digitization-batches` — multipart upload: `file` (PDF, JPEG, PNG; max 25 MB), `batchType`, optional `patientId`, optional `track` (default `backfill`)
- `GET /api/v1/digitization-batches/:id` — batch detail with document presigned URL (15-minute expiry)
- `GET /api/v1/digitization-batches` — filter by `status`, `batchType`, `assignedTo`, `track`; paginated
- `PATCH /api/v1/digitization-batches/:id/assign` — assign to entry clerk (Redis lock prevents double-assignment)
**Validation:**
- Accepted MIME types: `application/pdf`, `image/jpeg`, `image/png`
- Reject empty files
- **Duplicate detection (when `patientId` is set):** reject if `documentSha256` matches an existing batch for the **same patient** within 24 hours (`409 DUPLICATE_DOCUMENT`) — prevents accidental double-scan of the same chart page
- **Cross-patient duplicates:** the same physical form scanned for two different patients is allowed (distinct patients, distinct batches). Operators may still see a warning in the UI if the SHA matches any batch site-wide (informational only in v1)
**Concepts practiced:** Object storage for immutable document artifacts, content-addressed deduplication, presigned URLs for secure document viewing without proxying binary through the API.
---
### 2. Draft Data Entry
**Description:** Entry clerk views the scan alongside structured form fields. Saves draft patient updates, encounter context, and observations. Batch moves to `in_entry` on first save.
**Endpoints:**
- `GET /api/v1/digitization-batches/:id/draft` — full draft payload: patient fields, encounter, observations[]
- `PUT /api/v1/digitization-batches/:id/draft/patient` — upsert draft patient demographics or updates
- `PUT /api/v1/digitization-batches/:id/draft/encounter` — upsert draft encounter fields
- `POST /api/v1/digitization-batches/:id/draft/observations` — add observation row
- `PUT /api/v1/digitization-batches/:id/draft/observations/:obsId` — edit observation
- `DELETE /api/v1/digitization-batches/:id/draft/observations/:obsId` — remove observation from draft
- `POST /api/v1/digitization-batches/:id/submit-for-verification` — validates completeness, transitions to `pending_verification`
**Required fields before submit (by batch type):**
| batchType | Required draft content |
|---|---|
| `patient_registration` | Full name, date of birth, sex; MRN generated on approval if new |
| `encounter_summary` | Linked patient; encounter with admission date, department, admission reason |
| `vitals_sheet` | Linked patient, encounter context, ≥1 observation with `recordedAt` |
| `lab_results` | Linked patient, encounter, ≥1 lab observation code, `recordedAt` |
| `medication_list` | Linked patient; `medicationsJson` with ≥1 entry **or** explicit `noActiveMedications: true` |
| `allergy_update` | Linked patient, allergies list (may be empty with explicit `noKnownAllergies: true`) |
| `mixed` | Linked patient, encounter context, and **at least one** of: ≥1 observation with `recordedAt`, or complete encounter summary (admission date + department + admission reason) |
**Concurrent draft editing:** Redis assignment lock prevents double-assignment at intake (`PATCH .../assign`). Draft saves require the acting user to match `enteredByUserId` (or hold the `Administrator` role); otherwise `409 BATCH_NOT_ASSIGNED`. v1 does not use field-level optimistic locking — within an assigned session, last write wins. Administrators may reassign via work-queue tools, which clears the Redis lock and updates `enteredByUserId`.
**Plausibility validation:** Reuse VigilCareClinical observation plausibility ranges at draft save time. Out-of-range values return `422 OBSERVATION_OUT_OF_PLAUSIBLE_RANGE` with the same error shape — catch decimal errors (5.2 vs 52) before verification.
**UI requirement (digitization workstation):** Side-by-side layout — scan viewer (zoom, pan, rotate) on the left; structured entry form on the right. Field-level "verified" checkbox for verifier pass (stored in draft metadata, not live record).
**Concepts practiced:** Draft vs live data separation, optimistic UI with server validation, batch-type-driven validation rules.
---
### 3. Verification and Rejection
**Description:** Verifier reviews entry against the scan. Can approve field-by-field or reject the entire batch with a mandatory reason. Verifier cannot be the entry clerk.
On verify pass: status → `verified` or `awaiting_clinical_approval` per [site configuration](#site-configuration). On verify fail: status → `rejected` (returns to entry queue).
**Endpoints:**
- `GET /api/v1/work-queue/verification` — batches in `pending_verification`, sorted by `submittedAt ASC`
- `GET /api/v1/work-queue/clinical-approval` — batches in `awaiting_clinical_approval`, sorted by `submittedAt ASC`
- `POST /api/v1/digitization-batches/:id/verify` — body: `{ "fieldChecks": [{ "fieldPath": "observations[0].value", "passed": true }], "passed": true }`
- `POST /api/v1/digitization-batches/:id/reject` — body: `{ "reason": "..." }` → status `rejected`, notifies entry clerk
Reject is allowed from `pending_verification` (verifier; separation of duties applies) or `awaiting_clinical_approval` (clinical approver; separation of duties does **not** apply — approver may not have been the entry clerk by role design).
**Concepts practiced:** Separation of duties enforcement, work-queue patterns, structured rejection loops.
---
### 4. Approval and Promotion to VigilCareClinical
**Description:** Final approval triggers an atomic promotion: draft records become live Patient / Encounter / Observation rows in VigilCareClinical (same database in integrated deployment, or HTTP calls to VigilCareClinical API in split deployment). Batch status → `promoted`.
**Endpoints:**
- `POST /api/v1/digitization-batches/:id/approve` — requires `verified` or `awaiting_clinical_approval`; restricted to `ClinicalApprover` or `Administrator`; Idempotency-Key supported
- `GET /api/v1/digitization-batches/:id/promotion-result` — live IDs created: `patientId`, `encounterId`, `observationIds[]`
**Promotion transaction sequence:**
1. Begin database transaction (or saga with compensating actions in split deployment)
2. Create or update `Patient` in VigilCareClinical (assign MRN if new)
3. Create or match `Encounter` (open as `active` or `discharged` based on draft)
4. Insert each `DraftObservation` as live `Observation` with `source: digitization_backfill` or `source: live_capture`
5. Write outbox events for each observation (Kafka pipeline activates **only if** `enableRetroactiveAlerts` or `track: live_capture`)
6. Update batch status → `promoted`, set `promotedAt`, write `DigitizationEvent`
7. Commit
**Default alert behavior:**
| track | enableRetroactiveAlerts | Alert pipeline |
|---|---|---|
| `backfill` | `false` (default) | Observations stored; no alert evaluation |
| `backfill` | `true` | Full VigilCareClinical alert path |
| `live_capture` | n/a | Full alert path immediately |
**Concepts practiced:** Outbox pattern for promotion side effects, idempotent promotion, configurable clinical safety policy for historical data.
---
### 5. Corrections and Supersession
**Description:** Approved records are not silently edited. A correction creates a new batch with `supersedesBatchId` pointing to the original. Correction goes through the full entry → verify → approve cycle. On promotion, erroneous live observations are marked `superseded` (append-only — not deleted).
**Endpoints:**
- `POST /api/v1/digitization-batches` — body includes optional `supersedesBatchId`
- `GET /api/v1/patients/:id/digitization-history` — all batches for a patient with promotion status
**Concepts practiced:** Immutable clinical audit trail, correction-as-new-batch pattern (same as national digital services land registry approach).
---
### 6. Live Capture (Track B)
**Description:** Credentialed clinician enters vitals or labs at point of care on a tablet. No verification queue. Each submission creates an audit `DigitizationBatch` (see [Track B audit model](#track-b-audit-model)), writes draft observations for traceability, and promotes live observations synchronously with full alert evaluation.
**Endpoints:**
- `POST /api/v1/live-capture/encounters/:encounterId/observations` — body: observation fields + `clinicianAttestation: true` + password re-confirm or PIN
- `POST /api/v1/live-capture/encounters` — open encounter + initial vitals in one request (outpatient workflow)
**Validation:** Requires `Role: Clinician`. Creates batch + draft observations + live observations in one transaction; batch lands in `promoted` immediately. Returns VigilCareClinical observation IDs and any synchronous critical alerts generated.
**Concepts practiced:** Lighter gate for real-time care vs heavy gate for backfill; same underlying observation schema.
---
### 7. Patient Registry (Draft and Live)
**Description:** Search and link batches to patients. Support new patient registration through the draft pipeline.
**Endpoints:**
- `GET /api/v1/patients/search?q=` — search live VigilCareClinical patients by MRN or name
- `POST /api/v1/patients/draft` — create draft-only patient (no MRN until approval)
- `GET /api/v1/patients/:id/summary` — live patient + pending draft batches + digitization coverage stats
**Digitization coverage stat:** `approvedBatchCount / estimatedTotalBatches` — optional manual `estimatedChartSections` per patient for progress tracking.
---
### 8. Work Queues and Operational Dashboard
**Description:** Supervisors monitor backlog, assignment, and throughput.
**Endpoints:**
- `GET /api/v1/work-queue/entry` — batches awaiting or in entry
- `GET /api/v1/work-queue/clinical-approval` — batches awaiting physician sign-off after verification
- `GET /api/v1/work-queue/overview` — counts by status, average time-in-queue, reject rate
- `GET /api/v1/digitization-batches/:id/events` — cursor-paginated audit trail
**Metrics (Prometheus):**
- `digitization_batches_by_status` (gauge)
- `digitization_promotion_duration_seconds` (histogram)
- `digitization_rejection_total` (counter)
- `digitization_queue_age_seconds` (gauge — oldest pending verification)
---
### 9. Authentication and Audit
**Description:** JWT auth with role claims. Every state transition writes a `DigitizationEvent`. Document access logged.
**Endpoints:**
- `POST /api/v1/auth/login`
- `GET /api/v1/auth/me`
**Audit requirements:**
- Who viewed a scan and when
- Who changed which draft field (field-level diff in event metadata on save)
- Who approved promotion and which live record IDs were created
---
## Digitization Workstation UI
Separate Vue 3 SPA or Razor-hosted frontend. Four primary views:
| View | User | Purpose |
|---|---|---|
| **Intake** | Intake clerk | Upload, assign patient, print MRN label |
| **Entry** | Data entry clerk | Side-by-side scan + form |
| **Verification** | Verifier | Side-by-side with field checkboxes, approve/reject |
| **Queue dashboard** | Supervisor | Backlog, reject rate, clerk throughput |
Not a full EMR UI. No clinical alerting views — those remain in VigilCareClinical's ward dashboard.
---
## Data Storage
| Store | Purpose |
|---|---|
| **PostgreSQL** | Draft tables, batch metadata, digitization events, user/role data. Shares database with VigilCareClinical in integrated deployment. |
| **MinIO** | Scanned PDFs and images; content-addressed keys `scans/{year}/{month}/{batchId}/{sha256}.pdf` |
| **Redis** | Batch assignment locks (`SET batch:assign:{id} NX EX 3600`). Work-queue counters are derived from PostgreSQL queries in v1 (no Redis counter cache required) |
---
## Integration Contract with VigilCareClinical
**v1 deployment:** integrated PostgreSQL — promotion writes directly to VigilCareClinical tables in a single transaction. Split deployment (Records service calling VigilCareClinical REST + `PromotionRetryService`) remains supported as an alternate topology; see Phase 8.
On promotion, VigilCare Records writes to the same tables VigilCareClinical owns:
| Draft entity | Live entity | Notes |
|---|---|---|
| `DraftPatient` | `patients` | MRN generated via existing `GenerateMrnAsync` logic |
| `DraftEncounter` | `encounters` | Status from draft; `roomBed`, `admissionReason` mapped |
| `DraftObservation` | `observations` | Same `observationCode`, `value`, `unit`, `recordedAt`; adds `metadata.source` |
Observation codes must match VigilCareClinical's catalog: `HEART_RATE`, `TEMP_C`, `BP_SYSTOLIC`, `BP_DIASTOLIC`, `RESP_RATE`, `SPO2`, `POTASSIUM_MEQ_L`, `GLUCOSE_MG_DL`, `WBC_K_UL`, `LACTATE_MMOL_L`, etc.
If VigilCareClinical is unreachable in split deployment, batch remains `approved` and a background promotion retry job runs with exponential backoff. Batch does not revert to draft.
---
## Acceptance Criteria
| Criterion | Verification |
|---|---|
| Separation of duties | Entry clerk cannot verify or approve own batch — `409` |
| Draft isolation | Draft observations never appear in VigilCareClinical alert queries or ward dashboard |
| Promotion atomicity | Partial promotion (patient created, observations failed) never committed |
| Idempotent approval | Duplicate `Idempotency-Key` on approve returns same result, no duplicate observations |
| Rejection loop | Rejected batch returns to entry; resubmit reaches verification again |
| Backfill alert suppression | Default backfill promotion creates observations with zero alerts |
| Live capture alert path | Track B critical potassium triggers synchronous alert in VigilCareClinical |
| Audit completeness | Every status transition has a `DigitizationEvent` with actor and timestamp |
| Document immutability | Scan object in MinIO not modified or deleted on reject/correct |
| Plausibility at draft | Value 520 for potassium rejected at draft save, not at promotion |
| Clinical approval routing | `vitals_sheet` verify-pass → `awaiting_clinical_approval` when site config requires it |
| Track B audit batch | Live capture creates `DigitizationBatch` in `promoted` with draft observations and events |
| Draft assignment guard | Unassigned clerk receives `409 BATCH_NOT_ASSIGNED` on draft save |
| Cross-patient duplicate scan | Same SHA for two different patients allowed; same patient within 24h rejected |
---
## Build Order
| Phase | Focus |
|---|---|
| 1 | Schema, auth, roles, batch CRUD, MinIO upload, status machine |
| 2 | Draft entry API (patient, encounter, observations), submit-for-verification |
| 3 | Verification, rejection, separation of duties, work queues |
| 4 | Promotion service → VigilCareClinical live tables, outbox integration, idempotency |
| 5 | Corrections / supersession, patient digitization history |
| 6 | Track B live capture with clinician attestation |
| 7 | Digitization workstation UI (entry + verification side-by-side) |
| 8 | Prometheus metrics, supervisor dashboard, promotion retry job |
| 9 | Seed data, E2E verification script, clinical scenario documentation |
---
## Step-by-Step Guide
Complete phases in order. Promotion (Phase 4) must not be built until the draft state machine and separation of duties are correct — debugging promotion bugs alongside workflow bugs is painful.
---
### Phase 1 — Schema, Upload, and Status Machine
**What to do:**
1. Create `digitization_batches`, `draft_patients`, `draft_encounters`, `draft_observations`, `digitization_events` tables.
2. Implement batch status machine with explicit transition matrix; illegal transitions → `409`.
3. Wire MinIO upload with SHA-256 computation and presigned GET URLs.
4. Seed two users per role (entry, verifier, clinician) for separation-of-duties testing.
5. Implement JWT auth with role claims.
**Why:**
The status machine is the backbone. Getting transitions wrong means drafts leak into live data or approved batches get re-edited. Test every illegal transition before building entry forms.
---
### Phase 2 — Draft Entry
**What to do:**
1. Implement draft CRUD endpoints and batch-type validation on submit.
2. Port plausibility validator from VigilCareClinical (shared library or duplicated with comment linking source).
3. Write integration tests: incomplete vitals batch cannot submit; plausible observations save; implausible rejected.
**Why:**
Plausibility at draft save prevents the most common digitization error — decimal misplacement — from ever reaching verification.
---
### Phase 3 — Verification and Rejection
**What to do:**
1. Implement verification and rejection endpoints with separation-of-duties checks.
2. Build work-queue endpoints sorted by `submittedAt`.
3. Test: entry clerk A submits → verifier A attempts verify → `409`; verifier B succeeds.
**Why:**
Separation of duties is a clinical trust requirement, not a nice-to-have. Enforce in the service layer from day one.
---
### Phase 4 — Promotion
**What to do:**
1. Implement approval endpoint and promotion transaction against VigilCareClinical tables.
2. Wire outbox events for observations where alerting is enabled.
3. Implement `enableRetroactiveAlerts` flag — default `false`.
4. Test full path: upload → entry → verify → approve → observations in live table → zero alerts for backfill default.
5. Test idempotency: approve twice with same key → one set of live rows.
**Why:**
This phase connects Records to Clinical. Run against a VigilCareClinical Phase 2 environment where synchronous critical alerting exists, and verify Track B vs Track A behavior explicitly.
---
### Phase 5 — Corrections
**What to do:**
1. Add `supersedesBatchId` and supersession logic on promotion.
2. Mark superseded live observations inactive (soft flag, not delete).
3. Test: wrong potassium promoted → correction batch → new value live, old value superseded in audit.
---
### Phase 6 — Live Capture (Track B)
**What to do:**
1. Implement clinician attestation endpoint bypassing verification.
2. Test critical value entered via live capture → alert fires before response returns.
---
### Phases 7–9 — UI, Observability, Documentation
Build the side-by-side workstation UI. Add Prometheus metrics and a supervisor queue view. Write `docs/digitization-workstation-guide.md` and an E2E script `./scripts/run-vigilcare-records-verification.sh`.
---
## Deployment Notes (Small Island Context)
- **Single-site tenant:** One hospital or health district per deployment. No cross-island federation in v1.
- **Offline intake (optional extension):** Scan and draft entry on a local server; promotion queued until uplink to central VigilCareClinical returns. Aligns with VigilCareClinical climate-resilience Phases 20–24 — Records gateway can share the same ward-first sync pattern.
- **Staffing reality:** Same person may hold entry and intake roles, but **never** entry and verifier on the same batch. System enforces this even when staff roster is small.
- **Paper originals:** Scanned document is the working copy; physical chart remains legal original until jurisdiction defines otherwise. README must state this explicitly.
---
## Success Metrics (Operational)
| Metric | Target (6 months post go-live) |
|---|---|
| Charts with ≥1 approved batch | 80% of active patients |
| Average verification turnaround | < 24 hours |
| Rejection rate | < 15% (indicates entry quality or scan quality issues if higher) |
| Live capture share of new observations | Trending up month-over-month |
| VigilCareClinical alerts from live capture | ≥1 demonstrated critical-value workflow per site |
---
## Architecture Decisions
| Decision | Choice (v1) | Rationale |
|---|---|---|
| **Integrated DB vs split API** | Integrated PostgreSQL, shared instance, separate schemas | Atomic promotion in one transaction; simpler to build and demo; split path documented for production hardening |
| **Clinical approver routing** | Site config map (see [Site configuration](#site-configuration)); default requires physician queue for encounter/vitals/labs/medications/mixed | High-stakes chart sections get an extra gate; registration and allergy-only updates stay verifier → approver |
| **Retroactive alerts** | `enableRetroactiveAlerts: false` default on backfill | Prevents alert storms from historical critical values; opt-in per batch for facilities that accept the risk |
| **MRN issuance** | Locally generated MRN via VigilCareClinical `GenerateMrnAsync` | National health ID integration deferred per deployment |
## Open Questions (per deployment)
1. **Retroactive alerts policy:** Will the Ministry of Health allow backfilled critical values to trigger pages, or is storage-only the mandated default?
2. **MRN issuance:** National health ID integration vs locally generated MRN when a national registry becomes available?
3. **Split deployment:** When does the site require Records and Clinical in separate services (saga + retry) vs integrated database?
---
## References
- [vigilcare-clinical-api-prd.md](vigilcare-clinical-api-prd.md) — downstream alerting and observation ingest
- [Completed/VigilCareClinicalAPI/VigilCare-Partner-Brief.md](Completed/VigilCareClinicalAPI/VigilCare-Partner-Brief.md) — clinical positioning and scope boundaries
- [Completed/national-digital-services-architecture.md](Completed/national-digital-services-architecture.md) — scan-and-verify pattern for paper-to-digital government services