feature: Live Capture (Track B)

This commit is contained in:
voltsrage
2026-06-27 04:23:51 +08:00
parent 01000a2489
commit 88e70b3dbe
30 changed files with 4547 additions and 23 deletions
+129 -17
View File
@@ -2,7 +2,7 @@
A clinical records intake backend built with ASP.NET Core 8, PostgreSQL, MinIO, and Redis. The domain models the scan-to-approved lifecycle at the center of any paper chart digitization system: scanned document upload, human data entry, dual-human verification with separation of duties, site-configurable clinical approval routing, atomic promotion to live clinical tables, and governed correction via supersession.
**Implementation status:** Five planned phases are complete through Phase 5 — from schema, authentication, and batch CRUD through draft data entry, verification/rejection with separation of duties, approval with atomic promotion to VigilCareClinical live tables, and correction batches that supersede erroneous promoted observations without silent edits. See [Implemented Phases](#implemented-phases) for the full breakdown.
**Implementation status:** Six planned phases are complete through Phase 6 — from schema, authentication, and batch CRUD through draft data entry, verification/rejection with separation of duties, approval with atomic promotion to VigilCareClinical live tables, correction batches that supersede erroneous promoted observations without silent edits, and Track B live capture with clinician attestation and synchronous critical alerting. See [Implemented Phases](#implemented-phases) for the full breakdown.
## Domain Model — How It Maps to a Real Clinical System
@@ -56,6 +56,7 @@ Append-only audit log entry for every state transition, field-level correction,
- **Corrections and Supersession** — approved live observations are never silently edited; a correction uploads a new batch with `supersedesBatchId`, goes through the full entry → verify → approve cycle, and on promotion marks the original batch's `live_observations` rows as `is_superseded` (append-only — never deleted); `422 SUPERSEDED_BATCH_NOT_PROMOTED`, `409 BATCH_ALREADY_SUPERSEDED`, and `404 SUPERSEDED_BATCH_NOT_FOUND` guard invalid supersession targets; correction batches inherit patient context and reuse the original promotion encounter
- **Patient Digitization History** — `GET /patients/:id/digitization-history` returns all batches for a patient with correction chain metadata (`isCorrection`, `hasBeenSuperseded`, `supersededByBatchId`), live vs superseded observation counts, summary totals, and per-batch audit trails; `404 PATIENT_HISTORY_NOT_FOUND` when no batches exist for the patient
- **Work Queues** — `GET /work-queue/verification` (batches in `PENDING_VERIFICATION`, sorted by submission time ASC); `GET /work-queue/entry` (batches in `UPLOADED`, `IN_ENTRY`, or `REJECTED`); `GET /work-queue/clinical-approval` (batches in `AWAITING_CLINICAL_APPROVAL`); role-restricted access
- **Live Capture (Track B)** — `POST /live-capture/encounters/{encounterId}/observations` and `POST /live-capture/encounters` for credentialed clinicians entering vitals at bedside; clinician attestation + password re-confirm replaces the dual-human verification gate; observations promote synchronously to live tables in a single transaction with `source = live_capture`; critical threshold evaluation runs before the response returns, with inline `criticalAlert` per observation, committed `ClinicalAlert` rows, and `observation.recorded` / `alert.generated` outbox events; `422 ATTESTATION_REQUIRED`, `422 PASSWORD_CONFIRM_INVALID`, `422 EMPTY_OBSERVATIONS`, `409 ENCOUNTER_NOT_ACTIVE`, `409 ACTIVE_ENCOUNTER_EXISTS`
- **Batch Status Machine** — explicit transition matrix enforced in the service layer; illegal transitions return `409`; `PROMOTED` is terminal — corrections require a new batch with `supersedesBatchId`
- **JWT Authentication** — `POST /auth/login` returns access token (15 min) and refresh token (7 days); `POST /auth/refresh` rotates tokens; `POST /auth/logout` revokes server-side; `GET /auth/me` returns authenticated user profile; BCrypt password hashing
- **Role-Based Access** — six roles (`INTAKE_CLERK`, `DATA_ENTRY_CLERK`, `VERIFIER`, `CLINICAL_APPROVER`, `CLINICIAN`, `ADMINISTRATOR`) with role-based endpoint authorization; twelve seeded demo users (two per role)
@@ -85,6 +86,8 @@ HTTP request
├── IdempotencyService (Idempotency-Key storage + replay for safe promotion retries)
├── MrnGenerator (PostgreSQL sequence-backed MRN generation: VCR-000001)
├── WorkQueueService (verification, entry, clinical approval queues)
├── AttestationService (clinician role + password re-confirm for live capture)
├── LiveCaptureService (Track B synchronous promotion + critical alert evaluation)
├── DocumentStorageService (MinIO upload, SHA-256, presigned URLs)
├── PlausibilityValidator (per-code numeric range guard)
├── PostgreSQL (EF Core — writes, keyed reads, clinical schema for live tables)
@@ -120,7 +123,7 @@ HTTP request
|---|---|
| Server | ASP.NET Core 8 (.NET 8.0) |
| Database | PostgreSQL 16 with EF Core 8 (code-first migrations) |
| Cache / locking | Redis 7 (batch assignment locks) |
| Cache / locking | Redis 7 (batch assignment locks, alert threshold cache for live capture) |
| Object storage | MinIO (scanned documents — PDF, JPEG, PNG) |
| Authentication | JWT Bearer (Microsoft.AspNetCore.Authentication.JwtBearer) |
| Password hashing | BCrypt.Net-Next |
@@ -142,6 +145,7 @@ VigilCareRecordsAPI/
│ ├── DigitizationBatchesController.cs # Batch CRUD, document upload, assignment
│ ├── DraftController.cs # Draft data entry: patient, encounter, observations, submit
│ ├── PatientsController.cs # Patient digitization history with correction chain
│ ├── LiveCaptureController.cs # Track B live capture: record vitals, open encounter + vitals
│ ├── VerificationController.cs # Batch verification and rejection with separation of duties
│ └── WorkQueueController.cs # Work queues: verification, entry, clinical approval
├── Domain/
@@ -150,7 +154,9 @@ VigilCareRecordsAPI/
│ │ │ ├── Patient.cs # Live patient record with MRN (promoted from draft)
│ │ │ ├── Encounter.cs # Live encounter (promoted from draft)
│ │ │ ├── Observation.cs # Live observation with source traceability (batchId, draftObsId)
│ │ │ ├── OutboxEvent.cs # Transactional outbox for downstream consumers (Kafka)
│ │ │ ├── OutboxEvent.cs # Transactional outbox for downstream consumers (eventType, aggregateType, payloadJson)
│ │ │ ├── AlertThreshold.cs # Critical/warning bounds per observation code (live capture alerting)
│ │ │ ├── ClinicalAlert.cs # Synchronous critical alerts (AlertType, Severity, Details)
│ │ │ └── IdempotencyRecord.cs # Idempotency-Key storage for safe promotion retries
│ │ ├── Draft/
│ │ │ ├── DraftPatient.cs # Structured patient demographics from paper chart
@@ -171,9 +177,12 @@ VigilCareRecordsAPI/
│ ├── UserRole.cs # IntakeClerk, DataEntryClerk, Verifier, ClinicalApprover, Clinician, Administrator
│ ├── DigitizationEventType.cs # 18 event types covering full lifecycle + corrections + promotion retry
│ ├── BloodType.cs # A+, A-, B+, B-, AB+, AB-, O+, O-
── Department.cs # Clinical departments
── Department.cs # Clinical departments
│ ├── AlertType.cs # Critical/warning alert types (ported from VigilCareClinical)
│ ├── AlertSeverity.cs # WARNING, CRITICAL
│ └── AlertStatus.cs # OPEN, ACKNOWLEDGED, RESOLVED, ESCALATED
├── Services/
│ ├── Interfaces/ # IAuthService, IBatchService, IDraftService, IVerificationService, IWorkQueueService, IDocumentStorageService, IPromotionService, IIdempotencyService, IMrnGenerator, IDigitizationHistoryService
│ ├── Interfaces/ # IAuthService, IBatchService, IDraftService, IVerificationService, IWorkQueueService, IDocumentStorageService, IPromotionService, IIdempotencyService, IMrnGenerator, IDigitizationHistoryService, ILiveCaptureService, IAttestationService
│ ├── AuthService.cs # Login (BCrypt verify), JWT generation, refresh rotation, logout, audit logging
│ ├── BatchService.cs # Batch CRUD, status machine transitions, Redis assignment lock, duplicate detection, supersession validation on create
│ ├── DraftService.cs # Draft CRUD with plausibility validation, batch-type completeness on submit (relaxed for correction batches)
@@ -184,6 +193,8 @@ VigilCareRecordsAPI/
│ ├── MrnGenerator.cs # PostgreSQL sequence-backed MRN generation (VCR-000001)
│ ├── WorkQueueService.cs # Verification, entry, and clinical approval work queues
│ ├── DocumentStorageService.cs # MinIO upload with SHA-256, presigned URL generation
│ ├── AttestationService.cs # Clinician role + password re-confirm for live capture
│ ├── Interfaces/LiveCaptureService.cs # Track B synchronous promotion + critical alert evaluation
│ └── PlausibilityValidator.cs # Per-code numeric range guard (reused from VigilCareClinical)
├── Configurations/
│ ├── JwtOptions.cs # Issuer, audience, signing key, access/refresh token expiration
@@ -197,18 +208,19 @@ VigilCareRecordsAPI/
│ ├── DraftEncounter/ # DraftEncounterDto, UpsertDraftEncounterRequest
│ ├── DraftObservation/ # DraftObservationDto, CreateDraftObservationRequest, UpdateDraftObservationRequest
│ ├── WorkQueue/ # WorkQueueResponse, WorkQueueItemResponse
│ ├── LiveCapture/ # LiveCaptureResponse, RecordObservationsRequest, ThresholdCacheEntry, ...
│ └── Common/ # PagedResult
├── Data/
│ ├── AppDbContext.cs # EF Core context — entity configs, indexes, constraints
│ ├── Configurations/
│ │ ├── Clinical/ # PatientConfiguration, EncounterConfiguration, ObservationConfiguration, OutboxEventConfiguration, IdempotencyRecordConfiguration
│ │ ├── Clinical/ # PatientConfiguration, EncounterConfiguration, ObservationConfiguration, OutboxEventConfiguration, AlertThresholdConfiguration, ClinicalAlertConfiguration, IdempotencyRecordConfiguration
│ │ ├── Draft/ # DraftPatientConfiguration, DraftEncounterConfiguration, DraftObservationConfiguration
│ │ ├── LiveEncounterConfiguration.cs # live_encounters table mapping
│ │ ├── LiveObservationConfiguration.cs # live_observations with supersession columns and partial index
│ │ └── ... # DigitizationBatch, DigitizationEvent, ScannedDocument, User, RefreshToken, AuthAuditEvent configs
│ ├── Seed/
│ │ └── DataSeeder.cs # Seeds 12 demo users (two per role) on startup
│ └── Migrations/ # InitialCreate through AddLiveObservationAndLiveEncounter
│ └── Migrations/ # InitialCreate through AddAlertThresholdsAndClinicalAlerts
├── Common/
│ ├── ApiResponse.cs # { success, statusCode, data, error } envelope
│ └── Exceptions/
@@ -230,6 +242,7 @@ tests/
├── VerificationTests.cs # Verification, rejection, separation of duties, clinical approval routing
├── PromotionTests.cs # Approval, atomic promotion, idempotency, separation of duties, retroactive alerts, patient dedup
├── CorrectionSupersessionTests.cs # Correction batch supersession, validation guards, patient digitization history
├── LiveCaptureIntegrationTests.cs # Track B attestation, synchronous promotion, critical alerts, open encounter workflow
├── Fixtures/
│ ├── ApiFixture.cs # WebApplicationFactory with PostgreSQL, Redis, MinIO containers
│ └── DatabaseCollection.cs # Shared test collection
@@ -245,7 +258,8 @@ scripts/
├── run-vigilcare-records-phase-2-verification.sh # Phase 2 — draft entry, plausibility, submit-for-verification
├── run-vigilcare-records-phase-3-verification.sh # Phase 3 — verification, rejection, separation of duties, work queues
├── run-vigilcare-records-phase-4-verification.sh # Phase 4 — approval, atomic promotion, idempotency, patient dedup, outbox events
── run-vigilcare-records-phase-5-verification.sh # Phase 5 — corrections, supersession, digitization history, audit trail
── run-vigilcare-records-phase-5-verification.sh # Phase 5 — corrections, supersession, digitization history, audit trail
└── run-vigilcare-records-phase-6-verification.sh # Phase 6 — live capture, attestation, synchronous alerts, outbox events
docs/
├── plans/ # Phase 19 implementation and verification guides
@@ -343,11 +357,11 @@ The person who enters data cannot verify their own entry. This is enforced in th
### Two-Track Workflow (Backfill vs Live Capture)
Track A (backfill) is the full pipeline for historical charts: scan → entry → verification → clinical approval → promotion. Track B (live capture) is for credentialed clinicians entering vitals at bedside — attestation replaces the dual-human gate. Both tracks create `DigitizationBatch` records with full audit trails, keeping metrics and coverage stats consistent.
Track A (backfill) is the full pipeline for historical charts: scan → entry → verification → clinical approval → promotion. Track B (live capture) is for credentialed clinicians entering vitals at bedside via `POST /live-capture/encounters/{encounterId}/observations` or `POST /live-capture/encounters` — clinician attestation + password re-confirm replaces the dual-human gate, and observations promote synchronously with critical alerting before the response returns. Both tracks create `DigitizationBatch` records with full audit trails, keeping metrics and coverage stats consistent.
### Redis for Batch Assignment Locking
### Redis for Batch Assignment Locking and Alert Threshold Cache
Redis serves one purpose in this project: preventing double-assignment of batches to entry clerks. `SET batch:assign:{id} NX EX 3600` acquires an exclusive lock with a 1-hour TTL. Work-queue counters are derived from PostgreSQL queries, not Redis counters.
Redis serves two purposes in this project: (1) preventing double-assignment of batches to entry clerks via `SET batch:assign:{id} NX EX 3600`, and (2) caching alert threshold definitions for synchronous critical evaluation during live capture (`threshold:{observationCode}`). Work-queue counters are derived from PostgreSQL queries, not Redis counters.
### Integrated Database Deployment
@@ -376,7 +390,7 @@ docker compose up -d
|---|---|---|
| PostgreSQL 16 | 5437 | Database: `vigilcare_records`, user: `postgres`, password: `password` |
| Redis 7 | 6383 | No auth |
| Seq | 5346 | UI at `http://localhost:5346` |
| Seq | 5346 | UI at `http://localhost:5346`, login: `admin` / `seqadmin` |
| MinIO | 9012 (S3 API), 9013 (console) | login: `minioadmin` / `minioadmin` |
### Install and Run
@@ -407,6 +421,7 @@ Integration tests use `WebApplicationFactory` with PostgreSQL, Redis, and MinIO
| `VerificationTests` | 3 | Verification, rejection, separation of duties enforcement, clinical approval routing |
| `PromotionTests` | 4 | Approval, atomic promotion to live tables, idempotency replay, separation of duties, retroactive alert policy, patient deduplication, MRN generation |
| `CorrectionSupersessionTests` | 5 | Correction batch supersession, validation guards (non-promoted, already superseded), patient digitization history, unknown patient 404 |
| `LiveCaptureIntegrationTests` | 6 | Attestation and password confirm, synchronous promotion, critical low/high potassium alerts, mixed-batch alerting, open encounter + vitals, role and validation guards, audit events |
### Verification Scripts
@@ -418,6 +433,7 @@ With the API running (`dotnet run`) and Docker Compose up:
./scripts/run-vigilcare-records-phase-3-verification.sh # Phase 3 — verification, rejection, separation of duties
./scripts/run-vigilcare-records-phase-4-verification.sh # Phase 4 — approval, promotion, idempotency, patient dedup
./scripts/run-vigilcare-records-phase-5-verification.sh # Phase 5 — corrections, supersession, digitization history
./scripts/run-vigilcare-records-phase-6-verification.sh # Phase 6 — live capture, attestation, critical alerts
```
---
@@ -668,6 +684,72 @@ All work queue endpoints support pagination via `?page=1&pageSize=20`.
| 200 | History returned |
| 404 | No digitization batches for patient (`PATIENT_HISTORY_NOT_FOUND`) |
### Live Capture (Track B)
| Method | Path | Auth | Description |
|---|---|---|---|
| POST | `/live-capture/encounters/{encounterId}/observations` | Clinician | Record observations against an existing active encounter; promotes synchronously with inline critical alerts |
| POST | `/live-capture/encounters` | Clinician | Open a new encounter and record initial vitals in one request (outpatient workflow) |
**Request body (both endpoints):**
| Field | Type | Required | Description |
|---|---|---|---|
| `observations` | array | yes | One or more observation objects (see below) |
| `clinicianAttestation` | bool | yes | Must be `true` — clinician attests values are accurate |
| `passwordConfirm` | string | yes | Re-enter password to confirm identity |
**Open encounter only — additional fields:**
| Field | Type | Required | Description |
|---|---|---|---|
| `patientId` | Guid | yes | Existing patient ID |
| `department` | string | yes | e.g. `Outpatient Clinic`, `Internal Medicine` |
| `roomBed` | string | no | Ward/bed assignment |
| `admissionReason` | string | no | Reason for visit or admission |
**Observation object:**
| Field | Type | Required | Description |
|---|---|---|---|
| `observationCode` | string | yes | e.g. `HEART_RATE`, `POTASSIUM_MEQ_L`, `TEMP_C` |
| `value` | decimal | yes | Numeric measurement |
| `unit` | string | yes | Unit of measure |
| `recordedAt` | DateTimeOffset | yes | When the measurement was taken |
| `note` | string | no | Optional note |
**Response (`LiveCaptureResponse`):**
| Field | Type | Description |
|---|---|---|
| `batchId` | Guid | `DigitizationBatch` created in `PROMOTED` status with `track = LIVE_CAPTURE` |
| `encounterId` | Guid | Live encounter ID |
| `observations` | array | Promoted observations with `liveObservationId` and optional inline `criticalAlert` |
| `criticalAlertCount` | int | Number of synchronous critical alerts generated |
| `promotedAt` | DateTimeOffset | Promotion timestamp |
**Inline critical alert object (`criticalAlert` on each observation):**
| Field | Type | Description |
|---|---|---|
| `alertId` | Guid | Committed `ClinicalAlert` row ID |
| `severity` | string | `CRITICAL` |
| `thresholdBound` | string | `CRITICAL_LOW` or `CRITICAL_HIGH` |
| `thresholdValue` | decimal | Breached threshold value |
| `message` | string | Human-readable breach description |
**Status codes:**
| Code | Meaning |
|---|---|
| 201 | Observations promoted; critical alerts (if any) committed before response |
| 403 | Caller lacks `CLINICIAN` role |
| 404 | Patient or encounter not found |
| 409 | Encounter not active (`ENCOUNTER_NOT_ACTIVE`); patient already has active encounter (`ACTIVE_ENCOUNTER_EXISTS`) |
| 422 | Attestation false (`ATTESTATION_REQUIRED`); wrong password (`PASSWORD_CONFIRM_INVALID`); empty observations list (`EMPTY_OBSERVATIONS`) |
Track B still creates a full audit trail: each submission writes a `DigitizationBatch` (status `PROMOTED`, `documentRef = "live-capture"`), draft observation rows, `live_capture_attested` and `promoted` digitization events, live `Observation` rows with `source = live_capture`, and outbox events for downstream alerting.
---
## Data Models
@@ -759,7 +841,7 @@ uploadedAt DateTimeOffset
```
id Guid PK
batchId Guid FK → DigitizationBatch
eventType string uploaded | entry_started | submitted_for_verification | rejected | verified | awaiting_clinical_approval | approved | promoted | correction_uploaded | correction_promoted | superseded | ...
eventType string uploaded | entry_started | submitted_for_verification | rejected | verified | awaiting_clinical_approval | approved | promoted | live_capture_attested | correction_uploaded | correction_promoted | superseded | ...
actorUserId Guid FK → User
occurredAt DateTimeOffset
metadataJson string? optional JSON (field checks, rejection reason, assigned-to)
@@ -838,8 +920,8 @@ createdAt DateTimeOffset
```
id Guid PK
eventType string e.g. observation.created
aggregateType string e.g. Observation
eventType string e.g. observation.created | observation.recorded | alert.generated
aggregateType string e.g. Observation | ClinicalAlert
aggregateId Guid FK → the created entity
payloadJson string full event payload for downstream consumers
createdAt DateTimeOffset
@@ -847,6 +929,36 @@ processedAt DateTimeOffset? set when consumed
retryCount int default 0
```
### AlertThreshold
```
id Guid PK
observationCode string required, unique (e.g. POTASSIUM_MEQ_L)
displayName string required
unit string required
criticalLow decimal?
warningLow decimal?
warningHigh decimal?
criticalHigh decimal?
suppressionWindowMinutes int?
createdAt DateTimeOffset
```
### ClinicalAlert
```
id Guid PK
encounterId Guid FK → Encounter
patientId Guid FK → Patient
observationId Guid? FK → Observation (triggering value)
alertType string e.g. CRITICAL_POTASSIUM_MEQ_L
severity string WARNING | CRITICAL
details string human-readable breach message
observationCode string?
status string OPEN | ACKNOWLEDGED | RESOLVED | ESCALATED
triggeredAt DateTimeOffset
```
### IdempotencyRecord
```
@@ -922,7 +1034,7 @@ Response shape:
## Implemented Phases
Five phases from the project roadmap are implemented and verified. Integration tests (`dotnet test`) and per-phase verification scripts cover Phases 15.
Six phases from the project roadmap are implemented and verified. Integration tests (`dotnet test`) and per-phase verification scripts cover Phases 16.
| Phase | Feature | Status |
|---|---|---|
@@ -931,7 +1043,7 @@ Five phases from the project roadmap are implemented and verified. Integration t
| 3 | Verification with field-level checks (`ok`, `warning`, `error` per field), rejection with mandatory reason (min 10 chars), separation of duties enforcement (`SEPARATION_OF_DUTIES_VIOLATION`), site-configurable clinical approval routing per batch type, work queue endpoints (verification, entry, clinical approval) with role-based access, `VerificationTests` integration tests | Done |
| 4 | Approval with three-person separation of duties (entry ≠ verifier ≠ approver), atomic promotion to live `patients`/`encounters`/`observations` tables in a single PostgreSQL transaction, PostgreSQL sequence-backed MRN generation (`VCR-NNNNNN`), patient dedup by name + DOB, encounter matching by patient + department + active status, transactional outbox (`observation.created` events), retroactive alert policy per batch, `Idempotency-Key` header with 24h TTL for safe retries, `PromotionTests` integration tests, Phase 4 verification script | Done |
| 5 | Correction batches via `supersedesBatchId`, supersession validation on create (`422`/`404`/`409`), append-only `live_observations` supersession flags (`is_superseded`, `superseded_by_batch_id`, `superseded_at`), correction promotion reuses original encounter and linked patient, `correction_uploaded`/`correction_promoted`/`superseded` audit events, `GET /patients/:id/digitization-history` with correction chain and per-batch audit trails, `CorrectionSupersessionTests` integration tests, Phase 5 verification script | Done |
| 6 | Track B live capture with clinician attestation, synchronous alert evaluation | Planned |
| 6 | Track B live capture: `LiveCaptureController` with clinician-only endpoints, `AttestationService` (role + password re-confirm), synchronous promotion via `LiveCaptureService`, Redis-backed critical threshold evaluation, inline critical alerts + committed `ClinicalAlert` rows, `observation.recorded` and `alert.generated` outbox events, open-encounter + vitals outpatient workflow, `LiveCaptureIntegrationTests`, Phase 6 verification script | Done |
| 7 | Digitization workstation UI (Vue 3 side-by-side scan viewer + entry form) | Planned |
| 8 | Prometheus metrics, supervisor dashboard, promotion retry job | Planned |
| 9 | Seed data, E2E verification script, clinical scenario documentation | Planned |