test: run 21 verification script

chore: add readme
This commit is contained in:
voltsrage
2026-06-23 17:13:11 +08:00
parent 1bf8359097
commit c38334690c
3 changed files with 63 additions and 13 deletions
+59 -12
View File
@@ -2,11 +2,11 @@
A production-quality clinical backend built with ASP.NET Core 8, PostgreSQL, Apache Kafka, RabbitMQ, Elasticsearch, Redis, and MinIO. The domain models the observe-alert-acknowledge lifecycle at the center of any clinical monitoring system: patient encounters, continuous vital sign and lab result ingest, real-time sepsis and NEWS2 scoring, and clinician notification with automatic escalation. A production-quality clinical backend built with ASP.NET Core 8, PostgreSQL, Apache Kafka, RabbitMQ, Elasticsearch, Redis, and MinIO. The domain models the observe-alert-acknowledge lifecycle at the center of any clinical monitoring system: patient encounters, continuous vital sign and lab result ingest, real-time sepsis and NEWS2 scoring, and clinician notification with automatic escalation.
**Implementation status:** Twenty-seven planned phases are complete through Phase 31 (plus Phase 20) — from schema and CRUD through Kafka, Elasticsearch CQRS, sepsis detection, RabbitMQ paging with DLQ escalation, reconciliation jobs, Prometheus/Grafana observability, the MinIO Parquet data lake, clinical data model expansion, warning alerts and orders, the NEWS2 composite scoring engine, trend detection with alert suppression, qSOFA bedside screening, medication administration with alert correlation annotations, the console replay simulator, the **Vue 3 ward dashboard**, clinician feedback mode, **Glasgow Coma Scale (GCS) scoring**, **SOFA organ-dysfunction scoring with baseline tracking and delta sepsis alerts**, the **Sepsis-3 clinical refactor** (SIRS removed, qSOFA repositioned as screening, SOFA delta ≥ 2 triggers bundles), **frontend GCS entry and SOFA display**, **expanded simulator scenarios with clinical validation**, the **Site & Gateway Registry** with dual authentication, shared clinical sync contracts, and fleet health Prometheus gauges, the **FHIR R4 Inbound Facade** for EHR integration, and **Role-Based Access Control (RBAC) with clinical audit logging**. Post-phase hardening includes health check endpoints, Kafka poison pill protection, outbox dead-letter with retry tracking, data lake partial-commit safety, MRN sequence-based generation, FHIR bundle transaction rollback, **FHIR R4 read/search endpoints** (Patient and Encounter), **alert threshold deletion with audit trail**, **FHIR API key rotation** (constant-time multi-key validation), **authorization failure logging** with Prometheus metrics, **JWT signing key validation** at startup, and **concurrency hardening** (transactional sepsis bundle creation, unique active encounter constraint). See [Implemented Phases](#implemented-phases) for the full breakdown. Guides: [dashboard-guide.md](docs/dashboard-guide.md) (technical), [clinical-testing-guide.md](docs/clinical-testing-guide.md) (doctors & nurses). **Implementation status:** Twenty-eight planned phases are complete through Phase 31 (plus Phases 2021) — from schema and CRUD through Kafka, Elasticsearch CQRS, sepsis detection, RabbitMQ paging with DLQ escalation, reconciliation jobs, Prometheus/Grafana observability, the MinIO Parquet data lake, clinical data model expansion, warning alerts and orders, the NEWS2 composite scoring engine, trend detection with alert suppression, qSOFA bedside screening, medication administration with alert correlation annotations, the console replay simulator, the **Vue 3 ward dashboard**, clinician feedback mode, **Glasgow Coma Scale (GCS) scoring**, **SOFA organ-dysfunction scoring with baseline tracking and delta sepsis alerts**, the **Sepsis-3 clinical refactor** (SIRS removed, qSOFA repositioned as screening, SOFA delta ≥ 2 triggers bundles), **frontend GCS entry and SOFA display**, **expanded simulator scenarios with clinical validation**, the **Site & Gateway Registry** with dual authentication, shared clinical sync contracts, and fleet health Prometheus gauges, the **Ward Gateway Service** (local-first clinical path with offline buffering and central sync), the **FHIR R4 Inbound Facade** for EHR integration, and **Role-Based Access Control (RBAC) with clinical audit logging**. Post-phase hardening includes health check endpoints, Kafka poison pill protection, outbox dead-letter with retry tracking, data lake partial-commit safety, MRN sequence-based generation, FHIR bundle transaction rollback, **FHIR R4 read/search endpoints** (Patient and Encounter), **alert threshold deletion with audit trail**, **FHIR API key rotation** (constant-time multi-key validation), **authorization failure logging** with Prometheus metrics, **JWT signing key validation** at startup, and **concurrency hardening** (transactional sepsis bundle creation, unique active encounter constraint). See [Implemented Phases](#implemented-phases) for the full breakdown. Guides: [dashboard-guide.md](docs/dashboard-guide.md) (technical), [clinical-testing-guide.md](docs/clinical-testing-guide.md) (doctors & nurses).
## Domain Model — How It Maps to a Real Clinical System ## Domain Model — How It Maps to a Real Clinical System
In a hospital, a patient presents for care and an encounter is opened. Bedside monitors and lab systems post observations continuously against that encounter — either directly via the REST API or through the FHIR R4 facade that maps HL7 FHIR resources from integration engines (Mirth Connect, Rhapsody) into the internal domain. The FHIR facade also exposes read and search interactions so EHR systems can query patient and encounter data back in standard FHIR R4 format. A rules engine evaluates each observation against configured thresholds and flags abnormal values as clinical alerts. Composite scoring engines (NEWS2, GCS, SOFA, qSOFA) aggregate multiple vitals and labs into acuity scores. The sepsis pathway follows Sepsis-3 consensus: qSOFA ≥ 2 creates a bedside screening alert recommending SOFA labs; when SOFA delta ≥ 2 from baseline confirms organ dysfunction, a `SOFA_SEPSIS` alert triggers the treatment bundle. Clinicians authenticate via JWT, and role-based access control (RBAC) gates every endpoint by clinical role (Nurse, Physician, Admin, Integration). Clinicians acknowledge and resolve alerts. If a critical alert goes unacknowledged for five minutes, the system escalates to the on-call backup. All clinical write actions — patient registration, alert acknowledgment, threshold changes, encounter transitions — are recorded in an append-only audit log with user identity, IP address, correlation ID, and before/after state. All events flow through Kafka so the Elasticsearch dashboard, scoring engines, and data lake writer consume the same stream independently. In a hospital, a patient presents for care and an encounter is opened. Bedside monitors and lab systems post observations continuously against that encounter — either directly via the REST API, through the FHIR R4 facade that maps HL7 FHIR resources from integration engines (Mirth Connect, Rhapsody), or via ward gateway edge nodes that buffer observations locally during connectivity loss and sync to the central API when the link recovers. The FHIR facade also exposes read and search interactions so EHR systems can query patient and encounter data back in standard FHIR R4 format. A rules engine evaluates each observation against configured thresholds and flags abnormal values as clinical alerts. Composite scoring engines (NEWS2, GCS, SOFA, qSOFA) aggregate multiple vitals and labs into acuity scores. The sepsis pathway follows Sepsis-3 consensus: qSOFA ≥ 2 creates a bedside screening alert recommending SOFA labs; when SOFA delta ≥ 2 from baseline confirms organ dysfunction, a `SOFA_SEPSIS` alert triggers the treatment bundle. Clinicians authenticate via JWT, and role-based access control (RBAC) gates every endpoint by clinical role (Nurse, Physician, Admin, Integration). Clinicians acknowledge and resolve alerts. If a critical alert goes unacknowledged for five minutes, the system escalates to the on-call backup. All clinical write actions — patient registration, alert acknowledgment, threshold changes, encounter transitions — are recorded in an append-only audit log with user identity, IP address, correlation ID, and before/after state. All events flow through Kafka so the Elasticsearch dashboard, scoring engines, and data lake writer consume the same stream independently.
``` ```
Patient ─────────────────────────── one patient = one MRN, many lifetime encounters Patient ─────────────────────────── one patient = one MRN, many lifetime encounters
@@ -78,6 +78,7 @@ An `OutboxEvent` is written in the same transaction as any observation or alert,
- **Role-Based Access Control (RBAC)** — JWT bearer authentication (`POST /auth/login`); four clinical roles (`Nurse`, `Physician`, `Admin`, `Integration`) with 17 granular permissions (`patients:read`, `alerts:acknowledge`, `thresholds:write`, `fhir:ingest`, `fhir:read`, `audit:read`, etc.); `AuthorizePermission` attribute on every controller action; `PermissionAuthorizationHandler` resolves role → permission at runtime from `ClinicalRolePermissionMap` and logs authorization failures with structured details (user, role, permission, endpoint) plus `authorization_failures_total` Prometheus counter; `CurrentUserService` exposes authenticated identity (user ID, display name, role, IP address) to services; nurses and physicians get clinical read/write permissions; admins additionally get `thresholds:write`, `fhir:read`, `audit:read`, and `users:admin`; integration accounts get FHIR ingest and read access; FHIR endpoints accept both JWT and `X-Api-Key` authentication via `FhirApiKeyOrJwtMiddleware`; alert `acknowledgedBy` is set from the authenticated user identity, not the request body; startup validates JWT signing key is at least 256 bits (HMAC-SHA256 minimum); four seeded demo users (`nurse.demo`, `physician.demo`, `admin.demo`, `integration.mirth`); frontend login page with `localStorage` token persistence and automatic `Authorization: Bearer` header injection - **Role-Based Access Control (RBAC)** — JWT bearer authentication (`POST /auth/login`); four clinical roles (`Nurse`, `Physician`, `Admin`, `Integration`) with 17 granular permissions (`patients:read`, `alerts:acknowledge`, `thresholds:write`, `fhir:ingest`, `fhir:read`, `audit:read`, etc.); `AuthorizePermission` attribute on every controller action; `PermissionAuthorizationHandler` resolves role → permission at runtime from `ClinicalRolePermissionMap` and logs authorization failures with structured details (user, role, permission, endpoint) plus `authorization_failures_total` Prometheus counter; `CurrentUserService` exposes authenticated identity (user ID, display name, role, IP address) to services; nurses and physicians get clinical read/write permissions; admins additionally get `thresholds:write`, `fhir:read`, `audit:read`, and `users:admin`; integration accounts get FHIR ingest and read access; FHIR endpoints accept both JWT and `X-Api-Key` authentication via `FhirApiKeyOrJwtMiddleware`; alert `acknowledgedBy` is set from the authenticated user identity, not the request body; startup validates JWT signing key is at least 256 bits (HMAC-SHA256 minimum); four seeded demo users (`nurse.demo`, `physician.demo`, `admin.demo`, `integration.mirth`); frontend login page with `localStorage` token persistence and automatic `Authorization: Bearer` header injection
- **Clinical Audit Logging** — append-only `clinical_audit_logs` table records clinical write actions with user identity, entity type/ID, before/after state (JSONB), reason, IP address, and correlation ID; ten audit actions (`THRESHOLD_CREATED`, `THRESHOLD_UPDATED`, `THRESHOLD_DELETED`, `ALERT_ACKNOWLEDGED`, `ALERT_RESOLVED`, `ENCOUNTER_STATUS_CHANGED`, `PATIENT_REGISTERED`, `SUPPRESSION_WINDOW_SET`, `USER_LOGIN`, `AUTHORIZATION_DENIED`); `AuditService` writes log entries inline with domain operations; `GET /audit-logs` admin-only query endpoint with filters by entity type, entity ID, user ID, action, and time range; indexed on entity type, entity ID, user ID, and timestamp - **Clinical Audit Logging** — append-only `clinical_audit_logs` table records clinical write actions with user identity, entity type/ID, before/after state (JSONB), reason, IP address, and correlation ID; ten audit actions (`THRESHOLD_CREATED`, `THRESHOLD_UPDATED`, `THRESHOLD_DELETED`, `ALERT_ACKNOWLEDGED`, `ALERT_RESOLVED`, `ENCOUNTER_STATUS_CHANGED`, `PATIENT_REGISTERED`, `SUPPRESSION_WINDOW_SET`, `USER_LOGIN`, `AUTHORIZATION_DENIED`); `AuditService` writes log entries inline with domain operations; `GET /audit-logs` admin-only query endpoint with filters by entity type, entity ID, user ID, action, and time range; indexed on entity type, entity ID, user ID, and timestamp
- **Site & Gateway Registry** — `ClinicalSite` and `WardGateway` domain entities model ward edge nodes that buffer clinical data during connectivity loss; `POST /sites` creates clinical sites; `POST /sites/{siteId}/gateways` registers gateways under a site; `PATCH /gateways/{gatewayId}/heartbeat` (gateway API key auth) updates status (`ONLINE`, `DEGRADED`, `OFFLINE`) and reported buffer depth; `GET /sites/{siteId}/gateways` lists gateways with optional `?department=` filter; dual authentication — JWT + RBAC (`users:admin`) for admin CRUD, `GatewayApiKeyAuthenticationHandler` (`X-Api-Key` + `X-Gateway-Id`) for gateway heartbeat and future sync upload; constant-time key comparison via `CryptographicOperations.FixedTimeEquals`; `VigilCare.ClinicalContracts` shared class library with sync DTOs (`ClinicalSyncBatchRequest`, `SyncedObservation`, `SyncedAlertEvent`, `GatewayHeartbeatRequest`) consumed by both central API and ward gateway projects; Prometheus `ward_gateways_offline_gauge` and `ward_gateway_buffer_depth` via `WardGatewayMetricsCollector` (60s periodic); `GatewayRegistrySeeder` provides demo site and gateway for Docker Compose and tests; FluentValidation on all request DTOs; `GatewayRegistryTests` and `ClinicalContractsTests` integration tests - **Site & Gateway Registry** — `ClinicalSite` and `WardGateway` domain entities model ward edge nodes that buffer clinical data during connectivity loss; `POST /sites` creates clinical sites; `POST /sites/{siteId}/gateways` registers gateways under a site; `PATCH /gateways/{gatewayId}/heartbeat` (gateway API key auth) updates status (`ONLINE`, `DEGRADED`, `OFFLINE`) and reported buffer depth; `GET /sites/{siteId}/gateways` lists gateways with optional `?department=` filter; dual authentication — JWT + RBAC (`users:admin`) for admin CRUD, `GatewayApiKeyAuthenticationHandler` (`X-Api-Key` + `X-Gateway-Id`) for gateway heartbeat and future sync upload; constant-time key comparison via `CryptographicOperations.FixedTimeEquals`; `VigilCare.ClinicalContracts` shared class library with sync DTOs (`ClinicalSyncBatchRequest`, `SyncedObservation`, `SyncedAlertEvent`, `GatewayHeartbeatRequest`) consumed by both central API and ward gateway projects; Prometheus `ward_gateways_offline_gauge` and `ward_gateway_buffer_depth` via `WardGatewayMetricsCollector` (60s periodic); `GatewayRegistrySeeder` provides demo site and gateway for Docker Compose and tests; FluentValidation on all request DTOs; `GatewayRegistryTests` and `ClinicalContractsTests` integration tests
- **Ward Gateway Service** — `VigilCare.WardGateway` (`http://localhost:5081`) is a standalone ASP.NET Core 8 deployable with its own PostgreSQL, Redis, and RabbitMQ; ingests observations locally via `POST /encounters/:id/observations` with plausibility validation, Redis-cached threshold evaluation, and synchronous critical alert creation; `LocalWarningEvaluator` creates warning-range alerts; `BufferedSyncWriter` writes all clinical events to `buffered_sync_items` for central upload; `EncounterReplicaSyncService` pulls patient/encounter data from central API; `ThresholdCacheLoader` fetches thresholds from central into local Redis; `CentralReachabilityService` tracks central API connectivity; `GatewayHeartbeatService` reports status and buffer depth; `SyncUploaderService` batches and uploads buffered items when online; local RabbitMQ paging and escalation queues; `GET /encounters` ward list and `GET /encounters/:id` detail; `GET /health/live` and `GET /health/ready` (Redis, RabbitMQ, encounter replica readiness); Docker Compose `ward-gateway` profile
- **Health Check Endpoints** — `GET /health/live` (liveness — always returns 200 if the process is running) and `GET /health/ready` (readiness — checks PostgreSQL, Redis, Kafka, RabbitMQ, and Elasticsearch connectivity); both return structured JSON with per-check status and duration; anonymous access; suitable for Kubernetes probes and load balancer health checks - **Health Check Endpoints** — `GET /health/live` (liveness — always returns 200 if the process is running) and `GET /health/ready` (readiness — checks PostgreSQL, Redis, Kafka, RabbitMQ, and Elasticsearch connectivity); both return structured JSON with per-check status and duration; anonymous access; suitable for Kubernetes probes and load balancer health checks
- **Kafka Poison Pill Protection** — `PoisonPillGuard` prevents a single un-processable message from blocking a consumer partition forever; permanent errors (malformed JSON, bad format) are skipped immediately; transient errors are retried up to `MaxPoisonRetries` (default 5) before the offset is committed and the message is abandoned; all eight Kafka consumers use the guard; skipped messages are logged at CRITICAL with full payload and tracked by Prometheus `kafka_poison_pills_skipped_total` (labeled by consumer group and topic) - **Kafka Poison Pill Protection** — `PoisonPillGuard` prevents a single un-processable message from blocking a consumer partition forever; permanent errors (malformed JSON, bad format) are skipped immediately; transient errors are retried up to `MaxPoisonRetries` (default 5) before the offset is committed and the message is abandoned; all eight Kafka consumers use the guard; skipped messages are logged at CRITICAL with full payload and tracked by Prometheus `kafka_poison_pills_skipped_total` (labeled by consumer group and topic)
- **Outbox Dead-Letter with Retry Tracking** — `OutboxRelayService` tracks `RetryCount`, `LastError`, and `FailedAt` per event; events that fail `OutboxMaxRetries` (default 10) Kafka produce attempts are marked permanently failed (`FailedAt` set) and excluded from future relay polls; uses `FOR UPDATE SKIP LOCKED` for safe concurrent relay instances; idempotent Kafka producer (`EnableIdempotence = true`) prevents duplicate messages from network-level retries - **Outbox Dead-Letter with Retry Tracking** — `OutboxRelayService` tracks `RetryCount`, `LastError`, and `FailedAt` per event; events that fail `OutboxMaxRetries` (default 10) Kafka produce attempts are marked permanently failed (`FailedAt` set) and excluded from future relay polls; uses `FOR UPDATE SKIP LOCKED` for safe concurrent relay instances; idempotent Kafka producer (`EnableIdempotence = true`) prevents duplicate messages from network-level retries
@@ -140,6 +141,23 @@ IHostedServices (background):
KafkaConsumerLagCollector → polls four consumer groups every 30s → kafka_consumer_lag KafkaConsumerLagCollector → polls four consumer groups every 30s → kafka_consumer_lag
WardGatewayMetricsCollector → polls gateway status/buffer depth every 60s → ward_gateways_offline_gauge, ward_gateway_buffer_depth WardGatewayMetricsCollector → polls gateway status/buffer depth every 60s → ward_gateways_offline_gauge, ward_gateway_buffer_depth
ClinicalMetrics (singleton) → inline counters/histogram from ingest, qSOFA, NEWS2, GCS, SOFA, trend, suppression, bundle compliance, escalation paths ClinicalMetrics (singleton) → inline counters/histogram from ingest, qSOFA, NEWS2, GCS, SOFA, trend, suppression, bundle compliance, escalation paths
Ward Gateway (VigilCare.WardGateway — separate deployable on port 5081):
HTTP request → ExceptionHandlerMiddleware → Controllers (Observations, Alerts, Encounters, CentralRequired)
Services:
LocalObservationService → PostgreSQL (ward) + Redis (ward) threshold cache → critical alert + outbox
LocalWarningEvaluator → Redis threshold lookup → WARNING alert creation
LocalAlertService → Acknowledge/resolve local alerts
BufferedSyncWriter → Writes sync items to buffered_sync_items table
EncounterReadService → Ward encounter list and detail views
BackgroundServices:
EncounterReplicaSyncService → pulls patient/encounter data from central API
ThresholdCacheLoader → fetches thresholds from central → local Redis
CentralReachabilityService → polls central /health/ready every 30s
GatewayHeartbeatService → reports status + buffer depth to central registry
SyncUploaderService → batches buffered_sync_items → central API upload
LocalPagingWorkerService → local RabbitMQ paging queue → clinician page
LocalEscalationWorkerService → local RabbitMQ escalation queue → on-call backup
``` ```
**Why Kafka and RabbitMQ coexist:** Kafka is an append-only log — the same observation event reaches the Elasticsearch indexer, the sepsis engine, and the data lake independently without coordination. Each consumer holds its own offset and can replay from the beginning. RabbitMQ handles the action side: one message, one worker, one page. A duplicate page at 3am is a patient safety concern, not a minor inconvenience — RabbitMQ's acknowledgment-then-delete model is correct here. The DLQ TTL-based escalation has no equivalent in Kafka. **Why Kafka and RabbitMQ coexist:** Kafka is an append-only log — the same observation event reaches the Elasticsearch indexer, the sepsis engine, and the data lake independently without coordination. Each consumer holds its own offset and can replay from the beginning. RabbitMQ handles the action side: one message, one worker, one page. A duplicate page at 3am is a patient safety concern, not a minor inconvenience — RabbitMQ's acknowledgment-then-delete model is correct here. The DLQ TTL-based escalation has no equivalent in Kafka.
@@ -464,15 +482,18 @@ VigilCare.ClinicalContracts.Tests/ # Contracts round-trip seriali
VigilCare.WardGateway/ # Phase 21 — local-first ward edge API (separate DB/Redis/RabbitMQ) VigilCare.WardGateway/ # Phase 21 — local-first ward edge API (separate DB/Redis/RabbitMQ)
├── Dockerfile ├── Dockerfile
├── Program.cs ├── Program.cs
├── Common/ # ApiResponse envelope, CursorPage, PagedResult, domain exceptions
├── Data/ ├── Data/
│ ├── GatewayDbContext.cs │ ├── GatewayDbContext.cs
│ └── Configurations/ # snake_case mappings mirroring central API patterns │ └── Configurations/ # snake_case mappings mirroring central API patterns
├── Domain/ ├── Domain/
│ ├── Entities/ # ReplicaPatient, ReplicaEncounter, LocalObservation, │ ├── Entities/ # ReplicaPatient, ReplicaEncounter, LocalObservation, LocalClinicalAlert, BufferedSyncItem, SyncOutboxEntry, GatewaySyncState, ReplicaAlertThreshold
│ └── Enums/ # AlertType, EncounterStatus, BufferedSyncItemType, │ └── Enums/ # AlertType, AlertSeverity, AlertStatus, EncounterStatus, EncounterType, BufferedSyncItemType, SyncOutboxStatus, ObservationSource, Department
├── Models/Records/ThresholdCacheEntry.cs ├── Models/Records/ThresholdCacheEntry.cs
├── Models/Records/Observation/IngestObservationRequest.cs ├── Models/Records/Observation/IngestObservationRequest.cs
├── Models/Records/Alert/AcknowledgeAlertRequest.cs ├── Models/Records/Alert/AcknowledgeAlertRequest.cs
├── Models/Records/Encounter/ # EncounterDetail, WardEncounterSummary
├── Models/Records/LocalIngestResult.cs
├── Models/Central/CentralSyncDtos.cs # DTOs for central API sync responses ├── Models/Central/CentralSyncDtos.cs # DTOs for central API sync responses
├── Models/Central/CentralApiJson.cs ├── Models/Central/CentralApiJson.cs
├── Validators/IngestObservationRequestValidator.cs ├── Validators/IngestObservationRequestValidator.cs
@@ -492,16 +513,30 @@ VigilCare.WardGateway/ # Phase 21 — local-first war
│ ├── EncountersController.cs │ ├── EncountersController.cs
│ └── CentralRequiredController.cs │ └── CentralRequiredController.cs
├── Notifications/RabbitMqTopologyProvisioner.cs ├── Notifications/RabbitMqTopologyProvisioner.cs
├── Middlewares/ExceptionHandlerMiddleware.cs
├── BackgroundService/ ├── BackgroundService/
│ ├── EncounterReplicaSyncService.cs │ ├── EncounterReplicaSyncService.cs # Syncs patient/encounter data from central API on startup
│ ├── ThresholdCacheLoader.cs │ ├── ThresholdCacheLoader.cs # Pre-loads alert thresholds from central API into Redis
│ ├── CentralReachabilityService.cs │ ├── CentralReachabilityService.cs # Periodic central API health check (sets online/offline state)
│ ├── GatewayHeartbeatService.cs # Reports gateway status and buffer depth to central API
│ ├── SyncUploaderService.cs # Uploads buffered observations/alerts to central API when online
│ ├── LocalPagingWorkerService.cs │ ├── LocalPagingWorkerService.cs
│ └── LocalEscalationWorkerService.cs │ └── LocalEscalationWorkerService.cs
├── Configurations/ # GatewayOptions, CentralApiOptions, RabbitMqOptions ├── Configurations/ # GatewayOptions, CentralApiOptions, RabbitMqOptions, SuppressionOptions
├── Infrastructure/ # Health checks (Redis, RabbitMQ, encounter replica ready) ├── Infrastructure/ # Health checks (Redis, RabbitMQ, encounter replica ready)
└── Migrations/ # InitialGatewaySchema └── Migrations/ # InitialGatewaySchema
VigilCare.WardGateway.Tests/ # Phase 21 — ward gateway integration tests
├── VigilCare.WardGateway.Tests.csproj
├── Fixtures/
│ ├── GatewayApiFixture.cs # WebApplicationFactory with Testcontainers (PostgreSQL, Redis, RabbitMQ)
│ └── GatewayIntegrationCollection.cs
├── Helpers/
│ ├── GatewayDbResetHelper.cs # Database reset between tests
│ └── GatewayTestSeeder.cs # Seeds test patients, encounters, thresholds
├── WardGatewayLocalPathTests.cs # Local observation ingest, warning alerts, buffered sync items
└── WardGatewayPartitionTests.cs # Network partition simulation — offline buffering and sync upload
VigilCare.Simulator/ # Phase 16 — console replay simulator (HTTP-only, no direct DB/Kafka) VigilCare.Simulator/ # Phase 16 — console replay simulator (HTTP-only, no direct DB/Kafka)
├── Program.cs # CLI: replay, replay-all, validate, dry-run ├── Program.cs # CLI: replay, replay-all, validate, dry-run
├── Commands/ # System.CommandLine command handlers ├── Commands/ # System.CommandLine command handlers
@@ -543,6 +578,7 @@ scripts/
├── run-phase27-verification.sh # Phase 27 — Sepsis-3 refactor: SIRS removal, QSOFA_SCREEN, SOFA bundle trigger ├── run-phase27-verification.sh # Phase 27 — Sepsis-3 refactor: SIRS removal, QSOFA_SCREEN, SOFA bundle trigger
├── run-phase28-verification.sh # Phase 28 — Frontend GCS entry + SOFA display + sepsis UI refactor ├── run-phase28-verification.sh # Phase 28 — Frontend GCS entry + SOFA display + sepsis UI refactor
├── run-phase20-verification.sh # Phase 20 — Gateway registry tests + site/gateway/heartbeat curl checks ├── run-phase20-verification.sh # Phase 20 — Gateway registry tests + site/gateway/heartbeat curl checks
├── run-phase21-verification.sh # Phase 21 — Ward gateway local-first path, partition tests, sync upload
├── run-phase29-verification.sh # Phase 29 — Simulator scenario expansion + clinical validation ├── run-phase29-verification.sh # Phase 29 — Simulator scenario expansion + clinical validation
├── run-phase30-verification.sh # Phase 30 — FHIR R4 ingest integration tests + manual bundle/metadata checks ├── run-phase30-verification.sh # Phase 30 — FHIR R4 ingest integration tests + manual bundle/metadata checks
└── run-phase31-verification.sh # Phase 31 — RBAC integration tests + JWT login + audit log query └── run-phase31-verification.sh # Phase 31 — RBAC integration tests + JWT login + audit log query
@@ -720,7 +756,7 @@ On startup the application:
4. Provisions Kafka topics and Elasticsearch indices 4. Provisions Kafka topics and Elasticsearch indices
5. Declares the RabbitMQ exchange and queue topology 5. Declares the RabbitMQ exchange and queue topology
6. Starts all background consumers (outbox relay, ES indexer, sepsis engine with qSOFA screening, warning evaluator, NEWS2 scoring, GCS scoring, SOFA scoring, trend analyzer, notification workers, data lake writer, reconciliation scheduler, sepsis bundle monitor) 6. Starts all background consumers (outbox relay, ES indexer, sepsis engine with qSOFA screening, warning evaluator, NEWS2 scoring, GCS scoring, SOFA scoring, trend analyzer, notification workers, data lake writer, reconciliation scheduler, sepsis bundle monitor)
7. Starts Prometheus metric collectors (unacknowledged alerts, outbox pending, Kafka consumer lag) 7. Starts Prometheus metric collectors (unacknowledged alerts, outbox pending, Kafka consumer lag, ward gateway fleet health)
Swagger UI is available at `http://localhost:5270/swagger` in Development (API binds to `0.0.0.0:5270` per `launchSettings.json`). Swagger UI is available at `http://localhost:5270/swagger` in Development (API binds to `0.0.0.0:5270` per `launchSettings.json`).
@@ -796,13 +832,17 @@ Integration tests use `WebApplicationFactory` with a `Testing` environment and T
| `ClinicalContractsTests` | 20 | ClinicalSyncBatchRequest JSON round-trip serialization | | `ClinicalContractsTests` | 20 | ClinicalSyncBatchRequest JSON round-trip serialization |
| `FhirIngestTests` | 30 | FHIR R4 patient upsert idempotency, LOINC observation mapping, unknown code 422, transaction bundle | | `FhirIngestTests` | 30 | FHIR R4 patient upsert idempotency, LOINC observation mapping, unknown code 422, transaction bundle |
| `RbacTests` | 31 | Unauthenticated 401, nurse 403 on threshold write, admin threshold update with audit log creation, alert acknowledge uses authenticated user | | `RbacTests` | 31 | Unauthenticated 401, nurse 403 on threshold write, admin threshold update with audit log creation, alert acknowledge uses authenticated user |
| `BackgroundServiceTests` | — | Outbox relay, Kafka consumer, sepsis bundle monitor, reconciliation |
| `ConcurrencyTests` | — | Parallel patient registration unique MRNs, parallel sepsis alerts single bundle, parallel observation idempotency, parallel encounter open duplicate rejection | | `ConcurrencyTests` | — | Parallel patient registration unique MRNs, parallel sepsis alerts single bundle, parallel observation idempotency, parallel encounter open duplicate rejection |
| `WardGatewayLocalPathTests` | 21 | Local observation ingest, critical/warning alert creation, buffered sync item generation |
| `WardGatewayPartitionTests` | 21 | Network partition simulation — offline buffering, sync upload to central API, encounter replica sync |
### Verification Scripts ### Verification Scripts
With the API running (`dotnet run`) and Docker Compose up: With the API running (`dotnet run`) and Docker Compose up:
```bash ```bash
./scripts/run-phase21-verification.sh # Ward gateway local-first path, partition tests, sync upload verification
./scripts/run-phase20-verification.sh # Gateway registry tests, site/gateway/heartbeat API verification ./scripts/run-phase20-verification.sh # Gateway registry tests, site/gateway/heartbeat API verification
./scripts/run-phase8-verification.sh # Prometheus target UP, ten metrics, alerts_unacknowledged_gauge live update ./scripts/run-phase8-verification.sh # Prometheus target UP, ten metrics, alerts_unacknowledged_gauge live update
./scripts/run-phase9-verification.sh # DataLakePhase9Tests, Kafka consumer group, MinIO Parquet, DuckDB schema ./scripts/run-phase9-verification.sh # DataLakePhase9Tests, Kafka consumer group, MinIO Parquet, DuckDB schema
@@ -845,6 +885,12 @@ Phase 15 unit/integration tests only:
dotnet test --filter "FullyQualifiedName~Medication" dotnet test --filter "FullyQualifiedName~Medication"
``` ```
Phase 21 ward gateway tests only:
```bash
dotnet test --filter "FullyQualifiedName~WardGateway"
```
Phase 20 gateway tests only: Phase 20 gateway tests only:
```bash ```bash
@@ -1954,7 +2000,7 @@ Observation history uses cursor pagination on `(recorded_at DESC, id DESC)`. Off
## Implemented Phases ## Implemented Phases
Twenty-seven phases from the project roadmap are implemented and verified, including the **Site & Gateway Registry** (Phase 20), the **Sepsis-3 clinical refactor** (Phases 2729), the **FHIR R4 Inbound Facade** (Phase 30), and **RBAC with clinical audit logging** (Phase 31). Integration tests (`dotnet test`) and per-phase verification scripts cover Phases 815, 20, 2531. Phases 1719 add the Vue dashboard and clinician feedback (Vitest in `vigilcare-dashboard/`). Twenty-eight phases from the project roadmap are implemented and verified, including the **Site & Gateway Registry** (Phase 20), the **Ward Gateway Service** (Phase 21), the **Sepsis-3 clinical refactor** (Phases 2729), the **FHIR R4 Inbound Facade** (Phase 30), and **RBAC with clinical audit logging** (Phase 31). Integration tests (`dotnet test`) and per-phase verification scripts cover Phases 815, 2021, 2531. Phases 1719 add the Vue dashboard and clinician feedback (Vitest in `vigilcare-dashboard/`).
| Phase | Feature | Status | | Phase | Feature | Status |
|---|---|---| |---|---|---|
@@ -1975,6 +2021,7 @@ Twenty-seven phases from the project roadmap are implemented and verified, inclu
| 15 | Medication administration (`MedicationAdministration`, `MedicationsController`, `MedicationService`); drug-vital correlation config (`MedicationCorrelationOptions`); `MedicationCorrelationHelper` annotates `WarningEvaluator` and `News2Detector` alert details; `medication_administrations` table + migration; `MedicationServiceTests`, `MedicationCorrelationTests`, `MedicationValidationTests`; `run-phase15-verification.sh`; design doc in `docs/decisions/medication-correlation-design.md` | Done | | 15 | Medication administration (`MedicationAdministration`, `MedicationsController`, `MedicationService`); drug-vital correlation config (`MedicationCorrelationOptions`); `MedicationCorrelationHelper` annotates `WarningEvaluator` and `News2Detector` alert details; `medication_administrations` table + migration; `MedicationServiceTests`, `MedicationCorrelationTests`, `MedicationValidationTests`; `run-phase15-verification.sh`; design doc in `docs/decisions/medication-correlation-design.md` | Done |
| 16 | Console replay simulator (`VigilCare.Simulator`); scenario JSON schema; CLI commands `replay`, `replay-all`, `validate`, `dry-run`; speed multiplier and optional API polling; eight sample scenarios; `docs/simulator-guide.md` | Done | | 16 | Console replay simulator (`VigilCare.Simulator`); scenario JSON schema; CLI commands `replay`, `replay-all`, `validate`, `dry-run`; speed multiplier and optional API polling; eight sample scenarios; `docs/simulator-guide.md` | Done |
| 20 | **Site & Gateway Registry + Clinical Sync Contracts** — `VigilCare.ClinicalContracts` shared class library with sync DTOs (`ClinicalSyncBatchRequest`, `SyncedObservation`, `SyncedAlertEvent`, `GatewayHeartbeatRequest`); `ClinicalSite` and `WardGateway` domain entities with EF Core configurations and migrations; `GatewayApiKeyAuthenticationHandler` (constant-time `X-Api-Key` validation + `X-Gateway-Id` claim) registered alongside JWT bearer; `SitesController` (create, list, get) and `GatewaysController` (register, list, get, heartbeat) with dual auth — JWT + `users:admin` for admin CRUD, gateway API key for heartbeat; `SiteService`, `GatewayRegistryService`; FluentValidation on `CreateSiteRequest`, `RegisterGatewayRequest`, `GatewayHeartbeatRequest`; `GatewayRegistrySeeder` with fixed GUIDs for demo site and gateway; `WardGatewayMetricsCollector` (60s periodic) exports `ward_gateways_offline_gauge` and `ward_gateway_buffer_depth`; `GatewayRegistryTests` (register, heartbeat, API key 401, degraded status, department filter); `ClinicalContractsTests` (JSON round-trip); `run-phase20-verification.sh` | Done | | 20 | **Site & Gateway Registry + Clinical Sync Contracts** — `VigilCare.ClinicalContracts` shared class library with sync DTOs (`ClinicalSyncBatchRequest`, `SyncedObservation`, `SyncedAlertEvent`, `GatewayHeartbeatRequest`); `ClinicalSite` and `WardGateway` domain entities with EF Core configurations and migrations; `GatewayApiKeyAuthenticationHandler` (constant-time `X-Api-Key` validation + `X-Gateway-Id` claim) registered alongside JWT bearer; `SitesController` (create, list, get) and `GatewaysController` (register, list, get, heartbeat) with dual auth — JWT + `users:admin` for admin CRUD, gateway API key for heartbeat; `SiteService`, `GatewayRegistryService`; FluentValidation on `CreateSiteRequest`, `RegisterGatewayRequest`, `GatewayHeartbeatRequest`; `GatewayRegistrySeeder` with fixed GUIDs for demo site and gateway; `WardGatewayMetricsCollector` (60s periodic) exports `ward_gateways_offline_gauge` and `ward_gateway_buffer_depth`; `GatewayRegistryTests` (register, heartbeat, API key 401, degraded status, department filter); `ClinicalContractsTests` (JSON round-trip); `run-phase20-verification.sh` | Done |
| 21 | **Ward Gateway Service (Local-First Clinical Path)** — `VigilCare.WardGateway` standalone ASP.NET Core 8 deployable with its own PostgreSQL (`vigilcare_ward`), Redis, and RabbitMQ; domain entities mirror central API (`ReplicaPatient`, `ReplicaEncounter`, `LocalObservation`, `LocalClinicalAlert`, `ReplicaAlertThreshold`); `LocalObservationService` ingests observations locally with plausibility validation and synchronous critical alert creation; `LocalWarningEvaluator` creates warning-range alerts from Redis-cached thresholds; `BufferedSyncWriter` writes observation and alert events to `buffered_sync_items` table for upload when online; `EncounterReplicaSyncService` pulls patient/encounter data from central API on startup; `ThresholdCacheLoader` fetches alert thresholds from central API into local Redis; `CentralReachabilityService` polls central API health every 30s; `GatewayHeartbeatService` reports gateway status and buffer depth to central registry; `SyncUploaderService` batches buffered sync items and uploads to central API using `ClinicalSyncBatchRequest` contracts when online; local RabbitMQ paging (`LocalPagingWorkerService`) and escalation (`LocalEscalationWorkerService`) for ward-level clinician notification; `EncounterReadService` provides ward encounter list and detail views; Docker Compose `ward-gateway` profile with separate PostgreSQL, Redis, and RabbitMQ; health checks (Redis, RabbitMQ, encounter replica readiness); `WardGatewayLocalPathTests` and `WardGatewayPartitionTests` integration tests with Testcontainers; `run-phase21-verification.sh` | Done |
| 17 | Ward dashboard shell — Vue 3 + Vite + Pinia + Tailwind; virtual ward table (NEWS2-sorted, department filter); patient detail (vitals, scores, alerts, orders, sepsis bundle); alert center (global acknowledge/resolve); API polling; CORS-backed `GET /encounters` ward list | Done | | 17 | Ward dashboard shell — Vue 3 + Vite + Pinia + Tailwind; virtual ward table (NEWS2-sorted, department filter); patient detail (vitals, scores, alerts, orders, sepsis bundle); alert center (global acknowledge/resolve); API polling; CORS-backed `GET /encounters` ward list | Done |
| 18 | Clinical review mode — Chart.js vital sign trends (5 charts), NEWS2 history chart, local replay controls, alert reasoning panel, medication context on alerts; `fetchNews2History` / `fetchMedications`; Vitest composable and component tests; `docs/dashboard-guide.md` | Done | | 18 | Clinical review mode — Chart.js vital sign trends (5 charts), NEWS2 history chart, local replay controls, alert reasoning panel, medication context on alerts; `fetchNews2History` / `fetchMedications`; Vitest composable and component tests; `docs/dashboard-guide.md` | Done |
| 19 | Clinician feedback mode — six rating buttons per alert, optional notes, Feedback Summary with aggregate stats, JSON/CSV export, localStorage persistence; `docs/clinical-testing-guide.md` for doctor/nurse evaluation sessions | Done | | 19 | Clinician feedback mode — six rating buttons per alert, optional notes, Feedback Summary with aggregate stats, JSON/CSV export, localStorage persistence; `docs/clinical-testing-guide.md` for doctor/nurse evaluation sessions | Done |
@@ -1988,9 +2035,9 @@ Twenty-seven phases from the project roadmap are implemented and verified, inclu
**Ward dashboard:** backend APIs (`GET /encounters` ward list, `GET /qsofa/current`, CORS) and frontend SPA — `EncountersListTests`, `QsofaCurrentTests`, `vigilcare-dashboard` Vitest suite (replay scrubbing, feedback store, FeedbackButtons, FeedbackSummary, alert components, charts, ward table, GCS entry, SOFA panel, scores panel, alert labels). **Ward dashboard:** backend APIs (`GET /encounters` ward list, `GET /qsofa/current`, CORS) and frontend SPA — `EncountersListTests`, `QsofaCurrentTests`, `vigilcare-dashboard` Vitest suite (replay scrubbing, feedback store, FeedbackButtons, FeedbackSummary, alert components, charts, ward table, GCS entry, SOFA panel, scores panel, alert labels).
**Site & Gateway Registry (Phase 20):** Central API manages clinical sites and ward edge nodes (gateways). Gateways authenticate via API key for heartbeat and future sync upload. Shared `VigilCare.ClinicalContracts` class library defines sync DTOs consumed by both central API and ward gateway projects. Prometheus fleet health gauges track offline gateways and buffer depth per site. Foundation for Phase 21 (WardGateway standalone service) and Phase 22 (sync batch upload). **Site & Gateway Registry (Phase 20):** Central API manages clinical sites and ward edge nodes (gateways). Gateways authenticate via API key for heartbeat and sync upload. Shared `VigilCare.ClinicalContracts` class library defines sync DTOs consumed by both central API and ward gateway projects. Prometheus fleet health gauges track offline gateways and buffer depth per site.
**Ward Gateway (Phase 21, in progress):** `VigilCare.WardGateway` is a separate ASP.NET deployable with its own PostgreSQL (`vigilcare_ward`), Redis, and RabbitMQ. Docker Compose services use the `ward-gateway` profile — start with `docker compose --profile ward-gateway up -d`. Domain entities mirror central `Patient`/`Encounter`/`Observation`/`ClinicalAlert` as replica or local types; see `docs/plans/phase-21-plan.md` Step 2. **Ward Gateway (Phase 21):** `VigilCare.WardGateway` is a separate ASP.NET deployable with its own PostgreSQL (`vigilcare_ward`), Redis, and RabbitMQ — a local-first clinical path for ward edge nodes. Docker Compose services use the `ward-gateway` profile — start with `docker compose --profile ward-gateway up -d`. Observations are ingested locally with threshold evaluation and critical alert creation, then buffered for upload to the central API when the network link is available. Background services replicate encounter/patient data and alert thresholds from central on startup, report heartbeat status, and batch-upload buffered sync items. Local RabbitMQ provides ward-level paging and escalation independent of central connectivity. Integration tests (`WardGatewayLocalPathTests`, `WardGatewayPartitionTests`) validate the local ingest path and network partition/recovery workflow with Testcontainers.
**Scoring pipeline (Phases 2526):** GCS components → `gcs_scores` + `gcs.scored` → SOFA CNS organ system; SOFA lab/vital observations → `sofa_scores` with baseline tracking → delta sepsis alerts when organ dysfunction worsens. **Scoring pipeline (Phases 2526):** GCS components → `gcs_scores` + `gcs.scored` → SOFA CNS organ system; SOFA lab/vital observations → `sofa_scores` with baseline tracking → delta sepsis alerts when organ dysfunction worsens.
+3
View File
@@ -211,6 +211,9 @@ services:
Jwt__SigningKey: "dev-signing-key-minimum-32-bytes-long!!" Jwt__SigningKey: "dev-signing-key-minimum-32-bytes-long!!"
Jwt__Issuer: "vigilcare-gateway" Jwt__Issuer: "vigilcare-gateway"
Jwt__Audience: "vigilcare-dashboard" Jwt__Audience: "vigilcare-dashboard"
GATEWAY_SYNC_JWT: "${GATEWAY_SYNC_JWT:-}"
extra_hosts:
- "host.docker.internal:host-gateway"
depends_on: depends_on:
ward-gateway-db: ward-gateway-db:
condition: service_healthy condition: service_healthy
+1 -1
View File
@@ -45,7 +45,7 @@ sleep 35
echo "==> List active encounters from local replica" echo "==> List active encounters from local replica"
ENCOUNTER_ID=$(curl -sf "$GW/api/v1/encounters?status=ACTIVE&department=ICU" \ ENCOUNTER_ID=$(curl -sf "$GW/api/v1/encounters?status=ACTIVE&department=ICU" \
-H "Authorization: Bearer $GATEWAY_JWT" | jq -r '.data.items[0].id') -H "Authorization: Bearer $GATEWAY_JWT" | jq -r '.data.items[0].encounterId')
if [[ -z "$ENCOUNTER_ID" || "$ENCOUNTER_ID" == "null" ]]; then if [[ -z "$ENCOUNTER_ID" || "$ENCOUNTER_ID" == "null" ]]; then
echo "No ACTIVE ICU encounter on gateway — seed replica or set GATEWAY_SYNC_JWT for sync" >&2 echo "No ACTIVE ICU encounter on gateway — seed replica or set GATEWAY_SYNC_JWT for sync" >&2
exit 1 exit 1