50 lines
1.9 KiB
Markdown
50 lines
1.9 KiB
Markdown
# Experiment: Central API Down 30 Minutes
|
|
Date: 2026-06-24
|
|
Phase: 24 — Climate Resilience Verification
|
|
|
|
## 1. Hypothesis
|
|
If I stop the central API while the ward gateway continues running and replay
|
|
`ward-outage-reconnect-01` against the gateway, I expect critical potassium
|
|
observations to alert locally on the gateway with zero new rows on central,
|
|
because Tier 1 safety is ward-local and sync is deferred until reconnect.
|
|
|
|
## 2. Baseline
|
|
- Encounter: `0f81a139-9f33-47ad-9df9-09c7a6c011c8`
|
|
- Gateway status: ONLINE
|
|
- Gateway buffer depth: 0
|
|
- Central observation count (encounter): 7 (Phase A replay complete)
|
|
|
|
## 3. Break
|
|
```bash
|
|
SKIP_DOCKER=1 ./scripts/run-phase24-verification.sh
|
|
# Internally: pkill central VigilCareClinicalAPI process, wait ~35s for gateway
|
|
# to detect central unreachable, then replay scenario against gateway.
|
|
```
|
|
|
|
Manual equivalent:
|
|
```bash
|
|
pkill -f VigilCareClinicalAPI
|
|
sleep 35
|
|
dotnet run --project VigilCare.Simulator -- replay \
|
|
VigilCare.Simulator/Scenarios/List/ward-outage-reconnect-01.json \
|
|
--gateway --encounter-id 0f81a139-9f33-47ad-9df9-09c7a6c011c8 \
|
|
--gateway-token "$GATEWAY_JWT" --speed 0
|
|
```
|
|
|
|
## 4. Observe
|
|
- Gateway created `CriticalPotassiumMeqL` alert at scenario T+45 min (K+ 6.8 mEq/L)
|
|
- RN-Wu ack recorded locally at T+50 min (`status: Acknowledged`, `acknowledgedBy: RN-Wu`)
|
|
- Central API unreachable during outage — no new central observations ingested
|
|
- Gateway buffer depth (unsynced `buffered_sync_items`): **10**
|
|
- Phase B replay sent 7 observations, 1 critical alert, 1 ack (plus warning-tier alert)
|
|
- Gateway local alerts after Phase B:
|
|
- `CriticalPotassiumMeqL` — Acknowledged by RN-Wu
|
|
- `WarningPotassiumMeqL` — Open
|
|
|
|
## 5. Fix / Outcome
|
|
- Bug found? **no** — behavior matches design (local Tier 1 path + deferred sync)
|
|
- Fix applied: no fix needed — by design
|
|
|
|
## 6. Verify
|
|
(Post-reconnect results captured in `2026-06-24-central-down-fixed.json`)
|