Files
vigilcare-clinical/docs/resilience/climate/2026-06-24-central-down-broken.md
T
2026-06-24 04:35:15 +08:00

1.9 KiB

Experiment: Central API Down 30 Minutes

Date: 2026-06-24 Phase: 24 — Climate Resilience Verification

1. Hypothesis

If I stop the central API while the ward gateway continues running and replay ward-outage-reconnect-01 against the gateway, I expect critical potassium observations to alert locally on the gateway with zero new rows on central, because Tier 1 safety is ward-local and sync is deferred until reconnect.

2. Baseline

  • Encounter: 0f81a139-9f33-47ad-9df9-09c7a6c011c8
  • Gateway status: ONLINE
  • Gateway buffer depth: 0
  • Central observation count (encounter): 7 (Phase A replay complete)

3. Break

SKIP_DOCKER=1 ./scripts/run-phase24-verification.sh
# Internally: pkill central VigilCareClinicalAPI process, wait ~35s for gateway
# to detect central unreachable, then replay scenario against gateway.

Manual equivalent:

pkill -f VigilCareClinicalAPI
sleep 35
dotnet run --project VigilCare.Simulator -- replay \
  VigilCare.Simulator/Scenarios/List/ward-outage-reconnect-01.json \
  --gateway --encounter-id 0f81a139-9f33-47ad-9df9-09c7a6c011c8 \
  --gateway-token "$GATEWAY_JWT" --speed 0

4. Observe

  • Gateway created CriticalPotassiumMeqL alert at scenario T+45 min (K+ 6.8 mEq/L)
  • RN-Wu ack recorded locally at T+50 min (status: Acknowledged, acknowledgedBy: RN-Wu)
  • Central API unreachable during outage — no new central observations ingested
  • Gateway buffer depth (unsynced buffered_sync_items): 10
  • Phase B replay sent 7 observations, 1 critical alert, 1 ack (plus warning-tier alert)
  • Gateway local alerts after Phase B:
    • CriticalPotassiumMeqL — Acknowledged by RN-Wu
    • WarningPotassiumMeqL — Open

5. Fix / Outcome

  • Bug found? no — behavior matches design (local Tier 1 path + deferred sync)
  • Fix applied: no fix needed — by design

6. Verify

(Post-reconnect results captured in 2026-06-24-central-down-fixed.json)