# Experiment: Kafka Down During Sync Apply
Date: 2026-06-24
Phase: 24 — Climate Resilience Verification

## 1. Hypothesis
If I stop Kafka while a clinical sync batch is processed, I expect observations to
land in PostgreSQL but outbox events to remain pending, because the transactional
outbox pattern decouples durable writes from async Kafka publication.

## 2. Baseline
- Encounter: `0f81a139-9f33-47ad-9df9-09c7a6c011c8`
- Outbox pending (`processed_at IS NULL`): 0
- Central observation count: 306
- Kafka: running

## 3. Break
```bash
docker stop $(docker ps -qf name=kafka)

curl -X POST http://localhost:5270/api/v1/sync/batches \
  -H "X-Api-Key: dev-gateway-key-change-in-production" \
  -H "X-Gateway-Id: 22222222-2222-2222-2222-222222222222" \
  -H "Content-Type: application/json" \
  -d '{ "batchReference": "<uuid>", "gatewayId": "...", "siteId": "...",
        "capturedAtUtc": "<now>", "observations": [...], "alertEvents": [],
        "alertAcknowledgments": [], "alertResolutions": [] }'
```

## 4. Observe
- Batch `37618ec9-5fc8-45bf-8edd-fcfcfc47230a` reached status **APPLIED**
- Observation `kafka-exp3-1782245857` present in PostgreSQL (count = 1)
- Outbox pending events rose to **1** while Kafka was stopped
- `OutboxRelayService` could not publish `observation.recorded` until broker returned

## 5. Fix / Outcome
- Bug found? **no**
- Fix applied: no fix needed — by design (outbox holds events until Kafka heals)

## 6. Verify
(Post-heal results in `2026-06-24-kafka-down-sync-fixed.json`)