Add deployment
This commit is contained in:
voltsrage
@@ -31,7 +31,14 @@ Configuration lives in `appsettings.json` under `PhiEncryption` and `DataProtect
|
||||
| Environment | Data Protection key ring | Search HMAC key (`SearchTokenKey`) |
|
||||
|---|---|---|
|
||||
| Development | `./data-protection-keys/` on disk (`DataProtection:KeyPath`) | `appsettings.json` (dev placeholder only) |
|
||||
| Production | Azure Key Vault XML blob or AWS KMS-backed store | Key Vault / Secrets Manager secret — **not** appsettings |
|
||||
| Production | Docker named volume `vigilcare_dp_keys` mounted at `/app/data-protection-keys` (`docker-compose.prod.yml` uses `name: vigilcare`) | Host `.env` → `PHI_SEARCH_TOKEN_KEY` / secrets manager — **not** appsettings |
|
||||
|
||||
**Production custody rules (Phase 36 Step 10):**
|
||||
|
||||
1. The `dp_keys` volume is mandatory. Losing it makes every encrypted patient column permanently unreadable — a database backup alone cannot recover PHI.
|
||||
2. Back up the keyring **separately** from PostgreSQL, daily, and replicate off-host.
|
||||
3. Single API instance only with filesystem keys. Scaling past one replica requires moving to `PersistKeysToDbContext<AppDbContext>()` (or a shared store) so all instances share the ring.
|
||||
4. Keys on the volume are not encrypted at rest; ensure the host filesystem / volume store is encrypted, or add `ProtectKeysWithCertificate()` later.
|
||||
|
||||
---
|
||||
|
||||
@@ -122,6 +129,77 @@ Prometheus metric: `phi_access_logs_total{access_type="VIEW|LIST|SEARCH|CREATE|U
|
||||
|
||||
Rotating one key without the other does not require touching the other, but both rotations need a full patient re-save.
|
||||
|
||||
**`SearchTokenKey` is effectively permanent in production.** Rotating it invalidates every stored `name_search_token` and breaks patient name search until a full `encrypt-phi` re-tokenization pass completes. Prefer treating it like a root secret: generate once, store in the secrets system, never rotate casually.
|
||||
|
||||
---
|
||||
|
||||
## Production keyring backup
|
||||
|
||||
On the deploy host (after the API has started at least once and written keys into the volume):
|
||||
|
||||
```bash
|
||||
# From a checkout that includes scripts/, or copy the script to /opt/vigilcare/scripts/
|
||||
./scripts/backup-dp-keys.sh
|
||||
```
|
||||
|
||||
- Volume: `vigilcare_dp_keys` (from compose `name: vigilcare` + volume `dp_keys`)
|
||||
- Default destination: `/var/backups/vigilcare/dp-keys/dp-keys-<UTC>.tar.gz` (mode `0600`)
|
||||
- Retention: 30 days inside that directory
|
||||
- Override destination for off-host sync: `BACKUP_DIR=/mnt/offsite/vigilcare/dp-keys ./scripts/backup-dp-keys.sh`
|
||||
|
||||
Suggested cron (daily 02:15 UTC):
|
||||
|
||||
```
|
||||
15 2 * * * /opt/vigilcare/scripts/backup-dp-keys.sh >> /var/log/vigilcare-dp-backup.log 2>&1
|
||||
```
|
||||
|
||||
Replicate `/var/backups/vigilcare/dp-keys/` (or `BACKUP_DIR`) to a second site. A backup that only lives on the same disk as the volume is not a disaster-recovery backup.
|
||||
|
||||
---
|
||||
|
||||
## Keyring restore
|
||||
|
||||
Use when the volume is empty/corrupt, the host was rebuilt, or PHI decrypt fails after a redeploy.
|
||||
|
||||
```bash
|
||||
./scripts/restore-dp-keys.sh /var/backups/vigilcare/dp-keys/dp-keys-YYYYMMDDThhmmssZ.tar.gz
|
||||
```
|
||||
|
||||
The script stops the `api` service (if compose is present), extracts the archive into `vigilcare_dp_keys`, then prints the bring-up steps:
|
||||
|
||||
```bash
|
||||
docker compose -f /opt/vigilcare/docker-compose.prod.yml --env-file /opt/vigilcare/.env up -d api
|
||||
curl -fsS http://localhost:5270/health/ready
|
||||
# Then fetch a known patient and confirm firstName/lastName decrypt to plaintext.
|
||||
```
|
||||
|
||||
**Do not** invent a new empty keyring and restart the API against an existing encrypted database — that permanently orphans ciphertext.
|
||||
|
||||
---
|
||||
|
||||
## Test-restore cadence
|
||||
|
||||
A backup that has never been restored is not a backup. Cadence:
|
||||
|
||||
| Cadence | Action |
|
||||
|---|---|
|
||||
| After first production deploy | Take an immediate backup; restore into a **throwaway** Docker volume on a non-prod host (or a second named volume); start an API against a DB snapshot and decrypt one patient |
|
||||
| Quarterly | Repeat the throwaway restore drill; record date, archive name, operator, and pass/fail in the ops log |
|
||||
| Before any host migration / disk replacement | Fresh backup, then restore drill on the target host before cutting traffic |
|
||||
|
||||
Throwaway restore sketch (does not touch production volume):
|
||||
|
||||
```bash
|
||||
docker volume create vigilcare_dp_keys_drill
|
||||
docker run --rm \
|
||||
-v vigilcare_dp_keys_drill:/keys \
|
||||
-v /var/backups/vigilcare/dp-keys:/backup:ro \
|
||||
alpine tar xzf /backup/dp-keys-<stamp>.tar.gz -C /keys
|
||||
# Point a staging API at DataProtection__KeyPath=/app/data-protection-keys
|
||||
# with -v vigilcare_dp_keys_drill:/app/data-protection-keys and a DB clone.
|
||||
docker volume rm vigilcare_dp_keys_drill
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## PHI access log retention (HIPAA)
|
||||
@@ -156,10 +234,11 @@ These paths are **not** covered by column encryption in PostgreSQL:
|
||||
### `CryptographicException` / cannot decrypt patient
|
||||
|
||||
- Key ring missing or wrong `DataProtection:KeyPath`
|
||||
- App deployed to new host without copying `data-protection-keys/`
|
||||
- App deployed to new host without restoring `vigilcare_dp_keys` (or copying `data-protection-keys/`)
|
||||
- Production compose ran without the `dp_keys` volume — container regenerated a new empty ring
|
||||
- `ProtectorPurpose` changed without re-running `encrypt-phi`
|
||||
|
||||
**Fix:** Restore key ring from backup. Do not delete old keys until all data is re-encrypted.
|
||||
**Fix:** Restore key ring from backup (`scripts/restore-dp-keys.sh`). Do not delete old keys until all data is re-encrypted.
|
||||
|
||||
### Name search returns no results
|
||||
|
||||
@@ -194,5 +273,9 @@ Migration `WidenPhiEncryptedColumns` widens `first_name`, `last_name`, and emerg
|
||||
| `VigilCareClinicalAPI/Commands/EncryptPhiCommand.cs` | Bulk re-save CLI |
|
||||
| `VigilCareClinicalAPI/BackgroundServices/PatientPhiMigrationService.cs` | Startup token backfill |
|
||||
| `scripts/encrypt-existing-patient-phi.sh` | Wrapper for encrypt CLI |
|
||||
| `scripts/backup-dp-keys.sh` | Daily backup of `vigilcare_dp_keys` |
|
||||
| `scripts/restore-dp-keys.sh` | Restore keyring archive into the Docker volume |
|
||||
| `scripts/run-phase32-verification.sh` | End-to-end verification |
|
||||
| `docker-compose.prod.yml` | Mounts `dp_keys` → `/app/data-protection-keys` |
|
||||
| `docs/plans/phase-32-plan.md` | Implementation plan and design rationale |
|
||||
| `docs/plans/vigilcare-clinical-deployment-plan.md` | Phase 36 deployment (Step 10) |
|
||||
|
||||
Reference in New Issue
Block a user