Add deployment

2026-08-05 00:26:20 +08:00
parent 9e88ff6113
commit 2a3ef62a7d
86 changed files with 2320 additions and 174 deletions
@@ -0,0 +1,60 @@
+# Production observability wiring (Phase 36 Step 9)
+#
+# Local-dev Prometheus/Grafana/Seq remain in docker-compose.yml + infra/.
+# Production uses *existing external* instances — do not start those services
+# from docker-compose.prod.yml.
+
+## Prometheus
+
+1. Keep [infra/prometheus/prometheus.yml](../infra/prometheus/prometheus.yml) as the
+   local-dev scrape of `host.docker.internal:5270`. Do not add production jobs there.
+2. On the external Prometheus host, merge
+   [infra/prometheus/production/scrape-vigilcare.yml](../infra/prometheus/production/scrape-vigilcare.yml)
+   into `scrape_configs`.
+3. Load
+   [infra/prometheus/production/alert-rules.yml](../infra/prometheus/production/alert-rules.yml)
+   via `rule_files`.
+4. Optionally enable the commented blackbox probe for `/health/ready` (same scrape file).
+
+### `/metrics` access
+
+The Clinical API uses a global `FallbackPolicy` that requires an authenticated user.
+`/metrics` is explicitly `.AllowAnonymous()` so Prometheus can scrape without a JWT.
+**Restrict at the network layer** — firewall so only the Prometheus host (and
+operators) can reach `API_PORT` /metrics. Metric labels include department and
+gateway identifiers.
+
+## Grafana
+
+Prefer file provisioning over UI import (UI imports are lost on Grafana redeploy):
+
+1. Copy datasources:
+   [infra/grafana/production/datasources.yml](../infra/grafana/production/datasources.yml)
+   — keep `uid: prometheus` so existing dashboards keep working.
+2. Copy dashboards from [infra/grafana/dashboards/](../infra/grafana/dashboards/) to the
+   path referenced by
+   [infra/grafana/production/dashboards-provider.yml](../infra/grafana/production/dashboards-provider.yml).
+3. Confirm panels render against the external Prometheus (job labels
+   `vigilcare_api_prod` / `environment=production`).
+
+Minimum production alerts (also in Prometheus rules): API down, outbox backlog,
+Kafka consumer lag, ready probe failing, gateway offline, unacked CRITICAL alerts.
+
+## Seq
+
+1. Point production at Seq via env (compose already sets these):
+   - `Seq__ServerUrl` / `Serilog__WriteTo__1__Args__serverUrl` → `SEQ_URL`
+   - `Serilog__WriteTo__1__Args__apiKey` → `SEQ_API_KEY` (ingest-only key)
+2. `appsettings.Production.json` sets
+   `Microsoft.EntityFrameworkCore.Database.Command` to **Warning** so SQL with
+   patient identifiers is not shipped to Seq.
+3. Serilog `Properties:Application` is set to `VigilCareClinicalAPI` /
+   `VigilCare.WardGateway` for filtering.
+4. In Seq, create a signal (or shared dashboard) approximately:
+
+   ```
+   Application = 'VigilCareClinicalAPI' and @Level in ['Error', 'Fatal']
+   ```
+
+   Optionally a second signal for the ward gateway with
+   `Application = 'VigilCare.WardGateway'`.