vigilcare-records/docs/25-gitea-cicd-docker-deploy.md

# Guide 25: Gitea CI/CD with Docker Compose

How to wire a multi-service app for continuous integration and deployment on **Gitea Actions**, using VigilCare Clinical as a concrete example. The same layout works for any stack: keep local infra in one Compose file, ship only app images in production, put secrets on the host (not in git), and let workflows build, migrate, and deploy on version tags.

Related docs in this repo:

- [`.gitea/workflows/ci.yml`](../../.gitea/workflows/ci.yml) / [`.gitea/workflows/cd.yml`](../../.gitea/workflows/cd.yml)
- [`docker-compose.yml`](../../docker-compose.yml) (local + CI dependencies)
- [`docker-compose.prod.yml`](../../docker-compose.prod.yml) (production app stack)
- [`.env.example`](../../.env.example)
- [`docs/ops/cd-deploy-ssh-setup.md`](../ops/cd-deploy-ssh-setup.md)
- [`docs/instructions-for-env.md`](../instructions-for-env.md)

---

## Mental model

| Layer | What it is | Who owns it |
|-------|------------|-------------|
| **Source** | App code, Dockerfiles, Compose files, workflow YAML | Git repo |
| **CI** | On every push/PR: start deps, build, test | Gitea Actions + `act_runner` |
| **CD** | On `v*` tags: build/push images, migrate DB, SSH deploy | Same runner + container registry |
| **Secrets on host** | Production `.env` with DB URLs, JWT keys, etc. | Deploy VM only (never committed, never SCP’d by CD) |
| **Runtime** | Pulled images + Compose prod overlay | Deploy VM (`/opt/<app>/`) |

```
Developer push/PR ──► CI (compose deps + tests)
Developer git tag v1.2.3 ──► CD
                              ├─ build & push images → Gitea registry
                              ├─ apply EF migrations (DDL user)
                              └─ SSH → pull images, up -d, smoke test
```

Gitea Actions is largely compatible with GitHub Actions syntax (`on:`, `jobs:`, `uses: actions/checkout@v4`, etc.). Jobs run on a self-hosted **act_runner** that needs Docker, curl, ssh, scp, and bash, labeled `ubuntu-latest` (or whatever label you set in the workflow).

---

## 1. Split Compose: local/CI vs production

Do **not** reuse the same Compose file for laptop and production.

### Local / CI — `docker-compose.yml`

Defines **infrastructure** (Postgres, Redis, Kafka, RabbitMQ, Elasticsearch, MinIO, …) and optional app profiles. Developers and CI start only what tests need:

```bash
docker compose up -d postgres redis rabbitmq kafka elasticsearch minio
docker compose --profile ward-gateway up -d ward-gateway-db ward-gateway-redis ward-gateway-rabbitmq
```

Useful patterns (as in this repo):

- **Published ports** so host processes (or CI job containers via `host.docker.internal`) can reach brokers.
- **Profiles** (`ward-gateway`, `full`) so optional services stay off by default.
- **CI-friendly Kafka advertising** — override advertised host for runners:

```yaml
KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://${KAFKA_EXTERNAL_HOST:-localhost}:9092
```

CI sets `KAFKA_EXTERNAL_HOST: host.docker.internal` so the test process in a container reconnects to the published port on the Docker host, not to itself.

### Production — `docker-compose.prod.yml`

Runs **only the deployable apps** (here: `api`, `gateway`, `dashboard`). Databases and brokers are assumed to already exist; Compose wires them through environment variables from `.env`.

```yaml
services:
  api:
    image: ${REGISTRY}/clinical-api:${IMAGE_TAG}
    environment:
      ConnectionStrings__DefaultConnection: "${PG_CONNECTION}"
      # … Jwt, Kafka, Redis, etc. from .env
    volumes:
      - dp_keys:/app/data-protection-keys   # durable secrets / keyrings
    networks:
      - vigilcare_prod
      - shared-services   # external network owned by infra compose
```

Principles:

1. **Image coordinates** via `REGISTRY` + `IMAGE_TAG` — CD updates only `IMAGE_TAG`.
2. **External networks** for shared Postgres/Redis stacks already running on the host.
3. **No build:** on the VM — `docker compose pull` then `up -d`.
4. **Named volumes** for anything that must survive recreate (e.g. Data Protection keys).

CD copies **only** this file to the server each release; it does not copy `.env`.

---

## 2. Dockerfiles that CI and CD can trust

### Multi-stage builds

Keep a **SDK/build** stage and a thin **runtime** stage. Example (API):

```dockerfile
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS src
# restore with project files only → cache NuGet layer
# then COPY source, publish

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS runtime
COPY --from=build /app/publish .
USER app
HEALTHCHECK CMD curl -fsS http://localhost:8080/health/live || exit 1
ENTRYPOINT ["dotnet", "VigilCareClinicalAPI.dll"]
```

Frontend (Vite) must bake public API URLs at **build** time:

```dockerfile
ARG VITE_API_URL
ENV VITE_API_URL=$VITE_API_URL
RUN npm run build
```

CD passes `--build-arg VITE_API_URL="${{ vars.PROD_API_URL }}"`.

### Build context

Match Compose and CD:

| Image | Dockerfile | Context | Why |
|-------|------------|---------|-----|
| clinical-api | `VigilCareClinicalAPI/Dockerfile` | **repo root** | Sibling `ProjectReference`s |
| ward-gateway | `VigilCare.WardGateway/Dockerfile` | **repo root** | Same |
| dashboard | `vigilcare-dashboard/Dockerfile` | `vigilcare-dashboard/` | `package.json`, `nginx.conf` |

Wrong context is the most common “works on my machine, fails in CD” failure.

### Scrub secrets from published config

Dev `appsettings.json` often contains placeholder keys. Strip them in the image so production **must** supply env vars:

```dockerfile
RUN sed -i \
      -e 's/"SigningKey": "[^"]*"/"SigningKey": ""/' \
      -e 's/"Gateway": "dev-[^"]*"/"Gateway": ""/' \
      /app/publish/appsettings.json
```

### Optional: migration target in the same Dockerfile

```dockerfile
FROM src AS migrate
RUN dotnet ef migrations bundle ... --output /out/migrate-api
```

CD builds `--target migrate`, copies the binary out, and runs it with a **DDL** connection string that never enters the API container. Prefer `docker build` + `docker cp` over `docker run -v` on act_runner — bind mounts resolve on the Docker host, not the job workspace.

### `.dockerignore`

Exclude `bin/`, `obj/`, `node_modules/`, tests, `.env`, docs noise. Keep anything the image must ship (e.g. scenario JSON under `VigilCare.Simulator/Scenarios/`).

---

## 3. `.env.example` vs production `.env`

| File | In git? | Purpose |
|------|---------|---------|
| `.env.example` | Yes | Document every key; safe placeholders |
| `.env` (laptop) | No (`.gitignore`) | Local experimentation only |
| `/opt/vigilcare/.env` on VM | No | **Source of truth** for production |

`.gitignore` pattern used here:

```
.env
.env.*
!.env.example
```

Group keys clearly in `.env.example`:

1. `REGISTRY` / `IMAGE_TAG`
2. Host ports (`API_PORT`, …)
3. External service connection strings
4. App secrets (`JWT_SIGNING_KEY`, API keys) — generate with `openssl rand -base64 48`
5. Bootstrap / seed users

**Privilege split:** runtime `PG_CONNECTION` (DML app user) vs `PG_CONNECTION_DDL` (migrator only). The DDL string is a Gitea secret for the migrate job, not an API container env var.

### One-time place `.env` on the VM

CD never uploads `.env`. Before the first tag deploy:

```bash
ssh deploy@YOUR_HOST "mkdir -p /opt/vigilcare"
scp .env deploy@YOUR_HOST:/opt/vigilcare/.env
ssh deploy@YOUR_HOST "chmod 600 /opt/vigilcare/.env"
```

Later secret rotations: edit `/opt/vigilcare/.env` on the server (or replace your secret-management process). See [`docs/instructions-for-env.md`](../instructions-for-env.md).

---

## 4. Gitea Actions — CI workflow

Path: `.gitea/workflows/ci.yml`

### Triggers

```yaml
on:
  push:
    branches: [master]
  pull_request:
    branches: [master]
```

### Backend job pattern

1. Checkout  
2. `docker compose up -d` for dependencies  
3. Wait loops (`pg_isready`, Kafka broker API, Redis `PING`, RabbitMQ diagnostics as the `rabbitmq` user — avoid root creating a bad `.erlang.cookie`)  
4. Create test databases  
5. `dotnet restore` / `build` / `test` with connection env vars pointing at `host.docker.internal` and published ports  
6. Upload test artifacts; always tear down with `docker compose ... down -v`

### Frontend job pattern

```yaml
frontend:
  runs-on: ubuntu-latest
  container:
    image: node:22-alpine
  steps:
    - uses: actions/checkout@v4
    - run: npm ci && npm run test && npm run build
      working-directory: vigilcare-dashboard
```

### Runner requirement for Compose-based tests

The job (or its Docker sibling) must reach published ports. On Docker Desktop / many Linux runners, that means `host.docker.internal` and runner `extra_hosts: host-gateway`. Wire that into your act_runner config if tests hang on “connection refused”.

---

## 5. Gitea Actions — CD workflow

Path: `.gitea/workflows/cd.yml`

### Triggers

```yaml
on:
  push:
    tags: ["v*"]
  workflow_dispatch:
    inputs:
      image_tag:
        description: "Image tag to deploy"
        required: false
```

Release flow: merge to `master` → `git tag v1.2.3 && git push origin v1.2.3`.

### Jobs

```
build-and-push ──► migrate ──► deploy (smoke + rollback on failure)
```

**build-and-push**

1. Resolve tag (`GITHUB_REF_NAME` or `workflow_dispatch` input) and optional `vars.REGISTRY`
2. `docker login` to the Gitea package registry with `REGISTRY_USERNAME` / `REGISTRY_TOKEN`
3. Build each image; tag both `:v1.2.3` and `:latest`; push

**migrate**

1. Build `--target migrate`
2. Extract `migrate-api`
3. `./migrate-api --connection "${{ secrets.PG_CONNECTION_DDL }}"`

Migrations must be **backwards-compatible** with the still-running previous image (expand-then-contract). Rollback restores the old image tag only — it does not reverse schema.

**deploy**

1. Configure SSH from `DEPLOY_SSH_KEY` (see [`cd-deploy-ssh-setup.md`](../ops/cd-deploy-ssh-setup.md))
2. `scp docker-compose.prod.yml` → `/opt/vigilcare/`
3. On the host: save previous `IMAGE_TAG`, set new tag in `.env`, `pull`, `up -d`, prune dangling images
4. Smoke: curl `/health/ready`, gateway live, dashboard `/`
5. On failure: restore `.env.previous` and `up -d` again

Do not `source .env` in bash — Compose env files are not shell (semicolons in connection strings, spaces, CRLF). Read individual keys with `sed` if needed.

---

## 6. Gitea secrets and variables

Repo → **Settings** → **Actions**

### Secrets (sensitive)

| Secret | Used by |
|--------|---------|
| `REGISTRY_USERNAME` | `docker login` |
| `REGISTRY_TOKEN` | `docker login` (access token / PAT with package write) |
| `PG_CONNECTION_DDL` | migrate job only |
| `DEPLOY_HOST` | SSH / SCP |
| `DEPLOY_USER` | SSH / SCP |
| `DEPLOY_SSH_KEY` | Private key PEM / OpenSSH private key body |

### Variables (non-secret config)

| Variable | Used by |
|----------|---------|
| `PROD_API_URL` | Dashboard image build-arg (public API origin) |
| `REGISTRY` | Optional override of default `gitea.example.com/org` |

Enable **Packages** (container registry) for the org/user that owns `REGISTRY`.

---

## 7. One-time infrastructure checklist

Use this when cloning the pattern onto a new project or a fresh Gitea instance.

### Gitea + runner

- [ ] Gitea with Actions enabled  
- [ ] act_runner registered, label matches `runs-on:` (e.g. `ubuntu-latest`)  
- [ ] Runner can run Docker (socket or DinD) and has `docker compose`, `ssh`, `scp`, `curl`, `bash`  
- [ ] Container registry reachable from runner and from the deploy host  

### Repo layout

- [ ] `.gitea/workflows/ci.yml` and `cd.yml`  
- [ ] Dockerfile(s) with runtime HEALTHCHECK  
- [ ] `docker-compose.yml` for local/CI deps  
- [ ] `docker-compose.prod.yml` for app-only deploy  
- [ ] `.env.example` + `.gitignore` excluding `.env`  
- [ ] `.dockerignore` with correct exceptions  

### Deploy host

- [ ] User with Docker rights and home for SSH keys  
- [ ] Directory e.g. `/opt/<app>/` with filled `.env` (`chmod 600`)  
- [ ] External networks / infra Compose already up if prod overlay declares `external: true`  
- [ ] Public DNS / reverse proxy pointing at published ports  
- [ ] Deploy SSH key installed ([setup guide](../ops/cd-deploy-ssh-setup.md))  

### First release

- [ ] Secrets and variables set in Gitea  
- [ ] CI green on `master`  
- [ ] Tag `v0.1.0` (or `workflow_dispatch` with `image_tag`)  
- [ ] Confirm images in registry, migrate succeeded, smoke passed  

---

## 8. Adapting this to another project

Strip VigilCare-specific names; keep the skeleton:

1. **CI:** start your test deps → wait healthy → run unit/integration tests with host-reachable URLs.  
2. **CD build:** one `docker build`/`push` per deployable service; fix contexts.  
3. **CD migrate:** optional job if you have schema (EF bundle, Flyway, Prisma migrate, etc.) using a privileged secret.  
4. **CD deploy:** SCP prod Compose → patch `IMAGE_TAG` → pull/up → health curl → rollback tag on failure.  
5. **Host `.env`:** all runtime config; CD touches only the image tag line.

Minimal prod Compose for a single API:

```yaml
services:
  api:
    image: ${REGISTRY}/my-api:${IMAGE_TAG}
    ports:
      - "${API_PORT:-8080}:8080"
    environment:
      ConnectionStrings__Default: "${PG_CONNECTION}"
    restart: unless-stopped
```

Minimal CD deploy fragment:

```bash
sed -i "s|^IMAGE_TAG=.*|IMAGE_TAG=${IMAGE_TAG}|" .env
docker compose -f docker-compose.prod.yml --env-file .env pull
docker compose -f docker-compose.prod.yml --env-file .env up -d --remove-orphans
```

---

## 9. Common pitfalls

| Symptom | Likely cause |
|---------|----------------|
| Kafka / broker tests hang in CI | Advertised listener still `localhost` inside a job container — set `KAFKA_EXTERNAL_HOST=host.docker.internal` |
| RabbitMQ dies after health probe | Probing as root created a root-owned `.erlang.cookie` — probe as `rabbitmq` |
| `docker build` missing project references | Context not repo root |
| Dashboard calls wrong API in prod | Forgot `PROD_API_URL` / `VITE_*` build-arg |
| First CD fails at `cd /opt/...` | `.env` / directory never created on VM |
| Rollback “worked” but app errors | Schema already migrated; ensure expand-then-contract migrations |
| `source .env` breaks deploy scripts | Connection strings aren’t valid bash — don’t source Compose env files |

---

## 10. Day-to-day commands

```bash
# Local deps
docker compose up -d

# Production-shaped run (on the VM, after images exist)
docker compose -f docker-compose.prod.yml --env-file .env up -d

# Cut a release (after CI is green)
git tag v1.4.0
git push origin v1.4.0

# Manual redeploy of an existing tag (Gitea UI → Actions → CD → Run workflow)
```

That is the full loop this repository uses: Compose for deps and for prod apps, env files only on the host, Gitea workflows for test and tagged releases.