Trent/vigilcare-clinical
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
vigilcare-clinical/docs/docker-compose-usage-and-troubleshooting.md
T

9.6 KiB
Raw Permalink Blame History

Docker & Docker Compose Guide (VigilCare)

This guide is a practical reference for running this project with Docker, plus troubleshooting for common issues seen in this repo.

It is written for junior developers, so each section explains not just what to do, but why.

1) Quick start

From repo root:

docker compose up -d

Ward gateway services (ward-gateway-db, ward-gateway-redis, ward-gateway-rabbitmq, ward-gateway-api) use Compose profiles and do not start with the command above. Use --profile ward-gateway or see §10.

Check status:

docker compose ps

Stop everything (keep data):

docker compose stop

Start again:

docker compose start

2) Core concepts (simple mental model)

Container

  • A running process with its own filesystem and network namespace.
  • Example: vigilcare_prometheus is one container.

Service (in docker-compose.yml)

  • A recipe for how to run a container.
  • Example: the prometheus: section defines image, ports, volumes, networks.

Image

  • Template used to create a container.
  • Example: prom/prometheus:v2.52.0.

Volume

  • Persistent storage managed by Docker.
  • Survives container restarts/recreates.
  • Example: prometheus_data, grafana_data, pg_data.

Network

  • Virtual network connecting containers.
  • Containers can reach each other by service name (DNS).
  • Example: Grafana reaches Prometheus at http://prometheus:9090 inside Docker.

3) Host ports vs container ports

In Compose, this format is used:

ports:
  - "HOST:CONTAINER"

Example from this project:

  • Prometheus: "9101:9090"
    • Open in browser with http://localhost:9101
    • Inside Docker, service still listens on 9090
  • Grafana: "3101:3000"
    • Open in browser with http://localhost:3101

If a UI is not loading, first verify host port mappings in docker-compose.yml.

4) Project networking (vigilcare_net)

This project uses a user-defined bridge network:

networks:
  vigilcare_net:
    driver: bridge

All services should join it:

networks:
  - vigilcare_net

Why this matters:

  • Service-to-service DNS works (prometheus, grafana, postgres, etc.).
  • Keeps local environment predictable.

Important Linux note: host.docker.internal

Prometheus scrapes the API via host address in this project:

  • http://host.docker.internal:5270/metrics

On Linux, host.docker.internal may not resolve by default.
Fix by adding this to the prometheus service:

extra_hosts:
  - "host.docker.internal:host-gateway"

Then recreate Prometheus:

docker compose up -d --force-recreate prometheus

Symptom when missing:

  • Prometheus target vigilcare_api is down
  • Error: lookup host.docker.internal ... no such host

Important: API must listen on all interfaces (not only localhost)

After extra_hosts is fixed, Prometheus may still show:

dial tcp 172.17.0.1:5270: connect: connection refused

Why: dotnet run with http://localhost:5270 binds only to 127.0.0.1.
localhost inside a container means the container itself, not your host machine.
Prometheus inside Docker reaches the host via the gateway IP (172.17.0.1 via host.docker.internal), not host loopback.

Check binding:

ss -tlnp | rg ':5270'

If you see 127.0.0.1:5270, Prometheus cannot scrape from Docker.

Fix (local dev): bind on all interfaces in VigilCareClinicalAPI/Properties/launchSettings.json:

"applicationUrl": "http://0.0.0.0:5270"

Or start the API with:

ASPNETCORE_URLS=http://0.0.0.0:5270 dotnet run --project VigilCareClinicalAPI

Then restart the API and confirm:

ss -tlnp | rg ':5270'    # should show 0.0.0.0:5270
curl -sS http://localhost:5270/metrics | head

Security note: 0.0.0.0 is fine for local development. In production, bind explicitly and use proper network controls.

5) Volumes and persistence

This repo uses named volumes for persistent data:

  • pg_data
  • seq_data
  • kafka_data
  • es_data
  • minio_data
  • prometheus_data
  • grafana_data
  • ward_pg_data (ward gateway PostgreSQL — only created when ward profile is used)

Why your data still exists after restart

  • docker compose up -d --force-recreate recreates containers, but volumes remain.
  • This is expected and usually desired.

Full reset (destructive)

If you need a totally clean environment:

docker compose down -v

Warning:

  • -v removes named volumes (database/log/index data lost).

6) Common commands and when to use them

Apply config change to one service

Use when you changed only one section (e.g., Prometheus extra_hosts):

docker compose up -d --force-recreate prometheus

Restart service without recreate

Use when config did not change and you just want a restart:

docker compose restart prometheus

Rebuild image service

Use when Dockerfile/app code in image changed:

docker compose up -d --build <service>

View service logs

docker compose logs -f prometheus
docker compose logs -f grafana

7) Troubleshooting playbook

A) “Service is up but endpoint won’t open”

  1. Check container state: 
    docker compose ps
  2. Verify port mapping in docker-compose.yml.
  3. Check logs: 
    docker compose logs --tail=100 <service>

B) “Prometheus healthy, but target is DOWN”

  1. Open Prometheus targets page:
    • http://localhost:9101/targets
  2. Read the exact lastError.
  3. If error mentions host.docker.internal on Linux:
    • add extra_hosts fix (section 4)
    • recreate Prometheus.
  4. If error is connection refused to 172.17.0.1:5270:
    • API is likely bound to 127.0.0.1 only
    • use http://0.0.0.0:5270 and restart API (section 4).

C) “Docker compose command cannot connect to daemon”

Example:

  • failed to connect to the docker API at unix:///var/run/docker.sock

Fix:

  • Start Docker Desktop / Docker daemon.
  • Re-run docker compose ps.

D) “Permission denied writing files under bind-mounted folder”

This can happen when directories/files were created as root.

Symptoms:

  • Cannot create/edit files in folders like Grafana dashboard path.

Fix options:

  1. Correct ownership on host: 
    sudo chown -R $USER:$USER <folder>
  2. Recreate problematic directory as your user.

E) “Script fails preflight even though services seem running”

Check these directly:

curl -sS -o /dev/null -w "%{http_code}\n" http://localhost:5270/api/v1/alert-thresholds
curl -sS -o /dev/null -w "%{http_code}\n" http://localhost:9101/-/healthy
curl -sS -o /dev/null -w "%{http_code}\n" http://localhost:5345

Expected: 200, 200, 200.

8) Current project-specific paths and notes

  • Compose file: docker-compose.yml
  • Prometheus config: infra/prometheus/prometheus.yml
  • Grafana provisioning:
    • infra/grafana/provisioning/datasources/prometheus.yml
    • infra/grafana/provisioning/datasources/dashboards/config.yml
  • Grafana dashboards expected path:
    • infra/grafana/dashboards/

Note: if you accidentally create a typo folder like dashbpards, Grafana provisioning will not load dashboards from it.

9) Safe workflow for config changes (recommended)

  1. Edit docker-compose.yml.

  2. Recreate only changed services:

    docker compose up -d --force-recreate <service>

  3. Verify logs and health endpoints.

  4. Run project verification scripts (examples):

    ./scripts/run-phase8-verification.sh
./scripts/run-phase9-verification.sh

    Phase 9 also benefits from the MinIO client (mc) and DuckDB CLI for object and Parquet checks. Install without sudo — see docs/plans/phase-9-plan.md.

This avoids unnecessary full resets and speeds up local development.

10) Compose profiles (ward gateway stack)

Several services in docker-compose.yml declare a profile so they are optional:

profiles: ["ward-gateway", "full"]

Affected services: ward-gateway-db, ward-gateway-redis, ward-gateway-rabbitmq, ward-gateway-api.

Why ward services do not start with docker compose up

docker compose up (no --profile) only starts services without a profile. The central stack (postgres, redis, kafka, etc.) has no profile and starts normally. Ward services are gated behind ward-gateway or full.

Symptom: docker compose ps shows no ward-gateway-* containers after a plain docker compose up -d.

How to start the ward stack

# Ward gateway infrastructure + API (port 5081)
docker compose --profile ward-gateway up -d

# Central + ward together
docker compose --profile full up -d

Persist the profile in your environment so you do not need the flag every time:

export COMPOSE_PROFILES=ward-gateway
# or add to a repo-root .env file: COMPOSE_PROFILES=ward-gateway

Ward service ports (host)

Service Host port Purpose
ward-gateway-db 5437 PostgreSQL vigilcare_ward
ward-gateway-redis 6383 Threshold cache
ward-gateway-rabbitmq 5675 / 15675 Local paging (AMQP / management UI)
ward-gateway-api 5081 Gateway HTTP API

Connection strings in VigilCare.WardGateway/appsettings.json use these host ports when running dotnet run locally against Dockerized dependencies.

Verification

docker compose --profile ward-gateway ps
curl -sS -o /dev/null -w "%{http_code}\n" http://localhost:5081/health/ready

Phase 21 verification script already uses the correct profile:

./scripts/run-phase21-verification.sh
# internally: docker compose --profile ward-gateway up -d
Reference in New Issue View Git Blame Copy Permalink