vigilcare-clinical/docs/guides/04-postgresql-entity-framework-core.md

Guide 4: PostgreSQL with Entity Framework Core

What Are PostgreSQL and Entity Framework Core?

PostgreSQL (often called "Postgres") is an open-source relational database — it stores data in tables with rows and columns, and you query it using SQL. If you've used MySQL or SQL Server, PostgreSQL works similarly but offers advanced features like JSONB columns (storing JSON data that you can query), partial indexes (indexes that only cover some rows), and robust support for concurrent transactions.

Entity Framework Core (EF Core) is an ORM — an Object-Relational Mapper. Without an ORM, you'd write raw SQL strings in your C# code, manually map database columns to C# properties, and handle connection management yourself. An ORM lets you work with database rows as if they were regular C# objects:

// Without ORM — raw SQL, manual mapping
var sql = "SELECT id, encounter_id, value FROM observations WHERE encounter_id = @id";
// ... execute, read columns, create objects manually

// With EF Core — C# objects, LINQ queries
var observations = await db.Observations
    .Where(o => o.EncounterId == encounterId)
    .ToListAsync();

EF Core translates your LINQ queries into SQL, maps the results back to C# objects, tracks changes, and generates database migrations (versioned schema changes).

---

Why PostgreSQL + EF Core in This Project?

PostgreSQL is the primary relational database for all clinical data — patients, encounters, observations, alerts, scores, audit logs, and the transactional outbox. EF Core provides the ORM layer that maps C# entities to PostgreSQL tables, handles migrations, and generates SQL while allowing raw SQL when needed (like FOR UPDATE SKIP LOCKED for the outbox relay).

---

Architecture Overview

Application Code
      │
      │  _db.Observations.Add(...)
      │  _db.SaveChangesAsync()
      ▼
  AppDbContext (EF Core)
      │
      ├── Entity Configurations (IEntityTypeConfiguration<T>)
      │   ├── Column mappings (snake_case)
      │   ├── Check constraints
      │   ├── Indexes (unique, partial, composite)
      │   └── Value conversions (enum ↔ string, PHI encryption)
      │
      ├── Migrations (40+ tracked schema changes)
      │
      └──► PostgreSQL 16
           ├── JSONB columns (audit logs, staleness flags, alert explanations)
           ├── Partial unique indexes (idempotency, active encounters)
           ├── Sequences (MRN generation)
           └── FOR UPDATE SKIP LOCKED (outbox concurrency)

---

The DbContext

What is a DbContext? The DbContext is the main class you interact with in EF Core. It represents a session with the database — you use it to query data, add new records, and save changes. Think of it as a "database connection wrapper" that knows about your tables and how your C# classes map to them.

Each DbSet<T> property represents one table. DbSet<Patient> means "the patients table, where each row maps to a Patient C# object."

AppDbContext is this project's DbContext. It declares all 24 DbSet<T> properties and loads entity configurations from the assembly:

public class AppDbContext : DbContext
{
    private readonly IPhiEncryptionService? _phiCrypto;

    public AppDbContext(
        DbContextOptions<AppDbContext> options,
        IPhiEncryptionService? phiCrypto = null) : base(options)
    {
        _phiCrypto = phiCrypto;
    }

    public DbSet<Patient> Patients => Set<Patient>();
    public DbSet<Encounter> Encounters => Set<Encounter>();
    public DbSet<Observation> Observations => Set<Observation>();
    public DbSet<ClinicalAlert> ClinicalAlerts => Set<ClinicalAlert>();
    public DbSet<OutboxEvent> OutboxEvents => Set<OutboxEvent>();
    public DbSet<Order> Orders => Set<Order>();
    public DbSet<News2Score> News2Scores => Set<News2Score>();
    public DbSet<GcsScore> GcsScores => Set<GcsScore>();
    public DbSet<SofaScore> SofaScores => Set<SofaScore>();
    public DbSet<QsofaEvaluation> QsofaEvaluations => Set<QsofaEvaluation>();
    public DbSet<SepsisBundle> SepsisBundles => Set<SepsisBundle>();
    public DbSet<SepsisBundleElement> SepsisBundleElements => Set<SepsisBundleElement>();
    public DbSet<MedicationAdministration> MedicationAdministrations => Set<MedicationAdministration>();
    public DbSet<AlertThreshold> AlertThresholds => Set<AlertThreshold>();
    public DbSet<ClinicalAuditLog> ClinicalAuditLogs => Set<ClinicalAuditLog>();
    public DbSet<ClinicalUser> ClinicalUsers => Set<ClinicalUser>();
    public DbSet<ExternalResourceIdentifier> ExternalResourceIdentifiers => Set<ExternalResourceIdentifier>();
    public DbSet<PhiAccessLog> PhiAccessLogs => Set<PhiAccessLog>();
    public DbSet<ClinicalSite> ClinicalSites => Set<ClinicalSite>();
    public DbSet<WardGateway> WardGateways => Set<WardGateway>();
    public DbSet<ClinicalSyncBatch> ClinicalSyncBatches => Set<ClinicalSyncBatch>();
    public DbSet<ClinicalSyncConflict> ClinicalSyncConflicts => Set<ClinicalSyncConflict>();
    public DbSet<AlertFeedback> AlertFeedbacks => Set<AlertFeedback>();
    public DbSet<AlertQualityMetric> AlertQualityMetrics => Set<AlertQualityMetric>();
    public DbSet<ReconciliationAlert> ReconciliationAlerts => Set<ReconciliationAlert>();

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.ApplyConfigurationsFromAssembly(typeof(AppDbContext).Assembly);

        if (_phiCrypto is not null)
            modelBuilder.ConfigurePhiConverters(_phiCrypto);
    }
}

Key design decisions:

• ApplyConfigurationsFromAssembly automatically finds all entity configuration classes in the project (explained below) instead of registering them one by one
• Optional IPhiEncryptionService — PHI stands for Protected Health Information (patient names, dates of birth, etc.). When this service is provided, EF Core automatically encrypts PHI columns when writing to the database and decrypts when reading. Integration tests pass null to skip encryption for simpler test setup.
• Expression-body DbSets (=> Set<T>()) — a concise C# syntax. This is functionally the same as a property with a getter

Registration in Program.cs

builder.Services.AddDbContext<AppDbContext>((sp, opts) =>
{
    opts.UseNpgsql(builder.Configuration.GetConnectionString("DefaultConnection"));
});

Connection string from appsettings.json:

{
  "ConnectionStrings": {
    "DefaultConnection": "Host=localhost;Port=5436;Database=vigilcare;Username=postgres;Password=password"
  }
}

Port 5436 matches the Docker Compose mapping (host 5436 → container 5432).

---

Entity Configurations

What is an entity configuration? EF Core needs to know how your C# classes map to database tables — which property maps to which column, what the column type is, which columns have indexes, etc. You configure this by creating a class that implements IEntityTypeConfiguration<T> for each entity.

Each entity has a dedicated configuration class in Data/Configurations/. This keeps the DbContext clean and puts all the schema details for one table in one file.

Snake_Case Column Mapping

PostgreSQL convention is snake_case (like encounter_id). C# convention is PascalCase (like EncounterId). Every column is explicitly mapped to bridge the two naming styles:

builder.Property(o => o.EncounterId).HasColumnName("encounter_id");
builder.Property(o => o.ObservationCode).HasColumnName("observation_code")
    .HasMaxLength(50).IsRequired();
builder.Property(o => o.Value).HasColumnName("value")
    .HasColumnType("decimal(10,3)").IsRequired();
builder.Property(o => o.CreatedAt).HasColumnName("created_at")
    .HasDefaultValueSql("NOW()");

UUID Primary Keys

All entities use Guid primary keys, generated by PostgreSQL:

builder.Property(p => p.Id).HasColumnName("id")
    .HasDefaultValueSql("gen_random_uuid()");

gen_random_uuid() generates UUIDs server-side, so the application doesn't need to generate them unless it wants to (which it does for outbox events and alerts, where the ID is needed before the INSERT).

Enum-to-String Conversions

What is a value conversion? EF Core can automatically convert between your C# type and the database type when reading and writing. This is configured with HasConversion().

In C#, enums are typically stored as integers internally (Critical = 0, Warning = 1). But if you store those integers in the database, the data is unreadable without the code ("what does status 2 mean?"), and renumbering the enum breaks everything. Instead, this project stores enums as human-readable strings:

builder.Property(a => a.Severity)
    .HasColumnName("severity")
    .HasMaxLength(20)
    .HasConversion(
        v => v.ToDbString(),       // AlertSeverity.Critical → "CRITICAL"
        v => AlertSeverityExtensions.FromDbString(v))  // "CRITICAL" → AlertSeverity.Critical
    .IsRequired();

The ToDbString() / FromDbString() pattern is used for all enums: AlertStatus, AlertType, AlertSeverity, ObservationSource, AuditAction, etc.

JSONB Columns

What is JSONB? PostgreSQL can store JSON data in two column types: json (stored as text) and jsonb (stored as binary, pre-parsed JSON). JSONB is faster to query and can be indexed. It's useful for data that has a flexible or evolving structure — you don't need to define rigid columns for every possible field.

JSONB is a good fit for data that doesn't need relational querying (JOINs, foreign keys) but benefits from being stored alongside the row:

// Audit log — before/after snapshots as JSON
builder.Property(a => a.PreviousValueJson).HasColumnName("previous_value_json")
    .HasColumnType("jsonb");
builder.Property(a => a.NewValueJson).HasColumnName("new_value_json")
    .HasColumnType("jsonb");

// SOFA score — staleness tracking per organ
builder.Property(s => s.StalenessFlags).HasColumnName("staleness_flags")
    .HasColumnType("jsonb");

// Alert explanation — structured clinical reasoning
builder.Property(a => a.Explanation)
    .HasColumnName("explanation")
    .HasColumnType("jsonb")
    .HasConversion(
        v => v == null ? null : JsonSerializer.Serialize(v, JsonOptions),
        v => string.IsNullOrEmpty(v) ? null : JsonSerializer.Deserialize<AlertExplanation>(v, JsonOptions)!);

JSONB is stored as binary JSON in PostgreSQL — it's indexed for fast access and supports @> containment queries, though this project reads it as opaque blobs in most cases.

---

Check Constraints

What is a check constraint? A check constraint is a rule the database enforces on every INSERT and UPDATE. If the data violates the rule, the database rejects the operation with an error. This is a safety net — even if there's a bug in your application code, the database won't allow invalid data.

Think of it as a bouncer at the door: "you can only enter if your severity is 'WARNING' or 'CRITICAL' — anything else gets rejected."

builder.ToTable("observations", t =>
{
    t.HasCheckConstraint("chk_observations_source",
        "source IN ('MANUAL', 'DEVICE', 'LAB')");
});

builder.ToTable("clinical_alerts", t =>
{
    t.HasCheckConstraint("chk_clinical_alerts_severity",
        "severity IN ('WARNING', 'CRITICAL')");
    t.HasCheckConstraint("chk_clinical_alerts_status",
        "status IN ('OPEN', 'ACKNOWLEDGED', 'RESOLVED', 'ESCALATED')");
    t.HasCheckConstraint("chk_clinical_alerts_alert_type",
        "alert_type IN ('SEPSIS_WARNING', 'CRITICAL_HEART_RATE', ...)");
});

builder.ToTable("sepsis_bundles", t =>
{
    t.HasCheckConstraint("chk_sepsis_bundles_compliance_status",
        "compliance_status IN ('IN_PROGRESS', 'COMPLIANT', 'NON_COMPLIANT')");
});

These constraints are created by EF Core migrations and enforced by PostgreSQL on every INSERT and UPDATE.

---

Indexing Strategies

What is a database index? An index is a data structure that speeds up lookups, like the index at the back of a textbook. Without an index, the database has to scan every row in the table to find what you're looking for (a "full table scan"). With an index, it can jump directly to the matching rows. The tradeoff: indexes use extra disk space and slow down writes slightly (the index needs updating too).

What is a unique index? A unique index does two things: it speeds up lookups AND it prevents duplicate values. If you try to insert a row that would create a duplicate, the database rejects it with an error.

Unique Indexes

Simple uniqueness constraints:

// One MRN per patient
builder.HasIndex(p => p.Mrn).IsUnique();

// One external identifier per resource type + system + value
builder.HasIndex(e => new { e.ResourceType, e.System, e.Value }).IsUnique();

Partial Unique Indexes

What is a partial index? A standard index covers every row in the table. A partial index only covers rows that match a filter condition (the HasFilter() clause). This is a PostgreSQL-specific feature (not available in all databases) and is useful when you only need uniqueness or fast lookups on a subset of rows:

// Only non-null idempotency keys are deduplicated.
// Devices that don't send a key are not subject to deduplication.
builder.HasIndex(o => o.IdempotencyKey)
    .IsUnique()
    .HasFilter("idempotency_key IS NOT NULL");

// Only one gateway-synced alert per client_alert_id
builder.HasIndex(a => a.ClientAlertId)
    .IsUnique()
    .HasFilter("client_alert_id IS NOT NULL");

// Efficient baseline lookup for SOFA scoring
builder.HasIndex(s => s.EncounterId)
    .HasFilter("is_baseline = true")
    .HasDatabaseName("idx_sofa_scores_baseline");

The observation idempotency index is critical for safety — it prevents duplicate observations from device retries without requiring every observation to have an idempotency key.

Composite Indexes

A composite index covers multiple columns together. This is useful when your queries always filter on a combination of columns. The index on (EncounterId, ObservationCode, RecordedAt) speeds up queries like "get all heart rate observations for encounter X, sorted by time":

// Observations queried by encounter + code + time
builder.HasIndex(o => new { o.EncounterId, o.ObservationCode, o.RecordedAt });

// Alerts queried by encounter + time, patient + time
builder.HasIndex(a => new { a.EncounterId, a.TriggeredAt });
builder.HasIndex(a => new { a.PatientId, a.TriggeredAt });

// Open alerts by severity (for the unacknowledged collector)
builder.HasIndex(a => new { a.Severity, a.TriggeredAt })
    .HasFilter("status = 'OPEN'");

// Active alerts by encounter + type (for duplicate prevention)
builder.HasIndex(a => new { a.EncounterId, a.AlertType, a.ObservationCode })
    .HasFilter("status IN ('OPEN', 'ESCALATED')");

Append-Only Tables

Audit logs are never updated or deleted — only inserted:

// clinical_audit_logs — indexes for querying, no update patterns
builder.HasIndex(a => a.EntityType);
builder.HasIndex(a => a.EntityId);
builder.HasIndex(a => a.UserId);
builder.HasIndex(a => a.CreatedAt);

---

PHI Encryption with Value Converters

What is PHI? Protected Health Information — any data that could identify a patient (names, dates of birth, medical records). Healthcare regulations require PHI to be encrypted "at rest" (when stored on disk).

EF Core value converters can automatically encrypt data when writing to the database and decrypt when reading. Your application code works with plaintext strings as usual — the encryption is invisible:

public static class PatientPhiConverterConfigurator
{
    public static void ConfigurePhiConverters(
        this ModelBuilder modelBuilder, IPhiEncryptionService crypto)
    {
        var entity = modelBuilder.Entity<Patient>();

        entity.Property(p => p.FirstName)
            .HasColumnType("text")
            .HasConversion(
                v => crypto.Encrypt(v),       // encrypt on write
                v => crypto.Decrypt(v));       // decrypt on read

        entity.Property(p => p.LastName)
            .HasColumnType("text")
            .HasConversion(
                v => crypto.Encrypt(v),
                v => crypto.Decrypt(v));

        // DateOfBirth stored as encrypted ISO string
        entity.Property(p => p.DateOfBirth)
            .HasConversion(
                v => crypto.Encrypt(v.ToString("yyyy-MM-dd")),
                v => DateOnly.Parse(crypto.Decrypt(v)));
    }
}

This is transparent to application code — patient.FirstName always returns the plaintext value. The database stores ciphertext. The NameSearchToken column (HMAC-based) enables searching encrypted names without decrypting every row.

---

Relationship Configuration

All relationships use DeleteBehavior.Restrict to prevent accidental cascade deletes in a patient safety system:

builder.HasOne(o => o.Encounter)
    .WithMany(e => e.Observations)
    .HasForeignKey(o => o.EncounterId)
    .OnDelete(DeleteBehavior.Restrict);

builder.HasOne(b => b.Encounter)
    .WithMany()
    .HasForeignKey(b => b.EncounterId)
    .OnDelete(DeleteBehavior.Restrict);

builder.HasOne(b => b.TriggeringAlert)
    .WithMany()
    .HasForeignKey(b => b.TriggeringAlertId)
    .OnDelete(DeleteBehavior.Restrict);

---

Transactions and Atomicity

What is a transaction? A database transaction groups multiple operations into one atomic unit — either ALL of them succeed, or NONE of them do. If anything fails midway (power outage, constraint violation, application crash), the database rolls back all changes as if nothing happened. This is critical when you need to ensure consistency — for example, you never want an alert to be created without its corresponding observation.

Outbox Pattern Transaction

The observation ingest writes the observation, the outbox event, and optionally a critical alert in one transaction:

await using var tx = await _db.Database.BeginTransactionAsync();

_db.Observations.Add(observation);
_db.OutboxEvents.Add(BuildOutboxEvent("observation.recorded", ...));

if (IsCriticalBreach(req.Value, threshold))
{
    _db.ClinicalAlerts.Add(alert);
    _db.OutboxEvents.Add(BuildOutboxEvent("alert.generated", ...));
}

await _db.SaveChangesAsync();
await tx.CommitAsync();

If anything fails, the entire batch rolls back — no orphaned alerts or lost observations.

FOR UPDATE SKIP LOCKED

What is row locking? When two processes try to update the same database row simultaneously, you get a race condition. PostgreSQL's FOR UPDATE clause locks the selected rows so no other transaction can modify them until you're done. SKIP LOCKED is an additional modifier that says "if another transaction already locked some rows, skip those instead of waiting." This lets multiple relay instances run in parallel without blocking each other.

The outbox relay uses this raw SQL for concurrent-safe row locking:

var events = await db.OutboxEvents
    .FromSqlRaw("""
        SELECT id, topic, payload, partition_key, created_at, processed_at,
               retry_count, last_error, failed_at
        FROM outbox_events
        WHERE processed_at IS NULL AND failed_at IS NULL
        ORDER BY created_at ASC
        LIMIT {0}
        FOR UPDATE SKIP LOCKED
        """, _options.OutboxBatchSize)
    .ToListAsync(ct);

FOR UPDATE SKIP LOCKED means: lock these rows, but if another relay instance already locked some of them, skip those instead of blocking. Both instances make progress without duplicating work.

Conditional INSERT with NOT EXISTS

Sometimes you need to insert a row only if a certain condition is true. A naive approach (check first, then insert) has a race condition — between your check and your insert, another process might insert the same row. The INSERT ... SELECT ... WHERE NOT EXISTS pattern does both in one atomic SQL statement:

Alert creation uses this pattern to prevent duplicate alerts:

var affected = await db.Database.ExecuteSqlInterpolatedAsync($"""
    INSERT INTO clinical_alerts
        (id, encounter_id, patient_id, alert_type, severity, details, status, triggered_at)
    SELECT {alertId}, {encounterId}, {patientId},
           'QSOFA_SCREEN', 'WARNING', {details}, 'OPEN', {triggeredAt}
    WHERE NOT EXISTS (
        SELECT 1 FROM clinical_alerts
        WHERE  encounter_id = {encounterId}
          AND  alert_type   = 'QSOFA_SCREEN'
          AND  status       IN ('OPEN', 'ESCALATED')
    )
    """, ct);

This prevents duplicate alerts without application-level locking — PostgreSQL guarantees atomicity of the INSERT...SELECT.

---

Migrations

What is a migration? A migration is a versioned change to your database schema — like adding a column, creating a table, or adding an index. EF Core compares your current C# entity configurations to the last known database state and generates a migration file with the differences (the "up" to apply the change and the "down" to reverse it). Migrations are committed to version control so every developer and deployment environment applies the same schema changes in the same order.

The project has 40+ migrations tracked chronologically. Key commands:

# Create a new migration
dotnet ef migrations add AddNewFeature \
  --project VigilCareClinicalAPI \
  --startup-project VigilCareClinicalAPI

# Apply pending migrations
dotnet ef database update \
  --project VigilCareClinicalAPI \
  --startup-project VigilCareClinicalAPI

# Generate SQL script (for production deployments)
dotnet ef migrations script \
  --project VigilCareClinicalAPI \
  --startup-project VigilCareClinicalAPI

Migrations are applied automatically on startup during development (via db.Database.Migrate() in the seeder path).

---

Default Values and Sentinels

PostgreSQL can generate default values for columns when you don't provide one in the INSERT. But EF Core needs to know when you intentionally left a property unset vs when you set it to a value that happens to match the default.

builder.Property(a => a.Status)
    .HasDefaultValueSql("'OPEN'")
    .HasSentinel((AlertStatus)(-1));

builder.Property(o => o.Source)
    .HasDefaultValueSql("'MANUAL'")
    .HasSentinel((ObservationSource)(-1));

What is a sentinel value? A sentinel is a special "marker" value that means "not set." HasSentinel((AlertStatus)(-1)) tells EF Core: "if the C# property equals -1, that means it was never explicitly set, so let PostgreSQL provide the default ('OPEN')." This is needed because the C# enum default (0) might be a valid, meaningful enum value that you actually want to store — without a sentinel, EF Core couldn't distinguish between "I set this to 0" and "I forgot to set this."

---

Entity Summary

Entity	Table	Key Features
Patient	patients	PHI encryption, MRN unique index, name search token
Encounter	encounters	Status machine, active encounter uniqueness
Observation	observations	Partial unique index on idempotency_key, append-only
ClinicalAlert	clinical_alerts	Check constraints, 4 partial indexes, JSONB explanation
OutboxEvent	outbox_events	FOR UPDATE SKIP LOCKED, retry tracking
ClinicalAuditLog	clinical_audit_logs	Append-only, JSONB before/after snapshots
News2Score	news2_scores	7 parameter scores, risk level
GcsScore	gcs_scores	3 components, classification
SofaScore	sofa_scores	6 organ scores, baseline flag, partial index
QsofaEvaluation	qsofa_evaluations	3 criteria values, alert-fired flag
SepsisBundle	sepsis_bundles	Compliance status check constraint, deadline tracking
ExternalResourceIdentifier	external_resource_identifiers	Composite unique index for FHIR mapping