PostgreSQL application handoff fence

IDurableWorkHandoffFence lets a running Durable Work executor make one current database decision from two kinds of authority: Durable's live invocation and permit, plus the application's current authorization and conditional handoff record. The application owns its authority and ledger; Durable continues to own the Work claim and execution lifecycle. A prepared handoff is provisional until the caller commits.

Use this boundary when an already-permitted Work invocation must revalidate domain authority immediately before a possible provider effect. Use transactional enqueue to accept new Work. An application outbox can persist business intent and relay state, but does not by itself validate the live Durable invocation described here.

Eligibility and registration

Register the PostgreSQL provider through AddAppSurfaceDurablePostgreSql and inject IDurableWorkHandoffFence into the Work executor. The fence is scoped. The pump binds the same scoped instance only around the actual executor invocation, after permit commit and any existing invocation admission. Constructor or executor preparation runs unbound. A retained service loses authority when its invocation exits; an instance borrowed from another invocation cannot authorize the current one. There is no public binder, claim/permit input, or caller-selected scope.

The caller supplies an active NpgsqlTransaction on an open connection to the configured physical PostgreSQL target. Its isolation level must already be ReadCommitted. The fence does not begin, commit, roll back, dispose, or change the isolation of caller resources.

Public API

namespace ForgeTrust.AppSurface.Durable.PostgreSql;

public interface IDurableWorkHandoffFence
{
    ValueTask<DurableOperationResult<DurableWorkHandoffResult>> ExecuteAsync(
        NpgsqlTransaction transaction,
        Func<NpgsqlTransaction, CancellationToken, ValueTask<bool>> authorizeAndRecordHandoff,
        CancellationToken cancellationToken = default);
}

public sealed record DurableWorkHandoffResult(bool HandoffPrepared);

ExecuteAsync calls the delegate at most once for that call, passing the exact caller transaction and caller token. The delegate returns true only after it has revalidated application authority and made its conditional ledger write in that transaction. Sequential calls within one invocation are supported; the application ledger's conditional write must prevent duplicate logical handoffs. Overlapping or reentrant calls on one scoped instance throw InvalidOperationException before a second callback starts. The framework does not queue or retry callbacks.

The result is not a provider-delivery receipt, a durable application commit, or permission transferable to another invocation. Existing provider-effect safety and reconciliation rules still decide how to handle provider acceptance, rejection, or ambiguity.

Transaction and lock sequence

The caller begins and owns the outer transaction. The fence establishes an operation savepoint, captures the effective transaction-local Durable scope, acquires the shared migration/epoch fence, validates schema, StoreId and active epoch, binds the native scope, then locks and validates the active scope, exact Work, its dispatch row, and the exact permit. The callback savepoint is established only after those native locks are held.

The lock order is:

shared migration/epoch fence
  -> scope
  -> Work
  -> dispatch
  -> exact effect permit
  -> application authority and ledger rows (inside the callback)

After the callback, the fence restores the caller's prior effective Durable scope, samples authoritative database time after lock waits and callback work, and performs its final invocation-lifetime, cancellation, lease, and deadline checks before releasing the operation savepoint. Releasing savepoints leaves their successful effects and acquired locks in the caller's outer transaction. Therefore native locks remain held after ExecuteAsync returns until the caller commits, rolls back, or otherwise ends that transaction.

Pre-callback native refusal rolls back the operation savepoint; locks acquired by that operation may be released, while writes the caller made before the savepoint remain outside it. Callback false rolls back the callback savepoint, restores the prior scope, performs the final check, and returns a successful false result only if the final check succeeds. A post-callback native/time refusal removes callback writes while retaining the native locks until outer completion. If the callback throws while the caller token remains active, cleanup rolls back the callback savepoint, restores the prior scope, releases the operation savepoint, and propagates the original exception; native locks remain until outer completion. If caller cancellation is observed instead, cleanup rolls back the operation savepoint, so those operation-acquired locks may be released. The fence never completes the outer transaction.

Caller-token cancellation observed before the final success check throws OperationCanceledException after cleanup and removes operation writes, including callback writes if its savepoint has already been released. The last decision point is after callback handling and scope restoration, with database time sampled before the invocation/token checks and operation-savepoint release. A cancellation arriving after that point does not retroactively reverse the decision. Check the token before commit; if commit fails or its outcome is uncertain, do not send until the application ledger and existing provider-safety rules resolve the outcome.

The implementation uses a fresh internal five-second cleanup token for savepoint rollback/release and scope restoration. It is not a public option and does not change the host's PostgreSQL statement or transaction policy. Npgsql is pinned by the repository to 10.0.3 in Directory.Packages.props. In that version, NpgsqlTransaction.SaveAsync completes synchronously without a database round trip; it queues the SAVEPOINT statement with the next command on that same transaction. Therefore, a completed SaveAsync alone is not server acknowledgement. The two-savepoint proof must exercise the queued savepoints through subsequent commands on the exact supplied transaction and verify rollback/release visibility and lock behavior. See the Npgsql transaction API. If cleanup fails, the fence throws rather than returning success or a clean refusal; roll back and dispose the outer transaction and connection.

Native facts and refusal precedence

The validator stops at the first applicable check and does not inspect later rows merely to select a different description. For simultaneously invalid facts, the order is:

Caller-token cancellation is an exception, not a native refusal. A callback-thrown OperationCanceledException propagates even if the caller token is not canceled. PostgreSQL permission, serialization, deadlock, lock-timeout, statement-timeout, and transport errors propagate after feasible cleanup; they are not rewritten as Durable refusals. Safe native codes and user-facing guidance do not include raw server text, application payloads, credentials, recipients, or provider responses. The canonical Durable diagnostics catalog supplies shared Problem/Cause/Fix/Docs wording.

Refusal and exception cause-to-action checklist

Use the first applicable refusal in the precedence table. A failed DurableOperationResult is a stop signal for provider I/O, not an invitation to bypass the fence or retry the callback. The canonical diagnostics catalog above provides shared Problem/Cause/Fix/Docs wording; this table applies it to the handoff boundary.

These outcomes do not transfer responsibility for the outer transaction. After an exception, use the exception and transaction state to decide whether to roll back/dispose; after cleanup failure, rollback/disposal is mandatory.

Callback, RLS, and application writer obligations

Every authorization read and handoff write in the callback must use the supplied transaction and its exact connection. Do not open another connection, begin or use another transaction, or rely on an ambient transaction for those operations. The callback is database-only: it must not perform provider I/O, commit/roll back/dispose/replace the supplied resources, or leave work running beyond the executor invocation. The fence cannot sandbox application code holding runtime database credentials. An improper callback commit or write on another connection escapes the savepoints and cannot be undone by the framework; detection does not reverse it.

Call the fence before acquiring application authority or ledger locks. The canonical proof may make an unrelated earlier caller write, such as a marker row, only if it has no foreign-key or other lock dependency on authority/ledger rows. Inside the callback, lock and revalidate authority, then use a conditional write keyed by the stable logical operation and unchanged ActivityId/ProviderKey. Audit every competing writer of those rows, including implicit foreign-key locks and paths using transactional enqueue. All writers must follow native-before-application ordering and deterministic ordering when locking multiple application rows. A writer that takes application locks first and then requests conflicting native locks can deadlock. Do not nest enqueue into the canonical callback.

Use the restrictive runtime principal with separately granted application-table access and the applicable native and application row-level security policies. The fence sets only the bound transaction-local Durable scope and restores the caller's previous effective value, including the no-scope state, on usable outcomes. It does not change role, bypass RLS, or use migration-owner privileges. Prove scope restoration with both an existing scope and no scope, including pooled connection reuse.

The host owns finite, nonzero PostgreSQL lock_timeout, statement_timeout, and idle_in_transaction_session_timeout settings for the relevant runtime/application sessions. lock_timeout must be below statement_timeout. Record effective values and their owner in release evidence; select budgets from the deployment workload rather than copying universal seconds. The host also owns a bounded cancellation/cleanup policy and total transaction duration. These settings can bound individual waits/statements and terminate an idle abandoned transaction; they do not bound arbitrary callback CPU, an actively extended transaction, or an uncooperative application. The library does not change these settings or own caller completion.

Local prerequisites and supported proof commands

Run from the repository root. The equipped/ready path requires the .NET 10 SDK, Docker installed and running, and the checkout. The strict PostgreSQL rail uses the repository's pinned PostgreSQL 16.5 Testcontainers image; it can pull that image on first use. An explicitly configured APPSURFACE_POSTGRES_TEST_CONNECTION may select an external PostgreSQL 16.0+ server for the ordinary rail, but do not use that override to label Docker readiness evidence. The packed consumer rail uses a fresh coordinated package version and runs the public-only consumer against the strict pinned PostgreSQL path. It accepts no positional test-case selector.

For the public handoff consumer's first run, use the same supported packed command in either cache state. The ready path confirms the pinned image is already present; the cold path leaves it absent so Testcontainers downloads it during the command. Both paths still include isolated NuGet restore and package build work:

# Ready/equipped: .NET 10 and Docker are ready; the exact PostgreSQL image is cached.
dotnet --version
docker info
docker image inspect 'postgres:16.5@sha256:53f3e608f9475ce120ced2d0f430b89458d7faa28530e0b0977a6af64d294877' >/dev/null
CI=true APP_SURFACE_PACKAGE_VERSION=0.2.0-issue869.4 ./Durable/verify-packed-consumers.sh

# Cold image: .NET 10 and Docker are ready; the exact PostgreSQL image is not cached.
dotnet --version
docker info
CI=true APP_SURFACE_PACKAGE_VERSION=0.2.0-issue869.4 ./Durable/verify-packed-consumers.sh

Do not pre-pull the image when measuring the cold path; count its download and all package restore/build waits. These are verifier commands, not a handoff-only case selector.

The supported commands are the existing script interfaces; use the exact package version shown here for the #869 candidate and do not infer success from source presence:

./Durable/verify-postgresql.sh --ci
CI=true APP_SURFACE_PACKAGE_VERSION=0.2.0-issue869.4 ./Durable/verify-packed-consumers.sh
./scripts/coverage-solution.sh

The first command is the full strict PostgreSQL suite, not a handoff-only selection. The packed command packs the coordinated packages and executes the package-only PostgreSQL consumer, including the handoff proof, with required PostgreSQL tests configured to fail on skips. Its reported outcomes and artifact identity must come from that run. The final command is the unchanged solution coverage gate (95% line / 85% branch by default); it accepts no arguments. The first two are issue acceptance rails; run solution coverage only after their source/package state is ready. Record the exact command, source/dirty-tree identity, terminal result, package version and bytes, database image/server identity, and effective host timeout values with their owner. Do not copy generated logs as tracked receipts or describe a pending/failed run as passing.

The approved equipped-machine first-use target is an unmeasured 2–5 minute human aspiration, not a test result or release gate. It includes reading this entry point, checking a ready .NET 10/Docker/repository setup with no previous handoff run, restore/build, image-cache state, downloads and waits, then accurately explaining that commit makes preparation visible and callback denial removes only callback writes. The cold-image path uses the same commands on an otherwise equipped machine with the pinned image uncached; image download and setup time are included and must be reported. Installing .NET or Docker and running the full strict suite are separate clocks. No timing or comprehension result is claimed here.

Prepared, committed, and provider outcomes

active invocation
  -> caller ReadCommitted transaction
  -> native locks, then application authorization + conditional ledger write
  -> final database-time/lifetime/cancellation check
  -> HandoffPrepared (provisional)
  -> caller checks cancellation, commits, disposes transaction, releases connection
  -> same invocation applies existing provider-safety policy
  -> accepted / rejected / ambiguous provider outcome is reconciled under existing rules

Give the application ledger a stable key scoped to the logical operation. If commit acknowledgement is lost, a fresh appropriately scoped read that finds the key proves a committed database fact; absence alone is inconclusive while the original transaction could still commit. Keep provider I/O stopped while the result remains unresolved. A later native cancellation does not undo a committed ledger row. The executor may observe cancellation before provider I/O and suppress the call, leaving the ledger for reconciliation; cancellation alone does not prove that no provider effect occurred. A found row alone never creates invocation authority or permission to send outside the current invocation.

Internal contracts and verification

PostgreSqlDurableTransactionResources.RequireActive is the narrow shared resource check for the transaction writer and fence: it requires the supplied transaction's active open connection and configured physical target, preserves the writer's established failure convention, and takes no ownership. PostgreSqlDurableHandoffFacts.ReadAsync assumes schema and active epoch were already validated, the bound transaction-local scope was set, and the connection is the transaction's exact connection. It locks scope → exact Work → dispatch → only the supplied permit ID, supports legacy and opted-in Work, samples current database time around relevant lock waits, and returns current lease/deadline facts plus a bounded refusal code. It does not create scope, consume admission, mutate a permit, scan permit history, or complete the caller transaction. TimeProblem gives lease expiry precedence over an elapsed opted-in deadline. The shared epoch fence remains in PostgreSqlDurableWorkStore; no second admission transaction is run.

The minimal application flow is compiled and executed through the public native pump in the packed consumer. The fake provider used there is an explicit test adapter outside this snippet. A commit exception propagates and suppresses provider I/O; reconcile the application ledger under the existing uncertain-commit rules before any later delivery decision.

internal interface IHandoffApplicationProvider
{
    ValueTask SendAsync(
        HandoffWork work,
        string activityId,
        CancellationToken cancellationToken);
}

internal sealed class HandoffApplicationExecutor(
    IDurableWorkHandoffFence handoffFence,
    NpgsqlDataSource runtimeDataSource,
    IHandoffApplicationProvider provider)
    : IDurableWorkExitExecutor<HandoffWork, HandoffResult>
{
    public async ValueTask<DurableWorkExit<HandoffResult>> ExecuteAsync(
        DurableWorkerEnvelope<HandoffWork> envelope,
        CancellationToken cancellationToken = default)
    {
        var work = envelope.Payload ?? throw new InvalidOperationException("The handoff Work payload was missing.");
        var identity = envelope.ExecutionIdentity ?? throw new InvalidOperationException("The native execution identity was missing.");
        await using (var callerConnection = await runtimeDataSource.OpenConnectionAsync(cancellationToken))
        {
            await using (var callerTransaction = await callerConnection.BeginTransactionAsync(
                System.Data.IsolationLevel.ReadCommitted,
                cancellationToken))
            {
                var prepared = await handoffFence.ExecuteAsync(
                    callerTransaction,
                    async (transaction, token) =>
                    {
                        await using var authority = new NpgsqlCommand("""
                            SELECT is_authorized
                            FROM handoff_consumer.authority
                            WHERE scope_id = @scope_id AND operation_key = @operation_key
                            FOR UPDATE;
                            """, transaction.Connection, transaction);
                        authority.Parameters.AddWithValue("scope_id", work.ScopeId);
                        authority.Parameters.AddWithValue("operation_key", work.AuthorityKey);
                        if (await authority.ExecuteScalarAsync(token) is not true)
                        {
                            return false;
                        }

                        await using var ledger = new NpgsqlCommand("""
                            INSERT INTO handoff_consumer.ledger (scope_id, activity_id, logical_operation)
                            VALUES (@scope_id, @activity_id, @operation)
                            ON CONFLICT DO NOTHING
                            RETURNING activity_id;
                            """, transaction.Connection, transaction);
                        ledger.Parameters.AddWithValue("scope_id", work.ScopeId);
                        ledger.Parameters.AddWithValue("activity_id", identity.ActivityId);
                        ledger.Parameters.AddWithValue("operation", work.LogicalOperation);
                        return await ledger.ExecuteScalarAsync(token) is not null;
                    },
                    cancellationToken);

                if (!prepared.IsSuccess || prepared.Value?.HandoffPrepared != true)
                {
                    return DurableWorkExit<HandoffResult>.RetryBeforeEffect(
                        prepared.IsSuccess ? "handoff_denied" : "handoff_refused");
                }

                cancellationToken.ThrowIfCancellationRequested();
                // A failed or uncertain commit propagates without sending. Reconcile the ledger before a later send.
                await callerTransaction.CommitAsync(cancellationToken);
            }
        }

        cancellationToken.ThrowIfCancellationRequested();
        await provider.SendAsync(work, identity.ActivityId, cancellationToken);
        return DurableWorkExit<HandoffResult>.Succeeded(new HandoffResult(identity.ActivityId));
    }
}

Start with the compiled public handoff example above, its packed PostgreSQL consumer source, and the #869 proof plan. The package entry point, Work protocol, execution-policy reference, and issue progress record provide the surrounding adoption and verification context. The consumer uses package references only; do not treat source presence as a compiled or passing proof.