Design: AppSurface EvidenceHost — Contract-First CI Evidence

Generated by /office-hours on 2026-08-19 Branch: main Repo: forge-trust/AppSurface Status: APPROVED (office-hours and autoplan, 2026-08-20) Mode: Builder Supersedes: andrew-main-design-20260813-043247-issue147-rich-authoring.md

Problem Statement

AppSurface currently provides strong local coverage execution and gating, but it does not express or prove the complete evidence required for a particular change or release. A consumer selects coverage projects manually, separately provisions dependencies, separately runs browser and release checks, and then interprets many CI steps as a confidence claim.

Skoolit shows both failure modes that AppSurface should solve as a reusable product:

• PR 548 failed because three untouched DbContext structural accessors counted as patch lines (94.34% patch-line coverage) despite 100% patch-branch coverage. Executing those getters would add little behavioral evidence.
• PR 555 had 100% patch coverage but failed the global aggregate line threshold because the planner intentionally did not provision PostgreSQL. The observed aggregate was from an incomplete test environment, not a changed-code risk failure.
• Skoolit’s current coverage wrapper explicitly selects nine of ten test projects, omitting its Playwright project; Postgres and browser checks are separately planned and orchestrated. Its release-gate report validates an issue, while deployment does not itself invoke AppSurface coverage. A release therefore lacks one machine-verifiable statement of full evidence.

The developer consequence is either wasted work—tests that merely touch framework plumbing—or false confidence—green/release-adjacent flows assembled from a convenient subset of evidence. Engineers must inspect workflow YAML, planner outputs, environment variables, and artifacts to answer a simple question: what risk did this change require us to mediate, and did we do it?

What Makes This Cool

Coverage becomes quiet when it correctly mediates risk, and unmistakably useful when it blocks. A developer runs one command and gets a human- and agent-readable answer:

Plan: pr-database
Why: src/.../Persistence matched the versioned persistence rule
Evidence:
  ✓ PostgreSQL started and became healthy
  ✓ selected integration and coverage producers completed
  ✓ behavioral patch coverage satisfied
  ✓ migration preflight completed
Result: evidence complete for the changed risk

For a release, the same system can make an honest, stronger statement:

Release evidence incomplete
Missing: required browser E2E producer
Cause: browser dependency was unavailable before its bounded readiness deadline
No release-complete claim was emitted.

The “whoa” moment is not an extra dashboard or a higher percentage. It is a single auditable evidence graph that selects the minimum sufficient PR work, brings up exactly the required disposable topology, runs coverage and E2E under the same lifecycle, and refuses to mistake a partial run for release evidence.

Constraints

• AppSurface is the product boundary. Skoolit is the first serious consumer and acceptance case, not the place where product behavior is hard-coded.
• The design is a non-mutating CI evidence system. It must not deploy, change traffic, apply migrations, touch production data, use production credentials, or become a hosted CI/artifact platform.
• A consumer-owned, separate Aspire EvidenceHost is required. Normal development AppHosts and test topology should intermix as little as possible.
• E2E is first-class: dependencies, readiness, fixtures, browser artifacts, outcomes, and cleanup belong in the evidence graph.
• The selected approach must not duplicate the existing coverage runner’s discovery, scheduling, watchdog, merging, output ownership, or gate behavior. coverage run remains a producer.
• No generic “simple code,” no broad handwritten exclusion catalog, and no coverage-only assertions. Any behavioral-coverage exclusion must be mechanically proven and fail closed.
• Existing coverage commands remain usable for consumers that do not adopt Aspire/EvidenceHost.
• Added planning and verification cost must be trivial relative to the selected evidence. PR evidence should be narrower than release evidence by policy, not by hidden CI conditions.
• Plans and manifests must never serialize secret values. AppSurface-owned diagnostics redact declared sensitive values; consumer CI remains responsible for retention and sanitization of raw consumer-producer logs and media.

Premises

1. AppSurface Evidence is a product surface, not a coverage flag. It coordinates build, unit/integration coverage, E2E, migration preflight, contracts, policy, and package/artifact verification into one non-mutating evidence claim.
2. A diff-aware planner selects the minimum sufficient profile. A versioned consumer policy maps explicit diff input to profiles, producers, and dependencies with an explanation trace. Unknown, invalid, or ambiguous input chooses a conservative broader profile or fails before an evidence claim.
3. EvidenceHost is separate and consumer-owned. It is an ephemeral, noninteractive Aspire AppHost that composes resources, waits for declared readiness, projects scoped safe values, starts typed producers, gathers diagnostics, and cleans up.
4. E2E is equal to coverage as evidence. Browser journeys run as typed, bounded producers with application/identity/browser prerequisites, fixture provenance, approved traces/screenshots/video, and distinct product, flake, timeout, and infrastructure outcomes.
5. Profiles and obligations are enforceable claims. A profile chooses the topology and producer set; explicit risk obligations state which changed risk each producer must close. A final immutable manifest binds source revision, diff hash, policy identity, profile identity, selected/closed obligations, resources/readiness, producer identity/version/outcome/assertions, metric verdicts, timings, and artifact hashes. A partial run cannot claim full or release evidence.
6. Behavioral coverage is a narrow refinement. It excludes only source-and-symbol-proven, non-behavioral structural lines; it always shows excluded locations and reasons; near-misses remain ordinary coverage. The initial candidate is an exact EF Core DbContext DbSet<TEntity> => Set<TEntity>() accessor, not “any line with no branch.”
7. The boundary is coordination, not a CI platform. AppSurface owns planning, typed contracts, reusable standard components, and manifest semantics. It does not own workflow scheduling, arbitrary plug-in discovery, generic retries, secret management, hosted storage, cloud provisioning, or deployment.

Normative v1 Contract

Terms and claim rules

• Policy is a versioned, checked-in document selected explicitly by the caller. It maps a normalized, explicit diff to named profiles and risk obligations. v1 supports exact paths and segment globs only; no arbitrary code/content detectors. A rename evaluates both old and new paths, and a deletion evaluates the deleted path. Exact matches outrank globs; otherwise, the glob with more literal path segments wins. Equal-specificity overlap must declare precedence or resolve to an explicitly declared conservative combined profile. Generated-file treatment is explicit policy, never inferred. An unknown path selects the policy’s named conservative profile or makes planning fail.
• Profile is a closed list of required resource declarations, producer registrations, and obligations. It is targeted or release; release is a policy designation, not an inference from the Git branch. no_evidence_required is a valid targeted result only when an explicit policy rule selects no obligations and no producers. An unexplained empty profile is invalid.
• EvidenceObligation identifies one changed-risk requirement with id, riskClass, rationale, required producer IDs, and an explicit acceptance assertion ID. Producers return the assertion IDs they satisfied. A producer can pass without closing an obligation; an obligation closes only when every required producer passed and its declared assertion was returned.
• EvidencePlan is the immutable, validated resolution of one policy, one normalized diff, and exactly one profile. It contains schema version, base/head revision identifiers, diff SHA-256, policy SHA-256, profile ID, resolved producers/resources/obligations, rationale records, and an allowlisted execution envelope. v1 policy authors must define any needed combined profile explicitly; a policy without a required conservative profile fails planning on an unmatched path.
• Producer is an explicitly registered typed component, identified by immutable ID and semantic version. It declares its bounded timeout, required resources, assertion IDs it may satisfy, and allowed artifact declarations. Policy can select a registration; it cannot supply an executable command, assembly name, mount, image, or network destination.
• EvidenceManifest is schema-versioned JSON derived only from the resolved plan and producer/resource results. It includes selected, closed, and unmediated obligation IDs; hashes and bounded metadata; and never secret values or raw artifact contents. “Verified” in v1 means schema validation plus recomputation of all recorded plan/policy/diff/artifact hashes; it is not a cryptographic identity attestation.
• Evidence complete means every selected obligation is closed, every required resource is ready, every required producer has passed, every expected artifact is present and hash-valid, and cleanup completed. Any other required terminal status makes the verdict incomplete or invalid; it can never make a full/release claim.
• Partial evidence is a truthful result for a targeted profile with every selected obligation closed. It carries its profile identity and is never substitutable for release evidence. A no_evidence_required result records its matched rule, paths, rationale, and empty obligation set; it is not a release claim.

The plan owns the exact declaration sets that results must satisfy:

• A ResourceDeclaration has id, componentId, requires[], readiness (aspire_health or completion in v1), positive deadlineSeconds, failure outcome unavailable, and a closed list of projected value keys. Values are injected only at runtime and are never serialized. requires[] determines startup order; aspire_health waits for the resource’s registered Aspire health checks, and completion waits for a one-shot resource’s successful completion.
• A ProducerDeclaration has id, version, requiresResources[], positive timeoutSeconds, and an exact set of ArtifactSlots. An ArtifactSlot has logicalName, relative root, media type, required/optional flag, and maximum bytes. A producer result must return one result per declared slot and cannot add undeclared paths. Required absent slots, duplicate slots, path escape, media mismatch, or size/hash failure make the producer invalid.

Public contract ownership, canonical form, and compatibility

ForgeTrust.AppSurface.Evidence.Contracts is the sole owner of public Evidence DTOs, interfaces, enums, schema media types, assertion/diagnostic namespaces, canonical serialization, and compatibility rules. It has no dependency on Aspire, the CLI, the coverage implementation library, or a consumer producer. Planner, CLI, Aspire, and consumer packages depend inward on it.

• The v1 plan and manifest media types are application/vnd.forge-trust.appsurface.evidence-plan+json;version=1 and application/vnd.forge-trust.appsurface.evidence-manifest+json;version=1. EvidenceCanonicalJson serializes UTF-8 without insignificant whitespace, orders object properties by ordinal name, writes integral numbers without exponent form, and normalizes all declared paths and IDs before hashing. PolicyDigest, DiffDigest, PlanDigest, and ManifestDigest are SHA-256 of those canonical bytes.
• NormalizedDiff is a canonical ordered list of change records containing status plus old/new normalized paths where present; no raw diff text, line ending, checkout root, or Git display name is hashed into the evidence identity. That keeps policy resolution stable across equivalent CI checkouts.
• Contract major version is the compatibility boundary. v1 readers accept unknown additive object properties, preserve none of them when reserializing, and reject unknown values in claim-critical enums, media types, assertion descriptors, limits, and diagnostic codes. A writer declares minimumReaderContractVersion; a verifier rejects a manifest outside its supported major/minor range before evaluating a claim.
• A producer registration has a contract version and an implementation semantic version. An implementation version may change without changing assertion semantics; a contract-version or assertion-semantic change requires a new descriptor version and compatibility fixtures. Built-in assertion IDs use appsurface/<producer-id>/<name>@<major>; consumer IDs use consumer/<consumer-id>/<name>@<major>. Only trusted-base policy/registration code may define or select descriptors.
• Additive optional fields/descriptors are permitted inside v1 only when old readers can ignore them safely. Removing, renaming, changing claim eligibility, changing canonicalization, or changing assertion meaning is breaking and requires a new major contract. Each release runs old-reader/new-writer and new-reader/old-writer fixtures for every advertised supported pair plus packed-consumer restore/build tests.

Producer execution and assertion closure

IEvidenceProducer is a public contract type, but it is not an arbitrary plugin system. A consumer may implement it only by explicitly registering a trusted-base type in its EvidenceHost registration module. EvidenceProducerContext supplies immutable plan/producer identity, scoped projected resource values, CancellationToken, TimeProvider, bounded ArtifactWriter, and bounded structured DiagnosticSink; it has no raw command, host-path, secret, network, or policy mutation capability.

EvidenceAssertionDescriptor is policy/contract-owned and has ID, semantic version, human explanation, accepted result shape, and owning producer registration. A producer result can return only a unique subset of descriptor IDs predeclared by that producer registration. The collector rejects duplicate, unknown, stale, incompatible, or out-of-plan assertion IDs as invalid; a declared assertion lacking its required result shape leaves its obligation unclosed. An obligation maps to exact descriptor/version requirements, so producer success cannot close an obligation it was not selected to close.

Claim dimensions and gate eligibility

The manifest never uses a bare “complete” verdict. It carries separate dimensions:

Dimension	Values in v1	Meaning
ProfileScope	targeted, release	Policy-selected breadth of obligation set.
ExecutionVerdict	passed, incomplete, invalid	Whether this resolved run satisfied its declared contract.
ClaimKind	targeted_complete, release_complete, no_evidence_required, observation_only, none	Human and machine-readable claim that may be consumed downstream.
ClaimEligibility	pr_gate, release_gate, informational, none	Closed eligibility set computed by AppSurface, not inferred by callers.

Profile scope	Execution verdict	Claim kind	Eligible consumers
targeted	passed with all obligations closed	targeted_complete	PR gate, informational
release	passed with all obligations closed and accepted envelope status	release_complete	PR gate, release gate, informational
targeted no-obligation rule	passed with no resources/producers	no_evidence_required	PR gate, informational
either in observation mode	any non-invalid result	observation_only	informational
either	incomplete or invalid	none	none

Release consumers must explicitly opt in to accepting the validated but not_attested envelope state described below. No targeted, no-evidence, observation, incomplete, or invalid result is release-gate eligible.

Hard execution limits and quality methodology

v1 hard limits are versioned contract data and are recorded in every manifest: at most 16 resources, 32 producers, 128 obligations/assertions, and 128 artifact slots; 256 MiB total artifacts; 8 MiB bounded stdout and 8 MiB bounded stderr per producer; 1 MiB manifest; 4 MiB total structured diagnostics; 128-character IDs; 240-character normalized relative paths; four concurrent producers; and a 10-minute cumulative cleanup budget. Individual resource/producer deadlines and ArtifactSlot byte limits may lower but never raise these maxima. Count, path, identifier, schema, artifact, or output-limit breaches are invalid; elapsed deadlines are timed_out; cleanup-budget breaches are incomplete.

The pilot compares the same commit and resolved profile against the equivalent current selected evidence using a fixed reviewed fixture cohort. It reports warm and cold restore separately, records at least 20 successful observations before evaluating p50/p95, and treats a discarded/failed CI job as no performance sample. The target p95 planner-plus-manifest overhead (<30 seconds) and runner-minute delta (<=10%) are enforced as profile-promotion criteria: a profile that exceeds either stays opt-in until corrected.

Lifecycle and terminal states

The orchestrator has one closed lifecycle:

planned → validated → provisioning → ready → producing → collecting → cleaning
                                                          ↘ terminal verdict

validated fails closed on malformed/mismatched policy, diff, schema, registration, or execution-envelope input. provisioning and ready have per-resource deadlines and stop on an unavailable resource rather than waiting for an unbounded restart. producing starts each typed producer only after its resources are ready; producer execution is either an in-process trusted-base component or a supervised child process started by that component. collecting validates declared artifact paths, hashes, returned assertion IDs, and closure of every selected obligation. cleaning always runs after any terminal producer state and owns cancellation plus process-tree/resource cleanup.

Every required producer resolves to one of these terminal outcomes:

Outcome	Meaning	Claim eligibility
passed	Producer completed its declared assertion and artifacts validated.	Eligible only together with every other required passed producer.
failed	Product/test assertion failed.	No complete claim.
flaky	Producer detected nondeterministic/retry-inconsistent execution.	No complete claim; do not hide it behind an automatic pass.
timed_out	Bounded producer/resource deadline elapsed.	No complete claim.
unavailable	Required dependency did not become healthy or could not be provisioned.	No complete claim.
cancelled	CI/user cancellation occurred.	No claim.
invalid	Contract, provenance, artifact, or execution-envelope validation failed.	No claim.
skipped_not_required	Producer was not selected by the resolved profile.	Not part of that profile’s claim.

Cleanup failure changes an otherwise passing plan to incomplete and is recorded as a stable cleanup diagnostic. Duplicate producer IDs, duplicate artifact paths, or a producer result not declared by the plan are invalid. v1 has no retry policy; a producer may report flaky, but retry strategy remains the producer/CI owner’s concern.

An explicit no_evidence_required plan takes the shortened path planned → validated → collecting → cleaning → terminal verdict: it may not provision resources or start producers. explain must print the matched no-obligation rule, normalized paths, and rationale before that shortened path is accepted. A missing explanation, a non-empty resource/producer/obligation set, or a release designation is invalid.

v1 execution and ownership matrix

Concern	Owner	Enforcement point
Diff normalization, policy parsing, plan closure, rationale, manifest schema and hash validation	AppSurface Evidence	CLI/library before resource startup and before final verdict
Coverage test discovery, scheduling, watchdog, merge, patch targets and coverage thresholds	Existing AppSurface coverage core and CLI	First-party coverage producer calls the same friend-visible internal orchestration contract; the CLI remains its public owner
Resource lifecycle, health waits, connection/reference projection, resource state and disposal	Aspire plus consumer EvidenceHost	Consumer-owned separate AppHost
Domain topology, consumer resource components, E2E journey semantics, fixture content, contract/migration/package assertions	Consumer	Explicit C# registrations in its EvidenceHost
Trusted runtime isolation, credential exposure, network/container policy, and runner identity	Consumer CI provider	A base-owned registered EvidenceTrustEnvelope verifier validates the declared CI envelope before a complete claim; CI owns the actual sandbox
CI runner allocation, checkout, cache, parallel job/matrix scheduling, artifact upload, secrets, approvals and deployment	Consumer CI provider	Existing workflow; outside Evidence

This matrix is intentional: the EvidenceHost coordinates typed producer results inside one disposable topology. It does not own independent CI jobs or a general-purpose command/plugin model. v1 supports one EvidenceHost process and rejects sharded/matrix aggregation as unsupported rather than claiming cross-job completeness.

EvidenceHost bootstrap and Aspire lifecycle

EvidenceHostBootstrap.CreateAsync(EvidenceHostRegistration, EvidencePlan, CancellationToken) is the only v1 composition entry point. EvidenceHostRegistration is a base-owned, explicit module containing the permitted resources, producer registrations, envelope verifier, and package-compatible Aspire adapters. It is supplied before DistributedApplication builder creation and never scans the subject checkout, entry assembly, referenced assemblies, dependency injection registrations, or filesystem for components.

The bootstrap owns one builder and one DistributedApplication: validate plan and envelope; compose explicit registrations; build once; start; wait for declared readiness with bounded stop-on-unavailable behavior; run producers; collect; stop; then dispose resources and activation services. It preserves the first non-process-fatal failure while recording later cleanup failures, and an otherwise passing run with cleanup failure is incomplete. Cancellation during build/start/readiness stops the application and cancels descendants before disposal. A readiness regression before a dependent producer starts is unavailable; a completion resource that exits unsuccessfully is failed; and a post-producer health regression is recorded in collection/cleanup but cannot upgrade a claim.

EvidenceHost supports the same explicitly tested Aspire package range as the Evidence.Aspire package. Its compatibility suite includes normal build/start/dispose, failed partial build, cancellation at every lifecycle edge, readiness unavailable, completion failure, health regression, and app-before-bootstrap disposal. It may borrow internal lifecycle techniques from ForgeTrust.AppSurface.Aspire.Testing, but it exposes a separate API and lifecycle contract rather than extending the testing builder.

Trust and untrusted-code boundary

v1 has two modes:

• Trusted evidence mode is for same-repository, protected CI on a supported runner. The trusted base revision supplies policy, EvidenceHost registrations, resource definitions, typed allowlists, and one explicitly registered EvidenceTrustEnvelope verifier from a separate read-only trusted-tool checkout. The tested head revision exists only in the subject checkout and supplies the product/test inputs selected by the base-owned plan. Head-controlled files cannot supply policy, host code, producer registrations, commands, mounts, images, network policy, artifact roots, or the envelope verifier. Only this mode can emit an evidence complete claim.
• Observation mode is for untrusted/fork execution. It may run an explicitly consumer-approved, dependency-free subset but cannot provision privileged resources, receive secrets, or produce a complete/release claim.

The v1 execution envelope is closed by typed registrations and allowlisted resource definitions, while the consumer CI owns actual container, network, checkout, and secret isolation. A base-owned EvidenceTrustEnvelope registration declares a stable ID/version, supported runner class, trusted-tool/subject/artifact root identities, permitted producer classes, sensitive projection-key policy, and the CI-owned capability assertions it can verify. Its verifier must fail closed when any field is absent, unexpected, head-derived, or inconsistent with the resolved plan. A profile cannot declare a raw command, image, mount, egress destination, or envelope implementation.

The manifest records EnvelopeStatus as validated, unavailable, or not_attested; verifier ID/version; trusted-tool and subject revisions; root identities; runner/workflow/run identity; verification timestamp; and SHA-256 of bounded capability evidence. Verification happens immediately before provisioning, and a changed protected revision, root identity, runner/workflow/run identity, verifier version, or capability-evidence hash invalidates the run. A validated envelope says only that the registered verifier accepted those bound CI facts. It never becomes attested in v1.

Envelope validation proves that a registered verifier accepted declared CI conditions; it is not a cryptographic attestation or proof that the CI provider’s sandbox policy was correctly implemented. Consumer CI documentation and templates are responsible for showing that head-derived code cannot alter the control plane, credentials, mounts, or network policy. Producers must identify sensitive projected keys and avoid logging their values; raw logs, traces, screenshots, and videos remain consumer-CI artifacts under the consumer’s retention/sanitization policy. The manifest names only validated relative locations, media type, byte length, and SHA-256. A future signed attestation is a separate design, not implied by v1 manifest verification.

An AppSurface-owned Docker security runtime is deferred to a separately approved pilot. That pilot must prove denied egress, mount and process-tree containment, image identity enforcement, cancellation cleanup, and hostile-head-input resistance before it can replace a consumer-owned envelope. Existing non-Evidence coverage commands remain supported outside either envelope.

Staged delivery boundary

The product contract is broad enough to describe whole evidence; v1 implementation is deliberately narrow:

1. Planner, deterministic policy conformance fixtures, plan/manifest schemas, closed outcome taxonomy, explain/dry-run, and a dependency-free EvidenceHost sample with an explicit consumer registration path.
2. Existing coverage CLI as the only first-party producer, plus a standard ephemeral PostgreSQL component, a registered CI-provided trust-envelope verifier, and a Skoolit-like consumer-owned integration/E2E producer pilot.
3. Promote PostgreSQL or browser support into reusable managed components only when both AppSurface and a consumer prove the same lifecycle contract.
4. Treat the AppSurface-owned Docker security runtime, Roslyn structural-accessor classifier, content-sensitive planning, cross-job aggregation, signed attestations, and additional first-party producers as separately approved pilots with measured exit criteria.

Cross-Model Perspective

An independent adversarial review agreed that the separate, consumer-owned EvidenceHost is a strong fit for Aspire’s lifecycle and readiness model, but challenged the phrase “all CI evidence” as an implementation boundary. Its key warning: build, E2E, migrations, package checks, and policy checks have different ownership and security models; a second scheduler, plugin system, artifact store, and retry engine would make AppSurface a competing CI platform.

The review supports the selected approach if AppSurface remains a thin provenance-focused contract:

• an immutable EvidencePlan from explicit diff and policy;
• a narrow, explicitly registered one-shot producer contract;
• consumer-owned Aspire lifecycle and resource readiness;
• a secret-free evidence manifest;
• existing coverage machinery reused rather than rebuilt.

It requires bounded readiness/cancellation/cleanup, explicit image-command-mount-network allowlists for PR execution, distinct E2E outcome kinds, path-safe artifacts, producer-version validation, and measurement against current CI latency before scope expands.

Approaches Considered

Approach A: Contract-First EvidenceHost

AppSurface provides four small public concepts and first-party adapters:

1. Evidence policy, obligations, and planner. The planner accepts an explicit base/head diff plus a checked-in, versioned policy. It produces an immutable EvidencePlan: one selected profile, required resources, producer registrations, risk obligations and their acceptance assertions, rationale, source/policy hashes, and a conservative-selection flag or explicit no_evidence_required result.
2. Typed producer contract. An IEvidenceProducer-style contract declares stable identity/version, inputs, timeout, assertion IDs it may close, exact plan-owned artifact slots, and outcome kind. Producers are explicitly composed by the consumer’s EvidenceHost; there is no directory scanning or arbitrary shell discovery.
3. Aspire EvidenceHost kit. Reusable AppSurface components compose standard disposable dependencies and execute producers after readiness. A one-shot coverage producer calls the existing coverage orchestration through a dedicated friend-visible internal library shared with the CLI, rather than reimplementing it or spawning a second CLI process. A consumer supplies domain resources and typed E2E, contract, migration-preflight, or package producers.
4. Evidence manifest/verdict. A bounded JSON/Markdown result verifies provenance and outcome completeness. It contains artifact hashes and approved metadata, not raw logs, screenshots, or secrets.

This is the selected approach. It gives every consumer one evidence graph and a full final claim while keeping GitHub Actions, the consumer’s infrastructure choices, and producer-specific semantics in their rightful places.

Illustrative policy shape—not a committed file format:

profile release:
  requires: [postgres, browser, coverage, e2e, migration-preflight, package-proof]

rule persistence-change:
  selects: [pr-database]
  reason: changed persistence runtime needs PostgreSQL-backed evidence

Approach B: Fully Managed EvidenceHost Toolkit

AppSurface would additionally ship opinionated build, PostgreSQL, Playwright, migration-preflight, package, and policy producers, allowing consumers to declare mostly configuration. This is more magical for a narrow common .NET stack, but it expands the security, support, compatibility, and performance commitments sharply. It remains a future promotion path only after the contract-first model proves which producers truly share a stable contract across consumers.

Approach C: Full CI Controller — rejected

AppSurface would generate/manage workflows, schedules, retries, secret integrations, artifact upload, and deployment-adjacent orchestration. Rejected because it competes with CI providers and dilutes the evidence problem.

Recommended Approach

Adopt Contract-First EvidenceHost.

It is the smallest architecture that still delivers the champion-tier experience: one diff-to-plan explanation, one disposable Aspire test topology, one evidence graph containing coverage and E2E, and one truthful final manifest that names the risk obligations it closed. It makes AppSurface a reusable intent/compiler and evidence-contract layer rather than a generic workflow engine.

The first release proves the model through the staged boundary above. Its only built-in producer is the existing coverage orchestration, shared through an internal friend-visible seam with the CLI; PostgreSQL and one consumer-owned integration/E2E producer are pilots, not a promise of a managed browser or CI platform. Skoolit then becomes the proving consumer: a persistence change must select PostgreSQL evidence, an affected browser journey must appear in the graph, and a release profile must make omission of Skoolit.Playwright.Tests impossible to misrepresent as full evidence.

Open Questions

1. Which standardized dependency should follow PostgreSQL, if any, after the pilot demonstrates an identical lifecycle contract across AppSurface and a consumer? Browser tooling is first-class in the contract, but not necessarily a v1 managed component.
2. What cryptographic attestation and CI-identity model should a later release-consumption design require? v1 manifest verification is intentionally self-consistency verification only.
3. How should a later sharded/matrix evidence extension aggregate signed producer manifests without adding a second scheduler? v1 explicitly rejects it.
4. Which exact Roslyn/source availability constraints make the initial EF structural-accessor classifier safe and portable across consumer build layouts? It is a later, separately approved coverage-producer pilot.

Success Criteria

• A consumer can run explain/dry-run and see the selected profile, every selected risk obligation/producer/dependency, the registered execution envelope, deterministic path/rule precedence, and rationale before a resource or test starts. An explicit no-risk change displays no_evidence_required, its matched rule, and its empty obligation set.
• v1 rejects an unmatched path without a conservative policy profile, ambiguous rule precedence, a policy/diff/schema/registration hash mismatch, duplicate producer/artifact declarations, a sharded profile, an unexplained empty profile, or an execution-envelope mismatch before making a claim.
• A Skoolit-like persistence patch deterministically selects a PostgreSQL-backed evidence profile; no workflow-specific conditional may silently leave its required database evidence out.
• A release profile emits evidence complete only when every selected obligation is closed, every declared required producer is passed, every required resource was ready, artifacts hash-validate, and cleanup succeeds. failed, flaky, unavailable, timed_out, cancelled, and invalid are never claim-eligible.
• Product failure, flaky E2E, dependency health failure, timeout, policy mismatch, cleanup failure, and incomplete profile have distinct stable outcome/diagnostic codes and remediation text.
• The manifest is path-safe, bounded, secret-free, and self-consistently verifies its source/policy/profile/producer/artifact hash bindings; it makes no cryptographic attestation claim.
• The contract-first slice adds no test reinstrumentation or duplicate solution build. The targeted default keeps p95 planner-plus-manifest overhead below 30 seconds and adds no more than 10% runner minutes over the equivalent current selected evidence. A reviewed fixture corpus reports unnecessary-evidence selection and missed-risk selection; a profile that exceeds either budget remains opt-in until corrected.
• A trusted same-repository mode and an untrusted observation mode are tested separately: observation mode cannot provision privileged dependencies, receive secrets, or emit complete/release evidence.
• Trusted evidence mode is enabled only when the base-owned registered CI-envelope verifier accepts the trusted-tool/subject/artifact boundaries and capability claims. The verification result is documented as an envelope assertion, not an independently cryptographic or sandbox attestation. The later AppSurface-owned Docker runtime pilot must prove hostile-head-input, mount, egress, process-tree, cancellation, and cleanup containment separately.
• Existing appsurface coverage run/gate consumers retain their current behavior without an EvidenceHost.

Distribution Plan

Ship as AppSurface preview packages and additions to the existing appsurface .NET tool:

• ForgeTrust.AppSurface.Evidence.Contracts: public DTOs/interfaces/enums and canonical compatibility rules; no Aspire or CLI dependency;
• ForgeTrust.AppSurface.Evidence.Planner: policy/diff normalization and plan/manifest construction; depends only on Contracts;
• ForgeTrust.AppSurface.Evidence.Coverage: implementation-only coverage orchestration library with internal types and explicit friend access for the CLI and first-party Evidence adapter. It is distributed only as their dependency and is not a public consumer API; promotion requires a separate compatibility review;
• ForgeTrust.AppSurface.Evidence.Cli: Contracts/Planner plus first-party adapters and additions to the existing appsurface .NET tool; it has no Aspire dependency;
• ForgeTrust.AppSurface.Evidence.Aspire: separate EvidenceHost lifecycle package that references Contracts and the explicitly supported Aspire range. It must not extend or reinterpret ForgeTrust.AppSurface.Aspire.Testing: that package remains a typed application-test builder, while EvidenceHost owns plan execution, producer coordination, claim formation, and evidence cleanup;
• optional consumer producer packages layered above Contracts and Evidence.Aspire; and
• samples and complete reference documentation for a dependency-free, PostgreSQL-backed, and E2E-enabled EvidenceHost.

The existing AppSurface package build/publish pipeline distributes the packages. Preview packages use coordinated AppSurface versions, publish a tested Aspire minimum/maximum range, and reject unsupported downgrade/conflict combinations before startup. Removing the Evidence CLI/host registration rolls back evidence adoption without changing existing coverage commands or normal AppHosts. Each supported package combination receives packed-consumer restore/build fixtures, including coexistence of Evidence.Aspire with the existing Aspire testing package. Consumer workflows remain ordinary GitHub Actions or another CI provider: they invoke the plan/EvidenceHost and upload the generated evidence bundle using their own artifact policy.

Developer Experience Contract

The primary user is the consumer platform engineer or test-infrastructure maintainer who owns CI evidence for an existing AppSurface consumer. A feature engineer is a secondary reader of the explanation and evidence summary. The primary user must be able to get a truthful plan explanation in under two minutes without Docker, a database, a browser, secrets, a CI run, or a changed test suite. Full producer execution is deliberately a later, explicit operation; it must never be disguised as a fast local preview.

Guided command journey

Evidence extends the existing appsurface tool with five noninteractive commands. All commands support CI-safe machine-readable output, stable diagnostic codes, and a human explanation. None prints secret values, and none sends telemetry or contacts an AppSurface service.

Command	Responsibility	Safe default and first useful result
appsurface evidence init --sample	Create an explicitly marked EvidenceHost starter, sample policy, and copy/paste next steps in an existing repository.	Lists planned files first and never overwrites a non-Evidence file. The default uses a caller-chosen output location; overwrite requires an explicit force option and only replaces files bearing the generated AppSurface marker.
appsurface evidence doctor	Check local/CI prerequisites for the selected policy and profile.	Does not provision resources or run tests. It checks the SDK/package version, policy/diff readability, trusted-envelope inputs, and only the Docker, database, and browser capabilities declared by selected producers. Each unavailable capability reports the affected producer, whether it blocks run, and one next action.
appsurface evidence explain	Normalize the supplied diff, resolve the policy, and show why one profile, resource set, producer set, and obligation set were selected.	Does not start resources, execute producers, or create evidence claims. A successful result is the champion first-success moment and remains under the two-minute TTHW target.
appsurface evidence run	Validate the envelope, execute the resolved EvidenceHost lifecycle, collect outputs, and write the final evidence bundle.	Is the sole command that may start disposable resources or execute tests. It fails closed when doctor-equivalent validation or a required capability fails.
appsurface evidence verify <manifest>	Recompute the public manifest bindings and decide whether the declared claim is eligible for the named consumer.	Does not rerun producers or infer release eligibility. It verifies the supplied immutable evidence and returns a stable reason when the claim is not gate eligible.

init emits a non-overwriting starter rather than pretending to infer a consumer's topology. Its next command is doctor, followed by explain; generated sample output must be useful without a working database or browser. explain reports the matched policy rules, normalized paths, selected profile, selected and unmediated obligations, required capabilities, and the exact reason for a conservative choice or refusal. This makes "why did CI choose this work?" answerable before CI starts a slow job.

doctor has three human-facing outcomes: ready, ready_with_external_prerequisites, and blocked. ready_with_external_prerequisites is not a hidden pass: it names the optional-but-not-currently-executable producers and the command that would become blocked. blocked reports the first decisive problem and a concrete fix, for example “PostgreSQL evidence is required by profile pr-persistence; start the declared disposable dependency or select the documented CI environment.” An invalid policy, ambiguous diff, or invalid envelope is a stable diagnostic failure, not a prerequisite hint.

One human front door; machine evidence stays authoritative

Every successful explain and terminal run/verify writes a compact, bounded evidence-summary.json next to the authoritative plan or manifest. run additionally writes a concise terminal summary and, when GITHUB_STEP_SUMMARY is available, the equivalent GitHub job summary. The summary is intentionally not another gate format or dashboard; the versioned EvidenceManifest remains the authoritative machine claim.

The human summary answers five questions in a stable order: what change/policy/profile was evaluated; why that selection was made; which risk obligations are closed, unmediated, or missing; what resources/producers were required and their terminal state; and the single next action, if any. It distinguishes a low-risk explicit no_evidence_required result from a missing or incomplete run. A non-passing result opens with the failed claim dimension and reason—for example, “release evidence incomplete: required browser E2E producer unavailable”—rather than a raw process exit code or percentage. Machine consumers receive the same outcome through the existing manifest claim dimensions and stable diagnostic codes.

The generated summary is bounded, path-safe, and secret-free under the same limits as the manifest. Artifacts retain their hashes and logical names; large logs, traces, screenshots, and videos remain artifact references rather than being embedded in the summary or job annotation.

Discoverability, documentation, and upgrade path

The preview ships with a dedicated EvidenceHost start page linked from the root README, the appsurface CLI reference, the Evidence.Contracts reference, and the Aspire package README. The first section is a four-command existing-repository journey: init --sample, doctor, explain, then the explicit run. The page tells readers what EvidenceHost is not: a replacement for normal AppHosts, generic CI scheduler, deployment system, hosted artifact store, or a promise that an incomplete test profile is full evidence.

A task-focused cookbook complements, but does not duplicate, the contract reference:

• an explicit no_evidence_required documentation-only/no-risk change;
• a PostgreSQL-backed persistence change selected by a targeted profile;
• browser E2E as a required release obligation;
• a required dependency that becomes unavailable before its readiness deadline; and
• a release claim refused because a required producer, envelope, artifact, or cleanup result is missing.

Each scenario gives the policy intent, command sequence, expected human summary, manifest claim dimensions, common diagnostic, and a link to the canonical policy/producer/manifest reference. The PostgreSQL and browser guides state their external prerequisites plainly; users must not expect repository-wide coverage or a release claim from a deliberately skipped portion of the suite. The existing Docs or developer experience feedback form remains the voluntary route for confusing guidance or first-run friction; EvidenceHost introduces neither a bespoke intake process nor telemetry.

Preview packages carry the published contract major, supported Aspire range, producer descriptor versions, and migration notes. Additive preview upgrades preserve old-reader/new-writer fixtures; an incompatible policy, descriptor, or manifest change fails before execution with a migration link and never attempts a best-effort interpretation. The generated sample pins no hidden runtime behavior; doctor makes version incompatibility visible before run.

DX quality bar and explicit non-goals

Champion-tier DX is judged by whether the system is silent when policy selects no risk and highly specific when a selected risk lacks evidence. The preview acceptance suite therefore verifies that the generated sample yields a non-mutating doctor and explain result inside the two-minute TTHW target, that each command’s failure names the blocking capability or contract violation, and that GitHub/terminal summaries agree with the manifest. It does not add dashboard work, outbound telemetry, opt-in aggregate collection, a bespoke Evidence feedback workflow, automatic resource provisioning from doctor, or a convenience switch that treats skipped required tests as a complete claim.

Next Steps

1. Define and review the normative public contract: EvidencePlan, EvidenceObligation, exact/glob policy subset, deterministic diff normalization/precedence, conservative unmatched behavior, explicit no-risk results, producer registration/outcome/assertion/artifact declarations, manifest schema, lifecycle state machine, and stable diagnostics.
2. Build a dependency-free standalone AppSurface EvidenceHost sample with one typed producer, explicit consumer registration, readiness deadlines, cancellation, cleanup, registered CI-envelope validation, and a secret-safe manifest—without changing the existing development AppHost model.
3. Extract the existing coverage runner/gate orchestration into an implementation-only friend-visible library, then wrap it as the first producer. Preserve CLI output ownership, patch-target diagnostics, scheduling, and the backwards-compatible command surface.
4. Build a PostgreSQL plus consumer-owned integration/E2E Skoolit acceptance pilot; prove that required browser evidence cannot be omitted from a release-profile claim and that an observation run cannot claim release evidence.
5. Threat-model trusted versus untrusted mode and add hostile-input tests for policy/registration/envelope substitution, paths, artifact hashing, cancellation, and cleanup failure. Propose an AppSurface-owned Docker runtime only after its distinct containment tests and operational exit criteria are approved.
6. Capture p50/p95 baseline metrics for planning, manifest generation, resource startup, cleanup, total wall time, and runner minutes; evaluate reviewed change fixtures for unnecessary-evidence and missed-risk selection. Promote only genuinely shared producer components into AppSurface’s managed toolkit.
7. Submit the Roslyn-backed structural-accessor classifier as a separate coverage-producer pilot with positive, near-miss, source-missing, and semantic-binding-failure tests.

What I noticed about how you think

• You set a quality bar around the lived feeling, not the metric: coverage should be “working exactly to ensure risk is mediated,” and it should not stall someone chasing “the last 3 lines of low-value coverage.”
• You guarded the incentive system: developers should neither “BOLSTER” numbers with redundant tests nor be punished by missing low-value execution.
• You corrected the product boundary immediately—“we are in AppSurface here”—and consistently kept Skoolit in its proper role as a consumer proving real pain.
• You spotted the architectural lever in Aspire, then named the crucial separation: “Test and Regular code should be intermingled as little as possible.”
• You did not stop at a local coverage fix. You pushed EvidenceHost to facilitate “the WHOLE evidentiary process during a CI run,” while accepting the contract-first boundary when the independent review exposed the CI-platform risk.

Decision Audit Trail

Decision	Choice	Why	Evidence	Reversible?
Autoplan review lenses	Skip visual-design review; run CEO, engineering, and developer-experience reviews	EvidenceHost is a developer platform and CI topology. The plan has no screens, visual states, or human interaction flows to assess, but it does define a CLI/library/Aspire developer experience.	Normative v1 contract and distribution plan	Yes; add a design review if a dashboard or UI is later proposed.
Independent review voices	Use approved combo/sub reviewer; record unavailable local Codex CLI reviewer	The local reviewer probe produced no usable capability and the bundled Codex binary path is absent. The plan still receives an independent review without fabricating a second voice.	Autoplan preflight, 2026-08-19	Yes; rerun with a working Codex reviewer in a later session.
Implementation approach	Contract-First EvidenceHost, confirmed by the user	It is the smallest approach that makes selected dependencies, coverage, E2E, cleanup, and the final claim part of one enforceable disposable topology without AppSurface becoming a general CI platform.	User reply B, 2026-08-19; independent CEO review	Yes; the plan retains an explicit managed-toolkit promotion path after shared lifecycle evidence exists.
CEO review mode	Selective expansion, confirmed by the user	Preserve the approved EvidenceHost baseline while considering only discrete additions that could materially strengthen risk mediation or the developer experience.	User reply B to D2, 2026-08-19	Yes; each proposed addition requires its own decision.
Risk-obligation contract	Include EvidenceObligation in plan and manifest	A complete claim must prove that each selected changed-risk obligation was closed by its required producer assertions, not merely that producers completed successfully.	User reply A to D3.1, 2026-08-19	Yes; versioned as an additive schema concept before a stable release.
Explicit quiet result	Include a no_evidence_required targeted verdict	Only an explicit no-obligation policy rule may select an empty evidence set. The plan and manifest expose the matched rule, paths, rationale, and empty obligation set; unexplained empty profiles remain invalid.	User reply A to D3.2, 2026-08-19	Yes; versioned as an additive schema concept before a stable release.
Deterministic policy resolution	Include explicit diff normalization, precedence, and table-driven conformance fixtures	A policy must have one reproducible answer for renames, deletions, overlaps, generated files, and unknown paths, or it recreates hidden CI conditions.	User reply A to D3.3, 2026-08-19	No for v1 schema semantics; future versions can add new explicit cases compatibly.
Trusted execution ownership	Validate a CI-provided trusted envelope in v1; defer an AppSurface-owned Docker sandbox to a separate security pilot	Preserve fail-closed trusted claims without expanding the first Evidence release into a portable constrained-execution runtime.	User reply B to D3.4, 2026-08-19; independent CEO review	Yes; promoted only after hostile-input and operational exit criteria are separately approved.
Quietness measurement	Add pilot overhead budgets plus unnecessary-evidence and missed-risk selection metrics	The champion-tier promise must be falsifiable: targeted evidence cannot become default until it is both safe and measurably quieter than the equivalent current evidence.	User reply A to D3.5, 2026-08-19	Yes; budgets may be revised from observed pilot baseline data.
Coverage producer seam	Extract coverage orchestration into a dedicated friend-visible internal library shared by the CLI and Evidence adapter	Evidence reuses the existing workflow in-process without making the current CLI internals a public consumer API. The implementation remains version-coupled and may be promoted to public only through a separate compatibility review.	User reply B with friend-visible constraint to D4.1, 2026-08-19	Yes; the adapter can fall back to CLI supervision if a future package boundary proves the internal seam unsuitable.
Public contract owner	Create dependency-free ForgeTrust.AppSurface.Evidence.Contracts	One package owns public plans, manifests, producer interfaces, assertion/diagnostic namespaces, schema media types, canonical serialization, and compatibility policy. Planner, CLI, Aspire, and consumer producer packages depend inward on it.	User reply A to D4.2, 2026-08-19	No for v1 package ownership; fields and contract versions evolve under the published compatibility policy.
Typed producer assertions	Include versioned producer context, assertion descriptors, and result validation	A producer may close only selected, registered, semantically compatible assertions through a bounded context; string conventions cannot create a trusted risk claim.	User reply A to D4.3, 2026-08-19	No for v1 contract behavior; new descriptors are additive under the compatibility policy.
Claim dimensions	Separate profile scope, execution verdict, claim kind, and gate eligibility	A targeted success is useful but structurally non-substitutable for a release claim; every downstream gate reads the same machine-verifiable dimensions.	User reply A to D4.4, 2026-08-19	No for v1 semantics; future claim kinds are additive and must declare eligibility.
Envelope claim data	Bind validated/unavailable/not-attested status, verifier/version, protected inputs, and capability evidence into the manifest	A validated CI envelope remains a constrained assertion rather than an implied sandbox attestation; release consumers must explicitly accept it.	User reply A to D4.5, 2026-08-19	No for v1 security claim semantics; an attested status is a separate future contract.
EvidenceHost lifecycle	Add a dedicated base-owned bootstrap/registration contract with explicit Aspire ownership and failure ordering	Evidence lifecycle is independent of normal profile discovery and owns deterministic build/start/readiness/stop/dispose semantics.	User reply A to D4.6, 2026-08-19	No for v1 bootstrap shape; supported Aspire range may evolve under compatibility policy.
Deterministic acceptance matrix	Require golden schemas/hashes, CLI regressions, packed-consumer compatibility, fake seams, lifecycle cancellation, hostile inputs, and bounded real E2E	Claim semantics must be exhaustively verified without sleeps or wall-clock dependence; existing coverage test seams remain preserved.	User reply A to D4.7, 2026-08-19	No for v1 verification bar; fixture corpus evolves additively with the contract.
Hard execution bounds	Define aggregate resource/output/count/concurrency/time limits with stable terminal outcomes and a reproducible performance methodology	Quality budgets catch systemic noise; hard limits prevent a single producer from causing unbounded runner cost or artifact volume.	User reply A to D4.8, 2026-08-19	Yes; numeric defaults may change compatibly when the manifest records the applied limits.
Evidence package graph	Publish layered contracts, planner, CLI/coverage, Aspire host, and consumer-producer packages with compatibility/rollback rules	Non-Aspire consumers never acquire Aspire; the friend-visible coverage core remains an implementation dependency; packed consumers prove coexistence with existing Aspire testing.	User reply A to D4.9, 2026-08-19	No for v1 dependency direction; supported version ranges evolve through package compatibility policy.
Primary developer persona	Consumer platform engineer / test-infrastructure maintainer	This developer assembles CI evidence for a mature AppSurface consumer and needs deterministic policy, safe defaults, explicit trust boundaries, and a clear upgrade path; feature engineers remain a secondary read-only consumer of the result.	User reply A to D5.1, 2026-08-19	Yes; future adoption research may change which persona the default guides optimize for.
Champion TTHW target	Under two minutes to a trustworthy explain/dry-run result; real dependency/browser execution may take longer	The user repeatedly set champion-tier DX as the product bar. First success is a zero-risk plan explanation, not misleadingly fast full E2E execution.	Existing user direction; DX research, 2026-08-19	Yes; revise from observed preview onboarding data.
First-success delivery	appsurface evidence init --sample generates a non-overwriting starter, then explain/run emit terminal and GitHub summaries plus machine artifacts	Existing-repo adoption needs one obvious path to a dedicated EvidenceHost; a human-readable summary is the front door while JSON/manifest remain automation evidence.	User reply A to D5.3, 2026-08-20	Yes; a future template may complement init for greenfield repos.
Prerequisite preflight	Add appsurface evidence doctor as a non-provisioning capability preflight	Platform maintainers should learn about missing SDK, policy/diff, envelope, Docker, database, and browser prerequisites before a costly or confusing evidence run; the command reports one concrete next action without creating resources.	User reply A to D5.4, 2026-08-20	Yes; the checks can evolve as supported producer capabilities expand.
Developer-experience posture	DX expansion	EvidenceHost needs a small but complete adoption loop—guided setup, transparent planning, bounded execution, helpful summary artifacts, a cookbook, and claim-quality measurement—to reach the stated champion-tier bar.	User reply A to D5.5, 2026-08-20	Yes; future consumer evidence can prioritize, defer, or narrow individual adoption surfaces.
Guided command model	Separate init, doctor, explain, run, and verify commands	Each operation has a bounded, unsurprising responsibility: setup does not overwrite, doctor does not provision, explain does not execute, run is explicitly operational, and verify is usable by downstream gates.	User reply A to D5.6, 2026-08-20	Yes; aliases or a guided interactive wrapper may be added after the stable noninteractive contract proves itself.
Adoption telemetry	No structured preview adoption measurement	Keep the initial EvidenceHost preview free of outbound telemetry, opt-in aggregation, and a new feedback process. Per-run performance and correctness data remains only where it is required to support a truthful evidence claim.	User reply C to D5.7, 2026-08-20	Yes; collect consented adoption research later if a specific product decision needs it.
Preview documentation	Ship a first-class EvidenceHost guide and scenario cookbook	The generated starter is the shortest path to first success, while task-based guidance makes real persistence, browser, unavailable-dependency, and release cases legible without treating the reference contract as a tutorial.	User reply A to D5.8, 2026-08-20	Yes; expand or retire scenarios based on concrete consumer support needs.
Autoplan approval	Approve the reviewed Contract-First EvidenceHost plan as written	The plan meets the contract, safety, engineering, and champion-tier DX gates while preserving the explicit deferred boundary; implementation remains a separate authorized action.	User reply A to final approval gate, 2026-08-20	Yes; future approved revisions create a new review cycle.

Autoplan CEO Review

What already exists and will be reused

Existing surface	EvidenceHost reuse	Boundary to preserve
coverage run	Discovery, exclusive-project scheduling, watchdog, process supervision, merge, and owned coverage artifacts become the first built-in producer.	Its orchestration moves once into a friend-visible implementation library shared with the CLI; Evidence must not rebuild test execution.
coverage gate	Cobertura threshold and patch-target evaluation remains the coverage verdict producer.	Its orchestration moves once into the same friend-visible implementation library; risk obligations do not reinterpret line or branch arithmetic.
ForgeTrust.AppSurface.Aspire	Aspire resource composition, references, health checks, and disposal provide the disposable topology.	Existing entry-assembly component discovery is for normal AppHosts. EvidenceHost uses explicit, base-owned registrations only.
ForgeTrust.AppSurface.Aspire.Testing	Its lifecycle discipline is a useful reference for deterministic build/disposal behavior.	It remains an application-test builder; it is not the EvidenceHost package or claim engine.
Consumer CI	Checkout, runner, cache, secret, artifact-retention, and sandbox policy remain consumer-owned.	Evidence records validated outcomes but does not become a CI scheduler, artifact store, or deployment controller.

Premise challenge, alternatives, and temporal check

The core premise survives review: coverage alone is not an evidence-completeness model, because neither a branch percentage nor a green selected subset proves that the required dependency/E2E/release obligations ran. The plan also deliberately rejects the inverse mistake—turning every line into behavior or forcing repository-wide coverage from an incomplete profile. Policy may choose a no-risk result only explicitly; incomplete required evidence remains incomplete.

Approach	Effort / risk	Why it was not selected
Tune line/branch thresholds and add exclusions	Low effort; high false-signal risk	It cannot express skipped required tests, topology readiness, E2E, or release evidence; broad exclusions would recreate low-value gaming.
Consumer-specific workflow conventions	Medium effort; medium maintenance risk	It can help Skoolit once, but leaves each consumer to rebuild policy, claim semantics, and failure reporting.
Contract-First EvidenceHost	Medium effort; controlled public-contract risk	It centralizes reusable evidence semantics while leaving CI, deployment, secrets, and consumer topology ownership where they belong.
AppSurface-managed CI/sandbox platform	High effort; high security/operational risk	It expands into runner isolation, scheduling, artifact storage, and deployment territory before the core claim model is proven.

In hour one, init, doctor, and explain make the selected work legible without external infrastructure. In the first consumer pilot, the coverage adapter and PostgreSQL/browser obligations prove that a required profile cannot silently omit evidence. In six months, only producer lifecycle components repeatedly proven across consumers become managed toolkit candidates; the deferred sandbox, attestation, aggregation, content-sensitive planning, and structural classifier remain independent product decisions.

Dream state delta

CURRENT
  Coverage commands, browser tests, dependencies, and release checks are separate CI facts.
  A green percentage can be low-value; a missing dependency can silently remove proof.
       |
       v
THIS PLAN
  A deterministic policy compiles a diff into one profile, risk-obligation set,
  disposable Aspire topology, typed producer set, and truthful manifest.
       |
       v
12-MONTH IDEAL
  AppSurface consumers get quiet, risk-proportional evidence by default; shared
  producer lifecycles are promoted only after repeated consumer evidence proves them reusable.

Architecture and data-flow review

trusted-tool checkout (base-owned)                         subject checkout (head-derived)
  policy + registrations + envelope verifier                   product + tests + fixtures
                |                                                        |
                +-------------------+------------------------------------+
                                    v
                       Evidence CLI / planner
                 normalize diff -> resolve profile + obligations
                                    |
               invalid/ambiguous ---+---> invalid, no claim
                                    |
                                    v
                    consumer-owned explicit EvidenceHost
       registered CI envelope -> Aspire resources -> typed producers
                 |                    |                    |
                 |                    |                    +--> coverage CLI / consumer E2E
                 |                    |
                 |                    +--> unavailable/timed_out -> incomplete
                 v
    collect assertion closures + artifacts -> cleanup -> EvidenceManifest
                 |                                  |
                 +-- obligations closed ------------+--> complete / partial
                 +-- unmediated risk or cleanup failure --> incomplete / invalid

Input path	Happy path	Missing/empty path	Upstream/error path
Diff to policy resolution	A normalized diff selects one profile and obligation set with rationale.	Empty diff requires an explicit policy rule or fails; no implicit empty profile exists.	Invalid diff, unknown path without a conservative profile, or equal-precedence conflict fails before startup.
Envelope validation	Base-owned verifier accepts the registered CI declaration.	Missing or incomplete declaration fails closed for a complete claim.	Verifier mismatch is invalid; observation mode never upgrades itself.
Resource readiness	Every declared resource becomes healthy before dependent producers start.	No resources are permitted only for an explicit no-evidence plan.	Deadline or unavailable state yields stable unavailable and no claim.
Producer to obligation closure	Declared producer returns its declared assertions and artifacts.	A passed producer that returns no required assertion leaves the obligation unclosed.	Timeout, flake, invalid artifact, or assertion mismatch yields no complete claim.
Final manifest	Hashes and bounded metadata verify against the plan.	No-evidence manifest carries its matched rule and empty obligation set.	Missing required result, extra producer, path escape, cleanup failure, or hash mismatch is invalid/incomplete.

The lifecycle remains intentionally closed. An EvidenceHost registration is explicit, not discovered from the subject or from every referenced assembly; this prevents normal-development composition conveniences from becoming a trusted control-plane input.

Error and rescue registry

Codepath	What can go wrong	Stable outcome/diagnostic	Rescue action	Developer sees
Diff normalizer	Missing, malformed, or non-canonical diff	invalid	Stop before provisioning; show source and remediation.	“Evidence plan invalid: diff input …”
Policy resolver	Unknown path, ambiguous precedence, undeclared combined profile	invalid	Select the named conservative profile only where policy permits; otherwise stop.	Rule/path trace and required policy fix.
Envelope verifier	Missing, head-derived, or unsupported CI envelope	invalid for trusted claim	Refuse trusted execution; allow only separately selected observation path.	“Trusted evidence unavailable: envelope …”
Resource readiness	Health check fails or deadline elapses	unavailable / timed_out	Cancel dependent producers, collect bounded diagnostics, clean up.	Dependency ID, deadline, last state, remediation.
Producer execution	Product assertion fails, is flaky, times out, or is cancelled	failed, flaky, timed_out, cancelled	Preserve producer result and artifacts, then clean up.	Producer ID, obligation IDs, exact outcome, artifact links.
Artifact collector	Path escape, duplicate slot, media/size/hash mismatch	invalid	Reject result and clean up; never ingest raw undeclared content.	Slot and validation reason.
Obligation closer	A required assertion is absent despite a passed producer	incomplete	Do not synthesize coverage or success.	Unclosed obligation, required assertion, producer result.
Cleanup	Resource/process cleanup fails	incomplete	Record cleanup diagnostic and retain consumer-CI cleanup evidence.	Cleanup target and manual recovery instructions.

Failure modes and operational proof

Codepath	Failure mode	Rescued?	Test?	Developer sees	Logged?
Planner	Overlapping glob rule has no precedence	Yes: fail closed	Table-driven policy conformance	Rule IDs and conflicting paths	Yes
No-op result	Empty producer set is selected without explicit no-risk rule	Yes: invalid	Unit + manifest test	“Unexplained empty profile”	Yes
Readiness	Resource becomes unavailable before healthy	Yes: bounded stop	Aspire lifecycle integration test	Resource state/deadline	Yes
Producer	Coverage producer passes but does not close persistence obligation	Yes: incomplete	Contract integration test	Unclosed obligation and assertion ID	Yes
Artifact	Producer attempts a path escape	Yes: invalid	Artifact adversarial test	Rejected slot/path	Yes
Trust	Subject changes policy or envelope registration	Yes: invalid	Hostile-input integration test	Trusted-control-plane violation	Yes
Cleanup	Browser descendant survives cancellation	Yes: incomplete	Process-tree cleanup test	Cleanup recovery code	Yes
Release claim	Required browser E2E was omitted	Yes: incomplete	Skoolit-like acceptance test	Missing producer/obligation	Yes

No failure path is allowed to become a complete or release claim. The CI provider retains raw diagnostic/media retention; the AppSurface manifest retains only bounded, secret-free references and hashes.

Verification strategy

Policy unit/conformance
  -> exact, glob, rename, deletion, generated, unknown, overlap, no-risk selection

Contract unit
  -> schema/hash/registration/envelope/artifact/obligation closure validation

EvidenceHost integration
  -> readiness, timeout, cancellation, cleanup, coverage producer invocation

Consumer acceptance
  -> PostgreSQL persistence obligation + consumer-owned E2E + release omission failure

Security/adversarial
  -> head-control-plane substitution, path escape, undeclared producer, secret-free manifest

Future Docker runtime pilot
  -> denied egress, mount/process containment, image identity, descendant cleanup

The primary test pyramid is many deterministic policy/contract tests, fewer disposable Aspire integration tests, and a small number of expensive consumer browser journeys. The 2am-on-Friday test is a release-profile run that fails loudly when the browser producer is omitted or a passed producer leaves its risk obligation unclosed.

Performance, observability, rollout, and long-term trajectory

• Record planner and manifest duration, resource startup/cleanup duration, total wall time, runner minutes, profile selection frequency, producer outcomes, obligation-closure rate, unnecessary-evidence rate, and missed-risk rate in the evidence/fixture outputs required for claim and quality verification. Do not collect secrets, raw fixture data, head-controlled free-form labels, or adoption/usage telemetry.
• Ship as preview packages and opt-in policy/host registrations. Existing coverage run and coverage gate remain unchanged; disabling/removing an EvidenceHost registration is the immediate rollback.
• Rollout order: dependency-free sample, coverage producer, PostgreSQL plus consumer E2E acceptance, then promotion of only repeatedly proven managed components. No deployment, traffic, production data, or production-secret integration is part of this sequence.
• This is a 4/5 reversible architecture decision: public contract names need versioning discipline, but profiles, producer registrations, and managed-component promotion remain opt-in and additive.
• The explicitly deferred AppSurface-owned Docker runtime, signed attestation, cross-job aggregation, content-sensitive planning, and broad structural-code filtering remain separate decisions. They must not be smuggled into a consumer pilot.

Scope-expansion record

• Accepted: risk obligations, explicit no_evidence_required, deterministic policy resolution/conformance, CI-provided envelope validation with a separate Docker-runtime pilot, and quietness budgets/selection metrics.
• Deferred: AppSurface-owned Docker security runtime, signed attestations, cross-job aggregation, content-sensitive planning, Roslyn structural-accessor classification, and additional managed producers.
• Skipped: no accepted change was discarded as low value during this review.

Not in scope

EvidenceHost v1 does not become a hosted CI product, CI scheduler, deployment controller, secret manager, artifact store, generic plugin runtime, cross-job aggregator, or normal-development AppHost extension. It does not make coverage arithmetic itself a risk classifier, infer full evidence from a skipped test profile, silently run a release profile, or implement the deferred Docker sandbox, signed attestation, content-sensitive policy engine, or Roslyn structural-line classifier. Those exclusions preserve the first release’s central promise: a truthful, bounded, consumer-owned evidence claim.

Implementation Tasks

• T1 (P1, human: ~2d / CC: ~30min) — Evidence contracts — Add EvidencePlan, EvidenceObligation, manifest closure data, stable verdicts, and schema/hash validation.
  • Surfaced by: CEO Sections 1, 2, and 4 — producer success is not sufficient proof that a changed risk was mediated.
  • Files: new Evidence contract/planner package and contract tests.
  • Verify: deterministic unit tests for complete, partial, unclosed, invalid, and no-risk manifests.
• T2 (P1, human: ~1d / CC: ~20min) — Evidence policy — Implement normalized-diff resolution, deterministic precedence, conservative fallback, and explicit no-risk rules.
  • Surfaced by: CEO Sections 1 and 6 — hidden policy ambiguity recreates hidden CI conditions.
  • Files: new policy parser/resolver plus table-driven fixtures.
  • Verify: rename, deletion, overlap, generated, unknown, and empty-diff conformance matrix.
• T3 (P1, human: ~2d / CC: ~30min) — EvidenceHost lifecycle — Provide a separate explicit-registration Aspire EvidenceHost package with bounded readiness, cancellation, and cleanup.
  • Surfaced by: CEO Sections 1, 2, and 5 — normal AppHost auto-discovery cannot be a trusted evidence control plane.
  • Files: new EvidenceHost package, sample, lifecycle integration tests, and package documentation.
  • Verify: resource-unavailable, timeout, cancellation, cleanup-failure, and no-evidence lifecycle tests.
• T4 (P1, human: ~1d / CC: ~20min) — Trust boundary — Add the base-owned registered CI-envelope verifier and observation-mode claim restrictions.
  • Surfaced by: CEO Section 3 — a trusted claim must fail closed without pretending to be a CI sandbox attestation.
  • Files: EvidenceHost trust contracts, hostile-input tests, and CI-template documentation.
  • Verify: head-controlled registration/policy/envelope substitution is rejected; observation cannot emit complete/release.
• T5 (P2, human: ~1d / CC: ~20min) — First producer and acceptance pilot — Adapt existing coverage CLI, then prove PostgreSQL and consumer E2E obligations in a Skoolit-like host.
  • Surfaced by: CEO Sections 6 and 9 — existing coverage machinery must be reused and release omission must be visible.
  • Files: coverage producer adapter, sample/acceptance host, docs.
  • Verify: omitted browser producer and unclosed obligation both produce an incomplete release claim.
• T6 (P2, human: ~1d / CC: ~15min) — Evidence quality measurement — Produce secret-free overhead and selection-quality reports with reviewed fixtures and opt-in enforcement budgets; do not add adoption/usage telemetry.
  • Surfaced by: CEO Sections 7 and 8 — quietness requires measurable unnecessary-evidence and missed-risk rates.
  • Files: manifest diagnostics, fixture harness, preview documentation.
  • Verify: p95 and runner-minute reports; budget-exceeding profile stays opt-in.

CEO completion summary

+====================================================================+
| MEGA PLAN REVIEW — CEO SUMMARY                                    |
+====================================================================+
| Mode                 | SELECTIVE EXPANSION                        |
| System audit         | Coverage is producer-grade; Aspire auto-   |
|                      | discovery must not own trusted composition |
| Scope decisions      | 5 proposed, 5 accepted, 0 deferred         |
| Architecture         | Explicit registrations + obligation closure |
| Error/security       | Fail-closed taxonomy and CI-envelope truth  |
| Tests                | Conformance -> contract -> integration ->   |
|                      | consumer acceptance -> future containment   |
| Performance          | 30s p95 planning/manifest; <=10% runner     |
|                      | minutes; selection-quality fixture corpus   |
| Design & UX          | SKIPPED: no human-facing UI scope            |
| Outside voice        | Approved combo/sub review; 6 findings, all  |
|                      | resolved through explicit user decisions     |
+====================================================================+

Independent CEO review consensus

The local Codex reviewer was unavailable during preflight. The approved independent combo/sub reviewer supplied the strategy challenge; its concerns were resolved through the user-confirmed Contract-First approach, obligations, explicit quiet result, deterministic policy, bounded CI-envelope trust model, and quietness budgets.

CEO DUAL VOICES — CONSENSUS TABLE
═══════════════════════════════════════════════════════════════
  Dimension                           combo/sub    Codex        Consensus
  ──────────────────────────────────── ──────────── ──────────   ──────────────────
  1. Premises valid?                   revised      unavailable  resolved with explicit obligations/no-op
  2. Right problem to solve?           yes          unavailable  Contract-First EvidenceHost
  3. Scope calibration correct?        revised      unavailable  CI sandbox/runtime deferred
  4. Alternatives sufficiently explored? yes        unavailable  selected against CI-platform alternative
  5. Competitive/market risks covered? revised      unavailable  quieter-risk metrics and consumer pilot
  6. 6-month trajectory sound?         yes          unavailable  managed components only after proof
═══════════════════════════════════════════════════════════════
Missing voice is N/A, not a fabricated confirmation. No unresolved CEO finding remains.

Autoplan Engineering Review

Package and dependency architecture

                         Evidence.Contracts
                         (public, no Aspire/CLI)
                           /       |        \
                          v        v         v
                 Evidence.Planner  Evidence.Aspire  consumer producer package
                      |                  |                    |
                      v                  v                    v
              Evidence.Cli ------> EvidenceHostBootstrap <- explicit base-owned registration
                      |                  |
                      v                  +--> Aspire.Hosting (tested range only)
       Evidence.Coverage (internal; friend-visible only)
                      ^
                      |
       existing appsurface coverage run/gate CLI commands

• Contracts owns the public semantic language. Planner owns deterministic resolution. Aspire owns no policy or CLI dependency. The CLI owns the public coverage command. The coverage implementation library is internal and friend-visible only to the CLI and first-party Evidence adapter.
• EvidenceHostRegistration is explicit trusted-base code. Consumer producers compile against Contracts but are registered only by that module. No runtime discovery is allowed.
• Every package pins the coordinated preview version and declares its tested target-framework/Aspire range. Unsupported range, package downgrade, or contract-major mismatch fails before claim formation. Removal of Evidence packages/registration leaves normal AppHosts and current CLI coverage behavior unchanged.

Contract and lifecycle verification matrix

EvidenceProducerContext
  planId + producer registration + scoped values + TimeProvider + cancellation
       |
       +--> bounded ArtifactWriter ----> declared ArtifactSlots ----> hash/size/path validation
       |
       +--> bounded DiagnosticSink ----> manifest-safe diagnostics
       |
       +--> EvidenceAssertionDescriptor@version
                |
                +--> selected obligation requirement --> closed / unclosed

EvidenceHostBootstrap
  validate plan + envelope --> build once --> start --> readiness --> producers
       |                      |              |          |              |
       |                      +--> build fail |          |              +--> collect limits/assertions
       |                                     v |          v
       +--> invalid                         cleanup <--- unavailable/timed_out/cancelled

Test level	Required proof
Contract/golden	Canonical plan/manifest bytes and digests; reader/writer compatibility; diagnostic, assertion, claim-kind, and limit validation.
Policy conformance	Exact/glob precedence, renames, deletes, generated paths, unknown paths, no-risk results, and conservative combinations.
CLI regression	Existing coverage run/gate command syntax, exit codes, patch-target output, artifact names, output ownership, scheduling, and cleanup do not regress after extraction.
Lifecycle integration	Explicit bootstrap only; failed partial build; cancellation at validation/build/start/readiness/producer/collection/cleanup; readiness unavailable; completion failure; health regression; primary-error preservation.
Adversarial	Subject attempts policy/registration/envelope substitution; duplicate/unknown/stale assertions; Unicode/case/path normalization; symlink/reparse escape; oversized artifact/log/diagnostic; malformed JSON; descendant process.
Packed consumer	Restore/build consumers of Contracts/Planner/CLI/Aspire, including coexistence with ForgeTrust.AppSurface.Aspire.Testing; unsupported package combinations fail clearly.
Acceptance E2E	PostgreSQL-backed persistence obligation, consumer-owned browser journey, release producer omission, and observation-mode non-eligibility.

Deterministic tests use fake TimeProvider, process-tree supervisor, resource-state/readiness provider, and bounded virtual artifact filesystem. They never depend on sleeps or wall-clock timing. Real Aspire/browser tests are reserved for topology and user-journey proof.

Engineering failure and performance review

Area	Failure that must be tested	Contract response
Contract/version	Reader cannot consume contract major or claim-critical enum	invalid before claim evaluation.
Producer context	Producer attempts undeclared assertion/artifact or exceeds sink limit	invalid; no partial promotion of unvalidated result.
Envelope	Runner, protected checkout, root, verifier, or capability hash changes after validation	invalid; release eligibility never inferred.
Aspire host	Build/start/readiness fails or health regresses	Primary fault retained; dependant work cancelled; unavailable/failed/timed_out as applicable.
Aggregate limits	Count/byte/path/identifier/concurrency bound exceeded	invalid; deadline expiry is timed_out; cleanup-budget expiry is incomplete.
Coverage extraction	Existing CLI consumer receives changed artifacts or exit semantics	Regression failure blocks release of the friend-visible library.

The planner itself is bounded by linear normalized-diff/rule evaluation under v1 exact paths and segment globs. No content scanning, arbitrary code detector, plugin discovery, or cross-job aggregation enters the hot path. Aggregate limits cap memory and output; the profile-promotion quality methodology evaluates warm/cold restore separately, ignores failed-job samples, and requires at least 20 successful observations for p50/p95.

Engineering not in scope

The first implementation does not change the public behavior of existing coverage run/gate, publish the friend-visible coverage core as a consumer API, make normal Aspire component discovery trusted, provide a generic resource orchestration engine, or rely on wall-clock sleeps/reflection to test private internals. The later Docker runtime has its own hostile-input containment proof; it must not be folded into EvidenceHost lifecycle work merely because both touch ephemeral dependencies.

Parallel implementation lanes

Lane	Modules	Depends on
A	Evidence.Contracts, Evidence.Planner, contract fixtures	—
B	Coverage orchestration extraction and CLI regression suite	A only for adapter DTOs; can begin extraction independently.
C	Evidence.Aspire, bootstrap, envelope contracts, lifecycle tests	A
D	Dependency-free sample, PostgreSQL/E2E acceptance, metric fixture cohort, documentation	A, B, and C

Launch A and B in parallel. After A, launch C. Merge A+B+C, then execute D as the integration lane. The only predicted merge conflict is A/B around the friend-visible coverage adapter types; keep those types owned by A and have B consume the approved contract.

Engineering implementation tasks

• E1 (P1, human: ~1d / CC: ~20min) — Contracts — Create the dependency-free Contracts package, canonical serializers/digests, compatibility rules, claim dimensions, typed producer/assertion API, and golden fixtures.
  • Verify: reader/writer compatibility, invalid enum/version cases, claim-kind truth table, canonical hash tests.
• E2 (P1, human: ~1d / CC: ~20min) — Planner — Implement canonical normalized diff, deterministic policy resolution, no-risk result, obligation closure, and hard-limit validation.
  • Verify: table-driven change matrix plus aggregate-limit terminal outcomes.
• E3 (P1, human: ~1d / CC: ~20min) — Coverage extraction — Move existing orchestration once into friend-visible Evidence.Coverage internals used by CLI and the Evidence adapter.
  • Verify: existing CLI behavior, artifacts, exit codes, scheduling, and package consumer tests remain unchanged.
• E4 (P1, human: ~2d / CC: ~30min) — Evidence.Aspire — Implement explicit-registration bootstrap, lifecycle ownership, registered CI-envelope verifier, and cancellation/cleanup/error precedence.
  • Verify: integration matrix for failed build, readiness, completion, health regression, cancellation, cleanup, and envelope identity drift.
• E5 (P2, human: ~1d / CC: ~20min) — Packages/docs — Publish graph/support matrix, compatibility/rollback rules, packed consumers, and distinct guidance for Evidence.Aspire versus Aspire.Testing.
  • Verify: restore/build fixtures for each supported dependency combination.
• E6 (P2, human: ~1d / CC: ~20min) — Acceptance and metrics — Build the PostgreSQL plus consumer-E2E pilot and fixture-cohort performance/selection reports.
  • Verify: release omission, observation, quiet no-risk, 20-sample p50/p95, and runner-minute rules.

Engineering completion summary

Step 0 scope:       Accepted as revised by CEO decisions
Architecture:        8 issues found, 8 resolved through explicit decisions
Code quality:        Clear package ownership and no duplicate coverage workflow
Test review:         Contract-to-E2E matrix defined; deterministic seams mandatory
Performance:         Hard limits plus reproducible quality methodology
Critical gaps:       0 remaining
Outside voice:       combo/sub review; all substantive findings accepted
Parallelization:     4 lanes, A+B parallel, then C, then D

Independent engineering review consensus

The local Codex reviewer was unavailable during preflight. The approved independent combo/sub reviewer identified eight engineering gaps; the user accepted every corrective contract/lifecycle/package decision, and the engineering matrix above turns each one into a deterministic verification requirement.

ENGINEERING DUAL VOICES — CONSENSUS TABLE
═══════════════════════════════════════════════════════════════
  Dimension                           combo/sub     Codex        Consensus
  ──────────────────────────────────── ────────────  ──────────   ──────────────────
  1. Package ownership/layering        gap resolved  unavailable  Contracts inward; coverage friend-visible
  2. Public contract rigor             gap resolved  unavailable  typed producer/assertion semantics
  3. Claim/gate correctness            gap resolved  unavailable  explicit claim dimensions
  4. Trust boundary                    gap resolved  unavailable  structural CI-envelope facts
  5. Aspire lifecycle ownership        gap resolved  unavailable  explicit bootstrap/order
  6. Test determinism                  gap resolved  unavailable  full acceptance matrix/fake seams
  7. Resource/performance bounds       gap resolved  unavailable  hard limits and p95 method
  8. Package compatibility/rollback    gap resolved  unavailable  support graph and packed proof
═══════════════════════════════════════════════════════════════
Missing voice is N/A, not a fabricated confirmation. No unresolved engineering finding remains.

Autoplan Developer Experience Review

DX framing and market calibration

The primary persona is a consumer platform engineer integrating a new evidence surface into an existing repository and CI workflow. Their first concern is not “can I execute this test?” but “what work will policy require, what external capability is missing, and will a partial run be mistaken for release confidence?” The secondary persona is a feature engineer reading the generated explanation after a CI result.

The reviewed competitive patterns support a deliberately boring first run. .NET custom templates are a good greenfield mechanism, but appsurface evidence init --sample is better for adding a clearly separated EvidenceHost to an existing consumer. Testcontainers for .NET makes disposable dependency testing approachable but still requires a usable Docker runtime, so Docker/database/browser checks belong in an explicit preflight rather than a surprise during execution. GitHub job summaries are the natural CI front door for concise human results. The Dagger CI quickstart demonstrates why topology-first systems can be powerful, but its longer workshop-shaped setup reinforces the choice to make a non-mutating policy explanation—not full topology execution—the first success.

Independent DX review consensus

The local Codex reviewer was unavailable during autoplan preflight, so this phase used the approved independent combo/sub reviewer and does not fabricate two-model confirmation. Its six gaps—starter generation, explicit commands, preflight, human summary, task-based docs, and an improvement loop—were reviewed with the user. The user accepted the first five; the sixth was intentionally narrowed to existing voluntary feedback plus claim-quality measurements, with no adoption telemetry or new process.

DX DUAL VOICES — CONSENSUS TABLE
═══════════════════════════════════════════════════════════════
  Dimension                         combo/sub      Codex        Consensus
  ─────────────────────────────────  ────────────   ──────────   ──────────────────
  1. Getting started < 5 min?        gap resolved   unavailable  resolved by init/doctor/explain
  2. API/CLI naming guessable?       gap resolved   unavailable  resolved by five commands
  3. Error messages actionable?      gap resolved   unavailable  resolved by doctor + summaries
  4. Docs findable & complete?       gap resolved   unavailable  resolved by start page/cookbook
  5. Upgrade path safe?              preserved      unavailable  covered by contract/package policy
  6. Dev environment friction-free?  gap resolved   unavailable  resolved by non-provisioning preflight
═══════════════════════════════════════════════════════════════
Missing voice is N/A, not a fabricated confirmation. No unresolved DX finding remains.

Developer empathy narrative

“I own CI confidence for a product that already has coverage, integration tests, browser tests, and a normal Aspire host. I do not want another percentage gate or a tool that quietly launches a database and leaves me to infer why. I need a fast answer about what this diff requires, a straightforward way to see whether my runner can provide it, and a result that cannot overstate confidence when one required suite is skipped. If something fails, tell me whether I need to change my policy, test, environment, or release gate—and show the exact next move.”

Nine-stage journey, friction, and intentional fixes

Journey stage	Initial friction	Contract-first resolution	Acceptance proof
1. Discover	Evidence could appear to be another coverage switch, normal AppHost, or CI platform.	Root, CLI, contracts, and Aspire docs link to one EvidenceHost start page that states the boundary and names the primary user.	Documentation-link test and handbook review.
2. Assess fit	A maintainer cannot tell whether an incomplete profile is supported or whether Evidence replaces current coverage.	Start page names the exact non-goals and retains coverage run/gate as independent, compatible commands.	Documentation scenario distinguishes optional adoption from partial-claim misuse.
3. Install/init	A platform maintainer might hand-author an unsafe or mixed-topology host.	init --sample creates an explicitly marked, non-overwriting starter and prints the next command.	Existing-repository fixture verifies planned writes, no overwrite by default, explicit marked-file replacement only.
4. Configure policy	Silent rule overlap or a convenient omitted test suite could create an untruthful default.	Generated starter points at explicit policy/registration; policy resolution is deterministic and no unexplained empty profile is valid.	Table-driven policy fixtures and generated scenario comments.
5. Preflight	Docker/browser/CI availability could turn first contact into an infrastructure error.	doctor delivers a local, non-mutating capability report before execution.	Fixture needs no Docker, browser, secrets, resource, or producer execution.
6. First success	A plan command might be mistaken for an execution or coverage claim.	explain makes the selected profile and obligations visible without starting topology or creating a claim.	Truthful result inside the two-minute TTHW target.
7. Real integration	A successful producer could be mistaken for sufficient evidence, or omitted suite portions could be invisible.	run is explicit and claim closure is obligation-based; summaries name required, closed, and missing producers/obligations.	PostgreSQL/browser acceptance fixtures show complete, no-risk, unavailable, and omitted-producer states.
8. Debug	Raw exit codes and scattered CI artifacts make a three-line coverage failure feel arbitrary.	Terminal/GitHub summaries lead with the claim result and one next action; JSON/manifest retain exact evidence.	Golden terminal, job-summary, JSON summary, and manifest results agree for each terminal outcome.
9. Upgrade or exit	A preview mismatch could surface late, or adoption could become sticky.	Version/range checks run before execution; migration notes link to the required action; removing registration leaves normal hosts/current coverage unchanged.	Packed old/new consumer fixtures, unsupported-combination diagnostics, and rollback test.

Confusion report

Without the DX additions, a platform maintainer would reasonably be confused by at least four things: whether EvidenceHost belongs in the normal AppHost; whether a dry plan actually runs tests; whether missing Docker/Postgres/browser capability means a product failure; and whether a green coverage producer proves a release claim. The first three are eliminated by the dedicated host starter and the doctor/explain/run separation. The fourth is eliminated by the obligation and claim-dimension summary. The intentional remaining constraint is that an engineer must write explicit policy and consumer registration rather than allow topology discovery; that is a learning cost, but it is necessary to keep the trusted evidence control plane base-owned and deterministic.

DX scorecard

Scores are for the planned preview rather than the current repository; all axes are expected to meet the champion threshold before package promotion.

Dimension	Before DX decisions	Reviewed plan	Evidence for the reviewed score
Getting started / TTHW	4	9	Generated non-overwriting starter plus under-two-minute doctor/explain, without external dependencies.
CLI/API clarity	7	9	Five commands have one responsibility and stable noninteractive semantics.
Error quality	8	9	Capability-, contract-, and claim-specific messages lead with one next action.
Documentation	5	9	Dedicated start page, cross-links, reference, and five scenario cookbook entries.
Upgrade path	7	9	Contract/package range validation, packed fixtures, migration notes, and no best-effort interpretation.
Environment/setup	5	9	doctor makes selected producer requirements visible before resource/test execution.
Community / feedback	6	8	The existing voluntary Docs/DX issue form remains available; Evidence adds no telemetry or bespoke intake process.
Measurement / improvement	8	8	Claim/quality fixture outputs retain performance and selection evidence; the user intentionally excluded adoption telemetry and aggregation.
Overall	6.3	8.8	No reviewed dimension remains below 7; the design meets the DX quality gate.

DX implementation tasks

• DX1 (P1, human: ~1d / CC: ~20min) — Guided CLI — Add evidence init --sample, doctor, explain, run, and verify command contracts to the existing tool without changing coverage run/gate behavior.
  • Verify: help/output/exit-code fixtures prove each command has its defined non-mutating or execution responsibility; init cannot overwrite an unmarked file.
• DX2 (P1, human: ~1d / CC: ~20min) — Explain and diagnostics — Implement stable doctor outcome/capability details, explanation traces, next-action diagnostics, and contract-invalid versus unavailable distinction.
  • Verify: golden tests cover ready, external prerequisite, blocked, ambiguous policy, bad envelope, no-risk, dependency unavailable, and omitted release producer.
• DX3 (P1, human: ~1d / CC: ~20min) — Human evidence front door — Emit bounded terminal, GitHub step-summary, and evidence-summary.json views that agree with the plan/manifest but never become an alternate gate format.
  • Verify: golden outputs are path-safe/secret-free, point at bounded artifact references, and agree on every claim dimension.
• DX4 (P2, human: ~1d / CC: ~20min) — Discoverability and cookbook — Publish the EvidenceHost start page, cross-links, API reference, migration notes, and the five approved task-based scenarios.
  • Verify: link checker and rendered documentation review; every scenario includes policy intent, commands, expected summary, manifest result, prerequisite caveat, and remediation link.
• DX5 (P2, human: ~0.5d / CC: ~10min) — First-success regression — Add an isolated existing-repository fixture that runs generated doctor/explain without Docker, a browser, secrets, resources, or producer execution and records the two-minute bound.
  • Verify: deterministic test seam measures the bound without wall-clock sleeps; run remains the only resource/test-executing command.

DX completion summary

Primary persona:      Consumer platform engineer / test-infra maintainer
First success:        init -> doctor -> explain in under two minutes
Champion moment:      A precise, zero-resource explanation of why evidence is—or is not—required
Execution boundary:   run only; no hidden provisioning in init, doctor, or explain
Human front door:     terminal + GitHub summary; manifest remains authoritative
Privacy decision:     no outbound/adoption telemetry or bespoke feedback workflow
Documentation:        start page + API reference + five scenario cookbook
DX score:             8.8 / 10; no dimension below 7
Outside voice:        combo/sub DX reviewer; 6 gaps resolved through explicit decisions

Cross-Phase Themes

• Truthful claim over convenient green — CEO, engineering, and DX all converged on the same requirement: a passed producer, a percentage, or a selected subset must not be allowed to imply a broader evidence claim. Obligations, claim dimensions, the explicit run boundary, and human summaries all enforce that single principle.
• Separation of control planes — Strategy rejected an AppSurface CI platform, engineering rejected normal AppHost discovery as trusted composition, and DX rejected hidden execution during setup. The resulting shape is deliberately separated: consumer-owned EvidenceHost, base-owned registrations/envelope verification, and consumer-owned CI/environment.
• Quietness must be mechanically earned — The CEO review added no-risk results and selection-quality budgets; engineering made policy deterministic and bounded; DX made doctor/explain expose the why before work begins. The system is allowed to be quiet only with a versioned policy reason, never because a required suite was skipped.
• One evidence source, two audiences — Engineering keeps the manifest authoritative and hash-verifiable; DX adds terminal/GitHub/JSON summaries that explain the same claim without inventing a second gate or dashboard.