I have enough context. The task is brainstorming how to test the `pi-subagents` chain/prompt-workflow code. The package has no test directory but `package.json` declares `node --test` scripts. Let me deliver the brainstorm.

# Brainstorm: testing the prompt-workflow chain

## 1. Problem restatement

We need a test strategy for the `pi-subagents` chain / prompt-workflow layer — the code that parses `c-*.chain.md` recipes, validates the `chain` argument shape, resolves `{outputs.x}` references, builds the workflow graph, runs each step as a forked subagent, and serializes results back to disk. "Done" means: chain-related logic has fast, deterministic unit coverage; malformed inputs are caught with friendly errors; recipe parsing round-trips; the whole layer regresses safely when we touch it. Constraints that matter: the package is TypeScript ESM with a Jiti loader, no test framework is installed yet, `package.json` already declares `node --test` scripts (so Node's built-in runner is the intended harness), and the bulk of the chain logic is in ~3,800 LOC across `chain-validation.ts`, `chain-outputs.ts`, `chain-serializer.ts`, `workflow-graph.ts`, plus the giant `chain-execution.ts` / `chain-clarify.ts` orchestrators that need mocked providers.

## 2. Alternatives

### Alternative A — Pure unit tests of the pure modules with `node --test`

- **Summary.** Add `test/unit/*.test.ts` files that directly import and exercise `validateChainInput`, `validateChainOutputBindings`, `resolveOutputReferences`, `parseChainFile`, and `buildWorkflowGraphSnapshot` with hand-built inputs. Use Node's built-in `node:test` + `node:assert/strict`; no new deps.
- **How it works.** `package.json` already runs `node --experimental-strip-types --test test/unit/*.test.ts`. We create `test/unit/chain-validation.test.ts`, `test/unit/chain-outputs.test.ts`, `test/unit/chain-serializer.test.ts`, `test/unit/workflow-graph.test.ts`. Each file builds a fixture object, calls the function, asserts on the return value or thrown error. `node --test` discovers them via glob, runs in parallel, prints TAP.
- **Pros.** Zero new dependencies (Node ships the runner); matches the script that's already in `package.json`; very fast (milliseconds per file); pure functions are the highest-leverage targets — these are the modules where regressions silently break every chain run; the Jiti/strip-types dance is already configured; diff-friendly because the tests are pure data in / assertion out.
- **Cons.** Only covers the leaf modules. The orchestrators (`chain-execution.ts`, `chain-clarify.ts`) that talk to providers, sessions, and the TUI are not exercised; we still need mocking for those, and a pure-unit approach can't catch bugs in the wiring between `chain-validation` and `chain-execution`. `node --test`'s output is plain TAP, not as nice as Vitest's reporter.
- **Best when.** The priority is "lock down the easy-to-break pure logic first, ship today, defer the slow stuff." The codebase clearly intends `node --test` (per `package.json` scripts) and there's no existing test infra to contradict.

### Alternative B — Add Vitest on top of the existing `node --test` setup

- **Summary.** Install `vitest` as a dev dep, replace the `test:unit` script to use it, and gain watch mode, snapshot testing, ESM-friendly mocking, and a better reporter — at the cost of one dependency.
- **How it works.** `npm i -D vitest`. Add `vitest.config.ts` that points at `test/unit/`. Author tests with `describe / it / expect`, use `vi.mock` to stub modules we don't want to load (e.g., the TUI). Reuse the same fixture objects as Alternative A; the assertions are a near-mechanical port.
- **Pros.** Vitest's mock API is dramatically less painful than hand-rolling `node:module` hooks or `jest.mock` for the orchestrators; watch mode (`vitest --watch`) makes TDD cheap; snapshot testing fits recipe round-trips naturally; better DX (inline `expect`, colored output, stack traces that point at the source). The migration cost from `node --test` is low because both run TS through Vite.
- **Cons.** Adds a non-trivial dev dependency to a package that currently has *zero* dev deps beyond the `pi-coding-agent` peer; lock-in to Vite's transformer (we already use `jiti` elsewhere, so another transformer is a small tax); we have to choose between keeping the old `node --test` script for CI or removing it; potential version-drift friction with the existing `experimental-strip-types` setup.
- **Best when.** The team expects the test suite to grow well beyond pure functions, mock-heavy integration tests are coming, and the DX of `node --test` is going to be a friction point within a week.

### Alternative C — Property-based tests for output-reference resolution and validation

- **Summary.** Layer in a property-based testing library (`fast-check`) on top of whichever runner we pick, and characterize the *invariants* of `validateChainOutputBindings` and `resolveOutputReferences` rather than enumerating cases.
- **How it works.** For each property: generate random `ChainStep[]` arrays, random valid output names, random `outputs` maps; assert that (a) the validator never throws on syntactically valid names; (b) the resolver is a total function on its declared domain; (c) round-tripping a name through `resolveOutputReferences` is stable; (d) duplicate `as:` names always throw; (e) referencing a future output always throws. `fast-check` shrinks failing cases to minimal repros.
- **Pros.** Property tests famously find corner cases the author didn't think to enumerate — exactly the bug class we have in `chain-outputs.ts` (e.g., what happens if `{previous}` and `{outputs.x}` appear in the same template? what about unicode in names? what if `as:` is empty?); once written, they double as living documentation of the contract; shrinks to minimal repros automatically.
- **Cons.** Adds a heavier dev dep than Vitest alone; the property invariants need careful design or they become tautologies; the failure messages can be confusing the first time; doesn't replace example-based tests, only supplements them; setup overhead is real for a brand-new test suite.
- **Best when.** The chain-output reference machinery is a high-risk surface and we want maximum confidence per line of test code. Best as a *layer on top of* A or B, not a standalone pick.

### Alternative D — Integration tests that exercise the slash command end-to-end via a fixture recipe

- **Summary.** Author a `test/integration/chain-execution.test.ts` that writes a temporary `c-smoke.chain.md` recipe in a temp dir, invokes the chain runner with stubbed provider responses, and asserts the run state transitions correctly.
- **How it works.** Use the existing `test/integration/*.test.ts` script (declared in `package.json` but pointing at a directory that doesn't exist). The test creates a temp directory, writes a 2-step `c-smoke.chain.md` referencing a stub `p-stub` agent, invokes `executeChain` (or whatever the public entry is) with a mock `Provider` that returns canned completions, then asserts on the resulting `SingleResult[]` and the `workflow-graph` snapshot.
- **Pros.** This is the only approach that catches wiring bugs between modules; a green unit suite + a broken integration test is the most common "tests passed but prod is on fire" failure mode; exercises the `pi-subagents` extension the way real users will (via the `/prompt-workflow` slash).
- **Cons.** Hardest to write and slowest to run; the orchestrator depends on a real `pi-coding-agent` session host, TUI components, and a `Provider`; mocking all of that without leaking real LLM calls is non-trivial; flaky tests in this layer are reputation-damaging; needs Alt A or B as a foundation so we're not debugging everything at once.
- **Best when.** After A is green, before any user-visible release, to lock down the orchestrator. Specifically the right next step once the pure-module layer is solid.

## 3. Comparison table

| Approach | Complexity | Time-to-ship | Reversibility | Performance | Fit with existing code |
|---|---|---|---|---|---|
| **A. `node --test` unit tests** | Low — no deps, fixtures + asserts | Hours | Trivial (delete dir) | Fast (<1s per file) | **Excellent** — already wired in `package.json` |
| **B. Vitest** | Medium — config + 1 dev dep | ~Half day | Easy (rm dep, revert script) | Fast (~1s per file) | Good — slight overlap with existing `experimental-strip-types` |
| **C. Property-based (`fast-check`)** | Medium-high — invariant design | Day | Moderate (still useful if removed) | Slower (~5s per file) | Fair — orthogonal to runner choice |
| **D. Integration via slash command** | High — mocks + fixtures + temp fs | 2–4 days | Hard (orchestrator is the product) | Slow (5–30s) | Excellent — exercises real path |

## 4. Recommendation

**Pick Alternative A first, with B as the fallback if mock-heavy tests are needed within a sprint.** The `package.json` already commits to `node --test`, the high-leverage modules (`chain-validation.ts`, `chain-outputs.ts`, `chain-serializer.ts`, `workflow-graph.ts`) are pure functions with crisp contracts, and we can ship a meaningful safety net in hours without adding a single dependency. A unit suite also gives us a regression fence to lean on while we write the much riskier integration tests in D. If, after A lands, we find ourselves hand-rolling module stubs for the orchestrator or wishing for watch mode, that's the trigger to migrate to B (Vitest) — at that point the migration is mechanical because the fixture shape is the same.

**Fallback: B (Vitest).** If mock-heavy tests are needed *before* A is done, install Vitest now and use it as the runner for A as well; that avoids writing the suite twice.

**Do not start with C or D in isolation.** C is a complement to A/B, not a replacement. D is the right next move *after* A is green.

## 5. Unknowns

1. **What does "test the prompt workflow chain" mean in scope?** Cover the `pi-subagents` chain runner (my assumption), or also the `~/.pi/agent/prompts/c-*.md` recipe *format* (i.e., a `p-test`-style review of those four files for correctness/consistency)? The two need very different tests.
2. **Do we want the test suite to live inside `pi-subagents` (upstream-contributable) or in a sibling `pi-subagents-tests` package?** The right answer changes how we configure the loader.
3. **Is there a CI pipeline today, or is "tests pass" purely local?** If CI exists, what runs it? This determines whether `node --experimental-strip-types` is acceptable (it's a flag today, may behave differently across Node versions).
4. **What's the policy on the `pi-coding-agent` peer dependency for tests?** Several chain modules import types from it. Do we install it as a dev dep, stub the imports, or guard with `import type`?
5. **Is there an LLM-free mode we can lean on, or do integration tests *require* a real provider call?** This decides whether D is feasible at all without a per-run cost.
6. **What's the failure mode we most fear?** If it's "malformed chain arg silently gets a bad error message," A is sufficient. If it's "an orchestrator race leaves a step stuck," only D catches it.

## Residual risks

- The orchestrator modules (`chain-execution.ts`, `chain-clarify.ts`, ~2,700 LOC) are not unit-testable in any clean way; the chosen approach covers leaf modules, not the wiring. A green unit suite is *not* the same as "chains work end-to-end."
- `node --experimental-strip-types` is a moving target across Node versions; CI pinning matters.
- If the user's intent was "test the four `c-*.md` recipe files in `~/.pi/agent/prompts/`" rather than the `pi-subagents` chain runner, the entire above analysis is mis-scoped — clarify before implementation.
- No mocking framework means hand-rolled stubs if we ever do need to fake the provider; budget accordingly.