The subagent extension directory is essentially empty (just a placeholder). The test target is the prompt workflow chain mechanism itself, which is documented in `Doc/chain-commands.md` and the artifact directory under `.pi-subagents/`. I have enough — I do not need to write code, only the plan. Here it is.

---

# Implementation Plan: Test the Prompt Workflow Chain

## 1. Goal

Add a deterministic, repeatable test suite that exercises the **prompt workflow chain** system — the mechanism that resolves `c-*.md` recipes (e.g. `c-design-build`, `c-fix-commit`, `c-refactor-commit`, `c-security-loop`) into ordered `p-*.md` subagent runs and wires their inputs/outputs together — so that recipe parsing, prompt resolution, fork-context plumbing, and step-to-step data flow can be verified without a live LLM.

## 2. Constraints

- **Source surface is small and shell-script / JSON driven.** The chain mechanism itself is not a TypeScript module in this repo — it is the externally-installed `pi-subagents` adapter invoked via `/prompt-workflow <c-*.md> ...`. The only artifacts we own locally are the recipe files (`/home/liph/dotfiles/pi/.pi/agent/prompts/c-*.md`, the 4 recipes present), the prompt templates (`p-*.md`, 12 present), the docs (`Doc/prompts.md`, `Doc/chain-commands.md`), and the run artifacts under `.pi-subagents/artifacts/` and `.pi-subagents/chain-runs/`.
- **No new framework may be introduced.** The project has no test runner, no `package.json` for these prompts, and no `pyproject.toml` here. The only languages available are bash + `jq` + `node` (already on path for the pi agent).
- **No network calls and no live LLM.** The four `c-*.md` recipes trigger real model calls when invoked normally. Tests must stub or skip those, or use fixture artifacts.
- **Prompt bodies are stable contracts.** Tests must not mutate the `c-*.md` or `p-*.md` frontmatter; if frontmatter changes, existing chains break.
- **Acceptance level is `attested`.** Every test step needs concrete evidence (artifact paths, captured output, diff summaries), not prose.
- **Run artifacts already exist** under `.pi-subagents/artifacts/` (`5f390df5_*`, `68d5d3d2_*`, `bf21079c-*`, `f23fc33e_*`) and can be reused as golden fixtures without re-running anything.

## 3. Approach

**Strategy:** treat the chain system as a *contract test* target. We do not test the model — we test:

1. **Recipe syntax** — every `c-*.md` has valid `chain:` frontmatter that resolves to existing `p-*.md` files.
2. **Step compatibility** — every step named in a chain has `subagent: true` and `fork: true` (the two flags the chain runner depends on per `Doc/chain-commands.md`).
3. **Step-to-step wiring** — given a fixture chain run, the `N+1` step's `*_input.md` contains the `N` step's `*_output.md` (proves context handoff works).
4. **Artifact shape** — every chain run produces the expected file quartet (`*_input.md`, `*_meta.json`, `*_output.md`, `*_transcript.jsonl`) with the expected JSON keys in `*_meta.json`.
5. **Argument plumbing** — the user-supplied `$@` argument appears in step 0's `*_input.md` (proves `chain:` expansion of the recipe argument is wired in).

**Rationale:** these five axes cover every failure mode the troubleshooting table in `Doc/chain-commands.md` lists (Unknown prompt workflow, steps seem disconnected, chain does not run in background, `/c-...` pastes text) without depending on the model or the external adapter being invocable from a test harness.

**Test layout:** a new directory `prompts/.tests/` containing:
- `run.sh` — bash entrypoint that runs all checks, exits non-zero on any failure.
- `check_recipe_syntax.sh` — parses each `c-*.md` frontmatter, validates `chain:` is a non-empty `->` arrow list.
- `check_step_resolution.sh` — resolves every step name to a real `p-*.md` file in the same directory.
- `check_step_flags.sh` — asserts each resolved `p-*.md` has `subagent: true` and `fork: true`.
- `check_arg_plumbing.sh` — picks a fixture chain run and asserts step 0's input contains the argument.
- `check_context_handoff.sh` — for each N→N+1 pair in a fixture, asserts that `delegate_N_output.md` content appears in `delegate_(N+1)_input.md`.
- `check_artifact_shape.sh` — for every `*_meta.json` under `.pi-subagents/artifacts/`, asserts the required keys exist (`runId`, `agent`, `task`, `exitCode`, `acceptance`).
- `fixtures/` — copies (or symlinks) of two existing chain runs: one short (`68d5d3d2` or `f23fc33e`, 1 delegate) and one long (`5f390df5`, 4 delegates). Used by checks 5–7.
- `README.md` — how to run, how to add a new check, what each check protects against.
- `justfile` (or `Makefile`) — convenience targets `test`, `test-fast`, `test-fixtures`.

**No live runs.** All checks are static + JSON assertions. A `test-live` target *can* invoke `/prompt-workflow c-security-loop --bg "test fixture"` against a stub prompt and tear it down, but it is gated behind a `[live]` label and skipped by default so the suite stays fast and offline.

## 4. Steps

1. **Inventory existing artifacts.** Run `ls .pi-subagents/artifacts/` and record which chain runs exist and how many delegates each has. Confirm at least one run has ≥3 delegates (so step-to-step handoff can be tested). *Files touched:* none.
2. **Pick two fixtures.** Copy `68d5d3d2` (1-delegate, sanity) and `5f390df5` (4-delegate, full handoff chain) into `prompts/.tests/fixtures/artifacts/`. Do not modify originals. *Files touched:* `prompts/.tests/fixtures/artifacts/{68d5d3d2,5f390df5}/*` (copies).
3. **Create `prompts/.tests/check_recipe_syntax.sh`.** For each `../c-*.md`, extract the `chain:` value, split on `->`, trim, assert non-empty and that every step is a bare token (no spaces, no slashes). *Files touched:* `prompts/.tests/check_recipe_syntax.sh`.
4. **Create `prompts/.tests/check_step_resolution.sh`.** For every recipe, for every step, assert `../p-<step>.md` exists. On failure, print the missing file. *Files touched:* `prompts/.tests/check_step_resolution.sh`.
5. **Create `prompts/.tests/check_step_flags.sh`.** For every `../p-*.md`, parse YAML frontmatter, assert `subagent: true` and `fork: true`. Use a tiny bash + awk parser; do not introduce a YAML library. *Files touched:* `prompts/.tests/check_step_flags.sh`.
6. **Create `prompts/.tests/check_arg_plumbing.sh`.** Read `fixtures/artifacts/68d5d3d2/68d5d3d2_delegate_0_input.md` and assert it contains the literal argument that was passed in. (The argument is recorded in the meta file or recoverable from the transcript; if not recoverable, document the limitation in the check's stderr message and skip — do not fail — so the suite stays green for future runs that don't preserve it.) *Files touched:* `prompts/.tests/check_arg_plumbing.sh`.
7. **Create `prompts/.tests/check_context_handoff.sh`.** For the 4-delegate fixture, assert for each `N` in `0..2` that `delegate_N_output.md` (or a hash/summary of it) appears inside `delegate_(N+1)_input.md`. Use `grep -F` for substring match, or `jq` over a "previous-output" injection field if the adapter exposes one. *Files touched:* `prompts/.tests/check_context_handoff.sh`.
8. **Create `prompts/.tests/check_artifact_shape.sh`.** For every `*_meta.json` under `fixtures/artifacts/`, assert presence of keys: `runId`, `agent`, `task`, `exitCode`, `acceptance.status`, `acceptance.criteria`, `usage.input`, `usage.output`, `model`. *Files touched:* `prompts/.tests/check_artifact_shape.sh`.
9. **Create `prompts/.tests/run.sh`.** Sequentially run all `check_*.sh` scripts, collect pass/fail counts, print a summary, exit non-zero if any check failed. Use `set -uo pipefail` and a `trap` for cleanup. *Files touched:* `prompts/.tests/run.sh`.
10. **Create `prompts/.tests/justfile` (or `Makefile`).** Targets: `test` (all), `test-syntax` (steps 3–5 only, fast), `test-fixtures` (steps 6–8 only, needs fixtures), `test-live` (gated, runs `/prompt-workflow` against a stub, skipped by default). *Files touched:* `prompts/.tests/justfile`.
11. **Create `prompts/.tests/README.md`.** Document: what is tested, what is not tested (live model behavior, real adapter internals), how to add a new check, how to refresh fixtures (`cp -r .pi-subagents/artifacts/<runId> .tests/fixtures/artifacts/`), and the troubleshooting cross-ref to `Doc/chain-commands.md`. *Files touched:* `prompts/.tests/README.md`.
12. **Run the suite locally and capture output.** `bash prompts/.tests/run.sh` → record stdout/stderr into a `validation_output` note. *Files touched:* none (transient).
13. **(Optional, manual)** Invoke `/prompt-workflow c-security-loop --bg "fixtures"` once interactively to confirm the chain still runs end-to-end with the existing recipe set. This is a *smoke test*, not part of the automated suite. *Files touched:* none.

## 5. Verification

- **Step 1** — `ls .pi-subagents/artifacts/` shows ≥3 distinct run prefixes; at least one has `*_delegate_[2-9]_*` files. Manual eyeball.
- **Step 2** — `diff -r .pi-subagents/artifacts/68d5d3d2 prompts/.tests/fixtures/artifacts/68d5d3d2` returns no output (copies are byte-identical).
- **Step 3** — `bash check_recipe_syntax.sh` prints "4 recipes parsed, 0 errors". Mutate `c-design-build.md` to `chain: p-brainstorm -> p-nope -> p-test -> p-commit`; rerun; expect exit 1 and a clear error.
- **Step 4** — `bash check_step_resolution.sh` prints "all steps resolve". With the bad recipe from step 3, expect exit 1 and "missing: ../p-nope.md".
- **Step 5** — `bash check_step_flags.sh` prints "12 prompts, all compliant". Add `subagent: false` to `p-test.md` frontmatter temporarily; rerun; expect exit 1.
- **Step 6** — `bash check_arg_plumbing.sh` prints "arg plumbing OK for run 68d5d3d2". If the fixture's `*_input.md` doesn't contain the argument literal, the check prints a "SKIP: arg not recoverable" notice and exits 0.
- **Step 7** — `bash check_context_handoff.sh` prints "3 handoffs verified for 5f390df5". Truncate the last 200 bytes of `5f390df5_delegate_1_output.md` (a copy in fixtures, not the original); rerun; expect exit 1 with a specific `delegate 1 -> 2` failure message.
- **Step 8** — `bash check_artifact_shape.sh` prints "N metas validated, 0 missing keys". Delete a key from a fixture `*_meta.json`; rerun; expect exit 1.
- **Step 9** — `bash run.sh` prints summary, exits 0. With any single check failing, exits non-zero and the failing check is the last one printed.
- **Step 10** — `just test-syntax` runs in <1s; `just test` runs in <5s. Verify wall time with `time`.
- **Step 11** — `README.md` renders; cross-link to `Doc/chain-commands.md#troubleshooting` works (relative path resolves).
- **Step 12** — captured output saved as the `validation_output` field of the acceptance report.
- **Step 13** — `/subagents-fleet` shows the smoke chain started, then completes.

## 6. Risks & Rollbacks

- **Risk: a recipe is renamed.** `check_step_resolution.sh` will fail until the fixture docs are updated. Rollback: revert the rename; the recipes are not load-bearing for any external user.
- **Risk: `pi-subagents` adapter changes its artifact shape** (different key names, different filename pattern). `check_artifact_shape.sh` will fail. Rollback: re-record fixtures from a fresh chain run; update the key list. No data loss because original artifacts under `.pi-subagents/artifacts/` are never modified.
- **Risk: prompts gain/lose frontmatter flags** as the chain runner evolves. `check_step_flags.sh` will fail. Rollback: update the check to match the new contract; consult `Doc/chain-commands.md#why-individual-prompts-are-the-subagents` first.
- **Risk: a future contributor adds a `p-*.md` without `subagent: true` because it isn't intended for chains.** Currently all 12 prompts are chain-eligible; if that assumption breaks, the check needs an allowlist. Rollback: add an allowlist of "interactive-only" prompts in `check_step_flags.sh` with a comment.
- **Risk: a fixture `*_input.md` is large (>1MB)** and `grep -F` is slow. Mitigation: limit the search to the last 5KB of the input (where the prior step's output summary is injected), or hash the output and grep for the hash.
- **Rollback strategy for the whole suite:** the new files all live under `prompts/.tests/`, isolated from `c-*.md`, `p-*.md`, and the docs. `rm -rf prompts/.tests/` is a complete, safe rollback with no impact on existing chain execution.

## 7. Open Questions

- **Q1:** Is there a *real* test framework this suite should hook into (e.g. `bats`, `shellspec`), or is a plain bash + `jq` script acceptable given there is no existing test infrastructure? *Default if unanswered:* plain bash + `jq`, no new dependency.
- **Q2:** Should `test-live` (step 13) be wired into CI, or kept as a manual smoke test? *Default if unanswered:* manual only; CI runs `test` (steps 3–8) which is fully offline.
- **Q3:** When a recipe's `chain:` references a prompt that does *not* have `subagent: true` (e.g. `p-doc.md`, `p-explain.md`), is that a bug to flag, or an accepted pattern? The current 4 recipes don't use those, but the question matters for the `check_step_flags.sh` allowlist. *Default if unanswered:* treat as a bug and fail the check; the recipes are explicit about which steps are fork-context agents.
- **Q4:** Where in the docs tree should the `README.md` for the test suite live? Options: (a) `prompts/.tests/README.md` (alongside the code), (b) `prompts/Doc/prompts.md` section addition, (c) top-level `README.md` in `.pi/agent/`. *Default if unanswered:* (a), with a one-line pointer from `Doc/prompts.md`.
- **Q5:** Is it acceptable to copy fixture artifacts (duplicating ~500KB on disk), or should the tests symlink to the originals? Copying protects the suite from accidental edits to live artifacts; symlinking saves space but couples them. *Default if unanswered:* copy, with a `make refresh-fixtures` target.

---