Pilot report
Code that works is not the whole result
Four functional passes. A missing decision record. Why the first author-run pilot does not demonstrate a PEA advantage.
| Attempt | Feature checks | Original tests | Audit trail |
|---|---|---|---|
| Baseline 1 | 17 / 17 | 6 / 6 | No |
| PEA-labeled 1 | 17 / 17 | 6 / 6 | No |
| PEA-labeled 2 | 17 / 17 | 6 / 6 | No |
| Baseline 2 | 17 / 17 | 6 / 6 | No |
The question and the controlled task
On 11 November 2026, I ran four fresh agent sessions against a small, public cart-calculation fixture: two baseline sessions and two PEA-labeled sessions. The task was to add a percentage discount while preserving shipping eligibility, integer-cent arithmetic, validation and input immutability. The protocol was recorded before model execution, with the same task and external acceptance checks for both conditions. All four runtime captures identified claude-sonnet-5. This is author-run exploratory evidence on one synthetic task, not independent replication or a representative productivity benchmark.
The code passed; the intended workflow did not
All four attempts passed all 17 external feature checks. I also reran the six original regression tests from observer-owned copies rather than trusting agent-edited test files. Neither PEA-labeled attempt produced the required audit trail. Although the runtime listed the five operational skills as available, the captured tool sequences did not show an operational skill-content read or an Improve invocation. Functional completion was demonstrated for this fixture; application of the intended PEA workflow was not. Discoverable skills and applied skills are different acceptance criteria.
Why the comparison is inconclusive
The missing trace is a real end-to-end failure, but it does not isolate a cause. Working-directory ambiguity appeared in one attempt, and host routing, activation and instruction compliance remain confounded. Earlier host-only failures also exposed inherited skills, a Windows launcher problem and an unsupported resource cap; those invoked no model and remain separately recorded. No completed task result was silently replaced. Harness corrections now have local regression tests, but no further live-model sessions have been run. These results cannot support a claim that PEA improves safety, speed or reasoning fidelity.
Inspect or reproduce the evidence
The open pilot includes the frozen task, source identities, all named acceptance cases, final fixture code and all four outcomes. Raw private host captures are retained separately; the public receipt is not a disclosure of account details or a claim of access to true internal reasoning. The two-minute video is a silent, annotated explanation of the recorded evidence, not live agent footage or a successful PEA demonstration. A future comparison needs separately recorded conditions and verified activation. For now, the honest result is functional parity, an unmet trace requirement and an inconclusive intended comparison.