tests: PLAN.md for v0.2.0 (envelope, subscribers, lifecycle, perf)
Adds tests/PLAN.md — the test plan that implements HOOK_CONTRACT.md section 14. Defines what we test (sections A-G: envelope shape, subscriber API, error containment, lifecycle, anti-corruption, performance budget, system adapter loading), what we don't test (consumer domain logic, Foundry's correctness, system-specific derived events, older Foundry versions), and the definition of done (100% coverage of 'what we test', smoke <2s, Foundry <30s, perf budget <0.1ms median per fire). The v0.1.0 verify-hooks-lib.mjs (20 assertions) covers the wrong shape — it's slated for archive when v0.2.0 test files ship. Listed in 'future turns' as a git mv into tests/archive/v0.1.0/. Push: Gitea only.
This commit is contained in:
260
tests/PLAN.md
Normal file
260
tests/PLAN.md
Normal file
@@ -0,0 +1,260 @@
|
||||
# hooks-lib test plan — v0.2.0
|
||||
|
||||
**Status:** Proposed. Implements the contract in `docs/HOOK_CONTRACT.md`.
|
||||
**Drives:** `tests/verify-hooks-lib.mjs` (no-Foundry smoke) and
|
||||
`tests/verify-hooks-lib-foundry.mjs` (Playwright against a live
|
||||
Foundry instance).
|
||||
|
||||
The v0.1.0 test file (`tests/verify-hooks-lib.mjs`, 20 assertions) is
|
||||
**out of scope** for v0.2.0 — its assertions cover the curated-event
|
||||
catalog shape that v0.2.0 replaces. v0.2.0 ships a fresh test file;
|
||||
v0.1.0's stays in git history for reference but is gitignored from
|
||||
the test runner.
|
||||
|
||||
---
|
||||
|
||||
## What we test (must pass for "done")
|
||||
|
||||
### Section A — Envelope shape
|
||||
|
||||
Every registered hook in HOOK_CONTRACT.md §6 produces an envelope
|
||||
with **exactly** the fields from §2:
|
||||
|
||||
```js
|
||||
{ ts: number, hook: string, args: array }
|
||||
```
|
||||
|
||||
No extras, no missing. The test enumerates the registered hook set
|
||||
and fires each one with a stubbed Foundry, asserting the envelope
|
||||
shape per hook.
|
||||
|
||||
**Assertions per registered hook:**
|
||||
- `envelope.ts` is a number, `>= 0`, taken within the last second
|
||||
- `envelope.hook` equals the registered hook name (or the
|
||||
normalized name in §9 for hooks with anti-corruption mappings)
|
||||
- `envelope.args` is an array with the arity documented in
|
||||
HOOK_ARG_SHAPES.md (or arity 0 for hook families with no args)
|
||||
|
||||
**Why this matters:** if a hook ever emits an extra field, downstream
|
||||
consumers (system adapters, audit logs, replay tools) might depend on
|
||||
it. Locking the shape to §2 keeps the contract honest.
|
||||
|
||||
### Section B — Subscriber API
|
||||
|
||||
The four subscribe primitives from §3 work as documented.
|
||||
|
||||
**`subscribe(hookName, fn)`:**
|
||||
- Returns an unsubscribe function.
|
||||
- Unsubscribed fn no longer fires.
|
||||
- Multiple subscribes for the same hook stack in registration order.
|
||||
- Subscribing with an unregistered hook name throws `TypeError`
|
||||
synchronously (typo protection).
|
||||
|
||||
**`subscribeMany({...})`:**
|
||||
- Atomic: if any name is unregistered, the whole batch throws and
|
||||
no handlers are registered.
|
||||
- Successful batch registers all handlers; the returned unsubscribe
|
||||
removes them all.
|
||||
|
||||
**`subscribeAll(fn)`:**
|
||||
- One handler receives every fire from every registered hook.
|
||||
- Unsubscribed fn no longer fires.
|
||||
|
||||
**`unsubscribeAll()`:**
|
||||
- Removes every callback registered through any primitive.
|
||||
|
||||
### Section C — Error containment
|
||||
|
||||
A throwing consumer callback does not break the dispatch chain.
|
||||
|
||||
**Assertions:**
|
||||
- Register two callbacks; first throws, second runs.
|
||||
- The error is logged via `console.error` with the prefix
|
||||
`[hax-hooks-lib]` and the hook name.
|
||||
- The library does NOT re-throw the error to Foundry's hook chain.
|
||||
- 100 fires with a single throwing callback complete with the same
|
||||
total fires-to-other-callbacks count as 100 fires without.
|
||||
|
||||
### Section D — Lifecycle
|
||||
|
||||
**`init` registration:**
|
||||
- After the library's `init` runs, `listRegisteredHooks()` returns
|
||||
every hook in §6.
|
||||
- No `game.system` reads during init (the test verifies the library
|
||||
doesn't crash when `game.system` is undefined during init).
|
||||
|
||||
**`ready` adapter evaluation:**
|
||||
- With `game.system.id === "dnd5e"` and `game.system.version === "5.2.5"`,
|
||||
a registered dnd5e adapter whose `system.versions` covers 5.2.5
|
||||
is loaded.
|
||||
- With `game.system.id === "dnd5e"` and version outside the
|
||||
adapter's range, the adapter is NOT loaded; a warning is logged
|
||||
naming the adapter and the version mismatch.
|
||||
- With `game.system.id === "pf2e"` and the adapter declaring
|
||||
`system.id === "dnd5e"`, the adapter is NOT loaded; no warning
|
||||
(silent skip for non-matching system).
|
||||
- Throwing adapter factory is contained: the failure is logged, the
|
||||
library continues loading other adapters.
|
||||
|
||||
**`unregisterModule` cleanup:**
|
||||
- After `unregisterModule("hax-hooks-lib")`, `listRegisteredHooks()`
|
||||
returns `[]`.
|
||||
- All consumer callbacks are removed (verified by re-registering
|
||||
init + asserting the registered hook set is empty).
|
||||
- Foundry's `Hooks.off` is called for every library-registered
|
||||
listener (verified by asserting that subsequent Foundry fires do
|
||||
NOT produce envelopes).
|
||||
|
||||
**World change re-evaluation:**
|
||||
- Firing `ready` twice on the same world produces the same active
|
||||
adapter set (idempotent).
|
||||
- Firing `ready` again with a different `game.system.id` re-runs
|
||||
the adapter match; non-matching adapters unload, matching ones
|
||||
load.
|
||||
|
||||
### Section E — Anti-corruption
|
||||
|
||||
**Hook rename mapping (§9):**
|
||||
- A Foundry v13 fire of `renderChatLog` produces an envelope with
|
||||
`hook === "renderChatInput"`.
|
||||
- A Foundry v14 fire of `renderChatInput` produces the same.
|
||||
- A Foundry v13 `combatEnd` (when fired) AND a Foundry v14
|
||||
`updateCombat` with `active: false` both produce envelopes with
|
||||
`hook === "combatEnd"` (normalized).
|
||||
|
||||
**Arg normalization (§10):**
|
||||
- For every hook in §6 with documented arg-shape normalization,
|
||||
firing with raw Foundry args produces an `args` array matching the
|
||||
documented shape, regardless of Foundry version.
|
||||
- The exact normalization shapes live in `docs/HOOK_ARG_SHAPES.md`.
|
||||
v0.2.0 ships shapes for: `combatStart`, `combatEnd`, `combatTurn`,
|
||||
`combatRound`, `preUpdateActor`, `updateActor`, `preUpdateToken`,
|
||||
`updateToken`, `dnd5e.rollAttackV2`, `dnd5e.rollDamageV2`. The
|
||||
remaining hooks either have stable arity or no normalization.
|
||||
|
||||
### Section F — Performance budget
|
||||
|
||||
Measured by `tests/perf.mjs`:
|
||||
|
||||
- **Per-fire overhead:** median <0.1ms per fire over 10k fires.
|
||||
Median because some fires do more work (e.g. consumer with many
|
||||
subscribers) — median captures the typical case.
|
||||
- **Memory:** zero per-fire allocation beyond the envelope (verified
|
||||
by heap snapshot before/after a 10k-fire loop; heap delta < 1MB).
|
||||
- **Async dispatch:** for async-dispatched hooks, the wrapper returns
|
||||
before consumer callbacks run (verified by registering a consumer
|
||||
that records its invocation time and asserting the wrapper's
|
||||
return time precedes it).
|
||||
|
||||
### Section G — System adapter loading (deeper)
|
||||
|
||||
Beyond Section D's basic cases:
|
||||
|
||||
- An adapter registered with no `factory` is invalid; `registerSystemAdapter`
|
||||
throws synchronously.
|
||||
- An adapter whose `factory` returns a non-array is invalid; the
|
||||
library logs and marks the adapter failed.
|
||||
- A derived-event registered by a failed adapter does NOT fire (no
|
||||
partial registration on failure).
|
||||
- Multiple adapters with the same `id` are deduplicated; the second
|
||||
registration is a no-op with a warning.
|
||||
- The library does not call `factory` for an adapter whose system
|
||||
doesn't match, even if called multiple times.
|
||||
|
||||
---
|
||||
|
||||
## What we don't test (explicitly out of scope)
|
||||
|
||||
- **Consumer-side domain logic.** "When bob takes damage" is the
|
||||
consumer's test plan, not hooks-lib's. The library's tests assert
|
||||
that the envelope shape and subscriber API work; they do not
|
||||
assert that any specific consumer's interpretation is correct.
|
||||
- **Foundry's hook correctness.** We trust Foundry fires the right
|
||||
args. If Foundry has a bug, we patch around it in the library, but
|
||||
we don't test Foundry.
|
||||
- **System-specific derived events.** That's the adapter's test
|
||||
plan. hooks-lib's tests verify the adapter *protocol* (registration,
|
||||
matching, lifecycle) but not the adapter's *content*.
|
||||
- **Foundry-version compatibility beyond the latest two releases.**
|
||||
v0.2.0 tests Foundry v13.351 and v14.364. Older versions are
|
||||
"best effort" — the anti-corruption layer should handle them, but
|
||||
no CI gate.
|
||||
- **Real Foundry runtime for the smoke test.** The no-Foundry smoke
|
||||
test stubs `Hooks`, `game`, `ui`. It runs in <2s. Real-Foundry
|
||||
verification is the Playwright test (§ Playwright run below).
|
||||
|
||||
---
|
||||
|
||||
## Definition of done
|
||||
|
||||
A v0.2.0 release is "done" when:
|
||||
|
||||
1. **100% of the "What we test" bullets have an assertion.** No
|
||||
silent gaps. If we can't test something (e.g. a Foundry-version
|
||||
matrix we don't have installed), we either skip that bullet with
|
||||
an explicit comment in the test file OR we leave the bullet out
|
||||
of this plan.
|
||||
2. **`npm test` exits 0** in the no-Foundry smoke runner. Runs in
|
||||
<2s.
|
||||
3. **`npm run test:foundry` exits 0** against a live Foundry v14
|
||||
instance with a representative world (3 actors, Bard-vs-Goblin
|
||||
fixture reused from battle-focus's test helper). Runs in <30s.
|
||||
4. **`npm run lint` exits 0** (if we adopt a linter; otherwise skip).
|
||||
5. **Both pass on the Hermes shell** (Windows, git-bash, Node 18+).
|
||||
CI portability is a stretch goal, not a gate.
|
||||
6. **Performance budget holds** (§ F). Median per-fire overhead
|
||||
measured in CI; if it regresses >0.1ms, the release is blocked
|
||||
until the regression is explained.
|
||||
|
||||
---
|
||||
|
||||
## Test files (v0.2.0)
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `tests/verify-hooks-lib.mjs` | No-Foundry smoke. Sections A–F (F's perf-measurement subset that doesn't require Foundry). Runs in <2s. Replaces v0.1.0's `verify-hooks-lib.mjs`. |
|
||||
| `tests/verify-hooks-lib-foundry.mjs` | Playwright against a live Foundry. Loads the library as a Foundry module, asserts `mod.api.isReady() === true`, fires one synthetic `updateActor` and one `dnd5e.rollAttackV2` (stubbed at the Foundry level), asserts the library captures both. Runs in <30s. |
|
||||
| `tests/perf.mjs` | The performance budget measurement. Runs in <5s. Called from the smoke runner or standalone. |
|
||||
| `tests/test-helpers.mjs` | Shared stubs and Foundry helpers. Updates the v0.1.0 helper to expose the new Foundry hook stubbing. |
|
||||
|
||||
The smoke test is the gate for PRs. The Foundry test is the gate
|
||||
for releases. Performance test runs weekly or on demand.
|
||||
|
||||
---
|
||||
|
||||
## Future turns (when this repo is no longer the focus)
|
||||
|
||||
- The v0.1.0 `tests/verify-hooks-lib.mjs` and `tests/test-helpers.mjs`
|
||||
will be archived (git mv into `tests/archive/v0.1.0/`) once v0.2.0's
|
||||
test files are stable. They stay in git history but not in the
|
||||
test runner.
|
||||
- When battle-focus migrates to hooks-lib v0.2.0, the
|
||||
battle-focus test plan (separate `tests/PLAN.md` in that repo)
|
||||
will reference hooks-lib's plan for the contract assertions and
|
||||
add battle-focus-specific consumer assertions.
|
||||
- When its-achievable ships, its `tests/PLAN.md` will reference
|
||||
hooks-lib's adapter protocol and add adapter-specific assertions.
|
||||
|
||||
---
|
||||
|
||||
## Open questions for this plan
|
||||
|
||||
These are settled when v0.2.0 implementation starts, not before:
|
||||
|
||||
1. **Stub fidelity for the no-Foundry smoke test.** How close does
|
||||
the stub need to match Foundry's actual `Hooks.on` semantics
|
||||
(e.g. `callAll` return value, error propagation)? Pragmatic
|
||||
answer: match the contract §3.5's error containment, match the
|
||||
arity pass-through, and don't sweat the rest.
|
||||
2. **Performance measurement methodology.** §F says "median <0.1ms
|
||||
over 10k fires." In CI, that's `node --experimental-vm-modules`
|
||||
with a synthetic hook loop. On real hardware. No reproducible
|
||||
benchmark — the budget is a soft gate.
|
||||
3. **How to test `unregisterModule`.** Foundry's unregister hook
|
||||
fires when a module is disabled via the UI. In the smoke test,
|
||||
we call the library's unregister function directly. In the
|
||||
Foundry test, we trigger it via `game.modules.get(id).active = false`.
|
||||
Both paths are tested.
|
||||
|
||||
Push back on any of these or on the plan as a whole before the
|
||||
v0.2.0 implementation starts.
|
||||
Reference in New Issue
Block a user