fix

skills/test-runner/references/diagnose-failures.md

@@ -0,0 +1,131 @@
+# Diagnose Test Failures — Escalation Ladder
+
+When the default `--tb=short -q` output is not enough to identify a root cause,
+escalate through these levels in order. Stop as soon as the cause is clear.
+
+## Level 1 — Full Traceback
+
+Use when: short traceback cuts off the relevant frame.
+
+```bash
+pytest {failing_tests} --tb=long -v
+```
+
+What to look for:
+- The exact line that raises, with full local variable context
+- Which fixture or conftest call is involved
+- Whether the error originates in test code or production code
+
+---
+
+## Level 2 — Show Local Variables
+
+Use when: you see the line but not the values involved.
+
+```bash
+pytest {failing_tests} --tb=long -v --showlocals
+```
+
+What to look for:
+- Values of `self`, `result`, `response`, `exc` at the point of failure
+- Unexpected `None` where an object was expected
+- Wrong type (e.g. `str` instead of `int`)
+
+---
+
+## Level 3 — Verbose + Full Diff
+
+Use when: `AssertionError` with truncated diff output.
+
+```bash
+pytest {failing_tests} -vv --tb=long --showlocals
+```
+
+`-vv` forces pytest to print the full comparison diff without truncation.
+
+What to look for:
+- Exact difference between actual and expected dicts/lists/objects
+- Whitespace or encoding differences in string comparisons
+- Extra or missing keys in dicts
+
+---
+
+## Level 4 — Capture Disabled (see print/log output)
+
+Use when: test uses `print()` or `logging` for debug output, but it's suppressed.
+
+```bash
+pytest {failing_tests} -s --tb=long -v
+```
+
+`-s` disables stdout/stderr capture — all print and logging output becomes visible.
+
+What to look for:
+- Debug prints left in test or production code
+- Log messages from libraries (SQLAlchemy queries, httpx requests)
+- Side effects happening before the assertion
+
+---
+
+## Level 5 — Log Level Override
+
+Use when: need structured log output, not just prints.
+
+```bash
+pytest {failing_tests} --log-cli-level=DEBUG --tb=long -v
+```
+
+What to look for:
+- DB query output (sqlalchemy echo)
+- HTTP request/response details
+- Background task or worker lifecycle events
+
+---
+
+## Level 6 — Single Test Isolation
+
+Use when: a test passes alone but fails in suite (ordering issue or shared state).
+
+```bash
+# Run the failing test in complete isolation
+pytest {single_failing_test_id} --tb=long -v --forked
+# or without pytest-forked:
+pytest {single_failing_test_id} --tb=long -v -p no:randomly
+```
+
+Also try running with `--randomly-seed=0` to fix the order and reproduce reliably.
+
+What to look for:
+- Global state mutated by a previous test
+- Singleton or module-level cache not reset between tests
+- `autouse` fixture with session scope polluting state
+
+---
+
+## Level 7 — Collection Errors
+
+Use when: pytest crashes before running any test (`ERROR collecting ...`).
+
+```bash
+pytest {target} --collect-only --tb=long
+```
+
+What to look for:
+- `SyntaxError` in test file
+- Import-time crash (`ModuleNotFoundError`, circular import)
+- Missing fixture at discovery time
+- Plugin conflict (`conftest.py` error)
+
+---
+
+## Common Root Causes Quick Reference
+
+| Symptom | Likely Cause | Fix Hint |
+|---------|-------------|----------|
+| `fixture 'X' not found` | Fixture not in scope or conftest not loaded | Check conftest path; add `conftest.py` to package |
+| `coroutine was never awaited` | `async def` called without `await` in sync context | Add `@pytest.mark.asyncio` or set `asyncio_mode=auto` |
+| `TypeError: __init__() missing argument` | Production code signature changed | Update fixture to pass new required arg |
+| `AssertionError` with None | Mock not set up / fixture returns None | Check mock `return_value`; check fixture creates object |
+| `ImportError` on test file | Wrong `PYTHONPATH` or missing `__init__.py` | Check `pythonpath` in `[tool.pytest.ini_options]` |
+| `RuntimeError: Event loop closed` | Async fixture scope mismatch | Match fixture scope to `asyncio_mode`; use `loop_scope` |
+| All tests suddenly fail | Broken conftest or missing env var | Run `--collect-only` first; check `.env.test` |