github-copilot/.github/skills/test-runner/references/diagnose-failures.md

Diagnose Test Failures — Escalation Ladder

When the default --tb=short -q output is not enough to identify a root cause, escalate through these levels in order. Stop as soon as the cause is clear.

Level 1 — Full Traceback

Use when: short traceback cuts off the relevant frame.

pytest {failing_tests} --tb=long -v

What to look for:

• The exact line that raises, with full local variable context
• Which fixture or conftest call is involved
• Whether the error originates in test code or production code

---

Level 2 — Show Local Variables

Use when: you see the line but not the values involved.

pytest {failing_tests} --tb=long -v --showlocals

What to look for:

• Values of self, result, response, exc at the point of failure
• Unexpected None where an object was expected
• Wrong type (e.g. str instead of int)

---

Level 3 — Verbose + Full Diff

Use when: AssertionError with truncated diff output.

pytest {failing_tests} -vv --tb=long --showlocals

-vv forces pytest to print the full comparison diff without truncation.

What to look for:

• Exact difference between actual and expected dicts/lists/objects
• Whitespace or encoding differences in string comparisons
• Extra or missing keys in dicts

---

Level 4 — Capture Disabled (see print/log output)

Use when: test uses print() or logging for debug output, but it's suppressed.

pytest {failing_tests} -s --tb=long -v

-s disables stdout/stderr capture — all print and logging output becomes visible.

What to look for:

• Debug prints left in test or production code
• Log messages from libraries (SQLAlchemy queries, httpx requests)
• Side effects happening before the assertion

---

Level 5 — Log Level Override

Use when: need structured log output, not just prints.

pytest {failing_tests} --log-cli-level=DEBUG --tb=long -v

What to look for:

• DB query output (sqlalchemy echo)
• HTTP request/response details
• Background task or worker lifecycle events

---

Level 6 — Single Test Isolation

Use when: a test passes alone but fails in suite (ordering issue or shared state).

# Run the failing test in complete isolation
pytest {single_failing_test_id} --tb=long -v --forked
# or without pytest-forked:
pytest {single_failing_test_id} --tb=long -v -p no:randomly

Also try running with --randomly-seed=0 to fix the order and reproduce reliably.

What to look for:

• Global state mutated by a previous test
• Singleton or module-level cache not reset between tests
• autouse fixture with session scope polluting state

---

Level 7 — Collection Errors

Use when: pytest crashes before running any test (ERROR collecting ...).

pytest {target} --collect-only --tb=long

What to look for:

• SyntaxError in test file
• Import-time crash (ModuleNotFoundError, circular import)
• Missing fixture at discovery time
• Plugin conflict (conftest.py error)

---

Common Root Causes Quick Reference

Symptom	Likely Cause	Fix Hint
fixture 'X' not found	Fixture not in scope or conftest not loaded	Check conftest path; add conftest.py to package
coroutine was never awaited	async def called without await in sync context	Add @pytest.mark.asyncio or set asyncio_mode=auto
TypeError: __init__() missing argument	Production code signature changed	Update fixture to pass new required arg
AssertionError with None	Mock not set up / fixture returns None	Check mock return_value; check fixture creates object
ImportError on test file	Wrong PYTHONPATH or missing __init__.py	Check pythonpath in [tool.pytest.ini_options]
RuntimeError: Event loop closed	Async fixture scope mismatch	Match fixture scope to asyncio_mode; use loop_scope
All tests suddenly fail	Broken conftest or missing env var	Run --collect-only first; check .env.test