ВяткинАртём
132 lines
3.9 KiB
Markdown
132 lines
3.9 KiB
Markdown
# Diagnose Test Failures — Escalation Ladder
|
|
|
|
When the default `--tb=short -q` output is not enough to identify a root cause,
|
|
escalate through these levels in order. Stop as soon as the cause is clear.
|
|
|
|
## Level 1 — Full Traceback
|
|
|
|
Use when: short traceback cuts off the relevant frame.
|
|
|
|
```bash
|
|
pytest {failing_tests} --tb=long -v
|
|
```
|
|
|
|
What to look for:
|
|
- The exact line that raises, with full local variable context
|
|
- Which fixture or conftest call is involved
|
|
- Whether the error originates in test code or production code
|
|
|
|
---
|
|
|
|
## Level 2 — Show Local Variables
|
|
|
|
Use when: you see the line but not the values involved.
|
|
|
|
```bash
|
|
pytest {failing_tests} --tb=long -v --showlocals
|
|
```
|
|
|
|
What to look for:
|
|
- Values of `self`, `result`, `response`, `exc` at the point of failure
|
|
- Unexpected `None` where an object was expected
|
|
- Wrong type (e.g. `str` instead of `int`)
|
|
|
|
---
|
|
|
|
## Level 3 — Verbose + Full Diff
|
|
|
|
Use when: `AssertionError` with truncated diff output.
|
|
|
|
```bash
|
|
pytest {failing_tests} -vv --tb=long --showlocals
|
|
```
|
|
|
|
`-vv` forces pytest to print the full comparison diff without truncation.
|
|
|
|
What to look for:
|
|
- Exact difference between actual and expected dicts/lists/objects
|
|
- Whitespace or encoding differences in string comparisons
|
|
- Extra or missing keys in dicts
|
|
|
|
---
|
|
|
|
## Level 4 — Capture Disabled (see print/log output)
|
|
|
|
Use when: test uses `print()` or `logging` for debug output, but it's suppressed.
|
|
|
|
```bash
|
|
pytest {failing_tests} -s --tb=long -v
|
|
```
|
|
|
|
`-s` disables stdout/stderr capture — all print and logging output becomes visible.
|
|
|
|
What to look for:
|
|
- Debug prints left in test or production code
|
|
- Log messages from libraries (SQLAlchemy queries, httpx requests)
|
|
- Side effects happening before the assertion
|
|
|
|
---
|
|
|
|
## Level 5 — Log Level Override
|
|
|
|
Use when: need structured log output, not just prints.
|
|
|
|
```bash
|
|
pytest {failing_tests} --log-cli-level=DEBUG --tb=long -v
|
|
```
|
|
|
|
What to look for:
|
|
- DB query output (sqlalchemy echo)
|
|
- HTTP request/response details
|
|
- Background task or worker lifecycle events
|
|
|
|
---
|
|
|
|
## Level 6 — Single Test Isolation
|
|
|
|
Use when: a test passes alone but fails in suite (ordering issue or shared state).
|
|
|
|
```bash
|
|
# Run the failing test in complete isolation
|
|
pytest {single_failing_test_id} --tb=long -v --forked
|
|
# or without pytest-forked:
|
|
pytest {single_failing_test_id} --tb=long -v -p no:randomly
|
|
```
|
|
|
|
Also try running with `--randomly-seed=0` to fix the order and reproduce reliably.
|
|
|
|
What to look for:
|
|
- Global state mutated by a previous test
|
|
- Singleton or module-level cache not reset between tests
|
|
- `autouse` fixture with session scope polluting state
|
|
|
|
---
|
|
|
|
## Level 7 — Collection Errors
|
|
|
|
Use when: pytest crashes before running any test (`ERROR collecting ...`).
|
|
|
|
```bash
|
|
pytest {target} --collect-only --tb=long
|
|
```
|
|
|
|
What to look for:
|
|
- `SyntaxError` in test file
|
|
- Import-time crash (`ModuleNotFoundError`, circular import)
|
|
- Missing fixture at discovery time
|
|
- Plugin conflict (`conftest.py` error)
|
|
|
|
---
|
|
|
|
## Common Root Causes Quick Reference
|
|
|
|
| Symptom | Likely Cause | Fix Hint |
|
|
|---------|-------------|----------|
|
|
| `fixture 'X' not found` | Fixture not in scope or conftest not loaded | Check conftest path; add `conftest.py` to package |
|
|
| `coroutine was never awaited` | `async def` called without `await` in sync context | Add `@pytest.mark.asyncio` or set `asyncio_mode=auto` |
|
|
| `TypeError: __init__() missing argument` | Production code signature changed | Update fixture to pass new required arg |
|
|
| `AssertionError` with None | Mock not set up / fixture returns None | Check mock `return_value`; check fixture creates object |
|
|
| `ImportError` on test file | Wrong `PYTHONPATH` or missing `__init__.py` | Check `pythonpath` in `[tool.pytest.ini_options]` |
|
|
| `RuntimeError: Event loop closed` | Async fixture scope mismatch | Match fixture scope to `asyncio_mode`; use `loop_scope` |
|
|
| All tests suddenly fail | Broken conftest or missing env var | Run `--collect-only` first; check `.env.test` |
|