fix

2026-04-06 17:54:07 +03:00
parent 6d4e45d407
commit d00f858d98
15 changed files with 1319 additions and 0 deletions
@@ -0,0 +1,127 @@
+---
+name: test-runner
+description: "Run pytest tests, diagnose failures, escalate verbosity when output is insufficient, and fix broken tests. Use when: run tests, tests are failing, fix failing tests, pytest errors, запустить тесты, тесты падают, исправить тесты, почему падает тест, debug test failure, test errors, починить тесты, pytest fail, assertion error in test."
+argument-hint: "Test file or folder to run (e.g. tests/test_auth.py or tests/). Leave empty to run all."
+---
+
+# Test Runner & Fixer
+
+Runs pytest, diagnoses failures with escalating verbosity, and fixes broken tests automatically.
+
+## When to Use
+
+- Tests were just written and need to be verified
+- User reports that tests are failing
+- CI is broken and the cause is unknown
+- Need to understand why a specific test fails
+
+## Procedure
+
+### Step 1 — Read pytest Config
+
+Check `pyproject.toml` (or `pytest.ini`, `setup.cfg`) for `[tool.pytest.ini_options]`.
+
+Extract and respect:
+- `testpaths` — default path(s) to run
+- `addopts` — default flags (e.g. `-x`, `--tb=short`, `-q`)
+- `asyncio_mode` — `auto` or `strict`
+- `filterwarnings` — do not suppress these
+- Custom markers and their meanings
+- `env` or `env_files` (pytest-dotenv, pytest-env)
+
+Store the effective base command — all runs in this session build on it.
+
+### Step 2 — First Run (Standard)
+
+Run pytest with the base config flags plus minimal output:
+
+```bash
+pytest {target} {base_addopts} --tb=short -q
+```
+
+Parse the output:
+- ✅ All passed → report summary and stop
+- ⚠️ Warnings only → report warnings, ask if user wants to address them
+- ❌ Failures → proceed to Step 3
+
+### Step 3 — Triage Failures
+
+For each failing test, classify the error:
+
+| Error type | Signal |
+|---|---|
+| `ImportError` / `ModuleNotFoundError` | Missing dependency or wrong path |
+| `fixture not found` | Fixture name mismatch or missing conftest import |
+| `AssertionError` | Wrong expected value or broken logic |
+| `TypeError` / `AttributeError` | Signature mismatch, wrong mock, API change |
+| `asyncio` / `RuntimeError: coroutine` | Async test misconfiguration |
+| `ERRORS` (collection error) | Syntax error or import-time crash |
+| Timeout | Missing `await`, infinite loop, or slow test |
+
+If ≤ 3 failures — read the full test source for each failed test immediately.
+If > 3 failures — read the source of the first 3 and look for a common root cause first.
+
+### Step 4 — Escalate Verbosity (if needed)
+
+If the failure message from Step 2 is insufficient to diagnose, re-run with more detail.
+Follow the escalation ladder in
+[references/diagnose-failures.md](./references/diagnose-failures.md).
+
+Escalation stops as soon as the root cause is identified — do not run all levels blindly.
+
+### Step 5 — Propose Fixes
+
+For each failure, show the diagnosis and proposed fix **before editing**:
+
+```
+### ❌ test_login_wrong_password
+
+**Cause:** `UserService.__init__` signature changed — now requires `settings` argument.
+**Fix:** Update fixture `user_service` in conftest.py to pass `settings`.
+
+Before:
+    user_service = UserService(db_session)
+After:
+    user_service = UserService(db_session, settings=app_settings)
+```
+
+If multiple tests share the same root cause — group them under one fix proposal.
+
+Ask: "Apply fixes?" — unless there is only one trivial fix, in which case apply immediately.
+
+### Step 6 — Apply Fixes
+
+Apply only accepted fixes following the rules in the `python-review` skill's
+`references/fix-rules.md` (minimal diff, no behaviour changes, preserve comments).
+
+After applying:
+1. Re-run the previously failing tests in isolation:
+   ```bash
+   pytest {fixed_test_ids} --tb=short -q
+   ```
+2. If still failing → repeat from Step 3 (max 3 iterations before asking user)
+3. If passing → run the full suite to check for regressions:
+   ```bash
+   pytest {target} --tb=short -q
+   ```
+
+### Step 7 — Report
+
+Output a final summary:
+
+```
+## Test Run Summary
+
+| Status | Count |
+|--------|-------|
+| ✅ Passed | N |
+| ❌ Failed | N |
+| ⚠️ Warnings | N |
+| 🔧 Fixed | N |
+
+{List of fixed tests}
+{List of remaining failures if any, with reason}
+```
+
+If any failures remain after 3 fix iterations — explain what is needed from the user
+(e.g. missing env vars, external service not running, DB migration needed).
@@ -0,0 +1,131 @@
+# Diagnose Test Failures — Escalation Ladder
+
+When the default `--tb=short -q` output is not enough to identify a root cause,
+escalate through these levels in order. Stop as soon as the cause is clear.
+
+## Level 1 — Full Traceback
+
+Use when: short traceback cuts off the relevant frame.
+
+```bash
+pytest {failing_tests} --tb=long -v
+```
+
+What to look for:
+- The exact line that raises, with full local variable context
+- Which fixture or conftest call is involved
+- Whether the error originates in test code or production code
+
+---
+
+## Level 2 — Show Local Variables
+
+Use when: you see the line but not the values involved.
+
+```bash
+pytest {failing_tests} --tb=long -v --showlocals
+```
+
+What to look for:
+- Values of `self`, `result`, `response`, `exc` at the point of failure
+- Unexpected `None` where an object was expected
+- Wrong type (e.g. `str` instead of `int`)
+
+---
+
+## Level 3 — Verbose + Full Diff
+
+Use when: `AssertionError` with truncated diff output.
+
+```bash
+pytest {failing_tests} -vv --tb=long --showlocals
+```
+
+`-vv` forces pytest to print the full comparison diff without truncation.
+
+What to look for:
+- Exact difference between actual and expected dicts/lists/objects
+- Whitespace or encoding differences in string comparisons
+- Extra or missing keys in dicts
+
+---
+
+## Level 4 — Capture Disabled (see print/log output)
+
+Use when: test uses `print()` or `logging` for debug output, but it's suppressed.
+
+```bash
+pytest {failing_tests} -s --tb=long -v
+```
+
+`-s` disables stdout/stderr capture — all print and logging output becomes visible.
+
+What to look for:
+- Debug prints left in test or production code
+- Log messages from libraries (SQLAlchemy queries, httpx requests)
+- Side effects happening before the assertion
+
+---
+
+## Level 5 — Log Level Override
+
+Use when: need structured log output, not just prints.
+
+```bash
+pytest {failing_tests} --log-cli-level=DEBUG --tb=long -v
+```
+
+What to look for:
+- DB query output (sqlalchemy echo)
+- HTTP request/response details
+- Background task or worker lifecycle events
+
+---
+
+## Level 6 — Single Test Isolation
+
+Use when: a test passes alone but fails in suite (ordering issue or shared state).
+
+```bash
+# Run the failing test in complete isolation
+pytest {single_failing_test_id} --tb=long -v --forked
+# or without pytest-forked:
+pytest {single_failing_test_id} --tb=long -v -p no:randomly
+```
+
+Also try running with `--randomly-seed=0` to fix the order and reproduce reliably.
+
+What to look for:
+- Global state mutated by a previous test
+- Singleton or module-level cache not reset between tests
+- `autouse` fixture with session scope polluting state
+
+---
+
+## Level 7 — Collection Errors
+
+Use when: pytest crashes before running any test (`ERROR collecting ...`).
+
+```bash
+pytest {target} --collect-only --tb=long
+```
+
+What to look for:
+- `SyntaxError` in test file
+- Import-time crash (`ModuleNotFoundError`, circular import)
+- Missing fixture at discovery time
+- Plugin conflict (`conftest.py` error)
+
+---
+
+## Common Root Causes Quick Reference
+
+| Symptom | Likely Cause | Fix Hint |
+|---------|-------------|----------|
+| `fixture 'X' not found` | Fixture not in scope or conftest not loaded | Check conftest path; add `conftest.py` to package |
+| `coroutine was never awaited` | `async def` called without `await` in sync context | Add `@pytest.mark.asyncio` or set `asyncio_mode=auto` |
+| `TypeError: __init__() missing argument` | Production code signature changed | Update fixture to pass new required arg |
+| `AssertionError` with None | Mock not set up / fixture returns None | Check mock `return_value`; check fixture creates object |
+| `ImportError` on test file | Wrong `PYTHONPATH` or missing `__init__.py` | Check `pythonpath` in `[tool.pytest.ini_options]` |
+| `RuntimeError: Event loop closed` | Async fixture scope mismatch | Match fixture scope to `asyncio_mode`; use `loop_scope` |
+| All tests suddenly fail | Broken conftest or missing env var | Run `--collect-only` first; check `.env.test` |