ВяткинАртём
128 lines
4.3 KiB
Markdown
128 lines
4.3 KiB
Markdown
---
|
|
name: test-runner
|
|
description: "Run pytest tests, diagnose failures, escalate verbosity when output is insufficient, and fix broken tests. Use when: run tests, tests are failing, fix failing tests, pytest errors, запустить тесты, тесты падают, исправить тесты, почему падает тест, debug test failure, test errors, починить тесты, pytest fail, assertion error in test."
|
|
argument-hint: "Test file or folder to run (e.g. tests/test_auth.py or tests/). Leave empty to run all."
|
|
---
|
|
|
|
# Test Runner & Fixer
|
|
|
|
Runs pytest, diagnoses failures with escalating verbosity, and fixes broken tests automatically.
|
|
|
|
## When to Use
|
|
|
|
- Tests were just written and need to be verified
|
|
- User reports that tests are failing
|
|
- CI is broken and the cause is unknown
|
|
- Need to understand why a specific test fails
|
|
|
|
## Procedure
|
|
|
|
### Step 1 — Read pytest Config
|
|
|
|
Check `pyproject.toml` (or `pytest.ini`, `setup.cfg`) for `[tool.pytest.ini_options]`.
|
|
|
|
Extract and respect:
|
|
- `testpaths` — default path(s) to run
|
|
- `addopts` — default flags (e.g. `-x`, `--tb=short`, `-q`)
|
|
- `asyncio_mode` — `auto` or `strict`
|
|
- `filterwarnings` — do not suppress these
|
|
- Custom markers and their meanings
|
|
- `env` or `env_files` (pytest-dotenv, pytest-env)
|
|
|
|
Store the effective base command — all runs in this session build on it.
|
|
|
|
### Step 2 — First Run (Standard)
|
|
|
|
Run pytest with the base config flags plus minimal output:
|
|
|
|
```bash
|
|
pytest {target} {base_addopts} --tb=short -q
|
|
```
|
|
|
|
Parse the output:
|
|
- ✅ All passed → report summary and stop
|
|
- ⚠️ Warnings only → report warnings, ask if user wants to address them
|
|
- ❌ Failures → proceed to Step 3
|
|
|
|
### Step 3 — Triage Failures
|
|
|
|
For each failing test, classify the error:
|
|
|
|
| Error type | Signal |
|
|
|---|---|
|
|
| `ImportError` / `ModuleNotFoundError` | Missing dependency or wrong path |
|
|
| `fixture not found` | Fixture name mismatch or missing conftest import |
|
|
| `AssertionError` | Wrong expected value or broken logic |
|
|
| `TypeError` / `AttributeError` | Signature mismatch, wrong mock, API change |
|
|
| `asyncio` / `RuntimeError: coroutine` | Async test misconfiguration |
|
|
| `ERRORS` (collection error) | Syntax error or import-time crash |
|
|
| Timeout | Missing `await`, infinite loop, or slow test |
|
|
|
|
If ≤ 3 failures — read the full test source for each failed test immediately.
|
|
If > 3 failures — read the source of the first 3 and look for a common root cause first.
|
|
|
|
### Step 4 — Escalate Verbosity (if needed)
|
|
|
|
If the failure message from Step 2 is insufficient to diagnose, re-run with more detail.
|
|
Follow the escalation ladder in
|
|
[references/diagnose-failures.md](./references/diagnose-failures.md).
|
|
|
|
Escalation stops as soon as the root cause is identified — do not run all levels blindly.
|
|
|
|
### Step 5 — Propose Fixes
|
|
|
|
For each failure, show the diagnosis and proposed fix **before editing**:
|
|
|
|
```
|
|
### ❌ test_login_wrong_password
|
|
|
|
**Cause:** `UserService.__init__` signature changed — now requires `settings` argument.
|
|
**Fix:** Update fixture `user_service` in conftest.py to pass `settings`.
|
|
|
|
Before:
|
|
user_service = UserService(db_session)
|
|
After:
|
|
user_service = UserService(db_session, settings=app_settings)
|
|
```
|
|
|
|
If multiple tests share the same root cause — group them under one fix proposal.
|
|
|
|
Ask: "Apply fixes?" — unless there is only one trivial fix, in which case apply immediately.
|
|
|
|
### Step 6 — Apply Fixes
|
|
|
|
Apply only accepted fixes following the rules in the `python-review` skill's
|
|
`references/fix-rules.md` (minimal diff, no behaviour changes, preserve comments).
|
|
|
|
After applying:
|
|
1. Re-run the previously failing tests in isolation:
|
|
```bash
|
|
pytest {fixed_test_ids} --tb=short -q
|
|
```
|
|
2. If still failing → repeat from Step 3 (max 3 iterations before asking user)
|
|
3. If passing → run the full suite to check for regressions:
|
|
```bash
|
|
pytest {target} --tb=short -q
|
|
```
|
|
|
|
### Step 7 — Report
|
|
|
|
Output a final summary:
|
|
|
|
```
|
|
## Test Run Summary
|
|
|
|
| Status | Count |
|
|
|--------|-------|
|
|
| ✅ Passed | N |
|
|
| ❌ Failed | N |
|
|
| ⚠️ Warnings | N |
|
|
| 🔧 Fixed | N |
|
|
|
|
{List of fixed tests}
|
|
{List of remaining failures if any, with reason}
|
|
```
|
|
|
|
If any failures remain after 3 fix iterations — explain what is needed from the user
|
|
(e.g. missing env vars, external service not running, DB migration needed).
|