github-copilot/.github/skills/test-runner/SKILL.md

name, description, argument-hint

name	description	argument-hint
test-runner	Run pytest tests, diagnose failures, escalate verbosity when output is insufficient, and fix broken tests. Use when: run tests, tests are failing, fix failing tests, pytest errors, запустить тесты, тесты падают, исправить тесты, почему падает тест, debug test failure, test errors, починить тесты, pytest fail, assertion error in test.	Test file or folder to run (e.g. tests/test_auth.py or tests/). Leave empty to run all.

Test Runner & Fixer

Runs pytest, diagnoses failures with escalating verbosity, and fixes broken tests automatically.

When to Use

• Tests were just written and need to be verified
• User reports that tests are failing
• CI is broken and the cause is unknown
• Need to understand why a specific test fails

Procedure

Step 1 — Read pytest Config

Check pyproject.toml (or pytest.ini, setup.cfg) for [tool.pytest.ini_options].

Extract and respect:

• testpaths — default path(s) to run
• addopts — default flags (e.g. -x, --tb=short, -q)
• asyncio_mode — auto or strict
• filterwarnings — do not suppress these
• Custom markers and their meanings
• env or env_files (pytest-dotenv, pytest-env)

Store the effective base command — all runs in this session build on it.

Step 2 — First Run (Standard)

Run pytest with the base config flags plus minimal output:

pytest {target} {base_addopts} --tb=short -q

Parse the output:

• ✅ All passed → report summary and stop
• ⚠️ Warnings only → report warnings, ask if user wants to address them
• ❌ Failures → proceed to Step 3

Step 3 — Triage Failures

For each failing test, classify the error:

Error type	Signal
ImportError / ModuleNotFoundError	Missing dependency or wrong path
fixture not found	Fixture name mismatch or missing conftest import
AssertionError	Wrong expected value or broken logic
TypeError / AttributeError	Signature mismatch, wrong mock, API change
asyncio / RuntimeError: coroutine	Async test misconfiguration
ERRORS (collection error)	Syntax error or import-time crash
Timeout	Missing await, infinite loop, or slow test

If ≤ 3 failures — read the full test source for each failed test immediately. If > 3 failures — read the source of the first 3 and look for a common root cause first.

Step 4 — Escalate Verbosity (if needed)

If the failure message from Step 2 is insufficient to diagnose, re-run with more detail. Follow the escalation ladder in references/diagnose-failures.md.

Escalation stops as soon as the root cause is identified — do not run all levels blindly.

Step 5 — Propose Fixes

For each failure, show the diagnosis and proposed fix before editing:

### ❌ test_login_wrong_password

**Cause:** `UserService.__init__` signature changed — now requires `settings` argument.
**Fix:** Update fixture `user_service` in conftest.py to pass `settings`.

Before:
    user_service = UserService(db_session)
After:
    user_service = UserService(db_session, settings=app_settings)

If multiple tests share the same root cause — group them under one fix proposal.

Ask: "Apply fixes?" — unless there is only one trivial fix, in which case apply immediately.

Step 6 — Apply Fixes

Apply only accepted fixes following the rules in the python-review skill's references/fix-rules.md (minimal diff, no behaviour changes, preserve comments).

After applying:

1. Re-run the previously failing tests in isolation:

   pytest {fixed_test_ids} --tb=short -q
   

2. If still failing → repeat from Step 3 (max 3 iterations before asking user)
3. If passing → run the full suite to check for regressions:

   pytest {target} --tb=short -q
   

Step 7 — Report

Output a final summary:

## Test Run Summary

| Status | Count |
|--------|-------|
| ✅ Passed | N |
| ❌ Failed | N |
| ⚠️ Warnings | N |
| 🔧 Fixed | N |

{List of fixed tests}
{List of remaining failures if any, with reason}

If any failures remain after 3 fix iterations — explain what is needed from the user (e.g. missing env vars, external service not running, DB migration needed).