This commit is contained in:
ВяткинАртём
2026-04-07 09:33:47 +03:00
parent 67cb7bcdeb
commit b6eb535e25
18 changed files with 110 additions and 1 deletions
+101
View File
@@ -0,0 +1,101 @@
---
name: autotest-writer
description: "Write autotests that match the project's existing test style, framework, fixtures, and conventions. Use when: user asks to write tests, generate autotests, cover code with tests, add unit tests, add integration tests, написать тесты, покрыть тестами, написать автотесты, сгенерировать тесты, добавить тесты. Discovers patterns from existing tests first. Uses MCP context7 for library docs when needed. Applies python-review skill rules for type annotations."
argument-hint: "File or function to cover with tests (e.g. src/auth.py or src/auth.py::login)"
---
# Autotest Writer
Discovers how the project writes tests, then generates autotests that look like they were written by the same developer.
## When to Use
- Cover a new function or module with tests
- Generate tests from a TestLink mini-test decomposition (pairs with `testlink-decompose`)
- Add missing edge-case or negative tests
- Scaffold a test file for a new module
## Procedure
### Step 1 — Understand What to Test
From the user's request, identify:
- **Target**: specific file, class, function, or mini-test ID (e.g. `КМД-1831-MT1`)
- **Scope**: unit / integration / end-to-end
- **Source**: existing code to read, or mini-test structure from `testlink-decompose`
If the target is a mini-test ID — treat its steps and expected results as the test specification.
If the target is a source file — read it fully before proceeding.
### Step 2 — Discover Project Test Patterns
Follow the full discovery process in
[references/discover-patterns.md](./references/discover-patterns.md).
Output a one-line summary before continuing:
> Detected: **{framework}** · fixtures via **{conftest/class/factory}** · assertions: **{assert/custom}** · async: **{yes/no}** · mocks: **{unittest.mock / pytest-mock / none}**
### Step 3 — Resolve Library Documentation (if needed)
When the code under test uses a third-party library AND the project has no existing tests covering it:
1. Call `mcp_io_github_ups_resolve-library-id` with the library name to get the context7 ID
2. Call `mcp_io_github_ups_get-library-docs` with that ID to retrieve relevant docs
3. Extract: how to instantiate, patch, or assert against the library's objects
4. If context7 returns nothing — read the library's source from `site-packages/` in the virtualenv
Do NOT fetch docs for libraries already exercised in existing project tests.
### Step 4 — Plan the Tests
Before writing code, output a test plan as a table:
| # | Test name | Scenario type | Target | Fixtures needed |
|---|-----------|--------------|--------|-----------------|
| 1 | `test_login_valid_credentials` | happy path | `auth.login()` | `db_session`, `user_factory` |
| 2 | `test_login_wrong_password` | negative | `auth.login()` | `db_session`, `user_factory` |
| 3 | `test_login_user_not_found` | negative | `auth.login()` | `db_session` |
Coverage target: at minimum cover happy path + all negative/edge cases visible in the source.
Ask: "Продолжить? Или изменить план?" — wait for confirmation before writing code if the plan has more than 5 tests.
For ≤ 5 tests, proceed immediately.
### Step 5 — Write the Tests
Follow all rules in [references/write-rules.md](./references/write-rules.md).
Key constraints (never violate):
- Match the exact file naming convention discovered in Step 2
- Use the same fixture sources (conftest.py / inline / factory) as existing tests
- Use the same assertion style (plain `assert` vs custom matchers)
- Use the same import order and grouping
- Use Python 3.13+ type annotations on all new functions
- Match async/sync style of the surrounding test suite
### Step 6 — Place the File
Determine the output path:
- If a parallel test file exists (e.g. `src/auth.py``tests/test_auth.py`) — append to it
- If no test file exists — create one at the location matching the project layout
- Never create a test file in the source tree unless that is the project's convention
Output the full file path before writing.
### Step 7 — Validate
After writing, run:
```bash
# Syntax and type check
mypy --strict <test_file>
# Lint
ruff check <test_file>
# Execute tests (dry run — collect only)
pytest --collect-only <test_file>
```
Fix any collection errors or mypy/ruff violations silently.
If tests actually run — report pass/fail counts.
@@ -0,0 +1,112 @@
# Discover Project Test Patterns
Run this discovery process once at the start of every autotest-writer session.
Cache results — do not repeat searches for the same project.
## 1. Framework Detection
Search for test dependencies in `pyproject.toml` or `requirements*.txt`:
| Found | Framework |
|-------|-----------|
| `pytest` | pytest (default assumption) |
| `unittest` only | stdlib unittest |
| `anyio`, `pytest-asyncio` | async pytest |
| `hypothesis` | property-based testing present |
If none found — default to **pytest**.
## 2. Test File Layout
Run a glob search for `test_*.py` and `*_test.py` files.
Determine:
- **Root test folder**: `tests/`, `test/`, inline next to source, or mixed
- **Naming convention**: `test_{module}.py` or `{module}_test.py`
- **Mirrors source tree?** e.g. `src/auth/login.py``tests/auth/test_login.py`
## 3. conftest.py Analysis
Read all `conftest.py` files found (root + subdirectories).
Extract:
- All fixture names and their scopes (`function`, `session`, `module`)
- Factory fixtures (return a callable/class)
- Common parametrize patterns
- Any custom pytest plugins or hooks registered
## 4. Read Sample Tests
Read **23 representative test files** (pick files that seem most complete, not trivial ones).
Extract the following patterns:
### Import style
```python
# Note: stdlib first? third-party second? local last?
# Note: are there "from __future__ import annotations"?
```
### Class vs function tests
- Are tests in classes (`class TestLogin:`) or bare functions?
- If classes — do they inherit from anything?
### Assertion style
- Plain `assert a == b`
- `assert a == b, "message"`
- Custom matchers: `assert_that(a).equals(b)`, `expect(a).to.equal(b)`
### Mock / patch style
- `unittest.mock.patch` as decorator
- `unittest.mock.patch` as context manager
- `pytest-mock` via `mocker` fixture
- `MagicMock` / `AsyncMock` / `create_autospec`
### Async style
- `@pytest.mark.asyncio` + `async def test_`
- `anyio` backend marker
- Synchronous wrappers around async code
### Parametrize style
```python
@pytest.mark.parametrize("input,expected", [...])
```
or inline in class docstring (hypothesis)?
## 5. Factory / Builder Patterns
Search for:
- `factory_boy``UserFactory`, `PostFactory` classes
- `pytest-factoryboy``register()` usage
- Custom builder fixtures that return objects
- `faker` usage for test data generation
## 6. Database / External Service Patterns
- Does the project use `pytest-django`, `pytest-sqlalchemy`, or custom DB fixtures?
- Are there transaction rollback fixtures (`db`, `transactional_db`)?
- Are HTTP calls mocked via `responses`, `httpretty`, `respx`, or `pytest-httpx`?
- Is there a test database URL in `.env.test` or `pytest.ini`?
## 7. Markers and Config
Read `pytest.ini`, `setup.cfg`, or `[tool.pytest.ini_options]` in `pyproject.toml`.
Note:
- Custom markers (`@pytest.mark.integration`, `@pytest.mark.slow`)
- `asyncio_mode` setting (`auto` or `strict`)
- `testpaths` — where pytest looks for tests
- Any `filterwarnings` that should be preserved
## Output Format
Summarize findings as:
```
Framework: pytest 8.x
Async: pytest-asyncio, asyncio_mode=auto
Test root: tests/
Naming: test_{module}.py mirroring src/
Classes: no (bare functions)
Fixtures: conftest.py at root; session-scoped db_engine, function-scoped db_session
Factories: factory_boy (UserFactory, ProjectFactory)
Mocks: pytest-mock (mocker fixture)
HTTP mocks: respx
Markers: integration, slow, unit
Assertions: plain assert
```
@@ -0,0 +1,168 @@
# Test Writing Rules
Mandatory rules when generating test code. Never violate without explicit user approval.
## File Structure
```python
# 1. stdlib imports
# 2. third-party imports
# 3. local imports (blank line between groups — match project convention)
# 4. Module-level constants or type aliases (if needed)
# 5. Fixtures local to this file (if not in conftest)
# 6. Test functions / class
```
Always match the import grouping style found in existing tests.
## Naming
| Target | Convention | Example |
|--------|-----------|---------|
| Test file | `test_{module}.py` or `{module}_test.py` (match project) | `test_auth.py` |
| Test function | `test_{what}_{condition}` | `test_login_wrong_password` |
| Test class | `Test{Subject}` | `TestLoginService` |
| Fixture | `{noun}` or `{noun}_{qualifier}` | `db_session`, `admin_user` |
| Parametrize ID | descriptive string | `"valid"`, `"empty_email"` |
## Type Annotations — Python 3.13+
All test functions must have return type `-> None`.
Fixture functions must have explicit return type annotation.
```python
# Correct
def test_login_valid(db_session: AsyncSession, user: User) -> None:
...
@pytest.fixture
def admin_user(db_session: AsyncSession) -> User:
...
```
Never use `Optional[X]` — use `X | None`.
Never use `List[X]`, `Dict[K, V]` — use `list[X]`, `dict[K, V]`.
## Assertions
Use the same assertion style as the project (discovered in Step 2).
For plain `assert` style:
- Always compare with `==`, not `is`, for value equality
- Use `assert x is None` / `assert x is not None` for None checks
- Use `assert isinstance(x, SomeClass)` for type checks
- Include a failure message for non-obvious assertions:
`assert result.status == 200, f"Expected 200, got {result.status}"`
For exception assertions:
```python
with pytest.raises(ValueError, match="invalid email"):
service.create_user(email="bad")
```
Never assert `True` or `False` directly from a function that returns bool:
```python
# Bad
assert service.is_valid(x) == True
# Good
assert service.is_valid(x)
```
## Fixtures
- Prefer existing fixtures from conftest.py over creating new ones
- Create a new fixture only if it will be reused in ≥ 2 tests in this file
- One-off setup → inline in the test body
- Scope: use `function` by default; `module`/`session` only for read-only shared resources
- Never use `autouse=True` for fixtures defined inside a test file
## Parametrize
Use `@pytest.mark.parametrize` when the same logic is tested with ≥ 3 data variants:
```python
@pytest.mark.parametrize(
("email", "expected_error"),
[
("", "Email cannot be empty"),
("not-an-email", "Invalid email format"),
("a" * 256 + "@x.com", "Email too long"),
],
)
def test_create_user_invalid_email(
email: str,
expected_error: str,
db_session: AsyncSession,
) -> None:
with pytest.raises(ValueError, match=expected_error):
UserService(db_session).create_user(email=email)
```
## Mocking
Match the project's mock style (discovered in Step 2).
With `pytest-mock`:
```python
def test_sends_email(mocker: MockerFixture) -> None:
send = mocker.patch("myapp.notifications.send_email")
service.register(email="u@example.com")
send.assert_called_once_with("u@example.com", subject=mocker.ANY)
```
With `unittest.mock`:
```python
from unittest.mock import AsyncMock, patch
@patch("myapp.notifications.send_email", new_callable=AsyncMock)
async def test_sends_email(mock_send: AsyncMock) -> None:
await service.register(email="u@example.com")
mock_send.assert_awaited_once()
```
Use `create_autospec` when the interface must be enforced:
```python
mock_repo = create_autospec(UserRepository)
```
Never use `MagicMock()` without speccing — it silently accepts any attribute.
## Async Tests
Match `asyncio_mode` from `pyproject.toml`:
- `asyncio_mode = "auto"` → no decorator needed, just `async def test_`
- `asyncio_mode = "strict"` → must add `@pytest.mark.asyncio`
Never mix sync and async in the same test — use `AsyncMock` for async dependencies.
## Test Isolation Rules
- Each test must be independent — no shared mutable state between tests
- Never rely on test execution order
- Clean up side effects: files, DB rows, environment variables
- Use `monkeypatch` for environment variable overrides (not `os.environ` direct)
- Use `tmp_path` fixture for temporary file I/O
## Scenario Coverage per Function
At minimum, cover:
| Scenario | Must have |
|----------|-----------|
| Happy path | ✅ Always |
| Empty / zero input | ✅ If applicable |
| Boundary values | ✅ If numeric limits exist |
| Invalid type or format | ✅ If input is validated |
| Not found / missing resource | ✅ For DB/file lookups |
| Permission denied | ✅ If auth is involved |
| External service failure | ✅ If network/DB call exists |
## What NOT to Test
- Internal private methods (`_method`) directly — test via public API
- Framework internals (Django ORM, SQLAlchemy internals)
- Third-party library behaviour (mock at the boundary, don't re-test the lib)
- Trivial getters/setters with zero logic