fix
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
---
|
||||
name: autotest-writer
|
||||
description: "Write autotests that match the project's existing test style, framework, fixtures, and conventions. Use when: user asks to write tests, generate autotests, cover code with tests, add unit tests, add integration tests, написать тесты, покрыть тестами, написать автотесты, сгенерировать тесты, добавить тесты. Discovers patterns from existing tests first. Uses MCP context7 for library docs when needed. Applies python-review skill rules for type annotations."
|
||||
argument-hint: "File or function to cover with tests (e.g. src/auth.py or src/auth.py::login)"
|
||||
---
|
||||
|
||||
# Autotest Writer
|
||||
|
||||
Discovers how the project writes tests, then generates autotests that look like they were written by the same developer.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Cover a new function or module with tests
|
||||
- Generate tests from a TestLink mini-test decomposition (pairs with `testlink-decompose`)
|
||||
- Add missing edge-case or negative tests
|
||||
- Scaffold a test file for a new module
|
||||
|
||||
## Procedure
|
||||
|
||||
### Step 1 — Understand What to Test
|
||||
|
||||
From the user's request, identify:
|
||||
- **Target**: specific file, class, function, or mini-test ID (e.g. `КМД-1831-MT1`)
|
||||
- **Scope**: unit / integration / end-to-end
|
||||
- **Source**: existing code to read, or mini-test structure from `testlink-decompose`
|
||||
|
||||
If the target is a mini-test ID — treat its steps and expected results as the test specification.
|
||||
If the target is a source file — read it fully before proceeding.
|
||||
|
||||
### Step 2 — Discover Project Test Patterns
|
||||
|
||||
Follow the full discovery process in
|
||||
[references/discover-patterns.md](./references/discover-patterns.md).
|
||||
|
||||
Output a one-line summary before continuing:
|
||||
> Detected: **{framework}** · fixtures via **{conftest/class/factory}** · assertions: **{assert/custom}** · async: **{yes/no}** · mocks: **{unittest.mock / pytest-mock / none}**
|
||||
|
||||
### Step 3 — Resolve Library Documentation (if needed)
|
||||
|
||||
When the code under test uses a third-party library AND the project has no existing tests covering it:
|
||||
|
||||
1. Call `mcp_io_github_ups_resolve-library-id` with the library name to get the context7 ID
|
||||
2. Call `mcp_io_github_ups_get-library-docs` with that ID to retrieve relevant docs
|
||||
3. Extract: how to instantiate, patch, or assert against the library's objects
|
||||
4. If context7 returns nothing — read the library's source from `site-packages/` in the virtualenv
|
||||
|
||||
Do NOT fetch docs for libraries already exercised in existing project tests.
|
||||
|
||||
### Step 4 — Plan the Tests
|
||||
|
||||
Before writing code, output a test plan as a table:
|
||||
|
||||
| # | Test name | Scenario type | Target | Fixtures needed |
|
||||
|---|-----------|--------------|--------|-----------------|
|
||||
| 1 | `test_login_valid_credentials` | happy path | `auth.login()` | `db_session`, `user_factory` |
|
||||
| 2 | `test_login_wrong_password` | negative | `auth.login()` | `db_session`, `user_factory` |
|
||||
| 3 | `test_login_user_not_found` | negative | `auth.login()` | `db_session` |
|
||||
|
||||
Coverage target: at minimum cover happy path + all negative/edge cases visible in the source.
|
||||
|
||||
Ask: "Продолжить? Или изменить план?" — wait for confirmation before writing code if the plan has more than 5 tests.
|
||||
For ≤ 5 tests, proceed immediately.
|
||||
|
||||
### Step 5 — Write the Tests
|
||||
|
||||
Follow all rules in [references/write-rules.md](./references/write-rules.md).
|
||||
|
||||
Key constraints (never violate):
|
||||
- Match the exact file naming convention discovered in Step 2
|
||||
- Use the same fixture sources (conftest.py / inline / factory) as existing tests
|
||||
- Use the same assertion style (plain `assert` vs custom matchers)
|
||||
- Use the same import order and grouping
|
||||
- Use Python 3.13+ type annotations on all new functions
|
||||
- Match async/sync style of the surrounding test suite
|
||||
|
||||
### Step 6 — Place the File
|
||||
|
||||
Determine the output path:
|
||||
- If a parallel test file exists (e.g. `src/auth.py` → `tests/test_auth.py`) — append to it
|
||||
- If no test file exists — create one at the location matching the project layout
|
||||
- Never create a test file in the source tree unless that is the project's convention
|
||||
|
||||
Output the full file path before writing.
|
||||
|
||||
### Step 7 — Validate
|
||||
|
||||
After writing, run:
|
||||
|
||||
```bash
|
||||
# Syntax and type check
|
||||
mypy --strict <test_file>
|
||||
|
||||
# Lint
|
||||
ruff check <test_file>
|
||||
|
||||
# Execute tests (dry run — collect only)
|
||||
pytest --collect-only <test_file>
|
||||
```
|
||||
|
||||
Fix any collection errors or mypy/ruff violations silently.
|
||||
If tests actually run — report pass/fail counts.
|
||||
@@ -0,0 +1,112 @@
|
||||
# Discover Project Test Patterns
|
||||
|
||||
Run this discovery process once at the start of every autotest-writer session.
|
||||
Cache results — do not repeat searches for the same project.
|
||||
|
||||
## 1. Framework Detection
|
||||
|
||||
Search for test dependencies in `pyproject.toml` or `requirements*.txt`:
|
||||
|
||||
| Found | Framework |
|
||||
|-------|-----------|
|
||||
| `pytest` | pytest (default assumption) |
|
||||
| `unittest` only | stdlib unittest |
|
||||
| `anyio`, `pytest-asyncio` | async pytest |
|
||||
| `hypothesis` | property-based testing present |
|
||||
|
||||
If none found — default to **pytest**.
|
||||
|
||||
## 2. Test File Layout
|
||||
|
||||
Run a glob search for `test_*.py` and `*_test.py` files.
|
||||
Determine:
|
||||
- **Root test folder**: `tests/`, `test/`, inline next to source, or mixed
|
||||
- **Naming convention**: `test_{module}.py` or `{module}_test.py`
|
||||
- **Mirrors source tree?** e.g. `src/auth/login.py` → `tests/auth/test_login.py`
|
||||
|
||||
## 3. conftest.py Analysis
|
||||
|
||||
Read all `conftest.py` files found (root + subdirectories).
|
||||
Extract:
|
||||
- All fixture names and their scopes (`function`, `session`, `module`)
|
||||
- Factory fixtures (return a callable/class)
|
||||
- Common parametrize patterns
|
||||
- Any custom pytest plugins or hooks registered
|
||||
|
||||
## 4. Read Sample Tests
|
||||
|
||||
Read **2–3 representative test files** (pick files that seem most complete, not trivial ones).
|
||||
Extract the following patterns:
|
||||
|
||||
### Import style
|
||||
```python
|
||||
# Note: stdlib first? third-party second? local last?
|
||||
# Note: are there "from __future__ import annotations"?
|
||||
```
|
||||
|
||||
### Class vs function tests
|
||||
- Are tests in classes (`class TestLogin:`) or bare functions?
|
||||
- If classes — do they inherit from anything?
|
||||
|
||||
### Assertion style
|
||||
- Plain `assert a == b`
|
||||
- `assert a == b, "message"`
|
||||
- Custom matchers: `assert_that(a).equals(b)`, `expect(a).to.equal(b)`
|
||||
|
||||
### Mock / patch style
|
||||
- `unittest.mock.patch` as decorator
|
||||
- `unittest.mock.patch` as context manager
|
||||
- `pytest-mock` via `mocker` fixture
|
||||
- `MagicMock` / `AsyncMock` / `create_autospec`
|
||||
|
||||
### Async style
|
||||
- `@pytest.mark.asyncio` + `async def test_`
|
||||
- `anyio` backend marker
|
||||
- Synchronous wrappers around async code
|
||||
|
||||
### Parametrize style
|
||||
```python
|
||||
@pytest.mark.parametrize("input,expected", [...])
|
||||
```
|
||||
or inline in class docstring (hypothesis)?
|
||||
|
||||
## 5. Factory / Builder Patterns
|
||||
|
||||
Search for:
|
||||
- `factory_boy` — `UserFactory`, `PostFactory` classes
|
||||
- `pytest-factoryboy` — `register()` usage
|
||||
- Custom builder fixtures that return objects
|
||||
- `faker` usage for test data generation
|
||||
|
||||
## 6. Database / External Service Patterns
|
||||
|
||||
- Does the project use `pytest-django`, `pytest-sqlalchemy`, or custom DB fixtures?
|
||||
- Are there transaction rollback fixtures (`db`, `transactional_db`)?
|
||||
- Are HTTP calls mocked via `responses`, `httpretty`, `respx`, or `pytest-httpx`?
|
||||
- Is there a test database URL in `.env.test` or `pytest.ini`?
|
||||
|
||||
## 7. Markers and Config
|
||||
|
||||
Read `pytest.ini`, `setup.cfg`, or `[tool.pytest.ini_options]` in `pyproject.toml`.
|
||||
Note:
|
||||
- Custom markers (`@pytest.mark.integration`, `@pytest.mark.slow`)
|
||||
- `asyncio_mode` setting (`auto` or `strict`)
|
||||
- `testpaths` — where pytest looks for tests
|
||||
- Any `filterwarnings` that should be preserved
|
||||
|
||||
## Output Format
|
||||
|
||||
Summarize findings as:
|
||||
```
|
||||
Framework: pytest 8.x
|
||||
Async: pytest-asyncio, asyncio_mode=auto
|
||||
Test root: tests/
|
||||
Naming: test_{module}.py mirroring src/
|
||||
Classes: no (bare functions)
|
||||
Fixtures: conftest.py at root; session-scoped db_engine, function-scoped db_session
|
||||
Factories: factory_boy (UserFactory, ProjectFactory)
|
||||
Mocks: pytest-mock (mocker fixture)
|
||||
HTTP mocks: respx
|
||||
Markers: integration, slow, unit
|
||||
Assertions: plain assert
|
||||
```
|
||||
@@ -0,0 +1,168 @@
|
||||
# Test Writing Rules
|
||||
|
||||
Mandatory rules when generating test code. Never violate without explicit user approval.
|
||||
|
||||
## File Structure
|
||||
|
||||
```python
|
||||
# 1. stdlib imports
|
||||
# 2. third-party imports
|
||||
# 3. local imports (blank line between groups — match project convention)
|
||||
|
||||
# 4. Module-level constants or type aliases (if needed)
|
||||
|
||||
# 5. Fixtures local to this file (if not in conftest)
|
||||
|
||||
# 6. Test functions / class
|
||||
```
|
||||
|
||||
Always match the import grouping style found in existing tests.
|
||||
|
||||
## Naming
|
||||
|
||||
| Target | Convention | Example |
|
||||
|--------|-----------|---------|
|
||||
| Test file | `test_{module}.py` or `{module}_test.py` (match project) | `test_auth.py` |
|
||||
| Test function | `test_{what}_{condition}` | `test_login_wrong_password` |
|
||||
| Test class | `Test{Subject}` | `TestLoginService` |
|
||||
| Fixture | `{noun}` or `{noun}_{qualifier}` | `db_session`, `admin_user` |
|
||||
| Parametrize ID | descriptive string | `"valid"`, `"empty_email"` |
|
||||
|
||||
## Type Annotations — Python 3.13+
|
||||
|
||||
All test functions must have return type `-> None`.
|
||||
Fixture functions must have explicit return type annotation.
|
||||
|
||||
```python
|
||||
# Correct
|
||||
def test_login_valid(db_session: AsyncSession, user: User) -> None:
|
||||
...
|
||||
|
||||
@pytest.fixture
|
||||
def admin_user(db_session: AsyncSession) -> User:
|
||||
...
|
||||
```
|
||||
|
||||
Never use `Optional[X]` — use `X | None`.
|
||||
Never use `List[X]`, `Dict[K, V]` — use `list[X]`, `dict[K, V]`.
|
||||
|
||||
## Assertions
|
||||
|
||||
Use the same assertion style as the project (discovered in Step 2).
|
||||
|
||||
For plain `assert` style:
|
||||
- Always compare with `==`, not `is`, for value equality
|
||||
- Use `assert x is None` / `assert x is not None` for None checks
|
||||
- Use `assert isinstance(x, SomeClass)` for type checks
|
||||
- Include a failure message for non-obvious assertions:
|
||||
`assert result.status == 200, f"Expected 200, got {result.status}"`
|
||||
|
||||
For exception assertions:
|
||||
```python
|
||||
with pytest.raises(ValueError, match="invalid email"):
|
||||
service.create_user(email="bad")
|
||||
```
|
||||
|
||||
Never assert `True` or `False` directly from a function that returns bool:
|
||||
```python
|
||||
# Bad
|
||||
assert service.is_valid(x) == True
|
||||
# Good
|
||||
assert service.is_valid(x)
|
||||
```
|
||||
|
||||
## Fixtures
|
||||
|
||||
- Prefer existing fixtures from conftest.py over creating new ones
|
||||
- Create a new fixture only if it will be reused in ≥ 2 tests in this file
|
||||
- One-off setup → inline in the test body
|
||||
- Scope: use `function` by default; `module`/`session` only for read-only shared resources
|
||||
- Never use `autouse=True` for fixtures defined inside a test file
|
||||
|
||||
## Parametrize
|
||||
|
||||
Use `@pytest.mark.parametrize` when the same logic is tested with ≥ 3 data variants:
|
||||
|
||||
```python
|
||||
@pytest.mark.parametrize(
|
||||
("email", "expected_error"),
|
||||
[
|
||||
("", "Email cannot be empty"),
|
||||
("not-an-email", "Invalid email format"),
|
||||
("a" * 256 + "@x.com", "Email too long"),
|
||||
],
|
||||
)
|
||||
def test_create_user_invalid_email(
|
||||
email: str,
|
||||
expected_error: str,
|
||||
db_session: AsyncSession,
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match=expected_error):
|
||||
UserService(db_session).create_user(email=email)
|
||||
```
|
||||
|
||||
## Mocking
|
||||
|
||||
Match the project's mock style (discovered in Step 2).
|
||||
|
||||
With `pytest-mock`:
|
||||
```python
|
||||
def test_sends_email(mocker: MockerFixture) -> None:
|
||||
send = mocker.patch("myapp.notifications.send_email")
|
||||
service.register(email="u@example.com")
|
||||
send.assert_called_once_with("u@example.com", subject=mocker.ANY)
|
||||
```
|
||||
|
||||
With `unittest.mock`:
|
||||
```python
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
@patch("myapp.notifications.send_email", new_callable=AsyncMock)
|
||||
async def test_sends_email(mock_send: AsyncMock) -> None:
|
||||
await service.register(email="u@example.com")
|
||||
mock_send.assert_awaited_once()
|
||||
```
|
||||
|
||||
Use `create_autospec` when the interface must be enforced:
|
||||
```python
|
||||
mock_repo = create_autospec(UserRepository)
|
||||
```
|
||||
|
||||
Never use `MagicMock()` without speccing — it silently accepts any attribute.
|
||||
|
||||
## Async Tests
|
||||
|
||||
Match `asyncio_mode` from `pyproject.toml`:
|
||||
- `asyncio_mode = "auto"` → no decorator needed, just `async def test_`
|
||||
- `asyncio_mode = "strict"` → must add `@pytest.mark.asyncio`
|
||||
|
||||
Never mix sync and async in the same test — use `AsyncMock` for async dependencies.
|
||||
|
||||
## Test Isolation Rules
|
||||
|
||||
- Each test must be independent — no shared mutable state between tests
|
||||
- Never rely on test execution order
|
||||
- Clean up side effects: files, DB rows, environment variables
|
||||
- Use `monkeypatch` for environment variable overrides (not `os.environ` direct)
|
||||
- Use `tmp_path` fixture for temporary file I/O
|
||||
|
||||
## Scenario Coverage per Function
|
||||
|
||||
At minimum, cover:
|
||||
|
||||
| Scenario | Must have |
|
||||
|----------|-----------|
|
||||
| Happy path | ✅ Always |
|
||||
| Empty / zero input | ✅ If applicable |
|
||||
| Boundary values | ✅ If numeric limits exist |
|
||||
| Invalid type or format | ✅ If input is validated |
|
||||
| Not found / missing resource | ✅ For DB/file lookups |
|
||||
| Permission denied | ✅ If auth is involved |
|
||||
| External service failure | ✅ If network/DB call exists |
|
||||
|
||||
## What NOT to Test
|
||||
|
||||
- Internal private methods (`_method`) directly — test via public API
|
||||
- Framework internals (Django ORM, SQLAlchemy internals)
|
||||
- Third-party library behaviour (mock at the boundary, don't re-test the lib)
|
||||
- Trivial getters/setters with zero logic
|
||||
Reference in New Issue
Block a user