fix
This commit is contained in:
@@ -0,0 +1,112 @@
|
||||
# Discover Project Test Patterns
|
||||
|
||||
Run this discovery process once at the start of every autotest-writer session.
|
||||
Cache results — do not repeat searches for the same project.
|
||||
|
||||
## 1. Framework Detection
|
||||
|
||||
Search for test dependencies in `pyproject.toml` or `requirements*.txt`:
|
||||
|
||||
| Found | Framework |
|
||||
|-------|-----------|
|
||||
| `pytest` | pytest (default assumption) |
|
||||
| `unittest` only | stdlib unittest |
|
||||
| `anyio`, `pytest-asyncio` | async pytest |
|
||||
| `hypothesis` | property-based testing present |
|
||||
|
||||
If none found — default to **pytest**.
|
||||
|
||||
## 2. Test File Layout
|
||||
|
||||
Run a glob search for `test_*.py` and `*_test.py` files.
|
||||
Determine:
|
||||
- **Root test folder**: `tests/`, `test/`, inline next to source, or mixed
|
||||
- **Naming convention**: `test_{module}.py` or `{module}_test.py`
|
||||
- **Mirrors source tree?** e.g. `src/auth/login.py` → `tests/auth/test_login.py`
|
||||
|
||||
## 3. conftest.py Analysis
|
||||
|
||||
Read all `conftest.py` files found (root + subdirectories).
|
||||
Extract:
|
||||
- All fixture names and their scopes (`function`, `session`, `module`)
|
||||
- Factory fixtures (return a callable/class)
|
||||
- Common parametrize patterns
|
||||
- Any custom pytest plugins or hooks registered
|
||||
|
||||
## 4. Read Sample Tests
|
||||
|
||||
Read **2–3 representative test files** (pick files that seem most complete, not trivial ones).
|
||||
Extract the following patterns:
|
||||
|
||||
### Import style
|
||||
```python
|
||||
# Note: stdlib first? third-party second? local last?
|
||||
# Note: are there "from __future__ import annotations"?
|
||||
```
|
||||
|
||||
### Class vs function tests
|
||||
- Are tests in classes (`class TestLogin:`) or bare functions?
|
||||
- If classes — do they inherit from anything?
|
||||
|
||||
### Assertion style
|
||||
- Plain `assert a == b`
|
||||
- `assert a == b, "message"`
|
||||
- Custom matchers: `assert_that(a).equals(b)`, `expect(a).to.equal(b)`
|
||||
|
||||
### Mock / patch style
|
||||
- `unittest.mock.patch` as decorator
|
||||
- `unittest.mock.patch` as context manager
|
||||
- `pytest-mock` via `mocker` fixture
|
||||
- `MagicMock` / `AsyncMock` / `create_autospec`
|
||||
|
||||
### Async style
|
||||
- `@pytest.mark.asyncio` + `async def test_`
|
||||
- `anyio` backend marker
|
||||
- Synchronous wrappers around async code
|
||||
|
||||
### Parametrize style
|
||||
```python
|
||||
@pytest.mark.parametrize("input,expected", [...])
|
||||
```
|
||||
or inline in class docstring (hypothesis)?
|
||||
|
||||
## 5. Factory / Builder Patterns
|
||||
|
||||
Search for:
|
||||
- `factory_boy` — `UserFactory`, `PostFactory` classes
|
||||
- `pytest-factoryboy` — `register()` usage
|
||||
- Custom builder fixtures that return objects
|
||||
- `faker` usage for test data generation
|
||||
|
||||
## 6. Database / External Service Patterns
|
||||
|
||||
- Does the project use `pytest-django`, `pytest-sqlalchemy`, or custom DB fixtures?
|
||||
- Are there transaction rollback fixtures (`db`, `transactional_db`)?
|
||||
- Are HTTP calls mocked via `responses`, `httpretty`, `respx`, or `pytest-httpx`?
|
||||
- Is there a test database URL in `.env.test` or `pytest.ini`?
|
||||
|
||||
## 7. Markers and Config
|
||||
|
||||
Read `pytest.ini`, `setup.cfg`, or `[tool.pytest.ini_options]` in `pyproject.toml`.
|
||||
Note:
|
||||
- Custom markers (`@pytest.mark.integration`, `@pytest.mark.slow`)
|
||||
- `asyncio_mode` setting (`auto` or `strict`)
|
||||
- `testpaths` — where pytest looks for tests
|
||||
- Any `filterwarnings` that should be preserved
|
||||
|
||||
## Output Format
|
||||
|
||||
Summarize findings as:
|
||||
```
|
||||
Framework: pytest 8.x
|
||||
Async: pytest-asyncio, asyncio_mode=auto
|
||||
Test root: tests/
|
||||
Naming: test_{module}.py mirroring src/
|
||||
Classes: no (bare functions)
|
||||
Fixtures: conftest.py at root; session-scoped db_engine, function-scoped db_session
|
||||
Factories: factory_boy (UserFactory, ProjectFactory)
|
||||
Mocks: pytest-mock (mocker fixture)
|
||||
HTTP mocks: respx
|
||||
Markers: integration, slow, unit
|
||||
Assertions: plain assert
|
||||
```
|
||||
@@ -0,0 +1,168 @@
|
||||
# Test Writing Rules
|
||||
|
||||
Mandatory rules when generating test code. Never violate without explicit user approval.
|
||||
|
||||
## File Structure
|
||||
|
||||
```python
|
||||
# 1. stdlib imports
|
||||
# 2. third-party imports
|
||||
# 3. local imports (blank line between groups — match project convention)
|
||||
|
||||
# 4. Module-level constants or type aliases (if needed)
|
||||
|
||||
# 5. Fixtures local to this file (if not in conftest)
|
||||
|
||||
# 6. Test functions / class
|
||||
```
|
||||
|
||||
Always match the import grouping style found in existing tests.
|
||||
|
||||
## Naming
|
||||
|
||||
| Target | Convention | Example |
|
||||
|--------|-----------|---------|
|
||||
| Test file | `test_{module}.py` or `{module}_test.py` (match project) | `test_auth.py` |
|
||||
| Test function | `test_{what}_{condition}` | `test_login_wrong_password` |
|
||||
| Test class | `Test{Subject}` | `TestLoginService` |
|
||||
| Fixture | `{noun}` or `{noun}_{qualifier}` | `db_session`, `admin_user` |
|
||||
| Parametrize ID | descriptive string | `"valid"`, `"empty_email"` |
|
||||
|
||||
## Type Annotations — Python 3.13+
|
||||
|
||||
All test functions must have return type `-> None`.
|
||||
Fixture functions must have explicit return type annotation.
|
||||
|
||||
```python
|
||||
# Correct
|
||||
def test_login_valid(db_session: AsyncSession, user: User) -> None:
|
||||
...
|
||||
|
||||
@pytest.fixture
|
||||
def admin_user(db_session: AsyncSession) -> User:
|
||||
...
|
||||
```
|
||||
|
||||
Never use `Optional[X]` — use `X | None`.
|
||||
Never use `List[X]`, `Dict[K, V]` — use `list[X]`, `dict[K, V]`.
|
||||
|
||||
## Assertions
|
||||
|
||||
Use the same assertion style as the project (discovered in Step 2).
|
||||
|
||||
For plain `assert` style:
|
||||
- Always compare with `==`, not `is`, for value equality
|
||||
- Use `assert x is None` / `assert x is not None` for None checks
|
||||
- Use `assert isinstance(x, SomeClass)` for type checks
|
||||
- Include a failure message for non-obvious assertions:
|
||||
`assert result.status == 200, f"Expected 200, got {result.status}"`
|
||||
|
||||
For exception assertions:
|
||||
```python
|
||||
with pytest.raises(ValueError, match="invalid email"):
|
||||
service.create_user(email="bad")
|
||||
```
|
||||
|
||||
Never assert `True` or `False` directly from a function that returns bool:
|
||||
```python
|
||||
# Bad
|
||||
assert service.is_valid(x) == True
|
||||
# Good
|
||||
assert service.is_valid(x)
|
||||
```
|
||||
|
||||
## Fixtures
|
||||
|
||||
- Prefer existing fixtures from conftest.py over creating new ones
|
||||
- Create a new fixture only if it will be reused in ≥ 2 tests in this file
|
||||
- One-off setup → inline in the test body
|
||||
- Scope: use `function` by default; `module`/`session` only for read-only shared resources
|
||||
- Never use `autouse=True` for fixtures defined inside a test file
|
||||
|
||||
## Parametrize
|
||||
|
||||
Use `@pytest.mark.parametrize` when the same logic is tested with ≥ 3 data variants:
|
||||
|
||||
```python
|
||||
@pytest.mark.parametrize(
|
||||
("email", "expected_error"),
|
||||
[
|
||||
("", "Email cannot be empty"),
|
||||
("not-an-email", "Invalid email format"),
|
||||
("a" * 256 + "@x.com", "Email too long"),
|
||||
],
|
||||
)
|
||||
def test_create_user_invalid_email(
|
||||
email: str,
|
||||
expected_error: str,
|
||||
db_session: AsyncSession,
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match=expected_error):
|
||||
UserService(db_session).create_user(email=email)
|
||||
```
|
||||
|
||||
## Mocking
|
||||
|
||||
Match the project's mock style (discovered in Step 2).
|
||||
|
||||
With `pytest-mock`:
|
||||
```python
|
||||
def test_sends_email(mocker: MockerFixture) -> None:
|
||||
send = mocker.patch("myapp.notifications.send_email")
|
||||
service.register(email="u@example.com")
|
||||
send.assert_called_once_with("u@example.com", subject=mocker.ANY)
|
||||
```
|
||||
|
||||
With `unittest.mock`:
|
||||
```python
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
@patch("myapp.notifications.send_email", new_callable=AsyncMock)
|
||||
async def test_sends_email(mock_send: AsyncMock) -> None:
|
||||
await service.register(email="u@example.com")
|
||||
mock_send.assert_awaited_once()
|
||||
```
|
||||
|
||||
Use `create_autospec` when the interface must be enforced:
|
||||
```python
|
||||
mock_repo = create_autospec(UserRepository)
|
||||
```
|
||||
|
||||
Never use `MagicMock()` without speccing — it silently accepts any attribute.
|
||||
|
||||
## Async Tests
|
||||
|
||||
Match `asyncio_mode` from `pyproject.toml`:
|
||||
- `asyncio_mode = "auto"` → no decorator needed, just `async def test_`
|
||||
- `asyncio_mode = "strict"` → must add `@pytest.mark.asyncio`
|
||||
|
||||
Never mix sync and async in the same test — use `AsyncMock` for async dependencies.
|
||||
|
||||
## Test Isolation Rules
|
||||
|
||||
- Each test must be independent — no shared mutable state between tests
|
||||
- Never rely on test execution order
|
||||
- Clean up side effects: files, DB rows, environment variables
|
||||
- Use `monkeypatch` for environment variable overrides (not `os.environ` direct)
|
||||
- Use `tmp_path` fixture for temporary file I/O
|
||||
|
||||
## Scenario Coverage per Function
|
||||
|
||||
At minimum, cover:
|
||||
|
||||
| Scenario | Must have |
|
||||
|----------|-----------|
|
||||
| Happy path | ✅ Always |
|
||||
| Empty / zero input | ✅ If applicable |
|
||||
| Boundary values | ✅ If numeric limits exist |
|
||||
| Invalid type or format | ✅ If input is validated |
|
||||
| Not found / missing resource | ✅ For DB/file lookups |
|
||||
| Permission denied | ✅ If auth is involved |
|
||||
| External service failure | ✅ If network/DB call exists |
|
||||
|
||||
## What NOT to Test
|
||||
|
||||
- Internal private methods (`_method`) directly — test via public API
|
||||
- Framework internals (Django ORM, SQLAlchemy internals)
|
||||
- Third-party library behaviour (mock at the boundary, don't re-test the lib)
|
||||
- Trivial getters/setters with zero logic
|
||||
Reference in New Issue
Block a user