shelfmark/tests/README.md

# Test Suite

This directory contains the test suite for Shelfmark. Tests are organized by scope and component.

## Quick Start

```bash
# Run all unit tests (fast, no external dependencies)
docker exec test-cwabd python3 -m pytest tests/ -v -m "not integration and not e2e"

# Run E2E API tests
docker exec test-cwabd python3 -m pytest tests/e2e/ -v -m e2e

# Run everything except integration tests
docker exec test-cwabd python3 -m pytest tests/ -v -m "not integration"
```

## Test Structure

```
tests/
├── config/                 # Settings & configuration tests
│   ├── test_docker_volumes.py          # Docker volume mapping
│   ├── test_environment.py             # Environment variable handling
│   ├── test_mirror_settings_live_apply.py  # Mirror settings live reload
│   ├── test_mirror_settings_options.py # Mirror settings options
│   ├── test_security.py               # Security settings
│   └── test_oidc_settings.py          # OIDC settings fields & show_when conditions
│
├── core/                   # Core application logic tests
│   ├── test_admin_users_api.py         # Admin user CRUD API endpoints
│   ├── test_booklore_multiuser.py      # BookLore per-user override merging
│   ├── test_builtin_multiuser.py       # Builtin auth multi-user migration
│   ├── test_download_processing.py     # Download file processing
│   ├── test_hardlink.py               # Hardlink/copy operations
│   ├── test_library_processing.py      # Library file processing
│   ├── test_manual_query.py           # Manual search query handling
│   ├── test_mirrors_config.py         # Mirror configuration
│   ├── test_naming.py                 # File naming templates
│   ├── test_oidc_auth.py             # OIDC auth helpers (group claims, user provisioning)
│   ├── test_oidc_integration.py       # OIDC integration into auth system (logic mirror)
│   ├── test_oidc_routes.py           # OIDC Flask route handlers
│   ├── test_part_number_extraction.py  # Part number extraction
│   ├── test_per_user_downloads.py     # Per-user download queue filtering
│   ├── test_permission_handling.py    # File permission handling
│   ├── test_processing_integration.py  # Processing integration
│   ├── test_search_plan.py           # Search plan logic
│   ├── test_user_db.py               # UserDB CRUD operations
│   └── test_user_template_variable.py  # {User} template variable in naming
│
├── e2e/                    # End-to-end API tests
│   ├── conftest.py         # Fixtures (APIClient, DownloadTracker)
│   ├── test_api.py         # Core API endpoint tests
│   ├── test_download_flow.py   # Full download journey tests
│   └── test_prowlarr_flow.py   # Prowlarr-specific tests
│
├── prowlarr/               # Prowlarr plugin tests
│   ├── conftest.py         # Shared fixtures
│   ├── test_clients.py     # DownloadClient base, registry, DownloadStatus
│   ├── test_qbittorrent_client.py  # qBittorrent client unit tests
│   ├── test_transmission_client.py # Transmission client unit tests
│   ├── test_nzbget_client.py       # NZBGet client unit tests
│   ├── test_sabnzbd_client.py      # SABnzbd client unit tests
│   ├── test_handler.py     # ProwlarrHandler unit tests
│   ├── test_torrent_utils.py   # Bencode, hash extraction, URL parsing
│   ├── test_bencode.py     # Bencode encoding/decoding
│   ├── test_source.py      # Release source (size parsing, format detection)
│   ├── test_cache.py       # Release cache
│   ├── test_integration_clients.py  # Integration tests (require Docker stack)
│   └── test_integration_handler.py  # Handler integration tests
│
└── README.md               # This file
```

## Test Types

### Unit Tests
Fast tests that mock external dependencies. Run these frequently during development.

```bash
docker exec test-cwabd python3 -m pytest tests/prowlarr/ -v -m "not integration"
```

**What they test:**
- Download client logic (status mapping, URL handling, error cases)
- Bencode encoding/decoding for torrent files
- Hash extraction from magnet links and .torrent files
- Protocol detection (torrent vs usenet)
- Release cache operations
- Handler download flow logic
- User database (CRUD, settings, OIDC subject linking)
- OIDC authentication (group claims, user provisioning, route handlers)
- Admin user management API (create, update, delete, password, per-user settings)
- Multi-user download queue filtering and per-user overrides
- Settings configuration (OIDC fields, show_when conditions)

### E2E Tests
Test the full application through its HTTP API. Require the app to be running.

```bash
docker exec test-cwabd python3 -m pytest tests/e2e/ -v -m e2e
```

**What they test:**
- Health check endpoint
- Configuration endpoint
- Metadata provider search (Hardcover, etc.)
- Release source listing
- Download queue operations (add, cancel, reorder, clear)
- Settings API
- Prowlarr integration

### Integration Tests
Test against real services (qBittorrent, Transmission, etc.). Require the full Docker test stack.

```bash
# Start the test stack first
docker compose -f docker-compose.test-clients.yml up -d

# Run integration tests
docker exec test-cwabd python3 -m pytest tests/prowlarr/ -v -m integration
```

**What they test:**
- Real connections to download clients
- Adding/removing actual torrents
- Status polling from real clients

## Test Markers

| Marker | Description | When to Skip |
|--------|-------------|--------------|
| `integration` | Requires running services (qBittorrent, etc.) | Default skip with `-m "not integration"` |
| `e2e` | End-to-end API tests | When app isn't running |
| `slow` | Tests that take longer (network calls, polling) | Quick feedback with `-m "not slow"` |

## Common Commands

```bash
# Run specific test file
docker exec test-cwabd python3 -m pytest tests/prowlarr/test_clients.py -v

# Run specific test class
docker exec test-cwabd python3 -m pytest tests/e2e/test_api.py::TestHealthEndpoint -v

# Run specific test
docker exec test-cwabd python3 -m pytest tests/e2e/test_api.py::TestHealthEndpoint::test_health_returns_ok -v

# Run with short traceback (cleaner output)
docker exec test-cwabd python3 -m pytest tests/ -v --tb=short -m "not integration"

# Run and stop on first failure
docker exec test-cwabd python3 -m pytest tests/ -v -x -m "not integration"

# Run with coverage (if pytest-cov installed)
docker exec test-cwabd python3 -m pytest tests/ --cov=shelfmark -m "not integration"
```

## Writing New Tests

### Unit Test Example

```python
from unittest.mock import MagicMock, patch

class TestMyFeature:
    def test_something(self, monkeypatch):
        # Mock config values
        monkeypatch.setattr(
            "shelfmark.module.config.get",
            lambda key, default="": {"KEY": "value"}.get(key, default),
        )

        # Test your code
        result = my_function()
        assert result == expected
```

### E2E Test Example

```python
import pytest
from .conftest import APIClient, DownloadTracker

@pytest.mark.e2e
class TestMyEndpoint:
    def test_endpoint_works(self, api_client: APIClient):
        resp = api_client.get("/api/my-endpoint")
        assert resp.status_code == 200

    def test_with_cleanup(self, api_client: APIClient, download_tracker: DownloadTracker):
        # Track IDs for automatic cleanup after test
        download_tracker.track("some-id")
        # ... test code ...
```

## Test Fixtures

### E2E Fixtures (`tests/e2e/conftest.py`)

| Fixture | Scope | Description |
|---------|-------|-------------|
| `api_client` | session | HTTP client for API calls |
| `download_tracker` | function | Tracks downloads for cleanup |
| `server_config` | session | Cached server configuration |

### Prowlarr Fixtures (`tests/prowlarr/conftest.py`)

| Fixture | Scope | Description |
|---------|-------|-------------|
| `transmission_client` | module | Real Transmission client (integration) |
| `qbittorrent_client` | module | Real qBittorrent client (integration) |
| `deluge_client` | module | Real Deluge client (integration) |
| `nzbget_client` | module | Real NZBGet client (integration) |
| `sabnzbd_client` | module | Real SABnzbd client (integration) |

## Expected Skips

Some tests skip when external services aren't available. This is normal:

- **"No metadata providers available"** - Metadata provider not responding
- **"Prowlarr not configured"** - Prowlarr settings not set up
- **"No releases found"** - No indexers configured in Prowlarr
- **"Legacy search source unavailable"** - Direct download source offline
- **"Transmission/qBittorrent not available"** - Docker test stack not running

## Troubleshooting

### Tests can't connect to app
```bash
# Check the app is running
docker ps | grep test-cwabd

# Check app logs
docker logs test-cwabd
```

### Import errors
```bash
# Make sure you're running inside the container
docker exec test-cwabd python3 -m pytest ...

# Not from your local machine
pytest ...  # This won't work
```

### Integration tests failing
```bash
# Make sure test stack is running
docker compose -f docker-compose.test-clients.yml up -d

# Check client containers
docker ps | grep -E "qbittorrent|transmission|deluge|nzbget|sabnzbd"
```

### Stale test data
```bash
# Clear the queue between test runs
docker exec test-cwabd python3 -c "
from shelfmark.core.queue import book_queue
book_queue.clear()
"
```