96 lines
No EOL
2.3 KiB
Markdown
96 lines
No EOL
2.3 KiB
Markdown
# AI Router Test Suite
|
|
|
|
This test suite contains real API tests for the AI Router library using pytest.
|
|
|
|
## Setup
|
|
|
|
1. Install test dependencies:
|
|
```bash
|
|
pip install -r requirements-test.txt
|
|
```
|
|
|
|
2. Create a `.env` file in the project root with your API keys:
|
|
```bash
|
|
COHERE_API_KEY=your_cohere_key
|
|
GEMINI_API_KEY=your_gemini_key
|
|
OPENAI_API_KEY=your_openai_key
|
|
OPENAI_BASE_URL=your_openai_base_url # Optional custom endpoint
|
|
OLLAMA_BASE_URL=http://localhost:11434 # For local Ollama
|
|
```
|
|
|
|
3. For Ollama tests, ensure Ollama is running:
|
|
```bash
|
|
ollama serve
|
|
ollama pull nomic-embed-text:latest
|
|
ollama pull mxbai-embed-large:latest
|
|
```
|
|
|
|
## Running Tests with Pytest
|
|
|
|
### Run all tests:
|
|
```bash
|
|
pytest
|
|
```
|
|
|
|
### Run with verbose output:
|
|
```bash
|
|
pytest -v
|
|
```
|
|
|
|
### Run specific test file:
|
|
```bash
|
|
pytest tests/test_embeddings.py
|
|
pytest tests/test_generation.py -v
|
|
```
|
|
|
|
### Run specific test class or method:
|
|
```bash
|
|
pytest tests/test_embeddings.py::TestCohereEmbeddings
|
|
pytest tests/test_embeddings.py::TestCohereEmbeddings::test_single_text_embedding
|
|
```
|
|
|
|
### Run tests by marker:
|
|
```bash
|
|
# Run only integration tests
|
|
pytest -m integration
|
|
|
|
# Run tests excluding Ollama
|
|
pytest -m "not ollama"
|
|
|
|
# Run only Ollama tests
|
|
pytest -m ollama
|
|
```
|
|
|
|
### Run with coverage:
|
|
```bash
|
|
pytest --cov=router tests/
|
|
```
|
|
|
|
### Run tests in parallel:
|
|
```bash
|
|
# Install pytest-xdist first
|
|
pip install pytest-xdist
|
|
pytest -n auto
|
|
```
|
|
|
|
## Test Coverage
|
|
|
|
- **test_config.py**: Tests configuration loading and API key management
|
|
- **test_embeddings.py**: Tests embedding APIs (Cohere, Gemini, Ollama)
|
|
- **test_rerank.py**: Tests Cohere reranking functionality
|
|
- **test_generation.py**: Tests text generation (OpenAI-compatible, Gemini, Cohere)
|
|
|
|
## Test Markers
|
|
|
|
- `@pytest.mark.integration`: Marks integration tests that call real APIs
|
|
- `@pytest.mark.asyncio`: Marks async tests
|
|
- `@pytest.mark.ollama`: Marks tests that require Ollama to be running
|
|
- `@pytest.mark.parametrize`: Used for running tests with multiple input values
|
|
|
|
## Notes
|
|
|
|
- These tests use real APIs and will consume API credits
|
|
- Ollama tests are marked with `@pytest.mark.ollama` and can be skipped
|
|
- All tests include both sync and async variants
|
|
- Tests verify functionality without checking exact output values
|
|
- The test suite is configured for `asyncio_mode = auto` in pytest.ini |