template-with-ai/tests

AI Router Test Suite

This test suite contains real API tests for the AI Router library using pytest.

Setup

1. Install test dependencies:

pip install -r requirements-test.txt

2. Create a .env file in the project root with your API keys:

COHERE_API_KEY=your_cohere_key
GEMINI_API_KEY=your_gemini_key
OPENAI_API_KEY=your_openai_key
OPENAI_BASE_URL=your_openai_base_url  # Optional custom endpoint
OLLAMA_BASE_URL=http://localhost:11434  # For local Ollama

3. For Ollama tests, ensure Ollama is running:

ollama serve
ollama pull nomic-embed-text:latest
ollama pull mxbai-embed-large:latest

Running Tests with Pytest

Run all tests:

pytest

Run with verbose output:

pytest -v

Run specific test file:

pytest tests/test_embeddings.py
pytest tests/test_generation.py -v

Run specific test class or method:

pytest tests/test_embeddings.py::TestCohereEmbeddings
pytest tests/test_embeddings.py::TestCohereEmbeddings::test_single_text_embedding

Run tests by marker:

# Run only integration tests
pytest -m integration

# Run tests excluding Ollama
pytest -m "not ollama"

# Run only Ollama tests
pytest -m ollama

Run with coverage:

pytest --cov=router tests/

Run tests in parallel:

# Install pytest-xdist first
pip install pytest-xdist
pytest -n auto

Test Coverage

• test_config.py: Tests configuration loading and API key management
• test_embeddings.py: Tests embedding APIs (Cohere, Gemini, Ollama)
• test_rerank.py: Tests Cohere reranking functionality
• test_generation.py: Tests text generation (OpenAI-compatible, Gemini, Cohere)

Test Markers

• @pytest.mark.integration: Marks integration tests that call real APIs
• @pytest.mark.asyncio: Marks async tests
• @pytest.mark.ollama: Marks tests that require Ollama to be running
• @pytest.mark.parametrize: Used for running tests with multiple input values

Notes

• These tests use real APIs and will consume API credits
• Ollama tests are marked with @pytest.mark.ollama and can be skipped
• All tests include both sync and async variants
• Tests verify functionality without checking exact output values
• The test suite is configured for asyncio_mode = auto in pytest.ini