feat: Add comprehensive Python testing infrastructure with Poetry

[llbbl]

Add Python Testing Infrastructure

Summary

This PR sets up a comprehensive testing infrastructure for the QANet project using Poetry as the package manager and pytest as the testing framework. The setup provides a solid foundation for writing and running tests with proper organization, fixtures, and coverage reporting.

Changes Made

Package Management

• Poetry Setup: Created pyproject.toml with Poetry configuration as the primary package manager
• Dependency Migration: Migrated existing dependencies from requirements.txt to Poetry
• Development Dependencies: Added pytest, pytest-cov, and pytest-mock as development dependencies

Testing Configuration

• pytest Configuration: Configured pytest in pyproject.toml with:

  • Test discovery patterns for flexible test file naming
  • Custom markers for test categorization (unit, integration, slow)
  • Strict mode and verbose output options

• Coverage Settings: Set up coverage reporting with:

  • Multiple output formats (terminal, HTML, XML)
  • 80% coverage threshold (configurable)
  • Proper exclusions for test files and common patterns

Project Structure

tests/
├── __init__.py
├── conftest.py          # Shared fixtures and configuration
├── test_setup_validation.py  # Infrastructure validation tests
├── unit/
│   └── __init__.py
└── integration/
    └── __init__.py

Fixtures and Utilities

Created comprehensive shared fixtures in conftest.py:

• temp_dir: Temporary directory for test files
• mock_config: Mock configuration for testing
• sample_data: Sample data for NLP/QA testing
• mock_model_config: Mock model configuration
• reset_environment: Environment variable reset
• capture_logs: Log capture utility

Additional Files

• .gitignore: Updated with testing artifacts and Poetry entries
• run-tests: Convenience script for running tests
• pytest.ini: Additional pytest configuration for compatibility

Usage Instructions

Install Dependencies

poetry install

Run Tests

# Run all tests
poetry run pytest

# Run with coverage
poetry run pytest --cov

# Run specific test categories
poetry run pytest -m unit
poetry run pytest -m integration
poetry run pytest -m "not slow"

# Use the convenience script
./run-tests

# Run validation tests
poetry run pytest tests/test_setup_validation.py

Writing Tests

1. Place unit tests in tests/unit/
2. Place integration tests in tests/integration/
3. Use appropriate markers: @pytest.mark.unit, @pytest.mark.integration, @pytest.mark.slow
4. Leverage shared fixtures from conftest.py

Notes

• The testing infrastructure is ready for immediate use
• No actual unit tests for the codebase were written - only infrastructure setup
• Coverage is configured but not enforced by default (use --cov flag)
• The spacy dependency was updated from 2.0.9 to ^2.3.0 for compatibility
• Poetry lock file will be generated on first install