5b484c9226
Refactored LLM provider infrastructure to support sustainable additions of new providers with both embedding and text generation capabilities.
## Major Changes
### Unified Provider Architecture (ADR-015)
- Created `nextcloud_mcp_server/providers/` with unified Provider ABC
- Providers now support optional capabilities (embeddings and/or generation)
- Auto-detection registry with priority: Bedrock → Ollama → Simple
- Backward compatible - existing code continues to work
### New Providers
- **BedrockProvider**: Full Amazon Bedrock integration
- Embeddings: Titan Embed, Cohere Embed models
- Generation: Claude, Llama, Titan Text, Mistral models
- Model-specific request/response handling
- AWS credential chain integration
- **OllamaProvider**: Migrated with both capabilities support
- **AnthropicProvider**: Moved from test code to production providers
- **SimpleProvider**: Migrated in-memory fallback provider
### Breaking Changes
None - full backward compatibility maintained:
- `embedding.get_embedding_service()` still works
- RAG evaluation tests updated to use unified providers
- All existing tests pass (127 unit tests)
### Testing
- Added 9 comprehensive Bedrock unit tests with mocked boto3
- All existing unit tests pass
- Type checking (ty) and linting (ruff) pass
- Verified backward compatibility
### Documentation
- `docs/ADR-015-unified-provider-architecture.md`: Comprehensive ADR
- `docs/bedrock-setup.md`: AWS setup guide with IAM permissions
- `CLAUDE.md`: Updated with provider architecture section
### Dependencies
- Added `boto3>=1.35.0` to dev dependencies (optional)
## Environment Variables
### Bedrock
- `AWS_REGION`: AWS region (e.g., "us-east-1")
- `BEDROCK_EMBEDDING_MODEL`: Model ID for embeddings
- `BEDROCK_GENERATION_MODEL`: Model ID for generation
- `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`: Optional credentials
### Ollama
- `OLLAMA_BASE_URL`: API URL
- `OLLAMA_EMBEDDING_MODEL`: Embedding model (default: "nomic-embed-text")
- `OLLAMA_GENERATION_MODEL`: Generation model
## AWS Bedrock Permissions Required
Minimal IAM policy:
```json
{
"Effect": "Allow",
"Action": ["bedrock:InvokeModel"],
"Resource": ["arn:aws:bedrock:*::foundation-model/*"]
}
```
See `docs/bedrock-setup.md` for detailed setup instructions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
127 lines
4.5 KiB
Python
127 lines
4.5 KiB
Python
"""Provider registry and factory for auto-detection and instantiation."""
|
|
|
|
import logging
|
|
import os
|
|
|
|
from .base import Provider
|
|
from .bedrock import BedrockProvider
|
|
from .ollama import OllamaProvider
|
|
from .simple import SimpleProvider
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
class ProviderRegistry:
|
|
"""
|
|
Registry for provider auto-detection and instantiation.
|
|
|
|
Checks environment variables in priority order and creates appropriate provider:
|
|
1. Bedrock (AWS_REGION + BEDROCK_*_MODEL)
|
|
2. Ollama (OLLAMA_BASE_URL)
|
|
3. Simple (fallback for testing/development)
|
|
"""
|
|
|
|
@staticmethod
|
|
def create_provider() -> Provider:
|
|
"""
|
|
Auto-detect and create provider based on environment variables.
|
|
|
|
Priority order:
|
|
1. Bedrock - if AWS_REGION or BEDROCK_EMBEDDING_MODEL is set
|
|
2. Ollama - if OLLAMA_BASE_URL is set
|
|
3. Simple - fallback for testing/development
|
|
|
|
Returns:
|
|
Provider instance
|
|
|
|
Environment Variables:
|
|
Bedrock:
|
|
- AWS_REGION: AWS region (e.g., "us-east-1")
|
|
- AWS_ACCESS_KEY_ID: AWS access key (optional, uses credential chain)
|
|
- AWS_SECRET_ACCESS_KEY: AWS secret key (optional)
|
|
- BEDROCK_EMBEDDING_MODEL: Model ID for embeddings (e.g., "amazon.titan-embed-text-v2:0")
|
|
- BEDROCK_GENERATION_MODEL: Model ID for text generation (e.g., "anthropic.claude-3-sonnet-20240229-v1:0")
|
|
|
|
Ollama:
|
|
- OLLAMA_BASE_URL: Ollama API base URL (e.g., "http://localhost:11434")
|
|
- OLLAMA_EMBEDDING_MODEL: Model for embeddings (default: "nomic-embed-text")
|
|
- OLLAMA_GENERATION_MODEL: Model for text generation (e.g., "llama3.2:1b")
|
|
- OLLAMA_VERIFY_SSL: Verify SSL certificates (default: "true")
|
|
|
|
Simple (no configuration needed, fallback):
|
|
- SIMPLE_EMBEDDING_DIMENSION: Embedding dimension (default: 384)
|
|
"""
|
|
# 1. Check for Bedrock
|
|
aws_region = os.getenv("AWS_REGION")
|
|
bedrock_embedding_model = os.getenv("BEDROCK_EMBEDDING_MODEL")
|
|
bedrock_generation_model = os.getenv("BEDROCK_GENERATION_MODEL")
|
|
|
|
if aws_region or bedrock_embedding_model or bedrock_generation_model:
|
|
logger.info(
|
|
f"Using Bedrock provider: region={aws_region}, "
|
|
f"embedding_model={bedrock_embedding_model}, "
|
|
f"generation_model={bedrock_generation_model}"
|
|
)
|
|
return BedrockProvider(
|
|
region_name=aws_region,
|
|
embedding_model=bedrock_embedding_model,
|
|
generation_model=bedrock_generation_model,
|
|
aws_access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
|
|
aws_secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
|
|
)
|
|
|
|
# 2. Check for Ollama
|
|
ollama_url = os.getenv("OLLAMA_BASE_URL")
|
|
if ollama_url:
|
|
embedding_model = os.getenv("OLLAMA_EMBEDDING_MODEL", "nomic-embed-text")
|
|
generation_model = os.getenv("OLLAMA_GENERATION_MODEL")
|
|
verify_ssl = os.getenv("OLLAMA_VERIFY_SSL", "true").lower() == "true"
|
|
|
|
logger.info(
|
|
f"Using Ollama provider: {ollama_url}, "
|
|
f"embedding_model={embedding_model}, "
|
|
f"generation_model={generation_model}"
|
|
)
|
|
return OllamaProvider(
|
|
base_url=ollama_url,
|
|
embedding_model=embedding_model,
|
|
generation_model=generation_model,
|
|
verify_ssl=verify_ssl,
|
|
)
|
|
|
|
# 3. Fallback to Simple provider for development/testing
|
|
dimension = int(os.getenv("SIMPLE_EMBEDDING_DIMENSION", "384"))
|
|
logger.warning(
|
|
"No provider configured (AWS_REGION, OLLAMA_BASE_URL not set). "
|
|
"Using SimpleProvider for testing/development. "
|
|
"For production, configure Bedrock or Ollama."
|
|
)
|
|
return SimpleProvider(dimension=dimension)
|
|
|
|
|
|
# Singleton instance
|
|
_provider: Provider | None = None
|
|
|
|
|
|
def get_provider() -> Provider:
|
|
"""
|
|
Get singleton provider instance.
|
|
|
|
Returns:
|
|
Global Provider instance (auto-detected on first call)
|
|
"""
|
|
global _provider
|
|
if _provider is None:
|
|
_provider = ProviderRegistry.create_provider()
|
|
return _provider
|
|
|
|
|
|
def reset_provider():
|
|
"""
|
|
Reset singleton provider instance.
|
|
|
|
Useful for testing or reconfiguration.
|
|
"""
|
|
global _provider
|
|
_provider = None
|