nextcloud-mcp-server/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Development Commands

Testing

The test suite is organized in layers for fast feedback:

# FAST FEEDBACK (recommended for development)
# Unit tests only - ~5 seconds
uv run pytest tests/unit/ -v

# Smoke tests - critical path validation - ~30-60 seconds
uv run pytest -m smoke -v

# INTEGRATION TESTS
# Integration tests without OAuth - ~2-3 minutes
uv run pytest -m "integration and not oauth" -v

# Full test suite - ~4-5 minutes
uv run pytest

# OAuth tests only (slowest, requires Playwright) - ~3 minutes
uv run pytest -m oauth -v

# COVERAGE
# Run tests with coverage
uv run pytest --cov

# LEGACY COMMANDS (still work)
# Run all integration tests
uv run pytest -m integration -v

# Skip integration tests
uv run pytest -m "not integration" -v

! Hint: If the tests are failing due to missing environment variables, then usually the correct .env has not been created or not correctly configured yet.

Load Testing

# Run benchmark with default settings (10 workers, 30 seconds)
uv run python -m tests.load.benchmark

# Quick test with custom concurrency and duration
uv run python -m tests.load.benchmark --concurrency 20 --duration 60

# Extended load test (50 workers for 5 minutes)
uv run python -m tests.load.benchmark -c 50 -d 300

# Export results to JSON for analysis
uv run python -m tests.load.benchmark -c 20 -d 60 --output results.json

# Test OAuth server on port 8001
uv run python -m tests.load.benchmark --url http://127.0.0.1:8001/mcp

# Verbose mode with detailed logging
uv run python -m tests.load.benchmark -c 10 -d 30 --verbose

Load Testing Features:

• Mixed workload simulating realistic MCP usage (40% reads, 20% writes, 15% search, 25% other operations)
• Real-time progress bar with live RPS and error counts
• Detailed metrics:
  • Throughput (requests/second)
  • Latency percentiles (p50, p90, p95, p99)
  • Per-operation breakdown
  • Error rates and types
• Automatic cleanup of test data
• JSON export for CI/CD integration
• Server health checks before starting

Understanding Results:

• Requests/Second (RPS): Higher is better. Expected baseline: 50-200 RPS for mixed workload
• Latency:
  • p50 (median): Should be <100ms for most operations
  • p95: Should be <500ms
  • p99: Should be <1000ms
• Error Rate: Should be <1% under normal load

Common Bottlenecks:

1. Nextcloud backend API response times (most common)
2. Database connection limits
3. HTTP client connection pooling
4. Network I/O between containers

Code Quality

# Format and lint code
uv run ruff check
uv run ruff format

# Type checking
# No explicit type checker configured - this is a Python project using ruff for linting

Running the Server

# Local development - load environment variables and run
export $(grep -v '^#' .env | xargs)
mcp run --transport sse nextcloud_mcp_server.app:mcp

# Docker development environment with Nextcloud instance
docker-compose up

# After code changes, rebuild and restart the appropriate MCP server container:
# For basic auth changes (most common) - uses admin credentials
docker-compose up --build -d mcp

# For OAuth changes - uses OAuth authentication with JWT tokens
docker-compose up --build -d mcp-oauth

# Build Docker image
docker build -t nextcloud-mcp-server .

Important: MCP Server Containers

• mcp (port 8000): Uses basic auth with admin credentials. Use this for most development and testing.
• mcp-oauth (port 8001): Uses OAuth authentication with JWT tokens. Use this when working on OAuth-specific features or tests.
  • JWT tokens are used for testing (faster validation, scopes embedded in token)
  • The server can handle both JWT and opaque tokens via the token verifier

Environment Setup

# Install dependencies
uv sync

# Install development dependencies
uv sync --group dev

Database Inspection

Docker Compose Database Credentials:

• Root user: root / password: password
• App user: nextcloud / password: password
• Database: nextcloud

Common Database Commands:

# Connect to database as root (most common for inspection)
docker compose exec db mariadb -u root -ppassword nextcloud

# Check OAuth clients
docker compose exec db mariadb -u root -ppassword nextcloud -e "SELECT id, name, token_type FROM oc_oidc_clients ORDER BY id DESC LIMIT 10;"

# Check OAuth client scopes
docker compose exec db mariadb -u root -ppassword nextcloud -e "SELECT c.id, c.name, s.scope FROM oc_oidc_clients c LEFT JOIN oc_oidc_client_scopes s ON c.id = s.client_id WHERE c.name LIKE '%MCP%';"

# Check OAuth access tokens
docker compose exec db mariadb -u root -ppassword nextcloud -e "SELECT id, client_id, user_id, created_at FROM oc_oidc_access_tokens ORDER BY created_at DESC LIMIT 10;"

Important Tables:

• oc_oidc_clients - OAuth client registrations (DCR clients)
• oc_oidc_client_scopes - Client allowed scopes
• oc_oidc_access_tokens - Issued access tokens
• oc_oidc_authorization_codes - Authorization codes
• oc_oidc_registration_tokens - RFC 7592 registration tokens for client management
• oc_oidc_redirect_uris - Redirect URIs for each client

Architecture Overview

This is a Python MCP (Model Context Protocol) server that provides LLM integration with Nextcloud. The architecture follows a layered pattern:

Core Components

• nextcloud_mcp_server/app.py - Main MCP server entry point using FastMCP framework
• nextcloud_mcp_server/client/ - HTTP client implementations for different Nextcloud APIs
• nextcloud_mcp_server/server/ - MCP tool/resource definitions that expose client functionality
• nextcloud_mcp_server/controllers/ - Business logic controllers (e.g., notes search)

Client Architecture

• NextcloudClient - Main orchestrating client that manages all app-specific clients
• BaseNextcloudClient - Abstract base class providing common HTTP functionality and retry logic
• App-specific clients: NotesClient, CalendarClient, ContactsClient, TablesClient, WebDAVClient

Server Integration

Each Nextcloud app has a corresponding server module that:

1. Defines MCP tools using @mcp.tool() decorators
2. Defines MCP resources using @mcp.resource() decorators
3. Uses the context pattern to access the NextcloudClient instance

Supported Nextcloud Apps

• Notes - Full CRUD operations and search
• Calendar - CalDAV integration with events, recurring events, attendees, and tasks (VTODO)
  • Calendar Operations: List, create, delete calendars
  • Event Operations: Full CRUD, recurring events, attendees, reminders, bulk operations
  • Task Operations (VTODO): Full CRUD for CalDAV tasks with:
    • Status tracking (NEEDS-ACTION, IN-PROCESS, COMPLETED, CANCELLED)
    • Priority levels (0-9, 1=highest, 9=lowest)
    • Due dates, start dates, completion tracking
    • Percent complete (0-100%)
    • Categories and filtering
    • Search across all calendars
  • Note: Calendar implementation uses caldav library's AsyncDavClient
• Contacts - CardDAV integration with address book operations
• Tables - Row-level operations on Nextcloud Tables
• WebDAV - Complete file system access

Key Patterns

1. Environment-based configuration - Uses NextcloudClient.from_env() to load credentials from environment variables
2. Async/await throughout - All operations are async using httpx
3. Retry logic - @retry_on_429 decorator handles rate limiting
4. Context injection - MCP context provides access to the authenticated client instance
5. Modular design - Each Nextcloud app is isolated in its own client/server pair

MCP Response Patterns

CRITICAL: Never return raw List[Dict] from MCP tools - always wrap in Pydantic response models

FastMCP serialization issue: raw lists get mangled into dicts with numeric string keys.

Pattern:

1. Client methods return List[Dict] (raw data)
2. MCP tools convert to Pydantic models and wrap in response object
3. Response models inherit from BaseResponse, include results field + metadata

Reference implementations:

• SearchNotesResponse in nextcloud_mcp_server/models/notes.py:80
• SearchFilesResponse in nextcloud_mcp_server/models/webdav.py:113
• Tool examples: nextcloud_mcp_server/server/{notes,webdav}.py

Testing: Extract data["results"] from MCP responses, not data directly.

Testing Structure

The test suite follows a layered architecture for fast feedback:

tests/
├── unit/                    # Fast unit tests (~5s total)
│   ├── test_scope_decorator.py
│   └── test_response_models.py
├── smoke/                   # Critical path tests (~30-60s)
│   └── test_smoke.py
├── integration/
│   ├── client/             # Direct API layer tests
│   │   ├── notes/
│   │   ├── calendar/
│   │   └── ...
│   └── server/             # MCP tool layer tests
│       ├── oauth/          # OAuth-specific tests (slow, ~3min)
│       │   ├── test_oauth_core.py
│       │   ├── test_scope_authorization.py
│       │   └── ...
│       ├── test_mcp.py
│       └── ...
└── load/                   # Performance tests

Test Markers:

• @pytest.mark.unit - Fast unit tests with mocked dependencies
• @pytest.mark.integration - Integration tests requiring Docker containers
• @pytest.mark.oauth - OAuth tests requiring Playwright (slowest)
• @pytest.mark.smoke - Critical path smoke tests

Fixtures in tests/conftest.py - Shared test setup and utilities

• Important: Integration tests run against live Docker containers. After making code changes:
  • For basic auth tests: rebuild with docker-compose up --build -d mcp
  • For OAuth tests: rebuild with docker-compose up --build -d mcp-oauth

Testing Best Practices

• MANDATORY: Always run tests after implementing features or fixing bugs
  • Run tests to completion before considering any task complete
  • If tests require modifications to pass, ask for permission before proceeding
  • Rebuild the correct container after code changes:
    • For basic auth tests (most common): docker-compose up --build -d mcp
    • For OAuth tests: docker-compose up --build -d mcp-oauth
• Use existing fixtures from tests/conftest.py to avoid duplicate setup work:
  • nc_mcp_client - MCP client session for tool/resource testing (uses mcp container)
  • nc_mcp_oauth_client - MCP client session for OAuth testing (uses mcp-oauth container)
  • nc_client - Direct NextcloudClient for setup/cleanup operations
  • temporary_note - Creates and cleans up test notes automatically
  • temporary_addressbook - Creates and cleans up test address books
  • temporary_contact - Creates and cleans up test contacts
• Test specific functionality after changes:
  • For Notes changes: uv run pytest tests/server/test_mcp.py -k "notes" -v
  • For specific API changes: uv run pytest tests/client/notes/test_notes_api.py -v
  • For OAuth changes: uv run pytest tests/server/test_oauth*.py -v (remember to rebuild mcp-oauth container)
• Avoid creating standalone test scripts - use pytest with proper fixtures instead

Writing Mocked Unit Tests

For client-layer tests that verify response parsing logic, use mocked HTTP responses instead of real network calls:

Pattern:

import httpx
import pytest
from nextcloud_mcp_server.client.notes import NotesClient
from tests.conftest import create_mock_note_response

async def test_notes_api_get_note(mocker):
    """Test that get_note correctly parses the API response."""
    # Create mock response using helper functions
    mock_response = create_mock_note_response(
        note_id=123,
        title="Test Note",
        content="Test content",
        category="Test",
        etag="abc123",
    )

    # Mock the _make_request method
    mock_client = mocker.AsyncMock(spec=httpx.AsyncClient)
    mock_make_request = mocker.patch.object(
        NotesClient, "_make_request", return_value=mock_response
    )

    # Create client and test
    client = NotesClient(mock_client, "testuser")
    note = await client.get_note(note_id=123)

    # Verify the response was parsed correctly
    assert note["id"] == 123
    assert note["title"] == "Test Note"
    # Verify the correct API endpoint was called
    mock_make_request.assert_called_once_with("GET", "/apps/notes/api/v1/notes/123")

Mock Response Helpers in tests/conftest.py:

• create_mock_response() - Generic HTTP response builder
• create_mock_note_response() - Pre-configured note response
• create_mock_error_response() - Error responses (404, 412, etc.)

Benefits:

• ⚡ Fast execution (~0.1s vs minutes for integration tests)
• 🔒 No Docker dependency
• 🎯 Tests focus on response parsing logic
• ♻️ Repeatable and deterministic

When to use:

• Testing client methods that parse JSON responses
• Testing error handling (404, 412, etc.)
• Testing request parameter building

When NOT to use (keep as integration tests):

• Complex protocol interactions (CalDAV, CardDAV, WebDAV)
• Multi-component workflows (Notes + WebDAV attachments)
• OAuth flows
• End-to-end MCP tool testing

Reference Implementation:

• See tests/client/notes/test_notes_api.py for complete examples
• Mark unit tests with pytestmark = pytest.mark.unit
• Run with: uv run pytest tests/unit/ tests/client/notes/test_notes_api.py -v

OAuth/OIDC Testing

OAuth integration tests use automated Playwright browser automation to complete the OAuth flow programmatically.

OAuth Testing Setup:

• Main fixtures: nc_oauth_client, nc_mcp_oauth_client - Use Playwright automation
• Shared OAuth Client: All test users authenticate using a single OAuth client
  • Created fresh for each test session via Dynamic Client Registration (DCR)
  • Matches production MCP server behavior (one client, multiple user tokens)
  • Each user gets their own unique access token
  • Automatic cleanup: Client is registered at session start, deleted at session end (RFC 7592)
  • Implementation: shared_oauth_client_credentials fixture in tests/conftest.py
  • Note: Client deletion may fail due to Nextcloud middleware (logged as warning). This doesn't affect tests.
• Available fixtures: playwright_oauth_token, nc_oauth_client, nc_mcp_oauth_client
• Multi-user fixtures: alice_oauth_token, bob_oauth_token, charlie_oauth_token, diana_oauth_token
• Requirements: NEXTCLOUD_HOST, NEXTCLOUD_USERNAME, NEXTCLOUD_PASSWORD environment variables
• Uses pytest-playwright-asyncio for async Playwright fixtures
• Playwright configuration: Use pytest CLI args like --browser firefox --headed to customize
• Install browsers: uv run playwright install firefox (or chromium, webkit)

Example Commands:

# Run all OAuth tests with Playwright automation using Firefox
uv run pytest tests/server/oauth/ --browser firefox -v

# Run specific OAuth test file with visible browser for debugging
uv run pytest tests/server/oauth/test_oauth_core.py --browser firefox --headed -v

# Run with Chromium (default) - use -m oauth marker for all OAuth tests
uv run pytest -m oauth -v

Test Environment:

• Two MCP server containers are available:
  • mcp (port 8000): Uses basic auth with admin credentials - for most testing
  • mcp-oauth (port 8001): Uses OAuth authentication - for OAuth-specific testing
• Start OAuth MCP server: docker-compose up --build -d mcp-oauth
• Important: When working on OAuth functionality, always rebuild mcp-oauth container, not mcp

CI/CD Notes:

• Playwright tests run in CI/CD environments
• Use Firefox browser in CI: --browser firefox (Chromium may have issues with localhost redirects)

Keycloak OAuth/OIDC Testing (ADR-002 Integration)

The MCP server supports using Keycloak as an external OAuth/OIDC identity provider instead of Nextcloud's built-in OIDC app. This validates the ADR-002 architecture for background jobs and external identity providers.

Architecture:

MCP Client → Keycloak (OAuth) → MCP Server → Nextcloud user_oidc (validates token) → APIs

Key Benefits:

• ✅ No admin credentials needed - All API access uses user's Keycloak token
• ✅ External identity provider - Demonstrates integration with enterprise IdPs
• ✅ ADR-002 validation - Tests offline_access and refresh token patterns
• ✅ User provisioning - Nextcloud automatically provisions users from Keycloak

Setup and Testing:

# 1. Start Keycloak and MCP server with Keycloak OAuth
docker-compose up -d keycloak app mcp-keycloak

# 2. Verify Keycloak realm is available
curl http://localhost:8888/realms/nextcloud-mcp/.well-known/openid-configuration

# 3. Verify user_oidc provider is configured
docker compose exec app php occ user_oidc:provider keycloak

# 4. Generate encryption key for refresh token storage (optional, for ADR-002 Tier 1)
python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
# Set in environment: export TOKEN_ENCRYPTION_KEY='<key>'

# 5. Test OAuth flow manually
# Get token from Keycloak:
TOKEN=$(curl -s -X POST "http://localhost:8888/realms/nextcloud-mcp/protocol/openid-connect/token" \
  -d "grant_type=password" \
  -d "client_id=mcp-client" \
  -d "client_secret=mcp-secret-change-in-production" \
  -d "username=admin" \
  -d "password=admin" \
  -d "scope=openid profile email offline_access" | jq -r .access_token)

# Use token with Nextcloud API (validated by user_oidc):
curl -H "Authorization: Bearer $TOKEN" http://localhost:8080/ocs/v2.php/cloud/capabilities

# 6. Connect MCP client
# Point client to: http://localhost:8002
# Complete OAuth flow using Keycloak credentials: admin/admin

Three MCP Server Containers:

• mcp (port 8000): Basic auth with admin credentials
• mcp-oauth (port 8001): Nextcloud OIDC provider (JWT tokens)
• mcp-keycloak (port 8002): Keycloak OIDC provider (external IdP)

Keycloak Configuration:

• Realm: nextcloud-mcp (auto-imported from keycloak/realm-export.json)
• Client: mcp-client (pre-configured with PKCE, offline_access)
• Admin user: admin/admin (created in realm export)
• Redirect URIs: http://localhost:*/callback, http://127.0.0.1:*/callback

Environment Variables (Generic OIDC - works with any provider):

# Generic OIDC configuration (provider-agnostic)
OIDC_DISCOVERY_URL=http://keycloak:8080/realms/nextcloud-mcp/.well-known/openid-configuration
OIDC_CLIENT_ID=nextcloud-mcp-server        # OAuth client ID
OIDC_CLIENT_SECRET=mcp-secret-...          # OAuth client secret

# Nextcloud API configuration
NEXTCLOUD_HOST=http://app:80               # Nextcloud API (token validation in external IdP mode)

# Refresh tokens and token exchange (ADR-002)
ENABLE_OFFLINE_ACCESS=true                 # Enable refresh tokens
TOKEN_ENCRYPTION_KEY=<fernet-key>          # Encrypt refresh tokens
TOKEN_STORAGE_DB=/app/data/tokens.db       # Token storage path

# OAuth scopes (optional - uses defaults if not specified)
NEXTCLOUD_OIDC_SCOPES=openid profile email offline_access notes:read notes:write ...

Provider Mode Detection:

• External IdP mode: If OIDC_DISCOVERY_URL issuer ≠ NEXTCLOUD_HOST → Uses external provider (Keycloak, Auth0, Okta, etc.)
• Integrated mode: If OIDC_DISCOVERY_URL not set or issuer = NEXTCLOUD_HOST → Uses Nextcloud OIDC app

Nextcloud user_oidc Configuration: The user_oidc app is automatically configured by app-hooks/post-installation/15-setup-keycloak-provider.sh:

# Configured with:
--check-bearer=1          # Validate bearer tokens
--bearer-provisioning=1   # Auto-provision users
--unique-uid=1            # Hash user IDs
--scope="openid profile email offline_access"

Troubleshooting:

# Check Keycloak is running
docker-compose ps keycloak
docker-compose logs keycloak

# Check user_oidc provider configuration
docker compose exec app php occ user_oidc:provider keycloak

# Check MCP server logs
docker-compose logs -f mcp-keycloak

# Check Nextcloud logs for token validation
docker compose exec app tail -f /var/www/html/data/nextcloud.log

# Verify Keycloak is accessible from Nextcloud container
docker compose exec app curl http://keycloak:8080/realms/nextcloud-mcp/.well-known/openid-configuration

ADR-002 Offline Access Testing: The Keycloak integration enables testing ADR-002 Tier 1 (offline access with refresh tokens):

1. Refresh token storage: Tokens stored encrypted in SQLite (/app/data/tokens.db)
2. Token refresh: Access tokens refreshed automatically when expired
3. Background workers: Can access APIs using stored refresh tokens
4. No admin credentials: All operations use user's OAuth tokens

See docs/ADR-002-vector-sync-authentication.md for architectural details.

Audience Validation: Tokens include aud: ["mcp-server", "nextcloud"] claims for proper security:

• MCP server validates tokens are intended for it
• Nextcloud validates tokens include it as audience
• Prevents token misuse across services

See docs/audience-validation-setup.md for configuration details and docs/keycloak-multi-client-validation.md for realm-level validation behavior.

Configuration Files

• pyproject.toml - Python project configuration using uv for dependency management
• .env (from env.sample) - Environment variables for Nextcloud connection
• docker-compose.yml - Complete development environment with Nextcloud + database

Integration testing with docker

Nextcloud

• The app container is running nextcloud.
• Use docker compose exec app php occ ... to get a list of available commands

Mariadb

• The db container is running mariadb
• Use docker compose exec db mariadb -u [user] -p [password] [database] to execute queries. Check the docker-compose file for credentials