brandon/nextcloud-mcp-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
828 Commits 193 Branches 339 Tags
d16bcdcfbbdc0aa5612a453fd9b5c5bc649e1bb2
Commit Graph

37 Commits

Author SHA1 Message Date
Chris Coutinho babd60e08b feat: Implement ADR-004 Hybrid Flow with comprehensive integration tests 
Implement the ADR-004 Hybrid Flow OAuth pattern where the MCP server
intercepts the OAuth callback to obtain master refresh tokens while
maintaining PKCE security for clients.

## Implementation

### OAuth Routes (ADR-004 Hybrid Flow)
- Add `/oauth/authorize` endpoint: Intercepts client OAuth initiation
- Add `/oauth/callback` endpoint: Receives IdP callback, stores master token
- Add `/oauth/token` endpoint: Exchanges MCP code for client access token
- Implement PKCE code challenge/verifier validation
- Store OAuth sessions with state/challenge correlation

### MCP Server Integration
- Update `setup_oauth_config()` to return client_id and client_secret
- Initialize OAuth context in Starlette lifespan for login routes
- Add OAuth session storage to RefreshTokenStorage
- Configure authlib dependency for OAuth flow management

### Integration Tests
- Create `test_adr004_hybrid_flow.py` with Playwright automation
- Add `adr004_hybrid_flow_mcp_client` session-scoped fixture
- Test MCP session establishment with hybrid flow token
- Test tool execution using stored refresh tokens (on-behalf-of pattern)
- Test persistent access across multiple operations
- All tests passing: ✅ 3 passed in 8.82s

### Documentation
- Update ADR-004 with comprehensive Testing section
- Add integration test commands and coverage details
- Document test implementation and verification steps
- Create TESTING_INSTRUCTIONS.md for manual and automated testing
- Include manual test scripts for reference/debugging

## What This Enables

✅ PKCE code challenge/verifier flow
✅ MCP server intercepts OAuth callback and stores master refresh token
✅ Client receives MCP access token (not master token)
✅ MCP session establishment with hybrid flow token
✅ Tool execution using stored refresh tokens (on-behalf-of pattern)
✅ Multiple operations without re-authentication
✅ Proper token isolation (client never sees master token)

## Testing

Run ADR-004 integration tests:
```bash
uv run pytest tests/server/oauth/test_adr004_hybrid_flow.py --browser firefox -v
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-03 02:18:30 +01:00
Chris Coutinho 849c67c32a fix: Complete Keycloak external IdP integration with all tests passing 
This commit completes the Keycloak external IdP integration for the MCP
server, implementing ADR-002 Tier 2 (External Identity Provider) with
full Bearer token authentication support.

Key Changes:
1. **Keycloak backchannel-dynamic configuration**
   - Added --hostname-strict=false and --hostname-backchannel-dynamic=true
   - Allows external issuer (localhost:8888) with internal endpoints (keycloak:8080)
   - Solves Docker networking issue where containers can't reach localhost

2. **CORSMiddleware Bearer token patch**
   - Created app-hooks/patches/cors-bearer-token.patch from upstream commit 8fb5e77db82
   - Allows Bearer tokens to bypass CORS/CSRF checks (stateless authentication)
   - Applied via post-installation hook 20-apply-cors-bearer-token-patch.sh
   - Enables app-specific APIs (Notes, Calendar, etc.) to work with Bearer tokens

3. **Patch organization**
   - Moved patches to app-hooks/patches/ directory
   - Updated docker-compose.yml to mount entire app-hooks directory
   - Consolidated patch management for better maintainability

4. **Test improvements**
   - All 11 Keycloak integration tests passing
   - Tests validate OAuth token acquisition, MCP connectivity, token validation,
     tool execution, token persistence, user provisioning, scope filtering,
     and error handling

Architecture:
- Keycloak acts as external OAuth/OIDC identity provider
- MCP server uses Keycloak tokens to access Nextcloud APIs
- Nextcloud user_oidc app validates Bearer tokens from Keycloak
- No admin credentials needed - all API access uses user's OAuth tokens

Cache Note:
- Discovery and JWKS caches must be cleared when switching Keycloak configurations
- Use: docker compose exec redis redis-cli DEL "<cache-key>"
- Or: docker compose exec app php occ user_oidc:provider keycloak --clientid nextcloud

Related:
- ADR-002: Vector sync background jobs authentication
- Validates external IdP integration pattern
- Demonstrates offline_access with refresh tokens (Tier 1 & 2)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-02 22:03:20 +01:00
Chris Coutinho 6117aaaed3 fix: Complete Keycloak external IdP integration with all tests passing 
This commit completes the Keycloak external identity provider integration,
implementing the ADR-002 architecture where Keycloak acts as an external
OAuth/OIDC provider and Nextcloud validates tokens via the user_oidc app.

Architecture:
  MCP Client → Keycloak (OAuth) → MCP Server → Nextcloud user_oidc → APIs

Key Fixes:

1. Keycloak JWT token configuration
   - Added 'sub' claim protocol mapper to realm-export.json
   - Updated token_verifier.py to accept both 'sub' and 'preferred_username'
   - Ensures tokens contain required OIDC claims

2. Keycloak hostname configuration for Docker networking
   - Implemented --hostname-backchannel-dynamic=true in docker-compose.yml
   - External clients use localhost:8888 (public)
   - Internal services use keycloak:8080 (Docker network)
   - Same issuer (localhost:8888) everywhere for token consistency
   - Restored frontendUrl in realm attributes

3. MCP server provider mode detection
   - Fixed URL normalization to handle port differences (http://app vs http://app:80)
   - Correctly distinguishes integrated mode vs external IdP mode
   - Removes explicit default ports (80 for HTTP, 443 for HTTPS)

4. Nextcloud SSRF protection configuration
   - Added allow_local_remote_servers=true to user_oidc install script
   - Enables Nextcloud to fetch JWKS from internal Keycloak container
   - Required for external IdP token validation

5. OAuth lifespan cleanup
   - Fixed RefreshTokenStorage close() error (uses context managers)
   - Added safe cleanup for oauth_client with hasattr check
   - Prevents session crash on shutdown

6. Test suite fixes
   - Fixed test_user_auto_provisioning to reflect actual behavior
   - Fixed test_scope_filtering_with_keycloak tool name (nc_webdav_write_file)
   - Updated test_keycloak_oauth_client_credentials_discovery for hostname config
   - All 11 Keycloak external IdP tests now passing

Testing:
  ✅ All 11 tests in test_keycloak_external_idp.py passing
  ✅ OAuth token acquisition via Playwright automation
  ✅ Token validation through Nextcloud user_oidc app
  ✅ Write operations (Notes create, Calendar create, File upload)
  ✅ Read operations (search, list, get)
  ✅ Token persistence across multiple operations
  ✅ User authentication and bearer token validation
  ✅ Scope-based tool filtering
  ✅ Error handling for invalid operations

Implementation validates:
  - ADR-002 external identity provider architecture
  - No admin credentials needed in MCP server
  - Centralized identity management via Keycloak
  - Standards-based OAuth 2.0 / OIDC integration
  - User auto-provisioning from IdP claims

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-02 22:03:20 +01:00
Chris Coutinho 403f8be429 feat: Add Keycloak external IdP integration with custom scopes 
Add comprehensive support for using Keycloak as an external identity
provider with Nextcloud custom scopes. This enables testing of ADR-002
external IdP integration patterns.

**Keycloak Realm Configuration:**
- Add frontendUrl attribute to issue tokens with public issuer URL
- Define 18 Nextcloud custom client scopes (notes:read/write,
  calendar:read/write, contacts:read/write, cookbook:read/write,
  deck:read/write, tables:read/write, files:read/write,
  sharing:read/write, todo:read/write)
- Add all custom scopes to nextcloud-mcp-server client optional scopes
- Scopes include consent screen text for user-friendly OAuth flow

**MCP Server Configuration:**
- Add OIDC_JWKS_URI environment variable support
- Implement JWKS URI override logic for Docker networking
- Update NEXTCLOUD_PUBLIC_ISSUER_URL to include full realm path
- Enable MCP server to fetch JWKS from internal Docker network

**Test Infrastructure:**
- Add keycloak_oauth_client_credentials fixture (session-scoped)
- Add keycloak_oauth_token fixture with Playwright automation
- Implement PKCE (S256) support for Keycloak OAuth flow
- Add nc_mcp_keycloak_client fixture for MCP testing
- Create comprehensive test suite in test_keycloak_external_idp.py

**Tests Created:**
- test_keycloak_oauth_token_acquisition: Token acquisition via Playwright
- test_keycloak_oauth_client_credentials_discovery: OIDC discovery
- test_mcp_client_connects_to_keycloak_server: MCP connectivity
- test_external_idp_server_initialization: Server auto-detection
- test_external_idp_token_validation: Token validation flow
- test_tools_work_with_keycloak_token: End-to-end tool execution
- test_keycloak_token_persistence: Multi-operation token reuse
- test_user_auto_provisioning: Nextcloud user provisioning
- test_scope_filtering_with_keycloak: Scope-based tool filtering
- test_keycloak_error_handling: Error handling
- test_external_idp_architecture: Architecture documentation

**Current Status:**
- ✅ Keycloak realm configuration complete
- ✅ Custom scopes defined and available
- ✅ OAuth token acquisition working (1 test passing)
- ⚠️  Token validation needs additional work (external IdP userinfo)

**Files Modified:**
- keycloak/realm-export.json: Realm configuration with scopes
- tests/conftest.py: Keycloak OAuth fixtures (+285 lines)
- tests/server/oauth/test_keycloak_external_idp.py: New test suite
- docker-compose.yml: OIDC_JWKS_URI and issuer configuration
- nextcloud_mcp_server/app.py: JWKS URI override logic

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-02 22:03:20 +01:00
Chris Coutinho 2a1274d8a8 refactor: Unify OAuth configuration to be provider-agnostic 
Replace provider-specific environment variables (OAUTH_PROVIDER, KEYCLOAK_*)
with generic OIDC_* variables that work with any OIDC-compliant provider.

**Key Changes:**
- Auto-detect provider mode from OIDC_DISCOVERY_URL issuer
  - External IdP mode: issuer ≠ NEXTCLOUD_HOST (Keycloak, Auth0, Okta, etc.)
  - Integrated mode: issuer = NEXTCLOUD_HOST (Nextcloud OIDC app)
- Unified OIDC discovery flow (single code path)
- Generic client credential loading (static or DCR)
- Simplified docker-compose.yml environment variables

**Environment Variables:**
BEFORE:
  OAUTH_PROVIDER=keycloak
  KEYCLOAK_URL=http://keycloak:8080
  KEYCLOAK_REALM=nextcloud-mcp
  KEYCLOAK_CLIENT_ID=...
  KEYCLOAK_DISCOVERY_URL=...

AFTER:
  OIDC_DISCOVERY_URL=http://keycloak:8080/realms/nextcloud-mcp/.well-known/...
  OIDC_CLIENT_ID=nextcloud-mcp-server
  OIDC_CLIENT_SECRET=...

**Benefits:**
- Works with any OIDC provider without code changes
- No manual provider selection needed
- Cleaner environment variable naming
- Reduced code duplication (~150 lines removed)

**Testing:**
✅ mcp-keycloak auto-detects external IdP mode
✅ Token exchange test passes with generic config
✅ Backward compatible - integrated mode still works

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-02 22:03:20 +01:00
Chris Coutinho e331544cee feat: Implement RFC 8693 token exchange for Keycloak (ADR-002 Tier 2) 
Implements OAuth 2.0 Token Exchange (RFC 8693) enabling the MCP server to
exchange service account tokens for user-scoped tokens. This provides an
alternative to refresh tokens for background operations.

**Core Implementation:**
- Added `get_service_account_token()` method to KeycloakOAuthClient for
  client_credentials grant
- Added `exchange_token_for_user()` method implementing RFC 8693 token exchange
- Fixed Fernet encryption key handling in RefreshTokenStorage (was incorrectly
  base64 decoding already-encoded keys)
- Updated OAuth configuration to support offline_access scope and refresh token
  storage infrastructure

**Keycloak Configuration:**
- Enabled `serviceAccountsEnabled` in realm-export.json
- Added `token.exchange.grant.enabled` attribute
- Added `client.token.exchange.standard.enabled` attribute (required for
  Keycloak 26.2+ Standard Token Exchange V2)
- Fresh Keycloak imports now correctly enable token exchange

**Docker Compose:**
- Added TOKEN_ENCRYPTION_KEY and ENABLE_OFFLINE_ACCESS environment variables
- Created oauth-tokens volume for refresh token storage
- Configured both mcp-oauth and mcp-keycloak services

**Testing & Documentation:**
- Added tests/manual/test_token_exchange.py - Validates complete RFC 8693 flow
- Added tests/manual/test_nextcloud_impersonate.py - Documents session-based
  impersonation limitations
- Added docs/oauth-impersonation-findings.md - Comprehensive investigation
  findings and resolution documentation

**Verified Working:**
✅ Service account token acquisition (client_credentials grant)
✅ RFC 8693 token exchange for internal-to-internal tokens
✅ Exchanged tokens validate with Nextcloud APIs
✅ Keycloak 26.4.2 Standard Token Exchange V2 support

**Known Limitations:**
- User impersonation (requested_subject) requires Keycloak Legacy V1 with
  preview features
- Cross-client token exchange limited to same realm
- Refresh token storage infrastructure ready but unused (MCP protocol limitation)

Dependencies: aiosqlite>=0.20.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-02 22:03:19 +01:00
Chris Coutinho f34366a260 feat: Add Keycloak OAuth provider support with refresh token storage 
Implements Keycloak as an external OIDC provider following ADR-002
architecture for background job authentication using offline_access.

## Features

- Keycloak OAuth provider with PKCE and offline_access support
- Refresh token storage with Fernet encryption
- Token verifier for both JWT and opaque tokens
- Multi-client validation (realm-level trust)
- Sample configuration for Keycloak integration

## Implementation

### OAuth Provider (keycloak_oauth.py)
- Authorization Code Flow with PKCE
- Refresh token exchange
- OIDC discovery endpoint support
- Token validation with JWKS

### Token Storage (refresh_token_storage.py)
- Encrypted storage using Fernet symmetric encryption
- SQLite backend for persistence
- Token rotation support
- Per-user token management

### Token Verifier Updates
- Support both JWT (self-encoded) and opaque tokens
- JWKS-based JWT signature verification
- Introspection endpoint fallback for opaque tokens
- Scope extraction from both token types

### Configuration
- .env.keycloak.sample: Example configuration with Keycloak URLs
- docs/keycloak-multi-client-validation.md: Realm-level validation documentation
- app-hooks/post-installation/10-install-user_oidc-app.sh: Updated dependencies

## Architecture Notes

- MCP Server is a protected resource (requires OAuth)
- MCP Client initiates OAuth flow and shares refresh tokens
- Refresh tokens enable background operations without admin credentials
- Supports future token exchange delegation when Keycloak implements it

## References

- ADR-002: Vector Database Background Sync Authentication
- RFC 6749: OAuth 2.0 (offline_access, refresh tokens)
- RFC 7517: JSON Web Key (JWK)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-02 22:03:19 +01:00
Chris Coutinho 562c102711 feat(server): Add /live & /health endpoints 2025-10-29 10:29:30 +01:00
Chris Coutinho 415b1c901b docs: Parse available scopes from registered tools and update docs 2025-10-25 21:16:40 +02:00
Chris Coutinho a36038422b feat: Add text processing background worker for telling client about progress 2025-10-25 19:52:45 +02:00
Chris Coutinho 2147fc1696 refactor: Transform document parsing into pluggable processor architecture 
Refactors PR #190's hardcoded Unstructured.io integration into a flexible,
extensible plugin system supporting multiple text extraction engines.

- **`DocumentProcessor` ABC**: Abstract interface for all processors
- **`ProcessorRegistry`**: Central registry for discovery and routing
- **`ProcessingResult`**: Standardized output format across processors

- **`UnstructuredProcessor`**: Refactored from `UnstructuredClient`
- **`TesseractProcessor`**: Local OCR for images (lightweight alternative)
- **`CustomHTTPProcessor`**: Generic wrapper for custom HTTP APIs

- New `get_document_processor_config()` returns structured config
- Supports enabling/disabling individual processors
- Per-processor configuration via environment variables
- **Breaking Change**: `ENABLE_UNSTRUCTURED_PARSING` replaced with:
  - `ENABLE_DOCUMENT_PROCESSING=true/false` (master switch)
  - `ENABLE_UNSTRUCTURED=true/false` (per-processor)
  - `ENABLE_TESSERACT=true/false`
  - `ENABLE_CUSTOM_PROCESSOR=true/false`

- `parse_document()` now uses `ProcessorRegistry`
- Auto-selects appropriate processor based on MIME type
- Processor priority system (Unstructured=10, Tesseract=5, Custom=1)

- `initialize_document_processors()` registers processors at startup
- Integrated into both BasicAuth and OAuth lifespans
- Graceful degradation if processors fail to initialize

```env
ENABLE_DOCUMENT_PROCESSING=false

ENABLE_UNSTRUCTURED=false
UNSTRUCTURED_API_URL=http://unstructured:8000
UNSTRUCTURED_STRATEGY=auto  # auto|fast|hi_res
UNSTRUCTURED_LANGUAGES=eng,deu

ENABLE_TESSERACT=false
TESSERACT_LANG=eng

ENABLE_CUSTOM_PROCESSOR=false
CUSTOM_PROCESSOR_URL=http://localhost:9000/process
CUSTOM_PROCESSOR_TYPES=application/pdf,image/jpeg
```

- **Removed**: `tests/test_unstructured_config.py` (legacy tests)
- **Added**: `tests/unit/test_document_processor_config.py`
  - 7 unit tests for new config system
  - Tests individual and multi-processor configurations

- **Added**:
  - `nextcloud_mcp_server/document_processors/__init__.py`
  - `nextcloud_mcp_server/document_processors/base.py`
  - `nextcloud_mcp_server/document_processors/registry.py`
  - `nextcloud_mcp_server/document_processors/unstructured.py`
  - `nextcloud_mcp_server/document_processors/tesseract.py`
  - `nextcloud_mcp_server/document_processors/custom_http.py`
  - `tests/unit/test_document_processor_config.py`

- **Modified**:
  - `nextcloud_mcp_server/config.py` - New plugin config system
  - `nextcloud_mcp_server/app.py` - Processor initialization
  - `nextcloud_mcp_server/utils/document_parser.py` - Uses registry
  - `nextcloud_mcp_server/server/webdav.py` - Import updates
  - `env.sample` - New configuration format
  - `docker-compose.yml` - (profile changes from previous work)

- **Removed**:
  - `nextcloud_mcp_server/client/unstructured_client.py` - Replaced by UnstructuredProcessor
  - `tests/test_unstructured_config.py` - Replaced with new tests

✅ **Extensible**: Add processors without modifying core code
✅ **Testable**: Mock processors for unit tests
✅ **Configurable**: Enable only needed processors
✅ **Flexible**: Choose fast (Tesseract) vs accurate (Unstructured)
✅ **Opt-in**: Disabled by default, no mandatory dependencies

Users upgrading from PR #190 need to update environment variables:
```bash
ENABLE_UNSTRUCTURED_PARSING=true

ENABLE_DOCUMENT_PROCESSING=true
ENABLE_UNSTRUCTURED=true
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-10-25 19:28:35 +02:00
Chris Coutinho d452684535 feat: Split read/write scopes into app:read/write scopes 2025-10-24 04:38:49 +02:00
Chris Coutinho f2d2dd8068 feat: Enable token introspection for opaque tokens 2025-10-23 15:51:27 +02:00
Chris Coutinho 053cf7798b fix: Add CORS middleware to allow browser-based clients like MCP Inspector 2025-10-23 15:23:41 +02:00
Chris Coutinho 737780b417 chore: Make all env vars available to be overriden as cli options 2025-10-23 11:48:01 +02:00
Chris Coutinho e9a16c43b5 refactor: Update JWT client to use DCR, re-enable tool filtering 2025-10-23 09:33:06 +02:00
Chris Coutinho e48f5f3f30 feat(server): Add support for custom OIDC scopes and permissions via JWTs 2025-10-23 08:37:36 +02:00
Chris Coutinho c069d78f80 feat: Initialize JWT-scoped tools 2025-10-22 06:21:16 +02:00
Chris Coutinho 37164dbdbc chore: sort imports 2025-10-18 22:02:25 +02:00
Chris Coutinho 5e829fc7e7 refactor: Unify logging & remove factory deployment 2025-10-18 01:15:06 +02:00
Chris Coutinho 9de59db718 feat(cookbook): Add full Cookbook app support with 13 tools and 2 resources 
- Import recipes from URLs using schema.org metadata
- Full CRUD operations for recipes
- Search, categorize, and organize recipes
- Manage keywords/tags and categories
- Configure app settings and trigger reindexing
 2025-10-17 03:08:16 +02:00
Chris Coutinho a38c795124 feat: add sharing API client and server tools 2025-10-15 02:59:26 +02:00
Chris Coutinho afc82ce3dc chore: Validate auth server support for PKCE on startup 2025-10-14 01:23:45 +02:00
Chris Coutinho 057e25b653 chore: Add support for overriding public issuer URL 
test: Add patch for PKCE support
 2025-10-14 01:23:41 +02:00
Chris Coutinho 13e4915e38 test: Remove unused pytest fixtures 2025-10-14 01:23:39 +02:00
Chris Coutinho 33b962a7fc test: Setup interactive browser test 2025-10-14 01:23:30 +02:00
Chris Coutinho 4d7e4b9a4b feat(server): Experimental support for OAuth2/OIDC authentication 2025-10-14 01:22:15 +02:00
Chris Coutinho 71da620099 refactor: Add http to --transport option 2025-09-20 22:23:13 +02:00
Chris Coutinho b3cd2ace34 chore: Update README.md, move docs to directory 2025-09-11 17:28:13 +02:00
Chris Coutinho 2cd91ceee7 chore: Update README and help text 2025-09-11 17:10:58 +02:00
Chris Coutinho c1c5a61952 feat(server): Add support for streamable-http transport type 2025-09-11 17:01:29 +02:00
Chris Coutinho 167053578d feat(deck): Initialize Deck app client/server 2025-09-11 00:10:25 +02:00
Chris Coutinho bbd8d1cf63 feat(cli): Replace mcp run with click CLI and runtime options 
BREAKING CHANGE: FASTMCP_-prefixed env vars have been replaced by CLI
arguments. Refer to the README for updated usage.

Usage: python -m nextcloud_mcp_server.app [OPTIONS]

Options:
  -h, --host TEXT
  -p, --port INTEGER
  -w, --workers INTEGER
  -r, --reload
  --log-level [critical|error|warning|info|debug|trace]
  -t, --transport [sse|streamable-http]
  -e, --enable-app [notes|tables|webdav|calendar|contacts]
                                  Enable specific Nextcloud app APIs. Can be
                                  specified multiple times. If not specified,
                                  all apps are enabled.
  --help                          Show this message and exit.
 2025-09-10 17:19:12 +02:00
Chris Coutinho 72cb62a101 test(contacts): Add unit/integration tests for a few tools 2025-08-03 14:36:16 +02:00
Chris Coutinho 37b1057d2a feat(contacts): Initialize Contacts App 2025-08-03 14:15:37 +02:00
Chris Coutinho 8956945e9d chore: sort imports 2025-08-01 12:21:32 +02:00
Chris Coutinho e7598a5467 format 2025-07-29 15:00:23 +02:00