403f8be429
feat: Add Keycloak external IdP integration with custom scopes
...
Add comprehensive support for using Keycloak as an external identity
provider with Nextcloud custom scopes. This enables testing of ADR-002
external IdP integration patterns.
**Keycloak Realm Configuration:**
- Add frontendUrl attribute to issue tokens with public issuer URL
- Define 18 Nextcloud custom client scopes (notes:read/write,
calendar:read/write, contacts:read/write, cookbook:read/write,
deck:read/write, tables:read/write, files:read/write,
sharing:read/write, todo:read/write)
- Add all custom scopes to nextcloud-mcp-server client optional scopes
- Scopes include consent screen text for user-friendly OAuth flow
**MCP Server Configuration:**
- Add OIDC_JWKS_URI environment variable support
- Implement JWKS URI override logic for Docker networking
- Update NEXTCLOUD_PUBLIC_ISSUER_URL to include full realm path
- Enable MCP server to fetch JWKS from internal Docker network
**Test Infrastructure:**
- Add keycloak_oauth_client_credentials fixture (session-scoped)
- Add keycloak_oauth_token fixture with Playwright automation
- Implement PKCE (S256) support for Keycloak OAuth flow
- Add nc_mcp_keycloak_client fixture for MCP testing
- Create comprehensive test suite in test_keycloak_external_idp.py
**Tests Created:**
- test_keycloak_oauth_token_acquisition: Token acquisition via Playwright
- test_keycloak_oauth_client_credentials_discovery: OIDC discovery
- test_mcp_client_connects_to_keycloak_server: MCP connectivity
- test_external_idp_server_initialization: Server auto-detection
- test_external_idp_token_validation: Token validation flow
- test_tools_work_with_keycloak_token: End-to-end tool execution
- test_keycloak_token_persistence: Multi-operation token reuse
- test_user_auto_provisioning: Nextcloud user provisioning
- test_scope_filtering_with_keycloak: Scope-based tool filtering
- test_keycloak_error_handling: Error handling
- test_external_idp_architecture: Architecture documentation
**Current Status:**
- ✅ Keycloak realm configuration complete
- ✅ Custom scopes defined and available
- ✅ OAuth token acquisition working (1 test passing)
- ⚠️ Token validation needs additional work (external IdP userinfo)
**Files Modified:**
- keycloak/realm-export.json: Realm configuration with scopes
- tests/conftest.py: Keycloak OAuth fixtures (+285 lines)
- tests/server/oauth/test_keycloak_external_idp.py: New test suite
- docker-compose.yml: OIDC_JWKS_URI and issuer configuration
- nextcloud_mcp_server/app.py: JWKS URI override logic
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-02 22:03:20 +01:00
e331544cee
feat: Implement RFC 8693 token exchange for Keycloak (ADR-002 Tier 2)
...
Implements OAuth 2.0 Token Exchange (RFC 8693) enabling the MCP server to
exchange service account tokens for user-scoped tokens. This provides an
alternative to refresh tokens for background operations.
**Core Implementation:**
- Added `get_service_account_token()` method to KeycloakOAuthClient for
client_credentials grant
- Added `exchange_token_for_user()` method implementing RFC 8693 token exchange
- Fixed Fernet encryption key handling in RefreshTokenStorage (was incorrectly
base64 decoding already-encoded keys)
- Updated OAuth configuration to support offline_access scope and refresh token
storage infrastructure
**Keycloak Configuration:**
- Enabled `serviceAccountsEnabled` in realm-export.json
- Added `token.exchange.grant.enabled` attribute
- Added `client.token.exchange.standard.enabled` attribute (required for
Keycloak 26.2+ Standard Token Exchange V2)
- Fresh Keycloak imports now correctly enable token exchange
**Docker Compose:**
- Added TOKEN_ENCRYPTION_KEY and ENABLE_OFFLINE_ACCESS environment variables
- Created oauth-tokens volume for refresh token storage
- Configured both mcp-oauth and mcp-keycloak services
**Testing & Documentation:**
- Added tests/manual/test_token_exchange.py - Validates complete RFC 8693 flow
- Added tests/manual/test_nextcloud_impersonate.py - Documents session-based
impersonation limitations
- Added docs/oauth-impersonation-findings.md - Comprehensive investigation
findings and resolution documentation
**Verified Working:**
✅ Service account token acquisition (client_credentials grant)
✅ RFC 8693 token exchange for internal-to-internal tokens
✅ Exchanged tokens validate with Nextcloud APIs
✅ Keycloak 26.4.2 Standard Token Exchange V2 support
**Known Limitations:**
- User impersonation (requested_subject) requires Keycloak Legacy V1 with
preview features
- Cross-client token exchange limited to same realm
- Refresh token storage infrastructure ready but unused (MCP protocol limitation)
Dependencies: aiosqlite>=0.20.0
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-02 22:03:19 +01:00
37b0b4a281
fix: Update DCR token_type tests for OIDC app changes
...
The Nextcloud OIDC app has updated token_type parameter values:
- Changed from "Bearer" → "opaque" for opaque tokens
- Changed from "JWT" → "jwt" for JWT tokens
Updated test_dcr_token_type.py to use lowercase token_type values:
- token_type="jwt" for JWT-formatted tokens
- token_type="opaque" for opaque/bearer tokens
This fixes test failures where tests were using the old "Bearer" and
"JWT" (uppercase) values which are no longer recognized by the OIDC app.
Fixes test: test_dcr_respects_bearer_token_type
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-02 22:03:19 +01:00
ab7411d9fd
test: Fix tests
2025-10-25 22:07:46 +02:00
9414d9c9c3
test: Add integration marker to user/group tests
2025-10-25 20:16:14 +02:00
8a52df4a8e
test: Skip unstructured tests if not enabled
2025-10-25 20:13:41 +02:00
a36038422b
feat: Add text processing background worker for telling client about progress
2025-10-25 19:52:45 +02:00
2147fc1696
refactor: Transform document parsing into pluggable processor architecture
...
Refactors PR #190 's hardcoded Unstructured.io integration into a flexible,
extensible plugin system supporting multiple text extraction engines.
- **`DocumentProcessor` ABC**: Abstract interface for all processors
- **`ProcessorRegistry`**: Central registry for discovery and routing
- **`ProcessingResult`**: Standardized output format across processors
- **`UnstructuredProcessor`**: Refactored from `UnstructuredClient`
- **`TesseractProcessor`**: Local OCR for images (lightweight alternative)
- **`CustomHTTPProcessor`**: Generic wrapper for custom HTTP APIs
- New `get_document_processor_config()` returns structured config
- Supports enabling/disabling individual processors
- Per-processor configuration via environment variables
- **Breaking Change**: `ENABLE_UNSTRUCTURED_PARSING` replaced with:
- `ENABLE_DOCUMENT_PROCESSING=true/false` (master switch)
- `ENABLE_UNSTRUCTURED=true/false` (per-processor)
- `ENABLE_TESSERACT=true/false`
- `ENABLE_CUSTOM_PROCESSOR=true/false`
- `parse_document()` now uses `ProcessorRegistry`
- Auto-selects appropriate processor based on MIME type
- Processor priority system (Unstructured=10, Tesseract=5, Custom=1)
- `initialize_document_processors()` registers processors at startup
- Integrated into both BasicAuth and OAuth lifespans
- Graceful degradation if processors fail to initialize
```env
ENABLE_DOCUMENT_PROCESSING=false
ENABLE_UNSTRUCTURED=false
UNSTRUCTURED_API_URL=http://unstructured:8000
UNSTRUCTURED_STRATEGY=auto # auto|fast|hi_res
UNSTRUCTURED_LANGUAGES=eng,deu
ENABLE_TESSERACT=false
TESSERACT_LANG=eng
ENABLE_CUSTOM_PROCESSOR=false
CUSTOM_PROCESSOR_URL=http://localhost:9000/process
CUSTOM_PROCESSOR_TYPES=application/pdf,image/jpeg
```
- **Removed**: `tests/test_unstructured_config.py` (legacy tests)
- **Added**: `tests/unit/test_document_processor_config.py`
- 7 unit tests for new config system
- Tests individual and multi-processor configurations
- **Added**:
- `nextcloud_mcp_server/document_processors/__init__.py`
- `nextcloud_mcp_server/document_processors/base.py`
- `nextcloud_mcp_server/document_processors/registry.py`
- `nextcloud_mcp_server/document_processors/unstructured.py`
- `nextcloud_mcp_server/document_processors/tesseract.py`
- `nextcloud_mcp_server/document_processors/custom_http.py`
- `tests/unit/test_document_processor_config.py`
- **Modified**:
- `nextcloud_mcp_server/config.py` - New plugin config system
- `nextcloud_mcp_server/app.py` - Processor initialization
- `nextcloud_mcp_server/utils/document_parser.py` - Uses registry
- `nextcloud_mcp_server/server/webdav.py` - Import updates
- `env.sample` - New configuration format
- `docker-compose.yml` - (profile changes from previous work)
- **Removed**:
- `nextcloud_mcp_server/client/unstructured_client.py` - Replaced by UnstructuredProcessor
- `tests/test_unstructured_config.py` - Replaced with new tests
✅ **Extensible**: Add processors without modifying core code
✅ **Testable**: Mock processors for unit tests
✅ **Configurable**: Enable only needed processors
✅ **Flexible**: Choose fast (Tesseract) vs accurate (Unstructured)
✅ **Opt-in**: Disabled by default, no mandatory dependencies
Users upgrading from PR #190 need to update environment variables:
```bash
ENABLE_UNSTRUCTURED_PARSING=true
ENABLE_DOCUMENT_PROCESSING=true
ENABLE_UNSTRUCTURED=true
```
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-10-25 19:28:35 +02:00
f0e5333e43
Merge branch 'master' into feature/introduce_files_parsing_with_unstructured_service_for_webdav_files_retrieval
2025-10-25 17:23:38 +02:00
01b43c96ba
test: Update client id/secret -> client_info
2025-10-24 19:47:49 +02:00
50b69a2531
fix: Add support for RFC 7592 client registration and deletion
2025-10-24 19:19:27 +02:00
8e0a4d8ce5
feat(auth): Add support for client registration deletion
2025-10-24 18:54:24 +02:00
72fce189d2
test: Add tests for dcr endpoint and update oidc app
2025-10-24 18:48:05 +02:00
1e877f17f7
test: Replace persistent OAuth client cache with session-scoped fixtures
...
Remove file-based caching of OAuth client credentials and implement automatic
client lifecycle management for test fixtures.
Changes:
- Add RFC 7592 client deletion function in auth/client_registration.py
- Remove cache_file parameter from _create_oauth_client_with_scopes helper
- Update all OAuth credential fixtures to use yield/finalizer pattern
- Add automatic client cleanup at end of test session (best-effort)
- Remove persistent .nextcloud_oauth_*.json cache files
Benefits:
- No persistent cache files cluttering repository
- Fresh OAuth clients created for each test session via DCR
- Automatic cleanup attempts (RFC 7592 DELETE endpoint)
- Cleaner test environment with proper fixture lifecycle
Note: Client deletion may fail due to Nextcloud authentication middleware
(logged as warning). The key improvement is removing persistent cache files.
OAuth clients may accumulate in Nextcloud but can be cleaned manually.
2025-10-24 08:11:22 +02:00
13f76a7734
chore: Upgrade pydantic Config to ConfigDict
2025-10-24 06:18:13 +02:00
2f1bd1bbe9
test: Move client integration tests to mocked unit tests
2025-10-24 05:50:25 +02:00
d452684535
feat: Split read/write scopes into app:read/write scopes
2025-10-24 04:38:49 +02:00
d55e5708c7
ci: fix imports
2025-10-24 01:04:30 +02:00
d4ee5a74c2
test: Update default tokens to JWT, add to introspection tests
2025-10-24 00:51:50 +02:00
db79afacb9
improved tests - fixing the linting
2025-10-23 22:56:25 +02:00
6730dd4a4b
added new tests for unstructured api (pdf and docx workflow)
2025-10-23 22:38:27 +02:00
8734c4b292
add new tests for unstructured config
2025-10-23 22:37:52 +02:00
053cf7798b
fix: Add CORS middleware to allow browser-based clients like MCP Inspector
2025-10-23 15:23:41 +02:00
737780b417
chore: Make all env vars available to be overriden as cli options
2025-10-23 11:48:01 +02:00
54e975198f
test: Update all test network hosts to respect iss claims from JWTs
2025-10-23 11:09:51 +02:00
e48f5f3f30
feat(server): Add support for custom OIDC scopes and permissions via JWTs
2025-10-23 08:37:36 +02:00
1aecb099e6
fix: Use occ-created OAuth clients with allowed_scopes for all tests
...
The shared_oauth_client_credentials fixture was using Dynamic Client
Registration which doesn't support Nextcloud's allowed_scopes parameter.
This caused tokens to lack proper scope configuration, resulting in empty
tool lists when the server validated scopes.
Changes:
1. Updated shared_oauth_client_credentials to use occ oidc:create with
allowed_scopes="openid profile email nc:read nc:write"
2. Created opaque token client (not JWT) for port 8001 compatibility
3. Enhanced _create_oauth_client_with_scopes to support both JWT and
opaque token types via token_type parameter
This ensures:
- Regular OAuth tests (port 8001) get opaque tokens with proper scopes
- JWT OAuth tests (port 8002) get JWT tokens with embedded scopes
- Both token types have allowed_scopes configured on the OAuth client
Fixes test_mcp_oauth_server_connection which was getting empty tool list
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-10-22 07:38:16 +02:00
2c35e07675
fix: Separate OAuth fixtures for opaque vs JWT tokens
...
Previous fix created a JWT OAuth client for all tests, which broke the
regular OAuth server (port 8001) that expects opaque tokens.
This commit:
1. Reverts shared_oauth_client_credentials to use regular OAuth (opaque tokens)
2. Creates new shared_jwt_oauth_client_credentials for JWT OAuth clients
3. Creates new playwright_oauth_token_jwt fixture using JWT credentials
4. Updates nc_mcp_oauth_jwt_client to use JWT token fixture
This ensures:
- Regular OAuth tests (port 8001) use opaque tokens
- JWT OAuth tests (port 8002) use JWT tokens with embedded scopes
Fixes remaining CI failure in test_mcp_oauth_server_connection
2025-10-22 07:17:43 +02:00
5cfdff0faf
test: Create JWT OAuth client with explicit scopes for shared test fixture
...
The shared_oauth_client_credentials fixture was creating an OAuth client
without explicit allowed_scopes configuration. This caused JWT tokens to
lack nc:read and nc:write scope claims, resulting in the JWT MCP server
filtering out ALL tools when list_tools() was called.
Changed the fixture to use _create_oauth_client_with_scopes() helper to
create a JWT client with explicit allowed_scopes="openid profile email
nc:read nc:write", matching the scopes requested in the authorization
URL and the behavior of other scoped test fixtures.
This fixes CI test failures in:
- test_mcp_oauth.py::test_mcp_oauth_server_connection
- test_mcp_oauth_jwt.py::test_jwt_mcp_server_connection
- test_mcp_oauth_jwt.py::test_jwt_tool_list_operations
- test_mcp_oauth_jwt.py::test_jwt_automation_worked
All were failing with: assert len(result.tools) > 0 (result.tools was empty)
2025-10-22 07:02:40 +02:00
8a3269f366
test: Use separate docker compose command
2025-10-22 06:38:05 +02:00
c069d78f80
feat: Initialize JWT-scoped tools
2025-10-22 06:21:16 +02:00
63b898c0e3
chore: Update logs
2025-10-20 22:57:18 +02:00
a143123acc
fix(caldav): Check that calendar exists after creation to avoid race condition
...
Verify that field preservation tests still operate
2025-10-19 23:44:39 +02:00
1dc2ddfdb7
fix(caldav): Properly parse datetimes as vDDDTypes
2025-10-19 20:13:05 +02:00
92e18825bc
feat(caldav): Add support for tasks
2025-10-19 18:02:43 +02:00
5757f2582b
ci: Run oauth tests
2025-10-19 00:49:55 +02:00
b72514bb32
ci: Add pytest-timeout to dev deps
2025-10-19 00:27:19 +02:00
95da43ea0f
ci: Increase playwright timeout to 60s
2025-10-18 23:26:50 +02:00
963a504ae2
ci: Replace 0.5 stagger with 10s in CI
2025-10-18 22:57:47 +02:00
ead298c132
chore: revert conftest.py
2025-10-18 22:44:51 +02:00
2f805e54b7
test: Migrate load test benchmark scripts to anyio
...
Remove unused redis container
2025-10-18 22:40:50 +02:00
6158a890af
feat(webdav): Add search and list favorite response tools
2025-10-18 22:02:26 +02:00
240ceb3808
test: Migrate load test framework to anyio as well
2025-10-18 22:02:25 +02:00
1459fe9bc8
test: Replace pytest-asyncio plugin fixtures with anyio fixtures
2025-10-18 22:02:25 +02:00
37164dbdbc
chore: sort imports
2025-10-18 22:02:25 +02:00
c3ff92a8c1
test: Cleanup testing fixtures regarding canceled scopes
2025-10-18 22:02:25 +02:00
371d0c93a5
test: Update oauth benchmark tests
2025-10-18 22:02:25 +02:00
644c59bf78
docs: remove old docs
2025-10-18 22:02:25 +02:00
056b6fc9d6
test: Initialize load testing framework
2025-10-18 22:02:24 +02:00
83917b3786
perf(notes): Improve notes search performance using async iterators
2025-10-18 22:02:19 +02:00
Chris Coutinho