Commit Graph

828 Commits

Author SHA1 Message Date
Chris Coutinho d16bcdcfbb feat: Implement ADR-004 Progressive Consent foundation components
- Token Broker Service manages Nextcloud access tokens with audience validation
- Implements short-lived token caching (5-minute TTL) with early refresh
- Enhanced token storage schema with ADR-004 fields (flow_type, audience, provisioning)
- MCP provisioning tools for explicit Flow 2 resource authorization
- Comprehensive unit tests for Token Broker Service (14 tests, all passing)
- Environment configuration for Progressive Consent mode

This implements the foundation for the dual OAuth flow architecture where:
- Flow 1: MCP clients authenticate to MCP server (aud: "mcp-server")
- Flow 2: MCP server gets delegated Nextcloud access (aud: "nextcloud")

Users must explicitly call provision_nextcloud_access tool to grant resource access,
implementing the "stateless by default" principle from ADR-004.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 07:51:07 +01:00
Chris Coutinho 9d514f52b0 docs: Refactor ADR-004 to Progressive Consent architecture with dual OAuth flows
Replace hybrid flow model with true progressive consent where MCP client authenticates directly to IdP (Flow 1) and server requests separate explicit provisioning for Nextcloud access (Flow 2). This separates client authentication from resource authorization, uses distinct client_id for each flow, and keeps server stateless by default until user explicitly grants offline access via provision_nextcloud_access tool.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 02:55:27 +01:00
Chris Coutinho 4e1d143e54 Merge remote-tracking branch 'origin/master' into feature/keycloak 2025-11-03 02:49:00 +01:00
github-actions[bot] 02a2c4a16f bump: version 0.22.7 → 0.23.0 nextcloud-mcp-server-0.23.0 v0.23.0 2025-11-03 01:48:39 +00:00
Chris Coutinho f37008fdc3 Merge pull request #254 from cbcoutinho/feature/keycloak
feat: Complete Keycloak external IdP integration with ADR-002 implementation
2025-11-03 02:47:57 +01:00
Chris Coutinho 0d45120470 docs: Update ADR-004 with progressive consent architecture
Refactor ADR-004 to document the proper OAuth architecture where MCP
clients are registered at the IdP level (not with MCP server) and use
a progressive consent pattern with dual OAuth flows.

## Key Changes

### MCP Client Registration
- Document that MCP clients (Claude Desktop, etc.) register at IdP level
- Show DCR and pre-registration options
- Clarify client validation happens against IdP registry

### Progressive Consent Architecture
Replace single "Hybrid Flow" with three-phase progressive consent:

**Phase 1: MCP Client Authentication** (Always)
- MCP client uses own client_id (e.g., "claude-desktop")
- User consents to "Claude Desktop accessing MCP Server"
- MCP server validates client exists at IdP
- Stores MCP client access token

**Phase 2: Nextcloud Consent** (Conditional)
- Only if MCP server doesn't have refresh token for user
- MCP server uses own client_id ("nextcloud-mcp-server")
- User consents to "MCP Server accessing Nextcloud offline"
- MCP server stores master refresh token
- SSO: If already authenticated, only consent needed

**Phase 3: Token Exchange** (Standard PKCE)
- Client exchanges MCP authorization code
- Validates PKCE code_verifier
- Returns access token (aud: mcp-server)
- Client never sees master refresh token

### Implementation Status Section
- Document current implementation as "simplified hybrid flow"
- List what's implemented vs what needs refactoring
- Clarify current tests use simplified version
- Note progressive consent is target architecture

## Benefits of Progressive Consent

 Standards-compliant: Proper OAuth clients at IdP level
 Secure: Client validation against IdP registry
 Efficient: Nextcloud consent only once per user
 Transparent: Users understand each authorization step
 SSO-friendly: Minimal re-authentication in Phase 2

## Implementation Tracking

The refactoring from simplified hybrid flow to progressive consent will
be tracked in a separate issue. Current implementation demonstrates:
- MCP server can intercept OAuth callbacks
- Refresh tokens stored securely
- PKCE flow works end-to-end
- Tool execution succeeds

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 02:34:30 +01:00
Chris Coutinho babd60e08b feat: Implement ADR-004 Hybrid Flow with comprehensive integration tests
Implement the ADR-004 Hybrid Flow OAuth pattern where the MCP server
intercepts the OAuth callback to obtain master refresh tokens while
maintaining PKCE security for clients.

## Implementation

### OAuth Routes (ADR-004 Hybrid Flow)
- Add `/oauth/authorize` endpoint: Intercepts client OAuth initiation
- Add `/oauth/callback` endpoint: Receives IdP callback, stores master token
- Add `/oauth/token` endpoint: Exchanges MCP code for client access token
- Implement PKCE code challenge/verifier validation
- Store OAuth sessions with state/challenge correlation

### MCP Server Integration
- Update `setup_oauth_config()` to return client_id and client_secret
- Initialize OAuth context in Starlette lifespan for login routes
- Add OAuth session storage to RefreshTokenStorage
- Configure authlib dependency for OAuth flow management

### Integration Tests
- Create `test_adr004_hybrid_flow.py` with Playwright automation
- Add `adr004_hybrid_flow_mcp_client` session-scoped fixture
- Test MCP session establishment with hybrid flow token
- Test tool execution using stored refresh tokens (on-behalf-of pattern)
- Test persistent access across multiple operations
- All tests passing:  3 passed in 8.82s

### Documentation
- Update ADR-004 with comprehensive Testing section
- Add integration test commands and coverage details
- Document test implementation and verification steps
- Create TESTING_INSTRUCTIONS.md for manual and automated testing
- Include manual test scripts for reference/debugging

## What This Enables

 PKCE code challenge/verifier flow
 MCP server intercepts OAuth callback and stores master refresh token
 Client receives MCP access token (not master token)
 MCP session establishment with hybrid flow token
 Tool execution using stored refresh tokens (on-behalf-of pattern)
 Multiple operations without re-authentication
 Proper token isolation (client never sees master token)

## Testing

Run ADR-004 integration tests:
```bash
uv run pytest tests/server/oauth/test_adr004_hybrid_flow.py --browser firefox -v
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 02:18:30 +01:00
Chris Coutinho f48e039e9e docs: WIP with Hybrid token 2025-11-03 01:19:46 +01:00
Chris Coutinho 14a8f70503 docs: Correct ADR-004 to Token Broker Architecture with strict audience isolation
Critical architectural corrections to properly implement secure token brokering:

## Key Changes:

1. **Removed Dual Token Concept**: MCP server no longer generates its own JWTs.
   Instead, it acts as a token broker using IdP-issued tokens with proper
   audience validation.

2. **Strict Audience Isolation**:
   - Tokens with `aud: "mcp-server"` can ONLY authenticate to MCP server
   - Tokens with `aud: "nextcloud"` can ONLY access Nextcloud APIs
   - No tokens have multiple audiences (security boundary violation)
   - Compromised MCP tokens cannot access Nextcloud directly

3. **Linked Authorization Pattern**: Single OAuth flow obtains a master
   refresh token capable of minting tokens for different audiences as needed.
   This solves the challenge of needing both MCP authentication and Nextcloud
   access from a single user authorization.

4. **Token Broker Implementation**:
   - Validates incoming tokens have `audience: "mcp-server"`
   - Uses stored refresh tokens to obtain `audience: "nextcloud"` tokens
   - Never exposes Nextcloud tokens to MCP clients
   - Maintains short-lived cache for performance

5. **PKCE and Native Client Updates**:
   - Proper 302 redirects (no HTML pages)
   - Complete PKCE verification in token endpoint
   - IdP tokens returned directly (not MCP-generated)

6. **Security Enhancements**:
   - Comprehensive audience validation examples
   - Token exchange pattern documentation
   - Keycloak configuration for audience mapping
   - Trust boundary diagrams

This architecture maintains strict security boundaries while enabling the
MCP server to act on behalf of users for both authentication and resource
access, following OAuth best practices and enterprise security standards.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 00:44:34 +01:00
Chris Coutinho bf8120682e docs: Rewrite ADR-004 for Federated Authentication Architecture
Major rewrite of ADR-004 to reflect federated authentication pattern with
shared identity provider (IdP) instead of direct Nextcloud authentication.

Key changes:
- Replaced "Sign-in with Nextcloud" with "Federated Authentication"
- Added shared IdP (Keycloak, Okta, Azure AD) as central auth provider
- MCP server now acts as OAuth client to shared IdP, not Nextcloud
- Single user authentication grants both identity and Nextcloud access
- Updated all diagrams to show 4-party architecture
- Removed authorize_nextcloud tool - uses standard 401 flow
- Added proper token rotation with reuse detection
- Clarified Pattern 3 vs Pattern 4 differences in comparison doc
- Pattern 3 can use external IdPs via user_oidc (not limited to NC)

Architecture benefits:
- True single sign-on with enterprise IdP support
- OAuth-compliant on-behalf-of pattern
- Supports SAML/LDAP backends through IdP
- Nextcloud validates IdP tokens, not MCP-specific tokens

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 23:58:15 +01:00
Chris Coutinho f2af5a39a8 docs: Add ADR-004 - MCP Server as OAuth Client for Offline Access
- Supersedes ADR-002 which fundamentally misunderstood MCP protocol constraints
- Introduces "Sign-in with Nextcloud" architecture pattern
- MCP server becomes OAuth client to enable offline/background operations
- Implements full token rotation with reuse detection for security
- Includes comprehensive implementation details and migration strategy

Key architectural shift:
- From: Pass-through authentication (stateless, no offline access)
- To: MCP server as OAuth client (stateful, full offline capabilities)

The solution enables background workers to operate independently of MCP
sessions by storing and rotating refresh tokens securely.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 23:31:39 +01:00
Chris Coutinho 7cb616c7ce feat: Auto-configure impersonation role in Keycloak realm import
Add service account user with impersonation role to realm-export.json
so that Tier 1 impersonation works out-of-the-box without requiring
manual CLI configuration.

Changes:
- Add service-account-nextcloud-mcp-server user to realm import
- Grant "impersonation" role from "realm-management" client
- Eliminates need for manual `kcadm.sh add-roles` command

Benefits:
- Impersonation tests now pass automatically
- No manual permission configuration required
- Consistent development environment setup

Verified:
- Manual test: tests/manual/test_impersonation.py  PASS
- Integration tests: tests/integration/auth/test_token_exchange_legacy_v1.py  3 PASS

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:22 +01:00
Chris Coutinho 34df5f5b9a feat: Implement dual-tier token exchange (Standard V2 + Legacy V1 impersonation)
This commit implements and documents both RFC 8693 token exchange tiers
from ADR-002, enabling both production-ready delegation and advanced
impersonation capabilities.

- Enable Keycloak preview features (`--features=preview`) to support
  both Standard V2 and Legacy V1 token exchange modes

- Update Tier 1 status from "NOT IMPLEMENTED" to "IMPLEMENTED (Legacy V1)"
- Add detailed empirical testing results showing:
  - Standard V2 rejects `requested_subject` parameter
  - Legacy V1 accepts parameter but requires impersonation permissions
  - Complete configuration steps for enabling impersonation
- Add comparison table showing when to use each tier
- Add "When to Use" guidance for both tiers
- Document that Tier 2 (Delegation) is the recommended default

- Update docstring to document both Tier 1 and Tier 2 support
- Add tier-specific logging (shows which tier is being used)
- Document permission requirements for Tier 1 impersonation

**tests/integration/auth/test_token_exchange_standard_v2.py**:
- Test delegation without impersonation (Tier 2)
- Verify sub claim remains unchanged (service account identity)
- Verify no special permissions required
- Test exchanged tokens work with Nextcloud APIs
- All tests PASS 

**tests/integration/auth/test_token_exchange_legacy_v1.py**:
- Test impersonation with `requested_subject` (Tier 1)
- Verify sub claim changes to target user
- Auto-skip if impersonation permissions not configured
- Document permission requirements in test docstrings
- Test exchanged tokens work with Nextcloud APIs

**tests/manual/test_impersonation.py**:
- Comprehensive impersonation validation script
- Tests both Standard V2 and Legacy V1 behavior
- Decodes JWT tokens to verify sub claim changes
- Validates tokens against Nextcloud APIs

**tests/manual/configure_impersonation.py**:
- Automated permission configuration helper
- Documents manual Keycloak CLI configuration steps

Both token exchange tiers are now fully implemented and tested:

- **Tier 2 (Delegation)** -  RECOMMENDED
  - Standard V2 (production-ready)
  - No special permissions required
  - Service account identity preserved

- **Tier 1 (Impersonation)** -  Advanced use only
  - Legacy V1 (--features=preview required)
  - Requires manual permission grant via Keycloak CLI
  - Subject claim changes to target user

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:22 +01:00
Chris Coutinho e26c5128b7 docs: Reject service account tokens as OAuth authentication pattern
Service account tokens (client_credentials grant) violate OAuth "act on-behalf-of"
principles and have been moved to ADR-002's "Will Not Implement" section.

## Problem Discovery

Testing revealed that service account tokens create Nextcloud user accounts
(e.g., `service-account-nextcloud-mcp-server`) due to user_oidc's bearer
provisioning feature. This violates core OAuth principles:

-  Creates stateful server identity in Nextcloud
-  All actions attributed to service account, not real user
-  Breaks audit trail and user attribution
-  Service account becomes "admin by another name"

## Changes

### Documentation (ADR-002)
- Moved service account (old Tier 1) to "Will Not Implement" section
- Added "OAuth Act On-Behalf-Of Principle" section
- Renumbered tiers:
  - Tier 1: Impersonation (NOT IMPLEMENTED)
  - Tier 2: Delegation via token exchange (IMPLEMENTED)
- Updated status to reflect rejection of service accounts

### Code Warnings
- Added comprehensive warning to KeycloakOAuthClient.get_service_account_token()
- Clarified VALID use: only as subject_token for RFC 8693 token exchange
- Clarified INVALID use: direct API access with service account token

### Supporting Documentation
- CLAUDE.md: Removed outdated "Tier 1" references, added rejection note
- oauth-impersonation-findings.md: Added prominent update banner
- audience-validation-setup.md: Updated tier numbers, added rejection note
- tests/manual/test_token_exchange.py: Added warning comment

## Valid Patterns (ADR-002)

 Foreground operations: User's access token from MCP request
 Background operations: Token exchange (impersonation/delegation)
 Offline access: Refresh tokens with user consent
 Service accounts: Creates independent server identity (REJECTED)

## Alternative

If service account pattern is truly needed, use BasicAuth mode instead of
OAuth mode. OAuth mode MUST maintain "act on-behalf-of" semantics.

Related: c12df98 (revert of service account test)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:22 +01:00
Chris Coutinho ed813af45c Revert "test: Add automated test for service account token acquisition (ADR-002 Tier 1)"
This reverts commit cbc37f1d76687d66a771236903ccb88b2e7b0242.
2025-11-02 22:03:22 +01:00
Chris Coutinho 1e071c83a9 test: Add automated test for service account token acquisition (ADR-002 Tier 1)
Add comprehensive automated integration test for Keycloak service account
token acquisition via client_credentials grant, validating ADR-002 Tier 1
implementation for external IdP mode.

Changes:
- Add keycloak_oauth_client fixture in tests/conftest.py
  - Creates KeycloakOAuthClient instance for service account operations
  - Session-scoped fixture with automatic cleanup
  - Discovers Keycloak endpoints automatically

- Add test_keycloak_service_account_token_acquisition test
  - Tests client_credentials grant token acquisition
  - Verifies token response structure (access_token, token_type, expires_in)
  - Validates token works with Nextcloud APIs via capabilities endpoint
  - Documents limitation for Nextcloud OIDC app (integrated mode)

- Update ADR-002 documentation
  - Mark automated test as complete ()
  - Document supported providers (Keycloak , Nextcloud OIDC app )
  - Add note that KeycloakOAuthClient is provider-agnostic
  - Clarify that Nextcloud OIDC app support requires config only

Test results:
-  Service account token acquired successfully (300s expiry, Bearer type)
-  Token validated by Nextcloud user_oidc app
-  Token works with Nextcloud capabilities API

Note: Nextcloud OIDC app (integrated mode) service account token support
not yet implemented. See app.py:631-635 for current status.

Resolves: "TODO: Automated integration tests needed for both Keycloak and
Nextcloud OIDC app" from ADR-002
2025-11-02 22:03:22 +01:00
Chris Coutinho 76430bec21 docs: Update ADR-002 with OAuth-only focus and testing status [skip ci]
Major changes to ADR-002 (Vector Database Background Sync Authentication):

1. Reordered authentication tiers:
   - Tier 1: Service Account Token (client_credentials) - most compatible
   - Tier 2: Token Exchange with Impersonation - not implemented
   - Tier 3: Token Exchange with Delegation - implemented

2. Removed admin credentials fallback:
   - ADR now focuses exclusively on OAuth mode
   - Background sync unavailable without proper OAuth configuration
   - BasicAuth mode out of scope (credentials already available)

3. Clarified testing status:
   - Tier 1: Implemented but only manual tests exist
   - Tier 3: Implemented but only manual tests exist
   - Added TODO for automated integration tests

4. Removed "Offline Access with Refresh Tokens":
   - Documented as "Will Not Implement"
   - MCP protocol architecture prevents server from accessing refresh tokens
   - Violates OAuth security model (tokens must stay with client)

5. Simplified configuration:
   - Removed all admin credential references
   - OAuth-only environment variables
   - Automatic tier detection based on provider capabilities

The ADR now accurately reflects that refresh tokens should never be shared
between MCP client and server, following OAuth best practices and the
FastMCP SDK architecture.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:22 +01:00
Chris Coutinho e81c2ad33d docs: Update upstream OAuth status with completed oidc app PRs [skip ci]
Update oauth-upstream-status.md to clarify patch requirements and document
completed upstream work:

**Clarifications:**
- CORSMiddleware patch is for Nextcloud core server (not user_oidc app)
- Root cause: CORS middleware logs out sessions without CSRF tokens
- Solution: Allow Bearer tokens to bypass CORS/CSRF checks
- Updated all references with actual PR number: nextcloud/server#55878

**Completed oidc app PRs (now documented):**
-  H2CK/oidc#586: User consent management (v1.11.0+)
-  H2CK/oidc#585: JWT tokens, introspection, scope validation (v1.10.0+)
-  H2CK/oidc#584: PKCE support (RFC 7636) (v1.10.0+)

**Updated sections:**
- "What Works Without Patches" - Added JWT, scopes, consent features
- "Upstream PRs Status" - Added completed PRs table
- "Monitoring Upstream Progress" - Focus on remaining work
- Last updated date: 2025-11-02

All OAuth features except app-specific APIs now work out of the box
with oidc app v1.10.0+. Only CORSMiddleware patch remains pending.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:21 +01:00
Chris Coutinho 23360485a8 refactor: Remove NEXTCLOUD_OIDC_CLIENT_STORAGE environment variable
Remove the NEXTCLOUD_OIDC_CLIENT_STORAGE environment variable from all
configuration files. OAuth client credentials are now always stored in the
SQLite database, with no option to use a custom JSON file path.

Changes:
- Remove NEXTCLOUD_OIDC_CLIENT_STORAGE from .env.keycloak.sample
- Remove NEXTCLOUD_OIDC_CLIENT_STORAGE from docker-compose.yml (mcp-oauth and mcp-keycloak services)
- Remove NEXTCLOUD_OIDC_CLIENT_STORAGE from Helm deployment template
- Remove NEXTCLOUD_OIDC_CLIENT_STORAGE from test_cli.py test assertions
- Remove --headed flag from pytest addopts (use CLI arg instead)

This simplifies configuration by enforcing a single storage mechanism
(SQLite database) for OAuth client credentials.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:21 +01:00
Chris Coutinho 2ca6725fc6 docs: Replace .nextcloud_oauth_client.json references with SQLite storage
Replace all references to the JSON file-based OAuth client storage with
SQLite database storage in documentation. OAuth client credentials are now
stored in the SQLite database instead of .nextcloud_oauth_client.json.

Changes:
- Update oauth-architecture.md to reference SQLite database
- Update jwt-oauth-reference.md credential storage sections
- Update oauth-setup.md Docker volume mounts and security best practices
- Update oauth-troubleshooting.md file permission → database permission errors
- Update configuration.md to remove JSON file chmod instructions
- Update troubleshooting.md database permission troubleshooting

The code already uses SQLite (RefreshTokenStorage class), so only
documentation needed updating.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:21 +01:00
Chris Coutinho 4c7d1cfc8d test: Add scope-based authorization tests for Keycloak external IdP
This enhances the Keycloak integration test suite with comprehensive
scope-based authorization validation, matching the OIDC test structure.

Changes:
- Add 3 test users to Keycloak realm (read-only, write-only, no-custom-scopes)
- Create OAuth token fixtures with different scope combinations
- Create MCP client fixtures for each scope configuration
- Add 4 new tests validating scope-based tool filtering:
  * Read-only tokens filter out write tools
  * Write-only tokens filter out read tools
  * Full access tokens show all 90+ tools
  * No custom scopes result in zero tools

Test Results:
- All 15 Keycloak integration tests pass (11 existing + 4 new)
- Validates proper JWT scope enforcement in external IdP architecture
- Confirms security isolation when users decline custom scopes

This completes ADR-002 scope authorization testing for the Keycloak
external identity provider integration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:21 +01:00
Chris Coutinho b68c704c4d refactor: Remove unnecessary user_oidc patch - CORSMiddleware patch is sufficient
Testing confirmed that the CORSMiddleware Bearer token patch (from upstream
commit 8fb5e77db82) alone is sufficient to enable Bearer token authentication
for all Nextcloud APIs, including app-specific endpoints like Notes and Calendar.

The user_oidc patch (which sets the app_api session flag) is not required when
the CORSMiddleware patch is applied, as it fixes the root cause by allowing
Bearer tokens to bypass CORS/CSRF checks at the framework level.

Validation:
- Restarted Nextcloud with user_oidc patch disabled
- Ran all 11 Keycloak integration tests
- All tests passed without the user_oidc patch

Updated documentation in 10-install-user_oidc-app.sh to explain why the patch
is no longer needed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:21 +01:00
Chris Coutinho 849c67c32a fix: Complete Keycloak external IdP integration with all tests passing
This commit completes the Keycloak external IdP integration for the MCP
server, implementing ADR-002 Tier 2 (External Identity Provider) with
full Bearer token authentication support.

Key Changes:
1. **Keycloak backchannel-dynamic configuration**
   - Added --hostname-strict=false and --hostname-backchannel-dynamic=true
   - Allows external issuer (localhost:8888) with internal endpoints (keycloak:8080)
   - Solves Docker networking issue where containers can't reach localhost

2. **CORSMiddleware Bearer token patch**
   - Created app-hooks/patches/cors-bearer-token.patch from upstream commit 8fb5e77db82
   - Allows Bearer tokens to bypass CORS/CSRF checks (stateless authentication)
   - Applied via post-installation hook 20-apply-cors-bearer-token-patch.sh
   - Enables app-specific APIs (Notes, Calendar, etc.) to work with Bearer tokens

3. **Patch organization**
   - Moved patches to app-hooks/patches/ directory
   - Updated docker-compose.yml to mount entire app-hooks directory
   - Consolidated patch management for better maintainability

4. **Test improvements**
   - All 11 Keycloak integration tests passing
   - Tests validate OAuth token acquisition, MCP connectivity, token validation,
     tool execution, token persistence, user provisioning, scope filtering,
     and error handling

Architecture:
- Keycloak acts as external OAuth/OIDC identity provider
- MCP server uses Keycloak tokens to access Nextcloud APIs
- Nextcloud user_oidc app validates Bearer tokens from Keycloak
- No admin credentials needed - all API access uses user's OAuth tokens

Cache Note:
- Discovery and JWKS caches must be cleared when switching Keycloak configurations
- Use: docker compose exec redis redis-cli DEL "<cache-key>"
- Or: docker compose exec app php occ user_oidc:provider keycloak --clientid nextcloud

Related:
- ADR-002: Vector sync background jobs authentication
- Validates external IdP integration pattern
- Demonstrates offline_access with refresh tokens (Tier 1 & 2)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:20 +01:00
Chris Coutinho b3725dd2f5 test: Remove --headed from pytest addopts 2025-11-02 22:03:20 +01:00
Chris Coutinho 6117aaaed3 fix: Complete Keycloak external IdP integration with all tests passing
This commit completes the Keycloak external identity provider integration,
implementing the ADR-002 architecture where Keycloak acts as an external
OAuth/OIDC provider and Nextcloud validates tokens via the user_oidc app.

Architecture:
  MCP Client → Keycloak (OAuth) → MCP Server → Nextcloud user_oidc → APIs

Key Fixes:

1. Keycloak JWT token configuration
   - Added 'sub' claim protocol mapper to realm-export.json
   - Updated token_verifier.py to accept both 'sub' and 'preferred_username'
   - Ensures tokens contain required OIDC claims

2. Keycloak hostname configuration for Docker networking
   - Implemented --hostname-backchannel-dynamic=true in docker-compose.yml
   - External clients use localhost:8888 (public)
   - Internal services use keycloak:8080 (Docker network)
   - Same issuer (localhost:8888) everywhere for token consistency
   - Restored frontendUrl in realm attributes

3. MCP server provider mode detection
   - Fixed URL normalization to handle port differences (http://app vs http://app:80)
   - Correctly distinguishes integrated mode vs external IdP mode
   - Removes explicit default ports (80 for HTTP, 443 for HTTPS)

4. Nextcloud SSRF protection configuration
   - Added allow_local_remote_servers=true to user_oidc install script
   - Enables Nextcloud to fetch JWKS from internal Keycloak container
   - Required for external IdP token validation

5. OAuth lifespan cleanup
   - Fixed RefreshTokenStorage close() error (uses context managers)
   - Added safe cleanup for oauth_client with hasattr check
   - Prevents session crash on shutdown

6. Test suite fixes
   - Fixed test_user_auto_provisioning to reflect actual behavior
   - Fixed test_scope_filtering_with_keycloak tool name (nc_webdav_write_file)
   - Updated test_keycloak_oauth_client_credentials_discovery for hostname config
   - All 11 Keycloak external IdP tests now passing

Testing:
   All 11 tests in test_keycloak_external_idp.py passing
   OAuth token acquisition via Playwright automation
   Token validation through Nextcloud user_oidc app
   Write operations (Notes create, Calendar create, File upload)
   Read operations (search, list, get)
   Token persistence across multiple operations
   User authentication and bearer token validation
   Scope-based tool filtering
   Error handling for invalid operations

Implementation validates:
  - ADR-002 external identity provider architecture
  - No admin credentials needed in MCP server
  - Centralized identity management via Keycloak
  - Standards-based OAuth 2.0 / OIDC integration
  - User auto-provisioning from IdP claims

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:20 +01:00
Chris Coutinho 403f8be429 feat: Add Keycloak external IdP integration with custom scopes
Add comprehensive support for using Keycloak as an external identity
provider with Nextcloud custom scopes. This enables testing of ADR-002
external IdP integration patterns.

**Keycloak Realm Configuration:**
- Add frontendUrl attribute to issue tokens with public issuer URL
- Define 18 Nextcloud custom client scopes (notes:read/write,
  calendar:read/write, contacts:read/write, cookbook:read/write,
  deck:read/write, tables:read/write, files:read/write,
  sharing:read/write, todo:read/write)
- Add all custom scopes to nextcloud-mcp-server client optional scopes
- Scopes include consent screen text for user-friendly OAuth flow

**MCP Server Configuration:**
- Add OIDC_JWKS_URI environment variable support
- Implement JWKS URI override logic for Docker networking
- Update NEXTCLOUD_PUBLIC_ISSUER_URL to include full realm path
- Enable MCP server to fetch JWKS from internal Docker network

**Test Infrastructure:**
- Add keycloak_oauth_client_credentials fixture (session-scoped)
- Add keycloak_oauth_token fixture with Playwright automation
- Implement PKCE (S256) support for Keycloak OAuth flow
- Add nc_mcp_keycloak_client fixture for MCP testing
- Create comprehensive test suite in test_keycloak_external_idp.py

**Tests Created:**
- test_keycloak_oauth_token_acquisition: Token acquisition via Playwright
- test_keycloak_oauth_client_credentials_discovery: OIDC discovery
- test_mcp_client_connects_to_keycloak_server: MCP connectivity
- test_external_idp_server_initialization: Server auto-detection
- test_external_idp_token_validation: Token validation flow
- test_tools_work_with_keycloak_token: End-to-end tool execution
- test_keycloak_token_persistence: Multi-operation token reuse
- test_user_auto_provisioning: Nextcloud user provisioning
- test_scope_filtering_with_keycloak: Scope-based tool filtering
- test_keycloak_error_handling: Error handling
- test_external_idp_architecture: Architecture documentation

**Current Status:**
-  Keycloak realm configuration complete
-  Custom scopes defined and available
-  OAuth token acquisition working (1 test passing)
- ⚠️  Token validation needs additional work (external IdP userinfo)

**Files Modified:**
- keycloak/realm-export.json: Realm configuration with scopes
- tests/conftest.py: Keycloak OAuth fixtures (+285 lines)
- tests/server/oauth/test_keycloak_external_idp.py: New test suite
- docker-compose.yml: OIDC_JWKS_URI and issuer configuration
- nextcloud_mcp_server/app.py: JWKS URI override logic

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:20 +01:00
Chris Coutinho 2a1274d8a8 refactor: Unify OAuth configuration to be provider-agnostic
Replace provider-specific environment variables (OAUTH_PROVIDER, KEYCLOAK_*)
with generic OIDC_* variables that work with any OIDC-compliant provider.

**Key Changes:**
- Auto-detect provider mode from OIDC_DISCOVERY_URL issuer
  - External IdP mode: issuer ≠ NEXTCLOUD_HOST (Keycloak, Auth0, Okta, etc.)
  - Integrated mode: issuer = NEXTCLOUD_HOST (Nextcloud OIDC app)
- Unified OIDC discovery flow (single code path)
- Generic client credential loading (static or DCR)
- Simplified docker-compose.yml environment variables

**Environment Variables:**
BEFORE:
  OAUTH_PROVIDER=keycloak
  KEYCLOAK_URL=http://keycloak:8080
  KEYCLOAK_REALM=nextcloud-mcp
  KEYCLOAK_CLIENT_ID=...
  KEYCLOAK_DISCOVERY_URL=...

AFTER:
  OIDC_DISCOVERY_URL=http://keycloak:8080/realms/nextcloud-mcp/.well-known/...
  OIDC_CLIENT_ID=nextcloud-mcp-server
  OIDC_CLIENT_SECRET=...

**Benefits:**
- Works with any OIDC provider without code changes
- No manual provider selection needed
- Cleaner environment variable naming
- Reduced code duplication (~150 lines removed)

**Testing:**
 mcp-keycloak auto-detects external IdP mode
 Token exchange test passes with generic config
 Backward compatible - integrated mode still works

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:20 +01:00
Chris Coutinho e331544cee feat: Implement RFC 8693 token exchange for Keycloak (ADR-002 Tier 2)
Implements OAuth 2.0 Token Exchange (RFC 8693) enabling the MCP server to
exchange service account tokens for user-scoped tokens. This provides an
alternative to refresh tokens for background operations.

**Core Implementation:**
- Added `get_service_account_token()` method to KeycloakOAuthClient for
  client_credentials grant
- Added `exchange_token_for_user()` method implementing RFC 8693 token exchange
- Fixed Fernet encryption key handling in RefreshTokenStorage (was incorrectly
  base64 decoding already-encoded keys)
- Updated OAuth configuration to support offline_access scope and refresh token
  storage infrastructure

**Keycloak Configuration:**
- Enabled `serviceAccountsEnabled` in realm-export.json
- Added `token.exchange.grant.enabled` attribute
- Added `client.token.exchange.standard.enabled` attribute (required for
  Keycloak 26.2+ Standard Token Exchange V2)
- Fresh Keycloak imports now correctly enable token exchange

**Docker Compose:**
- Added TOKEN_ENCRYPTION_KEY and ENABLE_OFFLINE_ACCESS environment variables
- Created oauth-tokens volume for refresh token storage
- Configured both mcp-oauth and mcp-keycloak services

**Testing & Documentation:**
- Added tests/manual/test_token_exchange.py - Validates complete RFC 8693 flow
- Added tests/manual/test_nextcloud_impersonate.py - Documents session-based
  impersonation limitations
- Added docs/oauth-impersonation-findings.md - Comprehensive investigation
  findings and resolution documentation

**Verified Working:**
 Service account token acquisition (client_credentials grant)
 RFC 8693 token exchange for internal-to-internal tokens
 Exchanged tokens validate with Nextcloud APIs
 Keycloak 26.4.2 Standard Token Exchange V2 support

**Known Limitations:**
- User impersonation (requested_subject) requires Keycloak Legacy V1 with
  preview features
- Cross-client token exchange limited to same realm
- Refresh token storage infrastructure ready but unused (MCP protocol limitation)

Dependencies: aiosqlite>=0.20.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:19 +01:00
Chris Coutinho 37b0b4a281 fix: Update DCR token_type tests for OIDC app changes
The Nextcloud OIDC app has updated token_type parameter values:
- Changed from "Bearer" → "opaque" for opaque tokens
- Changed from "JWT" → "jwt" for JWT tokens

Updated test_dcr_token_type.py to use lowercase token_type values:
- token_type="jwt" for JWT-formatted tokens
- token_type="opaque" for opaque/bearer tokens

This fixes test failures where tests were using the old "Bearer" and
"JWT" (uppercase) values which are no longer recognized by the OIDC app.

Fixes test: test_dcr_respects_bearer_token_type

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:19 +01:00
Chris Coutinho f34366a260 feat: Add Keycloak OAuth provider support with refresh token storage
Implements Keycloak as an external OIDC provider following ADR-002
architecture for background job authentication using offline_access.

## Features

- Keycloak OAuth provider with PKCE and offline_access support
- Refresh token storage with Fernet encryption
- Token verifier for both JWT and opaque tokens
- Multi-client validation (realm-level trust)
- Sample configuration for Keycloak integration

## Implementation

### OAuth Provider (keycloak_oauth.py)
- Authorization Code Flow with PKCE
- Refresh token exchange
- OIDC discovery endpoint support
- Token validation with JWKS

### Token Storage (refresh_token_storage.py)
- Encrypted storage using Fernet symmetric encryption
- SQLite backend for persistence
- Token rotation support
- Per-user token management

### Token Verifier Updates
- Support both JWT (self-encoded) and opaque tokens
- JWKS-based JWT signature verification
- Introspection endpoint fallback for opaque tokens
- Scope extraction from both token types

### Configuration
- .env.keycloak.sample: Example configuration with Keycloak URLs
- docs/keycloak-multi-client-validation.md: Realm-level validation documentation
- app-hooks/post-installation/10-install-user_oidc-app.sh: Updated dependencies

## Architecture Notes

- MCP Server is a protected resource (requires OAuth)
- MCP Client initiates OAuth flow and shares refresh tokens
- Refresh tokens enable background operations without admin credentials
- Supports future token exchange delegation when Keycloak implements it

## References

- ADR-002: Vector Database Background Sync Authentication
- RFC 6749: OAuth 2.0 (offline_access, refresh tokens)
- RFC 7517: JSON Web Key (JWK)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:19 +01:00
Chris Coutinho 529dc4616b docs: Implement separate clients architecture for Keycloak integration
Implements proper OAuth 2.0 separation following RFC 8707 best practices
with distinct resource server and OAuth client configurations.

## Architecture Changes

- Create separate "nextcloud" bearer-only client (resource server)
- Configure "nextcloud-mcp-server" OAuth client with audience mapper
- Audience mapper targets "nextcloud" resource server
- Token flow: aud="nextcloud", azp="nextcloud-mcp-server"

## Benefits

- Proper OAuth client vs resource server separation
- Support for future multi-resource tokens: aud=["nextcloud", "other-service"]
- RFC 8707 Resource Indicators compliance
- Clear requester identification via azp claim

## Documentation Updates

- Correct OAuth flow: MCP Client initiates, handles redirect, shares tokens
- Explain MCP Server as protected resource architecture
- Document offline_access with refresh tokens (Tier 1, current)
- Document token exchange with delegation (Tier 2, future when Keycloak adds support)
- Reference Keycloak issue #38279 for delegation status

## Files

- keycloak/realm-export.json: Add separate clients configuration
- app-hooks/post-installation/15-setup-keycloak-provider.sh: Setup user_oidc with "nextcloud" client
- docs/audience-validation-setup.md: Comprehensive documentation with corrected OAuth flow and delegation comparison
- docker-compose.yml: Fix Keycloak healthcheck (bash TCP instead of curl)
- scripts/test_separate_clients.sh: Verification script for architecture

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 22:03:19 +01:00
Chris Coutinho f739330341 ci: fix typo 2025-11-02 22:03:19 +01:00
Chris Coutinho 136df2422b build: Add keykloak to docker-compose.yml 2025-11-02 22:03:19 +01:00
Chris Coutinho eb8ca92bca Merge pull request #252 from cbcoutinho/renovate/ghcr.io-astral-sh-uv-0.x
chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.9.7
2025-10-31 22:32:43 +01:00
Chris Coutinho 0f03541486 Merge branch 'master' of github.com:cbcoutinho/nextcloud-mcp-server 2025-10-31 02:59:53 +01:00
Chris Coutinho ef07b1a6c9 docs: Add ADRs 2025-10-31 02:59:44 +01:00
Chris Coutinho 4f82357f24 ci: update submodule 2025-10-31 02:59:35 +01:00
renovate-bot-cbcoutinho[bot] 9ef2311c71 chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.9.7 2025-10-30 23:08:17 +00:00
Chris Coutinho c4293b6750 Merge pull request #251 from cbcoutinho/renovate/docker.io-library-nginx-alpine
chore(deps): update docker.io/library/nginx:alpine docker digest to b3c656d
2025-10-30 20:23:52 +01:00
renovate-bot-cbcoutinho[bot] 72e4eb3d19 chore(deps): update docker.io/library/nginx:alpine docker digest to b3c656d 2025-10-30 17:06:28 +00:00
Chris Coutinho 47dd2df7aa Merge pull request #250 from cbcoutinho/renovate/ghcr.io-astral-sh-uv-0.x
chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.9.6
2025-10-30 12:55:02 +01:00
renovate-bot-cbcoutinho[bot] 9fd2022151 chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.9.6 2025-10-29 23:07:53 +00:00
Chris Coutinho b99dc52c95 docs: Update README with instructions on helm install 2025-10-29 12:47:20 +01:00
Chris Coutinho 78b27fb5e9 Merge pull request #249 from cbcoutinho/renovate/actions-checkout-5.x
chore(deps): update actions/checkout action to v5
2025-10-29 12:42:59 +01:00
renovate-bot-cbcoutinho[bot] 03e39a3f94 chore(deps): update actions/checkout action to v5 2025-10-29 11:28:09 +00:00
github-actions[bot] 5259658458 bump: version 0.22.6 → 0.22.7 nextcloud-mcp-server-0.22.7 v0.22.7 2025-10-29 11:18:41 +00:00
Chris Coutinho e03a3c2e83 fix(helm): Remove image tag overide 2025-10-29 12:18:12 +01:00
Chris Coutinho 94cbd3015d Merge pull request #248 from cbcoutinho/renovate/pin-dependencies
chore(deps): pin dependencies
2025-10-29 12:14:10 +01:00
renovate-bot-cbcoutinho[bot] 49a961cbcc chore(deps): pin dependencies 2025-10-29 11:06:51 +00:00
github-actions[bot] e1aca04aff bump: version 0.22.5 → 0.22.6 nextcloud-mcp-server-0.22.6 v0.22.6 2025-10-29 10:57:44 +00:00