diff --git a/docs/jwt-oauth-reference.md b/docs/jwt-oauth-reference.md new file mode 100644 index 0000000..b071fac --- /dev/null +++ b/docs/jwt-oauth-reference.md @@ -0,0 +1,761 @@ +# JWT OAuth Reference - Nextcloud MCP Server + +**Last Updated:** 2025-10-23 +**Status:** Production Ready + +## Table of Contents + +- [Overview](#overview) +- [JWT vs Opaque Tokens](#jwt-vs-opaque-tokens) +- [Scope-Based Authorization](#scope-based-authorization) +- [Configuration](#configuration) +- [Architecture](#architecture) +- [Testing](#testing) +- [Troubleshooting](#troubleshooting) +- [Production Deployment](#production-deployment) + +--- + +## Overview + +The Nextcloud MCP Server supports OAuth authentication with both **JWT** (RFC 9068) and **opaque** tokens. JWT tokens are recommended for production use as they enable: + +- **Faster validation** - No HTTP call needed for token verification +- **Direct scope extraction** - Scopes embedded in token claims +- **Dynamic tool filtering** - Users only see tools they have permission to use +- **Signature verification** - Cryptographic validation using JWKS + +### Key Features + +✅ **JWT Token Support** - RFC 9068 compliant access tokens with RS256 signatures +✅ **Custom Scopes** - `nc:read` and `nc:write` for read/write access control +✅ **Dynamic Tool Filtering** - Tools filtered based on user's token scopes +✅ **Automatic Client Creation** - JWT OAuth clients auto-generated on container startup +✅ **Scope Challenges** - RFC-compliant `WWW-Authenticate` headers for insufficient scopes +✅ **Protected Resource Metadata** - RFC 8959 endpoint for scope discovery +✅ **Backward Compatible** - BasicAuth mode bypasses all scope checks + +### Supported Scopes + +| Scope | Description | Tool Count | +|-------|-------------|------------| +| `nc:read` | Read-only access to Nextcloud data | 36 tools | +| `nc:write` | Write access to create/modify/delete data | 54 tools | + +All MCP tools (90 total) require at least one of these scopes. Standard OIDC scopes (`openid`, `profile`, `email`) are also supported. + +--- + +## JWT vs Opaque Tokens + +The Nextcloud OIDC app supports two token formats, configured per-client: + +### JWT Tokens (Recommended) + +**Advantages:** +- ✅ Fast validation - JWT signature verified locally using JWKS +- ✅ Direct scope extraction from `scope` claim in payload +- ✅ Standard approach (RFC 9068) +- ✅ No additional HTTP calls for validation + +**Disadvantages:** +- ⚠️ Larger size (~800-1200 chars vs 72 chars for opaque) +- ⚠️ Token payload visible to client (not an issue for access tokens) + +**Token Structure:** +```json +{ + "header": { + "typ": "at+JWT", + "alg": "RS256", + "kid": "..." + }, + "payload": { + "iss": "http://localhost:8080", + "sub": "admin", + "aud": "client_id", + "exp": 1234567890, + "iat": 1234567890, + "scope": "openid profile email nc:read nc:write", + "client_id": "...", + "jti": "..." + } +} +``` + +### Opaque Tokens + +**Advantages:** +- ✅ Smaller size (72 characters) +- ✅ No payload visible to client + +**Disadvantages:** +- ❌ Requires userinfo endpoint call for each validation +- ❌ Scopes must be inferred from userinfo response (not directly available) +- ❌ Higher latency (HTTP call required) + +**When to Use:** +- Use **JWT tokens** for production (better performance, direct scope access) +- Use **opaque tokens** for compatibility with clients that don't support JWT + +--- + +## Scope-Based Authorization + +### Scope Definitions + +The MCP server uses **coarse-grained scopes** for simplicity: + +| Scope | Operations | Examples | +|-------|------------|----------| +| `nc:read` | Read-only access | Get notes, search files, list calendars, read contacts | +| `nc:write` | Write operations | Create notes, update events, delete files, modify contacts | + +### Standard OIDC Scopes + +| Scope | Description | Required | +|-------|-------------|----------| +| `openid` | OIDC authentication | Yes | +| `profile` | User profile information | Recommended | +| `email` | Email address | Recommended | + +### Recommended Configurations + +**Full Access:** +``` +openid profile email nc:read nc:write +``` + +**Read-Only:** +``` +openid profile email nc:read +``` + +**No Custom Scopes (OIDC only):** +``` +openid profile email +``` + +### Implementation + +All 90 MCP tools are decorated with scope requirements: + +```python +@mcp.tool() +@require_scopes("nc:read") +async def nc_notes_get_note(note_id: int, ctx: Context): + """Get a note by ID (requires nc:read scope)""" + ... + +@mcp.tool() +@require_scopes("nc:write") +async def nc_notes_create_note(title: str, content: str, ctx: Context): + """Create a note (requires nc:write scope)""" + ... +``` + +**Coverage:** +- ✅ 36 read tools decorated with `@require_scopes("nc:read")` +- ✅ 54 write tools decorated with `@require_scopes("nc:write")` +- ✅ 90/90 tools covered (100%) + +### Dynamic Tool Filtering + +The MCP server implements **dynamic tool filtering** - users only see tools they have permission to use: + +**JWT with `nc:read` only:** +- `list_tools()` returns 36 read-only tools +- Write tools are hidden from the tool list + +**JWT with `nc:write` only:** +- `list_tools()` returns 54 write-only tools +- Read tools are hidden from the tool list + +**JWT with both scopes:** +- `list_tools()` returns all 90 tools + +**JWT with no custom scopes:** +- `list_tools()` returns 0 tools (all require `nc:read` or `nc:write`) + +**BasicAuth mode:** +- `list_tools()` returns all 90 tools (no filtering) + +### Scope Challenges + +When a tool is called without required scopes, the server returns a `403 Forbidden` response with a `WWW-Authenticate` header: + +```http +HTTP/1.1 403 Forbidden +WWW-Authenticate: Bearer error="insufficient_scope", + scope="nc:write", + resource_metadata="http://server/.well-known/oauth-protected-resource" +``` + +This enables **step-up authorization** - clients can detect missing scopes and trigger re-authentication to obtain additional permissions. + +### Protected Resource Metadata (PRM) + +The server implements RFC 8959's Protected Resource Metadata endpoint: + +**Endpoint:** `GET /.well-known/oauth-protected-resource` + +**Response:** +```json +{ + "resource": "http://localhost:8002", + "scopes_supported": ["nc:read", "nc:write"], + "authorization_servers": ["http://localhost:8080"], + "bearer_methods_supported": ["header"], + "resource_signing_alg_values_supported": ["RS256"] +} +``` + +This allows OAuth clients to discover supported scopes before requesting authorization. + +--- + +## Configuration + +### Docker Services + +The development environment includes three MCP server variants: + +| Service | Port | Auth Type | Token Type | Use Case | +|---------|------|-----------|------------|----------| +| `mcp` | 8000 | BasicAuth | N/A | Development, testing | +| `mcp-oauth` | 8001 | OAuth | Opaque | Standard OAuth flows | +| `mcp-oauth-jwt` | 8002 | OAuth | JWT | Production, JWT testing | + +### JWT Service Configuration + +The `mcp-oauth-jwt` service uses automatic client creation: + +```yaml +mcp-oauth-jwt: + build: . + command: ["--transport", "streamable-http", "--oauth", "--port", "8002"] + ports: + - 127.0.0.1:8002:8002 + environment: + - NEXTCLOUD_HOST=http://app:80 + - NEXTCLOUD_MCP_SERVER_URL=http://localhost:8002 + - NEXTCLOUD_PUBLIC_ISSUER_URL=http://localhost:8080 + - NEXTCLOUD_OIDC_CLIENT_STORAGE=/var/www/html/.oauth-jwt/nextcloud_oauth_client.json + - NEXTCLOUD_OIDC_SCOPES=openid profile email nc:read nc:write + - NEXTCLOUD_OIDC_TOKEN_TYPE=jwt + volumes: + - nextcloud:/var/www/html:ro +``` + +**Key Points:** +- No `NEXTCLOUD_OIDC_CLIENT_ID` or `CLIENT_SECRET` needed (loaded from storage) +- JWT client auto-created during Nextcloud initialization +- Credentials stored in `/var/www/html/.oauth-jwt/nextcloud_oauth_client.json` +- Volume mounted read-only for security + +### Environment Variables + +| Variable | Description | Default | +|----------|-------------|---------| +| `NEXTCLOUD_HOST` | Nextcloud base URL | `http://localhost:8080` | +| `NEXTCLOUD_MCP_SERVER_URL` | MCP server external URL | (required) | +| `NEXTCLOUD_PUBLIC_ISSUER_URL` | Public issuer URL for JWT validation | (uses `NEXTCLOUD_HOST`) | +| `NEXTCLOUD_OIDC_CLIENT_STORAGE` | Path to client credentials JSON | (optional - uses DCR if unset) | +| `NEXTCLOUD_OIDC_SCOPES` | Space-separated scopes to request | `"openid profile email nc:read nc:write"` | +| `NEXTCLOUD_OIDC_TOKEN_TYPE` | Token format: `"jwt"` or `"Bearer"` | `"Bearer"` | + +### Manual Client Creation + +If not using automatic creation, create a JWT client manually: + +```bash +docker compose exec app php occ oidc:create \ + --token_type=jwt \ + --allowed_scopes="openid profile email nc:read nc:write" \ + "Nextcloud MCP Server" \ + "http://localhost:8002/oauth/callback" +``` + +**Output:** +```json +{ + "client_id": "...", + "client_secret": "...", + "token_type": "jwt", + "allowed_scopes": "openid profile email nc:read nc:write" +} +``` + +Then configure the MCP server: +```bash +export NEXTCLOUD_OIDC_CLIENT_ID="" +export NEXTCLOUD_OIDC_CLIENT_SECRET="" +export NEXTCLOUD_OIDC_TOKEN_TYPE="jwt" +``` + +--- + +## Architecture + +### Component Overview + +``` +┌──────────────────┐ OAuth Flow ┌──────────────────┐ +│ OAuth Client │<─────────────────────>│ Nextcloud OIDC │ +│ (Claude, etc) │ │ Server │ +└────────┬─────────┘ └────────┬─────────┘ + │ │ + │ JWT Access Token │ + │ { │ + │ "scope": "openid nc:read nc:write" │ + │ ... │ + │ } │ + │ │ + v │ +┌────────────────────────────────────────────────────────────┐ +│ Nextcloud MCP Server │ +│ ┌───────────────────────────────────────────────────┐ │ +│ │ NextcloudTokenVerifier │ │ +│ │ - JWT signature verification (JWKS) │ │ +│ │ - Scope extraction from token payload │ │ +│ │ - Fallback to userinfo endpoint (opaque tokens) │ │ +│ └───────────────────┬───────────────────────────────┘ │ +│ │ │ +│ v │ +│ ┌───────────────────────────────────────────────────┐ │ +│ │ Dynamic Tool Filtering (list_tools) │ │ +│ │ - Get user scopes from verified token │ │ +│ │ - Filter tools based on @require_scopes metadata │ │ +│ │ - Return only accessible tools │ │ +│ └───────────────────┬───────────────────────────────┘ │ +│ │ │ +│ v │ +│ ┌───────────────────────────────────────────────────┐ │ +│ │ Tool Execution (@require_scopes decorator) │ │ +│ │ - Check token scopes before execution │ │ +│ │ - Raise InsufficientScopeError if missing │ │ +│ │ - Return 403 with WWW-Authenticate header │ │ +│ └───────────────────────────────────────────────────┘ │ +└────────────────────────────────────────────────────────────┘ +``` + +### Key Components + +**1. Token Verification** (`nextcloud_mcp_server/auth/token_verifier.py`) +- JWT signature verification using JWKS (RS256) +- Scope extraction from `scope` claim +- Token caching with TTL +- Fallback to userinfo endpoint for opaque tokens + +**2. Scope Authorization** (`nextcloud_mcp_server/auth/scope_authorization.py`) +- `@require_scopes()` decorator for tools +- `get_required_scopes()` - Extract scope requirements from functions +- `has_required_scopes()` - Check if user has necessary scopes +- `InsufficientScopeError` exception for WWW-Authenticate challenges + +**3. Dynamic Filtering** (`nextcloud_mcp_server/app.py:433-488`) +- Overrides FastMCP's `list_tools()` method +- Filters based on user's JWT token scopes +- Only active in OAuth mode +- Bypassed in BasicAuth mode + +**4. PRM Endpoint** (`nextcloud_mcp_server/app.py:503-532`) +- `GET /.well-known/oauth-protected-resource` +- Advertises `["nc:read", "nc:write"]` +- RFC 8959 compliant + +**5. Exception Handler** (`nextcloud_mcp_server/app.py:540-563`) +- Catches `InsufficientScopeError` +- Returns 403 with `WWW-Authenticate` header +- Includes missing scopes and PRM endpoint URL + +### Automatic Client Creation + +**Post-Installation Hook:** `app-hooks/post-installation/90-create-jwt-oauth-client.sh` + +On container startup, this script: +1. Installs/enables OIDC app +2. Creates JWT client via `occ oidc:create --token_type=jwt` +3. Parses JSON output for credentials +4. Saves to `/var/www/html/.oauth-jwt/nextcloud_oauth_client.json` + +**Credential Format:** +```json +{ + "client_id": "XBd2xqIisu3Kswg39Ub4BUhC36PEYjwwivx3G5nZdDgigvwKXrTHozs7m9DeoLSY", + "client_secret": "xNKcy0qpUSau36T60pGGdb03pMEVLXtqykxjK8YkDpoNxNcZ4ClyAT3IAEse2AKT", + "client_id_issued_at": 1761097039, + "client_secret_expires_at": 2076457039, + "redirect_uris": ["http://localhost:8002/oauth/callback"] +} +``` + +--- + +## Testing + +### Test Infrastructure + +The test suite includes comprehensive coverage for JWT OAuth and scope authorization: + +**Test Files:** +- `tests/server/test_scope_authorization.py` - Scope-based authorization tests (4 tests) +- `tests/server/test_mcp_oauth_jwt.py` - JWT OAuth integration tests +- `tests/conftest.py` - Shared fixtures for JWT testing + +### Consent Scenario Tests + +Four test scenarios verify scope-based tool filtering with different consent levels: + +#### 1. No Custom Scopes (0 tools) +```bash +uv run pytest tests/server/test_scope_authorization.py::test_jwt_with_no_custom_scopes_returns_zero_tools -v +``` + +**Scenario:** JWT token with only OIDC defaults (`openid profile email`) +**Expected:** 0 tools returned (all require `nc:read` or `nc:write`) +**Verifies:** Security - users who decline custom scopes cannot access any MCP tools + +#### 2. Read-Only Access (36 tools) +```bash +uv run pytest tests/server/test_scope_authorization.py::test_jwt_consent_scenarios_read_only -v +``` + +**Scenario:** JWT token with `nc:read` only +**Expected:** 36 read-only tools visible, write tools hidden +**Verifies:** Read tools accessible, write tools filtered out + +#### 3. Write-Only Access (54 tools) +```bash +uv run pytest tests/server/test_scope_authorization.py::test_jwt_consent_scenarios_write_only -v +``` + +**Scenario:** JWT token with `nc:write` only +**Expected:** 54 write tools visible, read tools hidden +**Verifies:** Write tools accessible, read tools filtered out + +#### 4. Full Access (90 tools) +```bash +uv run pytest tests/server/test_scope_authorization.py::test_jwt_consent_scenarios_full_access -v +``` + +**Scenario:** JWT token with both `nc:read` and `nc:write` +**Expected:** All 90 tools visible +**Verifies:** Full access when user grants all custom scopes + +### Test Fixtures + +**OAuth Client Fixtures:** +- `read_only_oauth_client_credentials` - Client with `nc:read` only +- `write_only_oauth_client_credentials` - Client with `nc:write` only +- `full_access_oauth_client_credentials` - Client with both scopes +- `no_custom_scopes_oauth_client_credentials` - Client with OIDC defaults only + +**Token Fixtures:** +- `playwright_oauth_token_read_only` - Obtains token with `nc:read` +- `playwright_oauth_token_write_only` - Obtains token with `nc:write` +- `playwright_oauth_token_full_access` - Obtains token with both scopes +- `playwright_oauth_token_no_custom_scopes` - Obtains token with no custom scopes + +**MCP Client Fixtures:** +- `nc_mcp_oauth_client_read_only` - MCP session with read-only token +- `nc_mcp_oauth_client_write_only` - MCP session with write-only token +- `nc_mcp_oauth_client_full_access` - MCP session with full access token +- `nc_mcp_oauth_client_no_custom_scopes` - MCP session with no custom scopes + +### Running Tests + +**All consent scenario tests:** +```bash +uv run pytest tests/server/test_scope_authorization.py -v +``` + +**JWT OAuth integration tests:** +```bash +uv run pytest tests/server/test_mcp_oauth_jwt.py -v --browser firefox +``` + +**With visible browser (debugging):** +```bash +uv run pytest tests/server/test_mcp_oauth_jwt.py -v --browser firefox --headed +``` + +### Test Configuration + +**Playwright Browser:** +- Default: Chromium +- Recommended for CI: Firefox (`--browser firefox`) +- Debugging: Add `--headed` flag + +**OAuth Flow:** +- Uses automated Playwright browser automation +- Completes OAuth consent flow programmatically +- Creates separate OAuth client for each scenario +- Each user gets unique access token + +--- + +## Troubleshooting + +### Issue: JWT Issuer Validation Failed + +**Symptom:** +``` +WARNING JWT issuer validation failed: Invalid issuer +WARNING JWT verification failed, will try other methods +✅ Extracted scopes from access token: {'openid', 'profile'} +``` + +**Cause:** Token's `iss` claim doesn't match expected issuer URL. This often happens when: +- Using `localhost` vs `127.0.0.1` inconsistently +- MCP server uses internal URL but clients use public URL + +**Solution:** +```bash +# Option 1: Use consistent URLs +export NEXTCLOUD_PUBLIC_ISSUER_URL=http://localhost:8080 +# Ensure all test fixtures also use localhost:8080 + +# Option 2: Check discovery document +curl http://localhost:8080/.well-known/openid-configuration | jq .issuer +# Use this exact issuer in NEXTCLOUD_PUBLIC_ISSUER_URL +``` + +**Impact if not fixed:** +- JWT validation falls back to userinfo endpoint +- Scopes inferred from userinfo (only standard OIDC scopes, no custom scopes) +- Result: 0 tools visible or incorrect tool filtering + +### Issue: Scopes Not Present in JWT + +**Symptom:** JWT token doesn't contain `scope` claim or contains empty string + +**Cause:** Client's `allowed_scopes` is empty or not configured + +**Solution:** +```bash +# Check client configuration +docker compose exec app php occ oidc:list + +# Look for allowed_scopes in output +# If empty, recreate client with --allowed_scopes +docker compose exec app php occ oidc:create \ + --token_type=jwt \ + --allowed_scopes="openid profile email nc:read nc:write" \ + "Client Name" \ + "http://callback/url" +``` + +### Issue: All Tools Visible Despite Read-Only Token + +**Symptom:** User with `nc:read` token can see all 90 tools including write tools + +**Cause:** Server running in BasicAuth mode, not OAuth mode + +**Solution:** +```bash +# Verify OAuth mode is active +docker compose logs mcp-oauth-jwt | grep "OAuth mode" + +# Should see: "Running in OAuth mode" + +# If not, check environment variables: +docker compose exec mcp-oauth-jwt env | grep NEXTCLOUD_OIDC + +# Ensure no NEXTCLOUD_USERNAME or NEXTCLOUD_PASSWORD set +``` + +### Issue: Dynamic Client Registration Doesn't Set Scopes + +**Symptom:** Client registered via DCR has empty `allowed_scopes` + +**Cause:** OIDC app's registration endpoint doesn't populate `allowed_scopes` from request + +**Workaround:** Use pre-configured clients instead of DCR for JWT tokens: +```bash +# Create client manually via occ +docker compose exec app php occ oidc:create \ + --token_type=jwt \ + --allowed_scopes="openid profile email nc:read nc:write" \ + "Client Name" \ + "http://callback" + +# Configure MCP server to use these credentials +export NEXTCLOUD_OIDC_CLIENT_ID="..." +export NEXTCLOUD_OIDC_CLIENT_SECRET="..." +``` + +### Issue: Token Type Case Sensitivity + +**Symptom:** JWT tokens not generated even though `token_type=JWT` set + +**Cause:** OIDC app checks `token_type === 'jwt'` (lowercase) + +**Solution:** Always use lowercase: +```bash +# Correct +export NEXTCLOUD_OIDC_TOKEN_TYPE=jwt + +# Incorrect (will generate opaque tokens) +export NEXTCLOUD_OIDC_TOKEN_TYPE=JWT +``` + +### Issue: Missing WWW-Authenticate Header + +**Symptom:** 403 error doesn't include `WWW-Authenticate` header + +**Cause:** Server not in OAuth mode, or exception not being caught + +**Solution:** +```bash +# Check server logs for OAuth mode +docker compose logs mcp-oauth-jwt | grep "WWW-Authenticate scope challenges enabled" + +# Should see this during startup + +# Check exception handling +docker compose logs mcp-oauth-jwt | grep "InsufficientScopeError" +``` + +### Debugging Tools + +**Check JWT contents:** +```bash +# Decode JWT (base64 decode the payload) +echo "JWT_PAYLOAD_PART" | base64 -d | jq . +``` + +**Check database scopes:** +```bash +# View access tokens with scopes +docker compose exec db mariadb -u nextcloud -ppassword nextcloud \ + -e "SELECT id, client_id, user_id, scope FROM oc_oidc_access_tokens ORDER BY id DESC LIMIT 5;" + +# View user consents +docker compose exec db mariadb -u nextcloud -ppassword nextcloud \ + -e "SELECT user_id, client_id, scopes_granted FROM oc_oidc_user_consents;" +``` + +**Check server logs:** +```bash +# Follow JWT verification logs +docker compose logs -f mcp-oauth-jwt | grep -E "JWT|scope|tool" + +# Check for issuer mismatches +docker compose logs mcp-oauth-jwt | grep -i issuer +``` + +--- + +## Production Deployment + +### Deployment Checklist + +✅ **Use JWT Tokens** - Enable `token_type=jwt` for better performance +✅ **Configure Allowed Scopes** - Always set `allowed_scopes` on OAuth clients +✅ **Use Pre-Configured Clients** - Avoid DCR limitation with manual client creation +✅ **Consistent URLs** - Use same URL for `NEXTCLOUD_HOST` and `PUBLIC_ISSUER_URL` +✅ **Secure Credentials** - Store client credentials securely (environment variables or secrets management) +✅ **Monitor Token Size** - JWT tokens are 10-15x larger than opaque (not usually an issue) +✅ **Enable Logging** - Configure appropriate log levels for JWT verification + +### Production Configuration Example + +```yaml +# docker-compose.yml (production) +mcp-oauth-jwt: + image: ghcr.io/yourusername/nextcloud-mcp-server:latest + environment: + - NEXTCLOUD_HOST=https://nextcloud.example.com + - NEXTCLOUD_MCP_SERVER_URL=https://mcp.example.com + - NEXTCLOUD_PUBLIC_ISSUER_URL=https://nextcloud.example.com + - NEXTCLOUD_OIDC_CLIENT_ID=${JWT_CLIENT_ID} + - NEXTCLOUD_OIDC_CLIENT_SECRET=${JWT_CLIENT_SECRET} + - NEXTCLOUD_OIDC_SCOPES=openid profile email nc:read nc:write + - NEXTCLOUD_OIDC_TOKEN_TYPE=jwt + ports: + - "8002:8002" +``` + +### Security Considerations + +**Token Storage:** +- Never commit credentials to version control +- Use environment variables or secrets management +- Rotate client secrets periodically + +**Scope Configuration:** +- Grant minimum necessary scopes to clients +- Use read-only tokens for AI assistants that don't need write access +- Review OAuth client list regularly + +**Network Security:** +- Use HTTPS in production +- Ensure issuer URL matches public URL +- Configure proper CORS headers + +### Monitoring + +**Key Metrics:** +- JWT verification success/failure rate +- Scope challenge frequency (indicates clients with insufficient scopes) +- Token validation latency +- Tool execution by scope (identify unused scopes) + +**Log Patterns:** +```bash +# Success +INFO JWT verified successfully for user: admin +INFO ✅ Extracted scopes from access token: {'openid', 'profile', 'email', 'nc:read', 'nc:write'} + +# Failures +WARNING JWT issuer validation failed: Invalid issuer +WARNING Missing required scopes: nc:write +``` + +### Known Limitations + +1. **No Fine-Grained Scopes** - Only coarse `nc:read` and `nc:write` (not per-app scopes) +2. **DCR Doesn't Set Allowed Scopes** - Must use pre-configured clients for JWT +3. **No Introspection Endpoint** - OIDC app lacks RFC 7662 introspection +4. **No Refresh Token Support** - Tokens must be reacquired when expired + +### Future Enhancements + +**Potential Improvements:** +- Per-app scopes (`nc:notes:read`, `nc:calendar:write`) +- Resource-level filtering (apply to MCP resources, not just tools) +- Automatic scope discovery from decorated tools +- Admin UI for scope management +- Token introspection endpoint (RFC 7662) + +--- + +## References + +### Standards + +- [RFC 9068: JWT Profile for OAuth 2.0 Access Tokens](https://www.rfc-editor.org/rfc/rfc9068.html) +- [RFC 7519: JSON Web Token (JWT)](https://www.rfc-editor.org/rfc/rfc7519.html) +- [RFC 7517: JSON Web Key (JWK)](https://www.rfc-editor.org/rfc/rfc7517.html) +- [RFC 8959: Protected Resource Metadata](https://www.rfc-editor.org/rfc/rfc8959.html) +- [RFC 7662: OAuth 2.0 Token Introspection](https://www.rfc-editor.org/rfc/rfc7662.html) + +### Related Documentation + +- [OAuth Setup Guide](oauth-setup.md) - Complete OAuth configuration guide +- [OAuth Architecture](oauth-architecture.md) - Detailed architecture documentation +- [OAuth Troubleshooting](oauth-troubleshooting.md) - Common OAuth issues and solutions +- [Authentication Guide](authentication.md) - BasicAuth vs OAuth comparison + +### External Resources + +- [Nextcloud OIDC App](https://github.com/H2CK/oidc) - OIDC identity provider for Nextcloud +- [PyJWT Documentation](https://pyjwt.readthedocs.io/) - JWT library used for verification +- [FastMCP Documentation](https://github.com/jlowin/fastmcp) - MCP server framework + +--- + +**Implementation Date:** 2025-10-21 to 2025-10-23 +**Version:** 1.0.0 +**Status:** ✅ Production Ready diff --git a/docs/testing-client-sessions-architecture.md b/docs/testing-client-sessions-architecture.md index 6347216..a5ad0ee 100644 --- a/docs/testing-client-sessions-architecture.md +++ b/docs/testing-client-sessions-architecture.md @@ -41,7 +41,7 @@ async def nc_mcp_client() -> AsyncGenerator[ClientSession, Any]: """Fixture with surgical exception handling for pytest-asyncio incompatibility.""" try: async for session in create_mcp_client_session( - url="http://127.0.0.1:8000/mcp", client_name="Basic MCP" + url="http://localhost:8000/mcp", client_name="Basic MCP" ): yield session except RuntimeError as e: @@ -88,7 +88,7 @@ async def nc_mcp_client() -> AsyncGenerator[ClientSession, Any]: async def create_and_hold_session(): """Runs in isolated task - creates session and keeps it alive.""" - async with streamablehttp_client("http://127.0.0.1:8000/mcp") as (read_stream, write_stream, _): + async with streamablehttp_client("http://localhost:8000/mcp") as (read_stream, write_stream, _): async with ClientSession(read_stream, write_stream) as session: await session.initialize() session_holder["session"] = session @@ -141,7 +141,7 @@ async def nc_mcp_client() -> AsyncGenerator[ClientSession, Any]: @pytest.fixture(scope="function") # Changed from session async def nc_mcp_client() -> AsyncGenerator[ClientSession, Any]: """Function-scoped fixture with natural LIFO cleanup.""" - async with streamablehttp_client("http://127.0.0.1:8000/mcp") as (read_stream, write_stream, _): + async with streamablehttp_client("http://localhost:8000/mcp") as (read_stream, write_stream, _): async with ClientSession(read_stream, write_stream) as session: await session.initialize() yield session @@ -150,11 +150,11 @@ async def nc_mcp_client() -> AsyncGenerator[ClientSession, Any]: @pytest.fixture(scope="function") async def multi_mcp_clients() -> AsyncGenerator[tuple[ClientSession, ClientSession], Any]: """Multiple clients with guaranteed LIFO cleanup through nesting.""" - async with streamablehttp_client("http://127.0.0.1:8000/mcp") as (read1, write1, _): + async with streamablehttp_client("http://localhost:8000/mcp") as (read1, write1, _): async with ClientSession(read1, write1) as session1: await session1.initialize() - async with streamablehttp_client("http://127.0.0.1:8001/mcp") as (read2, write2, _): + async with streamablehttp_client("http://localhost:8001/mcp") as (read2, write2, _): async with ClientSession(read2, write2) as session2: await session2.initialize() yield session1, session2 @@ -195,7 +195,7 @@ async def multi_mcp_clients() -> AsyncGenerator[tuple[ClientSession, ClientSessi # Fixtures work naturally with trio @pytest.fixture(scope="session") async def nc_mcp_client() -> AsyncGenerator[ClientSession, Any]: - async with streamablehttp_client("http://127.0.0.1:8000/mcp") as (read, write, _): + async with streamablehttp_client("http://localhost:8000/mcp") as (read, write, _): async with ClientSession(read, write) as session: await session.initialize() yield session diff --git a/docs/testing-oidc-consent.md b/docs/testing-oidc-consent.md new file mode 100644 index 0000000..4a00bdc --- /dev/null +++ b/docs/testing-oidc-consent.md @@ -0,0 +1,412 @@ +# Testing OIDC Consent Feature + +This guide explains how to test the OIDC consent feature using the development version of the OIDC app mounted into the Docker environment. + +## Setup + +### Volume Mount Configuration + +The development OIDC app is mounted from `~/Software/oidc` into the container at `/opt/apps/oidc`: + +```yaml +# docker-compose.yml +volumes: + - ../Software/oidc:/opt/apps/oidc:ro +``` + +**Why mount outside `/var/www/html/`?** +- The Nextcloud container uses `rsync` to initialize `/var/www/html/` from the image +- Mounting inside that path causes conflicts (rsync tries to delete mounted directories) +- Mounting to `/opt/apps/oidc` avoids rsync entirely +- Nextcloud supports multiple app directories via the `apps_paths` configuration + +**How multiple app paths work:** +- Nextcloud can load apps from multiple directories +- The post-installation hook registers `/opt/apps` as an additional app directory (index 2) +- Apps in default paths (index 0 and 1) are still available +- All directories are scanned for apps, but `/opt/apps` is read-only + +This setup allows you to: +- Test changes without rebuilding containers +- Avoid needing npm/node in the container (JS already built on host) +- Iterate quickly on development +- Install other Nextcloud apps normally (custom_apps remains writable) + +### How It Works + +1. **Mount Development App**: Docker mounts `~/Software/oidc` to `/opt/apps/oidc` (outside Nextcloud's path) +2. **Register App Path**: The `10-install-oidc-app.sh` hook configures `/opt/apps` as an additional app directory +3. **Enable App**: The hook enables the OIDC app from `/opt/apps/oidc` +4. **Run Migrations**: Nextcloud detects pending migrations and runs them automatically +5. **Configure OIDC**: Dynamic client registration and PKCE are enabled + +## Starting the Stack + +```bash +cd ~/Projects/nextcloud-mcp-server + +# Start fresh (recommended for first test) +docker compose down -v +docker compose up -d + +# Wait for initialization (check logs) +docker compose logs -f app +``` + +The post-installation hooks will: +1. Configure custom_apps path (already done) +2. Enable OIDC app from mounted directory +3. Run database migrations (including consent table creation) +4. Configure OIDC settings + +## Verifying Installation + +### Before Container Restart + +Before running `docker compose up -d`, the consent feature will NOT be active: +- ❌ No `oc_oidc_user_consents` table in database +- ❌ Migration 0015 not applied yet +- ❌ ConsentController class not loaded +- ❌ Consent routes not registered + +You can verify this with: +```bash +# Check migrations applied (should stop at 0014) +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e "SELECT version FROM oc_migrations WHERE app = 'oidc' ORDER BY version DESC LIMIT 3;" nextcloud + +# Check for consent table (should return empty) +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e "SHOW TABLES LIKE 'oc_oidc_user_consents';" nextcloud +``` + +### After Container Restart + +After `docker compose up -d` with the mounted OIDC directory, the consent feature should be active: +- ✅ `oc_oidc_user_consents` table exists +- ✅ Migration 0015 (Version0015Date20251123100100) applied +- ✅ ConsentController routes registered +- ✅ Consent screen appears during OAuth flows + +### Check App Status + +```bash +docker compose exec app php occ app:list | grep -A 2 oidc +``` + +Expected output: +``` + - oidc: 1.10.0 (enabled) +``` + +### Verify App Paths Configuration + +Verify that `/opt/apps` is registered as an additional app directory: + +```bash +# Check configured app paths +docker compose exec app php occ config:system:get apps_paths + +# Verify the mount is accessible +docker compose exec app ls -la /opt/apps/oidc/ + +# Verify custom_apps is writable (for normal app installation) +docker compose exec -u www-data app touch /var/www/html/custom_apps/.test && echo "✅ custom_apps is writable" || echo "❌ custom_apps NOT writable" +docker compose exec app rm -f /var/www/html/custom_apps/.test +``` + +Expected: Output should show multiple app paths including index 2 (/opt/apps). + +### Verify Consent Files + +```bash +# Check controller exists in mounted location +docker compose exec app ls -la /opt/apps/oidc/lib/Controller/ConsentController.php + +# Check Vue component exists +docker compose exec app ls -la /opt/apps/oidc/src/Consent.vue + +# Check built JS exists +docker compose exec app ls -lh /opt/apps/oidc/js/oidc-consent.js +``` + +### Verify Database Migration + +**Note**: These checks will only pass after restarting containers with the mounted OIDC app. + +```bash +# Check if consent table exists +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e "SHOW TABLES LIKE 'oc_oidc_user_consents';" + +# Check table structure +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e "DESCRIBE oc_oidc_user_consents;" + +# Verify migration 0015 was applied +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e "SELECT app, version FROM oc_migrations WHERE app = 'oidc' AND version LIKE '%0015%';" +``` + +Expected table structure: +- id: int(10) unsigned, auto_increment, primary key +- user_id: varchar(256), not null +- client_id: int(10) unsigned, not null +- scopes_granted: varchar(512), not null +- created_at: int(10) unsigned, not null +- updated_at: int(10) unsigned, not null +- expires_at: int(10) unsigned, nullable + +### Verify Routes + +```bash +docker compose exec app php occ router:list | grep consent +``` + +Expected output: +``` +oidc.Consent.show GET apps/oidc/consent +oidc.Consent.grant POST apps/oidc/consent/grant +oidc.Consent.deny POST apps/oidc/consent/deny +``` + +## Testing the Consent Flow + +### 1. Create an OAuth Client + +The JWT client is automatically created by the post-installation hooks: + +```bash +# Check if JWT client exists +docker compose exec app cat /var/www/html/.oauth-jwt/nextcloud_oauth_client.json +``` + +### 2. Initiate Authorization Flow + +You can test using the MCP OAuth container or manually: + +**Option A: Using MCP OAuth container** +```bash +# The mcp-oauth-jwt container will trigger the OAuth flow +docker compose logs -f mcp-oauth-jwt +``` + +**Option B: Manual browser test** +1. Get client_id from the JWT client JSON +2. Visit in browser: +``` +http://localhost:8080/apps/oidc/authorize?client_id=YOUR_CLIENT_ID&response_type=code&redirect_uri=http://localhost:8002/oauth/callback&scope=openid+profile+email+nc:read+nc:write&state=test123 +``` + +### 3. Expected Behavior + +**First Authorization:** +1. User logs in (if not already authenticated) +2. **Consent screen appears** with: + - Application name: "Nextcloud MCP Server JWT" + - List of requested scopes with descriptions: + - ✓ Basic authentication (openid) - required, cannot deselect + - ✓ Profile information (profile) + - ✓ Email address (email) + - ✓ nc:read (custom scope, shown as-is) + - ✓ nc:write (custom scope, shown as-is) + - "Allow" and "Deny" buttons +3. User selects scopes and clicks "Allow" +4. Authorization proceeds with selected scopes +5. Consent is stored in database + +**Subsequent Authorizations:** +- Same scopes → No consent screen (uses stored consent) +- Different scopes → Consent screen appears again +- If user clicks "Deny" → Returns `error=access_denied` to client + +### 4. Verify Consent Stored + +After granting consent: + +```bash +# View all stored consents with formatted timestamps +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e " +SELECT + user_id, + client_id, + scopes_granted, + FROM_UNIXTIME(created_at) as created, + FROM_UNIXTIME(updated_at) as updated, + FROM_UNIXTIME(expires_at) as expires +FROM oc_oidc_user_consents; +" nextcloud + +# Or for a compact view: +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e "SELECT * FROM oc_oidc_user_consents;" nextcloud +``` + +## Troubleshooting + +### Consent Screen Not Appearing + +**Check browser console** (F12 → Console tab): +``` +# Look for JS errors like: +Failed to load resource: js/oidc-consent.js +``` + +**Check Nextcloud logs:** +```bash +docker compose exec app tail -f /var/www/html/data/nextcloud.log | grep -i consent +``` + +**Verify JS file loaded:** +```bash +# Check file exists and has correct size (~73KB) +docker compose exec app ls -lh /opt/apps/oidc/js/oidc-consent.js +``` + +**Clear Nextcloud caches:** +```bash +docker compose exec app php occ maintenance:repair +docker compose restart app +``` + +### Migration Didn't Run + +**Check which migrations have been applied:** +```bash +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e "SELECT app, version FROM oc_migrations WHERE app = 'oidc' ORDER BY version;" nextcloud +``` + +Expected to see `Version0015Date20251123100100` in the list. + +**Manually trigger migrations:** +```bash +# Disable and re-enable app (triggers all pending migrations) +docker compose exec app php occ app:disable oidc +docker compose exec app php occ app:enable oidc + +# Verify migration 0015 was applied +docker compose exec -T db mariadb -u nextcloud -ppassword nextcloud -e "SELECT version FROM oc_migrations WHERE app = 'oidc' AND version LIKE '%0015%';" nextcloud +``` + +### Routes Not Registered + +If `router:list` doesn't show consent routes: + +```bash +# The autoloader might not have picked up new classes +# Restart the container +docker compose restart app + +# Wait for it to be ready +sleep 10 + +# Try again +docker compose exec app php occ router:list | grep consent +``` + +If still not working, check if ConsentController is accessible: +```bash +docker compose exec app php -r " +require_once '/var/www/html/lib/base.php'; +\$class = 'OCA\\OIDCIdentityProvider\\Controller\\ConsentController'; +if (class_exists(\$class)) { + echo \"Class exists\n\"; +} else { + echo \"Class not found\n\"; +} +" +``` + +## Making Changes + +### Frontend Changes (Vue.js) + +1. Edit source file on host: +```bash +cd ~/Software/oidc +# Edit src/Consent.vue +``` + +2. Rebuild JS: +```bash +npm run build +``` + +3. Refresh browser (container sees changes immediately via volume mount at /opt/apps/oidc) + +### Backend Changes (PHP) + +1. Edit files on host: +```bash +cd ~/Software/oidc +# Edit lib/Controller/ConsentController.php or other PHP files +``` + +2. Changes are immediately visible (PHP is interpreted, no build step) + +3. For new classes or major changes, restart container: +```bash +docker compose restart app +``` + +### Database Schema Changes + +If you modify the migration: + +```bash +# Changes won't be picked up if migration already ran +# Need to recreate the database: +docker compose down -v # Removes volumes +docker compose up -d # Fresh start with clean DB +``` + +## Cleanup + +### Reset Everything + +```bash +cd ~/Projects/nextcloud-mcp-server +docker compose down -v +``` + +This removes: +- All containers +- Database volume (all data) +- OAuth client credentials + +### Keep Data, Restart App + +```bash +docker compose restart app +``` + +This preserves: +- Database (consents, clients, users) +- OAuth client credentials + +## Development Workflow Summary + +1. **Make changes** in `~/Software/oidc` +2. **Build JS** if you changed Vue files: `npm run build` +3. **Test immediately** - refresh browser or restart container +4. **No need** to rebuild Docker images or reinstall app +5. **Iterate quickly** with instant feedback + +## Production Deployment + +When ready to deploy: + +1. **Create patch file** (already done): + ```bash + cd ~/Software/oidc + git format-patch master --stdout > user-consent-feature.patch + ``` + +2. **Test patch** in clean environment: + ```bash + # In a production-like environment + cd /path/to/production/oidc + git apply user-consent-feature.patch + npm install + npm run build + php occ app:disable oidc + php occ app:enable oidc + ``` + +3. **Verify migration** runs automatically on app enable + +4. **Submit pull request** to upstream repository