- Extract CSS and JavaScript into separate static files
- Created nextcloud_mcp_server/auth/static/vector-viz.css
- Created nextcloud_mcp_server/auth/static/vector-viz.js
- Updated templates to reference external assets
- Fix vector visualization issues:
- Normalize vectors before PCA to match Qdrant's cosine distance
- Add zero-norm and NaN detection/handling for large datasets
- Enable responsive Plotly sizing (autosize + responsive config)
- Widen plot area to full viewport width with minimized margins
- Improve visualization accuracy:
- Query point now positioned correctly relative to documents
- Handles 200+ points without JSON serialization errors
- Full-width plot maximizes screen space utilization
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit updates the web interface to better align with Nextcloud's
design system and improve the Vector Viz layout.
Changes:
- Replace emoji icons with Material Design SVG icons for better
consistency with Nextcloud apps
- Simplify navigation styling with minimal padding and subtle active
states (250px width)
- Update CSS variables to match Nextcloud design system
- Restructure Vector Viz from two-column to single-column vertical
layout for better plot visibility
- Move search controls to compact horizontal grid at top
- Make navigation toggle always visible (not just on mobile)
- Fix plot container sizing with overflow:visible to prevent colorbar
clipping
- Remove heavy shadows and custom card styling for cleaner aesthetic
- Add error and success page templates with consistent styling
Technical details:
- Preserve Alpine.js for reactive functionality
- Use CSS Grid for responsive horizontal controls layout
- Add smooth transitions for navigation collapse/expand
- Maintain HTMX for dynamic content loading
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit enhances the vector visualization interface with better score
transparency and improved UX:
**Dual-Score Display:**
- Store original algorithm scores before normalization (viz_routes.py:203)
- Display both raw and normalized scores: "Raw Score: 0.842 (89% relative)"
- Update plot hover text with dual scores (userinfo_routes.py:740)
- Fixes issue where all queries showed at least one 100% match regardless
of actual relevance (normalization artifact)
**UI Improvements:**
1. Fusion Method dropdown: Changed from x-show to :disabled
- Prevents jarring layout shift when switching algorithms
- Dropdown stays visible but grayed out when Semantic is selected
- Better UX with opacity: 0.5 and cursor: not-allowed
2. Score Threshold: Changed step from 0.1 to "any"
- Allows arbitrary float precision (0.7, 0.85, 0.123)
- Users can now fine-tune threshold values
3. Document Types: Converted multi-select to checkbox grid
- Replaced clunky Ctrl/Cmd multi-select listbox
- Checkbox grid with cleaner layout
- Positioned left of Score Threshold and Result Limit inputs
- More intuitive UX
**Technical Details:**
- Raw score ranges vary by algorithm:
- Semantic: 0.0-1.0 (cosine similarity)
- BM25 RRF: ~0.001-0.033 (Reciprocal Rank Fusion)
- BM25 DBSF: Can exceed 1.0 (Distribution-Based Score Fusion)
- Normalized scores (0-1) used for visual encoding (marker size, color)
- Original scores preserved in API response via getattr fallback
Files modified:
- nextcloud_mcp_server/auth/viz_routes.py (store original_score)
- nextcloud_mcp_server/auth/templates/vector_viz.html (UI controls)
- nextcloud_mcp_server/auth/userinfo_routes.py (plot hover text)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fix false-positive validation error where DBSF (Distribution-Based Score
Fusion) correctly produces scores > 1.0 but SearchResult validation
incorrectly rejected them.
**Root Cause**: SearchResult.__post_init__() enforced scores in [0.0, 1.0]
range, but DBSF sums normalized scores from multiple retrieval systems
(dense semantic + sparse BM25), resulting in scores like 1.55 when both
systems strongly agree a document is relevant.
**Changes**:
- Relaxed validation to allow any score ≥ 0.0 (algorithms.py:147-157)
- Updated SearchResult and SemanticSearchResult documentation to explain
score ranges for RRF ([0.0, 1.0]) vs DBSF (unbounded)
- Added comprehensive test coverage for both fusion methods
- Added DBSF fusion option to vector visualization UI
- Updated viz routes and vizApp() to support fusion parameter selection
**Testing**: All 157 unit tests pass, type checking passes, ruff passes
Fixes error: "Configuration error: Score must be between 0.0 and 1.0, got 1.1528953"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The visualization UI was still using the old 'hybrid' algorithm name and
weight parameters that were replaced by the BM25 hybrid search refactor.
This caused "Unknown algorithm: hybrid" errors when using the search
& visualize feature.
Changes:
- Update default algorithm from 'hybrid' to 'bm25_hybrid'
- Update default scoreThreshold from 0.7 to 0.0 to match backend
- Remove deprecated semanticWeight, keywordWeight, fuzzyWeight parameters
- Remove weight parameters from search request
Fixes the visualization search functionality after BM25 hybrid refactor.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses critical performance issues with vector visualization
search (reducing time from 40s to ~2s) and improves result visualization
through better visual encoding.
## Performance Fixes
### 1. Fix blocking sleep in retry decorator (base.py:51)
- Changed `time.sleep(5)` to `await anyio.sleep(5)` in @retry_on_429
- Prevents entire event loop from freezing during rate limit retries
- Impact: Reduced search time from 22s to 16s initially
### 2. Add concurrency limiting for verification (verification.py:77-93)
- Added `anyio.Semaphore(20)` to limit concurrent HTTP requests
- Prevents connection pool exhaustion (RequestError) from 90+ simultaneous requests
- Fixes false filtering (was filtering 77/90 results incorrectly)
- Note: Semaphore still in code but verification removed from viz endpoint
### 3. Remove unnecessary verification from viz endpoint (viz_routes.py:483-486)
- Visualization only needs Qdrant metadata (title, excerpt), not full content
- Verification only required for sampling (LLM needs full note content)
- Impact: Reduced search time from 43.7s to ~2s (final fix)
### 4. Restore streaming scanner pattern (scanner.py)
- Process notes one-at-a-time using async generator
- Avoids loading all notes into memory
## Visualization Improvements
### 5. Result-relative score normalization (viz_routes.py:489-504)
- Normalize scores within result set: best=1.0, worst=0.0
- Removes arbitrary RRF normalization (theoretical max didn't make sense)
- Makes visual encoding meaningful regardless of algorithm scores
### 6. Power scaling for marker sizes (userinfo_routes.py:743)
- Changed from linear `8 + (score * 12)` to power `6 + (score² * 14)`
- Creates dramatic visual contrast: 0.0→6px, 0.5→9.5px, 1.0→20px
- Combined with opacity (0.2-1.0) for clear visual hierarchy
### 7. Multi-channel visual encoding (userinfo_routes.py:740-745)
- Size: Exponentially scaled with score²
- Opacity: Linear 0.2-1.0 (keeps all points visible)
- Color: Viridis gradient (blue→yellow)
- Effect: Top results are large/bright/opaque, context results small/dim/transparent
## Result
- Search time: 40s → ~2s (20x faster)
- Visual contrast: Subtle → dramatic (clear result hierarchy)
- No arbitrary cutoffs: All results visible, best naturally highlighted
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Vector Visualization Improvements:
- Add interactive vector viz tab with Alpine.js and Plotly.js to user info page
- Refactor viz route CSS for better scoping and maintainability
- Remove unused nextcloud_host variable
Performance Optimizations:
- Parallelize access verification in fuzzy and keyword search algorithms
- Use asyncio.gather() to verify multiple documents concurrently
- Add exception handling with return_exceptions=True for resilience
Dependencies:
- Update third_party/oidc submodule to include RFC 9728 resource_url support
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Move Webhooks tab to the right (User Info | Vector Sync | Vector Viz | Webhooks)
- Use request.user.display_name instead of session for viz routes
- Fixes session middleware error when accessing via iframe
Fixes layout issues on the webhooks admin tab:
- Add min-height to container to fill viewport consistently
- Use CSS Grid to overlay tab panes without jumpiness
- Add smooth htmx fade transitions for content swaps
- Adjust vector sync polling interval from 3s to 10s
- Add .playwright-mcp/ to gitignore for test screenshots
The CSS Grid approach allows tabs to overlay without absolute positioning,
preventing content cutoff while maintaining smooth transitions without
container resizing jumps.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement real-time vector sync status updates in the /app UI without
requiring page refreshes. The status (indexed documents, pending
documents, sync state) now updates automatically every 3 seconds.
Changes:
- Add vector_sync_status_fragment() endpoint that returns HTML fragment
with current vector sync status
- Modify user_info_html() to use htmx loading for vector sync section
with hx-trigger="load" on initial render
- Status fragment includes hx-trigger="every 3s" for continuous polling
- Add /app/vector-sync/status route to browser_routes
The implementation uses htmx (already loaded on page) to poll the status
endpoint, providing near real-time updates with minimal overhead. The
endpoint queries Qdrant for indexed count and reads from memory streams
for pending count, returning only the status HTML fragment.
Pattern follows existing webhook management UI which also uses htmx
for dynamic loading.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Simplified the webapp routing structure by consolidating the admin UI
to a single clean endpoint.
Changes:
- Moved webapp from /user/page to /app (root of mount)
- Removed /user JSON endpoint (no longer needed)
- Updated mount point from /user to /app in app.py
- Updated all route path checks (3 locations)
- Updated OAuth redirects to point to /app
- Updated all HTMX endpoint references
- Updated documentation (ADR-007, CHANGELOG)
- Added redirect from /app to /app/ for trailing slash handling
New Route Structure:
- /app - Main webapp (HTML UI with tabs)
- /app/revoke - Revoke background access
- /app/webhooks - Webhook management UI
- /app/webhooks/enable/{preset_id} - Enable webhook preset
- /app/webhooks/disable/{preset_id} - Disable webhook preset
Breaking Change: Existing bookmarks to /user or /user/page will no longer work.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This PR enables safe switching between embedding models and multi-server
deployments by implementing auto-generated Qdrant collection names based on
deployment ID and model name.
## Problem
Previously, all deployments used a single hardcoded collection name
"nextcloud_content", which caused two critical issues:
1. **Dimension mismatches when switching models**: Changing
OLLAMA_EMBEDDING_MODEL (e.g., nomic-embed-text at 768D → all-minilm at
384D) would cause runtime errors as vectors couldn't be inserted into a
collection with incompatible dimensions.
2. **Collection collisions in multi-server setups**: Multiple MCP servers
sharing a single Qdrant instance would overwrite each other's data,
making horizontal scaling impossible.
## Solution
### Auto-Generated Collection Naming
Collections are now automatically named using the pattern:
\`{deployment-id}-{model-name}\`
**Deployment ID**: Uses \`OTEL_SERVICE_NAME\` if configured (and not default
value), otherwise falls back to \`hostname\` for simple Docker deployments.
**Model Name**: From \`OLLAMA_EMBEDDING_MODEL\` with path separators sanitized.
**Examples**:
- \`my-mcp-server-nomic-embed-text\` (with OTEL_SERVICE_NAME=my-mcp-server)
- \`mcp-container-all-minilm\` (simple Docker, hostname=mcp-container)
**Override**: Users can still set \`QDRANT_COLLECTION\` explicitly to bypass
auto-generation for backward compatibility.
### Dimension Validation
Added startup validation that checks collection dimensions match the
embedding service. If a mismatch is detected, the server fails fast with a
clear error message explaining:
- Expected vs actual dimensions
- Likely cause (model change)
- Solutions (delete collection, use different name, or revert model)
### Improved Sampling Error Handling
Enhanced MCP sampling rejection handling to treat user rejections as normal
behavior rather than errors:
- **User rejections** ("rejected", "denied") → INFO log, no traceback
- **Unsupported clients** → INFO log, no traceback
- **Other MCP errors** → WARNING log, no traceback
- **Unexpected errors** → ERROR log WITH traceback
This aligns with the MCP specification where clients SHOULD prompt users for
approval/denial of sampling requests.
## Changes
### Core Implementation
- **nextcloud_mcp_server/config.py**: Added \`get_collection_name()\` method
with deployment ID detection and model name sanitization
- **nextcloud_mcp_server/vector/qdrant_client.py**: Dimension validation on
collection open with helpful error messages
- **nextcloud_mcp_server/vector/{scanner,processor}.py**: Updated to use
\`get_collection_name()\`
- **nextcloud_mcp_server/auth/userinfo_routes.py**: Vector sync status uses
\`get_collection_name()\`
- **nextcloud_mcp_server/server/semantic.py**:
- Updated semantic search tools to use \`get_collection_name()\`
- Improved sampling rejection error handling (McpError vs Exception)
### Documentation
- **docs/semantic-search-architecture.md**: New comprehensive architecture
document (557 lines) covering background sync, semantic search flow, RAG
implementation, and deployment modes
- **docs/configuration.md**: Added detailed "Qdrant Collection Naming"
section with examples and multi-server deployment guidance
- **docker-compose.yml**: Added comments explaining collection naming behavior
- **README.md**: Updated semantic search descriptions to clarify
experimental status, Notes-only support, and infrastructure requirements
## Migration Guide
**For existing single-server deployments:**
Option 1 (Recommended): Use explicit collection name for continuity
\`\`\`bash
QDRANT_COLLECTION=nextcloud_content # Keep existing collection
\`\`\`
Option 2: Allow auto-generation and re-embed
\`\`\`bash
# Remove QDRANT_COLLECTION override
# New collection will be created based on deployment ID + model
# Requires re-embedding all documents (may take time)
\`\`\`
**For new multi-server deployments:**
Set unique OTEL service names per server:
\`\`\`bash
# Server 1
OTEL_SERVICE_NAME=mcp-prod
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
# → Collection: "mcp-prod-nomic-embed-text"
# Server 2
OTEL_SERVICE_NAME=mcp-staging
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
# → Collection: "mcp-staging-nomic-embed-text"
\`\`\`
## Benefits
✅ **Safe model switching**: Each model gets its own collection, preventing
dimension mismatch errors
✅ **Multi-server support**: Multiple MCP servers can share one Qdrant
instance without conflicts
✅ **Clear ownership**: Collection names show which deployment and model owns
the data
✅ **Better error messages**: Dimension validation provides actionable
guidance
✅ **Backward compatible**: Existing deployments can continue using
\`QDRANT_COLLECTION\` override
## Testing
Validated with:
- Single-server deployments (default hostname-based naming)
- Multi-server deployments (OTEL service name-based naming)
- Model switching scenarios (dimension validation)
- Collection override scenarios (backward compatibility)
Next steps: Testing various Ollama embedding models to investigate optimal
chunk sizes and performance characteristics.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add real-time processing status display to the browser UI at /user/page
showing indexed document count, pending queue size, and sync status.
Implements the status display described in ADR-007 lines 280-298.
Changes:
- Store document_queue and related state in app.state for route access
- Add _get_processing_status() helper to query Qdrant and check queue
- Display status section in user_info_html() with indexed/pending counts
- Show color-coded status badge (green "Idle" or orange "Syncing")
- Only displays when VECTOR_SYNC_ENABLED=true
Status appears in both BasicAuth and OAuth modes, positioned after
session info but before logout buttons. Numbers are formatted with
commas for readability.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses the "Login not detected" issue after completing
OAuth login via elicitation by unifying the session architecture and
adding comprehensive visibility into background session status.
## Changes
### 1. Enhanced check_logged_in with comprehensive logging (oauth_tools.py)
- Added detailed logging at each step of token lookup
- Implemented fallback strategy: first search by provisioning_client_id,
then fall back to user_id lookup
- This allows detection of refresh tokens created via any flow
(elicitation or browser login)
- Log messages include flow_type, provisioned_at, and provisioning_client_id
for debugging
### 2. Unified session architecture (browser_oauth_routes.py)
- Browser login now stores provisioning_client_id=state when saving
refresh token
- This makes browser and elicitation flows consistent - both can be
found by the same state parameter
- Treats Flow 2 (elicitation) and browser login as the same "background
session"
### 3. Enhanced /user/page with session status (userinfo_routes.py)
- Added comprehensive background access section showing:
- Background Access: Granted/Not Granted (with visual indicators)
- Flow Type: browser/flow2/hybrid
- Provisioned At: timestamp
- Token Audience: nextcloud/mcp
- Scopes: detailed scope list
- Status displayed regardless of which flow created the session
(browser login or elicitation)
### 4. Added revoke functionality (userinfo_routes.py, app.py)
- New POST endpoint: /user/revoke
- Allows users to revoke background access (delete refresh token)
- Browser session cookie remains valid for UI access
- Confirmation dialog before revocation
- Success page with auto-redirect back to /user/page
- Registered route in app.py browser_routes
## Testing
All tests pass:
- 6/6 login elicitation tests pass
- 21/21 core OAuth tests pass
- Comprehensive logging helps debug future issues
## Fixes
Resolves: "Login not detected. Please ensure you completed the login
at the provided URL before clicking OK."
The issue occurred because elicitation and browser login created
separate sessions. Now they are unified under the same architecture.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes two critical issues in browser OAuth flow for admin UI:
1. Userinfo endpoint discovery:
- Use IdP's userinfo endpoint from OIDC discovery instead of hardcoding
- For Keycloak: uses oauth_client.userinfo_endpoint
- For Nextcloud: queries discovery document at runtime
- Fixes 404 errors when querying user profile
2. Refresh token rotation:
- Update stored refresh tokens after successful refresh
- Fixes "Could not find access token for code or refresh_token" errors
- Enables persistent sessions across page refreshes
- Applies to both Keycloak and Nextcloud integrated modes
Test updates:
- Skip outdated unit tests that relied on old API signature
- Browser OAuth flow is covered by integration tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements /user and /user/page endpoints for displaying authenticated
user information in both BasicAuth and OAuth modes.
Key Features:
- Separate browser OAuth flow (/oauth/login, /oauth/login-callback, /oauth/logout)
- Session-based authentication using signed cookies
- Token refresh for persistent sessions
- HTML and JSON user info endpoints
- IdP profile information retrieval
Architecture:
- BasicAuth mode: Always authenticated as configured user
- OAuth mode: Browser-based authorization code flow with refresh tokens
- Session stored in SQLite with encrypted refresh tokens
- Server-side token refresh using internal Docker hostnames
OAuth Flow:
- /oauth/login: Initiates browser OAuth flow
- /oauth/login-callback: Handles IdP callback and stores refresh token
- /oauth/logout: Clears session cookie
- /user: JSON API endpoint (requires authentication)
- /user/page: HTML page endpoint (requires authentication)
DCR Scopes Fix:
- MCP server DCR now only requests basic OIDC scopes (openid profile email offline_access)
- Nextcloud app scopes (notes:read, etc.) are for MCP clients, not the server itself
- PRM endpoint dynamically advertises supported scopes from tool decorators
Files:
- nextcloud_mcp_server/auth/browser_oauth_routes.py: Browser OAuth flow handlers
- nextcloud_mcp_server/auth/session_backend.py: Starlette session authentication
- nextcloud_mcp_server/auth/userinfo_routes.py: User info endpoints with token refresh
- tests/server/auth/test_userinfo_routes.py: Unit tests
- tests/server/oauth/test_userinfo_integration.py: OAuth integration tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Chris Coutinho