Previously, pymupdf4llm.to_markdown() was called twice - once in
PyMuPDFProcessor during indexing and again in PDFHighlighter during
visualization. Different image path lengths caused different character
offsets, leading to highlighted pages not matching their chunks.
Also fixed issue where all chunks on the same page showed all highlights
instead of just their own highlight. Now restores original page contents
between chunks using xref stream caching.
Changes:
- Add PDFHighlighter class requiring pre-computed page_boundaries and
full_text from document processor (no fallback extraction)
- Pass pre-computed data from processor to highlighter
- Extract page-relative portion of chunk text for cross-page chunks
- Add bounding box highlighting using text anchor search
- Run highlight generation in parallel with embedding/BM25
- Cache and restore page contents to isolate highlights per chunk
Results: Highlighting success rate improved from 51% to 95% (121/128).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Major improvements to vector visualization page:
- Refactor PCA to display individual chunks instead of averaged documents
- Add context expansion module for fetching surrounding text from notes and PDFs
- Update deduplication to use (doc_id, doc_type, chunk_start, chunk_end) keys
- Fix Alpine.js rendering with chunk-specific keys including offsets
- Refactor authentication helper to return NextcloudClient for better reuse
- Add async context manager support to NextcloudClient
Technical details:
- viz_routes.py: Fetch specific chunk vectors instead of averaging per document
- context.py: New module supporting both notes and PDF text extraction via PyMuPDF
- search algorithms: Extract page_number, chunk_index, total_chunks from Qdrant
- vector-viz.js/html: Use chunk positions in expansion tracking keys
This enables users to see which specific chunks match their query
and view them with surrounding context in the PCA visualization.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Get container dimensions before creating Plotly layout to render at correct size immediately
- Add init() method with window resize listener for responsive plot sizing
- Remove post-render resize call (no longer needed with explicit dimensions)
- Improve colorbar positioning and scene domain configuration
This eliminates the visual "jump" during initial render and ensures the plot resizes smoothly when the browser window changes size.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Two fixes for the vector visualization page:
1. **CSS Loading Fix**: Moved CSS <link> from vector_viz.html fragment
to user_info.html <head> block. HTMX fragments don't process <link>
tags in <head>, causing unstyled page. Now CSS loads correctly.
2. **Camera Preservation**: Modified renderPlot() to preserve camera
position when toggling query point visibility. Previously, toggling
the "Show Query Point" checkbox would reset zoom/rotation to default.
Now reads existing camera settings from plot before updating.
Related: nextcloud_mcp_server/auth/static/vector-viz.js:123-130
Related: nextcloud_mcp_server/auth/templates/user_info.html:12
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Extract CSS and JavaScript into separate static files
- Created nextcloud_mcp_server/auth/static/vector-viz.css
- Created nextcloud_mcp_server/auth/static/vector-viz.js
- Updated templates to reference external assets
- Fix vector visualization issues:
- Normalize vectors before PCA to match Qdrant's cosine distance
- Add zero-norm and NaN detection/handling for large datasets
- Enable responsive Plotly sizing (autosize + responsive config)
- Widen plot area to full viewport width with minimized margins
- Improve visualization accuracy:
- Query point now positioned correctly relative to documents
- Handles 200+ points without JSON serialization errors
- Full-width plot maximizes screen space utilization
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Chris Coutinho