fix: implement deletion grace period and vector sync status tool

This commit addresses issues with vector database synchronization that
were causing test failures:

1. **Deletion Grace Period** (scanner.py)
   - Fixed premature deletion of documents due to pagination cursor
     inconsistencies in Notes API
   - Implemented 2-scan verification with 1.5x scan interval grace period
     (15 seconds default)
   - Documents must be missing for 2 consecutive scans before deletion
   - Documents that reappear are removed from deletion tracking
   - Prevents false deletions during concurrent note creation/indexing

2. **Vector Sync Status Tool** (server/notes.py, models/notes.py)
   - Added nc_notes_get_vector_sync_status MCP tool
   - Returns indexed_count, pending_count, status, and enabled fields
   - Enables tests and clients to wait for vector sync completion
   - Uses lifespan context to access document queue and Qdrant client

3. **Test Improvements** (test_sampling.py, conftest.py)
   - Added temporary_note_factory fixture for creating multiple test notes
   - Updated all sampling tests to wait for vector sync completion
   - Adjusted score_threshold to 0.0 for SimpleEmbeddingProvider
     (feature hashing produces low-quality embeddings)
   - Fixed CallToolResult extraction (removed ["result"] key access)
   - Removed invalid @pytest.mark.asyncio markers (anyio mode)

All integration tests now pass successfully.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2025-11-09 03:11:39 +01:00
parent bb5d4f464f
commit a854656d3c
5 changed files with 367 additions and 55 deletions
+37
View File
@@ -550,6 +550,43 @@ async def temporary_note(nc_client: NextcloudClient):
logger.error(f"Unexpected error deleting temporary note {note_id}: {e}")
@pytest.fixture
async def temporary_note_factory(nc_client: NextcloudClient):
"""
Factory fixture to create multiple temporary notes with custom parameters.
Returns a callable that creates notes and tracks them for automatic cleanup.
"""
created_notes = []
async def _create_note(title: str, content: str, category: str = ""):
"""Create a temporary note with custom title, content, and category."""
logger.info(f"Creating temporary note via factory: {title}")
note_data = await nc_client.notes.create_note(
title=title, content=content, category=category
)
note_id = note_data.get("id")
if note_id:
created_notes.append(note_id)
logger.info(f"Factory created note ID: {note_id}")
return note_data
yield _create_note
# Cleanup all created notes
for note_id in created_notes:
logger.info(f"Cleaning up factory-created note ID: {note_id}")
try:
await nc_client.notes.delete_note(note_id=note_id)
logger.info(f"Successfully deleted factory note ID: {note_id}")
except HTTPStatusError as e:
if e.response.status_code != 404:
logger.error(f"HTTP error deleting factory note {note_id}: {e}")
else:
logger.warning(f"Factory note {note_id} already deleted (404).")
except Exception as e:
logger.error(f"Unexpected error deleting factory note {note_id}: {e}")
@pytest.fixture
async def temporary_note_with_attachment(
nc_client: NextcloudClient, temporary_note: dict