feat: Normalize hybrid search RRF scores to 0-1 range
Improve user comprehension by scaling RRF scores to match the intuitive 0-1 range used by other search algorithms. ## Problem RRF (Reciprocal Rank Fusion) scores had a drastically different scale than semantic/keyword/fuzzy scores: - Semantic similarity: 0.0 to 1.0 (typical: 0.5-0.9) - RRF scores: 0.0 to ~0.016 (typical: 0.005-0.015) This caused user confusion - a score of 0.0078 looked terrible but was actually excellent (near theoretical maximum). ## Solution Normalize RRF scores using the formula: `normalized_score = rrf_score * (rrf_k + 1) / total_weight` Where: - rrf_k = 60 (RRF constant) - total_weight = sum of algorithm weights (default: 1.0) **Example transformation:** - Before: 0.0078 (confusing) - After: 0.477 (intuitive) ## Changes **nextcloud_mcp_server/search/hybrid.py:** - Store total_weight as instance variable (line 63) - Calculate normalization factor in _reciprocal_rank_fusion() (line 209) - Apply normalization to all RRF scores (line 217) - Preserve raw RRF score in metadata for debugging (line 222) ## Impact **User Experience:** - Hybrid search scores now comparable with semantic/keyword/fuzzy - Score of 0.5 indicates good match across all algorithms - Consistent scale improves score threshold usability **Backward Compatibility:** - Raw RRF scores preserved in metadata["rrf_score_raw"] - Result ordering unchanged (normalization is linear transformation) - Breaking change: Existing score thresholds need adjustment **Performance:** - Negligible overhead (single multiplication per result) ## Testing Verified with nc_semantic_search and nc_semantic_search_answer: - Hybrid scores now 0.47-0.7 range (was 0.003-0.011) - Semantic scores unchanged (0.75) - Result ordering preserved 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -60,6 +60,7 @@ class HybridSearchAlgorithm(SearchAlgorithm):
|
||||
self.keyword_weight = keyword_weight
|
||||
self.fuzzy_weight = fuzzy_weight
|
||||
self.rrf_k = rrf_k
|
||||
self.total_weight = total_weight
|
||||
|
||||
# Initialize sub-algorithms
|
||||
self.semantic = SemanticSearchAlgorithm()
|
||||
@@ -202,16 +203,25 @@ class HybridSearchAlgorithm(SearchAlgorithm):
|
||||
reverse=True,
|
||||
)[:limit]
|
||||
|
||||
# Build final results with RRF scores
|
||||
# Calculate normalization factor to scale RRF scores to 0-1 range
|
||||
# Theoretical max RRF score = total_weight / (rrf_k + 1)
|
||||
# Normalization factor = (rrf_k + 1) / total_weight
|
||||
normalization_factor = (self.rrf_k + 1) / self.total_weight
|
||||
|
||||
# Build final results with normalized RRF scores
|
||||
final_results = []
|
||||
for doc_key, rrf_score in sorted_docs:
|
||||
result = best_results[doc_key]
|
||||
|
||||
# Create new result with RRF score
|
||||
# Normalize RRF score to 0-1 range for better user comprehension
|
||||
normalized_score = rrf_score * normalization_factor
|
||||
|
||||
# Create new result with normalized score
|
||||
# Keep original metadata but add RRF details
|
||||
metadata = result.metadata or {}
|
||||
metadata["rrf_score"] = rrf_score
|
||||
metadata["original_score"] = result.score
|
||||
metadata["rrf_score_raw"] = rrf_score # Original RRF score
|
||||
metadata["original_score"] = result.score # Original algorithm score
|
||||
metadata["normalization_factor"] = normalization_factor
|
||||
|
||||
final_results.append(
|
||||
SearchResult(
|
||||
@@ -219,7 +229,7 @@ class HybridSearchAlgorithm(SearchAlgorithm):
|
||||
doc_type=result.doc_type,
|
||||
title=result.title,
|
||||
excerpt=result.excerpt,
|
||||
score=rrf_score, # Use RRF score as the primary score
|
||||
score=normalized_score, # Use normalized score (0-1 range)
|
||||
metadata=metadata,
|
||||
)
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user