feat(helm): Add document chunking configuration

Add support for configurable document chunking parameters to Helm chart
to match docker-compose and application capabilities.

Changes:
1. values.yaml:
   - Add documentChunking section with chunkSize (512) and chunkOverlap (50)
   - Include comprehensive comments explaining chunking strategies
   - Positioned between vectorSync and qdrant sections

2. templates/deployment.yaml:
   - Add DOCUMENT_CHUNK_SIZE and DOCUMENT_CHUNK_OVERLAP env vars
   - Always set (not conditional), used by vector sync processor
   - Environment variables follow same pattern as config.py defaults

3. README.md:
   - Add documentChunking parameter table in Vector Search section
   - Document chunking strategies (small/medium/large chunks)
   - Explain overlap recommendations (10-20% of chunk size)

Validation:
- helm lint: Passes
- helm template: Environment variables correctly generated
- Custom values: Work as expected (tested with chunkSize=1024)
- Always present: Not conditional on vectorSync.enabled

This maintains feature parity between Helm and docker-compose deployments,
allowing users to tune chunking for their embedding models and use cases.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2025-11-10 03:34:16 +01:00
parent 157e433d65
commit d80e54ff97
3 changed files with 32 additions and 0 deletions
+13
View File
@@ -219,6 +219,19 @@ Enable semantic search capabilities by deploying a vector database (Qdrant) and
| `vectorSync.processorWorkers` | Number of concurrent processor workers | `3` |
| `vectorSync.queueMaxSize` | Maximum queue size for pending documents | `10000` |
**Document Chunking Configuration:**
| Parameter | Description | Default |
|-----------|-------------|---------|
| `documentChunking.chunkSize` | Number of words per chunk for embedding | `512` |
| `documentChunking.chunkOverlap` | Number of overlapping words between chunks | `50` |
**Chunking Strategy:**
- **Small chunks (256-384)**: Better precision for searches, more storage overhead
- **Medium chunks (512-768)**: Balanced approach (recommended for most use cases)
- **Large chunks (1024+)**: Better context preservation, less precise matching
- **Overlap**: Should be 10-20% of chunk size to preserve context across boundaries
**Qdrant Vector Database:**
Qdrant is deployed as a subchart when `qdrant.enabled` is `true`. All configuration values are passed through to the [qdrant/qdrant](https://github.com/qdrant/qdrant-helm) chart.
@@ -158,6 +158,11 @@ spec:
- name: VECTOR_SYNC_QUEUE_MAX_SIZE
value: {{ .Values.vectorSync.queueMaxSize | quote }}
{{- end }}
# Document Chunking (always set, used by vector sync processor)
- name: DOCUMENT_CHUNK_SIZE
value: {{ .Values.documentChunking.chunkSize | quote }}
- name: DOCUMENT_CHUNK_OVERLAP
value: {{ .Values.documentChunking.chunkOverlap | quote }}
# Qdrant Vector Database
{{- if eq .Values.qdrant.mode "network" }}
# Network mode: Use dedicated Qdrant service
+14
View File
@@ -314,6 +314,20 @@ vectorSync:
# Maximum queue size for documents pending indexing
queueMaxSize: 10000
# Document Chunking Configuration
# Controls how documents are split into chunks before embedding
# Only relevant when vectorSync.enabled is true
documentChunking:
# Number of words per chunk (default: 512)
# Smaller chunks (256-384): Better for precise searches, more chunks to store
# Medium chunks (512-768): Balanced approach (recommended for most use cases)
# Larger chunks (1024+): Better for context, less precise matching
chunkSize: 512
# Number of overlapping words between chunks (default: 50)
# Recommended: 10-20% of chunkSize for context preservation across boundaries
# Must be less than chunkSize
chunkOverlap: 50
# Qdrant Vector Database Configuration
# Three deployment modes available:
# 1. Local In-Memory: Fast, ephemeral, zero-config (mode: "memory")