Commit Graph

65 Commits

Author SHA1 Message Date
github-actions[bot] 798958f20a bump: version 0.44.0 → 0.44.1 2025-11-21 00:39:23 +00:00
github-actions[bot] bf11f16e2f bump: version 0.43.0 → 0.44.0 2025-11-19 22:43:03 +00:00
github-actions[bot] 441d94301e bump: version 0.42.0 → 0.43.0 2025-11-18 12:56:15 +00:00
renovate-bot-cbcoutinho[bot] cefb438017 chore(deps): update helm release qdrant to v1.16.0 2025-11-17 17:08:54 +00:00
github-actions[bot] 8367208a03 bump: version 0.41.0 → 0.42.0 2025-11-17 07:25:33 +00:00
github-actions[bot] b1f7b1d30b bump: version 0.40.0 → 0.41.0 2025-11-17 05:57:12 +00:00
github-actions[bot] 39131cefcc bump: version 0.39.0 → 0.40.0 2025-11-16 11:09:40 +00:00
github-actions[bot] 050e9a56b9 bump: version 0.38.0 → 0.39.0 2025-11-16 11:02:48 +00:00
github-actions[bot] b58b200452 bump: version 0.37.0 → 0.38.0 2025-11-16 10:18:37 +00:00
github-actions[bot] 10129354d9 bump: version 0.36.0 → 0.37.0 2025-11-16 10:18:00 +00:00
github-actions[bot] 7a7ed79d56 bump: version 0.35.0 → 0.36.0 2025-11-15 23:32:55 +00:00
github-actions[bot] 682923dcc8 bump: version 0.34.2 → 0.35.0 2025-11-15 00:46:11 +00:00
github-actions[bot] 56a5c63994 bump: version 0.34.1 → 0.34.2 2025-11-13 21:11:36 +00:00
github-actions[bot] dd12c957f6 bump: version 0.34.0 → 0.34.1 2025-11-13 21:10:16 +00:00
github-actions[bot] 2f138e7539 bump: version 0.33.1 → 0.34.0 2025-11-13 16:15:29 +00:00
github-actions[bot] bd76902932 bump: version 0.33.0 → 0.33.1 2025-11-13 12:10:42 +00:00
Chris Coutinho 4e43d15153 fix: Move grafana_folder from labels to annotations
Fixes Kubernetes label validation error when deploying dashboard ConfigMap.

Problem:
- Kubernetes labels cannot contain spaces (validation regex: [A-Za-z0-9][-A-Za-z0-9_.]*[A-Za-z0-9])
- Previous implementation had grafana_folder: "Nextcloud MCP" as a label
- Deployment failed with: "Invalid value: 'Nextcloud MCP'"

Solution:
- Move grafana_folder from labels to annotations (annotations allow spaces)
- Keep grafana_dashboard="1" as label for ConfigMap discovery
- Grafana sidecar reads folder name from folderAnnotation parameter

Changes:
- dashboard-configmap.yaml: Move grafana_folder to annotations section
- dashboards/README.md: Fix kubectl commands to use annotations
- values.yaml: Update comments to clarify annotation usage

This follows the standard kube-prometheus-stack pattern where:
- Labels are used for ConfigMap discovery (strict validation)
- Annotations are used for metadata like folder names (relaxed validation)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 13:08:45 +01:00
github-actions[bot] 15951c38fa bump: version 0.32.1 → 0.33.0 2025-11-13 10:58:05 +00:00
Chris Coutinho 4ea5ed72d4 feat: Add Grafana dashboard and vector sync metric instrumentation
Implement comprehensive observability for vector database synchronization
with Grafana dashboard and Prometheus metrics.

## Part 1: Grafana Dashboard

Created all-in-one operations dashboard with 7 rows and 34 panels:

### Dashboard Structure:
- **Overview Row**: Request rate, error rate, P95 latency, active requests
- **HTTP Metrics (RED)**: Request/error rates by endpoint, latency percentiles
- **MCP Tools**: Call volume, error rates, execution duration by tool
- **Nextcloud API**: API calls/latency by app, retry patterns
- **OAuth & Authentication**: Token validations, exchanges, cache hit rate
- **Dependencies & Health**: Status for Nextcloud/Qdrant/Keycloak/Unstructured
- **Vector Sync**: Processing throughput, queue depth, Qdrant operations

### Helm Chart Integration:
- Added dashboard-configmap.yaml template for automatic provisioning
- Configured Grafana sidecar auto-discovery (label: grafana_dashboard="1")
- Added dashboards configuration section in values.yaml (opt-in)
- Updated Chart.yaml with dashboard annotations
- Enhanced NOTES.txt with dashboard deployment instructions
- Comprehensive documentation in dashboards/README.md

Dashboard supports dynamic filtering via variables:
- datasource: Prometheus data source selection
- namespace: Filter by Kubernetes namespace
- pod: Multi-select pod filtering
- interval: Query interval (1m/5m/10m/30m/1h)

## Part 2: Vector Sync Metric Instrumentation

Implemented metric recording throughout vector sync pipeline:

### metrics.py:
Added convenience functions:
- record_vector_sync_scan() - Track documents per scan
- record_vector_sync_processing() - Track processing duration/status
- record_qdrant_operation() - Track database operations
- update_vector_sync_queue_size() - Track queue depth

### scanner.py:
- Record number of documents found in each scan
- Enables monitoring of scan throughput

### processor.py:
- Record processing duration for each document
- Track success/failure status with timing
- Record Qdrant upsert/delete operations
- Handle all code paths (success, deletion, error)

### semantic.py:
- Wrap Qdrant query_points with try/except
- Record search operation success/failure

## Metrics Exposed:

- mcp_vector_sync_documents_scanned_total
- mcp_vector_sync_documents_processed_total{status}
- mcp_vector_sync_processing_duration_seconds (histogram)
- mcp_vector_sync_queue_size (gauge)
- mcp_qdrant_operations_total{operation,status}

This enables monitoring of:
- Scan and processing throughput
- Processing latency (P50/P95/P99)
- Error rates for processing and Qdrant operations
- Queue depth trends
- Complete observability of vector sync pipeline

## Testing:

Verified locally that metrics are recorded correctly:
- 36 documents scanned
- 3 documents processed (avg 7.5s each)
- 3 successful Qdrant upsert operations
- Search operations tracked

## Deployment:

Enable dashboard provisioning in Helm values:
```yaml
dashboards:
  enabled: true
  grafanaFolder: "Nextcloud MCP"
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 11:49:20 +01:00
github-actions[bot] 747d297008 bump: version 0.32.0 → 0.32.1 2025-11-12 02:16:57 +00:00
github-actions[bot] 49a9dd43c6 bump: version 0.31.1 → 0.32.0 2025-11-11 23:54:43 +00:00
github-actions[bot] ce666934f2 bump: version 0.31.0 → 0.31.1 2025-11-10 22:21:48 +00:00
Chris Coutinho a6e5f3d8ff refactor: simplify OpenTelemetry tracing configuration
Simplifies the OpenTelemetry tracing setup by removing the redundant
OTEL_ENABLED flag and using the presence of OTEL_EXPORTER_OTLP_ENDPOINT
to determine if tracing should be enabled. This follows the standard
OpenTelemetry environment variable conventions more closely.

Changes:
- Remove OTEL_ENABLED/tracing_enabled flag in favor of checking if
  OTEL_EXPORTER_OTLP_ENDPOINT is set
- Add OTEL_EXPORTER_VERIFY_SSL configuration option for OTLP endpoints
  with self-signed certificates (defaults to false for development)
- Move HTTPXClientInstrumentor initialization to module level to ensure
  httpx calls are traced across all Nextcloud API requests
- Add tracing spans to vector sync operations (scan_user_documents)
- Fix authorization header logging to only warn about missing headers
  in OAuth mode (BasicAuth mode doesn't use Authorization headers)
- Update observability documentation to reflect simplified configuration
- Refactor Dockerfile to use --no-editable flag for uv sync

Breaking changes:
- OTEL_ENABLED environment variable is removed
- Tracing is now automatically enabled when OTEL_EXPORTER_OTLP_ENDPOINT
  is set

Migration guide:
- Remove OTEL_ENABLED=true from environment configuration
- Tracing will be enabled automatically if OTEL_EXPORTER_OTLP_ENDPOINT
  is configured

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 22:48:37 +01:00
github-actions[bot] f44bf3e8f2 bump: version 0.30.0 → 0.31.0 2025-11-10 07:02:49 +00:00
github-actions[bot] 126b5a7626 bump: version 0.29.2 → 0.30.0 2025-11-10 02:50:11 +00:00
Chris Coutinho 4d3ff1abe1 Merge pull request #282 from cbcoutinho/feat/multi-embedding-model-support
feat(vector): Support multiple embedding models with auto-generated collection names
2025-11-10 03:49:48 +01:00
Chris Coutinho d80e54ff97 feat(helm): Add document chunking configuration
Add support for configurable document chunking parameters to Helm chart
to match docker-compose and application capabilities.

Changes:
1. values.yaml:
   - Add documentChunking section with chunkSize (512) and chunkOverlap (50)
   - Include comprehensive comments explaining chunking strategies
   - Positioned between vectorSync and qdrant sections

2. templates/deployment.yaml:
   - Add DOCUMENT_CHUNK_SIZE and DOCUMENT_CHUNK_OVERLAP env vars
   - Always set (not conditional), used by vector sync processor
   - Environment variables follow same pattern as config.py defaults

3. README.md:
   - Add documentChunking parameter table in Vector Search section
   - Document chunking strategies (small/medium/large chunks)
   - Explain overlap recommendations (10-20% of chunk size)

Validation:
- helm lint: Passes
- helm template: Environment variables correctly generated
- Custom values: Work as expected (tested with chunkSize=1024)
- Always present: Not conditional on vectorSync.enabled

This maintains feature parity between Helm and docker-compose deployments,
allowing users to tune chunking for their embedding models and use cases.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 03:34:16 +01:00
github-actions[bot] a0576aa9a2 bump: version 0.29.1 → 0.29.2 2025-11-09 18:28:34 +00:00
Chris Coutinho 4a6c60113b fix(helm): Set default strategy to Recreate 2025-11-09 19:27:55 +01:00
renovate-bot-cbcoutinho[bot] de4f1032aa chore(deps): update helm release qdrant to v1 2025-11-09 17:08:13 +00:00
renovate-bot-cbcoutinho[bot] 3147180ccd chore(deps): update helm release ollama to v1.34.0 2025-11-09 11:08:18 +00:00
github-actions[bot] 7772b1ac2e bump: version 0.29.0 → 0.29.1 2025-11-09 08:54:26 +00:00
github-actions[bot] af96378cb6 bump: version 0.28.0 → 0.29.0 2025-11-09 08:29:53 +00:00
Chris Coutinho c5da11aa4c Merge pull request #274 from cbcoutinho/feature/observability-monitoring
feature/observability monitoring
2025-11-09 09:29:25 +01:00
Chris Coutinho 093ac5b5ba feat(helm): Add observability support with ServiceMonitor and Grafana dashboard
Add comprehensive observability configuration to Helm chart:

**Helm Values:**
- Add observability configuration section for metrics, tracing, and logging
- Add serviceMonitor configuration (disabled by default)
- Add prometheusRule configuration (disabled by default)

**Templates:**
- Update deployment to include observability environment variables
- Update deployment to expose metrics port (9090)
- Update service to expose metrics port
- Add ServiceMonitor template for Prometheus Operator
- Add PrometheusRule template with critical and warning alerts

**Dashboards:**
- Add comprehensive Grafana dashboard JSON with 6 panels:
  - Request Rate (by method and endpoint)
  - Error Rate (5xx errors percentage)
  - Request Latency (P50/P95 by endpoint)
  - Top MCP Tools (by invocation volume)
  - Nextcloud API Latency (by app)
  - Vector Sync Queue Size
- Add dashboard README with import instructions

**Alert Rules:**
- Critical: Server down, high error rate (>5%), high latency (>1s), dependency down
- Warning: Token validation errors (>1%), vector sync queue high (>100), Qdrant slow (>500ms)

All features are opt-in via values.yaml configuration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 09:10:11 +01:00
github-actions[bot] ae81f0334e bump: version 0.27.3 → 0.28.0 2025-11-09 08:04:06 +00:00
github-actions[bot] 8f0f989c6d bump: version 0.27.2 → 0.27.3 2025-11-09 06:52:31 +00:00
Chris Coutinho f8a2935c22 fix(ci): Use helm dependency build instead of update to use Chart.lock 2025-11-09 07:52:00 +01:00
github-actions[bot] 137dc80075 bump: version 0.27.1 → 0.27.2 2025-11-09 06:45:44 +00:00
Chris Coutinho 725ac65e6a fix(helm): update Qdrant dependency condition to match new mode structure
The Qdrant subchart was being included by default even in memory/persistent
modes. Changed the dependency condition from `qdrant.enabled` to
`qdrant.networkMode.deploySubchart` to align with the three-mode structure.

Now the Qdrant subchart is ONLY deployed when:
- qdrant.mode: "network"
- qdrant.networkMode.deploySubchart: true

Verified all three modes:
- Memory mode (:memory:): No subchart, QDRANT_LOCATION=:memory:
- Persistent mode (path): No subchart, QDRANT_LOCATION=/app/data/qdrant, PVC created
- Network mode (subchart): Qdrant subchart deployed, QDRANT_URL=http://...:6333
- Network mode (external): No subchart, QDRANT_URL=<external-url>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 07:45:06 +01:00
github-actions[bot] f51edff25d bump: version 0.27.0 → 0.27.1 2025-11-09 06:22:00 +00:00
github-actions[bot] 538bbc375e bump: version 0.26.1 → 0.27.0 2025-11-09 06:15:27 +00:00
Chris Coutinho 167e49788e feat(helm): add Qdrant local mode support with three deployment options [skip ci]
Add support for three Qdrant deployment modes in Helm chart:
1. In-memory mode (:memory:) - Default, zero-config, ephemeral storage
2. Persistent local mode (path-based) - File-based storage with PVC
3. Network mode (URL-based) - Dedicated Qdrant service or external instance

Changes:
- Restructured qdrant configuration in values.yaml with mode selector
- Added conditional environment variable logic in deployment.yaml
- Created PVC template for persistent local mode with optional existingClaim
- Added qdrantPvcName helper template in _helpers.tpl
- Updated README.md with Helm registry URL (https://cbcoutinho.github.io/nextcloud-mcp-server)

Breaking change: Default changed from requiring qdrant.enabled to using
in-memory mode (:memory:) when no Qdrant configuration is provided.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 07:14:19 +01:00
Chris Coutinho e32c8f4aec feat: add optional vector database and semantic search to helm chart
Add support for deploying Qdrant vector database and Ollama embedding
service as optional helm chart dependencies. Enables semantic search
capabilities for Nextcloud content with flexible deployment options.

Chart Dependencies:
- Add Qdrant v0.9.0 from qdrant/qdrant-helm (conditional)
- Add Ollama v1.33.0 from otwld/ollama-helm (conditional)
- Both dependencies only deploy when enabled

Configuration (values.yaml):
- vectorSync: Background sync settings (interval, workers, queue size)
- qdrant: Subchart config with persistence, resources, clustering
- ollama: Subchart config with model pull, persistence, resources
  - Support for external Ollama via ollama.url (no subchart deployment)
- openai: Alternative embedding provider (OpenAI or compatible API)

Environment Variables (deployment.yaml):
- VECTOR_SYNC_* variables when vectorSync.enabled
- QDRANT_URL, QDRANT_COLLECTION when qdrant.enabled
- OLLAMA_BASE_URL, OLLAMA_EMBEDDING_MODEL when ollama enabled or URL set
- OPENAI_API_KEY when openai.enabled

Documentation:
- README: New "Vector Search & Semantic Capabilities" section
- README: Example 5 showing three deployment patterns
- NOTES.txt: Conditional guidance when vector features enabled
- Secret template for OpenAI API key management

All features disabled by default for backward compatibility.
Tested with helm template and helm lint.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 00:03:51 +01:00
github-actions[bot] 4ceaf45ffd bump: version 0.26.0 → 0.26.1 2025-11-08 03:59:28 +00:00
github-actions[bot] 218f0bd366 bump: version 0.25.0 → 0.26.0 2025-11-08 03:48:50 +00:00
github-actions[bot] 77e491beea bump: version 0.24.1 → 0.25.0 2025-11-05 23:02:25 +00:00
github-actions[bot] 7052c19de0 bump: version 0.24.0 → 0.24.1 2025-11-04 12:28:13 +00:00
github-actions[bot] f587a4e31f bump: version 0.23.0 → 0.24.0 2025-11-04 10:27:39 +00:00
github-actions[bot] 02a2c4a16f bump: version 0.22.7 → 0.23.0 2025-11-03 01:48:39 +00:00