Commit Graph

119 Commits

Author SHA1 Message Date
renovate-bot-cbcoutinho[bot] dd636e6a08 chore(deps): update docker.io/library/nginx:alpine docker digest to 9dacca6 2025-10-29 05:07:08 +00:00
renovate-bot-cbcoutinho[bot] 4fc0cb5a41 chore(deps): update docker.io/library/nextcloud:32.0.1 docker digest to 1e4eae5 2025-10-27 23:10:34 +00:00
renovate-bot-cbcoutinho[bot] 7db2a5c586 chore(deps): pin downloads.unstructured.io/unstructured-io/unstructured-api docker tag to a43ab55 2025-10-25 22:05:59 +00:00
Chris Coutinho a36038422b feat: Add text processing background worker for telling client about progress 2025-10-25 19:52:45 +02:00
Chris Coutinho 2147fc1696 refactor: Transform document parsing into pluggable processor architecture
Refactors PR #190's hardcoded Unstructured.io integration into a flexible,
extensible plugin system supporting multiple text extraction engines.

- **`DocumentProcessor` ABC**: Abstract interface for all processors
- **`ProcessorRegistry`**: Central registry for discovery and routing
- **`ProcessingResult`**: Standardized output format across processors

- **`UnstructuredProcessor`**: Refactored from `UnstructuredClient`
- **`TesseractProcessor`**: Local OCR for images (lightweight alternative)
- **`CustomHTTPProcessor`**: Generic wrapper for custom HTTP APIs

- New `get_document_processor_config()` returns structured config
- Supports enabling/disabling individual processors
- Per-processor configuration via environment variables
- **Breaking Change**: `ENABLE_UNSTRUCTURED_PARSING` replaced with:
  - `ENABLE_DOCUMENT_PROCESSING=true/false` (master switch)
  - `ENABLE_UNSTRUCTURED=true/false` (per-processor)
  - `ENABLE_TESSERACT=true/false`
  - `ENABLE_CUSTOM_PROCESSOR=true/false`

- `parse_document()` now uses `ProcessorRegistry`
- Auto-selects appropriate processor based on MIME type
- Processor priority system (Unstructured=10, Tesseract=5, Custom=1)

- `initialize_document_processors()` registers processors at startup
- Integrated into both BasicAuth and OAuth lifespans
- Graceful degradation if processors fail to initialize

```env
ENABLE_DOCUMENT_PROCESSING=false

ENABLE_UNSTRUCTURED=false
UNSTRUCTURED_API_URL=http://unstructured:8000
UNSTRUCTURED_STRATEGY=auto  # auto|fast|hi_res
UNSTRUCTURED_LANGUAGES=eng,deu

ENABLE_TESSERACT=false
TESSERACT_LANG=eng

ENABLE_CUSTOM_PROCESSOR=false
CUSTOM_PROCESSOR_URL=http://localhost:9000/process
CUSTOM_PROCESSOR_TYPES=application/pdf,image/jpeg
```

- **Removed**: `tests/test_unstructured_config.py` (legacy tests)
- **Added**: `tests/unit/test_document_processor_config.py`
  - 7 unit tests for new config system
  - Tests individual and multi-processor configurations

- **Added**:
  - `nextcloud_mcp_server/document_processors/__init__.py`
  - `nextcloud_mcp_server/document_processors/base.py`
  - `nextcloud_mcp_server/document_processors/registry.py`
  - `nextcloud_mcp_server/document_processors/unstructured.py`
  - `nextcloud_mcp_server/document_processors/tesseract.py`
  - `nextcloud_mcp_server/document_processors/custom_http.py`
  - `tests/unit/test_document_processor_config.py`

- **Modified**:
  - `nextcloud_mcp_server/config.py` - New plugin config system
  - `nextcloud_mcp_server/app.py` - Processor initialization
  - `nextcloud_mcp_server/utils/document_parser.py` - Uses registry
  - `nextcloud_mcp_server/server/webdav.py` - Import updates
  - `env.sample` - New configuration format
  - `docker-compose.yml` - (profile changes from previous work)

- **Removed**:
  - `nextcloud_mcp_server/client/unstructured_client.py` - Replaced by UnstructuredProcessor
  - `tests/test_unstructured_config.py` - Replaced with new tests

 **Extensible**: Add processors without modifying core code
 **Testable**: Mock processors for unit tests
 **Configurable**: Enable only needed processors
 **Flexible**: Choose fast (Tesseract) vs accurate (Unstructured)
 **Opt-in**: Disabled by default, no mandatory dependencies

Users upgrading from PR #190 need to update environment variables:
```bash
ENABLE_UNSTRUCTURED_PARSING=true

ENABLE_DOCUMENT_PROCESSING=true
ENABLE_UNSTRUCTURED=true
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 19:28:35 +02:00
yuisheaven f0e5333e43 Merge branch 'master' into feature/introduce_files_parsing_with_unstructured_service_for_webdav_files_retrieval 2025-10-25 17:23:38 +02:00
renovate-bot-cbcoutinho[bot] ff20031601 chore(deps): update docker.io/library/nextcloud docker tag to v32.0.1 2025-10-25 10:06:16 +00:00
Chris Coutinho d452684535 feat: Split read/write scopes into app:read/write scopes 2025-10-24 04:38:49 +02:00
yuisheaven 29df645d53 Merge branch 'master' into feature/introduce_files_parsing_with_unstructured_service_for_webdav_files_retrieval 2025-10-23 21:30:09 +02:00
Chris Coutinho 38e12db46a Merge pull request #233 from cbcoutinho/feature/jwt-scopes
feat: Initialize JWT-scoped tools
2025-10-23 12:23:12 +02:00
Chris Coutinho 54e975198f test: Update all test network hosts to respect iss claims from JWTs 2025-10-23 11:09:51 +02:00
Chris Coutinho e9a16c43b5 refactor: Update JWT client to use DCR, re-enable tool filtering 2025-10-23 09:33:06 +02:00
Chris Coutinho e48f5f3f30 feat(server): Add support for custom OIDC scopes and permissions via JWTs 2025-10-23 08:37:36 +02:00
Chris Coutinho c069d78f80 feat: Initialize JWT-scoped tools 2025-10-22 06:21:16 +02:00
renovate-bot-cbcoutinho[bot] e3436fecc0 chore(deps): update docker.io/library/nextcloud:32.0.0 docker digest to f9bec5c 2025-10-22 04:06:24 +00:00
yuisheaven 98627593d5 corrected smaller merge issues 2025-10-21 20:55:33 +02:00
yuisheaven 64649c902d Merge branch 'master' into feature/introduce_files_parsing_with_unstructured_service_for_webdav_files_retrieval 2025-10-21 20:37:00 +02:00
renovate-bot-cbcoutinho[bot] 08ebab9f48 chore(deps): update docker.io/library/nextcloud:32.0.0 docker digest to d3d8b9d 2025-10-21 16:06:08 +00:00
renovate-bot-cbcoutinho[bot] 27bb0a4b56 chore(deps): update docker.io/library/nextcloud:32.0.0 docker digest to 4fbd72f 2025-10-21 10:06:57 +00:00
Chris Coutinho 92e18825bc feat(caldav): Add support for tasks 2025-10-19 18:02:43 +02:00
Chris Coutinho 2f805e54b7 test: Migrate load test benchmark scripts to anyio
Remove unused redis container
2025-10-18 22:40:50 +02:00
renovate-bot-cbcoutinho[bot] 9b29eabfaa chore(deps): pin docker.io/library/nginx docker tag to 61e0128 2025-10-17 04:07:05 +00:00
Chris Coutinho 27519d0f62 test: Replace http server for recipes with nginx container 2025-10-17 04:30:03 +02:00
Chris Coutinho 0fd32ecd34 test: Fix test networking 2025-10-17 03:58:36 +02:00
Chris Coutinho 7004104873 test: Fix multi-user tests 2025-10-15 02:11:17 +02:00
Chris Coutinho 1023a7d9c7 chore: Remove comments 2025-10-14 01:23:49 +02:00
Chris Coutinho 057e25b653 chore: Add support for overriding public issuer URL
test: Add patch for PKCE support
2025-10-14 01:23:41 +02:00
Chris Coutinho f48d3714d2 test: Add restart to mcp containers in docker-compose.yml 2025-10-14 01:23:39 +02:00
Chris Coutinho 4d7e4b9a4b feat(server): Experimental support for OAuth2/OIDC authentication 2025-10-14 01:22:15 +02:00
renovate-bot-cbcoutinho[bot] f16af39b97 chore(deps): update docker.io/library/redis:alpine docker digest to 59b6e69 2025-10-10 22:09:45 +00:00
renovate-bot-cbcoutinho[bot] 391f418934 chore(deps): update docker.io/library/mariadb:lts docker digest to ae61197 2025-10-10 04:06:36 +00:00
renovate-bot-cbcoutinho[bot] 900d1bb462 chore(deps): update docker.io/library/redis:alpine docker digest to b4ab73c 2025-10-09 10:13:48 +00:00
renovate-bot-cbcoutinho[bot] 1402da0ac0 chore(deps): update docker.io/library/redis:alpine docker digest to 0ea5184 2025-10-09 04:06:45 +00:00
renovate-bot-cbcoutinho[bot] 5f3ff60531 chore(deps): update docker.io/library/nextcloud:32.0.0 docker digest to 3e70e4d 2025-10-04 16:07:30 +00:00
yuisheaven c9a687171a added envs for unstructured to control OCR quality and OCR languages 2025-10-04 05:21:02 +02:00
yuisheaven 642108ee91 added new "unstructured" docker service to compose stack and introduced new envs 2025-10-04 04:27:31 +02:00
renovate-bot-cbcoutinho[bot] 7463234ccb chore(deps): replace redis docker tag with docker.io/library/redis alpine 2025-10-03 22:07:16 +00:00
renovate-bot-cbcoutinho[bot] 0faa32fd10 chore(deps): replace mariadb docker tag with docker.io/library/mariadb lts 2025-10-02 10:04:52 +00:00
Chris Coutinho 8a9fa2a3c4 Merge pull request #180 from cbcoutinho/renovate/docker.io-library-nextcloud-32.0.0
chore(deps): update docker.io/library/nextcloud:32.0.0 docker digest to f4d0a4a
2025-10-02 08:08:04 +02:00
renovate-bot-cbcoutinho[bot] 9be03ef0de chore(deps): update mariadb:lts docker digest to 24264e9 2025-10-02 04:04:12 +00:00
renovate-bot-cbcoutinho[bot] eda6753253 chore(deps): update docker.io/library/nextcloud:32.0.0 docker digest to f4d0a4a 2025-10-01 22:06:55 +00:00
renovate-bot-cbcoutinho[bot] e6dc14c31f chore(deps): replace nextcloud docker tag with docker.io/library/nextcloud 32.0.0 2025-10-01 16:04:54 +00:00
renovate-bot-cbcoutinho[bot] 1a2a1f065f chore(deps): update nextcloud docker tag to v32 2025-09-30 22:11:16 +00:00
renovate-bot-cbcoutinho[bot] 3f8312e6f3 chore(deps): update nextcloud:31.0.9 docker digest to 88fe398 2025-09-28 22:05:34 +00:00
renovate-bot-cbcoutinho[bot] 290ad2edc2 chore(deps): update nextcloud:31.0.9 docker digest to 875511f 2025-09-27 10:05:24 +00:00
renovate-bot-cbcoutinho[bot] aafac732c6 chore(deps): update nextcloud:31.0.9 docker digest to 11f1580 2025-09-17 04:04:06 +00:00
renovate-bot-cbcoutinho[bot] 0600cea87b chore(deps): update mariadb:lts docker digest to 851a602 2025-09-16 04:05:11 +00:00
renovate-bot-cbcoutinho[bot] 948e7a4d91 chore(deps): update nextcloud docker tag to v31.0.9 2025-09-15 22:07:01 +00:00
renovate-bot-cbcoutinho[bot] 4ea6ce3477 chore(deps): update nextcloud:31.0.8 docker digest to 92bc503 2025-09-12 16:05:34 +00:00
Chris Coutinho 06042357f8 fix(docker): Provide --host 0.0.0.0 in default docker image 2025-09-11 17:44:45 +02:00