67 Commits

Author SHA1 Message Date
92f42ffce3 feat: implemented declarative schema-based settings system 2026-01-06 04:55:52 +01:00
58408d0dbe fix: fixed vectordb loneliness 2026-01-06 04:39:42 +01:00
2f1ac3c758 infra: simplified mongodb healthcheck 2026-01-06 04:36:52 +01:00
d3b69f7459 feat: enabled logging for alfred(-core) 2026-01-06 04:33:59 +01:00
50c8204fa0 infra: added granular logging configuration for mongodb 2026-01-06 02:50:49 +01:00
507fe0f40e chore: updated dependencies 2026-01-06 02:41:18 +01:00
b7b40eada1 fix: set proper database name for mongodb 2026-01-06 02:33:35 +01:00
9765386405 feat: named docker image to avoid docker picking the wrong one 2026-01-06 02:19:00 +01:00
aa89a3fb00 doc: updated README.md
All checks were successful
Renovate Bot / renovate (push) Successful in 22s
2026-01-04 13:30:54 +01:00
64aeb5fc80 chore: removed deprecated cli.py file 2026-01-04 13:29:38 +01:00
9540520dc4 feat: updated default start mode from core to full 2026-01-03 05:49:51 +01:00
300ed387f5 fix: fixed build vars generation not being called 2026-01-01 06:00:55 +01:00
dea81de5b5 fix: updated CI workflow and added .env.make generation for CI 2026-01-01 05:33:39 +01:00
01a00a12af chore: bump version 0.1.6 → 0.1.7
Some checks failed
CI/CD Awesome Pipeline / Test (push) Successful in 5m54s
CI/CD Awesome Pipeline / Build & Push to Registry (push) Failing after 1m9s
2026-01-01 05:04:37 +01:00
504d0162bb infra: added proper settings handling & orchestration and app bootstrap (.env)
Reviewed-on: https://gitea.iswearihadsomethingforthis.net/francwa/alfred/pulls/18
2026-01-01 03:55:35 +00:00
cda23d074f feat: added current alfred version from pyproject.py to healthcheck 2026-01-01 04:51:45 +01:00
0357108077 infra: added orchestration and app bootstrap (.env) 2026-01-01 04:48:32 +01:00
ab1df3dd0f fix: forgot to lint/format 2026-01-01 04:48:32 +01:00
c50091f6bf feat: added proper settings handling 2026-01-01 04:48:32 +01:00
8b406370f1 chore: updated some dependencies 2026-01-01 03:08:22 +01:00
c56bf2b92c feat: added Claude.AI to available providers 2025-12-29 02:13:11 +01:00
b1507db4d0 fix: fixed test image not being built before running tests
All checks were successful
Renovate Bot / renovate (push) Successful in 1m26s
2025-12-28 13:24:03 +01:00
3074962314 infra: proper librechat integration & improved configuration handling
Reviewed-on: https://gitea.iswearihadsomethingforthis.net/francwa/alfred/pulls/11
2025-12-28 11:56:53 +00:00
84799879bb fix: added data dir to .dockerignore 2025-12-28 12:02:44 +01:00
1052c1b619 infra: added modules version to .env.example 2025-12-28 06:07:34 +01:00
9958b8e848 feat: created placeholders for various API models 2025-12-28 06:06:28 +01:00
b15161dad7 fix: provided OpenAI API key and fixed docker-compose configuration to enable RAG service 2025-12-28 06:04:08 +01:00
52f025ae32 chore: ran linter & formatter again 2025-12-27 20:07:48 +01:00
2cfe7a035b infra: moved makefile logic to python and added .env setup 2025-12-27 19:51:40 +01:00
2441c2dc29 infra: rewrote docker-compose for proper integration of librechat 2025-12-27 19:49:43 +01:00
261a1f3918 fix: fixed real data directory being used in tests 2025-12-27 19:48:13 +01:00
253903a1e5 infra: made pyproject the SSOT 2025-12-27 19:46:27 +01:00
20a113e335 infra: updated .env.example 2025-12-27 19:43:31 +01:00
fed83e7d79 chore: bumped fastapi version 2025-12-27 19:42:16 +01:00
3880a4ec49 chore: ran linter and formatter 2025-12-27 19:41:22 +01:00
6195abbaa5 chore: fixed imports and tests configuration 2025-12-27 19:39:36 +01:00
b132554631 infra: updated .gitignore 2025-12-27 02:30:14 +01:00
561796cec1 infra: removed CORE_DIR variable from Makefile (not relevant anymore) 2025-12-24 11:28:28 +01:00
26d90acc16 infra: moved manifest files in librechat dir 2025-12-24 09:46:25 +01:00
d8234b2958 chore: cleaned up .env.example 2025-12-24 08:17:47 +01:00
156d1fe567 fix: fixed Dockerfile 2025-12-24 08:17:12 +01:00
f8eee120cf infra: backed up old deploy-compose 2025-12-24 08:16:30 +01:00
c5e4a5e1a7 infra: removed occurences of alfred_api_key (not implemented after all) 2025-12-24 08:11:30 +01:00
d10c9160f3 infra: renamed broken references to alfred 2025-12-24 08:09:26 +01:00
1f88e99e8b infra: reorganized repo 2025-12-24 07:50:09 +01:00
e097a13221 chore: upgraded to python 3.14 and some dependencies
Reviewed-on: https://gitea.iswearihadsomethingforthis.net/francwa/agent_media/pulls/8
2025-12-22 14:04:46 +00:00
086fff803d feat: configured CI/CD pipeline to allow build for feature branches 2025-12-22 14:48:50 +01:00
45fbf975b3 feat: improved rules for renovate 2025-12-22 14:25:16 +01:00
b8f2798e29 chore: updated fastapi and uvicorn 2025-12-22 14:20:22 +01:00
c762d91eb1 chore: updated to python 3.14 2025-12-22 14:08:27 +01:00
35a68387ab infra: use python version from pyproject.toml 2025-12-22 13:46:40 +01:00
9b13c69631 chore: updated meilisearch 2025-12-22 13:25:54 +01:00
2ca1ea29b2 infra: added meilisearch to renovate exclusion list 2025-12-22 13:22:55 +01:00
5e86615bde fix: added renovate token with proper permissions 2025-12-22 12:59:24 +01:00
6701a4b392 infra: added Renovate bot 2025-12-22 12:50:59 +01:00
68372405d6 fix: downgraded upload-artifact action to v3 from v4 2025-12-22 12:13:50 +01:00
f1ea0de247 fix: fixed indentation error 2025-12-22 12:04:47 +01:00
974d008825 feat: finalized CI/CD pipeline setup 2025-12-22 11:59:36 +01:00
8a87d94e6d fix: use docker image for trivy vulnerability scanner
Some checks failed
CI/CD Awesome Pipeline / Test (push) Successful in 1m23s
CI/CD Awesome Pipeline / Build & Push to Registry (push) Failing after 5m9s
2025-12-22 11:38:35 +01:00
ec99a501fc fix! added directive to Dockerfile 2025-12-22 11:37:48 +01:00
c256b26601 fix: fixed vulnerability scanner issue in CI/CD pipeline
Some checks failed
CI/CD Awesome Pipeline / Test (push) Successful in 48s
CI/CD Awesome Pipeline / Build & Push to Registry (push) Failing after 6m18s
2025-12-22 10:59:34 +01:00
56a3c1257d infra: added trivy vulnerability scanner to CI/CD
Some checks failed
CI/CD Awesome Pipeline / Test (push) Successful in 1m36s
CI/CD Awesome Pipeline / Build & Push to Registry (push) Failing after 7m10s
2025-12-22 10:01:52 +01:00
79d23f936a fix: fixed typo 2025-12-22 09:40:43 +01:00
f02e916d33 fix: fixed config gathering in ci.yml
Some checks failed
CI/CD Awesome Pipeline / Test (push) Successful in 1m34s
CI/CD Awesome Pipeline / Build & Push to Registry (push) Failing after 14m19s
2025-12-22 09:16:55 +01:00
4e64c83c4b fix: updated build and push CI/CD configuration
Some checks failed
CI/CD Awesome Pipeline / Test (push) Successful in 1m4s
CI/CD Awesome Pipeline / Build & Push to Registry (push) Failing after 2m32s
2025-12-22 08:45:31 +01:00
07cae9abd1 chore: bump version 0.1.5 → 0.1.6
All checks were successful
CI/CD Awesome Pipeline / Test (push) Successful in 1m23s
CI/CD Awesome Pipeline / Build & Push to Registry (push) Successful in 1m4s
2025-12-21 13:54:02 +01:00
21b2dffc37 fix: added gitea token 2025-12-21 13:53:49 +01:00
137 changed files with 5272 additions and 1669 deletions

View File

@@ -1,5 +1,5 @@
[tool.bumpversion]
current_version = "0.1.5"
current_version = "0.1.7"
parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)"
serialize = ["{major}.{minor}.{patch}"]
search = "{current_version}"

View File

@@ -22,8 +22,7 @@ venv
.venv
env
.env
.env.*
.env-
# IDE
.vscode
.idea
@@ -41,11 +40,8 @@ docs/
*.md
!README.md
# Tests
tests/
pytest.ini
# Data (will be mounted as volumes)
# Data
data/
memory_data/
logs/
*.log

View File

@@ -1,69 +1,93 @@
# Agent Media - Environment Variables
MAX_HISTORY_MESSAGES=10
MAX_TOOL_ITERATIONS=10
REQUEST_TIMEOUT=30
# LibreChat Security Keys
# Generate secure keys with: openssl rand -base64 32
JWT_SECRET=your-super-secret-jwt-key-change-this-in-production
JWT_REFRESH_SECRET=your-super-secret-refresh-key-change-this-too
# LLM Settings
LLM_TEMPERATURE=0.2
# Generate with: openssl rand -hex 16 (for CREDS_KEY)
CREDS_KEY=your-32-character-secret-key-here
# Persistence
DATA_STORAGE_DIR=data
# Generate with: openssl rand -hex 8 (for CREDS_IV)
CREDS_IV=your-16-character-iv-here
# Network configuration
HOST=0.0.0.0
PORT=3080
# LibreChat Configuration
DOMAIN_CLIENT=http://localhost:3080
DOMAIN_SERVER=http://localhost:3080
# Build informations (Synced with pyproject.toml via bootstrap)
ALFRED_VERSION=
IMAGE_NAME=
LIBRECHAT_VERSION=
PYTHON_VERSION=
PYTHON_VERSION_SHORT=
RAG_VERSION=
RUNNER=
SERVICE_NAME=
# Session expiry (in milliseconds)
# Default: 15 minutes
SESSION_EXPIRY=900000
# --- SECURITY KEYS (CRITICAL) ---
# These are used for session tokens and encrypting sensitive data in MongoDB.
# If you lose these, you lose access to encrypted stored credentials.
JWT_SECRET=
JWT_REFRESH_SECRET=
CREDS_KEY=
CREDS_IV=
# Refresh token expiry (in milliseconds)
# Default: 7 days
REFRESH_TOKEN_EXPIRY=604800000
# --- DATABASES (AUTO-SECURED) ---
# Alfred uses MongoDB for application state and PostgreSQL for Vector RAG.
# Passwords will be generated as 24-character secure tokens if left blank.
# Meilisearch Configuration
# Master key for Meilisearch (generate with: openssl rand -base64 32)
MEILI_MASTER_KEY=DrhYf7zENyR6AlUCKmnz0eYASOQdl6zxH7s7MKFSfFU
# MongoDB (Application Data)
MONGO_URI=
MONGO_HOST=mongodb
MONGO_PORT=27017
MONGO_USER=alfred
MONGO_PASSWORD=
MONGO_DB_NAME=LibreChat
# PostgreSQL Configuration (for RAG API)
POSTGRES_DB=librechat_rag
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
# PostgreSQL (Vector Database / RAG)
POSTGRES_URI=
POSTGRES_HOST=vectordb
POSTGRES_PORT=5432
POSTGRES_USER=alfred
POSTGRES_PASSWORD=
POSTGRES_DB_NAME=alfred
# RAG API Configuration (Vector Database)
RAG_COLLECTION_NAME=testcollection
RAG_EMBEDDINGS_PROVIDER=openai
RAG_EMBEDDINGS_MODEL=text-embedding-3-small
# --- EXTERNAL SERVICES ---
# Media Metadata (Required)
# Get your key at https://www.themoviedb.org/
TMDB_API_KEY=
TMDB_BASE_URL=https://api.themoviedb.org/3
# API Keys
# OpenAI API Key (required for RAG embeddings)
OPENAI_API_KEY=your-openai-api-key-here
# Deepseek API Key (for LLM in agent-brain)
DEEPSEEK_API_KEY=your-deepseek-api-key-here
# Agent Brain Configuration
# LLM Provider (deepseek or ollama)
LLM_PROVIDER=deepseek
# Memory storage directory (inside container)
MEMORY_STORAGE_DIR=/data/memory
# API Key for agent-brain (used by LibreChat custom endpoint)
AGENT_BRAIN_API_KEY=agent-brain-secret-key
# External Services (Optional)
# TMDB API Key (for movie metadata)
TMDB_API_KEY=your-tmdb-key
# qBittorrent Configuration
QBITTORRENT_URL=http://localhost:8080
# qBittorrent integration
QBITTORRENT_URL=http://qbittorrent:16140
QBITTORRENT_USERNAME=admin
QBITTORRENT_PASSWORD=adminpass
QBITTORRENT_PASSWORD=
QBITTORRENT_PORT=16140
# Debug Options
DEBUG_LOGGING=false
DEBUG_CONSOLE=false
# Meilisearch
MEILI_ENABLED=FALSE
MEILI_NO_ANALYTICS=TRUE
MEILI_HOST=http://meilisearch:7700
MEILI_MASTER_KEY=
# --- LLM CONFIGURATION ---
# Providers: 'local', 'openai', 'anthropic', 'deepseek', 'google', 'kimi'
DEFAULT_LLM_PROVIDER=local
# Local LLM (Ollama)
OLLAMA_BASE_URL=http://ollama:11434
OLLAMA_MODEL=llama3.3:latest
# --- API KEYS (OPTIONAL) ---
# Fill only the ones you intend to use.
ANTHROPIC_API_KEY=
DEEPSEEK_API_KEY=
GOOGLE_API_KEY=
KIMI_API_KEY=
OPENAI_API_KEY=
# --- RAG ENGINE ---
# Enable/Disable the Retrieval Augmented Generation system
RAG_ENABLED=TRUE
RAG_API_URL=http://rag_api:8000
RAG_API_PORT=8000
EMBEDDINGS_PROVIDER=ollama
EMBEDDINGS_MODEL=nomic-embed-text

View File

@@ -2,11 +2,10 @@ name: CI/CD Awesome Pipeline
on:
push:
branches: [main]
tags:
- 'v*.*.*'
pull_request:
branches: [main]
workflow_dispatch:
env:
REGISTRY_URL: ${{ vars.REGISTRY_URL || 'gitea.iswearihadsomethingforthis.net' }}
@@ -30,34 +29,64 @@ jobs:
name: Build & Push to Registry
runs-on: ubuntu-latest
needs: test
steps:
- name: Debug ref
run: |
echo "github.ref = ${{ github.ref }}"
echo "GITHUB_REF = $GITHUB_REF"
echo "github.event_name = ${{ github.event_name }}"
exit 1
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Generate build variables
run: python scripts/generate_build_vars.py
- name: Load config from Makefile
id: config
run: |
eval "$(make _ci-image-name)"
echo "image_name=${IMAGE_NAME}" >> $GITHUB_OUTPUT
run: make -s _ci-dump-config >> $GITHUB_OUTPUT
- name: Extract version from tag
id: version
run: echo "version=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT
- name: 🏷️ Docker Metadata (Tags & Labels)
id: meta
uses: docker/metadata-action@v5
with:
images: gitea.iswearihadsomethingforthis.net/francwa/${{ steps.config.outputs.image_name }}
tags: |
# Tagged (v1.2.3)
type=semver,pattern={{ version }}
# Latest (main)
type=raw,value=latest,enable={{ is_default_branch }}
# Feature branches
type=ref,event=branch
- name: Build production image
run: make build
- name: Login to Gitea Registry
uses: docker/login-action@v3
with:
registry: gitea.iswearihadsomethingforthis.net
username: ${{ gitea.actor }}
password: ${{ secrets.G1T34_TOKEN }}
- name: Tag and push to registry
run: |
docker tag ${{ steps.config.outputs.image_name }}:latest ${{ env.REGISTRY_URL }}/${{ env.REGISTRY_USER }}/${{ steps.config.outputs.image_name }}:${{ steps.version.outputs.version }}
docker tag ${{ steps.config.outputs.image_name }}:latest ${{ env.REGISTRY_URL }}/${{ env.REGISTRY_USER }}/${{ steps.config.outputs.image_name }}:latest
echo "${{ secrets.GITEA_TOKEN }}" | docker login ${{ env.REGISTRY_URL }} -u ${{ env.REGISTRY_USER }} --password-stdin
docker push ${{ env.REGISTRY_URL }}/${{ env.REGISTRY_USER }}/${{ steps.config.outputs.image_name }}:${{ steps.version.outputs.version }}
docker push ${{ env.REGISTRY_URL }}/${{ env.REGISTRY_USER }}/${{ steps.config.outputs.image_name }}:latest
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
PYTHON_VERSION=${{ steps.config.outputs.python_version }}
PYTHON_VERSION_SHORT=${{ steps.config.outputs.python_version_short }}
RUNNER=${{ steps.config.outputs.runner }}
- name: 🛡️ Run Trivy Vulnerability Scanner
uses: docker://aquasec/trivy:latest
env:
TRIVY_USERNAME: ${{ gitea.actor }}
TRIVY_PASSWORD: ${{ secrets.G1T34_TOKEN }}
# Unset the fake GITHUB_TOKEN injected by Gitea
GITHUB_TOKEN: ""
with:
args: image --format table --output trivy-report.txt --exit-code 0 --ignore-unfixed --severity CRITICAL,HIGH gitea.iswearihadsomethingforthis.net/francwa/${{ steps.config.outputs.image_name }}:latest
- name: 📤 Upload Security Report
uses: actions/upload-artifact@v3
with:
name: security-report
path: trivy-report.txt
retention-days: 7

View File

@@ -0,0 +1,22 @@
name: Renovate Bot
on:
schedule:
# Every Monday 4AM
- cron: '0 4 * * 1'
workflow_dispatch:
jobs:
renovate:
runs-on: ubuntu-latest
steps:
- name: Run Renovate
uses: docker://renovate/renovate:latest
env:
RENOVATE_PLATFORM: "gitea"
RENOVATE_ENDPOINT: "https://gitea.iswearihadsomethingforthis.net/api/v1"
RENOVATE_TOKEN: "${{ secrets.RENOVATE_TOKEN }}"
RENOVATE_REPOSITORIES: '["${{ gitea.repository }}"]'
RENOVATE_GIT_AUTHOR: "Renovate Bot <renovate@bot.local>"
# Might need a free github token if lots of depencies
# RENOVATE_GITHUB_TOKEN: "${{ secrets.GITHUB_COM_TOKEN }}"

6
.gitignore vendored
View File

@@ -59,3 +59,9 @@ Thumbs.db
# Backup files
*.backup
# Application data dir
data/*
# Application logs
logs/*

261
CONTRIBUTE.md Normal file
View File

@@ -0,0 +1,261 @@
# Contributing to Alfred
## Settings Management System
Alfred uses a **declarative, schema-based configuration system** that ensures type safety, validation, and maintainability.
### Architecture Overview
```
settings.toml # Schema definitions (single source of truth)
settings_schema.py # Parser & validation
settings_bootstrap.py # Generation & resolution
.env # Runtime configuration
.env.make # Build variables for Makefile
settings.py # Pydantic Settings (runtime validation)
```
### Key Files
- **`settings.toml`** — Declarative schema for all settings
- **`alfred/settings_schema.py`** — Schema parser and validation logic
- **`alfred/settings_bootstrap.py`** — Bootstrap logic (generates `.env` and `.env.make`)
- **`alfred/settings.py`** — Pydantic Settings class (runtime)
- **`.env`** — Generated configuration file (gitignored)
- **`.env.make`** — Build variables for Makefile (gitignored)
### Setting Sources
Settings can come from different sources:
| Source | Description | Example |
|--------|-------------|---------|
| `toml` | From `pyproject.toml` | Version numbers, build config |
| `env` | From `.env` file | User-provided values, API keys |
| `generated` | Auto-generated secrets | JWT secrets, passwords |
| `computed` | Calculated from other settings | Database URIs |
### How to Add a New Setting
#### 1. Define in `settings.toml`
```toml
[tool.alfred.settings_schema.MY_NEW_SETTING]
type = "string" # string, integer, float, boolean, secret, computed
source = "env" # env, toml, generated, computed
default = "default_value" # Optional: default value
description = "Description here" # Required: clear description
category = "app" # app, api, database, security, build
required = true # Optional: default is true
validator = "range:1:100" # Optional: validation rule
export_to_env_make = false # Optional: export to .env.make for Makefile
```
#### 2. Regenerate Configuration
```bash
make bootstrap
```
This will:
- Read the schema from `settings.toml`
- Generate/update `.env` with the new setting
- Generate/update `.env.make` if `export_to_env_make = true`
- Preserve existing secrets
#### 3. Validate
```bash
make validate
```
This ensures all settings are valid according to the schema.
#### 4. Use in Code
The setting is automatically available in `settings.py`:
```python
from alfred.settings import settings
print(settings.my_new_setting)
```
### Setting Types
#### String Setting
```toml
[tool.alfred.settings_schema.API_URL]
type = "string"
source = "env"
default = "https://api.example.com"
description = "API base URL"
category = "api"
```
#### Integer Setting with Validation
```toml
[tool.alfred.settings_schema.MAX_RETRIES]
type = "integer"
source = "env"
default = 3
description = "Maximum retry attempts"
category = "app"
validator = "range:1:10"
```
#### Secret (Auto-generated)
```toml
[tool.alfred.settings_schema.API_SECRET]
type = "secret"
source = "generated"
secret_rule = "32:b64" # 32 bytes, base64 encoded
description = "API secret key"
category = "security"
```
Secret rules:
- `"32:b64"` — 32 bytes, URL-safe base64
- `"16:hex"` — 16 bytes, hexadecimal
#### Computed Setting
```toml
[tool.alfred.settings_schema.DATABASE_URL]
type = "computed"
source = "computed"
compute_from = ["DB_HOST", "DB_PORT", "DB_NAME"]
compute_template = "postgresql://{DB_HOST}:{DB_PORT}/{DB_NAME}"
description = "Database connection URL"
category = "database"
```
#### From TOML (Build Variables)
```toml
[tool.alfred.settings_schema.APP_VERSION]
type = "string"
source = "toml"
toml_path = "tool.poetry.version"
description = "Application version"
category = "build"
export_to_env_make = true # Available in Makefile
```
### Validators
Available validators:
- **`range:min:max`** — Numeric range validation
```toml
validator = "range:0.0:2.0" # For floats
validator = "range:1:100" # For integers
```
### Categories
Organize settings by category:
- **`app`** — Application settings
- **`api`** — API keys and external services
- **`database`** — Database configuration
- **`security`** — Secrets and security keys
- **`build`** — Build-time configuration
### Best Practices
1. **Always add a description** — Make it clear what the setting does
2. **Use appropriate types** — Don't use strings for numbers
3. **Add validation** — Use validators for numeric ranges
4. **Categorize properly** — Helps with organization
5. **Use computed settings** — For values derived from others (e.g., URIs)
6. **Mark secrets as generated** — Let the system handle secret generation
7. **Export build vars** — Set `export_to_env_make = true` for Makefile variables
### Workflow Example
```bash
# 1. Edit settings.toml
vim settings.toml
# 2. Regenerate configuration
make bootstrap
# 3. Validate
make validate
# 4. Test
python -c "from alfred.settings import settings; print(settings.my_new_setting)"
# 5. Commit (settings.toml only, not .env)
git add settings.toml
git commit -m "Add MY_NEW_SETTING"
```
### Commands
```bash
make bootstrap # Generate .env and .env.make from schema
make validate # Validate all settings against schema
make help # Show all available commands
```
### Troubleshooting
**Setting not found in schema:**
```
KeyError: Missing [tool.alfred.settings_schema] section
```
→ Check that `settings.toml` exists and has the correct structure
**Validation error:**
```
ValueError: MY_SETTING must be between 1 and 100, got 150
```
→ Check the validator in `settings.toml` and adjust the value in `.env`
**Secret not preserved:**
→ Secrets are automatically preserved during `make bootstrap`. If lost, they were never in `.env` (check `.env` exists before running bootstrap)
### Testing
When adding a new setting, consider adding tests:
```python
# tests/test_settings_schema.py
def test_my_new_setting(self, create_schema_file):
"""Test MY_NEW_SETTING definition."""
schema_toml = """
[tool.alfred.settings_schema.MY_NEW_SETTING]
type = "string"
source = "env"
default = "test"
"""
base_dir = create_schema_file(schema_toml)
schema = load_schema(base_dir)
definition = schema.get("MY_NEW_SETTING")
assert definition.default == "test"
```
### Migration from Old System
If you're migrating from the old system:
1. Settings are now in `settings.toml` instead of scattered across files
2. No more `.env.example` — schema is the source of truth
3. Secrets are auto-generated and preserved
4. Validation happens at bootstrap time, not just runtime
---
## Questions?
Open an issue or check the existing settings in `settings.toml` for examples.

View File

@@ -1,5 +1,6 @@
# Dockerfile for Agent Media
# Multi-stage build for smaller image size
# syntax=docker/dockerfile:1
# check=skip=InvalidDefaultArgInFrom
ARG PYTHON_VERSION
ARG PYTHON_VERSION_SHORT
ARG RUNNER
@@ -26,10 +27,10 @@ RUN --mount=type=cache,target=/root/.cache/pip \
pip install $RUNNER
# Set working directory for dependency installation
WORKDIR /tmp
WORKDIR /app
# Copy dependency files
COPY brain/pyproject.toml brain/poetry.lock* brain/uv.lock* Makefile ./
COPY pyproject.toml poetry.lock* uv.lock* Makefile ./
# Install dependencies as root (to avoid permission issues with system packages)
RUN --mount=type=cache,target=/root/.cache/pip \
@@ -42,6 +43,11 @@ RUN --mount=type=cache,target=/root/.cache/pip \
uv pip install --system -r pyproject.toml; \
fi
COPY alfred/ ./alfred/
COPY scripts/ ./scripts/
COPY .env.example ./
COPY settings.toml ./
# ===========================================
# Stage 2: Testing
# ===========================================
@@ -58,12 +64,7 @@ RUN --mount=type=cache,target=/root/.cache/pip \
uv pip install --system -e .[dev]; \
fi
COPY brain/agent/ ./agent/
COPY brain/application/ ./application/
COPY brain/domain/ ./domain/
COPY brain/infrastructure/ ./infrastructure/
COPY brain/tests/ ./tests/
COPY brain/app.py .
COPY tests/ ./tests
# ===========================================
# Stage 3: Runtime
@@ -72,10 +73,11 @@ FROM python:${PYTHON_VERSION}-slim-bookworm AS runtime
ARG PYTHON_VERSION_SHORT
# TODO: A-t-on encore besoin de toutes les clés ?
ENV LLM_PROVIDER=deepseek \
MEMORY_STORAGE_DIR=/data/memory \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONPATH=/home/appuser/app \
PYTHONPATH=/home/appuser \
PYTHONUNBUFFERED=1
# Install runtime dependencies (needs root)
@@ -88,28 +90,28 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN useradd -m -u 1000 -s /bin/bash appuser
# Create data directories (needs root for /data)
RUN mkdir -p /data/memory /data/logs \
&& chown -R appuser:appuser /data
RUN mkdir -p /data /logs \
&& chown -R appuser:appuser /data /logs
# Switch to non-root user
USER appuser
# Set working directory (owned by appuser)
WORKDIR /home/appuser/app
WORKDIR /home/appuser
# Copy Python packages from builder stage
COPY --from=builder /usr/local/lib/python${PYTHON_VERSION_SHORT}/site-packages /usr/local/lib/python${PYTHON_VERSION_SHORT}/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
# Copy application code (already owned by appuser)
COPY --chown=appuser:appuser brain/agent/ ./agent/
COPY --chown=appuser:appuser brain/application/ ./application/
COPY --chown=appuser:appuser brain/domain/ ./domain/
COPY --chown=appuser:appuser brain/infrastructure/ ./infrastructure/
COPY --chown=appuser:appuser brain/app.py .
COPY --chown=appuser:appuser alfred/ ./alfred
COPY --chown=appuser:appuser scripts/ ./scripts
COPY --chown=appuser:appuser .env.example ./
COPY --chown=appuser:appuser pyproject.toml ./
COPY --chown=appuser:appuser settings.toml ./
# Create volumes for persistent data
VOLUME ["/data/memory", "/data/logs"]
VOLUME ["/data", "/logs"]
# Expose port
EXPOSE 8000
@@ -118,5 +120,4 @@ EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD python -c "import requests; requests.get('http://localhost:8000/health', timeout=5).raise_for_status()" || exit 1
# Run the application
CMD ["python", "-m", "uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
CMD ["python", "-m", "uvicorn", "alfred.app:app", "--host", "0.0.0.0", "--port", "8000"]

393
Makefile
View File

@@ -1,249 +1,194 @@
.POSIX:
.SUFFIXES:
.DEFAULT_GOAL := help
# --- SETTINGS ---
PYTHON_VERSION = 3.12.7
PYTHON_VERSION_SHORT = $(shell echo $(PYTHON_VERSION) | cut -d. -f1,2)
# Change to 'uv' when ready.
RUNNER ?= poetry
# --- Load Config from pyproject.toml ---
-include .env.make
export PYTHON_VERSION
export PYTHON_VERSION_SHORT
export RUNNER
export IMAGE_NAME
# --- Profiles management ---
# Usage: make up p=rag,meili
p ?= full
PROFILES_PARAM := COMPOSE_PROFILES=$(p)
# --- VARIABLES ---
CORE_DIR = brain
SERVICE_NAME = agent_media
IMAGE_NAME = agent_media
# --- ADAPTERS ---
# UV uses "sync", Poetry uses "install". Both install DEV deps by default.
INSTALL_CMD = $(if $(filter uv,$(RUNNER)),sync,install)
# --- MACROS ---
ARGS = $(filter-out $@,$(MAKECMDGOALS))
BUMP_CMD = cd $(CORE_DIR) && $(RUNNER) run bump-my-version bump
COMPOSE_CMD = docker-compose
DOCKER_CMD = docker build \
# --- Commands ---
DOCKER_COMPOSE := docker compose
DOCKER_BUILD := docker build --no-cache \
--build-arg PYTHON_VERSION=$(PYTHON_VERSION) \
--build-arg PYTHON_VERSION_SHORT=$(PYTHON_VERSION_SHORT) \
--build-arg RUNNER=$(RUNNER) \
-f $(CORE_DIR)/Dockerfile \
-t $(IMAGE_NAME):latest .
--build-arg RUNNER=$(RUNNER)
RUNNER_ADD = cd $(CORE_DIR) && $(RUNNER) add
RUNNER_HOOKS = cd $(CORE_DIR) && $(RUNNER) run pre-commit install -c ../.pre-commit-config.yaml
RUNNER_INSTALL = cd $(CORE_DIR) && $(RUNNER) $(INSTALL_CMD)
RUNNER_RUN = cd $(CORE_DIR) && $(RUNNER) run
RUNNER_UPDATE = cd $(CORE_DIR) && $(RUNNER) update
# --- Phony ---
.PHONY: .env bootstrap validate up down restart logs ps shell build build-test install \
update install-hooks test coverage lint format clean major minor patch help
# --- STYLES ---
B = \033[1m
G = \033[32m
T = \033[36m
R = \033[0m
# --- Setup ---
.env:
@echo "Initializing environment..."
@python scripts/bootstrap.py \
&& echo "✓ Environment ready" \
|| (echo "✗ Environment setup failed" && exit 1)
# --- TARGETS ---
.PHONY: add build build-test check-docker check-runner clean coverage down format help init-dotenv install install-hooks lint logs major minor patch prune ps restart run shell test up update _check_branch _ci-image-name _ci-run-tests
# .env.make is automatically generated by bootstrap.py when .env is created
.env.make: .env
# Catch-all for args
%:
@:
bootstrap: .env
add: check-runner
@echo "$(T) Adding dependency ($(RUNNER)): $(ARGS)$(R)"
$(RUNNER_ADD) $(ARGS)
validate:
@echo "Validating settings..."
@python scripts/validate_settings.py \
&& echo "✓ Settings valid" \
|| (echo "✗ Settings validation failed" && exit 1)
build: check-docker
@echo "$(T)🐳 Building Docker image...$(R)"
$(DOCKER_CMD)
@echo "✅ Image $(IMAGE_NAME):latest ready."
# --- Docker ---
up: .env
@echo "Starting containers with profiles: [full]..."
@$(PROFILES_PARAM) $(DOCKER_COMPOSE) up -d --remove-orphans \
&& echo "✓ Containers started" \
|| (echo "✗ Failed to start containers" && exit 1)
build-test: check-docker
@echo "$(T)🐳 Building test image (with dev deps)...$(R)"
docker build \
--build-arg RUNNER=$(RUNNER) \
--build-arg PYTHON_VERSION=$(PYTHON_VERSION) \
--build-arg PYTHON_VERSION_SHORT=$(PYTHON_VERSION_SHORT) \
-f $(CORE_DIR)/Dockerfile \
--target test \
-t $(IMAGE_NAME):test .
@echo "✅ Test image $(IMAGE_NAME):test ready."
down:
@echo "Stopping containers..."
@$(PROFILES_PARAM) $(DOCKER_COMPOSE) down \
&& echo "✓ Containers stopped" \
|| (echo "✗ Failed to stop containers" && exit 1)
check-docker:
@command -v docker >/dev/null 2>&1 || { echo "$(R)❌ Docker not installed$(R)"; exit 1; }
@docker info >/dev/null 2>&1 || { echo "$(R)❌ Docker daemon not running$(R)"; exit 1; }
restart:
@echo "Restarting containers..."
@$(PROFILES_PARAM) $(DOCKER_COMPOSE) restart \
&& echo "✓ Containers restarted" \
|| (echo "✗ Failed to restart containers" && exit 1)
check-runner:
@command -v $(RUNNER) >/dev/null 2>&1 || { echo "$(R)$(RUNNER) not installed$(R)"; exit 1; }
logs:
@echo "Following logs (Ctrl+C to exit)..."
@$(PROFILES_PARAM) $(DOCKER_COMPOSE) logs -f
ps:
@echo "Container status:"
@$(PROFILES_PARAM) $(DOCKER_COMPOSE) ps
shell:
@echo "Opening shell in $(SERVICE_NAME)..."
@$(DOCKER_COMPOSE) exec $(SERVICE_NAME) /bin/bash
# --- Build ---
build: .env.make
@echo "Building image $(IMAGE_NAME):latest ..."
@$(DOCKER_BUILD) -t $(IMAGE_NAME):latest . \
&& echo "✓ Build complete" \
|| (echo "✗ Build failed" && exit 1)
build-test: .env.make
@echo "Building test image $(IMAGE_NAME):test..."
@$(DOCKER_BUILD) --target test -t $(IMAGE_NAME):test . \
&& echo "✓ Test image built" \
|| (echo "✗ Build failed" && exit 1)
# --- Dependencies ---
install:
@echo "Installing dependencies with $(RUNNER)..."
@$(RUNNER) install \
&& echo "✓ Dependencies installed" \
|| (echo "✗ Installation failed" && exit 1)
install-hooks:
@echo "Installing pre-commit hooks..."
@$(RUNNER) run pre-commit install \
&& echo "✓ Hooks installed" \
|| (echo "✗ Hook installation failed" && exit 1)
update:
@echo "Updating dependencies with $(RUNNER)..."
@$(RUNNER) update \
&& echo "✓ Dependencies updated" \
|| (echo "✗ Update failed" && exit 1)
# --- Quality ---
test:
@echo "Running tests..."
@$(RUNNER) run pytest \
&& echo "✓ Tests passed" \
|| (echo "✗ Tests failed" && exit 1)
coverage:
@echo "Running tests with coverage..."
@$(RUNNER) run pytest --cov=. --cov-report=html --cov-report=term \
&& echo "✓ Coverage report generated" \
|| (echo "✗ Coverage failed" && exit 1)
lint:
@echo "Linting code..."
@$(RUNNER) run ruff check --fix . \
&& echo "✓ Linting complete" \
|| (echo "✗ Linting failed" && exit 1)
format:
@echo "Formatting code..."
@$(RUNNER) run ruff format . && $(RUNNER) run ruff check --fix . \
&& echo "✓ Code formatted" \
|| (echo "✗ Formatting failed" && exit 1)
clean:
@echo "$(T)🧹 Cleaning caches...$(R)"
cd $(CORE_DIR) && rm -rf .ruff_cache __pycache__ .pytest_cache
find $(CORE_DIR) -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find $(CORE_DIR) -type d -name ".pytest_cache" -exec rm -rf {} + 2>/dev/null || true
find $(CORE_DIR) -type f -name "*.pyc" -delete 2>/dev/null || true
@echo "✅ Caches cleaned."
@echo "Cleaning build artifacts..."
@rm -rf .ruff_cache __pycache__ .pytest_cache htmlcov .coverage
@find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
@echo "✓ Cleanup complete"
coverage: check-runner
@echo "$(T)📊 Running tests with coverage...$(R)"
$(RUNNER_RUN) pytest --cov=. --cov-report=html --cov-report=term $(ARGS)
@echo "✅ Report generated in htmlcov/"
# --- Versioning ---
major minor patch: _check-main
@echo "Bumping $@ version..."
@$(RUNNER) run bump-my-version bump $@ \
&& echo "✓ Version bumped" \
|| (echo "✗ Version bump failed" && exit 1)
down: check-docker
@echo "$(T)🛑 Stopping containers...$(R)"
$(COMPOSE_CMD) down
@echo "✅ System stopped."
@echo "Pushing tags..."
@git push --tags \
&& echo "✓ Tags pushed" \
|| (echo "✗ Push failed" && exit 1)
format: check-runner
@echo "$(T)✨ Formatting with Ruff...$(R)"
$(RUNNER_RUN) ruff format .
$(RUNNER_RUN) ruff check --fix .
@echo "✅ Code cleaned."
# CI/CD helpers
_ci-dump-config:
@echo "image_name=$(IMAGE_NAME)"
@echo "python_version=$(PYTHON_VERSION)"
@echo "python_version_short=$(PYTHON_VERSION_SHORT)"
@echo "runner=$(RUNNER)"
@echo "service_name=$(SERVICE_NAME)"
help:
@echo "$(B)Available commands:$(R)"
@echo ""
@echo "$(G)Setup:$(R)"
@echo " $(T)check-docker $(R) Verify Docker is installed and running."
@echo " $(T)check-runner $(R) Verify package manager ($(RUNNER))."
@echo " $(T)init-dotenv $(R) Create .env from .env.example with generated secrets."
@echo " $(T)install $(R) Install ALL dependencies (Prod + Dev)."
@echo " $(T)install-hooks $(R) Install git pre-commit hooks."
@echo ""
@echo "$(G)Docker:$(R)"
@echo " $(T)build $(R) Build the docker image (production)."
@echo " $(T)build-test $(R) Build the docker image (with dev deps for testing)."
@echo " $(T)down $(R) Stop and remove containers."
@echo " $(T)logs $(R) Follow logs."
@echo " $(T)prune $(R) Clean Docker system."
@echo " $(T)ps $(R) Show container status."
@echo " $(T)restart $(R) Restart all containers."
@echo " $(T)shell $(R) Open shell in container."
@echo " $(T)up $(R) Start the agent."
@echo ""
@echo "$(G)Development:$(R)"
@echo " $(T)add ... $(R) Add dependency (use --group dev or --dev if needed)."
@echo " $(T)clean $(R) Clean caches."
@echo " $(T)coverage $(R) Run tests with coverage."
@echo " $(T)format $(R) Format code (Ruff)."
@echo " $(T)lint $(R) Lint code without fixing."
@echo " $(T)test ... $(R) Run tests (local with $(RUNNER))."
@echo " $(T)update $(R) Update dependencies."
@echo ""
@echo "$(G)Versioning:$(R)"
@echo " $(T)major/minor/patch $(R) Bump version."
init-dotenv:
@echo "$(T)🔑 Initializing .env file...$(R)"
@if [ -f .env ]; then \
echo "$(R)⚠️ .env already exists. Skipping.$(R)"; \
exit 0; \
fi
@if [ ! -f .env.example ]; then \
echo "$(R)❌ .env.example not found$(R)"; \
exit 1; \
fi
@if ! command -v openssl >/dev/null 2>&1; then \
echo "$(R)❌ openssl not found. Please install it first.$(R)"; \
exit 1; \
fi
@echo "$(T) → Copying .env.example...$(R)"
@cp .env.example .env
@echo "$(T) → Generating secrets...$(R)"
@sed -i.bak "s|JWT_SECRET=.*|JWT_SECRET=$$(openssl rand -base64 32)|" .env
@sed -i.bak "s|JWT_REFRESH_SECRET=.*|JWT_REFRESH_SECRET=$$(openssl rand -base64 32)|" .env
@sed -i.bak "s|CREDS_KEY=.*|CREDS_KEY=$$(openssl rand -hex 16)|" .env
@sed -i.bak "s|CREDS_IV=.*|CREDS_IV=$$(openssl rand -hex 8)|" .env
@sed -i.bak "s|MEILI_MASTER_KEY=.*|MEILI_MASTER_KEY=$$(openssl rand -base64 32)|" .env
@sed -i.bak "s|AGENT_BRAIN_API_KEY=.*|AGENT_BRAIN_API_KEY=$$(openssl rand -base64 24)|" .env
@rm -f .env.bak
@echo "$(G)✅ .env created with generated secrets!$(R)"
@echo "$(T)⚠️ Don't forget to add your API keys:$(R)"
@echo " - OPENAI_API_KEY"
@echo " - DEEPSEEK_API_KEY"
@echo " - TMDB_API_KEY (optional)"
install: check-runner
@echo "$(T)📦 Installing FULL environment ($(RUNNER))...$(R)"
$(RUNNER_INSTALL)
@echo "✅ Environment ready (Prod + Dev)."
install-hooks: check-runner
@echo "$(T)🔧 Installing hooks...$(R)"
$(RUNNER_HOOKS)
@echo "✅ Hooks ready."
lint: check-runner
@echo "$(T)🔍 Linting code...$(R)"
$(RUNNER_RUN) ruff check .
logs: check-docker
@echo "$(T)📋 Following logs...$(R)"
$(COMPOSE_CMD) logs -f
major: _check_branch
@echo "$(T)💥 Bumping major...$(R)"
SKIP=all $(BUMP_CMD) major
minor: _check_branch
@echo "$(T)✨ Bumping minor...$(R)"
SKIP=all $(BUMP_CMD) minor
patch: _check_branch
@echo "$(T)🚀 Bumping patch...$(R)"
SKIP=all $(BUMP_CMD) patch
prune: check-docker
@echo "$(T)🗑️ Pruning Docker resources...$(R)"
docker system prune -af
@echo "✅ Docker cleaned."
ps: check-docker
@echo "$(T)📋 Container status:$(R)"
@$(COMPOSE_CMD) ps
restart: check-docker
@echo "$(T)🔄 Restarting containers...$(R)"
$(COMPOSE_CMD) restart
@echo "✅ Containers restarted."
run: check-runner
$(RUNNER_RUN) $(ARGS)
shell: check-docker
@echo "$(T)🐚 Opening shell in $(SERVICE_NAME)...$(R)"
$(COMPOSE_CMD) exec $(SERVICE_NAME) /bin/sh
test: check-runner
@echo "$(T)🧪 Running tests...$(R)"
$(RUNNER_RUN) pytest $(ARGS)
up: check-docker
@echo "$(T)🚀 Starting Agent Media...$(R)"
$(COMPOSE_CMD) up -d
@echo "✅ System is up."
update: check-runner
@echo "$(T)🔄 Updating dependencies...$(R)"
$(RUNNER_UPDATE)
@echo "✅ All packages up to date."
_check_branch:
@curr=$$(git rev-parse --abbrev-ref HEAD); \
if [ "$$curr" != "main" ]; then \
echo "❌ Error: not on the main branch"; exit 1; \
fi
_ci-image-name:
@echo "IMAGE_NAME=$(IMAGE_NAME)"
_ci-run-tests: build-test
@echo "$(T)🧪 Running tests in Docker...$(R)"
_ci-run-tests:build-test
@echo "Running tests in Docker..."
docker run --rm \
-e DEEPSEEK_API_KEY \
-e TMDB_API_KEY \
-e QBITTORRENT_URL \
$(IMAGE_NAME):test pytest
@echo " Tests passed."
@echo " Tests passed."
_check-main:
@test "$$(git rev-parse --abbrev-ref HEAD)" = "main" \
|| (echo "✗ ERROR: Not on main branch" && exit 1)
# --- Help ---
help:
@echo "Cleverly Crafted Unawareness - Management Commands"
@echo ""
@echo "Usage: make [target] [p=profile1,profile2]"
@echo ""
@echo "Setup:"
@echo " bootstrap Generate .env and .env.make from schema"
@echo " validate Validate settings against schema"
@echo ""
@echo "Docker:"
@echo " up Start containers (default profile: full)"
@echo " Example: make up p=rag,meili"
@echo " down Stop all containers"
@echo " restart Restart containers (supports p=...)"
@echo " logs Follow logs (supports p=...)"
@echo " ps Status of containers"
@echo " shell Open bash in the core container"
@echo " build Build the production Docker image"
@echo ""
@echo "Dev & Quality:"
@echo " install Install dependencies via $(RUNNER)"
@echo " test Run pytest suite"
@echo " coverage Run tests and generate HTML report"
@echo " lint/format Quality and style checks"
@echo ""
@echo "Release:"
@echo " major|minor|patch Bump version and push tags (main branch only)"

595
README.md Normal file
View File

@@ -0,0 +1,595 @@
# Alfred Media Organizer 🎬
An AI-powered agent for managing your local media library with natural language. Search, download, and organize movies and TV shows effortlessly through a conversational interface.
[![Python 3.14](https://img.shields.io/badge/python-3.14-blue.svg)](https://www.python.org/downloads/)
[![Poetry](https://img.shields.io/badge/dependency%20manager-poetry-blue)](https://python-poetry.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Code style: ruff](https://img.shields.io/badge/code%20style-ruff-000000.svg)](https://github.com/astral-sh/ruff)
## ✨ Features
- 🤖 **Natural Language Interface** — Talk to your media library in plain language
- 🔍 **Smart Search** — Find movies and TV shows via TMDB with rich metadata
- 📥 **Torrent Integration** — Search and download via qBittorrent
- 🧠 **Contextual Memory** — Remembers your preferences and conversation history
- 📁 **Auto-Organization** — Keeps your media library tidy and well-structured
- 🌐 **OpenAI-Compatible API** — Works with any OpenAI-compatible client
- 🖥️ **LibreChat Frontend** — Beautiful web UI included out of the box
- 🔒 **Secure by Default** — Auto-generated secrets and encrypted credentials
## 🏗️ Architecture
Built with **Domain-Driven Design (DDD)** principles for clean separation of concerns:
```
alfred/
├── agent/ # AI agent orchestration
│ ├── llm/ # LLM clients (Ollama, DeepSeek)
│ └── tools/ # Tool implementations
├── application/ # Use cases & DTOs
│ ├── movies/ # Movie search use cases
│ ├── torrents/ # Torrent management
│ └── filesystem/ # File operations
├── domain/ # Business logic & entities
│ ├── movies/ # Movie entities
│ ├── tv_shows/ # TV show entities
│ └── subtitles/ # Subtitle entities
└── infrastructure/ # External services & persistence
├── api/ # External API clients (TMDB, qBittorrent)
├── filesystem/ # File system operations
└── persistence/ # Memory & repositories
```
See [docs/architecture_diagram.md](docs/architecture_diagram.md) for detailed architectural diagrams.
## 🚀 Quick Start
### Prerequisites
- **Python 3.14+** (required)
- **Poetry** (dependency manager)
- **Docker & Docker Compose** (recommended for full stack)
- **API Keys:**
- TMDB API key ([get one here](https://www.themoviedb.org/settings/api))
- Optional: DeepSeek, OpenAI, Anthropic, or other LLM provider keys
### Installation
```bash
# Clone the repository
git clone https://github.com/francwa/alfred_media_organizer.git
cd alfred_media_organizer
# Install dependencies
make install
# Bootstrap environment (generates .env with secure secrets)
make bootstrap
# Edit .env with your API keys
nano .env
```
### Running with Docker (Recommended)
```bash
# Start all services (LibreChat + Alfred + MongoDB + Ollama)
make up
# Or start with specific profiles
make up p=rag,meili # Include RAG and Meilisearch
make up p=qbittorrent # Include qBittorrent
make up p=full # Everything
# View logs
make logs
# Stop all services
make down
```
The web interface will be available at **http://localhost:3080**
### Running Locally (Development)
```bash
# Install dependencies
poetry install
# Start the API server
poetry run uvicorn alfred.app:app --reload --port 8000
```
## ⚙️ Configuration
### Environment Bootstrap
Alfred uses a smart bootstrap system that:
1. **Generates secure secrets** automatically (JWT tokens, database passwords, encryption keys)
2. **Syncs build variables** from `pyproject.toml` (versions, image names)
3. **Preserves existing secrets** when re-running (never overwrites your API keys)
4. **Computes database URIs** automatically from individual components
```bash
# First time setup
make bootstrap
# Re-run after updating pyproject.toml (secrets are preserved)
make bootstrap
```
### Configuration File (.env)
The `.env` file is generated from `.env.example` with secure defaults:
```bash
# --- CORE SETTINGS ---
HOST=0.0.0.0
PORT=3080
MAX_HISTORY_MESSAGES=10
MAX_TOOL_ITERATIONS=10
# --- LLM CONFIGURATION ---
# Providers: 'local' (Ollama), 'deepseek', 'openai', 'anthropic', 'google'
DEFAULT_LLM_PROVIDER=local
# Local LLM (Ollama - included in Docker stack)
OLLAMA_BASE_URL=http://ollama:11434
OLLAMA_MODEL=llama3.3:latest
LLM_TEMPERATURE=0.2
# --- API KEYS (fill only what you need) ---
TMDB_API_KEY=your-tmdb-key-here # Required for movie search
DEEPSEEK_API_KEY= # Optional
OPENAI_API_KEY= # Optional
ANTHROPIC_API_KEY= # Optional
# --- SECURITY (auto-generated, don't modify) ---
JWT_SECRET=<auto-generated>
JWT_REFRESH_SECRET=<auto-generated>
CREDS_KEY=<auto-generated>
CREDS_IV=<auto-generated>
# --- DATABASES (auto-generated passwords) ---
MONGO_PASSWORD=<auto-generated>
POSTGRES_PASSWORD=<auto-generated>
```
### Security Keys
Security keys are defined in `pyproject.toml` and generated automatically:
```toml
[tool.alfred.security]
jwt_secret = "32:b64" # 32 bytes, base64 URL-safe
jwt_refresh_secret = "32:b64"
creds_key = "32:hex" # 32 bytes, hexadecimal (AES-256)
creds_iv = "16:hex" # 16 bytes, hexadecimal (AES IV)
mongo_password = "16:hex"
postgres_password = "16:hex"
```
**Formats:**
- `b64` — Base64 URL-safe (for JWT tokens)
- `hex` — Hexadecimal (for encryption keys, passwords)
## 🐳 Docker Services
### Service Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ alfred-net (bridge) │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ LibreChat │───▶│ Alfred │───▶│ MongoDB │ │
│ │ :3080 │ │ (core) │ │ :27017 │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │ │ │
│ │ ▼ │
│ │ ┌──────────────┐ │
│ │ │ Ollama │ │
│ │ │ (local) │ │
│ │ └──────────────┘ │
│ │ │
│ ┌──────┴───────────────────────────────────────────────┐ │
│ │ Optional Services (profiles) │ │
│ ├──────────────┬──────────────┬──────────────┬─────────┤ │
│ │ Meilisearch │ RAG API │ VectorDB │qBittor- │ │
│ │ :7700 │ :8000 │ :5432 │ rent │ │
│ │ [meili] │ [rag] │ [rag] │[qbit..] │ │
│ └──────────────┴──────────────┴──────────────┴─────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
### Docker Profiles
| Profile | Services | Use Case |
|---------|----------|----------|
| (default) | LibreChat, Alfred, MongoDB, Ollama | Basic setup |
| `meili` | + Meilisearch | Fast search |
| `rag` | + RAG API, VectorDB | Document retrieval |
| `qbittorrent` | + qBittorrent | Torrent downloads |
| `full` | All services | Complete setup |
```bash
# Start with specific profiles
make up p=rag,meili
make up p=full
```
### Docker Commands
```bash
make up # Start containers (default profile)
make up p=full # Start with all services
make down # Stop all containers
make restart # Restart containers
make logs # Follow logs
make ps # Show container status
make shell # Open bash in Alfred container
make build # Build production image
make build-test # Build test image
```
## 🛠️ Available Tools
The agent has access to these tools for interacting with your media library:
| Tool | Description |
|------|-------------|
| `find_media_imdb_id` | Search for movies/TV shows on TMDB by title |
| `find_torrent` | Search for torrents across multiple indexers |
| `get_torrent_by_index` | Get detailed info about a specific torrent result |
| `add_torrent_by_index` | Download a torrent by its index in search results |
| `add_torrent_to_qbittorrent` | Add a torrent via magnet link directly |
| `set_path_for_folder` | Configure folder paths for media organization |
| `list_folder` | List contents of a folder |
| `set_language` | Set preferred language for searches |
## 💬 Usage Examples
### Via Web Interface (LibreChat)
Navigate to **http://localhost:3080** and start chatting:
```
You: Find Inception in 1080p
Alfred: I found 3 torrents for Inception (2010):
1. Inception.2010.1080p.BluRay.x264 (150 seeders) - 2.1 GB
2. Inception.2010.1080p.WEB-DL.x265 (80 seeders) - 1.8 GB
3. Inception.2010.1080p.REMUX (45 seeders) - 25 GB
You: Download the first one
Alfred: ✓ Added to qBittorrent! Download started.
Saving to: /downloads/Movies/Inception (2010)/
You: What's downloading right now?
Alfred: You have 1 active download:
- Inception.2010.1080p.BluRay.x264 (45% complete, ETA: 12 min)
```
### Via API
```bash
# Health check
curl http://localhost:8000/health
# Chat with the agent (OpenAI-compatible)
curl -X POST http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "alfred",
"messages": [
{"role": "user", "content": "Find The Matrix 4K"}
]
}'
# List available models
curl http://localhost:8000/v1/models
# View memory state (debug)
curl http://localhost:8000/memory/state
# Clear session memory
curl -X POST http://localhost:8000/memory/clear-session
```
### Via OpenWebUI or Other Clients
Alfred is compatible with any OpenAI-compatible client:
1. Add as OpenAI-compatible endpoint: `http://localhost:8000/v1`
2. Model name: `alfred`
3. No API key required (or use any placeholder)
## 🧠 Memory System
Alfred uses a three-tier memory system for context management:
### Long-Term Memory (LTM)
- **Persistent** — Saved to JSON files
- **Contents:** Configuration, user preferences, media library state
- **Survives:** Application restarts
### Short-Term Memory (STM)
- **Session-based** — Stored in RAM
- **Contents:** Conversation history, current workflow state
- **Cleared:** On session end or restart
### Episodic Memory
- **Transient** — Stored in RAM
- **Contents:** Search results, active downloads, recent errors
- **Cleared:** Frequently, after task completion
## 🧪 Development
### Project Setup
```bash
# Install all dependencies (including dev)
poetry install
# Install pre-commit hooks
make install-hooks
# Run the development server
poetry run uvicorn alfred.app:app --reload
```
### Running Tests
```bash
# Run all tests (parallel execution)
make test
# Run with coverage report
make coverage
# Run specific test file
poetry run pytest tests/test_agent.py -v
# Run specific test
poetry run pytest tests/test_config_loader.py::TestBootstrapEnv -v
```
### Code Quality
```bash
# Lint and auto-fix
make lint
# Format code
make format
# Clean build artifacts
make clean
```
### Adding a New Tool
1. **Create the tool function** in `alfred/agent/tools/`:
```python
# alfred/agent/tools/api.py
def my_new_tool(param: str) -> dict[str, Any]:
"""
Short description of what this tool does.
This will be shown to the LLM to help it decide when to use this tool.
"""
memory = get_memory()
# Your implementation here
result = do_something(param)
return {
"status": "success",
"data": result
}
```
2. **Register in the registry** (`alfred/agent/registry.py`):
```python
tool_functions = [
# ... existing tools ...
api_tools.my_new_tool, # Add your tool here
]
```
The tool will be automatically registered with its parameters extracted from the function signature.
### Version Management
```bash
# Bump version (must be on main branch)
make patch # 0.1.7 -> 0.1.8
make minor # 0.1.7 -> 0.2.0
make major # 0.1.7 -> 1.0.0
```
## 📚 API Reference
### Endpoints
#### `GET /health`
Health check endpoint.
```json
{
"status": "healthy",
"version": "0.1.7"
}
```
#### `GET /v1/models`
List available models (OpenAI-compatible).
```json
{
"object": "list",
"data": [
{
"id": "alfred",
"object": "model",
"owned_by": "alfred"
}
]
}
```
#### `POST /v1/chat/completions`
Chat with the agent (OpenAI-compatible).
**Request:**
```json
{
"model": "alfred",
"messages": [
{"role": "user", "content": "Find Inception"}
],
"stream": false
}
```
**Response:**
```json
{
"id": "chatcmpl-xxx",
"object": "chat.completion",
"created": 1234567890,
"model": "alfred",
"choices": [{
"index": 0,
"message": {
"role": "assistant",
"content": "I found Inception (2010)..."
},
"finish_reason": "stop"
}]
}
```
#### `GET /memory/state`
View full memory state (debug endpoint).
#### `POST /memory/clear-session`
Clear session memories (STM + Episodic).
## 🔧 Troubleshooting
### Agent doesn't respond
1. Check API keys in `.env`
2. Verify LLM provider is running:
```bash
# For Ollama
docker logs alfred-ollama
# Check if model is pulled
docker exec alfred-ollama ollama list
```
3. Check Alfred logs: `docker logs alfred-core`
### qBittorrent connection failed
1. Verify qBittorrent is running: `docker ps | grep qbittorrent`
2. Check Web UI is enabled in qBittorrent settings
3. Verify credentials in `.env`:
```bash
QBITTORRENT_URL=http://qbittorrent:16140
QBITTORRENT_USERNAME=admin
QBITTORRENT_PASSWORD=<check-your-env>
```
### Database connection issues
1. Check MongoDB is healthy: `docker logs alfred-mongodb`
2. Verify credentials match in `.env`
3. Try restarting: `make restart`
### Memory not persisting
1. Check `data/` directory exists and is writable
2. Verify volume mounts in `docker-compose.yaml`
3. Check file permissions: `ls -la data/`
### Bootstrap fails
1. Ensure `.env.example` exists
2. Check `pyproject.toml` has required sections:
```toml
[tool.alfred.settings]
[tool.alfred.security]
```
3. Run manually: `python scripts/bootstrap.py`
### Tests failing
1. Update dependencies: `poetry install`
2. Check Python version: `python --version` (needs 3.14+)
3. Run specific failing test with verbose output:
```bash
poetry run pytest tests/test_failing.py -v --tb=long
```
## 🤝 Contributing
Contributions are welcome! Please follow these steps:
1. **Fork** the repository
2. **Create** a feature branch: `git checkout -b feature/my-feature`
3. **Make** your changes
4. **Run** tests: `make test`
5. **Run** linting: `make lint && make format`
6. **Commit**: `git commit -m "feat: add my feature"`
7. **Push**: `git push origin feature/my-feature`
8. **Create** a Pull Request
### Commit Convention
We use [Conventional Commits](https://www.conventionalcommits.org/):
- `feat:` New feature
- `fix:` Bug fix
- `docs:` Documentation
- `refactor:` Code refactoring
- `test:` Adding tests
- `chore:` Maintenance
## 📖 Documentation
- [Architecture Diagram](docs/architecture_diagram.md) — System architecture overview
- [Class Diagram](docs/class_diagram.md) — Class structure and relationships
- [Component Diagram](docs/component_diagram.md) — Component interactions
- [Sequence Diagram](docs/sequence_diagram.md) — Sequence flows
- [Flowchart](docs/flowchart.md) — System flowcharts
## 📄 License
MIT License — see [LICENSE](LICENSE) file for details.
## 🙏 Acknowledgments
- [LibreChat](https://github.com/danny-avila/LibreChat) — Beautiful chat interface
- [Ollama](https://ollama.ai/) — Local LLM runtime
- [DeepSeek](https://www.deepseek.com/) — LLM provider
- [TMDB](https://www.themoviedb.org/) — Movie database
- [qBittorrent](https://www.qbittorrent.org/) — Torrent client
- [FastAPI](https://fastapi.tiangolo.com/) — Web framework
- [Pydantic](https://docs.pydantic.dev/) — Data validation
## 📬 Support
- 📧 Email: francois.hodiaumont@gmail.com
- 🐛 Issues: [GitHub Issues](https://github.com/francwa/alfred_media_organizer/issues)
- 💬 Discussions: [GitHub Discussions](https://github.com/francwa/alfred_media_organizer/discussions)
---
<p align="center">Made with ❤️ by <a href="https://github.com/francwa">Francwa</a></p>

0
alfred/__init__.py Normal file
View File

View File

@@ -1,6 +1,7 @@
"""Agent module for media library management."""
from alfred.settings import settings
from .agent import Agent
from .config import settings
__all__ = ["Agent", "settings"]

View File

@@ -5,9 +5,9 @@ import logging
from collections.abc import AsyncGenerator
from typing import Any
from infrastructure.persistence import get_memory
from alfred.infrastructure.persistence import get_memory
from alfred.settings import settings
from .config import settings
from .prompts import PromptBuilder
from .registry import Tool, make_tools
@@ -21,17 +21,20 @@ class Agent:
Uses OpenAI-compatible tool calling API.
"""
def __init__(self, llm, max_tool_iterations: int = 5):
def __init__(self, settings, llm, max_tool_iterations: int = 5):
"""
Initialize the agent.
Args:
settings: Application settings instance
llm: LLM client with complete() method
max_tool_iterations: Maximum number of tool execution iterations
"""
self.settings = settings
self.llm = llm
self.tools: dict[str, Tool] = make_tools()
self.tools: dict[str, Tool] = make_tools(settings)
self.prompt_builder = PromptBuilder(self.tools)
self.settings = settings
self.max_tool_iterations = max_tool_iterations
def step(self, user_input: str) -> str:
@@ -78,7 +81,7 @@ class Agent:
tools_spec = self.prompt_builder.build_tools_spec()
# Tool execution loop
for _iteration in range(self.max_tool_iterations):
for _iteration in range(self.settings.max_tool_iterations):
# Call LLM with tools
llm_result = self.llm.complete(messages, tools=tools_spec)
@@ -230,7 +233,7 @@ class Agent:
tools_spec = self.prompt_builder.build_tools_spec()
# Tool execution loop
for _iteration in range(self.max_tool_iterations):
for _iteration in range(self.settings.max_tool_iterations):
# Call LLM with tools
llm_result = self.llm.complete(messages, tools=tools_spec)

View File

@@ -6,7 +6,8 @@ from typing import Any
import requests
from requests.exceptions import HTTPError, RequestException, Timeout
from ..config import settings
from alfred.settings import Settings, settings
from .exceptions import LLMAPIError, LLMConfigurationError
logger = logging.getLogger(__name__)
@@ -21,6 +22,7 @@ class DeepSeekClient:
base_url: str | None = None,
model: str | None = None,
timeout: int | None = None,
settings: Settings | None = None,
):
"""
Initialize DeepSeek client.
@@ -34,10 +36,10 @@ class DeepSeekClient:
Raises:
LLMConfigurationError: If API key is missing
"""
self.api_key = api_key or settings.deepseek_api_key
self.base_url = base_url or settings.deepseek_base_url
self.model = model or settings.model
self.timeout = timeout or settings.request_timeout
self.api_key = api_key or self.settings.deepseek_api_key
self.base_url = base_url or self.settings.deepseek_base_url
self.model = model or self.settings.deepseek_model
self.timeout = timeout or self.settings.request_timeout
if not self.api_key:
raise LLMConfigurationError(
@@ -94,7 +96,7 @@ class DeepSeekClient:
payload = {
"model": self.model,
"messages": messages,
"temperature": settings.temperature,
"temperature": settings.llm_temperature,
}
# Add tools if provided

View File

@@ -1,13 +1,13 @@
"""Ollama LLM client with robust error handling."""
import logging
import os
from typing import Any
import requests
from requests.exceptions import HTTPError, RequestException, Timeout
from ..config import settings
from alfred.settings import Settings
from .exceptions import LLMAPIError, LLMConfigurationError
logger = logging.getLogger(__name__)
@@ -32,6 +32,7 @@ class OllamaClient:
model: str | None = None,
timeout: int | None = None,
temperature: float | None = None,
settings: Settings | None = None,
):
"""
Initialize Ollama client.
@@ -45,13 +46,11 @@ class OllamaClient:
Raises:
LLMConfigurationError: If configuration is invalid
"""
self.base_url = base_url or os.getenv(
"OLLAMA_BASE_URL", "http://localhost:11434"
)
self.model = model or os.getenv("OLLAMA_MODEL", "llama3.2")
self.base_url = base_url or settings.ollama_base_url
self.model = model or settings.ollama_model
self.timeout = timeout or settings.request_timeout
self.temperature = (
temperature if temperature is not None else settings.temperature
temperature if temperature is not None else settings.llm_temperature
)
if not self.base_url:

View File

@@ -3,7 +3,7 @@
import json
from typing import Any
from infrastructure.persistence import get_memory
from alfred.infrastructure.persistence import get_memory
from .registry import Tool
@@ -52,7 +52,7 @@ class PromptBuilder:
# Show first 5 results
for i, result in enumerate(result_list[:5]):
name = result.get("name", "Unknown")
lines.append(f" {i+1}. {name}")
lines.append(f" {i + 1}. {name}")
if len(result_list) > 5:
lines.append(f" ... and {len(result_list) - 5} more")

View File

@@ -78,10 +78,13 @@ def _create_tool_from_function(func: Callable) -> Tool:
)
def make_tools() -> dict[str, Tool]:
def make_tools(settings) -> dict[str, Tool]:
"""
Create and register all available tools.
Args:
settings: Application settings instance
Returns:
Dictionary mapping tool names to Tool objects
"""

View File

@@ -3,12 +3,12 @@
import logging
from typing import Any
from application.movies import SearchMovieUseCase
from application.torrents import AddTorrentUseCase, SearchTorrentsUseCase
from infrastructure.api.knaben import knaben_client
from infrastructure.api.qbittorrent import qbittorrent_client
from infrastructure.api.tmdb import tmdb_client
from infrastructure.persistence import get_memory
from alfred.application.movies import SearchMovieUseCase
from alfred.application.torrents import AddTorrentUseCase, SearchTorrentsUseCase
from alfred.infrastructure.api.knaben import knaben_client
from alfred.infrastructure.api.qbittorrent import qbittorrent_client
from alfred.infrastructure.api.tmdb import tmdb_client
from alfred.infrastructure.persistence import get_memory
logger = logging.getLogger(__name__)

View File

@@ -2,8 +2,8 @@
from typing import Any
from application.filesystem import ListFolderUseCase, SetFolderPathUseCase
from infrastructure.filesystem import FileManager
from alfred.application.filesystem import ListFolderUseCase, SetFolderPathUseCase
from alfred.infrastructure.filesystem import FileManager
def set_path_for_folder(folder_name: str, path_value: str) -> dict[str, Any]:

View File

@@ -3,7 +3,7 @@
import logging
from typing import Any
from infrastructure.persistence import get_memory
from alfred.infrastructure.persistence import get_memory
logger = logging.getLogger(__name__)

View File

@@ -2,22 +2,21 @@
import json
import logging
import os
import time
import uuid
from pathlib import Path
from typing import Any
from fastapi import FastAPI, HTTPException
from fastapi.responses import JSONResponse, StreamingResponse
from fastapi.staticfiles import StaticFiles
from pydantic import BaseModel, Field, validator
from agent.agent import Agent
from agent.config import settings
from agent.llm.deepseek import DeepSeekClient
from agent.llm.exceptions import LLMAPIError, LLMConfigurationError
from agent.llm.ollama import OllamaClient
from infrastructure.persistence import get_memory, init_memory
from alfred.agent.agent import Agent
from alfred.agent.llm.deepseek import DeepSeekClient
from alfred.agent.llm.exceptions import LLMAPIError, LLMConfigurationError
from alfred.agent.llm.ollama import OllamaClient
from alfred.infrastructure.persistence import get_memory, init_memory
from alfred.settings import settings
logging.basicConfig(
level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
@@ -30,38 +29,33 @@ app = FastAPI(
version="0.2.0",
)
# TODO: Make a variable
manifests = "manifests"
# Sécurité : on vérifie que le dossier existe pour ne pas faire planter l'app au démarrage
if os.path.exists(manifests):
app.mount("/manifests", StaticFiles(directory=manifests), name="manifests")
else:
print(
f"⚠️ ATTENTION : Le dossier '{manifests}' est introuvable. Le plugin ne marchera pas."
)
# Initialize memory context at startup
# Use /data/memory in Docker, fallback to memory_data for local dev
storage_dir = os.getenv("MEMORY_STORAGE_DIR", "memory_data")
init_memory(storage_dir=storage_dir)
logger.info(f"Memory context initialized (storage: {storage_dir})")
memory_path = Path(settings.data_storage) / "memory"
init_memory(storage_dir=str(memory_path))
logger.info(f"Memory context initialized (path: {memory_path})")
# Initialize LLM based on environment variable
llm_provider = os.getenv("LLM_PROVIDER", "deepseek").lower()
llm_provider = settings.default_llm_provider.lower()
try:
if llm_provider == "ollama":
logger.info("Using Ollama LLM")
llm = OllamaClient()
else:
if llm_provider == "local":
logger.info("Using local Ollama LLM")
llm = OllamaClient(settings=settings)
elif llm_provider == "deepseek":
logger.info("Using DeepSeek LLM")
llm = DeepSeekClient()
elif llm_provider == "claude":
raise ValueError(f"LLM provider not fully implemented: {llm_provider}")
else:
raise ValueError(f"Unknown LLM provider: {llm_provider}")
except LLMConfigurationError as e:
logger.error(f"Failed to initialize LLM: {e}")
raise
# Initialize agent
agent = Agent(llm=llm, max_tool_iterations=settings.max_tool_iterations)
agent = Agent(
settings=settings, llm=llm, max_tool_iterations=settings.max_tool_iterations
)
logger.info("Agent Media API initialized")
@@ -116,7 +110,7 @@ def extract_last_user_content(messages: list[dict[str, Any]]) -> str:
@app.get("/health")
async def health_check():
"""Health check endpoint."""
return {"status": "healthy", "version": "0.2.0"}
return {"status": "healthy", "version": f"v{settings.alfred_version}"}
@app.get("/v1/models")

View File

@@ -2,7 +2,7 @@
import logging
from infrastructure.filesystem import FileManager
from alfred.infrastructure.filesystem import FileManager
from .dto import ListFolderResponse

View File

@@ -2,7 +2,7 @@
import logging
from infrastructure.filesystem import FileManager
from alfred.infrastructure.filesystem import FileManager
from .dto import SetFolderPathResponse

View File

@@ -2,7 +2,7 @@
import logging
from infrastructure.api.tmdb import (
from alfred.infrastructure.api.tmdb import (
TMDBAPIError,
TMDBClient,
TMDBConfigurationError,

View File

@@ -2,7 +2,7 @@
import logging
from infrastructure.api.qbittorrent import (
from alfred.infrastructure.api.qbittorrent import (
QBittorrentAPIError,
QBittorrentAuthError,
QBittorrentClient,

View File

@@ -2,7 +2,11 @@
import logging
from infrastructure.api.knaben import KnabenAPIError, KnabenClient, KnabenNotFoundError
from alfred.infrastructure.api.knaben import (
KnabenAPIError,
KnabenClient,
KnabenNotFoundError,
)
from .dto import SearchTorrentsResponse

View File

@@ -6,7 +6,7 @@ from typing import Any
import requests
from requests.exceptions import HTTPError, RequestException, Timeout
from agent.config import Settings, settings
from alfred.settings import Settings, settings
from .dto import TorrentResult
from .exceptions import KnabenAPIError, KnabenNotFoundError

View File

@@ -6,7 +6,7 @@ from typing import Any
import requests
from requests.exceptions import HTTPError, RequestException, Timeout
from agent.config import Settings, settings
from alfred.settings import Settings, settings
from .dto import TorrentInfo
from .exceptions import QBittorrentAPIError, QBittorrentAuthError

View File

@@ -6,7 +6,7 @@ from typing import Any
import requests
from requests.exceptions import HTTPError, RequestException, Timeout
from agent.config import Settings, settings
from alfred.settings import Settings, settings
from .dto import MediaResult
from .exceptions import (

View File

@@ -7,7 +7,7 @@ from enum import Enum
from pathlib import Path
from typing import Any
from infrastructure.persistence import get_memory
from alfred.infrastructure.persistence import get_memory
from .exceptions import PathTraversalError

View File

@@ -3,9 +3,9 @@
import logging
from pathlib import Path
from domain.movies.entities import Movie
from domain.tv_shows.entities import Episode, Season, TVShow
from domain.tv_shows.value_objects import SeasonNumber
from alfred.domain.movies.entities import Movie
from alfred.domain.tv_shows.entities import Episode, Season, TVShow
from alfred.domain.tv_shows.value_objects import SeasonNumber
logger = logging.getLogger(__name__)

View File

@@ -6,7 +6,7 @@ without passing it explicitly through all function calls.
Usage:
# At application startup
from infrastructure.persistence import init_memory, get_memory
from alfred.infrastructure.persistence import init_memory, get_memory
init_memory("memory_data")

View File

@@ -4,11 +4,11 @@ import logging
from datetime import datetime
from typing import Any
from domain.movies.entities import Movie
from domain.movies.repositories import MovieRepository
from domain.movies.value_objects import MovieTitle, Quality, ReleaseYear
from domain.shared.value_objects import FilePath, FileSize, ImdbId
from infrastructure.persistence import get_memory
from alfred.domain.movies.entities import Movie
from alfred.domain.movies.repositories import MovieRepository
from alfred.domain.movies.value_objects import MovieTitle, Quality, ReleaseYear
from alfred.domain.shared.value_objects import FilePath, FileSize, ImdbId
from alfred.infrastructure.persistence import get_memory
logger = logging.getLogger(__name__)

View File

@@ -3,11 +3,11 @@
import logging
from typing import Any
from domain.shared.value_objects import FilePath, ImdbId
from domain.subtitles.entities import Subtitle
from domain.subtitles.repositories import SubtitleRepository
from domain.subtitles.value_objects import Language, SubtitleFormat, TimingOffset
from infrastructure.persistence import get_memory
from alfred.domain.shared.value_objects import FilePath, ImdbId
from alfred.domain.subtitles.entities import Subtitle
from alfred.domain.subtitles.repositories import SubtitleRepository
from alfred.domain.subtitles.value_objects import Language, SubtitleFormat, TimingOffset
from alfred.infrastructure.persistence import get_memory
logger = logging.getLogger(__name__)

View File

@@ -4,11 +4,11 @@ import logging
from datetime import datetime
from typing import Any
from domain.shared.value_objects import ImdbId
from domain.tv_shows.entities import TVShow
from domain.tv_shows.repositories import TVShowRepository
from domain.tv_shows.value_objects import ShowStatus
from infrastructure.persistence import get_memory
from alfred.domain.shared.value_objects import ImdbId
from alfred.domain.tv_shows.entities import TVShow
from alfred.domain.tv_shows.repositories import TVShowRepository
from alfred.domain.tv_shows.value_objects import ShowStatus
from alfred.infrastructure.persistence import get_memory
logger = logging.getLogger(__name__)

292
alfred/settings.py Normal file
View File

@@ -0,0 +1,292 @@
"""
Application settings using Pydantic Settings.
Settings are loaded from .env file and validated against the schema
defined in pyproject.toml [tool.alfred.settings_schema].
"""
from pathlib import Path
from pydantic import Field, computed_field, field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
from .settings_schema import SCHEMA
BASE_DIR = Path(__file__).resolve().parent.parent
ENV_FILE_PATH = BASE_DIR / ".env"
class ConfigurationError(Exception):
"""Raised when configuration is invalid."""
pass
def _get_default_from_schema(setting_name: str):
"""Get default value from schema for a setting."""
definition = SCHEMA.get(setting_name.upper())
return definition.default if definition else None
def _get_secret_factory(rule: str):
"""Create a factory function for generating secrets."""
def factory():
from .settings_bootstrap import generate_secret # noqa: PLC0415
return generate_secret(rule)
return factory
class Settings(BaseSettings):
"""
Application settings management.
Initializes configuration with internal defaults from the settings module,
then overrides them with environment variables loaded from a .env file.
"""
model_config = SettingsConfigDict(
env_file=ENV_FILE_PATH,
env_file_encoding="utf-8",
extra="ignore",
case_sensitive=False,
)
# --- BUILD (from TOML) ---
alfred_version: str = Field(
default=_get_default_from_schema("ALFRED_VERSION"), description="Alfred version"
)
python_version: str = Field(
default=_get_default_from_schema("PYTHON_VERSION"), description="Python version"
)
python_version_short: str = Field(
default=_get_default_from_schema("PYTHON_VERSION_SHORT"),
description="Python version (short)",
)
runner: str = Field(
default=_get_default_from_schema("RUNNER"), description="Dependency manager"
)
image_name: str = Field(
default=_get_default_from_schema("IMAGE_NAME"), description="Docker image name"
)
service_name: str = Field(
default=_get_default_from_schema("SERVICE_NAME"),
description="Docker service name",
)
librechat_version: str = Field(
default=_get_default_from_schema("LIBRECHAT_VERSION"),
description="LibreChat version",
)
rag_version: str = Field(
default=_get_default_from_schema("RAG_VERSION"), description="RAG version"
)
# --- APP SETTINGS ---
host: str = Field(
default=_get_default_from_schema("HOST"), description="Server host"
)
port: int = Field(
default=_get_default_from_schema("PORT"), description="Server port"
)
max_history_messages: int = Field(
default=_get_default_from_schema("MAX_HISTORY_MESSAGES"),
description="Maximum conversation history",
)
max_tool_iterations: int = Field(
default=_get_default_from_schema("MAX_TOOL_ITERATIONS"),
description="Maximum tool iterations",
)
request_timeout: int = Field(
default=_get_default_from_schema("REQUEST_TIMEOUT"),
description="Request timeout in seconds",
)
llm_temperature: float = Field(
default=_get_default_from_schema("LLM_TEMPERATURE"),
description="LLM temperature",
)
data_storage_dir: str = Field(
default=_get_default_from_schema("DATA_STORAGE_DIR"),
description="Data storage directory",
alias="DATA_STORAGE_DIR",
)
# Legacy aliases
debug_logging: bool = False
debug_console: bool = False
data_storage: str = Field(default="data", exclude=True) # Deprecated
# --- API KEYS ---
tmdb_api_key: str | None = Field(None, description="TMDB API key")
deepseek_api_key: str | None = Field(None, description="DeepSeek API key")
openai_api_key: str | None = Field(None, description="OpenAI API key")
anthropic_api_key: str | None = Field(None, description="Anthropic API key")
google_api_key: str | None = Field(None, description="Google API key")
kimi_api_key: str | None = Field(None, description="Kimi API key")
# --- SECURITY SECRETS ---
jwt_secret: str = Field(
default_factory=_get_secret_factory("32:b64"), description="JWT signing secret"
)
jwt_refresh_secret: str = Field(
default_factory=_get_secret_factory("32:b64"), description="JWT refresh secret"
)
creds_key: str = Field(
default_factory=_get_secret_factory("32:hex"),
description="Credentials encryption key",
)
creds_iv: str = Field(
default_factory=_get_secret_factory("16:hex"),
description="Credentials encryption IV",
)
meili_master_key: str = Field(
default_factory=_get_secret_factory("32:b64"),
description="Meilisearch master key",
repr=False,
)
# --- DATABASE ---
mongo_host: str = Field(
default=_get_default_from_schema("MONGO_HOST"), description="MongoDB host"
)
mongo_port: int = Field(
default=_get_default_from_schema("MONGO_PORT"), description="MongoDB port"
)
mongo_user: str = Field(
default=_get_default_from_schema("MONGO_USER"), description="MongoDB user"
)
mongo_password: str = Field(
default_factory=_get_secret_factory("16:hex"),
description="MongoDB password",
repr=False,
exclude=True,
)
mongo_db_name: str = Field(
default=_get_default_from_schema("MONGO_DB_NAME"),
description="MongoDB database name",
)
@computed_field(repr=False)
@property
def mongo_uri(self) -> str:
"""MongoDB connection URI."""
return (
f"mongodb://{self.mongo_user}:{self.mongo_password}"
f"@{self.mongo_host}:{self.mongo_port}/{self.mongo_db_name}"
f"?authSource=admin"
)
postgres_host: str = Field(
default=_get_default_from_schema("POSTGRES_HOST"), description="PostgreSQL host"
)
postgres_port: int = Field(
default=_get_default_from_schema("POSTGRES_PORT"), description="PostgreSQL port"
)
postgres_user: str = Field(
default=_get_default_from_schema("POSTGRES_USER"), description="PostgreSQL user"
)
postgres_password: str = Field(
default_factory=_get_secret_factory("16:hex"),
description="PostgreSQL password",
repr=False,
exclude=True,
)
postgres_db_name: str = Field(
default=_get_default_from_schema("POSTGRES_DB_NAME"),
description="PostgreSQL database name",
)
@computed_field(repr=False)
@property
def postgres_uri(self) -> str:
"""PostgreSQL connection URI."""
return (
f"postgresql://{self.postgres_user}:{self.postgres_password}"
f"@{self.postgres_host}:{self.postgres_port}/{self.postgres_db_name}"
)
# --- EXTERNAL SERVICES ---
tmdb_base_url: str = "https://api.themoviedb.org/3"
qbittorrent_url: str = "http://qbittorrent:16140"
qbittorrent_username: str = "admin"
qbittorrent_password: str = Field(
default_factory=_get_secret_factory("16:hex"),
description="qBittorrent password",
)
# --- LLM CONFIG ---
default_llm_provider: str = "local"
ollama_base_url: str = "http://ollama:11434"
ollama_model: str = "llama3.3:latest"
deepseek_base_url: str = "https://api.deepseek.com"
deepseek_model: str = "deepseek-chat"
# --- RAG ENGINE ---
rag_enabled: bool = True
rag_api_url: str = "http://rag_api:8000"
embeddings_provider: str = "ollama"
embeddings_model: str = "nomic-embed-text"
# --- MEILISEARCH ---
meili_enabled: bool = True
meili_no_analytics: bool = True
meili_host: str = "http://meilisearch:7700"
# --- VALIDATORS (from schema) ---
@field_validator("llm_temperature")
@classmethod
def validate_temperature(cls, v: float) -> float:
"""Validate LLM temperature is in valid range."""
if not 0.0 <= v <= 2.0:
raise ConfigurationError(
f"Temperature must be between 0.0 and 2.0, got {v}"
)
return v
@field_validator("max_tool_iterations")
@classmethod
def validate_max_iterations(cls, v: int) -> int:
"""Validate max tool iterations is in valid range."""
if not 1 <= v <= 20:
raise ConfigurationError(
f"max_tool_iterations must be between 1 and 20, got {v}"
)
return v
@field_validator("request_timeout")
@classmethod
def validate_timeout(cls, v: int) -> int:
"""Validate request timeout is in valid range."""
if not 1 <= v <= 300:
raise ConfigurationError(
f"request_timeout must be between 1 and 300 seconds, got {v}"
)
return v
@field_validator("deepseek_base_url", "tmdb_base_url")
@classmethod
def validate_url(cls, v: str, info) -> str:
"""Validate URLs start with http:// or https://."""
if not v.startswith(("http://", "https://")):
raise ConfigurationError(f"Invalid {info.field_name}: must be a valid URL")
return v
# --- HELPER METHODS ---
def is_tmdb_configured(self) -> bool:
"""Check if TMDB API key is configured."""
return bool(self.tmdb_api_key)
def is_deepseek_configured(self) -> bool:
"""Check if DeepSeek API key is configured."""
return bool(self.deepseek_api_key)
def dump_safe(self) -> dict:
"""Dump settings excluding sensitive fields."""
return self.model_dump(exclude_none=False)
# Global settings instance
settings = Settings()

View File

@@ -0,0 +1,417 @@
"""
Settings bootstrap - Generate and validate configuration files.
This module uses the settings schema to generate .env and .env.make files
with proper validation and secret generation.
"""
import re
import secrets
from dataclasses import dataclass
from pathlib import Path
from typing import Any
import tomllib
from .settings_schema import (
SCHEMA,
SettingDefinition,
SettingSource,
SettingsSchema,
SettingType,
validate_value,
)
@dataclass
class ConfigSource:
"""Configuration source paths."""
base_dir: Path
toml_path: Path
env_path: Path
env_example_path: Path
@classmethod
def from_base_dir(cls, base_dir: Path | None = None) -> "ConfigSource":
"""Create ConfigSource from base directory."""
if base_dir is None:
# Don't import settings.py to avoid Pydantic dependency in pre-commit
base_dir = Path(__file__).resolve().parent.parent
return cls(
base_dir=base_dir,
toml_path=base_dir / "pyproject.toml",
env_path=base_dir / ".env",
env_example_path=base_dir / ".env.example",
)
def extract_python_version(version_string: str) -> tuple[str, str]:
"""
Extract Python version from poetry dependency string.
Examples:
"==3.14.2" -> ("3.14.2", "3.14")
"^3.14.2" -> ("3.14.2", "3.14")
"""
clean_version = re.sub(r"^[=^~><]+", "", version_string.strip())
parts = clean_version.split(".")
if len(parts) >= 2:
full_version = clean_version
short_version = f"{parts[0]}.{parts[1]}"
return full_version, short_version
else:
raise ValueError(f"Invalid Python version format: {version_string}")
def generate_secret(rule: str) -> str:
"""
Generate a cryptographically secure secret.
Args:
rule: Format "size:tech" (e.g., "32:b64", "16:hex")
"""
parts = rule.split(":")
if len(parts) != 2:
raise ValueError(f"Invalid security rule format: {rule}")
size_str, tech = parts
size = int(size_str)
match tech:
case "b64":
return secrets.token_urlsafe(size)
case "hex":
return secrets.token_hex(size)
case _:
raise ValueError(f"Invalid security format: {tech}")
def get_nested_value(data: dict, path: str) -> Any:
"""
Get nested value from dict using dot notation.
Example:
get_nested_value({"a": {"b": {"c": 1}}}, "a.b.c") -> 1
"""
keys = path.split(".")
value = data
for key in keys:
if not isinstance(value, dict):
raise KeyError(f"Cannot access {key} in non-dict value")
value = value[key]
return value
class SettingsBootstrap:
"""
Bootstrap settings from schema.
This class orchestrates the entire bootstrap process:
1. Load schema
2. Load sources (TOML, existing .env)
3. Resolve all settings
4. Validate
5. Write .env and .env.make
"""
def __init__(self, source: ConfigSource, schema: SettingsSchema | None = None):
"""
Initialize bootstrap.
Args:
source: Configuration source paths
schema: Settings schema (uses global SCHEMA if None)
"""
self.source = source
self.schema = schema or SCHEMA
self.toml_data: dict | None = None
self.existing_env: dict[str, str] = {}
self.resolved_settings: dict[str, Any] = {}
def bootstrap(self) -> None:
"""
Run complete bootstrap process.
This is the main entry point that orchestrates everything.
"""
print("<EFBFBD><EFBFBD><EFBFBD><EFBFBD> Starting settings bootstrap...")
# 1. Load sources
self._load_sources()
# 2. Resolve all settings
self._resolve_settings()
# 3. Validate
self._validate_settings()
# 4. Write files
self._write_env()
self._write_env_make()
print("✅ Bootstrap complete!")
print("\n⚠️ Reminder: Add your API keys to .env if needed")
def _load_sources(self) -> None:
"""Load TOML and existing .env."""
# Load TOML
if not self.source.toml_path.exists():
raise FileNotFoundError(
f"pyproject.toml not found: {self.source.toml_path}"
)
with open(self.source.toml_path, "rb") as f:
self.toml_data = tomllib.load(f)
# Load existing .env
if self.source.env_path.exists():
print("🔄 Reading existing .env...")
with open(self.source.env_path) as f:
for line in f:
if "=" in line and not line.strip().startswith("#"):
key, value = line.split("=", 1)
self.existing_env[key.strip()] = value.strip()
print(f" Found {len(self.existing_env)} existing keys")
else:
print("🔧 Creating new .env file...")
def _resolve_settings(self) -> None:
"""Resolve all settings from their sources."""
print("📋 Resolving settings...")
# First pass: resolve non-computed settings
for definition in self.schema:
if definition.source != SettingSource.COMPUTED:
self.resolved_settings[definition.name] = self._resolve_setting(
definition
)
# Second pass: resolve computed settings (they may depend on others)
for definition in self.schema:
if definition.source == SettingSource.COMPUTED:
self.resolved_settings[definition.name] = self._resolve_setting(
definition
)
def _resolve_setting(self, definition: SettingDefinition) -> Any:
"""Resolve a single setting value."""
match definition.source:
case SettingSource.TOML:
return self._resolve_from_toml(definition)
case SettingSource.ENV:
return self._resolve_from_env(definition)
case SettingSource.GENERATED:
return self._resolve_generated(definition)
case SettingSource.COMPUTED:
return self._resolve_computed(definition)
def _resolve_from_toml(self, definition: SettingDefinition) -> Any:
"""Resolve setting from TOML."""
if not definition.toml_path:
raise ValueError(
f"{definition.name}: toml_path is required for TOML source"
)
value = get_nested_value(self.toml_data, definition.toml_path)
# Apply transform if specified
if definition.transform:
match definition.transform:
case "extract_python_version_full":
value, _ = extract_python_version(value)
case "extract_python_version_short":
_, value = extract_python_version(value)
case _:
raise ValueError(f"Unknown transform: {definition.transform}")
return value
def _resolve_from_env(self, definition: SettingDefinition) -> Any:
"""Resolve setting from .env."""
# Check existing .env first
if definition.name in self.existing_env:
value = self.existing_env[definition.name]
elif definition.default is not None:
value = definition.default
elif not definition.required:
return None
else:
raise ValueError(f"{definition.name} is required but not found in .env")
# Convert type (only if value is a string from .env)
match definition.type:
case SettingType.INTEGER:
return int(value) if not isinstance(value, int) else value
case SettingType.FLOAT:
return float(value) if not isinstance(value, float) else value
case SettingType.BOOLEAN:
if isinstance(value, bool):
return value
return str(value).lower() in ("true", "1", "yes")
case _:
return str(value) if not isinstance(value, str) else value
def _resolve_generated(self, definition: SettingDefinition) -> str:
"""Resolve generated secret."""
# Preserve existing secret
if definition.name in self.existing_env:
print(f" ↻ Kept existing {definition.name}")
return self.existing_env[definition.name]
# Generate new secret
if not definition.secret_rule:
raise ValueError(
f"{definition.name}: secret_rule is required for GENERATED source"
)
secret = generate_secret(definition.secret_rule)
print(f" + Generated {definition.name} ({definition.secret_rule})")
return secret
def _resolve_computed(self, definition: SettingDefinition) -> str:
"""Resolve computed setting."""
if not definition.compute_template:
raise ValueError(
f"{definition.name}: compute_template is required for COMPUTED source"
)
# Build context from dependencies
context = {}
if definition.compute_from:
for dep in definition.compute_from:
if dep not in self.resolved_settings:
raise ValueError(
f"{definition.name}: dependency {dep} not resolved yet"
)
context[dep] = self.resolved_settings[dep]
# Format template
return definition.compute_template.format(**context)
def _validate_settings(self) -> None:
"""Validate all resolved settings."""
print("✓ Validating settings...")
errors = []
for definition in self.schema:
value = self.resolved_settings.get(definition.name)
try:
validate_value(definition, value)
except ValueError as e:
errors.append(str(e))
if errors:
raise ValueError(
"Validation errors:\n" + "\n".join(f" - {e}" for e in errors)
)
def _write_env(self) -> None:
"""
Write .env file using .env.example as template.
This preserves the structure, comments, and formatting of .env.example
while updating only the values of variables defined in the schema.
Custom variables from existing .env are appended at the end.
"""
print("📝 Writing .env...")
# Check if .env.example exists
if not self.source.env_example_path.exists():
raise FileNotFoundError(
f".env.example not found: {self.source.env_example_path}"
)
# Read .env.example as template
with open(self.source.env_example_path, encoding="utf-8") as f:
template_lines = f.readlines()
# Track which keys we've processed from .env.example
processed_keys = set()
# Process template line by line
output_lines = []
for line in template_lines:
stripped = line.strip()
# Keep comments and empty lines as-is
if not stripped or stripped.startswith("#"):
output_lines.append(line)
continue
# Check if line contains a variable assignment
if "=" in line:
key, _ = line.split("=", 1)
key = key.strip()
processed_keys.add(key)
# Check if this variable is in our schema
definition = self.schema.get(key)
if definition:
# Update with resolved value (including computed settings)
value = self.resolved_settings.get(key, "")
# Convert Python booleans to lowercase for .env compatibility
if isinstance(value, bool):
value = "true" if value else "false"
output_lines.append(f"{key}={value}\n")
# Variable not in schema
# If it exists in current .env, use that value, otherwise keep template
elif key in self.existing_env:
output_lines.append(f"{key}={self.existing_env[key]}\n")
else:
output_lines.append(line)
else:
# Keep any other lines as-is
output_lines.append(line)
# Append custom variables from existing .env that aren't in .env.example
custom_vars = {
k: v for k, v in self.existing_env.items() if k not in processed_keys
}
if custom_vars:
output_lines.append("\n# --- CUSTOM VARIABLES ---\n")
output_lines.append("# Variables added manually (not in .env.example)\n")
for key, value in sorted(custom_vars.items()):
output_lines.append(f"{key}={value}\n")
# Write updated .env
with open(self.source.env_path, "w", encoding="utf-8") as f:
f.writelines(output_lines)
print(f"{self.source.env_path.name} written (preserving template structure)")
if custom_vars:
print(f" Preserved {len(custom_vars)} custom variable(s)")
def _write_env_make(self) -> None:
"""Write .env.make for Makefile."""
print("📝 Writing .env.make...")
lines = ["# Auto-generated from pyproject.toml\n"]
for definition in self.schema.get_for_env_make():
value = self.resolved_settings.get(definition.name, "")
lines.append(f"export {definition.name}={value}\n")
env_make_path = self.source.base_dir / ".env.make"
with open(env_make_path, "w", encoding="utf-8") as f:
f.writelines(lines)
print("✅ .env.make written")
def bootstrap_env(source: ConfigSource) -> None: # noqa: PLC0415
"""
Bootstrap environment configuration.
This is the main entry point for bootstrapping.
Args:
source: Configuration source paths
"""
bootstrapper = SettingsBootstrap(source)
bootstrapper.bootstrap()

278
alfred/settings_schema.py Normal file
View File

@@ -0,0 +1,278 @@
"""
Settings schema parser and definitions.
This module loads the settings schema from pyproject.toml and provides
type-safe access to setting definitions.
"""
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from typing import Any
import tomllib
BASE_DIR = Path(__file__).resolve().parent.parent
class SettingType(Enum):
"""Type of setting value."""
STRING = "string"
INTEGER = "integer"
FLOAT = "float"
BOOLEAN = "boolean"
SECRET = "secret"
COMPUTED = "computed"
class SettingSource(Enum):
"""Source of setting value."""
ENV = "env" # From .env file
TOML = "toml" # From pyproject.toml
GENERATED = "generated" # Auto-generated (secrets)
COMPUTED = "computed" # Computed from other settings
@dataclass
class SettingDefinition:
"""
Complete definition of a setting.
This is the parsed representation of a setting from pyproject.toml.
"""
name: str
type: SettingType
source: SettingSource
description: str = ""
category: str = "general"
required: bool = True
default: str | int | float | bool | None = None
# For TOML source
toml_path: str | None = None
transform: str | None = None # Transform function name
# For SECRET source
secret_rule: str | None = None # e.g., "32:b64", "16:hex"
# For COMPUTED source
compute_from: list[str] | None = None # Dependencies
compute_template: str | None = None # Template string
# For validation
validator: str | None = None # e.g., "range:0.0:2.0"
# For export
export_to_env_make: bool = False
class SettingsSchema:
"""
Settings schema loaded from pyproject.toml.
Provides access to all setting definitions and utilities for
working with the schema.
"""
def __init__(self, schema_dict: dict[str, dict[str, Any]]):
"""
Initialize schema from parsed TOML.
Args:
schema_dict: Dictionary from [tool.alfred.settings_schema]
"""
self.definitions: dict[str, SettingDefinition] = {}
self._parse_schema(schema_dict)
def _parse_schema(self, schema_dict: dict[str, dict[str, Any]]) -> None:
"""Parse schema dictionary into SettingDefinition objects."""
for name, config in schema_dict.items():
# Skip non-setting entries
if not isinstance(config, dict):
continue
# Parse type
type_str = config.get("type", "string")
setting_type = SettingType(type_str)
# Parse source
source_str = config.get("source", "env")
source = SettingSource(source_str)
# Parse default value based on type
default = config.get("default")
if default is not None:
match setting_type:
case SettingType.INTEGER:
default = int(default)
case SettingType.FLOAT:
default = float(default)
case SettingType.BOOLEAN:
default = bool(default)
case _:
default = str(default) if default else None
# Create definition
definition = SettingDefinition(
name=name,
type=setting_type,
source=source,
description=config.get("description", ""),
category=config.get("category", "general"),
required=config.get("required", True),
default=default,
toml_path=config.get("toml_path"),
transform=config.get("transform"),
secret_rule=config.get("secret_rule"),
compute_from=config.get("compute_from"),
compute_template=config.get("compute_template"),
validator=config.get("validator"),
export_to_env_make=config.get("export_to_env_make", False),
)
self.definitions[name] = definition
def get(self, name: str) -> SettingDefinition | None:
"""Get setting definition by name."""
return self.definitions.get(name)
def get_by_category(self, category: str) -> list[SettingDefinition]:
"""Get all settings in a category."""
return [d for d in self.definitions.values() if d.category == category]
def get_by_source(self, source: SettingSource) -> list[SettingDefinition]:
"""Get all settings from a specific source."""
return [d for d in self.definitions.values() if d.source == source]
def get_required(self) -> list[SettingDefinition]:
"""Get all required settings."""
return [d for d in self.definitions.values() if d.required]
def get_for_env_make(self) -> list[SettingDefinition]:
"""Get all settings that should be exported to .env.make."""
return [d for d in self.definitions.values() if d.export_to_env_make]
def __iter__(self):
"""Iterate over all setting definitions."""
return iter(self.definitions.values())
def __len__(self):
"""Number of settings in schema."""
return len(self.definitions)
def load_schema(base_dir: Path | None = None) -> SettingsSchema:
"""
Load settings schema from settings.toml or pyproject.toml.
Priority:
1. settings.toml (if exists)
2. pyproject.toml [tool.alfred.settings_schema]
Args:
base_dir: Base directory containing config files
Returns:
SettingsSchema instance
Raises:
FileNotFoundError: If neither file exists
KeyError: If settings_schema section is missing
"""
if base_dir is None:
base_dir = BASE_DIR
# Load from settings.toml (required)
settings_toml_path = base_dir / "settings.toml"
if not settings_toml_path.exists():
raise FileNotFoundError(
f"settings.toml not found at {settings_toml_path}. "
f"This file is required and must be present in the application root."
)
with open(settings_toml_path, "rb") as f:
data = tomllib.load(f)
try:
schema_dict = data["tool"]["alfred"]["settings_schema"]
return SettingsSchema(schema_dict)
except KeyError as e:
raise KeyError(
f"Missing [tool.alfred.settings_schema] section in settings.toml at {settings_toml_path}"
) from e
def validate_value(definition: SettingDefinition, value: Any) -> bool:
"""
Validate a value against a setting definition.
Args:
definition: Setting definition with validation rules
value: Value to validate
Returns:
True if valid
Raises:
ValueError: If validation fails
"""
if value is None:
if definition.required:
raise ValueError(f"{definition.name} is required but got None")
return True
# Type validation
match definition.type:
case SettingType.INTEGER:
if not isinstance(value, int):
raise ValueError(
f"{definition.name} must be integer, got {type(value).__name__}"
)
case SettingType.FLOAT:
if not isinstance(value, (int, float)):
raise ValueError(
f"{definition.name} must be float, got {type(value).__name__}"
)
case SettingType.BOOLEAN:
if not isinstance(value, bool):
raise ValueError(
f"{definition.name} must be boolean, got {type(value).__name__}"
)
case SettingType.STRING | SettingType.SECRET:
if not isinstance(value, str):
raise ValueError(
f"{definition.name} must be string, got {type(value).__name__}"
)
# Custom validator
if definition.validator:
_apply_validator(definition.name, definition.validator, value)
return True
def _apply_validator(name: str, validator: str, value: Any) -> None:
"""Apply custom validator to value."""
if validator.startswith("range:"):
# Parse range validator: "range:min:max"
parts = validator.split(":")
if len(parts) != 3:
raise ValueError(f"Invalid range validator format: {validator}")
min_val = float(parts[1])
max_val = float(parts[2])
if not (min_val <= value <= max_val):
raise ValueError(
f"{name} must be between {min_val} and {max_val}, got {value}"
)
else:
raise ValueError(f"Unknown validator: {validator}")
# Load schema once at module import
SCHEMA = load_schema()

View File

@@ -1,400 +0,0 @@
# Agent Media 🎬
An AI-powered agent for managing your local media library with natural language. Search, download, and organize movies and TV shows effortlessly.
## Features
- 🤖 **Natural Language Interface**: Talk to your media library in plain language
- 🔍 **Smart Search**: Find movies and TV shows via TMDB
- 📥 **Torrent Integration**: Search and download via qBittorrent
- 🧠 **Contextual Memory**: Remembers your preferences and conversation history
- 📁 **Auto-Organization**: Keeps your media library tidy
- 🌐 **API Compatible**: OpenAI-compatible API for easy integration
## Architecture
Built with **Domain-Driven Design (DDD)** principles:
```
agent_media/
├── agent/ # AI agent orchestration
├── application/ # Use cases & DTOs
├── domain/ # Business logic & entities
└── infrastructure/ # External services & persistence
```
See [architecture_diagram.md](docs/architecture_diagram.md) for architectural details.
## Quick Start
### Prerequisites
- Python 3.12+
- Poetry
- qBittorrent (optional, for downloads)
- API Keys:
- DeepSeek API key (or Ollama for local LLM)
- TMDB API key
### Installation
```bash
# Clone the repository
git clone https://github.com/your-username/agent-media.git
cd agent-media
# Install dependencies
poetry install
# Copy environment template
cp .env.example .env
# Edit .env with your API keys
nano .env
```
### Configuration
Edit `.env`:
```bash
# LLM Provider (deepseek or ollama)
LLM_PROVIDER=deepseek
DEEPSEEK_API_KEY=your-api-key-here
# TMDB (for movie/TV show metadata)
TMDB_API_KEY=your-tmdb-key-here
# qBittorrent (optional)
QBITTORRENT_HOST=http://localhost:8080
QBITTORRENT_USERNAME=admin
QBITTORRENT_PASSWORD=adminadmin
```
### Run
```bash
# Start the API server
poetry run uvicorn app:app --reload
# Or with Docker
docker-compose up
```
The API will be available at `http://localhost:8000`
## Usage
### Via API
```bash
# Health check
curl http://localhost:8000/health
# Chat with the agent
curl -X POST http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "agent-media",
"messages": [
{"role": "user", "content": "Find Inception 1080p"}
]
}'
```
### Via OpenWebUI
Agent Media is compatible with [OpenWebUI](https://github.com/open-webui/open-webui):
1. Add as OpenAI-compatible endpoint: `http://localhost:8000/v1`
2. Model name: `agent-media`
3. Start chatting!
### Example Conversations
```
You: Find Inception in 1080p
Agent: I found 3 torrents for Inception:
1. Inception.2010.1080p.BluRay.x264 (150 seeders)
2. Inception.2010.1080p.WEB-DL.x265 (80 seeders)
3. Inception.2010.720p.BluRay (45 seeders)
You: Download the first one
Agent: Added to qBittorrent! Download started.
You: List my downloads
Agent: You have 1 active download:
- Inception.2010.1080p.BluRay.x264 (45% complete)
```
## Available Tools
The agent has access to these tools:
| Tool | Description |
|------|-------------|
| `find_media_imdb_id` | Search for movies/TV shows on TMDB |
| `find_torrents` | Search for torrents |
| `get_torrent_by_index` | Get torrent details by index |
| `add_torrent_by_index` | Download torrent by index |
| `add_torrent_to_qbittorrent` | Add torrent via magnet link |
| `set_path_for_folder` | Configure folder paths |
| `list_folder` | List folder contents |
## Memory System
Agent Media uses a three-tier memory system:
### Long-Term Memory (LTM)
- **Persistent** (saved to JSON)
- Configuration, preferences, media library
- Survives restarts
### Short-Term Memory (STM)
- **Session-based** (RAM only)
- Conversation history, current workflow
- Cleared on restart
### Episodic Memory
- **Transient** (RAM only)
- Search results, active downloads, recent errors
- Cleared frequently
## Development
### Project Structure
```
agent_media/
├── agent/
│ ├── agent.py # Main agent orchestrator
│ ├── prompts.py # System prompt builder
│ ├── registry.py # Tool registration
│ ├── tools/ # Tool implementations
│ └── llm/ # LLM clients (DeepSeek, Ollama)
├── application/
│ ├── movies/ # Movie use cases
│ ├── torrents/ # Torrent use cases
│ └── filesystem/ # Filesystem use cases
├── domain/
│ ├── movies/ # Movie entities & value objects
│ ├── tv_shows/ # TV show entities
│ ├── subtitles/ # Subtitle entities
│ └── shared/ # Shared value objects
├── infrastructure/
│ ├── api/ # External API clients
│ │ ├── tmdb/ # TMDB client
│ │ ├── knaben/ # Torrent search
│ │ └── qbittorrent/ # qBittorrent client
│ ├── filesystem/ # File operations
│ └── persistence/ # Memory & repositories
├── tests/ # Test suite (~500 tests)
└── docs/ # Documentation
```
### Running Tests
```bash
# Run all tests
poetry run pytest
# Run with coverage
poetry run pytest --cov
# Run specific test file
poetry run pytest tests/test_agent.py
# Run specific test
poetry run pytest tests/test_agent.py::TestAgent::test_step
```
### Code Quality
```bash
# Linting
poetry run ruff check .
# Formatting
poetry run black .
# Type checking (if mypy is installed)
poetry run mypy .
```
### Adding a New Tool
Quick example:
```python
# 1. Create the tool function in agent/tools/api.py
def my_new_tool(param: str) -> Dict[str, Any]:
"""Tool description."""
memory = get_memory()
# Implementation
return {"status": "ok", "data": "result"}
# 2. Register in agent/registry.py
Tool(
name="my_new_tool",
description="What this tool does",
func=api_tools.my_new_tool,
parameters={
"type": "object",
"properties": {
"param": {"type": "string", "description": "Parameter description"},
},
"required": ["param"],
},
),
```
## Docker
### Build
```bash
docker build -t agent-media .
```
### Run
```bash
docker run -p 8000:8000 \
-e DEEPSEEK_API_KEY=your-key \
-e TMDB_API_KEY=your-key \
-v $(pwd)/memory_data:/app/memory_data \
agent-media
```
### Docker Compose
```bash
# Start all services (agent + qBittorrent)
docker-compose up -d
# View logs
docker-compose logs -f
# Stop
docker-compose down
```
## API Documentation
### Endpoints
#### `GET /health`
Health check endpoint.
**Response:**
```json
{
"status": "healthy",
"version": "0.2.0"
}
```
#### `GET /v1/models`
List available models (OpenAI-compatible).
#### `POST /v1/chat/completions`
Chat with the agent (OpenAI-compatible).
**Request:**
```json
{
"model": "agent-media",
"messages": [
{"role": "user", "content": "Find Inception"}
],
"stream": false
}
```
**Response:**
```json
{
"id": "chatcmpl-xxx",
"object": "chat.completion",
"created": 1234567890,
"model": "agent-media",
"choices": [{
"index": 0,
"message": {
"role": "assistant",
"content": "I found Inception (2010)..."
},
"finish_reason": "stop"
}]
}
```
#### `GET /memory/state`
View full memory state (debug).
#### `POST /memory/clear-session`
Clear session memories (STM + Episodic).
## Troubleshooting
### Agent doesn't respond
- Check API keys in `.env`
- Verify LLM provider is running (Ollama) or accessible (DeepSeek)
- Check logs: `docker-compose logs agent-media`
### qBittorrent connection failed
- Verify qBittorrent is running
- Check `QBITTORRENT_HOST` in `.env`
- Ensure Web UI is enabled in qBittorrent settings
### Memory not persisting
- Check `memory_data/` directory exists and is writable
- Verify volume mounts in Docker
### Tests failing
- Run `poetry install` to ensure dependencies are up to date
- Check logs for specific error messages
## Contributing
Contributions are welcome!
### Development Workflow
1. Fork the repository
2. Create a feature branch: `git checkout -b feature/my-feature`
3. Make your changes
4. Run tests: `poetry run pytest`
5. Run linting: `poetry run ruff check . && poetry run black .`
6. Commit: `git commit -m "Add my feature"`
7. Push: `git push origin feature/my-feature`
8. Create a Pull Request
## Documentation
- [Architecture Diagram](docs/architecture_diagram.md) - System architecture overview
- [Class Diagram](docs/class_diagram.md) - Class structure and relationships
- [Component Diagram](docs/component_diagram.md) - Component interactions
- [Sequence Diagram](docs/sequence_diagram.md) - Sequence flows
- [Flowchart](docs/flowchart.md) - System flowcharts
## License
MIT License - see [LICENSE](LICENSE) file for details.
## Acknowledgments
- [DeepSeek](https://www.deepseek.com/) - LLM provider
- [TMDB](https://www.themoviedb.org/) - Movie database
- [qBittorrent](https://www.qbittorrent.org/) - Torrent client
- [FastAPI](https://fastapi.tiangolo.com/) - Web framework
## Support
- 📧 Email: francois.hodiaumont@gmail.com
- 🐛 Issues: [GitHub Issues](https://github.com/your-username/agent-media/issues)
- 💬 Discussions: [GitHub Discussions](https://github.com/your-username/agent-media/discussions)
---
Made with ❤️ by Francwa

View File

@@ -1,115 +0,0 @@
"""Configuration management with validation."""
import os
from dataclasses import dataclass, field
from pathlib import Path
from dotenv import load_dotenv
# Load environment variables from .env file
load_dotenv()
class ConfigurationError(Exception):
"""Raised when configuration is invalid."""
pass
@dataclass
class Settings:
"""Application settings loaded from environment variables."""
# LLM Configuration
deepseek_api_key: str = field(
default_factory=lambda: os.getenv("DEEPSEEK_API_KEY", "")
)
deepseek_base_url: str = field(
default_factory=lambda: os.getenv(
"DEEPSEEK_BASE_URL", "https://api.deepseek.com"
)
)
model: str = field(
default_factory=lambda: os.getenv("DEEPSEEK_MODEL", "deepseek-chat")
)
temperature: float = field(
default_factory=lambda: float(os.getenv("TEMPERATURE", "0.2"))
)
# TMDB Configuration
tmdb_api_key: str = field(default_factory=lambda: os.getenv("TMDB_API_KEY", ""))
tmdb_base_url: str = field(
default_factory=lambda: os.getenv(
"TMDB_BASE_URL", "https://api.themoviedb.org/3"
)
)
# Storage Configuration
memory_file: str = field(
default_factory=lambda: os.getenv("MEMORY_FILE", "memory.json")
)
# Security Configuration
max_tool_iterations: int = field(
default_factory=lambda: int(os.getenv("MAX_TOOL_ITERATIONS", "5"))
)
request_timeout: int = field(
default_factory=lambda: int(os.getenv("REQUEST_TIMEOUT", "30"))
)
# Memory Configuration
max_history_messages: int = field(
default_factory=lambda: int(os.getenv("MAX_HISTORY_MESSAGES", "10"))
)
def __post_init__(self):
"""Validate settings after initialization."""
self._validate()
def _validate(self) -> None:
"""Validate configuration values."""
# Validate temperature
if not 0.0 <= self.temperature <= 2.0:
raise ConfigurationError(
f"Temperature must be between 0.0 and 2.0, got {self.temperature}"
)
# Validate max_tool_iterations
if self.max_tool_iterations < 1 or self.max_tool_iterations > 20:
raise ConfigurationError(
f"max_tool_iterations must be between 1 and 20, got {self.max_tool_iterations}"
)
# Validate request_timeout
if self.request_timeout < 1 or self.request_timeout > 300:
raise ConfigurationError(
f"request_timeout must be between 1 and 300 seconds, got {self.request_timeout}"
)
# Validate URLs
if not self.deepseek_base_url.startswith(("http://", "https://")):
raise ConfigurationError(
f"Invalid deepseek_base_url: {self.deepseek_base_url}"
)
if not self.tmdb_base_url.startswith(("http://", "https://")):
raise ConfigurationError(f"Invalid tmdb_base_url: {self.tmdb_base_url}")
# Validate memory file path
memory_path = Path(self.memory_file)
if memory_path.exists() and not memory_path.is_file():
raise ConfigurationError(
f"memory_file exists but is not a file: {self.memory_file}"
)
def is_deepseek_configured(self) -> bool:
"""Check if DeepSeek API is properly configured."""
return bool(self.deepseek_api_key and self.deepseek_base_url)
def is_tmdb_configured(self) -> bool:
"""Check if TMDB API is properly configured."""
return bool(self.tmdb_api_key and self.tmdb_base_url)
# Global settings instance
settings = Settings()

210
docker-compose.yaml Normal file
View File

@@ -0,0 +1,210 @@
services:
# - CORE SERVICES -
# --- .ENV INIT ---
alfred-init:
container_name: alfred-init
build:
context: .
target: builder
args:
PYTHON_VERSION: ${PYTHON_VERSION}
PYTHON_VERSION_SHORT: ${PYTHON_VERSION_SHORT}
RUNNER: ${RUNNER}
command: python scripts/bootstrap.py
networks:
- alfred-net
# --- MAIN APPLICATION ---
alfred:
container_name: alfred-core
image: alfred_media_organizer:latest
build:
context: .
args:
PYTHON_VERSION: ${PYTHON_VERSION}
PYTHON_VERSION_SHORT: ${PYTHON_VERSION_SHORT}
RUNNER: ${RUNNER}
depends_on:
alfred-init:
condition: service_completed_successfully
restart: unless-stopped
env_file:
- path: .env
required: true
volumes:
- ./data:/data
- ./logs:/logs
# TODO: Hot reload (comment out in production)
#- ./alfred:/home/appuser/alfred
command: >
sh -c "python -u -m uvicorn alfred.app:app --host 0.0.0.0 --port 8000 2>&1 | tee -a /logs/alfred.log"
networks:
- alfred-net
# --- FRONTEND LIBRECHAT ---
librechat:
container_name: alfred-librechat
image: ghcr.io/danny-avila/librechat:${LIBRECHAT_VERSION}
depends_on:
alfred-init:
condition: service_completed_successfully
mongodb:
condition: service_healthy
restart: unless-stopped
env_file:
- path: .env
required: true
environment:
# Remap value name
- SEARCH=${MEILI_ENABLED}
ports:
- "${PORT}:${PORT}"
volumes:
- ./data/librechat/images:/app/client/public/images
- ./data/librechat/uploads:/app/client/uploads
- ./logs:/app/api/logs
# Mount custom endpoint
- ./librechat/manifests:/app/manifests:ro
- ./librechat/librechat.yaml:/app/librechat.yaml:ro
networks:
- alfred-net
# --- DATABASE #1 - APP STATE ---
mongodb:
container_name: alfred-mongodb
image: mongo:latest
restart: unless-stopped
depends_on:
alfred-init:
condition: service_completed_successfully
env_file:
- path: .env
required: true
environment:
# Remap value name
- MONGO_INITDB_ROOT_USERNAME=${MONGO_USER}
- MONGO_INITDB_ROOT_PASSWORD=${MONGO_PASSWORD}
ports:
- "${MONGO_PORT}:${MONGO_PORT}"
volumes:
- ./data/mongodb:/data/db
- ./mongod.conf:/etc/mongod.conf:ro
command: ["mongod", "--config", "/etc/mongod.conf"]
healthcheck:
test: mongosh --quiet -u "${MONGO_USER}" -p "${MONGO_PASSWORD}" --authenticationDatabase admin --eval "db.adminCommand('ping')"
interval: 10s
timeout: 5s
retries: 5
networks:
- alfred-net
# --- OLLAMA - LOCAL LLM ENGINE ---
ollama:
image: ollama/ollama:latest
container_name: alfred-ollama
depends_on:
alfred-init:
condition: service_completed_successfully
restart: unless-stopped
env_file:
- path: .env
required: true
volumes:
- ./data/ollama:/root/.ollama
networks:
- alfred-net
# - OPTIONAL SERVICES -
# --- SEARCH ENGINE SUPER FAST (Optional) ---
meilisearch:
container_name: alfred-meilisearch
image: getmeili/meilisearch:v1.12.3
depends_on:
alfred-init:
condition: service_completed_successfully
restart: unless-stopped
env_file:
- path: .env
required: true
volumes:
- ./data/meilisearch:/meili_data
profiles: ["meili", "full"]
networks:
- alfred-net
# --- RETRIEVAL AUGMENTED GENERATION SYSTEM (Optional) ---
rag_api:
container_name: alfred-rag
image: ghcr.io/danny-avila/librechat-rag-api-dev-lite:${RAG_VERSION}
depends_on:
alfred-init:
condition: service_completed_successfully
vectordb:
condition: service_healthy
restart: unless-stopped
env_file:
- path: .env
required: true
ports:
- "${RAG_API_PORT}:${RAG_API_PORT}"
volumes:
- ./data/rag/uploads:/app/uploads
profiles: ["rag", "full"]
networks:
- alfred-net
# --- DATABASE #2 - Vector RAG (Optional) ---
vectordb:
container_name: alfred-vectordb
image: pgvector/pgvector:0.8.0-pg16-bookworm
depends_on:
alfred-init:
condition: service_completed_successfully
restart: unless-stopped
env_file:
- path: .env
required: true
ports:
- "${POSTGRES_PORT}:${POSTGRES_PORT}"
volumes:
- ./data/vectordb:/var/lib/postgresql/data
profiles: ["rag", "full"]
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB_NAME}" ]
interval: 5s
timeout: 5s
retries: 5
networks:
alfred-net:
aliases:
- db
# --- QBITTORENT (Optional) ---
qbittorrent:
image: lscr.io/linuxserver/qbittorrent:latest
container_name: alfred-qbittorrent
depends_on:
alfred-init:
condition: service_completed_successfully
restart: unless-stopped
env_file:
- path: .env
required: true
environment:
- PUID=1000
- PGID=1000
- TZ=Europe/Paris
- WEBUI_PORT=${QBITTORRENT_PORT}
volumes:
- ./data/qbittorrent/config:/config
- ./data/qbittorrent/downloads:/downloads
profiles: ["qbittorrent", "full"]
ports:
- "${QBITTORRENT_PORT}:${QBITTORRENT_PORT}"
networks:
- alfred-net
networks:
alfred-net:
name: alfred-internal
driver: bridge

View File

@@ -1,206 +0,0 @@
version: "3.4"
services:
# Da brain
agent-brain:
build:
context: ./brain
dockerfile: Dockerfile
args:
RUNNER: ${RUNNER} # Get it from Makefile
container_name: agent-brain
restart: unless-stopped
env_file: .env
ports:
- "8000:8000"
volumes:
# Persistent data volumes (outside container /app)
- agent-memory:/data/memory
- agent-logs:/data/logs
# Development: mount code for hot reload (comment out in production)
# - ./brain:/app
environment:
# LLM Configuration
LLM_PROVIDER: ${LLM_PROVIDER:-deepseek}
DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY:-}
# Memory storage
MEMORY_STORAGE_DIR: /data/memory
# External services
TMDB_API_KEY: ${TMDB_API_KEY:-}
QBITTORRENT_URL: ${QBITTORRENT_URL:-}
QBITTORRENT_USERNAME: ${QBITTORRENT_USERNAME:-}
QBITTORRENT_PASSWORD: ${QBITTORRENT_PASSWORD:-}
networks:
- agent-network
# Da face (LibreChat)
librechat:
image: ghcr.io/danny-avila/librechat-dev:latest
container_name: librechat-frontend
restart: unless-stopped
ports:
- "3080:3080"
depends_on:
- mongodb
- meilisearch
- rag_api
- agent-brain
env_file: .env
environment:
# MongoDB connection (no auth, matching LibreChat default)
MONGO_URI: mongodb://mongodb:27017/LibreChat
# App configuration
HOST: 0.0.0.0
PORT: 3080
# Security
JWT_SECRET: ${JWT_SECRET:-your-super-secret-jwt-key-change-this-in-production}
JWT_REFRESH_SECRET: ${JWT_REFRESH_SECRET:-your-super-secret-refresh-key-change-this-too}
CREDS_KEY: ${CREDS_KEY:-your-32-character-secret-key-here}
CREDS_IV: ${CREDS_IV:-your-16-character-iv-here}
# Session
SESSION_EXPIRY: ${SESSION_EXPIRY:-1000 * 60 * 15}
REFRESH_TOKEN_EXPIRY: ${REFRESH_TOKEN_EXPIRY:-1000 * 60 * 60 * 24 * 7}
# Domain
DOMAIN_CLIENT: ${DOMAIN_CLIENT:-http://localhost:3080}
DOMAIN_SERVER: ${DOMAIN_SERVER:-http://localhost:3080}
# Meilisearch
MEILI_HOST: http://meilisearch:7700
MEILI_MASTER_KEY: ${MEILI_MASTER_KEY:-DrhYf7zENyR6AlUCKmnz0eYASOQdl6zxH7s7MKFSfFU}
# RAG API
RAG_API_URL: http://rag_api:8000
# Endpoints
ENDPOINTS: custom
# Custom endpoint pointing to agent-brain
CUSTOM_API_KEY: ${AGENT_BRAIN_API_KEY:-agent-brain-secret-key}
# Debug (optional)
DEBUG_LOGGING: ${DEBUG_LOGGING:-false}
DEBUG_CONSOLE: ${DEBUG_CONSOLE:-false}
volumes:
- ./librechat/librechat.yaml:/app/librechat.yaml:ro
- librechat-images:/app/client/public/images
- librechat-logs:/app/api/logs
networks:
- agent-network
# MongoDB for LibreChat
mongodb:
image: mongo:latest
container_name: librechat-mongodb
restart: unless-stopped
volumes:
- mongodb-data:/data/db
command: mongod --noauth
ports:
- "27017:27017"
networks:
- agent-network
# Meilisearch - Search engine for LibreChat
meilisearch:
image: getmeili/meilisearch:v1.11.3
container_name: librechat-meilisearch
restart: unless-stopped
volumes:
- meilisearch-data:/meili_data
environment:
MEILI_HOST: http://meilisearch:7700
MEILI_HTTP_ADDR: meilisearch:7700
MEILI_MASTER_KEY: ${MEILI_MASTER_KEY:-DrhYf7zENyR6AlUCKmnz0eYASOQdl6zxH7s7MKFSfFU}
ports:
- "7700:7700"
networks:
- agent-network
# PostgreSQL with pgvector for RAG API
pgvector:
image: ankane/pgvector:latest
container_name: librechat-pgvector
restart: unless-stopped
environment:
POSTGRES_DB: ${POSTGRES_DB:-librechat_rag}
POSTGRES_USER: ${POSTGRES_USER:-postgres}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-postgres}
volumes:
- pgvector-data:/var/lib/postgresql/data
ports:
- "5432:5432"
networks:
- agent-network
# RAG API - Vector database for LibreChat
rag_api:
image: ghcr.io/danny-avila/librechat-rag-api-dev-lite:latest
container_name: librechat-rag-api
restart: unless-stopped
depends_on:
- pgvector
environment:
PORT: 8000
HOST: 0.0.0.0
# PostgreSQL connection (multiple variable names for compatibility)
DB_HOST: pgvector
DB_PORT: 5432
DB_NAME: ${POSTGRES_DB:-librechat_rag}
DB_USER: ${POSTGRES_USER:-postgres}
DB_PASSWORD: ${POSTGRES_PASSWORD:-postgres}
POSTGRES_DB: ${POSTGRES_DB:-librechat_rag}
POSTGRES_USER: ${POSTGRES_USER:-postgres}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-postgres}
# RAG configuration
COLLECTION_NAME: ${RAG_COLLECTION_NAME:-testcollection}
EMBEDDINGS_PROVIDER: ${RAG_EMBEDDINGS_PROVIDER:-openai}
EMBEDDINGS_MODEL: ${RAG_EMBEDDINGS_MODEL:-text-embedding-3-small}
OPENAI_API_KEY: ${OPENAI_API_KEY:-}
RAG_UPLOAD_DIR: /app/uploads
volumes:
- rag-uploads:/app/uploads
ports:
- "8001:8000"
networks:
- agent-network
# Named volumes for persistent data
volumes:
# MongoDB data
mongodb-data:
driver: local
# Meilisearch data
meilisearch-data:
driver: local
# PostgreSQL pgvector data
pgvector-data:
driver: local
# RAG API uploads
rag-uploads:
driver: local
# LibreChat data
librechat-images:
driver: local
librechat-logs:
driver: local
# Agent Brain data
agent-memory:
driver: local
agent-logs:
driver: local
# Network for inter-service communication
networks:
agent-network:
driver: bridge

View File

@@ -1,27 +0,0 @@
#!/bin/bash
# Script to generate secure keys for LibreChat
# Run this script to generate random secure keys for your .env file
echo "==================================="
echo "LibreChat Security Keys Generator"
echo "==================================="
echo ""
echo "# MongoDB Password"
echo "MONGO_PASSWORD=$(openssl rand -base64 24)"
echo ""
echo "# JWT Secrets"
echo "JWT_SECRET=$(openssl rand -base64 32)"
echo "JWT_REFRESH_SECRET=$(openssl rand -base64 32)"
echo ""
echo "# Credentials Encryption Keys"
echo "CREDS_KEY=$(openssl rand -hex 16)"
echo "CREDS_IV=$(openssl rand -hex 8)"
echo ""
echo "==================================="
echo "Copy these values to your .env file"
echo "==================================="

View File

@@ -4,6 +4,16 @@
version: 1.2.1
cache: true
endpoints:
anthropic:
apiKey: "${ANTHROPIC_API_KEY}"
models:
default: ["claude-sonnet-4-5", "claude-haiku-4-5", "claude-opus-4-5"]
fetch: false
titleConvo: true
titleModel: "claude-haiku-4-5"
modelDisplayLabel: "Claude AI"
streamRate: 1
custom:
# Deepseek
- name: "Deepseek"
@@ -22,7 +32,7 @@ endpoints:
manifest:
schema:
type: openapi
url: "http://agent-brain:8000/manifests/find_media_imdb_id.json"
url: "http://alfred:8000/manifests/find_media_imdb_id.json"
auth:
type: none
@@ -32,7 +42,7 @@ endpoints:
manifest:
schema:
type: openapi
url: "http://agent-brain:8000/manifests/find_torrent.json"
url: "http://alfred:8000/manifests/find_torrent.json"
auth:
type: none
@@ -42,7 +52,7 @@ endpoints:
manifest:
schema:
type: openapi
url: "http://agent-brain:8000/manifests/add_torrent_by_index.json"
url: "http://alfred:8000/manifests/add_torrent_by_index.json"
auth:
type: none
@@ -52,7 +62,7 @@ endpoints:
manifest:
schema:
type: openapi
url: "http://agent-brain:8000/manifests/set_language.json"
url: "http://alfred:8000/manifests/set_language.json"
auth:
type: none
@@ -60,7 +70,7 @@ endpoints:
# Backend Local Agent
- name: "Local Agent"
apiKey: "dummy_key"
baseURL: "http://agent-brain:8000/v1"
baseURL: "http://alfred:8000/v1"
models:
default: ["local-deepseek-agent"]
fetch: false
@@ -75,7 +85,7 @@ endpoints:
manifest:
schema:
type: openapi
url: "http://agent-brain:8000/manifests/find_media_imdb_id.json"
url: "http://alfred:8000/manifests/find_media_imdb_id.json"
auth:
type: none
@@ -85,7 +95,7 @@ endpoints:
manifest:
schema:
type: openapi
url: "http://agent-brain:8000/manifests/find_torrent.json"
url: "http://alfred:8000/manifests/find_torrent.json"
auth:
type: none
@@ -95,7 +105,7 @@ endpoints:
manifest:
schema:
type: openapi
url: "http://agent-brain:8000/manifests/add_torrent_by_index.json"
url: "http://alfred:8000/manifests/add_torrent_by_index.json"
auth:
type: none
@@ -105,6 +115,6 @@ endpoints:
manifest:
schema:
type: openapi
url: "http://agent-brain:8000/manifests/set_language.json"
url: "http://alfred:8000/manifests/set_language.json"
auth:
type: none

Some files were not shown because too many files have changed in this diff Show More