Updated README and did a little bit of cleanup
This commit is contained in:
@@ -31,7 +31,4 @@ MAX_TOOL_ITERATIONS=10
|
||||
REQUEST_TIMEOUT=30
|
||||
|
||||
# Memory Configuration
|
||||
# Number of previous messages to include in context (default: 10)
|
||||
# Higher = more context but slower/more expensive
|
||||
# Lower = less context but faster
|
||||
MAX_HISTORY_MESSAGES=10
|
||||
|
||||
@@ -1,308 +0,0 @@
|
||||
# 🎯 Architecture Finale - 100% DDD
|
||||
|
||||
## ✅ Migration Complète Terminée
|
||||
|
||||
Toute la couche de compatibilité a été supprimée. L'architecture est maintenant **100% Domain-Driven Development**.
|
||||
|
||||
---
|
||||
|
||||
## 📁 Structure Finale
|
||||
|
||||
```
|
||||
agent_media/
|
||||
│
|
||||
├── domain/ # 🎯 LOGIQUE MÉTIER PURE
|
||||
│ ├── shared/
|
||||
│ │ ├── exceptions.py
|
||||
│ │ └── value_objects.py
|
||||
│ ├── movies/
|
||||
│ │ ├── entities.py
|
||||
│ │ ├── value_objects.py
|
||||
│ │ ├── repositories.py
|
||||
│ │ ├── services.py
|
||||
│ │ └── exceptions.py
|
||||
│ ├── tv_shows/
|
||||
│ │ ├── entities.py
|
||||
│ │ ├── value_objects.py
|
||||
│ │ ├── repositories.py
|
||||
│ │ ├── services.py
|
||||
│ │ └── exceptions.py
|
||||
│ └── subtitles/
|
||||
│ ├── entities.py
|
||||
│ ├── value_objects.py
|
||||
│ ├── repositories.py
|
||||
│ ├── services.py
|
||||
│ └── exceptions.py
|
||||
│
|
||||
├── infrastructure/ # 🔧 DÉTAILS TECHNIQUES
|
||||
│ ├── api/
|
||||
│ │ ├── tmdb/
|
||||
│ │ ├── knaben/
|
||||
│ │ └── qbittorrent/
|
||||
│ ├── persistence/
|
||||
│ │ ├── memory.py
|
||||
│ │ └── json/
|
||||
│ └── filesystem/
|
||||
│ ├── file_manager.py
|
||||
│ ├── organizer.py
|
||||
│ └── exceptions.py
|
||||
│
|
||||
├── application/ # 🎬 USE CASES
|
||||
│ ├── movies/
|
||||
│ │ ├── search_movie.py
|
||||
│ │ └── dto.py
|
||||
│ ├── torrents/
|
||||
│ │ ├── search_torrents.py
|
||||
│ │ ├── add_torrent.py
|
||||
│ │ └── dto.py
|
||||
│ └── filesystem/
|
||||
│ ├── set_folder_path.py
|
||||
│ ├── list_folder.py
|
||||
│ └── dto.py
|
||||
│
|
||||
├── agent/ # 🤖 INTERFACE LLM
|
||||
│ ├── llm/
|
||||
│ │ ├── __init__.py
|
||||
│ │ └── deepseek.py
|
||||
│ ├── tools/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── api.py
|
||||
│ │ └── filesystem.py
|
||||
│ ├── agent.py
|
||||
│ ├── registry.py
|
||||
│ ├── prompts.py
|
||||
│ ├── parameters.py
|
||||
│ └── config.py
|
||||
│
|
||||
└── app.py # 🚀 FASTAPI
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Imports Mis à Jour
|
||||
|
||||
### **app.py**
|
||||
```python
|
||||
# AVANT
|
||||
from agent.memory import Memory
|
||||
|
||||
# APRÈS
|
||||
from infrastructure.persistence.memory import Memory
|
||||
```
|
||||
|
||||
### **agent/agent.py**
|
||||
```python
|
||||
# AVANT
|
||||
from .memory import Memory
|
||||
|
||||
# APRÈS
|
||||
from infrastructure.persistence.memory import Memory
|
||||
```
|
||||
|
||||
### **agent/tools/api.py**
|
||||
```python
|
||||
# Utilise directement les use cases
|
||||
from application.movies import SearchMovieUseCase
|
||||
from infrastructure.api.tmdb import tmdb_client
|
||||
```
|
||||
|
||||
### **agent/tools/filesystem.py**
|
||||
```python
|
||||
# Utilise directement les use cases
|
||||
from application.filesystem import SetFolderPathUseCase
|
||||
from infrastructure.filesystem import FileManager
|
||||
from infrastructure.persistence.memory import Memory
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🗑️ Fichiers Supprimés
|
||||
|
||||
### **Ancienne Architecture**
|
||||
```
|
||||
❌ agent/api/themoviedb.py
|
||||
❌ agent/api/knaben.py
|
||||
❌ agent/api/qbittorrent.py
|
||||
❌ agent/api/__init__.py
|
||||
❌ agent/models/tv_show.py
|
||||
❌ agent/models/__init__.py
|
||||
❌ agent/memory.py
|
||||
```
|
||||
|
||||
### **Dossiers Supprimés**
|
||||
```
|
||||
❌ agent/api/
|
||||
❌ agent/models/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Fichiers Conservés
|
||||
|
||||
### **Agent Core**
|
||||
```
|
||||
✅ agent/agent.py # Agent principal (imports mis à jour)
|
||||
✅ agent/registry.py # Registry des tools
|
||||
✅ agent/prompts.py # Construction des prompts
|
||||
✅ agent/parameters.py # Schéma des paramètres
|
||||
✅ agent/config.py # Configuration
|
||||
```
|
||||
|
||||
### **Agent LLM**
|
||||
```
|
||||
✅ agent/llm/__init__.py
|
||||
✅ agent/llm/deepseek.py # Client DeepSeek
|
||||
```
|
||||
|
||||
### **Agent Tools**
|
||||
```
|
||||
✅ agent/tools/__init__.py
|
||||
✅ agent/tools/api.py # Wrappers vers use cases
|
||||
✅ agent/tools/filesystem.py # Wrappers vers use cases
|
||||
```
|
||||
|
||||
### **Application**
|
||||
```
|
||||
✅ app.py # FastAPI (imports mis à jour)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Flux de Données
|
||||
|
||||
```
|
||||
USER
|
||||
↓
|
||||
LibreChat
|
||||
↓
|
||||
app.py (FastAPI)
|
||||
↓
|
||||
Agent (agent/agent.py)
|
||||
↓
|
||||
Tools (agent/tools/)
|
||||
↓
|
||||
Use Cases (application/)
|
||||
↓
|
||||
Domain Services (domain/)
|
||||
↓
|
||||
Infrastructure (infrastructure/)
|
||||
↓
|
||||
External APIs / Storage
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Principes DDD Appliqués
|
||||
|
||||
### **1. Layered Architecture**
|
||||
✅ Séparation stricte : Domain → Application → Infrastructure → Interface
|
||||
|
||||
### **2. Dependency Inversion**
|
||||
✅ Domain ne dépend de rien
|
||||
✅ Infrastructure dépend de Domain
|
||||
✅ Application orchestre Domain et Infrastructure
|
||||
|
||||
### **3. Bounded Contexts**
|
||||
✅ Movies, TV Shows, Subtitles sont des domaines séparés
|
||||
|
||||
### **4. Ubiquitous Language**
|
||||
✅ Vocabulaire métier partagé (Movie, TVShow, Episode, etc.)
|
||||
|
||||
### **5. Entities & Value Objects**
|
||||
✅ Entities : Movie, TVShow, Episode, Subtitle
|
||||
✅ Value Objects : ImdbId, MovieTitle, SeasonNumber, etc.
|
||||
|
||||
### **6. Repositories**
|
||||
✅ Interfaces abstraites dans domain/
|
||||
✅ Implémentations concrètes dans infrastructure/
|
||||
|
||||
### **7. Domain Services**
|
||||
✅ MovieService, TVShowService, SubtitleService
|
||||
|
||||
### **8. Application Services (Use Cases)**
|
||||
✅ SearchMovieUseCase, SearchTorrentsUseCase, etc.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Commandes de Nettoyage
|
||||
|
||||
### **Script Automatique**
|
||||
```bash
|
||||
chmod +x FINAL_CLEANUP.sh
|
||||
./FINAL_CLEANUP.sh
|
||||
```
|
||||
|
||||
### **Manuel**
|
||||
```bash
|
||||
# Supprimer les dossiers
|
||||
rm -rf agent/api/
|
||||
rm -rf agent/models/
|
||||
|
||||
# Supprimer le fichier
|
||||
rm -f agent/memory.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Statistiques
|
||||
|
||||
### **Avant le Nettoyage**
|
||||
- Fichiers dans agent/ : ~15
|
||||
- Couches de compatibilité : 3 (api, models, memory)
|
||||
- Architecture : Hybride
|
||||
|
||||
### **Après le Nettoyage**
|
||||
- Fichiers dans agent/ : ~8
|
||||
- Couches de compatibilité : 0
|
||||
- Architecture : 100% DDD
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Résultat
|
||||
|
||||
### **Architecture Propre** ✅
|
||||
Plus aucune couche de compatibilité
|
||||
|
||||
### **Imports Directs** ✅
|
||||
Tous les imports pointent vers la nouvelle architecture
|
||||
|
||||
### **DDD Pur** ✅
|
||||
Respect strict des principes Domain-Driven Development
|
||||
|
||||
### **Maintenable** ✅
|
||||
Code clair, organisé, facile à comprendre
|
||||
|
||||
### **Évolutif** ✅
|
||||
Facile d'ajouter de nouvelles fonctionnalités
|
||||
|
||||
---
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
- `DDD_PHASE1_COMPLETE.md` - Phase 1 (Domain + Infrastructure)
|
||||
- `DDD_PHASE2_COMPLETE.md` - Phase 2 (Application + Agent)
|
||||
- `DDD_MIGRATION_COMPLETE.md` - Récapitulatif complet
|
||||
- `ARCHITECTURE_FINALE.md` - Ce fichier (architecture finale)
|
||||
- `DELETED_FILES.md` - Liste des fichiers supprimés
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Prochaines Étapes
|
||||
|
||||
1. **Tester l'application** : `uvicorn app:app --reload`
|
||||
2. **Vérifier que tout fonctionne**
|
||||
3. **Commencer à utiliser la nouvelle architecture**
|
||||
4. **Ajouter de nouveaux use cases si nécessaire**
|
||||
|
||||
---
|
||||
|
||||
## 🏆 Mission Accomplie
|
||||
|
||||
L'architecture est maintenant **100% Domain-Driven Development** !
|
||||
|
||||
✅ Aucune couche de compatibilité
|
||||
✅ Imports directs vers la nouvelle architecture
|
||||
✅ Code propre et maintenable
|
||||
✅ Prêt pour l'avenir
|
||||
|
||||
🎉 **Félicitations !** 🎉
|
||||
516
CHANGELOG.md
516
CHANGELOG.md
@@ -1,516 +0,0 @@
|
||||
# Changelog
|
||||
|
||||
## [Non publié] - 2024-01-XX
|
||||
|
||||
### 🎯 Objectif principal
|
||||
Correction massive des dépendances circulaires et refactoring complet du système pour utiliser les tool calls natifs OpenAI. Migration de l'architecture vers un système plus propre et maintenable.
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Corrections majeures
|
||||
|
||||
### 1. Agent Core (`agent/agent.py`)
|
||||
**Refactoring complet du système d'agent**
|
||||
|
||||
- **Suppression du système JSON custom** :
|
||||
- Retiré `_parse_intent()` qui parsait du JSON custom
|
||||
- Retiré `_execute_action()` remplacé par `_execute_tool_call()`
|
||||
- Migration vers les tool calls natifs OpenAI
|
||||
|
||||
- **Nouvelle interface LLM** :
|
||||
- Ajout du `Protocol` `LLMClient` pour typage fort
|
||||
- `complete()` retourne `Dict[str, Any]` (message avec tool_calls)
|
||||
- `complete_stream()` retourne `AsyncGenerator` pour streaming
|
||||
- Suppression du tuple `(response, usage)` - plus de comptage de tokens
|
||||
|
||||
- **Gestion des tool calls** :
|
||||
- `_execute_tool_call()` parse les tool calls OpenAI
|
||||
- Gestion des `tool_call_id` pour la conversation
|
||||
- Boucle d'itération jusqu'à réponse finale ou max iterations
|
||||
- Raise `MaxIterationsReachedError` si dépassement
|
||||
|
||||
- **Streaming asynchrone** :
|
||||
- `step_stream()` pour réponses streamées
|
||||
- Détection des tool calls avant streaming
|
||||
- Fallback non-streaming si tool calls nécessaires
|
||||
- Sauvegarde de la réponse complète en mémoire
|
||||
|
||||
- **Gestion de la mémoire** :
|
||||
- Utilisation de `get_memory()` au lieu de passer `memory` partout
|
||||
- `_prepare_messages()` pour construire le contexte
|
||||
- Sauvegarde automatique après chaque step
|
||||
- Ajout des messages user/assistant dans l'historique
|
||||
|
||||
### 2. LLM Clients
|
||||
|
||||
#### `agent/llm/deepseek.py`
|
||||
- **Nouvelle signature** : `complete(messages, tools=None) -> Dict[str, Any]`
|
||||
- **Streaming** : `complete_stream()` avec `httpx.AsyncClient`
|
||||
- **Support des tool calls** : Ajout de `tools` et `tool_choice` dans le payload
|
||||
- **Retour simplifié** : Retourne directement le message, pas de tuple
|
||||
- **Gestion d'erreurs** : Raise `LLMAPIError` pour toutes les erreurs
|
||||
|
||||
#### `agent/llm/ollama.py`
|
||||
- Même refactoring que DeepSeek
|
||||
- Support des tool calls (si Ollama le supporte)
|
||||
- Streaming avec `httpx.AsyncClient`
|
||||
|
||||
#### `agent/llm/exceptions.py` (NOUVEAU)
|
||||
- `LLMError` - Exception de base
|
||||
- `LLMConfigurationError` - Configuration invalide
|
||||
- `LLMAPIError` - Erreur API
|
||||
|
||||
### 3. Prompts (`agent/prompts.py`)
|
||||
|
||||
**Simplification massive du système de prompts**
|
||||
|
||||
- **Suppression du prompt verbeux** :
|
||||
- Plus de JSON context énorme
|
||||
- Plus de liste exhaustive des outils
|
||||
- Plus d'exemples JSON
|
||||
|
||||
- **Nouveau prompt court** :
|
||||
```
|
||||
You are a helpful AI assistant for managing a media library.
|
||||
Your first task is to determine the user's language...
|
||||
```
|
||||
|
||||
- **Contexte structuré** :
|
||||
- `_format_episodic_context()` : Dernières recherches, downloads, erreurs
|
||||
- `_format_stm_context()` : Topic actuel, langue de conversation
|
||||
- Affichage limité (5 résultats, 3 downloads, 3 erreurs)
|
||||
|
||||
- **Tool specs OpenAI** :
|
||||
- `build_tools_spec()` génère le format OpenAI
|
||||
- Les tools sont passés via l'API, pas dans le prompt
|
||||
|
||||
### 4. Registry (`agent/registry.py`)
|
||||
|
||||
**Correction des dépendances circulaires**
|
||||
|
||||
- **Nouveau système d'enregistrement** :
|
||||
- Décorateur `@tool` pour auto-enregistrement
|
||||
- Liste globale `_tools` pour stocker les tools
|
||||
- `make_tools()` appelle explicitement chaque fonction
|
||||
|
||||
- **Suppression des imports directs** :
|
||||
- Plus d'imports dans `agent/tools/__init__.py`
|
||||
- Imports dans `registry.py` au moment de l'enregistrement
|
||||
- Évite les boucles d'imports
|
||||
|
||||
- **Génération automatique des schemas** :
|
||||
- Inspection des signatures avec `inspect`
|
||||
- Génération des `parameters` JSON Schema
|
||||
- Extraction de la description depuis la docstring
|
||||
|
||||
### 5. Tools
|
||||
|
||||
#### `agent/tools/__init__.py`
|
||||
- **Vidé complètement** pour éviter les imports circulaires
|
||||
- Juste `__all__` pour la documentation
|
||||
|
||||
#### `agent/tools/api.py`
|
||||
**Refactoring complet avec gestion de la mémoire**
|
||||
|
||||
- **`find_media_imdb_id()`** :
|
||||
- Stocke le résultat dans `memory.stm.set_entity("last_media_search")`
|
||||
- Set topic à "searching_media"
|
||||
- Logging des résultats
|
||||
|
||||
- **`find_torrent()`** :
|
||||
- Stocke les résultats dans `memory.episodic.store_search_results()`
|
||||
- Set topic à "selecting_torrent"
|
||||
- Permet la référence par index
|
||||
|
||||
- **`get_torrent_by_index()` (NOUVEAU)** :
|
||||
- Récupère un torrent par son index dans les résultats
|
||||
- Utilisé pour "télécharge le 3ème"
|
||||
|
||||
- **`add_torrent_by_index()` (NOUVEAU)** :
|
||||
- Combine `get_torrent_by_index()` + `add_torrent_to_qbittorrent()`
|
||||
- Workflow simplifié
|
||||
|
||||
- **`add_torrent_to_qbittorrent()`** :
|
||||
- Ajoute le download dans `memory.episodic.add_active_download()`
|
||||
- Set topic à "downloading"
|
||||
- End workflow
|
||||
|
||||
#### `agent/tools/filesystem.py`
|
||||
- **Suppression du paramètre `memory`** :
|
||||
- `set_path_for_folder(folder_name, path_value)`
|
||||
- `list_folder(folder_type, path=".")`
|
||||
- Utilise `get_memory()` en interne via `FileManager`
|
||||
|
||||
#### `agent/tools/language.py` (NOUVEAU)
|
||||
- **`set_language(language_code)`** :
|
||||
- Définit la langue de conversation
|
||||
- Stocke dans `memory.stm.set_language()`
|
||||
- Permet au LLM de détecter et changer la langue
|
||||
|
||||
### 6. Exceptions (`agent/exceptions.py`)
|
||||
|
||||
**Nouvelles exceptions spécifiques**
|
||||
|
||||
- `AgentError` - Exception de base
|
||||
- `ToolExecutionError(tool_name, message)` - Échec d'exécution d'un tool
|
||||
- `MaxIterationsReachedError(max_iterations)` - Trop d'itérations
|
||||
|
||||
### 7. Config (`agent/config.py`)
|
||||
|
||||
**Amélioration de la validation**
|
||||
|
||||
- Validation stricte des valeurs (temperature, timeouts, etc.)
|
||||
- Messages d'erreur plus clairs
|
||||
- Docstrings complètes
|
||||
- Formatage avec Black
|
||||
|
||||
---
|
||||
|
||||
## 🌐 API (`app.py`)
|
||||
|
||||
### Refactoring complet
|
||||
|
||||
**Avant** : API simple avec un seul endpoint
|
||||
**Après** : API complète OpenAI-compatible avec gestion d'erreurs
|
||||
|
||||
### Nouveaux endpoints
|
||||
|
||||
1. **`GET /health`**
|
||||
- Health check avec version et service name
|
||||
- Retourne `{"status": "healthy", "version": "0.2.0", "service": "agent-media"}`
|
||||
|
||||
2. **`GET /v1/models`**
|
||||
- Liste des modèles disponibles (OpenAI-compatible)
|
||||
- Retourne format OpenAI avec `object: "list"`, `data: [...]`
|
||||
|
||||
3. **`GET /api/memory/state`**
|
||||
- État complet de la mémoire (LTM + STM + Episodic)
|
||||
- Pour debugging et monitoring
|
||||
|
||||
4. **`GET /api/memory/search-results`**
|
||||
- Derniers résultats de recherche
|
||||
- Permet de voir ce que l'agent a trouvé
|
||||
|
||||
5. **`POST /api/memory/clear`**
|
||||
- Efface la session (STM + Episodic)
|
||||
- Préserve la LTM (config, bibliothèque)
|
||||
|
||||
### Validation des messages
|
||||
|
||||
**Nouvelle fonction `validate_messages()`** :
|
||||
- Vérifie qu'il y a au moins un message user
|
||||
- Vérifie que le contenu n'est pas vide
|
||||
- Raise `HTTPException(422)` si invalide
|
||||
- Appelée avant chaque requête
|
||||
|
||||
### Gestion d'erreurs HTTP
|
||||
|
||||
**Codes d'erreur spécifiques** :
|
||||
- **504 Gateway Timeout** : `MaxIterationsReachedError` (agent bloqué en boucle)
|
||||
- **400 Bad Request** : `ToolExecutionError` (tool mal appelé)
|
||||
- **502 Bad Gateway** : `LLMAPIError` (API LLM down)
|
||||
- **500 Internal Server Error** : `AgentError` (erreur interne)
|
||||
- **422 Unprocessable Entity** : Validation des messages
|
||||
|
||||
### Streaming
|
||||
|
||||
**Amélioration du streaming** :
|
||||
- Utilise `agent.step_stream()` pour vraies réponses streamées
|
||||
- Gestion correcte des chunks
|
||||
- Envoi de `[DONE]` à la fin
|
||||
- Gestion d'erreurs dans le stream
|
||||
|
||||
---
|
||||
|
||||
## 🧠 Infrastructure
|
||||
|
||||
### Persistence (`infrastructure/persistence/`)
|
||||
|
||||
#### `memory.py`
|
||||
**Nouvelles méthodes** :
|
||||
- `get_full_state()` - Retourne tout l'état de la mémoire
|
||||
- `clear_session()` - Efface STM + Episodic, garde LTM
|
||||
|
||||
#### `context.py`
|
||||
**Singleton global** :
|
||||
- `init_memory(storage_dir)` - Initialise la mémoire
|
||||
- `get_memory()` - Récupère l'instance globale
|
||||
- `set_memory(memory)` - Définit l'instance (pour tests)
|
||||
|
||||
### Filesystem (`infrastructure/filesystem/`)
|
||||
|
||||
#### `file_manager.py`
|
||||
- **Suppression du paramètre `memory`** du constructeur
|
||||
- Utilise `get_memory()` en interne
|
||||
- Simplifie l'utilisation
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Tests
|
||||
|
||||
### Fixtures (`tests/conftest.py`)
|
||||
|
||||
**Mise à jour complète des mocks** :
|
||||
|
||||
1. **`MockLLMClient`** :
|
||||
- `complete()` retourne `Dict[str, Any]` (pas de tuple)
|
||||
- `complete_stream()` async generator
|
||||
- `set_next_response()` pour configurer les réponses
|
||||
|
||||
2. **`MockDeepSeekClient` global** :
|
||||
- Ajout de `complete_stream()` async
|
||||
- Évite les appels API réels dans tous les tests
|
||||
|
||||
3. **Nouvelles fixtures** :
|
||||
- `mock_agent_step` - Pour mocker `agent.step()`
|
||||
- Fixtures existantes mises à jour
|
||||
|
||||
### Tests corrigés
|
||||
|
||||
#### `test_agent.py`
|
||||
- **`MockLLMClient`** adapté pour nouvelle interface
|
||||
- **`test_step_stream`** : Double réponse mockée (check + stream)
|
||||
- **`test_max_iterations_reached`** : Arguments valides pour `set_language`
|
||||
- Suppression de tous les asserts sur `usage`
|
||||
|
||||
#### `test_api.py`
|
||||
- **Import corrigé** : `from agent.llm.exceptions import LLMAPIError`
|
||||
- **Variable `data`** ajoutée dans `test_list_models`
|
||||
- **Test streaming** : Utilisation de `side_effect` au lieu de `return_value`
|
||||
- Nouveaux tests pour `/health` et `/v1/models`
|
||||
|
||||
#### `test_prompts.py`
|
||||
- Tests adaptés au nouveau format de prompt court
|
||||
- Vérification de `CONVERSATION LANGUAGE` au lieu de texte long
|
||||
- Tests de `build_tools_spec()` pour format OpenAI
|
||||
|
||||
#### `test_prompts_edge_cases.py`
|
||||
- **Réécriture complète** pour nouveau prompt
|
||||
- Tests de `_format_episodic_context()`
|
||||
- Tests de `_format_stm_context()`
|
||||
- Suppression des tests sur sections obsolètes
|
||||
|
||||
#### `test_registry_edge_cases.py`
|
||||
- **Nom d'outil corrigé** : `find_torrents` → `find_torrent`
|
||||
- Ajout de `set_language` dans la liste des tools attendus
|
||||
|
||||
#### `test_agent_edge_cases.py`
|
||||
- **Réécriture complète** pour tool calls natifs
|
||||
- Tests de `_execute_tool_call()`
|
||||
- Tests de gestion d'erreurs avec tool calls
|
||||
- Tests de mémoire avec tool calls
|
||||
|
||||
#### `test_api_edge_cases.py`
|
||||
- **Tous les chemins d'endpoints corrigés** :
|
||||
- `/memory/state` → `/api/memory/state`
|
||||
- `/memory/episodic/search-results` → `/api/memory/search-results`
|
||||
- `/memory/clear-session` → `/api/memory/clear`
|
||||
- Tests de validation des messages
|
||||
- Tests des nouveaux endpoints
|
||||
|
||||
### Configuration pytest (`pyproject.toml`)
|
||||
|
||||
**Migration complète de `pytest.ini` vers `pyproject.toml`**
|
||||
|
||||
#### Options de coverage ajoutées :
|
||||
```toml
|
||||
"--cov=.", # Coverage de tout le projet
|
||||
"--cov-report=term-missing", # Lignes manquantes dans terminal
|
||||
"--cov-report=html", # Rapport HTML dans htmlcov/
|
||||
"--cov-report=xml", # Rapport XML pour CI/CD
|
||||
"--cov-fail-under=80", # Échoue si < 80%
|
||||
```
|
||||
|
||||
#### Options de performance :
|
||||
```toml
|
||||
"-n=auto", # Parallélisation automatique
|
||||
"--strict-markers", # Validation des markers
|
||||
"--disable-warnings", # Sortie plus propre
|
||||
```
|
||||
|
||||
#### Nouveaux markers :
|
||||
- `slow` - Tests lents
|
||||
- `integration` - Tests d'intégration
|
||||
- `unit` - Tests unitaires
|
||||
|
||||
#### Configuration coverage :
|
||||
```toml
|
||||
[tool.coverage.run]
|
||||
source = ["agent", "application", "domain", "infrastructure"]
|
||||
omit = ["tests/*", "*/__pycache__/*"]
|
||||
|
||||
[tool.coverage.report]
|
||||
exclude_lines = ["pragma: no cover", "def __repr__", ...]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Documentation
|
||||
|
||||
### Nouveaux fichiers
|
||||
|
||||
1. **`README.md`** (412 lignes)
|
||||
- Documentation complète du projet
|
||||
- Quick start, installation, usage
|
||||
- Exemples de conversations
|
||||
- Liste des tools disponibles
|
||||
- Architecture et structure
|
||||
- Guide de développement
|
||||
- Docker et CI/CD
|
||||
- API documentation
|
||||
- Troubleshooting
|
||||
|
||||
2. **`docs/PYTEST_CONFIG.md`**
|
||||
- Explication ligne par ligne de chaque option pytest
|
||||
- Guide des commandes utiles
|
||||
- Bonnes pratiques
|
||||
- Troubleshooting
|
||||
|
||||
3. **`TESTS_TO_FIX.md`**
|
||||
- Liste des tests à corriger (maintenant obsolète)
|
||||
- Recommandations pour l'approche complète
|
||||
|
||||
4. **`.pytest.ini.backup`**
|
||||
- Sauvegarde de l'ancien `pytest.ini`
|
||||
|
||||
### Fichiers mis à jour
|
||||
|
||||
1. **`.env`**
|
||||
- Ajout de commentaires pour chaque section
|
||||
- Nouvelles variables :
|
||||
- `LLM_PROVIDER` - Choix entre deepseek/ollama
|
||||
- `OLLAMA_BASE_URL`, `OLLAMA_MODEL`
|
||||
- `MAX_TOOL_ITERATIONS`
|
||||
- `MAX_HISTORY_MESSAGES`
|
||||
- Organisation par catégories
|
||||
|
||||
2. **`.gitignore`**
|
||||
- Ajout des fichiers de coverage :
|
||||
- `.coverage`, `.coverage.*`
|
||||
- `htmlcov/`, `coverage.xml`
|
||||
- Ajout de `.pytest_cache/`
|
||||
- Ajout de `memory_data/`
|
||||
- Ajout de `*.backup`
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Refactoring général
|
||||
|
||||
### Architecture
|
||||
- **Séparation des responsabilités** plus claire
|
||||
- **Dépendances circulaires** éliminées
|
||||
- **Injection de dépendances** via `get_memory()`
|
||||
- **Typage fort** avec `Protocol` et type hints
|
||||
|
||||
### Code quality
|
||||
- **Formatage** avec Black (line-length=88)
|
||||
- **Linting** avec Ruff
|
||||
- **Docstrings** complètes partout
|
||||
- **Logging** ajouté dans les tools
|
||||
|
||||
### Performance
|
||||
- **Parallélisation** des tests avec pytest-xdist
|
||||
- **Streaming** asynchrone pour réponses rapides
|
||||
- **Mémoire** optimisée (limitation des résultats affichés)
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Bugs corrigés
|
||||
|
||||
1. **Dépendances circulaires** :
|
||||
- `agent/tools/__init__.py` ↔ `agent/registry.py`
|
||||
- Solution : Imports dans `registry.py` uniquement
|
||||
|
||||
2. **Import manquant** :
|
||||
- `LLMAPIError` dans `test_api.py`
|
||||
- Solution : `from agent.llm.exceptions import LLMAPIError`
|
||||
|
||||
3. **Mock streaming** :
|
||||
- `test_step_stream` avec liste vide
|
||||
- Solution : Double réponse mockée (check + stream)
|
||||
|
||||
4. **Mock async generator** :
|
||||
- `return_value` au lieu de `side_effect`
|
||||
- Solution : `side_effect=mock_stream_generator`
|
||||
|
||||
5. **Nom d'outil** :
|
||||
- `find_torrents` vs `find_torrent`
|
||||
- Solution : Uniformisation sur `find_torrent`
|
||||
|
||||
6. **Validation messages** :
|
||||
- Endpoints acceptaient messages vides
|
||||
- Solution : `validate_messages()` avec HTTPException
|
||||
|
||||
7. **Décorateur mal placé** :
|
||||
- `@tool` dans `language.py` causait import circulaire
|
||||
- Solution : Suppression, enregistrement dans `registry.py`
|
||||
|
||||
8. **Imports manquants** :
|
||||
- `from typing import Dict, Any` dans plusieurs fichiers
|
||||
- Solution : Ajout des imports
|
||||
|
||||
---
|
||||
|
||||
## 📊 Métriques
|
||||
|
||||
### Avant
|
||||
- Tests : ~450 (beaucoup échouaient)
|
||||
- Coverage : Non mesuré
|
||||
- Endpoints : 1 (`/v1/chat/completions`)
|
||||
- Tools : 5
|
||||
- Dépendances circulaires : Oui
|
||||
- Système de prompts : Verbeux et complexe
|
||||
|
||||
### Après
|
||||
- Tests : ~500 (tous passent ✅)
|
||||
- Coverage : Configuré avec objectif 80%
|
||||
- Endpoints : 6 (5 nouveaux)
|
||||
- Tools : 8 (3 nouveaux)
|
||||
- Dépendances circulaires : Non ✅
|
||||
- Système de prompts : Simple et efficace
|
||||
|
||||
### Changements de code
|
||||
- **Fichiers modifiés** : ~30
|
||||
- **Lignes ajoutées** : ~2000
|
||||
- **Lignes supprimées** : ~1500
|
||||
- **Net** : +500 lignes (documentation comprise)
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Améliorations futures
|
||||
|
||||
### Court terme
|
||||
- [ ] Atteindre 100% de coverage
|
||||
- [ ] Tests d'intégration end-to-end
|
||||
- [ ] Benchmarks de performance
|
||||
|
||||
### Moyen terme
|
||||
- [ ] Support de plus de LLM providers
|
||||
- [ ] Interface web (OpenWebUI)
|
||||
- [ ] Métriques et monitoring
|
||||
|
||||
### Long terme
|
||||
- [ ] Multi-utilisateurs
|
||||
- [ ] Plugins système
|
||||
- [ ] API GraphQL
|
||||
|
||||
---
|
||||
|
||||
## 🙏 Notes
|
||||
|
||||
**Problème initial** : Gemini 3 Pro a introduit des dépendances circulaires et supprimé du code critique, rendant l'application non fonctionnelle.
|
||||
|
||||
**Solution** : Refactoring complet du système avec :
|
||||
- Migration vers tool calls natifs OpenAI
|
||||
- Élimination des dépendances circulaires
|
||||
- Simplification du système de prompts
|
||||
- Ajout de tests et documentation
|
||||
- Configuration pytest professionnelle
|
||||
|
||||
**Résultat** : Application stable, testée, documentée et prête pour la production ! 🎉
|
||||
|
||||
---
|
||||
|
||||
**Auteur** : Claude (avec l'aide de Francwa)
|
||||
**Date** : Janvier 2024
|
||||
**Version** : 0.2.0
|
||||
28
README.md
28
README.md
@@ -23,7 +23,7 @@ agent_media/
|
||||
└── infrastructure/ # External services & persistence
|
||||
```
|
||||
|
||||
See [ARCHITECTURE_FINALE.md](ARCHITECTURE_FINALE.md) for details.
|
||||
See [architecture_diagram.md](docs/architecture_diagram.md) for architectural details.
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -223,8 +223,6 @@ poetry run mypy .
|
||||
|
||||
### Adding a New Tool
|
||||
|
||||
See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for detailed instructions.
|
||||
|
||||
Quick example:
|
||||
|
||||
```python
|
||||
@@ -281,16 +279,6 @@ docker-compose logs -f
|
||||
docker-compose down
|
||||
```
|
||||
|
||||
## CI/CD
|
||||
|
||||
Includes Gitea Actions workflow for:
|
||||
- ✅ Linting & testing
|
||||
- 🐳 Docker image building
|
||||
- 📦 Container registry push
|
||||
- 🚀 Deployment (optional)
|
||||
|
||||
See [docs/CI_CD_GUIDE.md](docs/CI_CD_GUIDE.md) for setup instructions.
|
||||
|
||||
## API Documentation
|
||||
|
||||
### Endpoints
|
||||
@@ -364,12 +352,12 @@ Clear session memories (STM + Episodic).
|
||||
- Verify volume mounts in Docker
|
||||
|
||||
### Tests failing
|
||||
- See [docs/TEST_FAILURES_SUMMARY.md](docs/TEST_FAILURES_SUMMARY.md)
|
||||
- Run `poetry install` to ensure dependencies are up to date
|
||||
- Check logs for specific error messages
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! Please read [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) first.
|
||||
Contributions are welcome!
|
||||
|
||||
### Development Workflow
|
||||
|
||||
@@ -384,11 +372,11 @@ Contributions are welcome! Please read [docs/CONTRIBUTING.md](docs/CONTRIBUTING.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Architecture](ARCHITECTURE_FINALE.md) - System architecture
|
||||
- [Contributing Guide](docs/CONTRIBUTING.md) - How to contribute
|
||||
- [CI/CD Guide](docs/CI_CD_GUIDE.md) - Pipeline setup
|
||||
- [Flowcharts](docs/flowchart.md) - System flowcharts
|
||||
- [Test Failures](docs/TEST_FAILURES_SUMMARY.md) - Known test issues
|
||||
- [Architecture Diagram](docs/architecture_diagram.md) - System architecture overview
|
||||
- [Class Diagram](docs/class_diagram.md) - Class structure and relationships
|
||||
- [Component Diagram](docs/component_diagram.md) - Component interactions
|
||||
- [Sequence Diagram](docs/sequence_diagram.md) - Sequence flows
|
||||
- [Flowchart](docs/flowchart.md) - System flowcharts
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Script de nettoyage des fichiers obsolètes après migration DDD
|
||||
|
||||
echo "🗑️ Nettoyage des fichiers obsolètes..."
|
||||
|
||||
# Supprimer les anciens clients API (déplacés vers infrastructure/)
|
||||
echo "Suppression des anciens clients API..."
|
||||
rm -f agent/api/themoviedb.py
|
||||
rm -f agent/api/knaben.py
|
||||
rm -f agent/api/qbittorrent.py
|
||||
|
||||
echo "✅ Anciens clients API supprimés"
|
||||
|
||||
# Optionnel : Supprimer l'ancienne documentation
|
||||
read -p "Voulez-vous supprimer l'ancienne documentation ? (y/n) " -n 1 -r
|
||||
echo
|
||||
if [[ $REPLY =~ ^[Yy]$ ]]
|
||||
then
|
||||
echo "Suppression de l'ancienne documentation..."
|
||||
rm -f CHANGELOG_QUALITY.md
|
||||
rm -f CLEANUP_FINAL.md
|
||||
rm -f CLEANUP_SUMMARY.md
|
||||
rm -f CODE_QUALITY.md
|
||||
rm -f COMMANDS_REMOVAL.md
|
||||
rm -f DEPENDENCY_INJECTION.md
|
||||
rm -f DOCUMENTATION_INDEX.md
|
||||
rm -f EXECUTIVE_SUMMARY.md
|
||||
rm -f FILES_CHANGED.md
|
||||
rm -f IMPROVEMENTS_SUMMARY.md
|
||||
rm -f KNABEN_CLIENT.md
|
||||
rm -f MIGRATION_GUIDE.md
|
||||
rm -f MULTI_TOOL_EXECUTION.md
|
||||
rm -f PARAMETERS.md
|
||||
rm -f PROJECT_STRUCTURE.md
|
||||
rm -f QUALITY_REVIEW_COMPLETE.md
|
||||
rm -f README_QUALITY.md
|
||||
rm -f REFACTORING_COMPLETE.md
|
||||
rm -f REFACTORING_FINAL.md
|
||||
rm -f REFACTORING_FOLDERS.md
|
||||
rm -f REFACTORING_SUMMARY.md
|
||||
rm -f SECURITY.md
|
||||
rm -f TMDB_CLIENT_ARCHITECTURE.md
|
||||
rm -f TMDB_CLIENT_SUMMARY.md
|
||||
rm -f TOOLS_REFACTORING.md
|
||||
rm -f TV_SHOWS.md
|
||||
rm -f VERIFICATION.md
|
||||
echo "✅ Ancienne documentation supprimée"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "🎉 Nettoyage terminé !"
|
||||
echo ""
|
||||
echo "📋 Fichiers conservés (nécessaires) :"
|
||||
echo " - agent/api/__init__.py (re-exporte infrastructure)"
|
||||
echo " - agent/models/__init__.py (re-exporte domain)"
|
||||
echo " - agent/models/tv_show.py (compatibilité)"
|
||||
echo " - agent/memory.py (re-exporte infrastructure)"
|
||||
echo ""
|
||||
echo "📚 Nouvelle documentation DDD :"
|
||||
echo " - DDD_PHASE1_COMPLETE.md"
|
||||
echo " - DDD_PHASE2_COMPLETE.md"
|
||||
echo " - DDD_MIGRATION_COMPLETE.md"
|
||||
402
docs/architecture_diagram.md
Normal file
402
docs/architecture_diagram.md
Normal file
@@ -0,0 +1,402 @@
|
||||
# Architecture Diagram - Agent Media
|
||||
|
||||
## System Overview
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Client["👤 Client"]
|
||||
CHAT[Chat Interface<br/>OpenWebUI / CLI / Custom]
|
||||
end
|
||||
|
||||
subgraph AgentMedia["🎬 Agent Media"]
|
||||
subgraph API["API Layer"]
|
||||
FASTAPI[FastAPI Server<br/>:8000]
|
||||
end
|
||||
|
||||
subgraph Core["Core"]
|
||||
AGENT[🤖 Agent<br/>Orchestrator]
|
||||
MEMORY[🧠 Memory<br/>LTM + STM + Episodic]
|
||||
end
|
||||
|
||||
subgraph Tools["Tools"]
|
||||
T1[📁 Filesystem]
|
||||
T2[🔍 Search]
|
||||
T3[⬇️ Download]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph LLM["🧠 LLM Provider"]
|
||||
DEEPSEEK[DeepSeek API]
|
||||
OLLAMA[Ollama<br/>Local]
|
||||
end
|
||||
|
||||
subgraph External["☁️ External Services"]
|
||||
TMDB[(TMDB<br/>Movie Database)]
|
||||
KNABEN[(Knaben<br/>Torrent Search)]
|
||||
QBIT[qBittorrent<br/>Download Client]
|
||||
end
|
||||
|
||||
subgraph Storage["💾 Storage"]
|
||||
JSON[(memory_data/<br/>ltm.json)]
|
||||
MEDIA[(Media Folders<br/>/movies /tvshows)]
|
||||
end
|
||||
|
||||
CHAT <-->|OpenAI API| FASTAPI
|
||||
FASTAPI <--> AGENT
|
||||
AGENT <--> MEMORY
|
||||
AGENT <--> Tools
|
||||
AGENT <-->|Chat Completion| LLM
|
||||
|
||||
T1 <--> MEDIA
|
||||
T2 --> TMDB
|
||||
T2 --> KNABEN
|
||||
T3 --> QBIT
|
||||
|
||||
MEMORY <--> JSON
|
||||
QBIT --> MEDIA
|
||||
|
||||
style AgentMedia fill:#1a1a2e,color:#fff
|
||||
style AGENT fill:#ff6b6b,color:#fff
|
||||
style MEMORY fill:#4ecdc4,color:#fff
|
||||
```
|
||||
|
||||
## Detailed Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Clients["Clients"]
|
||||
direction LR
|
||||
OWU[OpenWebUI]
|
||||
CLI[CLI Client]
|
||||
CURL[cURL / HTTP]
|
||||
end
|
||||
|
||||
subgraph LoadBalancer["Entry Point"]
|
||||
NGINX[Nginx / Reverse Proxy<br/>Optional]
|
||||
end
|
||||
|
||||
subgraph Application["Agent Media Application"]
|
||||
direction TB
|
||||
|
||||
subgraph Presentation["Presentation Layer"]
|
||||
EP1["/v1/chat/completions"]
|
||||
EP2["/v1/models"]
|
||||
EP3["/health"]
|
||||
EP4["/memory/state"]
|
||||
end
|
||||
|
||||
subgraph AgentLayer["Agent Layer"]
|
||||
direction LR
|
||||
AG[Agent]
|
||||
PB[PromptBuilder]
|
||||
REG[Registry]
|
||||
end
|
||||
|
||||
subgraph ToolsLayer["Tools Layer"]
|
||||
direction LR
|
||||
FS_TOOL[Filesystem Tools<br/>set_path, list_folder]
|
||||
API_TOOL[API Tools<br/>find_torrent, add_torrent]
|
||||
end
|
||||
|
||||
subgraph AppLayer["Application Layer"]
|
||||
direction LR
|
||||
UC1[SearchMovie<br/>UseCase]
|
||||
UC2[SearchTorrents<br/>UseCase]
|
||||
UC3[AddTorrent<br/>UseCase]
|
||||
UC4[SetFolderPath<br/>UseCase]
|
||||
end
|
||||
|
||||
subgraph DomainLayer["Domain Layer"]
|
||||
direction LR
|
||||
ENT[Entities<br/>Movie, TVShow, Subtitle]
|
||||
VO[Value Objects<br/>ImdbId, Quality, FilePath]
|
||||
REPO_INT[Repository<br/>Interfaces]
|
||||
end
|
||||
|
||||
subgraph InfraLayer["Infrastructure Layer"]
|
||||
direction TB
|
||||
|
||||
subgraph Persistence["Persistence"]
|
||||
MEM[Memory Manager]
|
||||
REPO_IMPL[JSON Repositories]
|
||||
end
|
||||
|
||||
subgraph APIClients["API Clients"]
|
||||
TMDB_C[TMDB Client]
|
||||
KNAB_C[Knaben Client]
|
||||
QBIT_C[qBittorrent Client]
|
||||
end
|
||||
|
||||
subgraph FSManager["Filesystem"]
|
||||
FM[FileManager]
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
subgraph LLMProviders["LLM Providers"]
|
||||
direction LR
|
||||
DS[DeepSeek<br/>api.deepseek.com]
|
||||
OL[Ollama<br/>localhost:11434]
|
||||
end
|
||||
|
||||
subgraph ExternalAPIs["External APIs"]
|
||||
direction LR
|
||||
TMDB_API[TMDB API<br/>api.themoviedb.org]
|
||||
KNAB_API[Knaben API<br/>knaben.eu]
|
||||
QBIT_API[qBittorrent WebUI<br/>localhost:8080]
|
||||
end
|
||||
|
||||
subgraph DataStores["Data Stores"]
|
||||
direction LR
|
||||
LTM_FILE[(ltm.json<br/>Persistent Config)]
|
||||
MEDIA_DIR[(Media Directories<br/>/downloads /movies /tvshows)]
|
||||
end
|
||||
|
||||
%% Client connections
|
||||
Clients --> LoadBalancer
|
||||
LoadBalancer --> Presentation
|
||||
|
||||
%% Internal flow
|
||||
Presentation --> AgentLayer
|
||||
AgentLayer --> ToolsLayer
|
||||
ToolsLayer --> AppLayer
|
||||
AppLayer --> DomainLayer
|
||||
AppLayer --> InfraLayer
|
||||
InfraLayer -.->|implements| DomainLayer
|
||||
|
||||
%% Agent to LLM
|
||||
AgentLayer <-->|HTTP| LLMProviders
|
||||
|
||||
%% Infrastructure to External
|
||||
TMDB_C -->|HTTP| TMDB_API
|
||||
KNAB_C -->|HTTP| KNAB_API
|
||||
QBIT_C -->|HTTP| QBIT_API
|
||||
|
||||
%% Persistence
|
||||
MEM <--> LTM_FILE
|
||||
FM <--> MEDIA_DIR
|
||||
QBIT_API --> MEDIA_DIR
|
||||
```
|
||||
|
||||
## Memory System Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph MemoryManager["Memory Manager"]
|
||||
direction TB
|
||||
|
||||
subgraph LTM["💾 Long-Term Memory"]
|
||||
direction LR
|
||||
LTM_DESC["Persistent across restarts<br/>Stored in ltm.json"]
|
||||
|
||||
subgraph LTM_DATA["Data"]
|
||||
CONFIG["config{}<br/>folder paths, API keys"]
|
||||
PREFS["preferences{}<br/>quality, languages"]
|
||||
LIBRARY["library{}<br/>movies[], tv_shows[]"]
|
||||
FOLLOWING["following[]<br/>watchlist"]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph STM["🧠 Short-Term Memory"]
|
||||
direction LR
|
||||
STM_DESC["Session-based<br/>Cleared on restart"]
|
||||
|
||||
subgraph STM_DATA["Data"]
|
||||
HISTORY["conversation_history[]<br/>last 20 messages"]
|
||||
WORKFLOW["current_workflow{}<br/>type, stage, target"]
|
||||
ENTITIES["extracted_entities{}<br/>title, year, quality"]
|
||||
TOPIC["current_topic<br/>searching, downloading"]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph EPISODIC["⚡ Episodic Memory"]
|
||||
direction LR
|
||||
EPIS_DESC["Transient state<br/>Cleared on restart"]
|
||||
|
||||
subgraph EPIS_DATA["Data"]
|
||||
SEARCH["last_search_results{}<br/>indexed torrents"]
|
||||
DOWNLOADS["active_downloads[]<br/>in-progress"]
|
||||
ERRORS["recent_errors[]<br/>last 5 errors"]
|
||||
PENDING["pending_question{}<br/>awaiting user input"]
|
||||
EVENTS["background_events[]<br/>notifications"]
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Storage["Storage"]
|
||||
FILE[(memory_data/ltm.json)]
|
||||
end
|
||||
|
||||
subgraph Lifecycle["Lifecycle"]
|
||||
SAVE[save()]
|
||||
LOAD[load()]
|
||||
CLEAR[clear_session()]
|
||||
end
|
||||
|
||||
LTM <-->|read/write| FILE
|
||||
SAVE --> LTM
|
||||
LOAD --> LTM
|
||||
CLEAR --> STM
|
||||
CLEAR --> EPISODIC
|
||||
|
||||
style LTM fill:#4caf50,color:#fff
|
||||
style STM fill:#2196f3,color:#fff
|
||||
style EPISODIC fill:#ff9800,color:#fff
|
||||
```
|
||||
|
||||
## Request Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Request["1️⃣ Request"]
|
||||
USER[User Message]
|
||||
end
|
||||
|
||||
subgraph Parse["2️⃣ Parse"]
|
||||
FASTAPI[FastAPI<br/>Extract message]
|
||||
end
|
||||
|
||||
subgraph Context["3️⃣ Build Context"]
|
||||
PROMPT[PromptBuilder<br/>+ Memory context<br/>+ Tool descriptions]
|
||||
end
|
||||
|
||||
subgraph Think["4️⃣ Think"]
|
||||
LLM[LLM<br/>Decide action]
|
||||
end
|
||||
|
||||
subgraph Act["5️⃣ Act"]
|
||||
TOOL[Execute Tool<br/>or respond]
|
||||
end
|
||||
|
||||
subgraph Store["6️⃣ Store"]
|
||||
MEM[Update Memory<br/>STM + Episodic]
|
||||
end
|
||||
|
||||
subgraph Response["7️⃣ Response"]
|
||||
RESP[JSON Response]
|
||||
end
|
||||
|
||||
USER --> FASTAPI --> PROMPT --> LLM
|
||||
LLM -->|Tool call| TOOL --> MEM --> LLM
|
||||
LLM -->|Text response| MEM --> RESP
|
||||
|
||||
style Think fill:#ff6b6b,color:#fff
|
||||
style Act fill:#4ecdc4,color:#fff
|
||||
style Store fill:#45b7d1,color:#fff
|
||||
```
|
||||
|
||||
## Deployment Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Host["Host Machine"]
|
||||
subgraph Docker["Docker (Optional)"]
|
||||
AGENT_CONTAINER[Agent Media<br/>Container]
|
||||
end
|
||||
|
||||
subgraph Native["Native Services"]
|
||||
QBIT_SERVICE[qBittorrent<br/>:8080]
|
||||
OLLAMA_SERVICE[Ollama<br/>:11434]
|
||||
end
|
||||
|
||||
subgraph Storage["Local Storage"]
|
||||
CONFIG_DIR[/config<br/>memory_data/]
|
||||
MEDIA_DIR[/media<br/>downloads, movies, tvshows]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Cloud["Cloud Services"]
|
||||
DEEPSEEK[DeepSeek API]
|
||||
TMDB[TMDB API]
|
||||
KNABEN[Knaben API]
|
||||
end
|
||||
|
||||
subgraph Client["Client"]
|
||||
BROWSER[Browser<br/>OpenWebUI]
|
||||
end
|
||||
|
||||
BROWSER <-->|:8000| AGENT_CONTAINER
|
||||
AGENT_CONTAINER <-->|:8080| QBIT_SERVICE
|
||||
AGENT_CONTAINER <-->|:11434| OLLAMA_SERVICE
|
||||
AGENT_CONTAINER <--> CONFIG_DIR
|
||||
AGENT_CONTAINER <--> MEDIA_DIR
|
||||
QBIT_SERVICE --> MEDIA_DIR
|
||||
|
||||
AGENT_CONTAINER <-->|HTTPS| Cloud
|
||||
```
|
||||
|
||||
## Technology Stack
|
||||
|
||||
```mermaid
|
||||
mindmap
|
||||
root((Agent Media))
|
||||
API
|
||||
FastAPI
|
||||
Uvicorn
|
||||
OpenAI Compatible
|
||||
Agent
|
||||
Python 3.11+
|
||||
Dataclasses
|
||||
Protocol typing
|
||||
LLM
|
||||
DeepSeek
|
||||
Ollama
|
||||
OpenAI compatible
|
||||
Storage
|
||||
JSON files
|
||||
Filesystem
|
||||
External APIs
|
||||
TMDB
|
||||
Knaben
|
||||
qBittorrent WebUI
|
||||
Architecture
|
||||
DDD
|
||||
Clean Architecture
|
||||
Hexagonal
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Security["Security Layers"]
|
||||
direction TB
|
||||
|
||||
subgraph Input["Input Validation"]
|
||||
PATH_VAL[Path Traversal Protection<br/>FileManager._sanitize_path]
|
||||
INPUT_VAL[Input Sanitization<br/>Tool parameters]
|
||||
end
|
||||
|
||||
subgraph Auth["Authentication"]
|
||||
API_KEYS[API Keys<br/>Environment variables]
|
||||
QBIT_AUTH[qBittorrent Auth<br/>Username/Password]
|
||||
end
|
||||
|
||||
subgraph Access["Access Control"]
|
||||
FOLDER_RESTRICT[Folder Restrictions<br/>Configured paths only]
|
||||
SAFE_PATH[Safe Path Checks<br/>_is_safe_path()]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Env["Environment"]
|
||||
ENV_FILE[.env file<br/>DEEPSEEK_API_KEY<br/>TMDB_API_KEY<br/>QBITTORRENT_*]
|
||||
end
|
||||
|
||||
ENV_FILE --> Auth
|
||||
Input --> Access
|
||||
```
|
||||
|
||||
## Legend
|
||||
|
||||
| Icon | Meaning |
|
||||
|------|---------|
|
||||
| 🎬 | Agent Media System |
|
||||
| 🤖 | AI Agent |
|
||||
| 🧠 | Memory / LLM |
|
||||
| 💾 | Persistent Storage |
|
||||
| ⚡ | Transient / Fast |
|
||||
| 📁 | Filesystem |
|
||||
| 🔍 | Search |
|
||||
| ⬇️ | Download |
|
||||
| ☁️ | Cloud / External |
|
||||
| 👤 | User / Client |
|
||||
367
docs/class_diagram.md
Normal file
367
docs/class_diagram.md
Normal file
@@ -0,0 +1,367 @@
|
||||
# Class Diagram - Agent Media
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
direction TB
|
||||
|
||||
%% ===========================================
|
||||
%% MEMORY SYSTEM
|
||||
%% ===========================================
|
||||
|
||||
class Memory {
|
||||
+Path storage_dir
|
||||
+Path ltm_file
|
||||
+LongTermMemory ltm
|
||||
+ShortTermMemory stm
|
||||
+EpisodicMemory episodic
|
||||
+__init__(storage_dir: str)
|
||||
+save() void
|
||||
+get_context_for_prompt() Dict
|
||||
+get_full_state() Dict
|
||||
+clear_session() void
|
||||
}
|
||||
|
||||
class LongTermMemory {
|
||||
+Dict config
|
||||
+Dict preferences
|
||||
+Dict~str, List~ library
|
||||
+List~Dict~ following
|
||||
+get_config(key: str) Any
|
||||
+set_config(key: str, value: Any) void
|
||||
+has_config(key: str) bool
|
||||
+add_to_library(media_type: str, media: Dict) void
|
||||
+get_library(media_type: str) List
|
||||
+follow_show(show: Dict) void
|
||||
+to_dict() Dict
|
||||
+from_dict(data: Dict)$ LongTermMemory
|
||||
}
|
||||
|
||||
class ShortTermMemory {
|
||||
+List~Dict~ conversation_history
|
||||
+Dict current_workflow
|
||||
+Dict extracted_entities
|
||||
+str current_topic
|
||||
+int max_history
|
||||
+add_message(role: str, content: str) void
|
||||
+get_recent_history(n: int) List
|
||||
+start_workflow(type: str, target: Dict) void
|
||||
+update_workflow_stage(stage: str) void
|
||||
+end_workflow() void
|
||||
+set_entity(key: str, value: Any) void
|
||||
+get_entity(key: str) Any
|
||||
+clear() void
|
||||
+to_dict() Dict
|
||||
}
|
||||
|
||||
class EpisodicMemory {
|
||||
+Dict last_search_results
|
||||
+List~Dict~ active_downloads
|
||||
+List~Dict~ recent_errors
|
||||
+Dict pending_question
|
||||
+List~Dict~ background_events
|
||||
+store_search_results(query: str, results: List) void
|
||||
+get_result_by_index(index: int) Dict
|
||||
+get_search_results() Dict
|
||||
+add_active_download(download: Dict) void
|
||||
+complete_download(task_id: str, path: str) Dict
|
||||
+add_error(action: str, error: str) void
|
||||
+set_pending_question(question: str, options: List) void
|
||||
+resolve_pending_question(index: int) Dict
|
||||
+add_background_event(type: str, data: Dict) void
|
||||
+get_unread_events() List
|
||||
+clear() void
|
||||
+to_dict() Dict
|
||||
}
|
||||
|
||||
Memory *-- LongTermMemory : ltm
|
||||
Memory *-- ShortTermMemory : stm
|
||||
Memory *-- EpisodicMemory : episodic
|
||||
|
||||
%% ===========================================
|
||||
%% AGENT SYSTEM
|
||||
%% ===========================================
|
||||
|
||||
class Agent {
|
||||
+LLMClient llm
|
||||
+Memory memory
|
||||
+Dict~str, Tool~ tools
|
||||
+PromptBuilder prompt_builder
|
||||
+int max_tool_iterations
|
||||
+__init__(llm: LLMClient, memory: Memory)
|
||||
+step(user_input: str) str
|
||||
-_parse_intent(text: str) Dict
|
||||
-_execute_action(intent: Dict) Dict
|
||||
-_check_unread_events() str
|
||||
}
|
||||
|
||||
class LLMClient {
|
||||
<<Protocol>>
|
||||
+complete(messages: List) str
|
||||
}
|
||||
|
||||
class DeepSeekClient {
|
||||
+str api_key
|
||||
+str model
|
||||
+str base_url
|
||||
+complete(messages: List) str
|
||||
}
|
||||
|
||||
class OllamaClient {
|
||||
+str base_url
|
||||
+str model
|
||||
+complete(messages: List) str
|
||||
}
|
||||
|
||||
class PromptBuilder {
|
||||
+Dict~str, Tool~ tools
|
||||
+__init__(tools: Dict)
|
||||
+build_system_prompt(memory: Memory) str
|
||||
-_format_tools_description() str
|
||||
-_format_episodic_context(memory: Memory) str
|
||||
-_format_stm_context(memory: Memory) str
|
||||
}
|
||||
|
||||
class Tool {
|
||||
<<dataclass>>
|
||||
+str name
|
||||
+str description
|
||||
+Callable func
|
||||
+Dict parameters
|
||||
}
|
||||
|
||||
Agent --> LLMClient : uses
|
||||
Agent --> Memory : uses
|
||||
Agent --> PromptBuilder : uses
|
||||
Agent --> Tool : executes
|
||||
DeepSeekClient ..|> LLMClient
|
||||
OllamaClient ..|> LLMClient
|
||||
PromptBuilder --> Tool : formats
|
||||
|
||||
%% ===========================================
|
||||
%% DOMAIN - MOVIES
|
||||
%% ===========================================
|
||||
|
||||
class Movie {
|
||||
<<Entity>>
|
||||
+ImdbId imdb_id
|
||||
+MovieTitle title
|
||||
+ReleaseYear release_year
|
||||
+Quality quality
|
||||
+FilePath file_path
|
||||
+FileSize file_size
|
||||
+int tmdb_id
|
||||
+datetime added_at
|
||||
}
|
||||
|
||||
class MovieTitle {
|
||||
<<ValueObject>>
|
||||
+str value
|
||||
+__init__(value: str)
|
||||
}
|
||||
|
||||
class ReleaseYear {
|
||||
<<ValueObject>>
|
||||
+int value
|
||||
+__init__(value: int)
|
||||
}
|
||||
|
||||
class Quality {
|
||||
<<ValueObject>>
|
||||
+str value
|
||||
+__init__(value: str)
|
||||
}
|
||||
|
||||
class MovieRepository {
|
||||
<<Interface>>
|
||||
+save(movie: Movie) void
|
||||
+find_by_imdb_id(imdb_id: ImdbId) Movie
|
||||
+find_all() List~Movie~
|
||||
+delete(imdb_id: ImdbId) bool
|
||||
+exists(imdb_id: ImdbId) bool
|
||||
}
|
||||
|
||||
Movie --> MovieTitle
|
||||
Movie --> ReleaseYear
|
||||
Movie --> Quality
|
||||
Movie --> ImdbId
|
||||
|
||||
%% ===========================================
|
||||
%% DOMAIN - TV SHOWS
|
||||
%% ===========================================
|
||||
|
||||
class TVShow {
|
||||
<<Entity>>
|
||||
+ImdbId imdb_id
|
||||
+str title
|
||||
+int seasons_count
|
||||
+ShowStatus status
|
||||
+int tmdb_id
|
||||
+str first_air_date
|
||||
+datetime added_at
|
||||
}
|
||||
|
||||
class ShowStatus {
|
||||
<<Enum>>
|
||||
CONTINUING
|
||||
ENDED
|
||||
UNKNOWN
|
||||
+from_string(value: str)$ ShowStatus
|
||||
}
|
||||
|
||||
class TVShowRepository {
|
||||
<<Interface>>
|
||||
+save(show: TVShow) void
|
||||
+find_by_imdb_id(imdb_id: ImdbId) TVShow
|
||||
+find_all() List~TVShow~
|
||||
+delete(imdb_id: ImdbId) bool
|
||||
}
|
||||
|
||||
TVShow --> ShowStatus
|
||||
TVShow --> ImdbId
|
||||
|
||||
%% ===========================================
|
||||
%% DOMAIN - SHARED
|
||||
%% ===========================================
|
||||
|
||||
class ImdbId {
|
||||
<<ValueObject>>
|
||||
+str value
|
||||
+__init__(value: str)
|
||||
+__str__() str
|
||||
}
|
||||
|
||||
class FilePath {
|
||||
<<ValueObject>>
|
||||
+str value
|
||||
+__init__(value: str)
|
||||
}
|
||||
|
||||
class FileSize {
|
||||
<<ValueObject>>
|
||||
+int bytes
|
||||
+__init__(bytes: int)
|
||||
+to_human_readable() str
|
||||
}
|
||||
|
||||
%% ===========================================
|
||||
%% INFRASTRUCTURE - PERSISTENCE
|
||||
%% ===========================================
|
||||
|
||||
class JsonMovieRepository {
|
||||
+Memory memory
|
||||
+__init__(memory: Memory)
|
||||
+save(movie: Movie) void
|
||||
+find_by_imdb_id(imdb_id: ImdbId) Movie
|
||||
+find_all() List~Movie~
|
||||
+delete(imdb_id: ImdbId) bool
|
||||
}
|
||||
|
||||
class JsonTVShowRepository {
|
||||
+Memory memory
|
||||
+__init__(memory: Memory)
|
||||
+save(show: TVShow) void
|
||||
+find_by_imdb_id(imdb_id: ImdbId) TVShow
|
||||
+find_all() List~TVShow~
|
||||
+delete(imdb_id: ImdbId) bool
|
||||
}
|
||||
|
||||
JsonMovieRepository ..|> MovieRepository
|
||||
JsonTVShowRepository ..|> TVShowRepository
|
||||
JsonMovieRepository --> Memory
|
||||
JsonTVShowRepository --> Memory
|
||||
|
||||
%% ===========================================
|
||||
%% INFRASTRUCTURE - API CLIENTS
|
||||
%% ===========================================
|
||||
|
||||
class TMDBClient {
|
||||
+str api_key
|
||||
+str base_url
|
||||
+search_movie(title: str) TMDBSearchResult
|
||||
+search_tv(title: str) TMDBSearchResult
|
||||
+get_external_ids(tmdb_id: int) Dict
|
||||
}
|
||||
|
||||
class KnabenClient {
|
||||
+str base_url
|
||||
+search(query: str, limit: int) List~TorrentResult~
|
||||
}
|
||||
|
||||
class QBittorrentClient {
|
||||
+str host
|
||||
+str username
|
||||
+str password
|
||||
+add_torrent(magnet: str) bool
|
||||
+get_torrents() List
|
||||
}
|
||||
|
||||
%% ===========================================
|
||||
%% INFRASTRUCTURE - FILESYSTEM
|
||||
%% ===========================================
|
||||
|
||||
class FileManager {
|
||||
+Memory memory
|
||||
+__init__(memory: Memory)
|
||||
+set_folder_path(name: str, path: str) Dict
|
||||
+list_folder(type: str, path: str) Dict
|
||||
+move_file(source: str, dest: str) Dict
|
||||
}
|
||||
|
||||
FileManager --> Memory
|
||||
|
||||
%% ===========================================
|
||||
%% APPLICATION - USE CASES
|
||||
%% ===========================================
|
||||
|
||||
class SearchMovieUseCase {
|
||||
+TMDBClient tmdb_client
|
||||
+execute(title: str) SearchMovieResponse
|
||||
}
|
||||
|
||||
class SearchTorrentsUseCase {
|
||||
+KnabenClient knaben_client
|
||||
+execute(title: str, limit: int) SearchTorrentsResponse
|
||||
}
|
||||
|
||||
class AddTorrentUseCase {
|
||||
+QBittorrentClient qbittorrent_client
|
||||
+execute(magnet: str) AddTorrentResponse
|
||||
}
|
||||
|
||||
class SetFolderPathUseCase {
|
||||
+FileManager file_manager
|
||||
+execute(folder_name: str, path: str) SetFolderPathResponse
|
||||
}
|
||||
|
||||
class ListFolderUseCase {
|
||||
+FileManager file_manager
|
||||
+execute(folder_type: str, path: str) ListFolderResponse
|
||||
}
|
||||
|
||||
SearchMovieUseCase --> TMDBClient
|
||||
SearchTorrentsUseCase --> KnabenClient
|
||||
AddTorrentUseCase --> QBittorrentClient
|
||||
SetFolderPathUseCase --> FileManager
|
||||
ListFolderUseCase --> FileManager
|
||||
```
|
||||
|
||||
## Legend
|
||||
|
||||
| Symbol | Meaning |
|
||||
|--------|---------|
|
||||
| `<<Entity>>` | Domain entity with identity |
|
||||
| `<<ValueObject>>` | Immutable value object |
|
||||
| `<<Interface>>` | Abstract interface/protocol |
|
||||
| `<<Enum>>` | Enumeration |
|
||||
| `<<dataclass>>` | Python dataclass |
|
||||
| `<<Protocol>>` | Python Protocol (structural typing) |
|
||||
| `*--` | Composition (owns) |
|
||||
| `-->` | Association (uses) |
|
||||
| `..\|>` | Implementation |
|
||||
|
||||
## Architecture Layers
|
||||
|
||||
1. **Domain Layer** - Business entities and rules (Movie, TVShow, ValueObjects)
|
||||
2. **Application Layer** - Use cases orchestrating business logic
|
||||
3. **Infrastructure Layer** - External services (APIs, filesystem, persistence)
|
||||
4. **Agent Layer** - AI agent, LLM clients, tools, prompts
|
||||
311
docs/component_diagram.md
Normal file
311
docs/component_diagram.md
Normal file
@@ -0,0 +1,311 @@
|
||||
# Component Diagram - Agent Media (DDD Architecture)
|
||||
|
||||
```mermaid
|
||||
C4Component
|
||||
title Component Diagram - Agent Media
|
||||
|
||||
Container_Boundary(agent_layer, "Agent Layer") {
|
||||
Component(agent, "Agent", "Python", "Orchestrates LLM and tools")
|
||||
Component(prompt_builder, "PromptBuilder", "Python", "Builds system prompts with context")
|
||||
Component(registry, "Tool Registry", "Python", "Registers and binds tools")
|
||||
|
||||
Component_Boundary(llm_clients, "LLM Clients") {
|
||||
Component(deepseek, "DeepSeekClient", "Python", "DeepSeek API client")
|
||||
Component(ollama, "OllamaClient", "Python", "Ollama local client")
|
||||
}
|
||||
|
||||
Component_Boundary(tools, "Tools") {
|
||||
Component(api_tools, "API Tools", "Python", "find_torrent, add_torrent, etc.")
|
||||
Component(fs_tools, "Filesystem Tools", "Python", "set_path, list_folder")
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Layered Architecture (DDD)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Presentation["🌐 Presentation Layer"]
|
||||
API["FastAPI Server<br/>/v1/chat/completions"]
|
||||
end
|
||||
|
||||
subgraph Agent["🤖 Agent Layer"]
|
||||
AG[Agent]
|
||||
PB[PromptBuilder]
|
||||
TR[Tool Registry]
|
||||
|
||||
subgraph LLM["LLM Clients"]
|
||||
DS[DeepSeek]
|
||||
OL[Ollama]
|
||||
end
|
||||
|
||||
subgraph Tools["Tools"]
|
||||
AT[API Tools]
|
||||
FT[Filesystem Tools]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Application["⚙️ Application Layer"]
|
||||
subgraph UseCases["Use Cases"]
|
||||
UC1[SearchMovieUseCase]
|
||||
UC2[SearchTorrentsUseCase]
|
||||
UC3[AddTorrentUseCase]
|
||||
UC4[SetFolderPathUseCase]
|
||||
UC5[ListFolderUseCase]
|
||||
end
|
||||
|
||||
subgraph DTOs["DTOs"]
|
||||
DTO1[SearchMovieResponse]
|
||||
DTO2[SearchTorrentsResponse]
|
||||
DTO3[AddTorrentResponse]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Domain["📦 Domain Layer"]
|
||||
subgraph Movies["movies/"]
|
||||
ME[Movie Entity]
|
||||
MVO[MovieTitle, Quality, ReleaseYear]
|
||||
MR[MovieRepository Interface]
|
||||
end
|
||||
|
||||
subgraph TVShows["tv_shows/"]
|
||||
TE[TVShow Entity]
|
||||
TVO[ShowStatus]
|
||||
TR2[TVShowRepository Interface]
|
||||
end
|
||||
|
||||
subgraph Subtitles["subtitles/"]
|
||||
SE[Subtitle Entity]
|
||||
SVO[Language, SubtitleFormat]
|
||||
SR[SubtitleRepository Interface]
|
||||
end
|
||||
|
||||
subgraph Shared["shared/"]
|
||||
SH[ImdbId, FilePath, FileSize]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Infrastructure["🔧 Infrastructure Layer"]
|
||||
subgraph Persistence["persistence/"]
|
||||
MEM[Memory<br/>LTM + STM + Episodic]
|
||||
JMR[JsonMovieRepository]
|
||||
JTR[JsonTVShowRepository]
|
||||
JSR[JsonSubtitleRepository]
|
||||
end
|
||||
|
||||
subgraph APIs["api/"]
|
||||
TMDB[TMDBClient]
|
||||
KNAB[KnabenClient]
|
||||
QBIT[QBittorrentClient]
|
||||
end
|
||||
|
||||
subgraph FS["filesystem/"]
|
||||
FM[FileManager]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph External["☁️ External Services"]
|
||||
TMDB_API[(TMDB API)]
|
||||
KNAB_API[(Knaben API)]
|
||||
QBIT_API[(qBittorrent)]
|
||||
DISK[(Filesystem)]
|
||||
end
|
||||
|
||||
%% Connections
|
||||
API --> AG
|
||||
AG --> PB
|
||||
AG --> TR
|
||||
AG --> LLM
|
||||
TR --> Tools
|
||||
|
||||
AT --> UC1
|
||||
AT --> UC2
|
||||
AT --> UC3
|
||||
FT --> UC4
|
||||
FT --> UC5
|
||||
|
||||
UC1 --> TMDB
|
||||
UC2 --> KNAB
|
||||
UC3 --> QBIT
|
||||
UC4 --> FM
|
||||
UC5 --> FM
|
||||
|
||||
JMR --> MEM
|
||||
JTR --> MEM
|
||||
JSR --> MEM
|
||||
FM --> MEM
|
||||
|
||||
JMR -.->|implements| MR
|
||||
JTR -.->|implements| TR2
|
||||
JSR -.->|implements| SR
|
||||
|
||||
TMDB --> TMDB_API
|
||||
KNAB --> KNAB_API
|
||||
QBIT --> QBIT_API
|
||||
FM --> DISK
|
||||
MEM --> DISK
|
||||
|
||||
%% Styling
|
||||
classDef presentation fill:#e1f5fe
|
||||
classDef agent fill:#fff3e0
|
||||
classDef application fill:#f3e5f5
|
||||
classDef domain fill:#e8f5e9
|
||||
classDef infrastructure fill:#fce4ec
|
||||
classDef external fill:#f5f5f5
|
||||
|
||||
class API presentation
|
||||
class AG,PB,TR,DS,OL,AT,FT agent
|
||||
class UC1,UC2,UC3,UC4,UC5,DTO1,DTO2,DTO3 application
|
||||
class ME,MVO,MR,TE,TVO,TR2,SE,SVO,SR,SH domain
|
||||
class MEM,JMR,JTR,JSR,TMDB,KNAB,QBIT,FM infrastructure
|
||||
class TMDB_API,KNAB_API,QBIT_API,DISK external
|
||||
```
|
||||
|
||||
## Memory Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Memory["Memory System"]
|
||||
direction TB
|
||||
|
||||
subgraph LTM["💾 Long-Term Memory<br/>(Persistent - JSON)"]
|
||||
CONFIG[config<br/>download_folder, tvshow_folder...]
|
||||
PREFS[preferences<br/>quality, languages...]
|
||||
LIB[library<br/>movies[], tv_shows[]]
|
||||
FOLLOW[following<br/>watchlist]
|
||||
end
|
||||
|
||||
subgraph STM["🧠 Short-Term Memory<br/>(Session - RAM)"]
|
||||
HIST[conversation_history]
|
||||
WORKFLOW[current_workflow]
|
||||
ENTITIES[extracted_entities]
|
||||
TOPIC[current_topic]
|
||||
end
|
||||
|
||||
subgraph EPISODIC["⚡ Episodic Memory<br/>(Transient - RAM)"]
|
||||
SEARCH[last_search_results<br/>indexed torrents]
|
||||
DOWNLOADS[active_downloads]
|
||||
ERRORS[recent_errors]
|
||||
PENDING[pending_question]
|
||||
EVENTS[background_events]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Storage["Storage"]
|
||||
JSON[(ltm.json)]
|
||||
end
|
||||
|
||||
LTM -->|save| JSON
|
||||
JSON -->|load| LTM
|
||||
|
||||
STM -.->|cleared on| RESTART[Server Restart]
|
||||
EPISODIC -.->|cleared on| RESTART
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Input
|
||||
USER[User Request]
|
||||
end
|
||||
|
||||
subgraph Processing
|
||||
direction TB
|
||||
FASTAPI[FastAPI]
|
||||
AGENT[Agent]
|
||||
TOOLS[Tools]
|
||||
USECASES[Use Cases]
|
||||
end
|
||||
|
||||
subgraph External
|
||||
direction TB
|
||||
TMDB[(TMDB)]
|
||||
KNABEN[(Knaben)]
|
||||
QBIT[(qBittorrent)]
|
||||
end
|
||||
|
||||
subgraph Memory
|
||||
direction TB
|
||||
LTM[(LTM)]
|
||||
STM[(STM)]
|
||||
EPIS[(Episodic)]
|
||||
end
|
||||
|
||||
USER -->|HTTP POST| FASTAPI
|
||||
FASTAPI -->|step()| AGENT
|
||||
AGENT -->|execute| TOOLS
|
||||
TOOLS -->|call| USECASES
|
||||
|
||||
USECASES -->|search| TMDB
|
||||
USECASES -->|search| KNABEN
|
||||
USECASES -->|add| QBIT
|
||||
|
||||
AGENT <-->|read/write| LTM
|
||||
AGENT <-->|read/write| STM
|
||||
TOOLS <-->|read/write| EPIS
|
||||
|
||||
AGENT -->|response| FASTAPI
|
||||
FASTAPI -->|JSON| USER
|
||||
```
|
||||
|
||||
## Dependency Direction
|
||||
|
||||
```mermaid
|
||||
flowchart BT
|
||||
subgraph External["External"]
|
||||
EXT[APIs, Filesystem]
|
||||
end
|
||||
|
||||
subgraph Infra["Infrastructure"]
|
||||
INF[Clients, Repositories, Memory]
|
||||
end
|
||||
|
||||
subgraph App["Application"]
|
||||
APP[Use Cases, DTOs]
|
||||
end
|
||||
|
||||
subgraph Dom["Domain"]
|
||||
DOM[Entities, Value Objects, Interfaces]
|
||||
end
|
||||
|
||||
subgraph Agent["Agent"]
|
||||
AGT[Agent, Tools, Prompts]
|
||||
end
|
||||
|
||||
subgraph Pres["Presentation"]
|
||||
PRES[FastAPI]
|
||||
end
|
||||
|
||||
EXT --> Infra
|
||||
Infra --> App
|
||||
Infra -.->|implements| Dom
|
||||
App --> Dom
|
||||
Agent --> App
|
||||
Agent --> Infra
|
||||
Pres --> Agent
|
||||
|
||||
style Dom fill:#e8f5e9,stroke:#4caf50,stroke-width:3px
|
||||
style Infra fill:#fce4ec,stroke:#e91e63
|
||||
style App fill:#f3e5f5,stroke:#9c27b0
|
||||
style Agent fill:#fff3e0,stroke:#ff9800
|
||||
style Pres fill:#e1f5fe,stroke:#03a9f4
|
||||
```
|
||||
|
||||
## Legend
|
||||
|
||||
| Layer | Responsibility | Examples |
|
||||
|-------|---------------|----------|
|
||||
| 🌐 **Presentation** | HTTP interface, request/response handling | FastAPI endpoints |
|
||||
| 🤖 **Agent** | AI orchestration, LLM interaction, tools | Agent, PromptBuilder, Tools |
|
||||
| ⚙️ **Application** | Use case orchestration, DTOs | SearchMovieUseCase, SearchTorrentsResponse |
|
||||
| 📦 **Domain** | Business entities, rules, interfaces | Movie, TVShow, ImdbId, MovieRepository |
|
||||
| 🔧 **Infrastructure** | External service implementations | TMDBClient, JsonMovieRepository, Memory |
|
||||
| ☁️ **External** | Third-party services | TMDB API, qBittorrent, Filesystem |
|
||||
|
||||
## Key Principles
|
||||
|
||||
1. **Dependency Inversion**: Domain defines interfaces, Infrastructure implements them
|
||||
2. **Clean Architecture**: Dependencies point inward (toward Domain)
|
||||
3. **Separation of Concerns**: Each layer has a single responsibility
|
||||
4. **Memory Segregation**: LTM (persistent), STM (session), Episodic (transient)
|
||||
366
docs/flowchart.md
Normal file
366
docs/flowchart.md
Normal file
@@ -0,0 +1,366 @@
|
||||
# Flowcharts - Agent Media
|
||||
|
||||
## 1. Main Application Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
START([Application Start]) --> INIT_MEM[Initialize Memory Context<br/>init_memory]
|
||||
INIT_MEM --> INIT_LLM{LLM Provider?}
|
||||
|
||||
INIT_LLM -->|OLLAMA| OLLAMA[Create OllamaClient]
|
||||
INIT_LLM -->|DEEPSEEK| DEEPSEEK[Create DeepSeekClient]
|
||||
|
||||
OLLAMA --> INIT_AGENT[Create Agent]
|
||||
DEEPSEEK --> INIT_AGENT
|
||||
|
||||
INIT_AGENT --> INIT_TOOLS[Register Tools<br/>make_tools]
|
||||
INIT_TOOLS --> START_SERVER[Start FastAPI Server<br/>:8000]
|
||||
|
||||
START_SERVER --> WAIT_REQ[/Wait for Request/]
|
||||
|
||||
WAIT_REQ --> REQ_TYPE{Request Type?}
|
||||
|
||||
REQ_TYPE -->|GET /health| HEALTH[Return health status]
|
||||
REQ_TYPE -->|GET /v1/models| MODELS[Return model list]
|
||||
REQ_TYPE -->|GET /memory/state| MEM_STATE[Return memory state]
|
||||
REQ_TYPE -->|POST /memory/clear-session| CLEAR_SESSION[Clear STM + Episodic]
|
||||
REQ_TYPE -->|POST /v1/chat/completions| CHAT[Process Chat Request]
|
||||
|
||||
HEALTH --> WAIT_REQ
|
||||
MODELS --> WAIT_REQ
|
||||
MEM_STATE --> WAIT_REQ
|
||||
CLEAR_SESSION --> WAIT_REQ
|
||||
CHAT --> AGENT_STEP[agent.step]
|
||||
AGENT_STEP --> RETURN_RESP[Return Response]
|
||||
RETURN_RESP --> WAIT_REQ
|
||||
```
|
||||
|
||||
## 2. Agent Step Flow (Core Logic)
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
START([agent.step called]) --> GET_MEM[Get Memory from Context]
|
||||
GET_MEM --> CHECK_EVENTS[Check Unread Events]
|
||||
|
||||
CHECK_EVENTS --> HAS_EVENTS{Has Events?}
|
||||
HAS_EVENTS -->|Yes| FORMAT_EVENTS[Format Event Notifications]
|
||||
HAS_EVENTS -->|No| BUILD_PROMPT
|
||||
FORMAT_EVENTS --> BUILD_PROMPT
|
||||
|
||||
BUILD_PROMPT[Build System Prompt<br/>with Memory Context]
|
||||
BUILD_PROMPT --> INIT_MSGS[Initialize Messages Array]
|
||||
|
||||
INIT_MSGS --> ADD_SYSTEM[Add System Prompt]
|
||||
ADD_SYSTEM --> GET_HISTORY[Get STM History]
|
||||
GET_HISTORY --> ADD_HISTORY[Add History Messages]
|
||||
ADD_HISTORY --> ADD_NOTIF{Has Notifications?}
|
||||
|
||||
ADD_NOTIF -->|Yes| ADD_NOTIF_MSG[Add Notification Message]
|
||||
ADD_NOTIF -->|No| ADD_USER
|
||||
ADD_NOTIF_MSG --> ADD_USER[Add User Input]
|
||||
|
||||
ADD_USER --> LOOP_START[/Tool Execution Loop/]
|
||||
|
||||
LOOP_START --> CHECK_ITER{iteration < max?}
|
||||
CHECK_ITER -->|No| MAX_REACHED[Request Final Response]
|
||||
CHECK_ITER -->|Yes| CALL_LLM[Call LLM.complete]
|
||||
|
||||
MAX_REACHED --> FINAL_LLM[Call LLM.complete]
|
||||
FINAL_LLM --> SAVE_FINAL[Save to STM History]
|
||||
SAVE_FINAL --> RETURN_FINAL([Return Response])
|
||||
|
||||
CALL_LLM --> PARSE_INTENT[Parse Intent from Response]
|
||||
PARSE_INTENT --> IS_TOOL{Is Tool Call?}
|
||||
|
||||
IS_TOOL -->|No| SAVE_HISTORY[Save to STM History]
|
||||
SAVE_HISTORY --> SAVE_LTM[Save LTM]
|
||||
SAVE_LTM --> RETURN_TEXT([Return Text Response])
|
||||
|
||||
IS_TOOL -->|Yes| EXEC_TOOL[Execute Tool]
|
||||
EXEC_TOOL --> ADD_RESULT[Add Tool Result to Messages]
|
||||
ADD_RESULT --> INC_ITER[iteration++]
|
||||
INC_ITER --> LOOP_START
|
||||
```
|
||||
|
||||
## 3. Tool Execution Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
START([_execute_action called]) --> GET_ACTION[Extract action name & args]
|
||||
GET_ACTION --> FIND_TOOL{Tool exists?}
|
||||
|
||||
FIND_TOOL -->|No| UNKNOWN[Return unknown_tool error]
|
||||
UNKNOWN --> END_ERR([Return Error])
|
||||
|
||||
FIND_TOOL -->|Yes| CALL_FUNC[Call tool.func with args]
|
||||
|
||||
CALL_FUNC --> EXEC_OK{Execution OK?}
|
||||
|
||||
EXEC_OK -->|TypeError| BAD_ARGS[Log bad arguments error]
|
||||
EXEC_OK -->|Exception| EXEC_ERR[Log execution error]
|
||||
EXEC_OK -->|Success| CHECK_RESULT{Result has error?}
|
||||
|
||||
BAD_ARGS --> ADD_ERR_MEM[Add error to Episodic Memory]
|
||||
EXEC_ERR --> ADD_ERR_MEM
|
||||
ADD_ERR_MEM --> END_ERR
|
||||
|
||||
CHECK_RESULT -->|Yes| ADD_ERR_MEM2[Add error to Episodic Memory]
|
||||
ADD_ERR_MEM2 --> RETURN_RESULT
|
||||
CHECK_RESULT -->|No| RETURN_RESULT([Return Result])
|
||||
```
|
||||
|
||||
## 4. Prompt Building Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
START([build_system_prompt called]) --> GET_MEM[Get Memory from Context]
|
||||
|
||||
GET_MEM --> FORMAT_TOOLS[Format Tools Description]
|
||||
FORMAT_TOOLS --> FORMAT_PARAMS[Format Parameters Description]
|
||||
|
||||
FORMAT_PARAMS --> CHECK_MISSING[Check Missing Required Params]
|
||||
CHECK_MISSING --> HAS_MISSING{Has Missing?}
|
||||
|
||||
HAS_MISSING -->|Yes| FORMAT_MISSING[Format Missing Params Info]
|
||||
HAS_MISSING -->|No| FORMAT_EPISODIC
|
||||
FORMAT_MISSING --> FORMAT_EPISODIC
|
||||
|
||||
FORMAT_EPISODIC[Format Episodic Context]
|
||||
FORMAT_EPISODIC --> HAS_SEARCH{Has Search Results?}
|
||||
|
||||
HAS_SEARCH -->|Yes| ADD_SEARCH[Add Search Results Summary]
|
||||
HAS_SEARCH -->|No| CHECK_PENDING
|
||||
ADD_SEARCH --> CHECK_PENDING
|
||||
|
||||
CHECK_PENDING{Has Pending Question?}
|
||||
CHECK_PENDING -->|Yes| ADD_PENDING[Add Pending Question]
|
||||
CHECK_PENDING -->|No| CHECK_DOWNLOADS
|
||||
ADD_PENDING --> CHECK_DOWNLOADS
|
||||
|
||||
CHECK_DOWNLOADS{Has Active Downloads?}
|
||||
CHECK_DOWNLOADS -->|Yes| ADD_DOWNLOADS[Add Downloads Status]
|
||||
CHECK_DOWNLOADS -->|No| CHECK_ERRORS
|
||||
ADD_DOWNLOADS --> CHECK_ERRORS
|
||||
|
||||
CHECK_ERRORS{Has Recent Errors?}
|
||||
CHECK_ERRORS -->|Yes| ADD_ERRORS[Add Last Error]
|
||||
CHECK_ERRORS -->|No| FORMAT_STM
|
||||
ADD_ERRORS --> FORMAT_STM
|
||||
|
||||
FORMAT_STM[Format STM Context]
|
||||
FORMAT_STM --> HAS_WORKFLOW{Has Workflow?}
|
||||
|
||||
HAS_WORKFLOW -->|Yes| ADD_WORKFLOW[Add Workflow Info]
|
||||
HAS_WORKFLOW -->|No| CHECK_TOPIC
|
||||
ADD_WORKFLOW --> CHECK_TOPIC
|
||||
|
||||
CHECK_TOPIC{Has Topic?}
|
||||
CHECK_TOPIC -->|Yes| ADD_TOPIC[Add Current Topic]
|
||||
CHECK_TOPIC -->|No| CHECK_ENTITIES
|
||||
ADD_TOPIC --> CHECK_ENTITIES
|
||||
|
||||
CHECK_ENTITIES{Has Entities?}
|
||||
CHECK_ENTITIES -->|Yes| ADD_ENTITIES[Add Extracted Entities]
|
||||
CHECK_ENTITIES -->|No| BUILD_FINAL
|
||||
ADD_ENTITIES --> BUILD_FINAL
|
||||
|
||||
BUILD_FINAL[Assemble Final Prompt]
|
||||
BUILD_FINAL --> RETURN([Return System Prompt])
|
||||
```
|
||||
|
||||
## 5. Memory System Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph Initialization
|
||||
INIT([init_memory called]) --> CREATE_MEM[Create Memory Instance]
|
||||
CREATE_MEM --> LOAD_LTM{LTM file exists?}
|
||||
LOAD_LTM -->|Yes| READ_FILE[Read ltm.json]
|
||||
LOAD_LTM -->|No| CREATE_DEFAULT[Create Default LTM]
|
||||
READ_FILE --> PARSE_JSON{Parse OK?}
|
||||
PARSE_JSON -->|Yes| RESTORE_LTM[Restore LTM from Dict]
|
||||
PARSE_JSON -->|No| CREATE_DEFAULT
|
||||
CREATE_DEFAULT --> CREATE_STM[Create Empty STM]
|
||||
RESTORE_LTM --> CREATE_STM
|
||||
CREATE_STM --> CREATE_EPIS[Create Empty Episodic]
|
||||
CREATE_EPIS --> SET_CTX[Set in Context Variable]
|
||||
SET_CTX --> RETURN_MEM([Return Memory])
|
||||
end
|
||||
|
||||
subgraph Access
|
||||
GET([get_memory called]) --> CHECK_CTX{Context has Memory?}
|
||||
CHECK_CTX -->|Yes| RETURN_CTX([Return Memory])
|
||||
CHECK_CTX -->|No| RAISE_ERR[Raise RuntimeError]
|
||||
end
|
||||
|
||||
subgraph Save
|
||||
SAVE([memory.save called]) --> SERIALIZE[Serialize LTM to Dict]
|
||||
SERIALIZE --> WRITE_JSON[Write to ltm.json]
|
||||
WRITE_JSON --> SAVE_OK{Write OK?}
|
||||
SAVE_OK -->|Yes| DONE([Done])
|
||||
SAVE_OK -->|No| LOG_ERR[Log Error & Raise]
|
||||
end
|
||||
```
|
||||
|
||||
## 6. Torrent Search & Download Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph Search
|
||||
SEARCH_START([find_torrent called]) --> CREATE_UC[Create SearchTorrentsUseCase]
|
||||
CREATE_UC --> EXEC_SEARCH[Execute Search via Knaben API]
|
||||
EXEC_SEARCH --> SEARCH_OK{Results Found?}
|
||||
|
||||
SEARCH_OK -->|No| RETURN_ERR([Return Error])
|
||||
SEARCH_OK -->|Yes| GET_MEM[Get Memory]
|
||||
GET_MEM --> STORE_RESULTS[Store in Episodic Memory<br/>with indexes 1,2,3...]
|
||||
STORE_RESULTS --> SET_TOPIC[Set Topic: selecting_torrent]
|
||||
SET_TOPIC --> RETURN_RESULTS([Return Results])
|
||||
end
|
||||
|
||||
subgraph "Get by Index"
|
||||
GET_START([get_torrent_by_index called]) --> GET_MEM2[Get Memory]
|
||||
GET_MEM2 --> HAS_RESULTS{Has Search Results?}
|
||||
|
||||
HAS_RESULTS -->|No| NO_RESULTS([Return not_found Error])
|
||||
HAS_RESULTS -->|Yes| FIND_INDEX[Find Result by Index]
|
||||
FIND_INDEX --> FOUND{Found?}
|
||||
|
||||
FOUND -->|No| NOT_FOUND([Return not_found Error])
|
||||
FOUND -->|Yes| RETURN_TORRENT([Return Torrent Data])
|
||||
end
|
||||
|
||||
subgraph "Add by Index"
|
||||
ADD_START([add_torrent_by_index called]) --> CALL_GET[Call get_torrent_by_index]
|
||||
CALL_GET --> GET_OK{Got Torrent?}
|
||||
|
||||
GET_OK -->|No| RETURN_GET_ERR([Return Error])
|
||||
GET_OK -->|Yes| HAS_MAGNET{Has Magnet Link?}
|
||||
|
||||
HAS_MAGNET -->|No| NO_MAGNET([Return no_magnet Error])
|
||||
HAS_MAGNET -->|Yes| CALL_ADD[Call add_torrent_to_qbittorrent]
|
||||
CALL_ADD --> ADD_OK{Added OK?}
|
||||
|
||||
ADD_OK -->|No| RETURN_ADD_ERR([Return Error])
|
||||
ADD_OK -->|Yes| ADD_NAME[Add torrent_name to Result]
|
||||
ADD_NAME --> RETURN_SUCCESS([Return Success])
|
||||
end
|
||||
|
||||
subgraph "Add to qBittorrent"
|
||||
QB_START([add_torrent_to_qbittorrent called]) --> CREATE_UC2[Create AddTorrentUseCase]
|
||||
CREATE_UC2 --> EXEC_ADD[Execute Add via qBittorrent API]
|
||||
EXEC_ADD --> QB_OK{Added OK?}
|
||||
|
||||
QB_OK -->|No| QB_ERR([Return Error])
|
||||
QB_OK -->|Yes| GET_MEM3[Get Memory]
|
||||
GET_MEM3 --> FIND_NAME[Find Torrent Name from Search]
|
||||
FIND_NAME --> ADD_DOWNLOAD[Add to Active Downloads]
|
||||
ADD_DOWNLOAD --> SET_TOPIC2[Set Topic: downloading]
|
||||
SET_TOPIC2 --> END_WORKFLOW[End Current Workflow]
|
||||
END_WORKFLOW --> QB_SUCCESS([Return Success])
|
||||
end
|
||||
```
|
||||
|
||||
## 7. Filesystem Operations Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph "Set Folder Path"
|
||||
SET_START([set_path_for_folder called]) --> VALIDATE_NAME[Validate Folder Name]
|
||||
VALIDATE_NAME --> NAME_OK{Valid Name?}
|
||||
|
||||
NAME_OK -->|No| INVALID_NAME([Return validation_failed])
|
||||
NAME_OK -->|Yes| RESOLVE_PATH[Resolve Path]
|
||||
|
||||
RESOLVE_PATH --> PATH_EXISTS{Path Exists?}
|
||||
PATH_EXISTS -->|No| NOT_EXISTS([Return invalid_path])
|
||||
PATH_EXISTS -->|Yes| IS_DIR{Is Directory?}
|
||||
|
||||
IS_DIR -->|No| NOT_DIR([Return invalid_path])
|
||||
IS_DIR -->|Yes| IS_READABLE{Is Readable?}
|
||||
|
||||
IS_READABLE -->|No| NO_READ([Return permission_denied])
|
||||
IS_READABLE -->|Yes| GET_MEM[Get Memory]
|
||||
|
||||
GET_MEM --> SET_CONFIG[Set in LTM Config]
|
||||
SET_CONFIG --> SAVE_MEM[Save Memory]
|
||||
SAVE_MEM --> SET_SUCCESS([Return Success])
|
||||
end
|
||||
|
||||
subgraph "List Folder"
|
||||
LIST_START([list_folder called]) --> VALIDATE_TYPE[Validate Folder Type]
|
||||
VALIDATE_TYPE --> TYPE_OK{Valid Type?}
|
||||
|
||||
TYPE_OK -->|No| INVALID_TYPE([Return validation_failed])
|
||||
TYPE_OK -->|Yes| SANITIZE[Sanitize Path]
|
||||
|
||||
SANITIZE --> SAFE{Path Safe?}
|
||||
SAFE -->|No| TRAVERSAL([Return forbidden])
|
||||
SAFE -->|Yes| GET_MEM2[Get Memory]
|
||||
|
||||
GET_MEM2 --> GET_CONFIG[Get Folder from Config]
|
||||
GET_CONFIG --> CONFIGURED{Folder Configured?}
|
||||
|
||||
CONFIGURED -->|No| NOT_SET([Return folder_not_set])
|
||||
CONFIGURED -->|Yes| BUILD_TARGET[Build Target Path]
|
||||
|
||||
BUILD_TARGET --> CHECK_SAFE[Check Path is Safe]
|
||||
CHECK_SAFE --> SAFE2{Within Base?}
|
||||
|
||||
SAFE2 -->|No| FORBIDDEN([Return forbidden])
|
||||
SAFE2 -->|Yes| TARGET_EXISTS{Target Exists?}
|
||||
|
||||
TARGET_EXISTS -->|No| NOT_FOUND([Return not_found])
|
||||
TARGET_EXISTS -->|Yes| TARGET_DIR{Is Directory?}
|
||||
|
||||
TARGET_DIR -->|No| NOT_A_DIR([Return not_a_directory])
|
||||
TARGET_DIR -->|Yes| LIST_DIR[List Directory Contents]
|
||||
|
||||
LIST_DIR --> LIST_OK{Permission OK?}
|
||||
LIST_OK -->|No| PERM_DENIED([Return permission_denied])
|
||||
LIST_OK -->|Yes| LIST_SUCCESS([Return Entries])
|
||||
end
|
||||
```
|
||||
|
||||
## 8. LLM Communication Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph "DeepSeek Client"
|
||||
DS_START([complete called]) --> DS_BUILD[Build Request Body]
|
||||
DS_BUILD --> DS_HEADERS[Set Headers with API Key]
|
||||
DS_HEADERS --> DS_POST[POST to DeepSeek API]
|
||||
DS_POST --> DS_OK{Response OK?}
|
||||
|
||||
DS_OK -->|No| DS_ERR{Error Type?}
|
||||
DS_ERR -->|401| DS_AUTH([Raise LLMAuthenticationError])
|
||||
DS_ERR -->|429| DS_RATE([Raise LLMRateLimitError])
|
||||
DS_ERR -->|Other| DS_API([Raise LLMAPIError])
|
||||
|
||||
DS_OK -->|Yes| DS_PARSE[Parse JSON Response]
|
||||
DS_PARSE --> DS_EXTRACT[Extract Content]
|
||||
DS_EXTRACT --> DS_RETURN([Return Content String])
|
||||
end
|
||||
|
||||
subgraph "Ollama Client"
|
||||
OL_START([complete called]) --> OL_BUILD[Build Request Body]
|
||||
OL_BUILD --> OL_POST[POST to Ollama API]
|
||||
OL_POST --> OL_OK{Response OK?}
|
||||
|
||||
OL_OK -->|No| OL_ERR([Raise LLMAPIError])
|
||||
OL_OK -->|Yes| OL_PARSE[Parse JSON Response]
|
||||
OL_PARSE --> OL_EXTRACT[Extract Message Content]
|
||||
OL_EXTRACT --> OL_RETURN([Return Content String])
|
||||
end
|
||||
```
|
||||
|
||||
## Legend
|
||||
|
||||
| Symbol | Meaning |
|
||||
|--------|---------|
|
||||
| `([text])` | Start/End (Terminal) |
|
||||
| `[text]` | Process |
|
||||
| `{text}` | Decision |
|
||||
| `/text/` | Input/Output |
|
||||
| `-->` | Flow direction |
|
||||
| `-->\|label\|` | Conditional flow |
|
||||
264
docs/sequence_diagram.md
Normal file
264
docs/sequence_diagram.md
Normal file
@@ -0,0 +1,264 @@
|
||||
# Sequence Diagrams - Agent Media
|
||||
|
||||
## 1. Torrent Search and Download Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant User
|
||||
participant FastAPI as FastAPI Server
|
||||
participant Agent
|
||||
participant PromptBuilder
|
||||
participant LLM as LLM (DeepSeek/Ollama)
|
||||
participant Tools as Tool Registry
|
||||
participant Memory
|
||||
participant Knaben as Knaben API
|
||||
participant qBit as qBittorrent
|
||||
|
||||
User->>FastAPI: POST /v1/chat/completions<br/>"Find torrents for Inception 1080p"
|
||||
FastAPI->>Agent: step(user_input)
|
||||
|
||||
Agent->>Memory: stm.get_recent_history()
|
||||
Memory-->>Agent: conversation history
|
||||
|
||||
Agent->>PromptBuilder: build_system_prompt(memory)
|
||||
PromptBuilder->>Memory: ltm.config, episodic state
|
||||
Memory-->>PromptBuilder: context data
|
||||
PromptBuilder-->>Agent: system prompt with context
|
||||
|
||||
Agent->>LLM: complete(messages)
|
||||
LLM-->>Agent: {"action": {"name": "find_torrents", "args": {...}}}
|
||||
|
||||
Agent->>Agent: _parse_intent(response)
|
||||
Agent->>Tools: execute find_torrents
|
||||
Tools->>Knaben: search("Inception 1080p")
|
||||
Knaben-->>Tools: torrent results
|
||||
|
||||
Tools->>Memory: episodic.store_search_results()
|
||||
Memory-->>Tools: stored with indexes (1, 2, 3...)
|
||||
|
||||
Tools-->>Agent: {"status": "ok", "torrents": [...]}
|
||||
|
||||
Agent->>LLM: complete(messages + tool_result)
|
||||
LLM-->>Agent: "I found 5 torrents for Inception..."
|
||||
|
||||
Agent->>Memory: stm.add_message("user", input)
|
||||
Agent->>Memory: stm.add_message("assistant", response)
|
||||
Agent->>Memory: save()
|
||||
|
||||
Agent-->>FastAPI: final response
|
||||
FastAPI-->>User: JSON response
|
||||
|
||||
Note over User,qBit: User selects a torrent
|
||||
|
||||
User->>FastAPI: POST /v1/chat/completions<br/>"Download the 2nd one"
|
||||
FastAPI->>Agent: step(user_input)
|
||||
|
||||
Agent->>PromptBuilder: build_system_prompt(memory)
|
||||
PromptBuilder->>Memory: episodic.last_search_results
|
||||
Note right of Memory: Results still in memory:<br/>1. Inception.2010.1080p...<br/>2. Inception.1080p.BluRay...
|
||||
Memory-->>PromptBuilder: context with search results
|
||||
PromptBuilder-->>Agent: prompt showing available results
|
||||
|
||||
Agent->>LLM: complete(messages)
|
||||
LLM-->>Agent: {"action": {"name": "add_torrent_by_index", "args": {"index": 2}}}
|
||||
|
||||
Agent->>Tools: execute add_torrent_by_index(index=2)
|
||||
Tools->>Memory: episodic.get_result_by_index(2)
|
||||
Memory-->>Tools: torrent data with magnet link
|
||||
|
||||
Tools->>qBit: add_torrent(magnet_link)
|
||||
qBit-->>Tools: success
|
||||
|
||||
Tools->>Memory: episodic.add_active_download()
|
||||
Tools-->>Agent: {"status": "ok", "torrent_name": "Inception.1080p.BluRay"}
|
||||
|
||||
Agent->>LLM: complete(messages + tool_result)
|
||||
LLM-->>Agent: "I've added Inception to qBittorrent!"
|
||||
|
||||
Agent-->>FastAPI: final response
|
||||
FastAPI-->>User: JSON response
|
||||
```
|
||||
|
||||
## 2. Folder Configuration Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant User
|
||||
participant FastAPI as FastAPI Server
|
||||
participant Agent
|
||||
participant LLM as LLM
|
||||
participant Tools as Tool Registry
|
||||
participant FileManager
|
||||
participant Memory
|
||||
participant FS as Filesystem
|
||||
|
||||
User->>FastAPI: POST /v1/chat/completions<br/>"Set download folder to /mnt/media/downloads"
|
||||
FastAPI->>Agent: step(user_input)
|
||||
|
||||
Agent->>LLM: complete(messages)
|
||||
LLM-->>Agent: {"action": {"name": "set_path_for_folder", "args": {...}}}
|
||||
|
||||
Agent->>Tools: execute set_path_for_folder
|
||||
Tools->>FileManager: set_folder_path("download", "/mnt/media/downloads")
|
||||
|
||||
FileManager->>FS: Path.exists()?
|
||||
FS-->>FileManager: true
|
||||
FileManager->>FS: Path.is_dir()?
|
||||
FS-->>FileManager: true
|
||||
FileManager->>FS: os.access(R_OK)?
|
||||
FS-->>FileManager: true
|
||||
|
||||
FileManager->>Memory: ltm.set_config("download_folder", path)
|
||||
FileManager->>Memory: save()
|
||||
Memory->>FS: write ltm.json
|
||||
|
||||
FileManager-->>Tools: {"status": "ok", "path": "/mnt/media/downloads"}
|
||||
Tools-->>Agent: result
|
||||
|
||||
Agent->>LLM: complete(messages + tool_result)
|
||||
LLM-->>Agent: "Download folder set to /mnt/media/downloads"
|
||||
|
||||
Agent-->>FastAPI: final response
|
||||
FastAPI-->>User: JSON response
|
||||
```
|
||||
|
||||
## 3. Multi-Tool Workflow (Search Movie → Find Torrents → Download)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant User
|
||||
participant Agent
|
||||
participant LLM as LLM
|
||||
participant TMDB as TMDB API
|
||||
participant Knaben as Knaben API
|
||||
participant qBit as qBittorrent
|
||||
participant Memory
|
||||
|
||||
User->>Agent: "Download Dune 2 in 4K"
|
||||
|
||||
rect rgb(240, 248, 255)
|
||||
Note over Agent,TMDB: Step 1: Identify the movie
|
||||
Agent->>LLM: complete(messages)
|
||||
LLM-->>Agent: {"action": "find_media_imdb_id", "args": {"media_title": "Dune 2"}}
|
||||
Agent->>TMDB: search_movie("Dune 2")
|
||||
TMDB-->>Agent: {title: "Dune: Part Two", imdb_id: "tt15239678", year: 2024}
|
||||
Agent->>Memory: stm.set_entity("last_media_search", {...})
|
||||
end
|
||||
|
||||
rect rgb(255, 248, 240)
|
||||
Note over Agent,Knaben: Step 2: Search for torrents
|
||||
Agent->>LLM: complete(messages + movie_info)
|
||||
LLM-->>Agent: {"action": "find_torrents", "args": {"media_title": "Dune Part Two 2024 4K"}}
|
||||
Agent->>Knaben: search("Dune Part Two 2024 4K")
|
||||
Knaben-->>Agent: [torrent1, torrent2, torrent3...]
|
||||
Agent->>Memory: episodic.store_search_results()
|
||||
end
|
||||
|
||||
rect rgb(240, 255, 240)
|
||||
Note over Agent,qBit: Step 3: Add best torrent
|
||||
Agent->>LLM: complete(messages + torrents)
|
||||
LLM-->>Agent: {"action": "add_torrent_by_index", "args": {"index": 1}}
|
||||
Agent->>Memory: episodic.get_result_by_index(1)
|
||||
Memory-->>Agent: torrent with magnet
|
||||
Agent->>qBit: add_torrent(magnet)
|
||||
qBit-->>Agent: success
|
||||
Agent->>Memory: episodic.add_active_download()
|
||||
end
|
||||
|
||||
Agent->>LLM: complete(messages + all_results)
|
||||
LLM-->>Agent: "I found Dune: Part Two (2024) and added the 4K torrent to qBittorrent!"
|
||||
Agent-->>User: Final response
|
||||
```
|
||||
|
||||
## 4. Error Handling Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant User
|
||||
participant Agent
|
||||
participant LLM as LLM
|
||||
participant Tools as Tool Registry
|
||||
participant Memory
|
||||
participant API as External API
|
||||
|
||||
User->>Agent: "Download the 5th torrent"
|
||||
|
||||
Agent->>LLM: complete(messages)
|
||||
LLM-->>Agent: {"action": "add_torrent_by_index", "args": {"index": 5}}
|
||||
|
||||
Agent->>Tools: execute add_torrent_by_index(5)
|
||||
Tools->>Memory: episodic.get_result_by_index(5)
|
||||
|
||||
alt No search results
|
||||
Memory-->>Tools: None (no previous search)
|
||||
Tools-->>Agent: {"status": "error", "error": "not_found"}
|
||||
Agent->>Memory: episodic.add_error("add_torrent_by_index", "not_found")
|
||||
else Index out of range
|
||||
Memory-->>Tools: None (only 3 results)
|
||||
Tools-->>Agent: {"status": "error", "error": "not_found"}
|
||||
Agent->>Memory: episodic.add_error("add_torrent_by_index", "not_found")
|
||||
end
|
||||
|
||||
Agent->>LLM: complete(messages + error)
|
||||
LLM-->>Agent: "I couldn't find torrent #5. Please search for torrents first."
|
||||
|
||||
Agent-->>User: Error explanation
|
||||
|
||||
Note over User,API: User searches first
|
||||
|
||||
User->>Agent: "Search for Matrix 1999"
|
||||
Agent->>API: search("Matrix 1999")
|
||||
API-->>Agent: [3 results]
|
||||
Agent->>Memory: episodic.store_search_results()
|
||||
Agent-->>User: "Found 3 torrents..."
|
||||
|
||||
User->>Agent: "Download the 2nd one"
|
||||
Agent->>Memory: episodic.get_result_by_index(2)
|
||||
Memory-->>Agent: torrent data ✓
|
||||
Agent-->>User: "Added to qBittorrent!"
|
||||
```
|
||||
|
||||
## 5. Background Events Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant User
|
||||
participant Agent
|
||||
participant Memory
|
||||
participant qBit as qBittorrent
|
||||
participant LLM as LLM
|
||||
|
||||
Note over qBit,Memory: Background: Download completes
|
||||
qBit--)Memory: episodic.complete_download(task_id, file_path)
|
||||
Memory->>Memory: add_background_event("download_complete", {...})
|
||||
|
||||
Note over User,LLM: Later: User sends a message
|
||||
User->>Agent: "What's new?"
|
||||
|
||||
Agent->>Memory: episodic.get_unread_events()
|
||||
Memory-->>Agent: [{type: "download_complete", data: {name: "Inception.1080p"}}]
|
||||
|
||||
Agent->>Agent: _check_unread_events()
|
||||
Note right of Agent: Formats notification:<br/>"Download completed: Inception.1080p"
|
||||
|
||||
Agent->>LLM: complete(messages + notification)
|
||||
LLM-->>Agent: "Good news! Inception.1080p has finished downloading."
|
||||
|
||||
Agent-->>User: Response with notification
|
||||
```
|
||||
|
||||
## Legend
|
||||
|
||||
| Element | Description |
|
||||
|---------|-------------|
|
||||
| `rect rgb(...)` | Grouped steps in a workflow |
|
||||
| `alt/else` | Conditional branches |
|
||||
| `Note` | Explanatory notes |
|
||||
| `-->>` | Response/return |
|
||||
| `->>` | Request/call |
|
||||
| `--))` | Async event |
|
||||
@@ -1 +0,0 @@
|
||||
"""Test suite for Agent Media."""
|
||||
Reference in New Issue
Block a user