AgnosticRag-Q

AgnosticRag-Q is a cloud agnostic and LLM agnostic Agentic AI & Retrieval-Augmented Generation (RAG) platform built with a clean, modular architecture. It supports Text, CSV, Images, Audio, enterprise applications such as Odoo, hybrid search, multimodal retrieval, intelligent routing, and configurable runtime behavior.

It provides:

a Core RAG API
an intelligent routing layer (AgnosticRagRouter)
an MCP integration layer (AgnosticRagMCP)
enterprise application integration (Odoo, PostgreSQL, and custom MCP services)
A GraphRAG layer combining Qdrant vector search and Neo4j knowledge graphs
and a GUI application that consumes these backends

allowing you to build, extend, and deploy scalable multi-backend AI systems that combine private knowledge bases, enterprise applications, and multiple LLM providers.

The project is intentionally provider-agnostic, making it easy to switch between LLM providers, embedding models, vector stores, enterprise connectors, and retrieval strategies without changing business logic.

AgnosticRagRouter introduces an Agentic AI orchestration layer that automatically selects the most appropriate backend or enterprise service for each request. Depending on the query, it can route to document collections (TXT, CSV, Images, Audio), GraphRAG, Odoo ERP, PostgreSQL, or any custom MCP service using configurable routing strategies and LLM-assisted decision making.

The platform is designed for modular deployment, where multiple RAG backends and enterprise services run independently. Dedicated services can be deployed for text, images, audio, domain-specific knowledge, Odoo ERP, databases, and custom business systems, while the router dynamically dispatches requests to the most relevant destination.

AgnosticRagMCP extends the platform through the Model Context Protocol (MCP), enabling seamless integration with external applications, enterprise systems, and communication channels. It supports connectors such as Telegram, WhatsApp, Microsoft Teams, Odoo, PostgreSQL, Google Drive, and custom enterprise MCP servers, allowing users and business applications to interact with both RAG knowledge bases and enterprise systems through a unified AI interface.

Full RAG with translation	Full RAG With Images/PDFs
Full RAG with Medical Data csv Table	Full RAG with Images : Search by an image or text
Visual search AgnosticRAG	Search in TXT and CSV using AgnosticRagRouter
AgnosticRagMCP Telegram	AgnosticRagMCP MS Teams
GraphRAG using Qdrant + Neo4j	Full Example : Txt, CSV, Image, Audio and Graph

✨ Key Highlights

Core Multi-Modal RAG API built with a clean, extensible architecture
Config-Driven rag_settings.json (models, rerank, chunking, prompts), graph_rag_settings.json (allowed relation types, graph extraction prompts, entity/relation rules)
AgnosticRagRouter intelligent LLM-based backend routing and multi-backend orchestration
Config-Driven AgnosticRagRouter router_settings.json (routing rules, backends, prompts, retries, thresholds)
AgnosticRagMCP MCP-based external integrations layer (Odoo, PostgreSQL, Telegram, WhatsApp, MS Teams, Google Drive, external enterprise tools & drivers)
Config-Driven AgnosticRagMCP mcp_settings.json (whatsapp, telegram, microsoft_teams)
Enterprise AI integration with business systems such as Odoo ERP, PostgreSQL, and custom MCP services
Pluggable LLM providers (local and remote) : (OpenAI, Gemini, Ollama, Hugging Face Transformers, and local models)
Horisontal Scalable (cloud / on-prem)
Redis based Conversation history (app_id / user_id / session_id)
Qdrant-based vector search with multiple data source options
Qdrant-Hybrid search BM25 (sparse) + Dense embeddings
GraphRAG using Qdrant + Neo4j for hybrid semantic and relationship-based retrieval
Rewrite Prompt Create standalone retrieval query,from (Final Query + short history)
Flexible Ingestion CSV(table field extraction) , TXT (Paragraph-based chunking), PDF, Images and Audio (Speech)
Multi-vector image/pdf retrieval using ColQwen / ColPali
Optional Muvera Fixed-Dimensional Encoding (FDE) for fast image retrieval
Supports multimodal queries: text, image, audio, text+image, text+audio
Audio Search using Whisper + CLAP with Qdrant/Neo4j
Audio Query Support search by uploaded audio file or text
Hybrid Audio Transcript Search BM25 (sparse) + Dense embeddings
Built-in caching system:
- In-memory engine cache (avoid reloading heavy models)
- Optional media/query result cache for image & audio queries
- Configurable cache limits and TTL for performance tuning
Docker-based modular deployment:
- run separate RAG backends per modality (text / image / audio / CSV)
- each backend connected to its own collection or configuration
- Router dynamically dispatches queries to the best backend
- UI can dynamically select the target backend (e.g., text, image, audio, router)
- scalable, isolated, and production-ready architecture
Simple UI for quick testing that uses the Core API / AgnosticRagRouter as its backend

Rewrite Prompt

User query
   ↓
Load last N Redis turns (e.g., 2–3)
   ↓
Rewrite LLM (prompt above)
   ↓
Rewritten query
   ↓
Embedding
   ↓
Qdrant

Rewrite Prompt + GraphRAG Retrieval

User query
   ↓
Load last N Redis turns (e.g., 2–3)
   ↓
Rewrite LLM
   ↓
Rewritten query
   ↓
Embedding
   ↓
Qdrant Semantic Search
   ↓
(Optional) Neo4j Graph Search
   ↓
Merge Semantic + Graph Results
   ↓
LLM Answer

Config-Driven rag_settings.json (models, rerank, chunking, prompts)

{
  "api_base_url": "http://localhost:8000",
  "qdrant_url": "http://localhost:6333",
  "collection_prefix": "ManyImages-",
  "collection_name": "ManyImages-__vidore_colpali-v1.3__multivector_image__dim128__muvera",
  "enable_neo4j_with_qdrant": false,
  "graph_store_provider": "neo4j",  
  "topk_retrieve": 10,
  "topk_use": 5,
  "enable_rerank": false,
  "text_group_lines": 1,
  "txt_chunk_chars": 900,
  "txt_overlap": 120,
  "batch_size": 64,
  "is_paragraph_enable": false,
  "paragraph_end_str": "",
  "include_marker": false,
  "marker_contains": false,
  "enable_structured_csv_lookup": false,
  "enable_exact_filter_first": false,
  "enable_qdrant_hybrid_search_csv": false,
  "use_qdrant_hybrid_search_csv": false,
  "enable_muvera_for_multi_vector_image": true,
  "use_muvera_for_multi_vector_image": false,
  "send_images_to_final_chat_for_multi_vector_image": true,
  "docs_dir": "rag_data\\my_csvs\\docs",
  "rerank_prompt_template": "You are a retrieval re-ranker.\nGiven a user question and a list of candidate contexts, select the most relevant items.\nRules:\n- Choose exactly {choose_k} distinct indices.\n- Prefer contexts that directly contain facts needed to answer.\n- Avoid redundant/duplicate contexts.\n- Output ONLY valid JSON, no extra text.\n\nReturn JSON format:\n{{\n  \"selected_indices\": [0, 2, 5],\n  \"reasons\": [\"short reason 1\", \"short reason 2\", \"short reason 3\"]\n}}\n\nQuestion:\n{query}\n\nCandidates:\n{candidates}",
  "answer_prompt_template": "You are an assistant that answers strictly from retrieved context.\n\nIMPORTANT RULES:\n- Use ONLY the information in the Context.\n- First, check if there is an EXACT match to the Question (all provided key=value pairs) in a single Context row.\n- If there is an exact match:\n  - Output ONLY the value of Initial_ActivityDenialCode from that row.\n- If there is NO exact match but Context is not empty:\n  - Output:\n    Closest match (not exact).\n  - List differing fields (requested vs found).\n  - Then output ONLY the value of Initial_ActivityDenialCode from the closest-match row (the highest score row).\n- If Context is empty:\n  - Reply exactly: \"I don't know based on the provided context.\"\n- Do NOT add any other text.\n\nContext:\n{context}\n\nQuestion:\n{query}\n\nAnswer:",
  "answer_prompt_template_image": "  You are an assistant that answers the user's question based on retrieved image(s) and recent conversation history.\n  \n  The retrieved images are provided as attachments alongside this prompt.\n\n  Rules:\n  - For questions about the images, answer using the visual content of the attached images.\n  - For meta-questions about the conversation (e.g. \\\"what did I just ask?\\\"), answer from the Conversation history below.\n  - If the images do not contain enough information to answer, reply exactly: \\\"I don't know based on the retrieved images.\\\"\n  - Keep the answer clear and concise.\\n\\nConversation history (most recent last):\n  {history}\n  \n  Retrieved image references (for traceability):\n  {context}\n  \n  Question: {query}\n  \n  Answer:",
  "answer_prompt_template_audio": "You are an assistant that answers using BOTH the retrieved context and the recent conversation history.\n\nIMPORTANT RULES:\n- Prefer the Context for factual answers.\n- Use History only for continuity (follow-ups, pronouns).\n- DO NOT repeat the question.\n- DO NOT hallucinate or add information not present in the Context.\n- If the answer is not found in Context, reply exactly:\n  \"I don't know based on the provided context.\"\n\nOUTPUT FORMAT (STRICT):\n\n1) First: provide a clean and concise answer.\n2) Then: append the section EXACTLY as shown below.\n\nAssistant:\n<final answer>\n\nSelected Context:\n<context exactly as provided below, DO NOT modify, DO NOT summarize, DO NOT reorder>\n\n---\n\nHistory (last turns):\n{history}\n\nContext:\n{context}\n\nQuestion: {query}\n\nAnswer:",
  "translate_prompt_template": "You are a translation engine.\n\n    Task:\n    - Translate the user text into natural English.\n    Rules:\n    - Output ONLY the English translation.\n    - No explanations, no quotes, no extra text.\n    - Keep proper nouns, IDs, emails, URLs, and numbers unchanged.\n    - If the text is already English, output it unchanged.\n\n    User text:\n    {text}",
  "rewrite_prompt_template": "You are a query rewriting assistant for a retrieval-augmented generation (RAG) system.\n\nYour job is to rewrite the user's current message into a single, clear, standalone search query\nthat can be used to retrieve documents from a vector database.\n\nRules:\n- Use the conversation only to resolve references (e.g. \"it\", \"that\", \"the second one\").\n- Do NOT introduce new topics.\n- Do NOT answer the user.\n- Do NOT include chatty or conversational text.\n- Output ONLY the rewritten search query.\n- If the user query is already clear and standalone, return it unchanged.\n\nRecent conversation:\n{history}\n\nCurrent user question:\n{query}\n\nRewrite this into one concise standalone retrieval query.",
  "enable_auto_translate": false,
  "enable_rewrite": false,
  "embedding_model": "vidore/colqwen2-v1.0",
  "main_model": "gpt-4o-mini",
  "translate_model": "qwen3:1.7b",
  "rewrite_model": "gpt-4o-mini",
  "main_provider": "openai",
  "embedding_provider": "colqwen",
  "translate_provider": "ollama",
  "rewrite_provider": "openai",
  "transformer_device": "cpu",
  "transformer_trust_remote_code": true,
  "transformer_local_files_only": false,
  "transformer_embedding_trust_remote_code": false,
  "transformer_embedding_local_files_only": false,
  "transformer_embedding_backend": "auto",
  "transformer_embedding_normalize": true,
  "transformer_embedding_max_length": 512,
  "transformer_embedding_prompt_prefix": "query: ",
  "transformer_embedding_model_path": "",
  "transformer_chat_backend": "causal",
  "transformer_chat_model_path": "",
  "transformer_chat_prompt": "{joined}",
  "transformer_max_new_tokens": 128,
  "transformer_temperature": 0.0,
  "transformer_top_p": 0.9,
  "transformer_do_sample": false,
  "transformer_system_prompt": "",
  "transformer_max_input_length": 2048,
  "audio_clap_enable": false,
  "enable_text_transcript_audio_one_search_qdrant": false,
  "audio_clap_model": "laion/clap-htsat-unfused",
  "audio_transcript_embedding_provider": "transformer",
  "audio_transcript_embedding_model": "intfloat/multilingual-e5-small",
  "whisper_language": "en",
  "whisper_beam_size": 5,
  "whisper_vad_filter": false,
  "whisper_vad_threshold": 0.5,
  "whisper_word_timestamps": false,
  "whisper_device": "cpu",
  "whisper_compute_type": "int8",
  "audio_target_sr": 48000,
  "audio_whisper_sr": 16000,
  "silero_min_silence_ms": 250,
  "silero_speech_pad_ms": 150,
  "silero_threshold": 0.4,
  "silero_window_samples": 512,
  "audio_min_chunk_seconds": 1.2,
  "audio_max_chunk_seconds": 20.0,
  "audio_merge_gap_ms": 300,
  "env_path": "keys.env",
  "redis_url": "redis://localhost:6381",
  "redis_store": "redis_store",
  "redis_prefix": "rag",
  "app_id": "rag_api1",
  "session_id": "defaault1",
  "user_id": "local_user1",
  "history_turns": 5,
  "history_rewrite_turns": 2,
  "history_max_turns": 2000,
  "history_ttl_seconds": 604800
}

Config-Driven AgnosticRagRouter router_settings.json (routing, backends, prompts, retries, thresholds)

{
  "router_provider": "openai",
  "router_model": "gpt-4o-mini",
  "router_temperature": 0.0,
  "router_prompt": "You are a routing agent. Choose the best RAG backend for the user query.",
  "routing_mode": "rules_then_llm",
  "fallback_backend": "eyad_asil",
  "allow_multi_backend": false,
  "confidence_threshold": 0.6,
  "llm_call_timeout_seconds": 60.0,
  "llm_call_retries": 1,
  "backend_call_timeout_seconds": 180.0,
  "backend_call_retries": 1,
  "env_path": "keys.env",
  "transformer_device": "cuda",
  "transformer_trust_remote_code": true,
  "transformer_local_files_only": false,
  "transformer_chat_backend": "",
  "transformer_chat_model_path": "",
  "transformer_chat_prompt": "{joined}",
  "transformer_max_new_tokens": 256,
  "transformer_temperature": 0.0,
  "transformer_top_p": 0.9,
  "transformer_do_sample": false,
  "transformer_system_prompt": "",
  "transformer_max_input_length": 2048,
  "openai_api_key": "",
  "gemini_api_key": "",
  "backends": [
    {
      "name": "medical_csv",
      "title": "Medical CSV RAG",
      "url": "http://localhost:8002",
      "query_path": "/query",
      "index_path": "/index/build",
      "description": "Use for medical CSV questions, claims, claim denial codes, diseases, treatments, medicines, patient/insurance/medical table lookups.",
      "source_type": "csv",
      "docs_dir": "rag_data/my_csvs/docs",
      "source_files": [
        "claims_300.csv"
      ],
      "collection_prefix": "medical_csv",
      "enabled": true,
      "rules": {
        "keywords_any": [
          "medical",
          "medicine",
          "doctor",
          "disease",
          "treatment",
          "drug",
          "medicine",
          "patient",
          "claim",
          "claims",
          "denial",
          "denialcode",
          "insurance",
          "سكري",
          "ضغط",
          "مرض",
          "علاج",
          "دواء",
          "طبي",
          "مطالبة",
          "تأمين"
        ]
      },
      "query_stream_path": "/query_stream"
    },
    {
      "name": "eyad_asil",
      "title": "Eyad Asil TXT RAG",
      "url": "http://localhost:8001",
      "query_path": "/query",
      "index_path": "/index/build",
      "description": "Use for questions about Eyad, Asil, family/private TXT knowledge, children info, personal notes in Asil_Eyad.txt.",
      "source_type": "txt",
      "docs_dir": "rag_data/my_csvs/docs",
      "source_files": [
        "Asil_Eyad.txt"
      ],
      "collection_prefix": "eyad_asil",
      "enabled": true,
      "rules": {
        "keywords_any": [
          "eyad",
          "asil",
          "إياد",
          "اياد",
          "أسيل",
          "اسيل",
          "لجين",
          "lujain",
          "family",
          "child",
          "children",
          "عائلة",
          "طفل",
          "أطفال"
        ]
      },
      "query_stream_path": "/query_stream"
    }
  ],
  "routing_function_schema": "{\n  \"function\": \"query_<backend_name>_rag\",\n  \"arguments\": {\"query\": \"original user query\"},\n  \"backend_name\": \"backend_name\",\n  \"confidence\": 0.0,\n  \"reason\": \"short reason\"\n}",
  "routing_rules_prompt": "- Choose medical_csv for medical CSV claims, denial codes, patient/insurance/medical table questions.\n- Choose eyad_asil for Eyad/Asil/private TXT family information.\n- If uncertain, choose the closest backend with low confidence.\n- Keep the original user query unchanged in arguments.query.",
  "routing_prompt_template": "{router_prompt}\n\nYou are routing a user query to exactly one AgnosticRAG backend.\nReturn ONLY valid JSON. Do not use markdown. Do not add explanations outside JSON.\n\nAvailable tools/functions:\n{backend_tools}\n\nAvailable backend descriptions:\n{backend_descriptions}\n\nRequired JSON schema:\n{function_schema}\n\nRules:\n{routing_rules}\n\nUser query:\n{query}\n",
  "stream_media_type": "text/event-stream",
  "cors_allowed_origins": [
    "http://localhost:4200",
    "http://127.0.0.1:4200"
  ],
  "cors_allowed_methods": [
    "GET",
    "POST",
    "OPTIONS"
  ],
  "cors_allowed_headers": [
    "*"
  ]
}

🧠 Architecture Overview

GUI (Web / Desktop)
   ↓
AgnosticRagMCP (Optional)
   ├── Telegram
   ├── WhatsApp
   ├── MS Teams
   ├── Google Drive
   └── External Tools / Drivers
   ↓
AgnosticRagRouter (Optional)
   ↓
Core RAG API (FastAPI)
   ↓
RAG Engine
   ├── LLM Providers (Transformers, vLLM(Later), Ollama, OpenAI, Gemini, ...)
   ├── Vector Store (Qdrant)
   ├── Graph Store (Neo4j)   
   └── Data Sources (Txt, CSV, Images, Audio(Speech))

Core API: Handles retrieval, embeddings, GraphRAG, prompt construction, history, reranking, caching, and inference..
AgnosticRagRouter: Intelligent LLM-based routing and multi-backend orchestration.
AgnosticRagMCP: MCP-based integrations layer for Telegram, WhatsApp, MS Teams, Google Drive, enterprise connectors, and external AI tools/drivers.
GUI (Desktop/Web): Thin client that communicates with the Core API or Router for quick testing and configuration generation.
LLM Providers: Swappable providers selected dynamically through configuration.
Qdrant: Hybrid vector database for dense, sparse, multimodal, and multivector retrieval.
Neo4j: Knowledge graph database for entity relationships, GraphRAG retrieval, and multi-hop reasoning.

🚀 Core RAG API

The Core API is responsible for:

Query understanding
Context retrieval from Qdrant and Neo4j
Hybrid search (Dense + BM25)
GraphRAG retrieval and multi-hop reasoning
Prompt construction
LLM inference
Conversation history management
Query rewriting and reranking
Multi-modal retrieval (text, image, audio)

It can be used:

Directly as a FastAPI backend
As a backend for the provided GUI applications
As a foundation for custom enterprise applications
As a scalable backend for multi-RAG and GraphRAG deployments

🧠 AgnosticRagRouter

AgnosticRagRouter is an intelligent routing layer for multi-backend RAG systems.

It is responsible for:

LLM-based backend selection
Query routing across multiple RAG backends
Multi-domain orchestration
Dynamic backend dispatching
Agentic-AI style routing workflows
Configurable routing strategies and retry logic

Example:

Text RAG backend
CSV / Medical RAG backend
Image RAG backend
Audio / Speech RAG backend

The router can automatically select the most relevant backend depending on the user query.

🔌 AgnosticRagMCP

AgnosticRagMCP is an MCP-based external integrations layer.

It enables external systems and messaging platforms to communicate with AgnosticRagRouter and the Core RAG APIs.

Supported / planned integrations:

Telegram Bots
WhatsApp Bots
MS Teams Bots
Google Drive tools
Enterprise connectors & drivers
External AI tools
MCP orchestration workflows

Example flow:

Telegram / WhatsApp / MS Teams User ↓ AgnosticRagMCP ↓ AgnosticRagRouter ↓ Selected RAG Backend ↓ Qdrant Search + Neo4j Graph Search ↓ Merge Semantic + Graph Context ↓ LLM ↓ Reply to User

🖥️ GUI (Quick Testing)

The GUI is built on top of the Core API and provides:

Interactive chat interface
History-aware conversations
Easy switching between RAG configurations
Runtime configuration generation
Multi-backend testing support

The GUI does not contain RAG logic; it strictly consumes the Core API, this will help admin and developers to setup the deployment configuration quickly.

🔌 LLM Providers

Current / Supported

Ollama (local)
Transformer (local)
OpenAI (remote)
Gemini (remote)
ColPali (multivector image retrieval) (local)
ColQwen (multivector image retrieval) (local)
Whisper (speech/audio transcription) (local)

Embedding / Retrieval Support

Dense embeddings
BM25 sparse embeddings
Hybrid search
Multi-vector retrieval
CLAP audio embeddings
Muvera FDE acceleration for image retrieval

🔮 Incoming Features

Transformers provider & vLLM provider
- Local execution
- Hugging Face model support
- Transformers: Windows / Linux, GPU optional
- vLLM: Best for Ubuntu / Linux production
PostgreSQL Data Source
- Index relational data into Qdrant
- Enables RAG over structured datasets

📦 Vector Store

Qdrant is used as the primary vector database
Neo4j is used as the knowledge graph database for GraphRAG
Supports: Qdrant
- Dense vector search
- Metadata filtering
- Multi-collection setups
- Qdrant-Hybrid search : BM25 (sparse) + Dense embeddings

Neo4j

Entity and relationship storage
Graph traversal and relationship discovery
Multi-hop retrieval
GraphRAG context enrichment
Relationship-aware search

🛠️ Configuration

The system is configured via:

Environment variables
Config files
Provider-specific settings

This allows you to:

Switch LLM providers
Change chat, embedding, translate and rewrite models
Toggle rewrite prompt logic
Select data sources

Start Qdrant & Redis & RedisInsight By Docker `docker-compose.yml`

Start All

docker compose up -d

Stop All

docker compose down

🌐 URLs

Swagger UI: http://localhost:8000/docs
Qdrant Dashboard: http://localhost:6333/dashboard
Neo4j Browser: http://localhost:7474/browser/
RedisInsight: http://localhost:5540/

⚙️ Settings API

GET  http://localhost:8000/settings
POST http://localhost:8000/settings

📥 Index Data

POST http://localhost:8000/index/build

{
}

📡 Qdrant State

GET http://localhost:8000/ping/qdrant
GET http://localhost:8000/ping/qdrantWithInfo

❓ Query Stream API

POST http://localhost:8000/query_stream

{
  "query": "who is here Father?",
  "user_id": "local_user88777"
}

❓ Query API

POST http://localhost:8000/query

{
  "query": "who is here Father?",
  "user_id": "local_user88777"
}

Name		Name	Last commit message	Last commit date
Latest commit History 50 Commits
images		images
README.md		README.md

Folders and files

Latest commit

History

Repository files navigation

AgnosticRag-Q

✨ Key Highlights

🧠 Architecture Overview

🚀 Core RAG API

🧠 AgnosticRagRouter

🔌 AgnosticRagMCP

🖥️ GUI (Quick Testing)

🔌 LLM Providers

Current / Supported

Embedding / Retrieval Support

🔮 Incoming Features

📦 Vector Store

🛠️ Configuration

Start Qdrant & Redis & RedisInsight By Docker docker-compose.yml

🌐 URLs

⚙️ Settings API

📥 Index Data

📡 Qdrant State

❓ Query Stream API

❓ Query API

GUI Windows

Main Window : Search in txt Files

Main Window : Search in csv Files Tables

Main Window : Search in Images by text query

Main Window : Search in Images by image query

Main Window : Search in Audio(Speech) by text/audio_file query

Main Window : Search in TXT and CSV using AgnosticRagRouter

Results Window

Logs Window

History Window

Settings Window

Settings Window

Qdrant Collections

Redis Data

Swagger

Telegram

📄 License

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Start Qdrant & Redis & RedisInsight By Docker `docker-compose.yml`

Packages