Contextual RAG Chatbot

Agentic contextual RAG pipeline: ingest resources → markdown → chunk → embed → pgvector → retrieve → rerank → answer, with ZenML pipelines.

An advanced RAG (Retrieval-Augmented Generation) chatbot that uses contextual chunking, semantic embeddings, and agentic workflows to provide accurate, context-aware responses from your documents.

🚀 Features

Contextual Chunking: Intelligent document chunking with context preservation
Semantic Embeddings: Vector-based document retrieval using pgvector
Agentic Workflows: CrewAI-powered intelligent response generation
MLOps Pipeline: ZenML orchestration for reproducible ML workflows
Real-time Tracking: Phoenix observability for monitoring
FastAPI Integration: RESTful API endpoints for easy integration
OpenWebUI: Modern chat interface for user interaction

📋 Prerequisites

Python 3.13+
PostgreSQL with pgvector extension
Docker (for observability and UI)
uv package manager

⚡ Quick Start

1. Clone the Repository

git clone https://github.com/RaviVaishnav20/Contextual_RAG_Chatbot.git
cd Contextual_RAG_Chatbot

2. Environment Setup

Create your environment file:

cp env.example .env

Add the following to your .env file:

OPENAI_API_KEY=your_openai_api_key_here
CREWAI_TRACING_ENABLED=true

🏗️ Installation & Setup

Phase 1: ZenML Pipeline Setup

Note: There's a dependency conflict between ZenML and CrewAI, so we'll use separate pyproject.toml files for different phases.

Step 1: Initialize ZenML

# Use ZenML pyproject.toml (included in repo)
uv sync

# Initialize ZenML
uv run zenml init
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES uv run zenml login --local

Troubleshooting ZenML Setup: If you encounter port issues:

rm -rf "/Users/ravi/Library/Application Support/zenml/zen_server"
lsof -i :8237
kill -9 <PID>

Step 2: Install PostgreSQL with pgvector

Option 1: Install with Homebrew (Recommended)

brew install postgresql@15 pgvector

Option 2: Build pgvector from Source

brew install make gcc
git clone https://github.com/pgvector/pgvector.git
cd pgvector
make
make install

Step 3: Configure PostgreSQL

# Connect to PostgreSQL
psql -U ravi -d vector_db

# Enable vector extension
CREATE EXTENSION vector;
\dx

Troubleshooting PostgreSQL:

Check if psql is installed:

which psql

If missing, add Postgres to PATH:

nano ~/.zshrc
# Add this line:
export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"
source ~/.zshrc
psql --version

Connect explicitly:

psql -U ravi -d postgres -h localhost
psql -U ravi -d vector_db -h localhost

Phase 2: Data Processing Pipeline

Step 1: Ingest Resources

Convert documents to markdown using Langchain Docling:

uv run python -m tools.run ingest

This processes files from the resources/ folder and saves markdown to data/markdown/.

Step 2: Contextual Chunking

Generate semantic chunks with context:

uv run python -m tools.run chunking

This creates:

metadata_sementic_chunk.json: Initial semantic chunks
metadata_context_chunk.json: Contextually enriched chunks

Step 3: Build Embeddings

Create and store vector embeddings in pgvector:

uv run python -m tools.run embedding

Step 4: Verify Database Setup

psql -U ravi -d vector_db
\dt

Step 5: Test Sample Query

uv run python -m tools.run query "specifically in Article (137)"

Phase 3: Agentic Workflows & API

Step 1: Switch to CrewAI Environment

Replace the pyproject.toml with CrewAI version and sync:

# Replace pyproject.toml with CrewAI version (included in repo)
uv sync

Step 2: Setup Phoenix Observability

Start Phoenix container for real-time tracking:

docker run -d \
  -p 9090:9090 \
  -p 6006:6006 \
  -p 4317:4317 \
  --name phoenix \
  --restart always \
  arizephoenix/phoenix:latest

Verify Phoenix is running:

docker ps --filter "name=phoenix"

Access Phoenix at: http://localhost:6006

Step 3: Start FastAPI Server

Launch the API endpoints:

uv run -m contextual_rag.model.inference.api.main

Access Swagger documentation at: http://localhost:8000

Step 4: Setup OpenWebUI

Setup the chat interface:

# Stop existing container if running
docker stop open-webui && docker rm open-webui

# Start OpenWebUI
docker run -d \
  -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e OPENAI_API_BASE_URL=http://host.docker.internal:8000/v1 \
  -e OPENAI_API_KEY=dummy-key \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Access OpenWebUI at: http://localhost:3000

📁 Project Structure

Contextual_RAG_Chatbot/
├── configs/                    # Configuration files
├── contextual_rag/
│   ├── application/           # Application layer
│   │   ├── agents/           # CrewAI agents and tools
│   │   ├── extractors/       # Document extraction
│   │   ├── preprocessing/    # Data preprocessing
│   │   └── rag/             # RAG implementation
│   ├── infrastructure/       # Infrastructure components
│   └── model/               # ML models and API
├── data/
│   ├── artifacts/           # Generated chunks
│   └── markdown/           # Processed documents
├── pipelines/              # ZenML pipelines
├── resources/              # Source documents
├── steps/                  # Pipeline steps
└── tools/                  # Utility scripts

🔧 Configuration

Database Configuration (configs/config.yaml)

Configure your PostgreSQL connection settings in the config file.

Agent Configuration (contextual_rag/application/agents/crew/config/)

agents.yaml: Define AI agents and their roles
tasks.yaml: Configure agent tasks and workflows

🧪 Testing

Run various tests to verify functionality:

# Test basic functionality
python tests/simple_run.py

# Test RAG components
python tests/test_rag.py

# Test document conversion
python tests/test_document_conversion.py

# Test evaluation metrics
python tests/test_ragas.py

📊 Evaluation

The system includes RAGAS evaluation metrics for assessing RAG performance:

# Run evaluation
python -m contextual_rag.model.evaluation.ragas

🔍 Monitoring & Observability

Phoenix: Real-time tracing and monitoring at http://localhost:6006
ZenML Dashboard: Pipeline tracking and artifact management
FastAPI Metrics: API performance monitoring

🚀 Usage Examples

API Usage

import requests

# Query the RAG system
response = requests.post(
    "http://localhost:8000/rag/query",
    json={"query": "What are the HR policies regarding leave?"}
)
print(response.json())

Direct Python Usage

from contextual_rag.application.rag.rag import RAG

rag = RAG()
answer = rag.answer_question("What are the company bylaws?")
print(answer)

🤝 Contributing

Fork the repository
Create a feature branch
Make your changes
Add tests
Submit a pull request

Name		Name	Last commit message	Last commit date
Latest commit History 20 Commits
configs		configs
contextual_rag		contextual_rag
pipelines		pipelines
resources		resources
scripts		scripts
steps		steps
tests		tests
tools		tools
.env.example		.env.example
.gitignore		.gitignore
.python-version		.python-version
Contextual_RAG_Chatbot.svg		Contextual_RAG_Chatbot.svg
OpenWebUI_and_FastAPI_intergration.docx		OpenWebUI_and_FastAPI_intergration.docx
README.md		README.md
__init__.py		__init__.py
convert_format.py		convert_format.py
crew_pyproject.txt		crew_pyproject.txt
docker-compose.yml		docker-compose.yml
env.example		env.example
main.py		main.py
pycallgraph.png		pycallgraph.png
pyproject.toml		pyproject.toml
uv.lock		uv.lock
zenml_config.py		zenml_config.py
zenml_pyproject.txt		zenml_pyproject.txt

Folders and files

Latest commit

History

Repository files navigation