Skip to content

GovTechSG/oobee-fix

Repository files navigation

Oobee Fix

An LLM-powered pipeline and orchestrator to fix accessibility issues. Self-hosted, bring your own model — works with any OpenAI-compatible endpoint (Ollama, vLLM, LiteLLM, Azure OpenAI, OpenRouter, etc.).

Given code snippets with WCAG accessibility violations, this service returns fixed code with explanations — powered by a LangChain agent with bundled WCAG knowledge and a pre-built RAG knowledge base.

Quickstart

Prerequisites

  • Docker and Docker Compose
  • An OpenAI-compatible LLM endpoint (e.g., Ollama, Azure OpenAI, vLLM)

Setup

# 1. Clone and enter the repo
git clone https://github.com/GovTechSG/oobee-fix.git && cd oobee-fix

# 2. Configure your model endpoint
cp .env.example .env
# Edit .env — set LLM_BASE_URL, LLM_MODEL, and LLM_API_KEY

# 3. Start the services
docker compose up --build

Wait for both services to report healthy, then you're ready.

Usage

curl -X POST http://localhost:3000/api/fixes \
  -H "Content-Type: application/json" \
  -d '{
    "framework": "react",
    "framework_version": 19,
    "build_tool": "webpack",
    "issues": [
      {
        "code_snippet": "<img src=\"logo.png\" width=\"300\" height=\"200\" />",
        "language": "typescript",
        "coordinates": {
          "start": { "line": 10, "column": 4 },
          "end": { "line": 10, "column": 55 },
          "filepath": "src/components/Header.tsx"
        },
        "wcag_clause": "1.1.1"
      }
    ]
  }'

Response

{
  "success": true,
  "issues": [
    {
      "original": {
        "coordinates": { "start": { "line": 10, "column": 4 }, "end": { "line": 10, "column": 55 } },
        "code_snippet": "<img src=\"logo.png\" width=\"300\" height=\"200\" />"
      },
      "fixed": {
        "coordinates": { "start": { "line": 10, "column": 4 }, "end": { "line": 10, "column": 70 } },
        "code_snippet": "<img src=\"logo.png\" width=\"300\" height=\"200\" alt=\"Company logo\" />"
      },
      "filepath": "src/components/Header.tsx",
      "wcagClause": "1.1.1",
      "description": "Added alt attribute to provide text alternative for the image."
    }
  ]
}

Architecture

┌─────────────────────────────────────────────────────────────┐
│  docker compose up                                           │
│                                                              │
│  ┌─────────────────────┐      ┌───────────────────────────┐ │
│  │  Orchestrator         │      │  AI Pipeline               │ │
│  │  Node/Express         │      │  Python/FastAPI            │ │
│  │  Port 3000 (exposed)  │ ───▶ │  Port 8000 (internal)     │ │
│  │                        │      │                            │ │
│  │  • Input validation   │      │  • LangChain agent         │ │
│  │  • Sanitization       │      │  • ChromaDB RAG (pre-built)│ │
│  │  • SQLite cache       │      │  • WCAG docs (bundled)     │ │
│  │  • Response transform │      │  • Evals & guardrails      │ │
│  └────────────────────────┘      └────────────────────────────┘ │
│                                           │                     │
└───────────────────────────────────────────│─────────────────────┘
                                            ▼
                              ┌──────────────────────────┐
                              │  Your LLM Endpoint        │
                              │  (OpenAI-compatible API)   │
                              └──────────────────────────┘

Request Flow

  1. User sends POST /api/fixes with code snippets and WCAG clause IDs
  2. Orchestrator validates input, sanitizes code (strips file paths, data-attributes), checks SQLite cache
  3. Cache miss → transforms payload and forwards to AI pipeline
  4. AI Pipeline runs a LangChain agent per violation:
    • Retrieves relevant docs from ChromaDB (framework patterns, language APIs)
    • Loads WCAG clause documentation (bundled markdown)
    • Calls your LLM via OpenAI-compatible API
    • Runs quality evals (e.g., contrast ratio checks for WCAG 1.4.3)
    • Retries with feedback if eval fails; escalates to fallback model if configured
  5. Response returns → orchestrator maps fixes to source coordinates, caches results, responds to user

The agent validates its own fixes with deterministic checks and re-attempts with feedback when they fail — see SELF_VALIDATION.md for how the self-validation and fallback loop work.

What's Handled Automatically

  • Internal networking between services (Docker DNS)
  • SQLite cache creation and persistence (Docker volume)
  • WCAG 2.0/2.1/2.2 documentation bundled in the image
  • RAG knowledge base pre-built with framework docs (React, Vue, Angular, JS, TS)
  • Health checks and startup ordering
  • Sanitization of sensitive data before sending to LLM
  • Deduplication — identical violations only call the LLM once per batch
  • No telemetry by default — optionally enable Langfuse tracing via .env

Performance

Benchmarked against a golden dataset of 220 React and 230 Angular components with confirmed WCAG violations (26 clauses, all validated by oobee scanner):

Model Pass Rate Avg Latency
Kimi-K2.5 99.1% 16.8s
GPT-5.4 mini 97.7% ~5s
o4-mini 96.4% 24.4s
GPT-5.4 nano 92.3% ~3s

"Pass" = oobee re-scans the fixed code and finds no violations. See DETAILS.md for full methodology, iteration history, and design decisions.

Configuration

All configuration is via a single .env file:

Variable Required Default Description
LLM_BASE_URL Yes OpenAI-compatible API base URL
LLM_MODEL Yes Model name / deployment name
LLM_API_KEY Depends API key (not needed for local Ollama)
AI_SERVICE_TIMEOUT No 60000 Request timeout in milliseconds

Example Configurations

Ollama (local):

LLM_BASE_URL=http://host.docker.internal:11434/v1
LLM_MODEL=llama3.1
LLM_API_KEY=

Azure OpenAI / Foundry:

LLM_BASE_URL=https://your-resource.services.ai.azure.com/openai/v1/
LLM_MODEL=gpt-4o
LLM_API_KEY=your-api-key-here

OpenRouter:

LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_MODEL=anthropic/claude-sonnet-4-5-20250514
LLM_API_KEY=sk-or-...

API Reference

Interactive API docs (Swagger / OpenAPI)

Once the services are running, both expose live API documentation in the browser:

Service Docs URL
Orchestrator (public gateway) Swagger UI http://localhost:3000/api-docs
AI Pipeline (internal service) Swagger UI http://localhost:8000/docs

The orchestrator on port 3000 is the API you call (POST /api/fixes). The AI pipeline docs on port 8000 describe the internal /fix, /fix/batch, and /fix/eval endpoints the orchestrator forwards to.

POST /api/fixes

Submit accessibility issues for AI-powered fixing.

Request Body:

Field Type Required Description
framework string Yes Framework name (react, html, vue, angular)
framework_version number No Framework major version
build_tool string No Build tool (webpack, vite, etc.)
issues array Yes Array of issues to fix (min 1)
issues[].code_snippet string Yes The code containing the violation
issues[].language string Yes Programming language
issues[].coordinates object Yes Source location (start, end, filepath)
issues[].wcag_clause string Yes WCAG clause ID (e.g., 1.1.1, 1.4.3)

Response:

Field Type Description
success boolean Whether all fixes succeeded
issues array Array of fix results
issues[].original object Original code with coordinates
issues[].fixed object Fixed code with updated coordinates
issues[].filepath string Source file path
issues[].wcagClause string WCAG clause that was fixed
issues[].description string Explanation of what was changed

GET /health

Returns 200 OK with service status.

Supported WCAG Clauses

The bundled knowledge base covers WCAG 2.0, 2.1, and 2.2 guidelines. The agent has specialized handling for:

  • 1.1.1 — Non-text Content (alt text, ARIA labels)
  • 1.4.3 — Contrast Minimum (with built-in contrast ratio calculator)
  • 2.4.1 — Bypass Blocks (skip navigation)
  • 2.4.4 — Link Purpose
  • 2.5.3 — Label in Name
  • 3.1.1 — Language of Page
  • 4.1.2 — Name, Role, Value

Other clauses are supported via generic WCAG documentation retrieval.

Knowledge Base (RAG)

The AI pipeline includes a pre-built ChromaDB vector database with ~10,000+ chunks from:

Source Docs Description
React 166 files Component patterns, hooks, accessibility APIs
Vue 83 files Guide, API docs, ARIA patterns
Angular 260 files Components, directives, a11y best practices
JavaScript 1,041 files MDN Web Docs (full JS reference)
TypeScript 75 files Handbook, type definitions, project config
WCAG 87 files Success criteria, techniques, failure patterns

The index is pre-built during the Docker image build using a local embedding model (all-MiniLM-L6-v2). No external embedding service is needed at runtime.

Rebuilding with Custom Embeddings

If you want to use your own embedding model (e.g., from Ollama):

docker compose exec ai-pipeline python scripts/index_docs.py \
  --docs-dir ./docs \
  --output-dir ./chroma-data \
  --embedding-mode openai \
  --embedding-url http://host.docker.internal:11434/v1 \
  --embedding-model nomic-embed-text

Development

# Rebuild after code changes
docker compose up --build

# View logs
docker compose logs -f

# View only AI pipeline logs
docker compose logs -f ai-pipeline

# Stop services
docker compose down

# Reset cache (clear SQLite DB)
docker compose down -v

Project Structure

oobee-fix/
├── docker-compose.yml          # Service orchestration
├── .env.example                # Configuration template
├── orchestrator/               # Node.js API gateway
│   ├── Dockerfile              # Multi-stage (build TS → slim runtime)
│   ├── package.json
│   └── src/
│       ├── middleware/         # Validation, sanitization, transform
│       ├── services/           # AI client, SQLite cache
│       ├── routes/             # Express routes
│       └── types/              # TypeScript interfaces
├── ai-pipeline/                # Python LLM agent
│   ├── Dockerfile              # Multi-stage (index docs → slim runtime)
│   ├── pyproject.toml
│   ├── docs/                   # RAG source docs (React, Vue, Angular, JS, TS, WCAG)
│   ├── wcag/                   # WCAG clause markdown (used at runtime)
│   ├── scripts/
│   │   └── index_docs.py      # ChromaDB indexing script
│   └── src/wcag_fix_agent/
│       ├── agent.py            # LangChain agent with retry logic
│       ├── api.py              # FastAPI endpoints
│       ├── models.py           # LLM provider factory
│       ├── tools.py            # RAG retrieval (ChromaDB/Pinecone)
│       ├── evals/              # Quality evaluation checks
│       └── guardrails/         # Structural guardrails
└── scripts/
    └── setup.sh                # First-time setup helper

License

ISC

About

An LLM-powered pipeline and orchestrator to fix accessibility issues

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors