AI Caller Review Collector

AI Caller Review Collector is a portfolio prototype that turns a customer feedback conversation into a structured product-review record. Sarah, the AI agent, can talk in a browser or place a Twilio outbound call, then saves the transcript and extracted insights for review.

Vision and current position

The goal is to make qualitative product feedback easier to collect and use. Every completed conversation should produce more than a chat transcript: it should yield sentiment, rating, themes, praise, issues, suggestions, and notable quotes.

The core product loop is implemented:

Browser or phone -> speech-to-text -> Sarah's reply -> end call
-> review extraction -> SQLite -> UI or REST API

Implemented	Still to do for production
Browser click-to-talk demo	Authentication and access control
Shared conversation engine	Consent, opt-out, and webhook verification
Gemini replies and end-of-call extraction	Durable multi-worker call state
SQLite review store, API, and review card	Encryption, retention, monitoring, and retries
Twilio outbound calls and Media Streams bridge	Automated end-to-end tests and deployment hardening

This is a functional, single-process prototype, not a production-ready customer-feedback platform.

The product is currently configured for a Lifelong Professional Pickleball Set. Its agent and product defaults are centralized in app/services/product_config.py, making it a foundation for another scripted product-feedback flow rather than a configurable multi-product app.

Features and use cases

• Browser conversations with microphone capture, turn-by-turn transcript updates, spoken replies, and timing metrics.
• Twilio outbound calls with realtime phone transcription and Twilio-compatible audio responses.
• Structured review extraction with sentiment, score, inferred rating, recommendation intent, themes, praise, issues, suggestions, and quotes.
• Local persistence of the review and full transcript in reviews.db.
• REST endpoints to list and retrieve completed reviews.

It is suited to short product-feedback follow-ups, customer research experiments, and demonstrations of a shared AI voice engine across browser and phone channels. It is not intended for emergency, regulated, high-volume call-center, or autonomous sales workflows.

How it works

Component	Role
FastAPI + WebSockets	Serves the UI and realtime browser and Twilio routes.
AssemblyAI	Upload transcription for browser clips and realtime transcription for phone audio.
Gemini 2.5 Flash	Conversation replies and one final structured review extraction.
ElevenLabs	MP3 voice in browsers and ulaw_8000 voice for Twilio.
Twilio	Outbound calls and Media Streams.
SQLite	Stores completed review records.

Both channels use ConversationEngine, which preserves a consistent transcript, prompt, and review shape. The detailed architecture history is in PLAN.md, though its checklist is no longer current.

Quick start: browser demo

Requirements: Python 3.10+, Gemini API key (SECRET_KEY_GOOGLE_AI), ElevenLabs, and AssemblyAI API keys, plus a modern microphone-enabled browser.

git clone https://github.com/jaibhasin/AI-Caller-Review-Collector.git
cd AI-Caller-Review-Collector
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

Set the required keys in .env:

SECRET_KEY_GOOGLE_AI=your_google_ai_key
ELEVEN_LABS_API_KEY=your_elevenlabs_key
ASSEMBLYAI_API_KEY=your_assemblyai_key

uvicorn app.main:app --reload

Open http://localhost:8000, select Start Browser Call, record one or more turns, and select End Call. The extracted review appears in the UI and is saved locally.

Twilio phone setup

Phone calls also need a Twilio account, voice-enabled caller ID, and a public HTTPS URL so Twilio can reach the app. For local development, start the app on all interfaces and expose it through ngrok or Cloudflare Tunnel.

# Terminal 1
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Terminal 2
ngrok http 8000

Add these values to .env, restart the server, then use the phone-call panel with an E.164 destination such as +15551234567.

TWILIO_ACCOUNT_SID=...
TWILIO_AUTH_TOKEN=...
TWILIO_PHONE_NUMBER=+1...
PUBLIC_BASE_URL=https://your-subdomain.ngrok-free.app

API and configuration

Endpoint	Purpose
GET / and GET /api/health	Demo UI and health response.
WS /api/agent/voice	Browser conversation.
POST /api/calls/outbound	Start a call with {"to":"+15551234567"}.
GET /api/twilio/status	Twilio configuration status.
POST /api/twilio/voice, WS /api/twilio/media	Twilio call and audio routes.
POST /api/twilio/status-callback	Finalizes a completed phone review.
GET /api/reviews, GET /api/reviews/{call_id}	List or retrieve reviews.
GET /docs	Interactive API documentation.

Variable	Needed for
SECRET_KEY_GOOGLE_AI, ELEVEN_LABS_API_KEY, ASSEMBLYAI_API_KEY	Browser and phone conversations.
TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_PHONE_NUMBER, PUBLIC_BASE_URL	Phone calls only.

ASSEMBLY_API_KEY remains a backwards-compatible fallback, but new setups should use ASSEMBLYAI_API_KEY.

Important limitations

• Review APIs have no authentication, and stored transcripts and phone numbers have no encryption or retention controls.
• Twilio webhook signature validation and consent or opt-out handling are not yet implemented.
• Active phone calls live in process memory, so restarts and multiple workers are not safe.
• Browser transcription is turn-based upload and polling, not continuous realtime speech recognition.
• There is no queue, durable worker, monitoring, rate limiting, or automated test suite.

The next priority is to validate real calls end to end, then add privacy, security, durable state, and operational safeguards before expanding the feature set.

Troubleshooting

• Use the served app at http://localhost:8000, not frontend/index.html opened directly.
• Check microphone permissions, provider keys and quotas, and server logs for browser failures.
• For Twilio, verify all four Twilio variables, the live public tunnel URL, and Twilio debugger logs.

License

MIT License. See LICENSE.