final: refactor structure, add cache benchmark, update README

Author: mxxcissie

README.md

@@ -1,13 +1,11 @@
 # URL Shortener
 
-A production-style distributed URL shortener with a FastAPI backend and a lightweight React frontend, built using PostgreSQL, Redis, Docker, pytest, and GitHub Actions CI.
-
-This project emphasizes backend engineering beyond basic CRUD, focusing on scalability, performance optimization, fault tolerance, and production-ready system design.
+A production-style distributed URL shortener demonstrating caching, rate limiting, load balancing, and horizontal scalability. Built with a FastAPI backend, React frontend, PostgreSQL, Redis, Docker, and CI/CD (GitHub Actions).
 
 ## Live Demo
 
 - Frontend: https://url-shortener-frontend-av1x.onrender.com  
-- Backend API: https://url-shortener-gfp0.onrender.com
+- Backend API: https://url-shortener-gfp0.onrender.com/docs
 
 ## Frontend
 
@@ -30,7 +28,7 @@ The frontend communicates with the deployed FastAPI backend via REST APIs, enabl
 ## Quick Test
 
 - Frontend:
-Open the web UI: https://your-frontend-url.onrender.com
+  - Open the web UI: https://url-shortener-frontend-av1x.onrender.com
 
 - Backend API:
 ```bash
@@ -45,6 +43,7 @@ curl -X POST "https://url-shortener-gfp0.onrender.com/shorten" \
 This project was built to simulate a production-style distributed system incorporating real-world backend and system design principles, rather than a simple CRUD application.
 
 Key goals:
+
 - Design a scalable API with clear request/response contracts
 - Introduce caching and rate limiting as core system-level concerns
 - Support multiple runtime environments (local, Docker, cloud)
@@ -69,6 +68,24 @@ Key goals:
 - Run the full stack locally using Docker Compose
 - Validate system behavior with pytest and GitHub Actions CI
 - Support horizontal scaling via stateless application instances behind a load balancer
+- Benchmark cache performance (miss vs. hit latency)
+
+## Cache Performance Benchmark
+
+To validate the effectiveness of Redis caching, redirect latency was measured for cache misses (first request) and cache hits (subsequent requests).
+
+Run locally:
+```bash
+python scripts/benchmark_cache.py
+```
+
+Example results:
+
+- Cache miss latency: ~30–45 ms
+- Average cache hit latency: ~6 ms
+- Approximate speedup: ~5–7×
+
+This demonstrates that Redis caching significantly reduces redirect latency and minimizes repeated database queries in read-heavy workloads.
 
 ## Backend Highlights
 
@@ -79,6 +96,7 @@ Key goals:
 - Built automated test coverage with pytest to validate core workflows
 - Configured GitHub Actions CI to run tests on every push and pull request
 - Introduced Nginx as a load balancer to distribute traffic across multiple FastAPI instances
+- Validated Redis caching effectiveness using benchmark measurements (cache miss vs. hit latency)
 
 ## Tech Stack
 
@@ -96,11 +114,30 @@ Key goals:
 ## Project Structure
 
 ```text
-app/                  # FastAPI application code
+app/                  # FastAPI backend application
+  __init__.py
+  main.py             # application entrypoint
+  core/               # application configuration and environment setup
+    __init__.py
+    config.py
+  services/           # external services and infrastructure logic
+    __init__.py
+    cache.py
+    rate_limiter.py
+  crud.py             # database operations
+  database.py         # database connection and setup
+  models.py           # SQLAlchemy models
+  schemas.py          # Pydantic schemas
+  utils.py            # helper utilities
+
 frontend/             # React frontend (Vite, API integration)
 nginx/                # Nginx configuration for load balancing
 tests/                # automated tests
-scripts/              # helper scripts (e.g., seed data)
+
+scripts/
+  seed.py             # sample data loader
+  benchmark_cache.py  # measures cache miss vs. hit latency
+
 docker-compose.yml    # service orchestration
 Dockerfile            # app container definition
 requirements.txt      # backend dependencies
@@ -127,6 +164,7 @@ curl http://127.0.0.1:8000/health
 ## How to Run Locally
 
 ### Run Full Stack with Docker
+
 ```bash
 docker compose up --build
 ```
@@ -135,6 +173,7 @@ Open:
 - Backend health: http://127.0.0.1:8000/health
 
 ### Run Backend Locally (Services in Docker)
+
 - Start required services:
 ```bash
 docker compose up -d db redis
@@ -143,12 +182,17 @@ docker compose up -d db redis
 ```bash
 source venv/bin/activate
 ```
+- Install dependencies:
+```bash
+pip install -r requirements.txt
+```
 - Run backend:
 ```bash
 uvicorn app.main:app --reload
 ```
 
 ### Run Frontend Locally
+
 ```bash
 cd frontend
 npm install
@@ -180,7 +224,8 @@ docker compose up --build
 - Open the returned short URL in your browser to verify redirection
 - Check click statistics using the UI or via: `GET /stats/{short_code}`
 
-### Optional (API-level testing):
+### Optional (API-level testing)
+
 - Open API docs: http://127.0.0.1:8000/docs
 - Create a short URL using `POST /shorten`
 ```json
@@ -196,12 +241,14 @@ This project demonstrates a horizontally scalable, distributed system with a Rea
 The architecture is designed for scalability, fault tolerance, and consistent behavior across multiple application instances.
 
 ### Components
+
 - Nginx load balancer
 - 3 FastAPI application replicas
 - PostgreSQL as the durable, shared source of truth
 - Redis for shared caching and distributed rate limiting
 
 ### System Request Flow
+
 1. User interacts with the React frontend or sends a request directly to Nginx
 2. Nginx routes the request to one FastAPI replica
 3. The selected instance processes the request and interacts with shared PostgreSQL and Redis
@@ -247,12 +294,14 @@ Nginx Load Balancer
 ## Request Flow
 
 ### Create Short URL (`POST /shorten`)
+
 - Request enters the FastAPI service
 - Redis-backed rate limiter validates request frequency
 - A short code is generated and checked for uniqueness
 - Mapping is stored in PostgreSQL
 
 ### Redirect (`GET /{short_code}`)
+
 - FastAPI checks Redis cache for the short code
 - On cache hit → return redirect immediately (low latency)
 - On cache miss:
@@ -262,12 +311,14 @@ Nginx Load Balancer
   - Return redirect response
 
 ### Stats (`GET /stats/{short_code}`)
+
 - Retrieve URL metadata and click count from PostgreSQL
 
 ## Design Notes
 
 - PostgreSQL serves as the single source of truth for URL mappings and analytics, ensuring consistency across all application instances
 - Redis is used as a shared cache layer to optimize read-heavy redirect traffic and reduce database load
+- Cache effectiveness is validated through latency benchmarking, demonstrating faster response times for repeated requests
 - Rate limiting is enforced using Redis to ensure global limits across all replicas, preventing per-instance bypass
 - The application is stateless, allowing any FastAPI instance to handle any request
 - Click counts are updated even on cache hits to maintain consistency between cache and persistent storage

app/__init__.py

@@ -1,6 +1,6 @@
 from sqlalchemy import create_engine
 from sqlalchemy.orm import sessionmaker, declarative_base
-from app.config import DATABASE_URL
+from app.core.config import DATABASE_URL
 
 connect_args = {}
 

app/core/__init__.py

@@ -1,4 +1,5 @@
 import logging
+import time
 from contextlib import asynccontextmanager
 
 from fastapi import FastAPI, Depends, HTTPException, Request
@@ -8,9 +9,11 @@
 from sqlalchemy import text
 
 from app.database import engine, Base, get_db
-from app import schemas, crud, cache
-from app.rate_limiter import check_rate_limit
-from app.config import BASE_URL, INSTANCE_NAME
+from app import schemas, crud
+from app.services import cache
+from app.services.rate_limiter import check_rate_limit
+from app.core.config import BASE_URL, INSTANCE_NAME
+
 
 logging.basicConfig(level=logging.INFO)
 logger = logging.getLogger(__name__)
@@ -26,6 +29,7 @@ async def lifespan(app: FastAPI):
 
 app = FastAPI(title="URL Shortener API", lifespan=lifespan)
 
+
 app.add_middleware(
     CORSMiddleware,
     allow_origins=[
@@ -40,8 +44,17 @@ async def lifespan(app: FastAPI):
 
 @app.middleware("http")
 async def log_requests(request: Request, call_next):
-    logger.info("Instance %s handling %s %s", INSTANCE_NAME, request.method, request.url.path)
+    start = time.perf_counter()
     response = await call_next(request)
+    duration_ms = (time.perf_counter() - start) * 1000
+
+    logger.info(
+        "Instance %s handled %s %s in %.2f ms",
+        INSTANCE_NAME,
+        request.method,
+        request.url.path,
+        duration_ms,
+    )
     return response
 
 
@@ -66,6 +79,7 @@ def db_health():
 def redis_health():
     if not cache.redis_client:
         return {"redis_status": "not_configured"}
+
     try:
         cache_ping = cache.redis_client.ping()
         return {"redis_status": "ok", "ping": cache_ping}
@@ -78,18 +92,20 @@ def redis_health():
 def shorten_url(
     request: Request,
     body: schemas.ShortenRequest,
-    db: Session = Depends(get_db)
+    db: Session = Depends(get_db),
 ):
     check_rate_limit(request)
 
     try:
         db_url = crud.create_short_url(db, body.original_url)
+
         logger.info("Created short URL for %s", body.original_url)
 
         return {
             "short_code": db_url.short_code,
-            "short_url": f"{BASE_URL}/{db_url.short_code}"
+            "short_url": f"{BASE_URL}/{db_url.short_code}",
         }
+
     except Exception as exc:
         logger.exception("Failed to create short URL")
         raise HTTPException(status_code=500, detail="Failed to create short URL") from exc
@@ -106,7 +122,7 @@ def get_stats(short_code: str, db: Session = Depends(get_db)):
         "short_code": db_url.short_code,
         "original_url": db_url.original_url,
         "click_count": db_url.click_count,
-        "created_at": db_url.created_at
+        "created_at": db_url.created_at,
     }
 
 
@@ -117,18 +133,21 @@ def redirect_to_url(short_code: str, db: Session = Depends(get_db)):
 
         if cached_url:
             logger.info("Cache hit for short code %s", short_code)
+
             db_url = crud.get_url_by_code(db, short_code)
             if db_url:
                 crud.increment_click_count(db, db_url)
+
             return RedirectResponse(url=cached_url)
 
         logger.info("Cache miss for short code %s", short_code)
-        db_url = crud.get_url_by_code(db, short_code)
 
+        db_url = crud.get_url_by_code(db, short_code)
         if not db_url:
             raise HTTPException(status_code=404, detail="Short URL not found")
 
         cache.set_cached_url(short_code, db_url.original_url)
+
         crud.increment_click_count(db, db_url)
 
         return RedirectResponse(url=db_url.original_url)

app/config.py app/core/config.pyapp/config.py renamed to app/core/config.py

@@ -1,6 +1,6 @@
 import redis
 import logging
-from app.config import REDIS_URL
+from app.core.config import REDIS_URL
 
 logger = logging.getLogger(__name__)
 

app/database.py

@@ -1,5 +1,5 @@
 from fastapi import HTTPException, Request
-from app.cache import redis_client
+from app.services.cache import redis_client
 import logging
 
 logger = logging.getLogger(__name__)

app/main.py

@@ -0,0 +1,50 @@
+import statistics
+import time
+import requests
+
+BASE_URL = "http://127.0.0.1:8000"
+ORIGINAL_URL = "https://www.google.com"
+HIT_RUNS = 20
+
+def create_short_url() -> str:
+    resp = requests.post(
+        f"{BASE_URL}/shorten",
+        json={"original_url": ORIGINAL_URL},
+        timeout=10,
+    )
+    resp.raise_for_status()
+    data = resp.json()
+    return data["short_code"]
+
+def measure_once(url: str) -> float:
+    start = time.perf_counter()
+    resp = requests.get(url, allow_redirects=False, timeout=10)
+    resp.raise_for_status()
+    end = time.perf_counter()
+    return (end - start) * 1000.0
+
+def main() -> None:
+    short_code = create_short_url()
+    short_url = f"{BASE_URL}/{short_code}"
+
+    miss_ms = measure_once(short_url)
+
+    hit_results = []
+    for _ in range(HIT_RUNS):
+        hit_results.append(measure_once(short_url))
+
+    avg_hit = statistics.mean(hit_results)
+    min_hit = min(hit_results)
+    max_hit = max(hit_results)
+
+    print(f"Short URL: {short_url}")
+    print(f"Cache miss latency: {miss_ms:.2f} ms")
+    print(f"Cache hit avg latency over {HIT_RUNS} runs: {avg_hit:.2f} ms")
+    print(f"Cache hit min latency: {min_hit:.2f} ms")
+    print(f"Cache hit max latency: {max_hit:.2f} ms")
+
+    if avg_hit > 0:
+        print(f"Approx speedup (miss / avg hit): {miss_ms / avg_hit:.2f}x")
+
+if __name__ == "__main__":
+    main()