|
1 | | -# E-commerce Customer Segmentation and LTV Prediction Platform |
| 1 | +# End-to-End MLOps: Predicting Customer Satisfaction |
2 | 2 |
|
3 | | -## Project Goal |
| 3 | +This project demonstrates a complete, end-to-end machine learning pipeline for the Olist e-commerce dataset. The primary goal is to predict customer satisfaction by classifying their potential review score (1-5 stars) based on initial order data. The entire lifecycle, from raw data ingestion to a containerized API, is managed using professional MLOps practices. |
4 | 4 |
|
5 | | -This project aims to build a comprehensive, end-to-end data platform for analyzing customer behavior from the Olist e-commerce dataset. The primary goals are to perform customer segmentation to identify key customer groups and to predict Customer Lifetime Value (LTV). The entire pipeline, from data ingestion to model deployment, follows MLOps best practices. |
| 5 | +[](https://github.com/shaikhilhaam/ecommerce-analytics-platform/actions) |
| 6 | + |
| 7 | +**Docker Hub Image:** [`shaikhilhaam/olist-review-api:latest`](https://hub.docker.com/r/shaikhilhaam/olist-review-api) |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +## Tech Stack |
| 12 | + |
| 13 | +| Category | Technologies | |
| 14 | +| ----------------------- | -------------------------------------------------------------------------------------------------------- | |
| 15 | +| **Data Storage & Querying** | PostgreSQL, SQL | |
| 16 | +| **Data Science & EDA** | Python, Pandas, Jupyter Notebook, Matplotlib, Seaborn, Tableau | |
| 17 | +| **Feature Engineering** | Scikit-learn, Sentence-Transformers (for NLP), Haversine | |
| 18 | +| **Modeling & MLOps** | XGBoost, MLflow (Tracking & Model Registry), Optuna, SHAP | |
| 19 | +| **API & Deployment** | FastAPI, Uvicorn | |
| 20 | +| **Containerization** | Docker | |
| 21 | +| **Automation & CI/CD** | Git, GitHub Actions | |
6 | 22 |
|
7 | 23 | --- |
8 | 24 |
|
9 | | -## Tech Stack Overview |
| 25 | +## Project Pipeline Overview |
| 26 | + |
| 27 | +This project follows a structured, multi-stage pipeline: |
| 28 | + |
| 29 | +1. **Data Engineering:** Raw CSV files are ingested and structured into a **PostgreSQL** database. A comprehensive master table is generated using a single SQL query that joins 9 different tables, performing the heavy data lifting within the database. |
| 30 | + |
| 31 | +2. **Exploratory Data Analysis (EDA):** An extensive analysis revealed key business insights, most notably that **delivery performance (speed and accuracy) is the single biggest driver of customer satisfaction**, and that the platform has an extremely low customer retention rate (~3%). This insight guided the pivot from an LTV model to a more impactful review score prediction model. |
| 32 | + |
| 33 | +3. **Feature Engineering:** A sophisticated feature set was engineered, including: |
| 34 | + * **Logistical Features:** `delivery_time_vs_estimated`, `customer_seller_distance`. |
| 35 | + * **Temporal Features:** Cyclical encoding of month and day of the week. |
| 36 | + * **NLP Features:** State-of-the-art text embeddings from review comments, compressed with PCA. |
| 37 | + * **Seller Features:** Seller's average review score and total order count. |
10 | 38 |
|
11 | | -* **Data Storage:** PostgreSQL |
12 | | -* **Data Analysis & Transformation:** Python (Pandas, NumPy) |
13 | | -* **Experiment Tracking:** MLflow |
14 | | -* **Model Development:** Scikit-learn, PyTorch |
15 | | -* **API Development:** FastAPI |
16 | | -* **Containerization:** Docker |
17 | | -* **Automation (CI/CD):** GitHub Actions |
18 | | -* **Visualization:** Tableau Public |
| 39 | +4. **Modeling & Experiment Tracking:** |
| 40 | + * An **XGBoost** multiclass classifier was trained to predict the 1-5 star review score. |
| 41 | + * **MLflow** was used to manage the entire modeling lifecycle. Experiments were tracked, and the final, best-performing model was versioned and stored in the **MLflow Model Registry**. |
| 42 | + * Model performance was deeply analyzed using a confusion matrix, classification reports, and **SHAP** for expert-level explainability. |
| 43 | + |
| 44 | +5. **Containerization & Deployment:** |
| 45 | + * The registered model is served via a **FastAPI** application, which exposes a `/predict` endpoint. |
| 46 | + * The entire application is containerized using **Docker**, creating a lightweight, portable, and production-ready image. |
| 47 | + |
| 48 | +6. **Automation (CI/CD):** A **GitHub Actions** workflow automates the entire process. On every push to `main`, the workflow automatically builds the Docker image and pushes the `:latest` tag to Docker Hub, ensuring the application is always up-to-date. |
19 | 49 |
|
20 | 50 | --- |
21 | 51 |
|
22 | | -## Project Phases |
| 52 | +## How to Run This Project |
| 53 | + |
| 54 | +### 1. Setup |
| 55 | + |
| 56 | +* Clone the repository. |
| 57 | +* Create a Python virtual environment and install dependencies: |
| 58 | + ```bash |
| 59 | + pip install -r requirements.txt |
| 60 | + ``` |
| 61 | +* Set up a PostgreSQL database and populate it using the `ingest_data.py` script. |
| 62 | +* Create a `.env` file with your database credentials. |
| 63 | + |
| 64 | +### 2. Run the Data & Training Pipeline |
| 65 | + |
| 66 | +* Generate the final modeling dataset: |
| 67 | + ```bash |
| 68 | + python main.py |
| 69 | + ``` |
| 70 | +* Train the model and register it with MLflow: |
| 71 | + ```bash |
| 72 | + python src/train.py |
| 73 | + ``` |
| 74 | + |
| 75 | +### 3. Run the API Locally |
| 76 | + |
| 77 | +* Start the FastAPI server: |
| 78 | + ```bash |
| 79 | + uvicorn api:app --reload |
| 80 | + ``` |
| 81 | +* Access the interactive API documentation at `http://127.0.0.1:8000/docs`. |
| 82 | + |
| 83 | +### 4. Build and Run with Docker |
23 | 84 |
|
24 | | -1. **Data Engineering:** Ingest raw CSV data into a structured PostgreSQL database. Establish schema and table relationships. |
25 | | -2. **Exploratory Data Analysis (EDA) & Segmentation:** Analyze data using SQL and Python. Perform customer segmentation using RFM analysis and clustering (K-Means). Visualize segments in Tableau. |
26 | | -3. **NLP for Review Analysis:** Apply sentiment analysis to customer reviews to create features for product quality and satisfaction metrics. |
27 | | -4. **Predictive Modeling (LTV):** Train and track machine learning models using MLflow to predict future customer value. |
28 | | -5. **Deployment:** Expose the model through a REST API using FastAPI and containerize the application with Docker for production readiness. |
| 85 | +* Build the Docker image: |
| 86 | + ```bash |
| 87 | + docker build -t olist-review-api . |
| 88 | + ``` |
| 89 | +* Run the container: |
| 90 | + ```bash |
| 91 | + docker run -p 8000:8000 olist-review-api |
| 92 | + ``` |
0 commit comments