🧠 IEEE-CIS Financial Fraud Detection

Sequential Behavioural Modelling with 2-Layer GRU + Bahdanau Attention

End-to-end deep learning project — 590,540 real IEEE-CIS banking transactions, PySpark distributed sequence engineering, 2-layer GRU with custom Attention mechanism, attention weight visualisation, and ZAR business impact analysis. Extends the fraud-detection-ml XGBoost project from point-in-time to sequential fraud detection.

---

🔑 Why GRU — not XGBoost again and why this extends the fraud detection portfolio?

The fraud-detection-ml project scores one transaction at a time — a point-in-time decision using tabular features. This project scores the entire customer behavioural sequence — capturing fraud patterns that only emerge over time:

Card testing pattern (invisible to XGBoost):
  T1: R12  Groceries   ← fraudster tests if card is active
  T2: R45  Restaurant  ← small legitimate-looking purchase
  T3: R890 Electronics ← escalating amount
  T4: R8,500 Forex     ← large fraudulent withdrawal
         ↑
    GRU sees this sequence shape — XGBoost only sees T4

GRU maintains a hidden state — a memory that carries information forward through the transaction sequence. The Attention layer then reveals which specific transactions in the sequence most influenced the fraud prediction.

---

🧠 What is a GRU — and why not a stacked LSTM?

A Gated Recurrent Unit (GRU) is a recurrent neural network that maintains a memory (hidden state) across a sequence of transactions. At each timestep it decides what to remember and what to forget using two learned gates: For each transaction timestep t: h(t-1) ← Memory from all prior transactions

1. RESET GATE: sigmoid(Wr × [h(t-1), x(t)]) Determines how much of the past to forget. (0.0 = forget all history, 1.0 = keep all history)

2. UPDATE GATE: sigmoid(Wu × [h(t-1), x(t)]) Determines how much past vs new information to keep.

3. NEW HIDDEN STATE h(t): The updated memory passed to the next transaction timestep. Why GRU over a stacked LSTM for this dataset:

Property	Stacked LSTM	2-Layer GRU
Gates per unit	3 (forget, input, output)	2 (reset, update)
Parameters	Higher (~25% more)	Fewer — faster training
Sequence length suitability	Very long (500+ steps)	Short-medium (10–50 steps)
Overfitting risk on small datasets	Higher	Lower
Training stability	Prone to exploding gradients	More stable with clipnorm
Our median sequence length	4 transactions	✅ GRU optimal

The EDA (Section 2.4) confirmed the median card has only 4 transactions and 90.9% of cards have fewer than 50 transactions. A stacked LSTM with 3 gates per unit would introduce unnecessary parameters and overfitting risk on sequences this short. GRU's simpler 2-gate architecture is precisely calibrated for this sequence length regime — and the results confirm it: ROC-AUC = 0.8611, PR-AUC = 0.6671 (3.70× above random baseline of 0.1805).

---

⚡ Results at a glance

Property	Value
Dataset	590,540 IEEE-CIS real banking transactions
Transaction fraud rate	3.499% (real-world 28:1 imbalance)
Unique cards	13,553
Sequences built	8,419 (cards with ≥ 3 transactions)
Sequence fraud rate	18.03% (cards with any fraudulent transaction)
Sequence shape	(8,419 × 50 timesteps × 30 features)
Architecture	2-Layer GRU + Bahdanau Attention
Test ROC-AUC	0.8611
Test PR-AUC	0.6671 (vs random 0.1805 — 3.70× better)
Optimal threshold	0.08
Net financial benefit	R1,670,000+ per test set
MLflow model	ieee-cis-gru-attention-v1 / Version 1

---

📐 Architecture

590,540 IEEE-CIS Transactions
          │
          ▼  PySpark — Window functions, distributed rolling features
┌──────────────────────────────────────────────────────┐
│  Sequence Engineering                                │
│  Per-card transaction histories sorted by time       │
│  Rolling amount stats · velocity · time gaps         │
│  Cyclical time encoding · categorical encoding       │
└──────────────────────────────────────────────────────┘
          │  (n_cards, seq_len=50, n_features) numpy arrays
          ▼
┌──────────────────────────────────────────────────────┐
│  2-Layer GRU + Bahdanau Attention                    │
│                                                      │
│  Input (50, n_features)                              │
│    → Masking (ignore zero-padded timesteps)          │
│    → GRU(128, return_sequences=True)                 │
│    → BatchNorm + Dropout(0.3)                        │
│    → GRU(64, return_sequences=True)                  │
│    → BatchNorm + Dropout(0.3)                        │
│    → Attention (learns which timesteps matter)       │
│    → Dense(32, relu) + Dropout                       │
│    → Dense(1, sigmoid) — fraud probability           │
└──────────────────────────────────────────────────────┘
          │
          ▼
┌──────────────────────────────────────────────────────┐
│  Evaluation & Explainability                         │
│  ROC-AUC · PR-AUC · F1 · Confusion matrix            │
│  Attention heatmaps — which transactions drove score │
│  ZAR threshold optimisation · MLflow registration    │
└──────────────────────────────────────────────────────┘

---

🚀 Quick start

# 1. Download IEEE-CIS dataset from Kaggle
#    https://www.kaggle.com/competitions/ieee-fraud-detection/data
#    Download: train_transaction.csv + train_identity.csv
#    Place in: data/raw/

# 2. Install dependencies
pip install -r requirements.txt

# 3. Run preprocessing (builds sequences from raw CSVs)
python src/data/preprocess.py

# 4. Launch Jupyter
jupyter lab notebooks/ieee_fraud_gru.ipynb

# 5. Run tests
pytest tests/ -v

# 6. View MLflow experiment
mlflow ui   # → http://localhost:5000

⚠️ Windows + PySpark: Requires Java 17 as JAVA_HOME and PYSPARK_PYTHON set to your Python 3.11 executable.

---

📋 Table of contents

Section	Description	Key output
0	Environment setup	All imports verified
1	PySpark loading	590,540 rows · 434 columns · fraud rate 3.499%
2.1	Class distribution	28:1 imbalance · PR-AUC chosen as primary metric
2.2	Amount distribution	Fraud heavier right tail · log-transform justified
2.3	Fraud by product code	Product-level fraud rate variation
2.4	Sequence lengths	Median=4 · 90.9% covered by seq_len=50
3.1	Temporal features	Cyclical encoding · is_night · is_weekend
3.2	Categorical encoding	StringIndexer via SparkML Pipeline
3.3	Window features	Rolling stats · velocity · time gaps per card
4.1	Sequence features	30 features selected from 434 columns
4.2	Sequence construction	8,419 sequences · (8419 × 50 × 30) · 18.03% fraud
4.3	Split + normalisation	70/15/15 stratified · train-only normalisation
5.1	Model architecture	2-Layer GRU + Attention · ~85k parameters
5.2	Training	50 epochs · EarlyStopping · ReduceLROnPlateau
5.3	Training history	PR-AUC and loss curves — healthy convergence
6.1	Evaluation metrics	ROC-AUC=0.8611 · PR-AUC=0.6671
6.2	ROC & PR curves	Strong early climb · 3.70× above random
6.3	Confusion matrix	Fraud recall strong at default threshold
7.1	Attention extraction	(1000, 50) attention weight matrix
7.2	Local — Card #4	P=0.976 · peak timestep 24 · mid-sequence cluster
7.3	Average attention	Fraud sequences attend more to recent transactions
8	Business impact	Optimal threshold=0.08 · net benefit R1.67M+
9	MLflow logging	ieee-cis-gru-attention-v1 Version 1 registered
10	Final summary	Complete results + four-project portfolio context

---

📊 Dataset

Source: IEEE-CIS Fraud Detection (Kaggle)
Transactions: 590,540 rows × 394 columns
Identity: 144,233 rows × 41 columns
Fraud rate: ~3.5% (real-world imbalanced)
Features: Transaction amounts, card details, device info, Vesta-engineered V/C/D features, email domains

⚠️ The dataset is not committed to this repo — download from Kaggle and place train_transaction.csv and train_identity.csv in data/raw/.

---

🧠 Custom Attention Layer

class AttentionLayer(layers.Layer):
    """
    Bahdanau-style soft attention over GRU hidden states.

    For each timestep h_t:
        e_t = tanh(W_a × h_t + b_a)      ← score
        α_t = softmax(e_t)                ← attention weight
        c   = Σ α_t × h_t                ← context vector

    Returns: context_vector + attention_weights
    The weights show WHICH transactions drove the fraud score.
    """

Attention weights serve as the RNN equivalent of SHAP — providing per-timestep explanations that satisfy FSCA regulatory requirements for explainable automated decisions.

---

🔧 Tech stack

Component	Technology
Distributed processing	PySpark 3.5 (Window functions, rolling features)
Deep learning	TensorFlow 2.x / Keras
Architecture	2-Layer GRU + Custom Bahdanau Attention
Imbalance	class_weight balanced
Experiment tracking	MLflow 2.10
Visualisation	matplotlib, seaborn
Testing	pytest
CI/CD	GitHub Actions

---

📁 Project structure

ieee-fraud-gru/
│
├── 📓 notebooks/
│   └── ieee_fraud_gru.ipynb         # Main notebook — 10 sections
│
├── 📁 src/
│   ├── data/
│   │   └── preprocess.py            # PySpark sequence engineering pipeline
│   ├── models/
│   │   └── gru_attention.py         # GRU + Attention model + callbacks
│   └── evaluation/
│       └── evaluate.py              # Metrics + attention visualisation
│
├── 📁 data/
│   └── raw/                         # Place Kaggle CSVs here (not committed)
│       ├── train_transaction.csv
│       └── train_identity.csv
│
├── 📁 reports/
│   └── figures/                     # Generated charts (not committed)
│
├── 📁 tests/
│   └── test_pipeline.py             # 17 unit tests
│
├── 📁 .github/
│   └── workflows/
│       └── ci.yml                   # GitHub Actions
│
├── .gitignore
├── requirements.txt
└── README.md

---

🔗 Four-project portfolio

#	Project	What it demonstrates
1	idm-debt-pipeline	Medallion batch pipeline — ADF, Databricks, ADLS Gen2
2	nedbank-streaming-pipeline	Real-time ELT — Kafka, PySpark Streaming, Delta MERGE INTO
3	fraud-detection-ml	4-model shootout — XGBoost vs RF vs LR vs MLP, SHAP, MLflow
4	ieee-fraud-gru ← This	Sequential fraud detection — GRU + Attention, PySpark sequences

Projects 3 and 4 form a complementary fraud detection system:

• Project 3 (XGBoost): Scores individual transactions at point-in-time
• Project 4 (GRU): Scores entire customer behavioural sequences

Together they provide defence-in-depth — catching both isolated anomalous transactions AND sequential fraud patterns.

---

📄 License

MIT