India Runs Data & AI Challenge 2026 — Track 1
Build an AI system that ranks candidates the way a great recruiter would — not by matching keywords, but by actually understanding who fits the role.
https://huggingface.co/spaces/leo-011/redrob-candidate-ranking
Upload a candidates JSON/JSONL file (up to 100 candidates) or run directly with the pre-loaded sample.
Keyword-based ATS filters miss strong candidates because they match words, not meaning. A Search Engineer who built FAISS-based retrieval systems at scale will get filtered out if their resume doesn't say "vector database" — even though that's exactly what they did.
This system ranks candidates the way a great recruiter would: by reading the full picture — career trajectory, actual work done, behavioral signals, and claim credibility — not by counting keyword matches.
The pipeline runs in two stages:
Stage 1 — Retrieval (100K → 300)
Hybrid dense + sparse search (RRF fusion) via Qdrant narrows the full candidate pool to a 300-candidate shortlist. BGE-M3 generates both dense and sparse representations of the job description for retrieval.
Stage 2 — Scoring (300 → 100)
Five independent signals score each shortlisted candidate. The final score is a multiplicative combination tuned via grid search against a self-rated ground truth (NDCG@10 = 1.0).
final_score = rerank^3.0 × structured^1.0 × behavioral^0.2 × consistency × integrity
Signal 1 — Cross-Encoder Reranking
BAAI/bge-reranker-base scores each candidate's career history against a focused description of the JD's core requirement — not the full JD text. Cross-encoders attend jointly over both texts, capturing relevance that bi-encoder cosine similarity misses. The rerank score is cubed to sharpen separation between strong and weak matches.
Signal 2 — Structured Score
Three multiplicative factors: IT-services company background (×0.3 penalty), location relative to preferred hiring cities, and years of experience against the 5-9 year target band.
Signal 3 — Behavioral Score
Recency decay on last-active date, open-to-work flag, recruiter response rate, and interview completion rate. Exponent 0.2 keeps this signal from dominating — activity patterns are noisy proxies for fit.
Signal 4 — Skill-Career Consistency
Three-step check: (1) does the candidate's skills blob match JD-required categories, (2) is each individual skill evidenced in their career history — literal substring match first, embedding similarity fallback second, (3) of the unevidenced skills, which ones are in JD-critical categories. The literal-match-first approach prevents embedding similarity from flagging skills that are literally named in the candidate's career text but diluted by blob averaging.
Signal 5 — Integrity Score
Five independent fabrication checks — skill duration vs tool release date, skill duration vs total career length, graduation year vs implied experience start, expert proficiency vs near-zero duration, and education/career timeline overlaps including backward degree sequencing (Master's starting before a Bachelor's). Collapses to 0.05 only when a single check is decisive on its own, or when two or more independent checks corroborate each other. Redesigned from a single-flag-triggers-collapse approach after discovering a 51% false-positive rate in the original design against the 300-candidate shortlist.
├── app/
│ ├── pipeline.py # shared logic — RankerResources class + all signal functions
│ └── rank.py # thin CLI wrapper — argparse, calls pipeline.py, writes CSV
├── notebook/
│ ├── redrob_ranking_final_pipeline.ipynb # validated end-to-end notebook
│ └── embeddings/ # precomputed vectors (not in git — see setup)
├── data/
│ └── job_description.txt
├── submission/
│ └── submission.csv
└── README.md
1. Clone the repo
git clone https://github.com/AF011/redrob-intelligent-candidate-ranking
cd redrob-intelligent-candidate-ranking2. Create environment and install dependencies
pip install -r requirements.txt3. Download precomputed embeddings
Precomputed vectors and the Qdrant index (~1.61 GB) are hosted on HuggingFace: leo-011/redrob-embeddings
from huggingface_hub import snapshot_download
snapshot_download(
repo_id="leo-011/redrob-embeddings",
repo_type="dataset",
local_dir="./notebook/embeddings"
)This downloads:
candidate_ids.jsonskills_text_dense_vecs.npycareer_text_dense_vecs.npyqdrant_candidates_vec_db/— pre-built hybrid search index (100K candidates, dense + sparse)
4. Download model weights (first run only, requires internet)
Both models are pulled from HuggingFace Hub automatically on first run and cached locally. After that, no network access is required.
BAAI/bge-m3(~1.1 GB)BAAI/bge-reranker-base(~280 MB)
cd app
python rank.py --candidates ../data/candidates.jsonl --out ../submission/submission.csvExpected output:
[1/6] Loading BGE-M3... done (~9s)
[2/6] Loading reranker... done (~9s)
[3/6] Connecting to Qdrant... done (~70s)
[4/6] Loading precomputed vectors... done (~1s, 100000 candidates)
[5/6] Embedding job description... done (~10s)
[6/6] Embedding JD categories... done (~2s)
[1/5] Retrieving candidates (hybrid search, 100K -> 300)... done (~30-70s, 300 candidates loaded)
[2/5] Reranking against JD core requirement... done (~50-70s)
[3/5] Scoring structured, behavioral, consistency, integrity signals... done (~30-65s)
[4/5] Ranking and selecting top candidates... done (0.0s)
[5/5] Generating reasoning... done (0.0s)
Wrote 100 rows to ../submission/submission.csv
Total runtime (incl. model load): ~262s (budget: 300s)
Note: The
candidates.jsonlfile (full 100K dataset) is not included in this repo. The reproduction command assumes it is present atdata/candidates.jsonl.
Why cross-encoder reranking instead of bi-encoder cosine similarity for Signal 1?
Bi-encoder cosine similarity compares two independently-encoded vectors — the JD and the candidate — without any cross-attention between them. For a focused technical JD, this produces near-zero discriminative power between strong and weak candidates (NDCG@10 was 0.54 with cosine similarity). Cross-encoders attend jointly over both texts, which is fundamentally more expressive at the cost of higher compute. At 300 candidates, bge-reranker-base adds ~50s and moves NDCG@10 from 0.54 to 1.0.
Why a focused core-requirement description rather than the full JD for reranking?
The full JD is 8600+ characters including culture, logistics, and benefits text. Using it for reranking dilutes the signal — a candidate who mentions "remote-friendly" scores higher than one who built a production FAISS index. The focused description strips the JD to its irreducible technical ask.
Why literal substring matching before embedding similarity in Signal 4?
Embedding similarity between a skill name and a career blob can fall below the evidence threshold even when the skill is literally named in the career text — the skill's embedding gets diluted by averaging over hundreds of words in the career blob. Checking for literal presence first prevents false "unevidenced" flags on skills that are unambiguously present.
Why corroboration-based integrity rather than single-flag collapse?
The original single-flag design collapsed 51% of the 300-candidate shortlist to integrity score 0.05 — far above the ~80 honeypots stated for the full 100K dataset. Auditing showed Check A (skill duration vs tool release date) fires on overage magnitudes from 1 to 56 months with no clean separation between fabricated and legitimate early-adopter claims. The redesigned hierarchy requires either a single decisive check (e.g. expert proficiency with <12 months duration, or a Master's starting before any Bachelor's) or two independent corroborating checks. This dropped the flagged rate from 51% to 32% while keeping the confirmed honeypot (CAND_0002025) correctly caught via two-check corroboration.
Why these exponents in the final formula?
The exponents were found via grid search against a self-rated ground truth on the 300-candidate shortlist. The intuition behind each:
| Signal | Exponent | Reason |
|---|---|---|
| rerank_score | 3.0 | Sharpens separation between strong and weak career matches — small differences in relevance score become meaningful ranking gaps |
| structured_score | 1.0 | Linear, direct impact — location, company type, and experience band are clean signals with no need for amplification |
| behavioral_score | 0.2 | Noisy proxy for fit — a strong candidate can be inactive for unrelated reasons, so this signal is dampened to avoid dominating the formula |
| consistency, integrity | 1.0 | Near-binary gates (1.0 or 0.05/0.4/0.7) — raising a near-binary value to any exponent has negligible effect, so tuning was only applied to the three continuous signals |
