Skip to content

feat(mcp_lite): expose channel as no-LLM SDMX primitives over MCP - #510

Draft
bahdan111 wants to merge 8 commits into
developmentfrom
lite-mcp
Draft

feat(mcp_lite): expose channel as no-LLM SDMX primitives over MCP#510
bahdan111 wants to merge 8 commits into
developmentfrom
lite-mcp

Conversation

@bahdan111

@bahdan111 bahdan111 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds mcp_lite — a Model Context Protocol server that exposes a StatGPT channel as low-level, no-LLM SDMX primitives. Instead of the production data-query pipeline (LLM normalise → hybrid retrieve → LLM rerank → finished selection), mcp_lite hands the raw primitives to the agent driving the MCP client, which does discovery, code resolution, availability checks, and fetch itself.

Server is mounted alongside the existing MCP endpoint at
/api/v1/{deployment_id}/mcp-lite/ and shares the running app's channel facade, hybrid index, and auth context — no separate process.

Also ships an accompanying Claude Skill (using-statgpt-mcp-lite) that codifies the recommended agent workflow when driving the server.

Tool surface

Nine data primitives + one eval-only artifact writer. Each tool's own description is the authoritative spec.

tool purpose
list_datasets datasets in the channel
dataset_structure dimensions of one dataset (id, type, codelist size)
sample_dim_values peek / list values of a non-indicator dim
search_indicators cross-dataset concept search → datasets[] groups; each match carries a ready dimensions selection
search_codes atomic dim-value codes within one dataset
availability_query reachable dim values under a partial filter
execute_sdmx_query fetch observations for a resolved key
list_glossary_terms / get_glossary_term channel-specific vocabulary
write_data_query_artifact eval harness only — gated behind EVAL_* env vars

Typical agent loop: search_indicatorssearch_codes / sample_dim_valuesavailability_queryexecute_sdmx_query.

Claude Skill

.claude/skills/using-statgpt-mcp-lite/SKILL.md — a single-file Claude Skill auto-loaded when the statgpt-mcp-lite MCP server is registered for a session. It:

  • describes when the skill applies (quantitative questions about official statistics)
  • lays out the recommended tool-call sequence (locate dataset → resolve codes → verify availability → fetch)
  • surfaces the "one concept commonly lives across several datasets" affordance so the agent inspects search_indicators.datasets[] end-to-end rather than picking the top group only

No dependency on Claude Code specifically — the file is portable to any client that supports the Anthropic Skills format.

Design notes

  • Hybrid retrieval reused. search_indicators uses the same ES BM25 ∥ pgvector convex combination (α = 0.9) as HybridSearcher — just without the LLM normalise / LLM rerank / availability filter stages. Those move into the agent's tool-call loop.
  • Grouped output. search_indicators returns per-dataset groups (datasets[], sorted by best_score desc) rather than a flat matches[] list. Makes multi-dataset coverage structural rather than implicit — the agent can't skip past sibling datasets without reading the group list.
  • Shared facade. All tools take a ChannelServiceFacade via fastmcp.Depends; the channel is resolved from the X-DIAL-Application-Id header, matching the existing MCP endpoint's convention.
  • Timing middleware. Per-tools/call wall-clock is logged to /tmp/mcp_lite_timing.log (override with STATGPT_MCP_LITE_TIMING_LOG). Zero effect on tool payloads.
  • Two new facade methods (in chat_facade.py) let the search primitives narrow by dim_id — search_non_indicator_dimensions_scored(..., dimension_id=...) and search_special_dim_values_by_id(..., dim_id=...). Existing callers unaffected.

Files

  • statgpt/mcp_lite/ — server, schemas, timing middleware, tool modules
  • statgpt/app/application/app_factory.py — mount at /api/v1/{deployment_id}/mcp-lite
  • statgpt/app/services/chat_facade.py — new dim-id-scoped search methods
  • .claude/skills/using-statgpt-mcp-lite/SKILL.md — Claude Skill for driving the server
  • .gitignore — ignore locally-authored experiment notes, eval run artifacts, and the skill's .zip distribution artifact

Not in this PR

  • Eval methodology + comparison tables against the production data-query pipeline

Bahdan Kapionkin added 8 commits May 26, 2026 11:08
Not appropriate for the open-source distribution — HANDOFF and backlog
carry internal absolute paths and dev host names, PLAN references the
internal GitLab tracker. EXPERIMENTS.md is now gitignored so it can be
kept locally as an authoring space without accidentally re-committing.
Auto-loaded by Claude Code when the statgpt-mcp-lite MCP server is
registered for the session. Codifies the recommended workflow:
locate the right dataset(s), resolve dimension codes, verify data
availability, fetch observations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant