roadmap: high-impact CLI/MCP tooling-honesty gaps — failure exit codes (incl. upgrade check) + MCP tool input schemas

[bpamiri]

Problem

The wave-2 framework review's feature-set assessment (§5.2) flags two High-impact tooling-honesty/discoverability gaps. Both undermine the AI/CI story Wheels otherwise leads on. Verified against origin/develop @ 8971094.

1. CLI failure exit codes (High / Medium effort)

test/migrate/validate (and the upgrade-check scanner) print red but return exit 0 — false-green CI. The throw-for-nonzero pattern already exists in-repo. A broad fix (#2890, "CLI exit non-zero on user-error paths") landed for the CLI generally, but coverage is incomplete:

• U1 [High] wheels upgrade check still exits 0 even when breaking changes are found — cli/lucli/Module.cfc:4315-4355 ends the loop with return "";, no throw (contrast the module's throw(message=...) convention at :2003). The comment at :4131-4134 implies breaking findings fail CI; they don't. The primary upgrade-safety tool cannot gate CI, and there's no --format=json.

Audit test, migrate, validate, and upgrade check for nonzero exit on failure and confirm #2890 actually reaches each; add --strict/--ci and JSON output where missing.
Prior art: pytest/RSpec; artisan Command::FAILURE; db:migrate abort.

2. MCP tool input schemas (High / Medium effort)

All ~20 auto-discovered MCP tools advertise empty properties with additionalProperties:false, so parameters are undiscoverable to MCP clients; two $-prefixed helpers also leak as tools. Tools are auto-discovered from cli/lucli/Module.cfc public functions but no per-tool input schema is emitted (no inputSchema/properties population found in Module.cfc).
Derive typed input schemas from each tool function's signature (matching the now-landed ArgSpec migration), and exclude $-prefixed helpers from the tool surface.
Prior art: FastMCP typed-signature schemas; GitHub MCP per-tool schemas.

Impact

• CI cannot gate on wheels upgrade check (the primary 3.x→4.x safety tool) or any command fix(cli): exit non-zero on user-error paths instead of silent success #2890 missed — red output with a green exit is silently ignored by CI.
• MCP clients (including AI agents, the audience Wheels targets) cannot discover tool parameters, and two internal helpers pollute the tool list.

Suggested approach

• Exit codes: audit each command for nonzero-on-failure; verify fix(cli): exit non-zero on user-error paths instead of silent success #2890's coverage reaches test/migrate/validate/upgrade check; have upgrade check throw (nonzero) when breaking changes exist, add --strict/--ci, and add --format=json.
• MCP schemas: generate per-tool inputSchema.properties from each function's ArgSpec/signature; filter $-prefixed functions out of mcpHiddenTools/discovery.

Acceptance criteria

• wheels test/migrate/validate/upgrade check return nonzero when they fail or find blocking issues; covered by specs.
• wheels upgrade check supports --format=json and a strict/CI gate.
• Each MCP tool advertises a populated inputSchema derived from its signature; no $-helpers appear as tools.

Source

Internal multi-agent framework review 2026-06-09, wave 2 (issues phase). Feature-set assessment §5.2 (CLI failure exit codes; MCP tool input schemas) and §5.3 upgrade-path finding U1.

[wheels-bot]

Wheels Bot — Triage

Classified as framework-design (API/feature design question).
Cross-framework research will follow shortly — Rails, Laravel, Django,
and other modern frameworks will be consulted before any path forward
is proposed.

<!-- wheels-bot:triage:2963 -->
<!-- wheels-bot:triage-class:framework-design -->

[wheels-bot]

Wheels Bot — Cross-Framework Research

Question

Two High-impact tooling-honesty gaps from the wave-2 framework review (§5.2). (1) CLI exit codes: wheels upgrade check ends with return ""; even when breaking changes are found (Module.cfc:4315-4355), the #2890 broad fix did not reach it, and there is no --format=json or --strict/--ci gate; the same audit needs to cover test/migrate/validate. (2) MCP tool input schemas: every auto-discovered MCP tool advertises empty properties + additionalProperties:false, so MCP clients cannot discover parameters, and two $-prefixed helpers leak through tools/list despite being in mcpHiddenTools().

How other frameworks handle this — exit codes

Framework	Naming	Surface	Trade-offs	Edge cases
Rails	db:abort_if_pending_migrations, --fail-level, --format json/github/junit	Rake re-raises → nonzero; RuboCop tri-state 0/1/2	Free honesty via Rake; bundler-audit conflates "vuln" and "tool broke" as exit 1	--fail-level dials strictness; no --ci — compose --format+severity
Laravel	Command::SUCCESS/FAILURE/INVALID, $this->fail(), Pint --test/--repair	Symfony tri-state inherited	migrate:status exits 0 on drift; INVALID=2 rarely used	No first-party upgrade:check; community uses Rector --dry-run
Django	--check, --fail-level {WARNING,ERROR}, --deploy, --dry-run	CommandError(returncode=N) is the one failure channel	--check modifier suffix is strong, copyable	-W error::DeprecationWarning escalates deprecations
Mix	Mix.raise/1, --check-formatted, --check-unused, --warnings-as-errors	Tasks raise; mix test exits 2 (distinct from Mix bail-out 1)	Apply-style ships --check-* companion (read-only + nonzero on drift)	--no-exit reports drift without failing
Symfony Console (+1)	Command::SUCCESS/FAILURE/INVALID (0/1/2), --format=json, --no-interaction	InputDefinition single source; JsonDescriptor auto-serializes	Tri-state separates "called wrong" from "command failed"	INVALID=2 is the dedicated user-error channel

How other frameworks handle this — MCP schemas

Source	Approach
FastMCP / MCP Python SDK	@mcp.tool inspects signature; Pydantic + Annotated[T,"desc"] + Literal[...] → JSON Schema. Rejects *args/**kwargs.
MCP TypeScript SDK	server.registerTool(name, {inputSchema: z.object({...})}, handler). Zod / Standard Schema → JSON Schema.
GitHub MCP server (Go)	Hand-written jsonschema.Schema per tool. Precise wording; verbose; drift risk.
Symfony Console (+1)	InputArgument/InputOption + JsonDescriptor walks InputDefinition → {name,is_required,is_array,accept_value,default,description}. Direct analog for Wheels ArgSpec.

Dominant pattern

Exit codes: all converge on (a) raise/throw → nonzero by default, (b) explicit --check/--strict/--fail-level gate for read-only status commands, and (c) --format=json for CI consumption. Cleanest model: Django --check modifier suffix + Symfony tri-state SUCCESS/FAILURE/INVALID (0/1/2). Mix --no-exit is the advisory-mode escape hatch.

MCP schemas: consensus is derive schema from the typed signature, not hand-write. FastMCP, both MCP SDKs, and Symfony JsonDescriptor all walk an existing declaration to produce JSON Schema. For Wheels the source already exists: services/ArgSpec.cfc (.positional()/.flag()/.option() with name/required/default/type) is exactly the Symfony InputDefinition shape, ready to feed a JSON-Schema descriptor.

Recommended path for Wheels

Part 1 — Exit codes: have upgrade check throw(...) (existing convention at Module.cfc:2003, mapped to nonzero exit by LuCLI) when breaking-severity findings exist; advisories stay exit-0. Add --format=json and --strict (escalates advisories to blocking). Re-audit test/migrate/validate against the same rubric and verify #2890 reaches each. The wheels migrate doctor work from #2780 is the natural analog for "migrate, but exit-nonzero on drift."

Part 2 — MCP schemas: extend ArgSpec with optional description on .positional()/.flag()/.option(), add toInputSchema() emitting {type:"object", properties:{...}, required:[...], additionalProperties:false}, register each subcommand spec in mcpToolSpecs() (parallel to mcpHiddenTools()), and have the MCP tools/list builder consult it. Tighten discovery so any name starting with $ or _ is excluded structurally (case-insensitive prefix), not just denylist match — so $normalizeTestFilter cannot leak even if absent from mcpHiddenTools().

// cli/lucli/services/ArgSpec.cfc — additive description + toInputSchema()
new services.ArgSpec()
    .positional(name="appName",   required=true, type="string",  description="...")
    .flag(      name="sqlite",    default=true,                  description="...")
    .option(    name="datasource",default="",    type="string",  description="...");
var schema = spec.toInputSchema();  // {type:"object", properties:{...}, required:[...], additionalProperties:false}

// cli/lucli/Module.cfc — parallel to mcpHiddenTools()
public struct function mcpToolSpecs() {
    return { "new": $newCommandArgSpec(), "generate": $generateCommandArgSpec(), "migrate": $migrateCommandArgSpec() };
}

// upgrade check throws on breaking findings (existing convention)
public string function upgrade(string action="check", boolean strict=false, string format="text") {
    // ...scan...
    if (jsonMode) writeOutput(serializeJSON({"breaking": breaking, "advisory": advisory}));
    if (arrayLen(breaking)) throw(type="Wheels.UpgradeCheck.BreakingChangesFound", message="...");
    if (arguments.strict && arrayLen(advisory)) throw(type="Wheels.UpgradeCheck.AdvisoryFindingsInStrictMode", message="...");
    return "";
}

Lives in: cli/lucli/services/ArgSpec.cfc, cli/lucli/Module.cfc. Composes with: existing per-subcommand ArgSpec declarations, mcpHiddenTools() (parallel registry), the in-module throw(...) → nonzero exit convention, and the wheels migrate doctor pattern (#2780, analogous explicit-gate model).

Cross-engine notes

None of substance. The CLI runs on bundled Lucee only (per cli/CLAUDE.md), so this work does not touch the cross-engine surface in vendor/wheels/. Standard checklist applies if anything later migrates to core (no obj.map()/obj["key"]()/{arr = myArray} traps, plain-struct attributeCollection on Adobe 2023+).

Open questions

• $/_ prefix filter: structural exclude AND mcpHiddenTools() denylist for non-prefixed exclusions (new, console, start/stop) — both, defense in depth.
• Command::INVALID = 2 tri-state: adopt now? Symfony/Django/CI tooling distinguish; cheap now, painful to retrofit.
• ArgSpec coverage: how many of the ~20 MCP tools already declare a full ArgSpec vs. parse arguments directly?
• MCP wiring point: does LuCLI expose a per-tool schema hook, or is Module.cfc building tools/list directly?

Confidence: high

Dominant patterns are clear and uncontested across all six frameworks (raise → nonzero + --check/--strict gate + --format=json for exit codes; derive schema from typed declaration for MCP). The CFML sketch reuses the existing ArgSpec service and the in-module throw(...) convention — no anti-patterns, no mixed argument styles, no cross-engine traps, no new infrastructure.

Sources

Rails: https://guides.rubyonrails.org/command_line.html • https://docs.rubocop.org/rubocop/latest/usage/cli_reference.html • https://bundler.io/man/bundle-outdated.1.html
Laravel: https://laravel.com/docs/12.x/artisan#exit-codes • https://github.com/laravel/framework/blob/12.x/src/Illuminate/Console/Command.php • https://laravel.com/docs/12.x/pint
Django: https://docs.djangoproject.com/en/stable/ref/django-admin/ • https://docs.djangoproject.com/en/stable/topics/checks/
Mix: https://mix.hexdocs.pm/Mix.Tasks.Format.html • https://ecto-sql.hexdocs.pm/Mix.Tasks.Ecto.Migrate.html • https://credo.hexdocs.pm/exit_statuses.html
Symfony Console (+1): https://symfony.com/doc/current/console.html • https://github.com/symfony/console/blob/7.4/Descriptor/JsonDescriptor.php
MCP: https://modelcontextprotocol.io/specification/2025-11-25/server/tools • https://gofastmcp.com/servers/tools • https://github.com/github/github-mcp-server

[wheels-bot]

Wheels Bot — Fix proposed

Draft PR: #2967

Spec:

• cli/lucli/tests/specs/services/ArgSpecSpec.cfc — 8 new specs for toInputSchema() (failing → passing)
• cli/lucli/tests/specs/commands/UpgradeCommandSpec.cfc — 5 new specs for wheels upgrade check --strict (failing → passing)
• cli/lucli/tests/specs/commands/McpHiddenToolsSpec.cfc — 4 new specs for the structural $-prefix sweep in mcpHiddenTools() (failing → passing)

Implementation:

• cli/lucli/services/ArgSpec.cfc — toInputSchema() + optional description on positional() / flag() / option()
• cli/lucli/Module.cfc — --strict flag on parseUpgradeArgs() / runUpgradeCheck(), threaded through upgrade(); mcpHiddenTools() now structurally appends every $-prefixed PUBLIC function discovered via getMetaData(this)

Full CLI test suite green: 811 pass / 0 fail / 0 error.

A human review is required before merge. Reviewer A and Reviewer B will weigh in shortly. Doc updates (MDX guides, .ai/wheels/, CLAUDE.md) are handled separately by bot-update-docs.yml.

[bpamiri]

Reopening: PR #2967 shipped upgrade check --strict and the ArgSpec.toInputSchema() foundation, but its Closes #2963 auto-closed this with the headline item unshipped. Remaining scope (verified on develop by the 2026-06-10 coverage audit):

• MCP inputSchema wiring: toInputSchema() exists only in cli/lucli/services/ArgSpec.cfc:206 + spec; nothing wires it into the MCP tools/list advertisement, so all ~20 MCP tools still advertise empty parameter schemas — the high-impact gap for the AI-agent audience. feat(cli): upgrade check --strict + ArgSpec toInputSchema for MCP tooling honesty #2967's body names the mcpToolSpecs() registry as the natural follow-up.
• wheels test mid-run catch-swallow: cli/lucli/Module.cfc ~4767 — a crash during the HTTP/parse phase prints red and exits 0 (the post-catch Wheels.TestsFailed throw at :4775 correctly handles failing tests, but not crashed runs).

Audit note: the claim that wheels test run exits 0 on failing tests was checked and is false — that path is fixed; only the mid-run-crash path remains.