feat(base-llm): orchestration multi-modeles (MoA, Triumvirat) et supervision Langfuse#17
feat(base-llm): orchestration multi-modeles (MoA, Triumvirat) et supervision Langfuse#17cluster2600 wants to merge 9 commits into
Conversation
createMoaModel wraps N proposer models + 1 aggregator and conforms to the
existing LanguageModel port ({ id, complete, stream? }), so it drops in
anywhere a single model is expected.
Flow (Wang et al. 2024 MoA, the shape Hermes' `moa` provider uses):
proposers answer the same request in parallel with tools stripped (text
drafts only), then the aggregator synthesizes the drafts — folded into the
system message as private guidance — with the original tools intact so
tool-calling still works through it. Usage is summed across all models.
No ground-truth judge exists in chat, so the aggregator LLM is the selector,
matching Hermes' design. Resilient to partial proposer failures; throws only
if every proposer fails.
Tested with faux/stub models (no API keys): synthesis, tool-stripping,
partial-failure resilience, usage summation, config validation, streaming.
feat(base-llm): Mixture-of-Agents meta-model
createTriumviratModel implements the inference-time architecture of Sakana Fugu / TRINITY (arXiv:2512.04695). The name nods to the three roles it coordinates. Each turn a coordinator picks a model from a swappable pool and assigns it a role (Thinker plans, Worker executes and is the only role given tools, Verifier judges the draft and gates termination), looping until accepted or a turn budget hits. TRINITY's contribution is a CMA-ES-trained 0.6B coordinator, which can't be reproduced as a PR, so the default coordinator is prompted (an LLM choosing model+role+stop) with a deterministic Thinker->Worker->Verifier fallback. The coordinator.decide seam lets a heuristic or trained coordinator drop in later with no API change. Conforms to the LanguageModel port, sums usage, threads the abort signal, recovers from failed role turns. Tested with stub models (no keys): happy path, reject-then-fix feedback, budget exhaustion, tool routing, usage summation, prompted-coordinator JSON path, deterministic fallback, role-failure recovery, abort, config validation.
Add an `ensembles` block to .ai/studio.settings.json so MoA and Triumvirat are configurable without code. resolveModel checks ensembles before the provider path: an ensemble entry names a type (moa | triumvirat) plus member refs, and each member resolves through the existing <provider>/<model> registry, so the ensemble name then works anywhere a model ref does. Bad config (missing aggregator/pool, unknown type, unknown provider member) fails with a clear BAD_REQUEST. Tests cover both ensemble types, a plain provider ref regression, and each error path. Docs with a resolution diagram.
createLangfuseModel wraps any base-llm model and conforms to the same port
({ id, complete, stream? }), so it composes over a single adapter, a MoA, or
a Triumvirat. Every call is traced to Langfuse: input, output, token usage,
latency, and errors.
Zero new dependencies: no Langfuse SDK, just native fetch against the public
ingestion API (POST /api/public/ingestion, HTTP Basic auth). Ingestion is
fire-and-forget so it never adds latency; flush() drains in-flight sends.
A telemetry failure never breaks the wrapped call (routed to onError).
Keys default to env LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY; self-hosted
via LANGFUSE_HOST. Tested with a fake fetch (no network): trace+generation
emission, usage mapping, error traces, ingestion-failure resilience, env
keys, streaming capture, config validation. Docs with a mermaid sequence
diagram.
- packages/base-llm/README.md: a Composition section linking the MoA, Triumvirat, ensembles-settings and Langfuse docs so they are discoverable from the package root. - docs/reference/orchestration-et-supervision.md: a French reference page (authoritative) for the docs site, plus its English mirror under docs/en. Present-tense, em-dash-free, lexicon/translation/tree gates green. Makes the new capabilities visible on the default branch and in the rendered manual once merged.
Orchestration + monitoring: Triumvirat, ensembles, Langfuse, docs
|
Bonjour Maxime, et merci pour cette contribution. Elle suit le port de modèle existant, reste additive et n'introduit aucune dépendance d'exécution; les trois méta-modèles se composent entre eux, les tests tournent sur des modèles factices, et la documentation est bilingue avec diagrammes. Merci aussi d'avoir tenu la matrice des exigences à jour (UR-CORE-003): peu de contributions y pensent. Quelques points à régler avant que nous puissions l'intégrer.
Rien de rédhibitoire sur le fond. Réglons la question de l'egress et les trois points d'hygiène, et nous repassons volontiers en revue. Merci encore. |
cbardyn
added
the
en attente du contributeur
2 participants
Cette contribution ajoute trois meta-modeles a
@ai-swiss/base-llm, tous conformes au port de modele existant ({ id, complete, stream? }), donc utilisables partout ou un modele simple l'est (reglages, Studio, CLI). Aucune nouvelle dependance d'execution.createMoaModel): plusieurs proposeurs repondent en parallele, un agregateur synthetise leurs brouillons. Les proposeurs n'ont pas d'outils, l'agregateur conserve les outils d'origine. L'usage des jetons s'additionne; resilient si certains proposeurs echouent.createTriumviratModel): un coordinateur attribue a chaque tour un role (Penseur, Executant, Verificateur) a un modele d'un vivier interchangeable, en boucle jusqu'a acceptation ou limite de tours (architecture Sakana Fugu / TRINITY, arXiv:2512.04695). Le coordinateur par defaut est guide par invite, avec un repli deterministe; un crochetdecidepermet un coordinateur personnalise..ai/studio.settings.jsonsans code, via un blocensemblesdont les membres sont resolus par le registre<fournisseur>/<modele>existant.createLangfuseModel): enveloppe n'importe quel modele et trace chaque appel (entree, sortie, jetons, latence, erreurs) vers l'API d'ingestion publique viafetch, sans SDK. Envoi en arriere-plan (aucune latence ajoutee),flush()pour vider; un echec de supervision n'interrompt jamais l'appel.Documentation: pages dediees avec diagrammes mermaid (
packages/base-llm/docs/) et une page de reference du site (FR + miroir EN).Verification:
tscpasse; suite de tests verte (le seul echec,base init, preexiste et est sans lien, il echoue aussi sur une copie propre); portes docs et specs vertes. Tests ecrits avec des modeles factices, sans cle API.