Prompt Performance / Lazy Skill Loading Suggestion

[JoKenPo]

Recently I noticed my tokens were running out after only a few messages.

After inspecting the prompts being sent to my provider, I realized the agent was including the full description of every skill in the prompt on every single message exchange.

That resulted in roughly:

• ~140k tokens per message
• Massive context duplication
• Higher costs
• Worse scalability

What I changed

To reduce the overhead, I moved all skills from open-skills into another folder called lazy-skills.

Then I created a lightweight lazy skill inside open-skills containing only:

• the skill name
• a one-line description for each skill

The agent now loads the full skill only when needed.

Result

• ~90% reduction in token usage
• Smaller prompts
• Faster requests
• More scalable conversations

Suggestion

Would it make sense to have a native/vanilla mechanism for this?

Right now, sending the full content of every skill on every request feels extremely inefficient, especially as projects grow.

Some possible alternatives:

• Load full skills only on the first message of the session
• Lazy-load skills on demand
• Use summaries/indexes first, then expand only selected skills
• Cache skills server-side between turns

As projects become larger, prompt orchestration and context management start mattering a lot more than model size itself.

Curious to hear how others are handling this.

[ishita-0301]

That's a smart optimization. Sending the full definition of every skill on every turn doesn't scale, especially as the number of skills grows. A lightweight index with lazy loading seems like a much better default.

It also highlights another challenge with long running agents. Besides context bloat, they can get stuck repeating the same actions or waste tokens retrying failed paths. I've been exploring FailproofAI for that side of the problem since it focuses on runtime reliability and loop detection: https://github.com/FailproofAI/failproofai.

Feels like efficient context management and runtime guardrails go hand in hand for building scalable agents.