Building an effective AI proxy means not only choosing a powerful language model. As the Manus project found, it is crucial how you design and manage the “context” (information of AI processes to make decisions). This “context engineering” directly affects the speed, cost, reliability and intelligence of the agent.
Initially, the choice was clear: leveraging slow, iterative fine-tuning to leverage the intrinsic learning of the border model. This allows for rapid improvements, transporting changes over hours rather than weeks, allowing the product to adapt to evolving AI capabilities. However, this path is far from simple, thus leading to multiple frameworks reconstruction through affectionately called “random graduate descendants” (a process of experimental guessing).
Here are the key lessons Manus has learned in effective context engineering:
1. Design around KV-CACHE
KV-CACHE is crucial to proxy performance and directly affects latency and cost. The agent continuously adds actions and observations to its context, making the input significantly longer than the output. KV-CACHE reuses the same context prefix, greatly reducing processing time and cost (for example, the cost difference of Claude Sonnet is 10 times).
To maximize KV-CACHE hits:
- Stable prompt prefix: Even a single change can invalidate the cache even when the system prompts to begin. Avoid dynamic elements such as exact timestamps.
- Appendix context only: Do not modify past actions or observations. Ensure deterministic serialization of the data (e.g. JSON) to prevent subtle cache breaks.
- Explicitly cache breakpoints: Some frameworks require manually inserting cache breakpoints after a system prompts.
2. Mask, don’t delete it
As agents gain more tools, their action space becomes complicated and may “fool” the agent when trying to make the right choice. While dynamic tool loading seems intuitive, if the context in the past refers to an undefined tool, it invalidates KV-CACHE and confuses the model.
Manus uses context-aware state machines to manage tool availability Masking token logits During the decoding process. This prevents the model from selecting unavailable or inappropriate operations without changing the definition of the core tool, thus keeping context stable and focusing on agents.
3. Use the file system as context
Even with large context windows (128K+ tokens), real-world proxy observations (such as web pages or PDFs) can easily exceed limits, reducing performance and incur high costs. Irreversible compression risks lose the critical information required for future steps.
Manus treats the file system as the ultimate, infinite context. Agents will use the file system as an externalized, structured memory to read and write files to always be designed to be reliable (e.g., keeping URLs but deleting page content), effectively reducing context length without permanent data loss.
4. Manipulate attention through recitation
Agents may lose focus or forget long-term goals in complex multi-step tasks. Manus solves this problem by having agents constantly rewrite the todo.md file. By reciting its goals and developing to the end of the context, the model’s attention is biased towards its global plan, alleviating the “lost in the middle” problem and reducing target misalignment. This uses natural language to bias the focus of AI without architectural changes.
5. Keep the wrong stuff
Agents make mistakes – hallucinations, encounter mistakes, perform poorly. The natural impulse is to clean up these faults. However, Manus found that leaving failed actions and observations in the context implicitly updates the model’s internal beliefs. Seeing your own mistakes helps agents learn and reduces the chances of repeating the same mistakes, making error recovery a key indicator of true proxy behavior.
6. Don’t get hit hardly any
While the cues of few shooting are powerful for LLM, it can backfire by leading to imitation and suboptimal repetitive behavior. When the context is too uniform and has a similar action attack pair, the agent can fall into the rut, resulting in drift or hallucination.
The solution is Controlled diversity. Manus introduces small changes in serialization templates, wording, or formatting in the context. This “noise” helps break the repetitive pattern and change the model’s attention, thus preventing it from falling into a strict imitation of past actions.
Anyway, context engineering is very new, but it is a key area for AI agents. It goes beyond the original model power, dictating how the agent manages memory, interacts with the environment and learns from feedback. Mastering these principles is crucial to building powerful, scalable, and intelligent AI agents.
Sponsorship Opportunities: Attract the most influential AI developers in the United States and Europe. 1M+ monthly readers, 500K+ community builders, unlimited possibilities. [Explore Sponsorship]
Max is an AI analyst at Marktechpost, based in Silicon Valley, who actively shapes the future of technology. He teaches robotics at Brainvyne, uses comma to combat spam, and uses AI every day to transform complex technological advancements into clear, understandable insights
