Deferred Tools in LLMs: How Agents Use Thousands of Tools Without Loading Them All
An LLM agent may have access to a calculator, a browser, a code runner, GitHub, Slack, Jira, Google Drive, a database, and hundreds of internal APIs. Giving the agent more capabilities sounds useful. But there is a practical problem: every tool normally comes with a name, description, and input schema. If I place all of those definitions in every model request, the model has to read a large API catalog before it can answer a simple question. ...