Skip to content

Commit 4701149

Browse files
danbarrdanbarr
authored
Add concept guide for optimizing LLM context (#373)
Signed-off-by: Dan Barr <6922515+danbarr@users.noreply.github.com> Co-authored-by: Dan Barr <6922515+danbarr@users.noreply.github.com>
1 parent 18da3c5 commit 4701149

File tree

3 files changed

+261
-1
lines changed

3 files changed

+261
-1
lines changed

‎docs/toolhive/concepts/groups.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Organizing MCP servers with groups
3-
sidebar_label: Groups
3+
sidebar_label: Organizing MCP servers
44
description:
55
Understanding when and why to use groups for organizing MCP servers and
66
controlling client access.

‎docs/toolhive/concepts/tool-optimization.mdx‎

Lines changed: 259 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,259 @@
1+
---
2+
title: Optimizing LLM context with tool filtering and overrides
3+
sidebar_label: Optimizing LLM context
4+
description:
5+
Understanding when and why to use tool filtering and overrides to reduce
6+
context pollution and improve AI performance.
7+
---
8+
9+
When AI assistants interact with MCP servers, they receive information about
10+
every available tool. While having many tools provides flexibility, it can also
11+
create problems. This guide explains how tool filtering and overrides help you
12+
optimize your AI's context window for better performance and more focused
13+
results.
14+
15+
## The context pollution problem
16+
17+
Modern AI clients work by analyzing all available tools and selecting the most
18+
appropriate one for each task. This selection process happens in the AI model's
19+
context, which means every tool's name, description, and schema consumes tokens
20+
and processing time.
21+
22+
Consider what happens when you connect an AI client to multiple MCP servers:
23+
24+
- A GitHub server might expose 30+ tools for repositories, issues, pull
25+
requests, and more
26+
- A filesystem server adds another 10+ tools for file operations
27+
- A database server contributes 20+ tools for queries and schema management
28+
- Additional servers for Slack, Jira, monitoring systems, and other integrations
29+
30+
Before you know it, your AI client is evaluating 100+ tools for every request.
31+
32+
### Why this matters
33+
34+
When your AI receives too many tools, several problems emerge:
35+
36+
**Performance degradation**: More tools mean longer processing time as the AI
37+
model evaluates each option. Tool selection becomes a bottleneck, especially for
38+
complex queries.
39+
40+
**Higher costs**: Every tool description consumes tokens. In token-based pricing
41+
models, exposing unnecessary tools directly increases your costs for every AI
42+
interaction.
43+
44+
**Reduced accuracy**: When faced with many similar tools, AI models sometimes
45+
choose incorrectly. A client might use a production database tool when it should
46+
use a development one, or select a write operation when a read would suffice.
47+
48+
The solution is selective tool exposure - showing your AI only the tools it
49+
actually needs.
50+
51+
## Tool filtering
52+
53+
Tool filtering restricts which tools from an MCP server are available to
54+
clients. Think of it as creating a curated subset of functionality for specific
55+
use cases.
56+
57+
### How filtering works
58+
59+
ToolHive uses an allow-list approach. When you specify a filter, only the tools
60+
you explicitly list become available. The filtering happens at the HTTP proxy
61+
level, so:
62+
63+
- The AI only sees allowed tools in its tool list
64+
- Attempts to call filtered tools result in errors
65+
- The backend MCP server remains unchanged
66+
67+
An empty filter means all tools are available. Once you add any tool to the
68+
filter, only listed tools are exposed.
69+
70+
### When to use filtering
71+
72+
Filtering makes sense in several scenarios:
73+
74+
#### Improving AI tool selection
75+
76+
When an MCP server exposes many tools but you only need a subset, filtering
77+
improves the AI's ability to choose correctly. For example, enable only pull
78+
request tools from the GitHub server when doing code review, or limit a file
79+
system server to just `read_file` and `list_directory` for a documentation
80+
assistant. Removing irrelevant options helps the AI make more confident
81+
selections.
82+
83+
#### Limiting access to safe operations
84+
85+
An MCP server for database access might include both read and write operations.
86+
During development or analysis, you might want to expose only read operations
87+
like `query` and `list_tables`, while filtering out write operations like
88+
`insert`, `update`, and `delete` that modify data or perform destructive
89+
operations.
90+
91+
#### Creating role-specific tool sets
92+
93+
Different team members need different capabilities. Junior developers might get
94+
filtered access to safe operations, while senior developers see the full tool
95+
set. Security-sensitive tools like deployment commands might be filtered for
96+
most users but available to DevOps engineers.
97+
98+
#### Compliance and governance
99+
100+
When organizational policies restrict certain operations, you can enforce policy
101+
by only exposing approved tools, even if the underlying MCP server provides more
102+
capabilities.
103+
104+
## Tool overrides
105+
106+
Tool overrides let you rename tools and update their descriptions without
107+
modifying the backend MCP server. This is particularly valuable when tool names
108+
are unclear or when combining multiple servers.
109+
110+
### How overrides work
111+
112+
Overrides maintain a bidirectional mapping between original and user-facing
113+
names. When your AI sees the tool list, it receives the overridden names and
114+
descriptions. When it calls a tool, ToolHive translates the user-facing name
115+
back to the original name for the backend server.
116+
117+
You can override either the name, the description, or both for each tool.
118+
119+
### When to use overrides
120+
121+
Overrides solve several common problems:
122+
123+
#### Preventing name conflicts
124+
125+
When combining multiple MCP servers through Virtual MCP Server or running
126+
similar servers for different purposes, naming conflicts are common. Both GitHub
127+
and Jira might have a `create_issue` tool. Overriding these to
128+
`github_create_issue` and `jira_create_issue` eliminates ambiguity.
129+
130+
When you run the same MCP server multiple times with different configurations,
131+
tool names become identical. For example, running the GitHub server twice (once
132+
for your company's organization and once for open source contributions) requires
133+
renaming tools to `github_company_create_pr` and `github_oss_create_pr` to make
134+
the distinction clear.
135+
136+
#### Adding context and clarity
137+
138+
Tool names and descriptions can be improved to provide environment-specific or
139+
use-case-specific information. Renaming `deploy` to `deploy_to_staging` versus
140+
`deploy_to_production` makes the destination explicit and reduces mistakes.
141+
Similarly, you can update descriptions from generic text like "Deploy
142+
application" to specific guidance like "Deploy to staging environment -
143+
auto-rollback enabled."
144+
145+
## Combining filters and overrides
146+
147+
Filtering and overrides work together, but understanding their interaction is
148+
important: **filters apply to user-facing names after overrides**.
149+
150+
This means when you override a tool name, you must use the new name in your
151+
filter list, not the original name.
152+
153+
### Pattern: Secure subset with clear names
154+
155+
Start by overriding technical names to be more intuitive, then filter to only
156+
safe operations.
157+
158+
```json
159+
{
160+
"toolsOverride": {
161+
"exec_raw_sql": {
162+
"name": "run_database_query",
163+
"description": "Execute read-only SQL queries against the staging database"
164+
},
165+
"write_table": {
166+
"name": "update_database",
167+
"description": "Modify staging database tables (use with caution)"
168+
}
169+
}
170+
}
171+
```
172+
173+
Then filter using the new names:
174+
175+
```bash
176+
thv run --tools-override overrides.json --tools run_database_query my-db-server
177+
```
178+
179+
**Why this works**: Clear names guide the AI while filtering enforces safety by
180+
making destructive operations unavailable.
181+
182+
### Pattern: Environment-specific configurations
183+
184+
Different environments need different tool access. In development, you might
185+
expose many tools for flexibility. In production, filter to essential tools
186+
only.
187+
188+
Your development configuration could expose all tools with friendly names
189+
through overrides. Your production configuration uses the same overrides for
190+
consistency but adds strict filtering to expose only read and monitoring tools,
191+
blocking any write or deployment operations.
192+
193+
**Why this works**: Consistent naming across environments with
194+
environment-specific filtering prevents accidents while maintaining flexibility.
195+
196+
### Pattern: Multi-server aggregation
197+
198+
When using [Virtual MCP Server](vmcp.mdx) to combine multiple MCP servers,
199+
overrides prevent conflicts and improve clarity:
200+
201+
```json
202+
{
203+
"toolsOverride": {
204+
"search": {
205+
"name": "github_search",
206+
"description": "Search GitHub repositories and code"
207+
}
208+
}
209+
}
210+
```
211+
212+
You can override the `search` tool from different servers to `github_search`,
213+
`jira_search`, and `confluence_search`. Then filter each server to its relevant
214+
tools, creating a clean, conflict-free tool set.
215+
216+
**Why this works**: Prefixes eliminate ambiguity about which server a tool
217+
targets, while filtering prevents context overload from aggregating many
218+
services.
219+
220+
## Trade-offs to consider
221+
222+
While these optimization features provide significant benefits, they also
223+
introduce complexity:
224+
225+
**Configuration and maintenance overhead**: Filters and overrides require
226+
ongoing maintenance. When MCP servers update their tools, you'll need to adjust
227+
your configurations. When using both features together, remember that filters
228+
apply to overridden names, not original names.
229+
230+
**Flexibility vs. safety**: Aggressive filtering makes it harder to access tools
231+
you occasionally need. You may find yourself creating exceptions or
232+
reconfiguring access. The more you optimize, the less flexible your system
233+
becomes.
234+
235+
**Discovery and documentation**: When tools are filtered or renamed, team
236+
members may not realize what capabilities exist. Clear documentation becomes
237+
essential when your visible tools don't match what the MCP server actually
238+
provides.
239+
240+
Start simple and add complexity only where it provides clear value.
241+
242+
## Best practices
243+
244+
**Start minimal**: Begin with a focused tool set and expand as you discover
245+
needs, rather than filtering down from everything.
246+
247+
**Be specific**: When overriding names and descriptions, provide clear,
248+
context-specific information that helps the AI understand when to use each tool.
249+
250+
## Related information
251+
252+
Now that you understand when and why to use tool filtering and overrides, learn
253+
how to configure them:
254+
255+
- [Customize tools (CLI)](../guides-cli/run-mcp-servers.mdx)
256+
- [Customize tools (UI)](../guides-ui/customize-tools.mdx)
257+
- [Customize tools (Kubernetes)](../guides-k8s/customize-tools.mdx)
258+
- [MCPToolConfig CRD reference](../reference/crd-spec.mdx)
259+
- [Virtual MCP Server tool aggregation](../guides-vmcp/tool-aggregation.mdx)

‎sidebars.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -214,6 +214,7 @@ const sidebars: SidebarsConfig = {
214214
items: [
215215
'toolhive/concepts/mcp-primer',
216216
'toolhive/concepts/groups',
217+
'toolhive/concepts/tool-optimization',
217218
'toolhive/concepts/registry-criteria',
218219
'toolhive/concepts/observability',
219220
'toolhive/concepts/auth-framework',

0 commit comments

Comments
 (0)