Token-Efficient MCP Tool Response Formatting

Token-Efficient MCP Tool Response Formatting

1. Problem Statement

MCP tools currently return pretty-printed JSON (json.MarshalIndent) with all fields including internal IDs, display URLs, and rendering metadata. This is wasteful — agents don’t need fido_id, upc, image_url, or cta_url to reason about products. A single product context response costs 345 tokens; with field pruning and minification this drops to 111 tokens (68% reduction). Across a multi-tool conversation with batch results, this waste compounds into thousands of unnecessary tokens per episode, inflating cost and consuming context window.

2. Background & Context

2.1 Current State

• All MCP tool responses go through jsonResponse() in internal/mcp/tools.go:646 which uses json.MarshalIndent
• Every tool returns the same struct used by the REST API — no agent-specific view
• No instrumentation on token count per tool response
• Current tool response for a product context with 2 offers: 345 tokens

2.2 Key Decisions Already Made

• Field pruning > format switching. Research shows pruning agent-irrelevant fields saves 55% with zero format change, while format switching saves 19-33% with accuracy/maintenance tradeoffs. See docs/token-efficient-tool-responses.md.
• Frontier models are format-agnostic. Benchmarks across 30 models show accuracy approaches 100% for frontier models (Claude, GPT-4o) regardless of format [21]. Optimize for token cost, not format-driven accuracy.
• TOON is the leading alternative format. Outperforms compact JSON in both accuracy (76.4% vs 73.7%) and token efficiency [arXiv 2603.03306]. Go SDK available. Best for uniform arrays (offers, search results).

2.3 Glossary

Term	Definition
TOON	Token-Oriented Object Notation — LLM-optimized format combining YAML-like objects with CSV-like header-driven arrays
TRON	Token Reduced Object Notation — JSON superset with class deduplication (no Go SDK)
Markdown-KV	Non-standard layout with markdown headers and key-value bullet points; highest accuracy format
Field pruning	Removing fields from tool responses that the agent doesn’t need for reasoning
Pass-by-reference	Storing data server-side and returning a pointer instead of serializing into the prompt

3. Requirements

3.1 Functional Requirements

1. The system MUST minify all MCP tool responses (no pretty-printing)
2. The system MUST define per-tool “agent view” field sets that omit rendering/internal fields
3. The system MUST instrument token count (estimated via character count / 4) per MCP tool response
4. The system SHOULD support a format parameter on MCP tools to allow agents to request preferred format
5. The system MAY support TOON format for array-heavy responses (search, offers, candidates)

3.2 Non-Functional Requirements

1. MCP tool response token count for product context MUST be < 150 tokens (currently 345)
2. MCP tool response latency MUST NOT increase by more than 5ms at p99
3. REST API responses MUST remain unchanged

3.3 Acceptance Criteria

• AC-1: Given a get_product_context call, when the response is returned via MCP, then the response is minified JSON with agent-irrelevant fields removed, measuring < 150 tokens
• AC-2: Given any MCP tool call, when the response is returned, then the token count is logged as a metric
• AC-3: Given a search_offers call returning 10 results, when minified + pruned, then token count is < 60% of current pretty JSON baseline
• AC-4: Given any REST API call, when the response is returned, then it is unchanged from current behavior

4. Solution Design

4.1 Architecture

MCP tool call
  → tool handler (same business logic)
    → agentResponse(data, facts)     # NEW: prunes fields + minifies + adds facts
      → json.Marshal(agentView)      # minified, pruned
      → optional: toon.Encode()      # Phase 2, if format=toon requested

REST API call
  → handler (same business logic)
    → jsonResponse(data)             # UNCHANGED: full fields, pretty-printed

The key change is splitting the response path: MCP tools get agentResponse() which prunes and minifies; REST keeps jsonResponse() unchanged.

4.2 Data Model

Per-tool agent view structs. Example for product context:

// AgentProductContext is the pruned view of ProductContext for MCP responses.
type AgentProductContext struct {
    Name       string   `json:"name"`
    Brand      string   `json:"brand"`
    Category   string   `json:"category"`
    EVPoints   int      `json:"ev_points"`
    PPDRate    float64  `json:"ppd_rate"`
    OfferCount int      `json:"offer_count"`
    Stickers   []string `json:"stickers,omitempty"`
}

// AgentOffer is the pruned view of an offer for MCP responses.
type AgentOffer struct {
    Title    string `json:"title"`
    Points   int    `json:"points"`
    Expires  string `json:"expires"`
    Retailer string `json:"retailer"`
}

4.3 API Contracts

No API contract changes. MCP tool response format is content[].text (string) — the string content changes from pretty JSON to minified pruned JSON. The ToolResponse struct is unchanged.

4.4 Key Implementation Details

• agentResponse() helper alongside existing jsonResponse() — uses json.Marshal (not MarshalIndent) and accepts agent-view structs
• Each tool handler defines a toAgentView() method on its response type
• Token count estimation: len(responseText) / 4 is a reasonable approximation for cl100k_base. Log as OTel metric mcp.tool.response_tokens with tool_name attribute.
• Phase 2 TOON: add format param to tool definitions. When format=toon, use toon.Encode() instead of json.Marshal. Only for array-returning tools.

5. Dependencies

5.1 Spec Dependencies

Spec ID	What We Need From It	Why
005-mcp-tool-fact-responses	Fact types and response structure	Facts and pruned fields should ship together — both modify ToolResponse

5.2 External Dependencies

Dependency	Owner	Status	Blocker?
TOON Go SDK	toon-format/toon	Available	No (Phase 2 only)

6. Risks & Open Questions

#	Risk / Question	Impact	Mitigation / Answer
1	Pruned fields turn out to be needed by some agent	Agent gives incomplete answers	Per-tool review of what agents actually reference. Start conservative — prune obvious rendering fields only. Add back if agents request them.
2	TOON comprehension varies by model	Accuracy regression	Only adopt TOON in Phase 2 after measuring Phase 1 savings. TOON benchmarks show 76.4% accuracy — higher than compact JSON.
3	Token count estimation is approximate	Metrics are directional, not exact	len/4 is standard approximation for cl100k_base. Exact counting requires tiktoken which is Python-only. Directional metrics are sufficient.
4	What is TOON?	Answered	Token-Oriented Object Notation. LLM-optimized format. arXiv paper, Go SDK, spec at github.com/toon-format/toon. See research doc.

7. Testing Strategy

• Unit tests: Agent view conversion for each tool response type. Verify pruned structs omit the right fields.
• Integration tests: MCP tool calls return minified responses. REST calls remain pretty-printed.
• Token measurement: Baseline token counts per tool before/after. Include in PR description.
• Manual validation: Run sample queries through agent with pruned responses. Verify agent reasoning quality is unchanged.

8. Rollout & Observability

8.1 Rollout Plan

1. Phase 1 (1 PR): Minification + field pruning for all MCP tools. Token count instrumentation.
2. Phase 2 (1 PR): TOON support for array-heavy tools behind format param. Measure incremental savings.
3. Phase 3 (deferred): Pass-by-reference for assemble_context large payloads.

8.2 Metrics & Alerts

• mcp.tool.response_tokens histogram by tool_name — track before/after
• mcp.tool.response_bytes histogram by tool_name — correlate with tokens
• Alert if average response tokens increases >20% from baseline (regression detection)

8.3 Rollback Plan

Revert agentResponse() calls back to jsonResponse(). One-line change per tool handler.

9. Appendix

Research Documents

• Token-Efficient Tool Response Formats — our benchmarks with tiktoken on CCS sample data
• LLM Data Format Token Reduction Research — comprehensive external research covering TOON, TRON, Markdown-KV, pass-by-reference, and benchmarks across 30 models

Token Baseline (measured 2026-04-14)

Format	Tokens	vs Baseline
Pretty JSON (current)	345	—
Minified JSON	246	-28.7%
Minified + pruned	111	-67.8%
TOON	258	-25.2%
TOON + pruned	116	-66.4%