Research: OpenAI Conversations API + Tool Calls — LangChain Incompatibility

Research: OpenAI Conversations API + Tool Calls — LangChain Incompatibility

Date: 2026-04-23 Status: Blocked — LangChain does not support Conversations API with tool calls Related: Experiment: previous_response_id

Summary

OpenAI’s Conversations API provides server-side conversation state that survives tool calls — solving the previous_response_id chain-breaking problem we identified. However, LangChain/LangGraph does not support it correctly. When tools are involved, LangGraph’s agent loop re-sends items that the Conversations API already stored server-side, causing "Duplicate item found" errors.

This is a known, solved problem in OpenAI’s own Agents SDK (openai-agents-python) — but LangChain has not implemented the fix.

The Problem

When using conversation={"id": conv_id} with the Responses API, each API call’s output items are automatically stored in the conversation. On the next call, you only need to send new items — the server already has the rest.

LangGraph’s agent loop doesn’t know this. When a tool call happens:

1. Call 1: Model returns a function_call item → stored in conversation as fc_xxx
2. LangGraph executes tool locally
3. Call 2: LangGraph sends the full message list including the function_call + tool_result → server sees fc_xxx again → “Duplicate item found” error

The conversation is left in a broken state with an orphan function_call and no function_call_output, making all subsequent calls fail with "No tool output found for function call".

Validated Behavior

Raw OpenAI SDK — Works ✓

When manually managing the tool loop with the raw SDK, the Conversations API works perfectly:

conv = client.conversations.create()

# Turn 1: establish fact
r1 = client.responses.create(
    model="gpt-5.4-mini",
    input=[{"role": "user", "content": "My color is purple, dog is Biscuit"}],
    conversation={"id": conv.id}, tools=tools,
)

# Turn 2: tool call — manually send only the tool output
r2 = client.responses.create(
    model="gpt-5.4-mini",
    input=[{"role": "user", "content": "Echo hello"}],
    conversation={"id": conv.id}, tools=tools,
)
# Handle function_call response...
r2b = client.responses.create(
    model="gpt-5.4-mini",
    input=[{"type": "function_call_output", "call_id": call_id, "output": result}],
    conversation={"id": conv.id},
)

# Turn 3: recall — REMEMBERS ✓
r3 = client.responses.create(
    model="gpt-5.4-mini",
    input=[{"role": "user", "content": "What is my color and dog name?"}],
    conversation={"id": conv.id},
)
# → "Purple, Biscuit" ✓

LangChain + LangGraph — Fails ✗

Through our Agent.stream() wrapper (which uses LangGraph’s create_agent):

model = ChatOpenAI(
    model="gpt-5.4-mini",
    use_responses_api=True,
    model_kwargs={"conversation": conv.id},
)
agent = Agent(model=model, tools=[echo])

# Turn 1: works (no tool call)
# Turn 2: FAILS — "Duplicate item found with id fc_xxx"
# Turn 3: FAILS — "No tool output found for function call"

The conversation parameter IS being passed to the API (confirmed via traceback inspection), but LangGraph’s internal tool loop re-sends items that are already in the conversation.

Direct LangChain model.invoke() — Also Fails ✗

Even without LangGraph, using ChatOpenAI directly with tool binding fails the same way:

model_with_tools = model.bind(conversation={"id": conv.id}).bind_tools([echo])
r2 = model_with_tools.invoke([HumanMessage(content="Echo hello")])
# → "Duplicate item found" on the post-tool call

This confirms the issue is in LangChain’s ChatOpenAI response handling, not LangGraph specifically. When ChatOpenAI processes a tool call response, it constructs the follow-up call with all prior messages including the function_call — which the Conversations API already has.

Known Issue in OpenAI’s Agents SDK

This exact problem was reported and fixed in OpenAI’s own Agents SDK:

Python SDK

• Issue: openai/openai-agents-python#1789 — “Duplicate item found with id fc_xxxx when using conversation_id with function calling” (2025-09-23)
• Fix PR: openai/openai-agents-python#1827 — “fix: Fix multi-turn handling for conversation_id and previous_response_id: only send new items” (merged 2025-10-01)
• Solution: Added _ServerConversationTracker that tracks items already stored server-side and strips them from subsequent calls

JavaScript SDK

• Issue: openai/openai-agents-js#425 — “Agent + tool + conversationId throws invalid request error - Duplicate item found” (2025-09-03)
• Fix PR: openai/openai-agents-js#556 — “fix: improve the compatibility for conversationId / previousResponseId + tool calls” (merged 2025-10-07)

Related Issues

• openai/openai-agents-js#1190 — “run() abort with conversationId leaves orphan function_call items in the conversation store” (2026-04-21, open)
• openai/openai-agents-python#2865 — “_sanitize_openai_conversation_item strips required id from built-in tool-call items” (2026-04-09, closed)

Fix Description (from PR #1827)

The fix in OpenAI’s Agents SDK adds a _ServerConversationTracker that:

1. After each API call, records which items the server now has (from the response output)
2. Before the next API call (internal tool loop or next turn), filters the input items to only include new items not yet stored server-side
3. For conversation_id: strips all previously-sent items, sends only new ones
4. For previous_response_id: strips previously-sent items AND updates previous_response_id to the latest response ID on each internal call

This aligns with OpenAI’s Conversations API design: the server manages state, clients should only send deltas.

LangChain Status

• No native Conversations API support in langchain-openai as of v1.2.0 (2026-04-23)
• No conversation field on ChatOpenAI class — must be passed via model_kwargs or .bind()
• No item deduplication in the tool call loop — LangChain resends all messages on every internal call
• No GitHub issue filed for this specific problem in langchain-ai/langchain or langchain-ai/langchain-openai
• The LangChain forum post suggests using model_kwargs={"conversation": conversation.id} but does not test with tools

Options

Option	Effort	Risk	Notes
Use OpenAI Agents SDK instead of LangChain for agent loop	High	High	Different framework, major refactor, but has native fix
Build custom deduplication wrapper around ChatOpenAI	Medium	Medium	Mirror _ServerConversationTracker from PR #1827
File LangChain issue/PR with the fix	Medium	Low	Community benefit, but timeline uncertain
Wait for LangChain to add support	None	High	No issue exists yet, could be months
Stay with current approach (full history, no Conversations API)	None	None	Works today, just not optimal for tokens

Recommendation

Short-term: Stay with current approach (full message history from DynamoDB). The previous_response_id chain break with tools doesn’t cause quality loss because the history array provides all context. The token overhead is mitigated by ~96% prompt caching.

Medium-term: File a LangChain issue referencing the OpenAI Agents SDK fix (PR #1827) and the duplicate item problem. This puts it on LangChain’s radar and may prompt native support.

Long-term: Evaluate OpenAI Agents SDK as a potential replacement for LangChain’s agent loop, especially if Conversations API support becomes critical for token efficiency or compaction.

Appendix: Versions Tested

Package	Version
langchain	1.2.15
langchain-core	1.3.0
langchain-openai	1.2.0
langgraph	1.1.9
openai	2.32.0

All versions current as of 2026-04-23. The incompatibility exists in the latest releases.