Checked other resources
Reproduction Steps / Example Code
import operator
from typing import Annotated
from langgraph.channels.untracked_value import UntrackedValue
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import START, StateGraph
from langgraph.types import Send
from typing_extensions import TypedDict
class RuntimeResource:
def __init__(self, name: str):
self.name = name
class State(TypedDict):
messages: Annotated[list[str], operator.add]
resource: Annotated[RuntimeResource, UntrackedValue]
attempts = 0
def setup(state: State):
return {
"messages": ["setup"],
"resource": RuntimeResource("runtime-secret"),
}
def route_to_worker(state: State):
return [Send("worker", state)]
def worker(state: State):
global attempts
attempts += 1
# Present on the first execution, missing after checkpoint resume.
resource = state.get("resource")
print("attempt", attempts, "resource", resource)
if attempts == 1:
assert resource is not None
raise ValueError("intentional-worker-failure")
assert resource is not None, "resource was lost on resume"
return {"messages": [f"used {resource.name}"]}
builder = StateGraph(State)
builder.add_node("setup", setup)
builder.add_node("worker", worker)
builder.add_edge(START, "setup")
builder.add_conditional_edges("setup", route_to_worker)
graph = builder.compile(checkpointer=InMemorySaver())
config = {"configurable": {"thread_id": "send-untracked-resume"}}
try:
graph.invoke({}, config)
except ValueError:
pass
snapshot = graph.get_state(config)
print("checkpointed values:", snapshot.values)
print("next:", snapshot.next)
# `resource` is correctly absent because it is UntrackedValue.
assert "resource" not in snapshot.values
assert snapshot.next == ("worker",)
# The failed dynamic Send task is retried, but its original Send input
# can no longer be reconstructed because the UntrackedValue was not persisted.
graph.invoke(None, config)
Actual behavior
First execution:
attempt 1 resource <RuntimeResource ...>
After the injected failure:
checkpointed values: {'messages': ['setup']}
next: ('worker',)
On resume:
attempt 2 resource None
AssertionError: resource was lost on resume
The failed worker task is still considered resumable (state.next == ('worker',)), but the input it receives after resume is structurally different from the original Send input because the UntrackedValue is gone.
Expected behavior / contract question
I understand that UntrackedValue is intentionally excluded from checkpoints, so persistence of the value itself is not expected.
The surprising part is that LangGraph still retries the failed dynamic Send task on resume even though part of that task's original input can no longer be reconstructed.
I would expect one of the following contracts:
- LangGraph preserves enough pending
Send task input to retry the task faithfully, even when that input contains an UntrackedValue; or
- LangGraph detects that the failed task depends on untracked input and does not silently retry it with a different input shape; or
- This behavior is documented as an intentional limitation of
UntrackedValue with failed dynamic tasks.
Could maintainers confirm which behavior is intended?
Why this matters
UntrackedValue is useful for runtime-only resources such as sessions, clients, locks, or other non-serializable objects. Send is commonly used for dynamic fan-out. Combining the two works normally during the initial run, but after a task failure the same task is retried with different input, which can lead to secondary errors or different behavior after recovery.
If maintainers consider this a bug, I would be happy to add a focused regression test and work on a minimal fix. Please assign the issue to @Hello-world-Prakash if that direction is approved.
Checked other resources
Send,UntrackedValue, retry/resume, and checkpoint recovery and did not find a matching report.InMemorySaveronly.main(d56666f7f).Reproduction Steps / Example Code
Actual behavior
First execution:
After the injected failure:
On resume:
The failed
workertask is still considered resumable (state.next == ('worker',)), but the input it receives after resume is structurally different from the originalSendinput because theUntrackedValueis gone.Expected behavior / contract question
I understand that
UntrackedValueis intentionally excluded from checkpoints, so persistence of the value itself is not expected.The surprising part is that LangGraph still retries the failed dynamic
Sendtask on resume even though part of that task's original input can no longer be reconstructed.I would expect one of the following contracts:
Sendtask input to retry the task faithfully, even when that input contains anUntrackedValue; orUntrackedValuewith failed dynamic tasks.Could maintainers confirm which behavior is intended?
Why this matters
UntrackedValueis useful for runtime-only resources such as sessions, clients, locks, or other non-serializable objects.Sendis commonly used for dynamic fan-out. Combining the two works normally during the initial run, but after a task failure the same task is retried with different input, which can lead to secondary errors or different behavior after recovery.If maintainers consider this a bug, I would be happy to add a focused regression test and work on a minimal fix. Please assign the issue to
@Hello-world-Prakashif that direction is approved.