Skip to content

[FLINK-AGENTS][api] Fix Kryo deserialization failure for Event classes with immutable lists#677

Merged
wenjin272 merged 1 commit into
apache:mainfrom
avichaym:fix/kryo-immutable-list-serialization
May 16, 2026
Merged

[FLINK-AGENTS][api] Fix Kryo deserialization failure for Event classes with immutable lists#677
wenjin272 merged 1 commit into
apache:mainfrom
avichaym:fix/kryo-immutable-list-serialization

Conversation

@avichaym
Copy link
Copy Markdown
Contributor

fixes #673

Wraps list arguments in new ArrayList<>() before passing to setAttr() in ChatRequestEvent, ContextRetrievalResponseEvent, and ToolRequestEvent. Without this, passing List.of(...) crashes Kryo on JDK 17+ with InaccessibleObjectException.

@github-actions github-actions Bot added doc-label-missing The Bot applies this label either because none or multiple labels were provided. fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue. labels May 14, 2026
Copy link
Copy Markdown
Collaborator

@wenjin272 wenjin272 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @avichaym, Ty for this fix.

The current fix looks good to me, but I think two additional points need to be added:

  1. If user use Map.of to consturct ToolResponseEvent, there will likely be issues as well.
  2. Supplement with relevant tests. Since in the CI pipeline, the test cases in flink-agents-end-to-end-tests-integration module will be executed in jdk 17 with flink 1.20, 2.0, 2.1 and 2.2. I think we can add a EventKryoSerializationTest in this module like:
@Test
void chatRequestEventWithImmutableListSurvivesKryoRoundTrip() throws Exception {
    ChatRequestEvent event =
            new ChatRequestEvent(
                    "myModel", List.of(new ChatMessage(MessageRole.USER, "hello")));

    ChatRequestEvent roundTripped = roundTrip(event, ChatRequestEvent.class);

    assertEquals("myModel", roundTripped.getModel());
    assertNotNull(roundTripped.getMessages());
    assertEquals(1, roundTripped.getMessages().size());
    assertEquals("hello", roundTripped.getMessages().get(0).getContent());
}
private static <T extends Event> T roundTrip(T event, Class<T> cls) throws Exception {
    KryoSerializer<T> serializer = new KryoSerializer<>(cls, new SerializerConfigImpl());
    DataOutputSerializer out = new DataOutputSerializer(64);
    serializer.serialize(event, out);
    DataInputDeserializer in = new DataInputDeserializer(out.getCopyOfBuffer());
    return serializer.deserialize(in);
}

@wenjin272 wenjin272 added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing The Bot applies this label either because none or multiple labels were provided. labels May 15, 2026
@avichaym avichaym force-pushed the fix/kryo-immutable-list-serialization branch from 08bb545 to 751782b Compare May 15, 2026 09:01
@avichaym
Copy link
Copy Markdown
Contributor Author

Thanks for the careful review @wenjin272 - both points addressed in the new commit :

  1. ToolResponseEvent now defensive-copies all four maps (responses, success, error, externalIds) so Map.of(...) callers are safe - same shape as the list fix.

  2. Added EventKryoSerializationTest under flink-agents-end-to-end-tests-integration covering all four affected events.

Map.of(
"id", "call_1",
"name", "myTool",
"arguments", Map.of("x", 1))));
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, we only copy the outer ImmutableList/ImmutableMap without handling nested ImmutableList/ImmutableMap cases. Therefore, this test will fail under Flink 1.20.

I think there are probably two solutions to this problem:

  1. Supports copying arbitrarily nested ImmutableList/ImmutableMap structures into nested MutableList/MutableMap, but this might be too heavy for constructing Events.
  2. Since both ToolRequestEvent and ToolResponseEvent are constructed by built-in actions, users typically do not create these events themselves. Therefore, we forgo defensive checks for Tool Events.

I personally prefer option 2, WDYT?

Copy link
Copy Markdown
Contributor Author

@avichaym avichaym May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, option 2 makes more sense. Updated to revert the Tool event changes and keeping defensive copies only on ChatRequestEvent and ContextRetrievalResponseEvent. Test trimmed to those two cases.

@avichaym avichaym force-pushed the fix/kryo-immutable-list-serialization branch from 751782b to 0535311 Compare May 15, 2026 10:33
…ted events + add contract test

Wrap the List arg in new ArrayList<>() in the constructors of the events
that user code instantiates directly: ChatRequestEvent (messages) and
ContextRetrievalResponseEvent (documents). This avoids the JDK 17+ Kryo
InaccessibleObjectException when callers pass List.of(...) or any other
ImmutableCollections instance.

ToolRequestEvent / ToolResponseEvent are intentionally not changed -
they are produced only by built-in framework actions, never by user code,
so adding defensive copies there only adds overhead. (And shallow copies
wouldn't help anyway: tool-call payloads can carry nested Map.of(...).)

EventKryoSerializationTest in the e2e integration module asserts the
defensive-copy contract directly: the stored collection must be mutable.
Two tests, one per affected event.
@avichaym avichaym force-pushed the fix/kryo-immutable-list-serialization branch from 0535311 to 60a46a0 Compare May 15, 2026 10:45
Copy link
Copy Markdown
Collaborator

@wenjin272 wenjin272 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wenjin272 wenjin272 merged commit 2a4c858 into apache:main May 16, 2026
44 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Kryo serialization fails for Event classes with immutable lists on JDK 17+

2 participants