Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Storing Chat History in 3rd Party Storage
description: How to store agent chat history in external storage using a custom ChatMessageStore.
description: How to store agent chat history in external storage
zone_pivot_groups: programming-languages
author: westey-m
ms.topic: tutorial
Expand All @@ -13,7 +13,7 @@ ms.service: agent-framework

::: zone pivot="programming-language-csharp"

This tutorial shows how to store agent chat history in external storage by implementing a custom `ChatMessageStore` and using it with a `ChatClientAgent`.
This tutorial shows how to store agent chat history in external storage by implementing a custom `ChatHistoryProvider` and using it with a `ChatClientAgent`.

By default, when using `ChatClientAgent`, chat history is stored either in memory in the `AgentThread` object or the underlying inference service, if the service supports it.

Expand All @@ -39,9 +39,9 @@ In addition, you'll use the in-memory vector store to store chat messages.
dotnet add package Microsoft.SemanticKernel.Connectors.InMemory --prerelease
```

## Create a custom ChatMessage Store
## Create a custom ChatHistoryProvider

To create a custom `ChatMessageStore`, you need to implement the abstract `ChatMessageStore` class and provide implementations for the required methods.
To create a custom `ChatHistoryProvider`, you need to implement the abstract `ChatHistoryProvider` class and provide implementations for the required methods.

### Message storage and retrieval methods

Expand All @@ -56,13 +56,13 @@ Any chat history reduction logic, such as summarization or trimming, should be d

### Serialization

`ChatMessageStore` instances are created and attached to an `AgentThread` when the thread is created, and when a thread is resumed from a serialized state.
`ChatHistoryProvider` instances are created and attached to an `AgentThread` when the thread is created, and when a thread is resumed from a serialized state.

While the actual messages making up the chat history are stored externally, the `ChatMessageStore` instance might need to store keys or other state to identify the chat history in the external store.
While the actual messages making up the chat history are stored externally, the `ChatHistoryProvider` instance might need to store keys or other state to identify the chat history in the external store.

To allow persisting threads, you need to implement the `Serialize` method of the `ChatMessageStore` class. This method should return a `JsonElement` containing the state needed to restore the store later. When deserializing, the agent framework will pass this serialized state to the ChatMessageStoreFactory, allowing you to use it to recreate the store.
To allow persisting threads, you need to implement the `Serialize` method of the `ChatHistoryProvider` class. This method should return a `JsonElement` containing the state needed to restore the provider later. When deserializing, the agent framework will pass this serialized state to the ChatHistoryProviderFactory, allowing you to use it to recreate the provider.

### Sample ChatMessageStore implementation
### Sample ChatHistoryProvider implementation

The following sample implementation stores chat messages in a vector store.

Expand All @@ -87,11 +87,11 @@ using Microsoft.Extensions.AI;
using Microsoft.Extensions.VectorData;
using Microsoft.SemanticKernel.Connectors.InMemory;

internal sealed class VectorChatMessageStore : ChatMessageStore
internal sealed class VectorChatHistoryProvider : ChatHistoryProvider
{
private readonly VectorStore _vectorStore;

public VectorChatMessageStore(
public VectorChatHistoryProvider(
VectorStore vectorStore,
JsonElement serializedStoreState,
JsonSerializerOptions? jsonSerializerOptions = null)
Expand Down Expand Up @@ -185,13 +185,13 @@ internal sealed class VectorChatMessageStore : ChatMessageStore
}
```

## Using the custom ChatMessageStore with a ChatClientAgent
## Using the custom ChatHistoryProvider with a ChatClientAgent

To use the custom `ChatMessageStore`, you need to provide a `ChatMessageStoreFactory` when creating the agent. This factory allows the agent to create a new instance of the desired `ChatMessageStore` for each thread.
To use the custom `ChatHistoryProvider`, you need to provide a `ChatHistoryProviderFactory` when creating the agent. This factory allows the agent to create a new instance of the desired `ChatHistoryProvider` for each thread.

When creating a `ChatClientAgent` it is possible to provide a `ChatClientAgentOptions` object that allows providing the `ChatMessageStoreFactory` in addition to all other agent options.
When creating a `ChatClientAgent` it is possible to provide a `ChatClientAgentOptions` object that allows providing the `ChatHistoryProviderFactory` in addition to all other agent options.

The factory is an async function that receives a context object and a cancellation token, and returns a `ValueTask<ChatMessageStore>`.
The factory is an async function that receives a context object and a cancellation token, and returns a `ValueTask<ChatHistoryProvider>`.

```csharp
using Azure.AI.OpenAI;
Expand All @@ -210,11 +210,11 @@ AIAgent agent = new AzureOpenAIClient(
{
Name = "Joker",
ChatOptions = new() { Instructions = "You are good at telling jokes." },
ChatMessageStoreFactory = (ctx, ct) => new ValueTask<ChatMessageStore>(
// Create a new chat message store for this agent that stores the messages in a vector store.
// Each thread must get its own copy of the VectorChatMessageStore, since the store
ChatHistoryProviderFactory = (ctx, ct) => new ValueTask<ChatHistoryProvider>(
// Create a new chat history provider for this agent that stores the messages in a vector store.
// Each thread must get its own copy of the VectorChatHistoryProvider, since the provider
// also contains the id that the thread is stored under.
new VectorChatMessageStore(
new VectorChatHistoryProvider(
vectorStore,
ctx.SerializedState,
ctx.JsonSerializerOptions))
Expand Down
42 changes: 21 additions & 21 deletions agent-framework/user-guide/agents/agent-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Various chat history storage options are supported by Agent Framework. The avail

The two main supported scenarios are:

- **In-memory storage**: Agent is built on a service that doesn't support in-service storage of chat history (for example, OpenAI Chat Completion). By default, Agent Framework stores the full chat history in-memory in the `AgentThread` object, but developers can provide a custom `ChatMessageStore` implementation to store chat history in a third-party store if required.
- **In-memory storage**: Agent is built on a service that doesn't support in-service storage of chat history (for example, OpenAI Chat Completion). By default, Agent Framework stores the full chat history in-memory in the `AgentThread` object, but developers can provide a custom `ChatHistoryProvider` implementation to store chat history in a third-party store if required.
- **In-service storage**: Agent is built on a service that requires in-service storage of chat history (for example, Azure AI Foundry Persistent Agents). Agent Framework stores the ID of the remote chat history in the `AgentThread` object, and no other chat history storage options are supported.

### In-memory chat history storage
Expand Down Expand Up @@ -49,17 +49,17 @@ IList<ChatMessage>? messages = thread.GetService<IList<ChatMessage>>();

#### Chat history reduction with in-memory storage

The built-in `InMemoryChatMessageStore` that's used by default when the underlying service does not support in-service storage,
The built-in `InMemoryChatHistoryProvider` that's used by default when the underlying service does not support in-service storage,
can be configured with a reducer to manage the size of the chat history.
This is useful to avoid exceeding the context size limits of the underlying service.

The `InMemoryChatMessageStore` can take an optional `Microsoft.Extensions.AI.IChatReducer` implementation to reduce the size of the chat history.
The `InMemoryChatHistoryProvider` can take an optional `Microsoft.Extensions.AI.IChatReducer` implementation to reduce the size of the chat history.
It also allows you to configure the event during which the reducer is invoked, either after a message is added to the chat history
or before the chat history is returned for the next invocation.

To configure the `InMemoryChatMessageStore` with a reducer, you can provide a factory to construct a new `InMemoryChatMessageStore`
for each new `AgentThread` and pass it a reducer of your choice. The `InMemoryChatMessageStore` can also be passed an optional trigger event
which can be set to either `InMemoryChatMessageStore.ChatReducerTriggerEvent.AfterMessageAdded` or `InMemoryChatMessageStore.ChatReducerTriggerEvent.BeforeMessagesRetrieval`.
To configure the `InMemoryChatHistoryProvider` with a reducer, you can provide a factory to construct a new `InMemoryChatHistoryProvider`
for each new `AgentThread` and pass it a reducer of your choice. The `InMemoryChatHistoryProvider` can also be passed an optional trigger event
which can be set to either `InMemoryChatHistoryProvider.ChatReducerTriggerEvent.AfterMessageAdded` or `InMemoryChatHistoryProvider.ChatReducerTriggerEvent.BeforeMessagesRetrieval`.

The factory is an async function that receives a context object and a cancellation token.

Expand All @@ -70,17 +70,17 @@ AIAgent agent = new OpenAIClient("<your_api_key>")
{
Name = JokerName,
ChatOptions = new() { Instructions = JokerInstructions },
ChatMessageStoreFactory = (ctx, ct) => new ValueTask<ChatMessageStore>(
new InMemoryChatMessageStore(
ChatHistoryProviderFactory = (ctx, ct) => new ValueTask<ChatHistoryProvider>(
new InMemoryChatHistoryProvider(
new MessageCountingChatReducer(2),
ctx.SerializedState,
ctx.JsonSerializerOptions,
InMemoryChatMessageStore.ChatReducerTriggerEvent.AfterMessageAdded))
InMemoryChatHistoryProvider.ChatReducerTriggerEvent.AfterMessageAdded))
});
```

> [!NOTE]
> This feature is only supported when using the `InMemoryChatMessageStore`. When a service has in-service chat history storage, it is up to the service itself to manage the size of the chat history. Similarly, when using 3rd party storage (see below), it is up to the 3rd party storage solution to manage the chat history size. If you provide a `ChatMessageStoreFactory` for a message store but you use a service with built-in chat history storage, the factory will not be used.
> This feature is only supported when using the `InMemoryChatHistoryProvider`. When a service has in-service chat history storage, it is up to the service itself to manage the size of the chat history. Similarly, when using 3rd party storage (see below), it is up to the 3rd party storage solution to manage the chat history size. If you provide a `ChatHistoryProviderFactory` for a chat history provider but you use a service with built-in chat history storage, the factory will not be used.

### Inference service chat history storage

Expand All @@ -102,16 +102,16 @@ Console.WriteLine(await agent.RunAsync("Tell me a joke about a pirate.", thread)

### Third-party chat history storage

When using a service that does not support in-service storage of chat history, Agent Framework allows developers to replace the default in-memory storage of chat history with third-party chat history storage. The developer is required to provide a subclass of the base abstract `ChatMessageStore` class.
When using a service that does not support in-service storage of chat history, Agent Framework allows developers to replace the default in-memory storage of chat history with third-party chat history storage. The developer is required to provide a subclass of the base abstract `ChatHistoryProvider` class.

The `ChatMessageStore` class defines the interface for storing and retrieving chat messages. Developers must implement the `InvokedAsync` and `InvokingAsync` methods to add messages to the remote store as they are generated, and retrieve messages from the remote store before invoking the underlying service.
The `ChatHistoryProvider` class defines the interface for storing and retrieving chat messages. Developers must implement the `InvokedAsync` and `InvokingAsync` methods to add messages to the remote store as they are generated, and retrieve messages from the remote store before invoking the underlying service.

The agent will use all messages returned by `InvokingAsync` when processing a user query. It is up to the implementer of `ChatMessageStore` to ensure that the size of the chat history does not exceed the context window of the underlying service.
The agent will use all messages returned by `InvokingAsync` when processing a user query. It is up to the implementer of `ChatHistoryProvider` to ensure that the size of the chat history does not exceed the context window of the underlying service.

When implementing a custom `ChatMessageStore` which stores chat history in a remote store, the chat history for that thread should be stored under a key that is unique to that thread. The `ChatMessageStore` implementation should generate this key and keep it in its state. `ChatMessageStore` has a `Serialize` method that can be overridden to serialize its state when the thread is serialized. The `ChatMessageStore` should also provide a constructor that takes a <xref:System.Text.Json.JsonElement> as input to support deserialization of its state.
When implementing a custom `ChatHistoryProvider` which stores chat history in a remote store, the chat history for that thread should be stored under a key that is unique to that thread. The `ChatHistoryProvider` implementation should generate this key and keep it in its state. `ChatHistoryProvider` has a `Serialize` method that can be overridden to serialize its state when the thread is serialized. The `ChatHistoryProvider` should also provide a constructor that takes a <xref:System.Text.Json.JsonElement> as input to support deserialization of its state.

To supply a custom `ChatMessageStore` to a `ChatClientAgent`, you can use the `ChatMessageStoreFactory` option when creating the agent.
Here is an example showing how to pass the custom implementation of `ChatMessageStore` to a `ChatClientAgent` that is based on Azure OpenAI Chat Completion.
To supply a custom `ChatHistoryProvider` to a `ChatClientAgent`, you can use the `ChatHistoryProviderFactory` option when creating the agent.
Here is an example showing how to pass the custom implementation of `ChatHistoryProvider` to a `ChatClientAgent` that is based on Azure OpenAI Chat Completion.

The factory is an async function that receives a context object and a cancellation token.

Expand All @@ -124,11 +124,11 @@ AIAgent agent = new AzureOpenAIClient(
{
Name = JokerName,
ChatOptions = new() { Instructions = JokerInstructions },
ChatMessageStoreFactory = (ctx, ct) => new ValueTask<ChatMessageStore>(
// Create a new chat message store for this agent that stores the messages in a custom store.
// Each thread must get its own copy of the CustomMessageStore, since the store
ChatHistoryProviderFactory = (ctx, ct) => new ValueTask<ChatHistoryProvider>(
// Create a new chat history provider for this agent that stores the messages in a custom store.
// Each thread must get its own copy of the CustomChatHistoryProvider, since the provider
// also contains the ID that the thread is stored under.
new CustomMessageStore(
new CustomChatHistoryProvider(
vectorStore,
ctx.SerializedState,
ctx.JsonSerializerOptions))
Expand All @@ -153,7 +153,7 @@ It is important to be able to persist an `AgentThread` object between agent invo

Even if the chat history is stored in a remote store, the `AgentThread` object still contains an ID referencing the remote chat history. Losing the `AgentThread` state will therefore result in also losing the ID of the remote chat history.

The `AgentThread` as well as any objects attached to it, all therefore provide the `Serialize` method to serialize their state. The `AIAgent` also provides a `DeserializeThreadAsync` method that re-creates a thread from the serialized state. The `DeserializeThreadAsync` method re-creates the thread with the `ChatMessageStore` and `AIContextProvider` configured on the agent.
The `AgentThread` as well as any objects attached to it, all therefore provide the `Serialize` method to serialize their state. The `AIAgent` also provides a `DeserializeThreadAsync` method that re-creates a thread from the serialized state. The `DeserializeThreadAsync` method re-creates the thread with the `ChatHistoryProvider` and `AIContextProvider` configured on the agent.

```csharp
// Serialize the thread state to a JsonElement, so it can be stored for later use.
Expand Down
2 changes: 2 additions & 0 deletions agent-framework/user-guide/agents/agent-types/TOC.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
href: durable-agent/create-durable-agent.md
- name: Durable Agent Features
href: durable-agent/features.md
- name: GitHub Copilot Agents
href: github-copilot-agent.md
- name: A2A Agents
href: a2a-agent.md
- name: Custom Agents
Expand Down
Loading