You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: agent-framework/user-guide/agents/agent-memory.md
+21-22Lines changed: 21 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,18 +17,18 @@ Agent chat history and memory are crucial capabilities that allow agents to main
17
17
18
18
## Chat History
19
19
20
-
Various chat history storage options are supported by the Agent Framework. The available options vary by agent type and the underlying service(s) used to build the agent.
20
+
Various chat history storage options are supported by Agent Framework. The available options vary by agent type and the underlying service(s) used to build the agent.
21
21
22
-
Here are the two main scenarios supported:
22
+
The two main supported scenarios are:
23
23
24
-
1.**In-memory storage**: Agent is built on a service that does not support in-service storage of chat history (e.g. OpenAI Chat Completion). The Agent Framework will by default store the full chat history in-memory in the `AgentThread` object, but developers can provide a custom `ChatMessageStore` implementation to store chat history in a 3rd party store if required.
25
-
1.**In-service storage**: Agent is built on a service that requires in-service storage of chat history (e.g. Azure AI Foundry Persistent Agents). The Agent Framework will store the id of the remote chat history in the `AgentThread` object and no other chat history storage options are supported.
24
+
-**In-memory storage**: Agent is built on a service that doesn't support in-service storage of chat history (for example, OpenAI Chat Completion). By default, Agent Framework stores the full chat history in-memory in the `AgentThread` object, but developers can provide a custom `ChatMessageStore` implementation to store chat history in a third-party store if required.
25
+
-**In-service storage**: Agent is built on a service that requires in-service storage of chat history (for example, Azure AI Foundry Persistent Agents). Agent Framework stores the ID of the remote chat history in the `AgentThread` object, and no other chat history storage options are supported.
26
26
27
27
### In-memory chat history storage
28
28
29
-
When using a service that does not support in-service storage of chat history, the Agent Framework will default to storing chat history in-memory in the `AgentThread` object. In this case, the full chat history that is stored in the thread object, plus any new messages, will be provided to the underlying service on each agent run. This allows for a natural conversational experience with the agent, where the caller only provides the new user message, and the agent only returns new answers, but the agent has access to the full conversation history and will use it when generating its response.
29
+
When using a service that doesn't support in-service storage of chat history, Agent Framework defaults to storing chat history in-memory in the `AgentThread` object. In this case, the full chat history that's stored in the thread object, plus any new messages, will be provided to the underlying service on each agent run. This design allows for a natural conversational experience with the agent. The caller only provides the new user message, and the agent only returns new answers. But the agent has access to the full conversation history and will use it when generating its response.
30
30
31
-
When using OpenAI Chat Completion as the underlying service for agents, the following code will result in the thread object containing the chat history from the agent run.
31
+
When using OpenAI Chat Completion as the underlying service for agents, the following code results in the thread object containing the chat history from the agent run.
> Retrieving messages from the `AgentThread` object in this way will only work if in-memory storage is being used.
48
+
> Retrieving messages from the `AgentThread` object in this way only works if in-memory storage is being used.
49
49
50
-
#### Chat History reduction with In-Memory storage
50
+
#### Chat history reduction with in-memory storage
51
51
52
-
The built-in `InMemoryChatMessageStore` that is used by default when the underlying service does not support in-service storage,
52
+
The built-in `InMemoryChatMessageStore` that's used by default when the underlying service does not support in-service storage,
53
53
can be configured with a reducer to manage the size of the chat history.
54
54
This is useful to avoid exceeding the context size limits of the underlying service.
55
55
@@ -81,9 +81,9 @@ AIAgent agent = new OpenAIClient("<your_api_key>")
81
81
82
82
### Inference service chat history storage
83
83
84
-
When using a service that requires in-service storage of chat history, the Agent Framework will store the id of the remote chat history in the `AgentThread` object.
84
+
When using a service that requires in-service storage of chat history, Agent Framework stores the ID of the remote chat history in the `AgentThread` object.
85
85
86
-
E.g, when using OpenAI Responses with store=true as the underlying service for agents, the following code will result in the thread object containing the last response id returned by the service.
86
+
For example, when using OpenAI Responses with store=true as the underlying service for agents, the following code will result in the thread object containing the last response ID returned by the service.
87
87
88
88
```csharp
89
89
AIAgentagent=newOpenAIClient("<your_api_key>")
@@ -94,19 +94,18 @@ Console.WriteLine(await agent.RunAsync("Tell me a joke about a pirate.", thread)
94
94
```
95
95
96
96
> [!NOTE]
97
-
> Some services, e.g. OpenAI Responses support either in-service storage of chat history (store=true), or providing the full chat history on each invocation (store=false).
98
-
> Therefore, depending on the mode that the service is used in, the Agent Framework will either default to storing the full chat history in memory, or storing an id reference
99
-
> to the service stored chat history.
97
+
> Some services, for example, OpenAI Responses support either in-service storage of chat history (store=true), or providing the full chat history on each invocation (store=false).
98
+
> Therefore, depending on the mode that the service is used in, Agent Framework will either default to storing the full chat history in memory, or storing an ID reference to the service stored chat history.
100
99
101
-
### 3rd party chat history storage
100
+
### Third-party chat history storage
102
101
103
-
When using a service that does not support in-service storage of chat history, the Agent Framework allows developers to replace the default in-memory storage of chat history with 3rd party chat history storage. The developer is required to provide a subclass of the base abstract `ChatMessageStore` class.
102
+
When using a service that does not support in-service storage of chat history, Agent Framework allows developers to replace the default in-memory storage of chat history with third-party chat history storage. The developer is required to provide a subclass of the base abstract `ChatMessageStore` class.
104
103
105
104
The `ChatMessageStore` class defines the interface for storing and retrieving chat messages. Developers must implement the `AddMessagesAsync` and `GetMessagesAsync` methods to add messages to the remote store as they are generated, and retrieve messages from the remote store before invoking the underlying service.
106
105
107
106
The agent will use all messages returned by `GetMessagesAsync` when processing a user query. It is up to the implementer of `ChatMessageStore` to ensure that the size of the chat history does not exceed the context window of the underlying service.
108
107
109
-
When implementing a custom `ChatMessageStore` which stores chat history in a remote store, the chat history for that thread should be stored under a key that is unique to that thread. The `ChatMessageStore` implementation should generate this key and keep it in its state. `ChatMessageStore` has a `Serialize` method that can be overridden to serialize its state when the thread is serialized. The `ChatMessageStore` should also provide a constructor that takes a `JsonElement` as input to support deserialization of its state.
108
+
When implementing a custom `ChatMessageStore` which stores chat history in a remote store, the chat history for that thread should be stored under a key that is unique to that thread. The `ChatMessageStore` implementation should generate this key and keep it in its state. `ChatMessageStore` has a `Serialize` method that can be overridden to serialize its state when the thread is serialized. The `ChatMessageStore` should also provide a constructor that takes a <xref:System.Text.Json.JsonElement> as input to support deserialization of its state.
110
109
111
110
To supply a custom `ChatMessageStore` to a `ChatClientAgent`, you can use the `ChatMessageStoreFactory` option when creating the agent.
112
111
Here is an example showing how to pass the custom implementation of `ChatMessageStore` to a `ChatClientAgent` that is based on Azure OpenAI Chat Completion.
@@ -124,7 +123,7 @@ AIAgent agent = new AzureOpenAIClient(
124
123
{
125
124
// Create a new chat message store for this agent that stores the messages in a custom store.
126
125
// Each thread must get its own copy of the CustomMessageStore, since the store
127
-
// also contains the id that the thread is stored under.
126
+
// also contains the ID that the thread is stored under.
@@ -144,9 +143,9 @@ To implement such a memory component, the developer needs to subclass the `AICon
144
143
145
144
## AgentThread Serialization
146
145
147
-
It is important to be able to persist an `AgentThread` object between agent invocations. This allows for situations where a user may ask a question of the agent, and take a long time to ask follow up questions. This allows the `AgentThread` state to survive service or app restarts.
146
+
It is important to be able to persist an `AgentThread` object between agent invocations. This allows for situations where a user might ask a question of the agent, and take a long time to ask follow up questions. This allows the `AgentThread` state to survive service or app restarts.
148
147
149
-
Even if the chat history is stored in a remote store, the `AgentThread` object still contains an id referencing the remote chat history. Losing the `AgentThread` state will therefore result in also losing the id of the remote chat history.
148
+
Even if the chat history is stored in a remote store, the `AgentThread` object still contains an ID referencing the remote chat history. Losing the `AgentThread` state will therefore result in also losing the ID of the remote chat history.
150
149
151
150
The `AgentThread` as well as any objects attached to it, all therefore provide the `SerializeAsync` method to serialize their state. The `AIAgent` also provides a `DeserializeThread` method that re-creates a thread from the serialized state. The `DeserializeThread` method re-creates the thread with the `ChatMessageStore` and `AIContextProvider` configured on the agent.
> Always treat `AgentThread` objects as opaque objects, unless you are very sure of the internals. The contents may vary not just by agent type, but also by service type and configuration.
165
164
> [!WARNING]
166
-
> Deserializing a thread with a different agent than that which originally created it, or with an agent that has a different configuration than the original agent, may result in errors or unexpected behavior.
165
+
> Deserializing a thread with a different agent than that which originally created it, or with an agent that has a different configuration than the original agent, might result in errors or unexpected behavior.
Copy file name to clipboardExpand all lines: agent-framework/user-guide/agents/agent-middleware.md
+11-11Lines changed: 11 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,15 +11,15 @@ ms.service: agent-framework
11
11
12
12
# Agent Middleware
13
13
14
-
Middleware in the Agent Framework provides a powerful way to intercept, modify, and enhance agent interactions at various stages of execution. You can use middleware to implement cross-cutting concerns such as logging, security validation, error handling, and result transformation without modifying your core agent or function logic.
14
+
Middleware in Agent Framework provides a powerful way to intercept, modify, and enhance agent interactions at various stages of execution. You can use middleware to implement cross-cutting concerns such as logging, security validation, error handling, and result transformation without modifying your core agent or function logic.
15
15
16
16
::: zone pivot="programming-language-csharp"
17
17
18
-
The Agent Framework can be customized using three different types of middleware:
18
+
Agent Framework can be customized using three different types of middleware:
19
19
20
20
1. Agent Run middleware: Allows interception of all agent runs, so that input and output can be inspected and/or modified as needed.
21
21
1. Function calling middleware: Allows interception of all function calls executed by the agent, so that input and output can be inspected and modified as needed.
22
-
1.`IChatClient` middleware: Allows interception of calls to an `IChatClient` implementation, where an agent is using `IChatClient` for inference calls, e.g. when using `ChatClientAgent`.
22
+
1.<xref:Microsoft.Extensions.AI.IChatClient> middleware: Allows interception of calls to an `IChatClient` implementation, where an agent is using `IChatClient` for inference calls, for example, when using `ChatClientAgent`.
23
23
24
24
All the types of middleware are implemented via a function callback, and when multiple middleware instances of the same type are registered, they form a chain,
25
25
where each middleware instance is expected to call the next in the chain, via a provided `next``Func`.
@@ -35,10 +35,10 @@ var middlewareEnabledAgent = originalAgent
35
35
```
36
36
37
37
> [!IMPORTANT]
38
-
> Ideally both `runFunc` and `runStreamingFunc` should be provided, when providing just the non-streaming middleware, the agent will use it for both streaming and non-streaming invocations and this will block the streaming to run in non-streaming mode to suffice the middleware expectations.
38
+
> Ideally both `runFunc` and `runStreamingFunc` should be provided. When providing just the non-streaming middleware, the agent will use it for both streaming and non-streaming invocations. Streaming will only run in non-streaming mode to suffice the middleware expectations.
39
39
40
40
> [!NOTE]
41
-
> There's an additional overload `Use(sharedFunc: ...)` that allows you to provide the same middleware for non-streaming and streaming without blocking the streaming, however, the shared middleware won't be able intercept or override the output, make this the best option only for scenarios where you only need to inspect/modify the input before it reaches the agent.
41
+
> There's an additional overload,`Use(sharedFunc: ...)`, that allows you to provide the same middleware for non-streaming and streaming without blocking the streaming. However, the shared middleware won't be able to intercept or override the output. This overload should be used for scenarios where you only need to inspect or modify the input before it reaches the agent.
42
42
43
43
`IChatClient` middleware can be registered on an `IChatClient` before it is used with a `ChatClientAgent`, by using the chat client builder pattern.
> Function calling middleware is currently only supported with an `AIAgent` that uses `Microsoft.Extensions.AI.FunctionInvokingChatClient`, e.g.`ChatClientAgent`.
116
+
> Function calling middleware is currently only supported with an `AIAgent` that uses <xref:Microsoft.Extensions.AI.FunctionInvokingChatClient>, for example,`ChatClientAgent`.
117
117
118
118
Here is an example of function calling middleware, that can inspect and/or modify the function being called, and the result from the function call.
It is possible to terminate the function call loop with function calling middleware by setting the provided `FunctionInvocationContext.Terminate` to true.
136
136
This will prevent the function calling loop from issuing a request to the inference service containing the function call results after function invocation.
137
-
If there were more than one function available for invocation during this iteration, it may also prevent any remaining functions from being executed.
137
+
If there were more than one function available for invocation during this iteration, it might also prevent any remaining functions from being executed.
138
138
139
139
> [!WARNING]
140
-
> Terminating the function call loop may result in your thread being left in an inconsistent state, e.g. containing function call content with no function result content.
141
-
> This may result in the thread being unusable for further runs.
140
+
> Terminating the function call loop might result in your thread being left in an inconsistent state, for example, containing function call content with no function result content.
141
+
> This might result in the thread being unusable for further runs.
> For more information about `IChatClient` middleware, see [Custom IChatClient middleware](/dotnet/ai/microsoft-extensions-ai#custom-ichatclient-middleware)
164
-
> in the Microsoft.Extensions.AI documentation.
163
+
> For more information about `IChatClient` middleware, see [Custom IChatClient middleware](/dotnet/ai/microsoft-extensions-ai#custom-ichatclient-middleware).
0 commit comments