Skip to content

Commit 3b2ed6d

Browse files
Merge pull request #806 from gewarren/ug-pass-agents
Edit pass on user-guide/workflows docs
2 parents 3985027 + 78176aa commit 3b2ed6d

15 files changed

Lines changed: 188 additions & 192 deletions

agent-framework/user-guide/agents/agent-memory.md

Lines changed: 21 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -17,18 +17,18 @@ Agent chat history and memory are crucial capabilities that allow agents to main
1717

1818
## Chat History
1919

20-
Various chat history storage options are supported by the Agent Framework. The available options vary by agent type and the underlying service(s) used to build the agent.
20+
Various chat history storage options are supported by Agent Framework. The available options vary by agent type and the underlying service(s) used to build the agent.
2121

22-
Here are the two main scenarios supported:
22+
The two main supported scenarios are:
2323

24-
1. **In-memory storage**: Agent is built on a service that does not support in-service storage of chat history (e.g. OpenAI Chat Completion). The Agent Framework will by default store the full chat history in-memory in the `AgentThread` object, but developers can provide a custom `ChatMessageStore` implementation to store chat history in a 3rd party store if required.
25-
1. **In-service storage**: Agent is built on a service that requires in-service storage of chat history (e.g. Azure AI Foundry Persistent Agents). The Agent Framework will store the id of the remote chat history in the `AgentThread` object and no other chat history storage options are supported.
24+
- **In-memory storage**: Agent is built on a service that doesn't support in-service storage of chat history (for example, OpenAI Chat Completion). By default, Agent Framework stores the full chat history in-memory in the `AgentThread` object, but developers can provide a custom `ChatMessageStore` implementation to store chat history in a third-party store if required.
25+
- **In-service storage**: Agent is built on a service that requires in-service storage of chat history (for example, Azure AI Foundry Persistent Agents). Agent Framework stores the ID of the remote chat history in the `AgentThread` object, and no other chat history storage options are supported.
2626

2727
### In-memory chat history storage
2828

29-
When using a service that does not support in-service storage of chat history, the Agent Framework will default to storing chat history in-memory in the `AgentThread` object. In this case, the full chat history that is stored in the thread object, plus any new messages, will be provided to the underlying service on each agent run. This allows for a natural conversational experience with the agent, where the caller only provides the new user message, and the agent only returns new answers, but the agent has access to the full conversation history and will use it when generating its response.
29+
When using a service that doesn't support in-service storage of chat history, Agent Framework defaults to storing chat history in-memory in the `AgentThread` object. In this case, the full chat history that's stored in the thread object, plus any new messages, will be provided to the underlying service on each agent run. This design allows for a natural conversational experience with the agent. The caller only provides the new user message, and the agent only returns new answers. But the agent has access to the full conversation history and will use it when generating its response.
3030

31-
When using OpenAI Chat Completion as the underlying service for agents, the following code will result in the thread object containing the chat history from the agent run.
31+
When using OpenAI Chat Completion as the underlying service for agents, the following code results in the thread object containing the chat history from the agent run.
3232

3333
```csharp
3434
AIAgent agent = new OpenAIClient("<your_api_key>")
@@ -38,18 +38,18 @@ AgentThread thread = agent.GetNewThread();
3838
Console.WriteLine(await agent.RunAsync("Tell me a joke about a pirate.", thread));
3939
```
4040

41-
Where messages are stored in memory, it is possible to retrieve the list of messages from the thread and manipulate the mesages directly if required.
41+
Where messages are stored in memory, it's possible to retrieve the list of messages from the thread and manipulate the messages directly if required.
4242

4343
```csharp
4444
IList<ChatMessage>? messages = thread.GetService<IList<ChatMessage>>();
4545
```
4646

4747
> [!NOTE]
48-
> Retrieving messages from the `AgentThread` object in this way will only work if in-memory storage is being used.
48+
> Retrieving messages from the `AgentThread` object in this way only works if in-memory storage is being used.
4949
50-
#### Chat History reduction with In-Memory storage
50+
#### Chat history reduction with in-memory storage
5151

52-
The built-in `InMemoryChatMessageStore` that is used by default when the underlying service does not support in-service storage,
52+
The built-in `InMemoryChatMessageStore` that's used by default when the underlying service does not support in-service storage,
5353
can be configured with a reducer to manage the size of the chat history.
5454
This is useful to avoid exceeding the context size limits of the underlying service.
5555

@@ -81,9 +81,9 @@ AIAgent agent = new OpenAIClient("<your_api_key>")
8181
8282
### Inference service chat history storage
8383

84-
When using a service that requires in-service storage of chat history, the Agent Framework will store the id of the remote chat history in the `AgentThread` object.
84+
When using a service that requires in-service storage of chat history, Agent Framework stores the ID of the remote chat history in the `AgentThread` object.
8585

86-
E.g, when using OpenAI Responses with store=true as the underlying service for agents, the following code will result in the thread object containing the last response id returned by the service.
86+
For example, when using OpenAI Responses with store=true as the underlying service for agents, the following code will result in the thread object containing the last response ID returned by the service.
8787

8888
```csharp
8989
AIAgent agent = new OpenAIClient("<your_api_key>")
@@ -94,19 +94,18 @@ Console.WriteLine(await agent.RunAsync("Tell me a joke about a pirate.", thread)
9494
```
9595

9696
> [!NOTE]
97-
> Some services, e.g. OpenAI Responses support either in-service storage of chat history (store=true), or providing the full chat history on each invocation (store=false).
98-
> Therefore, depending on the mode that the service is used in, the Agent Framework will either default to storing the full chat history in memory, or storing an id reference
99-
> to the service stored chat history.
97+
> Some services, for example, OpenAI Responses support either in-service storage of chat history (store=true), or providing the full chat history on each invocation (store=false).
98+
> Therefore, depending on the mode that the service is used in, Agent Framework will either default to storing the full chat history in memory, or storing an ID reference to the service stored chat history.
10099
101-
### 3rd party chat history storage
100+
### Third-party chat history storage
102101

103-
When using a service that does not support in-service storage of chat history, the Agent Framework allows developers to replace the default in-memory storage of chat history with 3rd party chat history storage. The developer is required to provide a subclass of the base abstract `ChatMessageStore` class.
102+
When using a service that does not support in-service storage of chat history, Agent Framework allows developers to replace the default in-memory storage of chat history with third-party chat history storage. The developer is required to provide a subclass of the base abstract `ChatMessageStore` class.
104103

105104
The `ChatMessageStore` class defines the interface for storing and retrieving chat messages. Developers must implement the `AddMessagesAsync` and `GetMessagesAsync` methods to add messages to the remote store as they are generated, and retrieve messages from the remote store before invoking the underlying service.
106105

107106
The agent will use all messages returned by `GetMessagesAsync` when processing a user query. It is up to the implementer of `ChatMessageStore` to ensure that the size of the chat history does not exceed the context window of the underlying service.
108107

109-
When implementing a custom `ChatMessageStore` which stores chat history in a remote store, the chat history for that thread should be stored under a key that is unique to that thread. The `ChatMessageStore` implementation should generate this key and keep it in its state. `ChatMessageStore` has a `Serialize` method that can be overridden to serialize its state when the thread is serialized. The `ChatMessageStore` should also provide a constructor that takes a `JsonElement` as input to support deserialization of its state.
108+
When implementing a custom `ChatMessageStore` which stores chat history in a remote store, the chat history for that thread should be stored under a key that is unique to that thread. The `ChatMessageStore` implementation should generate this key and keep it in its state. `ChatMessageStore` has a `Serialize` method that can be overridden to serialize its state when the thread is serialized. The `ChatMessageStore` should also provide a constructor that takes a <xref:System.Text.Json.JsonElement> as input to support deserialization of its state.
110109

111110
To supply a custom `ChatMessageStore` to a `ChatClientAgent`, you can use the `ChatMessageStoreFactory` option when creating the agent.
112111
Here is an example showing how to pass the custom implementation of `ChatMessageStore` to a `ChatClientAgent` that is based on Azure OpenAI Chat Completion.
@@ -124,7 +123,7 @@ AIAgent agent = new AzureOpenAIClient(
124123
{
125124
// Create a new chat message store for this agent that stores the messages in a custom store.
126125
// Each thread must get its own copy of the CustomMessageStore, since the store
127-
// also contains the id that the thread is stored under.
126+
// also contains the ID that the thread is stored under.
128127
return new CustomMessageStore(vectorStore, ctx.SerializedState, ctx.JsonSerializerOptions);
129128
}
130129
});
@@ -144,9 +143,9 @@ To implement such a memory component, the developer needs to subclass the `AICon
144143
145144
## AgentThread Serialization
146145

147-
It is important to be able to persist an `AgentThread` object between agent invocations. This allows for situations where a user may ask a question of the agent, and take a long time to ask follow up questions. This allows the `AgentThread` state to survive service or app restarts.
146+
It is important to be able to persist an `AgentThread` object between agent invocations. This allows for situations where a user might ask a question of the agent, and take a long time to ask follow up questions. This allows the `AgentThread` state to survive service or app restarts.
148147

149-
Even if the chat history is stored in a remote store, the `AgentThread` object still contains an id referencing the remote chat history. Losing the `AgentThread` state will therefore result in also losing the id of the remote chat history.
148+
Even if the chat history is stored in a remote store, the `AgentThread` object still contains an ID referencing the remote chat history. Losing the `AgentThread` state will therefore result in also losing the ID of the remote chat history.
150149

151150
The `AgentThread` as well as any objects attached to it, all therefore provide the `SerializeAsync` method to serialize their state. The `AIAgent` also provides a `DeserializeThread` method that re-creates a thread from the serialized state. The `DeserializeThread` method re-creates the thread with the `ChatMessageStore` and `AIContextProvider` configured on the agent.
152151

@@ -163,7 +162,7 @@ AgentThread resumedThread = AIAgent.DeserializeThread(serializedThreadState);
163162
> [!IMPORTANT]
164163
> Always treat `AgentThread` objects as opaque objects, unless you are very sure of the internals. The contents may vary not just by agent type, but also by service type and configuration.
165164
> [!WARNING]
166-
> Deserializing a thread with a different agent than that which originally created it, or with an agent that has a different configuration than the original agent, may result in errors or unexpected behavior.
165+
> Deserializing a thread with a different agent than that which originally created it, or with an agent that has a different configuration than the original agent, might result in errors or unexpected behavior.
167166
168167
::: zone-end
169168
::: zone pivot="programming-language-python"

agent-framework/user-guide/agents/agent-middleware.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -11,15 +11,15 @@ ms.service: agent-framework
1111

1212
# Agent Middleware
1313

14-
Middleware in the Agent Framework provides a powerful way to intercept, modify, and enhance agent interactions at various stages of execution. You can use middleware to implement cross-cutting concerns such as logging, security validation, error handling, and result transformation without modifying your core agent or function logic.
14+
Middleware in Agent Framework provides a powerful way to intercept, modify, and enhance agent interactions at various stages of execution. You can use middleware to implement cross-cutting concerns such as logging, security validation, error handling, and result transformation without modifying your core agent or function logic.
1515

1616
::: zone pivot="programming-language-csharp"
1717

18-
The Agent Framework can be customized using three different types of middleware:
18+
Agent Framework can be customized using three different types of middleware:
1919

2020
1. Agent Run middleware: Allows interception of all agent runs, so that input and output can be inspected and/or modified as needed.
2121
1. Function calling middleware: Allows interception of all function calls executed by the agent, so that input and output can be inspected and modified as needed.
22-
1. `IChatClient` middleware: Allows interception of calls to an `IChatClient` implementation, where an agent is using `IChatClient` for inference calls, e.g. when using `ChatClientAgent`.
22+
1. <xref:Microsoft.Extensions.AI.IChatClient> middleware: Allows interception of calls to an `IChatClient` implementation, where an agent is using `IChatClient` for inference calls, for example, when using `ChatClientAgent`.
2323

2424
All the types of middleware are implemented via a function callback, and when multiple middleware instances of the same type are registered, they form a chain,
2525
where each middleware instance is expected to call the next in the chain, via a provided `next` `Func`.
@@ -35,10 +35,10 @@ var middlewareEnabledAgent = originalAgent
3535
```
3636

3737
> [!IMPORTANT]
38-
> Ideally both `runFunc` and `runStreamingFunc` should be provided, when providing just the non-streaming middleware, the agent will use it for both streaming and non-streaming invocations and this will block the streaming to run in non-streaming mode to suffice the middleware expectations.
38+
> Ideally both `runFunc` and `runStreamingFunc` should be provided. When providing just the non-streaming middleware, the agent will use it for both streaming and non-streaming invocations. Streaming will only run in non-streaming mode to suffice the middleware expectations.
3939
4040
> [!NOTE]
41-
> There's an additional overload `Use(sharedFunc: ...)` that allows you to provide the same middleware for non-streaming and streaming without blocking the streaming, however, the shared middleware won't be able intercept or override the output, make this the best option only for scenarios where you only need to inspect/modify the input before it reaches the agent.
41+
> There's an additional overload, `Use(sharedFunc: ...)`, that allows you to provide the same middleware for non-streaming and streaming without blocking the streaming. However, the shared middleware won't be able to intercept or override the output. This overload should be used for scenarios where you only need to inspect or modify the input before it reaches the agent.
4242
4343
`IChatClient` middleware can be registered on an `IChatClient` before it is used with a `ChatClientAgent`, by using the chat client builder pattern.
4444

@@ -113,7 +113,7 @@ async IAsyncEnumerable<AgentRunResponseUpdate> CustomAgentRunStreamingMiddleware
113113
## Function calling middleware
114114

115115
> [!NOTE]
116-
> Function calling middleware is currently only supported with an `AIAgent` that uses `Microsoft.Extensions.AI.FunctionInvokingChatClient`, e.g. `ChatClientAgent`.
116+
> Function calling middleware is currently only supported with an `AIAgent` that uses <xref:Microsoft.Extensions.AI.FunctionInvokingChatClient>, for example, `ChatClientAgent`.
117117
118118
Here is an example of function calling middleware, that can inspect and/or modify the function being called, and the result from the function call.
119119

@@ -134,11 +134,11 @@ async ValueTask<object?> CustomFunctionCallingMiddleware(
134134

135135
It is possible to terminate the function call loop with function calling middleware by setting the provided `FunctionInvocationContext.Terminate` to true.
136136
This will prevent the function calling loop from issuing a request to the inference service containing the function call results after function invocation.
137-
If there were more than one function available for invocation during this iteration, it may also prevent any remaining functions from being executed.
137+
If there were more than one function available for invocation during this iteration, it might also prevent any remaining functions from being executed.
138138

139139
> [!WARNING]
140-
> Terminating the function call loop may result in your thread being left in an inconsistent state, e.g. containing function call content with no function result content.
141-
> This may result in the thread being unusable for further runs.
140+
> Terminating the function call loop might result in your thread being left in an inconsistent state, for example, containing function call content with no function result content.
141+
> This might result in the thread being unusable for further runs.
142142
143143
## IChatClient middleware
144144

@@ -160,8 +160,7 @@ async Task<ChatResponse> CustomChatClientMiddleware(
160160
```
161161

162162
> [!NOTE]
163-
> For more information about `IChatClient` middleware, see [Custom IChatClient middleware](/dotnet/ai/microsoft-extensions-ai#custom-ichatclient-middleware)
164-
> in the Microsoft.Extensions.AI documentation.
163+
> For more information about `IChatClient` middleware, see [Custom IChatClient middleware](/dotnet/ai/microsoft-extensions-ai#custom-ichatclient-middleware).
165164
166165
::: zone-end
167166
::: zone pivot="programming-language-python"
@@ -451,6 +450,7 @@ async def blocking_middleware(
451450
Middleware can override results in both non-streaming and streaming scenarios, allowing you to modify or completely replace agent responses.
452451

453452
The result type in `context.result` depends on whether the agent invocation is streaming or non-streaming:
453+
454454
- **Non-streaming**: `context.result` contains an `AgentRunResponse` with the complete response
455455
- **Streaming**: `context.result` contains an async generator that yields `AgentRunResponseUpdate` chunks
456456

0 commit comments

Comments
 (0)