Skip to content

MCP Tools in Drills: Drill Create - #3320

Open
dynamicdhx wants to merge 9 commits into
mainfrom
users/dhruvbharuka/drillcreate
Open

MCP Tools in Drills: Drill Create#3320
dynamicdhx wants to merge 9 commits into
mainfrom
users/dhruvbharuka/drillcreate

Conversation

@dynamicdhx

Copy link
Copy Markdown
Contributor

What does this PR do?

[Provide a clear, concise description of the changes]

[Add additional context, screenshots, or information that helps reviewers]

GitHub issue number?

[Link to the GitHub issue this PR addresses]

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

@dynamicdhx

Copy link
Copy Markdown
Contributor Author

Tool Description Evaluator Results

Test 1

Expected Tool: resilience_drill_create
Prompt: Create a zonal resilience drill named <drill_name> in service group <service_group> using subscription , region , resource group <resource_group>, automated built-in roles, and recovery plan <recovery_plan_name>

Results

Rank Score Tool Status
1 0.731779 resilience_drill_create EXPECTED
2 0.572673 resilience_drill_get
3 0.549801 resilience_drill_resource_get
4 0.485367 resilience_recovery_plan_get
5 0.440381 resilience_usageplan_create

Test 2

Expected Tool: resilience_drill_create
Prompt: Create a regional resilience drill named <drill_name> in service group <service_group> using subscription , region , and manual RBAC setup

Results

Rank Score Tool Status
1 0.660446 resilience_drill_create EXPECTED
2 0.564984 resilience_drill_get
3 0.505161 resilience_drill_resource_get
4 0.415006 resilience_usageplan_enrollment_create
5 0.388699 sreagent_agents_tools_create

Test 3

Expected Tool: resilience_drill_create
Prompt: Create a resilience drill for service group <service_group>

Results

Rank Score Tool Status
1 0.649628 resilience_drill_create EXPECTED
2 0.625004 resilience_drill_get
3 0.517876 resilience_drill_resource_get
4 0.469316 resilience_recovery_plan_get
5 0.464734 resilience_usageplan_enrollment_create

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Azure Resilience Management MCP tool for creating/updating drills, wiring it into the toolset, server documentation, consolidated tool mappings, and test coverage.

Changes:

  • Added resilience_drill_create command (options, models, command implementation) plus service-layer support.
  • Added unit + live/integration test coverage for drill creation.
  • Updated Azure MCP Server docs/metadata (README prompts, azmcp-commands.md, e2e prompts, consolidated-tools mapping) and added a changelog entry.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/ResilienceManagementCommandTests.cs Adds live/integration test coverage for drill creation tool invocation.
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/Drills/DrillCreateCommandTests.cs Adds unit tests for input validation, success path, and error sanitization.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/ResilienceManagementService.cs Adds CreateDrillAsync implementation using ARM SDK types.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/IResilienceManagementService.cs Extends service interface with CreateDrillAsync.
tools/Azure.Mcp.Tools.ResilienceManagement/src/ResilienceManagementSetup.cs Registers the new command and exposes it under the resilience drill subgroup.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Options/Drills/DrillCreateOption.cs Introduces options model for drill create.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Models/DrillRbacSetupMode.cs Adds RBAC setup mode enum for drill creation inputs.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Models/DrillKind.cs Adds drill kind enum (zonal/regional).
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/ResilienceManagementJsonContext.cs Registers the new command result type for source-gen JSON serialization.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/Drills/DrillCreateCommand.cs Implements the create command (validation, execution, error mapping).
servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json Maps resilience_drill_create into the consolidated resilience “create resources” tool list.
servers/Azure.Mcp.Server/README.md Documents example prompts and updates service capability list to include drills.
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md Adds e2e prompts for the new drill create tool.
servers/Azure.Mcp.Server/docs/azmcp-commands.md Documents the new azmcp resilience drill create command and updates drill get/resource signatures.
servers/Azure.Mcp.Server/changelog-entries/resilience-drill-create.yml Adds a changelog entry for the new command.
Suppressed comments (1)

tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/Drills/DrillCreateCommand.cs:95

  • IsValidResourceGroupCharacter currently disallows parentheses, which can cause DrillCreate to reject resource groups that other commands accept (and that Azure allows).
    private static bool IsValidResourceGroupCharacter(char character) =>
        character is >= 'a' and <= 'z' or >= 'A' and <= 'Z' or >= '0' and <= '9' or '_' or '-' or '.';

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@g2vinay Vinay Gera (g2vinay) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good start, my review comments are in-line.

ArmOperation<ResilienceManagementDrillResource> operation = await drills.CreateOrUpdateAsync(WaitUntil.Started, drill, drillData, cancellationToken);
await WaitForLroCompletionAsync(operation, cancellationToken);

using JsonDocument document = JsonDocument.Parse(operation.GetRawResponse().Content.ToMemory());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After WaitUntil.Started + WaitForLroCompletionAsync, operation.GetRawResponse() is the async-operation status envelope, not the drill resource, so it has no id or properties. That is why Should_create_drill fails on the test legs: the asserted id, provisioningState, and drillType are not in that payload. The sibling creates (CreateUsagePlanAsync, CreateUsagePlanEnrollmentAsync) and the new GetDrillAsync all read ...Value.Data. Please return MapDrill(operation.Value.Data) here (that overload already exists) and drop the raw-response parse, then re-record the live test.

…drillcreate

# Conflicts:
#	servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
#	tools/Azure.Mcp.Tools.ResilienceManagement/src/ResilienceManagementSetup.cs
#	tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/IResilienceManagementService.cs
#	tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/ResilienceManagementCommandTests.cs
#	tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/assets.json
@dynamicdhx
dynamicdhx force-pushed the users/dhruvbharuka/drillcreate branch from 5f5f303 to 2a3933b Compare August 26, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants