Multi-Tenant M365 Service Desk Automation with LangSmith Fleets

Tue, 21 Jul 2026 22:45:00 +0530

Multi-Tenant M365 Service Desk Automation with LangSmith Fleets

A managed service provider operates Microsoft 365 environments for multiple customer organizations. Routine administrative requests arrive through ServiceNow, but each ticket still requires careful human interpretation: identify the customer, resolve the correct Microsoft tenant, connect with the right permissions, perform the change, verify the end state, update the ticket, and alert the right person if something fails.

This is repetitive work, but it is not low-risk work. A wrong tenant, wrong user, unapproved request, exposed credential, or premature ticket closure can create security and compliance impact for both the MSP and its customer.

AAIC designed an agentic service desk workflow using LangSmith Fleets to make these routine Microsoft 365 operations faster while keeping the safety controls explicit, auditable, and fail-closed.

Let’s discuss your service desk automation use case

Talk to an Agentic AI Expert

Business Problem

The service desk team needed a standardized way to automate eligible Microsoft 365 administrative work without turning ticket text into unchecked executable instructions.

The target workflow had to handle multiple customer tenants and still make conservative decisions at every step. It needed to:

Process ServiceNow Incidents and Requested Items assigned to a designated operational user.
Confirm formal ServiceNow approval for Requested Items before planning any action.
Resolve the customer and Microsoft tenant from the ServiceNow CMDB instead of ticket text or hardcoded mappings.
Extract one supported operation, one target object, and all required parameters.
Ask for explicit human approval of the exact planned action before execution.
Execute through Microsoft Graph only after confirming the connected tenant matches the CMDB tenant.
Verify the actual Microsoft 365 end state before resolving the ticket.
Leave a no-op work note and alert a human when the request is ambiguous, unsupported, or unsafe.

Intended Operation Scope

The first automation scope covers eight single-user Microsoft 365 operations:

Create a user.
Assign a user to a group.
Assign a license.
Remove a license.
Disable a user or block sign-in.
Enable a user.
Reset a password.
Grant an administrative role or privilege.

Some actions are treated as elevated risk depending on context. Disabling a user, granting administrative access, assigning a role-assignable group, removing a license that can deprovision a mailbox, and resetting a privileged account require stronger handling and unmistakable ticket notes.

Each operation must pass implementation, permission, verification, and test gates before it is enabled.

Agentic Solution

The LangSmith Fleet is structured around a supervisor agent that owns orchestration and safety gates, with a synchronous execution subagent dedicated to Microsoft operations.

The workflow is intentionally narrow:

Find open ServiceNow Incidents and Requested Items assigned to the designated operational user.
Decide whether the ticket describes exactly one supported Microsoft 365 operation on one target object.
Confirm ServiceNow approval for Requested Items.
Resolve the customer and authoritative Microsoft tenant through the CMDB.
Extract the exact operation, target, and required parameters.
Add an internal ServiceNow work note describing the planned action and flagging high-risk actions.
Wait for a designated approver to explicitly approve that specific planned action.
Authenticate against the CMDB-resolved tenant and confirm the connected tenant matches it.
Execute exactly one controlled Microsoft Graph operation.
Re-query Microsoft Graph and verify that the actual end state matches the intended end state.
On verified success, add a resolution work note and resolve the ticket.
On failure, leave the ticket open, add a failure work note without secrets, and notify the designated operational user in Microsoft Teams.

The operational user can have different ServiceNow and Teams identities, so the solution maps ticket assignment and failure notification separately. The operational user is also not treated as the action approver unless that approval authority is explicitly assigned.

Core Safety Rule

One ticket -> one customer -> one tenant -> one target object -> one operation

Bulk execution, hardcoded tenants, arbitrary commands generated from ticket text, and guessed parameters are prohibited. Ticket text can describe the requested outcome, but it is not trusted as the source of tenant identity, authorization, or executable instructions.

When any safety gate fails, the agent does nothing to Microsoft 365. It writes an auditable ServiceNow work note explaining why it stopped and alerts the designated operational user when required.

Solution Design

The design uses:

One LangSmith Fleet supervisor for ticket orchestration and safety gates.
One synchronous execution subagent for Microsoft operations.
A ServiceNow ticket-control skill.
A controlled Microsoft Graph execution skill.
A custom Python-based Linux snapshot for the per-thread computer.
OAuth-authenticated remote ServiceNow MCP integration.
Workspace-managed credentials and a restricted access profile.
Microsoft Graph for controlled execution and independent end-state verification.
A planned native Fleet Teams integration for failure alerts.

ServiceNow and the CMDB are authoritative for tickets, approvals, customer identity, and tenant mapping. Microsoft Graph is authoritative for the Microsoft starting state and ending state. The execution subagent runs on the same thread computer and creates a responsibility boundary for controlled operations.

Before and After Scorecard

The following scorecard defines the operating targets for the rollout and should be replaced with measured production values after the first deployment cycle.

Ticket handling time

Before: 6-8 minutes of repetitive engineer work per routine Microsoft 365 ticket.

After target: 1-2 minutes of engineer review for eligible tickets after the agent prepares, executes, verifies, and documents the controlled action.

Safety coverage

Before: Safety depended on individual engineer discipline across tenant lookup, approval checks, execution, and documentation.

After target: 100% of eligible write actions pass approval, CMDB tenant match, planned-action approval, and Microsoft Graph end-state verification gates.

Audit readiness

Before: Ticket notes could vary by engineer and by request type.

After target: Every supported action creates planned-action, verification, success, or failure notes in ServiceNow without exposing secrets.

What LangSmith Fleets Standardized

LangSmith Fleets provides the repeatable operating model for this agentic workflow:

Daily scheduled execution without a separate scheduler service.
Versioned prompts, skills, contracts, and runtime snapshots.
A per-thread computer for controlled execution.
Clear supervisor and execution-subagent responsibilities.
Remote MCP integration for ServiceNow.
Workspace-managed secrets and restricted access profiles.
Reusable approval, verification, failure, and audit patterns.
Fail-closed validation before Microsoft operations are enabled.

Impact

The design helps the MSP move routine Microsoft 365 administration from manual execution to controlled agent-assisted operations.

The value is not just faster ticket handling. The bigger operational gain is consistency: every eligible ticket follows the same tenant-resolution path, approval gate, execution boundary, verification step, ticket note format, and failure escalation behavior.

For MSPs managing multiple Microsoft 365 tenants, that consistency matters. It reduces avoidable risk while freeing engineers from repetitive administrative work that still needs strong governance.

Build a Similar Agentic Workflow

If your service desk runs high-volume operational tickets across ServiceNow, Microsoft 365, Azure, AWS, security tools, or internal systems, AAIC can help design the right agent boundary, approval model, integration pattern, and measurable rollout plan.

Discuss your agentic service desk workflow

ServiceNow | Applied AI Consulting

Multi-Tenant M365 Service Desk Automation with LangSmith Fleets