Microsoft Foundry adds trace analysis for uncovering unknown agent failures

Microsoft has made Insights in Foundry available in public preview. The tool automatically analyzes production traces from AI agents and flags recurring behaviors that teams did not know to look for. It runs within Microsoft Foundry, the company's AI agent development platform, and extends existing tracing, monitoring, and evaluation capabilities.

Predefined evaluations only cover failures a team already anticipated, while reading thousands of individual traces by hand is impractical. Insights scans agent traces stored in the Azure Monitor Application Insights resource connected to a Foundry project; no evaluator needs to be written first. One described example is an order-support agent whose status lookup fails but which still tells the customer the order was delivered. In a single trace that looks like an isolated mistake; across hundreds of requests it becomes a recognizable pattern.

Each finding can include a description, category, severity, status, affected agent version, linked traces, representative examples, and a likely cause or recommended action. Depending on configuration and available evidence, findings may cover tool-call failures, context handling, output quality, reliability, latency, or unnecessary token use. Insights is a starting point for investigation, not a confirmed root cause; an intermediate tool error differs from a failed task. Evidence quality depends on telemetry quality: missing agent identity, version information, spans, or content can limit analysis. Infrastructure outages, dependency failures, quotas, and service-level alerts remain with Azure Monitor and the teams that own those systems.

Insights complements monitoring and evaluation rather than replacing them. For a confirmed finding, Microsoft suggests four possible responses: changing an instruction, tool description, code path, or workflow; adding evaluation coverage so the behavior becomes an explicit test; routing the issue to the owner of a dependency or platform; or using Agent Optimizer, a separate Foundry capability that generates candidate improvements and evaluates them against developer-set goals. For prompt agents and code-based hosted agents, an Insight may include a proposed prompt or code change. Insights does not open pull requests, build evaluators, or deploy fixes by itself; changes are applied through normal source-control, evaluation, and deployment processes.

The first scan reviews the last seven days of traces, and analysis can be run again on demand. A scan that returns no findings does not mean an agent is problem-free. Preview documentation covers supported configurations, limitations, and the model and telemetry costs scans can incur. Microsoft is scheduled to discuss the tools at Microsoft Ignite, November 17–20 in San Francisco, and will hold a Foundry Friday AMA on September 25.