Loading
Guide developers through capturing diagnostic artifacts to diagnose production .NET performance issues. Use when the user needs help choosing diagnostic tools, collecting performance data, or understa
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add dotnet/skills --skill dotnet-trace-collect -a cursor -y
Guide developers through capturing diagnostic artifacts to diagnose production .NET performance issues. Use when the user needs help choosing diagnostic tools, collecting performance data, or understanding tool trade-offs across different environments (Windows/Linux, .NET Framework/modern .NET, container/non-container).
Guide developers through capturing diagnostic artifacts to diagnose production .NET performance issues. Use when the user needs help choosing diagnostic tools, collecting performance data, or understanding tool trade-offs across different environments (Windows/Linux, .NET Framework/modern .NET, container/non-container).
This skill helps developers diagnose production performance issues by recommending the right diagnostic tools for their environment, guiding data collection, and suggesting analysis approaches. It does not analyze code for anti-patterns or perform the analysis itself.
| Input | Required | Description |
|---|---|---|
| Symptom | Yes | What the developer is observing (high CPU, memory growth, slow requests, hangs, excessive GC, HTTP 5xx errors, networking timeouts, connection failures, assembly loading failures, etc.) |
| Runtime | Yes | .NET Framework or modern .NET (and version, especially whether .NET 10+) |
| OS | Yes | Windows or Linux |
| Deployment | Yes | Non-container, container, or Kubernetes |
| Admin privileges | Recommended | Whether the developer has admin/root access on the target machine |
| Repro characteristics | Recommended | Whether the issue is easy to reproduce or requires a long time to manifest |
Determine or ask the developer to clarify:
Use this information to select the right tool in Step 2.
Select tools based on the environment using the priority rules below. Once a tool is selected, load the corresponding reference file for detailed command-line usage.
| Environment | Reference file(s) |
|---|---|
| Windows + modern .NET + admin | references/perfview.md |
| Windows + modern .NET, no admin | references/dotnet-trace-collect.md |
| Windows + .NET Framework | references/perfview.md |
| Linux + .NET 10+ + root | references/dotnet-trace-collect-linux.md |
| Linux + pre-.NET 10 | references/dotnet-trace-collect.md |
| Linux + native stacks needed | references/perfcollect.md |
| Container/K8s (console access) | references/dotnet-trace-collect.md (or dotnet-trace-collect-linux.md) |
| Container/K8s (no console) | references/dotnet-monitor.md |
| Environment | Preferred tool | Fallback / Notes |
|---|---|---|
| Windows + modern .NET + admin | PerfView | If admin is unavailable, use dotnet-trace |
| Windows + .NET Framework + admin | PerfView | Without admin, there is no trace fallback; for hangs/memory leaks, provide dump commands directly (procdump -ma or Task Manager) since dump-collect does not support .NET Framework |
| Linux + .NET 10+ + root | dotnet-trace collect-linux | Use dotnet-trace if root or kernel prerequisites are not met |
| Linux + pre-.NET 10 | dotnet-trace | Add perfcollect when native stacks are needed (requires root) |
| Linux container/Kubernetes | Console tools if in workload context; dotnet-monitor if no console access | See Linux Container / Kubernetes section for details |
/ThreadTime to capture thread-level wait and block detail.dotnet-trace — fallback when admin privileges are not available./StopOn trigger that fires on the symptom you want to capture (e.g., /StopOnPerfCounter, /StopOnGCEvent, /StopOnException) and a circular buffer (/CircularMB + /BufferSizeMB). Critical: the stop trigger must fire on the interesting event, not the recovery. The circular buffer continuously overwrites old data, so if you trigger on recovery, the buffer may have already overwritten the interesting behavior by the time collection stops. Only add /StartOn if the start event is known to precede the stop event. For slow requests, do not include a stop trigger by default — let the user design one based on their specific scenario.PerfView — most Windows containers (including Kubernetes on Windows) use process-isolation by default. Collect from the host with /EnableEventsInContainers. After collection, you have two options:
.etl.zip into the container and run PerfViewCollect merge /ImageIDsOnly inside it to embed symbol information. Then copy the merged trace out. Without this merge step, symbols for binaries inside the container will be unresolvable on other machines.For the less common Hyper-V containers, collect inside the container directly. See references/perfview.md for detailed commands.
dotnet-monitor, dotnet-trace — inside the container if the tools are installed in the image. For dumps, invoke the dump-collect skill.
/StopOn triggers that fire on the symptom (e.g., /StopOnPerfCounter, /StopOnGCEvent, /StopOnException) with /CircularMB + /BufferSizeMB.procdump -ma <PID> or Task Manager) since the dump-collect skill does not support .NET Framework. Dumps can help diagnose hangs and memory leaks. However, for high CPU, slow requests, and excessive GC, there is no way to investigate on .NET Framework without admin access. Advise the user to obtain admin privileges.dotnet-trace collect-linux (preferred) — uses perf_events for richer traces including native call stacks and kernel events. Captures machine-wide by default (no PID required). Requires root and kernel >= 6.4.dotnet-trace — fallback when root privileges are not available or kernel requirements are not met. Managed stacks only.dotnet-trace (preferred) — managed trace collection; no admin required.perfcollect — when native call stacks are needed (requires admin/root).If running in the context of the workload (i.e., you have console access to the container), prefer console-based tools. These are easier to set up than dotnet-monitor, which requires authentication configuration and sidecar deployment:
dotnet-trace collect-linux (.NET 10+ with root) — produces the richest traces including native call stacks and kernel events.dotnet-trace — inside the container if the tool is installed in the image. For dumps, invoke the dump-collect skill.perfcollect — inside the container when native stacks are needed on pre-.NET 10 (requires SYS_ADMIN / --privileged).If not running in the workload context (no console access), or if dotnet-monitor is already deployed:
dotnet-monitor — designed for containers; runs as a sidecar. No tools needed in the app container. Easiest option when console access is not available.When dumps are needed (memory leaks, hangs), do not provide dump collection commands directly for modern .NET — invoke the dump-collect skill instead. The dump-collect skill only supports modern .NET (.NET Core 3.0+). For .NET Framework, provide dump collection guidance directly (e.g., procdump -ma <PID> or Task Manager). This skill focuses on trace collection only.
dump-collect skill for dump collection — do not provide dump commands directly. Diff the dumps in PerfView to see which objects have increased — this is the most effective way to identify what is leaking.dump-collect skill) for heap diff, plus dotnet-trace while memory is growing (for allocation tracking). No trigger needed — capture during the growth period. Both together give the best picture.dump-collect skill) for heap diff, plus dotnet-trace collect-linux while memory is growing (richer data including native stacks). No trigger needed.dump-collect skill does not support .NET Framework, so provide dump commands directly (e.g., procdump -ma <PID> or right-click → Create Dump File in Task Manager). No trigger is needed — just capture the trace during the growth period. Do not wait for an OutOfMemoryException.Excessive GC requires a trace to analyze GC events, pause times, and allocation patterns — a dump is not sufficient.
PerfView collect /GCCollectOnly to capture GC events.dotnet-trace collect -p <PID> --profile gc-verbose.dotnet-trace collect-linux --profile gc-verbose for richer data with native stacks.dotnet-monitor can capture GC traces via its REST API (/trace?profile=gc-verbose).Slow requests require a thread time trace to see where threads are spending time — waiting on locks, I/O, external calls, etc. Use larger buffers since thread time traces generate more data. For ASP.NET Core applications, also enable Microsoft.AspNetCore.Hosting and Microsoft-AspNetCore-Server-Kestrel providers to get server-side request lifecycle timing (when requests arrive, how long they take to process).
PerfView /ThreadTime collect /BufferSizeMB:1024 /CircularMB:2048. The /ThreadTime argument adds thread-level wait and block detail. For ASP.NET Core, add Kestrel providers: PerfView /ThreadTime collect /BufferSizeMB:1024 /CircularMB:2048 /Providers:*Microsoft.AspNetCore.Hosting,*Microsoft-AspNetCore-Server-Kestrel. Do not include a stop trigger by default — let the user design one based on their specific scenario.dotnet-trace captures thread time data by default — no special arguments needed. Use dotnet-trace collect -p <PID>. For ASP.NET Core, add Kestrel providers: dotnet-trace collect -p <PID> --providers Microsoft.AspNetCore.Hosting,Microsoft-AspNetCore-Server-Kestrel.dotnet-trace collect-linux --profile thread-time for richer data with native stacks. For ASP.NET Core, add: --providers Microsoft.AspNetCore.Hosting,Microsoft-AspNetCore-Server-Kestrel.dotnet-monitor can capture traces via its REST API (/trace?pid=<PID>&durationSeconds=30)./ThreadTime on Windows, dotnet-trace on Linux, dotnet-trace collect-linux --profile thread-time on .NET 10+ Linux with root). The trace can reveal:
dump-collect skill to collect a process dump — do not provide dump commands directly.lldb with the SOS debugger extension.Networking issues (HTTP 5xx errors from downstream services, request timeouts, connection failures, DNS resolution failures, TLS handshake failures, connection pool exhaustion) require both a thread-time trace and networking event providers. The thread-time trace shows where threads are blocked (slow downstream calls, thread starvation), while the networking events show the request lifecycle — which requests failed, what status codes came back, how long DNS resolution and TLS handshakes took, and how long requests waited for a connection from the pool.
For .NET Framework, PerfView /ThreadTime already collects the relevant networking events (from the System.Net ETW provider) — no additional providers are needed.
For modern .NET, you must explicitly enable the System.Net.* EventSource providers:
| Provider | What it covers |
|---|---|
System.Net.Http | HttpClient/SocketsHttpHandler — request lifecycle, HTTP status codes, connection pool |
System.Net.NameResolution | DNS lookups (start/stop, duration) |
System.Net.Security | TLS/SSL handshakes (SslStream) |
System.Net.Sockets | Low-level socket connect/disconnect |
Key events from System.Net.Http: RequestStart (scheme, host, port, path), RequestStop (statusCode — -1 if no response was received), RequestFailed (exception message for timeouts, connection refused, etc.), RequestLeftQueue (time waiting for a connection from the pool — indicates connection pool exhaustion), ConnectionEstablished, ConnectionClosed.
Collect a thread-time trace with networking providers enabled (modern .NET only — .NET Framework needs only PerfView /ThreadTime):
PerfView /ThreadTime collect /BufferSizeMB:1024 /CircularMB:2048 /Providers:*System.Net.Http,*System.Net.NameResolution,*System.Net.Security,*System.Net.Sockets. For .NET Framework, omit the /Providers flag — /ThreadTime already includes the networking events. The thread-time trace shows where threads are blocked while the networking events show what requests are failing and why.dotnet-trace captures thread time data by default, but specifying --providers overrides the defaults so you must also include --profile: dotnet-trace collect -p <PID> --profile dotnet-common,dotnet-sampled-thread-time --providers System.Net.Http,System.Net.NameResolution,System.Net.Security,System.Net.Sockets.dotnet-trace collect-linux --profile dotnet-common,cpu-sampling,thread-time --providers System.Net.Http,System.Net.NameResolution,System.Net.Security,System.Net.Sockets.dotnet-monitor can capture traces with custom providers via its REST API.For modern .NET, assembly loading issues (FileNotFoundException, FileLoadException, ReflectionTypeLoadException, version conflicts, duplicate assembly loads across AssemblyLoadContexts) require collecting assembly loader binder events from the Microsoft-Windows-DotNETRuntime provider with the Loader keyword (0x4). These events trace every step of the runtime's assembly resolution algorithm — which paths were probed, which AssemblyLoadContext handled the load, whether the load succeeded or failed, and why. For .NET Framework, the same provider and keyword work for ETW-based collection; additionally, the Fusion Log Viewer (fuslogvw.exe) can diagnose assembly binding failures without requiring a trace.
The provider specification is Microsoft-Windows-DotNETRuntime:0x4:4 (provider name, AssemblyLoader keyword, Informational verbosity).
PerfView collect with no extra providers. For a smaller trace file, use PerfView collect /ClrEvents:Default-Profile, which removes the most verbose default events while keeping the events necessary for diagnosing assembly loading issues.dotnet-trace collect --clrevents assemblyloader -- <path-to-built-exe> to launch and trace the process, or dotnet-trace collect --clrevents assemblyloader -p <PID> to attach to a running process.dotnet-trace collect-linux --clrevents assemblyloader.dotnet-monitor can capture traces with the loader provider via its REST API.For short-lived processes that fail on startup (common with assembly loading issues), prefer the dotnet-trace launch form (-- <path-to-built-exe>) over attaching by PID, since the process may exit before you can attach.
Explain the trade-offs when recommending a tool. For example:
dotnet-trace works cross-platform without admin but captures less system-level detail.perfcollect captures native call stacks but needs admin/root.dotnet-monitor is the best option for containers/K8s when console access is not available, but requires sidecar deployment and authentication configuration.Provide the specific commands for the recommended tool. Load the appropriate reference file from the tool reference lookup table for detailed command-line examples.
Key guidance to include:
dotnet tool install -g dotnet-trace). When recommending multiple tools, provide installation and usage instructions for each one — do not mention a tool without showing how to install and use it.-p <PID> command): Verify the target process first (for example: dotnet-trace ps, curl <monitor-endpoint>/processes, or ps inside a container). If the app is expected to be PID 1 in a container, still verify before collecting.kubectl cp to copy it in.dotnet-monitor as a sidecar with a shared diagnostic port (Unix domain socket in /tmp).dotnet-monitor as a sidecar container, or kubectl debug for ephemeral debug containers.After data is collected, recommend the appropriate tool for analysis. Do not perform the analysis — just point the developer to the right tool and documentation.
| Collected Data | Analysis Tool | Notes |
|---|---|---|
.nettrace file | PerfView (Windows), Speedscope (web) | PerfView gives the richest view on Windows |
.etl / .etl.zip file | PerfView | ETW traces from PerfView or perfcollect |
perf.data.nl from perfcollect | PerfView (Windows) | Copy the file to a Windows machine and open with PerfView |
| Pitfall | Solution |
|---|---|
Using dotnet-trace on .NET Framework | dotnet-trace only works with modern .NET (.NET Core 3.0+). Use PerfView for .NET Framework. |
| PerfView without admin privileges | PerfView requires admin for ETW tracing. Fall back to dotnet-trace if admin is not available. |
perfcollect in container without SYS_ADMIN | Containers drop SYS_ADMIN by default. Run with --privileged or add SYS_ADMIN capability, or fall back to dotnet-trace. |
| Huge trace files from long repros | On Windows, use PerfView /StopOn triggers that fire on the symptom you want to capture (e.g., /StopOnPerfCounter, /StopOnGCEvent, /StopOnException) with /CircularMB and /BufferSizeMB. Never trigger on recovery — the circular buffer continuously overwrites old data, so the interesting behavior may be lost by the time collection stops. |
| Diagnostic port not accessible in container | Mount /tmp as a shared volume between the app container and dotnet-monitor sidecar for the diagnostic Unix domain socket. |
| Forgetting to install tools in container image | Add dotnet tool install to your Dockerfile, or use dotnet-monitor as a sidecar to avoid modifying the app image. |
Exposing dotnet-monitor with --no-auth in production | Keep auth enabled, bind to localhost, and use kubectl port-forward for access. Use --no-auth only for short-lived isolated debugging. |
| Collecting only CPU/thread-time trace for networking issues | CPU and thread-time traces alone do not show HTTP status codes, DNS timing, or connection pool behavior. Add the networking providers (System.Net.Http, System.Net.NameResolution, System.Net.Security, System.Net.Sockets) alongside the thread-time trace. |
| Enabling all networking providers when only one is needed | Each networking provider adds overhead. If the issue is clearly HTTP-level (5xx status codes), System.Net.Http alone may be sufficient. Add DNS, TLS, and socket providers when the root cause is unclear. |