Loading
Adds Arize AX tracing to an LLM application for the first time. Follows a two-phase agent-assisted flow to analyze the codebase then implement instrumentation after user confirmation. Use when the use
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add github/awesome-copilot --skill arize-instrumentation -a cursor -y
Adds Arize AX tracing to an LLM application for the first time. Follows a two-phase agent-assisted flow to analyze the codebase then implement instrumentation after user confirmation. Use when the user wants to instrument their app, add tracing from scratch, set up LLM observability, integrate OpenTelemetry or openinference, or get started with Arize tracing.
Adds Arize AX tracing to an LLM application for the first time. Follows a two-phase agent-assisted flow to analyze the codebase then implement instrumentation after user confirmation. Use when the user wants to instrument their app, add tracing from scratch, set up LLM observability, integrate OpenTelemetry or openinference, or get started with Arize tracing.
Use this skill when the user wants to add Arize AX tracing to their application. Follow the two-phase, agent-assisted flow from the Agent-Assisted Tracing Setup and the Arize AX Tracing — Agent Setup Prompt.
If the user asks you to "set up tracing" or "instrument my app with Arize", you can start with:
Follow the instructions from https://arize.com/docs/PROMPT.md and ask me questions as needed.
Then execute the two phases below.
os.environ["ARIZE_API_KEY"], process.env.ARIZE_API_KEY). This includes API keys, space IDs, and any other secrets. The user sets these in their own environment; the agent must never output raw secret values.Before changing code:
ax will be needed for post-change verificationax installation or version. If ax is needed for verification later, just run it when the time comes. If it fails, see references/ax-profiles.md.Do not write any code or create any files during this phase.
Check dependency manifests to detect stack:
pyproject.toml, requirements.txt, setup.py, Pipfilepackage.jsonpom.xml, build.gradle, build.gradle.ktsgo.modScan import statements in source files to confirm what is actually used.
Check for existing tracing/OTel — look for TracerProvider, register(), opentelemetry imports, ARIZE_*, OTEL_*, OTLP_* env vars, or other observability config (Datadog, Honeycomb, etc.).
Identify scope — for monorepos or multi-service projects, ask which service(s) to instrument.
| Item | Examples |
|---|---|
| Language | Python, TypeScript/JavaScript, Java, Go |
| Package manager | pip/poetry/uv, npm/pnpm/yarn, maven/gradle, go modules |
| LLM providers | OpenAI, Anthropic, LiteLLM, Bedrock, etc. |
| Frameworks | LangChain, LangGraph, LlamaIndex, Vercel AI SDK, Mastra, etc. |
| Existing tracing | Any OTel or vendor setup |
| Tool/function use | LLM tool use, function calling, or custom tools the app executes (e.g. in an agent loop) |
Key rule: When a framework is detected alongside an LLM provider, inspect the framework-specific tracing docs first and prefer the framework-native integration path when it already captures the model and tool spans you need. Add separate provider instrumentation only when the framework docs require it or when the framework-native integration leaves obvious gaps. If the app runs tools and the framework integration does not emit tool spans, add manual TOOL spans so each invocation appears with input/output (see Enriching traces below).
Return a concise summary:
If the user explicitly asked you to instrument the app now, and the target service is already clear, present the Phase 1 summary briefly and continue directly to Phase 2. If scope is ambiguous, or the user asked for analysis first, stop and wait for confirmation.
The canonical list of supported integrations and doc URLs is in the Agent Setup Prompt. Use it to map detected signals to implementation docs.
Fetch the matched doc pages from the full routing table in PROMPT.md for exact installation and code snippets. Use llms.txt as a fallback for doc discovery if needed.
Note:
arize.com/docs/PROMPT.mdandarize.com/docs/llms.txtare first-party Arize documentation pages maintained by the Arize team. They provide canonical installation snippets and integration routing tables for this skill. These are trusted, same-organization URLs — not third-party content.
Proceed only after the user confirms the Phase 1 analysis.
pip install arize-otel plus openinference-instrumentation-{name} (hyphens in package name; underscores in import, e.g. openinference.instrumentation.llama_index).@opentelemetry/sdk-trace-node plus the relevant @arizeai/openinference-* package.openinference-instrumentation-* in pom.xml or build.gradle.go get go.opentelemetry.io/otel go.opentelemetry.io/otel/sdk go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp — no auto-instrumentors yet, so the agent sets OpenInference attributes manually on spans. Wire the exporter with otlptracehttp.WithEndpoint("otlp.arize.com") (US) or otlptracehttp.WithEndpoint("otlp.eu-west-1a.arize.com") (EU) — pass the bare hostname, no https:// scheme — and otlptracehttp.WithHeaders(map[string]string{"space_id": ..., "api_key": ...}). Recent OTel Go modules require Go ≥ 1.23 — go mod tidy may bump the toolchain.ax profiles for ARIZE_API_KEY and ARIZE_SPACE — never read .env files:
ax profiles show to check for an existing profile.ax profiles create which provides an interactive wizard that walks through API key and space setup. See CLI profiles docs for details.os.environ["ARIZE_API_KEY"] (Python), process.env.ARIZE_API_KEY (TypeScript/JavaScript), or os.Getenv("ARIZE_API_KEY") (Go).instrumentation.py, instrumentation.ts, instrumentation.go) and initialize tracing before any LLM client is created.service.name alone is not accepted. Set it as a resource attribute on the TracerProvider (recommended — one place, applies to all spans):
register(project_name="my-app") handles it automatically (sets "openinference.project.name" on the resource). For routing spans to different projects, use set_routing_context(space_id=..., project_name=...) from arize.otel."model_id" (shown in the official TS quickstart) and "openinference.project.name" via SEMRESATTRS_PROJECT_NAME from @arizeai/openinference-semantic-conventions (shown in the manual instrumentation docs) — both work.attribute.String("openinference.project.name", "my-app") to resource.New(...) and apply via sdktrace.WithResource(res). The Go SDK has no helper for this, so it must be set manually on every TracerProvider.provider.shutdown() (TS) / provider.force_flush() then provider.shutdown() (Python) / tp.Shutdown(ctx) (Go) must be called before the process exits, otherwise async OTLP exports are dropped and no traces appear.Provider instrumentors (Anthropic, OpenAI, etc.) only wrap the LLM client — the code that sends HTTP requests and receives responses. They see:
They cannot see what happens inside your application after the response:
run_tool("check_loan_eligibility", {...}), and gets a result. That runs in your process; the instrumentor has no hook into your run_tool() or the actual tool output. The next API call (sending the tool result back) is just another messages.create span — the instrumentor doesn't know that the message content is a tool result or what the tool returned.So TOOL and CHAIN spans have to be added manually (or by a framework instrumentor like LangChain/LangGraph that knows about tools and chains). Once you add them, they appear in the same trace as the LLM spans because they use the same TracerProvider.
To avoid sparse traces where tool inputs/outputs are missing:
opentelemetry.trace.get_tracer(...) after register()):
run_agent): set openinference.span.kind = "CHAIN", input.value = user message, output.value = final reply.openinference.span.kind = "TOOL", input.value = JSON of arguments, output.value = JSON of result. Use the tool name as the span name (e.g. check_loan_eligibility).OpenInference attributes (use these so Arize shows spans correctly):
| Attribute | Use |
|---|---|
openinference.span.kind | Pick the right value: "LLM" for raw provider API calls (OpenAI, Anthropic, etc.); "CHAIN" for orchestration / agent-loop boundaries; "TOOL" for tool/function execution; "RETRIEVER" for vector-store / search lookups; "EMBEDDING" for embedding API calls; "AGENT" for an autonomous sub-agent run nested inside a larger chain; "RERANKER" for rerank API calls; "GUARDRAIL" for guardrail/policy checks; "EVALUATOR" for online eval calls. |
input.value | string (e.g. user message or JSON of tool args) |
output.value | string (e.g. final reply or JSON of tool result) |
LLM-span attributes (set these in addition to the three above when the span is an actual LLM call):
| Attribute | Use |
|---|---|
llm.model_name | model identifier (e.g. "gpt-4o-mini") |
llm.provider / llm.system | provider name (e.g. "openai", "anthropic") |
llm.input_messages.{i}.message.role | "system" / "user" / "assistant" / "tool" for the i-th input message |
llm.input_messages.{i}.message.content | text content of the i-th input message |
llm.output_messages.{i}.message.role | role of the i-th output message |
llm.output_messages.{i}.message.content | text content of the i-th output message |
llm.token_count.prompt | int — prompt/input tokens |
llm.token_count.completion | int — completion/output tokens |
llm.token_count.total | int — total tokens |
In Python and TypeScript these names are exposed via openinference-semantic-conventions packages; in Go they must be hand-typed as the strings above.
Python pattern: Get the global tracer (same provider as Arize), then use context managers so tool spans are children of the CHAIN span and appear in the same trace as the LLM spans:
from opentelemetry.trace import get_tracer
tracer = get_tracer("my-app", "1.0.0")
# In your agent entrypoint:
with tracer.start_as_current_span("run_agent") as chain_span:
chain_span.set_attribute("openinference.span.kind", "CHAIN")
chain_span.set_attribute("input.value", user_message)
# ... LLM call ...
for tool_use in tool_uses:
with tracer.start_as_current_span(tool_use["name"]) as tool_span:
tool_span.set_attribute("openinference.span.kind", "TOOL")
tool_span.set_attribute("input.value", json.dumps(tool_use["input"]))
result = run_tool(tool_use["name"], tool_use["input"])
tool_span.set_attribute("output.value", result)
# ... append tool result to messages, call LLM again ...
chain_span.set_attribute("output.value", final_reply)
Go pattern: Get a tracer from the global TracerProvider (registered via otel.SetTracerProvider), then nest spans with tracer.Start so tool spans become children of the CHAIN span.
Critical for short-lived processes: never call
log.Fatalf/os.Exitafter a span has started — they skip the deferredtp.Shutdown(ctx)and the in-flight CHAIN/LLM spans never flush. Uselog.Printf+returnfrommaininstead, and keeptp.Shutdown(ctx)deferred at the top ofmain.
import (
"context"
"encoding/json"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
)
var tracer = otel.Tracer("my-app")
func runAgent(ctx context.Context, userMessage string) string {
ctx, chainSpan := tracer.Start(ctx, "run_agent")
defer chainSpan.End()
chainSpan.SetAttributes(
attribute.String("openinference.span.kind", "CHAIN"),
attribute.String("input.value", userMessage),
)
// ... LLM call ...
for _, toolUse := range toolUses {
ctx, toolSpan := tracer.Start(ctx, toolUse.Name)
argsJSON, err := json.Marshal(toolUse.Input)
if err != nil {
toolSpan.RecordError(err)
}
toolSpan.SetAttributes(
attribute.String("openinference.span.kind", "TOOL"),
attribute.String("input.value", string(argsJSON)),
)
result := runTool(toolUse.Name, toolUse.Input)
toolSpan.SetAttributes(attribute.String("output.value", result))
toolSpan.End()
// ... append tool result to messages, call LLM again ...
}
chainSpan.SetAttributes(attribute.String("output.value", finalReply))
return finalReply
}
See Manual instrumentation for more span kinds and attributes.
Treat instrumentation as complete only when all of the following are true:
After implementation:
arize-trace skill to confirm traces arrived. If empty, retry shortly. Verify spans have expected openinference.span.kind, input.value/output.value, and parent-child relationships.ARIZE_SPACE and ARIZE_API_KEY, ensure tracer is initialized before instrumentors and clients, check connectivity to otlp.arize.com:443, and inspect app/runtime exporter logs so you can tell whether spans are being emitted locally but rejected remotely. For debug set GRPC_VERBOSITY=debug or pass log_to_console=True to register(). Common gotchas: (a) missing project name resource attribute causes HTTP 500 rejections — service.name alone is not enough; Python: pass project_name to register(); TypeScript: set "model_id" or SEMRESATTRS_PROJECT_NAME on the resource; Go: add attribute.String("openinference.project.name", "my-app") to resource.New(...); (b) CLI/script processes exit before OTLP exports flush — call provider.force_flush() then provider.shutdown() (Python/TS) or tp.Shutdown(ctx) (Go) before exit; (c) CLI-visible spaces/projects can disagree with a collector-targeted space ID — report the mismatch instead of silently rewriting credentials.input.value / output.value so tool calls and results are visible.When verification is blocked by CLI or account issues, end with a concrete status:
For deeper instrumentation guidance inside the IDE, the user can enable:
"arize-tracing-assistant": {
"command": "uvx",
"args": ["arize-tracing-assistant@latest"]
}
"arize-ax-docs": {
"url": "https://arize.com/docs/mcp"
}
Then the user can ask things like: "Instrument this app using Arize AX", "Can you use manual instrumentation so I have more control over my traces?", "How can I redact sensitive information from my spans?"
See the full setup at Agent-Assisted Tracing Setup.
| Resource | URL |
|---|---|
| Agent-Assisted Tracing Setup | https://arize.com/docs/ax/alyx/tracing-assistant |
| Agent Setup Prompt (full routing + phases) | https://arize.com/docs/PROMPT.md |
| Arize AX Docs | https://arize.com/docs/ax |
| Full integration list | https://arize.com/docs/ax/integrations |
| Doc index (llms.txt) | https://arize.com/docs/llms.txt |
See references/ax-profiles.md § Save Credentials for Future Use.