- name
SKILL.md
スキルファイルは公開元のリポジトリに置かれたままです。
コピーしたコマンドをターミナルに貼り付けてください。この SKILL.md がそのエージェントにインストールされます。
npx skills add addyosmani/agent-skills --skill observability-and-instrumentation -a cursor -yコピーしたコマンドをターミナルに貼り付けてください。この SKILL.md がそのエージェントにインストールされます。
npx skills add addyosmani/agent-skills --skill observability-and-instrumentation -a claude-code -yコピーしたコマンドをターミナルに貼り付けてください。この SKILL.md がそのエージェントにインストールされます。
npx skills add addyosmani/agent-skills --skill observability-and-instrumentation -a codex -yコピーしたコマンドをターミナルに貼り付けてください。この SKILL.md がそのエージェントにインストールされます。
npx skills add addyosmani/agent-skills --skill observability-and-instrumentation -a grok -yコピーしたメッセージを Grok Bot のチャットで送信してください。Bot がこの SKILL.md を GitHub から取得し、すべての Bot が共有するスキルに追加します。
Install this skill: https://github.com/addyosmani/agent-skills/tree/HEAD/skills/observability-and-instrumentation公開元より
Code you can't observe is code you can't operate. Observability is the ability to answer "what is the system doing and why?" from the outside, using the telemetry the code emits. Instrumentation is not a post-launch add-on — it's written alongside the feature, the same way tests are. If a feature ships without telemetry, the first user-reported bug becomes archaeology instead of a query.
NOT for:
debugging-and-error-recovery skill (observability is what makes that skill fast next time)performance-optimization skillshipping-and-launch skill; this skill covers the instrumentation that feeds themTelemetry without a question is noise. Before adding any instrumentation, write down 2–4 questions an on-call engineer will ask about this feature:
FEATURE: checkout payment retry
QUESTIONS ON-CALL WILL ASK:
1. What fraction of payments succeed on first attempt vs after retry?
2. When a payment fails permanently, why? (provider error? timeout? validation?)
3. Is the payment provider slower than usual?
→ Every signal below must help answer one of these.
If you can't name the questions, you're not ready to instrument — you'll log everything and learn nothing.
Rule of thumb: metrics tell you that something is wrong, traces tell you where, logs tell you why.
Log events, not prose. Every log line is a JSON object with a stable event name and machine-readable fields:
// BAD: string interpolation — unqueryable, inconsistent
logger.info(`Payment ${id} failed for user ${userId} after ${n} retries`);
// GOOD: stable event name + structured fields
logger.warn({
event: 'payment_failed',
paymentId: id,
provider: 'stripe',
errorCode: err.code,
attempt: n,
}, 'payment failed');
Log levels — use them consistently:
Correlation IDs are mandatory. Generate (or accept) a request ID at the system boundary and attach it to every log line, span, and outbound call. Without it, you cannot reconstruct a single request from interleaved logs:
// Express: child logger per request, ID propagated downstream
app.use((req, res, next) => {
req.id = req.headers['x-request-id'] ?? crypto.randomUUID();
req.log = logger.child({ requestId: req.id });
res.setHeader('x-request-id', req.id);
next();
});
When several entry points write to one log, name the entry point. A correlation ID identifies a run; it does not say which code path started it. The same job reached by a scheduler, by a replay endpoint, and by a manual CLI run produces interchangeable lines in one sink, so attributing a line falls back to elimination — cross-reading the scheduler's history, the process table, a deploy log — and that argument holds only as long as those external records happen to still exist. Stamp the entry point where the run starts, next to the correlation ID, and propagate both the same way:
// One helper for every entry point: the run's own logger carries both fields.
// `entryPoint`, not `source` — ECS reserves `source.*` for network fields.
export const runLog = (entryPoint: 'scheduler' | 'replay_endpoint' | 'cli', runId: string) =>
logger.child({ entryPoint, requestId: runId });
// scheduler tick -> runLog('scheduler', crypto.randomUUID())
// POST /jobs/:id/replay -> runLog('replay_endpoint', req.id)
// CLI invocation -> runLog('cli', process.env.RUN_ID ?? crypto.randomUUID())
Both fields have to cross the same boundaries as the correlation ID — queue metadata, HTTP headers — or a worker re-derives the entry point and guesses. A field that merely correlates with an entry point is a hint, not an attribution: anything that can invoke the job can reproduce it.
Never log secrets, tokens, passwords, or full PII. This is a hard rule from the security-and-hardening skill — telemetry pipelines are a classic data-leak path. Allowlist fields; don't log whole request bodies.
For request-driven services, instrument RED on every endpoint and every external dependency: Rate (requests/sec), Errors (failure rate), Duration (latency histogram, not average). For resources (queues, pools, hosts), use USE: Utilization, Saturation, Errors.
As with tracing, the vendor-neutral path is the OpenTelemetry metrics API (same SDK and context as step 5). The example below uses Prometheus' prom-client — one common backend choice, not the only one; the RED/USE and cardinality rules are identical either way.
import { Histogram } from 'prom-client';
const httpDuration = new Histogram({
name: 'http_request_duration_seconds',
help: 'HTTP request duration',
labelNames: ['method', 'route', 'status_class'], // '2xx', not '200'
buckets: [0.05, 0.1, 0.25, 0.5, 1, 2.5, 5],
});
Cardinality is the failure mode. Every unique label combination is a separate time series. Labels must come from small, fixed sets (route template, status class, provider name). Never use user IDs, raw URLs, error messages, or other unbounded values as labels — that belongs in logs and traces.
OK as label: route="/api/tasks/:id" status_class="5xx" provider="stripe"
NEVER a label: user_id, email, request_id, full URL, error message text
Track averages never, percentiles always: an average hides the 1% of users having a terrible time. Use histograms and read p50/p95/p99.
Use OpenTelemetry — it's the vendor-neutral standard, and auto-instrumentation covers HTTP, gRPC, and common DB clients with near-zero code:
// tracing.ts — must be imported before anything else
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
const sdk = new NodeSDK({
serviceName: 'checkout-service',
instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();
Add manual spans only around meaningful internal units of work (e.g., applyDiscounts, chargeProvider) and attach the attributes on-call will filter by. Propagate context across every async boundary — HTTP headers, queue message metadata — or the trace dies at the gap. Sample head-based at a low rate by default; keep 100% of errors if your backend supports tail sampling.
Alert on symptoms users feel, not on causes:
SYMPTOM (page-worthy): CAUSE (dashboard, not a page):
error rate > 1% for 5 min CPU at 85%
p99 latency > 2s one pod restarted
queue age > 10 min disk at 70%
Cause-based alerts fire when nothing is wrong and miss failures you didn't predict. Symptom-based alerts fire exactly when users are hurt, regardless of the cause.
Rules for every alert you create:
Rule 2 above requires every alert to link to a runbook. A runbook's job is to answer three questions without requiring the reader to think: what is happening, what to check first, and who to call if that doesn't resolve it. Store in docs/runbooks/ named after the alert.
Minimum viable runbook (three lines):
# Runbook: High Error Rate on /api/tasks
**Means:** DB connection pool likely exhausted, or a bad deploy.
**First check:** `SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'client backend';`
— if count > pool limit, see Step 2. (Swap in the equivalent for your database.)
**Escalate to:** #db-oncall or engineering on-call rotation.
When to expand beyond three lines: add steps only when the first check alone isn't enough to decide. A five-step runbook that covers the three most common causes is better than a twenty-step document that covers every edge case and gets skimmed.
Keep runbooks current. Update the runbook as part of closing every incident it was used in — a stale runbook builds false confidence. If a step was wrong or missing, fix it before marking the incident resolved.
Instrumentation is code; it can be wrong. Before calling the work done, trigger the paths and look at the actual output:
requestId, confirm fields are structured (not [object Object])After instrumenting a feature, confirm:
For the at-a-glance version of this list, including the pre-launch instrumentation gate, see ../../references/observability-checklist.md.
Instruments code so production behavior is visible and diagnosable. Use when adding logging, metrics, tracing, or alerting. Use when shipping any feature that runs in production and you need evidence
---name: observability-and-instrumentationdescription: Instruments code so production behavior is visible and diagnosable. Use when adding logging, metrics, tracing…---# skills/observability-and-instrumentation/SKILL.md概要
Instruments code so production behavior is visible and diagnosable. Use when adding logging, metrics, tracing, or alerting. Use when shipping any feature that runs in production and you need evidence it works. Use when production issues are reported but you can't tell what happened from the available data.
よくある質問
公開元から skills/observability-and-instrumentation/SKILL.md をダウンロードするか、パスをコピーしてリポジトリを開いてください。PlugMCP はスキルを掲載するだけで、ファイルはホストしていません。
公開元は https://github.com/addyosmani/agent-skills で管理しています。
Claude、Cursor、Codex、Grok Build は、公開元のリポジトリから追加した SKILL.md を読み込めます。Grok Bot は、チャットでスキルの GitHub フォルダーのリンクを送るとインストールします。