- name
SKILL.md
스킬 파일은 게시자의 저장소에 그대로 있습니다.
복사한 명령을 터미널에 붙여넣습니다. 이 SKILL.md가 해당 에이전트에 설치됩니다.
npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation -a cursor -y복사한 명령을 터미널에 붙여넣습니다. 이 SKILL.md가 해당 에이전트에 설치됩니다.
npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation -a claude-code -y복사한 명령을 터미널에 붙여넣습니다. 이 SKILL.md가 해당 에이전트에 설치됩니다.
npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation -a codex -y복사한 명령을 터미널에 붙여넣습니다. 이 SKILL.md가 해당 에이전트에 설치됩니다.
npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation -a grok -y복사한 메시지를 Grok Bot 채팅에 보냅니다. Bot이 GitHub에서 이 SKILL.md를 가져와 모든 Bot이 공유하는 스킬에 추가합니다.
Install this skill: https://github.com/sickn33/agentic-awesome-skills/tree/HEAD/plugins/agentic-awesome-skills-claude/skills/agent-evaluation게시자 제공
Evaluate observable agent behavior against task-specific cases. Modified by AAS maintainers on 2026-09-05 to remove unsupported benchmark claims, correct uncertainty/error reporting and separate optional architecture sketches from the operating procedure.
Use when comparing a changed agent, prompt or tool configuration, reproducing an observed failure, or estimating reliability on a declared task distribution. Do not infer product readiness from a public benchmark percentage or a generic score threshold.
A synthetic agent changes how it chooses a tenant identifier for a read-only lookup. Freeze three cases: an authorized lookup must return the seeded fixture, an unauthorized tenant must be rejected without a tool call, and a simulated tool outage must be classified as infrastructure failure. Supply neither real customer records nor production credentials.
Predeclare five repeats per case with fresh state, the same budget for baseline and candidate, and zero tolerance for an unauthorized tool call. Suppose the candidate returns the expected authorized result in all five runs but makes one unauthorized call in the second case: the candidate fails the permission contract even if its aggregate success rate improves. Retain that run, fix argument authorization, verify the negative case, and rerun the frozen suite. If the outage detector itself crashes, mark that case inconclusive and repair the detector before comparing versions. These are illustrative outcomes, not measured agent results.
Expected output:
contract: case-set revision, rules, repeat plan and budget
versions: baseline, candidate, model, prompt, tool and runtime
runs: one record per attempt, classified outcome and bounded evidence reference
comparison: per-case results, uncertainty, regressions and critical violations
decision: pass | fail | inconclusive; reason; unresolved work
Ten successes in ten independent trials do not demonstrate 100% reliability. This dependency-free helper returns an approximate 95% Wilson interval; for 10/10 it is about [0.7225, 1]. For zero trials it rejects the input.
function wilson95(passes, trials) {
if (!Number.isSafeInteger(passes) || !Number.isSafeInteger(trials)
|| trials <= 0 || passes < 0 || passes > trials) throw new Error('Invalid counts');
const z = 1.959963984540054;
const p = passes / trials;
const denominator = 1 + z * z / trials;
const center = (p + z * z / (2 * trials)) / denominator;
const margin = z * Math.sqrt(p * (1 - p) / trials + z * z / (4 * trials * trials)) / denominator;
return [Math.max(0, center - margin), Math.min(1, center + margin)];
}
Expected checks: 0/10 has a positive upper bound; 10/10 has a lower bound below 1; 0/0 fails. Use case-level or clustered uncertainty when repeated runs share cases or state; pooling correlated runs as independent observations overstates confidence. See NIST interval guidance.
Read the corresponding section in the bundled architecture sketches only when designing a custom harness:
The classes require application-specific adapters and are not copy-and-run implementations. No listed tool, related skill or delegate is a required dependency.
Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.
---name: agent-evaluationdescription: Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt…---# plugins/agentic-awesome-skills-claude/skills/agent-evaluation/SKILL.mdFAQ
게시자에게서 plugins/agentic-awesome-skills-claude/skills/agent-evaluation/SKILL.md를 다운로드하거나 경로를 복사해 저장소를 엽니다. PlugMCP는 스킬을 목록에 올릴 뿐 파일을 호스팅하지 않습니다.
게시자가 https://github.com/sickn33/agentic-awesome-skills에서 관리합니다.
Claude, Cursor, Codex, Grok Build는 게시자의 저장소에서 추가한 SKILL.md를 읽을 수 있습니다. Grok Bot은 채팅에서 스킬의 GitHub 폴더 링크를 보내면 설치합니다.