- name
SKILL.md
스킬 파일은 게시자의 저장소에 그대로 있습니다.
복사한 명령을 터미널에 붙여넣습니다. 이 SKILL.md가 해당 에이전트에 설치됩니다.
npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation-reporting -a cursor -y복사한 명령을 터미널에 붙여넣습니다. 이 SKILL.md가 해당 에이전트에 설치됩니다.
npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation-reporting -a claude-code -y복사한 명령을 터미널에 붙여넣습니다. 이 SKILL.md가 해당 에이전트에 설치됩니다.
npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation-reporting -a codex -y복사한 명령을 터미널에 붙여넣습니다. 이 SKILL.md가 해당 에이전트에 설치됩니다.
npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation-reporting -a grok -y복사한 메시지를 Grok Bot 채팅에 보냅니다. Bot이 GitHub에서 이 SKILL.md를 가져와 모든 Bot이 공유하는 스킬에 추가합니다.
Install this skill: https://github.com/sickn33/agentic-awesome-skills/tree/HEAD/plugins/agentic-awesome-skills-claude/skills/agent-evaluation-reporting게시자 제공
Turn raw agent evaluation runs into a decision-ready report without hiding failures or overstating capability. Keep outcome populations, denominators, latency populations, and experiment conditions explicit so readers can reproduce every headline number.
Record the task set and sampling, model and provider, prompt or policy version, tool and harness versions, evaluator rubric, timeout and retry policy, token or cost budget, environment, and human-intervention policy. Assign the configuration a stable label or digest.
If a material condition differs between runs, mark the comparison as non-equivalent. Report a directional observation only; do not claim that the changed agent caused the difference.
Classify every scheduled attempt exactly once:
Preserve attempt ID, task ID or seed, retry index, parent attempt ID, configuration label, outcome, intervention count, duration, cost, evaluator evidence, and invalid reason when available. Never silently drop invalid or retried runs.
Also build a unique-task rollup. For each task, retain its first-attempt outcome and derive one eventual outcome after the predeclared retry policy finishes. An execution attempt may contribute once to attempt-level metrics, but a task may contribute only once to task-level completion metrics. If retry lineage or the retry policy is missing, do not report eventual task completion.
Let N_all be all execution attempts, including retries, and N_eval = N_all - N_invalid be evaluable attempts. Let T_all be unique scheduled tasks and T_eval be tasks with a valid task-level outcome under the fixed retry policy. Report counts beside every rate.
autonomous attempt success = N_autonomous / N_eval
assisted attempt success = N_assisted / N_eval
attempt non-completion = (N_failure + N_timeout) / N_eval
invalid-attempt rate = N_invalid / N_all
first-attempt completion = T_first_attempt_completed / T_all
eventual task completion = T_eventual_completed / T_eval
operational task delivery = T_eventual_completed / T_all
Label attempt-level and unique-task metrics explicitly; never call an attempt-level rate workflow completion. Report the retry rate and attempts per task so policy-dependent gains remain visible. Check that evaluable attempt outcomes sum to N_eval, all attempt outcomes sum to N_all, and the task rollup sums to T_all.
If N_eval == 0, report every attempt capability rate as unavailable rather than dividing by zero, and mark any gate that depends on those rates inconclusive. Apply the same rule to any metric whose denominator is zero, including task-level rates when T_all == 0 or T_eval == 0.
Report autonomous-completion latency, assisted end-to-end latency, and failure time-to-terminal separately. A success-only P50 is not an overall P50, and subgroup medians cannot be averaged or weighted to reconstruct a combined median.
Calculate an all-run percentile only from per-run observations and state how timeouts are handled. If durations are right-censored, report the censoring policy or use an appropriate survival estimate. Apply the same population labels to token and cost metrics.
For stochastic evaluations, show sample size and an interval or repeated-run distribution beside headline rates. For comparisons, report the absolute delta and verify that both sides share the frozen contract from Step 1. If data is missing, conditions differ, or intervals are too wide, use inconclusive rather than choosing a winner.
Define readiness gates before reading the result, such as minimum autonomous success, maximum timeout rate, zero critical safety violations, and latency or cost bounds. Return pass, fail, or inconclusive for each gate.
Do not infer production readiness from a success rate alone. When no thresholds or risk requirements were supplied, state that readiness is not determined and list the missing gates.
For 120 unique tasks with one attempt each, including 12 infrastructure-invalid runs, 48 autonomous successes, 24 assisted successes, 20 failures, and 16 timeouts:
Evaluable attempts: 108 / 120
Autonomous success: 48 / 108 = 44.4%
Assisted success: 24 / 108 = 22.2%
Attempt non-completion: 36 / 108 = 33.3%
First-attempt completion: 72 / 120 = 60.0%
Eventual task completion: 72 / 108 = 66.7% (no retries)
Operational task delivery: 72 / 120 = 60.0%
Infrastructure-invalid: 12 / 120 = 10.0%
Overall latency P50: unavailable from subgroup aggregates
Readiness: inconclusive until gates are declared
N_all before calculating metrics.inconclusive.@agent-evaluation - Design behavioral tests, benchmarks, and reliability evaluations.@run-deep-swe - Execute reproducible DeepSWE benchmark runs before reporting their results.Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
---name: agent-evaluation-reportingdescription: Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must…---# plugins/agentic-awesome-skills-claude/skills/agent-evaluation-reporting/SKILL.mdFAQ
게시자에게서 plugins/agentic-awesome-skills-claude/skills/agent-evaluation-reporting/SKILL.md를 다운로드하거나 경로를 복사해 저장소를 엽니다. PlugMCP는 스킬을 목록에 올릴 뿐 파일을 호스팅하지 않습니다.
게시자가 https://github.com/sickn33/agentic-awesome-skills에서 관리합니다.
Claude, Cursor, Codex, Grok Build는 게시자의 저장소에서 추가한 SKILL.md를 읽을 수 있습니다. Grok Bot은 채팅에서 스킬의 GitHub 폴더 링크를 보내면 설치합니다.