- name
SKILL.md
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add glebis/claude-skills --skill annotate -a cursor -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add glebis/claude-skills --skill annotate -a claude-code -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add glebis/claude-skills --skill annotate -a codex -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add glebis/claude-skills --skill annotate -a grok -ySend the copied message in a Grok Bot chat. The Bot fetches this SKILL.md from GitHub and adds it to the skills all your Bots share.
Install this skill: https://github.com/glebis/claude-skills/tree/HEAD/confide/skills/annotateFrom the publisher
Humans label PII spans in a transcript; you measure how much they agree (κ) and draft an adjudicated gold from their labels. Annotators are first-class here — most of this skill is plain instructions FOR a person doing the labelling, plus a coordinator path to score it.
confide:anon) and annotate the GREEN copy.assets/annotator.html — zero-install browser annotation tool (EN/RU, runs offline).references/codebook.md — the labelling rulebook (10 PII types, direct/quasi, harm).references/tool-guide.md — how to drive the tool + scorer step by step.scripts/score_iaa.py — Cohen's/Fleiss' κ, span-F1, disagreement queue, draft gold (stdlib).scripts/gold_to_labels.py — turn an existing gold into a "reference annotator" to test solo.assets/annotator.html (or open it in Chrome/Firefox/
Safari). It runs entirely in your browser — nothing is uploaded; labels stay on your
machine until you Export.references/codebook.md first. It defines the 10 types
(PERSON, LOCATION, ORG, PHONE, EMAIL, ID, DATE, MEDICATION, AGE, PROFESSION), what counts as
a span (the minimal identifying text), and direct vs. quasi-identifier.A, B, or your name).
Use only synthetic or consented text.QUESTION: on the
span (e.g. QUESTION: gym or city?). These flow straight into the adjudication queue.labels.<doc>.<annotator>.json
(schema: {doc_id, annotator, text, spans:[{start,end,text,type,...}]}). Keep it local and
hand only this file to the coordinator. Two+ people should label the same doc independently
(blind) for a meaningful κ.labels.<doc>.<annotator>.json into one folder, e.g. labels/.python3 skills/annotate/scripts/score_iaa.py --labels-dir labels/ --out-dir results/
It writes (per doc + overall): Cohen's κ (pairwise), Fleiss' κ (3+ annotators),
span-F1, a disagreement queue (*-iaa-disagreements.json: every cluster annotators
don't fully agree on, plus any QUESTION: spans), and a draft adjudicated gold
(*-adjudicated-gold-draft.json: majority span per overlap-cluster, ties/questions marked
needs_review:true). Character-level κ sidesteps tokenization disputes.needs_review cluster. The resulting label set is the published gold; report
post-adjudication κ too. Nothing is ever auto-finalised.Treat an existing gold JSONL as one "reference annotator", label the same doc yourself in
annotator.html as another, then score the pair:
python3 skills/annotate/scripts/gold_to_labels.py --gold GOLD.jsonl --name gold --out-dir labels/
# label the same doc yourself in annotator.html as "me" -> drop labels.<doc>.me.json into labels/
python3 skills/annotate/scripts/score_iaa.py --labels-dir labels/ --out-dir results/
(--sessions-dir DIR lets gold_to_labels.py read transcript text from disk so char offsets
match the gold exactly.)
IAA results (κ, F1) + a disagreement list + a draft adjudicated gold — labels/stats only. Transcript text and original PII stay local; only what's needed to adjudicate is shared.
Build and verify a PII gold set with HUMAN annotators (first-class). Launch the browser annotator, label spans per the codebook, export per-annotator label files, then compute inter-annotator agreemen
---name: annotatedescription: Build and verify a PII gold set with HUMAN annotators (first-class). Launch the browser annotator, label spans…---# confide/skills/annotate/SKILL.mdOverview
Build and verify a PII gold set with HUMAN annotators (first-class). Launch the browser annotator, label spans per the codebook, export per-annotator label files, then compute inter-annotator agreement (Cohen's/Fleiss' kappa) and draft an adjudicated gold. Use when the user says "annotate PII", "label this transcript", "build a gold set", "inter-annotator agreement", "review annotations", "adjudicate labels", or wants to measure/defend a de-identification gold standard. Local-only: synthetic or consented data only; annotators' names and transcript text stay on the machine — only labels/stats are collected, nothing PII is re-shared.
FAQ
Download confide/skills/annotate/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
The publisher keeps it at https://github.com/glebis/claude-skills.
Claude, Cursor, Codex, and Grok Build can read a SKILL.md you add from the publisher's repository. Grok Bot installs it when you send the skill's GitHub folder link in a chat.