- name
SKILL.md
Tệp skill vẫn nằm trong kho mã của nhà phát hành.
Dán lệnh đã sao chép vào terminal. Lệnh sẽ cài đặt tệp SKILL.md này vào agent đó.
npx skills add taishi-i/awesome-japanese-nlp-resources --skill discover -a cursor -yDán lệnh đã sao chép vào terminal. Lệnh sẽ cài đặt tệp SKILL.md này vào agent đó.
npx skills add taishi-i/awesome-japanese-nlp-resources --skill discover -a claude-code -yDán lệnh đã sao chép vào terminal. Lệnh sẽ cài đặt tệp SKILL.md này vào agent đó.
npx skills add taishi-i/awesome-japanese-nlp-resources --skill discover -a codex -yDán lệnh đã sao chép vào terminal. Lệnh sẽ cài đặt tệp SKILL.md này vào agent đó.
npx skills add taishi-i/awesome-japanese-nlp-resources --skill discover -a grok -yGửi tin nhắn đã sao chép trong cuộc trò chuyện Grok Bot. Bot sẽ lấy tệp SKILL.md này từ GitHub và thêm vào bộ skill mà mọi Bot của bạn dùng chung.
Install this skill: https://github.com/taishi-i/awesome-japanese-nlp-resources/tree/HEAD/plugins/awesome-japanese-nlp-resources/skills/discoverTừ nhà phát hành
Discover Japanese NLP resources related to the user's query — both what's already in awesome-japanese-nlp-resources and what's not yet listed.
This skill is shared by the Claude Code and Codex versions of the plugin. The steps are the same in both tools; only these details differ:
/awesome-japanese-nlp-resources:discover, appended at the end of this skill as ARGUMENTS: …. Codex: the user's message that invoked $awesome-japanese-nlp-resources:discover, minus that $… mention. If the skill was picked automatically rather than invoked by name, use the user's request as the query.${CLAUDE_PLUGIN_ROOT}. Codex: the directory two levels above this SKILL.md (use its absolute path).Bash tool or Codex's shell tool. Copy each Python script in full and run it as written, changing only its placeholders (RESOURCES_PATH, SEED, the keyword list, the candidate list) — don't shorten it, drop passes, or alter its scores and thresholds.WebSearch to search and WebFetch to read a page. Codex: the built-in web search tool (search, then open the page). Do not use the gh CLI, curl, or other network commands from the shell./awesome-japanese-nlp-resources:<skill> in Claude Code, $awesome-japanese-nlp-resources:<skill> in Codex.The "Already in the list" results and the "not yet in the list" check both come from the bundled data. If the data file can't be read (for example, shell commands are blocked or fail to start), say so and link https://github.com/taishi-i/awesome-japanese-nlp-resources instead of guessing which resources are listed.
Before anything else, run this once and remember the values — every step that mentions a year refers to them:
echo "YEAR_NOW=$(date +%Y)"
echo "YEAR_PREV=$(($(date +%Y) - 1))"
Substitute these wherever this skill writes ${YEAR_NOW} or ${YEAR_PREV}. Do not hardcode years.
If the query is empty or blank, treat it as a topic-mode request for a general search for the latest Japanese NLP resources. Use these defaults for the rest of the steps:
llm, bert, embed, speech, morpholog — short stems for local substring matching (a phrase like japanese language processing almost never occurs verbatim in a description and would silently match nothing; see Step 3's substring-matching note)${YEAR_PREV} ${YEAR_NOW} to every query, and include:
japanese NLP new library github ${YEAR_NOW}日本語 NLP 新しい ライブラリ github ${YEAR_NOW}japanese NLP new model huggingface ${YEAR_NOW}huggingface japanese nlp ${YEAR_PREV} ${YEAR_NOW} newDiscover: "<query>"Then skip Step 1 (mode is topic mode) and continue from Step 2.
Seed mode — the query names ONE specific existing repository, model, or tool:
(a) A direct identifier — a full GitHub URL (https://github.com/owner/repo), a full Hugging Face URL (https://huggingface.co/owner/name or https://huggingface.co/datasets/owner/name), an owner/repo pair, or a bare tool/model name (e.g. mecab). Pass it straight through as SEED.
(b) A natural-language phrase wrapping an identifier (e.g. mecabに似たツール, alternatives to fugashi) — extract the core identifier (the URL, owner/repo, or tool/model name) and pass only that as SEED.
If (a) or (b) applies, proceed to Step 2 in seed mode.
Topic mode — the query is a descriptive/topical phrase with no single specific name in it (e.g. 日本語の要約データセット, japanese sentiment analysis dataset, 形態素解析). There is no identifier to extract and no reason to force one — proceed to Step 2 in topic mode directly. Do not invent a seed; topic mode already covers this case via keyword search plus web discovery.
Keep the full query for the language-detection rule in Step 9.
The data ships with the plugin at data/resources.json under the plugin root (see "Claude Code and Codex" above). Resolve its absolute path, falling back to a scoped search only if the install is unusual:
PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT}" # Codex: replace with the plugin root, two levels above this SKILL.md
RESOURCES_PATH="$PLUGIN_ROOT/data/resources.json"
[ -f "$RESOURCES_PATH" ] || RESOURCES_PATH="$(find "${CODEX_HOME:-$HOME/.codex}/plugins" "${HOME}/.claude/plugins" -type f -name resources.json 2>/dev/null | grep "awesome-japanese-nlp-resources/" | head -1)"
echo "RESOURCES_PATH=$RESOURCES_PATH"
Write the resulting absolute path into every script below — shell variables may not persist between commands.
The plugin also ships data/multilingual_resources.json (same item format) listing multilingual libraries, models, and datasets (GitHub repositories) that also support Japanese, from docs/multilingual.md. The scripts below load it automatically when it exists; its items have categories like Multilingual (Speech recognition).
Seed mode: locate the seed and score every other item by shared category, shared semantic labels, and shared description tokens (IDF-weighted so rare, discriminative labels count more than ubiquitous ones). Substitute RESOURCES_PATH (Step 2) and SEED (from Step 1):
python3 << 'EOF'
import json, re, math, os
from collections import Counter
RESOURCES_PATH = "RESOURCES_PATH" # from Step 2
SEED_RAW = "SEED" # from Step 1
with open(RESOURCES_PATH) as f:
data = json.load(f)
multilingual_path = os.path.join(os.path.dirname(RESOURCES_PATH), "multilingual_resources.json")
if os.path.exists(multilingual_path):
with open(multilingual_path) as f:
data += json.load(f)
N = len(data)
STOP = {
"the","and","for","with","that","this","from","into","your","you","are","was",
"japanese","nlp","python","library","tool","tools","text","based","using","use",
"used","language","data","model","models","repository","repo","support","simple",
"fast","easy","also","can","via","etc","https","http","github","com","www","org",
"not","but","all","any","other","such","more","most","than","its","each","which",
}
def norm_url(u): return (u or "").lower().rstrip("/")
def subs_of(it): return set(t.strip().lower() for t in (it.get("s") or []))
def family(c): return (c or "").split("(")[0].strip().lower()
def toks(text):
return set(w for w in re.findall(r"[a-z0-9]+", (text or "").lower())
if len(w) >= 4 and w not in STOP)
seed = SEED_RAW.strip().lower().rstrip("/")
gh_m = re.search(r"github\.com/([^/#?]+/[^/#?]+)", seed)
hf_m = re.search(r"huggingface\.co/(?:datasets/)?([^/#?]+/[^/#?]+)", seed)
slug = (gh_m or hf_m).group(1) if (gh_m or hf_m) else (seed if seed.count("/") == 1 else None)
basename = seed.split("/")[-1]
def find_matches():
exact = [x for x in data if norm_url(x["u"]) == seed]
if exact: return exact, "exact URL"
if slug:
sm = [x for x in data if norm_url(x["u"]).endswith("/" + slug)]
if sm: return sm, "owner/repo"
return [], None
nm = [x for x in data if x["n"].lower() == basename]
if nm: return nm, "name"
loose = [x for x in data if basename and (basename in x["n"].lower() or ("/" + basename) in norm_url(x["u"]))]
if loose: return loose, "loose substring"
return [], None
matches, how = find_matches()
matches.sort(key=lambda x: -(max(x.get("ns") or 0, x.get("nd") or 0)))
seed_item = matches[0] if matches else None
if not seed_item:
print("SEED_NOT_FOUND")
near = [x for x in data if basename and basename[:4] and basename[:4] in x["n"].lower()]
near.sort(key=lambda x: -(max(x.get("ns") or 0, x.get("nd") or 0)))
for x in near[:8]:
print(f" near? {x['n']} | {x['c']} | {x['u']}")
raise SystemExit
print(f"SEED_FOUND via {how}: {seed_item['n']}")
print(f" url={seed_item['u']}")
print(f" c={seed_item['c']}")
print(f" s={seed_item.get('s','')}")
print(f" d={seed_item.get('d','')[:200]}")
if len(matches) > 1:
print(f" NOTE: {len(matches)} entries matched '{seed}' — chose the most popular. Others:")
for x in matches[1:6]:
print(f" - {x['n']} ({x['u']})")
print()
sub_df, tok_df = Counter(), Counter()
for x in data:
for s in subs_of(x): sub_df[s] += 1
for t in toks((x.get("d") or "") + " " + x["n"] + " " + " ".join(x.get("s") or [])): tok_df[t] += 1
SUB_FLOOR, TOK_FLOOR = 1.5, 1.5
def idf_sub(l): return max(0.0, math.log(N / sub_df.get(l, 1)) - SUB_FLOOR)
def idf_tok(t): return max(0.0, math.log(N / tok_df.get(t, 1)) - TOK_FLOOR)
SUB_W, TOK_W, THRESHOLD = 3.0, 1.5, 8.0
seed_cat = seed_item["c"]; seed_fam = family(seed_cat)
seed_subs = subs_of(seed_item)
seed_tok = toks((seed_item.get("d") or "") + " " + seed_item["n"] + " " + " ".join(seed_item.get("s") or []))
seed_name = seed_item["n"].lower()
seed_urls = {norm_url(seed_item["u"])} | {norm_url(x["u"]) for x in matches if x["n"].lower() == seed_name}
results = []
for x in data:
if x.get("status") == "not_found":
continue
if norm_url(x["u"]) in seed_urls:
continue
score = 0.0
if x["c"] == seed_cat: score += 10
elif family(x["c"]) == seed_fam: score += 5
sh_subs = seed_subs & subs_of(x)
score += SUB_W * sum(idf_sub(l) for l in sh_subs)
sh_tok = seed_tok & toks((x.get("d") or "") + " " + x["n"] + " " + " ".join(x.get("s") or []))
score += TOK_W * sum(idf_tok(t) for t in sh_tok)
if score < THRESHOLD:
continue
pop = max(x.get("ns") or 0, x.get("nd") or 0)
results.append((score + 0.5 * pop, score, sorted(sh_subs), sorted(sh_tok), x))
results.sort(key=lambda r: -r[0])
print(f"=== LOCAL SIMILAR ({len(results)} candidates; showing top 15) ===")
for combined, sc, sh_subs, sh_tok, x in results[:15]:
st = x.get("st", 0) or 0; dl = x.get("dl", 0) or 0
print(f"score={combined:.1f}")
print(f" n={x['n']}")
print(f" u={x['u']}")
print(f" c={x['c']}")
print(f" st={st} dl={dl}")
print(f" shared_labels={', '.join(sh_subs) if sh_subs else '-'}")
print(f" shared_terms={', '.join(sh_tok[:6]) if sh_tok else '-'}")
print(f" d={x.get('d','')[:120]}")
print()
EOF
Topic mode: translate the topic to 3–5 English keywords (same stem + tool-name conventions as the search skill — see its domain table if unsure), then score by keyword match instead of seed fingerprint. Matching is by literal substring, so prefer short stems (llm, embed, morpholog) over multi-word phrases, which rarely occur verbatim in a description:
python3 << 'EOF'
import json, os
with open("RESOURCES_PATH") as f: # from Step 2
data = json.load(f)
multilingual_path = os.path.join(os.path.dirname("RESOURCES_PATH"), "multilingual_resources.json")
if os.path.exists(multilingual_path):
with open(multilingual_path) as f:
data += json.load(f)
keywords = ["keyword1", "keyword2", "keyword3"] # from Step 1 (topic mode)
results = []
for item in data:
if item.get("status") == "not_found":
continue
n = item.get("n", "").lower()
d = item.get("d", "").lower()
s = " ".join(item.get("s") or []).lower()
c = item.get("c", "").lower()
al = " ".join(item.get("al") or []).lower()
text_score = 0
for kw in keywords:
kw = kw.lower()
if n == kw: text_score += 20
elif kw in n: text_score += 10
if kw in d: text_score += 5
if kw in s: text_score += 3
if kw in c: text_score += 2
if kw in al: text_score += 10
if text_score < 8:
continue
ns = item.get("ns") or 0
nd = item.get("nd") or 0
results.append((text_score + max(ns, nd) * 2.5, item))
results.sort(key=lambda x: -x[0])
print(f"=== LOCAL MATCHES ({len(results)} candidates; showing top 15) ===")
for combined, item in results[:15]:
st = item.get("st", 0) or 0; dl = item.get("dl", 0) or 0
print(f"score={combined:.1f} st={st} dl={dl}")
print(f" n={item['n']}")
print(f" u={item['u']}")
print(f" c={item['c']}")
print(f" s={item.get('s','')}")
print(f" d={item.get('d','')[:120]}")
print()
EOF
This prints the resolved seed (seed mode) or keyword matches (topic mode), plus up to 15 local candidates. Use the shared labels/terms (seed mode) or matched keywords (topic mode) to justify the "Why" column later.
If Step 3 printed SEED_NOT_FOUND:
owner/repo, fetch its page (Claude Code: WebFetch as below; Codex: open it with the web search tool and extract the same fields):
WebFetch url="https://github.com/<owner>/<repo>" prompt="Extract as JSON: name, one-line description, primary language, the NLP task it performs, star count, and whether it targets Japanese. If a field is unavailable, set it to null."
If it looks like a Hugging Face URL, fetch that page instead:
WebFetch url="https://huggingface.co/<owner>/<name>" prompt="Extract as JSON: name, one-line description / model card summary, pipeline or task tag, whether it is a model or a dataset, downloads count, likes count, and whether it targets Japanese. If a field is unavailable, set it to null."
If it's a bare name with no domain hint, run one web search (<name> japanese nlp) to identify what it does and where it lives.If the seed cannot be found locally or resolved on the web, this is the only case where you stop and report back: state what you tried, and suggest running the search skill with the same query.
Seed mode — using the seed's task and category, build 6–8 queries mixing GitHub and Hugging Face, English and Japanese:
GitHub-oriented: <seed-name> alternative github, <seed-name> vs, japanese <task> library github, 日本語 <task> ライブラリ github, <seed-name> 類似 ツール
Hugging Face-oriented: <seed-name> alternative huggingface, japanese <task> huggingface, huggingface japanese <task>
Topic mode — reuse the same keywords derived in Step 3 (the search skill's domain-table conventions), build 6–9 queries. Unlike Step 3's local substring matching, these are search-engine queries, so natural multi-word phrasing (e.g. japanese sentence embedding) works fine here even where it wouldn't as a literal substring:
GitHub-oriented: <keyword> japanese site:github.com, japanese <keyword> ${YEAR_NOW} site:github.com, <topic> 日本語 github, awesome japanese <keyword>
Hugging Face-oriented (always include at least 2–3): <keyword> japanese site:huggingface.co, huggingface japanese <keyword> ${YEAR_NOW}, <topic> 日本語 huggingface
Both modes: add ${YEAR_NOW} recency variants when the topic is fast-moving (LLMs, embeddings, speech).
Use the web search and page-fetch tools only (see "Claude Code and Codex" above) — do not use the gh CLI in this project.
Run the queries from Step 5 (6–9 of them). From each result, extract every URL matching:
https://github.com/<owner>/<repo> (ignore deeper paths like /issues, /pull/, /blob/, /tree/)https://huggingface.co/<owner>/<name> for models, https://huggingface.co/datasets/<owner>/<name> for datasets (ignore deeper paths)Collect them, lowercased, trailing slashes stripped, tagged by kind (github / hf_model / hf_dataset), de-duplicated by owner/repo (or owner/name) within each kind, and drop the seed itself (seed mode).
Keep only resources NOT already in the dataset — this section reports unlisted items only; anything already catalogued belongs in the "Already in the list" table from Step 3. Check all collected candidates in one run (both modes), substituting RESOURCES_PATH (Step 2) and the candidate URLs. Besides resources.json, the script also reads the multilingual list, a pre-built data/existing_urls.txt when the plugin ships one, and the list's own README.md / docs/huggingface.md when run from a clone of the repository — resources.json can lag behind the live README. It reads files only, so it also works in read-only sandboxes:
python3 << 'EOF'
import json, re, os
RESOURCES_PATH = "RESOURCES_PATH" # from Step 2
CANDIDATES = """
https://github.com/owner/repo
https://huggingface.co/owner/name
""".split() # every candidate URL collected above, one per line
def canon(u):
u = (u or "").strip().lower().rstrip("/")
m = re.search(r"(github\.com|huggingface\.co)/((?:datasets/|spaces/)?[^/#?\s]+/[^/#?\s]+)", u)
if not m:
return u
path = m.group(2)
if path.endswith(".git"):
path = path[:-4]
return f"https://{m.group(1)}/{path}"
data_dir = os.path.dirname(os.path.abspath(RESOURCES_PATH))
existing = set()
for name in ("resources.json", "multilingual_resources.json"):
path = os.path.join(data_dir, name)
if os.path.exists(path):
with open(path) as f:
existing.update(canon(item.get("u")) for item in json.load(f))
scan_files = [os.path.join(data_dir, "existing_urls.txt")]
p = data_dir
for _ in range(6):
p = os.path.dirname(p)
if os.path.exists(os.path.join(p, "awesome-japanese-nlp-resources.json")):
scan_files += [os.path.join(p, "README.md"), os.path.join(p, "docs", "huggingface.md")]
break
url_pattern = re.compile(r"https://(?:github\.com|huggingface\.co)/(?:datasets/)?[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+")
for scan_file in scan_files:
if os.path.exists(scan_file):
with open(scan_file) as f:
existing.update(canon(u) for u in url_pattern.findall(f.read()))
print(f"{len(existing)} URLs already in the list")
for c in CANDIDATES:
print(("in list - drop " if canon(c) in existing else "unlisted - keep ") + c)
EOF
Cap the survivors at 10–15 total (seed mode: 5, spread across both kinds), prioritizing candidates that appear in multiple result sets and keeping a mix of both platforms.
For each surviving candidate, fetch its page — in Claude Code with WebFetch (up to 5 calls in parallel — single message, multiple tool calls); in Codex, open it with the web search tool and extract the same fields:
For github candidates:
WebFetch url="https://github.com/<owner>/<repo>" prompt="Extract as JSON: name, one-line description, primary language, star count, last-updated date (YYYY-MM), whether archived, whether a fork, and what NLP task it performs. Note if it targets Japanese."
For hf_model / hf_dataset candidates:
WebFetch url="https://huggingface.co/<owner>/<name>" prompt="Extract as JSON: name, one-line description / model card summary, pipeline or task tag, whether model or dataset, downloads count, likes count, last-updated date (YYYY-MM), and what NLP task it performs. Note if it targets Japanese."
Drop any candidate that: is archived (GitHub); is an inactive/throwaway fork (keep only if independently maintained and published — e.g. on PyPI/crates.io/npm — with a distinct improvement); clearly does not perform the same/related task; is not Japanese-NLP-related (unless a widely-known direct equivalent); or (topic mode, general-search default) has stars < 3 / likes < 3 AND is >24 months stale. Keep stars/downloads/likes and last-updated for the survivors.
Group survivors under awesome-japanese-nlp-resources section headings, contribution-ready:
Hugging Face candidates go to "Hugging Face model" or "Hugging Face dataset", inferred from the fetched page (hf_model/hf_dataset kind, or whether the URL contains /datasets/).
GitHub candidates are grouped by primary language and description:
Language detection rule (apply before writing any output):
Apply the detected language to all headings and prose. Repository/model descriptions stay in English (the awesome list standard).
English output template:
## Discover: "<query>"
**Seed:** [<seed-name>](<seed-url>) — <category> — <one-line what it does> ← seed mode only
*(Similarity from the bundled dataset + web research across GitHub and Hugging Face)*
### Already in awesome-japanese-nlp-resources
| # | Repository | Category | Popularity | Why |
|---|---|---|---|---|
| 1 | [name](url) | category | ⭐N / 📥N | shares <task/labels> (seed mode) or matched <keywords> (topic mode) |
*(Omit this table, with a one-line note, if Step 3 found nothing.)*
### Not yet in the list — candidates
Found **N** resources not yet in awesome-japanese-nlp-resources (**G** GitHub, **H** Hugging Face).
#### Python library
* [repo-name](https://github.com/owner/repo) - One-line English description. (⭐ 123, last updated: YYYY-MM)
#### Hugging Face model
* [model-name](https://huggingface.co/owner/name) - 📥 {downloads} / ⭐ {likes} / One-line English description.
*(Only the section headings that actually have candidates. Omit the whole block, with a one-line note, if Step 6/7 found nothing unlisted.)*
### Recommendation
- **Closest alternative** (seed mode) or **Highlight** (topic mode): [name](url) — one-line reason.
- **Next step:** `/awesome-japanese-nlp-resources:search <query>` for the full catalog, or `/awesome-japanese-nlp-resources:research <topic>` for a trend/challenge report.
Sources:
- [Title 1](https://...)
Japanese output template (when the query is in Japanese): mirror the structure with ## "<query>" の発見結果, ### awesome-japanese-nlp-resources 内の既存リソース, ### 未収録の候補, ### おすすめ, and per-bullet suffixes (⭐ N, 最終更新: YYYY-MM).
Rules:
* [name](url) - description. (⭐ N, last updated: YYYY-MM); Hugging Face * [name](url) - 📥 {downloads} / ⭐ {likes} / description. (abbreviate with k/M, e.g. 📥 367k).Sources: is mandatory (web research requirement) — list the result URLs actually used.$awesome-japanese-nlp-resources:… in Codex.Given a Japanese NLP GitHub repo/model/dataset (URL / owner/repo / tool name) OR a topic, find what's already in awesome-japanese-nlp-resources and discover related resources NOT yet listed (contribut
---name: discoverdescription: Given a Japanese NLP GitHub repo/model/dataset (URL / owner/repo / tool name) OR a topic, find what's already…---# plugins/awesome-japanese-nlp-resources/skills/discover/SKILL.mdTổng quan
Given a Japanese NLP GitHub repo/model/dataset (URL / owner/repo / tool name) OR a topic, find what's already in awesome-japanese-nlp-resources and discover related resources NOT yet listed (contribution candidates). Mines the bundled dataset, then expands via web research across GitHub and Hugging Face. Use when the user names a SPECIFIC repository, model, or tool and wants alternatives/equivalents, OR wants to discover Japanese NLP resources for a topic that are NOT yet in the list, OR wants to prepare a contribution. Trigger phrases include 'mecabに似たツール', 'fugashiの代替', 'alternatives to fugashi', 'repos like manga-ocr', 'what else is like sudachi', 'リストに無い新しい日本語NLP', 'awesome-japanese-nlpに追加できそうな', '最近公開された日本語NLPツール', 'find unlisted Japanese NLP repos', 'new Japanese models on Hugging Face', 'contribute a new resource'. For a simple lookup of what already exists, use the search skill instead.
Câu hỏi thường gặp
Tải plugins/awesome-japanese-nlp-resources/skills/discover/SKILL.md từ nhà phát hành, hoặc sao chép đường dẫn và mở kho mã. PlugMCP chỉ liệt kê skill và không lưu trữ tệp này.
Nhà phát hành duy trì nó tại https://github.com/taishi-i/awesome-japanese-nlp-resources.
Claude, Cursor, Codex và Grok Build có thể đọc tệp SKILL.md mà bạn thêm từ kho mã của nhà phát hành. Grok Bot cài đặt skill khi bạn gửi liên kết thư mục GitHub của skill trong cuộc trò chuyện.