- name
SKILL.md
技能檔案保留在發布者的儲存庫中。
在終端機中貼上複製的指令。它會將這個 SKILL.md 安裝到對應的代理中。
npx skills add agentsope/SkillAlchemy --skill agentsop-dspy -a cursor -y在終端機中貼上複製的指令。它會將這個 SKILL.md 安裝到對應的代理中。
npx skills add agentsope/SkillAlchemy --skill agentsop-dspy -a claude-code -y在終端機中貼上複製的指令。它會將這個 SKILL.md 安裝到對應的代理中。
npx skills add agentsope/SkillAlchemy --skill agentsop-dspy -a codex -y在終端機中貼上複製的指令。它會將這個 SKILL.md 安裝到對應的代理中。
npx skills add agentsope/SkillAlchemy --skill agentsop-dspy -a grok -y在 Grok Bot 聊天中傳送複製的訊息。Bot 會從 GitHub 取得這個 SKILL.md,並加入你所有 Bot 共用的技能中。
Install this skill: https://github.com/agentsope/SkillAlchemy/tree/HEAD/skills/agentsop-dspy來自發布者
"DSPy isn't a prompt-optimization agent framework. It's the LLM compiler for the shortest, cleanest code." — Eito Miyamura [eito.substack.com/p/dspy-the-most-misunderstood-agent]
"Prompts are effectively the weights of an LLM application." — Core philosophy [arxiv.org/abs/2310.03714]
Activate this skill when any of the following triggers are present in the user's intent or codebase:
Do NOT activate when:
client.messages.create.DSPy's full name is Declarative Self-improving Python. The three primitives form a PyTorch-like compile chain [arxiv.org/abs/2310.03714]:
┌─────────────┐ ┌──────────┐ ┌──────────────┐ ┌─────────┐
│ Signature │ → │ Module │ → │ Teleprompter │ → │ Compile │
│ (what) │ │ (how) │ │ (optimizer) │ │ (tune) │
└─────────────┘ └──────────┘ └──────────────┘ └─────────┘
I/O spec Predict/CoT/ MIPROv2/GEPA/ Bake demos
field names ReAct/PoT BootstrapFewShot + instructions
= semantic = strategy = search algorithm into JSON
Three mental shifts the agent must internalize:
Prompts are weights. The prompt string is not the artifact you ship — the compiled program (a JSON of demonstrations + instructions + structural choices) is. You ship program.json, not a .txt prompt [dspy.ai/tutorials/saving/].
Signatures carry semantic load. question -> answer is not the same as query -> response. DSPy uses the field names as the only natural-language hint the optimizer has about intent before it sees data. Name them like you'd name function parameters in well-written code [dspy.ai/learn/programming/signatures/].
Compile is a hyperparameter search, not a one-shot call. Compilation runs hundreds-to-thousands of LM calls (typically $2–$3 USD, 6–20 minutes, 3.2k API calls in the reference run). Costs scale with num_trials × |trainset| × |program LM calls| [dspy.ai/faqs/].
The PyTorch analogy is load-bearing. Signatures ≈ nn.Module.forward() shape contract. Modules ≈ nn.Linear / nn.Transformer. Teleprompters ≈ torch.optim.Adam. compile() ≈ training loop. save()/load() ≈ checkpoint.
The DSPy team is explicit about a three-stage gate [dspy.ai/learn/]:
"It's unproductive to launch optimization runs using a poorly designed program or a bad metric."
Do not skip stages. Each stage has an exit criterion.
"question -> answer"); upgrade to a class-based dspy.Signature with InputField(desc=...) / OutputField(desc=...) when types matter or fields need disambiguation.dspy.ChainOfThought. Use dspy.Predict for trivial classification, dspy.ReAct only when tools are needed, dspy.ProgramOfThought for arithmetic-heavy tasks [dspy.ai/learn/programming/modules/].dspy.Module, instantiate sub-modules in __init__, call them in forward(). No special DSL.dspy.inspect_history(n=3).Exit criterion: the un-optimized program produces plausible outputs on 5+ examples. Not great — plausible.
def metric(example, pred, trace=None) -> float|bool. Start with exact-match; only escalate to LLM-as-judge when the task demands it (open-ended generation, multi-criteria).dspy.Evaluate(devset=dev, metric=metric, num_threads=16) and record a baseline score.Exit criterion: baseline score is stable across two runs (cache-free) AND the metric agrees with human judgment on 10 spot-checks.
auto="light". Only escalate to "medium"/"heavy" if dev-set gains flatten and budget allows.compiled.save("v1.json") for state, or compiled.save("./v1/", save_program=True) for whole-program (preferred for production with metadata) [dspy.ai/tutorials/saving/].dspy.asyncify) or MLflow (mlflow.dspy.log_model) [dspy.ai/tutorials/deployment/].Exit criterion: compiled program beats baseline on a held-out test set (not the val set used in optimization) by ≥ task-relevant delta.
Loop to Stage 1 if optimization plateaus. Per the docs: "Is your task well-defined? Do you need more data? Should your evaluation metric change?" — these are the questions to re-ask, not "should I try a different optimizer?" [dspy.ai/learn/optimization/overview/].
困境 (Dilemma): User has a 3-stage RAG pipeline. Hand-tuned prompts already hit 72% on dev. MIPROv2 auto="heavy" would cost ~$40 and 4 hours. Worth it?
约束 (Constraints):
决策步骤 (Decision steps):
auto="light" (~$2) typically yields 10–30%+ on hand-tuned baselines per the paper's GPT-3.5/Llama2 results (25%/65% lift over standard few-shot) [arxiv.org/abs/2310.03714].auto="light" first as a cheap signal. The docs explicitly recommend "start with moderate values, observe behavior, and scale up only if you see clear gains" [github.com/stanfordnlp/dspy issue #1596].light gives <2% lift, do not escalate to heavy. Instead, revisit Stage 1: is the signature ambiguous? Is the program structure (3 stages) actually right?light gives 5–10% lift, run medium. Only escalate to heavy if data ≥ 300 and you have a held-out test set distinct from val.结果 (Outcome): Typical: light exposes whether more compute helps. Often the answer is "no — fix the program/metric first."
可提取的操作 (Extractable operation): Never start compilation at auto="heavy". Always probe with light and use a cheap optimizer LM.
困境: Compiled program for GPT-4o works at 85%. Need to switch to Llama-3-8B for cost. Re-use the GPT-4o-compiled program.json or recompile?
约束:
决策步骤:
BootstrapFinetune as a follow-on: optimize prompts on the big model, then distill into a 1B–7B student. Typical setup: student=Llama-3.2-1B-Instruct, teacher=gpt-4o-mini [dspy.ai/api/optimizers/BootstrapFinetune/].program.gpt4o.json and program.llama8b.json checked in; A/B in production.结果: Recompiled programs typically recover 70–90% of the larger-model performance at 1/10–1/50 the per-call cost. The "transfer without recompile" path is reliably worse.
可提取的操作: Treat the compiled program as a (program × LM) pair. Changing the LM invalidates the artifact — recompile.
困境: Open-ended customer-support response task. No exact-match metric possible. LLM-as-judge "feels right" but the team worries the judge will be biased toward verbose, hedged outputs.
约束:
决策步骤:
dspy.Predict(Assess) call with a single yes/no question (factual? on-topic? concise? non-hedging?). Documented pattern [dspy.ai/learn/evaluation/metrics/].trace is not None to return bool during compile, float during eval — same metric function, two modes. Avoids the optimizer overfitting to score noise.dspy.GEPA instead of MIPROv2 — GEPA leverages text feedback for faster, more sample-efficient convergence [dspy.ai/api/optimizers/GEPA/overview/, arxiv.org/abs/2507.19457].结果: Multi-dimension metric with explicit length penalty + GEPA's textual feedback typically beats single-judge + MIPROv2 by 10–13% on AIME-style benchmarks [arxiv.org/abs/2507.19457] and is the empirically robust path.
可提取的操作: Never compile against a metric you haven't human-validated on ≥ 20 spot-checks. Decompose multi-criteria metrics. Prefer GEPA when you can express textual feedback.
困境: MIPROv2 compile stuck mid-trial (no progress logs for 30 min). Reported pattern in issue #1970 [github.com/stanfordnlp/dspy/issues/1970]. Abort and restart, or wait?
约束:
决策步骤:
dspy.inspect_history(n=3) — does the last LM call show truncation or rate-limit error?max_bootstrapped_demos and max_labeled_demos (default 4 each); the docs explicitly cite this as the #1 context-length fix [dspy.ai/faqs/].num_threads in the underlying Evaluate; add retry/backoff in the LM client.minibatch_size (default 35; try 16) and smaller num_trials. Hanging is a known failure mode without graceful resume.student retains best demo candidates — check compiled._predictors state.可提取的操作: Compile is not atomic. Treat long hangs as failure. The cost of restart < cost of indefinite wait.
auto="heavy". Always probe with light first [Case A].cache=False in Lambda / stateless deploys. Caches default to a writable dir and break in serverless [dspy.ai/faqs/].dspy.streamify from 2.6.0+ but it's newer than the rest of the stack — verify your version [dspy.ai/tutorials/deployment/].DSPy is not the same layer as LangChain / LlamaIndex / LangGraph. It sits underneath them as a compiler for the individual LM calls inside those orchestration layers [langwatch.ai/blog/best-ai-agent-frameworks-in-2025-...].
┌──────────────────────────────────────────────┐
│ Orchestration: LangGraph, CrewAI │ ← graphs, agents, state
├──────────────────────────────────────────────┤
│ Retrieval: LlamaIndex │ ← ingestion, indexing
├──────────────────────────────────────────────┤
│ Compiler: DSPy │ ← signatures, modules, compile
├──────────────────────────────────────────────┤
│ Generation: Guidance, LMQL, Outlines │ ← single-call grammar control
├──────────────────────────────────────────────┤
│ Inference: vLLM, llama.cpp, Anthropic │ ← serving
└──────────────────────────────────────────────┘
Predict(context, question -> answer) synthesizes. JetBlue's chatbot uses exactly this split — retrieval quality + answer quality as separate metrics, DSPy optimizes both [databricks.com/blog/optimizing-databricks-llm-pipelines-dspy].OutputField already pushes the LM toward structure but doesn't guarantee grammar conformance [dspy.ai/faqs/].import dspy
# 1. Signature
class BasicQA(dspy.Signature):
"""Answer questions with short factoid answers."""
question: str = dspy.InputField()
answer: str = dspy.OutputField(desc="often between 1 and 5 words")
# 2. Module
qa = dspy.ChainOfThought(BasicQA)
# 3. Metric
def metric(ex, pred, trace=None):
return ex.answer.lower() in pred.answer.lower()
# 4. Compile
from dspy.teleprompt import MIPROv2
optimizer = MIPROv2(metric=metric, auto="light")
compiled = optimizer.compile(qa, trainset=trainset)
# 5. Save / load
compiled.save("v1.json")
Have a metric? ─── No ──► Stop. Build a metric first. (Or skip DSPy.)
│
Yes
│
Have ≥ 30 examples? ─── No ──► Stop. Collect more data, or use LabeledFewShot(k=8) as floor.
│
Yes
│
Have textual error feedback? ─── Yes ──► dspy.GEPA
│
No
│
≤ 10 examples? ──► BootstrapFewShot
30–50? ──► BootstrapFewShotWithRandomSearch
50–200? ──► MIPROv2(auto="light", max_bootstrapped_demos=4)
200+? ──► MIPROv2(auto="light" → "medium" if gains; "heavy" only if 300+ and budget)
Need to ship small model? ──► chain BootstrapFinetune after MIPROv2
program.jsonAfter compiled.save("v1.json"), the file is plain JSON. Per-predictor it contains [dspy.ai/tutorials/saving/]:
{
"predictor_name": {
"signature_instructions": "Given the context, answer the question with a short factoid...",
"signature_prefix": "Answer:",
"extended_signature_instructions": "...",
"demos": [
{"question": "...", "reasoning": "...", "answer": "..."},
...
],
"signature": {
"instructions": "...",
"fields": [{"prefix": "Question:", "description": "..."}, ...]
}
}
}
What changes when you compile:
What does NOT change between LMs (so you can read across artifacts):
What DOES change between LMs (so you can't reuse):
dspy.Assert vs dspy.SuggestFor self-refining pipelines [dspy.ai/learn/programming/7-assertions/, arxiv.org/pdf/2312.13382]:
# Hard: halts after max retries with dspy.AssertionError
dspy.Assert(len(pred.answer) < 100, "Answer must be < 100 chars")
# Soft: retries with feedback in prompt, logs failure, continues
dspy.Suggest(is_valid_json(pred.output), "Output must be valid JSON")
When a constraint fails, DSPy backtracks to the previous module and re-runs with the error message injected into the prompt. This is self-refinement at inference time — distinct from compile-time optimization.
Use Assert during development (catch bugs hard). Use Suggest in production (degrade gracefully).
|
---name: agentsop-dspydescription: |---# skills/agentsop-dspy/SKILL.md常見問題
從發布者下載 skills/agentsop-dspy/SKILL.md,或複製路徑後開啟儲存庫。PlugMCP 只收錄這項技能,不代管檔案。
發布者將它存放在 https://github.com/agentsope/SkillAlchemy。
Claude、Cursor、Codex 和 Grok Build 可以讀取你從發布者儲存庫加入的 SKILL.md。在聊天中傳送技能的 GitHub 資料夾連結,Grok Bot 就會安裝它。