- name
SKILL.md
Le fichier du skill reste dans le dépôt de l'éditeur.
Collez la commande copiée dans un terminal. Elle installe ce SKILL.md dans cet agent.
npx skills add agentsope/SkillAlchemy --skill agentsop-dspy -a cursor -yCollez la commande copiée dans un terminal. Elle installe ce SKILL.md dans cet agent.
npx skills add agentsope/SkillAlchemy --skill agentsop-dspy -a claude-code -yCollez la commande copiée dans un terminal. Elle installe ce SKILL.md dans cet agent.
npx skills add agentsope/SkillAlchemy --skill agentsop-dspy -a codex -yCollez la commande copiée dans un terminal. Elle installe ce SKILL.md dans cet agent.
npx skills add agentsope/SkillAlchemy --skill agentsop-dspy -a grok -yEnvoyez le message copié dans un chat Grok Bot. Le Bot récupère ce SKILL.md sur GitHub et l'ajoute aux skills que partagent tous vos Bots.
Install this skill: https://github.com/agentsope/SkillAlchemy/tree/HEAD/skills/agentsop-dspyDe l'éditeur
"DSPy isn't a prompt-optimization agent framework. It's the LLM compiler for the shortest, cleanest code." — Eito Miyamura [eito.substack.com/p/dspy-the-most-misunderstood-agent]
"Prompts are effectively the weights of an LLM application." — Core philosophy [arxiv.org/abs/2310.03714]
Activate this skill when any of the following triggers are present in the user's intent or codebase:
Do NOT activate when:
client.messages.create.DSPy's full name is Declarative Self-improving Python. The three primitives form a PyTorch-like compile chain [arxiv.org/abs/2310.03714]:
┌─────────────┐ ┌──────────┐ ┌──────────────┐ ┌─────────┐
│ Signature │ → │ Module │ → │ Teleprompter │ → │ Compile │
│ (what) │ │ (how) │ │ (optimizer) │ │ (tune) │
└─────────────┘ └──────────┘ └──────────────┘ └─────────┘
I/O spec Predict/CoT/ MIPROv2/GEPA/ Bake demos
field names ReAct/PoT BootstrapFewShot + instructions
= semantic = strategy = search algorithm into JSON
Three mental shifts the agent must internalize:
Prompts are weights. The prompt string is not the artifact you ship — the compiled program (a JSON of demonstrations + instructions + structural choices) is. You ship program.json, not a .txt prompt [dspy.ai/tutorials/saving/].
Signatures carry semantic load. question -> answer is not the same as query -> response. DSPy uses the field names as the only natural-language hint the optimizer has about intent before it sees data. Name them like you'd name function parameters in well-written code [dspy.ai/learn/programming/signatures/].
Compile is a hyperparameter search, not a one-shot call. Compilation runs hundreds-to-thousands of LM calls (typically $2–$3 USD, 6–20 minutes, 3.2k API calls in the reference run). Costs scale with num_trials × |trainset| × |program LM calls| [dspy.ai/faqs/].
The PyTorch analogy is load-bearing. Signatures ≈ nn.Module.forward() shape contract. Modules ≈ nn.Linear / nn.Transformer. Teleprompters ≈ torch.optim.Adam. compile() ≈ training loop. save()/load() ≈ checkpoint.
The DSPy team is explicit about a three-stage gate [dspy.ai/learn/]:
"It's unproductive to launch optimization runs using a poorly designed program or a bad metric."
Do not skip stages. Each stage has an exit criterion.
"question -> answer"); upgrade to a class-based dspy.Signature with InputField(desc=...) / OutputField(desc=...) when types matter or fields need disambiguation.dspy.ChainOfThought. Use dspy.Predict for trivial classification, dspy.ReAct only when tools are needed, dspy.ProgramOfThought for arithmetic-heavy tasks [dspy.ai/learn/programming/modules/].dspy.Module, instantiate sub-modules in __init__, call them in forward(). No special DSL.dspy.inspect_history(n=3).Exit criterion: the un-optimized program produces plausible outputs on 5+ examples. Not great — plausible.
def metric(example, pred, trace=None) -> float|bool. Start with exact-match; only escalate to LLM-as-judge when the task demands it (open-ended generation, multi-criteria).dspy.Evaluate(devset=dev, metric=metric, num_threads=16) and record a baseline score.Exit criterion: baseline score is stable across two runs (cache-free) AND the metric agrees with human judgment on 10 spot-checks.
auto="light". Only escalate to "medium"/"heavy" if dev-set gains flatten and budget allows.compiled.save("v1.json") for state, or compiled.save("./v1/", save_program=True) for whole-program (preferred for production with metadata) [dspy.ai/tutorials/saving/].dspy.asyncify) or MLflow (mlflow.dspy.log_model) [dspy.ai/tutorials/deployment/].Exit criterion: compiled program beats baseline on a held-out test set (not the val set used in optimization) by ≥ task-relevant delta.
Loop to Stage 1 if optimization plateaus. Per the docs: "Is your task well-defined? Do you need more data? Should your evaluation metric change?" — these are the questions to re-ask, not "should I try a different optimizer?" [dspy.ai/learn/optimization/overview/].
困境 (Dilemma): User has a 3-stage RAG pipeline. Hand-tuned prompts already hit 72% on dev. MIPROv2 auto="heavy" would cost ~$40 and 4 hours. Worth it?
约束 (Constraints):
决策步骤 (Decision steps):
auto="light" (~$2) typically yields 10–30%+ on hand-tuned baselines per the paper's GPT-3.5/Llama2 results (25%/65% lift over standard few-shot) [arxiv.org/abs/2310.03714].auto="light" first as a cheap signal. The docs explicitly recommend "start with moderate values, observe behavior, and scale up only if you see clear gains" [github.com/stanfordnlp/dspy issue #1596].light gives <2% lift, do not escalate to heavy. Instead, revisit Stage 1: is the signature ambiguous? Is the program structure (3 stages) actually right?light gives 5–10% lift, run medium. Only escalate to heavy if data ≥ 300 and you have a held-out test set distinct from val.结果 (Outcome): Typical: light exposes whether more compute helps. Often the answer is "no — fix the program/metric first."
可提取的操作 (Extractable operation): Never start compilation at auto="heavy". Always probe with light and use a cheap optimizer LM.
困境: Compiled program for GPT-4o works at 85%. Need to switch to Llama-3-8B for cost. Re-use the GPT-4o-compiled program.json or recompile?
约束:
决策步骤:
BootstrapFinetune as a follow-on: optimize prompts on the big model, then distill into a 1B–7B student. Typical setup: student=Llama-3.2-1B-Instruct, teacher=gpt-4o-mini [dspy.ai/api/optimizers/BootstrapFinetune/].program.gpt4o.json and program.llama8b.json checked in; A/B in production.结果: Recompiled programs typically recover 70–90% of the larger-model performance at 1/10–1/50 the per-call cost. The "transfer without recompile" path is reliably worse.
可提取的操作: Treat the compiled program as a (program × LM) pair. Changing the LM invalidates the artifact — recompile.
困境: Open-ended customer-support response task. No exact-match metric possible. LLM-as-judge "feels right" but the team worries the judge will be biased toward verbose, hedged outputs.
约束:
决策步骤:
dspy.Predict(Assess) call with a single yes/no question (factual? on-topic? concise? non-hedging?). Documented pattern [dspy.ai/learn/evaluation/metrics/].trace is not None to return bool during compile, float during eval — same metric function, two modes. Avoids the optimizer overfitting to score noise.dspy.GEPA instead of MIPROv2 — GEPA leverages text feedback for faster, more sample-efficient convergence [dspy.ai/api/optimizers/GEPA/overview/, arxiv.org/abs/2507.19457].结果: Multi-dimension metric with explicit length penalty + GEPA's textual feedback typically beats single-judge + MIPROv2 by 10–13% on AIME-style benchmarks [arxiv.org/abs/2507.19457] and is the empirically robust path.
可提取的操作: Never compile against a metric you haven't human-validated on ≥ 20 spot-checks. Decompose multi-criteria metrics. Prefer GEPA when you can express textual feedback.
困境: MIPROv2 compile stuck mid-trial (no progress logs for 30 min). Reported pattern in issue #1970 [github.com/stanfordnlp/dspy/issues/1970]. Abort and restart, or wait?
约束:
决策步骤:
dspy.inspect_history(n=3) — does the last LM call show truncation or rate-limit error?max_bootstrapped_demos and max_labeled_demos (default 4 each); the docs explicitly cite this as the #1 context-length fix [dspy.ai/faqs/].num_threads in the underlying Evaluate; add retry/backoff in the LM client.minibatch_size (default 35; try 16) and smaller num_trials. Hanging is a known failure mode without graceful resume.student retains best demo candidates — check compiled._predictors state.可提取的操作: Compile is not atomic. Treat long hangs as failure. The cost of restart < cost of indefinite wait.
auto="heavy". Always probe with light first [Case A].cache=False in Lambda / stateless deploys. Caches default to a writable dir and break in serverless [dspy.ai/faqs/].dspy.streamify from 2.6.0+ but it's newer than the rest of the stack — verify your version [dspy.ai/tutorials/deployment/].DSPy is not the same layer as LangChain / LlamaIndex / LangGraph. It sits underneath them as a compiler for the individual LM calls inside those orchestration layers [langwatch.ai/blog/best-ai-agent-frameworks-in-2025-...].
┌──────────────────────────────────────────────┐
│ Orchestration: LangGraph, CrewAI │ ← graphs, agents, state
├──────────────────────────────────────────────┤
│ Retrieval: LlamaIndex │ ← ingestion, indexing
├──────────────────────────────────────────────┤
│ Compiler: DSPy │ ← signatures, modules, compile
├──────────────────────────────────────────────┤
│ Generation: Guidance, LMQL, Outlines │ ← single-call grammar control
├──────────────────────────────────────────────┤
│ Inference: vLLM, llama.cpp, Anthropic │ ← serving
└──────────────────────────────────────────────┘
Predict(context, question -> answer) synthesizes. JetBlue's chatbot uses exactly this split — retrieval quality + answer quality as separate metrics, DSPy optimizes both [databricks.com/blog/optimizing-databricks-llm-pipelines-dspy].OutputField already pushes the LM toward structure but doesn't guarantee grammar conformance [dspy.ai/faqs/].import dspy
# 1. Signature
class BasicQA(dspy.Signature):
"""Answer questions with short factoid answers."""
question: str = dspy.InputField()
answer: str = dspy.OutputField(desc="often between 1 and 5 words")
# 2. Module
qa = dspy.ChainOfThought(BasicQA)
# 3. Metric
def metric(ex, pred, trace=None):
return ex.answer.lower() in pred.answer.lower()
# 4. Compile
from dspy.teleprompt import MIPROv2
optimizer = MIPROv2(metric=metric, auto="light")
compiled = optimizer.compile(qa, trainset=trainset)
# 5. Save / load
compiled.save("v1.json")
Have a metric? ─── No ──► Stop. Build a metric first. (Or skip DSPy.)
│
Yes
│
Have ≥ 30 examples? ─── No ──► Stop. Collect more data, or use LabeledFewShot(k=8) as floor.
│
Yes
│
Have textual error feedback? ─── Yes ──► dspy.GEPA
│
No
│
≤ 10 examples? ──► BootstrapFewShot
30–50? ──► BootstrapFewShotWithRandomSearch
50–200? ──► MIPROv2(auto="light", max_bootstrapped_demos=4)
200+? ──► MIPROv2(auto="light" → "medium" if gains; "heavy" only if 300+ and budget)
Need to ship small model? ──► chain BootstrapFinetune after MIPROv2
program.jsonAfter compiled.save("v1.json"), the file is plain JSON. Per-predictor it contains [dspy.ai/tutorials/saving/]:
{
"predictor_name": {
"signature_instructions": "Given the context, answer the question with a short factoid...",
"signature_prefix": "Answer:",
"extended_signature_instructions": "...",
"demos": [
{"question": "...", "reasoning": "...", "answer": "..."},
...
],
"signature": {
"instructions": "...",
"fields": [{"prefix": "Question:", "description": "..."}, ...]
}
}
}
What changes when you compile:
What does NOT change between LMs (so you can read across artifacts):
What DOES change between LMs (so you can't reuse):
dspy.Assert vs dspy.SuggestFor self-refining pipelines [dspy.ai/learn/programming/7-assertions/, arxiv.org/pdf/2312.13382]:
# Hard: halts after max retries with dspy.AssertionError
dspy.Assert(len(pred.answer) < 100, "Answer must be < 100 chars")
# Soft: retries with feedback in prompt, logs failure, continues
dspy.Suggest(is_valid_json(pred.output), "Output must be valid JSON")
When a constraint fails, DSPy backtracks to the previous module and re-runs with the error message injected into the prompt. This is self-refinement at inference time — distinct from compile-time optimization.
Use Assert during development (catch bugs hard). Use Suggest in production (degrade gracefully).
|
---name: agentsop-dspydescription: |---# skills/agentsop-dspy/SKILL.mdFAQ
Téléchargez skills/agentsop-dspy/SKILL.md auprès de l'éditeur, ou copiez le chemin et ouvrez le dépôt. PlugMCP répertorie le skill et n'héberge pas le fichier.
L'éditeur le maintient sur https://github.com/agentsope/SkillAlchemy.
Claude, Cursor, Codex et Grok Build peuvent lire un SKILL.md que vous ajoutez depuis le dépôt de l'éditeur. Grok Bot l'installe lorsque vous envoyez dans un chat le lien du dossier GitHub du skill.