Processes and analyzes data with resident-kernel engines (DuckDB, Polars) and one-shot tools. Use for CSV/parquet/JSON analysis, group-by/join/aggregation, time series, distributions, cleaning, or plo
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add code-yeongyu/oh-my-openagent --skill data-scientist -a cursor -y
Processes and analyzes data with resident-kernel engines (DuckDB, Polars) and one-shot tools. Use for CSV/parquet/JSON analysis, group-by/join/aggregation, time series, distributions, cleaning, or plotting a dataset.
Processes and analyzes data with resident-kernel engines (DuckDB, Polars) and one-shot tools. Use for CSV/parquet/JSON analysis, group-by/join/aggregation, time series, distributions, cleaning, or plotting a dataset.
Answer data questions through the cheapest engine and surface that can prove the answer, and decide where the computation should live before touching the data.
A persistent REPL/eval kernel (many harnesses expose one for JavaScript and Python) is the default surface. Reason: each one-shot process pays roughly a second of spawn-plus-import overhead and re-scans the input file, while a resident connection amortizes both — after a one-time load, repeat queries return in milliseconds. Exploration is repeat queries, so this difference dominates the session.
scripts/ensure-js-deps.sh once; it prints the absolute
import path for @duckdb/node-api. Dynamic-import it, connect once, query across cells.scripts/ensure-py-deps.sh, which
installs them once into a user cache keyed to the kernel's interpreter —
sys.path.insert the printed directory and import. The interpreter itself is never
mutated.uv run --with ...): isolation for a heavy or crash-prone one-shot that
should not take the kernel down.bun -e for
DuckDB-js, uv run python -c for the Python stack — batching several questions per
process.Per-surface patterns and pitfalls: read references/execution-surfaces.md before first use.
ensure-py-deps.sh.
Read references/polars-lane.md — the current 1.x API differs from widely-memorized
older spellings.references/visualization.md first; it carries the
quality bar and a mandatory visual check.Performance folklore ("X is Nx faster at filtering") varies with data shape, cardinality, and hardware. When the engine choice materially matters, measure on the actual data instead of trusting remembered multipliers.
Probe before you compute — one cell: file size, free RAM, and (when unclear) a row count via a direct scan. Then place the work:
CREATE TABLE t AS SELECT ... (or a collected
DataFrame) once, then iterate. One scan up front converts every later query from a file
re-scan into milliseconds.FROM 'data.csv'); past RAM, cap DuckDB's memory and let it
spill, or use Polars' streaming engine in the Python kernel. NEVER load a larger-than-RAM
dataset fully into memory — swapping stalls the whole machine, while streaming merely
takes longer.Sizing heuristics and recipes: references/placement.md.
.df() on a DuckDB
result raises unless pandas is installed; convert with .pl() via Arrow instead.Answer the question; report row counts and timing for anything heavy; then stop — no bonus
charts, no extra exploration passes beyond what the question needed. Chart when asked, or
when the answer is a shape (trend, distribution, comparison) that prose cannot carry — then
follow references/visualization.md including its visual QA step.
| Read | When |
|---|---|
references/execution-surfaces.md | before the first query on any surface: kernel patterns, one-shot recipes, escalation rules |
references/polars-lane.md | DataFrame-shaped pipeline or data past RAM: current API, Arrow handoff, package sets |
references/placement.md | before heavy or remote work: sizing probe, memory limits, remote reads |
references/visualization.md | before any chart: type selection, quality bar, CJK fonts, visual QA |
references/uv-setup.md | uv missing or broken on this machine |
When no kernel or REPL surface exists, uv run scripts/quick-query.py <file> [SQL]
(--filter <polars-sql-expr>, --describe) answers ad-hoc questions with zero code.
Supports CSV, Parquet, JSON, NDJSON.