Datanika

Datanika

Open-source data pipeline platform — Extract, Load, Transform, and Orchestrate from a single UI.

Datanika combines dlt (extract + load) with dbt-core (transform) and adds visual pipeline management, scheduling, and monitoring — all in one Python-native platform.

Think Airbyte + dbt Cloud + Airflow — in one tool, self-hostable with Docker Compose.

Features

🔌 36 Connectors — PostgreSQL, MySQL, Oracle, MongoDB, BigQuery, Snowflake, Stripe, HubSpot, Salesforce, Kafka, ClickHouse, and more 🔄 dbt Transformations — SQL models, tests, snapshots, packages, and source freshness built in 📊 Visual Pipeline Builder — DAG editor with dependency management ⏰ Scheduling — Cron-based with APScheduler, persistent across restarts 📈 Monitoring — Run history, streaming logs, and dashboard stats 🔐 Enterprise Security — RBAC, SSO (SAML/OIDC), audit logging, encrypted credentials 🌍 9 Languages — English, German, French, Spanish, Russian, Greek, Chinese, Arabic, Serbian 🔌 REST API — Full CRUD with OpenAPI/Swagger docs, rate limiting, and scoped API keys 🤖 AI-Agent Ready — hosted + local MCP server (25 tools), /llms.txt, agent-guide.md, 5-tier capability API, compile+preview validation, typed error codes, ?wait=true, Idempotency-Key, run cancellation 🚀 Pipeline Templates — One-click starter templates (Stripe→Postgres, Postgres→BigQuery, CSV→DuckDB) with prefilled connection configs 🔔 Notifications — Slack, Telegram, email, and webhook alerts on run completion, plus an in-app notification center 📦 Self-Hostable — Single docker compose up — no Kubernetes required

Quick Start

Docker (recommended)

git clone https://github.com/datanika-io/datanika-core.git
cd datanika-core
cp .env.example .env
# Edit .env with your settings
docker compose up -d

App available at http://localhost:3000

Development

uv venv && source .venv/bin/activate
uv pip install -e ".[dev]"
docker compose up -d postgres redis  # infrastructure only
uv run reflex run                     # starts on :3000 + :8000

Releases & versioning

Datanika uses 0.x SemVer with tagged releases (v0.1.0, …). While pre-1.0, breaking changes bump the minor. There is no 1.0 yet — that will mean a committed, stable public API contract.

Self-hosting? Pin a release instead of tracking master. master is continuously deployed to our hosted app and moves several times a day.

git checkout v0.1.0            # source
docker compose up -d --build   # build the image from that source

⚠️ There is no publicly pullable image yet. ghcr.io/datanika-io/datanika-core exists but is private, so docker pull on any tag answers denied for everyone outside the org — this README told you otherwise until 2026-09-03. It stays private on purpose: the image grafts in our closed-source cloud plugin, so publishing it as built today would publish that too. Pin the source tag and build; that path is exercised by CI.

Building it yourself gives you the same artifact we would publish. Since 2026-09-04 the Dockerfile carries a DATANIKA_IMAGE_EDITION build arg, and the core-only variant is built and asserted on every PR (core-only-image in CI) from a context that does not contain the cloud tree at all — which is the context you have.

⚠️ The Dockerfile expects a monorepo-shaped context: it does COPY datanika/pyproject.toml, so the build context is the parent directory and this checkout has to be the datanika/ inside it. Clone it under that name and build from one level up:

git clone https://github.com/datanika-io/datanika-core.git datanika
docker build --build-arg DATANIKA_IMAGE_EDITION=core -f datanika/Dockerfile .

The resulting image carries no /cloud tree and cannot import datanika_cloud. Leave DATANIKA_EDITION unset — its default is core. Setting it to cloud on this image fails immediately and loudly (ModuleNotFoundError: No module named 'datanika_cloud'), which is deliberate: the alternative is a service that looks healthy and enforces nothing. Publishing this under a public GHCR package is tracked in #1014.

Every version's notes are on the Releases page. Security advisories cite the first patched release (e.g. Patched: v0.1.0), so a pinned tag tells you immediately whether you're affected.

The datanika-mcp sub-package is versioned and released independently (mcp-v* tags, published to PyPI).

Why Datanika?

Tech Stack

Roadmap

• 36 connectors (databases, SaaS APIs, files, streaming)
• dbt transformations, tests, snapshots, packages
• REST API v1 with OpenAPI/Swagger and typed per-connector inline schemas
• AI-agent compatibility (/llms.txt, agent-guide, 5-tier API, golden-path loop, ?wait=true, Idempotency-Key, run cancel, MCP server)
• Pipeline templates (one-click setup)
• In-app notification center with Slack, Telegram, Email, Webhook channels
• SSO (SAML/OIDC) for Enterprise
• Usage-based billing (cloud plugin)
• 2,300+ tests across unit, security, and E2E (SQLite in-memory for speed)
• Kubernetes Helm chart — in-tree at deploy/helm/datanika/; installs the same image as the Compose path. Bundled Postgres/Redis are single-replica and not production-grade — point it at managed databases (see the chart README)
• Data lineage visualization

AI Agent Integration

Datanika speaks MCP, so AI agents (Claude Desktop, Claude Code, Cursor, …) can browse connections, preview data, compile transformations, and monitor runs natively. 25 tools — 17 read-only, 8 write. There are two ways in.

Hosted — nothing to install. Paste this wherever your client accepts a remote MCP server and authorize in the browser:

https://app.datanika.io/mcp

OAuth 2.1, no API key to handle. Read-only unless you grant write at consent — a client that asks for nothing gets read-only, and a pasted API key stays read-only here even if its own scopes allow writes.

Local — stdio. Published on PyPI as datanika-mcp and listed on the official MCP registry as io.datanika/datanika-mcp:

# read-only by default; add --allow-write to enable the 8 write tools
uvx datanika-mcp --url https://app.datanika.io --api-key YOUR_KEY

See datanika-mcp/README.md for per-client config snippets and the full tool list, or datanika.io/docs/mcp-server for the hosted walkthrough.

Additional agent resources:

• /llms.txt — discovery document
• /api/v1/openapi.json — OpenAPI spec with typed inline schemas
• /api/v1/meta/agent-tiers — 5-tier capability stack (JSON)
• docs/api_versioning.md — stability tiers and deprecation policy

Open-Core Strategy

Core platform is open-source (AGPL-3.0). Cloud version adds billing, quotas, and usage metering via the datanika-cloud plugin.

Links

• 🌐 Website: datanika.io
• 🚀 Cloud Platform: app.datanika.io
• 📖 Documentation: datanika.io/docs
• 🔌 Connectors: datanika.io/connectors
• 📡 API Reference: datanika.io/docs/api

Security

Found a vulnerability? See SECURITY.md for our disclosure policy, supported versions, and reporting instructions.

Contributing

We welcome contributors and design partners. Open an issue or contact [email protected].

License

AGPL-3.0