OpenIngest
Open-source data ingestion engine. 500+ connectors. Log-based CDC. Schema drift handling. PII masking. Arrow → Parquet → Lakehouse.
接入你的智能体
從發布者安裝。不是 PlugMCP 端點。
遠端 URL 還是本機指令? 兩種安裝方式的差異
來自發布者
Open-source data ingestion engine. 500+ connectors. Log-based CDC. Schema drift handling. PII masking. Arrow → Parquet → Lakehouse.
Open-source data ingestion engine — rivaling Fivetran, Airbyte Cloud, and AWS DMS with 500+ connector architecture, CDC, schema drift, PII masking, Arrow → Parquet output, WASM plugins, and MCP server
# Install
curl -fsSL https://openingest.ai/install.sh | sh
# Run with demo
openingest daemon --config demo/pipeline.yaml
# Create a pipeline
openingest pipeline create -f my_pipeline.yaml
# Trigger a sync
openingest sync <pipeline-id>
# List connectors
openingest connector list
pipeline:
name: sales_cdc
source:
type: postgres
config:
host: localhost
database: prod
sync_mode: log_cdc
destination:
type: iceberg
config:
warehouse: s3://my-lakehouse/
transforms:
- type: pii_mask
rules:
- table: public.users
column: email
method: sha256
schedule: "*/5 * * * *"
schema_drift: auto_add
┌──────────────┐ ┌──────────────────┐ ┌────────────────┐
│ WASM Plugin │ │ Singer Adapter │ │ Airbyte Adapter│
│ (connectors)│ │ (Python taps) │ │ (Docker specs) │
└──────┬───────┘ └────────┬─────────┘ └───────┬────────┘
│ │ │
└────────────────────┼──────────────────────┘
│
┌───────▼────────┐
│ Pipeline │
│ Engine │
└───────┬────────┘
│
┌─────────────┼─────────────┐
│ │ │
┌─────▼────┐ ┌─────▼────┐ ┌─────▼────┐
│ Transform │ │ Schema │ │Checkpoint│
│ (PII) │ │ Drift │ │Manager │
└─────┬────┘ └─────┬────┘ └─────┬────┘
│ │ │
┌─────▼─────────────▼─────────────▼────┐
│ Arrow RecordBatch │
└────────────────┬─────────────────────┘
│
┌────────────────▼──────────────────────┐
│ Parquet File Writer │
│ ┌────────┐ ┌────────┐ ┌────────┐ │
│ │Iceberg │ │ Delta │ │ Hudi │ │
│ └────────┘ └────────┘ └────────┘ │
└────────────────────────────────────────┘
Benchmarks run on c6g.4xlarge (16 vCPU, 32GB RAM). Full methodology in BENCHMARKS.md.
30 Tier-1 (hand-crafted WASM): Postgres, MySQL, MariaDB, SQL Server, Oracle, MongoDB, DynamoDB, Cassandra, S3, GCS, Azure Blob, Kafka, Redpanda, RabbitMQ, HTTP/REST, GraphQL, CSV, JSON, Parquet, Snowflake, BigQuery, Redshift, ClickHouse, Elasticsearch, Redis, GitHub, Stripe, Salesforce, HubSpot, Shopify
200+ Tier-2 (declarative factory): Auto-generated from OpenAPI specs, JDBC drivers, and API templates
200+ Tier-3 (Singer/Airbyte compatibility): Run existing Singer taps and Airbyte sources
Apache 2.0
Open-source data ingestion engine — rivaling Fivetran, Airbyte Cloud, and AWS DMS with 500+ connector architecture, CDC, schema drift, PII masking, Arrow → Parquet output, WASM plugins, and MCP server
工具
尚無工具清單。遠端伺服器會在首次健康檢查後取得工具清單。
常見問題
在本頁複製適用於 Cursor、Claude Desktop、Claude Code 或 VS Code 的 stdio 設定。它使用發布者的安裝方式,而不是 PlugMCP 代理伺服器。
這筆記錄未列出必要的環境變數。
Openingest 透過 stdio 在本機執行。
上次檢查時間:2026-10-28。 Open-source data ingestion engine — rivaling Fivetran, Airbyte Cloud, and AWS DMS with 500+ connector architecture, CDC, schema drift, PII masking, Arrow → Parquet output, WASM plugins, and MCP server