Categories

Data Integration Frameworks

Frameworks for performing ELT / ETL

Repos

24

Stars

26,059

30 days

+742

More

PeerDB-io

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

Created Apr 2023

bigquery cdc clickhouse cloud-native distributed-systems
Total 3,325
30 days +20
datazip-inc

OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

Created Oct 2024

apache-iceberg cdc change-data-capture data-ingestion data-pipeline
Total 2,038
30 days +480
transferia

Open Source Cloud Native Ingestion engine

Created Jun 2024

bigdata cdc clickhouse elt go
Total 199
30 days +5
hw2499

etl engine 轻量级 跨平台 流批一体ETL引擎 数据抽取-转换-装载 ETL engine lightweight cross platform batch flow integration ETL engine data extraction transformation loading

Created Apr 2022

cdc clickhouse crontab datafusion elastic
Total 78
30 days +0
retail-ai-inc

Real-time, one-way CDC from production MongoDB, MySQL, MariaDB, PostgreSQL and Redis to secondary databases or GCS, with PII masking and encryption. Built for analytics, cold storage and disaster recovery.

Created Nov 2024

cdc change-data-capture cold-storage data-masking database-replication
Total 64
30 days +0

From awesome-go (MIT), plus repos sorted by Jev.