Categories

Data Integration Frameworks

Frameworks for performing ELT / ETL

Repos

24

Stars

26,059

30 days

+742

More

PeerDB-io

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

Created Apr 2023

bigquery cdc clickhouse cloud-native distributed-systems
Total 3,325
30 days +20
datazip-inc

OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

Created Oct 2024

apache-iceberg cdc change-data-capture data-ingestion data-pipeline
Total 2,038
30 days +480
ConduitIO

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Created Jan 2022

conduit data-engineering data-integration data-pipeline data-stream
Total 619
30 days +3
transferia

Open Source Cloud Native Ingestion engine

Created Jun 2024

bigdata cdc clickhouse elt go
Total 199
30 days +5
gitstliu

A Mysql data to file/elasticsearch/kafka/redis/redis-cluster/mongodb(audit and normal)/TiDB/postgresql with binlog tool.

Created May 2019

audit-log binlog elasticsearch golang kafka
Total 99
30 days +0
hw2499

etl engine 轻量级 跨平台 流批一体ETL引擎 数据抽取-转换-装载 ETL engine lightweight cross platform batch flow integration ETL engine data extraction transformation loading

Created Apr 2022

cdc clickhouse crontab datafusion elastic
Total 78
30 days +0

From awesome-go (MIT), plus repos sorted by Jev.