Categories

Data Integration Frameworks

Frameworks for performing ELT / ETL

Repos

24

Stars

26,059

30 days

+742

More

PeerDB-io

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

Created Apr 2023

bigquery cdc clickhouse cloud-native distributed-systems
Total 3,325
30 days +20
gitstliu

A Mysql data to file/elasticsearch/kafka/redis/redis-cluster/mongodb(audit and normal)/TiDB/postgresql with binlog tool.

Created May 2019

audit-log binlog elasticsearch golang kafka
Total 99
30 days +0
hw2499

etl engine 轻量级 跨平台 流批一体ETL引擎 数据抽取-转换-装载 ETL engine lightweight cross platform batch flow integration ETL engine data extraction transformation loading

Created Apr 2022

cdc clickhouse crontab datafusion elastic
Total 78
30 days +0
retail-ai-inc

Real-time, one-way CDC from production MongoDB, MySQL, MariaDB, PostgreSQL and Redis to secondary databases or GCS, with PII masking and encryption. Built for analytics, cold storage and disaster recovery.

Created Nov 2024

cdc change-data-capture cold-storage data-masking database-replication
Total 64
30 days +0
compose

Sync data between persistence engines, like ETL only not stodgy

Created Oct 2014 Archived

elasticsearch etl go mongodb mysql
Total 1,438
30 days +0

From awesome-go (MIT), plus repos sorted by Jev.