Categories

Data Integration Frameworks

Frameworks for performing ELT / ETL

Repos

24

Stars

26,059

30 days

+742

More

cloudquery

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

Created Nov 2020

airbyte attack-surface-management aws azure bigquery
Total 6,563
30 days +27
datazip-inc

OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

Created Oct 2024

apache-iceberg cdc change-data-capture data-ingestion data-pipeline
Total 2,038
30 days +480
slingdata-io

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.

Created Oct 2020

elt etl extract load
Total 929
30 days +12
galaxy-io

Pluggable data replication with checkpointing, batching, and integrity events

Created Jun 2026

change-data-capture data data-collection data-integration data-pipeline
Total 333
30 days +110
transferia

Open Source Cloud Native Ingestion engine

Created Jun 2024

bigdata cdc clickhouse elt go
Total 199
30 days +5
realdatadriven

ETL / ELT / Reverse ETL Framework powered by DuckDB, designed to seamlessly integrate and process data from diverse sources. It leverages Markdown as a configuration medium, where YAML blocks define metadata for each data source, and embedded SQL blocks specify the extraction, transformation, and loading logic.

Created Dec 2024

data-engineering data-lake data-lakehouse data-quality data-quality-checks
Total 57
30 days +2

From awesome-go (MIT), plus repos sorted by Jev.