Categories

Data Integration Frameworks

Frameworks for performing ELT / ETL

Repos

24

Stars

26,059

30 days

+742

More

cloudquery

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

Created Nov 2020

airbyte attack-surface-management aws azure bigquery
Total 6,563
30 days +27
apache

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

Created Jul 2021

dashboard-friendly data data-analysis data-engineering data-integration
Total 3,205
30 days +32
ConduitIO

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Created Jan 2022

conduit data-engineering data-integration data-pipeline data-stream
Total 619
30 days +3
realdatadriven

ETL / ELT / Reverse ETL Framework powered by DuckDB, designed to seamlessly integrate and process data from diverse sources. It leverages Markdown as a configuration medium, where YAML blocks define metadata for each data source, and embedded SQL blocks specify the extraction, transformation, and loading logic.

Created Dec 2024

data-engineering data-lake data-lakehouse data-quality data-quality-checks
Total 57
30 days +2
jitsucom

Service for bulk-loading data to databases with automatic schema management (Redshift, Snowflake, BigQuery, ClickHouse, Postgres, MySQL)

Created Jun 2022 Archived

data-engineering datawarehouse etl etl-pipeline ingestion
Total 224
30 days +0

From awesome-go (MIT), plus repos sorted by Jev.