GitHub topics

data-pipeline

Mostly in Data Integration Frameworks 27%

Repositories

11

Total stars

27,094

Trending

datazip-inc

OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

Created Oct 2024

apache-iceberg cdc change-data-capture data-ingestion data-pipeline
Total 2,038
30 days +480
dagucloud

Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.

Created Apr 2022

agentic-workflows ai-agents ai-workflows airflow-alternative cron
Total 4,644
30 days +250
galaxy-io

Pluggable data replication with checkpointing, batching, and integrity events

Created Jun 2026

change-data-capture data data-collection data-integration data-pipeline
Total 333
30 days +110
bruin-data

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

Created Feb 2024

bigquery copy-database data-ingestion data-integration data-pipeline
Total 4,053
30 days +37
clidey

Where data access meets operational intelligence

Created Jun 2024

ai-agent data-catalog data-governance data-integration data-lineage
Total 5,069
30 days +31
rudderlabs

Privacy and Security focused Segment-alternative, in Golang and React

Created Jul 2019

bigquery cdp customer-data customer-data-lake customer-data-pipeline
Total 4,523
30 days +18
renart-data

Open source data platform for SQL and Python pipelines, notebooks and dashboards, with type checking, scheduling and definitions in Git.

Created Apr 2026

dashboards data-engineering data-pipeline data-platform developer-tools
Total 96
30 days +3
ConduitIO

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Created Jan 2022

conduit data-engineering data-integration data-pipeline data-stream
Total 619
30 days +3
superstreamlabs

Memphis.dev is a highly scalable and effortless data streaming platform

Created Feb 2022

data data-engineering data-pipeline data-stream-processing data-streaming
Total 3,446
30 days +2
reugn

A lightweight stream processing library for Go

Created Apr 2019

aerospike data-pipeline data-stream etl kafka
Total 2,179
30 days +2

Top

clidey

Where data access meets operational intelligence

Created Jun 2024

ai-agent data-catalog data-governance data-integration data-lineage
Total 5,069
dagucloud

Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.

Created Apr 2022

agentic-workflows ai-agents ai-workflows airflow-alternative cron
Total 4,644
rudderlabs

Privacy and Security focused Segment-alternative, in Golang and React

Created Jul 2019

bigquery cdp customer-data customer-data-lake customer-data-pipeline
Total 4,523
bruin-data

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

Created Feb 2024

bigquery copy-database data-ingestion data-integration data-pipeline
Total 4,053
superstreamlabs

Memphis.dev is a highly scalable and effortless data streaming platform

Created Feb 2022

data data-engineering data-pipeline data-stream-processing data-streaming
Total 3,446
reugn

A lightweight stream processing library for Go

Created Apr 2019

aerospike data-pipeline data-stream etl kafka
Total 2,179
datazip-inc

OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

Created Oct 2024

apache-iceberg cdc change-data-capture data-ingestion data-pipeline
Total 2,038
ConduitIO

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Created Jan 2022

conduit data-engineering data-integration data-pipeline data-stream
Total 619
galaxy-io

Pluggable data replication with checkpointing, batching, and integrity events

Created Jun 2026

change-data-capture data data-collection data-integration data-pipeline
Total 333
renart-data

Open source data platform for SQL and Python pipelines, notebooks and dashboards, with type checking, scheduling and definitions in Git.

Created Apr 2026

dashboards data-engineering data-pipeline data-platform developer-tools
Total 96
wybiral

A cross-platform tool for data pipelines.

Created Feb 2018

data-pipeline fan-in fan-out firehose golang
Total 94

Related

All topics →