Repositories

24

Total stars

38,524

Trending

gocolly

Elegant Scraper and Crawler Framework for Golang

Created Sep 2017

crawler crawling framework go golang
Total 25,630
30 days +66
1broseidon

Fast, stateless CLI for web search and scrape. Built for AI agents.

Created Mar 2026

ai-agents cli golang scraping search
Total 754
30 days +31
agnosto

An all-in-one scraper/downloader for Fansly written in go with the aid of A.I. Download content, Record lives, and Interact with post from your favorite creators. Available as standalone binaries for all platforms.

Created Jul 2024

downloader fansly fanslydownloader scraping
Total 164
30 days +12
bjesus

Swiss-army tool for scraping and extracting data from online assets, made for hackers

Created Aug 2024

css curl gjson json playwright
Total 4,782
30 days +11
alpkeskin

A high-performance proxy rotation engine with automated IP management and real-time health monitoring

Created Jun 2024

golang http http-proxy ip-rotation proxy
Total 776
30 days +11
shurco

🌱 goClone - clone websites in seconds

Created Jul 2023

cloner cloning crawler crawling go
Total 132
30 days +5
geziyor

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

Created Jun 2019

crawler go scraper scraping spider
Total 2,784
30 days +4
usestring

MCP server enabling agents to debug HTTP requests better (using powhttp)

Created Jan 2026

antibot-detection charles-proxy debugging debugging-proxy http
Total 85
30 days +2
slotix

Extract structured data from web sites. Web sites scraping.

Created Feb 2017

cdp chrome-fetcher crawling extract-data go
Total 716
30 days +2
Hyper-Solutions

Go SDK for Bot Protection Bypass - Automate Akamai, Incapsula, Kasada, and DataDome. No browsers required. Solve challenges and generate valid sensors/cookies via API.

Created Dec 2023

akamai antibot api automation bot
Total 71
30 days +1

Top

gocolly

Elegant Scraper and Crawler Framework for Golang

Created Sep 2017

crawler crawling framework go golang
Total 25,630
bjesus

Swiss-army tool for scraping and extracting data from online assets, made for hackers

Created Aug 2024

css curl gjson json playwright
Total 4,782
geziyor

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

Created Jun 2019

crawler go scraper scraping spider
Total 2,784
DataHenHQ

DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.

Created Feb 2021

crawler man-in-the-middle mitm proxy-server scraper
Total 816
alpkeskin

A high-performance proxy rotation engine with automated IP management and real-time health monitoring

Created Jun 2024

golang http http-proxy ip-rotation proxy
Total 776
1broseidon

Fast, stateless CLI for web search and scrape. Built for AI agents.

Created Mar 2026

ai-agents cli golang scraping search
Total 754
slotix

Extract structured data from web sites. Web sites scraping.

Created Feb 2017

cdp chrome-fetcher crawling extract-data go
Total 716
infinilabs

🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)

Created Jul 2017

crawler crawling elasticsearch lightweight scraping
Total 311
antchfx

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Created Sep 2017

crawler crawling framework golang scraping
Total 265
semanser

Get structured JSON data from any page.

Created Oct 2023

api go golang gpt gpt-3
Total 178
agnosto

An all-in-one scraper/downloader for Fansly written in go with the aid of A.I. Download content, Record lives, and Interact with post from your favorite creators. Available as standalone binaries for all platforms.

Created Jul 2024

downloader fansly fanslydownloader scraping
Total 164
rookmoot

A fast, modern and intelligent proxy rotator perfect for crawling and scraping public data.

Created Jan 2017

crawler crawling proxy proxy-server rotating-proxy
Total 156
antchfx

Extract data or evaluate value from HTML/XML documents using XPath

Created Oct 2016

extracting golang html scraping xml
Total 156
lizongying

A web crawling framework implemented in Golang, it is simple to write and delivers powerful performance. It comes with a wide range of practical middleware and supports various parsing and storage methods. Additionally, it supports distributed deployment. 基于golang实现的爬虫框架,编写简单,性能强劲。内置了丰富的实用中间件,支持多种解析、保存方式,支持分布式部署。

Created Apr 2023

crawler golang scraping scrapy spider
Total 150
shurco

🌱 goClone - clone websites in seconds

Created Jul 2023

cloner cloning crawler crawling go
Total 132
juliensalinas

Torrengo is a CLI (command line) program written in Go which concurrently searches torrents from various sources.

Created Jun 2018

1337x archiveorg concurrency go golang
Total 106
lnenad

Newser is a simple utility to generate a pdf with you favorite news articles

Created Feb 2022

news pdf-generation scraping supernote
Total 92
dewey

Plugin based RSS feed generator for sites that don't offer any. Serves RSS, Atom and JSON Feeds.

Created Aug 2018

atom-feed jsonfeed-generator rss rss-generator scraping
Total 89
usestring

MCP server enabling agents to debug HTTP requests better (using powhttp)

Created Jan 2026

antibot-detection charles-proxy debugging debugging-proxy http
Total 85
Matthew17-21

All-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha, Anticaptcha, and Capsolver API's!

Created Apr 2021

2captcha 2captcha-api anticaptcha anticaptcha-client capsolver
Total 78
Hyper-Solutions

Go SDK for Bot Protection Bypass - Automate Akamai, Incapsula, Kasada, and DataDome. No browsers required. Solve challenges and generate valid sensors/cookies via API.

Created Dec 2023

akamai antibot api automation bot
Total 71
bosniankicks

A Golang based Undetected Web Automation Framework

Created Jan 2025

akamai anti antibot bot browser
Total 62
pzaino

an open, self-hosted framework for discovering, collecting, understanding, correlating, transforming, and acting on information from heterogeneous environments.

Created Nov 2023

automation blue-team-tool content-detection content-discovery crawler
Total 62

Related

All topics →