GitHub topics

scraper

Mostly in Text Processing 20%

Repositories

54

Total stars

91,485

Trending

Solr159

开箱即用的本地 JAV/视频 刮削、管理、播放软件,支持命令行一键安装和 Docker 部署。只需简单添加目录,即可打造你的私人 JAV/视频 媒体库,带给你顶级的浏览体验,懒人必备。| Your local JAV/video manager.

Created Mar 2026

golang jav javbus javdb javlibrary
Total 944
30 days +187
iawia002

👾 Fast and simple video download library and CLI tool written in Go

Created Feb 2018

bilibili crawler download downloader go
Total 31,878
30 days +116
gocolly

Elegant Scraper and Crawler Framework for Golang

Created Sep 2017

crawler crawling framework go golang
Total 25,630
30 days +66
go-rod

A Chrome DevTools Protocol driver for web automation and scraping.

Created Jan 2020

automation cdp chrome-devtools chrome-devtools-protocol chrome-headless
Total 7,165
30 days +29
Anakin-Inc

Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up

Created Mar 2026

ai anti-detect golang markdown open-source
Total 3,818
30 days +28
edoardottt

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

Created Apr 2021

bugbounty crawler crawling endpoint-discovery endpoints
Total 3,819
30 days +28
Dhairya3391

Kari just a tui for getting media from providers and playing it in your media player with nice to have features.

Created Jun 2026

anilist anime cartoons cli go
Total 72
30 days +11
bjesus

Swiss-army tool for scraping and extracting data from online assets, made for hackers

Created Aug 2024

css curl gjson json playwright
Total 4,782
30 days +11
Nhoya

OSINT Swiss Army Knife

Created Aug 2017

axfr bitbucket crawler forensics git
Total 680
30 days +6
gosom

Golang Crawling and scraping framework

Created Mar 2023

crawler go go-framework golang scraper
Total 219
30 days +5

Top

Top 50 of 54

iawia002

👾 Fast and simple video download library and CLI tool written in Go

Created Feb 2018

bilibili crawler download downloader go
Total 31,878
gocolly

Elegant Scraper and Crawler Framework for Golang

Created Sep 2017

crawler crawling framework go golang
Total 25,630
go-rod

A Chrome DevTools Protocol driver for web automation and scraping.

Created Jan 2020

automation cdp chrome-devtools chrome-devtools-protocol chrome-headless
Total 7,165
bjesus

Swiss-army tool for scraping and extracting data from online assets, made for hackers

Created Aug 2024

css curl gjson json playwright
Total 4,782
edoardottt

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

Created Apr 2021

bugbounty crawler crawling endpoint-discovery endpoints
Total 3,819
Anakin-Inc

Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up

Created Mar 2026

ai anti-detect golang markdown open-source
Total 3,818
geziyor

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

Created Jun 2019

crawler go scraper scraping spider
Total 2,784
metafates

📖 The most advanced (yet simple) cli manga downloader in the entire universe! Lua scrapers, export formats, anilist integration, fancy TUI and more!

Created May 2022

anilist anime cli comic-downloader command-line
Total 1,744
Solr159

开箱即用的本地 JAV/视频 刮削、管理、播放软件,支持命令行一键安装和 Docker 部署。只需简单添加目录,即可打造你的私人 JAV/视频 媒体库,带给你顶级的浏览体验,懒人必备。| Your local JAV/video manager.

Created Mar 2026

golang jav javbus javdb javlibrary
Total 944
DataHenHQ

DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.

Created Feb 2021

crawler man-in-the-middle mitm proxy-server scraper
Total 816
piquette

📊 Financial markets data library implemented in go.

Created Feb 2018

cryptocurrency data finance financial-data financial-markets
Total 791
slotix

Extract structured data from web sites. Web sites scraping.

Created Feb 2017

cdp chrome-fetcher crawling extract-data go
Total 716
Nhoya

OSINT Swiss Army Knife

Created Aug 2017

axfr bitbucket crawler forensics git
Total 680
lapwat

Scrape the web in the eink era. Convert websites into ebooks and markdown.

Created Sep 2021

cli command-line command-line-tool ebook ebook-reader
Total 406
yields

A web crawler for Go

Created Sep 2020

go golang scraper spider web-crawler
Total 282
tnychn

⚡️ A magnet link scraper for streaming videos (movies, tv shows, anime, porn) along with subtitles.

Created Mar 2019

anime bittorrent cli downloader golang
Total 274
R4yGM

DorkScout - Golang tool to automate google dork scan against the entiere internet or specific targets

Created Jul 2021

bug-bounty crawler ghdb golang google-dorks
Total 244
Dentrax

GMDB is the ultra-simple, cross-platform Movie Library with Features (Search, Take Note, Watch Later, Like, Import, Learn, Instantly Torrent Magnet Watch)

Created Mar 2019

goquery hacktoberfest imdb imdb-cli imdb-webscrapping
Total 241
xxxsen

一个正经的电影元数据刮削工具

Created Jun 2024

capture jellyfin metadata movie nfo
Total 233
gosom

Golang Crawling and scraping framework

Created Mar 2023

crawler go go-framework golang scraper
Total 219
cornelk

Web scraper that can create an offline readable version of a website

Created Feb 2017

go golang scraper webscraping
Total 218
gnur

Demeter is a tool for scraping the calibre web ui

Created Nov 2019

calibre demeter download ebook-downloader hacktoberfest
Total 201
YABOIpy

The Fast And Advanced Discord Multi-Tool

Created Nov 2022

automation bot discord discord-multi-tool discord-multitool
Total 200
saeeddhqan

Interactive CLI Web Crawler

Created Jul 2020

cli crawler data-mining fuzzing go
Total 183
lawzava

Concurrent Go CLI that crawls a website and extracts the email addresses it publishes, with headless Chrome support for JS-rendered pages

Created Oct 2019

cli cli-utility email golang scrape-emails
Total 182
semanser

Get structured JSON data from any page.

Created Oct 2023

api go golang gpt gpt-3
Total 178
adyzng

京东App自动登录,在线商品自动下单

Created Jun 2017

golang jingdong scraper
Total 175
twiny

Domain names collector - Crawl websites and collect domain names along with their availability status.

Created Jul 2020

backlinks crawler domain expired-domain golang
Total 165
rookmoot

A fast, modern and intelligent proxy rotator perfect for crawling and scraping public data.

Created Jan 2017

crawler crawling proxy proxy-server rotating-proxy
Total 156
henson

Tracking the most popular Github repos, updated daily.

Created Jan 2017

go markdown scraper
Total 147
PeterYangs

文章采集工具 Article collection tool

Created Feb 2021

article-spider go golang scraper spider
Total 143
luckylittle

Grabs all of the audio files from all of the Blinkist books

Created Jan 2019

audiobooks blinkist books crawler data-archiving
Total 139
Pilfer

A simple scraper for Ultimate-Guitar.com's mobile API, written in Go.

Created Jul 2019

frida golang guitar reverse-engineering scraper
Total 134
PxyUp

New way for collect information from the API's/Websites

Created Feb 2023

ai-agents api go golang json
Total 133
alash3al

Scraply a simple dom scraper to fetch information from any html based website

Created May 2019

crawler crawling dom golang scraper
Total 130
integrii

A Go package for working with headless Chrome. Run interactive JavaScript commands on web pages with Go and Chrome.

Created Sep 2017

chrome cli go headless macos
Total 123
tonywangcn

27.6% of the Top 10 Million Sites are Dead

Created Oct 2024

domain kubernetes scraper terraform webscraping
Total 118
badoux

Golang pkg to quickly return a preview of a webpage (title/description/images)

Created Jan 2017

crawler description engine go golang
Total 110
notdodo

Scrape/Parse Pastebin using GO and expression grammar (PEG)

Created May 2017

go osint pastebin peg scraper
Total 96
vegidio

A desktop app to easily download media files hosted on popular websites.

Created Nov 2024

bunkr coomer cyberdrop downloader erome
Total 94
fengqi

Command-line scraper for movies and TV shows. Fetches TMDB metadata and writes Kodi-compatible NFO files and artwork. Also works with Emby, Jellyfin and Plex.

Created Dec 2021

cli emby jellyfin kodi kodi-scraper
Total 92
piquette

A cli tool to streamline financial markets data analysis 🔧

Created Jun 2018

cli data data-science finance go
Total 84
parasyte-x

Reverse Engineered Twitter's API

Created Sep 2023

reverse-engineering scraper twitter twitter-api twitter-api-v2
Total 80
mahesh-hegde

Bulk image downloader for reddit.

Created Jun 2021

downloader golang reddit scraper
Total 80
Matthew17-21

All-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha, Anticaptcha, and Capsolver API's!

Created Apr 2021

2captcha 2captcha-api anticaptcha anticaptcha-client capsolver
Total 78
go-rod

go-rod API 中文文档

Created Aug 2022

automation go-rod headless rod scraper
Total 75
Dhairya3391

Kari just a tui for getting media from providers and playing it in your media player with nice to have features.

Created Jun 2026

anilist anime cartoons cli go
Total 72
averagesecurityguy

Extensible paste site scraper written in Golang.

Created Dec 2017

gists golang hacking-tool osint osint-tool
Total 71
philipjkim

Webpage summary extractor using Facebook Open Graph and arc90's readability

Created Apr 2016

opengraph readability scraper
Total 68
ofdl

A self-contained OnlyFans Scraper

Created Jul 2023

downloader onlyfans scraper
Total 64

Related

All topics →