Polite, slim and concurrent web crawler.

Last push 5 years ago

crawler robots-txt

Created

Sep 2012

Total stars

2,052

7 days

0

30 days

+1

12 months

+8

Star trends

README badge

Best of Go badge for PuerkitoBio/gocrawl
[![Best of Go](https://bestofgo.dev/badge/PuerkitoBio/gocrawl.svg)](https://bestofgo.dev/PuerkitoBio/gocrawl)