PuerkitoBio / gocrawl
Polite, slim and concurrent web crawler.
Last push 5 years ago
crawler
robots-txt
Created
Sep 2012
Total stars
2,052
Last 7 days
0
Last 30 days
+1
Last 12 months
+8
README badge
[](https://bestofgo.dev/PuerkitoBio/gocrawl)