A command-line tool for recursively spidering webpages for URLs. It works by actively traversing websites, following links embedded in webpages, and parsing files (including sitemaps & robots.txt), to uncover every URL.
Last push 3 days ago
bug-bounty
bug-bounty-tools
contentdiscovery
crawler
ethical-hacking
ethical-hacking-tools
go
golang
penetration-testing
penetration-testing-tools
reconnaissance
red-teaming
red-teaming-tools
web-security
Created
Jul 2021
Total stars
110
Last 7 days
0
Last 30 days
0
Last 12 months
+8
README badge
[](https://bestofgo.dev/hueristiq/xcrawl3r)