golang-collection

golang-collection / go-crawler-distributed

分布式爬虫项目,本项目支持个性化定制页面解析器二次开发,项目整体采用微服务架构,通过消息队列实现消息的异步发送,使用到的框架包括:redigo, gorm, goquery, easyjson, viper, amqp, zap, go-micro,并通过Docker实现容器化部署,中间爬虫节点支持水平拓展。

Last push 2 years ago

Created

Aug 2020

Total stars

53

7 days

0

30 days

0

12 months

+2

Star trends

README badge

Best of Go badge for golang-collection/go-crawler-distributed
[![Best of Go](https://bestofgo.dev/badge/golang-collection/go-crawler-distributed.svg)](https://bestofgo.dev/golang-collection/go-crawler-distributed)