Research on Detection Algorithm of Web Crawler 参考地址：https://ngte.cowtransfer.com/s/63c41532fba247 上一页 Design and Implementation of Scalable, Fully Distributed Web Crawler for a Web Search Engine
