返回浏览器检测 · iPhone 手机真假检测 · 搜索引擎蜘蛛记录 · 最新UA记录 · 更新日志

搜索引擎蜘蛛代码大全

 
最新UA记录 · 所有蜘蛛记录 · 百度 · Google 谷歌 · Bing 必应 · 360 搜索 · 搜狗 · 神马搜索 · Yandex · Yahoo 雅虎

本页面用来自然搜集来自搜索引擎抓取蜘蛛的 User-Agent 代码,当然有时候会发现除了百度Google搜狗360搜索之外,有很多没有见过的蜘蛛。

本页在2018年1月进行了升级,先按照已知的搜索引擎蜘蛛的顺序进行排列,然后就是未知的蜘蛛或者爬虫按照抓取的顺序自然收集和展示。

现在已知的蜘蛛出了前面提到的之外,还包括一些不常用的搜索引擎,比如 必应 bing神马搜索,还有国外的一些搜索引擎:YahooYandex 等。

相关文章:
蜘蛛与爬虫名称搜索引擎蜘蛛与网络爬虫记录
360 搜索
360Spider
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
360搜索
360Spider
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
360搜索
360Spider
Mozilla/5.0 (Linux; U; Android 4.0.2; en-us; Galaxy Nexus Build/ICL53F) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30; 360Spider
360 搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
360搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
360 搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.2.2661.102 Safari/537.36; 360Spider
360搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.2.2661.102 Safari/537.36; 360Spider
360 搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.4.2661.102 Safari/537.36; 360Spider
360搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.4.2661.102 Safari/537.36; 360Spider
百度搜索
Baiduspider
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html
百度搜索
Baiduspider
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
百度搜索
Baiduspider
Mozilla/5.0 (compatible;Baiduspider/2.0;+http://www.baidu.com/search/spider.html)
百度搜索
Baiduspider
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Bing 必应
bingbot
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Bing 必应
bingbot
Mozilla/5.0 (compatible;bingbot/2.0;+http://www.bing.com/bingbot.htm)
Bing 必应
bingbot
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
CloudFlare 爬虫
CloudFlare
Mozilla/5.0 (compatible; CloudFlare-AlwaysOnline/1.0; +http://www.cloudflare.com/always-online) AppleWebKit/534.34
Google 谷歌
Googlebot
Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Google 谷歌
Googlebot
Mediapartners-Google
Google 谷歌
Googlebot
MoMozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google 谷歌
Googlebot
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google 谷歌
Googlebot
Mozilla/5.0 (compatible; Googlebot/2.1; startmebot/1.0; +https://start.me/bot)
Google 谷歌
Googlebot
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Netcraft 爬虫
Netcraft
Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; [email protected])
Scrapy 爬虫
Scrapy
Scrapy/1.4.0 (+http://scrapy.org)
Scrapy 爬虫
Scrapy
Scrapy/1.5.0 (+https://scrapy.org)
搜狗搜索
Sogou web spider
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Yahoo 雅虎
Yahoo! Slurp
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Yandex
YandexBot
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
神马搜索
YisouSpider
YisouSpider
蜘蛛或爬虫
_unknow spider
ag_dm_spider v1.0
蜘蛛或爬虫
_unknow spider
Barkrowler/0.7 (+http://www.exensa.com/crawl)
蜘蛛或爬虫
_unknow spider
BUbiNG (+http://law.di.unimi.it/BUbiNG.html)
蜘蛛或爬虫
_unknow spider
CCBot/2.0 (http://commoncrawl.org/faq/)
蜘蛛或爬虫
_unknow spider
dcrawl/1.0
蜘蛛或爬虫
_unknow spider
Googlebot/2.1 (+http://www.google.com/bot.html)
蜘蛛或爬虫
_unknow spider
IAS crawler (ias_crawler; http://integralads.com/site-indexing-policy/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; baidubot/7.7; +http://www.baidu.com/bot.html)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; Cliqzbot/2.0; +http://cliqz.com/company/cliqzbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; Gluten Free Crawler/1.0; +http://glutenfreepleasure.com/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; linkdexbot/2.2; +http://www.linkdex.com/bots/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://mj12bot.com/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; oBot/2.3.1; http://filterdb.iss.net/crawler/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; proximic; +http://www.proximic.com/info/spider.php)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; proximic; +https://www.comscore.com/Web-Crawler)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; SMTBot/1.0; http://www.similartech.com/smtbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; SurdotlyBot/1.0; +http://sur.ly/bot.html)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25(compatible; Sogou wap spider/4.0; +http://www.sogou.com/docs/help/webmasters.htm#07)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexMobileBot/3.0; +http://yandex.com/bots)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0
蜘蛛或爬虫
_unknow spider
msnbot/1.1 (+http://search.msn.com/msnbot.htm)
蜘蛛或爬虫
_unknow spider
TelegramBot (like TwitterBot)
蜘蛛或爬虫
_unknow spider
TurnitinBot (https://turnitin.com/robot/crawlerinfo.html)
蜘蛛或爬虫
_unknow spider
Twitterbot/1.0
蜘蛛或爬虫
_unknow spider
Wotbox/2.01 (+http://www.wotbox.com/bot/)

逍遥峡谷 逍遥乱弹 张立博 爱之路 酷品网 宝贝派

网络违法犯罪举报