返回浏览器检测 · iPhone 手机真假检测 · 搜索引擎蜘蛛记录 · 最新UA记录 · 更新日志

搜索引擎蜘蛛代码大全

 
最新UA记录 · 所有蜘蛛记录 · 百度 · Google 谷歌 · Bing 必应 · 360 搜索 · 搜狗 · 神马搜索 · Yandex · Yahoo 雅虎

本页面用来自然搜集来自搜索引擎抓取蜘蛛的 User-Agent 代码,当然有时候会发现除了百度Google搜狗360搜索之外,有很多没有见过的蜘蛛。

本页在2018年1月进行了升级,先按照已知的搜索引擎蜘蛛的顺序进行排列,然后就是未知的蜘蛛或者爬虫按照抓取的顺序自然收集和展示。

现在已知的蜘蛛出了前面提到的之外,还包括一些不常用的搜索引擎,比如 必应 bing神马搜索,还有国外的一些搜索引擎:YahooYandex 等。

相关文章:
蜘蛛与爬虫名称搜索引擎蜘蛛与网络爬虫记录
360 搜索
360Spider
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
360搜索
360Spider
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
360搜索
360Spider
Mozilla/5.0 (Linux; U; Android 4.0.2; en-us; Galaxy Nexus Build/ICL53F) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30; 360Spider
360 搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
360搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
360 搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.2.2661.102 Safari/537.36; 360Spider
360搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.2.2661.102 Safari/537.36; 360Spider
360 搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.4.2661.102 Safari/537.36; 360Spider
360搜索
360Spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.4.2661.102 Safari/537.36; 360Spider
百度搜索
Baiduspider
Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
百度搜索
Baiduspider
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html
百度搜索
Baiduspider
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
百度搜索
Baiduspider
Mozilla/5.0 (compatible; Baiduspider/2.0;+http://www.baidu.com/search/spider.html)
百度搜索
Baiduspider
Mozilla/5.0 (compatible;Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
百度搜索
Baiduspider
Mozilla/5.0 (compatible;Baiduspider/2.0;+http://www.baidu.com/search/spider.html)
百度搜索
Baiduspider
Mozilla/5.0 (Linux; Android 8.0.0; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/63.0.3239.83 Mobile Safari/537.36 T7/10.12 Baiduspider/10.12.0.12 (Baidu; P1 8.0.0)
百度搜索
Baiduspider
Mozilla/5.0 (Linux; U; Android 4.1; en-us; GT-N7100 Build/JRO03C;Baiduspider-ads)AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
百度搜索
Baiduspider
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
百度搜索
Baiduspider
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3(compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Bing 必应
bingbot
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Bing 必应
bingbot
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
CloudFlare 爬虫
CloudFlare
Mozilla/5.0 (compatible; CloudFlare-AlwaysOnline/1.0; +http://www.cloudflare.com/always-online) AppleWebKit/534.34
Google 谷歌
Googlebot
Googlebot/2.1 (+http://www.google.com/bot.html)
Google 谷歌
Googlebot
Mediapartners-Google
Google 谷歌
Googlebot
MoMozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google 谷歌
Googlebot
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google 谷歌
Googlebot
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google 谷歌
Googlebot
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36
Netcraft 爬虫
Netcraft
Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; +info@netcraft.com)
人人爬虫
Renren Share Slurp
Mozilla/5.0 (compatible; Renren Share Slurp 1.0; http://share.renren.com/share/help/webmaster)
Scrapy 爬虫
Scrapy
Scrapy/1.3.0 (+http://scrapy.org)
Scrapy 爬虫
Scrapy
Scrapy/1.4.0 (+http://scrapy.org)
Scrapy 爬虫
Scrapy
Scrapy/1.5.0 (+https://scrapy.org)
Scrapy 爬虫
Scrapy
Scrapy/1.5.1 (+https://scrapy.org)
搜狗搜索
sogou web spider
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25(compatible; Sogou wap spider/4.0; +http://www.sogou.com/docs/help/webmasters.htm#07)
搜狗搜索
Sogou web spider
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Yahoo 雅虎
Yahoo! Slurp
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Yandex
YandexBot
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
Yandex
Yandexbot
Mozilla/5.0 (compatible; YandexDirect/3.0; +http://yandex.com/bots)
Yandex
Yandexbot
Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexMobileBot/3.0; +http://yandex.com/bots)
神马搜索
YisouSpider
YisouSpider
蜘蛛或爬虫
_unknow spider
360spider (http://webscan.360.cn)
蜘蛛或爬虫
_unknow spider
ag_dm_spider v1.0
蜘蛛或爬虫
_unknow spider
Barkrowler/0.7 (+http://www.exensa.com/crawl)
蜘蛛或爬虫
_unknow spider
BUbiNG (+http://law.di.unimi.it/BUbiNG.html)
蜘蛛或爬虫
_unknow spider
CCBot/2.0 (http://commoncrawl.org/faq/)
蜘蛛或爬虫
_unknow spider
CCBot/2.0 (https://commoncrawl.org/faq/)
蜘蛛或爬虫
_unknow spider
IAS crawler (ias_crawler; http://integralads.com/site-indexing-policy/)
蜘蛛或爬虫
_unknow spider
Linguee Bot (http://www.linguee.com/bot; bot@linguee.com)
蜘蛛或爬虫
_unknow spider
MauiBot (crawler.feedback+dc@gmail.com)
蜘蛛或爬虫
_unknow spider
MauiBot (crawler.feedback+wc@gmail.com)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; 007ac9 Crawler; http://crawler.007ac9.net/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; AhrefsBot/5.2; News; +http://ahrefs.com/robot/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; Cliqzbot/2.0; +http://cliqz.com/company/cliqzbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; Gluten Free Crawler/1.0; +http://glutenfreepleasure.com/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; Jetslide; +http://jetsli.de/crawler)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; linkdexbot/2.2; +http://www.linkdex.com/bots/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://mj12bot.com/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; oBot/2.3.1; http://filterdb.iss.net/crawler/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; proximic; +http://www.proximic.com/info/spider.php)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; proximic; +https://www.comscore.com/Web-Crawler)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; Seekport Crawler; http://seekport.com/)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; SemrushBot/2~bl; +http://www.semrush.com/bot.html)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; SMTBot/1.0; http://www.similartech.com/smtbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; SurdotlyBot/1.0; +http://sur.ly/bot.html)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; ToutiaoSpider/1.0; http://web.toutiao.com/media_cooperation/;)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (compatible; YaK/1.0; http://linkfluence.com/; bot@linkfluence.com)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e YisouSpider/5.0 Safari/602.1
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Laserlikebot/0.1)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.75 Safari/537.36 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.75 Safari/537.36 (compatible; SMTBot/1.0; http://www.similartech.com/smtbot)
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 YisouSpider/5.0 Safari/537.36
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 YisouSpider/5.0 Safari/537.36
蜘蛛或爬虫
_unknow spider
Mozilla/5.0 (YunSecurityBot/1.0) Gecko/20100101 Firefox/36.0
蜘蛛或爬虫
_unknow spider
msnbot/1.1 (+http://search.msn.com/msnbot.htm)
蜘蛛或爬虫
_unknow spider
netEstate NE Crawler (+http://www.website-datenbank.de/)
蜘蛛或爬虫
_unknow spider
pyspider/0.3.10 (+http://pyspider.org/)
蜘蛛或爬虫
_unknow spider
Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)
蜘蛛或爬虫
_unknow spider
TelegramBot (like TwitterBot)
蜘蛛或爬虫
_unknow spider
TurnitinBot (https://turnitin.com/robot/crawlerinfo.html)
蜘蛛或爬虫
_unknow spider
Twitterbot/1.0
蜘蛛或爬虫
_unknow spider
VelenPublicWebCrawler (velen.io)
蜘蛛或爬虫
_unknow spider
Wotbox/2.01 (+http://www.wotbox.com/bot/)

逍遥峡谷 逍遥乱弹 张立博 爱之路 酷品网 宝贝派

网络违法犯罪举报