Bots

The bot names identified below are exposed by our Application Monitors that our users use. This page is updated daily.

  • Name Times identified
  • Semrush 749,853
  • bingbot 622,112
  • Googlebot 593,713
  • AhrefsBot 190,676
  • Yandex 168,085
  • MJ12bot 157,590
  • WordPress/ 131,323
  • NextAnalyticsUptimeBot 98,531
  • PetalBot 70,256
  • DotBot 57,890
  • python-requests 55,159
  • Applebot 39,221
  • BLEXBot 31,867
  • facebookexternalhit 29,539
  • Bot 25,266
  • curl 20,758
  • AspiegelBot 16,944
  • SentiBot 16,817
  • Uptime 14,456
  • Apache-HttpClient 10,954
  • SEOkicks 10,552
  • MegaIndex.ru 10,112
  • serpstatbot 9,889
  • Google Favicon 7,845
  • Monitoring 7,315
  • Pinterestbot 6,791
  • msnbot 6,593
  • Google-Read-Aloud 5,407
  • AddThis 5,161
  • MauiBot 5,128
  • Mediapartners-Google 4,606
  • Mail.RU 3,584
  • linkfluence 3,569
  • Crawl 3,244
  • CCBot 3,231
  • Seznam 2,961
  • GrapeshotCrawl 2,911
  • Daum/4 2,809
  • proximic 2,778
  • BingPreview 2,573
  • Sogou web 2,284
  • HeadlessChrome 2,232
  • WhatsApp 2,146
  • Go-http-client 1,934
  • Bloglovin 1,847
  • Buck/ 1,816
  • SurdotlyBot 1,762
  • BorneoBot 1,681
  • Feedspot 1,602
  • owler 1,210
  • Discordbot 1,183
  • GoogleProducer 1,179
  • Re-re Studio 1,174
  • PaperLiBot 1,144
  • CloudFlare-AlwaysOnline 957
  • special_archiver 897
  • zgrab 878
  • coccoc 865
  • Datanyze 863
  • Twitterbot 835
  • Cliqzbot 827
  • ping.blo.gs 795
  • SMTBot 793
  • ubermetrics-technologies 717
  • FeedBurner 650
  • crawl 630
  • inoreader 627
  • cortex/ 620
  • DuckDuckGo-Favicons-Bot 614
  • AppEngine-Google 614
  • FeedFetcher 576
  • AppleNewsBot 558
  • Nimbostratus-Bot 531
  • cron 432
  • axios/ 361
  • Qwantify 339
  • SiteLockSpider 320
  • UniversalFeedParser 313
  • org_bot 306
  • DuckDuckBot 278
  • Test Certificate Info 275
  • TweetmemeBot 255
  • Wget 241
  • adscanner 240
  • Lighthouse 233
  • Scrapy 224
  • AdsTxtCrawl 221
  • feedbot 210
  • Baiduspider 205
  • Netcraft 193
  • Jersey/ 183
  • Twingly 180
  • github.com 162
  • Google Web Preview 148
  • linkdex 136
  • 360Spider 130
  • WPScan 127
  • DomainStatsBot 120
  • Panscient 118
  • Dataprovider 118
  • Mojeek 117
  • masscan/ 106
  • admantx 106
  • Microsoft Office 99
  • Java/ 96
  • SerendeputyBot 95
  • newspaper/ 94
  • Google-Site-Verification 91
  • TelegramBot 85
  • Trendsmap 81
  • nyu 78
  • okhttp 78
  • Clarabot 70
  • Internet-structure-research-project-bot 68
  • tweetedtimes 65
  • evc-batch 60
  • AdsBot 59
  • fetch/1 59
  • TprAdsTxtCrawl 59
  • Mediumbot-MetaTagFetcher 59
  • Python-urllib 58
  • Semanticbot 58
  • MagpieRSS 58
  • Microsoft Windows Network Diagnostics 57
  • eContext/ 57
  • oBot 55
  • FlipboardProxy 54
  • AwarioSmartBot 53
  • Mixnode 53
  • via ggpht.com GoogleImageProxy 50
  • Mediatoolkitbot 49
  • scalaj-http 48
  • DMBrowser 47
  • VoluumDSP-content-bot 47
  • ZoomBot 46
  • Jugendschutzprogramm-Crawl 45
  • woorankreview 45
  • contxbot 44
  • RyteBot 44
  • SkypeUriPreview 43
  • HuaweiWebCatBot 41
  • Blackboard Safeassign 40
  • Ruby 40
  • OdklBot 37
  • wpif 32
  • Yeti 30
  • bot 30
  • istellabot 30
  • LibVLC 29
  • checker 28
  • Google Page Speed Insights 27
  • AHC 27
  • Photon/ 26
  • Upflow 26
  • PhantomJS/ 26
  • adreview/ 25
  • developers.google.com/+/web/snippet/ 24
  • COMODO DCV 24
  • Hexometer 24
  • bidswitchbot 23
  • YisouSpider 23
  • Slackbot 22
  • KOCMOHABT 22
  • WordPress.com 21
  • domainsbot 21
  • httpscheck 20
  • nominet.uk 20
  • PTST/200427 19
  • ia_archiver 18
  • Webster 18
  • spaziodati 18
  • aiohttp/ 18
  • Siteimprove 18
  • LinkedInBot 17
  • SEMrush 17
  • BuiltWith 15
  • Snapbot 15
  • Slurp 15
  • CATExplorador 14
  • Bytespider 14
  • Wappalyzer 13
  • wkhtmlto 13
  • LightspeedSystems 12
  • ltx71 12
  • datagnionbot 11
  • NetSystemsResearch 10
  • panscient 10
  • 7Siters 10
  • Yahoo Link Preview 10
  • PeoplePal 9
  • Konturbot 9
  • OgScrper 9
  • GuzzleHttp 9
  • Symfony BrowserKit 9
  • The Knowledge AI 9
  • W3C-checklink 9
  • ShortLinkTranslate 8
  • DataXu 8
  • Screaming 8
  • AlphaBot 8
  • trendiction 8
  • Checker 8
  • Bidtellect 7
  • URL_server_crawl 7
  • HealthCheckBot 7
  • 7
  • PagePeeker 7
  • Bingbot 7
  • Google-Structured-Data-Testing-Tool 7
  • Validator 6
  • aiHit 6
  • Genieo 6
  • BDCbot 6
  • PTST/200420 6
  • Commons-HttpClient 5
  • PTST/200604 5
  • Turnitin 5
  • webtechbot 5
  • YahooBot 5
  • Scoop 5
  • spider 5
  • Riverbot 5
  • Curl 4
  • Eyeotabot 4
  • VelenPublicWebCrawl 4
  • FastCrawl 4
  • YahooCacheSystem 4
  • arquivo-web-crawl 4
  • Embedly 4
  • craw/ 4
  • Dispatch/ 4
  • Grammarly 4
  • SimplePie 4
  • Mojolicious (Perl) 3
  • Perlu - 3
  • GTmetrix 3
  • MSIECrawl 3
  • SiteCheckerBotCrawl 3
  • PTST/200506 3
  • RestSharp 3
  • Faraday v 3
  • PTST/200422 3
  • SMRF URL Expander 3
  • Sitemaps Generator 3
  • PocketParser 3
  • PTST 3
  • Pingdom 3
  • Y!J-ASR 3
  • Cocolyzebot 3
  • Nuzzel 3
  • PTST/200518 3
  • PTST/200415 3
  • bitlybot 2
  • MicroMessenger/ 2
  • Traackr.com 2
  • Cron 2
  • startmebot 2
  • googlebot 2
  • AdAuth/ 2
  • ips-agent 2
  • GoScraper 2
  • libwww 2
  • HttpRequest 2
  • MarkCrawl 2
  • LongURL API 2
  • Wheregoes? 2
  • PTST/200602 2
  • vkShare 2
  • PTST/200629 2
  • robot 2
  • PTST/200501 1
  • ruby/2 1
  • Unshorten.It 1
  • RedirectBot 1
  • aria2 1
  • LivelapBot 1
  • pilicanbot 1
  • Apache-HttpAsyncClient 1
  • octopus 1
  • PTST/200513 1
  • Jigsaw 1
  • PTST/200530 1
  • GetLinkInfo 1
  • Cloudinary 1
  • WebDataStats 1
  • yourls.org 1
  • MixrankBot 1
  • PostmanRuntime 1
  • fasthttp 1
  • shrinktheweb 1
  • monitoring 1
  • W3C_Validator 1
  • WebCorp 1
  • Google-Adwords 1