Bot crawling
How to see which AI bots and search bots read your site, how much, and for what purpose, plus how bots are categorized.
In Analytics > Bot crawling, you see the bots that read your site by service, by category, and by name. You can check whether AIs like ChatGPT actually read your site.
Bot categories
Weekerp sorts requests into the categories below. The first three together are called AI bots.
| Category | Meaning |
|---|---|
| AI search & answers | Indexes your site to cite it in AI answers and search. This is the core of GEO |
| AI live fetch | A request that fetched a page on the spot because someone asked an AI |
| AI training | Collection for model training. You can block it with robots.txt |
| Search engines | Indexing bots of search engines such as Google and Naver |
| Link previews | Bots that build previews when a link is shared on KakaoTalk or Facebook |
| Ad review | Bots that review ad landing pages |
| Webmaster tools | Uptime monitoring and verification tools |
| Infrastructure | Underlying services such as feeds and push |
| SEO tools | Ahrefs, Semrush, and so on |
| Automation tools | Requests with an unknown purpose, such as python or curl |
| Other crawlers | Other bots whose purpose is known |
| Humans (browsers) | Requests from people, not bots |
| Unclassified | Requests not yet classified |
Bot requests by AI service
Pick a service tab (GPT, Gemini, Claude, Perplexity, Meta AI, Copilot, Others) to see that AI's search & answers / live fetch / training requests, their trend, and the list of bots that AI uses. Gemini does not include the regular Googlebot.
Bot requests over time · Bot requests by category
- Bot requests over time: Shows the 6 platforms with the most requests plus the rest, by date. Use the chips at the top to pick all bots, all AI, or a single category.
- Bot requests by category: The number of requests per category. Click a row to narrow the chart above and the table below to that category.
Bot ranking
Shows requests, HTML share, 4xx and 5xx error counts, and the last request time for each bot name (User-Agent).
- Include human browsers: Turn it on to show requests from people too.
- Use the search box to find a bot by name or UA.
- Click a row to see the Raw User-Agent and the paths fetched by that bot.
- Mostly files: A bot that mostly fetched images and files rather than pages.
- Many 4xx: A bot that requested many pages that don't exist. Check for broken links or old addresses.
When AI bots don't show up
- Check that robots.txt isn't blocking that bot. Manage robots.txt
- Try a longer period. On small sites, AI bots come only once every few days.