# https://walzone.com/robots.txt
#
# There was no robots.txt at all until 2026-07-29 (it 404'd). A 404 means
# "allow everything", so nothing was ever blocked — but it also meant no
# crawler had a Sitemap: pointer. Google has the sitemap via Search Console;
# every other crawler (OAI-SearchBot, bingbot, PerplexityBot, ...) discovers
# sitemaps HERE and nowhere else. That is the main reason this file exists.

User-agent: *
Allow: /

# The JSON API is not content. Crawling it wastes crawl budget, and bots that
# render JS fire our /beacon endpoint, which pollutes the funnel analytics.
Disallow: /interview/api/

# ---------------------------------------------------------------------------
# AI search crawlers — explicitly welcome.
# These build the indexes behind ChatGPT / Perplexity answers. Listing them by
# name is redundant with "User-agent: *" above, but it is deliberate: it states
# the intent, and a named group survives any future tightening of the wildcard.
#
# NOTE: GPTBot (OpenAI's *training* crawler) and ClaudeBot are NOT listed here
# and are therefore allowed by the wildcard. That is the status quo, not a
# decision — blocking training crawlers while keeping search crawlers is a live
# option the owner has not ruled on. See TASK.md.
# ---------------------------------------------------------------------------

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /

User-agent: DuckAssistBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: Applebot
Allow: /

Sitemap: https://walzone.com/interview/questions/sitemap.xml
Sitemap: https://walzone.com/books/sitemap_index.xml