roazon-bot

how to identify us

every request we make carries this User-Agent, with a contactable address in it:

for robots.txt matching, our token is roazon-bot. we run on Cloudflare Workers, so requests arrive from Cloudflare datacentre IPs rather than a fixed address — match on the User-Agent, not on an IP allowlist.

what we limit ourselves to

these are enforced in our crawler, not just promised here:

• request ratemax 1 per second
• pages per roaster, per run40
• products per roaster, per run120
• request timeout15s
• redirects followed3
• response body readfirst 5 MB
• robots.txtalways honoured

we read HTML and any catalogue feed your shop already publishes for machines — Shopify's /products.json, the WooCommerce Store API, your sitemap, JSON-LD. we never log in, never submit a form, and never touch checkout.

letting us in

if your site sits behind a bot challenge, we stop at it and record your roastery as "awaiting the owner" — we do not try to work around it. to let us read your public catalogue, allow the User-Agent above.

robots.txt (start here) — if your robots.txt disallows unknown crawlers, add an explicit group. this is not a security control, so it costs you nothing:

User-agent: roazon-bot
Allow: /

Cloudflare — only if a bot challenge is what is actually stopping us. our User-Agent is not proof of identity — anyone can send it — so never skip your whole WAF on that string alone. scope the exception to the bot-protection component only, on read-only requests:

(http.user_agent contains "roazon-bot"
  and http.request.method eq "GET")

set the action to Skip and select only your bot-fight / bot-management feature — leave managed rules, rate limiting, and your other firewall rules switched on. that way a spoofed User-Agent buys an attacker nothing except the bot check, on read-only requests.

tighter still, if you would rather name the paths: we read your homepage and about/contact pages first, then whichever catalogue endpoint your platform exposes, then each product page it lists. add your own product path prefix to the last line — that part differs per shop, which is why we cannot print a list that is complete for everyone:

(http.user_agent contains "roazon-bot"
  and http.request.method eq "GET"
  and (http.request.uri.path in {"/" "/about" "/about-us" "/over-ons" "/contact" "/contact-us"}
    or http.request.uri.path in {"/products.json" "/sitemap.xml" "/sitemap_index.xml"}
    or starts_with(http.request.uri.path, "/wp-json/wc/store/")
    or starts_with(http.request.uri.path, "/collections/")
    or starts_with(http.request.uri.path, "/products/")))

if the roastery pages come back but the beans do not, that last prefix is the line to widen.

on Shopify, WooCommerce or Lightspeed with no extra WAF in front, no change is usually needed — your catalogue is already readable. once you have allowed us, email [email protected] and we will re-read your shop.

turning us off

you never have to explain why, and we will not ask twice. add this to your robots.txt and we stop on our next visit:

User-agent: roazon-bot
Disallow: /

prefer to tell a human, or want your roastery removed from roazon entirely? email [email protected] and we will take the page down and stop reading your site. no account, no form, no conditions.

if this is your roastery

already on roazon? open your roastery's page and use the "claim it" link there — it carries the id that ties the claim to your listing.

not on roazon yet, or not sure? just email [email protected] — no account needed. claiming lets you correct details and see what customers click.

questions

a real person reads this address: [email protected].