How we serve bots (and why we don't cloak)
One render, one cache, every bot. Why RenderBeam serves the same HTML to Googlebot, GPTBot, ClaudeBot, and PerplexityBot — and what we add automatically.
Updated 5/4/2026
One render, one cache, every bot
Every bot that hits your site through RenderBeam gets the same fully-rendered HTML — built once by a real headless Chromium, cached, and replayed for the next crawler that asks. Googlebot, Bingbot, GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Google-Extended, AppleBot — all served from the same cached document.
This is intentional. It's also how every reputable pre-rendering service works.
Why we don't serve different content per bot
Some competitors market "AI-optimized HTML" or "bot-specific caches" as if Googlebot and ChatGPT need fundamentally different pages. They don't.
- Google calls it cloaking. Showing different content to a search bot than to a user — or to one bot vs. another — violates Google's Spam Policies. The penalty is removal from the index.
- AI crawlers want the same thing humans want. A fully-rendered DOM, semantic HTML, a real
</code>, a real <code><meta name="description"></code>, Open Graph tags, JSON-LD, clean headings, crawlable links.</li><li><strong>"AI-specific schema" is largely marketing.</strong> GPTBot, ClaudeBot, and PerplexityBot read the same Schema.org vocabulary as Google.</li></ul> <h2 id="what-actually-matters-to-ai-crawlers">What actually matters to AI crawlers</h2> <p>The biggest gap for SPA sites isn't bot-specific tuning — it's that the bot sees a blank <code><div id="root"></code> because it doesn't run JavaScript. Once you fix that with real pre-rendering, you've solved 95% of the problem. The remaining 5% is page quality: unique titles, meta descriptions, OG tags, JSON-LD, semantic HTML, and crawlable internal links.</p> <h2 id="what-we-add-automatically">What we add automatically</h2> <p>If your rendered page doesn't already include any <code><script type="application/ld+json"></code> block, RenderBeam injects a minimal Schema.org <code>WebPage</code> block at serve time — never overwriting your own schema. You can verify it with the <code>X-RB-AutoSchema: webpage</code> response header.</p> <h2 id="what-you-should-still-do-yourself">What you should still do yourself</h2> <p>Auto-injected <code>WebPage</code> schema is a floor, not a ceiling. Add page-specific structured data:</p> <ul><li>Product pages → <code>Product</code> + <code>Offer</code></li><li>Articles / blog posts → <code>Article</code> or <code>BlogPosting</code></li><li>FAQ sections → <code>FAQPage</code></li><li>Homepage → <code>Organization</code> and <code>WebSite</code></li></ul> <p>Audit what crawlers see with our free <a href="/tools/visibility-checker">Visibility Checker</a>, <a href="/tools/llms-txt-generator">llms.txt Generator</a>, and <a href="/tools/sitemap-verifier">Sitemap Verifier</a>.</p>
Was this page helpful?