How to Block AI Crawlers on Bubble in 2026
Technical SEO

How to Block AI Crawlers on Bubble in 2026

 Rahul Sharma  Nov 12, 2026  11 min  ~2,114 words

You can block AI crawlers on a Bubble app, and the main tool is built into Bubble itself: a robots.txt editor under Settings, in the SEO / Metatags section. Add a stanza for each training bot — GPTBot, ClaudeBot, CCBot, Google-Extended and the rest — save, and Bubble serves your rules at your domain's /robots.txt. That stops the honest bots. For the ones that ignore robots.txt, you need a layer Bubble does not provide on its own, and this guide explains what that layer is and when you need it.

The honest summary first. Bubble gives you real control over robots.txt, which is more than some no-code platforms offer. It does not give you server access, so you cannot write firewall rules or return a 403 the way you can on Nginx or Apache. Your enforcement options are: a correct robots.txt, Bubble's own privacy rules and page controls for sensitive content, and a service in front of Bubble — Cloudflare, which Bubble already uses under the hood — for anything stricter. Most Bubble apps only need the first two.

Step 1: open Bubble's robots.txt editor

In the Bubble editor, go to Settings, then the SEO / Metatags tab. There you will find a text area for your robots.txt file. Bubble serves a default file if you never touch it. The default typically allows all crawlers, disallows the test version of your app at /version-test/, and includes a link to your sitemap. That default is fine for search engines, but it says nothing about AI training bots — which is why you need to edit it.

Before you change anything, open your live app's robots.txt in a browser — yourdomain.com/robots.txt — and read what is there now. You need a baseline so you can tell whether your edit actually took effect. Bubble serves the file from your app's root, so what you see there is what every crawler sees.

Step 2: add the AI crawler stanzas

Replace or extend the file with rules for the training crawlers. A solid starting point for a Bubble app looks like this:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: Amazonbot
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: DuckAssistBot
Disallow: /

User-agent: cohere-ai
Disallow: /

User-agent: *
Disallow: /version-test/
Disallow: /reset_pw/
Disallow: /api/1.1/

Sitemap: https://yourdomain.com/sitemap.xml

Read the list carefully before you copy it, because it makes choices for you. The first block stops OpenAI's training crawler, GPTBot. The second stops OAI-SearchBot, which powers citations in ChatGPT search — include it only if you want out of ChatGPT answers entirely, and remove it if you want your public pages to stay citable. The same judgment applies to PerplexityBot and DuckAssistBot: they drive citations and referral traffic from AI answer engines, so blocking them trades visibility for control.

The last group — the User-agent: * block — is Bubble-specific housekeeping that experienced Bubble developers recommend: keep the test version of your app out of every index, keep password-reset pages out, and keep Bubble's workflow API paths out of search results. Replace yourdomain.com in the sitemap line with your real domain, and keep that line at the bottom no matter what else you change.

Two rules that are easy to get wrong. First, do not block Googlebot, Bingbot, or Applebot. Those are search crawlers, and blocking them removes your app from search results. Note the difference between Applebot and Applebot-Extended: the first is Apple's search crawler, the second is its AI training crawler — only the second belongs in your blocklist. Second, do not use a blanket 'User-agent: * Disallow: /' unless you genuinely want your whole app invisible to every search engine. That is a valid choice for a private internal tool, and a disaster for a public product.

Step 3: save, deploy, and verify

Save the file in Settings, then deploy your app to live if your changes require a deployment — settings changes in Bubble generally apply to the live app once saved, but confirm by checking the live URL rather than assuming. Open yourdomain.com/robots.txt in a fresh browser tab and confirm your new stanzas are there. If you see the old default, wait a few minutes and check again: Bubble sits behind Cloudflare, and a cached copy of the old file can linger briefly.

Then validate the file. Paste it into our free robots.txt tester to check the syntax, or generate a clean version first with our robots.txt generator and paste the result into Bubble. A single typo — a missing slash, a misspelled bot name — silently breaks a rule, and robots.txt gives you no error message when that happens. Verification is not optional.

Finally, watch for the effect. Honest bots re-read robots.txt within a day or two, so AI crawler traffic to your app should fall within 48 hours. You cannot see that fall in Bubble's own analytics, which is a real limitation: Bubble does not expose a server access log to app owners. If you need proof, put Cloudflare in front of your domain (covered below) and read its analytics, or watch your app's capacity usage for a drop.

What robots.txt cannot do on Bubble

Say it plainly: robots.txt is a request, not a lock. A bot that ignores it will keep crawling your app, and on Bubble you have no server config to stop it. That matters more than it does on a traditional stack, because the usual fallback — a firewall rule that returns 403 to a named user agent — is not something Bubble exposes to app owners.

There is a second limit. Robots.txt only controls crawling. It does not control indexing of pages that are already indexed, and it does not protect content that is already public. If a page is already in an AI training dataset, a robots.txt change does not remove it. If a page is public and a bot fetches it through a link rather than a crawl, robots.txt never enters the picture.

That is why Bubble's own privacy features matter more than most app owners realize. Privacy rules — set on each data type in the Data tab — control what data a page can even load, regardless of who is asking. A page that shows user data only to logged-in users is not leaking that data to crawlers, because the crawler cannot log in. Page-level settings matter too: keep admin dashboards, user settings, and account pages behind login, and do not expose them in your sitemap. The strongest protection on Bubble is not a robots file; it is never serving sensitive content to anonymous visitors in the first place.

Layer 2: Cloudflare in front of your Bubble app

Bubble already routes traffic through Cloudflare, but that is Bubble's Cloudflare setup, not yours — you do not control its rules. If you want real enforcement, put your own Cloudflare setup in front of your domain. The practical approach Bubble developers use is to manage DNS through Cloudflare, with the orange-cloud proxy enabled for your domain, and then use Cloudflare's own controls on top of Bubble.

Cloudflare's AI Crawl Control, available on all plans including free, lets you allow or block AI crawlers by name with a toggle, and it shows you which bots visit and which ones break your robots.txt rules. Because Cloudflare sits in front of your app, its rules run before traffic reaches Bubble at all — a blocked bot never touches your app or your capacity. For a fuller walkthrough, read our guide to blocking AI bots with Cloudflare.

One caution from the Bubble community: Cloudflare also offers a managed robots.txt feature, and it does not always merge cleanly with an origin robots.txt — there have been reports of the managed rules failing to combine with Bubble's own file. If you use both, keep Bubble's file as the source of truth, test your live /robots.txt after enabling any Cloudflare feature, and purge Cloudflare's cache for that exact URL if you see a stale copy.

Protecting specific content: APIs, files, and user data

Most Bubble apps have three kinds of content that need more than a robots file. Handle each one separately.

Backend workflows and the Data API. Bubble exposes workflow APIs under /api/1.1/ and data through its Data API. Disallow those paths in robots.txt, as in the example above, and — more importantly — require authentication on every workflow and data endpoint that is not deliberately public. An unauthenticated public workflow is crawlable and callable by anyone, bot or human.

Uploaded files. Files uploaded to Bubble get public URLs on Bubble's file storage. A robots.txt rule cannot protect a file URL that a bot already knows, and Bubble does not offer per-file access controls the way a private S3 bucket does. If a file must stay private, do not rely on obscurity: serve it through an authenticated workflow, or store it in a service with signed URLs.

Public profile and listing pages. Many Bubble apps have public pages — profiles, listings, marketplace items — that the owner wants indexed by Google but not used for AI training. That is exactly what split robots.txt stanzas are for: allow Googlebot, disallow GPTBot and the other trainers. The same split is covered in detail in our guide to allowing Googlebot while blocking AI crawlers. Do not solve this by making the pages login-only unless you are willing to lose the search traffic too.

What to do when a bot ignores your rules

First, confirm it actually ignored them. Because Bubble gives you no access log, use Cloudflare's analytics if you have your own setup in front, or check whether the bot's visits show up in your app's logs or capacity charts. A single stray visit is usually a bot that cached your old robots.txt; persistent visits after a week mean the bot is non-compliant.

For a non-compliant bot, your options on Bubble are limited but real. Report the bot to its operator — OpenAI, Anthropic, and the rest all publish crawler documentation and contact points, and a documented robots.txt violation is worth reporting. Tighten Cloudflare rules to block the bot's user agent at the edge. And for the content that matters most, move it behind authentication: a bot cannot read what it cannot log in to see. What you cannot do on Bubble is write a server rule, so do not waste time looking for a plugin that promises one — any plugin claiming server-level blocking on Bubble is selling something the platform does not expose.

FAQ

Where is the robots.txt setting in Bubble?

In the Bubble editor, go to Settings, then the SEO / Metatags tab. The robots.txt text area is there. Save your changes and check your live domain's /robots.txt to confirm they took effect.

Will blocking GPTBot remove my Bubble app from Google?

No. GPTBot is OpenAI's training crawler and Googlebot is Google's search crawler — different user agents, different jobs. As long as your rules do not block Googlebot, your search visibility is unaffected.

Can I block AI crawlers on the free Bubble plan?

The robots.txt editor is a settings feature, not a server feature, so the robots.txt route works wherever you can set a custom domain and edit settings. Check Bubble's current plan documentation for the exact availability on your plan before you rely on it.

Does Bubble's default robots.txt block AI crawlers?

No. The default file allows all crawlers and only disallows the /version-test/ path. You have to add the AI crawler stanzas yourself.

Should I block OAI-SearchBot and PerplexityBot too?

Only if you want out of AI answers entirely. OAI-SearchBot powers citations in ChatGPT search and PerplexityBot powers Perplexity answers, so blocking them removes your pages from those results. If you only want to stop training, block GPTBot and the other trainers and leave the answer bots alone.

What if a bot keeps crawling after I update robots.txt?

Give it 48 hours — honest bots re-read the file on their own schedule. After that, the bot is likely non-compliant. Your enforcement options on Bubble are Cloudflare rules in front of your domain, reporting the bot to its operator, and moving sensitive content behind login.

The bottom line

Blocking AI crawlers on Bubble is a two-part job. The robots.txt editor in Settings handles the honest bots, and it is worth doing carefully — split stanzas, verified syntax, no accidental Googlebot block. But the durable protection comes from Bubble's own features: privacy rules, login-gated pages, and authenticated APIs. Use robots.txt to declare the policy, use Bubble's access controls to enforce it, and add your own Cloudflare layer when you need a bot stopped at the edge rather than asked politely at the door.

Related Topics & Keywords

Share this article

Related Articles