Short answer: On Framer, you get an automatic robots.txt file, and on Pro plans and above you can upload a custom one. On simpler no-code builders such as Carrd, you generally do not get a robots.txt editor at all. That does not leave you helpless, but it does change the plan: use the controls your platform actually gives you, put a firewall in front of the site where you control DNS, and keep anything truly private behind a login instead of trusting a file you cannot edit.
No-code builders are popular precisely because they hide the server. That convenience has a cost when you want fine control over AI crawlers. You cannot SSH into a Framer or Carrd server and edit a file. You work with toggles, uploads, and front-door services. This guide walks through each platform honestly — what it lets you do, what it does not, and the workarounds that actually hold up.
First, understand what your builder already does
Before you try to block anything, look at what is already live. Visit yourdomain.com/robots.txt and yourdomain.com/sitemap.xml in a browser and read them. On Framer, both files are generated automatically. The sitemap updates when you publish, and it leaves out pages you have set to noindex. The robots.txt file exists by default, but on lower plans you cannot rewrite it by hand. Many site owners skip this check, assume the worst, and then add a second layer of blocking that fights the first.
Also check your page-level settings. Framer gives every page a search-engine visibility toggle. Turning it off does not edit robots.txt — Framer adds a noindex meta tag to that page instead. That is actually the approach Google recommends for removing a page from search results, because a robots.txt block only stops crawling and does not guarantee removal from the index. The catch is that a noindex tag only works if crawlers can reach the page to read it. If you also block the page in robots.txt, crawlers may never see the noindex instruction, and the URL can linger in results if other pages link to it.
So the first rule for no-code sites is: do not stack robots.txt blocks and noindex tags on the same page and assume they reinforce each other. They can work against each other. Pick one mechanism per page, and write down which one you chose.
Framer: what you can and cannot control
Framer is the friendliest of the no-code builders for crawler control, with one important plan boundary. Out of the box, you get HTTPS, clean URLs, an automatic sitemap, automatic canonical tags, and per-page meta titles, descriptions, and social tags. For a marketing site, that covers most of what search engines need.
For AI crawlers specifically, here is the practical picture:
- Default robots.txt: auto-generated. Check it, but do not assume you can rewrite it from the editor on every plan.
- Custom robots.txt: on Pro plans and above, you can upload a fully custom robots.txt file through Site Settings and its well-known files area. This is the point where Framer stops being a limitation and becomes a normal site for crawler control. If AI-crawler rules matter to your business, this upgrade is often cheaper than the workarounds below.
- Page-level control: the search-engine toggle on each page adds a noindex tag. Use it for pages that should not appear in search at all — thank-you pages, duplicate variants, internal pages that slipped into the sitemap.
- llms.txt and similar files: the same well-known files area on Pro plans is how you publish extra discovery files. On lower plans, you cannot place a file at the site root, which limits what you can declare for AI tools.
A sensible Framer setup for a site that wants search visibility but not AI training use looks like this. Keep your main pages indexed. On a Pro plan, upload a custom robots.txt file that disallows the training crawlers — GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, and CCBot — while leaving search crawlers such as Googlebot and OAI-SearchBot allowed. On a lower plan, skip the custom file, keep the page toggles correct, and use the firewall approach in the next section if you need a hard block. Do not try to fake a custom robots.txt file by pasting rules into a page's custom code area; that text becomes page content, not a robots.txt file, and crawlers will never read it as one.
One more Framer trap: pages set to noindex drop out of the automatic sitemap. That is correct behaviour, but it means a single accidental toggle can make a key page vanish from discovery. After any bulk edit, re-check your sitemap and confirm your homepage, pricing, and main collection pages are still listed.
Carrd and simpler builders: work with the front door
Carrd is built for single-page sites, landing pages, and personal pages. It gives you far fewer server controls than Framer, and that is by design — a one-page site does not need a server dashboard. The honest starting point is this: if your builder gives you no robots.txt editor, stop looking for a hidden one. Check the platform's help pages for your plan, and if there is no file control, accept that your controls live elsewhere.
What you do have on a simple builder is usually a mix of the following: a site visibility or search-engine setting, password protection on some plans, custom domain support, and DNS control at your domain registrar. Each one solves a different problem, and none of them is a robots.txt file in disguise.
For a landing page that should be found by Google but not used for AI training, the realistic options are limited — and that limitation is worth stating plainly. A polite request in a file you cannot publish will not reach the crawler. Your levers are: keep the page public and accept that public pages can be crawled; move the page to a platform with file control if training opt-out is a business requirement; or put a firewall in front of the site, as described below, where you control DNS. Pretending a page setting blocks AI training when it only affects search indexing is how sites end up surprised.
For pages that must stay private — a client preview, an unreleased offer, a personal page shared with a few people — password protection is the right tool. A password gate stops crawlers, AI fetchers, and casual visitors alike, and it does not depend on any bot honouring a request. If your plan does not include password protection, do not publish the page on a public URL and hope nobody finds it. Host the preview somewhere private, or accept that a public URL is public.
The workaround that works on almost any builder: a firewall in front
If you control your domain's DNS, you can usually place a service such as Cloudflare in front of a no-code site and write firewall rules there, even though you cannot edit the origin server. The builder still serves the pages; the firewall decides which requests reach it. This is the closest no-code sites get to server-level bot control, and it works because it does not ask the builder for anything.
A practical firewall setup has three rules, in this order:
- Allow the crawlers you want. Googlebot, Bingbot, and the search-index crawlers behind the AI answers you want (such as OAI-SearchBot) should pass. Blocking a search crawler by accident is the most common firewall mistake, and it costs real traffic.
- Challenge or block the training crawlers you do not want. Use the published user-agent tokens — GPTBot, ClaudeBot, CCBot, and the other training agents — and, where your firewall supports it, verified bot lists rather than raw user-agent strings alone. A user-agent string is easy to fake, so a rule that blocks only the string will also catch some fakes and miss some real bots using other names. Treat the firewall as a strong filter, not a perfect wall.
- Scope sensitive rules to paths. If only a section of the site needs protection — a members area, a draft collection, an unreleased landing page — scope the rule to those paths. A site-wide block is harder to debug and easier to regret.
Before you switch DNS, check two things. First, confirm your builder supports the setup you plan: some builders need specific DNS records and can break if traffic is proxied in a way they do not expect. Test on a staging domain or during a quiet period, and confirm the site still loads, forms still submit, and the sitemap and robots.txt files still respond. Second, after the firewall is live, verify from the outside: fetch your robots.txt file, fetch a normal page, and check your firewall logs for blocked and allowed bot requests. A rule you never tested is a guess, not a control.
What does not work, however tempting
A few no-code workarounds circulate in forums and waste a surprising amount of time. Pasting robots.txt rules into a page's code block or footer does nothing — crawlers read the file at yourdomain.com/robots.txt, not the text on a page. Disabling right-click or copy on a page does nothing against bots, which never use a mouse. Adding a polite “do not scrape” note to your page copy is fine as a statement of intent, but no crawler is obliged to read it, and a note is not a substitute for a rule or a firewall.
Similarly, hiding a page from navigation does not hide it from crawlers. If the page is linked anywhere — a sitemap, another page, an old social post — it can be discovered. And deleting a page from navigation while leaving it in the sitemap is a common way for supposedly hidden pages to stay indexed. If a page should not be found, remove it from the sitemap (on Framer, that follows the noindex toggle), remove its links, and protect it properly.
A checklist for launching a no-code site with AI crawlers in mind
- Visit /robots.txt and /sitemap.xml and read them before you publish anything else.
- Decide, in one sentence, what you want: search visibility, AI citations, training opt-out, or some combination — then write rules for that goal, not for every bot you have heard of.
- On Framer Pro or above, upload a custom robots.txt file with separate rules for training crawlers and search crawlers. Test it with a robots.txt tester before and after publishing.
- On builders without file control, decide whether a firewall in front of the site is worth the DNS work. If it is, scope its rules narrowly and test search traffic afterwards.
- Use page-level noindex only for pages that should leave search entirely, and never combine it with a robots.txt block on the same page.
- Protect genuinely private pages with a password or a private host, not with a polite request.
- Recheck after every major publish. Builders regenerate sitemaps and default files on publish, and a platform update can quietly change what your site declares.
FAQ
Can I edit robots.txt on Framer's free plan?
Framer generates a robots.txt file automatically, but a fully custom upload is a Pro-plan-and-above feature through the well-known files area. On lower plans, use page-level visibility settings for search control and a front-door firewall if you need a harder block on AI crawlers.
Does turning off search visibility on a Framer page block AI crawlers?
No, not directly. The toggle adds a noindex tag, which tells search engines not to show the page in results. It is not a robots.txt block, and it does not by itself opt the page out of AI training collection. For training opt-out you need a robots.txt rule or a firewall rule naming the training crawlers.
Can I block AI crawlers on Carrd?
Carrd does not give you a robots.txt editor in the way a self-hosted site does, so you cannot publish crawler rules from inside Carrd itself. Your realistic options are the platform's visibility and password settings, a firewall in front of the site where you control DNS, or moving the page to a platform with file control if training opt-out is a hard requirement.
Will putting Cloudflare in front of my no-code site break it?
It can, if DNS is set up in a way the builder does not expect. Check your builder's DNS requirements first, test during a quiet period, and confirm pages, forms, and the sitemap still work afterwards. When it is set up correctly, a front-door firewall is one of the most reliable bot controls available to no-code sites.
Should I block all AI crawlers on a no-code marketing site?
Usually not. A blanket block also removes you from AI search answers that could send you traffic. Most marketing sites do better with a split: allow search crawlers, block training crawlers if that matters to you, and leave user-triggered fetchers alone so people can share your pages in chat.
How do I test my setup?
Fetch your live robots.txt file, check your sitemap for stray noindex pages, and run the file through a tester before publishing. After a firewall change, check its logs for a few days to confirm search crawlers still pass and the bots you meant to block are actually being stopped.
Build your file with the free robots.txt generator and check it with the robots.txt tester. For comparison, see blocking AI crawlers on WordPress and blocking AI crawlers on Squarespace.
