Here is the short answer: you cannot block AI crawlers on Medium by editing robots.txt, because Medium does not let writers edit it. But Medium’s own robots.txt already blocks GPTBot, OpenAI’s training crawler, while still allowing OAI-SearchBot, PerplexityBot, and Google-Extended. So your writing is already kept out of OpenAI’s training data, but it can still show up in ChatGPT search answers and Perplexity citations. If that split does not match what you want, your real options are to unlist or paywall individual stories, move your best work to a site you control, or stop publishing original work on Medium at all. This guide explains exactly what Medium allows and blocks, how to check it yourself, and which of those four moves fits your situation.
What Medium’s robots.txt Actually Says
Every website has a robots.txt file, and Medium is no exception. It lives at medium.com/robots.txt, and it is the same file for every writer on the platform. You cannot change it, but you should know what it says, because it decides which AI companies can read your stories.
A writer who checked Medium’s robots.txt on 31 October 2026 found that it blocks GPTBot, the crawler OpenAI uses to collect training data, while allowing OAI-SearchBot, the crawler behind ChatGPT search, along with PerplexityBot and Google-Extended. That is a deliberate split: training is out, AI search is in. The same check found that Substack and DEV block none of the four, while Indie Hackers blocks all four, so Medium sits in the middle of the pack.
What does this mean in practice? If you publish an essay on Medium today, OpenAI’s training crawler is told to stay away. But when someone asks ChatGPT a question your essay answers, OAI-SearchBot may still fetch and cite it. Perplexity may do the same. And Google-Extended being allowed means Google may use your text to train Gemini and to ground answers in Gemini apps. That last point surprises most writers, because Google-Extended sounds like a search setting. It is not. It is a training and grounding token, and it is separate from Googlebot, the crawler that puts pages into Google Search.
One more honest caveat: robots.txt is a request, not a lock. The big AI companies respect it, but nothing in the file forces a scraper to obey. Medium also serves its pages behind Cloudflare, and the same writer found that plain fetches of Medium pages hit a Cloudflare challenge, which is a second, accidental layer of protection against casual scrapers. It is not a policy you control, though, and it can change without notice.
Why Writers Can’t Edit Medium’s robots.txt
On a self-hosted site, robots.txt is a file you own. You open it, add two lines, and GPTBot is gone. On Medium, you rent space on someone else’s domain. The robots.txt file belongs to medium.com, and it applies to every story, every publication, and every author profile at once. Medium has no setting, toggle, or paid plan that lets an individual writer override it.
Publications with a custom domain do not escape this either. A custom domain changes the address your readers see, but the content is still served by Medium, and the robots.txt that governs it is still Medium’s. There is no per-publication or per-author robots.txt on the platform. If you have read a guide that tells you to “just add a Disallow rule to your Medium robots.txt,” that guide is wrong. There is no file for you to edit.
This is the single most important thing to understand before you spend an afternoon on workarounds. Any solution has to work within what Medium actually offers: story-level visibility settings, the paywall, and the choice of where else to publish.
What Medium Already Does for You
It is worth giving Medium credit where it is due. The platform’s default position is more protective than most writers realise. GPTBot, the most-discussed training crawler on the web, is already blocked. That covers OpenAI’s foundation-model training, which is the use most writers object to. Common Crawl’s CCBot, the crawler that builds the public dataset many models train on, is also widely reported as blocked on Medium, though you should verify the current file yourself rather than trust any article, including this one, because these files change.
Medium also has a "metered paywall" system. Stories behind the paywall are still readable by search engines in many cases, because Medium wants them discovered, but they are not freely copyable in full by every reader. That is not an AI block, and you should not treat it as one. A crawler that is allowed in robots.txt can still read a paywalled story’s text if Medium serves it to crawlers, which it generally does for indexing purposes.
The honest summary: Medium protects you from the one crawler most writers name first, and it leaves the door open to the AI search crawlers that can send you readers. For many writers, that is actually the right default. The problem is only that you cannot change it.
Your Four Real Options
Option 1: Unlist Individual Stories
Medium lets you unlist a story, which removes it from your profile, from topic pages, and from search results on Medium itself. An unlisted story is still readable by anyone with the direct link, and it is still on medium.com, so it is still governed by Medium’s robots.txt. Unlisting reduces discovery, which reduces the chance an AI search crawler finds and cites the piece, but it does not block any crawler. Treat unlisting as a visibility choice, not a privacy control. Use it for drafts, for pieces you shared with a small group, or for work you no longer want promoted. Do not use it as your AI opt-out.
Option 2: Put Work Behind the Paywall
If you are in the Medium Partner Program, you can put a story behind the paywall. This does not block AI crawlers either, for the reason above: Medium wants paywalled stories indexed so readers can find them. But it does change the economics. A crawler that copies a paywalled story is copying content Medium monetises, and Medium’s own commercial interests sit on your side of that argument. The paywall is a business decision first and an AI decision a distant second. Use it because you want to earn from the piece, not because you think it hides the text from bots.
Option 3: Publish Original Work on a Site You Control
This is the only option that gives you real control. If AI training is a red line for you, publish your original work on a domain you own, where you can write your own robots.txt, and use Medium only for reposts, excerpts, or essays you are comfortable sharing. A simple setup works: publish first on your own site, wait for Google to index it, then import or repost to Medium with a canonical link back to the original. Medium’s import tool sets the canonical URL for you, which also protects your search ranking.
On your own site, the robots.txt rules are yours to write. The standard block for training crawlers while staying visible in AI search looks like this:
User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: CCBot Disallow: / User-agent: Google-Extended Disallow: /
That file keeps OAI-SearchBot, Claude-SearchBot, and PerplexityBot allowed, so your pages can still be cited in AI answers, while the training crawlers are told to stay out. If you want to generate a file like this without writing it by hand, our free robots.txt generator builds it from checkboxes, and our robots.txt tester checks it before you publish. For the full reasoning behind splitting training crawlers from search crawlers, see our guide on whether you should block AI crawlers at all.
Option 4: Leave Medium for Original Work
Some writers decide the trade is not worth it. If your writing is your livelihood, if you publish research or reporting that took months, or if you simply do not want your words in any AI system, the cleanest answer is to stop publishing original work on platforms you do not control. Keep a Medium profile for distribution, but make your own site the home of everything that matters. This is a bigger decision than a settings toggle, and it is the only one that is fully enforceable, because it rests on ownership rather than on a request file.
How to Check What Medium Allows Right Now
Do not take any article’s word for a robots.txt file, including this one. Files change, and Medium could change its policy tomorrow. Here is how to check in under a minute:
- Open medium.com/robots.txt in your browser.
- Use your browser’s find function (Ctrl+F or Cmd+F) and search for GPTBot. If you see a group that names GPTBot with a Disallow rule, training crawls are blocked.
- Search for OAI-SearchBot, PerplexityBot, and Google-Extended. If a bot has no group at all, it falls through to the general rules, which usually allow it.
- Repeat the check every few months, or whenever Medium announces a policy change. Set a calendar reminder. A file you checked once is a snapshot, not a guarantee.
If you want a second opinion on what the file means, paste your own site’s robots.txt into our robots.txt tester and it will tell you, crawler by crawler, who is allowed in and who is kept out.
The Citation Trade-Off Nobody Mentions
There is a cost to blocking AI search crawlers, and Medium writers feel it more than most. The writer who checked Medium’s robots.txt in late October 2026 also ran a small experiment: he searched for his own posts by exact title in an AI assistant. Every one of his five DEV posts came up. None of his six Medium posts did, and for three of them the DEV copy of the same text appeared instead. The lesson was not about robots.txt, which allows the search crawlers on Medium. It was about findability: Medium pages are harder for AI systems to fetch and read, partly because of the Cloudflare challenge, partly because of how the pages are built.
That matters because the same split that protects your training data also shapes your reach. If you want ChatGPT and Perplexity to cite you, publish where the bots are let in and the text is in plain HTML. Medium is not the best place for that. Your own site is. Use Medium for the essay version, the personal version, the version that builds your audience, and use your own site for the version you want cited.
What About Republishing and Canonical Links?
If you publish on your own site first and then repost to Medium, use Medium’s import feature rather than copy-pasting. Import sets a canonical link pointing back to your original, which tells Google and AI search systems that your site is the source. Without it, Medium’s version can outrank yours, and AI citations may point to Medium instead of to you. This is a small step that protects both your search traffic and your authorship signal.
Also remember that blocking a crawler does not remove content already collected. If GPTBot crawled your Medium stories before a block existed, or if your text is already in a training dataset through another route, a robots.txt change cannot undo that. Blocks only prevent future crawling. That is true on every platform, and it is worth saying plainly so nobody expects a settings change to erase the past.
Frequently Asked Questions
Can I block GPTBot on my Medium stories?
Yes, and you do not need to do anything: Medium’s robots.txt already blocks GPTBot for the whole platform. You cannot change that rule, but you also do not need to. What you cannot do is block additional crawlers or allow GPTBot back in. Those decisions belong to Medium.
Does Medium block PerplexityBot?
No. As of the check on 31 October 2026, Medium’s robots.txt allows PerplexityBot, which means Perplexity can crawl and cite your stories. If you do not want Perplexity citations, your only real option is to move the work to a site you control and block PerplexityBot there.
Will unlisting a story stop AI crawlers reading it?
No. Unlisting removes a story from Medium’s own listings and search, but the page is still live on medium.com and still governed by Medium’s robots.txt. Any crawler Medium allows can still fetch it if it finds the link.
Does the Medium paywall hide my story from AI crawlers?
Not reliably. Medium wants paywalled stories to be discovered, so they are generally served to crawlers for indexing. The paywall limits human readers, not crawlers. Do not rely on it as an AI block.
Can I use a custom domain on Medium to control robots.txt?
No. A custom domain changes the address readers see, but Medium still serves the content and Medium’s robots.txt still governs it. There is no per-publication or per-author robots.txt on Medium.
What is the best option if I want full control?
Publish original work on a domain you own, where you write your own robots.txt, and use Medium only for republication with a canonical link. That is the only setup where the decision is actually yours. Our WordPress guide and our Substack guide show how the same decision plays out on two other platforms, and the complete list of AI crawlers explains what each bot actually does.
The bottom line is simple. Medium has already made the big decision for you: no training crawls, yes to AI search citations. If you agree with that split, keep writing there and check the file twice a year. If you do not, the fix is not a setting on Medium. It is a site of your own.
