AI Search
How to Block Meta AI Crawlers (Meta-ExternalAgent) With Robots.txt
Block Meta's AI training crawler with robots.txt: the exact tokens for Meta-ExternalAgent, Meta-WebIndexer, and Meta-ExternalFetcher — and how to opt out without breaking Facebook link previews.
Rahul Sharma
10 min
