Markdown vs HTML: A Randomized Controlled Experiment on AI Crawler Traffic

Profound ran a controlled A/B experiment across 381 pages on 6 websites tested whether serving Markdown to AI crawlers increases bot traffic compared to standard HTML. Results showed a marginal directional advantage for Markdown (~16% mean lift, ~1 extra median visit over 3 weeks), driven primarily by high-traffic outlier pages — not statistically significant.

February 26, 2026

What is the Markdown vs HTML Debate for AI Crawlers?

Over the past year, a popular theory has spread across SEO and GEO communities: serve Markdown instead of HTML to AI crawlers, and your content will appear more frequently in ChatGPT, Perplexity, and other AI tools. The logic seemed sound: Markdown is cleaner, more structured, and easier for LLMs to parse. But nearly all of these stories were anecdotal. Profound ran the first rigorous, randomized controlled experiment to find out if the theory holds up.

What This Research Reveals:

Markdown is not a game-changer, at least not yet: Across 381 pages on 6 websites over 3 weeks, Markdown pages received roughly one extra median bot visit. Directionally positive, but not statistically significant.
ChatGPT-User is the dominant AI crawler: It accounted for 73% of all bot traffic in the experiment, far ahead of Meta (20%), OAI-SearchBot (4%), ClaudeBot (2%), and GPTBot (1%).
High-traffic pages benefited most: The ~16% mean lift was driven almost entirely by pages already receiving heavy bot attention. The median page saw virtually no difference.
The experiment was powered to catch big effects: Any lift above 40% would have been detected. None was. If a meaningful advantage exists, it is modest and likely situational.