Optimize your site for AI search - Ghost Changelog

For decades, search engines like Google have been a major source of traffic for publishers. Ghost has long included built-in SEO tools to help your site appear in search results, automatically handling structured metadata and sitemaps.

More recently, people have increasingly started using new AI tools such as ChatGPT and Claude to answer their questions and discover new sources of information. Large Language Models (LLMs) search the web and reference sources they use to answer a user's queries.

As you might expect, new practices have emerged for website owners who want their site to be more likely to be cited and referenced in AI searches - sometimes referred to as GEO, or Generative Engine Optimization.

Ghost now includes built-in GEO tools that generate the files AI systems expect and provide Markdown versions of your public content, which you can activate in your admin panel under Settings → Meta data

When enabled, Ghost will output a new file at the root of your domain called llms.txt - and serve Markdown versions of your posts when .md is appended to the URL.

For example:

Much like SEO, these new GEO tools help AI agents discover, read and understand your content more easily and efficiently, making it more likely to be referenced when responding to user queries.

Only publicly available content is included. Members-only and paid posts are listed with their public title and excerpt, but their protected content is not. Private sites don’t serve these files at all.

We understand that not everyone wants to engage with AI, so this feature is off by default for existing sites. If you'd like to enable it, simply toggle it on in Settings. Otherwise, there's nothing you need to do; it will remain switched off.


Ghost(Pro) users can log in and start enjoying all of this right away! If you're a developer, self-hosting Ghost, you'll need to update to the latest version to get access to everything that's new.

8 Likes

This can be great with semantic search adopted with the default Ghost search app :slight_smile:

Thanks for building this. Keep it rocking!

1 Like

Hi,

I think this is a great feature, though I’m a bit concerned too.

LLM do ingest our content, and added value, and keep their audiences on their tools.

Thus, a decrease in the discoverability of our websites / newsletters.

I have of course a registration / pay wall, but keep most of the content open, to recruit subscribers. Most of LLM / AI tools will use my niche content but won’t redirect traffic.

Hence a question : wouldn’t be nice as well to be able to edit robots.txt (I’m on Ghost Pro) to prevent AI Bots from scraping / stealing the content ?

Would that be a possible feature to counterbalance this one ?

Great job done though, Ghost team !

3 Likes

Much thanks for keeping this off by default as I wont be using it. I have the bots attempt to be blocked in cloudflare, but that cant block everything when the scraping comes in from all over.

wouldn’t be nice as well to be able to edit robots.txt (I’m on Ghost Pro) to prevent AI Bots from scraping / stealing the content?

See, Robots.txt configuration

I am curious how LLMs disallowed in the Robots.txt file, would handle this AI Search feature if it was enabled.

I wouldn’t rely on bots to respect robots.txt . Rather, I suggest you block requests from known or suspected unwelcome user-agents, such as those listed at:
awesome-lists/Lists/suspicious_http_user_agents_list.csv at main · mthcht/awesome-lists · GitHub

The above assumes you have a reverse-proxy setup; if not, I suggest Nginx with the following relevant blocking configuration:
Banning Bots with nginx maps - eigenmagic

2 Likes

I have updated my self-hosted site to the latest release of Ghost (6.53.0) but this llms feature is not available on the settings → Meta data panel. Is it only available via some beta release channel?

Thanks for the hint.
Unfortunately I run on Ghost Pro, so I don’t have the hand on what you suggest.
I’ll wait for an update :smiling_face:

Simon

Is there any risk of SEO cannibalization by having both the standard html and the md versions live?

Just updated my self hosted ghost, and noticed that this option was turned ON by default, not off as the article claims. Not a problem for me, as I wanted it turned on, but worth noting.

2 Likes

interesting - was off for me on both my Ghost(Pro) and self hosted sites. :face_with_monocle:

Never mind, I tried updating again to the latest minor revision and discovered that the previous update had not completely succeeded, corrupting my configuration file. Once the config file was fixed, and the new update applied, the llms feature option now appears in the settings.

1 Like

Hi, this is from another topic thread I just reviewed, Ghost vs. Slack

Using Ghost by forum user @dearestleader on July 7.

Are there suggestions for addressing this, besides the anti-bot options mentioned in this thread?

btw, I att. file of screenshot here as I was blocked from copying text from one thread, to paste in a different thread: is there a way to enable that?

For me it was causing markdown instead of HTML (for few sites):

(the configuration is returned back to off - so now is working fine)

Sounds like cache poisoning.

I see you’re using Cloudflare. With llmx.txt enabled Ghost handles requests with the Accept: text/markdown or Accept: text/plain headers differently – by serving the markdown.

The straightforward fix would be to exclude all requests from caching when Accept includes these two keys.

If you do want to cache markdown separately there are a few ways. At Magic Pages I ended up with a Snippet. A worker would also work (but is far more expensive at our scale).

2 Likes