Site disappeared from Google - robots.txt missing Allow: / directive

My Ghost Pro site (cgstrategylab.com) disappeared from Google search results after making recent content updates.

Issue: My robots.txt file is missing the “Allow: /” directive, blocking Google from crawling my main content pages.

Current robots.txt:

User-agent: *
Sitemap: https://cgstrategylab.com/sitemap.xml
Disallow: /ghost/
Disallow: /email/
Disallow: /members/api/comments/counts/
Disallow: /r/
Disallow: /webmentions/receive/

Needed fix: Add “Allow: /” after “User-agent: *”

Impact: Google Search Console shows sitemap submitted successfully but 0 discovered pages due to robots.txt blocking.

This appears to be a known issue affecting multiple Ghost Pro users. Can Ghost support add the Allow directive to my robots.txt file?

I’ve also emailed support@ghost.org but posting here for faster resolution.

Thanks!

Are you on 6.0, or 5.something?

Updated to latest 6.0 version yesterday.

I looked at the code. Blaming Ghost/ghost/core/core/frontend/public/robots.txt at main · TryGhost/Ghost · GitHub

That file hasn’t change this year, so it’s not a 6.0 bug. My site uses the same file and is indexed fine.

It may be something about your configuration. Do you have published posts? When you visit sitemap-posts.xml, do you see them?

OK, so you submitted your sitemap in Google search console? How long ago? It should be able to pick up your posts.

I just submitted it but it is not able to see any pages.

I suggest giving it a couple hours. GSC updates slooooowwwwly. :slight_smile:

1 Like

Haha okay. I am not that technical when it comes to SEO, etc. But I was trying to get the robot.txt issue which seems like a legit one resolved (based on my chat with Open AI / Claude :p).

I have dropped a note to the support team. Seems like its something that is fixed from back end or by using custom code injection.

Custom code injection to solve this issue needs a Theme change / custom theme apparently.

And I prefer that it gets solved directly from the backend by the Ghost Support team. So waiting for their response.

Hey no problem. If you’d prefer a reply from the support team and/or AI, I’ll go ahead and unlist this (which is probably what I should have done when you said you were a Ghost Pro customer anyway, oops). Have a great one.

No. I dint mean for you to unlist this.
I just wanted to share that I have already written to Ghost Support. Appreciate all the insights you have provided. :slight_smile: