This afternoon, I received an email from Ahrefs notifying me that my Ghost blog couldn’t be indexed because the robots.txt file was missing. Just last week, the site was indexable and has been for a year, and I haven’t made any changes to my Ghost installation or configuration in over six months.
I’m running Ghost 5.72.0 (self-hosted), and when I checked my site, I confirmed that the robots.txt file is missing across all my Ghost installations. According to Ghost’s documentation, this file should be automatically generated, but in my case, it’s completely absent.
I’ve seen references stating that robots.txt is managed by Ghost itself, yet I can’t find any clear information on where it should be located or how to manually add one if Ghost isn’t generating it.
Here’s what I’ve checked so far:
• Running Ghost 5.109.0 (self-hosted).
• Using Casper on one site and Solo on another.
• robots.txt is missing on both installations.
• Documentation suggests it should be auto-generated, but it’s not.
Questions:
-
Where exactly does Ghost generate/store robots.txt on a self-hosted instance?
-
If it’s missing, is there a way to manually force Ghost to regenerate it?
-
If I need to add it manually, where should I place it within a self-hosted Ghost setup?
Any insights would be greatly appreciated!