Ghost Admin API and UI return 500 Error if I insert Markdown inline code into a post (consistent replication)

I’ve found a consistent method to cause the Ghost Admin API to fail to return posts, resulting in a 500 error whenever you try to access the drafts section of the admin interface.

I noticed this first when I was fleshing out a blog post and added an inline code example using the Markdown shorthand, like so:

I was talking about adding `<img>` elements

As soon as I typed that out the 500 error appeared and I was unable to access the drafts section (500 page with animated tumbleweed) and the API requests in the browser devtools were returning nothing for drafts (even though I had a lot).

I’ve since been able to replicate this reliably by simply copying the example in the replication steps below, effectively as soon as I type out the Markdown syntax for inline code, followed by the <img> element. The “crash” doesn’t occur if you don’t put <img> / HTML elements inside.

Restarting Ghost doesn’t change anything. The only way I was able to resolve this and get my posts back was to manually delete the bad post via the Admin API, which was a journey in itself as I couldn’t get the ID of the bad post via the Admin API as it returns nothing.

For what it’s worth, I was unable to replicate this locally on an instance that uses SQLite. I’ve only seen this and been able to replicate it on my production instance that uses MySQL. I don’t know if that is relevant but it’s worth mentioning.

I’ve outlined as much as I can above and below but appreciate that you’ll likely need more to help debug this. Let me know what else I can provide to help as I can replicate this reliably.

What’s your URL?

What version of Ghost are you using?

How was Ghost installed and configured?
DigitalOcean one-click install

What Node version, database, OS & browser are you using?

  • Node: v14.16.1
  • MySQL
  • Ubuntu Ubuntu 20.04.2 LTS (Ghost)
  • Chrome 92.0.4515.131

What errors or information do you see in the console?

  • No errors in the Ghost logs after saving a bad post.
  • Error in browser after saving bad post:

What steps could someone else take to reproduce the issue you’re having?

  1. Open drafts and click “New Post”
  2. Give the post any title and move to the body
  3. Type the following
This will crash the Ghost Admin API for posts if I add an `<img>` element with the Markdown code syntax
  1. If 500 Error hasn’t already occurred then force a save
  2. You should be seeing a 500 Error, which remains if you refresh
  3. Browse and post API endpoints now return nothing

Temporary resolution
Given that this completely disables the ability to add new draft posts or edit existing ones, I had a pretty immediate need to get the blog functional again. I was able to do this via the following method:

  1. Manually log into the MySQL database
  2. Get the latest post ID
select id, title from posts order by updated_at desc limit 1;
  1. Use the Admin API client to delete the bad post
    id: "the-bad-post-id",
  1. Refresh Ghost and all should be good

The only method I could get the bad post ID was via the database as the Admin API returns nothing when calling browse / posts after the bad post has been added, so it’s impossible to get the ID from the API, nor edit the post via the API.

SQL data for the bad post

{"version":"0.3.1","atoms":[],"cards":[],"markups":[["code"]],"sections":[[1,"p",[[0,[],0,"This will crash the Ghost Admin API for posts if I add an "],[0,[0],1,"<img>"],[0,[],0," element with the Markdown code syntax"]]]],"ghostVersion":"4.0"}


<p>This will crash the Ghost Admin API for posts if I add an <code>&lt;img&gt;</code> element with the Markdown code syntax</p>


This will crash the Ghost Admin API for posts if I add an <img> element with the
Markdown code syntax

Is there anything else I can provide to help debug this? I’d like to avoid it happening again as I’m inevitably going to hit it given the amount of code I’ll be referencing in my blog posts.

Happy to submit a ticket on GitHub too if that’s useful, though I get the impression the preference is to start investigation here first.

I’m not able to reproduce this with MariaDB 10.4.12 in both production and development :confused:

saving that and previewing the post works flawlessly

It’s really weird that the Admin API is returning nothing when your browse posts. Are you getting a 200 response? Also, are there any relevant (error) logs from ghost log or ghost log -e?

Thanks for responding.

So no errors in the logs. I double-checked this a few times when I was originally debugging and also switched the logging to various levels — nothing appears between writing out the bad content, the crashing, and then manually resetting it all.

And yes, 200 Response on the API calls but absolutely no content inside of them — not even an empty JSON array, which explains the browser error re: unexpected JSON input.

I’m wondering if this is related to MySQL, given that it works for you on MariaDB and that it also works for me on a local SQLite instance.

I can replicate it very easily so happy to attempt other debugging steps. I was hoping there’s perhaps something obviously bad in the mobiledoc payload I provided that would explain it :man_shrugging:

Do you have any security setup on the server or on the CDN, locally, or even a browser extension, that may be wrongly attempting to sanitise the API response?

Is it the same issue in all browsers?

I am using Cloudflare on that domain but it isn’t caching anything on the admin paths.

Browser extensions and browser versions aren’t an issue as I can replicate the blank response via a direct API call in Node to request the drafts. It’s only responses containing the bad post that return empty, all others seem to remain good and return data correctly (eg. API call for a single good post).

I will set up a temporary DigitalOcean One-Click instance later to see if I can replicate it again there, to hopefully save us all some time. If I can, then I’m happy to give a Ghost staff member access to tinker before throwing it away.

Do you have Cloudflare’s Web Application Firewall enabled for your domain? Their caching is separate and likely unrelated but it’s worth checking the response headers to make sure there’s no indication of caching.