I’m running Ghost version 3.14.0, self-hosted on a DigitalOcean droplet via Docker containers (ghost and mysql in separate containers). I’ve been experiencing sudden spikes in CPU usage that brings down the database, as shown in the graphs below:
You can see how the CPU spikes suddenly to 100% even though it usually barely uses 1%. I’ve narrowed down the problem to be caused by the MySQL container, bringing down the db.
Here are the logs from the Docker container:
It repeats this for quite a while before finally succeeding. It seems to be an out of memory error, but I don’t know why the droplet is using up so much memory. It has 1GB RAM, and from the statistics, it’s barely using 30-40% of that, yet the graphs show a consistent 90% RAM usage.
Attached are my free memory and top memory:
Not sure why the graphs show so much memory is used even though I’m only using at beast 40-50% memory. Should I add swap or some other thing although DigitalOcean doesn’t recommend adding swap?
Does anyone have any idea how to fix this issue? The server randomly crashes due to this (luckily, thanks to Docker, it restarts immediately). I can’t pinpoint the exact reason either - running top doesn’t show anything, and I can’t access the server through ssh during the flareup. Thank you!
Have you added any caching in front of your Ghost installation? It’s one of the things we recommend doing:
Hi David, I do have Cloudflare in front of my server that does serve ~70%+ cached documents (according to Cloudflare analytics). The memory issue seems to stem from MySQL, so I’m not sure if there’s any expensive database queries being done on a cron job but I’m not sure how to check that.
Either way, I’m going to set up swapping for the server so we’ll see if the downtime improves.
@chenningg seems I have a similar issue… Were you able to solve it?
I’m having the same issue! Weird… I’ve had 2 spikes in a span of 3 hrs and my site is currently down.
@andrii @dietrich Yes, I was able to solve it by setting up swap space in the droplet. It’s disabled by default: How To Add Swap Space on Ubuntu 20.04 | DigitalOcean
Alternatively, provision a higher memory (RAM) instance.
thanks @chenningg! doing that now. I’m also installing CloudFlare, will that help?
Cloudflare does help provide a layer of caching on your setup, so in theory that means less load on your server (and less reads and writes on your MySQL database). However, it does add a layer of complexity to deal with the certificates and all that. I do recommend setting it up though.
I’m curious why Digital Ocean have an article on how to add swap space in droplets when they say,
placing swap on SSDs can cause issues with hardware degradation over time. Due to this, we do not recommend enabling swap on DigitalOcean or any other provider that uses SSD storage
I guess that whatever hardware degradation occurs must be quite minimal.
Swap is often needed so servers don’t run out of memory when processes spike in memory. The reason DigitalOcean (and other hardware providers) discourage enabling swap is because it does degrade SSDs, and because they own the SSDs, they would naturally discourage it so they can save cost on replacing their hardware. However, since anyone with an above moderate use case would generally enable swapping, they do have an article on it.
@chenningg I haven’t got the chance to put the swap space. Has it solved the spiking on your end?
I’m no longer hosting a Ghost server, but yes, it did help last time.