Website goes down when the system needs restart

I have Ghost running on the Digital Ocean Droplet. The operating system is: Ubuntu 22.04.1 LTS
My droplet size is: 1 vCPU / 1GB / 25GB Disk.

The website is down each time I have this message in my SSH console:

93 updates can be applied immediately.
2 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable

*** System restart required ***

Also, the memory and CPU usage go up to 100%. All back to normal when I restart the system by:
sudo reboot

Are there any Linux commands to restart the system automatically when the system needs update?
Maybe there is a way to prevent server from falling when it needs updates at all?

Your Droplet is probably too small, i.e., insufficient memory.

What may help alleviate this is installing unattended upgrades. This will handle Ubuntu updates only, but you can set this to perform routine restarts.

sudo apt install unattended-upgrades --yes
sudo dpkg-reconfigure unattended-upgrades
sudo cp /etc/apt/apt.conf.d/50unattended-upgrades /etc/apt/apt.conf.d/50unattended-upgrades.bak
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

Then uncomment the relevant sections, and set a suitable time for restarting.

In addition, it’s useful to receive email notifications, so I also configure Postfix with Mailgun.

As ever, take a snapshot before making changes.

1 Like

@mjw, thank you! You are right about the small droplet. I guess this is the issue. Today I had a CPU usage spike above 100%. It lasted for 2.5 hrs and all this time my website was down. Those spikes are pretty frequent, as you can see on the picture.

How can I mitigate those spikes?
Or at least how powerful do I need the droplet to be to be able to handle them?

There are a few options. You could increase the size of the Droplet, or migrate to Hetzner, for example, that performs better in my experience and costs less so you can go bigger.

However, I’d first check the logs to determine what’s going on. You may find that bots are scanning for vulnerabilities, and this is essentially bringing the server down. Mitigate this by using a cloud firewall, and installing fail2ban on the server.

Unattended updates will also keep the server up-to-date, and can be used for scheduling reboots.

Use Cloudflare to reduce requests on your server.

Finally, consider optimizing Nginx and MySQL.

1 Like

@mjw, thank you for the extensive response.

Did optimizing Nginx and MySQL helped you much?
Would it have an impact for the CPU load, RAM and internal resources or website pagespeed score as well?

I’m thinking about it as a last resort, as I know nothing about those. Also, it might put the website offline for a while while I’m doing the maintenance.

Not as much as the other steps. I discovered a pattern, and it was clear that bots were attempting to access the server via SSH. Fail2ban, preventing password-based access, and using a non-standard port have eliminated this problem.

Next up, using Cloudflare helps significantly by reducing direct hits on the server; saving multiple GB of data transfer, and an extra layer of DDoS protection.

I still get occasional memory warnings when upgrading Ghost, but typically this occurs because a reboot is required following apt upgrade.

My PageSpeed score for desktop is around 95 on a membership site using 1 vCPU and 2 GB memory, i.e., €4.53 per month including back-ups and snapshots. Using Linode this cost $15 monthly. I can’t grumble.

Your issues are most likely related to having only 1 GB memory.

1 Like

@mjw that’s very informative, thank you!

If the password-based access is turned off bots can’t really even try to access the server, right? Because they don’t have an SSH key to attempt to connect, thus the request is not sent at all.

If I have a connection by an SSH key and disabled password-based access there is no need for me to change the port, or there is?

Also, fail2ban helps to blacklist IPs that make too many password requests. There is no password field exposed in Ghost. Only email, which blocks any requests for 10 minutes (don’t remember exactly) after several invalid email inputs. Thus, I’m thinking if installing fail2ban is relevant.

Am I correct here or am I missing something?

The bots don’t know this, and are simply trying for known exploits (with sshd). Even unsuccessful scans can overload the server.

If you use a non-standard port, and port 22 is therefore closed, scans may be reduced. It’s just an additional measure, and isn’t an inconvenience.

For http, Ghost has inbuilt measures to prevent multiple attempts logging into the admin console, which is password based.

1 Like

@mjw you were right, since I upgraded to 2GB of memory the website runs without any downtime.