Tried to update Ghost and everything is broken :(

[lobo]

Hello!!

I had my blog with an older version of Ghost (3.x.x) and I wanted to update, so I followed this guide step by step. First I updated to the latest minor version of the 3.x.x and everything worked fine there, I could see the version updated on my admin panel.
Now, when updating to the latest version (5.x.x) everything broke
There are no errors when running ghost doctor:

But Ghost is never able to start:

If I check the logs it doesn’t give me useful information (I think):

lobo@blog:/var/www/ghost$ cat /home/lobo/.ghost/logs/ghost-cli-debug-2024-05-14T17_35_03_935Z.log
Debug Information:
    OS: Ubuntu, v18.04.4 LTS
    Node Version: v20.13.1
    Ghost Version: 5.82.9
    Ghost-CLI Version: 1.26.0
    Environment: production
    Command: 'ghost start'
Message: Could not communicate with Ghost
Suggestion: journalctl -u ghost_jvlobo-com -n 50
Stack: Error: Could not communicate with Ghost
    at Server.<anonymous> (/usr/local/lib/node_modules/ghost-cli/lib/utils/port-polling.js:91:20)
    at Object.onceWrapper (node:events:633:28)
    at Server.emit (node:events:519:28)
    at emitCloseNT (node:net:2321:8)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)

I have the backup as the guide explains but if I tried ghost update --rollback it doesn’t work either:

I don’t know what else to do… my website is totally down, throwing a 502 error.

Any help is highly appreciated.

Thanks!

[mjw]

I’m sorry to hear about your troubles. Did you upgrade to the terminal release of Ghost v3, and then move straight to v5? Or did you upgrade v3 to v4 first? This is a necessary step.

Likewise, it would seem that you are running MariaDB not MySQL.

Did you take a snapshot of your VPS first?

[lobo]

Thanks for the quick reply.

I followed the documentation 100% step by step.
I checked the database as explained and I saw MySQL higher than version 8. Where do you see that I may be running MariaDB?
And for updating I did as the documentation also explains, and it went from v3 to v5
I was just re-reading in case I missed something but even in the example it says to go from v3 to v5:

Before updating to a new major version, you must update your site to the latest minor version, e.g if you are on 3.23.0 you need to be on 3.42.9 before updating to v5.

I didn’t take a snapshot of my VPS, nope…

[mjw]

Okay, the upgrade steps are good for v3 (terminal) to v5 … I think earlier needed more steps, so sorry for the confusion.

The reason I mentioned MariaDB is the warning. I don’t recall this is displayed if you’re running MySQL 8. So, what database are you running?

[lobo]

Running MySQL, I thought ver 14 but latest version is 8.4, so I’m running MySQL 5.7.42 (Distrib is the actual version it seems)

But ghost doctor doesn’t complain about the database, it should have if I was running MariaDB, right?

[mjw]

You need to be on MySQL 8, so I would look to upgrade this. However, before doing anything, run mysqldump.

[lobo]

I see my error there… the documentation explicitly says to have MySQL 8 and I got confused with the “Ver 14.14” :(

I’m going to update to MySQL 8 then… what should I do after that? just continue with the “normal” installation of the newest version of Ghost?

Thanks!

[mjw]

Please refer to this guide.

Supported databases in production for self-hosting Ghost - Ghost Developers

However, before proceeding, take a backup of the database using mysqldump.

[lobo]

Ok, thanks.

That is done:

lobo@blog:/var/www/ghost$ mysql --version
mysql  Ver 8.0.36-0ubuntu0.20.04.1 for Linux on x86_64 ((Ubuntu))

Now, if I try to start my Ghost instance I get an error:

I’m not sure what’s the best way to proceed here.
ghost doctor says everything is fine.

Thanks!

[mjw]

It’s possible that the systemd config is broken. Try ghost setup systemd.

[lobo]

This is what I get:

lobo@blog:/var/www/ghost$ ghost setup systemd

Love open source? We’re hiring JavaScript Engineers to work on Ghost full-time.
https://careers.ghost.org



Systemd service has already been set up. Skipping Systemd setup
ℹ Setting up Systemd [skipped]

[mjw]

Please post the output from journalctl -u <your-user-id> -n 50.

[lobo]

I’m not sure if I’m doing it properly, but the output doesn’t have any information:

Could also be a solution to do a fresh install? Would I be able to get everything as it was? (since I have the backup I did with ghost backup before trying to update to v5)

[mjw]

A couple of observations.

1. You need to use a user-id when executing journalctl:

journalctl -u ghost_jvlobo-com -n 50

2. But, the user-id you’re logged in with is lobo and the Ghost user is ghost_jvlobo-com.

Why is this?

[lobo]

I have no idea, I installed Ghost a few years ago and I think everything was done with lobo, I don’t seem to have any credentials for a ghost user, so I’m not too sure

At this point everything seems to be getting messier and messier, I only wanted to upgrade my already working blog
What do you think is my best option at this point?

Thanks for your kind help

[mjw]

Please execute the following:

sudo journalctl -u ghost_jvlobo-com -n 50

Not all appears lost; let’s try to get Ghost started.

[lobo]

Ok ok… I see good stuff now.
It seems like there is a problem with my custom theme:

-- Logs begin at Wed 2024-03-27 23:55:05 UTC, end at Wed 2024-05-15 19:04:32 UTC. --
May 15 19:04:32 blog node[27531]: Details:
May 15 19:04:32 blog node[27531]:     checkedVersion: 5.x
May 15 19:04:32 blog node[27531]:     name:           titan
May 15 19:04:32 blog node[27531]:     path:           /var/www/ghost/content/themes/titan
May 15 19:04:32 blog node[27531]:     version:        0.0.2
May 15 19:04:32 blog node[27531]:     errors:
May 15 19:04:32 blog node[27531]:       -
May 15 19:04:32 blog node[27531]:         fatal:     false
May 15 19:04:32 blog node[27531]:         level:     error
May 15 19:04:32 blog node[27531]:         rule:      The <code>.kg-width-wide</code> CSS class is required to appear styled in your theme
May 15 19:04:32 blog node[27531]:         details:   The <code>.kg-width-wide</code> CSS class is required otherwise wide images will appear unstyled.Find out more about required theme changes for the Koenig editor <a href="https://ghost.org/docs/themes/content/" target=_blank>here</a>.
May 15 19:04:32 blog node[27531]:         regex:
May 15 19:04:32 blog node[27531]:         className: .kg-width-wide
May 15 19:04:32 blog node[27531]:         css:       true
May 15 19:04:32 blog node[27531]:         failures:
May 15 19:04:32 blog node[27531]:           -
May 15 19:04:32 blog node[27531]:             ref: styles
May 15 19:04:32 blog node[27531]:         code:      GS050-CSS-KGWW
May 15 19:04:32 blog node[27531]:       -
May 15 19:04:32 blog node[27531]:         fatal:     false
May 15 19:04:32 blog node[27531]:         level:     error
May 15 19:04:32 blog node[27531]:         rule:      The <code>.kg-width-full</code> CSS class is required to appear styled in your theme
May 15 19:04:32 blog node[27531]:         details:   The <code>.kg-width-full</code> CSS class is required otherwise wide images will appear unstyled.Find out more about required theme changes for the Koenig editor <a href="https://ghost.org/docs/themes/content/" target=_blank>here</a>.
May 15 19:04:32 blog node[27531]:         regex:
May 15 19:04:32 blog node[27531]:         className: .kg-width-full
May 15 19:04:32 blog node[27531]:         css:       true
May 15 19:04:32 blog node[27531]:         failures:
May 15 19:04:32 blog node[27531]:           -
May 15 19:04:32 blog node[27531]:             ref: styles
May 15 19:04:32 blog node[27531]:         code:      GS050-CSS-KGWF
May 15 19:04:32 blog node[27531]:       -
May 15 19:04:32 blog node[27531]:         fatal:    false
May 15 19:04:32 blog node[27531]:         level:    error
May 15 19:04:32 blog node[27531]:         rule:     Not all page features are being used
May 15 19:04:32 blog node[27531]:         details:  <b>This error only applies to pages created with the Beta editor.</b> Some page features used by Ghost via the <code>{{@page}}</code> global are not implemented in this theme.&nbsp;Find more information about the <code>{{@page}}</code> global <a href="https://ghost.org/docs/themes/helpers/page/" target=_blank>here</a>.
May 15 19:04:32 blog node[27531]:         failures:
May 15 19:04:32 blog node[27531]:           -
May 15 19:04:32 blog node[27531]:             ref:     page.hbs
May 15 19:04:32 blog node[27531]:             message: @page.show_title_and_feature_image is not used
May 15 19:04:32 blog node[27531]:             rule:    GS110-NO-MISSING-PAGE-BUILDER-USAGE
May 15 19:04:32 blog node[27531]:         code:     GS110-NO-MISSING-PAGE-BUILDER-USAGE
May 15 19:04:32 blog node[27531]:     warnings:
May 15 19:04:32 blog node[27531]:       (empty array)
May 15 19:04:32 blog node[27531]: ----------------------------------------
May 15 19:04:32 blog node[27531]: ThemeValidationError: The currently active theme "titan" has errors, but will still work.
May 15 19:04:32 blog node[27531]:     at Object.getThemeValidationError (/var/www/ghost/versions/5.82.9/core/server/services/themes/validate.js:142:12)
May 15 19:04:32 blog node[27531]:     at module.exports.loadAndActivate (/var/www/ghost/versions/5.82.9/core/server/services/themes/activate.js:30:35)
May 15 19:04:32 blog node[27531]:     at async initServicesForFrontend (/var/www/ghost/versions/5.82.9/core/boot.js:192:5)
May 15 19:04:32 blog node[27531]:     at async bootGhost (/var/www/ghost/versions/5.82.9/core/boot.js:547:31)
May 15 19:04:32 blog node[27531]: 

I’ll try to sort that out and update the theme, or can I somehow set the theme to the default one and try to start ghost?

[mjw]

Run the same command with -n 100 or -n 200 etc., to view more lines, or simply sudo journalctl -u ghost_jvlobo-com to scroll backward and forward. Look for fatal errors that stop Ghost starting.

[lobo]

Apart from the theme issue, this is what I can see:

May 15 19:35:11 blog node[38344]: [2024-05-15 19:35:11] INFO Running milestone emails job on Sat May 18 2024 19:35:11 GMT+0000 (Coordinated Unive>
May 15 19:35:12 blog node[38319]: /usr/lib/node_modules/ghost-cli/lib/process-manager.js:46
May 15 19:35:12 blog node[38319]:         throw error;
May 15 19:35:12 blog node[38319]:         ^
May 15 19:35:12 blog node[38344]: [2024-05-15 19:35:12] WARN Ghost is shutting down
May 15 19:35:12 blog node[38319]: { message: undefined }
May 15 19:35:12 blog node[38319]: Node.js v20.13.1
May 15 19:35:12 blog systemd[1]: ghost_jvlobo-com.service: Main process exited, code=exited, status=1/FAILURE
May 15 19:35:12 blog node[38344]: [2024-05-15 19:35:12] WARN Ghost has shut down
May 15 19:35:12 blog node[38344]: [2024-05-15 19:35:12] WARN Your site is now offline
May 15 19:35:12 blog node[38344]: [2024-05-15 19:35:12] WARN Ghost was running for a few seconds
May 15 19:35:12 blog node[38344]: node:internal/errors:541
May 15 19:35:12 blog node[38344]:       throw error;
May 15 19:35:12 blog node[38344]:       ^
May 15 19:35:12 blog node[38344]: TypeError [ERR_INVALID_ARG_TYPE]: The "code" argument must be of type number. Received type string ('SIGTERM')
May 15 19:35:12 blog node[38344]:     at process.set [as exitCode] (node:internal/bootstrap/node:123:9)
May 15 19:35:12 blog node[38344]:     at process.exit (node:internal/process/per_thread:180:24)
May 15 19:35:12 blog node[38344]:     at Timeout._onTimeout (/var/www/ghost/versions/5.82.9/core/server/GhostServer.js:152:25)
May 15 19:35:12 blog node[38344]:     at listOnTimeout (node:internal/timers:573:17)
May 15 19:35:12 blog node[38344]:     at process.processTimers (node:internal/timers:514:7) {
May 15 19:35:12 blog node[38344]:   code: 'ERR_INVALID_ARG_TYPE'
May 15 19:35:12 blog node[38344]: }
May 15 19:35:12 blog node[38344]: Node.js v20.13.1
May 15 19:35:12 blog systemd[1]: ghost_jvlobo-com.service: Failed with result 'exit-code'.
May 15 19:35:12 blog systemd[1]: ghost_jvlobo-com.service: Scheduled restart job, restart counter is at 538.
May 15 19:35:12 blog systemd[1]: Stopped Ghost systemd service for blog: jvlobo-com.

Is this of any help?

[mjw]

You’re running Node 20 when Node 18 is recommended. I think I’d roll back to 18. How did you install Node, nvm? Or using https://deb.nodesource.com/setup_18.x?