After upgrading to Ghost 4.0.1 from the latest version I’m hitting 502 errors with my website. Most of the time it shows a 502 and sometimes I can see a " We’ll be right back." screen. Even sometimes it shows a skeleton site without images (only about 10% of the time).
$ ghost log -f
+ sudo systemctl is-active ghost_danielesalatti-com
[2021-03-18 00:52:15] INFO Your site is now available on https://danielesalatti.com/
[2021-03-18 00:52:15] INFO Ctrl+C to shut down
[2021-03-18 00:52:15] INFO Ghost server started in 3.17s
[2021-03-18 00:52:16] INFO Database is in a ready state.
[2021-03-18 00:52:16] INFO Ghost database ready in 4.952s
[2021-03-18 00:52:25] INFO Ghost booted in 13.706s
[2021-03-18 00:52:26] WARN Ghost is shutting down
[2021-03-18 00:52:26] WARN Ghost has shut down
[2021-03-18 00:52:26] WARN Your site is now offline
[2021-03-18 00:52:26] WARN Ghost was running for a few seconds
[2021-03-18 00:52:26] WARN Ghost is shutting down
[2021-03-18 00:52:26] WARN Ghost has shut down
[2021-03-18 00:52:26] WARN Your site is now offline
[2021-03-18 00:52:26] WARN Ghost was running for a few seconds
[2021-03-18 00:52:32] INFO Ghost is running in production...
[2021-03-18 00:52:32] INFO Your site is now available on https://danielesalatti.com/
[2021-03-18 00:52:32] INFO Ctrl+C to shut down
[2021-03-18 00:52:32] INFO Ghost server started in 3.177s
[2021-03-18 00:52:35] INFO Database is in a ready state.
[2021-03-18 00:52:35] INFO Ghost database ready in 6.414s
[2021-03-18 00:52:45] WARN Ghost is shutting down
[2021-03-18 00:52:45] WARN Ghost has shut down
[2021-03-18 00:52:45] WARN Your site is now offline
[2021-03-18 00:52:45] WARN Ghost was running for a few seconds
[2021-03-18 00:52:45] WARN Ghost is shutting down
[2021-03-18 00:52:45] WARN Ghost has shut down
[2021-03-18 00:52:45] WARN Your site is now offline
[2021-03-18 00:52:45] WARN Ghost was running for a few seconds
[2021-03-18 00:52:51] INFO Ghost is running in production...
[2021-03-18 00:52:51] INFO Your site is now available on https://danielesalatti.com/
[2021-03-18 00:52:51] INFO Ctrl+C to shut down
[2021-03-18 00:52:51] INFO Ghost server started in 3.162s
[2021-03-18 00:52:54] INFO Database is in a ready state.
[2021-03-18 00:52:54] INFO Ghost database ready in 6.14s
[2021-03-18 00:53:04] INFO Ghost booted in 15.418s
The above is from running “ghost start”.
journalctl (relevant bits):
Mar 18 01:06:41 web-01 systemd[1]: ghost_danielesalatti-com.service: Main process exited, code=exited, status=1/FAILURE
Mar 18 01:06:41 web-01 node[190778]: [2021-03-18 01:06:41] WARN Ghost was running for a few seconds
Mar 18 01:06:41 web-01 node[190778]: [2021-03-18 01:06:41] WARN Ghost is shutting down
Mar 18 01:06:41 web-01 node[190778]: [2021-03-18 01:06:41] WARN Ghost has shut down
Mar 18 01:06:42 web-01 node[190778]: [2021-03-18 01:06:41] WARN Your site is now offline
Mar 18 01:06:42 web-01 node[190778]: [2021-03-18 01:06:42] WARN Ghost was running for a few seconds
Mar 18 01:06:42 web-01 systemd[1]: ghost_danielesalatti-com.service: Failed with result 'exit-code'.
Mar 18 01:06:42 web-01 systemd[1]: ghost_danielesalatti-com.service: Scheduled restart job, restart counter is at 27.
Mar 18 01:06:42 web-01 systemd[1]: Stopped Ghost systemd service for blog: danielesalatti-com.
Mar 18 01:06:42 web-01 systemd[1]: Started Ghost systemd service for blog: danielesalatti-com.
Mar 18 01:06:44 web-01 node[190815]: - Inspecting operating system
Mar 18 01:06:48 web-01 node[190871]: [2021-03-18 01:06:48] INFO Ghost is running in production...
Mar 18 01:06:48 web-01 node[190871]: [2021-03-18 01:06:48] INFO Your site is now available on https://danielesalatti.com/
Mar 18 01:06:48 web-01 node[190871]: [2021-03-18 01:06:48] INFO Ctrl+C to shut down
Mar 18 01:06:48 web-01 node[190871]: [2021-03-18 01:06:48] INFO Ghost server started in 3.298s
Mar 18 01:06:50 web-01 node[190871]: [2021-03-18 01:06:50] INFO Database is in a ready state.
Mar 18 01:06:50 web-01 node[190871]: [2021-03-18 01:06:50] INFO Ghost database ready in 5.291s
Mar 18 01:06:59 web-01 node[190871]: [2021-03-18 01:06:59] INFO Ghost booted in 13.9s
Mar 18 01:07:00 web-01 node[190815]: /home/ubuntu/.nvm/versions/node/v12.18.3/lib/node_modules/ghost-cli/lib/process-manager.js:46
Mar 18 01:07:00 web-01 node[190815]: throw error;
Mar 18 01:07:00 web-01 node[190815]: ^
Mar 18 01:07:00 web-01 node[190871]: [2021-03-18 01:07:00] WARN Ghost is shutting down
Mar 18 01:07:00 web-01 node[190815]: { message: undefined }
Mar 18 01:07:00 web-01 node[190871]: [2021-03-18 01:07:00] WARN Ghost has shut down
Mar 18 01:07:00 web-01 systemd[1]: ghost_danielesalatti-com.service: Main process exited, code=exited, status=1/FAILURE
Interesting because that is not the version of nodejs that I have set as default with nvm. Switching to that one and trying to restart causes ghost to give an error (was not installed with this version, run ghost update v4.0.1 --force).
Uninstalled ghost-cli, switched back to LTS (node v14) and trying again.
@DanieleSalatti I install node v14.15.4, then run ghost restart, but some error occur:
One or more errors occurred.
1) Checking system Node.js version
Message: The version of Node.js you are using is not supported.
Supported: ^8.16.0 || ^10.13.0 || ^12.10.0
Installed: 14.15.4
See https://ghost.org/faq/node-versions/ for more information
2) Checking binary dependencies
Message: The installed node version has changed since Ghost was installed.
Help: Run ghost update 4.0.1 --force to re-install binary dependencies.
Debug Information:
OS: CentOS Linux, v7
Node Version: v14.15.4
Ghost Version: 4.0.1
Ghost-CLI Version: 1.13.1
Environment: production
Command: 'ghost doctor'
Try running ghost doctor to check your system for known issues.
You can always refer to https://ghost.org/docs/api/ghost-cli/ for troubleshooting.
It seems doesn’t support v14.15.4. How you swap the node version?
From what I can tell, everyone affected is on Node v14, and the problem is not with Node v14 itself but rather with how Node has been updated meaning Ghost is getting two different node versions.
We know the CLI is not good at catching and making these issues visible right now and it’s something we’re looking into to try to help.
If you are updating Node, it’s a really good idea to do this separately to updating Ghost to 4.0.
My recommendation for anyone updating is to do the following:
ghost version to get your current Ghost version
update node
run ghost update [ghost version] --force to do a reinstall of the current Ghost version
check everything is ok
now run ghost update to update Ghost itself
If you’ve already done the update and are running into problems, it will be easier to help if everyone starts their own threads.
Please share the normal debug info and also:
did you just update node? If so how?
output from which node
output from node --version
your systemd file
Important, take care: The output from running node --version with the full path from Exec Start in systemd, so if your ExecStart says ExecStart=/usr/bin/node /usr/bin/ghost run then share the output from running /usr/bin/node --version`
This should be a good start.
This thread was started by @bop and so let’s wait for more info from them.
Yep, I updated this thread instead of creating a new one with the assumption that the problem might have been the same. Hopefully it helps @bop and others.
@halfrost sorry, it was late so I ended up going to bed I see your issue was solved by following Hannah’s instructions in the other thread though
It doesn’t matter, we all have jet lag, and each of us has our own work and life. I don’t mind if the reply is late. I wish you a good dream. Good night~ Thank you again~
Hannah, I appreciate a lot your thorough feedback and response.
I updated node through the nodesource setup listed on the main DigitalOcean tutorial. I ran the same commands listed on the tutorial. I updated node before updating Ghost to the latest version.
This is the output of which node:
$ which node
/usr/bin/node
This is the output of node --version:
$ node --version
v14.16.0
This is my systemd file
[Unit]
Description=Ghost systemd service for blog
Documentation=https://docs.ghost.org
[Service]
Type=simple
WorkingDirectory=/var/www/ghost
User=999
Environment="NODE_ENV=production"
ExecStart=/usr/bin/node /usr/local/bin/ghost run
Restart=always
[Install]
WantedBy=multi-user.target
When I run ghost doctor everything is green.
If I ran ghost start and then ghost log -f this is what I see:
[2021-03-18 19:36:51] INFO Ghost is running in production...
[2021-03-18 19:36:51] INFO Your site is now available on ...
[2021-03-18 19:36:51] INFO Ctrl+C to shut down
[2021-03-18 19:36:51] INFO Ghost server started in 1.323s
[2021-03-18 19:36:52] INFO Database is in a ready state.
[2021-03-18 19:36:52] INFO Ghost database ready in 1.875s
[2021-03-18 19:36:54] INFO Ghost booted in 4.572s
[2021-03-18 19:36:55] WARN
NAME: HelperWarning
CODE: SLOW_GET_HELPER
MESSAGE: {{#get}} helper took 234ms to complete
level: normal
ERROR DETAILS:
{"api":"v4.postsPublic.browse","apiOptions":{"filter":"tag:hash-personal-work"},"returnedRows":1}
[2021-03-18 19:36:55] WARN
NAME: HelperWarning
CODE: SLOW_GET_HELPER
MESSAGE: {{#get}} helper took 239ms to complete
level: normal
ERROR DETAILS:
{"api":"v4.postsPublic.browse","apiOptions":{"filter":"tag:hash-client-work"},"returnedRows":4}
[2021-03-18 19:36:55] ERROR "GET /" 503 278ms
NAME: MaintenanceError
MESSAGE: Site is starting up, please wait a moment then retry.
level: normal
MaintenanceError: Site is starting up, please wait a moment then retry.
at new MaintenanceError (/var/www/ghost/versions/4.0.1/node_modules/@tryghost/errors/node_modules/ghost-ignition/lib/errors/index.js:162:23)
at maintenance (/var/www/ghost/versions/4.0.1/core/server/web/shared/middlewares/maintenance.js:14:21)
at Layer.handle [as handle_request] (/var/www/ghost/versions/4.0.1/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (/var/www/ghost/versions/4.0.1/node_modules/express/lib/router/index.js:317:13)
at /var/www/ghost/versions/4.0.1/node_modules/express/lib/router/index.js:284:7
at Function.process_params (/var/www/ghost/versions/4.0.1/node_modules/express/lib/router/index.js:335:12)
at next (/var/www/ghost/versions/4.0.1/node_modules/express/lib/router/index.js:275:10)
at servePublicFileMiddleware (/var/www/ghost/versions/4.0.1/core/server/web/site/middleware/serve-public-file.js:75:20)
at Layer.handle [as handle_request] (/var/www/ghost/versions/4.0.1/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (/var/www/ghost/versions/4.0.1/node_modules/express/lib/router/index.js:317:13)
at /var/www/ghost/versions/4.0.1/node_modules/express/lib/router/index.js:284:7
at Function.process_params (/var/www/ghost/versions/4.0.1/node_modules/express/lib/router/index.js:335:12)
at next (/var/www/ghost/versions/4.0.1/node_modules/express/lib/router/index.js:275:10)
at SendStream.error (/var/www/ghost/versions/4.0.1/node_modules/serve-static/index.js:121:7)
at SendStream.emit (events.js:315:20)
at SendStream.EventEmitter.emit (domain.js:467:12)
[2021-03-18 19:36:55] INFO "GET /assets/built/main.css?v=7b7730315d" 200 11ms
[2021-03-18 19:36:55] INFO "GET /assets/built/bundle.js?v=7b7730315d" 200 14ms
[2021-03-18 19:36:55] WARN Ghost is shutting down
[2021-03-18 19:36:55] WARN Ghost has shut down
[2021-03-18 19:36:55] WARN Your site is now offline
[2021-03-18 19:36:55] WARN Ghost was running for a few seconds
[2021-03-18 19:36:55] INFO "GET /assets/built/main.css?v=7b7730315d" 200 11ms
[2021-03-18 19:36:55] INFO "GET /assets/built/bundle.js?v=7b7730315d" 200 14ms
[2021-03-18 19:36:55] WARN Ghost is shutting down
[2021-03-18 19:36:55] WARN Ghost has shut down
[2021-03-18 19:36:55] WARN Your site is now offline
[2021-03-18 19:36:55] WARN Ghost was running for a few seconds
[2021-03-18 19:36:58] INFO Ghost is running in production...
[2021-03-18 19:36:58] INFO Your site is now available on ...
[2021-03-18 19:36:58] INFO Ctrl+C to shut down
[2021-03-18 19:36:58] INFO Ghost server started in 1.525s
[2021-03-18 19:36:59] INFO Database is in a ready state.
[2021-03-18 19:36:59] INFO Ghost database ready in 2.034s
[2021-03-18 19:37:01] INFO Ghost booted in 4.888s
[2021-03-18 19:37:02] WARN Ghost is shutting down
[2021-03-18 19:37:02] WARN Ghost has shut down
[2021-03-18 19:37:02] WARN Your site is now offline
[2021-03-18 19:37:02] WARN Ghost was running for a few seconds
[2021-03-18 19:37:05] INFO Ghost is running in production...
[2021-03-18 19:37:05] INFO Your site is now available on -...
[2021-03-18 19:37:05] INFO Ctrl+C to shut down
[2021-03-18 19:37:05] INFO Ghost server started in 1.087s
[2021-03-18 19:37:05] INFO Database is in a ready state.
[2021-03-18 19:37:05] INFO Ghost database ready in 1.599s
[2021-03-18 19:37:08] INFO Ghost booted in 4.231s
[2021-03-18 19:37:09] WARN Ghost is shutting down
[2021-03-18 19:37:09] WARN Ghost has shut down
[2021-03-18 19:37:09] WARN Your site is now offline
[2021-03-18 19:37:09] WARN Ghost was running for a few seconds
[2021-03-18 19:37:11] INFO Ghost is running in production...
[2021-03-18 19:37:11] INFO Your site is now available on ...
[2021-03-18 19:37:11] INFO Ctrl+C to shut down
[2021-03-18 19:37:11] INFO Ghost server started in 1.288s
[2021-03-18 19:37:12] INFO Database is in a ready state.
[2021-03-18 19:37:12] INFO Ghost database ready in 1.676s
[2021-03-18 19:37:14] INFO Ghost booted in 4.212s
[2021-03-18 19:37:15] WARN Ghost is shutting down
[2021-03-18 19:37:15] WARN Ghost has shut down
[2021-03-18 19:37:15] WARN Your site is now offline
[2021-03-18 19:37:15] WARN Ghost was running for a few seconds
[2021-03-18 19:37:17] INFO Ghost is running in production...
[2021-03-18 19:37:17] INFO Your site is now available on ...
It basically seems to go endlessly on a loop of reboots. If I reload the page fast enough sometimes I can catch this error between 502 and “We’re back” messages:
[2021-03-18 19:39:42] INFO Ghost booted in 3.845s
[2021-03-18 19:39:43] WARN Ghost is shutting down
[2021-03-18 19:39:43] ERROR
NAME: InternalServerError
CODE: IMAGE_SIZE_URL
MESSAGE: Unknown Request error.
level: critical
"https://my_webite.foo/favicon.ico"
InternalServerError: Unknown Request error.
at new InternalServerError (/var/www/ghost/versions/4.0.1/node_modules/@tryghost/errors/node_modules/ghost-ignition/lib/errors/index.js:90:23)
at /var/www/ghost/versions/4.0.1/core/server/lib/image/image-size.js:179:35
at tryCatcher (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/promise.js:725:18)
at _drainQueueStep (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/async.js:93:12)
at _drainQueue (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (/var/www/ghost/versions/4.0.1/node_modules/bluebird/js/release/async.js:15:14)
at processImmediate (internal/timers.js:461:21)
HTTPError: Response code 502 (Bad Gateway)
at EventEmitter.<anonymous> (/var/www/ghost/versions/4.0.1/node_modules/got/source/as-promise.js:74:19)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
[2021-03-18 19:39:43] INFO "GET /" 200 396ms
[2021-03-18 19:39:43] WARN Ghost has shut down
[2021-03-18 19:39:43] WARN Your site is now offline
[2021-03-18 19:39:43] WARN Ghost was running for a few seconds
[2021-03-18 19:39:45] INFO Ghost is running in production...
This only happens when I due ghost start, when I do ghost run and log the responses everything seems normal:
2021-03-18 19:43:50] INFO Ghost server started in 1.071s
[2021-03-18 19:43:50] INFO Database is in a ready state.
[2021-03-18 19:43:50] INFO Ghost database ready in 1.476s
[2021-03-18 19:43:53] INFO Ghost booted in 4.098s
[2021-03-18 19:44:39] INFO "GET /favicon.ico" 200 20ms
[2021-03-18 19:44:39] INFO "GET /" 200 488ms
[2021-03-18 19:44:39] INFO "GET /assets/built/main.css?v=dfcd71ef69" 200 33ms
[2021-03-18 19:44:39] INFO "GET /assets/built/bundle.js?v=dfcd71ef69" 200 36ms
[2021-03-18 19:44:39] INFO "GET /content/images/2021/03/01-copy.jpg" 304 3ms
[2021-03-18 19:44:39] INFO "GET /content/images/2021/03/01-4.jpg" 304 6ms
[2021-03-18 19:44:39] INFO "GET /members/api/site/" 200 3ms
[2021-03-18 19:44:39] INFO "GET /members/api/member/" 401 1ms
Thanks for your help, again, I really appreciate it.
Hi Hannah. Indeed, it doesn’t look like the other issue with nvm. I tried downgrading to node 12 using the instructions and no luck, ghost behaved in the same way.
Hello Hannah, I went through the reinstall of ghost-cli suggested in the thread above but unfortunately it didn’t work. I still get 502 errors and constant reboots when doing ghost start. ghost run continues to work without issue.
This is the output of sudo systemcl status ghost_service :
● ghost.service - Ghost systemd service for blog: .
Loaded: loaded (/var/www/ghost/system/files/ghost.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2021-03-19 22:44:25 UTC; 4min 23s ago
Docs: https://docs.ghost.org
Process: 14582 ExecStart=/usr/bin/node /usr/local/bin/ghost run (code=exited, status=1/FAILURE)
Main PID: 14582 (code=exited, status=1/FAILURE)
Mar 19 22:44:24 my_machine node[14582]: [2021-03-19 22:44:24] WARN Ghost has shut down
Mar 19 22:44:24 my_machine node[14582]: [2021-03-19 22:44:24] WARN Your site is now offline
Mar 19 22:44:24 my_machine node[14582]: [2021-03-19 22:44:24] WARN Ghost was running for a few seconds
Mar 19 22:44:25 my_machine node[14582]: /usr/local/lib/node_modules/ghost-cli/lib/process-manager.js:52
Mar 19 22:44:25 my_machine node[14582]: throw error;
Mar 19 22:44:25 my_machine node[14582]: ^
Mar 19 22:44:25 my_machine node[14582]: { message: undefined }
Mar 19 22:44:25 my_machine systemd[1]: ghost.service: Main process exited, code=exited, status=1/FAILURE
Mar 19 22:44:25 my_machine systemd[1]: ghost.service: Failed with result 'exit-code'.
Mar 19 22:44:25 my_machine systemd[1]: Stopped Ghost systemd service for blog: .
I found this similar issue on GitHub, but I don’t even have nvm installed.
After doing all of this now I’m getting a different output:
ghost-mgr@my_computer:/var/www/ghost$ sudo journalctl -u ghost-art -n 50
-- Logs begin at Mon 2020-08-31 19:23:31 UTC, end at Fri 2021-03-19 23:14:43 UTC. --
Mar 19 23:14:28 my_computer node[1925]: [2021-03-19 23:14:28] INFO Ghost server started in 1.305s
Mar 19 23:14:28 my_computer node[1925]: [2021-03-19 23:14:28] INFO Database is in a ready state.
Mar 19 23:14:28 my_computer node[1925]: [2021-03-19 23:14:28] INFO Ghost database ready in 1.665s
Mar 19 23:14:30 my_computer node[1925]: [2021-03-19 23:14:30] INFO Ghost booted in 3.917s
Mar 19 23:14:30 my_computer node[1925]: [2021-03-19 23:14:30] WARN Can't connect to the bootstrap socket (localhost 8000) ECONNREFUSED.
Mar 19 23:14:30 my_computer node[1925]: [2021-03-19 23:14:30] WARN Tries: 0
Mar 19 23:14:30 my_computer node[1925]: [2021-03-19 23:14:30] WARN Retrying...
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Can't connect to the bootstrap socket (localhost 8000) ECONNREFUSED.
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Tries: 1
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Retrying...
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Can't connect to the bootstrap socket (localhost 8000) ECONNREFUSED.
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Tries: 2
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Retrying...
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Can't connect to the bootstrap socket (localhost 8000) ECONNREFUSED.
Mar 19 23:14:31 my_computer node[1925]: /usr/local/lib/node_modules/ghost-cli/lib/process-manager.js:52
Mar 19 23:14:31 my_computer node[1925]: throw error;
Mar 19 23:14:31 my_computer node[1925]: ^
Mar 19 23:14:31 my_computer node[1925]: { message: undefined }
Mar 19 23:14:31 my_computer systemd[1]: ghost-art.service: Main process exited, code=exited, status=1/FAILURE
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Ghost is shutting down
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Ghost has shut down
Mar 19 23:14:31 my_computer node[1925]: [2021-03-19 23:14:31] WARN Your site is now offline
This seems like this error but my port is the right one. I tried removing the boostrap socket section on my config.production.json but it didn’t do anything and the key went back after running ghost start.