In my case, I needed nvm for other applications so It would be a hassle to have to uninstall it. Instead I did the following:
which ghost
# /home/<your_user>/.nvm/versions/node/v12.17.0/bin/ghost
Then I modified the ghost.service with the ghost binary path in nvm:
sudo vim /etc/systemd/system/ghost_your-blog-com
[Unit]
Description=Ghost systemd service for blog: your-blog-com
Documentation=https://ghost.org/docs/
[Service]
Type=simple
WorkingDirectory=/var/www/html/your-blog
User=998
Environment="NODE_ENV=production"
#ExecStart=/usr/bin/node /usr/bin/ghost run
ExecStart=/usr/bin/node /home/<your_user>/.nvm/versions/node/v12.17.0/bin/ghost run
Restart=always
[Install]
WantedBy=multi-user.target
Afterward, run
sudo systemctl daemon-reload
ghost start
Thanks @elergy for the hints