Thanks for the pointers, spot on.
User vagrant did have sudo permissions but not for user ghost so without alterations this command failed:
sudo -u ghost /usr/bin/whoami (error identical to the installer error)
After adding this config to /etc/sudoers.d/vagrant the above command does work:
# Grant user `vagrant` permission to run specified commands as
# user `ghost`. Required to get throught the ghost-cli installer.
vagrant ALL=(ALL) ALL