Simply using the new tag did not work for me in my docker setup. After using the new image, I encountered an exception, where the SQLite dependency could not be found. Probably, because the files for the 3.x version was not included in the docker image.
I instead opted to use the ghost:3 for the moment and upgrade using the ghost-cli npm package.
docker exec -it ghost-container-name
# >> Then in the attached container
npm i -g ghost-cli
su node
ghost update v3 # or whatever your current major version is
ghost update
exit
Then, back in your host machine, bump the tag to :4 und deploy the container as you normally would. As i used docker-compose it is as simple as docker-compose restart.