Error while starting instance after updating to 2.x from 1.x

I was running a very old version of Ghost (1.17) and I decided to upgrade at last. I updated the ghost-cli to latest and ran ghost update on my blog root. After the update I am trying to run ghost run but it terminates with the error

I have tried running ghost doctor and ghost update again but those complete successfully. So, I checked my MySql db for the blog and there was no mention of integrations table
It looks like the db migration didn’t happen successfully. Thankfully I have a json backup from the admin panel as well as the backup of the content folder. But I need to start the instance someway.
Any guidance is appreciated.

I tried --rollback but its failing too. I checked the logs and the logs say

Debug Information:
    OS: Ubuntu, v16.04
    Node Version: v8.11.4
    Ghost-CLI Version: 1.11.0
    Environment: production
    Command: 'ghost update --rollback'
Message: The database migration in Ghost encountered an error.
Suggestion: ghost update --rollback
Stack: Error: The database migration in Ghost encountered an error.
    at /<self-redacted-path-to-node>/node_modules/ghost-cli/lib/tasks/migrator.js:41:21
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)

Additional log info while trying to start

Debug Information:
    OS: Ubuntu, v16.04
    Node Version: v8.11.4
    Ghost-CLI Version: 1.11.0
    Environment: production
    Command: 'ghost start'
Message: select `integrations`.* from `integrations` where `integrations`.`slug` = 'zapier' limit 1 - ER_NO_SUCH_TABLE: Table 'ghost_cmdrgarud.integra$
Suggestion: journalctl -u ghost_ghost-cmdrgarud -n 50
Stack: Error: select `integrations`.* from `integrations` where `integrations`.`slug` = 'zapier' limit 1 - ER_NO_SUCH_TABLE: Table '$
    at Server.server.close (/<self-redacted-path-to-node>/node_modules/ghost-cli/lib/utils/port-polling.js:38:28)
    at Object.onceWrapper (events.js:313:30)
    at emitNone (events.js:106:13)
    at Server.emit (events.js:208:7)
    at emitCloseNT (net.js:1659:8)
    at _combinedTickCallback (internal/process/next_tick.js:135:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)

Do you have the log output from the upgrade available so that we can see what went wrong?

Also can you show us what’s in the migrations table?

Yeah sure, see below

Debug Information:
    OS: Ubuntu, v16.04
    Node Version: v8.11.4
    Ghost-CLI Version: 1.11.0
    Environment: production
    Command: 'ghost update --v1'
Message: The database migration in Ghost encountered an error.
Suggestion: ghost update --rollback
Stack: Error: The database migration in Ghost encountered an error.
    at /<self-redacted-path-to-node>/node_modules/ghost-cli/lib/tasks/migrator.js:41:21
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)

And this is what the migrations and its lock table looks like

Ok - I’m struggling to follow what’s happening here.

The error messages to do with integrations are for a Ghost v2 blog, but the output you provided was for ghost update --v1 which updates to the latest v1, not to v2.

CLI should have gotten you to run through an update to the latest v1 first, and then to v2. Your migrations table is missing a whole bunch of migrations for v1.

Are you able to take a look in the ghost log file (in content/logs) and put the full content that’s related to the upgrade process into a gist or similar?

I understand you just want to get it running, but I’m struggling to understand enough of what happened to be able to figure out a path foward…

Can you try running the following command inside your ghost install?

./current/node_modules/knex-migrator/bin/knex-migrator health --mgpath=./current

I got the DB migration failure when upgrading to the latest v1 itself. I tried rolling back and the rollback threw an error too. So, I proceeded to upgrade to v2 hoping that things would sort themeselves out. The upgrade to V2 didn’t throw any errors but I haven’t been able to run. Thus my error logs only have the error to v1 latest and not the v2 one.

Yeah sure, see {"name":"Log","hostname":"1337","pid":12315,"level":50,"err":{"id":400,"domain": -

I get the following output

I am not running init as I don’t wanna touch the db with a ten foot stick without knowing what that command does :sweat:

Yeah you definitely shouldn’t run init - that command I gave you should be picking up your DB and telling us what state it’s in.

Something has gone wrong for sure as you shouldn’t have been able to update to v2 without the latest v1 upgrade being in place first. I’m sorry that this happened and allowed you to get into such a broken state!

I’m still trying to figure out a way back or forwards…

Hopefully a last question for you, can you paste me the output from these 2 commands?

ls -al
ls -al versions

Sorry, also I’ve just gone through the logs you shared and these only cover the rollback errors, nothing from the original upgrade - I’m keen to understand what broke so I can make sure it doesn’t happen again.

Can you grab a tonne more of your logs? The main log rather than the error log would be preferable. All logs go to the main log, only errors go to the error log, and context is king!

This is weird. I am certain that everything was owned by the ghost user before the upgrade. I don’t know if the owner can break anything.
The logs are below:
Latest - {"name":"Log","hostname":"1337","pid":926,"level":30,"req":{"meta":{"requestId": -
Second Latest - {"name":"Log","hostname":"1337","pid":926,"level":30,"req":{"meta":{"requestId": -
Third Latest - {"name":"Log","hostname":"1337","pid":926,"level":30,"req":{"meta":{"requestId": -

Don’t worry about the file owner unless ghost doctor says you need to change something :slight_smile:

The reason I asked for the output from those ls commands is so that I could see exactly which versions you have installed and are currently in use.

What I recommend is this:

  1. Attempt a manual rollback, without rolling back the database (which is failing due to a bug)
  2. Work through the upgrade to v1 latest until we’ve got that working perfectly
  3. Re-attempt the upgrade to v2

For part 1 - the manual rollback, you’ll need to do the following:

ln -sfn /var/www/cmdrgarud/ghost/versions/1.26.0 current

This will override the symlink so that 1.26 is the current version, rather than 2.25.6 that won’t boot.

Next you need to try to start Ghost in debug mode so that we can see exactly what happens:

ghost run (!!! not ghost start).

If you can do these 2 commands and share the output, we should be able to do step 2 and debug the v1 upgrade based on this output.

Thanks, I got this output

Should I install and run the knex-migrator? Is there a ghost-cli way to do the same?

So - to provide some background here:

  • In Ghost v1 knex-migration commands were managed outside of Ghost in Ghost-CLI
  • In Ghost v2 Ghost itself handles running these commands

In either case you aren’t expected to install it or run any commands.

Because we’re in a weird path in this case doing manual stuff Ghost-CLI isn’t re-running the migrations, but I believe we can get it to do so by using

ghost update --v1 --force

We need to pin to v1 for now with --v1 to make sure we finish this upgrade fully.

We need to use --force because there are no more versions of v1 (1.26 is the latest) and by default Ghost CLI will tell you you are already up-to-date. However with --force Ghost CLI will download and reinstall the latest version.

Therefore this should trigger the migrations.

Can you give it a go?

Just tried ghost update --v1 --force and everything went smoothly. But I get the same error as above on ghost run.

So, I decided to check something else. I switched the symlink of current to my initial version of 1.17.1 and then ran ghost run; and it works. I can even open my blog in the browser.

But I have noted another thing. Doing ghost version still shows me 2.25.6. Also I tried updating from 1.17.1 which again went without any errors thrown but the symlink didnt get updated.

OK - I understand that it’s frustrating, but I’m deliberately trying to work through this with you step by step, jumping around makes it really tricky to follow and help.

Just tried ghost update --v1 --force and everything went smoothly.

Please paste the full output from that command.

Please set the version back to 1.26.0 (I understand it doesn’t work, but we’re not going to get anywhere unless we fix that).

Please then try updating the .ghost-cli file so that active-version is 1.26.0 and previous-version is 1.17.1 and re-run the command ghost update --v1 --force.

That made ghost download fresh and I encountered the error that I did the first time around.

Logs - {"name":"Log","hostname":"1337","pid":4250,"level":50,"err":{"id":200,"domain":" -
Error Logs - {"name":"Log","hostname":"1337","pid":4250,"level":50,"err":{"id":200,"domain":" -

Can you also share the contents of the cli debug log file output in that screenshot?

Ok - absolutely nothing is providing the context of what the actual error with the DB is :frowning:

I’m also not able to reproduce the behaviour.

I understand this isn’t a great experience - it’s not how we expect his to work and we will work to iron these issues out.

Moving forward:

If you update .ghost-cli to have 1.26.0 as the active version and 1.17.0 as the previous version, and then try ghost update --rollback --verbose, if there isn’t some underlying problem with the DB, that should set you back to 1.17.0 properly now without errors, including clearing out the migrations table, but leaving the 1.26.0 folder in versions/.

If that still tells you errors about the integrations table (which only exists in 2.0) then you can try either of the 2 following things:

A. modifying the migrations table so that in all the places where it says “currentVersion 2.25.6” it says 1.26.0 instead, and then try ghost update --rollback --verbose again. This is another attempt to get ghost-cli doing the right thing again.


B. Redo the manual revert and try to get ghost-cli back into the right state manually

  • change the symlink manually
  • change .ghost-cli to have 1.17.1 as the active version
  • change .ghost-cli to remove previous version entirely (ensuring the JSON stays valid)
  • edit the database manually to remove all the lines from migrations that refer to > 1.17.0

Hopefully one of these 3 ways gets you reverted properly.

After that it should be safe to try another upgrade, but I would probably try a single small upgrade

ghost update 1.18.0 --verbose (might be an idea to start using --verbose / -V a lot!)

If that also fails with an unknown DB error then it’s a case of either

  1. stay on 1.17 and wait for us to fix ghost-cli to output the error properly
  2. do a fresh Ghost install of 2.0 in another folder and import your content

Nope, none of the above worked. I guess I will go with a fresh install.

Thanks for taking the time to assist me anyway, I appreciate it.