Revive old Ghost blog snapshot (locally)

schsoma · September 23, 2019, 10:55am

Hi folks,

I used to have a nice old Ghost blog (v0.6.0), back in those ‘free Openshift’ days… But somewhere around the v2 -> v3 migration I made a backup and let the Openshift thing to vanish.
Now I just decided to export my content from this snapshot.

I looked through the docs and topics here, but couldn’t find anything useful so far. I see the import/export function supported by the ghost engine itself (faq/the-importer).
But my snapshot looks like a real ‘hard copy’ of the application.

The tar.gz file contains the following structure:

Snapshot content tree

├── app-deployments
│ ├── 2016-01-16_07-23-05.380
│ │ ├── build-dependencies
│ │ ├── dependencies
│ │ │ └── nodejs
│ │ ├── metadata.json
│ │ └── repo
│ │ ├── config.example.js
│ │ ├── config.js
│ │ ├── content
│ │ ├── index.js
│ │ ├── LICENSE
│ │ ├── Makefile
│ │ ├── node_modules
│ │ ├── package.json
│ │ └── README.md
│ ├── by-id
│ │ └── ec6e5280 -> …/2016-01-16_07-23-05.380
│ └── current -> 2016-01-16_07-23-05.380
├── app-root
│ ├── build-dependencies -> runtime/build-dependencies
│ ├── data
│ ├── dependencies -> runtime/dependencies
│ ├── logs
│ ├── repo -> runtime/repo
│ └── runtime
├── git
│ └── ghost.git
├── mysql
│ ├── bin
│ ├── conf
│ ├── COPYRIGHT
│ ├── data
│ ├── env
│ ├── hooks
│ ├── lib
│ ├── …
│ │
└── nodejs
├── bin
├── conf
├── configuration
├── COPYRIGHT
├── env
├── etc…

I was wondering if there is a way to revive this application on a plain npm thing locally on a linux machine?

(I want to get my old posts back, so in the worst case, I will try mining them from the all.sql somehow… So, I welcome suggestions!)

(Also, my apologies if I am asking something trivial/impossible. I’m coming from c++ development. )

DavidDarnes · September 23, 2019, 1:33pm

Hey there @schsoma! That’s a very old version of Ghost you’ve got there . I see in that file tree you had the sql data of the site. Try the the following steps:

Import the .sql file into a local mysql instance
Download and install this version of Ghost locally Release 0.11.14 · TryGhost/Ghost · GitHub (install steps in the zip readme)
Migrate to v1.x using these steps Ghost Docs
Use the export option in the v.1.x Ghost admin and generate an export file that will be compatible with a fresh install of v2.x

See how you get on and let us know if you need any more help!

schsoma · September 23, 2019, 1:50pm

Thanks, I’ll try it soon!
This sounds pretty strightforward to me.

schsoma · September 24, 2019, 3:06pm

Hi!

I’ve managed to import the all.sql into a mysql instance (which runs in fine docker container). I can see the ghost database there which is half success!

A link (v0.11/docs/installing-ghost-on-linux) from the readme:
https://docs.ghost.org/v0.11/docs/installing-ghost-on-linux

Not too useful about how to do it actually.
Any idea?

DavidDarnes · September 24, 2019, 4:34pm

Oh dear! Ha, well it is a very old version of Ghost. Found a link on the npm site for installation steps https://www.npmjs.com/package/ghost/v/0.11.0#developer-install-from-git

schsoma · September 25, 2019, 10:08am

Thanks David!

I started to do what the Developer Install section says… and it was a mess. Than I realized, there is a Quick Start Install section right above it. It is much easier!

Well, almost easier. The following issues appeared:

har-validator was added with an unpublished (5.1.2) version to the npm-shrinkwrap.json. It can be simply corrected to 5.1.3.
I had to use nvm to get a closer version of node.
Sqlite3 had an issue during the npm install, and I don’t really see why, but retrying couple of times solved it.

Right now, I have a living mysql instance (and it contains my old ghost database) and a running version of Ghost (v0.11.14).

What comes next? How can I mix these things together?

DavidDarnes · September 25, 2019, 10:18am

Progress! Have you hooked up the old Ghost install to your old Ghost database? Best to make sure the data and install are linked correctly, maybe even reset the login so you can get to your content too. After that you’ll need to follow the v1.x upgrade process Ghost Docs

The reason you need to update is to use the export option so that content is formatted to work with v2.x, the latest version of Ghost

schsoma · September 25, 2019, 10:26am

Nah. That’s what I’m asking. How can I hook them together?

DavidDarnes · September 25, 2019, 12:29pm

Ah sorry, I misread. In the downloaded version of Ghost you have there should be an example config file like this:

github.com

TryGhost/Ghost/blob/0.11.14/config.example.js

// # Ghost Configuration
// Setup your Ghost install for various [environments](https://docs.ghost.org/v0.11/docs/configuring-ghost#section-about-environments).

// Ghost runs in `development` mode by default. Full documentation can be found
// at https://docs.ghost.org/v0.11/docs/configuring-ghost

var path = require('path'),
    config;

config = {
    // ### Production
    // When running Ghost in the wild, use the production environment.
    // Configure your URL and mail settings here
    production: {
        url: 'http://my-ghost-blog.com',
        mail: {},
        database: {
            client: 'sqlite3',
            connection: {
                filename: path.join(__dirname, '/content/data/ghost.db')

This file has been truncated. show original

You’ll need to copy and rename it to config.js and update the values to match your local setup

schsoma · September 27, 2019, 5:00pm

There are some further issues with all.sql.

First, I had to remove all tables related to performance_schema. I was not able to import them.

1142: SELECT,LOCK TABL command denied to user ‘root’@’localhost’ for table ‘cond_instances’ when using LOCK

Later, I realized that the import process drops the mysql.user table and creates a new one (based on an older scheme version, with different column count) and adds some kind of random users to it. (The dumped mysql version is 5.5.52 and I tried to import it into 5.7)

Now I’m wondering how to fix this thing. Should I skip the user table recreation?
This is not about ghost.user table, I assume.

Anyway, I keep trying

schsoma · September 27, 2019, 6:00pm

Yaay, skipping the mysql.user hacking (drop table, create, insert into)… it seems to working.
Even some is visible at localhost:2368

It complained about the missing theme, but that was easy to fix.

After calling npm start (without a typo in config.js ) the following log comes:

Saving failed! Someone else is editing this post.

~/SANDBOX/Ghost-0.11.14 $ npm start

> ghost@0.11.14 start /home/vboxuser/SANDBOX/Ghost-0.11.14
> node index

WARNING: Ghost is attempting to use a direct method to send email. 
It is recommended that you explicitly configure an email service.
Help and documentation can be found at https://docs.ghost.org/v0.11/docs/mail-config.

Migrations: Creating database backup
Ghost is running in development... 
Listening on 127.0.0.1:2368 
Url configured as: http://localhost:2368 
Ctrl+C to shut down
Migrations: Database backup written to: /home/vboxuser/SANDBOX/Ghost-0.11.14/content/data/myveryoldblog.ghost.2019-09-27.json
Migrations: Migration required from 005 to 009
Migrations: Updating database to 006
Migrations: No database migration tasks found for this version
Migrations: Running fixture updates
Migrations: Transforming dates to UTC: (could take a while)...
Skipping Migrations: rolling back because of an Error:
Saving failed! Someone else is editing this post.
Error
    at Error.InternalServerError (/home/vboxuser/SANDBOX/Ghost-0.11.14/core/server/errors/internal-server-error.js:6:18)
    at Sync.update (/home/vboxuser/SANDBOX/Ghost-0.11.14/core/server/models/plugins/collision.js:58:31)
    at .<anonymous> (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bookshelf/lib/model.js:1009:36)
    at tryCatcher (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise.js:693:18)
    at Promise._fulfill (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise.js:638:18)
    at MappingPromiseArray.PromiseArray._resolve (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise_array.js:126:19)
    at MappingPromiseArray._promiseFulfilled (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/map.js:101:18)
    at Promise._settlePromise (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise.js:574:26)
    at Promise._settlePromise0 (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/home/vboxuser/SANDBOX/Ghost-0.11.14/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)

ERROR: Ghost is currently undergoing maintenance, please wait a moment then retry. 
...

Any suggestion? I think, I’m really close now…

schsoma · September 29, 2019, 9:33pm

Almost done…

I was writing a new post ages ago but never published it. I got the idea: what if I fire up a v0.6.0 ghost, publish the post, and dump the new database?

It went suprisingly easy. I could find the proper npm ghost package.
There was some issue with the given sqlite3 build, so I changed it to a slightly newer one (in npm-shrinkwrap.js).
I had to setup an older mysql server (5.5) to be compatible with it.

And that was all!
I have a shining v.0.6.4 ghost with all my stuffs.

I published the draft, exported a new mysql-dump. Started the mysql container (v5.7) and the suggested ghost (v0.11.14).

Unfortunately, the error message remained the same. The process went further somewhat, but still complains about

Saving failed! Someone else is editing this post.

Ahh, now I see…

When I tried to import my original backup db into the ghost v0.11.14 it said:
Migrations: Migrations required from 005 to 009

And stoped after:
Migrations: Updating database to 006
Migrations: Transforming dates to UTC: (could take a while)...

Strange thing: after dumping from v0.6.4 the migration started with:
Migrations: Migrations required from 003 to 009
But still stopped at 006 / transforming dates to UTC.

@DavidDarnes Why did you suggested v0.11.14 exactly?
What if there is a slightly newer version where this migration step is fixed?

DavidDarnes · September 30, 2019, 11:38am

Pleased you’ve got most of the way there! I was advised this version by at colleague. I’m not sure it was necessary to save that single post, I guess you could’ve just copied content into a brand new Ghost v2 install and imported the rest. What state are you at now?

schsoma · September 30, 2019, 12:23pm

I tried to import my dumped database into v0.6.0 and v0.6.4 and both went well. I have working instances of my blog using these engine versions.

But, I am not able to import any kind of sql dump into v0.11.14 because some misterious
Migrations: Updating database to 006
Migrations: Transforming dates to UTC: (could take a while)... error.
(Which results the “Saving failed! Someone else is editing this post.” error message.)

Mabye in v0.11.14 Ghost there was something wrong with the date transforming migration code, OR my exported data is corrupted somehow. In the former case, a newer release could be solution. I will look for changelogs.

schsoma · October 1, 2019, 1:48pm

I did my homework.
v0.11.14 was the latest release before v1.0.0 thus there is no more closer, more fixed version.
That’s kind of sad.

Anyway, I started to search the issues on github and I’ve found this one:

github.com/TryGhost/Ghost

Date Migrations for SQLite & PG when using UTC

opened 12:54PM - 09 Sep 16 UTC

closed 08:38PM - 30 Sep 16 UTC

ErisDS

bug p1:priority

We have had a couple of bugs reported which are in fact a symptom of an issue wi…th the Date migrations we did in Ghost 0.9. - https://github.com/TryGhost/Ghost/issues/7192 - https://github.com/TryGhost/Ghost/issues/7248 We have identified a logic mistake which means that dates are not in the right format in sqlite3 and postgres if your server was running in UTC when the migration ran. This mistake will be corrected with a 008 migration, and shipped as 0.11.0.

Even though the issue is about PG and Sqlite, it sounds related somehow. (I haven’t had much time for reading all the conversations, but I will do it later.)

DavidDarnes · October 1, 2019, 3:41pm

I’m afraid this is the grief you have to go through with such an old version of Ghost . You’ll be pleased to know once all your content is on the latest version handles content much better

schsoma · October 2, 2019, 6:04pm

Uh. Hey. I solved it!

There was the following code in the collision.js (it was called after transformDatesIntoUTC fired an UpdatePost):

parentSync.update = function update() {
  var changed = _.omit(self.changed, [
    'created_at', 'updated_at', 'author_id', 'id',
    'published_by', 'updated_by', 'html'
  ]),
  clientUpdatedAt = moment(self.clientData.updated_at || self.serverData.updated_at || new Date()), 
  serverUpdatedAt = moment(self.serverData.updated_at || clientUpdatedAt);

  if (Object.keys(changed).length) {
    if (clientUpdatedAt.diff(serverUpdatedAt) !== 0) {
      err = new errors.InternalServerError('Saving failed! Someone else is editing this post.');
      err.code = 'UPDATE_COLLISION';
      return Promise.reject(err);
    }
}

Where the client and server update timestamps were different:
if (clientUpdatedAt.diff(serverUpdatedAt) !== 0).
I had the strage idea: what if I skip this checking during the migration? And it worked. So far. The migration is done.

(Should I fix these dates in my db later?)

Finally, I can continue with Step #3: Migrate to Ghost v1.x!

DavidDarnes · October 3, 2019, 8:39am

Nice job!! I’m not sure about the dates to be honest, best to keep backups and find out what happens post updates. They could be fine or they might need some tweaking

schsoma · October 3, 2019, 5:33pm

I think, I am done with everything. :) The v2.31.1 looks and feels amazing!
What else I did:

Fired up (somehow trying hard with ghost install local --v1 succeeded ) a version v1.26.0.
I had some theme issue during the v1 → v2 ghost update. I don’t remember exactly, but at the end my casper symlink was broken.
Ok. I can start over installing with --v1 and stuff.
But. Unfortunately (or actually fortunately), I wrote ghost install -v1…
Some easy folder accessing issues (chown,chmod ) and my just works like a charm. Okay. Hm. I looks different this time.
Anyway, I imported the v1.26.0 json, and done. Hm. What version am I running?

Ofc. It’s the newest one. Fine.

(I need to fix the theme (bleak) what I used to use, beacuse “kn-bookmark css classes are missing.” But I’ll be fine from here.

So, @DavidDarnes, thanks for the replies and the support. I’ve learned a LOT and it feels so good!

DavidDarnes · October 4, 2019, 12:00pm

That’s great to hear! What a journey this was .

More than happy to help, however I don’t think I can take credit in this case as you did a great job debugging it all. Also, thanks for sharing the whole experience. I’m sure people in the community will appreciate this if they ever have to go through a similar process.

Enjoy your latest install of Ghost

Topic		Replies	Views
Backup from Content Directory? Developer help	3	870	June 18, 2018
Moving from self-hosted to ghost-hosted Self-hosting	5	206	January 19, 2024
Migrating Ghost blog/data to a new server (droplet) Migrations	12	5715	May 27, 2020
Recover Blog Content from the ghost directory Developer help	5	897	May 25, 2020
Hello & Migration Plan Developer help	2	92	March 8, 2024

Revive old Ghost blog snapshot (locally)

Related Topics