Wordpress to Ghost Migration Update

Hi @nuclearpengy - thanks for the fast reply.

  1. Total size of all files is 2.2GB. I ran the download multiple times. Each time, it generated both a JSON file and a partial zip file in the ghost-exports directory, so the overall size of my site grew pretty quickly on disk :slight_smile:
  2. Yes - it worked fine.

All image links in the current WordPress content are using the same protocol (https) and using the same domain, etc.

Thanks for this. :+1:

  1. OK. Thinking out loud: you probably need disk usage * 3 worth of space (not sure if you’ve got enough). It could also maybe be memory/ram related.
  2. OK awesome.

I’ve just deleted the plugin and done a fresh install and export and the URL replacement worked on both the zip download and the plain JSON download. If you don’t mind, please try uploading a new image to a test post and then download the JSON again and search the JSON for that specific post to see what URL is being used, to see if it’s been replaced or not.

Might help before the migration to run an image optimization plugin to ensure the images are not bigger than they need to be and also a search & replace plugin to fix potential database url issues.



1 Like

I have plenty of disk space - over 70GB free.

I use ShortPixel for image optimization, but the sad fact is that WordPress creates tons of thumbnails for each size statically, as we are all familiar with I’m sure, and my theme has a bunch of them. This is as optimized as it is going to get :slight_smile:

The good news is that I used Ghost for a year up until about a month ago, so I have a current-ish backup, and will end up just reverting to that. My full images package with Ghost is around 600MB - much smaller.

It’s throwing a 502 / bad gateway, so it could be that something with NGINX is mad at the time the script is running or something along those lines. This is a pretty straightforwards NGINX configuration, and works for many other things, so perhaps there’s another setting or tweak that is required…

Re: Disk space - OK cool, that should be more than enough so we can rule that out. It may very well be NGINX or memory related.

Re: JSON - did you come right with a test post? Did it set the base URL correctly?

To further investigate, try adding this code somewhere within your theme files temporarily.

<?php
    // test code begins.
    echo "<br/><hr/>";
    $upload_dir = wp_upload_dir();
    $testuploadurl = $upload_dir['baseurl'];
    $testuploadurl_escaped = addcslashes($testuploadurl,"/");
    echo "<strong>This is the upload URL:</strong> <code>" . $testuploadurl . "</code><br/><hr/>";
    echo "<strong>This is the upload URL escaped:</strong> <code>" . $testuploadurl_escaped . "</code><br/><hr/>";
    if (class_exists('ZipArchive'))
    {
        echo "ZipArchive is correctly installed and enabled.";
    } else
    {
        echo "ZipArchive is not installed or enabled.";
    };
    // Test code ends.
?>

It will display the URL that is getting replaced with /content/images/wordpress.

If this value is different from the URL being used by your images then that would explain why the update/replacement isn’t working.

Additionally, it’ll display whether the PHP ZipArchive extension is installed/enabled.

I did some further investigation and uploaded a bunch of extra files to my test site:

06

I then tried to generate the zip and it failed to download on first attempt.

I did some scratching and fiddling with the max execution time (increasing it) and eventually, the file downloaded in full and I was able to open/extract it.

47

So, if possible try increasing your max execution time to the highest value you can.

1 Like

Can’t export from wordpress - keep getting cloudflare database overloaded page or something like that. When I set the RAM to 16 gb I managed download the json, which is 100+ mb, but when trying to import it, it gives me an error. Max execution time is set to 30000 everywhere I could think of putting it(

Hi @unnanego - thank you for the feedback. :slightly_smiling_face: :+1:

Sounds like you’re working with quite a lot of data. How many WP posts are you trying to export?

1 Like

3327, each with multiple images

Thanks for this feedback, much appreciated.

To confirm:

  • The zip download failed
  • The JSON only download worked after increasing server resource allocation
  • The problem at the moment is importing the JSON into Ghost?

I’ve read the Ghost 3.0 announcement and also the news about the new version of the migration plugin. I’ve installed Ghost 3.0 on my VPS and it’s working fine, and I’d like to export my WordPress blog posts to the Ghost one.

I’ve installed the plugin but it doesn’t work. After a few seconds the following message appears: “The site is experiencing technical difficulties. Please check your site admin email inbox for instructions.”

I guess this is due to my blog being quite large. I’ve got over 3,100 posts (it’s been up since 2005) and I guess the script needs more time (or memory, or file size permissions) to finish its work. Could somebody help to find the exact problem and how to solve it?

I’ve tried to change some PHP parameters in my WordPress’ VPS (I use EasyEngine there to automate some processes, but I don’t use EE on the Ghost VPS, that’s a different one). According to this support article I’ve changed this on the “custom.ini” file:

max_input_vars = 1234
max_memory_limit = 1024M
max_execution_time = 2000
upload_max_filesize =100M
post_max_size =100M

After restarting nginx and PHP the plugin still doesn’t work. Same error, even when I try to download the .json file. It always shows the error after 30s of thinking. I guess there is some 30 seconds value somewhere that has to be edited but I don’t find where, according to the EasyEngine docs it should work. In fact the ghost migrator info shows 1GB memory limit and 2000s execution time, so the problem lies somewhere else.

My WordPress folder is 2.2 GB in size, and my VPS has 11 GB of space available.
ghost1

30sec is not particularly long for a large import, sometimes it can take a few minutes. What error are you getting? If the request times out, as long as the upload has finished (that part shouldn’t time out with most server setups) then it will continue to be processed in the background.

hi @javipas - thank you for the feedback. Much appreciated. :+1:

Regarding generating the zip - I was able to zip just under 3GB in around 20 to 30 min on a shared hosting so 2000 seconds should be a good base amount but if you can, try to increase it some more, maybe try 60min / 3600 seconds.

The PHP max_execution_time does however seem to be reporting correctly so maybe there’s something else on the system causing a timeout. I recommend checking the NGINX proxy_read_timeout value.

Please let us know if that makes a difference.

Thanks for the suggestion. I’m afraid it doesn’t work. I’ve looked for timeout values and there was a “keepalive_timeout 30;” line on the nginx.conf file. I changed it to 3600 and also added the proxy_read_timeout value (there was none) without luck. (I always restart nginx and php to apply the changes, of course).

In fact the message I got previously appears at random times, always lower than 30s. I’ve used the chronometer in the smartphone and it was 24, 26, 23, 21 seconds… No idea what’s happening here. I guess the EasyEngine config makes it a little bit difficult and obscure, I could try to setup a virtualmachine with a normal LEMP server (without Easy Engine), install there a backup of the WordPress blog and run the Ghost migrator plugin from there, but I’ll have to spend a little time with that and I’m not in a big hurry.

Thank you again for your suggestions, if someone has other ideas just let me know.

Try also with fastcgi timeouts.

Hi @javipas - thank you for the additional feedback. :+1:

I’ve just experienced a similar issue with a site hosted on WP Engine. The PHP memory limit and max execution time were both reporting high values but it would fail after ~60 seconds.

After some further investigation, I discovered that WP Engine kills any process running for ~60 seconds and there’s not much we can do about it so it’s definitely possible that there’s something not-PHP related that is causing the process to fail/timeout.

Then, on a test site, I used a random content generator to generate 3,576 posts. It initially failed with the original site settings (96MB memory limit) but I incrementally increased the memory limit to 256MB and then it downloaded successfully. Something I noticed during this process was that PHP was reporting a memory limit that differed from the value actually applied to/configured on the site/domain (presumably a global server value) so that’s something to keep in mind.

That makes sense, I guess there’s something on EasyEngine installation that automates that kind of behaviour. I’ve asked there if someone can think of a solution

Let’s see if that helps. Thanks for the quick replies :wink:

Sure thing. :+1: :slight_smile: I’m keen to know what the root cause of the timeout is.

I think that tweaking php and nginx configs is not the right way.

As I’ve mentioned in githubs issue, WP-CLI compatibility to export through command line will be much more effective and precise manner. The plugin author confirmed that WP-CLI support is on the go.

Hi @codemotion - Some people won’t have SSH and CLI access to their WordPress installations so we’d ideally like the GUI to work for as many people as possible.

I am however working on WP-CLI support and hope to have an initial version ready for release soon.