GhostAI - Ghost & OpenAI Integration

Integrations & API
1

Dear all,

William and I are the original developers of an Open Source integration between Ghost and OpenAI. Our goal is to enhance Ghost with AI integration as much as possible. We are pleased to present our first solution, which addresses the long-standing issue of finding similar posts in Ghost without requiring any deep customization of Ghost to support Elastic or similar technologies.

We would greatly appreciate your feedback and adoption of this solution. Depending on the response, we plan to release additional AI-powered solutions, including a custom chatbot trained from your own content and an OpenAI-powered search feature.

Thank you for your consideration.

Introducing GhostAI Related Posts

GhostAI Related Posts is a project that utilizes OpenAI Embedding API to generate related blog post tags for a given Ghost Blogging site. The project provides the functionality to display “Related Posts” at the bottom of each blog post. Currently, this feature is not available in Ghost.

Check out a sample blog post with related topics at the bottom: Experiences with the Standard Suspension on BMW iX50: What type of suspension should you pick?

The project comprises of two scripts that can be executed either on a local machine or directly on the server. Unless the site has tens of thousands of blog posts, running the scripts on a local machine should suffice.

The first script iterates through all public blog posts on the server and extracts the text content, which is then sent to the OpenAI (Ada model) to generate a set of vectors. These vectors are then stored in a text file (find them in ./output/). The script also supports incremental generation, which means that it only generates vectors for blog posts that have not yet been processed.

The second script iterates through all the vectors in the directory and ranks their similarity by giving a score. For each blog post, a few other blog posts with high similarity ranking will be grouped together. Ghost internal tagging feature is used for the grouping mechanism, and each blog post ID becomes the internal tag name.

To display the related blog posts at the bottom of another blog post, the post.hbs template needs to be edited by inserting a simple tag filter. This design works with both self-hosted Ghost sites and Ghost Pro sites (hosted by the Ghost team) and does not require any additional configuration, database changes, or other adjustments.

Users can contribute to the GhostAI Related Posts project by submitting pull requests on Github. We welcome any improvements to the project, including bug fixes, new features, and enhancements.

Need help setting up?

For the first few sites, we would be happy to assist you in setting up and running the integration. If you are interested, please feel free to direct message me here, and I would be glad to provide further assistance from there.

Enjoy!

1 Like
2

Interesting! What’s the mechanism for getting tags onto new posts and updating old posts with matches to new ones?

I just built something on top of Algolia’s Recommend that accomplishes a similar goal. Since my client already had a sync to Algolia fir search, that made hood sense.

3

Hi Cathy!

Currently there are two scripts that run independently from each other, one is to generate the vectors and one is to tag the posts. For new posts, you can just setup a corn job to run it for example once a day, which will detect new posts and generate the vectors. For updating old posts with new tags, you can likewise setup a corn job for the second script, which will update all the posts with the new tags.

I believe this is much more lightweight than other setup like Algolia or Elastic, especially for Ghost hosted sites that may not have any database / SSH access, as this setup can entirely run on your laptop.

Let me know what you think especially if there is anything we should improve. Take care.

4

Excellent work!
Is there a way to implement it for all users who use Ghost on Docker?
Thanks!

5

Hi Joan,

It works in any environment. Do you mean you only have access to the Docker image but no ability to run script / install cron outside of Docker image?

6

Ah, sorry. I can install this script on an external machine. And by specifying my Ghost API, it will automatically work? Thanks

7

Correct. You will also need to provide the OpenAI API key. You can get one by setting up an account there. If you don’t have a lot of blog posts, their free quota should be enough to get you going.

Good luck, and don’t forget to let me know once you have it running successfully. :)

1 Like
8

@peterwo Hi! t fails me when executing the first python file. Or it’s normal?

Captura de pantalla 2023-04-06 a las 19.59.50
Captura de pantalla 2023-04-06 a las 19.59.501000Ă—340 45.3 KB

At the end it generates all the .csv files inside the output folder. With the second file, it generates everything OK.

Captura de pantalla 2023-04-06 a las 20.27.17
Captura de pantalla 2023-04-06 a las 20.27.17992Ă—452 16.8 KB

I think it’s work because inside each article, it’s generates “random” news tags (see the image).

Captura de pantalla 2023-04-06 a las 20.33.16
Captura de pantalla 2023-04-06 a las 20.33.16760Ă—1252 105 KB

And another question. To “update” the related articles. Do I have to run the 2 scripts from time to time?
And… can it be done in a “random” way? That every time I do F5, it shows different related articles in the same article?

Thanks for all!

9

Regarding your first question, it’s possible that the script encountered a temporary capacity issue when accessing the OpenAI API. Please try re-running the first script to check if the problem still persists.

As for your second question, indeed, the script needs to be executed periodically to ensure up-to-date relationships when new blogs are added. You can incorporate both scripts into a crontab to automate the process on a daily basis.

Currently, the script does not support a “random” mode, but I believe that’s an excellent suggestion. :blush:

1 Like
10

The “random” suggestion is pretty cool. I need to looking handlebar if the template language supports rand() :slight_smile:

If there are more site using it, I think it will be worth the effort to setup web hook etc so it won’t need crontab to run. But right now I think it is the most straight forward way.

1 Like
11

Something is wrong. I tried re-running the first script but it’s show the same “temporary” error…

12

We’ve made some improvements to the script so that it now provides more detailed information when an error occurs.
Please download the latest version of the code and run it again. This will help us diagnose the problem and address it more effectively. We’ll resolve your issue as quickly as possible.

13

Thanks for your reply and help @lovecactus
With the new file, it’s show this:

Blog failed to convert due to error:
RetryError[<Future at 0x1481695be250 state=finished raised InvalidRequestError>]
ID:627d8f8b3377cc008a4d8075
TitleCĂłmo instalar desde cero macOS en una memoria USB o tarjeta SD
Post content:
Original mobiledoc:{'version': '0.3.1', 'atoms': [], 'cards': [['html', {'html': '<p><span class="rt-reading-time" style="display: block;"><span..............
14

The issue originated from an inability to parse raw HTML content. We have resolved the matter, please download the latest version and execute the script again to rectify this problem. In addition, we have updated our script log to enhance future issue tracking.
Would you please send us the log file located at “./log” to ensure the fix has been implemented accurately? This will allow us to verify that the original content has been parsed appropriately. We appreciate your cooperation!

1 Like
15

Thanks you so much for your work!!! @lovecactus
Now it’s work without errors! :smile:
If you could add the “random” when pressing F5 to the article it would be perfect :heart_eyes:.

Thanks again!!

16

Joan,

We’ve been thinking about that random response thing you mentioned. Right now, tags are sorted based on how much they relate to the main post, with the closest matches at the top. Adding a random feature might mess with the whole point of the project?

If you could shoot me a DM with your site URL, I can check it out and see if there’s a better way to do what you’re aiming for. :slight_smile:

Good luck and happy to see this working on other site!

1 Like
17

Hi @cactus @peterwo ,
The script does NOT work again. I’m with Ghost 5.45.1. Maybe something has been changed and it doesn’t work anymore. I show you the logs:

Captura de pantalla 2023-04-27 a las 11.52.55
Captura de pantalla 2023-04-27 a las 11.52.551842Ă—340 24.5 KB

CleanShot 2023-04-27 at 11.53.29@2x
CleanShot 2023-04-27 at 11.53.29@2x1480Ă—362 41.6 KB

18

Sorry to see it doesn’t work. Did you pull th latest code and it no longer works afterward? Or it stops working after you upgraded the Ghost version?

19

Thank you for your response @peterwo. Yes, I am using the latest version of your script available on GitHub.
It has been days since I launched the script for the last time. I guess it doesn’t work because Ghost has changed things in their latest versions…
Can you take a look at it and see…? It’s a very useful script… Thanks

20

Based on your logs, it appears that the issue was caused by the blog content being structured in a new way, which led to the post content not being parsed correctly and resulting in emptiness.
We have updated our script to improve compatibility with your content. Please download the latest version of the code and give it another try. If you still encounter any issues, kindly send us the log so we can continue working on resolving this problem. Thank you!