A better search for Ghost - and it's not Algolia (and open source!)

Share & showcase
1

A few weeks ago, one of my customers reached out looking for full-text search capabilities for their Ghost site. They mentioned Algolia, which I hadn’t worked with before, so I decided to dive in and experiment.

Algolia worked well with Ghost - the toolkit the Ghost team provides does its job. But as I implemented it, I realized the setup process is quite technical. And whenever something’s overly complicated, I have the urge to make it easier.

My first thought was to integrate Algolia directly into Magic Pages as a feature. But there was a problem: since Algolia is proprietary software, customers would still need to create and manage separate Algolia accounts. That’s friction I wanted to eliminate.

After about a week of research, I discovered Typesense - an excellent open source alternative to Algolia with similar capabilities and simpler implementation.

This changed my approach: instead of just adding Algolia support, I could build a complete Typesense integration for Magic Pages as a one-click feature (which is coming soon: https://www.magicpages.co/roadmap/better-search-on-pro-plans/ ).

But what’s an open source search without well…open sourcing it. This is something every Ghost site could benefit from, no matter where it’s hosted.

Today I’m releasing ghost-typesense on GitHub, to make this possible. This toolkit provides everything needed to power up your Ghost site’s search capabilities:

  • A beautiful search UI (with dark mode detection and a manual override)
  • Full-text search (and the ability to customise searchable fields)
  • A command-line tool to easily sync your content, similar to the Ghost team’s Algolia CLI
  • A webhook handler for keeping search results updated in real-time (with the ability to deploy it on Netlify, again, similar to the Ghost team’s Algolia implementation)

You can see it live in action on Magic Pages and my personal blog.

Why should you use this?

Ghost’s default sodo-search only looks at titles, excerpts, tags and authors. This works great for most sites. But, as your site grows, you might want to give people the opportunity to look for keywords that are embedded deep in your post content.

This Typesense integration enables this.

So far, Algolia has been the standard recommendation for advanced search in Ghost. And hey, it’s a great service, but it can get expensive as your site grows, and is closed-source proprietary software.

Typesense provides comparable search capabilities but as fully open source software. You can either self-host it (free) or use Typesense Cloud if you prefer a managed solution.

The GitHub repo contains complete documentation for implementing this on any Ghost site, regardless of where you host.

Would love to hear your thoughts on this and any suggestions for improvements (specifically on the search UI - trying to strike a balance between beautiful, yet not too opinionated).

12 Likes
2

This would be awesome @jannis. I just emailed you some of my thoughts :slight_smile:

1 Like
3

This is amazing. Perhaps we can do similar with chatbot? So it can provide answers on top of search?

4

Typesense can do something like that, yeah:

But I do not see this part of a Ghost Typesense toolkit. It is rather something that needs to be implemented on a Typesense server. If you’re already using the toolkit to index your Ghost site, it’s a great base for a RAG pipeline, though.

5

Thanks for sharing this integration. Did you consider meilisearch? I’ll check out your solution

6

@FanyangMeng put together a package for Meilisearch that works similarly:

2 Likes
7

Excellent @jannis … and typo-tolerant for fat-fingered typists like me :grinning:.
With many thanks :+1:

1 Like
8

Hello, thanks for sharing this!

May I know how to properly debug the CLI? I can’t sync my content and both API and configurations seems to be OK.

9

It should throw an error.

When you say you can’t sync content, how exactly does that show?

10

This is the error message: ✖ Failed to sync posts: Failed to fetch posts from Ghost.

The endpoints are both working for posts and typesense (tested on localhost and exposing domains).

I can see the beautiful UI while typing but there are no posts.

My issue seems to be specifically on the sync content step, can I do it manually, perhaps with some script?

Thanks for building this.

11

Quick thought on this - there’ve been some recent changes to endpoints that remove limit=“all” functionality. I’m sure @jannis is on top of it, but I wonder if maybe @satonotdead is running an older version of either the search code or of Ghost, and hitting an incompatibility there?

Just a thought. Worth almost exactly what you paid for it.

12

The CLI is technically the “manual step”.

Failed to fetch posts from Ghost is thrown when the Ghost Content API fails. So, this is where the issue is most likely.

Have you tested the Content API manually already? If yes, how did your API call look like and what did it return?

Since I had a crystal ball when I wrote the whole thing, they all have proper limits :joy:
(though my crystal ball was more thinking of cases where users want to sync tens of thousands of posts all at once on my server :upside_down_face:)

1 Like
13

I’m running the last versions of Ghost and ghost-typesense.

The endpoint confirms on 200, shows all posts and it’s OK. The CLI is the ‘manual step’ but there are a few scripts built on it.

If there is no more manual than manual :sweat_smile: it’s possible to debug more?

14

Bit tricky without knowing exactly what you’ve done :smiley:

This is what the CLI does:

It uses ts-ghost
under the hood, but that is just a typed wrapper for the content API.

Can you share the exact API call that returned the posts for you? I can then compare that to what ts-ghost does.

15 
curl -G   --header "Accept-Version: v5.0"   "https://ghost.org/ghost/api/content/posts/"   --data-urlencode "key=CONTENT-API"   --data-urlencode "limit=1"
16

Quick update for everyone reading: I am debugging this with @satonotdead in DMs. Will update things here, if we find an issue :slight_smile:

1 Like
17

I just add the config file I’m using on typesense-ghost for anyone that could be facing the same:

{
  "ghost": {
    "url": "http://127.0.0.1:2368",
    "key": "API-CONTENT",
    "version": "v5.0"
  },
  "typesense": {
    "nodes": [{
      "host": "http://127.0.0.1",
      "port": 8108,
      "protocol": "http"
    }],
    "apiKey": "API-TYPESENSE"
  },
  "collection": {
    "name": "ghost"
  }
}

satonotdead@nowhere:/path/to/ghost-typesense$ ghost-typesense init --config ghost-typesense.config.json
✔ Collection initialized successfully
satonotdead@nowhere:/path/to/ghost-typesense$ ghost-typesense sync --config ghost-typesense.config.json
✖ Failed to sync posts: Failed to fetch posts from Ghost

There is no debug to follow and both APIs are working as expected (Ghost and Typesense).

Tried with localhost and secured domains, the same error persists. Any ideas, tips or suggestions should be appreciated.

18

Thanks for sharing that, @satonotdead.

I have tried with that exact configuration (though correct API keys, of course) and cannot reproduce that on my end.

The only cases where I could reproduce it was when either Ghost’s API key or URL were wrong in the configuration.

I have updated the ghost-typesense packages to v1.7.1 to include a new --debug flag for more information and error handling for the two cases I found that threw your error.

Try updating the CLI with npm install -g @magicpages/ghost-typesense-cli and then run ghost-typesense --debug sync --config config.json. That should tell us more about what went wrong.

2 Likes
19

Hi Jannis, thanks again for your follow up!

You see, the address is correct like the API key (triple checked and changed a few times to be sure :sweat_smile:).

I updated and runned it again, tried on localhost and URL with SSL:

✅ Config validated successfully
🔧 Ghost URL: http://127.0.0.1:2368
🔑 Ghost API Key: ...
🔍 Typesense nodes: [
  {
    "host": "127.0.0.1",
    "port": 8108,
    "protocol": "http"
  }
]
📡 Fetching posts from Ghost...
❌ Error details:
Error: Ghost API error: TypeError: fetch failed
    at GhostTypesenseManager.indexAllPosts (/usr/lib/node_modules/@magicpages/ghost-typesense-cli/node_modules/@magicpages/ghost-typesense-core/dist/index.js:145:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Command.<anonymous> (file:///usr/lib/node_modules/@magicpages/ghost-typesense-cli/dist/index.js:59:5)
Stack trace: Error: Ghost API error: TypeError: fetch failed
    at GhostTypesenseManager.indexAllPosts (/usr/lib/node_modules/@magicpages/ghost-typesense-cli/node_modules/@magicpages/ghost-typesense-core/dist/index.js:145:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Command.<anonymous> (file:///usr/lib/node_modules/@magicpages/ghost-typesense-cli/dist/index.js:59:5)
✖ Failed to sync posts: Ghost API error: TypeError: fetch failed

My instance is working with N8N, so I think it’s not related with the Ghost instance but something misterious.

I hope to solve this, I really like Typesense and have a lot of data and projects for integrate into it.

20

The fact that you’re not running into any of the two error cases indicates it is indeed a different problem.

Looking at everything, my best guess is that the CLI cannot reach Ghost at all. There is no rejection from Ghost (we’d see that in the logs). It’s like the CLI doesn’t even see Ghost on the address and port.

So, next question: HOW are you running all of these components?

Ghost via CLI? In a Docker container? Is Ghost actually listening to 127.0.0.1 (127.0.0.1 for Ghost might be different than 127.0.0.1 for your ghost-typesense CLI)?

Is the ghost-typesense CLI installed in a Docker container?

This smells like a networking issue.

1 Like