Hacktoberfest Project: i18n for Portal, Member emails, comments-ui & SodoSearch

Translating strings in Portal and our member emails is now our most-voted for feature on the forum. Translating comments and search crop up regularly too.

We have had a few PRs that aim to add support to individual components of Ghost, but we believe that i18n needs to be solved with a single solution and therefore we have put together this plan for the requirements and how we believe it needs to work.

We’d love to work together with a group of contributors to deliver i18n as part of Hacktoberfest 2022.

Problem

At present, only themes are translatable using a custom-built i18n framework that we do not want to build on. Portal & emails are part of core, not themes. Comments & search are related to themes, but the text strings within are not. They each have one set of strings, which only need translating into each language once and so it makes sense to have these translations be part of Ghost core.

There is a secondary use-case, where users want to be able to customise copy, rather than translate it. For example, the default text shown in the empty search modal is “Search posts, tags and authors” but some sites use different terminology for tags. This use-case is different and may warrant an escape-valve system for custom strings in future, but our immediate focus is on solving core translations.

Solution

We want to be able to serve translated emails and widgets, based on the language set in general settings. Technically, the solution boils down to answering two key questions:

  1. Where do the translated strings come from?
  2. How do the replacement strings get interpolated?

The answer to 1 is “a JSON file” of some description. We do not want to manage translations in a 3rd party tool like transifex, but we’d like to keep all of the JSON files in GitHub. It needs to be possible to load the strings and pass them to all the places they’re needed.

The answer to 2 is a {{t}} helper that needs to exist in both react and email templates. We want to keep our existing pattern from theme i18n of using the default string as the key. This means that when a default string is changed, it needs to be re-translated in all languages, which is desirable as changes are usually more than cosmetic.

We will also need a way to collect all the default strings and compile them into a new JSON file ready to be translated.

For email templates, it makes sense for the interpolation to be dynamic, done at the time of email-sending as this is a server-side system. For our react “widgets”,we would like to create a custom build for each translation, so it becomes possible to load e.g. portal-es.min.js.

Several aspects of i18n will be made significantly easier by ensuring that all translatable components, and the translations themselves live together in the Ghost monorepo, which is WIP. Portal should be migrated this week, which will allow work to begin.

Technical Plan

Our basic requirements are:

  • Load a single set of translations + (maybe) a set of custom strings into react widgets
  • Load a single set of translations into Ghost core for emails
  • Future: Load translations for Admin
  • Interpolation tools for Handlebars and React
  • Be able to use the default string as the key
  • Be able to generate a new language file with all available keys
  • Not have to load e.g. all of portal’s translations into comments
  • Have a single place, in GitHub, where locales are defined
  • Ideally: generate a build per locale for each JS widget, e.g. portal-es.min.js

That last requirement is the hardest, as we will need to figure out a build step for our react projects… but this should be a one-off piece.

The i18next framework appears to support every single one of our requirements, including fallback-based keys, namespacing and having tools for extracting translation strings. It’s also extensible with plugins and has links to tooling for managing the translation process - so we could have a yarn check-translations command to view what’s not translated.

Expected structure:

  • A new ghost/locales folder in the monorepo contains namespaced JSON files e.g. locales/es/common.json, /ocales/es/portal.json , locales/es/email.json
  • For Node.js/email-templates we do one of:
    • (Preferred) The correct path is loaded from config based on environment
    • ghost/locales gets copied into ghost/core somehow
  • Then for react:
    • Use code-splitting to generate individual versions of portal, comments and search for each supported locale.

I was able to find some details of using i18next with webpack to create a a build step using “code splitting” but it doesn’t quite achieve what we want. We are currently investigating switching to Vitejs for our build tool - and this may end up being an additional prerequisite.

Milestones

This project has prerequisites, is fairly complex and has lots of different phases in which different community members may wish to get involved. Therefore, I’ve broken the project down into milestones and will keep this thread up to date with where we are.

Milestone 1: Monorepo

  • Portal needs to be merged into the monorepo
  • Comments and search will follow later, but the project can continue without them.

Who: This milestone is currently being worked on by the Ghost team.

Milestone 2: Framework and build tools

  • Implement the “expected structure” from above
  • Get the basic pieces of i18next in place
  • Figure out if we need to switch to vitejs in Portal (may need help from Ghost team)
  • Have 1 interpolate string in each of email and Portal
  • Tooling for extracting strings & generating files

Who: Looking for interested contributors, Ghost team will help and/or kick this off if needed.

Milestone 3: Interpolated Strings

  • All strings in Portal and Member emails (signup, signin, subscribe) are interpolated using the {{t}} helper and the default, existing US English string as the key.
  • comments-ui and SodoSearch should also be in the monorepo by now, and so these strings can also be interpolated.

Who: Looking for interested contributors

Milestone 4: Translations

  • Tooling for extracting strings into new locale files should be perfected
  • Getting all strings translated for additional languages.

Who: Looking for interested contributors


Next steps: Once Portal is available in the monorepo, I’ll raise a GitHub issue for milestone 2 and link it here.


If you’re interested in helping with milestones 2-4 of this project, I’d love to hear from you in the comments below :point_down:

13 Likes

Best news in a long time!

I can work on the Swedish translation.

2 Likes

I can do German if you need anyone.

Is there any ETA for the milestones or at least the project as a whole?

1 Like

That’s good news. Clearly a must have for wider adoption of Ghost.

Awesome!
I can work on the pt-BR translation.
When can we start?

Good news, and the hackathon is a challenge. Let me know if I can be of help with the Spanish ES translation :hugs:

1 Like

Hi everyone :wave:

Milestone 1 has now been completed - the Portal codebase is now part of the Ghost monorepo.

This means we’re ready to start work on Milestone 2, which is tracked over on our GitHub repo here:

Milestone 2 is the first of two phases of development work needed to implement i18n support.

So far, we’ve only had interest from people looking to do translation work - this is truly great for when we get there - however this is a community project and it’s therefore really not in the spirit of this thread to be asking for ETAs on milestone 4 when you can clearly see noone has volunteered to work on implementing the feature yet.

By all means continue to leave comments if you’re interested in translation work. You can also use the controls on the thread to watch this topic and make sure you get notified of replies.

I will share regular progress updates if and when implementation work starts :slightly_smiling_face:

If there’s no interest in doing this from the community, the Ghost team will eventually pick up the work, however it’s unlikely to be for a while. This is being offered up as a structured community project to make it possible sooner seeing as there are so many people in our community who have voted for it.

6 Likes

Hannah, will PRs towards milestone 3 be helpful? Or hold off?

Hey @Cathy_Sarisky - PRs for milestone 3 would be too early right now as we don’t have the helpers yet to support such changes.

1 Like

Hello,
Any news about the m2 completion and Hacktoberfest results?
As many, I’m willing to offer translation but not able to be relevant at coding.
Thanks

2 Likes

Hey!
Could you elaborate a bit on these two? Would be good to know if there are hard rules or rather suggestions to consider.

I am not sure I get this right, is it about loading translations separately for specifics parts of the portal ? Any reasoning behind this? Is it because of speed considerations ?

Similar question here - have you considered (and opted out of) using dynamic components instead of per-local builds? Could you share the reasoning behind that choice?

Thanks in advance
Dan

Hi, I can work on the Bulgarian (bg_BG) translation when possible.

I can’t help coding unfortunately. Would it be a good thing if this community project was communicated through regular Ghost medias? A note in an email, Twitter and whatnot?

As for translation, I would like to help for the FR version, or more specifically the fr-CA version, as French is far from a homogenous language/region.

1 Like

I can work on the Russian translation.

I can help with the Hungarian translation.

Hello! I’ll be glad to work on Russian (ru) and Sakha (sah) translations.

I can help with the Spanish translation.

We now have an open beta for anyone who is interested in helping to submit translations:

3 Likes