Upgrade ghost from 4 to 5 unable to get cert

Developer help
1

I’m running ghost in kubernetes and I just upgraded from the latest version of 4.x.x-alpine to 5.0.0-alpine. I updated the ‘url’ environment variable to https, however I’m getting the following error:

[2022-07-13 21:52:44] ERROR The server has encountered an error.

The server has encountered an error.

Error ID:
    20318500-02f6-11ed-a6cb-356b7ff82f2b

Error Code:
    UNABLE_TO_GET_ISSUER_CERT_LOCALLY

----------------------------------------

RequestError: unable to get local issuer certificate
    at /var/lib/ghost/versions/5.3.0/core/server/adapters/scheduling/SchedulingDefault.js:319:23
    at ClientRequest.<anonymous> (/var/lib/ghost/versions/5.3.0/node_modules/got/source/request-as-event-emitter.js:178:14)
    at Object.onceWrapper (node:events:642:26)
    at ClientRequest.emit (node:events:539:35)
    at ClientRequest.origin.emit (/var/lib/ghost/versions/5.3.0/node_modules/@szmarczak/http-timer/source/index.js:37:11)
    at TLSSocket.socketErrorListener (node:_http_client:454:9)
    at TLSSocket.emit (node:events:527:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

I also tried the latest version but the same error.

Our URL is not public facing. Here is my config.production.json:

    {
        "url": "http://localhost:2368",
        "server": {
          "port": 2368,
          "host": "0.0.0.0"
        },
        "database": {
          "client": "mysql",
          "connection": {
            "host": "redacted",
            "port": 3306,
            "user": "ghost",
            "password": "redacted,
            "database": "ghost",
            "ssl": "Amazon RDS"
          }
        },
        "mail": {
          "from": "redacted",
          "transport": "SMTP",
          "options": {
            "host": "redacted",
            "service": "redactedr",
            "port": "redacted"
            }
        },
        "logging": {
          "transports": [
            "file",
            "stdout"
          ]
        },
        "process": "systemd",
        "paths": {
          "contentPath": "/var/lib/ghost/content"
        }
      }
  • How was Ghost installed and configured? kubernetes
  • What Node version, database, OS & browser are you using? Prior to upgrading, node was on v14.18.1. MySQL 8, OS is 5.3.0 alpine.
  • What errors or information do you see in the console? See above
  • What steps could someone else take to reproduce the issue you’re having? Deploy any version of ghost 4.x, then upgrade to 5.x
2

Are you using a FQDN? If so, that should be reflected in config.production.json.

However, it looks like you’re trying to do this for localhost, and that’s not possible with a certificate authority.

3

Hi mjw, I’ve tried changing url to a FQDN which matches the environment variable “url”. However, I still receive the same error.

4

A little more information about your setup would help immensely, and a link the the site. How do you proxy Ghost? How have you setup SSL?

5

sadly I can’t link to the site, it’s running internally only. Besides, it can’t run because of the cert issue.

To answer the other questions, we’re running this in kubernetes and in 4.x the url environment variable in our deployment was http://ghost.dev.local. The config.production.json had url: http://localhost:2368 and everything worked great.

SSL was handled through an ingress, with the cert being issued by letsencrypt tied to the ingress. So users would connect via https ghost.dev.local which would then point to ghost running on port 80 (targetPort 2368).

Since 5.x, we had to change the url environment variable to https://ghost.dev.local and this is where all the pain started.

6

Since you’re using config.production.json, I assume you’ve not done a local install, so url should be the public URL, not localhost. Then use Nginx to proxy 443 or 80 to the port Ghost runs on.

ghost.dev.local is not a FQDN, so Let’s Encrypt will fail, and the error message alludes to this. Therefore, either use port 80 if it is a intranet site, or port 443 and a self issued certificate.

Incidentally, I use a FQDN for my intranet sites, overriding DNS on the DHCP server, and simply control access on the perimeter firewall.

7

The problem is, if I use http://ghost.dev.local in the URL environment variable and in the config.production.json, Ghost gives a 301 permanent redirect to HTTPS followed by the cert issue.

Then use Nginx to proxy 443 or 80 to the port Ghost runs on.

This is how it was setup in 4.x. Ghost was running on port 80, then instead of nginx, we had a kubernetes ingress that was proxying 443 to 80. Let’s encrypt works in this case because the issuer for “.dev.local” is our internal CA. So as long as the user is on VPN and has the root CA installed on their machine, there are no cert issues.

However, 5.x prevents this setup from working.

8

It should proxy to the Ghost port, i.e., 2368, not 80. The redirects are handled by your proxy, not Ghost, so I think this is where things are misconfigured.

9

Sorry, I was too vague.

The ingress forwards to the service “postal”, which has this setup:

  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 2368

The ingress forwards to the service like so:

paths:
          - backend:
              service:
                name: ghost
                port:
                  name: http

So you’re correct, the ingress proxies to port 2368 (in this case the name is http) on the service.

10

Hello again,
I decided to start from scratch with a brand new 5.3.0 image, empty database, etc. Ghost installed correctly, initialized the database and was running. I went to the /ghost page to setup the admin user and did so correctly. Except when I try to login as that user I just created I get:

[2022-07-14 18:49:53] ERROR "GET /ghost/api/admin/users/me/?include=roles" 403 2ms

Authorization failed

"Unable to determine the authenticated user or integration. Check that cookies are being passed through if using session authentication."

Error ID:
    bfacbf50-03a5-11ed-8e77-c97f2078f1b2

----------------------------------------

NoPermissionError: Authorization failed
    at authorizeAdminApi (/var/lib/ghost/versions/5.3.0/core/server/services/auth/authorize.js:33:25)
    at Layer.handle [as handle_request] (/var/lib/ghost/versions/5.3.0/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/lib/ghost/versions/5.3.0/node_modules/express/lib/router/route.js:144:13)
    at authenticate (/var/lib/ghost/versions/5.3.0/core/server/services/auth/session/middleware.js:28:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

So I can’t login as the user and re-import everything.

11

Sounds like Kubernetes Ingress isn’t passing the authentication cookies. Indeed, it seems that your choice of infrastructure, and removing Nginx, is the cause of your problems.

12

The ingress doesn’t strip headers. I changed the image to 5.0.0 and I can actually login to the admin console. However, I’m met with a new error when trying to upload redirects from the previous installation:


EACCES: permission denied, open '/var/lib/ghost/content/data/redirects.json'

Error ID:
    fae31370-03ab-11ed-bdc3-69044444ce1b

Error Code:
    EACCES

----------------------------------------

Error: EACCES: permission denied, open '/var/lib/ghost/content/data/redirects.json'
    at module.exports.prepareError (/var/lib/ghost/versions/5.0.0/node_modules/@tryghost/mw-error-handler/lib/mw-error-handler.js:79:19)

I tried port-forwarding to the pod so I’d be skipping the ingress and I get the same error. Again, none of this happened in 4.x, even the latest 4.48.2 I believe.