Mailgun not working when on Digital Ocean (Ghost running with docker-compose). 504 in web ui, but 200 in docker logs

Ghost is running with docker-compose, on a Digital Ocean droplet. I have setup Mailgun and am loading in variables via .env all on par with the documentation. I use nginx (nginx-proxy-manager also via docker-compose) and Cloudflare for DNS+SSL. The Ghost webui loads fine on https, however emails are not sending.

Here is my docker-compose.yml file:

version: '3'
services:
  ghost:
    image: ghost:5
    restart: always
    ports:
      - ${PORT_BLOG}:2368
    environment:
      database__client: mysql
      database__connection__host: db
      database__connection__database: blog_db
      database__connection__user: db_user
      database__connection__password: ${MYSQL_PASSWORD}
      url: "https://blog.mydomain.com"
      mail__transport: "${MAIL_TRANSPORT}"
      mail__options__host: "${MAIL_HOST}"
      mail__options__port: "${MAIL_PORT}"
      mail__options__secureConnection: "${MAIL_SECURE_CONNECTION}"
      mail__options__auth__user: "${MAIL_USER}"
      mail__options__auth__pass: "${MAIL_PASSWORD}"
    volumes:
      - ./ghost:/var/lib/ghost/content
    links:
      - db

  db:
    image: mysql:8.0
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: blog_db
      MYSQL_USER: db_user
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    cap_add:
      - SYS_NICE
    volumes:
      - ./mysql:/var/lib/mysql

The problem: When users try to subscribe, the /send-magic-link is not working. The behavior is a 504 timeout for /send-magic-link inside the user’s web UI and email is never sent. When I run docker logs {container_id} it says the /send-magic-link request was a 200 success (with 90006ms response time… so still seems like a timeout to me… but interesting they show differently).


When I use the same exact docker-compose.yml file on my local dev machine and point Cloudflare DNS to my dev machine, I can send emails fine! So I know Mailgun is fine and my .env settings are loading fine. Also, when I SSH into my Digital Ocean droplet, I can send SMTP emails via swaks, like this:

    ./swaks --auth \
        --server smtp.mailgun.org \
        --au postmaster@mail.mydomain.com \
        --ap <omitted> \
        --to testuser123@gmail.com \
        --h-Subject: "Hello" \
        --body 'Testing some Mailgun from SMTP!'

Since this^ works, I don’t think Digital Ocean is blocking SMTP emails (which was my first suspicion). However, I am still unsure why emails cannot send when hosted on Digital Ocean droplet. The behavior is easy to reproduce for me by having a new user attempt to subscribe and monitoring the /send-magic-link requests, but it also appears anytime emails are attempted to send they have the same 504 timeout behavior.

The Digital Ocean ufw firewall rules are set to allow in/out for ports 587, 465, 443, 80, 25, etc. I have also tried .env values for ${MAIL_SECURE_CONNECTION} to be set to true and false, and tried ${MAIL_PORT} as 587 and 465. No luck on any combination that I could find, unfortunately.

I’ve been jamming my head at the wall trying to resolve this, and I am wondering if anyone has ran into this before or has recommended steps to I could try to debug this further.

Any idea what I could try to fix this? Thank you in advance for taking a look :pray:

When you mentioned you checked docker logs, you didn’t mention /which/ container. Was it the Nginx container or the Ghost container?

Try opening the logs of the Ghost container and watching them live while you test initiating a send through your browser. If it fails, there should be more logging besides the basic HTTP status details.

The logs were from the Ghost container. Nginx runs on a separate docker container on the same host machine, and is routing the address and https correctly for the Ghost web ui. The same setup with nginx and ghost like this works on my dev machine, but I agree it could be some kind of container communication issue. I tried to open ports 587 and 465 on the docker-compose Ghost container but that did not work either. In Nginx, I forward the http traffic to the Ghost web container the port (2368) and the web ui is loading fine with that. It is just not sending the Mailgun mail.

Here’s some more logs from the Ghost container when I try to do a Mailgun bulk newsletter this time:

[2023-08-10 14:22:44] ERROR ECONNABORTED
[2023-08-10 14:22:44] ERROR [BULK_EMAIL_DB_RETRY] Sending email batch 64d4f2782132b20001fb11bd - After 0 retries

[BULK_EMAIL_DB_RETRY] Sending email batch 64d4f2782132b20001fb11bd - After 0 retries

"Mailgun Error 400: timeout of 60000ms exceeded"
"https://ghost.org/docs/newsletters/#bulk-email-configuration"

Error ID:
    5f8a2400-3789-11ee-88a7-3b8c747e01a4

Error Code:
    BULK_EMAIL_DB_RETRY

The bulk email send returns a 400, but looks like the same timeout issue.

Confusingly, Ghost has two different ways to send mail, and each requires different configuration.

For basic transactional emails, SMTP is used. For bulk email, the Mailgun API is used.

The error message you are sharing is referring to bulk email. The error message provides this link:

Following the link, it describes how to set up your Mailgun API key for bulk sending. Have you done that?

Yes, I have done that. And I tested the same settings on a local dev machine (and pointed the same Cloudflare DNS + Nginx SSL setup to my dev machine instead of prod Digital Ocean) and Mailgun SMTP and Bulk API mail works when coming from my dev machine. So I am pretty sure the Mailgun settings are correct. Happy to share any config files if it can help with debugging though.

Check on the Mailgun side to see if there’s any logging there.

Where exactly is Ghost trying to reach out to when sending the Mailgun SMTP or API messages? Is there another port I need to open other then the Ghost web-ui port (2368)? It seems like the Ghost container cannot reach out to Mailgun relay for some reason, but it only happens when I host the containers with Digital Ocean.

Mailgun does not see the request happen at all, unfortunately. It never hits the request on Mailgun’s side. I did ask Digital Ocean support, and they told me my droplet’s IP is not SMTP blocked.

The Mailgun API doesn’t use SMTP, it uses HTTPS. That’s confusing because transactional emails do to mailgun via SMTP.

It’s not clear why Ghost doesn’t use the API for everything.

Scroll down here to find some example calls to Mailgun. Try sending a basic message through your server directly to Mailgun.

If that works, try again, but inside your Ghost container

https://documentation.mailgun.com/en/latest/api-sending.html#examples

I realize containers help with isolation and use them in production myself, but for my personal Ghost installs, I use the official instructions that forego containers. It eliminates the whole layer of bugs that could possible exist in the container configuration layer.