Failing to setup SMTP

Aleklu · January 15, 2023, 9:48am

Hi everyone,

I am struggling to get the SMTP setup working for my self hosted Ghost installation. All the things I tried so far ended with “Failed to send magic link email” in Portal and “Failed to send email. Reason: Invalid login: 535 Authentication failed.” in the logs.

My knowledge leaves a lot to be desired, but maybe someone could help me still.

My setup:
Running Ghost on my own hardware on top of Unraid. I am using the official Ghost Docker image available on Unraid. I am using Cloudflare for my domains, DNS und SSL. I also have to fight with a dynamic IP, so my router talks to Cloudflare in the background - Cloudflare provides my SSL cert. I am using NGINX Proxy Manager, again on Unraid, to proxy my host to the domain known to Cloudflare. For both mass mails and transactional mails I am trying to get Mailgun working following the official docs. (I hope I used some of the correct words here.^^)

Ghost-CLI version: 1.24.0
Ghost version: 5.30.0 (at /var/lib/ghost)

Current open questions:

Where can I find config.production.json? My current very limited understanding is that something is cobbling together Ghost core during runtime. I can see both config.development.json and config.production.json and more with a “ls” in the console of the docker image. But for the life of me I cant find the “actual file” to edit.
- I think I understood correctly that I can just put a config.production.json in the root and it’ll override settings. But this does not seem to work in the /appdata/ghost folder, which is the only Ghost root I can see in Unraids file system. Should this work? Do I have a complete misunderstanding here? Where is root and the files?
- (Adjacent question - I managed to create a DE.json and place the translation handlebar in a copy of one of the default themes. Works good. How can I translate Core and/or Portal? Some content is missing from the theme but would need translation anyway. And again: where are the files? I read in some forum post here that core translation is on the roadmap - wait for progress or learn to do it myself?)
Assumed the root mentioned above actually is the correct root, is anything wrong with my config below?

"mail": {
	"transport": "SMTP",
	"options": {
		"service": "Mailgun",
		"host": "smtp.eu.mailgun.org",
		"port": 465,
		"secureConnection": true,
		"auth": {
			"user": "USER@mg.MYDOMAIN.XYZ",
			"pass": "PW DEFINED IN MAILGUN SMTP CREDENTIALS"
		}
	}
},

I tried many many iterations of this config. “secureConnection” true/false, “SecureConnection” renamed to “Secure”, port 465 & 587, host with “EU” and without. Nothing changed the error message in any way - leads me back to Ghost ignoring this config completely.

Any other troubleshoot steps you would recommend? Thanks a million in advance!

Aleklu · January 15, 2023, 9:55am

EDIT: Mass mailing works - I just tried to send a newsletter to myself. That’s great, at least this part of the Mailgun integration is good for me.

Now, how to diagnose and troubleshoot the SMTP stuff…

mjw · January 15, 2023, 4:20pm

Whilst config.production.json is used when setting up SMTP for a standard installation, Docker does it differently.

You need to supply environment variables using docker-compose, e.g., mail__transport=SMTP.

https://hub.docker.com/_/ghost

Aleklu · January 15, 2023, 9:33pm

Thanks for the pointer. I now created several more environment variables that seem to work. I also, for testing, set up gMail as the SMTP sender and this worked - but only after I left the variables for “host” and “port” empty. Seems like Nodemailer (?) is picking all this up automagically when “service: gmail” is set.

While looking at this I also found Proxy support :: Nodemailer - my Ghost installation IS behind a proxy, NGINX proxy manager. I guess I somehow need to tell Nodemailer that it sits behind a proxy. I don’t really know if this is also done via the docker env variables of Ghost. And the Nodemailer help site linked is currently overwhelming me.

Oh well, more to learn next week. Any additional pointers would still be much appreciated.

EDIT: In case this is helpful - I followed this tutorial to get to where I am: Self-Hosting a Blog with Ghost and Unraid.

Aleklu · January 16, 2023, 8:46am

I could verify with telnet that my Mailgun SMTP credentials in fact DO work, at least via telnet on ports 25 and 587.

So I tried to mirror my telnet input in the env variables as so:

mail__transport "SMTP"
mail__options__service "Mailgun"
mail__options__auth__user "My Mailgun SMTP user"
mail__options__auth__pass "My Mailgun SMTP pw" OR "My Mailgun SMTP pw in BASE64"
mail_options_port "587"
mail_options_secure "false"
mail_options_host "smtp.eu.mailgun.org"
mail_options_auth_type "auth login" OR "login" OR ""

When setting up the env variables as described I still get

Failed to send email. Reason: Invalid login: 535 Authentication failed.

"Please see https://ghost.org/docs/config/#mail for instructions on configuring email."

Error ID:
    3fd839b0-9579-11ed-a8a0-bfb98a8f37af

Error Code: 
    EAUTH

----------------------------------------

Error: Invalid login: 535 Authentication failed
    at createMailError (/var/lib/ghost/versions/5.30.0/core/server/services/mail/GhostMailer.js:68:12)
    at SMTPConnection._formatError (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:790:19)
    at SMTPConnection._actionAUTHComplete (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:1542:34)
    at SMTPConnection.<anonymous> (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:546:26)
    at SMTPConnection._processResponse (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:953:20)
    at SMTPConnection._onData (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:755:14)
    at TLSSocket.SMTPConnection._onSocketData (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:193:44)
    at TLSSocket.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at TLSSocket.Readable.push (node:internal/streams/readable:228:10)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)

So it looks like something gets malformed between env variables and Ghost talking to Mailgun. Any additional ideas would be lovely.

thimiraonline · January 16, 2023, 11:07am

Try this configuration. This is what I am using for my website’s SMTP in config.production.json file.

  "mail": {
    "transport": "SMTP",
    "options": {
      "service": "Mailgun",
      "host": "smtp.mailgun.org",
      "port": 587,
      "secure": false,
      "auth": {
        "user": "user address",
        "pass": "password"
      }
    }
  },

Aleklu · January 16, 2023, 11:38am

Unfortunately does not work either.

My current config

I tried this and many iterations of it.

markstos · January 16, 2023, 1:25pm

For config options that would be nested in JSON, you need to separate the config keys with two underscores, but some of your config keys here use a single underscore, which won’t work.

Also, if you are going to try quoting them, quote the whole string, ex: ‘mail__logger=’

Aleklu · January 16, 2023, 3:24pm

Thanks, that brought me a step further - I am getting a different error message now.

[2023-01-16 14:37:22] ERROR "POST /members/api/send-magic-link/" 500 48ms

Failed to send email. Reason: connect ECONNREFUSED 127.0.0.1:587.

"Please see https://ghost.org/docs/config/#mail for instructions on configuring email."

Error ID:
    498a4ca0-95ab-11ed-83ee-1f539c442ef4

Error Code: 
    ESOCKET

----------------------------------------

Error: connect ECONNREFUSED 127.0.0.1:587
    at createMailError (/var/lib/ghost/versions/5.30.0/core/server/services/mail/GhostMailer.js:68:12)
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16)

I omitted almost everything from my env variables now:

-e 'mail__transport'='SMTP'
  -e 'mail__options__service'='Mailgun'
  -e 'mail__options__auth__user'='USER@mg.MYDOMAIN.XYZ'
  -e 'mail__options__auth__pass'='USERS PASSWORD FROM MAILGUN'

The 127.0.0.1 in the error message now seems curious. One the one hand: No email setup in Ghost? - Reclaim Cloud - Reclaim Hosting Community Forums

I’ve updated and tested our installer to work better with email setups other than Mailgun. Ghost has a built-in fallback email option called “direct email” via the Nodemailer package it uses internally, and in my testing it doesn’t seem to work in Docker based setups.

But there the recommendation is to use Mailgun as external SMTP server, which is exactly what I am trying to do in the first place.

mjw · January 16, 2023, 3:31pm

You need to add RequireTLS as an environment variable. But double check syntax as I’m on mobile, so haven’t checked.

Additionally, Nginx is proxying HTTP and HTTPS, so won’t impact mail delivery.

Aleklu · January 16, 2023, 4:26pm

Once more thanks for the input. It didn’t change the results unfortunately.

This config - which gets loaded correctly -

  -e 'mail__transport'='SMTP'
  -e 'mail__options__service'='Mailgun'
  -e 'mail__options__auth__user'='User@mg.MYDOMAIN.xyz'
  -e 'mail__options__auth__pass'='Mailgun User PW'
  -e 'mail__options__port'=''
  -e 'mail__options__secure'='true'
  -e 'mail__options__host'=''
  -e 'mail__options__requireTLS'='true'

as well as false-false, true-false and false-true for secure and requireTLS also lead to the same result:

[2023-01-16 16:20:21] ERROR "POST /members/api/send-magic-link/" 500 122ms

Failed to send email. Reason: connect ECONNREFUSED 127.0.0.1:465.

"Please see https://ghost.org/docs/config/#mail for instructions on configuring email."

Error ID:
    acdb3950-95b9-11ed-b2e0-33769715db9a

Error Code: 
    ESOCKET

----------------------------------------

Error: connect ECONNREFUSED 127.0.0.1:465
    at createMailError (/var/lib/ghost/versions/5.30.0/core/server/services/mail/GhostMailer.js:68:12)
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16)

(same result on port 587 when secure = false).

Once again for your help and input, sorry for being annoying.

mjw · January 16, 2023, 4:35pm

Secure needs to be false; TLS starts the negotiation using plain text.

Are you using the key from Mailgun as the password (only displayed when setting up?)

markstos · January 16, 2023, 4:36pm

Your host value is empty, so it’s trying to connect to port 465 on your localhost.

mjw · January 16, 2023, 4:38pm

Good spot; this should resolve it.

Port and host must be populated.

Aleklu · January 16, 2023, 4:45pm

If it only would be so easy - but no, now I have come full circle. Error:

Failed to send email. Reason: Invalid login: 535 Authentication failed.

"Please see https://ghost.org/docs/config/#mail for instructions on configuring email."

Error ID:
    680c24d0-95bc-11ed-b4ef-abbf45b7ddda

Error Code: 
    EAUTH

----------------------------------------

Error: Invalid login: 535 Authentication failed
    at createMailError (/var/lib/ghost/versions/5.30.0/core/server/services/mail/GhostMailer.js:68:12)
    at SMTPConnection._formatError (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:790:19)
    at SMTPConnection._actionAUTHComplete (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:1542:34)
    at SMTPConnection.<anonymous> (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:546:26)
    at SMTPConnection._processResponse (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:953:20)
    at SMTPConnection._onData (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:755:14)
    at TLSSocket.SMTPConnection._onSocketData (/var/lib/ghost/versions/5.30.0/node_modules/nodemailer/lib/smtp-connection/index.js:193:44)
    at TLSSocket.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at TLSSocket.Readable.push (node:internal/streams/readable:228:10)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)

Config (tried port empty - default set depending on secure state; port=587 and port=465):

  -e 'mail__transport'='SMTP'
  -e 'mail__options__service'='Mailgun'
  -e 'mail__options__auth__user'='USER@mg.MYDOMAIN.xyz'
  -e 'mail__options__auth__pass'='MAILGUN USER PW'
  -e 'mail__options__port'=''
  -e 'mail__options__secure'='false'
  -e 'mail__options__host'='smtp.eu.mailgun.org'
  -e 'mail__options__requireTLS'='true'

And keep in mind - those credentials DO work in telnet with 587 (AFAIK telnet out of the box does not do SSL, hence 465 does not work in telnet).

I am really out of my depths here…^^

Aleklu · January 16, 2023, 5:11pm

Wohoo… great success. Sort of.

I think this is what happened:

Mailgun recommends during domain setup to setup a subdomain for Mailgun, e.g. mg.mydomain.com. I did so.

Because my SSL certs from Cloudflare only cover ONE level of subdomains for free, email.mg.mydomain.com was out of scope for the cert.

Hence I was unable to connect to Mailgun via SMTP. Somehow. I guess. I would be lying if I said I understand what’s going on.

I now created a new domain in Mailgun that omits the subdomain, so mydomain.com. After setting up all the needed DNS entries my sites portal now can send emails.

Yay.

I somehow feel not satisfied because I didn’t really fix it, but alas. Sidestepping is good enough, I think.

mjw · January 16, 2023, 5:17pm

Your MX records should be something like mg.your.domain, and optionally a CNAME for email.mg.your.domain for tracking.

If your MX records were email.mg.your.domain, that would explain the error. However, Mailgun usually validates the domain in the control panel.

Aleklu · January 16, 2023, 5:21pm

I set them up correctly with the mg.mydomain.com subdomain - both MX pointed to mg.mydomain.com, not email.mg.mydomain.com. As you say, the CNAME was the only one pointing to email.gm. .... I checked a million times. (And all but the CNAME also had the verified badge. The CNAME shows unverified in Mailgun, and also in Cloudflare with the note that its not covered by my certificates.)

So if it was not that then I don’t know. I changed nothing in my config but the new credentials for the new Mailgun domain, omitting the subdomain. I would be really curious to understand what the error was/is.

Topic		Replies	Views
Installed Ghost on Docker. But can't setup SMTP Self-hosting	4	1872	October 5, 2022
"Update:Ghost Unable to send emails" Integrations & API	14	373	March 19, 2023
SMTP Setup Issues Developer help	16	425	September 20, 2023
Can't figure out how to setup email Developer help	7	6532	December 24, 2018
Failed to send email. Reason: Invalid login: 535 Authentication failed Self-hosting	13	149	April 9, 2024

Failing to setup SMTP

Related Topics