Jannis is working hard to get MagicPages back online

Using Ghost
1

Hey guys, I noticed that there is no post about the current outage with MagicPages, so I figured I’d share a post about it here.
I contacted Jannis and he got back to me quick and instructed me to watch https://health.magicpages.co/

Currently:

Earlier today, two server nodes went offline. I have restored them in due time, however, an hour later, the issue has popped back up. Right now, it looks like Hetzner, the infrastructure provider, does not have enough servers available.

I am shifting to different server nodes in the meantime, hoping that this will eliviate pressure. Unfortunately, a small set of websites is affected by this and is currently offline.

Update 15:11 UTC: I have provisioned the new servers, however, these are in a different architecture (AMD64) compared to the old ones (ARM). I am rebuilding Docker images at the moment to get things up and running again.

Update 17:41 UTC: Getting an AMD64 image ready is turning out to be a lot more difficult than anticipated, since the Kubernetes cluster is running on two architectures now (some leftover ARM nodes, and new AMD64 nodes). Each image build takes about 30 minutes, and I am a few tries in. Will keep things updated here.

I’m sure many of you were worried or confused as I was. But @Jannis is on the case!

1 Like
2

Update 19:57 UTC: Right now, there are about 50 websites still affected. As server nodes become available, these will come back online. I am monitoring the progress, and actively add more server where I can.

1 Like
3

Update: my website is back online, I suspect that @Jannis got the final stretch complete.

1 Like
4

My site has been down all day. I’m hoping Jannis will help resolve it soon.

1 Like
5

Hey Rōy! Your site’s actually one that has not been affected by this :thinking:
Any chance it’s a cache issue? Just checked again, but it’s accessible for me from different devices.

2 Likes
6

Thanks Jannis, I received and replied to your email!

1 Like