Administrativa: Site Upgrade

TL:DR: This is a nerdy technical post about Web site administration. The site was down for several days due to a denial of service attack. It’s back up now.

Five days ago, I was notified by our hosting provider, Dreamhost, that our site had experienced a denial of service attack so they took it offline. I’m not sure who would actually want to attack a free, irregularly updated blog about economy class travel (maybe Xiamen Air wasn’t happy with my review), but I can state pretty definitively that DDoSing a site at Dreamhost is effective. Dreamhost totally punishes the victim. They completely removed our site, which had to be restored from a backup. After that, they required us to move it into Cloudflare. Dreamhost has some automated tools to do this, which they ran, but they didn’t actually work. Our site was left in a broken state. Three days later, after failing to solve the problem, Dreamhost more or less washed their hands of the problem and passed the buck to Cloudflare.

It was DNS meme

This time, it actually wasn’t DNS

I’ve been pretty busy and I normally give folks plenty of space to do their jobs, but five days of downtime is where my patience with junior tech staff exceeds its limit and I get personally involved. Ultimately, I blocked a few hours to roll up my sleeves and fix the problem (I have worked in IT pretty much my whole life, and am currently a senior information security architect).

The first thing I did was decouple the site from Dreamhost DNS which–inexplicably–is the default when you enable Cloudflare caching on a site hosted at Dreamhost. A million and one problems can be caused by DNS, so it’s generally a best practice to host DNS with the ISP that is actually serving your site (which, in the case of using Cloudflare, becomes Cloudflare). I thought there was a pretty good chance that this would fix the problem, but it only partially resolved the problem. The cipher mismatch error I was receiving when using SSL was replaced with a redirect loop.

I did some digging, and it turns out that there are two WordPress plugins that need to be installed in order to solve the problem. One fixes the redirect loop problem, and the other integrates WordPress with your Cloudflare account. Now, you’d think that Dreamhost support–given their extensive support for WordPress–might know about this, but somehow, they didn’t. Anyway, I wasn’t quite out of the woods when I did this, but at least the error message changed again. A quick search of the Cloudflare knowledge base revealed this could be fixed by setting SSL mode to Strict in Cloudflare. I made the change, and magically everything was working.

We’re back online. It took me about 4 hours to solve what Dreamhost couldn’t solve in 5 days. I apologize for the outage and the inconvenience. Hopefully the move to Cloudflare will have the side effect of making the site faster. And if you have any issues, please let me know.

Leave a Reply

Your email address will not be published. Required fields are marked *