The Battle of Beijing

At my current job, I take care of all things systems. That’s infrastructure, security, networking, in-house desktop support, and automation.  That’s a lot, I know, but someone has to do it.  Nonetheless, it’s a great job.  I’ve had a lot of jobs in my life, and this one is definitely top three.  That being said, there are a bunch of a-holes in China making my life very hard right now by constantly running DDOS (dynamic denial of service) attacks against our IP address space. They particularly like to target our Joomla and WordPress sites. I’ve been fighting with them off and on for a few weeks now, but this morning, they launched the mother of all attacks against us.  This was a geographically distributed, multi-datacenter DDOS attack.  They were using in excess of 100 IPs in separate subnets scattered across at least five countries.  So how did I fix this?  Let’s get into it.

There are a number of ways to handle DDOS attacks, but the two most common are simply block the perpetrators or scale out to handle the load.  I think the latter is an asinine approach, but I have first-knowledge that this is what Microsoft Azure recommends.  We are currently one of their customers.

I prefer the concept of just blocking the bad actors, and there are a number of ways to approach this.  If it’s a web application, you can utilize a .htaccess file.  If it’s something else, you use some type of firewall or IPS/IDS system.  For me, I’m old school and usually just revert to iptables.  It’s available on every Linux system and any systems administrator or engineer worth their salt knows at least the basics.

Historically, I try to act surgically, using iptables like a scalpel and not heavy artillery.  Over the last few weeks, I’ve been able to do this with the aforementioned hackers that had been scouting my weak spots via smaller DDOS attacks and penetration tests (pentests).  Unfortunately, their actions were just the tip of the spear preparing for the metaphorical beachhead amphibious assault.  Unfortunately, the surgical approach was no longer going to work and I needed to pull out the Humvee-mounted Browning .50 cal and lay down some serious suppression fire.  This ultimately culminated in me blocking the entire APNIC IP address range on our primary web server that serves a number of pages, both for us and our customers.  I did this with the local iptables on this specific server.

To save us all some time, I’m going to leave the rule set I’m using here.  If you find yourself in a similar situation, just plug this into your rules file (/etc/iptables/rules.v4) and run:

iptables-restore < /etc/iptables/rules.v4

One last thing, my boss is an old-school network engineer.  He prefers .htaccess to using iptables.  While .htaccess is much easier to setup a granular access protocol, it also works at layer 7 versus layer 4 of the OSI stack.  Additionally, because it does work at the application layer, it has a much larger impact on performance.  If you’re just blocking a handful of IP address or ranges, .htaccess will work just fine, but I have first hand experience of expensive .htaccess files slowing the performance on a web application.  It must be said that the .htaccess that caused this lag was in excess of 10,000 lines and had rewrites enabled.  However, I’ve had an iptables rule set that was blocking 500,000,000 IP addresses without any issues.

In conclusion, you must choose what works best for your environment and skill set, and “your mileage may vary,” but for me and my builds, we will stick with iptables.

 

 

Here is the rule set:

#
### block all Asian/APNIC IPs (includes NZ, AU)
#
-A INPUT -s 1.0.0.0/8 -j DROP
-A INPUT -s 14.0.0.0/8 -j DROP
-A INPUT -s 27.0.0.0/8 -j DROP
-A INPUT -s 36.0.0.0/8 -j DROP
-A INPUT -s 39.0.0.0/8 -j DROP
-A INPUT -s 42.0.0.0/8 -j DROP
-A INPUT -s 49.0.0.0/8 -j DROP
-A INPUT -s 58.0.0.0/8 -j DROP
-A INPUT -s 59.0.0.0/8 -j DROP
-A INPUT -s 60.0.0.0/8 -j DROP
-A INPUT -s 61.0.0.0/8 -j DROP
-A INPUT -s 101.0.0.0/8 -j DROP
-A INPUT -s 103.0.0.0/8 -j DROP
-A INPUT -s 106.0.0.0/8 -j DROP
-A INPUT -s 110.0.0.0/8 -j DROP
-A INPUT -s 111.0.0.0/8 -j DROP
-A INPUT -s 112.0.0.0/8 -j DROP
-A INPUT -s 113.0.0.0/8 -j DROP
-A INPUT -s 114.0.0.0/8 -j DROP
-A INPUT -s 115.0.0.0/8 -j DROP
-A INPUT -s 116.0.0.0/8 -j DROP
-A INPUT -s 117.0.0.0/8 -j DROP
-A INPUT -s 118.0.0.0/8 -j DROP
-A INPUT -s 119.0.0.0/8 -j DROP
-A INPUT -s 120.0.0.0/8 -j DROP
-A INPUT -s 121.0.0.0/8 -j DROP
-A INPUT -s 122.0.0.0/8 -j DROP
-A INPUT -s 123.0.0.0/8 -j DROP
-A INPUT -s 124.0.0.0/8 -j DROP
-A INPUT -s 125.0.0.0/8 -j DROP
-A INPUT -s 126.0.0.0/8 -j DROP
-A INPUT -s 169.208.0.0/12 -j DROP
-A INPUT -s 175.0.0.0/8 -j DROP
-A INPUT -s 180.0.0.0/8 -j DROP
-A INPUT -s 182.0.0.0/8 -j DROP
-A INPUT -s 183.0.0.0/8 -j DROP
-A INPUT -s 202.0.0.0/8 -j DROP
-A INPUT -s 203.0.0.0/8 -j DROP
-A INPUT -s 210.0.0.0/8 -j DROP
-A INPUT -s 211.0.0.0/8 -j DROP
-A INPUT -s 218.0.0.0/8 -j DROP
-A INPUT -s 219.0.0.0/8 -j DROP
-A INPUT -s 220.0.0.0/8 -j DROP
-A INPUT -s 221.0.0.0/8 -j DROP
-A INPUT -s 222.0.0.0/8 -j DROP
-A INPUT -s 223.0.0.0/8 -j DROP
-A INPUT -s 43.224.0.0/13 -j DROP
-A INPUT -s 43.236.0.0/14 -j DROP
-A INPUT -s 43.240.0.0/14 -j DROP
-A INPUT -s 43.245.0.0/16 -j DROP
-A INPUT -s 43.246.0.0/15 -j DROP
-A INPUT -s 43.248.0.0/14 -j DROP
-A INPUT -s 43.252.0.0/16 -j DROP
-A INPUT -s 43.254.0.0/15 -j DROP
-A INPUT -s 45.64.0.0/16 -j DROP
-A INPUT -s 45.65.0.0/20 -j DROP
-A INPUT -s 45.65.16.0/20 -j DROP
-A INPUT -s 45.65.32.0/19 -j DROP
-A INPUT -s 45.112.0.0/12 -j DROP
-A INPUT -s 45.248.0.0/13 -j DROP
-A INPUT -s 137.59.0.0/16 -j DROP
-A INPUT -s 139.5.0.0/16 -j DROP
-A INPUT -s 144.48.0.0/16 -j DROP
-A INPUT -s 146.196.32.0/19 -j DROP
-A INPUT -s 146.196.64.0/18 -j DROP
-A INPUT -s 150.107.0.0/16 -j DROP
-A INPUT -s 150.129.0.0/16 -j DROP
-A INPUT -s 150.242.0.0/16 -j DROP
-A INPUT -s 157.119.0.0/16 -j DROP
-A INPUT -s 160.19.20.0/22 -j DROP
-A INPUT -s 160.19.48.0/21 -j DROP
-A INPUT -s 160.19.64.0/22 -j DROP
-A INPUT -s 160.19.208.0/20 -j DROP
-A INPUT -s 160.19.224.0/22 -j DROP
-A INPUT -s 160.20.0.0/20 -j DROP
-A INPUT -s 160.20.40.0/21 -j DROP
-A INPUT -s 160.20.48.0/20 -j DROP
-A INPUT -s 160.20.72.0/22 -j DROP
-A INPUT -s 160.20.222.0/23 -j DROP
-A INPUT -s 160.202.8.0/21 -j DROP
-A INPUT -s 160.202.32.0/19 -j DROP
-A INPUT -s 160.202.128.0/17 -j DROP
-A INPUT -s 160.238.0.0/24 -j DROP
-A INPUT -s 160.238.12.0/22 -j DROP
-A INPUT -s 160.238.16.0/22 -j DROP
-A INPUT -s 160.238.34.0/23 -j DROP
-A INPUT -s 160.238.58.0/23 -j DROP
-A INPUT -s 160.238.64.0/19 -j DROP
-A INPUT -s 162.12.208.0/21 -j DROP
-A INPUT -s 162.12.240.0/21 -j DROP
-A INPUT -s 163.47.4.0/22 -j DROP
-A INPUT -s 163.47.8.0/21 -j DROP
-A INPUT -s 163.47.16.0/23 -j DROP
-A INPUT -s 163.47.18.0/24 -j DROP
-A INPUT -s 163.47.20.0/23 -j DROP
-A INPUT -s 163.47.32.0/21 -j DROP
-A INPUT -s 163.47.40.0/22 -j DROP
-A INPUT -s 163.47.44.0/23 -j DROP
-A INPUT -s 163.47.47.0/24 -j DROP
-A INPUT -s 163.47.48.0/20 -j DROP
-A INPUT -s 163.47.64.0/18 -j DROP
-A INPUT -s 163.47.128.0/17 -j DROP
-A INPUT -s 163.53.0.0/16 -j DROP
-A INPUT -s 192.26.110.0/24 -j DROP
-A INPUT -s 192.75.137.0/24 -j DROP
-A INPUT -s 192.135.90.0/23 -j DROP
-A INPUT -s 192.135.99.0/24 -j DROP
-A INPUT -s 192.140.128.0/17 -j DROP
-A INPUT -s 192.144.78.0/23 -j DROP
-A INPUT -s 192.144.80.0/20 -j DROP
-A INPUT -s 192.145.228.0/23 -j DROP
-A INPUT -s 192.156.144.0/24 -j DROP
-A INPUT -s 192.156.220.0/24 -j DROP
-A INPUT -s 192.188.82.0/23 -j DROP
-A INPUT -s 199.21.172.0/22 -j DROP
-A INPUT -s 216.250.96.0/20 -j DROP

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s