GitHub survives record-breaking DDoS attack that hitchhiked memcached servers

GitHub recently survived the largest-ever recorded distributed denial of service attack in history,  experiencing roughly 10 minutes of disruption during the onslaught, which was amplified using exposed memcached servers — a vector that has seen a significant increase in abuse since last month.

A Mar. 1 blog post by the GitHub Engineering team reported that the attack reached a peak of 1.35 terabytes per second, with a throughput of 126.9 million packets per second. The all-out assault rendered the web-based hosting service unavailable on Feb. 28 from 17:21 to 17:26 UTC and intermittently unavailable from 17:26 to 17:30 UTC.

Nevertheless, the site staved off the volumetric attack and quickly restored normal service, an achievement GitHub credits to doubling its transit capacity over the past year, as well as the decision to move traffic to Akamai Technologies’ content delivery platform, which was able to provide additional edge network capacity.

The post’s author, GitHub’s manager of site reliability engineering Sam Kottler, wrote that the attack “originated from over a thousand different autonomous systems (ASNs) across tens of thousands of unique endpoints.”

“The real issue here is not the DDoS attack itself, but the amplification mechanism used,” said Mounir Hahad, head of threat research at Juniper Networks, in comments emailed to SC Media. “Memcached is a service that helps websites scale, but it is supposed to be internal and never exposed to the internet. Yet, there are a little over 90,000 misconfigured installations accepting non-authenticated requests coming from the internet that could easily use spoofed return IP addresses. That’s a recipe for disaster.”

“It behooves us to remediate this issue immediately before the next wave of attacks hits,” Hahad continued. “In the meantime, it is recommended that all firewalls implement a rate limit on port 11211, or block it off completely.”

The use of memcached servers continues a rapidly emerging trend that was initially referenced by content delivery network provider Cloudflare, which reported earlier this week that misconfigured, publicly accessible memcached servers were being exploited via UDP (User Datagram Protocol) port 11211 to deliver malicious traffic. (Memcached is a free and open-source, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.)