Blog

  • How to Launch a 65Gbps DDoS, and How to Stop One

    posted by Keito
    2012-09-18 18:52:43
    'Yesterday I posted a post mortem on an outage we had Saturday. The outage was caused when we applied an overly aggressive rate limit to traffic on our network while battling a determined DDoS attacker. In the process of writing it I mentioned that we'd seen a 65Gbps DDoS earlier on Saturday. I've received several questions since that all go something like: "65Gbps DDoS!? Who launches such an attack and how do you defend yourself against it?!" So I thought I'd give a bit more detail.

    ### What Constitutes a Big DDoS?

    A 65Gbps DDoS is a big attack, easily in the top 5% of the biggest attacks we see. The graph below shows the volume of the attack hitting our EU data centers (the green line represents inbound traffic). When an attack is 65Gbps that means every second 65 Gigabits of data is sent to our network. That's the equivalent data volume of watching 3,400 HD TV channels all at the same time. It's a ton of data. Most network connections are measured in 100Mbps, 1Gbps or 10Gbps so attacks like this would quickly saturate even a large Internet connection.

    At CloudFlare, an attack needs to get over about 5Gbps to set off alarms with our ops team. Even then, our automated network defenses usually stop attacks without the need of any manual intervention. When an attack gets up in the tens of Gigabits of data per second, our ops team starts monitoring the attack: applying filters and shifting traffic to ensure the attacked customer's site stays online and none of the rest of our network is affected.

    ### So You Want to Launch a DDoS

    So how does an attacker generate 65Gbps of traffic? It is highly unlikely that the attacker has a single machine with a big enough Internet connection to generate that much traffic on its own. One way to generate that much traffic is through a botnet. A botnet is a collection of PCs that have been compromised with a virus and can be controlled by what is known as a botnet herder.

    Botnet herders will often rent out access to their botnets, often billing in 15 minute increments (just like lawyers). Rental prices depend on the size of the botnets. Traditionally, email spammers purchased time on botnets in order to send their messages to appear to come from a large number of sources. As email spam has become less profitable with the rise of better spam filters, botnet herders have increasingly turned to renting out their networks of compromised machines to attackers wanting to launch a DDoS attack.

    To launch a 65Gbps attack, you'd need a botnet with at least 65,000 compromised machines each capable of sending 1Mbps of upstream data. Given that many of these compromised computers are in the developing world where connections are slower, and many of the machines that make up part of a botnet may not be online at any given time, the actual size of the botnet necessary to launch that attack would likely need to be at least 10x that size. While by no means unheard of, that's a large botnet and using all its resources to launch a DDoS risks ISPs detecting many of the compromised machines and taking them offline.

    ### Amplifying the Attacks

    Since renting a large botnet can be expensive and unwieldy, attackers typically look for additional ways to amplify the size of their attacks. The attack on Saturday used one such amplification technique called DNS reflection. To understand how these work, you need to understand a bit about how DNS works.

    When you first sign up for an Internet connection, your ISP will provide you with a recursive DNS server, also known as a DNS resolver. When you click on a link, your computer sends a lookup to your ISP's DNS resolver. The lookup is asking a question, like: what is the IP address of the server for cloudflare.com? If the DNS resolver you query knows the answer, because someone has already asked it recently and the answer is cached, it responds. If it doesn't, it passes the request on to the authoritative DNS for the domain.

    Typically, an ISP's DNS resolvers are setup to only answer requests from the ISP's clients. Unfortunately, there are a large number of misconfigured DNS resolvers that will accept queries from anyone on the Internet. These are known as "open resolvers" and they are a sort of latent landmine on the Internet just waiting to explode when misused.

    DNS queries are typically sent via the UDP protocol. UDP is a fire-and-forget protocol, meaning that there is no handshake to establish that where a packet says it is coming from actually is where it is coming from. This means, if you're an attacker, you can forge the header of a UDP packet to say it is coming from a particular IP you want to attack and send that forged packet to an open DNS resolver. The DNS resolver will reply back with a response to the forged IP address with an answer to whatever question was asked.

    To amplify an attack, the attacker asks a question that will result in a very large response. For example, the attacker may request all the DNS records for a particular zone. Or they may request the DNSSEC records which, typically, are extremely large. Since resolvers typically have relatively high bandwidth connections to the Internet, they have no problem pumping out tons of bytes. In other words, the attacker can send a relatively small UDP request and use open resolvers to fire back at an intended target with a crippling amount of traffic.

    ### Mitigating DNS Reflection Attacks

    One of the great ironies when we deal with these attacks is we'll often get an email from the owner of the network where an open resolver is running asking us to shut down the attack our network is launching against them. They're seeing a large number of UDP packets with one of our IPs as the source coming in to their network and assume we're the ones launching it. In fact, it is actually their network which is being used to launch a network against us. What's great is that we can safely respond and ask them to block all DNS requests originating from our network since our IPs should never originate a DNS request to a resolver. Not only does that solve their problem, but it means there's a smaller pool of open resolvers that can be used to target sites on CloudFlare's network.

    There have been a number of efforts to clean up open resolvers that are currently active. Unfortunately, it is slow going and the default installation of many DNS clients still has them open by default. While we actively reach out to the worst offenders to protect our network, to protect the Internet generally there will need to be a concerted effort to clean up open DNS resolvers.

    In terms of stopping these attacks, CloudFlare uses a number of techniques. It starts with our network architecture. We use Anycast which means the response from a resolver, while targeting one particular IP address, will hit whatever data center is closest. This inherently dilutes the impact of an attack, distributing its effects across all 23 of our data centers. Given the hundreds of gigs of capacity we have across our network, even a big attack rarely saturates a connection.

    At each of our facilities we take additional steps to protect ourselves. We know, for example, that we haven't sent any DNS inquiries out from our network. We can therefore safely filter the responses from DNS resolvers: dropping the response packets from the open resolvers at our routers or, in some cases, even upstream at one of our bandwidth providers. The result is that these types of attacks are relatively easily mitigated.

    What was fun to watch was that while the customer under attack was being targeted by 65Gbps of traffic, not a single packet from that attack made it to their network or affected their operations. In fact, CloudFlare stopped the entire attack without the customer even knowing there was a problem. From the network graph you can see after about 30 minutes the attacker gave up. We think that's pretty cool and, as we continue to expand our network, we'll get even more resilient to attacks like this one.

    http://blog.cloudflare.com/65gbps-ddos-no-problem
  • Sir Tim Berners-Lee accuses government of 'draconian' internet snooping

    posted by Keito
    2012-09-06 20:47:39
    'The inventor of the world wide web, Sir Tim Berners-Lee, has accused the government of invading the privacy by monitoring internet use.

    Sir Tim warned that plans to monitor individuals' use of the internet would result in Britain losing its reputation as an upholder of web freedom

    The plans, by Theresa May, would force service providers to keep records of every phone call, email and website visit in Britain.

    Sir Tim told the Times: "“In Britain, like in the US, there has been a series of Bills that would give government very strong powers to, for example, collect data. I am worried about that."

    Yesterday was the launch of the World Wide Web Foundation's first global Web Index analysing the state of the web in 61 countries using indicators such as the political, economic and social impact of the web, connectivity and use.

    Britain came third in the list which was topped by Sweden and the United States in second place.

    Speaking at the launch, Sir Tim said that Britain would soon slip down the rankings if the draft Communications Data Bill became law.

    “If the UK introduces draconian legislation that allows the Government to block websites or to snoop on people, which decreases privacy, in future indexes they may find themselves farther down the list,” he said.

    The draft bill extends the type of data that internet service providers must store for at least 12 months. Providers would also be required to keep details of a much wider set of data, including use of social network sites, webmail and voice calls over the internet.

    Mrs May has justified the need for the new legislation by saying that it is necessary to combat organised crime and terrorism.

    Sir Tim's comments came on the same day as he denied that there was an 'off'; switch for the internet.

    He said the only way the internet could only ever be completely shut down is if governments across the world coordinated to make it a centralised system:

    "At the moment, because countries connect to each other in lots of different ways, there is no one off switch, there is no central place where you can turn it off.

    "In order to be able to turn the whole thing off or really block, suppress one particular idea then the countries and governments would have to get together and agree and coordinate and turn it from a decentralised system to being a centralised system.

    "And if that does happen it is really important that everybody fights against that sort of direction."'

    http://www.telegraph.co.uk/technology/internet/9524681/Sir-Tim-Berners-Lee-accuses-government-of-draconian-internet-snooping.html
  • RAP NEWS 15: Big Brother is WWWatching You

    posted by Keito
    2012-09-05 19:59:51
  • Unity is Strength

    posted by Keito
    2012-09-04 21:22:19
    Telecomix Crypto Munitions Bureau works for the benefit of cipherspace. Cipherspace is the state of crypto anarchy. This means that your identity is anonymous as long as you stay protected. There are no identities or authorities in cipherspace, and it is not possible to enforce laws where there is no identity, or where there are no authorities.

    Today there are several threats to the inhabitants of the internet. The politicians of oppressive regimes in the east and in the west, in north and south, are imposing surveillance. Surveillance of the entire networks. What people say to each other, what information is transmitted between bots and humans alike.

    This aggression must be met with the strongest encryption algorithms available to modern computers. With onion and garlic routing it is possible to erect the fractal cipherspace. With distributed hash tables it is possible to create networks that has no central node. There is no one that controls the fractal cipherspace. Internet as we know it, turns into darknet.

    Telecomix Crypto Munitions Bureau recommends that you use the following software: i2p, for anonymous and secure communications, Gnu Privacy Guard, for direct and verified communication. The onion router, TOR, to access the internets.

    Telecomix Munitions is a defense bureau.

    You can change the future of the internets by joining us in defending the networks and creating cipherspace.

    You can help defending yourself and your friends, yes, all inhabitants of the networks.

    By learning a few skills you can take control over technology.

    Telecomix munitions are currently developing and promoting advanced security devices, which can endure even the harshest forms of government or corporation surveillance.

    Your personal computer is an encryption device. Modern hardware can transform plain text to ciphertext with ease. So rapidly you barely notice the difference between unencrypted and encrypted data.

    The laws of mathematics are infinitely stronger than the laws of nations and corporations, as the human laws are really only ink on paper. The laws of mathematics, on the other hand, are the laws that define our very universe. With the use of modern crypto algorithms we can use this fact to defend free speech and the integrity of both bots and humans. Information is nothing but numbers, numbers governed not by human laws, but by the laws of mathematics.

    Networks that utilize the power of cryptography already exist. It will not be possible to stop the spread of the fractal cipherspace.

    To find out more, come to cryptoanarchy.org
  • Russia's Former KGB Invests In Political Propaganda Spambots

    posted by Keito
    2012-08-28 20:49:03
    '"The newspaper Kommersant reports that the Russian Foreign Intelligence Service (formerly part of the KGB) has invested 30 million roubles (USD $940,000) on 'blog and social network intelligence' programs (Google translation of Russian original). A small part of that money is used for surveillance and analytics, but 22 million roubles (USD $690,000) is invested in 'mass distribution of messages in social networks with a view to the formation of public opinion.' Which presumably can be rephrased as 'launching massive pro-Kremlin astroturfing propaganda spambots in order to stifle and undermine political dissent.' The brazen Russian government acknowledgement of this investment indicates that the Kremlin does not think of such activities to be in any way illegal or unethical. No words whether these spambots would respect any anti-spam laws or the Terms and Conditions of victim websites. But hey, now you can accuse anyone you disagree with online of being a 'KGB bot'!"'

    http://politics.slashdot.org/story/12/08/27/0555219/russias-former-kgb-invests-in-political-propaganda-spambots