Summer of 2008

Let’s start this thing by stepping back a few years in time.

The summer of 2008 was a BIG one for us aspiring network security headz. It definitely deservers a revisit. By that time I personally had reached the sufficient level of technical understanding to truly appreciate and enjoy the science and the art of the research published that summer. It was intellectually a very inspiring season for me. True eye opener to the possibilities of elite network hacking.

Some of the research summarized below got somewhat written off in the mainstream press as already known issues (you know the “network protocols were not designed with security in mind” response), but actually two very foundational network attacks got major updates published during the long hot summer of 2008.

The summer kicked off in grand manner with the public release of the Simple Network Management Protocol (SNMP) version 3 HMAC Authentication Bypass vulnerability in the beginning of June. It got documented with the CVE-2008-0960. The vulnerability allowed the attacker to possibly authenticate their arbitrary SNMP messages by getting only the first byte correct of any HMAC code of a valid username. Yep. That serious. Even without any knowledge of a valid username, the attacker had 1 in 256 chances to get it right with any byte sent. Fair.

Here the protocol was not broken, but rather the implementations of the protocol turned out to be vulnerable and as is common with this type of infrastructure software, the same SNMP implementation code is used by multiple vendors. The list of affected devices and systems in the US-CERT Vulnerability Note VU#878044 took time to scroll.

The SNMP version 3 was considered as a major upgrade of the protocol. It introduced security to the SNMP definitions. Version 3 was defined by the RFC 3411 and the RFC 3418. The Internet Engineering Task Force (IETF) later declared it an Internet Standard (STD0062) recognizing the full maturity of the RFC. The older versions of the protocol are considered as “obsolete” after the full release of SNMPv3 in 2004.

The security additions in version 3 largely centered on the use of Hash-based Message Authentication Code (HMAC) with SNMP messages. As is usual with the keyed hash function output, HMAC can be used to verify both the integrity and the authenticity of the messages. Both MD5 and SHA-1 are widely used to calculate HMAC codes, but practically any cryptographic hash function can be used as long as both participating entities know the chosen function and the secret key used in the hash encryption.

This far all good. We have a well secured SNMP messages that actually can be trusted to deliver the delicate service expected from them. But not quite. Turns out that a vast majority of the actual implementations of the new SNMP version included a peculiar detail in their functionality.

The clients were allowed to explicitly specify the applied HMAC length and HMAC codes with the minimum length of 1 byte were happily accepted for authentication. I do not know all the details behind the design error, but it would make an interesting study without a doubt. I find it curious that so many different development teams repeated the mistake that frankly speaking sounds incomprehensible to a n00b. As far as I can see the involved RFC documents can not be blamed here.

Exploit for the vulnerability was quickly published and is still available although this one should serve perfect for any private exploit development practices. The vulnerability is strikingly simple and straight forward despite the graveness of the threat it presents.

Maybe something to bounce back to with a future blog entry. The intention is to cover the full spectrum of IT security topics although the focus of the blog may be set somewhere closer to the attack and response tactiques for now.

As the summer of 2008 got hotter, so increased the heat on the internet infrastructure. The suspicions rose around July 8th when multiple vendors suddenly released very similar looking patches to address an issue in the source port assignment of the Domain Name System (DNS) queries. The patches all randomized the source port choice further. Hmm. Ok. US-CERT released the Vulnerability Note VU#800113 to publicly acknowledge the issue alongside the patches and the vulnerability was later documented with CVE-2008-1447.

Then appears a dude named Dan Kaminsky publicly asking for everybody to install the patches. Install them fast. The most critical DNS vulnerability ever had been found. One that gives you anything you want with DNS in the internet.

The big one, but to know the details, the community had to wait until the Kaminskys presentation in the Black Hat USA 2008 scheduled for August 6th.

The noise was loud. A lot of people had issues with Kaminskys chosen method of disclosure, there was the “network protocols were not designed with security in mind” all over the mainstream media for a moment, but there was also private disclosure by Kaminsky to three fellow researchers, which eventually led to public leak of the exploit description on July 21st.

In fact Kaminskys method was even used in the wild against an AT&T server in Texas, USA, before Kaminsky gave the presentation in the Black Hat. The google.com DNS entry for the local AT&T Internet subscribers was poisoned to point the traffic to attackers’ lookalike Google, which on the side hosted some ad-clicking services.

Despite all the excessive press prior to the presentation, I personally found Kaminskys research awesome, when it was published in the beginning of August. Chilling out at home with his laptop, trying to break things up, concentrating this time on DNS. He almost accidently stumbles upon an issue, but he was also capable of analysing it to the point that he understood the underlying root cause and could imagined the potential of the findings. He really had the global DNS administration privately handed over to him. king-in-the-middle position anyone? pwning traffic at will almost.

Kaminsky noticed that not only a singular DNS Resource Record can be poisoned by DNS flooding. Entire zone authorities can be hijacked using similar method. He proved that also the NS record AKA the authoritative name server field in the replies can be poisoned allowing him complete DNS control of the affected zone.

It would be quickly noticed by the network administrators due to lack of service? Not, if you do not deny the service, but just proxy it instead. The vulnerability allows true man-in-the-middle position for ALL desired DNS dependent traffic in the affected domain.

The attack is fairly simple. Once you can determine the DNS message ID (TXID) used by a recursive DNS server in its outbound queries, you can attack it by flooding it with DNS replies. Flooding, until you get the TXID correct before the legitimate source does and get your data cached by the recursive DNS server or the client. You get also to determine the Time To Live (TTL) for the record in cache. The maximum specified in the RFCs is over 68 years, but the servers in the public networks seem to generally cache entries for days or weeks maximum.

DNS is largely defined in the RFC 1034 and RFC 1035. The RFC 2181 was later published to clarify some details and "there might be some others as well" (as is the norm with RFCs). The message ID aka TXID is defined in RFC 1035 in the section 4.1.1. as a 16-bit field in the DNS message header. Due to the 16-bit size there is a limited supply (64k) of different message IDs and limited possibilities to randomize the TXIDs to provide a level of integrity to the query sessions when using User Datagram Protocol (UDP).

It’s a known weakness in the DNS protocol. The DNS security has been boosted up in the server implementations by having the DNS server to preallocate multiple UDP ports to use for the DNS queries and then use them randomly adding an extra randomization layer to "secure" the sessions.

This logically forces the attacker to predict an additional entry in order to successfully forge a response. He needs to get both the TXID _and_ the source port correct to get his response treated as the trusted one.

The patches released for the Kaminskys DNS bug reinforced the source port randomization. The DNS message headers obviously could not be redesigned and reimplemented successfully in one summer, so only the additional layers of defense had to be updated for the time being. The actual vulnerability that Kaminsky detected was that the randomization used by the majority of the DNS servers was not random enough to resist an attack for many seconds. Update was needed indeed. The Microsoft’s DNS update is said to have increased the source port variety from 64k to 134M.

There were some interesting discussions about moving the public DNS servers to use Transmission Control Protocol (TCP) for communications instead of UDP, but apparently this would cause too much load to the current global network infrastructure.  Apparently there are just simply so many DNS queries performed constantly that adding the TCP handshake traffic to each query would overwhelm the infra according to a lot of network administrators. As far as I can see, in order to implement DNSSEC succesfully we need to move the DNS traffic to use TCP anyway.

Using TCP would in any case add a third random variable to the queries. The sessions would be controlled additionally by the 32 bit TCP Sequence Numbers. By the DNS protocol specifications both TCP and UDP queries are allowed and majority of the DNS server applications support both, so using TCP could be considered as a mitigating factor to further secure the local recursive DNS server caches.

The various open DNS services available in the public networks scored well in defense by the way. I believe Kaminsky himself confirmed that OpenDNS already had the mitigation implemented and never was vulnerable to the attack. PowerDNS and MaraDNS have also stated that they both had the Query ID _and_ the source port heavily randomized already before Kaminskys findings and were never vulnerable.

Interestingly only one publicly available DNS server application seemed to be non-vulnerable. DJBDNS server authored by mister Daniel J. Bernstein always had the full mitigation in place even before Kaminsky discovered the vulnerability. Respect. Pretty much all other widely used DNS servers were vulnerable.

The summer was not over yet. In fact the August holiday season was only beginning. There were many sunny days left still in 2008.

I didn't know DEFCON originally meant "defense readiness condition" of the US armed forces. For me it has always been the biggest hacker convention of them all held annually in Las Vegas, USA. The Chaos Communication Congress held in Berlin annually seems to be the oldest one by the way. Chaos Computer Club has been organizing the Congress continuously since 1984.

The DEF CON 2008 was held (as is usual) in the end of August. That year the conference finished off with an unscheduled presentation given by Alex Pilotov and Anton Kapela. The talk was titled Stealing The Internet - An Internet-Scale Man In The Middle Attack. The community was just getting in grips with the near complete pwnage capabilities of the Kaminskys DNS cache poisoning attacks. We certainly did not expect another complete network pwnage to surface so soon.

Border Gateway Protocol (BGP) is one of the core routing protocols in the Internet. You got this blog post delivered to your browser largely due to BGP being used as the de-facto standard globally. BGP enables the core routers to decide routes to use (among some other things) when delivering Internet traffic from one system to another. The BGP routers advertise which IP address spaces they deliver to and cache similar advertisements from other BGP routers. In BGP talk these IP address ranges are known as Autonomous Systems (AS).

The route advertisements are not authenticated nor verified in any way. Whoever who pwns a BGP talking device, can advertise networks at will. This has been a known issue already since long. The hope has been placed on the easy detection of the possible attacks and (the frankly impressive) recovery capabilities of the BGP itself.

So I could very well advertise myself as owning the route to the Google IP ranges, but obviously routing traffic successfully to the Google servers and back to the clients would be difficult for me in the core networks. The common understanding was that I would only end up black holing the Google addresses by advertising their AS numbers.

The attack has been unintentionally proven multiple times in the public networks, but actually the mitigation also has been verified various times. There was the AS7007 incident in 1997 and there was the Pakistani Telecom (local ISP) accidentally hijacking the traffic to YouTube causing the service to become unavailable for a few hours on 24th of February 2008.

They intended to block YouTube from their subscribers due to government orders, but actually ended up advertising BGP routes shorter than the legitimate ones with the YouTube prefix and eventually hijacked the traffic to YouTube. The issue obviously was quickly detected and the responsibles were notified. Once Pakistani Telecom stopped the false announcements, it took less than 5 minutes for the global BGP routers to recover and return routing traffic to YouTube correctly. The incident served as a type of Business Continuity Planning (BCP) audit for the protocol among other things and it indeed verified the awesome auto recovery capabilities.

More recently on April 8th, 2010, a Chinese ISP called  IDC China Telecommunication hijacked about 10% of the internet traffic with similar "configuration error". Various major telecommunications companies including AT&T, Deutsche Telekom and Telefonica were affected, but the entire issue was over in 15 minutes.

Back in the DEF CON 2008 Pilotov and Kapela upgraded the attack and presented a method to successfully bypass this Denial-of-Service effect of the BGP hijacking attacks and to gain a nearly stealth man-in-the-middle (MITM) position for all traffic of the entire affected AS.

They used a legitimate BGP attribute called AS-PATH. They were able to define the path the traffic should take, route it successfully to the correct destination and inject their own device to the path thus being able to freely examine, manipulate and store any unencrypted traffic destined to the hijacked AS.

There is one limitation though. Only the traffic _destined_ to the victim AS can be intercepted by the methods presented by Pilotov and Kapela. The traffic sourced from the victim could only be intercepted in some unspecified cases.

As a proof-of-concept (PoC) Pilotov and Kapela successfully hijacked ALL Internet passing traffic to the DEF CON conference itself. Renesys monitored Pilosovs and Kapelas attack on DEF CON AS number and confirmed that it took only slightly over 80 seconds before the DEF CON traffic was completely hijacked in the monitored environment .

As stated earlier the issue itself is not new. There were papers published on the security weaknesses of BGP already in the end of eighties. As far as I am aware Pilotov and Kapela performed the first ever public demonstration of the attack. Apparently the issue has been disclosed and demonstrated privately to US government officials already earlier.

Summer of 2008 was a beautiful one :) Set some spirit no doubt both personally and professionally for me. Lit the fiya. All that. There was much more to it than the above, but let's leave some of that for later posts. I think this one is stretching the length limits already a bit. There was the issue with the OpenSSH Pseudo Random Number Generator (PRNG) in Debian Linux and Ubuntu creating predictable keys aka the CVE-2008-0166, there was a lot of research published about compromising hypervisors, there was the GSM-cracking-made-cheap making the rounds, the buffer overflow in the Citect CitectSCADA ODBC service documented in CVE-2008-2639 and of course soon after the summer the networks got seriously hit by a botnet called Conficker. But lets leave that for some time later.

Welcome to the blog! Expect to find a bit more focused and compact (or not) musings on all things information security in this one. Due to professional engagements I will probably record whole lotta stuff related to intrusion detection and attacks against corporate environments for now, but finally wish to provide coverage of the entire intriguing research output provided by the global information security community and keep you updated on all things related to network security.

With the agenda set, I quote Pep Guardiola. Fasten your seatbelts, let´s have a good ride. Let's go beyond.