July 28, 2010

SCADA Hard-Coded Admin Credentials Win The Information Security Candy of the Summer Award

The winner of the InfoSec Candy of The Summer Award has been announced. The 2010 award go to "The Hard-Coded Admin Credential Issue in Various SCADA Systems".

There has been a looot of talk about the supervisory control and data acquisition (SCADA) system security and compliance during the past few years, no? I am sure you have stumbled across it and possibly also wondered about the lack of concrete details and depth regarding the vulnerabilities in and attacks against the SCADA systems. You know...the "ok, I agree, we need SCADA compliance, but what does that really mean?" feeling many articles about the issue leave you with. What can we do to harden these obviously vulnerable systems?

Mr. James Arlen seems to feel it also as he has invited the community in the BlackHat USA 2010 to sit down, discuss openly and have a little "fireside chat" this afternoon (Las Vegas time) to define the baseline in SCADA security. I feel like this really is needed.

We have now one concrete SCADA compliance issue at hands. Various SCADA systems apparently lack reliable database access control. These systems really have to be run in isolated enviroments accessable only via well protected and properly authenticating hopping stations.

The database admin credentials for Simatic WinCC SCADA systems are hard coded, should not be changed according to the vendor and now very publicly known and maliciously used in the wild. The password was curiously initially leaked to public in April, 2008., but now even your grandmother knows them. Yep. She is just not telling them to you.

Very serious issue indeed, but still maybe not The Candy without checking out how it was re-disclosed publicly this summer. We were quite ironically linked to the WinCC default configuration flaw by a very curious Windows LNK file 0-day detected this summer in the wild by the Belarus security company VirusBlokAda.

See their initial advisory here.

They discovered a worm now commonly known as the Stuxnet. It exploited a previously unknown vulnerability in the Windows Shell affecting all Windows versions. An malware analyst named Frank Boldewin soon released results of his initial decrypting/unpacking of the code and showed that at the nth attack stage Stuxnet sample with MD5 hash 016169ebebf1cec2aad6c7f0d0ee9026 utilized the hard coded database admin credentials of the Simatic WinCC SCADA Systems to run SQL on the databases. His "original advisory" is still cached by Google here.

In my opinion it is "the next Operation Aurora". The next incident in the long lineage of targeted attacks or Advanced Persistent Threats (APT) as the latest term goes. Siemens initially reported that only one customer had been attacked and later studies show that notable majority of the USB key distributed malware are detected in the Middle East. Now with the automated attack in the form of a worm is out in wild, there will be undoubtedly more attacks to follow.

For further reading enjoy the KrebsOnSecurity.com article about the discovery of the malware and the Windows Shell vulnerability. It remains unpatched, but Microsoft has released the Security Advisory KB2286198 to address the issue and there is the CVE-2010-2568 assigned for the vulnerability.

There is also an interesting Microsoft Threat Research & Response blog post about Stuxnet and Wired.com has a good Threat Level blog post about the Simatic WinCC hard coded credentials issue.

July 8, 2010

Security Distro Roundup

There have been some interesting computer security related ISO images released recently.

As already mentioned in an earlier blog post, Metasploit project has made the Metasploitable server image available. It is a vulnerable server image based on the Ubuntu Server 8.04 and comes with various vulnerable applications and configuration flaws by default. Check the Metasploit blog post for further information and download the image [torrent] for hours of legal exploitation fun.

Guy Bruneau from SANS ISC released recently ISO images (32-bit and 64-bit) for a properly preconfigured DNS Sinkhole server. Check the ISC Diary blog post by the author himself for the full description, but in essence DNS Sinkhole is a stand alone server image based on Slackware Linux operating system. You have a choice of using the Bind DNS Server or the PowerDNS server.

The magic with DNS sink holing lies in the blacklists. The Bruneaus DNS Sinkhole server parses its blacklist from three different sources (namely the Malware Domain BlocklistZeuS Tracker and Malware Threat Center SRI) and then replies to all DNS queries involving any of the blacklisted malware domains with a non-routed or simply non-existing internal network address thus disabling effectively any communication to these domains.

IPFire is a recently released Linux firewall / internet gateway server image. It is a stateful inspection firewall based on the Linux netfilter framework and complete with filters for bad packets and full IDS integration by the Guardian IPS add-on. IPFire can also act as a VPN endpoint for secure remote access and it can be deployed as a proxy for FTP, HTTP and DNS traffic or as a DHCP server for local clients.

Kind of reminds me of Devil-Linux which is an older Linux firewall distribution, which by now has also expanded to a multiserver distribution which can practically be used to implement any (or all?) common DMZ or LAN servers securely.

Via another recent SANS ISC blog post comes a very interesting paper [PDF] about creating a Live CD specifically for incident response purposes. The paper was written by Bert Hayes for his SANS Gold certification process and it offers a very detailed instructions on how to compile a Knoppix based Live CD to be used when remotely investigating possibly compromised systems. The paper is complete with detailing how to set up secure connectivity to a remote administration point (called Mothership), but it is maybe worth to note that the actual Live CD should be run locally on the system being investigated. There is no ISO image available for the Live CD yet and it seems like the project is still work in progress as new tools are planned to be integrated to the compilation.

In the other custom distribution related news we have the recent release of the Ubuntu Customization Kit (UCK), which is a tool for customizing any of the available Ubuntu distributions and which allows you to add and remove packages, tweak various configuration items and boot maneuvers and then create Live CD ISO images of these customized systems.

The IDS Incident Handling 101

We came up with a decent short description of Intrusion Detection System (IDS) incident handling during a recent lunch discussion.

Handling IDS incidents is not very different to making a puzzle. You first gather and identify the individual pieces and then you place them together correctly. While traditional puzzles, once put together, present you with a complete picture, the IDS incidents present you with a possible attack against one of your IP addresses.

The alerts in the current generation of IDS systems come in five pieces. There are always...

the source IP address
the source port
the destination IP address
the destination port

...and the IDS signature, which was triggered by an pattern detected in the network traffic by the IDS sensors.

As with traditional puzzles, you have to examine each given piece thoroughly before attempting to put them together. Some preliminary questions regarding the involved systems are the installed operating systems and the applications. Large part of the IDS signatures detect exploits to specific vulnerabilities in software, so the patching level of the systems and of the destination especially is one of the initial things to check also.

Ideally the incident handler is able to securely connect to the monitored environment and verify all of the above from the systems themselves, but in case this is not possible, ensure the incident response team has a system database of some type available to them. ALL monitored IP addresses really should be listed. I know it may be a taunting task in large environments to identify the various switch port addresses and the virtual addresses used for load balancing and clustering et cetera, but I guarantee you that simply assessing and documenting properly your computing resources is a big step in hardening your environment against misuse.

Among many other things, a proper system database enables your incident response team to react fast and properly to any possibly true positive attacks. The database should include at least the following properties for each monitored IP address:

-DESCRIPTION of the system usage
(as in web server, internet gateway router, LAN server, workstation etc.)
-OPERATING SYSTEM and the patching level
(including the network device firmware etc.)
-APPLICATIONS installed on the systems
(the exact version numbers)
-CONTACT DETAILS of the system administrators
(for running commands on the systems etc.)

Do not overlook the system information carried by the IPv4 addresses. One of the steps to full system identification is identifying the IP address ranges involved. In case the monitored environment includes publicly available servers, pay attention to the source addresses. Any addresses in the private IPv4 address ranges are most likely internal systems and as such possibly indications of more serious compromise.

The next piece to look at are the TCP/UDP ports. Do multiple searches to identify all possible services known to use the involved ports on the given platform. The IANA Port Numbers list is the official resource for figuring out the legitimate use of the Well Known Ports and the Registered Ports (namely the ports between the 0 and the 49151).

The SANS Internet Storm Center (ISC) provides a quality port search on the upper right corner of their diary page often listing the malware known to use the given port also. The DefaultPorts.com is another good database to check and don't forget the frequently updated list of known and detected port usage hosted at nmap.org.

The Trojan List hosted at simovits.com can be sorted by the port numbers also. I am not sure if the list is still maintained actively, but it is worthy information source nevertheless.

Here again access to the monitored environment comes very handy as the incident handler can quickly copy the running processes and the open network connections to the case log for later analysis. You will notice in time, that the mentioned two listings from the destination will be constantly needed.

Take the network design into consideration also when doing preliminary analysis on the ports. It is often an easy way to identify false positives and further implement filters to the sensors. Ensure you know the firewalls and routers examining the monitored traffic AFTER the sensor examines it. Depending on the implementation, the IDS sensor may see the possible attacks before the traffic gets filtered by a firewall or the routers. There may be even high volumes of attacks against the environment, but if the destination port of the attacks is closed by the firewall, no permanent harm can be caused.

After some innovative system identification and port analysis exercises, the incident handler is left with the last piece only. The IDS signature. This is the piece that tells you what actually was detected. Note that the signature does not tell you what the attack was as often used description goes. It tells you what was detected in the traffic between the two end points.

Study the signature that got triggered. Have the relevant IDS signature description library available to you at all times when monitoring. Its THE starting point. I believe all IDS vendors now provide online access to such libraries. Study the description, check all related advisories and references and do some background study if necessary to fully appreciate what the signature detects. This piece really completes the puzzle often with the IDS alerts. Here the next steps most often are revealed. They may be as simple as verifying that a patch is installed or that the software has been updated on the destination, but whatever the next steps may be, they are to be found by analysing the signature as far as possible.

In my opinion the incident handler should have access to the attack packets. At least in order to copy them to the case log for later analysis. The Cisco IDS for example gives you often also the exact character pattern detected by the signature in the same window with the attack packet data.

The attack packet data is priceless when investigating alerts against non-patched vulnerabilities such as the ones in custom web applications, where there is not actually even any specific description of the vulnerability available a part from the generic XSS or SQL Injection papers.

In case you use the much loved Snort IDS, make sure you can read the Snort rule language effortlessly as Snort uses an open signature format which allows you a detailed view into the signature construction. It uses a specific syntax, but the variables are not too many.

Emerging Threats has a good wiki entry titled Snort Signatures 101 to get you over the fundamentals

In my opinion the IDS incident handling process is completed with identifying and disregarding the obvious false positive alerts (and later maybe permanently filtering the reoccurring false positive alerts from the IDS monitoring). After this point the incident handling is over and the incident response begins although they often should overlap each other and occur simultaneously.

I have been the dude introducing various junior agents to the IDS incident handling. Kind of like giving them the first idea of the science here and showing them their first real alerts. I find it very challenging to give them any type of short description or introductory note to start with. The puzzle comparison is the current favourite here by far.

July 5, 2010

Enviroment Hardening As Training Course

I have been giving some very rewarding trainings in the past few weeks. I want to publicly thank all the attendants. I truly enjoyed both experiences. I was blessed with very knowledgeable and motivated participants which turned the classes into enjoyable peer-to-peer experiences instead of hours of monologue in front of people clearly absent minded.

I lead two very different courses. The first was aimed at the internal IT staff of a company who are not exactly in the computer business, but whose business is very dependent on computers and who are very concerned about their IT administrators performance. I believe we had a full hand of application, system and network administrators participating.

I did my Security by OSI Layers -course for them which goes through the entire Open Systems Interconnection (OSI) model talking about the threats past and present affecting each layer. I find the OSI model to be very convenient actually for this type of generic computer security trainings. Gives them structure.

As usual we spent good part of the course reviewing the application layer security, but finally there is a lot to talk about on each layer. The layered approach allows me also to inject some history of hacking in to the course and thus illustrate the progress of the game and some of the original reasoning behind the concepts like Public Key Infrastructure (PKI) and such computer security mechanisms nowadays often taken for granted.

We obviously also covered various mitigation techniques and defensive measures against the reviewed threats. With this type of course it is relatively easy to provide concrete additional value to the customer as the example material can be all drawn from the real life customer environment as we did now. In this particular case all our course exercises were actually related to assessing and hardening their own environment.

The other course I did was a bit different and somewhat more theoretic. It was for a group of junior members of a Computer Security Incident Response Team (CSIRT). People who have been working from zero to few years responding to corporate computer security incidents and IDS alerts. The course was built around the CompTIA Security+ certification as passing the exam was one of the internal requirements for a senior seat in the team.

I personally find the Security+ exam to serve perfect for this type of junior agent graduation. These people often do not possess the full five years of work experience needed for the (ISC)2 CISSP exam, so the CompTIA alternative serves them well. While consisting of only six knowledge domains, the Security+ manages in my opinion to test the applicants knowledge of the fundamentals rather well.

On both courses we also had a little isolated lab environment for some hands on exercises. I find this essential. Many of the central concepts related to computer security are challenging to teach by word only. Check the original buffer overflow article by Aleph One for proof. While it is extremely important to understand the science behind the security vulnerabilities and computer attacks, I find it equally essential to have hands on experience on launching such attacks, witnessing them happening and examining the results of an successful attack. Especially to the system and network administrators who are not maybe directly involved with computer security research, but very much affected by it.

Central point in our little labs was the Metasploitable server image [torrent] recently released by the good man HD Moore and his associates. It is an extremely vulnerable Ubuntu 8.04 server that comes with various expired versions of applications and servers, with weak account credentials and multiple configuration flaws by default. Happy trainer toy for demonstrating what is this thing called computer compromise. Big up Metasploit crew once again.

Check the Metasploit blog post linked above to get started with the image, but I encourage you to explore also. There is much, much more insecurity to be found beyond the few attacks outlined in the post. Good for brute-force exercises also. We were not actually able to break into the root account during the course, but  we did get root in later attack stages with some privilege escalation exploitation. Lots of fun included.

We attacked the Metasploitable server with various BackTrack 4 systems. Another reason to give thanks and praise, but this time to the Offensive Security crew. I am sure you are aware of the BackTrack distro by now, so I only testify that it is very suitable for training lab use also.

There has been some major changes in the BackTrack version 4 by the way. BackTrack is now based on Debian. I find it only nice to have the Advanced Packaging Tool (APT) handling the update and install procedures among some other things now included as well. The BackTrack 4 comes to the network very quiet. Seems like the network interfaces are disabled by default and not even the DHCP client run automatically. You have to therefore run ifup eth0 (or whatever your connected interface may be) to enable the interface and run dhclient to get the DHCP configuration manually from the server, which in our case was the Metasploitable server.

So...I am available for trainings : ) Feel free to contact me by email for any queries regarding the courses I am able to lead. I am willing to create some custom courses also aimed at very specific audiences as in web application developers or IDS incident handlers for example, if needed.