Monday, July 14, 2008

AxBan Signature List v. 4

Errata has updated the content on the AxBan bad ActiveX control list.

We added entries including the Yahoo Messenger and the Microsoft Access Snapshot Viewer ActiveX controls.

Users with AxBan 1.5 or later will not need to manually download these entries. It will be updated automatically on launch if your computer is connected to the internet.

iPhone 3G - 3G coverage = iPhone

Why you have to reboot your router


This Slashdot post asks "Why Do We Have To Restart Routers?", referring to home routers like Linksys, D-Link, and Apple AirPort. It claims "It seems like routers, purpose-built with an embedded OS, should be the most stable devices on my network". The assumption that purpose-built/embedded OSes are more robust is wrong, the opposite is true. Purpose-built/embedded operating systems are fragile.

The cause of this is "confirmation-bias". Engineering is broken into multiple teams. One team writes the code, another team ("quality assurance" or "QA") tests the code. However, QA suffers from critical-thinking defects. They look for tests that the product can pass, they avoid doing tests where the product will fail.

In cognitive-science/critical-thinking, this is known as "confirmation-bias". The Wikipedia page on this has an excellent example of this, which I'll paraphrase here.

A guy named Peter Wason did a study where he gave subjects a triple of numbers, such as [2 4 6], and told the subjects that they conform to a particular rule. The subjects were asked to discover the hidden rule. The were to pick their own triplets, and be told "yes" or "no" whether they conformed to the hidden rule.

What Wason found is that people chose triplets that tried to "confirm" their theory rather than "falsify" it. If they were guessing "all even numbers", they would choose a triplet like [6 8 2] to prove their theory rather than a triplet like [3 4 5] to disprove it. Those using a confirmation-bias usually failed at finding the correct theory, whereas those using a falsification-bias quickly found it.

Falsification is what makes science "science". When testing theories, scientists design tests to prove their theory false. A solid scientific theory is not something proven true, but something that we tried hard, but failed, to prove false. This is why things like "Astrology" are not scientific - while there appears plausible from a confirmation-bias point-of-view, it utterly fails to hold up when approached with a falsification-bias. Conversely, scientific theories like Newton's Mechanics or Einstein's Relativity hold up quite well to attempts trying to disprove them.

When testing home-routers, the QA department is given a list of features the router must pass. For example, they must support typical web surfing for a number of desktops behind the router, so QA might test to make sure that the device can handle 100 concurrent connections. However, QA does not test what happens when devices try a thousand, ten thousand, or a million connections. They test the confirmation-bias of "can it handle 100 connections?", not a falsification-bias of "how many connections can it handle, and what happens when it reaches its maximum?".

Because of this, the first time somebody misconfigures BitTorrent to use too many connections, the router crashes. Likewise, internal processes within the router crash often and silently restarting without being visible from the outside - but still passes QA tests because they aren't looking for that. Anything unusual that the user does is likely to cause a crash.

Fuzzing, or sending intentionally corrupted traffic, is an example of falsification QA testing that is never done. In Errata Security's testing most home routers will crash if you WiFi fuzz them. Their web-interfaces will result in crashes if you fuzz their HTTP management port. They will crash if you fuzz TCP/IP on their management port. They will also crash in their "stateful-inspection" logic if you fuzz the traffic going through them. Errata Security has fuzzed many typical home gateways, and NONE of them have passed well-known, public fuzzing suites. We therefore conclude that fuzzing isn't part of their QA process.

Well-known operating-systems like Linux and Windows don't have this problem. First of all, they have a lot of eyes looking at these things from the outside trying to break them. At the same time, they have internal QA people with a falsification-bias, trying to create tests that break them.

Conversely, expensive enterprise products (like switches) have the same confirmation-bias as cheap home products. We crashed an enterprise switch the other day running Nessus against it. I've tested products from all the high-end vendors, and they all demonstrate confirmation-bias in their testing. The only products that seem to have falsification-bias in their QA process are the major operating systems and a few of the major applications.

Confirmation-bias is why you hire pen-testers, code-auditors, and security consultants. You are hiring people to falsify your assumptions, not confirm them. This is the secret reason why pentesters look so smart. It's not that they are actually smarter than your security team, it's that they have a falsification-bias rather than a confirmation-bias.

Indeed, it's the top-down (from management) confirmation-bias that is at the heart of cybersecurity woes. Everyone is trying to prove they are secure, anyone trying to show where they are not secure is suppressed, called a "trouble maker" or "not a team player".

Thursday, July 10, 2008

New Firmware

Thanks to lifehacker, http://lifehacker.com/398280/iphone-20-software-update-unofficially-available, My iphone is now running te 2.0 firmware.

Wireshark "TurboCap"

I just noticed that CACEtech is now selling a sniffing adapter "TurboCap for Windows. (CACEtech is the company that funds Wireshark development - and if you are a cybersecurity geek, you should have experience with Wireshark).

This product addresses the problem that operating-systems (Windows, Linux, BSD, et al.) are not optimized for sniffing packets. Thus, if you wanted to sniff a fast network with Wireshark, you'd be lucky sniffing at a rate of 300,000 packets per second. This product claims to allow sniffing at 3-million packets per second, which is the max theoretical speed for full-duplex gigabit Ethernet.

This product addresses the problem with a custom driver. You cannot use this card for normal networking. Although it physically is a network card, it will not appear as one of the network cards under Windows. It is a special "sniffing" card instead. It will only be available for custom sniffing applications, such as Wireshark.

The first product that replaced the network stack with a custom sniffing driver was the Network General "Sniffer"™ back in the 1980s. This is the product that gave us the name "packet-sniffer". It was the first to achieve "wire-speed" sniffing performance.

Many sniffing products have since used this idea. I used to work at Network General. When I founded my own company and created the BlackICE intrusion-detection system in 1998, I likewise used this concept. We wrote a custom sniffing driver for the 3c905 hardware. This happened to be the chip used in Dell notebooks of the time. The upshot of this was that my Dell notebook could do wirespeed 100-mbps intrusion-detection while other products at the time struggled at 10-mbps. This was an unbelievable speed back in the day, although custom drivers are more common now, so most intrusion-detection products now support wirespeed.

When writing a custom driver, or tweaking the existing drivers for better speed, there are a number of issues that you address.

BUFFER SIZE

Standard network drivers use tiny buffers, often a mere 64k. You want a lot more for a sniffing application. You might allocate a 100-megabyte buffer within the driver for holding packets.

FRAGMENT SIZE

In order to fit variable sized packets into a tiny buffer, most cards will fragment the packets in to 64 byte, 128 byte, or 256 byte chunks. The network driver must then reassemble the fragments back into whole packets before sending them up the network stack. Note that this is a wholly different sort of fragmentation at the hardware level unrelated to the fragmentation that occurs at the TCP/IP level.

A good choice for fragment size is 2048 bytes. It's large enough to hold standard Ethernet packets without needing reassembly. Only GigE jumbo frames would need to be reassembled.

POLLING INSTEAD OF INTERUPTS

The operating-system stack is designed so that incoming packets cause a hardware interrupt. This causes the operating system to halt its current task, run the driver code to deal with incoming packet, then resume. Handling an interrupt is efficient if there are few of then (less then 10,000 per second), but extremely inefficient if there are many. Sending 3-million interrupts per second at a typical operating system will cause it to lock up.

The alternative solution is "polling", where the software constantly tries to read the next packet. This means that there is no overhead from interrupts if the packets are arriving very fast, but means that the CPU is pegged at 100% utilization even if there is no traffic at all.

A hybrid method is to poll on a timer interrupt. In this method, you set up a timed interrupt (such as 10,000 per second), then poll the card to see if any packets have arrived since the last interval.

DATA TRANSFER

There are two ways of getting packets off the network card into memory. The first is "programmed input-output". In this mode, the CPU reads the bytes from the network chip, and then writes them into main memory. The second method is "direct memory access (DMA)". In this method, the network card writes the packets directly to memory, bypassing the CPU.

The CPU still needs to be involved with DMA. It must tell the adapter where buffers are in memory. The driver must continually refresh the list of free buffers. Thus, as the code processes incoming packets, it will free up those buffers and send them back to the network hardware for reuse in DMA.

KERNEL-MODE TO USE-MODE TRANSITIONS

In much the same way that handling an interrupt is expensive, there is a lot of overhead in transferring control from driver (which runs in kernel-mode) to the application (which runs in user-mode). You would likewise lock up the system trying to do this for every packet at 3-million packets per second.

The trick to get around this is to map the buffer in both kernel space and user space. In this manner, the user-mode sniffing application can read packets directly from the buffer without a kernel-mode transition.

CPU BUDGET

Consider a 3-GHz CPU trying to sniff packets on a full duplex Ethernet at 3-million packets per second. Simple math shows that you have only 1000 CPU cycles per packet. That is your "budget".

This budget gets used up pretty fast. The problem isn't necessarily the number of instructions that can be executed (CPUs can execute multiple instructions per cycle), but memory access. The CPU can access a register in 1-cycle, first level cache in 3-cycles, second level cache in 25-cycles, and main memory in 250-cycles. In other words, if the software attempts to read memory, and it's not in the cache (a "cache miss"), it must stop and wait 250-cycles for the data to be read.

Thus, a 1000-cycle-per-packet budget equates to a 4-cache-miss-per-packet budget.

The packets are DMAed by the driver into memory, but not the cache. Therefore, reading the first byte of a packet will result in a cache miss. This leave only 750-cycles remaining.

In addition, the header information (packet length, timestamp, etc.) are located in a different place in memory. This will also cause a cache miss, leaving only 500-cycles left. Multiple CPUs must be synchronized with a full memory access, which has the same cost as a cache miss. This leaves 250-cycles left to process the packet. If you do something like a TCP connection table lookup, you've probably got another cache miss. These leaves 0-cycles left to process the packet.

Thus, we've quickly exceeded our CPU budget without actually doing anything.

With most drivers, you can locate the packet headers with the packet data. By combining them into the same location, they can be read together without a separate cache miss. CPU's have a pre-fetch instruction. The packet-read API can be implemented so that whenever the software reads the current packet, it "pre-fetches" the next packet into cache. Thus, the headers and data will be available next time without a cache miss.

If you use a ring buffer and a producer-consumer relationship, you won't need to use a traditional memory lock to synchronize the driver with the application. If you are even more clever with your sniffing API, you pre-fetch several future packets, and you allow the application to peek at the next packet allowing it to pre-fetch its TCP connection entry.

Putting this all together, I've proven that you'll need at least 4-cache misses to process a packet, and thus handling 3-million packets per second is impossible. Then, I've shown tricks to show how you can get around this and process a packet without any cache misses.

OTHER TRICKS

There are a long list of other optimizations you can do. For example, you'll want to align your buffers on cache-line boundaries. You'll also want to set the processor affinity flags so that the driver uses one CPU core while the user-mode process uses the remaining cores.

CONCLUSION

CACEtech claims "wirespeed" performance, which implies 3-million packets-per-second. I don't know if they've implemented all these tricks. Their cards are a little pricey ($900 each), so I'm not willing to buy one just to play with it. However, for anybody running network tools like Wireshark or Snort, they should logically give a huge boost in performance.

DI #4


I found myself this week explaining to people the difference between a real threat and a perceived threat.

Tuesday, July 08, 2008

GMail now shows IP address log

Google is updating Gmail to allow users to see if anybody else is reading their e-mail.

This is one of the things I recommended for dealing with SideJacking. When hackers steal your password or your session-id, there is no way for you to know that they are reading your e-mail. It's the scariest sort of hacker attack since it's completely invisible.

Now, apparently, Google has made this visible.

I'm constantly sidejacking Dave's Gmail session cookies. We connect to the same network, I'm always sniffing the network, and Gmail will sometimes disclose session cookies in the clear even when using SSL (there is no way to use Gmail securely in a way that cannot be sniffed). I've never actually used the session-ids to log onto Dave's account, but how can he trust me?

With this feature, he'll know. He can use this feature to verify that nobody else has accessed his account.

The Gmail blog post does not mention "hackers". This is for a good reason. About 20% of the population is stupid and paranoid. When they see information on Google's page they don't understand, they will assume it's a hacker. Worse, a certain percentage of hard-core paranoids will fit this into their conspiracies no matter how benign the information. As a result, this is going to become a headache for Google as people call them about hackers in their account.

If you suspect that somebody else has been accessing your account, there are a number of other things you should look at.

First, look at your password. If you are like most people, you use the same password for everything. A lot of hackers get into your Gmail because they've hacked some other website were you've created an account (with your e-mail address and password), and simply checked to see if the passwords work for Gmail as well. Change your password to something unique that you only use for Gmail.

Second, check your "Filters" and "Forwarding" account settings. A lot of hackers aren't going to read your e-mail through your account, but are instead going to configure Gmail to forward copies to another account.

Third, check your "POP/IMAP" account settings. It's easier for hackers to download all your e-mail through POP/IMAP because it only takes a few minutes, rather than spend days browning through your mail by hand. I think POP and IMAP might be enabled by default, so you should disable them. If these get re-enabled, then it might mean somebody has hacked your account.

Fourth, I suggest that you use "https" instead of "http" so that your sessions are encrypted most of the time. Google will still sometimes send things unencrypted, but at least this will reduce your exposure.

Showing concurrent and past logons is such a useful feature, it should be considered a requirement for all Web 2.0 applications, whether they are PHPBB, eBay, your local bank, or your online e-mail. If you suspect a hacker, it's a sure fire way to see if something unusual is going on -- despite the ignorant paranoids that will confuse everyone by insisting there's evidence for a hacker where none exists.

Dan Peck

http://www.digitalbond.com/index.php/2008/07/08/why-do-binary-analysis-when-you-have-source/

Dan Peck has a good reaction to Rob's post on auditing.

Sunday, July 06, 2008

DI #3


In case you are wondering I am the guy on the right and Rob is the guy on the left. You can tell its Rob because he is holding a large square that is about as close as my artist abilities will let me get to drawing a Macbook Air.

Thursday, July 03, 2008

DI #2


Rob and I get this kind of question all the time. After Blackhat last year we got some many requests from people that wanted to learn to hack Gmail accounts it was funny. Last week I actually got an email that contained this rather cryptic string:

Hello, I do not know how to develop code but wish to hack wifi
drivers. Is it possible?
What do you say to that?

Oooohh, more cocktail parties at the airport Hilton...

Awesome, another industry organization. ICASI

From their website:
"ICASI intends to be a trusted forum for addressing international, multi-product security challenges. "

Its amazing these mutinational companies can put the desire to sell their own wares out of the way and work together for a common goal of...selling more products? I think what we need is an organization to stamp out industry organizations.

Wednesday, July 02, 2008

No such thing as bad press

TechCrunch wrote a great article about the backlash against negative reviews on Yelp.

I totally drink the kool-aid when it comes to Yelp, and I think it's one of the coolest things on the Internet right now. I yelp about every place I go. I use it every time I travel. I want a Windows Mobile Yelp Client for Xmas.

There was a comment to the TechCrunch article that said that the "concern is that competitors and not customers are posting poor reviews." The idea that business owners are going around making bad Yelp posts about their competitors seems a little far fetched to me, given that I personally have never come across a place that didn't deserve their rating, but even if this was true, wouldn't it just be canceled out with the glowing endorsement the competitor business owner gave his own page? On the whole, this dance routine just doesn't happen.

But the greatest thing of all about this story is every new website created to slander Yelp merely gets the word out further. Yelp.com, like many things on the Internet, only works with high visibility and user input. Every time the site is mentioned, it adds credibility. I think this double-edge sword is an interesting phenomena. People who do not like the site are left with little recourse, except to hope it's a fad that blows over... or it gets harvested by Google.

Code auditing: the abstract vs. concrete

[This issue comes up often enough I thought I'd write it down in a blog post]

Among the services Errata provides is "code auditing", where we read the source code (and/or binary code) of a product looking for security flaws.

Hackers think different than coders. Hackers deal with the concrete, coders with the abstract. This difference is at the root of hacking.

Part of the indoctrination of coders is to beat "concrete thinking" out of them. This is why universities teach very abstract languages like LISP, or object-oriented coding like C++ and Java.

The consequence of this is that coders rarely understand how their code actually works. This is why Java is so often unbearably slow - the coders don't understand what the software is really doing. The biggest problem for Java is the automated memory management. Unlike C/C++, coders don't have to worry about allocating and freeing memory, because Java takes care of that for them. The unfortunate consequence is that seemingly simple code will recklessly make copies of objects in memory, spending most of its time allocating/freeing memory invisible to the coder. The coders who study the mechanics of the Java virtual machine can avoid this problem and write fast code with few memory copies. Few Java programmers study these mechanics, however.

We see this tension between abstract and concrete in file-formats and network-protocols. When hackers attack your code (through buffer overflows, for example), they will try to corrupt the file-format or network-protocol. Unfortunately, most coders do not understand what their format/protocol looks like. This is a concrete detail lost in the code. The format of the file on the disk, or the packet on the wire, is essentially an accidental byproduct of the various abstractions in the code. (This is also why there are so many vulnerabilities - file/packet processing is spread throughout the code rather than located in one place).

Thus, the first thing we are going to do when analyzing code is fire up a packet-sniffer (like Wireshark) and look at the packets it is sending on the wire. We'll open up a binary editor (like Hexview) to see how the files are formatted. We'll use reverse engineering tools (like IDApro) to see what's in the code. We'll run debuggers (like gdb or Visual Studio) to step through the code.

Thus, the first step in auditing code isn't to figure out the abstract intent of what it's supposed to be doing, but to figure out the concrete reality of what it's actually doing. Even when we have the source, we are still going to reverse-engineer your binary.

Of course, the first day on any project we often hear customers say "...oh, I didn't know it was doing that".

The internet just causes problems

http://www.cnn.com/2008/TECH/07/01/cyberbullying.ap/index.html?eref=rss_topstories

It is a great day "for the children" now that cyberbullying is outlawed. Those state officials really worked hard to stop growing menace. Before you clap, ask yourself this: how will it be enforced? Will the statement "OMG, a girl on MySpace said that my dress makes me look FAT" get someone thrown in jail? Will this be left up to a judge who has never seen the internet tubes? In that case, we could all be guilty of cyberbullying.

Tuesday, July 01, 2008

New threat to physical security

If you're like me, then today your Google Alert set to "invisibility cloaks" finally paid off.

This promising new research on ways to become invisible is completely awesome, but I can't help but wonder what security threats loom on our horizon. Perhaps in the future, there will have to be some kind of "clothes line" device that swoops down after each person that uses an access badge to enter a building. Better learn how to move quickly folks. And if you hear a massive *thud* after you enter, you can assume you've either thwarted a highly backed criminal mastermind or a plucky young wizard.