Thursday, April 21, 2011

Why cybersecurity tests fail

The Christian Science Monitor, a newspaper, recently had an online quiz to test how well you know cybersecurity. This is a good demonstration of the sorts of problem all such tests (like the CISSP certification test) have.


The first question asks what term William Gibson coined in "Neuromancer”.
The correct answer is "none of the above”.

This is a sort of trick question. While Neuromancer is credited with popularizing the term cyberspace, Gibson "coined” the term in an earlier work.

But the answer given in the test isn’t correct, either. The term "Internet” was coined in the 1970s. By the time the "Internet Protocol” (the basis for today’s Internet) was specified in RFC791 (three years before Neuromancer), the term "Internet” was already in widespread use.

The cop-out from test designers is that when given a bad set of choices, you are supposed to choose the "best” one. Clearly the "best” answer is "cyberspace”, not "Internet”. At least, it’s clear to me, since I was on the Internet before Neuromancer was published.


Question #8 is probably the most amusing, because it demonstrates a lack of knowledge of the English language rather than cybersecurity:

If you parse the English language, the question asks "Who invaded Georgia?", not "Who cyber-attacked Georgia?". We don’t know who was responsible for the cyber-attacks against Georgia, but we do know that Russia invaded Georgia with ground troops.

There is also a confusion as to what "its” means: does it mean the invasion was preceded by attacks on Georgia’s computers, or on Russia’s computers?


The reason I point this out is not to beat up on the Christian Science Monitor, but to use this as an analogy. Certification tests (like the CISSP) are hardly better. They are written by generalists who know a little about everything, but aren't an expert in any one thing. They make themselves immune to criticism (the first rule of the CISSP is that you will not criticize the CISSP), so it’s hard to debate those questions openly.

Microsoft's "Coordinated Vulnerability Disclosure"

Microsoft has been finding vulns in other people's products since forever. That's because for those of us "skilled in the art", it's impossible not to. Remember: when software crashes for you, you simply restart it. When it crashes for us, we trap it in a debugger, and use tools like !exploitable in order to see if it's exploitable.

Until now, Microsoft's response to such bugs as been ad-hoc. I'll bet that they've simply ignored the majority of such bugs. It takes a fair amount of work to take a bug that's "probably exploitable" to prove that it's "reproducibly exploitable". Security engineers should do it to keep in practice, but it's costly.

But now Microsoft has created an official disclosure policy for their engineers. Now, when they find a bug in somebody else's product, their engineers know what policy to follow.

"Responsible"


Microsoft should be praised for not using the word "responsible" disclosure. The words "vulnerability disclosure" describe a fact, "responsible" describes an opinion. In a room of 10 cybersecurity experts, you'll get 15 opinions one what is "responsible" for disclosing a vulnerability. (And few would agree with my opinion that no disclosure is irresponsible, and that each of the different ways a bug is disclosed has a different set of tradeoffs).

The Golden Rule


Microsoft's policy of how they disclose bugs to others mirrors their policy of how they would like others to disclose bugs in Microsoft's products. This is the "coordinated" bit in their disclosure -- where "coordinated" gives the upper hand to the affected vendor rather than the vuln discoverer.

Most controversial is the idea "under no circumstances will Microsoft release details of an unpatched vulnerability unless evidence of public attacks exists". This gives the vendor the ("irresponsible") ability to bury bugs by never patching them. Microsoft has buried bugs on occasion by doing this.

In contrast, Mozilla and Chrome have policies that say if they fail to patch a bug within a certain timeframe, then it's OK for the discover to disclose it. Knowing how vendor's react, I prefer this policy.

I'm looking a fight


Here's what I want to see: Microsoft report a bug to Mozilla or Chrome that they bury. Which policy wins? Does Microsoft follow their policy and never publicly disclose the bug? Or do they switch and follow the Chrome/Mozilla policy and report anyway?

Thursday, April 14, 2011

Email disclaimers are not pointless

These articles from Gizmodo and The Economist articles claim that the disclaimers lawyers put on the bottom of emails are pointless. The Economist says:
Lawyers and experts on internet policy say no court case has ever turned on the presence or absence of such an automatic e-mail footer in America, the most litigious of rich countries.

That's not strictly true.

Tuesday, April 12, 2011

So what's wrong with optimism?

In a response to my post "Transactive Memory Systems" , Rob Graham was uncharacteristically gracious when he called my theory "optimistic." He goes on to disagree with me and describes the cybersecurity industry as full of false memes, the echo chamber, and groupthink.

Being Optimistic


So what's wrong with being optimistic? By definition these criticisms imply that the industry believes in ideas that are untrue. But not all generally agreed upon ideas are inaccurate. Most of the ideas are believed by the person first saying it, and can be backed up by their own research. I believe that the majority of the ideas discussed in the community have merit, and it's practical to be optimistic. When Rob says, "The market doesn't care about cybersecurity" it is merely a different kind of groupthink. Remember that in groupthink there is only one correct answer, and that the self-appointed 'mind guards' are the ones who have it. If the symptoms of groupthink are protecting the group, rejecting alternatives, and silencing opposition, then optimistic belief in the likelihood of accurate ideas is the ultimate rejection of groupthink. Said differently, I believe in the abilities of the best and brightest scientists of our industry because there is a reasonable likelihood that they are correct.

Transactive Memory


To answer Rob's criticisms of the transactive memory of the security community, I didn't say that transactive memory was a good thing, just an efficient way of making decisions. It's success or failure is based upon being able to communicate the skills each person has to each other. In this I think we are very successful. It doesn't claim that the ideas of the community are True, but merely describes people's motivation for believing them.

Rob said that the three components of transactive memory were not consistent with his experience of the security community.
"Specialization: People don't actually specialize. Certainly, there are people that talk a lot about something, but that doesn't make them specialists."
In the first post, I make the point that for the purposes of "metamemory," the person who speaks about a single topic frequently is labeled by the community as a specialist, not the person themselves or any board of certification. This is a result of our human nature to simplify things to a level we can process.
"Coordination: Marisa points to conferences as an example of "transactive memory", but the reverse is true. It is the ability to act without a lot of formal meetings that is the hallmark of this "transactive" model."
The theory doesn't say that there is not a time where people get to know each other's strengths. In fact the benefits of teamwork with transactive memory depend on this period of learning about each other.
"Credibility is totally misplaced. People get credibility in our industry by pimping themselves. Vendors market themselves. Market analysts (like Gartner) also market themselves. People with little ability nonetheless get "certifications". Hackers, using tools built by their betters, are able to gain notoriety despite being little more than "script kiddies". There are those with technical ability (e.g. Schneier) that really deserve respect, but they are in the minority."
Credibility is the crux of our debate. Who should we believe? I submit that we have to believe *someone.* As an industry, the fact is we're only as good as our "experts." People like Schneier and Rob are good representatives of people who make good experts, but lousy community members. They rarely ever believe the ideas of their fellow experts. They constantly have to double check. This is inefficient and doesn't work for the broader community. But I agree that we need a better way to sort out the experts from the marketing whizzes. Or a better understanding of the implications for being wrong about our ideas.

Conclusion


Saying the community just suffers groupthink is problematic because it necessitates that the commonly held beliefs of the security community are more often wrong than right, when in reality they are more often right than wrong. I don't have a source for this observation, but if the top scientific minds in our field can't even get their theories right more than half the time, we have bigger problems on our hands than who believes them and for what reason. Call me optimistic, but I've met a lot of smart people in my time in the community, and if they say they've got conclusions, I believe them. I believe them not because I am pressured by mind control or subliminal catch phrases, but because it is the healthy human reaction to respect the ideas of experts in a field I am not an expert in. (Because really, what choice do I have?) In the same vein of optimism, I believe it is my duty to produce excellent research in the field I may be an expert in, so that those left in a similar predicament of inexperience can trust my expertise. This arrangement is infinitely more efficient than having to learn *everything* on your own, and often is the reason we have seen such successful collaborations across organizations in the security community.

No, it really is "groupthink"

In an optimistic description of the cybersecurity industry, Marisa Fagan likens it to a "transactive memory system".

I disagree. I believe it's memetics. And an echo chamber. And worst of all, groupthink.

Saturday, April 09, 2011

Transactive Memory Systems: an answer to "groupthink"

In the Information Security community, when pervasive ideas are generally agreed upon, inevitably someone cries "groupthink"(often on this blog!) The criticism is that we have let our opinions form by the pressures of the community and not by critical thinking. For example that "strong passwords increase security" or that "SQL Injection vulnerabilities are preventable." A good sign that the community is dictating the opinions is when the topic requires a special level of expertise to grok. Topics like these breed a desire to reach consensus without the individual members of the group exposing themselves as unknowledgeable or foolish. The problem with the term groupthink is that it is a pejorative term that implies the generally agreed upon idea is wrong, regardless of how the group came to that conclusion.

Wednesday, April 06, 2011

A pre-review of 'breaking_in'

UPDATE: no, a smash-n-grab is not a pentest

Tonight, Fox debuts a comedy called "breaking_in", about a small pen-test company. I haven't seen the show, but I'm the CEO of a small pen-test company (i.e. Christian Slater's character), so I thought I'd create a "pre review" of the show. (Also, one of our exploits, FedEx-ing a iPhone to a company was already dramatized in the show "Leverage" -- we got a nice thank-you note from the producers -- this gives me license to pre-comment.)

Government Didn’t Create the Internet

People often say that the government created the Internet. This is not true.

The Internet is a trillion dollars of fiber optic cables laid in the ground and under our oceans. Fiber optic technology was developed by corporations, such as Corning Glasworks, not the government. The trillion dollars in capital that was used to pay for laying cable came from Wall Street, not the government.

Monday, April 04, 2011

How to protect yourself from future "Epsilon" breach

Your e-mail address was only exposed (1) you gave it to the company and (2) if you selected “Please send me e-mail notfications”.

Anatomy of a Twitter worm ("Profile Spy")

I woke up this morning and among the tweets I saw this:
(Name has been pixelated to protect the guilty)

This looks like a worm/scam (some news here, so I thought I'd write up a technical explanation.