Tuesday, April 29, 2008
Monday, April 28, 2008
Thank god, I thought I was the only person this had happened to...
I wonder how that system got QAed, it seems like it would be prone to false postives.
Racing to Zero
This article claims that AV vendors are upset about an anti-AV competition during DefCon. They claim that it provides too much help to hackers.
Yet, such contests also help customers. The 'protectors" (product vendors) have big marketing budgets to tell us their side of the story about how good they are protecting us from 'hackers'. The 'hackers' have neither the budget nor the inclination to tell us how good they are at breaking past the 'protectors'. What we are hearing is the sound of one hand clapping. We only get one side of the story.
This contest will tell us the other side of the story. That's what conferences like DefCon, BlackHat Briefings, and CanSecWest are for - they are the counterweight to product conferences like RSA. Educating the consumer unfortunately means hackers often get educated to.
The educating needed here is that the mainstream anti-virus technologies are easily evaded, and that such evasion happens a lot, especially against high-value targets like financial institutions and government organizations. I see it when I talk to customers, but there is no reliable statistics on the matter. Anti-virus vendors publish tests "proving" a 99% detection rate, when no such detection rate happens in the real world.
There are niche technologies that can improve this situation. However, customers aren't demanding them, so mainstream vendors don't invest in them, and the niche products don't get traction. The more the inherent deficiencies with anti-virus come to light, the more these technologies will find their way to the market.
This open-letter signed but some respected people in the field asserts the principle: "It is not necessary and it is not useful to write computer viruses to learn how to protect against them". This is absolutely true. However, that doesn't apply to customers. Often, the best way to test an anti-virus product is to create your own virus. When I was building such products, I felt no need to create viruses in order to develop defenses. Now that I'm hired to evaluate products, I have already built my own viruses to evaluate how anti-virus products work, and whether they live up to their claims.
I would hope that the contest organizers take this into account. While judging how well hackers defeat anti-virus products, I would hope that they likewise give an award to the anti-virus that is best at defeating the hackers.
Yet, such contests also help customers. The 'protectors" (product vendors) have big marketing budgets to tell us their side of the story about how good they are protecting us from 'hackers'. The 'hackers' have neither the budget nor the inclination to tell us how good they are at breaking past the 'protectors'. What we are hearing is the sound of one hand clapping. We only get one side of the story.
This contest will tell us the other side of the story. That's what conferences like DefCon, BlackHat Briefings, and CanSecWest are for - they are the counterweight to product conferences like RSA. Educating the consumer unfortunately means hackers often get educated to.
The educating needed here is that the mainstream anti-virus technologies are easily evaded, and that such evasion happens a lot, especially against high-value targets like financial institutions and government organizations. I see it when I talk to customers, but there is no reliable statistics on the matter. Anti-virus vendors publish tests "proving" a 99% detection rate, when no such detection rate happens in the real world.
There are niche technologies that can improve this situation. However, customers aren't demanding them, so mainstream vendors don't invest in them, and the niche products don't get traction. The more the inherent deficiencies with anti-virus come to light, the more these technologies will find their way to the market.
This open-letter signed but some respected people in the field asserts the principle: "It is not necessary and it is not useful to write computer viruses to learn how to protect against them". This is absolutely true. However, that doesn't apply to customers. Often, the best way to test an anti-virus product is to create your own virus. When I was building such products, I felt no need to create viruses in order to develop defenses. Now that I'm hired to evaluate products, I have already built my own viruses to evaluate how anti-virus products work, and whether they live up to their claims.
I would hope that the contest organizers take this into account. While judging how well hackers defeat anti-virus products, I would hope that they likewise give an award to the anti-virus that is best at defeating the hackers.
Thursday, April 24, 2008
Automatic patch-based exploit generation
This paper promises "automatic patch-based exploit generation". The paper is a bit overstated, this isn't possible. By "exploit" the paper does not mean "working exploit". That's an important difference. Generating fully functional exploits by reverse engineering a patch takes a lot of steps, this paper automates only one of them, and only in certain cases.
Hackers have been reverse-engineering patches for a while now. In the old days, hackers and vendors would fully disclose all the information about a vulnerability. Those who discovered the bug would often publish a "proof of concept" exploit. This made a lot of people angry because they would get exploited before being able to patch their systems. Therefore, the industry has changed to be largely anti-disclosure, and few details of new vulnerabilities are now reported.
In response, the industry figured out how to get that information anyway. By comparing the patch against the previous version of the software, people can figure out what changed. Software makers are conservative and change the minimum necessary to fix the bug, so comparing the differences is pretty easy.
A good case study of this was the bug that led to Blaster. The discoverers ("Last Stage of Delirium") recognized that this was a major new class of bugs ("remote root in the default install of the Windows desktop") and refused to disclose any details about it. This was a major break from the industry norms at the time.
This frustrated me. I developed signatures for the ISS Proventia intrusion-prevention system. Without full disclosure or a proof-of-concept exploit, I could not create signatures to protect against it. This frustrated me.
So we reverse-engineered the details anyway. We used a tool called "bindiff" to find out what changed, and then created our own proof-of-concept exploit from it, then signatures to protect against it. (For intrusion-detection buffs: these were "vulnerability" signatures, not "exploit" signatures).
Nowadays, this is rather common. Many companies in our industry reverse-engineer patches on a regular basis. Black-hat groups in China and Eastern Europe also do this. For a major new vulnerability, you can expect that chances are pretty good that somebody, somewhere, has a working exploit within the first 24 hours of a patch being released. Indeed, Errata Security does this a couple times a month.
The steps are pretty straightforward.
First, you find the differences. Unfortunately, while the logic of the software is mostly unchanged, the way software build tools lay out the software changes a lot. You need tools like "bindiff" that can find the "logical" differences rather than the "exact" differences, and you often need some experience to quickly sort the irrelevant changes from the relevant ones.
Second, you need to find out how to reach the changed code. This is what the proposed automated-tool does. Given a function, it tests all possible inputs in order to find out which inputs will cause the NEW code to execute, and consequently the OLD code to fail. Normal brute-force wouldn't work - the paper above figured out clever ways to narrow the combinations that need to be tested.
Thirdly, you need to reach that function. In my experience, this is the hardest part of exploit development. How do you create a network packet or a file formatted in just the precise way to reach that function.
Fourthly, you need to figure out how to get execution from the bug. For example, if NX bit is stopping you, then you need return-to-libc shellcode to get around this. There are a lot of exploit frameworks that make this easy, but "easy" shellcode only works for about half the time, the other half its really, really tough, and can often take several days to get it right.
Fifthly, you have to try it out on a bunch of different systems. Your exploit needs to be different for different versions of the target. You typically create versions for specific targets you are interested in, and leave the remaining targets unexploited, otherwise you'd spend forever for each minor difference in potential targets.
The second step, the step this paper automates, takes the least amount of time. Indeed, I never would have identified it as a separate step (it's really part of the next step) because it's usually very easy. Sure, there are probably some difficult cases where this step might take more than a couple minutes, but those cases are fairly rare. It's certainly an interesting development, but in the real world, it wouldn't significantly reduce the amount of time it takes to make fully functional exploits from reverse-engineered patches. However, this time is already worrisome short, which means while you shouldn't be more scared from this paper, you should already be very scared.
The above article quotes "Microsoft has not taken adequate steps to make such attempts more difficult". I'm not sure I'd phrase it that way. Microsoft is under no obligation to do anything about this problem. While it's true that they "don't do anything", but "don't do enough" is a different claim.
The reason is that fixing this is harder than people think. Users already complain that it takes to long for Microsoft to patch flaws, and that the patches are often buggy. Obfuscating changes would make this worse. Moreover, it would be an arms-race: there are many simple things Microsoft could do in order to make reversing harder, but it would be easy to write tools to counteract them. Still, I would hope that Microsoft would be researching this area heavily -- while it's a bad idea for most bugs, it's something to consider for certain critical bugs.
Hackers have been reverse-engineering patches for a while now. In the old days, hackers and vendors would fully disclose all the information about a vulnerability. Those who discovered the bug would often publish a "proof of concept" exploit. This made a lot of people angry because they would get exploited before being able to patch their systems. Therefore, the industry has changed to be largely anti-disclosure, and few details of new vulnerabilities are now reported.
In response, the industry figured out how to get that information anyway. By comparing the patch against the previous version of the software, people can figure out what changed. Software makers are conservative and change the minimum necessary to fix the bug, so comparing the differences is pretty easy.
A good case study of this was the bug that led to Blaster. The discoverers ("Last Stage of Delirium") recognized that this was a major new class of bugs ("remote root in the default install of the Windows desktop") and refused to disclose any details about it. This was a major break from the industry norms at the time.
This frustrated me. I developed signatures for the ISS Proventia intrusion-prevention system. Without full disclosure or a proof-of-concept exploit, I could not create signatures to protect against it. This frustrated me.
So we reverse-engineered the details anyway. We used a tool called "bindiff" to find out what changed, and then created our own proof-of-concept exploit from it, then signatures to protect against it. (For intrusion-detection buffs: these were "vulnerability" signatures, not "exploit" signatures).
Nowadays, this is rather common. Many companies in our industry reverse-engineer patches on a regular basis. Black-hat groups in China and Eastern Europe also do this. For a major new vulnerability, you can expect that chances are pretty good that somebody, somewhere, has a working exploit within the first 24 hours of a patch being released. Indeed, Errata Security does this a couple times a month.
The steps are pretty straightforward.
First, you find the differences. Unfortunately, while the logic of the software is mostly unchanged, the way software build tools lay out the software changes a lot. You need tools like "bindiff" that can find the "logical" differences rather than the "exact" differences, and you often need some experience to quickly sort the irrelevant changes from the relevant ones.
Second, you need to find out how to reach the changed code. This is what the proposed automated-tool does. Given a function, it tests all possible inputs in order to find out which inputs will cause the NEW code to execute, and consequently the OLD code to fail. Normal brute-force wouldn't work - the paper above figured out clever ways to narrow the combinations that need to be tested.
Thirdly, you need to reach that function. In my experience, this is the hardest part of exploit development. How do you create a network packet or a file formatted in just the precise way to reach that function.
Fourthly, you need to figure out how to get execution from the bug. For example, if NX bit is stopping you, then you need return-to-libc shellcode to get around this. There are a lot of exploit frameworks that make this easy, but "easy" shellcode only works for about half the time, the other half its really, really tough, and can often take several days to get it right.
Fifthly, you have to try it out on a bunch of different systems. Your exploit needs to be different for different versions of the target. You typically create versions for specific targets you are interested in, and leave the remaining targets unexploited, otherwise you'd spend forever for each minor difference in potential targets.
The second step, the step this paper automates, takes the least amount of time. Indeed, I never would have identified it as a separate step (it's really part of the next step) because it's usually very easy. Sure, there are probably some difficult cases where this step might take more than a couple minutes, but those cases are fairly rare. It's certainly an interesting development, but in the real world, it wouldn't significantly reduce the amount of time it takes to make fully functional exploits from reverse-engineered patches. However, this time is already worrisome short, which means while you shouldn't be more scared from this paper, you should already be very scared.
The above article quotes "Microsoft has not taken adequate steps to make such attempts more difficult". I'm not sure I'd phrase it that way. Microsoft is under no obligation to do anything about this problem. While it's true that they "don't do anything", but "don't do enough" is a different claim.
The reason is that fixing this is harder than people think. Users already complain that it takes to long for Microsoft to patch flaws, and that the patches are often buggy. Obfuscating changes would make this worse. Moreover, it would be an arms-race: there are many simple things Microsoft could do in order to make reversing harder, but it would be easy to write tools to counteract them. Still, I would hope that Microsoft would be researching this area heavily -- while it's a bad idea for most bugs, it's something to consider for certain critical bugs.
Monday, April 21, 2008
Why the "Pentrate and Patch" idea is not only great, but also essential: A response to Ranum and Lindstrom
Every few months or so a person will point me to an essay written by Marcus Ranum entitled "The six dumbest ideas in computer security". This essasy is held up as the gold standard of why vulnerability auditing or penetration testing is bad. These ideas are parroted, namely by Pete Lindstrom, as reasons by vulnerability auditing is a bad idea.
Let me pause for a second and discuss marketing spin and positioning. Since we are in the midst of primary elections in the United States voters are constantly overwhelmed with a wide range of marketing messages, stories, testimonials, and other such forms of persuasion to sway people to a political parties or candidate's viewpoints. One of the staples of these efforts is vilifying your opponent. For instance if you think that the 2nd amendment should be repealed and individual gun ownership should be banned an argument would start with something like this.
"The NRA wants to arm everyman woman and child with fully automatic weapons and a legal precedent to shoot anybody on site they want. That will undoubtedly return this country to a state of constant fear and bloodshed." Of course, this is not an accurate representation of pro 2nd amendment advocates but a fair and balanced view can't be presented or you may actually convert people to your opponents cause.
That is a no-no. Dismissing this type of misinformation should be simple with some extra research. That is the beauty as the person misrepresenting the information relies on human laziness to accept an authoritative statement with no question.
You may be asking what this has to do with Ranum and why you care.
Ranum is attempting to convince you he is right by painting the ideas he is arguing against in the worst possible light and even misrepresenting some facts. I have thoughts on all his points but the one I choose to talk about the most is the "penetrate and patch idea".
Ranum wants to paint the idea of penetration testing and vulnerability hunting in general a bad idea. He lists several reasons and even gives a cutesy outline of a basic program that demonstrates his perceived flaws in the idea. The problem is that with a bit of fact checking mixed with a bit of tree shaking his reasoning starts to fall apart.
To start with the logic is flawed. His argument is literally the equivalent of saying "now that there is medicine no one should get sick anymore." He then continues with an anecdote about a programmer friend that refers to the "penetrate and patch" model and "turd polishing". From the description of the process it does sound pretty worthless. Hire some people who do not know how your application works to look for bugs. They will find some, claim success, justify the paycheck, and then do it all over again. If security research actually worked like Ranum portrays, I would have to agree, that model is worthless.
Luckily, Ranum's description is not how it actually works. I offer an explanation of how these tests actually work, a courtesy Ranum did not extend to his readers. Penetration testing is a marketing name that covers a variety of different activities that are not all the same and have different requirements, goals, and results.
Penetration Test:
Who needs it: Anybody who wants to verify that their current security posture is working. Penetration tests are to enlighten C level executives to verify or dispel a current policy, spending, and staffing needs.
How does it work: A penetration test is an activity that will assess a network's survivability to a hacker attack. This is done by using the tools and techniques a hacker would to search for flaws in a networks design, architecture, and policy. The goal is to no break-in as any network can be compromised with enough money and time. The goal is similar to game theory to look for things that might not have been considered during the network architecture.
Goals: Although vulnerabilities are unearthed, they are interpreted from a larger point-of-view.
Examples: Discovering that a user account has an easy guessed or brute forced password does not mean the resolution ends with changing the password. It means that implementing or strengthening an organizations password policy while implemented constant password auditing with user notification would resolve the problem. Being able to exploit a buffer overflow on a server for access does not mean that patching the server will fix the problem. It means that implementing a policy that servers have built in survivability from attacks with non-executable memory, address space layout randomization, heap and stack protection, and server hardening techniques would solve the problem.
Application Assessment:
Who needs it: People who have large development shops for custom applications or products. They provide a method of accountability for a products development while education the developers to different potential threats and even unintended consequences.
How does it work: An application, an operating system, or even a device like a mobile phone is analyzed and potential weakness are discovered and mitigated. This is more than just looking for buffer overflows; it is the understanding of how application actually works as opposed to how the documentation says it works. This type of understanding can pinpoint design errors and unintended consequences.
One of the advantages an audit like this brings is that normally during the development of a large and complex product there are different teams who work on different parts. For instance in an anti-virus tool there may be several teams like a scan engine team, a GUI team, an update team, and a signature team. The problem is that often these teams do not communicate and if they do its through things like developer documentation or API references, which means that one team may not understand the ramifications of the changes they make on another team. An application assessment means that a single person, or if it is a large enough application, a team of people will analyze and understand all the operations while looking for potential missteps. An application assessment is not just QA on steroids.
Examples: A security product stops threats by running them in a virtual machine and attempting to determine if software is malicious. A lack of proper checking on the part of the VM allows an attacker who is aware of its present can use the environment to hide malware as well as make untraceable changes to the host operating system. In a different example, the ability to test a mobile device GSM connection means that a test application left behind that shows the use of undocumented API functions while giving a person the list of all the ARFCN numbers in use in the area. Outsourced code is audited before being integrated into a main project to ensure that basic security standards have been met.
Vulnerability Assessment:
Who needs it: Anybody who has to prove that they are following a set of guidelines on how security should be implemented on a network.
How it works: A vulnerability assessment is what most people are actually thinking of when the phrase penetration test is mentioned. In some fashion, an auditor will look for a known set of problems to ascertain the security posture of a network. This maybe done for compliance reasons or to certify that a network is reasonably secure to handle certain types of information or transactions. They are not supposed to be insightful or explore new areas but instead verify a network meets a certain security baseline. This is important to companies like credit card processors, health care providers, and anybody that handles personal and confidential information.
Examples: A credit card processor needs to see that a potential merchant has taken the proper steps to assure that credit card information is stored safely and securely and that a network is undergoing a measure of security due diligence.
As you can see, it is a bit disingenuous to combine them all under one heading and describe them in a wash, rinses, repeat description of finding vulnerabilities. One of the main arguments Ranum makes is that if the "penetrate and patch" philosophy works them we should be running out of bugs to find. On the surface, this is a good argument and in certain conditions, I would agree with it. Those conditions would be that no future development is done on operating systems why banning any new development of commercial code. In order for Ranum's 10-year window to be accurate, that would mean that we would have spent 10 years auditing Windows 98, Redhat 3, and MacOS 7.6. If we, as an industry, were able to stop development and focus on bug, fixing just these versions over a 10 year period there would be no vulnerabilities. However, this dark period of code development is not possible. The desire of new products and functionality means that every day there is new code that contains new vulnerabilities and new problems that go into wide use. At rate of growth the IT industry is moving it will be almost impossible to catch up let alone stay current with the new tide of vulnerabilities are unwittingly introduced. It is arguably that security effort on security proofing new products should be the focus rather than looking through old products. I agree that investing in anti-exploitation technology is wise. If you look at new OS versions like Redhat, OSX, and Vista there is a lot more attention on these types of technologies. These investments are offset by the slow adoption rate of the new operating systems and is compounded by the old troubled software is not going away quickly. Recently during a penetration test I actually ran across a Windows 95 machine, talk about a blast from the past.
Ranum gives an example about the futility of running a "penetration test" for Apache against a locked down server with custom C code. I would agree. Security is no different from anything else; knowing the right tool to use is the key to success while using the wrong tool means useless results. A penetration test is useful for that example.
When doing penetration tests we specifically ask to identify all custom applications before starting the test in order to examine them in a different way otherwise it would be a waste of both parties time and effort.
Now for the entertainment portion of this post, in order better explain this position lets apply Ranum’s theory to unicycle riding. Let us say in an alternate universe that Ranum owns a Unicycle store, Ranums One Wheel. If you are used to dealing with experienced Unicycle riders glib comments like “do it right the first time” could be warranted but what happens when a novice rider comes in?
Mr. Smith (the new Unicycle rider): Excuse me sir, I have been committed to riding a unicycle in a charity fundraiser and I bought one here last week. I seem to be having a problem in its proper operation.
Ranum: Whats your problem?
Mr. Smith: I keep falling. I can’t seem to get the hang ouf it, is there any advice you can offer me?
Ranum: Of course. Don’t fall down, that is counterproductive.
Mr. Smith: Yes, I understand that part, but how do I not fall down?
Ranum: If you are wearing elbow pads, knee pads, and a helmet then you shouldn’t worry, just do it right the first time.
Mr. Smith: That’s kind of the problem, I don’t know what right is, or how to do it. Although I am wearing the required safety gear I am failing to achieve my goal of riding the unicycle.
Ranum: Then you do not have a problem.
Mr. Smith: I was thinking of hiring an instructor to look at what I am doing and tell me how to avoid falling, can you suggest someone?
Ranum: That’s what we call the “try and fall” method of unicycle riding. You will not learn anything and you will find that you are vulnerable to all types of problems like new environments and wind. It’s best if you just do it right the first time.
Mr. Smith: But I don’t undersand what right is, what It looks like, or how I accomplish it. At least with a coach I could get guidance.
Ranum: That is the kind of guidance that will leave you taking lessons forever. I am telling you just do it right the first time and do not fall.
Mr. Smith: Also I was wearing all the safety gear you told me to but i fell in a bush and got poked in the eye and had to go to the hospital.
Ranum: Never heard that one before, it must not be dangerous.
The largest problem I see with Ranum’s aversion to “penetrate and patch” is that is its purest form he is advocating ignorance. The essay boils down to saying that although there are tools and methods to better understand your application, your development process, and ways to keep you from ending up on Bugtraq they should all be ignored in favor of “do it right the first time.” As in the unicycle example it is hard for most people to do it right the first time because they don’t know what right is. There are also flaws in logic. For instance how is a company suppose to know if the are vulnerable to the “exploit of the week” as most software developers don’t know what that is. I still run into programmers that don’t think stack and heap overflows are exploitable. How do you combat that? You show them of course. That’s not a problem for me but it’s a no-no according to the anti “penetrate and patch” crowd. A glib comment could be “you should fire that programmer and hire one that can do it right the first time”. If everything is that simple why bother doing any QA, you can just hire programmers that write bug free code every time.
I am sure Ranum is a very smart and capable individual. I have used a number of his products in the past and am a fan of the design and usefulness of them. My problem with his essay is that he seems to talk about a subject he hasn't thoroughly researched or participated in which leads to an uninformed opinion. In addition, I find it odd that Ranum is the CSO of a company, Tenable, that produces tools that fall into the "penetrate and patch" heading.
Let me pause for a second and discuss marketing spin and positioning. Since we are in the midst of primary elections in the United States voters are constantly overwhelmed with a wide range of marketing messages, stories, testimonials, and other such forms of persuasion to sway people to a political parties or candidate's viewpoints. One of the staples of these efforts is vilifying your opponent. For instance if you think that the 2nd amendment should be repealed and individual gun ownership should be banned an argument would start with something like this.
"The NRA wants to arm everyman woman and child with fully automatic weapons and a legal precedent to shoot anybody on site they want. That will undoubtedly return this country to a state of constant fear and bloodshed." Of course, this is not an accurate representation of pro 2nd amendment advocates but a fair and balanced view can't be presented or you may actually convert people to your opponents cause.
That is a no-no. Dismissing this type of misinformation should be simple with some extra research. That is the beauty as the person misrepresenting the information relies on human laziness to accept an authoritative statement with no question.
You may be asking what this has to do with Ranum and why you care.
Ranum is attempting to convince you he is right by painting the ideas he is arguing against in the worst possible light and even misrepresenting some facts. I have thoughts on all his points but the one I choose to talk about the most is the "penetrate and patch idea".
Ranum wants to paint the idea of penetration testing and vulnerability hunting in general a bad idea. He lists several reasons and even gives a cutesy outline of a basic program that demonstrates his perceived flaws in the idea. The problem is that with a bit of fact checking mixed with a bit of tree shaking his reasoning starts to fall apart.
To start with the logic is flawed. His argument is literally the equivalent of saying "now that there is medicine no one should get sick anymore." He then continues with an anecdote about a programmer friend that refers to the "penetrate and patch" model and "turd polishing". From the description of the process it does sound pretty worthless. Hire some people who do not know how your application works to look for bugs. They will find some, claim success, justify the paycheck, and then do it all over again. If security research actually worked like Ranum portrays, I would have to agree, that model is worthless.
Luckily, Ranum's description is not how it actually works. I offer an explanation of how these tests actually work, a courtesy Ranum did not extend to his readers. Penetration testing is a marketing name that covers a variety of different activities that are not all the same and have different requirements, goals, and results.
Penetration Test:
Who needs it: Anybody who wants to verify that their current security posture is working. Penetration tests are to enlighten C level executives to verify or dispel a current policy, spending, and staffing needs.
How does it work: A penetration test is an activity that will assess a network's survivability to a hacker attack. This is done by using the tools and techniques a hacker would to search for flaws in a networks design, architecture, and policy. The goal is to no break-in as any network can be compromised with enough money and time. The goal is similar to game theory to look for things that might not have been considered during the network architecture.
Goals: Although vulnerabilities are unearthed, they are interpreted from a larger point-of-view.
Examples: Discovering that a user account has an easy guessed or brute forced password does not mean the resolution ends with changing the password. It means that implementing or strengthening an organizations password policy while implemented constant password auditing with user notification would resolve the problem. Being able to exploit a buffer overflow on a server for access does not mean that patching the server will fix the problem. It means that implementing a policy that servers have built in survivability from attacks with non-executable memory, address space layout randomization, heap and stack protection, and server hardening techniques would solve the problem.
Application Assessment:
Who needs it: People who have large development shops for custom applications or products. They provide a method of accountability for a products development while education the developers to different potential threats and even unintended consequences.
How does it work: An application, an operating system, or even a device like a mobile phone is analyzed and potential weakness are discovered and mitigated. This is more than just looking for buffer overflows; it is the understanding of how application actually works as opposed to how the documentation says it works. This type of understanding can pinpoint design errors and unintended consequences.
One of the advantages an audit like this brings is that normally during the development of a large and complex product there are different teams who work on different parts. For instance in an anti-virus tool there may be several teams like a scan engine team, a GUI team, an update team, and a signature team. The problem is that often these teams do not communicate and if they do its through things like developer documentation or API references, which means that one team may not understand the ramifications of the changes they make on another team. An application assessment means that a single person, or if it is a large enough application, a team of people will analyze and understand all the operations while looking for potential missteps. An application assessment is not just QA on steroids.
Examples: A security product stops threats by running them in a virtual machine and attempting to determine if software is malicious. A lack of proper checking on the part of the VM allows an attacker who is aware of its present can use the environment to hide malware as well as make untraceable changes to the host operating system. In a different example, the ability to test a mobile device GSM connection means that a test application left behind that shows the use of undocumented API functions while giving a person the list of all the ARFCN numbers in use in the area. Outsourced code is audited before being integrated into a main project to ensure that basic security standards have been met.
Vulnerability Assessment:
Who needs it: Anybody who has to prove that they are following a set of guidelines on how security should be implemented on a network.
How it works: A vulnerability assessment is what most people are actually thinking of when the phrase penetration test is mentioned. In some fashion, an auditor will look for a known set of problems to ascertain the security posture of a network. This maybe done for compliance reasons or to certify that a network is reasonably secure to handle certain types of information or transactions. They are not supposed to be insightful or explore new areas but instead verify a network meets a certain security baseline. This is important to companies like credit card processors, health care providers, and anybody that handles personal and confidential information.
Examples: A credit card processor needs to see that a potential merchant has taken the proper steps to assure that credit card information is stored safely and securely and that a network is undergoing a measure of security due diligence.
As you can see, it is a bit disingenuous to combine them all under one heading and describe them in a wash, rinses, repeat description of finding vulnerabilities. One of the main arguments Ranum makes is that if the "penetrate and patch" philosophy works them we should be running out of bugs to find. On the surface, this is a good argument and in certain conditions, I would agree with it. Those conditions would be that no future development is done on operating systems why banning any new development of commercial code. In order for Ranum's 10-year window to be accurate, that would mean that we would have spent 10 years auditing Windows 98, Redhat 3, and MacOS 7.6. If we, as an industry, were able to stop development and focus on bug, fixing just these versions over a 10 year period there would be no vulnerabilities. However, this dark period of code development is not possible. The desire of new products and functionality means that every day there is new code that contains new vulnerabilities and new problems that go into wide use. At rate of growth the IT industry is moving it will be almost impossible to catch up let alone stay current with the new tide of vulnerabilities are unwittingly introduced. It is arguably that security effort on security proofing new products should be the focus rather than looking through old products. I agree that investing in anti-exploitation technology is wise. If you look at new OS versions like Redhat, OSX, and Vista there is a lot more attention on these types of technologies. These investments are offset by the slow adoption rate of the new operating systems and is compounded by the old troubled software is not going away quickly. Recently during a penetration test I actually ran across a Windows 95 machine, talk about a blast from the past.
Ranum gives an example about the futility of running a "penetration test" for Apache against a locked down server with custom C code. I would agree. Security is no different from anything else; knowing the right tool to use is the key to success while using the wrong tool means useless results. A penetration test is useful for that example.
When doing penetration tests we specifically ask to identify all custom applications before starting the test in order to examine them in a different way otherwise it would be a waste of both parties time and effort.
Now for the entertainment portion of this post, in order better explain this position lets apply Ranum’s theory to unicycle riding. Let us say in an alternate universe that Ranum owns a Unicycle store, Ranums One Wheel. If you are used to dealing with experienced Unicycle riders glib comments like “do it right the first time” could be warranted but what happens when a novice rider comes in?
Mr. Smith (the new Unicycle rider): Excuse me sir, I have been committed to riding a unicycle in a charity fundraiser and I bought one here last week. I seem to be having a problem in its proper operation.
Ranum: Whats your problem?
Mr. Smith: I keep falling. I can’t seem to get the hang ouf it, is there any advice you can offer me?
Ranum: Of course. Don’t fall down, that is counterproductive.
Mr. Smith: Yes, I understand that part, but how do I not fall down?
Ranum: If you are wearing elbow pads, knee pads, and a helmet then you shouldn’t worry, just do it right the first time.
Mr. Smith: That’s kind of the problem, I don’t know what right is, or how to do it. Although I am wearing the required safety gear I am failing to achieve my goal of riding the unicycle.
Ranum: Then you do not have a problem.
Mr. Smith: I was thinking of hiring an instructor to look at what I am doing and tell me how to avoid falling, can you suggest someone?
Ranum: That’s what we call the “try and fall” method of unicycle riding. You will not learn anything and you will find that you are vulnerable to all types of problems like new environments and wind. It’s best if you just do it right the first time.
Mr. Smith: But I don’t undersand what right is, what It looks like, or how I accomplish it. At least with a coach I could get guidance.
Ranum: That is the kind of guidance that will leave you taking lessons forever. I am telling you just do it right the first time and do not fall.
Mr. Smith: Also I was wearing all the safety gear you told me to but i fell in a bush and got poked in the eye and had to go to the hospital.
Ranum: Never heard that one before, it must not be dangerous.
The largest problem I see with Ranum’s aversion to “penetrate and patch” is that is its purest form he is advocating ignorance. The essay boils down to saying that although there are tools and methods to better understand your application, your development process, and ways to keep you from ending up on Bugtraq they should all be ignored in favor of “do it right the first time.” As in the unicycle example it is hard for most people to do it right the first time because they don’t know what right is. There are also flaws in logic. For instance how is a company suppose to know if the are vulnerable to the “exploit of the week” as most software developers don’t know what that is. I still run into programmers that don’t think stack and heap overflows are exploitable. How do you combat that? You show them of course. That’s not a problem for me but it’s a no-no according to the anti “penetrate and patch” crowd. A glib comment could be “you should fire that programmer and hire one that can do it right the first time”. If everything is that simple why bother doing any QA, you can just hire programmers that write bug free code every time.
I am sure Ranum is a very smart and capable individual. I have used a number of his products in the past and am a fan of the design and usefulness of them. My problem with his essay is that he seems to talk about a subject he hasn't thoroughly researched or participated in which leads to an uninformed opinion. In addition, I find it odd that Ranum is the CSO of a company, Tenable, that produces tools that fall into the "penetrate and patch" heading.
Friday, April 18, 2008
Thursday, April 17, 2008
Ole Pete responds...
Pete says:
The sad thing is that my point is people like him don't understnd the link between secure software development and bug hunting.
Keep chasing those clouds Pete...
I have it on good authority below is an actual picture of Pete Lindstrom
Old people rants are funny. I wish it was not true but it is. Old people ranting about how hot their coffee is, how disrespectful teenagers are, or how it is hard to dance to music today makes most people roll their eyes a bit and chuckle. I get the same kind of chuckle everytime Pete Lindstrom posts something to his blog. If you close your eyes you can almost hear him say something like…
“I don’t understand exploits so they are bad. We should go back to a simpler time where the Glenn Miller orchestra was the hottest ticket in town. Back then in order to commit a computer crime you would need to throw a computer at someone. Teenagers also scare me with their IM chatting and emailing.”
This week’s rant from Pete says it is stupid for Microsoft to measure the success of the SDL with a metric of how many bugs are reported. I wish I could feign outrage that an organization that created a new development process should dare to elect to measure its success with a set of parameters they define. Pete seems to indicate that he thinks these metrics are just ludicrous. Of course, his outrage may be because these metrics deal a deathblow to his theories that vulnerability research has no use. Its hard to continue the rheotic that vulnerability research has no use when one of the largest software companies in the world basis the success of failure of a major initiative on the results of bug finders. I can see why he would be mad, an analyst that received a slap in the face like that from Redmond would look like he didn’t know what he was talking about…
I am sure this set back won’t keep Grandpa Pete down and he will be back soon with fresh facts of why people who find bugs are evil. I will be eagerly awaiting those funny bone rattling facts as I am sure you will be as well.
Wednesday, April 09, 2008
Tuesday, April 08, 2008
Update on Apple and QuickTime
I just read at the Infosec Blog that a new version of QuickTime has been released that contain fixes that should make QuickTime harder to exploit on Vista. I say should because although it is a good start Apple did not completely close the loop. The reason ASLR is important to thwarting hackers is that the memory space of an application is randomized, or as the king would say, they are all shook up. Since most buffer overflows rely on knowing where a piece of code or data is in memory, the randomization can turn a remotely exploitable bug into nothing more than a Denial-of-Service. Although targeted attacks against individuals may still be possible, widespread QuickTime exploits will be much harder to write.
Not to signal doom and gloom but there is a problem or two. The main problem with implementing ASLR is that is really is all or nothing venture. If you have even one static shared library you open yourself to compromise. Below are screenshots of the new QuickTime from a filesystem and a process point of view using LookingGlass. Although most of the files are now marked as ASLR enabled there are still a few binaries that are not and could still provide an attacker a static location to utilize.
Don’t let these few oversights detract you from the huge stride forward Apple is making Vista users safer. It is good to see Apple embracing these security enhancements and I encourage other vendors, like Adobe, to follow their lead. I also hope that Apple extends these improvements to the other products offered to Windows users.
Not to signal doom and gloom but there is a problem or two. The main problem with implementing ASLR is that is really is all or nothing venture. If you have even one static shared library you open yourself to compromise. Below are screenshots of the new QuickTime from a filesystem and a process point of view using LookingGlass. Although most of the files are now marked as ASLR enabled there are still a few binaries that are not and could still provide an attacker a static location to utilize.
Don’t let these few oversights detract you from the huge stride forward Apple is making Vista users safer. It is good to see Apple embracing these security enhancements and I encourage other vendors, like Adobe, to follow their lead. I also hope that Apple extends these improvements to the other products offered to Windows users.
QuickTime File system scan withLookingGlass.
QuickTime Process scan with LookingGlass.
Sunday, April 06, 2008
LookingGlass Vendor of the week:ATT
I have a Dell XPS 1330 with a built-in HSPDA card. Becasue of this I get some software branded ATT and its next up in my Program File directory. Bad ATT, no donut. This scan was done with LookingGlass 1.1 and the Microsoft SDL policy.
The stats say it all:
The stats say it all:
LookingGlass 1.1
We are releasing LookingGlass 1.1.
In addition o some bugfixes we have added the ability to define which “unsafe clib functions” you care about. Now when LookingGlass starts it will check the current directory for a file called lg-policy.txt. This file contains the policies and functions they contain. Right now there is a “Microsoft SDL” policy and a “Errata SDL” policy. To create a new policy you can just edit the lg-policy.txt and start you new policy with “#Policy:
You can find it at http://portal.erratasec.com/lg/LookingGlass.exe
Subscribe to:
Posts (Atom)
