Tuesday, October 11, 2011

Scanning the Internet

As part of a research project we are port scanning the entire internet. The scans will come from 216.75.60.94 and now 66.240.192.147.

EDIT: Per a comment I realized I left alot of stuff out. Here ya go:
I am scanning everything from 1.0.0.1 to 223.255.255.255.
I am collecting hostname, IP address, OS type, and service version.
As far as how long I have no idea, I am guessing somewhere around 100 days.
I was way off on this, Its Almost a year later and we are still running.

I am aware Shodan offers this information now, I need to collect my own data for this project however.

EDIT: This isn't a big deal. Researchers like us frequently scan the IPv4 address space. At any point in time, there are a few "white-hat" researchers doing such scans (we know of one other group currently conducting a scan), and many more "black-hats" doing it. The reason for this post is simply to be on record about it.

EDIT: We have added a new IP Address we are scanning from: 66.240.192.147. This is a followup machine that takes the oldest entries in our databases and checks if they are still alove/resemble what we collected.

19 comments:

Richard Bradshaw said...

How many clients are you expecting to scan? What exact data are you collecting? How long are you expecting it to take?

Would be interesting to include info such as response time, OS etc as well as just which ports.

David Maynor said...

Thanks for the comment, I updated the post with more information. I was mostly using this as a place holder for a link I put on the scanning boxes web server incase anybody looked it up.

Robert Graham said...

Richard,

We are doing an IPv4 wide scan. It's supposed to be slow, taking at least a month. The intent is an independent record to compare to other sources of Internet-wide information.

The purpose of the blog post was to make sure we were upfront and clear about it. We could do a stealth scan, but we'd rather just have everyone know we are scanning.

Jason Ross said...
This comment has been removed by the author.
Jason Ross said...

Just so you know, the data will lack significant meaning as Im sure within minutes of posting this you have already been blocked at people border firewalls. And those that dont see this will block you when they start seeing sequential scans on their networks. This means you will be missing data from certain environments and will result in very skewed data. While I will be interested in seeing the results, I would be very skeptical of any attempt to derive meaning from them.

Robert Graham said...

Jason,

That is very much part of the goal, to see how people's reactions distort the data.

Juan Miguel Paredes said...

Are you planning on making the raw information you collect available for the general public?

Robert Graham said...

Yes, Miguel, we'll write up a report.

An early result is that we get more SYN-SYNACK-ACK-RST combinations than I thought we would.

QaSaR said...

Hi Robert,

As a follower of your blog, is my first coment, so first of all, thx for your blog, your time, and your posts and ideas, there is a long time i read you, but never posted before =)

Now, have i think about the fact that big ISP buy big ranges of IPs, and this makes some IPs dynamic asigned to ISP clients?
I mean, depending on result, maybe would be more interesting, filtering or tracing the jumps to the ISP node, asi those "ISP finnal client IPs" will change constantly his IP when loggoff or whatever makes Dynamic IP Clients change/reassign his IP, no?
But would be more clear to see with range belongs to wich ISP, and maybe have some way to see Static IP, or Dynamic IP (i dont think this would be easy/possible), but would be usefull to make a map of ISP IP ranges and so on?

B

0xdeadbeef said...

Been twitted by Team Cymru ;-)

Unknown said...

Will you make some of your results public/post your scanning methodology?

Kevin Halgren said...

Your scanner set off our sensor because is sent an echo request with code 9. Code 9 is undefined and will likely be blocked by many systems, Code 0 would be the only proper ICMP code for an echo request. Presumably you have something crafting the packets instead of using standard tools for some reason.

Anonymous said...

Now seeing your scanner wasting it's time on our darknets for the 2nd time it would be nice if emails towards info@erratasec.com would be answered.

Anyway, if you need an hour to scan an unused IP when do you expect to be through?

Anonymous said...

Guys - sometime while we are playing hookie from work, you will have to explain this to me...I'm completely ignorant, but the scope of this project seems huge and makes me curious!!!!

Dan

Anonymous said...

David, I am being scanned by 216.75.60.109. Is this one of yours, or are you the victim of a copy cat?

David Maynor said...

Its one of mine, I will compose a new list of IPs and publish them.

Andrea said...

I'm triggering today 'BOT:ZeroAcces traffic Detected" allerts about your traffic.
Source ip 209.126.230.71 in UDP protocoll destination port 16471.
Can you confirm it?

Andrea.

Unknown said...

Nice to make the security teams of the targets have to investigate.Sure I support security research, but this behavior causes companies to expend resources which = $$.

Unknown said...

Nice to make the security teams of the targets have to investigate.Sure I support security research, but this behavior causes companies to expend resources which = $$.