Lately we all heard of Silk Road the underground cyber marketplace were you can buy illegal drugs and guns and people say all the bad guy’s are using the dark web for crime stuff – yeah DuDe:—:. It’s is just the Tor onion network, if you want to visit the onion network just go to torproject.org and download their bundle software and go surfing in the onion network. Since there is no bing, google or yahoo in the onion network, if you want a directory of what’s out in onion land just go to the hidden wiki. “Cleaned Hidden Wiki”-http://3suaolltfj2xjksb.onion/hiddenwiki/index.php/Main_Page.
The wiki is built by one of the founders of the onion netowk the administrator of MyHiddenBlog in – (- “http://utup22qsb6ebeejs.onion/” — ) and volunteers built The “Cleaned Hidden Wiki” .It is one of the few places were you can find some of the hidden services (websites) in Tor, in other words the only websites in Tor that want to be found. You see in the Tor onion network your site is your secret, your site is hidden because there is no google or yahoo to send web crawler out into the onion network. The USCyberlabs Tor Network Directory Project will be the first time that we go out actively and collect all the websites (hidden services) that are hiding in the Tor onion network.
When I started to write about Tor and our new (“The Deep Dark Web”) -book, I was contacted by the FBI about what I was writing about Tor and the hidden services and attack vectors in Tor. They wanted to be gAtO’s bff. I must admit I was intimidated and walked a very careful line with my blog postings and my tweets. Why because the FBI want to fuck with lawful security researchers that come to close to the truth about Tor.
They do not want this mapping of the Tor onion network. Why? The mapping of the Tor onion network will show all sites even the ones that want to stay hidden. Like government sites? Like Spy sites? I mentioned Bots with Tor c&c yeah government stuff. You of course have your corporate presence in the hidden services of Tor what will these Tor website show. Maybe it’s not just the bad guy using Tor, maybe.
There are currently only 9 directory servers in the Tor infrastructure that know all the sites on Tor and getting this list is kind of hard. Tor is design not to give out directory information to just anyone. We also want more than a URL of a live site, we will gather all meta-data so we can understand what these sites are all about. Google’s web crawlers do it every second of the day so we will send out crawlers into the Tor onion network to generate our directory of Tor.
The ToR Directory Scan Project (TDS)
The uscyberlabs TDS Project- is to scan every address possibility and to generate a directory of every single live hidden service in the ToR-.onion network.
Figuring out the rendezvous for a hidden service is complicated, we attack the problem from the side —>> so the onion URL is 16 digits 2-7 a-z plus the .onion after the url address. It’s easy to have a simple web crawler program count and a,b,c and generate a sequential-alphabetized URL list. Now due to the ToR network things work slow – old style modem speed that you young kids are not used to. Now we feed a URL wait up to 25-35 seconds then list a positive or no-go. Once we have a live hit list of possible live hidden services then we visit manually. And build a working verified w/login and password list of every hidden service on ToR.
with 100 VM we can scan Tor in weeks with 1000 machines we can scan the Tor network within days.
I tested the unix “curl command” in Tor with sock5 and it’s very good at extracting information from a website. So a simple script in will feed all the machines and they will start the scan. Once finish we take all the results and we will have a directory of every single hidden service in Tor land.