Let’s detect the IoT search engines, from Fofa to Shodan
Hunting the hunters is fun, but let’s starts from the background.
In this article, I will show how can we detect Shodan and Fofa user-agents, and who already made progress.
Those search engines are dedicated to map the Internet Of Things and other sensitive devices.
I like them very much, but I think they come with a price, everything being exposed at once, with no time to fix vulnerabilities. Legally, those scanner activities are against the ruling made by the Supreme Court of Israel, but let’s leave it for now to focus on the technical aspect.
What can you do in order to prevent an IoT search engine from leaking sensitive databases, and scanning exploited devices, like smart houses?
As we all know, nowadays, many houses are being connected to the internet,just like critical infrastructure and other devices that have connected to the internet for many years.
Unlike Google, which is focusing mostly on websites, those search engines are dedicated to cataloging sensitive findings, connected devices, databases and other things we want to prevent from falling into the wrong hands.
What should smart house vendors do to protect users from those search engines?
Is it even legal to scan the house?
Today I had an interesting finding.
I was looking at errors in IoT search engine called Fofa, and realized something interesting.
It was saying:
E\x00\x00\x00\xffj\x04Host ‘*.*.*.*’ is not allowed to connect to this MySQL server
It was very interesting because I never saw anyone speaking about how to prevent those engines from entering houses. I did mention the legal aspect of it, but let’s forget about the law and keep digging.
After realizing that this is Fofa user-agent, I used Google to check if anyone mentioned this string before, none. Only Google was mapping Fofa activity in the wild.
So I was thinking, let’s see what Fofa has done before? How many times did it get blocked while using this string? Well, the numbers were very high, 840696 times.
Query: “E\x00\x00\x00\xffj\x04Host ‘*.*.*.*'”, Total results: 840696，took 4545 ms，mode: normal.
默认只显示一年内的数据，点击 all 链接查看所有。
Now I was thinking, what about Shodan, can we look for Shodan in the wild?
Googling this subject lead me to results from a website called “Webmaster World” and back in June 2016, someone shared information about Shodan’s strange behavior.
While reading the post, I gained user agents, which seems to be used by Shodan.
Now I had dorks to hunt.
While analyzing the findings, I was thinking maybe it’s a start. Why don’t we build a database of IoT search engines, so developers can use it and try to prevent them from adding devices and sensitive data?
However, after some searching, I’ve realized that some researchers from the academic field, already made progress and published a research paper about this subject during the 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN 2017).
The article name is “Abnormal Behavior-Based Detection of Shodan and Censys-Like Scanning”.
Here is the abstract they wrote:
“Shodan and Censys, also known as IP Device search engines, build searchable databases of internet devices and networks. Even these tools are useful for security, those also can provide the vulnerabilities to malicious users. To prevent the information disclosure of own IP devices on those search engines, a fundamental solution is blocking the access from the scanners of them. Therefore, it is needed to understand and consider their scanning mechanism. Therefore, we propose an abnormal behavior based scan detection of Shodan and Censys. To do this, several traditional scan detection approaches are combined and applied to satisfy their specification. Proposed idea is monitoring packets whether it is abnormal or not and adding on the suspicious list if it is. This is based on traditional threshold approaches. To figure out it is abnormal, stateful TCP stateful packet inspection is used. The response behavior during the connection can be identified with TCP flag and abnormal behavior can be classified with SYN Scan, Banner Grabbing, and Combined SYN and Banner Grabbing. Demonstration is simulated in a Censys-like environment and detected time variation per variance of distributed detectors and Threshold value is analyzed.”
Most of the projects offer solutions based on IP lists, and less user agents, or just looking only on Shodan and Censys, without giving attention to the Chinese based competitors.
As for Censys, in their website, they have an explanation of how to prevent them from scanning, yet, they won’t delete results.
“Can I opt-out of Censys scans?
Censys scans help the scientific community accurately study the Internet. The data is sometimes used to detect security problems and to inform operators of vulnerable systems so that they can be fixed. If you opt-out of the research, you might not receive these important security notifications.
However, if you wish to opt-out, you can configure your firewall to drop traffic from the subnets we use for the measurements: 18.104.22.168/24 and 22.214.171.124/24. We do not remove results from Censys, but if you have blocked these subnets, the results will automatically be pruned out.”
To summarize, I think IoT search engines are great, they are really helping security researchers and basically for the safety of all. Scanning engine activities might be illegal in some countries, yet, it’s helping to detect problems and push vendors into solutions.
As from the vendors and the end users’ aspects, they might not be happy to know that their house or product are now out there, not protected and easy to attack.
I know that tools that detect port scanning are nothing new, but being focused on search engine activity, and banning and blocking them locally from adding sensitive information into the catalog of things, might help in many cases when a solution is not coming soon, and fixing won’t be done before the attacker will be able to take advantage.
We should balance between the freedom to know everything, the interest of security researchers to get data about exploited devices, and the rights for personal and public safety.
Giving the public abilities to detect user-agents of Internet of Things devices is something to start with.
Now, let’s hunt the hunters
Let’s hunt Shodan, ZoomEye Fofa and Censys.
Let’s build databases of user agents that belong to IoT search engines.
Originally posted here: http://popshark11.blogspot.com/2018/10/lets-detect-iot-scanner.html?m=1