Use and integration of "Shodan"

In this article, I will talk about ** "Shodan" **, how it works, and how to incorporate it into tools and scripts.

What Shodan can do

Shodan detects devices that are connected to the Internet at any given time, the location of those devices, and the current user. Such devices can be present in almost any type of system, including business networks, surveillance cameras, industrial control systems (ICS), and smart homes. Shodan is the first important step in penetration testing because trying to get the system banner directly helps identify vulnerable systems. He can also do more by supporting the Boolean operator and provides filters to improve search efficiency. Search engines offer 50 results for free and offer paid subscriptions for a wider range of results.

How it works

There are three ways to use Shodan.

--Web interface

For this tutorial, we will use Alibaba Cloud Elastic Compute Service (ECS). In order to secure your instance, you need to configure it accordingly.

--Create rules to allow traffic in and out of security groups --Create a pen test approval We won't go into server configuration here, but see the ECS documentation (https://www.alibabacloud.com/help/ja/product/25365.htm) for more information.

After doing the above two setups, you can install Apache on your ECS and access it with a web browser to test it.

Shodan web interface

Shodan, like most other search engines, can be accessed by accessing shodan.io in a web browser. I can. image.png

I would like to know how many servers are using Apache around the world. Just enter the keyword "apache". image.png

As you can see in the image, the results show that 25,544,783 servers are running on the Internet.

Shodan has many other features, but you need to register to use them. An error occurred when trying to do a deep analysis without an account, as shown below. image.png

Now let's say you have successfully created an account and logged in. One of Shodan's features is the filter, but let's see what you can do with it.

Filters are special keywords that Shodan uses to filter search results based on service or device metadata. The format for entering the filter is

filtername:value

For example

For example, suppose you want to do an accurate search by searching all Apache servers with Tomcat.

The keyword is "product:" apache tomcat "". The results are in the screenshot below.

image.png

As a result, we found that there are 1,445,150 servers running Apache Tomcat in the world.

You can further refine your research by looking for all Apache Tomcat servers in China.

Product:"apache tomcat" country:cn

What I did was ask Shodan to search for Apache servers in Tomcat, China.

image.png

The results show that there are 409,609 Apache servers with Tomcat in China.

Now suppose you are running a service on one of the ECSs and want to see what is happening. In my case, as mentioned above, I use Alibaba ECS as an example.

To do the task, we need to use a net filter and the value will be our IP_address.

Net: IP address

Suppose you have Apache installed, considering that your security group has not yet created a rule to allow traffic in and out. When I try to access it by entering my IP address in the navigator, I can't see anything because Alibaba Cloud restricts traffic by default. This is to ensure the security of the ECS server even if you have not created any rules for the security group.

image.png

As you can see in this image, the server is running on Alibaba and the IPS is Aliyun Computing. Since we are using two ports (80,443), we have two services running on the server, and the web technology used is jQuery. This will be part of the information that hackers collect about your server to see if it is vulnerable, such as an SSL version. ..

Now let's see how to use Shodan using the command line interface (CLI).

Shodan command line interface

Shodan's command line interface (CLI) is packaged with the official Python library for Shodan. To install a new tool, just run it.

$ easy_install shodan

After installing the tool, you need to initialize it with your API key.

$ shodan init YOUR_API_KEY

To https://account.shodan.io Go to it and get the API key for your account. It will be displayed in the upper right corner of the My Account button. Once the API_KEY is initialized, you can now start using Shodan commands.

You can check the account information.

Now, imagine you are on an instance of ECS and want to know what your public IP address is. With ifconfig, Alibaba Cloud protects the service by default, so you only know the private IP address. If you want to know your public IP address, just enter the command Shodan my ip.

$ shodan myip

image.png

The result shows my public IP 47.89.249.0.

If you want to have information about a host, we can now use Shodan, such as where such a host is, which port is open, which organization owns the IP, and so on. Let's say 42.120.226.13 is our server and we want to test it to see what's running.

$ shodan host 42.120.226.13

image.png

Our server is in China, last updated 2019-01-21, 2 ports are working (80, 443), the result gave us multiple versions, so check the SSL version need to do it.

The CLI has other features such as network analysis, maltego add-ons, and browser plugins, but you will need to upgrade your plan account to use them.

Script Shodan

Shodan provides a developer API for programmatically accessing the collected information. All websites and tools, including Shodan's main site, use this API. Everything you can do via a website can be achieved from your own code.

The API is divided into two parts. "REST API" and "Streaming API". The REST API provides a variety of utility methods for searching for Shodan, looking up hosts, getting summary information for queries, and facilitating development. The Streaming API provides a raw, real-time feed of the data that Shodan is currently collecting. There are several feeds you can subscribe to, but you can't search for data or interact with it in any other way.

Usage restrictions

There are three API methods that are restricted by the API plan.

1, ** Searching **: Shodan uses query credits to limit the number of searches that can be performed in a month. One query credit is used when you perform a search that includes a filter or when you pass the first page. For example, if you search for "apache", no query credits will be used. Also, if you search for "apache country: US", 1 query credit will be used. Similarly, if you search for search results on the second page with "apache", use 1 query credit. Finally, the search query on the second page of "apache country: US" also uses 1 query credit.

2, ** Scanning **: The On-Demand Scanning API uses scan credits to limit the number of hosts that can request Shodan to scan each month. Deduct 1 scan credit for all hosts you request for Shodan scans.

3, ** Network Alerts **: The number of IPs that can be monitored using alerts is limited based on your API subscription. Only paid customers can access this feature. Also, you cannot create more than 100 alerts for your account.

** Note **: Query and scan credits will be reset at the beginning of each month.

To install the Shodan library for Python, run the following command.

$ easy_install shodan

If you already have it installed and want to upgrade to the latest version.

$ easy_install -U shodan

The first thing you must do is initialize the Shodan API object.

import shodan
api = shodan.Shodan('YOUR API KEY')

Here, your API key is the API key for your account and can be obtained from: https://account.shodan.io

Now that you have all the APIs, you can search with a small script. image.png

As you can see in this image, we are writing a small script that is trying to scan the internet for servers and queries. The code is written in python3.

When you run the script with a parameter called Apache, Shodan scans the internet and provides all the Apache server information.

Let's run the script and see if it works and what the result will be. image.png

The result of the script looks like the one above. As you can see, you can see that the Apache server is running on the IP address.

Conclusion

As technology evolves, we need to be up-to-date to ensure that our security is not compromised. Especially in IoT applications where multiple devices are exposed to the Web, security is important not only to ensure the proper functioning of assets, but also to protect privacy. One of the most powerful tools available for that is Shodan. But like all good tools, Shodan can also be a double-edged sword. It can easily be exploited by hackers, but it also helps you better understand your network. It is not the tool itself that is dangerous, but rather the person who uses it.

Recommended Posts

Use and integration of "Shodan"
python development environment -use of pyenv and virtualenv-
Data cleansing 3 Use of OpenCV and preprocessing of image data
Convenient use of ipython
Vue-Cli and Python integration
Use SQLAlchemy and multiprocessing
Graceful use of MySQLdb
Personal notes about the integration of vscode and anaconda
EP 7 Use List Comprehensions Instead of map and filter
Problems of liars and honesty
Pre-processing and post-processing of pytest
Use pyrtm and RTM CLI
Combination of recursion and generator
Combination of anyenv and direnv
Explanation and implementation of SocialFoceModel
Python application: Data cleansing # 3: Use of OpenCV and preprocessing of image data
[Python] Summary of how to use split and join functions
Differentiation of sort and generalization of sort
Coexistence of pyenv and autojump
Comparison of how to use higher-order functions in Python 2 and 3
Problems of liars and honesty
Introduction of DataLiner ver.1.3 and how to use Union Append
Occurrence and resolution of tensorflow.python.framework.errors_impl.FailedPreconditionError
Comparison of Apex and Lamvery
Source installation and installation of Python
Introduction and tips of mlflow.Tracking
Wagtail Recommendations (3) Understand and use the tree structure of pages
I want to use both key and value of Python iterator
Environment construction of python and opencv
Let's use usercustomize.py instead of sitecustomize.py
Various of Tweepy. Ma ♡ and ♡ me ♡
Basic knowledge of Linux and basic commands
The story of Python and the story of NaN
Explanation and implementation of PRML Chapter 4
Use instance method and class method properly
Introduction and Implementation of JoCoR-Loss (CVPR2020)
Benefits and examples of using RabbitMq
Explanation and implementation of ESIM algorithm
Danger of mixing! ndarray and matrix
Installation of SciPy and matplotlib (Python)
Significance of machine learning and mini-batch learning
Introduction and implementation of activation function
Summary of how to use pandas.DataFrame.loc
[Python / matplotlib] Understand and use FuncAnimation
Memorandum of saving and loading model
Let's use tomotopy instead of gensim
Proper use of Python visualization packages
Read and use Python files from Python
Throw away curl and use httpie
Summary of how to use pyenv-virtualenv
Misunderstandings and interpretations of Luigi's dependencies
Make full use of Python's str.format
Explanation and implementation of simple perceptron
Calculation of homebrew class and existing class
[Python Queue] Convenient use of Deque
[Note] WSL2 kernel build and use
This and that of python properties
How to use .bash_profile and .bashrc
How to install and use Graphviz
Survey for practical use of BlockChain
Design of experiments and combinatorial optimization