Lambda + Python is good at restricting access with a large number of IP address lists

Background

We operate a service called Cloud-based e-learning system eden LMS, and the other day we had such an exchange with a customer.

"Can eden LMS set IP address restrictions?" "Yes, you can." "There is a pattern like 192.168.1.0/24, can you handle it?" "Of course. By the way, how many patterns do you allow?" "There are more than 2000" "2000 or more" "And the contents of the list change often." "Oh, oh ..."

How to implement?

eden LMS is written in Python, and if you use a library called netaddr made by Python, you can easily implement the judgment itself such as "Is 192.168.1.100/32 included in 192.168.1.0/24?". However, if the subnets are disjointed, it will be necessary to determine whether or not they are included in the allowed IP address list for the number of elements in the list. This seems to be a considerable CPU load if implemented foolishly honestly. Since it is a multi-tenant cloud service, such processing must be avoided.

Expand all lists like "192.168.1.0/24" as a list of strings like "192.168.1.1, 192.168.1.2, 192.168.1.3 ...", and the string of the IP address of the access source. It seems that the implementation (judgment like ipaddr in allowed_list) will be enough speed even if the list is included ... However, I decided to use AWS Lambda to make it a microservice.

With AWS Lambda, you can keep your implementation simple for the following reasons:

--Since you don't have to worry about CPU load, you can use netaddr to judge a list like "192.168.1.0/24". --Include a list of IP addresses in the file on the AWS Lambda side, and when the list is updated, you only have to update the Lambda side without touching the service itself.

Actual source code

First of all, on the AWS Lambda side, we just need to match the IP address determination using the netaddr library to the AWS Lambda format.

lambda_function.py


from netaddr.ip import IPNetwork,IPAddress

def lambda_handler(event, context):
	ipaddr = event['ipaddr']
	for network in allowed_networks:
		if IPAddress(ipaddr) in network:
			return True
	return False

allowed_networks = [
	IPNetwork("192.168.2.1/24")
	,IPNetwork("192.168.3.1/24")
	,IPNetwork("192.168.4.1/32")
]

After zipping this lambda_function.py and netaddr library folder and uploading it to AWS Lambda side, add the API endpoint. Now you are ready to go.

The caller of the API looks like this: (Security is open for the time being, but in reality IAM etc. should be used)

lambda_call.py


# -*- coding: utf-8 -*-
import urllib2
endpoint = "https://xxx.amazonaws.com/prod/apiname" #Contains the actual endpoint
req = urllib2.Request(endpoint)
req.add_header('Content-Type','application/json')
response = urllib2.urlopen(req,'{"ipaddr":"192.168.4.1"}')
print response.read()

After that, if you cache the determined result in KVS such as memcached, it seems that calling AWS Lambda will be minimal.

Summary

For processing that "has a large difference in processing volume between peak and non-peak" and is "referentially transparent", it would be easier to make it a microservice with AWS Lambda.

Recommended Posts

Lambda + Python is good at restricting access with a large number of IP address lists
Accelerate a large number of simple queries with MySQL
[Python] Randomly generate a large number of English names
Consolidate a large number of CSV files in folders with python (data without header)
Scrapy-Redis is recommended for crawling a large number of domains
Executing a large number of Python3 Executor.submit may consume a lot of memory
[Python] Correlation is below a certain level ・ Maximum number of features
[Python] What is a with statement?
A good description of Python decorators
I want to backtest a large number of exchange pairs and strategies at once with Python's backtesting.py
What is the XX file at the root of a popular Python project?
ETL processing for a large number of GTFS Realtime files (Python edition)
Get the number of searches with a regular expression. SeleniumBasic VBA Python
Connect a large number of videos together!
[python] [meta] Is the type of python a type?
One-liner to create a large number of test files at once on Linux
Turn multiple lists with a for statement at the same time in Python
3. Natural language processing with Python 3-3. A year of corona looking back at TF-IDF
[AtCoder] Solve A problem of ABC101 ~ 169 with Python
Upload a large number of images to Wordpress
Calculate the total number of combinations with python
Solve A ~ D of yuki coder 247 with python
Rewriting elements in a loop of lists (Python)
Organize a large number of files into folders
Judge whether it is a prime number [Python]
Get a large amount of Starbucks Twitter data with python and try data analysis Part 1
How to identify the element with the smallest number of characters in a Python list?
[Shell art] Only when it is a multiple of 3 and a number with 3 becomes stupid
A tool to follow posters with a large number of likes on instagram [25 minutes to 1 second]
I thought about why Python self is necessary with the feeling of a Python interpreter