I heard that AWS Lambda supports Python 3.6, so I decided to write a Lambda Function written in Node.js in Python, but Lambda including a module written in C such as numpy When I deploy a Function from a Mac, it's hard to run it on Lambda because it says ** "It's a module built for Mac so I can't load it" **.
I solved this problem with Docker, so I will summarize how to do it.
[Added on 2020.03.25] I also wrote about how to create a Lambda Layer. Create a Layer for AWS Lambda Python with Docker-Qiita
This time, consider deploying a Lambda Function that does nothing but load numpy like this:
Directory structure
.
├── main.py
└── requirements.txt
main.py
import numpy
def handler(event, context):
pass
requirements.txt
numpy
The following command installs numpy in the same directory, but even if I zip and deploy the installed one, it doesn't work on Lambda because numpy is a build for Mac.
$ pip install -r requirements.txt -t .
You should probably get this error.
Unable to import module 'main':
Importing the multiarray numpy extension module failed. Most likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all files not under version control). Otherwise reinstall numpy.
Original error was: cannot import name 'multiarray'
When I google the solution, I get only a mysterious answer like "Launch an Amazon Linux instance and build it there". ** I want to be serverless with AWS Lambda, but I need a server for deployment ** How is this? ?? ??
In short, I wish I could build in a Linux environment. You can do that with Docker.
To execute the above command on the Docker container, do as follows.
$ docker run --rm -v $(pwd):/work -w /work python:3.6 pip install -r requirements.txt -t .
Start the container from the Python 3.6 image [^ linux], mount the current directory in the working directory / work, and install the package there. When the build is complete, this container is no longer needed, so I added the --rm option to remove it.
[^ linux]: python: 3.6 The image is based on Debian, so it is different from the Lambda execution environment (Amazon Linux), but it was installed numpy-1.13.1-cp36-cp36m-manylinux1_x86_64.whl So I think Linux is fine. There may be some bad modules other than numpy, but in that case you can drop the Amazon Linux Docker image.
Now that you have numpy built in a Linux environment, you should be able to solidify and deploy it.
You might wonder, "Once you've installed a module for Linux, you can't run it on your Mac during development."
Sure, it can't run on a Mac, but I think it's best to run it all in a Docker container when developing.
$ docker run --rm -v $(pwd):/work -w /work python:3.6 python -c 'import main; main.handler({}, None)'
Like this.
Recommended Posts