Import your own functions on AWS Glue

Stumble by importing your own function

Simple import is not possible when writing common processing or complicated functions in a separate file on AWS Glue. That said, I just added the file path when creating the job, but I didn't know the relevant part of the document, so make a note of it.

Import method

Basically Using Python libraries with AWS Glue (https://docs.aws.amazon.com/ja_jp/glue/latest/dg/aws-glue-programming-python-libraries.html) It can be handled by referring to the document in.

The basic flow is to zip the python file, put it in S3, and set the script path.

This time we will import the following two simple functions. It is one function and one file.

hello_world.py

def hello_world(name):
    return 'Hello World, ' + str(name)

calcu.py

def sum(x, y):
    return x + y

I put these two files together in a zip and created a zip file called lib.zip this time. I will give this zip file to s3. Suppose you put it in the path s3: //example_backet/lib.zip. Put this path in the choice Python Library Path when creating or editing the job. If there are multiple zip files, enter them separated by commas.

Now you can import as usual after creating the job.

sample.py

from hello_world import hello_world
from calcu import sum

hoge = hello_world('hoge') 
sum = sum(1, 2)