③ Create a Jupyter notebook

When the status becomes "In Service", press "Open Jupyter" to move to the top screen of the Jupyter notebook. When you go to the top screen of Jupyter notebook, specify "conda_python3" in "new" on the upper right and create a notebook, it will be as shown in the figure below.

Import the required configuration variables and libraries. Change the bucket item below to the S3 bucket name you decided earlier. Also, if the regions of SageMaker and S3 are different, it will not work, so be careful if the regions are the same. The folder specified by prefix is newly created in S3. No particular change is required, but there is no problem even if it is changed as appropriate. Finally, declare the roles of boto3 and IAM. Although it is boto3, it is a library that integrates Python developed by AWS and various AWS services.

To execute a cell, press Shift + Enter. When the execution is completed, it will be numbered as shown in In [1] in the figure below. Occasionally, the name of the cell will be In [*], but that's just because the cell is taking a long time to execute, so wait for a while before proceeding to the next step.

In[1]

#Please set the bucket name of S3 to the following
#Set S3 prefix (no change required)
bucket = 'bank-xgboost'
prefix = 'sagemaker/xgboost-dm'
 
#IAM role declaration
import boto3
import re
from sagemaker import get_execution_role
 
role = get_execution_role()

Next, import various libraries required for this model construction into. In addition to Numpy, Pandas, and Matplotlib, which are standard machine learning, modules for plotting tables with ipython (Jupyter Notebook), and SageMaker's Python SDK are also imported.

In[2]

import numpy as np                                #For matrix operations and numerical processing
import pandas as pd                               #For changing tabular data
import matplotlib.pyplot as plt                   #For visualization of figures, etc.
from IPython.display import Image                 #To display an image in a notebook
from IPython.display import display               #To display the output in notebook
from time import gmtime, strftime                 #For labeling SageMaker models, endpoints, etc.
import sys                                        #For writing output to notebook
import math                                       #For ceiling function
import json                                       #For analysis of hosting output
import os                                         #To manipulate the file path name
import sagemaker                                  #Many helper functions are provided by using Amazon SageMaker's Python SDK
from sagemaker.predictor import csv_serializer    #Convert the HTTP POST request string during inference

④ Download, survey, and convert training data

The dataset is available on the University of California, Irvine website, so get it directly from there. Download directly from the URL with wget and unizip.

In[3]

#Download the dataset from the public URL of the University of California, Irvine
!wget https://archive.ics.uci.edu/ml/machine-learning-databases/00222/bank-additional.zip
!unzip -o bank-additional.zip

Load the downloaded csv file dataset as a Pandas dataframe.

In[4]

# bank-additional-full.Store csv in data
data = pd.read_csv('./bank-additional/bank-additional-full.csv', sep=';')
 
#Changed Pandas maximum display column and row count settings
pd.set_option('display.max_columns', 500)
pd.set_option('display.max_rows', 30)
 
#Show first 10 lines
data.head(10)

If you look at the first 10 lines of the dataset, you'll see that it contains customer information. Here is a part of the outline of the items.

· Age – Customer age · Job – Job category ・ Marital – Marriage status ・ Education – Educational background · Default – Credit late payment status ・ Housing – Whether or not there is a real estate loan ・ Loan – Whether or not you have a personal loan, etc ...

Check for missing data. If there is missing data, it is troublesome because it is necessary to discuss how to deal with it separately.

In[5]

#A table function that collects missing data in a data frame
def missing_values_table(df): 
        mis_val = df.isnull().sum()
        mis_val_percent = 100 * df.isnull().sum()/len(df)
        mis_val_table = pd.concat([mis_val, mis_val_percent], axis=1)
        mis_val_table_ren_columns = mis_val_table.rename(
        columns = {0 : 'Missing Values', 1 : '% of Total Values'})
        return mis_val_table_ren_columns 
 
#Check if data is missing
missing_values_table(data)

Looking at the figure below, it can be observed that there is no data loss. As expected, bank datasets.

Next, data cleansing (data set preprocessing) is performed. Dataset cleansing has become an integral part of almost every machine learning process.

This time, four types of pretreatment are performed.

** ① Extract customers who have not been contacted from pdays ** The pdays (how many days have passed since the last contact) has a large amount of "999" data, and it can be said that most customers have not contacted since the last time. Data processing is performed with 999 days (that is, customers who have not been contacted) as "1" and the others as "0".

** ② Extract customers who are not currently in the job from job ** There are 12 types of job items, including unknown. This item includes customers who are not currently in employment, such as "student" or "unemployed". Therefore, the process of separating whether it is currently working or not and adding a new item "not_working" (not working) is performed.

** ③ Make categorical data a dummy variable ** The method of digitizing non-numerical data is called a dummy variable. For example, in the case of this data, the prediction target is "y", but the values it has are "yes" and "no". When this is made into a dummy variable, one item "y" is originally divided into two items "y_yes" and "y_no", and "0" and "1" are given according to each value.

** ④ Delete items that are not included in the forecast model ** Finally, use pd.drop to delete items not used in this training, such as the index of external environmental factors (emp.var.rate), from the data frame.

In[7]

#Addition of new items to identify people who have not been contacted before
data['no_previous_contact'] = np.where(data['pdays'] == 999, 1, 0)
 
#Added "People who are not in employment" (students, etc.) flags from occupation
data['not_working'] = np.where(np.in1d(data['job'], ['student', 'retired', 'unemployed']), 1, 0)
 
#Make categorical data a dummy variable
model_data = pd.get_dummies(data)
 
#Deleted items not used in this model
model_data = model_data.drop(['duration', 'emp.var.rate', 'cons.price.idx', 'cons.conf.idx', 'euribor3m', 'nr.employed'], axis=1)

After the pre-processing is completed, the next task is to divide the data into training data, verification data, and test data.

Regarding why it is divided, to put it simply, I want to train it many times to improve the accuracy of the model, so I will leave it to the point of view this time.

In[8]

#Preprocessed model_Randomly sort data into 3 data frames
train_data, validation_data, test_data = np.split(model_data.sample(frac=1, random_state=1729), [int(0.7 * len(model_data)), int(0.9 * len(model_data))])   # Randomly sort the data then split out first 70%, second 20%, and last 10%

Next is the final step of data preprocessing. The data format of the Amazon SageMaker XGBoost container is libSVM. Although it is in libSVM format, the feature (feature amount) and the prediction target (objective variable) must be set as separate arguments, so that process is performed. Finally, send this training dataset (libSVM format) to AWS S3 via boto3.

In[9]

#Export libSVM file
dump_svmlight_file(X=train_data.drop(['y_no', 'y_yes'], axis=1), y=train_data['y_yes'], f='train.libsvm')
dump_svmlight_file(X=validation_data.drop(['y_no', 'y_yes'], axis=1), y=validation_data['y_yes'], f='validation.libsvm')
dump_svmlight_file(X=test_data.drop(['y_no', 'y_yes'], axis=1), y=test_data['y_yes'], f='test.libsvm')
 
#Copy files to S3 using Boto3
boto3.Session().resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'train/train.libsvm')).upload_file('train.libsvm')
boto3.Session().resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'validation/validation.libsvm')).upload_file('validation.libsvm')

You now have a new folder called "sagemaker" in your first AWS S3 bucket. If train.libsvm is newly created in the directory of S3> bucket name> sagemaker> xgboost-dm> train as shown in the figure below, you can proceed without any problem.

⑤ Train the model

Now that the pre-processing is complete, it's time to build the model using XGBoost.

First, specify the location of the ECR container for XGBoost of Amazon SageMaker, and link the training data (libSVM) with S3.

In[10]

#Specify ECR container for SageMaker XGBoost
containers = {'us-west-2': '433757028032.dkr.ecr.us-west-2.amazonaws.com/xgboost:latest',
              'us-east-1': '811284229777.dkr.ecr.us-east-1.amazonaws.com/xgboost:latest',
              'us-east-2': '825641698319.dkr.ecr.us-east-2.amazonaws.com/xgboost:latest',
              'eu-west-1': '685385470294.dkr.ecr.eu-west-1.amazonaws.com/xgboost:latest'}

#I will link training data and S3
s3_input_train = sagemaker.s3_input(s3_data='s3://{}/{}/train'.format(bucket, prefix), content_type='libsvm')
s3_input_validation = sagemaker.s3_input(s3_data='s3://{}/{}/validation/'.format(bucket, prefix), content_type='libsvm')

Next, specify the necessary parameters and hyperparameters to SageMaker's Estimator and perform fitting. The code below uses "ml.m4.xlarge" </ font> as the training instance. The process is completed in about 10 minutes.

#SageMaker session
sess = sagemaker.Session()
 
#Specify the required items in the sagemaker estimator
xgb = sagemaker.estimator.Estimator(containers[boto3.Session().region_name],
                                    role, 
                                    train_instance_count=1, 
                                    train_instance_type='ml.m4.xlarge',
                                    output_path='s3://{}/{}/output'.format(bucket, prefix),
                                    sagemaker_session=sess)
 
#Specifying hyperparameters
xgb.set_hyperparameters(max_depth=5,
                        eta=0.2,
                        gamma=4,
                        min_child_weight=6,
                        subsample=0.8,
                        silent=0,
                        objective='binary:logistic',
                        num_round=100)
 
#Model fitting and output destination specification (S3)
xgb.fit({'train': s3_input_train, 'validation': s3_input_validation})

2019-11-18 16:36:46 Uploading - Uploading generated training model
2019-11-18 16:36:46 Completed - Training job completed
Training seconds: 63
Billable seconds: 63

# ml.t2.Deploy with an instance of medium
xgb_predictor = xgb.deploy(initial_instance_count=1,
                           instance_type='ml.t2.medium')

ResourceLimitExceeded: An error occurred (ResourceLimitExceeded) when calling the CreateEndpoint operation: The account-level service limit 'ml.c4.xlarge for endpoint usage' is 0 Instances, with current utilization of 0 Instances and a request delta of 1 Instances. Please contact AWS support to request an increase for this limit.

#Make settings for data transfer
xgb_predictor.content_type = 'text/csv'
xgb_predictor.serializer = csv_serializer

#Xgb in small batches of every 500 lines_Predict and calculate with predictor
def predict(data, rows=500):
    split_array = np.array_split(data, int(data.shape[0] / float(rows) + 1))
    predictions = ''
    for array in split_array:
        predictions = ','.join([predictions, xgb_predictor.predict(array).decode('utf-8')])
 
    return np.fromstring(predictions[1:], sep=',')
 
#Test created in the previous item_Delete the target item from data and output the forecast
predictions = predict(test_data.drop(['y_no', 'y_yes'], axis=1).as_matrix())
 
#Comparison table of prediction and correct answer data
pd.crosstab(index=test_data['y_yes'], columns=np.round(predictions), rownames=['actuals'], colnames=['predictions'])

#Delete the created endpoint
sagemaker.Session().delete_endpoint(xgb_predictor.endpoint)