Amazon Elasticsearch Service is Elasticsearch. You can use the endpoint as Elasticsearch normally, and choose the access policy from the following.
By the way, I thought about a request from Lambda based on IAM to lightly parse the file placed in S3 for a little convenience and thrust it into Elasticsearch, but at present it seems that I make my own signature request and throw it in.
result. I tried it, but now it seems easier to stream it to the CloudWatch Logs stream and then use the preset functions. If boto3 resource is implemented for ES, it seems that there will be no trouble.
It worked for the time being, so make a note of the code.
--Allow roles applied to Lambda in IAM template
--ʻEs: * , ʻes: ESHttpGet
, etc.
Aside from EventSource, just send a GET to the ES /
for the time being.
The following materials were helpful.
-Complete Version 4 Signing Process Example (Python)-Amazon Web Services
--Bringed a function to create an Authorization header.
--Javascript for LogsToElasticsearch
that can be created with CloudWatch Logs
--Used as a reference for headers and environment variables other than Authorization.
――It's almost like transplanting this.
Mostly the code to make the header. .. ..
function.py
import urllib2, datetime, os, sys
import hashlib, hmac
endpoint = 'search-sandbox01-xxxxxxxxxxxxxxxxxxxx.ap-northeast-1.es.amazonaws.com'
url = "https://" + endpoint
region = 'ap-northeast-1'
service = 'es'
method = 'GET'
access_key = os.environ.get('AWS_ACCESS_KEY_ID')
secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')
if access_key is None or secret_key is None:
print 'No access key is available.'
sys.exit()
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
def lambda_handler(event, context):
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
canonical_uri = '/'
canonical_querystring = ""
canonical_headers = 'host:' + endpoint + '\n' + 'x-amz-date:' + amzdate + '\n'
signed_headers = 'host;x-amz-date'
payload_hash = hashlib.sha256('').hexdigest()
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' + amzdate + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
headers = {
'Content-Type': 'application/json',
'Host': endpoint,
'X-Amz-Security-Token': os.environ.get('AWS_SESSION_TOKEN'),
'X-Amz-Date': amzdate,
'Authorization': authorization_header
}
req = urllib2.Request(url, None, headers)
try:
print urllib2.urlopen(req).read()
except urllib2.URLError, e:
print e.read()
Is it possible to make this header quickly by using the botocore module? While thinking.
Addendum: Click here for an example of "Make it quickly using the botocore module". [Small story] Use only the signing process of botocore's AWS API request | Developers.IO
You Know, for Search
When I try to cURL to ES from my hand, it is blocked properly.
$ curl search-sandbox01-xxxxxxxxxx.ap-northeast-1.es.amazonaws.com
{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet on resource: arn:aws:es:ap-northeast-1:xxxxxxxxx:domain/sandbox01/"}
When I opened Lambda and Save and Test
the above code, I got a reply from ES safely.
START RequestId: b6ce7399-85eb-11e5-b2c3-99a74c2c1765 Version: $LATEST
{
"status" : 200,
"name" : "Chief Examiner",
"cluster_name" : "xxxxxxxxxxx:sandbox01",
"version" : {
"number" : "1.5.2",
"build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c",
"build_timestamp" : "2015-04-27T09:21:06Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
END RequestId: b6ce7399-85eb-11e5-b2c3-99a74c2c1765
REPORT RequestId: b6ce7399-85eb-11e5-b2c3-99a74c2c1765 Duration: 64.87 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 16 MB
Recommended Posts