Query from python to Amazon Athena (using named profile)

Introduction

I sometimes connect to Athena with AWS CLI on windows command line, Anyway I wanted to get the query result and draw the graph, so I thought I would do it with python. There was a dead end in the named profile part, so organize it

Manual description method (named profile unused)

How to do with windows command line

aws athena start-query-execution
  --query-string "select * from table_name"
  --result-configuration "s3://path/to/query/bucket/"

AWS documentation: https://docs.aws.amazon.com/cli/latest/reference/athena/start-query-execution.html

Method in python


athena = boto3.client('athena')

#Query execution
exec_run = athena.start_query_execution(
    QueryString="select * from table_name",
    QueryExecutionContext={'Database': 'database_name'},
    ResultConfiguration={'OutputLocation': 's3://path/to/query/bucket/'})

AWS documentation: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena.html

What I want to do this time

I want to execute a query using the information of the named profile.

How to set up a named profile I wasted my time without noticing the difference between "windows command line" and "python".

How to set up a named profile windows command line: set as a parameter of the aws command python: Set profile information using boto3.session class

Windows command line method (using named profile)

aws athena start-query-execution
  --query-string "select * from table_name"
  --result-configuration "s3://path/to/query/bucket/"
 --profile "NRP"

AWS documentation: https://docs.aws.amazon.com/cli/latest/reference/athena/start-query-execution.html https://docs.aws.amazon.com/cli/latest/reference/athena/start-query-execution.html

Method in python (using named profile)


session = boto3.Session(profile_name='NRP')
athena = session.client('athena')

#Processing execution
exec_run = athena.start_query_execution(
    QueryString="select * from table_name",
    QueryExecutionContext={'Database': 'database_name'},
    ResultConfiguration={'OutputLocation': 's3://path/to/query/bucket/'})

AWS documentation: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena.html https://boto3.amazonaws.com/v1/documentation/api/latest/guide/session.html Articles about S3 connectivity: Use something other than the default profile with boto3

Finally

It was hard to get to boto3.session. It wasn't good to look it up on an error statement basis. It was written firmly in the manual Once I got there, I realized again that I would understand various things.

However, this time there are many issues such as not having all the acquisition results.

that's all.

Reference information

Described in each place

Recommended Posts

Query from python to Amazon Athena (using named profile)
Query Athena from Lambda Python
Try using Amazon DynamoDB from Python
From Python to using MeCab (and CaboCha)
I want to email from Gmail using Python.
Changes from Python 3.0 to Python 3.5
Changes from Python 2 to Python 3.0
Push notifications from Python to Android using Google's API
MessagePack-Call Python (or Python to Ruby) methods from Ruby using RPC
Copy S3 files from Python to GCS using GSUtil
Flatten using Python yield from
Post from Python to Slack
Cheating from PHP to Python
Post to Twitter using Python
Start to Selenium using python
Switch from python2.7 to python3.6 (centos7)
Connect to sqlite from python
Convert from Pandas DataFrame to System.Data.DataTable using Python for .NET
How to install python using anaconda
Using Rstan from Python with PypeR
Create folders from '01' to '12' with python
Notes on using MeCab from Python
How to get followers and followers from python using the Mastodon API
[Lambda] [Python] Post to Twitter from Lambda!
Connect to utf8mb4 database from python
Using Cloud Storage from Python3 (Introduction)
Python (from first time to execution)
Post images from Python to Tumblr
Terminal association from the server side to Amazon SNS (python + boto3)
How to access wikipedia from python
Python to switch from another language
Run Ansible from Python using API
Precautions when using phantomjs from python
Access spreadsheets using OAuth 2.0 from Python
Did not change from Python 2 to 3
Update Python on Mac from 2 to 3
From preparation for morphological analysis with python using polyglot to part-of-speech tagging
How to deal with OAuth2 error when using Google APIs from Python
Create a tool to automatically furigana with html using Mecab from Python3
Data integration from Python app on Linux to Amazon Redshift with ODBC
Try using the Python web framework Django (1)-From installation to server startup
How to get a value from a parameter store in lambda (using python)
Data integration from Python app on Windows to Amazon Redshift with ODBC
Copy data from Amazon S3 to Google Cloud Storage with Python (boto)
[Python] Fluid simulation: From linear to non-linear
Introduction to Discrete Event Simulation Using Python # 1
How to update Google Sheets from Python
Send a message from Python to Slack
I want to use jar from python
Log in to Slack using requests in Python
Connecting from python to MySQL on CentOS 6.4
I tried using UnityCloudBuild API from Python
Try to operate Excel using Python (Xlwings)
How to access RDS from Lambda (python)
Python> Output numbers from 1 to 100, 501 to 600> For csv
Introduction to Discrete Event Simulation Using Python # 2
Convert from Markdown to HTML in Python
API explanation to touch mastodon from python
Connect to coincheck's Websocket API from Python
I tried to execute Python code from .Net using Pythonnet (Hallo World edition)
Send a message from Slack to a Python server