Notes on importing data from MySQL or CSV with Python

Minimal notes

environment

For the time being, python2.7.x.

Please see here for how to confirm the existence in the first place.

MySQL

Use mysql-connector-python

It's hard to decide which module to use to communicate with MySQL, but for the time being, use mysql-connector-python.

#coding:utf-8

import mysql.connector 

con = mysql.connector.connect(
		host='localhost',
		db='testdb',
		user='root',
		password='root'
	)

cur = con.cursor(buffered=True)

sql = "select * from members"

cur.execute(sql)

rows = cur.fetchall()

for row in rows:
	print row[1]

cur.close()
con.close()

CSV

csv(utf-8)

You don't have to worry about utf-8 comma-separated files.

#coding:utf-8

import csv

file = "test.csv"

f = open(file,"r")

reader = csv.reader(f)

for row in reader:
	print row[0]

f.close()

csv (sjis or cp932)

It is common for csv created in Excel to be sent and said "Read!". There is no time to deal with it in detail, but for the time being, you can read it by doing the following.

#coding:utf-8

import csv

file = "test2.csv"

f = open(file,"r")

reader = csv.reader(f)

for row in reader:
	print row[0].decode('cp932')

f.close()

Raw file

For some reason, you may not be able to go without processing the raw file.

Split normally

The line break at the end of the line is removed with strip ().

#coding:utf-8

f = open('test.csv','r')

for row in f:
	item = row.strip().split(',')
	print item[0]

f.close()

Split with regular expression

If you want to use regular expressions, use re.split (pattern, string).

#coding:utf-8

import re

f = open('test.csv','r')

for row in f:
	item = re.split(',',row)
	print item[0]

f.close()

Other

The rest is related to Pandas. I would like to add it from time to time.

Recommended Posts

Notes on importing data from MySQL or CSV with Python
Receive textual data from mysql with python
Get data from MySQL on a VPS with Python 3 and SQLAlchemy
[Data science basics] I tried saving from csv to mysql with python
[Python] Notes on data analysis
Notes on using MeCab from Python
Notes on handling large amounts of data with python + pandas
Notes on using rstrip with python.
Notes on accessing dashDB from python
[Note] Get data from PostgreSQL with Python
Notes on doing Japanese OCR with Python
Connecting from python to MySQL on CentOS 6.4
Let's do MySQL data manipulation with Python
Data integration from Python app on Linux to Amazon Redshift with ODBC
Data integration from Python app on Windows to Amazon Redshift with ODBC
Csv output from Google search with [Python]! 【Easy】
[Python] Convert from DICOM to PNG or CSV
Folium: Visualize data on a map with Python
Generate an insert statement from CSV with Python.
Write CSV data to AWS-S3 with AWS-Lambda + Python
Make JSON into CSV with Python from Splunk
Try importing MLB data on Mac and Python
Connect to MySQL with Python on Raspberry Pi
Extract data from a web page with Python
Notes on oct2py calling Octave scripts from Python
(Miscellaneous notes) Data update pattern from CSV data acquisition / processing to Excel by Python
Data analysis with python 2
Touch MySQL from Python 3
Python from or import
Use MySQL from Python
Use MySQL from Python
Csv tinkering with python
Data analysis with Python
Reading CSV data from DSX object storage Python code
How to scrape image data from flickr with python
Notes on HDR and RAW image processing with Python
Manipulate excel files from python with xlrd (personal notes)
Yum command to access MySQL with Python 3 on Linux
Interact with Python on Android from PC via adb
Process csv data with python (count processing using pandas)
[Basics of data science] Collecting data from RSS with python
Get data from database via ODBC with Python (Access)
Remove headings from multiple format CSV files with python
Sample data created with python
OpenJTalk on Windows10 (Speak Japanese with Python from environment construction)
Extract Twitter data with CSV
Get Youtube data with python
Use MySQL from Anaconda (python)
Python data analysis learning notes
[Python] Extract text data from XML data of 10GB or more.
Notes on installing Python on Mac
[Python / Ruby] Understanding with code How to get data from online and write it to CSV
Stylish technique for pasting CSV data into Excel with Python
How to know the number of GPUs from python ~ Notes on using multiprocessing with pytorch ~
Xpath summary when extracting data from websites with Python Scrapy
Write to csv with Python
Get data from analytics API with Google API Client for python
Collecting information from Twitter with Python (MySQL and Python work together)
Notes on deploying pyenv with Homebrew and managing Python versions
With skype, notify with skype from python!
Download csv file with python