[Technical book] Introduction to data analysis using Python -1 Chapter Introduction-

The data files and related materials used in this document are the GitHub repository below. https://github.com/wesm/pydata-book A memorandum summarizing the main points

table of contents

  1. Things to remember
  2. Required Python library
  3. Technical terms, etc.
  4. Impressions, etc.

Things to remember

--A book written for the purpose of becoming a good data analyst, and learning the knowledge to do the programming necessary for data analysis with Python. --Python is often used to build websites using web frameworks such as Djnago. One of the most important languages in data science, machine learning and general software development. --With improved support for libraries such as pandas and scikit-learn, Python has become a powerful choice in data analysis. --Python is an interpreted language, so its execution speed is slow. For applications that require low latency or applications that require effective use of resources (for example, high-frequency trading systems), it is more effective to maximize performance in a low-level language such as C ++. How to spend your time. --Python is a difficult language for developing parallel and multithreaded applications (by a mechanism called GIL).

Required Python library

Numpy Provides data structures and algorithms on the basis of numerical calculations in Python Typical examples are ndarray, a high-speed and efficient multidimensional array object, and mathematical operations.

pandas Introduced in 2010. The main object is DataFrame = tabular and columnar data structure It has both the high-performance array calculation function of numpy and the ability to flexibly manipulate data in spreadsheets and relational databases (like SQL). Pandas is one of the main things in this book. Data can be manipulated, prepared, and cleaned.

Matplotlab The most common Python library used for visualization of 2D formats such as graphs. A safe choice as a visualization tool to use by default

** Ipython and Jupyter ** IPython is recommended for use in situations where you edit, run, and try and error In 2014, the IPython web notebook was replaced by the Jupyter Notebook, which now supports more than 40 programming languages. Ipython is used as a kernel for using Python with Jupyter. Jupyter Notebook is a "notebook" for writing code on a web basis. Since the content can be edited with Markdown and HTML, you can create rich documents with a mixture of code and sentences.

SciPy A collection of packages dealing with common problems in the field of scientific computing. By using Numpy and SciPy together, they can be used as a rational and mature computational base and can be applied to many traditional scientific calculations.

scikit-learn At the top of general "machine learning tools". Submodules such as classification, regression, and clustering, cross-validation, preprocessing, etc.

statsmodels A classic statistical analysis package compared to scikit-learn.

Technical terms, etc.

--Python 2.x is called "legacy Python", Python 3.x is simply called "Python" -** Manging, Langling … The whole process of manipulating unstructured or messy data into a clean, structured format. - Pseudo code … Explains in a format similar to source code to explain algorithms and processes. - Syntax sugar **… A programming language grammar that does not add new features but makes input convenient.

Impressions, etc.

The first qiita post is a memorandum of technical books Let's edit according to the review Disseminate technology with qiita and post what you want people to see Plus, I want to make a machine learning / AI web application with python Write in your own words as much as possible. Don't seek perfection too much. It is a very good motivation to have people see it, and studying progresses efficiently. I can't write everything, I'll summarize what I want to remember, what I tried to understand, and what I was interested in

Recommended Posts

[Technical book] Introduction to data analysis using Python -1 Chapter Introduction-
[Introduction to Python3, Day 17] Chapter 8 Data Destinations (8.1-8.2.5)
[Introduction to Python3, Day 17] Chapter 8 Data Destinations (8.3-8.3.6.1)
[Introduction to Python3 Day 19] Chapter 8 Data Destinations (8.4-8.5)
Data analysis using Python 0
Python for Data Analysis Chapter 4
Data analysis using python pandas
Python for Data Analysis Chapter 3
Reading Note: An Introduction to Data Analysis with Python
[Introduction to Python3 Day 13] Chapter 7 Strings (7.1-7.1.1.1)
Introduction to image analysis opencv python
[Introduction to Python3 Day 21] Chapter 10 System (10.1 to 10.5)
20200329_Introduction to Data Analysis with Python Second Edition Personal Summary
[Introduction] Artificial satellite data analysis using Python (Google Colab environment)
"Introduction to effect verification Chapter 3 Analysis using propensity score" + α is tried in Python
Introduction to Discrete Event Simulation Using Python # 1
Data analysis python
Introduction to Data Analysis with Python P32-P43 [ch02 3.US Baby Names 1880-2010]
Introduction to Data Analysis with Python P17-P26 [ch02 1.usa.gov data from bit.ly]
Introduction to Discrete Event Simulation Using Python # 2
Introduction to Effectiveness Verification Chapter 1 in Python
[Introduction to Data Scientists] Basics of Python ♬
I tried fMRI data analysis with python (Introduction to brain information decoding)
Introduction to effectiveness verification Chapter 3 written in Python
[Introduction to Python3 Day 22] Chapter 11 Concurrency and Networking (11.1 to 11.3)
[Introduction to Python] How to handle JSON format data
[Introduction to Python3 Day 11] Chapter 6 Objects and Classes (6.1-6.2)
[Introduction to Python3 Day 23] Chapter 12 Become a Paisonista (12.1 to 12.6)
[Python] Flow from web scraping to data analysis
[Introduction to Python3 Day 20] Chapter 9 Unraveling the Web (9.1-9.4)
Introduction to Effectiveness Verification Chapter 2 Written in Python
Data analysis with python 2
Data analysis using xarray
Data analysis overview python
Introduction to Python language
Introduction to OpenCV (python)-(2)
Python data analysis template
Data analysis with Python
An introduction to data analysis using Python-To increase the number of video views-
[Introduction to Python3 Day 8] Chapter 4 Py Skin: Code Structure (4.1-4.13)
[For beginners] How to study Python3 data analysis exam
[Chapter 5] Introduction to Python with 100 knocks of language processing
Write data to KINTONE using the Python requests module
[Chapter 2] Introduction to Python with 100 knocks of language processing
[Introduction to Python] How to stop the loop using break?
[Introduction to Python] How to write repetitive statements using for statements
[Chapter 4] Introduction to Python with 100 knocks of language processing
My python data analysis container
Introduction to Python Django (2) Win
Post to Twitter using Python
Start to Selenium using python
[Python] Notes on data analysis
Introduction to serial communication [Python]
Python data analysis learning notes
[Introduction to Python] <list> [edit: 2020/02/22]
Introduction to Python (Python version APG4b)
An introduction to Python Programming
Introduction to discord.py (3) Using voice
Introduction to Python For, While
[Introduction to Python] How to write conditional branches using if statements
Introduction to Bayesian Modeling Using pymc3 Bayesian-Modeling-in-Python Japanese Translation (Chapter 0-2)