"Getting stock price time series data from k-db.com with Python" Program environment creation memo

In the Cloud9 development environment, install jupyter and run the stock price acquisition program from k-db.com in that environment.

Postscript 2018.2.22 It was said that k-db will end its service at the end of 2017.

I am studying deep learning and am thinking of a program to make stock price forecasts using stock data. However, there is almost no way to get the data of all TSE stocks in 5 minutes even if you search variously. And finally I found k-db.com. This site is for 4 days, but information on 5 minute bars of individual stocks is posted. However, it takes a lot of time to download and edit this one by one. And I found a program that automatically acquires stock price data in 5 minutes from this site. You can find it at the URL below.

Program to get stock price from k-db.com I used this program for "Getting time series data from k-db.com with Python". http://qiita.com/sawadybomb/items/03c3814268d3e2904e6c cloud9, python I'm new to python and it doesn't work very well.

environment

cloud9 (paid version) jupyter python 3.5.3

procedure

First, create an environment for anaconda and jupyter.

See here http://qiita.com/KenjiKitagawa/items/e0225d9be10ea3dbdafc

Download and deploy stock price acquisition program from k-db.com

https://github.com/sawadyrr5/pykdb Download the program from here to your PC and deploy it under the jupyter of cloud9.

Launch a file called setup.py

Execute setup of the downloaded file. Execute python setup.py install from the command line

When I started "tutorial_stocks.ipynb" under the example folder from jupyter, it worked properly.

However, depending on the brand, there was a part where the data was missing, and even when I looked at the acquisition source k-db.com, the data was still not included. The coping method is written below.

Program execution result NAN has no data stock1.jpg

Example of countermeasures when part of the data becomes NAN

Since the original data of k-db.com is NAN, it is impossible to get the data. Originally, this data needs a proper numerical value to be used for deep learning of artificial intelligence, but since the number of NANs is small, I decided to input the numerical value by predicting the numerical value from the previous and next numerical values (linear numerical value).

Put df = df.interpolate (hoge) under df = obj.hoge in the sample program under the example directory. Example

df = obj.price(date_from=start, date_to=end, symbol=i_old, freq=stock_freq)   
df = df.interpolate(method='index') #Interpolate NAN data with front and back linear values

After complementation stock2.jpg

Recommended Posts

"Getting stock price time series data from k-db.com with Python" Program environment creation memo
Get time series data from k-db.com in Python
[Time series with plotly] Dynamic visualization with plotly [python, stock price]
Download Japanese stock price data with python
Get stock price data with Quandl API [Python]
Rewrite the field creation node of SPSS Modeler with Python. Feature extraction from time series sensor data
Data is missing when getting stock price data with Pandas-datareader
[Python] Plot time series data
Get stock price with Python
A memo that reads data from dashDB with Python & Spark
A Python program that aggregates time usage from icalendar data
Python: Time Series Analysis: Preprocessing Time Series Data
[Stock price analysis] Learning pandas with fictitious data (001: environment preparation-file reading)
Get US stock price from Python with Web API with Raspberry Pi
Plot CSV of time series data with unixtime value in Python (matplotlib)
Forecasting time series data with Simplex Projection
Receive textual data from mysql with python
[Note] Get data from PostgreSQL with Python
Getting started with Dynamo from Python boto
Format and display time series data with different scales and units with Python or Matplotlib
Collecting information from Twitter with Python (Environment construction)
Acquisition of time series data (daily) of stock prices
[Python] Conversion memo between time data and numerical data
A memo when creating a python environment with miniconda
View details of time series data with Remotte
Flow memo when getting json data with urllib
Automatic acquisition of stock price data with docker-compose
Let's do web scraping with Python (stock price)
From Python environment construction to virtual environment construction with anaconda
Python & Machine Learning Study Memo ⑦: Stock Price Forecast
Extract data from a web page with Python
Reading, summarizing, visualizing, and exporting time series data to an Excel file with Python
Implementation of clustering k-shape method for time series data [Unsupervised learning with python Chapter 13]
Predict from various data in Python using Facebook Prophet, a time series prediction tool
Features that can be extracted from time series data
Receive dictionary data from a Python program in AppleScript
Python 3.4 Create Windows7-64bit environment (for financial time series analysis)
Output the time from the time the program was started in python
Data analysis environment construction with Python (IPython notebook + Pandas)
[Basics of data science] Collecting data from RSS with python
Get data from database via ODBC with Python (Access)
"Measurement Time Series Analysis of Economic and Finance Data" Solving Chapter End Problems with Python