Stock price forecast using deep learning [Data acquisition]

Introduction

I decided to write this article because I wanted to make some ingenuity and try stock price forecasting using deep learning. The purpose of this article is to programmatically download the data used for stock price forecasting from "yahoo finance".

By the way, the model for forecasting the stock price is still undecided, and I would like to think about it from now on. If you have a good idea or a recommended method, I would appreciate it if you could tell me.

Then, I would like to do it.

Target data

The source of this time is "yahoo finance". As for the target, you can download the csv file of the listed stock list from Kabusapo, so I would like to narrow down the data to the first section of the Tokyo Stock Exchange. I think.

The download period is from 2010 to the present.

code

stock_data.py


import requests
import io
import re
import pandas as pd

cols = ['Id', 'Date', 'Open', 'High', 'Low', 'Close', 'Adj_Close', 'Volume']
stock_data = pd.DataFrame(columns=cols)

code_list = pd.read_csv('stock_code_list.csv') #File obtained from Kabusapo
code_list = code_list[code_list['Market name'] == 'First Section of the Tokyo Stock Exchange']['Stock code']

for code in code_list:
    url = 'https://query1.finance.yahoo.com/v7/finance/download/{}.T?period1=1262304000&period2=1589241600&interval=1d&events=history'.format(code)
    res = requests.get(url)
    
    if res.status_code != 200:
        print('NotFound:', code)
        
    else:
        stock = pd.read_csv(io.StringIO(res.text))
        stock['Id'] = code
        stock_data = pd.concat([stock_data, stock], ignore_index=True, sort=False, keys=cols)
        print('OK:', code)

You have now downloaded it. The data size was about 5 million rows.

in conclusion

Now that we have the data, let's consider a predictive approach. Personally, it's not just about whether the stock price will go up or down the next day, but more for individual investors. I would like to have a program that detects uptrends and downtrends. I also think that the accuracy will improve if qualitative data is added.

If you have a good idea ...

I would like to write a little bit like this.

Recommended Posts

Stock price forecast using deep learning [Data acquisition]
Stock price forecast using deep learning (TensorFlow)
Stock Price Forecast Using Deep Learning (TensorFlow) -Part 2-
Stock price forecast using machine learning (scikit-learn)
Stock price forecast using machine learning (regression)
Stock price data acquisition tips
Stock Price Forecast 2 Chapter 2
Stock price forecast by machine learning Numerai Signals
Automatic acquisition of stock price data with docker-compose
Python & Machine Learning Study Memo ⑦: Stock Price Forecast
Stock Price Forecast 1 Chapter 1
Stock price forecast with tensorflow
Python: Stock Price Forecast Part 2
Stock Price Forecasting Using LSTM_1
Python: Stock Price Forecast Part 1
[Stock price analysis] Learning pandas with fictitious data (002: Log output)
Stock price forecast by machine learning Let's get started Numerai
Data acquisition using python googlemap api
[Python] My stock price forecast [HFT]
Data acquisition memo using Backlog API
Machine learning in Delemas (data acquisition)
Preprocessing in machine learning 2 Data acquisition
I tried deep learning using Theano
Predicting stock price changes using metal labeling and two-step machine learning
I tried using GLM (generalized linear model) for stock price data
Stock price forecast by machine learning is so true Numerai Signals
[Stock price analysis] Learning pandas with fictitious data (001: environment preparation-file reading)
Stock price acquisition code by scraping (Selenium)
Sentiment analysis of corporate word-of-mouth data of career change meetings using deep learning
Image recognition model using deep learning in 2016
Try Bitcoin Price Forecasting with Deep Learning
Download Japanese stock price data with python
Python: Preprocessing in machine learning: Data acquisition
Deep Learning
About data expansion processing for deep learning
Data supply tricks using deques in machine learning
An amateur tried Deep Learning using Caffe (Introduction)
An amateur tried Deep Learning using Caffe (Practice)
Acquisition of time series data (daily) of stock prices
Get stock price data with Quandl API [Python]
[Python3] Let's analyze data using machine learning! (Regression)
An amateur tried Deep Learning using Caffe (Overview)
Deep Learning Memorandum
Start Deep learning
Deep learning × Python
Thinking about party attack-like growth tactics using deep learning
Stock Price Forecast with TensorFlow (Multilayer Perceptron: MLP) ~ Stock Forecast Part 2 ~
Stock investment by deep reinforcement learning (policy gradient method) (1)
Data is missing when getting stock price data with Pandas-datareader
Collection and automation of erotic images using deep learning