Stock price forecast using deep learning [Data acquisition]

Introduction

I decided to write this article because I wanted to make some ingenuity and try stock price forecasting using deep learning. The purpose of this article is to programmatically download the data used for stock price forecasting from "yahoo finance".

By the way, the model for forecasting the stock price is still undecided, and I would like to think about it from now on. If you have a good idea or a recommended method, I would appreciate it if you could tell me.

Then, I would like to do it.

Target data

The source of this time is "yahoo finance". As for the target, you can download the csv file of the listed stock list from Kabusapo, so I would like to narrow down the data to the first section of the Tokyo Stock Exchange. I think.

The download period is from 2010 to the present.

code

stock_data.py

import requests
import io
import re
import pandas as pd

cols = ['Id', 'Date', 'Open', 'High', 'Low', 'Close', 'Adj_Close', 'Volume']
stock_data = pd.DataFrame(columns=cols)

code_list = pd.read_csv('stock_code_list.csv') #File obtained from Kabusapo
code_list = code_list[code_list['Market name'] == 'First Section of the Tokyo Stock Exchange']['Stock code']

for code in code_list:
    url = 'https://query1.finance.yahoo.com/v7/finance/download/{}.T?period1=1262304000&period2=1589241600&interval=1d&events=history'.format(code)
    res = requests.get(url)
    
    if res.status_code != 200:
        print('NotFound:', code)
        
    else:
        stock = pd.read_csv(io.StringIO(res.text))
        stock['Id'] = code
        stock_data = pd.concat([stock_data, stock], ignore_index=True, sort=False, keys=cols)
        print('OK:', code)

You have now downloaded it. The data size was about 5 million rows.

in conclusion

Now that we have the data, let's consider a predictive approach. Personally, it's not just about whether the stock price will go up or down the next day, but more for individual investors. I would like to have a program that detects uptrends and downtrends. I also think that the accuracy will improve if qualitative data is added.

If you have a good idea ...

I would like to write a little bit like this.