I tried to get started with Bitcoin Systre on the weekend

background

――When I was surfing the internet, I found a very helpful site about Systre. -Understand even in the humanities! How to start Bitcoin BOT automatic trading ――It was explained in a very easy-to-understand manner ――I feel like I can do it even if I have no experience with Systre. ――I was interested, so I decided to write a code on the weekend to check the feel. -Github This is the code I wrote this time. (Notebook format)

--By the way, before, Challenge to predict virtual currency by LSTM, it didn't work very well. ――This time, I wrote a simple algorithm using a moving average line without jumping to machine learning suddenly.

What you did

――We will try the effect by acquiring past data without actually trading. ――I wrote only the parts that seemed to be the basis of acquisition of past data and transaction simulation.

1. Get Bitcoin data from CryptWatch API

--Defined the following classes --Since it is difficult to see the JSON that can be obtained via API, we added a process to convert it to DataFrame format.

import json
import requests
import pandas as pd
import datetime
from pprint import pprint
import time

class PriceGetterCryptwatch():
#Omission
    def get_price(self, periods, after=None, before=None, exchange="bitflyer", currency="btcjpy"):
        """
        input
        periods:Cycle 86400 for daily bars
        after:Get chart after after with unixTime
        before:Get chart before before with unixTime
        exchange:Cryptocurrency exchange
        currency:Currency pair

        output
API results:JSON format
        """
        url = "https://api.cryptowat.ch/markets/"+exchange+"/"+currency+"/ohlc?periods="+str(periods)

        if after:
            after=int(after)
            url += "&after=" + str(after)
        elif before:
            before=int(before)
            url += "&before=" + str(before)

        print(url)
        responce = requests.request("GET", url)
        return responce.json()

    def apidata_to_dataframe(self, data, periods):
        """
        [ CloseTime , OpenPrice , HighPrice , LowPrice , ClosePrice , Volume]
        [Date and time,Open price,High price,Low price,closing price,Volume]
        """
        #Since the last column is unknown, delete it for the time being
        df = pd.DataFrame(data["result"][str(periods)],columns=[ "CloseTime" , "OpenPrice" , "HighPrice" , "LowPrice" , "ClosePrice" , "Volume", "None"])
        df = df.drop("None", axis=1)
        #Convert Unix time to real time
        df["CloseTime"] = df["CloseTime"].map(self.unixtime_to_datetime)
        return df

--If you use it, you will get the following table スクリーンショット 2019-11-11 13.11.48.png

Saving past data

--Cryptwatch seems to be able to collect up to 6000 data from the latest. ――The reference also described how to save and accumulate past data. --Save in JSON format --I wanted to save it in csv with the appearance of DataFrame, so I rewrote it with pandas

function

def save_data(periods):
    getter = PriceGetterCryptwatch()
    date = getter.datetime_to_obj(2017,1,1)
    date = getter.datetime_to_unixtime(date)

    # designate start date
    data = getter.get_price(periods, after=date)
    df = getter.apidata_to_dataframe(data, periods)

    start_time = (df["CloseTime"].iloc[0])
    last_time = (df["CloseTime"].iloc[-1]) # unixtime
    filename = str(start_time) + " - " + str(last_time)  + "period " + str(periods)
    filename += ".csv"
    filename = filename.replace(":", "-")

    df.to_csv(filename, index=False, float_format="%.6f")

def update_data(old_filename, periods):
    df_old = pd.read_csv(old_filename)
    old_size = df_old.shape[0]

    # get new data
    getter = PriceGetterCryptwatch()
    date = getter.datetime_to_obj(2018,1,1)
    date = getter.datetime_to_unixtime(date)

    # designate start date
    data = getter.get_price(periods,after=date)
    df = getter.apidata_to_dataframe(data, periods)
    
    df_temp = pd.concat([df_old, df], axis=0)
    df_temp["CloseTime"] = pd.to_datetime(df_temp["CloseTime"]) 
    df_save = df_temp.drop_duplicates(subset="CloseTime")
    df_save = df_save.sort_values("CloseTime")
    df_save = df_save.reset_index(drop=True)
    new_size = df_save.shape[0] - old_size

    if new_size > 0:
        print("NewData : {}".format(new_size))

        start_time = df_save["CloseTime"].iloc[0]
        last_time = df_save["CloseTime"].iloc[-1] # unixtime
        filename = str(start_time) + " - " + str(last_time)  + "period " + str(periods)
        filename += ".csv"
        filename = filename.replace(":", "-")

        dir_name = os.path.dirname(old_filename)
        filename = os.path.join(dir_name, filename)
        df_save.to_csv(filename, index=False, float_format="%.6f")
    else:
        print("NewData None")

--I got a little stuck in deleting and combining duplicate data in update_data --Combine old and new data with concat --After that, if you do not convert the date to Datetime again, drop_duplicates may not be judged as duplicate. --Even when saving to csv, if the number of digits of the float is not the same, it may not be judged as duplicate.

--Execute --When acquired in 1 minute

    PERIOD_1MIN = 60
    FILENAME_OLD_1MIN = "./Data/2019-11-06 12-45-00 - 2019-11-11 09-03-00period 60.csv"
    update_data(FILENAME_OLD_1MIN, PERIOD_1MIN)

--Result --It seems that 258 new data have been added. Update is quick because it is 1 minute

https://api.cryptowat.ch/markets/bitflyer/btcjpy/ohlc?periods=60&after=1514732400
NewData : 258

--Acquired data .csv --More than 6000 data have been accumulated ――We will utilize it as learning data in the future.

CloseTime,OpenPrice,HighPrice,LowPrice,ClosePrice,Volume
2019-11-06 12:45:00,1013343,1013343,1013343,1013343,0.090000
2019-11-06 12:46:00,1013343,1013622,1012862,1013091,2.730759
2019-11-06 12:47:00,1013375,1013839,1013345,1013345,1.170000
...

2019-11-11 13:26:00,981670,982509,981670,982509,0.778839
2019-11-11 13:27:00,982489,982489,982003,982003,0.205953
2019-11-11 13:28:00,982052,982052,982052,982052,0.010000

2. Transaction simulation

--Try to make a transaction based on the data obtained by hitting the API

BitCoin price plot (4 hour bar and moving average (12h, 20h))

--Obtained BTC-JPY data from January 1, 2018 to the present --The vertical axis is the price, and the horizontal axis is the time. --The 12-hour moving average and the 20-hour moving average are also plotted. ――When you look at it again, the crash in 2018 is terrifying.

スクリーンショット 2019-11-11 13.34.02.png

――I will expand it --If orange (short-term average line) and green (long-term average line) cross, it will be regarded as a trend change and you will buy and sell.

スクリーンショット 2019-11-11 13.39.36.png

Implementation of trading algorithm

--Created a basic trading parts class and inherited it to create a trade algo ――The moment the short-term line crosses the long-term line and exceeds BUY --The moment the short-term line crosses the long-term line and falls below SELL

--First, set your own JPY and BitCoin and repeat buying and selling based on Argo. ――It is a gambler specification that puts all your JPY and BitCoin assets into each transaction.

class TradeBase():
#Omission

    def __init__(self, jpy_asset, bitcoin_asset, period):
        self.jpy_asset = jpy_asset
        self.bitcoin_asset = bitcoin_asset
        self.period = period
        self.df = None

    def buy_coin(self, bitcoin_price, jpy_asset, bitcoin_asset, jpy_buy_amount=None):
        """
For backtesting
        """
        if jpy_buy_amount == None:
            jpy_buy_amount = jpy_asset

        bitcoin_buy_amount = jpy_buy_amount / bitcoin_price
        
        new_jpy_asset =  jpy_asset-jpy_buy_amount
        new_bitcoin_asset =  bitcoin_asset+bitcoin_buy_amount
        
        return new_jpy_asset, new_bitcoin_asset
        
    def sell_coin(self, bitcoin_price, jpy_asset, bitcoin_asset, bitcoin_sell_amount=None):
        """
For backtesting
        """
        if bitcoin_sell_amount== None:
            bitcoin_sell_amount = bitcoin_asset

        jpy_sell_amount = bitcoin_price * bitcoin_sell_amount
        
        new_jpy_asset =  jpy_asset + jpy_sell_amount
        new_bitcoin_asset =  bitcoin_asset-bitcoin_sell_amount
        
        return new_jpy_asset, new_bitcoin_asset


class TradeWithMovingAverage(TradeBase):
    """
When buying and selling only at the intersection of two types of moving averages
    """
    def __init__(self, jpy_asset, bitcoin_asset, period, period_ma_type1, period_ma_type2):
        super().__init__(jpy_asset, bitcoin_asset, period)
        self.period_ma_type1 = period_ma_type1
        self.period_ma_type2 = period_ma_type2
        self.rolling_win_type1 = int(period_ma_type1 / period)
        self.rolling_win_type2 = int(period_ma_type2 / period)
        # for trade func
        self.prev_mean_type1 = None
        self.prev_mean_type2 = None
        self.jpy_asset_result = []
        self.data = None
    
    def judge_sell_or_buy(self, new_mean_fast, prev_mean_fast, new_mean_slow, prev_mean_slow):
        """
Judgment of buying and selling by looking at the moving average line
        """
        if prev_mean_fast > prev_mean_slow:
            if new_mean_fast < new_mean_slow:
                return "SELL"
            else:
                return "NOTHING"

        elif prev_mean_fast < prev_mean_slow:
            if new_mean_fast > new_mean_slow:
                return "BUY"
            else:
                return "NOTHING"
        else:
            return "NOTHING"
        
    def trade(self, init_var=True):

        if self.data is None:
            print(" Data is None")
            return
        
        if init_var:
            self.prev_mean_type1 = None
            self.prev_mean_type2 = None
            self.jpy_asset_result = []

        mean_type1 = None
        mean_type2 = None
        
        last_sell_jpyasset = None
        for i, value in enumerate(self.data):
            # Average
            if i > self.rolling_win_type1:
                mean_type1 = self.data.iloc[int(i-self.rolling_win_type1):i].mean()

            if i > self.rolling_win_type2:
                mean_type2 = self.data.iloc[int(i-self.rolling_win_type2):i].mean()
            
            if mean_type1 is None or mean_type2 is None:
                self.jpy_asset_result.append(self.jpy_asset)
                continue
            
            # Trade
            bitcoin_price = value
            if self.prev_mean_type1 and self.prev_mean_type2:
                judge = self.judge_sell_or_buy(mean_type1, self.prev_mean_type1,
                    mean_type2, self.prev_mean_type2)
                
                if judge == "SELL":
                    if self.bitcoin_asset > 0:
                        self.jpy_asset, self.bitcoin_asset = self.sell_coin(bitcoin_price, self.jpy_asset, self.bitcoin_asset)
                        last_sell_jpyasset = self.jpy_asset

                elif judge == "BUY":
                    if self.jpy_asset > 0:
                        self.jpy_asset, self.bitcoin_asset = self.buy_coin(bitcoin_price, self.jpy_asset, self.bitcoin_asset, jpy_buy_amount=self.jpy_asset)

                else:
                    pass

            self.prev_mean_type1 = mean_type1
            self.prev_mean_type2 = mean_type2
            self.jpy_asset_result.append(self.jpy_asset)
            
        self.last_sell_jpyasset = last_sell_jpyasset
        print("Result: Jpy{:.1f} bitcoin{:.4f} (LAST JPY: {})".format(self.jpy_asset, self.bitcoin_asset, self.last_sell_jpyasset))
        return self.jpy_asset, self.bitcoin_asset

Execution result

--Actually do this --Start trading from 10,000 Japanese Yen and 0 BTC ――Based on 4-hour data

jpy = 10000
bitcoin = 0
period = 60*60*4    #4 hours

trader = TradeWithMovingAverage(jpy_asset=jpy, bitcoin_asset=bitcoin, period=period,
                                period_ma_type1=period*3, period_ma_type2=period*5)
#Set of data(Manual)
trader.df = df
trader.data = df["ClosePrice"]
trader.trade()
trader.show_trade_result()

Asset transition (Japanese yen)

――It's more than tripled from 10,000 yen at the starting point, and it seems to be working unexpectedly. ――Assets have not decreased much even after the crash in the first half of 2018 --Argo has successfully contributed to the mid-2019 surge ――However, if there is no time for a surge, there seems to be no sign of increasing assets so much.

スクリーンショット 2019-11-11 13.52.02.png

--I tried using it for the latest (2019/9 ~ 2019/11) data --Starting on 2019/9 with 10000 on hand --Not very effective. It seems that we have to wait for the next surge ――If you include the fee, you will lose it

スクリーンショット 2019-11-11 14.04.50.png

3. Impressions

――It was clear to me that mechanical trading seems to be more stable without bias than humans do. ――I'm glad I got a sense of building a trading algorithm. ――I think we will make improvements that incorporate other indicators. ――It seems that there is still a lot of anxiety and it will take time to use it in practice. --Deep Learning, I would like to incorporate reinforcement learning into the trading judgment algorithm. ――Thank you for reading.

Recommended Posts

I tried to get started with Bitcoin Systre on the weekend
I tried to get started with Hy
I tried to get started with blender python script_Part 01
I tried to get started with blender python script_Part 02
I tried to get started with Hy ・ Define a class
[Python] A memo that I tried to get started with asyncio
The easiest way to get started with Django
I tried to save the data with discord
I tried to get CloudWatch data with Python
I tried playing with the calculator on tkinter
I tried to get the authentication code of Qiita API with Python.
I tried with the top 100 PyPI packages> I tried to graph the packages installed on Python
I tried to get the movie information of TMDb API with Python
I tried to implement Minesweeper on terminal with python
I tried to touch the CSV file with Python
I tried to solve the soma cube with python
I tried to solve the problem with Python Vol.1
What I did to get started with Linux commands
I tried to notify the honeypot report on LINE
I want to get started with the Linux kernel, what is the list head structure?
I tried to find the entropy of the image with python
I tried to simulate how the infection spreads with Python
I tried to analyze the whole novel "Weathering with You" ☔️
I tried to get the location information of Odakyu Bus
Memo to get the value on the html-javascript side with jupyter
I tried to find the average of the sequence with TensorFlow
I tried to notify the train delay information with LINE Notify
Minimum knowledge to get started with the Python logging module
I tried changing the python script from 2.7.11 to 3.6.0 on windows10
I tried to launch ipython cluster to the minimum on AWS
I tried to divide the file into folders with Python
I tried to get various information from the codeforces API
Link to get started with python
Get started with MicroPython (on macOS)
How to get started with Scrapy
How to get started with Python
How to get started with Django
I tried to move the ball
I tried to estimate the interval.
I tried to get the number of days of the month holidays (Saturdays, Sundays, and holidays) with python
[Shell startup] I tried to display the shell on the TV with a cheap Linux board G-cluster
I tried to describe the traffic in real time with WebSocket
I tried to solve the ant book beginner's edition with python
Get started with the Python framework Django on Mac OS X
I tried to get the index of the list using the enumerate function
I tried to automate the watering of the planter with Raspberry Pi
A memorandum when I tried to get it automatically with selenium
I tried cross-validation based on the grid search results with scikit-learn
I wrote a script to get you started with AtCoder fast!
I tried to process the image in "sketch style" with OpenCV
I tried to digitize the stamp stamped on paper using OpenCV
I tried to register a station on the IoT platform "Rimotte"
I tried to display GUI on Mac with X Window System
I tried to process the image in "pencil style" with OpenCV
I tried to expand the size of the logical volume with LVM
I tried to summarize everyone's remarks on slack with wordcloud (Python)
I tried to improve the efficiency of daily work with Python
PhytoMine-I tried to get the genetic information of plants with Python
Step notes to get started with django
I tried to implement Autoencoder with TensorFlow
I tried to summarize the umask command