I tried to make a strategy for blackjack while studying Python and reinforcement learning. There is a probability-based strategy called a basic strategy, but I will try to catch up with it.
I will proceed like this
--Looks good at studying programming (Graduation exam from programming beginners should develop "Blackjack") --Compare post-learning strategies with basic strategies (benchmarks available) --Blackjack I want to get better
Blackjack is a popular table game in casinos. I will briefly introduce the rules.
--Distribute cards between dealers and players --The one whose total hand is close to 21 points wins --Loss (burst) when the total points in your hand exceed 22 points --How to count points ―― 2-9 ・ ・ ・ 2-9 points as it is ―― 10 and picture cards ・ ・ ・ 10 points --A (Ace) ・ ・ ・ 1 point or 11 points
Make a blackjack like this.
--Use 6 sets of playing cards --BET function available (However, the amount is fixed at $ 100. The return will be a reward for reinforcement learning.) --Initial possession is $ 1000 --Player choices ――Stand ・ ・ ・ Compete without drawing a card --Hit ... Draw another card --Double Down ・ ・ ・ Double the BET and draw only one more card --Surrender ・ ・ ・ Abandon half of BET and get off play --Do not implement --Split: If the two cards dealt have the same number, add the same amount as the first BET and play in two. --Insurance: When the dealer's face-up card is A, add half of the BET to insure. --BlackJack ・ ・ ・ A + picture card or 10 will give you 21 points
The entire code is listed at the end.
Generate playing cards. How to count the points of special blackjack picture cards is defined here. A is also special, but here it is generated as one point. The score of A is determined by considering the score of other hands in the Hand class.
class Card:
'''
Generate card
Numbers: A, 2-10, J, Q, K
Suits: spades, hearts, diamonds, clubs
'''
SUITS = '♠♥♦♣'
RANKS = range(1, 14) #Normal Rank
SYMBOLS = "A", "2", "3", "4", "5", "6", "7", "8", "9", "10", "J", "Q", "K"
POINTS = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 10, 10, 10] #Points for BlackJack
def __init__(self, suit, rank):
self.suit = suit
self.rank = rank
self.index = suit + self.SYMBOLS[rank - self.RANKS[0]]
self.point = self.POINTS[rank - self.RANKS[0]]
def __repr__(self):
return self.index
Define NUM_DECK = 6
and generate a deck with 6 sets of playing cards.
Shuffle is done at the same time as deck generation.
class Deck:
'''
Deck with shuffled cards (generate deck)
'''
def __init__(self):
self.cards = [Card(suit, rank) \
for suit in Card.SUITS \
for rank in Card.RANKS]
if NUM_DECK > 1:
temp_cards = copy.deepcopy(self.cards)
for i in range(NUM_DECK - 1):
self.cards.extend(temp_cards)
random.shuffle(self.cards)
・ ・ ・ Abbreviation
Add cards to your hand and calculate points. The judgment of soft hand (hand including A) is also included.
class Hand:
"""
Hand class
"""
def __init__(self):
self.hand = []
self.is_soft_hand = False
def add_card(self, card):
#The process of adding a card to your hand.
・ ・ ・ Abbreviation
def check_soft_hand(self):
#Check if it is a soft hand (hand including A)
・ ・ ・ Abbreviation
def sum_point(self):
#Calculate the points in your hand.
#For soft hands, calculate both when A is 1 and when A is 11.
・ ・ ・ Abbreviation
def calc_final_point(self):
#Calculate points when competing with Dealer
#Returns the final point not BUST
・ ・ ・ Abbreviation
def is_bust(self):
#Determine if your hand is BUST (more than 21)
・ ・ ・ Abbreviation
def deal(self, card):
#Processing at the time of Deal
・ ・ ・ Abbreviation
def hit(self, card):
#Processing at the time of Hit
・ ・ ・ Abbreviation
Choices such as Stand and Hit are defined here. If you want to add unimplemented Split and Insurance, go here. As a bonus, I added an automatic play mode. You can play with the appropriate AI described later.
class Player:
def __init__(self):
self.hand = Hand()
self.chip = Chip()
self.done = False #Flag indicating the end of the player's turn
self.hit_flag = False #Flag indicating whether Player has selected Hit
self.is_human = False # True:People play, False:Auto play
def init_player(self):
#Initialize your hand and each flag
・ ・ ・ Abbreviation
def deal(self, card):
#Processing at Stand
・ ・ ・ Abbreviation
def hit(self, card):
#Processing at the time of Hit
・ ・ ・ Abbreviation
def stand(self):
#Processing at Stand
・ ・ ・ Abbreviation
def double_down(self, card):
#Processing at the time of Double down
・ ・ ・ Abbreviation
def surrender(self):
#Processing at the time of Surrender
・ ・ ・ Abbreviation
It might have been nice to be with the Player class. I thought that a method unique to a dealer was necessary, but I didn't have any.
Chip exchange after the game. If you win, you will receive twice the amount you bet, and if you lose, you will forfeit the entire amount. If it is a draw, the bet amount will be returned to Player as it is.
class Chip:
def __init__(self):
self.balance = INITIAL_CHIP
self.bet = 0
def bet_chip(self, bet):
self.balance -= bet
self.bet = bet
def pay_chip_win(self):
self.balance += self.bet * 2
def pay_chip_lose(self):
self.balance = self.balance
def pay_chip_push(self):
self.balance += self.bet
Contains a suitable AI for automatic play. Bonus level. It is for a test before reinforcement learning.
class AI:
・ ・ ・ Abbreviation
def select_random3(self, hand, n):
if hand < 11:
selection = 'h' # hit
elif hand == 11 and n == 2:
selection = 'd' # double down
elif hand == 16 and n == 2:
selection = 'r' # surrender
elif hand > 17:
selection = 's' # stand
else:
r = random.randint(0, 1)
if r > 0.5:
selection = 'h' # hit
else:
selection = 's' # stand
return selection
This is the main function of blackjack. It may be difficult to see because display-related methods are mixed. There is room for improvement. The player_step function is also used in the gym's step function.
class Game:
def __init__(self):
self.game_mode = 0 # 0:Waiting for start, 1:In-game, 2:Game over
self.deck = Deck()
self.player = Player()
self.dealer = Dealer()
self.judgment = 0 # 1:Win, 0:draw, -1:Lose
self.game_count = 0
self.start()
self.message_on = True #self.player.is_human # True:Display a message on the console, False:Do not display messages on the console
def start(self):
#Shuffle the deck, Player,Initialize Dealer
・ ・ ・ Abbreviation
def reset_game(self):
# Player,Reset Dealer's hand
・ ・ ・ Abbreviation
def bet(self, bet):
#Player bets
・ ・ ・ Abbreviation
def deal(self, n=2):
# Player,Deal cards to Dealer
・ ・ ・ Abbreviation
def player_turn(self):
#Player's turn. Input the action from the console or automatically decide with the appropriate AI
・ ・ ・ Abbreviation
def player_step(self, action):
# Stand, Hit, Double down,Processing according to Surrender
・ ・ ・ Abbreviation
def show_card(self):
#Display the card. Dealer's face down card is displayed as "?"
・ ・ ・ Abbreviation
def dealer_turn(self):
#Dealer's turn. Draw a card until you have 17 or more points
・ ・ ・ Abbreviation
def open_dealer(self):
#Open Dealer's face down card
・ ・ ・ Abbreviation
def judge(self):
#Judgment of victory or defeat
・ ・ ・ Abbreviation
def pay_chip(self):
#Chip settlement
・ ・ ・ Abbreviation
def check_chip(self):
#Check if Player's possession is less than the Minimum Bet (minimum bet amount)
#If it is below, end the game
・ ・ ・ Abbreviation
def show_judgement(self):
#Show the result of victory or defeat
・ ・ ・ Abbreviation
def ask_next_game(self):
#Ask if you want to continue the game
・ ・ ・ Abbreviation
def check_deck(self):
#Check the remaining number of cards and shuffle if there are few
・ ・ ・ Abbreviation
Write the processing along the flow of the game in the main function. If you do this, you can play blackjack. By the way, ・ NUM_DECK = 6 # number of decks ・ INITIAL_CHIP = 1000 # Initial chip ・ MINIMUM_BET = 100 # Minimum bet amount It is said.
def main():
game = Game()
game.start()
while game.game_mode == 1:
game.reset_game() #Reset various
game.bet(bet=100) #bet
game.deal() #Deal cards
game.player_turn() #Player turn
game.dealer_turn() #Dealer turn
game.judge() #Judgment of victory or defeat
game.pay_chip() #Tip settlement
game.check_chip() #Check the player's balance
game.ask_next_game() #Ask if you want to continue the game
game.check_deck() #Check the number of remaining cards
print("Exit BlackJack")
print(str(game.game_count) + "I played the game times")
return game.player.chip, game.game_count
After Hit (h) or Stand (s) or Double down (d) or Surrender (r):
Enter characters from the keyboard to decide the player selection.
$I bet 100
The rest$900
Player's turn
Player : [♠A, ♦9] = [10, 20], soft card : True
Dealer : ♣6, ? = 6
Hit(h) or Stand(s) or Double down(d) or Surrender(r): s
Dealer's turn
Player : [♠A, ♦9] = 20
Dealer : [♣6, ♥K] = 16
Dealer's turn
Player : [♠A, ♦9] = 20
Dealer : [♣6, ♥K, ♥3] = 19
Player wins
Player's chips$1100
Do you want to continue? y/n: y
The number of remaining cards is 307
$I bet 100
The rest$1000
Player's turn
Player : [♠2, ♥K] = [12], soft card : False
Dealer : ♥3, ? = 3
Hit(h) or Stand(s) or Double down(d) or Surrender(r): h
Player's turn
Player : [♠2, ♥K, ♥2] = [14], soft card : False
Dealer : ♥3, ? = 3
Hit(h) or Stand(s) or Double down(d) or Surrender(r): h
Player's turn
Player : [♠2, ♥K, ♥2, ♠Q] = [24], soft card : False
Dealer : ♥3, ? = 3
Player BUST
Player loses
Player's chips$1000
Do you want to continue? y/n: y
301 cards left
$I bet 100
The rest$900
Player's turn
Player : [♥7, ♥5] = [12], soft card : False
Dealer : ♠8, ? = 8
Hit(h) or Stand(s) or Double down(d) or Surrender(r): d
Player's turn
Player : [♥7, ♥5, ♠8] = [20], soft card : False
Dealer : ♠8, ? = 8
Double down has been selected. Doubled the bet
The rest$800
Dealer's turn
Player : [♥7, ♥5, ♠8] = 20
Dealer : [♠8, ♥2] = 10
Dealer's turn
Player : [♥7, ♥5, ♠8] = 20
Dealer : [♠8, ♥2, ♣7] = 17
Player wins
Player's chips$1200
Do you want to continue? y/n: n
295 cards left
Exit BlackJack
I played the game 3 times
Keep it for your records.
blackjack.py
import random
import copy
#constant
NUM_DECK = 6 #Number of decks
NUM_PLAYER = 1 #Number of players
INITIAL_CHIP = 1000 #Initial chip
MINIMUM_BET = 100
class Card:
'''
Generate card
Numbers: A, 2-10, J, Q, K
Suits: spades, hearts, diamonds, clubs
'''
SUITS = '♠♥♦♣'
RANKS = range(1, 14) #Normal Rank
SYMBOLS = "A", "2", "3", "4", "5", "6", "7", "8", "9", "10", "J", "Q", "K"
POINTS = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 10, 10, 10] #Points for BlackJack
def __init__(self, suit, rank):
self.suit = suit
self.rank = rank
self.index = suit + self.SYMBOLS[rank - self.RANKS[0]]
self.point = self.POINTS[rank - self.RANKS[0]]
def __repr__(self):
return self.index
class Deck:
'''
Deck with shuffled cards (generate deck)
'''
def __init__(self):
self.cards = [Card(suit, rank) \
for suit in Card.SUITS \
for rank in Card.RANKS]
if NUM_DECK > 1:
temp_cards = copy.deepcopy(self.cards)
for i in range(NUM_DECK - 1):
self.cards.extend(temp_cards)
random.shuffle(self.cards)
def draw(self, n=1):
'''
A function that subtracts the specified number of cards from the deck
'''
cards = self.cards[:n]
del self.cards[:n] #Delete the drawn card from the deck
return cards
def shuffle(self):
'''
Shuffle the deck
'''
random.shuffle(self.cards)
return
def count_cards(self):
"""
Return the remaining number of decks
"""
count = len(self.cards)
return count
class Hand:
"""
Hand class
"""
def __init__(self):
self.hand = []
self.is_soft_hand = False
def add_card(self, card):
self.hand.append(card)
def check_soft_hand(self):
"""
Check if it is a soft hand (hand including A)
"""
hand_list = []
for i in range(len(self.hand)):
hand_list.append(self.hand[i].point)
hand_list.sort() #Sort your hand in ascending order
if hand_list[0] == 1 and sum(hand_list[1:]) < 11: #With a soft hand
self.is_soft_hand = True
else:
self.is_soft_hand = False
def sum_point(self):
"""
Returns the total value of points
"""
self.check_soft_hand()
hand_list = []
for i in range(len(self.hand)):
hand_list.append(self.hand[i].point)
hand_list.sort() #Sort your hand in ascending order
s1 = 0 #Initial value when counting A as 1
for i in range(len(self.hand)):
s1 += self.hand[i].point
if self.is_soft_hand == True: #With a soft hand
s2 = 11 #Count the first A as 11
for i in range(len(hand_list)-1):
s2 += hand_list[i+1]
s = [s1, s2]
else:
s = [s1]
return s
def calc_final_point(self):
"""
Returns the final point not BUST
"""
temp_point = self.sum_point()
if max(temp_point) > 22:
p = temp_point[0] #If the one with the larger points is BUST, the one with the smaller points
else:
p = max(temp_point)
return p
def is_bust(self):
"""
Determine if it is BUST
"""
if min(self.sum_point()) > 21: #If the smaller point exceeds 21
return True
else:
return False
def deal(self, card):
"""
Add the Dealed card to your hand
"""
for i in range(len(card)):
self.add_card(card[i])
def hit(self, card):
#Hit one by one
if len(card) == 1:
self.add_card(card[0])
else:
print("The number of cards is incorrect")
class Player:
def __init__(self):
self.hand = Hand()
self.chip = Chip()
self.done = False #Flag indicating the end of the player's turn
self.hit_flag = False #Flag indicating whether Player has selected Hit
self.is_human = True # True:People play, False:Auto play
def init_player(self):
self.hand = Hand()
self.done = False
self.hit_flag = False
def deal(self, card):
self.hand.deal(card)
def hit(self, card):
#Add 1 card to your hand
self.hand.hit(card)
self.hit_flag = True
def stand(self):
#End of turn
self.done = True
def double_down(self, card):
#Double the bet and hit only once to end the turn
self.chip.balance -= self.chip.bet
self.chip.bet = self.chip.bet * 2
self.hand.hit(card)
self.done = True #Make it a rule that you can hit only once after double down
def surrender(self):
#Halve the Bet (return half of the bet to hand) and end the turn
# self.chip.balance += int(self.chip.bet / 2)
self.chip.bet = int(self.chip.bet / 2)
self.chip.balance += self.chip.bet
self.done = True
def insurance(self):
#Unimplemented
pass
def split(self):
#Unimplemented
pass
class Dealer:
def __init__(self):
self.hand = Hand()
def init_dealer(self):
self.hand = Hand()
def deal(self, card):
self.hand.deal(card)
def hit(self, card):
self.hand.hit(card)
class Chip:
def __init__(self):
self.balance = INITIAL_CHIP
self.bet = 0
def bet_chip(self, bet):
#If you hang Chip, reduce from hand
self.balance -= bet
self.bet = bet
def pay_chip_win(self):
#When you win, you get twice as much as BET
self.balance += self.bet * 2
def pay_chip_lose(self):
#When you lose, you lose all BET
self.balance = self.balance
def pay_chip_push(self):
#When drawing, return by the amount of BET
self.balance += self.bet
class AI:
def select_random1(self):
r = random.randint(0, 1)
if r > 0.5:
selection = 'h' # hit
else:
selection = 's' # stand
return selection
def select_random2(self, hand):
if hand <= 11:
selection = 'h'
else:
r = random.randint(0, 1)
if r > 0.5:
selection = 'h' # hit
else:
selection = 's' # stand
return selection
def select_random3(self, hand, n):
if hand < 11:
selection = 'h' # hit
elif hand == 11 and n == 2:
selection = 'd' # double down
elif hand == 16 and n == 2:
selection = 'r' # surrender
elif hand > 17:
selection = 's' # stand
else:
r = random.randint(0, 1)
if r > 0.5:
selection = 'h' # hit
else:
selection = 's' # stand
return selection
class Game:
def __init__(self):
self.game_mode = 0 # 0:Waiting for start, 1:In-game, 2:Game over
self.deck = Deck()
self.player = Player()
self.dealer = Dealer()
self.judgment = 0
self.game_count = 0
self.start()
self.message_on = True #self.player.is_human # True:Display a message on the console, False:Do not display messages on the console
def start(self):
self.deck.shuffle()
self.game_mode = 1
self.player = Player()
self.dealer = Dealer()
self.game_count = 0
def reset_game(self):
self.player.init_player()
self.dealer.init_dealer()
self.game_count += 1
def bet(self, bet):
self.player.chip.bet_chip(bet=bet)
if self.message_on:
print("$" + str(self.player.chip.bet) + "I bet")
print("The rest$" + str(self.player.chip.balance))
#Deal cards
def deal(self, n=2):
'''
Deal cards
'''
card = self.deck.draw(n)
self.player.deal(card)
# print(self.player.hand.hand)
card = self.deck.draw(n)
self.dealer.deal(card)
# print(self.dealer.hand.hand)
self.judgment = 0 #Judgment of victory or defeat
self.player.done = False
self.show_card()
#Player's turn
def player_turn(self):
'''
Player turn
'''
if self.player.hand.calc_final_point() == 21: #If the total is 21, immediately go to Dealer's turn
self.player.done = True
while not self.player.done and not self.player.hand.is_bust():
if self.player.is_human is True:
action = input("Hit(h) or Stand(s) or Double down(d) or Surrender(r): ")
elif self.player.is_human is True and self.player.hit_flag:
action = input("Hit(h) or Stand(s): ") #Hit after hit/stand only
else:
action = AI().select_random3(hand=self.player.hand.calc_final_point(), n=len(self.player.hand.hand))
self.player_step(action=action)
def player_step(self, action):
if action == 'h': # Hit
card = self.deck.draw(1)
self.player.hit(card)
self.show_card()
if self.player.hand.calc_final_point() == 21: #No more hits when the total score reaches 21
self.player.done = True
if self.player.hand.is_bust():
self.player.done = True
self.judgment = -1 #If Player BUST, lose immediately
if self.message_on:
print("Player BUST")
elif action == 's': # Stand
self.player.stand()
elif action == 'd' and self.player.hit_flag is False: # Double down.Yes if Hit is not selected
card = self.deck.draw(1)
if self.player.chip.balance >= self.player.chip.bet: #Double Down is possible if the balance is more than the bet amount
self.player.double_down(card)
self.show_card()
if self.message_on:
print("Double down has been selected. Doubled the bet")
print("The rest$" + str(self.player.chip.balance))
if self.player.hand.is_bust():
self.player.done = True
self.judgment = -1 #If Player BUST, lose immediately
if self.message_on:
print("Player BUST")
else: #Hit if the balance is less than the bet amount
print("Hit because there are not enough chips")
self.player.hit(card)
self.show_card()
if self.player.hand.calc_final_point() == 21: #No more hits when the total score reaches 21
self.player.done = True
if self.player.hand.is_bust():
self.player.done = True
self.judgment = -1 #If Player BUST, lose immediately
if self.message_on:
print("Player BUST")
elif action == 'r' and self.player.hit_flag is False: # Surrender.Yes if Hit is not selected
self.player.surrender()
self.judgment = -1 #Lost because I chose Surrender
if self.message_on:
print("Surrender selected")
else:
if self.message_on:
print("Choose the right option")
def show_card(self):
'''
Show player card
'''
if self.message_on:
print("Player's turn")
print("Player : " + str(self.player.hand.hand) + " = " +
str(self.player.hand.sum_point()) + ", soft card : " + str(self.player.hand.is_soft_hand))
print("Dealer : " + str(self.dealer.hand.hand[0].index) +
", ? = " + str(self.dealer.hand.hand[0].point))
else:
pass
def dealer_turn(self):
'''
Dealer turn
'''
if self.judgment == -1:
return
self.open_dealer()
while self.dealer.hand.calc_final_point() < 17 and self.judgment == 0:
card = self.deck.draw(1)
self.dealer.hit(card)
self.open_dealer()
if self.dealer.hand.calc_final_point() > 21:
self.judgment = 1
if self.message_on:
print("Dealer BUST")
def open_dealer(self):
'''
Open a hole card
'''
if self.message_on:
print("Dealer's turn")
print("Player : " + str(self.player.hand.hand) + " = " +
str(self.player.hand.calc_final_point()))
print("Dealer : " + str(self.dealer.hand.hand) + " = " +
str(self.dealer.hand.calc_final_point()))
else:
pass
def judge(self):
'''
Judgment of victory or defeat
'''
if self.judgment == 0 and self.player.hand.calc_final_point() > \
self.dealer.hand.calc_final_point():
self.judgment = 1
elif self.judgment == 0 and self.player.hand.calc_final_point() < \
self.dealer.hand.calc_final_point():
self.judgment = -1
elif self.judgment == 0 and self.player.hand.calc_final_point() == \
self.dealer.hand.calc_final_point():
self.judgment = 0
if self.message_on:
self.show_judgement()
def pay_chip(self):
previous_chip = self.player.chip.balance
if self.judgment == 1: # Player win
self.player.chip.pay_chip_win()
elif self.judgment == -1: # Player lose
self.player.chip.pay_chip_lose()
elif self.judgment == 0: # Push
self.player.chip.pay_chip_push()
if self.message_on:
print("Player's chips$" + str(self.player.chip.balance))
reward = self.player.chip.balance - previous_chip #Rewards earned in this game
return reward
def check_chip(self):
if self.player.chip.balance < MINIMUM_BET:
self.game_mode = 2
if self.message_on:
print("The game ends because the chips are below the Minimum Bet")
def show_judgement(self):
'''
Display of victory or defeat
'''
if self.message_on:
print("")
if self.judgment == 1:
print("Player wins")
elif self.judgment == -1:
print("Player loses")
elif self.judgment == 0:
print("draw")
print("")
else:
pass
def ask_next_game(self):
'''
Ask if you want to continue the game
'''
if self.player.is_human == True:
while self.game_mode == 1:
player_input = input("Do you want to continue? y/n: ")
if player_input == 'y':
break
elif player_input == 'n':
self.game_mode = 2
break
else:
print('y/Please enter n')
else:
pass #Continue with automatic play
print('The number of remaining cards' + str(self.deck.count_cards()))
print("")
def check_deck(self):
'''
Check the remaining number of cards and shuffle if there are few
'''
if self.deck.count_cards() < NUM_PLAYER * 10 + 5:
self.deck = Deck()
if self.message_on:
print("Initialized the deck")
print('The number of remaining cards' + str(self.deck.count_cards()))
print("")
def main():
game = Game()
game.start()
while game.game_mode == 1:
game.reset_game() #Reset various
game.bet(bet=100) #bet
game.deal() #Deal cards
game.player_turn() #Player turn
game.dealer_turn() #Dealer turn
game.judge() #Judgment of victory or defeat
game.pay_chip() #Tip settlement
game.check_chip() #Check the player's balance
game.ask_next_game() #Ask if you want to continue the game
game.check_deck() #Check the number of remaining cards
print("Exit BlackJack")
print(str(game.game_count) + "I played the game times")
return game.player.chip, game.game_count
if __name__ == '__main__':
main()
I made blackjack with Python. I also tried to implement Double down and Surrender to get as close as possible to the basic strategy. I want to add Split someday. I added the BET function on the assumption that I will learn, but even though I have zero money, I double down, etc. Many bugs were found and it was difficult to fix. (Please point out if there are still bugs)
Next, register the blackjack created this time as your own environment in the gym of OpenAI. Create a blackjack strategy with reinforcement learning (② Register the environment in gym)
-I made blackjack with Python -Graduation exams from programming beginners should develop "Blackjack" -Blackjack Basic Glossary
Recommended Posts