Learning how to play Flappy Bird using ANN and genetic algorithm
I decided to try creating a self-learning algorithm on a simple problem. I know that the solution is an overkill, but nonetheless I wanted to explore the possibilities connected to using GA with ANN.
The plan was simple:
1. Capture the screen using OpenCV.
2. Get the X,Y of the bird and columns. Let algorithm know if it’s even playing.
3. Put it into an ANN to make a “decision”
4. Score the ANN basing on the total time played
5. Breed new ANNs using GA on the best parent ANNs
6. Repeat
Step zero was finding the right game version to use. I opted for this one. It was mainly due to the fact that you can jump with space and that I was able to slow the game down a little bit by editing its JS script.
- Capture the screen using OpenCV.
Capturing the screen using Python’s OpenCV is fairly easy. I put the webbrowser of the left side of the screen, so that the game windows had always the same coordinates (612,162,1090,781). The code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
import numpy as np from PIL import ImageGrab import cv2 #To get the right colors def process_img(original_image): processed_img = cv2.cvtColor(original_image, cv2.COLOR_BGR2RGB) return processed_img screen1 = np.array(ImageGrab.grab(bbox=(612,162,1090, 781))) new_screen = process_img(screen1) cv2.imshow('Flappy bird tracking',new_screen) if cv2.waitKey(25) & 0xFF == ord('q'): cv2.destroyAllWindows() break |
This gave me the access to the screen, which was saved as an numpy array. Because of that I could apply some logic to find the bird and columns, and also to let the algorithm know if it’s playing or not.
- Get the X,Y of the bird and columns. Let algorithm know if it’s even playing.
Finding column was done by finding a template column on the screen. I uploaded the column template and read its height and width:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
template = cv2.imread('column.jpg',0) w, h = template.shape[::-1] res = cv2.matchTemplate(cv2.cvtColor(new_screen,cv2.COLOR_BGR2GRAY),template,cv2.TM_CCOEFF_NORMED) threshold = 0.8 loc_column = np.where( res >= threshold) for pt in zip(*loc_column[::-1]): cv2.rectangle(new_screen, pt , (pt[0] + w+55, pt[1] + h), (0,255,255), 2) if loc_column[::-1][0].any(): bird_distance = -bird_dist+loc_column[::-1][0][0] column_height = loc_column[::-1][1][0] else: bird_distance = -bird_dist+300 column_height = -bird_height+280 |
To find a moving and animated bird I could not use template matching, which is a very basic technique. I decided to find the bird by locating the colors that only appear on the bird. It was more of a trial and error approach, but gave me satisfactory results.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
lower_red = np.array([30,180,200]) upper_red = np.array([50,200,220]) mask = cv2.inRange(new_screen, lower_red, upper_red) threshhold = 1 loc_bird = np.where(mask >= threshhold) for pt in zip(*loc_bird[::-1]): cv2.rectangle(new_screen, (loc_bird[::-1][0][0] -20, loc_bird[::-1][1][0] - 5), (loc_bird[::-1][0][0] +30, loc_bird[::-1][1][0] +30), (0,255,255), 2) if loc_bird[::-1][0].any(): #print('Ptak na wysokosci' + str(560-loc_bird[::-1][1][0])) bird_height = loc_bird[::-1][1][0] bird_dist = loc_bird[::-1][0][0] #print(bird_dist) else: bird_height = 0 bird_dist = 72 |
To check if algorithm is playing or not, I was looking for the number of points rendered above the bird. If they vanish, it means that the game ended. You can also see the whole algorithm here:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 |
#If there are no white points in the screen upper part: while(i<10): if(not([0,0,0] in screen1[100:120,197:267])): time.sleep(0.5) #Press space to start playing: if(state != 'Play'): state_update = 'Menu' state=state_update pyautogui.press('space') #The game just ended, score everything: else: state_update = 'End' if(state_update != state): state=state_update czas_gry = time.time()-last_time print(str(bird_height_old) + ' ' + str(czas_gry-0.5)) if((bird_height_old != 600) and (czas_gry-0.5 > 10)): ranking_sieci.append([lista_sieci[i],czas_gry-0.5]) i=i+1 if(i == 10): if(len(ranking_sieci)>1): mutation_rate = 0.2 sieci_ranking = sorted(ranking_sieci,key=lambda x: (x[1]),reverse=True) ranking_sieci=[] print(sieci_ranking) lista_sieci = [] sieci_ranking[0][0].save(str(sieci_ranking[0][1]).replace(".","")) print('tworze nowa populacje') evolution.create_new_pop(sieci_ranking[0][0],sieci_ranking[1][0],lista_sieci,mutation_rate) i=0 else: print(ranking_sieci) print('musze zaorac') mutation_rate = 0.2 test = sorted(ranking_sieci,key=lambda x: (x[1]),reverse=True) lista_sieci = [neural_net.NN(2,4,1) for i in range(10) ] i=0 ranking_sieci=[] if (len(test)>0): test[0][0].save(str(test[0][1]).replace(".","")) lista_sieci[0] = evolution.mutate(test[0][0],0.05) lista_sieci[1] = evolution.mutate(test[0][0],0.1) lista_sieci[2] = evolution.mutate(test[0][0],0.2) time.sleep(1) pyautogui.press('space') else: state_update = 'Play' if(state == 'Menu'): last_time= time.time() state=state_update pyautogui.press('space') else: decision = lista_sieci[i].runNN([column_height,bird_height,]) if(column_height == 600): time.sleep(1.4) if(decision[0]>0.5): pyautogui.press('space') bird_height_old = bird_height |
Put it into an ANN to make a “decision”
Ok, so I have X,Y of the bird and columns. I then calculate the distance between the bird and the column in both dimensions. Those will be inputs of our ANNs. I create random ANNs using this and this script.
This line creates 10 ANNs, which make the first population:
|
1 |
lista_sieci = [neural_net.NN(2,4,1) for i in range(10) ] |
Then I play the game in a loop, with i being the counter for iterations. To make a decision I feed the ANN our inputs and act basing on the result:
|
1 2 3 |
decision = lista_sieci[i].runNN([column_height,bird_height,]) if(decision[0]>0.5): pyautogui.press('space') |
Score the ANN basing on the total time played
To score an ANN I calculate the time between a start and an end of a game. The clock starts ticking, when the algorithm clicks space in menu and stops when the bird dies. Ranking_sieci is a list with both ANNs and their scores. When sorted by scores it is the list of best parents, that will breed.
|
1 2 3 |
czas_gry = time.time()-last_time print(str(bird_height_old) + ' ' + str(czas_gry-0.5)) ranking_sieci.append([lista_sieci[i],czas_gry-0.5]) |
Breed new ANNs using GA on the best parent ANNs
Basing on the best scores, I use crossover and mutate methods by invoking create_new_pop method from evolution.py file:
|
1 |
evolution.create_new_pop(sieci_ranking[0][0],sieci_ranking[1][0],lista_sieci,mutation_rate) |
If less than 2 ANNs passed minimum requirements, I create a new random list of ANNs and insert into it mutations of the best ANN from the previous population. This is done to not lose on the previous learnings.
Repeat
After creating a new population, I set the i to 0, to make the loop run again.
The results are surprisingly good. After only a few generations the algorithm became “good enough” being able to play for as long as I cared to wait 🙂
