![\"\"](\"http:\/\/cwiok.pl\/wp-content\/uploads\/2017\/11\/image3-e1510661204990.png\")
<\/p>\nI decided to try creating a self-learning algorithm on a simple problem. I know that the solution is an overkill, but nonetheless I wanted to explore the possibilities connected to using GA with ANN.<\/p>\n
<\/p>\n
<\/p>\n
The plan was simple:
\n1. Capture the screen using OpenCV.
\n2. Get the X,Y of the bird and columns. Let algorithm know if it’s even playing.
\n3. Put it into an ANN to make a “decision”
\n4. Score the ANN basing on the total time played
\n5. Breed new ANNs using GA on the best parent ANNs
\n6. Repeat
\n<\/p>\n
<\/p>\n
Step zero was finding the right game version to use. I opted for\u00a0this one<\/a>. It was mainly due to the fact that you can jump with space and that I was able to slow the game down a little bit by editing its JS script.<\/p>\n Capturing the screen using Python’s OpenCV is fairly easy. I put the webbrowser of the left side of the screen, so that the game windows had always the same coordinates (612,162,1090,781). The code:<\/p>\n This gave me the access to the screen, which was saved as an numpy array. Because of that I could apply some logic to find the bird and columns, and also to let the algorithm know if it’s playing or not.<\/p>\n Finding column was done by finding a template column on the screen. I uploaded the column template and read its height and width:<\/p>\n To find a moving and animated bird I could not use template matching, which is a very basic technique. I decided to find the bird by locating the colors that only appear on the bird. It was more of a trial and error approach, but gave me satisfactory results.<\/p>\n To check if algorithm is playing or not, I was looking for the number of points rendered above the bird. If they vanish, it means that the game ended. You can also see the whole algorithm here:<\/p>\n Put it into an ANN to make a “decision”<\/strong><\/p>\n Ok, so I have X,Y of the bird and columns. I then calculate the distance between the bird and the column in both dimensions. Those will be inputs of our ANNs. I create random ANNs using this <\/a>and this <\/a> script.<\/p>\n This line creates 10 ANNs, which make the first population:<\/p>\n Then I play the game in a loop, with i being the counter for iterations. To make a decision I feed the ANN our inputs and act basing on the result:<\/p>\n Score the ANN basing on the total time played<\/strong><\/p>\n To score an ANN I calculate the time between a start and an end of a game. The clock starts ticking, when the algorithm clicks space in menu and stops when the bird dies. Ranking_sieci is a list with both ANNs and their scores. When sorted by scores it is the list of best parents, that will breed.<\/p>\n Breed new ANNs using GA on the best parent ANNs<\/strong><\/p>\n Basing on the best scores, I use crossover and mutate methods by invoking create_new_pop method from evolution.py file:<\/p>\n If less than 2 ANNs passed minimum requirements, I create a new random list of ANNs and insert into it mutations of the best ANN from the previous population. This is done to not lose on the previous learnings.<\/p>\n Repeat<\/strong><\/p>\n After creating a new population, I set the i to 0, to make the loop run again.<\/p>\n The results are surprisingly good. After only a few generations the algorithm became “good enough” being able to play for as long as I cared to wait \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":" I decided to try creating a self-learning algorithm on a simple problem. I know that the solution is an overkill, but nonetheless I wanted to explore the possibilities connected to using GA with ANN.<\/p>\n <\/p>\n\n
import numpy as np\r\nfrom PIL import ImageGrab\r\nimport cv2\r\n\r\n#To get the right colors\r\ndef process_img(original_image):\r\n processed_img = cv2.cvtColor(original_image, cv2.COLOR_BGR2RGB)\r\n return processed_img\r\nscreen1 = np.array(ImageGrab.grab(bbox=(612,162,1090, 781)))\r\nnew_screen = process_img(screen1)\r\ncv2.imshow('Flappy bird tracking',new_screen)\r\nif cv2.waitKey(25) & 0xFF == ord('q'):\r\n cv2.destroyAllWindows()\r\n break\r\n<\/pre>\n\n
template = cv2.imread('column.jpg',0)\r\nw, h = template.shape[::-1]\r\nres = cv2.matchTemplate(cv2.cvtColor(new_screen,cv2.COLOR_BGR2GRAY),template,cv2.TM_CCOEFF_NORMED)\r\nthreshold = 0.8\r\nloc_column = np.where( res >= threshold)\r\nfor pt in zip(*loc_column[::-1]):\r\n cv2.rectangle(new_screen, pt , (pt[0] + w+55, pt[1] + h), (0,255,255), 2)\r\nif loc_column[::-1][0].any(): \r\n bird_distance = -bird_dist+loc_column[::-1][0][0]\r\n column_height = loc_column[::-1][1][0] \r\nelse:\r\n bird_distance = -bird_dist+300\r\n column_height = -bird_height+280\r\n<\/pre>\nlower_red = np.array([30,180,200])\r\nupper_red = np.array([50,200,220])\r\nmask = cv2.inRange(new_screen, lower_red, upper_red)\r\nthreshhold = 1\r\nloc_bird = np.where(mask >= threshhold)\r\nfor pt in zip(*loc_bird[::-1]):\r\n cv2.rectangle(new_screen, (loc_bird[::-1][0][0] -20, loc_bird[::-1][1][0] - 5), (loc_bird[::-1][0][0] +30, loc_bird[::-1][1][0] +30), (0,255,255), 2)\r\nif loc_bird[::-1][0].any():\r\n #print('Ptak na wysokosci' + str(560-loc_bird[::-1][1][0]))\r\n bird_height = loc_bird[::-1][1][0]\r\n bird_dist = loc_bird[::-1][0][0]\r\n #print(bird_dist)\r\nelse:\r\n bird_height = 0\r\n bird_dist = 72\r\n<\/pre>\n #If there are no white points in the screen upper part:\r\nwhile(i<10):\r\n if(not([0,0,0] in screen1[100:120,197:267])):\r\n time.sleep(0.5) \r\n #Press space to start playing:\r\n if(state != 'Play'):\r\n state_update = 'Menu'\r\n state=state_update\r\n pyautogui.press('space')\r\n #The game just ended, score everything:\r\n else:\r\n state_update = 'End'\r\n if(state_update != state):\r\n state=state_update\r\n czas_gry = time.time()-last_time\r\n \r\n print(str(bird_height_old) + ' ' + str(czas_gry-0.5))\r\n if((bird_height_old != 600) and (czas_gry-0.5 > 10)):\r\n ranking_sieci.append([lista_sieci[i],czas_gry-0.5])\r\n \r\n i=i+1\r\n if(i == 10):\r\n if(len(ranking_sieci)>1):\r\n mutation_rate = 0.2\r\n \r\n sieci_ranking = sorted(ranking_sieci,key=lambda x: (x[1]),reverse=True)\r\n ranking_sieci=[]\r\n print(sieci_ranking)\r\n lista_sieci = []\r\n sieci_ranking[0][0].save(str(sieci_ranking[0][1]).replace(\".\",\"\"))\r\n print('tworze nowa populacje')\r\n evolution.create_new_pop(sieci_ranking[0][0],sieci_ranking[1][0],lista_sieci,mutation_rate)\r\n i=0\r\n else:\r\n print(ranking_sieci)\r\n print('musze zaorac')\r\n mutation_rate = 0.2\r\n test = sorted(ranking_sieci,key=lambda x: (x[1]),reverse=True)\r\n lista_sieci = [neural_net.NN(2,4,1) for i in range(10) ]\r\n i=0\r\n ranking_sieci=[]\r\n \r\n if (len(test)>0): \r\n \r\n test[0][0].save(str(test[0][1]).replace(\".\",\"\"))\r\n lista_sieci[0] = evolution.mutate(test[0][0],0.05)\r\n lista_sieci[1] = evolution.mutate(test[0][0],0.1)\r\n lista_sieci[2] = evolution.mutate(test[0][0],0.2)\r\n \r\n \r\n \r\n time.sleep(1) \r\n pyautogui.press('space')\r\n \r\n else:\r\n \r\n state_update = 'Play'\r\n \r\n if(state == 'Menu'):\r\n last_time= time.time()\r\n state=state_update\r\n \r\n pyautogui.press('space')\r\n \r\n \r\n \r\n else:\r\n \r\n decision = lista_sieci[i].runNN([column_height,bird_height,])\r\n \r\n if(column_height == 600):\r\n time.sleep(1.4)\r\n \r\n \r\n if(decision[0]>0.5):\r\n pyautogui.press('space')\r\n \r\n \r\n bird_height_old = bird_height \r\n\r\n<\/pre>\nlista_sieci = [neural_net.NN(2,4,1) for i in range(10) ]\r\n<\/pre>\n
decision = lista_sieci[i].runNN([column_height,bird_height,])\r\nif(decision[0]>0.5):\r\n pyautogui.press('space')\r\n<\/pre>\nczas_gry = time.time()-last_time \r\nprint(str(bird_height_old) + ' ' + str(czas_gry-0.5))\r\nranking_sieci.append([lista_sieci[i],czas_gry-0.5])\r\n<\/pre>\n
evolution.create_new_pop(sieci_ranking[0][0],sieci_ranking[1][0],lista_sieci,mutation_rate)\r\n<\/pre>\n
![\"\"](\"http:\/\/cwiok.pl\/wp-content\/uploads\/2018\/05\/artyku\u0142_03_nauka-gry-flappy-bird.jpg\")
<\/p>\n