I am trying to teach a computer to learn to play Tic Tac Toe. Writing the code for this I found that it was becoming too specific for Tic Tac Toe. I decided I wanted to try to write a program that learns how to play games. I was really unsure how to do this, so I wanted to learn from myself. I played a bunch of old Nintendo games, games that I have never played before, and watched how I adapted to playing the games. I picked 3 games at random to log my learning, Baltron (A 2d shooter), Fighting Road (a karate style fighter), and Flipull (a puzzle game).

Starting off with Baltron, I immediately recognized it as a 2d shooter. First I examined all the buttons, the D-Pad moved my ship around, and the A button fired a vertical and a horizontal shot at the same time. For this style of game, I knew that the object was to shoot anything that moved, and to avoid running into anything. Playing with this knowledge I was able to do OK. After a few minutes I learned the following skills: Enemies can attack from behind, enemies can be on the ground, enemy's ammo travels at certain speeds so you can time your motions to not hit their bullets, shooting enemy's bullets does nothing, and if you shoot the red/yellow spinning thing, your bullets will reflect and hit you. So to sum up Baltron, I had the basic knowledge of how a 2d shooter played (try to shoot stuff, dont hit the ground), but I had the learn the specifics of this game (enemies can shoot at you, enemies can come from behind, shooting the spinning thing can hurt you).

Next I played Fighting Road. This game I immediately recognized as a kung fu style karate game. I knew the objective was to kick and punch my opponent. First I pushed all the buttons and saw what they did. Punching and kicking my opponent made his health go down, and when I was attacked my health went down. I had to learn not to push the D-Pad and attack at the same time as nothing would happen. I also learned that kicking did more damage than punching. I also learned other skills like jump-kicking and duck-kicking. Also I found that I could get 2-3 punches in before I was attacked, so I would punch 2-3 times, then back up as I expected the opponent to strike.

The final game I played was a puzzle game entitled Flipull. I entered the game with basically no knowledge of this game. I have never seen anything like it. There is a pile of blocks on the left, and my character is on the right on a ladder. Moving the D-Pad to the right or left does nothing, moving it up and down makes my character move up and down the ladder. There is a timer counting down from 3 minutes. I moved my character all the way to the top. As I did that, arrows started appearing on the pile of blocks. So I hit the button, and immediately threw the block I was holding onto the pile where the arrow was. The block I threw was bounced back to me. I then moved down the ladder a rung and threw it again. This time it landed on a block that was the same color that I threw. I received some points for this. So now I know I need to match up colors. Playing around with it for a while, I figured out that after a certain height, when I threw the block, it would fall vertically on the pile. Below that height, I would throw the block horizontally at the pile. Once you clear the board, you get more points for finishing the game, and more points for a 'Time Bonus'. If you get into the situation where you cannot place a block, you are penalized. So the strategy here is to plan a head in such a way that you will always be able to place a block somewhere.

For all of the games, I had certain knowledge. I knew how to use the buttons on the controller. For each individual game, I had to learn what these buttons did. In the first two games, I had previous knowledge of those 'style' of games and what the basic idea was behind playing them. I also received immediate feedback in the matter of scores. In the shooter, when I did something good, my score went up. When I did something bad, I lost a life. In the karate game, when I did something good, my opponent lost health, when I did something bad, I lost health. The puzzle game was a little different. I did not know how to play it. The feedback I received from the game was very vague at first. Doing something wrong had no effect either positive or negative. After pushing a few buttons, I found the first positive – I matched up 2 blocks and received points. Using that as a general rule, I played the game until I could not match up the block. At that point, I lost a life (the first negative reinforcement). So now I knew that planning was in the works. I had to plan my block placements in such a way that the next block would have somewhere to go.

Now I need some way to relate my learning experiences to a way that a computer can learn. First off, the computer program must be equipped with some basic knowledge. It must know that there exists directional buttons that usually move things around. It must also know about the existence of 'action buttons'. When the game first starts off, the computer must 'figure out' what all these buttons do. This is done simply by pressing them one at a time and seeing what happens. This can be stored in a table of some sort. (Up – y coord goes down, etc...). During this time, any positive or negatives can be recorded. I think using this methodology, you can come up with rule sets pretty easily that will make an effective game player for the first 2 games (shooter and kung fu style).

Chaining rulesets together will work good for the first two types of games, but for the puzzle game, I think something more needs to be done. The learning process would start the same. Figuring out what each button does. Even the part of matching up similar blocks would be pretty straight forward. The problem comes as you try to map out a strategy for planning ahead. You could use a simple tree to go through every possible move or set of moves. This actually would not be that hard, or time consuming, as the depth of your tree would be really easy to evaluate (how many blocks are left). I guess the big problem comes as to how to decide that you need plan. What makes this different than the straight shooter or kung fu game?

2005-03-10T15:32:52