Learnfun & Playfun                          Tom 7       17 August 2013

 Experimental code for automating the play of NES games.
 This directory contains some false starts and the playfun and learnfun
 algorithms, which are described here

   http://tom7.org/mario/

 The intended platform is 64-bit windows 7. There's no windows-specific
 stuff here, but compiling in this era is a nightmare and so I don't
 expect it to work straightforwardly on other platforms. See HACKING
 for my notes on compiling. (For what it's worth, compiling on linux
 should be less painful but I've done none of the work there.)

 The fceu subdirectory is the fork of FCEUX. I deleted a bunch of
 stuff from it, and made it compile cleanly under 64-bit mingw for
 x64. It is licensed under the GPL (see fceu/COPYING), including my
 modifications.

 Despite depending on SDL (for networking) this is currently a
 headless compile; no graphics or sound or input. It compiles for me
 with the makefile I built, after installing the cygwin packages like
 x86_64-w64-mingw32-g++ (don't get the i686 ones; I hear they are a
 trap!). Make sure you get libz since it's required by this. Follow
 the instructions on http://mingw-w64.sourceforge.net/ to make sure
 that it's running the right binary (this is all so very confusing),
 but I installed via cygwin and that worked and so that was +1 for
 cygwin. You will also need Google's protocol buffers to compile the
 network stuff. This part is super annoying and I'm not good at it
 either.

 It's now possible to use learnfun and playfun as executables if you
 have a 64-bit windows computer. For anything serious you'll want to
 be able to compile, since many constants are just embedded in the
 source code.

 learnfun - learns a lexicographic ordering of RAM values, as well as
            capturing input motifs, that can be used to automatically
            try to play the game (with playfun)

 playfun  - plays the game given the output of learnfun. More or less
            works for "easy" games like Super Mario Bros.

 You should read the paper at tom7.org/mario, or at least watch the
 video, to understand the basics.

 Learnfun and Playfun work well enough to play "easy" games without
 any customization. The steps are:

 - Use FCEUX to record a movie (FM2 format) of you playing the game.
   I usually do a few thousand frames and I try to keep it simple.

 - Optionally, modify the beginning of the movie to have only a single
   frame of the "START" button. You don't need to do this, but since
   the learning (and playing) process begins with the first button
   press, it may be cleaner training data if it never sees you press
   START after that, including on the next frame. In most games,
   having START among the input motifs is an annoyance, because
   Playfun will pause the game sometimes. (This does sometimes allow
   it to play better because it can wait when all its futures are
   bad.) If it's pausing a lot, it probably has no idea what it's
   doing, anyway. You might consider editing the training data to
   be cheating. Up to you.

 - Modify config.txt to contain e.g.

   game zelda
   movie me-playing-zelda.fm2

   and make sure you have a file called "zelda.nes" in the current
   directory as well as the FM2 file you made, called
   "me-playing-zelda.fm2". Make sure there's a newline after every
   line.

 - Run ./learnfun.exe to produce an .objectives and .motifs file
   based on your inputs. It also makes some SVGs that are optional.
   The outputs are all based on the game name from the config file.
   This is usually pretty fast.

 - Now you want to run Playfun to automate playing the game. It uses
   the same config file. I strongly recommend running Playfun in
   MARIONET mode. I don't even think single-cpu mode really works any
   more and it's very slow (you would at least need to turn down the
   constants like NFUTURES). This means first starting a helper for
   each logical CPU (this is probably 2x the number of cores, since
   you probably have a hyperthreaded CPU), then a single master.

   ./playfun.exe --helper 8000
   ./playfun.exe --helper 8001
   ./playfun.exe --helper 8002
   ./playfun.exe --helper 8003
   ./playfun.exe --helper 8004
   ./playfun.exe --helper 8005

   (wait until each helper says Ready.)
 
   ./playfun.exe --master 8000 8001 8002 8003 8004 8005

   These of course need to keep running, so you should do them in
   different console windows. They output ANSI colors and escape
   sequences to draw progress bars. The program "ansicon" works
   for me in Windows 7 to render ANSI control codes. I usually
   run ansicon, then from the command prompt run cygwin's bash,
   then playfun. One window per playfun.

 - Playfun will run forever. Every once in a while it writes
   an .fm2 file (*-playfun-futures-progress.fm2) which you can
   view in FCEUX to see what it's doing! Note that playfun is
   very slow; on a 6-core Intel Extreme whatever, I often run it
   for many days to produce a few minutes of gameplay.

 - You can load up the fm2 files it outputs in an emulator to
   watch them play the original ROM. You should probably use FCEUX,
   since many emulators have subtle differences.

 Other stuff in here:

 scopefun - Generates a series of PNGs and a WAV file containing the
            frames and sound from a .fm2 movie and learnfun's objectives.
            I used this to make my second playfun video, which will
            be up at tom7.org/mario soon.

 tasbot   - A*-ish search for solutions to games. Needs a hand-written
            objective function. Very slow.

 pinviz   - Creates a graphic showing the distribution of the ball
            in the three screens from the NES game Pinball. This was
            a write and run once kind of thing.