Skip to content

Latest commit

 

History

History
125 lines (87 loc) · 4.46 KB

README.md

File metadata and controls

125 lines (87 loc) · 4.46 KB
Raw

LinkedIn


Logo

Maplestory AI Trainer

Uses screen captures, OCR, and Reinforcement Learning to optimize training on a specific map in Maplestory

Table of Contents
  1. About The Project
  2. Getting Started
  3. Usage
  4. Roadmap
  5. Contributing
  6. License
  7. Contact
  8. Acknowledgments

Demo

Following demo is completely run via AI.

alt text

About The Project

The projects functions by first taking a screenshot of the maplestory screen. It then crops out the exp and health locations of the screen. The reward is comprised of amount of exp gained - amount of health lost. There are 3 multi discrete actions the AI can take and any one time using.

  1. Moving: left, right, up down
  2. Attacking: basic attack, power attack (lucky seven)
  3. Misc: Pick up item, Jump

At first, the model was done training after about 300 steps. It was just walking into a wall and attaching constantly. Researched a bit and lowered the learning rate, adjusted the gamma, and raised the batch size. This produced decent results.

Creates a customer environment using stable baselines3. Utilzes the PPO modules with CnnPolicy as it's good for pixel-based input.

Getting Started

Prerequisites

  • Install python 3.11
  • Install Tesseract-ocr source

Installation

  1. Clone the repo

    git clone https://github.com/GrahamMThomas/MapleAITrainer.git

  2. Create virtualEnv and install requirements

    python -m venv venv
./venv/scripts/Activate
pip install -r requirements.txt

  3. Run Training

    python train.py
  • check_env.py - Checks env for method signatures and input outputs are valid
  • test_env.py - Runs random commands on your environment to ensure it works
  • run_latest.py - Loads the latest_model and runs the environment using it to drive commands w/o training

Running Metrics

Launch Tensorboard

tensorboard --logdir .\logs\

alt text

Had to backtrack around 15k steps as training was invalid

(back to top)

Roadmap

  • Use ratios to load health, exp, and maple admin locations. Currently breaks if you go to another computer which runs at a different resolution
  • Preprocess data. Currently we send a lower rez screenshot to the model as input. I think there is a lot of noise and stuff like ladders/steps aren't detected well. By using OCR to locate player, enemies, ladders, steps, floors etc, I may be able to improve model performance.
  • Use minimap to punish the AI for staying in one area too long as enemies may have accumulated in other areas.

See the open issues for a full list of proposed features (and known issues).

(back to top)

(back to top)