First run on Mac OS

[Dario-Mantegazza]

Hi,
I'm trying to run gym-mupen64plus on Mac OS, but when I arrive at
conda env create -f environment.yml
I get the following error
``
By editing the environment.yml file and deleting - wx>=3.0.3 I manage to create the environment. After that i would activate the environment and `pip install wxPyhton` https://wiki.wxpython.org/How%20to%20install%20wxPython.
Using the test file described in your readme I get the following error

('ControllerHTTPServer started on port ', 8082)
Traceback (most recent call last):
File "test.py", line 4, in
env = gym.make('Mario-Kart-Luigi-Raceway-v0')
File "/Users/dariomantegazza/anaconda3/envs/gym-mupen64plus/lib/python2.7/site-packages/gym/envs/registration.py", line 164, in make
return registry.make(id)
File "/Users/dariomantegazza/anaconda3/envs/gym-mupen64plus/lib/python2.7/site-packages/gym/envs/registration.py", line 122, in make
env = spec.make()
File "/Users/dariomantegazza/anaconda3/envs/gym-mupen64plus/lib/python2.7/site-packages/gym/envs/registration.py", line 89, in make
env = cls(**self._kwargs)
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/MarioKart64/mario_kart_env.py", line 37, in init
super(MarioKartEnv, self).init(mk_config['ROM_NAME'])
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 59, in init
self.xvfb_process, self.emulator_process = self._start_emulator(rom_name=rom_name)
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 192, in _start_emulator
initial_disp = os.environ["DISPLAY"]
File "/Users/dariomantegazza/anaconda3/envs/gym-mupen64plus/lib/python2.7/UserDict.py", line 40, in getitem
raise KeyError(key)
KeyError: 'DISPLAY'
Close called!
I've tried to debug it but I've found nothing helpful.

[bzier]

Hi @Mardox91,

Thanks for the detailed information; that's very helpful. The error you are getting is related to the attempted use of the environment variable DISPLAY. It turns out that Mac OS doesn't use this (see here). The answer on that page suggests using XQuartz on Mac. However at least one other person has tried this and bumped into issues (see #15). Unfortunately I don't have a Mac, or access to one, to test with.

My first suggestion would be to experiment with XQuartz, although I know nothing about it and am not even sure it will work. The environment currently uses xvfb by default and that may or may not work (see here).

Alternatively, you could disable xvfb in the config file and attempt to run straight on your main display. As mentioned in the README, this is a little more cumbersome because you have to ensure the emulator is the topmost window and statically positioned for the duration of execution. Also, because this failure is occurring when reading the DISPLAY variable, you will also need to comment out a couple lines (at least L192 & L234 here).

It's worth mentioning, since you brought up wxPython, that I recently replaced that dependency with mss. I merged that pull request around November 13th or so. If you haven't done a pull lately, I would suggest getting the updated code. If you're also working on setting up TensorKart, that still does have a dependency on wxPython for now, but it may be removed in the future.

One last thing for now: I am also going to be adding Docker support soon (no promises on a date). This should make the project completely platform independent, and make the setup / getting started process pretty negligible. That's my hope anyway. I've still got some details to sort out and I need to do a write-up on its use, but I do have it working. So keep an eye out for that commit soon.

Ok, so all that said, please keep me updated with the direction you take and if you're able to make any progress. Also, let me know if you have any other questions or things I can help with along the way.

[bzier]

BTW, I just noticed that I forgot to update the environment.yml file when I changed from wxPython to mss. Thanks for pointing that out (whether you intended to or not). I will get that fixed.

[Dario-Mantegazza]

Hi,
first of all thank you for your immediate response.
Me and my colleague (we are doing a university project about Reinforcement Learning) have implemented the line code commenting, installed the xQuartz (from here) but we've got some other issue.
Running the test code the emulator would run and give us multiple errors.
The last one being

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/utils/closer.py", line 67, in close
closeable.close()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/core.py", line 166, in close
self._close()
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 145, in _close
self._stop_controller_server()
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 158, in _stop_controller_server
if self.controller_server is not None:
AttributeError: 'MarioKartLuigiRacewayEnv' object has no attribute 'controller_server'
Error in sys.exitfunc:
Traceback (most recent call last):
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/utils/closer.py", line 67, in close
closeable.close()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/core.py", line 166, in close
self._close()
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 145, in _close
self._stop_controller_server()
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 158, in _stop_controller_server
if self.controller_server is not None:
AttributeError: 'MarioKartLuigiRacewayEnv' object has no attribute 'controller_server'

We have solved for now this issue with the introduction of hasattr(self, 'controller_server') at line 158 of mupen64plus_env.py.
After this "correction" we can launch the emulator, that would run until the automatic race selector starts the race and as soon as lakitu appears with the traffic light the emulator would crash with the following error:

Traceback (most recent call last):
File "test.py", line 6, in
env.render()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/core.py", line 150, in render
return self._render(mode=mode, close=close)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/core.py", line 286, in _render
return self.env.render(mode, close)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/core.py", line 150, in render
return self._render(mode=mode, close=close)
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 136, in _render
from gym.envs.classic_control import rendering
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/envs/classic_control/rendering.py", line 23, in
from pyglet.gl import *
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/gl/init.py", line 100, in
from pyglet.gl.lib import GLException
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/gl/lib.py", line 141, in
from pyglet.gl.lib_agl import link_GL, link_GLU, link_AGL
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/gl/lib_agl.py", line 43, in
import pyglet.lib
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/lib.py", line 61, in
script_path = pyglet.resource.get_script_home()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/init.py", line 384, in getattr
import(import_name)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/resource.py", line 88, in
standard_library.install_aliases()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/future/standard_library/init.py", line 483, in install_aliases
import test
File "/Users/dariomantegazza/Desktop/PROGETTONE/test.py", line 4, in
env = gym.make('Mario-Kart-Luigi-Raceway-v0')
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/envs/registration.py", line 164, in make
return registry.make(id)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/envs/registration.py", line 122, in make
env = spec.make()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/envs/registration.py", line 89, in make
env = cls(**self._kwargs)
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/MarioKart64/mario_kart_env.py", line 37, in init
super(MarioKartEnv, self).init(mk_config['ROM_NAME'])
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 58, in init
self.controller_server, self.controller_server_thread = self._start_controller_server()
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 149, in _start_controller_server
config['ACTION_TIMEOUT'])
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 303, in init
super(ControllerHTTPServer, self).init(server_address, self.ControllerRequestHandler)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/SocketServer.py", line 417, in init
self.server_bind()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/SocketServer.py", line 431, in server_bind
self.socket.bind(self.server_address)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 48] Address already in use
Close called!
Close called!

Now we are trying to fix this issue. We will keep you updated.

[bzier]

Thanks for the update. I'm glad to hear you are making some progress. I've seen that error, socket.error: [Errno 48] Address already in use, typically when a process crashes or doesn't shutdown properly. I hope to address that at some point (kind of goes along with #3 and #4). I would suggest making sure you close/kill any running Python process (and children), and see if that makes a difference.

I'm at work right now, so can't look closely into the rest of this until later, but will try to see if I have any other suggestions this evening.

[Dario-Mantegazza]

Thank you for your response
We have tried to run the whole project on ubuntu but we get the same error.
Also running the test.py after fresh restart gives the same error. Personally I don't think it's related to some broken pipe or badly killed processes.
We are stuck and not sure on how to move from here.
I'll add the whole output that we get

('ControllerHTTPServer started on port ', 8082)
Starting emulator with comand: ['mupen64plus', '--resolution', '640x480', '--audio', 'dummy', '--input', '/usr/local/lib/mupen64plus/mupen64plus-input-bot.so', '/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/ROMs/marioKart.n64']
Calling mss.mss() with DISPLAY: /private/tmp/com.apple.launchd.UQJbDZAVRL/org.macosforge.xquartz:0
Frame 10: [0, 0, 1, 0, 0]
Frame 80: [0, 0, 1, 0, 0]
Frame 120: [0, 0, 1, 0, 0]
Frame 125: [0, -80, 0, 0, 0]
Frame 130: [0, 0, 1, 0, 0]
Frame 132: [0, 0, 1, 0, 0]
Frame 134: [0, 0, 1, 0, 0]
Player row: 0
Player col: 0
Frame 160: [0, 0, 1, 0, 0]
Frame 162: [0, 0, 1, 0, 0]
Map series: 0
Map choice: 0
Frame 202: [0, 0, 1, 0, 0]
Frame 230: [0, 0, 1, 0, 0]
Frame 232: [0, 0, 1, 0, 0]
Reset called!
Close called!Traceback (most recent call last):

File "/Users/dariomantegazza/Desktop/nervoso/test.py", line 6, in
env.render()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/core.py", line 150, in render
return self._render(mode=mode, close=close)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/core.py", line 286, in _render
return self.env.render(mode, close)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/core.py", line 150, in render
return self._render(mode=mode, close=close)
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 136, in _render
from gym.envs.classic_control import rendering
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/envs/classic_control/rendering.py", line 23, in
from pyglet.gl import *
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/gl/init.py", line 100, in
from pyglet.gl.lib import GLException
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/gl/lib.py", line 141, in
from pyglet.gl.lib_agl import link_GL, link_GLU, link_AGL
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/gl/lib_agl.py", line 43, in
import pyglet.lib
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/lib.py", line 61, in
script_path = pyglet.resource.get_script_home()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/init.py", line 384, in getattr
import(import_name)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/pyglet/resource.py", line 88, in
standard_library.install_aliases()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/future/standard_library/init.py", line 483, in install_aliases
import test
File "/Users/dariomantegazza/Desktop/nervoso/test.py", line 4, in
env = gym.make('Mario-Kart-Luigi-Raceway-v0')
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/envs/registration.py", line 164, in make
return registry.make(id)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/envs/registration.py", line 122, in make
env = spec.make()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/site-packages/gym/envs/registration.py", line 89, in make
env = cls(**self._kwargs)
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/MarioKart64/mario_kart_env.py", line 37, in init
super(MarioKartEnv, self).init(mk_config['ROM_NAME'])
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 58, in init
self.controller_server, self.controller_server_thread = self._start_controller_server()
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 149, in _start_controller_server
config['ACTION_TIMEOUT'])
File "/Users/dariomantegazza/Desktop/PROGETTONE/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 303, in init
super(ControllerHTTPServer, self).init(server_address, self.ControllerRequestHandler)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/SocketServer.py", line 417, in init
self.server_bind()
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/SocketServer.py", line 431, in server_bind
self.socket.bind(self.server_address)
File "/Users/dariomantegazza/anaconda3/envs/mirko/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 48] Address already in use
Close called!

Process finished with exit code 1

[Dario-Mantegazza]

So
We've found something.
The file that we used to do some testing was called test.py. This naming produced some erratic result when some of the lib tried to import test resulting in the code to be runned again and ending on using the same address.
So I think that is good to add in the Readme.md to not call the test file test.py.
Right know we are not sure if everything is working correctly because at the start of the race, another windows appears and seems black-ish and flickering.
Thank you for your help :)

[bzier]

Just to clarify, the latest output you included looks like it is still from the Mac, not from Ubuntu. Is that correct? If so, was the error message any different on Ubuntu?

The error is complaining that the address is in use when trying to start the controller http server. I wonder if you have some other process running on the machine using that port. The port is configured for 8082. Please verify if anything else may be listening on that port.

The port number is configurable in the config.yml file. However, the custom input driver is currently hard-coded to connect to 8082 here. It would be easiest to stop whatever process may be using that port, but if you need to change it, you will have to recompile the input driver.

---

Ok after writing all of that and reviewing the error stack trace again, I'm starting to think the issue may be different. It almost looks to me like there is a recursive loop going on. At the beginning, the call that is failing is env.render(). This occurs after the environment has been created (gym.make(...)). But about halfway through the stack trace, pyglet is doing something with the standard library, which is attempting to import test. This appears to be picking up your local test.py file, which is importing the gym and attempting to create the environment (again!). This second instance also attempts to start the controller server on the same port, but the first instance already has a hold of it, hence the error.

Two suggestions to move forward. First, simply rename the file to mkTest.py or something. Anything 'unique' should be fine. That should prevent the standard library import from incorrectly picking it up. Secondly, rather than saving a file and executing it, try starting an interactive python session and just copying/pasting that test code in. I think either of those approaches should work.

Please let me know if any of that doesn't make sense, or if it gets things working. If so, I'll have to make a note in the README about that test code. Really interesting behavior that I haven't seen before.

[bzier]

Wow, as I was writing that you guys found the same thing. Glad to hear that you fixed it.
Let me know if you encounter other issues, and what you find regarding that flickering black screen

[Dario-Mantegazza]

LoL Thank you.
Everything is working now.
We will let you know about our improvement on RL model.

[bzier]

Yes, please do keep me posted. I look forward to seeing any results you are able to achieve. Be aware that the currently implemented reward function does not seem to work well (in my experience). I'm working on changes to that, as well as changes to the architecture to more easily support variations to the reward function. You may discover that you need to make changes as well in order to get good agent behavior. If you encounter any other issues, or have ideas for improvements, feel free to open new issues and/or submit pull requests.

[bzier]

@Mardox91 It just occurred to me that the other window that pops up is likely from the call to env.render in the test code. If you guys did disable xvfb to get up and running on XQuartz, that's probably the case (I don't usually make calls to env.render when xvfb is disabled, just because the emulator window is already visible). This call is directly rendering the observation that the environment is providing to the agent at each step. If it is not showing the Mario Kart screen, either the render function is behaving improperly in XQuartz, or that is exactly what the agent would 'see' as well.

You may want to confirm the data that is coming back as an observation is a valid image from Mario Kart. Otherwise, the issue might be with the screenshot library mss not playing well with XQuartz. Again, not having used it, I can't say for sure.

Sorry I didn't consider this a little more the other day and I hope you guys haven't been stuck as a result.

[bzier]

@Mardox91 Sorry to keep spamming this issue, but I also wanted to give you a heads-up that I'm going to merge in PR #32. I made some changes to the way Xvfb is started up. The changes I made won't likely help with the XQuartz issues described above, but the lines changed are near the lines that you guys had to change to get things working. You may encounter a merge conflict as a result. It shouldn't be too bad, but I didn't want it to catch you by surprise.

[Dario-Mantegazza]

Hi @bzier,
In advance, sorry for the long response :)

Even with xQuartz we had some issues with what the agent would see. The popup screen had some interlacing problems(e.g. on odd rows we had the correct screen and on even we had a zoomed version of the view) we've tried to solve this problem by showing only the odd rows of the view.
We had some problems with the x,y coordinates that, for some unknown reason, we had to multiply by 2(both x and y).
In the end we got stuck at the reward function.

With that 'problem' in mind I've ended up around the web in in search of similar projects and in the end I've come up with an idea:
Use the RAM value of the distance for the reward function.
This reward function will work without additional problems (e.g. segmentation or computation overload).

On a youtube video comments I've found that with another emulator BizHawk there is an already present RAM analizer.
So given the short amount of time that we can spend on this project, right now we are working on that emulator even if in mupen64plus there is the ability to write python plugins to read the RAM value.(everything is talked in this video's comments https://www.youtube.com/watch?v=tmltm0ZHkHw)

I think that given some of the link provided above and more time it is possible to replicate this method on mupen64plus.
After the course(I' doing this project for an university exam) end I will publish the project code and link it here.

[kevinhughes27]

Thanks for sharing! I'm excited to see what you come up with 😄

[bzier]

@Mardox91 Thank you for the update. I am curious to see your results as well.

I'm currently working through changes to the reward function as well, although only due to poor learning and not the graphical glitches you described. It looks like that BizHawk emulator is actually built on mupen64plus (see here). I have been considering some architectural changes for performance reasons and may look into what they've done as one reference.

Besides the changes from the environment/emulator standpoint, I'm really excited to see what kind of learning results you are able to achieve. If you write something up (for the university course or otherwise), I'd be interested in reading it.