envs robotics - touch sensors - Shadow Hand

[ndrwmlnk]

@matthiasplappert
We (@ndrwmlnk @llach) covered the hand model with 92 touch sensors and extended the HandManipulate{Block, Egg, Pen}-v0 environments to HandManipulate{Block, Egg, Pen}TouchSensors-v0. Now, an observation additionally contains 92 touch values (0 - no touch, 1 - touch detected). Sensory information allows faster and better convergence (preliminary results).

Test the environments with pre-trained weights (currently in training):
https://rebrand.ly/TouchSensors
python baselines/her/experiment/play.py HandManipulateBlockTouchSensors.pkl
python baselines/her/experiment/play.py HandManipulateEggTouchSensors.pkl
python baselines/her/experiment/play.py HandManipulatePenTouchSensors.pkl

Touch sensor sites are tailored to physical geoms (robot0:DC_Hand), thus may visually overlap with meshes (robot0:D_Vizual). Active touch sensors are highlighted in red, inactive sensors - in green:
https://rebrand.ly/TouchSensors
sensors_cube_geoms.mp4
sensors_cube_mesh.mp4
sensors_cube.mp4
sensors.mp4

[matthiasplappert]

Thanks for your PR! I'm excited to get this merged and will work with @pzhokhov to figure out the best next steps.

[matthiasplappert]

Thanks again for this PR! I'm excited to get it merged soon.

I did an initial review and left some comments. Please address them and then I'll take another look. This PR will automatically update when you push to your branch, so just do that and let me know once ready.

[pzhokhov]

my 2 cents here are the following - a couple of tests of the new environments are failing (tried running manually, unfortunately, those won't show in travis because this is an external PR and as such our mujoco key is not injected => no tests of mujoco-based envs are run). Please run them manually (via pytest . with MUJOCO_KEY env variable set up and make sure 1) tests pass and 2) test count is 359 (that's how much it counted for me when I tried cloning the PR branch and running tests there). If there are some failures that seem unrelated to your changes, let's work together and figure out what's wrong

[ndrwmlnk]

@matthiasplappert @pzhokhov
Thanks for reviewing the code and tips with pytest .

I improved the code and fixed requested changes. 376 tests passed without errors. The number of tests exceeds 359, probably because of the new touch-sensory robotic environments.
The reason for the previous pytest error was the length of the observation vector:

gym/gym/envs/tests/test_envs.py

Line 23 in 49cd480

assert ob_space.contains(ob), 'Reset observation: {!r} not in space'.format(ob)

I added obs to ManipulateTouchSensorsEnv.init here:
https://github.com/ndrwmlnk/gym/blob/0d71b27103cd434173eb24ca50a99b3968866cfd/gym/envs/robotics/hand/manipulate_touch_sensors.py#L59
which now overrides obs from RobotEnv.init here (without touch values):

gym/gym/envs/robotics/robot_env.py

Line 38 in 49cd480

obs = self._get_obs()

Please let me know if I can make any further improvements.
Check pre-trained weights and videos here: https://rebrand.ly/TouchSensors

[pzhokhov]

@matthiasplappert @ndrwmlnk confirmed - tests are passing; if @matthiasplappert is happy, I am happy :)

[pzhokhov]

here's a successful travis build with the changes: https://travis-ci.org/openai/gym/jobs/489766042 (pay the "All checks have failed" no heed, I messed something up with github webhook when trying to set a build with the changes up)

[matthiasplappert]

Will take another pass tomorrow (crazy busy week so far) and then we can hopefully get this merged. Thanks everyone for helping!

[matthiasplappert]

LGTM now. Also verified that this works properly when building for a PyPI release and that all necessary assets are included. Merging this now.

[matthiasplappert]

And thanks for working with us on this PR; happy that we got it merged :)