Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pickle tests #53

Merged
merged 34 commits into from
Mar 31, 2023
Merged
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
7f90eb8
Add multi-agent dm control dockerfile and workflow
elliottower Mar 27, 2023
b30e99a
Fix typo in dm control multiagent workflow
elliottower Mar 27, 2023
6a0e2f2
Merge remote-tracking branch 'upstream/HEAD' into dm-lab-ci
elliottower Mar 28, 2023
9e3dc4e
Add dm-lab dockerfile and workflow
elliottower Mar 28, 2023
e8f9d87
Fix typo in dm_lab dockerfile
elliottower Mar 28, 2023
ae08a50
Add shimmy[dm-lab] pip installation to match other envs
elliottower Mar 28, 2023
1bc5a35
Add pickling tests for meltingpot, openspiel, bsuite, EzPickle for op…
elliottower Mar 28, 2023
2e97a55
Add initial pickle test to all third party environments (besides gym)
elliottower Mar 29, 2023
9f0760d
Merge branch 'main' into pickle-tests
elliottower Mar 29, 2023
f0afeef
Update PZ version after 1.22.4 yank
elliottower Mar 29, 2023
706db5f
Add importorskip for dm_lab so main tests don't fail
elliottower Mar 29, 2023
dc9df9c
Try old import deepmind_lab inside of test_check_env()
elliottower Mar 29, 2023
5917bd5
Add dm-env requirement to dm-lab dockerfile (fix CI error)
elliottower Mar 30, 2023
55e13a7
Fix typo in multiagent dm control test
elliottower Mar 30, 2023
29a0e4f
Update dm-lab tests to correct action type (from int to dict)
elliottower Mar 30, 2023
86fc8bc
Fix dm control multiagent init error (recursion limit)
elliottower Mar 30, 2023
efac866
Add all dm-lab levels to test, comment out obs test (not matching)
elliottower Mar 30, 2023
45de236
Attempt to fix dm-lab seeding, fix pickling test typo
elliottower Mar 30, 2023
6be6af8
Fix typo in dm lab test
elliottower Mar 30, 2023
e6a1c88
Fix meltingpot isort issues (ignore files, works locally just not in CI)
elliottower Mar 30, 2023
ad564f8
Fix dm control to take 10x less time for seed testing (1+hrs currently)
elliottower Mar 30, 2023
320f498
Fix typo in dm lab test
elliottower Mar 30, 2023
4992a4c
Make seed warning a print statement so execution doesn't stop during …
elliottower Mar 30, 2023
084d9b2
Skip dm lab tests
elliottower Mar 30, 2023
ef0f81a
Switch dm lab tests to do lt_chasm (env used in official examples)
elliottower Mar 30, 2023
246c8e1
Skip dm lab tests again due to erros
elliottower Mar 30, 2023
9413b02
Fix typo in install script
elliottower Mar 30, 2023
60f7482
Change dm_control_multi_agent test skip reason (weakref can't be pick…
elliottower Mar 30, 2023
0240502
Fix typo in dm control test
elliottower Mar 30, 2023
492a364
Fix typo in dm control mutliagent test
elliottower Mar 30, 2023
aa0bb38
Fix typo in dm control multiagent test
elliottower Mar 30, 2023
746a823
Remove isort ignore and don't run local pre-commit hooks
elliottower Mar 31, 2023
ca39529
Remove repeated noqa ignores for file-wide ignores, remove extra imports
elliottower Mar 31, 2023
cce032f
Fix pre-commit in meltingpot test
elliottower Mar 31, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add dm-lab dockerfile and workflow
  • Loading branch information
elliottower committed Mar 28, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 9e3dc4e12b20de10b67fad2e7fb5db489e6fcee1
10 changes: 9 additions & 1 deletion .github/workflows/optional-install-test.yml
Original file line number Diff line number Diff line change
@@ -10,7 +10,15 @@ jobs:
steps:
- uses: actions/checkout@v2

# dm-control
# dm-lab
- run: |
docker build -f bin/dm_lab.Dockerfile \
--build-arg PYTHON_VERSION='3.10' \
--tag shimmy-dm_lab-docker .
- name: Run dm-lab tests
run: docker run shimmy-dm_lab-docker pytest tests/test_dm_lab.py

# dm-control-multi-agent
- run: |
docker build -f bin/dm_control_multiagent.Dockerfile \
--build-arg PYTHON_VERSION='3.10' \
54 changes: 54 additions & 0 deletions bin/dm_lab.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# A Dockerfile that sets up a full shimmy install with test dependencies
ARG PYTHON_VERSION=3.9
FROM python:$PYTHON_VERSION

SHELL ["/bin/bash", "-o", "pipefail", "-c"]

# Install Shimmy requirements
RUN apt-get -y update \
&& apt-get install --no-install-recommends -y \
unzip \
libglu1-mesa-dev \
libgl1-mesa-dev \
libosmesa6-dev \
xvfb \
patchelf \
ffmpeg cmake \
&& apt-get autoremove -y \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

COPY . /usr/local/shimmy/
WORKDIR /usr/local/shimmy/

# Include Shimmy in Python path
ENV PYTHONPATH="$PYTHONPATH:/usr/local/shimmy/"

RUN pip install ".[testing]" --no-cache-dir

# Install DM lab requirements
RUN apt-get -y update \
&& apt-get install --no-install-recommends -y \
build-essential curl freeglut3 gettext git libffi-dev libglu1-mesa \
libglu1-mesa-dev libjpeg-dev liblua5.1-0-dev libosmesa6-dev \
libsdl2-dev lua5.1 pkg-config python-setuptools python3-dev \
software-properties-common unzip zip zlib1g-dev g++

# Install Bazel
RUN apt-get install -y apt-transport-https curl gnupg \
&& curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg \
&& mv bazel.gpg /etc/apt/trusted.gpg.d/ \
&& echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list \
&& apt-get update && apt-get install -y bazel

# Build DM lab
RUN git clone https://github.com/deepmind/lab.git \
&& cd lab \
&& echo 'build --cxxopt=-std=c++17' > .bazelrc \
&& bazel build -c opt //python/pip_package:build_pip_package \
&& ./bazel-bin/python/pip_package/build_pip_package /tmp/dmlab_pkg \
&& pip3 install --force-reinstall /tmp/dmlab_pkg/deepmind_lab-*.whl \
&& cd .. \
&& rm -rf lab

ENTRYPOINT ["/usr/local/shimmy/bin/docker_entrypoint"]
44 changes: 44 additions & 0 deletions scripts/install_dm_lab.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/bin/sh
set -eu

if [[ "$(uname -s)" == 'Linux' ]]; then
# Dependencies
apt-get update && apt-get install -y \
build-essential curl freeglut3 gettext git libffi-dev libglu1-mesa \
libglu1-mesa-dev libjpeg-dev liblua5.1-0-dev libosmesa6-dev \
libsdl2-dev lua5.1 pkg-config python-setuptools python3-dev \
software-properties-common unzip zip zlib1g-dev g++

# Bazel
apt-get install -y apt-transport-https curl gnupg
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg
mv bazel.gpg /etc/apt/trusted.gpg.d/
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list
apt-get update && apt-get install -y bazel

elif [[ "$(uname -s)" == 'Darwin' ]]; then
which -s brew
if [[ $? != 0 ]] ; then
echo "Homebrew not installed, run: `ruby -e '$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)'`"
exit
else
brew install bazel
fi
else
echo "Install script does not support windows, melting pot and dmlab2d must be installed manually and are not natively supported"
exit 1
fi


pip3 install numpy

# Build
if [ ! -d "lab" ]; then
git clone https://github.com/deepmind/lab.git
cd lab
echo 'build --cxxopt=-std=c++17' > .bazelrc
bazel build -c opt //python/pip_package:build_pip_package
./bazel-bin/python/pip_package/build_pip_package /tmp/dmlab_pkg
pip3 install --force-reinstall /tmp/dmlab_pkg/deepmind_lab-*.whl
cd ..
rm -rf lab
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
@@ -43,6 +43,7 @@ def get_version():
"h5py>=3.7.0",
"pettingzoo>=1.22.4",
],
"dm-lab": [],
"openspiel": ["open_spiel>=1.2", "pettingzoo>=1.22.4"],
"meltingpot": ["pettingzoo>=1.22.4"],
"bsuite": ["bsuite>=0.3.5"],