Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RL Python ManyLinux 2_28 #58

Merged
merged 2 commits into from
Oct 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build_deploy_latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ jobs:
- { image_name: manylinux2014_aarch64-build, dockerfile: manylinux2014_aarch64-build.Dockerfile, context: ./vowpal_wabbit/manylinux/ }
- { image_name: manylinux2014_x86_64-build, dockerfile: manylinux2014_x86_64-build.Dockerfile, context: ./vowpal_wabbit/manylinux/ }
- { image_name: rl-manylinux2010-build, dockerfile: rl-manylinux2010-build.Dockerfile, context: ./reinforcement_learning/manylinux-2010/ }
- { image_name: rl-manylinux-2_28-build, dockerfile: rl-manylinux-2_28-build.Dockerfile, context: ./reinforcement_learning/manylinux-2_28/ }
- { image_name: rl-ubuntu-1804, dockerfile: reinforcement_learning/ubuntu1804-build.Dockerfile, context: ./}
steps:
- uses: actions/checkout@master
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/build_deploy_tagged.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ jobs:
- { image_name: manylinux2014_aarch64-build, dockerfile: manylinux2014_aarch64-build.Dockerfile, context: ./vowpal_wabbit/manylinux/ }
- { image_name: manylinux2014_x86_64-build, dockerfile: manylinux2014_x86_64-build.Dockerfile, context: ./vowpal_wabbit/manylinux/ }
- { image_name: rl-manylinux2010-build, dockerfile: rl-manylinux2010-build.Dockerfile, context: ./reinforcement_learning/manylinux-2010/ }
- { image_name: rl-manylinux-2_28-build, dockerfile: rl-manylinux-2_28-build.Dockerfile, context: ./reinforcement_learning/manylinux-2_28/ }
- { image_name: rl-ubuntu-1804, dockerfile: reinforcement_learning/ubuntu1804-build.Dockerfile, context: ./}
steps:
- uses: actions/checkout@master
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/build_pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ jobs:
dockerfile: rl-manylinux2010-build.Dockerfile
context: ./reinforcement_learning/manylinux-2010/
changes_path_filter: reinforcement_learning/manylinux-2010/**
- image_name: rl-manylinux-2_28-build
dockerfile: rl-manylinux-2_28-build.Dockerfile
context: ./reinforcement_learning/manylinux-2_28/
changes_path_filter: reinforcement_learning/manylinux-2_28/**
- image_name: rl-ubuntu-1804
dockerfile: reinforcement_learning/ubuntu1804-build.Dockerfile
context: ./
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Tags on DockerHub correspond to [tags](https://github.com/VowpalWabbit/docker-im
| [CentOS 7.6.1810](./vowpal_wabbit/centos7_6_1810-build.Dockerfile) | [vowpalwabbit/centos7_6_1810-build](https://hub.docker.com/r/vowpalwabbit/centos7_6_1810-build) | CI |
| [RL Ubuntu 18.04](./reinforcement_learning/ubuntu1804-build.Dockerfile) | [vowpalwabbit/rl-ubuntu-1804](https://hub.docker.com/r/vowpalwabbit/rl-ubuntu-1804) | CI |
| [RL Python ManyLinux 2010](./reinforcement_learning/manylinux-2010/rlclientlib-manylinux2010-build.Dockerfile) | [vowpalwabbit/rlclientlib-manylinux2010-build](https://hub.docker.com/r/vowpalwabbit/rlclientlib-manylinux2010-build) | CI/Python packaging |

| [RL Python ManyLinux 2_28](./reinforcement_learning/manylinux-2_28/rlclientlib-manylinux-2_28-build.Dockerfile) | [vowpalwabbit/rlclientlib-manylinux-2_28-build](https://hub.docker.com/r/vowpalwabbit/rlclientlib-manylinux-2_28-build) | CI/Python packaging |

## Release steps

Expand Down
22 changes: 22 additions & 0 deletions reinforcement_learning/manylinux-2_28/build-boost.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/bash

# The only difference here to the other one is the addition of uuid and random

set -ex

BOOST_MAJOR=1
BOOST_MINOR=70
BOOST_PATCH=0

wget --no-check-certificate -O /tmp/boost.tar.gz "https://sourceforge.net/projects/boost/files/boost/${BOOST_MAJOR}.${BOOST_MINOR}.${BOOST_PATCH}/boost_${BOOST_MAJOR}_${BOOST_MINOR}_${BOOST_PATCH}.tar.gz/download"
mkdir /tmp/boost_src/
tar -xzf /tmp/boost.tar.gz -C /tmp/boost_src/
# This helps find then nested folder irrespective of verison.
BOOST_ROOT="$( find /tmp/boost_src/* -maxdepth 0 -type d -name 'boost*' )"
cd $BOOST_ROOT

./bootstrap.sh --with-libraries=program_options,system,thread,test,chrono,date_time,atomic,random,filesystem,regex
./b2 -j$(nproc) cxxflags="-fPIC" variant=release link=static install

cd /
rm -rf /tmp/boost_src/ /tmp/boost.tar.gz
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
FROM quay.io/pypa/manylinux_2_28_x86_64:latest

RUN yum install -y wget ninja-build && yum clean all

RUN version=3.18 && build=2 \
&& wget -qO- "https://cmake.org/files/v$version/cmake-$version.$build-Linux-x86_64.tar.gz" \
| tar --strip-components=1 -xz -C /usr/local

# Build and install OpenSSL from source with static libraries
RUN wget https://www.openssl.org/source/openssl-1.1.1l.tar.gz \
&& tar -xzf openssl-1.1.1l.tar.gz \
&& cd openssl-1.1.1l \
&& ./config no-shared --prefix=/usr/local/openssl --openssldir=/usr/local/openssl \
&& make -j$(nproc) \
&& make install \
&& cd .. \
&& rm -rf openssl-1.1.1l \
&& rm openssl-1.1.1l.tar.gz

RUN wget -O zlib.tar.gz 'https://zlib.net/fossils/zlib-1.2.8.tar.gz' \
&& tar xvzf zlib.tar.gz \
&& cd zlib-1.2.8 \
&& ./configure --static --archs=-fPIC \
&& make -j$(nproc) \
&& make install \
&& cd .. && rm -rf zlib*

# Install FlatBuffers 1.12.0
RUN version=1.12.0 && \
wget https://github.com/google/flatbuffers/archive/v$version.tar.gz \
&& tar -xzf v$version.tar.gz \
&& cd flatbuffers-$version \
&& mkdir build \
&& cd build \
&& cmake -G "Unix Makefiles" -DFLATBUFFERS_BUILD_TESTS=Off -DFLATBUFFERS_INSTALL=On -DCMAKE_BUILD_TYPE=Release -DFLATBUFFERS_BUILD_FLATHASH=Off .. \
&& make install -j$(nproc) \
&& cd ../../ \
&& rm -rf flatbuffers-$version \
&& rm v$version.tar.gz

COPY build-boost.sh build-boost.sh
RUN chmod +x build-boost.sh && ./build-boost.sh

# Checkout 2.10.18 version of cpprestsdk
RUN git clone https://github.com/Microsoft/cpprestsdk.git cpprestsdk \
&& cd cpprestsdk \
&& git checkout 122d09549201da5383321d870bed45ecb9e168c5 \
&& git submodule update --init --recursive \
&& cd Release \
&& mkdir build \
&& cd build \
&& cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DBoost_USE_STATIC_LIBS=On -DWERROR=OFF -DBUILD_TESTS=OFF -DBUILD_SAMPLES=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=On -DOPENSSL_ROOT_DIR=/usr/local/openssl -DOPENSSL_LIBRARIES=/usr/local/openssl/lib -DOPENSSL_INCLUDE_DIR=/usr/local/openssl/include \
&& make -j `nproc` \
&& make install \
&& cd ../../../ \
&& rm -rf cpprestsdk
Loading