Add support for running Linux on ARM (e..g Raspberry Pi)

[NapalmCodes]

Goal

What do you want to achieve?

Use of realm in a self hosted asp.net console application (ultimately to be ran on mono via Raspberry Pi)

Expected Results

Realm to be compatible and not throw unsupported exceptions

Xamarin/Visual Studio version: Vs 2015

Which operating system version and device: Windows 10/raspbian

[kristiandupont]

Closing this as a duplicate of #509.

[AndyDentFree]

Re-opened after discussion that this is not the same as UWP

[nuseaki]

+1

[rupreetg]

+1. If Realm supports .NET Core, it'll be cross platform by default and will run on all 3 platforms seamlessly. Would love to use it in Desktop based application.

[AndyDentFree]

If Realm supports .NET Core, it'll be cross platform by default

No it won't - we have a native C++ engine (which, confusingly, we refer to as our Core ). There is nothing in .NET Core which makes that go away - we will still need to build that engine for any given platform.

[rupreetg]

Ah yeah, either you write that engine in .NET core which might not be a good idea or you build for each platform as you mentioned. Fair enough.

[haogg]

it will be better if java version can support PC.

[NapalmCodes]

To provide some clarity .Net Core will not resolve all problems this initial request of mine set out to resolve. Currently .Net Core is not supported on Raspberry Pi. Per meetups, it is my understanding MS intends to bring it back but there is no ETA. Also, while I am sure it is still Raspbian behind the scenes, the new OS for Raspberry Pi is branded PIXEL now. Just in case there is any confusion. I would still like to see this happen for desktop/self-hosting scenarios for what it is worth.

[AndyDentFree]

Regarding hacking on Pi - in case people hadn't realised, the entire core is also open source and a good start for any platform would be to get that ported, if it is a platform you don't think we will get to in a desired timeframe.

The other C++ layer on top of that is object store.

There are parts of the server and sync code which remain proprietary but a clean job of porting the above two would be a time-saver and major proof of seriousness from the Pi community ;-)

(wholly personal opinion from just another engineer)

[RealmBot]

➤ Brian Munkholm commented:

Unblocked. Realm is now compiled for RPi and a release is available for Realm-JS.

[Hackmodford]

Just had a need for this to work on a project published for linux-arm

[jsantos98]

Hello,

I have the same exact requirement. Not only for Raspbery pi, but for other SBC arm devices (arm and arm64)
I already have some experience on this matter, so I spend some time building the wrappers.

I'm currently on a leave so I don't have a raspberry with me, only a nano pi nano 2 (arm64).

Hope the following instructions might help others and hopefully, manage to become a starting point for this library to supply them out-of-the-box!

1. First of all, prepare the docker installation to create images with other architectures. For this, you need to enable experimental features to true: https://docs.docker.com/engine/reference/commandline/dockerd/#description
2. Now, create a buildx builder (Note: some may fail if this is the first time you enter them, so don't worry)

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
docker buildx rm builder
docker buildx create --name builder --driver docker-container --use
docker buildx inspect --bootstrap

3. Create a centos.arm64.Dockerfile file for the arm64 build (I reused the one available, but had to tweek some things to make it work in ARM64)

FROM centos:7

# Install EPEL & devtoolset
RUN yum install -y \
        epel-release \
        centos-release-scl-rh \
 && yum-config-manager --enable rhel-server-rhscl-7-rpms

RUN yum install -y \
        chrpath \
        devtoolset-9 \
        jq \
        libconfig-devel \
        openssh-clients \
        rh-git218 \
        zlib-devel \
 && yum clean all

ENV PATH /opt/cmake/bin:/opt/rh/rh-git218/root/usr/bin:/opt/rh/devtoolset-9/root/usr/bin:$PATH
ENV LD_LIBRARY_PATH /opt/rh/devtoolset-9/root/usr/lib64:/opt/rh/devtoolset-9/root/usr/lib:/opt/rh/devtoolset-9/root/usr/lib64/dyninst:/opt/rh/devtoolset-9/root/usr/lib/dyninst:/opt/rh/devtoolset-9/root/usr/lib64:/opt/rh/devtoolset-9/root/usr/lib

RUN mkdir -p /opt/cmake \
 && curl https://cmake.org/files/v3.19/cmake-3.19.3-Linux-aarch64.sh -o /cmake.sh \
 && sh /cmake.sh --prefix=/opt/cmake --skip-license \
 && rm /cmake.sh

RUN mkdir -p /etc/ssh && \
    echo "Host github.com\n\tStrictHostKeyChecking no\n" >> /etc/ssh/ssh_config && \
    ssh-keyscan github.com >> /etc/ssh/ssh_known_hosts

# For some reason, OpenSsl is not detected as in the x64 image (cmake doesn't recognize it). 
# Compiling from source does the trick and bypasses the problem, but it will take a LONG TIME!
## https://gist.github.com/fernandoaleman/5459173e24d59b45ae2cfc618e20fe06?permalink_comment_id=4160063
RUN yum -y update
RUN yum install -y make gcc perl-core pcre-devel wget zlib-devel
RUN wget https://ftp.openssl.org/source/openssl-1.1.1k.tar.gz
RUN tar -xzvf openssl-1.1.1k.tar.gz
WORKDIR /openssl-1.1.1k
RUN ls -la
RUN ./config --prefix=/usr --openssldir=/etc/ssl --libdir=lib no-shared zlib-dynamic

## !! Remove or change the "-j 8" argument to other number of concurrent jobs
## In my scenario, because the build is in an emulated environment, more jobs
## mean faster build
RUN make -j 8
RUN make install
RUN export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/lib64:$LD_LIBRARY_PATH

# Override the build.sh parameters to run within the docker image
ENV REALM_CMAKE_CONFIGURATION=Release
ENV EXTRA_CMAKE_ARGS=-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON

VOLUME /source
CMD ["/source/build.sh"]
#ENTRYPOINT ["/bin/bash"]

4. Follow the instructions from https://github.com/realm/realm-dotnet/tree/main/wrappers, with some small changes:

docker buildx build . -f centos.arm64.Dockerfile --platform linux/arm64 --build-arg ARCH=arm64 --load  -t realm-dotnet/wrappers/arm64
docker run -v path/to/wrappers:/source realm-dotnet/wrappers/arm64

Notes:

• This process, because will be an emulated environment using qemu will be slow as hell! It can take more than one hour!
• To speed things up, I've used parallel jobs -j 8 in the build process for both openssl and the wrapper itself inside the build.sh file: cmake --build . --target $REALM_CMAKE_INSTALL_TARGET --config $REALM_CMAKE_CONFIGURATION -j 8
• I've hardcoded some env parameters used by the build.sh script.
• I had to build the openssl from source because the version included in the yum command was not properly detected by cmake. Didn't have much time to investigate it further.
• The result file was successfully tested in real arm64 hardware using Armbian_22.05.4_Nanopineo2_bullseye_current_5.15.48.img with an application build in .NET 6 as PublishSingleFile, SelfContained and EnableCompressionInSingleFile flags cross build in a Windows x64 computer.

Now, regarding Arm 32 bits (default/recommended by raspbian until very recently)...

As I mentioned, during this month, I don't have with me a board. I've tried to repeat the same process for arm, but in the centos image, for some reason, I've kept getting segmentation fault errors.
I've tried to use ubuntu instead and it seemed to work, but, without the real hardware, I cannot test it out properly.
Nevertheless, here are the docker and commands I used:

Edit Just got my hands on a Pi4 and the binary generated using the last image was not working due to older libs in the raspbian versus the ones in ubuntu 22.04.

pi@pi32:~/temp $ ldd librealm-wrappers.so
./librealm-wrappers.so.before: /lib/arm-linux-gnueabihf/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by ./librealm-wrappers.so.before)
./librealm-wrappers.so.before: /lib/arm-linux-gnueabihf/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ./librealm-wrappers.so.before)
./librealm-wrappers.so.before: /lib/arm-linux-gnueabihf/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by ./librealm-wrappers.so.before)
./librealm-wrappers.so.before: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.32' not found (required by ./librealm-wrappers.so.before)
./librealm-wrappers.so.before: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.33' not found (required by ./librealm-wrappers.so.before)
./librealm-wrappers.so.before: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.34' not found (required by ./librealm-wrappers.so.before)
        linux-vdso.so.1 (0xbef03000)
        /usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so => /usr/lib/arm-linux-gnueabihf/libarmmem-v7l.so (0xb68e9000)
        libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0xb68c1000)
        libstdc++.so.6 => /lib/arm-linux-gnueabihf/libstdc++.so.6 (0xb6739000)
        libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb66ca000)
        libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb669d000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6549000)
        /lib/ld-linux-armhf.so.3 (0xb6f76000)

I've updated the Docker file using another version of ubuntu and this time it is working properly in the Pi hardware.

# to build image:
# docker buildx build . -f ubuntu.arm.Dockerfile --platform linux/arm/v7 --build-arg ARCH=arm/v7 --load  -t realm-dotnet/wrappers/arm

FROM ubuntu:18.04

RUN apt update && \
        apt install -y \
                build-essential \
                checkinstall \
                zlib1g-dev \
                wget \
                libssl-dev

# Compile and install cmake from source
RUN apt-get install -y wget
RUN wget https://github.com/Kitware/CMake/releases/download/v3.15.0/cmake-3.15.0.tar.gz
RUN tar -zxvf cmake-3.15.0.tar.gz
WORKDIR /cmake-3.15.0
RUN ./bootstrap
RUN make -j 4
RUN make install

# Install and use gcc 10 as default
Run apt-get install -y software-properties-common
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test
RUN apt-get update
RUN apt-get install -y gcc-10 g++-10

RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 700 --slave /usr/bin/g++ g++ /usr/bin/g++-7
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 800 --slave /usr/bin/g++ g++ /usr/bin/g++-10

# Override the build.sh parameters to run within the docker image
ENV REALM_CMAKE_CONFIGURATION=Release
ENV EXTRA_CMAKE_ARGS=-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON

VOLUME /source
CMD ["/source/build.sh"]
#ENTRYPOINT ["/bin/bash"]

Hope this helps!

Regards
João

[bwachter]

I just noticed this one, which probably also would benefit from this: realm/realm-dart#1064

Are there other projects which also would need that?

[nirinchev]

Supporting dart on aarch64 is not a high priority right now, though if any work done on .NET makes it easier, that will be nice.

[jvyden]

Hello! I saw this issue had the gathering-interest tag, so I'd like to note here that this is wanted. We'd like to introduce Docker support with ARM in our project but the one thing stopping us is the lack of linux-arm/linux-arm64 binaries for Realm.

[jsantos98]

Hello! I saw this issue had the gathering-interest tag, so I'd like to note here that this is wanted. We'd like to introduce Docker support with ARM in our project but the one thing stopping us is the lack of linux-arm/linux-arm64 binaries for Realm.

You can build the binaries yourself. Try to use the instructions I left a few comments above.

[jvyden]

Yeah, we've considered this but in the end we'd prefer to just have a solution that's officially supported by the Realm team. While it is possible its probably not viable for us to maintain if something goes wrong with it.