Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow compiling with Rust code #3137

Merged
merged 61 commits into from
Jan 16, 2025
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
4ebc2d6
Squashed Rust impl
nyurik Jul 24, 2024
e5b038b
cmake
nyurik Aug 3, 2024
cad4ff4
cxx
nyurik Aug 4, 2024
dc0defe
wip
nyurik Aug 4, 2024
46ac304
add armv7-linux-androideabi
nyurik Aug 5, 2024
0aacaaf
add rust to qt docker build
nyurik Aug 5, 2024
2847d27
fix
nyurik Aug 5, 2024
4752a7d
fix
nyurik Aug 5, 2024
cc8eb72
build all qt
nyurik Aug 5, 2024
e07f5c8
build all qt
nyurik Aug 5, 2024
0afbe40
wip
nyurik Aug 6, 2024
c157da8
wip
nyurik Aug 7, 2024
61f7d3f
wip
nyurik Aug 7, 2024
5e404f4
wip
nyurik Aug 7, 2024
f712840
Merge branch 'main' into rust-css-color
louwers Jan 10, 2025
f2b7b7b
Add Bazel support
louwers Jan 10, 2025
d3836b6
Add back license
louwers Jan 10, 2025
90ba3ef
Cleanup unused code
louwers Jan 10, 2025
6f618ec
Revert some files
louwers Jan 10, 2025
729dafb
CI fixes
louwers Jan 10, 2025
c7aceda
Remove README.md
louwers Jan 10, 2025
5fc141e
revert qt.cmake
louwers Jan 10, 2025
671a41f
revert README.md
louwers Jan 10, 2025
8ed4303
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 10, 2025
53a33c8
revert
louwers Jan 10, 2025
a5853d9
revert
louwers Jan 10, 2025
bda511d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 10, 2025
a2fb48c
PR comments
louwers Jan 10, 2025
a43b2e7
Merge branch 'rust-css-color' of github.com:nyurik/maplibre-native in…
louwers Jan 10, 2025
627c4a1
Merge branch 'rust-css-color' of github.com:louwers/maplibre-native i…
louwers Jan 10, 2025
fca3163
fix merge oops
louwers Jan 10, 2025
643de09
fix setup-java
louwers Jan 10, 2025
29a2bb2
PR comments, fix CMake
louwers Jan 10, 2025
bfafa2b
Add rustfmt to pre-commit hooks
louwers Jan 10, 2025
3e06aa3
change CMake file
louwers Jan 10, 2025
6d848bb
put docs under right heading
louwers Jan 10, 2025
f74e5bb
fix .pre-commit.yaml
louwers Jan 10, 2025
a6162cb
Update docker/Dockerfile
louwers Jan 13, 2025
2ebf02a
Update rustutils/cpp/generate.sh
louwers Jan 13, 2025
4d64771
Update src/mbgl/util/color.rs.cpp
louwers Jan 13, 2025
0101522
allow modules, remove generates sources, update license
louwers Jan 13, 2025
cc5bf2a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 13, 2025
5ae0610
remove rust from android-ci workflow
louwers Jan 14, 2025
96757f3
add extra_target_triples to configure Rust toolchain iOS
louwers Jan 14, 2025
11dafe6
add basic CI for Rust
louwers Jan 14, 2025
32b377f
Merge branch 'rust-css-color' of github.com:louwers/maplibre-native i…
louwers Jan 14, 2025
8d016f3
yaml oopsie
louwers Jan 14, 2025
7355d6b
formatting
louwers Jan 14, 2025
1f2aeb9
fix linux-ci
louwers Jan 14, 2025
32f3f1c
improve rust.md
louwers Jan 14, 2025
06b4bef
improve rustutils.cmake
louwers Jan 14, 2025
4228b6e
improve linux-ci
louwers Jan 14, 2025
2f608a6
use cargo fmt
louwers Jan 14, 2025
991f1e1
add newlines
louwers Jan 14, 2025
af5b67b
Fix issue in linux-ci
louwers Jan 14, 2025
6afef70
fix typo in linux-ci
louwers Jan 14, 2025
038fa16
Merge branch 'main' of github.com:maplibre/maplibre-native into rust-…
louwers Jan 15, 2025
014f2ba
Fix color.rs.cpp impl, add test
louwers Jan 15, 2025
04091e0
fix factor premultiplication
louwers Jan 16, 2025
e671d23
Add Rust toolchains for iOS, build cxxbridge with Bazel
louwers Jan 16, 2025
22df4de
fix rustutils/BUILD.bazel
louwers Jan 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions .github/workflows/android-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ jobs:
distribution: "temurin"
java-version: "17"

- name: Add aarch64-linux-android for Rust toolchian
run: rustup target add --toolchain stable-x86_64-unknown-linux-gnu aarch64-linux-android armv7-linux-androideabi i686-linux-android x86_64-linux-android
louwers marked this conversation as resolved.
Show resolved Hide resolved
louwers marked this conversation as resolved.
Show resolved Hide resolved
- name: Get CMake and Ninja
uses: lukka/get-cmake@latest
with:
Expand Down Expand Up @@ -234,6 +236,10 @@ jobs:
distribution: "temurin"
java-version: "17"

- name: Add aarch64-linux-android for Rust toolchian
if: ${{ github.ref == 'refs/heads/rust' || github.event.pull_request.base.ref == 'rust' }}
run: rustup target add --toolchain stable-x86_64-unknown-linux-gnu aarch64-linux-android armv7-linux-androideabi i686-linux-android x86_64-linux-android

- name: Get CMake and Ninja
uses: lukka/get-cmake@latest
with:
Expand Down Expand Up @@ -296,6 +302,11 @@ jobs:
submodules: recursive
fetch-depth: 0

- uses: actions/setup-java@v4
with:
distribution: "temurin"
java-version: "17"

- uses: hendrikmuhs/[email protected]
with:
key: ${{ github.job }}
Expand All @@ -308,10 +319,9 @@ jobs:
cmakeVersion: 3.24.1
ninjaVersion: latest

- uses: actions/setup-java@v4
with:
distribution: "temurin"
java-version: "17"
- name: Add aarch64-linux-android for Rust toolchian
if: ${{ github.ref == 'refs/heads/rust' || github.event.pull_request.base.ref == 'rust' }}
run: rustup target add --toolchain stable-x86_64-unknown-linux-gnu aarch64-linux-android armv7-linux-androideabi i686-linux-android x86_64-linux-android

- name: Build and Upload Render Test APKs (${{ matrix.flavor }})
uses: ./.github/actions/android-build-and-upload-render-test
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/linux-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,12 @@ jobs:
xvfb \
x11-xserver-utils

- name: Install cxxbridge-cmd (compile)
if: ${{ github.ref == 'refs/heads/rust' || github.event.pull_request.base.ref == 'rust' }}
uses: baptiste0928/cargo-install@v3
with:
crate: cxxbridge-cmd
louwers marked this conversation as resolved.
Show resolved Hide resolved

- name: Cache Bazel
uses: actions/cache@v4
with:
Expand Down
25 changes: 22 additions & 3 deletions BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
load("@aspect_rules_js//js:defs.bzl", "js_binary", "js_library", "js_run_binary")
load("@bazel_skylib//rules:common_settings.bzl", "string_flag")
load("@bazel_skylib//rules:common_settings.bzl", "string_flag", "bool_flag")
load("@npm//:defs.bzl", "npm_link_all_packages")
load(
"//bazel:core.bzl",
Expand Down Expand Up @@ -124,7 +124,6 @@ cc_library(
":mbgl-core-generated-private-artifacts",
":mbgl-core-generated-public-artifacts",
"//vendor:boost",
"//vendor:csscolorparser",
"//vendor:earcut.hpp",
"//vendor:eternal",
"//vendor:mapbox-base",
Expand All @@ -149,7 +148,14 @@ cc_library(
"//vendor:metal-cpp",
],
"//conditions:default": [],
}),
}) + select({
":rust": [
"//rustutils:rustutilslib",
],
"//conditions:default": [
"//vendor:csscolorparser",
]
})
)

genrule(
Expand Down Expand Up @@ -195,6 +201,19 @@ config_setting(
},
)

bool_flag(
name = "use_rust",
build_setting_default = False,
visibility = ["//visibility:public"],
)

config_setting(
name = "rust",
flag_values = {
"//:use_rust": "true",
},
)

exports_files(
[
"LICENSE.md",
Expand Down
21 changes: 14 additions & 7 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ option(MLN_LEGACY_RENDERER "Include the legacy rendering pathway" ON)
option(MLN_DRAWABLE_RENDERER "Include the drawable rendering pathway" OFF)
option(MLN_USE_UNORDERED_DENSE "Use ankerl dense containers for performance" ON)
option(MLN_USE_TRACY "Enable Tracy instrumentation" OFF)
option(MLN_USE_RUST "Use components in Rust" OFF)

if (MLN_WITH_CLANG_TIDY)
find_program(CLANG_TIDY_COMMAND NAMES clang-tidy)
Expand Down Expand Up @@ -940,7 +941,7 @@ list(APPEND SRC_FILES
${PROJECT_SOURCE_DIR}/src/mbgl/util/bounding_volumes.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/chrono.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/client_options.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/color.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/color$<IF:$<BOOL:${MLN_USE_RUST}>,.rs.cpp,.cpp>
${PROJECT_SOURCE_DIR}/src/mbgl/util/constants.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/convert.cpp
${PROJECT_SOURCE_DIR}/src/mbgl/util/event.cpp
Expand Down Expand Up @@ -1458,6 +1459,8 @@ include(${PROJECT_SOURCE_DIR}/vendor/vector-tile.cmake)
include(${PROJECT_SOURCE_DIR}/vendor/wagyu.cmake)
include(${PROJECT_SOURCE_DIR}/vendor/metal-cpp.cmake)

include(${PROJECT_SOURCE_DIR}/rustutils/rustutils.cmake)

target_link_libraries(
mbgl-core
PRIVATE
Expand All @@ -1468,7 +1471,6 @@ target_link_libraries(
Mapbox::Base::cheap-ruler-cpp
mbgl-compiler-options
mbgl-vendor-boost
mbgl-vendor-csscolorparser
mbgl-vendor-earcut.hpp
mbgl-vendor-eternal
mbgl-vendor-parsedate
Expand All @@ -1479,6 +1481,7 @@ target_link_libraries(
mbgl-vendor-vector-tile
mbgl-vendor-wagyu
$<$<BOOL:${MLN_WITH_METAL}>:mbgl-vendor-metal-cpp>
$<IF:$<BOOL:${MLN_USE_RUST}>,mbgl-rustutils,mbgl-vendor-csscolorparser>
PUBLIC
Mapbox::Base
Mapbox::Base::Extras::expected-lite
Expand All @@ -1490,9 +1493,8 @@ target_link_libraries(
unordered_dense
)

export(TARGETS
set(EXPORT_TARGETS
mbgl-core

mapbox-base
mapbox-base-cheap-ruler-cpp
mapbox-base-extras-expected-lite
Expand All @@ -1508,7 +1510,6 @@ export(TARGETS
mapbox-base-variant
mbgl-compiler-options
mbgl-vendor-boost
mbgl-vendor-csscolorparser
mbgl-vendor-earcut.hpp
mbgl-vendor-eternal
mbgl-vendor-parsedate
Expand All @@ -1520,10 +1521,16 @@ export(TARGETS
mbgl-vendor-wagyu
mbgl-vendor-metal-cpp
unordered_dense

FILE MapboxCoreTargets.cmake
)

if(MLN_USE_RUST)
list(APPEND EXPORT_TARGETS mbgl-rustutils rustutils)
else()
list(APPEND EXPORT_TARGETS mbgl-vendor-csscolorparser)
endif()

export(TARGETS ${EXPORT_TARGETS} FILE MapboxCoreTargets.cmake)

if(MLN_WITH_VULKAN)
include(${PROJECT_SOURCE_DIR}/vendor/vulkan.cmake)

Expand Down
23 changes: 22 additions & 1 deletion MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -59,4 +59,25 @@ darwin_config = use_repo_rule("//platform/darwin:bazel/darwin_config_repository_

darwin_config(
name = "darwin_config",
)
)

bazel_dep(name = "rules_rust", version = "0.56.0")

rust = use_extension("@rules_rust//rust:extensions.bzl", "rust")

rust.toolchain(
edition = "2021",
versions = ["1.80.0"]
)
use_repo(rust, "rust_toolchains")

register_toolchains("@rust_toolchains//:all")

crate = use_extension("@rules_rust//crate_universe:extensions.bzl", "crate")

crate.from_cargo(
name = "crates",
cargo_lockfile = "//rustutils:Cargo.lock",
manifests = ["//rustutils:Cargo.toml"],
)
use_repo(crate, "crates")
10 changes: 8 additions & 2 deletions bazel/core.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -584,7 +584,6 @@ MLN_CORE_SOURCE = [
"src/mbgl/util/bounding_volumes.cpp",
"src/mbgl/util/chrono.cpp",
"src/mbgl/util/client_options.cpp",
"src/mbgl/util/color.cpp",
"src/mbgl/util/constants.cpp",
"src/mbgl/util/convert.cpp",
"src/mbgl/util/event.cpp",
Expand Down Expand Up @@ -652,7 +651,14 @@ MLN_CORE_SOURCE = [
"src/mbgl/util/version.cpp",
"src/mbgl/util/version.hpp",
"src/mbgl/util/work_request.cpp",
]
] + select({
"//:rust": [
"src/mbgl/util/color.rs.cpp",
],
"//conditions:default": [
"src/mbgl/util/color.cpp",
]
})

MLN_CORE_HEADERS = [
"include/mbgl/gfx/context.hpp",
Expand Down
22 changes: 18 additions & 4 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Ubuntu version should match SWIFT version below
louwers marked this conversation as resolved.
Show resolved Hide resolved
louwers marked this conversation as resolved.
Show resolved Hide resolved
FROM ubuntu:22.04

# Install build tools and dependencies
Expand All @@ -23,20 +24,23 @@ RUN apt-get update \
ccache \
ninja-build \
pkg-config \
python3 \
python3-pip \
python-is-python3 \
clang-tidy \
&& : # end of the RUN cmd - easier to keep a colon at the end of the list, than to keep the backslashes in check

# This could also be `.../releases/latest/download/bazelisk-linux-amd64` for the latest version, but for predictability better hardcode it
# Detect if current CPU is x64 or ARM64 and download the appropriate binary
RUN if [ "$(uname -m)" = "aarch64" ]; then \
RUN echo "Download and install Bazel" \
&& if [ "$(uname -m)" = "aarch64" ]; then \
curl -fsSL https://github.com/bazelbuild/bazelisk/releases/download/v1.20.0/bazelisk-linux-arm64 -o /usr/local/bin/bazel ;\
else \
curl -fsSL https://github.com/bazelbuild/bazelisk/releases/download/v1.20.0/bazelisk-linux-amd64 -o /usr/local/bin/bazel ;\
fi \
&& chmod +x /usr/local/bin/bazel \
&& :

WORKDIR /app

ARG USERNAME=user
ARG USER_UID=1000
Expand All @@ -50,11 +54,21 @@ RUN groupadd --force --gid $USER_GID $USERNAME \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 0440 /etc/sudoers.d/$USERNAME

# This allows users to `docker run` without specifying -u and -g
USER $USERNAME

RUN pip install pre-commit

ENV RUSTUP_HOME=/home/$USERNAME/.cache/.rustup \
CARGO_HOME=/home/$USERNAME/.cache/.cargo \
PATH=/home/$USERNAME/.cache/.cargo/bin:$PATH

# As the very last step, copy the startup script
USER root
COPY startup.sh /usr/local/bin/startup.sh
RUN chmod +x /usr/local/bin/startup.sh

# This allows users to `docker run` without specifying -u and -g
USER $USERNAME

WORKDIR /app
ENTRYPOINT ["/usr/local/bin/startup.sh"]
CMD ["bash"]
42 changes: 40 additions & 2 deletions docker/startup.sh
Original file line number Diff line number Diff line change
@@ -1,13 +1,51 @@
#!/bin/sh

if [ ! -d /app/.github ] || [ ! -d /home/user/.cache ]; then
if [ ! -d /app/.github ] || [ ! -d ~/.cache ]; then
echo " "
echo "ERROR: Docker container was not started properly."
echo " From the root of this repo, run the following command."
echo " You may add any command to perform in the container at the end of this command."
echo " "
echo ' docker run --rm -it -v "$PWD:/app/" -v "$PWD/docker/.cache:/home/user/.cache" maplibre-native-image'
echo " docker run --rm -it -v \"$PWD:/app/\" -v \"$PWD/docker/.cache:/home/$USERNAME/.cache\" maplibre-native-image"
louwers marked this conversation as resolved.
Show resolved Hide resolved
exit 1
fi

export PATH="$PATH:~/.local/bin/"


# Work in progress: install and configure Swift and pre-commit
# Detect if current CPU is x64 or ARM64 and download the appropriate binary
#RUN echo "Download and install SWIFT" \
# && if [ "$(uname -m)" = "aarch64" ]; then \
# curl -fsSL https://download.swift.org/swift-5.10.1-release/ubuntu2204-aarch64/swift-5.10.1-RELEASE/swift-5.10.1-RELEASE-ubuntu22.04-aarch64.tar.gz \
# -o /tmp/swift.tar.gz ;\
# else \
# curl -fsSL https://download.swift.org/swift-5.10.1-release/ubuntu2204/swift-5.10.1-RELEASE/swift-5.10.1-RELEASE-ubuntu22.04.tar.gz \
# -o /tmp/swift.tar.gz ;\
# fi \
# && tar -xzf /tmp/swift.tar.gz -C / --strip-components=1 \
# && rm /tmp/swift.tar.gz \
# && :
#if [ ! -f "/app/.git/hooks/pre-commit" ]; then
# echo "Configuring pre-commit git hooks by creating a .git/hooks/pre-commit file..."
# ~/.local/bin/pre-commit install
#fi



if [ ! -f "$CARGO_HOME/env" ]; then
echo "Downloading and installing Rust..."
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
fi
. "$CARGO_HOME/env"



if ! command -v cxxbridge > /dev/null; then
echo "Installing cxxbridge..."
cargo install cxxbridge-cmd
fi



exec "$@"
2 changes: 2 additions & 0 deletions docs/mdbook/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,5 @@

- [Profiling applications that use MapLibre Native](./profiling/README.md)
- [Tracy profiling](./profiling/tracy-profiling.md)

- [Rust](./rust.md)
27 changes: 27 additions & 0 deletions docs/mdbook/src/rust.md
louwers marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Rust

We have added experimental support for intergrating Rust code into the source tree.

## Rust Bridge

The Rust bridge lives in `rustutils`. To regenerate the C++ bridge run the following script from the root of the repository.

```
rustutils/cpp/generate.sh
```

Check in the generated files under `rustutils/cpp`.

We might intergrate this generation process into the build, but the tools to do so are experimental and immature as of January 2025.

## Building

You need to have the correct Rust toolchain(s) installed.
louwers marked this conversation as resolved.
Show resolved Hide resolved

### CMake

Set `MLN_USE_RUST` to `ON`.
louwers marked this conversation as resolved.
Show resolved Hide resolved

### Bazel

Pass `--//:use_rust` to Bazel.
1 change: 1 addition & 0 deletions rustutils/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/target
Loading
Loading