Sync

.github/ISSUE_TEMPLATE/bug_report.md

@@ -7,7 +7,7 @@ assignees: ''
 
 ---
 
-<!-- We've collected some common issue solutions in https://docs.taichi.graphics/lang/articles/misc/install. Make sure you've check them out first. Hopefully they could address your problem. -->
+<!-- We've collected some common issue solutions in https://docs.taichi-lang.org/lang/articles/install. Make sure you've check them out first. Hopefully they could address your problem. -->
 
 **Describe the bug**
 A clear and concise description of what the bug is, ideally within 20 words.

.github/ISSUE_TEMPLATE/config.yml

@@ -2,7 +2,7 @@
 blank_issues_enabled: true
 contact_links:
   - name: Contributor Guideline
-    url: https://docs.taichi.graphics/lang/articles/contributor_guide
+    url: https://docs.taichi-lang.org/lang/articles/contributor_guide
     about: Please check this out if you'd like to contribute by opening a PR :)
   - name: Taichi Forum
     url: https://forum.taichi.graphics

.github/ISSUE_TEMPLATE/question.md

@@ -12,6 +12,6 @@ Before asking a question, please first consider:
 
 - Searching Google
 - Searching [existing issues](https://github.com/taichi-dev/taichi/issues)
-- Searching [Taichi Doc](https://docs.taichi.graphics/)
+- Searching [Taichi Doc](https://docs.taichi-lang.org/)
 - Searching [Taichi Forum](https://forum.taichi.graphics/)
 -->

.github/pull_request_template.md

@@ -4,11 +4,11 @@ Related issue = #
 Thank you for your contribution!
 
 If it is your first time contributing to Taichi, please read our Contributor Guidelines:
-  https://docs.taichi.graphics/lang/articles/contribution/contributor_guide
+  https://docs.taichi-lang.org/lang/articles/contributor_guide
 
 - Please always prepend your PR title with tags such as [CUDA], [Lang], [Doc], [Example]. For a complete list of valid PR tags, please check out https://github.com/taichi-dev/taichi/blob/master/misc/prtags.json.
 - Use upper-case tags (e.g., [Metal]) for PRs that change public APIs. Otherwise, please use lower-case tags (e.g., [metal]).
-- More details: https://docs.taichi.graphics/lang/articles/contribution/contributor_guide#pr-title-format-and-tags
+- More details: https://docs.taichi-lang.org/lang/articles/contributor_guide#pr-title-format-and-tags
 
 - Please fill in the issue number that this PR relates to.
 - If your PR fixes the issue **completely**, use the `close` or `fixes` prefix so that GitHub automatically closes the issue when the PR is merged. For example,

.github/workflows/scripts/check_clang_tidy.sh

@@ -5,8 +5,5 @@ CI_SETUP_CMAKE_ARGS=$1
 cd taichi
 python3 -m pip install -r requirements_dev.txt
 
-rm -rf build && mkdir build && cd build
-cmake $CI_SETUP_CMAKE_ARGS ..
-
-cd ..
-python3 ./scripts/run_clang_tidy.py $PWD/taichi -clang-tidy-binary clang-tidy-10 -checks=-*,performance-inefficient-string-concatenation,readability-identifier-naming -header-filter=$PWD/taichi -p $PWD/build -j2
+export CI_SETUP_CMAKE_ARGS
+python3 ./scripts/run_clang_tidy.py $PWD/taichi -clang-tidy-binary clang-tidy-10 -checks=-*,performance-inefficient-string-concatenation,readability-identifier-naming -header-filter=$PWD/taichi -j2

.github/workflows/scripts/unix_build.sh

@@ -49,7 +49,7 @@ setup_python() {
     python3 -m pip install -r requirements_dev.txt
 }
 
-build() {
+build_taichi_wheel() {
     git fetch origin master
     PROJECT_TAGS=""
     EXTRA_ARGS=""
@@ -69,12 +69,21 @@ build() {
     sccache -s
 }
 
+build_libtaichi_export() {
+    git fetch origin master
+    python3 setup.py build_ext
+}
+
 setup_sccache
 setup_python
-build
-cat "$SCCACHE_ERROR_LOG" || true
-NUM_WHL=$(ls dist/*.whl | wc -l)
-if [ $NUM_WHL -ne 1 ]; then echo "ERROR: created more than 1 whl." && exit 1; fi
+
+if [ "$EXPORT_CORE" == "1" ]; then
+    build_libtaichi_export
+else
+    build_taichi_wheel
+    NUM_WHL=$(ls dist/*.whl | wc -l)
+    if [ $NUM_WHL -ne 1 ]; then echo "ERROR: created more than 1 whl." && exit 1; fi
+fi
 
 chmod -R 777 "$SCCACHE_DIR"
 rm -f python/CHANGELOG.md

.github/workflows/scripts/unix_test.sh

@@ -22,11 +22,16 @@ python3 -m pip install dist/*.whl
 if [ -z "$GPU_TEST" ]; then
     python3 -m pip install -r requirements_test.txt
     python3 -m pip install "torch; python_version < '3.10'"
+    # Paddle's develop package doesn't support CI's MACOS machine at present
+    if [[ $OSTYPE == "linux-"* ]]; then
+        python3 -m pip install "paddlepaddle==0.0.0; python_version < '3.10'" -f https://www.paddlepaddle.org.cn/whl/linux/cpu-mkl/develop.html
+    fi
 else
     ## Only GPU machine uses system python.
     export PATH=$PATH:$HOME/.local/bin
     # pip will skip packages if already installed
     python3 -m pip install -r requirements_test.txt
+    # Import Paddle's develop GPU package will occur error `Illegal Instruction`.
 fi
 ti diagnose
 ti changelog
@@ -38,27 +43,32 @@ TI_LIB_DIR="$TI_PATH/_lib/runtime" ./build/taichi_cpp_tests
 if [ -z "$GPU_TEST" ]; then
     if [[ $PLATFORM == *"m1"* ]]; then
 	# Split per arch to avoid flaky test
-        python3 tests/run_tests.py -vr2 -t4 -k "not torch" -a cpu
+        python3 tests/run_tests.py -vr2 -t4 -k "not torch and not paddle" -a cpu
         # Run metal and vulkan separately so that they don't use M1 chip simultaneously.
-        python3 tests/run_tests.py -vr2 -t4 -k "not torch" -a vulkan
-        python3 tests/run_tests.py -vr2 -t2 -k "not torch" -a metal
+        python3 tests/run_tests.py -vr2 -t4 -k "not torch and not paddle" -a vulkan
+        python3 tests/run_tests.py -vr2 -t2 -k "not torch and not paddle" -a metal
         python3 tests/run_tests.py -vr2 -t1 -k "torch" -a "$TI_WANTED_ARCHS"
     else
-        python3 tests/run_tests.py -vr2 -t4 -a "$TI_WANTED_ARCHS"
+        # Fail fast, give priority to the error-prone tests
+        if [[ $OSTYPE == "linux-"* ]]; then
+            python3 tests/run_tests.py -vr2 -t1 -k "paddle" -a "$TI_WANTED_ARCHS"
+        fi
+        python3 tests/run_tests.py -vr2 -t4 -k "not paddle" -a "$TI_WANTED_ARCHS"
     fi
 else
     # Split per arch to increase parallelism for linux GPU tests
     if [[ $TI_WANTED_ARCHS == *"cuda"* ]]; then
-        python3 tests/run_tests.py -vr2 -t4 -k "not torch" -a cuda
+        python3 tests/run_tests.py -vr2 -t4 -k "not torch and not paddle" -a cuda
     fi
     if [[ $TI_WANTED_ARCHS == *"cpu"* ]]; then
-        python3 tests/run_tests.py -vr2 -t8 -k "not torch" -a cpu
+        python3 tests/run_tests.py -vr2 -t8 -k "not torch and not paddle" -a cpu
     fi
     if [[ $TI_WANTED_ARCHS == *"vulkan"* ]]; then
-        python3 tests/run_tests.py -vr2 -t8 -k "not torch" -a vulkan
+        python3 tests/run_tests.py -vr2 -t8 -k "not torch and not paddle" -a vulkan
     fi
     if [[ $TI_WANTED_ARCHS == *"opengl"* ]]; then
-        python3 tests/run_tests.py -vr2 -t4 -k "not torch" -a opengl
+        python3 tests/run_tests.py -vr2 -t4 -k "not torch and not paddle" -a opengl
     fi
     python3 tests/run_tests.py -vr2 -t1 -k "torch" -a "$TI_WANTED_ARCHS"
+    # Paddle's paddle.fluid.core.Tensor._ptr() is only available on develop branch, and CUDA version on linux will get error `Illegal Instruction`
 fi

.github/workflows/scripts/win_build.ps1

@@ -76,7 +76,6 @@ python -m venv venv
 . venv\Scripts\activate.ps1
 python -m pip install wheel
 python -m pip install -r requirements_dev.txt
-python -m pip install -r requirements_test.txt
 if (-not $?) { exit 1 }
 WriteInfo("Building Taichi")
 $env:TAICHI_CMAKE_ARGS += " -DCLANG_EXECUTABLE=$libsDir\\taichi_clang\\bin\\clang++.exe"

.github/workflows/scripts/win_test.ps1

@@ -9,20 +9,24 @@ pip install -r requirements_test.txt
 # TODO relax this when torch supports 3.10
 if ("$env:TI_WANTED_ARCHS".Contains("cuda")) {
     pip install "torch==1.10.1+cu113; python_version < '3.10'" -f https://download.pytorch.org/whl/cu113/torch_stable.html
+    pip install "paddlepaddle-gpu==0.0.0.post112; python_version < '3.10'" -f https://www.paddlepaddle.org.cn/whl/windows/gpu/develop.html
 } else {
     pip install "torch; python_version < '3.10'"
+    pip install "paddlepaddle==0.0.0; python_version < '3.10'" -f https://www.paddlepaddle.org.cn/whl/windows/cpu-mkl-avx/develop.html
 }
+# Fail fast, give priority to the error-prone tests
+python tests/run_tests.py -vr2 -t1 -k "paddle" -a "$env:TI_WANTED_ARCHS"
 if ("$env:TI_WANTED_ARCHS".Contains("cuda")) {
-  python tests/run_tests.py -vr2 -t4 -k "not torch" -a cuda
+  python tests/run_tests.py -vr2 -t4 -k "not torch and not paddle" -a cuda
   if (-not $?) { exit 1 }
 }
 if ("$env:TI_WANTED_ARCHS".Contains("cpu")) {
-  python tests/run_tests.py -vr2 -t6 -k "not torch" -a cpu
+  python tests/run_tests.py -vr2 -t6 -k "not torch and not paddle" -a cpu
   if (-not $?) { exit 1 }
 }
 if ("$env:TI_WANTED_ARCHS".Contains("opengl")) {
-  python tests/run_tests.py -vr2 -t4 -k "not torch" -a opengl
+  python tests/run_tests.py -vr2 -t4 -k "not torch and not paddle" -a opengl
   if (-not $?) { exit 1 }
 }
-python tests/run_tests.py -vr2 -t2 -k "torch" -a "$env:TI_WANTED_ARCHS"
+python tests/run_tests.py -vr2 -t1 -k "torch" -a "$env:TI_WANTED_ARCHS"
 if (-not $?) { exit 1 }

.github/workflows/testing.yml

@@ -415,3 +415,57 @@ jobs:
           PY: ${{ matrix.python }}
           PLATFORM: 'm1'
           TI_CI: 1
+
+  build_libtaichi_export:
+    name: Build libtaichi_export.so(GPU)
+    needs: check_files
+    runs-on: [self-hosted, cuda, vulkan, cn]
+    timeout-minutes: 60
+    strategy:
+      matrix:
+        include:
+          - os: ubuntu-latest
+            python: py39
+            with_cc: ON
+    permissions:
+      packages: read
+      contents: read
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          submodules: "recursive"
+
+      - name: Get sccache cache
+        uses: actions/cache@v2
+        with:
+          path: sccache_cache
+          key: sccache-linux-${{matrix.with_cc}}-${{ github.sha }}
+          restore-keys: |
+            sccache-linux-${{matrix.with_cc}}-
+
+      - name: Build For Desktop
+        run: |
+          if [[ ${{needs.check_files.outputs.run_job}} == false ]]; then
+            exit 0
+          fi
+          docker create --user dev --name taichi_build_desktop --gpus all -v /tmp/.X11-unix:/tmp/.X11-unix \
+            -e PY -e GPU_BUILD -e PROJECT_NAME -e TAICHI_CMAKE_ARGS -e DISPLAY -e EXPORT_CORE\
+            registry.taichigraphics.com/taichidev-ubuntu18.04:v0.3.0 \
+            /home/dev/taichi/.github/workflows/scripts/unix_build.sh
+          # A tarball is needed because sccache needs some permissions that only the file owner has.
+          # 1000 is the uid and gid of user "dev" in the container.
+          # If the uid or gid of the user inside the docker changes, please change the uid and gid in the following line.
+          tar -cf - ../${{ github.event.repository.name }} --mode u=+rwx,g=+rwx,o=+rwx --owner 1000 --group 1000 | docker cp - taichi_build_desktop:/home/dev/
+          docker start -a taichi_build_desktop
+        env:
+          PY: ${{ matrix.python }}
+          GPU_BUILD: ON
+          PROJECT_NAME: taichi
+          TAICHI_CMAKE_ARGS: -DTI_WITH_VULKAN:BOOL=ON -DTI_WITH_CUDA:BOOL=OFF -DTI_WITH_OPENGL:BOOL=OFF -DTI_WITH_LLVM:BOOL=OFF -DCMAKE_C_COMPILER_LAUNCHER=sccache -DCMAKE_CXX_COMPILER_LAUNCHER=sccache -DTI_EXPORT_CORE:BOOL=ON
+          EXPORT_CORE: 1
+          DISPLAY: :1
+
+      - name: clean docker container
+        if: always()
+        run: |
+          docker rm taichi_build_desktop -f

.gitignore

@@ -85,3 +85,4 @@ _build
 !docs/**/*.json
 imgui.ini
 /venv/
+/_skbuild/

.pre-commit-config.yaml

@@ -1,32 +1,34 @@
 ci:
   autoupdate_schedule: quarterly
+  autoupdate_commit_msg: '[misc] Update pre-commit hooks'
 
 exclude: ^((tests/python/test_exception|.*/examples/.*)\.py$|external/)
 repos:
   - repo: https://github.com/google/yapf
-    rev: v0.31.0
+    rev: v0.32.0
     hooks:
       - id: yapf
         additional_dependencies: [toml]
 
-  - repo: https://github.com/timothycrosley/isort
+  - repo: https://github.com/PyCQA/isort
     rev: 5.10.1
     hooks:
       - id: isort
 
   - repo: https://github.com/pre-commit/mirrors-clang-format
-    rev: v10.0.1
+    rev: v13.0.1
     hooks:
       - id: clang-format
+        exclude: .*\.json$
 
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.0.1
+    rev: v4.1.0
     hooks:
       - id: trailing-whitespace
       - id: end-of-file-fixer
 
   - repo: https://github.com/PyCQA/pylint
-    rev: v2.13.1
+    rev: v2.13.4
     hooks:
       - id: pylint
         args: ['-rn', '-sn']

CMakeLists.txt

@@ -2,12 +2,12 @@
 # The Taichi Programming Language
 #*********************************************************************
 
-cmake_minimum_required(VERSION 3.12)
+cmake_minimum_required(VERSION 3.13)
 
 project(taichi)
 
 if (NOT DEFINED TI_VERSION_MAJOR)
-    message(WARNING "It seems that you are running cmake manually, which may cause issues. Please use setup.py to build taichi from source, see https://docs.taichi.graphics/lang/articles/dev_install for more details.")
+    message(WARNING "It seems that you are running cmake manually, which may cause issues. Please use setup.py to build taichi from source, see https://docs.taichi-lang.org/lang/articles/dev_install for more details.")
     set(TI_VERSION_MAJOR 0)
     set(TI_VERSION_MINOR 0)
     set(TI_VERSION_PATCH 0)
@@ -49,7 +49,6 @@ else ()
 endif ()
 
 set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/build")
-set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/build")
 
 find_program(CCACHE_PROGRAM ccache)
 if(CCACHE_PROGRAM)
@@ -149,15 +148,7 @@ if (${CLANG_VERSION_MAJOR} VERSION_GREATER ${CLANG_HIGHEST_VERSION})
   endif()
 endif()
 
-# Build llvm-runtime for host arch and cuda (if available)
-foreach(arch IN LISTS HOST_ARCH CUDA_ARCH)
-  add_custom_target(
-      "generate_llvm_runtime_${arch}"
-      COMMAND ${CLANG_EXECUTABLE} ${CLANG_OSX_FLAGS} -c runtime.cpp -o "runtime_${arch}.bc" -fno-exceptions -emit-llvm -std=c++17 -D "ARCH_${arch}" -I ${PROJECT_SOURCE_DIR};
-      WORKING_DIRECTORY "${PROJECT_SOURCE_DIR}/taichi/runtime/llvm"
-  )
-  add_dependencies(${CORE_LIBRARY_NAME} "generate_llvm_runtime_${arch}")
-endforeach()
+add_subdirectory(taichi/runtime/llvm)
 
 configure_file(taichi/common/version.h.in ${CMAKE_SOURCE_DIR}/taichi/common/version.h)
 configure_file(taichi/common/commit_hash.h.in ${CMAKE_SOURCE_DIR}/taichi/common/commit_hash.h)

CONTRIBUTING.md

@@ -1,12 +1,12 @@
 # Contributing Guide
 
-Thank you for your interest in contributing to Taichi! Please check out the [Contribution Guidelines](https://docs.taichi.graphics/lang/articles/contributor_guide) for how to make a contribution.
+Thank you for your interest in contributing to Taichi! Please check out the [Contribution Guidelines](https://docs.taichi-lang.org/lang/articles/contributor_guide) for how to make a contribution.
 
 All contributors are expected to follow the [code of conduct](https://github.com/taichi-dev/taichi/blob/master/CODE_OF_CONDUCT.md).
 
 ## Developer installation
 
-Taichi is developed mainly in C++17 and Python3. Please check out the [Developer Installation](https://docs.taichi.graphics/lang/articles/dev_install) to build Taichi from source. Note that Taichi is LLVM-10.0.0 dependent and that we recommend installing [our pre-built LLVM libraries](https://docs.taichi.graphics/lang/articles/dev_install#install-llvm) for your platform.
+Taichi is developed mainly in C++17 and Python3. Please check out the [Developer Installation](https://docs.taichi-lang.org/lang/articles/dev_install) to build Taichi from source. Note that Taichi is LLVM-10.0.0 dependent and that we recommend installing [our pre-built LLVM libraries](https://docs.taichi-lang.org/lang/articles/dev_install#install-llvm) for your platform.
 
 ## Contribution opportunities
 

MANIFEST.in

@@ -1,8 +1,3 @@
-include MANIFEST.in
-include version.txt
-include python/*.txt
-include python/*.py
-include *.cfg
 include python/taichi/*.md
 recursive-include python/taichi/assets *
 recursive-include python/taichi/examples *.py

README.md

@@ -36,10 +36,10 @@ The language has broad applications spanning real-time physical simulation, numb
 ## Why Taichi?
 
 - Built around Python: Taichi shares almost the same syntax with Python, allowing you to write algorithms with minimal language barrier. It is also well integrated into the Python ecosystem, including NumPy and PyTorch.
-- Flexibility: Taichi provides a set of generic data containers known as *SNode* (/ˈsnoʊd/), an effective mechanism for composing hierarchical, multi-dimensional fields. This can cover many use patterns in numerical simulation (e.g. [spatially sparse computing](https://docs.taichi.graphics/lang/articles/sparse)).
+- Flexibility: Taichi provides a set of generic data containers known as *SNode* (/ˈsnoʊd/), an effective mechanism for composing hierarchical, multi-dimensional fields. This can cover many use patterns in numerical simulation (e.g. [spatially sparse computing](https://docs.taichi-lang.org/lang/articles/sparse)).
 - Performance: With the `@ti.kernel` decorator, Taichi's JIT compiler automatically compiles your Python functions into efficient GPU or CPU machine code for parallel execution.
 - Portability: Write your code once and run it everywhere. Currently, Taichi supports most mainstream GPU APIs, such as CUDA and Vulkan.
-- ... and many more features! A cross-platform, Vulkan-based 3D visualizer, [differentiable programming](https://docs.taichi.graphics/lang/articles/differentiable_programming),  [quantized computation](https://github.com/taichi-dev/quantaichi) (experimental), etc.
+- ... and many more features! A cross-platform, Vulkan-based 3D visualizer, [differentiable programming](https://docs.taichi-lang.org/lang/articles/differentiable_programming),  [quantized computation](https://github.com/taichi-dev/quantaichi) (experimental), etc.
 
 ## Getting Started
 
@@ -120,17 +120,17 @@ for i in range(1000000):
 
 <a href="https://github.com/taichi-dev/taichi/blob/master/python/taichi/examples/simulation/fractal.py#L1-L31"> </a><img src="https://raw.githubusercontent.com/taichi-dev/public_files/master/taichi/fractal_small.gif" height="270px">
 
-See [Get started](https://docs.taichi.graphics) for more information.
+See [Get started](https://docs.taichi-lang.org) for more information.
 
 ### Build from source
 
-If you wish to try our our experimental features or build Taichi for your own environments, see [Developer installation](https://docs.taichi.graphics/lang/articles/dev_install).
+If you wish to try our our experimental features or build Taichi for your own environments, see [Developer installation](https://docs.taichi-lang.org/lang/articles/dev_install).
 
 ## Documentation
 
-- [Technical documents](https://docs.taichi.graphics/)
-- [API Reference](https://docs.taichi.graphics/api/)
-- [Blog](https://docs.taichi.graphics/blog)
+- [Technical documents](https://docs.taichi-lang.org/)
+- [API Reference](https://docs.taichi-lang.org/api/)
+- [Blog](https://docs.taichi-lang.org/blog)
 
 ## Contributing
 

benchmarks/microbenchmarks/_items.py

@@ -67,7 +67,7 @@ def __init__(self):
 class MathOps(BenchmarkItem):
     name = 'math_op'
 
-    #reference: https://docs.taichi.graphics/lang/articles/operator
+    #reference: https://docs.taichi-lang.org/lang/articles/operator
     def __init__(self):
         self._items = {
             # Trigonometric