Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Refactor] Use setup.py from root dir. #2540

Merged
merged 1 commit into from
Jul 19, 2021
Merged

Conversation

ailzhang
Copy link
Contributor

@ailzhang ailzhang commented Jul 19, 2021

This is a copy of #2531 to merge into master.

A few things included in this PR:

  • Move setup.py up to root dir so that we can run python setup.py install/develop.
  • Remove dependency of taichi_core in build process. We used to call
    ti.core.get_runtime_dir()/get_version_string()/libdevice_path() from
    C++, but these information are already available in build process.
  • Simplify python/build.py to call setup.py in the root. Tested
    build/test commands still work as expected.
  • Update github workflows to use python setup.py install in linux/mac instead of
    setting env TAICHI_REPO_DIR.
  • Developers used to customize taichi_core.so build using args passed
    into cmake. Under this new mechanism they can put everything inside
    TAICHI_CMAKE_ARGS and run setup.py using: TAICHI_CMAKE_ARGS="-DXXXX -DXXXX" python setup.py ..
  • Move tests/conftest.py into tests/python folder. I think it was a
    bug that we skipped it when we copied tests folder when building wheels.

To explain how build works after this PR:
In short, c++ build is no longer a separate step. It's built as an
extension when you build taichi python package.

Developers can use

  • python setup.py develop(or pip install -e .) in root dir to
    install taichi into their current python environment. Note in
    develop mode any local changes to python files take effect
    immediately. You only need to rerun this command when you change C++
    code.
  • python setup.py install(or pip install .) to install taichi into
    your their current python environment. Note in install mode files
    are copied over to python's site-packages so your local changes won't
    take effect until you rerun the command.
  • No more manipulation with TAICHI_REPO_DIR and PYTHONPATH.

A few known followups:

  • I left windows job to remain using TAICHI_REPO_DIR since I don't
    have a windows machine. But it should be straightfoward to update if
    someone else can help.
  • We can start removing TAICHI_REPO_DIR in the codebase after this PR since our
    development mode has changed.
  • Update developer install documentation.
  • I'm pretty sure the handling for windows in setup.py might have
    minor problems about relative paths but will leave it to debug in CI.
    It should be a simple fix.
  • We probably need to reconsider if we want to include assets/examples
    and a few other .py files under python/taichi in the release.

ghstack-source-id: b82db21c4293c3281dbf3869cc91a927d234a4fa
Pull Request resolved: #2531

Related issue = #

A few things included in this PR:
- Move `setup.py` up to root dir so that we can run `python setup.py
  install/develop`.
- Remove dependency of `taichi_core` in build process. We used to call
  `ti.core.get_runtime_dir()/get_version_string()/libdevice_path()` from
C++, but these information are already available in build process.
- Simplify `python/build.py` to call `setup.py` in the root. Tested
  `build/test` commands still work as expected.
- Update github workflows to use `python setup.py install` in linux/mac instead of
  setting env `TAICHI_REPO_DIR`.
- Developers used to customize taichi_core.so build using args passed
  into cmake. Under this new mechanism they can put everything inside
`TAICHI_CMAKE_ARGS` and run `setup.py` using: `TAICHI_CMAKE_ARGS="-DXXXX -DXXXX" python setup.py ..`
- Move `tests/conftest.py` into `tests/python` folder. I think it was a
  bug that we skipped it when we copied `tests` folder when building wheels.

To explain how build works after this PR:
In short, c++ build is no longer a separate step. It's built as an
extension when you build `taichi` python package.

Developers can use
- `python setup.py develop`(or `pip install -e .`) in root dir to
  install taichi into their current python environment. Note in
`develop` mode any local changes to python files take effect
immediately. You only need to rerun this command when you change C++
code.
- `python setup.py install`(or `pip install .`) to install taichi into
  your their current python environment. Note in `install` mode files
are copied over to python's `site-packages` so your local changes won't
take effect until you rerun the command.
- No more manipulation with `TAICHI_REPO_DIR` and `PYTHONPATH`.

A few known followups:
- I left windows job to remain using `TAICHI_REPO_DIR` since I don't
  have a windows machine. But it should be straightfoward to update if
  someone else can help.
- We can start removing `TAICHI_REPO_DIR` in the codebase after this PR since our
  development mode has changed.
- Update developer install documentation.
- I'm pretty sure the handling for windows in `setup.py` might have
  minor problems about relative paths but will leave it to debug in CI.
   It should be a simple fix.
- We probably need to reconsider if we want to include assets/examples
  and a few other .py files under `python/taichi` in the release.

ghstack-source-id: b82db21c4293c3281dbf3869cc91a927d234a4fa
Pull Request resolved: taichi-dev#2531
@ailzhang ailzhang changed the title [DEBUG] Use setup.py from root dir. [Refactor] Use setup.py from root dir. Jul 19, 2021
@ailzhang ailzhang requested a review from k-ye July 19, 2021 05:54
Copy link
Member

@k-ye k-ye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!

@ailzhang ailzhang merged commit a2a45a9 into taichi-dev:master Jul 19, 2021
@Leonz5288 Leonz5288 mentioned this pull request Jul 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants