Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fetching @gapic_generator_python_pip_deps takes 5+ minutes #400

Closed
aohren opened this issue Apr 22, 2020 · 1 comment · Fixed by #402
Closed

Fetching @gapic_generator_python_pip_deps takes 5+ minutes #400

aohren opened this issue Apr 22, 2020 · 1 comment · Fixed by #402
Assignees
@vam-google
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release.

Comments

@aohren
Copy link
Contributor

aohren commented Apr 22, 2020

Breaking this out as a separate issue from #361 (where I first mentioned this).

When trying to build the py_gapic_library Bazel rule, the Fetching @gapic_generator_python_pip_deps step takes a little >5 mins from a clean build every time.

Reproducible by doing bazel clean --expunge before building.
@aohren aohren mentioned this issue Apr 22, 2020
Closed
@hkdevandla
Copy link
Member

I guess @vam-google is looking into this. Vadym, please update with your findings.

@hkdevandla hkdevandla added the priority: p2 Moderately-important priority. Fix may not be included in next release. label Apr 22, 2020
@vam-google vam-google mentioned this issue Apr 25, 2020
Merged
vam-google added a commit that referenced this issue May 4, 2020
@vam-google
feat: Make GAPIC Bazel rules production ready (#402)
d18ed41 
Shoud fix #400 and #390, plus a bunch of other not-yet-opened issues.

This includes:
1) Fix long initial load time (5+ min). This was caused by python_rules buildling `grpcio` dependency from sources in one core (which was super slow). Switched to using bazel-native `"@com_github_grpc_grpc//src/python/grpcio/grpc:grpcio"` target instead, which is not only much faster, but is also already used in googleapis, so there is no additional cost for reusing it in microgenerator rules.

2) Properly handle `pandoc` dependency (platform-sepcific version of pandoc is properly pulled by bazel itself using toolchains).

3) Add simplistic version of the `py_gapic_assembly_pkg` rule, to make output of microgenerator compatible with `GAPICBazel` class in synthtool.

4) Add `plugin_args` argument for python_gapic_library rule to pass custom argumetns to the plugin (similar to PHP rules).

5) Add compatibility with  `python3.6` runtime (otherwise `python3.7` is minimum because of dependency on `dataclasses` module). Python 3.6 compatibility can be enabled by adding `--define=gapic_gen_python=3.6` command line argument to `bazel build` command. 

6) Add support for Python runtimes installed with `pyenv`. To tell bazel using Python3 installed via pyenv add `--extra_toolchains=@gapic_generator_python//:pyenv3_toolchain` argument to `bazel build` command.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vam-google vam-google

Labels
priority: p2 Moderately-important priority. Fix may not be included in next release.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Make GAPIC Bazel rules production ready vam-google/gapic-generator-python
3 participants
@aohren @vam-google @hkdevandla