Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(components): buildCpuTrainer failure #4405

Merged
merged 32 commits into from
Aug 25, 2020

Conversation

jingzhang36
Copy link
Contributor

@jingzhang36 jingzhang36 commented Aug 24, 2020

Description of your changes:

We've observed persistent errors in buildCpuTrainer and buildGpuTrainer which seem to be caused by a version issue with setuptools. However, since we use tensorflow image 1.6.0, the latest setuptools via apt-get is 20.7.0. Therefore, in order to increase setuptools version, we increase tensorflow image to 2.3.0 and as a result, tensorflow transform and tensorflow model analysis need to be increased too. Let's see if this can get rid of the error.

Part of #4361
Screen Shot 2020-08-24 at 5 18 39 PM

Screen Shot 2020-08-24 at 5 11 54 PM

Checklist:

  • The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

    PR titles examples:

    • fix(frontend): fixes empty page. Fixes #1234
      Use fix to indicate that this PR fixes a bug.
    • feat(backend): configurable service account. Fixes #1234, fixes #1235
      Use feat to indicate that this PR adds a new feature.
    • chore: set up changelog generation tools
      Use chore to indicate that this PR makes some changes that users don't need to know.
    • test: fix CI failure. Part of #1234
      Use part of to indicate that a PR is working on an issue, but shouldn't close the issue when merged.
  • Do you want this pull request (PR) cherry-picked into the current release branch?

    If yes, use one of the following options:

    • (Recommended.) Ask the PR approver to add the cherrypick-approved label to this PR. The release manager adds this PR to the release branch in a batch update.
    • After this PR is merged, create a cherry-pick PR to add these changes to the release branch. (For more information about creating a cherry-pick PR, see the Kubeflow Pipelines release guide.)

…page and the archive page"

This reverts commit 5b67273.
@kubeflow-bot
Copy link

This change is Reviewable

@jingzhang36
Copy link
Contributor Author

/test kubeflow-pipeline-e2e-test

@jingzhang36
Copy link
Contributor Author

/assign @rmgogogo
/assign @Bobgy

@@ -19,7 +19,7 @@ while getopts ":hp:t:i:b:l:" opt; do
echo "-t: tag name"
echo "-i: image name. If provided, project name and tag name are not necessary"
echo "-b: tensorflow base image tag. Optional. The value can be tags listed under \
https://hub.docker.com/r/tensorflow/tensorflow/tags. Defaults to '1.6.0'."
https://hub.docker.com/r/tensorflow/tensorflow/tags. Defaults to '2.3.0'."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:
0.23.0?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 2.3.0 is correct

Copy link
Contributor Author

@jingzhang36 jingzhang36 Aug 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tensorflow is 2.3.0, while tensorflow transform and tensorflow model analysis is 0.23.0

@Bobgy Bobgy changed the title fix: buildCpuTrainer failure build(components): buildCpuTrainer failure Aug 24, 2020
@@ -53,7 +53,7 @@ if [ -z "${TAG_NAME}" ]; then
fi

if [ -z "${TF_BASE_TAG}" ]; then
TF_BASE_TAG=1.6.0
TF_BASE_TAG=0.23.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2.3.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for catching this. Revised.

@Ark-kun
Copy link
Contributor

Ark-kun commented Aug 24, 2020

Thank you for taking on this issue, Jing.

@Bobgy
Copy link
Contributor

Bobgy commented Aug 25, 2020

/lgtm

@jingzhang36
Copy link
Contributor Author

/test kubeflow-pipeline-backend-test

@jingzhang36
Copy link
Contributor Author

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jingzhang36
To complete the pull request process, please assign bobgy
You can assign the PR to them by writing /assign @bobgy in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jingzhang36 jingzhang36 merged commit 7e68991 into kubeflow:master Aug 25, 2020
@Ark-kun
Copy link
Contributor

Ark-kun commented Aug 25, 2020

The following packages are not found for licenses tracking.
Please add an entry in /ml/third_party_licenses.csv for each of them.
argon2-cffi astunparse avro-python3 backcall importlib-metadata packaging QtPy tensorboard-plugin-wit zipp

@Bobgy Bobgy added the cherrypick-approved area OWNER approves to cherry pick this PR to current active release branch label Sep 4, 2020
Bobgy pushed a commit to Bobgy/pipelines that referenced this pull request Sep 4, 2020
* enable pagination when expanding experiment in both the home page and the archive page

* Revert "enable pagination when expanding experiment in both the home page and the archive page"

This reverts commit 5b67273.

* fix buildcputrainer error

* update args when calling build image shell script

* remove tmp test file

* tensorflow is 2.3.0

* remove accidently checked in file
Bobgy pushed a commit to Bobgy/pipelines that referenced this pull request Sep 4, 2020
* enable pagination when expanding experiment in both the home page and the archive page

* Revert "enable pagination when expanding experiment in both the home page and the archive page"

This reverts commit 5b67273.

* fix buildcputrainer error

* update args when calling build image shell script

* remove tmp test file

* tensorflow is 2.3.0

* remove accidently checked in file
Bobgy pushed a commit that referenced this pull request Sep 4, 2020
* enable pagination when expanding experiment in both the home page and the archive page

* Revert "enable pagination when expanding experiment in both the home page and the archive page"

This reverts commit 5b67273.

* fix buildcputrainer error

* update args when calling build image shell script

* remove tmp test file

* tensorflow is 2.3.0

* remove accidently checked in file
@Bobgy Bobgy added the cherrypicked cherry picked to release branch `release-x.y` label Sep 4, 2020
Jeffwan pushed a commit to Jeffwan/pipelines that referenced this pull request Dec 9, 2020
* enable pagination when expanding experiment in both the home page and the archive page

* Revert "enable pagination when expanding experiment in both the home page and the archive page"

This reverts commit 5b67273.

* fix buildcputrainer error

* update args when calling build image shell script

* remove tmp test file

* tensorflow is 2.3.0

* remove accidently checked in file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherrypick-approved area OWNER approves to cherry pick this PR to current active release branch cherrypicked cherry picked to release branch `release-x.y` cla: yes lgtm size/S
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants