Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] rerun_ci script #279

Merged
merged 7 commits into from
Aug 9, 2018
Merged

[WIP] rerun_ci script #279

merged 7 commits into from
Aug 9, 2018

Conversation

mathias-luedtke
Copy link
Member

rerun_ci runs local tests like run_ci, but speeds up recurring builds by reusing the image from the last run.

Each run will commit the container to industrial-ci/run_ci/$REPO_NAME:$CLI_HASH. The latter hashes the command line arguments. Thee will added to the images as a comment as well.
So each combination of (absolute) target path and enviromnent settings gets a dedicated image.

There are some differences between run_ci and rerun_ci:

  • the first argument must be the target path, "." is okay as well.
  • the images will persist, the users must delete them manually
  • the environement gets filled only with the command line arguments

Please give it a try.

@mathias-luedtke
introduce DOCKER_PULL (default: true)
2b93722 
If image in DOCKER_IMAGE has been created locally, docker pull would fail.
@mathias-luedtke mathias-luedtke force-pushed the feature/rerun_ci branch 3 times, most recently from ef3477f to c47a88c Compare May 21, 2018 00:58
@mathias-luedtke mathias-luedtke mentioned this pull request May 21, 2018
Merged
miguelprada
miguelprada requested changes May 22, 2018
View reviewed changes
Copy link
Member

@miguelprada miguelprada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The install rule for the script is missing.

Also, it would be nice to update the section on running industrial_ci on the local host with a brief description and usage example.

Other than that, it works like a charm!

@mathias-luedtke
Copy link
Member Author

The install rule for the script is missing.

Shame on me!

I added the rule and some documentation.
Furthermore, the run_ci exit code is now returned by rerun_ci properly.

@mathias-luedtke mathias-luedtke changed the title [WIP] rerun_ci script rerun_ci script May 22, 2018
miguelprada
miguelprada approved these changes May 23, 2018
View reviewed changes
Copy link
Member

@miguelprada miguelprada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👍

@mathias-luedtke mathias-luedtke force-pushed the feature/rerun_ci branch 2 times, most recently from fe4505b to dfb1172 Compare May 23, 2018 16:58
@mathias-luedtke mathias-luedtke mentioned this pull request May 25, 2018
Merged
This was referenced Jun 2, 2018
Merged
Closed
130s
130s requested changes Jun 11, 2018
View reviewed changes
Copy link
Member

@130s 130s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty useful! Just spotted easy .rst formatting issues.

doc/index.rst Outdated

As an alternative `rerun_ci` could be used. It take the same argument as `run_ci`, but will run the build incrementally and only download or compile after changes.
This results in much faster execution for recurring runs, but has some disadvantages as well:
* The user needs to clean-up manually, an instruction to do so is printed at the end of all runs.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably blank space is needed to enable bullet points in .rst.

doc/index.rst Outdated
$ rosrun industrial_ci rerun_ci . ROS_DISTRO=melodic ROS_REPO=ros-shadow-fixed

This will run the tests and commit the result to a Docker image `industrial-ci/rerun_ci/ros_canopen:$HASH`.
The hash is unique for each argument list, so `rerun_ci . ROS_DISTRO=melodic` and `rerun_ci . ROS_DISTRO=kinetic` do not mix up.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so rerun_ci . ROS_DISTRO=melodic and rerun_ci . ROS_DISTRO=kinetic do not mix up.

github.com does not render it nicely enough so that it's hard to tell apart the 2 sets of commands. Maybe separate lines makes it the most clear.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently, inline code has to be wrapped in double backticks.

@mathias-luedtke mathias-luedtke changed the title rerun_ci script [WIP] rerun_ci script Jun 11, 2018
@mathias-luedtke
Copy link
Member Author

Thanks for the review!
This one needs to be changed a little bit after #283 was merged.
In addition _COMMIT_IMAGE will become a public option (#286)

@130s 130s changed the title [WIP] rerun_ci script rerun_ci script Jun 23, 2018
130s
130s approved these changes Jun 23, 2018
View reviewed changes
Copy link
Member

@130s 130s left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#279 (comment)

This one needs to be changed a little bit after #283 was merged.
In addition _COMMIT_IMAGE will become a public option (#286)

I assume those are future TODOs, not for this PR. Merging. -> Found some jobs failing #279 (comment)

@130s
Copy link
Member

130s commented Jun 23, 2018

Found a couple of CI jobs failed. They are restarted. Once they pass we can merge this.

@130s
Copy link
Member

130s commented Jun 23, 2018

https://travis-ci.org/ros-industrial/industrial_ci/jobs/394076779#L554

ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
industrial_ci: No definition of [catkin] for OS [debian]
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Function rosdep_install returned with code '1' after 0 min 0 sec

I assume this is hitting #291?

Is this ignorable or not, regarding PR merge? I'm inclined to ignore it for now.

@mathias-luedtke mathias-luedtke changed the title rerun_ci script [WIP] rerun_ci script Jun 23, 2018
@mathias-luedtke
Copy link
Member Author

I assume those are future TODOs, not for this PR. Merging.

No, #283 is an extended version of 2b93722

I assume this is hitting #291?

Yes, all our builds will hit this.

Is this ignorable or not, regarding PR merge? I'm inclined to ignore it for now.

Yes, but don't this PR merge now. ;)
The isssue is fixed in #292.

@130s
Copy link
Member

130s commented Jul 11, 2018

Just a comment; at my work, people have been frustrated with CI gitlab.com for a number of reasons (along with many advantages) and kept asking around more legit way to run CI locally. So this will be very useful.

An inevitable downside (correct me if wrong) is that we won't be able to count the number of local runs as it won't clone industrial_ci from github.

@130s 130s mentioned this pull request Jul 11, 2018
Closed
@130s
Copy link
Member

130s commented Aug 9, 2018

@ipa-mdl Could you rebase to the latest? I think #292 is needed to pass Travis.

@130s
Copy link
Member

130s commented Aug 9, 2018

I'm running rebased branch of yours @ipa-mdl on my fork. https://travis-ci.org/130s/industrial_ci/builds/414177337. If it runs fine then I'll go ahead merge this.

@130s
Copy link
Member

130s commented Aug 9, 2018

Merging as https://travis-ci.org/130s/industrial_ci/builds/414177337 passed.

@130s 130s merged commit d8ac934 into ros-industrial:master Aug 9, 2018
@mathias-luedtke
Copy link
Member Author

This really was WIP!
It should not have been merged like this.

@mathias-luedtke mathias-luedtke deleted the feature/rerun_ci branch August 9, 2018 23:17
@130s 130s mentioned this pull request Aug 10, 2018
Closed
@mathias-luedtke mathias-luedtke mentioned this pull request Sep 15, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@130s 130s 130s approved these changes

@miguelprada miguelprada miguelprada approved these changes

Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mathias-luedtke @130s @miguelprada