Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Jenkins CI - npx nx affected --target=build throws as error #2422

Closed
mikel67 opened this issue Feb 3, 2020 · 16 comments
Closed

Jenkins CI - npx nx affected --target=build throws as error #2422

mikel67 opened this issue Feb 3, 2020 · 16 comments
Labels
blocked: repro needed outdated scope: misc Misc issues type: question / discussion

Comments

@mikel67
Copy link

mikel67 commented Feb 3, 2020

Expected Behavior

Build should execute on Jenkins with same source as executes on local environment

Current Behavior

Source checked into git, Jenkins multi branch pipeline executes

Build stage fails with error:

  • npx nx affected --target=build -- master HEAD
    fatal: Not a valid object name affected
    Command failed: git merge-base affected master
    fatal: Not a valid object name affected

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

  1. Source repo available at https://git.symbiotics.co.za/mike.love/NXPoC.git
  2. Checkout locally, build via npm install & npm run build => successful nx build
  3. Run as Jenkinsmulti pipeline build

Context

Please provide any relevant information about your setup:

  • version of Nx used (Please run nx report at the root of your workspace and copy the results here if you are using Nx 8.6.1 or greater)
  • 3rd-party libraries and their versions
  • and most importantly - a use-case that fails

A minimal reproduction scenario allows us to quickly confirm a bug (or point out coding problem) as well as confirm that we are fixing the right problem.

Failure Logs

Extract of log file included below:

[Pipeline] }
[Pipeline] // nodejs
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] archive
The archive step is deprecated, please use archiveArtifacts instead.
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
@mehrad-rafigh
Copy link
Contributor

Jenkins only does a sparse checkout. Have you tried checking out the branch which has to be built/tested etc?

@vsavkin
Copy link
Member

vsavkin commented Feb 5, 2020

Could you upload a repro to github?

@mikel67
Copy link
Author

mikel67 commented Feb 5, 2020

Thanks for the response

Repo pushed to Github.
https://github.com/mikel67/NXPoC.git

@mehrad-rafigh, I have not yet verified direct checkout on the Jenkins agent. Will feedback when I get a chance to test

@zakhenry
Copy link
Contributor

I have a similar problem - the issue arises when trying to compute affected when there is a shallow clone. Ideally it should be possible to just have a shallow clone of the current branch and the target branch (i.e. only 2 commits pulled for the whole repo).

The problem arises in

nx/packages/workspace/src/command-line/shared.ts

Lines 74 to 81 in 74716df

function getFilesUsingBaseAndHead(base: string, head: string): string[] {
const mergeBase = execSync(`git merge-base ${base} ${head}`, {
maxBuffer: TEN_MEGABYTES
})
.toString()
.trim();
return parseGitOutput(`git diff --name-only --relative ${mergeBase} ${head}`);
}
where git merge-base is used to find a common ancestor.

I'm not sure if there is an alternative to git merge-base to work out how to find the minimal set of changed files in head only given no history

@mikel67
Copy link
Author

mikel67 commented Feb 22, 2020

@zakhenry appreciate the comprehensive response.

@vsavkin
Copy link
Member

vsavkin commented Feb 27, 2020

@zakhenry you could pass the files yourself, like this:

nx affected --target=build --files=list-of-paths-with-commas

Where you generate the list by running git diff --name-only --relative SHA1 SHA2. It's a bit annoying but it should work.

@zakhenry
Copy link
Contributor

@vsavkin yep that is true however it will be quite a bit more eager than the git-merge technique as the git diff will pick up every single non matching file, whether or not it was caused by a change in the current branches divergence from the common ancestor. Weighed up it is probably faster overall to just clone deeply.

@vsavkin
Copy link
Member

vsavkin commented Mar 4, 2020

@zakhenry yup. Another option, which isn't really a replacement for affected, is to use the the NxCloud distributed caching. So you run nx run-many --target=build --all, but it will only rebuild things that weren't cached. This isn't exactly the same (see last section of this: https://github.com/nrwl/nx-distributed-cache-example).

I'm going to close this issue. I don't think there is a good fix for it. The workaround: Since you can pass the list of touched files directly, you can compute the list in the way that works in your CI env.

@vsavkin vsavkin closed this as completed Mar 4, 2020
@tsoehnlin-aura
Copy link

@vsavkin This solution has limitations in the fact that any non-trivial change (e.g. adding in a whole new submodule) will result in a large file list. This has impact on reporting in the CI system as the concatentated list of files will be embedded in the output/headings and the like. This could also have impact on shell limitations on the number of characters that are safely passed for given arguments or overall commmands.

Would it be possible to formally support this model via arguments e.g. nx affected --target=build --diff=SHA1..SHA2 and run this git operation internally?

@SanthoshKumar-Prodapt
Copy link

@vsavkin ,

Just a suggestion, I had the same issue with jenkins where the origin develop was not fetched as jenkins was doing shadow cloning, I resolved the issue by enabling advanced clone option in multibranch pipline and enable fetch tags. It helped me.
I hope it helps you.
Screenshot 2021-03-25 at 23 24 56

@tomalaforge
Copy link
Contributor

Hello,

Does anyone succeded in using nx affected in a jenkins pipeline?

@nycynik
Copy link

nycynik commented May 31, 2021 •
edited
Loading

I don't have a solution, but wanted to share some information.

In pipeline jobs, it's not a problem, since you can indicate the branch to build, and sure that you have the base on that branch's last commit.

The problem is multi-branch pipelines. And the issue is the affected command requires you to tell it what base you want to compare it to. This branch also needs to be checked out in order for the comparison to work. This is rarely the case - when you have a PR from some other branch (feature* for example) - this is when origin/main (or master) is not checked out and the comparison has

The solutions above all are trying to solve that issue, how to keep the checkout shallow/fast, but also have everything needed to do the comparison.

There are two scenarios, one is where you are building a branch and the other where you are building a PR, they have different env variables and are different.

The solution offered in the docs is to use something like this:

def String baseSha = env.CHANGE_ID ? 'origin/main' : 'origin/main~1'

This is what is used to generate the base, but this only works in certain situations, like when you are on the main branch, to begin with.

@santoshyadavdev 's solution worked for me.

@santoshyadavdev
Copy link
Contributor

santoshyadavdev commented May 31, 2021 •
edited
Loading

I don't have a solution, but wanted to share some information.

In pipeline jobs, it's not a problem, since you can indicate the branch to build, and sure that you have the base on that branch's last commit.

The problem is multi-branch pipelines. And the issue is the affected command requires you to tell it what base you want to compare it to. This branch also needs to be checked out in order for the comparison to work. This is rarely the case - when you have a PR from some other branch (feature* for example) - this is when origin/main (or master) is not checked out and the comparison has

The solutions above all are trying to solve that issue, how to keep the checkout shallow/fast, but also have everything needed to do the comparison.

There are two scenarios, one is where you are building a branch and the other where you are building a PR, they have different env variables and are different.

The solution offered in the docs is to use something like this:

def String baseSha = env.CHANGE_ID ? 'origin/main' : 'origin/main~1'

This is what is used to generate the base, but this only works in certain situations, like when you are on the main branch, to begin with.

@santoshyadavdev 's solution worked for me.

Looks like you tagged me by mistake 😃 it's @SanthoshKumar-Prodapt

@maxisam
Copy link

maxisam commented Aug 10, 2021

I think this is worth to mention in the document.

@vespertilian
Copy link

Just for anyone else finding this thread. @SanthoshKumar-Prodapt fix worked for me ... just make sure you use origin in your branch name. Originally I just used develop and it did not work.

Also I found it easier to add the command to my package.json and call that from jenkins.

    "test-affected-dev": "npm run affected:test -- --base=origin/develop --head=HEAD",
    "lint-affected-dev": "npm run affected:lint -- --base=origin/develop --head=HEAD",

@github-actions
Copy link

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
blocked: repro needed outdated scope: misc Misc issues type: question / discussion
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests