Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples for resource pipeline and source inputs are not clear. #594

Open
PramodKumarYadav opened this issue Jun 2, 2023 · 9 comments
Open

Comments

@PramodKumarYadav
Copy link

This issue is with reference to examples listed here: https://github.com/microsoft/azure-pipelines-yaml/blob/master/design/pipeline-resources.md#examples

For example, it is not clear what the distinction between input at "pipeline: SmartHotel" and "source: SmartHotel-CI" from a source side is. If you could given some examples on how to get these names and from where with an example, that would help a lot. Currently its a bit fuzzy and making it difficult to make this part work.

resources:
pipelines:

  • pipeline: SmartHotel
    project: DevOpsProject
    source: SmartHotel-CI
    branch: releases/M142
@DmitriiBobreshev
Copy link
Contributor

DmitriiBobreshev commented Jun 5, 2023

Hi @PramodKumarYadav, thank you for the suggestion! The main goal of the docs is to support collaborative contributions to the design process. For the actual and relevant information you could check the Azure Devops Docs, speaking about the pipeline resources, you could check detailed
documentation here. We'll also change the example for the file we will have enough capacity, but we also very welcome any PRs to make the documentation in the repository more helpful.

@PramodKumarYadav
Copy link
Author

PramodKumarYadav commented Jun 5, 2023

Thanks for the response Dmitrii.

We'll also change the example for the file we will have enough capacity, but we also very welcome any PRs to make the documentation in the repository more helpful.

On your above suggestion I would have made a PR but its not clear to me how & what values for these fields should be based on an upstream jobs pipeline. I went through the docs more than a few times and tried some combinations but all of them resulted in error saying "Pipeline Resource trigger-tests-on-merge-to-main Input Must be Valid."

Now it could be because of some simple thing missing and I tried a lot of trial and error combinations but since there is no one-to-one reference on the values to be filled in downstream job vs upstream job pipeline source, its not clear to me what is getting missed here and why it is not working. If you can help give me an example and if it works, I will be glad to create a PR to update the docs.

Thanks again for looking into it and appreciate your support on this 🙌

@DmitriiBobreshev
Copy link
Contributor

DmitriiBobreshev commented Jun 5, 2023

@PramodKumarYadav Sure, If I understood you correctly, you want to run one pipeline from another, you can check this detailed example as a reference. If you already saw it, could you please write in detail what you want to do, maybe I could help you with it?

btw, maybe the problem is that you need to put branch into the trigger input as in the example

# app-ci YAML pipeline
resources:
  pipelines:
  - pipeline: securitylib
    source: security-lib-ci
    trigger: 
      branches:
        include: 
        - releases/*
        exclude:
        - releases/old*

@PramodKumarYadav
Copy link
Author

Hello @DmitriiBobreshev , my use case is to use option number 5 (Pipeline completion triggers) here:

I have a "test repository" which contains a pipeline that can run tests. What I want is when an "application" repository pipeline finishes (say for deployment), then I can immediately run my tests in the "test" repository post the application pipeline completion.

btw, maybe the problem is that you need to put branch into the trigger input as in the example

P.S: I did include branch name in the pipeline I setup but it didnt work. So maybe a working example that shows how the downsteam (test) repo should refer to resource names in an upstream (app) repo that would be great!

@DmitriiBobreshev
Copy link
Contributor

@PramodKumarYadav, Have you tried to enable build completion triggers using GUI?
In the "application's" pipeline, you need to select triggers
image
and set your"test repository's" pipeline
image

here you can find the description: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/pipeline-triggers?view=azure-devops

@PramodKumarYadav
Copy link
Author

PramodKumarYadav commented Jun 12, 2023

@DmitriiBobreshev : Classic build triggers are not allowed in our company. So I have to use pipeline triggers and I keep getting this error.

on-internal-trigger.yml (Line: 7, Col: 15): Pipeline Resource securitylib Input Must be Valid.

even when I use exactly the same trigger example as specified here: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/pipeline-triggers?view=azure-devops

P.S: This has been so much more simpler to setup when I worked with Github Actions. I wish it wasn't so difficult here :). Maybe an example public test repo that shows all these pipelines working examples would be great.

Step1:
Pipeline to run first: security-lib-ci.yml file
Result: pipeline succeds

# security-lib-ci YAML pipeline
steps:
- bash: echo "The security-lib-ci pipeline runs first"

Step2:
Pipeline to run second: on-internal-trigger.yml
Result: fails with error on-internal-trigger.yml (Line: 7, Col: 15): Pipeline Resource securitylib Input Must be Valid.

# app-ci YAML pipeline
# We are setting up a pipeline resource that references the security-lib-ci
# pipeline and setting up a pipeline completion trigger so that our app-ci
# pipeline runs when a run of the security-lib-ci pipeline completes
resources:
  pipelines:
  - pipeline: securitylib # Name of the pipeline resource.
    source: security-lib-ci # The name of the pipeline referenced by this pipeline resource.
    # project: app # Required only if the source pipeline is in another project
    trigger: true # Run app-ci pipeline when any run of security-lib-ci completes

steps:
- bash: echo "app-ci runs after security-lib-ci completes"

Location of files (both in root repository)
Screenshot 2023-06-12 at 18 54 45

@PramodKumarYadav
Copy link
Author

Okay, I finally got to make it work but it seems the documentation here would need some updating if others dont go through the same issues.
In short the name of pipeline is not the nae of the file but the name of pipeline as seen in the GUI.
source: security-lib-ci # The name of the pipeline referenced by this pipeline resource.
So for example the name of the first pipeline to finish is:

  • main-ci-trigger (custom name given by the user or name of the project by default) if its the only pipeline or with a number after project name if not renamed.
Screenshot 2023-06-12 at 22 14 45 - However the name of the file is security-lib-ci (which is what the docs tell to fill in). Telling users to go to the pipeline section and see the name as written on the GUI for this pipeline is the right steps to write in the document.

Now this again is not enough. After user add the correct pipeline name in the source, they would also need to add a pipeline via GUI for these secondary pipeline triggers. Unless we do that the the yaml files in themselves are not enough to trigger the runs (something which works automatically in tools like github action). So unless you add this instruction also in the read me file, it would not work.

As a last step, If this trigger should really run "after" the first pipeline is finished then we need to add a trigger: none on the top so that our second pipeline only run one time "after" the first pipeline is finished. If we dont add this our pipeline will run two times. First in parallel to the first line and then again after first pipeline is finished.

P.S: If you could ping me the location to update documentation, I will be more than happy to create a PR with detailed instructions on how to do this properly. Hopefully no other user has to face this issue again ever.

@PramodKumarYadav
Copy link
Author

PramodKumarYadav commented Jun 12, 2023

My bad. I see this is the repo where all the documentation is :). I will create a PR right away with detailed examples.

Edit: Spoken too soon. I dont see the files for this page in this repo: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/pipeline-triggers?view=azure-devops

If you could ping me the repo where this documentation is updated, I will clone and create a PR there.

@DmitriiBobreshev
Copy link
Contributor

Hi @PramodKumarYadav, I'm glad that it finally working! The documentation for ms learn is placed here. You could open most of the ms learn sources by pressing the button
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants