Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(datasource/azure-pipelines-tasks): Azure DevOps API based datasource #32966

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

bdovaz
Copy link

@bdovaz bdovaz commented Dec 7, 2024

Changes

Initially as I do not have much experience of the Renovate API, I tried (looking at other code examples in this repository) that only changes the behavior for the case that you are running it from platform: azure because this way there is no need to create any new variable and it can use the values of RENOVATE_PLATFORM, RENOVATE_ENDPOINT and RENOVATE_TOKEN.

This PR does not solve the case that platform is a value different to azure (in that case it makes fallback to what it was already doing) because for that case I suppose that new variables will be needed and I consider that this can go in another future PR because it requires more analysis.

Context

The need comes from the fact that we currently take as a source of data the static files that are generated from this repository: https://github.com/renovatebot/azure-devops-marketplace

The problem is that task updates are not propagated at the same speed to all Azure Devops organizations so there are times when renovate mistakenly suggests updating to versions of tasks that do not exist in the organization and if you do not know why it happens it is very confusing.

There is an open discussion: #24820

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

@CLAassistant
Copy link

CLAassistant commented Dec 7, 2024

CLA assistant check
All committers have signed the CLA.

@bdovaz bdovaz changed the title Azure DevOps API based datasource feat(datasource/azure-pipelines-tasks): Azure DevOps API based datasource Dec 7, 2024
Copy link
Collaborator

@rarkins rarkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure that this can replace the existing functionality in a backwards-compatible way?

How efficient is this approach? e.g. how many API calls and how many MB should it require each run?

Any caching possible?

return { releases };
if (platform === 'azure' && endpoint) {
const auth = Buffer.from(
`renovate:${process.env.RENOVATE_TOKEN}`,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not certain that RENOVATE_TOKEN is defined. Instead, use hostRules to get the token

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have used the GlobalConfig and hostRules based on other data sources but I don't know if I have done it right.

I don't see any API documentation, I've only seen the pages you recommend here: https://github.com/renovatebot/renovate/blob/main/.github/contributing.md#code

@bdovaz
Copy link
Author

bdovaz commented Dec 14, 2024

Are you sure that this can replace the existing functionality in a backwards-compatible way?

How efficient is this approach? e.g. how many API calls and how many MB should it require each run?

Any caching possible?

Doesn't the caching already do it with the code that was there?

https://github.com/bdovaz/renovate/blob/620fa90803e995e404e29c478b1cfcd32ad028f8/lib/modules/datasource/azure-pipelines-tasks/index.ts#L81

And the backwards-compatible thing is for when these conditions are not met that it at least does what it did before:

https://github.com/bdovaz/renovate/blob/620fa90803e995e404e29c478b1cfcd32ad028f8/lib/modules/datasource/azure-pipelines-tasks/index.ts#L36

I've also added more tests to compare function so we have 100% code coverage.

@rarkins
Copy link
Collaborator

rarkins commented Dec 17, 2024

So it caches once per 24 hours, if the user has persistent datasource caching? (note: most people running Renovate in pipelines do not have such persistence)

I don't understand your answer to my backwards-compatibility question. Is it replacing existing logic, or adding to it? If it's replacing it, is it fully backwards compatible?

How many API calls and how many MB should it require?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants