Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Azure authentication #785

Open
wants to merge 22 commits into
base: main
Choose a base branch
from

Conversation

jkroepke
Copy link
Contributor

@jkroepke jkroepke commented Mar 2, 2024

Fixes #768

This PR adds native Azure authentication, provided by github.com/grafana/grafana-azure-sdk-go.

It supports:

  • ClientCredentials (this was possible via OAuth2)
  • ManangedIdentity (credential-less authentication inside Azure Virtual Machines)
  • WorkloadIdentity (credential-less authentication inside Kubernetes)

@jkroepke jkroepke requested a review from a team as a code owner March 2, 2024 23:11
# Conflicts:
#	go.mod
#	go.sum
#	pkg/models/settings.go
@jkroepke
Copy link
Contributor Author

I fixed the conflict.

@yesoreyeram let me know If I can help/assist here.

@yesoreyeram
Copy link
Collaborator

Hi @jkroepke - Thank you so much for the contribution. I am planning to take a look at this later this week. Sorry for the delay. Will keep you posted.

Copy link
Collaborator

@yesoreyeram yesoreyeram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good to me. Left some nit comments.

pkg/infinity/azure.go Outdated Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
src/editors/config/MicrosoftInput.tsx Outdated Show resolved Hide resolved
src/editors/config/MicrosoftInput.tsx Outdated Show resolved Hide resolved
src/editors/config/MicrosoftInput.tsx Outdated Show resolved Hide resolved
src/types/config.types.ts Outdated Show resolved Hide resolved
src/editors/config/Auth.tsx Outdated Show resolved Hide resolved
@yesoreyeram yesoreyeram changed the title Add support for native Azure authentication Add support for Microsoft authentication Apr 15, 2024
# Conflicts:
#	go.mod
#	go.sum
Signed-off-by: Jan-Otto Kröpke <[email protected]>
@jkroepke
Copy link
Contributor Author

@yesoreyeram

I may ask myself, how this may can work, if os.LookupEnv does not work.

At the end, grafana/grafana-azure-sdk-go is using Azure/azure-sdk-for-go is expect some OS environment variables.

Ref:

@yesoreyeram
Copy link
Collaborator

Hi.. Thanks for the pointers. We have discussed this lot internally on similar topics. As grafana plugins are proceeding towards multitenant/remote plugins, we are already removing such os level things in our plugins code already. Adding this now to infinity will make things harder. In future, yes we can re-evaluate this decision and come up with better alternatives.

@jkroepke
Copy link
Contributor Author

jkroepke commented Apr 15, 2024

I figure out that Grafana admins needs to opt-in to allow such authentication.

https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#azure

Grafana pass the azure settings to plugins context, and the grafana-azure-sdk can be make aware of them

I had rewrite the plugin datasource config. The plugins will take care of the azure settings for now. It will allow only the specific authentications, if the Grafana admin has allowed it via grafana.ini.

For an out of the box experience, I would recommend to add this plugin there: https://github.com/grafana/grafana/blob/cc87281d7153d4b6b0edd365efee94d199a4f79b/conf/defaults.ini#L968

Maybe you can take an eye here.

@jkroepke jkroepke marked this pull request as draft April 15, 2024 17:26
Signed-off-by: Jan-Otto Kröpke <[email protected]>
@jkroepke jkroepke marked this pull request as ready for review April 15, 2024 18:47
@jkroepke jkroepke requested a review from yesoreyeram April 15, 2024 18:47
Signed-off-by: Jan-Otto Kröpke <[email protected]>
@yesoreyeram
Copy link
Collaborator

Again sorry for the delay and thank you for the patience. I am still on top of this PR and hopefully can merge this week.

Copy link
Collaborator

@yesoreyeram yesoreyeram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the delay and multiple roundtrips..

now getting the following error..

image

also can you update the documentation section about different authentication types and how to use it

pkg/models/settings.go Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
pkg/infinity/azure.go Outdated Show resolved Hide resolved
@jkroepke
Copy link
Contributor Author

jkroepke commented May 20, 2024

Seems like something broke after a merge commit.

I'm I still have a compiled version locally, but I did do a retest after the merge conflict

@jkroepke
Copy link
Contributor Author

jkroepke commented May 20, 2024

@yesoreyeram It seems like the error you get is coming from Grafana SDK, if user_identity_enabled = true is set on Grafana instance.

The errors comes from the Grafana SDK: https://github.com/grafana/grafana-azure-sdk-go/blob/c271284be0be9f85c64b6d2c8d00d22e458dc80f/azsettings/env.go#L86

grafana/grafana-azure-sdk-go#144

@jkroepke jkroepke changed the title Add support for Microsoft authentication Add support for Azure authentication May 20, 2024
@jkroepke jkroepke closed this Jul 6, 2024
@jkroepke jkroepke reopened this Jul 6, 2024
@yesoreyeram
Copy link
Collaborator

Hi there.. again sorry for the delay. you can do yarn spellcheck locally and fix the missing keywords in cspell.json file

Signed-off-by: Jan-Otto Kröpke <[email protected]>
@jkroepke
Copy link
Contributor Author

jkroepke commented Jul 6, 2024

great, guessing it is ready to merge now?

Signed-off-by: Jan-Otto Kröpke <[email protected]>
@jkroepke
Copy link
Contributor Author

Hey @yesoreyeram

my motivation to resolve a merge conflict one more is quite low.

Should I close this PR? It seems like that grafana-infinity-datasource has serious issues with external contributions

@yesoreyeram
Copy link
Collaborator

Hi @jkroepke - Sorry for the annoyance. We are quite busy and couldn't focus much on this PR. also last time when I tried to merge, CI was failing. Let's home this goes through. Happy to merge once CI passed.

@yesoreyeram
Copy link
Collaborator

seems CI tests failing again. Can you check?

@jkroepke
Copy link
Contributor Author

jkroepke commented Jul 16, 2024

Done.

It seems like additional check (lint) where introduced.

If the CI would automatic start, I could fix the CI issues on my own. However I have to ask someone each time to trigger the CI which makes to process terrible slow.

Maybe switch from CircleCI to GH Actions to solve such issues.

@jkroepke
Copy link
Contributor Author

tests are green here now

image

@jkroepke
Copy link
Contributor Author

CI is green now

@jkroepke
Copy link
Contributor Author

jkroepke commented Jul 22, 2024

Hey @yesoreyeram can we merge this before an merge conflict is introduced again resulting into red CI pipelines?

@jkroepke
Copy link
Contributor Author

jkroepke commented Jul 28, 2024

Hey @yesoreyeram

what are the next steps here? tbh, it's also time waste for you to recurring check this.

I recommend to merge this and any additional errors can be resolved than.

jkroepke and others added 2 commits July 28, 2024 08:30
Signed-off-by: Jan-Otto Kröpke <[email protected]>
Signed-off-by: Jan-Otto Kröpke <[email protected]>
@ivanahuckova
Copy link
Member

Hi @jkroepke this is blocked on our side, due to internal discussions on authentication architecture, which may take a little while. We understand that you put a lot of effort in this PR and we are sorry it is taking so long.

@gabor gabor removed their request for review August 12, 2024 10:55
@mrmiyagietas
Copy link

Hi @jkroepke @yesoreyeram @ivanahuckova
Found this PR when i tried to solve my issue: currently blocked since i cannot integrate Azure Key Vaults into Grafana because of problems with OAuth2 integration (cannot set the "Scopes" to https://vault.azure.net/.default). Seems that the OAuth2 part will not be taken over into any queries. Can you tell me if this would be solved with this PR too?

Background information:
We want to scrape the Key Vault Endpoints for getting expiry dates for proper monitoring, alerting, can improve release plannings of PROD environments (automotive company with automotive customers). Would be very happy to also have this native support provided (and hope that it will also solve my problem).

@jkroepke
Copy link
Contributor Author

jkroepke commented Aug 13, 2024

Can you tell me if this would be solved with this PR too?

Yep, within this PR, scopes can be configured. You can also use Managed Identities to gain access to KeyVault password-less.

https://github.com/grafana/grafana-infinity-datasource/pull/785/files#diff-337c34e7ed021093c90e6727c53d96c01e4023e9c15b193f1cf794eb907d2d35R114-R136

@mrmiyagietas
Copy link

Can you tell me if this would be solved with this PR too?

Yep, within this PR, scopes can be configured. You can also use Managed Identities to gain access to KeyVault password-less.

https://github.com/grafana/grafana-infinity-datasource/pull/785/files#diff-337c34e7ed021093c90e6727c53d96c01e4023e9c15b193f1cf794eb907d2d35R114-R136

awesome

This would be awesome! Would solve a lot of problems and simplify several things, nowadays i would need to write custom prometheus exporter for doing this but which would lead to more effort instead of using alternatives like this one here <3

@jkroepke
Copy link
Contributor Author

nowadays i would need to write custom prometheus exporter

Not sure, if it helps: https://github.com/webdevops/azure-keyvault-exporter

@mrmiyagietas
Copy link

mrmiyagietas commented Aug 14, 2024

Not sure, if it helps: https://github.com/webdevops/azure-keyvault-exporter

Thanks for the link, already familiar with it :) Markus Blaschke is a former colleague of mine :) This looks really good but the issue here this that you also would need to dockerize it, create a helm chart, deploy it, maintain it, patch it....and you have to think about where to deploy it later. Makes it pretty difficult if you do not have one single environment but several ones without any multi tenancy setup. All about processes: In enterprise companies you then have also to take care about OSS, license and vulnerability such as security stuff.

Converting that into money would mean high costs. That is why it would be nice to use the plugin for doing that. You can automate the whole deployment via terraform, starting with Grafana, then the dashboards and last one the datasources (like this one here: https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/data_source.html) Since already mentioned above in Grafana would make the setup more attractive since i would then only need to take care about Grafana and not where i put also then the exporter -> there i would think about using Kubernetes or any other Container Solution with Prometheus integration.

@mrmiyagietas
Copy link

@gwdawson @yesoreyeram

cat

@jkroepke
Copy link
Contributor Author

jkroepke commented Sep 24, 2024

@mrmiyagietas we wrote a custom proxy between grafana-infinity-datasource and the upstream URL which injects the authentication and may do other things, e.g. pagination.

In other words, I give up here.

@yesoreyeram yesoreyeram self-assigned this Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

5 participants