Full Cilogon integration #1089

GeorgianaElena · 2022-03-11T23:39:54Z

This PR adds another way to use CILogon auth for our hubs, one that doesn't require relying on Auth0.

It adds a script, cilogon_app.py that:

uses the administrative client credentials that we received from CILogon to create/udpate/delete/get approved OAuth applications for our hubs.
stores and ecrypts the credentials of each hub in `config/clusters/<cluster_name>/enc-<hub_name>.secret.values.yaml

The process to activate CILogon for a hub using the credentials generated above is very similar to the current process of using GitHubOAuthenticator.

Also, this PR enables CILogon for our 2i2c dask-staging hub and 2i2c demo hub. It also adds credentials for the 2i2c staging hub also, but without activating it for this hub.

The dask-staging hub uses this, so you can play with it. It works from what I've tested so far.

Reference #967

deployer/deploy_actions.py

config/clusters/2i2c/enc-cilogon-clients.secret.yaml

choldgraf

This looks nice to me! A few comments and questions. I'll let others on the @2i2c-org/tech-team give a more technical review!

Could we add the start of some lightweight docs for how we'd use CILogon auth? I think it could be helpful exercise to think about the changes from a user's perspective. It doesn't need to be totally complete since I know you'd like to get the PR in quickly, but something to help people start would be useful.
Given what you've seen, are there any major decision-points that you can think of where we'd want to use Auth0 vs. CILogon?
Have you encountered any major downsides or concerns about this approach? Or have you encountered any major benefits?

choldgraf · 2022-03-15T03:04:56Z

config/clusters/2i2c/cluster.yaml

-      connection: google-oauth2
+      enabled: false
+      # connection: google-oauth2
+    cilogon:


I like that we are allowing them to co-exist for a bit! As long as we don't have too much headaches maintaining both, this should give us time to try out CILogon without disrupting our current workflows as much

deployer/cilogon_auth.py

yuvipanda

Thank you so much for working on this, @GeorgianaElena!!!!

The primary reason the auth0 stuff was baked into the deployer was so we won't have to keep secret files in the repo. However, given that is not possible with CILogon (we can not retrieve secret key multiple times), I think we should instead change this to something where the script just spits out appropriate sops encrypted files as enc-<hub-name>.secret.values.yaml file (just using the CILogon authenticator), which we can just check-in. This also simplifies the deployer (once we can get rid of auth0 at least), and enhances right to replicate.

So we could have something like a new-hub script or cookie-cutter that talks to the CILogon API, generates the appropriate secrets and puts it in place. I think the code for this already exists here.

Once we move off auth0 completely, we can just get rid of the dynamic auth part of the deployer completely.

GeorgianaElena · 2022-03-17T17:05:30Z

Thanks for the feedback @yuvipanda. I just pushed some changes with the cilogon client provider turned into a script.
It's not ready, but I would appreciate an early feedback so that I don't get into another rabbit hole.

Also, weird thing, I just got a 500 error with a message saying that the cilogon credentials I was trying to use don't have an "admin" role. @choldgraf do you mind asking the CILogon folks if they've revoked our admin credentials? I'm afraid they think we've abused their system since I've created quite a few dummy clients while testing the implementation 😕

requests.exceptions.JSONDecodeError: [Errno Expecting value] error="server_error"
error_description="Error%3A+the+given+id+of+%22cilogon%3A%2Fclient_id%<encrypted-client-id>+is+not+recognized+as+an+admin+client."
: 0

choldgraf · 2022-03-17T17:50:40Z

@GeorgianaElena will send them an email today!

GeorgianaElena · 2022-03-17T18:45:11Z

Thanks a lot @choldgraf 🌼

yuvipanda · 2022-03-18T10:25:30Z

@GeorgianaElena i haven't done a deep dive into the code but overall this seems the right direction to me!

for more information, see https://pre-commit.ci

Co-authored-by: Sarah Gibson <[email protected]>

…ams are not implemented

…cilogon

for more information, see https://pre-commit.ci

…ot-hubs into cilogon-admin

GeorgianaElena · 2022-03-23T13:07:40Z

@2i2c-org/tech-team, this should be now ready for a new review 👀 or a merge

We can put default values there even if they aren't enabled

yuvipanda · 2022-03-23T22:41:37Z

@GeorgianaElena i've left some suggested changes in GeorgianaElena#2! I'm going to now test deploying this on the demo hub.

yuvipanda · 2022-03-23T23:05:20Z

GeorgianaElena#3 is me trying to deploy this to the demo hub, but ran into problems :( I've detailed them in the PR. I think if we can resolve this, this is good to go.

This is such a huge step forward, thank you for working on this @GeorgianaElena!

yuvipanda · 2022-03-23T23:12:01Z

The test failure is a link to a file introduced in this PR, so is alright to let fail.

Suggested changes

docs/howto/configure/auth-management.md

Co-authored-by: Georgiana Elena <[email protected]>

Move demo hub to use cilogon directly

Continue to rely on username_pattern right now as we figure out how to use IDPs

yuvipanda

@GeorgianaElena this is super awesome, am very excited to get this to land!

Remove allowed_idps for demo hub GeorgianaElena/pilot-hubs#4 keeps status quo for demo hub, and we can figure out what to do with IDPs
Remove CILogonOAuthenticator default scopes GeorgianaElena/pilot-hubs#5 removes some extra config
Can you update the PR description here too?

I think you can merge after these two are done.

These are already actually set in the base CILogonOAuthenticator itself! https://github.com/jupyterhub/oauthenticator/blob/8c1cf0c616d5e19d1f93238673f17db2daf57fa0/oauthenticator/cilogon.py#L69

Remove allowed_idps for demo hub

Remove CILogonOAuthenticator default scopes

GeorgianaElena · 2022-03-25T12:13:10Z

Merging this now! THANK YOU! 🚀

choldgraf · 2022-03-25T17:03:12Z

damianavila · 2022-04-05T14:02:17Z

Lovely piece of work, @GeorgianaElena!!

GeorgianaElena changed the title ~~Cilogon admin~~ Full Cilogon integration Mar 11, 2022

GeorgianaElena marked this pull request as draft March 11, 2022 23:41

GeorgianaElena marked this pull request as ready for review March 14, 2022 16:24

GeorgianaElena requested a review from a team March 14, 2022 16:24

sgibson91 reviewed Mar 14, 2022

View reviewed changes

deployer/deploy_actions.py Outdated Show resolved Hide resolved

consideRatio reviewed Mar 14, 2022

View reviewed changes

config/clusters/2i2c/enc-cilogon-clients.secret.yaml Outdated Show resolved Hide resolved

choldgraf reviewed Mar 15, 2022

View reviewed changes

choldgraf mentioned this pull request Mar 15, 2022

Simplify UToronto hub deployment so it is easier to manage #1088

Closed

yuvipanda requested changes Mar 16, 2022

View reviewed changes

GeorgianaElena marked this pull request as draft March 17, 2022 16:21

GeorgianaElena and others added 16 commits March 21, 2022 17:18

Temporarly use CILogon JupyterHub OAuthenticator for the staging hub

f2e46a3

First version of a cilogon client provider

6540d37

Add schema defs for cilogon

6bddb3b

Enable new CIlogon for dask-staging

ebdee91

Add interface class

d6df2a9

Update Auth0 class to follow base class

8d17070

Create the appropriate client provider based on config

c04965c

Update generated config to suport cilogon

a7d10e4

Working verison of the CILogon client provider

7027cf7

Add encrypted file storing first hubs client and secret

63e7ab9

[pre-commit.ci] auto fixes from pre-commit.com hooks

d007c77

for more information, see https://pre-commit.ci

Update deployer/deploy_actions.py

fd5f18d

Co-authored-by: Sarah Gibson <[email protected]>

Make allowed_idps a list

d517f2c

Raise errors if functions we call and for which we assume certain par…

1343e88

…ams are not implemented

Rename to reflect there can be multiple connections now that we have …

708a455

…cilogon

[pre-commit.ci] auto fixes from pre-commit.com hooks

9b4f48c

for more information, see https://pre-commit.ci

pre-commit-ci bot and others added 3 commits March 23, 2022 11:47

[pre-commit.ci] auto fixes from pre-commit.com hooks

adeba0f

for more information, see https://pre-commit.ci

Fix location

d51df2a

Merge branch 'cilogon-admin' of https://github.com/GeorgianaElena/pil…

85c6449

…ot-hubs into cilogon-admin

yuvipanda added 4 commits March 23, 2022 14:43

Move CILogonOAuthenticator default config to values.yaml

e98cae3

We can put default values there even if they aren't enabled

Add warning about picking USERNAME_KEY

2094892

Invoke python as python3 rather than just as python

09925bf

Fix typo in name of cilogon script

d18ed4f

Move demo hub to use cilogon directly

ca3631c

Merge pull request #2 from yuvipanda/cilogon-admin-suggestions

012a084

Suggested changes

consideRatio reviewed Mar 24, 2022

View reviewed changes

docs/howto/configure/auth-management.md Outdated Show resolved Hide resolved

yuvipanda and others added 3 commits March 24, 2022 15:09

Fix typo

f23cce4

Co-authored-by: Georgiana Elena <[email protected]>

Merge pull request #3 from yuvipanda/cilogon-admin-demo-try

b56efc4

Move demo hub to use cilogon directly

Remove allowed_idps for demo hub

7305f63

Continue to rely on username_pattern right now as we figure out how to use IDPs

yuvipanda approved these changes Mar 24, 2022

View reviewed changes

yuvipanda mentioned this pull request Mar 24, 2022

Write documentation on where to look for config #1144

Closed

Remove CILogonOAuthenticator default scopes

aadf12f

These are already actually set in the base CILogonOAuthenticator itself! https://github.com/jupyterhub/oauthenticator/blob/8c1cf0c616d5e19d1f93238673f17db2daf57fa0/oauthenticator/cilogon.py#L69

yuvipanda mentioned this pull request Mar 24, 2022

Investigate using CILogon instead of Auth0 #967

Closed

6 tasks

GeorgianaElena added 3 commits March 25, 2022 13:50

Merge pull request #4 from yuvipanda/cilogon-admin-2

4b9d523

Remove allowed_idps for demo hub

Merge pull request #5 from yuvipanda/cilogon-admin-3

44fb3ef

Remove CILogonOAuthenticator default scopes

Print clint list preetier

939a91c

GeorgianaElena merged commit dc43c6e into 2i2c-org:master Mar 25, 2022

GeorgianaElena deleted the cilogon-admin branch March 28, 2022 10:28

choldgraf mentioned this pull request Apr 2, 2022

[blog] 2022 - Q1 highlights 2i2c-org/team-compass#405

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Full Cilogon integration #1089

Full Cilogon integration #1089

GeorgianaElena commented Mar 11, 2022 •

edited

Loading

choldgraf left a comment

choldgraf Mar 15, 2022

yuvipanda left a comment

GeorgianaElena commented Mar 17, 2022 •

edited

Loading

choldgraf commented Mar 17, 2022

GeorgianaElena commented Mar 17, 2022

yuvipanda commented Mar 18, 2022

GeorgianaElena commented Mar 23, 2022 •

edited

Loading

yuvipanda commented Mar 23, 2022

yuvipanda commented Mar 23, 2022

yuvipanda commented Mar 23, 2022

yuvipanda left a comment •

edited

Loading

GeorgianaElena commented Mar 25, 2022

choldgraf commented Mar 25, 2022

damianavila commented Apr 5, 2022

Full Cilogon integration #1089

Full Cilogon integration #1089

Conversation

GeorgianaElena commented Mar 11, 2022 • edited Loading

choldgraf left a comment

Choose a reason for hiding this comment

choldgraf Mar 15, 2022

Choose a reason for hiding this comment

yuvipanda left a comment

Choose a reason for hiding this comment

GeorgianaElena commented Mar 17, 2022 • edited Loading

choldgraf commented Mar 17, 2022

GeorgianaElena commented Mar 17, 2022

yuvipanda commented Mar 18, 2022

GeorgianaElena commented Mar 23, 2022 • edited Loading

yuvipanda commented Mar 23, 2022

yuvipanda commented Mar 23, 2022

yuvipanda commented Mar 23, 2022

yuvipanda left a comment • edited Loading

Choose a reason for hiding this comment

GeorgianaElena commented Mar 25, 2022

choldgraf commented Mar 25, 2022

damianavila commented Apr 5, 2022

GeorgianaElena commented Mar 11, 2022 •

edited

Loading

GeorgianaElena commented Mar 17, 2022 •

edited

Loading

GeorgianaElena commented Mar 23, 2022 •

edited

Loading

yuvipanda left a comment •

edited

Loading