[Request deployment] New Hub: Neurohackedemy 2023

[colliand]

The GitHub handle of the community representative

arokem

Hub important dates

• Target Start Date: 2023-06-30
• Required Start Date: 2023-07-07
• Important usage dates: The event will take place August 7-18, 2023.

Hub Authentication Type

GitHub (e.g., @MyGitHubHandle)

First Hub Administrators

• Ariel Rokem, @arokem

[GitHub Auth only] How would you like to manage your users?

Allowing members of a specific GitHub organization

[GitHub Teams Auth only] Profile restriction based on team membership

pending

Hub logo image URL

pending

Hub logo website URL

https://neurohackademy.org/

Hub user image GitHub repository

pending

Hub user image tag and name

pending

Extra features you would like to enable

• Dedicated Kubernetes cluster
• Scalable Dask Cluster

(Optional) Preferred cloud provider

AWS

(Optional) Billing and Cloud account

None

Other relevant information to the features above

The GitHub organization that will be used to manage this hub: https://github.com/NeuroHackademy-2023

This is a request for a DaskHub on a shared cluster.

2i2c operated a similar hub for NeuroHacademy 2022. See #1300, #1505, and #1532, #1536, #1620 for additional background.

Tasks to deploy the hub

• 1. Deploy information filled in above
• 2. Engineer who will deploy the hub is assigned
• 3. If using GitHub Orgs/Teams Auth, Engineer is given Owner rights to the org to set this up.
• 4. Initial Hub deployment PR
• 5. Administrators able to log on -> Hub now in steady-state

[colliand]

Hi @arokem!

• Can you please share a URL that points to the splash image you'd like shown on the Neurohackademy 2023 login page? For example, see the openscapes logo at the top of this page.
• Can you please share a URL that points to the software image that should be used for your hub? I believe that last year you assembled the software image here.

[arokem]

Hi @colliand! Thanks for kicking this off.

Can you please share a URL that points to the splash image you'd like shown on the Neurohackademy 2023 login page? For example, see the openscapes logo at the top of this page.

We can use this: https://jinhyuncheong.com/assets/post15/neurohackademylogo.png

Can you please share a URL that points to the software image that should be used for your hub? I believe that last year you assembled the software image here.

I have not made any adjustments yet, but this is where I am planning to customize the image: https://github.com/NeuroHackademy-2023/jhub-image

Regarding extra features, I would also like to have the same additional web-page that we had last year (along the lines of this conversation). To remind: this is a single page that should be accessible only to authenticated users of the hub and should be pulled from a private repo every time the user restarts their server.

[damianavila]

@sgibson91 is going to take care of this deployment in the current sprint.

@colliand, in the above request, you are asking for a daskhub in AWS, but given the last response from @arokem, it seems he wants pretty much the same hub he had before which was a basehub in a shared GCP cluster (as per https://github.com/2i2c-org/infrastructure/pull/1721/files), can you clarify the discrepancy?

Btw, Sarah, if you have any further doubts or questions, feel free to ping @colliand on this very same issue so he can secure answers to those questions (and I presume @arokem will also reply when he is available).

[arokem]

@colliand, in the above request, you are asking for a daskhub in AWS, but given the last response from @arokem, it seems he wants pretty much the same hub he had before which was a basehub in a shared GCP cluster (as per https://github.com/2i2c-org/infrastructure/pull/1721/files), can you clarify the discrepancy?

We are not strongly attached to either. Some of the datasets we are interested in using are accessible through the AWS Open Data program, so we'd be happy to be on AWS. But whatever is easier for y'all works for us.

[colliand]

I will monitor and provide assistance as I am able. Thanks Ariel for replying to this clarification requests.

[sgibson91]

So probably the simplest thing for us from an engineering standpoint, is to just revert the PR for the old hub on GCP. As well as this one for the special nodepool https://github.com/2i2c-org/infrastructure/pull/1726/files

[sgibson91]

I have opened PR #2762 to add the hub. @arokem I've asked for some feedback from you regarding various links, etc.

I tried to add a dedicated nodepool as before in #2758 but ran into troubles with terraform.

[sgibson91]

The hub is up at https://neurohackademy.2i2c.cloud - I currently don't have the static site working yet though

[sgibson91]

I tried to add a dedicated nodepool as before in #2758 but ran into troubles with terraform.

This was resolved yesterday by Yuvi

[sgibson91]

Current plan:

• Review and merge Add neurohackademy hub #2762 without the static site config for now
• Timebox investigations into network policies blocking the dex/oauth2-proxy pod to one week
  • @yuvipanda to do some initial investigations into this on Wednesday and report back

[damianavila]

AFAIK, reopened for:

Timebox investigations into network policies blocking the dex/oauth2-proxy pod to one week
@yuvipanda to do some initial investigations into this on Wednesday and report back

[yuvipanda]

I've suggested a different plan in #2848, and wrote up docs on how to do it in #2849. I'll ping @sgibson91 and ask if she can pick this up.

[sgibson91]

Hi @arokem. I have a PR ready to go with Yuvi's alternative method in #2852 but it definitely won't work until you have git-credentials-helper installed in your image

[arokem]

Great! Added in NeuroHackademy-2023/jhub-image@894883d

[sgibson91]

Hmmm, we are receiving a 128 error code

• https://neurohackademy.2i2c.cloud/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2FNeuroHackademy-2023%2Fzoom&urlpath=lab%2Ftree%2F%2F&branch=main

[I 2023-07-21 12:51:43.356 SingleUserLabApp handlers:38] Traceback (most recent call last):
    File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbgitpuller/handlers.py", line 79, in get
        gp = GitPuller(repo, repo_dir, branch=branch, depth=depth, parent=self.settings['nbapp'])
    File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbgitpuller/pull.py", line 77, in __init__
        elif not self.branch_exists(self.branch_name):
    File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbgitpuller/pull.py", line 89, in branch_exists
        heads = subprocess.run(
    File "/srv/conda/envs/notebook/lib/python3.10/subprocess.py", line 526, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['git', 'ls-remote', '--heads', '--', 'https://github.com/NeuroHackademy-2023/zoom']' returned non-zero exit status 128.

@arokem can you confirm that https://github.com/apps/2i2c-neurohackademyhub-static-pull is installed correctly on the NeuroHackademy-2023/zoom repo and isn't requesting updated permissions please?

[arokem]

Does this look correct?

[arokem]

TBC, I didn't change anything just now.

[sgibson91]

Hmmmm, yes that looks correct

I didn't really expect you to need to change anything but I wanted to make sure. Thank you.

[yuvipanda]

am takin a look

[sgibson91]

@arokem we got it working! The following link will open up the list of zoom links for the event:

• https://neurohackademy.2i2c.cloud/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2FNeuroHackademy-2023%2Fzoom&urlpath=lab%2Ftree%2Fzoom%2Findex.html&branch=main

Whenever someone visits just https://neurohackademy.2i2c.cloud, the curriculum repo will be pulled

[arokem]

Yep - I see that now! Thank you for setting that up!

[sgibson91]

No problem! I'm going to close this issue now, please let us know if you need anything else via our support process :)