Misc updates on testing hub server spawing of binderhubs and documentation

[GeorgianaElena]

Fixes #4167

This is currently failing for unauthenticated binderhubs because of some issue with the token being passed to /user/api/kernelspecs or dropped. Seeing this in the notebook logs:

[I 2024-06-20 14:22:15.949 ServerApp] 302 GET /user/deployment-service-check/ (@192.168.15.175) 0.51ms
[W 2024-06-20 14:22:21.607 ServerApp] wrote error: 'Forbidden'
    Traceback (most recent call last):
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/tornado/web.py", line 1788, in _execute
        result = method(*self.path_args, **self.path_kwargs)
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/tornado/web.py", line 3289, in wrapper
        url = self.get_login_url()
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 782, in get_login_url
        raise web.HTTPError(403)
    tornado.web.HTTPError: HTTP 403: Forbidden
[W 2024-06-20 14:22:21.608 ServerApp] 403 GET /user/deployment-service-check/api/kernelspecs (@192.168.14.203) 1.89ms referer=None

But in that get request, the token should be passed because it's done under the same session here

https://github.com/Quansight/jhub-client/blob/860dff7365bdccc5208724390529b00b8198ccef/jhub_client/api.py#L226

Which gets called by https://github.com/Quansight/jhub-client/blob/860dff7365bdccc5208724390529b00b8198ccef/jhub_client/execute.py#L87
so not sure what's that about.

Can it be because of aio-libs/aiohttp#5783? I do see a 302 in the logs

[GeorgianaElena]

This should be special-cased for hubs not requiring authentication because of where the Spawner expects to find the token:

infrastructure/helm-charts/basehub/values.yaml

Lines 901 to 910 in 6214ce1

	if self.auth_enabled:
	args = super().get_args()
	else:
	args = [
	"--ip=0.0.0.0",
	f"--port={self.port}",
	f"--NotebookApp.base_url={self.server.base_url}",
	f"--NotebookApp.token={self.user_options['token']}",
	"--NotebookApp.trust_xheaders=True",
	]

[GeorgianaElena]

This is what binderhub does too. Otherwise, in images that don't have jupyter-lab installed, spawning a server will fail

[GeorgianaElena]

Based on @yuvipanda's feedback in #4167 I've updated this PR to:

• skip health tests for binderhubs that don't use authentication
• updated the singleuser cmd to cover the cases when jupyter-lab is not installed in the image being launched (similar to what binderhub does) and updated the docs
• updated the docs to mention that in the cluster.yaml we should enter the hub's domain rather than the binderhub's so that the tests work (for authenticated setups)

[github-actions]

🎉🎉🎉🎉

Monitor the deployment of the hubs here 👉 https://github.com/2i2c-org/infrastructure/actions/runs/9612884778