Reduce mybinder startup time

[maxmargraf]

The binder re-builds itself after every commit whether the environment.yml is changed or not. This takes a long time and will in future, with even more dependencies even be slower. There are several ways to reduce the startup time and the most promising one from this jupyter discourse page are presented here which splits up the environment and the content in two repositories:

Use two repositories: one for the environment, one for your content - many people change their content much more often than they change the environment needed for it. However, Binder will re-build the environment for any changes to a repository. A hack to get around this is to define an “environment repository” that Binder builds, and use a hook to pull in new content at launch from a “content repository”. This means that your “environment repository” changes less-often, which should result in fewer new builds and reduced launch times. See the instructions in this post to get started.

Use the nbgitpuller.link page to automate separate content/environment repos. The above step can be (mostly) automated by using nbgitpuller.link. This is a little web form that generates JupyterHub links for you. To quickly create a link for content/environment repositories, go here:

nbgitpuller.link?tab=binder

and fill out the form.

You can also pre-populate the form with some fields. For example:

nbgitpuller.link/?tab=binder&repo=https://github.com/binder-examples/requirements

will use the binder-examples repository as the “environment” repo.

The steps towards the use of nbgitpuller should be tracked by this issue.

[georges-schutz]

The split of content and environment is started:

• There is now a new repository https://github.com/OpenSenseAction/OPENSENSE_sandbox_environment where the binder environment is going to be specified.
• There is a test https://github.com/georges-schutz/OPENSENSE_sandbox_outsource_environment where the testing of this split is done.

[cchwala]

Trying out

https://mybinder.org/v2/gh/OpenSenseAction/OPENSENSE_sandbox_environment/main?urlpath=git-pull?repo=https://github.com/OpenSenseAction/OPENSENSE_sandbox

[cchwala]

This is the one i created via filling out the form at nbgitpuller.link?tab=binder

https://mybinder.org/v2/gh/OpenSenseAction/OPENSENSE_sandbox_environment/main?urlpath=git-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252FOpenSenseAction%252FOPENSENSE_sandbox%26urlpath%3Dlab%252Ftree%252FOPENSENSE_sandbox%252Fintro_to_R.ipynb%26branch%3Dmain

[georges-schutz]

In the OPENSENSE_sandbox_outsource_environment fork has the environment removed and the README.md adapted points to the binder url using the OPENSENSE_sandbox_environment.
Try with this fork https://github.com/georges-schutz/OPENSENSE_sandbox_outsource_environment
Next is to get this button automatically created correctly when a fork is done

[cchwala]

To me it looks as if we just have to adjust the URL template here

With this URL will always reference the env defined in the main branch of the sandbox env repo. That means that, if somebody has to adjust the env, he/she has to do an updated of the env repo first.

The bot that adds the binder button should add a note that users have to take care of env updates in the env repo if they need an env update. In such a case, I would suggest to create a branch or PR of the env repo, do the update of the env there, and then manually adjust the binder URL to test if binder can build the env and if everything runs correctly.

[georges-schutz]

We are close and there is a pull request for it but the readme.md url of the binder button is not loading the notebook
Pointing only too the root is OK it only opens the file tree

[cchwala]

@georges-schutz This is the full path that they use in their README

https://mybinder.org/v2/gh/ipython/ipython-in-depth/master?filepath=binder%2FIndex.ipynb

[cchwala]

maybe like this

https://mybinder.org/v2/gh/OpenSenseAction/OPENSENSE_sandbox_environment/main?urlpath=git-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252FOpenSenseAction%252FOPENSENSE_sandbox%26urlpath%3Dlab%252Ftree%252FOPENSENSE_sandbox%252Fintro_to_R.ipynb%26branch%3Dmain

(created with nbgitpuller web form)

[georges-schutz]

The pull request PR#31 is ready to be merged back to the upstream.
There is still one other open PR and of cause several forks that may need to merge from the upstream

[cchwala]

With #31 now being merge we have the problem that the submodules are not recursively cloned.

nbgitpuller might support that, see jupyterhub/nbgitpuller#123, but I do not (yet) know why.

[cchwala]

closed via #31, but with some issues, see #30