Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: upgrade R and RStudio versions #218

Merged
merged 14 commits into from
Feb 7, 2022

Conversation

gavin-k-lee
Copy link
Contributor

@gavin-k-lee gavin-k-lee commented Jan 25, 2022

We have been out-of-date with R/RStudio for a while now (#154 & #160) due to external dependencies. Since these external dependencies have been resolved, we can build newer versions based on Jupyter proxies.

@olevski
Copy link
Member

olevski commented Jan 31, 2022

@gavin-k-lee I think all the pieces (i.e. dependent libraries) that needed new releases for this to work are now done. I did some tests in a different branch. Are you ok for me to transfer these changes here so that we can have images with newer R and Rstudio versions?

@olevski
Copy link
Member

olevski commented Jan 31, 2022

Some related issues we can close with this: #154 and #160.

@gavin-k-lee
Copy link
Contributor Author

gavin-k-lee commented Feb 1, 2022

@olevski yes sure we can transfer them or make a new PR and scrap this one. up to you, i don't mind.

btw on an un-related note, was there any reason why we are building bioconductor-renku images as well? @rokroskar historical reason?

@gavin-k-lee gavin-k-lee changed the title Rocker 4.1.2 rstudio 1.4.1717 3 upgrade R and RStudio versions Feb 1, 2022
@rokroskar
Copy link
Member

@gavin-k-lee the bioc image was requested some time back - it takes a long time to build so it was nice to have a base image as a starting point.

docker/r/Dockerfile Outdated Show resolved Hide resolved
@olevski
Copy link
Member

olevski commented Feb 1, 2022

Also this is a good guide on the insanity that is R session environment variables. If I change only the Renviron file the path is not updated. I could only get it to work when I change Renviron.site - and that is what is actually recommended.

See: https://rviews.rstudio.com/2017/04/19/r-for-enterprise-understanding-r-s-startup/

@rokroskar
Copy link
Member

@olevski yes the RStudio environment setup is mind-boggling. It basically ignores any reasonable convention that you might expect 😭 🤦

@rokroskar
Copy link
Member

Though I'm a bit confused why adding the paths to Renviron.site is needed - it used to work as is

@olevski olevski force-pushed the rocker-4.1.2-rstudio-1.4.1717-3 branch from 7b018d4 to e524791 Compare February 1, 2022 23:26
@gavin-k-lee
Copy link
Contributor Author

btw I think the bioc builds might fail since the R and RBioc versions aren't corresponding exactly.
Bioc 3.11 ~ R 4.0.0
Bioc 3.12 ~ R 4.0.3
Bioc 3.13 ~ R 4.1.0
Bioc 3.14 ~ R 4.1.1

@olevski olevski force-pushed the rocker-4.1.2-rstudio-1.4.1717-3 branch from e524791 to 0422a7e Compare February 2, 2022 00:29
@olevski
Copy link
Member

olevski commented Feb 2, 2022

@gavin-k-lee the bioc builds were failing because of mistakes (that I just introduced) in the RSTUDIO_VERSION environment variable not being properly defined in the dockerfile. They should all pass now.

@olevski olevski force-pushed the rocker-4.1.2-rstudio-1.4.1717-3 branch 7 times, most recently from 645ede5 to 463e898 Compare February 2, 2022 02:48
@olevski olevski force-pushed the rocker-4.1.2-rstudio-1.4.1717-3 branch from 463e898 to ade8dce Compare February 2, 2022 02:58
@olevski olevski changed the title upgrade R and RStudio versions chore: upgrade R and RStudio versions Feb 3, 2022
@olevski olevski marked this pull request as ready for review February 3, 2022 09:20
@olevski
Copy link
Member

olevski commented Feb 3, 2022

Ok this is ready to go now.

Here is a test project with bioconductor 13: https://dev.renku.ch/projects/tasko.olevski/test-image-upgrade-r-1

@olevski
Copy link
Member

olevski commented Feb 3, 2022

Here is a project with just R 4.1.0: https://dev.renku.ch/projects/tasko.olevski/test-r-project-1

@olevski
Copy link
Member

olevski commented Feb 3, 2022

Both of the test projects work!

@gavin-k-lee
Copy link
Contributor Author

gavin-k-lee commented Feb 3, 2022

nice! I just tried both projects and when opening in an external tab it worked great and I tested the plotting (which was the most prominent problem before). In the iframe RStudio didn't load if i didn't touch it, but if i refreshed it took a long while to load - not sure if you experience that all? Maybe it is a resource issue?

@olevski
Copy link
Member

olevski commented Feb 3, 2022

@gavin-k-lee I have seen the loading issues in the iframe you mention on other versions (in production) too. I am pretty sure it is related to resource usage. I never saw it in test but then I made sure to request 1cpu with the sessions - not sure if that helped really. I think the cpu should not be limited in k8s.

Either way I think this is a problem that shows up sometimes and is not a result of the new images.

@olevski
Copy link
Member

olevski commented Feb 3, 2022

On another note...

I def lost some sanity just trying to get these images to build btw. I did notice that building the dockerfiles worked differently on my mac locally and in github CI. But then after I switched to my personal latptop (which runs linux) I could finally get the same outcomes I see in the CI workflows locally and eventually fix them. Just a PSA if some else is having such weird issues.

@gavin-k-lee
Copy link
Contributor Author

gavin-k-lee commented Feb 3, 2022

yeah it seems like you really need to build them with linux to have them equivalent, which is annoying.

@SwissDataScienceCenter/renku-ui-maintainers would we be able to get some help to understand what's going on with the iframes? I tried the highest cpu and ram on dev and the iframe didn't load (but the new tab worked as expected).
I just tried the R 4.1 image on renkulab: https://renkulab.io/projects/lee.gavin.k/test-renku-r with Resources 2 cpu | 4G memory | 1G storage or Resources 1 cpu | 2G memory | 1G storage and it loaded up in the iframe quickly, so it's likely a resource issue. Strange though.

@gavin-k-lee
Copy link
Contributor Author

gavin-k-lee commented Feb 4, 2022

I think we can also retire some of the 4.0.X images and bioc images. We could go with:
R 4.1.0, 4.1.1, 4.1.2, devel
Bioc 3.13, 3.14, devel

R 4.0.5 came out in March 2021 and Bioc 3.12 in October 2020.

Thoughts?

@ciyer
Copy link
Contributor

ciyer commented Feb 4, 2022

I just tried both the test-image-upgrade and the test-r-project above, and I did not see anything unusual in the iframe experience. Both projects took 40-45s to start, but I had the spinner/bouncer and then they transitioned seamlessly to RStudio once the pods were up. It seems that the problem is intermittent.

If you could note the time (UTC) when you next have issues, we can see if there is anything in the logs that sheds light on what is going on.

@olevski
Copy link
Member

olevski commented Feb 7, 2022

@rokroskar I tested the latest version of the R images with the devel images for R and bioconductor. Things work. So this is ready to go!

If you or anyone else is interested here are the projects:

Copy link
Member

@rokroskar rokroskar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is awesome, thanks @gavin-k-lee and @olevski !!!

@gavin-k-lee gavin-k-lee merged commit 79ea965 into master Feb 7, 2022
@gavin-k-lee gavin-k-lee deleted the rocker-4.1.2-rstudio-1.4.1717-3 branch February 7, 2022 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

R 4.1.0 is not compatible with RStudio 1.3.959 (plots) RStudio version is out of date
4 participants