Override scratch directory with preference or environmental variable

[kescobo]

Pursuant to a discussion on Slack, this PR adds the ability to specify an alternate location for scratch directories using Preferences.jl or using an environmental variable. My personal reason for needing this is that for large files on HPC, I typically want to put stuff on filesystems that are specifically meant for scratch data / fast access, which is often a different
filesystem than $HOME

Possible sources of bikeshedding:

• Is it ok to take on Preferences as a dependency? If not, I still think this would be useful with just the environmental variable option (I couldn't think of a way to do this with a conditional dependency, but I'm happy to take suggestions there).
• The names of the preference / env variable. I currently have "scratch_dir" and JULIA_SCRATCH_DIR respectively, but don't feel strongly.

Obviously happy to take any other comments / suggestions, but if this approach is amenable, I can start working on docs and tests.

[codecov]

Codecov Report

Merging #43 (df6f59e) into master (7df2dd3) will decrease coverage by 0.59%.
Report is 1 commits behind head on master.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##           master      #43      +/-   ##
==========================================
- Coverage   87.61%   87.03%   -0.59%     
==========================================
  Files           1        1              
  Lines         105      108       +3     
==========================================
+ Hits           92       94       +2     
- Misses         13       14       +1     

Files	Coverage Δ
src/Scratch.jl	87.03% <75.00%> (-0.59%)	⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[kescobo]

No idea why this is failing on nightly :-(

[StefanKarpinski]

I don't know that this can be done in this way. One of the requirements I tried to convey on Slack was that even when this option isn't set, it has to be possible to find the places where scratch spaces have been placed because this option was used. We could record it in a log file in the depot or something but there has to be some record of it so that it can be cleaned up by another (potentially automated) Julia process without knowledge of this option.

[kescobo]

Something like the way Pkg keeps track of all of the environments for GC? I can dig around and see how that's done 👍

[StefanKarpinski]

Yes, exactly like that. Some log file like ~/.julia/logs/scratch_usage.toml or similar.

[kescobo]

Just a heads up - given the higher complexity and my current life circumstances, it may be a while until I can address this adequately. I'm happy to leave the PR open or close it, but I don't want to be a cookie licker in case someone else wants this feature and can implement it.

[fingolfin]

Just a note orthogonal to this PR, in case it helps @kescobo: depending on your setup, perhaps you could also try to turn ~/.julia/scratchspaces into a symlink to a dir on a separate filesystem. I do something similar (actually, for me ~/.julia is a symlink -- my homedir is on NFS and thus small and slow; but on all machines I used I have a stable (local) scratch dir in a fixed location.