Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use .julia/scratchspaces instead of .julia/datadeps #167

Merged
merged 4 commits into from
Jan 19, 2024
Merged

use .julia/scratchspaces instead of .julia/datadeps #167

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fe5dc95
use .julia/scratchspaces instead
IanButterworth Dec 1, 2023
ee16c3d
set during init
IanButterworth Dec 1, 2023
d114733
Update src/locations.jl
IanButterworth Dec 13, 2023
3c28d82
pushfirst to standard_loadpath during init
IanButterworth Dec 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@ HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
Reexport = "189a3867-3050-52da-a836-e630ba90ab69"
SHA = "ea8e919c-243c-51af-8825-aaa63cd721ce"
Scratch = "6c6a2e73-6563-6170-7368-637461726353"
p7zip_jll = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

[compat]
ExpectationStubs = "0.2"
HTTP = "1"
Reexport = "0.2, 1.0"
Scratch = "1"
julia = "1.6"
p7zip_jll = "16.2.0, 17"

Expand Down
6 changes: 3 additions & 3 deletions docs/src/z10-for-end-users.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ Moving data is a great idea.
DataDeps.jl is in favour of moving data.
When data is automatically downloaded it will almost always go to the same location:
the first (existent, writable) directory on your `DATADEPS_LOAD_PATH`.
Which by-default is `~/.julia/datadeps/`.
(If you delete this, it will go to another location).
Which by-default is DataDeps's scratch space under `~/.julia/scratchspaces/124859b0-ceae-595e-8997-d05f6a7a8dfe/datadeps`,
such that `Pkg.gc()` can automatically delete data if DataDeps is uninstalled.
But you can move them from there to anywhere in the `DATADEPS_LOAD_PATH`. (See below)

If you have a large chunk of data that everyone in your lab is using (e.g. a 1TB video corpora),
Expand Down Expand Up @@ -47,7 +47,7 @@ You can (and should when desired) move things around between any folder in the l
For the user **oxinabox**

```bash
/home/wheel/oxinabox/.julia/datadeps
/home/wheel/oxinabox/.julia/scratchspaces/124859b0-ceae-595e-8997-d05f6a7a8dfe/datadeps
/home/wheel/oxinabox/datadeps
/scratch/datadeps
/staging/datadeps
Expand Down
8 changes: 8 additions & 0 deletions src/DataDeps.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ using p7zip_jll

using HTTP
using Reexport
using Scratch
@reexport using SHA

export DataDep, ManualDataDep
Expand All @@ -29,6 +30,13 @@ include("preupload.jl")
include("fetch_helpers.jl")
include("post_fetch_helpers.jl")

# populated by __init__()
datadeps_scratch_dir = ""

function __init__()
global datadeps_scratch_dir = @get_scratch!("datadeps")
pushfirst!(standard_loadpath, datadeps_scratch_dir)
end

function _precompile_()
ccall(:jl_generating_output, Cint, ()) == 1 || return nothing
Expand Down
5 changes: 4 additions & 1 deletion src/locations.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
## Core path determining stuff

const standard_loadpath = joinpath.([
Base.DEPOT_PATH; homedir(); # Common all systems
homedir(); # Common all systems

@static if Sys.iswindows()
vcat(get.(Ref(ENV),
Expand All @@ -15,6 +15,9 @@ const standard_loadpath = joinpath.([
"/usr/share", "/usr/local/share"] # Unix Filestructure
end], "datadeps")

# NOTE: the scratchspace is pushed to the front during __init__()


# ensure at least something in the loadpath exists when instaleld
mkpath(first(standard_loadpath))

Expand Down
Loading