gittargets-like functionality for raw data?

[joelnitta]

I just started using gittargets, and it's great! Thanks so much @wlandau for the excellent documentation. Git can be ahem rather confusing, but I found {gittargets} to be a breeze so far 👍

I had an idea though... {gittargets} + code = a fully reproducible project, but what about external data files ("raw data")? If these are not under version control, the reproducibility can break down. I think the exact same logic applied to the targets data store with gittargets could also be applied to a raw data cache. Perhaps a function (or even just arguments) could be added to {gittargets}?

[wlandau]

I just started using gittargets, and it's great! Thanks so much @wlandau for the excellent documentation. Git can be ahem rather confusing, but I found {gittargets} to be a breeze so far +1

Glad to hear it!

I had an idea though... {gittargets} + code = a fully reproducible project, but what about external data files ("raw data")? If these are not under version control, the reproducibility can break down. I think the exact same logic applied to the targets data store with gittargets could also be applied to a raw data cache. Perhaps a function (or even just arguments) could be added to {gittargets}?

I recommend putting those files in _targets/user so gittargets can snapshot them. (In fact, newer versions of targets automatically create this folder.) Related: https://docs.ropensci.org/gittargets/articles/git.html#custom-data-files, ropensci/gittargets#3

[joelnitta]

Ah, as usual one step ahead of me! I totally missed that in the vignette, that's exactly what I needed.

In that case should I just read in files using a path directly into _targets or is there some recommended function (akin to tar_load()) for that?

[wlandau]

The files in _targets/user/ are best combined with tar_target(..., format = "file"), so tar_load() itself is going to return file paths. Other than that, these are entirely custom files, so targets cannot make assumptions about how to load the contents into memory.

[joelnitta]

Got it, thanks!