Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure the snapshot directory #1623

Open
ghill-fusion opened this issue May 12, 2022 · 3 comments
Open

Configure the snapshot directory #1623

ghill-fusion opened this issue May 12, 2022 · 3 comments
Labels
feature a feature request or enhancement snapshot 📷

Comments

@ghill-fusion
Copy link

I'm working on a project that's using snapshot tests. For confidentiality reasons, we can't commit the snapshots to the main GitHub repository as the documentation recommends. Instead, we've set up another repo (testdata), hosted on an on-premises server. This repo should include test inputs and CSV-formatted test outputs in addition to the snapshot files themselves. I figured the easiest way to set this up would be to put the _snaps directory inside the local checkout of testdata. But I couldn't see a way to configure the snapshot directory; it seems to be hardcoded to tests/testthat/_snaps (at least in the publicly facing functions; the internals all seem to let you pass in a snapshot directory).

Is a configuration setting to change the snapshot directory something that can be added? Or is there a reasonable workaround for this use case that fits with the spirit of testthat?

Other options I've thought of/tried:

  • Just using tests/testthat/_snaps as the local checkout of testdata. Doesn't work because testthat clears out the snapshot directory after running the tests, obliterating all the test inputs and CSV-formatted outputs.
  • Making tests/testthat the local checkout of testdata. Doesn't work because now the tests themselves get caught up in the testdata repo, and I can't ignore them without also ignoring them in the main repo.
  • Automatically copying the files back and forth. This seems to have timing issues; I can't see how to automatically copy existing snapshots from testdata before they get locked in as the reference snapshots, and copying new snapshots back into testdata should really happen at the same time as snapshot_accept, so we'd have to use our own function instead of snapshot_accept.
  • Replacing _snaps with a symlink to testdata/_snaps. Unfortunately, we're stuck on Windows workstations that require administrator permission to make symlinks. I don't want to add "go find an administrator" as a step in the setup procedure, there are enough barriers to adopting a testing workflow as it is.

In any case, all of these seem unnecessarily convoluted compared to just letting me choose where _snaps goes.

@scottporter
Copy link

We also have problems with this. We like to use testthat even for code that isn't an R package. This code gets executed in a cloud environment, and when the code is checked out from our git repo, it's checked out as read-only. Even though we aren't even using snapshots, the _snaps folders that get written cause us grief.

@hadley hadley added feature a feature request or enhancement snapshot 📷 labels Sep 19, 2022
@aronatkins
Copy link

Similar to @scottporter - our CI runs against R code outside an R package that is in a read-only directory structure. We want to use the _snaps directory, but treat that location as read-only. We would like failures only reported to screen, or written to some alternate location.

@nanxstats
Copy link

In case it helps anyone before a proper solution appears, here is a simple workaround to customize the snapshot directory by monkey patching testthat.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature a feature request or enhancement snapshot 📷
Projects
None yet
Development

No branches or pull requests

5 participants