Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixtures: files and insert-or-replace mode #2039

Closed
tiborsimko opened this issue Dec 6, 2017 · 4 comments
Closed

fixtures: files and insert-or-replace mode #2039

tiborsimko opened this issue Dec 6, 2017 · 4 comments

Comments

@tiborsimko
Copy link
Member

It seems we have a problem with files when using fixtures with --mode insert-or-replace. To be tested locally and fixed.

Here's one scenario to test:

  1. Upload some record with --skip-files --mode-insert. (so no files)
  2. Upload the same record with --mode insert-or-replace. (so with files)

Might be good to test other combinations of (1) initial record state (with or without files); and (2) update mode (replace or insert-or-replace, with or without files).

@pamfilos
Copy link
Member

pamfilos commented Dec 6, 2017

yes when replacing or updating a file with update_record, the record['files'] is replaced with files from fixtures.

Easy fix: is to just update metadata for this records..and skip files leaving them in their before_update state. Else, all files need to be removed and reloaded , if we have no clue which file is the one updated (if there is any )

ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 7, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 8, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
@tiborsimko
Copy link
Member Author

If we update only the metadata, then we would not cover the original scenario described above. It is possible that the record metadata has changed, or the file information such as some file size has changed, or some new files were added... We need to have an easy way to update records in all these scenarios.

@ioannistsanaktsidis
Copy link
Contributor

@tiborsimko the only and easy way to do this now is to remove bucket and files and recreate them again when someone asks for update . Is that ok for you ?

@tiborsimko
Copy link
Member Author

@ioannistsanaktsidis Yes, that would be perfect, since we always do a full "replace" operation, and never a partial "update".

ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 12, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 13, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 13, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 13, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
Co-authored-by: Pamfilos Fokianos <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 13, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
Co-authored-by: Pamfilos Fokianos <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 13, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
Co-authored-by: Pamfilos Fokianos <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 13, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
Co-authored-by: Pamfilos Fokianos <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 14, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
Co-authored-by: Pamfilos Fokianos <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 14, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
Co-authored-by: Pamfilos Fokianos <[email protected]>
ioannistsanaktsidis added a commit to ioannistsanaktsidis/opendata.cern.ch that referenced this issue Dec 14, 2017
 * Closes cernopendata#2039.

Signed-off-by: Ioannis Tsanaktsidis <[email protected]>
Co-authored-by: Pamfilos Fokianos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants