Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object metadata does not update if a key is removed #1468

Open
KeithKeithDev opened this issue Jan 30, 2024 · 4 comments
Open

Object metadata does not update if a key is removed #1468

KeithKeithDev opened this issue Jan 30, 2024 · 4 comments
Labels

Comments

@KeithKeithDev
Copy link

KeithKeithDev commented Jan 30, 2024

Using version: 1.47.6

I create an object with the initial custom metadata:

{foo=bar, baz=qux}

When trying to update the object's metadata with the following map (e.g. the 'foo' entry is now removed)
{baz=qux}

using the following code (as per the google gcs documentation here):

val updated = blob.toBuilder.setMetadata(metadata.asJava).build().update()
println(s"Updated metadata ${updated.getMetadata.asScala}")

The following is logged and we can see that the foo entry is still there.
Updated metadata Map(baz -> qux, foo -> bar)

However, if I update the object by changing the mapping of a single entry, so I update with:
{baz=blah}

Then I can see that the object's metadata is now updated to have just the 1 key and the foo mapping is gone, as expected:
Updated metadata Map(baz -> blah)

Is this a bug?

@fsouza fsouza added the bug label Feb 8, 2024
@fsouza
Copy link
Owner

fsouza commented Feb 8, 2024

@KeithKeithDev thanks for reporting! This is definitely unexpected.

@KeithKeithDev
Copy link
Author

KeithKeithDev commented Mar 4, 2024

@fsouza actually, I may have been making the wrong API call. To delete a key, you need to set a mapping to null for that key as per https://cloud.google.com/storage/docs/json_api#patch. So in reality, I should have called setMetadata with foo -> null.

However, I see another potential issue.

  1. I create an object with initial metadata mapping "foo" -> "bar"
  2. I run the curl command to see the object in fake-gcs-server: curl http://localhost:55007/storage/v1/b/my-bucket/o. I can see "metadata":{"foo":"bar"}
  3. In my code, I call val updated = blob.toBuilder.setMetadata(metadata.asJava).build().update() with a new metadata mapping of "baz" -> "qux".
  4. When I run the same curl command against fake-gcs-server again, to see the object's metadata, curl http://localhost:55007/storage/v1/b/my-bucket/o. I can see "metadata":{"baz":"qux"}

It looks like the original metadata is wiped. https://cloud.google.com/storage/docs/json_api#patch. Can you reproduce this and confirm if it's an issue?

@KeithKeithDev
Copy link
Author

KeithKeithDev commented Mar 4, 2024

@fsouza - also, another potential issue.

When deleting a metadata item, it appears that the keys still exist, although the mapped value is "null". For example, if I call:

val toDelete: Map[String, String] = Map("foo" -> null)
val updated = blob.toBuilder.setMetadata(toDelete.asJava).build().update()

If I then call updated.getMetadata, I see: Map(foo -> ). If I do the same test against GCS, I can see it deletes the 'foo' metadata item.

@noamshvacher
Copy link

Hi, any updates on that? I'm experiencing the same issue here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants