Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3 retrieve can fail with non-obvious error message if MetadataFileExtension is not provided #108

Open
discentem opened this issue May 25, 2023 · 0 comments
Labels
AWS S3 FR/Bugs related to AWS S3 store implementation bug Something isn't working

Comments

@discentem
Copy link
Owner

func TestS3StoreRetrieve(t *testing.T) {
	mTime, _ := time.Parse("2006-01-02T15:04:05.000Z", "2014-11-12T11:45:26.371Z")
	// create bucket content
	bucketfs, err := testutils.MemMapFsWith(map[string]testutils.MapFile{
		"someObject": {
			Content: []byte("tla"),
			ModTime: &mTime,
		},
	})
	assert.NoError(t, err)

	fakeS3Server := aferoS3Server{
		buckets: map[string]afero.Fs{
			// create a bucket in our fake s3 server with the content
			"aFakeBucket": *bucketfs,
		},
	}

	localFs, err := testutils.MemMapFsWith(map[string]testutils.MapFile{
		"someObject.cfile": {
			Content: []byte(`{
				"name": "someObject",
				"checksum": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
				"date_modified": "2014-11-12T11:45:26.371Z"
			   }`),
			ModTime: &mTime,
		},
	})

	store := S3Store{
		Options: Options{
			BackendAddress:        "s3://aFakeBucket",
			MetadataFileExtension: "cfile",
		},
		fsys:         *localFs,
		awsRegion:    "us-east-1",
		s3Uploader:   fakeS3Server,
		s3Downloader: fakeS3Server,
	}

	err = store.Retrieve(context.Background(), "someObject") # NO .cfile EXTENSION HERE
	assert.NoError(t, err)

}

This results in the following error:

json marshal failed: invalid character 'l' in literal true (expecting 'r')

We should catch the user if they don't provide whatever MetadataFileExtension is in their input

cavorite retrieve thing.cfile <--- good input
cavorite retrieve thing <--- bad input, should fail early

@discentem discentem added bug Something isn't working AWS S3 FR/Bugs related to AWS S3 store implementation labels May 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AWS S3 FR/Bugs related to AWS S3 store implementation bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant