Skip to content

Commit

Permalink
Skip folder from s3 listing
Browse files Browse the repository at this point in the history
When list and s3 for state folders may be created as separate object with 0 size, such objects should be skipped

closes #130

---

Pull Request resolved: #131
commit_hash:1bb587c1259b843d1797e06aaaaff4d68db5d1f6
  • Loading branch information
laskoviymishka authored and robot-piglet committed Dec 5, 2024
1 parent d4f3a5c commit 8032ee3
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions pkg/coordinator/s3coordinator/coordinator_s3.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,10 @@ func (c *CoordinatorS3) GetTransferState(transferID string) (map[string]*coordin

// Fetch each object and deserialize the JSON
for _, obj := range listResp.Contents {
if obj.Size == nil || *obj.Size == 0 {
// see: https://stackoverflow.com/questions/75620230/aws-s3-listobjectsv2-returns-folder-as-an-object
continue
}
key := strings.TrimPrefix(*obj.Key, prefix)
getInput := &s3.GetObjectInput{
Bucket: aws.String(c.bucket),
Expand Down

0 comments on commit 8032ee3

Please sign in to comment.