Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Information for doc: MUST respect server's request of upload type #124

Open
binn opened this issue Dec 20, 2021 · 2 comments
Open

Information for doc: MUST respect server's request of upload type #124

binn opened this issue Dec 20, 2021 · 2 comments
Labels
Priority: Medium Normal priority Type: Enhancement This will **improve** existing or **add** something new

Comments

@binn
Copy link

binn commented Dec 20, 2021

Simple

If VRChat returns an upload type of simple, you MUST respect it and upload all the data in one big HTTP request to S3.

In this scenario, you need to add the content-md5 and content-type headers to the request.

Multipart

Otherwise, if you get multipart, you HAVE to upload only 10485760 bytes at a time, and update your part number when doing startFileUpload.

The content-md5 and content-type headers MUST be omitted otherwise your request will be rejected.

It is expected that you calculate the number of required parts yourself.

An implementation of this goes as follows:

FileStream fs = File.Open("file.txt");

int chunkSize = 10485760;
byte[] buffer = new byte[chunkSize * 2];
int parts = (int)MathF.Max(1f, MathF.Floor((float)fs.Length / (float)chunkSize));
for(int part = 0; part <= parts; part++)
{
    int bytesToRead = part < parts ? chunkSize : (int)(fs.Length - fs.Position);
    int bytesRead = fs.Read(buffer, 0, bytesToRead);

   // example method
   var file = StartMultipartUpload(type, part);
   UploadToS3(buffer, bytesRead, file.URL);

   // VRChat throws an exception if bytesRead != bytesToRead
}

Queued

Not documented yet, not sure what queued does or means.

If this isn't explicitly followed, the file is rejected on the server end later on, with the S3 URL returning 403 Forbidden.

@binn
Copy link
Author

binn commented Dec 20, 2021

When completing a multipart File Upload, you must save the etags returned from the S3 API and provide it to the finishFileUpload request.

@Foorack Foorack added Priority: Medium Normal priority Type: Enhancement This will **improve** existing or **add** something new labels Jan 20, 2022
@Foorack
Copy link
Member

Foorack commented Jan 20, 2022

Thank you. I had actually missed this one.

Better documentation of the File API is drastically needed. It is one of the most complex areas, and also one that's requested the most. Will look into this more in the coming weeks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: Medium Normal priority Type: Enhancement This will **improve** existing or **add** something new
Development

No branches or pull requests

2 participants