Uploads

API

{
    "id": "<upload_id>",
    "region": "eu-west-1",
    "bucket": "useruploadbucket",
    "destinations": [
        {
            "type": "youtube",
            "status": "waiting|upload|transcoding",
            "progress": 25.56,
            "details": "as returned from YouTube API"
            "error": "..." # optional
        },
        {
            "type": "s3Bucket",
            "status": "waiting|transcoding",
            "progress": 56.54,
            "details": "free-form, do we need this?"
            "error": "..." # optional
        }
    ],
    "parts": [
        {
            "start": 0,
            "end": 265716,
            "key": "uploads/<atom_id>/<upload_id>/parts/0",
            "complete": true
        },
        {
            "start": 265716,
            "end": 24617583,
            "key": "uploads/<atom_id>/<upload_id>/parts/1",
            "complete": false
        }
    ],
    #credentials are only returned on initial POST or the refresh end-point
    # i.e. they are not stored in the progress table 
    "credentials": {
        "temporaryAccessId": "...",
        "temporarySecretKey": "...",
        "sessionToken": "...",
        "expiry": 1275617265
    }
}

YouTube progress is a combination of reading the progress dynamo table and calling the YouTube API for transcoding progress (if the upload already complete).

List uploads for atom

GET /api2/atom/<atom_id>/uploads

{
    "uploads": [
        ... see format above ...
    ]
}

### Start an upload

POST /api2/atom/<atom_id>/uploads

.. see format above ..

Refresh credentials for an upload

POST /api2/atom/<atom_id>/uploads/<upload_id>/credentials

{
    "credentials": {
        "temporaryAccessId": "...",
        "temporarySecretKey": "...",
        "sessionToken": "...",
        "expiry": 1275617265
    }
}

Lambdas

New object in S3

Key: /uploads/atom_id/upload_id/parts/part_number

if part_number == number_of_parts:
    issue multipart upload copy to reconstruct entire video into one key
        /uploads/atom_id/upload_id/full

else if destination == "youtube" and
        no youtube uploads running and
        part_number == next part in progress table:

    launch youtube upload against part

Key: /uploads/atom_id/upload_id/ful

if s3bucket destination:
    - launch transcoding

YouTube upload

Key: /uploads/atom_id/upload_id/parts/part_number

uploadToYouTube();
markAsCompleteInProgressTable();

if part_number == number_of_parts:
    add YouTube asset to atom
else if key for next part number exists:
    launch youtube upload against next part

Progress Table

• Use consistent reads at start and end of lambdas

  • Required because YouTube doesn't like concurrent uploads

• General reads (like getting progress) use normal eventual consistency

Reference

https://developers.google.com/youtube/v3/docs/videos#processingDetails