Implement changed flow storage to return list of media object URLs

The response of /<flow id>/storage has "segments" renamed to "media_objects" to reflect there is no longer a need for the concept of a storage segment that is distinct from the media object. The request and response have no timeranges. The response no longer has paging headers. Some wording has been changed to use media object instead of "flow segment media object" or "flow segment object" or "storage segment". The start timestamp falling within the storage segment timerange restriction has been removed sem-ver: api-break
bbc · Mar 21, 2024 · 57df67b · 57df67b
1 parent 3e777c7
commit 57df67b
Show file tree

Hide file tree

Showing 6 changed files with 44 additions and 74 deletions.
diff --git a/README.md b/README.md
@@ -76,11 +76,10 @@ The process of reading from the store is:
 The process of writing to the store is:
 
 1. Client creates a Flow if necessary by making a request to [`PUT flows/<flow_id>`](https://bbc.github.io/tams/#/operations/PUT_flows-flowId)
-2. Client makes a request to [`POST flows/<flow_id>/storage`](https://bbc.github.io/tams/#/operations/POST_flows-flowId-storage) with the timerange to be written
-3. Store responds with a list of Flow Segment timeranges and URLs to PUT media data into, along with an optional `pre` URL to call before writing
-4. If a `pre` URL was given, client calls it
-5. Client breaks content into Flow Segments as instructed and uploads it
-6. Client makes requests to [`POST flows/<flow_id>/segments`](https://bbc.github.io/tams/#/operations/POST_flows-flowId-segments) with details of each new Flow Segment created, to register them on the timeline
+2. Client makes a request to [`POST flows/<flow_id>/storage`](https://bbc.github.io/tams/#/operations/POST_flows-flowId-storage) and receives a list of URLs to PUT media data into, along with an optional `pre` URL to call before writing
+3. If a `pre` URL was given, client calls it
+4. Client breaks content into Flow Segments and uploads the corresponding media data
+5. Client makes requests to [`POST flows/<flow_id>/segments`](https://bbc.github.io/tams/#/operations/POST_flows-flowId-segments) with details of each new Flow Segment created, to register them on the timeline
 
 ### Sources
 

diff --git a/api/TimeAddressableMediaStore.yaml b/api/TimeAddressableMediaStore.yaml
@@ -929,8 +929,6 @@ paths:
 
         For newly-written objects, the client is responsible for ensuring that the segment written to the store obeys the following restrictions:
         - The object id provided for a segment MUST be one which was received in a POST from /storage for this flow.
-        - The timestamp of the first sample written to the object MUST be located inside the timerange
-          that was associated with this object id in the response from the POST request to /storage for this flow.
         - All samples in the object SHOULD be used by the segment.
         - The timestamps of each sample in the media object MUST equal its position on the Flow timeline, OR `ts_offset` MUST
           be set such that `media_object_ts + ts_offset = segment_ts`
@@ -1008,25 +1006,22 @@ paths:
         description: The flow identifier.
     post:
       description: |
-        Allocate storage locations for writing flow segment objects that would fall within the specified timerange.
-        The returned storage locations will be arranged according to the rate stored in the flow.
+        Allocate storage locations for writing media objects.
 
         The media store type, which is indicated in the /service resource, determines the information provided
         in the response. The examples and description below are for the "http_object_store" media store type.
         This media store type provides HTTP URLs for uploading and downloading media objects in buckets.
 
-        The response will include a PUT URL for each storage location that a client uses to upload the flow segment
-        object. The client is expected to register the flow segment using the /flows/{flowId}/segments endpoint
-        once the upload is complete. Implementations need to handle situations where objects were uploaded but no
-        flow segment was registered successfully. The only requirement on the client writing data into these segments
-        is that the first timestamp of the data in that object MUST lie within the timerange that was indicated for
-        that storage location.
+        The response will include a PUT URL that a client uses to upload the media object. The client is expected
+        to register the flow segment using the /flows/{flowId}/segments endpoint once the upload is complete.
+        Implementations need to handle situations where objects were uploaded but no flow segment was registered
+        successfully.
 
-        The response may include PUT URLs for creating buckets for the flow segment objects. These PUT URLs should
-        be used before uploading flow segment objects. The object_id associated with each storage location has the
+        The response may include PUT URLs for creating buckets for the media objects. These PUT URLs should
+        be used before uploading media objects. The object_id associated with each storage location has the
         bucket name as its prefix.
 
-        The response may include PUT URLs for setting the CORS properties for the buckets and flow segment objects.
+        The response may include PUT URLs for setting the CORS properties for the buckets and media objects.
       operationId: POST_flows-flowId-storage
       tags:
         - MediaStorage
@@ -1039,24 +1034,7 @@ paths:
               $ref: examples/flow-storage-post.json
       responses:
         "201":
-          description: "Paging headers with information about the current and next page"
-          headers:
-            X-Paging-Next-Timerange:
-              description: The 'timerange' to use in the Post data to get the next page. If 'X-Paging-Next-Timerange' is not present then it is the last page.
-              schema:
-                $ref: 'schemas/timerange.json'
-            X-Paging-Limit:
-              description: Identifies the current limit being used for paging. This may not match the requested value if the requested value was too high for the implementation.
-              schema:
-                type: integer
-            X-Paging-Timerange:
-              description: Identifies the timerange for the returned data set.
-              schema:
-                $ref: 'schemas/timerange.json'
-            X-Paging-Count:
-              description: The number of items in the returned data set.
-              schema:
-                type: integer
+          description: Storage locations for writing media objects.
           content:
             application/json:
               schema:

diff --git a/api/examples/flow-storage-post-201.json b/api/examples/flow-storage-post-201.json
@@ -17,76 +17,76 @@
             }
         }
     ],
-    "segments": {
-        "[0:0_0:20000000)": {
+    "media_objects": [
+        {
             "object_id": "tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/846023d3-612d-5014-bc47-88f6eb2d04bb",
             "put_url": {
                 "url": "https://example.store.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/846023d3-612d-5014-bc47-88f6eb2d04bb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[0:20000000_0:40000000)": {
+        {
             "object_id": "tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/25be83fc-11d1-5743-9d47-6865cef5ea35",
             "put_url": {
                 "url": "https://example.store.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/25be83fc-11d1-5743-9d47-6865cef5ea35?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[0:40000000_0:60000000)": {
+        {
             "object_id": "tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/8b785422-6a82-5d60-b25a-f77e0a748321",
             "put_url": {
                 "url": "https://example.store.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/8b785422-6a82-5d60-b25a-f77e0a748321?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[0:60000000_0:80000000)": {
+        {
             "object_id": "tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/a94cee18-1a40-5676-b788-1ac74c8a26e9",
             "put_url": {
                 "url": "https://example.store.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/a94cee18-1a40-5676-b788-1ac74c8a26e9?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[0:80000000_0:100000000)": {
+        {
             "object_id": "tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9d2b1d66-d785-5dcd-82d8-70d37d39259e",
             "put_url": {
                 "url": "https://example.store.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9d2b1d66-d785-5dcd-82d8-70d37d39259e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[0:980000000_1:0)": {
+        {
             "object_id": "tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/e3a116a0-416c-5530-ae79-58f2ecea31cb",
             "put_url": {
                 "url": "https://example.store.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/e3a116a0-416c-5530-ae79-58f2ecea31cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[1:0_1:20000000)": {
+        {
             "object_id": "tams-c6b8e7cc-edd3-5f6d-9d79-4467d06eb8bf/8859eec0-cf59-54b2-8572-51ffb755b369",
             "put_url": {
                 "url": "https://example.store.com/tams-c6b8e7cc-edd3-5f6d-9d79-4467d06eb8bf/8859eec0-cf59-54b2-8572-51ffb755b369?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[1:20000000_1:40000000)": {
+        {
             "object_id": "tams-c6b8e7cc-edd3-5f6d-9d79-4467d06eb8bf/5b143aae-fa75-50b4-9cba-b7c0676dba15",
             "put_url": {
                 "url": "https://example.store.com/tams-c6b8e7cc-edd3-5f6d-9d79-4467d06eb8bf/5b143aae-fa75-50b4-9cba-b7c0676dba15?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[1:40000000_1:60000000)": {
+        {
             "object_id": "tams-c6b8e7cc-edd3-5f6d-9d79-4467d06eb8bf/028ce16f-96ab-5334-bf3c-db95fdc73b17",
             "put_url": {
                 "url": "https://example.store.com/tams-c6b8e7cc-edd3-5f6d-9d79-4467d06eb8bf/028ce16f-96ab-5334-bf3c-db95fdc73b17?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         },
-        "[1:60000000_1:80000000)": {
+        {
             "object_id": "tams-c6b8e7cc-edd3-5f6d-9d79-4467d06eb8bf/8264b3d3-d903-58b1-ab28-0ea164a22d2b",
             "put_url": {
                 "url": "https://example.store.com/tams-c6b8e7cc-edd3-5f6d-9d79-4467d06eb8bf/8264b3d3-d903-58b1-ab28-0ea164a22d2b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0&X-Amz-Date=20230316T120329Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=0",
                 "content-type": "video/mp2t"
             }
         }
-    }
+    ]
 }
diff --git a/api/examples/flow-storage-post.json b/api/examples/flow-storage-post.json
@@ -1,4 +1,3 @@
 {
-  "timerange": "[0:0_100:0)",
   "limit": 50
 }
diff --git a/api/schemas/flow-storage-post.json b/api/schemas/flow-storage-post.json
@@ -3,11 +3,6 @@
     "description": "Post data for the flow storage endpoint",
     "type": "object",
     "properties": {
-        "timerange": {
-            "description": "Limit the target timerange for the storage segments",
-            "type": "string",
-            "pattern": "^(\\[|\\()?-?\\d+:\\d+?(_(-?\\d+:\\d+)?)?(\\]|\\))?$"
-        },
         "limit": {
             "description": "Limit the number of storage segments in each response page. Implementations may specify their own default and maximum for the limit",
             "type": "integer"

diff --git a/api/schemas/flow-storage.json b/api/schemas/flow-storage.json
@@ -1,11 +1,11 @@
 {
   "type": "object",
-  "title": "Media Bucket Segment Store",
-  "description": "Gives information on storage for particular flow segments. This example is for a media store that provided URLs for storing media objects in bucket. Different backends may add additional/different fields",
+  "title": "Media Bucket Object Store",
+  "description": "Gives information on storage for media objects. This example is for a media store that provided URLs for storing media objects in bucket. Different backends may add additional/different fields",
   "properties": {
     "pre": {
       "type": "array",
-      "description": "Actions that need to be taken before the segments can be written to",
+      "description": "Actions that need to be taken before the media object can be written",
       "items": {
         "type": "object",
         "description": "An action",
@@ -28,23 +28,22 @@
         }
       }
     },
-    "segments": {
-      "type": "object",
-      "description": "Mapping of timeranges (validated by regex) to segment storage locations. Format as described by the [TimeRange](../schemas/timerange#top) type",
-      "patternProperties": {
-        "^[\\[\\(]?([0-9]+:[0-9]+)?_([0-9]+:[0-9]+)?[\\]\\)]?$": {
-          "type": "object",
-          "required": [
-            "object_id"
-          ],
-          "properties": {
-            "object_id": {
-              "description": "The object store identifier for the media object covering this timerange.",
-              "type": "string"
-            },
-            "put_url": { "$ref": "http-request.json" },
-            "put_cors_url": { "$ref": "http-request.json" }
-          }
+    "media_objects": {
+      "type": "array",
+      "description": "List of information for identifying and uploading media objects",
+      "items": {
+        "type": "object",
+        "description": "Information for a media object",
+        "required": [
+          "object_id"
+        ],
+        "properties": {
+          "object_id": {
+            "description": "The object store identifier for the media object.",
+            "type": "string"
+          },
+          "put_url": { "$ref": "http-request.json" },
+          "put_cors_url": { "$ref": "http-request.json" }
         }
       }
     }