diff --git a/index.src.html b/index.src.html index a0c58418..33f7052e 100644 --- a/index.src.html +++ b/index.src.html @@ -13,7 +13,7 @@ Editor: Bernard Aboba, w3cid 65611, Microsoft Corporation https://www.microsoft.com/ Abstract: This specification defines interfaces to codecs for encoding and - decoding of audio and video. + decoding of audio, video, and images. This specification does not specify or require any particular codec or method of encoding or decoding. The purpose of this specification is to @@ -45,6 +45,7 @@ type: attribute; text: resizeWidth; url:#dom-imagebitmapoptions-resizewidth type: attribute; text: resizeHeight; url:#dom-imagebitmapoptions-resizeheight type: dfn; text: global object; url: webappapis.html#global-object + type: dfn; text: live; url: infrastructure.html#live spec: mediacapture-streams; urlPrefix: https://www.w3.org/TR/mediacapture-streams/ for: mediaDevices; @@ -60,6 +61,7 @@ spec: mimesniff; urlPrefix: https://mimesniff.spec.whatwg.org/# type: dfn; text: MIME type; url: mime-type + type: dfn; text: valid MIME type string; url:valid-mime-type spec: infra; urlPrefix: https://infra.spec.whatwg.org/# type: dfn; text: queue; url: queues @@ -76,6 +78,12 @@ type: attribute; text: powerEfficient; url: dom-mediacapabilitiesinfo-powerefficient + + Definitions {#definitions} ========================== @@ -101,8 +109,22 @@ objects. Such resources may be quickly exhuasted and should be released immediately when no longer in use. -Codec Processing Model {#codec-processing-model} -================================================ +: Progressive Image +:: An image that supports decoding to multiple levels of detail, with lower + levels becoming available while the encoded data is not yet fully buffered. + +: Progressive Image Frame Generation +:: A generational identifier for a given [=Progressive Image=] decoded output. + Each successive generation adds additional detail to the decoded output. + The mechanism for computing a frame's generation is implementer defined. + +: Primary Image Track +:: An image track that is marked by the given image file as being the default + track. The mechanism for indiciating a primary track is format defined. + + +Codec Processing Model {#codec-processing-model-section} +=================================================================== Background {#processing-model-background} ----------------------------------------- @@ -277,7 +299,8 @@ 2. Increment {{VideoDecoder/decodeQueueSize}}. 3. [=Queue a control message=] to decode the |chunk|. - Running a control message to decode the chunk means performing these steps: + [=Running a control message=] to decode the chunk means performing these + steps: 1. Attempt to use {{VideoDecoder/[[codec implementation]]}} to decode the chunk. 2. If decoding results in an error, queue a task on the [=control thread=] @@ -304,7 +327,7 @@ 3. [=Queue a control message=] to flush the codec with |promise|. 4. Return |promise|. - Running a control message to flush the codec means performing these steps + [=Running a control message=] to flush the codec means performing these steps with |promise|. 1. Signal {{AudioDecoder/[[codec implementation]]}} to emit all [=internal pending outputs=]. @@ -504,7 +527,7 @@ 2. Increment {{VideoDecoder/decodeQueueSize}}. 3. [=Queue a control message=] to decode the |chunk|. - Running a control message to decode the chunk means performing these steps: + [=Running a control message=] to decode the chunk means performing these steps: 1. Attempt to use {{VideoDecoder/[[codec implementation]]}} to decode the chunk. 2. If decoding results in an error, queue a task on the [=control thread=] @@ -531,7 +554,7 @@ 3. [=Queue a control message=] to flush the codec with |promise|. 4. Return |promise|. - Running a control message to flush the codec means performing these steps + [=Running a control message=] to flush the codec means performing these steps with |promise|. 1. Signal {{VideoDecoder/[[codec implementation]]}} to emit all [=internal pending outputs=]. @@ -596,28 +619,36 @@
Run these steps: 1. For each |output| in |outputs|: - 1. Let |planes| be a sequence of {{Plane}}s containing the decoded video + 1. Let |timestamp| and |duration| be the + {{EncodedVideoChunk/timestamp}} and {{EncodedVideoChunk/duration}} + from the {{EncodedVideoChunk}} associated with |output|. + 2. Let |frame| be the result of running the [=Create a VideoFrame=] + algorithm with |output|, |timestamp|, and |duration|. + 3. Invoke {{VideoDecoder/[[output callback]]}} with |frame|. +
+
+ Create a VideoFrame (with |output|, |timestamp|, and |duration|) +
+
+ 1. Let |planes| be a sequence of {{Plane}}s containing the decoded video frame data from |output|. - 2. Let |pixelFormat| be the {{PixelFormat}} of |planes|. - 3. Let |frameInit| be a {{VideoFrameInit}} with the following keys: - 1. Let {{VideoFrameInit/timestamp}} and {{VideoFrameInit/duration}} - be the {{EncodedVideoChunk/timestamp}} and - {{EncodedVideoChunk/duration}} from the {{EncodedVideoChunk}} - associated with |output|. - 2. Let {{VideoFrameInit/codedWidth}} and - {{VideoFrameInit/codedHeight}} - be the width and height of the decoded video frame |output| in - pixels, prior to any cropping or aspect ratio adjustments. - 3. Let {{VideoFrameInit/cropLeft}}, {{VideoFrameInit/cropTop}}, - {{VideoFrameInit/cropWidth}}, and {{VideoFrameInit/cropHeight}} - be the crop region of the decoded video frame |output| in - pixels, prior to any aspect ratio adjustments. - 4. Let {{VideoFrameInit/displayWidth}} and - {{VideoFrameInit/displayHeight}} be the display size of the - decoded video frame in pixels. - 4. Let |frame| be a {{VideoFrame}}, constructed with |pixelFormat|, - |planes|, and |frameInit|. - 5. Invoke {{VideoDecoder/[[output callback]]}} with |frame|. + 2. Let |pixelFormat| be the {{PixelFormat}} of |planes|. + 3. Let |frameInit| be a {{VideoFrameInit}} with the following keys: + 1. Assign |timestamp| to {{VideoFrameInit/timestamp}}. + 2. Assign |duration| to {{VideoFrameInit/duration}}. + 3. Let {{VideoFrameInit/codedWidth}} and + {{VideoFrameInit/codedHeight}} + be the width and height of the decoded video frame |output| in + pixels, prior to any cropping or aspect ratio adjustments. + 4. Let {{VideoFrameInit/cropLeft}}, {{VideoFrameInit/cropTop}}, + {{VideoFrameInit/cropWidth}}, and {{VideoFrameInit/cropHeight}} + be the crop region of the decoded video frame |output| in + pixels, prior to any aspect ratio adjustments. + 5. Let {{VideoFrameInit/displayWidth}} and + {{VideoFrameInit/displayHeight}} be the display size of the + decoded video frame in pixels. + 4. Return a new {{VideoFrame}}, constructed with |pixelFormat|, + |planes|, and |frameInit|.
Reset VideoDecoder
@@ -728,7 +759,7 @@ 3. Set {{AudioEncoder/state}} to `"configured"`. 4. [=Queue a control message=] to configure the encoder using |config|. - Running a control message to configure the encoder means performing these + [=Running a control message=] to configure the encoder means performing these steps: 1. Let |supported| be the result of running the Check Configuration Support algorith with |config|. @@ -753,7 +784,7 @@ 4. Increment {{AudioEncoder/encodeQueueSize}}. 5. [=Queue a control message=] to encode |frameClone|. - Running a control message to encode the frame means performing these steps. + [=Running a control message=] to encode the frame means performing these steps. 1. Attempt to use {{AudioEncoder/[[codec implementation]]}} to encode |frameClone|. 2. If encoding results in an error, queue a task on the [=control thread=] @@ -780,7 +811,7 @@ 3. [=Queue a control message=] to flush the codec with |promise|. 4. Return |promise|. - Running a control message to flush the codec means performing these steps + [=Running a control message=] to flush the codec means performing these steps with |promise|. 1. Signal {{AudioEncoder/[[codec implementation]]}} to emit all [=internal pending outputs=]. @@ -973,7 +1004,7 @@ 3. Set {{VideoEncoder/state}} to `"configured"`. 4. [=Queue a control message=] to configure the encoder using |config|. - Running a control message to configure the encoder means performing these + [=Running a control message=] to configure the encoder means performing these steps: 1. Let |supported| be the result of running the Check Configuration Support algorith with |config|. @@ -999,7 +1030,7 @@ 4. Increment {{VideoEncoder/encodeQueueSize}}. 5. [=Queue a control message=] to encode |frameClone|. - Running a control message to encode the frame means performing these steps. + [=Running a control message=] to encode the frame means performing these steps. 1. Attempt to use {{VideoEncoder/[[codec implementation]]}} to encode |frameClone| according to |options|. 2. If encoding results in an error, queue a task on the [=control thread=] @@ -1026,7 +1057,7 @@ 3. [=Queue a control message=] to flush the codec with |promise|. 4. Return |promise|. - Running a control message to flush the codec means performing these steps + [=Running a control message=] to flush the codec means performing these steps with |promise|. 1. Signal {{VideoEncoder/[[codec implementation]]}} to emit all [=internal pending outputs=]. @@ -2343,14 +2374,816 @@ NOTE: User Agents are encouraged to avoid expensive copies of large objects (for instance, {{VideoFrame}} pixel data). Frame types are - immutable, so the above step may be implemented using memory sharing - techniques such as reference counting. + immutable, so the above step may be implemented using memory + sharing techniques such as reference counting. 3. Return |cloneFrame|.
+Image Decoding {#image-decoding} +==================================== + +Background {#image-decoding-background} +------------------------------------- + +This section is non-normative. + +Image codec definitions are typically accompanied by a definition for a +corresponding file format. Hence image decoders often perform both duties of +unpacking (demuxing) as well as decoding the encoded image data. The WebCodecs +{{ImageDecoder}} follows this pattern, which motivates an interface design that +is notably different from that of {{VideoDecoder}} and {{AudioDecoder}}. + +In spite of these differences, {{ImageDecoder}} uses the same +[=codec processing model=] as the other codec interfaces. Additionally, +{{ImageDecoder}} uses the {{VideoFrame}} interface to describe decoded outputs. + +ImageDecoder Interface {#imagedecoder-interface} +------------------------------------------------ + +
+
+[Exposed=(Window,DedicatedWorker)]
+interface ImageDecoder {
+  constructor(ImageDecoderInit init);
+
+  readonly attribute boolean complete;
+  readonly attribute Promise<undefined> completed;
+  readonly attribute ImageTrackList tracks;
+
+  Promise<ImageDecodeResult> decode(optional ImageDecodeOptions options);
+  undefined reset();
+  undefined close();
+
+  static Promise<boolean> isTypeSupported(DOMString type);
+};
+
+
+ +### Internal Slots ### {#imagedecoder-internal-slots} + +: \[[ImageTrackList]] +:: An {{ImageTrackList}} describing the tracks found in + {{ImageDecoder/[[encoded data]]}} + +: \[[complete]] +:: A boolean indicating whether {{ImageDecoder/[[encoded data]]}} is completely + buffered. + +: [[completed promise]] +:: The promise used to signal when {{ImageDecoder/[[complete]]}} becomes + `true`. + +: [[codec implementation]] +:: An underlying image decoder implementation provided by the User Agent. + +: [[encoded data]] +:: A [=byte sequence=] containing the encoded image data to be decoded. + +: [[prefer animation]] +:: A boolean reflecting the value of {{ImageDecoderInit/preferAnimation}} given + at construction. + +: [[pending decode promises]] +:: A list of unresolved promises returned by calls to decode(). + +: [[internal selected track index]] +:: Identifies the image track within {{ImageDecoder/[[encoded data]]}} that is + used by decoding algorithms on the [=codec thread=]. + +: [[tracks established]] +:: A boolean indicating whether the track list has been established in + {{ImageDecoder/[[ImageTrackList]]}}. + +: \[[closed]] +:: A boolean indicating that the ImageDecoder is in a permanent closed state + and can no longer be used. + +: [[progressive frame generations]] +:: A mapping of frame indices to [=Progressive Image Frame Generations=]. The + values represent the Progressive Image Frame Generation for the + {{VideoFrame}} which was most recently output by a call to + {{ImageDecoder/decode()}} with the given frame index. + + +### Constructor ### {#imagedecoder-constructor} + +: + ImageDecoder(init) + +:: NOTE: Calling {{ImageDecoder/decode()}} on the constructed {{ImageDecoder}} + will trigger a {{NotSupportedError}} if the user agent does not support + |type|. Authors should first check support by calling + {{ImageDecoder/isTypeSupported()}} with |type|. User agents are not + required to support any particular type. + + When invoked, run these steps: + 1. If |init| is not [=valid ImageDecoderInit=], throw a {{TypeError}}. + 2. Let |d| be a new {{ImageDecoder}} object. In the steps below, all + mentions of {{ImageDecoder}} members apply to |d| unless stated + otherwise. + 3. Assign {{ImageDecoder/[[ImageTrackList]]}} a new {{ImageTrackList}} + initialized as follows: + 1. Assign a new [=list=] to {{ImageTrackList/[[track list]]}}. + 2. Assign `-1` to {{ImageTrackList/[[selected index]]}}. + 4. Assign `null` to {{ImageDecoder/[[codec implementation]]}}. + 5. If `init.preferAnimation` [=map/exists=], assign `init.preferAnimation` + to the {{ImageDecoder/[[prefer animation]]}} internal slot. Otherwise, + assign 'null' to {{ImageDecoder/[[prefer animation]]}} internal slot. + 7. Assign a new [=list=] to {{ImageDecoder/[[pending decode promises]]}}. + 8. Assign `-1` to {{ImageDecoder/[[internal selected track index]]}}. + 9. Assign `false` to {{ImageDecoder/[[tracks established]]}}. + 10. Assign `false` to {{ImageDecoder/[[closed]]}}. + 11. Assign a new [=map=] to {{ImageDecoder/[[progressive frame + generations]]}}. + 12. If |init|'s {{ImageDecoderInit/data}} member is of type + {{ReadableStream}}: + 1. Assign a new [=list=] to {{ImageDecoder/[[encoded data]]}}. + 2. Assign `false` to {{ImageDecoder/complete}} + 3. [=Queue a control message=] to [=configure the image decoder=] with + |init|. + 4. Let |reader| be the result of [=getting a reader=] for + {{ImageDecoderInit/data}}. + 5. In parallel, perform the [=Fetch Stream Data Loop=] on |d| with + |reader|. + 13. Otherwise: + 1. Assert that `init.data` is of type {{BufferSource}}. + 2. Assign a copy of `init.data` to {{ImageDecoder/[[encoded data]]}}. + 3. Assign `true` to {{ImageDecoder/complete}}. + 4. Reslove {{ImageDecoder/[[completed promise]]}}. + 5. Queue a control message to [=configure the image decoder=] with + |init|. + 6. Queue a control message to [=decode track metadata=]. + 14. return |d|. + + [=Running a control message=] to configure the image decoder + means running these steps: + 1. Let |supported| be the result of running the [=ImageDecoder/Check Type + Support=] algorithm with `init.type`. + 2. If |supported| is `false`, queue a task on the [=control thread=] event + loop to run the [=ImageDecoder/Close ImageDecoder=] algorithm + with a {{NotSupportedError}} {{DOMException}} and abort + these steps. + 3. If |supported| is `true`, assign the + {{ImageDecoder/[[codec implementation]]}} internal slot with an + implementation supporting `init.type` + 4. Configure {{ImageDecoder/[[codec implementation]]}} in accordance with + the values given for {{ImageDecoderInit/premultiplyAlpha}}, + {{ImageDecoderInit/colorSpaceConversion}}, + {{ImageDecoderInit/desiredWidth}}, and + {{ImageDecoderInit/desiredHeight}}. + + [=Running a control message=] to decode track metadata means + running these steps: + 1. Run the [=ImageDecoder/Establish Tracks=] algorithm. + +### Attributes ### {#imagedecoder-attributes} +: complete +:: Indicates whether {{ImageDecoder/[[encoded data]]}} is completely buffered. + + The {{ImageDecoder/complete}} getter steps are to return + {{ImageDecoder/[[complete]]}}. + +: completed +:: The promise used to signal when {{ImageDecoder/complete}} becomes `true`. + + The {{ImageDecoder/completed}} getter steps are to return + {{ImageDecoder/[[completed promise]]}}. + +: tracks +:: Returns a [=live=] {{ImageTrackList}}, which provides metadata + for the available tracks and a mechanism for selecting a track to decode. + + The {{ImageDecoder/tracks}} getter steps are to return + {{ImageDecoder/[[ImageTrackList]]}}. + +### Methods ### {#imagedecoder-methods} +: decode(options) +:: Enqueues a control message to decode the frame according to |options|. + + When invoked, run these steps: + 1. If {{ImageDecoder/[[closed]]}} is `true`, return a {{Promise}} + rejected with an {{InvalidStateError}} {{DOMException}}. + 2. If {{ImageDecoder/[[ImageTrackList]]}}'s + {{ImageTrackList/[[selected index]]}} is '-1', return a {{Promise}} + rejected with an {{InvalidStateError}} {{DOMException}}. + 3. If |options| is `undefined`, assign a new {{ImageDecodeOptions}} to + |options|. + 4. Let |promise| be a new {{Promise}}. + 5. [=Queue a control message=] to decode the the image with |options|, and + |promise|. + 6. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}. + 7. Return |promise|. + + [=Running a control message=] to decode the image means running these + steps: + 1. Wait for {{ImageDecoder/[[tracks established]]}} to become `true`. + 2. If |options|.{{ImageDecodeOptions/completeFramesOnly}} is `false` and + the image is a [=Progressive Image=] for which the user agent supports + progressive decoding, run the [=Decode Progressive Frame=] algorithm with |options|.{{ImageDecodeOptions/frameIndex}} and |promise|. + 3. Otherwise, run the [=Decode Complete Frame=] algorithm with + |options|.{{ImageDecodeOptions/frameIndex}} and |promise|. + +: reset() +:: Immediately aborts all pending work. + + When invoked, run the [=ImageDecoder/Reset ImageDecoder=] algorithm with + and {{AbortError}} {{DOMException}}. + +: close() +:: Immediately aborts all pending work and releases system resources. Close is + final. + + When invoked, run the [=ImageDecoder/Close ImageDecoder=] algorithm with + and {{AbortError}} {{DOMException}}. + +: isTypeSupported(type) +:: Returns a promise indicating whether the provided config is supported by the + user agent. + + When invoked, run these steps: + 1. If |type| is not a [=valid image MIME type=], return a {{Promise}} + rejected with {{TypeError}}. + 2. Let |p| be a new {{Promise}}. + 3. In parallel, resolve |p| with the result of running the + [=Check Type Support=] algorithm with |type|. + 4. Return |p|. + +### Algorithms ### {#imagedecoder-algorithms} + +: Fetch Stream Data Loop (with |reader|) +:: Run these steps: + 1. Let |readRequest| be the following [=read request=]. + + : [=read request/chunk steps=], given |chunk| + :: 1. If {{ImageDecoder/[[closed]]}} is `true`, abort these steps. + 2. If |chunk| is not a Uint8Array object, queue a task on the + [=control thread=] event loop to run the + [=ImageDecoder/Close ImageDecoder=] algorithm with a + {{DataError}} {{DOMException}} and abort these steps. + 3. Let |bytes| be the byte sequence represented by the Uint8Array + object. + 4. Append |bytes| to the {{ImageDecoder/[[encoded data]]}} + internal slot. + 5. If {{ImageDecoder/[[tracks established]]}} is `false`, run the + [=Establish Tracks=] algorithm. + 6. Otherwise, run the [=Update Tracks=] algorithm. + 7. Run the [=Fetch Stream Data Loop=] algorithm with |reader|. + + : [=read request/close steps=] + :: 1. Assign `true` to {{ImageDecoder/complete}} + 2. Resolve {{ImageDecoder/[[completed promise]]}}. + + : [=read request/error steps=] + :: 1. Queue a task on the [=control thread=] event loop to run the + [=ImageDecoder/Close ImageDecoder=] algorithm with a + {{NotReadableError}} {{DOMException}} + + 2. Read a chunk from |reader| given |readRequest|. + +: Establish Tracks +:: Run these steps: + 1. Assert {{ImageDecoder/[[tracks established]]}} is `false`. + 2. If {{ImageDecoder/[[encoded data]]}} does not contain enough data to + determine the number of tracks: + 1. If {{ImageDecoder/complete}} is `true`, queue a task on the + [=control thread=] event loop to run the [=ImageDecoder/Close ImageDecoder=] algorithm. + 2. Abort these steps. + 3. If the number of tracks is found to be `0`, queue a task on the + [=control thread=] event loop to run the + [=ImageDecoder/Close ImageDecoder=] algorithm and abort these steps. + 4. Let |newTrackList| be a new [=list=]. + 5. For each |image track| found in {{ImageDecoder/[[encoded data]]}}: + 1. Let |newTrack| be a new {{ImageTrack}}, initialized as follows: + 1. Assign [=this=] to {{ImageTrack/[[ImageDecoder]]}}. + 2. Assign {{ImageDecoder/tracks}} to + {{ImageTrack/[[ImageTrackList]]}}. + 3. If |image track| is found to be animated, assign `true` to + |newTrack|'s {{ImageTrack/[[animated]]}} internal slot. + Otherwise, assign `false`. + 4. If |image track| is found to describe a frame count, assign + that count to |newTrack|'s {{ImageTrack/[[frame count]]}} + internal slot. Otherwise, assign `0`. + + NOTE: If [=this=] was constructed with + {{ImageDecoderInit/data}} as a {{ReadableStream}}, the + {{ImageTrack/frameCount}} may change as additional bytes are + appended to {{ImageDecoder/[[encoded data]]}}. See the + [=Update Tracks=] algorithm. + + 5. If |image track| is found to describe a repetition count, + assign that count to {{ImageTrack/[[repetition count]]}} + internal slot. Otherwise, assign `0`. + + NOTE: A value of `Infinity` indicates infinite repetitions. + + 6. Assign `false` to |newTrack|'s {{ImageTrack/[[selected]]}} + internal slot. + 2. Append |newTrack| to |newTrackList|. + 6. Let |selectedTrackIndex| be the result of running the + [=ImageDecoder/Get Default Selected Track Index=] algorithm with + |newTrackList|. + 7. Let |selectedTrack| be the track at position |selectedTrackIndex| within + |newTrackList|. + 8. Assign `true` to |selectedTrack|'s {{ImageTrack/[[selected]]}} internal + slot. + 8. Assign |selectedTrackIndex| to {{ImageDecoder/[[internal selected track + index]]}}. + 9. Assign `true` to {{ImageDecoder/[[tracks established]]}}. + 10. Queue a task on the [=control thread=] event loop to perform the + following steps: + 1. Assign |newTrackList| to the {{ImageDecoder/tracks}} + {{ImageTrackList/[[track list]]}} internal slot. + 2. Assign |selectedTrackIndex| to {{ImageDecoder/tracks}} + {{ImageTrackList/[[selected index]]}}. + 3. Resolve {{ImageTrackList/[[ready promise]]}}. + +: Get Default Selected Track Index (with + |trackList|) +:: Run these steps: + 1. If {{ImageDecoder/[[encoded data]]}} identifies a [=Primary Image + Track=]: + 1. Let |primaryTrack| be the {{ImageTrack}} from |trackList| that + describes the [=Primary Image Track=]. + 2. Let |primaryTrackIndex| be position of |primaryTrack| within + |trackList|. + 3. If {{ImageDecoder/[[prefer animation]]}} is `null`, return + |primaryTrackIndex|. + 4. If |primaryTrack|.{{ImageTrack/animated}} equals + {{ImageDecoder/[[prefer animation]]}}, return |primaryTrackIndex|. + 2. If any {{ImageTrack}}s in |trackList| have {{ImageTrack/animated}} equal + to {{ImageDecoder/[[prefer animation]]}}, return the position of the + earliest such track in |trackList|. + 3. Return `0`. + +: Update Tracks +:: A track update struct is a [=struct=] that consists of a + track index ({{unsigned long}}) + and a frame count + ({{unsigned long}}). + + Run these steps: + 1. Assert {{ImageDecoder/[[tracks established]]}} is `true`. + 2. Let |trackChanges| be a new [=list=]. + 3. Let |trackList| be a copy of {{ImageDecoder/tracks}}' + {{ImageTrackList/[[track list]]}}. + 4. For each |track| in |trackList|: + 1. Let |trackIndex| be the position of |track| in |trackList|. + 2. Let |latestFrameCount| be the frame count as indicated by + {{ImageDecoder/[[encoded data]]}} for the track corresponding to + |track|. + 3. Assert that |latestFrameCount| is greater than or equal to + `track.frameCount`. + 4. If |latestFrameCount| is greater than `track.frameCount`: + 1. Let |change| be a [=track update struct=] whose + [=track update struct/track index=] is |trackIndex| and + [=track update struct/frame count=] is |latestFrameCount|. + 2. Append |change| to |tracksChanges|. + 5. If |tracksChanges| is [=list/empty=], abort these steps. + 6. Queue a task on the [=control thread=] event loop to perform the + following steps: + 1. For each update in |trackChanges|: + 1. Let |updateTrack| be the {{ImageTrack}} at position + `update.trackIndex` within {{ImageDecoder/tracks}}' + {{ImageTrackList/[[track list]]}}. + 2. Assign `update.frameCount` to |updateTrack|'s + {{ImageTrack/[[frame count]]}}. + 3. Fire a simple event named {{ImageTrack/change}} at the + {{ImageDecoder/tracks}} object. + +: Decode Complete Frame (with |frameIndex| and + |promise|) +:: 1. Assert that {{ImageDecoder/[[tracks established]]}} is `true`. + 2. Assert that {{ImageDecoder/[[internal selected track index]]}} is not + `-1`. + 3. Let |encodedFrame| be the encoded frame identified by |frameIndex| and + {{ImageDecoder/[[internal selected track index]]}}. + 4. Wait for any of the following conditions to be true (whichever happens + first): + 1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to + completely decode |encodedFrame|. + 2. {{ImageDecoder/[[encoded data]]}} is found to be malformed. + 3. {{ImageDecoder/complete}} is `true`. + 4. {{ImageDecoder/[[closed]]}} is `true`. + 5. If {{ImageDecoder/[[encoded data]]}} is found to be malformed, run the + [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these + steps. + 6. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to + completely decode |encodedFrame|, run the + [=ImageDecoder/Reject Infeasible Decode=] algorithm with |promise| and + abort these steps. + 7. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode + |encodedFrame|. + 8. If decoding produces an error, run the + [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these + steps. + 9. If {{ImageDecoder/[[progressive frame generations]]}} contains an entry + keyed by |frameIndex|, remove the entry from the map. + 10. Let |output| be the decoded image data emitted by + {{ImageDecoder/[[codec implementation]]}} corresponding to + |encodedFrame|. + 11. Let |decodeResult| be a new {{ImageDecodeResult}} initialized as + follows: + 1. Assign 'true' to {{ImageDecodeResult/complete}}. + 2. Let |timestamp| and |duration| be the presentation timestamp and + duration for |output| as described by |encodedFrame|. If + |encodedFrame| does not describe a timestamp or + duration, assign `null` to the corresponding variable. + 3. Assign {{ImageDecodeResult/image}} with the result of running the + [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and + |duration|. + 12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and + |decodeResult|. + +: Decode Progressive Frame (with |frameIndex| and + |promise|) +:: 1. Assert that {{ImageDecoder/[[tracks established]]}} is `true`. + 2. Assert that {{ImageDecoder/[[internal selected track index]]}} is not + `-1`. + 3. Let |encodedFrame| be the encoded frame identified by |frameIndex| and + {{ImageDecoder/[[internal selected track index]]}}. + 4. Let |lastFrameGeneration| be `null`. + 5. If {{ImageDecoder/[[progressive frame generations]]}} contains a map + entry with the key |frameIndex|, assign the value of the map entry to + |lastFrameGeneration|. + 6. Wait for any of the following conditions to be true (whichever happens + first): + 1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to decode + |encodedFrame| to produce an output who's [=Progressive Image + Frame Generation=] exceeds |lastFrameGeneration|. + 2. {{ImageDecoder/[[encoded data]]}} is found to be malformed. + 3. {{ImageDecoder/complete}} is `true`. + 4. {{ImageDecoder/[[closed]]}} is `true`. + 7. If {{ImageDecoder/[[encoded data]]}} is found to be malformed, run the + [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these + steps. + 8. Otherwise, if {{ImageDecoder/[[encoded data]]}} does not contain enough + bytes to decode |encodedFrame| to produce an output who's + [=Progressive Image Frame Generation=] exceeds |lastFrameGeneration|, + run the [=ImageDecoder/Reject Infeasible Decode=] algorithm with + |promise| and abort these steps. + 9. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode + |encodedFrame|. + 10. If decoding produces an error, run the + [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these + steps. + 11. Let |output| be the decoded image data emitted by + {{ImageDecoder/[[codec implementation]]}} corresponding to + |encodedFrame|. + 12. Let |decodeResult| be a new {{ImageDecodeResult}}. + 13. If |output| is the final full-detail progressive output corresponding + to |encodedFrame|: + 1. Assign `true` to |decodeResult|'s {{ImageDecodeResult/complete}}. + 2. If {{ImageDecoder/[[progressive frame generations]]}} contains an + entry keyed by |frameIndex|, remove the entry from the map. + 14. Otherwise: + 1. Assign `false` to |decodeResult|'s {{ImageDecodeResult/complete}}. + 2. Let |frameGeneration| be the [=Progressive Image Frame Generation=] + for |output|. + 3. Add a new entry to {{ImageDecoder/[[progressive frame + generations]]}} with key |frameIndex| and value |frameGeneration|. + 15. Let |timestamp| and |duration| be the presentation timestamp and + duration for |output| as described by |encodedFrame|. If + |encodedFrame| does not describe a timestamp or + duration, assign `null` to the corresponding variable. + 16. Assign {{ImageDecodeResult/image}} with the result of running the + [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and + |duration|. + 17. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 18. Resolve |promise| with |decodeResult|. + +: Resolve Decode (with |promise| and |result|) +:: 1. Queue a task on the [=control thread=] event loop to run these steps: + 1. If {{ImageDecoder/[[closed]]}}, abort these steps. + 2. Assert that |promise| is an element of + {{ImageDecoder/[[pending decode promises]]}}. + 3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 4. Resolve |promise| with |result|. + +: Reject Infeasible Decode (with |promise|) +:: 1. Assert that {{ImageDecoder/complete}} is `true` or + {{ImageDecoder/[[closed]]}} is `true`. + 2. If {{ImageDecoder/complete}} is `true`, let |exception| be a + {{RangeError}}. Otherwise, let |exception| be an + {{InvalidStateError}} {{DOMException}}. + 3. Queue a task on the [=control thread=] event loop to run these steps: + 1. If {{ImageDecoder/[[closed]]}}, abort these steps. + 2. Assert that |promise| is an element of + {{ImageDecoder/[[pending decode promises]]}}. + 3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 4. Reject |promise| with |exception|. + +: Fatally Reject Bad Data +:: 1. Queue a task on the [=control thread=] event loop to run these steps: + 1. If {{ImageDecoder/[[closed]]}}, abort these steps. + 2. Run the [=ImageDecoder/Close ImageDecoder=] algorithm with an + {{EncodingError}} {{DOMException}}. + +: Check Type Support (with |type|) +:: 1. If the user agent can provide a codec to support decoding |type|, return + `true`. + 2. Otherwise, return `false`. + +: Reset ImageDecoder (with |exception|) +:: 1. Signal {{ImageDecoder/[[codec implementation]]}} to abort any active + decoding operation. + 2. For each |decodePromise| in + {{ImageDecoder/[[pending decode promises]]}}: + 1. Reject |decodePromise| with |exception|. + 2. Remove |decodePromise| from + {{ImageDecoder/[[pending decode promises]]}}. + +: Close ImageDecoder (with |exception|) +:: 1. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm with |exception|. + 1. Assign `true` to {{ImageDecoder/[[closed]]}}. + 2. Clear {{ImageDecoder/[[codec implementation]]}} and release associated + [=system resources=]. + 3. Remove all entries from {{ImageDecoder/[[ImageTrackList]]}}. + 4. Assign `-1` to {{ImageDecoder/[[ImageTrackList]]}}'s + {{ImageTrackList/[[selected index]]}}. + + +ImageDecoderInit Interface {#imagedecoderinit-interface} +-------------------------------------------------------- +
+
+typedef (BufferSource or ReadableStream) ImageBufferSource;
+dictionary ImageDecoderInit {
+  required DOMString type;
+  required ImageBufferSource data;
+  PremultiplyAlpha premultiplyAlpha = "default";
+  ColorSpaceConversion colorSpaceConversion = "default";
+  unsigned long desiredWidth;
+  unsigned long desiredHeight;
+  boolean preferAnimation;
+};
+
+
+ +To determine if an {{ImageDecoderInit}} is a valid ImageDecoderInit, +run these steps: +1. If |type| is not a [=valid image MIME type=], return `false`. +2. If |data| is of type {{ReadableStream}} and the ReadableStream is + [=ReadableStream/disturbed=] or [=ReadableStream/locked=], return `false`. +3. If |data| is of type {{BufferSource}}: + 1. If the result of running IsDetachedBuffer (described in + [[!ECMASCRIPT]]) on |data| is `false`, return `false`. + 2. If |data| is [=empty=], return `false`. +4. If {{ImageDecoderInit/desiredWidth}} [=map/exists=] and + {{ImageDecoderInit/desiredHeight}} does not exist, return `false`. +5. If {{ImageDecoderInit/desiredHeight}} [=map/exists=] and + {{ImageDecoderInit/desiredWidth}} does not exist, return `false`. +6. Return `true`. + +A valid image MIME type is a string that is a [=valid MIME type +string=] and for which the `type`, per Section 3.1.1.1 of [[RFC7231]], is +`image`. + +: type +:: String containing the MIME type of the image file to be decoded. + +: data +:: {{BufferSource}} or {{ReadableStream}} of bytes representing an encoded + image file as described by {{ImageDecoderInit/type}}. + +: premultiplyAlpha +:: Controls whether decoded outputs' color channels are to be premultiplied by + their alpha channel, as defined by {{ImageBitmapOptions/premultiplyAlpha}} + in {{ImageBitmapOptions}}. + +: colorSpaceConversion +:: Controls whether decoded outputs' color space is converted or ignored, as + defined by {{ImageBitmapOptions/colorSpaceConversion}} in + {{ImageBitmapOptions}}. + +: desiredWidth +:: Indicates a desired width for decoded outputs. Implementation is best + effort; decoding to a desired width may not be supported by all formats/ + decoders. + +: desiredHeight +:: Indicates a desired height for decoded outputs. Implementation is best + effort; decoding to a desired height may not be supported by all + formats/decoders. + +: preferAnimation +:: For images with multiple tracks, this indicates whether the + initial track selection should prefer an animated track. + + NOTE: See the [=ImageDecoder/Get Default Selected Track Index=] algorithm. + +ImageDecodeOptions Interface {#imagedecodeoptions-interface} +------------------------------------------------------------ +
+
+dictionary ImageDecodeOptions {
+  unsigned long frameIndex = 0;
+  boolean completeFramesOnly = true;
+};
+
+
+ +: frameIndex +:: The index of the frame to decode. + +: completeFramesOnly +:: For [=Progressive Images=], a value of `false` indicates that the decoder + may output an {{ImageDecodeResult/image}} with reduced detail. Each + subsequent call to {{ImageDecoder/decode()}} for the same + {{ImageDecodeOptions/frameIndex}} will resolve to produce an image with a + higher [=Progressive Image Frame Generation=] (more image detail) than the + previous call, until finally the full-detail image is produced. + + If {{ImageDecodeOptions/completeFramesOnly}} is assigned `true`, or if the + image is not a [=Progressive Image=], or if the user agent does not support + progressive decoding for the given image type, calls to + {{ImageDecoder/decode()}} will only resolve once the full detail image is + decoded. + +
+ NOTE: For [=Progressive Images=], setting + {{ImageDecodeOptions/completeFramesOnly}} to `false` may be used to + offer users a preview an image that is still being buffered from the + network (via the {{ImageDecoderInit/data}} {{ReadableStream}}). + + Upon decoding the full detail image, the {{ImageDecodeResult}}'s + {{ImageDecodeResult/complete}} will be set to true. +
+ + +ImageDecodeResult Interface {#imagedecoderesult-interface} +---------------------------------------------------------- +
+
+dictionary ImageDecodeResult {
+  required VideoFrame image;
+  required boolean complete;
+};
+
+
+ +: image +:: The decoded image. + +: complete +:: Indicates whether {{ImageDecodeResult/image}} contains the final full-detail + output. + + NOTE: {{ImageDecodeResult/complete}} is always `true` when + {{ImageDecoder/decode()}} is invoked with + {{ImageDecodeOptions/completeFramesOnly}} set to `true`. + +ImageTrackList Interface {#imagetracklist-interface} +---------------------------------------------------- +
+
+interface ImageTrackList {
+  getter ImageTrack (unsigned long index);
+
+  readonly attribute Promise<undefined> ready;
+  readonly attribute unsigned long length;
+  readonly attribute long selectedIndex;
+  readonly attribute ImageTrack? selectedTrack;
+};
+
+
+ +### Internal Slots ### {#imagetracklist-internal-slots} +: [[ready promise]] +:: The promise used to signal when the {{ImageTrackList}} has been populated + with {{ImageTrack}}s. + + NOTE: {{ImageTrack}} {{ImageTrack/frameCount}} may receive subsequent + updates until {{ImageDecoder/complete}} is `true`. + +: [[track list]] +:: The list of {{ImageTrack}}s describe by this {{ImageTrackList}}. + +: \[[selected index]] +:: The index of the selected track in {{ImageTrackList/[[track list]]}}. A + value of `-1` indeicates that no track is selected. + +### Attributes ### {#imagetracklist-attributes} +: ready +:: The {{ImageTrackList/ready}} getter steps are to return the + {{ImageTrackList/[[ready promise]]}}. + +: length +:: The {{ImageTrackList/length}} getter steps are to return the length of + {{ImageTrackList/[[track list]]}}. + +: selectedIndex +:: The {{ImageTrackList/selectedIndex}} getter steps are to return + {{ImageTrackList/[[selected index]]}}; + +: selectedTrack +:: The {{ImageTrackList/selectedTrack}} getter steps are: + 1. If {{ImageTrackList/[[selected index]]}} is `-1`, return `null`. + 2. Otherwise, return the ImageTrack from {{ImageTrackList/[[track list]]}} + at the position indicated by {{ImageTrackList/[[selected index]]}}. + +ImageTrack Interface {#imagetrack-interface} +-------------------------------------------- +
+
+interface ImageTrack : EventTarget {
+  readonly attribute boolean animated;
+  readonly attribute unsigned long frameCount;
+  readonly attribute unrestricted float repetitionCount;
+  attribute EventHandler onchange;
+  attribute boolean selected;
+};
+
+
+ +### Internal Slots ### {#imagetrack-internal-slots} +: \[[ImageDecoder]] +:: The {{ImageDecoder}} instance that constructed this {{ImageTrack}}. + +: \[[ImageTrackList]] +:: The {{ImageTrackList}} instance that lists this {{ImageTrack}}. + +: \[[animated]] +:: Indicates whether this track contains an animated image with multiple + frames. + +: [[frame count]] +:: The number of frames in this track. + +: [[repetition count]] +:: The number of times the animation is intended to repeat. + +: \[[selected]] +:: Indicates whether this track is selected for decoding. + +### Attributes ### {#imagetrack-attributes} + +: animated +:: The {{ImageTrack/animated}} getter steps are to return the value of + {{ImageTrack/[[animated]]}}. + + NOTE: This attribute provides an early indication that + {{ImageTrack/frameCount}} will ultimately exceed 0 for images where the + {{ImageTrack/frameCount}} starts at `0` and later increments as new + chunks of the {{ReadableStream}} {{ImageDecoderInit/data}} arrive. + +: frameCount +:: The {{ImageTrack/frameCount}} getter steps are to return the value of + {{ImageTrack/[[frame count]]}}. + +: repetitionCount +:: The {{ImageTrack/repetitionCount}} getter steps are to return the value of + {{ImageTrack/[[repetition count]]}}. + +: onchange +:: An [=event handler IDL attribute=] whose [=event handler event type=] is + {{ImageTrack/change}}. + +: selected +:: The {{ImageTrack/selected}} getter steps are to return the value of + {{ImageTrack/[[selected]]}}. + + The {{ImageTrack/selected}} setter steps are: + 1. If {{ImageTrack/[[ImageDecoder]]}}'s {{ImageDecoder/[[closed]]}} slot is + `true`, abort these steps. + 2. Let |newValue| be [=the given value=]. + 3. If |newValue| equals {{ImageTrack/[[selected]]}}, abort these steps. + 4. Assign |newValue| to {{ImageTrack/[[selected]]}}. + 5. Let |parentTrackList| be {{ImageTrack/[[ImageTrackList]]}} + 6. Let |oldSelectedIndex| be the value of |parentTrackList| + {{ImageTrackList/[[selected index]]}}. + 7. If |oldSelectedIndex| is not `-1`: + 1. Let |oldSelectedTrack| be the {{ImageTrack}} in |parentTrackList| + {{ImageTrackList/[[track list]]}} at the position of + |oldSelectedIndex|. + 2. Assign `false` to |oldSelectedTrack| {{ImageTrack/[[selected]]}} + + 8. If |newValue| is `true`, let |selectedIndex| be the index of [=this=] + {{ImageTrack}} within |parentTrackList|'s + {{ImageTrackList/[[track list]]}}. Otherwise, let |selectedIndex| be + `-1`. + 9. Assign |selectedIndex| to |parentTrackList| + {{ImageTrackList/[[selected index]]}}. + 10. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm on + {{ImageTrack/[[ImageDecoder]]}}. + 11. [=Queue a control message=] to {{ImageTrack/[[ImageDecoder]]}}'s + [=control message queue=] to update the internal selected track + index with |selectedIndex|. + + [=Running a control message=] to update the internal selected track index + means running these steps: + 1. Assign |selectedIndex| to + {{ImageDecoder/[[internal selected track index]]}}. + 2. Remove all entries from + {{ImageDecoder/[[progressive frame generations]]}}. + + +### Event Summary ### {#imagetracklist-eventsummary} + +: change +:: Fired at the {{ImageTrack}} when the {{ImageTrack/frameCount}} is altered. Security Considerations{#security-considerations}