Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CompressedVideo format strings #155

Merged
merged 4 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion internal/__snapshots__/exportTypeScriptSchemas.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -204,13 +204,32 @@ export type CompressedVideo = {
* Compressed video frame data.
*
* For packet-based video codecs this data must begin and end on packet boundaries (no partial packets), and must contain enough video packets to decode exactly one image (either a keyframe or delta frame). Note: Foxglove does not support video streams that include B frames because they require lookahead.
*
* Specifically, the requirements for different \`format\` values are:
*
* - \`h264\`
* - Use Annex B formatted data
* - Each CompressedVideo message should contain enough NAL units to decode exactly one video frame
* - Each message containing a key frame (IDR) must also include a SPS NAL unit
*
* - \`h265\` (HEVC)
* - Use Annex B formatted data
* - Each CompressedVideo message should contain enough NAL units to decode exactly one video frame
* - Each message containing a key frame (IRAP) must also include relevant VPS/SPS/PPS NAL units
*
* - \`av1\`
* - Use the "Low overhead bitstream format" (section 5.2)
* - Each CompressedVideo message should contain enough OBUs to decode exactly one video frame
* - Each message containing a key frame must also include a Sequence Header OBU
*/
data: Uint8Array;

/**
* Video format.
*
* Supported values: \`h264\` (Annex B formatted data only)
* Supported values: \`h264\`, \`h265\`, \`av1\`.
*
* Note: compressed video support is subject to hardware limitations and patent licensing, so not all encodings may be supported on all platforms. See more about [H.265 support](https://caniuse.com/hevc) and [AV1 support](https://caniuse.com/av1).
*/
format: string;
};
Expand Down
25 changes: 22 additions & 3 deletions internal/schemas.ts
Original file line number Diff line number Diff line change
Expand Up @@ -810,13 +810,32 @@ const CompressedVideo: FoxgloveMessageSchema = {
{
name: "data",
type: { type: "primitive", name: "bytes" },
description:
"Compressed video frame data.\n\nFor packet-based video codecs this data must begin and end on packet boundaries (no partial packets), and must contain enough video packets to decode exactly one image (either a keyframe or delta frame). Note: Foxglove does not support video streams that include B frames because they require lookahead.",
description: `Compressed video frame data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we point here to the format field for more info on what this should look like?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured since they are right next to each other maybe it’s fine? Could add it for redundancy though.


For packet-based video codecs this data must begin and end on packet boundaries (no partial packets), and must contain enough video packets to decode exactly one image (either a keyframe or delta frame). Note: Foxglove does not support video streams that include B frames because they require lookahead.

Specifically, the requirements for different \`format\` values are:

- \`h264\`
- Use Annex B formatted data
- Each CompressedVideo message should contain enough NAL units to decode exactly one video frame
- Each message containing a key frame (IDR) must also include a SPS NAL unit

- \`h265\` (HEVC)
- Use Annex B formatted data
- Each CompressedVideo message should contain enough NAL units to decode exactly one video frame
- Each message containing a key frame (IRAP) must also include relevant VPS/SPS/PPS NAL units

- \`av1\`
- Use the "Low overhead bitstream format" (section 5.2)
- Each CompressedVideo message should contain enough OBUs to decode exactly one video frame
- Each message containing a key frame must also include a Sequence Header OBU`,
},
{
name: "format",
type: { type: "primitive", name: "string" },
description: "Video format.\n\nSupported values: `h264` (Annex B formatted data only)",
description:
"Video format.\n\nSupported values: `h264`, `h265`, `av1`.\n\nNote: compressed video support is subject to hardware limitations and patent licensing, so not all encodings may be supported on all platforms. See more about [H.265 support](https://caniuse.com/hevc) and [AV1 support](https://caniuse.com/av1).",
},
],
};
Expand Down
21 changes: 20 additions & 1 deletion ros_foxglove_msgs/ros1/CompressedVideo.msg

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 20 additions & 1 deletion ros_foxglove_msgs/ros2/CompressedVideo.msg

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 20 additions & 1 deletion schemas/README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 20 additions & 1 deletion schemas/flatbuffer/CompressedVideo.fbs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions schemas/jsonschema/CompressedVideo.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions schemas/jsonschema/index.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 20 additions & 1 deletion schemas/omgidl/foxglove/CompressedVideo.idl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 20 additions & 1 deletion schemas/proto/foxglove/CompressedVideo.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 20 additions & 1 deletion schemas/ros1/CompressedVideo.msg

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading