The ideal container format

[MasterInQuestion]

[[
[ MasterInQuestion @ CE 2024-05-28 18:18:26 UTC:
https://github.com/exiftool/exiftool/issues/94#issuecomment-1546820018
    Based on the context of a few posts that have gone missing asserting CFR (Constant Frame Rate) and superiority of MP4.

----

    MKV may be favored for having less restriction on the media formats (codecs) to contain.
    Any post-analog adequately designed container format (codec too) should be able to handle arbitrary frame rate (and VFR).
    For digital media, there's really no frame rate (nor time base [1]): only presentation timestamps (PTS).

    MP4 is far from the most optimal container format. (so is MKV) [4]
    For the typical application (local media playback), the container may not even be needed.

[ [1]
    For typical timestamp implementations: time base shouldn't be of concern.
    But when tailored to optimize for specific applications: time base may actually become a thing.

    Typical high-precision timestamps are based on Int64 and of a granularity of μs (i.e. 1/1,000,000 time base).
    Which should allow a max duration of (( (2^63 - 1) / 1,000,000 / (3,600 * 24) / 365.2425 ≈ 292,277 )) years in this configuration.
    .
    However support for granularity smaller than 1/10,000 may be pointless for common systems. (unreachable target accuracy for underlying system; due to beyond threshold latency [3]) [2]
    And using Int32 instead may provide relatively significant performance benefit in certain cases.
    .
    Thus an alternative configuration based on Int32 and of granularity of ms (1/1,000 s) is plausible:
    Which should still allow a max duration of (( (2^31 - 1) / 1,000 / (3,600 * 24) ≈ 24.8551348 )) days. (and max 1,000 FPS) ]

[ [2]
    Significant amount of audio handling are of 48,000 Hz. (1/48,000 < 1/10,000 indeed)
    But what handles the final audio playback is typically dedicated hardware (not software based on CPU [ Precisely, general purpose processors. ]).
    And which should load far more than 1 sample at once (more than 1 sample per frame), and is FIFO like.

    Note audio playback is somewhat different than video: the sample value (content) is uttered at the timestamp but not maintained across the whole duration. ]

[ [3]
    The system's latency may be estimated using `while ( 1 ) { printTS(); };` alike structure.
    E.g. in Bash:
[[

	x0="${EPOCHREALTIME%%.*}";
	while true;
	do {
	x1="$EPOCHREALTIME";
	x2="${x1%%.*}";
	[ "$x2" == "$x0" ] && continue;
	echo -E "$x1";
	x0="$x2";
	};
	done;
#
#	Less likely to work:
# [[
#	while true; do { [ "${EPOCHREALTIME##*.}" -eq 0 ] && echo -E "$EPOCHREALTIME"; }; done;
# ]]
#

]] ]

[ [4]
    Why exactly?
    MP4 had the information critical [1] for media playback stored at 1 location (usually near the end of file; or the start of file).
    Such design has caused some problems. [ See FFmpeg's documentation on related muxers. ]
    .
    Timestamps could be wrapped as part of each packet (encoded frame), thereby achieve superior performance meanwhile getting rid of all aforementioned problems.
    Such was somewhat done in MKV. But of significant overhead. (complicated "cues", and many more)

    More on the topic:
    https://github.com/MasterInQuestion/talk/discussions/3

[1] "moov atom", in MP4 terminology. Mostly contained the presentation timestamps, and frame dimensions. ]

    ----

    The ideal container format should be based on a general purpose container format: 7z should be a good start point. ]

----

[ MasterInQuestion @ CE 2023-05-14 09:55:21 UTC:
https://github.com/exiftool/exiftool/issues/94#issuecomment-1546859236
    https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Containers#mpeg-4_mp4

    The description covers more than just H.264: intended for all codecs (both audio/video) in general.
    It's a functional analysis on the underpinnings: whose validity is irrelevant of whether particular implementations work or not.

    Unnecessary elements may indeed exist (or even be required).
    But indeed they do not have to. ]
]]

[MasterInQuestion]

    Name it MPEG-X, essentially 7z archive with alike content:
[ "eg.mpx"
    "etc/Comment on Random Non-Sense.txt"
    "!.mpp" [ Media Presentation Profile ]
    "a1.flac"
    "a2.opus"
    "a3.vorbis"
    "a4.aac"
    "v1.vp9"
    "v2.h264"
    "v3.h265"
    "v4.av1" ]

    It could also sort of address some legacy JPEG problems:
[ "eg.jpx"
    "!.mpp" [ Color Profile etc. ]
    "v.jpg" ]
    Some JPEG may receive further size reduction this way... Real losslessly.


    Media Presentation Profile is meant to describe whatsoever possible rendering in Plain Text.
    Complex blob like MXF, PSD, 3DS, DWG etc. [1] would there be pointless.
    [ This should also supersede MIDI audio. Extending further could even become a game engine..? (and somewhat complement HTML) ]

    For some cases a single ".mpp" file should suffice. (XVG)

    Attention shall be taken to not retrace the serious failure of SVG:
    https://github.com/MasterInQuestion/Markup/issues/2
    https://github.com/MasterInQuestion/Markup/blob/main/Junkyard%20Tornado.svg?short_path=1

[1] https://www.google.com/search?hl=en&gl=us&num=100&q=site:adobe.com/creativecloud/file-types+PSD|3DS|DWG


    The Presentation Profile shall be more focused on the Presentation part:
    I.e. there shall be the distinction between "sources" and "post-rendering" (manipulation based on the sources).

    Which means the MPP itself doesn't promote (though support) embedding raw vector sources right inline.
    (using independent files and referencing shall be preferable)

    The format defining such vector sources shall be named XVG.
    Note the Graphics may contain audio too.
    .
    There shall be no essential difference between MPP/XVG:
    XVG is essentially MPP without referring any external source. (thus independently presentable)

[MasterInQuestion]

    The simplest exemplar:
    "_00_00_00.000000.webp"
    "_00_00_00.033333.webp"
    "_00_00_00.066667.webp"
    "_00_00_00.100000.webp"
    "_10_00_00.000000.jxl"
    "_10_00_01.000000" -> "./1__00_00_00.000000.jxl" [ Symlink. ]
    "_10_00_01.500000" -> "./_00_00_00.066667.webp"
    "_10_00_02.000000.webp" -> "./_00_00_00.100000.webp"
    "_23_59_59.999999.heif"
    "1__00_00_00.000000.jxl"

    Done with the SMPTE non-sense:
    https://trac.ffmpeg.org/ticket/11249#comment:13

[MasterInQuestion]

= Where major current container formats exactly failed =

    ...