Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add: simple st20tx plugin for gstreamer #1015

Merged
merged 3 commits into from
Dec 6, 2024

Conversation

DawidWesierski4
Copy link
Collaborator

add simple st20tx plugin for gstreamer, allows for simple st2110 tx stream creation.
Known bugs

Tested against the ffmpeg st20 rx plugin and found corruption in the video stream. This is likely due to frame creation mechanizm in the _chain function

Working on fixing it now

add simple st20tx plugin for gstreamer, allows for
simple st2110 tx stream creation.
Known bugs

Tested against the ffmpeg st20 rx plugin and found
corruption in the video stream. This is likely due
to frame creation mechanizm in the _chain function

Working on fixing it now
@DawidWesierski4
Copy link
Collaborator Author

DawidWesierski4 commented Dec 2, 2024

image
this is how the corruption looks like

even though
image
ffmpeg reports correct number of frames received when you try to extract them with

with

ffmpeg -f \
rawvideo \
-video_size \
1920x1080 \
-pix_fmt \
yuv422p10le \
-i "$OUTPUT" \
-r 60 \
src/frames/debug%d.png

it reports 1440 frames 2.4 times over
This i don't know why
image

working on ironing out the quirks

@DawidWesierski4
Copy link
Collaborator Author

Also analysis of the binary files indicates i have a stable 0x90 Bytes offset in the data figuring it out why
image

@DawidWesierski4
Copy link
Collaborator Author

the data shift seem to originate from the data handed me by

rawvideoparse format=y42b width=1920 height=1080 framerate=60/1
0x7fffe000f010:	0x10	0xf9	0x59	0x55	0x55	0x55	0x00	0x00
0x7fffe000f018:	0x01	0x00	0x00	0x00	0x00	0x00	0x01	0x00
0x7fffe000f020:	0x01	0x00	0x00	0x00	0x00	0x00	0x00	0x00
0x7fffe000f028:	0xc0	0x99	0xec	0xf7	0xff	0x7f	0x00	0x00
0x7fffe000f030:	0x00	0x00	0x00	0x00	0x00	0x00	0x00	0x00
0x7fffe000f038:	0x40	0xb9	0xec	0xf7	0xff	0x7f	0x00	0x00
0x7fffe000f040:	0x02	0x00	0x00	0x00	0x00	0x00	0x00	0x00
0x7fffe000f048:	0x00	0xa9	0x83	0x55	0x55	0x55	0x00	0x00
0x7fffe000f050:	0x50	0xf0	0x59	0x55	0x55	0x55	0x00	0x00
0x7fffe000f058:	0x00	0x00	0x00	0x00	0x00	0x00	0x00	0x00
0x7fffe000f060:	0x1f	0x48	0x3f	0x00	0x00	0x00	0x00	0x00
0x7fffe000f068:	0x1f	0x00	0x00	0x00	0x00	0x00	0x00	0x00
0x7fffe000f070:	0x00	0x00	0x00	0x00	0x00	0x00	0x00	0x00
0x7fffe000f078:	0x00	0x48	0x3f	0x00	0x00	0x00	0x00	0x00
0x7fffe000f080:	0xaf	0x48	0x3f	0x00	0x00	0x00	0x00	0x00
0x7fffe000f088:	0xa0	0xf0	0x00	0xe0	0xff	0x7f	0x00	0x00
0x7fffe000f090:	0x00	0x00	0x00	0x00	0x00	0x00	0x00	0x00
0x7fffe000f098:	0x00	0x00	0x00	0x00	0x00	0x00	0x00	0x00

It seems that this data is always the same added at the beginning

@DawidWesierski4
Copy link
Collaborator Author

Hi the issue was the format read the format wrong the y42b has 8bit depth

https://gstreamer.freedesktop.org/documentation/additional/design/mediatype-video-raw.html?gi-language=c
the correct one for yuv 422 planar 10 bit depth little endian would be I422_10LE

huge thx for Mateusz for noticing

Add the status
Fix the initialization

Tested and there is an problem with small
Corruption in the left top corner 73 x 9 pixels
seem to be discolored and 1 x 9 pixels have quite
big discoloration

The initialization of plugin after the deinitialization
needs to be fixed
@DawidWesierski4
Copy link
Collaborator Author

Ok ironed out most of the issue working on small corruption on left top corner in buxfix

also -> the IMTL doesn't really mesh well with rapind init and deinit which we can address later

@DawidWesierski4 DawidWesierski4 marked this pull request as ready for review December 5, 2024 17:37
Copy link
Contributor

@MateuszGrabuszynski MateuszGrabuszynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few things to fix, but nothing major. We discussed the topic of fractions on fps values already - I suggest adding a TODO comment.

ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
ecosystem/gstreamer_plugin/gstmtltxsink.c Outdated Show resolved Hide resolved
@DawidWesierski4 DawidWesierski4 force-pushed the gst-plugin branch 9 times, most recently from 2fa23f3 to 809e672 Compare December 6, 2024 10:28
@DawidWesierski4 DawidWesierski4 changed the title Add simple st20tx plugin for gstreamer Add: simple st20tx plugin for gstreamer Dec 6, 2024
Copy link
Collaborator

@skolelis skolelis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DawidWesierski4 DawidWesierski4 merged commit 0de0be7 into OpenVisualCloud:main Dec 6, 2024
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants