Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(client): ♻️ Move decoder out of ClientCoreContext; fix IDR race condition #2421

Merged
merged 4 commits into from
Sep 28, 2024

Conversation

zmerp
Copy link
Member

@zmerp zmerp commented Sep 24, 2024

This PR creates a new API for using the decoder. Removed the distinction between internal and external decoder, there is only an "external" implementation. The C API had been tweaked to reflect the internal changes. A significant improvement of this refactor is zero-copy decoder integration for visionOS.

This PR also fixed one bug where the video stream might start in a corrupted state because of a race condition.

@zmerp zmerp changed the title refactor(client): ♻️ Move decoder out of ClientContext; fix IDR race condition refactor(client): ♻️ Move decoder out of ClientCoreContext; fix IDR race condition Sep 24, 2024
@zmerp zmerp force-pushed the decoder-refactor branch 2 times, most recently from d4cda2c to 2c9442b Compare September 24, 2024 15:59
if let Some(source) = self.decoder_source.as_mut() {
while frame_result.is_none() && Instant::now() < frame_poll_deadline {
frame_result = source.get_frame();
thread::yield_now();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the image queue lock actually cause waiting or is this just a busy loop?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually a busy loop. When there is a healthy amount of buffering, this will succeed immediately without busy looping. The code is the same as before just moved from lib.rs to stream.rs.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea ik, but I think we should still add like a tiny wait if we're at it

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like 200 microsecs or 500 microsecs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By reading online, seems that yield() is generally discouraged, mainly because the implementation is platform dependent and even within the same platform is quite unpredictable. The thread could actually resume immediately or wait up to 3ms. So it should never be used unless we did extensive profiling checking it does exactly what we want. It's much better to use our own sleep to balance out CPU usage and latency depending on our needs. So i will change this.

@The-personified-devil
Copy link
Collaborator

The-personified-devil commented Sep 25, 2024

Lgtm so far, but I also can't properly review due to complexity
also converting to draft because it wasn't tested at all

@The-personified-devil The-personified-devil marked this pull request as draft September 25, 2024 00:01
@zmerp zmerp marked this pull request as ready for review September 28, 2024 18:58
@zmerp
Copy link
Member Author

zmerp commented Sep 28, 2024

Tested and working

@zmerp zmerp merged commit 370455d into master Sep 28, 2024
9 checks passed
@zmerp zmerp deleted the decoder-refactor branch September 28, 2024 22:54
zmerp added a commit that referenced this pull request Dec 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants