You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FelixMcFelix opened this issue
Aug 8, 2022
· 0 comments
Labels
breakingWill either cause existing code to fail to compile, or cause substantial behaviour changesdriverRelates to the driver or one of its sub-tasks.eventsRelates to driver event handling/generation.receiveRelated to voice receive, decoding, decryption, or events
For users to correctly handle packet reordering or jitter due to network instability, they currently need to disable our decoding features and reimplement decoding logic themselves -- this gets more complex when handling audio data from several users, which is interspersed and can arrive at different parts of each 20ms window. This makes downmixing particular nasty.
Ideally, we should take this complexity on ourselves. Doing so will require a few changes:
Packet events must be separated from decode events. We likely want to write and store audio data into an Arc<[u8]> to minimise copying between these.
Decode events must be generated every 20ms, and include audio from every active participant (accessible by SSRC).
Each user needs their own resizable jitter/reorder buffer (likely a deque of Arcs). This should have user-configurable target and max depths. If the buffer was allowed to empty (e.g., user disconnects or stops speaking), then it must build back up to its target occupancy before we decode packets again.
Packet events should no longer include audio data -- this is to be moved to the decode event.
The text was updated successfully, but these errors were encountered:
FelixMcFelix
added
driver
Relates to the driver or one of its sub-tasks.
events
Relates to driver event handling/generation.
breaking
Will either cause existing code to fail to compile, or cause substantial behaviour changes
receive
Related to voice receive, decoding, decryption, or events
labels
Aug 8, 2022
This PR Introduces a new `VoiceTick` event which collects and reorders all RTP packets to smooth over network instability, as well as to synchronise user audio streams. Raw packet events have been moved to `RtpPacket`, while `SpeakingUpdate`s have been removed as they can be easily computed using the `silent`/`speaking` audio maps included in each event.
Closes#146.
This PR Introduces a new `VoiceTick` event which collects and reorders all RTP packets to smooth over network instability, as well as to synchronise user audio streams. Raw packet events have been moved to `RtpPacket`, while `SpeakingUpdate`s have been removed as they can be easily computed using the `silent`/`speaking` audio maps included in each event.
Closesserenity-rs#146.
breakingWill either cause existing code to fail to compile, or cause substantial behaviour changesdriverRelates to the driver or one of its sub-tasks.eventsRelates to driver event handling/generation.receiveRelated to voice receive, decoding, decryption, or events
For users to correctly handle packet reordering or jitter due to network instability, they currently need to disable our decoding features and reimplement decoding logic themselves -- this gets more complex when handling audio data from several users, which is interspersed and can arrive at different parts of each 20ms window. This makes downmixing particular nasty.
Ideally, we should take this complexity on ourselves. Doing so will require a few changes:
Arc
s). This should have user-configurable target and max depths. If the buffer was allowed to empty (e.g., user disconnects or stops speaking), then it must build back up to its target occupancy before we decode packets again.The text was updated successfully, but these errors were encountered: