AudioServer.get_time_to_next_mix() can return negative values

[qqi0O0]

Godot version:
3.3.2

OS/device including version:
Macbook Pro 2018 13 inch, Big Sur 11.3.1

Issue description:
AudioServer.get_time_to_next_mix() can return negative values. It seems that this happens if a relatively longer stream (eg 2 minutes long) was just set in the audio player.
Expectation: time to the next mix should always be non negative.

Steps to reproduce:
In the same call to _process, set the stream of an AudioStreamPlayer then print AudioServer.get_time_to_next_mix().

Minimal reproduction project:
AudioServerBug.zip

[azagaya]

As far as i tested, you dont need to set the stream and call get_time_to_next_mix in the same call to _process. Just doing this also shows negative values sometimes:

func _process(_delta):
	print(AudioServer.get_time_to_next_mix())

[JestemStefan]

Can't reproduce in master build v4.0.dev.20210916.official [104a619]. This is wrong actually

But this still exists in 3.3.3. I will check what is the difference

[JestemStefan]

Update 3: I checked pre-alpha versions on 4.0 and

• v4.0.dev.20210727 - Issue exists
• v4.0.dev.20210811.official [7188cb6] - Issue exists
• v4.0.dev.20210820.official [75697c0] - Issue exists
• v4.0.dev.20210916.official [104a619] - Issue exists

Here is a code:

[akien-mga]

CC @godotengine/audio

[ellenhp]

Does this cause problems in practice? get_time_to_next_mix seems to me like a convenient way to say latency - time_since_last_mix which can be negative if a mix takes more time than expected or is delayed for any reason.

That said, I can't figure out what the default implementation of this method is doing:

godot/servers/audio_server.cpp

Lines 87 to 95 in d187bb4

	double AudioDriver::get_time_to_next_mix() {
	lock();
	uint64_t last_mix_time = _last_mix_time;
	uint64_t last_mix_frames = _last_mix_frames;
	unlock();
	double total = (OS::get_singleton()->get_ticks_usec() - last_mix_time) / 1000000.0;
	double mix_buffer = last_mix_frames / (double)get_mix_rate();
	return mix_buffer - total;
	}

[jitspoe]

Took a look at this, and _last_mix_frames is a bit of a misnomer. It's the number of frames to mix, set by void AudioDriver::update_mix_time (and only used in this function). Dividing that by the mix rate will give the time, in seconds, for each mix buffer.

So it's calculating how much time has passed since the last mix, converting that to seconds, and finding how long the mix buffer, in seconds is, and returning the difference between the two.

The thing that's tricky here is that, since the mixing happens in a different thread, you could query the time to the next mix and get a negative value since enough time has passed to start a mix, but it hasn't started doing the mix on the audio thread yet.

So if you play your sound, it might mix that frame, or it might be delayed 1 full mix. In fact, that has potential to happen even with values that are greater than 0.

UNLESS, the audio mixing does not start until the end of the frame, in which case, the fix for this is to simply return 0 if the value is less than 0.

All that being said, I'm trying to figure out how to actually use get_time_to_next_mix() in practice. For example, if I have some sounds I want to play exactly on a beat, knowing the time to the next mix could mean I could play them sooner, but they'd still be off. It's like I'd need a way to play a sound with a delay that happens within the mixing thread. This is probably not the place for this discussion, but I found this github issue when searching for a practical use for get_time_to_next_mix(), so if somebody knows how to apply it, maybe throw a link in here for wild internet wanderers like myself who might stumble upon this.

[akien-mga]

Is this still reproducible in 4.0 RC 3 or later?

[jitspoe]

Confirmed happening with latest mainline (synced within 24 hours). Printed the values in the _physics_process():

0.00624
0.008073
0.006923
0.009763
0.008351
0.000281
0.008812
0.009825
0.008703
0.001796
0.000551
-0.000373
0.005808
0.004655
0.004895
0.003486
0.008016
0.006418
0.001161
0.009841
0.004332
0.002797
0.005589
0.004317
-0.000922
0.009149
0.003952
0.002609
0.00648
0.005104
0.00961
0.008232
0.002127
0.000425
0.00399
0.002942
0.00674
0.005585
0.009908
0.004405
0.002604
0.00669
0.005392
0.000525
0.003639
0.002056
0.006845
0.00495
0.009323
0.007802
0.002259
-0.000091
0.008799
0.007804
0.002842
0.001793
0.005498
0.004205
0.008262
0.007158
0.00138
0.000334
0.003988
0.002972
0.007543
0.006121
0.000457
0.003648
0.002351
0.006701
0.005727
0.009792
0.008822
0.002839
0.001782
0.00575
0.008966
0.007932
0.00201
0.000966
0.005014
0.003922
0.007589
0.006646
0.001168
0.009952
0.004247
0.007264
0.006075
0.000407
0.009262
0.00347
0.002513
0.006177
0.004272
0.009567
0.002422
0.000398
0.005846
0.003941
0.00779
0.006468
-0.000706
0.009325
0.004489
0.003282
0.007297
0.000743
0.008988
0.002585
0.009816
0.004717
0.003466
0.008379
0.006977
0.00176
0.0061
0.004897
-0.000928
0.008913
0.003126
0.00194
0.006196
0.009403
0.008038
0.002511
0.001413

[MJacred]

Confirmed on Ubuntu 20.04, driver PulseAudio.

Could trigger it thrice in 16 attempts of running the scene of the MRP (after using the auto-upgrade). All negative values were always before Load music into player.

---

Godot Engine v4.0.2.stable.official.7a0977ce2 - https://godotengine.org
Vulkan API 1.3.194 - Forward+ - Using Vulkan Device #0: NVIDIA - NVIDIA GeForce GTX 970

iter	time to next mix	time since last mix
0	-0.00099402267574	0.012604
1	-0.00152202267574	0.013132
2	0.00925997732426	0.00235
3	0.00904397732426	0.002567
4	0.00889597732426	0.002714
Load music into player
5	0.00228797732426	0.009322
6	0.00791997732426	0.003691
7	0.00773397732426	0.003876
8	0.01125197732426	0.000358
9	0.01097397732426	0.000636
10	0.01082897732426	0.000782

[Calinou]

reduz said about the return value of AudioServer.get_time_to_next_mix():

I think its the estimated time until you get an audio mix callback. I think it needs to be used together with the audio latency value to get the time it will take for a sound played to be actually heard.