Problem getting length of ogg/opus blob on edge

[NeoCoderMatrix86]

I use your framework inside a blazor application (https://github.com/NeoCoderMatrix86/AudioCuesheetEditor) for editing cuesheets.

If I record some ogg/opus inside the application and play that record with your framework, I get an exception like this:

System.Text.Json.JsonException: The JSON value could not be converted to System.Int32. Path: $ | LineNumber: 0 | BytePositionInLine: 4.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'Null' as a number.
   at System.Text.Json.Utf8JsonReader.TryGetInt32(Int32& value)
   at System.Text.Json.Utf8JsonReader.GetInt32()
   at System.Text.Json.Serialization.Converters.Int32Converter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1[[System.Int32, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, Int32& value)
   at System.Text.Json.Serialization.JsonConverter`1[[System.Int32, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.Serialization.JsonConverter`1[[System.Int32, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.JsonConverter`1[[System.Int32, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ReadCoreAsObject(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadCore[Object](JsonConverter jsonConverter, Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadValueCore[Object](JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Deserialize(Utf8JsonReader& reader, Type returnType, JsonSerializerOptions options)
   at Microsoft.JSInterop.Infrastructure.DotNetDispatcher.ParseArguments(JSRuntime jsRuntime, String methodIdentifier, String arguments, Type[] parameterTypes)
   at Microsoft.JSInterop.Infrastructure.DotNetDispatcher.InvokeSynchronously(JSRuntime jsRuntime, DotNetInvocationInfo& callInfo, IDotNetObjectReference objectReference, String argsJson)
   at Microsoft.JSInterop.Infrastructure.DotNetDispatcher.BeginInvokeDotNet(JSRuntime jsRuntime, DotNetInvocationInfo invocationInfo, String argsJson)

[StefH]

Hello @NeoCoderMatrix86,

That's a very nice application !

I was able to import + play a opus file without any issues. See attached opus file which I used.

Brad Fiedel - Terminator Main Theme (Terminator OST).zip

[StefH]

@NeoCoderMatrix86
If you want to use a faster download service, take a look at : https://github.com/StefH/Blazor.DownloadFileFast

And if you are also interested in reading .mkv / .webm files, take a look here: https://github.com/StefH/Matroska

[NeoCoderMatrix86]

Thanks for the fast reply :). I'm currently investigating the bug and found some information:
The bug only occurs, when I record an audio file via "Record mode". Like you used some standard opus file, the application works fine. Maybe it is related to getting length of length of the recorded blob?!
I use the following code:
Record (javascript):

function setupAudioRecording() {
    navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => { handleAudioRecording(stream) }).catch(function (err) { handleAudioRecordingData = false; });
}

function handleAudioRecording(stream) {
    rec = new MediaRecorder(stream);
    rec.ondataavailable = e => {
        audioChunks.push(e.data);
    }
    rec.onstop = () => {
        let blob = new Blob(audioChunks, { 'type': 'audio/ogg; codecs=opus' });
        var url = URL.createObjectURL(blob);
        if (GLOBAL.ViewModeRecord !== null) {
            GLOBAL.ViewModeRecord.invokeMethodAsync("AudioRecordingFinished", url);
        }
    }
}

function startAudioRecording() {
    if (handleAudioRecordingData == true) {
        audioChunks = [];
        rec.start();
    }
}

function stopAudioRecording() {
    if (handleAudioRecordingData == true) {
        rec.stop();
    }
}

ViewModeRecord:

await JSRuntime.InvokeVoidAsync("startAudioRecording");

[JSInvokable()]
    public void AudioRecordingFinished(String url)
    {
        //TODO: Customizable via options
        Cuesheet.AudioFile = new AudioFile("Recording.ogg", url, "audio/ogg", true);
    }

AudioPlayer:

private async Task OnPlayClicked()
    {
        if (soundId != null)
        {
            await Howl.Pause(soundId);
        }
        else
        {
            if ((Cuesheet == null) || (Cuesheet.AudioFile == null) || (Cuesheet.AudioFile.PlaybackPossible == false))
            {
                throw new ArgumentNullException(nameof(Cuesheet.AudioFile));
            }
            var options = new HowlOptions
            {
                Sources = new[] { Cuesheet.AudioFile.ObjectURL },
                Formats = new[] { Cuesheet.AudioFile.AudioFileType.ToLower() },
                Html5 = true
            };
            soundId = await Howl.Play(options);
        }
    }

[StefH]

Sorry, I cannot really help you here.

One thing which I noticed is that ".ogg" is normally using "Ogg / Vorbis"

The demo file is an "Ogg - Opus" file.

[NeoCoderMatrix86]

Is it possible to Debug into the exception, to get to know, what is causing the problem? I'm currently stuck here, because I'm not so deep into sound processing with howler framework. Maybe an update of howler could go good?

[NeoCoderMatrix86]

Ok, I managed to debug the problem and this seems to be a Bug in your wrapper for me:

1. Recorded audio files have at default no duration on edge. Firefox handles it a bit different, but the problem comes with duration.
2. Inside your "JsInteropHowl.js" you call:

onplay: async function (id) {
                const duration = Math.round(howl.duration()); <-----duration is Infinite
                await dotnetReference.invokeMethodAsync('OnPlayCallback', id, duration);
            },

3. I don't know, how you handle infinite duration inside the "OnPlayCallback", but the error comes from there. So maybe a fix for no valid duration should be done?!

[StefH]

Did you try upgrading howler.js to a newer version?

[NeoCoderMatrix86]

I updated howler.js to version 2.2.1, but that doesn't fix the problem. I'm still investigating the problem, seems to be a known bug of chrome/edge, that the stream has no duration.

[StefH]

Can you attach your audio file which has this problem?

Then I can investigate / debug this on my system.

(And maybe searching / crossposting an issue to the howler.js github?)

[NeoCoderMatrix86]

Well I think its not a problem of howler, since the playback starts even with the exception above. I think its more a conversion in .NET in this wrapper here, that makes the exception.

Recording.txt

I attached the file and renamed it for uploading.

[StefH]

Playing this test file in my WASM example (play local OGG at https://stefh.github.io/Howler.Blazor) works fine on Edge. However, I did change the integer to double and passed the play-id to the 'duration' function, maybe that helps?

Please test version 0.9.4

[NeoCoderMatrix86]

With the new Version 0.9.4 I get this exception:

Uncaught (in promise) Error: System.Text.Json.JsonException: The JSON value could not be converted to System.Double. Path: $ | LineNumber: 0 | BytePositionInLine: 4.
---> System.InvalidOperationException: Cannot get the value of a token type 'Null' as a number.
at System.Text.Json.Utf8JsonReader.TryGetDouble(Double& value)
at System.Text.Json.Utf8JsonReader.GetDouble()
at System.Text.Json.Serialization.Converters.DoubleConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter1[[System.Double, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, Double& value) at System.Text.Json.Serialization.JsonConverter1[[System.Double, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
--- End of inner exception stack trace ---
at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
at System.Text.Json.Serialization.JsonConverter1[[System.Double, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) at System.Text.Json.Serialization.JsonConverter1[[System.Double, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ReadCoreAsObject(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadCore[Object](JsonConverter jsonConverter, Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadValueCore[Object](JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& state)
at System.Text.Json.JsonSerializer.Deserialize(Utf8JsonReader& reader, Type returnType, JsonSerializerOptions options)
at Microsoft.JSInterop.Infrastructure.DotNetDispatcher.ParseArguments(JSRuntime jsRuntime, String methodIdentifier, String arguments, Type[] parameterTypes)
at Microsoft.JSInterop.Infrastructure.DotNetDispatcher.InvokeSynchronously(JSRuntime jsRuntime, DotNetInvocationInfo& callInfo, IDotNetObjectReference objectReference, String argsJson)
at Microsoft.JSInterop.Infrastructure.DotNetDispatcher.BeginInvokeDotNet(JSRuntime jsRuntime, DotNetInvocationInfo invocationInfo, String argsJson)
at Object.endInvokeDotNetFromJS (https://localhost:44359/_framework/blazor.webassembly.js:1:4191)
at Object.invokeJSFromDotNet (https://localhost:44359/_framework/blazor.webassembly.js:1:3797)
at Object.w [as invokeJSFromDotNet] (https://localhost:44359/_framework/blazor.webassembly.js:1:64287)
at _mono_wasm_invoke_js_blazor (https://localhost:44359/_framework/dotnet.5.0.2.js:1:190800)
at do_icall (:wasm-function[10596]:0x194e58)
at do_icall_wrapper (:wasm-function[3305]:0x79df9)
at interp_exec_method (:wasm-function[2155]:0x44ad3)
at interp_runtime_invoke (:wasm-function[7862]:0x12efff)
at mono_jit_runtime_invoke (:wasm-function[7347]:0x118e5f)
at do_runtime_invoke (:wasm-function[3304]:0x79d42)

I'm quite not shure, why it works in your samples but not in my code. Did you reencode the audio file? Because it has an duration of 5 seconds on your test site? Which version of .NET do you use?

[StefH]

My example application is

<PropertyGroup>
    <TargetFramework>netstandard2.1</TargetFramework>
    <RazorLangVersion>3.0</RazorLangVersion>
    ...
  </PropertyGroup>

And you use .NET 5.0 I see.

Maybe there's a difference?

[NeoCoderMatrix86]

Ok, can you run a test on .NET 5.0? I can currently not switch to .netstandard since other libraries are not working than.

[StefH]

I created the same example application with NET 5
https://github.com/StefH/Howler.Blazor/tree/duration-net5/examples/Howler.Blazor-WASM-AudioPlayer-NET5

And in Edge, it still works fine using your recording.ogg

[NeoCoderMatrix86]

I just checked, how you implemented your example and what I do in my application. To get the total time, you use the code

Howl.OnPlay += e =>
        {
            ErrorMessage = string.Empty;
            State = "Playing";
            TotalTime = e.TotalTime;

            StateHasChanged();
        };

While I use an update timer:

audioUpdateTimer = new Timer(500);
        audioUpdateTimer.AutoReset = true;
        audioUpdateTimer.Elapsed += async delegate
        {
            AudioIsPlaying = await Howl.IsPlaying();
            if (AudioIsPlaying == true)
            {
                CurrentPlaybackPosition = await Howl.GetCurrentTime();
                if (TotalTime == null)
                {
                    TotalTime = Howl.TotalTime;
                }
            }
            StateHasChanged();
        };

So I think, in this case we have a problem of the function Howl.TotalTime. Maybe this is the Bug. Could you investigate, if you can call it inside the OnPlay Event?

[NeoCoderMatrix86]

I just tried, getting TotalTime in the OnPlay Event. My application also crashes when I use the example code. I'm currently a bit stuck, it must have to do with the duration.

[StefH]

The only thing what you can try is copy that "JsInteropHowl.js" file from this project into your project, use it in the html

and change this code:

getTotalTime: function () {
        if (howl) {
            const duration = howl.duration();
            return Math.round(duration || 0);
        }

        return 0;
    }

to

getTotalTime: function () {
        if (howl) {
            const duration = howl.duration();
            if (duration === Infinity) {
              return 0;
            }
            return Math.round(duration || 0);
        }

        return 0;
    }

and the same for

onplay: async function (id) {
                const duration = Math.round(howl.duration(id));
                await dotnetReference.invokeMethodAsync('OnPlayCallback', id, duration);
            },

[NeoCoderMatrix86]

Ok, good idea. I made the changes you suggested and debugged the javascript.
The code crashes as before here:

onplay: async function (id) {
                const duration = Math.round(howl.duration(id)); <--------duration is Infinite
                await dotnetReference.invokeMethodAsync('OnPlayCallback', id, duration); <-------crash comes here
            },

What happens inside OnPlayCallback?

[StefH]

I don't know why the duration is Infinite in your case.
The workaround what you can try is:

onplay: async function (id) {
                const duration = howl.duration(id);
                if (duration === Infinity) {
                    await dotnetReference.invokeMethodAsync('OnPlayCallback', id, 0);
                }
                else {
                  await dotnetReference.invokeMethodAsync('OnPlayCallback', id, Math.round(duration));
                }
            },

This make sure to post 0 to .NET callback.

[NeoCoderMatrix86]

Ok, this fixes the bug. You should include the javascript code in your release. The duration is infinite because it is a recording saved in a blob. Edge and Chrome doesn't automatically calculate the length and attach it, since Firefox does it. After one play of the recording, the length is calculated and the duration is no more infinite.

[StefH]

In that case, I will also change the C# code.

I'll change the duration from double to int? so that this seems more logical.

You can expect a new NuGet later today.

[StefH]

#13