Make it easy to play audio input in Audio pane

[MarcSkovMadsen]

I believe its hard to use the Audio pane as it does not support

• raw bytes
• in memory (BytesIO) files
• data urls
• webm format
• an empty string or empty bytes string as input.
  • It does support None. It should behave the same for other "empty" objects. Because that is what you might get back from FileInput or some custom Audio input.

Please support these. And consider adding the similar support for the Video pane.

Motivation

I've been with working on the missing AudioInput widget. By default you get audio in the browser in webm format. The easist is to transfer this as a data url to the python side. But the data url no matter whether its webm, wav or any other format is not supported by the Audio pane.

Thus you will have to convert it to replay it.

If you do what the FileInput does, i.e. convert to the raw bytes, then the Audio player cannot play these. Thus its also hard to use the Audio pane with the FileInput widget.

As far as I can see you will have to save to a temporary file on disk and provide the path to the file. This is cumbersome and inefficient.

But as webm is not supported by Audio pane you will have to convert to a supported format. As far as I can see this requires you to install ffmpeg which is not always possible for example in a controlled enterprise environment. If if you try to use ffmpeg wasm in the browser you will also have to figure out how to support a web worker in a custom JScomponent or similar.

All in all its very hard for a "normal" data scientist to input Audio from JSComponent or FileInput and play it in Audio pane.

[dillonroach]

@MarcSkovMadsen drive by comment, as I saw this while looking for something else; I've done some rough-and-ready audio loading with panel using librosa as the intermediary (e.g. below is dropped audio file to playback pane)

            file_name, file_content = next(iter(file_dropper.value.items()))
            audio = librosa.load(io.BytesIO(file_content))
            audio_pane.sample_rate = sample_rate=audio[1]
            audio_pane.object = np.int16(np.array(audio[0],dtype=np.float32)* 32767)

That's at least a way to get from raw (file_content) or bytesIO (skipping step one heh) object to audio pane out. Nobody wants another requirement, naturally, so you'd want to reproduce the librosa.load basically.