support vorbis and flac sound encoding

[totaam]

Issue migrated from trac ticket # 246

component: core | priority: minor | resolution: fixed

2013-02-04 07:28:47: onlyjob created the issue

---

At the moment xpra-0.8.x has incomplete "vorbis" and "flac" audio forwarding support leaving only "mp3" as working alternative (I couldn't make "aac" work on client despite having the same packages installed).

MP3 is just retarded comparing to Vorbis in regards to bandwidth and audio quality (not to mention support for more than 2 channels).

AAC is not-so-free?

Flac is great license- and quality-wise (loseless) so IMHO the best order for codecs would be Vorbis > Flac > MP3 > AAC.

Please support Vorbis and Flac.

[totaam]

2013-02-04 07:35:35: antoine changed status from new to accepted

[totaam]

2013-02-04 07:35:35: antoine changed owner from ** to antoine

[totaam]

2013-02-04 07:35:35: antoine commented

---

Yes, this is planned, in fact the code already supports specifying the codecs, the gstreamer support code is mostly there, etc - only needs a little bit more work.
But there was not enough time to get this tested properly for 0.8, hopefully this can be completed in time for 0.9. There are other, more important issues to do with sound though: buffering and latency being the main problem.

[[BR]]

As can be seen here: [/browser/tags/v0.8.x/src/xpra/sound/gstreamer_util.py], the codecs that are meant to be supported (eventually) are exactly what you ask for, namely: vorbis, flac, aac and mp3.

[totaam]

2013-02-17 14:08:01: antoine edited the issue description

[totaam]

2013-02-17 14:08:01: antoine changed title from 0.8.1: please support vorbis and flac encoding. to support vorbis and flac sound encoding

[totaam]

2013-02-27 15:54:11: antoine commented

---

r2833 adds support for wav and flac. vorbis and aac just stubbornly refuse to work.

Here's how you can test the code from the command line. This is easier than starting a session with a pulseaudio server and all the codec arguments and then starting a media player, etc:

• record whatever is playing from your soundcard at the time into "test.mp3":

XPRA_SOUND_DEBUG=1 ./xpra/sound/src.py test.mp3

• verify that the file exists and has valid mp3 data in it:

mplayer test.mp3

• play it back using xpra's code:

XPRA_SOUND_DEBUG=1 ./xpra/sound/sink.py test.mp3

• then test with a real session and a pulseaudio server (longer - see Sound for details), ie:

xpra attach --speaker-codec=flac
(...)
using audio codec: FLAC

You can repeat this with the other supported file extensions: ".flac" and ".wav".

---

Now, to add support for more formats (say vorbis, ac3 or aac for example - or maybe even some more obscure ones like amr, gsm or dca?), here is what is needed:

• find a pipeline that records a valid file in that format (inc container format), ie:

gst-launch-0.10 pulsesrc device=alsa_output.pci-0000_00_14.2.analog-surround-51.monitor ! \
    vorbisenc ! oggmux ! filesink location=test.vorbis

• verify that the file is valid (look for container error messages - the file needs to play "cleanly"):

mplayer test.vorbis

• find the corresponding play pipeline:

gst-launch filesrc location=test.vorbis ! oggdemux ! vorbisdec ! \
    audioconvert ! audioresample ! autoaudiosink

• modify CODECS and CODEC_ORDER definitions in xpra.sound.gstreamer_util accordingly, ie:

#format: encoder, formatter, decoder, parser
CODECS = {
            VORBIS : ("vorbisenc", "oggmux", "vorbisdec", "oggdemux"),
...
CODEC_ORDER = [MP3, FLAC, WAV, VORBIS]

• as per the comments above, verify that the src.py and sink.py work as expected. The pipeline should be (almost) identical to the one used with gst-launch.
  When it does not, like the "vorbis" example above, weep.

---

Notes:

• the workflow above definitely works as this is what I have used to add support for opus, speex and wavpack in r2834
• If we need a completely different pipeline to support other formats, we can do that too - it's not too late for that as the code is still fairly small and self-contained. But hopefully we don't have to.
• The container format is important, I've used "oggmux" / "oggdemux" for most of the formats because it is one of the most widely used container format and well supported. But there are others... Worth a try. Especially when we start dropping some buffers because of network congestion, the container format will become more important in dealing with those "broken" streams.
• The list of gstreamer plugins that you can use can be found here (note that you may need to install extra packages to support all the plugins)
• vorbis is a good format, it would be good to find out what is wrong with it..

[totaam]

2013-03-21 03:02:07: antoine changed status from accepted to closed

[totaam]

2013-03-21 03:02:07: antoine changed resolution from ** to fixed

[totaam]

2013-03-21 03:02:07: antoine commented

---

Not going to bother with vorbis and aac since there are better alternatives.

As of now, we support:

• flac
• mp3
• wav
• opus (*)
• speex
• wavpack

win32 needs a newer build of gstreamer to support "opus", maybe #299 will help