Add support for LHDC v2 A2DP source and sink

[arkq]

This PR is a follow-up for #672

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.42%. Comparing base (2357d98) to head (ab7ad97).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #742      +/-   ##
==========================================
- Coverage   70.45%   70.42%   -0.03%     
==========================================
  Files          96       96              
  Lines       16024    16024              
  Branches     2516     2515       -1     
==========================================
- Hits        11289    11285       -4     
- Misses       4616     4620       +4     
  Partials      119      119              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[arkq]

@anonymix007 I'm trying to add support for LHDC v2 using your library, but it seems that something is not right. As I've mentioned earlier, I have a retail device which is able to stream LHDC v2 codec (Huawei Mate 20 Pro). Unfortunately, the lhdcBT_dec_decode is not able to properly decode the stream. Captured RTP stream (from the phone) you can find here:
a2dp-sink-LHDC-v2-s32-48000-2c.zip. The *.rtp file is a hex dump of each A2DP packet, while the *.btd file is a dump which can be used with ./test/test-io to feed BT stream directly to the a2dp_lhdc_dec_thread function as follows (the --dump option will create decoded wav file):

./test/test-io lhdc-v2 --input-bt hw20-a2dp-sink-LHDC-v2-s32-48000-2c.btd --dump

It seems that the decoder is not able to decode most of the frames...

Used A2DP configuration:

$ a2dpconf lhdc-v2:3a050000324c140001
LHDC v2 <hex:3a050000324c140001> {
  vendor-id:32 = 0x0000053a [Savitech Corp.,]
  vendor-codec-id:16 = 0x4c32
  <reserved>:2
  bit-depth:2 = 24
  sample-rate:4 = 48000
  low-latency:1 = false
  max-bitrate:3 = 900
  version:4 = 0
  <reserved>:4
  ch-split-mode:4 = None
}

However, when stream is created with lhdcBT_encode (in order to create LHDC v2), the decoder is able to decode something. I'm not sure whether I've used the lhdcBT_encode in a right way (I've been trying to follow code from your lhdcenc_stereo.c), but the decoder is able to decode only the right channel. The left channel seems to be some garbage/noise. Also, the PCM format for lhdcBT_encode does not seem to be interleaved stereo s32le, but only a single channel.

Encoded and decoded sine with patch as follows:

diff --git a/src/a2dp-lhdc.c b/src/a2dp-lhdc.c
index f6d50e7..08fc4a2 100644
--- a/src/a2dp-lhdc.c
+++ b/src/a2dp-lhdc.c
@@ -303,7 +303,11 @@ void *a2dp_lhdc_enc_thread(struct ba_transport_pcm *t_pcm) {
if (codec_id == A2DP_CODEC_VENDOR_ID(LHDC_V2_VENDOR_ID, LHDC_V2_CODEC_ID)) {
-                               if ((rv = lhdcBT_encode(handle, input, bt.tail)) < 0) {
+                               int32_t xxx[2048] = { 0 };
+                               for (size_t i = 0; i < lhdc_ch_samples; i++)
+                                       xxx[i] = input[i*2];
+                               if ((rv = lhdcBT_encode(handle, xxx, bt.tail)) < 0) {
error("LHDC encoding error: %d", rv);
break;
}

./test/test-io lhdc-v2 --dump

Left channel is garbage, and the encoded channel is decoded as a right one.

Also, it seems that the lhdcBT_encode does not respect MTU. Even though the MTU is set to something like 900 bytes, the encoder writes ~2k of data. So, maybe the payload should be fragmented here on the RTP level?

Do you have any idea what might be wrong with v2 here?

In next few days I'll try to get some Android phone with LHDC v3 to test the v3 decoder.

[anonymix007]

PCM data ends up in this function:

/**
 * LHDC encode function
 *  fb: LHDC control block.
 *  wav : The PCM data. please input non-planer and compact PCM data.
 *        (eg. The input stream format is 96KHz/24bits stereo and the LHDC request 512 samples for each frame.
 *         So the PCM data length should be 512 * 2 * (24/8) = 3072 bytes. The data order should be L/R, L/R, L/R....).
 *  ns  : The number of samples, not PCM data byte length. The LHDC encoder only supports 512.
 *  final   : Fixed to 0.
 *  out : Output buffer pointer.
 *  out_len : The output buffer size to protect overwrite.
 *
 *  Return value :
 *      The return value should be the encoded size, otherwise an error occurs (less than or equal to 0)
 *
 */
int LossyEncoderProcessWav(FFT_BLOCK *fb, unsigned char *wav, int ns, int final, unsigned char *out, int out_len);

I'll take a look at the dump a bit later.

Also, it seems that the lhdcBT_encode does not respect MTU. Even though the MTU is set to something like 900 bytes, the encoder writes ~2k of data.

This could be a bug in my implementation or normal behavior, I'm not sure. Is it happening for just a few first frames or is it consistent?

So, maybe the payload should be fragmented here on the RTP level?

AFAIU it won't work because sink devices do not do anything to defragment payloads afterwards.

[anonymix007]

The *.rtp file is a hex dump of each A2DP packet

These don't look like correct LHDC frames to me.
For example, these are first 2 lines:

8060000200000200000000ffc8014330824c03001818ff000000ffffffff00feffff8888000088888888888888888888888888888888888888888888888888888888ffff8088ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff888810a0fffff0031001f003f0033000f003f003f003f0033003300050000600f1033f0009003f001f003f00120000d0000400fc00240074003c80fc00fc800b8016801f0018800b801f801f0019c01f001ec00f800a4008c00fc003c00f400440054003c00f88888888af888888f080f02081020b200078a422600700006900080203401380037e004a06c00f8001f80128007800f8003f003f003f003f003f003f003f003f003f003f000f003f2c01e007fc00fc00380f4003f801f8014d01f80107c0000006e007e000fc0024871f80d21f801f8003100780007e001e580ec0181a006300004c0220804900fc5602d0120440480080fc01a85e02500909403f00009d01580000a01d4474824c030330300fa6fdff252167c44ecc727eaddaac21966da2c49b3f7255ba7bb532f24dd68a78292bd1607699c6059ffdffb0d298afb847f7c1a2e0083d51508ec1a19d17a3e6256401444297001055722c386d6ea12c774969e0d759504ca4
8060000300000200000000ffa002195b293f6153e794fca878fda6fdc755cf9837c16d7fb34a88a92835fe7204e066d9308f190b5a4a27f41d7d9526f4c63f7d7c8b3d6ef28fdaa61e928816be4cfbc8dc69aadf9d6580a09d4c32c872ef3141aa9854d8cf7ccd75bb57a6cd3a55aafdf48f923b9688a22fd020592a54a25846255e789271a67a6ce5c49934cee89021096dc24d50843ca24fbd238d858d93b4de8c005194d0c35e43df68a1a4563deaf6a3265331bda8ed4d48536a72d1f2d75316b2a6e5c03d7b530fd533dad6055743a02a964ea0f648b9f2797bb6d2686a41562c4506e1724b2619b5985c405b6d27667b72ca0a1127d6a7cd6521e55cfbb944c5ddb5c706f31ea1ec1edf852cc459be0b3550b67e6d6985b193970f5c4b8ececd9be4e8acca65f2564d9251744b53357346584695c8b9ab655f6766c50c459aeae5cb7609c91befe59a2fa75d4ff225a2b31ef6a45fbac569882dd2ccd68ed059c4aca13ed5ba58637feda2c4bdc24ca909f78d56e28b271013aecb934cf620bc5b8d48946f52cd24c75022ada4e6248ea4a86a1012db568f92136a421abe888d4f1ea40959840c6a1098ddab2fc99e2955c83b2ff566a71194b877641df7a1d7bc84b2a985a71e55bb9c137be414b90816275b56966b4b3d483836998709641791dce3fa2967f2c49ddd5dad57668990aa904d3dce2b868edad42bb28c46abbdda91e6a0bab7e8710970f1f8b008b3cf7fd6b89142d01fb1d10000009e

The actual LHDC frames are:

4330824c03001818ff000000ffffffff00feffff8888000088888888888888888888888888888888888888888888888888888888ffff8088ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff8888ffffffff8888ffff88888888888888888888888888888888888888888888888888888888ffff888810a0fffff0031001f003f0033000f003f003f003f0033003300050000600f1033f0009003f001f003f00120000d0000400fc00240074003c80fc00fc800b8016801f0018800b801f801f0019c01f001ec00f800a4008c00fc003c00f400440054003c00f88888888af888888f080f02081020b200078a422600700006900080203401380037e004a06c00f8001f80128007800f8003f003f003f003f003f003f003f003f003f003f000f003f2c01e007fc00fc00380f4003f801f8014d01f80107c0000006e007e000fc0024871f80d21f801f8003100780007e001e580ec0181a006300004c0220804900fc5602d0120440480080fc01a85e02500909403f00009d01580000a01d
4474824c030330300fa6fdff252167c44ecc727eaddaac21966da2c49b3f7255ba7bb532f24dd68a78292bd1607699c6059ffdffb0d298afb847f7c1a2e0083d51508ec1a19d17a3e6256401444297001055722c386d6ea12c774969e0d759504ca4195b293f6153e794fca878fda6fdc755cf9837c16d7fb34a88a92835fe7204e066d9308f190b5a4a27f41d7d9526f4c63f7d7c8b3d6ef28fdaa61e928816be4cfbc8dc69aadf9d6580a09d4c32c872ef3141aa9854d8cf7ccd75bb57a6cd3a55aafdf48f923b9688a22fd020592a54a25846255e789271a67a6ce5c49934cee89021096dc24d50843ca24fbd238d858d93b4de8c005194d0c35e43df68a1a4563deaf6a3265331bda8ed4d48536a72d1f2d75316b2a6e5c03d7b530fd533dad6055743a02a964ea0f648b9f2797bb6d2686a41562c4506e1724b2619b5985c405b6d27667b72ca0a1127d6a7cd6521e55cfbb944c5ddb5c706f31ea1ec1edf852cc459be0b3550b67e6d6985b193970f5c4b8ececd9be4e8acca65f2564d9251744b53357346584695c8b9ab655f6766c50c459aeae5cb7609c91befe59a2fa75d4ff225a2b31ef6a45fbac569882dd2ccd68ed059c4aca13ed5ba58637feda2c4bdc24ca909f78d56e28b271013aecb934cf620bc5b8d48946f52cd24c75022ada4e6248ea4a86a1012db568f92136a421abe888d4f1ea40959840c6a1098ddab2fc99e2955c83b2ff566a71194b877641df7a1d7bc84b2a985a71e55bb9c137be414b90816275b56966b4b3d483836998709641791dce3fa2967f2c49ddd5dad57668990aa904d3dce2b868edad42bb28c46abbdda91e6a0bab7e8710970f1f8b008b3cf7fd6b89142d01fb1d10000009e

So first packet has one frame and a part of the second one and second one contains the rest.
Note that 0x4c is a sync byte and always should be at position 3.
First frame header decodes to

sync 0x4c, down_step 1, size 560, side 0
btr 3, btr_ext 0
btr 3, btr_ext 0
compression type {diff3, diff2}, div {8, 8}
channel 0
residual type rice
channel 1
residual type rice
Total hdr size 8

Second is

sync 0x4c, down_step 1, size 628, side 0
btr 6, btr_ext 0
btr 6, btr_ext 0
compression type {diff3, diff3}, div {64, 64}
channel 0
residual type rice
channel 1
residual type rice
Total hdr size 8

Note that total size is 560+628=1188 which is actually the combined length of payloads for the first 2 packets.

Also, the PCM format for lhdcBT_encode does not seem to be interleaved stereo s32le, but only a single channel.

It isn't s32le. Depending on the bit depth parameter, it will be either s24le (packed/3 bytes) or s16le interleaved stereo.

So, maybe the payload should be fragmented here on the RTP level?

Actually, now that I'm looking at the AOSP patches for V2 support, it seems that there indeed is some sort of fragmentation.

However, I don't think my liblhdcBT_dec supports A2DP_LHDC_HDR_*_MSK. In the end, it was never intended for v2, neither was the original library.

[arkq]

However, I don't think my liblhdcBT_dec supports A2DP_LHDC_HDR_*_MSK. In the end, it was never intended for v2, neither was the original library.

This mask is a copy of rtp_media_header for fragmentation (big-endian ordering for readability):

struct rtp_lhdc_media_header {
	uint8_t fragmented:1;
	uint8_t first_fragment:1;
	uint8_t last_fragment:1;
	uint8_t frame_count:3;
	uint8_t latency:2;
	uint8_t seq_number;
}

I was able to add defragmentation logic into the decoder, unfortunately, the decoded audio is still far from original :D So, I have another question, is the lhdcBT_dec_decode able to decode more than one frame in the v2 mode? Currently I'm feeding it with defragmented v2 frames with fragmented, first_fragment and last_fragment cleared ( lhdcBT_dec_decode does not need to support that) , but a single bitstream contains 0, 1 or 2 frames. Actually, in case of 0 frames library spits (the frame len at the frame_len(p) offest is also 0 in such case):

ERROR: cirbuf_get_no_copy: failed! len=12, s_len=0 

And in case of 2 frames I can see:

ERROR: lhdc_dec_decode: alignment bits are not zero
ERROR: [LHDC]lhdcDecodeProcess: Type LHDC : Decode error (-3) 

But the lhdcBT_dec_decode returns 0 with some decoded PCM.

I'll try to feed frame by frame (maybe later today) and see what will happen :)

[anonymix007]

So, I have another question, is the lhdcBT_dec_decode able to decode more than one frame in the v2 mode?

I think it should be. You may try by converting packets to the format that my lhdcdec.c tool understands, that is, each packet is prepended by

struct hdr {
    uint16_t len;
    uint16_t frames;
};

alignment bits are not zero

This is bad, correct frames should not trigger this.

But the lhdcBT_dec_decode returns 0 with some decoded PCM.

How much samples does it decode? Maybe the issue is with the second frame.

[arkq]

Splitting LHDC packets with 2 frames into two packets yields the same result. But maybe I'm doing something wrong (something is not 100% right). Steps are as follows:

1. Defragmenting packet (without RTP header) [len=1022]:

c80b4500824c0202383846c1ffff9e24998c743bb52c29f72ef6d0cdb117b0b849d0a9d8b92b482c29a2bf01004092327a3199422f3ad4621c5943ac403838083a848483206539a8f01f9a3883b272d6d3919a4c845036cd6c24b1ad5ffd6f2462332aefe12144babe5f4ba2d98c75518960458ea8a8c25a0b72612a5363cf0a7f4cd69ce608f1c4d2a580de6e1689c868b3babe5aa462274854a2771639848c54c714338d1adad311cb3765f6b9523405e11fe3a01d4b936d27b32a3a6cb4b9b19ab3220bbd2a72eca47dbb98227d4db36e92c22b3d7ba31121c9610f4bd984a4f65c2ab7aef1806cd4c5ac308a8b1deb195842313f8b15ad1842529163c433477c52480b39ba8c2344cd343470bf7eafe2dd6a6444725ebd681633193309d9c53a65ac9548cfd09b27a4428e2d2695b8e4654516f77822624212ee79a5e4f24db246c7bc76fbfcb73811d1e4f99d44231b5cb5623d76b9cabbd59099add7fb5311167324f01c196b656e7d5a2181e0211405842b42c9215a0c5392596bc58e4e3b5b5ae4b099d8a86e2e7996b6a5f2fbb6e2e674b9e266356c39cf6e2b00deb56074b3123714b7acb87252084b0aa15ba164a5fa27de87f4a6a64254fd28080b2952d35940ff59df7641e27b7a556f1a130af5d07934c6866df37569126854a07d54eca39244998af5ec2747b44c3117528d4ece56152b5ac95b49e51e000000e444f0814c02023838cdf9feffdcba4493aeb2f8b636d944b2159d3ed138ed90aa3962268ac20aec8a31ffff8095586a09d7ffd63532958aafbb39991a0aa2226ec8a6ac654e1d692456e08e722235ba3917716292857bec101c51a42395a6089abec72b1eedf26ecbd05e40125da33142efda25ba7f54f8e2b2955a9e4a95164d39b1585e001b1ab337977dd7dee5534dc9360000000000000000000000009a4a8ada9675b4d3e5923732416249f077d9d9bbeea24e35e95223996d939c7aba699aa4329f234142b4b580d735683add3c4b4fdbcc22b6efdda9f489b5936abae4d4d8fee5d7eb0c8d08e3f6af941b42309c43c151328c99b6b2244a181ab6f5aafe428e900937173a15a1ad807bddebadcb1472a1c9daa513354776a24c44e4a46fa1d64e3100f279b06c25905e5451a5b3938eaeb176ada54653f953199999d8dbdfa6914d6246f06ddd0fdf646484ccd71697fa9a1213d59254b254f4b1fcfe3389b8842b481529d04c1155c7b3db4a46f75a5efad33464c7ef74b3d1385e51d74565846a77e9a29b48379a2953321273a4232d724c9d0a7246655ec0894a6a173af7e6f87d507167948c42b151c96db322a2a1c4dab9daada824bad9506272f9b6f8e24b22225292f65def511192ee13925e17d06b8a90001e9eb70f7f74641862671d94c324c384f661a1d0cccccc098d6e969f349988007857f5

2. Searching for sync byte 0x4c starting after the LHDC media header (first 2 bytes)
3. Byte found at offset 3: c80b + 4500824c
4. Getting frame size (2 bytes after sync): (0x0F & "02") << 8 + "02" == 514
5. It seems that 514 is a frame length including LHDC media header (first 2 bytes)
6. Decoding frame (with cleared fragmentation bits) => no errors, produced 1024 int32_t samples (4k bytes)

040b4500824c0202383846c1ffff9e24998c743bb52c29f72ef6d0cdb117b0b849d0a9d8b92b482c29a2bf01004092327a3199422f3ad4621c5943ac403838083a848483206539a8f01f9a3883b272d6d3919a4c845036cd6c24b1ad5ffd6f2462332aefe12144babe5f4ba2d98c75518960458ea8a8c25a0b72612a5363cf0a7f4cd69ce608f1c4d2a580de6e1689c868b3babe5aa462274854a2771639848c54c714338d1adad311cb3765f6b9523405e11fe3a01d4b936d27b32a3a6cb4b9b19ab3220bbd2a72eca47dbb98227d4db36e92c22b3d7ba31121c9610f4bd984a4f65c2ab7aef1806cd4c5ac308a8b1deb195842313f8b15ad1842529163c433477c52480b39ba8c2344cd343470bf7eafe2dd6a6444725ebd681633193309d9c53a65ac9548cfd09b27a4428e2d2695b8e4654516f77822624212ee79a5e4f24db246c7bc76fbfcb73811d1e4f99d44231b5cb5623d76b9cabbd59099add7fb5311167324f01c196b656e7d5a2181e0211405842b42c9215a0c5392596bc58e4e3b5b5ae4b099d8a86e2e7996b6a5f2fbb6e2e674b9e266356c39cf6e2b00deb56074b3123714b7acb87252084b0aa15ba164a5fa27de87f4a6a64254fd28080b2952d35940ff59df7641e27b7a556f1a130af5d07934c6866df37569126854a07d54eca39244998af5ec2747b44c3117528d4ece56152b5ac95b49e51e000000e4

7. The rest of the payload looks like (ends with: ...007857f5)

44f0814c02023838cdf9feffdcba4493aeb2f8b636d944b2159d3ed138ed90aa3962268ac20aec8a31ffff8095586a09d7ffd63532958aafbb39991a0aa2226ec8a6ac654e1d692456e08e722235ba3917716292857bec101c51a42395a6089abec72b1eedf26ecbd05e40125da33142efda25ba7f54f8e2b2955a9e4a95164d39b1585e001b1ab337977dd7dee5534dc9360000000000000000000000009a4a8ada9675b4d3e5923732416249f077d9d9bbeea24e35e95223996d939c7aba699aa4329f234142b4b580d735683add3c4b4fdbcc22b6efdda9f489b5936abae4d4d8fee5d7eb0c8d08e3f6af941b42309c43c151328c99b6b2244a181ab6f5aafe428e900937173a15a1ad807bddebadcb1472a1c9daa513354776a24c44e4a46fa1d64e3100f279b06c25905e5451a5b3938eaeb176ada54653f953199999d8dbdfa6914d6246f06ddd0fdf646484ccd71697fa9a1213d59254b254f4b1fcfe3389b8842b481529d04c1155c7b3db4a46f75a5efad33464c7ef74b3d1385e51d74565846a77e9a29b48379a2953321273a4232d724c9d0a7246655ec0894a6a173af7e6f87d507167948c42b151c96db322a2a1c4dab9daada824bad9506272f9b6f8e24b22225292f65def511192ee13925e17d06b8a90001e9eb70f7f74641862671d94c324c384f661a1d0cccccc098d6e969f349988007857f5

8. Searching for sync byte => found at offset 3 (just like with the 1st frame, but this time we do not have media header to skip)
9. Prepending media header with next sequence number:

040c + 44f0814...

10. Getting frame size (2 bytes after sync): (0x0F & "02") << 8 + "02" == 514 (just like before)
11. Now the funny part:
    If calculating frame size just like before (with media header), 514 bytes is bigger than the rest of the payload, 4 bytes are missing... so I've appended the same bytes like in the 1st frame (so the length will match): 000000e4
12. Decoding... and error:

ERROR: lhdc_dec_decode: alignment bits are not zero
ERROR: [LHDC]lhdcDecodeProcess: Type LHDC : Decode error (-3)

13. The same but with appending zeros 00000000, the same result, alignment error

So, I guest that the same algorithm is in your decoding library, so it doesn't matter whether I pass 2 frames in a single packet, or 2 packets with a single frame.

But there is one disclaimer, I'm not sure whether the Huawei implementation is correct :D I do not have a retail sink device which could consume LHDC v2 from my phone.... There was a time when Android's AAC implementation was incorrect....