Duplicate call detection and suppression regressed in latest nightly versus 0.6.1-beta

[thisisfakediy]

sdrtrunk Version
'master-branch'

Describe the bug
Duplicate call detection and suppression is not working normally in the latest nightly.

To Reproduce
Steps to reproduce the behavior:

1. Start SDRtrunk and start multiple control channels on a trunked system.
2. Enable recording of talkgroups.
3. Wait for a call that is simulcast across two or more talkgroups.

Expected behavior
In prior stable release versions, duplicate call detection would a) suppress simulcast calls, even across different talkgroups and even if there was a delay of several seconds in the audio between talkgroups, and b) suppress recording of all simulcast talkgroups, only recording one copy.

Application Log
sdrtrunk_app.log

Desktop (optional - complete the following information):

• OS: Windows 10
• CPU Cores: 8 cores (Intel i7-9700)
• RAM: 16 GB

Additional context
I have one radio (RSPdxR2) monitoring three sites of a P25 700 MHz trunked system. Local EMS and fire are co-dispatched and their calls go out over 2 or 3 talkgroups at a time. In 0.6.1-beta the call and recording suppression would result in just one call playing out; in this nightly it's playing the same call in both left and right channels and also recording both calls despite them being duplicates. Not every simulcast call is doing this, just ones where there seems to be a delay or glitch in one broadcast. When this happens, the recorded files are not showing From/Alias data in the mp3 files, either. I'd go back to the beta version but it apparently does not support the right API for this newer SDRPlay model radio.

[DSheirer]

Where you said that the duplicate calls that made it through, undetected, didn't have any FROM/ALIAS data in the MP3 files, that information is needed to do the duplicate call detection when the calls are from the same source radio but to different talkgroups in a simulcast setup. If the call decode was poor with lots of errors and didn't capture the FROM identifiers, sdrtrunk can't make the comparisons to detect duplicates.

I'm writing some test cases for this now, but I don't think there was a code regression in the duplicate call detection. The recent changes to the way that P25 calls are managed in the traffic channel manager may be contributing to the call FROM identifiers not being populated correctly.

If this is happening fairly consistently for you, can you please capture some bitstream recordings of the control channel and traffic channels and send them to me. Please indicatet the source radio and the simulcast talkgroups for the call(s) that went undetected, so that I can inspect the demodulated bitstreams to see if something else might be interferring with capturing the FROM identifiers.

[thisisfakediy]

I've enabled bitstream recording of the control and traffic channels on one of the three sites I monitor and will upload them later after there's been activity during the day. Since originally posting this, however, I've noticed that the duplicate calls are almost always now showing a FROM ID instead of being blank, but the duplicate calls are still being played out and recorded. Right now I'd say maybe one call in a hundred is filtered properly, the rest are not being suppressed, so it'll be interesting to see if you can find a cause in the bitstream recordings. I'm listening to the same three sites of one trunked system (non-simulcast) so I don't think something local has changed, just my switching from the beta to the nightly for the RSPdxR2. My local system dispatches fire calls out over both a set of regional channels (north/south/east/west/central) but also each individual fire department has their own channel. If EMS is requested, they get the page on their own channel as well, meaning the same call is playing three times in a row. Usually what happens is the same call will play with a slight delay from the left/right channels and then when they're done, the third will play in the left channel by itself, ha ha. It can get quite annoying with all the page-out tones they play! In a few scenarios the page tones are different, but most of the time they're the same on all talkgroups, and the files are the same size and written to disk at the same time. Sent with Shortwave <https://www.shortwave.com?utm_medium=email&utm_content=signature&utm_source=c29yZGlkYW5kc3VibGltZUBnbWFpbC5jb20=>

…

On Wed Sep 4, 2024, 08:03 AM GMT, Denny Sheirer ***@***.***> wrote: Where you said that the duplicate calls that made it through, undetected, didn't have any FROM/ALIAS data in the MP3 files, that information is needed to do the duplicate call detection when the calls are from the same source radio but to different talkgroups in a simulcast setup. If the call decode was poor with lots of errors and didn't capture the FROM identifiers, sdrtrunk can't make the comparisons to detect duplicates. I'm writing some test cases for this now, but I don't think there was a code regression. If this is happening fairly consistently for you, can you please capture some bitstream recordings of the control channel and traffic channels and send them to me. Please indicatet the source radio and the simulcast talkgroups for the call(s) that went undetected, so that I can inspect the demodulated bitstreams to see if something else might be interferring with capturing the FROM identifiers. — Reply to this email directly, view it on GitHub <#1939 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2TSTLPIY5LCEBOGJTPWPJDZU25GJAVCNFSM6AAAAABJLJ3D6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRYGE4DKMBSGA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[thisisfakediy]

Robertsdale-AIRS.zip

Here's some of the .bits from around the time that a duplicate call was recorded. If you need more than this, let me know. I've never done bitstream recording before and can't quite tell what matches what with regard to the mp3 audio that's recorded.

[DSheirer]

I've never done bitstream recording before and can't quite tell what matches what with regard to the mp3 audio that's recorded.

It can be challenging to capture. Usually I just turn on bitstream recording for both the control and traffic channels and then watch/listen for the simulcast call event to happen and the call to end and then shut down the application and grab the (.bits) files from the recording directory. There can be a lot of recording files generated depending on how long you have to wait for the simulcast event. You can periodically stop the software, delete the recordings and start again, to limit the total quantity of recording files that I'll have to scan through, to find the event.

If you're able to shutdown sdrtrunk right after the simulcast call happens, then the recording would naturally be one of the files with the latest timestamp.

[DSheirer]

I made some slight changes to the duplicate call detection, but there wasn't any shortfall or regression in the code that had to be changed. Can you test out the nightly release (in a few minutes, once it's built) and let me know if you're still seeing any problems. We can reopen this issue if the problem persists.

I added a number of automated unit tests to ensure the duplicate call detector is working as designed.

[thisisfakediy]

Thanks, I have the latest nightly installed and will be listening for duplicate calls. With the way my local system is operating, a fire/EMS call will go out on two or three talkgroups simultaneously, so it shouldn't be long before I can tell if anything has changed.

…

On Sun Sep 15, 2024, 12:42 PM GMT, Denny Sheirer ***@***.***> wrote: I made some slight changes to the duplicate call detection, but there wasn't any shortfall or regression in the code that had to be changed. Can you test out the nightly release (in a few minutes, once it's built) and let me know if you're still seeing any problems. We can reopen this issue if the problem persists. I added a number of automated unit tests to ensure the duplicate call detector is working as designed. — Reply to this email directly, view it on GitHub <#1939 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2TSTLIQC4YDKZPF6PAZQPLZWV6EDAVCNFSM6AAAAABJLJ3D6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJRGU3TMMZZHA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[thisisfakediy]

After doing some extensive listening today, I have discovered something that I did not notice prior to filing the bug report, and I think it's safe to say it's something the duplicate call detection algorithm won't be able to handle.

It seems that the way my local agencies are paging out calls has changed slightly; page-out tones previously played out across multiple talkgroups, followed by a dropped carrier, then after a second or two, they'd key back up for the actual voice page. The voice page is the same across talkgroups and had been previously suppressed up until a few weeks before I filed the bug report.

Now, they're apparently playing out the tones and keeping the carrier active, still with a few seconds pause before the voice activity starts. The tones are slightly different (usually) across different talkgroups, meaning that each call is technically unique from a granular viewpoint even though the voice part, length and file sizes were all the same.

So, what I'm concluding is that the initial page tones being different on different talkgroups is enough to throw off the call detection. I was able to verify that the duplicate detector does work on other transmissions where the data is the same, like simulcast police dispatch calls across multiple TX sites. So it has to be this little change to the way the fire/EMS calls are being paged out that caused this issue.

It was something I didn't really notice since I usually tune out those tones. But when listening carefully with headphones it became obvious that the simultaneous calls are different at the start, even if everything else is exactly the same.

Some calls that weren't being detected ARE being suppressed now, though, and it seems that one agency in my area just has the same tones for fire as EMS, so they are still being properly suppressed. So whatever changes you made did make a difference in that particular case.

[DSheirer]

The logic for detecting duplicate calls does not look at the audio content. It looks at either:

1. The system is the same and the TO talkgroup for each call is the same, indicating that it's the same call being simulcast across two or more sites or,

2. The system is the same and the TO talkgroups are different but the FROM radio is the same, indicating that a single dispatcher is simulcasting the same call to two or more talkgroups.

Both of these steps are controlled by the user preferences settings for detecting duplicate calls by TALKGROUP or by source RADIO.

Even though the two calls you're seeing are having different tone-out preambles, as long as the source radio ID is the same for both calls, it should be flagging one of those calls as duplicate.

[thisisfakediy]

Okay, that's really strange, then. Watching calls live, I can see the radio ID is the same in the FROM field while the talkgroups are different, so it should work. It does work if the radio AND talkgroups are the same but across different sites, as happens with some single-agency talkgroups I monitor.

[DSheirer]

To confirm, do you have duplicate call detection turned on for both talkgroup and radio, in the user preferences?

[thisisfakediy]

No, Radio ID is unselected. That's strange, I don't remember turning it off. It's on by default, isn't it? I'll turn it on and see if that makes a difference.

[DSheirer]

@thisisfakediy Any updates with the radio ID enabled?

[thisisfakediy]

Yes, it seems the majority of the calls are now being suppressed again, unless the radio happens to be missing.