strange audio distortion when using filter envelope after merging #1345

[mrbumpy409]

FluidSynth version

2.4 (and master)

Describe the bug

Since #1345 was merged, there is now a strange audio distortion when the filter is modulated by the modulation envelope. I created a demo MIDI file and audio renderings that should hopefully make this quite evident: filter envelope noise.zip

Inside the zip file, you will find the following files:

• filter envelope noise.mid – A MIDI file that uses the GS preset 008:038 Acid Bass featured in GeneralUser GS 2.0.1. This preset uses high filter emphasis and has both the modulation envelope and modulation LFO modulating the filter.
• filter envelope noise-FluidSynth 2.4.flac – The above MIDI file rendered using the latest FluidSynth master (one commit ahead of 2.4) via FluidSynth Plugin in REAPER. You can hear a strange, grainy distortion throughout correlating to the filter being modulated by the modulation envelope.
• filter envelope noise-1345 reverted.flac – Same as above but with Remove interpolation of IIR filter coefficients #1345 reverted. You can hear no more distortion. Everything sounds as it should.
• filter envelope noise-Roland SC8820.flac – The above MIDI file rendered on Roland Sound Canvas VA (SC8820 model, reverb disabled). Filter popping with high Q and quick modulation is a known issue with IIR filters. I included this recording to show that FluidSynth handles filter popping much better than the Roland Sound Canvas, even without Remove interpolation of IIR filter coefficients #1345.

It is unfortunate that such a substantial change was made to FluidSynth's behavior so soon before a major new release. From my perspective, FluidSynth's filter behavior was far better than any other SoundFont synth I had tested, and now it's broken. I'm really upset that I didn't have the time to test this until now, otherwise I would have been able to say something sooner.

[ReinholdH]

I can confirm the distortion in the sample and agree to all your points. More to #1415.

[derselbst]

It is unfortunate that such a substantial change was made to FluidSynth's behavior so soon before a major new release

Let me clarify a few things:

• 2.4.0 is a minor release, technically, since it's a minor version bump.
• Remove interpolation of IIR filter coefficients #1345 stayed open for a month, which I usually consider long enough, and it was already merged 4 months ago, which I don't consider as "soon".
• I was convinced - given the linked example rendering - that Remove interpolation of IIR filter coefficients #1345 would make things better.

Yet, I appreciate your input Chris and will look into that when I find the time.

[klerg]

I must admit the Roland Sound Canvas sample has the most clicks and cracks. And the distortion is more pronounced at around 7 seconds or so in FluidSynth 2.4 sample. My hope is that it can be fixed or at least reduced

[derselbst]

I started investigating this. The good thing is, that we now got several different use-cases and test data, thanks for that.

I incorporated the test data into the buildsystem's unit tests. The data is available in the following directory, on the iir_tests branch currently: https://github.com/FluidSynth/fluidsynth/tree/iir-tests/test/manual/iir_filter

The tests can be run during build by executing make check_manual. This will render the tunes into build/test/manual/iir_filter/, so one can at least manually verify by listening whether everything works as expected when changing the filter implementation.

Additionally, in order to get a better understanding of the filter's behavior, I created an interactive Bode plot by using Matlab. The script is also available in this directory.

It let's you adjust fc and Q with sliders. One can observe that the filter's phase is quite sensitive whenever any of those two parameters change.

The clicks heard in all three issues (#1415, #1424, and this one) are much more subtle and decent, compared to the artifacts I attempted to fix by #1345. I believe the distortions here (and clicks in the other two issues) are the result of the filter's phase changing rapidly, i.e. causing the frequency-dependent delay inside the filter to change. Therefore I agree with Chris' comment that the fc (and ideally Q as well) need to be smoothed out.

In the next days I'll attempt to implement this smoothing by using a linear interpolation of the fc. We'll see how this works.

Currently, I still believe that #1345 itself was correct. I don't see how the linear interpolation of the filter coefficients itself can preserve any state of the filter at all - the coefficients do not change in a linear way, but rather in a sinusoidal way. So yes, while there are cases where this linear interpolation of the coefficients apparently yields the desired effect (as heard in the example files posted by Chris here), it can also break stuff in a much more louder way as heard in #1345.

[spessasus]

Looks like:

• a feature was requested to add support for NRPN for an old sound card.
• a clicking bug was discovered after adding that feature. It only happened in a MIDI file using the said feature, nowhere else.
• 13-year old code (or potentially, 17-year old code?) for the filter interpolation was removed.
• it fixed that new NRPN bug, while breaking the filter envelope. I.E. Fixed a minor bug and introduced a major one.

Therefore, I think #1345 should be reverted and AWE implementation changed. There's probably a good reason for the coefficient interpolation, given that it stayed for 13 (17?) years without any issues. Reverting and fixing AWE would fix this bug (along with the one i reported) and nervous filter would be fine too.

And then there's no need to interpolate the Fc or Q in ModEnv, which is not good according to Christian Collins:

If NRPN control of filter cutoff is causing pops, you might want to incorporate "smoothing" logic to limit the speed at which a NRPN or modulator can increase/decrease the cutoff, though I would advise against also applying this smoothing logic to modulation from the modulation envelope, as this may mess with the sound of presets.

[derselbst]

Why are you so focused on the new AWE NRPN feature? The NRPNs just modulate the fc in a way, that could be achieved with any custom modulator and CC as well. It was just that particular MIDI file, modulating the fc in that particular way to cause the observed artifacts.

AWE implementation changed. [...] Reverting and fixing AWE would fix this bug

So, what exactly makes you believe that the AWE implementation needs to be changed / reverted / fixed? Can you pls. be a bit more specific?

There's probably a good reason for the coefficient interpolation, given that it stayed for 13 (17?) years without any issues

Preservation of tradition is not a legitimate reason. Pls. argue on a technical level. I have tried to explain that the clicks are related to the filter's changing phase whenever fc and Q are changed. For the filter it doesn't matter through which mechanism these parameters are changed.

And then there's no need to interpolate the Fc or Q in ModEnv

I'm planning to introduce a smoothing, that is agnostic of what has caused the parameter changes. Just like the previous coefficient smoothing also was agnostic of its cause. If all that effort doesn't yield the desired effect, reverting #1345 will obviously be a last resort.

[spessasus]

So, what exactly makes you believe that the AWE implementation needs to be changed / reverted / fixed? Can you pls. be a bit more specific?

Sorry for not being specific, I simply meant to smooth out AWE params instead of messing with the filter, for example by setting a target value when the NRPN is received and changing fc to it slowly enough for no clicks. I'm not against removing this feature (i meant reverting the PR, not AWE!), I just want to keep the filter as it was before, because it just worked and it worked perfectly. Now it does not.

I hope this clears things up for you : )

[derselbst]

I have applied the previous linear smoothing logic to the fc only in f123fa1. With that clicks of #1417 and #1424 can no longer be heard. #1415 still has a potential for clicks, which is likely due to the fact that it also changes the filter's Q, for which I'm currently not accounting at all. But I think I'm on the right track. I'll try to get one or two more tests incorporated esp. with the one observed in #1345. When it's ready, I'll provide the renderings for comparison so you can have a listen as well.

---

I just want to keep the filter as it was before, because it just worked and it worked perfectly.

I'm glad that it worked for you. I've been messing around with the filter for the past seven years using custom modulators, and I can tell you that it wasn't "perfect" and probably will never be. Yet, implementing the smoothing on NRPN-level only is not a meaningful solution.

[mrbumpy409]

I can confirm that the current iir-tests branch eliminates the distortion that was introduced after the merge of #1345.

[klerg]

I'm glad that we decided to keep the AWE32 NRPN support as the issue seems to be with #1345, and hope there will be no need to revert it but I guess will find out soon

[derselbst]

Since we now have a nice test suite in place, I would like to share some renderings, that I have created with different implementations of fluidsynth's filter.

The first test renderings are created from fluidsynth's recent master (4cdd9af) which should be identical to the offical(-ly broken) 2.4.0 release:
TestSuite-2.4.0.zip

On top of that, I have applied the change of #1430 to get a bit more modulations in the fc of the AWE test files - if audible at all.

Next, I'm providing renderings where I have

• reverted Remove interpolation of IIR filter coefficients #1345, i.e. giving 2.4.0 back the previous linear filter coefficient smoothing: TestSuite-2.4.0-with-1345-reverted.zip,
• applied my new filter parameter smoothing as found in Smooth linear filter parameter change #1432, with
  • A maximum of 5*64 samples of smoothing duration: TestSuite-1432_smooth5.zip, and
  • A maximum of 8*64 smoothing duration: TestSuite-1432_smooth8.zip

I cannot hear a difference between TestSuite-1432_smooth5.zip and TestSuite-1432_smooth8.zip, yet TestSuite-1432_smooth5.zip stays my favorite, as the smoothing duration and therefore the delay for the parameter change is reasonably short.

From my perception, there is a "muffled side tone / vibration" present at the beginning of 1417_filter-envelope-noise in TestSuite-2.4.0-with-1345-reverted, which is not present in my smoothing approach.

We can also hear that none of the changes affects the testcase for #1427. So this will be addressed separately. Anyway, in case all of you are happy with the result, I'd make some final performance tweaks to #1432, merge it and then release a fix.

[spessasus]

We can also hear that none of the changes affects the testcase for #1427. So this will be addressed separately.

As I have stated before, this has nothing to do with the filter itself (or the coefficients). It's just that its output is not processed by the Volume envelope when it should be.

[derselbst]

We can also hear that none of the changes affects the testcase for #1427. So this will be addressed separately.

As I have stated before, this has nothing to do with the filter itself (or the coefficients). It's just that its output is not processed by the Volume envelope when it should be.

Yes, understood. I wanted to express that the tests prove this to be true.

[klerg]

Well, I definitely hear clicks and pops in The Nervous Filter.mid with 1345 reverted so we can toss that out completely. And it is good that you applied #1430 but the effect is not very pronounced and only minor, at best. Based on what my ears hear, I agree that 1432-smooth5 has the best overall sound. 1432-smooth 8 to me, has a lower volume to my ears, but I have no clue why

Also, I noticed in Uplift.mid at around 2:10 or so it plays the wrong instrument, on my Live! (and Audigy2?) it should be a synth, but in Fluidsynth it is strings, is Fluid loading bank 1 properly, or any idea what is going on here ?