Retry attempts are done earlier than the defined delay

[manumoreira]

General Description

The delays defined in the survey settings for the re-contact attempts are not working as expected.
We were used to see delays on those attempts and that was caused by the inability of Surveda to handle contact attempts according to channel capacity. i.e. The delay was 26h, Surveda sent the attempt to the channel, but it was so busy that the attampt was done 4h later getting a 30h delay.
Since the Channel Broker (ChB) the delays are not happening that often, but we noticed another issue: Some attempts are done earlier than expected.

Why is this happening?

The strongest hypothesis is that the Survey Broker (SB) is considering the Enqueuing call time (for IVR of course) to calculate the delay.
When the SB was designed the difference between the Enqueueing of a contact attempt and the attempts itself was trivial (in theory), but after years of interacting with real channels and after having a ChB that handles priorities (prioritizing re-contacts over first contacts) the difference got bigger and evident.

A test case

In this survey we have defined a re-contact delay of 4h

The call was queued at 17:45, but it waited in the ChB queue until 19:30 when it finally went out.
At 21:45 4h after the call was queued it gets the new attempt

What should we do?

We might want to change the behavior of the SB and start counting for the re-contact delay when the attempt is actually done. There are (at least) two options for this

1. when the ChB actually sends the call to the channel
2. when the channel sends the first callback related to that attempt

Probably the choice will be clear after checking to existing code.

Steps to reproduce

1. Create a IVR Survey
2. Add (at least) 1 re-contact with a delay > n hours and a batch-size < channel capacity by hour x n
3. Wait until 1 re-contact is completed for the first batch to download the interactions file and calculate the delay

Calculations were done using this notebook https://colab.research.google.com/drive/14pkYblNRNvevkBl42uvdmIIX4VZMhGrf?usp=sharing

[matiasgarciaisaia]

This finding comes from analyzing the interactions files, and we have confirmed via Verboice that the calls were effectively done before they should - their not_before time was miscalculated.

Analyzing interactions files from old surveys (with versions of Surveda that didn't have the Channel Broker) @manumoreira couldn't find instances of this behaviour outside the staging environment. Our most probable cause for this absense of the issue right now is that the channels would have been so busy then (without channel broker) that even if Surveda queued a contact earlier than it should have, the contact would get queued in the channel long enough to be actually attempted after the correct time. Nonetheless, we should also check if there was any other behaviour in Surveda that "recalculated" this not_before time, thus preventing the bug in old versions (ie, something like when we "refreshed" expired calls, overwriting some of the dates we store for each respondent).