Goes into an infinite loop if the script takes too long

[Wanchai]

Since merging PR #24, I tried to add a folder containing ~700 files and the proccess started all over after file 141:
22.08.2019, 00:24:44: Script called for BitBucket JSON payload
22.08.2019, 00:24:44: Received a push from {###}
22.08.2019, 00:24:44: Modifications detected on the branch {master}
22.08.2019, 00:24:44: Commencing transfer for change 1
22.08.2019, 00:24:46: 703 files changed.
22.08.2019, 00:24:46: Created new directory fol2 copy
22.08.2019, 00:24:47: Uploaded: ../bb/fol2 copy/001.txt
22.08.2019, 00:24:47: Uploaded: ../bb/fol2 copy/002.txt
22.08.2019, 00:24:48: Uploaded: ../bb/fol2 copy/003.txt
22.08.2019, 00:24:48: Uploaded: ../bb/fol2 copy/004.txt
[...]
22.08.2019, 00:25:43: Uploaded: ../bb/fol2 copy/140.txt
22.08.2019, 00:25:44: Uploaded: ../bb/fol2 copy/141.txt
22.08.2019, 00:25:44: Script called for BitBucket JSON payload
22.08.2019, 00:25:44: Received a push from {###}
22.08.2019, 00:25:44: Modifications detected on the branch {master}
22.08.2019, 00:25:44: Commencing transfer for change 1
22.08.2019, 00:25:44: Uploaded: ../bb/fol2 copy/142.txt
22.08.2019, 00:25:45: Uploaded: ../bb/fol2 copy/143.txt
22.08.2019, 00:25:45: Uploaded: ../bb/fol2 copy/144.txt
22.08.2019, 00:25:45: Uploaded: ../bb/fol2 copy/145.txt
22.08.2019, 00:25:46: Uploaded: ../bb/fol2 copy/146.txt
22.08.2019, 00:25:46: 703 files changed.
22.08.2019, 00:25:46: Uploaded: ../bb/fol2 copy/147.txt
22.08.2019, 00:25:46: Uploaded: ../bb/fol2 copy/001.txt
22.08.2019, 00:25:47: Uploaded: ../bb/fol2 copy/148.txt
22.08.2019, 00:25:47: Uploaded: ../bb/fol2 copy/002.txt

[d2roth]

Did you fix this? It looks like two pushes are running at the same time. I know working with this script we always needed to wait before doing a second push because if we didn't we would have unpredictable results.

[d2roth]

@Wanchai so I think this is a case of handle_rate_limit coming into play. Basically when a 429 HTTP status code is hit the rate limiter stops execution for a bit so BitBucket isn't hammered.

Did both of your pushes affect the same files?

I just did a test with about 4k files and this is what the results were:
lgofile.txt

05.09.2019, 13:10:20: Script called for BitBucket JSON payload
05.09.2019, 13:10:20: Received a push from {...}
05.09.2019, 13:10:20: Modifications detected on the branch {master}
05.09.2019, 13:10:20: Commencing transfer for change 1
05.09.2019, 13:10:57: 3996 files changed.
05.09.2019, 13:10:57: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-1-1.txt
05.09.2019, 13:10:57: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-1-10.txt
05.09.2019, 13:10:58: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-1-100.txt
...
05.09.2019, 14:00:53: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-846.txt
05.09.2019, 14:00:53: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-847.txt
05.09.2019, 14:00:53: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-848.txt
05.09.2019, 14:00:53: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-849.txt
05.09.2019, 14:04:55: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-85.txt
05.09.2019, 14:04:55: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-850.txt
05.09.2019, 14:04:56: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-851.txt
05.09.2019, 14:04:56: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-852.txt
05.09.2019, 14:04:56: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-853.txt
05.09.2019, 14:04:56: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-854.txt
05.09.2019, 14:04:57: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-855.txt
05.09.2019, 14:04:58: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-856.txt
05.09.2019, 14:04:59: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-2-857.txt
...
05.09.2019, 16:11:48: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-4-996.txt
05.09.2019, 16:11:48: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-4-997.txt
05.09.2019, 16:11:48: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-4-998.txt
05.09.2019, 16:11:49: Uploaded: /public_html/bitbucket-v2-ftpbucket-test/test-4-999.txt
05.09.2019, 16:11:49: Transfer done for branch {master}
05.09.2019, 16:11:49: Push completed. There were {4052} total CURL calls made.

logconnection.txt

A PUSH was started at 05.09.2019, 13:10:20
The push finally finished at 05.09.2019, 16:11:49.
In total it took 10889 seconds.

So basically I think what was happening for you might have been that you started the push and then it was still running because there is no end "Push completed" line in your sample.

Does that match your findings?