Parallel tracker evaluation

[davidyang180]

Hello! We participated in the VOT challenge for the first time, and initially tried the testing process of the toolkit. It seems that we can only test in a single thread; are we only able to evaluate through the official VOT toolkit?Can we do multi-threaded testing on the basis of VOT toolkit?

[lukacu]

At the moment we only have single-threaded testing as multi-threaded tests can impact the overall performance in uncertain ways. Still, I know that this would be desired and it can be added (I will see how much time will I have in the upcoming weeks).

[lukacu]

Well, I am not sure if you can complete all the experiments this way, pytracking does not have realtime challenge implemented. All experiments must be complete to submit to the challenge. But there is plenty of time left so you can easily test with pytracking and then finally with vot-toolkit.

…

On Thu, Mar 31, 2022 at 12:17 PM davidyang ***@***.***> wrote: At the moment we only have single-threaded testing as multi-threaded tests can impact the overall performance in uncertain ways. Still, I know that this would be desired and it can be added (I will see how much time will I have in the upcoming weeks). Hello! Recently, I tested a multi-threaded test program for the VOT2022-STB public data set locally through the pytracking toolkit, but after evaluating the test results through the VOT toolkit, I found that there is a certain error compared with the results running in the VOT toolkit. After comparing the two result output files, except for a slight difference in decimal places of 3-6 orders of magnitude, the output results are basically the same, but the EAO of the evaluation results differs by 0.5-1 percentage points; What is the reason for this, I don't think the difference after 3 decimal places will cause such a big difference in the evaluation results. — Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVY6MHWDBL4CZNFBABIALVCV3SZANCNFSM5RZTK7KA> . You are receiving this because you commented.Message ID: ***@***.***>

-- Luka Čehovin Zajc http://luka.tnode.com