possible mistake in apply_camera_motion

[arnaud-nt2i]

Hi @viplix3

Your code in track.cpp/apply_camera_motion is different than the official
Botsort https://github.com/NirAharon/BoT-SORT/blob/main/tracker/bot_sort.py#L68
and yolo_tracking repo https://github.com/mikel-brostrom/yolo_tracking/blob/master/boxmot/trackers/botsort/bot_sort.py#L96

your code:
Eigen::Matrix<float, 8, 8> R8x8 = Eigen::Matrix<float, 8, 8>::Identity();
R8x8.block(0, 0, 2, 2) = R;
mean = R8x8 * mean.transpose();
mean.head(2) += t;
covariance = R8x8 * covariance * R8x8.transpose();

The other:
R = H[:2, :2]
R8x8 = np.kron(np.eye(4, dtype=float), R)
t = H[:2, 2]
for i, (mean, cov) in enumerate(zip(multi_mean, multi_covariance)):
mean = R8x8.dot(mean)
mean[:2] += t
cov = R8x8.dot(cov).dot(R8x8.transpose())
stracks[i].mean = mean
stracks[i].covariance = cov

So it seems you are applying camera motion compensation only to x,y ( and not w,h) while original code apply rotation and shear transform to x,y, w and h.
See np.kron() for Kronecker product.
Is it a mistake, or there is something else ?

Sorry for editing, code writing is broken in github

[arnaud-nt2i]

Correct code should be :

Eigen::Matrix<float, 8, 8> R8x8 = Eigen::Matrix<float, 8, 8>::Identity();
R8x8.block(0, 0, 2, 2) = R;
R8x8.block(2, 2, 2, 2) = R;
R8x8.block(4, 4, 2, 2) = R;
R8x8.block(6, 6, 2, 2) = R;
mean = R8x8 * mean.transpose();
mean.head(2) += t;
covariance = R8x8 * covariance * R8x8.transpose();

[viplix3]

I will look into it this weekend