vdpa/mlx5: fix unregister kick handler order

The mlx5_vdpa_virtq_kick_handler function may still be running and waiting
on virtq->virtq_lock while mlx5_vdpa_cqe_event_unset function is trying to
re-initialize the virtq->virtq_lock.

This causes mlx5_vdpa_virtq_kick_handler thread can't be wake up and can't
be unregister. Following print may loop forever when calling
rte_vhost_driver_unregister(socket_path):

    mlx5_vdpa: Try again to unregister fd 154 of virtq 11 interrupt
    mlx5_vdpa: Try again to unregister fd 154 of virtq 11 interrupt
    ...

The fix is to move mlx5_vdpa_virtq_unregister_intr_handle before
mlx5_vdpa_cqe_event_unset.

Fixes: 057f7d2 ("vdpa/mlx5: optimize datapath-control synchronization")
Cc: [email protected]

Signed-off-by: Yajun Wu <[email protected]>
Acked-by: Matan Azrad <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>

drivers/vdpa/mlx5/mlx5_vdpa.c

@@ -282,6 +282,7 @@ _internal_mlx5_vdpa_dev_close(struct mlx5_vdpa_priv *priv,
 	int ret = 0;
 	int vid = priv->vid;
 
+	mlx5_vdpa_virtq_unreg_intr_handle_all(priv);
 	mlx5_vdpa_cqe_event_unset(priv);
 	if (priv->state == MLX5_VDPA_STATE_CONFIGURED) {
 		ret |= mlx5_vdpa_lm_log(priv);

drivers/vdpa/mlx5/mlx5_vdpa_cthread.c

@@ -189,7 +189,6 @@ mlx5_vdpa_c_thread_handle(void *arg)
 			pthread_mutex_unlock(&virtq->virtq_lock);
 			break;
 		case MLX5_VDPA_TASK_DEV_CLOSE_NOWAIT:
-			mlx5_vdpa_virtq_unreg_intr_handle_all(priv);
 			pthread_mutex_lock(&priv->steer_update_lock);
 			mlx5_vdpa_steer_unset(priv);
 			pthread_mutex_unlock(&priv->steer_update_lock);