common/cnxk: fix race condition between up and down mailbox

Fixing a possible case for race condition where an up mbox
interrupt over writes the down mbox message.
Although mbox_wait_for_zero() makes sure no up/down message
is pending before raising an up mbox interrupt. But there is a
small window were a VF may send a down mbox request to PF after
mbox_wait_for_zero() and before PF attempts to send a up
message to same VF. In such scenario interrupt register which has
down message bit set will get overwritten by up message bit.

As a solution, read interrupt register and OR the status with
required up/down bit before writing to the interrupt register.

Fixes: fa4ee2d ("common/cnxk: sync between mbox up and down messages")
Cc: [email protected]

Signed-off-by: Harman Kalra <[email protected]>

drivers/common/cnxk/roc_mbox.c

@@ -209,10 +209,9 @@ static void
 mbox_msg_send_data(struct mbox *mbox, int devid, uint8_t data)
 {
 	struct mbox_dev *mdev = &mbox->dev[devid];
-	struct mbox_hdr *tx_hdr =
-		(struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->tx_start);
-	struct mbox_hdr *rx_hdr =
-		(struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->rx_start);
+	struct mbox_hdr *tx_hdr = (struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->tx_start);
+	struct mbox_hdr *rx_hdr = (struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->rx_start);
+	uint64_t intr_val;
 
 	/* Reset header for next messages */
 	tx_hdr->msg_size = mdev->msg_size;
@@ -229,11 +228,16 @@ mbox_msg_send_data(struct mbox *mbox, int devid, uint8_t data)
 	/* Sync mbox data into memory */
 	plt_wmb();
 
+	/* Check for any pending interrupt */
+	intr_val = plt_read64(
+		(volatile void *)(mbox->reg_base + (mbox->trigger | (devid << mbox->tr_shift))));
+
+	intr_val |= (uint64_t)data;
 	/* The interrupt should be fired after num_msgs is written
 	 * to the shared memory
 	 */
-	plt_write64(data, (volatile void *)(mbox->reg_base +
-				(mbox->trigger | (devid << mbox->tr_shift))));
+	plt_write64(intr_val, (volatile void *)(mbox->reg_base +
+						(mbox->trigger | (devid << mbox->tr_shift))));
 }
 
 /**