diff mbox series

[7/9] nvme-apple: Serialize command issue

Message ID 20220321165049.35985-8-sven@svenpeter.dev (mailing list archive)
State New, archived
Headers show
Series Apple M1 (Pro/Max) NVMe driver | expand

Commit Message

Sven Peter March 21, 2022, 4:50 p.m. UTC 
From: Jens Axboe <axboe@kernel.dk>

This controller shouldn't need serialization of command issue since
the SQ is replaced by a simple array and commands are issued by writing
the array index to a MMIO register.
Without serialization however sometimes commands are still executed
correctly and appear in the CQ but never trigger an interrupt.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
[sven: added our best guess why this needs to be done]
Signed-off-by: Sven Peter <sven@svenpeter.dev>
---
 drivers/nvme/host/apple.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

Comments

Christoph Hellwig March 24, 2022, 6:16 a.m. UTC | #1 
On Mon, Mar 21, 2022 at 05:50:47PM +0100, Sven Peter wrote:
> From: Jens Axboe <axboe@kernel.dk>
> 
> This controller shouldn't need serialization of command issue since
> the SQ is replaced by a simple array and commands are issued by writing
> the array index to a MMIO register.
> Without serialization however sometimes commands are still executed
> correctly and appear in the CQ but never trigger an interrupt.
> 
> Signed-off-by: Jens Axboe <axboe@kernel.dk>
> [sven: added our best guess why this needs to be done]
> Signed-off-by: Sven Peter <sven@svenpeter.dev>

This really should go into the previous patch.
Sven Peter April 2, 2022, 12:42 p.m. UTC | #2 
On Thu, Mar 24, 2022, at 07:16, Christoph Hellwig wrote:
> On Mon, Mar 21, 2022 at 05:50:47PM +0100, Sven Peter wrote:
>> From: Jens Axboe <axboe@kernel.dk>
>> 
>> This controller shouldn't need serialization of command issue since
>> the SQ is replaced by a simple array and commands are issued by writing
>> the array index to a MMIO register.
>> Without serialization however sometimes commands are still executed
>> correctly and appear in the CQ but never trigger an interrupt.
>> 
>> Signed-off-by: Jens Axboe <axboe@kernel.dk>
>> [sven: added our best guess why this needs to be done]
>> Signed-off-by: Sven Peter <sven@svenpeter.dev>
>
> This really should go into the previous patch.

Ok, I'll squash it (and the following two patches) into the previous one.

Sven
diff mbox series

Patch

diff --git a/drivers/nvme/host/apple.c b/drivers/nvme/host/apple.c
index 587d6c7014a0..a4193429564e 100644
--- a/drivers/nvme/host/apple.c
+++ b/drivers/nvme/host/apple.c
@@ -292,6 +292,7 @@  static void apple_nvmmu_inval(struct apple_nvme_queue *q, unsigned int tag)
 static void apple_nvme_submit_cmd(struct apple_nvme_queue *q,
 				  struct nvme_command *cmd)
 {
+	struct apple_nvme *anv = queue_to_apple_nvme(q);
 	u32 tag = nvme_tag_from_cid(cmd->common.command_id);
 	struct apple_nvmmu_tcb *tcb = &q->tcbs[tag];
 
@@ -308,7 +309,18 @@  static void apple_nvme_submit_cmd(struct apple_nvme_queue *q,
 		tcb->dma_flags |= APPLE_ANS_TCB_DMA_FROM_DEVICE;
 
 	memcpy(&q->sqes[tag], cmd, sizeof(*cmd));
+
+	/*
+	 * This lock here doesn't make much sense at a first glace but
+	 * removing it will result in occasional missed completetion
+	 * interrupts even though the commands still appear on the CQ.
+	 * It's unclear why this happens but our best guess is that
+	 * there is a bug in the firmware triggered when a write to the
+	 * CQ and the SQ happen simultaneously.
+	 */
+	spin_lock_irq(&anv->lock);
 	writel(tag, q->sq_db);
+	spin_unlock_irq(&anv->lock);
 }
 
 /*