lpfc: Fix interrupt assignments when multiple vectors are supported on same cpu

Commit Message

James Smart July 6, 2020, 8:42 p.m. UTC

With certain platforms its possible pci_alloc_irq_vectors() may
affinitize irq vectors to multiple (all?) cpus. The driver is currently
assuming exclusivity and vectors being doled out to different cpus and
is assigning primary ownership of each vector to the first cpu in the
mask.  The code doesn't bother to check if the cpu already owns a vector
and will unconditionally overwrite the cpu to vector mapping. This causes
the relationships between eq's and cq's to get confused and gets worse
when cpus start to offline. The net results are interrupts are skipped
resulting in mailbox timeouts and there are oops's in cpu offling flows.

Fix this changing up the primary vector assignment. Now assign the eq to
a cpu only if it is the cpu in the mask that does not have a prior
assignment. And once the primary ownership is assigned, break from the
loop. For cpu's that may have been set before but not the primary owner,
the lpfc_cpu_affinity_check() routine will balance the cpu to eq
assignment.

Signed-off-by: Dick Kennedy <[e-mail]>
Signed-off-by: James Smart <[e-mail]>
---
 drivers/scsi/lpfc/lpfc_init.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

Comments

Martin K. Petersen July 8, 2020, 6:06 a.m. UTC | #1

On Mon, 6 Jul 2020 13:42:30 -0700, James Smart wrote:

> With certain platforms its possible pci_alloc_irq_vectors() may
> affinitize irq vectors to multiple (all?) cpus. The driver is currently
> assuming exclusivity and vectors being doled out to different cpus and
> is assigning primary ownership of each vector to the first cpu in the
> mask.  The code doesn't bother to check if the cpu already owns a vector
> and will unconditionally overwrite the cpu to vector mapping. This causes
> the relationships between eq's and cq's to get confused and gets worse
> when cpus start to offline. The net results are interrupts are skipped
> resulting in mailbox timeouts and there are oops's in cpu offling flows.
> 
> [...]

Applied to 5.9/scsi-queue, thanks!

[1/1] scsi: lpfc: Fix interrupt assignments when multiple vectors are supported on same CPU
      https://git.kernel.org/mkp/scsi/c/17105d959b26

Patch

diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 33d334ac8d20..4ba8202d391b 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -11522,9 +11522,9 @@  lpfc_sli4_enable_msix(struct lpfc_hba *phba)
 	char *name;
 	const struct cpumask *aff_mask = NULL;
 	unsigned int cpu = 0, cpu_cnt = 0, cpu_select = nr_cpu_ids;
+	struct lpfc_vector_map_info *cpup;
 	struct lpfc_hba_eq_hdl *eqhdl;
 	const struct cpumask *maskp;
-	bool first;
 	unsigned int flags = PCI_IRQ_MSIX;
 
 	/* Set up MSI-X multi-message vectors */
@@ -11597,18 +11597,28 @@  lpfc_sli4_enable_msix(struct lpfc_hba *phba)
 		} else {
 			maskp = pci_irq_get_affinity(phba->pcidev, index);
 
-			first = true;
 			/* Loop through all CPUs associated with vector index */
 			for_each_cpu_and(cpu, maskp, cpu_present_mask) {
+				cpup = &phba->sli4_hba.cpu_map[cpu];
+
 				/* If this is the first CPU thats assigned to
 				 * this vector, set LPFC_CPU_FIRST_IRQ.
+				 *
+				 * With certain platforms its possible that irq
+				 * vectors are affinitized to all the cpu's.
+				 * This can result in each cpu_map.eq to be set
+				 * to the last vector, resulting in overwrite
+				 * of all the previous cpu_map.eq.  Ensure that
+				 * each vector receives a place in cpu_map.
+				 * Later call to lpfc_cpu_affinity_check will
+				 * ensure we are nicely balanced out.
 				 */
+				if (cpup->eq != LPFC_VECTOR_MAP_EMPTY)
+					continue;
 				lpfc_assign_eq_map_info(phba, index,
-							first ?
-							LPFC_CPU_FIRST_IRQ : 0,
+							LPFC_CPU_FIRST_IRQ,
 							cpu);
-				if (first)
-					first = false;
+				break;
 			}
 		}
 	}