| Message ID | 20220222150319.28397-1-jpittman@redhat.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | scsi: mpt3sas: decrease potential frequency of scsi_dma_map errors | expand |
On Tue, 2022-02-22 at 10:03 -0500, John Pittman wrote: > When scsi_dma_map() fails by returning a sges_left value less than > zero, the amount of logging can be extremely high. In a recent > end-user environment, 1200 messages per second were being sent to > the log buffer. This eventually overwhelmed the system and it > stalled. As the messages are almost all identical, use > pr_err_ratelimited() instead of sdev_printk() to print the > scsi_dma_map failure messages. > > Signed-off-by: John Pittman <jpittman@redhat.com> > --- > drivers/scsi/mpt3sas/mpt3sas_base.c | 15 ++++++--------- > 1 file changed, 6 insertions(+), 9 deletions(-) > > diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c > b/drivers/scsi/mpt3sas/mpt3sas_base.c > index 511726f92d9a..ac9ccde6f3f8 100644 > --- a/drivers/scsi/mpt3sas/mpt3sas_base.c > +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c > @@ -2594,9 +2594,8 @@ _base_check_pcie_native_sgl(struct > MPT3SAS_ADAPTER *ioc, > /* Get the SG list pointer and info. */ > sges_left = scsi_dma_map(scmd); > if (sges_left < 0) { > - sdev_printk(KERN_ERR, scmd->device, > - "scsi_dma_map failed: request for %d bytes!\n", > - scsi_bufflen(scmd)); > + pr_err_ratelimited("sd %s: scsi_dma_map failed: request > for %d bytes!\n", > + dev_name(&scmd->device->sdev_gendev), > scsi_bufflen(scmd)); > return 1; > } > > @@ -2706,9 +2705,8 @@ _base_build_sg_scmd(struct MPT3SAS_ADAPTER > *ioc, > sg_scmd = scsi_sglist(scmd); > sges_left = scsi_dma_map(scmd); > if (sges_left < 0) { > - sdev_printk(KERN_ERR, scmd->device, > - "scsi_dma_map failed: request for %d bytes!\n", > - scsi_bufflen(scmd)); > + pr_err_ratelimited("sd %s: scsi_dma_map failed: request > for %d bytes!\n", > + dev_name(&scmd->device->sdev_gendev), > scsi_bufflen(scmd)); > return -ENOMEM; > } > > @@ -2854,9 +2852,8 @@ _base_build_sg_scmd_ieee(struct MPT3SAS_ADAPTER > *ioc, > sg_scmd = scsi_sglist(scmd); > sges_left = scsi_dma_map(scmd); > if (sges_left < 0) { > - sdev_printk(KERN_ERR, scmd->device, > - "scsi_dma_map failed: request for %d bytes!\n", > - scsi_bufflen(scmd)); > + pr_err_ratelimited("sd %s: scsi_dma_map failed: request > for %d bytes!\n", > + dev_name(&scmd->device->sdev_gendev), > scsi_bufflen(scmd)); > return -ENOMEM; > } > This high message rate triggered a double completion at a customer and this patch will help prevent the issue of the latency causing the race window for the double completion. This was triaged by David Jeffery Reviewed-by: Laurence Oberman <loberman@redhat.com>
On Tue, Feb 22, 2022 at 10:03:19AM -0500, John Pittman wrote: > When scsi_dma_map() fails by returning a sges_left value less than > zero, the amount of logging can be extremely high. In a recent > end-user environment, 1200 messages per second were being sent to > the log buffer. This eventually overwhelmed the system and it > stalled. As the messages are almost all identical, use > pr_err_ratelimited() instead of sdev_printk() to print the > scsi_dma_map failure messages. I'd remove the message entirely.
> ACK by BRCM.
Please resubmit with message removed as requested by Christoph.
On Wed, Mar 2, 2022 at 9:54 AM Martin K. Petersen <martin.petersen@oracle.com> wrote: > > > > ACK by BRCM. > > Please resubmit with message removed as requested by Christoph. Posted below patch to remove these error messages, mpt3sas: Remove scsi_dma_map errors messages https://patchwork.kernel.org/project/linux-scsi/patch/20220303140203.12642-1-sreekanth.reddy@broadcom.com/ > > -- > Martin K. Petersen Oracle Linux Engineering
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 511726f92d9a..ac9ccde6f3f8 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -2594,9 +2594,8 @@ _base_check_pcie_native_sgl(struct MPT3SAS_ADAPTER *ioc, /* Get the SG list pointer and info. */ sges_left = scsi_dma_map(scmd); if (sges_left < 0) { - sdev_printk(KERN_ERR, scmd->device, - "scsi_dma_map failed: request for %d bytes!\n", - scsi_bufflen(scmd)); + pr_err_ratelimited("sd %s: scsi_dma_map failed: request for %d bytes!\n", + dev_name(&scmd->device->sdev_gendev), scsi_bufflen(scmd)); return 1; } @@ -2706,9 +2705,8 @@ _base_build_sg_scmd(struct MPT3SAS_ADAPTER *ioc, sg_scmd = scsi_sglist(scmd); sges_left = scsi_dma_map(scmd); if (sges_left < 0) { - sdev_printk(KERN_ERR, scmd->device, - "scsi_dma_map failed: request for %d bytes!\n", - scsi_bufflen(scmd)); + pr_err_ratelimited("sd %s: scsi_dma_map failed: request for %d bytes!\n", + dev_name(&scmd->device->sdev_gendev), scsi_bufflen(scmd)); return -ENOMEM; } @@ -2854,9 +2852,8 @@ _base_build_sg_scmd_ieee(struct MPT3SAS_ADAPTER *ioc, sg_scmd = scsi_sglist(scmd); sges_left = scsi_dma_map(scmd); if (sges_left < 0) { - sdev_printk(KERN_ERR, scmd->device, - "scsi_dma_map failed: request for %d bytes!\n", - scsi_bufflen(scmd)); + pr_err_ratelimited("sd %s: scsi_dma_map failed: request for %d bytes!\n", + dev_name(&scmd->device->sdev_gendev), scsi_bufflen(scmd)); return -ENOMEM; }
When scsi_dma_map() fails by returning a sges_left value less than zero, the amount of logging can be extremely high. In a recent end-user environment, 1200 messages per second were being sent to the log buffer. This eventually overwhelmed the system and it stalled. As the messages are almost all identical, use pr_err_ratelimited() instead of sdev_printk() to print the scsi_dma_map failure messages. Signed-off-by: John Pittman <jpittman@redhat.com> --- drivers/scsi/mpt3sas/mpt3sas_base.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)