From patchwork Thu May 26 10:16:58 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaitra P B X-Patchwork-Id: 9136895 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A48E56075C for ; Thu, 26 May 2016 10:17:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9703D27EE9 for ; Thu, 26 May 2016 10:17:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B711281D4; Thu, 26 May 2016 10:17:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB82B27EE9 for ; Thu, 26 May 2016 10:17:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752944AbcEZKRD (ORCPT ); Thu, 26 May 2016 06:17:03 -0400 Received: from mail-qk0-f169.google.com ([209.85.220.169]:35901 "EHLO mail-qk0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751355AbcEZKRA convert rfc822-to-8bit (ORCPT ); Thu, 26 May 2016 06:17:00 -0400 Received: by mail-qk0-f169.google.com with SMTP id x7so54636585qkd.3 for ; Thu, 26 May 2016 03:17:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:cc:content-transfer-encoding; bh=ZoQw51DDslHDtNWpMRFG9Zyc/JGY2H0lYqwrZPD19c8=; b=U/aDjNzs2V+sfpc/CAFWLpjy73PKZLya/zlNsyIJpVUOwroJsIdNNYe62outUdK7jY s5FjWE+FAKxrjO2WdzYTG3sVralfodc56wxXjQxPeKKJCRTBjmCxSsseNKRSxeraHa1P BPW8iwIskuGyQB2zcT6kpsINDQ21KZlpnXBPw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc :content-transfer-encoding; bh=ZoQw51DDslHDtNWpMRFG9Zyc/JGY2H0lYqwrZPD19c8=; b=CfNGXOEwQ7ZA7W5tfzIQTiQ7njpUxLh6iuDr+7v/I2ma8+5dXnnfvyiBGEagHxOqpf BWSMN+eTdHl0YyOJ7YSLFsd9DUnAAv3P0MdRUMpn5Kzbw/m6Oe7x60OeLImMGUFhEPA7 qX2Y0Hkrj0ZAD/C3FUEbc2V91ClyrlaQ0ksYgcnevJJ3WvQsVBKsIXd9qF10KFuqoV+g Xn3y4rJDwlVXKonrRwNyeDlLgD5TqgruqPo39Yu7F1hfnEiqX4mR/HY4G6eZrMuG0je2 99QkWhJLSf9fsiYA4IXWHwSjRfAfBOZyjR+9178S3BQRnD6xIArLIAQ9Tcol7fdbbRXc ElsQ== X-Gm-Message-State: ALyK8tJKVQV5tjtUrYddMfOMqDrrWaJyJAmi7Vl4+mo8MpVdn6xUYOnm9q9dKlo/g5TJQBJb5KSbyxU3ue8MsT1V X-Received: by 10.55.8.12 with SMTP id 12mr7904640qki.200.1464257819552; Thu, 26 May 2016 03:16:59 -0700 (PDT) From: Chaitra Basappa References: <20160525091853.psv6kml6oingj4fx@c203.arch.suse.de> In-Reply-To: <20160525091853.psv6kml6oingj4fx@c203.arch.suse.de> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQLQn4nNdpugBXUQmMAh8zLsl4EQ1Z3NKyfA Date: Thu, 26 May 2016 15:46:58 +0530 Message-ID: Subject: RE: mpt3sas: memory allocation for firmware upgrade DMA memory question To: Johannes Thumshirn , Suganath Prabu Subramani Cc: Linux SCSI Mailinglist , Jeff Mahoney Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Johannes, Could you please let us know which application is being used for firmware upgrade ?? Whether it is your own customized application or LSI provided applications? Also our application use single contiguous memory buffer for ioctls and hence mpt3sas driver is using single contiguous memory for the DMA operation. If we use GFP_KERNEL flag then it may be possible that ioctl thread may hang/wait for long, if it doesn't get required memory from the system. We may need to test below patch thoroughly , as I don’t see allocation of several non-contiguous chunks of memory in below patch..., Thanks, Chaitra -----Original Message----- From: Johannes Thumshirn [mailto:jthumshirn@suse.de] Sent: Wednesday, May 25, 2016 2:49 PM To: Chaitra P B; Suganath Prabu Subramani Cc: Linux SCSI Mailinglist; Jeff Mahoney Subject: mpt3sas: memory allocation for firmware upgrade DMA memory question Hi Chaitra and Suganath, I've got a question regarding mpt3sas' memory allocation used when doing a firmware upgrade. Currently you're doing a pci_alloc_consitent() which tries to allocate memory via GFP_ATOMIC. This memory then is passed as a single element on a sg_list. Jeff reported it returned -ENOMEM on his Server due to highly fragmented memory. Is it required to have the memory for the DMA operation contiguous, or can I just allocate several non-contiguous junks of memory and map it to a sg_list? If not, is GFP_ATMOIC really needed? I've converted the driver to use GFP_KERNEL but I'm a bit reluctant to test below patch on real hardware to not brick the HBA. Thanks, Johannes RFC patch for GFP_KERNEL allocation, though splitting into multiple sg mapped elements is the preferred fix here: From 06e63654d887df7f740dc5abcb40d441a8da7fa5 Mon Sep 17 00:00:00 2001 From: Johannes Thumshirn Date: Tue, 24 May 2016 17:25:59 +0200 Subject: [RFC PATCH] mpt3sas: Don't do atomic memory allocations for firmware update DMA Currently mpt3sas uses pci_alloc_consistent() to allocate memory for the DMA used to do firmware updates. pci_alloc_consistent() in turn uses GFP_ATOMIC allocations. On a host with high memory fragmention this can lead to page allocation failures, as the DMA buffer holds the complete firmware update and thus can need page allocations of higher orders. As the firmware update code path may sleep, convert allocation to a normal kzalloc() call with GFP_KERNEL and map it to the DMA buffers. Reported-by: Jeff Mahoney Signed-off-by: Johannes Thumshirn --- drivers/scsi/mpt3sas/mpt3sas_ctl.c | 39 ++++++++++++++++++++++++++++---------- 1 file changed, 29 insertions(+), 10 deletions(-) __LINE__, __func__); @@ -760,6 +759,14 @@ _ctl_do_mpt_command(struct MPT3SAS_ADAPTER *ioc, struct mpt3_ioctl_command karg, mpt3sas_base_free_smid(ioc, smid); goto out; } + data_out_dma = pci_map_single(ioc->pdev, data_out, + data_out_sz, PCI_DMA_TODEVICE); + if (dma_mapping_error(&ioc->pdev->dev, data_out_dma)) { + ret = -EINVAL; + mpt3sas_base_free_smid(ioc, smid); + goto out_free_data_out; + } + if (copy_from_user(data_out, karg.data_out_buf_ptr, data_out_sz)) { pr_err("failure at %s:%d/%s()!\n", __FILE__, @@ -771,8 +778,7 @@ _ctl_do_mpt_command(struct MPT3SAS_ADAPTER *ioc, struct mpt3_ioctl_command karg, } if (data_in_sz) /* READ */ { - data_in = pci_alloc_consistent(ioc->pdev, data_in_sz, - &data_in_dma); + data_in = kzalloc(data_in_sz, GFP_KERNEL); if (!data_in) { pr_err("failure at %s:%d/%s()!\n", __FILE__, __LINE__, __func__); @@ -780,6 +786,13 @@ _ctl_do_mpt_command(struct MPT3SAS_ADAPTER *ioc, struct mpt3_ioctl_command karg, mpt3sas_base_free_smid(ioc, smid); goto out; } + data_in_dma = pci_map_single(ioc->pdev, data_in, + data_in_sz, PCI_DMA_FROMDEVICE); + if (dma_mapping_error(&ioc->pdev->dev, data_in_dma)) { + ret = -EINVAL; + mpt3sas_base_free_smid(ioc, smid); + goto out_free_data_in; + } } psge = (void *)request + (karg.data_sge_offset*4); @@ -1013,13 +1026,19 @@ _ctl_do_mpt_command(struct MPT3SAS_ADAPTER *ioc, struct mpt3_ioctl_command karg, out: /* free memory associated with sg buffers */ - if (data_in) - pci_free_consistent(ioc->pdev, data_in_sz, data_in, - data_in_dma); + if (data_in) { + pci_unmap_single(ioc->pdev, data_in_dma, + data_in_sz, PCI_DMA_TODEVICE); +out_free_data_in: + kfree(data_in); + } - if (data_out) - pci_free_consistent(ioc->pdev, data_out_sz, data_out, - data_out_dma); + if (data_out) { + pci_unmap_single(ioc->pdev, data_out_dma, + data_out_sz, PCI_DMA_FROMDEVICE); +out_free_data_out: + kfree(data_out); + } kfree(mpi_request); ioc->ctl_cmds.status = MPT3_CMD_NOT_USED; -- 1.8.5.6 -- Johannes Thumshirn Storage jthumshirn@suse.de +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850 -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/scsi/mpt3sas/mpt3sas_ctl.c b/drivers/scsi/mpt3sas/mpt3sas_ctl.c index 7d00f09..14be3cf 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_ctl.c +++ b/drivers/scsi/mpt3sas/mpt3sas_ctl.c @@ -751,8 +751,7 @@ _ctl_do_mpt_command(struct MPT3SAS_ADAPTER *ioc, struct mpt3_ioctl_command karg, /* obtain dma-able memory for data transfer */ if (data_out_sz) /* WRITE */ { - data_out = pci_alloc_consistent(ioc->pdev, data_out_sz, - &data_out_dma); + data_out = kzalloc(data_out_sz, GFP_KERNEL); if (!data_out) { pr_err("failure at %s:%d/%s()!\n", __FILE__,