From patchwork Sun Jan 8 09:14:49 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shamir Rabinovitch X-Patchwork-Id: 9503385 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id ECF2C6071A for ; Sun, 8 Jan 2017 09:15:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD36128391 for ; Sun, 8 Jan 2017 09:15:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9F2B6283BA; Sun, 8 Jan 2017 09:15:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 18B8A28391 for ; Sun, 8 Jan 2017 09:15:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755670AbdAHJPM (ORCPT ); Sun, 8 Jan 2017 04:15:12 -0500 Received: from aserp1040.oracle.com ([141.146.126.69]:29444 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755818AbdAHJPH (ORCPT ); Sun, 8 Jan 2017 04:15:07 -0500 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v089F2oV021746 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 8 Jan 2017 09:15:03 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v089F2Lb015956 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 8 Jan 2017 09:15:02 GMT Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v089F1tA020854; Sun, 8 Jan 2017 09:15:01 GMT Received: from shamir-net-srv.us.oracle.com (/10.211.3.142) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sun, 08 Jan 2017 01:15:01 -0800 From: Shamir Rabinovitch To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, shamir.rabinovitch@oracle.com Subject: [PATCH v1] net/mlx4_core: VF probe fail when HW support 64-bit coherent DMA Date: Sun, 8 Jan 2017 04:14:49 -0500 Message-Id: <1483866889-10700-1-git-send-email-shamir.rabinovitch@oracle.com> X-Mailer: git-send-email 1.7.1 X-Source-IP: aserv0022.oracle.com [141.146.126.234] Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP CX3 firmware is unable to work with 64-bit coherent DMA. If IOMMU support 64-bit coherent DMA mlx4_core driver will try to use it and VF probe will fail due to firmware error. Force all mlx4_core coherent DMA to 32-bit only! Kernel log when issue occur: [1383654.766249] mlx4_core 0006:01:00.1: Sending reset [1383654.775971] mlx4_core 0006:01:00.0: Received reset from slave:1 [1383654.788087] mlx4_core 0006:01:00.1: Sending vhcr0 [1383664.318338] mlx4_core 0006:01:00.0: command 0x2e failed: fw status = 0x1 [1383664.318342] mlx4_core 0006:01:00.0: mlx4_master_process_vhcr: Failed reading vhcr ret: 0xfffffffb [1383664.318345] mlx4_core 0006:01:00.0: Failed processing vhcr for slave:1, resetting slave [1383664.318352] mlx4_core 0006:01:00.0: Turn on internal error to force reset, slave=1, cmd=0x5 [1383664.318415] mlx4_core 0006:01:00.0: slave:1 is out of sync, cmd=0x5, last command=0x0, reset is needed [1383664.318418] mlx4_core 0006:01:00.0: Turn on internal error to force reset, slave=1, cmd=0x5 [1383664.318501] mlx4_core 0006:01:00.0: slave:1 is out of sync, cmd=0x5, last command=0x0, reset is needed [1383664.318504] mlx4_core 0006:01:00.0: Turn on internal error to force reset, slave=1, cmd=0x5 [1383664.318513] mlx4_core 0006:01:00.1: HCA minimum page size:1 [1383664.318515] mlx4_core 0006:01:00.1: UAR size:4096 != kernel PAGE_SIZE of 8192 [1383664.318517] mlx4_core 0006:01:00.1: Failed to obtain slave caps Signed-off-by: Shamir Rabinovitch --- drivers/net/ethernet/mellanox/mlx4/main.c | 11 ++++------- 1 files changed, 4 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c index 5e7840a..851d7f4 100644 --- a/drivers/net/ethernet/mellanox/mlx4/main.c +++ b/drivers/net/ethernet/mellanox/mlx4/main.c @@ -3701,14 +3701,11 @@ static int __mlx4_init_one(struct pci_dev *pdev, int pci_dev_data, goto err_release_regions; } } - err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64)); + /* CX3 firmware 2.11.1280 does not support 64-bit coherent DMA */ + err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32)); if (err) { - dev_warn(&pdev->dev, "Warning: couldn't set 64-bit consistent PCI DMA mask\n"); - err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32)); - if (err) { - dev_err(&pdev->dev, "Can't set consistent PCI DMA mask, aborting\n"); - goto err_release_regions; - } + dev_err(&pdev->dev, "Can't set consistent PCI DMA mask, aborting\n"); + goto err_release_regions; } /* Allow large DMA segments, up to the firmware limit of 1 GB */