From patchwork Fri Mar 20 23:18:04 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Luis R. Rodriguez" X-Patchwork-Id: 6061861 Return-Path: X-Original-To: patchwork-linux-fbdev@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id CD9A99F314 for ; Fri, 20 Mar 2015 23:51:58 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 19C7020529 for ; Fri, 20 Mar 2015 23:51:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 74F55204E3 for ; Fri, 20 Mar 2015 23:51:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751107AbbCTXvq (ORCPT ); Fri, 20 Mar 2015 19:51:46 -0400 Received: from mail-pa0-f41.google.com ([209.85.220.41]:33521 "EHLO mail-pa0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751090AbbCTXvp (ORCPT ); Fri, 20 Mar 2015 19:51:45 -0400 Received: by pabxg6 with SMTP id xg6so110613862pab.0; Fri, 20 Mar 2015 16:51:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=NJQxkOTDZZrLgN8FvaCAAR7YdvmUJcktRV6p6iU/uT0=; b=fNJmXgPS/FgTQZ/If3gbypOy4HVLdwi3AZNVZ6SKuAfiWWSDJQi8X/mrJNK4B1NL87 lQYjnI8sy3Ggj+PZxfmu/vPCxcXdVHnzX6r+PU6o3E1fJtVZ9dzz1Qd5oYVRxfniL72I wYpqrEQEXaUKK7iJDuNSwHbnQZW23gFA+VVPyySfo1LTNPTeEOOv/7vEovuwlMYtJX3B cIHNM36Yfg6HjT2sl9bGxN+6xu/yxO4Eq1EkqlFndcSjMtbQcuQuzPq4PyZDoI4t+GRy KxdFkRhgtdhGEfnrW6NU0oZVftxlKXvbPf3j1Inv3EVlOgpiyjoTgdmpAheZ8JjJuDHF 1ayQ== X-Received: by 10.70.100.39 with SMTP id ev7mr42116080pdb.29.1426895505377; Fri, 20 Mar 2015 16:51:45 -0700 (PDT) Received: from mcgrof@gmail.com (c-98-234-145-61.hsd1.ca.comcast.net. [98.234.145.61]) by mx.google.com with ESMTPSA id jx5sm173092pbc.85.2015.03.20.16.51.41 (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 20 Mar 2015 16:51:44 -0700 (PDT) Received: by mcgrof@gmail.com (sSMTP sendmail emulation); Fri, 20 Mar 2015 16:49:32 -0700 From: "Luis R. Rodriguez" To: luto@amacapital.net, mingo@redhat.com, tglx@linutronix.de, hpa@zytor.com, jgross@suse.com, JBeulich@suse.com, bp@suse.de, suresh.b.siddha@intel.com, venkatesh.pallipadi@intel.com, airlied@redhat.com Cc: linux-kernel@vger.kernel.org, linux-fbdev@vger.kernel.org, x86@kernel.org, xen-devel@lists.xenproject.org, "Luis R. Rodriguez" , Rickard Strandqvist , Mike Marciniszyn , Roland Dreier , Ingo Molnar , Linus Torvalds , Daniel Vetter , Bjorn Helgaas , Antonino Daplas , Jean-Christophe Plagniol-Villard , Tomi Valkeinen , Dave Hansen , Arnd Bergmann , "Michael S. Tsirkin" , Stefan Bader , konrad.wilk@oracle.com, ville.syrjala@linux.intel.com, david.vrabel@citrix.com, jbeulich@suse.com, toshi.kani@hp.com, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , xen-devel@lists.xensource.com Subject: [PATCH v1 14/47] IB/ipath: use __arch_phys_wc_add() Date: Fri, 20 Mar 2015 16:18:04 -0700 Message-Id: <1426893517-2511-15-git-send-email-mcgrof@do-not-panic.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1426893517-2511-1-git-send-email-mcgrof@do-not-panic.com> References: <1426893517-2511-1-git-send-email-mcgrof@do-not-panic.com> MIME-Version: 1.0 Sender: linux-fbdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, T_DKIM_INVALID, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: "Luis R. Rodriguez" This driver sadly does not have the MMIO registers and WC desired areas (PIO buffers in this case) properly split up and addressing a split is considerable work, as such this such requires using the __arch_phys_wc_add() call to ensure write combining is enforced using MTRR on x86 even when PAT is available. Cc: Rickard Strandqvist Cc: Mike Marciniszyn Cc: Roland Dreier Cc: Andy Lutomirski Cc: Suresh Siddha Cc: Venkatesh Pallipadi Cc: Ingo Molnar Cc: Linus Torvalds Cc: Thomas Gleixner Cc: Juergen Gross Cc: Daniel Vetter Cc: Dave Airlie Cc: Bjorn Helgaas Cc: Antonino Daplas Cc: Jean-Christophe Plagniol-Villard Cc: Tomi Valkeinen Cc: Dave Hansen Cc: Arnd Bergmann Cc: Michael S. Tsirkin Cc: venkatesh.pallipadi@intel.com Cc: Stefan Bader Cc: konrad.wilk@oracle.com Cc: ville.syrjala@linux.intel.com Cc: david.vrabel@citrix.com Cc: jbeulich@suse.com Cc: toshi.kani@hp.com Cc: Roger Pau Monné Cc: linux-fbdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: xen-devel@lists.xensource.com Signed-off-by: Luis R. Rodriguez --- drivers/infiniband/hw/ipath/ipath_driver.c | 7 ++-- drivers/infiniband/hw/ipath/ipath_kernel.h | 4 +-- drivers/infiniband/hw/ipath/ipath_wc_x86_64.c | 47 ++++++++++----------------- 3 files changed, 20 insertions(+), 38 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index bd0caed..464f39c 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -542,6 +542,7 @@ static int ipath_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dd->ipath_kregbase = __ioremap(addr, len, (_PAGE_NO_CACHE|_PAGE_WRITETHRU)); #else + /* XXX: split pio on a separate ioremap_wc() */ dd->ipath_kregbase = ioremap_nocache(addr, len); #endif @@ -587,12 +588,8 @@ static int ipath_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) ret = ipath_enable_wc(dd); - if (ret) { - ipath_dev_err(dd, "Write combining not enabled " - "(err %d): performance may be poor\n", - -ret); + if (ret) ret = 0; - } ipath_verify_pioperf(dd); diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index e08db70..f0f9471 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -463,9 +463,7 @@ struct ipath_devdata { /* offset in HT config space of slave/primary interface block */ u8 ipath_ht_slave_off; /* for write combining settings */ - unsigned long ipath_wc_cookie; - unsigned long ipath_wc_base; - unsigned long ipath_wc_len; + int wc_cookie; /* ref count for each pkey */ atomic_t ipath_pkeyrefs[4]; /* shadow copy of struct page *'s for exp tid pages */ diff --git a/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c b/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c index 70c1f3a..88709c1 100644 --- a/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c +++ b/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c @@ -37,7 +37,6 @@ */ #include -#include #include #include "ipath_kernel.h" @@ -122,27 +121,26 @@ int ipath_enable_wc(struct ipath_devdata *dd) } if (!ret) { - int cookie; ipath_cdbg(VERBOSE, "Setting mtrr for chip to WC " "(addr %llx, len=0x%llx)\n", (unsigned long long) pioaddr, (unsigned long long) piolen); - cookie = mtrr_add(pioaddr, piolen, MTRR_TYPE_WRCOMB, 1); - if (cookie < 0) { - { - dev_info(&dd->pcidev->dev, - "mtrr_add() WC for PIO bufs " - "failed (%d)\n", - cookie); - ret = -EINVAL; - } - } else { - ipath_cdbg(VERBOSE, "Set mtrr for chip to WC, " - "cookie is %d\n", cookie); - dd->ipath_wc_cookie = cookie; - dd->ipath_wc_base = (unsigned long) pioaddr; - dd->ipath_wc_len = (unsigned long) piolen; - } + dd->wc_cookie = __arch_phys_wc_add(pioaddr, piolen); + if (dd->wc_cookie <= 0) { + /* + * If MTRR is not available on an architecture + * or if it could not be enabled at run time + * folks who care should work towards the + * ioremap_wc() split. + */ + if (!dd->wc_cookie) + ipath_dev_err(dd, "System does not support MTRR\n"); + else { + ipath_dev_err(dd, "Seting mtrr failed on PIO buffers\n"); + ret = -EINVAL; + } + } else + ipath_cdbg(VERBOSE, "Set mtrr for chip to WC\n"); } return ret; @@ -154,16 +152,5 @@ int ipath_enable_wc(struct ipath_devdata *dd) */ void ipath_disable_wc(struct ipath_devdata *dd) { - if (dd->ipath_wc_cookie) { - int r; - ipath_cdbg(VERBOSE, "undoing WCCOMB on pio buffers\n"); - r = mtrr_del(dd->ipath_wc_cookie, dd->ipath_wc_base, - dd->ipath_wc_len); - if (r < 0) - dev_info(&dd->pcidev->dev, - "mtrr_del(%lx, %lx, %lx) failed: %d\n", - dd->ipath_wc_cookie, dd->ipath_wc_base, - dd->ipath_wc_len, r); - dd->ipath_wc_cookie = 0; /* even on failure */ - } + arch_phys_wc_del(dd->wc_cookie); }