From patchwork Wed Aug 21 23:39:44 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 2847958 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 73E6BBF546 for ; Wed, 21 Aug 2013 23:39:50 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 55A7C205CD for ; Wed, 21 Aug 2013 23:39:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 48B1A205B9 for ; Wed, 21 Aug 2013 23:39:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753002Ab3HUXjr (ORCPT ); Wed, 21 Aug 2013 19:39:47 -0400 Received: from mail-oa0-f52.google.com ([209.85.219.52]:48506 "EHLO mail-oa0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752413Ab3HUXjr (ORCPT ); Wed, 21 Aug 2013 19:39:47 -0400 Received: by mail-oa0-f52.google.com with SMTP id f4so1185017oah.25 for ; Wed, 21 Aug 2013 16:39:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=subject:to:from:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=PFgTzy4ba3cYjBixQDl57z6k5JX7vSFP8NM6i3qgS/Y=; b=NJ5A37uju5fRvGtQhW4Lgbo1bsf27hVfj8TuN07Sx4HbIvNXoaKGXxTKcwO58LyWUC GlgLSLJUHyWXMBZ9XR+MXImFgPiTuIG0PQDmBm826MdXi+Q9nuF6OvSUeay6TthpmaFg q+Ut5z5qeIspsGVz6KH7yLUv4wDNC7wWdFInuuwHMosVqCQdiwk3+6SkuBHkJQ6FlBYj rg4M0fR5F2Jc03amsYIdgycE5Zhambkhvx9CkYtQ/LgwI5iwq6Gsgt6P1jUke9p5l4j3 fNXhTtNXmpVPJYK0VdeS/jxs9DdvgfgB6mBEJveQhkQJKUKF9eTvJMVnN1KzykBV5vjo Ii4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:subject:to:from:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-type :content-transfer-encoding; bh=PFgTzy4ba3cYjBixQDl57z6k5JX7vSFP8NM6i3qgS/Y=; b=BByi8wBwjE4VG58FnTd2YmUPBoKV8YWYeKZ3vvWPhOsYLn0/5P+tLqlpiDU0RUV3Mx nRO6xra8K1soaPl6B695lz1/1k6wqUzmbclgh0r5Sa7gc7zfM3o6BbHqfuhbZjjtBZBP AImeaeGjl/HCwhX0P8Qv1rF7B19ppw/yJ9mIyyt4hxeOi85AjlKJdafwLxTQAgTjSsjT Z1kcEwxP+cZrRQPl3uSzW+6V18DbVCXcLpk2AZ/7+Bi88ycBHktTEgR+PcROz827YPmi dC/Po04Fy73+eKnmElDiwzPC4yeNyvxlm1PIS6guB31O4myH0KxNH/EPwkPXyIM4gLtU nPJQ== X-Gm-Message-State: ALoCoQkbrtH3CP2WP3mcPXwv+e8YF8mWa6a5lB4V/E0Nh5v8394rmCnzAI/efa5Uw8kEQVEPor66FnJ9bQrLc8NTEtaw3C10XoGY2MgPIbo+s7Mlsb0RmEoMBoDr6sYjegGFGqNFG8jRzhqoyVGIbbrLhWqPFh8V2tR5VP5YTekUG6bRLexI4AEsfMgJDwUtLjMSRwD8rHWbnowI5nr1tVyzCoqUqIP5PA== X-Received: by 10.60.94.108 with SMTP id db12mr5136263oeb.81.1377128386773; Wed, 21 Aug 2013 16:39:46 -0700 (PDT) Received: from localhost ([172.29.122.86]) by mx.google.com with ESMTPSA id b5sm13943574obj.8.1969.12.31.16.00.00 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 21 Aug 2013 16:39:46 -0700 (PDT) Subject: [PATCH v7 5/5] PCI: Don't restrict MPS for slots below Root Ports To: Yijing Wang From: Bjorn Helgaas Cc: linux-pci@vger.kernel.org, Jon Mason , jiang.liu@huawei.com, Hanjun Guo Date: Wed, 21 Aug 2013 17:39:44 -0600 Message-ID: <20130821233944.2300.19147.stgit@bhelgaas-glaptop> In-Reply-To: <20130821233722.2300.81773.stgit@bhelgaas-glaptop> References: <20130821233722.2300.81773.stgit@bhelgaas-glaptop> User-Agent: StGit/0.16 MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Yijing Wang When booting with "pci=pcie_bus_safe", we previously limited the fabric MPS to 128 when we found: (1) A hotplug-capable Downstream Port ("dev->is_hotplug_bridge && pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT"), or (2) A hotplug-capable Root Port with a slot that was either empty or contained a multi-function device ("dev->is_hotplug_bridge && !list_is_singular(&dev->bus->devices)") Part (1) is valid, but part (2) is not. After a hot-add in the slot below a Root Port, we can reconfigure all MPS values in the fabric below the Root Port because the new device is the only thing below the Root Port and there are no active drivers. Therefore, there's no reason to limit the MPS for Root Ports, no matter what's in the slot. Test info: -+-[0000:40]-+-07.0-[0000:46]--+-00.0 Intel 82576 NIC \-00.1 Intel 82576 NIC 0000:40:07.0 Root Port bridge to [bus 46] (MPS supported=256) 0000:46:00.0 Endpoint (MPS supported=512) 0000:46:00.1 Endpoint (MPS supported=512) # echo 0 > /sys/bus/pci/slots/7/power # echo 1 > /sys/bus/pci/slots/7/power # dmesg ... pcieport 0000:40:07.0: PCI-E Max Payload Size set to 256/ 256 (was 256) pci 0000:46:00.0: PCI-E Max Payload Size set to 256/ 512 (was 128) pci 0000:46:00.1: PCI-E Max Payload Size set to 256/ 512 (was 128) Before this change, we set MPS to 128 for the Root Port and both NICs because the slot contained a multi-function device and dev->is_hotplug_bridge && !list_is_singular(&dev->bus->devices) was true. After this change, we set it to 256. [bhelgaas: changelog, comments, split out upstream bridge check] Signed-off-by: Yijing Wang Signed-off-by: Bjorn Helgaas Cc: Jon Mason --- drivers/pci/probe.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 0591b08..94b1d22 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1491,23 +1491,23 @@ static int pcie_find_smpss(struct pci_dev *dev, void *data) if (!pci_is_pcie(dev)) return 0; - /* For PCIE hotplug enabled slots not connected directly to a - * PCI-E root port, there can be problems when hotplugging - * devices. This is due to the possibility of hotplugging a - * device into the fabric with a smaller MPS that the devices - * currently running have configured. Modifying the MPS on the - * running devices could cause a fatal bus error due to an - * incoming frame being larger than the newly configured MPS. - * To work around this, the MPS for the entire fabric must be - * set to the minimum size. Any devices hotplugged into this - * fabric will have the minimum MPS set. If the PCI hotplug - * slot is directly connected to the root port and there are not - * other devices on the fabric (which seems to be the most - * common case), then this is not an issue and MPS discovery - * will occur as normal. + /* + * We don't have a way to change MPS settings on devices that have + * drivers attached. A hot-added device might support only the minimum + * MPS setting (MPS=128). Therefore, if the fabric contains a bridge + * where devices may be hot-added, we limit the fabric MPS to 128 so + * hot-added devices will work correctly. + * + * However, if we hot-add a device to a slot directly below a Root + * Port, it's impossible for there to be other existing devices below + * the port. We don't limit the MPS in this case because we can + * reconfigure MPS on both the Root Port and the hot-added device, + * and there are no other devices involved. + * + * Note that this PCIE_BUS_SAFE path assumes no peer-to-peer DMA. */ - if (dev->is_hotplug_bridge && (!list_is_singular(&dev->bus->devices) || - pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT)) + if (dev->is_hotplug_bridge && + pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT) *smpss = 0; if (*smpss > dev->pcie_mpss)