From patchwork Thu Oct 15 19:31:16 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 7408981 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 311AFBEEA4 for ; Thu, 15 Oct 2015 19:31:25 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 18EAF20793 for ; Thu, 15 Oct 2015 19:31:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1E6442078B for ; Thu, 15 Oct 2015 19:31:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752064AbbJOTbV (ORCPT ); Thu, 15 Oct 2015 15:31:21 -0400 Received: from mail.kernel.org ([198.145.29.136]:51350 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751741AbbJOTbU (ORCPT ); Thu, 15 Oct 2015 15:31:20 -0400 Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id DD70F2078D; Thu, 15 Oct 2015 19:31:18 +0000 (UTC) Received: from localhost (unknown [69.55.156.182]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DED022078B; Thu, 15 Oct 2015 19:31:17 +0000 (UTC) Date: Thu, 15 Oct 2015 14:31:16 -0500 From: Bjorn Helgaas To: Ben Shelton Cc: bhelgaas@google.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2] PCI: IOV: read SRIOV_NUM_VF after enabling ARI Message-ID: <20151015193116.GE17702@localhost> References: <1444317617-13399-1-git-send-email-benjamin.h.shelton@intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1444317617-13399-1-git-send-email-benjamin.h.shelton@intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Oct 08, 2015 at 10:20:17AM -0500, Ben Shelton wrote: > For some SR-IOV devices, the number of available virtual functions increases > after enabling ARI. Currently, SRIOV_NUM_VF is read and saved off before the > ARI control bit is enabled in SRIOV_CTRL. This causes an issue when VFs are > enabled. > > At device init, SRIOV_INITIAL_VF and SRIOV_NUM_VF are specified to contain the > number of available VFs for the device. sriov_enable() does a sanity check > that SRIOV_INITIAL_VF is not greater than iov->total_VFs, the saved-off value > of SRIOV_NUM_VF. Since the value of both SRIOV_INITIAL_VF and SRIOV_NUM_VF has > increased after enabling the ARI bit, the check fails, and the VFs cannot be > enabled. > > To fix the issue, write SRIOV_CTRL first, and then read SRIOV_NUM_VF. > > Signed-off-by: Ben Shelton I applied this as follows to pci/virtualization for v4.4, thanks, Ben! This is on top of a NumVFs-related patch, so the diff looks slightly different, but I think it's functionally equivalent. commit 3aa71da412fedaee133b4b6e4be4b801c59d6c91 Author: Ben Shelton Date: Thu Oct 15 12:35:17 2015 -0500 PCI: Enable SR-IOV ARI Capable Hierarchy before reading TotalVFs For some SR-IOV devices, the number of available virtual functions, i.e., TotalVFs, increases after setting the ARI Capable Hierarchy bit in the SR-IOV Control register. This violates the SR-IOV spec, r1.1, sec 3.3.6, which says TotalVFs is HwInit, but we don't need TotalVFs before setting the ARI Capable bit anyway. Set the ARI Capable Hierarchy bit (if ARI is enabled in the upstream bridge) before reading TotalVFs. [bhelgaas: changelog] Signed-off-by: Ben Shelton Signed-off-by: Bjorn Helgaas --- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index 0202ab0..f8bfc1d 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -399,10 +399,6 @@ static int sriov_init(struct pci_dev *dev, int pos) ssleep(1); } - pci_read_config_word(dev, pos + PCI_SRIOV_TOTAL_VF, &total); - if (!total) - return 0; - ctrl = 0; list_for_each_entry(pdev, &dev->bus->devices, bus_list) if (pdev->is_physfn) @@ -415,6 +411,10 @@ static int sriov_init(struct pci_dev *dev, int pos) found: pci_write_config_word(dev, pos + PCI_SRIOV_CTRL, ctrl); + pci_read_config_word(dev, pos + PCI_SRIOV_TOTAL_VF, &total); + if (!total) + return 0; + pci_read_config_dword(dev, pos + PCI_SRIOV_SUP_PGSIZE, &pgsz); i = PAGE_SHIFT > 12 ? PAGE_SHIFT - 12 : 0; pgsz &= ~((1 << i) - 1);