From patchwork Thu Jul 6 21:07:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sinan Kaya X-Patchwork-Id: 9829023 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A7155602BD for ; Thu, 6 Jul 2017 21:07:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 956C6284B2 for ; Thu, 6 Jul 2017 21:07:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 881772851E; Thu, 6 Jul 2017 21:07:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D0A1284B2 for ; Thu, 6 Jul 2017 21:07:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752117AbdGFVHZ (ORCPT ); Thu, 6 Jul 2017 17:07:25 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:47386 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751873AbdGFVHY (ORCPT ); Thu, 6 Jul 2017 17:07:24 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 6FBA860996; Thu, 6 Jul 2017 21:07:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1499375243; bh=JAVue3nbdIopEEoRkUicumz60lcBHTdM6ffjwMzQBkI=; h=From:To:Cc:Subject:Date:From; b=kTtVPbl1+xmOPQt3OMONyp9KBA/R9Ja+HcSk7dPIGzlVVHrqrdhO24wjqQX9ZwPxz odWBWFSnXFjmJ6XZZYBaEIsz54O4m2yN/wsemAve59W4JF8nPuFmShSvqytsAtspHj OlBFG+3Rq/8RPERNHIRXYgFrgaICg7Lx4Xs8uszE= Received: from drakthul.qualcomm.com (global_nat1_iad_fw.qualcomm.com [129.46.232.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: okaya@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id B914E6081E; Thu, 6 Jul 2017 21:07:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1499375242; bh=JAVue3nbdIopEEoRkUicumz60lcBHTdM6ffjwMzQBkI=; h=From:To:Cc:Subject:Date:From; b=WmcrryREOxJL6plqnSQ8z9SQ2cK0UrHWU8QtjC/KLOVWLetANFrKIp4uCC66CI45E kVou6PHzriUyAHJnhzeMmOUqNRD9PBwF9p6aICmxhhpx1D8/T4RAm9INgGspsfwN8n Bd3yOm8dCBAHaTxj6mY8oW7Tq05y29rPRiyElQuk= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org B914E6081E Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=okaya@codeaurora.org From: Sinan Kaya To: linux-pci@vger.kernel.org, timur@codeaurora.org, alex.williamson@redhat.com, vikrams@codeaurora.org Cc: Lorenzo.Pieralisi@arm.com, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Sinan Kaya , Bjorn Helgaas , linux-kernel@vger.kernel.org Subject: [PATCH V4] PCI: handle CRS returned by device after FLR Date: Thu, 6 Jul 2017 17:07:14 -0400 Message-Id: <1499375234-23928-1-git-send-email-okaya@codeaurora.org> X-Mailer: git-send-email 1.9.1 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP An endpoint is allowed to issue Configuration Request Retry Status (CRS) following a Function Level Reset (FLR) request to indicate that it is not ready to accept new requests. Seen a timeout message with Intel 750 NVMe drive and FLR reset. Kernel enables CRS visibility in pci_enable_crs() function for each bridge it discovers. The OS observes a special vendor ID read value of 0xFFFF0001 in this case. We need to keep polling until this special read value disappears. pci_bus_read_dev_vendor_id() takes care of CRS handling for a given vendor id read request under the covers. Adding a vendor ID read if this is a physical function before attempting to read any other registers on the endpoint. A CRS indication will only be given if the address to be read is vendor ID register. Note that virtual functions report their vendor ID through another mechanism. The spec is calling to wait up to 1 seconds if the device is sending CRS. The NVMe device seems to be requiring more. Relax this up to 60 seconds. Signed-off-by: Sinan Kaya --- drivers/pci/pci.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index aab9d51..83a9784 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -3723,10 +3723,16 @@ static void pci_flr_wait(struct pci_dev *dev) int i = 0; u32 id; - do { - msleep(100); - pci_read_config_dword(dev, PCI_COMMAND, &id); - } while (i++ < 10 && id == ~0); + if (dev->is_virtfn) { + do { + msleep(100); + pci_read_config_dword(dev, PCI_COMMAND, &id); + } while (i++ < 10 && id == ~0); + } else { + if (!pci_bus_read_dev_vendor_id(dev->bus, dev->devfn, &id, + 60*1000)) + id = ~0; + } if (id == ~0) dev_warn(&dev->dev, "Failed to return from FLR\n");