From patchwork Mon Mar 25 03:07:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Hao" X-Patchwork-Id: 10867865 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 642A715AC for ; Mon, 25 Mar 2019 03:25:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E7C8291CD for ; Mon, 25 Mar 2019 03:25:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3F9B629027; Mon, 25 Mar 2019 03:25:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C00F929027 for ; Mon, 25 Mar 2019 03:25:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729466AbfCYDZI (ORCPT ); Sun, 24 Mar 2019 23:25:08 -0400 Received: from mga18.intel.com ([134.134.136.126]:19152 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729548AbfCYDYB (ORCPT ); Sun, 24 Mar 2019 23:24:01 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Mar 2019 20:24:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,256,1549958400"; d="scan'208";a="310109657" Received: from hao-dev.bj.intel.com ([10.238.157.65]) by orsmga005.jf.intel.com with ESMTP; 24 Mar 2019 20:23:58 -0700 From: Wu Hao To: atull@kernel.org, mdf@kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Cc: linux-api@vger.kernel.org, Wu Hao , Zhang Yi Z , Xu Yilun Subject: [PATCH 06/17] fpga: dfl: pci: enable SRIOV support. Date: Mon, 25 Mar 2019 11:07:33 +0800 Message-Id: <1553483264-5379-7-git-send-email-hao.wu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1553483264-5379-1-git-send-email-hao.wu@intel.com> References: <1553483264-5379-1-git-send-email-hao.wu@intel.com> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fpga@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch enables the standard sriov support. It allows user to enable SRIOV (and VFs), then user could pass through accelerators (VFs) into virtual machine or use VFs directly in host. Signed-off-by: Zhang Yi Z Signed-off-by: Xu Yilun Signed-off-by: Wu Hao Acked-by: Alan Tull --- drivers/fpga/dfl-pci.c | 40 ++++++++++++++++++++++++++++++++++++++++ drivers/fpga/dfl.c | 41 +++++++++++++++++++++++++++++++++++++++++ drivers/fpga/dfl.h | 1 + 3 files changed, 82 insertions(+) diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c index 66b5720..2fa571b 100644 --- a/drivers/fpga/dfl-pci.c +++ b/drivers/fpga/dfl-pci.c @@ -223,8 +223,46 @@ int cci_pci_probe(struct pci_dev *pcidev, const struct pci_device_id *pcidevid) return ret; } +static int cci_pci_sriov_configure(struct pci_dev *pcidev, int num_vfs) +{ + struct cci_drvdata *drvdata = pci_get_drvdata(pcidev); + struct dfl_fpga_cdev *cdev = drvdata->cdev; + int ret = 0; + + mutex_lock(&cdev->lock); + + if (!num_vfs) { + /* + * disable SRIOV and then put released ports back to default + * PF access mode. + */ + pci_disable_sriov(pcidev); + + __dfl_fpga_cdev_config_port_vf(cdev, false); + + } else if (cdev->released_port_num == num_vfs) { + /* + * only enable SRIOV if cdev has matched released ports, put + * released ports into VF access mode firstly. + */ + __dfl_fpga_cdev_config_port_vf(cdev, true); + + ret = pci_enable_sriov(pcidev, num_vfs); + if (ret) + __dfl_fpga_cdev_config_port_vf(cdev, false); + } else { + ret = -EINVAL; + } + + mutex_unlock(&cdev->lock); + return ret; +} + static void cci_pci_remove(struct pci_dev *pcidev) { + if (dev_is_pf(&pcidev->dev)) + cci_pci_sriov_configure(pcidev, 0); + cci_remove_feature_devs(pcidev); pci_disable_pcie_error_reporting(pcidev); } @@ -234,6 +272,7 @@ static struct pci_driver cci_pci_driver = { .id_table = cci_pcie_id_tbl, .probe = cci_pci_probe, .remove = cci_pci_remove, + .sriov_configure = cci_pci_sriov_configure, }; module_pci_driver(cci_pci_driver); @@ -241,3 +280,4 @@ module_pci_driver(cci_pci_driver); MODULE_DESCRIPTION("FPGA DFL PCIe Device Driver"); MODULE_AUTHOR("Intel Corporation"); MODULE_LICENSE("GPL v2"); +MODULE_VERSION(DRV_VERSION); diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index a6b6d38..c5aa287 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -1098,6 +1098,47 @@ int dfl_fpga_cdev_config_port(struct dfl_fpga_cdev *cdev, } EXPORT_SYMBOL_GPL(dfl_fpga_cdev_config_port); +static void config_port_vf(struct device *fme_dev, int port_id, bool is_vf) +{ + void __iomem *base; + u64 v; + + base = dfl_get_feature_ioaddr_by_id(fme_dev, FME_FEATURE_ID_HEADER); + + v = readq(base + FME_HDR_PORT_OFST(port_id)); + + v &= ~FME_PORT_OFST_ACC_CTRL; + v |= FIELD_PREP(FME_PORT_OFST_ACC_CTRL, + is_vf ? FME_PORT_OFST_ACC_VF : FME_PORT_OFST_ACC_PF); + + writeq(v, base + FME_HDR_PORT_OFST(port_id)); +} + +/** + * __dfl_fpga_cdev_config_port_vf - configure port to VF access mode + * + * @cdev: parent container device. + * @if_vf: true for VF access mode, and false for PF access mode + * + * Return: 0 on success, negative error code otherwise. + * + * This function is needed in sriov configuration routine. It could be used to + * configures the released ports access mode to VF or PF. + * The caller needs to hold lock for protection. + */ +void __dfl_fpga_cdev_config_port_vf(struct dfl_fpga_cdev *cdev, bool is_vf) +{ + struct dfl_feature_platform_data *pdata; + + list_for_each_entry(pdata, &cdev->port_dev_list, node) { + if (device_is_registered(&pdata->dev->dev)) + continue; + + config_port_vf(cdev->fme_dev, pdata->id, is_vf); + } +} +EXPORT_SYMBOL_GPL(__dfl_fpga_cdev_config_port_vf); + static int __init dfl_fpga_init(void) { int ret; diff --git a/drivers/fpga/dfl.h b/drivers/fpga/dfl.h index 63f39ab..1350e8e 100644 --- a/drivers/fpga/dfl.h +++ b/drivers/fpga/dfl.h @@ -421,5 +421,6 @@ dfl_fpga_cdev_find_port(struct dfl_fpga_cdev *cdev, void *data, int dfl_fpga_cdev_config_port(struct dfl_fpga_cdev *cdev, u32 port_id, bool release); +void __dfl_fpga_cdev_config_port_vf(struct dfl_fpga_cdev *cdev, bool is_vf); #endif /* __FPGA_DFL_H */