From patchwork Mon Mar 16 04:16:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11439693 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3D7F46CA for ; Mon, 16 Mar 2020 04:19:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 12EDB20719 for ; Mon, 16 Mar 2020 04:19:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729558AbgCPETS (ORCPT ); Mon, 16 Mar 2020 00:19:18 -0400 Received: from mga12.intel.com ([192.55.52.136]:63079 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728940AbgCPETS (ORCPT ); Mon, 16 Mar 2020 00:19:18 -0400 IronPort-SDR: s2F4LwM7v2ylNp6o6oVdiKWQENx7XC/SsOnXZvVdY6IIv+AvFWaHTAZCfBqWabGIMr7AeB8Vfp dSgwBOZwdCyQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2020 21:19:17 -0700 IronPort-SDR: knwCU92MwkqdUHJc+UxGYL+4QbrhInFR902EAsFZ1LDLjqFc77u6k+/TeGeZTJiL42FCZ3MVnd 7NtmRdEZ+t4w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,559,1574150400"; d="scan'208";a="417007228" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga005.jf.intel.com with ESMTP; 15 Mar 2020 21:19:15 -0700 From: Xu Yilun To: mdf@kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Xu Yilun , Luwei Kang , Wu Hao Subject: [PATCH v2 1/7] fpga: dfl: parse interrupt info for feature devices on enumeration Date: Mon, 16 Mar 2020 12:16:56 +0800 Message-Id: <1584332222-26652-2-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> References: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fpga@vger.kernel.org DFL based FPGA devices could support interrupts for different purposes, but current DFL framework only supports feature device enumeration with given MMIO resources information via common DFL headers. This patch introduces one new API dfl_fpga_enum_info_add_irq for low level bus drivers (e.g. PCIe device driver) to pass its interrupt resources information to DFL framework for enumeration, and also adds interrupt enumeration code in framework to parse and assign interrupt resources for enumerated feature devices and their own sub features. With this patch, DFL framework enumerates interrupt resources for core features, including PORT Error Reporting, FME (FPGA Management Engine) Error Reporting and also AFU User Interrupts. Signed-off-by: Luwei Kang Signed-off-by: Wu Hao Signed-off-by: Xu Yilun ---- v2: early validating irq table for each feature in parse_feature_irq(). Some code improvement and minor fix for Hao's comments. --- drivers/fpga/dfl.c | 152 +++++++++++++++++++++++++++++++++++++++++++++++++++-- drivers/fpga/dfl.h | 40 ++++++++++++++ 2 files changed, 188 insertions(+), 4 deletions(-) diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index 9909948..28e2cd8 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -11,6 +11,7 @@ * Xiao Guangrong */ #include +#include #include "dfl.h" @@ -421,6 +422,9 @@ EXPORT_SYMBOL_GPL(dfl_fpga_dev_ops_unregister); * * @dev: device to enumerate. * @cdev: the container device for all feature devices. + * @nr_irqs: number of irqs for all feature devices. + * @irq_table: Linux IRQ numbers for all irqs, indexed by local irq index of + * this device. * @feature_dev: current feature device. * @ioaddr: header register region address of feature device in enumeration. * @sub_features: a sub features linked list for feature device in enumeration. @@ -429,6 +433,9 @@ EXPORT_SYMBOL_GPL(dfl_fpga_dev_ops_unregister); struct build_feature_devs_info { struct device *dev; struct dfl_fpga_cdev *cdev; + unsigned int nr_irqs; + int *irq_table; + struct platform_device *feature_dev; void __iomem *ioaddr; struct list_head sub_features; @@ -442,12 +449,16 @@ struct build_feature_devs_info { * @mmio_res: mmio resource of this sub feature. * @ioaddr: mapped base address of mmio resource. * @node: node in sub_features linked list. + * @irq_base: start of irq index in this sub feature. + * @nr_irqs: number of irqs of this sub feature. */ struct dfl_feature_info { u64 fid; struct resource mmio_res; void __iomem *ioaddr; struct list_head node; + unsigned int irq_base; + unsigned int nr_irqs; }; static void dfl_fpga_cdev_add_port_dev(struct dfl_fpga_cdev *cdev, @@ -520,6 +531,8 @@ static int build_info_commit_dev(struct build_feature_devs_info *binfo) /* fill features and resource information for feature dev */ list_for_each_entry_safe(finfo, p, &binfo->sub_features, node) { struct dfl_feature *feature = &pdata->features[index]; + struct dfl_feature_irq_ctx *ctx; + int i; /* save resource information for each feature */ feature->id = finfo->fid; @@ -527,6 +540,20 @@ static int build_info_commit_dev(struct build_feature_devs_info *binfo) feature->ioaddr = finfo->ioaddr; fdev->resource[index++] = finfo->mmio_res; + if (finfo->nr_irqs) { + ctx = devm_kcalloc(binfo->dev, finfo->nr_irqs, + sizeof(*ctx), GFP_KERNEL); + if (!ctx) + return -ENOMEM; + + for (i = 0; i < finfo->nr_irqs; i++) + ctx[i].irq = + binfo->irq_table[finfo->irq_base + i]; + + feature->irq_ctx = ctx; + feature->nr_irqs = finfo->nr_irqs; + } + list_del(&finfo->node); kfree(finfo); } @@ -648,7 +675,8 @@ static u64 feature_id(void __iomem *start) static int create_feature_instance(struct build_feature_devs_info *binfo, struct dfl_fpga_enum_dfl *dfl, resource_size_t ofst, - resource_size_t size, u64 fid) + resource_size_t size, u64 fid, unsigned int irq_base, + unsigned int nr_irqs) { struct dfl_feature_info *finfo; @@ -667,6 +695,8 @@ create_feature_instance(struct build_feature_devs_info *binfo, finfo->mmio_res.start = dfl->start + ofst; finfo->mmio_res.end = finfo->mmio_res.start + size - 1; finfo->mmio_res.flags = IORESOURCE_MEM; + finfo->irq_base = irq_base; + finfo->nr_irqs = nr_irqs; finfo->ioaddr = dfl->ioaddr + ofst; list_add_tail(&finfo->node, &binfo->sub_features); @@ -684,7 +714,8 @@ static int parse_feature_port_afu(struct build_feature_devs_info *binfo, WARN_ON(!size); - return create_feature_instance(binfo, dfl, ofst, size, FEATURE_ID_AFU); + return create_feature_instance(binfo, dfl, ofst, size, FEATURE_ID_AFU, + 0, 0); } static int parse_feature_afu(struct build_feature_devs_info *binfo, @@ -724,7 +755,7 @@ static int parse_feature_fiu(struct build_feature_devs_info *binfo, if (ret) return ret; - ret = create_feature_instance(binfo, dfl, ofst, 0, 0); + ret = create_feature_instance(binfo, dfl, ofst, 0, 0, 0, 0); if (ret) return ret; /* @@ -742,17 +773,86 @@ static int parse_feature_fiu(struct build_feature_devs_info *binfo, return ret; } +static void parse_feature_irqs(struct build_feature_devs_info *binfo, + struct dfl_fpga_enum_dfl *dfl, + resource_size_t ofst, + unsigned int *irq_base, unsigned int *nr_irqs) +{ + unsigned int i; + u64 id, v; + int virq; + + /* + * Ideally DFL framework should only read info from DFL header, but + * current version DFL only provides mmio resources information for + * each feature in DFL Header, no field for interrupt resources. + * Some interrupt resources information are provided by specific + * mmio registers of each components(e.g. different private features) + * which supports interrupt. So in order to parse and assign irq + * resources to different components, DFL framework has to look into + * specific capability registers of these core private features. + * + * Once future DFL version supports generic interrupt resources + * information in common DFL headers, some generic interrupt parsing + * code could be added. But in order to be compatible to old version + * DFL, driver may still fall back to these quirks. + */ + + id = feature_id((dfl->ioaddr + ofst)); + + if (id == PORT_FEATURE_ID_UINT) { + v = readq(dfl->ioaddr + ofst + PORT_UINT_CAP); + *irq_base = FIELD_GET(PORT_UINT_CAP_FST_VECT, v); + *nr_irqs = FIELD_GET(PORT_UINT_CAP_INT_NUM, v); + } else if (id == PORT_FEATURE_ID_ERROR) { + v = readq(dfl->ioaddr + ofst + PORT_ERROR_CAP); + *irq_base = FIELD_GET(PORT_ERROR_CAP_INT_VECT, v); + *nr_irqs = FIELD_GET(PORT_ERROR_CAP_SUPP_INT, v); + } else if (id == FME_FEATURE_ID_GLOBAL_ERR) { + v = readq(dfl->ioaddr + ofst + FME_ERROR_CAP); + *irq_base = FIELD_GET(FME_ERROR_CAP_INT_VECT, v); + *nr_irqs = FIELD_GET(FME_ERROR_CAP_SUPP_INT, v); + } else { + return; + } + + dev_dbg(binfo->dev, "feature: 0x%llx, nr_irqs: %u, irq_base: %u\n", + (unsigned long long)id, *nr_irqs, *irq_base); + + if (*irq_base + *nr_irqs > binfo->nr_irqs) + goto parse_irq_fail; + + for (i = 0; i < *nr_irqs; i++) { + virq = binfo->irq_table[*irq_base + i]; + if (virq < 0 || virq > NR_IRQS) + goto parse_irq_fail; + } + + return; + +parse_irq_fail: + *irq_base = 0; + *nr_irqs = 0; + dev_warn(binfo->dev, "Invalid interrupt number in feature 0x%llx\n", + (unsigned long long)id); +} + static int parse_feature_private(struct build_feature_devs_info *binfo, struct dfl_fpga_enum_dfl *dfl, resource_size_t ofst) { + unsigned int irq_base = 0, nr_irqs = 0; + if (!binfo->feature_dev) { dev_err(binfo->dev, "the private feature %llx does not belong to any AFU.\n", (unsigned long long)feature_id(dfl->ioaddr + ofst)); return -EINVAL; } - return create_feature_instance(binfo, dfl, ofst, 0, 0); + parse_feature_irqs(binfo, dfl, ofst, &irq_base, &nr_irqs); + + return create_feature_instance(binfo, dfl, ofst, 0, 0, irq_base, + nr_irqs); } /** @@ -853,6 +953,10 @@ void dfl_fpga_enum_info_free(struct dfl_fpga_enum_info *info) devm_kfree(dev, dfl); } + /* remove irq table */ + if (info->irq_table) + devm_kfree(dev, info->irq_table); + devm_kfree(dev, info); put_device(dev); } @@ -892,6 +996,42 @@ int dfl_fpga_enum_info_add_dfl(struct dfl_fpga_enum_info *info, } EXPORT_SYMBOL_GPL(dfl_fpga_enum_info_add_dfl); +/** + * dfl_fpga_enum_info_add_irq - add irq table to enum info + * + * @info: ptr to dfl_fpga_enum_info + * @nr_irqs: number of irqs of the DFL fpga device to be enumerated. + * @irq_table: Linux IRQ numbers for all irqs, indexed by local irq index of + * this device. + * + * One FPGA device may have several interrupts. This function adds irq + * information of the DFL fpga device to enum info for next step enumeration. + * This function should be called before dfl_fpga_feature_devs_enumerate(). + * Adding multiply irq tables is not supported so it will fail on a second + * call. + * + * Return: 0 on success, negative error code otherwise. + */ +int dfl_fpga_enum_info_add_irq(struct dfl_fpga_enum_info *info, + unsigned int nr_irqs, int *irq_table) +{ + if (!nr_irqs) + return -EINVAL; + + if (info->irq_table) + return -EEXIST; + + info->irq_table = devm_kmemdup(info->dev, irq_table, + sizeof(int) * nr_irqs, GFP_KERNEL); + if (!info->irq_table) + return -ENOMEM; + + info->nr_irqs = nr_irqs; + + return 0; +} +EXPORT_SYMBOL_GPL(dfl_fpga_enum_info_add_irq); + static int remove_feature_dev(struct device *dev, void *data) { struct platform_device *pdev = to_platform_device(dev); @@ -959,6 +1099,10 @@ dfl_fpga_feature_devs_enumerate(struct dfl_fpga_enum_info *info) binfo->dev = info->dev; binfo->cdev = cdev; + binfo->nr_irqs = info->nr_irqs; + if (info->nr_irqs) + binfo->irq_table = info->irq_table; + /* * start enumeration for all feature devices based on Device Feature * Lists. diff --git a/drivers/fpga/dfl.h b/drivers/fpga/dfl.h index 4a9a33c..6a498cd 100644 --- a/drivers/fpga/dfl.h +++ b/drivers/fpga/dfl.h @@ -112,6 +112,13 @@ #define FME_PORT_OFST_ACC_VF 1 #define FME_PORT_OFST_IMP BIT_ULL(60) +/* FME Error Capability Register */ +#define FME_ERROR_CAP 0x70 + +/* FME Error Capability Register Bitfield */ +#define FME_ERROR_CAP_SUPP_INT BIT_ULL(0) /* Interrupt Support */ +#define FME_ERROR_CAP_INT_VECT GENMASK_ULL(12, 1) /* Interrupt vector */ + /* PORT Header Register Set */ #define PORT_HDR_DFH DFH #define PORT_HDR_GUID_L GUID_L @@ -145,6 +152,20 @@ #define PORT_STS_PWR_STATE_AP2 2 /* 90% throttling */ #define PORT_STS_PWR_STATE_AP6 6 /* 100% throttling */ +/* Port Error Capability Register */ +#define PORT_ERROR_CAP 0x38 + +/* Port Error Capability Register Bitfield */ +#define PORT_ERROR_CAP_SUPP_INT BIT_ULL(0) /* Interrupt Support */ +#define PORT_ERROR_CAP_INT_VECT GENMASK_ULL(12, 1) /* Interrupt vector */ + +/* Port Uint Capability Register */ +#define PORT_UINT_CAP 0x8 + +/* Port Uint Capability Register Bitfield */ +#define PORT_UINT_CAP_INT_NUM GENMASK_ULL(11, 0) /* Interrupts num */ +#define PORT_UINT_CAP_FST_VECT GENMASK_ULL(23, 12) /* First Vector */ + /** * struct dfl_fpga_port_ops - port ops * @@ -189,6 +210,15 @@ struct dfl_feature_driver { }; /** + * struct dfl_feature_irq_ctx - dfl private feature interrupt context + * + * @irq: Linux IRQ number of this interrupt. + */ +struct dfl_feature_irq_ctx { + int irq; +}; + +/** * struct dfl_feature - sub feature of the feature devices * * @id: sub feature id. @@ -196,12 +226,16 @@ struct dfl_feature_driver { * this index is used to find its mmio resource from the * feature dev (platform device)'s reources. * @ioaddr: mapped mmio resource address. + * @irq_ctx: interrupt context list. + * @nr_irqs: number of interrupt contexts. * @ops: ops of this sub feature. */ struct dfl_feature { u64 id; int resource_index; void __iomem *ioaddr; + struct dfl_feature_irq_ctx *irq_ctx; + unsigned int nr_irqs; const struct dfl_feature_ops *ops; }; @@ -388,10 +422,14 @@ static inline u8 dfl_feature_revision(void __iomem *base) * * @dev: parent device. * @dfls: list of device feature lists. + * @nr_irqs: number of irqs for all feature devices. + * @irq_table: Linux IRQ numbers for all irqs, indexed by hw irq numbers. */ struct dfl_fpga_enum_info { struct device *dev; struct list_head dfls; + unsigned int nr_irqs; + int *irq_table; }; /** @@ -415,6 +453,8 @@ struct dfl_fpga_enum_info *dfl_fpga_enum_info_alloc(struct device *dev); int dfl_fpga_enum_info_add_dfl(struct dfl_fpga_enum_info *info, resource_size_t start, resource_size_t len, void __iomem *ioaddr); +int dfl_fpga_enum_info_add_irq(struct dfl_fpga_enum_info *info, + unsigned int nr_irqs, int *irq_table); void dfl_fpga_enum_info_free(struct dfl_fpga_enum_info *info); /** From patchwork Mon Mar 16 04:16:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11439695 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8835A6CA for ; Mon, 16 Mar 2020 04:19:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 70FCF20738 for ; Mon, 16 Mar 2020 04:19:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729586AbgCPETV (ORCPT ); Mon, 16 Mar 2020 00:19:21 -0400 Received: from mga12.intel.com ([192.55.52.136]:63079 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728940AbgCPETV (ORCPT ); Mon, 16 Mar 2020 00:19:21 -0400 IronPort-SDR: 5A4rjyD6Dh70x28j3k9f01DFF6mCeJDL4G3Brxet31nHLnKcOIw122Mlu5U+mq3EclcEXBI/s9 31ZG7UKgdb6A== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2020 21:19:19 -0700 IronPort-SDR: hyeQeyd7hG16QZOg/F8qqhXU1KU2KvJ6cHwQyCHwH+n3DoyzMMOucpC1FKJY9wNecl+XdW5b7t rUYJnjSc/4Cg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,559,1574150400"; d="scan'208";a="417007232" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga005.jf.intel.com with ESMTP; 15 Mar 2020 21:19:18 -0700 From: Xu Yilun To: mdf@kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Xu Yilun , Luwei Kang , Wu Hao Subject: [PATCH v2 2/7] fpga: dfl: pci: add irq info for feature devices enumeration Date: Mon, 16 Mar 2020 12:16:57 +0800 Message-Id: <1584332222-26652-3-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> References: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fpga@vger.kernel.org Some DFL FPGA PCIe cards (e.g. Intel FPGA Programmable Acceleration Card) support MSI-X based interrupts. This patch allows PCIe driver to prepare and pass interrupt resources to DFL via enumeration API. These interrupt resources could then be assigned to actual features which use them. Signed-off-by: Luwei Kang Signed-off-by: Wu Hao Signed-off-by: Xu Yilun ---- v2: put irq resources init code inside cce_enumerate_feature_dev() Some minor changes for Hao's comments. --- drivers/fpga/dfl-pci.c | 78 ++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 69 insertions(+), 9 deletions(-) diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c index 5387550..0b1ee7d 100644 --- a/drivers/fpga/dfl-pci.c +++ b/drivers/fpga/dfl-pci.c @@ -39,6 +39,28 @@ static void __iomem *cci_pci_ioremap_bar(struct pci_dev *pcidev, int bar) return pcim_iomap_table(pcidev)[bar]; } +static int cci_pci_alloc_irq(struct pci_dev *pcidev) +{ + int nvec = pci_msix_vec_count(pcidev); + int ret; + + if (nvec <= 0) { + dev_dbg(&pcidev->dev, "fpga interrupt not supported\n"); + return 0; + } + + ret = pci_alloc_irq_vectors(pcidev, nvec, nvec, PCI_IRQ_MSIX); + if (ret < 0) + return ret; + + return nvec; +} + +static void cci_pci_free_irq(struct pci_dev *pcidev) +{ + pci_free_irq_vectors(pcidev); +} + /* PCI Device ID */ #define PCIE_DEVICE_ID_PF_INT_5_X 0xBCBD #define PCIE_DEVICE_ID_PF_INT_6_X 0xBCC0 @@ -78,17 +100,33 @@ static void cci_remove_feature_devs(struct pci_dev *pcidev) /* remove all children feature devices */ dfl_fpga_feature_devs_remove(drvdata->cdev); + cci_pci_free_irq(pcidev); +} + +static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) +{ + unsigned int i; + int *table; + + table = kcalloc(nvec, sizeof(int), GFP_KERNEL); + if (table) { + for (i = 0; i < nvec; i++) + table[i] = pci_irq_vector(pcidev, i); + } + + return table; } /* enumerate feature devices under pci device */ static int cci_enumerate_feature_devs(struct pci_dev *pcidev) { struct cci_drvdata *drvdata = pci_get_drvdata(pcidev); + int port_num, bar, i, nvec, ret = 0; struct dfl_fpga_enum_info *info; struct dfl_fpga_cdev *cdev; resource_size_t start, len; - int port_num, bar, i, ret = 0; void __iomem *base; + int *irq_table; u32 offset; u64 v; @@ -97,11 +135,32 @@ static int cci_enumerate_feature_devs(struct pci_dev *pcidev) if (!info) return -ENOMEM; + /* add irq info for enumeration if the device support irq */ + nvec = cci_pci_alloc_irq(pcidev); + if (nvec < 0) { + dev_err(&pcidev->dev, "Fail to alloc irq %d.\n", nvec); + ret = nvec; + goto enum_info_free_exit; + } + + if (nvec) { + irq_table = cci_pci_create_irq_table(pcidev, nvec); + if (!irq_table) { + ret = -ENOMEM; + goto error_free_irq; + } + + ret = dfl_fpga_enum_info_add_irq(info, nvec, irq_table); + kfree(irq_table); + if (ret) + goto error_free_irq; + } + /* start to find Device Feature List from Bar 0 */ base = cci_pci_ioremap_bar(pcidev, 0); if (!base) { ret = -ENOMEM; - goto enum_info_free_exit; + goto error_free_irq; } /* @@ -154,7 +213,7 @@ static int cci_enumerate_feature_devs(struct pci_dev *pcidev) dfl_fpga_enum_info_add_dfl(info, start, len, base); } else { ret = -ENODEV; - goto enum_info_free_exit; + goto error_free_irq; } /* start enumeration with prepared enumeration information */ @@ -162,11 +221,14 @@ static int cci_enumerate_feature_devs(struct pci_dev *pcidev) if (IS_ERR(cdev)) { dev_err(&pcidev->dev, "Enumeration failure\n"); ret = PTR_ERR(cdev); - goto enum_info_free_exit; + goto error_free_irq; } drvdata->cdev = cdev; +error_free_irq: + if (ret) + cci_pci_free_irq(pcidev); enum_info_free_exit: dfl_fpga_enum_info_free(info); @@ -211,12 +273,10 @@ int cci_pci_probe(struct pci_dev *pcidev, const struct pci_device_id *pcidevid) } ret = cci_enumerate_feature_devs(pcidev); - if (ret) { - dev_err(&pcidev->dev, "enumeration failure %d.\n", ret); - goto disable_error_report_exit; - } + if (!ret) + return ret; - return ret; + dev_err(&pcidev->dev, "enumeration failure %d.\n", ret); disable_error_report_exit: pci_disable_pcie_error_reporting(pcidev); From patchwork Mon Mar 16 04:16:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11439697 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 359A76CA for ; Mon, 16 Mar 2020 04:19:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1D66D20738 for ; Mon, 16 Mar 2020 04:19:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729630AbgCPETY (ORCPT ); Mon, 16 Mar 2020 00:19:24 -0400 Received: from mga12.intel.com ([192.55.52.136]:63079 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728940AbgCPETX (ORCPT ); Mon, 16 Mar 2020 00:19:23 -0400 IronPort-SDR: 5Mmncp6jJ1LqZTak2e6Utf175micLkreyk/tTJjkHT+4SQRJdVhkUzFDSzWRGLYI3V5kutoeBj Qv3gVFt2qx7g== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2020 21:19:22 -0700 IronPort-SDR: wgsTnc1titjg4FV8GMZN7owaSzOATWanEZ5O2JiDuD/cD6TgfrZs2AUNUw7Hi2hzHiuq5UXIMb DyB1p+rCFIgw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,559,1574150400"; d="scan'208";a="417007241" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga005.jf.intel.com with ESMTP; 15 Mar 2020 21:19:21 -0700 From: Xu Yilun To: mdf@kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Xu Yilun , Luwei Kang , Wu Hao Subject: [PATCH v2 3/7] fpga: dfl: introduce interrupt trigger setting API Date: Mon, 16 Mar 2020 12:16:58 +0800 Message-Id: <1584332222-26652-4-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> References: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fpga@vger.kernel.org FPGA user applications may be interested in interrupts generated by DFL features. For example, users can implement their own FPGA logics with interrupts enabled in AFU (Accelerated Function Unit, dynamic region of DFL based FPGA). So user applications need to be notified to handle these interrupts. In order to allow userspace applications to monitor interrupts, driver requires userspace to provide eventfds as interrupt notification channels. Applications then poll/select on the eventfds to get notified. This patch introduces a generic helper function for sub features to do eventfds binding with given interrupts. Signed-off-by: Luwei Kang Signed-off-by: Wu Hao Signed-off-by: Xu Yilun ---- v2: use unsigned int instead of int for irq array indexes in dfl_fpga_set_irq_triggers() Improves comments for NULL fds param in dfl_fpga_set_irq_triggers() --- drivers/fpga/dfl.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ drivers/fpga/dfl.h | 11 +++++++ 2 files changed, 106 insertions(+) diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index 28e2cd8..8dcc4e2 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -535,6 +535,7 @@ static int build_info_commit_dev(struct build_feature_devs_info *binfo) int i; /* save resource information for each feature */ + feature->dev = fdev; feature->id = finfo->fid; feature->resource_index = index; feature->ioaddr = finfo->ioaddr; @@ -1385,6 +1386,100 @@ int dfl_fpga_cdev_config_ports_vf(struct dfl_fpga_cdev *cdev, int num_vfs) } EXPORT_SYMBOL_GPL(dfl_fpga_cdev_config_ports_vf); +static irqreturn_t dfl_irq_handler(int irq, void *arg) +{ + struct eventfd_ctx *trigger = arg; + + eventfd_signal(trigger, 1); + return IRQ_HANDLED; +} + +static int do_set_irq_trigger(struct dfl_feature *feature, unsigned int idx, + int fd) +{ + struct platform_device *pdev = feature->dev; + struct eventfd_ctx *trigger; + int irq, ret; + + if (idx >= feature->nr_irqs) + return -EINVAL; + + irq = feature->irq_ctx[idx].irq; + + if (feature->irq_ctx[idx].trigger) { + free_irq(irq, feature->irq_ctx[idx].trigger); + kfree(feature->irq_ctx[idx].name); + eventfd_ctx_put(feature->irq_ctx[idx].trigger); + feature->irq_ctx[idx].trigger = NULL; + } + + if (fd < 0) + return 0; + + feature->irq_ctx[idx].name = + kasprintf(GFP_KERNEL, "fpga-irq[%u](%s-%llx)", idx, + dev_name(&pdev->dev), + (unsigned long long)feature->id); + if (!feature->irq_ctx[idx].name) + return -ENOMEM; + + trigger = eventfd_ctx_fdget(fd); + if (IS_ERR(trigger)) { + ret = PTR_ERR(trigger); + goto free_name; + } + + ret = request_irq(irq, dfl_irq_handler, 0, + feature->irq_ctx[idx].name, trigger); + if (!ret) { + feature->irq_ctx[idx].trigger = trigger; + return ret; + } + + eventfd_ctx_put(trigger); +free_name: + kfree(feature->irq_ctx[idx].name); + + return ret; +} + +/** + * dfl_fpga_set_irq_triggers - set eventfd triggers for dfl feature interrupts + * + * @feature: dfl sub feature. + * @start: start of irq index in this dfl sub feature. + * @count: number of irqs. + * @fds: eventfds to bind with irqs. + * + * Bind given eventfds with irqs in this dfl sub feature. Use NULL or negative + * fds as parameter to unbind irqs. + * + * Return: 0 on success, negative error code otherwise. + */ +int dfl_fpga_set_irq_triggers(struct dfl_feature *feature, unsigned int start, + unsigned int count, int32_t *fds) +{ + unsigned int i, j; + int ret = 0; + + if (start + count < start || start + count > feature->nr_irqs) + return -EINVAL; + + for (i = 0, j = start; i < count && !ret; i++, j++) { + int fd = fds ? fds[i] : -1; + + ret = do_set_irq_trigger(feature, j, fd); + } + + if (ret) { + for (--j; j >= start; j--) + do_set_irq_trigger(feature, j, -1); + } + + return ret; +} +EXPORT_SYMBOL_GPL(dfl_fpga_set_irq_triggers); + static void __exit dfl_fpga_exit(void) { dfl_chardev_uinit(); diff --git a/drivers/fpga/dfl.h b/drivers/fpga/dfl.h index 6a498cd..6b60077 100644 --- a/drivers/fpga/dfl.h +++ b/drivers/fpga/dfl.h @@ -24,6 +24,8 @@ #include #include #include +#include +#include /* maximum supported number of ports */ #define MAX_DFL_FPGA_PORT_NUM 4 @@ -213,14 +215,19 @@ struct dfl_feature_driver { * struct dfl_feature_irq_ctx - dfl private feature interrupt context * * @irq: Linux IRQ number of this interrupt. + * @trigger: eventfd context to signal when interrupt happens. + * @name: irq name needed when requesting irq. */ struct dfl_feature_irq_ctx { int irq; + struct eventfd_ctx *trigger; + char *name; }; /** * struct dfl_feature - sub feature of the feature devices * + * @dev: ptr to pdev of the feature device which has the sub feature. * @id: sub feature id. * @resource_index: each sub feature has one mmio resource for its registers. * this index is used to find its mmio resource from the @@ -231,6 +238,7 @@ struct dfl_feature_irq_ctx { * @ops: ops of this sub feature. */ struct dfl_feature { + struct platform_device *dev; u64 id; int resource_index; void __iomem *ioaddr; @@ -506,4 +514,7 @@ int dfl_fpga_cdev_release_port(struct dfl_fpga_cdev *cdev, int port_id); int dfl_fpga_cdev_assign_port(struct dfl_fpga_cdev *cdev, int port_id); void dfl_fpga_cdev_config_ports_pf(struct dfl_fpga_cdev *cdev); int dfl_fpga_cdev_config_ports_vf(struct dfl_fpga_cdev *cdev, int num_vf); + +int dfl_fpga_set_irq_triggers(struct dfl_feature *feature, unsigned int start, + unsigned int count, int32_t *fds); #endif /* __FPGA_DFL_H */ From patchwork Mon Mar 16 04:16:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11439705 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CD476CA for ; Mon, 16 Mar 2020 04:19:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3C61A20739 for ; Mon, 16 Mar 2020 04:19:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728940AbgCPET2 (ORCPT ); Mon, 16 Mar 2020 00:19:28 -0400 Received: from mga12.intel.com ([192.55.52.136]:63079 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729642AbgCPET0 (ORCPT ); Mon, 16 Mar 2020 00:19:26 -0400 IronPort-SDR: /vkzcVnen1xGjZbiGT1KSdZVNxAU0rDyMFp6PHM9Wyh46/DjNtPCeC9gv2ABMeU6Pdm4VjZhSY jKramvQ6Kybw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2020 21:19:25 -0700 IronPort-SDR: 6lXW79JZt9PXBfgnIS1y8LAQntAO/Oo5k4ZZDg8dBP+YafvjmHAbBaJrzXnZfXzXbXnf6XwVbm b4M/PyovPG8w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,559,1574150400"; d="scan'208";a="417007249" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga005.jf.intel.com with ESMTP; 15 Mar 2020 21:19:24 -0700 From: Xu Yilun To: mdf@kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Xu Yilun , Luwei Kang , Wu Hao Subject: [PATCH v2 4/7] fpga: dfl: afu: add interrupt support for error reporting Date: Mon, 16 Mar 2020 12:16:59 +0800 Message-Id: <1584332222-26652-5-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> References: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fpga@vger.kernel.org Error reporting interrupt is very useful to notify users that some errors are detected by the hardware. Once users are notified, they could query hardware logged error states, no need to continuously poll on these states. This patch follows the common DFL interrupt notification and handling mechanism, implements two ioctl commands below for user to query number of irqs supported, and set/unset interrupt triggers. Ioctls: * DFL_FPGA_PORT_ERR_GET_IRQ_NUM get the number of irqs, which is used to determine whether/how many interrupts error reporting feature supports. * DFL_FPGA_PORT_ERR_SET_IRQ set/unset given eventfds as error interrupt triggers. Signed-off-by: Luwei Kang Signed-off-by: Wu Hao Signed-off-by: Xu Yilun ---- v2: use DFL_FPGA_PORT_ERR_GET_IRQ_NUM instead of DFL_FPGA_PORT_ERR_GET_INFO Delete flag field for DFL_FPGA_PORT_ERR_SET_IRQ param --- drivers/fpga/dfl-afu-error.c | 64 +++++++++++++++++++++++++++++++++++++++++++ drivers/fpga/dfl-afu-main.c | 4 +++ include/uapi/linux/fpga-dfl.h | 29 ++++++++++++++++++++ 3 files changed, 97 insertions(+) diff --git a/drivers/fpga/dfl-afu-error.c b/drivers/fpga/dfl-afu-error.c index c1467ae..4d478b2f 100644 --- a/drivers/fpga/dfl-afu-error.c +++ b/drivers/fpga/dfl-afu-error.c @@ -15,6 +15,7 @@ */ #include +#include #include "dfl-afu.h" @@ -219,6 +220,68 @@ static void port_err_uinit(struct platform_device *pdev, afu_port_err_mask(&pdev->dev, true); } +static long +port_err_get_num_irqs(struct platform_device *pdev, + struct dfl_feature *feature, unsigned long arg) +{ + if (copy_to_user((void __user *)arg, &feature->nr_irqs, + sizeof(feature->nr_irqs))) + return -EFAULT; + + return 0; +} + +static long port_err_set_irq(struct platform_device *pdev, + struct dfl_feature *feature, unsigned long arg) +{ + struct dfl_feature_platform_data *pdata = dev_get_platdata(&pdev->dev); + struct dfl_fpga_irq_set hdr; + s32 *fds; + long ret; + + if (!feature->nr_irqs) + return -ENOENT; + + if (copy_from_user(&hdr, (void __user *)arg, sizeof(hdr))) + return -EFAULT; + + if (!hdr.count || (hdr.start + hdr.count > feature->nr_irqs) || + (hdr.start + hdr.count < hdr.start)) + return -EINVAL; + + fds = memdup_user((void __user *)(arg + sizeof(hdr)), + hdr.count * sizeof(s32)); + if (IS_ERR(fds)) + return PTR_ERR(fds); + + mutex_lock(&pdata->lock); + ret = dfl_fpga_set_irq_triggers(feature, hdr.start, hdr.count, fds); + mutex_unlock(&pdata->lock); + + kfree(fds); + return ret; +} + +static long +port_err_ioctl(struct platform_device *pdev, struct dfl_feature *feature, + unsigned int cmd, unsigned long arg) +{ + long ret = -ENODEV; + + switch (cmd) { + case DFL_FPGA_PORT_ERR_GET_IRQ_NUM: + ret = port_err_get_num_irqs(pdev, feature, arg); + break; + case DFL_FPGA_PORT_ERR_SET_IRQ: + ret = port_err_set_irq(pdev, feature, arg); + break; + default: + dev_dbg(&pdev->dev, "%x cmd not handled", cmd); + } + + return ret; +} + const struct dfl_feature_id port_err_id_table[] = { {.id = PORT_FEATURE_ID_ERROR,}, {0,} @@ -227,4 +290,5 @@ const struct dfl_feature_id port_err_id_table[] = { const struct dfl_feature_ops port_err_ops = { .init = port_err_init, .uinit = port_err_uinit, + .ioctl = port_err_ioctl, }; diff --git a/drivers/fpga/dfl-afu-main.c b/drivers/fpga/dfl-afu-main.c index 435bde4..fc8b9cf 100644 --- a/drivers/fpga/dfl-afu-main.c +++ b/drivers/fpga/dfl-afu-main.c @@ -577,6 +577,7 @@ static int afu_release(struct inode *inode, struct file *filp) { struct platform_device *pdev = filp->private_data; struct dfl_feature_platform_data *pdata; + struct dfl_feature *feature; dev_dbg(&pdev->dev, "Device File Release\n"); @@ -586,6 +587,9 @@ static int afu_release(struct inode *inode, struct file *filp) dfl_feature_dev_use_end(pdata); if (!dfl_feature_dev_use_count(pdata)) { + dfl_fpga_dev_for_each_feature(pdata, feature) + dfl_fpga_set_irq_triggers(feature, 0, + feature->nr_irqs, NULL); __port_reset(pdev); afu_dma_region_destroy(pdata); } diff --git a/include/uapi/linux/fpga-dfl.h b/include/uapi/linux/fpga-dfl.h index ec70a0746..ced859d 100644 --- a/include/uapi/linux/fpga-dfl.h +++ b/include/uapi/linux/fpga-dfl.h @@ -151,6 +151,35 @@ struct dfl_fpga_port_dma_unmap { #define DFL_FPGA_PORT_DMA_UNMAP _IO(DFL_FPGA_MAGIC, DFL_PORT_BASE + 4) +/** + * DFL_FPGA_PORT_ERR_GET_IRQ_NUM - _IOR(DFL_FPGA_MAGIC, DFL_PORT_BASE + 5, + * __u32 num_irqs) + * + * Get the number of irqs supported by the fpga port error reporting private + * feature. + * Return: 0 on success, -errno on failure. + */ +#define DFL_FPGA_PORT_ERR_GET_IRQ_NUM _IOR(DFL_FPGA_MAGIC, \ + DFL_PORT_BASE + 5, __u32) + +/** + * DFL_FPGA_PORT_ERR_SET_IRQ - _IOW(DFL_FPGA_MAGIC, DFL_PORT_BASE + 6, + * struct dfl_fpga_irq_set) + * + * Set fpga port error reporting interrupt trigger if evtfds[n] is valid. + * Unset related interrupt trigger if evtfds[n] is a NULL or negative value. + * Return: 0 on success, -errno on failure. + */ +struct dfl_fpga_irq_set { + __u32 start; /* First irq number */ + __u32 count; /* The number of eventfd handler */ + __s32 evtfds[]; /* Eventfd handler */ +}; + +#define DFL_FPGA_PORT_ERR_SET_IRQ _IOW(DFL_FPGA_MAGIC, \ + DFL_PORT_BASE + 6, \ + struct dfl_fpga_irq_set) + /* IOCTLs for FME file descriptor */ /** From patchwork Mon Mar 16 04:17:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11439701 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2590A139A for ; Mon, 16 Mar 2020 04:19:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 055C220737 for ; Mon, 16 Mar 2020 04:19:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729687AbgCPET3 (ORCPT ); Mon, 16 Mar 2020 00:19:29 -0400 Received: from mga12.intel.com ([192.55.52.136]:63079 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729682AbgCPET3 (ORCPT ); Mon, 16 Mar 2020 00:19:29 -0400 IronPort-SDR: 4fVhEMtIKZGYCEXzZLCqgaZ97AxyzOwrag/x+VeSrvY1praymw6ahez76JkT5PLdCrdfhiLDkL 2Be7xtrTf4ug== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2020 21:19:28 -0700 IronPort-SDR: 1FH1+zhjaLE7Ihare5/UA52RNJoxSB1JoownkOKf8O9Jrnm6CcV4WN93CLBpS7dUeJUCjWt/X0 1S2i79MVRPxg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,559,1574150400"; d="scan'208";a="417007254" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga005.jf.intel.com with ESMTP; 15 Mar 2020 21:19:26 -0700 From: Xu Yilun To: mdf@kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Xu Yilun , Luwei Kang , Wu Hao Subject: [PATCH v2 5/7] fpga: dfl: fme: add interrupt support for global error reporting Date: Mon, 16 Mar 2020 12:17:00 +0800 Message-Id: <1584332222-26652-6-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> References: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fpga@vger.kernel.org Error reporting interrupt is very useful to notify users that some errors are detected by the hardware. Once users are notified, they could query hardware logged error states, no need to continuously poll on these states. This patch follows the common DFL interrupt notification and handling mechanism. And it implements two ioctls below for user to query number of irqs supported, and set/unset interrupt triggers. Ioctls: * DFL_FPGA_FME_ERR_GET_IRQ_NUM get the number of irqs, which is used to determine whether/how many interrupts fme error reporting feature supports. * DFL_FPGA_FME_ERR_SET_IRQ set/unset given eventfds as fme error reporting interrupt triggers. Signed-off-by: Luwei Kang Signed-off-by: Wu Hao Signed-off-by: Xu Yilun ---- v2: use DFL_FPGA_FME_ERR_GET_IRQ_NUM instead of DFL_FPGA_FME_ERR_GET_INFO Delete flags field for DFL_FPGA_FME_ERR_SET_IRQ --- drivers/fpga/dfl-fme-error.c | 66 +++++++++++++++++++++++++++++++++++++++++++ drivers/fpga/dfl-fme-main.c | 6 ++++ include/uapi/linux/fpga-dfl.h | 23 +++++++++++++++ 3 files changed, 95 insertions(+) diff --git a/drivers/fpga/dfl-fme-error.c b/drivers/fpga/dfl-fme-error.c index f897d41..42853e6 100644 --- a/drivers/fpga/dfl-fme-error.c +++ b/drivers/fpga/dfl-fme-error.c @@ -16,6 +16,7 @@ */ #include +#include #include "dfl.h" #include "dfl-fme.h" @@ -348,6 +349,70 @@ static void fme_global_err_uinit(struct platform_device *pdev, fme_err_mask(&pdev->dev, true); } +static long +fme_global_err_get_num_irqs(struct platform_device *pdev, + struct dfl_feature *feature, unsigned long arg) +{ + if (copy_to_user((void __user *)arg, &feature->nr_irqs, + sizeof(feature->nr_irqs))) + return -EFAULT; + + return 0; +} + +static long +fme_global_err_set_irq(struct platform_device *pdev, + struct dfl_feature *feature, unsigned long arg) +{ + struct dfl_feature_platform_data *pdata = dev_get_platdata(&pdev->dev); + struct dfl_fpga_irq_set hdr; + s32 *fds; + long ret; + + if (!feature->nr_irqs) + return -ENOENT; + + if (copy_from_user(&hdr, (void __user *)arg, sizeof(hdr))) + return -EFAULT; + + if (!hdr.count || (hdr.start + hdr.count > feature->nr_irqs) || + (hdr.start + hdr.count < hdr.start)) + return -EINVAL; + + fds = memdup_user((void __user *)(arg + sizeof(hdr)), + hdr.count * sizeof(s32)); + if (IS_ERR(fds)) + return PTR_ERR(fds); + + mutex_lock(&pdata->lock); + ret = dfl_fpga_set_irq_triggers(feature, hdr.start, hdr.count, fds); + mutex_unlock(&pdata->lock); + + kfree(fds); + return ret; +} + +static long +fme_global_error_ioctl(struct platform_device *pdev, + struct dfl_feature *feature, + unsigned int cmd, unsigned long arg) +{ + long ret = -ENODEV; + + switch (cmd) { + case DFL_FPGA_FME_ERR_GET_IRQ_NUM: + ret = fme_global_err_get_num_irqs(pdev, feature, arg); + break; + case DFL_FPGA_FME_ERR_SET_IRQ: + ret = fme_global_err_set_irq(pdev, feature, arg); + break; + default: + dev_dbg(&pdev->dev, "%x cmd not handled", cmd); + } + + return ret; +} + const struct dfl_feature_id fme_global_err_id_table[] = { {.id = FME_FEATURE_ID_GLOBAL_ERR,}, {0,} @@ -356,4 +421,5 @@ const struct dfl_feature_id fme_global_err_id_table[] = { const struct dfl_feature_ops fme_global_err_ops = { .init = fme_global_err_init, .uinit = fme_global_err_uinit, + .ioctl = fme_global_error_ioctl, }; diff --git a/drivers/fpga/dfl-fme-main.c b/drivers/fpga/dfl-fme-main.c index 56d720c..ab3722d 100644 --- a/drivers/fpga/dfl-fme-main.c +++ b/drivers/fpga/dfl-fme-main.c @@ -616,11 +616,17 @@ static int fme_release(struct inode *inode, struct file *filp) { struct dfl_feature_platform_data *pdata = filp->private_data; struct platform_device *pdev = pdata->dev; + struct dfl_feature *feature; dev_dbg(&pdev->dev, "Device File Release\n"); mutex_lock(&pdata->lock); dfl_feature_dev_use_end(pdata); + + if (!dfl_feature_dev_use_count(pdata)) + dfl_fpga_dev_for_each_feature(pdata, feature) + dfl_fpga_set_irq_triggers(feature, 0, + feature->nr_irqs, NULL); mutex_unlock(&pdata->lock); return 0; diff --git a/include/uapi/linux/fpga-dfl.h b/include/uapi/linux/fpga-dfl.h index ced859d..dc8d370 100644 --- a/include/uapi/linux/fpga-dfl.h +++ b/include/uapi/linux/fpga-dfl.h @@ -223,4 +223,27 @@ struct dfl_fpga_fme_port_pr { */ #define DFL_FPGA_FME_PORT_ASSIGN _IOW(DFL_FPGA_MAGIC, DFL_FME_BASE + 2, int) +/** + * DFL_FPGA_FME_ERR_GET_IRQ_NUM - _IOR(DFL_FPGA_MAGIC, DFL_FME_BASE + 3, + * __u32 num_irqs) + * + * Get the number of irqs supported by the fpga fme error reporting private + * feature. + * Return: 0 on success, -errno on failure. + */ +#define DFL_FPGA_FME_ERR_GET_IRQ_NUM _IOR(DFL_FPGA_MAGIC, \ + DFL_FME_BASE + 3, __u32) + +/** + * DFL_FPGA_FME_ERR_SET_IRQ - _IOW(DFL_FPGA_MAGIC, DFL_FME_BASE + 4, + * struct dfl_fpga_irq_set) + * + * Set fpga fme error reporting interrupt trigger if evtfds[n] is valid. + * Unset related interrupt trigger if evtfds[n] is a NULL or negative value. + * Return: 0 on success, -errno on failure. + */ +#define DFL_FPGA_FME_ERR_SET_IRQ _IOW(DFL_FPGA_MAGIC, \ + DFL_FME_BASE + 4, \ + struct dfl_fpga_irq_set) + #endif /* _UAPI_LINUX_FPGA_DFL_H */ From patchwork Mon Mar 16 04:17:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11439699 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A0C46CA for ; Mon, 16 Mar 2020 04:19:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 190E920738 for ; Mon, 16 Mar 2020 04:19:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729735AbgCPETc (ORCPT ); Mon, 16 Mar 2020 00:19:32 -0400 Received: from mga12.intel.com ([192.55.52.136]:63079 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729682AbgCPETc (ORCPT ); Mon, 16 Mar 2020 00:19:32 -0400 IronPort-SDR: JBqGKjQMx8aXGoHkJMspc5mNz9NZ+KO+z0knNRg66+qRmWkHfh7sloiR6eOY9F5dDZZUJQEwRz M7HM0eYHu6Yg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2020 21:19:31 -0700 IronPort-SDR: zVBwDtWTIxTWJpwz/E/ujHhQzH7TXCFBQsxvhQsbEUmWYHn/WVG9sTKkRRX8zcE+cKHb4R/9gS 0Ze9X1AbodCw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,559,1574150400"; d="scan'208";a="417007264" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga005.jf.intel.com with ESMTP; 15 Mar 2020 21:19:29 -0700 From: Xu Yilun To: mdf@kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Xu Yilun , Luwei Kang , Wu Hao Subject: [PATCH v2 6/7] fpga: dfl: afu: add user interrupt support Date: Mon, 16 Mar 2020 12:17:01 +0800 Message-Id: <1584332222-26652-7-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> References: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fpga@vger.kernel.org AFU (Accelerated Function Unit) is dynamic region of the DFL based FPGA, and always defined by users. Some DFL based FPGA cards allow users to implement their own interrupts in AFU. In order to support this, hardware implements a new UINT (User Interrupt) private feature with related capability register which describes the number of supported user interrupts as well as the local index of the interrupts for software enumeration, and from software side, driver follows the common DFL interrupt notification and handling mechanism, and it implements two ioctls below for user to query number of irqs supported and set/unset interrupt triggers. Ioctls: * DFL_FPGA_PORT_UINT_GET_IRQ_NUM get the number of irqs, which is used to determine how many interrupts UINT feature supports. * DFL_FPGA_PORT_UINT_SET_IRQ set/unset eventfds as AFU user interrupt triggers. Signed-off-by: Luwei Kang Signed-off-by: Wu Hao Signed-off-by: Xu Yilun ---- v2: use DFL_FPGA_PORT_UINT_GET_IRQ_NUM instead of DFL_FPGA_PORT_UINT_GET_INFO Delete flags field for DFL_FPGA_PORT_UINT_SET_IRQ --- drivers/fpga/dfl-afu-main.c | 74 +++++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/fpga-dfl.h | 23 ++++++++++++++ 2 files changed, 97 insertions(+) diff --git a/drivers/fpga/dfl-afu-main.c b/drivers/fpga/dfl-afu-main.c index fc8b9cf..784817c 100644 --- a/drivers/fpga/dfl-afu-main.c +++ b/drivers/fpga/dfl-afu-main.c @@ -529,6 +529,76 @@ static const struct dfl_feature_ops port_stp_ops = { .init = port_stp_init, }; +static long +port_uint_get_num_irqs(struct platform_device *pdev, + struct dfl_feature *feature, unsigned long arg) +{ + if (copy_to_user((void __user *)arg, &feature->nr_irqs, + sizeof(feature->nr_irqs))) + return -EFAULT; + + return 0; +} + +static long port_uint_set_irq(struct platform_device *pdev, + struct dfl_feature *feature, unsigned long arg) +{ + struct dfl_feature_platform_data *pdata = dev_get_platdata(&pdev->dev); + struct dfl_fpga_irq_set hdr; + s32 *fds; + long ret; + + if (!feature->nr_irqs) + return -ENOENT; + + if (copy_from_user(&hdr, (void __user *)arg, sizeof(hdr))) + return -EFAULT; + + if (!hdr.count || (hdr.start + hdr.count > feature->nr_irqs) || + (hdr.start + hdr.count < hdr.start)) + return -EINVAL; + + fds = memdup_user((void __user *)(arg + sizeof(hdr)), + hdr.count * sizeof(s32)); + if (IS_ERR(fds)) + return PTR_ERR(fds); + + mutex_lock(&pdata->lock); + ret = dfl_fpga_set_irq_triggers(feature, hdr.start, hdr.count, fds); + mutex_unlock(&pdata->lock); + + kfree(fds); + return ret; +} + +static long +port_uint_ioctl(struct platform_device *pdev, struct dfl_feature *feature, + unsigned int cmd, unsigned long arg) +{ + long ret = -ENODEV; + + switch (cmd) { + case DFL_FPGA_PORT_UINT_GET_IRQ_NUM: + ret = port_uint_get_num_irqs(pdev, feature, arg); + break; + case DFL_FPGA_PORT_UINT_SET_IRQ: + ret = port_uint_set_irq(pdev, feature, arg); + break; + default: + dev_dbg(&pdev->dev, "%x cmd not handled", cmd); + } + return ret; +} + +static const struct dfl_feature_id port_uint_id_table[] = { + {.id = PORT_FEATURE_ID_UINT,}, + {0,} +}; + +static const struct dfl_feature_ops port_uint_ops = { + .ioctl = port_uint_ioctl, +}; + static struct dfl_feature_driver port_feature_drvs[] = { { .id_table = port_hdr_id_table, @@ -547,6 +617,10 @@ static struct dfl_feature_driver port_feature_drvs[] = { .ops = &port_stp_ops, }, { + .id_table = port_uint_id_table, + .ops = &port_uint_ops, + }, + { .ops = NULL, } }; diff --git a/include/uapi/linux/fpga-dfl.h b/include/uapi/linux/fpga-dfl.h index dc8d370..ca2dc6c 100644 --- a/include/uapi/linux/fpga-dfl.h +++ b/include/uapi/linux/fpga-dfl.h @@ -180,6 +180,29 @@ struct dfl_fpga_irq_set { DFL_PORT_BASE + 6, \ struct dfl_fpga_irq_set) +/** + * DFL_FPGA_PORT_UINT_GET_IRQ_NUM - _IOR(DFL_FPGA_MAGIC, DFL_PORT_BASE + 7, + * __u32 num_irqs) + * + * Get the number of irqs supported by the fpga AFU user interrupt private + * feature. + * Return: 0 on success, -errno on failure. + */ +#define DFL_FPGA_PORT_UINT_GET_IRQ_NUM _IOR(DFL_FPGA_MAGIC, \ + DFL_PORT_BASE + 7, __u32) + +/** + * DFL_FPGA_PORT_UINT_SET_IRQ - _IOW(DFL_FPGA_MAGIC, DFL_PORT_BASE + 8, + * struct dfl_fpga_irq_set) + * + * Set fpga afu user interrupt trigger if evtfds[n] is valid. + * Unset related interrupt trigger if evtfds[n] is a NULL or negative value. + * Return: 0 on success, -errno on failure. + */ +#define DFL_FPGA_PORT_UINT_SET_IRQ _IOW(DFL_FPGA_MAGIC, \ + DFL_PORT_BASE + 8, \ + struct dfl_fpga_irq_set) + /* IOCTLs for FME file descriptor */ /** From patchwork Mon Mar 16 04:17:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11439703 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 447D517E6 for ; Mon, 16 Mar 2020 04:19:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2DDC820737 for ; Mon, 16 Mar 2020 04:19:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729585AbgCPETf (ORCPT ); Mon, 16 Mar 2020 00:19:35 -0400 Received: from mga12.intel.com ([192.55.52.136]:63079 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729682AbgCPETe (ORCPT ); Mon, 16 Mar 2020 00:19:34 -0400 IronPort-SDR: Gf8T2cdm+3G4PAJJDr6jtdcWYYFF7ll42jTpnKtNV/iR276ILOZ1P0Izw3dXXrGJM68OmvEfmK TIDy4RMNQz3A== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2020 21:19:34 -0700 IronPort-SDR: xlKcmrYga1mEeBpU3mPgoXH77ySE1Oqbga2hyP3X8cfDs0u7aj7d9pJoVKDgN4Y9GTgk0OhwcZ +OOhabbkxmPg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,559,1574150400"; d="scan'208";a="417007273" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga005.jf.intel.com with ESMTP; 15 Mar 2020 21:19:32 -0700 From: Xu Yilun To: mdf@kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Xu Yilun , Luwei Kang , Wu Hao Subject: [PATCH v2 7/7] Documentation: fpga: dfl: add descriptions for interrupt related interfaces. Date: Mon, 16 Mar 2020 12:17:02 +0800 Message-Id: <1584332222-26652-8-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> References: <1584332222-26652-1-git-send-email-yilun.xu@intel.com> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fpga@vger.kernel.org This patch adds introductions of interrupt related interfaces for FME error reporting, port error reporting and AFU user interrupts features. Signed-off-by: Luwei Kang Signed-off-by: Wu Hao Signed-off-by: Xu Yilun ---- v2: Update Documents cause change of irq ioctl interfaces. --- Documentation/fpga/dfl.rst | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/Documentation/fpga/dfl.rst b/Documentation/fpga/dfl.rst index 094fc8a..c7ed3e4 100644 --- a/Documentation/fpga/dfl.rst +++ b/Documentation/fpga/dfl.rst @@ -89,6 +89,8 @@ The following functions are exposed through ioctls: - Program bitstream (DFL_FPGA_FME_PORT_PR) - Assign port to PF (DFL_FPGA_FME_PORT_ASSIGN) - Release port from PF (DFL_FPGA_FME_PORT_RELEASE) +- Get number of irqs of FME global error (DFL_FPGA_FME_ERR_GET_IRQ_NUM) +- Set interrupt trigger for FME error (DFL_FPGA_FME_ERR_SET_IRQ) More functions are exposed through sysfs (/sys/class/fpga_region/regionX/dfl-fme.n/): @@ -144,6 +146,10 @@ The following functions are exposed through ioctls: - Map DMA buffer (DFL_FPGA_PORT_DMA_MAP) - Unmap DMA buffer (DFL_FPGA_PORT_DMA_UNMAP) - Reset AFU (DFL_FPGA_PORT_RESET) +- Get number of irqs of port error (DFL_FPGA_PORT_ERR_GET_IRQ_NUM) +- Set interrupt trigger for port error (DFL_FPGA_PORT_ERR_SET_IRQ) +- Get number of irqs of UINT (DFL_FPGA_PORT_UINT_GET_IRQ_NUM) +- Set interrupt trigger for UINT (DFL_FPGA_PORT_UINT_SET_IRQ) DFL_FPGA_PORT_RESET: reset the FPGA Port and its AFU. Userspace can do Port @@ -378,6 +384,17 @@ The device nodes used for ioctl() or mmap() can be referenced through:: /sys/class/fpga_region///dev +Interrupt support +================= +Some FME and AFU private features are able to generate interrupts. As mentioned +above, users could call ioctl (DFL_FPGA_*_GET_IRQ_NUM) to know whether or how +many interrupts are supported for this private feature. Drivers also implement +an eventfd based interrupt handling mechanism for users to get notified when +interrupt happens. Users could set eventfds to driver via +ioctl (DFL_FPGA_*_SET_IRQ), and then poll/select on these eventfds waiting for +notification. + + Add new FIUs support ==================== It's possible that developers made some new function blocks (FIUs) under this