| Message ID | 1486634439-7892-1-git-send-email-agust@denx.de (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Headers | show |
Hi Anatolij Gustschin, I've been working on a driver for what I think is the same PCIe card with an Arria10 FPGA on it, but the design on the card has a different pci product id than yours. The difference in product id may be related to a different design in the fpga. I am using an Altera A10 PCIe DevKit, and the product id I am targeting is 5052 compared to the product id of e005 supported by your driver. Regardless of the pci product id, it looks like we are using the same Partial Reconfiguration IP Core. I plan on trying this code tomorrow with my card. Given our apparent, similar development, I'm looking forward to constructive discussions. I want to begin with there is a lot of great work in this patch. You've done an excellent job creating a driver for the PR IP Core that plugs into Alan Tull's FPGA Manager framework. You've also done a great job at a very concise PCIe driver that focuses on just supporting reconfiguring the fpga over pcie. I think a PCIe driver that only is used for Partial Reconfiguration can be very useful. As it turns out, the PR IP Core can be instantiated in any Arria10 FPGA, and we are currently supporting designs using the PR IP core that are not connected to a PCIe bus. For this reason, I have a version of code for the PR IP that is separated into two parts. The first part is considered the base code that accesses the IP core after its registers have been mapped. This code is essentially the set of functions registered to the fpga-mgr. Your PCIe portion of this patch could call the common code. I then have a separate file of code used for a platform_driver implementation of the PR IP. Should I go ahead a post a patch of this separated PR IP driver for further discussion? There are also changes to the PR IP code being released in acds/17.0 that the driver will need to address. I think your patch also brings up a very interesting issue. Semantically, by writing the rbf filename to /sys/class/fpga_maganger/fpgaN/firmware, you are configuring what is connected to that fpga manager. However, a single fpga manager can individually reconfigure more than one piece of an fpga. Conversely, one could have a single fpga manager per piece of fpga that can be reconfigured. In addition, reconfiguring a piece of an FPGA involves more than a fpga manager. Best practices of (re)configuring a fpga involves disabling bridges around the area being configured before actually performing the configuration. Once configuration is complete the bridges are reenabled. The bridges prevent random signals getting out during the configuration process. Alan Tull has used the term fpga-region to describe a piece of fpga that can be (re)configured. A fpga-region is defined by the fpga-mgr that controls region and the set of bridges surrounding the configurable region. The set of bridges surrounding the region could be empty, but that is not recommended. A fpga region could be the entire fpga, or a single fpga can have many regions that can be individually reconfigured without affecting the other regions. So semantically, one does not reconfigure a fpga manager, but one does reconfigure a fpga region. Alan and I have been reconfiguring fpga regions with device tree overlays on SOCFPGAs for quite some time and are recently looking to bring this notion to fpgas connected to PCIe buses. It may be that your patch will accelerate this work. Looking forward to further conversations on this topic, Matthew Gerlach On Thu, 9 Feb 2017, Anatolij Gustschin wrote: > Partial reconfiguration (PR) allows to reconfigure part of the design's > core logic while the remainder of the design continues running. This > driver loads a new design (RBF file under /lib/firmware) over PCIe. > Writing new rbf file name to /sys/class/fpga_manager/fpgaN/firmware > interface triggers FPGA reconfiguration. > > Signed-off-by: Anatolij Gustschin <agust@denx.de> > --- > drivers/fpga/Kconfig | 7 ++ > drivers/fpga/Makefile | 1 + > drivers/fpga/a10-pcie-pr.c | 299 +++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 307 insertions(+) > create mode 100644 drivers/fpga/a10-pcie-pr.c > > diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig > index ce861a2..aa50fa4 100644 > --- a/drivers/fpga/Kconfig > +++ b/drivers/fpga/Kconfig > @@ -20,6 +20,13 @@ config FPGA_REGION > FPGA Regions allow loading FPGA images under control of > the Device Tree. > > +config FPGA_MGR_A10_PCIE_PR > + tristate "Arria 10 partial reconfiguration over PCIe PR IP" > + depends on PCI > + help > + FPGA manager driver support for Arria 10 partial > + reconfiguration over PCIe > + > config FPGA_MGR_SOCFPGA > tristate "Altera SOCFPGA FPGA Manager" > depends on ARCH_SOCFPGA || COMPILE_TEST > diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile > index 8df07bc..b90b3f5 100644 > --- a/drivers/fpga/Makefile > +++ b/drivers/fpga/Makefile > @@ -6,6 +6,7 @@ > obj-$(CONFIG_FPGA) += fpga-mgr.o > > # FPGA Manager Drivers > +obj-$(CONFIG_FPGA_MGR_A10_PCIE_PR) += a10-pcie-pr.o > obj-$(CONFIG_FPGA_MGR_SOCFPGA) += socfpga.o > obj-$(CONFIG_FPGA_MGR_SOCFPGA_A10) += socfpga-a10.o > obj-$(CONFIG_FPGA_MGR_ZYNQ_FPGA) += zynq-fpga.o > diff --git a/drivers/fpga/a10-pcie-pr.c b/drivers/fpga/a10-pcie-pr.c > new file mode 100644 > index 0000000..09de181 > --- /dev/null > +++ b/drivers/fpga/a10-pcie-pr.c > @@ -0,0 +1,299 @@ > +/* > + * FPGA Manager Driver for Arria 10 partial reconfiguration over PCIe. > + * Firmware must be in binary "rbf" format. > + * > + * Copyright (C) 2017 DENX Software Engineering > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation; version 2 of the License. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + */ > + > +#include <linux/delay.h> > +#include <linux/fpga/fpga-mgr.h> > +#include <linux/errno.h> > +#include <linux/kernel.h> > +#include <linux/limits.h> > +#include <linux/module.h> > +#include <linux/pci.h> > +#include <linux/types.h> > + > +#define DRV_NAME "a10-pcie-pr" > +#define PCIE_PR_MGR_NAME "PCIe PR Manager" > +#define DFLT_FIRMWARE_STR "a10_pr_region.rbf" > +#define PR_IP_BAR 4 /* PCIe PR IP core BAR */ > +#define PR_CONTROLLER_OFFSET 0xcfb0 > +#define CMD_PR 0x01 > +#define CMD_DOUBLE_PR 0x03 > + > +static char firmware[NAME_MAX]; > +module_param_string(firmware, firmware, sizeof(firmware), 0444); > +MODULE_PARM_DESC(firmware, " RBF file in /lib/firmware to load via PCIe PR"); > + > +/* use value 1 for debugging only */ > +static int chkcfg; > +module_param(chkcfg, int, 0664); > +MODULE_PARM_DESC(chkcfg, " 1 - check error status, 0 - skip check (default 0)"); > + > +/* PR IP core registers */ > +struct pr_regs { > + u32 pr_data; > + u32 pr_csr; > +}; > + > +struct pcie_pr_conf { > + struct fpga_manager *mgr; > + struct pci_dev *pci_dev; > + struct pr_regs *regs; > + void __iomem *map; > + u32 fw_size; > +}; > + > +static enum fpga_mgr_states pcie_pr_state(struct fpga_manager *mgr) > +{ > + return mgr->state; > +} > + > +static int pcie_pr_write_init(struct fpga_manager *mgr, > + struct fpga_image_info *info, > + const char *buf, size_t count) > +{ > + struct pcie_pr_conf *conf = mgr->priv; > + u32 status; > + > + if (info) > + dev_dbg(&mgr->dev, "%s(): flags 0x%x\n", __func__, info->flags); > + > + if (info && !(info->flags & FPGA_MGR_PARTIAL_RECONFIG)) { > + dev_err(&mgr->dev, "Only partial reconfiguration supported.\n"); > + return -EINVAL; > + } > + > + status = readl(&conf->regs->pr_csr); > + dev_dbg(&mgr->dev, "PR IP status 0x%08X\n", status); > + > + dev_dbg(&mgr->dev, "PR IP command 0x%08X\n", CMD_PR); > + writel(CMD_PR, &conf->regs->pr_csr); > + > + status = readl(&conf->regs->pr_csr); > + dev_dbg(&mgr->dev, "PR IP status: 0x%08X\n", (int) status); > + if ((status != 0x10) && (status != 0x0)) { > + dev_err(&mgr->dev, "status 0x%x\n", status); > + return -EFAULT; > + } > + > + return 0; > +} > + > +static int pcie_pr_write(struct fpga_manager *mgr, const char *buf, > + size_t count) > +{ > + struct pcie_pr_conf *conf = mgr->priv; > + const u32 *data, *data_end; > + size_t bytes = 0; > + int ret = 0; > + int status; > + > + dev_dbg(&mgr->dev, "%s: count %li\n", __func__, count); > + > + conf->fw_size = 0; > + data = (u32 *)buf; > + data_end = (u32 *)((char *)buf + count); > + > + while (data < data_end) { > + writel(*data++, &conf->regs->pr_data); > + bytes += 4; > + if (chkcfg) { > + status = readl(&conf->regs->pr_csr) >> 2; > + switch (status) { > + case 1 ... 3: > + dev_err(&mgr->dev, "PR error: 0x%x\n", status); > + ret = -EIO; > + break; > + } > + } > + } > + > + dev_dbg(&mgr->dev, "PR bytes written: %li\n", bytes); > + > + if (!ret) { > + /* > + * remember the loaded rbf image size to show this > + * info via sysfs firmware file when reading it > + */ > + conf->fw_size = count; > + } > + > + return ret; > +} > + > +static int pcie_pr_write_complete(struct fpga_manager *mgr, > + struct fpga_image_info *info) > +{ > + struct pcie_pr_conf *conf = mgr->priv; > + int status = 0; > + > + dev_dbg(&mgr->dev, "%s\n", __func__); > + > + status = readl(&conf->regs->pr_csr); > + if (status == 0x14) { > + dev_dbg(&mgr->dev, "PR done: 0x%08X\n", status); > + return 0; > + } > + > + dev_err(&mgr->dev, "PR error: 0x%08X\n", status); > + > + return -EIO; > +} > + > +static const struct fpga_manager_ops pcie_pr_ops = { > + .state = pcie_pr_state, > + .write_init = pcie_pr_write_init, > + .write = pcie_pr_write, > + .write_complete = pcie_pr_write_complete, > +}; > + > +static ssize_t show_firmware(struct device *dev, struct device_attribute *attr, > + char *buf) > +{ > + struct fpga_manager *mgr = to_fpga_manager(dev); > + struct pcie_pr_conf *conf = mgr->priv; > + > + return snprintf(buf, PAGE_SIZE, "%s %i\n", > + firmware, conf->fw_size); > +} > + > +static ssize_t store_firmware(struct device *dev, struct device_attribute *attr, > + const char *buf, size_t count) > +{ > + struct fpga_manager *mgr = to_fpga_manager(dev); > + int ret; > + > + if (!count || count > NAME_MAX) > + return -EINVAL; > + > + ret = sscanf(buf, "%s", firmware); > + if (ret != 1) > + return -EINVAL; > + > + ret = fpga_mgr_firmware_load(mgr, NULL, firmware); > + > + return ret ? ret : count; > +} > + > +static DEVICE_ATTR(firmware, 0664, show_firmware, store_firmware); > + > +static int pcie_pr_probe(struct pci_dev *pdev, > + const struct pci_device_id *dev_id) > +{ > + struct pcie_pr_conf *conf; > + int ret; > + > + dev_dbg(&pdev->dev, "%s\n", __func__); > + > + ret = pci_enable_device(pdev); > + if (ret < 0) { > + dev_err(&pdev->dev, "can't enable pci device: %d\n", ret); > + return ret; > + } > + > + pci_set_master(pdev); > + > + conf = devm_kzalloc(&pdev->dev, sizeof(*conf), GFP_KERNEL); > + if (!conf) { > + ret = -ENOMEM; > + goto err; > + } > + > + conf->pci_dev = pdev; > + > + if (pci_request_region(pdev, PR_IP_BAR, "PR IP") < 0) { > + dev_err(&pdev->dev, "Requesting PR BAR region failed\n"); > + ret = -ENODEV; > + goto err; > + } > + > + conf->map = pci_iomap(pdev, PR_IP_BAR, 0); > + if (!conf->map) { > + dev_warn(&pdev->dev, "Mapping PR BAR failed\n"); > + ret = -ENODEV; > + goto err_reg; > + } > + > + conf->regs = conf->map + PR_CONTROLLER_OFFSET; > + > + ret = fpga_mgr_register(&pdev->dev, PCIE_PR_MGR_NAME, > + &pcie_pr_ops, conf); > + if (ret) > + goto err_unmap; > + > + conf->mgr = fpga_mgr_get(&pdev->dev); > + if (IS_ERR(conf->mgr)) { > + dev_err(&pdev->dev, "Getting fpga-mgr reference failed\n"); > + ret = -ENODEV; > + goto err_mgr; > + } > + > + if (!strnlen(firmware, NAME_MAX)) > + strncpy(firmware, DFLT_FIRMWARE_STR, sizeof(firmware)); > + > + ret = fpga_mgr_firmware_load(conf->mgr, NULL, firmware); > + if (ret) > + dev_warn(&pdev->dev, "loading fpga image failed\n"); > + > + ret = device_create_file(&conf->mgr->dev, &dev_attr_firmware); > + if (ret) > + dev_warn(&pdev->dev, "can't create reconfig interface %d\n", > + ret); > + > + return 0; > + > +err_mgr: > + fpga_mgr_unregister(&pdev->dev); > +err_unmap: > + pci_iounmap(pdev, conf->map); > +err_reg: > + pci_release_region(pdev, PR_IP_BAR); > +err: > + pci_disable_device(pdev); > + return ret; > +} > + > +static void pcie_pr_remove(struct pci_dev *pdev) > +{ > + struct fpga_manager *mgr = pci_get_drvdata(pdev); > + struct pcie_pr_conf *conf = mgr->priv; > + > + fpga_mgr_put(conf->mgr); > + fpga_mgr_unregister(&pdev->dev); > + pci_iounmap(pdev, conf->map); > + pci_release_region(pdev, PR_IP_BAR); > + pci_disable_device(pdev); > +} > + > +#define PCI_VENDOR_ID_ALTERA 0x1172 > +#define PCI_DEVICE_ID_FPGA 0xe003 > + > +static struct pci_device_id pcie_pr_id_tbl[] = { > + { PCI_VDEVICE(ALTERA, PCI_DEVICE_ID_FPGA) }, > + { } > +}; > + > +static struct pci_driver pcie_pr_driver = { > + .name = DRV_NAME, > + .id_table = pcie_pr_id_tbl, > + .probe = pcie_pr_probe, > + .remove = pcie_pr_remove, > +}; > + > +module_pci_driver(pcie_pr_driver) > + > +MODULE_DEVICE_TABLE(pci, pcie_pr_id_tbl); > +MODULE_LICENSE("GPL"); > +MODULE_AUTHOR("Anatolij Gustschin <agust@denx.de>"); > +MODULE_DESCRIPTION("Arria 10 FPGA reconfiguration via PCIe PR IP"); > -- > 2.7.4 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-fpga" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Matthew, On Thu, 9 Feb 2017 18:55:18 -0800 (PST) matthew.gerlach@linux.intel.com matthew.gerlach@linux.intel.com wrote: >Hi Anatolij Gustschin, > >I've been working on a driver for what I think is the same >PCIe card with an Arria10 FPGA on it, but the design on the card has a >different pci product id than yours. The difference in product id may be >related to a different design in the fpga. I am using an Altera A10 PCIe >DevKit, and the product id I am targeting is 5052 compared to the product >id of e005 supported by your driver. Regardless of the pci product id, it >looks like we are using the same Partial Reconfiguration IP Core. I >plan on trying this code tomorrow with my card. Given our apparent, >similar development, I'm looking forward to constructive >discussions. I'm currently using Attila Instant-DevKit Arria 10 FPGA FMC IDK, with PR over PCIe Reference Design here [1]. We are interested in a mainline FPGA manager driver for A10 partial reconfiguration. It would be optimal for our use case if we could implement partial reconfiguration over PCIe with similar userspace interface on DT based platforms (ARM) and on x86 platform. >I want to begin with there is a lot of great work in this patch. You've >done an excellent job creating a driver for the PR IP Core that plugs into >Alan Tull's FPGA Manager framework. You've also done a great job at a >very concise PCIe driver that focuses on just supporting reconfiguring >the fpga over pcie. I think a PCIe driver that only is used for >Partial Reconfiguration can be very useful. > >As it turns out, the PR IP Core can be instantiated in any Arria10 FPGA, >and we are currently supporting designs using the PR IP core that are not >connected to a PCIe bus. For this reason, I have a version of code for >the PR IP that is separated into two parts. The first part is considered >the base code that accesses the IP core after its registers have been >mapped. This code is essentially the set of functions registered to the >fpga-mgr. Your PCIe portion of this patch could call the common code. >I then have a separate file of code used for a platform_driver implementation >of the PR IP. Should I go ahead a post a patch of this separated PR IP >driver for further discussion? There are also changes to the PR IP code >being released in acds/17.0 that the driver will need to address. Yes, posting the patch would help I think. Before I started the work on this patch I checked linux-socfpga tree on github (socfpga-4.1.22-ltsi branch and later socfpga-4.7 branch) but didn't find anything for partial reconfiguration. I could look at your code and need to figure out what must be changed in the driver so that PR IP over PCIe is also supported in mainline kernel. >I think your patch also brings up a very interesting issue. Semantically, >by writing the rbf filename to /sys/class/fpga_maganger/fpgaN/firmware, >you are configuring what is connected to that fpga manager. However, a >single fpga manager can individually reconfigure more than one piece of an >fpga. Conversely, one could have a single fpga manager per piece of >fpga that can be reconfigured. In addition, reconfiguring a piece of an >FPGA involves more than a fpga manager. Best practices of (re)configuring >a fpga involves disabling bridges around the area being configured before >actually performing the configuration. Once configuration is complete the >bridges are reenabled. The bridges prevent random signals getting out >during the configuration process. Is my understanding correct that by bridges you mean the interfaces between an fpga and host CPU and also interfaces between reconfigurable pieces of an fpga? Is disabling the bridges between reconfigurable pieces handled by PR IP automatically when requesting the reconfiguration? >Alan Tull has used the term fpga-region to describe a piece of fpga that >can be (re)configured. A fpga-region is defined by the fpga-mgr that >controls region and the set of bridges surrounding the configurable >region. The set of bridges surrounding the region could be empty, but >that is not recommended. A fpga region could be the entire fpga, or a >single fpga can have many regions that can be individually reconfigured >without affecting the other regions. So semantically, one does not >reconfigure a fpga manager, but one does reconfigure a fpga region. Alan and I >have been reconfiguring fpga regions with device tree overlays on SOCFPGAs >for quite some time and are recently looking to bring this notion to fpgas >connected to PCIe buses. It may be that your patch will accelerate this work. I'm also looking for possibilities to use device tree overlays with fpgas on PCIe buses. Additional issue here is that we normally do not use device tree (and overlays) on x86 platform. Was this topic already discussed on the mailing list? Best regards, Anatolij [1] https://github.com/01org/fpga-partial-reconfig/tree/master/ref_designs/a10_pcie_devkit_cvp -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Anatolij, On Fri, Feb 10, 2017 at 9:02 AM, Anatolij Gustschin <agust@denx.de> wrote: > I'm currently using Attila Instant-DevKit Arria 10 FPGA FMC IDK, with > PR over PCIe Reference Design here [1]. We are interested in a mainline > FPGA manager driver for A10 partial reconfiguration. It would be optimal > for our use case if we could implement partial reconfiguration over PCIe > with similar userspace interface on DT based platforms (ARM) and on x86 > platform. Well currently we don't really have a userspace interface for DT based platforms, either in mainline. The takeaway from discussions on that subject seemed that we need to come up with something along the lines of Pantelis' dt-connector concept, which I cannot make work for fpgas in my head. > Yes, posting the patch would help I think. Before I started the work > on this patch I checked linux-socfpga tree on github (socfpga-4.1.22-ltsi > branch and later socfpga-4.7 branch) but didn't find anything for > partial reconfiguration. I could look at your code and need to figure > out what must be changed in the driver so that PR IP over PCIe is > also supported in mainline kernel. Well you should be able to separate it out into a PR core (common) driver part, a platform driver and a pci driver that both implement a low level fpga manager driver. >>I think your patch also brings up a very interesting issue. Semantically, >>by writing the rbf filename to /sys/class/fpga_maganger/fpgaN/firmware, >>you are configuring what is connected to that fpga manager. However, a >>single fpga manager can individually reconfigure more than one piece of an >>fpga. Conversely, one could have a single fpga manager per piece of >>fpga that can be reconfigured. In addition, reconfiguring a piece of an >>FPGA involves more than a fpga manager. Best practices of (re)configuring >>a fpga involves disabling bridges around the area being configured before >>actually performing the configuration. Once configuration is complete the >>bridges are reenabled. The bridges prevent random signals getting out >>during the configuration process. It also completely bypasses the driver model, i.e. whatever happens inside of the fpga is unknown to the kernel. > Is my understanding correct that by bridges you mean the interfaces > between an fpga and host CPU and also interfaces between reconfigurable > pieces of an fpga? Is disabling the bridges between reconfigurable pieces > handled by PR IP automatically when requesting the reconfiguration? Yes to the first part. I don't know enough about that part of IP for whether that happens automatic, I'd guess probably not, though. > I'm also looking for possibilities to use device tree overlays with > fpgas on PCIe buses. Additional issue here is that we normally do not > use device tree (and overlays) on x86 platform. Was this topic already > discussed on the mailing list? We had a longer discussion at the Linux Plumbers conference. The overlap of people that know about ACPI based PCIe platforms and DT based ARM / other platforms seemed pretty small. Notes are here: https://etherpad.openstack.org/p/LPC2016_FPGA I'd be happy to discuss if people have ideas. I don't know who's working on what currently. Personally I've done a lot of thinking, but couldn't come to a good solution for the DT case, and don't know enough about ACPI / PCIe (yet). Cheers & thanks for contributing! Moritz -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Feb 10, 2017 at 11:19 AM, Moritz Fischer <moritz.fischer@ettus.com> wrote: > Hi Anatolij, > > On Fri, Feb 10, 2017 at 9:02 AM, Anatolij Gustschin <agust@denx.de> wrote: > >> I'm currently using Attila Instant-DevKit Arria 10 FPGA FMC IDK, with >> PR over PCIe Reference Design here [1]. We are interested in a mainline >> FPGA manager driver for A10 partial reconfiguration. It would be optimal >> for our use case if we could implement partial reconfiguration over PCIe >> with similar userspace interface on DT based platforms (ARM) and on x86 >> platform. > > Well currently we don't really have a userspace interface for DT based > platforms, > either in mainline. I've been using Pantelis Antoniou's configfs interface for applying DT overlays. It hasn't been accepted upstream yet, but you can get it from his git repo at: https://github.com/pantoniou/linux-beagle-track-mainline.git Patch name I'm currently using is "OF: DT-Overlay configfs interface (v7)". Basically the user can mkdir a directory under configfs and there are files under that directory that allow applying an overlay. > The takeaway from discussions on that subject seemed that we > need to come up with something along the lines of Pantelis' > dt-connector concept, > which I cannot make work for fpgas in my head. Yes, I like that idea, but haven't had time to work on it myself. > >> Yes, posting the patch would help I think. Before I started the work >> on this patch I checked linux-socfpga tree on github (socfpga-4.1.22-ltsi >> branch and later socfpga-4.7 branch) but didn't find anything for >> partial reconfiguration. I could look at your code and need to figure >> out what must be changed in the driver so that PR IP over PCIe is >> also supported in mainline kernel. > > Well you should be able to separate it out into a PR core (common) driver part, > a platform driver and a pci driver that both implement a low level fpga manager > driver. > >>>I think your patch also brings up a very interesting issue. Semantically, >>>by writing the rbf filename to /sys/class/fpga_maganger/fpgaN/firmware, >>>you are configuring what is connected to that fpga manager. However, a >>>single fpga manager can individually reconfigure more than one piece of an >>>fpga. Conversely, one could have a single fpga manager per piece of >>>fpga that can be reconfigured. In addition, reconfiguring a piece of an >>>FPGA involves more than a fpga manager. Best practices of (re)configuring >>>a fpga involves disabling bridges around the area being configured before >>>actually performing the configuration. Once configuration is complete the >>>bridges are reenabled. The bridges prevent random signals getting out >>>during the configuration process. > > It also completely bypasses the driver model, i.e. whatever happens inside of > the fpga is unknown to the kernel. Yes there also has to be some sort of enumeration after successfully programming. > >> Is my understanding correct that by bridges you mean the interfaces >> between an fpga and host CPU and also interfaces between reconfigurable >> pieces of an fpga? Is disabling the bridges between reconfigurable pieces >> handled by PR IP automatically when requesting the reconfiguration? > > Yes to the first part. I don't know enough about that part of IP for > whether that > happens automatic, I'd guess probably not, though. It's not automatic. That's why we need FPGA Region in the kernel to represent FPGA regions on the FPGA. Each kernel region knows which FPGA manager and which bridge to control. So telling a kernel FPGA region to reconfigure a region initializes the sequence where the bridge is disabled, the region is programmed, and the bridge is reenabled. For a discussion of DTO and FPGA regions, please read: Documentation/devicetree/bindings/fpga/fpga-region.txt The DT overlay support which is upstream (and also on the Altera github branches) has FPGA regions that are controlled by applying an overlay. I"m working on a patchset that breaks out the concept of regions from device tree so that we may be able to create a region by some mechanism other than DT. Hopefully on the list this weekend as a RFC. > >> I'm also looking for possibilities to use device tree overlays with >> fpgas on PCIe buses. Additional issue here is that we normally do not >> use device tree (and overlays) on x86 platform. Was this topic already >> discussed on the mailing list? > > We had a longer discussion at the Linux Plumbers conference. The > overlap of people > that know about ACPI based PCIe platforms and DT based ARM / other platforms > seemed pretty small. > > Notes are here: > https://etherpad.openstack.org/p/LPC2016_FPGA > > I'd be happy to discuss if people have ideas. I don't know who's > working on what currently. > Personally I've done a lot of thinking, but couldn't come to a good > solution for the DT case, > and don't know enough about ACPI / PCIe (yet). > > Cheers & thanks for contributing! > > Moritz -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Anatolij, On Fri, 10 Feb 2017, Anatolij Gustschin wrote: > Hi Matthew, > > On Thu, 9 Feb 2017 18:55:18 -0800 (PST) > matthew.gerlach@linux.intel.com matthew.gerlach@linux.intel.com wrote: > >> Hi Anatolij Gustschin, >> >> I've been working on a driver for what I think is the same >> PCIe card with an Arria10 FPGA on it, but the design on the card has a >> different pci product id than yours. The difference in product id may be >> related to a different design in the fpga. I am using an Altera A10 PCIe >> DevKit, and the product id I am targeting is 5052 compared to the product >> id of e005 supported by your driver. Regardless of the pci product id, it >> looks like we are using the same Partial Reconfiguration IP Core. I >> plan on trying this code tomorrow with my card. Given our apparent, >> similar development, I'm looking forward to constructive >> discussions. > > I'm currently using Attila Instant-DevKit Arria 10 FPGA FMC IDK, with > PR over PCIe Reference Design here [1]. We are interested in a mainline > FPGA manager driver for A10 partial reconfiguration. It would be optimal > for our use case if we could implement partial reconfiguration over PCIe > with similar userspace interface on DT based platforms (ARM) and on x86 > platform. I guess we are using different boards, but the base A10 PR over PCIe Reference Design is similar. I am working the owners of the design to update the design to include appropriate bridges around the PR region. I agree that it would be optimal to have a similar userspace interface for using FPGA with DT based platforms and x86. At the moment I am using a very rudimentary debugfs based interface to control PR. > >> I want to begin with there is a lot of great work in this patch. You've >> done an excellent job creating a driver for the PR IP Core that plugs into >> Alan Tull's FPGA Manager framework. You've also done a great job at a >> very concise PCIe driver that focuses on just supporting reconfiguring >> the fpga over pcie. I think a PCIe driver that only is used for >> Partial Reconfiguration can be very useful. >> >> As it turns out, the PR IP Core can be instantiated in any Arria10 FPGA, >> and we are currently supporting designs using the PR IP core that are not >> connected to a PCIe bus. For this reason, I have a version of code for >> the PR IP that is separated into two parts. The first part is considered >> the base code that accesses the IP core after its registers have been >> mapped. This code is essentially the set of functions registered to the >> fpga-mgr. Your PCIe portion of this patch could call the common code. >> I then have a separate file of code used for a platform_driver implementation >> of the PR IP. Should I go ahead a post a patch of this separated PR IP >> driver for further discussion? There are also changes to the PR IP code >> being released in acds/17.0 that the driver will need to address. > > Yes, posting the patch would help I think. Before I started the work > on this patch I checked linux-socfpga tree on github (socfpga-4.1.22-ltsi > branch and later socfpga-4.7 branch) but didn't find anything for > partial reconfiguration. I could look at your code and need to figure > out what must be changed in the driver so that PR IP over PCIe is > also supported in mainline kernel. I will post the patch and look forward to feedback. We are in alignment with the desire for PR IP over PCIe being suppported in the mainline kernel. > >> I think your patch also brings up a very interesting issue. Semantically, >> by writing the rbf filename to /sys/class/fpga_maganger/fpgaN/firmware, >> you are configuring what is connected to that fpga manager. However, a >> single fpga manager can individually reconfigure more than one piece of an >> fpga. Conversely, one could have a single fpga manager per piece of >> fpga that can be reconfigured. In addition, reconfiguring a piece of an >> FPGA involves more than a fpga manager. Best practices of (re)configuring >> a fpga involves disabling bridges around the area being configured before >> actually performing the configuration. Once configuration is complete the >> bridges are reenabled. The bridges prevent random signals getting out >> during the configuration process. > > Is my understanding correct that by bridges you mean the interfaces > between an fpga and host CPU and also interfaces between reconfigurable > pieces of an fpga? Is disabling the bridges between reconfigurable pieces > handled by PR IP automatically when requesting the reconfiguration? Your understanding is correct. I am referring to bridges between the fpga and the host CPU and also bridges between reconfigurable pieces of the fpga. With Altera/Intel FPGAs, the component that controls the bridges is a different component that configures the FPGA. The PR IP does not automatically disable bridges. > >> Alan Tull has used the term fpga-region to describe a piece of fpga that >> can be (re)configured. A fpga-region is defined by the fpga-mgr that >> controls region and the set of bridges surrounding the configurable >> region. The set of bridges surrounding the region could be empty, but >> that is not recommended. A fpga region could be the entire fpga, or a >> single fpga can have many regions that can be individually reconfigured >> without affecting the other regions. So semantically, one does not >> reconfigure a fpga manager, but one does reconfigure a fpga region. Alan and I >> have been reconfiguring fpga regions with device tree overlays on SOCFPGAs >> for quite some time and are recently looking to bring this notion to fpgas >> connected to PCIe buses. It may be that your patch will accelerate this work. > > I'm also looking for possibilities to use device tree overlays with > fpgas on PCIe buses. Additional issue here is that we normally do not > use device tree (and overlays) on x86 platform. Was this topic already > discussed on the mailing list? I don't think using device tree (and overlays) on x86 platform has been discussed too much on this mailing list. As Moritz pointed out, the topic was discussed at the last Linux Plumber's conference. At the 2015 Linux Plumber's conference, there was a talk of someone actually using device tree overlays with x86 based system, Juniper Networks I think, but I don't think the work was ever upstreamed. > > Best regards, > > Anatolij > > [1] https://github.com/01org/fpga-partial-reconfig/tree/master/ref_designs/a10_pcie_devkit_cvp > Matthew Gerlach -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, 10 Feb 2017, Alan Tull wrote: > On Fri, Feb 10, 2017 at 11:19 AM, Moritz Fischer > <moritz.fischer@ettus.com> wrote: >> Hi Anatolij, >> >> On Fri, Feb 10, 2017 at 9:02 AM, Anatolij Gustschin <agust@denx.de> wrote: >> >>> I'm currently using Attila Instant-DevKit Arria 10 FPGA FMC IDK, with >>> PR over PCIe Reference Design here [1]. We are interested in a mainline >>> FPGA manager driver for A10 partial reconfiguration. It would be optimal >>> for our use case if we could implement partial reconfiguration over PCIe >>> with similar userspace interface on DT based platforms (ARM) and on x86 >>> platform. >> >> Well currently we don't really have a userspace interface for DT based >> platforms, >> either in mainline. > > I've been using Pantelis Antoniou's configfs interface for applying DT > overlays. It hasn't been accepted upstream yet, but you can get it > from his git repo at: > > https://github.com/pantoniou/linux-beagle-track-mainline.git > > Patch name I'm currently using is "OF: DT-Overlay configfs interface (v7)". > > Basically the user can mkdir a directory under configfs and there are > files under that directory that allow applying an overlay. > This configfs interface also allows for removing of overlays, but one needs to be careful about the order of applying and removing overlays. >> The takeaway from discussions on that subject seemed that we >> need to come up with something along the lines of Pantelis' >> dt-connector concept, >> which I cannot make work for fpgas in my head. > > Yes, I like that idea, but haven't had time to work on it myself. > I have not had any time to work in this area much either. At the moment I'm using a debugfs interface, which is really for debugging and prototyping purposes. >> >>> Yes, posting the patch would help I think. Before I started the work >>> on this patch I checked linux-socfpga tree on github (socfpga-4.1.22-ltsi >>> branch and later socfpga-4.7 branch) but didn't find anything for >>> partial reconfiguration. I could look at your code and need to figure >>> out what must be changed in the driver so that PR IP over PCIe is >>> also supported in mainline kernel. >> >> Well you should be able to separate it out into a PR core (common) driver part, >> a platform driver and a pci driver that both implement a low level fpga manager >> driver. >> >>>> I think your patch also brings up a very interesting issue. Semantically, >>>> by writing the rbf filename to /sys/class/fpga_maganger/fpgaN/firmware, >>>> you are configuring what is connected to that fpga manager. However, a >>>> single fpga manager can individually reconfigure more than one piece of an >>>> fpga. Conversely, one could have a single fpga manager per piece of >>>> fpga that can be reconfigured. In addition, reconfiguring a piece of an >>>> FPGA involves more than a fpga manager. Best practices of (re)configuring >>>> a fpga involves disabling bridges around the area being configured before >>>> actually performing the configuration. Once configuration is complete the >>>> bridges are reenabled. The bridges prevent random signals getting out >>>> during the configuration process. >> >> It also completely bypasses the driver model, i.e. whatever happens inside of >> the fpga is unknown to the kernel. > > Yes there also has to be some sort of enumeration after successfully > programming. One of the best parts of using device tree overlays for programming FPGA regions is that it solves the problem of enumerating the contents of the FPGA quite elegantly. The overlay can describe the contents of the newly programmed region, and the platform code will instantiate any necessary drivers. > >> >>> Is my understanding correct that by bridges you mean the interfaces >>> between an fpga and host CPU and also interfaces between reconfigurable >>> pieces of an fpga? Is disabling the bridges between reconfigurable pieces >>> handled by PR IP automatically when requesting the reconfiguration? >> >> Yes to the first part. I don't know enough about that part of IP for >> whether that >> happens automatic, I'd guess probably not, though. > > It's not automatic. That's why we need FPGA Region in the kernel to > represent FPGA regions on the FPGA. Each kernel region knows which > FPGA manager and which bridge to control. So telling a kernel FPGA > region to reconfigure a region initializes the sequence where the > bridge is disabled, the region is programmed, and the bridge is > reenabled. > > For a discussion of DTO and FPGA regions, please read: > > Documentation/devicetree/bindings/fpga/fpga-region.txt > > The DT overlay support which is upstream (and also on the Altera > github branches) has FPGA regions that are controlled by applying an > overlay. I"m working on a patchset that breaks out the concept of > regions from device tree so that we may be able to create a region by > some mechanism other than DT. Hopefully on the list this weekend as a > RFC. > >> >>> I'm also looking for possibilities to use device tree overlays with >>> fpgas on PCIe buses. Additional issue here is that we normally do not >>> use device tree (and overlays) on x86 platform. Was this topic already >>> discussed on the mailing list? >> >> We had a longer discussion at the Linux Plumbers conference. The >> overlap of people >> that know about ACPI based PCIe platforms and DT based ARM / other platforms >> seemed pretty small. >> >> Notes are here: >> https://etherpad.openstack.org/p/LPC2016_FPGA >> >> I'd be happy to discuss if people have ideas. I don't know who's >> working on what currently. >> Personally I've done a lot of thinking, but couldn't come to a good >> solution for the DT case, >> and don't know enough about ACPI / PCIe (yet). >> >> Cheers & thanks for contributing! >> >> Moritz > -- > To unsubscribe from this list: send the line "unsubscribe linux-fpga" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Feb 9, 2017 at 4:00 AM, Anatolij Gustschin <agust@denx.de> wrote: Hi Anatolij, > Partial reconfiguration (PR) allows to reconfigure part of the design's > core logic while the remainder of the design continues running. This > driver loads a new design (RBF file under /lib/firmware) over PCIe. > Writing new rbf file name to /sys/class/fpga_manager/fpgaN/firmware > interface triggers FPGA reconfiguration. > > Signed-off-by: Anatolij Gustschin <agust@denx.de> > --- > drivers/fpga/Kconfig | 7 ++ > drivers/fpga/Makefile | 1 + > drivers/fpga/a10-pcie-pr.c | 299 +++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 307 insertions(+) > create mode 100644 drivers/fpga/a10-pcie-pr.c > > diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig > index ce861a2..aa50fa4 100644 > --- a/drivers/fpga/Kconfig > +++ b/drivers/fpga/Kconfig > @@ -20,6 +20,13 @@ config FPGA_REGION > FPGA Regions allow loading FPGA images under control of > the Device Tree. > > +config FPGA_MGR_A10_PCIE_PR > + tristate "Arria 10 partial reconfiguration over PCIe PR IP" > + depends on PCI > + help > + FPGA manager driver support for Arria 10 partial > + reconfiguration over PCIe > + > config FPGA_MGR_SOCFPGA > tristate "Altera SOCFPGA FPGA Manager" > depends on ARCH_SOCFPGA || COMPILE_TEST > diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile > index 8df07bc..b90b3f5 100644 > --- a/drivers/fpga/Makefile > +++ b/drivers/fpga/Makefile > @@ -6,6 +6,7 @@ > obj-$(CONFIG_FPGA) += fpga-mgr.o > > # FPGA Manager Drivers > +obj-$(CONFIG_FPGA_MGR_A10_PCIE_PR) += a10-pcie-pr.o > obj-$(CONFIG_FPGA_MGR_SOCFPGA) += socfpga.o > obj-$(CONFIG_FPGA_MGR_SOCFPGA_A10) += socfpga-a10.o > obj-$(CONFIG_FPGA_MGR_ZYNQ_FPGA) += zynq-fpga.o > diff --git a/drivers/fpga/a10-pcie-pr.c b/drivers/fpga/a10-pcie-pr.c > new file mode 100644 > index 0000000..09de181 > --- /dev/null > +++ b/drivers/fpga/a10-pcie-pr.c > @@ -0,0 +1,299 @@ > +/* > + * FPGA Manager Driver for Arria 10 partial reconfiguration over PCIe. > + * Firmware must be in binary "rbf" format. > + * > + * Copyright (C) 2017 DENX Software Engineering > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation; version 2 of the License. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + */ > + > +#include <linux/delay.h> > +#include <linux/fpga/fpga-mgr.h> > +#include <linux/errno.h> > +#include <linux/kernel.h> > +#include <linux/limits.h> > +#include <linux/module.h> > +#include <linux/pci.h> > +#include <linux/types.h> > + > +#define DRV_NAME "a10-pcie-pr" > +#define PCIE_PR_MGR_NAME "PCIe PR Manager" > +#define DFLT_FIRMWARE_STR "a10_pr_region.rbf" > +#define PR_IP_BAR 4 /* PCIe PR IP core BAR */ > +#define PR_CONTROLLER_OFFSET 0xcfb0 > +#define CMD_PR 0x01 > +#define CMD_DOUBLE_PR 0x03 > + > +static char firmware[NAME_MAX]; > +module_param_string(firmware, firmware, sizeof(firmware), 0444); > +MODULE_PARM_DESC(firmware, " RBF file in /lib/firmware to load via PCIe PR"); > + > +/* use value 1 for debugging only */ > +static int chkcfg; > +module_param(chkcfg, int, 0664); > +MODULE_PARM_DESC(chkcfg, " 1 - check error status, 0 - skip check (default 0)"); > + > +/* PR IP core registers */ > +struct pr_regs { > + u32 pr_data; > + u32 pr_csr; > +}; > + > +struct pcie_pr_conf { > + struct fpga_manager *mgr; > + struct pci_dev *pci_dev; > + struct pr_regs *regs; > + void __iomem *map; > + u32 fw_size; > +}; > + > +static enum fpga_mgr_states pcie_pr_state(struct fpga_manager *mgr) > +{ > + return mgr->state; > +} This will need to get state from device or return that it's unknown. > + > +static int pcie_pr_write_init(struct fpga_manager *mgr, > + struct fpga_image_info *info, > + const char *buf, size_t count) > +{ > + struct pcie_pr_conf *conf = mgr->priv; > + u32 status; > + > + if (info) > + dev_dbg(&mgr->dev, "%s(): flags 0x%x\n", __func__, info->flags); > + > + if (info && !(info->flags & FPGA_MGR_PARTIAL_RECONFIG)) { > + dev_err(&mgr->dev, "Only partial reconfiguration supported.\n"); > + return -EINVAL; > + } > + > + status = readl(&conf->regs->pr_csr); > + dev_dbg(&mgr->dev, "PR IP status 0x%08X\n", status); > + > + dev_dbg(&mgr->dev, "PR IP command 0x%08X\n", CMD_PR); > + writel(CMD_PR, &conf->regs->pr_csr); > + > + status = readl(&conf->regs->pr_csr); > + dev_dbg(&mgr->dev, "PR IP status: 0x%08X\n", (int) status); > + if ((status != 0x10) && (status != 0x0)) { > + dev_err(&mgr->dev, "status 0x%x\n", status); > + return -EFAULT; > + } > + > + return 0; > +} > + > +static int pcie_pr_write(struct fpga_manager *mgr, const char *buf, > + size_t count) > +{ > + struct pcie_pr_conf *conf = mgr->priv; > + const u32 *data, *data_end; > + size_t bytes = 0; > + int ret = 0; > + int status; > + > + dev_dbg(&mgr->dev, "%s: count %li\n", __func__, count); > + > + conf->fw_size = 0; > + data = (u32 *)buf; > + data_end = (u32 *)((char *)buf + count); > + > + while (data < data_end) { > + writel(*data++, &conf->regs->pr_data); > + bytes += 4; > + if (chkcfg) { > + status = readl(&conf->regs->pr_csr) >> 2; > + switch (status) { > + case 1 ... 3: > + dev_err(&mgr->dev, "PR error: 0x%x\n", status); > + ret = -EIO; > + break; > + } > + } > + } > + > + dev_dbg(&mgr->dev, "PR bytes written: %li\n", bytes); > + > + if (!ret) { > + /* > + * remember the loaded rbf image size to show this > + * info via sysfs firmware file when reading it > + */ > + conf->fw_size = count; > + } > + > + return ret; > +} > + > +static int pcie_pr_write_complete(struct fpga_manager *mgr, > + struct fpga_image_info *info) > +{ > + struct pcie_pr_conf *conf = mgr->priv; > + int status = 0; > + > + dev_dbg(&mgr->dev, "%s\n", __func__); > + > + status = readl(&conf->regs->pr_csr); > + if (status == 0x14) { > + dev_dbg(&mgr->dev, "PR done: 0x%08X\n", status); > + return 0; > + } > + > + dev_err(&mgr->dev, "PR error: 0x%08X\n", status); > + > + return -EIO; > +} > + > +static const struct fpga_manager_ops pcie_pr_ops = { > + .state = pcie_pr_state, > + .write_init = pcie_pr_write_init, > + .write = pcie_pr_write, > + .write_complete = pcie_pr_write_complete, > +}; > + > +static ssize_t show_firmware(struct device *dev, struct device_attribute *attr, > + char *buf) > +{ > + struct fpga_manager *mgr = to_fpga_manager(dev); > + struct pcie_pr_conf *conf = mgr->priv; > + > + return snprintf(buf, PAGE_SIZE, "%s %i\n", > + firmware, conf->fw_size); > +} > + > +static ssize_t store_firmware(struct device *dev, struct device_attribute *attr, > + const char *buf, size_t count) > +{ > + struct fpga_manager *mgr = to_fpga_manager(dev); > + int ret; > + > + if (!count || count > NAME_MAX) > + return -EINVAL; > + > + ret = sscanf(buf, "%s", firmware); > + if (ret != 1) > + return -EINVAL; > + > + ret = fpga_mgr_firmware_load(mgr, NULL, firmware); > + > + return ret ? ret : count; > +} > + > +static DEVICE_ATTR(firmware, 0664, show_firmware, store_firmware); I mentioned in another thread - please don't add an interface at this level. > + > +static int pcie_pr_probe(struct pci_dev *pdev, > + const struct pci_device_id *dev_id) > +{ > + struct pcie_pr_conf *conf; > + int ret; > + > + dev_dbg(&pdev->dev, "%s\n", __func__); > + > + ret = pci_enable_device(pdev); > + if (ret < 0) { > + dev_err(&pdev->dev, "can't enable pci device: %d\n", ret); > + return ret; > + } > + > + pci_set_master(pdev); > + > + conf = devm_kzalloc(&pdev->dev, sizeof(*conf), GFP_KERNEL); > + if (!conf) { > + ret = -ENOMEM; > + goto err; > + } > + > + conf->pci_dev = pdev; > + > + if (pci_request_region(pdev, PR_IP_BAR, "PR IP") < 0) { > + dev_err(&pdev->dev, "Requesting PR BAR region failed\n"); > + ret = -ENODEV; > + goto err; > + } > + > + conf->map = pci_iomap(pdev, PR_IP_BAR, 0); > + if (!conf->map) { > + dev_warn(&pdev->dev, "Mapping PR BAR failed\n"); > + ret = -ENODEV; > + goto err_reg; > + } > + > + conf->regs = conf->map + PR_CONTROLLER_OFFSET; > + > + ret = fpga_mgr_register(&pdev->dev, PCIE_PR_MGR_NAME, > + &pcie_pr_ops, conf); > + if (ret) > + goto err_unmap; > + > + conf->mgr = fpga_mgr_get(&pdev->dev); > + if (IS_ERR(conf->mgr)) { > + dev_err(&pdev->dev, "Getting fpga-mgr reference failed\n"); > + ret = -ENODEV; > + goto err_mgr; > + } > + > + if (!strnlen(firmware, NAME_MAX)) > + strncpy(firmware, DFLT_FIRMWARE_STR, sizeof(firmware)); > + > + ret = fpga_mgr_firmware_load(conf->mgr, NULL, firmware); > + if (ret) > + dev_warn(&pdev->dev, "loading fpga image failed\n"); Please don't do fpga_mgr_get/firmware_load/put here. That would tie this particular FPGA device programming method to only one use case. > + > + ret = device_create_file(&conf->mgr->dev, &dev_attr_firmware); > + if (ret) > + dev_warn(&pdev->dev, "can't create reconfig interface %d\n", > + ret); > + > + return 0; > + > +err_mgr: > + fpga_mgr_unregister(&pdev->dev); > +err_unmap: > + pci_iounmap(pdev, conf->map); > +err_reg: > + pci_release_region(pdev, PR_IP_BAR); > +err: > + pci_disable_device(pdev); > + return ret; > +} > + > +static void pcie_pr_remove(struct pci_dev *pdev) > +{ > + struct fpga_manager *mgr = pci_get_drvdata(pdev); > + struct pcie_pr_conf *conf = mgr->priv; > + > + fpga_mgr_put(conf->mgr); Won't need fpga_mgr_put if you don't 'get' in probe. Thanks for submitting! Alan Tull > + fpga_mgr_unregister(&pdev->dev); > + pci_iounmap(pdev, conf->map); > + pci_release_region(pdev, PR_IP_BAR); > + pci_disable_device(pdev); > +} > + > +#define PCI_VENDOR_ID_ALTERA 0x1172 > +#define PCI_DEVICE_ID_FPGA 0xe003 > + > +static struct pci_device_id pcie_pr_id_tbl[] = { > + { PCI_VDEVICE(ALTERA, PCI_DEVICE_ID_FPGA) }, > + { } > +}; > + > +static struct pci_driver pcie_pr_driver = { > + .name = DRV_NAME, > + .id_table = pcie_pr_id_tbl, > + .probe = pcie_pr_probe, > + .remove = pcie_pr_remove, > +}; > + > +module_pci_driver(pcie_pr_driver) > + > +MODULE_DEVICE_TABLE(pci, pcie_pr_id_tbl); > +MODULE_LICENSE("GPL"); > +MODULE_AUTHOR("Anatolij Gustschin <agust@denx.de>"); > +MODULE_DESCRIPTION("Arria 10 FPGA reconfiguration via PCIe PR IP"); > -- > 2.7.4 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-fpga" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 13 Feb 2017, matthew.gerlach@linux.intel.com wrote: Hi Anatolij, It has been a little while since I've heard from you on this conversation, but I realized I did not explicitly put you on the list of receipients for the first four revisions of patches I have sent out that are mentioned below. I apologize. Please see more comments inline below. Matthew Gerlach > > Hi Anatolij, > > > > On Fri, 10 Feb 2017, Anatolij Gustschin wrote: > >> Hi Matthew, >> >> On Thu, 9 Feb 2017 18:55:18 -0800 (PST) >> matthew.gerlach@linux.intel.com matthew.gerlach@linux.intel.com wrote: >> >>> Hi Anatolij Gustschin, >>> >>> I've been working on a driver for what I think is the same >>> PCIe card with an Arria10 FPGA on it, but the design on the card has a >>> different pci product id than yours. The difference in product id may be >>> related to a different design in the fpga. I am using an Altera A10 PCIe >>> DevKit, and the product id I am targeting is 5052 compared to the product >>> id of e005 supported by your driver. Regardless of the pci product id, it >>> looks like we are using the same Partial Reconfiguration IP Core. I >>> plan on trying this code tomorrow with my card. Given our apparent, >>> similar development, I'm looking forward to constructive >>> discussions. >> >> I'm currently using Attila Instant-DevKit Arria 10 FPGA FMC IDK, with >> PR over PCIe Reference Design here [1]. We are interested in a mainline >> FPGA manager driver for A10 partial reconfiguration. It would be optimal >> for our use case if we could implement partial reconfiguration over PCIe >> with similar userspace interface on DT based platforms (ARM) and on x86 >> platform. > > I guess we are using different boards, but the base A10 PR over PCIe > Reference Design is similar. I am working the owners of the design to update > the design to include appropriate bridges around the PR region. > I agree that it would be optimal to have a similar userspace interface for > using FPGA with DT based platforms and x86. At the moment I am using a very > rudimentary debugfs based interface to control PR. > >> >>> I want to begin with there is a lot of great work in this patch. You've >>> done an excellent job creating a driver for the PR IP Core that plugs into >>> Alan Tull's FPGA Manager framework. You've also done a great job at a >>> very concise PCIe driver that focuses on just supporting reconfiguring >>> the fpga over pcie. I think a PCIe driver that only is used for >>> Partial Reconfiguration can be very useful. >>> >>> As it turns out, the PR IP Core can be instantiated in any Arria10 FPGA, >>> and we are currently supporting designs using the PR IP core that are not >>> connected to a PCIe bus. For this reason, I have a version of code for >>> the PR IP that is separated into two parts. The first part is considered >>> the base code that accesses the IP core after its registers have been >>> mapped. This code is essentially the set of functions registered to the >>> fpga-mgr. Your PCIe portion of this patch could call the common code. >>> I then have a separate file of code used for a platform_driver >>> implementation >>> of the PR IP. Should I go ahead a post a patch of this separated PR IP >>> driver for further discussion? There are also changes to the PR IP code >>> being released in acds/17.0 that the driver will need to address. >> >> Yes, posting the patch would help I think. Before I started the work >> on this patch I checked linux-socfpga tree on github (socfpga-4.1.22-ltsi >> branch and later socfpga-4.7 branch) but didn't find anything for >> partial reconfiguration. I could look at your code and need to figure >> out what must be changed in the driver so that PR IP over PCIe is >> also supported in mainline kernel. I went and looked at socfpga git hub, and it turns out, the driver for the a10 PR IP component first shows up in socfpga-4.1.33-ltsi. This seems like a case of unfortunate timing. > > I will post the patch and look forward to feedback. We are in alignment with > the desire for PR IP over PCIe being suppported in the mainline kernel. I just posted v5 of these patches with you on the "to" list. It should be fairly simple to modify the pcie driver you posted to call the register/unregister core functions. I am successfully calling the core functions in both a SOCFPGA environment and a x86 with PCIe environment. If it would be helpful, I could try modifying your post to call the functions. I believe that separating the fpga-mgr driver for the A10 PR IP from the PCIe driver and any user space interface discussions will provide a good first step of getting all the necessary pieces for PR IP over PCIe into the mainline kernel. I am looking forward to your feedback. > >> >>> I think your patch also brings up a very interesting issue. Semantically, >>> by writing the rbf filename to /sys/class/fpga_maganger/fpgaN/firmware, >>> you are configuring what is connected to that fpga manager. However, a >>> single fpga manager can individually reconfigure more than one piece of an >>> fpga. Conversely, one could have a single fpga manager per piece of >>> fpga that can be reconfigured. In addition, reconfiguring a piece of an >>> FPGA involves more than a fpga manager. Best practices of (re)configuring >>> a fpga involves disabling bridges around the area being configured before >>> actually performing the configuration. Once configuration is complete the >>> bridges are reenabled. The bridges prevent random signals getting out >>> during the configuration process. >> >> Is my understanding correct that by bridges you mean the interfaces >> between an fpga and host CPU and also interfaces between reconfigurable >> pieces of an fpga? Is disabling the bridges between reconfigurable pieces >> handled by PR IP automatically when requesting the reconfiguration? > > Your understanding is correct. I am referring to bridges between the fpga > and the host CPU and also bridges between reconfigurable pieces of the fpga. > With Altera/Intel FPGAs, the component that controls the bridges is a > different component that configures the FPGA. The PR IP does not > automatically disable bridges. > >> >>> Alan Tull has used the term fpga-region to describe a piece of fpga that >>> can be (re)configured. A fpga-region is defined by the fpga-mgr that >>> controls region and the set of bridges surrounding the configurable >>> region. The set of bridges surrounding the region could be empty, but >>> that is not recommended. A fpga region could be the entire fpga, or a >>> single fpga can have many regions that can be individually reconfigured >>> without affecting the other regions. So semantically, one does not >>> reconfigure a fpga manager, but one does reconfigure a fpga region. Alan >>> and I >>> have been reconfiguring fpga regions with device tree overlays on SOCFPGAs >>> for quite some time and are recently looking to bring this notion to fpgas >>> connected to PCIe buses. It may be that your patch will accelerate this >>> work. >> >> I'm also looking for possibilities to use device tree overlays with >> fpgas on PCIe buses. Additional issue here is that we normally do not >> use device tree (and overlays) on x86 platform. Was this topic already >> discussed on the mailing list? > > I don't think using device tree (and overlays) on x86 platform has been > discussed too much on this mailing list. As Moritz pointed out, the topic > was discussed at the last Linux Plumber's conference. At the 2015 Linux > Plumber's conference, there was a talk of someone actually using device tree > overlays with x86 based system, Juniper Networks I think, but I don't think > the work was ever upstreamed. > > > >> >> Best regards, >> >> Anatolij >> >> [1] >> https://github.com/01org/fpga-partial-reconfig/tree/master/ref_designs/a10_pcie_devkit_cvp >> > > > Matthew Gerlach > -- > To unsubscribe from this list: send the line "unsubscribe linux-fpga" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Matthew, On Fri, 10 Mar 2017 12:28:33 -0800 (PST) matthew.gerlach@linux.intel.com matthew.gerlach@linux.intel.com wrote: >On Mon, 13 Feb 2017, matthew.gerlach@linux.intel.com wrote: > > >Hi Anatolij, > >It has been a little while since I've heard from you on this conversation, >but I realized I did not explicitly put you on the list of receipients for >the first four revisions of patches I have sent out that are mentioned >below. I apologize. > >Please see more comments inline below. > >Matthew Gerlach >> >> Hi Anatolij, >> >> >> >> On Fri, 10 Feb 2017, Anatolij Gustschin wrote: >> >>> Hi Matthew, >>> >>> On Thu, 9 Feb 2017 18:55:18 -0800 (PST) >>> matthew.gerlach@linux.intel.com matthew.gerlach@linux.intel.com wrote: >>> >>>> Hi Anatolij Gustschin, >>>> >>>> I've been working on a driver for what I think is the same >>>> PCIe card with an Arria10 FPGA on it, but the design on the card has a >>>> different pci product id than yours. The difference in product id may be >>>> related to a different design in the fpga. I am using an Altera A10 PCIe >>>> DevKit, and the product id I am targeting is 5052 compared to the product >>>> id of e005 supported by your driver. Regardless of the pci product id, it >>>> looks like we are using the same Partial Reconfiguration IP Core. I >>>> plan on trying this code tomorrow with my card. Given our apparent, >>>> similar development, I'm looking forward to constructive >>>> discussions. >>> >>> I'm currently using Attila Instant-DevKit Arria 10 FPGA FMC IDK, with >>> PR over PCIe Reference Design here [1]. We are interested in a mainline >>> FPGA manager driver for A10 partial reconfiguration. It would be optimal >>> for our use case if we could implement partial reconfiguration over PCIe >>> with similar userspace interface on DT based platforms (ARM) and on x86 >>> platform. >> >> I guess we are using different boards, but the base A10 PR over PCIe >> Reference Design is similar. I am working the owners of the design to update >> the design to include appropriate bridges around the PR region. >> I agree that it would be optimal to have a similar userspace interface for >> using FPGA with DT based platforms and x86. At the moment I am using a very >> rudimentary debugfs based interface to control PR. >> >>> >>>> I want to begin with there is a lot of great work in this patch. You've >>>> done an excellent job creating a driver for the PR IP Core that plugs into >>>> Alan Tull's FPGA Manager framework. You've also done a great job at a >>>> very concise PCIe driver that focuses on just supporting reconfiguring >>>> the fpga over pcie. I think a PCIe driver that only is used for >>>> Partial Reconfiguration can be very useful. >>>> >>>> As it turns out, the PR IP Core can be instantiated in any Arria10 FPGA, >>>> and we are currently supporting designs using the PR IP core that are not >>>> connected to a PCIe bus. For this reason, I have a version of code for >>>> the PR IP that is separated into two parts. The first part is considered >>>> the base code that accesses the IP core after its registers have been >>>> mapped. This code is essentially the set of functions registered to the >>>> fpga-mgr. Your PCIe portion of this patch could call the common code. >>>> I then have a separate file of code used for a platform_driver >>>> implementation >>>> of the PR IP. Should I go ahead a post a patch of this separated PR IP >>>> driver for further discussion? There are also changes to the PR IP code >>>> being released in acds/17.0 that the driver will need to address. >>> >>> Yes, posting the patch would help I think. Before I started the work >>> on this patch I checked linux-socfpga tree on github (socfpga-4.1.22-ltsi >>> branch and later socfpga-4.7 branch) but didn't find anything for >>> partial reconfiguration. I could look at your code and need to figure >>> out what must be changed in the driver so that PR IP over PCIe is >>> also supported in mainline kernel. > >I went and looked at socfpga git hub, and it turns out, the driver for the >a10 PR IP component first shows up in socfpga-4.1.33-ltsi. This seems >like a case of unfortunate timing. Ok, thanks for the pointer! >> I will post the patch and look forward to feedback. We are in alignment with >> the desire for PR IP over PCIe being suppported in the mainline kernel. > >I just posted v5 of these patches with you on the "to" list. It should be >fairly simple to modify the pcie driver you posted to call the >register/unregister core functions. I am successfully calling the core >functions in both a SOCFPGA environment and a x86 with PCIe environment. >If it would be helpful, I could try modifying your post to call the >functions. I believe that separating the fpga-mgr driver for the A10 PR IP >from the PCIe driver and any user space interface discussions will provide >a good first step of getting all the necessary pieces for PR IP over >PCIe into the mainline kernel. I am looking forward to your feedback. Thanks for your series! I've been busy with other stuff and will be away for next tree days. I'll rework the PCIe PR patch based on your series when I'm back on Friday. Best regards, Anatolij -- To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig index ce861a2..aa50fa4 100644 --- a/drivers/fpga/Kconfig +++ b/drivers/fpga/Kconfig @@ -20,6 +20,13 @@ config FPGA_REGION FPGA Regions allow loading FPGA images under control of the Device Tree. +config FPGA_MGR_A10_PCIE_PR + tristate "Arria 10 partial reconfiguration over PCIe PR IP" + depends on PCI + help + FPGA manager driver support for Arria 10 partial + reconfiguration over PCIe + config FPGA_MGR_SOCFPGA tristate "Altera SOCFPGA FPGA Manager" depends on ARCH_SOCFPGA || COMPILE_TEST diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile index 8df07bc..b90b3f5 100644 --- a/drivers/fpga/Makefile +++ b/drivers/fpga/Makefile @@ -6,6 +6,7 @@ obj-$(CONFIG_FPGA) += fpga-mgr.o # FPGA Manager Drivers +obj-$(CONFIG_FPGA_MGR_A10_PCIE_PR) += a10-pcie-pr.o obj-$(CONFIG_FPGA_MGR_SOCFPGA) += socfpga.o obj-$(CONFIG_FPGA_MGR_SOCFPGA_A10) += socfpga-a10.o obj-$(CONFIG_FPGA_MGR_ZYNQ_FPGA) += zynq-fpga.o diff --git a/drivers/fpga/a10-pcie-pr.c b/drivers/fpga/a10-pcie-pr.c new file mode 100644 index 0000000..09de181 --- /dev/null +++ b/drivers/fpga/a10-pcie-pr.c @@ -0,0 +1,299 @@ +/* + * FPGA Manager Driver for Arria 10 partial reconfiguration over PCIe. + * Firmware must be in binary "rbf" format. + * + * Copyright (C) 2017 DENX Software Engineering + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; version 2 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include <linux/delay.h> +#include <linux/fpga/fpga-mgr.h> +#include <linux/errno.h> +#include <linux/kernel.h> +#include <linux/limits.h> +#include <linux/module.h> +#include <linux/pci.h> +#include <linux/types.h> + +#define DRV_NAME "a10-pcie-pr" +#define PCIE_PR_MGR_NAME "PCIe PR Manager" +#define DFLT_FIRMWARE_STR "a10_pr_region.rbf" +#define PR_IP_BAR 4 /* PCIe PR IP core BAR */ +#define PR_CONTROLLER_OFFSET 0xcfb0 +#define CMD_PR 0x01 +#define CMD_DOUBLE_PR 0x03 + +static char firmware[NAME_MAX]; +module_param_string(firmware, firmware, sizeof(firmware), 0444); +MODULE_PARM_DESC(firmware, " RBF file in /lib/firmware to load via PCIe PR"); + +/* use value 1 for debugging only */ +static int chkcfg; +module_param(chkcfg, int, 0664); +MODULE_PARM_DESC(chkcfg, " 1 - check error status, 0 - skip check (default 0)"); + +/* PR IP core registers */ +struct pr_regs { + u32 pr_data; + u32 pr_csr; +}; + +struct pcie_pr_conf { + struct fpga_manager *mgr; + struct pci_dev *pci_dev; + struct pr_regs *regs; + void __iomem *map; + u32 fw_size; +}; + +static enum fpga_mgr_states pcie_pr_state(struct fpga_manager *mgr) +{ + return mgr->state; +} + +static int pcie_pr_write_init(struct fpga_manager *mgr, + struct fpga_image_info *info, + const char *buf, size_t count) +{ + struct pcie_pr_conf *conf = mgr->priv; + u32 status; + + if (info) + dev_dbg(&mgr->dev, "%s(): flags 0x%x\n", __func__, info->flags); + + if (info && !(info->flags & FPGA_MGR_PARTIAL_RECONFIG)) { + dev_err(&mgr->dev, "Only partial reconfiguration supported.\n"); + return -EINVAL; + } + + status = readl(&conf->regs->pr_csr); + dev_dbg(&mgr->dev, "PR IP status 0x%08X\n", status); + + dev_dbg(&mgr->dev, "PR IP command 0x%08X\n", CMD_PR); + writel(CMD_PR, &conf->regs->pr_csr); + + status = readl(&conf->regs->pr_csr); + dev_dbg(&mgr->dev, "PR IP status: 0x%08X\n", (int) status); + if ((status != 0x10) && (status != 0x0)) { + dev_err(&mgr->dev, "status 0x%x\n", status); + return -EFAULT; + } + + return 0; +} + +static int pcie_pr_write(struct fpga_manager *mgr, const char *buf, + size_t count) +{ + struct pcie_pr_conf *conf = mgr->priv; + const u32 *data, *data_end; + size_t bytes = 0; + int ret = 0; + int status; + + dev_dbg(&mgr->dev, "%s: count %li\n", __func__, count); + + conf->fw_size = 0; + data = (u32 *)buf; + data_end = (u32 *)((char *)buf + count); + + while (data < data_end) { + writel(*data++, &conf->regs->pr_data); + bytes += 4; + if (chkcfg) { + status = readl(&conf->regs->pr_csr) >> 2; + switch (status) { + case 1 ... 3: + dev_err(&mgr->dev, "PR error: 0x%x\n", status); + ret = -EIO; + break; + } + } + } + + dev_dbg(&mgr->dev, "PR bytes written: %li\n", bytes); + + if (!ret) { + /* + * remember the loaded rbf image size to show this + * info via sysfs firmware file when reading it + */ + conf->fw_size = count; + } + + return ret; +} + +static int pcie_pr_write_complete(struct fpga_manager *mgr, + struct fpga_image_info *info) +{ + struct pcie_pr_conf *conf = mgr->priv; + int status = 0; + + dev_dbg(&mgr->dev, "%s\n", __func__); + + status = readl(&conf->regs->pr_csr); + if (status == 0x14) { + dev_dbg(&mgr->dev, "PR done: 0x%08X\n", status); + return 0; + } + + dev_err(&mgr->dev, "PR error: 0x%08X\n", status); + + return -EIO; +} + +static const struct fpga_manager_ops pcie_pr_ops = { + .state = pcie_pr_state, + .write_init = pcie_pr_write_init, + .write = pcie_pr_write, + .write_complete = pcie_pr_write_complete, +}; + +static ssize_t show_firmware(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct fpga_manager *mgr = to_fpga_manager(dev); + struct pcie_pr_conf *conf = mgr->priv; + + return snprintf(buf, PAGE_SIZE, "%s %i\n", + firmware, conf->fw_size); +} + +static ssize_t store_firmware(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct fpga_manager *mgr = to_fpga_manager(dev); + int ret; + + if (!count || count > NAME_MAX) + return -EINVAL; + + ret = sscanf(buf, "%s", firmware); + if (ret != 1) + return -EINVAL; + + ret = fpga_mgr_firmware_load(mgr, NULL, firmware); + + return ret ? ret : count; +} + +static DEVICE_ATTR(firmware, 0664, show_firmware, store_firmware); + +static int pcie_pr_probe(struct pci_dev *pdev, + const struct pci_device_id *dev_id) +{ + struct pcie_pr_conf *conf; + int ret; + + dev_dbg(&pdev->dev, "%s\n", __func__); + + ret = pci_enable_device(pdev); + if (ret < 0) { + dev_err(&pdev->dev, "can't enable pci device: %d\n", ret); + return ret; + } + + pci_set_master(pdev); + + conf = devm_kzalloc(&pdev->dev, sizeof(*conf), GFP_KERNEL); + if (!conf) { + ret = -ENOMEM; + goto err; + } + + conf->pci_dev = pdev; + + if (pci_request_region(pdev, PR_IP_BAR, "PR IP") < 0) { + dev_err(&pdev->dev, "Requesting PR BAR region failed\n"); + ret = -ENODEV; + goto err; + } + + conf->map = pci_iomap(pdev, PR_IP_BAR, 0); + if (!conf->map) { + dev_warn(&pdev->dev, "Mapping PR BAR failed\n"); + ret = -ENODEV; + goto err_reg; + } + + conf->regs = conf->map + PR_CONTROLLER_OFFSET; + + ret = fpga_mgr_register(&pdev->dev, PCIE_PR_MGR_NAME, + &pcie_pr_ops, conf); + if (ret) + goto err_unmap; + + conf->mgr = fpga_mgr_get(&pdev->dev); + if (IS_ERR(conf->mgr)) { + dev_err(&pdev->dev, "Getting fpga-mgr reference failed\n"); + ret = -ENODEV; + goto err_mgr; + } + + if (!strnlen(firmware, NAME_MAX)) + strncpy(firmware, DFLT_FIRMWARE_STR, sizeof(firmware)); + + ret = fpga_mgr_firmware_load(conf->mgr, NULL, firmware); + if (ret) + dev_warn(&pdev->dev, "loading fpga image failed\n"); + + ret = device_create_file(&conf->mgr->dev, &dev_attr_firmware); + if (ret) + dev_warn(&pdev->dev, "can't create reconfig interface %d\n", + ret); + + return 0; + +err_mgr: + fpga_mgr_unregister(&pdev->dev); +err_unmap: + pci_iounmap(pdev, conf->map); +err_reg: + pci_release_region(pdev, PR_IP_BAR); +err: + pci_disable_device(pdev); + return ret; +} + +static void pcie_pr_remove(struct pci_dev *pdev) +{ + struct fpga_manager *mgr = pci_get_drvdata(pdev); + struct pcie_pr_conf *conf = mgr->priv; + + fpga_mgr_put(conf->mgr); + fpga_mgr_unregister(&pdev->dev); + pci_iounmap(pdev, conf->map); + pci_release_region(pdev, PR_IP_BAR); + pci_disable_device(pdev); +} + +#define PCI_VENDOR_ID_ALTERA 0x1172 +#define PCI_DEVICE_ID_FPGA 0xe003 + +static struct pci_device_id pcie_pr_id_tbl[] = { + { PCI_VDEVICE(ALTERA, PCI_DEVICE_ID_FPGA) }, + { } +}; + +static struct pci_driver pcie_pr_driver = { + .name = DRV_NAME, + .id_table = pcie_pr_id_tbl, + .probe = pcie_pr_probe, + .remove = pcie_pr_remove, +}; + +module_pci_driver(pcie_pr_driver) + +MODULE_DEVICE_TABLE(pci, pcie_pr_id_tbl); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Anatolij Gustschin <agust@denx.de>"); +MODULE_DESCRIPTION("Arria 10 FPGA reconfiguration via PCIe PR IP");
Partial reconfiguration (PR) allows to reconfigure part of the design's core logic while the remainder of the design continues running. This driver loads a new design (RBF file under /lib/firmware) over PCIe. Writing new rbf file name to /sys/class/fpga_manager/fpgaN/firmware interface triggers FPGA reconfiguration. Signed-off-by: Anatolij Gustschin <agust@denx.de> --- drivers/fpga/Kconfig | 7 ++ drivers/fpga/Makefile | 1 + drivers/fpga/a10-pcie-pr.c | 299 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 307 insertions(+) create mode 100644 drivers/fpga/a10-pcie-pr.c