| Message ID | 0bdd4f3f-965a-fe39-f7af-d8a40b7a68c6@suse.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | PCI: re-work pci_get_pdev() and friends | expand |
On 11/08/2022 11:52, Jan Beulich wrote: > Rename the latter, subsuming the functionality of the former when passed > NULL as first argument. > > Since this requires touching all call sites anyway, take the opportunity > and fold the remaining three parameters into a single pci_sbdf_t one. > > No functional change intended. In particular the locking related > assertion needs to continue to be kept silent when a non-NULL domain > pointer is passed - both vpci_read() and vpci_write() call the function > without holding the lock (adding respective locking to vPCI [or finding > an alternative to doing so] is the topic of a separate series). > > Signed-off-by: Jan Beulich <jbeulich@suse.com> > --- > v2: New. > > --- a/xen/arch/x86/irq.c > +++ b/xen/arch/x86/irq.c > @@ -2162,7 +2162,7 @@ int map_domain_pirq( > if ( !cpu_has_apic ) > goto done; > > - pdev = pci_get_pdev_by_domain(d, msi->seg, msi->bus, msi->devfn); > + pdev = pci_get_pdev(d, PCI_SBDF(msi->seg, msi->bus, msi->devfn)); Oh, I should really have read this patch before trying to do the sbdf conversion in patch 1. However, it occurs to me that this: diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h index 117379318f2c..6f0ab845017c 100644 --- a/xen/arch/x86/include/asm/msi.h +++ b/xen/arch/x86/include/asm/msi.h @@ -59,9 +59,14 @@ #define FIX_MSIX_MAX_PAGES 512 struct msi_info { - u16 seg; - u8 bus; - u8 devfn; + union { + struct { + u8 devfn; + u8 bus; + u16 seg; + }; + pci_sbdf_t sbdf; + }; int irq; int entry_nr; uint64_t table_base; will simplify several hunks in this patch, because you can just pass msi->sbdf rather than reconstructing it by reversing 32 bits worth of data from their in-memory representation. Preferably with something to this effect included, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
On 11.08.2022 15:21, Andrew Cooper wrote: > On 11/08/2022 11:52, Jan Beulich wrote: >> --- a/xen/arch/x86/irq.c >> +++ b/xen/arch/x86/irq.c >> @@ -2162,7 +2162,7 @@ int map_domain_pirq( >> if ( !cpu_has_apic ) >> goto done; >> >> - pdev = pci_get_pdev_by_domain(d, msi->seg, msi->bus, msi->devfn); >> + pdev = pci_get_pdev(d, PCI_SBDF(msi->seg, msi->bus, msi->devfn)); > > Oh, I should really have read this patch before trying to do the sbdf > conversion in patch 1. > > However, it occurs to me that this: > > diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h > index 117379318f2c..6f0ab845017c 100644 > --- a/xen/arch/x86/include/asm/msi.h > +++ b/xen/arch/x86/include/asm/msi.h > @@ -59,9 +59,14 @@ > #define FIX_MSIX_MAX_PAGES 512 > > struct msi_info { > - u16 seg; > - u8 bus; > - u8 devfn; > + union { > + struct { > + u8 devfn; > + u8 bus; > + u16 seg; > + }; > + pci_sbdf_t sbdf; > + }; > int irq; > int entry_nr; > uint64_t table_base; > > will simplify several hunks in this patch, because you can just pass > msi->sbdf rather than reconstructing it by reversing 32 bits worth of > data from their in-memory representation. No, I'm strictly against introducing a 2nd instance of such aliasing (we already have one in struct pci_dev, and that's bad enough). There could be _only_ an "sbdf" field here, yes, but that'll have knock-on effects elsewhere, so wants to be a separate change. And there are far more places where imo we'll want to use pci_sbdf_t. > Preferably with something to this effect included, Reviewed-by: Andrew > Cooper <andrew.cooper3@citrix.com> Thanks. Jan
On 11/08/2022 14:26, Jan Beulich wrote: > On 11.08.2022 15:21, Andrew Cooper wrote: >> On 11/08/2022 11:52, Jan Beulich wrote: >>> --- a/xen/arch/x86/irq.c >>> +++ b/xen/arch/x86/irq.c >>> @@ -2162,7 +2162,7 @@ int map_domain_pirq( >>> if ( !cpu_has_apic ) >>> goto done; >>> >>> - pdev = pci_get_pdev_by_domain(d, msi->seg, msi->bus, msi->devfn); >>> + pdev = pci_get_pdev(d, PCI_SBDF(msi->seg, msi->bus, msi->devfn)); >> Oh, I should really have read this patch before trying to do the sbdf >> conversion in patch 1. >> >> However, it occurs to me that this: >> >> diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h >> index 117379318f2c..6f0ab845017c 100644 >> --- a/xen/arch/x86/include/asm/msi.h >> +++ b/xen/arch/x86/include/asm/msi.h >> @@ -59,9 +59,14 @@ >> #define FIX_MSIX_MAX_PAGES 512 >> >> struct msi_info { >> - u16 seg; >> - u8 bus; >> - u8 devfn; >> + union { >> + struct { >> + u8 devfn; >> + u8 bus; >> + u16 seg; >> + }; >> + pci_sbdf_t sbdf; >> + }; >> int irq; >> int entry_nr; >> uint64_t table_base; >> >> will simplify several hunks in this patch, because you can just pass >> msi->sbdf rather than reconstructing it by reversing 32 bits worth of >> data from their in-memory representation. > No, I'm strictly against introducing a 2nd instance of such aliasing > (we already have one in struct pci_dev, and that's bad enough). There > could be _only_ an "sbdf" field here, yes, but that'll have knock-on > effects elsewhere, so wants to be a separate change. And there are far > more places where imo we'll want to use pci_sbdf_t. What's the likelihood of getting to that before 4.17 goes out? I'd prefer to see it fixed, and obviously even more conversion to sbdf_t is better. Basically, I'm happy for the conversion to not be in this patch, as long it's not going to get forgotten. ~Andrew
On 11.08.2022 17:09, Andrew Cooper wrote: > On 11/08/2022 14:26, Jan Beulich wrote: >> On 11.08.2022 15:21, Andrew Cooper wrote: >>> On 11/08/2022 11:52, Jan Beulich wrote: >>>> --- a/xen/arch/x86/irq.c >>>> +++ b/xen/arch/x86/irq.c >>>> @@ -2162,7 +2162,7 @@ int map_domain_pirq( >>>> if ( !cpu_has_apic ) >>>> goto done; >>>> >>>> - pdev = pci_get_pdev_by_domain(d, msi->seg, msi->bus, msi->devfn); >>>> + pdev = pci_get_pdev(d, PCI_SBDF(msi->seg, msi->bus, msi->devfn)); >>> Oh, I should really have read this patch before trying to do the sbdf >>> conversion in patch 1. >>> >>> However, it occurs to me that this: >>> >>> diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h >>> index 117379318f2c..6f0ab845017c 100644 >>> --- a/xen/arch/x86/include/asm/msi.h >>> +++ b/xen/arch/x86/include/asm/msi.h >>> @@ -59,9 +59,14 @@ >>> #define FIX_MSIX_MAX_PAGES 512 >>> >>> struct msi_info { >>> - u16 seg; >>> - u8 bus; >>> - u8 devfn; >>> + union { >>> + struct { >>> + u8 devfn; >>> + u8 bus; >>> + u16 seg; >>> + }; >>> + pci_sbdf_t sbdf; >>> + }; >>> int irq; >>> int entry_nr; >>> uint64_t table_base; >>> >>> will simplify several hunks in this patch, because you can just pass >>> msi->sbdf rather than reconstructing it by reversing 32 bits worth of >>> data from their in-memory representation. >> No, I'm strictly against introducing a 2nd instance of such aliasing >> (we already have one in struct pci_dev, and that's bad enough). There >> could be _only_ an "sbdf" field here, yes, but that'll have knock-on >> effects elsewhere, so wants to be a separate change. And there are far >> more places where imo we'll want to use pci_sbdf_t. > > What's the likelihood of getting to that before 4.17 goes out? Well, I can try to get to making a patch tomorrow, just in time to meet the submission deadline. But that's not really a promise ... Jan > I'd prefer to see it fixed, and obviously even more conversion to sbdf_t > is better. > > Basically, I'm happy for the conversion to not be in this patch, as long > it's not going to get forgotten. > > ~Andrew
Hi Jan, > On 11 Aug 2022, at 11:52 am, Jan Beulich <jbeulich@suse.com> wrote: > > Rename the latter, subsuming the functionality of the former when passed > NULL as first argument. > > Since this requires touching all call sites anyway, take the opportunity > and fold the remaining three parameters into a single pci_sbdf_t one. > > No functional change intended. In particular the locking related > assertion needs to continue to be kept silent when a non-NULL domain > pointer is passed - both vpci_read() and vpci_write() call the function > without holding the lock (adding respective locking to vPCI [or finding > an alternative to doing so] is the topic of a separate series). > > Signed-off-by: Jan Beulich <jbeulich@suse.com> Reviewed-by: Rahul Singh <rahul.singh@arm.com> Tested-by: Rahul Singh <rahul.singh@arm.com> Regards, Rahul
--- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2162,7 +2162,7 @@ int map_domain_pirq( if ( !cpu_has_apic ) goto done; - pdev = pci_get_pdev_by_domain(d, msi->seg, msi->bus, msi->devfn); + pdev = pci_get_pdev(d, PCI_SBDF(msi->seg, msi->bus, msi->devfn)); if ( !pdev ) goto done; --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -683,7 +683,8 @@ static u64 read_pci_mem_bar(u16 seg, u8 if ( vf >= 0 ) { - struct pci_dev *pdev = pci_get_pdev(seg, bus, PCI_DEVFN(slot, func)); + struct pci_dev *pdev = pci_get_pdev(NULL, + PCI_SBDF(seg, bus, slot, func)); unsigned int pos = pci_find_ext_capability(seg, bus, PCI_DEVFN(slot, func), PCI_EXT_CAP_ID_SRIOV); @@ -1000,7 +1001,7 @@ static int __pci_enable_msi(struct msi_i struct msi_desc *old_desc; ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(msi->seg, msi->bus, msi->devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(msi->seg, msi->bus, msi->devfn)); if ( !pdev ) return -ENODEV; @@ -1055,7 +1056,7 @@ static int __pci_enable_msix(struct msi_ struct msi_desc *old_desc; ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(msi->seg, msi->bus, msi->devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(msi->seg, msi->bus, msi->devfn)); if ( !pdev || !pdev->msix ) return -ENODEV; @@ -1146,7 +1147,7 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 return 0; pcidevs_lock(); - pdev = pci_get_pdev(seg, bus, devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); if ( !pdev ) rc = -ENODEV; else if ( pdev->msix->used_entries != !!off ) --- a/xen/arch/x86/pci.c +++ b/xen/arch/x86/pci.c @@ -90,7 +90,7 @@ int pci_conf_write_intercept(unsigned in pcidevs_lock(); - pdev = pci_get_pdev(seg, PCI_BUS(bdf), PCI_DEVFN(bdf)); + pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bdf)); if ( pdev ) rc = pci_msi_conf_write_intercept(pdev, reg, size, data); --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -531,7 +531,8 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_H break; pcidevs_lock(); - pdev = pci_get_pdev(0, restore_msi.bus, restore_msi.devfn); + pdev = pci_get_pdev(NULL, + PCI_SBDF(0, restore_msi.bus, restore_msi.devfn)); ret = pdev ? pci_restore_msi_state(pdev) : -ENODEV; pcidevs_unlock(); break; @@ -546,7 +547,7 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_H break; pcidevs_lock(); - pdev = pci_get_pdev(dev.seg, dev.bus, dev.devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(dev.seg, dev.bus, dev.devfn)); ret = pdev ? pci_restore_msi_state(pdev) : -ENODEV; pcidevs_unlock(); break; --- a/xen/common/sysctl.c +++ b/xen/common/sysctl.c @@ -447,7 +447,7 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xe } pcidevs_lock(); - pdev = pci_get_pdev(dev.seg, dev.bus, dev.devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(dev.seg, dev.bus, dev.devfn)); if ( !pdev ) node = XEN_INVALID_DEV; else if ( pdev->node == NUMA_NO_NODE ) --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -748,8 +748,7 @@ static bool_t __init set_iommu_interrupt } pcidevs_lock(); - iommu->msi.dev = pci_get_pdev(iommu->seg, PCI_BUS(iommu->bdf), - PCI_DEVFN(iommu->bdf)); + iommu->msi.dev = pci_get_pdev(NULL, PCI_SBDF(iommu->seg, iommu->bdf)); pcidevs_unlock(); if ( !iommu->msi.dev ) { @@ -1286,7 +1285,7 @@ static int __init cf_check amd_iommu_set if ( !pci_init ) continue; pcidevs_lock(); - pdev = pci_get_pdev(seg, PCI_BUS(bdf), PCI_DEVFN(bdf)); + pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bdf)); pcidevs_unlock(); } --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -727,7 +727,7 @@ int cf_check amd_iommu_get_reserved_devi const struct pci_dev *pdev; pcidevs_lock(); - pdev = pci_get_pdev(seg, sbdf.bus, sbdf.devfn); + pdev = pci_get_pdev(NULL, sbdf); pcidevs_unlock(); if ( pdev ) --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -576,23 +576,6 @@ int __init pci_ro_device(int seg, int bu return 0; } -struct pci_dev *pci_get_pdev(uint16_t seg, uint8_t bus, uint8_t devfn) -{ - const struct pci_seg *pseg = get_pseg(seg); - struct pci_dev *pdev; - - ASSERT(pcidevs_locked()); - - if ( !pseg ) - return NULL; - - list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) - if ( pdev->bus == bus && pdev->devfn == devfn ) - return pdev; - - return NULL; -} - struct pci_dev *pci_get_real_pdev(int seg, int bus, int devfn) { struct pci_dev *pdev; @@ -601,12 +584,12 @@ struct pci_dev *pci_get_real_pdev(int se if ( seg < 0 || bus < 0 || devfn < 0 ) return NULL; - for ( pdev = pci_get_pdev(seg, bus, devfn), stride = 4; + for ( pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)), stride = 4; !pdev && stride; stride >>= 1 ) { if ( !(devfn & (8 - stride)) ) continue; - pdev = pci_get_pdev(seg, bus, devfn & ~(8 - stride)); + pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn & ~(8 - stride))); if ( pdev && stride != pdev->phantom_stride ) pdev = NULL; } @@ -614,32 +597,33 @@ struct pci_dev *pci_get_real_pdev(int se return pdev; } -struct pci_dev *pci_get_pdev_by_domain(const struct domain *d, uint16_t seg, - uint8_t bus, uint8_t devfn) +struct pci_dev *pci_get_pdev(const struct domain *d, pci_sbdf_t sbdf) { struct pci_dev *pdev; + ASSERT(d || pcidevs_locked()); + /* * The hardware domain owns the majority of the devices in the system. * When there are multiple segments, traversing the per-segment list is * likely going to be faster, whereas for a single segment the difference * shouldn't be that large. */ - if ( is_hardware_domain(d) ) + if ( !d || is_hardware_domain(d) ) { - const struct pci_seg *pseg = get_pseg(seg); + const struct pci_seg *pseg = get_pseg(sbdf.seg); if ( !pseg ) return NULL; list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) - if ( pdev->bus == bus && pdev->devfn == devfn && - pdev->domain == d ) + if ( pdev->sbdf.bdf == sbdf.bdf && + (!d || pdev->domain == d) ) return pdev; } else list_for_each_entry ( pdev, &d->pdev_list, domain_list ) - if ( pdev->bus == bus && pdev->devfn == devfn ) + if ( pdev->sbdf.bdf == sbdf.bdf ) return pdev; return NULL; @@ -746,7 +730,9 @@ int pci_add_device(u16 seg, u8 bus, u8 d else if ( info->is_virtfn ) { pcidevs_lock(); - pdev = pci_get_pdev(seg, info->physfn.bus, info->physfn.devfn); + pdev = pci_get_pdev(NULL, + PCI_SBDF(seg, info->physfn.bus, + info->physfn.devfn)); if ( pdev ) pf_is_extfn = pdev->info.is_extfn; pcidevs_unlock(); @@ -924,7 +910,7 @@ static int deassign_device(struct domain return -EINVAL; ASSERT(pcidevs_locked()); - pdev = pci_get_pdev_by_domain(d, seg, bus, devfn); + pdev = pci_get_pdev(d, PCI_SBDF(seg, bus, devfn)); if ( !pdev ) return -ENODEV; @@ -1201,7 +1187,8 @@ static int __hwdom_init cf_check _setup_ { for ( devfn = 0; devfn < 256; devfn++ ) { - struct pci_dev *pdev = pci_get_pdev(pseg->nr, bus, devfn); + struct pci_dev *pdev = pci_get_pdev(NULL, + PCI_SBDF(pseg->nr, bus, devfn)); if ( !pdev ) continue; @@ -1475,7 +1462,7 @@ static int device_assigned(u16 seg, u8 b int rc = 0; ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(seg, bus, devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); if ( !pdev ) rc = -ENODEV; @@ -1506,7 +1493,7 @@ static int assign_device(struct domain * /* device_assigned() should already have cleared the device for assignment */ ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(seg, bus, devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); ASSERT(pdev && (pdev->domain == hardware_domain || pdev->domain == dom_io)); --- a/xen/drivers/passthrough/vtd/quirks.c +++ b/xen/drivers/passthrough/vtd/quirks.c @@ -417,7 +417,7 @@ static int __must_check map_me_phantom_f int rc; /* find ME VT-d engine base on a real ME device */ - pdev = pci_get_pdev(0, 0, PCI_DEVFN(dev, 0)); + pdev = pci_get_pdev(NULL, PCI_SBDF(0, 0, dev, 0)); drhd = acpi_find_matched_drhd_unit(pdev); /* map or unmap ME phantom function */ --- a/xen/drivers/video/vga.c +++ b/xen/drivers/video/vga.c @@ -118,7 +118,7 @@ void __init video_endboot(void) u8 b = bus, df = devfn, sb; pcidevs_lock(); - pdev = pci_get_pdev(0, bus, devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(0, bus, devfn)); pcidevs_unlock(); if ( !pdev || --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -325,7 +325,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsi } /* Find the PCI dev matching the address. */ - pdev = pci_get_pdev_by_domain(d, sbdf.seg, sbdf.bus, sbdf.devfn); + pdev = pci_get_pdev(d, sbdf); if ( !pdev ) return vpci_read_hw(sbdf, reg, size); @@ -435,7 +435,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigne * Find the PCI dev matching the address. * Passthrough everything that's not trapped. */ - pdev = pci_get_pdev_by_domain(d, sbdf.seg, sbdf.bus, sbdf.devfn); + pdev = pci_get_pdev(d, sbdf); if ( !pdev ) { vpci_write_hw(sbdf, reg, size, data); --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -177,10 +177,8 @@ int pci_add_device(u16 seg, u8 bus, u8 d int pci_remove_device(u16 seg, u8 bus, u8 devfn); int pci_ro_device(int seg, int bus, int devfn); int pci_hide_device(unsigned int seg, unsigned int bus, unsigned int devfn); -struct pci_dev *pci_get_pdev(uint16_t seg, uint8_t bus, uint8_t devfn); +struct pci_dev *pci_get_pdev(const struct domain *d, pci_sbdf_t sbdf); struct pci_dev *pci_get_real_pdev(int seg, int bus, int devfn); -struct pci_dev *pci_get_pdev_by_domain(const struct domain *, uint16_t seg, - uint8_t bus, uint8_t devfn); void pci_check_disable_device(u16 seg, u8 bus, u8 devfn); uint8_t pci_conf_read8(pci_sbdf_t sbdf, unsigned int reg);
Rename the latter, subsuming the functionality of the former when passed NULL as first argument. Since this requires touching all call sites anyway, take the opportunity and fold the remaining three parameters into a single pci_sbdf_t one. No functional change intended. In particular the locking related assertion needs to continue to be kept silent when a non-NULL domain pointer is passed - both vpci_read() and vpci_write() call the function without holding the lock (adding respective locking to vPCI [or finding an alternative to doing so] is the topic of a separate series). Signed-off-by: Jan Beulich <jbeulich@suse.com> --- v2: New.