| Message ID | 20210709123813.8700-2-ameynarkhede03@gmail.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
| Series | Expose and manage PCI device reset | expand |
On Fri, 9 Jul 2021 18:08:06 +0530 Amey Narkhede <ameynarkhede03@gmail.com> wrote: > Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe > FLR to avoid reading PCI_EXP_DEVCAP multiple times. > > Currently there is separate function pcie_has_flr() to probe if PCIe FLR > is supported by the device which does not match the calling convention > followed by reset methods which use second function argument to decide > whether to probe or not. Add new function pcie_reset_flr() that follows > the calling convention of reset methods. > > Signed-off-by: Amey Narkhede <ameynarkhede03@gmail.com> > --- > drivers/crypto/cavium/nitrox/nitrox_main.c | 4 +- > drivers/pci/pci.c | 59 +++++++++++----------- > drivers/pci/pcie/aer.c | 12 ++--- > drivers/pci/probe.c | 6 ++- > drivers/pci/quirks.c | 9 ++-- > include/linux/pci.h | 3 +- > 6 files changed, 45 insertions(+), 48 deletions(-) Looks good to me, Reviewed-by: Alex Williamson <alex.williamson@redhat.com>
On Fri, Jul 09, 2021 at 06:08:06PM +0530, Amey Narkhede wrote: > Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe > FLR to avoid reading PCI_EXP_DEVCAP multiple times. > > Currently there is separate function pcie_has_flr() to probe if PCIe FLR > is supported by the device which does not match the calling convention > followed by reset methods which use second function argument to decide > whether to probe or not. Add new function pcie_reset_flr() that follows > the calling convention of reset methods. > > Signed-off-by: Amey Narkhede <ameynarkhede03@gmail.com> > --- > drivers/crypto/cavium/nitrox/nitrox_main.c | 4 +- > drivers/pci/pci.c | 59 +++++++++++----------- > drivers/pci/pcie/aer.c | 12 ++--- > drivers/pci/probe.c | 6 ++- > drivers/pci/quirks.c | 9 ++-- > include/linux/pci.h | 3 +- > 6 files changed, 45 insertions(+), 48 deletions(-) > > diff --git a/drivers/crypto/cavium/nitrox/nitrox_main.c b/drivers/crypto/cavium/nitrox/nitrox_main.c > index facc8e6bc..15d6c8452 100644 > --- a/drivers/crypto/cavium/nitrox/nitrox_main.c > +++ b/drivers/crypto/cavium/nitrox/nitrox_main.c > @@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev) > return -ENOMEM; > } > > - /* check flr support */ > - if (pcie_has_flr(pdev)) > - pcie_flr(pdev); > + pcie_reset_flr(pdev, 0); I'm not really a fan of exposing the "probe" argument outside drivers/pci/. I think this would be the only occurrence. Is there a way to avoid that? Can we just make pcie_flr() do the equivalent of pcie_has_flr() internally? > static int delay_250ms_after_flr(struct pci_dev *dev, int probe) > { > - if (!pcie_has_flr(dev)) > - return -ENOTTY; > + int ret = pcie_reset_flr(dev, probe); > > if (probe) > - return 0; > - > - pcie_flr(dev); > + return ret; > > msleep(250); Can we structure this like the following? I think it's easier to understand. if (probe) return pcie_reset_flr(dev, 1); pcie_reset_flr(dev, 0); msleep(250); return 0; > diff --git a/include/linux/pci.h b/include/linux/pci.h > index c20211e59..d432428fd 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -337,6 +337,7 @@ struct pci_dev { > u8 msi_cap; /* MSI capability offset */ > u8 msix_cap; /* MSI-X capability offset */ > u8 pcie_mpss:3; /* PCIe Max Payload Size Supported */ > + u8 has_pcie_flr:1; /* PCIe FLR supported */ Let's add a devcap member instead. Then we can use it for some ASPM-related things as well. We *could* use it to replace pcie_mpss, since that comes from PCI_EXP_DEVCAP, too, but for now I think it's easier to just keep it because it's encoded, and some drivers and quirks use it so it would be a fair amount of work to change that. Example patch below that could become the first in the series. diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index aacf575c15cf..5a99061ea53a 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4635,8 +4635,7 @@ bool pcie_has_flr(struct pci_dev *dev) if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) return false; - pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap); - return cap & PCI_EXP_DEVCAP_FLR; + return dev->devcap & PCI_EXP_DEVCAP_FLR; } EXPORT_SYMBOL_GPL(pcie_has_flr); diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 79177ac37880..52ae26bcc68c 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1498,8 +1498,8 @@ void set_pcie_port_type(struct pci_dev *pdev) pdev->pcie_cap = pos; pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16); pdev->pcie_flags_reg = reg16; - pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, ®16); - pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD; + pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap); + pdev->pcie_mpss = pdev->devcap & PCI_EXP_DEVCAP_PAYLOAD; parent = pci_upstream_bridge(pdev); if (!parent) diff --git a/include/linux/pci.h b/include/linux/pci.h index 540b377ca8f6..294d1c857a57 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -334,6 +334,7 @@ struct pci_dev { struct pci_dev *rcec; /* Associated RCEC device */ #endif u8 pcie_cap; /* PCIe capability offset */ + u32 devcap; /* PCIe Device Capabilities */ u8 msi_cap; /* MSI capability offset */ u8 msix_cap; /* MSI-X capability offset */ u8 pcie_mpss:3; /* PCIe Max Payload Size Supported */
Hi Bjorn, On 7/27/21 5:12 PM, Bjorn Helgaas wrote: > External email: Use caution opening links or attachments > > > On Fri, Jul 09, 2021 at 06:08:06PM +0530, Amey Narkhede wrote: >> Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe >> FLR to avoid reading PCI_EXP_DEVCAP multiple times. >> >> Currently there is separate function pcie_has_flr() to probe if PCIe FLR >> is supported by the device which does not match the calling convention >> followed by reset methods which use second function argument to decide >> whether to probe or not. Add new function pcie_reset_flr() that follows >> the calling convention of reset methods. >> >> Signed-off-by: Amey Narkhede <ameynarkhede03@gmail.com> >> --- >> drivers/crypto/cavium/nitrox/nitrox_main.c | 4 +- >> drivers/pci/pci.c | 59 +++++++++++----------- >> drivers/pci/pcie/aer.c | 12 ++--- >> drivers/pci/probe.c | 6 ++- >> drivers/pci/quirks.c | 9 ++-- >> include/linux/pci.h | 3 +- >> 6 files changed, 45 insertions(+), 48 deletions(-) >> >> diff --git a/drivers/crypto/cavium/nitrox/nitrox_main.c b/drivers/crypto/cavium/nitrox/nitrox_main.c >> index facc8e6bc..15d6c8452 100644 >> --- a/drivers/crypto/cavium/nitrox/nitrox_main.c >> +++ b/drivers/crypto/cavium/nitrox/nitrox_main.c >> @@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev) >> return -ENOMEM; >> } >> >> - /* check flr support */ >> - if (pcie_has_flr(pdev)) >> - pcie_flr(pdev); >> + pcie_reset_flr(pdev, 0); > I'm not really a fan of exposing the "probe" argument outside > drivers/pci/. I think this would be the only occurrence. Is there a > way to avoid that? > > Can we just make pcie_flr() do the equivalent of pcie_has_flr() > internally? > I like your suggestion don't change the existing definition of pcie_has_flr()/pcie_flr() and define a new function pcie_reset_flr() to fit into the reset framework. This way no need to modify logic/drivers outside of driver/pci/xxx. int pcie_reset_flr(struct pci_dev *dev, int probe) { if (!pcie_has_flr(dev)) return -ENOTTY; if (probe) return 0; return pcie_flr(dev); } And add a new patch to begging of the series for caching 'devcap' in pci_dev structure. --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -333,6 +333,7 @@ struct pci_dev { struct rcec_ea *rcec_ea; /* RCEC cached endpoint association */ struct pci_dev *rcec; /* Associated RCEC device */ #endif + u32 devcap; /* Cached PCIe device capabilities */ u8 pcie_cap; /* PCIe capability offset */ u8 msi_cap; /* MSI capability offset */ u8 msix_cap; /* MSI-X capability offset */ --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -31,6 +31,7 @@ #include <linux/vmalloc.h> #include <asm/dma.h> #include <linux/aer.h> +#include <linux/bitfield.h> #include "pci.h" DEFINE_MUTEX(pci_slot_mutex); @@ -4630,13 +4631,10 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction); */ bool pcie_has_flr(struct pci_dev *dev) { - u32 cap; - if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) return false; - pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap); - return cap & PCI_EXP_DEVCAP_FLR; + return !!FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap); } --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -19,6 +19,7 @@ #include <linux/hypervisor.h> #include <linux/irqdomain.h> #include <linux/pm_runtime.h> +#include <linux/bitfield.h> #include "pci.h" #define CARDBUS_LATENCY_TIMER 176 /* secondary latency timer */ @@ -1498,8 +1499,8 @@ void set_pcie_port_type(struct pci_dev *pdev) pdev->pcie_cap = pos; pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16); pdev->pcie_flags_reg = reg16; - pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, ®16); - pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD; + pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap); + pdev->pcie_mpss = FIELD_GET(PCI_EXP_DEVCAP_PAYLOAD, pdev->devcap);
On Wed, Jul 28, 2021 at 01:54:16PM -0500, Shanker R Donthineni wrote: > On 7/27/21 5:12 PM, Bjorn Helgaas wrote: > > On Fri, Jul 09, 2021 at 06:08:06PM +0530, Amey Narkhede wrote: > >> Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe > >> FLR to avoid reading PCI_EXP_DEVCAP multiple times. > >> > >> Currently there is separate function pcie_has_flr() to probe if PCIe FLR > >> is supported by the device which does not match the calling convention > >> followed by reset methods which use second function argument to decide > >> whether to probe or not. Add new function pcie_reset_flr() that follows > >> the calling convention of reset methods. > >> > >> Signed-off-by: Amey Narkhede <ameynarkhede03@gmail.com> > >> --- > >> drivers/crypto/cavium/nitrox/nitrox_main.c | 4 +- > >> drivers/pci/pci.c | 59 +++++++++++----------- > >> drivers/pci/pcie/aer.c | 12 ++--- > >> drivers/pci/probe.c | 6 ++- > >> drivers/pci/quirks.c | 9 ++-- > >> include/linux/pci.h | 3 +- > >> 6 files changed, 45 insertions(+), 48 deletions(-) > >> > >> diff --git a/drivers/crypto/cavium/nitrox/nitrox_main.c b/drivers/crypto/cavium/nitrox/nitrox_main.c > >> index facc8e6bc..15d6c8452 100644 > >> --- a/drivers/crypto/cavium/nitrox/nitrox_main.c > >> +++ b/drivers/crypto/cavium/nitrox/nitrox_main.c > >> @@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev) > >> return -ENOMEM; > >> } > >> > >> - /* check flr support */ > >> - if (pcie_has_flr(pdev)) > >> - pcie_flr(pdev); > >> + pcie_reset_flr(pdev, 0); > > I'm not really a fan of exposing the "probe" argument outside > > drivers/pci/. I think this would be the only occurrence. Is there a > > way to avoid that? > > > > Can we just make pcie_flr() do the equivalent of pcie_has_flr() > > internally? > > > I like your suggestion don't change the existing definition of > pcie_has_flr()/pcie_flr() and define a new function pcie_reset_flr() > to fit into the reset framework. This way no need to modify > logic/drivers outside of driver/pci/xxx. > > int pcie_reset_flr(struct pci_dev *dev, int probe) > { > if (!pcie_has_flr(dev)) > return -ENOTTY; > > if (probe) > return 0; > > return pcie_flr(dev); > } Can't remember the whole context of this in the series, but I assume this would be static? > And add a new patch to begging of the series for caching 'devcap' in > pci_dev structure. > > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -333,6 +333,7 @@ struct pci_dev { > struct rcec_ea *rcec_ea; /* RCEC cached endpoint association */ > struct pci_dev *rcec; /* Associated RCEC device */ > #endif > + u32 devcap; /* Cached PCIe device capabilities */ > u8 pcie_cap; /* PCIe capability offset */ > u8 msi_cap; /* MSI capability offset */ > u8 msix_cap; /* MSI-X capability offset */ > > > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -31,6 +31,7 @@ > #include <linux/vmalloc.h> > #include <asm/dma.h> > #include <linux/aer.h> > +#include <linux/bitfield.h> > #include "pci.h" > > DEFINE_MUTEX(pci_slot_mutex); > @@ -4630,13 +4631,10 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction); > */ > bool pcie_has_flr(struct pci_dev *dev) > { > - u32 cap; > - > if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) > return false; > > - pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap); > - return cap & PCI_EXP_DEVCAP_FLR; > + return !!FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap); Nice, thanks for reminding me of FIELD_GET(). I like how that works without having to #define *_SHIFT values. I personally don't care for "!!" and would probably write something like: return FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap) == 1; > } > > --- a/drivers/pci/probe.c > +++ b/drivers/pci/probe.c > @@ -19,6 +19,7 @@ > #include <linux/hypervisor.h> > #include <linux/irqdomain.h> > #include <linux/pm_runtime.h> > +#include <linux/bitfield.h> > #include "pci.h" > > #define CARDBUS_LATENCY_TIMER 176 /* secondary latency timer */ > @@ -1498,8 +1499,8 @@ void set_pcie_port_type(struct pci_dev *pdev) > pdev->pcie_cap = pos; > pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16); > pdev->pcie_flags_reg = reg16; > - pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, ®16); > - pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD; > + pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap); > + pdev->pcie_mpss = FIELD_GET(PCI_EXP_DEVCAP_PAYLOAD, pdev->devcap); >
Hi Bjorn, On 7/28/21 3:23 PM, Bjorn Helgaas wrote: > External email: Use caution opening links or attachments > > > On Wed, Jul 28, 2021 at 01:54:16PM -0500, Shanker R Donthineni wrote: >> On 7/27/21 5:12 PM, Bjorn Helgaas wrote: >>> On Fri, Jul 09, 2021 at 06:08:06PM +0530, Amey Narkhede wrote: >>>> Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe >>>> FLR to avoid reading PCI_EXP_DEVCAP multiple times. >>>> >>>> Currently there is separate function pcie_has_flr() to probe if PCIe FLR >>>> is supported by the device which does not match the calling convention >>>> followed by reset methods which use second function argument to decide >>>> whether to probe or not. Add new function pcie_reset_flr() that follows >>>> the calling convention of reset methods. >>>> >>>> Signed-off-by: Amey Narkhede <ameynarkhede03@gmail.com> >>>> --- >>>> drivers/crypto/cavium/nitrox/nitrox_main.c | 4 +- >>>> drivers/pci/pci.c | 59 +++++++++++----------- >>>> drivers/pci/pcie/aer.c | 12 ++--- >>>> drivers/pci/probe.c | 6 ++- >>>> drivers/pci/quirks.c | 9 ++-- >>>> include/linux/pci.h | 3 +- >>>> 6 files changed, 45 insertions(+), 48 deletions(-) >>>> >>>> diff --git a/drivers/crypto/cavium/nitrox/nitrox_main.c b/drivers/crypto/cavium/nitrox/nitrox_main.c >>>> index facc8e6bc..15d6c8452 100644 >>>> --- a/drivers/crypto/cavium/nitrox/nitrox_main.c >>>> +++ b/drivers/crypto/cavium/nitrox/nitrox_main.c >>>> @@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev) >>>> return -ENOMEM; >>>> } >>>> >>>> - /* check flr support */ >>>> - if (pcie_has_flr(pdev)) >>>> - pcie_flr(pdev); >>>> + pcie_reset_flr(pdev, 0); >>> I'm not really a fan of exposing the "probe" argument outside >>> drivers/pci/. I think this would be the only occurrence. Is there a >>> way to avoid that? >>> >>> Can we just make pcie_flr() do the equivalent of pcie_has_flr() >>> internally? >>> >> I like your suggestion don't change the existing definition of >> pcie_has_flr()/pcie_flr() and define a new function pcie_reset_flr() >> to fit into the reset framework. This way no need to modify >> logic/drivers outside of driver/pci/xxx. >> >> int pcie_reset_flr(struct pci_dev *dev, int probe) >> { >> if (!pcie_has_flr(dev)) >> return -ENOTTY; >> >> if (probe) >> return 0; >> >> return pcie_flr(dev); >> } > Can't remember the whole context of this in the series, but I assume > this would be static? It should be static since it's referenced in driver/pci/qrirk.c and aer.c. >> And add a new patch to begging of the series for caching 'devcap' in >> pci_dev structure. >> >> --- a/include/linux/pci.h >> +++ b/include/linux/pci.h >> @@ -333,6 +333,7 @@ struct pci_dev { >> struct rcec_ea *rcec_ea; /* RCEC cached endpoint association */ >> struct pci_dev *rcec; /* Associated RCEC device */ >> #endif >> + u32 devcap; /* Cached PCIe device capabilities */ >> u8 pcie_cap; /* PCIe capability offset */ >> u8 msi_cap; /* MSI capability offset */ >> u8 msix_cap; /* MSI-X capability offset */ >> >> >> --- a/drivers/pci/pci.c >> +++ b/drivers/pci/pci.c >> @@ -31,6 +31,7 @@ >> #include <linux/vmalloc.h> >> #include <asm/dma.h> >> #include <linux/aer.h> >> +#include <linux/bitfield.h> >> #include "pci.h" >> >> DEFINE_MUTEX(pci_slot_mutex); >> @@ -4630,13 +4631,10 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction); >> */ >> bool pcie_has_flr(struct pci_dev *dev) >> { >> - u32 cap; >> - >> if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) >> return false; >> >> - pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap); >> - return cap & PCI_EXP_DEVCAP_FLR; >> + return !!FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap); > Nice, thanks for reminding me of FIELD_GET(). I like how that works > without having to #define *_SHIFT values. I personally don't care for > "!!" and would probably write something like: > > return FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap) == 1; Both are same since FLR is a single bit value. >> } >> >> --- a/drivers/pci/probe.c >> +++ b/drivers/pci/probe.c >> @@ -19,6 +19,7 @@ >> #include <linux/hypervisor.h> >> #include <linux/irqdomain.h> >> #include <linux/pm_runtime.h> >> +#include <linux/bitfield.h> >> #include "pci.h" >> >> #define CARDBUS_LATENCY_TIMER 176 /* secondary latency timer */ >> @@ -1498,8 +1499,8 @@ void set_pcie_port_type(struct pci_dev *pdev) >> pdev->pcie_cap = pos; >> pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16); >> pdev->pcie_flags_reg = reg16; >> - pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, ®16); >> - pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD; >> + pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap); >> + pdev->pcie_mpss = FIELD_GET(PCI_EXP_DEVCAP_PAYLOAD, pdev->devcap); >>
On 7/28/21 4:58 PM, Shanker R Donthineni wrote: > External email: Use caution opening links or attachments > > > Hi Bjorn, > > On 7/28/21 3:23 PM, Bjorn Helgaas wrote: >> External email: Use caution opening links or attachments >> >> >> On Wed, Jul 28, 2021 at 01:54:16PM -0500, Shanker R Donthineni wrote: >>> On 7/27/21 5:12 PM, Bjorn Helgaas wrote: >>>> On Fri, Jul 09, 2021 at 06:08:06PM +0530, Amey Narkhede wrote: >>>>> Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe >>>>> FLR to avoid reading PCI_EXP_DEVCAP multiple times. >>>>> >>>>> Currently there is separate function pcie_has_flr() to probe if PCIe FLR >>>>> is supported by the device which does not match the calling convention >>>>> followed by reset methods which use second function argument to decide >>>>> whether to probe or not. Add new function pcie_reset_flr() that follows >>>>> the calling convention of reset methods. >>>>> >>>>> Signed-off-by: Amey Narkhede <ameynarkhede03@gmail.com> >>>>> --- >>>>> drivers/crypto/cavium/nitrox/nitrox_main.c | 4 +- >>>>> drivers/pci/pci.c | 59 +++++++++++----------- >>>>> drivers/pci/pcie/aer.c | 12 ++--- >>>>> drivers/pci/probe.c | 6 ++- >>>>> drivers/pci/quirks.c | 9 ++-- >>>>> include/linux/pci.h | 3 +- >>>>> 6 files changed, 45 insertions(+), 48 deletions(-) >>>>> >>>>> diff --git a/drivers/crypto/cavium/nitrox/nitrox_main.c b/drivers/crypto/cavium/nitrox/nitrox_main.c >>>>> index facc8e6bc..15d6c8452 100644 >>>>> --- a/drivers/crypto/cavium/nitrox/nitrox_main.c >>>>> +++ b/drivers/crypto/cavium/nitrox/nitrox_main.c >>>>> @@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev) >>>>> return -ENOMEM; >>>>> } >>>>> >>>>> - /* check flr support */ >>>>> - if (pcie_has_flr(pdev)) >>>>> - pcie_flr(pdev); >>>>> + pcie_reset_flr(pdev, 0); >>>> I'm not really a fan of exposing the "probe" argument outside >>>> drivers/pci/. I think this would be the only occurrence. Is there a >>>> way to avoid that? >>>> >>>> Can we just make pcie_flr() do the equivalent of pcie_has_flr() >>>> internally? >>>> >>> I like your suggestion don't change the existing definition of >>> pcie_has_flr()/pcie_flr() and define a new function pcie_reset_flr() >>> to fit into the reset framework. This way no need to modify >>> logic/drivers outside of driver/pci/xxx. >>> >>> int pcie_reset_flr(struct pci_dev *dev, int probe) >>> { >>> if (!pcie_has_flr(dev)) >>> return -ENOTTY; >>> >>> if (probe) >>> return 0; >>> >>> return pcie_flr(dev); >>> } >> Can't remember the whole context of this in the series, but I assume >> this would be static? > It should be static since it's referenced in driver/pci/qrirk.c and aer.c. > Sorry it should not be static since it's referenced in driver/pci/quirk.c and aer.c
On Wed, Jul 28, 2021 at 04:58:14PM -0500, Shanker R Donthineni wrote: > On 7/28/21 3:23 PM, Bjorn Helgaas wrote: > > On Wed, Jul 28, 2021 at 01:54:16PM -0500, Shanker R Donthineni wrote: > >> On 7/27/21 5:12 PM, Bjorn Helgaas wrote: > >>> On Fri, Jul 09, 2021 at 06:08:06PM +0530, Amey Narkhede wrote: > >>>> Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe > >>>> FLR to avoid reading PCI_EXP_DEVCAP multiple times. > >> - pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap); > >> - return cap & PCI_EXP_DEVCAP_FLR; > >> + return !!FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap); > > > > Nice, thanks for reminding me of FIELD_GET(). I like how that works > > without having to #define *_SHIFT values. I personally don't care for > > "!!" and would probably write something like: > > > > return FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap) == 1; > > Both are same since FLR is a single bit value. Right; this is a style preference, not a correctness question.
diff --git a/drivers/crypto/cavium/nitrox/nitrox_main.c b/drivers/crypto/cavium/nitrox/nitrox_main.c index facc8e6bc..15d6c8452 100644 --- a/drivers/crypto/cavium/nitrox/nitrox_main.c +++ b/drivers/crypto/cavium/nitrox/nitrox_main.c @@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev) return -ENOMEM; } - /* check flr support */ - if (pcie_has_flr(pdev)) - pcie_flr(pdev); + pcie_reset_flr(pdev, 0); pci_restore_state(pdev); diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 452351025..fefa6d7b3 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4611,32 +4611,12 @@ int pci_wait_for_pending_transaction(struct pci_dev *dev) } EXPORT_SYMBOL(pci_wait_for_pending_transaction); -/** - * pcie_has_flr - check if a device supports function level resets - * @dev: device to check - * - * Returns true if the device advertises support for PCIe function level - * resets. - */ -bool pcie_has_flr(struct pci_dev *dev) -{ - u32 cap; - - if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) - return false; - - pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap); - return cap & PCI_EXP_DEVCAP_FLR; -} -EXPORT_SYMBOL_GPL(pcie_has_flr); - /** * pcie_flr - initiate a PCIe function level reset * @dev: device to reset * - * Initiate a function level reset on @dev. The caller should ensure the - * device supports FLR before calling this function, e.g. by using the - * pcie_has_flr() helper. + * Initiate a function level reset unconditionally on @dev without + * checking any flags and DEVCAP */ int pcie_flr(struct pci_dev *dev) { @@ -4659,6 +4639,28 @@ int pcie_flr(struct pci_dev *dev) } EXPORT_SYMBOL_GPL(pcie_flr); +/** + * pcie_reset_flr - initiate a PCIe function level reset + * @dev: device to reset + * @probe: If set, only check if the device can be reset this way. + * + * Initiate a function level reset on @dev. + */ +int pcie_reset_flr(struct pci_dev *dev, int probe) +{ + if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) + return -ENOTTY; + + if (!dev->has_pcie_flr) + return -ENOTTY; + + if (probe) + return 0; + + return pcie_flr(dev); +} +EXPORT_SYMBOL_GPL(pcie_reset_flr); + static int pci_af_flr(struct pci_dev *dev, int probe) { int pos; @@ -5139,11 +5141,9 @@ int __pci_reset_function_locked(struct pci_dev *dev) rc = pci_dev_specific_reset(dev, 0); if (rc != -ENOTTY) return rc; - if (pcie_has_flr(dev)) { - rc = pcie_flr(dev); - if (rc != -ENOTTY) - return rc; - } + rc = pcie_reset_flr(dev, 0); + if (rc != -ENOTTY) + return rc; rc = pci_af_flr(dev, 0); if (rc != -ENOTTY) return rc; @@ -5174,8 +5174,9 @@ int pci_probe_reset_function(struct pci_dev *dev) rc = pci_dev_specific_reset(dev, 1); if (rc != -ENOTTY) return rc; - if (pcie_has_flr(dev)) - return 0; + rc = pcie_reset_flr(dev, 1); + if (rc != -ENOTTY) + return rc; rc = pci_af_flr(dev, 1); if (rc != -ENOTTY) return rc; diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index ec943cee5..98077595a 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1405,13 +1405,11 @@ static pci_ers_result_t aer_root_reset(struct pci_dev *dev) } if (type == PCI_EXP_TYPE_RC_EC || type == PCI_EXP_TYPE_RC_END) { - if (pcie_has_flr(dev)) { - rc = pcie_flr(dev); - pci_info(dev, "has been reset (%d)\n", rc); - } else { - pci_info(dev, "not reset (no FLR support)\n"); - rc = -ENOTTY; - } + rc = pcie_reset_flr(dev, 0); + if (!rc) + pci_info(dev, "has been reset\n"); + else + pci_info(dev, "not reset (no FLR support: %d)\n", rc); } else { rc = pci_bus_error_reset(dev); pci_info(dev, "%s Port link has been reset (%d)\n", diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 3a62d09b8..072a3d4dc 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1487,6 +1487,7 @@ void set_pcie_port_type(struct pci_dev *pdev) { int pos; u16 reg16; + u32 reg32; int type; struct pci_dev *parent; @@ -1497,8 +1498,9 @@ void set_pcie_port_type(struct pci_dev *pdev) pdev->pcie_cap = pos; pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16); pdev->pcie_flags_reg = reg16; - pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, ®16); - pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD; + pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, ®32); + pdev->pcie_mpss = reg32 & PCI_EXP_DEVCAP_PAYLOAD; + pdev->has_pcie_flr = !!(reg32 & PCI_EXP_DEVCAP_FLR); parent = pci_upstream_bridge(pdev); if (!parent) diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index d85914afe..f977ba79a 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -3819,7 +3819,7 @@ static int nvme_disable_and_flr(struct pci_dev *dev, int probe) u32 cfg; if (dev->class != PCI_CLASS_STORAGE_EXPRESS || - !pcie_has_flr(dev) || !pci_resource_start(dev, 0)) + pcie_reset_flr(dev, 1) || !pci_resource_start(dev, 0)) return -ENOTTY; if (probe) @@ -3888,13 +3888,10 @@ static int nvme_disable_and_flr(struct pci_dev *dev, int probe) */ static int delay_250ms_after_flr(struct pci_dev *dev, int probe) { - if (!pcie_has_flr(dev)) - return -ENOTTY; + int ret = pcie_reset_flr(dev, probe); if (probe) - return 0; - - pcie_flr(dev); + return ret; msleep(250); diff --git a/include/linux/pci.h b/include/linux/pci.h index c20211e59..d432428fd 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -337,6 +337,7 @@ struct pci_dev { u8 msi_cap; /* MSI capability offset */ u8 msix_cap; /* MSI-X capability offset */ u8 pcie_mpss:3; /* PCIe Max Payload Size Supported */ + u8 has_pcie_flr:1; /* PCIe FLR supported */ u8 rom_base_reg; /* Config register controlling ROM */ u8 pin; /* Interrupt pin this device uses */ u16 pcie_flags_reg; /* Cached PCIe Capabilities Register */ @@ -1225,7 +1226,7 @@ u32 pcie_bandwidth_available(struct pci_dev *dev, struct pci_dev **limiting_dev, enum pci_bus_speed *speed, enum pcie_link_width *width); void pcie_print_link_status(struct pci_dev *dev); -bool pcie_has_flr(struct pci_dev *dev); +int pcie_reset_flr(struct pci_dev *dev, int probe); int pcie_flr(struct pci_dev *dev); int __pci_reset_function_locked(struct pci_dev *dev); int pci_reset_function(struct pci_dev *dev);
Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times. Currently there is separate function pcie_has_flr() to probe if PCIe FLR is supported by the device which does not match the calling convention followed by reset methods which use second function argument to decide whether to probe or not. Add new function pcie_reset_flr() that follows the calling convention of reset methods. Signed-off-by: Amey Narkhede <ameynarkhede03@gmail.com> --- drivers/crypto/cavium/nitrox/nitrox_main.c | 4 +- drivers/pci/pci.c | 59 +++++++++++----------- drivers/pci/pcie/aer.c | 12 ++--- drivers/pci/probe.c | 6 ++- drivers/pci/quirks.c | 9 ++-- include/linux/pci.h | 3 +- 6 files changed, 45 insertions(+), 48 deletions(-)