diff mbox series

[v6,1/7] PCI: Expose reset_type to users of __pci_reset_function_locked()

Message ID 20181019021132.14743-1-okaya@kernel.org (mailing list archive)
State New, archived
Delegated to: Bjorn Helgaas
Headers show
Series [v6,1/7] PCI: Expose reset_type to users of __pci_reset_function_locked() | expand

Commit Message

Sinan Kaya Oct. 19, 2018, 2:11 a.m. UTC
We need a contract between the reset API users and the PCI core about the
types of reset that a user needs vs. what PCI core can do internally.
If a platform supports hotplug, we need to do hotplug reset as an example.

Expose the reset types to the drivers and try different reset types based
on the new reset_type parameter.

Most users are expected to use PCI_RESET_ANY, PCI_RESET_FUNC or
PCI_RESET_LINK parameters.

Link: https://www.spinics.net/lists/linux-pci/msg75828.html
Suggested-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Sinan Kaya <okaya@kernel.org>
---
 .../net/ethernet/cavium/liquidio/lio_main.c   |  2 +-
 drivers/pci/pci.c                             | 59 ++++++++++++-------
 drivers/xen/xen-pciback/pci_stub.c            |  6 +-
 include/linux/pci.h                           | 58 +++++++++++++++++-
 4 files changed, 100 insertions(+), 25 deletions(-)

Comments

Bjorn Helgaas Oct. 20, 2018, 2:09 a.m. UTC | #1
This series (and v5, it looks like) lost the cover letter, which had a
nice diffstat overview :)

On Fri, Oct 19, 2018 at 02:11:21AM +0000, Sinan Kaya wrote:
> We need a contract between the reset API users and the PCI core about the
> types of reset that a user needs vs. what PCI core can do internally.
> If a platform supports hotplug, we need to do hotplug reset as an example.

Somewhere this needs a description of a bug that's being fixed or some
other justification, e.g., code simplification.  The above is a little
too abstract for me to grasp it.

> Expose the reset types to the drivers and try different reset types based
> on the new reset_type parameter.
> 
> Most users are expected to use PCI_RESET_ANY, PCI_RESET_FUNC or
> PCI_RESET_LINK parameters.

There are fewer than a dozen callers of all these functions and the
complication of these interfaces doesn't seem commensurate with the
problem.  With six different interfaces and five independent bit
flags, the possibilities are way more than necessary.

It seems like there are only three main cases (possibly extended by
locked/unlocked versions):

  - a generic reset, used by ipc, genwqe, qlcnic, sfc, liquidio,
    mwifiex, xen

  - something special for VFIO that is affected by the set of devices
    owned by the user

  - a link reset for places like these where a device needs help to
    train the link correctly:

      cik_pcie_gen3_enable()       # amdgpu, radeon
      si_pcie_gen3_enable()        # amdgpu, radeon
      do_pcie_gen3_transition()    # infiniband/hfi1

The interface for the generic case should be simple and made to be the
obvious choice for drivers, i.e., requires only the pci_dev and no
magic flags.  The link reset case is different enough that it might
deserve its own special interface.

> Link: https://www.spinics.net/lists/linux-pci/msg75828.html

Can you switch to using https://lore.kernel.org/linux-pci/<Message-ID>
URLs so we don't depend on 3rd-party archives like spinics?  See
https://www.kernel.org/lore.html .  I usually silently convert to the
lore URLs, but I guess doing it silently only makes more work for
myself.

Bjorn
Sinan Kaya Oct. 20, 2018, 2:58 a.m. UTC | #2
On 10/19/2018 10:09 PM, Bjorn Helgaas wrote:
> This series (and v5, it looks like) lost the cover letter, which had a
> nice diffstat overview :)

Yeah, I was too lazy. I was rushing to get it out of my laptop before I go
back to my daily work.

> 
> On Fri, Oct 19, 2018 at 02:11:21AM +0000, Sinan Kaya wrote:
>> We need a contract between the reset API users and the PCI core about the
>> types of reset that a user needs vs. what PCI core can do internally.
>> If a platform supports hotplug, we need to do hotplug reset as an example.
> 
> Somewhere this needs a description of a bug that's being fixed or some
> other justification, e.g., code simplification.  The above is a little
> too abstract for me to grasp it.
> 

This series was in response to a request from Alex to have external users
some level of control on what PCI core can do internally.

On the other hand, I have been posting patches to remove direct access from
external users to PCI core internals and hide all reset semantics like
save/restore from the users. I like giving less power to the users :)

>> Expose the reset types to the drivers and try different reset types based
>> on the new reset_type parameter.
>>
>> Most users are expected to use PCI_RESET_ANY, PCI_RESET_FUNC or
>> PCI_RESET_LINK parameters.
> 
> There are fewer than a dozen callers of all these functions and the
> complication of these interfaces doesn't seem commensurate with the
> problem.  With six different interfaces and five independent bit
> flags, the possibilities are way more than necessary.
> 

True, I posted an RFC to reduce 5 function reset API flavors to 1 and
rely on the flags.

In the end, there would be two reset APIs only.

> It seems like there are only three main cases (possibly extended by
> locked/unlocked versions):
> 
>    - a generic reset, used by ipc, genwqe, qlcnic, sfc, liquidio,
>      mwifiex, xen
> 
>    - something special for VFIO that is affected by the set of devices
>      owned by the user
> 
>    - a link reset for places like these where a device needs help to
>      train the link correctly:
> 
>        cik_pcie_gen3_enable()       # amdgpu, radeon
>        si_pcie_gen3_enable()        # amdgpu, radeon
>        do_pcie_gen3_transition()    # infiniband/hfi1
> 
> The interface for the generic case should be simple and made to be the
> obvious choice for drivers, i.e., requires only the pci_dev and no
> magic flags.  

Even these generic reset examples are not really generic today. Some of them
call reset from probe path and use the locked reset API 
(pci_reset_function_locked()).

Others call the reset but prefer to not save/restore context.
(__pci_reset_function_locked()). Some drivers even implemented their own
context save/restore code themselves.

There are others that do obtain the lock as well as save/restore context.
(pci_reset_function())

I really don't like having these many reset API flavors and I thought
we could do something about that.

We can also drop the series if we think that current API are good enough
and nuances are well understood.

Alex wanted to have some more control into the reset APIs. Thus, this series
+ the API reduction in RFC from me (again Alex didn't like this much).

> The link reset case is different enough that it might
> deserve its own special interface.
> 
>> Link: https://www.spinics.net/lists/linux-pci/msg75828.html
> 
> Can you switch to using https://lore.kernel.org/linux-pci/<Message-ID>
> URLs so we don't depend on 3rd-party archives like spinics?  See
> https://www.kernel.org/lore.html .  I usually silently convert to the
> lore URLs, but I guess doing it silently only makes more work for
> myself.
> 
> Bjorn
>
Bjorn Helgaas Oct. 20, 2018, 3:03 p.m. UTC | #3
On Fri, Oct 19, 2018 at 10:58:00PM -0400, Sinan Kaya wrote:
> On 10/19/2018 10:09 PM, Bjorn Helgaas wrote:
> > On Fri, Oct 19, 2018 at 02:11:21AM +0000, Sinan Kaya wrote:
> > > We need a contract between the reset API users and the PCI core about the
> > > types of reset that a user needs vs. what PCI core can do internally.
> > > If a platform supports hotplug, we need to do hotplug reset as an example.
> > 
> > Somewhere this needs a description of a bug that's being fixed or some
> > other justification, e.g., code simplification.  The above is a little
> > too abstract for me to grasp it.
> 
> This series was in response to a request from Alex to have external users
> some level of control on what PCI core can do internally.
> 
> On the other hand, I have been posting patches to remove direct access from
> external users to PCI core internals and hide all reset semantics like
> save/restore from the users. I like giving less power to the users :)

That's a good goal.  Along the way we might need to tweak the users to
align their usage to some common cases.

> > > Expose the reset types to the drivers and try different reset types based
> > > on the new reset_type parameter.
> > > 
> > > Most users are expected to use PCI_RESET_ANY, PCI_RESET_FUNC or
> > > PCI_RESET_LINK parameters.
> > 
> > There are fewer than a dozen callers of all these functions and the
> > complication of these interfaces doesn't seem commensurate with the
> > problem.  With six different interfaces and five independent bit
> > flags, the possibilities are way more than necessary.
> 
> True, I posted an RFC to reduce 5 function reset API flavors to 1 and
> rely on the flags.

I don't like the flags.  Even one API with five bit flags has 32
possible combinations, most of which are meaningless.

> Even these generic reset examples are not really generic today. Some of them
> call reset from probe path and use the locked reset API
> (pci_reset_function_locked()).
> 
> Others call the reset but prefer to not save/restore context.
> (__pci_reset_function_locked()). Some drivers even implemented their own
> context save/restore code themselves.
> 
> There are others that do obtain the lock as well as save/restore context.
> (pci_reset_function())
> 
> I really don't like having these many reset API flavors and I thought
> we could do something about that.

I agree we have too many flavors.  I'm not convinced all the variation
in save/restore is necessary; I suspect most of that is accidental and
drivers could probably all live with a single solution.

> We can also drop the series if we think that current API are good enough
> and nuances are well understood.

I don't think the current API is good enough :)  It's just a small
matter of sorting out a better one.

Bjorn
Sinan Kaya Oct. 20, 2018, 4:21 p.m. UTC | #4
On 10/20/2018 11:03 AM, Bjorn Helgaas wrote:
>> We can also drop the series if we think that current API are good enough
>> and nuances are well understood.
> I don't think the current API is good enough:)   It's just a small
> matter of sorting out a better one.

I guess the next logical question is if you have any suggestion.

merge __pci_reset_function_locked() and pci_reset_function_locked().
Then save the context all the time?

What about locked vs. unlocked APIs? If there is a way to know if lock
is being held by the currently running task, we could skip the locking business
and unify these two APIs.
Alex Williamson Nov. 8, 2018, 8:31 p.m. UTC | #5
On Sat, 20 Oct 2018 10:03:53 -0500
Bjorn Helgaas <helgaas@kernel.org> wrote:

> On Fri, Oct 19, 2018 at 10:58:00PM -0400, Sinan Kaya wrote:
> > On 10/19/2018 10:09 PM, Bjorn Helgaas wrote:  
> > > On Fri, Oct 19, 2018 at 02:11:21AM +0000, Sinan Kaya wrote:  
> > > > We need a contract between the reset API users and the PCI core about the
> > > > types of reset that a user needs vs. what PCI core can do internally.
> > > > If a platform supports hotplug, we need to do hotplug reset as an example.  
> > > 
> > > Somewhere this needs a description of a bug that's being fixed or some
> > > other justification, e.g., code simplification.  The above is a little
> > > too abstract for me to grasp it.  
> > 
> > This series was in response to a request from Alex to have external users
> > some level of control on what PCI core can do internally.
> > 
> > On the other hand, I have been posting patches to remove direct access from
> > external users to PCI core internals and hide all reset semantics like
> > save/restore from the users. I like giving less power to the users :)  
> 
> That's a good goal.  Along the way we might need to tweak the users to
> align their usage to some common cases.
> 
> > > > Expose the reset types to the drivers and try different reset types based
> > > > on the new reset_type parameter.
> > > > 
> > > > Most users are expected to use PCI_RESET_ANY, PCI_RESET_FUNC or
> > > > PCI_RESET_LINK parameters.  
> > > 
> > > There are fewer than a dozen callers of all these functions and the
> > > complication of these interfaces doesn't seem commensurate with the
> > > problem.  With six different interfaces and five independent bit
> > > flags, the possibilities are way more than necessary.  
> > 
> > True, I posted an RFC to reduce 5 function reset API flavors to 1 and
> > rely on the flags.  
> 
> I don't like the flags.  Even one API with five bit flags has 32
> possible combinations, most of which are meaningless.
> 
> > Even these generic reset examples are not really generic today. Some of them
> > call reset from probe path and use the locked reset API
> > (pci_reset_function_locked()).
> > 
> > Others call the reset but prefer to not save/restore context.
> > (__pci_reset_function_locked()). Some drivers even implemented their own
> > context save/restore code themselves.
> > 
> > There are others that do obtain the lock as well as save/restore context.
> > (pci_reset_function())
> > 
> > I really don't like having these many reset API flavors and I thought
> > we could do something about that.  
> 
> I agree we have too many flavors.  I'm not convinced all the variation
> in save/restore is necessary; I suspect most of that is accidental and
> drivers could probably all live with a single solution.
> 
> > We can also drop the series if we think that current API are good enough
> > and nuances are well understood.  
> 
> I don't think the current API is good enough :)  It's just a small
> matter of sorting out a better one.

The thing that I was really hoping to fix was what happened in
811c5cb37df4 where we used to have separate bus vs slot reset
interfaces that vfio-pci made use of, but these were abstracted away
such that even though vfio-pci is specifically trying to achieve one or
the other, not just whichever the pci-core chooses to use, we no longer
have an interface to do that.  It doesn't make sense that we have
pci_probe_reset_slot() and pci_probe_reset_bus() exported for drivers
yet only pci_reset_bus() to execute the reset, which actually does a
slot reset, not a bus reset, if it's available (and hopefully it picks
the one that aligns with what the caller was intending).

I agree that we should have the "I don't really know what I'm doing,
please make the correct choice obvious" interface, but at the same time
I don't see the value in penalizing drivers that do know what they're
doing and want to make a specific choice.  In that sense a desire to
give less power to the user (ie. GPL kernel drivers) seems misplaced, I
think we need to aim for a balance where the interface is hard to
misuse, but at the same time retains useful functionality.  It can't
just be one or the other.

I was also in a discussion today that seems like it could leverage
aspects of this series where there's a desire to allow system admin
level influence over the type of reset used for a device.  We can do
this in the kernel via device specific resets, but it's not always the
case that we can develop a universal recipe for blacklisting a reset
mechanism for a given device.  Consider for instance a device where FLR
works well only with some firmware revisions, but different system
vendors might report similar firmware revisions, some afflicted, others
not.

The bar is relatively high to not introduce device regressions and
incorrectly requiring a device to use SBR rather than FLR can have
functionality repercussions.  I know Bjorn has a desire to have things
work well by default and command line options and sysfs tweaking of
devices are obstacles to that, but there are cases where I don't see
that we have a clear path to impose a device specific quirk without
fear of regressions, and some mechanism of letting userspace policy
influence the operation seems a viable path.  IMO it would also be a
useful debugging tool if I could select a specific method of reset for a
device from userspace and perhaps the infrastructure for this might
make it easier to test and implement device specific quirks that simply
need to pick a reset mechanism different from the one the kernel
chooses by default.

The interface in this proposal maybe needs some work, but at some level
it seems we need to consider which resets the device/bus/slot is capable
of, resets blacklisted/selected/prioritized by the user/driver, and
allow the caller to identify the type of reset they want to achieve.
Can we agree that's a desirable goal?  Thanks,

Alex
Bjorn Helgaas Nov. 8, 2018, 9:13 p.m. UTC | #6
On Thu, Nov 08, 2018 at 01:31:47PM -0700, Alex Williamson wrote:
> On Sat, 20 Oct 2018 10:03:53 -0500
> Bjorn Helgaas <helgaas@kernel.org> wrote:
> > I don't think the current API is good enough :)  It's just a small
> > matter of sorting out a better one.
> 
> The thing that I was really hoping to fix was what happened in
> 811c5cb37df4 where we used to have separate bus vs slot reset
> interfaces that vfio-pci made use of, but these were abstracted away
> such that even though vfio-pci is specifically trying to achieve one or
> the other, not just whichever the pci-core chooses to use, we no longer
> have an interface to do that.  It doesn't make sense that we have
> pci_probe_reset_slot() and pci_probe_reset_bus() exported for drivers
> yet only pci_reset_bus() to execute the reset, which actually does a
> slot reset, not a bus reset, if it's available (and hopefully it picks
> the one that aligns with what the caller was intending).
> 
> I agree that we should have the "I don't really know what I'm doing,
> please make the correct choice obvious" interface, but at the same time
> I don't see the value in penalizing drivers that do know what they're
> doing and want to make a specific choice.  In that sense a desire to
> give less power to the user (ie. GPL kernel drivers) seems misplaced, I
> think we need to aim for a balance where the interface is hard to
> misuse, but at the same time retains useful functionality.  It can't
> just be one or the other.
> 
> I was also in a discussion today that seems like it could leverage
> aspects of this series where there's a desire to allow system admin
> level influence over the type of reset used for a device.  We can do
> this in the kernel via device specific resets, but it's not always the
> case that we can develop a universal recipe for blacklisting a reset
> mechanism for a given device.  Consider for instance a device where FLR
> works well only with some firmware revisions, but different system
> vendors might report similar firmware revisions, some afflicted, others
> not.
> 
> The bar is relatively high to not introduce device regressions and
> incorrectly requiring a device to use SBR rather than FLR can have
> functionality repercussions.  I know Bjorn has a desire to have things
> work well by default and command line options and sysfs tweaking of
> devices are obstacles to that, but there are cases where I don't see
> that we have a clear path to impose a device specific quirk without
> fear of regressions, and some mechanism of letting userspace policy
> influence the operation seems a viable path.  IMO it would also be a
> useful debugging tool if I could select a specific method of reset for a
> device from userspace and perhaps the infrastructure for this might
> make it easier to test and implement device specific quirks that simply
> need to pick a reset mechanism different from the one the kernel
> chooses by default.
> 
> The interface in this proposal maybe needs some work, but at some level
> it seems we need to consider which resets the device/bus/slot is capable
> of, resets blacklisted/selected/prioritized by the user/driver, and
> allow the caller to identify the type of reset they want to achieve.
> Can we agree that's a desirable goal?  Thanks,

I can't sign up completely to that whole list because it basically
mandates much of the complexity I want to avoid.

I do certainly agree we need to figure out which resets are supported
by a device.  Along that line, I want to get rid of all the "probe"
arguments.  In almost all cases we can figure out what's supported at
enumeration-time, and I think we should cache that information in a
series of bits in the pci_dev.  We already do much of that in
pci_probe_reset_function(), but we only save overall "can this device
be reset by any method", not the results for each individual method.

Of course, the parent bus reset is slightly tricky because we can't
decide for each device individually; it depends on whether there are
peer devices.  But I think that's still doable.

If we cache each bit, then it's a small step to have quirks or some
sort of sysfs hook to disable specific resets for firmware issues or
sysadmin control.

I don't want to penalize drivers that need to do something specific.
I just don't believe there are very many of those cases.  I think we
should list them explicitly, combine the ones that are needlessly
different, and build something that's sufficient for those cases, but
not overly generalized.

Bjorn
diff mbox series

Patch

diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 6fb13fa73b27..0ff76722734d 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -989,7 +989,7 @@  static void octeon_pci_flr(struct octeon_device *oct)
 	pci_write_config_word(oct->pci_dev, PCI_COMMAND,
 			      PCI_COMMAND_INTX_DISABLE);
 
-	rc = __pci_reset_function_locked(oct->pci_dev);
+	rc = __pci_reset_function_locked(oct->pci_dev, PCI_RESET_ANY);
 
 	if (rc != 0)
 		dev_err(&oct->pci_dev->dev, "Error %d resetting PCI function %d\n",
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 1835f3a7aa8d..e292ea589d3e 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -4673,6 +4673,7 @@  static void pci_dev_restore(struct pci_dev *dev)
  * __pci_reset_function_locked - reset a PCI device function while holding
  * the @dev mutex lock.
  * @dev: PCI device to reset
+ * @reset_type: reset mask to try
  *
  * Some devices allow an individual function to be reset without affecting
  * other functions in the same device.  The PCI device must be responsive
@@ -4688,9 +4689,9 @@  static void pci_dev_restore(struct pci_dev *dev)
  * Returns 0 if the device function was successfully reset or negative if the
  * device doesn't support resetting a single function.
  */
-int __pci_reset_function_locked(struct pci_dev *dev)
+int __pci_reset_function_locked(struct pci_dev *dev, u32 reset_type)
 {
-	int rc;
+	int rc = -EINVAL;
 
 	might_sleep();
 
@@ -4702,24 +4703,42 @@  int __pci_reset_function_locked(struct pci_dev *dev)
 	 * other error, we're also finished: this indicates that further
 	 * reset mechanisms might be broken on the device.
 	 */
-	rc = pci_dev_specific_reset(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	if (pcie_has_flr(dev)) {
-		rc = pcie_flr(dev);
+	if (reset_type & PCI_RESET_DEV_SPECIFIC) {
+		rc = pci_dev_specific_reset(dev, 0);
 		if (rc != -ENOTTY)
 			return rc;
 	}
-	rc = pci_af_flr(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_pm_reset(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_dev_reset_slot_function(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	return pci_parent_bus_reset(dev, 0);
+
+	if (reset_type & PCI_RESET_FLR) {
+		if (pcie_has_flr(dev)) {
+			rc = pcie_flr(dev);
+			if (rc != -ENOTTY)
+				return rc;
+		}
+		rc = pci_af_flr(dev, 0);
+		if (rc != -ENOTTY)
+			return rc;
+	}
+
+	if (reset_type & PCI_RESET_PM) {
+		rc = pci_pm_reset(dev, 0);
+		if (rc != -ENOTTY)
+			return rc;
+	}
+
+	if (reset_type & PCI_RESET_SLOT) {
+		rc = pci_dev_reset_slot_function(dev, 0);
+		if (rc != -ENOTTY)
+			return rc;
+	}
+
+	if (reset_type & PCI_RESET_BUS) {
+		rc = pci_parent_bus_reset(dev, 0);
+		if (rc != -ENOTTY)
+			return rc;
+	}
+
+	return rc;
 }
 EXPORT_SYMBOL_GPL(__pci_reset_function_locked);
 
@@ -4784,7 +4803,7 @@  int pci_reset_function(struct pci_dev *dev)
 	pci_dev_lock(dev);
 	pci_dev_save_and_disable(dev);
 
-	rc = __pci_reset_function_locked(dev);
+	rc = __pci_reset_function_locked(dev, PCI_RESET_ANY);
 
 	pci_dev_restore(dev);
 	pci_dev_unlock(dev);
@@ -4819,7 +4838,7 @@  int pci_reset_function_locked(struct pci_dev *dev)
 
 	pci_dev_save_and_disable(dev);
 
-	rc = __pci_reset_function_locked(dev);
+	rc = __pci_reset_function_locked(dev, PCI_RESET_ANY);
 
 	pci_dev_restore(dev);
 
@@ -4844,7 +4863,7 @@  int pci_try_reset_function(struct pci_dev *dev)
 		return -EAGAIN;
 
 	pci_dev_save_and_disable(dev);
-	rc = __pci_reset_function_locked(dev);
+	rc = __pci_reset_function_locked(dev, PCI_RESET_ANY);
 	pci_dev_restore(dev);
 	pci_dev_unlock(dev);
 
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index 59661db144e5..6dfb805bcb19 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -105,7 +105,7 @@  static void pcistub_device_release(struct kref *kref)
 	/* Call the reset function which does not take lock as this
 	 * is called from "unbind" which takes a device_lock mutex.
 	 */
-	__pci_reset_function_locked(dev);
+	__pci_reset_function_locked(dev, PCI_RESET_ANY);
 	if (pci_load_and_free_saved_state(dev, &dev_data->pci_saved_state))
 		dev_info(&dev->dev, "Could not reload PCI state\n");
 	else
@@ -283,7 +283,7 @@  void pcistub_put_pci_dev(struct pci_dev *dev)
 	 * (so it's ready for the next domain)
 	 */
 	device_lock_assert(&dev->dev);
-	__pci_reset_function_locked(dev);
+	__pci_reset_function_locked(dev, PCI_RESET_ANY);
 
 	dev_data = pci_get_drvdata(dev);
 	ret = pci_load_saved_state(dev, dev_data->pci_saved_state);
@@ -417,7 +417,7 @@  static int pcistub_init_device(struct pci_dev *dev)
 		dev_err(&dev->dev, "Could not store PCI conf saved state!\n");
 	else {
 		dev_dbg(&dev->dev, "resetting (FLR, D3, etc) the device\n");
-		__pci_reset_function_locked(dev);
+		__pci_reset_function_locked(dev, PCI_RESET_ANY);
 		pci_restore_state(dev);
 	}
 	/* Now disable the device (this also ensures some private device
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 6925828f9f25..7ace46b3e479 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -849,6 +849,62 @@  enum {
 	PCI_SCAN_ALL_PCIE_DEVS	= 0x00000040,	/* Scan all, not just dev 0 */
 };
 
+/*
+ * A common theme here is that the driver wants some degree of
+ * control of the type of reset used, vfio specifically wants to specify a
+ * bus or slot reset while hfi1 just wants to specify that the link is
+ * reset and doesn't care if it's a bus or slot reset that accomplishes
+ * that.  The right "unified" interface is one that takes a parameter
+ * allowing the caller to specify the scope or type of reset
+ * with aliases so drivers can ignore the hotplug interface if they wish
+ * for special cases.
+ *
+ * PCI_RESET_ANY tries all reset type one by one until device is successfully
+ * reset. Under normal circumstances, most drivers are expected to use
+ * PCI_RESET_ANY as they don't usually care about the type of reset as long
+ * as device is reset.
+ *
+ * PCI_RESET_FUNC is useful when you want to reset one particular PCI device
+ * but you don't want to impact other devices or cause a temporary service
+ * outage.
+ *
+ * PCI_RESET_LINK can be used to cause a link retraining. This operation will
+ * cause service outage for the PCI bus if there are other devices on the same
+ * bus. PCI_RESET_LINK determines if a platform supports hotplug or not and
+ * suppresses hotplug interrupts during secondary bus reset.
+ *
+ * PCI_RESET_DEV_SPECIFIC can be used to reset a device by help from the
+ * device driver. Not all device drivers support this option.
+ *
+ * PCI_RESET_FLR can be used to issue a Function Level Reset to a device. This
+ * option will fail if FLR is not supported.
+ *
+ * PCI_RESET_PM can be used to reset the device via D3->D0 and D0->D3 sleep
+ * transition. This assumes that device supports PM based reset.
+ *
+ * PCI_RESET_SLOT forces a slot/hotplug reset. This will not work on platforms
+ * without hotplug capability. This option should only be used for advanced
+ * use-cases where driver developer absolutely knows that the device will never
+ * be used on non-hotplug environments.
+ * Not recommended for scalability. Please refer to PCI_RESET_LINK and let the
+ * PCI core do the hotplug detection.
+ *
+ * PCI_RESET_BUS performs a secondary bus reset on the link and causes a link
+ * recovery. Using this option directly and bypassing hotplug driver may
+ * cause a deadlock if platform supports hotplug. Please refer to
+ * PCI_RESET_LINK and let the PCI core do the hotplug detection.
+ */
+#define PCI_RESET_DEV_SPECIFIC	(1 << 0)
+#define PCI_RESET_FLR		(1 << 1)
+#define PCI_RESET_PM		(1 << 2)
+#define PCI_RESET_SLOT		(1 << 3)
+#define PCI_RESET_BUS		(1 << 4)
+
+#define PCI_RESET_ANY		(~0)
+#define PCI_RESET_FUNC		(PCI_RESET_DEV_SPECIFIC | \
+				 PCI_RESET_FLR | PCI_RESET_PM)
+#define PCI_RESET_LINK		(PCI_RESET_SLOT | PCI_RESET_BUS)
+
 /* These external functions are only available when PCI support is enabled */
 #ifdef CONFIG_PCI
 
@@ -1111,7 +1167,7 @@  u32 pcie_bandwidth_available(struct pci_dev *dev, struct pci_dev **limiting_dev,
 void pcie_print_link_status(struct pci_dev *dev);
 bool pcie_has_flr(struct pci_dev *dev);
 int pcie_flr(struct pci_dev *dev);
-int __pci_reset_function_locked(struct pci_dev *dev);
+int __pci_reset_function_locked(struct pci_dev *dev, u32 reset_type);
 int pci_reset_function(struct pci_dev *dev);
 int pci_reset_function_locked(struct pci_dev *dev);
 int pci_try_reset_function(struct pci_dev *dev);