[v1,2/3] PCI: prevent work_on_cpu's probe to execute on isolated CPUs
diff mbox series

Message ID 20200610161226.424337-3-nitesh@redhat.com
State Superseded
Headers show
Series
  • Preventing job distribution to isolated CPUs
Related show

Commit Message

Nitesh Narayan Lal June 10, 2020, 4:12 p.m. UTC
From: Alex Belits <abelits@marvell.com>

pci_call_probe() prevents the nesting of work_on_cpu()
for a scenario where a VF device is probed from work_on_cpu()
of the Physical device.
This patch replaces the cpumask used in pci_call_probe()
from all online CPUs to only housekeeping CPUs. This is to
ensure that there are no additional latency overheads
caused due to the pinning of jobs on isolated CPUs.

Signed-off-by: Alex Belits <abelits@marvell.com>
Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com>
---
 drivers/pci/pci-driver.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Comments

Bjorn Helgaas June 16, 2020, 8:05 p.m. UTC | #1
"git log --oneline drivers/pci/pci-driver.c" tells you that the
subject should be something like:

  PCI: Restrict probe functions to housekeeping CPUs

On Wed, Jun 10, 2020 at 12:12:25PM -0400, Nitesh Narayan Lal wrote:
> From: Alex Belits <abelits@marvell.com>
> 
> pci_call_probe() prevents the nesting of work_on_cpu()
> for a scenario where a VF device is probed from work_on_cpu()
> of the Physical device.
> This patch replaces the cpumask used in pci_call_probe()
> from all online CPUs to only housekeeping CPUs. This is to
> ensure that there are no additional latency overheads
> caused due to the pinning of jobs on isolated CPUs.

s/Physical/PF/ (since you used "VF" earlier, this should match that)

s/This patch replaces/Replace the/

Please rewrap this to fill a 75 column line (so it doesn't overflow 80
columns when "git log" adds 4 spaces).

This should be two paragraphs; add a blank line between them.

> Signed-off-by: Alex Belits <abelits@marvell.com>
> Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com>

Acked-by: Bjorn Helgaas <bhelgaas@google.com>

> ---
>  drivers/pci/pci-driver.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> index da6510af1221..449466f71040 100644
> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -12,6 +12,7 @@
>  #include <linux/string.h>
>  #include <linux/slab.h>
>  #include <linux/sched.h>
> +#include <linux/sched/isolation.h>
>  #include <linux/cpu.h>
>  #include <linux/pm_runtime.h>
>  #include <linux/suspend.h>
> @@ -333,6 +334,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
>  			  const struct pci_device_id *id)
>  {
>  	int error, node, cpu;
> +	int hk_flags = HK_FLAG_DOMAIN | HK_FLAG_WQ;
>  	struct drv_dev_and_id ddi = { drv, dev, id };
>  
>  	/*
> @@ -353,7 +355,8 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
>  	    pci_physfn_is_probed(dev))
>  		cpu = nr_cpu_ids;
>  	else
> -		cpu = cpumask_any_and(cpumask_of_node(node), cpu_online_mask);
> +		cpu = cpumask_any_and(cpumask_of_node(node),
> +				      housekeeping_cpumask(hk_flags));
>  
>  	if (cpu < nr_cpu_ids)
>  		error = work_on_cpu(cpu, local_pci_probe, &ddi);
> -- 
> 2.18.4
>
Nitesh Narayan Lal June 16, 2020, 10:03 p.m. UTC | #2
On 6/16/20 4:05 PM, Bjorn Helgaas wrote:
> "git log --oneline drivers/pci/pci-driver.c" tells you that the
> subject should be something like:
>
>   PCI: Restrict probe functions to housekeeping CPUs
>
> On Wed, Jun 10, 2020 at 12:12:25PM -0400, Nitesh Narayan Lal wrote:
>> From: Alex Belits <abelits@marvell.com>
>>
>> pci_call_probe() prevents the nesting of work_on_cpu()
>> for a scenario where a VF device is probed from work_on_cpu()
>> of the Physical device.
>> This patch replaces the cpumask used in pci_call_probe()
>> from all online CPUs to only housekeeping CPUs. This is to
>> ensure that there are no additional latency overheads
>> caused due to the pinning of jobs on isolated CPUs.
> s/Physical/PF/ (since you used "VF" earlier, this should match that)
>
> s/This patch replaces/Replace the/
>
> Please rewrap this to fill a 75 column line (so it doesn't overflow 80
> columns when "git log" adds 4 spaces).
>
> This should be two paragraphs; add a blank line between them.

Thanks for pointing these out.
I will correct it in the next posting before that I will wait for any comments
on other patches.

>
>> Signed-off-by: Alex Belits <abelits@marvell.com>
>> Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com>
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>
>
>> ---
>>  drivers/pci/pci-driver.c | 5 ++++-
>>  1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
>> index da6510af1221..449466f71040 100644
>> --- a/drivers/pci/pci-driver.c
>> +++ b/drivers/pci/pci-driver.c
>> @@ -12,6 +12,7 @@
>>  #include <linux/string.h>
>>  #include <linux/slab.h>
>>  #include <linux/sched.h>
>> +#include <linux/sched/isolation.h>
>>  #include <linux/cpu.h>
>>  #include <linux/pm_runtime.h>
>>  #include <linux/suspend.h>
>> @@ -333,6 +334,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
>>  			  const struct pci_device_id *id)
>>  {
>>  	int error, node, cpu;
>> +	int hk_flags = HK_FLAG_DOMAIN | HK_FLAG_WQ;
>>  	struct drv_dev_and_id ddi = { drv, dev, id };
>>  
>>  	/*
>> @@ -353,7 +355,8 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
>>  	    pci_physfn_is_probed(dev))
>>  		cpu = nr_cpu_ids;
>>  	else
>> -		cpu = cpumask_any_and(cpumask_of_node(node), cpu_online_mask);
>> +		cpu = cpumask_any_and(cpumask_of_node(node),
>> +				      housekeeping_cpumask(hk_flags));
>>  
>>  	if (cpu < nr_cpu_ids)
>>  		error = work_on_cpu(cpu, local_pci_probe, &ddi);
>> -- 
>> 2.18.4
>>
Frederic Weisbecker June 16, 2020, 11:22 p.m. UTC | #3
On Wed, Jun 10, 2020 at 12:12:25PM -0400, Nitesh Narayan Lal wrote:
> From: Alex Belits <abelits@marvell.com>
> 
> pci_call_probe() prevents the nesting of work_on_cpu()
> for a scenario where a VF device is probed from work_on_cpu()
> of the Physical device.
> This patch replaces the cpumask used in pci_call_probe()
> from all online CPUs to only housekeeping CPUs. This is to
> ensure that there are no additional latency overheads
> caused due to the pinning of jobs on isolated CPUs.
> 
> Signed-off-by: Alex Belits <abelits@marvell.com>
> Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com>
> ---
>  drivers/pci/pci-driver.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> index da6510af1221..449466f71040 100644
> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -12,6 +12,7 @@
>  #include <linux/string.h>
>  #include <linux/slab.h>
>  #include <linux/sched.h>
> +#include <linux/sched/isolation.h>
>  #include <linux/cpu.h>
>  #include <linux/pm_runtime.h>
>  #include <linux/suspend.h>
> @@ -333,6 +334,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
>  			  const struct pci_device_id *id)
>  {
>  	int error, node, cpu;
> +	int hk_flags = HK_FLAG_DOMAIN | HK_FLAG_WQ;
>  	struct drv_dev_and_id ddi = { drv, dev, id };
>  
>  	/*
> @@ -353,7 +355,8 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
>  	    pci_physfn_is_probed(dev))
>  		cpu = nr_cpu_ids;
>  	else
> -		cpu = cpumask_any_and(cpumask_of_node(node), cpu_online_mask);
> +		cpu = cpumask_any_and(cpumask_of_node(node),
> +				      housekeeping_cpumask(hk_flags));

Looks like cpumask_of_node() is based on online CPUs. So that all
looks good. Thanks!

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>


>  
>  	if (cpu < nr_cpu_ids)
>  		error = work_on_cpu(cpu, local_pci_probe, &ddi);
> -- 
> 2.18.4
>

Patch
diff mbox series

diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index da6510af1221..449466f71040 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -12,6 +12,7 @@ 
 #include <linux/string.h>
 #include <linux/slab.h>
 #include <linux/sched.h>
+#include <linux/sched/isolation.h>
 #include <linux/cpu.h>
 #include <linux/pm_runtime.h>
 #include <linux/suspend.h>
@@ -333,6 +334,7 @@  static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
 			  const struct pci_device_id *id)
 {
 	int error, node, cpu;
+	int hk_flags = HK_FLAG_DOMAIN | HK_FLAG_WQ;
 	struct drv_dev_and_id ddi = { drv, dev, id };
 
 	/*
@@ -353,7 +355,8 @@  static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
 	    pci_physfn_is_probed(dev))
 		cpu = nr_cpu_ids;
 	else
-		cpu = cpumask_any_and(cpumask_of_node(node), cpu_online_mask);
+		cpu = cpumask_any_and(cpumask_of_node(node),
+				      housekeeping_cpumask(hk_flags));
 
 	if (cpu < nr_cpu_ids)
 		error = work_on_cpu(cpu, local_pci_probe, &ddi);