[RFC,v3,03/21] ACPI: processor: Register CPUs that are online, but not described in the DSDT

Commit Message

Russell King (Oracle) Dec. 13, 2023, 12:49 p.m. UTC

From: James Morse <[e-mail]>

ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
says "Each processor in the system must be declared in the ACPI
namespace"). Having two descriptions allows firmware authors to get
this wrong.

If CPUs are described in the MADT/APIC, they will be brought online
early during boot. Once the register_cpu() calls are moved to ACPI,
they will be based on the DSDT description of the CPUs. When CPUs are
missing from the DSDT description, they will end up online, but not
registered.

Add a helper that runs after acpi_init() has completed to register
CPUs that are online, but weren't found in the DSDT. Any CPU that
is registered by this code triggers a firmware-bug warning and kernel
taint.

Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
is configured.

Signed-off-by: James Morse <[e-mail]>
Reviewed-by: Jonathan Cameron <[e-mail]>
Reviewed-by: Gavin Shan <[e-mail]>
Tested-by: Miguel Luis <[e-mail]>
Tested-by: Vishnu Pajjuri <[e-mail]>
Tested-by: Jianyong Wu <[e-mail]>
Signed-off-by: Russell King (Oracle) <[e-mail]>
---
 drivers/acpi/acpi_processor.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

Comments

Rafael J. Wysocki Dec. 18, 2023, 8:22 p.m. UTC | #1

On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:
>
> From: James Morse <[e-mail]>
>
> ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> says "Each processor in the system must be declared in the ACPI
> namespace"). Having two descriptions allows firmware authors to get
> this wrong.
>
> If CPUs are described in the MADT/APIC, they will be brought online
> early during boot. Once the register_cpu() calls are moved to ACPI,
> they will be based on the DSDT description of the CPUs. When CPUs are
> missing from the DSDT description, they will end up online, but not
> registered.
>
> Add a helper that runs after acpi_init() has completed to register
> CPUs that are online, but weren't found in the DSDT. Any CPU that
> is registered by this code triggers a firmware-bug warning and kernel
> taint.
>
> Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> is configured.

So why is this a kernel problem?

> Signed-off-by: James Morse <[e-mail]>
> Reviewed-by: Jonathan Cameron <[e-mail]>
> Reviewed-by: Gavin Shan <[e-mail]>
> Tested-by: Miguel Luis <[e-mail]>
> Tested-by: Vishnu Pajjuri <[e-mail]>
> Tested-by: Jianyong Wu <[e-mail]>
> Signed-off-by: Russell King (Oracle) <[e-mail]>
> ---
>  drivers/acpi/acpi_processor.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
>
> diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
> index 6a542e0ce396..0511f2bc10bc 100644
> --- a/drivers/acpi/acpi_processor.c
> +++ b/drivers/acpi/acpi_processor.c
> @@ -791,6 +791,25 @@ void __init acpi_processor_init(void)
>         acpi_pcc_cpufreq_init();
>  }
>
> +static int __init acpi_processor_register_missing_cpus(void)
> +{
> +       int cpu;
> +
> +       if (acpi_disabled)
> +               return 0;
> +
> +       for_each_online_cpu(cpu) {
> +               if (!get_cpu_device(cpu)) {
> +                       pr_err_once(FW_BUG "CPU %u has no ACPI namespace description!\n", cpu);
> +                       add_taint(TAINT_FIRMWARE_WORKAROUND, LOCKDEP_STILL_OK);
> +                       arch_register_cpu(cpu);

Which part of this code is related to ACPI?

> +               }
> +       }
> +
> +       return 0;
> +}
> +subsys_initcall_sync(acpi_processor_register_missing_cpus);
> +
>  #ifdef CONFIG_ACPI_PROCESSOR_CSTATE
>  /**
>   * acpi_processor_claim_cst_control - Request _CST control from the platform.
> --

Russell King (Oracle) Jan. 15, 2024, 11:06 a.m. UTC | #2

On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:
> On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:
> >
> > From: James Morse <[e-mail]>
> >
> > ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> > in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> > says "Each processor in the system must be declared in the ACPI
> > namespace"). Having two descriptions allows firmware authors to get
> > this wrong.
> >
> > If CPUs are described in the MADT/APIC, they will be brought online
> > early during boot. Once the register_cpu() calls are moved to ACPI,
> > they will be based on the DSDT description of the CPUs. When CPUs are
> > missing from the DSDT description, they will end up online, but not
> > registered.
> >
> > Add a helper that runs after acpi_init() has completed to register
> > CPUs that are online, but weren't found in the DSDT. Any CPU that
> > is registered by this code triggers a firmware-bug warning and kernel
> > taint.
> >
> > Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> > is configured.
> 
> So why is this a kernel problem?

So what are you proposing should be the behaviour here? What this
statement seems to be saying is that QEMU as it exists today only
describes the first CPU in DSDT.

As this patch series changes when arch_register_cpu() gets called (as
described in the paragraph above) we obviously need to preserve the
_existing_ behaviour to avoid causing regressions. So, if changing the
kernel causes user visible regressions (e.g. sysfs entries to
disappear) then it obviously _is_ a kernel problem that needs to be
solved.

We can't say "well fix QEMU then" without invoking the wrath of Linus.

> > Signed-off-by: James Morse <[e-mail]>
> > Reviewed-by: Jonathan Cameron <[e-mail]>
> > Reviewed-by: Gavin Shan <[e-mail]>
> > Tested-by: Miguel Luis <[e-mail]>
> > Tested-by: Vishnu Pajjuri <[e-mail]>
> > Tested-by: Jianyong Wu <[e-mail]>
> > Signed-off-by: Russell King (Oracle) <[e-mail]>
> > ---
> >  drivers/acpi/acpi_processor.c | 19 +++++++++++++++++++
> >  1 file changed, 19 insertions(+)
> >
> > diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
> > index 6a542e0ce396..0511f2bc10bc 100644
> > --- a/drivers/acpi/acpi_processor.c
> > +++ b/drivers/acpi/acpi_processor.c
> > @@ -791,6 +791,25 @@ void __init acpi_processor_init(void)
> >         acpi_pcc_cpufreq_init();
> >  }
> >
> > +static int __init acpi_processor_register_missing_cpus(void)
> > +{
> > +       int cpu;
> > +
> > +       if (acpi_disabled)
> > +               return 0;
> > +
> > +       for_each_online_cpu(cpu) {
> > +               if (!get_cpu_device(cpu)) {
> > +                       pr_err_once(FW_BUG "CPU %u has no ACPI namespace description!\n", cpu);
> > +                       add_taint(TAINT_FIRMWARE_WORKAROUND, LOCKDEP_STILL_OK);
> > +                       arch_register_cpu(cpu);
> 
> Which part of this code is related to ACPI?

That's a good question, and I suspect it would be more suited to being
placed in drivers/base/cpu.c except for the problem that the error
message refers to ACPI.

As long as we keep the acpi_disabled test, I guess that's fine.
cpu_dev_register_generic() there already tests acpi_disabled.

Jonathan Cameron Jan. 22, 2024, 4:02 p.m. UTC | #3

On Mon, 15 Jan 2024 11:06:29 +0000
"Russell King (Oracle)" <[e-mail]> wrote:

> On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:
> > On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:  
> > >
> > > From: James Morse <[e-mail]>
> > >
> > > ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> > > in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> > > says "Each processor in the system must be declared in the ACPI
> > > namespace"). Having two descriptions allows firmware authors to get
> > > this wrong.
> > >
> > > If CPUs are described in the MADT/APIC, they will be brought online
> > > early during boot. Once the register_cpu() calls are moved to ACPI,
> > > they will be based on the DSDT description of the CPUs. When CPUs are
> > > missing from the DSDT description, they will end up online, but not
> > > registered.
> > >
> > > Add a helper that runs after acpi_init() has completed to register
> > > CPUs that are online, but weren't found in the DSDT. Any CPU that
> > > is registered by this code triggers a firmware-bug warning and kernel
> > > taint.
> > >
> > > Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> > > is configured.  
> > 
> > So why is this a kernel problem?  
> 
> So what are you proposing should be the behaviour here? What this
> statement seems to be saying is that QEMU as it exists today only
> describes the first CPU in DSDT.

This confuses me somewhat, because I'm far from sure which machines this
is true for in QEMU.  I'm guessing it's a legacy thing with
some old distro version of QEMU - so we'll have to paper over it anyway
but for current QEMU I'm not sure it's true.

Helpfully there are a bunch of ACPI table tests so I've been checking
through all the multi CPU cases.

CPU hotplug not enabled.
pc/DSDT.dimmpxm  - 4x Processor entries.  -smp 4
pc/DSDT.acpihmat - 2x Processor entries.  -smp 2
q35/DSDT.acpihmat - 2x Processor entries. -smp 2
virt/DSDT.acpihmatvirt - 4x ACPI0007 entries -smp 4
q35/DSDT.acpihmat-noinitiator - 4 x Processor () entries -smp 4 
virt/DSDT.topology - 8x ACPI0007 entries

I've also looked at the code and we have various types of
CPU hotplug on x86 but they all build appropriate numbers of
Processor() entries in DSDT.
Arm likewise seems to build the right number of ACPI0007 entries
(and doesn't yet have CPU HP support).

If anyone can add a reference on why this is needed that would be very
helpful.

> 
> As this patch series changes when arch_register_cpu() gets called (as
> described in the paragraph above) we obviously need to preserve the
> _existing_ behaviour to avoid causing regressions. So, if changing the
> kernel causes user visible regressions (e.g. sysfs entries to
> disappear) then it obviously _is_ a kernel problem that needs to be
> solved.
> 
> We can't say "well fix QEMU then" without invoking the wrath of Linus.

Overall I'm fine with the defensive nature of this patch as there
'might' be firmware out there with this problem - I just can't establish
that there is!  If anyone else recalls the history of this then give
a shout.  I vaguely wondered if this was an ia64 thing but nope, QEMU
never generated tables for ia64 before dropping support back in QEMU 2.11


> 
> > > Signed-off-by: James Morse <[e-mail]>
> > > Reviewed-by: Jonathan Cameron <[e-mail]>
> > > Reviewed-by: Gavin Shan <[e-mail]>
> > > Tested-by: Miguel Luis <[e-mail]>
> > > Tested-by: Vishnu Pajjuri <[e-mail]>
> > > Tested-by: Jianyong Wu <[e-mail]>
> > > Signed-off-by: Russell King (Oracle) <[e-mail]>
> > > ---
> > >  drivers/acpi/acpi_processor.c | 19 +++++++++++++++++++
> > >  1 file changed, 19 insertions(+)
> > >
> > > diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
> > > index 6a542e0ce396..0511f2bc10bc 100644
> > > --- a/drivers/acpi/acpi_processor.c
> > > +++ b/drivers/acpi/acpi_processor.c
> > > @@ -791,6 +791,25 @@ void __init acpi_processor_init(void)
> > >         acpi_pcc_cpufreq_init();
> > >  }
> > >
> > > +static int __init acpi_processor_register_missing_cpus(void)
> > > +{
> > > +       int cpu;
> > > +
> > > +       if (acpi_disabled)
> > > +               return 0;
> > > +
> > > +       for_each_online_cpu(cpu) {
> > > +               if (!get_cpu_device(cpu)) {
> > > +                       pr_err_once(FW_BUG "CPU %u has no ACPI namespace description!\n", cpu);
> > > +                       add_taint(TAINT_FIRMWARE_WORKAROUND, LOCKDEP_STILL_OK);
> > > +                       arch_register_cpu(cpu);  
> > 
> > Which part of this code is related to ACPI?  
> 
> That's a good question, and I suspect it would be more suited to being
> placed in drivers/base/cpu.c except for the problem that the error
> message refers to ACPI.
> 
> As long as we keep the acpi_disabled test, I guess that's fine.
> cpu_dev_register_generic() there already tests acpi_disabled.
> 
Moving it seems fine to me.

Jonathan

Rafael J. Wysocki Jan. 22, 2024, 4:22 p.m. UTC | #4

On Mon, Jan 22, 2024 at 5:02 PM Jonathan Cameron
<[e-mail]> wrote:
>
> On Mon, 15 Jan 2024 11:06:29 +0000
> "Russell King (Oracle)" <[e-mail]> wrote:
>
> > On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:
> > > On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:
> > > >
> > > > From: James Morse <[e-mail]>
> > > >
> > > > ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> > > > in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> > > > says "Each processor in the system must be declared in the ACPI
> > > > namespace"). Having two descriptions allows firmware authors to get
> > > > this wrong.
> > > >
> > > > If CPUs are described in the MADT/APIC, they will be brought online
> > > > early during boot. Once the register_cpu() calls are moved to ACPI,
> > > > they will be based on the DSDT description of the CPUs. When CPUs are
> > > > missing from the DSDT description, they will end up online, but not
> > > > registered.
> > > >
> > > > Add a helper that runs after acpi_init() has completed to register
> > > > CPUs that are online, but weren't found in the DSDT. Any CPU that
> > > > is registered by this code triggers a firmware-bug warning and kernel
> > > > taint.
> > > >
> > > > Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> > > > is configured.
> > >
> > > So why is this a kernel problem?
> >
> > So what are you proposing should be the behaviour here? What this
> > statement seems to be saying is that QEMU as it exists today only
> > describes the first CPU in DSDT.
>
> This confuses me somewhat, because I'm far from sure which machines this
> is true for in QEMU.  I'm guessing it's a legacy thing with
> some old distro version of QEMU - so we'll have to paper over it anyway
> but for current QEMU I'm not sure it's true.
>
> Helpfully there are a bunch of ACPI table tests so I've been checking
> through all the multi CPU cases.
>
> CPU hotplug not enabled.
> pc/DSDT.dimmpxm  - 4x Processor entries.  -smp 4
> pc/DSDT.acpihmat - 2x Processor entries.  -smp 2
> q35/DSDT.acpihmat - 2x Processor entries. -smp 2
> virt/DSDT.acpihmatvirt - 4x ACPI0007 entries -smp 4
> q35/DSDT.acpihmat-noinitiator - 4 x Processor () entries -smp 4
> virt/DSDT.topology - 8x ACPI0007 entries
>
> I've also looked at the code and we have various types of
> CPU hotplug on x86 but they all build appropriate numbers of
> Processor() entries in DSDT.
> Arm likewise seems to build the right number of ACPI0007 entries
> (and doesn't yet have CPU HP support).
>
> If anyone can add a reference on why this is needed that would be very
> helpful.

Yes, it would.

Personally, I would prefer to assume that it is not necessary until it
turns out that (1) there is firmware with this issue actually in use
and (2) updating the firmware in question to follow the specification
is not practical.

Otherwise, we'd make it easier to ship non-compliant firmware for no
good reason.

Russell King (Oracle) Jan. 22, 2024, 5:27 p.m. UTC | #5

On Mon, Jan 22, 2024 at 04:02:27PM +0000, Jonathan Cameron wrote:
> On Mon, 15 Jan 2024 11:06:29 +0000
> "Russell King (Oracle)" <[e-mail]> wrote:
> 
> > On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:
> > > On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:  
> > > >
> > > > From: James Morse <[e-mail]>
> > > >
> > > > ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> > > > in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> > > > says "Each processor in the system must be declared in the ACPI
> > > > namespace"). Having two descriptions allows firmware authors to get
> > > > this wrong.
> > > >
> > > > If CPUs are described in the MADT/APIC, they will be brought online
> > > > early during boot. Once the register_cpu() calls are moved to ACPI,
> > > > they will be based on the DSDT description of the CPUs. When CPUs are
> > > > missing from the DSDT description, they will end up online, but not
> > > > registered.
> > > >
> > > > Add a helper that runs after acpi_init() has completed to register
> > > > CPUs that are online, but weren't found in the DSDT. Any CPU that
> > > > is registered by this code triggers a firmware-bug warning and kernel
> > > > taint.
> > > >
> > > > Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> > > > is configured.  
> > > 
> > > So why is this a kernel problem?  
> > 
> > So what are you proposing should be the behaviour here? What this
> > statement seems to be saying is that QEMU as it exists today only
> > describes the first CPU in DSDT.
> 
> This confuses me somewhat, because I'm far from sure which machines this
> is true for in QEMU.  I'm guessing it's a legacy thing with
> some old distro version of QEMU - so we'll have to paper over it anyway
> but for current QEMU I'm not sure it's true.
> 
> Helpfully there are a bunch of ACPI table tests so I've been checking
> through all the multi CPU cases.
> 
> CPU hotplug not enabled.
> pc/DSDT.dimmpxm  - 4x Processor entries.  -smp 4
> pc/DSDT.acpihmat - 2x Processor entries.  -smp 2
> q35/DSDT.acpihmat - 2x Processor entries. -smp 2
> virt/DSDT.acpihmatvirt - 4x ACPI0007 entries -smp 4
> q35/DSDT.acpihmat-noinitiator - 4 x Processor () entries -smp 4 
> virt/DSDT.topology - 8x ACPI0007 entries
> 
> I've also looked at the code and we have various types of
> CPU hotplug on x86 but they all build appropriate numbers of
> Processor() entries in DSDT.
> Arm likewise seems to build the right number of ACPI0007 entries
> (and doesn't yet have CPU HP support).
> 
> If anyone can add a reference on why this is needed that would be very
> helpful.

Maybe Salil can shed some light on this?

Russell King (Oracle) Jan. 22, 2024, 5:30 p.m. UTC | #6

On Mon, Jan 22, 2024 at 05:22:46PM +0100, Rafael J. Wysocki wrote:
> On Mon, Jan 22, 2024 at 5:02 PM Jonathan Cameron
> <[e-mail]> wrote:
> >
> > On Mon, 15 Jan 2024 11:06:29 +0000
> > "Russell King (Oracle)" <[e-mail]> wrote:
> >
> > > On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:
> > > > On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:
> > > > >
> > > > > From: James Morse <[e-mail]>
> > > > >
> > > > > ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> > > > > in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> > > > > says "Each processor in the system must be declared in the ACPI
> > > > > namespace"). Having two descriptions allows firmware authors to get
> > > > > this wrong.
> > > > >
> > > > > If CPUs are described in the MADT/APIC, they will be brought online
> > > > > early during boot. Once the register_cpu() calls are moved to ACPI,
> > > > > they will be based on the DSDT description of the CPUs. When CPUs are
> > > > > missing from the DSDT description, they will end up online, but not
> > > > > registered.
> > > > >
> > > > > Add a helper that runs after acpi_init() has completed to register
> > > > > CPUs that are online, but weren't found in the DSDT. Any CPU that
> > > > > is registered by this code triggers a firmware-bug warning and kernel
> > > > > taint.
> > > > >
> > > > > Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> > > > > is configured.
> > > >
> > > > So why is this a kernel problem?
> > >
> > > So what are you proposing should be the behaviour here? What this
> > > statement seems to be saying is that QEMU as it exists today only
> > > describes the first CPU in DSDT.
> >
> > This confuses me somewhat, because I'm far from sure which machines this
> > is true for in QEMU.  I'm guessing it's a legacy thing with
> > some old distro version of QEMU - so we'll have to paper over it anyway
> > but for current QEMU I'm not sure it's true.
> >
> > Helpfully there are a bunch of ACPI table tests so I've been checking
> > through all the multi CPU cases.
> >
> > CPU hotplug not enabled.
> > pc/DSDT.dimmpxm  - 4x Processor entries.  -smp 4
> > pc/DSDT.acpihmat - 2x Processor entries.  -smp 2
> > q35/DSDT.acpihmat - 2x Processor entries. -smp 2
> > virt/DSDT.acpihmatvirt - 4x ACPI0007 entries -smp 4
> > q35/DSDT.acpihmat-noinitiator - 4 x Processor () entries -smp 4
> > virt/DSDT.topology - 8x ACPI0007 entries
> >
> > I've also looked at the code and we have various types of
> > CPU hotplug on x86 but they all build appropriate numbers of
> > Processor() entries in DSDT.
> > Arm likewise seems to build the right number of ACPI0007 entries
> > (and doesn't yet have CPU HP support).
> >
> > If anyone can add a reference on why this is needed that would be very
> > helpful.
> 
> Yes, it would.
> 
> Personally, I would prefer to assume that it is not necessary until it
> turns out that (1) there is firmware with this issue actually in use
> and (2) updating the firmware in question to follow the specification
> is not practical.
> 
> Otherwise, we'd make it easier to ship non-compliant firmware for no
> good reason.

If Salil can't come up with a reason, then I'm in favour of dropping
the patch like already done for patch 2. If the code change serves no
useful purpose, there's no point in making the change.

Jonathan Cameron Jan. 23, 2024, 9:27 a.m. UTC | #7

On Mon, 22 Jan 2024 17:30:05 +0000
"Russell King (Oracle)" <[e-mail]> wrote:

> On Mon, Jan 22, 2024 at 05:22:46PM +0100, Rafael J. Wysocki wrote:
> > On Mon, Jan 22, 2024 at 5:02 PM Jonathan Cameron
> > <[e-mail]> wrote:  
> > >
> > > On Mon, 15 Jan 2024 11:06:29 +0000
> > > "Russell King (Oracle)" <[e-mail]> wrote:
> > >  
> > > > On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:  
> > > > > On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:  
> > > > > >
> > > > > > From: James Morse <[e-mail]>
> > > > > >
> > > > > > ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> > > > > > in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> > > > > > says "Each processor in the system must be declared in the ACPI
> > > > > > namespace"). Having two descriptions allows firmware authors to get
> > > > > > this wrong.
> > > > > >
> > > > > > If CPUs are described in the MADT/APIC, they will be brought online
> > > > > > early during boot. Once the register_cpu() calls are moved to ACPI,
> > > > > > they will be based on the DSDT description of the CPUs. When CPUs are
> > > > > > missing from the DSDT description, they will end up online, but not
> > > > > > registered.
> > > > > >
> > > > > > Add a helper that runs after acpi_init() has completed to register
> > > > > > CPUs that are online, but weren't found in the DSDT. Any CPU that
> > > > > > is registered by this code triggers a firmware-bug warning and kernel
> > > > > > taint.
> > > > > >
> > > > > > Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> > > > > > is configured.  
> > > > >
> > > > > So why is this a kernel problem?  
> > > >
> > > > So what are you proposing should be the behaviour here? What this
> > > > statement seems to be saying is that QEMU as it exists today only
> > > > describes the first CPU in DSDT.  
> > >
> > > This confuses me somewhat, because I'm far from sure which machines this
> > > is true for in QEMU.  I'm guessing it's a legacy thing with
> > > some old distro version of QEMU - so we'll have to paper over it anyway
> > > but for current QEMU I'm not sure it's true.
> > >
> > > Helpfully there are a bunch of ACPI table tests so I've been checking
> > > through all the multi CPU cases.
> > >
> > > CPU hotplug not enabled.
> > > pc/DSDT.dimmpxm  - 4x Processor entries.  -smp 4
> > > pc/DSDT.acpihmat - 2x Processor entries.  -smp 2
> > > q35/DSDT.acpihmat - 2x Processor entries. -smp 2
> > > virt/DSDT.acpihmatvirt - 4x ACPI0007 entries -smp 4
> > > q35/DSDT.acpihmat-noinitiator - 4 x Processor () entries -smp 4
> > > virt/DSDT.topology - 8x ACPI0007 entries
> > >
> > > I've also looked at the code and we have various types of
> > > CPU hotplug on x86 but they all build appropriate numbers of
> > > Processor() entries in DSDT.
> > > Arm likewise seems to build the right number of ACPI0007 entries
> > > (and doesn't yet have CPU HP support).
> > >
> > > If anyone can add a reference on why this is needed that would be very
> > > helpful.  
> > 
> > Yes, it would.
> > 
> > Personally, I would prefer to assume that it is not necessary until it
> > turns out that (1) there is firmware with this issue actually in use
> > and (2) updating the firmware in question to follow the specification
> > is not practical.
> > 
> > Otherwise, we'd make it easier to ship non-compliant firmware for no
> > good reason.  
> 
> If Salil can't come up with a reason, then I'm in favour of dropping
> the patch like already done for patch 2. If the code change serves no
> useful purpose, there's no point in making the change.
> 

Salil's out today, but I've messaged him to follow up later in the week.

It 'might' be the odd cold plug path where QEMU half comes up, then extra
CPUs are added, then it boots. (used by some orchestration frameworks)
I don't have a set up for that and I won't get to creating one today anyway
(we all love start of the year planning workshops!)

I've +CC'd a few people have run tests on the various iterations of this
work in the past.  Maybe one of them can shed some light on this?

Jonathan

Miguel Luis Jan. 25, 2024, 1:56 p.m. UTC | #8

Hi

> On 23 Jan 2024, at 08:27, Jonathan Cameron <[e-mail]> wrote:
> 
> On Mon, 22 Jan 2024 17:30:05 +0000
> "Russell King (Oracle)" <[e-mail]> wrote:
> 
>> On Mon, Jan 22, 2024 at 05:22:46PM +0100, Rafael J. Wysocki wrote:
>>> On Mon, Jan 22, 2024 at 5:02 PM Jonathan Cameron
>>> <[e-mail]> wrote:  
>>>> 
>>>> On Mon, 15 Jan 2024 11:06:29 +0000
>>>> "Russell King (Oracle)" <[e-mail]> wrote:
>>>> 
>>>>> On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:  
>>>>>> On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:  
>>>>>>> 
>>>>>>> From: James Morse <[e-mail]>
>>>>>>> 
>>>>>>> ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
>>>>>>> in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
>>>>>>> says "Each processor in the system must be declared in the ACPI
>>>>>>> namespace"). Having two descriptions allows firmware authors to get
>>>>>>> this wrong.
>>>>>>> 
>>>>>>> If CPUs are described in the MADT/APIC, they will be brought online
>>>>>>> early during boot. Once the register_cpu() calls are moved to ACPI,
>>>>>>> they will be based on the DSDT description of the CPUs. When CPUs are
>>>>>>> missing from the DSDT description, they will end up online, but not
>>>>>>> registered.
>>>>>>> 
>>>>>>> Add a helper that runs after acpi_init() has completed to register
>>>>>>> CPUs that are online, but weren't found in the DSDT. Any CPU that
>>>>>>> is registered by this code triggers a firmware-bug warning and kernel
>>>>>>> taint.
>>>>>>> 
>>>>>>> Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
>>>>>>> is configured.  
>>>>>> 
>>>>>> So why is this a kernel problem?  
>>>>> 
>>>>> So what are you proposing should be the behaviour here? What this
>>>>> statement seems to be saying is that QEMU as it exists today only
>>>>> describes the first CPU in DSDT.  
>>>> 
>>>> This confuses me somewhat, because I'm far from sure which machines this
>>>> is true for in QEMU.  I'm guessing it's a legacy thing with
>>>> some old distro version of QEMU - so we'll have to paper over it anyway
>>>> but for current QEMU I'm not sure it's true.
>>>> 
>>>> Helpfully there are a bunch of ACPI table tests so I've been checking
>>>> through all the multi CPU cases.
>>>> 
>>>> CPU hotplug not enabled.
>>>> pc/DSDT.dimmpxm  - 4x Processor entries.  -smp 4
>>>> pc/DSDT.acpihmat - 2x Processor entries.  -smp 2
>>>> q35/DSDT.acpihmat - 2x Processor entries. -smp 2
>>>> virt/DSDT.acpihmatvirt - 4x ACPI0007 entries -smp 4
>>>> q35/DSDT.acpihmat-noinitiator - 4 x Processor () entries -smp 4
>>>> virt/DSDT.topology - 8x ACPI0007 entries
>>>> 
>>>> I've also looked at the code and we have various types of
>>>> CPU hotplug on x86 but they all build appropriate numbers of
>>>> Processor() entries in DSDT.
>>>> Arm likewise seems to build the right number of ACPI0007 entries
>>>> (and doesn't yet have CPU HP support).
>>>> 
>>>> If anyone can add a reference on why this is needed that would be very
>>>> helpful.  
>>> 
>>> Yes, it would.
>>> 
>>> Personally, I would prefer to assume that it is not necessary until it
>>> turns out that (1) there is firmware with this issue actually in use
>>> and (2) updating the firmware in question to follow the specification
>>> is not practical.
>>> 
>>> Otherwise, we'd make it easier to ship non-compliant firmware for no
>>> good reason.  
>> 
>> If Salil can't come up with a reason, then I'm in favour of dropping
>> the patch like already done for patch 2. If the code change serves no
>> useful purpose, there's no point in making the change.
>> 
> 
> Salil's out today, but I've messaged him to follow up later in the week.
> 
> It 'might' be the odd cold plug path where QEMU half comes up, then extra
> CPUs are added, then it boots. (used by some orchestration frameworks)
> I don't have a set up for that and I won't get to creating one today anyway
> (we all love start of the year planning workshops!)
> 
> I've +CC'd a few people have run tests on the various iterations of this
> work in the past.  Maybe one of them can shed some light on this?
> 

IIUC, this patch covers a scenario for non compliant firmware and in which my 
tests for AArch64 using RFC v2 have been unable to trigger its error message so 
far. This does not mean, however, this patch should not be taken forward though.

It seems benevolent enough detecting non compliant firmware and still proceed 
while having whoever uses that firmware to get to know that.

I'm not sure, however, whether the reference to a specific VMM should be in the 
commit message though. That might not be anything to do with the kernel so a 
more meaningful rewrite on this separation of concerns could be useful.

Miguel

> Jonathan

Rafael J. Wysocki Jan. 25, 2024, 2:42 p.m. UTC | #9

Hi,

On Thu, Jan 25, 2024 at 2:56 PM Miguel Luis <[e-mail]> wrote:
>
> Hi
>
> > On 23 Jan 2024, at 08:27, Jonathan Cameron <[e-mail]> wrote:
> >
> > On Mon, 22 Jan 2024 17:30:05 +0000
> > "Russell King (Oracle)" <[e-mail]> wrote:
> >
> >> On Mon, Jan 22, 2024 at 05:22:46PM +0100, Rafael J. Wysocki wrote:
> >>> On Mon, Jan 22, 2024 at 5:02 PM Jonathan Cameron
> >>> <[e-mail]> wrote:
> >>>>
> >>>> On Mon, 15 Jan 2024 11:06:29 +0000
> >>>> "Russell King (Oracle)" <[e-mail]> wrote:
> >>>>
> >>>>> On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:
> >>>>>> On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:
> >>>>>>>
> >>>>>>> From: James Morse <[e-mail]>
> >>>>>>>
> >>>>>>> ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> >>>>>>> in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> >>>>>>> says "Each processor in the system must be declared in the ACPI
> >>>>>>> namespace"). Having two descriptions allows firmware authors to get
> >>>>>>> this wrong.
> >>>>>>>
> >>>>>>> If CPUs are described in the MADT/APIC, they will be brought online
> >>>>>>> early during boot. Once the register_cpu() calls are moved to ACPI,
> >>>>>>> they will be based on the DSDT description of the CPUs. When CPUs are
> >>>>>>> missing from the DSDT description, they will end up online, but not
> >>>>>>> registered.
> >>>>>>>
> >>>>>>> Add a helper that runs after acpi_init() has completed to register
> >>>>>>> CPUs that are online, but weren't found in the DSDT. Any CPU that
> >>>>>>> is registered by this code triggers a firmware-bug warning and kernel
> >>>>>>> taint.
> >>>>>>>
> >>>>>>> Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> >>>>>>> is configured.
> >>>>>>
> >>>>>> So why is this a kernel problem?
> >>>>>
> >>>>> So what are you proposing should be the behaviour here? What this
> >>>>> statement seems to be saying is that QEMU as it exists today only
> >>>>> describes the first CPU in DSDT.
> >>>>
> >>>> This confuses me somewhat, because I'm far from sure which machines this
> >>>> is true for in QEMU.  I'm guessing it's a legacy thing with
> >>>> some old distro version of QEMU - so we'll have to paper over it anyway
> >>>> but for current QEMU I'm not sure it's true.
> >>>>
> >>>> Helpfully there are a bunch of ACPI table tests so I've been checking
> >>>> through all the multi CPU cases.
> >>>>
> >>>> CPU hotplug not enabled.
> >>>> pc/DSDT.dimmpxm  - 4x Processor entries.  -smp 4
> >>>> pc/DSDT.acpihmat - 2x Processor entries.  -smp 2
> >>>> q35/DSDT.acpihmat - 2x Processor entries. -smp 2
> >>>> virt/DSDT.acpihmatvirt - 4x ACPI0007 entries -smp 4
> >>>> q35/DSDT.acpihmat-noinitiator - 4 x Processor () entries -smp 4
> >>>> virt/DSDT.topology - 8x ACPI0007 entries
> >>>>
> >>>> I've also looked at the code and we have various types of
> >>>> CPU hotplug on x86 but they all build appropriate numbers of
> >>>> Processor() entries in DSDT.
> >>>> Arm likewise seems to build the right number of ACPI0007 entries
> >>>> (and doesn't yet have CPU HP support).
> >>>>
> >>>> If anyone can add a reference on why this is needed that would be very
> >>>> helpful.
> >>>
> >>> Yes, it would.
> >>>
> >>> Personally, I would prefer to assume that it is not necessary until it
> >>> turns out that (1) there is firmware with this issue actually in use
> >>> and (2) updating the firmware in question to follow the specification
> >>> is not practical.
> >>>
> >>> Otherwise, we'd make it easier to ship non-compliant firmware for no
> >>> good reason.
> >>
> >> If Salil can't come up with a reason, then I'm in favour of dropping
> >> the patch like already done for patch 2. If the code change serves no
> >> useful purpose, there's no point in making the change.
> >>
> >
> > Salil's out today, but I've messaged him to follow up later in the week.
> >
> > It 'might' be the odd cold plug path where QEMU half comes up, then extra
> > CPUs are added, then it boots. (used by some orchestration frameworks)
> > I don't have a set up for that and I won't get to creating one today anyway
> > (we all love start of the year planning workshops!)
> >
> > I've +CC'd a few people have run tests on the various iterations of this
> > work in the past.  Maybe one of them can shed some light on this?
> >
>
> IIUC, this patch covers a scenario for non compliant firmware and in which my
> tests for AArch64 using RFC v2 have been unable to trigger its error message so
> far. This does not mean, however, this patch should not be taken forward though.
>
> It seems benevolent enough detecting non compliant firmware and still proceed
> while having whoever uses that firmware to get to know that.

There is one issue with this approach, though.

If this is done by Linux and Linux is used as a main testing vehicle
for whoever produced that firmware, it may pass the tests and be
shipped causing a problem for the rest of the industry (because other
operating systems will not support that firmware and now they will be
put in an awkward position).

I've seen enough breakage resulting from a similar policy in some
other OS and with Linux on the receiving end that I'd rather avoid
doing this to someone else.

So if the firmware is not compliant, the best way to go is to ask
whoever ships it to please fix their stuff, or if other OSes already
work around the non-compliance, it's time to update the spec to
reflect the reality (aka "industry practice").

Thanks!

Jonathan Cameron Jan. 29, 2024, 1:03 p.m. UTC | #10

On Tue, 23 Jan 2024 09:27:25 +0000
Jonathan Cameron <[e-mail]> wrote:

> On Mon, 22 Jan 2024 17:30:05 +0000
> "Russell King (Oracle)" <[e-mail]> wrote:
> 
> > On Mon, Jan 22, 2024 at 05:22:46PM +0100, Rafael J. Wysocki wrote:  
> > > On Mon, Jan 22, 2024 at 5:02 PM Jonathan Cameron
> > > <[e-mail]> wrote:    
> > > >
> > > > On Mon, 15 Jan 2024 11:06:29 +0000
> > > > "Russell King (Oracle)" <[e-mail]> wrote:
> > > >    
> > > > > On Mon, Dec 18, 2023 at 09:22:03PM +0100, Rafael J. Wysocki wrote:    
> > > > > > On Wed, Dec 13, 2023 at 1:49 PM Russell King <[e-mail]> wrote:    
> > > > > > >
> > > > > > > From: James Morse <[e-mail]>
> > > > > > >
> > > > > > > ACPI has two descriptions of CPUs, one in the MADT/APIC table, the other
> > > > > > > in the DSDT. Both are required. (ACPI 6.5's 8.4 "Declaring Processors"
> > > > > > > says "Each processor in the system must be declared in the ACPI
> > > > > > > namespace"). Having two descriptions allows firmware authors to get
> > > > > > > this wrong.
> > > > > > >
> > > > > > > If CPUs are described in the MADT/APIC, they will be brought online
> > > > > > > early during boot. Once the register_cpu() calls are moved to ACPI,
> > > > > > > they will be based on the DSDT description of the CPUs. When CPUs are
> > > > > > > missing from the DSDT description, they will end up online, but not
> > > > > > > registered.
> > > > > > >
> > > > > > > Add a helper that runs after acpi_init() has completed to register
> > > > > > > CPUs that are online, but weren't found in the DSDT. Any CPU that
> > > > > > > is registered by this code triggers a firmware-bug warning and kernel
> > > > > > > taint.
> > > > > > >
> > > > > > > Qemu TCG only describes the first CPU in the DSDT, unless cpu-hotplug
> > > > > > > is configured.    
> > > > > >
> > > > > > So why is this a kernel problem?    
> > > > >
> > > > > So what are you proposing should be the behaviour here? What this
> > > > > statement seems to be saying is that QEMU as it exists today only
> > > > > describes the first CPU in DSDT.    
> > > >
> > > > This confuses me somewhat, because I'm far from sure which machines this
> > > > is true for in QEMU.  I'm guessing it's a legacy thing with
> > > > some old distro version of QEMU - so we'll have to paper over it anyway
> > > > but for current QEMU I'm not sure it's true.
> > > >
> > > > Helpfully there are a bunch of ACPI table tests so I've been checking
> > > > through all the multi CPU cases.
> > > >
> > > > CPU hotplug not enabled.
> > > > pc/DSDT.dimmpxm  - 4x Processor entries.  -smp 4
> > > > pc/DSDT.acpihmat - 2x Processor entries.  -smp 2
> > > > q35/DSDT.acpihmat - 2x Processor entries. -smp 2
> > > > virt/DSDT.acpihmatvirt - 4x ACPI0007 entries -smp 4
> > > > q35/DSDT.acpihmat-noinitiator - 4 x Processor () entries -smp 4
> > > > virt/DSDT.topology - 8x ACPI0007 entries
> > > >
> > > > I've also looked at the code and we have various types of
> > > > CPU hotplug on x86 but they all build appropriate numbers of
> > > > Processor() entries in DSDT.
> > > > Arm likewise seems to build the right number of ACPI0007 entries
> > > > (and doesn't yet have CPU HP support).
> > > >
> > > > If anyone can add a reference on why this is needed that would be very
> > > > helpful.    
> > > 
> > > Yes, it would.
> > > 
> > > Personally, I would prefer to assume that it is not necessary until it
> > > turns out that (1) there is firmware with this issue actually in use
> > > and (2) updating the firmware in question to follow the specification
> > > is not practical.
> > > 
> > > Otherwise, we'd make it easier to ship non-compliant firmware for no
> > > good reason.    
> > 
> > If Salil can't come up with a reason, then I'm in favour of dropping
> > the patch like already done for patch 2. If the code change serves no
> > useful purpose, there's no point in making the change.
> >   
> 
> Salil's out today, but I've messaged him to follow up later in the week.
> 
> It 'might' be the odd cold plug path where QEMU half comes up, then extra
> CPUs are added, then it boots. (used by some orchestration frameworks)

I poked this on x86 - it only applies with hotplug enabled anyway so
same result as doing the hotplug later - All possible Processor() entries
already exist in DSDT. Hence this isn't the source of the mysterious
broken configuration.

If anyone does poke this path, the old discussion between James
and Salil provides some instructions (mostly the thread is about
another issue).
https://op-lists.linaro.org/archives/list/[e-mail]/thread/DNAGB2FB5ALVLV2BYWYOCLKGNF77PNXS/

Also on x86 a test involving smp 2,max-cpus=4 and adding cpu-id 3
(so skipping 2) doesn't boot. (this is without Salil's QEMU patches).
I guess there are some well known rules in there that I don't know about
and QEMU isn't preventing people shooting themselves in the foot.

As I'm concerned, drop this patch.
If there are platforms out there doing this wrong they'll surface once
we get this into more test farms (so linux-next).  If we need this
'fix' we can apply it when we have a problem firmware to point at.

Thanks,

Jonathan

> I don't have a set up for that and I won't get to creating one today anyway
> (we all love start of the year planning workshops!)

> 
> I've +CC'd a few people have run tests on the various iterations of this
> work in the past.  Maybe one of them can shed some light on this?
> 
> Jonathan
> 
> 
> 
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> [e-mail]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Russell King (Oracle) Jan. 29, 2024, 3:32 p.m. UTC | #11

On Mon, Jan 29, 2024 at 01:03:54PM +0000, Jonathan Cameron wrote:
> I poked this on x86 - it only applies with hotplug enabled anyway so
> same result as doing the hotplug later - All possible Processor() entries
> already exist in DSDT. Hence this isn't the source of the mysterious
> broken configuration.
> 
> If anyone does poke this path, the old discussion between James
> and Salil provides some instructions (mostly the thread is about
> another issue).
> https://op-lists.linaro.org/archives/list/[e-mail]/thread/DNAGB2FB5ALVLV2BYWYOCLKGNF77PNXS/
> 
> Also on x86 a test involving smp 2,max-cpus=4 and adding cpu-id 3
> (so skipping 2) doesn't boot. (this is without Salil's QEMU patches).
> I guess there are some well known rules in there that I don't know about
> and QEMU isn't preventing people shooting themselves in the foot.
> 
> As I'm concerned, drop this patch.
> If there are platforms out there doing this wrong they'll surface once
> we get this into more test farms (so linux-next).  If we need this
> 'fix' we can apply it when we have a problem firmware to point at.

Now dropped.

Patch

diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
index 6a542e0ce396..0511f2bc10bc 100644
--- a/drivers/acpi/acpi_processor.c
+++ b/drivers/acpi/acpi_processor.c
@@ -791,6 +791,25 @@  void __init acpi_processor_init(void)
 	acpi_pcc_cpufreq_init();
 }
 
+static int __init acpi_processor_register_missing_cpus(void)
+{
+	int cpu;
+
+	if (acpi_disabled)
+		return 0;
+
+	for_each_online_cpu(cpu) {
+		if (!get_cpu_device(cpu)) {
+			pr_err_once(FW_BUG "CPU %u has no ACPI namespace description!\n", cpu);
+			add_taint(TAINT_FIRMWARE_WORKAROUND, LOCKDEP_STILL_OK);
+			arch_register_cpu(cpu);
+		}
+	}
+
+	return 0;
+}
+subsys_initcall_sync(acpi_processor_register_missing_cpus);
+
 #ifdef CONFIG_ACPI_PROCESSOR_CSTATE
 /**
  * acpi_processor_claim_cst_control - Request _CST control from the platform.