| Message ID | 20230414225551.858160935@linutronix.de (mailing list archive) |
|---|---|
| Headers | show |
| Series | cpu/hotplug, x86: Reworked parallel CPU bringup | expand |
On 15.04.23 01:44, Thomas Gleixner wrote: > Hi! > > This is a complete rework of the parallel bringup patch series (V17) > > https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com > > to address the issues which were discovered in review: > > 1) The X86 microcode loader serialization requirement > > https://lore.kernel.org/lkml/87v8iirxun.ffs@tglx > > Microcode loading on HT enabled X86 CPUs requires that the microcode is > loaded on the primary thread. The sibling thread(s) must be in > quiescent state; either looping in a place which is aware of potential > changes by the microcode update (see late loading) or in fully quiescent > state, i.e. waiting for INIT/SIPI. > > This is required by hardware/firmware on Intel. Aside of that it's a > vendor independent software correctness issue. Assume the following > sequence: > > CPU1.0 CPU1.1 > CPUID($A) > Load microcode. > Changes CPUID($A, $B) > CPUID($B) > > CPU1.1 makes a decision on $A and $B which might be inconsistent due > to the microcode update. > > The solution for this is to bringup the primary threads first and after > that the siblings. Loading microcode on the siblings is a NOOP on Intel > and on AMD it is guaranteed to only modify thread local state. > > This ensures that the APs can load microcode before reaching the alive > synchronization point w/o doing any further x86 specific > synchronization between the core siblings. > > 2) The general design issues discussed in V16 > > https://lore.kernel.org/lkml/87pm8y6yme.ffs@tglx > > The previous parallel bringup patches just glued this mechanism into > the existing code without a deeper analysis of the synchronization > mechanisms and without generalizing it so that the control logic is > mostly in the core code and not made an architecture specific tinker > space. > > Much of that had been pointed out 2 years ago in the discussions about > the early versions of parallel bringup already. > > > The series is based on: > > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip x86/apic > > and also available from git: > > git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug > > > Background > ---------- > > The reason why people are interested in parallel bringup is to shorten > the (kexec) reboot time of cloud servers to reduce the downtime of the > VM tenants. There are obviously other interesting use cases for this > like VM startup time, embedded devices... > > The current fully serialized bringup does the following per AP: > > 1) Prepare callbacks (allocate, intialize, create threads) > 2) Kick the AP alive (e.g. INIT/SIPI on x86) > 3) Wait for the AP to report alive state > 4) Let the AP continue through the atomic bringup > 5) Let the AP run the threaded bringup to full online state > > There are two significant delays: > > #3 The time for an AP to report alive state in start_secondary() on x86 > has been measured in the range between 350us and 3.5ms depending on > vendor and CPU type, BIOS microcode size etc. > > #4 The atomic bringup does the microcode update. This has been measured > to take up to ~8ms on the primary threads depending on the microcode > patch size to apply. > > On a two socket SKL server with 56 cores (112 threads) the boot CPU spends > on current mainline about 800ms busy waiting for the APs to come up and > apply microcode. That's more than 80% of the actual onlining procedure. > > By splitting the actual bringup mechanism into two parts this can be > reduced to waiting for the first AP to report alive or if the system is > large enough the first AP is already waiting when the boot CPU finished the > wake-up of the last AP. > > > The actual solution comes in several parts > ------------------------------------------ > > 1) [P 1-2] General cleanups (init annotations, kernel doc...) > > 2) [P 3] The obvious > > Avoid pointless delay calibration when TSC is synchronized across > sockets. That removes a whopping 100ms delay for the first CPU of a > socket. This is an improvement independent of parallel bringup and had > been discussed two years ago already. > > 2) [P 3-6] Removal of the CPU0 hotplug hack. > > This was added 11 years ago with the promise to make this a real > hardware mechanism, but that never materialized. As physical CPU > hotplug is not really supported and the physical unplugging of CPU0 > never materialized there is no reason to keep this cruft around. It's > just maintenance ballast for no value and the removal makes > implementing the parallel bringup feature way simpler. > > 3) [P 7-16] Cleanup of the existing bringup mechanism: > > a) Code reorganisation so that the general hotplug specific code is > in smpboot.c and not sprinkled all over the place > > b) Decouple MTRR/PAT initialization from smp_callout_mask to prepare > for replacing that mask with a hotplug core code synchronization > mechanism. > > c) Make TSC synchronization function call based so that the control CPU > does not have to busy wait for nothing if synchronization is not > required. > > d) Remove the smp_callin_mask synchronization point as its not longer > required due to #3c. > > e) Rework the sparse_irq_lock held region in the core code so that the > next polling synchronization point in the x86 code can be removed to. > > f) Due to #3e it's not longer required to spin wait for the AP to set > it's online bit. Remove wait_cpu_online() and the XENPV > counterpart. So the control CPU can directly wait for the online > idle completion by the AP and free the control CPU up for other > work. > > This reduces the synchronization points in the x86 code to one, which > is the AP alive one. This synchronization will be moved to core > infrastructure in the next section. > > 4) [P 17-27] Replace the disconnected CPU state tracking > > The extra CPU state tracking which is used by a few architectures is > completely separate from the CPU hotplug core code. > > Replacing it by a variant integrated in the core hotplug machinery > allows to reduce architecture specific code and provides a generic > synchronization mechanism for (parallel) CPU bringup/teardown. > > - Convert x86 over and replace the AP alive synchronization on x86 with > the core variant which removes the remaining x86 hotplug > synchronization masks. > > - Convert the other architectures usage and remove the old interface > and code. > > 5) [P 28-30] Split the bringup into two steps > > First step invokes the wakeup function on the BP, e.g. SIPI/STARTUP on > x86. The second one waits on the BP for the AP to report alive and > releases it for the complete onlining. > > As the hotplug state machine allows partial bringup this allows later > to kick all APs alive in a first iteration and then bring them up > completely one by one afterwards. > > 6) [P 31] Switch the primary thread detection to a cpumask > > This makes the parallel bringup a simple cpumask based mechanism > without tons of conditionals and checks for primary threads. > > 7) [P 32] Implement the parallel bringup core code > > The parallel bringup looks like this: > > 1) Bring up the primary SMT threads to the CPUHP_KICK_AP_ALIVE step > one by one > > 2) Bring up the primary SMT threads to the CPUHP_ONLINE step one by > one > > 3) Bring up the secondary SMT threads to the CPUHP_KICK_AP_ALIVE > step one by one > > 4) Bring up the secondary SMT threads to the CPUHP_ONLINE > step one by one > > In case that SMT is not supported this is obviously reduced to step #1 > and #2. > > 8) [P 33-37] Prepare X86 for parallel bringup and enable it > > > Caveats > ------- > > The non X86 changes have been all compile tested. Boot and runtime > testing has only be done on a few real hardware platforms and qemu as > available. That definitely needs some help from the people who have > these systems at their fingertips. > > > Results and analysis > -------------------- > > Here are numbers for a dual socket SKL 56 cores/ 112 threads machine. All > numbers in milliseconds. The time measured is the time which the cpu_up() > call takes for each CPU and phase. It's not exact as the system is already > scheduling, handling interrupts and soft interrupts, which is obviously > skewing the picture slightly. > > Baseline tip tree x86/apic branch. > > total avg/CPU min max > total : 912.081 8.217 3.720 113.271 > > The max of 100ms is due to the silly delay calibration for the second > socket which takes 100ms and was eliminated first. Also the other initial > cleanups and improvements take some time away. > > So the real baseline becomes: > > total avg/CPU min max > total : 785.960 7.081 3.752 36.098 > > The max here is on the first CPU of the second socket. 20ms of that is due > to TSC synchronization and an extra 2ms to react on the SIPI. > > With parallel bootup enabled this becomes: > > total avg/CPU min max > prepare: 39.108 0.352 0.238 0.883 > online : 45.166 0.407 0.170 20.357 > total : 84.274 0.759 0.408 21.240 > > That's a factor ~9.3 reduction on average. > > Looking at the 27 primary threads of socket 0 then this becomes even more > interesting: > > total avg/CPU min max > total : 325.764 12.065 11.981 14.125 > > versus: > total avg/CPU min max > prepare: 8.945 0.331 0.238 0.834 > online : 4.830 0.179 0.170 0.212 > total : 13.775 0.510 0.408 1.046 > > So the reduction factor is ~23.5 here. That's mostly because the 20ms TSC > sync is not skewing the picture. > > For all 55 primaries, i.e with the 20ms TSC sync extra for socket 1 this > becomes: > > total avg/CPU min max > total : 685.489 12.463 11.975 36.098 > > versus: > > total avg/CPU min max > prepare: 19.080 0.353 0.238 0.883 > online : 30.283 0.561 0.170 20.357 > total : 49.363 0.914 0.408 21.240 > > The TSC sync reduces the win to a factor of ~13.8 > > With 'tsc=reliable' on the command line the socket sync is disabled which > brings it back to the socket 0 numbers: > > total avg/CPU min max > prepare: 18.970 0.351 0.231 0.874 > online : 10.328 0.191 0.169 0.358 > total : 29.298 0.543 0.400 1.232 > > Now looking at the secondary threads only: > > total avg/CPU min max > total : 100.471 1.794 0.375 4.745 > > versus: > total avg/CPU min max > prepare: 19.753 0.353 0.257 0.512 > online : 14.671 0.262 0.179 3.461 > total : 34.424 0.615 0.436 3.973 > > Still a factor of ~3. > > The average on the secondaries for the serialized bringup is significantly > lower than for the primaries because the SIPI response time is shorter and > the microcode update takes no time. > > This varies wildly with the system, whether microcode in BIOS is already up > to date, how big the microcode patch is and how long the INIT/SIPI response > time is. On an AMD Zen3 machine INIT/SIPI response time is amazingly fast > (350us), but then it lacks TSC_ADJUST and does a two millisecond TSC sync > test for _every_ AP. All of this sucks... > > > Possible further enhancements > ----------------------------- > > It's definitely worthwhile to look into reducing the cross socket TSC sync > test time. It's probably safe enough to use 5ms or even 2ms instead of 20ms > on systems with TSC_ADJUST and a few other 'TSC is sane' indicators. Moving > it out of the hotplug path is eventually possible, but that needs some deep > thoughts. > > Let's take the TSC sync out of the picture by adding 'tsc=reliable" to the > kernel command line. So the bringup of 111 APs takes: > > total avg/CPU min max > prepare: 38.936 0.351 0.231 0.874 > online : 25.231 0.227 0.169 3.465 > total : 64.167 0.578 0.400 4.339 > > Some of the outliers are not necessarily in the state callbacks as the > system is already scheduling and handles interrupts and soft > interrupts. Haven't analyzed that yet in detail. > > In the prepare stage which runs on the control CPU the larger steps are: > > smpcfd:prepare 16us avg/CPU > threads:prepare 98us avg/CPU > workqueue:prepare 43us avg/CPU > trace/RB:prepare 135us avg/CPU > > The trace ringbuffer initialization allocates 354 pages and 354 control > structures one by one. That probably should allocate a large page and an > array of control structures and work from there. I'm sure that would reduce > this significantly. Steven? > > smpcfd does just a percpu allocation. No idea why that takes that long. > > Vs. threads and workqueues. David thought about spreading out the > preparation work and do it really in parallel. That's a nice idea, but the > threads and workqueue prepare steps are self serializing. The workqueue one > has a global mutex and aside of that both steps create kernel threads which > implicitely serialize on kthreadd. alloc_percpu(), which is used by > smpcfd:prepare is also globally serialized. > > The rest of the prepare steps is pretty much in the single digit > microseconds range. > > On the AP side it should be possible to move some of the initialization > steps before the alive synchronization point, but that really needs a lot > of analysis whether the functions are safe to invoke that early and outside > of the cpu_hotplug_lock held region for the case of two stage parallel > bringup; see below. > > The largest part is: > > identify_secondary_cpu() 99us avg/CPU > > Inside of identify_secondary_cpu() the largest offender: > > mcheck_init() 73us avg/CPU > > This part is definitly worth to be looked at whether it can be at least > partially moved to the early startup code before the alive > synchronization point. There's a lot of deep analysis required and > ideally we just rewrite the whole CPUID evaluation trainwreck > completely. > > The rest of the AP side is low single digit microseconds except of: > > perf/x86:starting 14us avg/CPU > > smpboot/threads:online 13us avg/CPU > workqueue:online 17us avg/CPU > mm/vmstat:online 17us avg/CPU > sched:active 30us avg/CPU > > sched:active is special. Onlining the first secondary HT thread on the > second socket creates a 3.2ms outlier which skews the whole picture. That's > caused by enabling the static key sched_smt_present which patches the world > and some more. For all other APs this is really in the 1us range. This > definitely could be postponed during bootup like the scheduler domain > rebuild is done after the bringup. But that's still fully serialized and > single threaded and obviously could be done later in the context of async > parallel init. It's unclear why this is different with the fully serialized > bringup where it takes significantly less time, but that's something which > needs to be investigated. > > > Is truly parallel bringup feasible? > ----------------------------------- > > In theory yes, realistically no. Why? > > 1) The preparation phase > > Allocating memory, creating threads for the to be brought up CPU must > obviously happen on an already online CPU. > > While it would be possible to bring up a subset of CPUs first and let > them do the preparation steps for groups of still offline CPUs > concurrently, the actual benefit of doing so is dubious. > > The prime example is kernel thread creation, which is implicitely > serialized on kthreadd. > > A simple experiment shows that 4 concurrent workers on 4 different > CPUs where each is creating 14 * 5 = 70 kernel threads are 5% slower > than a single worker creating 4 * 14 * 5 = 280 threads. > > So we'd need to have multiple kthreadd instances to handle that, > which would then serialize on tasklist lock and other things. > > That aside the preparation phase is also affected by the problem > below. > > 2) Assumptions about hotplug serialization > > a) There are quite some assumptions about CPU bringup being fully > serialized across state transitions. A lot of state callbacks rely > on that and would require local locking. > > Adding that local locking is surely possible, but that has several > downsides: > > - It adds complexity and makes it harder for developers to get > this correct. The subtle bugs resulting out of that are going > to be interesting > > - Fine grained locking has a charm, but only if the time spent > for the actual work is larger than the time required for > serialization and synchronization. > > Serializing a callback which takes less than a microsecond and > then having a large number of CPUs contending on the lock will > not make it any faster at all. That's a well known issue of > parallelizing and neither made up nor kernel specific. > > b) Some operations definitely require to be protected by the > cpu_hotplug_lock, especially those which affect cpumasks as the > masks are guaranteed to be stable in a cpus_read_lock()'ed region. > > As this lock cannot be taken in atomic contexts, it's required > that the control CPU holds the lock write locked across these > state transitions. And no, we are not making this a spinlock just > for that and we even can't. > > Just slapping a lock into the x86 specific part of the cpumask > update function does not solve anything. The relevant patch in V17 > is completely useless as it only serializes the actual cpumask/map > modifications, but all read side users are hosed if the update > would be moved before the alive synchronization point, i.e. into a > non hotplug lock protected region. > > Even if the hotplug lock would be held accross the whole parallel > bringup operation then this would still expose all usage of these > masks and maps in the actual hotplug state callbacks to concurrent > modifications. > > And no, we are not going to expose an architecture specific raw > spinlock to the hotplug state callbacks, especially not to those > in generic code. > > c) Some cpu_read_lock()'ed regions also expect that there is no CPU > state transition happening which would modify their local > state. This would again require local serialization. > > 3) The amount of work and churn: > > - Analyze the per architecture low level startup functions plus their > descendant functions and make them ready for concurrency if > necessary. > > - Analyze ~300 hotplug state callbacks and their descendant functions > and make them ready for concurrency if necessary. > > - Analyze all cpus_read_lock()'ed regions and address their > requirements. > > - Rewrite the core code to handle the cpu_hotplug_lock requirements > only in distinct phases of the state machine. > > - Rewrite the core code to handle state callback failure and the > related rollback in the context of the new rules. > > - ... > > Even if some people are dedicated enough to do that, it's very > questionable whether the resulting complexity is justified. > > We've spent a serious amount of time to sanitize hotplug and bring it > into a state where it is correct. This also made it reasonably simple > for developers to implement hotplug state callbacks without having to > become hotplug experts. > > Breaking this completely up will result in a flood of hard to diagnose > subtle issues for sure. Who is going to deal with them? > > The experience with this series so far does not make me comfortable > about that thought in any way. > > > Summary > ------- > > The obvious and low hanging fruits have to be solved first: > > - The CPUID evaluation and related setup mechanisms > > - The trace/ringbuffer oddity > > - The sched:active oddity for the first sibling on the second socket > > - Some other expensive things which I'm not seeing in my test setup due > to lack of hardware or configuration. > > Anything else is pretty much wishful thinking in my opinion. > > To be clear. I'm not standing in the way if there is a proper solution, > but that requires to respect the basic engineering rules: > > 1) Correctness first > 2) Keep it maintainable > 3) Keep it simple > > So far this stuff failed already at #1. > > I completely understand why this is important for cloud people, but > the real question to ask here is what are the actual requirements. > > As far as I understand the main goal is to make a (kexec) reboot > almost invisible to VM tenants. > > Now lets look at how this works: > > A) Freeze VMs and persist state > B) kexec into the new kernel > C) Restore VMs from persistant memory > D) Thaw VMs > > So the key problem is how long it takes to get from #B to #C and finally > to #D. > > As far as I understand #C takes a serious amount of time and cannot be > parallelized for whatever reasons. > > At the same time the number of online CPUs required to restore the VMs > state is less than the number of online CPUs required to actually > operate them in #D. > > That means it would be good enough to return to userspace with a > limited number of online CPUs as fast as possible. A certain amount of > CPUs are going to be busy with restoring the VMs state, i.e. one CPU > per VM. Some remaining non-busy CPU can bringup the rest of the system > and the APs in order to be functional for #D, i.e the restore of VM > operation. > > Trying to optimize this purely in kernel space by adding complexity of > dubious value is simply bogus in my opinion. > > It's already possible today to limit the number of CPUs which are > initially onlined and online the rest later from user space. > > There are two issue there: > > a) The death by MCE broadcast problem > > Quite some (contemporary) x86 CPU generations are affected by > this: > > - MCE can be broadcasted to all CPUs and not only issued locally > to the CPU which triggered it. > > - Any CPU which has CR4.MCE == 0, even if it sits in a wait > for INIT/SIPI state, will cause an immediate shutdown of the > machine if a broadcasted MCE is delivered. > > b) Do the parallel bringup via sysfs control knob > > The per CPU target state interface allows to do that today one > by one, but it's akward and has quite some overhead. > > A knob to online the rest of the not yet onlined present CPUs > with the benefit of the parallel bringup mechanism is > missing. > > #a) That's a risk to take by the operator. > > Even the regular serialized bringup does not protect against this > issue up to the point where all present CPUs have at least > initialized CR4. > > Limiting the number of APs to online early via the kernel command > line widens that window and increases the risk further by > executing user space before all APs have CR4 initialized. > > But the same applies to a deferred online mechanism implemented in > the kernel where some worker brings up the not yet online APs while > the early online CPUs are already executing user space code. > > #b) Is a no brainer to implement on top of this. > > > Conclusion > ---------- > > Adding the basic parallel bringup mechanism as provided by this series > makes a lot of sense. Improving particular issues as pointed out in the > analysis makes sense too. > > But trying to solve an application specific problem fully in the kernel > with tons of complexity, without exploring straight forward and simple > approaches first, does not make any sense at all. > > Thanks, > > tglx > > --- > Documentation/admin-guide/kernel-parameters.txt | 20 > Documentation/core-api/cpu_hotplug.rst | 13 > arch/Kconfig | 23 + > arch/arm/Kconfig | 1 > arch/arm/include/asm/smp.h | 2 > arch/arm/kernel/smp.c | 18 > arch/arm64/Kconfig | 1 > arch/arm64/include/asm/smp.h | 2 > arch/arm64/kernel/smp.c | 14 > arch/csky/Kconfig | 1 > arch/csky/include/asm/smp.h | 2 > arch/csky/kernel/smp.c | 8 > arch/mips/Kconfig | 1 > arch/mips/cavium-octeon/smp.c | 1 > arch/mips/include/asm/smp-ops.h | 1 > arch/mips/kernel/smp-bmips.c | 1 > arch/mips/kernel/smp-cps.c | 14 > arch/mips/kernel/smp.c | 8 > arch/mips/loongson64/smp.c | 1 > arch/parisc/Kconfig | 1 > arch/parisc/kernel/process.c | 4 > arch/parisc/kernel/smp.c | 7 > arch/riscv/Kconfig | 1 > arch/riscv/include/asm/smp.h | 2 > arch/riscv/kernel/cpu-hotplug.c | 14 > arch/x86/Kconfig | 45 -- > arch/x86/include/asm/apic.h | 5 > arch/x86/include/asm/cpu.h | 5 > arch/x86/include/asm/cpumask.h | 5 > arch/x86/include/asm/processor.h | 1 > arch/x86/include/asm/realmode.h | 3 > arch/x86/include/asm/sev-common.h | 3 > arch/x86/include/asm/smp.h | 26 - > arch/x86/include/asm/topology.h | 23 - > arch/x86/include/asm/tsc.h | 2 > arch/x86/kernel/acpi/sleep.c | 9 > arch/x86/kernel/apic/apic.c | 22 - > arch/x86/kernel/callthunks.c | 4 > arch/x86/kernel/cpu/amd.c | 2 > arch/x86/kernel/cpu/cacheinfo.c | 21 > arch/x86/kernel/cpu/common.c | 50 -- > arch/x86/kernel/cpu/topology.c | 3 > arch/x86/kernel/head_32.S | 14 > arch/x86/kernel/head_64.S | 121 +++++ > arch/x86/kernel/sev.c | 2 > arch/x86/kernel/smp.c | 3 > arch/x86/kernel/smpboot.c | 508 ++++++++---------------- > arch/x86/kernel/topology.c | 98 ---- > arch/x86/kernel/tsc.c | 20 > arch/x86/kernel/tsc_sync.c | 36 - > arch/x86/power/cpu.c | 37 - > arch/x86/realmode/init.c | 3 > arch/x86/realmode/rm/trampoline_64.S | 27 + > arch/x86/xen/enlighten_hvm.c | 11 > arch/x86/xen/smp_hvm.c | 16 > arch/x86/xen/smp_pv.c | 56 +- > drivers/acpi/processor_idle.c | 4 > include/linux/cpu.h | 4 > include/linux/cpuhotplug.h | 17 > kernel/cpu.c | 397 +++++++++++++++++- > kernel/smp.c | 2 > kernel/smpboot.c | 163 ------- > 62 files changed, 953 insertions(+), 976 deletions(-) > > Tested with a Xen PV dom0 on an 8 cpu system, no issues found. Tested-by: Juergen Gross <jgross@suse.com> Juergen
On Sat, Apr 15, 2023 at 01:44:13AM +0200, Thomas Gleixner wrote: > Background > ---------- > > The reason why people are interested in parallel bringup is to shorten > the (kexec) reboot time of cloud servers to reduce the downtime of the > VM tenants. There are obviously other interesting use cases for this > like VM startup time, embedded devices... ... > There are two issue there: > > a) The death by MCE broadcast problem > > Quite some (contemporary) x86 CPU generations are affected by > this: > > - MCE can be broadcasted to all CPUs and not only issued locally > to the CPU which triggered it. > > - Any CPU which has CR4.MCE == 0, even if it sits in a wait > for INIT/SIPI state, will cause an immediate shutdown of the > machine if a broadcasted MCE is delivered. When doing kexec, CR4.MCE should already have been set to 1 by the prior kernel, no?
On 17/04/2023 11:30 am, Peter Zijlstra wrote: > On Sat, Apr 15, 2023 at 01:44:13AM +0200, Thomas Gleixner wrote: > >> Background >> ---------- >> >> The reason why people are interested in parallel bringup is to shorten >> the (kexec) reboot time of cloud servers to reduce the downtime of the >> VM tenants. There are obviously other interesting use cases for this >> like VM startup time, embedded devices... > ... > >> There are two issue there: >> >> a) The death by MCE broadcast problem >> >> Quite some (contemporary) x86 CPU generations are affected by >> this: >> >> - MCE can be broadcasted to all CPUs and not only issued locally >> to the CPU which triggered it. >> >> - Any CPU which has CR4.MCE == 0, even if it sits in a wait >> for INIT/SIPI state, will cause an immediate shutdown of the >> machine if a broadcasted MCE is delivered. > When doing kexec, CR4.MCE should already have been set to 1 by the prior > kernel, no? No(ish). Purgatory can't take #MC, or NMIs for that matter. It's cleaner to explicitly disable CR4.MCE and let the system reset (with all the MC banks properly preserved), than it is to take #MC while the IDT isn't in sync with the handlers, and wander off into the weeds. ~Andrew
Dear Thomas, Am 15.04.23 um 01:44 schrieb Thomas Gleixner: > This is a complete rework of the parallel bringup patch series (V17) > > https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com > > to address the issues which were discovered in review: […] Thank you very much for your rework. I tested this on the ASUS F2A85-M PRO, and get a delay of ten seconds. ``` […] [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1) […] [ 0.259329] smp: Bringing up secondary CPUs ... [ 0.259527] x86: Booting SMP configuration: [ 0.259528] .... node #0, CPUs: #1 [ 0.261007] After schedule_preempt_disabled [ 10.260990] CPU1 failed to report alive state [ 10.261070] smp: Brought up 1 node, 1 CPU [ 10.261073] smpboot: Max logical packages: 2 [ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS) [ 10.261601] devtmpfs: initialized [ 10.261697] x86/mm: Memory block size: 128MB ``` This delay has been there with v6.3-rc6-46-gde4664485abbc and some custom (printk) patches on top and merging dwmw2/parallel-6.2-rc3-v16 into it. I only tested this. I think dwmw2/parallel-6.2-v17 failed to build for me, when trying to merge it into Linus’ master version at that time. I didn’t come around to report it, and you posted your rework, so I am replying here. I am going to try your branch directly in the next days, but just wanted to report back already. Kind regards, Paul [ 0.000000] Linux version 6.3.0-rc6-00311-gde8224969f66 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #446 SMP PREEMPT_DYNAMIC Sat Apr 15 14:12:29 UTC 2023 [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable [ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-g9917d2d915 04/17/2023 [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Initial usec timer 6035615 [ 0.000000] tsc: Detected 3900.273 MHz processor [ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000 [ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000942] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000 [ 0.004000] Using GB pages for direct mapping [ 0.004000] ACPI: Early table checksum verification disabled [ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4) [ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925) [ 0.004000] ACPI: FACS 0x000000005FE5A240 000040 [ 0.004000] ACPI: FACS 0x000000005FE5A240 000040 [ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925) [ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000) [ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000) [ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001) [ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3] [ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9] [ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f] [ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69] [ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab] [ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11] [ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57] [ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f] [ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71] [ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8] [ 0.004000] No NUMA configuration found [ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff] [ 0.004000] NODE_DATA(0) allocated [mem 0x17efe7000-0x17effdfff] [ 0.004000] Zone ranges: [ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff] [ 0.004000] Device empty [ 0.004000] Movable zone start for each node [ 0.004000] Early memory node ranges [ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff] [ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff] [ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff] [ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges [ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges [ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges [ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges [ 0.004000] ACPI: PM-Timer IO Port: 0x818 [ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) [ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23 [ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level) [ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information [ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000 [ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs [ 0.004000] smpboot: smpboot: XXX end of prefill_possible_map [ 0.004000] After prefill_possible_map [ 0.004000] After init_cpu_to_node [ 0.004000] After init_gi_nodes [ 0.004000] After io_apic_init_mappings [ 0.004000] After x86_init.hyper.guest_late_init [ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices [ 0.004000] After e820 [ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns [ 0.004000] After unwind_init [ 0.004000] After setup_arch [ 0.004000] After setup_command_line [ 0.004000] After setup_nr_cpu_ids [ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1 [ 0.004000] percpu: Embedded 55 pages/cpu s188328 r8192 d28760 u1048576 [ 0.004000] pcpu-alloc: s188328 r8192 d28760 u1048576 alloc=1*2097152 [ 0.004000] pcpu-alloc: [0] 0 1 [ 0.004000] After setup_per_cpu_areas [ 0.004000] After smp_perpare_boot_cpu [ 0.004000] After boot_cpu_hotplug_init [ 0.004000] Fallback order for Node 0: 0 [ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451 [ 0.004000] Policy zone: Normal [ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 [ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66", will be passed to user space. [ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) [ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash [ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.004000] software IO TLB: area num 2. [ 0.004000] Memory: 3477160K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11060K bss, 174080K reserved, 0K cma-reserved) [ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 [ 0.004000] After mm_init [ 0.004000] After poking_init [ 0.004000] ftrace: allocating 38664 entries in 152 pages [ 0.004000] ftrace: allocated 152 pages with 3 groups [ 0.004000] Dynamic Preempt: full [ 0.004000] After sched_init [ 0.004000] rcu: Preemptible hierarchical RCU implementation. [ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. [ 0.004000] Trampoline variant of Tasks RCU enabled. [ 0.004000] Rude variant of Tasks RCU enabled. [ 0.004000] Tracing variant of Tasks RCU enabled. [ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 [ 0.004000] After rcu_init [ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16 [ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.004000] After random_init() [ 0.004000] After boot_init_stack_canary [ 0.004000] spurious 8259A interrupt: IRQ7. [ 0.004000] Console: colour VGA+ 80x25 [ 0.004000] printk: console [tty0] enabled [ 0.004000] ACPI: Core revision 20221020 [ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns [ 0.004000] APIC: Switch to symmetric I/O mode setup [ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0 [ 0.004000] APIC: Done [ 0.004000] Before apic_bsb_setup [ 0.004000] check_timer begin [ 0.004000] check_timer after local_irq_disable [ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7070b6fd6fc, max_idle_ns: 881590611710 ns [ 0.144982] Calibrating delay loop (skipped), value calculated using timer frequency.. 7800.54 BogoMIPS (lpj=15601092) [ 0.144986] pid_max: default: 32768 minimum: 301 [ 0.145079] LSM: initializing lsm=capability [ 0.145173] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) [ 0.145189] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) [ 0.145564] Bit 30 in CPUID ECX not set. [ 0.145587] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512 [ 0.145589] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0 [ 0.145594] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 0.145597] Spectre V2 : Mitigation: Retpolines [ 0.145598] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 0.145599] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT [ 0.145600] Spectre V2 : Enabling Speculation Barrier for firmware calls [ 0.145600] RETBleed: Mitigation: untrained return thunk [ 0.145602] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier [ 0.145604] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl [ 0.150046] Freeing SMP alternatives memory: 32K [ 0.150051] After check_bugs [ 0.150052] After acpi_subsystem_init [ 0.150053] After arch_post_acpi_subsys_init [ 0.150054] After rcu_scheduler_starting [ 0.150126] After find_task_by_pid_ns and PF_NO_SETAFFINITY [ 0.150131] After numa_default_policy [ 0.150151] After rcu_read_lock [ 0.150152] After rcu_read_unlock [ 0.150153] After kthreadd_done [ 0.150165] smpboot: Start of smp_prepare_cpus_common [ 0.150167] smpboot: smpboot: zalloc 0 [ 0.150168] smpboot: smpboot: zalloc 1 [ 0.150169] smpboot: smpboot: After set_sched_topology() [ 0.150171] smpboot: smpboot: After smp_sanity_check() [ 0.150172] smpboot: smpboot: Before x86_init.timers.setup_percpu_clockev() [ 0.258192] smpboot: smpboot: After x86_init.timers.setup_percpu_clockev() [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1) [ 0.258425] cblist_init_generic: Setting adjustable number of callback queues. [ 0.258427] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258458] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258484] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258513] Performance Events: Fam15h core perfctr, AMD PMU driver. [ 0.258536] ... version: 0 [ 0.258537] ... bit width: 48 [ 0.258538] ... generic registers: 6 [ 0.258539] ... value mask: 0000ffffffffffff [ 0.258540] ... max period: 00007fffffffffff [ 0.258541] ... fixed-purpose events: 0 [ 0.258542] ... event mask: 000000000000003f [ 0.258662] rcu: Hierarchical SRCU implementation. [ 0.258663] rcu: Max phase no-delay instances is 1000. [ 0.259257] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. [ 0.259329] smp: Bringing up secondary CPUs ... [ 0.259527] x86: Booting SMP configuration: [ 0.259528] .... node #0, CPUs: #1 [ 0.261007] After schedule_preempt_disabled [ 10.260990] CPU1 failed to report alive state [ 10.261070] smp: Brought up 1 node, 1 CPU [ 10.261073] smpboot: Max logical packages: 2 [ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS) [ 10.261601] devtmpfs: initialized [ 10.261697] x86/mm: Memory block size: 128MB [ 10.262788] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 10.262795] futex hash table entries: 512 (order: 3, 32768 bytes, linear) [ 10.262883] pinctrl core: initialized pinctrl subsystem [ 10.262953] PM: RTC time: 07:39:50, date: 2023-04-17 [ 10.263709] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 10.263937] audit: initializing netlink subsys (disabled) [ 10.264192] thermal_sys: Registered thermal governor 'fair_share' [ 10.264193] thermal_sys: Registered thermal governor 'bang_bang' [ 10.264194] thermal_sys: Registered thermal governor 'step_wise' [ 10.264195] thermal_sys: Registered thermal governor 'user_space' [ 10.264215] cpuidle: using governor ladder [ 10.264220] cpuidle: using governor menu [ 10.264427] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000) [ 10.264432] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry [ 10.264444] PCI: Using configuration type 1 for base access [ 10.264646] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 10.269064] audit: type=2000 audit(1681717190.140:1): state=initialized audit_enabled=0 res=1 [ 10.281080] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages [ 10.281084] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page [ 10.281085] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages [ 10.281086] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page [ 10.286166] cryptd: max_cpu_qlen set to 1000 [ 10.289615] ACPI: Added _OSI(Module Device) [ 10.289617] ACPI: Added _OSI(Processor Device) [ 10.289618] ACPI: Added _OSI(3.0 _SCP Extensions) [ 10.289620] ACPI: Added _OSI(Processor Aggregator Device) [ 10.293942] ACPI: DSDT successfully acquired and loaded [ 10.295626] ACPI: 4 ACPI AML tables successfully acquired and loaded [ 10.297161] ACPI: Interpreter enabled [ 10.297184] ACPI: PM: (supports S0 S1 S3 S5) [ 10.297186] ACPI: Using IOAPIC for interrupt routing [ 10.297236] HEST: Table parsing has been initialized. [ 10.297258] GHES: Failed to enable APEI firmware first mode. [ 10.297261] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 10.297262] PCI: Ignoring E820 reservations for host bridge windows [ 10.297510] ACPI: Enabled 8 GPEs in block 00 to 1F [ 10.303230] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 10.303241] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] [ 10.303320] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR] [ 10.303335] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge [ 10.303414] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored [ 10.303651] PCI host bridge to bus 0000:00 [ 10.303653] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 10.303655] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 10.303658] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff] [ 10.303660] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff] [ 10.303662] pci_bus 0000:00: root bus resource [bus 00-ff] [ 10.303686] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000 [ 10.303830] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600 [ 10.303921] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000 [ 10.303929] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref] [ 10.303934] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff] [ 10.303939] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff] [ 10.303955] pci 0000:00:01.0: enabling Extended Tags [ 10.303965] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] [ 10.303983] pci 0000:00:01.0: supports D1 D2 [ 10.304048] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300 [ 10.304056] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff] [ 10.304077] pci 0000:00:01.1: enabling Extended Tags [ 10.304101] pci 0000:00:01.1: supports D1 D2 [ 10.304185] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601 [ 10.304198] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417] [ 10.304206] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423] [ 10.304213] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f] [ 10.304221] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427] [ 10.304228] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f] [ 10.304235] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff] [ 10.304392] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310 [ 10.304406] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff] [ 10.304587] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320 [ 10.304601] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff] [ 10.304666] pci 0000:00:12.2: supports D1 D2 [ 10.304667] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot [ 10.304668] pci 0000:00:12.2: pme_poll = true [ 10.304669] pci 0000:00:12.2: after device_set_wakeup_capable() [ 10.304673] pci 0000:00:12.2: after pci_pme_active() [ 10.304814] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310 [ 10.304827] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff] [ 10.305566] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320 [ 10.305583] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff] [ 10.305654] pci 0000:00:13.2: supports D1 D2 [ 10.305656] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot [ 10.305657] pci 0000:00:13.2: pme_poll = true [ 10.305659] pci 0000:00:13.2: after device_set_wakeup_capable() [ 10.305662] pci 0000:00:13.2: after pci_pme_active() [ 10.305803] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500 [ 10.305975] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300 [ 10.305992] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit] [ 10.306047] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold [ 10.306049] pci 0000:00:14.2: pme_poll = true [ 10.306050] pci 0000:00:14.2: after device_set_wakeup_capable() [ 10.306052] pci 0000:00:14.2: after pci_pme_active() [ 10.306185] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100 [ 10.306360] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401 [ 10.306502] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310 [ 10.306515] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff] [ 10.306682] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400 [ 10.306711] pci 0000:00:15.0: enabling Extended Tags [ 10.306751] pci 0000:00:15.0: supports D1 D2 [ 10.306911] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400 [ 10.306942] pci 0000:00:15.1: enabling Extended Tags [ 10.306981] pci 0000:00:15.1: supports D1 D2 [ 10.307137] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400 [ 10.307166] pci 0000:00:15.2: enabling Extended Tags [ 10.307205] pci 0000:00:15.2: supports D1 D2 [ 10.307280] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310 [ 10.307293] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff] [ 10.307467] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320 [ 10.307481] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff] [ 10.307546] pci 0000:00:16.2: supports D1 D2 [ 10.307547] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot [ 10.307548] pci 0000:00:16.2: pme_poll = true [ 10.307549] pci 0000:00:16.2: after device_set_wakeup_capable() [ 10.307552] pci 0000:00:16.2: after pci_pme_active() [ 10.307691] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000 [ 10.307758] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000 [ 10.307819] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000 [ 10.307880] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000 [ 10.308016] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000 [ 10.308079] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000 [ 10.308153] pci_bus 0000:01: extended config space not accessible [ 10.308218] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode) [ 10.308227] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode) [ 10.308230] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode) [ 10.308232] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode) [ 10.308235] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode) [ 10.308283] pci 0000:00:15.0: PCI bridge to [bus 02] [ 10.308368] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330 [ 10.308404] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit] [ 10.308579] pci 0000:03:00.0: PME# supported from D3hot D3cold [ 10.308581] pci 0000:03:00.0: pme_poll = true [ 10.308582] pci 0000:03:00.0: after device_set_wakeup_capable() [ 10.308587] pci 0000:03:00.0: after pci_pme_active() [ 10.308625] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link) [ 10.317133] pci 0000:00:15.1: PCI bridge to [bus 03] [ 10.317145] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff] [ 10.317154] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring [ 10.317278] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000 [ 10.317296] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff] [ 10.317318] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref] [ 10.317332] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref] [ 10.317440] pci 0000:04:00.0: supports D1 D2 [ 10.317442] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 10.317444] pci 0000:04:00.0: pme_poll = true [ 10.317445] pci 0000:04:00.0: after device_set_wakeup_capable() [ 10.317449] pci 0000:04:00.0: after pci_pme_active() [ 10.329042] pci 0000:00:15.2: PCI bridge to [bus 04-ff] [ 10.329053] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff] [ 10.329057] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff] [ 10.329062] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref] [ 10.329065] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04 [ 10.329575] ACPI: PCI: Interrupt link INTA configured for IRQ 0 [ 10.329669] ACPI: PCI: Interrupt link INTB configured for IRQ 0 [ 10.329759] ACPI: PCI: Interrupt link INTC configured for IRQ 0 [ 10.329849] ACPI: PCI: Interrupt link INTD configured for IRQ 0 [ 10.329940] ACPI: PCI: Interrupt link INTE configured for IRQ 0 [ 10.330029] ACPI: PCI: Interrupt link INTF configured for IRQ 0 [ 10.330119] ACPI: PCI: Interrupt link INTG configured for IRQ 0 [ 10.330210] ACPI: PCI: Interrupt link INTH configured for IRQ 0 [ 10.330444] iommu: Default domain type: Translated [ 10.330446] iommu: DMA domain TLB invalidation policy: lazy mode [ 10.330624] SCSI subsystem initialized [ 10.330718] libata version 3.00 loaded. [ 10.330752] ACPI: bus type USB registered [ 10.330775] usbcore: registered new interface driver usbfs [ 10.330786] usbcore: registered new interface driver hub [ 10.330798] usbcore: registered new device driver usb [ 10.331125] PCI: Using ACPI for IRQ routing [ 10.332694] PCI: pci_cache_line_size set to 64 bytes [ 10.332745] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 10.332748] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff] [ 10.332750] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff] [ 10.332794] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 10.332799] hpet0: 3 comparators, 32-bit 14.318180 MHz counter [ 10.334067] clocksource: Switched to clocksource tsc-early [ 10.350705] VFS: Disk quotas dquot_6.6.0 [ 10.350734] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 10.350847] pnp: PnP ACPI init [ 10.351137] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved [ 10.351446] pnp: PnP ACPI: found 2 devices [ 10.358408] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 10.358544] NET: Registered PF_INET protocol family [ 10.358686] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 10.360256] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear) [ 10.360271] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 10.360277] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear) [ 10.360344] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear) [ 10.360670] TCP: Hash tables configured (established 32768 bind 32768) [ 10.360739] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear) [ 10.360763] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear) [ 10.360870] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 10.360906] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref] [ 10.360912] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff] [ 10.360917] pci 0000:00:14.4: PCI bridge to [bus 01] [ 10.360928] pci 0000:00:15.0: PCI bridge to [bus 02] [ 10.360936] pci 0000:00:15.1: PCI bridge to [bus 03] [ 10.360939] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff] [ 10.360947] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref] [ 10.360959] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref] [ 10.360971] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff] [ 10.360976] pci 0000:00:15.2: PCI bridge to [bus 04] [ 10.361116] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff] [ 10.361122] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref] [ 10.361128] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 10.361130] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 10.361132] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff] [ 10.361133] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff] [ 10.361135] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window] [ 10.361137] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window] [ 10.361138] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff] [ 10.361140] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff] [ 10.361141] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff] [ 10.361143] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff] [ 10.361144] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref] [ 10.361239] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0 [ 10.361456] pci 0000:00:12.0: AMD USB device [ 10.361478] pci 0000:00:12.0: AMD USB ohci handoff [ 10.361761] pci 0000:00:12.2: AMD USB device [ 10.361775] pci 0000:00:12.2: AMD USB ehci handoff [ 10.361903] pci 0000:00:12.2: PME# does not work under D3, disabling it [ 10.362049] pci 0000:00:13.0: AMD USB device [ 10.362062] pci 0000:00:13.0: AMD USB ohci handoff [ 10.362328] pci 0000:00:13.2: AMD USB device [ 10.362339] pci 0000:00:13.2: AMD USB ehci handoff [ 10.362469] pci 0000:00:13.2: PME# does not work under D3, disabling it [ 10.362622] pci 0000:00:14.5: AMD USB device [ 10.362634] pci 0000:00:14.5: AMD USB ohci handoff [ 10.362915] pci 0000:00:16.0: AMD USB device [ 10.362928] pci 0000:00:16.0: AMD USB ohci handoff [ 10.363195] pci 0000:00:16.2: AMD USB device [ 10.363206] pci 0000:00:16.2: AMD USB ehci handoff [ 10.363334] pci 0000:00:16.2: PME# does not work under D3, disabling it [ 10.363561] pci 0000:03:00.0: AMD USB xhci handoff [ 10.363610] PCI: CLS 64 bytes, default 64 [ 10.363723] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround [ 10.363810] pci 0000:00:01.0: Adding to iommu group 0 [ 10.363826] pci 0000:00:01.1: Adding to iommu group 0 [ 10.363851] pci 0000:00:11.0: Adding to iommu group 1 [ 10.363886] pci 0000:00:12.0: Adding to iommu group 2 [ 10.363903] pci 0000:00:12.2: Adding to iommu group 2 [ 10.363937] pci 0000:00:13.0: Adding to iommu group 3 [ 10.363953] pci 0000:00:13.2: Adding to iommu group 3 [ 10.363991] pci 0000:00:14.0: Adding to iommu group 4 [ 10.364009] pci 0000:00:14.2: Adding to iommu group 4 [ 10.364025] pci 0000:00:14.3: Adding to iommu group 4 [ 10.364048] pci 0000:00:14.4: Adding to iommu group 5 [ 10.364070] pci 0000:00:14.5: Adding to iommu group 6 [ 10.364104] pci 0000:00:15.0: Adding to iommu group 7 [ 10.364123] pci 0000:00:15.1: Adding to iommu group 7 [ 10.364139] pci 0000:00:15.2: Adding to iommu group 7 [ 10.364178] pci 0000:00:16.0: Adding to iommu group 8 [ 10.364194] pci 0000:00:16.2: Adding to iommu group 8 [ 10.364246] pci 0000:00:18.0: Adding to iommu group 9 [ 10.364266] pci 0000:00:18.1: Adding to iommu group 9 [ 10.364283] pci 0000:00:18.2: Adding to iommu group 9 [ 10.364303] pci 0000:00:18.3: Adding to iommu group 9 [ 10.364321] pci 0000:00:18.4: Adding to iommu group 9 [ 10.364340] pci 0000:00:18.5: Adding to iommu group 9 [ 10.364352] pci 0000:03:00.0: Adding to iommu group 7 [ 10.364360] pci 0000:04:00.0: Adding to iommu group 7 [ 10.366500] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 [ 10.366505] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA [ 10.366510] AMD-Vi: Interrupt remapping enabled [ 10.366691] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 10.366693] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB) [ 10.366742] LVT offset 0 assigned for vector 0x400 [ 10.366763] perf: AMD IBS detected (0x000000ff) [ 10.366771] amd_uncore: 4 amd_nb counters detected [ 10.367574] workingset: timestamp_bits=37 max_order=20 bucket_order=0 [ 10.367603] zbud: loaded [ 10.368066] NET: Registered PF_ALG protocol family [ 10.368071] Key type asymmetric registered [ 10.368073] Asymmetric key parser 'x509' registered [ 10.368346] alg: self-tests disabled [ 10.368439] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251) [ 10.368477] io scheduler mq-deadline registered [ 10.368479] io scheduler kyber registered [ 10.370011] pcieport 0000:00:15.0: PME: Signaling with IRQ 25 [ 10.370174] pcieport 0000:00:15.1: PME: Signaling with IRQ 26 [ 10.370247] pcieport 0000:00:15.2: enabling device (0000 -> 0003) [ 10.370449] pcieport 0000:00:15.2: PME: Signaling with IRQ 27 [ 10.370706] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 10.370767] ACPI: button: Power Button [PWRF] [ 10.370823] ACPI: \_SB_.P000: Found 2 idle states [ 10.370937] ACPI: \_SB_.P001: Found 2 idle states [ 10.371827] thermal LNXTHERM:00: registered as thermal_zone0 [ 10.371830] ACPI: thermal: Thermal Zone [TZ00] (0 C) [ 10.372146] Non-volatile memory driver v1.3 [ 10.372218] AMD-Vi: AMD IOMMUv2 loaded and initialized [ 10.372338] ahci 0000:00:11.0: version 3.0 [ 10.372612] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode [ 10.372616] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio [ 10.374133] scsi host0: ahci [ 10.374333] scsi host1: ahci [ 10.374510] scsi host2: ahci [ 10.374710] scsi host3: ahci [ 10.374879] scsi host4: ahci [ 10.375057] scsi host5: ahci [ 10.375241] scsi host6: ahci [ 10.375421] scsi host7: ahci [ 10.375508] ata port1: DUMMY [ 10.375510] ata port2: DUMMY [ 10.375511] ata port3: DUMMY [ 10.375512] ata port4: DUMMY [ 10.375514] ata port5: DUMMY [ 10.375515] ata port6: DUMMY [ 10.375517] ata port7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19 [ 10.375519] ata port8: DUMMY [ 10.375597] ACPI: bus type drm_connector registered [ 10.375823] i8042: PNP: No PS/2 controller found. [ 10.375824] i8042: Probing ports directly. [ 10.378675] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 10.378751] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 10.378874] mousedev: PS/2 mouse device common for all mice [ 10.378927] rtc_cmos 00:01: RTC can wake from S4 [ 10.379173] rtc_cmos 00:01: registered as rtc0 [ 10.379197] rtc_cmos 00:01: setting system clock to 2023-04-17T07:39:50 UTC (1681717190) [ 10.379234] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs [ 10.379268] device-mapper: uevent: version 1.0.3 [ 10.379337] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com [ 10.379496] hid: raw HID events driver (C) Jiri Kosina [ 10.379531] usbcore: registered new interface driver usbhid [ 10.379532] usbhid: USB HID core driver [ 10.379638] Initializing XFRM netlink socket [ 10.379648] NET: Registered PF_PACKET protocol family [ 10.379650] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending. [ 10.379817] microcode: CPU0: patch_level=0x0600111f [ 10.379827] microcode: Microcode Update Driver: v2.2. [ 10.379831] IPI shorthand broadcast: enabled [ 10.379838] AVX version of gcm_enc/dec engaged. [ 10.379854] AES CTR mode by8 optimization enabled [ 10.380695] ata link7: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 10.380969] ata dev7.0: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133 [ 10.380972] ata dev7.0: 125045424 sectors, multi 1: LBA48 NCQ (depth 32) [ 10.381178] ata dev7.0: configured for UDMA/133 [ 10.381289] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5 [ 10.381718] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB) [ 10.381730] sd 6:0:0:0: [sda] Write Protect is off [ 10.381733] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00 [ 10.381749] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 10.381775] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes [ 10.382547] sda: sda1 sda2 sda3 [ 10.382736] sd 6:0:0:0: [sda] Attached SCSI disk [ 10.385344] sched_clock: Marking stable (10268007974, 116977866)->(10387423156, -2437316) [ 10.385519] registered taskstats version 1 [ 10.385734] zswap: loaded using pool lzo/zbud [ 10.390028] kmemleak: Kernel memory leak detector initialized (mem pool available: 15677) [ 10.390033] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 10.393894] kmemleak: Automatic memory scanning thread started [ 10.394488] Key type encrypted registered [ 10.397453] PM: Magic number: 3:139:673 [ 10.397466] workqueue scsi_tmf_3: hash matches [ 10.410334] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none. [ 10.410378] VFS: Mounted root (ext4 filesystem) on device 8:3. [ 10.412268] devtmpfs: mounted [ 10.412286] After kernel_init_freeable [ 10.416793] Freeing unused kernel image (initmem) memory: 2908K [ 10.421291] Write protecting the kernel read-only data: 20480k [ 10.421559] Freeing unused kernel image (rodata/data gap) memory: 836K [ 10.458614] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 10.458620] rodata_test: all tests were successful [ 10.458621] After mark_readonly [ 10.458621] After pti_finalize [ 10.458637] rcu_end_inkernel_boot [ 10.458644] Run /sbin/init as init process [ 10.458646] with arguments: [ 10.458648] /sbin/init [ 10.458649] noisapnp [ 10.458650] with environment: [ 10.458650] HOME=/ [ 10.458651] TERM=linux [ 10.458652] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 [ 10.637190] systemd[1]: Inserted module 'autofs4' [ 10.663625] NET: Registered PF_INET6 protocol family [ 10.664473] Segment Routing with IPv6 [ 10.664500] In-situ OAM (IOAM) with IPv6 [ 10.691376] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) [ 10.691387] systemd[1]: Detected architecture x86-64. [ 10.696468] systemd[1]: Hostname set to <kodi>. [ 11.000700] systemd[1]: Queued start job for default target graphical.target. [ 11.011651] systemd[1]: Created slice system-getty.slice - Slice /system/getty. [ 11.012747] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe. [ 11.013600] systemd[1]: Created slice user.slice - User and Session Slice. [ 11.013784] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. [ 11.013902] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch. [ 11.014324] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point. [ 11.014366] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes. [ 11.014416] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes. [ 11.014455] systemd[1]: Reached target paths.target - Path Units. [ 11.014485] systemd[1]: Reached target remote-fs.target - Remote File Systems. [ 11.014515] systemd[1]: Reached target slices.target - Slice Units. [ 11.014554] systemd[1]: Reached target swap.target - Swaps. [ 11.014592] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes. [ 11.017065] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket. [ 11.017317] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket. [ 11.017479] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe. [ 11.017786] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket. [ 11.018059] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log). [ 11.018337] systemd[1]: Listening on systemd-journald.socket - Journal Socket. [ 11.018594] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket. [ 11.019420] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket. [ 11.019693] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket. [ 11.022504] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System... [ 11.025189] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System... [ 11.029590] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System... [ 11.046849] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System... [ 11.053730] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes... [ 11.066015] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs... [ 11.069414] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod... [ 11.076547] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm... [ 11.090852] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore... [ 11.097815] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse... [ 11.109388] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop... [ 11.109456] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes). [ 11.109525] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/). [ 11.109562] systemd[1]: Reached target local-fs.target - Local File Systems. [ 11.109629] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor). [ 11.120810] loop: module loaded [ 11.122003] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats... [ 11.129598] fuse: init (API version 7.38) [ 11.137426] systemd[1]: Starting systemd-journald.service - Journal Service... [ 11.145752] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed... [ 11.157494] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables... [ 11.172634] systemd[1]: Starting systemd-sysusers.service - Create System Users... [ 11.197939] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices... [ 11.216247] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System. [ 11.216451] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System. [ 11.216611] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System. [ 11.216772] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System. [ 11.233481] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes. [ 11.234279] systemd[1]: modprobe@configfs.service: Deactivated successfully. [ 11.238519] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs. [ 11.239177] systemd[1]: modprobe@dm_mod.service: Deactivated successfully. [ 11.247227] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod. [ 11.247980] systemd[1]: modprobe@drm.service: Deactivated successfully. [ 11.253523] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm. [ 11.254233] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully. [ 11.255540] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore. [ 11.256109] systemd[1]: modprobe@fuse.service: Deactivated successfully. [ 11.261747] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse. [ 11.262375] systemd[1]: modprobe@loop.service: Deactivated successfully. [ 11.265991] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop. [ 11.267213] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables. [ 11.268186] systemd[1]: Finished systemd-sysusers.service - Create System Users. [ 11.268690] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 133 (systemd-binfmt) [ 11.282799] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System... [ 11.309441] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System... [ 11.358583] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System... [ 11.358687] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore). [ 11.358816] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met. [ 11.377438] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev... [ 11.378820] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System. [ 11.385019] tsc: Refined TSC clocksource calibration: 3900.223 MHz [ 11.385026] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns [ 11.385038] clocksource: Switched to clocksource tsc [ 11.397465] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats. [ 11.397822] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System. [ 11.397985] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System. [ 11.448710] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. [ 11.457254] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files... [ 11.509082] systemd[1]: Started systemd-journald.service - Journal Service. [ 11.569863] systemd-journald[134]: Received client request to flush runtime journal. [ 12.053221] sd 6:0:0:0: Attached scsi generic sg0 type 0 [ 12.140997] random: crng init done [ 12.367954] acpi_cpufreq: overriding BIOS provided _PSD data [ 12.510747] QUIRK: Enable AMD PLL fix [ 12.510804] ehci-pci 0000:00:12.2: EHCI Host Controller [ 12.510833] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1 [ 12.510844] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 12.510853] ehci-pci 0000:00:12.2: debug port 1 [ 12.511023] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000 [ 12.515767] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0 [ 12.515773] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection [ 12.516241] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20 [ 12.525014] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00 [ 12.525397] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 12.525400] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.525402] usb usb1: Product: EHCI Host Controller [ 12.525404] usb usb1: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd [ 12.525405] usb usb1: SerialNumber: 0000:00:12.2 [ 12.525862] hub 1-0:1.0: USB hub found [ 12.525889] hub 1-0:1.0: 5 ports detected [ 12.526592] ehci-pci 0000:00:13.2: EHCI Host Controller [ 12.526612] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2 [ 12.526623] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 12.526632] ehci-pci 0000:00:13.2: debug port 1 [ 12.526767] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000 [ 12.541014] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00 [ 12.541263] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 12.541266] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.541268] usb usb2: Product: EHCI Host Controller [ 12.541270] usb usb2: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd [ 12.541271] usb usb2: SerialNumber: 0000:00:13.2 [ 12.541718] hub 2-0:1.0: USB hub found [ 12.541746] hub 2-0:1.0: 5 ports detected [ 12.542418] ehci-pci 0000:00:16.2: EHCI Host Controller [ 12.542436] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3 [ 12.542447] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 12.542456] ehci-pci 0000:00:16.2: debug port 1 [ 12.542584] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000 [ 12.557023] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00 [ 12.557405] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 12.557408] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.557410] usb usb3: Product: EHCI Host Controller [ 12.557411] usb usb3: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd [ 12.557413] usb usb3: SerialNumber: 0000:00:16.2 [ 12.557858] hub 3-0:1.0: USB hub found [ 12.557884] hub 3-0:1.0: 4 ports detected [ 12.558447] ohci-pci 0000:00:12.0: OHCI PCI host controller [ 12.558466] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4 [ 12.558608] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000 [ 12.558618] ohci-pci 0000:00:13.0: OHCI PCI host controller [ 12.558635] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 5 [ 12.558703] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000 [ 12.558710] ohci-pci 0000:00:14.5: OHCI PCI host controller [ 12.558725] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 6 [ 12.558793] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000 [ 12.558800] ohci-pci 0000:00:16.0: OHCI PCI host controller [ 12.558814] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 7 [ 12.558888] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000 [ 12.635709] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 12.635716] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.635718] usb usb7: Product: OHCI PCI host controller [ 12.635720] usb usb7: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd [ 12.635721] usb usb7: SerialNumber: 0000:00:16.0 [ 12.636145] hub 7-0:1.0: USB hub found [ 12.636172] hub 7-0:1.0: 4 ports detected [ 12.636849] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 12.636851] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.636853] usb usb5: Product: OHCI PCI host controller [ 12.636854] usb usb5: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd [ 12.636856] usb usb5: SerialNumber: 0000:00:13.0 [ 12.639549] hub 5-0:1.0: USB hub found [ 12.639579] hub 5-0:1.0: 5 ports detected [ 12.640465] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 12.640468] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.640470] usb usb4: Product: OHCI PCI host controller [ 12.640471] usb usb4: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd [ 12.640473] usb usb4: SerialNumber: 0000:00:12.0 [ 12.640848] hub 4-0:1.0: USB hub found [ 12.640874] hub 4-0:1.0: 5 ports detected [ 12.644079] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 12.644084] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.644086] usb usb6: Product: OHCI PCI host controller [ 12.644088] usb usb6: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd [ 12.644089] usb usb6: SerialNumber: 0000:00:14.5 [ 12.644484] hub 6-0:1.0: USB hub found [ 12.644509] hub 6-0:1.0: 2 ports detected [ 12.658738] r8169 0000:04:00.0: enabling device (0000 -> 0003) [ 12.697540] xhci_hcd 0000:03:00.0: xHCI Host Controller [ 12.697566] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 8 [ 12.709878] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28 [ 12.709886] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko] [ 12.735756] 1 [ 12.735806] 2 [ 12.736276] snd_hda_intel 0000:00:01.1: Force to non-snoop mode [ 12.736282] 3 [ 12.736283] 4 [ 12.736284] 5 [ 12.736284] 7 [ 12.736288] 8 [ 12.736288] 9 [ 12.740797] 1 [ 12.740846] 2 [ 12.741376] 3 [ 12.741377] 4 [ 12.741378] 5 [ 12.741379] 7 [ 12.741382] 8 [ 12.741383] 9 [ 12.769260] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1 [ 12.769555] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2 [ 12.784439] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100 [ 12.785682] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line [ 12.785688] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0) [ 12.785691] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0) [ 12.785693] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0 [ 12.785694] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0 [ 12.785696] snd_hda_codec_realtek hdaudioC1D0: inputs: [ 12.785697] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18 [ 12.785699] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19 [ 12.785700] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a [ 12.785701] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c [ 12.807528] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010 [ 12.813178] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3 [ 12.813514] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4 [ 12.813782] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5 [ 12.814048] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6 [ 12.814317] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7 [ 12.814575] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8 [ 12.814833] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9 [ 12.815087] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10 [ 12.816233] xhci_hcd 0000:03:00.0: xHCI Host Controller [ 12.816249] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 9 [ 12.816260] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed [ 12.830985] usb usb8: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 12.830993] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.830995] usb usb8: Product: xHCI Host Controller [ 12.830996] usb usb8: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd [ 12.830998] usb usb8: SerialNumber: 0000:03:00.0 [ 12.835543] hub 8-0:1.0: USB hub found [ 12.842296] hub 8-0:1.0: 2 ports detected [ 12.850842] usb usb9: We don't know the algorithms for LPM for this host, disabling LPM. [ 12.850983] usb usb9: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03 [ 12.850986] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.850988] usb usb9: Product: xHCI Host Controller [ 12.850989] usb usb9: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd [ 12.850991] usb usb9: SerialNumber: 0000:03:00.0 [ 12.856278] r8169 0000:04:00.0 enp4s0: renamed from eth0 [ 12.862771] hub 9-0:1.0: USB hub found [ 12.873601] hub 9-0:1.0: 2 ports detected [ 13.025026] usb 4-1: new low-speed USB device number 2 using ohci-pci [ 13.103377] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2 [ 13.103389] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2) [ 13.103907] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC) [ 13.175505] r8169 0000:04:00.0 enp4s0: Link is Down [ 13.179870] [drm] radeon kernel modesetting enabled. [ 13.181586] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00). [ 13.181652] ATOM BIOS: 113 [ 13.181757] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used) [ 13.181761] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF [ 13.181769] [drm] Detected VRAM RAM=512M, BAR=256M [ 13.181770] [drm] RAM width 64bits DDR [ 13.181947] [drm] radeon: 512M of VRAM memory ready [ 13.181952] [drm] radeon: 1024M of GTT memory ready. [ 13.181994] [drm] Loading ARUBA Microcode [ 13.190350] [drm] Internal thermal controller without fan control [ 13.191100] [drm] radeon: dpm initialized [ 13.196015] [drm] Found VCE firmware/feedback version 50.0.1 / 17! [ 13.196073] [drm] GART: num cpu pages 262144, num gpu pages 262144 [ 13.234337] [drm] GART: Restore entries: num cpu pages 262144, num gpu pages 262144 [ 13.237934] [drm] GART: Done restoring entries [ 13.237938] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000). [ 13.238177] radeon 0000:00:01.0: WB enabled [ 13.238180] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00 [ 13.238558] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18 [ 13.246045] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01 [ 13.246050] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 13.246052] usb 4-1: Product: Dell QuietKey Keyboard [ 13.246054] usb 4-1: Manufacturer: DELL [ 13.253819] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11 [ 13.260936] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18 [ 13.260940] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c [ 13.260942] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04 [ 13.260944] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08 [ 13.260945] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c [ 13.260947] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10 [ 13.262964] radeon 0000:00:01.0: radeon: MSI limited to 32-bit [ 13.263151] radeon 0000:00:01.0: radeon: using MSI. [ 13.263222] [drm] radeon: irq initialized. [ 13.281648] [drm] ring test on 0 succeeded in 3 usecs [ 13.281658] [drm] ring test on 3 succeeded in 4 usecs [ 13.281665] [drm] ring test on 4 succeeded in 4 usecs [ 13.295659] [drm] ring test on 5 succeeded in 2 usecs [ 13.297656] [drm] UVD initialized successfully. [ 13.313663] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0 [ 13.446953] [drm] ring test on 6 succeeded in 18 usecs [ 13.446967] [drm] ring test on 7 succeeded in 3 usecs [ 13.446968] [drm] VCE initialized successfully. [ 13.447122] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon]) [ 13.447293] [drm] ib test on ring 0 succeeded in 0 usecs [ 13.447346] [drm] ib test on ring 3 succeeded in 0 usecs [ 13.447396] [drm] ib test on ring 4 succeeded in 0 usecs [ 13.465099] [drm] ib test on ring 5 succeeded [ 13.481132] [drm] ib test on ring 6 succeeded in 1 usecs [ 13.497085] [drm] ib test on ring 7 succeeded in 1 usecs [ 13.500056] [drm] Radeon Display Connectors [ 13.500060] [drm] Connector 0: [ 13.500061] [drm] DP-1 [ 13.500062] [drm] HPD1 [ 13.500062] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c [ 13.500065] [drm] Encoders: [ 13.500065] [drm] DFP1: INTERNAL_UNIPHY2 [ 13.500066] [drm] Connector 1: [ 13.500067] [drm] VGA-1 [ 13.500068] [drm] HPD2 [ 13.500069] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c [ 13.500070] [drm] Encoders: [ 13.500071] [drm] CRT1: INTERNAL_UNIPHY2 [ 13.500072] [drm] CRT1: NUTMEG [ 13.500072] [drm] Connector 2: [ 13.500073] [drm] HDMI-A-1 [ 13.500074] [drm] HPD3 [ 13.500075] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c [ 13.500076] [drm] Encoders: [ 13.500077] [drm] DFP2: INTERNAL_UNIPHY [ 13.741081] usb 4-2: new low-speed USB device number 3 using ohci-pci [ 13.779818] [drm] fb mappable at 0xE03E9000 [ 13.779826] [drm] vram apper at 0xE0000000 [ 13.779828] [drm] size 5242880 [ 13.779830] [drm] fb depth is 24 [ 13.779832] [drm] pitch is 5120 [ 13.780398] fbcon: radeondrmfb (fb0) is primary device [ 13.936158] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40 [ 13.936167] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 13.936170] usb 4-2: Product: Optical USB Mouse [ 13.936173] usb 4-2: Manufacturer: Logitech [ 13.945345] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12 [ 13.946178] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0 [ 13.968191] Console: switching to colour frame buffer device 160x64 [ 13.973460] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device [ 13.981574] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0 [ 15.876613] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx [ 15.876628] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready [ 16.713266] [drm] amdgpu kernel modesetting enabled. [ 17.110008] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=249 'systemd'
[Correct David’s address] Am 17.04.23 um 13:19 schrieb Paul Menzel: > Dear Thomas, > > > Am 15.04.23 um 01:44 schrieb Thomas Gleixner: > >> This is a complete rework of the parallel bringup patch series (V17) >> >> >> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com >> >> to address the issues which were discovered in review: > > […] > > Thank you very much for your rework. > > I tested this on the ASUS F2A85-M PRO, and get a delay of ten seconds. > > ``` > […] > [   0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD > Graphics (family: 0x15, model: 0x13, stepping: 0x1) > […] > [   0.259329] smp: Bringing up secondary CPUs ... > [   0.259527] x86: Booting SMP configuration: > [   0.259528] .... node #0, CPUs:     #1 > [   0.261007] After schedule_preempt_disabled > [  10.260990] CPU1 failed to report alive state > [  10.261070] smp: Brought up 1 node, 1 CPU > [  10.261073] smpboot: Max logical packages: 2 > [  10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS) > [  10.261601] devtmpfs: initialized > [  10.261697] x86/mm: Memory block size: 128MB > ``` > > This delay has been there with v6.3-rc6-46-gde4664485abbc and some > custom (printk) patches on top and merging dwmw2/parallel-6.2-rc3-v16 > into it. I only tested this. I think dwmw2/parallel-6.2-v17 failed to > build for me, when trying to merge it into Linus’ master version at that > time. I didn’t come around to report it, and you posted your rework, so > I am replying here. > > I am going to try your branch directly in the next days, but just wanted > to report back already. > > > Kind regards, > > Paul
Paul! On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote: > Am 15.04.23 um 01:44 schrieb Thomas Gleixner: > [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD > Graphics (family: 0x15, model: 0x13, stepping: 0x1) > […] > [ 0.259329] smp: Bringing up secondary CPUs ... > [ 0.259527] x86: Booting SMP configuration: > [ 0.259528] .... node #0, CPUs: #1 > [ 0.261007] After schedule_preempt_disabled > [ 10.260990] CPU1 failed to report alive state Weird. CPU1 fails to come up and report that it has reached the synchronization point. Does it work when you add cpuhp.parallel=off on the kernel command line? Thanks, tglx
Dear Thomas, Am 17.04.23 um 16:48 schrieb Thomas Gleixner: > On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote: >> Am 15.04.23 um 01:44 schrieb Thomas Gleixner: >> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD >> Graphics (family: 0x15, model: 0x13, stepping: 0x1) >> […] >> [ 0.259329] smp: Bringing up secondary CPUs ... >> [ 0.259527] x86: Booting SMP configuration: >> [ 0.259528] .... node #0, CPUs: #1 >> [ 0.261007] After schedule_preempt_disabled >> [ 10.260990] CPU1 failed to report alive state > > Weird. CPU1 fails to come up and report that it has reached the > synchronization point. > > Does it work when you add cpuhp.parallel=off on the kernel command line? Yes, the ten seconds delay is gone with `cpuhp.parallel=off`. There was a patch set in the past, that worked on that device. I think up to v4 it did *not* work at all and hung [1]. I need some days to collect the results again. Kind regards, Paul [1]: https://lore.kernel.org/lkml/ab28d2ce-4a9c-387d-9eda-558045a0c35b@molgen.mpg.de/ [ 0.000000] Linux version 6.3.0-rc6-00311-gde8224969f66 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #446 SMP PREEMPT_DYNAMIC Sat Apr 15 14:12:29 UTC 2023 [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 cpuhp.parallel=off [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable [ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-gb640ed51b2 04/17/2023 [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Initial usec timer 9249065 [ 0.000000] tsc: Detected 3899.954 MHz processor [ 0.000755] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000 [ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000938] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000 [ 0.004000] Using GB pages for direct mapping [ 0.004000] ACPI: Early table checksum verification disabled [ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4) [ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925) [ 0.004000] ACPI: FACS 0x000000005FE5A240 000040 [ 0.004000] ACPI: FACS 0x000000005FE5A240 000040 [ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925) [ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000) [ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000) [ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001) [ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3] [ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9] [ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f] [ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69] [ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab] [ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11] [ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57] [ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f] [ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71] [ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8] [ 0.004000] No NUMA configuration found [ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff] [ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff] [ 0.004000] Zone ranges: [ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff] [ 0.004000] Device empty [ 0.004000] Movable zone start for each node [ 0.004000] Early memory node ranges [ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff] [ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff] [ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff] [ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges [ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges [ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges [ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges [ 0.004000] ACPI: PM-Timer IO Port: 0x818 [ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) [ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23 [ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level) [ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information [ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000 [ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs [ 0.004000] smpboot: smpboot: XXX end of prefill_possible_map [ 0.004000] After prefill_possible_map [ 0.004000] After init_cpu_to_node [ 0.004000] After init_gi_nodes [ 0.004000] After io_apic_init_mappings [ 0.004000] After x86_init.hyper.guest_late_init [ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices [ 0.004000] After e820 [ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns [ 0.004000] After unwind_init [ 0.004000] After setup_arch [ 0.004000] After setup_command_line [ 0.004000] After setup_nr_cpu_ids [ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1 [ 0.004000] percpu: Embedded 55 pages/cpu s188328 r8192 d28760 u1048576 [ 0.004000] pcpu-alloc: s188328 r8192 d28760 u1048576 alloc=1*2097152 [ 0.004000] pcpu-alloc: [0] 0 1 [ 0.004000] After setup_per_cpu_areas [ 0.004000] After smp_perpare_boot_cpu [ 0.004000] After boot_cpu_hotplug_init [ 0.004000] Fallback order for Node 0: 0 [ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451 [ 0.004000] Policy zone: Normal [ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 cpuhp.parallel=off [ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66", will be passed to user space. [ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) [ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash [ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.004000] software IO TLB: area num 2. [ 0.004000] Memory: 3477168K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11060K bss, 174072K reserved, 0K cma-reserved) [ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 [ 0.004000] After mm_init [ 0.004000] After poking_init [ 0.004000] ftrace: allocating 38664 entries in 152 pages [ 0.004000] ftrace: allocated 152 pages with 3 groups [ 0.004000] Dynamic Preempt: full [ 0.004000] After sched_init [ 0.004000] rcu: Preemptible hierarchical RCU implementation. [ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. [ 0.004000] Trampoline variant of Tasks RCU enabled. [ 0.004000] Rude variant of Tasks RCU enabled. [ 0.004000] Tracing variant of Tasks RCU enabled. [ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 [ 0.004000] After rcu_init [ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16 [ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.004000] After random_init() [ 0.004000] After boot_init_stack_canary [ 0.004000] spurious 8259A interrupt: IRQ7. [ 0.004000] Console: colour VGA+ 80x25 [ 0.004000] printk: console [tty0] enabled [ 0.004000] ACPI: Core revision 20221020 [ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns [ 0.004000] APIC: Switch to symmetric I/O mode setup [ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0 [ 0.004000] APIC: Done [ 0.004000] Before apic_bsb_setup [ 0.004000] check_timer begin [ 0.004000] check_timer after local_irq_disable [ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x706e603bb55, max_idle_ns: 881590819133 ns [ 0.145156] Calibrating delay loop (skipped), value calculated using timer frequency.. 7799.90 BogoMIPS (lpj=15599816) [ 0.145160] pid_max: default: 32768 minimum: 301 [ 0.145254] LSM: initializing lsm=capability [ 0.145348] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) [ 0.145365] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) [ 0.145738] Bit 30 in CPUID ECX not set. [ 0.145762] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512 [ 0.145764] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0 [ 0.145769] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 0.145772] Spectre V2 : Mitigation: Retpolines [ 0.145773] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 0.145774] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT [ 0.145774] Spectre V2 : Enabling Speculation Barrier for firmware calls [ 0.145775] RETBleed: Mitigation: untrained return thunk [ 0.145777] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier [ 0.145779] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl [ 0.150244] Freeing SMP alternatives memory: 32K [ 0.150249] After check_bugs [ 0.150250] After acpi_subsystem_init [ 0.150251] After arch_post_acpi_subsys_init [ 0.150252] After rcu_scheduler_starting [ 0.150324] After find_task_by_pid_ns and PF_NO_SETAFFINITY [ 0.150329] After numa_default_policy [ 0.150349] After rcu_read_lock [ 0.150350] After rcu_read_unlock [ 0.150351] After kthreadd_done [ 0.150363] smpboot: Start of smp_prepare_cpus_common [ 0.150365] smpboot: smpboot: zalloc 0 [ 0.150366] smpboot: smpboot: zalloc 1 [ 0.150367] smpboot: smpboot: After set_sched_topology() [ 0.150369] smpboot: smpboot: After smp_sanity_check() [ 0.150369] smpboot: smpboot: Before x86_init.timers.setup_percpu_clockev() [ 0.258381] smpboot: smpboot: After x86_init.timers.setup_percpu_clockev() [ 0.258382] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1) [ 0.258615] cblist_init_generic: Setting adjustable number of callback queues. [ 0.258618] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258648] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258675] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258703] Performance Events: Fam15h core perfctr, AMD PMU driver. [ 0.258726] ... version: 0 [ 0.258727] ... bit width: 48 [ 0.258728] ... generic registers: 6 [ 0.258729] ... value mask: 0000ffffffffffff [ 0.258730] ... max period: 00007fffffffffff [ 0.258731] ... fixed-purpose events: 0 [ 0.258732] ... event mask: 000000000000003f [ 0.258852] rcu: Hierarchical SRCU implementation. [ 0.258853] rcu: Max phase no-delay instances is 1000. [ 0.259441] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. [ 0.259515] smp: Bringing up secondary CPUs ... [ 0.259714] x86: Booting SMP configuration: [ 0.259715] .... node #0, CPUs: #1 [ 0.121151] Bit 30 in CPUID ECX not set. [ 0.259844] After schedule_preempt_disabled [ 0.259849] smp: Brought up 1 node, 2 CPUs [ 0.259849] smpboot: Max logical packages: 1 [ 0.259849] smpboot: Total of 2 processors activated (15599.81 BogoMIPS) [ 0.261311] devtmpfs: initialized [ 0.261311] x86/mm: Memory block size: 128MB [ 0.262220] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 0.262220] futex hash table entries: 512 (order: 3, 32768 bytes, linear) [ 0.262220] pinctrl core: initialized pinctrl subsystem [ 0.262220] PM: RTC time: 17:24:09, date: 2023-04-17 [ 0.262220] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 0.262434] audit: initializing netlink subsys (disabled) [ 0.262453] audit: type=2000 audit(1681752249.140:1): state=initialized audit_enabled=0 res=1 [ 0.262453] thermal_sys: Registered thermal governor 'fair_share' [ 0.262453] thermal_sys: Registered thermal governor 'bang_bang' [ 0.262453] thermal_sys: Registered thermal governor 'step_wise' [ 0.262453] thermal_sys: Registered thermal governor 'user_space' [ 0.262453] cpuidle: using governor ladder [ 0.262453] cpuidle: using governor menu [ 0.262453] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000) [ 0.262453] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry [ 0.262453] PCI: Using configuration type 1 for base access [ 0.262453] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 0.273235] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages [ 0.273235] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page [ 0.273235] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages [ 0.273235] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page [ 0.275428] cryptd: max_cpu_qlen set to 1000 [ 0.275428] ACPI: Added _OSI(Module Device) [ 0.275428] ACPI: Added _OSI(Processor Device) [ 0.275428] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.275428] ACPI: Added _OSI(Processor Aggregator Device) [ 0.280879] ACPI: DSDT successfully acquired and loaded [ 0.281154] ACPI: 4 ACPI AML tables successfully acquired and loaded [ 0.281649] ACPI: Interpreter enabled [ 0.281669] ACPI: PM: (supports S0 S1 S3 S5) [ 0.281671] ACPI: Using IOAPIC for interrupt routing [ 0.281724] HEST: Table parsing has been initialized. [ 0.281743] GHES: Failed to enable APEI firmware first mode. [ 0.281745] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.281746] PCI: Ignoring E820 reservations for host bridge windows [ 0.282003] ACPI: Enabled 8 GPEs in block 00 to 1F [ 0.286735] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.286746] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] [ 0.286838] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR] [ 0.286852] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge [ 0.286942] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored [ 0.287197] PCI host bridge to bus 0000:00 [ 0.287199] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.287201] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.287203] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff] [ 0.287205] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff] [ 0.287210] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.287234] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000 [ 0.287388] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600 [ 0.287479] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000 [ 0.287488] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref] [ 0.287493] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff] [ 0.287498] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff] [ 0.287514] pci 0000:00:01.0: enabling Extended Tags [ 0.287524] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] [ 0.287541] pci 0000:00:01.0: supports D1 D2 [ 0.287611] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300 [ 0.287618] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff] [ 0.287640] pci 0000:00:01.1: enabling Extended Tags [ 0.287663] pci 0000:00:01.1: supports D1 D2 [ 0.287750] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601 [ 0.287763] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417] [ 0.287771] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423] [ 0.287778] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f] [ 0.287785] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427] [ 0.287793] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f] [ 0.287800] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff] [ 0.287954] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310 [ 0.287967] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff] [ 0.288152] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320 [ 0.288165] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff] [ 0.288230] pci 0000:00:12.2: supports D1 D2 [ 0.288231] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot [ 0.288233] pci 0000:00:12.2: pme_poll = true [ 0.288234] pci 0000:00:12.2: after device_set_wakeup_capable() [ 0.288237] pci 0000:00:12.2: after pci_pme_active() [ 0.288373] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310 [ 0.288387] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff] [ 0.288570] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320 [ 0.288583] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff] [ 0.288650] pci 0000:00:13.2: supports D1 D2 [ 0.288652] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot [ 0.288653] pci 0000:00:13.2: pme_poll = true [ 0.288654] pci 0000:00:13.2: after device_set_wakeup_capable() [ 0.288657] pci 0000:00:13.2: after pci_pme_active() [ 0.288798] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500 [ 0.288980] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300 [ 0.288997] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit] [ 0.289052] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold [ 0.289054] pci 0000:00:14.2: pme_poll = true [ 0.289055] pci 0000:00:14.2: after device_set_wakeup_capable() [ 0.289058] pci 0000:00:14.2: after pci_pme_active() [ 0.290131] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100 [ 0.290335] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401 [ 0.290490] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310 [ 0.290504] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff] [ 0.290677] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400 [ 0.290705] pci 0000:00:15.0: enabling Extended Tags [ 0.290746] pci 0000:00:15.0: supports D1 D2 [ 0.290907] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400 [ 0.290939] pci 0000:00:15.1: enabling Extended Tags [ 0.290978] pci 0000:00:15.1: supports D1 D2 [ 0.291146] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400 [ 0.291174] pci 0000:00:15.2: enabling Extended Tags [ 0.291214] pci 0000:00:15.2: supports D1 D2 [ 0.291289] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310 [ 0.291303] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff] [ 0.291473] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320 [ 0.291486] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff] [ 0.291551] pci 0000:00:16.2: supports D1 D2 [ 0.291552] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot [ 0.291554] pci 0000:00:16.2: pme_poll = true [ 0.291555] pci 0000:00:16.2: after device_set_wakeup_capable() [ 0.291558] pci 0000:00:16.2: after pci_pme_active() [ 0.291684] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000 [ 0.291748] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000 [ 0.291806] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000 [ 0.291866] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000 [ 0.291998] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000 [ 0.292064] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000 [ 0.292136] pci_bus 0000:01: extended config space not accessible [ 0.292200] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode) [ 0.292208] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode) [ 0.292211] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode) [ 0.292214] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode) [ 0.292216] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode) [ 0.292261] pci 0000:00:15.0: PCI bridge to [bus 02] [ 0.292345] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330 [ 0.292382] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit] [ 0.292559] pci 0000:03:00.0: PME# supported from D3hot D3cold [ 0.292560] pci 0000:03:00.0: pme_poll = true [ 0.292562] pci 0000:03:00.0: after device_set_wakeup_capable() [ 0.292567] pci 0000:03:00.0: after pci_pme_active() [ 0.292605] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link) [ 0.305215] pci 0000:00:15.1: PCI bridge to [bus 03] [ 0.305228] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff] [ 0.305237] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring [ 0.305359] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000 [ 0.305378] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff] [ 0.305399] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref] [ 0.305413] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref] [ 0.305520] pci 0000:04:00.0: supports D1 D2 [ 0.305522] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.305524] pci 0000:04:00.0: pme_poll = true [ 0.305525] pci 0000:04:00.0: after device_set_wakeup_capable() [ 0.305529] pci 0000:04:00.0: after pci_pme_active() [ 0.321217] pci 0000:00:15.2: PCI bridge to [bus 04-ff] [ 0.321228] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff] [ 0.321232] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff] [ 0.321237] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref] [ 0.321240] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04 [ 0.321759] ACPI: PCI: Interrupt link INTA configured for IRQ 0 [ 0.321855] ACPI: PCI: Interrupt link INTB configured for IRQ 0 [ 0.321949] ACPI: PCI: Interrupt link INTC configured for IRQ 0 [ 0.322041] ACPI: PCI: Interrupt link INTD configured for IRQ 0 [ 0.322134] ACPI: PCI: Interrupt link INTE configured for IRQ 0 [ 0.322232] ACPI: PCI: Interrupt link INTF configured for IRQ 0 [ 0.322325] ACPI: PCI: Interrupt link INTG configured for IRQ 0 [ 0.322418] ACPI: PCI: Interrupt link INTH configured for IRQ 0 [ 0.322640] iommu: Default domain type: Translated [ 0.322642] iommu: DMA domain TLB invalidation policy: lazy mode [ 0.322839] SCSI subsystem initialized [ 0.325159] libata version 3.00 loaded. [ 0.325159] ACPI: bus type USB registered [ 0.325159] usbcore: registered new interface driver usbfs [ 0.325159] usbcore: registered new interface driver hub [ 0.325159] usbcore: registered new device driver usb [ 0.325159] PCI: Using ACPI for IRQ routing [ 0.325159] PCI: pci_cache_line_size set to 64 bytes [ 0.325159] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 0.325159] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff] [ 0.325159] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff] [ 0.325159] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.325159] hpet0: 3 comparators, 32-bit 14.318180 MHz counter [ 0.330232] clocksource: Switched to clocksource tsc-early [ 0.330474] VFS: Disk quotas dquot_6.6.0 [ 0.330500] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.330620] pnp: PnP ACPI init [ 0.330969] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved [ 0.331362] pnp: PnP ACPI: found 2 devices [ 0.337986] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.338186] NET: Registered PF_INET protocol family [ 0.338359] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 0.340153] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear) [ 0.340169] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 0.340177] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear) [ 0.340251] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear) [ 0.340614] TCP: Hash tables configured (established 32768 bind 32768) [ 0.340685] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear) [ 0.340711] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear) [ 0.340827] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 0.340862] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref] [ 0.340869] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff] [ 0.340874] pci 0000:00:14.4: PCI bridge to [bus 01] [ 0.340886] pci 0000:00:15.0: PCI bridge to [bus 02] [ 0.340894] pci 0000:00:15.1: PCI bridge to [bus 03] [ 0.340898] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff] [ 0.340908] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref] [ 0.340922] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref] [ 0.340934] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff] [ 0.340941] pci 0000:00:15.2: PCI bridge to [bus 04] [ 0.340943] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff] [ 0.340949] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref] [ 0.340956] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.340958] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.340960] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff] [ 0.340964] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff] [ 0.340966] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window] [ 0.340968] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window] [ 0.340970] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff] [ 0.340973] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff] [ 0.340975] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff] [ 0.340977] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff] [ 0.340979] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref] [ 0.341104] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0 [ 0.341412] pci 0000:00:12.0: AMD USB device [ 0.341436] pci 0000:00:12.0: AMD USB ohci handoff [ 0.341833] pci 0000:00:12.2: AMD USB device [ 0.341847] pci 0000:00:12.2: AMD USB ehci handoff [ 0.342032] pci 0000:00:12.2: PME# does not work under D3, disabling it [ 0.342235] pci 0000:00:13.0: AMD USB device [ 0.342250] pci 0000:00:13.0: AMD USB ohci handoff [ 0.342633] pci 0000:00:13.2: AMD USB device [ 0.342650] pci 0000:00:13.2: AMD USB ehci handoff [ 0.342830] pci 0000:00:13.2: PME# does not work under D3, disabling it [ 0.343041] pci 0000:00:14.5: AMD USB device [ 0.343057] pci 0000:00:14.5: AMD USB ohci handoff [ 0.343577] pci 0000:00:16.0: AMD USB device [ 0.343598] pci 0000:00:16.0: AMD USB ohci handoff [ 0.343993] pci 0000:00:16.2: AMD USB device [ 0.344008] pci 0000:00:16.2: AMD USB ehci handoff [ 0.344202] pci 0000:00:16.2: PME# does not work under D3, disabling it [ 0.344510] pci 0000:03:00.0: AMD USB xhci handoff [ 0.344577] PCI: CLS 64 bytes, default 64 [ 0.344678] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround [ 0.344790] pci 0000:00:01.0: Adding to iommu group 0 [ 0.344813] pci 0000:00:01.1: Adding to iommu group 0 [ 0.344844] pci 0000:00:11.0: Adding to iommu group 1 [ 0.344885] pci 0000:00:12.0: Adding to iommu group 2 [ 0.344905] pci 0000:00:12.2: Adding to iommu group 2 [ 0.344949] pci 0000:00:13.0: Adding to iommu group 3 [ 0.344970] pci 0000:00:13.2: Adding to iommu group 3 [ 0.345020] pci 0000:00:14.0: Adding to iommu group 4 [ 0.345043] pci 0000:00:14.2: Adding to iommu group 4 [ 0.345066] pci 0000:00:14.3: Adding to iommu group 4 [ 0.345102] pci 0000:00:14.4: Adding to iommu group 5 [ 0.345128] pci 0000:00:14.5: Adding to iommu group 6 [ 0.345172] pci 0000:00:15.0: Adding to iommu group 7 [ 0.345198] pci 0000:00:15.1: Adding to iommu group 7 [ 0.345219] pci 0000:00:15.2: Adding to iommu group 7 [ 0.345261] pci 0000:00:16.0: Adding to iommu group 8 [ 0.345285] pci 0000:00:16.2: Adding to iommu group 8 [ 0.345349] pci 0000:00:18.0: Adding to iommu group 9 [ 0.345374] pci 0000:00:18.1: Adding to iommu group 9 [ 0.345400] pci 0000:00:18.2: Adding to iommu group 9 [ 0.345428] pci 0000:00:18.3: Adding to iommu group 9 [ 0.345451] pci 0000:00:18.4: Adding to iommu group 9 [ 0.345479] pci 0000:00:18.5: Adding to iommu group 9 [ 0.345489] pci 0000:03:00.0: Adding to iommu group 7 [ 0.345499] pci 0000:04:00.0: Adding to iommu group 7 [ 0.348265] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 [ 0.348272] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA [ 0.348279] AMD-Vi: Interrupt remapping enabled [ 0.348484] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 0.348486] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB) [ 0.348547] LVT offset 0 assigned for vector 0x400 [ 0.348595] perf: AMD IBS detected (0x000000ff) [ 0.348602] amd_uncore: 4 amd_nb counters detected [ 0.352225] workingset: timestamp_bits=37 max_order=20 bucket_order=0 [ 0.352259] zbud: loaded [ 0.352752] NET: Registered PF_ALG protocol family [ 0.352758] Key type asymmetric registered [ 0.352759] Asymmetric key parser 'x509' registered [ 0.353113] alg: self-tests disabled [ 0.353209] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251) [ 0.353263] io scheduler mq-deadline registered [ 0.353265] io scheduler kyber registered [ 0.354479] pcieport 0000:00:15.0: PME: Signaling with IRQ 25 [ 0.354644] pcieport 0000:00:15.1: PME: Signaling with IRQ 26 [ 0.354712] pcieport 0000:00:15.2: enabling device (0000 -> 0003) [ 0.354941] pcieport 0000:00:15.2: PME: Signaling with IRQ 27 [ 0.355211] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 0.355298] ACPI: button: Power Button [PWRF] [ 0.355363] ACPI: \_SB_.P000: Found 2 idle states [ 0.355495] ACPI: \_SB_.P001: Found 2 idle states [ 0.356442] thermal LNXTHERM:00: registered as thermal_zone0 [ 0.356445] ACPI: thermal: Thermal Zone [TZ00] (21 C) [ 0.356797] Non-volatile memory driver v1.3 [ 0.356864] AMD-Vi: AMD IOMMUv2 loaded and initialized [ 0.357065] ahci 0000:00:11.0: version 3.0 [ 0.357356] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode [ 0.357360] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio [ 0.358785] scsi host0: ahci [ 0.358999] scsi host1: ahci [ 0.359215] scsi host2: ahci [ 0.359424] scsi host3: ahci [ 0.359624] scsi host4: ahci [ 0.359836] scsi host5: ahci [ 0.360048] scsi host6: ahci [ 0.360243] scsi host7: ahci [ 0.360336] ata port1: DUMMY [ 0.360338] ata port2: DUMMY [ 0.360339] ata port3: DUMMY [ 0.360341] ata port4: DUMMY [ 0.360342] ata port5: DUMMY [ 0.360343] ata port6: DUMMY [ 0.360345] ata port7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19 [ 0.360347] ata port8: DUMMY [ 0.360431] ACPI: bus type drm_connector registered [ 0.360734] i8042: PNP: No PS/2 controller found. [ 0.360736] i8042: Probing ports directly. [ 0.363078] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 0.363185] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 0.363347] mousedev: PS/2 mouse device common for all mice [ 0.363420] rtc_cmos 00:01: RTC can wake from S4 [ 0.363924] rtc_cmos 00:01: registered as rtc0 [ 0.363950] rtc_cmos 00:01: setting system clock to 2023-04-17T17:24:09 UTC (1681752249) [ 0.364011] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs [ 0.364048] device-mapper: uevent: version 1.0.3 [ 0.364136] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com [ 0.364301] hid: raw HID events driver (C) Jiri Kosina [ 0.364335] usbcore: registered new interface driver usbhid [ 0.364336] usbhid: USB HID core driver [ 0.364427] Initializing XFRM netlink socket [ 0.364436] NET: Registered PF_PACKET protocol family [ 0.364438] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending. [ 0.364652] microcode: CPU1: patch_level=0x0600111f [ 0.364652] microcode: CPU0: patch_level=0x0600111f [ 0.364664] microcode: Microcode Update Driver: v2.2. [ 0.364668] IPI shorthand broadcast: enabled [ 0.364678] AVX version of gcm_enc/dec engaged. [ 0.364710] AES CTR mode by8 optimization enabled [ 0.368578] sched_clock: Marking stable (249972593, 117151517)->(369867515, -2743405) [ 0.368823] registered taskstats version 1 [ 0.369080] zswap: loaded using pool lzo/zbud [ 0.371663] ata link7: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 0.371939] ata dev7.0: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133 [ 0.371942] ata dev7.0: 125045424 sectors, multi 1: LBA48 NCQ (depth 32) [ 0.372148] ata dev7.0: configured for UDMA/133 [ 0.372283] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5 [ 0.373496] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679) [ 0.373501] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 0.373959] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB) [ 0.373982] sd 6:0:0:0: [sda] Write Protect is off [ 0.373987] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00 [ 0.374015] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 0.374059] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes [ 0.380407] kmemleak: Automatic memory scanning thread started [ 0.380633] sda: sda1 sda2 sda3 [ 0.381120] Key type encrypted registered [ 0.381248] sd 6:0:0:0: [sda] Attached SCSI disk [ 0.384294] PM: Magic number: 3:443:441 [ 0.396721] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none. [ 0.396766] VFS: Mounted root (ext4 filesystem) on device 8:3. [ 0.398650] devtmpfs: mounted [ 0.398668] After kernel_init_freeable [ 0.403170] Freeing unused kernel image (initmem) memory: 2908K [ 0.415639] Write protecting the kernel read-only data: 20480k [ 0.415922] Freeing unused kernel image (rodata/data gap) memory: 836K [ 0.453024] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 0.453030] rodata_test: all tests were successful [ 0.453031] After mark_readonly [ 0.453031] After pti_finalize [ 0.453045] rcu_end_inkernel_boot [ 0.453054] Run /sbin/init as init process [ 0.453055] with arguments: [ 0.453057] /sbin/init [ 0.453058] noisapnp [ 0.453059] with environment: [ 0.453060] HOME=/ [ 0.453060] TERM=linux [ 0.453061] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 [ 0.629518] systemd[1]: Inserted module 'autofs4' [ 0.656601] NET: Registered PF_INET6 protocol family [ 0.657495] Segment Routing with IPv6 [ 0.657523] In-situ OAM (IOAM) with IPv6 [ 0.683745] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) [ 0.683756] systemd[1]: Detected architecture x86-64. [ 0.688485] systemd[1]: Hostname set to <kodi>. [ 0.959071] systemd[1]: Queued start job for default target graphical.target. [ 0.982198] systemd[1]: Created slice system-getty.slice - Slice /system/getty. [ 0.983286] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe. [ 0.984123] systemd[1]: Created slice user.slice - User and Session Slice. [ 0.984302] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. [ 0.984417] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch. [ 0.984855] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point. [ 0.984894] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes. [ 0.984934] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes. [ 0.984972] systemd[1]: Reached target paths.target - Path Units. [ 0.985006] systemd[1]: Reached target remote-fs.target - Remote File Systems. [ 0.985038] systemd[1]: Reached target slices.target - Slice Units. [ 0.985078] systemd[1]: Reached target swap.target - Swaps. [ 0.985116] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes. [ 0.987550] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket. [ 0.987802] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket. [ 0.987975] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe. [ 0.988274] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket. [ 0.988543] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log). [ 0.988822] systemd[1]: Listening on systemd-journald.socket - Journal Socket. [ 0.989072] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket. [ 0.989946] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket. [ 0.990210] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket. [ 0.992904] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System... [ 0.995944] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System... [ 0.999142] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System... [ 1.002682] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System... [ 1.007256] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes... [ 1.014276] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs... [ 1.017399] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod... [ 1.020657] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm... [ 1.024006] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore... [ 1.027278] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse... [ 1.036066] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop... [ 1.036151] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes). [ 1.036237] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/). [ 1.036279] systemd[1]: Reached target local-fs.target - Local File Systems. [ 1.036362] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor). [ 1.041356] loop: module loaded [ 1.043749] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats... [ 1.048983] systemd[1]: Starting systemd-journald.service - Journal Service... [ 1.050740] fuse: init (API version 7.38) [ 1.052824] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed... [ 1.063924] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables... [ 1.068061] systemd[1]: Starting systemd-sysusers.service - Create System Users... [ 1.071328] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices... [ 1.093517] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System. [ 1.094134] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System. [ 1.094447] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System. [ 1.094985] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System. [ 1.095974] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes. [ 1.096888] systemd[1]: modprobe@configfs.service: Deactivated successfully. [ 1.097257] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs. [ 1.098001] systemd[1]: modprobe@dm_mod.service: Deactivated successfully. [ 1.098339] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod. [ 1.099077] systemd[1]: modprobe@drm.service: Deactivated successfully. [ 1.099446] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm. [ 1.100518] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully. [ 1.100846] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore. [ 1.101749] systemd[1]: modprobe@fuse.service: Deactivated successfully. [ 1.102064] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse. [ 1.102810] systemd[1]: modprobe@loop.service: Deactivated successfully. [ 1.103142] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop. [ 1.103830] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 138 (systemd-binfmt) [ 1.123242] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System... [ 1.140028] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System... [ 1.148223] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System... [ 1.148346] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore). [ 1.148515] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met. [ 1.159909] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables. [ 1.184529] systemd[1]: Finished systemd-sysusers.service - Create System Users. [ 1.186000] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System. [ 1.186357] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System. [ 1.186600] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System. [ 1.209823] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev... [ 1.210512] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats. [ 1.254273] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. [ 1.273469] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files... [ 1.313286] systemd[1]: Started systemd-journald.service - Journal Service. [ 1.359636] systemd-journald[139]: Received client request to flush runtime journal. [ 1.369173] tsc: Refined TSC clocksource calibration: 3900.223 MHz [ 1.369184] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns [ 1.369200] clocksource: Switched to clocksource tsc [ 1.594559] sd 6:0:0:0: Attached scsi generic sg0 type 0 [ 1.731704] acpi_cpufreq: overriding BIOS provided _PSD data [ 1.953174] random: crng init done [ 1.992654] QUIRK: Enable AMD PLL fix [ 1.992706] ehci-pci 0000:00:12.2: EHCI Host Controller [ 1.992738] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1 [ 1.992751] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 1.992760] ehci-pci 0000:00:12.2: debug port 1 [ 1.992971] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000 [ 2.005197] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00 [ 2.005468] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 2.005471] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.005474] usb usb1: Product: EHCI Host Controller [ 2.005476] usb usb1: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd [ 2.005478] usb usb1: SerialNumber: 0000:00:12.2 [ 2.006279] hub 1-0:1.0: USB hub found [ 2.006321] hub 1-0:1.0: 5 ports detected [ 2.007319] ehci-pci 0000:00:13.2: EHCI Host Controller [ 2.007345] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2 [ 2.007360] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 2.007369] ehci-pci 0000:00:13.2: debug port 1 [ 2.007523] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000 [ 2.016141] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0 [ 2.016147] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection [ 2.016788] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20 [ 2.021184] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00 [ 2.021523] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 2.021527] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.021529] usb usb2: Product: EHCI Host Controller [ 2.021531] usb usb2: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd [ 2.021533] usb usb2: SerialNumber: 0000:00:13.2 [ 2.032086] hub 2-0:1.0: USB hub found [ 2.032171] hub 2-0:1.0: 5 ports detected [ 2.033420] ehci-pci 0000:00:16.2: EHCI Host Controller [ 2.033455] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3 [ 2.033470] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 2.033479] ehci-pci 0000:00:16.2: debug port 1 [ 2.033641] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000 [ 2.049177] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00 [ 2.049454] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 2.049459] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.049461] usb usb3: Product: EHCI Host Controller [ 2.049463] usb usb3: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd [ 2.049465] usb usb3: SerialNumber: 0000:00:16.2 [ 2.050025] hub 3-0:1.0: USB hub found [ 2.050070] hub 3-0:1.0: 4 ports detected [ 2.050881] ohci-pci 0000:00:16.0: OHCI PCI host controller [ 2.050910] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 4 [ 2.050999] ohci-pci 0000:00:12.0: OHCI PCI host controller [ 2.051021] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 5 [ 2.051110] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000 [ 2.051111] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000 [ 2.051120] ohci-pci 0000:00:13.0: OHCI PCI host controller [ 2.051141] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 6 [ 2.051240] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000 [ 2.051250] ohci-pci 0000:00:14.5: OHCI PCI host controller [ 2.051268] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 7 [ 2.051356] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000 [ 2.113579] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 2.113586] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.113589] usb usb4: Product: OHCI PCI host controller [ 2.113591] usb usb4: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd [ 2.113593] usb usb4: SerialNumber: 0000:00:16.0 [ 2.114096] hub 4-0:1.0: USB hub found [ 2.114127] hub 4-0:1.0: 4 ports detected [ 2.126194] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 2.126202] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.126204] usb usb5: Product: OHCI PCI host controller [ 2.126207] usb usb5: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd [ 2.126209] usb usb5: SerialNumber: 0000:00:12.0 [ 2.127341] hub 5-0:1.0: USB hub found [ 2.127380] hub 5-0:1.0: 5 ports detected [ 2.128522] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 2.128528] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.128531] usb usb6: Product: OHCI PCI host controller [ 2.128533] usb usb6: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd [ 2.128535] usb usb6: SerialNumber: 0000:00:13.0 [ 2.129049] hub 6-0:1.0: USB hub found [ 2.129082] hub 6-0:1.0: 5 ports detected [ 2.130311] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 2.130316] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.130319] usb usb7: Product: OHCI PCI host controller [ 2.130321] usb usb7: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd [ 2.130323] usb usb7: SerialNumber: 0000:00:14.5 [ 2.130855] hub 7-0:1.0: USB hub found [ 2.130895] hub 7-0:1.0: 2 ports detected [ 2.148137] 1 [ 2.148196] 2 [ 2.148820] snd_hda_intel 0000:00:01.1: Force to non-snoop mode [ 2.148832] 3 [ 2.148833] 4 [ 2.148834] 5 [ 2.148835] 7 [ 2.148837] 8 [ 2.148837] 9 [ 2.148929] 1 [ 2.148979] 2 [ 2.149637] 3 [ 2.149641] 4 [ 2.149642] 5 [ 2.149643] 7 [ 2.149645] 8 [ 2.149646] 9 [ 2.166674] xhci_hcd 0000:03:00.0: xHCI Host Controller [ 2.166707] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 8 [ 2.206853] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1 [ 2.208133] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2 [ 2.221615] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100 [ 2.222377] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line [ 2.222384] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0) [ 2.222387] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0) [ 2.222390] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0 [ 2.222392] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0 [ 2.222393] snd_hda_codec_realtek hdaudioC1D0: inputs: [ 2.222395] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18 [ 2.222398] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19 [ 2.222399] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a [ 2.222401] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c [ 2.231811] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010 [ 2.235245] r8169 0000:04:00.0: enabling device (0000 -> 0003) [ 2.265051] xhci_hcd 0000:03:00.0: xHCI Host Controller [ 2.265076] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 9 [ 2.265092] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed [ 2.266951] usb usb8: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 2.266958] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.266961] usb usb8: Product: xHCI Host Controller [ 2.266963] usb usb8: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd [ 2.266965] usb usb8: SerialNumber: 0000:03:00.0 [ 2.267014] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3 [ 2.267354] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4 [ 2.267750] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5 [ 2.268089] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6 [ 2.268437] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7 [ 2.268783] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8 [ 2.269132] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9 [ 2.269714] hub 8-0:1.0: USB hub found [ 2.269815] hub 8-0:1.0: 2 ports detected [ 2.269956] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10 [ 2.272826] usb usb9: We don't know the algorithms for LPM for this host, disabling LPM. [ 2.273006] usb usb9: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03 [ 2.273010] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.273012] usb usb9: Product: xHCI Host Controller [ 2.273014] usb usb9: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd [ 2.273016] usb usb9: SerialNumber: 0000:03:00.0 [ 2.275521] hub 9-0:1.0: USB hub found [ 2.275649] hub 9-0:1.0: 2 ports detected [ 2.295318] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 32 [ 2.295327] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko] [ 2.311845] r8169 0000:04:00.0 enp4s0: renamed from eth0 [ 2.421403] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2 [ 2.421415] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2) [ 2.421903] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC) [ 2.465163] usb 5-1: new low-speed USB device number 2 using ohci-pci [ 2.472735] [drm] radeon kernel modesetting enabled. [ 2.474200] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00). [ 2.474271] ATOM BIOS: 113 [ 2.474372] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used) [ 2.474376] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF [ 2.474384] [drm] Detected VRAM RAM=512M, BAR=256M [ 2.474385] [drm] RAM width 64bits DDR [ 2.474589] [drm] radeon: 512M of VRAM memory ready [ 2.474595] [drm] radeon: 1024M of GTT memory ready. [ 2.474642] [drm] Loading ARUBA Microcode [ 2.481608] [drm] Internal thermal controller without fan control [ 2.481974] [drm] radeon: dpm initialized [ 2.486664] [drm] Found VCE firmware/feedback version 50.0.1 / 17! [ 2.486711] [drm] GART: num cpu pages 262144, num gpu pages 262144 [ 2.492402] r8169 0000:04:00.0 enp4s0: Link is Down [ 2.526396] [drm] GART: Restore entries: num cpu pages 262144, num gpu pages 262144 [ 2.529843] [drm] GART: Done restoring entries [ 2.529847] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000). [ 2.530087] radeon 0000:00:01.0: WB enabled [ 2.530090] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00 [ 2.530468] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18 [ 2.550526] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18 [ 2.550531] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c [ 2.550533] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04 [ 2.550535] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08 [ 2.550536] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c [ 2.550538] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10 [ 2.550809] radeon 0000:00:01.0: radeon: MSI limited to 32-bit [ 2.551003] radeon 0000:00:01.0: radeon: using MSI. [ 2.551071] [drm] radeon: irq initialized. [ 2.569579] [drm] ring test on 0 succeeded in 3 usecs [ 2.569589] [drm] ring test on 3 succeeded in 4 usecs [ 2.569596] [drm] ring test on 4 succeeded in 4 usecs [ 2.583595] [drm] ring test on 5 succeeded in 2 usecs [ 2.585593] [drm] UVD initialized successfully. [ 2.670213] usb 5-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01 [ 2.670218] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 2.670220] usb 5-1: Product: Dell QuietKey Keyboard [ 2.670222] usb 5-1: Manufacturer: DELL [ 2.678027] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb5/5-1/5-1:1.0/0003:413C:2106.0001/input/input11 [ 2.694994] [drm] ring test on 6 succeeded in 18 usecs [ 2.695006] [drm] ring test on 7 succeeded in 3 usecs [ 2.695007] [drm] VCE initialized successfully. [ 2.695174] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon]) [ 2.695346] [drm] ib test on ring 0 succeeded in 0 usecs [ 2.695400] [drm] ib test on ring 3 succeeded in 0 usecs [ 2.695449] [drm] ib test on ring 4 succeeded in 0 usecs [ 2.713280] [drm] ib test on ring 5 succeeded [ 2.729270] [drm] ib test on ring 6 succeeded in 1 usecs [ 2.737964] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0 [ 2.745270] [drm] ib test on ring 7 succeeded in 1 usecs [ 2.748783] [drm] Radeon Display Connectors [ 2.748789] [drm] Connector 0: [ 2.748790] [drm] DP-1 [ 2.748791] [drm] HPD1 [ 2.748792] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c [ 2.748795] [drm] Encoders: [ 2.748796] [drm] DFP1: INTERNAL_UNIPHY2 [ 2.748797] [drm] Connector 1: [ 2.748798] [drm] VGA-1 [ 2.748799] [drm] HPD2 [ 2.748800] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c [ 2.748802] [drm] Encoders: [ 2.748803] [drm] CRT1: INTERNAL_UNIPHY2 [ 2.748804] [drm] CRT1: NUTMEG [ 2.748805] [drm] Connector 2: [ 2.748806] [drm] HDMI-A-1 [ 2.748807] [drm] HPD3 [ 2.748808] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c [ 2.748811] [drm] Encoders: [ 2.748811] [drm] DFP2: INTERNAL_UNIPHY [ 3.019999] [drm] fb mappable at 0xE03E9000 [ 3.020006] [drm] vram apper at 0xE0000000 [ 3.020009] [drm] size 5242880 [ 3.020011] [drm] fb depth is 24 [ 3.020012] [drm] pitch is 5120 [ 3.020508] fbcon: radeondrmfb (fb0) is primary device [ 3.201239] usb 5-2: new low-speed USB device number 3 using ohci-pci [ 3.215988] Console: switching to colour frame buffer device 160x64 [ 3.217767] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device [ 3.237462] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0 [ 3.397485] usb 5-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40 [ 3.397493] usb 5-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 3.397495] usb 5-2: Product: Optical USB Mouse [ 3.397497] usb 5-2: Manufacturer: Logitech [ 3.406006] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb5/5-2/5-2:1.0/0003:046D:C016.0002/input/input12 [ 3.406790] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0 [ 5.100538] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx [ 5.100555] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready [ 5.945432] [drm] amdgpu kernel modesetting enabled. [ 10.066361] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=258 'systemd'
Paul! On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote: > Am 17.04.23 um 16:48 schrieb Thomas Gleixner: > >> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote: >>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner: >>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD >>> Graphics (family: 0x15, model: 0x13, stepping: 0x1) >>> […] >>> [ 0.259329] smp: Bringing up secondary CPUs ... >>> [ 0.259527] x86: Booting SMP configuration: >>> [ 0.259528] .... node #0, CPUs: #1 >>> [ 0.261007] After schedule_preempt_disabled >>> [ 10.260990] CPU1 failed to report alive state >> >> Weird. CPU1 fails to come up and report that it has reached the >> synchronization point. >> >> Does it work when you add cpuhp.parallel=off on the kernel command line? > > Yes, the ten seconds delay is gone with `cpuhp.parallel=off`. > > There was a patch set in the past, that worked on that device. I think > up to v4 it did *not* work at all and hung [1]. I need some days to > collect the results again. Can you please apply the patch below on top of the pile remove the command line option again? Thanks, tglx --- kernel/cpu.c | 1 + 1 file changed, 1 insertion(+) --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1777,6 +1777,7 @@ static void __init cpuhp_bringup_mask(co */ WARN_ON(cpuhp_invoke_callback_range(false, cpu, st, CPUHP_OFFLINE)); } + msleep(20); } }
On Tue, Apr 18 2023 at 08:58, Thomas Gleixner wrote: > On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote: >> Am 17.04.23 um 16:48 schrieb Thomas Gleixner: >> >>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote: >>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner: >>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD >>>> Graphics (family: 0x15, model: 0x13, stepping: 0x1) >>>> […] >>>> [ 0.259329] smp: Bringing up secondary CPUs ... >>>> [ 0.259527] x86: Booting SMP configuration: >>>> [ 0.259528] .... node #0, CPUs: #1 >>>> [ 0.261007] After schedule_preempt_disabled >>>> [ 10.260990] CPU1 failed to report alive state >>> >>> Weird. CPU1 fails to come up and report that it has reached the >>> synchronization point. >>> >>> Does it work when you add cpuhp.parallel=off on the kernel command line? >> >> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`. >> >> There was a patch set in the past, that worked on that device. I think >> up to v4 it did *not* work at all and hung [1]. I need some days to >> collect the results again. > > Can you please apply the patch below on top of the pile remove the > command line option again? Bah. That patch does not make any sense at all. Not enough coffee. Can you please provide the output of cpuid? Thanks, tglx
Dear Thomas, Am 18.04.23 um 10:40 schrieb Thomas Gleixner: > On Tue, Apr 18 2023 at 08:58, Thomas Gleixner wrote: >> On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote: >>> Am 17.04.23 um 16:48 schrieb Thomas Gleixner: >>> >>>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote: >>>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner: >>>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1) >>>>> […] >>>>> [ 0.259329] smp: Bringing up secondary CPUs ... >>>>> [ 0.259527] x86: Booting SMP configuration: >>>>> [ 0.259528] .... node #0, CPUs: #1 >>>>> [ 0.261007] After schedule_preempt_disabled >>>>> [ 10.260990] CPU1 failed to report alive state >>>> >>>> Weird. CPU1 fails to come up and report that it has reached the >>>> synchronization point. >>>> >>>> Does it work when you add cpuhp.parallel=off on the kernel command line? >>> >>> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`. >>> >>> There was a patch set in the past, that worked on that device. I think >>> up to v4 it did *not* work at all and hung [1]. I need some days to >>> collect the results again. >> >> Can you please apply the patch below on top of the pile remove the >> command line option again? > > Bah. That patch does not make any sense at all. Not enough coffee. > > Can you please provide the output of cpuid? Of course. Here the top, and the whole output is attached. ``` CPU 0: vendor_id = "AuthenticAMD" version information (1/eax): processor type = primary processor (0) family = 0xf (15) model = 0x3 (3) stepping id = 0x1 (1) extended family = 0x6 (6) extended model = 0x1 (1) (family synth) = 0x15 (21) (model synth) = 0x13 (19) (simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm […] ``` Kind regards, Paul CPU 0: vendor_id = "AuthenticAMD" version information (1/eax): processor type = primary processor (0) family = 0xf (15) model = 0x3 (3) stepping id = 0x1 (1) extended family = 0x6 (6) extended model = 0x1 (1) (family synth) = 0x15 (21) (model synth) = 0x13 (19) (simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm miscellaneous (1/ebx): process local APIC physical ID = 0x0 (0) maximum IDs for CPUs in pkg = 0x2 (2) CLFLUSH line size = 0x8 (8) brand index = 0x0 (0) brand id = 0x00 (0): unknown feature information (1/edx): x87 FPU on chip = true VME: virtual-8086 mode enhancement = true DE: debugging extensions = true PSE: page size extensions = true TSC: time stamp counter = true RDMSR and WRMSR support = true PAE: physical address extensions = true MCE: machine check exception = true CMPXCHG8B inst. = true APIC on chip = true SYSENTER and SYSEXIT = true MTRR: memory type range registers = true PTE global bit = true MCA: machine check architecture = true CMOV: conditional move/compare instr = true PAT: page attribute table = true PSE-36: page size extension = true PSN: processor serial number = false CLFLUSH instruction = true DS: debug store = false ACPI: thermal monitor and clock ctrl = false MMX Technology = true FXSAVE/FXRSTOR = true SSE extensions = true SSE2 extensions = true SS: self snoop = false hyper-threading / multi-core supported = true TM: therm. monitor = false IA64 = false PBE: pending break event = false feature information (1/ecx): PNI/SSE3: Prescott New Instructions = true PCLMULDQ instruction = true DTES64: 64-bit debug store = false MONITOR/MWAIT = true CPL-qualified debug store = false VMX: virtual machine extensions = false SMX: safer mode extensions = false Enhanced Intel SpeedStep Technology = false TM2: thermal monitor 2 = false SSSE3 extensions = true context ID: adaptive or shared L1 data = false SDBG: IA32_DEBUG_INTERFACE = false FMA instruction = true CMPXCHG16B instruction = true xTPR disable = false PDCM: perfmon and debug = false PCID: process context identifiers = false DCA: direct cache access = false SSE4.1 extensions = true SSE4.2 extensions = true x2APIC: extended xAPIC support = false MOVBE instruction = false POPCNT instruction = true time stamp counter deadline = false AES instruction = true XSAVE/XSTOR states = true OS-enabled XSAVE/XSTOR = true AVX: advanced vector extensions = true F16C half-precision convert instruction = true RDRAND instruction = false hypervisor guest status = false cache and TLB information (2): processor serial number = 0061-0F31-0000-0000-0000-0000 deterministic cache parameters (4): --- cache 0 --- cache type = no more caches (0) MONITOR/MWAIT (5): smallest monitor-line size (bytes) = 0x40 (64) largest monitor-line size (bytes) = 0x40 (64) enum of Monitor-MWAIT exts supported = true supports intrs as break-event for MWAIT = true number of C0 sub C-states using MWAIT = 0x0 (0) number of C1 sub C-states using MWAIT = 0x0 (0) number of C2 sub C-states using MWAIT = 0x0 (0) number of C3 sub C-states using MWAIT = 0x0 (0) number of C4 sub C-states using MWAIT = 0x0 (0) number of C5 sub C-states using MWAIT = 0x0 (0) number of C6 sub C-states using MWAIT = 0x0 (0) number of C7 sub C-states using MWAIT = 0x0 (0) Thermal and Power Management Features (6): digital thermometer = false Intel Turbo Boost Technology = false ARAT always running APIC timer = false PLN power limit notification = false ECMD extended clock modulation duty = false PTM package thermal management = false HWP base registers = false HWP notification = false HWP activity window = false HWP energy performance preference = false HWP package level request = false HDC base registers = false Intel Turbo Boost Max Technology 3.0 = false HWP capabilities = false HWP PECI override = false flexible HWP = false IA32_HWP_REQUEST MSR fast access mode = false HW_FEEDBACK MSRs supported = false ignoring idle logical processor HWP req = false Thread Director = false IA32_HW_FEEDBACK_THREAD_CONFIG bit 25 = false digital thermometer thresholds = 0x0 (0) hardware coordination feedback = true ACNT2 available = false performance-energy bias capability = false number of enh hardware feedback classes = 0x0 (0) performance capability reporting = false energy efficiency capability reporting = false size of feedback struct (4KB pages) = 0x1 (1) index of CPU's row in feedback struct = 0x0 (0) extended feature flags (7): FSGSBASE instructions = false IA32_TSC_ADJUST MSR supported = false SGX: Software Guard Extensions supported = false BMI1 instructions = true HLE hardware lock elision = false AVX2: advanced vector extensions 2 = false FDP_EXCPTN_ONLY = false SMEP supervisor mode exec protection = false BMI2 instructions = false enhanced REP MOVSB/STOSB = false INVPCID instruction = false RTM: restricted transactional memory = false RDT-CMT/PQoS cache monitoring = false deprecated FPU CS/DS = false MPX: intel memory protection extensions = false RDT-CAT/PQE cache allocation = false AVX512F: AVX-512 foundation instructions = false AVX512DQ: double & quadword instructions = false RDSEED instruction = false ADX instructions = false SMAP: supervisor mode access prevention = false AVX512IFMA: integer fused multiply add = false PCOMMIT instruction = false CLFLUSHOPT instruction = false CLWB instruction = false Intel processor trace = false AVX512PF: prefetch instructions = false AVX512ER: exponent & reciprocal instrs = false AVX512CD: conflict detection instrs = false SHA instructions = false AVX512BW: byte & word instructions = false AVX512VL: vector length = false PREFETCHWT1 = false AVX512VBMI: vector byte manipulation = false UMIP: user-mode instruction prevention = false PKU protection keys for user-mode = false OSPKE CR4.PKE and RDPKRU/WRPKRU = false WAITPKG instructions = false AVX512_VBMI2: byte VPCOMPRESS, VPEXPAND = false CET_SS: CET shadow stack = false GFNI: Galois Field New Instructions = false VAES instructions = false VPCLMULQDQ instruction = false AVX512_VNNI: neural network instructions = false AVX512_BITALG: bit count/shiffle = false TME: Total Memory Encryption = false AVX512: VPOPCNTDQ instruction = false LA57: 57-bit addrs & 5-level paging = false BNDLDX/BNDSTX MAWAU value in 64-bit mode = 0x0 (0) RDPID: read processor ID supported = false KL: key locker = false bus lock detection = false CLDEMOTE supports cache line demote = false MOVDIRI instruction = false MOVDIR64B instruction = false ENQCMD instruction = false SGX_LC: SGX launch config supported = false PKS: supervisor protection keys = false SGX-KEYS: SGX attestation services = false AVX512_4VNNIW: neural network instrs = false AVX512_4FMAPS: multiply acc single prec = false fast short REP MOV = false UINTR: user interrupts = false AVX512_VP2INTERSECT: intersect mask regs = false IA32_MCU_OPT_CTRL SRBDS mitigation MSR = false VERW MD_CLEAR microcode support = false RTM transaction always aborts = false IA32_TSX_FORCE_ABORT MSR = false SERIALIZE instruction = false hybrid part = false TSXLDTRK: TSX suspend load addr tracking = false PCONFIG instruction = false LBR: architectural last branch records = false CET_IBT: CET indirect branch tracking = false AMX-BF16: tile bfloat16 support = false AVX512_FP16: fp16 support = false AMX-TILE: tile architecture support = false AMX-INT8: tile 8-bit integer support = false IBRS/IBPB: indirect branch restrictions = false STIBP: 1 thr indirect branch predictor = false L1D_FLUSH: IA32_FLUSH_CMD MSR = false IA32_ARCH_CAPABILITIES MSR = false IA32_CORE_CAPABILITIES MSR = false SSBD: speculative store bypass disable = false Direct Cache Access Parameters (9): PLATFORM_DCA_CAP MSR bits = 0 Architecture Performance Monitoring Features (0xa): version ID = 0x0 (0) number of counters per logical processor = 0x0 (0) bit width of counter = 0x0 (0) length of EBX bit vector = 0x0 (0) core cycle event = not available instruction retired event = not available reference cycles event = not available last-level cache ref event = not available last-level cache miss event = not available branch inst retired event = not available branch mispred retired event = not available top-down slots event = not available fixed counter 0 supported = false fixed counter 1 supported = false fixed counter 2 supported = false fixed counter 3 supported = false fixed counter 4 supported = false fixed counter 5 supported = false fixed counter 6 supported = false fixed counter 7 supported = false fixed counter 8 supported = false fixed counter 9 supported = false fixed counter 10 supported = false fixed counter 11 supported = false fixed counter 12 supported = false fixed counter 13 supported = false fixed counter 14 supported = false fixed counter 15 supported = false fixed counter 16 supported = false fixed counter 17 supported = false fixed counter 18 supported = false fixed counter 19 supported = false fixed counter 20 supported = false fixed counter 21 supported = false fixed counter 22 supported = false fixed counter 23 supported = false fixed counter 24 supported = false fixed counter 25 supported = false fixed counter 26 supported = false fixed counter 27 supported = false fixed counter 28 supported = false fixed counter 29 supported = false fixed counter 30 supported = false fixed counter 31 supported = false number of contiguous fixed counters = 0x0 (0) bit width of fixed counters = 0x0 (0) anythread deprecation = false x2APIC features / processor topology (0xb): extended APIC ID = 0 --- level 0 --- level number = 0x0 (0) level type = invalid (0) bit width of level = 0x0 (0) number of logical processors at level = 0x0 (0) XSAVE features (0xd/0): XCR0 valid bit field mask = 0x4000000000000007 x87 state = true SSE state = true AVX state = true MPX BNDREGS = false MPX BNDCSR = false AVX-512 opmask = false AVX-512 ZMM_Hi256 = false AVX-512 Hi16_ZMM = false PKRU state = false XTILECFG state = false XTILEDATA state = false bytes required by fields in XCR0 = 0x00000340 (832) bytes required by XSAVE/XRSTOR area = 0x000003c0 (960) XSAVEOPT instruction = false XSAVEC instruction = false XGETBV instruction = false XSAVES/XRSTORS instructions = false XFD: extended feature disable supported = false SAVE area size in bytes = 0x00000000 (0) IA32_XSS valid bit field mask = 0x0000000000000000 PT state = false PASID state = false CET_U user state = false CET_S supervisor state = false HDC state = false UINTR state = false LBR state = false HWP state = false AVX/YMM features (0xd/2): AVX/YMM save state byte size = 0x00000100 (256) AVX/YMM save state byte offset = 0x00000240 (576) supported in IA32_XSS or XCR0 = XCR0 (user state) 64-byte alignment in compacted XSAVE = false XFD faulting supported = false LWP features (0xd/0x3e): LWP save state byte size = 0x00000080 (128) LWP save state byte offset = 0x00000340 (832) supported in IA32_XSS or XCR0 = XCR0 (user state) 64-byte alignment in compacted XSAVE = false XFD faulting supported = false extended processor signature (0x80000001/eax): family/generation = 0xf (15) model = 0x3 (3) stepping id = 0x1 (1) extended family = 0x6 (6) extended model = 0x1 (1) (family synth) = 0x15 (21) (model synth) = 0x13 (19) (simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm extended feature flags (0x80000001/edx): x87 FPU on chip = true virtual-8086 mode enhancement = true debugging extensions = true page size extensions = true time stamp counter = true RDMSR and WRMSR support = true physical address extensions = true machine check exception = true CMPXCHG8B inst. = true APIC on chip = true SYSCALL and SYSRET instructions = true memory type range registers = true global paging extension = true machine check architecture = true conditional move/compare instruction = true page attribute table = true page size extension = true multiprocessing capable = false no-execute page protection = true AMD multimedia instruction extensions = true MMX Technology = true FXSAVE/FXRSTOR = true SSE extensions = true 1-GB large page support = true RDTSCP = true long mode (AA-64) = true 3DNow! instruction extensions = false 3DNow! instructions = false extended brand id (0x80000001/ebx): raw = 0x20000000 (536870912) BrandId = 0x0 (0) PkgType = FM2 (PGA) (2) AMD feature flags (0x80000001/ecx): LAHF/SAHF supported in 64-bit mode = true CMP Legacy = true SVM: secure virtual machine = true extended APIC space = true AltMovCr8 = true LZCNT advanced bit manipulation = true SSE4A support = true misaligned SSE mode = true 3DNow! PREFETCH/PREFETCHW instructions = true OS visible workaround = true instruction based sampling = true XOP support = true SKINIT/STGI support = true watchdog timer support = true lightweight profiling support = true 4-operand FMA instruction = true TCE: translation cache extension = true NodeId MSR C001100C = true TBM support = true topology extensions = true core performance counter extensions = true NB/DF performance counter extensions = true data breakpoint extension = false performance time-stamp counter support = false LLC performance counter extensions = false MWAITX/MONITORX supported = false Address mask extension support = false brand = "AMD A6-6400K APU with Radeon(tm) HD Graphics " L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax): instruction # entries = 0x18 (24) instruction associativity = 0xff (255) data # entries = 0x40 (64) data associativity = 0xff (255) L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx): instruction # entries = 0x30 (48) instruction associativity = 0xff (255) data # entries = 0x40 (64) data associativity = 0xff (255) L1 data cache information (0x80000005/ecx): line size (bytes) = 0x40 (64) lines per tag = 0x1 (1) associativity = 0x4 (4) size (KB) = 0x10 (16) L1 instruction cache information (0x80000005/edx): line size (bytes) = 0x40 (64) lines per tag = 0x1 (1) associativity = 0x2 (2) size (KB) = 0x40 (64) L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax): instruction # entries = 0x400 (1024) instruction associativity = 8 to 15-way (6) data # entries = 0x400 (1024) data associativity = 8 to 15-way (6) L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx): instruction # entries = 0x200 (512) instruction associativity = 4 to 5-way (4) data # entries = 0x400 (1024) data associativity = 8 to 15-way (6) L2 unified cache information (0x80000006/ecx): line size (bytes) = 0x40 (64) lines per tag = 0x1 (1) associativity = 16 to 31-way (8) size (KB) = 0x400 (1024) L3 cache information (0x80000006/edx): line size (bytes) = 0x0 (0) lines per tag = 0x0 (0) associativity = L2 off (0) size (in 512KB units) = 0x0 (0) RAS Capability (0x80000007/ebx): MCA overflow recovery support = false SUCCOR support = false HWA: hardware assert support = false scalable MCA support = false Advanced Power Management Features (0x80000007/ecx): CmpUnitPwrSampleTimeRatio = 0x0 (0) Advanced Power Management Features (0x80000007/edx): TS: temperature sensing diode = true FID: frequency ID control = false VID: voltage ID control = false TTP: thermal trip = true TM: thermal monitor = true STC: software thermal control = false 100 MHz multiplier control = true hardware P-State control = true TscInvariant = true CPB: core performance boost = true read-only effective frequency interface = true processor feedback interface = false APM power reporting = false connected standby = false RAPL: running average power limit = false Physical Address and Linear Address Size (0x80000008/eax): maximum physical address bits = 0x30 (48) maximum linear (virtual) address bits = 0x30 (48) maximum guest physical address bits = 0x0 (0) Extended Feature Extensions ID (0x80000008/ebx): CLZERO instruction = false instructions retired count support = false always save/restore error pointers = false INVLPGB instruction = false RDPRU instruction = false memory bandwidth enforcement = false MCOMMIT instruction = false WBNOINVD instruction = false IBPB: indirect branch prediction barrier = true interruptible WBINVD, WBNOINVD = false IBRS: indirect branch restr speculation = false STIBP: 1 thr indirect branch predictor = false CPU prefers: IBRS always on = false CPU prefers: STIBP always on = false IBRS preferred over software solution = false IBRS provides same mode protection = false EFER[LMSLE] not supported = false INVLPGB supports TLB flush guest nested = false ppin processor id number supported = false SSBD: speculative store bypass disable = false virtualized SSBD = false SSBD fixed in hardware = false CPPC: collaborative processor perf ctrl = false PSFD: predictive store forward disable = false not vulnerable to branch type confusion = false branch sampling feature support = false (vuln to branch type confusion synth) = true Size Identifiers (0x80000008/ecx): number of CPU cores = 0x2 (2) ApicIdCoreIdSize = 0x4 (4) performance time-stamp counter size = 40 bits (0) Feature Extended Size (0x80000008/edx): max page count for INVLPGB instruction = 0x0 (0) RDPRU instruction max input support = 0x0 (0) SVM Secure Virtual Machine (0x8000000a/eax): SvmRev: SVM revision = 0x1 (1) SVM Secure Virtual Machine (0x8000000a/edx): nested paging = true LBR virtualization = true SVM lock = true NRIP save = true MSR based TSC rate control = true VMCB clean bits support = true flush by ASID = true decode assists = true SSSE3/SSE5 opcode set disable = false pause intercept filter = true pause filter threshold = true AVIC: AMD virtual interrupt controller = false virtualized VMLOAD/VMSAVE = false virtualized global interrupt flag (GIF) = false GMET: guest mode execute trap = false X2AVIC: virtualized X2APIC = false supervisor shadow stack = false guest Spec_ctl support = false ROGPT: read-only guest page table = false host MCE override = false INVLPGB/TLBSYNC hyperv interc enable = false VNMI: NMI virtualization = false IBS virtualization = false guest SVME addr check = false NASID: number of address space identifiers = 0x10000 (65536): L1 TLB information: 1G pages (0x80000019/eax): instruction # entries = 0x18 (24) instruction associativity = full (15) data # entries = 0x40 (64) data associativity = full (15) L2 TLB information: 1G pages (0x80000019/ebx): instruction # entries = 0x400 (1024) instruction associativity = 8 to 15-way (6) data # entries = 0x400 (1024) data associativity = 8 to 15-way (6) Performance Optimization Identifiers (0x8000001a/eax): 128-bit SSE executed full-width = true MOVU* better than MOVL*/MOVH* = true 256-bit SSE executed full-width = false Instruction Based Sampling Identifiers (0x8000001b/eax): IBS feature flags valid = true IBS fetch sampling = true IBS execution sampling = true read write of op counter = true op counting mode = true branch target address reporting = true IbsOpCurCnt and IbsOpMaxCnt extend 7 = true invalid RIP indication support = true fused branch micro-op indication support = false IBS fetch control extended MSR support = false IBS op data 4 MSR support = false IBS L3 miss filtering support = false Lightweight Profiling Capabilities: Availability (0x8000001c/eax): lightweight profiling = false LWPVAL instruction = false instruction retired event = false branch retired event = false DC miss event = false core clocks not halted event = false core reference clocks not halted event = false continuous mode sampling = false tsc in event record = false interrupt on threshold overflow = false Lightweight Profiling Capabilities: Supported (0x8000001c/edx): lightweight profiling = true LWPVAL instruction = true instruction retired event = true branch retired event = true DC miss event = false core clocks not halted event = false core reference clocks not halted event = false continuous mode sampling = false tsc in event record = false interrupt on threshold overflow = true Lightweight Profiling Capabilities (0x8000001c/ebx): LWPCB byte size = 0x13 (19) event record byte size = 0x20 (32) maximum EventId = 0x3 (3) EventInterval1 field offset = 0x80 (128) Lightweight Profiling Capabilities (0x8000001c/ecx): latency counter bit size = 0x0 (0) data cache miss address valid = false amount cache latency is rounded = 0x0 (0) LWP implementation version = 0x1 (1) event ring buffer size in records = 0x1 (1) branch prediction filtering = false IP filtering = false cache level filtering = false cache latency filteing = false Cache Properties (0x8000001d): --- cache 0 --- type = data (1) level = 0x1 (1) self-initializing = true fully associative = false extra cores sharing this cache = 0x0 (0) line size in bytes = 0x40 (64) physical line partitions = 0x1 (1) number of ways = 0x4 (4) number of sets = 64 write-back invalidate = false cache inclusive of lower levels = false (synth size) = 16384 (16 KB) --- cache 1 --- type = instruction (2) level = 0x1 (1) self-initializing = true fully associative = false extra cores sharing this cache = 0x1 (1) line size in bytes = 0x40 (64) physical line partitions = 0x1 (1) number of ways = 0x2 (2) number of sets = 512 write-back invalidate = false cache inclusive of lower levels = false (synth size) = 65536 (64 KB) --- cache 2 --- type = unified (3) level = 0x2 (2) self-initializing = true fully associative = false extra cores sharing this cache = 0x1 (1) line size in bytes = 0x40 (64) physical line partitions = 0x1 (1) number of ways = 0x10 (16) number of sets = 1024 write-back invalidate = true cache inclusive of lower levels = false (synth size) = 1048576 (1024 KB) extended APIC ID = 16 Compute Unit Identifiers (0x8000001e/ebx): compute unit ID = 0x0 (0) cores per compute unit = 0x2 (2) Node Identifiers (0x8000001e/ecx): node ID = 0x0 (0) nodes per processor = 0x1 (1) (instruction supported synth): CMPXCHG8B = true conditional move/compare = true PREFETCH/PREFETCHW = true (multi-processing synth) = multi-core (c=2) (multi-processing method) = AMD (APIC widths synth): CORE_width=1 SMT_width=0 (APIC synth): PKG_ID=0 CORE_ID=0 SMT_ID=0 (uarch synth) = AMD Piledriver, 32nm (synth) = AMD A-Series (Richland RL-A1) [Piledriver], 32nm CPU 1: vendor_id = "AuthenticAMD" version information (1/eax): processor type = primary processor (0) family = 0xf (15) model = 0x3 (3) stepping id = 0x1 (1) extended family = 0x6 (6) extended model = 0x1 (1) (family synth) = 0x15 (21) (model synth) = 0x13 (19) (simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm miscellaneous (1/ebx): process local APIC physical ID = 0x1 (1) maximum IDs for CPUs in pkg = 0x2 (2) CLFLUSH line size = 0x8 (8) brand index = 0x0 (0) brand id = 0x00 (0): unknown feature information (1/edx): x87 FPU on chip = true VME: virtual-8086 mode enhancement = true DE: debugging extensions = true PSE: page size extensions = true TSC: time stamp counter = true RDMSR and WRMSR support = true PAE: physical address extensions = true MCE: machine check exception = true CMPXCHG8B inst. = true APIC on chip = true SYSENTER and SYSEXIT = true MTRR: memory type range registers = true PTE global bit = true MCA: machine check architecture = true CMOV: conditional move/compare instr = true PAT: page attribute table = true PSE-36: page size extension = true PSN: processor serial number = false CLFLUSH instruction = true DS: debug store = false ACPI: thermal monitor and clock ctrl = false MMX Technology = true FXSAVE/FXRSTOR = true SSE extensions = true SSE2 extensions = true SS: self snoop = false hyper-threading / multi-core supported = true TM: therm. monitor = false IA64 = false PBE: pending break event = false feature information (1/ecx): PNI/SSE3: Prescott New Instructions = true PCLMULDQ instruction = true DTES64: 64-bit debug store = false MONITOR/MWAIT = true CPL-qualified debug store = false VMX: virtual machine extensions = false SMX: safer mode extensions = false Enhanced Intel SpeedStep Technology = false TM2: thermal monitor 2 = false SSSE3 extensions = true context ID: adaptive or shared L1 data = false SDBG: IA32_DEBUG_INTERFACE = false FMA instruction = true CMPXCHG16B instruction = true xTPR disable = false PDCM: perfmon and debug = false PCID: process context identifiers = false DCA: direct cache access = false SSE4.1 extensions = true SSE4.2 extensions = true x2APIC: extended xAPIC support = false MOVBE instruction = false POPCNT instruction = true time stamp counter deadline = false AES instruction = true XSAVE/XSTOR states = true OS-enabled XSAVE/XSTOR = true AVX: advanced vector extensions = true F16C half-precision convert instruction = true RDRAND instruction = false hypervisor guest status = false cache and TLB information (2): processor serial number = 0061-0F31-0000-0000-0000-0000 deterministic cache parameters (4): --- cache 0 --- cache type = no more caches (0) MONITOR/MWAIT (5): smallest monitor-line size (bytes) = 0x40 (64) largest monitor-line size (bytes) = 0x40 (64) enum of Monitor-MWAIT exts supported = true supports intrs as break-event for MWAIT = true number of C0 sub C-states using MWAIT = 0x0 (0) number of C1 sub C-states using MWAIT = 0x0 (0) number of C2 sub C-states using MWAIT = 0x0 (0) number of C3 sub C-states using MWAIT = 0x0 (0) number of C4 sub C-states using MWAIT = 0x0 (0) number of C5 sub C-states using MWAIT = 0x0 (0) number of C6 sub C-states using MWAIT = 0x0 (0) number of C7 sub C-states using MWAIT = 0x0 (0) Thermal and Power Management Features (6): digital thermometer = false Intel Turbo Boost Technology = false ARAT always running APIC timer = false PLN power limit notification = false ECMD extended clock modulation duty = false PTM package thermal management = false HWP base registers = false HWP notification = false HWP activity window = false HWP energy performance preference = false HWP package level request = false HDC base registers = false Intel Turbo Boost Max Technology 3.0 = false HWP capabilities = false HWP PECI override = false flexible HWP = false IA32_HWP_REQUEST MSR fast access mode = false HW_FEEDBACK MSRs supported = false ignoring idle logical processor HWP req = false Thread Director = false IA32_HW_FEEDBACK_THREAD_CONFIG bit 25 = false digital thermometer thresholds = 0x0 (0) hardware coordination feedback = true ACNT2 available = false performance-energy bias capability = false number of enh hardware feedback classes = 0x0 (0) performance capability reporting = false energy efficiency capability reporting = false size of feedback struct (4KB pages) = 0x1 (1) index of CPU's row in feedback struct = 0x0 (0) extended feature flags (7): FSGSBASE instructions = false IA32_TSC_ADJUST MSR supported = false SGX: Software Guard Extensions supported = false BMI1 instructions = true HLE hardware lock elision = false AVX2: advanced vector extensions 2 = false FDP_EXCPTN_ONLY = false SMEP supervisor mode exec protection = false BMI2 instructions = false enhanced REP MOVSB/STOSB = false INVPCID instruction = false RTM: restricted transactional memory = false RDT-CMT/PQoS cache monitoring = false deprecated FPU CS/DS = false MPX: intel memory protection extensions = false RDT-CAT/PQE cache allocation = false AVX512F: AVX-512 foundation instructions = false AVX512DQ: double & quadword instructions = false RDSEED instruction = false ADX instructions = false SMAP: supervisor mode access prevention = false AVX512IFMA: integer fused multiply add = false PCOMMIT instruction = false CLFLUSHOPT instruction = false CLWB instruction = false Intel processor trace = false AVX512PF: prefetch instructions = false AVX512ER: exponent & reciprocal instrs = false AVX512CD: conflict detection instrs = false SHA instructions = false AVX512BW: byte & word instructions = false AVX512VL: vector length = false PREFETCHWT1 = false AVX512VBMI: vector byte manipulation = false UMIP: user-mode instruction prevention = false PKU protection keys for user-mode = false OSPKE CR4.PKE and RDPKRU/WRPKRU = false WAITPKG instructions = false AVX512_VBMI2: byte VPCOMPRESS, VPEXPAND = false CET_SS: CET shadow stack = false GFNI: Galois Field New Instructions = false VAES instructions = false VPCLMULQDQ instruction = false AVX512_VNNI: neural network instructions = false AVX512_BITALG: bit count/shiffle = false TME: Total Memory Encryption = false AVX512: VPOPCNTDQ instruction = false LA57: 57-bit addrs & 5-level paging = false BNDLDX/BNDSTX MAWAU value in 64-bit mode = 0x0 (0) RDPID: read processor ID supported = false KL: key locker = false bus lock detection = false CLDEMOTE supports cache line demote = false MOVDIRI instruction = false MOVDIR64B instruction = false ENQCMD instruction = false SGX_LC: SGX launch config supported = false PKS: supervisor protection keys = false SGX-KEYS: SGX attestation services = false AVX512_4VNNIW: neural network instrs = false AVX512_4FMAPS: multiply acc single prec = false fast short REP MOV = false UINTR: user interrupts = false AVX512_VP2INTERSECT: intersect mask regs = false IA32_MCU_OPT_CTRL SRBDS mitigation MSR = false VERW MD_CLEAR microcode support = false RTM transaction always aborts = false IA32_TSX_FORCE_ABORT MSR = false SERIALIZE instruction = false hybrid part = false TSXLDTRK: TSX suspend load addr tracking = false PCONFIG instruction = false LBR: architectural last branch records = false CET_IBT: CET indirect branch tracking = false AMX-BF16: tile bfloat16 support = false AVX512_FP16: fp16 support = false AMX-TILE: tile architecture support = false AMX-INT8: tile 8-bit integer support = false IBRS/IBPB: indirect branch restrictions = false STIBP: 1 thr indirect branch predictor = false L1D_FLUSH: IA32_FLUSH_CMD MSR = false IA32_ARCH_CAPABILITIES MSR = false IA32_CORE_CAPABILITIES MSR = false SSBD: speculative store bypass disable = false Direct Cache Access Parameters (9): PLATFORM_DCA_CAP MSR bits = 0 Architecture Performance Monitoring Features (0xa): version ID = 0x0 (0) number of counters per logical processor = 0x0 (0) bit width of counter = 0x0 (0) length of EBX bit vector = 0x0 (0) core cycle event = not available instruction retired event = not available reference cycles event = not available last-level cache ref event = not available last-level cache miss event = not available branch inst retired event = not available branch mispred retired event = not available top-down slots event = not available fixed counter 0 supported = false fixed counter 1 supported = false fixed counter 2 supported = false fixed counter 3 supported = false fixed counter 4 supported = false fixed counter 5 supported = false fixed counter 6 supported = false fixed counter 7 supported = false fixed counter 8 supported = false fixed counter 9 supported = false fixed counter 10 supported = false fixed counter 11 supported = false fixed counter 12 supported = false fixed counter 13 supported = false fixed counter 14 supported = false fixed counter 15 supported = false fixed counter 16 supported = false fixed counter 17 supported = false fixed counter 18 supported = false fixed counter 19 supported = false fixed counter 20 supported = false fixed counter 21 supported = false fixed counter 22 supported = false fixed counter 23 supported = false fixed counter 24 supported = false fixed counter 25 supported = false fixed counter 26 supported = false fixed counter 27 supported = false fixed counter 28 supported = false fixed counter 29 supported = false fixed counter 30 supported = false fixed counter 31 supported = false number of contiguous fixed counters = 0x0 (0) bit width of fixed counters = 0x0 (0) anythread deprecation = false x2APIC features / processor topology (0xb): extended APIC ID = 0 --- level 0 --- level number = 0x0 (0) level type = invalid (0) bit width of level = 0x0 (0) number of logical processors at level = 0x0 (0) XSAVE features (0xd/0): XCR0 valid bit field mask = 0x4000000000000007 x87 state = true SSE state = true AVX state = true MPX BNDREGS = false MPX BNDCSR = false AVX-512 opmask = false AVX-512 ZMM_Hi256 = false AVX-512 Hi16_ZMM = false PKRU state = false XTILECFG state = false XTILEDATA state = false bytes required by fields in XCR0 = 0x00000340 (832) bytes required by XSAVE/XRSTOR area = 0x000003c0 (960) XSAVEOPT instruction = false XSAVEC instruction = false XGETBV instruction = false XSAVES/XRSTORS instructions = false XFD: extended feature disable supported = false SAVE area size in bytes = 0x00000000 (0) IA32_XSS valid bit field mask = 0x0000000000000000 PT state = false PASID state = false CET_U user state = false CET_S supervisor state = false HDC state = false UINTR state = false LBR state = false HWP state = false AVX/YMM features (0xd/2): AVX/YMM save state byte size = 0x00000100 (256) AVX/YMM save state byte offset = 0x00000240 (576) supported in IA32_XSS or XCR0 = XCR0 (user state) 64-byte alignment in compacted XSAVE = false XFD faulting supported = false LWP features (0xd/0x3e): LWP save state byte size = 0x00000080 (128) LWP save state byte offset = 0x00000340 (832) supported in IA32_XSS or XCR0 = XCR0 (user state) 64-byte alignment in compacted XSAVE = false XFD faulting supported = false extended processor signature (0x80000001/eax): family/generation = 0xf (15) model = 0x3 (3) stepping id = 0x1 (1) extended family = 0x6 (6) extended model = 0x1 (1) (family synth) = 0x15 (21) (model synth) = 0x13 (19) (simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm extended feature flags (0x80000001/edx): x87 FPU on chip = true virtual-8086 mode enhancement = true debugging extensions = true page size extensions = true time stamp counter = true RDMSR and WRMSR support = true physical address extensions = true machine check exception = true CMPXCHG8B inst. = true APIC on chip = true SYSCALL and SYSRET instructions = true memory type range registers = true global paging extension = true machine check architecture = true conditional move/compare instruction = true page attribute table = true page size extension = true multiprocessing capable = false no-execute page protection = true AMD multimedia instruction extensions = true MMX Technology = true FXSAVE/FXRSTOR = true SSE extensions = true 1-GB large page support = true RDTSCP = true long mode (AA-64) = true 3DNow! instruction extensions = false 3DNow! instructions = false extended brand id (0x80000001/ebx): raw = 0x20000000 (536870912) BrandId = 0x0 (0) PkgType = FM2 (PGA) (2) AMD feature flags (0x80000001/ecx): LAHF/SAHF supported in 64-bit mode = true CMP Legacy = true SVM: secure virtual machine = true extended APIC space = true AltMovCr8 = true LZCNT advanced bit manipulation = true SSE4A support = true misaligned SSE mode = true 3DNow! PREFETCH/PREFETCHW instructions = true OS visible workaround = true instruction based sampling = true XOP support = true SKINIT/STGI support = true watchdog timer support = true lightweight profiling support = true 4-operand FMA instruction = true TCE: translation cache extension = true NodeId MSR C001100C = true TBM support = true topology extensions = true core performance counter extensions = true NB/DF performance counter extensions = true data breakpoint extension = false performance time-stamp counter support = false LLC performance counter extensions = false MWAITX/MONITORX supported = false Address mask extension support = false brand = "AMD A6-6400K APU with Radeon(tm) HD Graphics " L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax): instruction # entries = 0x18 (24) instruction associativity = 0xff (255) data # entries = 0x40 (64) data associativity = 0xff (255) L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx): instruction # entries = 0x30 (48) instruction associativity = 0xff (255) data # entries = 0x40 (64) data associativity = 0xff (255) L1 data cache information (0x80000005/ecx): line size (bytes) = 0x40 (64) lines per tag = 0x1 (1) associativity = 0x4 (4) size (KB) = 0x10 (16) L1 instruction cache information (0x80000005/edx): line size (bytes) = 0x40 (64) lines per tag = 0x1 (1) associativity = 0x2 (2) size (KB) = 0x40 (64) L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax): instruction # entries = 0x400 (1024) instruction associativity = 8 to 15-way (6) data # entries = 0x400 (1024) data associativity = 8 to 15-way (6) L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx): instruction # entries = 0x200 (512) instruction associativity = 4 to 5-way (4) data # entries = 0x400 (1024) data associativity = 8 to 15-way (6) L2 unified cache information (0x80000006/ecx): line size (bytes) = 0x40 (64) lines per tag = 0x1 (1) associativity = 16 to 31-way (8) size (KB) = 0x400 (1024) L3 cache information (0x80000006/edx): line size (bytes) = 0x0 (0) lines per tag = 0x0 (0) associativity = L2 off (0) size (in 512KB units) = 0x0 (0) RAS Capability (0x80000007/ebx): MCA overflow recovery support = false SUCCOR support = false HWA: hardware assert support = false scalable MCA support = false Advanced Power Management Features (0x80000007/ecx): CmpUnitPwrSampleTimeRatio = 0x0 (0) Advanced Power Management Features (0x80000007/edx): TS: temperature sensing diode = true FID: frequency ID control = false VID: voltage ID control = false TTP: thermal trip = true TM: thermal monitor = true STC: software thermal control = false 100 MHz multiplier control = true hardware P-State control = true TscInvariant = true CPB: core performance boost = true read-only effective frequency interface = true processor feedback interface = false APM power reporting = false connected standby = false RAPL: running average power limit = false Physical Address and Linear Address Size (0x80000008/eax): maximum physical address bits = 0x30 (48) maximum linear (virtual) address bits = 0x30 (48) maximum guest physical address bits = 0x0 (0) Extended Feature Extensions ID (0x80000008/ebx): CLZERO instruction = false instructions retired count support = false always save/restore error pointers = false INVLPGB instruction = false RDPRU instruction = false memory bandwidth enforcement = false MCOMMIT instruction = false WBNOINVD instruction = false IBPB: indirect branch prediction barrier = true interruptible WBINVD, WBNOINVD = false IBRS: indirect branch restr speculation = false STIBP: 1 thr indirect branch predictor = false CPU prefers: IBRS always on = false CPU prefers: STIBP always on = false IBRS preferred over software solution = false IBRS provides same mode protection = false EFER[LMSLE] not supported = false INVLPGB supports TLB flush guest nested = false ppin processor id number supported = false SSBD: speculative store bypass disable = false virtualized SSBD = false SSBD fixed in hardware = false CPPC: collaborative processor perf ctrl = false PSFD: predictive store forward disable = false not vulnerable to branch type confusion = false branch sampling feature support = false (vuln to branch type confusion synth) = true Size Identifiers (0x80000008/ecx): number of CPU cores = 0x2 (2) ApicIdCoreIdSize = 0x4 (4) performance time-stamp counter size = 40 bits (0) Feature Extended Size (0x80000008/edx): max page count for INVLPGB instruction = 0x0 (0) RDPRU instruction max input support = 0x0 (0) SVM Secure Virtual Machine (0x8000000a/eax): SvmRev: SVM revision = 0x1 (1) SVM Secure Virtual Machine (0x8000000a/edx): nested paging = true LBR virtualization = true SVM lock = true NRIP save = true MSR based TSC rate control = true VMCB clean bits support = true flush by ASID = true decode assists = true SSSE3/SSE5 opcode set disable = false pause intercept filter = true pause filter threshold = true AVIC: AMD virtual interrupt controller = false virtualized VMLOAD/VMSAVE = false virtualized global interrupt flag (GIF) = false GMET: guest mode execute trap = false X2AVIC: virtualized X2APIC = false supervisor shadow stack = false guest Spec_ctl support = false ROGPT: read-only guest page table = false host MCE override = false INVLPGB/TLBSYNC hyperv interc enable = false VNMI: NMI virtualization = false IBS virtualization = false guest SVME addr check = false NASID: number of address space identifiers = 0x10000 (65536): L1 TLB information: 1G pages (0x80000019/eax): instruction # entries = 0x18 (24) instruction associativity = full (15) data # entries = 0x40 (64) data associativity = full (15) L2 TLB information: 1G pages (0x80000019/ebx): instruction # entries = 0x400 (1024) instruction associativity = 8 to 15-way (6) data # entries = 0x400 (1024) data associativity = 8 to 15-way (6) Performance Optimization Identifiers (0x8000001a/eax): 128-bit SSE executed full-width = true MOVU* better than MOVL*/MOVH* = true 256-bit SSE executed full-width = false Instruction Based Sampling Identifiers (0x8000001b/eax): IBS feature flags valid = true IBS fetch sampling = true IBS execution sampling = true read write of op counter = true op counting mode = true branch target address reporting = true IbsOpCurCnt and IbsOpMaxCnt extend 7 = true invalid RIP indication support = true fused branch micro-op indication support = false IBS fetch control extended MSR support = false IBS op data 4 MSR support = false IBS L3 miss filtering support = false Lightweight Profiling Capabilities: Availability (0x8000001c/eax): lightweight profiling = false LWPVAL instruction = false instruction retired event = false branch retired event = false DC miss event = false core clocks not halted event = false core reference clocks not halted event = false continuous mode sampling = false tsc in event record = false interrupt on threshold overflow = false Lightweight Profiling Capabilities: Supported (0x8000001c/edx): lightweight profiling = true LWPVAL instruction = true instruction retired event = true branch retired event = true DC miss event = false core clocks not halted event = false core reference clocks not halted event = false continuous mode sampling = false tsc in event record = false interrupt on threshold overflow = true Lightweight Profiling Capabilities (0x8000001c/ebx): LWPCB byte size = 0x13 (19) event record byte size = 0x20 (32) maximum EventId = 0x3 (3) EventInterval1 field offset = 0x80 (128) Lightweight Profiling Capabilities (0x8000001c/ecx): latency counter bit size = 0x0 (0) data cache miss address valid = false amount cache latency is rounded = 0x0 (0) LWP implementation version = 0x1 (1) event ring buffer size in records = 0x1 (1) branch prediction filtering = false IP filtering = false cache level filtering = false cache latency filteing = false Cache Properties (0x8000001d): --- cache 0 --- type = data (1) level = 0x1 (1) self-initializing = true fully associative = false extra cores sharing this cache = 0x0 (0) line size in bytes = 0x40 (64) physical line partitions = 0x1 (1) number of ways = 0x4 (4) number of sets = 64 write-back invalidate = false cache inclusive of lower levels = false (synth size) = 16384 (16 KB) --- cache 1 --- type = instruction (2) level = 0x1 (1) self-initializing = true fully associative = false extra cores sharing this cache = 0x1 (1) line size in bytes = 0x40 (64) physical line partitions = 0x1 (1) number of ways = 0x2 (2) number of sets = 512 write-back invalidate = false cache inclusive of lower levels = false (synth size) = 65536 (64 KB) --- cache 2 --- type = unified (3) level = 0x2 (2) self-initializing = true fully associative = false extra cores sharing this cache = 0x1 (1) line size in bytes = 0x40 (64) physical line partitions = 0x1 (1) number of ways = 0x10 (16) number of sets = 1024 write-back invalidate = true cache inclusive of lower levels = false (synth size) = 1048576 (1024 KB) extended APIC ID = 17 Compute Unit Identifiers (0x8000001e/ebx): compute unit ID = 0x0 (0) cores per compute unit = 0x2 (2) Node Identifiers (0x8000001e/ecx): node ID = 0x0 (0) nodes per processor = 0x1 (1) (instruction supported synth): CMPXCHG8B = true conditional move/compare = true PREFETCH/PREFETCHW = true (multi-processing synth) = multi-core (c=2) (multi-processing method) = AMD (APIC widths synth): CORE_width=1 SMT_width=0 (APIC synth): PKG_ID=0 CORE_ID=1 SMT_ID=0 (uarch synth) = AMD Piledriver, 32nm (synth) = AMD A-Series (Richland RL-A1) [Piledriver], 32nm
Paul! On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote: > Am 18.04.23 um 10:40 schrieb Thomas Gleixner: >> Can you please provide the output of cpuid? > > Of course. Here the top, and the whole output is attached. Thanks for the data. Can you please apply the debug patch below and provide the dmesg output? Just the line which is added by the patch is enough. You can boot with cpuhp.parallel=off so you don't have wait for 10 seconds. Thanks, tglx --- --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -814,6 +814,7 @@ static int wakeup_secondary_cpu_via_init unsigned long send_status = 0, accept_status = 0; int maxlvt, num_starts, j; + pr_info("Kicking AP alive: %d\n", phys_apicid); preempt_disable(); maxlvt = lapic_get_maxlvt();
On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote: > On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote: >> Am 18.04.23 um 10:40 schrieb Thomas Gleixner: >>> Can you please provide the output of cpuid? >> >> Of course. Here the top, and the whole output is attached. > > Thanks for the data. Can you please apply the debug patch below and > provide the dmesg output? Just the line which is added by the patch is > enough. You can boot with cpuhp.parallel=off so you don't have wait for > 10 seconds. Borislav found some a machine which also refuses to boot. It turns of the debug patch was spot on: [ 0.462724] .... node #0, CPUs: #1 [ 0.462731] smpboot: Kicking AP alive: 17 [ 0.465723] #2 [ 0.465732] smpboot: Kicking AP alive: 18 [ 0.467641] #3 [ 0.467641] smpboot: Kicking AP alive: 19 So the kernel gets APICID 17, 18, 19 from ACPI but CPUID leaf 0x1 ebx[31:24], which is the initial APICID has: CPU1 0x01 CPU2 0x02 CPU3 0x03 Which means the APICID to Linux CPU number lookup based on CPUID 0x01 fails for all of them and stops them dead in the low level startup code. IOW, the BIOS assignes random numbers to the AP APICs for whatever raisins, which leaves the parallel startup low level code up a creek without a paddle, except for actually reading the APICID back from the APIC. *SHUDDER* I'm leaning towards disabling the CPUID lead 0x01 based discovery and be done with it. Thanks, tglx
On Wed, 2023-04-19 at 14:38 +0200, Thomas Gleixner wrote: > > I'm leaning towards disabling the CPUID lead 0x01 based discovery and be > done with it. Makes sense. The large machines where users really want the parallel startup all ought to have X2APIC and hence CPUID 0x0b.
On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote: > On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote: > IOW, the BIOS assignes random numbers to the AP APICs for whatever > raisins, which leaves the parallel startup low level code up a creek > without a paddle, except for actually reading the APICID back from the > APIC. *SHUDDER* So Andrew just pointed out on IRC that this might be related to the ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the ID space, but that system is definitely post 3-wire APIC :) Thanks, tglx
On 19/04/2023 2:43 pm, Thomas Gleixner wrote: > On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote: >> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote: >> IOW, the BIOS assignes random numbers to the AP APICs for whatever >> raisins, which leaves the parallel startup low level code up a creek >> without a paddle, except for actually reading the APICID back from the >> APIC. *SHUDDER* > So Andrew just pointed out on IRC that this might be related to the > ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the > ID space, but that system is definitely post 3-wire APIC :) Doesn't mean the BIOS code was updated adequately following that. What I'm confused by is why this system boots in the first place. I can only think that's is a system which only has 4-bit APIC IDs, and happens to function when bit 4 gets truncated off the top of the SIPI destination... ~Andrew
On 19/04/2023 2:50 pm, Andrew Cooper wrote: > On 19/04/2023 2:43 pm, Thomas Gleixner wrote: >> On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote: >>> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote: >>> IOW, the BIOS assignes random numbers to the AP APICs for whatever >>> raisins, which leaves the parallel startup low level code up a creek >>> without a paddle, except for actually reading the APICID back from the >>> APIC. *SHUDDER* >> So Andrew just pointed out on IRC that this might be related to the >> ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the >> ID space, but that system is definitely post 3-wire APIC :) > Doesn't mean the BIOS code was updated adequately following that. > > What I'm confused by is why this system boots in the first place. I can > only think that's is a system which only has 4-bit APIC IDs, and happens > to function when bit 4 gets truncated off the top of the SIPI destination... https://www.amd.com/system/files/TechDocs/42300_15h_Mod_10h-1Fh_BKDG.pdf This system does still require the IO-APICs to be at 0, and the LAPICs to start at some offset, which is clearly 16 in this case. Also, this system has configurable 4-bit or 8-bit wide APIC IDs, and I can't tell which mode is active just from the manual. But, it does mean that the BIOS has genuinely modified the APIC IDs of the logic processors. This does highlight an error in reasoning with the parallel bringup code. For xAPIC, the APIC_ID register is writeable (at least, model specifically), and CPUID is only the value it would have had at reset. So the AP bringup logic can't actually use CPUID reliably. This was changed in x2APIC, which made the x2APIC_ID immutable. I don't see an option other than the AP bringup code query for xAPIC vs x2APIC mode, and either looking at the real APIC_ID register, or falling back to CPUID. ~Andrew
On Wed, Apr 19 2023 at 17:21, Andrew Cooper wrote: > On 19/04/2023 2:50 pm, Andrew Cooper wrote: >> What I'm confused by is why this system boots in the first place. I can >> only think that's is a system which only has 4-bit APIC IDs, and happens >> to function when bit 4 gets truncated off the top of the SIPI destination... > > https://www.amd.com/system/files/TechDocs/42300_15h_Mod_10h-1Fh_BKDG.pdf > > This system does still require the IO-APICs to be at 0, and the LAPICs > to start at some offset, which is clearly 16 in this case. Also, this > system has configurable 4-bit or 8-bit wide APIC IDs, and I can't tell > which mode is active just from the manual. That document contradicts itself: "The ApicId of core j must be enumerated/assigned as: ApicId[core=j] = (OFFSET_IDX) * MNC + j Where OFFSET_IDX is an integer offset (0 to N) used to shift up the core ApicId values to allow room for IOAPIC devices. It is recommended that BIOS use the following APIC ID assignments for the broadest operating system sup- port. Given N = MNC and M = Number_Of_IOAPICs: • Assign the core ApicId’s first from 0 to N-1, and the IOAPIC IDs from N to N+(M-1)." Oh well. If the rest of these docs is of the same quality then it's not a surprise that BIOSes are trainwrecks. > But, it does mean that the BIOS has genuinely modified the APIC IDs of > the logic processors. This does highlight an error in reasoning with > the parallel bringup code. Yes. > For xAPIC, the APIC_ID register is writeable (at least, model > specifically), and CPUID is only the value it would have had at reset. > So the AP bringup logic can't actually use CPUID reliably. > > This was changed in x2APIC, which made the x2APIC_ID immutable. > > I don't see an option other than the AP bringup code query for xAPIC vs > x2APIC mode, and either looking at the real APIC_ID register, or falling > back to CPUID. I'm pondering to simply deny parallel mode if x2APIC is not there. Thanks, tglx
On 20/04/2023 9:32 am, Thomas Gleixner wrote: > On Wed, Apr 19 2023 at 17:21, Andrew Cooper wrote: >> On 19/04/2023 2:50 pm, Andrew Cooper wrote: >> For xAPIC, the APIC_ID register is writeable (at least, model >> specifically), and CPUID is only the value it would have had at reset. >> So the AP bringup logic can't actually use CPUID reliably. >> >> This was changed in x2APIC, which made the x2APIC_ID immutable. >> >> I don't see an option other than the AP bringup code query for xAPIC vs >> x2APIC mode, and either looking at the real APIC_ID register, or falling >> back to CPUID. > I'm pondering to simply deny parallel mode if x2APIC is not there. I'm not sure if that will help much. Just because x2APIC is there doesn't mean it's in use. There are several generations of Intel system which have x2APIC but also use the opt-out bit in ACPI tables. There are some machines which have mismatched APIC-ness settings in the BIOS->OS handover. There's very little you can do on the BSP alone to know for certain that the APs come out of wait-for-SIPI already in x2APIC mode. One way is the ÆPIC Leak "locked into x2APIC mode" giant security bodge. If the system really does have a CPU with an APIC ID above 0xfe, then chances are good that the APs come out consistently... ~Andrew
On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote: > On 20/04/2023 9:32 am, Thomas Gleixner wrote: >> I'm pondering to simply deny parallel mode if x2APIC is not there. > > I'm not sure if that will help much. Spoilsport. > Just because x2APIC is there doesn't mean it's in use. There are > several generations of Intel system which have x2APIC but also use the > opt-out bit in ACPI tables. There are some machines which have > mismatched APIC-ness settings in the BIOS->OS handover. > > There's very little you can do on the BSP alone to know for certain that > the APs come out of wait-for-SIPI already in x2APIC mode. Yeah. Reading the APIC that early is going to be entertaining too :) > One way is the ÆPIC Leak "locked into x2APIC mode" giant security > bodge. Bah. > If the system really does have a CPU with an APIC ID above 0xfe, then > chances are good that the APs come out consistently... Anything else would be really magic :)
On Thu, Apr 20, 2023, Thomas Gleixner wrote: > On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote: > > On 20/04/2023 9:32 am, Thomas Gleixner wrote: > > > On Wed, Apr 19, 2023, Andrew Cooper wrote: > > > > This was changed in x2APIC, which made the x2APIC_ID immutable. > > >> I'm pondering to simply deny parallel mode if x2APIC is not there. > > > > I'm not sure if that will help much. > > Spoilsport. LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM: A value previously written by software to the 8-bit APIC_ID register (MMIO offset 30h) is converted by hardware into the appropriate format and reflected into the 32-bit x2APIC_ID register (MSR 802h). FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above behavior described in the APM holds true on at least Rome and Milan. My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only was introduced when x2APIC came along, but I didn't test farther back than Haswell.
On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote: > On Thu, Apr 20, 2023, Thomas Gleixner wrote: >> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote: >> > On 20/04/2023 9:32 am, Thomas Gleixner wrote: >> > > On Wed, Apr 19, 2023, Andrew Cooper wrote: >> > > > This was changed in x2APIC, which made the x2APIC_ID immutable. >> >> >> I'm pondering to simply deny parallel mode if x2APIC is not there. >> > >> > I'm not sure if that will help much. >> >> Spoilsport. > > LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The > ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID > while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM: > > A value previously written by software to the 8-bit APIC_ID register (MMIO offset > 30h) is converted by hardware into the appropriate format and reflected into the > 32-bit x2APIC_ID register (MSR 802h). > > FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively > read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above > behavior described in the APM holds true on at least Rome and Milan. > > My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only > was introduced when x2APIC came along, but I didn't test farther back than Haswell. I'm not so worried about modern hardware. The horrorshow is the old muck as demonstrated and of course there is virt :) Something like the completely untested below should just work whatever APIC ID the BIOS decided to dice. That might just work on SEV too without that GHCB muck, but what do I know. Thanks, tglx --- --- a/arch/x86/include/asm/apicdef.h +++ b/arch/x86/include/asm/apicdef.h @@ -138,7 +138,8 @@ #define APIC_EILVT_MASKED (1 << 16) #define APIC_BASE (fix_to_virt(FIX_APIC_BASE)) -#define APIC_BASE_MSR 0x800 +#define APIC_BASE_MSR 0x800 +#define APIC_X2APIC_ID_MSR 0x802 #define XAPIC_ENABLE (1UL << 11) #define X2APIC_ENABLE (1UL << 10) @@ -162,6 +163,7 @@ #define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK) #define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT) +#ifndef __ASSEMBLY__ /* * the local APIC register structure, memory mapped. Not terribly well * tested, but we might eventually use this one in the future - the @@ -435,4 +437,5 @@ enum apic_delivery_modes { APIC_DELIVERY_MODE_EXTINT = 7, }; +#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_APICDEF_H */ --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -195,14 +195,13 @@ extern void nmi_selftest(void); #endif extern unsigned int smpboot_control; +extern unsigned long apic_mmio_base; #endif /* !__ASSEMBLY__ */ /* Control bits for startup_64 */ -#define STARTUP_APICID_CPUID_1F 0x80000000 -#define STARTUP_APICID_CPUID_0B 0x40000000 -#define STARTUP_APICID_CPUID_01 0x20000000 -#define STARTUP_APICID_SEV_ES 0x10000000 +#define STARTUP_READ_APICID 0x80000000 +#define STARTUP_APICID_SEV_ES 0x40000000 /* Top 8 bits are reserved for control */ #define STARTUP_PARALLEL_MASK 0xFF000000 --- a/arch/x86/kernel/apic/apic.c +++ b/arch/x86/kernel/apic/apic.c @@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init = */ static bool virt_ext_dest_id __ro_after_init; +unsigned long apic_mmio_base __ro_after_init; + /* * Map cpu index to physical APIC ID */ @@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig if (!x2apic_mode) { set_fixmap_nocache(FIX_APIC_BASE, address); + apic_mmio_base = APIC_BASE; apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n", APIC_BASE, address); } --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -24,8 +24,10 @@ #include "../entry/calling.h" #include <asm/export.h> #include <asm/nospec-branch.h> +#include <asm/apicdef.h> #include <asm/fixmap.h> #include <asm/smp.h> + #include <asm/sev-common.h> /* @@ -237,37 +239,24 @@ SYM_INNER_LABEL(secondary_startup_64_no_ #ifdef CONFIG_SMP /* - * For parallel boot, the APIC ID is retrieved from CPUID, and then - * used to look up the CPU number. For booting a single CPU, the - * CPU number is encoded in smpboot_control. + * For parallel boot, the APIC ID is either retrieved the APIC or + * from CPUID, and then used to look up the CPU number. + * For booting a single CPU, the CPU number is encoded in + * smpboot_control. * - * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f) - * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b) - * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01) - * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR) + * Bit 31 STARTUP_APICID_READ (Read APICID from APIC) + * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR) * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set */ movl smpboot_control(%rip), %ecx + testl $STARTUP_READ_APICID, %ecx #ifdef CONFIG_AMD_MEM_ENCRYPT testl $STARTUP_APICID_SEV_ES, %ecx jnz .Luse_sev_cpuid_0b #endif - testl $STARTUP_APICID_CPUID_1F, %ecx - jnz .Luse_cpuid_1f - testl $STARTUP_APICID_CPUID_0B, %ecx - jnz .Luse_cpuid_0b - testl $STARTUP_APICID_CPUID_01, %ecx - jnz .Luse_cpuid_01 andl $(~STARTUP_PARALLEL_MASK), %ecx jmp .Lsetup_cpu -.Luse_cpuid_01: - mov $0x01, %eax - cpuid - mov %ebx, %edx - shr $24, %edx - jmp .Lsetup_AP - #ifdef CONFIG_AMD_MEM_ENCRYPT .Luse_sev_cpuid_0b: /* Set the GHCB MSR to request CPUID 0x0B_EDX */ @@ -292,24 +281,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_ jmp .Lsetup_AP #endif -.Luse_cpuid_0b: - mov $0x0B, %eax - xorl %ecx, %ecx - cpuid - jmp .Lsetup_AP +.Lread_apicid: + mov $MSR_IA32_APICBASE, %ecx + rdmsr + testl $X2APIC_ENABLE, %eax + jnz read_apicid_msr + + /* Read the APIC ID from the fix-mapped MMIO space. */ + movq apic_mmio_base(%rip), %rcx + addq $APIC_ID, %rcx + movl (%rcx), %eax + shr $24, %eax + jnz .Lread_apicid -.Luse_cpuid_1f: - mov $0x1f, %eax - xorl %ecx, %ecx - cpuid +.Lread_apicid_msr: + mov $APIC_X2APIC_ID_MSR, %ecx + rdmsr .Lsetup_AP: - /* EDX contains the APIC ID of the current CPU */ + /* EAX contains the APIC ID of the current CPU */ xorq %rcx, %rcx leaq cpuid_to_apicid(%rip), %rbx .Lfind_cpunr: - cmpl (%rbx,%rcx,4), %edx + cmpl (%rbx,%rcx,4), %eax jz .Lsetup_cpu inc %ecx #ifdef CONFIG_FORCE_NR_CPUS --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri return false; } - /* Encrypted guests require special CPUID handling. */ + /* Encrypted guests require special handling. */ if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) { switch (cc_get_vendor()) { case CC_VENDOR_AMD: ctrl = STARTUP_APICID_SEV_ES; if (topology_extended_leaf == 0x0b) - goto setup; + break; fallthrough; default: pr_info("Parallel CPU startup disabled due to guest state encryption\n"); return false; } + } else { + ctrl = STARTUP_READ_APICID; } - switch (topology_extended_leaf) { - case 0x0b: - ctrl = STARTUP_APICID_CPUID_0B; - break; - case 0x1f: - ctrl = STARTUP_APICID_CPUID_1F; - break; - case 0x00: - /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */ - if (!x2apic_mode) { - ctrl = STARTUP_APICID_CPUID_01; - break; - } - fallthrough; - default: - pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n", - topology_extended_leaf); - return false; - } - -setup: pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl); smpboot_control = ctrl; return true;
Dear Thomas, Am 20.04.23 um 17:57 schrieb Thomas Gleixner: > On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote: >> On Thu, Apr 20, 2023, Thomas Gleixner wrote: >>> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote: >>>> On 20/04/2023 9:32 am, Thomas Gleixner wrote: >>>>> On Wed, Apr 19, 2023, Andrew Cooper wrote: >>>>>> This was changed in x2APIC, which made the x2APIC_ID immutable. >>> >>>>> I'm pondering to simply deny parallel mode if x2APIC is not there. >>>> >>>> I'm not sure if that will help much. >>> >>> Spoilsport. >> >> LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The >> ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID >> while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM: >> >> A value previously written by software to the 8-bit APIC_ID register (MMIO offset >> 30h) is converted by hardware into the appropriate format and reflected into the >> 32-bit x2APIC_ID register (MSR 802h). >> >> FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively >> read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above >> behavior described in the APM holds true on at least Rome and Milan. >> >> My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only >> was introduced when x2APIC came along, but I didn't test farther back than Haswell. > > I'm not so worried about modern hardware. The horrorshow is the old muck > as demonstrated and of course there is virt :) > > Something like the completely untested below should just work whatever > APIC ID the BIOS decided to dice. > > That might just work on SEV too without that GHCB muck, but what do I > know. > > Thanks, > > tglx > --- > --- a/arch/x86/include/asm/apicdef.h > +++ b/arch/x86/include/asm/apicdef.h > @@ -138,7 +138,8 @@ > #define APIC_EILVT_MASKED (1 << 16) > > #define APIC_BASE (fix_to_virt(FIX_APIC_BASE)) > -#define APIC_BASE_MSR 0x800 > +#define APIC_BASE_MSR 0x800 > +#define APIC_X2APIC_ID_MSR 0x802 > #define XAPIC_ENABLE (1UL << 11) > #define X2APIC_ENABLE (1UL << 10) > > @@ -162,6 +163,7 @@ > #define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK) > #define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT) > > +#ifndef __ASSEMBLY__ > /* > * the local APIC register structure, memory mapped. Not terribly well > * tested, but we might eventually use this one in the future - the > @@ -435,4 +437,5 @@ enum apic_delivery_modes { > APIC_DELIVERY_MODE_EXTINT = 7, > }; > > +#endif /* !__ASSEMBLY__ */ > #endif /* _ASM_X86_APICDEF_H */ > --- a/arch/x86/include/asm/smp.h > +++ b/arch/x86/include/asm/smp.h > @@ -195,14 +195,13 @@ extern void nmi_selftest(void); > #endif > > extern unsigned int smpboot_control; > +extern unsigned long apic_mmio_base; > > #endif /* !__ASSEMBLY__ */ > > /* Control bits for startup_64 */ > -#define STARTUP_APICID_CPUID_1F 0x80000000 > -#define STARTUP_APICID_CPUID_0B 0x40000000 > -#define STARTUP_APICID_CPUID_01 0x20000000 > -#define STARTUP_APICID_SEV_ES 0x10000000 > +#define STARTUP_READ_APICID 0x80000000 > +#define STARTUP_APICID_SEV_ES 0x40000000 > > /* Top 8 bits are reserved for control */ > #define STARTUP_PARALLEL_MASK 0xFF000000 > --- a/arch/x86/kernel/apic/apic.c > +++ b/arch/x86/kernel/apic/apic.c > @@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init = > */ > static bool virt_ext_dest_id __ro_after_init; > > +unsigned long apic_mmio_base __ro_after_init; > + > /* > * Map cpu index to physical APIC ID > */ > @@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig > > if (!x2apic_mode) { > set_fixmap_nocache(FIX_APIC_BASE, address); > + apic_mmio_base = APIC_BASE; > apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n", > APIC_BASE, address); > } > --- a/arch/x86/kernel/head_64.S > +++ b/arch/x86/kernel/head_64.S > @@ -24,8 +24,10 @@ > #include "../entry/calling.h" > #include <asm/export.h> > #include <asm/nospec-branch.h> > +#include <asm/apicdef.h> > #include <asm/fixmap.h> > #include <asm/smp.h> > + > #include <asm/sev-common.h> > > /* > @@ -237,37 +239,24 @@ SYM_INNER_LABEL(secondary_startup_64_no_ > > #ifdef CONFIG_SMP > /* > - * For parallel boot, the APIC ID is retrieved from CPUID, and then > - * used to look up the CPU number. For booting a single CPU, the > - * CPU number is encoded in smpboot_control. > + * For parallel boot, the APIC ID is either retrieved the APIC or > + * from CPUID, and then used to look up the CPU number. > + * For booting a single CPU, the CPU number is encoded in > + * smpboot_control. > * > - * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f) > - * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b) > - * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01) > - * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR) > + * Bit 31 STARTUP_APICID_READ (Read APICID from APIC) > + * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR) > * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set > */ > movl smpboot_control(%rip), %ecx > + testl $STARTUP_READ_APICID, %ecx > #ifdef CONFIG_AMD_MEM_ENCRYPT > testl $STARTUP_APICID_SEV_ES, %ecx > jnz .Luse_sev_cpuid_0b > #endif > - testl $STARTUP_APICID_CPUID_1F, %ecx > - jnz .Luse_cpuid_1f > - testl $STARTUP_APICID_CPUID_0B, %ecx > - jnz .Luse_cpuid_0b > - testl $STARTUP_APICID_CPUID_01, %ecx > - jnz .Luse_cpuid_01 > andl $(~STARTUP_PARALLEL_MASK), %ecx > jmp .Lsetup_cpu > > -.Luse_cpuid_01: > - mov $0x01, %eax > - cpuid > - mov %ebx, %edx > - shr $24, %edx > - jmp .Lsetup_AP > - > #ifdef CONFIG_AMD_MEM_ENCRYPT > .Luse_sev_cpuid_0b: > /* Set the GHCB MSR to request CPUID 0x0B_EDX */ > @@ -292,24 +281,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_ > jmp .Lsetup_AP > #endif > > -.Luse_cpuid_0b: > - mov $0x0B, %eax > - xorl %ecx, %ecx > - cpuid > - jmp .Lsetup_AP > +.Lread_apicid: > + mov $MSR_IA32_APICBASE, %ecx > + rdmsr > + testl $X2APIC_ENABLE, %eax > + jnz read_apicid_msr > + > + /* Read the APIC ID from the fix-mapped MMIO space. */ > + movq apic_mmio_base(%rip), %rcx > + addq $APIC_ID, %rcx > + movl (%rcx), %eax > + shr $24, %eax > + jnz .Lread_apicid > > -.Luse_cpuid_1f: > - mov $0x1f, %eax > - xorl %ecx, %ecx > - cpuid > +.Lread_apicid_msr: > + mov $APIC_X2APIC_ID_MSR, %ecx > + rdmsr > > .Lsetup_AP: > - /* EDX contains the APIC ID of the current CPU */ > + /* EAX contains the APIC ID of the current CPU */ > xorq %rcx, %rcx > leaq cpuid_to_apicid(%rip), %rbx > > .Lfind_cpunr: > - cmpl (%rbx,%rcx,4), %edx > + cmpl (%rbx,%rcx,4), %eax > jz .Lsetup_cpu > inc %ecx > #ifdef CONFIG_FORCE_NR_CPUS > --- a/arch/x86/kernel/smpboot.c > +++ b/arch/x86/kernel/smpboot.c > @@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri > return false; > } > > - /* Encrypted guests require special CPUID handling. */ > + /* Encrypted guests require special handling. */ > if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) { > switch (cc_get_vendor()) { > case CC_VENDOR_AMD: > ctrl = STARTUP_APICID_SEV_ES; > if (topology_extended_leaf == 0x0b) > - goto setup; > + break; > fallthrough; > default: > pr_info("Parallel CPU startup disabled due to guest state encryption\n"); > return false; > } > + } else { > + ctrl = STARTUP_READ_APICID; > } > > - switch (topology_extended_leaf) { > - case 0x0b: > - ctrl = STARTUP_APICID_CPUID_0B; > - break; > - case 0x1f: > - ctrl = STARTUP_APICID_CPUID_1F; > - break; > - case 0x00: > - /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */ > - if (!x2apic_mode) { > - ctrl = STARTUP_APICID_CPUID_01; > - break; > - } > - fallthrough; > - default: > - pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n", > - topology_extended_leaf); > - return false; > - } > - > -setup: > pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl); > smpboot_control = ctrl; > return true; I quickly applied it on top of your branch, but I am getting: ``` $ wget https://lore.kernel.org/lkml/87v8hq35sk.ffs@tglx/raw $ patch -p1 < raw $ make […] LD .tmp_vmlinux.kallsyms1 ld: arch/x86/kernel/head_64.o: in function `secondary_startup_64_no_verify': (.head.text+0xbf): undefined reference to `read_apicid_msr' make[1]: *** [scripts/Makefile.vmlinux:35: vmlinux] Error 1 make: *** [Makefile:1249: vmlinux] Error 2 ``` Kind regards, Paul
On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote: > Am 20.04.23 um 17:57 schrieb Thomas Gleixner: > I quickly applied it on top of your branch, but I am getting: As I said it was untested. I was traveling and did not have access to a machine to even build it completely. Fixed up and tested version below. Thanks, tglx --- --- a/arch/x86/include/asm/apicdef.h +++ b/arch/x86/include/asm/apicdef.h @@ -138,7 +138,8 @@ #define APIC_EILVT_MASKED (1 << 16) #define APIC_BASE (fix_to_virt(FIX_APIC_BASE)) -#define APIC_BASE_MSR 0x800 +#define APIC_BASE_MSR 0x800 +#define APIC_X2APIC_ID_MSR 0x802 #define XAPIC_ENABLE (1UL << 11) #define X2APIC_ENABLE (1UL << 10) @@ -162,6 +163,7 @@ #define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK) #define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT) +#ifndef __ASSEMBLY__ /* * the local APIC register structure, memory mapped. Not terribly well * tested, but we might eventually use this one in the future - the @@ -435,4 +437,5 @@ enum apic_delivery_modes { APIC_DELIVERY_MODE_EXTINT = 7, }; +#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_APICDEF_H */ --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -195,14 +195,13 @@ extern void nmi_selftest(void); #endif extern unsigned int smpboot_control; +extern unsigned long apic_mmio_base; #endif /* !__ASSEMBLY__ */ /* Control bits for startup_64 */ -#define STARTUP_APICID_CPUID_1F 0x80000000 -#define STARTUP_APICID_CPUID_0B 0x40000000 -#define STARTUP_APICID_CPUID_01 0x20000000 -#define STARTUP_APICID_SEV_ES 0x10000000 +#define STARTUP_READ_APICID 0x80000000 +#define STARTUP_APICID_SEV_ES 0x40000000 /* Top 8 bits are reserved for control */ #define STARTUP_PARALLEL_MASK 0xFF000000 --- a/arch/x86/kernel/apic/apic.c +++ b/arch/x86/kernel/apic/apic.c @@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init = */ static bool virt_ext_dest_id __ro_after_init; +unsigned long apic_mmio_base __ro_after_init; + /* * Map cpu index to physical APIC ID */ @@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig if (!x2apic_mode) { set_fixmap_nocache(FIX_APIC_BASE, address); + apic_mmio_base = APIC_BASE; apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n", APIC_BASE, address); } --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -24,8 +24,10 @@ #include "../entry/calling.h" #include <asm/export.h> #include <asm/nospec-branch.h> +#include <asm/apicdef.h> #include <asm/fixmap.h> #include <asm/smp.h> + #include <asm/sev-common.h> /* @@ -237,37 +239,25 @@ SYM_INNER_LABEL(secondary_startup_64_no_ #ifdef CONFIG_SMP /* - * For parallel boot, the APIC ID is retrieved from CPUID, and then - * used to look up the CPU number. For booting a single CPU, the - * CPU number is encoded in smpboot_control. + * For parallel boot, the APIC ID is either retrieved the APIC or + * from CPUID, and then used to look up the CPU number. + * For booting a single CPU, the CPU number is encoded in + * smpboot_control. * - * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f) - * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b) - * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01) - * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR) + * Bit 31 STARTUP_APICID_READ (Read APICID from APIC) + * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR) * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set */ movl smpboot_control(%rip), %ecx + testl $STARTUP_READ_APICID, %ecx + jnz .Lread_apicid #ifdef CONFIG_AMD_MEM_ENCRYPT testl $STARTUP_APICID_SEV_ES, %ecx jnz .Luse_sev_cpuid_0b #endif - testl $STARTUP_APICID_CPUID_1F, %ecx - jnz .Luse_cpuid_1f - testl $STARTUP_APICID_CPUID_0B, %ecx - jnz .Luse_cpuid_0b - testl $STARTUP_APICID_CPUID_01, %ecx - jnz .Luse_cpuid_01 andl $(~STARTUP_PARALLEL_MASK), %ecx jmp .Lsetup_cpu -.Luse_cpuid_01: - mov $0x01, %eax - cpuid - mov %ebx, %edx - shr $24, %edx - jmp .Lsetup_AP - #ifdef CONFIG_AMD_MEM_ENCRYPT .Luse_sev_cpuid_0b: /* Set the GHCB MSR to request CPUID 0x0B_EDX */ @@ -292,24 +282,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_ jmp .Lsetup_AP #endif -.Luse_cpuid_0b: - mov $0x0B, %eax - xorl %ecx, %ecx - cpuid +.Lread_apicid: + mov $MSR_IA32_APICBASE, %ecx + rdmsr + testl $X2APIC_ENABLE, %eax + jnz .Lread_apicid_msr + + /* Read the APIC ID from the fix-mapped MMIO space. */ + movq apic_mmio_base(%rip), %rcx + addq $APIC_ID, %rcx + movl (%rcx), %eax + shr $24, %eax jmp .Lsetup_AP -.Luse_cpuid_1f: - mov $0x1f, %eax - xorl %ecx, %ecx - cpuid +.Lread_apicid_msr: + mov $APIC_X2APIC_ID_MSR, %ecx + rdmsr .Lsetup_AP: - /* EDX contains the APIC ID of the current CPU */ + /* EAX contains the APIC ID of the current CPU */ xorq %rcx, %rcx leaq cpuid_to_apicid(%rip), %rbx .Lfind_cpunr: - cmpl (%rbx,%rcx,4), %edx + cmpl (%rbx,%rcx,4), %eax jz .Lsetup_cpu inc %ecx #ifdef CONFIG_FORCE_NR_CPUS --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri return false; } - /* Encrypted guests require special CPUID handling. */ + /* Encrypted guests require special handling. */ if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) { switch (cc_get_vendor()) { case CC_VENDOR_AMD: ctrl = STARTUP_APICID_SEV_ES; if (topology_extended_leaf == 0x0b) - goto setup; + break; fallthrough; default: pr_info("Parallel CPU startup disabled due to guest state encryption\n"); return false; } + } else { + ctrl = STARTUP_READ_APICID; } - switch (topology_extended_leaf) { - case 0x0b: - ctrl = STARTUP_APICID_CPUID_0B; - break; - case 0x1f: - ctrl = STARTUP_APICID_CPUID_1F; - break; - case 0x00: - /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */ - if (!x2apic_mode) { - ctrl = STARTUP_APICID_CPUID_01; - break; - } - fallthrough; - default: - pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n", - topology_extended_leaf); - return false; - } - -setup: pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl); smpboot_control = ctrl; return true;
On Thu, Apr 20 2023 at 21:10, Thomas Gleixner wrote: > On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote: >> Am 20.04.23 um 17:57 schrieb Thomas Gleixner: >> I quickly applied it on top of your branch, but I am getting: > > As I said it was untested. I was traveling and did not have access to a > machine to even build it completely. Fixed up and tested version below. I've updated git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug for your conveniance. Thanks, tglx
Dear Thomas, Am 20.04.23 um 21:10 schrieb Thomas Gleixner: > On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote: >> Am 20.04.23 um 17:57 schrieb Thomas Gleixner: >> I quickly applied it on top of your branch, but I am getting: > > As I said it was untested. I was traveling and did not have access to a > machine to even build it completely. Fixed up and tested version below. Sorry, if it sounded like a complaint. I just wanted to give a quick feedback. […] I tested your new version even on Friday, and it worked fine – no ten seconds delay. Please find the messages attached. Thank you all for your great work. Kind regards, Paul PS: I am going to try to test your updated branch at the end of the week. [ 0.000000] Linux version 6.3.0-rc3-00046-g8ba643d7e1c7 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #452 SMP PREEMPT_DYNAMIC Thu Apr 20 20:15:01 UTC 2023 [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable [ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-gb640ed51b2 04/17/2023 [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Detected 3900.440 MHz processor [ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000 [ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000938] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000 [ 0.004000] Using GB pages for direct mapping [ 0.004000] ACPI: Early table checksum verification disabled [ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4) [ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925) [ 0.004000] ACPI: FACS 0x000000005FE5A240 000040 [ 0.004000] ACPI: FACS 0x000000005FE5A240 000040 [ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925) [ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000) [ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000) [ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001) [ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925) [ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3] [ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9] [ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f] [ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69] [ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab] [ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11] [ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57] [ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f] [ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be] [ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71] [ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8] [ 0.004000] No NUMA configuration found [ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff] [ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff] [ 0.004000] Zone ranges: [ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff] [ 0.004000] Device empty [ 0.004000] Movable zone start for each node [ 0.004000] Early memory node ranges [ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff] [ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff] [ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff] [ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges [ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges [ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges [ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges [ 0.004000] ACPI: PM-Timer IO Port: 0x818 [ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) [ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23 [ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level) [ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information [ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000 [ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs [ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices [ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns [ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1 [ 0.004000] percpu: Embedded 55 pages/cpu s188392 r8192 d28696 u1048576 [ 0.004000] pcpu-alloc: s188392 r8192 d28696 u1048576 alloc=1*2097152 [ 0.004000] pcpu-alloc: [0] 0 1 [ 0.004000] Fallback order for Node 0: 0 [ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451 [ 0.004000] Policy zone: Normal [ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 [ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7", will be passed to user space. [ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) [ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash [ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.004000] software IO TLB: area num 2. [ 0.004000] Memory: 3477168K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11064K bss, 174072K reserved, 0K cma-reserved) [ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 [ 0.004000] ftrace: allocating 38652 entries in 151 pages [ 0.004000] ftrace: allocated 151 pages with 5 groups [ 0.004000] Dynamic Preempt: full [ 0.004000] rcu: Preemptible hierarchical RCU implementation. [ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. [ 0.004000] Trampoline variant of Tasks RCU enabled. [ 0.004000] Rude variant of Tasks RCU enabled. [ 0.004000] Tracing variant of Tasks RCU enabled. [ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 [ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16 [ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.004000] spurious 8259A interrupt: IRQ7. [ 0.004000] Console: colour VGA+ 80x25 [ 0.004000] printk: console [tty0] enabled [ 0.004000] ACPI: Core revision 20221020 [ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns [ 0.004000] APIC: Switch to symmetric I/O mode setup [ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0 [ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7071f4ed18f, max_idle_ns: 881590420850 ns [ 0.145332] Calibrating delay loop (skipped), value calculated using timer frequency.. 7800.88 BogoMIPS (lpj=15601760) [ 0.145336] pid_max: default: 32768 minimum: 301 [ 0.145430] LSM: initializing lsm=capability [ 0.145526] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) [ 0.145542] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) [ 0.145940] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512 [ 0.145943] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0 [ 0.145947] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 0.145950] Spectre V2 : Mitigation: Retpolines [ 0.145951] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 0.145952] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT [ 0.145952] Spectre V2 : Enabling Speculation Barrier for firmware calls [ 0.145953] RETBleed: Mitigation: untrained return thunk [ 0.145955] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier [ 0.145957] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl [ 0.150469] Freeing SMP alternatives memory: 32K [ 0.258608] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1) [ 0.258845] cblist_init_generic: Setting adjustable number of callback queues. [ 0.258846] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258878] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258905] cblist_init_generic: Setting shift to 1 and lim to 1. [ 0.258933] Performance Events: Fam15h core perfctr, AMD PMU driver. [ 0.258955] ... version: 0 [ 0.258956] ... bit width: 48 [ 0.258957] ... generic registers: 6 [ 0.258958] ... value mask: 0000ffffffffffff [ 0.258959] ... max period: 00007fffffffffff [ 0.258960] ... fixed-purpose events: 0 [ 0.258961] ... event mask: 000000000000003f [ 0.259083] rcu: Hierarchical SRCU implementation. [ 0.259084] rcu: Max phase no-delay instances is 1000. [ 0.259676] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. [ 0.259750] smp: Bringing up secondary CPUs ... [ 0.259952] x86: Booting SMP configuration: [ 0.259953] .... node #0, CPUs: #1 [ 0.259958] smpboot: Kicking AP alive: 17 [ 0.260088] smp: Brought up 1 node, 2 CPUs [ 0.260088] smpboot: Max logical packages: 1 [ 0.260088] smpboot: Total of 2 processors activated (15601.76 BogoMIPS) [ 0.261513] devtmpfs: initialized [ 0.261513] x86/mm: Memory block size: 128MB [ 0.262387] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 0.262387] futex hash table entries: 512 (order: 3, 32768 bytes, linear) [ 0.262387] pinctrl core: initialized pinctrl subsystem [ 0.262387] PM: RTC time: 20:56:07, date: 2023-04-21 [ 0.262387] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 0.262600] audit: initializing netlink subsys (disabled) [ 0.262617] audit: type=2000 audit(1682110567.140:1): state=initialized audit_enabled=0 res=1 [ 0.262617] thermal_sys: Registered thermal governor 'fair_share' [ 0.262617] thermal_sys: Registered thermal governor 'bang_bang' [ 0.262617] thermal_sys: Registered thermal governor 'step_wise' [ 0.262617] thermal_sys: Registered thermal governor 'user_space' [ 0.262617] cpuidle: using governor ladder [ 0.262617] cpuidle: using governor menu [ 0.262617] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000) [ 0.262617] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry [ 0.262617] PCI: Using configuration type 1 for base access [ 0.262617] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 0.273356] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages [ 0.273356] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page [ 0.273356] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages [ 0.273356] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page [ 0.280509] cryptd: max_cpu_qlen set to 1000 [ 0.280509] ACPI: Added _OSI(Module Device) [ 0.280509] ACPI: Added _OSI(Processor Device) [ 0.280509] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.280509] ACPI: Added _OSI(Processor Aggregator Device) [ 0.287039] ACPI: 4 ACPI AML tables successfully acquired and loaded [ 0.288072] ACPI: Interpreter enabled [ 0.288072] ACPI: PM: (supports S0 S1 S3 S5) [ 0.288072] ACPI: Using IOAPIC for interrupt routing [ 0.288072] HEST: Table parsing has been initialized. [ 0.288072] GHES: Failed to enable APEI firmware first mode. [ 0.288072] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.288072] PCI: Ignoring E820 reservations for host bridge windows [ 0.288072] ACPI: Enabled 8 GPEs in block 00 to 1F [ 0.290922] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.290932] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] [ 0.291018] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR] [ 0.291034] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge [ 0.291117] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored [ 0.291369] PCI host bridge to bus 0000:00 [ 0.291371] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.291374] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.291376] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff] [ 0.291379] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff] [ 0.291381] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.291406] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000 [ 0.291559] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600 [ 0.291648] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000 [ 0.291656] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref] [ 0.291661] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff] [ 0.291665] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff] [ 0.291681] pci 0000:00:01.0: enabling Extended Tags [ 0.291694] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] [ 0.291711] pci 0000:00:01.0: supports D1 D2 [ 0.291776] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300 [ 0.291784] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff] [ 0.291805] pci 0000:00:01.1: enabling Extended Tags [ 0.291828] pci 0000:00:01.1: supports D1 D2 [ 0.291918] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601 [ 0.291931] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417] [ 0.291938] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423] [ 0.291946] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f] [ 0.291953] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427] [ 0.291960] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f] [ 0.291968] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff] [ 0.292125] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310 [ 0.292139] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff] [ 0.292322] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320 [ 0.292335] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff] [ 0.292400] pci 0000:00:12.2: supports D1 D2 [ 0.292402] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot [ 0.292542] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310 [ 0.292555] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff] [ 0.292737] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320 [ 0.292750] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff] [ 0.292815] pci 0000:00:13.2: supports D1 D2 [ 0.292816] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot [ 0.292955] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500 [ 0.293139] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300 [ 0.293156] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit] [ 0.293210] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold [ 0.293355] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100 [ 0.293541] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401 [ 0.293703] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310 [ 0.293717] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff] [ 0.293906] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400 [ 0.293935] pci 0000:00:15.0: enabling Extended Tags [ 0.293975] pci 0000:00:15.0: supports D1 D2 [ 0.294151] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400 [ 0.294182] pci 0000:00:15.1: enabling Extended Tags [ 0.294221] pci 0000:00:15.1: supports D1 D2 [ 0.294398] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400 [ 0.294426] pci 0000:00:15.2: enabling Extended Tags [ 0.294468] pci 0000:00:15.2: supports D1 D2 [ 0.294547] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310 [ 0.294560] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff] [ 0.294737] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320 [ 0.294751] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff] [ 0.294815] pci 0000:00:16.2: supports D1 D2 [ 0.294817] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot [ 0.294958] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000 [ 0.295024] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000 [ 0.295086] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000 [ 0.295152] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000 [ 0.295289] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000 [ 0.295353] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000 [ 0.295436] pci_bus 0000:01: extended config space not accessible [ 0.295499] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode) [ 0.295510] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode) [ 0.295513] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode) [ 0.295515] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode) [ 0.295517] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode) [ 0.295571] pci 0000:00:15.0: PCI bridge to [bus 02] [ 0.295653] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330 [ 0.295689] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit] [ 0.295863] pci 0000:03:00.0: PME# supported from D3hot D3cold [ 0.295907] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link) [ 0.309393] pci 0000:00:15.1: PCI bridge to [bus 03] [ 0.309405] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff] [ 0.309414] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring [ 0.309545] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000 [ 0.309563] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff] [ 0.309585] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref] [ 0.309598] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref] [ 0.309705] pci 0000:04:00.0: supports D1 D2 [ 0.309707] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.325403] pci 0000:00:15.2: PCI bridge to [bus 04-ff] [ 0.325414] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff] [ 0.325417] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff] [ 0.325422] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref] [ 0.325425] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04 [ 0.325919] ACPI: PCI: Interrupt link INTA configured for IRQ 0 [ 0.326011] ACPI: PCI: Interrupt link INTB configured for IRQ 0 [ 0.326102] ACPI: PCI: Interrupt link INTC configured for IRQ 0 [ 0.326193] ACPI: PCI: Interrupt link INTD configured for IRQ 0 [ 0.326284] ACPI: PCI: Interrupt link INTE configured for IRQ 0 [ 0.326375] ACPI: PCI: Interrupt link INTF configured for IRQ 0 [ 0.326465] ACPI: PCI: Interrupt link INTG configured for IRQ 0 [ 0.326556] ACPI: PCI: Interrupt link INTH configured for IRQ 0 [ 0.326777] iommu: Default domain type: Translated [ 0.326778] iommu: DMA domain TLB invalidation policy: lazy mode [ 0.326974] SCSI subsystem initialized [ 0.329389] libata version 3.00 loaded. [ 0.329395] ACPI: bus type USB registered [ 0.329418] usbcore: registered new interface driver usbfs [ 0.329428] usbcore: registered new interface driver hub [ 0.329437] usbcore: registered new device driver usb [ 0.329564] PCI: Using ACPI for IRQ routing [ 0.331132] PCI: pci_cache_line_size set to 64 bytes [ 0.331184] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 0.331187] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff] [ 0.331189] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff] [ 0.331235] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.331240] hpet0: 3 comparators, 32-bit 14.318180 MHz counter [ 0.334410] clocksource: Switched to clocksource tsc-early [ 0.334656] VFS: Disk quotas dquot_6.6.0 [ 0.334682] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.334803] pnp: PnP ACPI init [ 0.335147] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved [ 0.335525] pnp: PnP ACPI: found 2 devices [ 0.342072] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.342260] NET: Registered PF_INET protocol family [ 0.342434] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 0.344041] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear) [ 0.344058] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 0.344070] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear) [ 0.344141] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear) [ 0.344505] TCP: Hash tables configured (established 32768 bind 32768) [ 0.344576] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear) [ 0.344601] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear) [ 0.344722] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 0.344757] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref] [ 0.344763] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff] [ 0.344769] pci 0000:00:14.4: PCI bridge to [bus 01] [ 0.344781] pci 0000:00:15.0: PCI bridge to [bus 02] [ 0.344790] pci 0000:00:15.1: PCI bridge to [bus 03] [ 0.344793] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff] [ 0.344807] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref] [ 0.344821] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref] [ 0.344833] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff] [ 0.344840] pci 0000:00:15.2: PCI bridge to [bus 04] [ 0.344842] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff] [ 0.344847] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref] [ 0.344854] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.344857] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.344860] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff] [ 0.344862] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff] [ 0.344864] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window] [ 0.344867] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window] [ 0.344869] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff] [ 0.344871] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff] [ 0.344873] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff] [ 0.344875] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff] [ 0.344877] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref] [ 0.345003] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0 [ 0.345887] pci 0000:00:12.2: PME# does not work under D3, disabling it [ 0.346671] pci 0000:00:13.2: PME# does not work under D3, disabling it [ 0.347980] pci 0000:00:16.2: PME# does not work under D3, disabling it [ 0.348353] PCI: CLS 64 bytes, default 64 [ 0.348449] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround [ 0.348560] pci 0000:00:01.0: Adding to iommu group 0 [ 0.348584] pci 0000:00:01.1: Adding to iommu group 0 [ 0.348616] pci 0000:00:11.0: Adding to iommu group 1 [ 0.348659] pci 0000:00:12.0: Adding to iommu group 2 [ 0.348681] pci 0000:00:12.2: Adding to iommu group 2 [ 0.348723] pci 0000:00:13.0: Adding to iommu group 3 [ 0.348743] pci 0000:00:13.2: Adding to iommu group 3 [ 0.348792] pci 0000:00:14.0: Adding to iommu group 4 [ 0.348817] pci 0000:00:14.2: Adding to iommu group 4 [ 0.348840] pci 0000:00:14.3: Adding to iommu group 4 [ 0.348879] pci 0000:00:14.4: Adding to iommu group 5 [ 0.348903] pci 0000:00:14.5: Adding to iommu group 6 [ 0.348946] pci 0000:00:15.0: Adding to iommu group 7 [ 0.348974] pci 0000:00:15.1: Adding to iommu group 7 [ 0.349000] pci 0000:00:15.2: Adding to iommu group 7 [ 0.349040] pci 0000:00:16.0: Adding to iommu group 8 [ 0.349063] pci 0000:00:16.2: Adding to iommu group 8 [ 0.349128] pci 0000:00:18.0: Adding to iommu group 9 [ 0.349150] pci 0000:00:18.1: Adding to iommu group 9 [ 0.349188] pci 0000:00:18.2: Adding to iommu group 9 [ 0.349210] pci 0000:00:18.3: Adding to iommu group 9 [ 0.349234] pci 0000:00:18.4: Adding to iommu group 9 [ 0.349255] pci 0000:00:18.5: Adding to iommu group 9 [ 0.349269] pci 0000:03:00.0: Adding to iommu group 7 [ 0.349279] pci 0000:04:00.0: Adding to iommu group 7 [ 0.352005] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 [ 0.352011] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA [ 0.352020] AMD-Vi: Interrupt remapping enabled [ 0.352238] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 0.352240] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB) [ 0.352297] LVT offset 0 assigned for vector 0x400 [ 0.352343] perf: AMD IBS detected (0x000000ff) [ 0.352351] amd_uncore: 4 amd_nb counters detected [ 0.356598] workingset: timestamp_bits=37 max_order=20 bucket_order=0 [ 0.356626] zbud: loaded [ 0.357112] NET: Registered PF_ALG protocol family [ 0.357118] Key type asymmetric registered [ 0.357119] Asymmetric key parser 'x509' registered [ 0.357471] alg: self-tests disabled [ 0.357567] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251) [ 0.357610] io scheduler mq-deadline registered [ 0.357612] io scheduler kyber registered [ 0.358782] pcieport 0000:00:15.0: PME: Signaling with IRQ 25 [ 0.358948] pcieport 0000:00:15.1: PME: Signaling with IRQ 26 [ 0.359013] pcieport 0000:00:15.2: enabling device (0000 -> 0003) [ 0.359228] pcieport 0000:00:15.2: PME: Signaling with IRQ 27 [ 0.359503] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 0.359567] ACPI: button: Power Button [PWRF] [ 0.359621] ACPI: \_SB_.P000: Found 2 idle states [ 0.359743] ACPI: \_SB_.P001: Found 2 idle states [ 0.360670] thermal LNXTHERM:00: registered as thermal_zone0 [ 0.360673] ACPI: thermal: Thermal Zone [TZ00] (15 C) [ 0.360984] Non-volatile memory driver v1.3 [ 0.361051] AMD-Vi: AMD IOMMUv2 loaded and initialized [ 0.361079] ACPI: bus type drm_connector registered [ 0.361292] ahci 0000:00:11.0: version 3.0 [ 0.361580] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode [ 0.361584] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio [ 0.362741] scsi host0: ahci [ 0.362967] scsi host1: ahci [ 0.363157] scsi host2: ahci [ 0.363372] scsi host3: ahci [ 0.363571] scsi host4: ahci [ 0.363773] scsi host5: ahci [ 0.363984] scsi host6: ahci [ 0.364182] scsi host7: ahci [ 0.364275] ata1: DUMMY [ 0.364277] ata2: DUMMY [ 0.364278] ata3: DUMMY [ 0.364278] ata4: DUMMY [ 0.364279] ata5: DUMMY [ 0.364280] ata6: DUMMY [ 0.364282] ata7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19 [ 0.364284] ata8: DUMMY [ 0.364564] i8042: PNP: No PS/2 controller found. [ 0.364565] i8042: Probing ports directly. [ 0.366981] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 0.366988] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 0.367123] mousedev: PS/2 mouse device common for all mice [ 0.367188] rtc_cmos 00:01: RTC can wake from S4 [ 0.367487] rtc_cmos 00:01: registered as rtc0 [ 0.367510] rtc_cmos 00:01: setting system clock to 2023-04-21T20:56:07 UTC (1682110567) [ 0.367553] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs [ 0.367590] device-mapper: uevent: version 1.0.3 [ 0.367665] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com [ 0.367827] hid: raw HID events driver (C) Jiri Kosina [ 0.367913] usbcore: registered new interface driver usbhid [ 0.367914] usbhid: USB HID core driver [ 0.368010] Initializing XFRM netlink socket [ 0.368020] NET: Registered PF_PACKET protocol family [ 0.368021] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending. [ 0.368340] microcode: CPU1: patch_level=0x0600111f [ 0.368342] microcode: CPU0: patch_level=0x0600111f [ 0.368353] microcode: Microcode Update Driver: v2.2. [ 0.368358] IPI shorthand broadcast: enabled [ 0.368370] AVX version of gcm_enc/dec engaged. [ 0.368401] AES CTR mode by8 optimization enabled [ 0.372495] sched_clock: Marking stable (254511749, 117328013)->(374590709, -2750947) [ 0.372745] registered taskstats version 1 [ 0.372987] zswap: loaded using pool lzo/zbud [ 0.377461] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679) [ 0.377464] kmemleak: Automatic memory scanning thread started [ 0.377465] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 0.382250] Key type encrypted registered [ 0.385395] PM: Magic number: 3:579:953 [ 0.385449] memory memory6: hash matches [ 0.483426] ata7: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 0.483587] ata7.00: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133 [ 0.483590] ata7.00: 125045424 sectors, multi 1: LBA48 NCQ (depth 32) [ 0.483792] ata7.00: configured for UDMA/133 [ 0.484004] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5 [ 0.485006] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB) [ 0.485028] sd 6:0:0:0: [sda] Write Protect is off [ 0.485033] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00 [ 0.485062] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 0.485103] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes [ 0.486554] sda: sda1 sda2 sda3 [ 0.487033] sd 6:0:0:0: [sda] Attached SCSI disk [ 0.508543] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none. [ 0.508591] VFS: Mounted root (ext4 filesystem) on device 8:3. [ 0.510618] devtmpfs: mounted [ 0.514506] Freeing unused kernel image (initmem) memory: 2908K [ 0.532141] Write protecting the kernel read-only data: 20480k [ 0.532425] Freeing unused kernel image (rodata/data gap) memory: 836K [ 0.569736] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 0.569742] rodata_test: all tests were successful [ 0.569773] Run /sbin/init as init process [ 0.569775] with arguments: [ 0.569776] /sbin/init [ 0.569777] noisapnp [ 0.569778] with environment: [ 0.569779] HOME=/ [ 0.569780] TERM=linux [ 0.569781] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 [ 0.751269] systemd[1]: Inserted module 'autofs4' [ 0.777767] NET: Registered PF_INET6 protocol family [ 0.778674] Segment Routing with IPv6 [ 0.778702] In-situ OAM (IOAM) with IPv6 [ 0.804133] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) [ 0.804143] systemd[1]: Detected architecture x86-64. [ 0.808903] systemd[1]: Hostname set to <kodi>. [ 1.081619] systemd[1]: Queued start job for default target graphical.target. [ 1.102404] systemd[1]: Created slice system-getty.slice - Slice /system/getty. [ 1.103505] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe. [ 1.104345] systemd[1]: Created slice user.slice - User and Session Slice. [ 1.104531] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. [ 1.104655] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch. [ 1.105082] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point. [ 1.105122] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes. [ 1.105171] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes. [ 1.105213] systemd[1]: Reached target paths.target - Path Units. [ 1.105248] systemd[1]: Reached target remote-fs.target - Remote File Systems. [ 1.105276] systemd[1]: Reached target slices.target - Slice Units. [ 1.105302] systemd[1]: Reached target swap.target - Swaps. [ 1.105344] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes. [ 1.107911] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket. [ 1.108154] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket. [ 1.108318] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe. [ 1.108624] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket. [ 1.108886] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log). [ 1.109157] systemd[1]: Listening on systemd-journald.socket - Journal Socket. [ 1.109414] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket. [ 1.110215] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket. [ 1.110475] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket. [ 1.113168] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System... [ 1.116585] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System... [ 1.121085] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System... [ 1.124574] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System... [ 1.128227] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes... [ 1.132312] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs... [ 1.135610] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod... [ 1.138876] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm... [ 1.147425] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore... [ 1.150601] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse... [ 1.153921] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop... [ 1.154024] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes). [ 1.154112] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/). [ 1.154158] systemd[1]: Reached target local-fs.target - Local File Systems. [ 1.154259] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor). [ 1.167112] fuse: init (API version 7.38) [ 1.167255] loop: module loaded [ 1.168586] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats... [ 1.174035] systemd[1]: Starting systemd-journald.service - Journal Service... [ 1.177353] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed... [ 1.180848] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables... [ 1.195025] systemd[1]: Starting systemd-sysusers.service - Create System Users... [ 1.201956] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices... [ 1.217695] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System. [ 1.218409] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System. [ 1.218683] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System. [ 1.218957] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System. [ 1.225546] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes. [ 1.226534] systemd[1]: modprobe@configfs.service: Deactivated successfully. [ 1.226893] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs. [ 1.227645] systemd[1]: modprobe@dm_mod.service: Deactivated successfully. [ 1.228209] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod. [ 1.230383] systemd[1]: modprobe@drm.service: Deactivated successfully. [ 1.230725] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm. [ 1.231459] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully. [ 1.231774] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore. [ 1.232474] systemd[1]: modprobe@fuse.service: Deactivated successfully. [ 1.232795] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse. [ 1.234105] systemd[1]: modprobe@loop.service: Deactivated successfully. [ 1.234462] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop. [ 1.235099] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 140 (systemd-binfmt) [ 1.261703] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System... [ 1.270160] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System... [ 1.273274] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System... [ 1.273402] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore). [ 1.273574] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met. [ 1.290481] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables. [ 1.294159] systemd[1]: Finished systemd-sysusers.service - Create System Users. [ 1.294555] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System. [ 1.303356] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev... [ 1.304817] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System. [ 1.320909] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System. [ 1.323560] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats. [ 1.369356] tsc: Refined TSC clocksource calibration: 3900.224 MHz [ 1.369367] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns [ 1.370036] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. [ 1.373207] clocksource: Switched to clocksource tsc [ 1.389831] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files... [ 1.427455] systemd[1]: Started systemd-journald.service - Journal Service. [ 1.472737] systemd-journald[142]: Received client request to flush runtime journal. [ 1.742223] sd 6:0:0:0: Attached scsi generic sg0 type 0 [ 1.866042] acpi_cpufreq: overriding BIOS provided _PSD data [ 2.005347] random: crng init done [ 2.137832] QUIRK: Enable AMD PLL fix [ 2.137832] QUIRK: Enable AMD PLL fix [ 2.137887] ehci-pci 0000:00:13.2: EHCI Host Controller [ 2.137922] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 1 [ 2.137936] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 2.137945] ehci-pci 0000:00:13.2: debug port 1 [ 2.138141] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000 [ 2.139491] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0 [ 2.139505] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection [ 2.140107] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20 [ 2.154632] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00 [ 2.155284] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 2.155291] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.155294] usb usb1: Product: EHCI Host Controller [ 2.155296] usb usb1: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd [ 2.155299] usb usb1: SerialNumber: 0000:00:13.2 [ 2.174521] hub 1-0:1.0: USB hub found [ 2.174560] hub 1-0:1.0: 5 ports detected [ 2.175582] ehci-pci 0000:00:12.2: EHCI Host Controller [ 2.175610] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 2 [ 2.175623] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 2.175632] ehci-pci 0000:00:12.2: debug port 1 [ 2.175789] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000 [ 2.189373] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00 [ 2.189720] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 2.189725] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.189727] usb usb2: Product: EHCI Host Controller [ 2.189730] usb usb2: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd [ 2.189732] usb usb2: SerialNumber: 0000:00:12.2 [ 2.190283] hub 2-0:1.0: USB hub found [ 2.190323] hub 2-0:1.0: 5 ports detected [ 2.191801] ehci-pci 0000:00:16.2: EHCI Host Controller [ 2.191831] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3 [ 2.191845] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround [ 2.191855] ehci-pci 0000:00:16.2: debug port 1 [ 2.192018] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000 [ 2.205373] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00 [ 2.205651] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 2.205655] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.205658] usb usb3: Product: EHCI Host Controller [ 2.205660] usb usb3: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd [ 2.205662] usb usb3: SerialNumber: 0000:00:16.2 [ 2.206200] hub 3-0:1.0: USB hub found [ 2.206236] hub 3-0:1.0: 4 ports detected [ 2.221053] ohci-pci 0000:00:12.0: OHCI PCI host controller [ 2.221085] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4 [ 2.221264] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000 [ 2.237796] r8169 0000:04:00.0: enabling device (0000 -> 0003) [ 2.261308] xhci_hcd 0000:03:00.0: xHCI Host Controller [ 2.261394] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 5 [ 2.272125] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28 [ 2.272143] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko] [ 2.282633] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 2.282644] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.282650] usb usb4: Product: OHCI PCI host controller [ 2.282655] usb usb4: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd [ 2.282659] usb usb4: SerialNumber: 0000:00:12.0 [ 2.283372] hub 4-0:1.0: USB hub found [ 2.283431] hub 4-0:1.0: 5 ports detected [ 2.297576] ohci-pci 0000:00:13.0: OHCI PCI host controller [ 2.297609] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 6 [ 2.297727] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000 [ 2.320955] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010 [ 2.322099] xhci_hcd 0000:03:00.0: xHCI Host Controller [ 2.322121] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 7 [ 2.322136] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed [ 2.324029] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03 [ 2.324036] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.324039] usb usb5: Product: xHCI Host Controller [ 2.324041] usb usb5: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 xhci-hcd [ 2.324043] usb usb5: SerialNumber: 0000:03:00.0 [ 2.324687] hub 5-0:1.0: USB hub found [ 2.324727] hub 5-0:1.0: 2 ports detected [ 2.325467] usb usb7: We don't know the algorithms for LPM for this host, disabling LPM. [ 2.325624] usb usb7: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03 [ 2.325628] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.325630] usb usb7: Product: xHCI Host Controller [ 2.325632] usb usb7: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 xhci-hcd [ 2.325634] usb usb7: SerialNumber: 0000:03:00.0 [ 2.326195] hub 7-0:1.0: USB hub found [ 2.327034] hub 7-0:1.0: 2 ports detected [ 2.349205] snd_hda_intel 0000:00:01.1: Force to non-snoop mode [ 2.366399] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 2.366406] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.366409] usb usb6: Product: OHCI PCI host controller [ 2.366411] usb usb6: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd [ 2.366413] usb usb6: SerialNumber: 0000:00:13.0 [ 2.367210] hub 6-0:1.0: USB hub found [ 2.367303] hub 6-0:1.0: 5 ports detected [ 2.391447] r8169 0000:04:00.0 enp4s0: renamed from eth0 [ 2.420037] ohci-pci 0000:00:14.5: OHCI PCI host controller [ 2.420070] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 8 [ 2.421556] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000 [ 2.445281] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2 [ 2.445292] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2) [ 2.445868] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC) [ 2.481250] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1 [ 2.481880] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2 [ 2.488733] usb usb8: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 2.488740] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.488743] usb usb8: Product: OHCI PCI host controller [ 2.488745] usb usb8: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd [ 2.488747] usb usb8: SerialNumber: 0000:00:14.5 [ 2.492816] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100 [ 2.492902] hub 8-0:1.0: USB hub found [ 2.492953] hub 8-0:1.0: 2 ports detected [ 2.493612] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line [ 2.493620] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0) [ 2.493623] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0) [ 2.493626] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0 [ 2.493628] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0 [ 2.493629] snd_hda_codec_realtek hdaudioC1D0: inputs: [ 2.493631] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18 [ 2.493633] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19 [ 2.493635] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a [ 2.493636] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c [ 2.495003] ohci-pci 0000:00:16.0: OHCI PCI host controller [ 2.495036] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 9 [ 2.495172] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000 [ 2.510556] [drm] radeon kernel modesetting enabled. [ 2.513510] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00). [ 2.513588] ATOM BIOS: 113 [ 2.513893] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used) [ 2.513898] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF [ 2.513911] [drm] Detected VRAM RAM=512M, BAR=256M [ 2.513913] [drm] RAM width 64bits DDR [ 2.514805] [drm] radeon: 512M of VRAM memory ready [ 2.514815] [drm] radeon: 1024M of GTT memory ready. [ 2.514888] [drm] Loading ARUBA Microcode [ 2.517185] r8169 0000:04:00.0 enp4s0: Link is Down [ 2.519927] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3 [ 2.520307] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4 [ 2.520824] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5 [ 2.521180] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6 [ 2.521594] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7 [ 2.522160] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8 [ 2.522536] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9 [ 2.522888] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10 [ 2.532990] [drm] Internal thermal controller without fan control [ 2.558842] usb usb9: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03 [ 2.558849] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.558852] usb usb9: Product: OHCI PCI host controller [ 2.558855] usb usb9: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd [ 2.558857] usb usb9: SerialNumber: 0000:00:16.0 [ 2.558984] [drm] radeon: dpm initialized [ 2.564667] [drm] Found VCE firmware/feedback version 50.0.1 / 17! [ 2.564728] [drm] GART: num cpu pages 262144, num gpu pages 262144 [ 2.594051] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000). [ 2.594314] radeon 0000:00:01.0: WB enabled [ 2.594318] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00 [ 2.594697] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18 [ 2.595711] hub 9-0:1.0: USB hub found [ 2.595753] hub 9-0:1.0: 4 ports detected [ 2.618866] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18 [ 2.618874] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c [ 2.618877] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04 [ 2.618879] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08 [ 2.618881] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c [ 2.618883] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10 [ 2.647823] radeon 0000:00:01.0: radeon: MSI limited to 32-bit [ 2.648048] radeon 0000:00:01.0: radeon: using MSI. [ 2.648125] [drm] radeon: irq initialized. [ 2.667958] [drm] ring test on 0 succeeded in 3 usecs [ 2.667970] [drm] ring test on 3 succeeded in 4 usecs [ 2.667977] [drm] ring test on 4 succeeded in 4 usecs [ 2.713784] [drm] ring test on 5 succeeded in 2 usecs [ 2.733666] [drm] UVD initialized successfully. [ 2.741341] usb 4-1: new low-speed USB device number 2 using ohci-pci [ 2.843040] [drm] ring test on 6 succeeded in 18 usecs [ 2.843052] [drm] ring test on 7 succeeded in 3 usecs [ 2.843053] [drm] VCE initialized successfully. [ 2.843202] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon]) [ 2.843370] [drm] ib test on ring 0 succeeded in 0 usecs [ 2.843424] [drm] ib test on ring 3 succeeded in 0 usecs [ 2.843474] [drm] ib test on ring 4 succeeded in 0 usecs [ 2.942595] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01 [ 2.942607] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 2.942611] usb 4-1: Product: Dell QuietKey Keyboard [ 2.942615] usb 4-1: Manufacturer: DELL [ 2.951006] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11 [ 3.010775] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0 [ 3.385454] [drm] ib test on ring 5 succeeded [ 3.405349] usb 4-2: new low-speed USB device number 3 using ohci-pci [ 3.600597] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40 [ 3.600609] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 3.600613] usb 4-2: Product: Optical USB Mouse [ 3.600617] usb 4-2: Manufacturer: Logitech [ 3.610963] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12 [ 3.611917] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0 [ 3.929454] [drm] ib test on ring 6 succeeded [ 4.441455] [drm] ib test on ring 7 succeeded [ 4.446983] [drm] Radeon Display Connectors [ 4.446989] [drm] Connector 0: [ 4.446992] [drm] DP-1 [ 4.446995] [drm] HPD1 [ 4.446997] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c [ 4.447003] [drm] Encoders: [ 4.447005] [drm] DFP1: INTERNAL_UNIPHY2 [ 4.447007] [drm] Connector 1: [ 4.447009] [drm] VGA-1 [ 4.447011] [drm] HPD2 [ 4.447013] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c [ 4.447018] [drm] Encoders: [ 4.447020] [drm] CRT1: INTERNAL_UNIPHY2 [ 4.447022] [drm] CRT1: NUTMEG [ 4.447024] [drm] Connector 2: [ 4.447026] [drm] HDMI-A-1 [ 4.447028] [drm] HPD3 [ 4.447030] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c [ 4.447035] [drm] Encoders: [ 4.447036] [drm] DFP2: INTERNAL_UNIPHY [ 4.716171] [drm] fb mappable at 0xE03E9000 [ 4.716179] [drm] vram apper at 0xE0000000 [ 4.716181] [drm] size 5242880 [ 4.716183] [drm] fb depth is 24 [ 4.716185] [drm] pitch is 5120 [ 4.716671] fbcon: radeondrmfb (fb0) is primary device [ 4.908197] Console: switching to colour frame buffer device 160x64 [ 4.909974] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device [ 4.933682] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0 [ 5.125043] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx [ 5.125059] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready [ 7.669093] [drm] amdgpu kernel modesetting enabled. [ 11.216542] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=272 'systemd'
On Thu, Apr 20 2023 at 17:57, Thomas Gleixner wrote: > On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote: > Something like the completely untested below should just work whatever > APIC ID the BIOS decided to dice. > > That might just work on SEV too without that GHCB muck, but what do I > know. It does not. RDMSR(X2APIC_ID) is trapped via #VC which cannot be handled at that point. Unfortunately the GHCB protocol does not provide a RDMSR mechanism similar to the CPUID mechanism. Neither does the secure firmware enforce CPUID(0xb):APICID to real APIC ID consistency. So the hypervisor can dice the APIC IDs as long as they are consistent with the provided ACPI/MADT table. So no parallel startup for SEV for now. Thanks, tglx
From: Thomas Gleixner <tglx@linutronix.de> Sent: Friday, April 14, 2023 4:44 PM [snip] > > Conclusion > ---------- > > Adding the basic parallel bringup mechanism as provided by this series > makes a lot of sense. Improving particular issues as pointed out in the > analysis makes sense too. > > But trying to solve an application specific problem fully in the kernel > with tons of complexity, without exploring straight forward and simple > approaches first, does not make any sense at all. > > Thanks, > > tglx > > --- > Documentation/admin-guide/kernel-parameters.txt | 20 > Documentation/core-api/cpu_hotplug.rst | 13 > arch/Kconfig | 23 + > arch/arm/Kconfig | 1 > arch/arm/include/asm/smp.h | 2 > arch/arm/kernel/smp.c | 18 > arch/arm64/Kconfig | 1 > arch/arm64/include/asm/smp.h | 2 > arch/arm64/kernel/smp.c | 14 > arch/csky/Kconfig | 1 > arch/csky/include/asm/smp.h | 2 > arch/csky/kernel/smp.c | 8 > arch/mips/Kconfig | 1 > arch/mips/cavium-octeon/smp.c | 1 > arch/mips/include/asm/smp-ops.h | 1 > arch/mips/kernel/smp-bmips.c | 1 > arch/mips/kernel/smp-cps.c | 14 > arch/mips/kernel/smp.c | 8 > arch/mips/loongson64/smp.c | 1 > arch/parisc/Kconfig | 1 > arch/parisc/kernel/process.c | 4 > arch/parisc/kernel/smp.c | 7 > arch/riscv/Kconfig | 1 > arch/riscv/include/asm/smp.h | 2 > arch/riscv/kernel/cpu-hotplug.c | 14 > arch/x86/Kconfig | 45 -- > arch/x86/include/asm/apic.h | 5 > arch/x86/include/asm/cpu.h | 5 > arch/x86/include/asm/cpumask.h | 5 > arch/x86/include/asm/processor.h | 1 > arch/x86/include/asm/realmode.h | 3 > arch/x86/include/asm/sev-common.h | 3 > arch/x86/include/asm/smp.h | 26 - > arch/x86/include/asm/topology.h | 23 - > arch/x86/include/asm/tsc.h | 2 > arch/x86/kernel/acpi/sleep.c | 9 > arch/x86/kernel/apic/apic.c | 22 - > arch/x86/kernel/callthunks.c | 4 > arch/x86/kernel/cpu/amd.c | 2 > arch/x86/kernel/cpu/cacheinfo.c | 21 > arch/x86/kernel/cpu/common.c | 50 -- > arch/x86/kernel/cpu/topology.c | 3 > arch/x86/kernel/head_32.S | 14 > arch/x86/kernel/head_64.S | 121 +++++ > arch/x86/kernel/sev.c | 2 > arch/x86/kernel/smp.c | 3 > arch/x86/kernel/smpboot.c | 508 ++++++++---------------- > arch/x86/kernel/topology.c | 98 ---- > arch/x86/kernel/tsc.c | 20 > arch/x86/kernel/tsc_sync.c | 36 - > arch/x86/power/cpu.c | 37 - > arch/x86/realmode/init.c | 3 > arch/x86/realmode/rm/trampoline_64.S | 27 + > arch/x86/xen/enlighten_hvm.c | 11 > arch/x86/xen/smp_hvm.c | 16 > arch/x86/xen/smp_pv.c | 56 +- > drivers/acpi/processor_idle.c | 4 > include/linux/cpu.h | 4 > include/linux/cpuhotplug.h | 17 > kernel/cpu.c | 397 +++++++++++++++++- > kernel/smp.c | 2 > kernel/smpboot.c | 163 ------- > 62 files changed, 953 insertions(+), 976 deletions(-) > I smoke-tested several Linux guest configurations running on Hyper-V, using the "kernel/git/tglx/devel.git hotplug" tree as updated on April 26th. No functional issues, but encountered one cosmetic issue (details below). Configurations tested: * 16 vCPUs and 32 vCPUs * 1 NUMA node and 2 NUMA nodes * Parallel bring-up enabled and disabled via kernel boot line * "Normal" VMs and SEV-SNP VMs running with a paravisor on Hyper-V. This config can use parallel bring-up because most of the SNP-ness is hidden in the paravisor. I was glad to see this work properly. There's not much difference in performance with and without parallel bring-up on the 32 vCPU VM. Without parallel, the time is about 26 milliseconds. With parallel, it's about 24 ms. So bring-up is already fast in the virtual environment. The cosmetic issue is in the dmesg log, and arises because Hyper-V enumerates SMT CPUs differently from many other environments. In a Hyper-V guest, the SMT threads in a core are numbered as <even, odd> pairs. Guest CPUs #0 & #1 are SMT threads in core, as are #2 & #3, etc. With parallel bring-up, here's the dmesg output: [ 0.444345] smp: Bringing up secondary CPUs ... [ 0.445139] .... node #0, CPUs: #2 #4 #6 #8 #10 #12 #14 #16 #18 #20 #22 #24 #26 #28 #30 [ 0.454112] x86: Booting SMP configuration: [ 0.456035] #1 #3 #5 #7 #9 #11 #13 #15 #17 #19 #21 #23 #25 #27 #29 #31 [ 0.466120] smp: Brought up 1 node, 32 CPUs [ 0.467036] smpboot: Max logical packages: 1 [ 0.468035] smpboot: Total of 32 processors activated (153240.06 BogoMIPS) The function announce_cpu() is specifically testing for CPU #1 to output the "Booting SMP configuration" message. In a Hyper-V guest, CPU #1 is the second SMT thread in a core, so it isn't started until all the even-numbered CPUs are started. I don't know if this cosmetic issue is worth fixing, but I thought I'd point it out. In any case, Tested-by: Michael Kelley <mikelley@microsoft.com>
Michael! On Thu, Apr 27 2023 at 14:48, Michael Kelley wrote: > From: Thomas Gleixner <tglx@linutronix.de> Sent: Friday, April 14, 2023 4:44 PM > > I smoke-tested several Linux guest configurations running on Hyper-V, > using the "kernel/git/tglx/devel.git hotplug" tree as updated on April 26th. > No functional issues, but encountered one cosmetic issue (details below). > > Configurations tested: > * 16 vCPUs and 32 vCPUs > * 1 NUMA node and 2 NUMA nodes > * Parallel bring-up enabled and disabled via kernel boot line > * "Normal" VMs and SEV-SNP VMs running with a paravisor on Hyper-V. > This config can use parallel bring-up because most of the SNP-ness is > hidden in the paravisor. I was glad to see this work properly. > > There's not much difference in performance with and without parallel > bring-up on the 32 vCPU VM. Without parallel, the time is about 26 > milliseconds. With parallel, it's about 24 ms. So bring-up is already > fast in the virtual environment. Depends on the environment :) > The cosmetic issue is in the dmesg log, and arises because Hyper-V > enumerates SMT CPUs differently from many other environments. In > a Hyper-V guest, the SMT threads in a core are numbered as <even, odd> > pairs. Guest CPUs #0 & #1 are SMT threads in core, as are #2 & #3, etc. With > parallel bring-up, here's the dmesg output: > > [ 0.444345] smp: Bringing up secondary CPUs ... > [ 0.445139] .... node #0, CPUs: #2 #4 #6 #8 #10 #12 #14 #16 #18 #20 #22 #24 #26 #28 #30 > [ 0.454112] x86: Booting SMP configuration: > [ 0.456035] #1 #3 #5 #7 #9 #11 #13 #15 #17 #19 #21 #23 #25 #27 #29 #31 > [ 0.466120] smp: Brought up 1 node, 32 CPUs > [ 0.467036] smpboot: Max logical packages: 1 > [ 0.468035] smpboot: Total of 32 processors activated (153240.06 BogoMIPS) > > The function announce_cpu() is specifically testing for CPU #1 to output the > "Booting SMP configuration" message. In a Hyper-V guest, CPU #1 is the second > SMT thread in a core, so it isn't started until all the even-numbered CPUs are > started. Ah. Didn't notice that because SMT siblings are usually enumerated after all primary ones in ACPI. > I don't know if this cosmetic issue is worth fixing, but I thought I'd point it out. That's trivial enough to fix. I'll amend the topmost patch before posting V2. Thanks for giving it a ride! tglx
Hi! This is a complete rework of the parallel bringup patch series (V17) https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com to address the issues which were discovered in review: 1) The X86 microcode loader serialization requirement https://lore.kernel.org/lkml/87v8iirxun.ffs@tglx Microcode loading on HT enabled X86 CPUs requires that the microcode is loaded on the primary thread. The sibling thread(s) must be in quiescent state; either looping in a place which is aware of potential changes by the microcode update (see late loading) or in fully quiescent state, i.e. waiting for INIT/SIPI. This is required by hardware/firmware on Intel. Aside of that it's a vendor independent software correctness issue. Assume the following sequence: CPU1.0 CPU1.1 CPUID($A) Load microcode. Changes CPUID($A, $B) CPUID($B) CPU1.1 makes a decision on $A and $B which might be inconsistent due to the microcode update. The solution for this is to bringup the primary threads first and after that the siblings. Loading microcode on the siblings is a NOOP on Intel and on AMD it is guaranteed to only modify thread local state. This ensures that the APs can load microcode before reaching the alive synchronization point w/o doing any further x86 specific synchronization between the core siblings. 2) The general design issues discussed in V16 https://lore.kernel.org/lkml/87pm8y6yme.ffs@tglx The previous parallel bringup patches just glued this mechanism into the existing code without a deeper analysis of the synchronization mechanisms and without generalizing it so that the control logic is mostly in the core code and not made an architecture specific tinker space. Much of that had been pointed out 2 years ago in the discussions about the early versions of parallel bringup already. The series is based on: git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip x86/apic and also available from git: git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug Background ---------- The reason why people are interested in parallel bringup is to shorten the (kexec) reboot time of cloud servers to reduce the downtime of the VM tenants. There are obviously other interesting use cases for this like VM startup time, embedded devices... The current fully serialized bringup does the following per AP: 1) Prepare callbacks (allocate, intialize, create threads) 2) Kick the AP alive (e.g. INIT/SIPI on x86) 3) Wait for the AP to report alive state 4) Let the AP continue through the atomic bringup 5) Let the AP run the threaded bringup to full online state There are two significant delays: #3 The time for an AP to report alive state in start_secondary() on x86 has been measured in the range between 350us and 3.5ms depending on vendor and CPU type, BIOS microcode size etc. #4 The atomic bringup does the microcode update. This has been measured to take up to ~8ms on the primary threads depending on the microcode patch size to apply. On a two socket SKL server with 56 cores (112 threads) the boot CPU spends on current mainline about 800ms busy waiting for the APs to come up and apply microcode. That's more than 80% of the actual onlining procedure. By splitting the actual bringup mechanism into two parts this can be reduced to waiting for the first AP to report alive or if the system is large enough the first AP is already waiting when the boot CPU finished the wake-up of the last AP. The actual solution comes in several parts ------------------------------------------ 1) [P 1-2] General cleanups (init annotations, kernel doc...) 2) [P 3] The obvious Avoid pointless delay calibration when TSC is synchronized across sockets. That removes a whopping 100ms delay for the first CPU of a socket. This is an improvement independent of parallel bringup and had been discussed two years ago already. 2) [P 3-6] Removal of the CPU0 hotplug hack. This was added 11 years ago with the promise to make this a real hardware mechanism, but that never materialized. As physical CPU hotplug is not really supported and the physical unplugging of CPU0 never materialized there is no reason to keep this cruft around. It's just maintenance ballast for no value and the removal makes implementing the parallel bringup feature way simpler. 3) [P 7-16] Cleanup of the existing bringup mechanism: a) Code reorganisation so that the general hotplug specific code is in smpboot.c and not sprinkled all over the place b) Decouple MTRR/PAT initialization from smp_callout_mask to prepare for replacing that mask with a hotplug core code synchronization mechanism. c) Make TSC synchronization function call based so that the control CPU does not have to busy wait for nothing if synchronization is not required. d) Remove the smp_callin_mask synchronization point as its not longer required due to #3c. e) Rework the sparse_irq_lock held region in the core code so that the next polling synchronization point in the x86 code can be removed to. f) Due to #3e it's not longer required to spin wait for the AP to set it's online bit. Remove wait_cpu_online() and the XENPV counterpart. So the control CPU can directly wait for the online idle completion by the AP and free the control CPU up for other work. This reduces the synchronization points in the x86 code to one, which is the AP alive one. This synchronization will be moved to core infrastructure in the next section. 4) [P 17-27] Replace the disconnected CPU state tracking The extra CPU state tracking which is used by a few architectures is completely separate from the CPU hotplug core code. Replacing it by a variant integrated in the core hotplug machinery allows to reduce architecture specific code and provides a generic synchronization mechanism for (parallel) CPU bringup/teardown. - Convert x86 over and replace the AP alive synchronization on x86 with the core variant which removes the remaining x86 hotplug synchronization masks. - Convert the other architectures usage and remove the old interface and code. 5) [P 28-30] Split the bringup into two steps First step invokes the wakeup function on the BP, e.g. SIPI/STARTUP on x86. The second one waits on the BP for the AP to report alive and releases it for the complete onlining. As the hotplug state machine allows partial bringup this allows later to kick all APs alive in a first iteration and then bring them up completely one by one afterwards. 6) [P 31] Switch the primary thread detection to a cpumask This makes the parallel bringup a simple cpumask based mechanism without tons of conditionals and checks for primary threads. 7) [P 32] Implement the parallel bringup core code The parallel bringup looks like this: 1) Bring up the primary SMT threads to the CPUHP_KICK_AP_ALIVE step one by one 2) Bring up the primary SMT threads to the CPUHP_ONLINE step one by one 3) Bring up the secondary SMT threads to the CPUHP_KICK_AP_ALIVE step one by one 4) Bring up the secondary SMT threads to the CPUHP_ONLINE step one by one In case that SMT is not supported this is obviously reduced to step #1 and #2. 8) [P 33-37] Prepare X86 for parallel bringup and enable it Caveats ------- The non X86 changes have been all compile tested. Boot and runtime testing has only be done on a few real hardware platforms and qemu as available. That definitely needs some help from the people who have these systems at their fingertips. Results and analysis -------------------- Here are numbers for a dual socket SKL 56 cores/ 112 threads machine. All numbers in milliseconds. The time measured is the time which the cpu_up() call takes for each CPU and phase. It's not exact as the system is already scheduling, handling interrupts and soft interrupts, which is obviously skewing the picture slightly. Baseline tip tree x86/apic branch. total avg/CPU min max total : 912.081 8.217 3.720 113.271 The max of 100ms is due to the silly delay calibration for the second socket which takes 100ms and was eliminated first. Also the other initial cleanups and improvements take some time away. So the real baseline becomes: total avg/CPU min max total : 785.960 7.081 3.752 36.098 The max here is on the first CPU of the second socket. 20ms of that is due to TSC synchronization and an extra 2ms to react on the SIPI. With parallel bootup enabled this becomes: total avg/CPU min max prepare: 39.108 0.352 0.238 0.883 online : 45.166 0.407 0.170 20.357 total : 84.274 0.759 0.408 21.240 That's a factor ~9.3 reduction on average. Looking at the 27 primary threads of socket 0 then this becomes even more interesting: total avg/CPU min max total : 325.764 12.065 11.981 14.125 versus: total avg/CPU min max prepare: 8.945 0.331 0.238 0.834 online : 4.830 0.179 0.170 0.212 total : 13.775 0.510 0.408 1.046 So the reduction factor is ~23.5 here. That's mostly because the 20ms TSC sync is not skewing the picture. For all 55 primaries, i.e with the 20ms TSC sync extra for socket 1 this becomes: total avg/CPU min max total : 685.489 12.463 11.975 36.098 versus: total avg/CPU min max prepare: 19.080 0.353 0.238 0.883 online : 30.283 0.561 0.170 20.357 total : 49.363 0.914 0.408 21.240 The TSC sync reduces the win to a factor of ~13.8 With 'tsc=reliable' on the command line the socket sync is disabled which brings it back to the socket 0 numbers: total avg/CPU min max prepare: 18.970 0.351 0.231 0.874 online : 10.328 0.191 0.169 0.358 total : 29.298 0.543 0.400 1.232 Now looking at the secondary threads only: total avg/CPU min max total : 100.471 1.794 0.375 4.745 versus: total avg/CPU min max prepare: 19.753 0.353 0.257 0.512 online : 14.671 0.262 0.179 3.461 total : 34.424 0.615 0.436 3.973 Still a factor of ~3. The average on the secondaries for the serialized bringup is significantly lower than for the primaries because the SIPI response time is shorter and the microcode update takes no time. This varies wildly with the system, whether microcode in BIOS is already up to date, how big the microcode patch is and how long the INIT/SIPI response time is. On an AMD Zen3 machine INIT/SIPI response time is amazingly fast (350us), but then it lacks TSC_ADJUST and does a two millisecond TSC sync test for _every_ AP. All of this sucks... Possible further enhancements ----------------------------- It's definitely worthwhile to look into reducing the cross socket TSC sync test time. It's probably safe enough to use 5ms or even 2ms instead of 20ms on systems with TSC_ADJUST and a few other 'TSC is sane' indicators. Moving it out of the hotplug path is eventually possible, but that needs some deep thoughts. Let's take the TSC sync out of the picture by adding 'tsc=reliable" to the kernel command line. So the bringup of 111 APs takes: total avg/CPU min max prepare: 38.936 0.351 0.231 0.874 online : 25.231 0.227 0.169 3.465 total : 64.167 0.578 0.400 4.339 Some of the outliers are not necessarily in the state callbacks as the system is already scheduling and handles interrupts and soft interrupts. Haven't analyzed that yet in detail. In the prepare stage which runs on the control CPU the larger steps are: smpcfd:prepare 16us avg/CPU threads:prepare 98us avg/CPU workqueue:prepare 43us avg/CPU trace/RB:prepare 135us avg/CPU The trace ringbuffer initialization allocates 354 pages and 354 control structures one by one. That probably should allocate a large page and an array of control structures and work from there. I'm sure that would reduce this significantly. Steven? smpcfd does just a percpu allocation. No idea why that takes that long. Vs. threads and workqueues. David thought about spreading out the preparation work and do it really in parallel. That's a nice idea, but the threads and workqueue prepare steps are self serializing. The workqueue one has a global mutex and aside of that both steps create kernel threads which implicitely serialize on kthreadd. alloc_percpu(), which is used by smpcfd:prepare is also globally serialized. The rest of the prepare steps is pretty much in the single digit microseconds range. On the AP side it should be possible to move some of the initialization steps before the alive synchronization point, but that really needs a lot of analysis whether the functions are safe to invoke that early and outside of the cpu_hotplug_lock held region for the case of two stage parallel bringup; see below. The largest part is: identify_secondary_cpu() 99us avg/CPU Inside of identify_secondary_cpu() the largest offender: mcheck_init() 73us avg/CPU This part is definitly worth to be looked at whether it can be at least partially moved to the early startup code before the alive synchronization point. There's a lot of deep analysis required and ideally we just rewrite the whole CPUID evaluation trainwreck completely. The rest of the AP side is low single digit microseconds except of: perf/x86:starting 14us avg/CPU smpboot/threads:online 13us avg/CPU workqueue:online 17us avg/CPU mm/vmstat:online 17us avg/CPU sched:active 30us avg/CPU sched:active is special. Onlining the first secondary HT thread on the second socket creates a 3.2ms outlier which skews the whole picture. That's caused by enabling the static key sched_smt_present which patches the world and some more. For all other APs this is really in the 1us range. This definitely could be postponed during bootup like the scheduler domain rebuild is done after the bringup. But that's still fully serialized and single threaded and obviously could be done later in the context of async parallel init. It's unclear why this is different with the fully serialized bringup where it takes significantly less time, but that's something which needs to be investigated. Is truly parallel bringup feasible? ----------------------------------- In theory yes, realistically no. Why? 1) The preparation phase Allocating memory, creating threads for the to be brought up CPU must obviously happen on an already online CPU. While it would be possible to bring up a subset of CPUs first and let them do the preparation steps for groups of still offline CPUs concurrently, the actual benefit of doing so is dubious. The prime example is kernel thread creation, which is implicitely serialized on kthreadd. A simple experiment shows that 4 concurrent workers on 4 different CPUs where each is creating 14 * 5 = 70 kernel threads are 5% slower than a single worker creating 4 * 14 * 5 = 280 threads. So we'd need to have multiple kthreadd instances to handle that, which would then serialize on tasklist lock and other things. That aside the preparation phase is also affected by the problem below. 2) Assumptions about hotplug serialization a) There are quite some assumptions about CPU bringup being fully serialized across state transitions. A lot of state callbacks rely on that and would require local locking. Adding that local locking is surely possible, but that has several downsides: - It adds complexity and makes it harder for developers to get this correct. The subtle bugs resulting out of that are going to be interesting - Fine grained locking has a charm, but only if the time spent for the actual work is larger than the time required for serialization and synchronization. Serializing a callback which takes less than a microsecond and then having a large number of CPUs contending on the lock will not make it any faster at all. That's a well known issue of parallelizing and neither made up nor kernel specific. b) Some operations definitely require to be protected by the cpu_hotplug_lock, especially those which affect cpumasks as the masks are guaranteed to be stable in a cpus_read_lock()'ed region. As this lock cannot be taken in atomic contexts, it's required that the control CPU holds the lock write locked across these state transitions. And no, we are not making this a spinlock just for that and we even can't. Just slapping a lock into the x86 specific part of the cpumask update function does not solve anything. The relevant patch in V17 is completely useless as it only serializes the actual cpumask/map modifications, but all read side users are hosed if the update would be moved before the alive synchronization point, i.e. into a non hotplug lock protected region. Even if the hotplug lock would be held accross the whole parallel bringup operation then this would still expose all usage of these masks and maps in the actual hotplug state callbacks to concurrent modifications. And no, we are not going to expose an architecture specific raw spinlock to the hotplug state callbacks, especially not to those in generic code. c) Some cpu_read_lock()'ed regions also expect that there is no CPU state transition happening which would modify their local state. This would again require local serialization. 3) The amount of work and churn: - Analyze the per architecture low level startup functions plus their descendant functions and make them ready for concurrency if necessary. - Analyze ~300 hotplug state callbacks and their descendant functions and make them ready for concurrency if necessary. - Analyze all cpus_read_lock()'ed regions and address their requirements. - Rewrite the core code to handle the cpu_hotplug_lock requirements only in distinct phases of the state machine. - Rewrite the core code to handle state callback failure and the related rollback in the context of the new rules. - ... Even if some people are dedicated enough to do that, it's very questionable whether the resulting complexity is justified. We've spent a serious amount of time to sanitize hotplug and bring it into a state where it is correct. This also made it reasonably simple for developers to implement hotplug state callbacks without having to become hotplug experts. Breaking this completely up will result in a flood of hard to diagnose subtle issues for sure. Who is going to deal with them? The experience with this series so far does not make me comfortable about that thought in any way. Summary ------- The obvious and low hanging fruits have to be solved first: - The CPUID evaluation and related setup mechanisms - The trace/ringbuffer oddity - The sched:active oddity for the first sibling on the second socket - Some other expensive things which I'm not seeing in my test setup due to lack of hardware or configuration. Anything else is pretty much wishful thinking in my opinion. To be clear. I'm not standing in the way if there is a proper solution, but that requires to respect the basic engineering rules: 1) Correctness first 2) Keep it maintainable 3) Keep it simple So far this stuff failed already at #1. I completely understand why this is important for cloud people, but the real question to ask here is what are the actual requirements. As far as I understand the main goal is to make a (kexec) reboot almost invisible to VM tenants. Now lets look at how this works: A) Freeze VMs and persist state B) kexec into the new kernel C) Restore VMs from persistant memory D) Thaw VMs So the key problem is how long it takes to get from #B to #C and finally to #D. As far as I understand #C takes a serious amount of time and cannot be parallelized for whatever reasons. At the same time the number of online CPUs required to restore the VMs state is less than the number of online CPUs required to actually operate them in #D. That means it would be good enough to return to userspace with a limited number of online CPUs as fast as possible. A certain amount of CPUs are going to be busy with restoring the VMs state, i.e. one CPU per VM. Some remaining non-busy CPU can bringup the rest of the system and the APs in order to be functional for #D, i.e the restore of VM operation. Trying to optimize this purely in kernel space by adding complexity of dubious value is simply bogus in my opinion. It's already possible today to limit the number of CPUs which are initially onlined and online the rest later from user space. There are two issue there: a) The death by MCE broadcast problem Quite some (contemporary) x86 CPU generations are affected by this: - MCE can be broadcasted to all CPUs and not only issued locally to the CPU which triggered it. - Any CPU which has CR4.MCE == 0, even if it sits in a wait for INIT/SIPI state, will cause an immediate shutdown of the machine if a broadcasted MCE is delivered. b) Do the parallel bringup via sysfs control knob The per CPU target state interface allows to do that today one by one, but it's akward and has quite some overhead. A knob to online the rest of the not yet onlined present CPUs with the benefit of the parallel bringup mechanism is missing. #a) That's a risk to take by the operator. Even the regular serialized bringup does not protect against this issue up to the point where all present CPUs have at least initialized CR4. Limiting the number of APs to online early via the kernel command line widens that window and increases the risk further by executing user space before all APs have CR4 initialized. But the same applies to a deferred online mechanism implemented in the kernel where some worker brings up the not yet online APs while the early online CPUs are already executing user space code. #b) Is a no brainer to implement on top of this. Conclusion ---------- Adding the basic parallel bringup mechanism as provided by this series makes a lot of sense. Improving particular issues as pointed out in the analysis makes sense too. But trying to solve an application specific problem fully in the kernel with tons of complexity, without exploring straight forward and simple approaches first, does not make any sense at all. Thanks, tglx --- Documentation/admin-guide/kernel-parameters.txt | 20 Documentation/core-api/cpu_hotplug.rst | 13 arch/Kconfig | 23 + arch/arm/Kconfig | 1 arch/arm/include/asm/smp.h | 2 arch/arm/kernel/smp.c | 18 arch/arm64/Kconfig | 1 arch/arm64/include/asm/smp.h | 2 arch/arm64/kernel/smp.c | 14 arch/csky/Kconfig | 1 arch/csky/include/asm/smp.h | 2 arch/csky/kernel/smp.c | 8 arch/mips/Kconfig | 1 arch/mips/cavium-octeon/smp.c | 1 arch/mips/include/asm/smp-ops.h | 1 arch/mips/kernel/smp-bmips.c | 1 arch/mips/kernel/smp-cps.c | 14 arch/mips/kernel/smp.c | 8 arch/mips/loongson64/smp.c | 1 arch/parisc/Kconfig | 1 arch/parisc/kernel/process.c | 4 arch/parisc/kernel/smp.c | 7 arch/riscv/Kconfig | 1 arch/riscv/include/asm/smp.h | 2 arch/riscv/kernel/cpu-hotplug.c | 14 arch/x86/Kconfig | 45 -- arch/x86/include/asm/apic.h | 5 arch/x86/include/asm/cpu.h | 5 arch/x86/include/asm/cpumask.h | 5 arch/x86/include/asm/processor.h | 1 arch/x86/include/asm/realmode.h | 3 arch/x86/include/asm/sev-common.h | 3 arch/x86/include/asm/smp.h | 26 - arch/x86/include/asm/topology.h | 23 - arch/x86/include/asm/tsc.h | 2 arch/x86/kernel/acpi/sleep.c | 9 arch/x86/kernel/apic/apic.c | 22 - arch/x86/kernel/callthunks.c | 4 arch/x86/kernel/cpu/amd.c | 2 arch/x86/kernel/cpu/cacheinfo.c | 21 arch/x86/kernel/cpu/common.c | 50 -- arch/x86/kernel/cpu/topology.c | 3 arch/x86/kernel/head_32.S | 14 arch/x86/kernel/head_64.S | 121 +++++ arch/x86/kernel/sev.c | 2 arch/x86/kernel/smp.c | 3 arch/x86/kernel/smpboot.c | 508 ++++++++---------------- arch/x86/kernel/topology.c | 98 ---- arch/x86/kernel/tsc.c | 20 arch/x86/kernel/tsc_sync.c | 36 - arch/x86/power/cpu.c | 37 - arch/x86/realmode/init.c | 3 arch/x86/realmode/rm/trampoline_64.S | 27 + arch/x86/xen/enlighten_hvm.c | 11 arch/x86/xen/smp_hvm.c | 16 arch/x86/xen/smp_pv.c | 56 +- drivers/acpi/processor_idle.c | 4 include/linux/cpu.h | 4 include/linux/cpuhotplug.h | 17 kernel/cpu.c | 397 +++++++++++++++++- kernel/smp.c | 2 kernel/smpboot.c | 163 ------- 62 files changed, 953 insertions(+), 976 deletions(-)