| Message ID | 20240203093054.412135-1-zhao1.liu@linux.intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Intel Thread Director Virtualization Support in QEMU | expand |
On Sat, Feb 03, 2024 at 05:30:48PM +0800, Zhao Liu wrote: > Date: Sat, 3 Feb 2024 17:30:48 +0800 > From: Zhao Liu <zhao1.liu@linux.intel.com> > Subject: [RFC 0/6] Intel Thread Director Virtualization Support in QEMU > X-Mailer: git-send-email 2.34.1 > > From: Zhao Liu <zhao1.liu@intel.com> > > Hi list, > > This is our refreshed RFC to support our ITD virtualization patch > series [1] in KVM, and bases on bd2e12310b18 ("Merge tag > 'qga-pull-2024-01-30' of https://github.com/kostyanf14/qemu into > staging"). > > ITD is Intel's client specific feature to optimize scheduling on Intel > hybrid platforms. Though this feature depends on hybrid topology s/depends/doesn't depend/ Regards, Zhao > details, in our parctice on Win11 Guest, ITD works with hyrbid topolohy > and CPU affinity can achieve the most performance improvement in Win11 > Guest (for example, on i9-13900K, up to 14%+ improvement on > 3DMARK). More data or details, can be found in [1]. Thus, the ITD for > Win11 is also a typical use case of hybrid topology. > > > Welcome your feedback! > > > 1. Background and Motivation > ============================ > > ITD allows the hardware to provide scheduling hints to the OS to help > optimize scheduling performance, and under the Intel hybrid > architecture, since Core and Atom have different capabilities > (performance, energy effency, etc.), scheduling based on hardware > hints can take full advantage of this hybrid architecture. This is also > the most ideal scheduling model for intel hybrid architecture. > > Therefore, we want to virtualize the ITD feature so that ITD can benefit > performance of the virtual machines on the hybrid machines as well. > > Currently, our ITD virtualization is a software virtualization solution. > > > 2. Introduction to HFI and ITD > ============================== > > Intel provides Hardware Feedback Interface (HFI) feature to allow > hardware to provide guidance to the OS scheduler to perform optimal > workload scheduling through a hardware feedback interface structure in > memory [2]. This hfi structure is called HFI table. > > As for now, the guidance includes performance and energy enficency hints, > and it could update via thermal interrupt as the actual operating > conditions of the processor change during run time. > > And Intel Thread Director (ITD) feature extends the HFI to provide > performance and energy efficiency data for advanced classes of > instructions. > > The virtual HFI table is maintained in KVM, and for QEMU, we just need > to handle HFI/ITD/HRESET (and their dependent features: ACPI, TM and > PTS) related CPUIDs and MSRs. > > > 3. Package level MSRs handling > ============================== > > PTS, HFI and ITD are all have package level features, such as package > level MSRs and package level HFI tables. But since KVM hasn't > support msr-topology and it just handle these package-level MSRs and > HFI table at VM level, in order to avoid potential contention problems > caused by multiple virtual-packages, we restrict VMs to be able to > enable PTC/HFI/ITD iff there's only 1 package (and only 1 die for > ITD/HFI). > > > 4. HFI/ITD related info in CPUID > ================================ > > KVM provides some basic HFI info in CPUID.0x06 leaf, which is associated > with the virtual HFI table in KVM. > > QEMU should configure HFI table index for each vCPU. Here we set the HFI > table index to vCPU index so that different vCPUs have different HFI > entries to avoid unnecessary competition problems. > > > 5. Compatibility issues > ======================= > > HFI is supported in both server (SPR) and client (ADL/RPL/MTL) platform > products while ITD is the client specific feature. > > For client platform, ITD (with HFI) could be enabled in Guest to improve > scheduling, but for server platform, HFI (without ITD) is only useful > on Host and Guest doesn't need it. > > To simplify the enabling logic and avoid impacting the common topology > of the Guest, we set PTS, HFI, and ITD as feature bits that are not > automatically enabled. > > Only when the user actively specifies these features, QEMU will check > and decide whether to enable them based on the topology constraints and > the ITD constraints. > > > 6. New option "enable-itd" > ============================ > > ITD-related features include PTS, HFI, ITD, and HRESET. > > To make it easier for users to enable ITD for Guest without specifying > the above feature bits one by one, we provide a new option "enable-itd" > to set the above feature bits for Guest all at once. > > "enable-itd" does not guarantee that ITD will be enabled for Guest. > The success of enabling ITD for guest depends on topology constraints, > platform support, etc., which are checked in QEMU. > > > 7. Patch Summary > ================ > > Patch 1: Add support save/load for ACPI feature related thermal MSRs > since ACPI feature CPUID has been added in QEMU. > Patch 2: Add support for PTS (package) thermal MSRs and its CPUID > Patch 3: Add support for HFI MSRs and its CPUID > Patch 4: Add support ITD CPUID and MSR_IA32_HW_FEEDBACK_THREAD_CONFIG. > Patch 5: Add support HRESET CPUID and MSR_IA32_HW_HRESET_ENABLE. > Patch 6: Add "enable-itd" to help user set ITD related feature bits. > > # 8. References > > [1]: KVM RFC: [RFC 00/26] Intel Thread Director Virtualization > https://lore.kernel.org/kvm/20240203091214.411862-1-zhao1.liu@linux.intel.com/T/#t > [2]: SDM, vol. 3B, section 15.6 HARDWARE FEEDBACK INTERFACE AND INTEL > THREAD DIRECTOR > > Thanks and Best Regards, > Zhao > --- > Zhao Liu (2): > target/i386: Add support for Intel Thread Director feature > i386: Add a new property to set ITD related feature bits for Guest > > Zhuocheng Ding (4): > target/i386: Add support for save/load of ACPI thermal MSRs > target/i386: Add support for Package Thermal Management feature > target/i386: Add support for Hardware Feedback Interface feature > target/i386: Add support for HRESET feature > > target/i386/cpu.c | 108 ++++++++++++++++++++++++++++++++++++++++-- > target/i386/cpu.h | 37 +++++++++++++++ > target/i386/kvm/kvm.c | 84 ++++++++++++++++++++++++++++++++ > 3 files changed, 225 insertions(+), 4 deletions(-) > > -- > 2.34.1 >
From: Zhao Liu <zhao1.liu@intel.com> Hi list, This is our refreshed RFC to support our ITD virtualization patch series [1] in KVM, and bases on bd2e12310b18 ("Merge tag 'qga-pull-2024-01-30' of https://github.com/kostyanf14/qemu into staging"). ITD is Intel's client specific feature to optimize scheduling on Intel hybrid platforms. Though this feature depends on hybrid topology details, in our parctice on Win11 Guest, ITD works with hyrbid topolohy and CPU affinity can achieve the most performance improvement in Win11 Guest (for example, on i9-13900K, up to 14%+ improvement on 3DMARK). More data or details, can be found in [1]. Thus, the ITD for Win11 is also a typical use case of hybrid topology. Welcome your feedback! 1. Background and Motivation ============================ ITD allows the hardware to provide scheduling hints to the OS to help optimize scheduling performance, and under the Intel hybrid architecture, since Core and Atom have different capabilities (performance, energy effency, etc.), scheduling based on hardware hints can take full advantage of this hybrid architecture. This is also the most ideal scheduling model for intel hybrid architecture. Therefore, we want to virtualize the ITD feature so that ITD can benefit performance of the virtual machines on the hybrid machines as well. Currently, our ITD virtualization is a software virtualization solution. 2. Introduction to HFI and ITD ============================== Intel provides Hardware Feedback Interface (HFI) feature to allow hardware to provide guidance to the OS scheduler to perform optimal workload scheduling through a hardware feedback interface structure in memory [2]. This hfi structure is called HFI table. As for now, the guidance includes performance and energy enficency hints, and it could update via thermal interrupt as the actual operating conditions of the processor change during run time. And Intel Thread Director (ITD) feature extends the HFI to provide performance and energy efficiency data for advanced classes of instructions. The virtual HFI table is maintained in KVM, and for QEMU, we just need to handle HFI/ITD/HRESET (and their dependent features: ACPI, TM and PTS) related CPUIDs and MSRs. 3. Package level MSRs handling ============================== PTS, HFI and ITD are all have package level features, such as package level MSRs and package level HFI tables. But since KVM hasn't support msr-topology and it just handle these package-level MSRs and HFI table at VM level, in order to avoid potential contention problems caused by multiple virtual-packages, we restrict VMs to be able to enable PTC/HFI/ITD iff there's only 1 package (and only 1 die for ITD/HFI). 4. HFI/ITD related info in CPUID ================================ KVM provides some basic HFI info in CPUID.0x06 leaf, which is associated with the virtual HFI table in KVM. QEMU should configure HFI table index for each vCPU. Here we set the HFI table index to vCPU index so that different vCPUs have different HFI entries to avoid unnecessary competition problems. 5. Compatibility issues ======================= HFI is supported in both server (SPR) and client (ADL/RPL/MTL) platform products while ITD is the client specific feature. For client platform, ITD (with HFI) could be enabled in Guest to improve scheduling, but for server platform, HFI (without ITD) is only useful on Host and Guest doesn't need it. To simplify the enabling logic and avoid impacting the common topology of the Guest, we set PTS, HFI, and ITD as feature bits that are not automatically enabled. Only when the user actively specifies these features, QEMU will check and decide whether to enable them based on the topology constraints and the ITD constraints. 6. New option "enable-itd" ============================ ITD-related features include PTS, HFI, ITD, and HRESET. To make it easier for users to enable ITD for Guest without specifying the above feature bits one by one, we provide a new option "enable-itd" to set the above feature bits for Guest all at once. "enable-itd" does not guarantee that ITD will be enabled for Guest. The success of enabling ITD for guest depends on topology constraints, platform support, etc., which are checked in QEMU. 7. Patch Summary ================ Patch 1: Add support save/load for ACPI feature related thermal MSRs since ACPI feature CPUID has been added in QEMU. Patch 2: Add support for PTS (package) thermal MSRs and its CPUID Patch 3: Add support for HFI MSRs and its CPUID Patch 4: Add support ITD CPUID and MSR_IA32_HW_FEEDBACK_THREAD_CONFIG. Patch 5: Add support HRESET CPUID and MSR_IA32_HW_HRESET_ENABLE. Patch 6: Add "enable-itd" to help user set ITD related feature bits. # 8. References [1]: KVM RFC: [RFC 00/26] Intel Thread Director Virtualization https://lore.kernel.org/kvm/20240203091214.411862-1-zhao1.liu@linux.intel.com/T/#t [2]: SDM, vol. 3B, section 15.6 HARDWARE FEEDBACK INTERFACE AND INTEL THREAD DIRECTOR Thanks and Best Regards, Zhao --- Zhao Liu (2): target/i386: Add support for Intel Thread Director feature i386: Add a new property to set ITD related feature bits for Guest Zhuocheng Ding (4): target/i386: Add support for save/load of ACPI thermal MSRs target/i386: Add support for Package Thermal Management feature target/i386: Add support for Hardware Feedback Interface feature target/i386: Add support for HRESET feature target/i386/cpu.c | 108 ++++++++++++++++++++++++++++++++++++++++-- target/i386/cpu.h | 37 +++++++++++++++ target/i386/kvm/kvm.c | 84 ++++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+), 4 deletions(-)