Message ID | 5ef38d72-3a80-e2ef-68d9-ec498866d4eb@citrix.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | livepatch-build-tools regression | expand |
> On 18. Nov 2019, at 17:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: > > Hello, > > Trying to build a simple version of XSA-304 Live-Patch for 4.13 gives > the following error during LP upload: > > (XEN) livepatch: lp: Unknown symbol: .LC7 > > Bisecting identified the first bad commit as: > > commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections" > > Base version of Xen used for this experiment is d13dfb02aafab > The patch file used for LP is attached. > > -- > Thanks, > Sergey > <0001-live-patch.patch> Could you give this a try? https://patchwork.kernel.org/patch/11228191/ https://patchwork.kernel.org/patch/11228189/ Best Regards, Pawel Wieczorkiewicz Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
On 18/11/2019 16:47, Wieczorkiewicz, Pawel wrote: > > >> On 18. Nov 2019, at 17:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: >> >> Hello, >> >> Trying to build a simple version of XSA-304 Live-Patch for 4.13 gives >> the following error during LP upload: >> >> (XEN) livepatch: lp: Unknown symbol: .LC7 >> >> Bisecting identified the first bad commit as: >> >> commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections" >> >> Base version of Xen used for this experiment is d13dfb02aafab >> The patch file used for LP is attached. >> >> -- >> Thanks, >> Sergey >> <0001-live-patch.patch> > > Could you give this a try? > > https://patchwork.kernel.org/patch/11228191/ > https://patchwork.kernel.org/patch/11228189/ Unfortunately, those patches didn't resolve the issue for me. Forgot to add, my gcc version is gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 -- Thanks, Sergey
> On 18. Nov 2019, at 18:09, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: > > On 18/11/2019 16:47, Wieczorkiewicz, Pawel wrote: >> >> >>> On 18. Nov 2019, at 17:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: >>> >>> Hello, >>> >>> Trying to build a simple version of XSA-304 Live-Patch for 4.13 gives >>> the following error during LP upload: >>> >>> (XEN) livepatch: lp: Unknown symbol: .LC7 >>> >>> Bisecting identified the first bad commit as: >>> >>> commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections" >>> >>> Base version of Xen used for this experiment is d13dfb02aafab >>> The patch file used for LP is attached. >>> >>> -- >>> Thanks, >>> Sergey >>> <0001-live-patch.patch> >> >> Could you give this a try? >> >> https://patchwork.kernel.org/patch/11228191/ >> https://patchwork.kernel.org/patch/11228189/ > > Unfortunately, those patches didn't resolve the issue for me. > > Forgot to add, my gcc version is > > gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 Could you build the lp with debug (-d) and provide me with the create-diff-object.log file? > > -- > Thanks, > Sergey Best Regards, Pawel Wieczorkiewicz Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote: > > Could you build the lp with debug (-d) and provide me with the create-diff-object.log file? > I've attached the log. Btw, I think I provided all the necessary information for others to repeat my experiment. -- Thanks, Sergey
> On 18. Nov 2019, at 18:41, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: > > On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote: >> >> Could you build the lp with debug (-d) and provide me with the create-diff-object.log file? >> > > I've attached the log. Btw, I think I provided all the necessary information > for others to repeat my experiment. > Sorry for another request, but I do not seem to be able to reproduce this locally. Could you send me the livepatch module binary that fails to upload? > -- > Thanks, > Sergey > <create-diff-object.log.xz> Best Regards, Pawel Wieczorkiewicz Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
On 19/11/2019 17:21, Wieczorkiewicz, Pawel wrote: > > >> On 18. Nov 2019, at 18:41, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: >> >> On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote: >>> >>> Could you build the lp with debug (-d) and provide me with the create-diff-object.log file? >>> >> >> I've attached the log. Btw, I think I provided all the necessary information >> for others to repeat my experiment. >> > > Sorry for another request, but I do not seem to be able to reproduce this locally. > Could you send me the livepatch module binary that fails to upload? That's interesting. I've attached the binary that my system produces. What version of gcc do you use? -- Thanks, Sergey
> On 20. Nov 2019, at 12:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: > > On 19/11/2019 17:21, Wieczorkiewicz, Pawel wrote: >> >> >>> On 18. Nov 2019, at 18:41, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: >>> >>> On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote: >>>> >>>> Could you build the lp with debug (-d) and provide me with the create-diff-object.log file? >>>> >>> >>> I've attached the log. Btw, I think I provided all the necessary information >>> for others to repeat my experiment. >>> >> >> Sorry for another request, but I do not seem to be able to reproduce this locally. >> Could you send me the livepatch module binary that fails to upload? > > That's interesting. I've attached the binary that my system produces. > What version of gcc do you use? The version used was: gcc (GCC) 7.2.1 20170915 But I have finally managed to reproduce the issue with: 1. gcc (Ubuntu 6.5.0-2ubuntu1~18.04) 6.5.0 20181026 2. gcc-7 (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 I think it is not related to the commit: commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections" I managed to reproduce it also with earlier version commit: "0c10457 Remove section alignment requirement" But this time a different symbol causes the failure: (XEN) livepatch: 0001-live-patch: Unknown symbol: hvm.c#lastpage.22856 > > -- > Thanks, > Sergey > <0001-live-patch-stripped.livepatch> Best Regards, Pawel Wieczorkiewicz Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
It looks like gcc plays the usual dirty tricks with local variables renaming: - xen-syms 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857 - livepatch 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856 Then, symbols resolution by name fails.. Can you please try to build the livepatch module with additional option '—prelink' and give it a try ? > On 26. Nov 2019, at 18:51, Wieczorkiewicz, Pawel <wipawel@amazon.de> wrote: > > > >> On 20. Nov 2019, at 12:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: >> >> On 19/11/2019 17:21, Wieczorkiewicz, Pawel wrote: >>> >>> >>>> On 18. Nov 2019, at 18:41, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: >>>> >>>> On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote: >>>>> >>>>> Could you build the lp with debug (-d) and provide me with the create-diff-object.log file? >>>>> >>>> >>>> I've attached the log. Btw, I think I provided all the necessary information >>>> for others to repeat my experiment. >>>> >>> >>> Sorry for another request, but I do not seem to be able to reproduce this locally. >>> Could you send me the livepatch module binary that fails to upload? >> >> That's interesting. I've attached the binary that my system produces. >> What version of gcc do you use? > > The version used was: gcc (GCC) 7.2.1 20170915 > > But I have finally managed to reproduce the issue with: > 1. gcc (Ubuntu 6.5.0-2ubuntu1~18.04) 6.5.0 20181026 > 2. gcc-7 (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 > > I think it is not related to the commit: > commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections" > > I managed to reproduce it also with earlier version commit: > "0c10457 Remove section alignment requirement" > > But this time a different symbol causes the failure: > > (XEN) livepatch: 0001-live-patch: Unknown symbol: hvm.c#lastpage.22856 > >> >> -- >> Thanks, >> Sergey >> <0001-live-patch-stripped.livepatch> > > Best Regards, > Pawel Wieczorkiewicz Best Regards, Pawel Wieczorkiewicz Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
On 26/11/2019 18:37, Wieczorkiewicz, Pawel wrote: > It looks like gcc plays the usual dirty tricks with local variables renaming: > > - xen-syms > 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857 > - livepatch > 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856 > > Then, symbols resolution by name fails.. > > Can you please try to build the livepatch module with additional option '—prelink' and give it a try ? My LP loading error is: (XEN) livepatch: lp: Unknown symbol: .LC7 When I pass --prelink to livepatch-build, it complains in a similar way: livepatch-build-tools/prelink: ERROR: output.o: livepatch_resolve_symbols: 80: lookup_local_symbol .LC7 (p2m.c) -- Thanks, Sergey
> On 27. Nov 2019, at 12:16, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: > > On 26/11/2019 18:37, Wieczorkiewicz, Pawel wrote: >> It looks like gcc plays the usual dirty tricks with local variables renaming: >> >> - xen-syms >> 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857 >> - livepatch >> 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856 >> >> Then, symbols resolution by name fails.. >> >> Can you please try to build the livepatch module with additional option '—prelink' and give it a try ? > > My LP loading error is: > > (XEN) livepatch: lp: Unknown symbol: .LC7 > > When I pass --prelink to livepatch-build, it complains in a similar way: > > livepatch-build-tools/prelink: ERROR: output.o: livepatch_resolve_symbols: 80: lookup_local_symbol .LC7 (p2m.c) > Could you give this testing patch a try? diff --git a/create-diff-object.c b/create-diff-object.c index 8d63940..10807d2 100644 --- a/create-diff-object.c +++ b/create-diff-object.c @@ -839,8 +839,10 @@ static void kpatch_compare_symbols(struct list_head *symlist) list_for_each_entry(sym, symlist, list) { if (sym->twin) kpatch_compare_correlated_symbol(sym); - else + else { sym->status = NEW; + sym->include = 1; + } log_debug("symbol %s is %s\n", sym->name, status_str(sym->status)); } > -- > Thanks, > Sergey Best Regards, Pawel Wieczorkiewicz Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
On 27/11/2019 15:22, Wieczorkiewicz, Pawel wrote: > > >> On 27. Nov 2019, at 12:16, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: >> >> On 26/11/2019 18:37, Wieczorkiewicz, Pawel wrote: >>> It looks like gcc plays the usual dirty tricks with local variables renaming: >>> >>> - xen-syms >>> 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857 >>> - livepatch >>> 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856 >>> >>> Then, symbols resolution by name fails.. >>> >>> Can you please try to build the livepatch module with additional option '—prelink' and give it a try ? >> >> My LP loading error is: >> >> (XEN) livepatch: lp: Unknown symbol: .LC7 >> >> When I pass --prelink to livepatch-build, it complains in a similar way: >> >> livepatch-build-tools/prelink: ERROR: output.o: livepatch_resolve_symbols: 80: lookup_local_symbol .LC7 (p2m.c) >> > > Could you give this testing patch a try? > > diff --git a/create-diff-object.c b/create-diff-object.c > index 8d63940..10807d2 100644 > --- a/create-diff-object.c > +++ b/create-diff-object.c > @@ -839,8 +839,10 @@ static void kpatch_compare_symbols(struct list_head *symlist) > list_for_each_entry(sym, symlist, list) { > if (sym->twin) > kpatch_compare_correlated_symbol(sym); > - else > + else { > sym->status = NEW; > + sym->include = 1; > + } > > log_debug("symbol %s is %s\n", sym->name, status_str(sym->status)); > } > Looks like this change fixed the issue for me! One thing to notice is that the size of a stripped LP binary increased from 45K to 60K. -- Thanks, Sergey
> On 27. Nov 2019, at 16:56, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: > > On 27/11/2019 15:22, Wieczorkiewicz, Pawel wrote: >> >> >>> On 27. Nov 2019, at 12:16, Sergey Dyasli <sergey.dyasli@citrix.com> wrote: >>> >>> On 26/11/2019 18:37, Wieczorkiewicz, Pawel wrote: >>>> It looks like gcc plays the usual dirty tricks with local variables renaming: >>>> >>>> - xen-syms >>>> 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857 >>>> - livepatch >>>> 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856 >>>> >>>> Then, symbols resolution by name fails.. >>>> >>>> Can you please try to build the livepatch module with additional option '—prelink' and give it a try ? >>> >>> My LP loading error is: >>> >>> (XEN) livepatch: lp: Unknown symbol: .LC7 >>> >>> When I pass --prelink to livepatch-build, it complains in a similar way: >>> >>> livepatch-build-tools/prelink: ERROR: output.o: livepatch_resolve_symbols: 80: lookup_local_symbol .LC7 (p2m.c) >>> >> >> Could you give this testing patch a try? >> >> diff --git a/create-diff-object.c b/create-diff-object.c >> index 8d63940..10807d2 100644 >> --- a/create-diff-object.c >> +++ b/create-diff-object.c >> @@ -839,8 +839,10 @@ static void kpatch_compare_symbols(struct list_head *symlist) >> list_for_each_entry(sym, symlist, list) { >> if (sym->twin) >> kpatch_compare_correlated_symbol(sym); >> - else >> + else { >> sym->status = NEW; >> + sym->include = 1; >> + } >> >> log_debug("symbol %s is %s\n", sym->name, status_str(sym->status)); >> } >> > > Looks like this change fixed the issue for me! > One thing to notice is that the size of a stripped LP binary increased > from 45K to 60K. > Yes, this was not supposed to be a proper fix. I was merely trying to establish if we were looking at the same issue. I did fix it now the proper way, though. I shall send the patches soon. > -- > Thanks, > Sergey Best Regards, Pawel Wieczorkiewicz Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 451d213c8c..5e427a1cf8 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -918,7 +918,7 @@ Controls for interacting with the system Extended Firmware Interface. uncacheable. ### ept -> `= List of [ ad=<bool>, pml=<bool> ]` +> `= List of [ ad=<bool>, pml=<bool>, exec-sp=<bool> ]` > Applicability: Intel @@ -949,6 +949,31 @@ introduced with the Nehalem architecture. disable PML. `pml=0` can be used to prevent the use of PML on otherwise capable hardware. +* The `exec-sp` boolean controls whether EPT superpages with execute + permissions are permitted. In general this is good for performance. + + However, on processors vulnerable CVE-2018-12207, HVM guest kernels can + use executable superpages to crash the host. By default, executable + superpages are disabled on affected hardware. + + If HVM guest kernels are trusted not to mount a DoS against the system, + this option can enabled to regain performance. + + This boolean may be modified at runtime using `xl set-parameters + ept=[no-]exec-sp` to switch between fast and secure. + + * When switching from secure to fast, preexisting HVM domains will run + at their current performance until they are rebooted; new domains will + run without any overhead. + + * When switching from fast to secure, all HVM domains will immediately + suffer a performance penalty. + + **Warning: No guarantee is made that this runtime option will be retained + indefinitely, or that it will retain this exact behaviour. It is + intended as an emergency option for people who first chose fast, then + change their minds to secure, and wish not to reboot.** + ### extra_guest_irqs > `= [<domU number>][,<dom0 number>]` diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 06a7b40107..818e705fd1 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -1833,6 +1833,24 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla, break; } + /* + * Workaround for XSA-304 / CVE-2018-12207. If we take an execution + * fault against a non-executable superpage, shatter it to regain + * execute permissions. + */ + if ( page_order > 0 && npfec.insn_fetch && npfec.present && !violation ) + { + int res = p2m_set_entry(p2m, _gfn(gfn), mfn, PAGE_ORDER_4K, + p2mt, p2ma); + + if ( res ) + printk(XENLOG_ERR "Failed to shatter gfn %"PRI_gfn": %d\n", + gfn, res); + + rc = !res; + goto out_put_gfn; + } + if ( violation ) { /* Should #VE be emulated for this fault? */ diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index ed27e8def7..668986eeaa 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -67,6 +67,7 @@ integer_param("ple_window", ple_window); static bool __read_mostly opt_ept_pml = true; static s8 __read_mostly opt_ept_ad = -1; +int8_t opt_ept_exec_sp = -1; static int __init parse_ept_param(const char *s) { @@ -92,6 +93,40 @@ static int __init parse_ept_param(const char *s) } custom_param("ept", parse_ept_param); +int parse_ept_param_runtime(const char *s) +{ + int val; + + if ( !cpu_has_vmx_ept || !hvm_funcs.hap_supported || + !(hvm_funcs.hap_capabilities & + (HVM_HAP_SUPERPAGE_2MB | HVM_HAP_SUPERPAGE_1GB)) ) + { + printk("VMX: EPT not available, or not in use - ignoring\n"); + return 0; + } + + if ( (val = parse_boolean("exec-sp", s, NULL)) < 0 ) + return -EINVAL; + + if ( val != opt_ept_exec_sp ) + { + struct domain *d; + + opt_ept_exec_sp = val; + + rcu_read_lock(&domlist_read_lock); + for_each_domain ( d ) + if ( paging_mode_hap(d) ) + p2m_change_entry_type_global(d, p2m_ram_rw, p2m_ram_rw); + rcu_read_unlock(&domlist_read_lock); + } + + printk("VMX: EPT executable superpages %sabled\n", + val ? "en" : "dis"); + + return 0; +} + /* Dynamic (run-time adjusted) execution control flags. */ u32 vmx_pin_based_exec_control __read_mostly; u32 vmx_cpu_based_exec_control __read_mostly; diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c index 220990f017..f06e51904a 100644 --- a/xen/arch/x86/mm/p2m-ept.c +++ b/xen/arch/x86/mm/p2m-ept.c @@ -174,6 +174,12 @@ static void ept_p2m_type_to_flags(struct p2m_domain *p2m, ept_entry_t *entry, break; } + /* + * Don't create executable superpages if we need to shatter them to + * protect against CVE-2018-12207. + */ + if ( !opt_ept_exec_sp && is_epte_superpage(entry) ) + entry->x = 0; } #define GUEST_TABLE_MAP_FAILED 0 diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c index e5e4349dea..ba126f790a 100644 --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -289,15 +289,20 @@ static void change_entry_type_global(struct p2m_domain *p2m, p2m_type_t ot, p2m_type_t nt) { p2m->change_entry_type_global(p2m, ot, nt); - p2m->global_logdirty = (nt == p2m_ram_logdirty); + /* Don't allow 'recalculate' operations to change the logdirty state. */ + if ( ot != nt ) + p2m->global_logdirty = (nt == p2m_ram_logdirty); } +/* + * May be called with ot = nt = p2m_ram_rw for its side effect of + * recalculating all PTEs in the p2m. + */ void p2m_change_entry_type_global(struct domain *d, p2m_type_t ot, p2m_type_t nt) { struct p2m_domain *hostp2m = p2m_get_hostp2m(d); - ASSERT(ot != nt); ASSERT(p2m_is_changeable(ot) && p2m_is_changeable(nt)); p2m_lock(hostp2m); diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h index ebaa74449b..371b912887 100644 --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -28,6 +28,8 @@ #include <asm/hvm/trace.h> #include <asm/hvm/vmx/vmcs.h> +extern int8_t opt_ept_exec_sp; + typedef union { struct { u64 r : 1, /* bit 0 - Read permission */ diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h index 637259bd1f..32746aa8ae 100644 --- a/xen/include/asm-x86/msr-index.h +++ b/xen/include/asm-x86/msr-index.h @@ -52,6 +52,7 @@ #define ARCH_CAPS_SKIP_L1DFL (_AC(1, ULL) << 3) #define ARCH_CAPS_SSB_NO (_AC(1, ULL) << 4) #define ARCH_CAPS_MDS_NO (_AC(1, ULL) << 5) +#define ARCH_CAPS_IF_PSCHANGE_MC_NO (_AC(1, ULL) << 6) #define MSR_FLUSH_CMD 0x0000010b #define FLUSH_CMD_L1D (_AC(1, ULL) << 0)