| Message ID | 80ceea17-958d-f409-5f39-9f353e780f5b@suse.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | x86: some assembler macro rework | expand |
On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: > Under certain conditions CPUs can speculate into the instruction stream > past a RET instruction. Guard against this just like 3b7dab93f240 > ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") > did - by inserting an "INT $3" insn. It's merely the mechanics of how to > achieve this that differ: A set of macros gets introduced to post- > process RET insns issued by the compiler (or living in assembly files). > > Unfortunately for clang this requires further features their built-in > assembler doesn't support: We need to be able to override insn mnemonics > produced by the compiler (which may be impossible, if internally > assembly mnemonics never get generated), and we want to use \(text) > escaping / quoting in the auxiliary macro. Could this have an option to enable/disable at build time? FreeBSD will drop GNU as quite soon from base, and albeit it can be installed as a package I would like to be able to build Xen using a toolchain based on LLVM exclusively. Thanks, Roger.
On 10.11.2020 10:31, Roger Pau Monné wrote: > On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: >> Under certain conditions CPUs can speculate into the instruction stream >> past a RET instruction. Guard against this just like 3b7dab93f240 >> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") >> did - by inserting an "INT $3" insn. It's merely the mechanics of how to >> achieve this that differ: A set of macros gets introduced to post- >> process RET insns issued by the compiler (or living in assembly files). >> >> Unfortunately for clang this requires further features their built-in >> assembler doesn't support: We need to be able to override insn mnemonics >> produced by the compiler (which may be impossible, if internally >> assembly mnemonics never get generated), and we want to use \(text) >> escaping / quoting in the auxiliary macro. > > Could this have an option to enable/disable at build time? Well, a subsequent patch adds a config option for this, which in the worst case could be turned off. I'm afraid though I'm not clear about the question, because ... > FreeBSD will drop GNU as quite soon from base, and albeit it can be > installed as a package I would like to be able to build Xen using a > toolchain based on LLVM exclusively. ... it's not clear to me what the implications here are: Are you saying -no-integrated-as is not going to function anymore, unless people explicitly install gas? If that's not what you meant to indicate, then I don't see how building would become impossible. Depending on the situation as a whole, we might then be in need of a 2nd config option... And btw, good that you pointed me back at this: The v3 change wasn't quite complete, since we now don't need to check for proper \(text) handling anymore in our logic to establish CLANG_FLAGS. I've dropped that part for v4. Jan
On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote: > On 10.11.2020 10:31, Roger Pau Monné wrote: > > On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: > >> Under certain conditions CPUs can speculate into the instruction stream > >> past a RET instruction. Guard against this just like 3b7dab93f240 > >> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") > >> did - by inserting an "INT $3" insn. It's merely the mechanics of how to > >> achieve this that differ: A set of macros gets introduced to post- > >> process RET insns issued by the compiler (or living in assembly files). > >> > >> Unfortunately for clang this requires further features their built-in > >> assembler doesn't support: We need to be able to override insn mnemonics > >> produced by the compiler (which may be impossible, if internally > >> assembly mnemonics never get generated), and we want to use \(text) > >> escaping / quoting in the auxiliary macro. > > > > Could this have an option to enable/disable at build time? > > Well, a subsequent patch adds a config option for this, which in > the worst case could be turned off. I'm afraid though I'm not > clear about the question, because ... > > > FreeBSD will drop GNU as quite soon from base, and albeit it can be > > installed as a package I would like to be able to build Xen using a > > toolchain based on LLVM exclusively. > > ... it's not clear to me what the implications here are: Are you > saying -no-integrated-as is not going to function anymore, unless > people explicitly install gas? If that's not what you meant to > indicate, then I don't see how building would become impossible. I'm still inquiring about this, but I would say that when gas is removed from FreeBSD then the 'as' command would be mapped to llvm-as, and thus -no-integrated-as would hit the same issues as the integrated as. So far in Xen we have assumed that -no-integrated-as would fallback to an as capable of doing what the integrated clang as doesn't support, but that might not be the case. Ideally we would have to re-run the tests with -no-integrated-as, in order to assert that the external as is really capable of what the internal one is not. Roger.
On 10.11.2020 12:16, Roger Pau Monné wrote: > On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote: >> On 10.11.2020 10:31, Roger Pau Monné wrote: >>> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: >>>> Under certain conditions CPUs can speculate into the instruction stream >>>> past a RET instruction. Guard against this just like 3b7dab93f240 >>>> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") >>>> did - by inserting an "INT $3" insn. It's merely the mechanics of how to >>>> achieve this that differ: A set of macros gets introduced to post- >>>> process RET insns issued by the compiler (or living in assembly files). >>>> >>>> Unfortunately for clang this requires further features their built-in >>>> assembler doesn't support: We need to be able to override insn mnemonics >>>> produced by the compiler (which may be impossible, if internally >>>> assembly mnemonics never get generated), and we want to use \(text) >>>> escaping / quoting in the auxiliary macro. >>> >>> Could this have an option to enable/disable at build time? >> >> Well, a subsequent patch adds a config option for this, which in >> the worst case could be turned off. I'm afraid though I'm not >> clear about the question, because ... >> >>> FreeBSD will drop GNU as quite soon from base, and albeit it can be >>> installed as a package I would like to be able to build Xen using a >>> toolchain based on LLVM exclusively. >> >> ... it's not clear to me what the implications here are: Are you >> saying -no-integrated-as is not going to function anymore, unless >> people explicitly install gas? If that's not what you meant to >> indicate, then I don't see how building would become impossible. > > I'm still inquiring about this, but I would say that when gas is > removed from FreeBSD then the 'as' command would be mapped to llvm-as, > and thus -no-integrated-as would hit the same issues as the integrated > as. So far in Xen we have assumed that -no-integrated-as would > fallback to an as capable of doing what the integrated clang as > doesn't support, but that might not be the case. At which point Xen couldn't be built anyway, I expect. If llvm-as isn't sufficiently gas-compatible, we've lost (right now at least). > Ideally we would have to re-run the tests with -no-integrated-as, in > order to assert that the external as is really capable of what the > internal one is not. And if it doesn't, what would we do other than failing the build (which it would also if we didn't do the 2nd round of checks)? Jan
On Tue, Nov 10, 2020 at 02:19:40PM +0100, Jan Beulich wrote: > On 10.11.2020 12:16, Roger Pau Monné wrote: > > On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote: > >> On 10.11.2020 10:31, Roger Pau Monné wrote: > >>> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: > >>>> Under certain conditions CPUs can speculate into the instruction stream > >>>> past a RET instruction. Guard against this just like 3b7dab93f240 > >>>> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") > >>>> did - by inserting an "INT $3" insn. It's merely the mechanics of how to > >>>> achieve this that differ: A set of macros gets introduced to post- > >>>> process RET insns issued by the compiler (or living in assembly files). > >>>> > >>>> Unfortunately for clang this requires further features their built-in > >>>> assembler doesn't support: We need to be able to override insn mnemonics > >>>> produced by the compiler (which may be impossible, if internally > >>>> assembly mnemonics never get generated), and we want to use \(text) > >>>> escaping / quoting in the auxiliary macro. > >>> > >>> Could this have an option to enable/disable at build time? > >> > >> Well, a subsequent patch adds a config option for this, which in > >> the worst case could be turned off. I'm afraid though I'm not > >> clear about the question, because ... > >> > >>> FreeBSD will drop GNU as quite soon from base, and albeit it can be > >>> installed as a package I would like to be able to build Xen using a > >>> toolchain based on LLVM exclusively. > >> > >> ... it's not clear to me what the implications here are: Are you > >> saying -no-integrated-as is not going to function anymore, unless > >> people explicitly install gas? If that's not what you meant to > >> indicate, then I don't see how building would become impossible. > > > > I'm still inquiring about this, but I would say that when gas is > > removed from FreeBSD then the 'as' command would be mapped to llvm-as, > > and thus -no-integrated-as would hit the same issues as the integrated > > as. So far in Xen we have assumed that -no-integrated-as would > > fallback to an as capable of doing what the integrated clang as > > doesn't support, but that might not be the case. > > At which point Xen couldn't be built anyway, I expect. If llvm-as > isn't sufficiently gas-compatible, we've lost (right now at least). > > > Ideally we would have to re-run the tests with -no-integrated-as, in > > order to assert that the external as is really capable of what the > > internal one is not. > > And if it doesn't, what would we do other than failing the build > (which it would also if we didn't do the 2nd round of checks)? I would always prefer a clear message (ie: your toolstack is not capable of building Xen) rather than a weird build time failure. Also we could maybe disable certain options by default if the toolstack doesn't have the required support to build them? Has anyone reported this shortcoming to upstream llvm, so they are aware and can work on this or maybe provide an alternative way to achieve the same result? Thanks, Roger.
On 10.11.2020 15:08, Roger Pau Monné wrote: > On Tue, Nov 10, 2020 at 02:19:40PM +0100, Jan Beulich wrote: >> On 10.11.2020 12:16, Roger Pau Monné wrote: >>> On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote: >>>> On 10.11.2020 10:31, Roger Pau Monné wrote: >>>>> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: >>>>>> Under certain conditions CPUs can speculate into the instruction stream >>>>>> past a RET instruction. Guard against this just like 3b7dab93f240 >>>>>> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") >>>>>> did - by inserting an "INT $3" insn. It's merely the mechanics of how to >>>>>> achieve this that differ: A set of macros gets introduced to post- >>>>>> process RET insns issued by the compiler (or living in assembly files). >>>>>> >>>>>> Unfortunately for clang this requires further features their built-in >>>>>> assembler doesn't support: We need to be able to override insn mnemonics >>>>>> produced by the compiler (which may be impossible, if internally >>>>>> assembly mnemonics never get generated), and we want to use \(text) >>>>>> escaping / quoting in the auxiliary macro. >>>>> >>>>> Could this have an option to enable/disable at build time? >>>> >>>> Well, a subsequent patch adds a config option for this, which in >>>> the worst case could be turned off. I'm afraid though I'm not >>>> clear about the question, because ... >>>> >>>>> FreeBSD will drop GNU as quite soon from base, and albeit it can be >>>>> installed as a package I would like to be able to build Xen using a >>>>> toolchain based on LLVM exclusively. >>>> >>>> ... it's not clear to me what the implications here are: Are you >>>> saying -no-integrated-as is not going to function anymore, unless >>>> people explicitly install gas? If that's not what you meant to >>>> indicate, then I don't see how building would become impossible. >>> >>> I'm still inquiring about this, but I would say that when gas is >>> removed from FreeBSD then the 'as' command would be mapped to llvm-as, >>> and thus -no-integrated-as would hit the same issues as the integrated >>> as. So far in Xen we have assumed that -no-integrated-as would >>> fallback to an as capable of doing what the integrated clang as >>> doesn't support, but that might not be the case. >> >> At which point Xen couldn't be built anyway, I expect. If llvm-as >> isn't sufficiently gas-compatible, we've lost (right now at least). >> >>> Ideally we would have to re-run the tests with -no-integrated-as, in >>> order to assert that the external as is really capable of what the >>> internal one is not. >> >> And if it doesn't, what would we do other than failing the build >> (which it would also if we didn't do the 2nd round of checks)? > > I would always prefer a clear message (ie: your toolstack is not > capable of building Xen) rather than a weird build time failure. Fair point in general. > Also we could maybe disable certain options by default if the > toolstack doesn't have the required support to build them? We could, but I'm afraid this will go down the route of embedding tool chain capabilities in xen/.config, which I continue to not consider a good idea (and the thread got stalled, as expected). In fact (also to Andrew and Anthony), recently I've become aware of another shortcoming of this model: Our kernel packages contain .config files for the various architectures and specific per- architecture flavors. It used to be easy to update them on any system, until the tool chain capability checks got introduced. Now, in order to update them, one has to use the precise versions of the various tool chain parts that will be used on the build hosts, or else an error may result (for unexpected changes to the file), or one may unknowingly turn off options that are expected to be enabled. Put more generally - if I handed someone a specific .config, I'd expect their resulting binary to contain what I did set up. Or for them to report back that they can't build the thing. But it should not be the case that the .config got silently changed and certain functionality disabled just because they use a different tool chain. > Has anyone reported this shortcoming to upstream llvm, so they are > aware and can work on this or maybe provide an alternative way to > achieve the same result? I didn't and I'm unaware of anyone else possibly having done so. That said, I consider it sort of obvious though that the goal of replacing the GNU tool chain implies being fully compatible (and presumably better in certain areas). Jan
On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: > Under certain conditions CPUs can speculate into the instruction stream > past a RET instruction. Guard against this just like 3b7dab93f240 > ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") > did - by inserting an "INT $3" insn. It's merely the mechanics of how to > achieve this that differ: A set of macros gets introduced to post- > process RET insns issued by the compiler (or living in assembly files). > > Unfortunately for clang this requires further features their built-in > assembler doesn't support: We need to be able to override insn mnemonics > produced by the compiler (which may be impossible, if internally > assembly mnemonics never get generated), and we want to use \(text) > escaping / quoting in the auxiliary macro. > > Signed-off-by: Jan Beulich <jbeulich@suse.com> > Acked-by: Roger Pau Monné <roger.pau@citrix.com> > --- > TBD: Would be nice to avoid the additions in .init.text, but a query to > the binutils folks regarding the ability to identify the section > stuff is in (by Peter Zijlstra over a year ago: > https://sourceware.org/pipermail/binutils/2019-July/107528.html) > has been left without helpful replies. > --- > v3: Use .byte 0xc[23] instead of the nested macros. > v2: Fix build with newer clang. Use int3 mnemonic. Also override retq. > > --- a/xen/Makefile > +++ b/xen/Makefile > @@ -145,7 +145,15 @@ t2 = $(call as-insn,$(CC) -I$(BASEDIR)/i > # https://bugs.llvm.org/show_bug.cgi?id=36110 > t3 = $(call as-insn,$(CC),".macro FOO;.endm"$(close); asm volatile $(open)".macro FOO;.endm",-no-integrated-as) > > -CLANG_FLAGS += $(call or,$(t1),$(t2),$(t3)) > +# Check whether \(text) escaping in macro bodies is supported. > +t4 = $(call as-insn,$(CC),".macro m ret:req; \\(ret) $$\\ret; .endm; m 8",,-no-integrated-as) > + > +# Check whether macros can override insn mnemonics in inline assembly. > +t5 = $(call as-insn,$(CC),".macro ret; .error; .endm; .macro retq; .error; .endm",-no-integrated-as) I was going over this to post a bug report to LLVM, but it seems like gcc also doesn't overwrite ret when using the above snippet: https://godbolt.org/z/oqsPTv Roger.
On Tue, Nov 10, 2020 at 03:32:43PM +0100, Jan Beulich wrote: > On 10.11.2020 15:08, Roger Pau Monné wrote: > > On Tue, Nov 10, 2020 at 02:19:40PM +0100, Jan Beulich wrote: > >> On 10.11.2020 12:16, Roger Pau Monné wrote: > >>> On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote: > >>>> On 10.11.2020 10:31, Roger Pau Monné wrote: > >>>>> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: > >>>>>> Under certain conditions CPUs can speculate into the instruction stream > >>>>>> past a RET instruction. Guard against this just like 3b7dab93f240 > >>>>>> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") > >>>>>> did - by inserting an "INT $3" insn. It's merely the mechanics of how to > >>>>>> achieve this that differ: A set of macros gets introduced to post- > >>>>>> process RET insns issued by the compiler (or living in assembly files). > >>>>>> > >>>>>> Unfortunately for clang this requires further features their built-in > >>>>>> assembler doesn't support: We need to be able to override insn mnemonics > >>>>>> produced by the compiler (which may be impossible, if internally > >>>>>> assembly mnemonics never get generated), and we want to use \(text) > >>>>>> escaping / quoting in the auxiliary macro. > >>>>> > >>>>> Could this have an option to enable/disable at build time? > >>>> > >>>> Well, a subsequent patch adds a config option for this, which in > >>>> the worst case could be turned off. I'm afraid though I'm not > >>>> clear about the question, because ... > >>>> > >>>>> FreeBSD will drop GNU as quite soon from base, and albeit it can be > >>>>> installed as a package I would like to be able to build Xen using a > >>>>> toolchain based on LLVM exclusively. > >>>> > >>>> ... it's not clear to me what the implications here are: Are you > >>>> saying -no-integrated-as is not going to function anymore, unless > >>>> people explicitly install gas? If that's not what you meant to > >>>> indicate, then I don't see how building would become impossible. > >>> > >>> I'm still inquiring about this, but I would say that when gas is > >>> removed from FreeBSD then the 'as' command would be mapped to llvm-as, > >>> and thus -no-integrated-as would hit the same issues as the integrated > >>> as. So far in Xen we have assumed that -no-integrated-as would > >>> fallback to an as capable of doing what the integrated clang as > >>> doesn't support, but that might not be the case. > >> > >> At which point Xen couldn't be built anyway, I expect. If llvm-as > >> isn't sufficiently gas-compatible, we've lost (right now at least). > >> > >>> Ideally we would have to re-run the tests with -no-integrated-as, in > >>> order to assert that the external as is really capable of what the > >>> internal one is not. > >> > >> And if it doesn't, what would we do other than failing the build > >> (which it would also if we didn't do the 2nd round of checks)? > > > > I would always prefer a clear message (ie: your toolstack is not > > capable of building Xen) rather than a weird build time failure. > > Fair point in general. > > > Also we could maybe disable certain options by default if the > > toolstack doesn't have the required support to build them? > > We could, but I'm afraid this will go down the route of embedding > tool chain capabilities in xen/.config, which I continue to not > consider a good idea (and the thread got stalled, as expected). > > In fact (also to Andrew and Anthony), recently I've become aware > of another shortcoming of this model: Our kernel packages contain > .config files for the various architectures and specific per- > architecture flavors. It used to be easy to update them on any > system, until the tool chain capability checks got introduced. > Now, in order to update them, one has to use the precise versions > of the various tool chain parts that will be used on the build > hosts, or else an error may result (for unexpected changes to > the file), or one may unknowingly turn off options that are > expected to be enabled. I think the options should only be set based on toolchain capabilities when there's no .config. If there's an existing .config we should just check whether the toolchain is capable of building the selected set of options, or else report an error. I guess this would apply to defconfig selecting options based on toolchain capabilties. > Put more generally - if I handed someone a specific .config, I'd > expect their resulting binary to contain what I did set up. Or > for them to report back that they can't build the thing. But it > should not be the case that the .config got silently changed and > certain functionality disabled just because they use a different > tool chain. Yes, I agree with this. > > Has anyone reported this shortcoming to upstream llvm, so they are > > aware and can work on this or maybe provide an alternative way to > > achieve the same result? > > I didn't and I'm unaware of anyone else possibly having done so. > That said, I consider it sort of obvious though that the goal of > replacing the GNU tool chain implies being fully compatible (and > presumably better in certain areas). Well, I think we have to keep in mind that the usage of the compiler and the linker by Xen is far more advanced than what most applications do, and we are likely to hit corner cases. I bet the LLVM people weren't even aware of such usage. Thanks, Roger.
On 11.11.2020 12:15, Roger Pau Monné wrote: > On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: >> Under certain conditions CPUs can speculate into the instruction stream >> past a RET instruction. Guard against this just like 3b7dab93f240 >> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") >> did - by inserting an "INT $3" insn. It's merely the mechanics of how to >> achieve this that differ: A set of macros gets introduced to post- >> process RET insns issued by the compiler (or living in assembly files). >> >> Unfortunately for clang this requires further features their built-in >> assembler doesn't support: We need to be able to override insn mnemonics >> produced by the compiler (which may be impossible, if internally >> assembly mnemonics never get generated), and we want to use \(text) >> escaping / quoting in the auxiliary macro. >> >> Signed-off-by: Jan Beulich <jbeulich@suse.com> >> Acked-by: Roger Pau Monné <roger.pau@citrix.com> >> --- >> TBD: Would be nice to avoid the additions in .init.text, but a query to >> the binutils folks regarding the ability to identify the section >> stuff is in (by Peter Zijlstra over a year ago: >> https://sourceware.org/pipermail/binutils/2019-July/107528.html) >> has been left without helpful replies. >> --- >> v3: Use .byte 0xc[23] instead of the nested macros. >> v2: Fix build with newer clang. Use int3 mnemonic. Also override retq. >> >> --- a/xen/Makefile >> +++ b/xen/Makefile >> @@ -145,7 +145,15 @@ t2 = $(call as-insn,$(CC) -I$(BASEDIR)/i >> # https://bugs.llvm.org/show_bug.cgi?id=36110 >> t3 = $(call as-insn,$(CC),".macro FOO;.endm"$(close); asm volatile $(open)".macro FOO;.endm",-no-integrated-as) >> >> -CLANG_FLAGS += $(call or,$(t1),$(t2),$(t3)) >> +# Check whether \(text) escaping in macro bodies is supported. >> +t4 = $(call as-insn,$(CC),".macro m ret:req; \\(ret) $$\\ret; .endm; m 8",,-no-integrated-as) >> + >> +# Check whether macros can override insn mnemonics in inline assembly. >> +t5 = $(call as-insn,$(CC),".macro ret; .error; .endm; .macro retq; .error; .endm",-no-integrated-as) > > I was going over this to post a bug report to LLVM, but it seems like > gcc also doesn't overwrite ret when using the above snippet: > > https://godbolt.org/z/oqsPTv I can't see what's different from void test(void) { asm volatile (".macro ret; .error; .endm; .macro retq; .error; .endm"); } but this one produces "Error: .error directive invoked in source file" for me with both old and new gcc. Jan
On Wed, Nov 11, 2020 at 02:33:34PM +0100, Jan Beulich wrote: > On 11.11.2020 12:15, Roger Pau Monné wrote: > > On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: > >> Under certain conditions CPUs can speculate into the instruction stream > >> past a RET instruction. Guard against this just like 3b7dab93f240 > >> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") > >> did - by inserting an "INT $3" insn. It's merely the mechanics of how to > >> achieve this that differ: A set of macros gets introduced to post- > >> process RET insns issued by the compiler (or living in assembly files). > >> > >> Unfortunately for clang this requires further features their built-in > >> assembler doesn't support: We need to be able to override insn mnemonics > >> produced by the compiler (which may be impossible, if internally > >> assembly mnemonics never get generated), and we want to use \(text) > >> escaping / quoting in the auxiliary macro. > >> > >> Signed-off-by: Jan Beulich <jbeulich@suse.com> > >> Acked-by: Roger Pau Monné <roger.pau@citrix.com> > >> --- > >> TBD: Would be nice to avoid the additions in .init.text, but a query to > >> the binutils folks regarding the ability to identify the section > >> stuff is in (by Peter Zijlstra over a year ago: > >> https://sourceware.org/pipermail/binutils/2019-July/107528.html) > >> has been left without helpful replies. > >> --- > >> v3: Use .byte 0xc[23] instead of the nested macros. > >> v2: Fix build with newer clang. Use int3 mnemonic. Also override retq. > >> > >> --- a/xen/Makefile > >> +++ b/xen/Makefile > >> @@ -145,7 +145,15 @@ t2 = $(call as-insn,$(CC) -I$(BASEDIR)/i > >> # https://bugs.llvm.org/show_bug.cgi?id=36110 > >> t3 = $(call as-insn,$(CC),".macro FOO;.endm"$(close); asm volatile $(open)".macro FOO;.endm",-no-integrated-as) > >> > >> -CLANG_FLAGS += $(call or,$(t1),$(t2),$(t3)) > >> +# Check whether \(text) escaping in macro bodies is supported. > >> +t4 = $(call as-insn,$(CC),".macro m ret:req; \\(ret) $$\\ret; .endm; m 8",,-no-integrated-as) > >> + > >> +# Check whether macros can override insn mnemonics in inline assembly. > >> +t5 = $(call as-insn,$(CC),".macro ret; .error; .endm; .macro retq; .error; .endm",-no-integrated-as) > > > > I was going over this to post a bug report to LLVM, but it seems like > > gcc also doesn't overwrite ret when using the above snippet: > > > > https://godbolt.org/z/oqsPTv > > I can't see what's different from > > void test(void) { > asm volatile (".macro ret; .error; .endm; .macro retq; .error; .endm"); > } > > but this one produces "Error: .error directive invoked in source file" > for me with both old and new gcc. You are right, I think godbolt is somehow busted? I can reproduce your results with my version of gcc, so will just report to LLVM. Roger.
On 11.11.2020 15:19, Roger Pau Monné wrote: > On Wed, Nov 11, 2020 at 02:33:34PM +0100, Jan Beulich wrote: >> On 11.11.2020 12:15, Roger Pau Monné wrote: >>> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: >>>> Under certain conditions CPUs can speculate into the instruction stream >>>> past a RET instruction. Guard against this just like 3b7dab93f240 >>>> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") >>>> did - by inserting an "INT $3" insn. It's merely the mechanics of how to >>>> achieve this that differ: A set of macros gets introduced to post- >>>> process RET insns issued by the compiler (or living in assembly files). >>>> >>>> Unfortunately for clang this requires further features their built-in >>>> assembler doesn't support: We need to be able to override insn mnemonics >>>> produced by the compiler (which may be impossible, if internally >>>> assembly mnemonics never get generated), and we want to use \(text) >>>> escaping / quoting in the auxiliary macro. >>>> >>>> Signed-off-by: Jan Beulich <jbeulich@suse.com> >>>> Acked-by: Roger Pau Monné <roger.pau@citrix.com> >>>> --- >>>> TBD: Would be nice to avoid the additions in .init.text, but a query to >>>> the binutils folks regarding the ability to identify the section >>>> stuff is in (by Peter Zijlstra over a year ago: >>>> https://sourceware.org/pipermail/binutils/2019-July/107528.html) >>>> has been left without helpful replies. >>>> --- >>>> v3: Use .byte 0xc[23] instead of the nested macros. >>>> v2: Fix build with newer clang. Use int3 mnemonic. Also override retq. >>>> >>>> --- a/xen/Makefile >>>> +++ b/xen/Makefile >>>> @@ -145,7 +145,15 @@ t2 = $(call as-insn,$(CC) -I$(BASEDIR)/i >>>> # https://bugs.llvm.org/show_bug.cgi?id=36110 >>>> t3 = $(call as-insn,$(CC),".macro FOO;.endm"$(close); asm volatile $(open)".macro FOO;.endm",-no-integrated-as) >>>> >>>> -CLANG_FLAGS += $(call or,$(t1),$(t2),$(t3)) >>>> +# Check whether \(text) escaping in macro bodies is supported. >>>> +t4 = $(call as-insn,$(CC),".macro m ret:req; \\(ret) $$\\ret; .endm; m 8",,-no-integrated-as) >>>> + >>>> +# Check whether macros can override insn mnemonics in inline assembly. >>>> +t5 = $(call as-insn,$(CC),".macro ret; .error; .endm; .macro retq; .error; .endm",-no-integrated-as) >>> >>> I was going over this to post a bug report to LLVM, but it seems like >>> gcc also doesn't overwrite ret when using the above snippet: >>> >>> https://godbolt.org/z/oqsPTv >> >> I can't see what's different from >> >> void test(void) { >> asm volatile (".macro ret; .error; .endm; .macro retq; .error; .endm"); >> } >> >> but this one produces "Error: .error directive invoked in source file" >> for me with both old and new gcc. > > You are right, I think godbolt is somehow busted? Or maybe they really only compile to assembly, while the error results from the assembler? Jan
On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote: > Under certain conditions CPUs can speculate into the instruction stream > past a RET instruction. Guard against this just like 3b7dab93f240 > ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") > did - by inserting an "INT $3" insn. It's merely the mechanics of how to > achieve this that differ: A set of macros gets introduced to post- > process RET insns issued by the compiler (or living in assembly files). > > Unfortunately for clang this requires further features their built-in > assembler doesn't support: We need to be able to override insn mnemonics > produced by the compiler (which may be impossible, if internally > assembly mnemonics never get generated) FTR I've reported this to LLVM upstream: https://bugs.llvm.org/show_bug.cgi?id=48159 Roger.
--- a/xen/Makefile +++ b/xen/Makefile @@ -145,7 +145,15 @@ t2 = $(call as-insn,$(CC) -I$(BASEDIR)/i # https://bugs.llvm.org/show_bug.cgi?id=36110 t3 = $(call as-insn,$(CC),".macro FOO;.endm"$(close); asm volatile $(open)".macro FOO;.endm",-no-integrated-as) -CLANG_FLAGS += $(call or,$(t1),$(t2),$(t3)) +# Check whether \(text) escaping in macro bodies is supported. +t4 = $(call as-insn,$(CC),".macro m ret:req; \\(ret) $$\\ret; .endm; m 8",,-no-integrated-as) + +# Check whether macros can override insn mnemonics in inline assembly. +t5 = $(call as-insn,$(CC),".macro ret; .error; .endm; .macro retq; .error; .endm",-no-integrated-as) + +acc1 := $(call or,$(t1),$(t2),$(t3),$(t4)) + +CLANG_FLAGS += $(call or,$(acc1),$(t5)) endif CLANG_FLAGS += -Werror=unknown-warning-option --- a/xen/include/asm-x86/asm-defns.h +++ b/xen/include/asm-x86/asm-defns.h @@ -50,3 +50,19 @@ .macro INDIRECT_JMP arg:req INDIRECT_BRANCH jmp \arg .endm + +/* + * To guard against speculation past RET, insert a breakpoint insn + * immediately after them. + */ +.macro ret operand:vararg + retq \operand +.endm +.macro retq operand:vararg + .ifb \operand + .byte 0xc3 + .else + .byte 0xc2 + .word \operand + .endif +.endm
Under certain conditions CPUs can speculate into the instruction stream past a RET instruction. Guard against this just like 3b7dab93f240 ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") did - by inserting an "INT $3" insn. It's merely the mechanics of how to achieve this that differ: A set of macros gets introduced to post- process RET insns issued by the compiler (or living in assembly files). Unfortunately for clang this requires further features their built-in assembler doesn't support: We need to be able to override insn mnemonics produced by the compiler (which may be impossible, if internally assembly mnemonics never get generated), and we want to use \(text) escaping / quoting in the auxiliary macro. Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Roger Pau Monné <roger.pau@citrix.com> --- TBD: Would be nice to avoid the additions in .init.text, but a query to the binutils folks regarding the ability to identify the section stuff is in (by Peter Zijlstra over a year ago: https://sourceware.org/pipermail/binutils/2019-July/107528.html) has been left without helpful replies. --- v3: Use .byte 0xc[23] instead of the nested macros. v2: Fix build with newer clang. Use int3 mnemonic. Also override retq.