diff mbox series

riscv: Handle zicsr/zifencei issue between gcc and binutils

Message ID 20230725170405.251011-1-xingmingzheng@iscas.ac.cn (mailing list archive)
State Superseded
Headers show
Series riscv: Handle zicsr/zifencei issue between gcc and binutils | expand

Checks

Context Check Description
conchuod/cover_letter success Single patches do not need cover letters
conchuod/tree_selection success Guessed tree name to be for-next at HEAD 471aba2e4760
conchuod/fixes_present success Fixes tag not required for -next series
conchuod/maintainers_pattern success MAINTAINERS pattern errors before the patch: 4 and now 4
conchuod/verify_signedoff success Signed-off-by tag matches author and committer
conchuod/kdoc success Errors and warnings before: 0 this patch: 0
conchuod/build_rv64_clang_allmodconfig success Errors and warnings before: 9 this patch: 9
conchuod/module_param success Was 0 now: 0
conchuod/build_rv64_gcc_allmodconfig success Errors and warnings before: 9 this patch: 9
conchuod/build_rv32_defconfig success Build OK
conchuod/dtb_warn_rv64 success Errors and warnings before: 3 this patch: 3
conchuod/header_inline success No static functions without inline keyword in header files
conchuod/checkpatch success total: 0 errors, 0 warnings, 0 checks, 16 lines checked
conchuod/build_rv64_nommu_k210_defconfig success Build OK
conchuod/verify_fixes success No Fixes tag
conchuod/build_rv64_nommu_virt_defconfig success Build OK

Commit Message

Mingzheng Xing July 25, 2023, 5:04 p.m. UTC
When compiling the kernel with the toolchain composed of GCC >= 12.1.0 and
binutils < 2.38, default ISA spec used when building binutils and GCC, the
following build failure will appear because the
CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI is not turned on.
(i.e, gcc-12.1.0 and binutils-2.36, or gcc-12.3.0 and binutils-2.37, use
default ISA spec.)

  CC      arch/riscv/kernel/vdso/vgettimeofday.o
  <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h: Assembler messages:
  <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:79: Error: unrecognized opcode `csrr a5,0xc01'

Binutils has updated the default ISA spec version, and the community has
responded well to this[1][2][3], but it appears that this is not over yet.

We also need to consider the situation of binutils < 2.38 but
GCC >= 12.1.0, since the combination between different versions of GCC and
binutils is not unique, which is to some extent flexible. GCC release
12.1.0 updated the default ISA spec version in GCC commit[4].

For more information, please refer to:

commit 6df2a016c0c8 ("riscv: fix build with binutils 2.38")
commit e89c2e815e76 ("riscv: Handle zicsr/zifencei issues between clang and binutils")

[1]: https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4
[2]: https://lore.kernel.org/all/20230308220842.1231003-1-conor@kernel.org
[3]: https://lore.kernel.org/all/20230223220546.52879-1-conor@kernel.org
[4]: https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd

Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
---
 arch/riscv/Kconfig | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

Comments

Nathan Chancellor July 25, 2023, 5:23 p.m. UTC | #1
Hi Mingzheng,

Thanks for the patch!

On Wed, Jul 26, 2023 at 01:04:05AM +0800, Mingzheng Xing wrote:
> When compiling the kernel with the toolchain composed of GCC >= 12.1.0 and
> binutils < 2.38, default ISA spec used when building binutils and GCC, the
> following build failure will appear because the
> CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI is not turned on.
> (i.e, gcc-12.1.0 and binutils-2.36, or gcc-12.3.0 and binutils-2.37, use
> default ISA spec.)
> 
>   CC      arch/riscv/kernel/vdso/vgettimeofday.o
>   <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h: Assembler messages:
>   <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:79: Error: unrecognized opcode `csrr a5,0xc01'

The gift that keeps on giving :/

> Binutils has updated the default ISA spec version, and the community has
> responded well to this[1][2][3], but it appears that this is not over yet.
> 
> We also need to consider the situation of binutils < 2.38 but
> GCC >= 12.1.0, since the combination between different versions of GCC and
> binutils is not unique, which is to some extent flexible. GCC release
> 12.1.0 updated the default ISA spec version in GCC commit[4].

I suspect this combination is not too common because binutils 2.38 came
out before GCC 12.1.0 but as you note, it is obviously possible. What
toolchain has this combination in the wild, which would be helpful for
documentation purposes?

> For more information, please refer to:
> 
> commit 6df2a016c0c8 ("riscv: fix build with binutils 2.38")
> commit e89c2e815e76 ("riscv: Handle zicsr/zifencei issues between clang and binutils")
> 
> [1]: https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4
> [2]: https://lore.kernel.org/all/20230308220842.1231003-1-conor@kernel.org
> [3]: https://lore.kernel.org/all/20230223220546.52879-1-conor@kernel.org
> [4]: https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
> 
> Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
> ---
>  arch/riscv/Kconfig | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 4c07b9189c86..b49cea30f6cc 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -570,11 +570,15 @@ config TOOLCHAIN_HAS_ZIHINTPAUSE
>  config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI
>  	def_bool y
>  	# https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc
> -	depends on AS_IS_GNU && AS_VERSION >= 23800
> +	# https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
> +	depends on CC_IS_GCC && GCC_VERSION >= 120100 || \
> +		   AS_IS_GNU && AS_VERSION >= 23800

GCC_VERSION will be 0 for clang, so you don't need the CC_IS_GCC check.
With that change, this should be able to stay on one line:

    depends on GCC_VERSION >= 120100 || (AS_IS_GNU && AS_VERSION >= 23800)

>  	help
>  	  Newer binutils versions default to ISA spec version 20191213 which
>  	  moves some instructions from the I extension to the Zicsr and Zifencei
>  	  extensions.
> +	  Similarly, GCC release 12.1.0 has changed the default ISA spec version to
> +	  20191213, so the above situation requires this option to be enabled.
>  
>  config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
>  	def_bool y
> -- 
> 2.34.1
> 

Cheers,
Nathan
Conor Dooley July 25, 2023, 6:57 p.m. UTC | #2
On Tue, Jul 25, 2023 at 10:23:44AM -0700, Nathan Chancellor wrote:
> Hi Mingzheng,
> 
> Thanks for the patch!
> 
> On Wed, Jul 26, 2023 at 01:04:05AM +0800, Mingzheng Xing wrote:
> > When compiling the kernel with the toolchain composed of GCC >= 12.1.0 and
> > binutils < 2.38, default ISA spec used when building binutils and GCC, the
> > following build failure will appear because the
> > CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI is not turned on.
> > (i.e, gcc-12.1.0 and binutils-2.36, or gcc-12.3.0 and binutils-2.37, use
> > default ISA spec.)
> > 
> >   CC      arch/riscv/kernel/vdso/vgettimeofday.o
> >   <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h: Assembler messages:
> >   <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:79: Error: unrecognized opcode `csrr a5,0xc01'
> 
> The gift that keeps on giving :/
> 
> > Binutils has updated the default ISA spec version, and the community has
> > responded well to this[1][2][3], but it appears that this is not over yet.
> > 
> > We also need to consider the situation of binutils < 2.38 but
> > GCC >= 12.1.0, since the combination between different versions of GCC and
> > binutils is not unique, which is to some extent flexible. GCC release
> > 12.1.0 updated the default ISA spec version in GCC commit[4].
> 
> I suspect this combination is not too common because binutils 2.38 came
> out before GCC 12.1.0 but as you note, it is obviously possible. What
> toolchain has this combination in the wild, which would be helpful for
> documentation purposes?

Yeah, that'd be great to know, at least the other niche stuff that we
are working around had a clear use-case (testing LLVM in debian containers)
whereas there's no clear user for this.
That's doubly interesting, as this patch seems to break things for binutils
< 2.35, and if we have to make a trade-off between those too, then it'd
be good to be able to weigh up the options.
Do we perhaps need the misa-spec workaround instead for this case?
Haven't tested that though, trying to dig myself out of email backlog.

> > For more information, please refer to:
> > 
> > commit 6df2a016c0c8 ("riscv: fix build with binutils 2.38")
> > commit e89c2e815e76 ("riscv: Handle zicsr/zifencei issues between clang and binutils")
> > 
> > [1]: https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4
> > [2]: https://lore.kernel.org/all/20230308220842.1231003-1-conor@kernel.org
> > [3]: https://lore.kernel.org/all/20230223220546.52879-1-conor@kernel.org
> > [4]: https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd

btw, please make these regular Link: tags (with a [N] at EOL) and drop
the space between them and the sign off. Also, this probably needs to be
CC: stable@vger.kernel.org too.

Cheers,
Conor.

> > 
> > Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
> > ---
> >  arch/riscv/Kconfig | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 4c07b9189c86..b49cea30f6cc 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -570,11 +570,15 @@ config TOOLCHAIN_HAS_ZIHINTPAUSE
> >  config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI
> >  	def_bool y
> >  	# https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc
> > -	depends on AS_IS_GNU && AS_VERSION >= 23800
> > +	# https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
> > +	depends on CC_IS_GCC && GCC_VERSION >= 120100 || \
> > +		   AS_IS_GNU && AS_VERSION >= 23800
> 
> GCC_VERSION will be 0 for clang, so you don't need the CC_IS_GCC check.
> With that change, this should be able to stay on one line:
> 
>     depends on GCC_VERSION >= 120100 || (AS_IS_GNU && AS_VERSION >= 23800)
> 
> >  	help
> >  	  Newer binutils versions default to ISA spec version 20191213 which
> >  	  moves some instructions from the I extension to the Zicsr and Zifencei
> >  	  extensions.
> > +	  Similarly, GCC release 12.1.0 has changed the default ISA spec version to
> > +	  20191213, so the above situation requires this option to be enabled.
> >  
> >  config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
> >  	def_bool y
> > -- 
> > 2.34.1
> >
Conor Dooley July 25, 2023, 10:17 p.m. UTC | #3
On Tue, Jul 25, 2023 at 07:57:54PM +0100, Conor Dooley wrote:
> On Tue, Jul 25, 2023 at 10:23:44AM -0700, Nathan Chancellor wrote:
> > On Wed, Jul 26, 2023 at 01:04:05AM +0800, Mingzheng Xing wrote:
> > > When compiling the kernel with the toolchain composed of GCC >= 12.1.0 and
> > > binutils < 2.38, default ISA spec used when building binutils and GCC, the
> > > following build failure will appear because the
> > > CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI is not turned on.
> > > (i.e, gcc-12.1.0 and binutils-2.36, or gcc-12.3.0 and binutils-2.37, use
> > > default ISA spec.)
> > > 
> > >   CC      arch/riscv/kernel/vdso/vgettimeofday.o
> > >   <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h: Assembler messages:
> > >   <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:79: Error: unrecognized opcode `csrr a5,0xc01'
> > 
> > The gift that keeps on giving :/
> > 
> > > Binutils has updated the default ISA spec version, and the community has
> > > responded well to this[1][2][3], but it appears that this is not over yet.

Also, I just noticed this comment. I disagree with the wording "well",
and more like "with weeping and gnashing of teeth" ;) This stuff is a
huge pain in the ass, and mixing toolchains between LLVM & GNU stuff (or
using an older binutils with a newer GCC) really makes it a lot worse.
Thanks for submitting a fix for this so that Nathan or I didn't have to!

> > > 
> > > We also need to consider the situation of binutils < 2.38 but
> > > GCC >= 12.1.0, since the combination between different versions of GCC and
> > > binutils is not unique, which is to some extent flexible. GCC release
> > > 12.1.0 updated the default ISA spec version in GCC commit[4].
> > 
> > I suspect this combination is not too common because binutils 2.38 came
> > out before GCC 12.1.0 but as you note, it is obviously possible. What
> > toolchain has this combination in the wild, which would be helpful for
> > documentation purposes?
> 
> Yeah, that'd be great to know, at least the other niche stuff that we
> are working around had a clear use-case (testing LLVM in debian containers)
> whereas there's no clear user for this.
> That's doubly interesting, as this patch seems to break things for binutils
> < 2.35, and if we have to make a trade-off between those too, then it'd
> be good to be able to weigh up the options.
> Do we perhaps need the misa-spec workaround instead for this case?
> Haven't tested that though, trying to dig myself out of email backlog.

I don't think the misa-spec stuff is what we need actually. Instead, the
workaround/fix that this patch implements just needs to be constrained to
versions of GAS greater than 2.35.

Thanks,
Conor.

> 
> > > For more information, please refer to:
> > > 
> > > commit 6df2a016c0c8 ("riscv: fix build with binutils 2.38")
> > > commit e89c2e815e76 ("riscv: Handle zicsr/zifencei issues between clang and binutils")
> > > 
> > > [1]: https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4
> > > [2]: https://lore.kernel.org/all/20230308220842.1231003-1-conor@kernel.org
> > > [3]: https://lore.kernel.org/all/20230223220546.52879-1-conor@kernel.org
> > > [4]: https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
> 
> btw, please make these regular Link: tags (with a [N] at EOL) and drop
> the space between them and the sign off. Also, this probably needs to be
> CC: stable@vger.kernel.org too.

> > > 
> > > Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
> > > ---
> > >  arch/riscv/Kconfig | 6 +++++-
> > >  1 file changed, 5 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index 4c07b9189c86..b49cea30f6cc 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -570,11 +570,15 @@ config TOOLCHAIN_HAS_ZIHINTPAUSE
> > >  config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI
> > >  	def_bool y
> > >  	# https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc
> > > -	depends on AS_IS_GNU && AS_VERSION >= 23800
> > > +	# https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
> > > +	depends on CC_IS_GCC && GCC_VERSION >= 120100 || \
> > > +		   AS_IS_GNU && AS_VERSION >= 23800
> > 
> > GCC_VERSION will be 0 for clang, so you don't need the CC_IS_GCC check.
> > With that change, this should be able to stay on one line:
> > 
> >     depends on GCC_VERSION >= 120100 || (AS_IS_GNU && AS_VERSION >= 23800)
> > 
> > >  	help
> > >  	  Newer binutils versions default to ISA spec version 20191213 which
> > >  	  moves some instructions from the I extension to the Zicsr and Zifencei
> > >  	  extensions.
> > > +	  Similarly, GCC release 12.1.0 has changed the default ISA spec version to
> > > +	  20191213, so the above situation requires this option to be enabled.
> > >  
> > >  config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
> > >  	def_bool y
> > > -- 
> > > 2.34.1
> > > 
>
Mingzheng Xing July 26, 2023, 4:48 p.m. UTC | #4
On 7/26/23 01:23, Nathan Chancellor wrote:
> Hi Mingzheng,
>
> Thanks for the patch!
>
> On Wed, Jul 26, 2023 at 01:04:05AM +0800, Mingzheng Xing wrote:
>> When compiling the kernel with the toolchain composed of GCC >= 12.1.0 and
>> binutils < 2.38, default ISA spec used when building binutils and GCC, the
>> following build failure will appear because the
>> CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI is not turned on.
>> (i.e, gcc-12.1.0 and binutils-2.36, or gcc-12.3.0 and binutils-2.37, use
>> default ISA spec.)
>>
>>    CC      arch/riscv/kernel/vdso/vgettimeofday.o
>>    <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h: Assembler messages:
>>    <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:79: Error: unrecognized opcode `csrr a5,0xc01'
> The gift that keeps on giving :/
Yeah, but things will get better :)
>> Binutils has updated the default ISA spec version, and the community has
>> responded well to this[1][2][3], but it appears that this is not over yet.
>>
>> We also need to consider the situation of binutils < 2.38 but
>> GCC >= 12.1.0, since the combination between different versions of GCC and
>> binutils is not unique, which is to some extent flexible. GCC release
>> 12.1.0 updated the default ISA spec version in GCC commit[4].
> I suspect this combination is not too common because binutils 2.38 came
> out before GCC 12.1.0 but as you note, it is obviously possible. What
> toolchain has this combination in the wild, which would be helpful for
> documentation purposes?
Actually, this issue was discovered during the upgrade of the distribution
openEuler for RISC-V. It is a temporary phenomenon caused by
inconsistent upgrade speeds of packages such as GCC and binutils.
 From my limited understanding, GCC and binutils are not strictly
version-bound in some other distributions, so I can't rule out this issue
happening in other scenarios. But once it happens it can cause problems
with compiling the kernel.
>> For more information, please refer to:
>>
>> commit 6df2a016c0c8 ("riscv: fix build with binutils 2.38")
>> commit e89c2e815e76 ("riscv: Handle zicsr/zifencei issues between clang and binutils")
>>
>> [1]:https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4
>> [2]:https://lore.kernel.org/all/20230308220842.1231003-1-conor@kernel.org
>> [3]:https://lore.kernel.org/all/20230223220546.52879-1-conor@kernel.org
>> [4]:https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
>>
>> Signed-off-by: Mingzheng Xing<xingmingzheng@iscas.ac.cn>
>> ---
>>   arch/riscv/Kconfig | 6 +++++-
>>   1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> index 4c07b9189c86..b49cea30f6cc 100644
>> --- a/arch/riscv/Kconfig
>> +++ b/arch/riscv/Kconfig
>> @@ -570,11 +570,15 @@ config TOOLCHAIN_HAS_ZIHINTPAUSE
>>   config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI
>>   	def_bool y
>>   	#https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc
>> -	depends on AS_IS_GNU && AS_VERSION >= 23800
>> +	#https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
>> +	depends on CC_IS_GCC && GCC_VERSION >= 120100 || \
>> +		   AS_IS_GNU && AS_VERSION >= 23800
> GCC_VERSION will be 0 for clang, so you don't need the CC_IS_GCC check.
> With that change, this should be able to stay on one line:
>
>      depends on GCC_VERSION >= 120100 || (AS_IS_GNU && AS_VERSION >= 23800)
OK, I'll change it in v2.

Thanks,
Mingzheng.
>>   	help
>>   	  Newer binutils versions default to ISA spec version 20191213 which
>>   	  moves some instructions from the I extension to the Zicsr and Zifencei
>>   	  extensions.
>> +	  Similarly, GCC release 12.1.0 has changed the default ISA spec version to
>> +	  20191213, so the above situation requires this option to be enabled.
>>   
>>   config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
>>   	def_bool y
>> -- 
>> 2.34.1
>>
> Cheers,
> Nathan
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv
Mingzheng Xing July 26, 2023, 4:53 p.m. UTC | #5
On 7/26/23 02:57, Conor Dooley wrote:
> On Tue, Jul 25, 2023 at 10:23:44AM -0700, Nathan Chancellor wrote:
>> Hi Mingzheng,
>>
>> Thanks for the patch!
>>
>> On Wed, Jul 26, 2023 at 01:04:05AM +0800, Mingzheng Xing wrote:
>>> When compiling the kernel with the toolchain composed of GCC >= 12.1.0 and
>>> binutils < 2.38, default ISA spec used when building binutils and GCC, the
>>> following build failure will appear because the
>>> CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI is not turned on.
>>> (i.e, gcc-12.1.0 and binutils-2.36, or gcc-12.3.0 and binutils-2.37, use
>>> default ISA spec.)
>>>
>>>    CC      arch/riscv/kernel/vdso/vgettimeofday.o
>>>    <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h: Assembler messages:
>>>    <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:79: Error: unrecognized opcode `csrr a5,0xc01'
>> The gift that keeps on giving :/
>>
>>> Binutils has updated the default ISA spec version, and the community has
>>> responded well to this[1][2][3], but it appears that this is not over yet.
>>>
>>> We also need to consider the situation of binutils < 2.38 but
>>> GCC >= 12.1.0, since the combination between different versions of GCC and
>>> binutils is not unique, which is to some extent flexible. GCC release
>>> 12.1.0 updated the default ISA spec version in GCC commit[4].
>> I suspect this combination is not too common because binutils 2.38 came
>> out before GCC 12.1.0 but as you note, it is obviously possible. What
>> toolchain has this combination in the wild, which would be helpful for
>> documentation purposes?
> Yeah, that'd be great to know, at least the other niche stuff that we
> are working around had a clear use-case (testing LLVM in debian containers)
> whereas there's no clear user for this.
> That's doubly interesting, as this patch seems to break things for binutils
> < 2.35, and if we have to make a trade-off between those too, then it'd
> be good to be able to weigh up the options.
> Do we perhaps need the misa-spec workaround instead for this case?
> Haven't tested that though, trying to dig myself out of email backlog.
Well, what I encountered use-case was a temporary thing caused by the
inconsistent pace of distro package upgrades, but it really happened and
took quite a bit of time to explore why. There are sites like [1] that count
the "Successful Builds" between different GCC and binutils releases,
(Though they don't seem to be updated much...), but it seems to indicate
that all kinds of available pairings are possible.

Before replying to the e-mail, I tested some "extreme" cases and things
seemed to be a little clearer.

1. binutils-2.38 and GCC-12.1.0 each changed default ISA spec version,
     updating from 2.2 to 20191213.

2. binutils>=2.38 or GCC>=12.1.0
     when meet any of these it is recommended to turn on
     TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI, the good outweighs
     the bad. (My personal understanding.)

3. (Extreme case) binutils>=2.38 AND GCC<11.1.0
     GCC-11.1.0 starts to support zicsr and zifencei extension for 
-march[2].
     In this case just turn on TOOLCHAIN_NEEDS_OLD_ISA_SPEC.
     (Otherwise compiling the kernel will report an error whether
     TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI is turned on or not.)

4. (Extreme case) GCC>=12.1.0, binutils<2.36
     I tested GCC-12.1.0 + binutils-2.36 and it compiles the kernel 
fine(after
     hitting this patch). Also tested GCC-12.1.0 + binutils2.35 and
     GCC-12.1.0 + binutils-2.34.0, but both pairings gave errors and failed
     to produce a usable toolchain (Default ISA versions used). It seems 
safe
     to assume that GCC-12.1.0 + binutils<2.35 is almost non-existent.

I'm no expert on toolchains, so thanks for correcting me if I'm wrong 
somewhere...

[1] https://wiki.osdev.org/Cross-Compiler_Successful_Builds
[2] 
https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=b03be74bad08c382da47e048007a78fa3fb4ef49
>>> For more information, please refer to:
>>>
>>> commit 6df2a016c0c8 ("riscv: fix build with binutils 2.38")
>>> commit e89c2e815e76 ("riscv: Handle zicsr/zifencei issues between clang and binutils")
>>>
>>> [1]:https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4
>>> [2]:https://lore.kernel.org/all/20230308220842.1231003-1-conor@kernel.org
>>> [3]:https://lore.kernel.org/all/20230223220546.52879-1-conor@kernel.org
>>> [4]:https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
> btw, please make these regular Link: tags (with a [N] at EOL) and drop
> the space between them and the sign off. Also, this probably needs to be
> CC:stable@vger.kernel.org  too.
OK, I'll fix it.

Thanks,
Mingzheng.
> Cheers,
> Conor.
>
>>> Signed-off-by: Mingzheng Xing<xingmingzheng@iscas.ac.cn>
>>> ---
>>>   arch/riscv/Kconfig | 6 +++++-
>>>   1 file changed, 5 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>> index 4c07b9189c86..b49cea30f6cc 100644
>>> --- a/arch/riscv/Kconfig
>>> +++ b/arch/riscv/Kconfig
>>> @@ -570,11 +570,15 @@ config TOOLCHAIN_HAS_ZIHINTPAUSE
>>>   config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI
>>>   	def_bool y
>>>   	#https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc
>>> -	depends on AS_IS_GNU && AS_VERSION >= 23800
>>> +	#https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
>>> +	depends on CC_IS_GCC && GCC_VERSION >= 120100 || \
>>> +		   AS_IS_GNU && AS_VERSION >= 23800
>> GCC_VERSION will be 0 for clang, so you don't need the CC_IS_GCC check.
>> With that change, this should be able to stay on one line:
>>
>>      depends on GCC_VERSION >= 120100 || (AS_IS_GNU && AS_VERSION >= 23800)
>>
>>>   	help
>>>   	  Newer binutils versions default to ISA spec version 20191213 which
>>>   	  moves some instructions from the I extension to the Zicsr and Zifencei
>>>   	  extensions.
>>> +	  Similarly, GCC release 12.1.0 has changed the default ISA spec version to
>>> +	  20191213, so the above situation requires this option to be enabled.
>>>   
>>>   config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
>>>   	def_bool y
>>> -- 
>>> 2.34.1
>>>
Mingzheng Xing July 26, 2023, 4:55 p.m. UTC | #6
On 7/26/23 06:17, Conor Dooley wrote:
> On Tue, Jul 25, 2023 at 07:57:54PM +0100, Conor Dooley wrote:
>> On Tue, Jul 25, 2023 at 10:23:44AM -0700, Nathan Chancellor wrote:
>>> On Wed, Jul 26, 2023 at 01:04:05AM +0800, Mingzheng Xing wrote:
>>>> When compiling the kernel with the toolchain composed of GCC >= 12.1.0 and
>>>> binutils < 2.38, default ISA spec used when building binutils and GCC, the
>>>> following build failure will appear because the
>>>> CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI is not turned on.
>>>> (i.e, gcc-12.1.0 and binutils-2.36, or gcc-12.3.0 and binutils-2.37, use
>>>> default ISA spec.)
>>>>
>>>>    CC      arch/riscv/kernel/vdso/vgettimeofday.o
>>>>    <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h: Assembler messages:
>>>>    <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:79: Error: unrecognized opcode `csrr a5,0xc01'
>>> The gift that keeps on giving :/
>>>
>>>> Binutils has updated the default ISA spec version, and the community has
>>>> responded well to this[1][2][3], but it appears that this is not over yet.
> Also, I just noticed this comment. I disagree with the wording "well",
> and more like "with weeping and gnashing of teeth" ;) This stuff is a
> huge pain in the ass, and mixing toolchains between LLVM & GNU stuff (or
> using an older binutils with a newer GCC) really makes it a lot worse.
At least in the usual way , except in my unusual case...
> Thanks for submitting a fix for this so that Nathan or I didn't have to!
Hopefully this patch will make things better.
>>>> We also need to consider the situation of binutils < 2.38 but
>>>> GCC >= 12.1.0, since the combination between different versions of GCC and
>>>> binutils is not unique, which is to some extent flexible. GCC release
>>>> 12.1.0 updated the default ISA spec version in GCC commit[4].
>>> I suspect this combination is not too common because binutils 2.38 came
>>> out before GCC 12.1.0 but as you note, it is obviously possible. What
>>> toolchain has this combination in the wild, which would be helpful for
>>> documentation purposes?
>> Yeah, that'd be great to know, at least the other niche stuff that we
>> are working around had a clear use-case (testing LLVM in debian containers)
>> whereas there's no clear user for this.
>> That's doubly interesting, as this patch seems to break things for binutils
>> < 2.35, and if we have to make a trade-off between those too, then it'd
>> be good to be able to weigh up the options.
>> Do we perhaps need the misa-spec workaround instead for this case?
>> Haven't tested that though, trying to dig myself out of email backlog.
> I don't think the misa-spec stuff is what we need actually. Instead, the
> workaround/fix that this patch implements just needs to be constrained to
> versions of GAS greater than 2.35.
I'm sorry, I didn't quite understand this, could you provide some more
information, thank you very much!
I'll change the handling in v2. After testing it myself, it seems to 
work fine.

Thanks,
Mingzheng.
> Thanks,
> Conor.
>
>>>> For more information, please refer to:
>>>>
>>>> commit 6df2a016c0c8 ("riscv: fix build with binutils 2.38")
>>>> commit e89c2e815e76 ("riscv: Handle zicsr/zifencei issues between clang and binutils")
>>>>
>>>> [1]:https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4
>>>> [2]:https://lore.kernel.org/all/20230308220842.1231003-1-conor@kernel.org
>>>> [3]:https://lore.kernel.org/all/20230223220546.52879-1-conor@kernel.org
>>>> [4]:https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
>> btw, please make these regular Link: tags (with a [N] at EOL) and drop
>> the space between them and the sign off. Also, this probably needs to be
>> CC:stable@vger.kernel.org  too.
>>>> Signed-off-by: Mingzheng Xing<xingmingzheng@iscas.ac.cn>
>>>> ---
>>>>   arch/riscv/Kconfig | 6 +++++-
>>>>   1 file changed, 5 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>>> index 4c07b9189c86..b49cea30f6cc 100644
>>>> --- a/arch/riscv/Kconfig
>>>> +++ b/arch/riscv/Kconfig
>>>> @@ -570,11 +570,15 @@ config TOOLCHAIN_HAS_ZIHINTPAUSE
>>>>   config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI
>>>>   	def_bool y
>>>>   	#https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc
>>>> -	depends on AS_IS_GNU && AS_VERSION >= 23800
>>>> +	#https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
>>>> +	depends on CC_IS_GCC && GCC_VERSION >= 120100 || \
>>>> +		   AS_IS_GNU && AS_VERSION >= 23800
>>> GCC_VERSION will be 0 for clang, so you don't need the CC_IS_GCC check.
>>> With that change, this should be able to stay on one line:
>>>
>>>      depends on GCC_VERSION >= 120100 || (AS_IS_GNU && AS_VERSION >= 23800)
>>>
>>>>   	help
>>>>   	  Newer binutils versions default to ISA spec version 20191213 which
>>>>   	  moves some instructions from the I extension to the Zicsr and Zifencei
>>>>   	  extensions.
>>>> +	  Similarly, GCC release 12.1.0 has changed the default ISA spec version to
>>>> +	  20191213, so the above situation requires this option to be enabled.
>>>>   
>>>>   config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
>>>>   	def_bool y
>>>> -- 
>>>> 2.34.1
>>>>
diff mbox series

Patch

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 4c07b9189c86..b49cea30f6cc 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -570,11 +570,15 @@  config TOOLCHAIN_HAS_ZIHINTPAUSE
 config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI
 	def_bool y
 	# https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc
-	depends on AS_IS_GNU && AS_VERSION >= 23800
+	# https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=98416dbb0a62579d4a7a4a76bab51b5b52fec2cd
+	depends on CC_IS_GCC && GCC_VERSION >= 120100 || \
+		   AS_IS_GNU && AS_VERSION >= 23800
 	help
 	  Newer binutils versions default to ISA spec version 20191213 which
 	  moves some instructions from the I extension to the Zicsr and Zifencei
 	  extensions.
+	  Similarly, GCC release 12.1.0 has changed the default ISA spec version to
+	  20191213, so the above situation requires this option to be enabled.
 
 config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
 	def_bool y