| Message ID | 20211202143331.15259-1-wangkefeng.wang@huawei.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v3] arm64: Enable KCSAN | expand |
On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote: > > This patch enables KCSAN for arm64, with updates to build rules > to not use KCSAN for several incompatible compilation units. > > Resent GCC version(at least GCC10) made outline-atomics as the s/Resent/Recent/ > default option(unlike Clang), which will cause linker errors > for kernel/kcsan/core.o. Disables the out-of-line atomics by > no-outline-atomics to fix the linker errors. > > Meanwhile, as Mark said[1], there is a specific issue on arm64 > about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed > by Clang 12, add CLANG_VERSION check. And also some latent issues > are need to be fixed which isn't just a KCSAN problem, we make > the KCSAN depends on EXPERT for now. > > Tested selftest and kcsan_test(built with GCC11 and Clang 13), > and all passed. > > [1] https://lkml.org/lkml/2021/12/1/354 Please use lore/kernel.org permalinks. For this one it'd be: https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N (But I think if this is the final version of the patch, hopefully a maintainer can amend the commit message.) > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Acked-by: Marco Elver <elver@google.com> # kernel/kcsan is still valid, given nothing changed there. I leave the rest to Mark. Thanks, -- Marco > --- > Tested on Qemu with clang 13 / gcc 11, based on 5.16-rc3. > > [ 0.221518] kcsan: enabled early > [ 0.222422] kcsan: strict mode configured > ... > [ 5.839223] kcsan: selftest: 3/3 tests passed > ... > [ 517.895102] # kcsan: pass:24 fail:0 skip:0 total:24 > [ 517.896393] # Totals: pass:168 fail:0 skip:0 total:168 > [ 517.897502] ok 1 - kcsan > > v3: > - add EXPERT and CLANG_VERSION depends suggested by Mark Rutland > v2: > - tested on GCC11 and disable outline-atomics for kernel/kcsan/core.c > suggested by Marco Elver > > arch/arm64/Kconfig | 1 + > arch/arm64/kernel/vdso/Makefile | 1 + > arch/arm64/kvm/hyp/nvhe/Makefile | 1 + > kernel/kcsan/Makefile | 1 + > 4 files changed, 4 insertions(+) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index 4ff73299f8a9..3a7b17742cde 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -150,6 +150,7 @@ config ARM64 > select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN > select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN > select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE) > + select HAVE_ARCH_KCSAN if EXPERT && (CC_IS_GCC || CLANG_VERSION >= 120000) > select HAVE_ARCH_KFENCE > select HAVE_ARCH_KGDB > select HAVE_ARCH_MMAP_RND_BITS > diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile > index 700767dfd221..60813497a381 100644 > --- a/arch/arm64/kernel/vdso/Makefile > +++ b/arch/arm64/kernel/vdso/Makefile > @@ -32,6 +32,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO > CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS) \ > $(CC_FLAGS_LTO) > KASAN_SANITIZE := n > +KCSAN_SANITIZE := n > UBSAN_SANITIZE := n > OBJECT_FILES_NON_STANDARD := y > KCOV_INSTRUMENT := n > diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile > index c3c11974fa3b..24b2c2425b38 100644 > --- a/arch/arm64/kvm/hyp/nvhe/Makefile > +++ b/arch/arm64/kvm/hyp/nvhe/Makefile > @@ -89,6 +89,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI) > # cause crashes. Just disable it. > GCOV_PROFILE := n > KASAN_SANITIZE := n > +KCSAN_SANITIZE := n > UBSAN_SANITIZE := n > KCOV_INSTRUMENT := n > > diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile > index c2bb07f5bcc7..e893b0e1d62a 100644 > --- a/kernel/kcsan/Makefile > +++ b/kernel/kcsan/Makefile > @@ -8,6 +8,7 @@ CFLAGS_REMOVE_debugfs.o = $(CC_FLAGS_FTRACE) > CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE) > > CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \ > + $(call cc-option,-mno-outline-atomics) \ > -fno-stack-protector -DDISABLE_BRANCH_PROFILING > > obj-y := core.o debugfs.o report.o > -- > 2.26.2 >
On Thu, Dec 02, 2021 at 03:36:06PM +0100, Marco Elver wrote: > On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote: > > > > This patch enables KCSAN for arm64, with updates to build rules > > to not use KCSAN for several incompatible compilation units. > > > > Resent GCC version(at least GCC10) made outline-atomics as the > > s/Resent/Recent/ > > > default option(unlike Clang), which will cause linker errors > > for kernel/kcsan/core.o. Disables the out-of-line atomics by > > no-outline-atomics to fix the linker errors. > > > > Meanwhile, as Mark said[1], there is a specific issue on arm64 > > about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed > > by Clang 12, add CLANG_VERSION check. And also some latent issues > > are need to be fixed which isn't just a KCSAN problem, we make > > the KCSAN depends on EXPERT for now. > > > > Tested selftest and kcsan_test(built with GCC11 and Clang 13), > > and all passed. > > > > [1] https://lkml.org/lkml/2021/12/1/354 > > Please use lore/kernel.org permalinks. For this one it'd be: > https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N > > (But I think if this is the final version of the patch, hopefully a > maintainer can amend the commit message.) > > > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> > > Acked-by: Marco Elver <elver@google.com> # kernel/kcsan > > is still valid, given nothing changed there. I leave the rest to Mark. I want to do some compiler / config testing before I ack this (just to make sure there isn't some latent issue I've forgotten about), but otherwise I think this should be fine. I'll try to have that done in the next few days. Any other review/testing would be appreciated! Thanks, Mark. > > Thanks, > -- Marco > > > --- > > Tested on Qemu with clang 13 / gcc 11, based on 5.16-rc3. > > > > [ 0.221518] kcsan: enabled early > > [ 0.222422] kcsan: strict mode configured > > ... > > [ 5.839223] kcsan: selftest: 3/3 tests passed > > ... > > [ 517.895102] # kcsan: pass:24 fail:0 skip:0 total:24 > > [ 517.896393] # Totals: pass:168 fail:0 skip:0 total:168 > > [ 517.897502] ok 1 - kcsan > > > > v3: > > - add EXPERT and CLANG_VERSION depends suggested by Mark Rutland > > v2: > > - tested on GCC11 and disable outline-atomics for kernel/kcsan/core.c > > suggested by Marco Elver > > > > arch/arm64/Kconfig | 1 + > > arch/arm64/kernel/vdso/Makefile | 1 + > > arch/arm64/kvm/hyp/nvhe/Makefile | 1 + > > kernel/kcsan/Makefile | 1 + > > 4 files changed, 4 insertions(+) > > > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > > index 4ff73299f8a9..3a7b17742cde 100644 > > --- a/arch/arm64/Kconfig > > +++ b/arch/arm64/Kconfig > > @@ -150,6 +150,7 @@ config ARM64 > > select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN > > select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN > > select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE) > > + select HAVE_ARCH_KCSAN if EXPERT && (CC_IS_GCC || CLANG_VERSION >= 120000) > > select HAVE_ARCH_KFENCE > > select HAVE_ARCH_KGDB > > select HAVE_ARCH_MMAP_RND_BITS > > diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile > > index 700767dfd221..60813497a381 100644 > > --- a/arch/arm64/kernel/vdso/Makefile > > +++ b/arch/arm64/kernel/vdso/Makefile > > @@ -32,6 +32,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO > > CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS) \ > > $(CC_FLAGS_LTO) > > KASAN_SANITIZE := n > > +KCSAN_SANITIZE := n > > UBSAN_SANITIZE := n > > OBJECT_FILES_NON_STANDARD := y > > KCOV_INSTRUMENT := n > > diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile > > index c3c11974fa3b..24b2c2425b38 100644 > > --- a/arch/arm64/kvm/hyp/nvhe/Makefile > > +++ b/arch/arm64/kvm/hyp/nvhe/Makefile > > @@ -89,6 +89,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI) > > # cause crashes. Just disable it. > > GCOV_PROFILE := n > > KASAN_SANITIZE := n > > +KCSAN_SANITIZE := n > > UBSAN_SANITIZE := n > > KCOV_INSTRUMENT := n > > > > diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile > > index c2bb07f5bcc7..e893b0e1d62a 100644 > > --- a/kernel/kcsan/Makefile > > +++ b/kernel/kcsan/Makefile > > @@ -8,6 +8,7 @@ CFLAGS_REMOVE_debugfs.o = $(CC_FLAGS_FTRACE) > > CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE) > > > > CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \ > > + $(call cc-option,-mno-outline-atomics) \ > > -fno-stack-protector -DDISABLE_BRANCH_PROFILING > > > > obj-y := core.o debugfs.o report.o > > -- > > 2.26.2 > >
Hi all, On Thu, Dec 02, 2021 at 10:33:31PM +0800, Kefeng Wang wrote: > This patch enables KCSAN for arm64, with updates to build rules > to not use KCSAN for several incompatible compilation units. > > Resent GCC version(at least GCC10) made outline-atomics as the > default option(unlike Clang), which will cause linker errors > for kernel/kcsan/core.o. Disables the out-of-line atomics by > no-outline-atomics to fix the linker errors. > > Meanwhile, as Mark said[1], there is a specific issue on arm64 > about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed > by Clang 12, add CLANG_VERSION check. And also some latent issues > are need to be fixed which isn't just a KCSAN problem, we make > the KCSAN depends on EXPERT for now. > > Tested selftest and kcsan_test(built with GCC11 and Clang 13), > and all passed. > > [1] https://lkml.org/lkml/2021/12/1/354 > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Confirming that I get KCSAN errors such as: [ 285.188576] ================================================================== [ 285.189514] BUG: KCSAN: data-race in el0_svc_common.constprop.0+0x84/0x14c [ 285.190599] [ 285.191092] race at unknown origin, with read to 0xffff0000e3ce3700 of 8 bytes by task 19946 on cpu 5: [ 285.192576] el0_svc_common.constprop.0+0x84/0x14c [ 285.193601] do_el0_svc+0x30/0x40 [ 285.194457] el0_svc+0x3c/0x70 [ 285.194954] el0t_64_sync_handler+0x9c/0x120 [ 285.195618] el0t_64_sync+0x18c/0x190 [ 285.196371] [ 285.196823] value changed: 0x0000000000000008 -> 0x000000000000000a [ 285.197674] [ 285.198172] Reported by Kernel Concurrency Sanitizer on: [ 285.198941] CPU: 5 PID: 19946 Comm: stress-ng Not tainted 5.16.0-rc3+ #842 95517f70e70f029451088b9dec0b9074aa29ae52 [ 285.201191] ================================================================== This particular case is fixed by Mark's thread flag series [1], which is in linux-next now. Tested with gcc 11 and clang 14.0 (built from git) on qemu and FVP. Tested-by: Joey Gouly <joey.gouly@arm.com> Thanks, Joey [1] https://lore.kernel.org/lkml/20211129130653.2037928-1-mark.rutland@arm.com/
On Thu, Dec 02, 2021 at 10:33:31PM +0800, Kefeng Wang wrote: > This patch enables KCSAN for arm64, with updates to build rules > to not use KCSAN for several incompatible compilation units. > > Resent GCC version(at least GCC10) made outline-atomics as the > default option(unlike Clang), which will cause linker errors > for kernel/kcsan/core.o. Disables the out-of-line atomics by > no-outline-atomics to fix the linker errors. > > Meanwhile, as Mark said[1], there is a specific issue on arm64 > about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed > by Clang 12, add CLANG_VERSION check. And also some latent issues I do not think this addition is necessary because we make BTI depend on clang 12 or newer after commit 8cdd23c23c3d ("arm64: Restrict ARM64_BTI_KERNEL to clang 12.0.0 and newer") upstream. I figured it was better to go after the root cause rather than add a bunch of version checks like this to the various sanitizers. > are need to be fixed which isn't just a KCSAN problem, we make > the KCSAN depends on EXPERT for now. > > Tested selftest and kcsan_test(built with GCC11 and Clang 13), > and all passed. > > [1] https://lkml.org/lkml/2021/12/1/354 > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> > --- > Tested on Qemu with clang 13 / gcc 11, based on 5.16-rc3. > > [ 0.221518] kcsan: enabled early > [ 0.222422] kcsan: strict mode configured > ... > [ 5.839223] kcsan: selftest: 3/3 tests passed > ... > [ 517.895102] # kcsan: pass:24 fail:0 skip:0 total:24 > [ 517.896393] # Totals: pass:168 fail:0 skip:0 total:168 > [ 517.897502] ok 1 - kcsan > > v3: > - add EXPERT and CLANG_VERSION depends suggested by Mark Rutland > v2: > - tested on GCC11 and disable outline-atomics for kernel/kcsan/core.c > suggested by Marco Elver > > arch/arm64/Kconfig | 1 + > arch/arm64/kernel/vdso/Makefile | 1 + > arch/arm64/kvm/hyp/nvhe/Makefile | 1 + > kernel/kcsan/Makefile | 1 + > 4 files changed, 4 insertions(+) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index 4ff73299f8a9..3a7b17742cde 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -150,6 +150,7 @@ config ARM64 > select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN > select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN > select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE) > + select HAVE_ARCH_KCSAN if EXPERT && (CC_IS_GCC || CLANG_VERSION >= 120000) > select HAVE_ARCH_KFENCE > select HAVE_ARCH_KGDB > select HAVE_ARCH_MMAP_RND_BITS > diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile > index 700767dfd221..60813497a381 100644 > --- a/arch/arm64/kernel/vdso/Makefile > +++ b/arch/arm64/kernel/vdso/Makefile > @@ -32,6 +32,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO > CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS) \ > $(CC_FLAGS_LTO) > KASAN_SANITIZE := n > +KCSAN_SANITIZE := n > UBSAN_SANITIZE := n > OBJECT_FILES_NON_STANDARD := y > KCOV_INSTRUMENT := n > diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile > index c3c11974fa3b..24b2c2425b38 100644 > --- a/arch/arm64/kvm/hyp/nvhe/Makefile > +++ b/arch/arm64/kvm/hyp/nvhe/Makefile > @@ -89,6 +89,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI) > # cause crashes. Just disable it. > GCOV_PROFILE := n > KASAN_SANITIZE := n > +KCSAN_SANITIZE := n > UBSAN_SANITIZE := n > KCOV_INSTRUMENT := n > > diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile > index c2bb07f5bcc7..e893b0e1d62a 100644 > --- a/kernel/kcsan/Makefile > +++ b/kernel/kcsan/Makefile > @@ -8,6 +8,7 @@ CFLAGS_REMOVE_debugfs.o = $(CC_FLAGS_FTRACE) > CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE) > > CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \ > + $(call cc-option,-mno-outline-atomics) \ > -fno-stack-protector -DDISABLE_BRANCH_PROFILING > > obj-y := core.o debugfs.o report.o > -- > 2.26.2 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On 2021/12/2 22:44, Mark Rutland wrote: > On Thu, Dec 02, 2021 at 03:36:06PM +0100, Marco Elver wrote: >> On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote: >>> This patch enables KCSAN for arm64, with updates to build rules >>> to not use KCSAN for several incompatible compilation units. >>> >>> Resent GCC version(at least GCC10) made outline-atomics as the >> s/Resent/Recent/ >> >>> default option(unlike Clang), which will cause linker errors >>> for kernel/kcsan/core.o. Disables the out-of-line atomics by >>> no-outline-atomics to fix the linker errors. >>> >>> Meanwhile, as Mark said[1], there is a specific issue on arm64 >>> about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed >>> by Clang 12, add CLANG_VERSION check. And also some latent issues >>> are need to be fixed which isn't just a KCSAN problem, we make >>> the KCSAN depends on EXPERT for now. >>> >>> Tested selftest and kcsan_test(built with GCC11 and Clang 13), >>> and all passed. >>> >>> [1] https://lkml.org/lkml/2021/12/1/354 >> Please use lore/kernel.org permalinks. For this one it'd be: >> https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N >> >> (But I think if this is the final version of the patch, hopefully a >> maintainer can amend the commit message.) >> >>> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> >> Acked-by: Marco Elver <elver@google.com> # kernel/kcsan >> >> is still valid, given nothing changed there. I leave the rest to Mark. > I want to do some compiler / config testing before I ack this (just to > make sure there isn't some latent issue I've forgotten about), but > otherwise I think this should be fine. > > I'll try to have that done in the next few days. I will wait for some time, thanks Marco/Mark. > > Any other review/testing would be appreciated! As Nathan points, commit 8cdd23c23c3d ("arm64: Restrict ARM64_BTI_KERNEL to clang 12.0.0 and newer"), so need to add Clang version check, which is v2 does. is there some other requirement that we need this check, what's your option, thanks. > > Thanks, > Mark. > >
On Fri, Dec 03, 2021 at 10:29:22AM +0800, Kefeng Wang wrote: > > On 2021/12/2 22:44, Mark Rutland wrote: > > On Thu, Dec 02, 2021 at 03:36:06PM +0100, Marco Elver wrote: > > > On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote: > > > > This patch enables KCSAN for arm64, with updates to build rules > > > > to not use KCSAN for several incompatible compilation units. > > > > > > > > Resent GCC version(at least GCC10) made outline-atomics as the > > > s/Resent/Recent/ > > > > > > > default option(unlike Clang), which will cause linker errors > > > > for kernel/kcsan/core.o. Disables the out-of-line atomics by > > > > no-outline-atomics to fix the linker errors. > > > > > > > > Meanwhile, as Mark said[1], there is a specific issue on arm64 > > > > about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed > > > > by Clang 12, add CLANG_VERSION check. And also some latent issues > > > > are need to be fixed which isn't just a KCSAN problem, we make > > > > the KCSAN depends on EXPERT for now. > > > > > > > > Tested selftest and kcsan_test(built with GCC11 and Clang 13), > > > > and all passed. > > > > > > > > [1] https://lkml.org/lkml/2021/12/1/354 > > > Please use lore/kernel.org permalinks. For this one it'd be: > > > https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N > > > > > > (But I think if this is the final version of the patch, hopefully a > > > maintainer can amend the commit message.) > > > > > > > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> > > > Acked-by: Marco Elver <elver@google.com> # kernel/kcsan > > > > > > is still valid, given nothing changed there. I leave the rest to Mark. > > I want to do some compiler / config testing before I ack this (just to > > make sure there isn't some latent issue I've forgotten about), but > > otherwise I think this should be fine. > > > > I'll try to have that done in the next few days. > > I will wait for some time, thanks Marco/Mark. > > > > > Any other review/testing would be appreciated! > > As Nathan points, commit 8cdd23c23c3d ("arm64: Restrict ARM64_BTI_KERNEL > > to clang 12.0.0 and newer"), so need to add Clang version check, which is v2 > does. > > is there some other requirement that we need this check, what's your option, I'm not immediately aware of another reason, so I'm fine with not checking that for KCSAN -- If I discover another reason I will let you know. Thanks, Mark.
On 2021/12/3 18:41, Mark Rutland wrote: > On Fri, Dec 03, 2021 at 10:29:22AM +0800, Kefeng Wang wrote: >> On 2021/12/2 22:44, Mark Rutland wrote: >>> On Thu, Dec 02, 2021 at 03:36:06PM +0100, Marco Elver wrote: >>>> On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote: >>>>> This patch enables KCSAN for arm64, with updates to build rules >>>>> to not use KCSAN for several incompatible compilation units. >>>>> >>>>> Resent GCC version(at least GCC10) made outline-atomics as the >>>> s/Resent/Recent/ >>>> >>>>> default option(unlike Clang), which will cause linker errors >>>>> for kernel/kcsan/core.o. Disables the out-of-line atomics by >>>>> no-outline-atomics to fix the linker errors. >>>>> >>>>> Meanwhile, as Mark said[1], there is a specific issue on arm64 >>>>> about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed >>>>> by Clang 12, add CLANG_VERSION check. And also some latent issues >>>>> are need to be fixed which isn't just a KCSAN problem, we make >>>>> the KCSAN depends on EXPERT for now. >>>>> >>>>> Tested selftest and kcsan_test(built with GCC11 and Clang 13), >>>>> and all passed. >>>>> >>>>> [1] https://lkml.org/lkml/2021/12/1/354 >>>> Please use lore/kernel.org permalinks. For this one it'd be: >>>> https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N >>>> >>>> (But I think if this is the final version of the patch, hopefully a >>>> maintainer can amend the commit message.) >>>> >>>>> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> >>>> Acked-by: Marco Elver <elver@google.com> # kernel/kcsan >>>> >>>> is still valid, given nothing changed there. I leave the rest to Mark. >>> I want to do some compiler / config testing before I ack this (just to >>> make sure there isn't some latent issue I've forgotten about), but >>> otherwise I think this should be fine. >>> >>> I'll try to have that done in the next few days. >> I will wait for some time, thanks Marco/Mark. Hi Mark, I will send v4 to address the comments in v3, is there new update about this? >> >>> Any other review/testing would be appreciated! >> As Nathan points, commit 8cdd23c23c3d ("arm64: Restrict ARM64_BTI_KERNEL >> >> to clang 12.0.0 and newer"), so need to add Clang version check, which is v2 >> does. >> >> is there some other requirement that we need this check, what's your option, > I'm not immediately aware of another reason, so I'm fine with not checking that > for KCSAN -- If I discover another reason I will let you know. Ok, will drop the Clang version check in v4. Thanks. > > Thanks, > Mark. > .
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 4ff73299f8a9..3a7b17742cde 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -150,6 +150,7 @@ config ARM64 select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE) + select HAVE_ARCH_KCSAN if EXPERT && (CC_IS_GCC || CLANG_VERSION >= 120000) select HAVE_ARCH_KFENCE select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile index 700767dfd221..60813497a381 100644 --- a/arch/arm64/kernel/vdso/Makefile +++ b/arch/arm64/kernel/vdso/Makefile @@ -32,6 +32,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS) \ $(CC_FLAGS_LTO) KASAN_SANITIZE := n +KCSAN_SANITIZE := n UBSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y KCOV_INSTRUMENT := n diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index c3c11974fa3b..24b2c2425b38 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -89,6 +89,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI) # cause crashes. Just disable it. GCOV_PROFILE := n KASAN_SANITIZE := n +KCSAN_SANITIZE := n UBSAN_SANITIZE := n KCOV_INSTRUMENT := n diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile index c2bb07f5bcc7..e893b0e1d62a 100644 --- a/kernel/kcsan/Makefile +++ b/kernel/kcsan/Makefile @@ -8,6 +8,7 @@ CFLAGS_REMOVE_debugfs.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE) CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \ + $(call cc-option,-mno-outline-atomics) \ -fno-stack-protector -DDISABLE_BRANCH_PROFILING obj-y := core.o debugfs.o report.o
This patch enables KCSAN for arm64, with updates to build rules to not use KCSAN for several incompatible compilation units. Resent GCC version(at least GCC10) made outline-atomics as the default option(unlike Clang), which will cause linker errors for kernel/kcsan/core.o. Disables the out-of-line atomics by no-outline-atomics to fix the linker errors. Meanwhile, as Mark said[1], there is a specific issue on arm64 about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed by Clang 12, add CLANG_VERSION check. And also some latent issues are need to be fixed which isn't just a KCSAN problem, we make the KCSAN depends on EXPERT for now. Tested selftest and kcsan_test(built with GCC11 and Clang 13), and all passed. [1] https://lkml.org/lkml/2021/12/1/354 Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> --- Tested on Qemu with clang 13 / gcc 11, based on 5.16-rc3. [ 0.221518] kcsan: enabled early [ 0.222422] kcsan: strict mode configured ... [ 5.839223] kcsan: selftest: 3/3 tests passed ... [ 517.895102] # kcsan: pass:24 fail:0 skip:0 total:24 [ 517.896393] # Totals: pass:168 fail:0 skip:0 total:168 [ 517.897502] ok 1 - kcsan v3: - add EXPERT and CLANG_VERSION depends suggested by Mark Rutland v2: - tested on GCC11 and disable outline-atomics for kernel/kcsan/core.c suggested by Marco Elver arch/arm64/Kconfig | 1 + arch/arm64/kernel/vdso/Makefile | 1 + arch/arm64/kvm/hyp/nvhe/Makefile | 1 + kernel/kcsan/Makefile | 1 + 4 files changed, 4 insertions(+)