mbox series

[v6,0/6] KASan for arm

Message ID 20190617221134.9930-1-f.fainelli@gmail.com (mailing list archive)
Headers show
Series KASan for arm | expand

Message

Florian Fainelli June 17, 2019, 10:11 p.m. UTC
Hi all,

Abbott submitted a v5 about a year ago here:

and the series was not picked up since then, so I rebased it against
v5.2-rc4 and re-tested it on a Brahma-B53 (ARMv8 running AArch32 mode)
and Brahma-B15, both LPAE and test-kasan is consistent with the ARM64
counter part.

We were in a fairly good shape last time with a few different people
having tested it, so I am hoping we can get that included for 5.4 if
everything goes well.

Changelog:

v6 - v5
- Resolve conflicts during rebase, and updated to make use of
  kasan_early_shadow_pte instead of kasan_zero_pte

v5 - v4
- Modify Andrey Ryabinin's email address.

v4 - v3
- Remove the fix of type conversion in kasan_cache_create because it has
  been fix in the latest version in:
  git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
- Change some Reviewed-by tag into Reported-by tag to avoid misleading.
  ---Reported by: Marc Zyngier <marc.zyngier@arm.com>
                  Russell King - ARM Linux <linux@armlinux.org.uk>
- Disable instrumentation for arch/arm/mm/physaddr.c

v3 - v2
- Remove this patch: 2 1-byte checks more safer for memory_is_poisoned_16
  because a unaligned load/store of 16 bytes is rare on arm, and this
  patch is very likely to affect the performance of modern CPUs.
  ---Acked by: Russell King - ARM Linux <linux@armlinux.org.uk>
- Fixed some link error which kasan_pmd_populate,kasan_pte_populate and
  kasan_pud_populate are in section .meminit.text but the function
  kasan_alloc_block which is called by kasan_pmd_populate,
  kasan_pte_populate and kasan_pud_populate is in section .init.text. So
  we need change kasan_pmd_populate,kasan_pte_populate and
  kasan_pud_populate into the section .init.text.
  ---Reported by: Florian Fainelli <f.fainelli@gmail.com>
- Fixed some compile error which caused by the wrong access instruction in
  arch/arm/kernel/entry-common.S.
  ---Reported by: kbuild test robot <lkp@intel.com>
- Disable instrumentation for arch/arm/kvm/hyp/*.
  ---Acked by: Marc Zyngier <marc.zyngier@arm.com>
- Update the set of supported architectures in
  Documentation/dev-tools/kasan.rst.
  ---Acked by:Dmitry Vyukov <dvyukov@google.com>
- The version 2 is tested by:
  Florian Fainelli <f.fainelli@gmail.com> (compile test)
  kbuild test robot <lkp@intel.com>       (compile test)
  Joel Stanley <joel@jms.id.au>           (on ASPEED ast2500(ARMv5))

v2 - v1
- Fixed some compiling error which happens on changing kernel compression
  mode to lzma/xz/lzo/lz4.
  ---Reported by: Florian Fainelli <f.fainelli@gmail.com>,
             Russell King - ARM Linux <linux@armlinux.org.uk>
- Fixed a compiling error cause by some older arm instruction set(armv4t)
  don't suppory movw/movt which is reported by kbuild.
- Changed the pte flag from _L_PTE_DEFAULT | L_PTE_DIRTY | L_PTE_XN to
  pgprot_val(PAGE_KERNEL).
  ---Reported by: Russell King - ARM Linux <linux@armlinux.org.uk>
- Moved Enable KASan patch as the last one.
  ---Reported by: Florian Fainelli <f.fainelli@gmail.com>,
     Russell King - ARM Linux <linux@armlinux.org.uk>
- Moved the definitions of cp15 registers from
  arch/arm/include/asm/kvm_hyp.h to arch/arm/include/asm/cp15.h.
  ---Asked by: Mark Rutland <mark.rutland@arm.com>
- Merge the following commits into the commit
  Define the virtual space of KASan's shadow region:
  1) Define the virtual space of KASan's shadow region;
  2) Avoid cleaning the KASan shadow area's mapping table;
  3) Add KASan layout;
- Merge the following commits into the commit
  Initialize the mapping of KASan shadow memory:
  1) Initialize the mapping of KASan shadow memory;
  2) Add support arm LPAE;
  3) Don't need to map the shadow of KASan's shadow memory;
     ---Reported by: Russell King - ARM Linux <linux@armlinux.org.uk>
  4) Change mapping of kasan_zero_page int readonly.
- The version 1 is tested by Florian Fainelli <f.fainelli@gmail.com>
  on a Cortex-A5 (no LPAE).

Hi,all:
   These patches add arch specific code for kernel address sanitizer
(see Documentation/kasan.txt).

   1/8 of kernel addresses reserved for shadow memory. There was no
big enough hole for this, so virtual addresses for shadow were
stolen from user space.

   At early boot stage the whole shadow region populated with just
one physical page (kasan_zero_page). Later, this page reused
as readonly zero shadow for some memory that KASan currently
don't track (vmalloc).

  After mapping the physical memory, pages for shadow memory are
allocated and mapped.

  KASan's stack instrumentation significantly increases stack's
consumption, so CONFIG_KASAN doubles THREAD_SIZE.

  Functions like memset/memmove/memcpy do a lot of memory accesses.
If bad pointer passed to one of these function it is important
to catch this. Compiler's instrumentation cannot do this since
these functions are written in assembly.

  KASan replaces memory functions with manually instrumented variants.
Original functions declared as weak symbols so strong definitions
in mm/kasan/kasan.c could replace them. Original functions have aliases
with '__' prefix in name, so we could call non-instrumented variant
if needed.

  Some files built without kasan instrumentation (e.g. mm/slub.c).
Original mem* function replaced (via #define) with prefixed variants
to disable memory access checks for such files.

  On arm LPAE architecture,  the mapping table of KASan shadow memory(if
PAGE_OFFSET is 0xc0000000, the KASan shadow memory's virtual space is
0xb6e000000~0xbf000000) can't be filled in do_translation_fault function,
because kasan instrumentation maybe cause do_translation_fault function
accessing KASan shadow memory. The accessing of KASan shadow memory in
do_translation_fault function maybe cause dead circle. So the mapping table
of KASan shadow memory need be copyed in pgd_alloc function.

Most of the code comes from:
https://github.com/aryabinin/linux/commit/0b54f17e70ff50a902c4af05bb92716eb95acefe

These patches are tested on vexpress-ca15, vexpress-ca9


Abbott Liu (2):
  ARM: Add TTBR operator for kasan_init
  ARM: Define the virtual space of KASan's shadow region

Andrey Ryabinin (4):
  ARM: Disable instrumentation for some code
  ARM: Replace memory function for kasan
  ARM: Initialize the mapping of KASan shadow memory
  ARM: Enable KASan for arm

 Documentation/dev-tools/kasan.rst     |   4 +-
 arch/arm/Kconfig                      |   1 +
 arch/arm/boot/compressed/Makefile     |   1 +
 arch/arm/boot/compressed/decompress.c |   2 +
 arch/arm/boot/compressed/libfdt_env.h |   2 +
 arch/arm/include/asm/cp15.h           | 106 +++++++++
 arch/arm/include/asm/kasan.h          |  35 +++
 arch/arm/include/asm/kasan_def.h      |  64 ++++++
 arch/arm/include/asm/kvm_hyp.h        |  54 -----
 arch/arm/include/asm/memory.h         |   5 +
 arch/arm/include/asm/pgalloc.h        |   7 +-
 arch/arm/include/asm/string.h         |  17 ++
 arch/arm/include/asm/thread_info.h    |   4 +
 arch/arm/kernel/entry-armv.S          |   5 +-
 arch/arm/kernel/entry-common.S        |   9 +-
 arch/arm/kernel/head-common.S         |   7 +-
 arch/arm/kernel/setup.c               |   2 +
 arch/arm/kernel/unwind.c              |   3 +-
 arch/arm/kvm/hyp/cp15-sr.c            |  12 +-
 arch/arm/kvm/hyp/switch.c             |   6 +-
 arch/arm/lib/memcpy.S                 |   3 +
 arch/arm/lib/memmove.S                |   5 +-
 arch/arm/lib/memset.S                 |   3 +
 arch/arm/mm/Makefile                  |   4 +
 arch/arm/mm/kasan_init.c              | 301 ++++++++++++++++++++++++++
 arch/arm/mm/mmu.c                     |   7 +-
 arch/arm/mm/pgd.c                     |  14 ++
 arch/arm/vdso/Makefile                |   2 +
 28 files changed, 608 insertions(+), 77 deletions(-)
 create mode 100644 arch/arm/include/asm/kasan.h
 create mode 100644 arch/arm/include/asm/kasan_def.h
 create mode 100644 arch/arm/mm/kasan_init.c

Comments

Linus Walleij July 2, 2019, 9:06 p.m. UTC | #1
Hi Florian,

On Tue, Jun 18, 2019 at 12:11 AM Florian Fainelli <f.fainelli@gmail.com> wrote:

> Abbott submitted a v5 about a year ago here:
>
> and the series was not picked up since then, so I rebased it against
> v5.2-rc4 and re-tested it on a Brahma-B53 (ARMv8 running AArch32 mode)
> and Brahma-B15, both LPAE and test-kasan is consistent with the ARM64
> counter part.
>
> We were in a fairly good shape last time with a few different people
> having tested it, so I am hoping we can get that included for 5.4 if
> everything goes well.

Thanks for picking this up. I was trying out KASan in the past,
got sidetracked and honestly lost interest a bit because it was
boring. But I do realize that it is really neat, so I will try to help
out with some review and test on a bunch of hardware I have.

At one point I even had this running on the ARMv4 SA1100
(no joke!) and if I recall correctly, I got stuck because of things
that might very well have been related to using a very fragile
Arm testchip that later broke down completely in the l2cache
when we added the spectre/meltdown fixes.

I start reviewing and testing.

Yours,
Linus Walleij
Florian Fainelli July 11, 2019, 5 p.m. UTC | #2
On 7/2/19 2:06 PM, Linus Walleij wrote:
> Hi Florian,
> 
> On Tue, Jun 18, 2019 at 12:11 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
>> Abbott submitted a v5 about a year ago here:
>>
>> and the series was not picked up since then, so I rebased it against
>> v5.2-rc4 and re-tested it on a Brahma-B53 (ARMv8 running AArch32 mode)
>> and Brahma-B15, both LPAE and test-kasan is consistent with the ARM64
>> counter part.
>>
>> We were in a fairly good shape last time with a few different people
>> having tested it, so I am hoping we can get that included for 5.4 if
>> everything goes well.
> 
> Thanks for picking this up. I was trying out KASan in the past,
> got sidetracked and honestly lost interest a bit because it was
> boring. But I do realize that it is really neat, so I will try to help
> out with some review and test on a bunch of hardware I have.
> 
> At one point I even had this running on the ARMv4 SA1100
> (no joke!) and if I recall correctly, I got stuck because of things
> that might very well have been related to using a very fragile
> Arm testchip that later broke down completely in the l2cache
> when we added the spectre/meltdown fixes.

A blast from the past!

> 
> I start reviewing and testing.

Great, thanks a lot for taking a look. FYI, I will be on holiday from
July 19th till August 12th, if you think you have more feedback between
now and then, I can try to pick it up and submit a v7 with that feedback
addressed, or it will happen when I return, or you can pick it up if you
refer, all options are possible!

@Arnd, should we squash your patches in as well?
Arnd Bergmann July 18, 2019, 7:51 a.m. UTC | #3
On Thu, Jul 11, 2019 at 7:00 PM Florian Fainelli
<florian.fainelli@broadcom.com> wrote:
> On 7/2/19 2:06 PM, Linus Walleij wrote:

>
> Great, thanks a lot for taking a look. FYI, I will be on holiday from
> July 19th till August 12th, if you think you have more feedback between
> now and then, I can try to pick it up and submit a v7 with that feedback
> addressed, or it will happen when I return, or you can pick it up if you
> refer, all options are possible!
>
> @Arnd, should we squash your patches in as well?

Yes, please do. I don't remember if I sent you all of them already,
here is the list of patches that I have applied locally on top of your
series to get a clean randconfig build:

123c3262f872 KASAN: push back KASAN_STACK to clang-10
d63dd9e2afd9 [HACK] ARM: disable KASAN+XIP_KERNEL
879eb3c22240 kasan: increase 32-bit stack frame warning limit
053555034bdf kasan: disable CONFIG_KASAN_STACK with clang on arm32
6c1a78a448c2 ARM: fix kasan link failures

      Arnd
Florian Fainelli Oct. 7, 2019, 9:34 p.m. UTC | #4
On 7/18/19 12:51 AM, Arnd Bergmann wrote:
> On Thu, Jul 11, 2019 at 7:00 PM Florian Fainelli
> <florian.fainelli@broadcom.com> wrote:
>> On 7/2/19 2:06 PM, Linus Walleij wrote:
> 
>>
>> Great, thanks a lot for taking a look. FYI, I will be on holiday from
>> July 19th till August 12th, if you think you have more feedback between
>> now and then, I can try to pick it up and submit a v7 with that feedback
>> addressed, or it will happen when I return, or you can pick it up if you
>> refer, all options are possible!
>>
>> @Arnd, should we squash your patches in as well?
> 
> Yes, please do. I don't remember if I sent you all of them already,
> here is the list of patches that I have applied locally on top of your
> series to get a clean randconfig build:
> 
> 123c3262f872 KASAN: push back KASAN_STACK to clang-10

This one seems to have received some feedback, not sure if it was
addressed or not in a subsequent patch?

> d63dd9e2afd9 [HACK] ARM: disable KASAN+XIP_KERNEL

That one has been squashed, we could always lift the XIP_KERNEL
restriction later once someone with suitable hardware confirms it works.

> 879eb3c22240 kasan: increase 32-bit stack frame warning limit

That one should be pushed separately.

> 053555034bdf kasan: disable CONFIG_KASAN_STACK with clang on arm32

This one I did not take based on Linus' feedback that is breaks booting
on his RealView board.

> 6c1a78a448c2 ARM: fix kasan link failures

This one was squashed relevant and will be sent out as v7.
Arnd Bergmann Oct. 7, 2019, 10:10 p.m. UTC | #5
On Mon, Oct 7, 2019 at 11:35 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
> On 7/18/19 12:51 AM, Arnd Bergmann wrote:
> > On Thu, Jul 11, 2019 at 7:00 PM Florian Fainelli
> > <florian.fainelli@broadcom.com> wrote:
> >> On 7/2/19 2:06 PM, Linus Walleij wrote:
> >
> >>
> >> Great, thanks a lot for taking a look. FYI, I will be on holiday from
> >> July 19th till August 12th, if you think you have more feedback between
> >> now and then, I can try to pick it up and submit a v7 with that feedback
> >> addressed, or it will happen when I return, or you can pick it up if you
> >> refer, all options are possible!
> >>
> >> @Arnd, should we squash your patches in as well?
> >
> > Yes, please do. I don't remember if I sent you all of them already,
> > here is the list of patches that I have applied locally on top of your
> > series to get a clean randconfig build:
> >
> > 123c3262f872 KASAN: push back KASAN_STACK to clang-10
>
> This one seems to have received some feedback, not sure if it was
> addressed or not in a subsequent patch?

ebb6d35a74ce ("kasan: remove clang version check for KASAN_STACK")

got applied, it seems clang will remain broken with KASAN_STACK
for a while.

> > 053555034bdf kasan: disable CONFIG_KASAN_STACK with clang on arm32
>
> This one I did not take based on Linus' feedback that is breaks booting
> on his RealView board.

That likely means that there is still a bigger problem somewhere.

      Arnd
Linus Walleij Oct. 8, 2019, 8:47 a.m. UTC | #6
On Tue, Oct 8, 2019 at 12:10 AM Arnd Bergmann <arnd@arndb.de> wrote:
> On Mon, Oct 7, 2019 at 11:35 PM Florian Fainelli <f.fainelli@gmail.com> wrote:

> > > 053555034bdf kasan: disable CONFIG_KASAN_STACK with clang on arm32
> >
> > This one I did not take based on Linus' feedback that is breaks booting
> > on his RealView board.
>
> That likely means that there is still a bigger problem somewhere.

I will try to look into it. I got pretty puzzled by this, it makes no sense.

One possible problem is that some of the test chips on the RealViews
are not that stable, especially with caches. The plan is to test in QEMU
and hardware in parallel.

Yours,
Linus Walleij
Marco Felsch Nov. 14, 2019, 6:12 p.m. UTC | #7
Hi Florian,

first of all, many thanks for your work on this series =) I picked your
and Arnd patches to make it compilable. Now it's compiling but my imx6q
board didn't boot anymore. I debugged the code and found that the branch
to 'start_kernel' won't be reached

8<------- arch/arm/kernel/head-common.S -------
....

#ifdef CONFIG_KASAN
        bl      kasan_early_init
#endif
	mov     lr, #0
	b       start_kernel
ENDPROC(__mmap_switched)

....
8<----------------------------------------------

Now, I found also that 'KASAN_SHADOW_OFFSET' isn't set due to missing
'CONFIG_KASAN_SHADOW_OFFSET' and so no '-fasan-shadow-offset=xxxxx' is
added. Can that be the reason why my board isn't booted anymore?

Thanks for your reply.

Regards,
  Marco

On 19-06-17 15:11, Florian Fainelli wrote:
> Hi all,
> 
> Abbott submitted a v5 about a year ago here:
> 
> and the series was not picked up since then, so I rebased it against
> v5.2-rc4 and re-tested it on a Brahma-B53 (ARMv8 running AArch32 mode)
> and Brahma-B15, both LPAE and test-kasan is consistent with the ARM64
> counter part.
> 
> We were in a fairly good shape last time with a few different people
> having tested it, so I am hoping we can get that included for 5.4 if
> everything goes well.
> 
> Changelog:
> 
> v6 - v5
> - Resolve conflicts during rebase, and updated to make use of
>   kasan_early_shadow_pte instead of kasan_zero_pte
> 
> v5 - v4
> - Modify Andrey Ryabinin's email address.
> 
> v4 - v3
> - Remove the fix of type conversion in kasan_cache_create because it has
>   been fix in the latest version in:
>   git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> - Change some Reviewed-by tag into Reported-by tag to avoid misleading.
>   ---Reported by: Marc Zyngier <marc.zyngier@arm.com>
>                   Russell King - ARM Linux <linux@armlinux.org.uk>
> - Disable instrumentation for arch/arm/mm/physaddr.c
> 
> v3 - v2
> - Remove this patch: 2 1-byte checks more safer for memory_is_poisoned_16
>   because a unaligned load/store of 16 bytes is rare on arm, and this
>   patch is very likely to affect the performance of modern CPUs.
>   ---Acked by: Russell King - ARM Linux <linux@armlinux.org.uk>
> - Fixed some link error which kasan_pmd_populate,kasan_pte_populate and
>   kasan_pud_populate are in section .meminit.text but the function
>   kasan_alloc_block which is called by kasan_pmd_populate,
>   kasan_pte_populate and kasan_pud_populate is in section .init.text. So
>   we need change kasan_pmd_populate,kasan_pte_populate and
>   kasan_pud_populate into the section .init.text.
>   ---Reported by: Florian Fainelli <f.fainelli@gmail.com>
> - Fixed some compile error which caused by the wrong access instruction in
>   arch/arm/kernel/entry-common.S.
>   ---Reported by: kbuild test robot <lkp@intel.com>
> - Disable instrumentation for arch/arm/kvm/hyp/*.
>   ---Acked by: Marc Zyngier <marc.zyngier@arm.com>
> - Update the set of supported architectures in
>   Documentation/dev-tools/kasan.rst.
>   ---Acked by:Dmitry Vyukov <dvyukov@google.com>
> - The version 2 is tested by:
>   Florian Fainelli <f.fainelli@gmail.com> (compile test)
>   kbuild test robot <lkp@intel.com>       (compile test)
>   Joel Stanley <joel@jms.id.au>           (on ASPEED ast2500(ARMv5))
> 
> v2 - v1
> - Fixed some compiling error which happens on changing kernel compression
>   mode to lzma/xz/lzo/lz4.
>   ---Reported by: Florian Fainelli <f.fainelli@gmail.com>,
>              Russell King - ARM Linux <linux@armlinux.org.uk>
> - Fixed a compiling error cause by some older arm instruction set(armv4t)
>   don't suppory movw/movt which is reported by kbuild.
> - Changed the pte flag from _L_PTE_DEFAULT | L_PTE_DIRTY | L_PTE_XN to
>   pgprot_val(PAGE_KERNEL).
>   ---Reported by: Russell King - ARM Linux <linux@armlinux.org.uk>
> - Moved Enable KASan patch as the last one.
>   ---Reported by: Florian Fainelli <f.fainelli@gmail.com>,
>      Russell King - ARM Linux <linux@armlinux.org.uk>
> - Moved the definitions of cp15 registers from
>   arch/arm/include/asm/kvm_hyp.h to arch/arm/include/asm/cp15.h.
>   ---Asked by: Mark Rutland <mark.rutland@arm.com>
> - Merge the following commits into the commit
>   Define the virtual space of KASan's shadow region:
>   1) Define the virtual space of KASan's shadow region;
>   2) Avoid cleaning the KASan shadow area's mapping table;
>   3) Add KASan layout;
> - Merge the following commits into the commit
>   Initialize the mapping of KASan shadow memory:
>   1) Initialize the mapping of KASan shadow memory;
>   2) Add support arm LPAE;
>   3) Don't need to map the shadow of KASan's shadow memory;
>      ---Reported by: Russell King - ARM Linux <linux@armlinux.org.uk>
>   4) Change mapping of kasan_zero_page int readonly.
> - The version 1 is tested by Florian Fainelli <f.fainelli@gmail.com>
>   on a Cortex-A5 (no LPAE).
> 
> Hi,all:
>    These patches add arch specific code for kernel address sanitizer
> (see Documentation/kasan.txt).
> 
>    1/8 of kernel addresses reserved for shadow memory. There was no
> big enough hole for this, so virtual addresses for shadow were
> stolen from user space.
> 
>    At early boot stage the whole shadow region populated with just
> one physical page (kasan_zero_page). Later, this page reused
> as readonly zero shadow for some memory that KASan currently
> don't track (vmalloc).
> 
>   After mapping the physical memory, pages for shadow memory are
> allocated and mapped.
> 
>   KASan's stack instrumentation significantly increases stack's
> consumption, so CONFIG_KASAN doubles THREAD_SIZE.
> 
>   Functions like memset/memmove/memcpy do a lot of memory accesses.
> If bad pointer passed to one of these function it is important
> to catch this. Compiler's instrumentation cannot do this since
> these functions are written in assembly.
> 
>   KASan replaces memory functions with manually instrumented variants.
> Original functions declared as weak symbols so strong definitions
> in mm/kasan/kasan.c could replace them. Original functions have aliases
> with '__' prefix in name, so we could call non-instrumented variant
> if needed.
> 
>   Some files built without kasan instrumentation (e.g. mm/slub.c).
> Original mem* function replaced (via #define) with prefixed variants
> to disable memory access checks for such files.
> 
>   On arm LPAE architecture,  the mapping table of KASan shadow memory(if
> PAGE_OFFSET is 0xc0000000, the KASan shadow memory's virtual space is
> 0xb6e000000~0xbf000000) can't be filled in do_translation_fault function,
> because kasan instrumentation maybe cause do_translation_fault function
> accessing KASan shadow memory. The accessing of KASan shadow memory in
> do_translation_fault function maybe cause dead circle. So the mapping table
> of KASan shadow memory need be copyed in pgd_alloc function.
> 
> Most of the code comes from:
> https://github.com/aryabinin/linux/commit/0b54f17e70ff50a902c4af05bb92716eb95acefe
> 
> These patches are tested on vexpress-ca15, vexpress-ca9
> 
> 
> Abbott Liu (2):
>   ARM: Add TTBR operator for kasan_init
>   ARM: Define the virtual space of KASan's shadow region
> 
> Andrey Ryabinin (4):
>   ARM: Disable instrumentation for some code
>   ARM: Replace memory function for kasan
>   ARM: Initialize the mapping of KASan shadow memory
>   ARM: Enable KASan for arm
> 
>  Documentation/dev-tools/kasan.rst     |   4 +-
>  arch/arm/Kconfig                      |   1 +
>  arch/arm/boot/compressed/Makefile     |   1 +
>  arch/arm/boot/compressed/decompress.c |   2 +
>  arch/arm/boot/compressed/libfdt_env.h |   2 +
>  arch/arm/include/asm/cp15.h           | 106 +++++++++
>  arch/arm/include/asm/kasan.h          |  35 +++
>  arch/arm/include/asm/kasan_def.h      |  64 ++++++
>  arch/arm/include/asm/kvm_hyp.h        |  54 -----
>  arch/arm/include/asm/memory.h         |   5 +
>  arch/arm/include/asm/pgalloc.h        |   7 +-
>  arch/arm/include/asm/string.h         |  17 ++
>  arch/arm/include/asm/thread_info.h    |   4 +
>  arch/arm/kernel/entry-armv.S          |   5 +-
>  arch/arm/kernel/entry-common.S        |   9 +-
>  arch/arm/kernel/head-common.S         |   7 +-
>  arch/arm/kernel/setup.c               |   2 +
>  arch/arm/kernel/unwind.c              |   3 +-
>  arch/arm/kvm/hyp/cp15-sr.c            |  12 +-
>  arch/arm/kvm/hyp/switch.c             |   6 +-
>  arch/arm/lib/memcpy.S                 |   3 +
>  arch/arm/lib/memmove.S                |   5 +-
>  arch/arm/lib/memset.S                 |   3 +
>  arch/arm/mm/Makefile                  |   4 +
>  arch/arm/mm/kasan_init.c              | 301 ++++++++++++++++++++++++++
>  arch/arm/mm/mmu.c                     |   7 +-
>  arch/arm/mm/pgd.c                     |  14 ++
>  arch/arm/vdso/Makefile                |   2 +
>  28 files changed, 608 insertions(+), 77 deletions(-)
>  create mode 100644 arch/arm/include/asm/kasan.h
>  create mode 100644 arch/arm/include/asm/kasan_def.h
>  create mode 100644 arch/arm/mm/kasan_init.c
> 
> -- 
> 2.17.1
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
Florian Fainelli Nov. 14, 2019, 11:01 p.m. UTC | #8
Hello Marco,

On 11/14/19 10:12 AM, Marco Felsch wrote:
> Hi Florian,
> 
> first of all, many thanks for your work on this series =) I picked your
> and Arnd patches to make it compilable. Now it's compiling but my imx6q
> board didn't boot anymore. I debugged the code and found that the branch
> to 'start_kernel' won't be reached
> 
> 8<------- arch/arm/kernel/head-common.S -------
> ....
> 
> #ifdef CONFIG_KASAN
>         bl      kasan_early_init
> #endif
> 	mov     lr, #0
> 	b       start_kernel
> ENDPROC(__mmap_switched)
> 
> ....
> 8<----------------------------------------------
> 
> Now, I found also that 'KASAN_SHADOW_OFFSET' isn't set due to missing
> 'CONFIG_KASAN_SHADOW_OFFSET' and so no '-fasan-shadow-offset=xxxxx' is
> added. Can that be the reason why my board isn't booted anymore?

The latest that I have is here, though not yet submitted since I needed
to solve one issue on a specific platform with a lot of memory:

https://github.com/ffainelli/linux/pull/new/kasan-v7

Can you share your branch as well? I did not pick all of Arnd's patches
since some appeared to be seemingly independent from KASan on ARM. This
is the KASAN related options that are set in my configuration:

grep KASAN build/linux-custom/.config
CONFIG_HAVE_ARCH_KASAN=y
CONFIG_CC_HAS_KASAN_GENERIC=y
CONFIG_KASAN=y
CONFIG_KASAN_GENERIC=y
CONFIG_KASAN_OUTLINE=y
# CONFIG_KASAN_INLINE is not set
CONFIG_KASAN_STACK=1
CONFIG_TEST_KASAN=m

are you using something different by any chance?
Marco Felsch Nov. 15, 2019, 7:08 a.m. UTC | #9
Hi Florian,

On 19-11-14 15:01, Florian Fainelli wrote:
> Hello Marco,
> 
> On 11/14/19 10:12 AM, Marco Felsch wrote:
> > Hi Florian,
> > 
> > first of all, many thanks for your work on this series =) I picked your
> > and Arnd patches to make it compilable. Now it's compiling but my imx6q
> > board didn't boot anymore. I debugged the code and found that the branch
> > to 'start_kernel' won't be reached
> > 
> > 8<------- arch/arm/kernel/head-common.S -------
> > ....
> > 
> > #ifdef CONFIG_KASAN
> >         bl      kasan_early_init
> > #endif
> > 	mov     lr, #0
> > 	b       start_kernel
> > ENDPROC(__mmap_switched)
> > 
> > ....
> > 8<----------------------------------------------
> > 
> > Now, I found also that 'KASAN_SHADOW_OFFSET' isn't set due to missing
> > 'CONFIG_KASAN_SHADOW_OFFSET' and so no '-fasan-shadow-offset=xxxxx' is
> > added. Can that be the reason why my board isn't booted anymore?
> 
> The latest that I have is here, though not yet submitted since I needed
> to solve one issue on a specific platform with a lot of memory:
> 
> https://github.com/ffainelli/linux/pull/new/kasan-v7

Thanks for that hint, I will try this series too :) I read that you
wanna prepare a v7 but didn't found it ^^

> Can you share your branch as well? I did not pick all of Arnd's patches
> since some appeared to be seemingly independent from KASan on ARM. This
> is the KASAN related options that are set in my configuration:

Of course I will push it to github and inform you shortly.

> grep KASAN build/linux-custom/.config
> CONFIG_HAVE_ARCH_KASAN=y
> CONFIG_CC_HAS_KASAN_GENERIC=y
> CONFIG_KASAN=y
> CONFIG_KASAN_GENERIC=y
> CONFIG_KASAN_OUTLINE=y
> # CONFIG_KASAN_INLINE is not set
> CONFIG_KASAN_STACK=1
> CONFIG_TEST_KASAN=m

My config is:

CONFIG_HAVE_ARCH_KASAN=y
CONFIG_CC_HAS_KASAN_GENERIC=y
CONFIG_KASAN=y
CONFIG_KASAN_GENERIC=y
CONFIG_KASAN_OUTLINE=y
# CONFIG_KASAN_INLINE is not set
CONFIG_KASAN_STACK=1
# CONFIG_TEST_KASAN is not set

> are you using something different by any chance?

Unfortunately not.

Regards,
  Marco

> -- 
> Florian
>
Marco Felsch Nov. 15, 2019, 11:44 a.m. UTC | #10
Hi Florian,

On 19-11-15 08:08, Marco Felsch wrote:
> Hi Florian,
> 
> On 19-11-14 15:01, Florian Fainelli wrote:
> > Hello Marco,
> > 
> > On 11/14/19 10:12 AM, Marco Felsch wrote:
> > > Hi Florian,
> > > 
> > > first of all, many thanks for your work on this series =) I picked your
> > > and Arnd patches to make it compilable. Now it's compiling but my imx6q
> > > board didn't boot anymore. I debugged the code and found that the branch
> > > to 'start_kernel' won't be reached
> > > 
> > > 8<------- arch/arm/kernel/head-common.S -------
> > > ....
> > > 
> > > #ifdef CONFIG_KASAN
> > >         bl      kasan_early_init
> > > #endif
> > > 	mov     lr, #0
> > > 	b       start_kernel
> > > ENDPROC(__mmap_switched)
> > > 
> > > ....
> > > 8<----------------------------------------------
> > > 
> > > Now, I found also that 'KASAN_SHADOW_OFFSET' isn't set due to missing
> > > 'CONFIG_KASAN_SHADOW_OFFSET' and so no '-fasan-shadow-offset=xxxxx' is
> > > added. Can that be the reason why my board isn't booted anymore?
> > 
> > The latest that I have is here, though not yet submitted since I needed
> > to solve one issue on a specific platform with a lot of memory:
> > 
> > https://github.com/ffainelli/linux/pull/new/kasan-v7
> 
> Thanks for that hint, I will try this series too :) I read that you
> wanna prepare a v7 but didn't found it ^^
> 
> > Can you share your branch as well? I did not pick all of Arnd's patches
> > since some appeared to be seemingly independent from KASan on ARM. This
> > is the KASAN related options that are set in my configuration:
> 
> Of course I will push it to github and inform you shortly.

Here comes the link:
https://github.com/medude/linux/tree/v5.4/topic/kasan-arm.v7

I just applied Arnds Patche which you didn't added into your v7.

> > grep KASAN build/linux-custom/.config
> > CONFIG_HAVE_ARCH_KASAN=y
> > CONFIG_CC_HAS_KASAN_GENERIC=y
> > CONFIG_KASAN=y
> > CONFIG_KASAN_GENERIC=y
> > CONFIG_KASAN_OUTLINE=y
> > # CONFIG_KASAN_INLINE is not set
> > CONFIG_KASAN_STACK=1
> > CONFIG_TEST_KASAN=m
> 
> My config is:
> 
> CONFIG_HAVE_ARCH_KASAN=y
> CONFIG_CC_HAS_KASAN_GENERIC=y
> CONFIG_KASAN=y
> CONFIG_KASAN_GENERIC=y
> CONFIG_KASAN_OUTLINE=y
> # CONFIG_KASAN_INLINE is not set
> CONFIG_KASAN_STACK=1
> # CONFIG_TEST_KASAN is not set
> 
> > are you using something different by any chance?
> 
> Unfortunately not.

With your v7 it is working on my imx6 but unfortunately I can't run my
gstreamer testcase. My CPU load goes to 100% after starting gstreamer
and nothing happens.. But the test_kasan module works =) So I decided to
check a imx6quadplus but this target did not boot.. I used another
toolchain for the imx6quadplus gcc-9 instead of gcc-8. So it seems that
something went wrong during compilation. Because you didn't changed
something within the logic.

I wonder why we must not define the CONFIG_KASAN_SHADOW_OFFSET for arm.

Regards,
  Marco

> Regards,
>   Marco
> 
> > -- 
> > Florian
> > 
>
Florian Fainelli Nov. 19, 2019, 12:13 a.m. UTC | #11
On 11/15/19 3:44 AM, Marco Felsch wrote:
> 
> With your v7 it is working on my imx6 but unfortunately I can't run my
> gstreamer testcase. My CPU load goes to 100% after starting gstreamer
> and nothing happens.. But the test_kasan module works =) So I decided to
> check a imx6quadplus but this target did not boot.. I used another
> toolchain for the imx6quadplus gcc-9 instead of gcc-8. So it seems that
> something went wrong during compilation. Because you didn't changed
> something within the logic.
> 
> I wonder why we must not define the CONFIG_KASAN_SHADOW_OFFSET for arm.

That is was oversight. I have pushed updates to the branch here:

https://github.com/ffainelli/linux/pull/new/kasan-v7

which defines CONFIG_KASAN_SHADOW_OFFSET from the PAGE_OFFSET value
directly, and recalculate KASAN_SHADOW_START/END accordingly using the
same formula as ARM64.

can you try them out? If that still does not work, can you detail the
imx6qdp memory layout a little more? Any chance of running a kernel with
DEBUG_LL/EARLYPRINTK turned on so we can see where the problem could be?
Linus Walleij Jan. 17, 2020, 10:13 a.m. UTC | #12
On Tue, Nov 19, 2019 at 1:14 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
> On 11/15/19 3:44 AM, Marco Felsch wrote:
> >
> > With your v7 it is working on my imx6 but unfortunately I can't run my
> > gstreamer testcase. My CPU load goes to 100% after starting gstreamer
> > and nothing happens.. But the test_kasan module works =) So I decided to
> > check a imx6quadplus but this target did not boot.. I used another
> > toolchain for the imx6quadplus gcc-9 instead of gcc-8. So it seems that
> > something went wrong during compilation. Because you didn't changed
> > something within the logic.
> >
> > I wonder why we must not define the CONFIG_KASAN_SHADOW_OFFSET for arm.
>
> That is was oversight. I have pushed updates to the branch here:
>
> https://github.com/ffainelli/linux/pull/new/kasan-v7

I just git Kasan back on my radar because it needs to be fixed some day.

I took this branch for a ride on some QEMU and some real hardware.
Here I use the test module and just hacked it into the kernel instead of
as a module, it then crashes predictably but performs all the KASan
tests first and it works file, as in provokes the right warnings from
KASan.

Tested systems:

QEMU ARM RealView PBA8
QEMU ARM RealView PBX A9
QEMU ARM Versatile AB
Hardware Integrator CP
Hardware Versatile AB with IB2

Can we start to submit these patches to Russell's patch tracker?
Any more testing I should be doing?

Yours,
Linus Walleij
Florian Fainelli Jan. 17, 2020, 7:55 p.m. UTC | #13
On 1/17/20 2:13 AM, Linus Walleij wrote:
> On Tue, Nov 19, 2019 at 1:14 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
>> On 11/15/19 3:44 AM, Marco Felsch wrote:
>>>
>>> With your v7 it is working on my imx6 but unfortunately I can't run my
>>> gstreamer testcase. My CPU load goes to 100% after starting gstreamer
>>> and nothing happens.. But the test_kasan module works =) So I decided to
>>> check a imx6quadplus but this target did not boot.. I used another
>>> toolchain for the imx6quadplus gcc-9 instead of gcc-8. So it seems that
>>> something went wrong during compilation. Because you didn't changed
>>> something within the logic.
>>>
>>> I wonder why we must not define the CONFIG_KASAN_SHADOW_OFFSET for arm.
>>
>> That is was oversight. I have pushed updates to the branch here:
>>
>> https://github.com/ffainelli/linux/pull/new/kasan-v7
> 
> I just git Kasan back on my radar because it needs to be fixed some day.
> 
> I took this branch for a ride on some QEMU and some real hardware.
> Here I use the test module and just hacked it into the kernel instead of
> as a module, it then crashes predictably but performs all the KASan
> tests first and it works file, as in provokes the right warnings from
> KASan.
> 
> Tested systems:
> 
> QEMU ARM RealView PBA8
> QEMU ARM RealView PBX A9
> QEMU ARM Versatile AB
> Hardware Integrator CP
> Hardware Versatile AB with IB2
> 
> Can we start to submit these patches to Russell's patch tracker?
> Any more testing I should be doing?

Let me submit and rebase v7 get the auto builders some days to see if it
exposes a new build issue and then we toss it to RMK's patch tracker and
fix bugs from there?
Linus Walleij Jan. 17, 2020, 9:05 p.m. UTC | #14
On Fri, Jan 17, 2020 at 8:55 PM Florian Fainelli <f.fainelli@gmail.com> wrote:

> [Me]
> > Can we start to submit these patches to Russell's patch tracker?
> > Any more testing I should be doing?
>
> Let me submit and rebase v7 get the auto builders some days to see if it
> exposes a new build issue and then we toss it to RMK's patch tracker and
> fix bugs from there?

OK you can add my Tested-by: Linus Walleij <linus.walleij@linaro.org>
to the patches.

Thanks,
Linus Walleij
Linus Walleij March 5, 2020, 8:43 a.m. UTC | #15
Hi Florian,

On Fri, Jan 17, 2020 at 8:55 PM Florian Fainelli <f.fainelli@gmail.com> wrote:

> Let me submit and rebase v7 get the auto builders some days to see if it
> exposes a new build issue and then we toss it to RMK's patch tracker and
> fix bugs from there?

Sorry for hammering, can we get some initial patches going into
Russell's patch tracker here? I can sign them off and put them in
if you don't have time.

Thanks,
Linus Walleij
Marco Felsch March 5, 2020, 9:43 a.m. UTC | #16
On 20-03-05 09:43, Linus Walleij wrote:
> Hi Florian,
> 
> On Fri, Jan 17, 2020 at 8:55 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
> > Let me submit and rebase v7 get the auto builders some days to see if it
> > exposes a new build issue and then we toss it to RMK's patch tracker and
> > fix bugs from there?
> 
> Sorry for hammering, can we get some initial patches going into
> Russell's patch tracker here? I can sign them off and put them in
> if you don't have time.

I've tested the branch on several imx6 based boards with different
toolchains. Some boards booting normal and some of them are lost in
space... I didn't debugged it yet just wanted to inform you.

Regards,
  Marco

> Thanks,
> Linus Walleij
Linus Walleij March 20, 2020, 9:26 a.m. UTC | #17
On Thu, Mar 5, 2020 at 10:44 AM Marco Felsch <m.felsch@pengutronix.de> wrote:
> On 20-03-05 09:43, Linus Walleij wrote:
> > Hi Florian,
> >
> > On Fri, Jan 17, 2020 at 8:55 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
> >
> > > Let me submit and rebase v7 get the auto builders some days to see if it
> > > exposes a new build issue and then we toss it to RMK's patch tracker and
> > > fix bugs from there?
> >
> > Sorry for hammering, can we get some initial patches going into
> > Russell's patch tracker here? I can sign them off and put them in
> > if you don't have time.
>
> I've tested the branch on several imx6 based boards with different
> toolchains. Some boards booting normal and some of them are lost in
> space... I didn't debugged it yet just wanted to inform you.

Hm. I will bring up the KASan stack on more boards.

If the system is anywhere close to being low on memory they
will naturally crash, this is an unavoidable side effect of KASan
or anything else that just chew of a big chunk of memory, that I ran into,
as I was booting from initramfs on very memory
constrained systems.

Yours,
Linus Walleij