[v3,0/9] Add Kernel Concurrency Sanitizer (KCSAN)
mbox series

Message ID 20191104142745.14722-1-elver@google.com
Headers show
Series
  • Add Kernel Concurrency Sanitizer (KCSAN)
Related show

Message

Marco Elver Nov. 4, 2019, 2:27 p.m. UTC
This is the patch-series for the Kernel Concurrency Sanitizer (KCSAN).
KCSAN is a sampling watchpoint-based data-race detector. More details
are included in Documentation/dev-tools/kcsan.rst. This patch-series
only enables KCSAN for x86, but we expect adding support for other
architectures is relatively straightforward (we are aware of
experimental ARM64 and POWER support).

To gather early feedback, we announced KCSAN back in September, and
have integrated the feedback where possible:
http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com

We want to point out and acknowledge the work surrounding the LKMM,
including several articles that motivate why data-races are dangerous
[1, 2], justifying a data-race detector such as KCSAN.
[1] https://lwn.net/Articles/793253/
[2] https://lwn.net/Articles/799218/

The current list of known upstream fixes for data-races found by KCSAN
can be found here:
https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan

Changelog
---------
v3:
* Major changes:
 - Add microbenchmark.
 - Add instruction watchpoint skip randomization.
 - Refactor API and core runtime fast-path and slow-path. Compared to
   the previous version, with a default config and benchmarked using the
   added microbenchmark, this version is 3.8x faster.
 - Make __tsan_unaligned __alias of generic accesses.
 - Rename kcsan_{begin,end}_atomic ->
   kcsan_{nestable,flat}_atomic_{begin,end}
 - For filter list in debugfs.c use kmalloc+krealloc instead of
   kvmalloc.
 - Split Documentation into separate patch.

v2: http://lkml.kernel.org/r/20191017141305.146193-1-elver@google.com
* Major changes:
 - Replace kcsan_check_access(.., {true, false}) with
   kcsan_check_{read,write}.
 - Change atomic-instrumented.h to use __atomic_check_{read,write}.
 - Use common struct kcsan_ctx in task_struct and for per-CPU interrupt
   contexts.

v1: http://lkml.kernel.org/r/20191016083959.186860-1-elver@google.com

Marco Elver (9):
  kcsan: Add Kernel Concurrency Sanitizer infrastructure
  kcsan: Add Documentation entry in dev-tools
  objtool, kcsan: Add KCSAN runtime functions to whitelist
  build, kcsan: Add KCSAN build exceptions
  seqlock, kcsan: Add annotations for KCSAN
  seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
  asm-generic, kcsan: Add KCSAN instrumentation for bitops
  locking/atomics, kcsan: Add KCSAN instrumentation
  x86, kcsan: Enable KCSAN for x86

 Documentation/dev-tools/index.rst         |   1 +
 Documentation/dev-tools/kcsan.rst         | 217 +++++++++
 MAINTAINERS                               |  11 +
 Makefile                                  |   3 +-
 arch/x86/Kconfig                          |   1 +
 arch/x86/boot/Makefile                    |   2 +
 arch/x86/boot/compressed/Makefile         |   2 +
 arch/x86/entry/vdso/Makefile              |   3 +
 arch/x86/include/asm/bitops.h             |   6 +-
 arch/x86/kernel/Makefile                  |   7 +
 arch/x86/kernel/cpu/Makefile              |   3 +
 arch/x86/lib/Makefile                     |   4 +
 arch/x86/mm/Makefile                      |   3 +
 arch/x86/purgatory/Makefile               |   2 +
 arch/x86/realmode/Makefile                |   3 +
 arch/x86/realmode/rm/Makefile             |   3 +
 drivers/firmware/efi/libstub/Makefile     |   2 +
 include/asm-generic/atomic-instrumented.h | 393 +++++++--------
 include/asm-generic/bitops-instrumented.h |  18 +
 include/linux/compiler-clang.h            |   9 +
 include/linux/compiler-gcc.h              |   7 +
 include/linux/compiler.h                  |  35 +-
 include/linux/kcsan-checks.h              |  97 ++++
 include/linux/kcsan.h                     | 115 +++++
 include/linux/sched.h                     |   4 +
 include/linux/seqlock.h                   |  51 +-
 init/init_task.c                          |   8 +
 init/main.c                               |   2 +
 kernel/Makefile                           |   6 +
 kernel/kcsan/Makefile                     |  11 +
 kernel/kcsan/atomic.h                     |  27 ++
 kernel/kcsan/core.c                       | 560 ++++++++++++++++++++++
 kernel/kcsan/debugfs.c                    | 275 +++++++++++
 kernel/kcsan/encoding.h                   |  94 ++++
 kernel/kcsan/kcsan.h                      | 131 +++++
 kernel/kcsan/report.c                     | 306 ++++++++++++
 kernel/kcsan/test.c                       | 121 +++++
 kernel/sched/Makefile                     |   6 +
 lib/Kconfig.debug                         |   2 +
 lib/Kconfig.kcsan                         | 119 +++++
 lib/Makefile                              |   3 +
 mm/Makefile                               |   8 +
 scripts/Makefile.kcsan                    |   6 +
 scripts/Makefile.lib                      |  10 +
 scripts/atomic/gen-atomic-instrumented.sh |  17 +-
 tools/objtool/check.c                     |  18 +
 46 files changed, 2526 insertions(+), 206 deletions(-)
 create mode 100644 Documentation/dev-tools/kcsan.rst
 create mode 100644 include/linux/kcsan-checks.h
 create mode 100644 include/linux/kcsan.h
 create mode 100644 kernel/kcsan/Makefile
 create mode 100644 kernel/kcsan/atomic.h
 create mode 100644 kernel/kcsan/core.c
 create mode 100644 kernel/kcsan/debugfs.c
 create mode 100644 kernel/kcsan/encoding.h
 create mode 100644 kernel/kcsan/kcsan.h
 create mode 100644 kernel/kcsan/report.c
 create mode 100644 kernel/kcsan/test.c
 create mode 100644 lib/Kconfig.kcsan
 create mode 100644 scripts/Makefile.kcsan

Comments

Paul E. McKenney Nov. 4, 2019, 4:47 p.m. UTC | #1
On Mon, Nov 04, 2019 at 03:27:36PM +0100, Marco Elver wrote:
> This is the patch-series for the Kernel Concurrency Sanitizer (KCSAN).
> KCSAN is a sampling watchpoint-based data-race detector. More details
> are included in Documentation/dev-tools/kcsan.rst. This patch-series
> only enables KCSAN for x86, but we expect adding support for other
> architectures is relatively straightforward (we are aware of
> experimental ARM64 and POWER support).
> 
> To gather early feedback, we announced KCSAN back in September, and
> have integrated the feedback where possible:
> http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com
> 
> We want to point out and acknowledge the work surrounding the LKMM,
> including several articles that motivate why data-races are dangerous
> [1, 2], justifying a data-race detector such as KCSAN.
> [1] https://lwn.net/Articles/793253/
> [2] https://lwn.net/Articles/799218/
> 
> The current list of known upstream fixes for data-races found by KCSAN
> can be found here:
> https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan

Making this more accessible to more people seems like a good thing.
So, for the series:

Acked-by: Paul E. McKenney <paulmck@kernel.org>

> Changelog
> ---------
> v3:
> * Major changes:
>  - Add microbenchmark.
>  - Add instruction watchpoint skip randomization.
>  - Refactor API and core runtime fast-path and slow-path. Compared to
>    the previous version, with a default config and benchmarked using the
>    added microbenchmark, this version is 3.8x faster.
>  - Make __tsan_unaligned __alias of generic accesses.
>  - Rename kcsan_{begin,end}_atomic ->
>    kcsan_{nestable,flat}_atomic_{begin,end}
>  - For filter list in debugfs.c use kmalloc+krealloc instead of
>    kvmalloc.
>  - Split Documentation into separate patch.
> 
> v2: http://lkml.kernel.org/r/20191017141305.146193-1-elver@google.com
> * Major changes:
>  - Replace kcsan_check_access(.., {true, false}) with
>    kcsan_check_{read,write}.
>  - Change atomic-instrumented.h to use __atomic_check_{read,write}.
>  - Use common struct kcsan_ctx in task_struct and for per-CPU interrupt
>    contexts.
> 
> v1: http://lkml.kernel.org/r/20191016083959.186860-1-elver@google.com
> 
> Marco Elver (9):
>   kcsan: Add Kernel Concurrency Sanitizer infrastructure
>   kcsan: Add Documentation entry in dev-tools
>   objtool, kcsan: Add KCSAN runtime functions to whitelist
>   build, kcsan: Add KCSAN build exceptions
>   seqlock, kcsan: Add annotations for KCSAN
>   seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
>   asm-generic, kcsan: Add KCSAN instrumentation for bitops
>   locking/atomics, kcsan: Add KCSAN instrumentation
>   x86, kcsan: Enable KCSAN for x86
> 
>  Documentation/dev-tools/index.rst         |   1 +
>  Documentation/dev-tools/kcsan.rst         | 217 +++++++++
>  MAINTAINERS                               |  11 +
>  Makefile                                  |   3 +-
>  arch/x86/Kconfig                          |   1 +
>  arch/x86/boot/Makefile                    |   2 +
>  arch/x86/boot/compressed/Makefile         |   2 +
>  arch/x86/entry/vdso/Makefile              |   3 +
>  arch/x86/include/asm/bitops.h             |   6 +-
>  arch/x86/kernel/Makefile                  |   7 +
>  arch/x86/kernel/cpu/Makefile              |   3 +
>  arch/x86/lib/Makefile                     |   4 +
>  arch/x86/mm/Makefile                      |   3 +
>  arch/x86/purgatory/Makefile               |   2 +
>  arch/x86/realmode/Makefile                |   3 +
>  arch/x86/realmode/rm/Makefile             |   3 +
>  drivers/firmware/efi/libstub/Makefile     |   2 +
>  include/asm-generic/atomic-instrumented.h | 393 +++++++--------
>  include/asm-generic/bitops-instrumented.h |  18 +
>  include/linux/compiler-clang.h            |   9 +
>  include/linux/compiler-gcc.h              |   7 +
>  include/linux/compiler.h                  |  35 +-
>  include/linux/kcsan-checks.h              |  97 ++++
>  include/linux/kcsan.h                     | 115 +++++
>  include/linux/sched.h                     |   4 +
>  include/linux/seqlock.h                   |  51 +-
>  init/init_task.c                          |   8 +
>  init/main.c                               |   2 +
>  kernel/Makefile                           |   6 +
>  kernel/kcsan/Makefile                     |  11 +
>  kernel/kcsan/atomic.h                     |  27 ++
>  kernel/kcsan/core.c                       | 560 ++++++++++++++++++++++
>  kernel/kcsan/debugfs.c                    | 275 +++++++++++
>  kernel/kcsan/encoding.h                   |  94 ++++
>  kernel/kcsan/kcsan.h                      | 131 +++++
>  kernel/kcsan/report.c                     | 306 ++++++++++++
>  kernel/kcsan/test.c                       | 121 +++++
>  kernel/sched/Makefile                     |   6 +
>  lib/Kconfig.debug                         |   2 +
>  lib/Kconfig.kcsan                         | 119 +++++
>  lib/Makefile                              |   3 +
>  mm/Makefile                               |   8 +
>  scripts/Makefile.kcsan                    |   6 +
>  scripts/Makefile.lib                      |  10 +
>  scripts/atomic/gen-atomic-instrumented.sh |  17 +-
>  tools/objtool/check.c                     |  18 +
>  46 files changed, 2526 insertions(+), 206 deletions(-)
>  create mode 100644 Documentation/dev-tools/kcsan.rst
>  create mode 100644 include/linux/kcsan-checks.h
>  create mode 100644 include/linux/kcsan.h
>  create mode 100644 kernel/kcsan/Makefile
>  create mode 100644 kernel/kcsan/atomic.h
>  create mode 100644 kernel/kcsan/core.c
>  create mode 100644 kernel/kcsan/debugfs.c
>  create mode 100644 kernel/kcsan/encoding.h
>  create mode 100644 kernel/kcsan/kcsan.h
>  create mode 100644 kernel/kcsan/report.c
>  create mode 100644 kernel/kcsan/test.c
>  create mode 100644 lib/Kconfig.kcsan
>  create mode 100644 scripts/Makefile.kcsan
> 
> -- 
> 2.24.0.rc1.363.gb1bccd3e3d-goog
>
Marco Elver Nov. 4, 2019, 6:41 p.m. UTC | #2
On Mon, 4 Nov 2019 at 17:47, Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Mon, Nov 04, 2019 at 03:27:36PM +0100, Marco Elver wrote:
> > This is the patch-series for the Kernel Concurrency Sanitizer (KCSAN).
> > KCSAN is a sampling watchpoint-based data-race detector. More details
> > are included in Documentation/dev-tools/kcsan.rst. This patch-series
> > only enables KCSAN for x86, but we expect adding support for other
> > architectures is relatively straightforward (we are aware of
> > experimental ARM64 and POWER support).
> >
> > To gather early feedback, we announced KCSAN back in September, and
> > have integrated the feedback where possible:
> > http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com
> >
> > We want to point out and acknowledge the work surrounding the LKMM,
> > including several articles that motivate why data-races are dangerous
> > [1, 2], justifying a data-race detector such as KCSAN.
> > [1] https://lwn.net/Articles/793253/
> > [2] https://lwn.net/Articles/799218/
> >
> > The current list of known upstream fixes for data-races found by KCSAN
> > can be found here:
> > https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan
>
> Making this more accessible to more people seems like a good thing.
> So, for the series:
>
> Acked-by: Paul E. McKenney <paulmck@kernel.org>

Much appreciated. Thanks, Paul!

Any suggestions which tree this could eventually land in?

Thanks,
-- Marco

> > Changelog
> > ---------
> > v3:
> > * Major changes:
> >  - Add microbenchmark.
> >  - Add instruction watchpoint skip randomization.
> >  - Refactor API and core runtime fast-path and slow-path. Compared to
> >    the previous version, with a default config and benchmarked using the
> >    added microbenchmark, this version is 3.8x faster.
> >  - Make __tsan_unaligned __alias of generic accesses.
> >  - Rename kcsan_{begin,end}_atomic ->
> >    kcsan_{nestable,flat}_atomic_{begin,end}
> >  - For filter list in debugfs.c use kmalloc+krealloc instead of
> >    kvmalloc.
> >  - Split Documentation into separate patch.
> >
> > v2: http://lkml.kernel.org/r/20191017141305.146193-1-elver@google.com
> > * Major changes:
> >  - Replace kcsan_check_access(.., {true, false}) with
> >    kcsan_check_{read,write}.
> >  - Change atomic-instrumented.h to use __atomic_check_{read,write}.
> >  - Use common struct kcsan_ctx in task_struct and for per-CPU interrupt
> >    contexts.
> >
> > v1: http://lkml.kernel.org/r/20191016083959.186860-1-elver@google.com
> >
> > Marco Elver (9):
> >   kcsan: Add Kernel Concurrency Sanitizer infrastructure
> >   kcsan: Add Documentation entry in dev-tools
> >   objtool, kcsan: Add KCSAN runtime functions to whitelist
> >   build, kcsan: Add KCSAN build exceptions
> >   seqlock, kcsan: Add annotations for KCSAN
> >   seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
> >   asm-generic, kcsan: Add KCSAN instrumentation for bitops
> >   locking/atomics, kcsan: Add KCSAN instrumentation
> >   x86, kcsan: Enable KCSAN for x86
> >
> >  Documentation/dev-tools/index.rst         |   1 +
> >  Documentation/dev-tools/kcsan.rst         | 217 +++++++++
> >  MAINTAINERS                               |  11 +
> >  Makefile                                  |   3 +-
> >  arch/x86/Kconfig                          |   1 +
> >  arch/x86/boot/Makefile                    |   2 +
> >  arch/x86/boot/compressed/Makefile         |   2 +
> >  arch/x86/entry/vdso/Makefile              |   3 +
> >  arch/x86/include/asm/bitops.h             |   6 +-
> >  arch/x86/kernel/Makefile                  |   7 +
> >  arch/x86/kernel/cpu/Makefile              |   3 +
> >  arch/x86/lib/Makefile                     |   4 +
> >  arch/x86/mm/Makefile                      |   3 +
> >  arch/x86/purgatory/Makefile               |   2 +
> >  arch/x86/realmode/Makefile                |   3 +
> >  arch/x86/realmode/rm/Makefile             |   3 +
> >  drivers/firmware/efi/libstub/Makefile     |   2 +
> >  include/asm-generic/atomic-instrumented.h | 393 +++++++--------
> >  include/asm-generic/bitops-instrumented.h |  18 +
> >  include/linux/compiler-clang.h            |   9 +
> >  include/linux/compiler-gcc.h              |   7 +
> >  include/linux/compiler.h                  |  35 +-
> >  include/linux/kcsan-checks.h              |  97 ++++
> >  include/linux/kcsan.h                     | 115 +++++
> >  include/linux/sched.h                     |   4 +
> >  include/linux/seqlock.h                   |  51 +-
> >  init/init_task.c                          |   8 +
> >  init/main.c                               |   2 +
> >  kernel/Makefile                           |   6 +
> >  kernel/kcsan/Makefile                     |  11 +
> >  kernel/kcsan/atomic.h                     |  27 ++
> >  kernel/kcsan/core.c                       | 560 ++++++++++++++++++++++
> >  kernel/kcsan/debugfs.c                    | 275 +++++++++++
> >  kernel/kcsan/encoding.h                   |  94 ++++
> >  kernel/kcsan/kcsan.h                      | 131 +++++
> >  kernel/kcsan/report.c                     | 306 ++++++++++++
> >  kernel/kcsan/test.c                       | 121 +++++
> >  kernel/sched/Makefile                     |   6 +
> >  lib/Kconfig.debug                         |   2 +
> >  lib/Kconfig.kcsan                         | 119 +++++
> >  lib/Makefile                              |   3 +
> >  mm/Makefile                               |   8 +
> >  scripts/Makefile.kcsan                    |   6 +
> >  scripts/Makefile.lib                      |  10 +
> >  scripts/atomic/gen-atomic-instrumented.sh |  17 +-
> >  tools/objtool/check.c                     |  18 +
> >  46 files changed, 2526 insertions(+), 206 deletions(-)
> >  create mode 100644 Documentation/dev-tools/kcsan.rst
> >  create mode 100644 include/linux/kcsan-checks.h
> >  create mode 100644 include/linux/kcsan.h
> >  create mode 100644 kernel/kcsan/Makefile
> >  create mode 100644 kernel/kcsan/atomic.h
> >  create mode 100644 kernel/kcsan/core.c
> >  create mode 100644 kernel/kcsan/debugfs.c
> >  create mode 100644 kernel/kcsan/encoding.h
> >  create mode 100644 kernel/kcsan/kcsan.h
> >  create mode 100644 kernel/kcsan/report.c
> >  create mode 100644 kernel/kcsan/test.c
> >  create mode 100644 lib/Kconfig.kcsan
> >  create mode 100644 scripts/Makefile.kcsan
> >
> > --
> > 2.24.0.rc1.363.gb1bccd3e3d-goog
> >
>
> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191104164717.GE20975%40paulmck-ThinkPad-P72.
Paul E. McKenney Nov. 4, 2019, 7:46 p.m. UTC | #3
On Mon, Nov 04, 2019 at 07:41:30PM +0100, Marco Elver wrote:
> On Mon, 4 Nov 2019 at 17:47, Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Mon, Nov 04, 2019 at 03:27:36PM +0100, Marco Elver wrote:
> > > This is the patch-series for the Kernel Concurrency Sanitizer (KCSAN).
> > > KCSAN is a sampling watchpoint-based data-race detector. More details
> > > are included in Documentation/dev-tools/kcsan.rst. This patch-series
> > > only enables KCSAN for x86, but we expect adding support for other
> > > architectures is relatively straightforward (we are aware of
> > > experimental ARM64 and POWER support).
> > >
> > > To gather early feedback, we announced KCSAN back in September, and
> > > have integrated the feedback where possible:
> > > http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com
> > >
> > > We want to point out and acknowledge the work surrounding the LKMM,
> > > including several articles that motivate why data-races are dangerous
> > > [1, 2], justifying a data-race detector such as KCSAN.
> > > [1] https://lwn.net/Articles/793253/
> > > [2] https://lwn.net/Articles/799218/
> > >
> > > The current list of known upstream fixes for data-races found by KCSAN
> > > can be found here:
> > > https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan
> >
> > Making this more accessible to more people seems like a good thing.
> > So, for the series:
> >
> > Acked-by: Paul E. McKenney <paulmck@kernel.org>
> 
> Much appreciated. Thanks, Paul!
> 
> Any suggestions which tree this could eventually land in?

I would guess that Dmitry might have some suggestions.

							Thanx, Paul

> Thanks,
> -- Marco
> 
> > > Changelog
> > > ---------
> > > v3:
> > > * Major changes:
> > >  - Add microbenchmark.
> > >  - Add instruction watchpoint skip randomization.
> > >  - Refactor API and core runtime fast-path and slow-path. Compared to
> > >    the previous version, with a default config and benchmarked using the
> > >    added microbenchmark, this version is 3.8x faster.
> > >  - Make __tsan_unaligned __alias of generic accesses.
> > >  - Rename kcsan_{begin,end}_atomic ->
> > >    kcsan_{nestable,flat}_atomic_{begin,end}
> > >  - For filter list in debugfs.c use kmalloc+krealloc instead of
> > >    kvmalloc.
> > >  - Split Documentation into separate patch.
> > >
> > > v2: http://lkml.kernel.org/r/20191017141305.146193-1-elver@google.com
> > > * Major changes:
> > >  - Replace kcsan_check_access(.., {true, false}) with
> > >    kcsan_check_{read,write}.
> > >  - Change atomic-instrumented.h to use __atomic_check_{read,write}.
> > >  - Use common struct kcsan_ctx in task_struct and for per-CPU interrupt
> > >    contexts.
> > >
> > > v1: http://lkml.kernel.org/r/20191016083959.186860-1-elver@google.com
> > >
> > > Marco Elver (9):
> > >   kcsan: Add Kernel Concurrency Sanitizer infrastructure
> > >   kcsan: Add Documentation entry in dev-tools
> > >   objtool, kcsan: Add KCSAN runtime functions to whitelist
> > >   build, kcsan: Add KCSAN build exceptions
> > >   seqlock, kcsan: Add annotations for KCSAN
> > >   seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
> > >   asm-generic, kcsan: Add KCSAN instrumentation for bitops
> > >   locking/atomics, kcsan: Add KCSAN instrumentation
> > >   x86, kcsan: Enable KCSAN for x86
> > >
> > >  Documentation/dev-tools/index.rst         |   1 +
> > >  Documentation/dev-tools/kcsan.rst         | 217 +++++++++
> > >  MAINTAINERS                               |  11 +
> > >  Makefile                                  |   3 +-
> > >  arch/x86/Kconfig                          |   1 +
> > >  arch/x86/boot/Makefile                    |   2 +
> > >  arch/x86/boot/compressed/Makefile         |   2 +
> > >  arch/x86/entry/vdso/Makefile              |   3 +
> > >  arch/x86/include/asm/bitops.h             |   6 +-
> > >  arch/x86/kernel/Makefile                  |   7 +
> > >  arch/x86/kernel/cpu/Makefile              |   3 +
> > >  arch/x86/lib/Makefile                     |   4 +
> > >  arch/x86/mm/Makefile                      |   3 +
> > >  arch/x86/purgatory/Makefile               |   2 +
> > >  arch/x86/realmode/Makefile                |   3 +
> > >  arch/x86/realmode/rm/Makefile             |   3 +
> > >  drivers/firmware/efi/libstub/Makefile     |   2 +
> > >  include/asm-generic/atomic-instrumented.h | 393 +++++++--------
> > >  include/asm-generic/bitops-instrumented.h |  18 +
> > >  include/linux/compiler-clang.h            |   9 +
> > >  include/linux/compiler-gcc.h              |   7 +
> > >  include/linux/compiler.h                  |  35 +-
> > >  include/linux/kcsan-checks.h              |  97 ++++
> > >  include/linux/kcsan.h                     | 115 +++++
> > >  include/linux/sched.h                     |   4 +
> > >  include/linux/seqlock.h                   |  51 +-
> > >  init/init_task.c                          |   8 +
> > >  init/main.c                               |   2 +
> > >  kernel/Makefile                           |   6 +
> > >  kernel/kcsan/Makefile                     |  11 +
> > >  kernel/kcsan/atomic.h                     |  27 ++
> > >  kernel/kcsan/core.c                       | 560 ++++++++++++++++++++++
> > >  kernel/kcsan/debugfs.c                    | 275 +++++++++++
> > >  kernel/kcsan/encoding.h                   |  94 ++++
> > >  kernel/kcsan/kcsan.h                      | 131 +++++
> > >  kernel/kcsan/report.c                     | 306 ++++++++++++
> > >  kernel/kcsan/test.c                       | 121 +++++
> > >  kernel/sched/Makefile                     |   6 +
> > >  lib/Kconfig.debug                         |   2 +
> > >  lib/Kconfig.kcsan                         | 119 +++++
> > >  lib/Makefile                              |   3 +
> > >  mm/Makefile                               |   8 +
> > >  scripts/Makefile.kcsan                    |   6 +
> > >  scripts/Makefile.lib                      |  10 +
> > >  scripts/atomic/gen-atomic-instrumented.sh |  17 +-
> > >  tools/objtool/check.c                     |  18 +
> > >  46 files changed, 2526 insertions(+), 206 deletions(-)
> > >  create mode 100644 Documentation/dev-tools/kcsan.rst
> > >  create mode 100644 include/linux/kcsan-checks.h
> > >  create mode 100644 include/linux/kcsan.h
> > >  create mode 100644 kernel/kcsan/Makefile
> > >  create mode 100644 kernel/kcsan/atomic.h
> > >  create mode 100644 kernel/kcsan/core.c
> > >  create mode 100644 kernel/kcsan/debugfs.c
> > >  create mode 100644 kernel/kcsan/encoding.h
> > >  create mode 100644 kernel/kcsan/kcsan.h
> > >  create mode 100644 kernel/kcsan/report.c
> > >  create mode 100644 kernel/kcsan/test.c
> > >  create mode 100644 lib/Kconfig.kcsan
> > >  create mode 100644 scripts/Makefile.kcsan
> > >
> > > --
> > > 2.24.0.rc1.363.gb1bccd3e3d-goog
> > >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191104164717.GE20975%40paulmck-ThinkPad-P72.
Marco Elver Nov. 5, 2019, 11:10 a.m. UTC | #4
On Mon, 4 Nov 2019 at 20:47, Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Mon, Nov 04, 2019 at 07:41:30PM +0100, Marco Elver wrote:
> > On Mon, 4 Nov 2019 at 17:47, Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Mon, Nov 04, 2019 at 03:27:36PM +0100, Marco Elver wrote:
> > > > This is the patch-series for the Kernel Concurrency Sanitizer (KCSAN).
> > > > KCSAN is a sampling watchpoint-based data-race detector. More details
> > > > are included in Documentation/dev-tools/kcsan.rst. This patch-series
> > > > only enables KCSAN for x86, but we expect adding support for other
> > > > architectures is relatively straightforward (we are aware of
> > > > experimental ARM64 and POWER support).
> > > >
> > > > To gather early feedback, we announced KCSAN back in September, and
> > > > have integrated the feedback where possible:
> > > > http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com
> > > >
> > > > We want to point out and acknowledge the work surrounding the LKMM,
> > > > including several articles that motivate why data-races are dangerous
> > > > [1, 2], justifying a data-race detector such as KCSAN.
> > > > [1] https://lwn.net/Articles/793253/
> > > > [2] https://lwn.net/Articles/799218/
> > > >
> > > > The current list of known upstream fixes for data-races found by KCSAN
> > > > can be found here:
> > > > https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan
> > >
> > > Making this more accessible to more people seems like a good thing.
> > > So, for the series:
> > >
> > > Acked-by: Paul E. McKenney <paulmck@kernel.org>
> >
> > Much appreciated. Thanks, Paul!
> >
> > Any suggestions which tree this could eventually land in?
>
> I would guess that Dmitry might have some suggestions.

I checked and we're both unclear what the most obvious tree to land in
is (the other sanitizers are mm related, which KCSAN is not).

One suggestion that comes to my mind is for KCSAN to go through the
same tree (rcu?) as the LKMM due to their inherent relationship. Would
that make most sense?

Thanks,
-- Marco

> >
> > > > Changelog
> > > > ---------
> > > > v3:
> > > > * Major changes:
> > > >  - Add microbenchmark.
> > > >  - Add instruction watchpoint skip randomization.
> > > >  - Refactor API and core runtime fast-path and slow-path. Compared to
> > > >    the previous version, with a default config and benchmarked using the
> > > >    added microbenchmark, this version is 3.8x faster.
> > > >  - Make __tsan_unaligned __alias of generic accesses.
> > > >  - Rename kcsan_{begin,end}_atomic ->
> > > >    kcsan_{nestable,flat}_atomic_{begin,end}
> > > >  - For filter list in debugfs.c use kmalloc+krealloc instead of
> > > >    kvmalloc.
> > > >  - Split Documentation into separate patch.
> > > >
> > > > v2: http://lkml.kernel.org/r/20191017141305.146193-1-elver@google.com
> > > > * Major changes:
> > > >  - Replace kcsan_check_access(.., {true, false}) with
> > > >    kcsan_check_{read,write}.
> > > >  - Change atomic-instrumented.h to use __atomic_check_{read,write}.
> > > >  - Use common struct kcsan_ctx in task_struct and for per-CPU interrupt
> > > >    contexts.
> > > >
> > > > v1: http://lkml.kernel.org/r/20191016083959.186860-1-elver@google.com
> > > >
> > > > Marco Elver (9):
> > > >   kcsan: Add Kernel Concurrency Sanitizer infrastructure
> > > >   kcsan: Add Documentation entry in dev-tools
> > > >   objtool, kcsan: Add KCSAN runtime functions to whitelist
> > > >   build, kcsan: Add KCSAN build exceptions
> > > >   seqlock, kcsan: Add annotations for KCSAN
> > > >   seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
> > > >   asm-generic, kcsan: Add KCSAN instrumentation for bitops
> > > >   locking/atomics, kcsan: Add KCSAN instrumentation
> > > >   x86, kcsan: Enable KCSAN for x86
> > > >
> > > >  Documentation/dev-tools/index.rst         |   1 +
> > > >  Documentation/dev-tools/kcsan.rst         | 217 +++++++++
> > > >  MAINTAINERS                               |  11 +
> > > >  Makefile                                  |   3 +-
> > > >  arch/x86/Kconfig                          |   1 +
> > > >  arch/x86/boot/Makefile                    |   2 +
> > > >  arch/x86/boot/compressed/Makefile         |   2 +
> > > >  arch/x86/entry/vdso/Makefile              |   3 +
> > > >  arch/x86/include/asm/bitops.h             |   6 +-
> > > >  arch/x86/kernel/Makefile                  |   7 +
> > > >  arch/x86/kernel/cpu/Makefile              |   3 +
> > > >  arch/x86/lib/Makefile                     |   4 +
> > > >  arch/x86/mm/Makefile                      |   3 +
> > > >  arch/x86/purgatory/Makefile               |   2 +
> > > >  arch/x86/realmode/Makefile                |   3 +
> > > >  arch/x86/realmode/rm/Makefile             |   3 +
> > > >  drivers/firmware/efi/libstub/Makefile     |   2 +
> > > >  include/asm-generic/atomic-instrumented.h | 393 +++++++--------
> > > >  include/asm-generic/bitops-instrumented.h |  18 +
> > > >  include/linux/compiler-clang.h            |   9 +
> > > >  include/linux/compiler-gcc.h              |   7 +
> > > >  include/linux/compiler.h                  |  35 +-
> > > >  include/linux/kcsan-checks.h              |  97 ++++
> > > >  include/linux/kcsan.h                     | 115 +++++
> > > >  include/linux/sched.h                     |   4 +
> > > >  include/linux/seqlock.h                   |  51 +-
> > > >  init/init_task.c                          |   8 +
> > > >  init/main.c                               |   2 +
> > > >  kernel/Makefile                           |   6 +
> > > >  kernel/kcsan/Makefile                     |  11 +
> > > >  kernel/kcsan/atomic.h                     |  27 ++
> > > >  kernel/kcsan/core.c                       | 560 ++++++++++++++++++++++
> > > >  kernel/kcsan/debugfs.c                    | 275 +++++++++++
> > > >  kernel/kcsan/encoding.h                   |  94 ++++
> > > >  kernel/kcsan/kcsan.h                      | 131 +++++
> > > >  kernel/kcsan/report.c                     | 306 ++++++++++++
> > > >  kernel/kcsan/test.c                       | 121 +++++
> > > >  kernel/sched/Makefile                     |   6 +
> > > >  lib/Kconfig.debug                         |   2 +
> > > >  lib/Kconfig.kcsan                         | 119 +++++
> > > >  lib/Makefile                              |   3 +
> > > >  mm/Makefile                               |   8 +
> > > >  scripts/Makefile.kcsan                    |   6 +
> > > >  scripts/Makefile.lib                      |  10 +
> > > >  scripts/atomic/gen-atomic-instrumented.sh |  17 +-
> > > >  tools/objtool/check.c                     |  18 +
> > > >  46 files changed, 2526 insertions(+), 206 deletions(-)
> > > >  create mode 100644 Documentation/dev-tools/kcsan.rst
> > > >  create mode 100644 include/linux/kcsan-checks.h
> > > >  create mode 100644 include/linux/kcsan.h
> > > >  create mode 100644 kernel/kcsan/Makefile
> > > >  create mode 100644 kernel/kcsan/atomic.h
> > > >  create mode 100644 kernel/kcsan/core.c
> > > >  create mode 100644 kernel/kcsan/debugfs.c
> > > >  create mode 100644 kernel/kcsan/encoding.h
> > > >  create mode 100644 kernel/kcsan/kcsan.h
> > > >  create mode 100644 kernel/kcsan/report.c
> > > >  create mode 100644 kernel/kcsan/test.c
> > > >  create mode 100644 lib/Kconfig.kcsan
> > > >  create mode 100644 scripts/Makefile.kcsan
> > > >
> > > > --
> > > > 2.24.0.rc1.363.gb1bccd3e3d-goog
> > > >
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> > > To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191104164717.GE20975%40paulmck-ThinkPad-P72.
>
> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191104194658.GK20975%40paulmck-ThinkPad-P72.
Paul E. McKenney Nov. 5, 2019, 2:20 p.m. UTC | #5
On Tue, Nov 05, 2019 at 12:10:56PM +0100, Marco Elver wrote:
> On Mon, 4 Nov 2019 at 20:47, Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Mon, Nov 04, 2019 at 07:41:30PM +0100, Marco Elver wrote:
> > > On Mon, 4 Nov 2019 at 17:47, Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Mon, Nov 04, 2019 at 03:27:36PM +0100, Marco Elver wrote:
> > > > > This is the patch-series for the Kernel Concurrency Sanitizer (KCSAN).
> > > > > KCSAN is a sampling watchpoint-based data-race detector. More details
> > > > > are included in Documentation/dev-tools/kcsan.rst. This patch-series
> > > > > only enables KCSAN for x86, but we expect adding support for other
> > > > > architectures is relatively straightforward (we are aware of
> > > > > experimental ARM64 and POWER support).
> > > > >
> > > > > To gather early feedback, we announced KCSAN back in September, and
> > > > > have integrated the feedback where possible:
> > > > > http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com
> > > > >
> > > > > We want to point out and acknowledge the work surrounding the LKMM,
> > > > > including several articles that motivate why data-races are dangerous
> > > > > [1, 2], justifying a data-race detector such as KCSAN.
> > > > > [1] https://lwn.net/Articles/793253/
> > > > > [2] https://lwn.net/Articles/799218/
> > > > >
> > > > > The current list of known upstream fixes for data-races found by KCSAN
> > > > > can be found here:
> > > > > https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan
> > > >
> > > > Making this more accessible to more people seems like a good thing.
> > > > So, for the series:
> > > >
> > > > Acked-by: Paul E. McKenney <paulmck@kernel.org>
> > >
> > > Much appreciated. Thanks, Paul!
> > >
> > > Any suggestions which tree this could eventually land in?
> >
> > I would guess that Dmitry might have some suggestions.
> 
> I checked and we're both unclear what the most obvious tree to land in
> is (the other sanitizers are mm related, which KCSAN is not).
> 
> One suggestion that comes to my mind is for KCSAN to go through the
> same tree (rcu?) as the LKMM due to their inherent relationship. Would
> that make most sense?

It works for me, though you guys have to continue to be the main
developers.  ;-)

I will go through the patches more carefully, and please look into the
kbuild test robot complaint.

							Thanx, Paul

> Thanks,
> -- Marco
> 
> > >
> > > > > Changelog
> > > > > ---------
> > > > > v3:
> > > > > * Major changes:
> > > > >  - Add microbenchmark.
> > > > >  - Add instruction watchpoint skip randomization.
> > > > >  - Refactor API and core runtime fast-path and slow-path. Compared to
> > > > >    the previous version, with a default config and benchmarked using the
> > > > >    added microbenchmark, this version is 3.8x faster.
> > > > >  - Make __tsan_unaligned __alias of generic accesses.
> > > > >  - Rename kcsan_{begin,end}_atomic ->
> > > > >    kcsan_{nestable,flat}_atomic_{begin,end}
> > > > >  - For filter list in debugfs.c use kmalloc+krealloc instead of
> > > > >    kvmalloc.
> > > > >  - Split Documentation into separate patch.
> > > > >
> > > > > v2: http://lkml.kernel.org/r/20191017141305.146193-1-elver@google.com
> > > > > * Major changes:
> > > > >  - Replace kcsan_check_access(.., {true, false}) with
> > > > >    kcsan_check_{read,write}.
> > > > >  - Change atomic-instrumented.h to use __atomic_check_{read,write}.
> > > > >  - Use common struct kcsan_ctx in task_struct and for per-CPU interrupt
> > > > >    contexts.
> > > > >
> > > > > v1: http://lkml.kernel.org/r/20191016083959.186860-1-elver@google.com
> > > > >
> > > > > Marco Elver (9):
> > > > >   kcsan: Add Kernel Concurrency Sanitizer infrastructure
> > > > >   kcsan: Add Documentation entry in dev-tools
> > > > >   objtool, kcsan: Add KCSAN runtime functions to whitelist
> > > > >   build, kcsan: Add KCSAN build exceptions
> > > > >   seqlock, kcsan: Add annotations for KCSAN
> > > > >   seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
> > > > >   asm-generic, kcsan: Add KCSAN instrumentation for bitops
> > > > >   locking/atomics, kcsan: Add KCSAN instrumentation
> > > > >   x86, kcsan: Enable KCSAN for x86
> > > > >
> > > > >  Documentation/dev-tools/index.rst         |   1 +
> > > > >  Documentation/dev-tools/kcsan.rst         | 217 +++++++++
> > > > >  MAINTAINERS                               |  11 +
> > > > >  Makefile                                  |   3 +-
> > > > >  arch/x86/Kconfig                          |   1 +
> > > > >  arch/x86/boot/Makefile                    |   2 +
> > > > >  arch/x86/boot/compressed/Makefile         |   2 +
> > > > >  arch/x86/entry/vdso/Makefile              |   3 +
> > > > >  arch/x86/include/asm/bitops.h             |   6 +-
> > > > >  arch/x86/kernel/Makefile                  |   7 +
> > > > >  arch/x86/kernel/cpu/Makefile              |   3 +
> > > > >  arch/x86/lib/Makefile                     |   4 +
> > > > >  arch/x86/mm/Makefile                      |   3 +
> > > > >  arch/x86/purgatory/Makefile               |   2 +
> > > > >  arch/x86/realmode/Makefile                |   3 +
> > > > >  arch/x86/realmode/rm/Makefile             |   3 +
> > > > >  drivers/firmware/efi/libstub/Makefile     |   2 +
> > > > >  include/asm-generic/atomic-instrumented.h | 393 +++++++--------
> > > > >  include/asm-generic/bitops-instrumented.h |  18 +
> > > > >  include/linux/compiler-clang.h            |   9 +
> > > > >  include/linux/compiler-gcc.h              |   7 +
> > > > >  include/linux/compiler.h                  |  35 +-
> > > > >  include/linux/kcsan-checks.h              |  97 ++++
> > > > >  include/linux/kcsan.h                     | 115 +++++
> > > > >  include/linux/sched.h                     |   4 +
> > > > >  include/linux/seqlock.h                   |  51 +-
> > > > >  init/init_task.c                          |   8 +
> > > > >  init/main.c                               |   2 +
> > > > >  kernel/Makefile                           |   6 +
> > > > >  kernel/kcsan/Makefile                     |  11 +
> > > > >  kernel/kcsan/atomic.h                     |  27 ++
> > > > >  kernel/kcsan/core.c                       | 560 ++++++++++++++++++++++
> > > > >  kernel/kcsan/debugfs.c                    | 275 +++++++++++
> > > > >  kernel/kcsan/encoding.h                   |  94 ++++
> > > > >  kernel/kcsan/kcsan.h                      | 131 +++++
> > > > >  kernel/kcsan/report.c                     | 306 ++++++++++++
> > > > >  kernel/kcsan/test.c                       | 121 +++++
> > > > >  kernel/sched/Makefile                     |   6 +
> > > > >  lib/Kconfig.debug                         |   2 +
> > > > >  lib/Kconfig.kcsan                         | 119 +++++
> > > > >  lib/Makefile                              |   3 +
> > > > >  mm/Makefile                               |   8 +
> > > > >  scripts/Makefile.kcsan                    |   6 +
> > > > >  scripts/Makefile.lib                      |  10 +
> > > > >  scripts/atomic/gen-atomic-instrumented.sh |  17 +-
> > > > >  tools/objtool/check.c                     |  18 +
> > > > >  46 files changed, 2526 insertions(+), 206 deletions(-)
> > > > >  create mode 100644 Documentation/dev-tools/kcsan.rst
> > > > >  create mode 100644 include/linux/kcsan-checks.h
> > > > >  create mode 100644 include/linux/kcsan.h
> > > > >  create mode 100644 kernel/kcsan/Makefile
> > > > >  create mode 100644 kernel/kcsan/atomic.h
> > > > >  create mode 100644 kernel/kcsan/core.c
> > > > >  create mode 100644 kernel/kcsan/debugfs.c
> > > > >  create mode 100644 kernel/kcsan/encoding.h
> > > > >  create mode 100644 kernel/kcsan/kcsan.h
> > > > >  create mode 100644 kernel/kcsan/report.c
> > > > >  create mode 100644 kernel/kcsan/test.c
> > > > >  create mode 100644 lib/Kconfig.kcsan
> > > > >  create mode 100644 scripts/Makefile.kcsan
> > > > >
> > > > > --
> > > > > 2.24.0.rc1.363.gb1bccd3e3d-goog
> > > > >
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> > > > To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191104164717.GE20975%40paulmck-ThinkPad-P72.
> >
> > --
> > You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191104194658.GK20975%40paulmck-ThinkPad-P72.
Marco Elver Nov. 5, 2019, 3:25 p.m. UTC | #6
On Tue, 5 Nov 2019 at 15:20, Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Tue, Nov 05, 2019 at 12:10:56PM +0100, Marco Elver wrote:
> > On Mon, 4 Nov 2019 at 20:47, Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Mon, Nov 04, 2019 at 07:41:30PM +0100, Marco Elver wrote:
> > > > On Mon, 4 Nov 2019 at 17:47, Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > >
> > > > > On Mon, Nov 04, 2019 at 03:27:36PM +0100, Marco Elver wrote:
> > > > > > This is the patch-series for the Kernel Concurrency Sanitizer (KCSAN).
> > > > > > KCSAN is a sampling watchpoint-based data-race detector. More details
> > > > > > are included in Documentation/dev-tools/kcsan.rst. This patch-series
> > > > > > only enables KCSAN for x86, but we expect adding support for other
> > > > > > architectures is relatively straightforward (we are aware of
> > > > > > experimental ARM64 and POWER support).
> > > > > >
> > > > > > To gather early feedback, we announced KCSAN back in September, and
> > > > > > have integrated the feedback where possible:
> > > > > > http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com
> > > > > >
> > > > > > We want to point out and acknowledge the work surrounding the LKMM,
> > > > > > including several articles that motivate why data-races are dangerous
> > > > > > [1, 2], justifying a data-race detector such as KCSAN.
> > > > > > [1] https://lwn.net/Articles/793253/
> > > > > > [2] https://lwn.net/Articles/799218/
> > > > > >
> > > > > > The current list of known upstream fixes for data-races found by KCSAN
> > > > > > can be found here:
> > > > > > https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan
> > > > >
> > > > > Making this more accessible to more people seems like a good thing.
> > > > > So, for the series:
> > > > >
> > > > > Acked-by: Paul E. McKenney <paulmck@kernel.org>
> > > >
> > > > Much appreciated. Thanks, Paul!
> > > >
> > > > Any suggestions which tree this could eventually land in?
> > >
> > > I would guess that Dmitry might have some suggestions.
> >
> > I checked and we're both unclear what the most obvious tree to land in
> > is (the other sanitizers are mm related, which KCSAN is not).
> >
> > One suggestion that comes to my mind is for KCSAN to go through the
> > same tree (rcu?) as the LKMM due to their inherent relationship. Would
> > that make most sense?
>
> It works for me, though you guys have to continue to be the main
> developers.  ;-)

Great, thanks. We did add an entry to MAINTAINERS, so yes of course. :-)

> I will go through the patches more carefully, and please look into the
> kbuild test robot complaint.

I just responded to that, it seems to be a sparse problem.

Thanks,
-- Marco

>                                                         Thanx, Paul
>
> > Thanks,
> > -- Marco
> >
> > > >
> > > > > > Changelog
> > > > > > ---------
> > > > > > v3:
> > > > > > * Major changes:
> > > > > >  - Add microbenchmark.
> > > > > >  - Add instruction watchpoint skip randomization.
> > > > > >  - Refactor API and core runtime fast-path and slow-path. Compared to
> > > > > >    the previous version, with a default config and benchmarked using the
> > > > > >    added microbenchmark, this version is 3.8x faster.
> > > > > >  - Make __tsan_unaligned __alias of generic accesses.
> > > > > >  - Rename kcsan_{begin,end}_atomic ->
> > > > > >    kcsan_{nestable,flat}_atomic_{begin,end}
> > > > > >  - For filter list in debugfs.c use kmalloc+krealloc instead of
> > > > > >    kvmalloc.
> > > > > >  - Split Documentation into separate patch.
> > > > > >
> > > > > > v2: http://lkml.kernel.org/r/20191017141305.146193-1-elver@google.com
> > > > > > * Major changes:
> > > > > >  - Replace kcsan_check_access(.., {true, false}) with
> > > > > >    kcsan_check_{read,write}.
> > > > > >  - Change atomic-instrumented.h to use __atomic_check_{read,write}.
> > > > > >  - Use common struct kcsan_ctx in task_struct and for per-CPU interrupt
> > > > > >    contexts.
> > > > > >
> > > > > > v1: http://lkml.kernel.org/r/20191016083959.186860-1-elver@google.com
> > > > > >
> > > > > > Marco Elver (9):
> > > > > >   kcsan: Add Kernel Concurrency Sanitizer infrastructure
> > > > > >   kcsan: Add Documentation entry in dev-tools
> > > > > >   objtool, kcsan: Add KCSAN runtime functions to whitelist
> > > > > >   build, kcsan: Add KCSAN build exceptions
> > > > > >   seqlock, kcsan: Add annotations for KCSAN
> > > > > >   seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
> > > > > >   asm-generic, kcsan: Add KCSAN instrumentation for bitops
> > > > > >   locking/atomics, kcsan: Add KCSAN instrumentation
> > > > > >   x86, kcsan: Enable KCSAN for x86
> > > > > >
> > > > > >  Documentation/dev-tools/index.rst         |   1 +
> > > > > >  Documentation/dev-tools/kcsan.rst         | 217 +++++++++
> > > > > >  MAINTAINERS                               |  11 +
> > > > > >  Makefile                                  |   3 +-
> > > > > >  arch/x86/Kconfig                          |   1 +
> > > > > >  arch/x86/boot/Makefile                    |   2 +
> > > > > >  arch/x86/boot/compressed/Makefile         |   2 +
> > > > > >  arch/x86/entry/vdso/Makefile              |   3 +
> > > > > >  arch/x86/include/asm/bitops.h             |   6 +-
> > > > > >  arch/x86/kernel/Makefile                  |   7 +
> > > > > >  arch/x86/kernel/cpu/Makefile              |   3 +
> > > > > >  arch/x86/lib/Makefile                     |   4 +
> > > > > >  arch/x86/mm/Makefile                      |   3 +
> > > > > >  arch/x86/purgatory/Makefile               |   2 +
> > > > > >  arch/x86/realmode/Makefile                |   3 +
> > > > > >  arch/x86/realmode/rm/Makefile             |   3 +
> > > > > >  drivers/firmware/efi/libstub/Makefile     |   2 +
> > > > > >  include/asm-generic/atomic-instrumented.h | 393 +++++++--------
> > > > > >  include/asm-generic/bitops-instrumented.h |  18 +
> > > > > >  include/linux/compiler-clang.h            |   9 +
> > > > > >  include/linux/compiler-gcc.h              |   7 +
> > > > > >  include/linux/compiler.h                  |  35 +-
> > > > > >  include/linux/kcsan-checks.h              |  97 ++++
> > > > > >  include/linux/kcsan.h                     | 115 +++++
> > > > > >  include/linux/sched.h                     |   4 +
> > > > > >  include/linux/seqlock.h                   |  51 +-
> > > > > >  init/init_task.c                          |   8 +
> > > > > >  init/main.c                               |   2 +
> > > > > >  kernel/Makefile                           |   6 +
> > > > > >  kernel/kcsan/Makefile                     |  11 +
> > > > > >  kernel/kcsan/atomic.h                     |  27 ++
> > > > > >  kernel/kcsan/core.c                       | 560 ++++++++++++++++++++++
> > > > > >  kernel/kcsan/debugfs.c                    | 275 +++++++++++
> > > > > >  kernel/kcsan/encoding.h                   |  94 ++++
> > > > > >  kernel/kcsan/kcsan.h                      | 131 +++++
> > > > > >  kernel/kcsan/report.c                     | 306 ++++++++++++
> > > > > >  kernel/kcsan/test.c                       | 121 +++++
> > > > > >  kernel/sched/Makefile                     |   6 +
> > > > > >  lib/Kconfig.debug                         |   2 +
> > > > > >  lib/Kconfig.kcsan                         | 119 +++++
> > > > > >  lib/Makefile                              |   3 +
> > > > > >  mm/Makefile                               |   8 +
> > > > > >  scripts/Makefile.kcsan                    |   6 +
> > > > > >  scripts/Makefile.lib                      |  10 +
> > > > > >  scripts/atomic/gen-atomic-instrumented.sh |  17 +-
> > > > > >  tools/objtool/check.c                     |  18 +
> > > > > >  46 files changed, 2526 insertions(+), 206 deletions(-)
> > > > > >  create mode 100644 Documentation/dev-tools/kcsan.rst
> > > > > >  create mode 100644 include/linux/kcsan-checks.h
> > > > > >  create mode 100644 include/linux/kcsan.h
> > > > > >  create mode 100644 kernel/kcsan/Makefile
> > > > > >  create mode 100644 kernel/kcsan/atomic.h
> > > > > >  create mode 100644 kernel/kcsan/core.c
> > > > > >  create mode 100644 kernel/kcsan/debugfs.c
> > > > > >  create mode 100644 kernel/kcsan/encoding.h
> > > > > >  create mode 100644 kernel/kcsan/kcsan.h
> > > > > >  create mode 100644 kernel/kcsan/report.c
> > > > > >  create mode 100644 kernel/kcsan/test.c
> > > > > >  create mode 100644 lib/Kconfig.kcsan
> > > > > >  create mode 100644 scripts/Makefile.kcsan
> > > > > >
> > > > > > --
> > > > > > 2.24.0.rc1.363.gb1bccd3e3d-goog
> > > > > >
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> > > > > To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191104164717.GE20975%40paulmck-ThinkPad-P72.
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> > > To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191104194658.GK20975%40paulmck-ThinkPad-P72.
>
> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20191105142035.GR20975%40paulmck-ThinkPad-P72.
Marco Elver Nov. 14, 2019, 6:05 p.m. UTC | #7
On Tue, 5 Nov 2019 at 16:25, Marco Elver <elver@google.com> wrote:
>
> On Tue, 5 Nov 2019 at 15:20, Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Tue, Nov 05, 2019 at 12:10:56PM +0100, Marco Elver wrote:
> > > On Mon, 4 Nov 2019 at 20:47, Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Mon, Nov 04, 2019 at 07:41:30PM +0100, Marco Elver wrote:
> > > > > On Mon, 4 Nov 2019 at 17:47, Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > > >
> > > > > > On Mon, Nov 04, 2019 at 03:27:36PM +0100, Marco Elver wrote:
> > > > > > > This is the patch-series for the Kernel Concurrency Sanitizer (KCSAN).
> > > > > > > KCSAN is a sampling watchpoint-based data-race detector. More details
> > > > > > > are included in Documentation/dev-tools/kcsan.rst. This patch-series
> > > > > > > only enables KCSAN for x86, but we expect adding support for other
> > > > > > > architectures is relatively straightforward (we are aware of
> > > > > > > experimental ARM64 and POWER support).
> > > > > > >
> > > > > > > To gather early feedback, we announced KCSAN back in September, and
> > > > > > > have integrated the feedback where possible:
> > > > > > > http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com
> > > > > > >
> > > > > > > We want to point out and acknowledge the work surrounding the LKMM,
> > > > > > > including several articles that motivate why data-races are dangerous
> > > > > > > [1, 2], justifying a data-race detector such as KCSAN.
> > > > > > > [1] https://lwn.net/Articles/793253/
> > > > > > > [2] https://lwn.net/Articles/799218/
> > > > > > >
> > > > > > > The current list of known upstream fixes for data-races found by KCSAN
> > > > > > > can be found here:
> > > > > > > https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan
> > > > > >
> > > > > > Making this more accessible to more people seems like a good thing.
> > > > > > So, for the series:
> > > > > >
> > > > > > Acked-by: Paul E. McKenney <paulmck@kernel.org>
> > > > >
> > > > > Much appreciated. Thanks, Paul!
> > > > >
> > > > > Any suggestions which tree this could eventually land in?
> > > >
> > > > I would guess that Dmitry might have some suggestions.
> > >
> > > I checked and we're both unclear what the most obvious tree to land in
> > > is (the other sanitizers are mm related, which KCSAN is not).
> > >
> > > One suggestion that comes to my mind is for KCSAN to go through the
> > > same tree (rcu?) as the LKMM due to their inherent relationship. Would
> > > that make most sense?
> >
> > It works for me, though you guys have to continue to be the main
> > developers.  ;-)
>
> Great, thanks. We did add an entry to MAINTAINERS, so yes of course. :-)
>
> > I will go through the patches more carefully, and please look into the
> > kbuild test robot complaint.
>
> I just responded to that, it seems to be a sparse problem.
>
> Thanks,
> -- Marco

v4 was sent out:
http://lkml.kernel.org/r/20191114180303.66955-1-elver@google.com

Thanks,
-- Marco
Paul E. McKenney Nov. 14, 2019, 7:48 p.m. UTC | #8
On Thu, Nov 14, 2019 at 07:05:34PM +0100, Marco Elver wrote:
> On Tue, 5 Nov 2019 at 16:25, Marco Elver <elver@google.com> wrote:
> > On Tue, 5 Nov 2019 at 15:20, Paul E. McKenney <paulmck@kernel.org> wrote:

[ . . . ]

> > > It works for me, though you guys have to continue to be the main
> > > developers.  ;-)
> >
> > Great, thanks. We did add an entry to MAINTAINERS, so yes of course. :-)
> >
> > > I will go through the patches more carefully, and please look into the
> > > kbuild test robot complaint.
> >
> > I just responded to that, it seems to be a sparse problem.
> >
> > Thanks,
> > -- Marco
> 
> v4 was sent out:
> http://lkml.kernel.org/r/20191114180303.66955-1-elver@google.com

And I have queued it and pushed it to -rcu.  It is still in the section
of -rcu subject to rebasing, so if you have a later v5, I can replace
this with the newer version.

I am assuming that you do -not- wish to target the upcoming merge window
(v5.5), but rather then next one (v5.6).  Please let me know right away
if I am assuming wrong.

							Thanx, Paul