[v7,00/25] Unify vDSOs across more architectures
mbox series

Message ID 20190621095252.32307-1-vincenzo.frascino@arm.com
Headers show
Series
  • Unify vDSOs across more architectures
Related show

Message

Vincenzo Frascino June 21, 2019, 9:52 a.m. UTC
vDSO (virtual dynamic shared object) is a mechanism that the Linux
kernel provides as an alternative to system calls to reduce where
possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do
not write any data and return one or more values that are stored
in the kernel, which makes relatively safe calling them directly
as a library function.

Even if the mechanism is pretty much standard, every architecture
in the last few years ended up implementing their own vDSO library
in the architectural code.

The purpose of this patch-set is to identify the commonalities in
between the architectures and try to consolidate the common code
paths, starting with gettimeofday().

This implementation contains the following design choices:
 * Every architecture defines the arch specific code in an header in
   "asm/vdso/".
 * The generic implementation includes the arch specific one and lives
   in "lib/vdso".
 * The arch specific code for gettimeofday lives in
   "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
 * The generic implementation of update_vsyscall and update_vsyscall_tz
   lives in kernel/vdso and provide the bindings that can be implemented
   by each architecture.
 * Each architecture provides its implementation of the bindings in
   "asm/vdso/vsyscall.h".
 * This approach allows to consolidate the common code in a single place
   with the benefit of avoiding code duplication.

This implementation contains the portings to the common library for: arm64,
compat mode for arm64, arm, mips, x86_64, x32, compat mode for x86_64 and
i386.

The mips porting has been tested on qemu for mips32el. A configuration to
repeat the tests can be found at [4].

The x86_64 porting has been tested on an Intel Xeon 5120T based machine
running Ubuntu 18.04 and using the Ubuntu provided defconfig.

The i386 porting has been tested on qemu using the i386_defconfig
configuration.

Last but not least from this porting arm64, compat arm64, arm and mips gain
the support for:
 * CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps
   track of the time during sleep as well.
 * CLOCK_TAI that is like CLOCK_REALTIME, but uses the International
   Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
   second updates.
for both clock_gettime and clock_getres.

The porting has been validated using the vdsotest test-suite [1] extended
to cover all the clock ids [2].

A new test has been added to the linux kselftest in order to validate the
newly added library.

The porting has been benchmarked and the performance results are
provided as part of this cover letter.

To simplify the testing, a copy of the patchset on top of a recent linux
tree can be found at [3] and [4].

The v7 of this patchseries has been rebased on [5].

[1] https://github.com/nathanlynch/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v7
[4] git://linux-arm.org/linux-vf.git vdso-mips/v7
[5] git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git hyperv-next

Changes:
--------
v7:
  - Rebased on [5] (5.2-rc3).
  - Added performance numbers for arm64 provided by Shijith Thotton.
  - Aimed at 1:1 replacement for pre-exisiting vDSO libraries.
  - Provided separate patches for newly added API.
  - Addressed review comments.
v6:
  - Rebased on 5.2-rc2.
  - Added performance numbers.
  - Removed vdso_types.h.
  - Unified update_vsyscall and update_vsyscall_tz.
  - Reworked the kselftest included in this patchset.
  - Addressed review comments.
v5:
  - Rebased on 5.0-rc7.
  - Added x86_64, compat mode for x86_64 and i386 portings.
  - Extended vDSO kselftest.
  - Addressed review comments.
v4:
  - Rebased on 5.0-rc2.
  - Addressed review comments.
  - Disabled compat vdso on arm64 when the kernel is compiled with
    clang.
v3:
  - Ported the latest fixes and optimizations done on the x86
    architecture to the generic library.
  - Addressed review comments.
  - Improved the documentation of the interfaces.
  - Changed the HAVE_ARCH_TIMER config option to a more generic
    HAVE_HW_COUNTER.
v2:
  - Added -ffixed-x18 to arm64
  - Repleced occurrences of timeval and timespec
  - Modified datapage.h to be compliant with y2038 on all the architectures
  - Removed __u_vdso type

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Dmitry Safonov <0x7f454c46@gmail.com>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Huw Davies <huw@codeweavers.com>
Cc: Shijith Thotton <sthotton@marvell.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Tested-by: Shijith Thotton <sthotton@marvell.com>
Tested-by: Andre Przywara <andre.przywara@arm.com>

Performance Numbers: Linux 5.2.0-rc2 - Xeon Gold 5120T
======================================================

Unified vDSO:
-------------

clock-gettime-monotonic: syscall: 342 nsec/call
clock-gettime-monotonic:    libc: 25 nsec/call
clock-gettime-monotonic:    vdso: 24 nsec/call
clock-getres-monotonic: syscall: 296 nsec/call
clock-getres-monotonic:    libc: 296 nsec/call
clock-getres-monotonic:    vdso: 3 nsec/call
clock-gettime-monotonic-coarse: syscall: 294 nsec/call
clock-gettime-monotonic-coarse:    libc: 5 nsec/call
clock-gettime-monotonic-coarse:    vdso: 5 nsec/call
clock-getres-monotonic-coarse: syscall: 295 nsec/call
clock-getres-monotonic-coarse:    libc: 292 nsec/call
clock-getres-monotonic-coarse:    vdso: 5 nsec/call
clock-gettime-monotonic-raw: syscall: 343 nsec/call
clock-gettime-monotonic-raw:    libc: 25 nsec/call
clock-gettime-monotonic-raw:    vdso: 23 nsec/call
clock-getres-monotonic-raw: syscall: 290 nsec/call
clock-getres-monotonic-raw:    libc: 290 nsec/call
clock-getres-monotonic-raw:    vdso: 4 nsec/call
clock-gettime-tai: syscall: 332 nsec/call
clock-gettime-tai:    libc: 24 nsec/call
clock-gettime-tai:    vdso: 23 nsec/call
clock-getres-tai: syscall: 288 nsec/call
clock-getres-tai:    libc: 288 nsec/call
clock-getres-tai:    vdso: 3 nsec/call
clock-gettime-boottime: syscall: 342 nsec/call
clock-gettime-boottime:    libc: 24 nsec/call
clock-gettime-boottime:    vdso: 23 nsec/call
clock-getres-boottime: syscall: 284 nsec/call
clock-getres-boottime:    libc: 291 nsec/call
clock-getres-boottime:    vdso: 3 nsec/call
clock-gettime-realtime: syscall: 337 nsec/call
clock-gettime-realtime:    libc: 24 nsec/call
clock-gettime-realtime:    vdso: 23 nsec/call
clock-getres-realtime: syscall: 287 nsec/call
clock-getres-realtime:    libc: 284 nsec/call
clock-getres-realtime:    vdso: 3 nsec/call
clock-gettime-realtime-coarse: syscall: 307 nsec/call
clock-gettime-realtime-coarse:    libc: 4 nsec/call
clock-gettime-realtime-coarse:    vdso: 4 nsec/call
clock-getres-realtime-coarse: syscall: 294 nsec/call
clock-getres-realtime-coarse:    libc: 291 nsec/call
clock-getres-realtime-coarse:    vdso: 4 nsec/call
getcpu: syscall: 246 nsec/call
getcpu:    libc: 14 nsec/call
getcpu:    vdso: 11 nsec/call
gettimeofday: syscall: 293 nsec/call
gettimeofday:    libc: 26 nsec/call
gettimeofday:    vdso: 25 nsec/call

Stock Kernel:
-------------

clock-gettime-monotonic: syscall: 338 nsec/call
clock-gettime-monotonic:    libc: 24 nsec/call
clock-gettime-monotonic:    vdso: 23 nsec/call
clock-getres-monotonic: syscall: 291 nsec/call
clock-getres-monotonic:    libc: 304 nsec/call
clock-getres-monotonic:    vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-monotonic-coarse: syscall: 297 nsec/call
clock-gettime-monotonic-coarse:    libc: 5 nsec/call
clock-gettime-monotonic-coarse:    vdso: 4 nsec/call
clock-getres-monotonic-coarse: syscall: 281 nsec/call
clock-getres-monotonic-coarse:    libc: 286 nsec/call
clock-getres-monotonic-coarse:    vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-monotonic-raw: syscall: 336 nsec/call
clock-gettime-monotonic-raw:    libc: 340 nsec/call
clock-gettime-monotonic-raw:    vdso: 346 nsec/call
clock-getres-monotonic-raw: syscall: 297 nsec/call
clock-getres-monotonic-raw:    libc: 301 nsec/call
clock-getres-monotonic-raw:    vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-tai: syscall: 351 nsec/call
clock-gettime-tai:    libc: 24 nsec/call
clock-gettime-tai:    vdso: 23 nsec/call
clock-getres-tai: syscall: 298 nsec/call
clock-getres-tai:    libc: 290 nsec/call
clock-getres-tai:    vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-boottime: syscall: 342 nsec/call
clock-gettime-boottime:    libc: 347 nsec/call
clock-gettime-boottime:    vdso: 355 nsec/call
clock-getres-boottime: syscall: 296 nsec/call
clock-getres-boottime:    libc: 295 nsec/call
clock-getres-boottime:    vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-realtime: syscall: 346 nsec/call
clock-gettime-realtime:    libc: 24 nsec/call
clock-gettime-realtime:    vdso: 22 nsec/call
clock-getres-realtime: syscall: 295 nsec/call
clock-getres-realtime:    libc: 291 nsec/call
clock-getres-realtime:    vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-realtime-coarse: syscall: 292 nsec/call
clock-gettime-realtime-coarse:    libc: 5 nsec/call
clock-gettime-realtime-coarse:    vdso: 4 nsec/call
clock-getres-realtime-coarse: syscall: 300 nsec/call
clock-getres-realtime-coarse:    libc: 301 nsec/call
clock-getres-realtime-coarse:    vdso: not tested
Note: vDSO version of clock_getres not found
getcpu: syscall: 252 nsec/call
getcpu:    libc: 14 nsec/call
getcpu:    vdso: 11 nsec/call
gettimeofday: syscall: 293 nsec/call
gettimeofday:    libc: 24 nsec/call
gettimeofday:    vdso: 25 nsec/call

Performance Numbers: Linux 5.2.0-rc2 - ThunderX2 (arm64)
========================================================

Provided by: Shijith Thotton <sthotton@marvell.com>

Unified vDSO:
-------------

clock-gettime-monotonic: syscall: 346 nsec/call
clock-gettime-monotonic:    libc: 38 nsec/call
clock-gettime-monotonic:    vdso: 36 nsec/call
clock-getres-monotonic: syscall: 262 nsec/call
clock-getres-monotonic:    libc: 6 nsec/call
clock-getres-monotonic:    vdso: 5 nsec/call
clock-gettime-monotonic-coarse: syscall: 296 nsec/call
clock-gettime-monotonic-coarse:    libc: 39 nsec/call
clock-gettime-monotonic-coarse:    vdso: 38 nsec/call
clock-getres-monotonic-coarse: syscall: 260 nsec/call
clock-getres-monotonic-coarse:    libc: 8 nsec/call
clock-getres-monotonic-coarse:    vdso: 5 nsec/call
clock-gettime-monotonic-raw: syscall: 345 nsec/call
clock-gettime-monotonic-raw:    libc: 35 nsec/call
clock-gettime-monotonic-raw:    vdso: 34 nsec/call
clock-getres-monotonic-raw: syscall: 261 nsec/call
clock-getres-monotonic-raw:    libc: 7 nsec/call
clock-getres-monotonic-raw:    vdso: 5 nsec/call
clock-gettime-tai: syscall: 357 nsec/call
clock-gettime-tai:    libc: 38 nsec/call
clock-gettime-tai:    vdso: 36 nsec/call
clock-getres-tai: syscall: 257 nsec/call
clock-getres-tai:    libc: 7 nsec/call
clock-getres-tai:    vdso: 5 nsec/call
clock-gettime-boottime: syscall: 356 nsec/call
clock-gettime-boottime:    libc: 38 nsec/call
clock-gettime-boottime:    vdso: 36 nsec/call
clock-getres-boottime: syscall: 257 nsec/call
clock-getres-boottime:    libc: 6 nsec/call
clock-getres-boottime:    vdso: 5 nsec/call
clock-gettime-realtime: syscall: 345 nsec/call
clock-gettime-realtime:    libc: 38 nsec/call
clock-gettime-realtime:    vdso: 36 nsec/call
clock-getres-realtime: syscall: 257 nsec/call
clock-getres-realtime:    libc: 7 nsec/call
clock-getres-realtime:    vdso: 5 nsec/call
clock-gettime-realtime-coarse: syscall: 295 nsec/call
clock-gettime-realtime-coarse:    libc: 39 nsec/call
clock-gettime-realtime-coarse:    vdso: 38 nsec/call
clock-getres-realtime-coarse: syscall: 260 nsec/call
clock-getres-realtime-coarse:    libc: 8 nsec/call
clock-getres-realtime-coarse:    vdso: 5 nsec/call
getcpu: syscall: 244 nsec/call
getcpu:    libc: 247 nsec/call
getcpu:    vdso: not tested
Note: vDSO version of getcpu not found
gettimeofday: syscall: 383 nsec/call
gettimeofday:    libc: 39 nsec/call
gettimeofday:    vdso: 35 nsec/call

Stock Kernel:
-------------

clock-gettime-monotonic: syscall: 344 nsec/call
clock-gettime-monotonic:    libc: 74 nsec/call
clock-gettime-monotonic:    vdso: 73 nsec/call
clock-getres-monotonic: syscall: 258 nsec/call
clock-getres-monotonic:    libc: 6 nsec/call
clock-getres-monotonic:    vdso: 4 nsec/call
clock-gettime-monotonic-coarse: syscall: 300 nsec/call
clock-gettime-monotonic-coarse:    libc: 36 nsec/call
clock-gettime-monotonic-coarse:    vdso: 34 nsec/call
clock-getres-monotonic-coarse: syscall: 261 nsec/call
clock-getres-monotonic-coarse:    libc: 6 nsec/call
clock-getres-monotonic-coarse:    vdso: 4 nsec/call
clock-gettime-monotonic-raw: syscall: 346 nsec/call
clock-gettime-monotonic-raw:    libc: 74 nsec/call
clock-gettime-monotonic-raw:    vdso: 72 nsec/call
clock-getres-monotonic-raw: syscall: 254 nsec/call
clock-getres-monotonic-raw:    libc: 6 nsec/call
clock-getres-monotonic-raw:    vdso: 4 nsec/call
clock-gettime-tai: syscall: 345 nsec/call
clock-gettime-tai:    libc: 361 nsec/call
clock-gettime-tai:    vdso: 359 nsec/call
clock-getres-tai: syscall: 259 nsec/call
clock-getres-tai:    libc: 262 nsec/call
clock-getres-tai:    vdso: 258 nsec/call
clock-gettime-boottime: syscall: 353 nsec/call
clock-gettime-boottime:    libc: 365 nsec/call
clock-gettime-boottime:    vdso: 362 nsec/call
clock-getres-boottime: syscall: 260 nsec/call
clock-getres-boottime:    libc: 267 nsec/call
clock-getres-boottime:    vdso: 259 nsec/call
clock-gettime-realtime: syscall: 344 nsec/call
clock-gettime-realtime:    libc: 73 nsec/call
clock-gettime-realtime:    vdso: 72 nsec/call
clock-getres-realtime: syscall: 255 nsec/call
clock-getres-realtime:    libc: 7 nsec/call
clock-getres-realtime:    vdso: 4 nsec/call
clock-gettime-realtime-coarse: syscall: 296 nsec/call
clock-gettime-realtime-coarse:    libc: 35 nsec/call
clock-gettime-realtime-coarse:    vdso: 33 nsec/call
clock-getres-realtime-coarse: syscall: 258 nsec/call
clock-getres-realtime-coarse:    libc: 6 nsec/call
clock-getres-realtime-coarse:    vdso: 4 nsec/call
getcpu: syscall: 237 nsec/call
getcpu:    libc: 242 nsec/call
getcpu:    vdso: not tested
Note: vDSO version of getcpu not found
gettimeofday: syscall: 378 nsec/call
gettimeofday:    libc: 73 nsec/call
gettimeofday:    vdso: 70 nsec/call

Peter Collingbourne (1):
  arm64: Build vDSO with -ffixed-x18

Vincenzo Frascino (24):
  kernel: Standardize vdso_datapage
  kernel: Define gettimeofday vdso common code
  kernel: Unify update_vsyscall implementation
  arm64: Substitute gettimeofday with C implementation
  arm64: compat: Add missing syscall numbers
  arm64: compat: Expose signal related structures
  arm64: compat: Generate asm offsets for signals
  lib: vdso: Add compat support
  arm64: compat: Add vDSO
  arm64: Refactor vDSO code
  arm64: compat: vDSO setup for compat layer
  arm64: elf: vDSO code page discovery
  arm64: compat: Get sigreturn trampolines from vDSO
  arm64: Add vDSO compat support
  arm: Add support for generic vDSO
  arm: Add clock_getres entry point
  arm: Add clock_gettime64 entry point
  mips: Add support for generic vDSO
  mips: Add clock_getres entry point
  mips: Add clock_gettime64 entry point
  x86: Add support for generic vDSO
  x86: Add clock_getres entry point
  x86: Add clock_gettime64 entry point
  kselftest: Extend vDSO selftest

 arch/arm/Kconfig                              |   3 +
 arch/arm/include/asm/vdso/gettimeofday.h      |  94 +++++
 arch/arm/include/asm/vdso/vsyscall.h          |  71 ++++
 arch/arm/include/asm/vdso_datapage.h          |  29 +-
 arch/arm/kernel/vdso.c                        |  87 +----
 arch/arm/vdso/Makefile                        |  13 +-
 arch/arm/vdso/note.c                          |  15 +
 arch/arm/vdso/vdso.lds.S                      |   2 +
 arch/arm/vdso/vgettimeofday.c                 | 256 +------------
 arch/arm64/Kconfig                            |   3 +
 arch/arm64/Makefile                           |  23 +-
 arch/arm64/include/asm/elf.h                  |  14 +
 arch/arm64/include/asm/signal32.h             |  46 +++
 arch/arm64/include/asm/unistd.h               |   5 +
 arch/arm64/include/asm/vdso.h                 |   3 +
 arch/arm64/include/asm/vdso/compat_barrier.h  |  51 +++
 .../include/asm/vdso/compat_gettimeofday.h    | 110 ++++++
 arch/arm64/include/asm/vdso/gettimeofday.h    |  86 +++++
 arch/arm64/include/asm/vdso/vsyscall.h        |  53 +++
 arch/arm64/include/asm/vdso_datapage.h        |  48 ---
 arch/arm64/kernel/Makefile                    |   6 +-
 arch/arm64/kernel/asm-offsets.c               |  39 +-
 arch/arm64/kernel/signal32.c                  |  72 ++--
 arch/arm64/kernel/vdso.c                      | 356 ++++++++++++------
 arch/arm64/kernel/vdso/Makefile               |  34 +-
 arch/arm64/kernel/vdso/gettimeofday.S         | 334 ----------------
 arch/arm64/kernel/vdso/vgettimeofday.c        |  28 ++
 arch/arm64/kernel/vdso32/.gitignore           |   2 +
 arch/arm64/kernel/vdso32/Makefile             | 186 +++++++++
 arch/arm64/kernel/vdso32/note.c               |  15 +
 arch/arm64/kernel/vdso32/sigreturn.S          |  62 +++
 arch/arm64/kernel/vdso32/vdso.S               |  19 +
 arch/arm64/kernel/vdso32/vdso.lds.S           |  82 ++++
 arch/arm64/kernel/vdso32/vgettimeofday.c      |  59 +++
 arch/mips/Kconfig                             |   2 +
 arch/mips/include/asm/vdso.h                  |  78 +---
 arch/mips/include/asm/vdso/gettimeofday.h     | 177 +++++++++
 arch/mips/{ => include/asm}/vdso/vdso.h       |   6 +-
 arch/mips/include/asm/vdso/vsyscall.h         |  43 +++
 arch/mips/kernel/vdso.c                       |  37 +-
 arch/mips/vdso/Makefile                       |  27 +-
 arch/mips/vdso/elf.S                          |   2 +-
 arch/mips/vdso/sigreturn.S                    |   2 +-
 arch/mips/vdso/vdso.lds.S                     |   4 +
 arch/mips/vdso/vgettimeofday.c                |  58 +++
 arch/x86/Kconfig                              |   3 +
 arch/x86/entry/vdso/Makefile                  |   9 +
 arch/x86/entry/vdso/vclock_gettime.c          | 249 +++---------
 arch/x86/entry/vdso/vdso.lds.S                |   2 +
 arch/x86/entry/vdso/vdso32/vdso32.lds.S       |   2 +
 arch/x86/entry/vdso/vdsox32.lds.S             |   1 +
 arch/x86/entry/vsyscall/Makefile              |   2 -
 arch/x86/entry/vsyscall/vsyscall_gtod.c       |  83 ----
 arch/x86/include/asm/pvclock.h                |   2 +-
 arch/x86/include/asm/vdso/gettimeofday.h      | 205 ++++++++++
 arch/x86/include/asm/vdso/vsyscall.h          |  44 +++
 arch/x86/include/asm/vgtod.h                  |  75 +---
 arch/x86/include/asm/vvar.h                   |   7 +-
 arch/x86/kernel/pvclock.c                     |   1 +
 include/asm-generic/vdso/vsyscall.h           |  56 +++
 include/linux/hrtimer.h                       |  15 +-
 include/linux/hrtimer_defs.h                  |  25 ++
 include/linux/timekeeper_internal.h           |   9 +
 include/vdso/datapage.h                       |  93 +++++
 include/vdso/helpers.h                        |  56 +++
 include/vdso/vsyscall.h                       |  11 +
 kernel/Makefile                               |   1 +
 kernel/vdso/Makefile                          |   2 +
 kernel/vdso/vsyscall.c                        | 131 +++++++
 lib/Kconfig                                   |   5 +
 lib/vdso/Kconfig                              |  36 ++
 lib/vdso/Makefile                             |  22 ++
 lib/vdso/gettimeofday.c                       | 231 ++++++++++++
 tools/testing/selftests/vDSO/Makefile         |   2 +
 tools/testing/selftests/vDSO/vdso_config.h    |  90 +++++
 tools/testing/selftests/vDSO/vdso_full_test.c | 244 ++++++++++++
 76 files changed, 3044 insertions(+), 1412 deletions(-)
 create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
 create mode 100644 arch/arm/include/asm/vdso/vsyscall.h
 create mode 100644 arch/arm/vdso/note.c
 create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
 create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
 create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
 create mode 100644 arch/arm64/include/asm/vdso/vsyscall.h
 delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
 delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
 create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
 create mode 100644 arch/arm64/kernel/vdso32/.gitignore
 create mode 100644 arch/arm64/kernel/vdso32/Makefile
 create mode 100644 arch/arm64/kernel/vdso32/note.c
 create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
 create mode 100644 arch/arm64/kernel/vdso32/vdso.S
 create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
 create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
 create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
 rename arch/mips/{ => include/asm}/vdso/vdso.h (89%)
 create mode 100644 arch/mips/include/asm/vdso/vsyscall.h
 create mode 100644 arch/mips/vdso/vgettimeofday.c
 delete mode 100644 arch/x86/entry/vsyscall/vsyscall_gtod.c
 create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
 create mode 100644 arch/x86/include/asm/vdso/vsyscall.h
 create mode 100644 include/asm-generic/vdso/vsyscall.h
 create mode 100644 include/linux/hrtimer_defs.h
 create mode 100644 include/vdso/datapage.h
 create mode 100644 include/vdso/helpers.h
 create mode 100644 include/vdso/vsyscall.h
 create mode 100644 kernel/vdso/Makefile
 create mode 100644 kernel/vdso/vsyscall.c
 create mode 100644 lib/vdso/Kconfig
 create mode 100644 lib/vdso/Makefile
 create mode 100644 lib/vdso/gettimeofday.c
 create mode 100644 tools/testing/selftests/vDSO/vdso_config.h
 create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c

Comments

Thomas Gleixner June 24, 2019, 12:34 a.m. UTC | #1
Vincenzo,

On Fri, 21 Jun 2019, Vincenzo Frascino wrote:
> vDSO (virtual dynamic shared object) is a mechanism that the Linux
> kernel provides as an alternative to system calls to reduce where
> possible the costs in terms of cycles.
> This is possible because certain syscalls like gettimeofday() do
> not write any data and return one or more values that are stored
> in the kernel, which makes relatively safe calling them directly
> as a library function.
> 
> Even if the mechanism is pretty much standard, every architecture
> in the last few years ended up implementing their own vDSO library
> in the architectural code.

....
 
> This implementation contains the portings to the common library for: arm64,
> compat mode for arm64, arm, mips, x86_64, x32, compat mode for x86_64 and
> i386.

I picked up the core implementation and the ARM64 and x86 conversion. I did
some refinements in several places, coding style, naming conventions,
comments and changelogs including subject prefixes. Please double check!

I did not merge the ARM and MIPS parts as they lack any form of
acknowlegment from their maintainers. Please talk to those folks. If they
ack/review the changes then I can pick them up and they go into 5.3 or they
have to go in a later cycle. Nevertheless it was well worth the trouble to
have those conversions done to confirm that the new common library fits a
bunch of different architectures.

As you can see from the commit dates, this has soaked for some time in a
WIP branch and I did extensive regression testing. So far so good.

Thanks a lot for going through several iterations. It's a very much
appreciated effort!

Especially with the upcoming time namespaces this will avoid a lot of
duplicated and pointlessly different horrors all over the architecture
space. Any architecture which wants to gain that support needs to convert
to the generic VDSO first.

As you have become the dude who knows almost everything about VDSO
including all the nasty pitfalls, I propose the patch below.

Thanks,

	tglx

8<------------
Subject: MAINTAINERS: Add entry for the generic VDSO library
From: Thomas Gleixner <tglx@linutronix.de>
Date: Mon, 24 Jun 2019 02:03:50 +0200

Asign the following folks in alphabetic order:

 - Andy for being the VDSO wizard of x86 and in general. He's also the
   performance monitor of choice and the code in the generic library is
   heavily influenced by his previous x86 VDSO work.

 - Thomas for being the dude who has to deal with any form of time(r)
   nonsense anyway

 - Vincenzo for being the poor sod who went through all the different
   architecture implementations in order to unify them. A lot of knowledge
   gained from VDSO implementation details to the intricacies of taming the
   build system.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 MAINTAINERS |   12 ++++++++++++
 1 file changed, 12 insertions(+)

--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6665,6 +6665,18 @@ L:	kvm@vger.kernel.org
 S:	Supported
 F:	drivers/uio/uio_pci_generic.c
 
+GENERIC VDSO LIBRARY:
+M:	Andy Lutomirksy <luto@kernel.org>
+M:	Thomas Gleixner <tglx@linutronix.de>
+M:	Vincenzo Frascino <vincenzo.frascino@arm.com>
+L:	linux-kernel@vger.kernel.org
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/vdso
+S:	Maintained
+F:	lib/vdso
+F:	kernel/time/vsyscall.c
+F:	include/vdso
+F:	include/asm-generic/vdso/vsyscall.h
+
 GENWQE (IBM Generic Workqueue Card)
 M:	Frank Haverkamp <haver@linux.ibm.com>
 S:	Supported
Andy Lutomirski June 24, 2019, 1:15 a.m. UTC | #2
On Sun, Jun 23, 2019 at 5:34 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> +GENERIC VDSO LIBRARY:
> +M:     Andy Lutomirksy <luto@kernel.org>

Lutomirski, perhaps?  Although I do appreciate the opportunity to say
"not me!" :)
Thomas Gleixner June 24, 2019, 7:42 a.m. UTC | #3
On Sun, 23 Jun 2019, Andy Lutomirski wrote:

> On Sun, Jun 23, 2019 at 5:34 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> > +GENERIC VDSO LIBRARY:
> > +M:     Andy Lutomirksy <luto@kernel.org>
> 
> Lutomirski, perhaps?

Ooops. Where did I copy that from?

> Although I do appreciate the opportunity to say "not me!" :)

You just gave me the perfect exit plan. I'll change my surname to Gleyxner
and head off to the goat farm :)

Thanks,

	tglx
Andre Przywara June 24, 2019, 12:50 p.m. UTC | #4
On Fri, 21 Jun 2019 10:52:27 +0100
Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:

Hi,

> vDSO (virtual dynamic shared object) is a mechanism that the Linux
> kernel provides as an alternative to system calls to reduce where
> possible the costs in terms of cycles.
[ ... ]

Some numbers for the ARM(32) part:

I booted my trusted old Calxeda Midway server (Cortex A-15 cores) and ran
the vdsotest benchmark on it. The results are:
(vdso: times, in nsec/call; n/t: "not tested" (=not implemented))
call				5.2-rc3	5.2-rc3-vdso
clock-gettime-monotonic:        147     142
clock-getres-monotonic:         n/t     34
clock-gettime-monotonic-coarse: 90      96
clock-getres-monotonic-coarse:  n/t     36
clock-gettime-monotonic-raw:    431     142
clock-getres-monotonic-raw:     n/t     35
clock-gettime-tai:              598     150
clock-getres-tai:               n/t     34
clock-gettime-boottime:         592     142
clock-getres-boottime:          n/t     34
clock-gettime-realtime:         149     142
clock-getres-realtime:          n/t     34
clock-gettime-realtime-coarse:  86      96
clock-getres-realtime-coarse:   n/t     36
getcpu:                         n/t     n/t
gettimeofday:                   133     110

So there are some minor improvements, two minor regressions, some
significant improvements (factor 3-4), and some dramatic improvements
(where we actually gained VDSO support).
Overall a pretty impressive outcome for an "Odd fixes" architecture,
especially as it should reduce the future maintenance burden.

Cheers,
Andre.
Vincenzo Frascino June 24, 2019, 1:21 p.m. UTC | #5
Hi Thomas,

On 24/06/2019 01:34, Thomas Gleixner wrote:
> Vincenzo,
> 
> On Fri, 21 Jun 2019, Vincenzo Frascino wrote:
>> vDSO (virtual dynamic shared object) is a mechanism that the Linux
>> kernel provides as an alternative to system calls to reduce where
>> possible the costs in terms of cycles.
>> This is possible because certain syscalls like gettimeofday() do
>> not write any data and return one or more values that are stored
>> in the kernel, which makes relatively safe calling them directly
>> as a library function.
>>
>> Even if the mechanism is pretty much standard, every architecture
>> in the last few years ended up implementing their own vDSO library
>> in the architectural code.
> 
> ....
>  
>> This implementation contains the portings to the common library for: arm64,
>> compat mode for arm64, arm, mips, x86_64, x32, compat mode for x86_64 and
>> i386.
> 
> I picked up the core implementation and the ARM64 and x86 conversion. I did
> some refinements in several places, coding style, naming conventions,
> comments and changelogs including subject prefixes. Please double check!
>

I tested your changes and they seem OK
(git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/vdso).

...
> As you can see from the commit dates, this has soaked for some time in a
> WIP branch and I did extensive regression testing. So far so good.
> 
> Thanks a lot for going through several iterations. It's a very much
> appreciated effort!
> 

It has been a lot of fun and I learned many many things about the vDSOs and the
kernel that I did not know before. Thanks to you for your patience and guidance.

> Especially with the upcoming time namespaces this will avoid a lot of
> duplicated and pointlessly different horrors all over the architecture
> space. Any architecture which wants to gain that support needs to convert
> to the generic VDSO first.
> 
> As you have become the dude who knows almost everything about VDSO
> including all the nasty pitfalls, I propose the patch below.
>

Thanks for this, it means a lot to me.

> Thanks,
> 
> 	tglx
> 
> 8<------------
> Subject: MAINTAINERS: Add entry for the generic VDSO library
> From: Thomas Gleixner <tglx@linutronix.de>
> Date: Mon, 24 Jun 2019 02:03:50 +0200
> 
> Asign the following folks in alphabetic order:
> 
>  - Andy for being the VDSO wizard of x86 and in general. He's also the
>    performance monitor of choice and the code in the generic library is
>    heavily influenced by his previous x86 VDSO work.
> 
>  - Thomas for being the dude who has to deal with any form of time(r)
>    nonsense anyway
> 
>  - Vincenzo for being the poor sod who went through all the different
>    architecture implementations in order to unify them. A lot of knowledge
>    gained from VDSO implementation details to the intricacies of taming the
>    build system.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  MAINTAINERS |   12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -6665,6 +6665,18 @@ L:	kvm@vger.kernel.org
>  S:	Supported
>  F:	drivers/uio/uio_pci_generic.c
>  
> +GENERIC VDSO LIBRARY:
> +M:	Andy Lutomirksy <luto@kernel.org>
> +M:	Thomas Gleixner <tglx@linutronix.de>
> +M:	Vincenzo Frascino <vincenzo.frascino@arm.com>
> +L:	linux-kernel@vger.kernel.org
> +T:	git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/vdso
> +S:	Maintained
> +F:	lib/vdso
> +F:	kernel/time/vsyscall.c
> +F:	include/vdso
> +F:	include/asm-generic/vdso/vsyscall.h
> +
>  GENWQE (IBM Generic Workqueue Card)
>  M:	Frank Haverkamp <haver@linux.ibm.com>
>  S:	Supported
>
Thomas Gleixner June 24, 2019, 2:18 p.m. UTC | #6
Vincenzo,

On Mon, 24 Jun 2019, Thomas Gleixner wrote:

> I did not merge the ARM and MIPS parts as they lack any form of
> acknowlegment from their maintainers. Please talk to those folks. If they
> ack/review the changes then I can pick them up and they go into 5.3 or they
> have to go in a later cycle. Nevertheless it was well worth the trouble to
> have those conversions done to confirm that the new common library fits a
> bunch of different architectures.

I talked to Russell King and he suggested to file the ARM parts into his
patch system and he'll pick them up after 5.3-rc1.

   https://www.arm.linux.org.uk/developer/patches/

I paged out how to deal with it, but you'll surely manage :)

Thanks,

	tglx
Russell King - ARM Linux admin June 24, 2019, 2:23 p.m. UTC | #7
On Mon, Jun 24, 2019 at 04:18:28PM +0200, Thomas Gleixner wrote:
> Vincenzo,
> 
> On Mon, 24 Jun 2019, Thomas Gleixner wrote:
> 
> > I did not merge the ARM and MIPS parts as they lack any form of
> > acknowlegment from their maintainers. Please talk to those folks. If they
> > ack/review the changes then I can pick them up and they go into 5.3 or they
> > have to go in a later cycle. Nevertheless it was well worth the trouble to
> > have those conversions done to confirm that the new common library fits a
> > bunch of different architectures.
> 
> I talked to Russell King and he suggested to file the ARM parts into his
> patch system and he'll pick them up after 5.3-rc1.
> 
>    https://www.arm.linux.org.uk/developer/patches/
> 
> I paged out how to deal with it, but you'll surely manage :)

Easy way: ask git to add the "KernelVersion" tag as a header to the
email using --add-header to e.g. git format-patch, and just mail them
to patches@armlinux.org.uk
Catalin Marinas June 24, 2019, 2:49 p.m. UTC | #8
On Mon, Jun 24, 2019 at 03:23:46PM +0100, Russell King wrote:
> On Mon, Jun 24, 2019 at 04:18:28PM +0200, Thomas Gleixner wrote:
> > Vincenzo,
> > 
> > On Mon, 24 Jun 2019, Thomas Gleixner wrote:
> > 
> > > I did not merge the ARM and MIPS parts as they lack any form of
> > > acknowlegment from their maintainers. Please talk to those folks. If they
> > > ack/review the changes then I can pick them up and they go into 5.3 or they
> > > have to go in a later cycle. Nevertheless it was well worth the trouble to
> > > have those conversions done to confirm that the new common library fits a
> > > bunch of different architectures.
> > 
> > I talked to Russell King and he suggested to file the ARM parts into his
> > patch system and he'll pick them up after 5.3-rc1.
> > 
> >    https://www.arm.linux.org.uk/developer/patches/
> > 
> > I paged out how to deal with it, but you'll surely manage :)
> 
> Easy way: ask git to add the "KernelVersion" tag as a header to the
> email using --add-header to e.g. git format-patch, and just mail them
> to patches@armlinux.org.uk

Although I haven't send patches to Russell in a while, I still have a
git alias in my .gitconfig (only works with one patch at a time IIRC,
sending multiple patches may arrive in a different order):

[alias]
	send-rmk-email = !git send-email --add-header=\"KernelVersion: $(git describe --abbrev=0)\" --no-thread --suppress-cc=all --to="patches@arm.linux.org.uk"
Vincenzo Frascino June 24, 2019, 4:20 p.m. UTC | #9
On 24/06/2019 15:49, Catalin Marinas wrote:
> On Mon, Jun 24, 2019 at 03:23:46PM +0100, Russell King wrote:
>> On Mon, Jun 24, 2019 at 04:18:28PM +0200, Thomas Gleixner wrote:
>>> Vincenzo,
>>>
>>> On Mon, 24 Jun 2019, Thomas Gleixner wrote:
>>>
>>>> I did not merge the ARM and MIPS parts as they lack any form of
>>>> acknowlegment from their maintainers. Please talk to those folks. If they
>>>> ack/review the changes then I can pick them up and they go into 5.3 or they
>>>> have to go in a later cycle. Nevertheless it was well worth the trouble to
>>>> have those conversions done to confirm that the new common library fits a
>>>> bunch of different architectures.
>>>
>>> I talked to Russell King and he suggested to file the ARM parts into his
>>> patch system and he'll pick them up after 5.3-rc1.
>>>
>>>    https://www.arm.linux.org.uk/developer/patches/
>>>
>>> I paged out how to deal with it, but you'll surely manage :)
>>
>> Easy way: ask git to add the "KernelVersion" tag as a header to the
>> email using --add-header to e.g. git format-patch, and just mail them
>> to patches@armlinux.org.uk
> 
> Although I haven't send patches to Russell in a while, I still have a
> git alias in my .gitconfig (only works with one patch at a time IIRC,
> sending multiple patches may arrive in a different order):
> 
> [alias]
> 	send-rmk-email = !git send-email --add-header=\"KernelVersion: $(git describe --abbrev=0)\" --no-thread --suppress-cc=all --to="patches@arm.linux.org.uk"
> 

Thanks to all for the hints and the support. I will send the patches to Russel
as agreed.
Paul Burton June 24, 2019, 6:41 p.m. UTC | #10
Hello,

On Mon, Jun 24, 2019 at 02:34:24AM +0200, Thomas Gleixner wrote:
> I did not merge the ARM and MIPS parts as they lack any form of
> acknowlegment from their maintainers. Please talk to those folks. If they
> ack/review the changes then I can pick them up and they go into 5.3 or they
> have to go in a later cycle. Nevertheless it was well worth the trouble to
> have those conversions done to confirm that the new common library fits a
> bunch of different architectures.

Apologies for not being more proactive on the MIPS front here; life &
work are extra busy at the moment... But thanks Vincenzo for including
MIPS in the work here.

Unfortunately after applying the 3 MIPS patches (19-21) atop the current
tip.git timers/vdso branch at ecf9db3d1f1a ("x86/vdso: Give the
[ph]vclock_page declarations real types") I see build failures for the
o32 compat VDSO, shown below. This is using the gcc 8.1.0 mips-linux
toolchain from here:

  https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.1.0/x86_64-gcc-8.1.0-nolibc-mips-linux.tar.xz

Configuration is 64r6el_defconfig. The following helps remove the
implicit declaration warnings (and eww to including C files via CFLAGS),
but it still doesn't build:

  diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
  index 95df49402a53..aa38049bdb24 100644
  --- a/arch/mips/vdso/Makefile
  +++ b/arch/mips/vdso/Makefile
  @@ -36,6 +36,8 @@ aflags-vdso := $(ccflags-vdso) \
  
   ifneq ($(c-gettimeofday-y),)
   CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
  +CFLAGS_vgettimeofday-o32.o = -include $(c-gettimeofday-y)
  +CFLAGS_vgettimeofday-n32.o = -include $(c-gettimeofday-y)
   endif

 CFLAGS_REMOVE_vgettimeofday.o = -pg

So the MIPS bits here need more work.

Thanks,
    Paul

  CC      arch/mips/vdso/vgettimeofday-o32.o
In file included from ./include/linux/bitops.h:19,
                 from ./include/linux/kernel.h:12,
                 from ./include/linux/list.h:9,
                 from ./include/linux/preempt.h:11,
                 from ./include/linux/spinlock.h:51,
                 from ./include/linux/seqlock.h:36,
                 from ./include/linux/time.h:6,
                 from arch/mips/vdso/vgettimeofday.c:10:
./arch/mips/include/asm/bitops.h: In function '__fls':
./arch/mips/include/asm/bitops.h:518:21: warning: left shift count >= width of type [-Wshift-count-overflow]
  if (!(word & (~0ul << 32))) {
                     ^~
./arch/mips/include/asm/bitops.h:520:8: warning: left shift count >= width of type [-Wshift-count-overflow]
   word <<= 32;
        ^~~
./arch/mips/include/asm/bitops.h:523:21: warning: left shift count >= width of type [-Wshift-count-overflow]
  if (!(word & (~0ul << (BITS_PER_LONG-16)))) {
                     ^~
./arch/mips/include/asm/bitops.h:527:21: warning: left shift count >= width of type [-Wshift-count-overflow]
  if (!(word & (~0ul << (BITS_PER_LONG-8)))) {
                     ^~
./arch/mips/include/asm/bitops.h:531:21: warning: left shift count >= width of type [-Wshift-count-overflow]
  if (!(word & (~0ul << (BITS_PER_LONG-4)))) {
                     ^~
./arch/mips/include/asm/bitops.h:535:21: warning: left shift count >= width of type [-Wshift-count-overflow]
  if (!(word & (~0ul << (BITS_PER_LONG-2)))) {
                     ^~
./arch/mips/include/asm/bitops.h:539:21: warning: left shift count >= width of type [-Wshift-count-overflow]
  if (!(word & (~0ul << (BITS_PER_LONG-1))))
                     ^~
In file included from ./arch/mips/include/asm/mmiowb.h:5,
                 from ./include/linux/spinlock.h:60,
                 from ./include/linux/seqlock.h:36,
                 from ./include/linux/time.h:6,
                 from arch/mips/vdso/vgettimeofday.c:10:
./arch/mips/include/asm/io.h: In function 'phys_to_virt':
./arch/mips/include/asm/io.h:136:9: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  return (void *)(address + PAGE_OFFSET - PHYS_OFFSET);
         ^
In file included from ./include/linux/bitops.h:5,
                 from ./include/linux/kernel.h:12,
                 from ./include/linux/list.h:9,
                 from ./include/linux/preempt.h:11,
                 from ./include/linux/spinlock.h:51,
                 from ./include/linux/seqlock.h:36,
                 from ./include/linux/time.h:6,
                 from arch/mips/vdso/vgettimeofday.c:10:
./arch/mips/include/asm/mips-cm.h: In function 'mips_cm_max_vp_width':
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:152:28: note: in expansion of macro 'GENMASK'
 #define CM_GCR_REV_MAJOR   GENMASK(15, 8)
                            ^~~~~~~
./arch/mips/include/asm/mips-cm.h:156:22: note: in expansion of macro 'CM_GCR_REV_MAJOR'
   (((major) << __ffs(CM_GCR_REV_MAJOR)) | \
                      ^~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:161:23: note: in expansion of macro 'CM_ENCODE_REV'
 #define CM_REV_CM3    CM_ENCODE_REV(8, 0)
                       ^~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:367:28: note: in expansion of macro 'CM_REV_CM3'
  if (mips_cm_revision() >= CM_REV_CM3)
                            ^~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:153:28: note: in expansion of macro 'GENMASK'
 #define CM_GCR_REV_MINOR   GENMASK(7, 0)
                            ^~~~~~~
./arch/mips/include/asm/mips-cm.h:157:22: note: in expansion of macro 'CM_GCR_REV_MINOR'
    ((minor) << __ffs(CM_GCR_REV_MINOR)))
                      ^~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:161:23: note: in expansion of macro 'CM_ENCODE_REV'
 #define CM_REV_CM3    CM_ENCODE_REV(8, 0)
                       ^~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:367:28: note: in expansion of macro 'CM_REV_CM3'
  if (mips_cm_revision() >= CM_REV_CM3)
                            ^~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:239:36: note: in expansion of macro 'GENMASK'
 #define CM_GCR_SYS_CONFIG2_MAXVPW  GENMASK(3, 0)
                                    ^~~~~~~
./arch/mips/include/asm/mips-cm.h:368:35: note: in expansion of macro 'CM_GCR_SYS_CONFIG2_MAXVPW'
   return read_gcr_sys_config2() & CM_GCR_SYS_CONFIG2_MAXVPW;
                                   ^~~~~~~~~~~~~~~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:293:33: note: in expansion of macro 'GENMASK'
 #define CM_GCR_Cx_CONFIG_PVPE   GENMASK(9, 0)
                                 ^~~~~~~
./arch/mips/include/asm/mips-cm.h:376:32: note: in expansion of macro 'CM_GCR_Cx_CONFIG_PVPE'
   cfg = read_gcr_cl_config() & CM_GCR_Cx_CONFIG_PVPE;
                                ^~~~~~~~~~~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:293:33: note: in expansion of macro 'GENMASK'
 #define CM_GCR_Cx_CONFIG_PVPE   GENMASK(9, 0)
                                 ^~~~~~~
./arch/mips/include/asm/mips-cm.h:377:24: note: in expansion of macro 'CM_GCR_Cx_CONFIG_PVPE'
   return (cfg >> __ffs(CM_GCR_Cx_CONFIG_PVPE)) + 1;
                        ^~~~~~~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_numclusters':
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:152:28: note: in expansion of macro 'GENMASK'
 #define CM_GCR_REV_MAJOR   GENMASK(15, 8)
                            ^~~~~~~
./arch/mips/include/asm/mips-cm.h:156:22: note: in expansion of macro 'CM_GCR_REV_MAJOR'
   (((major) << __ffs(CM_GCR_REV_MAJOR)) | \
                      ^~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
 #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
                         ^~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h:117:27: note: in expansion of macro 'CM_REV_CM3_5'
  if (mips_cm_revision() < CM_REV_CM3_5)
                           ^~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:153:28: note: in expansion of macro 'GENMASK'
 #define CM_GCR_REV_MINOR   GENMASK(7, 0)
                            ^~~~~~~
./arch/mips/include/asm/mips-cm.h:157:22: note: in expansion of macro 'CM_GCR_REV_MINOR'
    ((minor) << __ffs(CM_GCR_REV_MINOR)))
                      ^~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
 #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
                         ^~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h:117:27: note: in expansion of macro 'CM_REV_CM3_5'
  if (mips_cm_revision() < CM_REV_CM3_5)
                           ^~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:133:37: note: in expansion of macro 'GENMASK'
 #define CM_GCR_CONFIG_NUM_CLUSTERS  GENMASK(29, 23)
                                     ^~~~~~~
./arch/mips/include/asm/mips-cps.h:120:37: note: in expansion of macro 'CM_GCR_CONFIG_NUM_CLUSTERS'
  num_clusters = read_gcr_config() & CM_GCR_CONFIG_NUM_CLUSTERS;
                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:133:37: note: in expansion of macro 'GENMASK'
 #define CM_GCR_CONFIG_NUM_CLUSTERS  GENMASK(29, 23)
                                     ^~~~~~~
./arch/mips/include/asm/mips-cps.h:121:25: note: in expansion of macro 'CM_GCR_CONFIG_NUM_CLUSTERS'
  num_clusters >>= __ffs(CM_GCR_CONFIG_NUM_CLUSTERS);
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_cluster_config':
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:152:28: note: in expansion of macro 'GENMASK'
 #define CM_GCR_REV_MAJOR   GENMASK(15, 8)
                            ^~~~~~~
./arch/mips/include/asm/mips-cm.h:156:22: note: in expansion of macro 'CM_GCR_REV_MAJOR'
   (((major) << __ffs(CM_GCR_REV_MAJOR)) | \
                      ^~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
 #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
                         ^~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h:137:27: note: in expansion of macro 'CM_REV_CM3_5'
  if (mips_cm_revision() < CM_REV_CM3_5) {
                           ^~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:153:28: note: in expansion of macro 'GENMASK'
 #define CM_GCR_REV_MINOR   GENMASK(7, 0)
                            ^~~~~~~
./arch/mips/include/asm/mips-cm.h:157:22: note: in expansion of macro 'CM_GCR_REV_MINOR'
    ((minor) << __ffs(CM_GCR_REV_MINOR)))
                      ^~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
 #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
                         ^~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h:137:27: note: in expansion of macro 'CM_REV_CM3_5'
  if (mips_cm_revision() < CM_REV_CM3_5) {
                           ^~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_numcores':
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:135:32: note: in expansion of macro 'GENMASK'
 #define CM_GCR_CONFIG_PCORES   GENMASK(7, 0)
                                ^~~~~~~
./arch/mips/include/asm/mips-cps.h:172:50: note: in expansion of macro 'CM_GCR_CONFIG_PCORES'
  return (mips_cps_cluster_config(cluster) + 1) & CM_GCR_CONFIG_PCORES;
                                                  ^~~~~~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_numiocu':
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:134:33: note: in expansion of macro 'GENMASK'
 #define CM_GCR_CONFIG_NUMIOCU   GENMASK(15, 8)
                                 ^~~~~~~
./arch/mips/include/asm/mips-cps.h:189:48: note: in expansion of macro 'CM_GCR_CONFIG_NUMIOCU'
  num_iocu = mips_cps_cluster_config(cluster) & CM_GCR_CONFIG_NUMIOCU;
                                                ^~~~~~~~~~~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:134:33: note: in expansion of macro 'GENMASK'
 #define CM_GCR_CONFIG_NUMIOCU   GENMASK(15, 8)
                                 ^~~~~~~
./arch/mips/include/asm/mips-cps.h:190:21: note: in expansion of macro 'CM_GCR_CONFIG_NUMIOCU'
  num_iocu >>= __ffs(CM_GCR_CONFIG_NUMIOCU);
                     ^~~~~~~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_numvps':
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:152:28: note: in expansion of macro 'GENMASK'
 #define CM_GCR_REV_MAJOR   GENMASK(15, 8)
                            ^~~~~~~
./arch/mips/include/asm/mips-cm.h:156:22: note: in expansion of macro 'CM_GCR_REV_MAJOR'
   (((major) << __ffs(CM_GCR_REV_MAJOR)) | \
                      ^~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
 #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
                         ^~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h:216:27: note: in expansion of macro 'CM_REV_CM3_5'
  if (mips_cm_revision() < CM_REV_CM3_5) {
                           ^~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:153:28: note: in expansion of macro 'GENMASK'
 #define CM_GCR_REV_MINOR   GENMASK(7, 0)
                            ^~~~~~~
./arch/mips/include/asm/mips-cm.h:157:22: note: in expansion of macro 'CM_GCR_REV_MINOR'
    ((minor) << __ffs(CM_GCR_REV_MINOR)))
                      ^~~~~~~~~~~~~~~~
./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
 #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
                         ^~~~~~~~~~~~~
./arch/mips/include/asm/mips-cps.h:216:27: note: in expansion of macro 'CM_REV_CM3_5'
  if (mips_cm_revision() < CM_REV_CM3_5) {
                           ^~~~~~~~~~~~
./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
  (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
                                       ^~
./arch/mips/include/asm/mips-cm.h:293:33: note: in expansion of macro 'GENMASK'
 #define CM_GCR_Cx_CONFIG_PVPE   GENMASK(9, 0)
                                 ^~~~~~~
./arch/mips/include/asm/mips-cps.h:233:21: note: in expansion of macro 'CM_GCR_Cx_CONFIG_PVPE'
  return (cfg + 1) & CM_GCR_Cx_CONFIG_PVPE;
                     ^~~~~~~~~~~~~~~~~~~~~
arch/mips/vdso/vgettimeofday.c: In function '__vdso_clock_gettime':
arch/mips/vdso/vgettimeofday.c:17:9: error: implicit declaration of function '__cvdso_clock_gettime32'; did you mean '__vdso_clock_gettime'? [-Werror=implicit-function-declaration]
  return __cvdso_clock_gettime32(clock, ts);
         ^~~~~~~~~~~~~~~~~~~~~~~
         __vdso_clock_gettime
arch/mips/vdso/vgettimeofday.c: In function '__vdso_gettimeofday':
arch/mips/vdso/vgettimeofday.c:23:9: error: implicit declaration of function '__cvdso_gettimeofday'; did you mean '__vdso_gettimeofday'? [-Werror=implicit-function-declaration]
  return __cvdso_gettimeofday(tv, tz);
         ^~~~~~~~~~~~~~~~~~~~
         __vdso_gettimeofday
arch/mips/vdso/vgettimeofday.c: In function '__vdso_clock_getres':
arch/mips/vdso/vgettimeofday.c:29:9: error: implicit declaration of function '__cvdso_clock_getres_time32'; did you mean '__vdso_clock_gettime'? [-Werror=implicit-function-declaration]
  return __cvdso_clock_getres_time32(clock_id, res);
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
         __vdso_clock_gettime
arch/mips/vdso/vgettimeofday.c: In function '__vdso_clock_gettime64':
arch/mips/vdso/vgettimeofday.c:35:9: error: implicit declaration of function '__cvdso_clock_gettime'; did you mean '__vdso_clock_gettime'? [-Werror=implicit-function-declaration]
  return __cvdso_clock_gettime(clock, ts);
         ^~~~~~~~~~~~~~~~~~~~~
         __vdso_clock_gettime
cc1: some warnings being treated as errors
make[1]: *** [arch/mips/vdso/Makefile:148: arch/mips/vdso/vgettimeofday-o32.o] Error 1
make: *** [Makefile:1746: arch/mips/vdso/vgettimeofday-o32.o] Error 2
Vincenzo Frascino June 24, 2019, 11:16 p.m. UTC | #11
Hi Paul,

thank you for your review.

On 6/24/19 7:41 PM, Paul Burton wrote:
> Hello,
> 
> On Mon, Jun 24, 2019 at 02:34:24AM +0200, Thomas Gleixner wrote:
>> I did not merge the ARM and MIPS parts as they lack any form of
>> acknowlegment from their maintainers. Please talk to those folks. If they
>> ack/review the changes then I can pick them up and they go into 5.3 or they
>> have to go in a later cycle. Nevertheless it was well worth the trouble to
>> have those conversions done to confirm that the new common library fits a
>> bunch of different architectures.
> 
> Apologies for not being more proactive on the MIPS front here; life &
> work are extra busy at the moment... But thanks Vincenzo for including
> MIPS in the work here.
> 

No problem.

> Unfortunately after applying the 3 MIPS patches (19-21) atop the current
> tip.git timers/vdso branch at ecf9db3d1f1a ("x86/vdso: Give the
> [ph]vclock_page declarations real types") I see build failures for the
> o32 compat VDSO, shown below. This is using the gcc 8.1.0 mips-linux
> toolchain from here:
> 
>   https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.1.0/x86_64-gcc-8.1.0-nolibc-mips-linux.tar.xz
> 
> Configuration is 64r6el_defconfig. The following helps remove the
> implicit declaration warnings (and eww to including C files via CFLAGS),
> but it still doesn't build:
> 
>   diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
>   index 95df49402a53..aa38049bdb24 100644
>   --- a/arch/mips/vdso/Makefile
>   +++ b/arch/mips/vdso/Makefile
>   @@ -36,6 +36,8 @@ aflags-vdso := $(ccflags-vdso) \
>   
>    ifneq ($(c-gettimeofday-y),)
>    CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
>   +CFLAGS_vgettimeofday-o32.o = -include $(c-gettimeofday-y)
>   +CFLAGS_vgettimeofday-n32.o = -include $(c-gettimeofday-y)
>    endif
> 
>  CFLAGS_REMOVE_vgettimeofday.o = -pg
> 
> So the MIPS bits here need more work.
> 

I admit, the one proposed was a nice challenge and it took me a while to
understand the differences in between the O32, N32 and N64 binaries and what was
causing the reported issue.

In the end I concluded that all the errors seen here depend on the fact that I
tested my vdso implementation on MIPS32el only (as stated in the cover letter)
and that when I tried to compile a 32BIT binary on a 64BIT configuration I did
it wrongly for two reasons, for N32 and O32 binaries:
 - we need to undefine CONFIG_64BIT and define CONFIG_32BIT
 - we need to define CONFIG_GENERIC_ATOMIC64


I have a fix for this (patch in attachment), but I do not have the hardware to
test it. If you could provide some feedback would be appreciated (really want to
see MIPS merged with the other archs in 5.3 :) ).

> Thanks,
>     Paul
> 
>   CC      arch/mips/vdso/vgettimeofday-o32.o
> In file included from ./include/linux/bitops.h:19,
>                  from ./include/linux/kernel.h:12,
>                  from ./include/linux/list.h:9,
>                  from ./include/linux/preempt.h:11,
>                  from ./include/linux/spinlock.h:51,
>                  from ./include/linux/seqlock.h:36,
>                  from ./include/linux/time.h:6,
>                  from arch/mips/vdso/vgettimeofday.c:10:
> ./arch/mips/include/asm/bitops.h: In function '__fls':
> ./arch/mips/include/asm/bitops.h:518:21: warning: left shift count >= width of type [-Wshift-count-overflow]
>   if (!(word & (~0ul << 32))) {
>                      ^~
> ./arch/mips/include/asm/bitops.h:520:8: warning: left shift count >= width of type [-Wshift-count-overflow]
>    word <<= 32;
>         ^~~
> ./arch/mips/include/asm/bitops.h:523:21: warning: left shift count >= width of type [-Wshift-count-overflow]
>   if (!(word & (~0ul << (BITS_PER_LONG-16)))) {
>                      ^~
> ./arch/mips/include/asm/bitops.h:527:21: warning: left shift count >= width of type [-Wshift-count-overflow]
>   if (!(word & (~0ul << (BITS_PER_LONG-8)))) {
>                      ^~
> ./arch/mips/include/asm/bitops.h:531:21: warning: left shift count >= width of type [-Wshift-count-overflow]
>   if (!(word & (~0ul << (BITS_PER_LONG-4)))) {
>                      ^~
> ./arch/mips/include/asm/bitops.h:535:21: warning: left shift count >= width of type [-Wshift-count-overflow]
>   if (!(word & (~0ul << (BITS_PER_LONG-2)))) {
>                      ^~
> ./arch/mips/include/asm/bitops.h:539:21: warning: left shift count >= width of type [-Wshift-count-overflow]
>   if (!(word & (~0ul << (BITS_PER_LONG-1))))
>                      ^~
> In file included from ./arch/mips/include/asm/mmiowb.h:5,
>                  from ./include/linux/spinlock.h:60,
>                  from ./include/linux/seqlock.h:36,
>                  from ./include/linux/time.h:6,
>                  from arch/mips/vdso/vgettimeofday.c:10:
> ./arch/mips/include/asm/io.h: In function 'phys_to_virt':
> ./arch/mips/include/asm/io.h:136:9: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
>   return (void *)(address + PAGE_OFFSET - PHYS_OFFSET);
>          ^
> In file included from ./include/linux/bitops.h:5,
>                  from ./include/linux/kernel.h:12,
>                  from ./include/linux/list.h:9,
>                  from ./include/linux/preempt.h:11,
>                  from ./include/linux/spinlock.h:51,
>                  from ./include/linux/seqlock.h:36,
>                  from ./include/linux/time.h:6,
>                  from arch/mips/vdso/vgettimeofday.c:10:
> ./arch/mips/include/asm/mips-cm.h: In function 'mips_cm_max_vp_width':
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:152:28: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_REV_MAJOR   GENMASK(15, 8)
>                             ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:156:22: note: in expansion of macro 'CM_GCR_REV_MAJOR'
>    (((major) << __ffs(CM_GCR_REV_MAJOR)) | \
>                       ^~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:161:23: note: in expansion of macro 'CM_ENCODE_REV'
>  #define CM_REV_CM3    CM_ENCODE_REV(8, 0)
>                        ^~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:367:28: note: in expansion of macro 'CM_REV_CM3'
>   if (mips_cm_revision() >= CM_REV_CM3)
>                             ^~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:153:28: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_REV_MINOR   GENMASK(7, 0)
>                             ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:157:22: note: in expansion of macro 'CM_GCR_REV_MINOR'
>     ((minor) << __ffs(CM_GCR_REV_MINOR)))
>                       ^~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:161:23: note: in expansion of macro 'CM_ENCODE_REV'
>  #define CM_REV_CM3    CM_ENCODE_REV(8, 0)
>                        ^~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:367:28: note: in expansion of macro 'CM_REV_CM3'
>   if (mips_cm_revision() >= CM_REV_CM3)
>                             ^~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:239:36: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_SYS_CONFIG2_MAXVPW  GENMASK(3, 0)
>                                     ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:368:35: note: in expansion of macro 'CM_GCR_SYS_CONFIG2_MAXVPW'
>    return read_gcr_sys_config2() & CM_GCR_SYS_CONFIG2_MAXVPW;
>                                    ^~~~~~~~~~~~~~~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:293:33: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_Cx_CONFIG_PVPE   GENMASK(9, 0)
>                                  ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:376:32: note: in expansion of macro 'CM_GCR_Cx_CONFIG_PVPE'
>    cfg = read_gcr_cl_config() & CM_GCR_Cx_CONFIG_PVPE;
>                                 ^~~~~~~~~~~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:293:33: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_Cx_CONFIG_PVPE   GENMASK(9, 0)
>                                  ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:377:24: note: in expansion of macro 'CM_GCR_Cx_CONFIG_PVPE'
>    return (cfg >> __ffs(CM_GCR_Cx_CONFIG_PVPE)) + 1;
>                         ^~~~~~~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_numclusters':
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:152:28: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_REV_MAJOR   GENMASK(15, 8)
>                             ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:156:22: note: in expansion of macro 'CM_GCR_REV_MAJOR'
>    (((major) << __ffs(CM_GCR_REV_MAJOR)) | \
>                       ^~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
>  #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
>                          ^~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h:117:27: note: in expansion of macro 'CM_REV_CM3_5'
>   if (mips_cm_revision() < CM_REV_CM3_5)
>                            ^~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:153:28: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_REV_MINOR   GENMASK(7, 0)
>                             ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:157:22: note: in expansion of macro 'CM_GCR_REV_MINOR'
>     ((minor) << __ffs(CM_GCR_REV_MINOR)))
>                       ^~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
>  #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
>                          ^~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h:117:27: note: in expansion of macro 'CM_REV_CM3_5'
>   if (mips_cm_revision() < CM_REV_CM3_5)
>                            ^~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:133:37: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_CONFIG_NUM_CLUSTERS  GENMASK(29, 23)
>                                      ^~~~~~~
> ./arch/mips/include/asm/mips-cps.h:120:37: note: in expansion of macro 'CM_GCR_CONFIG_NUM_CLUSTERS'
>   num_clusters = read_gcr_config() & CM_GCR_CONFIG_NUM_CLUSTERS;
>                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:133:37: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_CONFIG_NUM_CLUSTERS  GENMASK(29, 23)
>                                      ^~~~~~~
> ./arch/mips/include/asm/mips-cps.h:121:25: note: in expansion of macro 'CM_GCR_CONFIG_NUM_CLUSTERS'
>   num_clusters >>= __ffs(CM_GCR_CONFIG_NUM_CLUSTERS);
>                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_cluster_config':
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:152:28: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_REV_MAJOR   GENMASK(15, 8)
>                             ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:156:22: note: in expansion of macro 'CM_GCR_REV_MAJOR'
>    (((major) << __ffs(CM_GCR_REV_MAJOR)) | \
>                       ^~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
>  #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
>                          ^~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h:137:27: note: in expansion of macro 'CM_REV_CM3_5'
>   if (mips_cm_revision() < CM_REV_CM3_5) {
>                            ^~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:153:28: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_REV_MINOR   GENMASK(7, 0)
>                             ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:157:22: note: in expansion of macro 'CM_GCR_REV_MINOR'
>     ((minor) << __ffs(CM_GCR_REV_MINOR)))
>                       ^~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
>  #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
>                          ^~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h:137:27: note: in expansion of macro 'CM_REV_CM3_5'
>   if (mips_cm_revision() < CM_REV_CM3_5) {
>                            ^~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_numcores':
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:135:32: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_CONFIG_PCORES   GENMASK(7, 0)
>                                 ^~~~~~~
> ./arch/mips/include/asm/mips-cps.h:172:50: note: in expansion of macro 'CM_GCR_CONFIG_PCORES'
>   return (mips_cps_cluster_config(cluster) + 1) & CM_GCR_CONFIG_PCORES;
>                                                   ^~~~~~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_numiocu':
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:134:33: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_CONFIG_NUMIOCU   GENMASK(15, 8)
>                                  ^~~~~~~
> ./arch/mips/include/asm/mips-cps.h:189:48: note: in expansion of macro 'CM_GCR_CONFIG_NUMIOCU'
>   num_iocu = mips_cps_cluster_config(cluster) & CM_GCR_CONFIG_NUMIOCU;
>                                                 ^~~~~~~~~~~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:134:33: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_CONFIG_NUMIOCU   GENMASK(15, 8)
>                                  ^~~~~~~
> ./arch/mips/include/asm/mips-cps.h:190:21: note: in expansion of macro 'CM_GCR_CONFIG_NUMIOCU'
>   num_iocu >>= __ffs(CM_GCR_CONFIG_NUMIOCU);
>                      ^~~~~~~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h: In function 'mips_cps_numvps':
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:152:28: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_REV_MAJOR   GENMASK(15, 8)
>                             ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:156:22: note: in expansion of macro 'CM_GCR_REV_MAJOR'
>    (((major) << __ffs(CM_GCR_REV_MAJOR)) | \
>                       ^~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
>  #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
>                          ^~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h:216:27: note: in expansion of macro 'CM_REV_CM3_5'
>   if (mips_cm_revision() < CM_REV_CM3_5) {
>                            ^~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:153:28: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_REV_MINOR   GENMASK(7, 0)
>                             ^~~~~~~
> ./arch/mips/include/asm/mips-cm.h:157:22: note: in expansion of macro 'CM_GCR_REV_MINOR'
>     ((minor) << __ffs(CM_GCR_REV_MINOR)))
>                       ^~~~~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cm.h:162:25: note: in expansion of macro 'CM_ENCODE_REV'
>  #define CM_REV_CM3_5    CM_ENCODE_REV(9, 0)
>                          ^~~~~~~~~~~~~
> ./arch/mips/include/asm/mips-cps.h:216:27: note: in expansion of macro 'CM_REV_CM3_5'
>   if (mips_cm_revision() < CM_REV_CM3_5) {
>                            ^~~~~~~~~~~~
> ./include/linux/bits.h:20:39: warning: right shift count >= width of type [-Wshift-count-overflow]
>   (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>                                        ^~
> ./arch/mips/include/asm/mips-cm.h:293:33: note: in expansion of macro 'GENMASK'
>  #define CM_GCR_Cx_CONFIG_PVPE   GENMASK(9, 0)
>                                  ^~~~~~~
> ./arch/mips/include/asm/mips-cps.h:233:21: note: in expansion of macro 'CM_GCR_Cx_CONFIG_PVPE'
>   return (cfg + 1) & CM_GCR_Cx_CONFIG_PVPE;
>                      ^~~~~~~~~~~~~~~~~~~~~
> arch/mips/vdso/vgettimeofday.c: In function '__vdso_clock_gettime':
> arch/mips/vdso/vgettimeofday.c:17:9: error: implicit declaration of function '__cvdso_clock_gettime32'; did you mean '__vdso_clock_gettime'? [-Werror=implicit-function-declaration]
>   return __cvdso_clock_gettime32(clock, ts);
>          ^~~~~~~~~~~~~~~~~~~~~~~
>          __vdso_clock_gettime
> arch/mips/vdso/vgettimeofday.c: In function '__vdso_gettimeofday':
> arch/mips/vdso/vgettimeofday.c:23:9: error: implicit declaration of function '__cvdso_gettimeofday'; did you mean '__vdso_gettimeofday'? [-Werror=implicit-function-declaration]
>   return __cvdso_gettimeofday(tv, tz);
>          ^~~~~~~~~~~~~~~~~~~~
>          __vdso_gettimeofday
> arch/mips/vdso/vgettimeofday.c: In function '__vdso_clock_getres':
> arch/mips/vdso/vgettimeofday.c:29:9: error: implicit declaration of function '__cvdso_clock_getres_time32'; did you mean '__vdso_clock_gettime'? [-Werror=implicit-function-declaration]
>   return __cvdso_clock_getres_time32(clock_id, res);
>          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
>          __vdso_clock_gettime
> arch/mips/vdso/vgettimeofday.c: In function '__vdso_clock_gettime64':
> arch/mips/vdso/vgettimeofday.c:35:9: error: implicit declaration of function '__cvdso_clock_gettime'; did you mean '__vdso_clock_gettime'? [-Werror=implicit-function-declaration]
>   return __cvdso_clock_gettime(clock, ts);
>          ^~~~~~~~~~~~~~~~~~~~~
>          __vdso_clock_gettime
> cc1: some warnings being treated as errors
> make[1]: *** [arch/mips/vdso/Makefile:148: arch/mips/vdso/vgettimeofday-o32.o] Error 1
> make: *** [Makefile:1746: arch/mips/vdso/vgettimeofday-o32.o] Error 2
>
Paul Burton June 25, 2019, 5:11 p.m. UTC | #12
Hi Vincenzo,

On Tue, Jun 25, 2019 at 12:16:55AM +0100, Vincenzo Frascino wrote:
> In the end I concluded that all the errors seen here depend on the fact that I
> tested my vdso implementation on MIPS32el only (as stated in the cover letter)
> and that when I tried to compile a 32BIT binary on a 64BIT configuration I did
> it wrongly for two reasons, for N32 and O32 binaries:
>  - we need to undefine CONFIG_64BIT and define CONFIG_32BIT
>  - we need to define CONFIG_GENERIC_ATOMIC64
> 
> I have a fix for this (patch in attachment), but I do not have the hardware to
> test it. If you could provide some feedback would be appreciated (really want to
> see MIPS merged with the other archs in 5.3 :) ).

Thanks for the quick turnaround on your patch!

I'm certainly willing to test it, but in a few hours I'll be spending
the bulk of a day on airplanes[1] so it might take a few days until I
get to it.

Thanks,
    Paul

[1] ...and travel isn't the hackathon it used to be with my 9 month old
    son around :)
Vincenzo Frascino June 25, 2019, 5:17 p.m. UTC | #13
Hi Paul,

On 25/06/2019 18:11, Paul Burton wrote:
> Hi Vincenzo,
> 
> On Tue, Jun 25, 2019 at 12:16:55AM +0100, Vincenzo Frascino wrote:
>> In the end I concluded that all the errors seen here depend on the fact that I
>> tested my vdso implementation on MIPS32el only (as stated in the cover letter)
>> and that when I tried to compile a 32BIT binary on a 64BIT configuration I did
>> it wrongly for two reasons, for N32 and O32 binaries:
>>  - we need to undefine CONFIG_64BIT and define CONFIG_32BIT
>>  - we need to define CONFIG_GENERIC_ATOMIC64
>>
>> I have a fix for this (patch in attachment), but I do not have the hardware to
>> test it. If you could provide some feedback would be appreciated (really want to
>> see MIPS merged with the other archs in 5.3 :) ).
> 
> Thanks for the quick turnaround on your patch!
> 
> I'm certainly willing to test it, but in a few hours I'll be spending
> the bulk of a day on airplanes[1] so it might take a few days until I
> get to it.
> 

Sounds like a plan. Let us know when you have an update.

> Thanks,
>     Paul
> 
> [1] ...and travel isn't the hackathon it used to be with my 9 month old
>     son around :)
>
Joe Perches June 26, 2019, 3:41 p.m. UTC | #14
On Wed, 2019-06-26 at 05:36 -0700, tip-bot for Thomas Gleixner wrote:
> MAINTAINERS: Add entry for the generic VDSO library
[]
> diff --git a/MAINTAINERS b/MAINTAINERS
[]
> @@ -6664,6 +6664,18 @@ L:	kvm@vger.kernel.org
>  S:	Supported
>  F:	drivers/uio/uio_pci_generic.c
>  
> +GENERIC VDSO LIBRARY:
> +M:	Andy Lutomirksi <luto@kernel.org>
> +M:	Thomas Gleixner <tglx@linutronix.de>
> +M:	Vincenzo Frascino <vincenzo.frascino@arm.com>
> +L:	linux-kernel@vger.kernel.org
> +T:	git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/vdso
> +S:	Maintained
> +F:	lib/vdso

Please use trailing slashes for directories.

> +F:	kernel/time/vsyscall.c
> +F:	include/vdso

and here.

> +F:	include/asm-generic/vdso/vsyscall.h
> +
>  GENWQE (IBM Generic Workqueue Card)
>  M:	Frank Haverkamp <haver@linux.ibm.com>
>  S:	Supported
Andy Lutomirski June 26, 2019, 4:31 p.m. UTC | #15
On Wed, Jun 26, 2019 at 5:37 AM tip-bot for Thomas Gleixner
<tipbot@zytor.com> wrote:
>
> Commit-ID:  e70980312a946a56173843cbc0104b3b0e57a0c7
> Gitweb:     https://git.kernel.org/tip/e70980312a946a56173843cbc0104b3b0e57a0c7
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Mon, 24 Jun 2019 02:34:24 +0200
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Wed, 26 Jun 2019 07:28:11 +0200
>
> MAINTAINERS: Add entry for the generic VDSO library
>
> Assign the following folks in alphabetic order:
>
>  - Andy for being the VDSO wizard of x86 and in general. He's also the
>    performance monitor of choice and the code in the generic library is
>    heavily influenced by his previous x86 VDSO work.
>
>  - Thomas for being the dude who has to deal with any form of time(r)
>    nonsense anyway
>
>  - Vincenzo for being the poor sod who went through all the different
>    architecture implementations in order to unify them. A lot of knowledge
>    gained from VDSO implementation details to the intricacies of taming the
>    build system.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Cc: linux-arch@vger.kernel.org
> Cc: LAK <linux-arm-kernel@lists.infradead.org>
> Cc: linux-mips@vger.kernel.org
> Cc: linux-kselftest@vger.kernel.org
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Ralf Baechle <ralf@linux-mips.org>
> Cc: Paul Burton <paul.burton@mips.com>
> Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
> Cc: Mark Salyzyn <salyzyn@android.com>
> Cc: Peter Collingbourne <pcc@google.com>
> Cc: Shuah Khan <shuah@kernel.org>
> Cc: Dmitry Safonov <0x7f454c46@gmail.com>
> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
> Cc: Huw Davies <huw@codeweavers.com>
> Cc: Shijith Thotton <sthotton@marvell.com>
> Cc: Andre Przywara <andre.przywara@arm.com>
> Cc: Dmitry Safonov <dima@arista.com>
> Cc: Andrei Vagin <avagin@openvz.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Michael Kelley <mikelley@microsoft.com>
> Cc: Sasha Levin <sashal@kernel.org>
> Link: https://lkml.kernel.org/r/alpine.DEB.2.21.1906240142000.32342@nanos.tec.linutronix.de
>
> ---
>  MAINTAINERS | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index d0ed735994a5..13ece5479167 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -6664,6 +6664,18 @@ L:       kvm@vger.kernel.org
>  S:     Supported
>  F:     drivers/uio/uio_pci_generic.c
>
> +GENERIC VDSO LIBRARY:
> +M:     Andy Lutomirksi <luto@kernel.org>

s/ksi/ski :)

Yes, it's a mouthful.

--Andy
Thomas Gleixner June 26, 2019, 4:38 p.m. UTC | #16
On Wed, 26 Jun 2019, Andy Lutomirski wrote:
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index d0ed735994a5..13ece5479167 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -6664,6 +6664,18 @@ L:       kvm@vger.kernel.org
> >  S:     Supported
> >  F:     drivers/uio/uio_pci_generic.c
> >
> > +GENERIC VDSO LIBRARY:
> > +M:     Andy Lutomirksi <luto@kernel.org>
> 
> s/ksi/ski :)
> 
> Yes, it's a mouthful.

/me blushes some more