mbox series

[GIT,PULL] nolibc changes for 6.6-rc1

Message ID 20230806172245.GA26239@1wt.eu (mailing list archive)
State New
Headers show
Series [GIT,PULL] nolibc changes for 6.6-rc1 | expand

Pull-request

https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1

Message

Willy Tarreau Aug. 6, 2023, 5:22 p.m. UTC
Hi Shuah, hi Paul,

I'm sending you the list of planned nolibc changes for 6.6. A doc update
may possibly follow a bit later to try to document the contribution
process. We also noticed a slight increase in binary sizes that might
be fixed soon but I wouldn't bet on this since it will require lot of
testing again and I'd rather postpone this by default. In any case I
have no intent to push any significant updates/fixes for 6.6 at this
point.

I'm also pasting a summary of the changes in this pull request, feel
free to use it for the merge commit message if you need.

For any question or if anything is not clear, do not hesitate to ask!

Thanks,
Willy

----- changes ------
Nolibc:
  - improved portability by removing build errors with -ENOSYS
  - added syscall6() on MIPS to support pselect6() and mmap()
  - added setvbuf(), rmdir(), pipe(), pipe2()
  - add support for ppc/ppc64
  - environ is no longer optional
  - fixed frame pointer issues at -O0
  - dropped sys_stat() in favor of sys_statx()
  - centralized _start_c() to remove lots of asm code
  - switched size_t to __SIZE_TYPE__

Selftests:
  - improved status reporting (success/warning/failure counts, path to log file)
  - various code cleanups (indent, unused variables, ...)
  - more consistent test numbering
  - enabled compiler warnings
  - dropped unreliable chmod_net test
  - improved reliability (create /dev/zero & /tmp, rely less on /proc)
  - new tests (brk/sbrk/mmap/munmap)
  - improved compatibility with musl
  - new run-nolibc-test target to build and run natively
  - new run-libc-test target to build and run against native libc
  - made the cmdline parser more reliable against boolean arguments
  - dropped dependency on memfd for vfprintf() test
  - nolibc-test is no longer stripped
  - added support for extending ARCH via XARCH

Other:
  - add Thomas as co-maintainer
-----------

The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:

  Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)

are available in the Git repository at:

  https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1

for you to fetch changes up to d98c1e27e46e47a3ae67e1d048f153598ba82611:

  tools/nolibc: stackprotector.h: make __stack_chk_init static (2023-08-06 18:44:47 +0200)

----------------------------------------------------------------
Ryan Roberts (1):
      tools/nolibc/stdio: add setvbuf() to set buffering mode

Thomas Weißschuh (22):
      selftests/nolibc: drop test chmod_net
      selftests/nolibc: simplify call to ioperm
      tools/nolibc: completely remove optional environ support
      selftests/nolibc: make evaluation of test conditions
      selftests/nolibc: simplify status printing
      selftests/nolibc: avoid gaps in test numbers
      selftests/nolibc: avoid buffer underrun in space printing
      tools/nolibc: drop unused variables
      tools/nolibc: fix return type of getpagesize()
      tools/nolibc: setvbuf: avoid unused parameter warnings
      tools/nolibc: sys: avoid implicit sign cast
      tools/nolibc: stdint: use __SIZE_TYPE__ for size_t
      selftests/nolibc: drop unused variables
      selftests/nolibc: mark test helpers as potentially unused
      selftests/nolibc: make functions static if possible
      selftests/nolibc: avoid unused parameter warnings
      selftests/nolibc: avoid sign-compare warnings
      selftests/nolibc: use correct return type for read() and write()
      selftests/nolibc: prevent out of bounds access in expect_vfprintf
      selftests/nolibc: don't strip nolibc-test
      selftests/nolibc: enable compiler warnings
      MAINTAINERS: nolibc: add myself as co-maintainer

Willy Tarreau (1):
      selftests/nolibc: avoid warnings during intptr tests

Yuan Tan (2):
      tools/nolibc: add pipe() and pipe2() support
      selftests/nolibc: add testcase for pipe

Zhangjin Wu (74):
      selftests/nolibc: add a standalone test report macro
      selftests/nolibc: always print the path to test log file
      selftests/nolibc: restore the failed tests print
      tools/nolibc: fix up #error compile failures with -ENOSYS
      tools/nolibc: fix up undeclared syscall macros with #ifdef and -ENOSYS
      tools/nolibc: sys.h: add a syscall return helper
      tools/nolibc: unistd.h: apply __sysret() helper
      tools/nolibc: sys.h: apply __sysret() helper
      tools/nolibc: unistd.h: reorder the syscall macros
      tools/nolibc: arch-*.h: fix up code indent errors
      toolc/nolibc: arch-*.h: clean up whitespaces after __asm__
      tools/nolibc: arch-loongarch.h: shrink with _NOLIBC_SYSCALL_CLOBBERLIST
      tools/nolibc: arch-mips.h: shrink with _NOLIBC_SYSCALL_CLOBBERLIST
      tools/nolibc: add missing my_syscall6() for mips
      tools/nolibc: __sysret: support syscalls who return a pointer
      tools/nolibc: clean up mmap() routine
      tools/nolibc: clean up sbrk() routine
      selftests/nolibc: export argv0 for some tests
      selftests/nolibc: prepare: create /dev/zero
      selftests/nolibc: add EXPECT_PTREQ, EXPECT_PTRNE and EXPECT_PTRER
      selftests/nolibc: add sbrk_0 to test current brk getting
      selftests/nolibc: add mmap_bad test case
      selftests/nolibc: add munmap_bad test case
      selftests/nolibc: add mmap_munmap_good test case
      selftests/nolibc: add run-libc-test target
      selftests/nolibc: stat_fault: silence NULL argument warning with glibc
      selftests/nolibc: gettid: restore for glibc and musl
      selftests/nolibc: add _LARGEFILE64_SOURCE for musl
      selftests/nolibc: fix up int_fast16/32_t test cases for musl
      tools/nolibc: types.h: add RB_ flags for reboot()
      selftests/nolibc: prefer <sys/reboot.h> to <linux/reboot.h>
      selftests/nolibc: fix up kernel parameters support
      selftests/nolibc: link_cross: use /proc/self/cmdline
      tools/nolibc: add rmdir() support
      selftests/nolibc: add a new rmdir() test case
      selftests/nolibc: fix up failures when CONFIG_PROC_FS=n
      selftests/nolibc: prepare /tmp for tests that need to write
      selftests/nolibc: vfprintf: remove MEMFD_CREATE dependency
      selftests/nolibc: chdir_root: restore current path after test
      selftests/nolibc: stat_timestamps: remove procfs dependency
      selftests/nolibc: chroot_exe: remove procfs dependency
      selftests/nolibc: add chmod_argv0 test
      selftests/nolibc: report: print a summarized test status
      selftests/nolibc: report: print total tests
      selftests/nolibc: report: align passed, skipped and failed
      selftests/nolibc: report: extrude the test status line
      selftests/nolibc: report: add newline before test failures
      tools/nolibc: arch-*.h: add missing space after ','
      tools/nolibc: fix up startup failures for -O0 under gcc < 11.1.0
      tools/nolibc: remove the old sys_stat support
      tools/nolibc: add new crt.h with _start_c
      tools/nolibc: stackprotector.h: add empty __stack_chk_init for !_NOLIBC_STACKPROTECTOR
      tools/nolibc: crt.h: initialize stack protector
      tools/nolibc: arm: shrink _start with _start_c
      tools/nolibc: aarch64: shrink _start with _start_c
      tools/nolibc: i386: shrink _start with _start_c
      tools/nolibc: x86_64: shrink _start with _start_c
      tools/nolibc: mips: shrink _start with _start_c
      tools/nolibc: loongarch: shrink _start with _start_c
      tools/nolibc: riscv: shrink _start with _start_c
      tools/nolibc: s390: shrink _start with _start_c
      selftests/nolibc: add EXPECT_PTRGE, EXPECT_PTRGT, EXPECT_PTRLE, EXPECT_PTRLT
      selftests/nolibc: add testcases for startup code
      selftests/nolibc: allow run nolibc-test locally
      selftests/nolibc: allow test -include /path/to/nolibc.h
      selftests/nolibc: mmap_munmap_good: fix up return value
      tools/nolibc: add support for powerpc
      tools/nolibc: add support for powerpc64
      selftests/nolibc: add XARCH and ARCH mapping support
      selftests/nolibc: add test support for ppc
      selftests/nolibc: add test support for ppc64le
      selftests/nolibc: add test support for ppc64
      selftests/nolibc: allow report with existing test log
      tools/nolibc: stackprotector.h: make __stack_chk_init static

 MAINTAINERS                                  |   1 +
 tools/include/nolibc/Makefile                |   1 +
 tools/include/nolibc/arch-aarch64.h          |  85 +---
 tools/include/nolibc/arch-arm.h              | 111 +----
 tools/include/nolibc/arch-i386.h             |  86 +---
 tools/include/nolibc/arch-loongarch.h        |  83 +---
 tools/include/nolibc/arch-mips.h             | 147 +++----
 tools/include/nolibc/arch-powerpc.h          | 213 ++++++++++
 tools/include/nolibc/arch-riscv.h            |  83 +---
 tools/include/nolibc/arch-s390.h             |  77 +---
 tools/include/nolibc/arch-x86_64.h           |  86 +---
 tools/include/nolibc/arch.h                  |   2 +
 tools/include/nolibc/crt.h                   |  61 +++
 tools/include/nolibc/nolibc.h                |   9 +-
 tools/include/nolibc/stackprotector.h        |   5 +-
 tools/include/nolibc/stdint.h                |   2 +-
 tools/include/nolibc/stdio.h                 |  27 ++
 tools/include/nolibc/stdlib.h                |  12 +-
 tools/include/nolibc/sys.h                   | 554 +++++++-----------------
 tools/include/nolibc/types.h                 |  22 +-
 tools/include/nolibc/unistd.h                |  13 +-
 tools/testing/selftests/nolibc/Makefile      | 109 +++--
 tools/testing/selftests/nolibc/nolibc-test.c | 609 ++++++++++++++++++++-------
 23 files changed, 1216 insertions(+), 1182 deletions(-)
 create mode 100644 tools/include/nolibc/arch-powerpc.h
 create mode 100644 tools/include/nolibc/crt.h

Comments

Shuah Khan Aug. 15, 2023, 2:25 p.m. UTC | #1
On 8/6/23 11:22, Willy Tarreau wrote:
> Hi Shuah, hi Paul,
> 
> I'm sending you the list of planned nolibc changes for 6.6. A doc update
> may possibly follow a bit later to try to document the contribution
> process. We also noticed a slight increase in binary sizes that might
> be fixed soon but I wouldn't bet on this since it will require lot of
> testing again and I'd rather postpone this by default. In any case I
> have no intent to push any significant updates/fixes for 6.6 at this
> point.
> 
> I'm also pasting a summary of the changes in this pull request, feel
> free to use it for the merge commit message if you need.
> 
> For any question or if anything is not clear, do not hesitate to ask!
> 
> Thanks,
> Willy
> 
> ----- changes ------
> Nolibc:
>    - improved portability by removing build errors with -ENOSYS
>    - added syscall6() on MIPS to support pselect6() and mmap()
>    - added setvbuf(), rmdir(), pipe(), pipe2()
>    - add support for ppc/ppc64
>    - environ is no longer optional
>    - fixed frame pointer issues at -O0
>    - dropped sys_stat() in favor of sys_statx()
>    - centralized _start_c() to remove lots of asm code
>    - switched size_t to __SIZE_TYPE__
> 
> Selftests:
>    - improved status reporting (success/warning/failure counts, path to log file)
>    - various code cleanups (indent, unused variables, ...)
>    - more consistent test numbering
>    - enabled compiler warnings
>    - dropped unreliable chmod_net test
>    - improved reliability (create /dev/zero & /tmp, rely less on /proc)
>    - new tests (brk/sbrk/mmap/munmap)
>    - improved compatibility with musl
>    - new run-nolibc-test target to build and run natively
>    - new run-libc-test target to build and run against native libc
>    - made the cmdline parser more reliable against boolean arguments
>    - dropped dependency on memfd for vfprintf() test
>    - nolibc-test is no longer stripped
>    - added support for extending ARCH via XARCH
> 
> Other:
>    - add Thomas as co-maintainer
> -----------
> 
> The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
> 
>    Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
> 
> are available in the Git repository at:
> 
>    https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
> 
> for you to fetch changes up to d98c1e27e46e47a3ae67e1d048f153598ba82611:
> 
>    tools/nolibc: stackprotector.h: make __stack_chk_init static (2023-08-06 18:44:47 +0200)
> 

Hi Willy,

I am sorry for the delay on this. I was traveling last week
and getting back to digging myself out of emails.

I am having trouble pulling this request though:

git request-pull https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1

gives me the following error

fatal: Not a valid revision: git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/

I don't see a tag at https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git

thanks,
-- Shuah
Willy Tarreau Aug. 15, 2023, 2:35 p.m. UTC | #2
On Tue, Aug 15, 2023 at 08:25:51AM -0600, Shuah Khan wrote:
> > The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
> > 
> >    Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
> > 
> > are available in the Git repository at:
> > 
> >    https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
> > 
> > for you to fetch changes up to d98c1e27e46e47a3ae67e1d048f153598ba82611:
> > 
> >    tools/nolibc: stackprotector.h: make __stack_chk_init static (2023-08-06 18:44:47 +0200)
> > 
> 
> Hi Willy,
> 
> I am sorry for the delay on this. I was traveling last week
> and getting back to digging myself out of emails.

No problem, thanks for getting back :-)

> I am having trouble pulling this request though:
> 
> git request-pull https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
> 
> gives me the following error
> 
> fatal: Not a valid revision: git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/
> 
> I don't see a tag at https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git

Ah sorry for the misunderstanding, that's the branch name, I'll set a
tag then.

Thanks!
willy
Shuah Khan Aug. 15, 2023, 2:39 p.m. UTC | #3
On 8/15/23 08:35, Willy Tarreau wrote:
> On Tue, Aug 15, 2023 at 08:25:51AM -0600, Shuah Khan wrote:
>>> The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
>>>
>>>     Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
>>>
>>> are available in the Git repository at:
>>>
>>>     https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
>>>
>>> for you to fetch changes up to d98c1e27e46e47a3ae67e1d048f153598ba82611:
>>>
>>>     tools/nolibc: stackprotector.h: make __stack_chk_init static (2023-08-06 18:44:47 +0200)
>>>
>>
>> Hi Willy,
>>
>> I am sorry for the delay on this. I was traveling last week
>> and getting back to digging myself out of emails.
> 
> No problem, thanks for getting back :-)
> 
>> I am having trouble pulling this request though:
>>
>> git request-pull https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
>>
>> gives me the following error
>>
>> fatal: Not a valid revision: git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/
>>
>> I don't see a tag at https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
> 
> Ah sorry for the misunderstanding, that's the branch name, I'll set a
> tag then.
> 

No worries. Could you also share the test you run? I will pull
you request and run tests.

thanks,
-- Shuah
Shuah Khan Aug. 15, 2023, 6:13 p.m. UTC | #4
On 8/15/23 08:39, Shuah Khan wrote:
> On 8/15/23 08:35, Willy Tarreau wrote:
>> On Tue, Aug 15, 2023 at 08:25:51AM -0600, Shuah Khan wrote:
>>>> The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
>>>>
>>>>     Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
>>>>
>>>> are available in the Git repository at:
>>>>
>>>>     https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
>>>>
>>>> for you to fetch changes up to d98c1e27e46e47a3ae67e1d048f153598ba82611:
>>>>
>>>>     tools/nolibc: stackprotector.h: make __stack_chk_init static (2023-08-06 18:44:47 +0200)
>>>>
>>>
>>> Hi Willy,
>>>
>>> I am sorry for the delay on this. I was traveling last week
>>> and getting back to digging myself out of emails.
>>
>> No problem, thanks for getting back :-)
>>
>>> I am having trouble pulling this request though:
>>>
>>> git request-pull https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
>>>
>>> gives me the following error
>>>
>>> fatal: Not a valid revision: git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/
>>>
>>> I don't see a tag at https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
>>
>> Ah sorry for the misunderstanding, that's the branch name, I'll set a
>> tag then.
>>
> 
> No worries. Could you also share the test you run? I will pull
> you request and run tests.
> 

Please send either another pull request or send the tag details.

Also please join IRC channel #kselftest on OFTC. It will be easier
to chat. Unless I am traveling you can find me there Nick: shuah

thanks,
-- Shuah
Willy Tarreau Aug. 15, 2023, 6:48 p.m. UTC | #5
Hello Shuah,

On Tue, Aug 15, 2023 at 12:13:24PM -0600, Shuah Khan wrote:
> On 8/15/23 08:39, Shuah Khan wrote:
> > On 8/15/23 08:35, Willy Tarreau wrote:
> > > On Tue, Aug 15, 2023 at 08:25:51AM -0600, Shuah Khan wrote:
> > > > > The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
> > > > > 
> > > > >     Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
> > > > > 
> > > > > are available in the Git repository at:
> > > > > 
> > > > >     https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
> > > > > 
> > > > > for you to fetch changes up to d98c1e27e46e47a3ae67e1d048f153598ba82611:
> > > > > 
> > > > >     tools/nolibc: stackprotector.h: make __stack_chk_init static (2023-08-06 18:44:47 +0200)
> > > > > 
> > > > 
> > > > Hi Willy,
> > > > 
> > > > I am sorry for the delay on this. I was traveling last week
> > > > and getting back to digging myself out of emails.
> > > 
> > > No problem, thanks for getting back :-)
> > > 
> > > > I am having trouble pulling this request though:
> > > > 
> > > > git request-pull https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
> > > > 
> > > > gives me the following error
> > > > 
> > > > fatal: Not a valid revision: git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/
> > > > 
> > > > I don't see a tag at https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
> > > 
> > > Ah sorry for the misunderstanding, that's the branch name, I'll set a
> > > tag then.
> > > 
> > 
> > No worries. Could you also share the test you run? I will pull
> > you request and run tests.
> > 
> 
> Please send either another pull request or send the tag details.

I've pushed a tag named 20230815-for-6.6-2 in the repo below:

  https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git

Please let me know if you want me to resend a PR.

I'm also attaching a script I'm using here to run the tests on the
various archs. It's quite limited but gets the job done for now.
There are simplifications coming for future versions. The simplest
way to use it is:

  1) make sure you have the relevant toolchains. We're currently
     using those Arnd maintains and that are located here:

       https://mirrors.edge.kernel.org/pub/tools/crosstool/

     If that helps I'm attaching a tiny script I have which downloads
     all needed ones from a given version and extracts them in a same
     directory.

     Please note, if you've downloaded version 13.1.0, I've had a
     segfault on gcc-aarch64 only on this one and that was fixed with
     13.2.0. I preferred to tell you upfront instead of wasting your
     time on it.

  2) from the kernel top dir, copy the test-all-full4.sh script into
     tools/testing/selftests/nolibc/ then indicate where you extracted
     the crosstools:

       $ cd tools/testing/selftests/nolibc
       $ TOOLCHAIN_BASE=/path/to/gcc-13.2.0-nolibc ./test-all-full4.sh

     For each supported arch, it will build the test program, run it in
     userland under qemu with your current kernel, and will also build
     a kernel, install the program in it and run it under qemu. Support
     for loongarch in Qemu is recent, so if the one from your distro
     doesn't have it, you can instead build for all other archs by
     passing their names in argument to the program:

       $ TOOLCHAIN_BASE=... ./test-all-full4.sh \
         i386 x86_64 arm64 arm mips ppc ppc64 ppc64le riscv s390

     Note that the archs most of the time match the kernel ones, except
     when there are variants, where we defined extra ones (typically to
     force 32/64 bits), and in this case we use distinct names to avoid
     any ambiguity. This explains "ppc" and "ppc64" above, which both
     map to "powerpc" when building the kernel.

  3) At the end it will show a synthetic report indicating what archs
     succeeded/failed and some will show "warning" if there is at
     least one test skipped:

     #### Synthetic summary:
     USER     i386  160   test(s):  158      passed,  2  skipped,  0  failed  =>  status:  warning
     KERN     i386  160   test(s):  160      passed,  0  skipped,  0  failed  =>  status:  success

     Two "skipped" are expected for the userland tests, due to privileged
     operations being skipped. That is OK. On MIPS and s390 depending on
     the toolchains, another skipped can appear due to the stack-protector
     test being skipped. In this case it will be 3 skipped for userland,
     and 1 skipped for the kernel. There should never be any failure.

  4) Save precious time: it is often useful to just run the userland
     tests when you're not testing for a kernel regression: the program
     is exactly the same and if you didn't change your kernel between
     tests there is no reason you'll get a different result once you know
     what the userland reports. For this, set USER_ONLY=1 when calling
     the script:

       $ TOOLCHAIN_BASE=... ./test-all-full4.sh \
         USER_ONLY=1 \
         i386 x86_64 arm64 arm mips ppc ppc64 ppc64le riscv s390

     It will skip all the kernel parts and will only install UAPI headers,
     build nolibc-test for each arch and run it through Qemu. Quite
     honestly this is the recommended way to use it when you're not
     seeking a specific kernel issue. On my laptop it takes 24 seconds
     (loongarch was not tested here):

     #### Synthetic summary:
     USER  i386     160  test(s):  158  passed,  2  skipped,  0  failed  =>  status:  warning
     USER  x86_64   160  test(s):  158  passed,  2  skipped,  0  failed  =>  status:  warning
     USER  arm64    160  test(s):  158  passed,  2  skipped,  0  failed  =>  status:  warning
     USER  arm      160  test(s):  158  passed,  2  skipped,  0  failed  =>  status:  warning
     USER  mips     160  test(s):  157  passed,  3  skipped,  0  failed  =>  status:  warning
     USER  ppc      160  test(s):  158  passed,  2  skipped,  0  failed  =>  status:  warning
     USER  ppc64    160  test(s):  158  passed,  2  skipped,  0  failed  =>  status:  warning
     USER  ppc64le  160  test(s):  158  passed,  2  skipped,  0  failed  =>  status:  warning
     USER  riscv    160  test(s):  158  passed,  2  skipped,  0  failed  =>  status:  warning
     USER  s390     160  test(s):  157  passed,  3  skipped,  0  failed  =>  status:  warning

  5) Other variables: it's possible to force a specific cross-compiler
     prefix with CROSS_COMPILE_$arch, or when building for a single arch,
     CROSS_COMPILE, a specific CC with CC, and to enable verbose build
     with V=1.

  6) A complete capture of all outputs is done in "report.out", and a
     copy of each used nolibc-test executable is kept with the suffix
     -$arch.

The script contains a "set -e" statement to stop on any error. There is
one known case (missing qemu binary for userlan) where the error is
ignored in the makefile and the build will continue. But this stuff is
currently being improved so I consider there's no point re-changing this
at the last minute at the risk of breaking other stuff.

Do not hesitate to ask if you encounter difficulties or if anything is
not clear.

Thank you!
Willy
Shuah Khan Aug. 15, 2023, 9:06 p.m. UTC | #6
On 8/15/23 12:48, Willy Tarreau wrote:
> Hello Shuah,
> 
> On Tue, Aug 15, 2023 at 12:13:24PM -0600, Shuah Khan wrote:
>> On 8/15/23 08:39, Shuah Khan wrote:
>>> On 8/15/23 08:35, Willy Tarreau wrote:
>>>> On Tue, Aug 15, 2023 at 08:25:51AM -0600, Shuah Khan wrote:
>>>>>> The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
>>>>>>
>>>>>>      Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
>>>>>>
>>>>>> are available in the Git repository at:
>>>>>>
>>>>>>      https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
>>>>>>
>>>>>> for you to fetch changes up to d98c1e27e46e47a3ae67e1d048f153598ba82611:
>>>>>>
>>>>>>      tools/nolibc: stackprotector.h: make __stack_chk_init static (2023-08-06 18:44:47 +0200)
>>>>>>
>>>>>
>>>>> Hi Willy,
>>>>>
>>>>> I am sorry for the delay on this. I was traveling last week
>>>>> and getting back to digging myself out of emails.
>>>>
>>>> No problem, thanks for getting back :-)
>>>>
>>>>> I am having trouble pulling this request though:
>>>>>
>>>>> git request-pull https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ 20230806-for-6.6-1
>>>>>
>>>>> gives me the following error
>>>>>
>>>>> fatal: Not a valid revision: git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/
>>>>>
>>>>> I don't see a tag at https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
>>>>
>>>> Ah sorry for the misunderstanding, that's the branch name, I'll set a
>>>> tag then.
>>>>
>>>
>>> No worries. Could you also share the test you run? I will pull
>>> you request and run tests.
>>>
>>
>> Please send either another pull request or send the tag details.
> 
> I've pushed a tag named 20230815-for-6.6-2 in the repo below:
> 
>    https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
> 
> Please let me know if you want me to resend a PR.
> 

Pull worked fine and my verify_fixes script failed on the following patches.

Commit: 6c931bf0c732 ("selftests/nolibc: avoid buffer underrun in space printing")
	Fixes tag: Fixes: 8a27526f49f9 ("selftests/nolibc: add EXPECT_PTREQ, EXPECT_PTRNE and EXPECT_PTRER")
	Has these problem(s):
		- Target SHA1 does not exist
Commit: 40f12898b479 ("tools/nolibc/stdio: add setvbuf() to set buffering mode")
	Fixes tag: Fixes: ecb7fe2cd610 ("selftests: line buffer test program's stdout")
	Has these problem(s):
		- Target SHA1 does not exist


Will you be able to add the right commit IDs and resend the pull, so there
are no mistakes if I guess it wrong.

thanks,
-- Shuah
Willy Tarreau Aug. 16, 2023, 3:12 a.m. UTC | #7
On Tue, Aug 15, 2023 at 03:06:39PM -0600, Shuah Khan wrote:
> Pull worked fine and my verify_fixes script failed on the following patches.
> 
> Commit: 6c931bf0c732 ("selftests/nolibc: avoid buffer underrun in space printing")
> 	Fixes tag: Fixes: 8a27526f49f9 ("selftests/nolibc: add EXPECT_PTREQ, EXPECT_PTRNE and EXPECT_PTRER")
> 	Has these problem(s):
> 		- Target SHA1 does not exist
> Commit: 40f12898b479 ("tools/nolibc/stdio: add setvbuf() to set buffering mode")
> 	Fixes tag: Fixes: ecb7fe2cd610 ("selftests: line buffer test program's stdout")
> 	Has these problem(s):
> 		- Target SHA1 does not exist
> 
> 
> Will you be able to add the right commit IDs and resend the pull, so there
> are no mistakes if I guess it wrong.

Will do, many thanks for the check!
Willy
Willy Tarreau Aug. 16, 2023, 8:35 a.m. UTC | #8
Hello Shuah,

On Tue, Aug 15, 2023 at 03:06:39PM -0600, Shuah Khan wrote:
> > I've pushed a tag named 20230815-for-6.6-2 in the repo below:
> > 
> >    https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
> > 
> > Please let me know if you want me to resend a PR.
> > 
> 
> Pull worked fine and my verify_fixes script failed on the following patches.
> 
> Commit: 6c931bf0c732 ("selftests/nolibc: avoid buffer underrun in space printing")
> 	Fixes tag: Fixes: 8a27526f49f9 ("selftests/nolibc: add EXPECT_PTREQ, EXPECT_PTRNE and EXPECT_PTRER")
> 	Has these problem(s):
> 		- Target SHA1 does not exist

Sadly this one was within the same branch and got rebased, now fixed.
Another one had the same issue after the fix, I've dropped the fixes
tag.

> Commit: 40f12898b479 ("tools/nolibc/stdio: add setvbuf() to set buffering mode")
> 	Fixes tag: Fixes: ecb7fe2cd610 ("selftests: line buffer test program's stdout")
> 	Has these problem(s):
> 		- Target SHA1 does not exist

This one mentioned a commit that's only in linux-next, I simply dropped
the fixes tag.

> Will you be able to add the right commit IDs and resend the pull, so there
> are no mistakes if I guess it wrong.

Please use the new following one instead.

Thanks!
Willy

The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:

  Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)

are available in the Git repository at:

  https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ tags/20230816-for-6.6-3

for you to fetch changes up to 938315d721db3b08c958e92b8237fb4986c66b7e:

  tools/nolibc: avoid undesired casts in the __sysret() macro (2023-08-16 10:31:17 +0200)

----------------------------------------------------------------
Ryan Roberts (1):
      tools/nolibc/stdio: add setvbuf() to set buffering mode

Thomas Weißschuh (22):
      selftests/nolibc: drop test chmod_net
      selftests/nolibc: simplify call to ioperm
      tools/nolibc: completely remove optional environ support
      selftests/nolibc: make evaluation of test conditions
      selftests/nolibc: simplify status printing
      selftests/nolibc: avoid gaps in test numbers
      selftests/nolibc: avoid buffer underrun in space printing
      tools/nolibc: drop unused variables
      tools/nolibc: fix return type of getpagesize()
      tools/nolibc: setvbuf: avoid unused parameter warnings
      tools/nolibc: sys: avoid implicit sign cast
      tools/nolibc: stdint: use __SIZE_TYPE__ for size_t
      selftests/nolibc: drop unused variables
      selftests/nolibc: mark test helpers as potentially unused
      selftests/nolibc: make functions static if possible
      selftests/nolibc: avoid unused parameter warnings
      selftests/nolibc: avoid sign-compare warnings
      selftests/nolibc: use correct return type for read() and write()
      selftests/nolibc: prevent out of bounds access in expect_vfprintf
      selftests/nolibc: don't strip nolibc-test
      selftests/nolibc: enable compiler warnings
      MAINTAINERS: nolibc: add myself as co-maintainer

Willy Tarreau (3):
      selftests/nolibc: avoid warnings during intptr tests
      tools/nolibc: keep brk(), sbrk(), mmap() away from __sysret()
      tools/nolibc: avoid undesired casts in the __sysret() macro

Yuan Tan (2):
      tools/nolibc: add pipe() and pipe2() support
      selftests/nolibc: add testcase for pipe

Zhangjin Wu (76):
      selftests/nolibc: add a standalone test report macro
      selftests/nolibc: always print the path to test log file
      selftests/nolibc: restore the failed tests print
      tools/nolibc: fix up #error compile failures with -ENOSYS
      tools/nolibc: fix up undeclared syscall macros with #ifdef and -ENOSYS
      tools/nolibc: sys.h: add a syscall return helper
      tools/nolibc: unistd.h: apply __sysret() helper
      tools/nolibc: sys.h: apply __sysret() helper
      tools/nolibc: unistd.h: reorder the syscall macros
      tools/nolibc: arch-*.h: fix up code indent errors
      toolc/nolibc: arch-*.h: clean up whitespaces after __asm__
      tools/nolibc: arch-loongarch.h: shrink with _NOLIBC_SYSCALL_CLOBBERLIST
      tools/nolibc: arch-mips.h: shrink with _NOLIBC_SYSCALL_CLOBBERLIST
      tools/nolibc: add missing my_syscall6() for mips
      tools/nolibc: __sysret: support syscalls who return a pointer
      tools/nolibc: clean up mmap() routine
      tools/nolibc: clean up sbrk() routine
      selftests/nolibc: export argv0 for some tests
      selftests/nolibc: prepare: create /dev/zero
      selftests/nolibc: add EXPECT_PTREQ, EXPECT_PTRNE and EXPECT_PTRER
      selftests/nolibc: add sbrk_0 to test current brk getting
      selftests/nolibc: add mmap_bad test case
      selftests/nolibc: add munmap_bad test case
      selftests/nolibc: add mmap_munmap_good test case
      selftests/nolibc: add run-libc-test target
      selftests/nolibc: stat_fault: silence NULL argument warning with glibc
      selftests/nolibc: gettid: restore for glibc and musl
      selftests/nolibc: add _LARGEFILE64_SOURCE for musl
      selftests/nolibc: fix up int_fast16/32_t test cases for musl
      tools/nolibc: types.h: add RB_ flags for reboot()
      selftests/nolibc: prefer <sys/reboot.h> to <linux/reboot.h>
      selftests/nolibc: fix up kernel parameters support
      selftests/nolibc: link_cross: use /proc/self/cmdline
      tools/nolibc: add rmdir() support
      selftests/nolibc: add a new rmdir() test case
      selftests/nolibc: fix up failures when CONFIG_PROC_FS=n
      selftests/nolibc: prepare /tmp for tests that need to write
      selftests/nolibc: vfprintf: remove MEMFD_CREATE dependency
      selftests/nolibc: chdir_root: restore current path after test
      selftests/nolibc: stat_timestamps: remove procfs dependency
      selftests/nolibc: chroot_exe: remove procfs dependency
      selftests/nolibc: add chmod_argv0 test
      selftests/nolibc: report: print a summarized test status
      selftests/nolibc: report: print total tests
      selftests/nolibc: report: align passed, skipped and failed
      selftests/nolibc: report: extrude the test status line
      selftests/nolibc: report: add newline before test failures
      tools/nolibc: arch-*.h: add missing space after ','
      tools/nolibc: fix up startup failures for -O0 under gcc < 11.1.0
      tools/nolibc: remove the old sys_stat support
      tools/nolibc: add new crt.h with _start_c
      tools/nolibc: stackprotector.h: add empty __stack_chk_init for !_NOLIBC_STACKPROTECTOR
      tools/nolibc: crt.h: initialize stack protector
      tools/nolibc: arm: shrink _start with _start_c
      tools/nolibc: aarch64: shrink _start with _start_c
      tools/nolibc: i386: shrink _start with _start_c
      tools/nolibc: x86_64: shrink _start with _start_c
      tools/nolibc: mips: shrink _start with _start_c
      tools/nolibc: loongarch: shrink _start with _start_c
      tools/nolibc: riscv: shrink _start with _start_c
      tools/nolibc: s390: shrink _start with _start_c
      selftests/nolibc: add EXPECT_PTRGE, EXPECT_PTRGT, EXPECT_PTRLE, EXPECT_PTRLT
      selftests/nolibc: add testcases for startup code
      selftests/nolibc: allow run nolibc-test locally
      selftests/nolibc: allow test -include /path/to/nolibc.h
      selftests/nolibc: mmap_munmap_good: fix up return value
      tools/nolibc: add support for powerpc
      tools/nolibc: add support for powerpc64
      selftests/nolibc: add XARCH and ARCH mapping support
      selftests/nolibc: add test support for ppc
      selftests/nolibc: add test support for ppc64le
      selftests/nolibc: add test support for ppc64
      selftests/nolibc: allow report with existing test log
      tools/nolibc: stackprotector.h: make __stack_chk_init static
      selftests/nolibc: libc-test: use HOSTCC instead of CC
      tools/nolibc: silence ppc64 compile warnings

 MAINTAINERS                                  |   1 +
 tools/include/nolibc/Makefile                |   1 +
 tools/include/nolibc/arch-aarch64.h          |  85 +---
 tools/include/nolibc/arch-arm.h              | 111 +----
 tools/include/nolibc/arch-i386.h             |  86 +---
 tools/include/nolibc/arch-loongarch.h        |  83 +---
 tools/include/nolibc/arch-mips.h             | 147 +++----
 tools/include/nolibc/arch-powerpc.h          | 221 ++++++++++
 tools/include/nolibc/arch-riscv.h            |  83 +---
 tools/include/nolibc/arch-s390.h             |  77 +---
 tools/include/nolibc/arch-x86_64.h           |  86 +---
 tools/include/nolibc/arch.h                  |   2 +
 tools/include/nolibc/crt.h                   |  61 +++
 tools/include/nolibc/nolibc.h                |   9 +-
 tools/include/nolibc/stackprotector.h        |   5 +-
 tools/include/nolibc/stdint.h                |   2 +-
 tools/include/nolibc/stdio.h                 |  27 ++
 tools/include/nolibc/stdlib.h                |  12 +-
 tools/include/nolibc/sys.h                   | 534 +++++++----------------
 tools/include/nolibc/types.h                 |  22 +-
 tools/include/nolibc/unistd.h                |  13 +-
 tools/testing/selftests/nolibc/Makefile      | 111 +++--
 tools/testing/selftests/nolibc/nolibc-test.c | 609 ++++++++++++++++++++-------
 23 files changed, 1221 insertions(+), 1167 deletions(-)
 create mode 100644 tools/include/nolibc/arch-powerpc.h
 create mode 100644 tools/include/nolibc/crt.h
Shuah Khan Aug. 16, 2023, 2:21 p.m. UTC | #9
On 8/16/23 02:35, Willy Tarreau wrote:
> Hello Shuah,
> 

> 
> The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
> 
>    Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
> 
> are available in the Git repository at:
> 
>    https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git/ tags/20230816-for-6.6-3
> 
> for you to fetch changes up to 938315d721db3b08c958e92b8237fb4986c66b7e:
> 
>    tools/nolibc: avoid undesired casts in the __sysret() macro (2023-08-16 10:31:17 +0200)
> 

Applied. Thank you.

https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/log/?h=nolibc

I will run tests later on today and keep you updated.

thanks,
-- Shuah