diff mbox series

certs: buffer stderr from openssl unless error

Message ID 20230517-genkey-v1-1-b887424da4a8@google.com (mailing list archive)
State New
Headers show
Series certs: buffer stderr from openssl unless error | expand

Commit Message

Nick Desaulniers May 17, 2023, 4:23 p.m. UTC 
Running `openssl req` prints a progress meter consisting of `.`, `*`,
and `+` characters to stderr which we redirect to stdout. During a build
with `make -j`, the output from this command becomes interspersed
throughout the rest of the quiet_cmd_* output, messing up the
indentation.

Suppress the output from this command unless the return code is
non-zero. If `openssl req` prints additional information to stderr
without setting a non-zero return code, it will be missed.

Suggested-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
 certs/Makefile   | 4 +---
 certs/gen_key.sh | 7 +++++++
 2 files changed, 8 insertions(+), 3 deletions(-)


---
base-commit: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
change-id: 20230517-genkey-24a835572835

Best regards,

Comments

Jarkko Sakkinen May 18, 2023, 6:32 p.m. UTC | #1 
On Wed May 17, 2023 at 7:23 PM EEST,  wrote:
> Running `openssl req` prints a progress meter consisting of `.`, `*`,
> and `+` characters to stderr which we redirect to stdout. During a build
> with `make -j`, the output from this command becomes interspersed
> throughout the rest of the quiet_cmd_* output, messing up the
> indentation.
>
> Suppress the output from this command unless the return code is
> non-zero. If `openssl req` prints additional information to stderr
> without setting a non-zero return code, it will be missed.
>
> Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
>  certs/Makefile   | 4 +---
>  certs/gen_key.sh | 7 +++++++
>  2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/certs/Makefile b/certs/Makefile
> index 799ad7b9e68a..9b4fee56780d 100644
> --- a/certs/Makefile
> +++ b/certs/Makefile
> @@ -45,9 +45,7 @@ ifeq ($(CONFIG_MODULE_SIG_KEY),certs/signing_key.pem)
>  keytype-$(CONFIG_MODULE_SIG_KEY_TYPE_ECDSA) := -newkey ec -pkeyopt ec_paramgen_curve:secp384r1
>  
>  quiet_cmd_gen_key = GENKEY  $@
> -      cmd_gen_key = openssl req -new -nodes -utf8 -$(CONFIG_MODULE_SIG_HASH) -days 36500 \
> -		-batch -x509 -config $< \
> -		-outform PEM -out $@ -keyout $@ $(keytype-y) 2>&1
> +      cmd_gen_key = $(srctree)/$(src)/gen_key.sh $(CONFIG_MODULE_SIG_HASH) $< $@ $(keytype-y)
>  
>  $(obj)/signing_key.pem: $(obj)/x509.genkey FORCE
>  	$(call if_changed,gen_key)
> diff --git a/certs/gen_key.sh b/certs/gen_key.sh
> new file mode 100755
> index 000000000000..1de1f22be484
> --- /dev/null
> +++ b/certs/gen_key.sh
> @@ -0,0 +1,7 @@
> +#!/usr/bin/env bash
> +# SPDX-License-Identifier: GPL-2.0
> +OUT=$(openssl req -new -nodes -utf8 -"$1" -days 36500 -batch -x509 \
> +	-config "$2" -outform PEM -out "$3" -keyout "$3" $4 2>&1)
> +if [[ $? -ne 0 ]]; then
> +	echo "$OUT"
> +fi
>
> ---
> base-commit: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
> change-id: 20230517-genkey-24a835572835
>
> Best regards,
> -- 
> Nick Desaulniers <ndesaulniers@google.com>

Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>

BR, Jarkko
Nick Desaulniers May 18, 2023, 10:36 p.m. UTC | #2 
On Thu, May 18, 2023 at 3:23 AM kernel test robot <lkp@intel.com> wrote:
>
> Hi,
>
> kernel test robot noticed the following build errors:
>
> [auto build test ERROR on f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6]
>
> url:    https://github.com/intel-lab-lkp/linux/commits/ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
> base:   f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
> patch link:    https://lore.kernel.org/r/20230517-genkey-v1-1-b887424da4a8%40google.com
> patch subject: [PATCH] certs: buffer stderr from openssl unless error
> config: csky-randconfig-m041-20230517
> compiler: csky-linux-gcc (GCC) 12.1.0
> reproduce (this is a W=1 build):
>         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross

^ should encourage `mkdir ~/bin`; the debian docker container doesn't
have `~/bin`.

>         chmod +x ~/bin/make.cross
>         # https://github.com/intel-lab-lkp/linux/commit/0e437a41fdb41c84834de6776bf38951b197792a
>         git remote add linux-review https://github.com/intel-lab-lkp/linux
>         git fetch --no-tags linux-review ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753

^ holy crap that takes forever. maybe recommend `b4 shazam <lore link>`?

>         git checkout 0e437a41fdb41c84834de6776bf38951b197792a
>         # save the config file
>         mkdir build_dir && cp config build_dir/.config
>         COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky olddefconfig

^ should use `~/bin/make.cross` since `~/bin` may not exist in $PATH.

>         COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky SHELL=/bin/bash

Hi Philip,
I've run into some issues with make.cross when trying to install the
csky toolchain. Maybe you can help?

```
$ COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0
~/bin/make.cross W=1 O=build_dir ARCH=csky olddefconfig
Compiler will be installed in /root/0day
make: gcc: No such file or directory
lftpget -c https://download.01.org/0day-ci/cross-package/./gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
/linux
tar Jxf /root/0day/gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
-C /root/0day
Please update: libc6 or glibc
ldd /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc
/root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found
(required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
/root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found
(required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
setup_crosstool failed

$ ldd -v /lib/x86_64-linux-gnu/libc.so.6
/lib64/ld-linux-x86-64.so.2 (0x00007f11f3478000)
linux-vdso.so.1 (0x00007ffcf45b2000)

Version information:
/lib/x86_64-linux-gnu/libc.so.6:
ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2

$ cat /etc/debian_version
11.6
```
Perhaps I MUST update my debian container to 11.7?

Are the 0day toolchains not statically linked? Can they use the ones
from kernel.org which are?
https://mirrors.edge.kernel.org/pub/tools/crosstool/

>
> If you fix the issue, kindly add following tag where applicable
> | Reported-by: kernel test robot <lkp@intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202305181712.xjvv1fbD-lkp@intel.com/
>
> All errors (new ones prefixed by >>):
>
>    At main.c:152:
>    - SSL error:FFFFFFFF80000002:system library::No such file or directory: ../crypto/bio/bss_file.c:67
>    - SSL error:10000080:BIO routines::no such file: ../crypto/bio/bss_file.c:75
> >> extract-cert: certs/signing_key.pem: No such file or directory
>
> --
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki
kernel test robot May 22, 2023, 3:35 p.m. UTC | #3 
Hi,

kernel test robot noticed the following build errors:

[auto build test ERROR on f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6]

url:    https://github.com/intel-lab-lkp/linux/commits/ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
base:   f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
patch link:    https://lore.kernel.org/r/20230517-genkey-v1-1-b887424da4a8%40google.com
patch subject: [PATCH] certs: buffer stderr from openssl unless error
config: arm-randconfig-r046-20230521 (https://download.01.org/0day-ci/archive/20230522/202305222321.45ezNpB8-lkp@intel.com/config)
compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project b0fb98227c90adf2536c9ad644a74d5e92961111)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install arm cross compiling tool for clang build
        # apt-get install binutils-arm-linux-gnueabi
        # https://github.com/intel-lab-lkp/linux/commit/0e437a41fdb41c84834de6776bf38951b197792a
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
        git checkout 0e437a41fdb41c84834de6776bf38951b197792a
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm olddefconfig
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm SHELL=/bin/bash

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202305222321.45ezNpB8-lkp@intel.com/

All errors (new ones prefixed by >>):

   At main.c:152:
   - SSL error:FFFFFFFF80000002:system library::No such file or directory: ../crypto/bio/bss_file.c:67
   - SSL error:10000080:BIO routines::no such file: ../crypto/bio/bss_file.c:75
>> extract-cert: certs/signing_key.pem: No such file or directory
diff mbox series

Patch

diff --git a/certs/Makefile b/certs/Makefile
index 799ad7b9e68a..9b4fee56780d 100644
--- a/certs/Makefile
+++ b/certs/Makefile
@@ -45,9 +45,7 @@  ifeq ($(CONFIG_MODULE_SIG_KEY),certs/signing_key.pem)
 keytype-$(CONFIG_MODULE_SIG_KEY_TYPE_ECDSA) := -newkey ec -pkeyopt ec_paramgen_curve:secp384r1
 
 quiet_cmd_gen_key = GENKEY  $@
-      cmd_gen_key = openssl req -new -nodes -utf8 -$(CONFIG_MODULE_SIG_HASH) -days 36500 \
-		-batch -x509 -config $< \
-		-outform PEM -out $@ -keyout $@ $(keytype-y) 2>&1
+      cmd_gen_key = $(srctree)/$(src)/gen_key.sh $(CONFIG_MODULE_SIG_HASH) $< $@ $(keytype-y)
 
 $(obj)/signing_key.pem: $(obj)/x509.genkey FORCE
 	$(call if_changed,gen_key)
diff --git a/certs/gen_key.sh b/certs/gen_key.sh
new file mode 100755
index 000000000000..1de1f22be484
--- /dev/null
+++ b/certs/gen_key.sh
@@ -0,0 +1,7 @@ 
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+OUT=$(openssl req -new -nodes -utf8 -"$1" -days 36500 -batch -x509 \
+	-config "$2" -outform PEM -out "$3" -keyout "$3" $4 2>&1)
+if [[ $? -ne 0 ]]; then
+	echo "$OUT"
+fi