[v6] kbuild: add script and target to generate pacman package

Commit Message

Thomas Weißschuh July 16, 2024, 5:52 p.m. UTC

pacman is the package manager used by Arch Linux and its derivates.
Creating native packages from the kernel tree has multiple advantages:

* The package triggers the correct hooks for initramfs generation and
  bootloader configuration
* Uninstallation is complete and also invokes the relevant hooks
* New UAPI headers can be installed without any manual bookkeeping

The PKGBUILD file is a simplified version of the one used for the
downstream Arch Linux "linux" package.
Extra steps that should not be necessary for a development kernel have
been removed and an UAPI header package has been added.

Signed-off-by: Thomas Weißschuh <[e-mail]>
Reviewed-by: Nathan Chancellor <[e-mail]>
Tested-by: Nathan Chancellor <[e-mail]>
Reviewed-by: Nicolas Schier <[e-mail]>
---
Changes in v6:
- Drop reference to srctree/Makefile
- Drop $(realpath $(srctree))
- Make use of the fact that $(objtree) is always "."
- Align coding style to kernel and drop vim config line
- Drop indirection through `$MAKE run-command`
- Unify shell variable syntax to "${var}"
- Add explanations to custom variables
- Add makedepends
- Link to v5: https://lore.kernel.org/r/[e-mail]

Changes in v5:
- Rebase onto kbuild/for-next
- Use new path to build-version script (from kbuild/for-next)
- Ensure submake jobserver delegation works
- Simplify $modulesdir/pkgbase file creation
- Add Reviewed-by from Nicolas
- Link to v4: https://lore.kernel.org/r/[e-mail]

Changes in v4:
- Update MRPROPER_FILES
- Unify shell variable syntax
- Link to v3: https://lore.kernel.org/r/[e-mail]

Changes in v3:
- Enforce matching architectures for installation
- Add Reviewed-by and Tested-by from Nathan
- Link to v2: https://lore.kernel.org/r/[e-mail]

Changes in v2:
- Replace ${MAKE} with $MAKE for consistency with other variables
- Use $MAKE for "-s image_name"
- Avoid permission warnings from build directory
- Clarify reason for /build symlink removal
- Install System.map and config
- Install dtbs where available
- Allow cross-build through arch=any
- Sort Contributor/Maintainer chronologically
- Disable some unneeded makepkg options
- Use DEPMOD=true for consistency with rpm-package
- Link to v1: https://lore.kernel.org/r/[e-mail]
---
 .gitignore               |  6 +++
 Makefile                 |  2 +-
 scripts/Makefile.package | 14 +++++++
 scripts/package/PKGBUILD | 99 ++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 120 insertions(+), 1 deletion(-)


---
base-commit: 818e9c998b04d6c69a510d5255a93d0e3b8d4993
change-id: 20240625-kbuild-pacman-pkg-b4f87e19d036

Best regards,

Comments

Nathan Chancellor July 17, 2024, 1:15 a.m. UTC | #1

Hi Thomas,

On Tue, Jul 16, 2024 at 07:52:14PM +0200, Thomas Weißschuh wrote:
> pacman is the package manager used by Arch Linux and its derivates.
> Creating native packages from the kernel tree has multiple advantages:
> 
> * The package triggers the correct hooks for initramfs generation and
>   bootloader configuration
> * Uninstallation is complete and also invokes the relevant hooks
> * New UAPI headers can be installed without any manual bookkeeping
> 
> The PKGBUILD file is a simplified version of the one used for the
> downstream Arch Linux "linux" package.
> Extra steps that should not be necessary for a development kernel have
> been removed and an UAPI header package has been added.
> 
> Signed-off-by: Thomas Weißschuh <[e-mail]>
> Reviewed-by: Nathan Chancellor <[e-mail]>
> Tested-by: Nathan Chancellor <[e-mail]>
> Reviewed-by: Nicolas Schier <[e-mail]>
> ---
> Changes in v6:
> - Drop reference to srctree/Makefile
> - Drop $(realpath $(srctree))
> - Make use of the fact that $(objtree) is always "."
> - Align coding style to kernel and drop vim config line
> - Drop indirection through `$MAKE run-command`
> - Unify shell variable syntax to "${var}"
> - Add explanations to custom variables
> - Add makedepends
> - Link to v5: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v5:
> - Rebase onto kbuild/for-next
> - Use new path to build-version script (from kbuild/for-next)
> - Ensure submake jobserver delegation works
> - Simplify $modulesdir/pkgbase file creation
> - Add Reviewed-by from Nicolas
> - Link to v4: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v4:
> - Update MRPROPER_FILES
> - Unify shell variable syntax
> - Link to v3: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v3:
> - Enforce matching architectures for installation
> - Add Reviewed-by and Tested-by from Nathan
> - Link to v2: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v2:
> - Replace ${MAKE} with $MAKE for consistency with other variables
> - Use $MAKE for "-s image_name"
> - Avoid permission warnings from build directory
> - Clarify reason for /build symlink removal
> - Install System.map and config
> - Install dtbs where available
> - Allow cross-build through arch=any
> - Sort Contributor/Maintainer chronologically
> - Disable some unneeded makepkg options
> - Use DEPMOD=true for consistency with rpm-package
> - Link to v1: https://lore.kernel.org/r/[e-mail]
> ---
>  .gitignore               |  6 +++
>  Makefile                 |  2 +-
>  scripts/Makefile.package | 14 +++++++
>  scripts/package/PKGBUILD | 99 ++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 120 insertions(+), 1 deletion(-)
> 
> diff --git a/.gitignore b/.gitignore
> index c59dc60ba62e..7902adf4f7f1 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -92,6 +92,12 @@ modules.order
>  #
>  /tar-install/
>  
> +#
> +# pacman files (make pacman-pkg)
> +#
> +/PKGBUILD
> +/pacman/
> +
>  #
>  # We don't want to ignore the following even if they are dot-files
>  #
> diff --git a/Makefile b/Makefile
> index 7372ea45ed3f..768d3dc107f8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1481,7 +1481,7 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
>  # Directories & files removed with 'make mrproper'
>  MRPROPER_FILES += include/config include/generated          \
>  		  arch/$(SRCARCH)/include/generated .objdiff \
> -		  debian snap tar-install \
> +		  debian snap tar-install PKGBUILD pacman \
>  		  .config .config.old .version \
>  		  Module.symvers \
>  		  certs/signing_key.pem \
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index bf016af8bf8a..0aaa0832279c 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -141,6 +141,19 @@ snap-pkg:
>  	cd $(objtree)/snap && \
>  	snapcraft --target-arch=$(UTS_MACHINE)
>  
> +# pacman-pkg
> +# ---------------------------------------------------------------------------
> +
> +PHONY += pacman-pkg
> +pacman-pkg:
> +	@ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> +	+objtree="$(realpath $(objtree))" \
> +		BUILDDIR=pacman \
> +		CARCH="$(UTS_MACHINE)" \
> +		KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> +		KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> +		makepkg
> +
>  # dir-pkg tar*-pkg - tarball targets
>  # ---------------------------------------------------------------------------
>  
> @@ -221,6 +234,7 @@ help:
>  	@echo '  bindeb-pkg          - Build only the binary kernel deb package'
>  	@echo '  snap-pkg            - Build only the binary kernel snap package'
>  	@echo '                        (will connect to external hosts)'
> +	@echo '  pacman-pkg          - Build only the binary kernel pacman package'
>  	@echo '  dir-pkg             - Build the kernel as a plain directory structure'
>  	@echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
>  	@echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
> diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> new file mode 100644
> index 000000000000..eb3957fad915
> --- /dev/null
> +++ b/scripts/package/PKGBUILD
> @@ -0,0 +1,99 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +# Maintainer: Thomas Weißschuh <[e-mail]>
> +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> +
> +pkgbase=linux-upstream
> +pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> +pkgver="${KERNELRELEASE//-/_}"
> +# The PKGBUILD is evaluated multiple times.
> +# Running scripts/build-version from here would introduce inconsistencies.
> +pkgrel="${KBUILD_REVISION}"
> +pkgdesc='Linux'
> +url='https://www.kernel.org/'
> +# Enable flexible cross-compilation
> +arch=(${CARCH})
> +license=(GPL-2.0-only)
> +makedepends=(
> +	base-devel
> +	bc
> +	cpio
> +	gettext
> +	libelf
> +	openssl
> +	pahole
> +	perl
> +	python
> +	rsync
> +	tar
> +)
> +options=(!debug !strip !buildflags !makeflags)
> +
> +build() {
> +	# MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
> +	# Bypass this override with a custom variable.
> +	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +	cd "${objtree}"
> +
> +	# makepkg does a "chmod a-srw", triggering warnings during kbuild
> +	chmod 0755 "${pkgdirbase}" || true
> +
> +	${MAKE}
> +}
> +
> +package_linux-upstream() {
> +	pkgdesc="The ${pkgdesc} kernel and modules"
> +
> +	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +	cd "${objtree}"
> +	local modulesdir="${pkgdir}/usr/${MODLIB}"
> +
> +	echo "Installing boot image..."
> +	# systemd expects to find the kernel here to allow hibernation
> +	# https://github.com/systemd/systemd/commit/edda44605f06a41fb86b7ab8128dcf99161d2344
> +	install -Dm644 "$(${MAKE} -s image_name)" "${modulesdir}/vmlinuz"
> +
> +	# Used by mkinitcpio to name the kernel
> +	echo "${pkgbase}" > "${modulesdir}/pkgbase"
> +
> +	echo "Installing modules..."
> +	${MAKE} INSTALL_MOD_PATH="${pkgdir}/usr" INSTALL_MOD_STRIP=1 \
> +		DEPMOD=true modules_install
> +
> +	if [ -d "${srctree}/arch/${SRCARCH}/boot/dts" ]; then

Does this reference to srctree...

> +		echo "Installing dtbs..."
> +		${MAKE} INSTALL_DTBS_PATH="${modulesdir}/dtb" dtbs_install
> +	fi
> +
> +	# remove build link, will be part of -headers package
> +	rm -f "${modulesdir}/build"
> +}
> +
> +package_linux-upstream-headers() {
> +	pkgdesc="Headers and scripts for building modules for the ${pkgdesc} kernel"
> +
> +	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +	cd "${objtree}"
> +	local builddir="${pkgdir}/usr/${MODLIB}/build"
> +
> +	echo "Installing build files..."
> +	"${srctree}/scripts/package/install-extmod-build" "${builddir}"

and this one still work with srctree no longer being passed to makepkg?

I tried myself but it looks like this version of the patch might be
broken?

$ make -skj"$(nproc)" ARCH=x86_64 CROSS_COMPILE=x86_64-linux- O=/mnt/nvme/tmp/build/linux mrproper defconfig pacman-pkg
find: ‘./pacman/linux-upstream/src/pacman/linux-upstream/pkg’: Permission denied
==> Making package: linux-upstream 6.10.0_rc7_00051_g818e9c998b04_dirty-1 (Tue 16 Jul 2024 06:08:29 PM MST)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
==> Extracting sources...
==> Starting build()...
chmod: cannot access 'pacman/linux-upstream/pkg': No such file or directory
find: ‘./pacman/linux-upstream/src/pacman/linux-upstream/pkg’: Permission denied
==> Entering fakeroot environment...
chmod: cannot access '/mnt/nvme/tmp/build/linux/pacman/linux-upstream/pkg': No such file or directory
/usr/bin/fakeroot: line 178: 2633185 User defined signal 1   FAKEROOTKEY=$FAKEROOTKEY LD_LIBRARY_PATH="$PATHS" LD_PRELOAD="$FAKEROOT_LIB" "$@"
make[4]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138

Cheers,
Nathan

> +	echo "Installing System.map and config..."
> +	cp System.map "${builddir}/System.map"
> +	cp .config "${builddir}/.config"
> +
> +	echo "Adding symlink..."
> +	mkdir -p "${pkgdir}/usr/src"
> +	ln -sr "${builddir}" "${pkgdir}/usr/src/${pkgbase}"
> +}
> +
> +package_linux-upstream-api-headers() {
> +	pkgdesc="Kernel headers sanitized for use in userspace"
> +	provides=(linux-api-headers)
> +	conflicts=(linux-api-headers)
> +
> +	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +	cd "${objtree}"
> +
> +	${MAKE} headers_install INSTALL_HDR_PATH="${pkgdir}/usr"
> +}
> 
> ---
> base-commit: 818e9c998b04d6c69a510d5255a93d0e3b8d4993
> change-id: 20240625-kbuild-pacman-pkg-b4f87e19d036
> 
> Best regards,
> -- 
> Thomas Weißschuh <[e-mail]>
>

Masahiro Yamada July 17, 2024, 8:34 a.m. UTC | #2

On Wed, Jul 17, 2024 at 10:15 AM Nathan Chancellor <[e-mail]> wrote:
>
> Hi Thomas,
>
> On Tue, Jul 16, 2024 at 07:52:14PM +0200, Thomas Weißschuh wrote:
> > pacman is the package manager used by Arch Linux and its derivates.
> > Creating native packages from the kernel tree has multiple advantages:
> >
> > * The package triggers the correct hooks for initramfs generation and
> >   bootloader configuration
> > * Uninstallation is complete and also invokes the relevant hooks
> > * New UAPI headers can be installed without any manual bookkeeping
> >
> > The PKGBUILD file is a simplified version of the one used for the
> > downstream Arch Linux "linux" package.
> > Extra steps that should not be necessary for a development kernel have
> > been removed and an UAPI header package has been added.
> >
> > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > Reviewed-by: Nathan Chancellor <[e-mail]>
> > Tested-by: Nathan Chancellor <[e-mail]>
> > Reviewed-by: Nicolas Schier <[e-mail]>
> > ---
> > Changes in v6:
> > - Drop reference to srctree/Makefile
> > - Drop $(realpath $(srctree))
> > - Make use of the fact that $(objtree) is always "."
> > - Align coding style to kernel and drop vim config line
> > - Drop indirection through `$MAKE run-command`
> > - Unify shell variable syntax to "${var}"
> > - Add explanations to custom variables
> > - Add makedepends
> > - Link to v5: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v5:
> > - Rebase onto kbuild/for-next
> > - Use new path to build-version script (from kbuild/for-next)
> > - Ensure submake jobserver delegation works
> > - Simplify $modulesdir/pkgbase file creation
> > - Add Reviewed-by from Nicolas
> > - Link to v4: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v4:
> > - Update MRPROPER_FILES
> > - Unify shell variable syntax
> > - Link to v3: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v3:
> > - Enforce matching architectures for installation
> > - Add Reviewed-by and Tested-by from Nathan
> > - Link to v2: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v2:
> > - Replace ${MAKE} with $MAKE for consistency with other variables
> > - Use $MAKE for "-s image_name"
> > - Avoid permission warnings from build directory
> > - Clarify reason for /build symlink removal
> > - Install System.map and config
> > - Install dtbs where available
> > - Allow cross-build through arch=any
> > - Sort Contributor/Maintainer chronologically
> > - Disable some unneeded makepkg options
> > - Use DEPMOD=true for consistency with rpm-package
> > - Link to v1: https://lore.kernel.org/r/[e-mail]
> > ---
> >  .gitignore               |  6 +++
> >  Makefile                 |  2 +-
> >  scripts/Makefile.package | 14 +++++++
> >  scripts/package/PKGBUILD | 99 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  4 files changed, 120 insertions(+), 1 deletion(-)
> >
> > diff --git a/.gitignore b/.gitignore
> > index c59dc60ba62e..7902adf4f7f1 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -92,6 +92,12 @@ modules.order
> >  #
> >  /tar-install/
> >
> > +#
> > +# pacman files (make pacman-pkg)
> > +#
> > +/PKGBUILD
> > +/pacman/
> > +
> >  #
> >  # We don't want to ignore the following even if they are dot-files
> >  #
> > diff --git a/Makefile b/Makefile
> > index 7372ea45ed3f..768d3dc107f8 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -1481,7 +1481,7 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
> >  # Directories & files removed with 'make mrproper'
> >  MRPROPER_FILES += include/config include/generated          \
> >                 arch/$(SRCARCH)/include/generated .objdiff \
> > -               debian snap tar-install \
> > +               debian snap tar-install PKGBUILD pacman \
> >                 .config .config.old .version \
> >                 Module.symvers \
> >                 certs/signing_key.pem \
> > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > index bf016af8bf8a..0aaa0832279c 100644
> > --- a/scripts/Makefile.package
> > +++ b/scripts/Makefile.package
> > @@ -141,6 +141,19 @@ snap-pkg:
> >       cd $(objtree)/snap && \
> >       snapcraft --target-arch=$(UTS_MACHINE)
> >
> > +# pacman-pkg
> > +# ---------------------------------------------------------------------------
> > +
> > +PHONY += pacman-pkg
> > +pacman-pkg:
> > +     @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> > +     +objtree="$(realpath $(objtree))" \
> > +             BUILDDIR=pacman \
> > +             CARCH="$(UTS_MACHINE)" \
> > +             KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> > +             KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > +             makepkg
> > +
> >  # dir-pkg tar*-pkg - tarball targets
> >  # ---------------------------------------------------------------------------
> >
> > @@ -221,6 +234,7 @@ help:
> >       @echo '  bindeb-pkg          - Build only the binary kernel deb package'
> >       @echo '  snap-pkg            - Build only the binary kernel snap package'
> >       @echo '                        (will connect to external hosts)'
> > +     @echo '  pacman-pkg          - Build only the binary kernel pacman package'
> >       @echo '  dir-pkg             - Build the kernel as a plain directory structure'
> >       @echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
> >       @echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
> > diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> > new file mode 100644
> > index 000000000000..eb3957fad915
> > --- /dev/null
> > +++ b/scripts/package/PKGBUILD
> > @@ -0,0 +1,99 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +# Maintainer: Thomas Weißschuh <[e-mail]>
> > +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> > +
> > +pkgbase=linux-upstream
> > +pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> > +pkgver="${KERNELRELEASE//-/_}"
> > +# The PKGBUILD is evaluated multiple times.
> > +# Running scripts/build-version from here would introduce inconsistencies.
> > +pkgrel="${KBUILD_REVISION}"
> > +pkgdesc='Linux'
> > +url='https://www.kernel.org/'
> > +# Enable flexible cross-compilation
> > +arch=(${CARCH})
> > +license=(GPL-2.0-only)
> > +makedepends=(
> > +     base-devel
> > +     bc
> > +     cpio
> > +     gettext
> > +     libelf
> > +     openssl
> > +     pahole
> > +     perl
> > +     python
> > +     rsync
> > +     tar
> > +)
> > +options=(!debug !strip !buildflags !makeflags)
> > +
> > +build() {
> > +     # MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
> > +     # Bypass this override with a custom variable.
> > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +     cd "${objtree}"
> > +
> > +     # makepkg does a "chmod a-srw", triggering warnings during kbuild
> > +     chmod 0755 "${pkgdirbase}" || true
> > +
> > +     ${MAKE}
> > +}
> > +
> > +package_linux-upstream() {
> > +     pkgdesc="The ${pkgdesc} kernel and modules"
> > +
> > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +     cd "${objtree}"
> > +     local modulesdir="${pkgdir}/usr/${MODLIB}"
> > +
> > +     echo "Installing boot image..."
> > +     # systemd expects to find the kernel here to allow hibernation
> > +     # https://github.com/systemd/systemd/commit/edda44605f06a41fb86b7ab8128dcf99161d2344
> > +     install -Dm644 "$(${MAKE} -s image_name)" "${modulesdir}/vmlinuz"
> > +
> > +     # Used by mkinitcpio to name the kernel
> > +     echo "${pkgbase}" > "${modulesdir}/pkgbase"
> > +
> > +     echo "Installing modules..."
> > +     ${MAKE} INSTALL_MOD_PATH="${pkgdir}/usr" INSTALL_MOD_STRIP=1 \
> > +             DEPMOD=true modules_install
> > +
> > +     if [ -d "${srctree}/arch/${SRCARCH}/boot/dts" ]; then
>
> Does this reference to srctree...


'srctree' is exported by the top Makefile.


This is based on the assumption that we always run
'make pacman-pkg', and we never do 'makepkg' directly.


Do you mean PKGBUILD should be self-contained
so that 'makepkg' works from the command line?



Debian and RPM work that way because we can
build a source package, then we can run
dpkg-buildpackage / rpmbuild later.



>
> > +             echo "Installing dtbs..."
> > +             ${MAKE} INSTALL_DTBS_PATH="${modulesdir}/dtb" dtbs_install
> > +     fi
> > +
> > +     # remove build link, will be part of -headers package
> > +     rm -f "${modulesdir}/build"
> > +}
> > +
> > +package_linux-upstream-headers() {
> > +     pkgdesc="Headers and scripts for building modules for the ${pkgdesc} kernel"
> > +
> > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +     cd "${objtree}"
> > +     local builddir="${pkgdir}/usr/${MODLIB}/build"
> > +
> > +     echo "Installing build files..."
> > +     "${srctree}/scripts/package/install-extmod-build" "${builddir}"
>
> and this one still work with srctree no longer being passed to makepkg?
>
> I tried myself but it looks like this version of the patch might be
> broken?
>
> $ make -skj"$(nproc)" ARCH=x86_64 CROSS_COMPILE=x86_64-linux- O=/mnt/nvme/tmp/build/linux mrproper defconfig pacman-pkg
> find: ‘./pacman/linux-upstream/src/pacman/linux-upstream/pkg’: Permission denied
> ==> Making package: linux-upstream 6.10.0_rc7_00051_g818e9c998b04_dirty-1 (Tue 16 Jul 2024 06:08:29 PM MST)
> ==> Checking runtime dependencies...
> ==> Checking buildtime dependencies...
> ==> Retrieving sources...
> ==> Extracting sources...
> ==> Starting build()...
> chmod: cannot access 'pacman/linux-upstream/pkg': No such file or directory
> find: ‘./pacman/linux-upstream/src/pacman/linux-upstream/pkg’: Permission denied
> ==> Entering fakeroot environment...
> chmod: cannot access '/mnt/nvme/tmp/build/linux/pacman/linux-upstream/pkg': No such file or directory
> /usr/bin/fakeroot: line 178: 2633185 User defined signal 1   FAKEROOTKEY=$FAKEROOTKEY LD_LIBRARY_PATH="$PATHS" LD_PRELOAD="$FAKEROOT_LIB" "$@"
> make[4]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138


Sorry, this is due to my suggestion, BUILDDIR=pacman.


Unfortunately, makepkg is tripped up by a relative path.

The 'pkg' directory is nested under the 'src' directory.

  pacman/linux-upstream/src/pacman/linux-upstream/pkg













> Cheers,
> Nathan
>
> > +     echo "Installing System.map and config..."
> > +     cp System.map "${builddir}/System.map"
> > +     cp .config "${builddir}/.config"
> > +
> > +     echo "Adding symlink..."
> > +     mkdir -p "${pkgdir}/usr/src"
> > +     ln -sr "${builddir}" "${pkgdir}/usr/src/${pkgbase}"
> > +}
> > +
> > +package_linux-upstream-api-headers() {
> > +     pkgdesc="Kernel headers sanitized for use in userspace"
> > +     provides=(linux-api-headers)
> > +     conflicts=(linux-api-headers)
> > +
> > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +     cd "${objtree}"
> > +
> > +     ${MAKE} headers_install INSTALL_HDR_PATH="${pkgdir}/usr"
> > +}
> >
> > ---
> > base-commit: 818e9c998b04d6c69a510d5255a93d0e3b8d4993
> > change-id: 20240625-kbuild-pacman-pkg-b4f87e19d036
> >
> > Best regards,
> > --
> > Thomas Weißschuh <[e-mail]>
> >
>

Masahiro Yamada July 17, 2024, 8:51 a.m. UTC | #3

On Wed, Jul 17, 2024 at 5:34 PM Masahiro Yamada <[e-mail]> wrote:
>
> On Wed, Jul 17, 2024 at 10:15 AM Nathan Chancellor <[e-mail]> wrote:
> >
> > Hi Thomas,
> >
> > On Tue, Jul 16, 2024 at 07:52:14PM +0200, Thomas Weißschuh wrote:
> > > pacman is the package manager used by Arch Linux and its derivates.
> > > Creating native packages from the kernel tree has multiple advantages:
> > >
> > > * The package triggers the correct hooks for initramfs generation and
> > >   bootloader configuration
> > > * Uninstallation is complete and also invokes the relevant hooks
> > > * New UAPI headers can be installed without any manual bookkeeping
> > >
> > > The PKGBUILD file is a simplified version of the one used for the
> > > downstream Arch Linux "linux" package.
> > > Extra steps that should not be necessary for a development kernel have
> > > been removed and an UAPI header package has been added.
> > >
> > > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > > Reviewed-by: Nathan Chancellor <[e-mail]>
> > > Tested-by: Nathan Chancellor <[e-mail]>
> > > Reviewed-by: Nicolas Schier <[e-mail]>
> > > ---
> > > Changes in v6:
> > > - Drop reference to srctree/Makefile
> > > - Drop $(realpath $(srctree))
> > > - Make use of the fact that $(objtree) is always "."
> > > - Align coding style to kernel and drop vim config line
> > > - Drop indirection through `$MAKE run-command`
> > > - Unify shell variable syntax to "${var}"
> > > - Add explanations to custom variables
> > > - Add makedepends
> > > - Link to v5: https://lore.kernel.org/r/[e-mail]
> > >
> > > Changes in v5:
> > > - Rebase onto kbuild/for-next
> > > - Use new path to build-version script (from kbuild/for-next)
> > > - Ensure submake jobserver delegation works
> > > - Simplify $modulesdir/pkgbase file creation
> > > - Add Reviewed-by from Nicolas
> > > - Link to v4: https://lore.kernel.org/r/[e-mail]
> > >
> > > Changes in v4:
> > > - Update MRPROPER_FILES
> > > - Unify shell variable syntax
> > > - Link to v3: https://lore.kernel.org/r/[e-mail]
> > >
> > > Changes in v3:
> > > - Enforce matching architectures for installation
> > > - Add Reviewed-by and Tested-by from Nathan
> > > - Link to v2: https://lore.kernel.org/r/[e-mail]
> > >
> > > Changes in v2:
> > > - Replace ${MAKE} with $MAKE for consistency with other variables
> > > - Use $MAKE for "-s image_name"
> > > - Avoid permission warnings from build directory
> > > - Clarify reason for /build symlink removal
> > > - Install System.map and config
> > > - Install dtbs where available
> > > - Allow cross-build through arch=any
> > > - Sort Contributor/Maintainer chronologically
> > > - Disable some unneeded makepkg options
> > > - Use DEPMOD=true for consistency with rpm-package
> > > - Link to v1: https://lore.kernel.org/r/[e-mail]
> > > ---
> > >  .gitignore               |  6 +++
> > >  Makefile                 |  2 +-
> > >  scripts/Makefile.package | 14 +++++++
> > >  scripts/package/PKGBUILD | 99 ++++++++++++++++++++++++++++++++++++++++++++++++
> > >  4 files changed, 120 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/.gitignore b/.gitignore
> > > index c59dc60ba62e..7902adf4f7f1 100644
> > > --- a/.gitignore
> > > +++ b/.gitignore
> > > @@ -92,6 +92,12 @@ modules.order
> > >  #
> > >  /tar-install/
> > >
> > > +#
> > > +# pacman files (make pacman-pkg)
> > > +#
> > > +/PKGBUILD
> > > +/pacman/
> > > +
> > >  #
> > >  # We don't want to ignore the following even if they are dot-files
> > >  #
> > > diff --git a/Makefile b/Makefile
> > > index 7372ea45ed3f..768d3dc107f8 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -1481,7 +1481,7 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
> > >  # Directories & files removed with 'make mrproper'
> > >  MRPROPER_FILES += include/config include/generated          \
> > >                 arch/$(SRCARCH)/include/generated .objdiff \
> > > -               debian snap tar-install \
> > > +               debian snap tar-install PKGBUILD pacman \
> > >                 .config .config.old .version \
> > >                 Module.symvers \
> > >                 certs/signing_key.pem \
> > > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > > index bf016af8bf8a..0aaa0832279c 100644
> > > --- a/scripts/Makefile.package
> > > +++ b/scripts/Makefile.package
> > > @@ -141,6 +141,19 @@ snap-pkg:
> > >       cd $(objtree)/snap && \
> > >       snapcraft --target-arch=$(UTS_MACHINE)
> > >
> > > +# pacman-pkg
> > > +# ---------------------------------------------------------------------------
> > > +
> > > +PHONY += pacman-pkg
> > > +pacman-pkg:
> > > +     @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> > > +     +objtree="$(realpath $(objtree))" \
> > > +             BUILDDIR=pacman \
> > > +             CARCH="$(UTS_MACHINE)" \
> > > +             KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> > > +             KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > > +             makepkg
> > > +
> > >  # dir-pkg tar*-pkg - tarball targets
> > >  # ---------------------------------------------------------------------------
> > >
> > > @@ -221,6 +234,7 @@ help:
> > >       @echo '  bindeb-pkg          - Build only the binary kernel deb package'
> > >       @echo '  snap-pkg            - Build only the binary kernel snap package'
> > >       @echo '                        (will connect to external hosts)'
> > > +     @echo '  pacman-pkg          - Build only the binary kernel pacman package'
> > >       @echo '  dir-pkg             - Build the kernel as a plain directory structure'
> > >       @echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
> > >       @echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
> > > diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> > > new file mode 100644
> > > index 000000000000..eb3957fad915
> > > --- /dev/null
> > > +++ b/scripts/package/PKGBUILD
> > > @@ -0,0 +1,99 @@
> > > +# SPDX-License-Identifier: GPL-2.0-only
> > > +# Maintainer: Thomas Weißschuh <[e-mail]>
> > > +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> > > +
> > > +pkgbase=linux-upstream
> > > +pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> > > +pkgver="${KERNELRELEASE//-/_}"
> > > +# The PKGBUILD is evaluated multiple times.
> > > +# Running scripts/build-version from here would introduce inconsistencies.
> > > +pkgrel="${KBUILD_REVISION}"
> > > +pkgdesc='Linux'
> > > +url='https://www.kernel.org/'
> > > +# Enable flexible cross-compilation
> > > +arch=(${CARCH})
> > > +license=(GPL-2.0-only)
> > > +makedepends=(
> > > +     base-devel
> > > +     bc
> > > +     cpio
> > > +     gettext
> > > +     libelf
> > > +     openssl
> > > +     pahole
> > > +     perl
> > > +     python
> > > +     rsync
> > > +     tar
> > > +)
> > > +options=(!debug !strip !buildflags !makeflags)
> > > +
> > > +build() {
> > > +     # MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
> > > +     # Bypass this override with a custom variable.
> > > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > > +     cd "${objtree}"
> > > +
> > > +     # makepkg does a "chmod a-srw", triggering warnings during kbuild
> > > +     chmod 0755 "${pkgdirbase}" || true
> > > +
> > > +     ${MAKE}
> > > +}
> > > +
> > > +package_linux-upstream() {
> > > +     pkgdesc="The ${pkgdesc} kernel and modules"
> > > +
> > > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > > +     cd "${objtree}"
> > > +     local modulesdir="${pkgdir}/usr/${MODLIB}"
> > > +
> > > +     echo "Installing boot image..."
> > > +     # systemd expects to find the kernel here to allow hibernation
> > > +     # https://github.com/systemd/systemd/commit/edda44605f06a41fb86b7ab8128dcf99161d2344
> > > +     install -Dm644 "$(${MAKE} -s image_name)" "${modulesdir}/vmlinuz"
> > > +
> > > +     # Used by mkinitcpio to name the kernel
> > > +     echo "${pkgbase}" > "${modulesdir}/pkgbase"
> > > +
> > > +     echo "Installing modules..."
> > > +     ${MAKE} INSTALL_MOD_PATH="${pkgdir}/usr" INSTALL_MOD_STRIP=1 \
> > > +             DEPMOD=true modules_install
> > > +
> > > +     if [ -d "${srctree}/arch/${SRCARCH}/boot/dts" ]; then
> >
> > Does this reference to srctree...
>
>
> 'srctree' is exported by the top Makefile.
>
>
> This is based on the assumption that we always run
> 'make pacman-pkg', and we never do 'makepkg' directly.
>
>
> Do you mean PKGBUILD should be self-contained
> so that 'makepkg' works from the command line?
>


Perhaps, this may make sense.

Currently,
pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
is hard-coded.

I do not think linux-upstream-headers make sense
when CONFIG_MODULE is disabled.



scripts/package/mkspec turns off with_devel
when CONFIG_MODULE is disabled.

Nathan Chancellor July 17, 2024, 1:44 p.m. UTC | #4

On Wed, Jul 17, 2024 at 05:34:09PM +0900, Masahiro Yamada wrote:
> On Wed, Jul 17, 2024 at 10:15 AM Nathan Chancellor <[e-mail]> wrote:
> > On Tue, Jul 16, 2024 at 07:52:14PM +0200, Thomas Weißschuh wrote:
...
> > > +     if [ -d "${srctree}/arch/${SRCARCH}/boot/dts" ]; then
> >
> > Does this reference to srctree...
> 
> 
> 'srctree' is exported by the top Makefile.

Aha, I did not realize that srctree was exported, ignore me then :)

> This is based on the assumption that we always run
> 'make pacman-pkg', and we never do 'makepkg' directly.
> 
> 
> Do you mean PKGBUILD should be self-contained
> so that 'makepkg' works from the command line?

No, just my misunderstanding. I think it is reasonable to tie the
PKGBUILD to the kernel tree in this manner.

Cheers,
Nathan

Christian Heusel July 17, 2024, 1:48 p.m. UTC | #5

On 24/07/16 07:52PM, Thomas Weißschuh wrote:
> pacman is the package manager used by Arch Linux and its derivates.
> Creating native packages from the kernel tree has multiple advantages:
>
> * The package triggers the correct hooks for initramfs generation and
>   bootloader configuration
> * Uninstallation is complete and also invokes the relevant hooks
> * New UAPI headers can be installed without any manual bookkeeping

Additionally, to what already has been mentioned above, this would also
greatly simplify the instructions and the needed setup for bisection if
people run into regressions, as for example in the following documents:

- https://docs.kernel.org/admin-guide/verify-bugs-and-bisect-regressions.html
- https://wiki.archlinux.org/title/Bisecting_bugs_with_Git

So thank you (up front) for your effort's here, this could greatly help
to improve the debugging experience!

I have tested this patch by just running "make pacman-pkg" and
everything worked as expected (currently booted from the generated
kernel).

As noted in another mail in this thread it would also be cool if just
plain makepkg could be executed (i.e. -s/--syncdeps, -i/--install and
--packagelist) would be interesting, but I think with the current setup
this could become a bit complicated 

Nathan Chancellor July 17, 2024, 1:57 p.m. UTC | #6

On Wed, Jul 17, 2024 at 05:51:21PM +0900, Masahiro Yamada wrote:
...
> > > > diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> > > > new file mode 100644
> > > > index 000000000000..eb3957fad915
> > > > --- /dev/null
> > > > +++ b/scripts/package/PKGBUILD
> > > > @@ -0,0 +1,99 @@
> > > > +# SPDX-License-Identifier: GPL-2.0-only
> > > > +# Maintainer: Thomas Weißschuh <[e-mail]>
> > > > +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> > > > +
> > > > +pkgbase=linux-upstream
...
> Perhaps, this may make sense.
> 
> Currently,
> pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> is hard-coded.
> 
> I do not think linux-upstream-headers make sense
> when CONFIG_MODULE is disabled.
> 
> scripts/package/mkspec turns off with_devel
> when CONFIG_MODULE is disabled.

Yes, I think that is a reasonable change to make. In the face of that
potential change, would it make sense to slightly adjust the
makedepends? pahole is only needed when CONFIG_DEBUG_INFO_BTF is enabled
but I guess no other package building infrastructure makes dependencies
conditional in that manner.

Another thing I wonder about would be allowing the user to customize the
value of pkgbase, like

  pkgbase=${PACMAN_PKGBASE:-linux-upstream}

because unlike Debian and Fedora, multiple versions of the same kernel
package cannot be installed at once. If I wanted to build a package
against mainline and -next and install them side by side, I could only
do so if they are named differently. This would allow one to provide
PACMAN_PKGBASE=linux-mainline and PACMAN_PKGBASE=linux-next to
accomplish that. Might be a hyper specific use case though, so I am not
opposed to disregarding it.

Cheers,
Nathan

Thomas Weißschuh July 17, 2024, 4:26 p.m. UTC | #7

On 2024-07-17 15:48:05+0000, Christian Heusel wrote:
> On 24/07/16 07:52PM, Thomas Weißschuh wrote:
> > pacman is the package manager used by Arch Linux and its derivates.
> > Creating native packages from the kernel tree has multiple advantages:
> >
> > * The package triggers the correct hooks for initramfs generation and
> >   bootloader configuration
> > * Uninstallation is complete and also invokes the relevant hooks
> > * New UAPI headers can be installed without any manual bookkeeping
> 
> Additionally, to what already has been mentioned above, this would also
> greatly simplify the instructions and the needed setup for bisection if
> people run into regressions, as for example in the following documents:
> 
> - https://docs.kernel.org/admin-guide/verify-bugs-and-bisect-regressions.html
> - https://wiki.archlinux.org/title/Bisecting_bugs_with_Git
> 
> So thank you (up front) for your effort's here, this could greatly help
> to improve the debugging experience!

Sounds reasonable, but IMO this can be done after the original
implementation is done.

> I have tested this patch by just running "make pacman-pkg" and
> everything worked as expected (currently booted from the generated
> kernel).

Thanks for the test!

> As noted in another mail in this thread it would also be cool if just
> plain makepkg could be executed (i.e. -s/--syncdeps, -i/--install and
> --packagelist) would be interesting, but I think with the current setup
> this could become a bit complicated 

Thomas Weißschuh July 17, 2024, 4:33 p.m. UTC | #8

On 2024-07-17 06:57:47+0000, Nathan Chancellor wrote:
> On Wed, Jul 17, 2024 at 05:51:21PM +0900, Masahiro Yamada wrote:
> ...
> > > > > diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> > > > > new file mode 100644
> > > > > index 000000000000..eb3957fad915
> > > > > --- /dev/null
> > > > > +++ b/scripts/package/PKGBUILD
> > > > > @@ -0,0 +1,99 @@
> > > > > +# SPDX-License-Identifier: GPL-2.0-only
> > > > > +# Maintainer: Thomas Weißschuh <[e-mail]>
> > > > > +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> > > > > +
> > > > > +pkgbase=linux-upstream
> ...
> > Perhaps, this may make sense.
> > 
> > Currently,
> > pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> > is hard-coded.
> > 
> > I do not think linux-upstream-headers make sense
> > when CONFIG_MODULE is disabled.
> > 
> > scripts/package/mkspec turns off with_devel
> > when CONFIG_MODULE is disabled.

Ack.

> Yes, I think that is a reasonable change to make. In the face of that
> potential change, would it make sense to slightly adjust the
> makedepends? pahole is only needed when CONFIG_DEBUG_INFO_BTF is enabled
> but I guess no other package building infrastructure makes dependencies
> conditional in that manner.

The pahole dependency optimization seems like an overcomplication.
Arch Linux in general is not hell-bent on minimizing dependencies,
for example there are no dev-packages at all.
Also if the kernel will require pahole in more cases it will create
churn.

> Another thing I wonder about would be allowing the user to customize the
> value of pkgbase, like
> 
>   pkgbase=${PACMAN_PKGBASE:-linux-upstream}
> 
> because unlike Debian and Fedora, multiple versions of the same kernel
> package cannot be installed at once. If I wanted to build a package
> against mainline and -next and install them side by side, I could only
> do so if they are named differently. This would allow one to provide
> PACMAN_PKGBASE=linux-mainline and PACMAN_PKGBASE=linux-next to
> accomplish that. Might be a hyper specific use case though, so I am not
> opposed to disregarding it.

Ack.

FYI linux-upstream was picked to be consistent with pkg-debian and not
to conflict with the existing linux-mainline package in AUR [0].

[0] https://aur.archlinux.org/packages/linux-mainline

Masahiro Yamada July 17, 2024, 5:18 p.m. UTC | #9

On Thu, Jul 18, 2024 at 1:33 AM Thomas Weißschuh <[e-mail]> wrote:
>
> On 2024-07-17 06:57:47+0000, Nathan Chancellor wrote:
> > On Wed, Jul 17, 2024 at 05:51:21PM +0900, Masahiro Yamada wrote:
> > ...
> > > > > > diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> > > > > > new file mode 100644
> > > > > > index 000000000000..eb3957fad915
> > > > > > --- /dev/null
> > > > > > +++ b/scripts/package/PKGBUILD
> > > > > > @@ -0,0 +1,99 @@
> > > > > > +# SPDX-License-Identifier: GPL-2.0-only
> > > > > > +# Maintainer: Thomas Weißschuh <[e-mail]>
> > > > > > +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> > > > > > +
> > > > > > +pkgbase=linux-upstream
> > ...
> > > Perhaps, this may make sense.
> > >
> > > Currently,
> > > pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> > > is hard-coded.
> > >
> > > I do not think linux-upstream-headers make sense
> > > when CONFIG_MODULE is disabled.
> > >
> > > scripts/package/mkspec turns off with_devel
> > > when CONFIG_MODULE is disabled.
>
> Ack.
>
> > Yes, I think that is a reasonable change to make. In the face of that
> > potential change, would it make sense to slightly adjust the
> > makedepends? pahole is only needed when CONFIG_DEBUG_INFO_BTF is enabled
> > but I guess no other package building infrastructure makes dependencies
> > conditional in that manner.
>
> The pahole dependency optimization seems like an overcomplication.
> Arch Linux in general is not hell-bent on minimizing dependencies,
> for example there are no dev-packages at all.
> Also if the kernel will require pahole in more cases it will create
> churn.
>

FWIW, we conditionally added build-dependency for the deb package
in the past, but I gave up on maintaining it.



See b88365b6d74edc88a9d283c837fec05b13d401a6


--
Best Regards
Masahiro Yamada

Masahiro Yamada July 17, 2024, 6:38 p.m. UTC | #10

On Wed, Jul 17, 2024 at 2:52 AM Thomas Weißschuh <[e-mail]> wrote:
>
> pacman is the package manager used by Arch Linux and its derivates.
> Creating native packages from the kernel tree has multiple advantages:
>
> * The package triggers the correct hooks for initramfs generation and
>   bootloader configuration
> * Uninstallation is complete and also invokes the relevant hooks
> * New UAPI headers can be installed without any manual bookkeeping
>
> The PKGBUILD file is a simplified version of the one used for the
> downstream Arch Linux "linux" package.
> Extra steps that should not be necessary for a development kernel have
> been removed and an UAPI header package has been added.
>
> Signed-off-by: Thomas Weißschuh <[e-mail]>
> Reviewed-by: Nathan Chancellor <[e-mail]>
> Tested-by: Nathan Chancellor <[e-mail]>
> Reviewed-by: Nicolas Schier <[e-mail]>
> ---
> Changes in v6:
> - Drop reference to srctree/Makefile
> - Drop $(realpath $(srctree))
> - Make use of the fact that $(objtree) is always "."
> - Align coding style to kernel and drop vim config line
> - Drop indirection through `$MAKE run-command`
> - Unify shell variable syntax to "${var}"
> - Add explanations to custom variables
> - Add makedepends
> - Link to v5: https://lore.kernel.org/r/[e-mail]
>
> Changes in v5:
> - Rebase onto kbuild/for-next
> - Use new path to build-version script (from kbuild/for-next)
> - Ensure submake jobserver delegation works
> - Simplify $modulesdir/pkgbase file creation
> - Add Reviewed-by from Nicolas
> - Link to v4: https://lore.kernel.org/r/[e-mail]
>
> Changes in v4:
> - Update MRPROPER_FILES
> - Unify shell variable syntax
> - Link to v3: https://lore.kernel.org/r/[e-mail]
>
> Changes in v3:
> - Enforce matching architectures for installation
> - Add Reviewed-by and Tested-by from Nathan
> - Link to v2: https://lore.kernel.org/r/[e-mail]
>
> Changes in v2:
> - Replace ${MAKE} with $MAKE for consistency with other variables
> - Use $MAKE for "-s image_name"
> - Avoid permission warnings from build directory
> - Clarify reason for /build symlink removal
> - Install System.map and config
> - Install dtbs where available
> - Allow cross-build through arch=any
> - Sort Contributor/Maintainer chronologically
> - Disable some unneeded makepkg options
> - Use DEPMOD=true for consistency with rpm-package
> - Link to v1: https://lore.kernel.org/r/[e-mail]
> ---
>  .gitignore               |  6 +++
>  Makefile                 |  2 +-
>  scripts/Makefile.package | 14 +++++++
>  scripts/package/PKGBUILD | 99 ++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 120 insertions(+), 1 deletion(-)
>
> diff --git a/.gitignore b/.gitignore
> index c59dc60ba62e..7902adf4f7f1 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -92,6 +92,12 @@ modules.order
>  #
>  /tar-install/
>
> +#
> +# pacman files (make pacman-pkg)
> +#
> +/PKGBUILD
> +/pacman/
> +
>  #
>  # We don't want to ignore the following even if they are dot-files
>  #
> diff --git a/Makefile b/Makefile
> index 7372ea45ed3f..768d3dc107f8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1481,7 +1481,7 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
>  # Directories & files removed with 'make mrproper'
>  MRPROPER_FILES += include/config include/generated          \
>                   arch/$(SRCARCH)/include/generated .objdiff \
> -                 debian snap tar-install \
> +                 debian snap tar-install PKGBUILD pacman \
>                   .config .config.old .version \
>                   Module.symvers \
>                   certs/signing_key.pem \
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index bf016af8bf8a..0aaa0832279c 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -141,6 +141,19 @@ snap-pkg:
>         cd $(objtree)/snap && \
>         snapcraft --target-arch=$(UTS_MACHINE)
>
> +# pacman-pkg
> +# ---------------------------------------------------------------------------
> +
> +PHONY += pacman-pkg
> +pacman-pkg:
> +       @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> +       +objtree="$(realpath $(objtree))" \
> +               BUILDDIR=pacman \
> +               CARCH="$(UTS_MACHINE)" \
> +               KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> +               KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> +               makepkg
> +
>  # dir-pkg tar*-pkg - tarball targets
>  # ---------------------------------------------------------------------------
>
> @@ -221,6 +234,7 @@ help:
>         @echo '  bindeb-pkg          - Build only the binary kernel deb package'
>         @echo '  snap-pkg            - Build only the binary kernel snap package'
>         @echo '                        (will connect to external hosts)'
> +       @echo '  pacman-pkg          - Build only the binary kernel pacman package'
>         @echo '  dir-pkg             - Build the kernel as a plain directory structure'
>         @echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
>         @echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
> diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> new file mode 100644
> index 000000000000..eb3957fad915
> --- /dev/null
> +++ b/scripts/package/PKGBUILD
> @@ -0,0 +1,99 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +# Maintainer: Thomas Weißschuh <[e-mail]>
> +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> +
> +pkgbase=linux-upstream
> +pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> +pkgver="${KERNELRELEASE//-/_}"
> +# The PKGBUILD is evaluated multiple times.
> +# Running scripts/build-version from here would introduce inconsistencies.
> +pkgrel="${KBUILD_REVISION}"








> +pkgdesc='Linux'
> +url='https://www.kernel.org/'
> +# Enable flexible cross-compilation
> +arch=(${CARCH})
> +license=(GPL-2.0-only)
> +makedepends=(
> +       base-devel


The base-devel group includes autoconf, automake, libtool, etc.

Kbuild does not use those.


I like a list of individual packages, as seen in arch linux:

https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/blob/6.9.9.arch1-1/PKGBUILD?ref_type=tags#L11






> +       bc
> +       cpio
> +       gettext
> +       libelf
> +       openssl
> +       pahole
> +       perl
> +       python
> +       rsync
> +       tar
> +)
> +options=(!debug !strip !buildflags !makeflags)
> +
> +build() {
> +       # MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
> +       # Bypass this override with a custom variable.
> +       export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +       cd "${objtree}"
> +
> +       # makepkg does a "chmod a-srw", triggering warnings during kbuild
> +       chmod 0755 "${pkgdirbase}" || true


Please remove this.

The warning is emitted by

  find . -name '*.usyms' | xargs rm -f

in scripts/remove-stale-files.


I will apply this first:
https://lore.kernel.org/linux-kbuild/[e-mail]/T/#u






> +
> +       ${MAKE}


This will cause a revision mismatch between the package and
'uname -a' in the installed kernel image.

You execute scripts/build-version in scripts/Makefile.package,
and once again during ${MAKE}.


The revision in include/generated/utsversion.h is bigger
than ${pkgrel}.


kernel.spec does like this:

  %{make} %{makeflags} KERNELRELEASE=%{KERNELRELEASE}
KBUILD_BUILD_VERSION=%{release}



You need to do something similar.





--
Best Regards
Masahiro Yamada

Thomas Weißschuh July 17, 2024, 7:27 p.m. UTC | #11

On 2024-07-18 03:38:17+0000, Masahiro Yamada wrote:
> On Wed, Jul 17, 2024 at 2:52 AM Thomas Weißschuh <[e-mail]> wrote:
> >
> > pacman is the package manager used by Arch Linux and its derivates.
> > Creating native packages from the kernel tree has multiple advantages:
> >
> > * The package triggers the correct hooks for initramfs generation and
> >   bootloader configuration
> > * Uninstallation is complete and also invokes the relevant hooks
> > * New UAPI headers can be installed without any manual bookkeeping
> >
> > The PKGBUILD file is a simplified version of the one used for the
> > downstream Arch Linux "linux" package.
> > Extra steps that should not be necessary for a development kernel have
> > been removed and an UAPI header package has been added.
> >
> > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > Reviewed-by: Nathan Chancellor <[e-mail]>
> > Tested-by: Nathan Chancellor <[e-mail]>
> > Reviewed-by: Nicolas Schier <[e-mail]>
> > ---
> > Changes in v6:
> > - Drop reference to srctree/Makefile
> > - Drop $(realpath $(srctree))
> > - Make use of the fact that $(objtree) is always "."
> > - Align coding style to kernel and drop vim config line
> > - Drop indirection through `$MAKE run-command`
> > - Unify shell variable syntax to "${var}"
> > - Add explanations to custom variables
> > - Add makedepends
> > - Link to v5: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v5:
> > - Rebase onto kbuild/for-next
> > - Use new path to build-version script (from kbuild/for-next)
> > - Ensure submake jobserver delegation works
> > - Simplify $modulesdir/pkgbase file creation
> > - Add Reviewed-by from Nicolas
> > - Link to v4: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v4:
> > - Update MRPROPER_FILES
> > - Unify shell variable syntax
> > - Link to v3: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v3:
> > - Enforce matching architectures for installation
> > - Add Reviewed-by and Tested-by from Nathan
> > - Link to v2: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v2:
> > - Replace ${MAKE} with $MAKE for consistency with other variables
> > - Use $MAKE for "-s image_name"
> > - Avoid permission warnings from build directory
> > - Clarify reason for /build symlink removal
> > - Install System.map and config
> > - Install dtbs where available
> > - Allow cross-build through arch=any
> > - Sort Contributor/Maintainer chronologically
> > - Disable some unneeded makepkg options
> > - Use DEPMOD=true for consistency with rpm-package
> > - Link to v1: https://lore.kernel.org/r/[e-mail]
> > ---
> >  .gitignore               |  6 +++
> >  Makefile                 |  2 +-
> >  scripts/Makefile.package | 14 +++++++
> >  scripts/package/PKGBUILD | 99 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  4 files changed, 120 insertions(+), 1 deletion(-)
> >
> > diff --git a/.gitignore b/.gitignore
> > index c59dc60ba62e..7902adf4f7f1 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -92,6 +92,12 @@ modules.order
> >  #
> >  /tar-install/
> >
> > +#
> > +# pacman files (make pacman-pkg)
> > +#
> > +/PKGBUILD
> > +/pacman/
> > +
> >  #
> >  # We don't want to ignore the following even if they are dot-files
> >  #
> > diff --git a/Makefile b/Makefile
> > index 7372ea45ed3f..768d3dc107f8 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -1481,7 +1481,7 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
> >  # Directories & files removed with 'make mrproper'
> >  MRPROPER_FILES += include/config include/generated          \
> >                   arch/$(SRCARCH)/include/generated .objdiff \
> > -                 debian snap tar-install \
> > +                 debian snap tar-install PKGBUILD pacman \
> >                   .config .config.old .version \
> >                   Module.symvers \
> >                   certs/signing_key.pem \
> > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > index bf016af8bf8a..0aaa0832279c 100644
> > --- a/scripts/Makefile.package
> > +++ b/scripts/Makefile.package
> > @@ -141,6 +141,19 @@ snap-pkg:
> >         cd $(objtree)/snap && \
> >         snapcraft --target-arch=$(UTS_MACHINE)
> >
> > +# pacman-pkg
> > +# ---------------------------------------------------------------------------
> > +
> > +PHONY += pacman-pkg
> > +pacman-pkg:
> > +       @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> > +       +objtree="$(realpath $(objtree))" \
> > +               BUILDDIR=pacman \
> > +               CARCH="$(UTS_MACHINE)" \
> > +               KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> > +               KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > +               makepkg
> > +
> >  # dir-pkg tar*-pkg - tarball targets
> >  # ---------------------------------------------------------------------------
> >
> > @@ -221,6 +234,7 @@ help:
> >         @echo '  bindeb-pkg          - Build only the binary kernel deb package'
> >         @echo '  snap-pkg            - Build only the binary kernel snap package'
> >         @echo '                        (will connect to external hosts)'
> > +       @echo '  pacman-pkg          - Build only the binary kernel pacman package'
> >         @echo '  dir-pkg             - Build the kernel as a plain directory structure'
> >         @echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
> >         @echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
> > diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> > new file mode 100644
> > index 000000000000..eb3957fad915
> > --- /dev/null
> > +++ b/scripts/package/PKGBUILD
> > @@ -0,0 +1,99 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +# Maintainer: Thomas Weißschuh <[e-mail]>
> > +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> > +
> > +pkgbase=linux-upstream
> > +pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> > +pkgver="${KERNELRELEASE//-/_}"
> > +# The PKGBUILD is evaluated multiple times.
> > +# Running scripts/build-version from here would introduce inconsistencies.
> > +pkgrel="${KBUILD_REVISION}"
> 
> > +pkgdesc='Linux'
> > +url='https://www.kernel.org/'
> > +# Enable flexible cross-compilation
> > +arch=(${CARCH})
> > +license=(GPL-2.0-only)
> > +makedepends=(
> > +       base-devel
> 
> 
> The base-devel group includes autoconf, automake, libtool, etc.
> 
> Kbuild does not use those.
> 
> 
> I like a list of individual packages, as seen in arch linux:
> 
> https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/blob/6.9.9.arch1-1/PKGBUILD?ref_type=tags#L11

From the wiki:

Note:
    The package base-devel is assumed to be already installed when
    building with makepkg. Dependencies of this package should not be
    included in makedepends array.

https://wiki.archlinux.org/title/PKGBUILD

Theoretically this shouldn't even be needed in the makedepends array.
I added it so if users are not aware that makepkg is used under the hood
these basic dependencies are met.
So the linked PKGBUILD assumes that all of this is already installed.

But I can see why you dislike it and will provide a more accurate list
of dependencies.

> > +       bc
> > +       cpio
> > +       gettext
> > +       libelf
> > +       openssl
> > +       pahole
> > +       perl
> > +       python
> > +       rsync
> > +       tar
> > +)
> > +options=(!debug !strip !buildflags !makeflags)
> > +
> > +build() {
> > +       # MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
> > +       # Bypass this override with a custom variable.
> > +       export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +       cd "${objtree}"
> > +
> > +       # makepkg does a "chmod a-srw", triggering warnings during kbuild
> > +       chmod 0755 "${pkgdirbase}" || true
> 
> 
> Please remove this.
> 
> The warning is emitted by
> 
>   find . -name '*.usyms' | xargs rm -f
> 
> in scripts/remove-stale-files.
> 
> 
> I will apply this first:
> https://lore.kernel.org/linux-kbuild/[e-mail]/T/#u

Thank you very much for the analysis and fix!

> > +
> > +       ${MAKE}
> 
> 
> This will cause a revision mismatch between the package and
> 'uname -a' in the installed kernel image.
> 
> You execute scripts/build-version in scripts/Makefile.package,
> and once again during ${MAKE}.
> 
> 
> The revision in include/generated/utsversion.h is bigger
> than ${pkgrel}.
> 
> 
> kernel.spec does like this:
> 
>   %{make} %{makeflags} KERNELRELEASE=%{KERNELRELEASE}
> KBUILD_BUILD_VERSION=%{release}
> 
> 
> 
> You need to do something similar.

Ack, thanks!

Nathan Chancellor July 17, 2024, 9:13 p.m. UTC | #12

On Wed, Jul 17, 2024 at 06:26:29PM +0200, Thomas Weißschuh wrote:
> On 2024-07-17 15:48:05+0000, Christian Heusel wrote:
> > Also is there a canonical way to be notified whenever people report
> > issues with this or there are patches to it? I'd love to help out if
> > that is desirable from your side ..
> 
> I've added you to my Cc list.

Another option would be adding scripts/package/PKGBUILD in a separate
section of MAINTAINERS, so that Christian could be added as a reviewer.
FWIW, I would also be interested in patches touching this (although I
should get notified as is since I am a Kbuild reviewer but still).

Cheers,
Nathan

Patch

diff --git a/.gitignore b/.gitignore
index c59dc60ba62e..7902adf4f7f1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -92,6 +92,12 @@  modules.order
 #
 /tar-install/
 
+#
+# pacman files (make pacman-pkg)
+#
+/PKGBUILD
+/pacman/
+
 #
 # We don't want to ignore the following even if they are dot-files
 #
diff --git a/Makefile b/Makefile
index 7372ea45ed3f..768d3dc107f8 100644
--- a/Makefile
+++ b/Makefile
@@ -1481,7 +1481,7 @@  CLEAN_FILES += vmlinux.symvers modules-only.symvers \
 # Directories & files removed with 'make mrproper'
 MRPROPER_FILES += include/config include/generated          \
 		  arch/$(SRCARCH)/include/generated .objdiff \
-		  debian snap tar-install \
+		  debian snap tar-install PKGBUILD pacman \
 		  .config .config.old .version \
 		  Module.symvers \
 		  certs/signing_key.pem \
diff --git a/scripts/Makefile.package b/scripts/Makefile.package
index bf016af8bf8a..0aaa0832279c 100644
--- a/scripts/Makefile.package
+++ b/scripts/Makefile.package
@@ -141,6 +141,19 @@  snap-pkg:
 	cd $(objtree)/snap && \
 	snapcraft --target-arch=$(UTS_MACHINE)
 
+# pacman-pkg
+# ---------------------------------------------------------------------------
+
+PHONY += pacman-pkg
+pacman-pkg:
+	@ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
+	+objtree="$(realpath $(objtree))" \
+		BUILDDIR=pacman \
+		CARCH="$(UTS_MACHINE)" \
+		KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
+		KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
+		makepkg
+
 # dir-pkg tar*-pkg - tarball targets
 # ---------------------------------------------------------------------------
 
@@ -221,6 +234,7 @@  help:
 	@echo '  bindeb-pkg          - Build only the binary kernel deb package'
 	@echo '  snap-pkg            - Build only the binary kernel snap package'
 	@echo '                        (will connect to external hosts)'
+	@echo '  pacman-pkg          - Build only the binary kernel pacman package'
 	@echo '  dir-pkg             - Build the kernel as a plain directory structure'
 	@echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
 	@echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
new file mode 100644
index 000000000000..eb3957fad915
--- /dev/null
+++ b/scripts/package/PKGBUILD
@@ -0,0 +1,99 @@ 
+# SPDX-License-Identifier: GPL-2.0-only
+# Maintainer: Thomas Weißschuh <[e-mail]>
+# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
+
+pkgbase=linux-upstream
+pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
+pkgver="${KERNELRELEASE//-/_}"
+# The PKGBUILD is evaluated multiple times.
+# Running scripts/build-version from here would introduce inconsistencies.
+pkgrel="${KBUILD_REVISION}"
+pkgdesc='Linux'
+url='https://www.kernel.org/'
+# Enable flexible cross-compilation
+arch=(${CARCH})
+license=(GPL-2.0-only)
+makedepends=(
+	base-devel
+	bc
+	cpio
+	gettext
+	libelf
+	openssl
+	pahole
+	perl
+	python
+	rsync
+	tar
+)
+options=(!debug !strip !buildflags !makeflags)
+
+build() {
+	# MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
+	# Bypass this override with a custom variable.
+	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
+	cd "${objtree}"
+
+	# makepkg does a "chmod a-srw", triggering warnings during kbuild
+	chmod 0755 "${pkgdirbase}" || true
+
+	${MAKE}
+}
+
+package_linux-upstream() {
+	pkgdesc="The ${pkgdesc} kernel and modules"
+
+	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
+	cd "${objtree}"
+	local modulesdir="${pkgdir}/usr/${MODLIB}"
+
+	echo "Installing boot image..."
+	# systemd expects to find the kernel here to allow hibernation
+	# https://github.com/systemd/systemd/commit/edda44605f06a41fb86b7ab8128dcf99161d2344
+	install -Dm644 "$(${MAKE} -s image_name)" "${modulesdir}/vmlinuz"
+
+	# Used by mkinitcpio to name the kernel
+	echo "${pkgbase}" > "${modulesdir}/pkgbase"
+
+	echo "Installing modules..."
+	${MAKE} INSTALL_MOD_PATH="${pkgdir}/usr" INSTALL_MOD_STRIP=1 \
+		DEPMOD=true modules_install
+
+	if [ -d "${srctree}/arch/${SRCARCH}/boot/dts" ]; then
+		echo "Installing dtbs..."
+		${MAKE} INSTALL_DTBS_PATH="${modulesdir}/dtb" dtbs_install
+	fi
+
+	# remove build link, will be part of -headers package
+	rm -f "${modulesdir}/build"
+}
+
+package_linux-upstream-headers() {
+	pkgdesc="Headers and scripts for building modules for the ${pkgdesc} kernel"
+
+	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
+	cd "${objtree}"
+	local builddir="${pkgdir}/usr/${MODLIB}/build"
+
+	echo "Installing build files..."
+	"${srctree}/scripts/package/install-extmod-build" "${builddir}"
+
+	echo "Installing System.map and config..."
+	cp System.map "${builddir}/System.map"
+	cp .config "${builddir}/.config"
+
+	echo "Adding symlink..."
+	mkdir -p "${pkgdir}/usr/src"
+	ln -sr "${builddir}" "${pkgdir}/usr/src/${pkgbase}"
+}
+
+package_linux-upstream-api-headers() {
+	pkgdesc="Kernel headers sanitized for use in userspace"
+	provides=(linux-api-headers)
+	conflicts=(linux-api-headers)
+
+	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
+	cd "${objtree}"
+
+	${MAKE} headers_install INSTALL_HDR_PATH="${pkgdir}/usr"
+}