| Message ID | 20240807-macos-build-support-v1-0-4cd1ded85694@samsung.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Enable build system on macOS hosts | expand |
On Wed, Aug 07, 2024 at 01:09:14AM +0200, Daniel Gomez via B4 Relay wrote: > This patch set allows for building the Linux kernel for arm64 in macOS with > LLVM. Is this a requirement somewhere that this must work? It seems like an odd request, what workflows require cross-operating-system builds like this? thanks, greg k-h
On Wed, Aug 07, 2024 at 01:09:22AM +0200, Daniel Gomez via B4 Relay wrote: > From: Daniel Gomez <da.gomez@samsung.com> > > Add a copy of elf/elf.h header from the GNU C Library (glibc), version > glibc-2.40 into include/elf. Update Makefiles where elf.h header is used > to ensure the compiler can find all necessary headers, for macOS host > where these headers are not provided by the system. > > Signed-off-by: Daniel Gomez <da.gomez@samsung.com> > --- > arch/arm64/kernel/pi/Makefile | 1 + > arch/arm64/kernel/vdso32/Makefile | 1 + > arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- > include/elf/elf.h | 4491 +++++++++++++++++++++++++++++++++++++ > scripts/Makefile | 3 +- > scripts/mod/Makefile | 6 + > 6 files changed, 4502 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile > index 4d11a8c29181..eb782aaa6585 100644 > --- a/arch/arm64/kernel/pi/Makefile > +++ b/arch/arm64/kernel/pi/Makefile > @@ -20,6 +20,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) > KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS)) > > hostprogs := relacheck > +HOSTCFLAGS_relacheck.o = -I$(srctree)/include/elf > > quiet_cmd_piobjcopy = $(quiet_cmd_objcopy) > cmd_piobjcopy = $(cmd_objcopy) && $(obj)/relacheck $(@) $(<) > diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile > index 25a2cb6317f3..e1ac384e6332 100644 > --- a/arch/arm64/kernel/vdso32/Makefile > +++ b/arch/arm64/kernel/vdso32/Makefile > @@ -107,6 +107,7 @@ VDSO_LDFLAGS += --orphan-handling=$(CONFIG_LD_ORPHAN_WARN_LEVEL) > # $(hostprogs) with $(obj) > munge := ../../../arm/vdso/vdsomunge > hostprogs := $(munge) > +HOSTCFLAGS_$(munge).o = -I$(objtree)/include/elf > > c-obj-vdso := note.o > c-obj-vdso-gettimeofday := vgettimeofday.o > diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile > index 782b34b004be..40541c0812bf 100644 > --- a/arch/arm64/kvm/hyp/nvhe/Makefile > +++ b/arch/arm64/kvm/hyp/nvhe/Makefile > @@ -15,7 +15,7 @@ ccflags-y += -fno-stack-protector \ > $(DISABLE_STACKLEAK_PLUGIN) > > hostprogs := gen-hyprel > -HOST_EXTRACFLAGS += -I$(objtree)/include > +HOST_EXTRACFLAGS += -I$(objtree)/include -I$(srctree)/include/elf > > lib-objs := clear_page.o copy_page.o memcpy.o memset.o > lib-objs := $(addprefix ../../../lib/, $(lib-objs)) > diff --git a/include/elf/elf.h b/include/elf/elf.h > new file mode 100644 > index 000000000000..33aea7f743b8 > --- /dev/null > +++ b/include/elf/elf.h > @@ -0,0 +1,4491 @@ > +/* This file defines standard ELF types, structures, and macros. > + Copyright (C) 1995-2024 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ I understand your want/need for this, but new files need a SPDX license header instead of this type of license boilerplate. Didn't glibc already convert to SPDX? Also, as this is not internal for the kernel, but rather for userspace builds, shouldn't the include/ path be different? thanks, greg k-h
On Wed, Aug 07, 2024 at 01:01:08PM GMT, Greg Kroah-Hartman wrote: > On Wed, Aug 07, 2024 at 01:09:14AM +0200, Daniel Gomez via B4 Relay wrote: > > This patch set allows for building the Linux kernel for arm64 in macOS with > > LLVM. > > Is this a requirement somewhere that this must work? It seems like an > odd request, what workflows require cross-operating-system builds like > this? This isn't a requirement, but it would, for example, support workflows for QEMU users and developers on macOS. They could build/compile the kernel natively and use it to launch QEMU instances, simplifying their process. > > thanks, > > greg k-h
On Wed, Aug 07, 2024 at 01:04:29PM GMT, Greg Kroah-Hartman wrote: > On Wed, Aug 07, 2024 at 01:09:22AM +0200, Daniel Gomez via B4 Relay wrote: > > From: Daniel Gomez <da.gomez@samsung.com> > > > > Add a copy of elf/elf.h header from the GNU C Library (glibc), version > > glibc-2.40 into include/elf. Update Makefiles where elf.h header is used > > to ensure the compiler can find all necessary headers, for macOS host > > where these headers are not provided by the system. > > > > Signed-off-by: Daniel Gomez <da.gomez@samsung.com> > > --- > > arch/arm64/kernel/pi/Makefile | 1 + > > arch/arm64/kernel/vdso32/Makefile | 1 + > > arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- > > include/elf/elf.h | 4491 +++++++++++++++++++++++++++++++++++++ > > scripts/Makefile | 3 +- > > scripts/mod/Makefile | 6 + > > 6 files changed, 4502 insertions(+), 2 deletions(-) > > > > diff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile > > index 4d11a8c29181..eb782aaa6585 100644 > > --- a/arch/arm64/kernel/pi/Makefile > > +++ b/arch/arm64/kernel/pi/Makefile > > @@ -20,6 +20,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) > > KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS)) > > > > hostprogs := relacheck > > +HOSTCFLAGS_relacheck.o = -I$(srctree)/include/elf > > > > quiet_cmd_piobjcopy = $(quiet_cmd_objcopy) > > cmd_piobjcopy = $(cmd_objcopy) && $(obj)/relacheck $(@) $(<) > > diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile > > index 25a2cb6317f3..e1ac384e6332 100644 > > --- a/arch/arm64/kernel/vdso32/Makefile > > +++ b/arch/arm64/kernel/vdso32/Makefile > > @@ -107,6 +107,7 @@ VDSO_LDFLAGS += --orphan-handling=$(CONFIG_LD_ORPHAN_WARN_LEVEL) > > # $(hostprogs) with $(obj) > > munge := ../../../arm/vdso/vdsomunge > > hostprogs := $(munge) > > +HOSTCFLAGS_$(munge).o = -I$(objtree)/include/elf > > > > c-obj-vdso := note.o > > c-obj-vdso-gettimeofday := vgettimeofday.o > > diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile > > index 782b34b004be..40541c0812bf 100644 > > --- a/arch/arm64/kvm/hyp/nvhe/Makefile > > +++ b/arch/arm64/kvm/hyp/nvhe/Makefile > > @@ -15,7 +15,7 @@ ccflags-y += -fno-stack-protector \ > > $(DISABLE_STACKLEAK_PLUGIN) > > > > hostprogs := gen-hyprel > > -HOST_EXTRACFLAGS += -I$(objtree)/include > > +HOST_EXTRACFLAGS += -I$(objtree)/include -I$(srctree)/include/elf > > > > lib-objs := clear_page.o copy_page.o memcpy.o memset.o > > lib-objs := $(addprefix ../../../lib/, $(lib-objs)) > > diff --git a/include/elf/elf.h b/include/elf/elf.h > > new file mode 100644 > > index 000000000000..33aea7f743b8 > > --- /dev/null > > +++ b/include/elf/elf.h > > @@ -0,0 +1,4491 @@ > > +/* This file defines standard ELF types, structures, and macros. > > + Copyright (C) 1995-2024 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > I understand your want/need for this, but new files need a SPDX license > header instead of this type of license boilerplate. Didn't glibc > already convert to SPDX? I don't think they've done the conversion. But I can add the SPDX header if we move forward with the patch series, and ask them if they have any plan to adopt the SPDX standard. > > Also, as this is not internal for the kernel, but rather for userspace > builds, shouldn't the include/ path be different? Can you suggest an alternative path or provide documentation that could help identify the correct location? Perhaps usr/include? > > thanks, > > greg k-h
On Wed, Aug 07, 2024 at 02:13:57PM +0000, Daniel Gomez wrote: > > Also, as this is not internal for the kernel, but rather for userspace > > builds, shouldn't the include/ path be different? > > Can you suggest an alternative path or provide documentation that could help > identify the correct location? Perhaps usr/include? That is better than the generic include path as you are attempting to mix userspace and kernel headers in the same directory :(
On Wed, Aug 07, 2024 at 01:56:38PM +0000, Daniel Gomez wrote: > On Wed, Aug 07, 2024 at 01:01:08PM GMT, Greg Kroah-Hartman wrote: > > On Wed, Aug 07, 2024 at 01:09:14AM +0200, Daniel Gomez via B4 Relay wrote: > > > This patch set allows for building the Linux kernel for arm64 in macOS with > > > LLVM. > > > > Is this a requirement somewhere that this must work? It seems like an > > odd request, what workflows require cross-operating-system builds like > > this? > > This isn't a requirement, but it would, for example, support workflows for QEMU > users and developers on macOS. They could build/compile the kernel natively and > use it to launch QEMU instances, simplifying their process. But that's not a real workload of anyone? How often does this ever come up? Who is going to maintain this cross-build functionality over time? thanks, greg k-h
On Wed, Aug 07, 2024 at 04:18:54PM +0200, Greg Kroah-Hartman wrote: > On Wed, Aug 07, 2024 at 02:13:57PM +0000, Daniel Gomez wrote: > > > Also, as this is not internal for the kernel, but rather for userspace > > > builds, shouldn't the include/ path be different? > > > > Can you suggest an alternative path or provide documentation that could help > > identify the correct location? Perhaps usr/include? > > That is better than the generic include path as you are attempting to > mix userspace and kernel headers in the same directory :( Please keep in mind, that usr/include/ currently does not hold a single header file but is used for dynamically composing the UAPI header tree. In general, I do not like the idea of keeping a elf.h file here that possibly is out-of-sync with the actual system's version (even though elf.h should not see that much changes). Might it be more helpful to provide a "development kit" for Linux devs that need to build on MacOS that provides necessary missing system header files, instead of merging those into upstream? Kind regards, Nicolas
On Thu, Aug 8, 2024 at 2:20 AM Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote: > > On Wed, Aug 07, 2024 at 01:56:38PM +0000, Daniel Gomez wrote: > > On Wed, Aug 07, 2024 at 01:01:08PM GMT, Greg Kroah-Hartman wrote: > > > On Wed, Aug 07, 2024 at 01:09:14AM +0200, Daniel Gomez via B4 Relay wrote: > > > > This patch set allows for building the Linux kernel for arm64 in macOS with > > > > LLVM. > > > > > > Is this a requirement somewhere that this must work? It seems like an > > > odd request, what workflows require cross-operating-system builds like > > > this? > > > > This isn't a requirement, but it would, for example, support workflows for QEMU > > users and developers on macOS. They could build/compile the kernel natively and > > use it to launch QEMU instances, simplifying their process. > > But that's not a real workload of anyone? How often does this ever come > up? Who is going to maintain this cross-build functionality over time? it is a real workload of me, i was running qemu-system-aarch64 on x86 and it was pretty slow. so got a M3 pro to accelerate my development. frequently changing kernel's source code, i am using qemu-system-aarch64 to do quick verification. > > thanks, > > greg k-h > Thanks Barry
On Aug 17 13:11, Barry Song wrote: > On Thu, Aug 8, 2024 at 2:20 AM Greg Kroah-Hartman > <gregkh@linuxfoundation.org> wrote: > > > > On Wed, Aug 07, 2024 at 01:56:38PM +0000, Daniel Gomez wrote: > > > On Wed, Aug 07, 2024 at 01:01:08PM GMT, Greg Kroah-Hartman wrote: > > > > On Wed, Aug 07, 2024 at 01:09:14AM +0200, Daniel Gomez via B4 Relay wrote: > > > > > This patch set allows for building the Linux kernel for arm64 in macOS with > > > > > LLVM. > > > > > > > > Is this a requirement somewhere that this must work? It seems like an > > > > odd request, what workflows require cross-operating-system builds like > > > > this? > > > > > > This isn't a requirement, but it would, for example, support workflows for QEMU > > > users and developers on macOS. They could build/compile the kernel natively and > > > use it to launch QEMU instances, simplifying their process. > > > > But that's not a real workload of anyone? How often does this ever come > > up? Who is going to maintain this cross-build functionality over time? > > it is a real workload of me, i was running qemu-system-aarch64 on x86 > and it was > pretty slow. so got a M3 pro to accelerate my development. frequently changing > kernel's source code, i am using qemu-system-aarch64 to do quick verification. > Allow me to chime in as well. Working with qemu-system-aarch64 on an Mx are a daily thing for me as well. Working with custom built kernels have been a pain, and this really helps.
On Wed, Aug 07, 2024 at 04:19:42PM +0200, Greg Kroah-Hartman wrote: > On Wed, Aug 07, 2024 at 01:56:38PM +0000, Daniel Gomez wrote: > > On Wed, Aug 07, 2024 at 01:01:08PM GMT, Greg Kroah-Hartman wrote: > > > On Wed, Aug 07, 2024 at 01:09:14AM +0200, Daniel Gomez via B4 Relay wrote: > > > > This patch set allows for building the Linux kernel for arm64 in macOS with > > > > LLVM. > > > > > > Is this a requirement somewhere that this must work? It seems like an > > > odd request, what workflows require cross-operating-system builds like > > > this? > > > > This isn't a requirement, but it would, for example, support workflows for QEMU > > users and developers on macOS. They could build/compile the kernel natively and > > use it to launch QEMU instances, simplifying their process. > > But that's not a real workload of anyone? How often does this ever come > up? Who is going to maintain this cross-build functionality over time? The delta is becoming very small thanks to the latest patches from Masahiro. Earlier this week (next-20240820) [1] I rebased the work with all the feedback and the patch series has been reduced to 7. For the maintenance part, I suggest keeping a CI to build and boot the lastest linux-next tag available. I can set this up here [2] and take the responsability for maintaining that. But I would be convenient to add documentation for it in the LLVM section and mark this as 'experimental'. If that's okay, I will prepare a v2 with this. [1] https://github.com/SamsungDS/linux/commits/b4/macos-build-support/ [2] https://github.com/SamsungDS/linux > > thanks, > > greg k-h
On Wed, Aug 07, 2024 at 05:46:03PM +0200, Nicolas Schier wrote: > On Wed, Aug 07, 2024 at 04:18:54PM +0200, Greg Kroah-Hartman wrote: > > On Wed, Aug 07, 2024 at 02:13:57PM +0000, Daniel Gomez wrote: > > > > Also, as this is not internal for the kernel, but rather for userspace > > > > builds, shouldn't the include/ path be different? > > > > > > Can you suggest an alternative path or provide documentation that could help > > > identify the correct location? Perhaps usr/include? > > > > That is better than the generic include path as you are attempting to > > mix userspace and kernel headers in the same directory :( > > Please keep in mind, that usr/include/ currently does not hold a single > header file but is used for dynamically composing the UAPI header tree. > > In general, I do not like the idea of keeping a elf.h file here that > possibly is out-of-sync with the actual system's version (even though > elf.h should not see that much changes). Might it be more helpful to > provide a "development kit" for Linux devs that need to build on MacOS > that provides necessary missing system header files, instead of merging > those into upstream? I took this suggestion and tried pushing a Homebrew formula/package here [1]. I think I chose a wrong name and maybe something like "development kit" would have been better. However, would it be possible instead to include the *.rb file in the scripts/ directory? So users of this can generate the development kit in their environments. I would maintain the script to keep it in sync with the required glibc version for the latest kernel version. [1] https://github.com/Homebrew/homebrew-core/pull/181885 > > Kind regards, > Nicolas > > -- > Nicolas
On Sat, Aug 24, 2024 at 12:37:36AM +0200, Daniel Gomez wrote: > On Wed, Aug 07, 2024 at 04:19:42PM +0200, Greg Kroah-Hartman wrote: > > On Wed, Aug 07, 2024 at 01:56:38PM +0000, Daniel Gomez wrote: > > > On Wed, Aug 07, 2024 at 01:01:08PM GMT, Greg Kroah-Hartman wrote: > > > > On Wed, Aug 07, 2024 at 01:09:14AM +0200, Daniel Gomez via B4 Relay wrote: > > > > > This patch set allows for building the Linux kernel for arm64 in macOS with > > > > > LLVM. > > > > > > > > Is this a requirement somewhere that this must work? It seems like an > > > > odd request, what workflows require cross-operating-system builds like > > > > this? > > > > > > This isn't a requirement, but it would, for example, support workflows for QEMU > > > users and developers on macOS. They could build/compile the kernel natively and > > > use it to launch QEMU instances, simplifying their process. > > > > But that's not a real workload of anyone? How often does this ever come > > up? Who is going to maintain this cross-build functionality over time? > > The delta is becoming very small thanks to the latest patches from Masahiro. > Earlier this week (next-20240820) [1] I rebased the work with all the feedback > and the patch series has been reduced to 7. > > For the maintenance part, I suggest keeping a CI to build and boot the lastest > linux-next tag available. I can set this up here [2] and take the responsability > for maintaining that. But I would be convenient to add documentation for it in > the LLVM section and mark this as 'experimental'. If that's okay, I will prepare > a v2 with this. Let's see what v2 looks like and go from there. thanks, greg k-h
On Sat, Aug 24, 2024 at 12:54:50AM +0200 Daniel Gomez wrote: > On Wed, Aug 07, 2024 at 05:46:03PM +0200, Nicolas Schier wrote: > > On Wed, Aug 07, 2024 at 04:18:54PM +0200, Greg Kroah-Hartman wrote: > > > On Wed, Aug 07, 2024 at 02:13:57PM +0000, Daniel Gomez wrote: > > > > > Also, as this is not internal for the kernel, but rather for userspace > > > > > builds, shouldn't the include/ path be different? > > > > > > > > Can you suggest an alternative path or provide documentation that could help > > > > identify the correct location? Perhaps usr/include? > > > > > > That is better than the generic include path as you are attempting to > > > mix userspace and kernel headers in the same directory :( > > > > Please keep in mind, that usr/include/ currently does not hold a single > > header file but is used for dynamically composing the UAPI header tree. > > > > In general, I do not like the idea of keeping a elf.h file here that > > possibly is out-of-sync with the actual system's version (even though > > elf.h should not see that much changes). Might it be more helpful to > > provide a "development kit" for Linux devs that need to build on MacOS > > that provides necessary missing system header files, instead of merging > > those into upstream? > > I took this suggestion and tried pushing a Homebrew formula/package here [1]. > I think I chose a wrong name and maybe something like "development kit" would > have been better. However, would it be possible instead to include the *.rb file > in the scripts/ directory? So users of this can generate the development kit in > their environments. I would maintain the script to keep it in sync with the > required glibc version for the latest kernel version. > > [1] https://github.com/Homebrew/homebrew-core/pull/181885 I think it sounds sensible to hold that formula file in the upstream tree. But I am not sure if scripts/ is the best location. Masahiro, what do you think? Kind regards, Nicolas
On Sat, Aug 31, 2024 at 4:54 AM Nicolas Schier <nicolas@fjasle.eu> wrote: > > On Sat, Aug 24, 2024 at 12:54:50AM +0200 Daniel Gomez wrote: > > On Wed, Aug 07, 2024 at 05:46:03PM +0200, Nicolas Schier wrote: > > > On Wed, Aug 07, 2024 at 04:18:54PM +0200, Greg Kroah-Hartman wrote: > > > > On Wed, Aug 07, 2024 at 02:13:57PM +0000, Daniel Gomez wrote: > > > > > > Also, as this is not internal for the kernel, but rather for userspace > > > > > > builds, shouldn't the include/ path be different? > > > > > > > > > > Can you suggest an alternative path or provide documentation that could help > > > > > identify the correct location? Perhaps usr/include? > > > > > > > > That is better than the generic include path as you are attempting to > > > > mix userspace and kernel headers in the same directory :( > > > > > > Please keep in mind, that usr/include/ currently does not hold a single > > > header file but is used for dynamically composing the UAPI header tree. > > > > > > In general, I do not like the idea of keeping a elf.h file here that > > > possibly is out-of-sync with the actual system's version (even though > > > elf.h should not see that much changes). Might it be more helpful to > > > provide a "development kit" for Linux devs that need to build on MacOS > > > that provides necessary missing system header files, instead of merging > > > those into upstream? > > > > I took this suggestion and tried pushing a Homebrew formula/package here [1]. > > I think I chose a wrong name and maybe something like "development kit" would > > have been better. However, would it be possible instead to include the *.rb file > > in the scripts/ directory? So users of this can generate the development kit in > > their environments. I would maintain the script to keep it in sync with the > > required glibc version for the latest kernel version. > > > > [1] https://github.com/Homebrew/homebrew-core/pull/181885 > > I think it sounds sensible to hold that formula file in the upstream tree. But > I am not sure if scripts/ is the best location. > > Masahiro, what do you think? I do not know much about the homebrew, but why does the upstream kernel need to merge such masOS stuff? > > Kind regards, > Nicolas
On Mon, Sep 02, 2024 at 01:15:01AM +0900, Masahiro Yamada wrote: > On Sat, Aug 31, 2024 at 4:54 AM Nicolas Schier <nicolas@fjasle.eu> wrote: > > > > On Sat, Aug 24, 2024 at 12:54:50AM +0200 Daniel Gomez wrote: > > > On Wed, Aug 07, 2024 at 05:46:03PM +0200, Nicolas Schier wrote: > > > > On Wed, Aug 07, 2024 at 04:18:54PM +0200, Greg Kroah-Hartman wrote: > > > > > On Wed, Aug 07, 2024 at 02:13:57PM +0000, Daniel Gomez wrote: > > > > > > > Also, as this is not internal for the kernel, but rather for userspace > > > > > > > builds, shouldn't the include/ path be different? > > > > > > > > > > > > Can you suggest an alternative path or provide documentation that could help > > > > > > identify the correct location? Perhaps usr/include? > > > > > > > > > > That is better than the generic include path as you are attempting to > > > > > mix userspace and kernel headers in the same directory :( > > > > > > > > Please keep in mind, that usr/include/ currently does not hold a single > > > > header file but is used for dynamically composing the UAPI header tree. > > > > > > > > In general, I do not like the idea of keeping a elf.h file here that > > > > possibly is out-of-sync with the actual system's version (even though > > > > elf.h should not see that much changes). Might it be more helpful to > > > > provide a "development kit" for Linux devs that need to build on MacOS > > > > that provides necessary missing system header files, instead of merging > > > > those into upstream? > > > > > > I took this suggestion and tried pushing a Homebrew formula/package here [1]. > > > I think I chose a wrong name and maybe something like "development kit" would > > > have been better. However, would it be possible instead to include the *.rb file > > > in the scripts/ directory? So users of this can generate the development kit in > > > their environments. I would maintain the script to keep it in sync with the > > > required glibc version for the latest kernel version. > > > > > > [1] https://protect2.fireeye.com/v1/url?k=96027706-f7896236-9603fc49-000babffaa23-452f645d7a72e234&q=1&e=343dd31c-5e5b-4b09-8ee5-6c59a1ff826e&u=https%3A%2F%2Fgithub.com%2FHomebrew%2Fhomebrew-core%2Fpull%2F181885 > > > > I think it sounds sensible to hold that formula file in the upstream tree. But > > I am not sure if scripts/ is the best location. > > > > Masahiro, what do you think? > > > I do not know much about the homebrew, but why does the upstream > kernel need to merge such masOS stuff? The missing headers (in macOS) need to be provided somehow. One way can be having the formula (*.rb file) in-tree, so users of this can install them in their systems. This would also require to have a tarball with the missing headers either in-tree or somewhere accessible so it can be fetched. To avoid having the formula and a tarball in-tree, I've created a Homebrew Tap (3rd-Party Repository) called 'Bee Headers Project' [1][2][3] that can provision the missing headers. The project provides a bee-headers package and formula that will install byteswap.h, elf.h and endian.h in the user's system Hombrew directory. It also provides a *.pc file so pkg-config can be used to find the location of these headers. I have a v2 with this solution ready, perhaps is easier to discuss this with the code. I think we can extend the same package and include extra headers if we need more in the future. I see for x86_64 asm/types.h and others might be required. The bee-headers package can then be the repository to place and distribute them. Please, let me know if you think of an alternative solution, I can give a try and explore. [1] Project: https://github.com/bee-headers [2] Headers repository: https://github.com/bee-headers/headers.git [3] Homebrew Tap formula: https://github.com/bee-headers/homebrew-bee-headers.git > > > > > > > Kind regards, > > Nicolas > > > > -- > Best Regards > Masahiro Yamada
On Thu, Sep 5, 2024 at 5:56 PM Daniel Gomez <da.gomez@samsung.com> wrote: > > On Mon, Sep 02, 2024 at 01:15:01AM +0900, Masahiro Yamada wrote: > > On Sat, Aug 31, 2024 at 4:54 AM Nicolas Schier <nicolas@fjasle.eu> wrote: > > > > > > On Sat, Aug 24, 2024 at 12:54:50AM +0200 Daniel Gomez wrote: > > > > On Wed, Aug 07, 2024 at 05:46:03PM +0200, Nicolas Schier wrote: > > > > > On Wed, Aug 07, 2024 at 04:18:54PM +0200, Greg Kroah-Hartman wrote: > > > > > > On Wed, Aug 07, 2024 at 02:13:57PM +0000, Daniel Gomez wrote: > > > > > > > > Also, as this is not internal for the kernel, but rather for userspace > > > > > > > > builds, shouldn't the include/ path be different? > > > > > > > > > > > > > > Can you suggest an alternative path or provide documentation that could help > > > > > > > identify the correct location? Perhaps usr/include? > > > > > > > > > > > > That is better than the generic include path as you are attempting to > > > > > > mix userspace and kernel headers in the same directory :( > > > > > > > > > > Please keep in mind, that usr/include/ currently does not hold a single > > > > > header file but is used for dynamically composing the UAPI header tree. > > > > > > > > > > In general, I do not like the idea of keeping a elf.h file here that > > > > > possibly is out-of-sync with the actual system's version (even though > > > > > elf.h should not see that much changes). Might it be more helpful to > > > > > provide a "development kit" for Linux devs that need to build on MacOS > > > > > that provides necessary missing system header files, instead of merging > > > > > those into upstream? > > > > > > > > I took this suggestion and tried pushing a Homebrew formula/package here [1]. > > > > I think I chose a wrong name and maybe something like "development kit" would > > > > have been better. However, would it be possible instead to include the *.rb file > > > > in the scripts/ directory? So users of this can generate the development kit in > > > > their environments. I would maintain the script to keep it in sync with the > > > > required glibc version for the latest kernel version. > > > > > > > > [1] https://protect2.fireeye.com/v1/url?k=96027706-f7896236-9603fc49-000babffaa23-452f645d7a72e234&q=1&e=343dd31c-5e5b-4b09-8ee5-6c59a1ff826e&u=https%3A%2F%2Fgithub.com%2FHomebrew%2Fhomebrew-core%2Fpull%2F181885 > > > > > > I think it sounds sensible to hold that formula file in the upstream tree. But > > > I am not sure if scripts/ is the best location. > > > > > > Masahiro, what do you think? > > > > > > I do not know much about the homebrew, but why does the upstream > > kernel need to merge such masOS stuff? > > The missing headers (in macOS) need to be provided somehow. One way can be > having the formula (*.rb file) in-tree, so users of this can install them in > their systems. This would also require to have a tarball with the missing > headers either in-tree or somewhere accessible so it can be fetched. > > To avoid having the formula and a tarball in-tree, I've created a Homebrew Tap > (3rd-Party Repository) called 'Bee Headers Project' [1][2][3] that can provision > the missing headers. The project provides a bee-headers package and formula > that will install byteswap.h, elf.h and endian.h in the user's system Hombrew > directory. It also provides a *.pc file so pkg-config can be used to find the > location of these headers. I have a v2 with this solution ready, perhaps is > easier to discuss this with the code. It is up to you what project you start in Github. Then, what do you need to submit v2 for this? > > I think we can extend the same package and include extra headers if we need > more in the future. I see for x86_64 asm/types.h and others might be required. > The bee-headers package can then be the repository to place and distribute them. > > Please, let me know if you think of an alternative solution, I can give a try > and explore. > > [1] Project: > https://github.com/bee-headers > [2] Headers repository: > https://github.com/bee-headers/headers.git > [3] Homebrew Tap formula: > https://github.com/bee-headers/homebrew-bee-headers.git > > > > > > > > > > > > > > Kind regards, > > > Nicolas > > > > > > > > -- > > Best Regards > > Masahiro Yamada
On Fri, Sep 06, 2024 at 08:58:15AM +0900, Masahiro Yamada wrote: > On Thu, Sep 5, 2024 at 5:56 PM Daniel Gomez <da.gomez@samsung.com> wrote: > > > > On Mon, Sep 02, 2024 at 01:15:01AM +0900, Masahiro Yamada wrote: > > > On Sat, Aug 31, 2024 at 4:54 AM Nicolas Schier <nicolas@fjasle.eu> wrote: > > > > > > > > On Sat, Aug 24, 2024 at 12:54:50AM +0200 Daniel Gomez wrote: > > > > > On Wed, Aug 07, 2024 at 05:46:03PM +0200, Nicolas Schier wrote: > > > > > > On Wed, Aug 07, 2024 at 04:18:54PM +0200, Greg Kroah-Hartman wrote: > > > > > > > On Wed, Aug 07, 2024 at 02:13:57PM +0000, Daniel Gomez wrote: > > > > > > > > > Also, as this is not internal for the kernel, but rather for userspace > > > > > > > > > builds, shouldn't the include/ path be different? > > > > > > > > > > > > > > > > Can you suggest an alternative path or provide documentation that could help > > > > > > > > identify the correct location? Perhaps usr/include? > > > > > > > > > > > > > > That is better than the generic include path as you are attempting to > > > > > > > mix userspace and kernel headers in the same directory :( > > > > > > > > > > > > Please keep in mind, that usr/include/ currently does not hold a single > > > > > > header file but is used for dynamically composing the UAPI header tree. > > > > > > > > > > > > In general, I do not like the idea of keeping a elf.h file here that > > > > > > possibly is out-of-sync with the actual system's version (even though > > > > > > elf.h should not see that much changes). Might it be more helpful to > > > > > > provide a "development kit" for Linux devs that need to build on MacOS > > > > > > that provides necessary missing system header files, instead of merging > > > > > > those into upstream? > > > > > > > > > > I took this suggestion and tried pushing a Homebrew formula/package here [1]. > > > > > I think I chose a wrong name and maybe something like "development kit" would > > > > > have been better. However, would it be possible instead to include the *.rb file > > > > > in the scripts/ directory? So users of this can generate the development kit in > > > > > their environments. I would maintain the script to keep it in sync with the > > > > > required glibc version for the latest kernel version. > > > > > > > > > > [1] https://protect2.fireeye.com/v1/url?k=96027706-f7896236-9603fc49-000babffaa23-452f645d7a72e234&q=1&e=343dd31c-5e5b-4b09-8ee5-6c59a1ff826e&u=https%3A%2F%2Fgithub.com%2FHomebrew%2Fhomebrew-core%2Fpull%2F181885 > > > > > > > > I think it sounds sensible to hold that formula file in the upstream tree. But > > > > I am not sure if scripts/ is the best location. > > > > > > > > Masahiro, what do you think? > > > > > > > > > I do not know much about the homebrew, but why does the upstream > > > kernel need to merge such masOS stuff? > > > > The missing headers (in macOS) need to be provided somehow. One way can be > > having the formula (*.rb file) in-tree, so users of this can install them in > > their systems. This would also require to have a tarball with the missing > > headers either in-tree or somewhere accessible so it can be fetched. > > > > To avoid having the formula and a tarball in-tree, I've created a Homebrew Tap > > (3rd-Party Repository) called 'Bee Headers Project' [1][2][3] that can provision > > the missing headers. The project provides a bee-headers package and formula > > that will install byteswap.h, elf.h and endian.h in the user's system Hombrew > > directory. It also provides a *.pc file so pkg-config can be used to find the > > location of these headers. I have a v2 with this solution ready, perhaps is > > easier to discuss this with the code. > > > It is up to you what project you start in Github. > > Then, what do you need to submit v2 for this? I'll just include a reference in the documentation. And will send the v2 later today. > > > > > > > > > I think we can extend the same package and include extra headers if we need > > more in the future. I see for x86_64 asm/types.h and others might be required. > > The bee-headers package can then be the repository to place and distribute them. > > > > Please, let me know if you think of an alternative solution, I can give a try > > and explore. > > > > [1] Project: > > https://protect2.fireeye.com/v1/url?k=8f1b9871-ee908d54-8f1a133e-74fe485cbff6-b9b8f875df8b37c5&q=1&e=860f3982-d1e4-4fcb-ab79-f181bb7fcda3&u=https%3A%2F%2Fgithub.com%2Fbee-headers > > [2] Headers repository: > > https://protect2.fireeye.com/v1/url?k=7c995f7b-1d124a5e-7c98d434-74fe485cbff6-a20d4e643f5978f8&q=1&e=860f3982-d1e4-4fcb-ab79-f181bb7fcda3&u=https%3A%2F%2Fgithub.com%2Fbee-headers%2Fheaders.git > > [3] Homebrew Tap formula: > > https://protect2.fireeye.com/v1/url?k=148f40fc-750455d9-148ecbb3-74fe485cbff6-5d795246adb22931&q=1&e=860f3982-d1e4-4fcb-ab79-f181bb7fcda3&u=https%3A%2F%2Fgithub.com%2Fbee-headers%2Fhomebrew-bee-headers.git > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > Nicolas > > > > > > > > > > > > -- > > > Best Regards > > > Masahiro Yamada > > > > -- > Best Regards > Masahiro Yamada
This patch set allows for building the Linux kernel for arm64 in macOS with LLVM. Patches are based on previous Nick's work and suggestions [1][2][3] to enable build system in macOS hosts. Since macOS does not provide some of the headers available in the GNU C Library (glibc), the patches include a copy of these headers from glibc-2.40, with minor modifications detailed in the commit message. To set up the environment: * Provide build dependencies (installed via Homebrew): coreutils, findutils, gnu-sed, gnu-tar, grep, llvm, make and pkg-config. * A case sensitive volume for building: diskutil apfs addVolume /dev/disk<N> "Case-sensitive APFS" linux * And include in your PATH all GNU tools required by the Linux kernel as well as LLVM: PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH" PATH="/opt/homebrew/opt/findutils/libexec/gnubin:$PATH" PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH" PATH="/opt/homebrew/opt/gnu-tar/libexec/gnubin:$PATH" PATH="/opt/homebrew/opt/grep/libexec/gnubin:$PATH" PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH" PATH="/opt/homebrew/opt/llvm/bin:$PATH" * Start the build using LLVM: make LLVM=1 allyesconfig make LLVM=1 -j$(nproc) I believe other architectures could also be supported if we can move forward this initiative. Additionally, we could incorporate Rust support. I understand that Finn Behrens has some patches [4][5] based on Nick's previous work. [1]: WIP: build Linux on MacOS https://github.com/ClangBuiltLinux/linux/commit/f06333e29addbc3d714adb340355f471c1dfe95a [2] Subject: [PATCH] scripts: subarch.include: fix SUBARCH on MacOS hosts https://lore.kernel.org/all/20221113233812.36784-1-nick.desaulniers@gmail.com/ [3] Subject: Any interest in building the Linux kernel from a MacOS host? https://lore.kernel.org/all/CAH7mPvj64Scp6_Nbaj8KOfkoV5f7_N5L=Tv5Z9zGyn5SS+gsUw@mail.gmail.com/ [4] https://github.com/kloenk/linux/commits/rust-project_macos-dylib/ [5] https://kloenk.eu/posts/build-linux-on-m1-macos/ To: Masahiro Yamada <masahiroy@kernel.org> To: Nathan Chancellor <nathan@kernel.org> To: Nicolas Schier <nicolas@fjasle.eu> To: Lucas De Marchi <lucas.demarchi@intel.com> To: Thomas Hellström <thomas.hellstrom@linux.intel.com> To: Rodrigo Vivi <rodrigo.vivi@intel.com> To: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> To: Maxime Ripard <mripard@kernel.org> To: Thomas Zimmermann <tzimmermann@suse.de> To: David Airlie <airlied@gmail.com> To: Daniel Vetter <daniel@ffwll.ch> To: William Hubbs <w.d.hubbs@gmail.com> To: Chris Brannon <chris@the-brannons.com> To: Kirk Reiser <kirk@reisers.ca> To: Samuel Thibault <samuel.thibault@ens-lyon.org> To: Paul Moore <paul@paul-moore.com> To: Stephen Smalley <stephen.smalley.work@gmail.com> To: Ondrej Mosnacek <omosnace@redhat.com> To: Catalin Marinas <catalin.marinas@arm.com> To: Will Deacon <will@kernel.org> To: Marc Zyngier <maz@kernel.org> To: Oliver Upton <oliver.upton@linux.dev> To: James Morse <james.morse@arm.com> To: Suzuki K Poulose <suzuki.poulose@arm.com> To: Zenghui Yu <yuzenghui@huawei.com> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org> To: Jiri Slaby <jirislaby@kernel.org> To: Nick Desaulniers <ndesaulniers@google.com> To: Bill Wendling <morbo@google.com> To: Justin Stitt <justinstitt@google.com> Cc: linux-kernel@vger.kernel.org Cc: linux-kbuild@vger.kernel.org Cc: intel-xe@lists.freedesktop.org Cc: dri-devel@lists.freedesktop.org Cc: speakup@linux-speakup.org Cc: selinux@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: kvmarm@lists.linux.dev Cc: linux-serial@vger.kernel.org Cc: llvm@lists.linux.dev Cc: Finn Behrens <me@kloenk.dev> Cc: Daniel Gomez (Samsung) <d+samsung@kruces.com> Cc: gost.dev@samsung.com Signed-off-by: Daniel Gomez <da.gomez@samsung.com> --- Daniel Gomez (11): kbuild: add header_install dependency to scripts file2alias: fix uuid_t definitions for macos drm/xe: xe_gen_wa_oob: fix program_invocation_short_name for macos accessiblity/speakup: genmap and makemapdata require linux/version.h selinux/genheaders: include bitsperlong and posix_types headers selinux/mdp: include bitsperlong and posix_types headers include: add elf.h support include: add endian.h support scripts/mod: add byteswap support tty/vt: conmakehash requires linux/limits.h scripts/kallsyms: add compatibility support for macos Nick Desaulniers (1): scripts: subarch.include: fix SUBARCH on MacOS hosts Makefile | 2 +- arch/arm64/kernel/pi/Makefile | 1 + arch/arm64/kernel/vdso32/Makefile | 1 + arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- drivers/accessibility/speakup/Makefile | 2 + drivers/gpu/drm/xe/xe_gen_wa_oob.c | 8 +- drivers/tty/vt/Makefile | 1 + include/byteswap/byteswap.h | 35 + include/elf/elf.h | 4491 ++++++++++++++++++++++++++++++++ include/endian/bits/uintn-identity.h | 48 + include/endian/endian.h | 63 + scripts/Makefile | 3 +- scripts/kallsyms.c | 4 + scripts/mod/Makefile | 6 + scripts/mod/file2alias.c | 3 + scripts/selinux/genheaders/Makefile | 3 +- scripts/selinux/mdp/Makefile | 3 +- scripts/subarch.include | 3 +- 18 files changed, 4672 insertions(+), 7 deletions(-) --- base-commit: 1e391b34f6aa043c7afa40a2103163a0ef06d179 change-id: 20240807-macos-build-support-9ca0d77adb17 Best regards,