| Message ID | 20240906-macos-build-support-v2-8-06beff418848@samsung.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Enable build system on macOS hosts | expand |
On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay <devnull+da.gomez.samsung.com@kernel.org> wrote: > > From: Daniel Gomez <da.gomez@samsung.com> > > Add documentation under kbuild/llvm to inform about the experimental > support for building the Linux kernel in macOS hosts environments. > > Signed-off-by: Daniel Gomez <da.gomez@samsung.com> Instead, you can add this instruction to: https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md > --- > Documentation/kbuild/llvm.rst | 78 +++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 78 insertions(+) > > diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst > index 6dc66b4f31a7..de3bde925793 100644 > --- a/Documentation/kbuild/llvm.rst > +++ b/Documentation/kbuild/llvm.rst > @@ -186,6 +186,84 @@ yet. Bug reports are always welcome at the issue tracker below! > - Supported > - ``LLVM=1`` > > +Experimental Build in macOS > +--------------------------- > + > +Building on macOS with LLVM is experimental. This section provides steps to > +install dependencies via Homebrew, set up the environment, and start the build > +process. > + > +1. **Create a Case-Sensitive Volume** > + > + For fetching and building the project, you need a case-sensitive volume. Use the following > + command to create one: > + > + .. code-block:: shell > + > + diskutil apfs addVolume /dev/disk<N> "Case-sensitive APFS" linux > + > + Replace `/dev/disk<N>` with the appropriate disk identifier. > + > +2. **Install Build Dependencies** > + > +Use Homebrew to install the required build dependencies. > + > +- **Core Utilities**: `coreutils`, `findutils`, `gnu-sed`, `gnu-tar`, `grep`, > + `llvm`, `make`, and `pkg-config`. > + > + .. code-block:: shell > + > + brew install coreutils findutils gnu-sed gnu-tar grep llvm make pkg-config > + > +- **Bee Headers**: Install byteswap, elf and endian headers using the > + `Bee Headers Project <https://github.com/bee-headers/headers>`_. > + > + .. code-block:: shell > + > + brew tap bee-headers/bee-headers > + brew install bee-headers/bee-headers/bee-headers > + > + After installation, verify the `CFLAGS` with `pkg-config`: > + > + .. code-block:: shell > + > + pkg-config --cflags bee-headers > + -I/opt/homebrew/Cellar/bee-headers/0.1/include > + > +3. **Configure the PATH** > + > + Include all the required GNU tools and LLVM in your `PATH`. This ensures that > + the necessary tools are available during the build process. > + > + .. code-block:: shell > + > + PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH" > + PATH="/opt/homebrew/opt/findutils/libexec/gnubin:$PATH" > + PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH" > + PATH="/opt/homebrew/opt/gnu-tar/libexec/gnubin:$PATH" > + PATH="/opt/homebrew/opt/grep/libexec/gnubin:$PATH" > + PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH" > + PATH="/opt/homebrew/opt/llvm/bin:$PATH" > + > +Building the Project > +-------------------- > + > +Once the environment is set up, you can start the build process using LLVM. Run > +the following commands to initiate the build: > + > +.. code-block:: shell > + > + make LLVM=1 allyesconfig > + make LLVM=1 -j$(nproc) > + > +Supported in macOS > +~~~~~~~~~~~~~~~~~~ > + > +At the moment, only arm64 is supported and tested with `allyesconfig` Makefile > +configuration target. Other Kconfig options not included in `allyesconfig` > +target and architectures may be supported as well as support in macOS is based > +on LLVM effort and maintenance. > + > Getting Help > ------------ > > > -- > 2.46.0 > > -- Best Regards Masahiro Yamada
On Sat, Sep 7, 2024 at 10:33 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay > <devnull+da.gomez.samsung.com@kernel.org> wrote: > > > > From: Daniel Gomez <da.gomez@samsung.com> > > > > Add documentation under kbuild/llvm to inform about the experimental > > support for building the Linux kernel in macOS hosts environments. > > > > Signed-off-by: Daniel Gomez <da.gomez@samsung.com> > > > Instead, you can add this instruction to: > > https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md Sure, that can be done as well. But the effort here is to have this integrated. So, I think documentation should be in-tree. > > > > > > > --- > > Documentation/kbuild/llvm.rst | 78 +++++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 78 insertions(+) > > > > diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst > > index 6dc66b4f31a7..de3bde925793 100644 > > --- a/Documentation/kbuild/llvm.rst > > +++ b/Documentation/kbuild/llvm.rst > > @@ -186,6 +186,84 @@ yet. Bug reports are always welcome at the issue tracker below! > > - Supported > > - ``LLVM=1`` > > > > +Experimental Build in macOS > > +--------------------------- > > + > > +Building on macOS with LLVM is experimental. This section provides steps to > > +install dependencies via Homebrew, set up the environment, and start the build > > +process. > > + > > +1. **Create a Case-Sensitive Volume** > > + > > + For fetching and building the project, you need a case-sensitive volume. Use the following > > + command to create one: > > + > > + .. code-block:: shell > > + > > + diskutil apfs addVolume /dev/disk<N> "Case-sensitive APFS" linux > > + > > + Replace `/dev/disk<N>` with the appropriate disk identifier. > > + > > +2. **Install Build Dependencies** > > + > > +Use Homebrew to install the required build dependencies. > > + > > +- **Core Utilities**: `coreutils`, `findutils`, `gnu-sed`, `gnu-tar`, `grep`, > > + `llvm`, `make`, and `pkg-config`. > > + > > + .. code-block:: shell > > + > > + brew install coreutils findutils gnu-sed gnu-tar grep llvm make pkg-config > > + > > +- **Bee Headers**: Install byteswap, elf and endian headers using the > > + `Bee Headers Project <https://github.com/bee-headers/headers>`_. > > + > > + .. code-block:: shell > > + > > + brew tap bee-headers/bee-headers > > + brew install bee-headers/bee-headers/bee-headers > > + > > + After installation, verify the `CFLAGS` with `pkg-config`: > > + > > + .. code-block:: shell > > + > > + pkg-config --cflags bee-headers > > + -I/opt/homebrew/Cellar/bee-headers/0.1/include > > + > > +3. **Configure the PATH** > > + > > + Include all the required GNU tools and LLVM in your `PATH`. This ensures that > > + the necessary tools are available during the build process. > > + > > + .. code-block:: shell > > + > > + PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH" > > + PATH="/opt/homebrew/opt/findutils/libexec/gnubin:$PATH" > > + PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH" > > + PATH="/opt/homebrew/opt/gnu-tar/libexec/gnubin:$PATH" > > + PATH="/opt/homebrew/opt/grep/libexec/gnubin:$PATH" > > + PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH" > > + PATH="/opt/homebrew/opt/llvm/bin:$PATH" > > + > > +Building the Project > > +-------------------- > > + > > +Once the environment is set up, you can start the build process using LLVM. Run > > +the following commands to initiate the build: > > + > > +.. code-block:: shell > > + > > + make LLVM=1 allyesconfig > > + make LLVM=1 -j$(nproc) > > + > > +Supported in macOS > > +~~~~~~~~~~~~~~~~~~ > > + > > +At the moment, only arm64 is supported and tested with `allyesconfig` Makefile > > +configuration target. Other Kconfig options not included in `allyesconfig` > > +target and architectures may be supported as well as support in macOS is based > > +on LLVM effort and maintenance. > > + > > Getting Help > > ------------ > > > > > > -- > > 2.46.0 > > > > > > > -- > Best Regards > Masahiro Yamada
On Sat, Sep 7, 2024 at 6:32 PM Daniel Gomez (Samsung) <d+samsung@kruces.com> wrote: > > On Sat, Sep 7, 2024 at 10:33 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay > > <devnull+da.gomez.samsung.com@kernel.org> wrote: > > > > > > From: Daniel Gomez <da.gomez@samsung.com> > > > > > > Add documentation under kbuild/llvm to inform about the experimental > > > support for building the Linux kernel in macOS hosts environments. > > > > > > Signed-off-by: Daniel Gomez <da.gomez@samsung.com> > > > > > > Instead, you can add this instruction to: > > > > https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md > > Sure, that can be done as well. But the effort here is to have this > integrated. So, I think documentation should be in-tree. I do not think so. Most people are not compile-testing on macOS. This is an unofficial tip, which you can advertise somewhere else.
On Sat, 07 Sep 2024 10:32:20 +0100, "Daniel Gomez (Samsung)" <d+samsung@kruces.com> wrote: > > On Sat, Sep 7, 2024 at 10:33 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay > > <devnull+da.gomez.samsung.com@kernel.org> wrote: > > > > > > From: Daniel Gomez <da.gomez@samsung.com> > > > > > > Add documentation under kbuild/llvm to inform about the experimental > > > support for building the Linux kernel in macOS hosts environments. > > > > > > Signed-off-by: Daniel Gomez <da.gomez@samsung.com> > > > > > > Instead, you can add this instruction to: > > > > https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md > > Sure, that can be done as well. But the effort here is to have this > integrated. So, I think documentation should be in-tree. I think this ship sailed the moment you ended-up with an external dependency. Having looked at this series (and in particular patch #4 which falls under my remit), I can't help but think that the whole thing should simply live as a wrapper around the pristine build system instead of hacking things inside of it. You already pull external dependencies (the include files). Just add a script that sets things up (environment variables that already exist) and calls 'make' in the kernel tree. I also dislike that this is forcing "native" developers to cater for an operating system they are unlikely to have access to. If I break this hack tomorrow by adding a new dependency that MacOS doesn't provide, how do I fix it? Should I drop my changes on the floor? As an alternative, and since you already have to create a special file-system to contain your kernel tree, you may as well run Linux in a VM, which I am told works pretty well (QEMU supports HVF, and there are plenty of corporate-friendly alternatives). This would solve your problem once and for all. Please don't take the above the wrong way. I'm sympathetic to what you are trying to do. But this is IMO going in the wrong direction. Thanks, M.
On 9/8/2024 11:03 AM, Marc Zyngier wrote: > On Sat, 07 Sep 2024 10:32:20 +0100, > "Daniel Gomez (Samsung)" <d+samsung@kruces.com> wrote: >> >> On Sat, Sep 7, 2024 at 10:33 AM Masahiro Yamada <masahiroy@kernel.org> wrote: >>> >>> On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay >>> <devnull+da.gomez.samsung.com@kernel.org> wrote: >>>> >>>> From: Daniel Gomez <da.gomez@samsung.com> >>>> >>>> Add documentation under kbuild/llvm to inform about the experimental >>>> support for building the Linux kernel in macOS hosts environments. >>>> >>>> Signed-off-by: Daniel Gomez <da.gomez@samsung.com> >>> >>> >>> Instead, you can add this instruction to: >>> >>> https://protect2.fireeye.com/v1/url?k=f33af8a2-92b1ed94-f33b73ed-74fe485cbff1-7d382b34bfd617fc&q=1&e=c7a3e869-d48e-4168-88a9-03cd717797f0&u=https%3A%2F%2Fgithub.com%2Fbee-headers%2Fhomebrew-bee-headers%2Fblob%2Fmain%2FREADME.md >> >> Sure, that can be done as well. But the effort here is to have this >> integrated. So, I think documentation should be in-tree. > > I think this ship sailed the moment you ended-up with an external > dependency. The external dependency is not different in Linux hosts. We depend on byteswap.h, elf.h and endian.h headers. However, these are provided by glibc and musl, and macOS (at least in arm64/Homebrew) does not provide any of these. To fix the dependency with these missing headers, it was suggested in v1 [1][2] to create a "development kit" for this hosts. And that is what Bee Headers [3][4] aims to provide. [1]https://lore.kernel.org/all/20240807-mottled-stoic-degu-d1e4cb@lindesnes/ [2]https://lore.kernel.org/all/ZrSoOM9z4VnqhOf2@fjasle.eu/ [3] https://github.com/bee-headers/headers [4] https://github.com/bee-headers/homebrew-bee-headers I don't see any other alternative to provide the missing headers other than the suggested. Please let me know if you think other paths can be explored and tested. > > Having looked at this series (and in particular patch #4 which falls > under my remit), I can't help but think that the whole thing should > simply live as a wrapper around the pristine build system instead of > hacking things inside of it. You already pull external dependencies > (the include files). Just add a script that sets things up > (environment variables that already exist) and calls 'make' in the > kernel tree. Agree. This aligns very well with other feedback. I've added a script to the Bee Headers [5] to help users init their shell/Makefile environment. [5] https://github.com/bee-headers/homebrew-bee-headers/blob/main/bee-headers.rb#L28 > > I also dislike that this is forcing "native" developers to cater for > an operating system they are unlikely to have access to. If I break > this hack tomorrow by adding a new dependency that MacOS doesn't > provide, how do I fix it? Should I drop my changes on the floor? > > As an alternative, and since you already have to create a special > file-system to contain your kernel tree, you may as well run Linux in > a VM, which I am told works pretty well (QEMU supports HVF, and there > are plenty of corporate-friendly alternatives). This would solve your > problem once and for all. This is a use case I'm trying to avoid with this series. QEMU HVF works very well indeed but extracting the built objects/vmlinux/*.ko etc is tedious and slow. Building natively allows users to boot a VM with QEMU and -kernel argument directly. This is way faster than the suggested alternative. In addition, lldb can be used to debug the kernel from the host. > > Please don't take the above the wrong way. I'm sympathetic to what you > are trying to do. But this is IMO going in the wrong direction. First patch has been merged already and after some fixes in the build system from Masahiro, this series only requires a small change in xe_gen_wa_oob to make this work if users of this provide the missing headers and set up the build environment properly. I will send a v3 later today with the remaining patch. Thanks Marc for sharing your views! > > Thanks, > > M. >
diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst index 6dc66b4f31a7..de3bde925793 100644 --- a/Documentation/kbuild/llvm.rst +++ b/Documentation/kbuild/llvm.rst @@ -186,6 +186,84 @@ yet. Bug reports are always welcome at the issue tracker below! - Supported - ``LLVM=1`` +Experimental Build in macOS +--------------------------- + +Building on macOS with LLVM is experimental. This section provides steps to +install dependencies via Homebrew, set up the environment, and start the build +process. + +1. **Create a Case-Sensitive Volume** + + For fetching and building the project, you need a case-sensitive volume. Use the following + command to create one: + + .. code-block:: shell + + diskutil apfs addVolume /dev/disk<N> "Case-sensitive APFS" linux + + Replace `/dev/disk<N>` with the appropriate disk identifier. + +2. **Install Build Dependencies** + +Use Homebrew to install the required build dependencies. + +- **Core Utilities**: `coreutils`, `findutils`, `gnu-sed`, `gnu-tar`, `grep`, + `llvm`, `make`, and `pkg-config`. + + .. code-block:: shell + + brew install coreutils findutils gnu-sed gnu-tar grep llvm make pkg-config + +- **Bee Headers**: Install byteswap, elf and endian headers using the + `Bee Headers Project <https://github.com/bee-headers/headers>`_. + + .. code-block:: shell + + brew tap bee-headers/bee-headers + brew install bee-headers/bee-headers/bee-headers + + After installation, verify the `CFLAGS` with `pkg-config`: + + .. code-block:: shell + + pkg-config --cflags bee-headers + -I/opt/homebrew/Cellar/bee-headers/0.1/include + +3. **Configure the PATH** + + Include all the required GNU tools and LLVM in your `PATH`. This ensures that + the necessary tools are available during the build process. + + .. code-block:: shell + + PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH" + PATH="/opt/homebrew/opt/findutils/libexec/gnubin:$PATH" + PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH" + PATH="/opt/homebrew/opt/gnu-tar/libexec/gnubin:$PATH" + PATH="/opt/homebrew/opt/grep/libexec/gnubin:$PATH" + PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH" + PATH="/opt/homebrew/opt/llvm/bin:$PATH" + +Building the Project +-------------------- + +Once the environment is set up, you can start the build process using LLVM. Run +the following commands to initiate the build: + +.. code-block:: shell + + make LLVM=1 allyesconfig + make LLVM=1 -j$(nproc) + +Supported in macOS +~~~~~~~~~~~~~~~~~~ + +At the moment, only arm64 is supported and tested with `allyesconfig` Makefile +configuration target. Other Kconfig options not included in `allyesconfig` +target and architectures may be supported as well as support in macOS is based +on LLVM effort and maintenance. + Getting Help ------------