| Message ID | 20241218-objtool-strict-v2-0-a5297c961434@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | objtool: Add option to fail build on vmlinux warnings | expand |
On Wed, 18 Dec 2024 14:58:54 +0000 Brendan Jackman <jackmanb@google.com> wrote: > This adds an option to objtool to exit with an error when it enounters > warnings. Why is it optional? Can we always fail the build so stuff gets fixed?
On Wed, Dec 18, 2024 at 04:06:56PM -0800, Andrew Morton wrote: > On Wed, 18 Dec 2024 14:58:54 +0000 Brendan Jackman <jackmanb@google.com> wrote: > > > This adds an option to objtool to exit with an error when it enounters > > warnings. > > Why is it optional? Can we always fail the build so stuff gets fixed? Eventually, yes. But right now, I'm not so sure... it will likely break a lot of robot builds, there are some obscure randconfigs out there triggering some "interesting" edge cases, e.g.: - ftrace branch profiling which is fundamentally incompatible with uaccess STAC/CLAC rules - similar weirdnesses with UBSAN/KASAN/etc and other compiler features/plugins - obscure toolchain bugs in certain compiler versions which do weird things with control flow. some of these bugs break the kernel, some don't. Problem is, it usually falls on the objtool maintainers to figure out the root of the problem and the resolution, neither of which is necessarily straightforward, especially the latter. There's only two of us maintainers at the moment, with limited bandwidth. So yes, it *should* always fail the build. But unless we get more maintainer bandwidth, I don't think we're ready for that. We might end up being able to make CONFIG_OBJTOOL_WERROR=y the default, and then just require broken features to depend on CONFIG_OBJTOOL_WERROR=n. And then print a big fat warning message at build and/or runtime in the case of warnings. We also might need to add some features, like a way to mark certain compiler versions as bad, or a way to silence objtool warnings for certain known harmless cases, or improve the specificity and usefulness of certain vague warnings. But as a first step I'll planning on just throwing these patches on a robot-monitored branch with CONFIG_OBJTOOL_WERROR=y over the holidays to see how bad the damage is.
Hi Josh, On Wed, Dec 18, 2024 at 05:00:54PM -0800, Josh Poimboeuf wrote: ... > - obscure toolchain bugs in certain compiler versions which do weird > things with control flow. some of these bugs break the kernel, some > don't. > > Problem is, it usually falls on the objtool maintainers to figure out > the root of the problem and the resolution, neither of which is > necessarily straightforward, especially the latter. There's only two of > us maintainers at the moment, with limited bandwidth. > > So yes, it *should* always fail the build. But unless we get more > maintainer bandwidth, I don't think we're ready for that. > > We might end up being able to make CONFIG_OBJTOOL_WERROR=y the default, > and then just require broken features to depend on > CONFIG_OBJTOOL_WERROR=n. And then print a big fat warning message at > build and/or runtime in the case of warnings. > > We also might need to add some features, like a way to mark certain > compiler versions as bad, or a way to silence objtool warnings for > certain known harmless cases, or improve the specificity and usefulness > of certain vague warnings. > > But as a first step I'll planning on just throwing these patches on a > robot-monitored branch with CONFIG_OBJTOOL_WERROR=y over the holidays to > see how bad the damage is. For the record, I plan to monitor these reports for LLVM and try to investigate and triage all other known objtool warnings for LLVM after the holidays to try and prepare for this. I felt blind sided by the compiler -Werror change so I'd rather not go through that again :) one reason I would like to be objtool clean is to catch changed compiler behavior quicker, as I tend to notice it is easier to get problems addressed when the problem is reported as close as possible to the original change. I do agree with you that figuring our the root problem and resolution to some of these warnings is not always the easiest, especially when they are on the toolchain side, so I have often kicked the can down the road. I know there is some documentation in objtool.txt around various warnings, is that pretty up to date/accurate? Are there any other resources I could look at to help with this work? I know Arnd just recently fixed a set [1] that I saw in our builds as well due to a bare unreachable(), which I think tend to hurt Clang more than GCC but maybe I am imagining things there. Some objtool reports get sent to only llvm@lists.linux.dev when clang is involved (due to a historical filter IIRC, I cannot find the original request), so you may want to glance at [2] to see if anything new pops up. [1]: https://git.kernel.org/netdev/net/c/cff865c700711ecc3824b2dfe181637f3ed23c80 [2]: https://lore.kernel.org/llvm/?q=objtool+f:lkp@intel.com Cheers, Nathan
On Thu, Dec 19, 2024 at 03:19:13PM -0700, Nathan Chancellor wrote: > I do agree with you that figuring our the root problem and resolution to > some of these warnings is not always the easiest, especially when they > are on the toolchain side, so I have often kicked the can down the road. > I know there is some documentation in objtool.txt around various > warnings, is that pretty up to date/accurate? Are there any other > resources I could look at to help with this work? I think the document is pretty up to date. Some warnings are self explanatory but others like "unreachable instruction" or "stack state mismatch" do require digging. One thing that can help is to "export OBJTOOL_VERBOSE=1", which will tell objtool to disassemble any affected functions and show a backtrace with all the taken branches leading up to the warning (if applicable). Maybe that should be the default for --Werror. I'd definitely like more people to be able to debug objtool warnings. Any ideas on making that easier or educating people or improving warnings are very welcome. I'll be keeping that in mind when looking at the build errors over the holidays. > Some objtool reports get sent to only llvm@lists.linux.dev when clang is > involved (due to a historical filter IIRC, I cannot find the original > request), so you may want to glance at [2] to see if anything new pops > up. We need to figure out how to get that fixed, the commit author really needs to know if their code causes a warning/error.
On Thu, Dec 19, 2024 at 2:19 PM Nathan Chancellor <nathan@kernel.org> wrote: > > Some objtool reports get sent to only llvm@lists.linux.dev when clang is > involved (due to a historical filter IIRC, I cannot find the original > request), https://lore.kernel.org/all/CAKwvOdmJvWpmbP3GyzaZxyiuwooFXA8D7ui05QE7+f8Oaz+rXg@mail.gmail.com/ perhaps?
On Thu, 19 Dec 2024 at 22:56, Josh Poimboeuf <jpoimboe@kernel.org> wrote: > One thing that can help is to "export OBJTOOL_VERBOSE=1", which will > tell objtool to disassemble any affected functions and show a backtrace > with all the taken branches leading up to the warning (if applicable). > Maybe that should be the default for --Werror. This sounds very useful, I will do this for v3 unless anyone objects. If we're failing the build the risk of a mega-verbose splat seems like a good tradeoff vs the risk of the user having to go and read the objtool code.
On Thu, Dec 19, 2024 at 02:56:42PM -0800, Josh Poimboeuf wrote: > On Thu, Dec 19, 2024 at 03:19:13PM -0700, Nathan Chancellor wrote: > > I do agree with you that figuring our the root problem and resolution to > > some of these warnings is not always the easiest, especially when they > > are on the toolchain side, so I have often kicked the can down the road. > > I know there is some documentation in objtool.txt around various > > warnings, is that pretty up to date/accurate? Are there any other > > resources I could look at to help with this work? > > I think the document is pretty up to date. Some warnings are self > explanatory but others like "unreachable instruction" or "stack state > mismatch" do require digging. > > One thing that can help is to "export OBJTOOL_VERBOSE=1", which will > tell objtool to disassemble any affected functions and show a backtrace > with all the taken branches leading up to the warning (if applicable). > Maybe that should be the default for --Werror. Yeah, that does not sound like a bad idea. If the build is going to break, it seems reasonable to give developers as much pertinent information as possible so they can address the problem properly and move on. > I'd definitely like more people to be able to debug objtool warnings. > Any ideas on making that easier or educating people or improving > warnings are very welcome. I'll be keeping that in mind when looking at > the build errors over the holidays. I will definitely ponder on that as well since I should have a good perspective on that front. > > Some objtool reports get sent to only llvm@lists.linux.dev when clang is > > involved (due to a historical filter IIRC, I cannot find the original > > request), so you may want to glance at [2] to see if anything new pops > > up. > > We need to figure out how to get that fixed, the commit author really > needs to know if their code causes a warning/error. I have started a new thread with the 0day folks to get that adjusted with you and Peter on CC: https://lore.kernel.org/20241220200617.GA936171@ax162/ Cheers, Nathan
This adds an option to objtool to exit with an error when it enounters warnings. Then, it adds a config to enable that flag. This enables you to fail the build e.g. when noinstr is violated. Signed-off-by: Brendan Jackman <jackmanb@google.com> --- Changes in v2: - Renamed flag/config to -Werror/CONFIG_*_WERROR - Applied to all objool runs instead of just vmlinux. - Link to v1: https://lore.kernel.org/r/20241213-objtool-strict-v1-0-fd388f9d971f@google.com --- Brendan Jackman (2): objtool: Add --Werror kbuild: Add option to fail build on vmlinux objtool issues lib/Kconfig.debug | 10 ++++++++++ scripts/Makefile.lib | 1 + tools/objtool/builtin-check.c | 6 ++++++ tools/objtool/check.c | 7 ++----- tools/objtool/include/objtool/builtin.h | 1 + 5 files changed, 20 insertions(+), 5 deletions(-) --- base-commit: f932fb9b40749d1c9a539d89bb3e288c077aafe5 change-id: 20241213-objtool-strict-cb9a0a75139e Best regards,