| Message ID | 20210302185852.5790-1-broonie@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: defconfig: Disable DEBUG_INFO | expand |
Mark Brown <broonie@kernel.org> writes: > We've had DEBUG_INFO enabled for arm64 defconfigs since the initial > commit. This is probably not that frequently used but substantially > inflates the size of the build tree and amount of I/O needed during the > build. This was causing issues with storage usage in some automated CI > environments which don't expect defconfigs to be quite this big, and > generally increases the resource consumption for both them and people > doing local builds. Turn the option off by default to remove these > overheads from the default, people can always enable DEBUG_INFO if they > need it. > > Without this patch my build tree is 6.8G, with it the size drops to 909M. > > Reported-by: Guillaume Tucker <guillaume.tucker@collabora.com> > Signed-off-by: Mark Brown <broonie@kernel.org> > Cc: Kevin Hilman <khilman@baylibre.com> Yes please. Acked-by: Kevin Hilman <khilman@baylibre.com> > --- > arch/arm64/configs/defconfig | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig > index d612f633b771..c246dfc0a175 100644 > --- a/arch/arm64/configs/defconfig > +++ b/arch/arm64/configs/defconfig > @@ -1155,7 +1155,6 @@ CONFIG_CRYPTO_DEV_HISI_HPRE=m > CONFIG_CRYPTO_DEV_HISI_TRNG=m > CONFIG_CMA_SIZE_MBYTES=32 > CONFIG_PRINTK_TIME=y > -CONFIG_DEBUG_INFO=y > CONFIG_MAGIC_SYSRQ=y > CONFIG_DEBUG_FS=y > CONFIG_DEBUG_KERNEL=y > -- > 2.20.1
On Tue, Mar 02, 2021 at 06:58:52PM +0000, Mark Brown wrote: > We've had DEBUG_INFO enabled for arm64 defconfigs since the initial > commit. This is probably not that frequently used but substantially > inflates the size of the build tree and amount of I/O needed during the > build. This was causing issues with storage usage in some automated CI > environments which don't expect defconfigs to be quite this big, and > generally increases the resource consumption for both them and people > doing local builds. Turn the option off by default to remove these > overheads from the default, people can always enable DEBUG_INFO if they > need it. > > Without this patch my build tree is 6.8G, with it the size drops to 909M. > > Reported-by: Guillaume Tucker <guillaume.tucker@collabora.com> > Signed-off-by: Mark Brown <broonie@kernel.org> > Cc: Kevin Hilman <khilman@baylibre.com> > --- > arch/arm64/configs/defconfig | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig > index d612f633b771..c246dfc0a175 100644 > --- a/arch/arm64/configs/defconfig > +++ b/arch/arm64/configs/defconfig > @@ -1155,7 +1155,6 @@ CONFIG_CRYPTO_DEV_HISI_HPRE=m > CONFIG_CRYPTO_DEV_HISI_TRNG=m > CONFIG_CMA_SIZE_MBYTES=32 > CONFIG_PRINTK_TIME=y > -CONFIG_DEBUG_INFO=y > CONFIG_MAGIC_SYSRQ=y > CONFIG_DEBUG_FS=y > CONFIG_DEBUG_KERNEL=y Hmm. Doing this means ./scripts/faddr2line no longer works with the vmlinux, which means if somebody forgets to enable DEBUG_INFO they're in for a really hard time debugging when something goes wrong. Why can't the CI systems just disable DEBUG_INFO themselves instead of changing defconfig for everybody? Will
On Thu, Mar 04, 2021 at 12:56:24PM +0000, Will Deacon wrote: > Hmm. Doing this means ./scripts/faddr2line no longer works with the vmlinux, > which means if somebody forgets to enable DEBUG_INFO they're in for a > really hard time debugging when something goes wrong. Assuming they're aware of that script in the first place and try to translate addreses with it rather just using the function name in the stack trace, and aren't able to easily rebuild if they decide that it's something that's useful. What people do is going to depend a lot on their use cases. > Why can't the CI systems just disable DEBUG_INFO themselves instead of > changing defconfig for everybody? What's more likely is that they can just increase the amount of space they allocate to jobs (that's certainly what KernelCI does). Testing modified versions of configurations isn't great as half the point of using the standard configurations is that everyone's working to the same thing and should in theory be seeing the same stuff, it's easier to name a standard config than name a standard config and a list of tweaks applied to it. This is about picking a sensible default, there's always going to be cases where someone wants the other value (otherwise it wouldn't be a config option). The contention is that there's a lot more builds being slowed down by the extra I/O and disk space being burned than benefit to people who end up with the debug info turned on and actively use it but these aren't direct tradeoffs so you can't categorically say something one way or the other. At the minute defconfig actually results in a bigger build tree than an allmodconfig for me (6.8G vs 5.2G) which doesn't seem like what I'd expect.
On Thu, Mar 04, 2021 at 02:36:47PM +0000, Mark Brown wrote: > On Thu, Mar 04, 2021 at 12:56:24PM +0000, Will Deacon wrote: > > > Hmm. Doing this means ./scripts/faddr2line no longer works with the vmlinux, > > which means if somebody forgets to enable DEBUG_INFO they're in for a > > really hard time debugging when something goes wrong. > > Assuming they're aware of that script in the first place and try to > translate addreses with it rather just using the function name in the > stack trace, and aren't able to easily rebuild if they decide that it's > something that's useful. What people do is going to depend a lot on > their use cases. I use ./scripts/faddr2line all the time to decode the function+offset from the stacktrace. For example: $ ./scripts/faddr2line vmlinux gic_handle_irq+0x64/0x120 gic_handle_irq+0x64/0x120: gic_handle_irq at drivers/irqchip/irq-gic-v3.c:671 where I pulled "gic_handle_irq+0x64/0x120" straight out of this backtrace: [ 1.947342] Call trace: [ 1.947576] dump_backtrace+0x0/0x1cc [ 1.948304] show_stack+0x18/0x64 [ 1.948637] dump_stack+0xcc/0x14c [ 1.948938] panic+0x14c/0x360 [ 1.949188] sysrq_handle_term+0x0/0x90 [ 1.949645] __handle_sysrq+0x150/0x178 [ 1.950044] handle_sysrq+0x2c/0x38 [ 1.950323] pl011_fifo_to_tty+0x104/0x218 [ 1.950659] pl011_int+0x200/0x484 [ 1.951025] __handle_irq_event_percpu+0x84/0x168 [ 1.951633] handle_irq_event+0x60/0x13c [ 1.952200] handle_fasteoi_irq+0xf4/0x1c8 [ 1.952688] __handle_domain_irq+0x88/0xcc [ 1.953075] gic_handle_irq+0x64/0x120 [ 1.953447] el1_irq+0xb4/0x180 [ 1.953778] arch_cpu_idle+0x18/0x28 [ 1.954109] default_idle_call+0x20/0x5c [ 1.954459] do_idle+0xfc/0x290 [ 1.954709] cpu_startup_entry+0x24/0x28 [ 1.955179] rest_init+0xd8/0xe8 [ 1.955516] arch_call_rest_init+0x10/0x1c [ 1.955929] start_kernel+0x32c/0x44c > > Why can't the CI systems just disable DEBUG_INFO themselves instead of > > changing defconfig for everybody? > > What's more likely is that they can just increase the amount of space > they allocate to jobs (that's certainly what KernelCI does). Testing > modified versions of configurations isn't great as half the point of > using the standard configurations is that everyone's working to the same > thing and should in theory be seeing the same stuff, it's easier to name > a standard config than name a standard config and a list of tweaks > applied to it. I'd be fine if arm64 build reports came back as "defconfig+DEBUG_INFO=n" and the CI just ran ./scripts/config -d DEBUG_INFO as part of its build step. For runtime testing, however, having the full vmlinux available is really helpful if we need to debug. > This is about picking a sensible default, there's always going to be > cases where someone wants the other value (otherwise it wouldn't be a > config option). The contention is that there's a lot more builds being > slowed down by the extra I/O and disk space being burned than benefit to > people who end up with the debug info turned on and actively use it but > these aren't direct tradeoffs so you can't categorically say something > one way or the other. At the minute defconfig actually results in a > bigger build tree than an allmodconfig for me (6.8G vs 5.2G) which > doesn't seem like what I'd expect. I suppose I'm of the opinion that debug info is a waste of time until you need it, and then it's suddenly invaluable. So I'd prefer it to be there by default, as I don't think the extra I/O or disk space is a concern outside of CI. But it would be good to hear what others have to say. Will
On Thu, Mar 04, 2021 at 02:48:34PM +0000, Will Deacon wrote: > On Thu, Mar 04, 2021 at 02:36:47PM +0000, Mark Brown wrote: > > they allocate to jobs (that's certainly what KernelCI does). Testing > > modified versions of configurations isn't great as half the point of > > using the standard configurations is that everyone's working to the same > > thing and should in theory be seeing the same stuff, it's easier to name > > a standard config than name a standard config and a list of tweaks > > applied to it. > I'd be fine if arm64 build reports came back as "defconfig+DEBUG_INFO=n" > and the CI just ran ./scripts/config -d DEBUG_INFO as part of its build > step. For runtime testing, however, having the full vmlinux available is > really helpful if we need to debug. Note that one of the things people are working on at the minute is trying to aggregate results from different systems together. Like I say it's more a case of just allocating more space to these builds rather than some insurmountable thing that can't be handled. > > This is about picking a sensible default, there's always going to be > > cases where someone wants the other value (otherwise it wouldn't be a > > config option). The contention is that there's a lot more builds being > > slowed down by the extra I/O and disk space being burned than benefit to > > people who end up with the debug info turned on and actively use it but > > these aren't direct tradeoffs so you can't categorically say something > > one way or the other. At the minute defconfig actually results in a > > bigger build tree than an allmodconfig for me (6.8G vs 5.2G) which > > doesn't seem like what I'd expect. > I suppose I'm of the opinion that debug info is a waste of time until you > need it, and then it's suddenly invaluable. So I'd prefer it to be there by > default, as I don't think the extra I/O or disk space is a concern outside > of CI. But it would be good to hear what others have to say. FWIW with laptops the I/O cost tends to make a difference to the edit/compile/run time for me, disks are slow and RAM not plentiful. Right now I'm sitting at a rather high speced desktop so the build trees I got the storage numbers from were all in RAM backed tmpfs and it makes very little odds.
On Thu, Mar 04, 2021 at 03:18:53PM +0000, Mark Brown wrote: > On Thu, Mar 04, 2021 at 02:48:34PM +0000, Will Deacon wrote: > > On Thu, Mar 04, 2021 at 02:36:47PM +0000, Mark Brown wrote: > > > they allocate to jobs (that's certainly what KernelCI does). Testing > > > modified versions of configurations isn't great as half the point of > > > using the standard configurations is that everyone's working to the same > > > thing and should in theory be seeing the same stuff, it's easier to name > > > a standard config than name a standard config and a list of tweaks > > > applied to it. > > > I'd be fine if arm64 build reports came back as "defconfig+DEBUG_INFO=n" > > and the CI just ran ./scripts/config -d DEBUG_INFO as part of its build > > step. For runtime testing, however, having the full vmlinux available is > > really helpful if we need to debug. I found DEBUG_INFO pretty useful as well and always hated it in the past when I had to recompile a kernel just to rerun the tests and identify the source/line of an address (I guess that's why we ended up with this in defconfig). > > > This is about picking a sensible default, there's always going to be > > > cases where someone wants the other value (otherwise it wouldn't be a > > > config option). The contention is that there's a lot more builds being > > > slowed down by the extra I/O and disk space being burned than benefit to > > > people who end up with the debug info turned on and actively use it but > > > these aren't direct tradeoffs so you can't categorically say something > > > one way or the other. At the minute defconfig actually results in a > > > bigger build tree than an allmodconfig for me (6.8G vs 5.2G) which > > > doesn't seem like what I'd expect. > > > I suppose I'm of the opinion that debug info is a waste of time until you > > need it, and then it's suddenly invaluable. So I'd prefer it to be there by > > default, as I don't think the extra I/O or disk space is a concern outside > > of CI. But it would be good to hear what others have to say. > > FWIW with laptops the I/O cost tends to make a difference to the > edit/compile/run time for me, disks are slow and RAM not plentiful. > Right now I'm sitting at a rather high speced desktop so the build trees > I got the storage numbers from were all in RAM backed tmpfs and it makes > very little odds. How about enabling DEBUG_INFO_REDUCED as a middle ground? For me the build tree goes from 5.9GB to 2.0GB. I tried faddr2line and still works as expected (only tried gcc, not sure whether clang honours this option).
On Thu, Mar 04, 2021 at 04:22:32PM +0000, Catalin Marinas wrote: > How about enabling DEBUG_INFO_REDUCED as a middle ground? For me the > build tree goes from 5.9GB to 2.0GB. I tried faddr2line and still works > as expected (only tried gcc, not sure whether clang honours this > option). That seems like a good tradeoff for me - it's only 100% bigger than no debug info at all.
Catalin Marinas <catalin.marinas@arm.com> writes: > On Thu, Mar 04, 2021 at 03:18:53PM +0000, Mark Brown wrote: >> On Thu, Mar 04, 2021 at 02:48:34PM +0000, Will Deacon wrote: >> > On Thu, Mar 04, 2021 at 02:36:47PM +0000, Mark Brown wrote: >> > > they allocate to jobs (that's certainly what KernelCI does). Testing >> > > modified versions of configurations isn't great as half the point of >> > > using the standard configurations is that everyone's working to the same >> > > thing and should in theory be seeing the same stuff, it's easier to name >> > > a standard config than name a standard config and a list of tweaks >> > > applied to it. >> >> > I'd be fine if arm64 build reports came back as "defconfig+DEBUG_INFO=n" >> > and the CI just ran ./scripts/config -d DEBUG_INFO as part of its build >> > step. For runtime testing, however, having the full vmlinux available is >> > really helpful if we need to debug. > > I found DEBUG_INFO pretty useful as well and always hated it in the past > when I had to recompile a kernel just to rerun the tests and identify > the source/line of an address (I guess that's why we ended up with this > in defconfig). > >> > > This is about picking a sensible default, there's always going to be >> > > cases where someone wants the other value (otherwise it wouldn't be a >> > > config option). The contention is that there's a lot more builds being >> > > slowed down by the extra I/O and disk space being burned than benefit to >> > > people who end up with the debug info turned on and actively use it but >> > > these aren't direct tradeoffs so you can't categorically say something >> > > one way or the other. At the minute defconfig actually results in a >> > > bigger build tree than an allmodconfig for me (6.8G vs 5.2G) which >> > > doesn't seem like what I'd expect. >> >> > I suppose I'm of the opinion that debug info is a waste of time until you >> > need it, and then it's suddenly invaluable. So I'd prefer it to be there by >> > default, as I don't think the extra I/O or disk space is a concern outside >> > of CI. But it would be good to hear what others have to say. >> >> FWIW with laptops the I/O cost tends to make a difference to the >> edit/compile/run time for me, disks are slow and RAM not plentiful. >> Right now I'm sitting at a rather high speced desktop so the build trees >> I got the storage numbers from were all in RAM backed tmpfs and it makes >> very little odds. > > How about enabling DEBUG_INFO_REDUCED as a middle ground? For me the > build tree goes from 5.9GB to 2.0GB. I tried faddr2line and still works > as expected (only tried gcc, not sure whether clang honours this > option). Sounds great to me. Kevin
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig index d612f633b771..c246dfc0a175 100644 --- a/arch/arm64/configs/defconfig +++ b/arch/arm64/configs/defconfig @@ -1155,7 +1155,6 @@ CONFIG_CRYPTO_DEV_HISI_HPRE=m CONFIG_CRYPTO_DEV_HISI_TRNG=m CONFIG_CMA_SIZE_MBYTES=32 CONFIG_PRINTK_TIME=y -CONFIG_DEBUG_INFO=y CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y CONFIG_DEBUG_KERNEL=y
We've had DEBUG_INFO enabled for arm64 defconfigs since the initial commit. This is probably not that frequently used but substantially inflates the size of the build tree and amount of I/O needed during the build. This was causing issues with storage usage in some automated CI environments which don't expect defconfigs to be quite this big, and generally increases the resource consumption for both them and people doing local builds. Turn the option off by default to remove these overheads from the default, people can always enable DEBUG_INFO if they need it. Without this patch my build tree is 6.8G, with it the size drops to 909M. Reported-by: Guillaume Tucker <guillaume.tucker@collabora.com> Signed-off-by: Mark Brown <broonie@kernel.org> Cc: Kevin Hilman <khilman@baylibre.com> --- arch/arm64/configs/defconfig | 1 - 1 file changed, 1 deletion(-)