| Message ID | 20240926130159.19e6d0e2@rorschach.local.home (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Documentation/tracing: Mention that RESET_ATTACK_MITIGATION can clear memory | expand |
Hi, On 26-Sep-24 7:01 PM, Steven Rostedt wrote: > From: Steven Rostedt <rostedt@goodmis.org> > > At the 2024 Linux Plumbers Conference, I was talking with Hans de Goede > about the persistent buffer to display traces from previous boots. He > mentioned that UEFI can clear memory. In my own tests I have not seen > this. He later informed me that it requires the config option: > > CONFIG_RESET_ATTACK_MITIGATION > > It appears that setting this will allow the memory to be cleared on boot > up, which will definitely clear out the trace of the previous boot. > > Add this information under the trace_instance in kernel-parameters.txt > to let people know that this can cause issues. > > Link: https://lore.kernel.org/all/20170825155019.6740-2-ard.biesheuvel@linaro.org/ > > Reported-by: Hans de Goede <hdegoede@redhat.com> > Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> Thanks, patch looks good to me: Reviewed-by: Hans de Goede <hdegoede@redhat.com> Regards, Hans > --- > Documentation/admin-guide/kernel-parameters.txt | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index bb48ae24ae69..f9b79294f84a 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -6850,6 +6850,9 @@ > > reserve_mem=12M:4096:trace trace_instance=boot_map^traceoff^traceprintk@trace,sched,irq > > + Note, CONFIG_RESET_ATTACK_MITIGATION can force a memory reset on boot which > + will clear any trace that was stored. > + > See also Documentation/trace/debugging.rst > >
On Thu, 26 Sep 2024 19:54:48 +0200 Hans de Goede <hdegoede@redhat.com> wrote: > Hi, > > On 26-Sep-24 7:01 PM, Steven Rostedt wrote: > > From: Steven Rostedt <rostedt@goodmis.org> > > > > At the 2024 Linux Plumbers Conference, I was talking with Hans de Goede > > about the persistent buffer to display traces from previous boots. He > > mentioned that UEFI can clear memory. In my own tests I have not seen > > this. He later informed me that it requires the config option: > > > > CONFIG_RESET_ATTACK_MITIGATION > > > > It appears that setting this will allow the memory to be cleared on boot > > up, which will definitely clear out the trace of the previous boot. > > > > Add this information under the trace_instance in kernel-parameters.txt > > to let people know that this can cause issues. > > > > Link: https://lore.kernel.org/all/20170825155019.6740-2-ard.biesheuvel@linaro.org/ > > > > Reported-by: Hans de Goede <hdegoede@redhat.com> > > Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> > > Thanks, patch looks good to me: > > Reviewed-by: Hans de Goede <hdegoede@redhat.com> Thanks, And I forgot to send this to the Documentation maintainers :-p Jon, could you take this? Do you need me to resend, or can you just pull it from lore? https://lore.kernel.org/linux-trace-kernel/20240926130159.19e6d0e2@rorschach.local.home/ -- Steve > > Regards, > > Hans > > > > > --- > > Documentation/admin-guide/kernel-parameters.txt | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > > index bb48ae24ae69..f9b79294f84a 100644 > > --- a/Documentation/admin-guide/kernel-parameters.txt > > +++ b/Documentation/admin-guide/kernel-parameters.txt > > @@ -6850,6 +6850,9 @@ > > > > reserve_mem=12M:4096:trace trace_instance=boot_map^traceoff^traceprintk@trace,sched,irq > > > > + Note, CONFIG_RESET_ATTACK_MITIGATION can force a memory reset on boot which > > + will clear any trace that was stored. > > + > > See also Documentation/trace/debugging.rst > > > >
Steven Rostedt <rostedt@goodmis.org> writes: > On Thu, 26 Sep 2024 19:54:48 +0200 > Hans de Goede <hdegoede@redhat.com> wrote: > >> Hi, >> >> On 26-Sep-24 7:01 PM, Steven Rostedt wrote: >> > From: Steven Rostedt <rostedt@goodmis.org> >> > >> > At the 2024 Linux Plumbers Conference, I was talking with Hans de Goede >> > about the persistent buffer to display traces from previous boots. He >> > mentioned that UEFI can clear memory. In my own tests I have not seen >> > this. He later informed me that it requires the config option: >> > >> > CONFIG_RESET_ATTACK_MITIGATION >> > >> > It appears that setting this will allow the memory to be cleared on boot >> > up, which will definitely clear out the trace of the previous boot. >> > >> > Add this information under the trace_instance in kernel-parameters.txt >> > to let people know that this can cause issues. >> > >> > Link: https://lore.kernel.org/all/20170825155019.6740-2-ard.biesheuvel@linaro.org/ >> > >> > Reported-by: Hans de Goede <hdegoede@redhat.com> >> > Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> >> >> Thanks, patch looks good to me: >> >> Reviewed-by: Hans de Goede <hdegoede@redhat.com> > > Thanks, > > And I forgot to send this to the Documentation maintainers :-p > > Jon, could you take this? Do you need me to resend, or can you just pull it > from lore? I'll grab it in a bit. jon
On Mon, 30 Sep 2024 11:20:48 -0600 Jonathan Corbet <corbet@lwn.net> wrote: > > And I forgot to send this to the Documentation maintainers :-p > > > > Jon, could you take this? Do you need me to resend, or can you just pull it > > from lore? > > I'll grab it in a bit. No rush. Thanks! -- Steve
On Thu, 26 Sept 2024 at 19:02, Steven Rostedt <rostedt@goodmis.org> wrote: > > From: Steven Rostedt <rostedt@goodmis.org> > > At the 2024 Linux Plumbers Conference, I was talking with Hans de Goede > about the persistent buffer to display traces from previous boots. He > mentioned that UEFI can clear memory. In my own tests I have not seen > this. He later informed me that it requires the config option: > > CONFIG_RESET_ATTACK_MITIGATION > > It appears that setting this will allow the memory to be cleared on boot > up, which will definitely clear out the trace of the previous boot. > > Add this information under the trace_instance in kernel-parameters.txt > to let people know that this can cause issues. > > Link: https://lore.kernel.org/all/20170825155019.6740-2-ard.biesheuvel@linaro.org/ > > Reported-by: Hans de Goede <hdegoede@redhat.com> > Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> > --- > Documentation/admin-guide/kernel-parameters.txt | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index bb48ae24ae69..f9b79294f84a 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -6850,6 +6850,9 @@ > > reserve_mem=12M:4096:trace trace_instance=boot_map^traceoff^traceprintk@trace,sched,irq > > + Note, CONFIG_RESET_ATTACK_MITIGATION can force a memory reset on boot which > + will clear any trace that was stored. > + CONFIG_RESET_ATTACK_MITIGATION can force a wipe of system RAM at warm reboot on systems that have a TPM enabled, but disabling it does not prevent it. Also, there are many other reasons why the trace buffer region may be wiped and/or reused for other purposes, so singling out CONFIG_RESET_ATTACK_MITIGATION like this is not that useful imo. As I indicated when this feature was under review, it should be made very clear that any kernel side changes that affect the system's behavior in this regard should not be considered regressions. So instead of mentioning CONFIG_RESET_ATTACK_MITIGATION here, perhaps it would be better to document that system firmware generally makes no guarantees about preserving memory contents, and so this feature may break without warning. Note that on UEFI systems, there is a so-called 'capsule' interface which does allow the OS to preserve an arbitrary buffer across a warm reboot. However, whether it works on all systems is anybody's guess.
Hi, On 1-Oct-24 8:17 AM, Ard Biesheuvel wrote: > On Thu, 26 Sept 2024 at 19:02, Steven Rostedt <rostedt@goodmis.org> wrote: >> >> From: Steven Rostedt <rostedt@goodmis.org> >> >> At the 2024 Linux Plumbers Conference, I was talking with Hans de Goede >> about the persistent buffer to display traces from previous boots. He >> mentioned that UEFI can clear memory. In my own tests I have not seen >> this. He later informed me that it requires the config option: >> >> CONFIG_RESET_ATTACK_MITIGATION >> >> It appears that setting this will allow the memory to be cleared on boot >> up, which will definitely clear out the trace of the previous boot. >> >> Add this information under the trace_instance in kernel-parameters.txt >> to let people know that this can cause issues. >> >> Link: https://lore.kernel.org/all/20170825155019.6740-2-ard.biesheuvel@linaro.org/ >> >> Reported-by: Hans de Goede <hdegoede@redhat.com> >> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> >> --- >> Documentation/admin-guide/kernel-parameters.txt | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt >> index bb48ae24ae69..f9b79294f84a 100644 >> --- a/Documentation/admin-guide/kernel-parameters.txt >> +++ b/Documentation/admin-guide/kernel-parameters.txt >> @@ -6850,6 +6850,9 @@ >> >> reserve_mem=12M:4096:trace trace_instance=boot_map^traceoff^traceprintk@trace,sched,irq >> >> + Note, CONFIG_RESET_ATTACK_MITIGATION can force a memory reset on boot which >> + will clear any trace that was stored. >> + > > CONFIG_RESET_ATTACK_MITIGATION can force a wipe of system RAM at warm > reboot on systems that have a TPM enabled, but disabling it does not > prevent it. Also, there are many other reasons why the trace buffer > region may be wiped and/or reused for other purposes, so singling out > CONFIG_RESET_ATTACK_MITIGATION like this is not that useful imo. Since the userspace parts to clear the CONFIG_RESET_ATTACK_MITIGATION related EFI variable after cleaning cryptographic keys from RAM has never materialized CONFIG_RESET_ATTACK_MITIGATION is pretty much guaranteed to clear any traces on any modern machine (and at least in Fedora's kernel config it is disabled because of this). I agree that there are more ways the RAM might get cleared, but since this will clear the RAM almost 100% of the time it is worth documenting this IMHO. I get the feeling you (Ard) see documenting this as some sorta bug report against CONFIG_RESET_ATTACK_MITIGATION, that is not the intention. Quite the opposite the documentation is there to let the user know that CONFIG_RESET_ATTACK_MITIGATION works as advertised and that it will (almost) always clear the RAM on reboot and thus conflicts with keeping traces over reboot. Regards, Hans
Hi Hans, On Tue, 1 Oct 2024 at 10:57, Hans de Goede <hdegoede@redhat.com> wrote: > > Hi, > > On 1-Oct-24 8:17 AM, Ard Biesheuvel wrote: > > On Thu, 26 Sept 2024 at 19:02, Steven Rostedt <rostedt@goodmis.org> wrote: > >> > >> From: Steven Rostedt <rostedt@goodmis.org> > >> > >> At the 2024 Linux Plumbers Conference, I was talking with Hans de Goede > >> about the persistent buffer to display traces from previous boots. He > >> mentioned that UEFI can clear memory. In my own tests I have not seen > >> this. He later informed me that it requires the config option: > >> > >> CONFIG_RESET_ATTACK_MITIGATION > >> > >> It appears that setting this will allow the memory to be cleared on boot > >> up, which will definitely clear out the trace of the previous boot. > >> > >> Add this information under the trace_instance in kernel-parameters.txt > >> to let people know that this can cause issues. > >> > >> Link: https://lore.kernel.org/all/20170825155019.6740-2-ard.biesheuvel@linaro.org/ > >> > >> Reported-by: Hans de Goede <hdegoede@redhat.com> > >> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> > >> --- > >> Documentation/admin-guide/kernel-parameters.txt | 3 +++ > >> 1 file changed, 3 insertions(+) > >> > >> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > >> index bb48ae24ae69..f9b79294f84a 100644 > >> --- a/Documentation/admin-guide/kernel-parameters.txt > >> +++ b/Documentation/admin-guide/kernel-parameters.txt > >> @@ -6850,6 +6850,9 @@ > >> > >> reserve_mem=12M:4096:trace trace_instance=boot_map^traceoff^traceprintk@trace,sched,irq > >> > >> + Note, CONFIG_RESET_ATTACK_MITIGATION can force a memory reset on boot which > >> + will clear any trace that was stored. > >> + > > > > CONFIG_RESET_ATTACK_MITIGATION can force a wipe of system RAM at warm > > reboot on systems that have a TPM enabled, but disabling it does not > > prevent it. Also, there are many other reasons why the trace buffer > > region may be wiped and/or reused for other purposes, so singling out > > CONFIG_RESET_ATTACK_MITIGATION like this is not that useful imo. > > Since the userspace parts to clear the CONFIG_RESET_ATTACK_MITIGATION > related EFI variable after cleaning cryptographic keys from RAM has > never materialized CONFIG_RESET_ATTACK_MITIGATION is pretty much > guaranteed to clear any traces on any modern machine (and at least > in Fedora's kernel config it is disabled because of this). > Any modern x86 PC, sure. > I agree that there are more ways the RAM might get cleared, but > since this will clear the RAM almost 100% of the time it is worth > documenting this IMHO. > Fair enough. > I get the feeling you (Ard) see documenting this as some sorta bug > report against CONFIG_RESET_ATTACK_MITIGATION, that is not the intention. > Quite the opposite the documentation is there to let the user know > that CONFIG_RESET_ATTACK_MITIGATION works as advertised and that it > will (almost) always clear the RAM on reboot and thus conflicts with > keeping traces over reboot. > I am not against documenting this. CONFIG_RESET_ATTACK_MITIGATION is simply incompatible with this feature so it makes sense calling that out. But there are so many other ways in which the trace buffer might get clobbered, and mentioning CONFIG_RESET_ATTACK_MITIGATION in particular might suggest to some that those other causes are due to kernel bugs. All I am asking for is a line in the documentation that says that clobbered trace buffers could occur at any time, regardless of whether CONFIG_RESET_ATTACK_MITIGATION is enabled or not. That way, we have something to refer to when people start filing bugs against the EFI component when this breaks.
On Tue, 1 Oct 2024 11:35:53 +0200 Ard Biesheuvel <ardb@kernel.org> wrote: > All I am asking for is a line in the documentation that says that > clobbered trace buffers could occur at any time, regardless of whether > CONFIG_RESET_ATTACK_MITIGATION is enabled or not. That way, we have > something to refer to when people start filing bugs against the EFI > component when this breaks. How about if I change the comment to this? Note, saving the trace buffer across reboots does require that the system is set up to not wipe memory. For instance, CONFIG_RESET_ATTACK_MITIGATION can force a memory reset on boot which will clear any trace that was stored. This is just one of many ways that can clear memory. Make sure your system keeps the content of memory across reboots before relying on this option. Would that be better? I want to stress that this doesn't work for every setup. It just so happens to work for mine and others. I do not want people thinking that it's a bug if it doesn't work for them. -- Steve
On Tue, 1 Oct 2024 at 14:52, Steven Rostedt <rostedt@goodmis.org> wrote: > > On Tue, 1 Oct 2024 11:35:53 +0200 > Ard Biesheuvel <ardb@kernel.org> wrote: > > > All I am asking for is a line in the documentation that says that > > clobbered trace buffers could occur at any time, regardless of whether > > CONFIG_RESET_ATTACK_MITIGATION is enabled or not. That way, we have > > something to refer to when people start filing bugs against the EFI > > component when this breaks. > > How about if I change the comment to this? > > Note, saving the trace buffer across reboots does require that the system > is set up to not wipe memory. For instance, CONFIG_RESET_ATTACK_MITIGATION > can force a memory reset on boot which will clear any trace that was stored. > This is just one of many ways that can clear memory. Make sure your system > keeps the content of memory across reboots before relying on this option. > > Would that be better? > Yes, that looks much better. Thanks. Acked-by: Ard Biesheuvel <ardb@kernel.org> > I want to stress that this doesn't work for every setup. It just so happens > to work for mine and others. I do not want people thinking that it's a bug > if it doesn't work for them. Exactly. I can see how this feature is very useful if it happens to work on your system firmware, but it should be documented as not being covered by the 'no user visible regressions' policy.
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index bb48ae24ae69..f9b79294f84a 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -6850,6 +6850,9 @@ reserve_mem=12M:4096:trace trace_instance=boot_map^traceoff^traceprintk@trace,sched,irq + Note, CONFIG_RESET_ATTACK_MITIGATION can force a memory reset on boot which + will clear any trace that was stored. + See also Documentation/trace/debugging.rst