| Message ID | 20230823132816.1011054-1-Quirin.Gylstorff@siemens.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [isar-cip-core] initramfs-verity-hook: Ensure sync on rebuild | expand |
On 23.08.23 15:28, Quirin Gylstorff wrote: > From: Quirin Gylstorff <quirin.gylstorff@siemens.com> > > Disable the timestamp of the do_install() task to > re-execute when ever a dependency on do_install > is executed. > > Without this change the sstate needs to be clean for > every rebuild. > > Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com> > --- > .../initramfs-verity-hook/initramfs-verity-hook_0.1.bb | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb b/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb > index 3fc63ed..7435649 100644 > --- a/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb > +++ b/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb > @@ -40,6 +40,9 @@ do_install[cleandirs] += " \ > ${D}/usr/share/initramfs-tools/scripts/local-top \ > ${D}/usr/share/initramfs-tools/conf-hooks.d" > > +# Ensure VERITY_ENV_FILE is updated also when rebuilding initramfs-verity-hook > +do_install[nostamp] = "1" > + > do_install() { > # Insert the veritysetup commandline into the script > if [ -f "${VERITY_ENV_FILE}" ]; then No other dpkg-raw packages require this, do they? What makes initramfs packages special here? And what about those in isar upstream? Jan
On 8/28/23 08:47, Jan Kiszka wrote: > On 23.08.23 15:28, Quirin Gylstorff wrote: >> From: Quirin Gylstorff <quirin.gylstorff@siemens.com> >> >> Disable the timestamp of the do_install() task to >> re-execute when ever a dependency on do_install >> is executed. >> >> Without this change the sstate needs to be clean for >> every rebuild. >> >> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com> >> --- >> .../initramfs-verity-hook/initramfs-verity-hook_0.1.bb | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb b/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb >> index 3fc63ed..7435649 100644 >> --- a/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb >> +++ b/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb >> @@ -40,6 +40,9 @@ do_install[cleandirs] += " \ >> ${D}/usr/share/initramfs-tools/scripts/local-top \ >> ${D}/usr/share/initramfs-tools/conf-hooks.d" >> >> +# Ensure VERITY_ENV_FILE is updated also when rebuilding initramfs-verity-hook >> +do_install[nostamp] = "1" >> + >> do_install() { >> # Insert the veritysetup commandline into the script >> if [ -f "${VERITY_ENV_FILE}" ]; then > > No other dpkg-raw packages require this, do they? What makes initramfs > packages special here? And what about those in isar upstream? > This is necessary for the initramfs package which contain the checksum/uuid from the root file system build. This ensure that the checksum/uuid is refreshed. It fixes the problem that an rebuild of a verity image leads to a broken boot after the build. I will send a v2 to clarify. Quirin
On 12.09.23 15:37, Gylstorff Quirin wrote: > > > On 8/28/23 08:47, Jan Kiszka wrote: >> On 23.08.23 15:28, Quirin Gylstorff wrote: >>> From: Quirin Gylstorff <quirin.gylstorff@siemens.com> >>> >>> Disable the timestamp of the do_install() task to >>> re-execute when ever a dependency on do_install >>> is executed. >>> >>> Without this change the sstate needs to be clean for >>> every rebuild. >>> >>> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com> >>> --- >>> .../initramfs-verity-hook/initramfs-verity-hook_0.1.bb | 3 +++ >>> 1 file changed, 3 insertions(+) >>> >>> diff --git >>> a/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb b/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb >>> index 3fc63ed..7435649 100644 >>> --- >>> a/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb >>> +++ >>> b/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb >>> @@ -40,6 +40,9 @@ do_install[cleandirs] += " \ >>> ${D}/usr/share/initramfs-tools/scripts/local-top \ >>> ${D}/usr/share/initramfs-tools/conf-hooks.d" >>> +# Ensure VERITY_ENV_FILE is updated also when rebuilding >>> initramfs-verity-hook >>> +do_install[nostamp] = "1" >>> + >>> do_install() { >>> # Insert the veritysetup commandline into the script >>> if [ -f "${VERITY_ENV_FILE}" ]; then >> >> No other dpkg-raw packages require this, do they? What makes initramfs >> packages special here? And what about those in isar upstream? >> > > This is necessary for the initramfs package which contain the > checksum/uuid from the root file system build. > This ensure that the checksum/uuid is refreshed. > > It fixes the problem that an rebuild of a verity image leads to a broken > boot after the build. And what will happen when the user decides to pin the UUID to fix value in order to have reproducibility? Won't that cause a needless re-run? Shouldn't this recipe rather make sure to depend on the UUID value? Jan
On Tue, 2023-09-12 at 15:55 +0530, Jan Kiszka wrote: > On 12.09.23 15:37, Gylstorff Quirin wrote: > > > > > > On 8/28/23 08:47, Jan Kiszka wrote: > > > On 23.08.23 15:28, Quirin Gylstorff wrote: > > > > From: Quirin Gylstorff <quirin.gylstorff@siemens.com> > > > > > > > > Disable the timestamp of the do_install() task to > > > > re-execute when ever a dependency on do_install > > > > is executed. > > > > > > > > Without this change the sstate needs to be clean for > > > > every rebuild. > > > > > > > > Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com> > > > > --- > > > > .../initramfs-verity-hook/initramfs-verity- > > > > hook_0.1.bb | 3 +++ > > > > 1 file changed, 3 insertions(+) > > > > > > > > diff --git > > > > a/recipes-initramfs/initramfs-verity-hook/initramfs-verity- > > > > hook_0.1.bb b/recipes-initramfs/initramfs-verity- > > > > hook/initramfs-verity-hook_0.1.bb > > > > index 3fc63ed..7435649 100644 > > > > --- > > > > a/recipes-initramfs/initramfs-verity-hook/initramfs-verity- > > > > hook_0.1.bb > > > > +++ > > > > b/recipes-initramfs/initramfs-verity-hook/initramfs-verity- > > > > hook_0.1.bb > > > > @@ -40,6 +40,9 @@ do_install[cleandirs] += " \ > > > > ${D}/usr/share/initramfs-tools/scripts/local-top \ > > > > ${D}/usr/share/initramfs-tools/conf-hooks.d" > > > > +# Ensure VERITY_ENV_FILE is updated also when rebuilding > > > > initramfs-verity-hook > > > > +do_install[nostamp] = "1" > > > > + > > > > do_install() { > > > > # Insert the veritysetup commandline into the script > > > > if [ -f "${VERITY_ENV_FILE}" ]; then > > > > > > No other dpkg-raw packages require this, do they? What makes > > > initramfs > > > packages special here? And what about those in isar upstream? > > > > > > > This is necessary for the initramfs package which contain the > > checksum/uuid from the root file system build. > > This ensure that the checksum/uuid is refreshed. > > > > It fixes the problem that an rebuild of a verity image leads to a > > broken > > boot after the build. > > And what will happen when the user decides to pin the UUID to fix > value > in order to have reproducibility? Won't that cause a needless re-run? > Shouldn't this recipe rather make sure to depend on the UUID value? This is a general reproducibility topic: The task checksums are computed at parsing time and by that the tasks that need to be re-run are selected. However, this is based on the assumption that tasks are deterministic, i.e. same input data also generates same output data. Unfortunately we are still not 100% reproducible. By that this pre- condition is not met. For normal packages this does not really matter, but for the dm-verity checksum this very well matters. I'm just wondering if we could work around that by making this component non-cacheable. Putting Adriaan in CC for the sstate part. Felix > > Jan >
On 9/13/23 03:26, MOESSBAUER, Felix (T CED INW-CN) wrote: > On Tue, 2023-09-12 at 15:55 +0530, Jan Kiszka wrote: >> On 12.09.23 15:37, Gylstorff Quirin wrote: >>> >>> >>> On 8/28/23 08:47, Jan Kiszka wrote: >>>> On 23.08.23 15:28, Quirin Gylstorff wrote: >>>>> From: Quirin Gylstorff <quirin.gylstorff@siemens.com> >>>>> >>>>> Disable the timestamp of the do_install() task to >>>>> re-execute when ever a dependency on do_install >>>>> is executed. >>>>> >>>>> Without this change the sstate needs to be clean for >>>>> every rebuild. >>>>> >>>>> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com> >>>>> --- >>>>> .../initramfs-verity-hook/initramfs-verity- >>>>> hook_0.1.bb | 3 +++ >>>>> 1 file changed, 3 insertions(+) >>>>> >>>>> diff --git >>>>> a/recipes-initramfs/initramfs-verity-hook/initramfs-verity- >>>>> hook_0.1.bb b/recipes-initramfs/initramfs-verity- >>>>> hook/initramfs-verity-hook_0.1.bb >>>>> index 3fc63ed..7435649 100644 >>>>> --- >>>>> a/recipes-initramfs/initramfs-verity-hook/initramfs-verity- >>>>> hook_0.1.bb >>>>> +++ >>>>> b/recipes-initramfs/initramfs-verity-hook/initramfs-verity- >>>>> hook_0.1.bb >>>>> @@ -40,6 +40,9 @@ do_install[cleandirs] += " \ >>>>> ${D}/usr/share/initramfs-tools/scripts/local-top \ >>>>> ${D}/usr/share/initramfs-tools/conf-hooks.d" >>>>> +# Ensure VERITY_ENV_FILE is updated also when rebuilding >>>>> initramfs-verity-hook >>>>> +do_install[nostamp] = "1" >>>>> + >>>>> do_install() { >>>>> # Insert the veritysetup commandline into the script >>>>> if [ -f "${VERITY_ENV_FILE}" ]; then >>>> >>>> No other dpkg-raw packages require this, do they? What makes >>>> initramfs >>>> packages special here? And what about those in isar upstream? >>>> >>> >>> This is necessary for the initramfs package which contain the >>> checksum/uuid from the root file system build. >>> This ensure that the checksum/uuid is refreshed. >>> >>> It fixes the problem that an rebuild of a verity image leads to a >>> broken >>> boot after the build. >> >> And what will happen when the user decides to pin the UUID to fix >> value >> in order to have reproducibility? Won't that cause a needless re-run? >> Shouldn't this recipe rather make sure to depend on the UUID value? > Even with pinning and same salt and uuid veritysetup will generate a new roothash during the build. Quirin > This is a general reproducibility topic: The task checksums are > computed at parsing time and by that the tasks that need to be re-run > are selected. However, this is based on the assumption that tasks are > deterministic, i.e. same input data also generates same output data. > > Unfortunately we are still not 100% reproducible. By that this pre- > condition is not met. For normal packages this does not really matter, > but for the dm-verity checksum this very well matters. > > I'm just wondering if we could work around that by making this > component non-cacheable. Putting Adriaan in CC for the sstate part. The same solution as is used by meta-security for Yocto[1]. [1]: https://lists.yoctoproject.org/g/yocto/message/50624 Quirin > > Felix > >> >> Jan >> >
On 13.09.23 18:21, Gylstorff Quirin wrote: > > > On 9/13/23 03:26, MOESSBAUER, Felix (T CED INW-CN) wrote: >> On Tue, 2023-09-12 at 15:55 +0530, Jan Kiszka wrote: >>> On 12.09.23 15:37, Gylstorff Quirin wrote: >>>> >>>> >>>> On 8/28/23 08:47, Jan Kiszka wrote: >>>>> On 23.08.23 15:28, Quirin Gylstorff wrote: >>>>>> From: Quirin Gylstorff <quirin.gylstorff@siemens.com> >>>>>> >>>>>> Disable the timestamp of the do_install() task to >>>>>> re-execute when ever a dependency on do_install >>>>>> is executed. >>>>>> >>>>>> Without this change the sstate needs to be clean for >>>>>> every rebuild. >>>>>> >>>>>> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com> >>>>>> --- >>>>>> .../initramfs-verity-hook/initramfs-verity- >>>>>> hook_0.1.bb | 3 +++ >>>>>> 1 file changed, 3 insertions(+) >>>>>> >>>>>> diff --git >>>>>> a/recipes-initramfs/initramfs-verity-hook/initramfs-verity- >>>>>> hook_0.1.bb b/recipes-initramfs/initramfs-verity- >>>>>> hook/initramfs-verity-hook_0.1.bb >>>>>> index 3fc63ed..7435649 100644 >>>>>> --- >>>>>> a/recipes-initramfs/initramfs-verity-hook/initramfs-verity- >>>>>> hook_0.1.bb >>>>>> +++ >>>>>> b/recipes-initramfs/initramfs-verity-hook/initramfs-verity- >>>>>> hook_0.1.bb >>>>>> @@ -40,6 +40,9 @@ do_install[cleandirs] += " \ >>>>>> ${D}/usr/share/initramfs-tools/scripts/local-top \ >>>>>> ${D}/usr/share/initramfs-tools/conf-hooks.d" >>>>>> +# Ensure VERITY_ENV_FILE is updated also when rebuilding >>>>>> initramfs-verity-hook >>>>>> +do_install[nostamp] = "1" >>>>>> + >>>>>> do_install() { >>>>>> # Insert the veritysetup commandline into the script >>>>>> if [ -f "${VERITY_ENV_FILE}" ]; then >>>>> >>>>> No other dpkg-raw packages require this, do they? What makes >>>>> initramfs >>>>> packages special here? And what about those in isar upstream? >>>>> >>>> >>>> This is necessary for the initramfs package which contain the >>>> checksum/uuid from the root file system build. >>>> This ensure that the checksum/uuid is refreshed. >>>> >>>> It fixes the problem that an rebuild of a verity image leads to a >>>> broken >>>> boot after the build. >>> >>> And what will happen when the user decides to pin the UUID to fix >>> value >>> in order to have reproducibility? Won't that cause a needless re-run? >>> Shouldn't this recipe rather make sure to depend on the UUID value? >> > Even with pinning and same salt and uuid veritysetup will generate a new > roothash during the build. > That sounds like other reproducibility issues we have, eg. with mkfs.* Can't that be fixed? I think it has to, otherwise dm-verity images will never be fully reproducible. > Quirin > > >> This is a general reproducibility topic: The task checksums are >> computed at parsing time and by that the tasks that need to be re-run >> are selected. However, this is based on the assumption that tasks are >> deterministic, i.e. same input data also generates same output data. >> >> Unfortunately we are still not 100% reproducible. By that this pre- >> condition is not met. For normal packages this does not really matter, >> but for the dm-verity checksum this very well matters. >> > >> I'm just wondering if we could work around that by making this >> component non-cacheable. Putting Adriaan in CC for the sstate part. > > The same solution as is used by meta-security for Yocto[1]. > > [1]: https://lists.yoctoproject.org/g/yocto/message/50624 Interesting - Yocto is NOT reproducible when it comes to verity... Jan
diff --git a/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb b/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb index 3fc63ed..7435649 100644 --- a/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb +++ b/recipes-initramfs/initramfs-verity-hook/initramfs-verity-hook_0.1.bb @@ -40,6 +40,9 @@ do_install[cleandirs] += " \ ${D}/usr/share/initramfs-tools/scripts/local-top \ ${D}/usr/share/initramfs-tools/conf-hooks.d" +# Ensure VERITY_ENV_FILE is updated also when rebuilding initramfs-verity-hook +do_install[nostamp] = "1" + do_install() { # Insert the veritysetup commandline into the script if [ -f "${VERITY_ENV_FILE}" ]; then