| Message ID | 20141031075209.GH3290@darkstar.nay.redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Hi Dave, Thanks for testing. On Fri, 2014-10-31 at 15:52 +0800, Dave Young wrote: > I tested your patches. The macihne is using spin-table cpu enable method > so I tried maxcpus=1 as you suggested. > > There's below issues for me, thoughts? > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > to use. You can supply a DTB with the kexec --dtb option, then kexec will not need /proc/device-tree. Please try if you can and let me know what happens. > 2. adding "acpi=off" to 1st kernel boot cmdline, kexec load fails with error > like below: > machine_apply_elf_rel: ERROR Unknown type: 261 > > I did below hack then kexec load works fine, OK, thanks, I'll add support for R_AARCH64_PREL32 in. > but `kexec -e` still does not work > there's nothing more than "Disableing non-boot CPUS ...\n Bye!": It is really tough to say what happened. The 'Bye!' message is printed just before the 1st stage kernel exits and the 2nd stage kernel is entered. If you have a debugger you can put some breakpoints in there to see how far it gets. That is generally how I figure out what went wrong. You could try the kexec --lite option, it will do a non-purgatory boot. You could also try my master branch that has more debugging output. Some of it is through the VE's serial port, so you may need to change that to what your platform has. -Geoff
On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: > Hi Geoff > > I tested your patches. The macihne is using spin-table cpu enable method > so I tried maxcpus=1 as you suggested. > > There's below issues for me, thoughts? > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > to use. Are you absolutely certain of this? To use ACPI, you must have booted via EFI, as the only mechanism for finding the ACPI tables is via EFI. If booted via EFI, the stub will have created a stub DTB if there is no provided DTB, to pass the command line and pointers to EFI data structures. This stub DTB should be present in the usual place. Thanks, Mark.
On 11/03/14 at 07:46pm, Mark Rutland wrote: > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: > > Hi Geoff > > > > I tested your patches. The macihne is using spin-table cpu enable method > > so I tried maxcpus=1 as you suggested. > > > > There's below issues for me, thoughts? > > > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > > to use. > > Are you absolutely certain of this? > > To use ACPI, you must have booted via EFI, as the only mechanism for > finding the ACPI tables is via EFI. If booted via EFI, the stub will > have created a stub DTB if there is no provided DTB, to pass the command > line and pointers to EFI data structures. This stub DTB should be > present in the usual place. Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs maybe I can pass external dtb to kexec-tools. What you mentioned should be true though but I have not get idea how to get the dtb which kernel is using for boot since it is not unflattened. Thanks Dave
On 10/31/14 at 04:25pm, Geoff Levand wrote: > Hi Dave, > > Thanks for testing. > > On Fri, 2014-10-31 at 15:52 +0800, Dave Young wrote: > > I tested your patches. The macihne is using spin-table cpu enable method > > so I tried maxcpus=1 as you suggested. > > > > There's below issues for me, thoughts? > > > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > > to use. > > You can supply a DTB with the kexec --dtb option, then kexec will not need > /proc/device-tree. Please try if you can and let me know what happens. I remember I tried but kexec load fails due to kexec-tools is still trying to access proc flattened dtb. I have little time last few days, will test it again to confirm. > > > 2. adding "acpi=off" to 1st kernel boot cmdline, kexec load fails with error > > like below: > > machine_apply_elf_rel: ERROR Unknown type: 261 > > > > I did below hack then kexec load works fine, > > OK, thanks, I'll add support for R_AARCH64_PREL32 in. > > > but `kexec -e` still does not work > > there's nothing more than "Disableing non-boot CPUS ...\n Bye!": > > It is really tough to say what happened. The 'Bye!' message is printed > just before the 1st stage kernel exits and the 2nd stage kernel is > entered. If you have a debugger you can put some breakpoints in > there to see how far it gets. That is generally how I figure out > what went wrong. > > You could try the kexec --lite option, it will do a non-purgatory > boot. > > You could also try my master branch that has more debugging output. > Some of it is through the VE's serial port, so you may need to > change that to what your platform has. I think I have will have to check if I can find a way to debug as you have done in your master branch. Thanks a lot Dave
On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: > On 11/03/14 at 07:46pm, Mark Rutland wrote: > > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: > > > Hi Geoff > > > > > > I tested your patches. The macihne is using spin-table cpu enable method > > > so I tried maxcpus=1 as you suggested. > > > > > > There's below issues for me, thoughts? > > > > > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > > > to use. > > > > Are you absolutely certain of this? > > > > To use ACPI, you must have booted via EFI, as the only mechanism for > > finding the ACPI tables is via EFI. If booted via EFI, the stub will > > have created a stub DTB if there is no provided DTB, to pass the command > > line and pointers to EFI data structures. This stub DTB should be > > present in the usual place. > > Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs > maybe I can pass external dtb to kexec-tools. What you mentioned should be true > though but I have not get idea how to get the dtb which kernel is using for boot > since it is not unflattened. Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, so there's nothing to expose at in sysfs/procfs. Somehow we need to unflatten the DTB without exposing it to drivers, such that it can be exposed to userspace in the usual place but drivers don't being probing based off of it. Thanks, Mark.
On Thu, Nov 6, 2014 at 3:08 PM, Mark Rutland <mark.rutland@arm.com> wrote: > On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: >> On 11/03/14 at 07:46pm, Mark Rutland wrote: >> > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: >> > > Hi Geoff >> > > >> > > I tested your patches. The macihne is using spin-table cpu enable method >> > > so I tried maxcpus=1 as you suggested. >> > > >> > > There's below issues for me, thoughts? >> > > >> > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb >> > > to use. >> > >> > Are you absolutely certain of this? >> > >> > To use ACPI, you must have booted via EFI, as the only mechanism for >> > finding the ACPI tables is via EFI. If booted via EFI, the stub will >> > have created a stub DTB if there is no provided DTB, to pass the command >> > line and pointers to EFI data structures. This stub DTB should be >> > present in the usual place. >> >> Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs >> maybe I can pass external dtb to kexec-tools. What you mentioned should be true >> though but I have not get idea how to get the dtb which kernel is using for boot >> since it is not unflattened. > > Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, > so there's nothing to expose at in sysfs/procfs. > > Somehow we need to unflatten the DTB without exposing it to drivers, > such that it can be exposed to userspace in the usual place but drivers > don't being probing based off of it. Is that even necessary? If the tree isn't unflattened, then it is just a stub tree. There really isn't anything interesting in it. Kexec should recreate the tree from scratch in that case. g.
On Fri, Nov 07, 2014 at 12:41:45AM +0000, Grant Likely wrote: > On Thu, Nov 6, 2014 at 3:08 PM, Mark Rutland <mark.rutland@arm.com> wrote: > > On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: > >> On 11/03/14 at 07:46pm, Mark Rutland wrote: > >> > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: > >> > > Hi Geoff > >> > > > >> > > I tested your patches. The macihne is using spin-table cpu enable method > >> > > so I tried maxcpus=1 as you suggested. > >> > > > >> > > There's below issues for me, thoughts? > >> > > > >> > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > >> > > to use. > >> > > >> > Are you absolutely certain of this? > >> > > >> > To use ACPI, you must have booted via EFI, as the only mechanism for > >> > finding the ACPI tables is via EFI. If booted via EFI, the stub will > >> > have created a stub DTB if there is no provided DTB, to pass the command > >> > line and pointers to EFI data structures. This stub DTB should be > >> > present in the usual place. > >> > >> Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs > >> maybe I can pass external dtb to kexec-tools. What you mentioned should be true > >> though but I have not get idea how to get the dtb which kernel is using for boot > >> since it is not unflattened. > > > > Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, > > so there's nothing to expose at in sysfs/procfs. > > > > Somehow we need to unflatten the DTB without exposing it to drivers, > > such that it can be exposed to userspace in the usual place but drivers > > don't being probing based off of it. > > Is that even necessary? If the tree isn't unflattened, then it is just > a stub tree. There really isn't anything interesting in it. We need to UEFI properties [1] from /chosen to access the memory map and system table (both of which are necessary to access any ACPI tables). > Kexec should recreate the tree from scratch in that case. I'm not sure if the required information is exposed to userspace elsewhere. Ard, Leif? Mark. [1] Documentation/arm/uefi.txt
On 7 November 2014 11:16, Mark Rutland <mark.rutland@arm.com> wrote: > On Fri, Nov 07, 2014 at 12:41:45AM +0000, Grant Likely wrote: >> On Thu, Nov 6, 2014 at 3:08 PM, Mark Rutland <mark.rutland@arm.com> wrote: >> > On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: >> >> On 11/03/14 at 07:46pm, Mark Rutland wrote: >> >> > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: >> >> > > Hi Geoff >> >> > > >> >> > > I tested your patches. The macihne is using spin-table cpu enable method >> >> > > so I tried maxcpus=1 as you suggested. >> >> > > >> >> > > There's below issues for me, thoughts? >> >> > > >> >> > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb >> >> > > to use. >> >> > >> >> > Are you absolutely certain of this? >> >> > >> >> > To use ACPI, you must have booted via EFI, as the only mechanism for >> >> > finding the ACPI tables is via EFI. If booted via EFI, the stub will >> >> > have created a stub DTB if there is no provided DTB, to pass the command >> >> > line and pointers to EFI data structures. This stub DTB should be >> >> > present in the usual place. >> >> >> >> Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs >> >> maybe I can pass external dtb to kexec-tools. What you mentioned should be true >> >> though but I have not get idea how to get the dtb which kernel is using for boot >> >> since it is not unflattened. >> > >> > Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, >> > so there's nothing to expose at in sysfs/procfs. >> > >> > Somehow we need to unflatten the DTB without exposing it to drivers, >> > such that it can be exposed to userspace in the usual place but drivers >> > don't being probing based off of it. >> >> Is that even necessary? If the tree isn't unflattened, then it is just >> a stub tree. There really isn't anything interesting in it. > > We need to UEFI properties [1] from /chosen to access the memory map and > system table (both of which are necessary to access any ACPI tables). > >> Kexec should recreate the tree from scratch in that case. > > I'm not sure if the required information is exposed to userspace > elsewhere. Ard, Leif? > Personally, I think we should not be using /proc/device-tree at all, but instead retain the original blob in some way and expose that. We already have /sys/firmware/efi/systab which contains the physical addresses of the UEFI configuration tables. We should probably add an entry for the FDT there anyway, but we would still be looking at mmap(/dev/mem) to access it, which is not a practice we want to encourage, I suppose. Also, we *must* take the secure boot scenario into account. Booting with an arbitrary user generated DTB is nice, but if you are doing kexec without an initrd, for instance, it would also be nice if we could just reuse the existing DTB without bothering the user for it at all, which would be something we could also allow when running securely.
On 7 November 2014 11:41, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > On 7 November 2014 11:16, Mark Rutland <mark.rutland@arm.com> wrote: >> On Fri, Nov 07, 2014 at 12:41:45AM +0000, Grant Likely wrote: >>> On Thu, Nov 6, 2014 at 3:08 PM, Mark Rutland <mark.rutland@arm.com> wrote: >>> > On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: >>> >> On 11/03/14 at 07:46pm, Mark Rutland wrote: >>> >> > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: >>> >> > > Hi Geoff >>> >> > > >>> >> > > I tested your patches. The macihne is using spin-table cpu enable method >>> >> > > so I tried maxcpus=1 as you suggested. >>> >> > > >>> >> > > There's below issues for me, thoughts? >>> >> > > >>> >> > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb >>> >> > > to use. >>> >> > >>> >> > Are you absolutely certain of this? >>> >> > >>> >> > To use ACPI, you must have booted via EFI, as the only mechanism for >>> >> > finding the ACPI tables is via EFI. If booted via EFI, the stub will >>> >> > have created a stub DTB if there is no provided DTB, to pass the command >>> >> > line and pointers to EFI data structures. This stub DTB should be >>> >> > present in the usual place. >>> >> >>> >> Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs >>> >> maybe I can pass external dtb to kexec-tools. What you mentioned should be true >>> >> though but I have not get idea how to get the dtb which kernel is using for boot >>> >> since it is not unflattened. >>> > >>> > Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, >>> > so there's nothing to expose at in sysfs/procfs. >>> > >>> > Somehow we need to unflatten the DTB without exposing it to drivers, >>> > such that it can be exposed to userspace in the usual place but drivers >>> > don't being probing based off of it. >>> >>> Is that even necessary? If the tree isn't unflattened, then it is just >>> a stub tree. There really isn't anything interesting in it. >> >> We need to UEFI properties [1] from /chosen to access the memory map and >> system table (both of which are necessary to access any ACPI tables). >> >>> Kexec should recreate the tree from scratch in that case. >> >> I'm not sure if the required information is exposed to userspace >> elsewhere. Ard, Leif? >> > > Personally, I think we should not be using /proc/device-tree at all, > but instead retain the original blob in some way and expose that. > > We already have /sys/firmware/efi/systab which contains the physical > addresses of the UEFI configuration tables. We should probably add an > entry for the FDT there anyway, but we would still be looking at > mmap(/dev/mem) to access it, which is not a practice we want to > encourage, I suppose. > Nah, strike that. The configuration table entry contains the original fdt, so with the device nodes etc still present. The stub makes changes to the DTB, and /that/ is the version we want to retain so subsequent kexec reboots can use it. > Also, we *must* take the secure boot scenario into account. Booting > with an arbitrary user generated DTB is nice, but if you are doing > kexec without an initrd, for instance, it would also be nice if we > could just reuse the existing DTB without bothering the user for it at > all, which would be something we could also allow when running > securely. > > -- > Ard.
On 7 November 2014 11:45, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > On 7 November 2014 11:41, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: >> On 7 November 2014 11:16, Mark Rutland <mark.rutland@arm.com> wrote: >>> On Fri, Nov 07, 2014 at 12:41:45AM +0000, Grant Likely wrote: >>>> On Thu, Nov 6, 2014 at 3:08 PM, Mark Rutland <mark.rutland@arm.com> wrote: >>>> > On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: >>>> >> On 11/03/14 at 07:46pm, Mark Rutland wrote: >>>> >> > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: >>>> >> > > Hi Geoff >>>> >> > > >>>> >> > > I tested your patches. The macihne is using spin-table cpu enable method >>>> >> > > so I tried maxcpus=1 as you suggested. >>>> >> > > >>>> >> > > There's below issues for me, thoughts? >>>> >> > > >>>> >> > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb >>>> >> > > to use. >>>> >> > >>>> >> > Are you absolutely certain of this? >>>> >> > >>>> >> > To use ACPI, you must have booted via EFI, as the only mechanism for >>>> >> > finding the ACPI tables is via EFI. If booted via EFI, the stub will >>>> >> > have created a stub DTB if there is no provided DTB, to pass the command >>>> >> > line and pointers to EFI data structures. This stub DTB should be >>>> >> > present in the usual place. >>>> >> >>>> >> Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs >>>> >> maybe I can pass external dtb to kexec-tools. What you mentioned should be true >>>> >> though but I have not get idea how to get the dtb which kernel is using for boot >>>> >> since it is not unflattened. >>>> > >>>> > Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, >>>> > so there's nothing to expose at in sysfs/procfs. >>>> > >>>> > Somehow we need to unflatten the DTB without exposing it to drivers, >>>> > such that it can be exposed to userspace in the usual place but drivers >>>> > don't being probing based off of it. >>>> >>>> Is that even necessary? If the tree isn't unflattened, then it is just >>>> a stub tree. There really isn't anything interesting in it. >>> >>> We need to UEFI properties [1] from /chosen to access the memory map and >>> system table (both of which are necessary to access any ACPI tables). >>> >>>> Kexec should recreate the tree from scratch in that case. >>> >>> I'm not sure if the required information is exposed to userspace >>> elsewhere. Ard, Leif? >>> >> >> Personally, I think we should not be using /proc/device-tree at all, >> but instead retain the original blob in some way and expose that. >> >> We already have /sys/firmware/efi/systab which contains the physical >> addresses of the UEFI configuration tables. We should probably add an >> entry for the FDT there anyway, but we would still be looking at >> mmap(/dev/mem) to access it, which is not a practice we want to >> encourage, I suppose. >> > > Nah, strike that. The configuration table entry contains the original > fdt, so with the device nodes etc still present. The stub makes *memory* nodes not device nodes > changes to the DTB, and /that/ is the version we want to retain so > subsequent kexec reboots can use it. > Sorry for the noise.
On Fri, Nov 07, 2014 at 10:41:11AM +0000, Ard Biesheuvel wrote: > On 7 November 2014 11:16, Mark Rutland <mark.rutland@arm.com> wrote: > > On Fri, Nov 07, 2014 at 12:41:45AM +0000, Grant Likely wrote: > >> On Thu, Nov 6, 2014 at 3:08 PM, Mark Rutland <mark.rutland@arm.com> wrote: > >> > On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: > >> >> On 11/03/14 at 07:46pm, Mark Rutland wrote: > >> >> > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: > >> >> > > Hi Geoff > >> >> > > > >> >> > > I tested your patches. The macihne is using spin-table cpu enable method > >> >> > > so I tried maxcpus=1 as you suggested. > >> >> > > > >> >> > > There's below issues for me, thoughts? > >> >> > > > >> >> > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > >> >> > > to use. > >> >> > > >> >> > Are you absolutely certain of this? > >> >> > > >> >> > To use ACPI, you must have booted via EFI, as the only mechanism for > >> >> > finding the ACPI tables is via EFI. If booted via EFI, the stub will > >> >> > have created a stub DTB if there is no provided DTB, to pass the command > >> >> > line and pointers to EFI data structures. This stub DTB should be > >> >> > present in the usual place. > >> >> > >> >> Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs > >> >> maybe I can pass external dtb to kexec-tools. What you mentioned should be true > >> >> though but I have not get idea how to get the dtb which kernel is using for boot > >> >> since it is not unflattened. > >> > > >> > Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, > >> > so there's nothing to expose at in sysfs/procfs. > >> > > >> > Somehow we need to unflatten the DTB without exposing it to drivers, > >> > such that it can be exposed to userspace in the usual place but drivers > >> > don't being probing based off of it. > >> > >> Is that even necessary? If the tree isn't unflattened, then it is just > >> a stub tree. There really isn't anything interesting in it. > > > > We need to UEFI properties [1] from /chosen to access the memory map and > > system table (both of which are necessary to access any ACPI tables). > > > >> Kexec should recreate the tree from scratch in that case. > > > > I'm not sure if the required information is exposed to userspace > > elsewhere. Ard, Leif? > > > > Personally, I think we should not be using /proc/device-tree at all, > but instead retain the original blob in some way and expose that. Grant took objection to that approach previously. > We already have /sys/firmware/efi/systab which contains the physical > addresses of the UEFI configuration tables. We should probably add an > entry for the FDT there anyway, but we would still be looking at > mmap(/dev/mem) to access it, which is not a practice we want to > encourage, I suppose. We should not encourage use of /dev/mem. Using /sys/firmware/efi/systab doesn't get us the memory map though, unless that's also exposed? > Also, we *must* take the secure boot scenario into account. Booting > with an arbitrary user generated DTB is nice, but if you are doing > kexec without an initrd, for instance, it would also be nice if we > could just reuse the existing DTB without bothering the user for it at > all, which would be something we could also allow when running > securely. Secure boot has to be handled completely differently. That will require new syscall support, in-kernel purgatory, and so on. That should not affect the non-secureboot cases where a user wants to load their own DTB (or other objects) into memory for the next OS (which might not be Linux). Mark.
On 7 November 2014 12:35, Mark Rutland <mark.rutland@arm.com> wrote: > On Fri, Nov 07, 2014 at 10:41:11AM +0000, Ard Biesheuvel wrote: >> On 7 November 2014 11:16, Mark Rutland <mark.rutland@arm.com> wrote: >> > On Fri, Nov 07, 2014 at 12:41:45AM +0000, Grant Likely wrote: >> >> On Thu, Nov 6, 2014 at 3:08 PM, Mark Rutland <mark.rutland@arm.com> wrote: >> >> > On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: >> >> >> On 11/03/14 at 07:46pm, Mark Rutland wrote: >> >> >> > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: >> >> >> > > Hi Geoff >> >> >> > > >> >> >> > > I tested your patches. The macihne is using spin-table cpu enable method >> >> >> > > so I tried maxcpus=1 as you suggested. >> >> >> > > >> >> >> > > There's below issues for me, thoughts? >> >> >> > > >> >> >> > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb >> >> >> > > to use. >> >> >> > >> >> >> > Are you absolutely certain of this? >> >> >> > >> >> >> > To use ACPI, you must have booted via EFI, as the only mechanism for >> >> >> > finding the ACPI tables is via EFI. If booted via EFI, the stub will >> >> >> > have created a stub DTB if there is no provided DTB, to pass the command >> >> >> > line and pointers to EFI data structures. This stub DTB should be >> >> >> > present in the usual place. >> >> >> >> >> >> Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs >> >> >> maybe I can pass external dtb to kexec-tools. What you mentioned should be true >> >> >> though but I have not get idea how to get the dtb which kernel is using for boot >> >> >> since it is not unflattened. >> >> > >> >> > Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, >> >> > so there's nothing to expose at in sysfs/procfs. >> >> > >> >> > Somehow we need to unflatten the DTB without exposing it to drivers, >> >> > such that it can be exposed to userspace in the usual place but drivers >> >> > don't being probing based off of it. >> >> >> >> Is that even necessary? If the tree isn't unflattened, then it is just >> >> a stub tree. There really isn't anything interesting in it. >> > >> > We need to UEFI properties [1] from /chosen to access the memory map and >> > system table (both of which are necessary to access any ACPI tables). >> > >> >> Kexec should recreate the tree from scratch in that case. >> > >> > I'm not sure if the required information is exposed to userspace >> > elsewhere. Ard, Leif? >> > >> >> Personally, I think we should not be using /proc/device-tree at all, >> but instead retain the original blob in some way and expose that. > > Grant took objection to that approach previously. > >> We already have /sys/firmware/efi/systab which contains the physical >> addresses of the UEFI configuration tables. We should probably add an >> entry for the FDT there anyway, but we would still be looking at >> mmap(/dev/mem) to access it, which is not a practice we want to >> encourage, I suppose. > > We should not encourage use of /dev/mem. > Agreed. > Using /sys/firmware/efi/systab doesn't get us the memory map though, > unless that's also exposed? > The address of the memory map is in /chosen/linux,uefi-mmap-start, but -as I replied-to-self earlier- the DT UEFI configuration table does not contain the properties added by the stub, so that is not going to work anyway. >> Also, we *must* take the secure boot scenario into account. Booting >> with an arbitrary user generated DTB is nice, but if you are doing >> kexec without an initrd, for instance, it would also be nice if we >> could just reuse the existing DTB without bothering the user for it at >> all, which would be something we could also allow when running >> securely. > > Secure boot has to be handled completely differently. That will require > new syscall support, in-kernel purgatory, and so on. > > That should not affect the non-secureboot cases where a user wants to > load their own DTB (or other objects) into memory for the next OS (which > might not be Linux). > OK, fair enough. So ideally, we should expose the binary blob somewhere, in a similar fashion to how config.gz is exposed, perhaps?
On 7 Nov 2014 11:36, "Mark Rutland" <mark.rutland@arm.com> wrote: > > On Fri, Nov 07, 2014 at 10:41:11AM +0000, Ard Biesheuvel wrote: > > On 7 November 2014 11:16, Mark Rutland <mark.rutland@arm.com> wrote: > > > On Fri, Nov 07, 2014 at 12:41:45AM +0000, Grant Likely wrote: > > >> On Thu, Nov 6, 2014 at 3:08 PM, Mark Rutland <mark.rutland@arm.com> wrote: > > >> > On Thu, Nov 06, 2014 at 01:56:42AM +0000, Dave Young wrote: > > >> >> On 11/03/14 at 07:46pm, Mark Rutland wrote: > > >> >> > On Fri, Oct 31, 2014 at 07:52:09AM +0000, Dave Young wrote: > > >> >> > > Hi Geoff > > >> >> > > > > >> >> > > I tested your patches. The macihne is using spin-table cpu enable method > > >> >> > > so I tried maxcpus=1 as you suggested. > > >> >> > > > > >> >> > > There's below issues for me, thoughts? > > >> >> > > > > >> >> > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > > >> >> > > to use. > > >> >> > > > >> >> > Are you absolutely certain of this? > > >> >> > > > >> >> > To use ACPI, you must have booted via EFI, as the only mechanism for > > >> >> > finding the ACPI tables is via EFI. If booted via EFI, the stub will > > >> >> > have created a stub DTB if there is no provided DTB, to pass the command > > >> >> > line and pointers to EFI data structures. This stub DTB should be > > >> >> > present in the usual place. > > >> >> > > >> >> Mark, I used kexec-tools from Geoff's git tree, it will create dtb from procfs > > >> >> maybe I can pass external dtb to kexec-tools. What you mentioned should be true > > >> >> though but I have not get idea how to get the dtb which kernel is using for boot > > >> >> since it is not unflattened. > > >> > > > >> > Ah, sorry. I see the problem now. For ACPI you don't unflatten the tree, > > >> > so there's nothing to expose at in sysfs/procfs. > > >> > > > >> > Somehow we need to unflatten the DTB without exposing it to drivers, > > >> > such that it can be exposed to userspace in the usual place but drivers > > >> > don't being probing based off of it. > > >> > > >> Is that even necessary? If the tree isn't unflattened, then it is just > > >> a stub tree. There really isn't anything interesting in it. > > > > > > We need to UEFI properties [1] from /chosen to access the memory map and > > > system table (both of which are necessary to access any ACPI tables). > > > > > >> Kexec should recreate the tree from scratch in that case. > > > > > > I'm not sure if the required information is exposed to userspace > > > elsewhere. Ard, Leif? > > > > > > > Personally, I think we should not be using /proc/device-tree at all, > > but instead retain the original blob in some way and expose that. > > Grant took objection to that approach previously. I don't strongly object, but using the original DTB is problematic if the kernel makes changes to the tree that should be passed through to the next kernel. The only big objection I have is to using the existing debugfs implementation. Care also need to be taken to make sure the flat tree has not been modified on the fly. I think there are some scenarios where the kernel modifies the DT buffer in-place. That said, for a stub DT, the best place to generate it is still probably in kernel space. > > > We already have /sys/firmware/efi/systab which contains the physical > > addresses of the UEFI configuration tables. We should probably add an > > entry for the FDT there anyway, but we would still be looking at > > mmap(/dev/mem) to access it, which is not a practice we want to > > encourage, I suppose. > > We should not encourage use of /dev/mem. > > Using /sys/firmware/efi/systab doesn't get us the memory map though, > unless that's also exposed? > > > Also, we *must* take the secure boot scenario into account. Booting > > with an arbitrary user generated DTB is nice, but if you are doing > > kexec without an initrd, for instance, it would also be nice if we > > could just reuse the existing DTB without bothering the user for it at > > all, which would be something we could also allow when running > > securely. > > Secure boot has to be handled completely differently. That will require > new syscall support, in-kernel purgatory, and so on. > > That should not affect the non-secureboot cases where a user wants to > load their own DTB (or other objects) into memory for the next OS (which > might not be Linux). > > Mark.
On 10/31/14 at 04:25pm, Geoff Levand wrote: > Hi Dave, > > Thanks for testing. > > On Fri, 2014-10-31 at 15:52 +0800, Dave Young wrote: > > I tested your patches. The macihne is using spin-table cpu enable method > > so I tried maxcpus=1 as you suggested. > > > > There's below issues for me, thoughts? > > > > 1. For acpi booting there's no /proc/device-tree so kexec can not find dtb > > to use. > > You can supply a DTB with the kexec --dtb option, then kexec will not need > /proc/device-tree. Please try if you can and let me know what happens. > In your kexec-tools code, by default it will create dtb1 from procfs, as for external --dtb it will be dtb2 for comparing with dtb1. Since dtb1 is a must so it will not work without /proc/device-tree. There had some discussion about this in the thread. But I think we have no conclusion where to go. There maybe below possibilities: 1) unflattern dtb both for EFI/acpi booting even if acpi boot does not need it. 2) copy the bootloader passed dtb stub to somewhere, and export the stub to userspace ie. in debugfs or procfs like /proc/saved_dtb Thoughts to continue the discuss? Thanks Dave
On Thu, 2014-11-13 at 16:37 +0800, Dave Young wrote: > In your kexec-tools code, by default it will create dtb1 from procfs, as for external > --dtb it will be dtb2 for comparing with dtb1. > > Since dtb1 is a must so it will not work without /proc/device-tree. I changed that to allow for no /proc/device-tree if --dtb is passed since it will also allow a kernel built with CONFIG_PROC_FS=n to do kexec re-boot. Please try it and let me know. > There had some discussion about this in the thread. But I think we have no > conclusion where to go. > > There maybe below possibilities: > 1) unflattern dtb both for EFI/acpi booting even if acpi boot does not need it. > > 2) copy the bootloader passed dtb stub to somewhere, and export the stub to > userspace ie. in debugfs or procfs > like /proc/saved_dtb > > Thoughts to continue the discuss? Some progress has been made, let's see what they come up with. -Geoff
On 11/13/14 at 03:50pm, Geoff Levand wrote: > On Thu, 2014-11-13 at 16:37 +0800, Dave Young wrote: > > In your kexec-tools code, by default it will create dtb1 from procfs, as for external > > --dtb it will be dtb2 for comparing with dtb1. > > > > Since dtb1 is a must so it will not work without /proc/device-tree. > > I changed that to allow for no /proc/device-tree if --dtb is passed > since it will also allow a kernel built with CONFIG_PROC_FS=n to do > kexec re-boot. > > Please try it and let me know. Tested the master branch or your kexec-tools, I got below with "mount -o bind /root /proc" because there's some other issues for my customized kernel to boot: arch_process_options:95: initrd: (null) arch_process_options:96: dtb: apm-mustang.dtb arch_process_options:97: lite: 1 arch_process_options:99: port: 0x0 kernel: 0x3ff9cbb0010 kernel_size: 0x602c947 unrecoverable error: could not scan "/proc/device-tree/": No such file or directory Thanks Dave
diff --git a/kexec/arch/arm64/kexec-arm64.c b/kexec/arch/arm64/kexec-arm64.c index 9db1c07..2df24f0 100644 --- a/kexec/arch/arm64/kexec-arm64.c +++ b/kexec/arch/arm64/kexec-arm64.c @@ -790,6 +790,10 @@ void machine_apply_elf_rel(struct mem_ehdr *ehdr, unsigned long r_type, # define R_AARCH64_ABS64 257 #endif +#if !defined(R_AARCH64_PREL32) +#define R_AARCH64_PREL32 261 +#endif + #if !defined(R_AARCH64_LD_PREL_LO19) # define R_AARCH64_LD_PREL_LO19 273 #endif @@ -806,6 +810,7 @@ void machine_apply_elf_rel(struct mem_ehdr *ehdr, unsigned long r_type, # define R_AARCH64_CALL26 283 #endif + uint64_t *location1 = (uint64_t *)ptr; uint32_t *location = (uint32_t *)ptr; uint32_t data = *location; const char *type = NULL; @@ -813,7 +818,11 @@ void machine_apply_elf_rel(struct mem_ehdr *ehdr, unsigned long r_type, switch(r_type) { case R_AARCH64_ABS64: type = "ABS64"; - *location += value; + *location1 += value; + break; + case R_AARCH64_PREL32: + type = "PREL32"; + *location += value - address; break; case R_AARCH64_LD_PREL_LO19: type = "LD_PREL_LO19"; @@ -839,7 +848,7 @@ void machine_apply_elf_rel(struct mem_ehdr *ehdr, unsigned long r_type, break; } - dbgprintf("%s: %s %x->%x\n", __func__, type, data, *location); + dbgprintf("%s: %s %x->%lx\n", __func__, type, data, (r_type == R_AARCH64_ABS64)? *location1: *location); } void arch_reuse_initrd(void)