| Message ID | 5342AF59.3030405@linaro.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Mon, Apr 07, 2014 at 02:59:53PM +0100, Julien Grall wrote: > Hello all, Hi Julien, > Currently ELF is supporting notes (see include/linux/elfnote.h) which > is used by Xen to know the feature supported by the kernel. What kind of features do you intend for Xen discover through this mechanism on ARM? Can support for these not be negotiated at run-time (how does this work with other OSs under Xen)? > On ARM world, the zImage format is mainly used by the distribution. > Rather than ELF, it's currently not possible to add notes on it. > > ==== Current zImage format ==== > > The header starts at offset 0x24: > uint32_t magic /* Magic number: 0x016f2818 */ > uint32_t start /* absolute load/run zImage address */ > uint32_t end /* zImage end address */ > > ==== Extension proposed to zImage format === > > The header stays compatible with the current format and starts at offset 0x24: > uint32_t magic0 /* Magic number : 0x16f2818 */ > uint32_t start /* absolute load/run zImage address */ > uint32_t end /* zImage end address */ > uint32_t magic1 /* Magic number : "ARM\x32" */ > uint32_t note_start /* Notes section offset */ > uint32_t note_end /* Notes sections end */ On arm64 we seem to have a notes section but no pointer to it in the Image header. I assume you'd consider a notes section useful for 64-bit too? I have some patches extending the header for arm64 which I intend to post shortly; if we're going to clash there and this is needed, perhaps I can fold in a similar update. Cheers, Mark.
On 04/07/2014 05:04 PM, Mark Rutland wrote: > On Mon, Apr 07, 2014 at 02:59:53PM +0100, Julien Grall wrote: > Hi Julien, Hello Mark, >> Currently ELF is supporting notes (see include/linux/elfnote.h) which >> is used by Xen to know the feature supported by the kernel. > > What kind of features do you intend for Xen discover through this > mechanism on ARM? tl;dr: To know if DOM0 Linux will correctly work when IOMMU is enabled by Xen. Actually Xen doesn't support IOMMU. To allow DMA-capable devices correctly work, we use an hybrid approach: - 1:1 mapping for DOM0 memory - Use swiotlb for DMA on a foreign page (i.e. page which belongs to another domain). For the latter, the swiotlb will return a physical address rather than an IPA. I'm actually working to add SMMU support in Xen for ARM. Once Xen will enable the IOMMU, Linux will have to program DMA with IPA. As the swiotlb is returning physical address, Linux will have to know whether the device is protected by IOMMU or not. For the former case, Linux shouldn't use swiotlb for this device (see https://lkml.org/lkml/2014/2/20/351). Since the Xen ABI is stable for ARM (since Linux 3.13 and Xen 4.4), we want to be able to boot Linux 3.13 on newer version of Xen. For this version and onward (i.e. as long as swiotlb is used by default in DOM0), Xen has to boot with IOMMU disabled. With the zImage format extension, we will be able to add a note saying: "I don't use swiotlb by default, you can turn on the IOMMU" once it's case. > Can support for these not be negotiated at run-time (how does this work > with other OSs under Xen)? We though about runtime approach (see the whole discussion here https://patches.linaro.org/26034/), but it's not as extensible as notes. In fact, we might need more notes in the future when new feature will be added and requires a DOM0/guest side support. On x86 world, the dom0/guest features is discovered via ELF notes. I think we should use the same way on ARM when zImage is used to boot DOM0. > On arm64 we seem to have a notes section but no pointer to it in the > Image header. I assume you'd consider a notes section useful for 64-bit > too? Yes, we will need to same section on arm64. > I have some patches extending the header for arm64 which I intend to > post shortly; if we're going to clash there and this is needed, perhaps > I can fold in a similar update. I didn't start to work on arm64 format extension. I wanted to wait for feedback on the zImage format extension for arm32. I plan to use one (may two) field from the reserved ones. Sincerely yours,
On Mon, Apr 07, 2014 at 06:28:12PM +0100, Julien Grall wrote: > On 04/07/2014 05:04 PM, Mark Rutland wrote: > > On Mon, Apr 07, 2014 at 02:59:53PM +0100, Julien Grall wrote: > > Hi Julien, > > Hello Mark, > > >> Currently ELF is supporting notes (see include/linux/elfnote.h) which > >> is used by Xen to know the feature supported by the kernel. > > > > What kind of features do you intend for Xen discover through this > > mechanism on ARM? > > tl;dr: To know if DOM0 Linux will correctly work when IOMMU is enabled > by Xen. > > Actually Xen doesn't support IOMMU. To allow DMA-capable devices > correctly work, we use an hybrid approach: > - 1:1 mapping for DOM0 memory > - Use swiotlb for DMA on a foreign page (i.e. page which belongs to > another domain). > > For the latter, the swiotlb will return a physical address rather than > an IPA. > > I'm actually working to add SMMU support in Xen for ARM. Once Xen will > enable the IOMMU, Linux will have to program DMA with IPA. I'm probably being thick here, but is there anything preventing those foreign pages from being idmapped into DOM0 (at least for the duration of the DMA access) such that there is a 1-1 PA<->IPA mapping? That would mean that from the point of view of Linux, there's nothing to care about, as every address it knows it can deal with is a valid IPA. Existing kernels would function with the IOMMU(s) on. I assume I'm missing something fairly obvious :) > As the swiotlb is returning physical address, Linux will have to know > whether the device is protected by IOMMU or not. For the former case, > Linux shouldn't use swiotlb for this device (see > https://lkml.org/lkml/2014/2/20/351). For some reason I can only see the cover letter and not the patches when following that link. > Since the Xen ABI is stable for ARM (since Linux 3.13 and Xen 4.4), we > want to be able to boot Linux 3.13 on newer version of Xen. For this > version and onward (i.e. as long as swiotlb is used by default in DOM0), > Xen has to boot with IOMMU disabled. A mentioned above, I think I'm missing some part of the reasoning as to why Linux needs to know the difference. > With the zImage format extension, we will be able to add a note saying: > "I don't use swiotlb by default, you can turn on the IOMMU" once it's case. > > > Can support for these not be negotiated at run-time (how does this work > > with other OSs under Xen)? > > We though about runtime approach (see the whole discussion here > https://patches.linaro.org/26034/), but it's not as extensible as notes. > In fact, we might need more notes in the future when new feature will be > added and requires a DOM0/guest side support. I can certainly see the usefulness of notes. > > On x86 world, the dom0/guest features is discovered via ELF notes. > I think we should use the same way on ARM when zImage is used to boot DOM0. > > > On arm64 we seem to have a notes section but no pointer to it in the > > Image header. I assume you'd consider a notes section useful for 64-bit > > too? > > Yes, we will need to same section on arm64. If it makes sense for this case then I can see why we would want it on arm64. > > I have some patches extending the header for arm64 which I intend to > > post shortly; if we're going to clash there and this is needed, perhaps > > I can fold in a similar update. > > I didn't start to work on arm64 format extension. I wanted to wait for > feedback on the zImage format extension for arm32. Certainly. I just wanted to make sure we weren't going to end up stepping on each others' toes whenever we try to get our patches merged. Cheers, Mark.
Hi Mark, On 04/07/2014 07:25 PM, Mark Rutland wrote: > > I'm probably being thick here, but is there anything preventing those > foreign pages from being idmapped into DOM0 (at least for the duration > of the DMA access) such that there is a 1-1 PA<->IPA mapping? > > That would mean that from the point of view of Linux, there's nothing to > care about, as every address it knows it can deal with is a valid IPA. > Existing kernels would function with the IOMMU(s) on. > > I assume I'm missing something fairly obvious :) I didn't though about this solution. It works on midway only when the physical address size is 32 bits. I didn't yet figure out why. I guess this is because I have a bug in my SMMU driver in Xen. >> We though about runtime approach (see the whole discussion here >> https://patches.linaro.org/26034/), but it's not as extensible as notes. >> In fact, we might need more notes in the future when new feature will be >> added and requires a DOM0/guest side support. > > I can certainly see the usefulness of notes. [..] > If it makes sense for this case then I can see why we would want it on > arm64. With the solution of idmapping, the RFC seems irrelevant for now. But we may need to add notes in the future. I'm wondering if it would be worth to agree on a solution now. Thus, we will be able to add a note quickly when it will be necessary. Any thoughts? Sincerely yours,
==== Current zImage format ====
The header starts at offset 0x24:
uint32_t magic /* Magic number: 0x016f2818 */
uint32_t start /* absolute load/run zImage address */
uint32_t end /* zImage end address */
==== Extension proposed to zImage format ===
The header stays compatible with the current format and starts at offset 0x24:
uint32_t magic0 /* Magic number : 0x16f2818 */
uint32_t start /* absolute load/run zImage address */
uint32_t end /* zImage end address */
uint32_t magic1 /* Magic number : "ARM\x32" */
uint32_t note_start /* Notes section offset */
uint32_t note_end /* Notes sections end */
I was thinking to also add a version field in case we might want to extend the
format in the future.
The notes section will contains a list of note (see description below).
==== Format of a note ====
The format is compatible with ELF note:
Name size : 4 bytes
Desc size : 4 bytes
Type : 4 bytes: identify the type of the note
Name : variable size, padded to 4 byte boundary
Desc : variable size, padded to 4 byte boundary : contains data related to the note type.
The "Name size", "Desc size" and "Type" fields are interpreted as an integer.
I wrote a quick patch to show what would be the modification on Linux side (see below).
Comments and questions are welcomed.
Sincerely yours,
diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 066b034..104d862 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -133,6 +138,12 @@ start:
.word 0x016f2818 @ Magic numbers to help the loader
.word start @ absolute load/run zImage address
.word _edata @ zImage end address
+ .byte 0x41 @ Magic number, "ARM\x32"
+ .byte 0x52
+ .byte 0x4d
+ .byte 0x32
+ .word _note_start @ Notes section offset
+ .word _note_end @ Notes section end
THUMB( .thumb )
1:
ARM_BE8( setend be ) @ go BE8 if compiled for BE8
diff --git a/arch/arm/boot/compressed/vmlinux.lds.in b/arch/arm/boot/compressed/vmlinux.lds.in
index 4919f2a..d3287ff 100644
--- a/arch/arm/boot/compressed/vmlinux.lds.in
+++ b/arch/arm/boot/compressed/vmlinux.lds.in
@@ -52,6 +52,10 @@ SECTIONS
.got : { *(.got) }
_got_end = .;
+ _note_start = .;
+ .note : { *(.note*) }
+ _note_end = .;
+
/* ensure the zImage file size is always a multiple of 64 bits */
/* (without a dummy byte, ld just ignores the empty section) */
.pad : { BYTE(0); . = ALIGN(8); }
diff --git a/include/linux/elfnote.h b/include/linux/elfnote.h
index 278e3ef..9ad4488 100644
--- a/include/linux/elfnote.h
+++ b/include/linux/elfnote.h
@@ -39,7 +39,7 @@
* ELFNOTE(XYZCo, 12, .long, 0xdeadbeef)
*/
#define ELFNOTE_START(name, type, flags) \
-.pushsection .note.name, flags,@note ; \
+.pushsection .note.name, flags,%note ; \
.balign 4 ; \
.long 2f - 1f /* namesz */ ; \
.long 4484f - 3f /* descsz */ ; \