| Message ID | 1349796183-30648-6-git-send-email-arnd@arndb.de (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Tue, 2012-10-09 at 16:22 +0100, Arnd Bergmann wrote: > * The XEN_BALLOON code requires the balloon infrastructure that is not > getting built on ARM. I've got patches to enable this, but not for 3.7 so this looks good for now. > * The tmem hypercall is not available on ARM > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the missing the end of this sentence? > * sys-hypervisor.c needs to include linux/err.h in order to use the > IS_ERR/PTR_ERR/ERR_PTR family of functions. I tripped over this too. Not sure where that patch got to though. > Signed-off-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Ian Campbell <ian.campbell@citrix.com> > Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > Cc: Jeremy Fitzhardinge <jeremy@goop.org> > Cc: xen-devel@lists.xensource.com > --- > arch/arm/Kconfig | 1 + > drivers/xen/Kconfig | 2 ++ > drivers/xen/sys-hypervisor.c | 1 + > 3 files changed, 4 insertions(+) > > diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig > index 6d2f7f5..85eaac3 100644 > --- a/arch/arm/Kconfig > +++ b/arch/arm/Kconfig > @@ -1846,6 +1846,7 @@ config XEN_DOM0 > config XEN > bool "Xen guest support on ARM (EXPERIMENTAL)" > depends on EXPERIMENTAL && ARM && OF > + depends on !CPU_V6 > help > Say Y if you want to run Linux in a Virtual Machine on Xen on ARM. > > diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig > index d4dffcd..126d8ce 100644 > --- a/drivers/xen/Kconfig > +++ b/drivers/xen/Kconfig > @@ -3,6 +3,7 @@ menu "Xen driver support" > > config XEN_BALLOON > bool "Xen memory balloon driver" > + depends on !ARM > default y > help > The balloon driver allows the Xen domain to request more memory from > @@ -145,6 +146,7 @@ config SWIOTLB_XEN > > config XEN_TMEM > bool > + depends on !ARM > default y if (CLEANCACHE || FRONTSWAP) > help > Shim to interface in-kernel Transcendent Memory hooks > diff --git a/drivers/xen/sys-hypervisor.c b/drivers/xen/sys-hypervisor.c > index 5e5ad7e..66a0a14 100644 > --- a/drivers/xen/sys-hypervisor.c > +++ b/drivers/xen/sys-hypervisor.c > @@ -11,6 +11,7 @@ > #include <linux/kernel.h> > #include <linux/module.h> > #include <linux/kobject.h> > +#include <linux/err.h> > > #include <asm/xen/hypervisor.h> > #include <asm/xen/hypercall.h>
Thanks for the patch and sorry for the build breakage! On Tue, 9 Oct 2012, Arnd Bergmann wrote: > * The XEN_BALLOON code requires the balloon infrastructure that is not > getting built on ARM. > > * The tmem hypercall is not available on ARM > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the > > * sys-hypervisor.c needs to include linux/err.h in order to use the > IS_ERR/PTR_ERR/ERR_PTR family of functions. > > Signed-off-by: Arnd Bergmann <arnd@arndb.de> > Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > Cc: Jeremy Fitzhardinge <jeremy@goop.org> > Cc: xen-devel@lists.xensource.com > --- > arch/arm/Kconfig | 1 + > drivers/xen/Kconfig | 2 ++ > drivers/xen/sys-hypervisor.c | 1 + > 3 files changed, 4 insertions(+) > > diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig > index 6d2f7f5..85eaac3 100644 > --- a/arch/arm/Kconfig > +++ b/arch/arm/Kconfig > @@ -1846,6 +1846,7 @@ config XEN_DOM0 > config XEN > bool "Xen guest support on ARM (EXPERIMENTAL)" > depends on EXPERIMENTAL && ARM && OF > + depends on !CPU_V6 > help > Say Y if you want to run Linux in a Virtual Machine on Xen on ARM. Considering that we rely on the virtualization extensions, this one can be: depends on CPU_V7 The rest looks fine. I can submit a second patch to change !CPU_V6 into CPU_V7 later, if you prefer. > diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig > index d4dffcd..126d8ce 100644 > --- a/drivers/xen/Kconfig > +++ b/drivers/xen/Kconfig > @@ -3,6 +3,7 @@ menu "Xen driver support" > > config XEN_BALLOON > bool "Xen memory balloon driver" > + depends on !ARM > default y > help > The balloon driver allows the Xen domain to request more memory from > @@ -145,6 +146,7 @@ config SWIOTLB_XEN > > config XEN_TMEM > bool > + depends on !ARM > default y if (CLEANCACHE || FRONTSWAP) > help > Shim to interface in-kernel Transcendent Memory hooks > > diff --git a/drivers/xen/sys-hypervisor.c b/drivers/xen/sys-hypervisor.c > index 5e5ad7e..66a0a14 100644 > --- a/drivers/xen/sys-hypervisor.c > +++ b/drivers/xen/sys-hypervisor.c > @@ -11,6 +11,7 @@ > #include <linux/kernel.h> > #include <linux/module.h> > #include <linux/kobject.h> > +#include <linux/err.h> > > #include <asm/xen/hypervisor.h> > #include <asm/xen/hypercall.h> > -- > 1.7.10 >
On Tuesday 09 October 2012, Ian Campbell wrote: > > * The tmem hypercall is not available on ARM > > > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the > > missing the end of this sentence? Right, I meant to say * ARMv6 does not support cmpxchg on 16-bit words that are used in the Xen grant table code, so we must ensure that Xen support is only built on ARMv7-only kernels not combined ARMv6/v7 kernels. This should be fixed differently in the future. Arnd
On Tue, Oct 09, 2012 at 05:22:59PM +0200, Arnd Bergmann wrote: > * The XEN_BALLOON code requires the balloon infrastructure that is not > getting built on ARM. > > * The tmem hypercall is not available on ARM > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the "in the" what?
On Tue, 2012-10-09 at 16:39 +0100, Arnd Bergmann wrote: > On Tuesday 09 October 2012, Ian Campbell wrote: > > > * The tmem hypercall is not available on ARM > > > > > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the > > > > missing the end of this sentence? > > Right, I meant to say > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the > Xen grant table code, so we must ensure that Xen support is only built > on ARMv7-only kernels not combined ARMv6/v7 kernels. > > This should be fixed differently in the future. Is this is a build time failure because gcc/gas/etc refuses to generate these instructions if it is configured for v6? I ask because if it is only a runtime issue then we can reason that if we are running Xen specific grant table code, then we must be running on Xen and therefore must necessarily be running on a v7 (because Xen only support v7+virt extensions) even if the kernel happens to be capable of running on v6 too. Ian.
On Tuesday 09 October 2012, Ian Campbell wrote: > On Tue, 2012-10-09 at 16:39 +0100, Arnd Bergmann wrote: > > On Tuesday 09 October 2012, Ian Campbell wrote: > > > > * The tmem hypercall is not available on ARM > > > > > > > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the > > > > > > missing the end of this sentence? > > > > Right, I meant to say > > > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the > > Xen grant table code, so we must ensure that Xen support is only built > > on ARMv7-only kernels not combined ARMv6/v7 kernels. > > > > This should be fixed differently in the future. > > Is this is a build time failure because gcc/gas/etc refuses to generate > these instructions if it is configured for v6? > > I ask because if it is only a runtime issue then we can reason that if > we are running Xen specific grant table code, then we must be running on > Xen and therefore must necessarily be running on a v7 (because Xen only > support v7+virt extensions) even if the kernel happens to be capable of > running on v6 too. The underlying reason of course is that ARMv6 doesn't have those instructions. The symptom you see is a link error because gcc emits a reference to the (intentionally missing) __bad_cmpxchg() function from static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, unsigned long new, int size) { unsigned long oldval, res; switch (size) { #ifndef CONFIG_CPU_V6 /* min ARCH >= ARMv6K */ case 1: ... break; case 2: ... break; #endif case 4: ... break; default: __bad_cmpxchg(ptr, size); oldval = 0; } return oldval; } The possible solutions I can see for this are: * change the grant table format to use 32 bits for the flags on ARM * change the code to always cmpxchg the entire 32 bit word including the flags. * implement your own cmpxchg wrapper that may be implemented using a spinlock rather than cmpxchg if ARMv6 is enabled. Arnd
On Tuesday 09 October 2012, Stefano Stabellini wrote: > > config XEN > > bool "Xen guest support on ARM (EXPERIMENTAL)" > > depends on EXPERIMENTAL && ARM && OF > > + depends on !CPU_V6 > > help > > Say Y if you want to run Linux in a Virtual Machine on Xen on ARM. > > Considering that we rely on the virtualization extensions, this one can > be: > > depends on CPU_V7 > > The rest looks fine. I can submit a second patch to change !CPU_V6 into > CPU_V7 later, if you prefer. CPU_V6 and CPU_V7 are not exclusive, I saw the problem when building a combined kernel for both V6 and V7. The code also needs to depend on ARMv7 with virtualization extensions, but that is a different issue. We don't actually have a configuration symbol for that yet, as far as I can tell. Arnd
On Tue, 2012-10-09 at 19:21 +0100, Arnd Bergmann wrote: > On Tuesday 09 October 2012, Stefano Stabellini wrote: > > > config XEN > > > bool "Xen guest support on ARM (EXPERIMENTAL)" > > > depends on EXPERIMENTAL && ARM && OF > > > + depends on !CPU_V6 > > > help > > > Say Y if you want to run Linux in a Virtual Machine on Xen on ARM. > > > > Considering that we rely on the virtualization extensions, this one can > > be: > > > > depends on CPU_V7 > > > > The rest looks fine. I can submit a second patch to change !CPU_V6 into > > CPU_V7 later, if you prefer. > > CPU_V6 and CPU_V7 are not exclusive, I saw the problem when building a > combined kernel for both V6 and V7. The code also needs to depend on ARMv7 > with virtualization extensions, but that is a different issue. We don't > actually have a configuration symbol for that yet, as far as I can tell. I don't think the guest kernels (including dom0) need the extensions to run under Xen, they are only need by Xen itself. The guests should just see a relatively normal v7 processor. Stefano, does that sound right? Ian.
On Tue, 2012-10-09 at 19:19 +0100, Arnd Bergmann wrote: > On Tuesday 09 October 2012, Ian Campbell wrote: > > On Tue, 2012-10-09 at 16:39 +0100, Arnd Bergmann wrote: > > > On Tuesday 09 October 2012, Ian Campbell wrote: > > > > > * The tmem hypercall is not available on ARM > > > > > > > > > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the > > > > > > > > missing the end of this sentence? > > > > > > Right, I meant to say > > > > > > * ARMv6 does not support cmpxchg on 16-bit words that are used in the > > > Xen grant table code, so we must ensure that Xen support is only built > > > on ARMv7-only kernels not combined ARMv6/v7 kernels. > > > > > > This should be fixed differently in the future. > > > > Is this is a build time failure because gcc/gas/etc refuses to generate > > these instructions if it is configured for v6? > > > > I ask because if it is only a runtime issue then we can reason that if > > we are running Xen specific grant table code, then we must be running on > > Xen and therefore must necessarily be running on a v7 (because Xen only > > support v7+virt extensions) even if the kernel happens to be capable of > > running on v6 too. > > The underlying reason of course is that ARMv6 doesn't have those > instructions. The symptom you see is a link error because gcc emits > a reference to the (intentionally missing) __bad_cmpxchg() function > from OK, then your fix is the best one for now. > [...] > The possible solutions I can see for this are: > > * change the grant table format to use 32 bits for the flags on ARM > * change the code to always cmpxchg the entire 32 bit word including the flags. I'd need to check the grant table semantics to see if this will be possible. > * implement your own cmpxchg wrapper that may be implemented using a spinlock > rather than cmpxchg if ARMv6 is enabled. Even if ARMv6 is enabled the grant table code will never be running on one so so it might be ok to just have our own wrapper which unconditionally uses the v7 instruction? That might upset gas though. > > Arnd
On Wednesday 10 October 2012, Ian Campbell wrote: > > * implement your own cmpxchg wrapper that may be implemented using a spinlock > > rather than cmpxchg if ARMv6 is enabled. > > Even if ARMv6 is enabled the grant table code will never be running on > one so so it might be ok to just have our own wrapper which > unconditionally uses the v7 instruction? That might upset gas though. Yes, that would be possible. You can tell gas to ignore the instruction set in this case. If you do this, you can implement the update functions more efficiently using direct ldrexh/strexh in assembler to avoid doing two nested loops. I assume that you don't need the v1 grant table code on ARM anyway, so the only code you need to look at is while (!((flags = *pflags) & GTF_transfer_committed)) { if (sync_cmpxchg(pflags, flags, 0) == flags) return 0; cpu_relax(); } which should transform nicely into a few lines of inline assembly. Arnd
On Wed, 10 Oct 2012, Ian Campbell wrote: > On Tue, 2012-10-09 at 19:21 +0100, Arnd Bergmann wrote: > > On Tuesday 09 October 2012, Stefano Stabellini wrote: > > > > config XEN > > > > bool "Xen guest support on ARM (EXPERIMENTAL)" > > > > depends on EXPERIMENTAL && ARM && OF > > > > + depends on !CPU_V6 > > > > help > > > > Say Y if you want to run Linux in a Virtual Machine on Xen on ARM. > > > > > > Considering that we rely on the virtualization extensions, this one can > > > be: > > > > > > depends on CPU_V7 > > > > > > The rest looks fine. I can submit a second patch to change !CPU_V6 into > > > CPU_V7 later, if you prefer. > > > > CPU_V6 and CPU_V7 are not exclusive, I saw the problem when building a > > combined kernel for both V6 and V7. The code also needs to depend on ARMv7 > > with virtualization extensions, but that is a different issue. We don't > > actually have a configuration symbol for that yet, as far as I can tell. > > I don't think the guest kernels (including dom0) need the extensions to > run under Xen, they are only need by Xen itself. The guests should just > see a relatively normal v7 processor. > > Stefano, does that sound right? Keep in mind that we are using HVC to issue hypercalls, and HVC has been introduced with the virtualization extensions, if I am not mistaken.
On Wed, 2012-10-10 at 15:47 +0100, Stefano Stabellini wrote: > On Wed, 10 Oct 2012, Ian Campbell wrote: > > On Tue, 2012-10-09 at 19:21 +0100, Arnd Bergmann wrote: > > > On Tuesday 09 October 2012, Stefano Stabellini wrote: > > > > > config XEN > > > > > bool "Xen guest support on ARM (EXPERIMENTAL)" > > > > > depends on EXPERIMENTAL && ARM && OF > > > > > + depends on !CPU_V6 > > > > > help > > > > > Say Y if you want to run Linux in a Virtual Machine on Xen on ARM. > > > > > > > > Considering that we rely on the virtualization extensions, this one can > > > > be: > > > > > > > > depends on CPU_V7 > > > > > > > > The rest looks fine. I can submit a second patch to change !CPU_V6 into > > > > CPU_V7 later, if you prefer. > > > > > > CPU_V6 and CPU_V7 are not exclusive, I saw the problem when building a > > > combined kernel for both V6 and V7. The code also needs to depend on ARMv7 > > > with virtualization extensions, but that is a different issue. We don't > > > actually have a configuration symbol for that yet, as far as I can tell. > > > > I don't think the guest kernels (including dom0) need the extensions to > > run under Xen, they are only need by Xen itself. The guests should just > > see a relatively normal v7 processor. > > > > Stefano, does that sound right? > > Keep in mind that we are using HVC to issue hypercalls, and HVC has been > introduced with the virtualization extensions, if I am not mistaken. I think we can ignore that in this context since it is only used by bits which are already virtualisation aware -- i.e. you wouldn't want to add it to the Kconfig as a dependency for Xen. Ian.
On Wed, 10 Oct 2012, Ian Campbell wrote: > On Wed, 2012-10-10 at 15:47 +0100, Stefano Stabellini wrote: > > On Wed, 10 Oct 2012, Ian Campbell wrote: > > > On Tue, 2012-10-09 at 19:21 +0100, Arnd Bergmann wrote: > > > > On Tuesday 09 October 2012, Stefano Stabellini wrote: > > > > > > config XEN > > > > > > bool "Xen guest support on ARM (EXPERIMENTAL)" > > > > > > depends on EXPERIMENTAL && ARM && OF > > > > > > + depends on !CPU_V6 > > > > > > help > > > > > > Say Y if you want to run Linux in a Virtual Machine on Xen on ARM. > > > > > > > > > > Considering that we rely on the virtualization extensions, this one can > > > > > be: > > > > > > > > > > depends on CPU_V7 > > > > > > > > > > The rest looks fine. I can submit a second patch to change !CPU_V6 into > > > > > CPU_V7 later, if you prefer. > > > > > > > > CPU_V6 and CPU_V7 are not exclusive, I saw the problem when building a > > > > combined kernel for both V6 and V7. The code also needs to depend on ARMv7 > > > > with virtualization extensions, but that is a different issue. We don't > > > > actually have a configuration symbol for that yet, as far as I can tell. > > > > > > I don't think the guest kernels (including dom0) need the extensions to > > > run under Xen, they are only need by Xen itself. The guests should just > > > see a relatively normal v7 processor. > > > > > > Stefano, does that sound right? > > > > Keep in mind that we are using HVC to issue hypercalls, and HVC has been > > introduced with the virtualization extensions, if I am not mistaken. > > I think we can ignore that in this context since it is only used by bits > which are already virtualisation aware -- i.e. you wouldn't want to add > it to the Kconfig as a dependency for Xen. Considering that ARM CPU_V* symbols are not mutually exclusive, if Linux had a CPU_V7_VIRTEXT symbol I don't see why we shouldn't add it to our list of dependencies. It is more accurate than CPU_V7 after all. But I don't think that it is necessary to introduce one now only for Xen, because like you said, we only need HVC and that is only used in virtualization aware code.
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 6d2f7f5..85eaac3 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -1846,6 +1846,7 @@ config XEN_DOM0 config XEN bool "Xen guest support on ARM (EXPERIMENTAL)" depends on EXPERIMENTAL && ARM && OF + depends on !CPU_V6 help Say Y if you want to run Linux in a Virtual Machine on Xen on ARM. diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig index d4dffcd..126d8ce 100644 --- a/drivers/xen/Kconfig +++ b/drivers/xen/Kconfig @@ -3,6 +3,7 @@ menu "Xen driver support" config XEN_BALLOON bool "Xen memory balloon driver" + depends on !ARM default y help The balloon driver allows the Xen domain to request more memory from @@ -145,6 +146,7 @@ config SWIOTLB_XEN config XEN_TMEM bool + depends on !ARM default y if (CLEANCACHE || FRONTSWAP) help Shim to interface in-kernel Transcendent Memory hooks diff --git a/drivers/xen/sys-hypervisor.c b/drivers/xen/sys-hypervisor.c index 5e5ad7e..66a0a14 100644 --- a/drivers/xen/sys-hypervisor.c +++ b/drivers/xen/sys-hypervisor.c @@ -11,6 +11,7 @@ #include <linux/kernel.h> #include <linux/module.h> #include <linux/kobject.h> +#include <linux/err.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h>
* The XEN_BALLOON code requires the balloon infrastructure that is not getting built on ARM. * The tmem hypercall is not available on ARM * ARMv6 does not support cmpxchg on 16-bit words that are used in the * sys-hypervisor.c needs to include linux/err.h in order to use the IS_ERR/PTR_ERR/ERR_PTR family of functions. Signed-off-by: Arnd Bergmann <arnd@arndb.de> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Jeremy Fitzhardinge <jeremy@goop.org> Cc: xen-devel@lists.xensource.com --- arch/arm/Kconfig | 1 + drivers/xen/Kconfig | 2 ++ drivers/xen/sys-hypervisor.c | 1 + 3 files changed, 4 insertions(+)