| Message ID | 20160914074149.GA28922@linux-7smt.suse (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Hello, On 14/09/16 08:41, Peng Fan wrote: > On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c > index 35ab08d..cc71e6f 100644 > --- a/xen/arch/arm/domain_build.c > +++ b/xen/arch/arm/domain_build.c > @@ -28,6 +28,8 @@ > > static unsigned int __initdata opt_dom0_max_vcpus; > integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); > +static bool_t __initdata opt_dom0_use_lowmem; > +boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); > > int dom0_11_mapping = 1; > > @@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct > kernel_info *kinfo) > unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); > int i; > > - bool_t lowmem = is_32bit_domain(d); > + bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; > unsigned int bits; > > > Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. Again, what is the benefit to have a command line option for that? Regards,
Hello Julien, On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: >Hello, > >On 14/09/16 08:41, Peng Fan wrote: >>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: >>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c >>index 35ab08d..cc71e6f 100644 >>--- a/xen/arch/arm/domain_build.c >>+++ b/xen/arch/arm/domain_build.c >>@@ -28,6 +28,8 @@ >> >>static unsigned int __initdata opt_dom0_max_vcpus; >>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >>+static bool_t __initdata opt_dom0_use_lowmem; >>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); >> >>int dom0_11_mapping = 1; >> >>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct >>kernel_info *kinfo) >> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >> int i; >> >>- bool_t lowmem = is_32bit_domain(d); >>+ bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; >> unsigned int bits; >> >> >>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. > >Again, what is the benefit to have a command line option for that? Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? I just want to give user a choice. Thanks, Peng. > >Regards, > >-- >Julien Grall
On 14/09/16 13:03, Peng Fan wrote: > Hello Julien, Hello Peng, > On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: >> Hello, >> >> On 14/09/16 08:41, Peng Fan wrote: >>> On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: >>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c >>> index 35ab08d..cc71e6f 100644 >>> --- a/xen/arch/arm/domain_build.c >>> +++ b/xen/arch/arm/domain_build.c >>> @@ -28,6 +28,8 @@ >>> >>> static unsigned int __initdata opt_dom0_max_vcpus; >>> integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >>> +static bool_t __initdata opt_dom0_use_lowmem; >>> +boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); >>> >>> int dom0_11_mapping = 1; >>> >>> @@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct >>> kernel_info *kinfo) >>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >>> int i; >>> >>> - bool_t lowmem = is_32bit_domain(d); >>> + bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; >>> unsigned int bits; >>> >>> >>> Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. >> >> Again, what is the benefit to have a command line option for that? > > Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? > I just want to give user a choice. We don't add new command line parameter just because they look cool to have. So far, you did not explain why it would be good to let the choice to the user and how it could be used. Regards,
Hello Julien, On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote: > > >On 14/09/16 13:03, Peng Fan wrote: >>Hello Julien, > >Hello Peng, > >>On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: >>>Hello, >>> >>>On 14/09/16 08:41, Peng Fan wrote: >>>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: >>>>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c >>>>index 35ab08d..cc71e6f 100644 >>>>--- a/xen/arch/arm/domain_build.c >>>>+++ b/xen/arch/arm/domain_build.c >>>>@@ -28,6 +28,8 @@ >>>> >>>>static unsigned int __initdata opt_dom0_max_vcpus; >>>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >>>>+static bool_t __initdata opt_dom0_use_lowmem; >>>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); >>>> >>>>int dom0_11_mapping = 1; >>>> >>>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct >>>>kernel_info *kinfo) >>>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >>>> int i; >>>> >>>>- bool_t lowmem = is_32bit_domain(d); >>>>+ bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; >>>> unsigned int bits; >>>> >>>> >>>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. >>> >>>Again, what is the benefit to have a command line option for that? >> >>Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? >>I just want to give user a choice. > >We don't add new command line parameter just because they look cool to have. >So far, you did not explain why it would be good to let the choice to the >user and how it could be used. I have not try, if there is no lowmem. I have not look into alloc_domheap_pages. I am not sure whether there is such a platform or not, just thinking if there is soc that dram memory starts from 4GB, and there is no dram below 4GB. If we still can get memory when lowmem is true, I am ok to change directly assign lowmem with value true. Anyway I have not look into the internals of domheap and not sure whether there is such a platform that no lowmem (:- while ( order >= min_low_order ) { for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ ) { pg = alloc_domheap_pages(d, order, MEMF_bits(bits)); if ( pg != NULL ) goto got_bank0; } order--; } panic("Unable to allocate first memory bank"); Thanks, Peng. > >Regards, > >-- >Julien Grall
On 14/09/16 13:18, Peng Fan wrote: > Hello Julien, > > On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote: >> >> >> On 14/09/16 13:03, Peng Fan wrote: >>> Hello Julien, >> >> Hello Peng, >> >>> On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: >>>> Hello, >>>> >>>> On 14/09/16 08:41, Peng Fan wrote: >>>>> On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: >>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c >>>>> index 35ab08d..cc71e6f 100644 >>>>> --- a/xen/arch/arm/domain_build.c >>>>> +++ b/xen/arch/arm/domain_build.c >>>>> @@ -28,6 +28,8 @@ >>>>> >>>>> static unsigned int __initdata opt_dom0_max_vcpus; >>>>> integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >>>>> +static bool_t __initdata opt_dom0_use_lowmem; >>>>> +boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); >>>>> >>>>> int dom0_11_mapping = 1; >>>>> >>>>> @@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct >>>>> kernel_info *kinfo) >>>>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >>>>> int i; >>>>> >>>>> - bool_t lowmem = is_32bit_domain(d); >>>>> + bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; >>>>> unsigned int bits; >>>>> >>>>> >>>>> Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. >>>> >>>> Again, what is the benefit to have a command line option for that? >>> >>> Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? >>> I just want to give user a choice. >> >> We don't add new command line parameter just because they look cool to have. >> So far, you did not explain why it would be good to let the choice to the >> user and how it could be used. > > I have not try, if there is no lowmem. > > I have not look into alloc_domheap_pages. > I am not sure whether there is such a platform or not, > just thinking if there is soc that dram memory starts from 4GB, and there is no dram > below 4GB. If we still can get memory when lowmem is true, I am ok to change directly assign > lowmem with value true. Anyway I have not look into the internals of domheap and > not sure whether there is such a platform that no lowmem (:- We cannot exclude this possibility. However, the only reason that Xen is requiring to allocate a bank below 4GB for 32-bit domain is to handle non-LPAE kernel. I personally don't think this is a hard requirement and instead of panicking we should print a warning to say "no bank has been allocated below 4GB" or "a bank of N MB has been allocated below 4GB". So my suggestion is to allocate as much as possible lowmem (i.e below 4GB) and if it does not work print a warning then allocate the first bank above 4GB. I much prefer to let Xen decide itself what to do when possible rather than asking the user to specify himself whether lowmem is required. This is making life easier for distribution to support multiple platforms with Xen without having to modify the command line. Regards,
On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote: > > >On 14/09/16 13:18, Peng Fan wrote: >>Hello Julien, >> >>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote: >>> >>> >>>On 14/09/16 13:03, Peng Fan wrote: >>>>Hello Julien, >>> >>>Hello Peng, >>> >>>>On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: >>>>>Hello, >>>>> >>>>>On 14/09/16 08:41, Peng Fan wrote: >>>>>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: >>>>>>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c >>>>>>index 35ab08d..cc71e6f 100644 >>>>>>--- a/xen/arch/arm/domain_build.c >>>>>>+++ b/xen/arch/arm/domain_build.c >>>>>>@@ -28,6 +28,8 @@ >>>>>> >>>>>>static unsigned int __initdata opt_dom0_max_vcpus; >>>>>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >>>>>>+static bool_t __initdata opt_dom0_use_lowmem; >>>>>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); >>>>>> >>>>>>int dom0_11_mapping = 1; >>>>>> >>>>>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct >>>>>>kernel_info *kinfo) >>>>>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >>>>>> int i; >>>>>> >>>>>>- bool_t lowmem = is_32bit_domain(d); >>>>>>+ bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; >>>>>> unsigned int bits; >>>>>> >>>>>> >>>>>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. >>>>> >>>>>Again, what is the benefit to have a command line option for that? >>>> >>>>Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? >>>>I just want to give user a choice. >>> >>>We don't add new command line parameter just because they look cool to have. >>>So far, you did not explain why it would be good to let the choice to the >>>user and how it could be used. >> >>I have not try, if there is no lowmem. >> >>I have not look into alloc_domheap_pages. >>I am not sure whether there is such a platform or not, >>just thinking if there is soc that dram memory starts from 4GB, and there is no dram >>below 4GB. If we still can get memory when lowmem is true, I am ok to change directly assign >>lowmem with value true. Anyway I have not look into the internals of domheap and >>not sure whether there is such a platform that no lowmem (:- > >We cannot exclude this possibility. However, the only reason that Xen is >requiring to allocate a bank below 4GB for 32-bit domain is to handle >non-LPAE kernel. Now also need to handle device that have DMA limitation -:) > >I personally don't think this is a hard requirement and instead of panicking >we should print a warning to say "no bank has been allocated below 4GB" or "a >bank of N MB has been allocated below 4GB". > >So my suggestion is to allocate as much as possible lowmem (i.e below 4GB) >and if it does not work print a warning then allocate the first bank above >4GB. > >I much prefer to let Xen decide itself what to do when possible rather than >asking the user to specify himself whether lowmem is required. > >This is making life easier for distribution to support multiple platforms >with Xen without having to modify the command line. Thanks. I'll follow you suggestion and do some test. Then send out a new patch. Thanks, Peng. > >Regards, > >-- >Julien Grall
On Wed, Sep 14, 2016 at 08:40:09PM +0800, Peng Fan wrote: > On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote: > > > > > >On 14/09/16 13:18, Peng Fan wrote: > >>Hello Julien, > >> > >>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote: > >>> > >>> > >>>On 14/09/16 13:03, Peng Fan wrote: > >>>>Hello Julien, > >>> > >>>Hello Peng, > >>> > >>>>On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: > >>>>>Hello, > >>>>> > >>>>>On 14/09/16 08:41, Peng Fan wrote: > >>>>>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: > >>>>>>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c > >>>>>>index 35ab08d..cc71e6f 100644 > >>>>>>--- a/xen/arch/arm/domain_build.c > >>>>>>+++ b/xen/arch/arm/domain_build.c > >>>>>>@@ -28,6 +28,8 @@ > >>>>>> > >>>>>>static unsigned int __initdata opt_dom0_max_vcpus; > >>>>>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); > >>>>>>+static bool_t __initdata opt_dom0_use_lowmem; > >>>>>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); > >>>>>> > >>>>>>int dom0_11_mapping = 1; > >>>>>> > >>>>>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct > >>>>>>kernel_info *kinfo) > >>>>>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); > >>>>>> int i; > >>>>>> > >>>>>>- bool_t lowmem = is_32bit_domain(d); > >>>>>>+ bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; > >>>>>> unsigned int bits; > >>>>>> > >>>>>> > >>>>>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. > >>>>> > >>>>>Again, what is the benefit to have a command line option for that? > >>>> > >>>>Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? > >>>>I just want to give user a choice. > >>> > >>>We don't add new command line parameter just because they look cool to have. > >>>So far, you did not explain why it would be good to let the choice to the > >>>user and how it could be used. > >> > >>I have not try, if there is no lowmem. > >> > >>I have not look into alloc_domheap_pages. > >>I am not sure whether there is such a platform or not, > >>just thinking if there is soc that dram memory starts from 4GB, and there is no dram > >>below 4GB. If we still can get memory when lowmem is true, I am ok to change directly assign > >>lowmem with value true. Anyway I have not look into the internals of domheap and > >>not sure whether there is such a platform that no lowmem (:- > > > >We cannot exclude this possibility. However, the only reason that Xen is > >requiring to allocate a bank below 4GB for 32-bit domain is to handle > >non-LPAE kernel. > > Now also need to handle device that have DMA limitation -:) Hi Peng, Doesn't your platform have an IOMMU/SMMU? Cheers, Edgar
Hi Edgar, On Wed, Sep 14, 2016 at 04:16:58PM +0200, Edgar E. Iglesias wrote: >On Wed, Sep 14, 2016 at 08:40:09PM +0800, Peng Fan wrote: >> On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote: >> > >> > >> >On 14/09/16 13:18, Peng Fan wrote: >> >>Hello Julien, >> >> >> >>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote: >> >>> >> >>> >> >>>On 14/09/16 13:03, Peng Fan wrote: >> >>>>Hello Julien, >> >>> >> >>>Hello Peng, >> >>> >> >>>>On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: >> >>>>>Hello, >> >>>>> >> >>>>>On 14/09/16 08:41, Peng Fan wrote: >> >>>>>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: >> >>>>>>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c >> >>>>>>index 35ab08d..cc71e6f 100644 >> >>>>>>--- a/xen/arch/arm/domain_build.c >> >>>>>>+++ b/xen/arch/arm/domain_build.c >> >>>>>>@@ -28,6 +28,8 @@ >> >>>>>> >> >>>>>>static unsigned int __initdata opt_dom0_max_vcpus; >> >>>>>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >> >>>>>>+static bool_t __initdata opt_dom0_use_lowmem; >> >>>>>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); >> >>>>>> >> >>>>>>int dom0_11_mapping = 1; >> >>>>>> >> >>>>>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct >> >>>>>>kernel_info *kinfo) >> >>>>>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >> >>>>>> int i; >> >>>>>> >> >>>>>>- bool_t lowmem = is_32bit_domain(d); >> >>>>>>+ bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; >> >>>>>> unsigned int bits; >> >>>>>> >> >>>>>> >> >>>>>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. >> >>>>> >> >>>>>Again, what is the benefit to have a command line option for that? >> >>>> >> >>>>Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? >> >>>>I just want to give user a choice. >> >>> >> >>>We don't add new command line parameter just because they look cool to have. >> >>>So far, you did not explain why it would be good to let the choice to the >> >>>user and how it could be used. >> >> >> >>I have not try, if there is no lowmem. >> >> >> >>I have not look into alloc_domheap_pages. >> >>I am not sure whether there is such a platform or not, >> >>just thinking if there is soc that dram memory starts from 4GB, and there is no dram >> >>below 4GB. If we still can get memory when lowmem is true, I am ok to change directly assign >> >>lowmem with value true. Anyway I have not look into the internals of domheap and >> >>not sure whether there is such a platform that no lowmem (:- >> > >> >We cannot exclude this possibility. However, the only reason that Xen is >> >requiring to allocate a bank below 4GB for 32-bit domain is to handle >> >non-LPAE kernel. >> >> Now also need to handle device that have DMA limitation -:) > >Hi Peng, > >Doesn't your platform have an IOMMU/SMMU? We have SMMU. This is not related to SMMU. Dom0 use 1:1 mapping and no SMMU involved, the physical memory assigned to Dom0 maybe higher than 4GB, but Some IPs only supports 32bits DMA in Dom0. Then assign a 64bits dma address to a device only supports 32 bits device in Linux will hang the device or else. Regards, Peng. > >Cheers, >Edgar
On Thu, Sep 15, 2016 at 08:20:33AM +0800, Peng Fan wrote: > Hi Edgar, > On Wed, Sep 14, 2016 at 04:16:58PM +0200, Edgar E. Iglesias wrote: > >On Wed, Sep 14, 2016 at 08:40:09PM +0800, Peng Fan wrote: > >> On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote: > >> > > >> > > >> >On 14/09/16 13:18, Peng Fan wrote: > >> >>Hello Julien, > >> >> > >> >>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote: > >> >>> > >> >>> > >> >>>On 14/09/16 13:03, Peng Fan wrote: > >> >>>>Hello Julien, > >> >>> > >> >>>Hello Peng, > >> >>> > >> >>>>On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: > >> >>>>>Hello, > >> >>>>> > >> >>>>>On 14/09/16 08:41, Peng Fan wrote: > >> >>>>>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: > >> >>>>>>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c > >> >>>>>>index 35ab08d..cc71e6f 100644 > >> >>>>>>--- a/xen/arch/arm/domain_build.c > >> >>>>>>+++ b/xen/arch/arm/domain_build.c > >> >>>>>>@@ -28,6 +28,8 @@ > >> >>>>>> > >> >>>>>>static unsigned int __initdata opt_dom0_max_vcpus; > >> >>>>>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); > >> >>>>>>+static bool_t __initdata opt_dom0_use_lowmem; > >> >>>>>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); > >> >>>>>> > >> >>>>>>int dom0_11_mapping = 1; > >> >>>>>> > >> >>>>>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct > >> >>>>>>kernel_info *kinfo) > >> >>>>>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); > >> >>>>>> int i; > >> >>>>>> > >> >>>>>>- bool_t lowmem = is_32bit_domain(d); > >> >>>>>>+ bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; > >> >>>>>> unsigned int bits; > >> >>>>>> > >> >>>>>> > >> >>>>>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. > >> >>>>> > >> >>>>>Again, what is the benefit to have a command line option for that? > >> >>>> > >> >>>>Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? > >> >>>>I just want to give user a choice. > >> >>> > >> >>>We don't add new command line parameter just because they look cool to have. > >> >>>So far, you did not explain why it would be good to let the choice to the > >> >>>user and how it could be used. > >> >> > >> >>I have not try, if there is no lowmem. > >> >> > >> >>I have not look into alloc_domheap_pages. > >> >>I am not sure whether there is such a platform or not, > >> >>just thinking if there is soc that dram memory starts from 4GB, and there is no dram > >> >>below 4GB. If we still can get memory when lowmem is true, I am ok to change directly assign > >> >>lowmem with value true. Anyway I have not look into the internals of domheap and > >> >>not sure whether there is such a platform that no lowmem (:- > >> > > >> >We cannot exclude this possibility. However, the only reason that Xen is > >> >requiring to allocate a bank below 4GB for 32-bit domain is to handle > >> >non-LPAE kernel. > >> > >> Now also need to handle device that have DMA limitation -:) > > > >Hi Peng, > > > >Doesn't your platform have an IOMMU/SMMU? > > We have SMMU. This is not related to SMMU. Dom0 use 1:1 mapping and no SMMU involved, > the physical memory assigned to Dom0 maybe higher than 4GB, but Some IPs only > supports 32bits DMA in Dom0. Then assign a 64bits dma address to a device only supports 32 > bits device in Linux will hang the device or else. Well, I think it is somewhat related to the IOMMU. If your SMMU supports S1 + S2 translations, allthough not supported by Xen/ARM today, we could support nested SMMU's so that Dom0 could get it's own private S1 portion of the SMMU. Another option is to perhaps join into the efforts of PV-IOMMU and try to see if it would work for Xen on ARM: https://lists.xen.org/archives/html/xen-devel/2016-02/msg01428.html For platforms that have an IOMMU, I think both of these options may solve the use-case of dom0 using 32bit DMA devs without any lowmem. In addition to that, these features would be very nice as they also enable DMA API isolation and VFIO user-space drivers in dom0. Spending time on these kind of options seems worthwhile to me. With 32bit DMA devs, without an IOMMU, lowmem becomes critical but such systems are not really secure as has already been mentioned. I'm not sure it's worth introducing workarounds/hax for such systems. Cheers, Edgar
Hi, On 15/09/2016 09:26, Edgar E. Iglesias wrote: > On Thu, Sep 15, 2016 at 08:20:33AM +0800, Peng Fan wrote: >> Hi Edgar, >> On Wed, Sep 14, 2016 at 04:16:58PM +0200, Edgar E. Iglesias wrote: >>> On Wed, Sep 14, 2016 at 08:40:09PM +0800, Peng Fan wrote: >>>> On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote: >>>>> >>>>> >>>>> On 14/09/16 13:18, Peng Fan wrote: >>>>>> Hello Julien, >>>>>> >>>>>> On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote: >>>>>>> >>>>>>> >>>>>>> On 14/09/16 13:03, Peng Fan wrote: >>>>>>>> Hello Julien, >>>>>>> >>>>>>> Hello Peng, >>>>>>> >>>>>>>> On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> On 14/09/16 08:41, Peng Fan wrote: >>>>>>>>>> On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: >>>>>>>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c >>>>>>>>>> index 35ab08d..cc71e6f 100644 >>>>>>>>>> --- a/xen/arch/arm/domain_build.c >>>>>>>>>> +++ b/xen/arch/arm/domain_build.c >>>>>>>>>> @@ -28,6 +28,8 @@ >>>>>>>>>> >>>>>>>>>> static unsigned int __initdata opt_dom0_max_vcpus; >>>>>>>>>> integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >>>>>>>>>> +static bool_t __initdata opt_dom0_use_lowmem; >>>>>>>>>> +boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); >>>>>>>>>> >>>>>>>>>> int dom0_11_mapping = 1; >>>>>>>>>> >>>>>>>>>> @@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct >>>>>>>>>> kernel_info *kinfo) >>>>>>>>>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >>>>>>>>>> int i; >>>>>>>>>> >>>>>>>>>> - bool_t lowmem = is_32bit_domain(d); >>>>>>>>>> + bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; >>>>>>>>>> unsigned int bits; >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. >>>>>>>>> >>>>>>>>> Again, what is the benefit to have a command line option for that? >>>>>>>> >>>>>>>> Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? >>>>>>>> I just want to give user a choice. >>>>>>> >>>>>>> We don't add new command line parameter just because they look cool to have. >>>>>>> So far, you did not explain why it would be good to let the choice to the >>>>>>> user and how it could be used. >>>>>> >>>>>> I have not try, if there is no lowmem. >>>>>> >>>>>> I have not look into alloc_domheap_pages. >>>>>> I am not sure whether there is such a platform or not, >>>>>> just thinking if there is soc that dram memory starts from 4GB, and there is no dram >>>>>> below 4GB. If we still can get memory when lowmem is true, I am ok to change directly assign >>>>>> lowmem with value true. Anyway I have not look into the internals of domheap and >>>>>> not sure whether there is such a platform that no lowmem (:- >>>>> >>>>> We cannot exclude this possibility. However, the only reason that Xen is >>>>> requiring to allocate a bank below 4GB for 32-bit domain is to handle >>>>> non-LPAE kernel. >>>> >>>> Now also need to handle device that have DMA limitation -:) >>> >>> Hi Peng, >>> >>> Doesn't your platform have an IOMMU/SMMU? >> >> We have SMMU. This is not related to SMMU. Dom0 use 1:1 mapping and no SMMU involved, >> the physical memory assigned to Dom0 maybe higher than 4GB, but Some IPs only >> supports 32bits DMA in Dom0. Then assign a 64bits dma address to a device only supports 32 >> bits device in Linux will hang the device or else. > > Well, I think it is somewhat related to the IOMMU. > > If your SMMU supports S1 + S2 translations, allthough not supported > by Xen/ARM today, we could support nested SMMU's so that Dom0 could > get it's own private S1 portion of the SMMU. > > Another option is to perhaps join into the efforts of PV-IOMMU > and try to see if it would work for Xen on ARM: > https://lists.xen.org/archives/html/xen-devel/2016-02/msg01428.html > > For platforms that have an IOMMU, I think both of these options may > solve the use-case of dom0 using 32bit DMA devs without any lowmem. > In addition to that, these features would be very nice as they also > enable DMA API isolation and VFIO user-space drivers in dom0. > Spending time on these kind of options seems worthwhile to me. There is a third option. The direct memory mapping for DOM0 is a workaround for platform where not all DMA-capable devices are protected by an SMMU. If you know the platform does not have such devices, it would be possible to remove the direct memory mapping. In this case, Xen will allocate the RAM anywhere in the physical memory but will be mapped in DOM0 using the memory region find the device tree. (e.g if the first bank is below 4GB, DOM0 will have memory mapped below 4GB). You would also have to remove specific xen_dma_ops (swiotlb-xen callback) as the device will be fully protected (and no workaround is necessary). I suggested a generic way few years ago [1] that is a step towards removing direct mapping. But i never had the time to continue it. > > With 32bit DMA devs, without an IOMMU, lowmem becomes critical but > such systems are not really secure as has already been mentioned. > I'm not sure it's worth introducing workarounds/hax for such > systems. Well, Dom0 (aka hardware domain) is a trusted domain as it owns most of the devices. There are so many other way to screw up the platform. So it would be fine to allocate lowmem for that domain. The SMMU is only mandatory for any device passthrough to all the other domain. Regards, [1] https://lists.xen.org/archives/html/xen-devel/2014-02/msg01897.html
Hi Edgar, On Thu, Sep 15, 2016 at 10:26:46AM +0200, Edgar E. Iglesias wrote: >On Thu, Sep 15, 2016 at 08:20:33AM +0800, Peng Fan wrote: >> Hi Edgar, >> On Wed, Sep 14, 2016 at 04:16:58PM +0200, Edgar E. Iglesias wrote: >> >On Wed, Sep 14, 2016 at 08:40:09PM +0800, Peng Fan wrote: >> >> On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote: >> >> > >> >> > >> >> >On 14/09/16 13:18, Peng Fan wrote: >> >> >>Hello Julien, >> >> >> >> >> >>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote: >> >> >>> >> >> >>> >> >> >>>On 14/09/16 13:03, Peng Fan wrote: >> >> >>>>Hello Julien, >> >> >>> >> >> >>>Hello Peng, >> >> >>> >> >> >>>>On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote: >> >> >>>>>Hello, >> >> >>>>> >> >> >>>>>On 14/09/16 08:41, Peng Fan wrote: >> >> >>>>>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote: >> >> >>>>>>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c >> >> >>>>>>index 35ab08d..cc71e6f 100644 >> >> >>>>>>--- a/xen/arch/arm/domain_build.c >> >> >>>>>>+++ b/xen/arch/arm/domain_build.c >> >> >>>>>>@@ -28,6 +28,8 @@ >> >> >>>>>> >> >> >>>>>>static unsigned int __initdata opt_dom0_max_vcpus; >> >> >>>>>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >> >> >>>>>>+static bool_t __initdata opt_dom0_use_lowmem; >> >> >>>>>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); >> >> >>>>>> >> >> >>>>>>int dom0_11_mapping = 1; >> >> >>>>>> >> >> >>>>>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct >> >> >>>>>>kernel_info *kinfo) >> >> >>>>>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >> >> >>>>>> int i; >> >> >>>>>> >> >> >>>>>>- bool_t lowmem = is_32bit_domain(d); >> >> >>>>>>+ bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; >> >> >>>>>> unsigned int bits; >> >> >>>>>> >> >> >>>>>> >> >> >>>>>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible. >> >> >>>>> >> >> >>>>>Again, what is the benefit to have a command line option for that? >> >> >>>> >> >> >>>>Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t lowmem = true" ? >> >> >>>>I just want to give user a choice. >> >> >>> >> >> >>>We don't add new command line parameter just because they look cool to have. >> >> >>>So far, you did not explain why it would be good to let the choice to the >> >> >>>user and how it could be used. >> >> >> >> >> >>I have not try, if there is no lowmem. >> >> >> >> >> >>I have not look into alloc_domheap_pages. >> >> >>I am not sure whether there is such a platform or not, >> >> >>just thinking if there is soc that dram memory starts from 4GB, and there is no dram >> >> >>below 4GB. If we still can get memory when lowmem is true, I am ok to change directly assign >> >> >>lowmem with value true. Anyway I have not look into the internals of domheap and >> >> >>not sure whether there is such a platform that no lowmem (:- >> >> > >> >> >We cannot exclude this possibility. However, the only reason that Xen is >> >> >requiring to allocate a bank below 4GB for 32-bit domain is to handle >> >> >non-LPAE kernel. >> >> >> >> Now also need to handle device that have DMA limitation -:) >> > >> >Hi Peng, >> > >> >Doesn't your platform have an IOMMU/SMMU? >> >> We have SMMU. This is not related to SMMU. Dom0 use 1:1 mapping and no SMMU involved, >> the physical memory assigned to Dom0 maybe higher than 4GB, but Some IPs only >> supports 32bits DMA in Dom0. Then assign a 64bits dma address to a device only supports 32 >> bits device in Linux will hang the device or else. > >Well, I think it is somewhat related to the IOMMU. > >If your SMMU supports S1 + S2 translations, allthough not supported >by Xen/ARM today, we could support nested SMMU's so that Dom0 could >get it's own private S1 portion of the SMMU. For DomU or hardware domain, I think supporting SMMU with Linux S1 + XEN S2 is a good feature to have. For Dom0, I think no need to use S1 + S2 for SMMU, It is control domain. > >Another option is to perhaps join into the efforts of PV-IOMMU >and try to see if it would work for Xen on ARM: >https://lists.xen.org/archives/html/xen-devel/2016-02/msg01428.html This is new to me -:) Will get back to you when I know more about this. > >For platforms that have an IOMMU, I think both of these options may >solve the use-case of dom0 using 32bit DMA devs without any lowmem. >In addition to that, these features would be very nice as they also >enable DMA API isolation and VFIO user-space drivers in dom0. >Spending time on these kind of options seems worthwhile to me. If SMMU S1 is supported, we could use VFIO. This's good feature to have. If you have any progress or new update, please kindly CC me. I am happy to see this. > >With 32bit DMA devs, without an IOMMU, lowmem becomes critical but >such systems are not really secure as has already been mentioned. >I'm not sure it's worth introducing workarounds/hax for such >systems. The proposed two options may cost big efforts I think -:) I prefer Julien's suggestion to fix the issue I met, since easy to implement. Anyway from the long run, your two options are good feature to have. Thanks, Peng. > >Cheers, >Edgar
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 35ab08d..cc71e6f 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -28,6 +28,8 @@ static unsigned int __initdata opt_dom0_max_vcpus; integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); +static bool_t __initdata opt_dom0_use_lowmem; +boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem); int dom0_11_mapping = 1; @@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct kernel_info *kinfo) unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); int i; - bool_t lowmem = is_32bit_domain(d); + bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem; unsigned int bits;