[for-4.14,2/3] xen/arm: Take into account the DMA width when allocating Dom0 memory banks
diff mbox series

Message ID 20200518113008.15422-3-julien@xen.org
State Superseded
Headers show
Series
  • Remove the 1GB limitation on Rasberry Pi 4
Related show

Commit Message

Julien Grall May 18, 2020, 11:30 a.m. UTC
From: Julien Grall <jgrall@amazon.com>

At the moment, Xen is assuming that all the devices are at least 32-bit
DMA capable. However, some SoCs have devices that may be able to access
a much restricted range. For instance, the Raspberry PI 4 has devices
that can only access the first GB of RAM.

The function arch_get_dma_bit_size() will return the lowest DMA width on
the platform. Use it to decide what is the limit for the low memory.

Signed-off-by: Julien GralL <jgrall@amazon.com>
---
 xen/arch/arm/domain_build.c | 32 +++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 13 deletions(-)

Comments

Volodymyr Babchuk May 18, 2020, 8:34 p.m. UTC | #1
Hi Julien,

On Mon, 2020-05-18 at 12:30 +0100, Julien Grall wrote:
> From: Julien Grall <jgrall@amazon.com>
> 
> At the moment, Xen is assuming that all the devices are at least 32-bit
> DMA capable. However, some SoCs have devices that may be able to access
> a much restricted range. For instance, the Raspberry PI 4 has devices
> that can only access the first GB of RAM.
> 
> The function arch_get_dma_bit_size() will return the lowest DMA width on
> the platform. Use it to decide what is the limit for the low memory.
> 
> Signed-off-by: Julien GralL <jgrall@amazon.com>
> ---
>  xen/arch/arm/domain_build.c | 32 +++++++++++++++++++-------------
>  1 file changed, 19 insertions(+), 13 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 430708753642..abc4e463d27c 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -211,10 +211,13 @@ fail:
>   *    the ramdisk and DTB must be placed within a certain proximity of
>   *    the kernel within RAM.
>   * 3. For dom0 we want to place as much of the RAM as we reasonably can
> - *    below 4GB, so that it can be used by non-LPAE enabled kernels (32-bit)
> + *    below 4GB, so that it can be used by non-LPAE enabled kernels (32-bit).
Is full stop really needed there?

>   *    or when a device assigned to dom0 can only do 32-bit DMA access.
> - * 4. For 32-bit dom0 the kernel must be located below 4GB.
> - * 5. We want to have a few largers banks rather than many smaller ones.
> + * 4. Some devices assigned to dom0 can only do 32-bit DMA access or
> + *    even be more restricted. We want to allocate as much of the RAM
> + *    as we reasonably can that can be accessed from all the devices..
> + * 5. For 32-bit dom0 the kernel must be located below 4GB.
> + * 6. We want to have a few largers banks rather than many smaller ones.
>   *
>   * For the first two requirements we need to make sure that the lowest
>   * bank is sufficiently large.
> @@ -245,9 +248,9 @@ fail:
>   * we give up.
>   *
>   * For 32-bit domain we require that the initial allocation for the
> - * first bank is under 4G. For 64-bit domain, the first bank is preferred
> - * to be allocated under 4G. Then for the subsequent allocations we
> - * initially allocate memory only from below 4GB. Once that runs out
> + * first bank is part of the low mem. For 64-bit, the first bank is preferred
> + * to be allocated in the low mem. Then for subsequent allocation, we
> + * initially allocate memory only from low mem. Once that runs out out
>   * (as described above) we allow higher allocations and continue until
>   * that runs out (or we have allocated sufficient dom0 memory).
>   */
> @@ -262,6 +265,7 @@ static void __init allocate_memory_11(struct domain *d,
>      int i;
>  
>      bool lowmem = true;
> +    unsigned int lowmem_bitsize = min(32U, arch_get_dma_bitsize());
>      unsigned int bits;
>  
>      /*
> @@ -282,7 +286,7 @@ static void __init allocate_memory_11(struct domain *d,
>       */
>      while ( order >= min_low_order )
>      {
> -        for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ )
> +        for ( bits = order ; bits <= lowmem_bitsize; bits++ )
>          {
>              pg = alloc_domheap_pages(d, order, MEMF_bits(bits));
>              if ( pg != NULL )
> @@ -296,24 +300,26 @@ static void __init allocate_memory_11(struct domain *d,
>          order--;
>      }
>  
> -    /* Failed to allocate bank0 under 4GB */
> +    /* Failed to allocate bank0 in the lowmem region. */
>      if ( is_32bit_domain(d) )
>          panic("Unable to allocate first memory bank\n");
>  
> -    /* Try to allocate memory from above 4GB */
> -    printk(XENLOG_INFO "No bank has been allocated below 4GB.\n");
> +    /* Try to allocate memory from above the lowmem region */
> +    printk(XENLOG_INFO "No bank has been allocated below %u-bit.\n",
> +           lowmem_bitsize);
>      lowmem = false;
>  
>   got_bank0:
>  
>      /*
> -     * If we failed to allocate bank0 under 4GB, continue allocating
> -     * memory from above 4GB and fill in banks.
> +     * If we failed to allocate bank0 in the lowmem region,
> +     * continue allocating from above the lowmem and fill in banks.
>       */
>      order = get_allocation_size(kinfo->unassigned_mem);
>      while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS )
>      {
> -        pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0);
> +        pg = alloc_domheap_pages(d, order,
> +                                 lowmem ? MEMF_bits(lowmem_bitsize) : 0);
>          if ( !pg )
>          {
>              order --;
Julien Grall May 19, 2020, 4:55 p.m. UTC | #2
On 18/05/2020 21:34, Volodymyr Babchuk wrote:
> Hi Julien,

Hi Volodymyr,

Thank you for the review.

> 
> On Mon, 2020-05-18 at 12:30 +0100, Julien Grall wrote:
>> From: Julien Grall <jgrall@amazon.com>
>>
>> At the moment, Xen is assuming that all the devices are at least 32-bit
>> DMA capable. However, some SoCs have devices that may be able to access
>> a much restricted range. For instance, the Raspberry PI 4 has devices
>> that can only access the first GB of RAM.
>>
>> The function arch_get_dma_bit_size() will return the lowest DMA width on
>> the platform. Use it to decide what is the limit for the low memory.
>>
>> Signed-off-by: Julien GralL <jgrall@amazon.com>
>> ---
>>   xen/arch/arm/domain_build.c | 32 +++++++++++++++++++-------------
>>   1 file changed, 19 insertions(+), 13 deletions(-)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 430708753642..abc4e463d27c 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -211,10 +211,13 @@ fail:
>>    *    the ramdisk and DTB must be placed within a certain proximity of
>>    *    the kernel within RAM.
>>    * 3. For dom0 we want to place as much of the RAM as we reasonably can
>> - *    below 4GB, so that it can be used by non-LPAE enabled kernels (32-bit)
>> + *    below 4GB, so that it can be used by non-LPAE enabled kernels (32-bit).
> Is full stop really needed there?

I was meant to remove the line below as it is now part of 4). I will 
remove it in the next version.

Best regards,

Patch
diff mbox series

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 430708753642..abc4e463d27c 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -211,10 +211,13 @@  fail:
  *    the ramdisk and DTB must be placed within a certain proximity of
  *    the kernel within RAM.
  * 3. For dom0 we want to place as much of the RAM as we reasonably can
- *    below 4GB, so that it can be used by non-LPAE enabled kernels (32-bit)
+ *    below 4GB, so that it can be used by non-LPAE enabled kernels (32-bit).
  *    or when a device assigned to dom0 can only do 32-bit DMA access.
- * 4. For 32-bit dom0 the kernel must be located below 4GB.
- * 5. We want to have a few largers banks rather than many smaller ones.
+ * 4. Some devices assigned to dom0 can only do 32-bit DMA access or
+ *    even be more restricted. We want to allocate as much of the RAM
+ *    as we reasonably can that can be accessed from all the devices..
+ * 5. For 32-bit dom0 the kernel must be located below 4GB.
+ * 6. We want to have a few largers banks rather than many smaller ones.
  *
  * For the first two requirements we need to make sure that the lowest
  * bank is sufficiently large.
@@ -245,9 +248,9 @@  fail:
  * we give up.
  *
  * For 32-bit domain we require that the initial allocation for the
- * first bank is under 4G. For 64-bit domain, the first bank is preferred
- * to be allocated under 4G. Then for the subsequent allocations we
- * initially allocate memory only from below 4GB. Once that runs out
+ * first bank is part of the low mem. For 64-bit, the first bank is preferred
+ * to be allocated in the low mem. Then for subsequent allocation, we
+ * initially allocate memory only from low mem. Once that runs out out
  * (as described above) we allow higher allocations and continue until
  * that runs out (or we have allocated sufficient dom0 memory).
  */
@@ -262,6 +265,7 @@  static void __init allocate_memory_11(struct domain *d,
     int i;
 
     bool lowmem = true;
+    unsigned int lowmem_bitsize = min(32U, arch_get_dma_bitsize());
     unsigned int bits;
 
     /*
@@ -282,7 +286,7 @@  static void __init allocate_memory_11(struct domain *d,
      */
     while ( order >= min_low_order )
     {
-        for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ )
+        for ( bits = order ; bits <= lowmem_bitsize; bits++ )
         {
             pg = alloc_domheap_pages(d, order, MEMF_bits(bits));
             if ( pg != NULL )
@@ -296,24 +300,26 @@  static void __init allocate_memory_11(struct domain *d,
         order--;
     }
 
-    /* Failed to allocate bank0 under 4GB */
+    /* Failed to allocate bank0 in the lowmem region. */
     if ( is_32bit_domain(d) )
         panic("Unable to allocate first memory bank\n");
 
-    /* Try to allocate memory from above 4GB */
-    printk(XENLOG_INFO "No bank has been allocated below 4GB.\n");
+    /* Try to allocate memory from above the lowmem region */
+    printk(XENLOG_INFO "No bank has been allocated below %u-bit.\n",
+           lowmem_bitsize);
     lowmem = false;
 
  got_bank0:
 
     /*
-     * If we failed to allocate bank0 under 4GB, continue allocating
-     * memory from above 4GB and fill in banks.
+     * If we failed to allocate bank0 in the lowmem region,
+     * continue allocating from above the lowmem and fill in banks.
      */
     order = get_allocation_size(kinfo->unassigned_mem);
     while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS )
     {
-        pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0);
+        pg = alloc_domheap_pages(d, order,
+                                 lowmem ? MEMF_bits(lowmem_bitsize) : 0);
         if ( !pg )
         {
             order --;