diff mbox

mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?

Message ID aecf1ec0-7172-ec41-1759-be261f3e0dd2@de.bosch.com (mailing list archive)
State Not Applicable
Delegated to: Geert Uytterhoeven
Headers show

Commit Message

Dirk Behme Oct. 12, 2017, 9:13 a.m. UTC
On 12.10.2017 10:20, Dirk Behme wrote:
> On 11.10.2017 15:20, Geert Uytterhoeven wrote:
>> Hi Dirk,
>>
>> On Wed, Oct 11, 2017 at 2:57 PM, Dirk Behme <dirk.behme@de.bosch.com> 
>> wrote:
>>> On 11.10.2017 14:42, Geert Uytterhoeven wrote:
>>>> On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme <dirk.behme@de.bosch.com>
>>>> wrote:
>>>>> trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board 
>>>>> with
>>>>> rootfs on SD card I'm getting [1].
>>>>>
>>>>> Last time I think I used v4.13 in the same environment and I think it
>>>>> worked
>>>>> fine, most probably because renesas_sdhi_internal_dmac wasn't 
>>>>> there, yet
>>>>> ;)
>>>>>
>>>>> I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything 
>>>>> newer
>>>>> regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't 
>>>>> look
>>>>> so.
>>>>>
>>>>> Any idea what I missed?
>>>>>
>>>> How much swiotlb memory do you have? The default is 64 MiB:
>>>>
>>>>       software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at
>>>> [ffffffc037fff000-ffffffc03bffefff]
>>>
>>> Same here:
>>
>> OK.
>>
>>>> Then, who else is consuming lots of swiotlb memory?
>>
>> Or, not freeing allocated memory.
>>
>>> Hmm, any idea how to find that?
>>>
>>> The dmesg doesn't seem to have more infos about that (?)
>>
>> I'm afraid you have to add some prints to swiotlb_tbl_map_single()
>> (and swiotlb_tbl_unmap_single())...
> 
> 
> With [1] I'm getting [2]. What doesn't look that bad, at least quite 
> symmetric. Having a quick look I can't find any obvious not freed mem. 
> Hmm ...


Ok, it seems to be somehow easy ;) It seems that the 524288 bytes them 
self are the issue, independent on the usage of the swiotlb memory.

Doing a hack like [3], i.e. trying to map 524288 bytes at the first 
access, already, crashes, too [4].

So seems we have two issues?

a) swiotlb_tbl_map_single() can't handle 524288 bytes

and

b) failing swiotlb_tbl_map_single() results in a kernel crash


Now, I'm not sure whom to address both to? Is it an issue of 
renesas_sdhi_internal_dmac() which shouldn't call swiotlb with 524288 
bytes? Or is this an error of swiotlb? Same for the kernel crash.


Best regards

Dirk

[3]

+
         mask = dma_get_seg_boundary(hwdev);

         tbl_dma_addr &= mask;

[4]

renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: trying 
to map 8 bytes
renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: test: 
size modified to 524288 bytes
renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full (sz: 
524288 bytes)
renesas_sdhi_internal_dmac ee100000.sd: DMA: Out of SW-IOMMU space for 8 
bytes
Unable to handle kernel paging request at virtual address ffffffffc0000000

Comments

Yoshihiro Shimoda Oct. 16, 2017, 8:36 a.m. UTC | #1
Hi Dirk-san,

> From: Dirk Behme, Sent: Thursday, October 12, 2017 6:13 PM

> 

> On 12.10.2017 10:20, Dirk Behme wrote:

> > On 11.10.2017 15:20, Geert Uytterhoeven wrote:

> >> Hi Dirk,

> >>

> >> On Wed, Oct 11, 2017 at 2:57 PM, Dirk Behme <dirk.behme@de.bosch.com>

> >> wrote:

> >>> On 11.10.2017 14:42, Geert Uytterhoeven wrote:

> >>>> On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme <dirk.behme@de.bosch.com>

> >>>> wrote:

> >>>>> trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board

> >>>>> with

> >>>>> rootfs on SD card I'm getting [1].

> >>>>>

> >>>>> Last time I think I used v4.13 in the same environment and I think it

> >>>>> worked

> >>>>> fine, most probably because renesas_sdhi_internal_dmac wasn't

> >>>>> there, yet

> >>>>> ;)

> >>>>>

> >>>>> I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything

> >>>>> newer

> >>>>> regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't

> >>>>> look

> >>>>> so.

> >>>>>

> >>>>> Any idea what I missed?

> >>>>>

> >>>> How much swiotlb memory do you have? The default is 64 MiB:

> >>>>

> >>>>       software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at

> >>>> [ffffffc037fff000-ffffffc03bffefff]

> >>>

> >>> Same here:

> >>

> >> OK.

> >>

> >>>> Then, who else is consuming lots of swiotlb memory?

> >>

> >> Or, not freeing allocated memory.

> >>

> >>> Hmm, any idea how to find that?

> >>>

> >>> The dmesg doesn't seem to have more infos about that (?)

> >>

> >> I'm afraid you have to add some prints to swiotlb_tbl_map_single()

> >> (and swiotlb_tbl_unmap_single())...

> >

> >

> > With [1] I'm getting [2]. What doesn't look that bad, at least quite

> > symmetric. Having a quick look I can't find any obvious not freed mem.

> > Hmm ...

> 

> 

> Ok, it seems to be somehow easy ;) It seems that the 524288 bytes them

> self are the issue, independent on the usage of the swiotlb memory.

> 

> Doing a hack like [3], i.e. trying to map 524288 bytes at the first

> access, already, crashes, too [4].

> 

> So seems we have two issues?

> 

> a) swiotlb_tbl_map_single() can't handle 524288 bytes

> 

> and

> 

> b) failing swiotlb_tbl_map_single() results in a kernel crash

> 

> 

> Now, I'm not sure whom to address both to? Is it an issue of

> renesas_sdhi_internal_dmac() which shouldn't call swiotlb with 524288

> bytes? Or is this an error of swiotlb? Same for the kernel crash.


Our team also reproduced this issue if we did the following command on H3.
 # dd if=/dev/mmcblk0 of=/dev/null bs=512k count=200

I investigate the issue and then if I modified the value of IO_TLB_SEGSIZE to 256,
the issue disappeared. However, other issue happens if I dropped "count=200"...
So, I will investigate the issues more...

IIUC,
- Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size.
 - So, swiotlb will got request 64kbyte+ size. This is different with previous version.

Best regards,
Yoshihiro Shimoda

> 

> Best regards

> 

> Dirk

> 

> [3]

> 

> --- a/lib/swiotlb.c

> +++ b/lib/swiotlb.c

> @@ -510,6 +510,10 @@ phys_addr_t swiotlb_tbl_map_single(struct device

> *hwdev,

>          if (sme_active())

>                  pr_warn_once("SME is active and system is using DMA

> bounce buffers\n");

> 

> +       dev_warn(hwdev, "swiotlb_tbl_map_single: trying to map %zd

> bytes\n", size);

> +       size = (size_t)524288;

> +       dev_warn(hwdev, "swiotlb_tbl_map_single: test: size modified to

> %zd bytes\n", size);

> +

>          mask = dma_get_seg_boundary(hwdev);

> 

>          tbl_dma_addr &= mask;

> 

> [4]

> 

> renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: trying

> to map 8 bytes

> renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: test:

> size modified to 524288 bytes

> renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full (sz:

> 524288 bytes)

> renesas_sdhi_internal_dmac ee100000.sd: DMA: Out of SW-IOMMU space for 8

> bytes

> Unable to handle kernel paging request at virtual address ffffffffc0000000
Geert Uytterhoeven Oct. 16, 2017, 9:02 a.m. UTC | #2
CC LinusW, Ulf, linux-mmc

On Mon, Oct 16, 2017 at 10:36 AM, Yoshihiro Shimoda
<yoshihiro.shimoda.uh@renesas.com> wrote:
>> From: Dirk Behme, Sent: Thursday, October 12, 2017 6:13 PM
>> On 12.10.2017 10:20, Dirk Behme wrote:
>> > On 11.10.2017 15:20, Geert Uytterhoeven wrote:
>> >> On Wed, Oct 11, 2017 at 2:57 PM, Dirk Behme <dirk.behme@de.bosch.com>
>> >> wrote:
>> >>> On 11.10.2017 14:42, Geert Uytterhoeven wrote:
>> >>>> On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme <dirk.behme@de.bosch.com>
>> >>>> wrote:
>> >>>>> trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board
>> >>>>> with
>> >>>>> rootfs on SD card I'm getting [1].
>> >>>>>
>> >>>>> Last time I think I used v4.13 in the same environment and I think it
>> >>>>> worked
>> >>>>> fine, most probably because renesas_sdhi_internal_dmac wasn't
>> >>>>> there, yet
>> >>>>> ;)
>> >>>>>
>> >>>>> I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything
>> >>>>> newer
>> >>>>> regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't
>> >>>>> look
>> >>>>> so.
>> >>>>>
>> >>>>> Any idea what I missed?
>> >>>>>
>> >>>> How much swiotlb memory do you have? The default is 64 MiB:
>> >>>>
>> >>>>       software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at
>> >>>> [ffffffc037fff000-ffffffc03bffefff]
>> >>>
>> >>> Same here:
>> >>
>> >> OK.
>> >>
>> >>>> Then, who else is consuming lots of swiotlb memory?
>> >>
>> >> Or, not freeing allocated memory.
>> >>
>> >>> Hmm, any idea how to find that?
>> >>>
>> >>> The dmesg doesn't seem to have more infos about that (?)
>> >>
>> >> I'm afraid you have to add some prints to swiotlb_tbl_map_single()
>> >> (and swiotlb_tbl_unmap_single())...
>> >
>> >
>> > With [1] I'm getting [2]. What doesn't look that bad, at least quite
>> > symmetric. Having a quick look I can't find any obvious not freed mem.
>> > Hmm ...
>>
>>
>> Ok, it seems to be somehow easy ;) It seems that the 524288 bytes them
>> self are the issue, independent on the usage of the swiotlb memory.
>>
>> Doing a hack like [3], i.e. trying to map 524288 bytes at the first
>> access, already, crashes, too [4].
>>
>> So seems we have two issues?
>>
>> a) swiotlb_tbl_map_single() can't handle 524288 bytes
>>
>> and
>>
>> b) failing swiotlb_tbl_map_single() results in a kernel crash
>>
>>
>> Now, I'm not sure whom to address both to? Is it an issue of
>> renesas_sdhi_internal_dmac() which shouldn't call swiotlb with 524288
>> bytes? Or is this an error of swiotlb? Same for the kernel crash.
>
> Our team also reproduced this issue if we did the following command on H3.
>  # dd if=/dev/mmcblk0 of=/dev/null bs=512k count=200
>
> I investigate the issue and then if I modified the value of IO_TLB_SEGSIZE to 256,
> the issue disappeared. However, other issue happens if I dropped "count=200"...
> So, I will investigate the issues more...
>
> IIUC,
> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size.
>  - So, swiotlb will got request 64kbyte+ size. This is different with previous version.
>
> Best regards,
> Yoshihiro Shimoda
>
>>
>> Best regards
>>
>> Dirk
>>
>> [3]
>>
>> --- a/lib/swiotlb.c
>> +++ b/lib/swiotlb.c
>> @@ -510,6 +510,10 @@ phys_addr_t swiotlb_tbl_map_single(struct device
>> *hwdev,
>>          if (sme_active())
>>                  pr_warn_once("SME is active and system is using DMA
>> bounce buffers\n");
>>
>> +       dev_warn(hwdev, "swiotlb_tbl_map_single: trying to map %zd
>> bytes\n", size);
>> +       size = (size_t)524288;
>> +       dev_warn(hwdev, "swiotlb_tbl_map_single: test: size modified to
>> %zd bytes\n", size);
>> +
>>          mask = dma_get_seg_boundary(hwdev);
>>
>>          tbl_dma_addr &= mask;
>>
>> [4]
>>
>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: trying
>> to map 8 bytes
>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: test:
>> size modified to 524288 bytes
>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full (sz:
>> 524288 bytes)
>> renesas_sdhi_internal_dmac ee100000.sd: DMA: Out of SW-IOMMU space for 8
>> bytes
>> Unable to handle kernel paging request at virtual address ffffffffc0000000

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
Linus Walleij Oct. 16, 2017, 11:41 a.m. UTC | #3
On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:

>> IIUC,
>> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size.
>>  - So, swiotlb will got request 64kbyte+ size. This is different with previous version.

Other people reported problems with that. Sorry about this...

This problem should be solved by upstream
commit de3ee99b097dd51938276e3af388cd4ad0f2750a
"mmc: Delete bounce buffer handling"

I just killed off the bounce buffers so they can't hurt anyone anymore.

Can you check that you have this patch in your tree?

Yours,
Linus Walleij
Geert Uytterhoeven Oct. 16, 2017, 12:07 p.m. UTC | #4
Hi Linus,

On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij <linus.walleij@linaro.org> wrote:
> On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven
> <geert@linux-m68k.org> wrote:
>>> IIUC,
>>> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size.
>>>  - So, swiotlb will got request 64kbyte+ size. This is different with previous version.
>
> Other people reported problems with that. Sorry about this...
>
> This problem should be solved by upstream
> commit de3ee99b097dd51938276e3af388cd4ad0f2750a
> "mmc: Delete bounce buffer handling"
>
> I just killed off the bounce buffers so they can't hurt anyone anymore.
>
> Can you check that you have this patch in your tree?

Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already
included :-(

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
Linus Walleij Oct. 16, 2017, 12:22 p.m. UTC | #5
On Mon, Oct 16, 2017 at 2:07 PM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij <linus.walleij@linaro.org> wrote:
>> On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven
>> <geert@linux-m68k.org> wrote:
>>>> IIUC,
>>>> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size.
>>>>  - So, swiotlb will got request 64kbyte+ size. This is different with previous version.
>>
>> Other people reported problems with that. Sorry about this...
>>
>> This problem should be solved by upstream
>> commit de3ee99b097dd51938276e3af388cd4ad0f2750a
>> "mmc: Delete bounce buffer handling"
>>
>> I just killed off the bounce buffers so they can't hurt anyone anymore.
>>
>> Can you check that you have this patch in your tree?
>
> Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already
> included :-(

So the first issue is definately gone.

Then it seems the driver is specifying to high ->max_seg_size
since the buffer cannot be mapped?

Unless you're using ADMA drivers/mmc/host/sdhci.c sets it as:
mmc->max_seg_size = mmc->max_req_size;

And:
mmc->max_req_size = 524288;

>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: test:
>> size modified to 524288 bytes
>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full (sz:
>> 524288 bytes)

So if your driver cannot handle 524288 bytes, why is it saying it can
handle 524288 bytes?

The MMC core will believe the claim is true, I guess maybe something
in the current tree (such as coalesced blocks or whatever) is finally
making this size of requests hit the driver (which is a good thing...)
just that it was never doing that before and now it turns out it can't
really handle that because of how the memory architecture or something
works.

I guess you need to figure out how big requests/segments this hardware
can really handle and pass that to mmc->mas_req_size, mmc->max_seg_size.

Sorry if my reasoning is dumb... need more coffee.

Yours,
Linus Walleij
Yoshihiro Shimoda Oct. 17, 2017, 4:51 a.m. UTC | #6
Hi Linus-san,

> From: Linus Walleij, Sent: Monday, October 16, 2017 9:22 PM

> 

> On Mon, Oct 16, 2017 at 2:07 PM, Geert Uytterhoeven

> <geert@linux-m68k.org> wrote:

> > On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij <linus.walleij@linaro.org> wrote:

> >> On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven

> >> <geert@linux-m68k.org> wrote:

> >>>> IIUC,

> >>>> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size.

> >>>>  - So, swiotlb will got request 64kbyte+ size. This is different with previous version.

> >>

> >> Other people reported problems with that. Sorry about this...

> >>

> >> This problem should be solved by upstream

> >> commit de3ee99b097dd51938276e3af388cd4ad0f2750a

> >> "mmc: Delete bounce buffer handling"

> >>

> >> I just killed off the bounce buffers so they can't hurt anyone anymore.

> >>

> >> Can you check that you have this patch in your tree?

> >

> > Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already

> > included :-(

> 

> So the first issue is definately gone.

> 

> Then it seems the driver is specifying to high ->max_seg_size

> since the buffer cannot be mapped?

> 

> Unless you're using ADMA drivers/mmc/host/sdhci.c sets it as:

> mmc->max_seg_size = mmc->max_req_size;


Thank you for the comment!
On R-Car Gen3, it's using drivers/mmc/host/renesas_sdhi_internal_dmac.c,
and sets the max_seg_size to very big size (0xffffffff * 512) now...

So, I modified the value to 262144 and rebase the kernel to v4.14-rc5
(-rc4 seems have other issue in block driver.), the "swiotlb buffer is full" issue disappeared.

> And:

> mmc->max_req_size = 524288;

> 

> >> renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: test:

> >> size modified to 524288 bytes

> >> renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full (sz:

> >> 524288 bytes)

> 

> So if your driver cannot handle 524288 bytes, why is it saying it can

> handle 524288 bytes?


I think the "swiotlb buffer is full" came from lib/swiotlb.c.
I'm not sure the detail of swiotlb.c, but IIUC, it has a memory size limitation.

- In lib/swiotlb.c, it calculates nslots.
 - The size of one slot is 2048 bytes because IO_TLB_SHIFT in include/linux/swiotlb.h is set to 11.
- IO_TLB_SEGSIZE is set to 128 in include/linux/swiotlb.h.
 - This value is number of maximum slots.
  - So, maximum size for a single map of swiotlb is 256kbytes (2048 * 128).

So, I will submit a patch to set the max_req_size to 256kbytes today.

Best regards,
Yoshihiro Shimoda
Dirk Behme Oct. 17, 2017, 5:07 a.m. UTC | #7
Limiting the thread to Renesas only, again, to discuss the other issue:

On 17.10.2017 06:51, Yoshihiro Shimoda wrote:
> Hi Linus-san,
> 
>> From: Linus Walleij, Sent: Monday, October 16, 2017 9:22 PM
>>
>> On Mon, Oct 16, 2017 at 2:07 PM, Geert Uytterhoeven
>> <geert@linux-m68k.org> wrote:
>>> On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij <linus.walleij@linaro.org> wrote:
>>>> On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven
>>>> <geert@linux-m68k.org> wrote:
>>>>>> IIUC,
>>>>>> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size.
>>>>>>   - So, swiotlb will got request 64kbyte+ size. This is different with previous version.
>>>>
>>>> Other people reported problems with that. Sorry about this...
>>>>
>>>> This problem should be solved by upstream
>>>> commit de3ee99b097dd51938276e3af388cd4ad0f2750a
>>>> "mmc: Delete bounce buffer handling"
>>>>
>>>> I just killed off the bounce buffers so they can't hurt anyone anymore.
>>>>
>>>> Can you check that you have this patch in your tree?
>>>
>>> Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already
>>> included :-(
>>
>> So the first issue is definately gone.
>>
>> Then it seems the driver is specifying to high ->max_seg_size
>> since the buffer cannot be mapped?
>>
>> Unless you're using ADMA drivers/mmc/host/sdhci.c sets it as:
>> mmc->max_seg_size = mmc->max_req_size;
> 
> Thank you for the comment!
> On R-Car Gen3, it's using drivers/mmc/host/renesas_sdhi_internal_dmac.c,
> and sets the max_seg_size to very big size (0xffffffff * 512) now...
> 
> So, I modified the value to 262144 and rebase the kernel to v4.14-rc5
> (-rc4 seems have other issue in block driver.), the "swiotlb buffer is full" issue disappeared.
> 
>> And:
>> mmc->max_req_size = 524288;
>>
>>>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: test:
>>>> size modified to 524288 bytes
>>>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full (sz:
>>>> 524288 bytes)
>>
>> So if your driver cannot handle 524288 bytes, why is it saying it can
>> handle 524288 bytes?
> 
> I think the "swiotlb buffer is full" came from lib/swiotlb.c.
> I'm not sure the detail of swiotlb.c, but IIUC, it has a memory size limitation.
> 
> - In lib/swiotlb.c, it calculates nslots.
>   - The size of one slot is 2048 bytes because IO_TLB_SHIFT in include/linux/swiotlb.h is set to 11.
> - IO_TLB_SEGSIZE is set to 128 in include/linux/swiotlb.h.
>   - This value is number of maximum slots.
>    - So, maximum size for a single map of swiotlb is 256kbytes (2048 * 128).
> 
> So, I will submit a patch to set the max_req_size to 256kbytes today.


Great, many thanks! I'll try to test that in the next days, then.

While above fixes the swiotlb full issue, we still have the issue that 
it looks like the driver doesn't handle that issue correctly and 
crashes, then:

renesas_sdhi_internal_dmac ee100000.sd: DMA: Out of SW-IOMMU space for 
524288 bytes
-> Unable to handle kernel paging request at virtual address 
ffffffffc0000000 <-

I'd propose to fix that, too. Have you looked into that issue, too?

I'm no expert on this, but it looks to me that other drivers are 
checking the return value of dma_map_sg() differently?

ret = dma_map_sg(&host->pdev->dev, sg, host->sg_len, dir);
	if (ret < 0)
		return;

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/mmc/host/renesas_sdhi_internal_dmac.c#n165

They check for (ret == 0)?

Sorry if I'm wrong on that ;)

Best regards

Dirk
Yoshihiro Shimoda Oct. 17, 2017, 6:06 a.m. UTC | #8
Hi Dirk-san,

> From: Dirk Behme, Sent: Tuesday, October 17, 2017 2:08 PM

> 

> Limiting the thread to Renesas only, again, to discuss the other issue:

> 

> On 17.10.2017 06:51, Yoshihiro Shimoda wrote:

> > Hi Linus-san,

> >

> >> From: Linus Walleij, Sent: Monday, October 16, 2017 9:22 PM

> >>

> >> On Mon, Oct 16, 2017 at 2:07 PM, Geert Uytterhoeven

> >> <geert@linux-m68k.org> wrote:

> >>> On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij <linus.walleij@linaro.org> wrote:

> >>>> On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven

> >>>> <geert@linux-m68k.org> wrote:

> >>>>>> IIUC,

> >>>>>> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size.

> >>>>>>   - So, swiotlb will got request 64kbyte+ size. This is different with previous version.

> >>>>

> >>>> Other people reported problems with that. Sorry about this...

> >>>>

> >>>> This problem should be solved by upstream

> >>>> commit de3ee99b097dd51938276e3af388cd4ad0f2750a

> >>>> "mmc: Delete bounce buffer handling"

> >>>>

> >>>> I just killed off the bounce buffers so they can't hurt anyone anymore.

> >>>>

> >>>> Can you check that you have this patch in your tree?

> >>>

> >>> Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already

> >>> included :-(

> >>

> >> So the first issue is definately gone.

> >>

> >> Then it seems the driver is specifying to high ->max_seg_size

> >> since the buffer cannot be mapped?

> >>

> >> Unless you're using ADMA drivers/mmc/host/sdhci.c sets it as:

> >> mmc->max_seg_size = mmc->max_req_size;

> >

> > Thank you for the comment!

> > On R-Car Gen3, it's using drivers/mmc/host/renesas_sdhi_internal_dmac.c,

> > and sets the max_seg_size to very big size (0xffffffff * 512) now...

> >

> > So, I modified the value to 262144 and rebase the kernel to v4.14-rc5

> > (-rc4 seems have other issue in block driver.), the "swiotlb buffer is full" issue disappeared.

> >

> >> And:

> >> mmc->max_req_size = 524288;

> >>

> >>>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb_tbl_map_single: test:

> >>>> size modified to 524288 bytes

> >>>> renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full (sz:

> >>>> 524288 bytes)

> >>

> >> So if your driver cannot handle 524288 bytes, why is it saying it can

> >> handle 524288 bytes?

> >

> > I think the "swiotlb buffer is full" came from lib/swiotlb.c.

> > I'm not sure the detail of swiotlb.c, but IIUC, it has a memory size limitation.

> >

> > - In lib/swiotlb.c, it calculates nslots.

> >   - The size of one slot is 2048 bytes because IO_TLB_SHIFT in include/linux/swiotlb.h is set to 11.

> > - IO_TLB_SEGSIZE is set to 128 in include/linux/swiotlb.h.

> >   - This value is number of maximum slots.

> >    - So, maximum size for a single map of swiotlb is 256kbytes (2048 * 128).

> >

> > So, I will submit a patch to set the max_req_size to 256kbytes today.

> 

> 

> Great, many thanks! I'll try to test that in the next days, then.

> 

> While above fixes the swiotlb full issue, we still have the issue that

> it looks like the driver doesn't handle that issue correctly and

> crashes, then:

> 

> renesas_sdhi_internal_dmac ee100000.sd: DMA: Out of SW-IOMMU space for

> 524288 bytes

> -> Unable to handle kernel paging request at virtual address

> ffffffffc0000000 <-

> 

> I'd propose to fix that, too. Have you looked into that issue, too?

> 

> I'm no expert on this, but it looks to me that other drivers are

> checking the return value of dma_map_sg() differently?

> 

> ret = dma_map_sg(&host->pdev->dev, sg, host->sg_len, dir);

> 	if (ret < 0)

> 		return;

> 

> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/mmc/host/renesas_sdhi_internal_dmac.

> c#n165

> 

> They check for (ret == 0)?

> 

> Sorry if I'm wrong on that ;)


Thank you for the comments! You're correct. We should check for (ret == 0) at least.
I also think we should set host->force_pio = true and
call renesas_sdhi_internal_dmac_enable_dma(host, false) to do PIO if "ret == 0".

I will make such a patch and submit it.

Best regards,
Yoshihiro Shimoda

> Best regards

> 

> Dirk
diff mbox

Patch

--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -510,6 +510,10 @@  phys_addr_t swiotlb_tbl_map_single(struct device 
*hwdev,
         if (sme_active())
                 pr_warn_once("SME is active and system is using DMA 
bounce buffers\n");

+       dev_warn(hwdev, "swiotlb_tbl_map_single: trying to map %zd 
bytes\n", size);
+       size = (size_t)524288;
+       dev_warn(hwdev, "swiotlb_tbl_map_single: test: size modified to 
%zd bytes\n", size);