diff mbox

[v2,00/26] IB: Optimize DMA mapping

Message ID 1484338667.2679.15.camel@sandisk.com (mailing list archive)
State Superseded
Headers show

Commit Message

Bart Van Assche Jan. 13, 2017, 8:18 p.m. UTC
On Fri, 2017-01-13 at 18:27 +0000, Estrin, Alex wrote:
> This series applied to 4.10-rc3 breaks hfi1 sdma engines.
> [   34.712343] hfi1 0000:82:00.0: hfi1_0: SDMA (0) engine error: 0x21
> state s50_HwHaltWait
> [   34.722752] hfi1 0000:82:00.0: hfi1_0: SDMA (0) descq_head: 0
> descq_tail: 3 freecnt: 2044 FLE 0
> [   34.733933] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[0]: flags:--F-
> addr:0x00008808248c8000 gen:0 len:36 bytes
> [   34.746595] hfi1 0000:82:00.0: hfi1_0:       desc0:0x80248808248c8000
> desc1 0x0000000000000000
> [   34.757218] hfi1 0000:82:00.0: hfi1_0:       aidx: 0 amode: 0 alen: 0
> [   34.765448] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[1]: flags:----
> addr:0x0000880829fd5000 gen:0 len:24 bytes
> [   34.778184] hfi1 0000:82:00.0: hfi1_0:       desc0:0x0018880829fd5000
> desc1 0x0000000000000000
> [   34.788941] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[2]: flags:-H-L
> addr:0x0000880829fd5018 gen:0 len:440 bytes
> [   34.801849] hfi1 0000:82:00.0: hfi1_0:       desc0:0x41b8880829fd5018
> desc1 0x0000000000000001
> ...etc
> 
> BTW, what was the base for this series, as 19/26 did not apply cleanly to
> 4.10-rc3?

Hello Alex,

Sorry for this. Can you check whether the patch below fixes this regression?

-- 
2.11.0

I will make sure that this series applies cleanly on top of v4.10-rc3 when I
repost it.

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Estrin, Alex Jan. 13, 2017, 9:08 p.m. UTC | #1
> 
> Hello Alex,
> 
> Sorry for this. Can you check whether the patch below fixes this regression?
> 
> diff --git a/drivers/infiniband/sw/rdmavt/vt.c
> b/drivers/infiniband/sw/rdmavt/vt.c
> index 6a81b179f631..f7fcd015ec09 100644
> --- a/drivers/infiniband/sw/rdmavt/vt.c
> +++ b/drivers/infiniband/sw/rdmavt/vt.c
> @@ -778,8 +778,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
>  	}
> 
>  	/* DMA Operations */
> -	if (rdi->ibdev.dma_device->dma_ops == NULL)
> -		set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
> +	set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
> 
>  	/* Protection Domain */
>  	spin_lock_init(&rdi->n_pds_lock);
> --
> 2.11.0
> 
> I will make sure that this series applies cleanly on top of v4.10-rc3 when I
> repost it.
> 
> Bart.

Hi Bart,
It didn't fix the failure... Apparently there is an issue with generic mapping itself. 
Thanks,
Alex.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Bart Van Assche Jan. 13, 2017, 9:59 p.m. UTC | #2
On Fri, 2017-01-13 at 21:08 +0000, Estrin, Alex wrote:
> It didn't fix the failure... Apparently there is an issue with generic
> mapping itself. 

Hello Alex,

The generic mapping code in lib/dma-virt.c works fine with at least the rxe
driver. Additionally, as far as I can see the mapping code in lib/dma-virt.c
is identical to the mapping in drivers/infiniband/sw/rdmavt/dma.c. What is
not clear to me however is how my patches can have caused the SDMA engine
error messages? As far as I can see the code in
drivers/infiniband/hw/hfi1/sdma.c uses PCI DMA mapping operations instead of
the generic dma_virt_ops. Is that correct?

Thanks,

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Estrin, Alex Jan. 14, 2017, 2:05 a.m. UTC | #3
> -----Original Message-----
> From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> Sent: Friday, January 13, 2017 5:00 PM
> To: Estrin, Alex <alex.estrin@intel.com>; dledford@redhat.com
> Cc: linux-kernel@vger.kernel.org; linux-rdma@vger.kernel.org;
> gregkh@linuxfoundation.org
> Subject: Re: [PATCH v2 00/26] IB: Optimize DMA mapping
> 
> On Fri, 2017-01-13 at 21:08 +0000, Estrin, Alex wrote:
> > It didn't fix the failure... Apparently there is an issue with generic
> > mapping itself.
> 
> Hello Alex,
> 
> The generic mapping code in lib/dma-virt.c works fine with at least the rxe
> driver. Additionally, as far as I can see the mapping code in lib/dma-virt.c
> is identical to the mapping in drivers/infiniband/sw/rdmavt/dma.c. What is
> not clear to me however is how my patches can have caused the SDMA engine
> error messages? As far as I can see the code in
> drivers/infiniband/hw/hfi1/sdma.c uses PCI DMA mapping operations instead of
> the generic dma_virt_ops. Is that correct?

Hi Bart,
If you refer to code like this 
from sdma.c: 
/* dma map the coalesce buffer */
                addr = dma_map_single(&dd->pcidev->dev,
..... 
please see hfi1/verbs.c @ hfi1_register_ib_device()  

'dma_device' field is initialized here
->	ibdev->dma_device = &dd->pcidev->dev;
	followed by 
	ret = rvt_register_device(&dd->verbs_dev.rdi);
	->	set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
 
Thanks,
Alex.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/infiniband/sw/rdmavt/vt.c
b/drivers/infiniband/sw/rdmavt/vt.c
index 6a81b179f631..f7fcd015ec09 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -778,8 +778,7 @@  int rvt_register_device(struct rvt_dev_info *rdi)
 	}
 
 	/* DMA Operations */
-	if (rdi->ibdev.dma_device->dma_ops == NULL)
-		set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
+	set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
 
 	/* Protection Domain */
 	spin_lock_init(&rdi->n_pds_lock);