| Message ID | 20190214121054.11693-3-rajur@chelsio.com (mailing list archive) |
|---|---|
| State | Mainlined |
| Commit | f09ef134a7ca3f0d2ce485a757f5b79809ebb803 |
| Delegated to: | Jason Gunthorpe |
| Headers | show |
| Series | iw_cxgb4: Adjust the cq/qp mask | expand |
On Thu, Feb 14, 2019 at 05:40:54PM +0530, Raju Rangoju wrote: > Adjust the cq/qp mask based on no.of bar2 pages in a host page. > > For user-mode rdma, the granularity of the BAR2 memory mapped > to a user rdma process during queue allocation must be based > on the host page size. The lld attributes udb_density and > ucq_density are used to figure out how many sge contexts are > in a bar2 page. So the rdev->qpmask and rdev->cqmask in > iw_cxgb4 need to now be adjusted based on how many sge bar2 > pages are in a host page. Why is this rc? Do certain arches fail to work or something? Jason
On Thursday, February 02/14/19, 2019 at 15:41:34 +0000, Jason Gunthorpe wrote: > On Thu, Feb 14, 2019 at 05:40:54PM +0530, Raju Rangoju wrote: > > Adjust the cq/qp mask based on no.of bar2 pages in a host page. > > > > For user-mode rdma, the granularity of the BAR2 memory mapped > > to a user rdma process during queue allocation must be based > > on the host page size. The lld attributes udb_density and > > ucq_density are used to figure out how many sge contexts are > > in a bar2 page. So the rdev->qpmask and rdev->cqmask in > > iw_cxgb4 need to now be adjusted based on how many sge bar2 > > pages are in a host page. > > Why is this rc? Do certain arches fail to work or something? > Yes, this series fixes a regression that was introduced by commit 2391b0030e (v5.0-rc1~129^2~272) > Jason
> -----Original Message----- > From: linux-rdma-owner@vger.kernel.org <linux-rdma- > owner@vger.kernel.org> On Behalf Of Raju Rangoju > Sent: Thursday, February 14, 2019 11:28 AM > To: Jason Gunthorpe <jgg@mellanox.com> > Cc: davem@davemloft.net; linux-rdma@vger.kernel.org; > netdev@vger.kernel.org; swise@opengridcomputing.com > Subject: Re: [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2 > pages in a host page > > On Thursday, February 02/14/19, 2019 at 15:41:34 +0000, Jason Gunthorpe > wrote: > > On Thu, Feb 14, 2019 at 05:40:54PM +0530, Raju Rangoju wrote: > > > Adjust the cq/qp mask based on no.of bar2 pages in a host page. > > > > > > For user-mode rdma, the granularity of the BAR2 memory mapped > > > to a user rdma process during queue allocation must be based > > > on the host page size. The lld attributes udb_density and > > > ucq_density are used to figure out how many sge contexts are > > > in a bar2 page. So the rdev->qpmask and rdev->cqmask in > > > iw_cxgb4 need to now be adjusted based on how many sge bar2 > > > pages are in a host page. > > > > Why is this rc? Do certain arches fail to work or something? > > > > Yes, this series fixes a regression that was introduced by commit > 2391b0030e (v5.0-rc1~129^2~272) > > > Jason Rdma over cxgb4 on arches with a non-4K page size are busted w/o this fix. That was the motivation for -rc.
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c index c13c0ba30f63..d499cd61c0e8 100644 --- a/drivers/infiniband/hw/cxgb4/device.c +++ b/drivers/infiniband/hw/cxgb4/device.c @@ -783,6 +783,7 @@ void c4iw_init_dev_ucontext(struct c4iw_rdev *rdev, static int c4iw_rdev_open(struct c4iw_rdev *rdev) { int err; + unsigned int factor; c4iw_init_dev_ucontext(rdev, &rdev->uctx); @@ -806,8 +807,18 @@ static int c4iw_rdev_open(struct c4iw_rdev *rdev) return -EINVAL; } - rdev->qpmask = rdev->lldi.udb_density - 1; - rdev->cqmask = rdev->lldi.ucq_density - 1; + /* This implementation requires a sge_host_page_size <= PAGE_SIZE. */ + if (rdev->lldi.sge_host_page_size > PAGE_SIZE) { + pr_err("%s: unsupported sge host page size %u\n", + pci_name(rdev->lldi.pdev), + rdev->lldi.sge_host_page_size); + return -EINVAL; + } + + factor = PAGE_SIZE / rdev->lldi.sge_host_page_size; + rdev->qpmask = (rdev->lldi.udb_density * factor) - 1; + rdev->cqmask = (rdev->lldi.ucq_density * factor) - 1; + pr_debug("dev %s stag start 0x%0x size 0x%0x num stags %d pbl start 0x%0x size 0x%0x rq start 0x%0x size 0x%0x qp qid start %u size %u cq qid start %u size %u srq size %u\n", pci_name(rdev->lldi.pdev), rdev->lldi.vr->stag.start, rdev->lldi.vr->stag.size, c4iw_num_stags(rdev),
Adjust the cq/qp mask based on no.of bar2 pages in a host page. For user-mode rdma, the granularity of the BAR2 memory mapped to a user rdma process during queue allocation must be based on the host page size. The lld attributes udb_density and ucq_density are used to figure out how many sge contexts are in a bar2 page. So the rdev->qpmask and rdev->cqmask in iw_cxgb4 need to now be adjusted based on how many sge bar2 pages are in a host page. Fixes: 2391b0030e ("cxgb4: Remove SGE_HOST_PAGE_SIZE dependency on page size") Signed-off-by: Raju Rangoju <rajur@chelsio.com> --- drivers/infiniband/hw/cxgb4/device.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)