| Message ID | 55C93C85.6090003@sandisk.com (mailing list archive) |
|---|---|
| State | Accepted |
| Headers | show |
On 8/11/2015 3:06 AM, Bart Van Assche wrote: > During a discussion in 2011 nobody recalled why FMR was not used for > non-page aligned buffers (see also > http://thread.gmane.org/gmane.linux.drivers.rdma/7149). Re-enable FMR > for such buffers. For the reason why the srp_map_fmr() function needs > to be modified, see also patch "IB/srp: rework mapping engine to use > multiple FMR entries" (commit ID 8f26c9ff9cd0; January 2011). > > Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com> Hi Bart, Just wanted to confirm that this was tested with FMR and non-aligned IO. (the change log doesn't say it). -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 08/16/15 03:53, Sagi Grimberg wrote: > On 8/11/2015 3:06 AM, Bart Van Assche wrote: >> During a discussion in 2011 nobody recalled why FMR was not used for >> non-page aligned buffers (see also >> http://thread.gmane.org/gmane.linux.drivers.rdma/7149). Re-enable FMR >> for such buffers. For the reason why the srp_map_fmr() function needs >> to be modified, see also patch "IB/srp: rework mapping engine to use >> multiple FMR entries" (commit ID 8f26c9ff9cd0; January 2011). >> >> Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com> > > Hi Bart, > > Just wanted to confirm that this was tested with FMR and non-aligned IO. > (the change log doesn't say it). Hello Sagi, This patch series has been tested with FMR and I/O buffers that were not aligned to a page boundary by setting the ib_srp kernel module parameter prefer_fr to 0. Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 2968b7b..887e8ca 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -1272,6 +1272,8 @@ static void srp_map_desc(struct srp_map_state *state, dma_addr_t dma_addr, static int srp_map_finish_fmr(struct srp_map_state *state, struct srp_rdma_ch *ch) { + struct srp_target_port *target = ch->target; + struct srp_device *dev = target->srp_host->srp_dev; struct ib_pool_fmr *fmr; u64 io_addr = 0; @@ -1283,7 +1285,8 @@ static int srp_map_finish_fmr(struct srp_map_state *state, *state->next_fmr++ = fmr; state->nmdesc++; - srp_map_desc(state, 0, state->dma_len, fmr->fmr->rkey); + srp_map_desc(state, state->base_dma_addr & ~dev->mr_page_mask, + state->dma_len, fmr->fmr->rkey); return 0; } @@ -1390,14 +1393,7 @@ static int srp_map_sg_entry(struct srp_map_state *state, return 0; } - /* - * Since not all RDMA HW drivers support non-zero page offsets for - * FMR, if we start at an offset into a page, don't merge into the - * current FMR mapping. Finish it out, and use the kernel's MR for - * this sg entry. - */ - if ((!dev->use_fast_reg && dma_addr & ~dev->mr_page_mask) || - dma_len > dev->mr_max_size) { + if (dma_len > dev->mr_max_size) { ret = srp_finish_mapping(state, ch); if (ret) return ret;
During a discussion in 2011 nobody recalled why FMR was not used for non-page aligned buffers (see also http://thread.gmane.org/gmane.linux.drivers.rdma/7149). Re-enable FMR for such buffers. For the reason why the srp_map_fmr() function needs to be modified, see also patch "IB/srp: rework mapping engine to use multiple FMR entries" (commit ID 8f26c9ff9cd0; January 2011). Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com> --- drivers/infiniband/ulp/srp/ib_srp.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)