| Message ID | 1493904187-13686-1-git-send-email-sagi@grimberg.me (mailing list archive) |
|---|---|
| State | Accepted |
| Headers | show |
On Thu, May 04, 2017 at 04:23:07PM +0300, Sagi Grimberg wrote: > Their completely logical, so don't pose an artificial limitation. > > Signed-off-by: Sagi Grimberg <sagi@grimberg.me> > --- Thanks a lot, Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
> + dev->num_comp_vectors = num_possible_cpus();
why not num_online_cpus()?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
>> + dev->num_comp_vectors = num_possible_cpus(); > why not num_online_cpus()? Because that can change on the fly and should not effect on the number of logical completion vectors rxe exposes. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> Because that can change on the fly and should not effect on > the number of logical completion vectors rxe exposes. > > -- BTW, what will be the affect of this if rxe driver doesn't use the comp_vector parameter in create_cq()? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Because that can change on the fly and should not effect on >> the number of logical completion vectors rxe exposes. >> >> -- > BTW, what will be the affect of this if rxe driver doesn't use the > comp_vector parameter in create_cq()? Some ULPs won't create more queue-pairs than what the rdma device supports (per multi-queue session). One example for that exists in a patch set I submitted for exposing to ULPs IRQ affinity mappings through the RDMA core so they in turn can make smart assignment decisions for queue<->cpu mappings in a multi-queue environment. I think its safe to assume that any modern HCA drivers supports a fair share of completion vectors. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
So you say that even if rxe driver doesn't do anything with the comp_vector parameter passed to create cq, a ULP can benefit from rxe reporting num_comp_vectors = number of cpus. I can agree with that even though that IMHO the benefit deepens on how the real eth device handles received packets. Acked-by: Moni Shoua <monis@mellanox.com> thanks On Thu, May 4, 2017 at 5:26 PM, Sagi Grimberg <sagi@grimberg.me> wrote: > >>> Because that can change on the fly and should not effect on >>> the number of logical completion vectors rxe exposes. >>> >>> -- >> >> BTW, what will be the affect of this if rxe driver doesn't use the >> comp_vector parameter in create_cq()? > > > Some ULPs won't create more queue-pairs than what the rdma device > supports (per multi-queue session). One example for that exists in a > patch set I submitted for exposing to ULPs IRQ affinity mappings > through the RDMA core so they in turn can make smart assignment > decisions for queue<->cpu mappings in a multi-queue environment. > > I think its safe to assume that any modern HCA drivers supports a fair > share of completion vectors. > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2017-05-04 at 16:23 +0300, Sagi Grimberg wrote: > Their completely logical, so don't pose an artificial limitation. > > Signed-off-by: Sagi Grimberg <sagi@grimberg.me> *They're/*impose. Fixed those up during commit, applied for second pull request, thanks.
diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index 13ed2cc6eaa2..1b596fbbe251 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -114,7 +114,6 @@ enum rxe_device_param { RXE_MAX_UCONTEXT = 512, RXE_NUM_PORT = 1, - RXE_NUM_COMP_VECTORS = 1, RXE_MIN_QP_INDEX = 16, RXE_MAX_QP_INDEX = 0x00020000, diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 4c9080c94c06..3be4cbe365c6 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -1237,7 +1237,7 @@ int rxe_register_device(struct rxe_dev *rxe) dev->owner = THIS_MODULE; dev->node_type = RDMA_NODE_IB_CA; dev->phys_port_cnt = 1; - dev->num_comp_vectors = RXE_NUM_COMP_VECTORS; + dev->num_comp_vectors = num_possible_cpus(); dev->dev.parent = rxe_dma_device(rxe); dev->local_dma_lkey = 0; dev->node_guid = rxe_node_guid(rxe);
Their completely logical, so don't pose an artificial limitation. Signed-off-by: Sagi Grimberg <sagi@grimberg.me> --- drivers/infiniband/sw/rxe/rxe_param.h | 1 - drivers/infiniband/sw/rxe/rxe_verbs.c | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-)