From patchwork Thu Oct 23 00:59:15 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minh Duc Tran X-Patchwork-Id: 5137611 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id B92799F349 for ; Thu, 23 Oct 2014 00:59:59 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 919A72024C for ; Thu, 23 Oct 2014 00:59:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7C0CC2025A for ; Thu, 23 Oct 2014 00:59:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933562AbaJWA7v (ORCPT ); Wed, 22 Oct 2014 20:59:51 -0400 Received: from cmexedge2.emulex.com ([138.239.224.100]:33584 "EHLO CMEXEDGE2.ext.emulex.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933545AbaJWA7p convert rfc822-to-8bit (ORCPT ); Wed, 22 Oct 2014 20:59:45 -0400 Received: from CMEXHTCAS1.ad.emulex.com (138.239.115.217) by CMEXEDGE2.ext.emulex.com (138.239.224.100) with Microsoft SMTP Server (TLS) id 14.3.174.1; Wed, 22 Oct 2014 17:59:23 -0700 Received: from CMEXMB1.ad.emulex.com ([169.254.1.58]) by CMEXHTCAS1.ad.emulex.com ([2002:8aef:71b7::8aef:71b7]) with mapi id 14.03.0174.001; Wed, 22 Oct 2014 17:59:16 -0700 From: Minh Duc Tran To: Sagi Grimberg , Or Gerlitz , Jay Kallickal CC: "michaelc@cs.wisc.edu" , "linux-rdma@vger.kernel.org" , "Jayamohan Kallickal" Subject: [PATCH 1/1] IB/iser: re-adjust number of max_cqe and send_wr to hw supported number. Thread-Topic: [PATCH 1/1] IB/iser: re-adjust number of max_cqe and send_wr to hw supported number. Thread-Index: Ac/uWlrIOG9yS0a0TyidN3hzNJv+dQ== Date: Thu, 23 Oct 2014 00:59:15 +0000 Message-ID: <2cee0ee5-dd62-451a-a6a9-a237b59e8dd1@CMEXHTCAS1.ad.emulex.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [138.239.44.26] MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-8.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi Or and Sagi, After getting your feedbacks from the other thread, we have reworked with this new patch. Thanks. -Minh ------------- From: Minh Tran This patch allows iser to re-adjust accordingly to the resources being supported by underlying hardwares for max cqe per CQ and max send_wr per QP. Signed-off-by: Minh Tran Signed-off-by: Jayamohan Kallickal --- drivers/infiniband/ulp/iser/iscsi_iser.h | 4 ++++ drivers/infiniband/ulp/iser/iser_verbs.c | 22 +++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h index cd4174c..c75d99a 100644 --- a/drivers/infiniband/ulp/iser/iscsi_iser.h +++ b/drivers/infiniband/ulp/iser/iscsi_iser.h @@ -135,6 +135,10 @@ ISER_MAX_TX_MISC_PDUS + \ ISER_MAX_RX_MISC_PDUS) +#define ISER_GET_MAX_XMIT_CMDS(send_wr) (send_wr - ISER_MAX_TX_MISC_PDUS - \ + ISER_MAX_RX_MISC_PDUS) / \ + (1 + ISER_INFLIGHT_DATAOUTS) + /* Max registration work requests per command */ #define ISER_MAX_REG_WR_PER_CMD 5 diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c index 67225bb..41d7dec 100644 --- a/drivers/infiniband/ulp/iser/iser_verbs.c +++ b/drivers/infiniband/ulp/iser/iser_verbs.c @@ -76,7 +76,7 @@ static void iser_event_handler(struct ib_event_handler *handler, static int iser_create_device_ib_res(struct iser_device *device) { struct ib_device_attr *dev_attr = &device->dev_attr; - int ret, i; + int ret, i, max_cqe; ret = ib_query_device(device->ib_device, dev_attr); if (ret) { @@ -106,9 +106,12 @@ static int iser_create_device_ib_res(struct iser_device *device) device->comps_used = min(ISER_MAX_CQ, device->ib_device->num_comp_vectors); - iser_info("using %d CQs, device %s supports %d vectors\n", + + max_cqe = min(ISER_MAX_CQ_LEN, dev_attr->max_cqe); + + iser_info("using %d CQs, device %s supports %d vectors max_cqe %d\n", device->comps_used, device->ib_device->name, - device->ib_device->num_comp_vectors); + device->ib_device->num_comp_vectors, max_cqe); device->pd = ib_alloc_pd(device->ib_device); if (IS_ERR(device->pd)) @@ -118,11 +121,12 @@ static int iser_create_device_ib_res(struct iser_device *device) struct iser_comp *comp = &device->comps[i]; comp->device = device; + comp->cq = ib_create_cq(device->ib_device, iser_cq_callback, iser_cq_event_callback, (void *)comp, - ISER_MAX_CQ_LEN, i); + max_cqe, i); if (IS_ERR(comp->cq)) { comp->cq = NULL; goto cq_err; @@ -426,6 +430,7 @@ void iser_free_fastreg_pool(struct ib_conn *ib_conn) static int iser_create_ib_conn_res(struct ib_conn *ib_conn) { struct iser_device *device; + struct ib_device_attr *dev_attr; struct ib_qp_init_attr init_attr; int ret = -ENOMEM; int index, min_index = 0; @@ -433,6 +438,7 @@ static int iser_create_ib_conn_res(struct ib_conn *ib_conn) BUG_ON(ib_conn->device == NULL); device = ib_conn->device; + dev_attr = &device->dev_attr; memset(&init_attr, 0, sizeof init_attr); @@ -461,7 +467,13 @@ static int iser_create_ib_conn_res(struct ib_conn *ib_conn) init_attr.cap.max_send_wr = ISER_QP_SIG_MAX_REQ_DTOS + 1; init_attr.create_flags |= IB_QP_CREATE_SIGNATURE_EN; } else { - init_attr.cap.max_send_wr = ISER_QP_MAX_REQ_DTOS + 1; + if (dev_attr->max_qp_wr >= ISER_QP_MAX_REQ_DTOS) + init_attr.cap.max_send_wr = ISER_QP_MAX_REQ_DTOS; + else { + init_attr.cap.max_send_wr = dev_attr->max_qp_wr; + iser_err("lowering max QueueDepth to %d per qp\n", + ISER_GET_MAX_XMIT_CMDS(dev_attr->max_qp_wr)); + } } ret = rdma_create_qp(ib_conn->cma_id, device->pd, &init_attr);