From patchwork Wed Sep 28 20:26:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 9354867 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C17696077B for ; Wed, 28 Sep 2016 20:26:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B4196289AF for ; Wed, 28 Sep 2016 20:26:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A8ED029758; Wed, 28 Sep 2016 20:26:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47787289AF for ; Wed, 28 Sep 2016 20:26:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753091AbcI1U0s (ORCPT ); Wed, 28 Sep 2016 16:26:48 -0400 Received: from mail-pf0-f196.google.com ([209.85.192.196]:34629 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752880AbcI1U0s (ORCPT ); Wed, 28 Sep 2016 16:26:48 -0400 Received: by mail-pf0-f196.google.com with SMTP id 21so2563307pfy.1 for ; Wed, 28 Sep 2016 13:26:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=s3lHKtd1NBr3wUTv/Pi3PMXsemAY+y7NxGvBv7owqag=; b=bECdhNObhZx6u4nHP5s8415Dkb5HgG+qkB5e2xHaHrL0B9khxTqRgd7oV6tO2u8UbB RxiBrtTU0VZnbRbJQLXYd5hxq1mYtmXQhNOv67YEiaoWj1NGANXP1uZa5SmL+8ymJb0V IQgiCkGKWRz3ChKy7yGgcaK436ffQgKBDiMuk+ZwHP0NG7uL5Fw7WA9t86TMWaDvcDv3 rkDga9c97m0Kmwowz8jLiUmY6RyTSiKdAPLRCCuGqVBIfffe8bjFPgeWwjY+5AlnLBH4 KKGD1DU+xJDJ5jDKsGAKPsqA72M4I8VvvxLndt1kvTeRjvWat3uOhMjmo+br8XXEHwIB cNMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=s3lHKtd1NBr3wUTv/Pi3PMXsemAY+y7NxGvBv7owqag=; b=kx6CXkvpB1XncVcLF6flzj8ObzXYG9SHihRzzODdopSq7xGnjNZrP1ygBIs3iQbVn7 CmWzYBahh6ONEt3mewCiKGK2AFkcJRxxhSdP+rRiyz4Il+Ame0mGdFN8rlYuQQ4ZbLTG n/wsBufYoMXEmMRRSG1sUu2dD38ATZIeZ7bzRCgokHLWzLLLX3u7Kr67hyliU/fY7bW0 fLiKgj797jfx1l0z+5t0nPtdmT04Hw9waoziDk0cuL31kOu8lmWi0sRrHvmm4JvSKOsX OnZxrLRf77z9b1iEJVKf8s23DMAtpzShtRHJjsd9dqAcAY95v+opkqR9o0RebaBftHoH duzA== X-Gm-Message-State: AE9vXwNmsVCZSAjimJeroPEHOmOw22HApmaZmolSGN7mY3BWO6FKWugAz4tPAkzNlNmgNQ== X-Received: by 10.98.37.130 with SMTP id l124mr59543572pfl.60.1475094407233; Wed, 28 Sep 2016 13:26:47 -0700 (PDT) Received: from ip-172-31-4-47.us-west-2.compute.internal (ec2-52-43-58-33.us-west-2.compute.amazonaws.com. [52.43.58.33]) by smtp.gmail.com with ESMTPSA id s12sm14541456pfj.73.2016.09.28.13.26.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Sep 2016 13:26:46 -0700 (PDT) From: Parav Pandit To: monis@mellanox.com, linux-rdma@vger.kernel.org, dledford@redhat.com Cc: pandit.parav@gmail.com Subject: [PATCH] IB/rxe: Fix honoring max IRD value for rd/atomic. Date: Wed, 28 Sep 2016 20:26:44 +0000 Message-Id: <1475094404-18494-1-git-send-email-pandit.parav@gmail.com> X-Mailer: git-send-email 2.7.4 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch fixes honoring max incoming read request count instead of outgoing read req count (a) during modify qp by allocating response queue metadata (b) during incoming read request processing Signed-off-by: Parav Pandit --- drivers/infiniband/sw/rxe/rxe_loc.h | 2 +- drivers/infiniband/sw/rxe/rxe_qp.c | 24 +++++++++++++----------- drivers/infiniband/sw/rxe/rxe_resp.c | 2 +- 3 files changed, 15 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index 4a5484e..73849a5a 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -198,7 +198,7 @@ void free_rd_atomic_resource(struct rxe_qp *qp, struct resp_res *res); static inline void rxe_advance_resp_resource(struct rxe_qp *qp) { qp->resp.res_head++; - if (unlikely(qp->resp.res_head == qp->attr.max_rd_atomic)) + if (unlikely(qp->resp.res_head == qp->attr.max_dest_rd_atomic)) qp->resp.res_head = 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index f2c0dc2..b8036cf 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -146,7 +146,7 @@ static void free_rd_atomic_resources(struct rxe_qp *qp) if (qp->resp.resources) { int i; - for (i = 0; i < qp->attr.max_rd_atomic; i++) { + for (i = 0; i < qp->attr.max_dest_rd_atomic; i++) { struct resp_res *res = &qp->resp.resources[i]; free_rd_atomic_resource(qp, res); @@ -174,7 +174,7 @@ static void cleanup_rd_atomic_resources(struct rxe_qp *qp) struct resp_res *res; if (qp->resp.resources) { - for (i = 0; i < qp->attr.max_rd_atomic; i++) { + for (i = 0; i < qp->attr.max_dest_rd_atomic; i++) { res = &qp->resp.resources[i]; free_rd_atomic_resource(qp, res); } @@ -596,14 +596,21 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, if (mask & IB_QP_MAX_QP_RD_ATOMIC) { int max_rd_atomic = __roundup_pow_of_two(attr->max_rd_atomic); + qp->attr.max_rd_atomic = max_rd_atomic; + atomic_set(&qp->req.rd_atomic, max_rd_atomic); + } + + if (mask & IB_QP_MAX_DEST_RD_ATOMIC) { + int max_dest_rd_atomic = + __roundup_pow_of_two(attr->max_dest_rd_atomic); + + qp->attr.max_dest_rd_atomic = max_dest_rd_atomic; + free_rd_atomic_resources(qp); - err = alloc_rd_atomic_resources(qp, max_rd_atomic); + err = alloc_rd_atomic_resources(qp, max_dest_rd_atomic); if (err) return err; - - qp->attr.max_rd_atomic = max_rd_atomic; - atomic_set(&qp->req.rd_atomic, max_rd_atomic); } if (mask & IB_QP_CUR_STATE) @@ -704,11 +711,6 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, pr_debug("qp#%d set req psn = 0x%x\n", qp_num(qp), qp->req.psn); } - if (mask & IB_QP_MAX_DEST_RD_ATOMIC) { - qp->attr.max_dest_rd_atomic = - __roundup_pow_of_two(attr->max_dest_rd_atomic); - } - if (mask & IB_QP_PATH_MIG_STATE) qp->attr.path_mig_state = attr->path_mig_state; diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index d0be8f2..dd3d88a 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -383,7 +383,7 @@ static enum resp_states check_resource(struct rxe_qp *qp, * too many read/atomic ops, we just * recycle the responder resource queue */ - if (likely(qp->attr.max_rd_atomic > 0)) + if (likely(qp->attr.max_dest_rd_atomic > 0)) return RESPST_CHK_LENGTH; else return RESPST_ERR_TOO_MANY_RDMA_ATM_REQ;