From patchwork Wed Mar 15 14:52:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 13175999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E823C61DA4 for ; Wed, 15 Mar 2023 14:55:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233282AbjCOOzs (ORCPT ); Wed, 15 Mar 2023 10:55:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36586 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233281AbjCOOz2 (ORCPT ); Wed, 15 Mar 2023 10:55:28 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BAEB11E91 for ; Wed, 15 Mar 2023 07:54:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678892073; x=1710428073; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VC9vf+pZ8kV7hM/D8Xm0DQjYINcphuz+F0BEGj0tXDU=; b=K7abnaNCWrGE1Srz7DwBcdumn/4zeDfh8VCDpbJB+Zt69k91wHWwYS13 7tAnnu+lY/sKAYbTNjCh66fDqjdbKG3IlqrYTtFsvM9bLMJxVZpVGoyNl xYRfo+FdPa6AatXMd1gMwk99HQkS0niGIFl2u5jFquysq1159lYGZBHXr Tp7FGOS8ENSUzuHq1BTUN03cuvbuLJ8dJ59uy34A6gFxiEstXwGKaNbyt wbi6FlKeA38thwarXhgAcF1ZnBiDCnxxpPPchOn7n9xKSJnZPle0M9yy1 HpzkP7HQSot1lqD6MHtNk9JgDgttwIJeRo4nhfREHdQBps/v014KZd7+E Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10649"; a="321561526" X-IronPort-AV: E=Sophos;i="5.98,262,1673942400"; d="scan'208";a="321561526" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2023 07:52:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10649"; a="748457444" X-IronPort-AV: E=Sophos;i="5.98,262,1673942400"; d="scan'208";a="748457444" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.255.35.84]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2023 07:52:45 -0700 From: Shiraz Saleem To: jgg@nvidia.com, leon@kernel.org Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-rc 1/4] RDMA/irdma: Do not generate SW completions for NOPs Date: Wed, 15 Mar 2023 09:52:28 -0500 Message-Id: <20230315145231.931-2-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230315145231.931-1-shiraz.saleem@intel.com> References: <20230315145231.931-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail Currently, artificial SW completions are generated for NOP wqes which can generate unexpected completions with wr_id = 0. Skip the generation of artificial completions for NOPs. Fixes: 81091d7696ae ("RDMA/irdma: Add SW mechanism to generate completions on error") Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/utils.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c index 445e69e8..7887230 100644 --- a/drivers/infiniband/hw/irdma/utils.c +++ b/drivers/infiniband/hw/irdma/utils.c @@ -2595,7 +2595,10 @@ void irdma_generate_flush_completions(struct irdma_qp *iwqp) /* remove the SQ WR by moving SQ tail*/ IRDMA_RING_SET_TAIL(*sq_ring, sq_ring->tail + qp->sq_wrtrk_array[sq_ring->tail].quanta); - + if (cmpl->cpi.op_type == IRDMAQP_OP_NOP) { + kfree(cmpl); + continue; + } ibdev_dbg(iwqp->iwscq->ibcq.device, "DEV: %s: adding wr_id = 0x%llx SQ Completion to list qp_id=%d\n", __func__, cmpl->cpi.wr_id, qp->qp_id); From patchwork Wed Mar 15 14:52:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 13175998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE1F2C6FD1D for ; Wed, 15 Mar 2023 14:55:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232491AbjCOOzf (ORCPT ); Wed, 15 Mar 2023 10:55:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232925AbjCOOzJ (ORCPT ); Wed, 15 Mar 2023 10:55:09 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95F939028 for ; Wed, 15 Mar 2023 07:54:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678892056; x=1710428056; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aTl2vtN3cwYPirEjRT4yCJZMmsTz4eHmoTimWby6ZTE=; b=CQ952CHNArRqdnVSOfu1wgO081ggWLJItHOqNLR8HJlC0RWfMIWuluMQ 0eT+Pdasib1BgRkQUkzt5nlPiWxLnXF3SS6PgP03RpDF7/Ilt2Z410M+q 1n2i3DoKPVfwTHrW8/MB+Gkgt14NDOKXk4IQAK0oKax9klxp0P61IyDyN gV90S0XpLCry7R9SfsFbdmrf/Z71RdP8PoN8Zbvigdqrm9Dicx8bniqEJ RjYgtJ+d+LxiUmZt94+r7gA+/W8CzouPUMciBBPZcU8YFYiktaeXHGEhC 4EOUNdhLtXSl/BmULGlicdegqg+WIIfJM6Y9JduiZ0rXRgiVGhjmJtl4e Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10649"; a="321561534" X-IronPort-AV: E=Sophos;i="5.98,262,1673942400"; d="scan'208";a="321561534" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2023 07:52:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10649"; a="748457447" X-IronPort-AV: E=Sophos;i="5.98,262,1673942400"; d="scan'208";a="748457447" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.255.35.84]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2023 07:52:46 -0700 From: Shiraz Saleem To: jgg@nvidia.com, leon@kernel.org Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-rc 2/4] RDMA/irdma: Fix memory leak of PBLE objects Date: Wed, 15 Mar 2023 09:52:29 -0500 Message-Id: <20230315145231.931-3-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230315145231.931-1-shiraz.saleem@intel.com> References: <20230315145231.931-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail On rmmod of irdma, the PBLE object memory is not being freed. PBLE object memory are not statically pre-allocated at function initialization time unlike other HMC objects. PBLEs objects and the Segment Descriptors (SD) for it can be dynamically allocated during scale up and SD's remain allocated till function deinitialization. Fix this leak by adding IRDMA_HMC_IW_PBLE to the iw_hmc_obj_types[] table and skip pbles in irdma_create_hmc_obj but not in irdma_del_hmc_objects(). Fixes: 44d9e52977a1 ("RDMA/irdma: Implement device initialization definitions") Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/hw.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c index 2e1e2ba..43dfa47 100644 --- a/drivers/infiniband/hw/irdma/hw.c +++ b/drivers/infiniband/hw/irdma/hw.c @@ -41,6 +41,7 @@ IRDMA_HMC_IW_XFFL, IRDMA_HMC_IW_Q1, IRDMA_HMC_IW_Q1FL, + IRDMA_HMC_IW_PBLE, IRDMA_HMC_IW_TIMER, IRDMA_HMC_IW_FSIMC, IRDMA_HMC_IW_FSIAV, @@ -827,6 +828,8 @@ static int irdma_create_hmc_objs(struct irdma_pci_f *rf, bool privileged, info.entry_type = rf->sd_type; for (i = 0; i < IW_HMC_OBJ_TYPE_NUM; i++) { + if (iw_hmc_obj_types[i] == IRDMA_HMC_IW_PBLE) + continue; if (dev->hmc_info->hmc_obj[iw_hmc_obj_types[i]].cnt) { info.rsrc_type = iw_hmc_obj_types[i]; info.count = dev->hmc_info->hmc_obj[info.rsrc_type].cnt; From patchwork Wed Mar 15 14:52:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 13176000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0016BC6FD1D for ; Wed, 15 Mar 2023 14:56:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233330AbjCOO4D (ORCPT ); Wed, 15 Mar 2023 10:56:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233270AbjCOOzs (ORCPT ); Wed, 15 Mar 2023 10:55:48 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83C4D93E04 for ; Wed, 15 Mar 2023 07:54:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678892089; x=1710428089; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tnADPxFQMHiIvqBjqalGRZjmxxBP6LwWe0CVM1b6Bd0=; b=EHs1jbf56NnYvc7Aij3uvlpkpU01JOISkrGPI95iSBDu8/wSKty4IL8V NpCBJNZnBBZ8rSpU1dxK+XSk93nO3yWvGc0GKJOhi6CRPQyaaZiZu00r+ xSU9cQo33vdk31BB+I7qWEfZcLxCJ8HDU/umy3btDJm9NkXHN4C2zsQo+ Z08d8r6BHoSR3O9F9Xn21F382vG/0vjK/DWwV7DljYSKHUfdOjzcM997d qTwFAeJ3dXDA7XAsuqVMzEYxXU1/uRvkY9DHYCJfuOOY7mGtYBAqrwHSb 4M5jcUnhrVy4eDogK4QDjcBcszLY6JG60hY2RKYx0tMk8UWMIIfjCDlPh Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10649"; a="321561537" X-IronPort-AV: E=Sophos;i="5.98,262,1673942400"; d="scan'208";a="321561537" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2023 07:52:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10649"; a="748457451" X-IronPort-AV: E=Sophos;i="5.98,262,1673942400"; d="scan'208";a="748457451" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.255.35.84]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2023 07:52:47 -0700 From: Shiraz Saleem To: jgg@nvidia.com, leon@kernel.org Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-rc 3/4] RDMA/irdma: Increase iWARP CM default rexmit count Date: Wed, 15 Mar 2023 09:52:30 -0500 Message-Id: <20230315145231.931-4-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230315145231.931-1-shiraz.saleem@intel.com> References: <20230315145231.931-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail When running perftest with large number of connections in iWARP mode, the passive side could be slow to respond. Increase the rexmit counter default to allow scaling connections. Fixes: 146b9756f14c ("RDMA/irdma: Add connection manager") Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/cm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/irdma/cm.h b/drivers/infiniband/hw/irdma/cm.h index 19c2849..7feadb3 100644 --- a/drivers/infiniband/hw/irdma/cm.h +++ b/drivers/infiniband/hw/irdma/cm.h @@ -41,7 +41,7 @@ #define TCP_OPTIONS_PADDING 3 #define IRDMA_DEFAULT_RETRYS 64 -#define IRDMA_DEFAULT_RETRANS 8 +#define IRDMA_DEFAULT_RETRANS 32 #define IRDMA_DEFAULT_TTL 0x40 #define IRDMA_DEFAULT_RTT_VAR 6 #define IRDMA_DEFAULT_SS_THRESH 0x3fffffff From patchwork Wed Mar 15 14:52:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 13176001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4F08C6FD1D for ; Wed, 15 Mar 2023 14:56:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233318AbjCOO4J (ORCPT ); Wed, 15 Mar 2023 10:56:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233325AbjCOOzy (ORCPT ); Wed, 15 Mar 2023 10:55:54 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AEE9823A79 for ; Wed, 15 Mar 2023 07:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678892099; x=1710428099; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=80rnPN658u/2vQrbjAP78NXf33A3bqVGKcEBJ7VETvQ=; b=T4uicMSN8Ayx7hOrVge5UtGN5U/sfkBwE4ZL+4FRW/R3hsP+umStUc9s x+1Bl6H+e2zoWlECRBuC0FWxdL4huTzwxSoFXEFQxM0X8ndCb2wizVbhU +JO7FGjQn9c/+WkW6wgv+O0lRu34gRUabayxj/tCI81U5Q3EvZDesxGTH xaNR/x3LNfOE2OGgRk3ep1p4kaYJu0E9APM55PMCH+GFDc7gZqNduVicQ gEWhH+0a6ShcTty9SFeRn/4OK6gb3v8vuQJhtDzJQ3mIAZ+knq+oqCxot mKOtLGgpAM9UfQcaElXLIFpdZZj/pxFroJ8wqQttpfbQUJRiQvvwkkiWZ g==; X-IronPort-AV: E=McAfee;i="6500,9779,10649"; a="321561550" X-IronPort-AV: E=Sophos;i="5.98,262,1673942400"; d="scan'208";a="321561550" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2023 07:52:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10649"; a="748457454" X-IronPort-AV: E=Sophos;i="5.98,262,1673942400"; d="scan'208";a="748457454" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.255.35.84]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Mar 2023 07:52:47 -0700 From: Shiraz Saleem To: jgg@nvidia.com, leon@kernel.org Cc: linux-rdma@vger.kernel.org, Tatyana Nikolova , Shiraz Saleem Subject: [PATCH for-rc 4/4] RDMA/irdma: Add ipv4 check to irdma_find_listener() Date: Wed, 15 Mar 2023 09:52:31 -0500 Message-Id: <20230315145231.931-5-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230315145231.931-1-shiraz.saleem@intel.com> References: <20230315145231.931-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Tatyana Nikolova Add ipv4 check to irdma_find_listener(). Otherwise the function incorrectly finds and returns a listener with a different addr family for the zero IP addr, if a listener with a zero IP addr and the same port as the one searched for has already been created. Fixes: 146b9756f14c ("RDMA/irdma: Add connection manager") Signed-off-by: Tatyana Nikolova Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/cm.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c index 195aa9e..8817864 100644 --- a/drivers/infiniband/hw/irdma/cm.c +++ b/drivers/infiniband/hw/irdma/cm.c @@ -1458,13 +1458,15 @@ static int irdma_send_fin(struct irdma_cm_node *cm_node) * irdma_find_listener - find a cm node listening on this addr-port pair * @cm_core: cm's core * @dst_addr: listener ip addr + * @ipv4: flag indicating IPv4 when true * @dst_port: listener tcp port num * @vlan_id: virtual LAN ID * @listener_state: state to match with listen node's */ static struct irdma_cm_listener * -irdma_find_listener(struct irdma_cm_core *cm_core, u32 *dst_addr, u16 dst_port, - u16 vlan_id, enum irdma_cm_listener_state listener_state) +irdma_find_listener(struct irdma_cm_core *cm_core, u32 *dst_addr, bool ipv4, + u16 dst_port, u16 vlan_id, + enum irdma_cm_listener_state listener_state) { struct irdma_cm_listener *listen_node; static const u32 ip_zero[4] = { 0, 0, 0, 0 }; @@ -1477,7 +1479,7 @@ static int irdma_send_fin(struct irdma_cm_node *cm_node) list_for_each_entry (listen_node, &cm_core->listen_list, list) { memcpy(listen_addr, listen_node->loc_addr, sizeof(listen_addr)); listen_port = listen_node->loc_port; - if (listen_port != dst_port || + if (listen_node->ipv4 != ipv4 || listen_port != dst_port || !(listener_state & listen_node->listener_state)) continue; /* compare node pair, return node handle if a match */ @@ -2902,9 +2904,10 @@ static void irdma_process_pkt(struct irdma_cm_node *cm_node, unsigned long flags; /* cannot have multiple matching listeners */ - listener = irdma_find_listener(cm_core, cm_info->loc_addr, - cm_info->loc_port, cm_info->vlan_id, - IRDMA_CM_LISTENER_EITHER_STATE); + listener = + irdma_find_listener(cm_core, cm_info->loc_addr, cm_info->ipv4, + cm_info->loc_port, cm_info->vlan_id, + IRDMA_CM_LISTENER_EITHER_STATE); if (listener && listener->listener_state == IRDMA_CM_LISTENER_ACTIVE_STATE) { refcount_dec(&listener->refcnt); @@ -3153,6 +3156,7 @@ void irdma_receive_ilq(struct irdma_sc_vsi *vsi, struct irdma_puda_buf *rbuf) listener = irdma_find_listener(cm_core, cm_info.loc_addr, + cm_info.ipv4, cm_info.loc_port, cm_info.vlan_id, IRDMA_CM_LISTENER_ACTIVE_STATE);