From patchwork Wed Apr 3 10:35:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 13615767 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F4B1433AD for ; Wed, 3 Apr 2024 10:36:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712140574; cv=none; b=dLs2YaY982cENb9lFQwL5tOnR4sAd8hfDzPdrTI4WdhNa7HPd7xL/+YSSGs2zBBqk2+uAPSNdHgbzn4UWoKe+qFdP6mwlUrQ7hpQZ58EwEtbZXmeyipzPz+vmVnuAggeZOk0vTyNUqxApwHxybVYmHV6R/3GfQGRGfrLRqtMX+U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712140574; c=relaxed/simple; bh=oUJRha4fMxW+pvW8Uz39rjHv6c2cWJRFI++Ii6JasxE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FRLHjLz9+Fy3Aij3NRtJ1Odox84ZM/e3Q5ZFQBrO7uUk4SrPKMH4macwfXgF7PiPP7IRxf7h52ioMnVeBiDCjjH0Qp30HUowh+C3/WkXrJR/8onWQ3iJIe3np50Pw3Y7FVe1RkqWE14Zrdw3b/tiwL4APz8HVMdGdGaY+DE6VVU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ef6wwbix; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ef6wwbix" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 446C3C433C7; Wed, 3 Apr 2024 10:36:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712140573; bh=oUJRha4fMxW+pvW8Uz39rjHv6c2cWJRFI++Ii6JasxE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ef6wwbix2X1/kHRDFJfOsQ9+FSzNxCgjAeP5cq+6dO+aUPl4haLCbh1FCXWgXp+ol yTzAKn/niCJLEHRrFuYkp5hnly/NbWIvD6rgSnwNxZ3NheuVOw8HPleYj+sNYbzLrG XritGJ2tUfowHOWFp08+qydtvWbuTT5yLqbm0ANzGLKrifXs9qKg+7Tqsuv/VVUvAI mt7G4ZJ4xTfGTR1y4UqKtPE8Mwg7ka9QCqBqt2GJZalFMgtRxM5a7pO3Gz+i0Kintg Bgx9/vUGZ/nbzV8ZTTei7BjaGheclkAPy2if5s02dvNS1VLAyK7htnQe9RWoL+Mke0 Onv/WJI8EdbfA== From: Leon Romanovsky To: Jason Gunthorpe Cc: Or Har-Toov , linux-rdma@vger.kernel.org, Michael Guralnik Subject: [PATCH rdma-next v2 1/3] RDMA/mlx5: Uncacheable mkey has neither rb_key or cache_ent Date: Wed, 3 Apr 2024 13:35:59 +0300 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Or Har-Toov As some mkeys can't be modified with UMR due to some UMR limitations, like the size of translation that can be updated, not all user mkeys can be cached. Fixes: dd1b913fb0d0 ("RDMA/mlx5: Cache all user cacheable mkeys on dereg MR flow") Signed-off-by: Or Har-Toov Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index a8de35c07c9e..e74f04865062 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -643,7 +643,7 @@ struct mlx5_ib_mkey { unsigned int ndescs; struct wait_queue_head wait; refcount_t usecount; - /* User Mkey must hold either a rb_key or a cache_ent. */ + /* Cacheable user Mkey must hold either a rb_key or a cache_ent. */ struct mlx5r_cache_rb_key rb_key; struct mlx5_cache_ent *cache_ent; }; From patchwork Wed Apr 3 10:36:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 13615768 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13483137C45 for ; Wed, 3 Apr 2024 10:36:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712140578; cv=none; b=GmS3GF3bkEIet20pxEsOGsIXL7TaOG9z24WsSULmWQ8y7mU+nILWrcbK3+cB+8opr3Sh2v02vgIFCGYU2LPS7ciTeUT2qlprXkMUNjrQMH3o2ZUUEewtJA0L/ELgdiulxDmGIf0WPq7ysTCqyx5vFybIODuOqCu688Rxs9ql2cw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712140578; c=relaxed/simple; bh=EB9jUR+T28hPiUhH8QaOjW9Lc8PgZoAWRc+UUme0Aa8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ff27ehnORb6hOHUXCdW4gUpGBP5hi2Rtlr1E12TWFEgg4MJfpKds+zx4RLPq7YhG0YRBLApn1BEm0fQaAU2cS8U7rKB6Fd7mWf4QV5dbxEhR94eB6ROfxVpAxesmg9nL4F6tv2S703OHiavnm4YESYGwHONtZ6C6MEufYOC2wj0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Cfdxy1+u; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Cfdxy1+u" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F2325C433F1; Wed, 3 Apr 2024 10:36:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712140577; bh=EB9jUR+T28hPiUhH8QaOjW9Lc8PgZoAWRc+UUme0Aa8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cfdxy1+ukAf639SrQO0D2CQDzSw4zQd055f+8TaXfGPrZQrWJaO5x4XgZkdfYM43C LSoeoeKull9cuFVHLRa77wx2aKpPGEECAhFUegglXZvnE3kZTVC4FMiurzlZWnrevd vW+C9BoL4nJJ0zGlpI+8VHAO3Tx9YFM/cLEprvCK/yoWqUtj4nGYI8F1jASLmU2jSy MKakCfCih2Vvql9eotO04wFmxT3iqU+mh/DVJdNw0Rvxd2/63NvEz1tFC/YFnhGnvE rk4hcO7NA9Q9OLLUHHQ0qy7sU22U4jBXvSkz5aqT/ZV1rcVsLj4HwDl7jv756P7Ais mP7GqGW2mqyEQ== From: Leon Romanovsky To: Jason Gunthorpe Cc: Or Har-Toov , linux-rdma@vger.kernel.org, Michael Guralnik Subject: [PATCH rdma-next v2 2/3] RDMA/mlx5: Change check for cacheable mkeys Date: Wed, 3 Apr 2024 13:36:00 +0300 Message-ID: <2690bc5c6896bcb937f89af16a1ff0343a7ab3d0.1712140377.git.leon@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Or Har-Toov umem can be NULL for user application mkeys in some cases. Therefore umem can't be used for checking if the mkey is cacheable and it is changed for checking a flag that indicates it. Also make sure that all mkeys which are not returned to the cache will be destroyed. Fixes: dd1b913fb0d0 ("RDMA/mlx5: Cache all user cacheable mkeys on dereg MR flow") Signed-off-by: Or Har-Toov Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 + drivers/infiniband/hw/mlx5/mr.c | 32 +++++++++++++++++++--------- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index e74f04865062..f255a12e26a0 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -646,6 +646,7 @@ struct mlx5_ib_mkey { /* Cacheable user Mkey must hold either a rb_key or a cache_ent. */ struct mlx5r_cache_rb_key rb_key; struct mlx5_cache_ent *cache_ent; + u8 cacheable : 1; }; #define MLX5_IB_MTT_PRESENT (MLX5_IB_MTT_READ | MLX5_IB_MTT_WRITE) diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index a8ee2ca1f4a1..7f7b1f59b5f0 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -1158,6 +1158,7 @@ static struct mlx5_ib_mr *alloc_cacheable_mr(struct ib_pd *pd, if (IS_ERR(mr)) return mr; mr->mmkey.rb_key = rb_key; + mr->mmkey.cacheable = true; return mr; } @@ -1168,6 +1169,7 @@ static struct mlx5_ib_mr *alloc_cacheable_mr(struct ib_pd *pd, mr->ibmr.pd = pd; mr->umem = umem; mr->page_shift = order_base_2(page_size); + mr->mmkey.cacheable = true; set_mr_fields(dev, mr, umem->length, access_flags, iova); return mr; @@ -1835,6 +1837,23 @@ static int cache_ent_find_and_store(struct mlx5_ib_dev *dev, return ret; } +static int mlx5_revoke_mr(struct mlx5_ib_mr *mr) +{ + struct mlx5_ib_dev *dev = to_mdev(mr->ibmr.device); + struct mlx5_cache_ent *ent = mr->mmkey.cache_ent; + + if (mr->mmkey.cacheable && !mlx5r_umr_revoke_mr(mr) && !cache_ent_find_and_store(dev, mr)) + return 0; + + if (ent) { + spin_lock_irq(&ent->mkeys_queue.lock); + ent->in_use--; + mr->mmkey.cache_ent = NULL; + spin_unlock_irq(&ent->mkeys_queue.lock); + } + return destroy_mkey(dev, mr); +} + int mlx5_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata) { struct mlx5_ib_mr *mr = to_mmr(ibmr); @@ -1880,16 +1899,9 @@ int mlx5_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata) } /* Stop DMA */ - if (mr->umem && mlx5r_umr_can_load_pas(dev, mr->umem->length)) - if (mlx5r_umr_revoke_mr(mr) || - cache_ent_find_and_store(dev, mr)) - mr->mmkey.cache_ent = NULL; - - if (!mr->mmkey.cache_ent) { - rc = destroy_mkey(to_mdev(mr->ibmr.device), mr); - if (rc) - return rc; - } + rc = mlx5_revoke_mr(mr); + if (rc) + return rc; if (mr->umem) { bool is_odp = is_odp_mr(mr); From patchwork Wed Apr 3 10:36:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 13615769 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1CA3134CCA for ; Wed, 3 Apr 2024 10:36:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712140582; cv=none; b=DeWW+Y1F/aV7RD5RppeEFLIZPdWZDUxavgi+mSVWWAkTGCs14j+BmRs5uULo7secV9/+mL5RKCNYTySMjCosgwjou7uRPSibHor/7GBI51FlKDMZCuB2C/eP1BEt49VCDK4pnsn3XsdH410bH3PWBL7GeCFa1CWcZxvtdzqINMU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712140582; c=relaxed/simple; bh=hfGZW4mkZ5Kxage6JgCUtZjIlIoeRXr7pULrTdQgC2E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Aj65fQyGVuDnCdXiQy8a/MuPOklzZ3lA3+nm7X2PjEB1N3Sxc7YrJqT+3di9cxkHZdpDJ+phaW4CcaiJnNr0wFTrkp30PAjNMjPjonOU9S77KAmIzeOzbPwcZw4u0jl1Utmq+5KSeBtYfTPS+oAdK5+AaaBusuCdPdxZCsdyx8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eDvYS7kH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eDvYS7kH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02618C433C7; Wed, 3 Apr 2024 10:36:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712140581; bh=hfGZW4mkZ5Kxage6JgCUtZjIlIoeRXr7pULrTdQgC2E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eDvYS7kHPzP+2cWNK1XdyDfuxhTKb5dUrXSPUt86PnWAP72Db05UQqa0RbHpAwern hkOiJRPF88FU3zXWCWKqoa4oZ/pQWpCuIK1Lj8VU0+6LDawKffQdeia5EP8pZqBxdw jcVidrCeur+tr4CXHY2/deRMd9/OHwo/mIxVqf3ffTnqW3nMEAq9PZhI8P6btiJyp6 V4DX9uPTsdOkhOXS2/8EQoOpuc9A4PNDVC4nzVRZL05h9Q7sbnjr5Bv7BhNaOFnTqS JtbY66zHZCgRf31x93xi4XFhGARAJK+xaJlXhjfOZVrlqoiHo4+E8rdCJsFiPTRTml 0wD23vr+DTWsA== From: Leon Romanovsky To: Jason Gunthorpe Cc: Or Har-Toov , linux-rdma@vger.kernel.org, Michael Guralnik Subject: [PATCH rdma-next v2 3/3] RDMA/mlx5: Adding remote atomic access flag to updatable flags Date: Wed, 3 Apr 2024 13:36:01 +0300 Message-ID: <24dac73e2fa48cb806f33a932d97f3e402a5ea2c.1712140377.git.leon@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Or Har-Toov Currently IB_ACCESS_REMOTE_ATOMIC is blocked from being updated via UMR although in some cases it should be possible. These cases are checked in mlx5r_umr_can_reconfig function. Fixes: ef3642c4f54d ("RDMA/mlx5: Fix error unwinds for rereg_mr") Signed-off-by: Or Har-Toov Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/mr.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 7f7b1f59b5f0..ecc111ed5d86 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -1572,7 +1572,8 @@ static bool can_use_umr_rereg_access(struct mlx5_ib_dev *dev, unsigned int diffs = current_access_flags ^ target_access_flags; if (diffs & ~(IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | - IB_ACCESS_REMOTE_READ | IB_ACCESS_RELAXED_ORDERING)) + IB_ACCESS_REMOTE_READ | IB_ACCESS_RELAXED_ORDERING | + IB_ACCESS_REMOTE_ATOMIC)) return false; return mlx5r_umr_can_reconfig(dev, current_access_flags, target_access_flags);