From patchwork Fri Aug 9 13:15:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758793 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8005A194C93 for ; Fri, 9 Aug 2024 13:15:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209359; cv=none; b=YvL9NAVBpXJ6hSlSMAVwvFB/YfOz3Fir2Z3VL+Ia9PihykMIfOZe36vkINWbMt0OpGB64b7kT+o58VbWSyN2viBOylxHKkDybvzb+jLRU8TcnoiQvlNMA9fjHYOpFVPemggQAWW+9ngDgdJ1UBEzCO245t55F2EX08sLnv9hTD0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209359; c=relaxed/simple; bh=YWucwtpfgTcqBS05BBr3Qv4JpdW+smVL766x3V5mssE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gww7Dmnnz2gaM1XmQftkbEBkZyN56toIK8I5dQthNv5x0SslyvaPLfM/ZcBkyLEIy7q5Ar26ivLUCtmeKTbmfR1VxPCVdOp0YLL4eK26/latbKlF7ujn6aQUi6jV/v/A3YOMudN03SHmfO3vjgqEUEWG+pTcufn/mENAjFFHts8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=DtrQ2rb/; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="DtrQ2rb/" Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-428e0d184b4so13720325e9.2 for ; Fri, 09 Aug 2024 06:15:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209355; x=1723814155; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PivggR7AYjcEmDoWWdQztk4HyavK50yywVBFX8fK/ns=; b=DtrQ2rb/4KIrU8rilDRbnq2PmQKHhdnMoI11kz6r41vh4NQnD44So05jp8inJZD1l6 onqHbQxoVhDDQPAj24d/rgVzWTlxYCC0n/NgpsmnapbIWPiPZ8pzxDs5CuvuR6EtSiM/ J7lAFYOQWcyMpA/av9KXY6o1m0DXpa8Pfdj4CZSlo7WFs5AXIpTnqd1kxRwv3vwEFoLD 1EavM7RwmnaPKt4tf2Zbpc3sDcGDya0p0YUo2c1GicttgakDnzfMYfs1TcD5vAKHiJmk Ml6sWP/1hTS3u1M1Q1SDy/S1zg2SBzBDnaE9bIhrrRQMUkij1zBgkkD4rDktuQM5a+fW HdLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209355; x=1723814155; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PivggR7AYjcEmDoWWdQztk4HyavK50yywVBFX8fK/ns=; b=wnilzLjiev/McgcqQGPD89gFRAatE84z7A0PZxk7m6QTQZ4H6iwDM7PTmz3FC/BidR mbclhAfYC/1sK7zkKep/WVsgMb96rVhFl5xT+12JQE4O6/zmssu2Brvivr0Zy8CTMpg+ sA/qCJlPlu8cPvQEbVjqxx6h5IbNbW26VyPYe7opuGfN1xCZ/5QwmTDy/h5ia7sJLUcT FPPit4HQiyG4e5RjmdyLb7BV3hb7nphX77wZaNSQMs6d+Mnzx1j7IzAUos/s3l/jR4ra rdOU6K3rgCnYvQhlPJPDl1s7G9ADGDMsYpSL5/7GsTURFQTa9fCAvzaHuUvdFYlS6npI KyzA== X-Gm-Message-State: AOJu0YyGiFR6OrlcOhvQXWZzPWVgi5Tn8GJv14HvhgRFrt7fYX3G15DS HYs4e9HZlYY48bqY8y3Nmtz82XVE5DlnLiJLHzBXSUDKp3AKQfmqTWTf3/GNQWf89dZAuQvaoCW tL64= X-Google-Smtp-Source: AGHT+IEAsM7ho9eHZ+5FfgHNos7ZimcVajSJAQPuMFdYKoCK3Nn9HwwKB23vOwmDvV83TCciUYJlhw== X-Received: by 2002:a05:600c:3146:b0:426:5dd0:a1ea with SMTP id 5b1f17b1804b1-429c3a52138mr12274335e9.28.1723209354581; Fri, 09 Aug 2024 06:15:54 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:54 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 01/13] RDMA/rtrs-srv: Make rtrs_srv_open fail if its a second call Date: Fri, 9 Aug 2024 15:15:26 +0200 Message-Id: <20240809131538.944907-2-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Do not allow opening RTRS server if it is already in use and print proper error message. Signed-off-by: Md Haris Iqbal Signed-off-by: Jack Wang Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 27 +++++++++++++++++++++++--- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 1 + 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 1d33efb8fb03..fb67b58a7f62 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -2174,9 +2174,18 @@ struct rtrs_srv_ctx *rtrs_srv_open(struct rtrs_srv_ops *ops, u16 port) struct rtrs_srv_ctx *ctx; int err; + mutex_lock(&ib_ctx.rtrs_srv_ib_mutex); + if (ib_ctx.srv_ctx) { + pr_err("%s: Already in use.\n", __func__); + ctx = ERR_PTR(-EEXIST); + goto out; + } + ctx = alloc_srv_ctx(ops); - if (!ctx) - return ERR_PTR(-ENOMEM); + if (!ctx) { + ctx = ERR_PTR(-ENOMEM); + goto out; + } mutex_init(&ib_ctx.ib_dev_mutex); ib_ctx.srv_ctx = ctx; @@ -2185,9 +2194,11 @@ struct rtrs_srv_ctx *rtrs_srv_open(struct rtrs_srv_ops *ops, u16 port) err = ib_register_client(&rtrs_srv_client); if (err) { free_srv_ctx(ctx); - return ERR_PTR(err); + ctx = ERR_PTR(err); } +out: + mutex_unlock(&ib_ctx.rtrs_srv_ib_mutex); return ctx; } EXPORT_SYMBOL(rtrs_srv_open); @@ -2221,10 +2232,16 @@ static void close_ctx(struct rtrs_srv_ctx *ctx) */ void rtrs_srv_close(struct rtrs_srv_ctx *ctx) { + mutex_lock(&ib_ctx.rtrs_srv_ib_mutex); + WARN_ON(ib_ctx.srv_ctx != ctx); + ib_unregister_client(&rtrs_srv_client); mutex_destroy(&ib_ctx.ib_dev_mutex); close_ctx(ctx); free_srv_ctx(ctx); + + ib_ctx.srv_ctx = NULL; + mutex_unlock(&ib_ctx.rtrs_srv_ib_mutex); } EXPORT_SYMBOL(rtrs_srv_close); @@ -2282,6 +2299,9 @@ static int __init rtrs_server_init(void) goto out_dev_class; } + mutex_init(&ib_ctx.rtrs_srv_ib_mutex); + ib_ctx.srv_ctx = NULL; + return 0; out_dev_class: @@ -2292,6 +2312,7 @@ static int __init rtrs_server_init(void) static void __exit rtrs_server_exit(void) { + mutex_destroy(&ib_ctx.rtrs_srv_ib_mutex); destroy_workqueue(rtrs_wq); class_unregister(&rtrs_dev_class); rtrs_rdma_dev_pd_deinit(&dev_pd); diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.h b/drivers/infiniband/ulp/rtrs/rtrs-srv.h index 5e325b82ff33..4924dde0a708 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.h @@ -127,6 +127,7 @@ struct rtrs_srv_ib_ctx { u16 port; struct mutex ib_dev_mutex; int ib_dev_count; + struct mutex rtrs_srv_ib_mutex; }; extern const struct class rtrs_dev_class; From patchwork Fri Aug 9 13:15:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758794 Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89DCC195FEC for ; Fri, 9 Aug 2024 13:15:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209359; cv=none; b=UkaLlJZdXgDWbjbgSUQdfumTBEZD+gM3n8/2rrEYuSOUjeFbFy/b7X17OU1Nr2pnGiN32o3LZkCsQNMW4YAwiZiPHywAKit0ZCmrt2rbw6b+JvyVNDxkiCptsXfM/MlU3zgNLwWMQx3RtkEtK5OxjFUnT7HUtZG8KPRifYXit7E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209359; c=relaxed/simple; bh=UKZdlURp9y8m7Wfiru0pj4BLF+QKyPW96UV/lMHLhqo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QVAW+wIa+38Hl/MwLLE9gBms2vI9WSkJTlCvtMumkbUmhzBkTCpmg+Lh9DjaDtsCfjcPK74I2yYneMKOeZx6KAvOjt/cXeP6PJl0gOVrAo+vV5LSEsk6aPrHP6JyzzbZsIRaI3mXFt5MVAGquqPuog8eTZVzeHy3QSgH/Mtmb0s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=aLfLUCaM; arc=none smtp.client-ip=209.85.208.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="aLfLUCaM" Received: by mail-lj1-f194.google.com with SMTP id 38308e7fff4ca-2f0dfdc9e16so24145041fa.2 for ; Fri, 09 Aug 2024 06:15:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209355; x=1723814155; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t07LWNbbJ1CWMDPd+GXylKpgK0rHVigw8wrUTpYaBe4=; b=aLfLUCaM3d09BJDZqjGYYzgHusCu67CVjn8RDFcRAQen7FxXxFzHz05T3ojvEgtyKc e2uweTniJdzWMz6yuOHIH73+Mnw9vjUeHtg0OZZB9XGFfmrRG74vMW7ClRoTheGVLUhP CtY+2PSFXFYDv2JcdAXP/Zyqi+X0hfSBLHHj9NrDTCtHNaKjuA558gb+/p0yKudj3Nn4 ypp4Y37y6H7M+0Ty2pkpjwVD72jqMQyyyz+QWhCqrb0H45nO9X0Ktg8NxvcTY8XoLukX ovRJvMP/Y65izQsBaeXRjUEfFA6edF/JE5PovhaeFOVmeG/SLFEpTU05gEKMh6dUxCFX +3aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209355; x=1723814155; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t07LWNbbJ1CWMDPd+GXylKpgK0rHVigw8wrUTpYaBe4=; b=QCWJjbjEJ82fFhQrUwf0hlpIWlimHB+O4B798dvHHO9feszHPpx4Q8IRuG8I8g9j29 tr+VpScjb9pjB84xEB3kNEByOUm2dDd5QWOvz3MmOH5G1LSxRiMcdV17trAVn+em5b1w bqKB8zTMIUvIpeyevEdpld26RcV6kvOjWozp2h/Y6AEgVZXzEEnZk/GplvVyM+McEifl 8It0X4AgqLDkfA+xoB33NQHFIjJ/XDGFczQaukOxk5oaSjJn96jM0rxzhgNekM2sBl0I 1DiM6SkllvL+w++5F1wClv41X77sNXbgNvKlEthDqRpseK6S5takqFiOINTDA64WHEL4 H1kQ== X-Gm-Message-State: AOJu0YyGY+iykaI/kytlhCtYA4Ailpoi+O/66JmhcY+Q2NFfwjQ/Q2WF 6+1+eamqyz+qlHgfA72bR/qcwP6K3KfygYmHfG1+wPFhQeD9HA/ycujIvGf5N4xsU8ZAQVq5jtM S5I82WA== X-Google-Smtp-Source: AGHT+IGTxh2gGuDmuiFwDZ65xgTy1VL3jAe4KYJ+qkdKO/F3mPdd6mY+GN8BC3Sl2JB9+hUFHbCVzw== X-Received: by 2002:a2e:9f4f:0:b0:2ef:392e:e45 with SMTP id 38308e7fff4ca-2f1a6d32166mr11382171fa.33.1723209355294; Fri, 09 Aug 2024 06:15:55 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:54 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 02/13] RDMA/rtrs-srv: Fix use-after-free during session establishment Date: Fri, 9 Aug 2024 15:15:27 +0200 Message-Id: <20240809131538.944907-3-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jack Wang In case of error happening during session stablishment, close_work is running. A new RDMA CM event may arrive since we don't destroy cm_id before destroying qp. To fix this, we first destroy cm_id after drain_qp, so no new RDMA CM event will arrive afterwards. Fixes: 9cb837480424 ("RDMA/rtrs: server: main functionality") Signed-off-by: Jack Wang Signed-off-by: Grzegorz Prajsner Signed-off-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 2 +- drivers/infiniband/ulp/rtrs/rtrs.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index fb67b58a7f62..90ea25ad6720 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1540,6 +1540,7 @@ static void rtrs_srv_close_work(struct work_struct *work) con = to_srv_con(srv_path->s.con[i]); rdma_disconnect(con->c.cm_id); ib_drain_qp(con->c.qp); + rdma_destroy_id(con->c.cm_id); } /* @@ -1564,7 +1565,6 @@ static void rtrs_srv_close_work(struct work_struct *work) continue; con = to_srv_con(srv_path->s.con[i]); rtrs_cq_qp_destroy(&con->c); - rdma_destroy_id(con->c.cm_id); kfree(con); } rtrs_ib_dev_put(srv_path->s.dev); diff --git a/drivers/infiniband/ulp/rtrs/rtrs.c b/drivers/infiniband/ulp/rtrs/rtrs.c index 4e17d546d4cc..44167fd1c958 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs.c +++ b/drivers/infiniband/ulp/rtrs/rtrs.c @@ -318,7 +318,7 @@ EXPORT_SYMBOL_GPL(rtrs_cq_qp_create); void rtrs_cq_qp_destroy(struct rtrs_con *con) { if (con->qp) { - rdma_destroy_qp(con->cm_id); + ib_destroy_qp(con->qp); con->qp = NULL; } destroy_cq(con); From patchwork Fri Aug 9 13:15:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758795 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B83B7194AF4 for ; Fri, 9 Aug 2024 13:15:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209359; cv=none; b=LBYDzOHl5BptrvDH4jcURPz3ZiTTJPWtVOyGbJDiQ0zhDAeKzyVhDGFTXCA9hOc0n+K+Ve9rWCPVWSN6KB8ZvyGE4RWpBm/JQxUM3SW1TXGvlvDwW3SZfw8IuFlR4rx4IUC70K2HFmYstV/0t0Q/nxCyKnvU3KD8XA3pdKScYuc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209359; c=relaxed/simple; bh=UX6CYtwSpi23VKoZoiBb1839/I25OP7qWo97jVNyyvU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ty02PJvDZd7sXmpJ0u5cOM5bnM+5pbajEBfcn0Jv9yjKn8OdMmMNW5+v42KefNzSg9/usYigWf/i7HSuXnG3fnvCRpZdM2fdj1HyPLni1woQe3M1y8Z5o/437DkcseHGOY22GPj6I4pTuhsripvfxC/gZt7wc49i2L2Dw7IatS8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=RciXoIx1; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="RciXoIx1" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4280bca3960so15296385e9.3 for ; Fri, 09 Aug 2024 06:15:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209356; x=1723814156; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9VftxWQ8DmE0g8jGp2R6vgB3PyXWNM5bcwEfKFwmO7k=; b=RciXoIx12qr6sz3ANzjqHs3Y5CmAsBSg5I1kkX2CyORd/LyfOJw8l72E7p3FnjrQH9 5V7X6X3bVAp3fzjktIHLoflD2qssEMoBlgVvUQXZxvp4yzb4Z0MGwrgmsRlyhSNfRI/6 utS3LXopnv99mwpa93EqAjpeVzHj9UnP18wBbHrpyS6+mN0exBrRkM/WLOqkfRN8lois n4uDa5hILGHV+YDREgcPWxkmg5ZS1mfLhkvmnK9t/NBEdi/MmpHkNbAA4ETjJYgVtWXd xTHh+g0WqEkoZ3fhIjkQEx5V4wbwwBEvwggoooGm4udcBim1RcG/sxqQnW4bzVfdksTX gzYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209356; x=1723814156; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9VftxWQ8DmE0g8jGp2R6vgB3PyXWNM5bcwEfKFwmO7k=; b=dARFpVef+iK4K7m2Xv5KpskrM/ZybxrN0KgOiyhm/mKSxBLzq2ZRDkTPg8wtjPk/GL U+x2LSnow2kwqRT+jQRSOTlfVBaxUu+h3vLVBLjW5+HRPqmLisEzozpEtWrFwj0dHgR9 r2Fvn41f7vJtAZUCrmwCO3eDbGivEdWHFG42fnB+ploGir+P3znhp0Bf3FH1MtrKiBVU /kAYv6VzcCGK5IFcZbmmTc3KVf30080ucjbNzDfCb6OmECHH6PlyzonE5m/TFhXfRHPx JHu/o/CHRckJaPbsk9zbL64oyNvkUfXHTCZ1Kp9utJx+cJggCydorbiiMpX9dIxcEsWJ FrmA== X-Gm-Message-State: AOJu0YxojgI/Es732aXz/YHTTFYxIGjpB0ml0iByMwTlKwKzA8DK7biZ nZXxhB/67hABwfDkWkYpCGJ+ySQ99HAlNBwgMrd6VzEB8B2eed3gEXl5ptQ05q5sS5WYP2W+I8R LNhE= X-Google-Smtp-Source: AGHT+IExrlVjE+lm25xBam+bIIzEuM10jao7kd5SS9wyUlDp9JDkX6MQAnSSrZYLay7IkU7hbKUa8w== X-Received: by 2002:a05:600c:4e8a:b0:426:6667:5c42 with SMTP id 5b1f17b1804b1-429c3a17ff5mr11456125e9.4.1723209355950; Fri, 09 Aug 2024 06:15:55 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:55 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 03/13] RDMA/rtrs: For HB error add additional clt/srv specific logging Date: Fri, 9 Aug 2024 15:15:28 +0200 Message-Id: <20240809131538.944907-4-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In case of HB error, we need to know the specific path on which it happened, for better debugging. Since the clt/srv path structures are not available in rtrs.c, it needs to be done in the individual HB error handler. This commit add those loging. A sample kernel log output after this commit: rtrs_core L357: : HB missed max reached. rtrs_server L717: : HB err handler for path=ip:x.x.x.x@ip:x.x.x.x . . rtrs_core L357: : HB missed max reached. rtrs_client L1519: : HB err handler for path=ip:x.x.x.x@ip:x.x.x.x Signed-off-by: Md Haris Iqbal Reviewed-by: Jack Wang Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 2 ++ drivers/infiniband/ulp/rtrs/rtrs-srv.c | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 88106cf5ce55..66ac4dba990f 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -1494,7 +1494,9 @@ static bool rtrs_clt_change_state_get_old(struct rtrs_clt_path *clt_path, static void rtrs_clt_hb_err_handler(struct rtrs_con *c) { struct rtrs_clt_con *con = container_of(c, typeof(*con), c); + struct rtrs_clt_path *clt_path = to_clt_path(con->c.path); + rtrs_err(con->c.path, "HB err handler for path=%s\n", kobject_name(&clt_path->kobj)); rtrs_rdma_error_recovery(con); } diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 90ea25ad6720..e6885c88fa41 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -672,6 +672,10 @@ static int map_cont_bufs(struct rtrs_srv_path *srv_path) static void rtrs_srv_hb_err_handler(struct rtrs_con *c) { + struct rtrs_srv_con *con = container_of(c, typeof(*con), c); + struct rtrs_srv_path *srv_path = to_srv_path(con->c.path); + + rtrs_err(con->c.path, "HB err handler for path=%s\n", kobject_name(&srv_path->kobj)); close_path(to_srv_path(c->path)); } From patchwork Fri Aug 9 13:15:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758796 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D0B1194C7A for ; Fri, 9 Aug 2024 13:15:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209360; cv=none; b=Wz8sup7dD6GE5CgbIX8bHZpVT9pJcy49Yaw7Ehc4ar8XAvz0/BNjkHZcbekICEvykTsfxwKrQSot75hpeHiZZRBOrP0ftN3/ayQB2865Gj9Ao46rKc1IUgMatcO/d1ie7k+TQ0bidTIxinwuV6ELLa2+gbtkXY/R8RKAL9axNWM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209360; c=relaxed/simple; bh=aPdeol5t6wwSSHxVfJDH5JF9MYqzC9pPQihHi5HNRHY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YqX26JZrGOFJyIU8BSdKYY7zinitlL5MrJ+sJgKVXypLtGPH9cPRTXvbwwO9/w8EidejEg/RkqeGPEkSXJNO7ml0qu7sZ2kzu851O5+zdG4O6nlSFke8LH4Tjt7v81D2edjHeccZbo4eohS0cPKM5sJpdNj7hpydWaWiAjmb5wU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=g9QJBbkT; arc=none smtp.client-ip=209.85.128.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="g9QJBbkT" Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-428e0d18666so14762345e9.3 for ; Fri, 09 Aug 2024 06:15:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209357; x=1723814157; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4qJ3V7YJGTaGdVFsVivjHiBFvmyNekMesA0jctJoY0g=; b=g9QJBbkTQTyIrTSnjceuS5cDSADGHqhd6sk7WoQbcvi/qBtK3La2ybK8hbJDZHVAuI g6zlR8aN56OOT/SXB/vFf+rc7EF0BY74IhPHG+5zHSO9pSJolKrYr+ccxuadneLYekdt RxnlZpa1+rE3x+NlFzpLZyzhZwdDFBRjXCFdG6g66cQ3Dm5d1+vEPVzaxe0ZcjOiJfAo r8W0k0lyNCO3QZZq2562L2cdXRZvS6efT4dwKqUud+vp0Lha1L7MWZmrECwQ0S7dNzNg F8ToOL6GoZScUPiDUjyXIIeZ3kxkog9w7EOKwEpiIFzN3lyKUaWSXIaHHdS8q6FSr7PO r3hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209357; x=1723814157; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4qJ3V7YJGTaGdVFsVivjHiBFvmyNekMesA0jctJoY0g=; b=LdJsvZyqpxLFZm3EVcvlL18UMZq4HkoXDtUpY3FmAyw9/Dg71V0M+MDG2uNsqUInP6 Wl8vFom+6UHf0G8F/bo7x06cntRQXhT8AcK6O2dHP0NqfL83XjZJODUQ3CGrJ0Fr7tcI Xan+8ukWhsUkhn96XCGZilzSbD7ftTwdWeSJ8rJiK3JZaZwLcRxqdNSj4r96VzZKlCib 1dHP/liYD5VUbu8q+Z4a/yXzUu8rNzeu/zpmZbx5i0fAayrnAqNsuelrEWSx+hof9G8d MQWMIjKaXYePGqwng237iJ5oSer4vG1t33Rjr8cG5Ch1GBbxEmIbxTB0USt4V8yWU4dO iJOQ== X-Gm-Message-State: AOJu0Yw72uf1JZjK+JFpmEM2SmAsHEjZCJhpzRUjSRFW7gPpG5pTMSRT 2xWB6A6Kg3NbQjjrviPmJTuGnkgjhG8CNvDdU6iECueQiBKJYkf/m2YWkuKCTa5KfFf8Sn/Zbz1 A82s= X-Google-Smtp-Source: AGHT+IHlt1+BIV9S4Y0Nh5RjJBXf5glbMu0f9sBcmKh1HWNwwD1nNhO3bJ+cIo25rAT/MRFWkGJAyw== X-Received: by 2002:a05:600c:314a:b0:426:6eb9:db07 with SMTP id 5b1f17b1804b1-429c3a1bc51mr11322155e9.13.1723209356576; Fri, 09 Aug 2024 06:15:56 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:56 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 04/13] RDMA/rtrs-clt: Fix need_inv setting in error case Date: Fri, 9 Aug 2024 15:15:29 +0200 Message-Id: <20240809131538.944907-5-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jack Wang In some cases need_inv can be missed for write requests, additionally driver has to handle missing invalidates for write requests. While at it, remove the else case from write invalidate path as it is possible to reach there. Signed-off-by: Jack Wang Signed-off-by: Md Haris Iqbal Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 66ac4dba990f..d09018c11ece 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -391,11 +391,12 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, clt_path = to_clt_path(con->c.path); if (req->sg_cnt) { - if (req->dir == DMA_FROM_DEVICE && req->need_inv) { + if (req->need_inv) { /* - * We are here to invalidate read requests + * We are here to invalidate read/write requests * ourselves. In normal scenario server should - * send INV for all read requests, but + * send INV for all read requests, we do chained local + * invalidate for write requests, but * we are here, thus two things could happen: * * 1. this is failover, when errno != 0 @@ -422,14 +423,6 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, req->mr->rkey, err); } else if (can_wait) { wait_for_completion(&req->inv_comp); - } else { - /* - * Something went wrong, so request will be - * completed from INV callback. - */ - WARN_ON_ONCE(1); - - return; } if (!refcount_dec_and_test(&req->ref)) return; @@ -1146,6 +1139,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) }; wr = &rwr.wr; fr_en = true; + req->need_inv = true; refcount_inc(&req->ref); } /* @@ -1164,6 +1158,10 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) clt_path->hca_port); if (req->mp_policy == MP_POLICY_MIN_INFLIGHT) atomic_dec(&clt_path->stats->inflight); + if (req->need_inv) { + req->need_inv = false; + refcount_dec(&req->ref); + } if (req->sg_cnt) ib_dma_unmap_sg(clt_path->s.dev->ib_dev, req->sglist, req->sg_cnt, req->dir); From patchwork Fri Aug 9 13:15:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758797 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 099C11953BB for ; Fri, 9 Aug 2024 13:15:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209360; cv=none; b=fbvGYNiNJcPU+yzPXjdm1p5cT/r7NtM11sIjRnZnE7YpWM3DWPnvFVLbNcxN6nhWAVyoBFHKjerZQgjh5Ym+q1Vx0yeIo3Wcvv5eSdWwmkkPUR1LVK9hp3HwmZWwLSDWB30pLeaCwmN8+sMw+HPgSEXEtTD9q0zhU1/iaLA3EL0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209360; c=relaxed/simple; bh=wSG8mJWVwK1/KFG56H2h9FYqiJNGfBvw1cfDruXjyG8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BpZNvxVLaO3NYJD1cWY6Eb2WLcnCRLOL0qvutb5GqKOZW4SUVzCNngeRWUvmi4oSiIZkqxyzD+enKjfEIQt3Lmc1YN1aP/Qn/0EBXmMdkg1Vaaa4T0K+nY677bsNUecFf6bWKuJDUUVY1U+polNr8atarugwrgbIGayYoDZj0Pw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=BJuNNMUm; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="BJuNNMUm" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-3687f8fcab5so1020627f8f.3 for ; Fri, 09 Aug 2024 06:15:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209357; x=1723814157; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3fjw2Ta8LUJKcy0S2Y9A72sgh/iD12aMXAxb+1vnFQ4=; b=BJuNNMUmRGk04kapXi6OxBEPLMcPPjOr2LEAbhh2nRO1gxhH3IV9y5gvpv+MckI4qD plbqX/XlpdnhInjCukAPi7TWM5Hd58D7dUEPKBexypM+poyZYdgtcnF39+r7Uqo+vf5W MmeMya8pT+zpGuAN/2DNvGyBDS0V+xh3jsnbM0b9jHNIuhMPPfAVJZFacw77VP8kt1Nq qY++wQsrbpMk1x2Vvd+yz2tQeuH1zB0pKfGUxYr6dYSGUTBbcKaAtc4o6kZQA6syO+nF b6Oi5JDQmKPSY8llEXpAwfeYVP1ijUuQWw8K3p+/bBvd5/yOmVPFT8eSzGb9jyVS8eGn 2ayQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209357; x=1723814157; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3fjw2Ta8LUJKcy0S2Y9A72sgh/iD12aMXAxb+1vnFQ4=; b=FwAO7PKEBcqz2FJfq0yuPAwiKilGrsXHgGRQRwZywwofa5jWBG95xrIi2Q1ydLPsdW fJpqXE+CVlW3fQCu4eDAJDKd5fYeOgwGSRs6x7eHg94LbH/AYI4LdkqAInNnAjlrBsib Ea4B34wWwVdeuvAjEt3DTgvSHnFc237jVLBKrO5vbi1iGB9Kk68yKc051oWzZ/3jcA3d QuTfeH/elrp+ByVMqAy42jk/DL4EtzjREGxP2yzZ4OC7CI4bqnnJxI1GtF99632WvvZ1 36iPqsFGc4iH3yFTjX3RzG4CGqw6eX9gZSEGOx64ZziUrd+716Ms371tSgnH1A9qTCXl eCHQ== X-Gm-Message-State: AOJu0YyK3s2sl/oyQwcLF1USOaahgoEQXNTFPFV1rdLLU2TEnUgWPPlq ybKiWPt/XyYqZ2anUpPtKXg/vc60UcNpLg5yLBbQN4aCL3RKKckBuvYPdAYWUKLUeg3c2UvSaFv ZHDk= X-Google-Smtp-Source: AGHT+IH8u9UzHXh++14Gi2rWpw8AwEymQjk+lTqtXiCxSLQqgU0I2s8aFQyVjyk17RUcKUOmVtmQNw== X-Received: by 2002:adf:f349:0:b0:368:664a:d4f9 with SMTP id ffacd0b85a97d-36d5eb08646mr1274872f8f.28.1723209357248; Fri, 09 Aug 2024 06:15:57 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:56 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 05/13] RDMA/rtrs-clt: Rate limit errors in IO path Date: Fri, 9 Aug 2024 15:15:30 +0200 Message-Id: <20240809131538.944907-6-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jack Wang On network errors, a large number of these logs are printed due to all the inflight IOs, rate limit them so they do not clutter kernel log. Signed-off-by: Jack Wang Signed-off-by: Md Haris Iqbal Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index d09018c11ece..b34eb4908185 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -331,7 +331,7 @@ static void rtrs_clt_fast_reg_done(struct ib_cq *cq, struct ib_wc *wc) struct rtrs_clt_con *con = to_clt_con(wc->qp->qp_context); if (wc->status != IB_WC_SUCCESS) { - rtrs_err(con->c.path, "Failed IB_WR_REG_MR: %s\n", + rtrs_err_rl(con->c.path, "Failed IB_WR_REG_MR: %s\n", ib_wc_status_msg(wc->status)); rtrs_rdma_error_recovery(con); } @@ -351,7 +351,7 @@ static void rtrs_clt_inv_rkey_done(struct ib_cq *cq, struct ib_wc *wc) struct rtrs_clt_con *con = to_clt_con(wc->qp->qp_context); if (wc->status != IB_WC_SUCCESS) { - rtrs_err(con->c.path, "Failed IB_WR_LOCAL_INV: %s\n", + rtrs_err_rl(con->c.path, "Failed IB_WR_LOCAL_INV: %s\n", ib_wc_status_msg(wc->status)); rtrs_rdma_error_recovery(con); } @@ -419,7 +419,7 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, refcount_inc(&req->ref); err = rtrs_inv_rkey(req); if (err) { - rtrs_err(con->c.path, "Send INV WR key=%#x: %d\n", + rtrs_err_rl(con->c.path, "Send INV WR key=%#x: %d\n", req->mr->rkey, err); } else if (can_wait) { wait_for_completion(&req->inv_comp); From patchwork Fri Aug 9 13:15:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758798 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93EF0194C86 for ; Fri, 9 Aug 2024 13:15:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209361; cv=none; b=J0h4M4iKQn2+SVp+hyfW0XOJH04SGroPGtgT1qEhiEfa+MmMzR8lm/gph8N7yE1OyP27sBhWgT2t1meEgbD9dI+nLI4VTw/lIx4WlyDacQJv5VdSZekYy1fgnnKD/qvodSpWqK/zgGJXKttOcauUcGc9dT0/yROk8TLWylLVzqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209361; c=relaxed/simple; bh=mKmt/dUi7HOvG9hMniQLGyjZyDjxEOHIIhmOdSb5+Zw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hkUGVeBwd+4aDRuZGpew6g3dgRTlHJSZKUmCdzKnumEqFwh0foPkCP0LGok8DphKR/T75RlxTDiAf/YLWsazImYiGhi8cucxEJ/9iH/nGz/B9gmrofFgc5OqLvR2DssZsjHFe/7hjYu2TnTlUaq9auTvf4i0n37YfF+ZI6ynREs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=hb8ofc3s; arc=none smtp.client-ip=209.85.128.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="hb8ofc3s" Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-42816ca782dso15158305e9.2 for ; Fri, 09 Aug 2024 06:15:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209358; x=1723814158; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CHI7eUFb+AIheGv9VgUZkWaCMRJAb04OZNzPB+16rtY=; b=hb8ofc3szKIIzok/EuBS13RjgzqR/aKoAUkDDLdotiPQPIoLW7j9OxTa0bQCgJ2y0l G36UCZ7pHx1Ugn2ucyOu4NxujcPHL/mzYOamv4PekLwIFg0JpNaIvlm9GSUJHz2PBYUz CPIy1InYwbIQ7K9y37kQldxapauTtuNH/T8NEOLiA8/ErMU+Af3nUpGWFqVhRwQjc1zi bgRpIVFg+Nd5KKDBvDdq1//IotSNu5QPikPi6QKZz0Zch31a1fckPhi7jn/DCKCoGKkn fmua0fM6r544ee3JBy+M+nmrVp6jZZEkCjDhAjP55Rl7EKE+jKgZLy77PvR5EynI/JJq QzMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209358; x=1723814158; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CHI7eUFb+AIheGv9VgUZkWaCMRJAb04OZNzPB+16rtY=; b=VbK28HYrirsbw/FUe1JkfPWEZDZGB05dxTkjqMB2aHfpJDogmEwnbyc5AOA+/sEK8K ol8bbcvCfmgcV1mPYVYIYXNmaYV0TxEsFtXFvqCVMe5cHZF+qU316KOaNKovNBa519sp PrJXCY3nnf2zFF2p6L8f9O2oyqr3TwBertmHDRLyLgd0MjtoMkrXJq3Sz7Tj2ah067PX LR7jmxdTjXsSGgSRdPgCmgYZYOR1WDo5nagLELB+MEmlV1KKx97+bQls0P9UqAqa/vJj +R5wqHaXBAZYJzX/Hv/mfNkgBnlG+D+5BFr0zXRvmJ0OQuraczc5kzEYJgvcD378n0Rj ov7w== X-Gm-Message-State: AOJu0YwIlTJtLOHNhr68z+hlrd1ZeMP890UEOGGi0H0y52eIo4ZXN5Cl 65VGsVC/x3+P/9Gw6pEpbMeZCJL1v6hvp3Jjbq8QuuxkU+2u6qsWJGwfjxzHPLAF/plmSiMdg4g SuQc= X-Google-Smtp-Source: AGHT+IFkLlK92Fs26CagIKRdvS+zJzllhMVfBdQSkqbECQQHmdqsnyIxMrIlXpPP5JJhFySPcKADrQ== X-Received: by 2002:a05:600c:468b:b0:426:55a3:71af with SMTP id 5b1f17b1804b1-429c3a5b3c1mr10755915e9.33.1723209357867; Fri, 09 Aug 2024 06:15:57 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:57 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 06/13] RDMA/rtrs: Reset hb_missed_cnt after receiving other traffic from peer Date: Fri, 9 Aug 2024 15:15:31 +0200 Message-Id: <20240809131538.944907-7-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jack Wang Reset hb_missed_cnt after receiving traffic from other peer, so hb is more robust again high load on host or network. Fixes: 6a98d71daea1 ("RDMA/rtrs: client: main functionality") Signed-off-by: Jack Wang Signed-off-by: Md Haris Iqbal Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 3 ++- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index b34eb4908185..c1bca8972015 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -619,6 +619,7 @@ static void rtrs_clt_rdma_done(struct ib_cq *cq, struct ib_wc *wc) */ if (WARN_ON(wc->wr_cqe->done != rtrs_clt_rdma_done)) return; + clt_path->s.hb_missed_cnt = 0; rtrs_from_imm(be32_to_cpu(wc->ex.imm_data), &imm_type, &imm_payload); if (imm_type == RTRS_IO_RSP_IMM || @@ -636,7 +637,6 @@ static void rtrs_clt_rdma_done(struct ib_cq *cq, struct ib_wc *wc) return rtrs_clt_recv_done(con, wc); } else if (imm_type == RTRS_HB_ACK_IMM) { WARN_ON(con->c.cid); - clt_path->s.hb_missed_cnt = 0; clt_path->s.hb_cur_latency = ktime_sub(ktime_get(), clt_path->s.hb_last_sent); if (clt_path->flags & RTRS_MSG_NEW_RKEY_F) @@ -663,6 +663,7 @@ static void rtrs_clt_rdma_done(struct ib_cq *cq, struct ib_wc *wc) /* * Key invalidations from server side */ + clt_path->s.hb_missed_cnt = 0; WARN_ON(!(wc->wc_flags & IB_WC_WITH_INVALIDATE || wc->wc_flags & IB_WC_WITH_IMM)); WARN_ON(wc->wr_cqe->done != rtrs_clt_rdma_done); diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index e6885c88fa41..c7aec40d4934 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1233,6 +1233,7 @@ static void rtrs_srv_rdma_done(struct ib_cq *cq, struct ib_wc *wc) */ if (WARN_ON(wc->wr_cqe != &io_comp_cqe)) return; + srv_path->s.hb_missed_cnt = 0; err = rtrs_post_recv_empty(&con->c, &io_comp_cqe); if (err) { rtrs_err(s, "rtrs_post_recv(), err: %d\n", err); From patchwork Fri Aug 9 13:15:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758799 Received: from mail-lj1-f172.google.com (mail-lj1-f172.google.com [209.85.208.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F8F7194C9E for ; Fri, 9 Aug 2024 13:16:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209362; cv=none; b=rQK3BtsaytpJYJQACDmzp6SoyiqZJ9K1CiSsc5R6YkVJqTSI71ahK6tVcKRXrLh9GCKwwuHBcZowen/RCNtMdx51dg0qd3NtAnfrmReImXLNdvCVoanNKPsBnTebCcMbWD+GTEqdmltAZZi2fyhcSRfgJRAaUNUMLXFMaRDCxrc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209362; c=relaxed/simple; bh=r4XLSg/aoYdYcbzTL1SRlVWjZce8no+6qF2vTB5o4I8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jkWusQDgprt2kNTt9Ep7/CI6EALGqKnzT/Fk2IOpFq0MuR/74Ro/iqLKTmM79So2pO3DEHsnhnhqENeCyv0bdLSC4YbA9F/Mn5YJCdBIhR2sQTboI8T0s4/Md1yZdmUvY13Pk15VErPmgaTKI/Gl6fimrgoAd7DNlHytEOHq2sE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=QlKnVfX/; arc=none smtp.client-ip=209.85.208.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="QlKnVfX/" Received: by mail-lj1-f172.google.com with SMTP id 38308e7fff4ca-2eeb1ba0468so26283421fa.0 for ; Fri, 09 Aug 2024 06:16:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209359; x=1723814159; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Fx7gqp5M1vTXN5tlxGBDKn1NzWyNR+YNEd/cHoZlL1U=; b=QlKnVfX/GWU51VDNjkCHvl1yG+rvD/rhReOZWLLEe8UB7Z2/Za1EGQG7UgVAyFWn6V gLvt5HG3OI4EbmD8Be2Ek1O/k3kP/kiLUN90pUOAbMm7xp1e3kB/f1o3eRPgTXD2WPnz WxirIKTwNVH2F1FVEqyStAiCSW0+KTpnKNc08z1IOI3uB302yGGxu210TmXtAwpDwSCE FqR927qYzeMu2HFhjQxrJcHVjHkT96fVfa/IPs4FX1jjToE3/6JpsKAArbv4sfnOcxYh e9NGTWbU/xJz2fqwP6jRHmKf4rJschaDTsr6lLt8LKgFjTd7zfTCye4uZfqQ9dNM2C18 7GQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209359; x=1723814159; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Fx7gqp5M1vTXN5tlxGBDKn1NzWyNR+YNEd/cHoZlL1U=; b=mO6cuj+nb2Dkgk2tc0jleIWvSQTdDSLouW9WnWEhQpDP7JErL5RVJEFtzWVlrjhmmc 3m86sAsB/akRGL9Vtkc7JrbflyGKoQzALfRGklAu//aZknZSZ3Ed5XMCuSa7U4R/VUhh v4Cp3mBzBhdJMceBU3aplzRRLFhKHRqiZ9f22nnrGM8JRIVQ7oHGCFuG9TTnKLlHcbfE SHvsQhrMkk1rmCkLqJfciuah5WKJZfQ5sZmqUcy10DawrxgGNyEIot4M2RB3Ty4PW25X qoY8Z5hyCv2U2wxHgtGonMgTO/AbRfdg90JagHCym44/vdpou7XjbOvvNliu/fKBp1sk xD1g== X-Gm-Message-State: AOJu0YyQj3rCEDW4H0piz9hpkkE3q1QxV3ziqHHnKP/eKLOFuE/xG8CC ACHL6frW55aGnjdYXiKUpCs0EfW1ea61U0oCajGfjcWcqxp/j+Ghhx0cjAkdsTqo5GKf7WLWjwF EbIg= X-Google-Smtp-Source: AGHT+IHO70Sk0c5LS7d0WqZAUkebEsmEGtfT7vZG+YoMiRmlorb8zoe+yJ6xdIWIfxTQV7KLHbo/2g== X-Received: by 2002:a2e:bd13:0:b0:2f1:8624:a4c with SMTP id 38308e7fff4ca-2f1a6d037b8mr14594661fa.47.1723209358500; Fri, 09 Aug 2024 06:15:58 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:58 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 07/13] RDMA/rtrs-clt: Reuse need_inval from mr Date: Fri, 9 Aug 2024 15:15:32 +0200 Message-Id: <20240809131538.944907-8-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jack Wang mr has a member need_inval, which can be used to indicate if local invalidate is needed, switch to it and remove need_inv from rtrs_clt_io_req. Signed-off-by: Jack Wang Signed-off-by: Md Haris Iqbal Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 18 +++++++++--------- drivers/infiniband/ulp/rtrs/rtrs-clt.h | 1 - 2 files changed, 9 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index c1bca8972015..e1557b0cda05 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -355,7 +355,7 @@ static void rtrs_clt_inv_rkey_done(struct ib_cq *cq, struct ib_wc *wc) ib_wc_status_msg(wc->status)); rtrs_rdma_error_recovery(con); } - req->need_inv = false; + req->mr->need_inval = false; if (req->need_inv_comp) complete(&req->inv_comp); else @@ -391,7 +391,7 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, clt_path = to_clt_path(con->c.path); if (req->sg_cnt) { - if (req->need_inv) { + if (req->mr->need_inval) { /* * We are here to invalidate read/write requests * ourselves. In normal scenario server should @@ -494,7 +494,7 @@ static void process_io_rsp(struct rtrs_clt_path *clt_path, u32 msg_id, req = &clt_path->reqs[msg_id]; /* Drop need_inv if server responded with send with invalidation */ - req->need_inv &= !w_inval; + req->mr->need_inval &= !w_inval; complete_rdma_req(req, errno, true, false); } @@ -961,7 +961,7 @@ static void rtrs_clt_init_req(struct rtrs_clt_io_req *req, req->dir = dir; req->con = rtrs_permit_to_clt_con(clt_path, permit); req->conf = conf; - req->need_inv = false; + req->mr->need_inval = false; req->need_inv_comp = false; req->inv_errno = 0; refcount_set(&req->ref, 1); @@ -1140,8 +1140,8 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) }; wr = &rwr.wr; fr_en = true; - req->need_inv = true; refcount_inc(&req->ref); + req->mr->need_inval = true; } /* * Update stats now, after request is successfully sent it is not @@ -1159,8 +1159,8 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) clt_path->hca_port); if (req->mp_policy == MP_POLICY_MIN_INFLIGHT) atomic_dec(&clt_path->stats->inflight); - if (req->need_inv) { - req->need_inv = false; + if (req->mr->need_inval) { + req->mr->need_inval = false; refcount_dec(&req->ref); } if (req->sg_cnt) @@ -1236,7 +1236,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req) msg->desc[0].len = cpu_to_le32(req->mr->length); /* Further invalidation is required */ - req->need_inv = !!RTRS_MSG_NEED_INVAL_F; + req->mr->need_inval = !!RTRS_MSG_NEED_INVAL_F; } else { msg->sg_cnt = 0; @@ -1269,7 +1269,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req) clt_path->hca_port); if (req->mp_policy == MP_POLICY_MIN_INFLIGHT) atomic_dec(&clt_path->stats->inflight); - req->need_inv = false; + req->mr->need_inval = false; if (req->sg_cnt) ib_dma_unmap_sg(dev->ib_dev, req->sglist, req->sg_cnt, req->dir); diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.h b/drivers/infiniband/ulp/rtrs/rtrs-clt.h index f848c0392d98..45dac15825f4 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.h @@ -115,7 +115,6 @@ struct rtrs_clt_io_req { struct completion inv_comp; int inv_errno; bool need_inv_comp; - bool need_inv; refcount_t ref; }; From patchwork Fri Aug 9 13:15:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758800 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D34FF19580F for ; Fri, 9 Aug 2024 13:16:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209362; cv=none; b=M2rYQRfvTmMIVwoJVa/JEWikaSrEMmQ4sYl72NEyDqvWfnJLnzJMC/K8MUdzJunIvBhzBw0ndf8klpXVucv1Ljm8QymKdFun/+Y8QrPfZbrk/xPiJcbzoigp5tooRykkXHE4/ro9u4BwS19zlxq4xgFG8zRkwA0rgVOe/qkMx1M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209362; c=relaxed/simple; bh=WRV7DY/vIWFiZ3gU6V1V+cCtOFglJblnZJhmSfbRmuY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=E7MVeljPZq/aFJREkKDtT1IxeMlXUnXrKHyE8rFFVeb3dTtyVHQEik3NJM462mEX7VMPzylPn3x2zW7vRpnJDSPkH1YaFB4P7JJ4WlDs7kr5GuGjlYWqBumUKArJH5igFfc+FsE83WmwIlQZqHSVvZMQP+omt2zSdqqeA69deNY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=aLToIZvU; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="aLToIZvU" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-42817f1eb1fso13873075e9.1 for ; Fri, 09 Aug 2024 06:16:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209359; x=1723814159; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+XXHfR6ei5DWieETKO7ps9vaiG5tNzbNEBSlnAsZkFM=; b=aLToIZvUao40ePz+phS6m3v09/MBaBNLc6XevK+cuxteGGW3hygR6Us/INPmH5XGxW tUIL/hH9/NCiGh5MEF8endQ8giwrD9ECCQzXxpft1MOB0bmAmTr6w4g1aKIbHi2wexHY IAn/buOGekuv43m3ErGhlZv3b3A8kRkzNQPPorOfCP7xV5hRCMDlwAdnChUFyiYhPRHw 0qzRo0zCWQFXtCcp7gSv211Vu5b5IyP50gqO5TVqHWa02rl2mXseyyS11xj2Y5708Mau QmY61NV7oCm/WZcLmF5vTDlApBfCiIugSaJj/vKpzRq1tkcGXp9NqqBUoPSyM1KfqHom 9VMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209359; x=1723814159; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+XXHfR6ei5DWieETKO7ps9vaiG5tNzbNEBSlnAsZkFM=; b=wdToHw+QJpQtVrK6E1UMXwA8yMNAsuGY1jV9cmjx/LLhIDildIlX+/BF7Qei0r7TgA s0faT306VIm0eSS1yz3HU7SfLgwZNkGTynUSDxaAaoxDjQiaBQclLsjxGg/TU7pEzwhB SgRZLdltqIxssX2tKe4+2Hl4cUYE4+neMLJjWuhVBcQme8ITI4BurYykMyPSwr0jHqaU 35jzg+2btQFfZiCc1CMtLdkX/Idgyv3qxXJdZ3vcQqZ0s66mi24Acmm2l+yFZyc1QFQd hW2+LUYqbjZyN5LlHmKId7Y9puEcoZ8Vu2YCnRTPKvgkPA4Us8dwZoxUbMVOs+83+Gwe oHBw== X-Gm-Message-State: AOJu0YxAA8cANyLHRjcFxSL5MLwmCz1mEVA2StWYxaKVc6HI1SizXIo+ 7iCK6jvwZP5O1F1T/wHHyyYvHVFE5lZIH6rBh2Ekgcy6mx83iPH+Y5QU+sOBM58LP3VZtBBl7IP ucDY= X-Google-Smtp-Source: AGHT+IFOy/Vp1or3cPmYTvwJRTcH+PWPhvz/v7WX2cpz3oAkiABEA+9m0lWzfee0JDQvtSz+n9zC3g== X-Received: by 2002:a05:600c:4708:b0:427:dae6:8416 with SMTP id 5b1f17b1804b1-429c3a56c64mr10586625e9.36.1723209359181; Fri, 09 Aug 2024 06:15:59 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:58 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 08/13] RDMA/rtrs-clt: Reset cid to con_num - 1 to stay in bounds Date: Fri, 9 Aug 2024 15:15:33 +0200 Message-Id: <20240809131538.944907-9-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In the function init_conns(), after the create_con() and create_cm() for loop if something fails. In the cleanup for loop after the destroy tag, we access out of bound memory because cid is set to clt_path->s.con_num. This commits resets the cid to clt_path->s.con_num - 1, to stay in bounds in the cleanup loop later. Fixes: 6a98d71daea1 ("RDMA/rtrs: client: main functionality") Signed-off-by: Md Haris Iqbal Signed-off-by: Jack Wang Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index e1557b0cda05..777f8e52ed7c 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -2347,6 +2347,12 @@ static int init_conns(struct rtrs_clt_path *clt_path) if (err) goto destroy; } + + /* + * Set the cid to con_num - 1, since if we fail later, we want to stay in bounds. + */ + cid = clt_path->s.con_num - 1; + err = alloc_path_reqs(clt_path); if (err) goto destroy; From patchwork Fri Aug 9 13:15:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758801 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DE5B195B14 for ; Fri, 9 Aug 2024 13:16:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209363; cv=none; b=FdxfLbunFkMuhiwMbErnwZTc8nwkGAYczvg3qVfTtF2nPU/jOhQd1VI2Zey1VucMKmc3EFQYoUhnOKx8PZn2QUHnILnCRcYCwnyzq1M+/R7YxkjcMBYnzLLoDLHKcRfQQ3WqzYJMLMapAnQfEZGqtV0YHYRX6JJN8B9+Disct4s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209363; c=relaxed/simple; bh=E4Tkz+NNXf0Rqt+KeIW7YV0+OPjkqtWHuBsqB6QLKqw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZhpPWh6dlq9cDb75RLD/dgJqV1PXvQfVTYSpEtOax0WNgWLAF3WojUcAU+yTTCSPoS45iR5y4PzWR54i/V6Y8LUNo1Ys0mvZAFYuwiLYuVn0Lttgg0BZxb4h/2VH5/omhNXp6tPV2ZMWDOIA3TF5Cr/kAQ9H+kIy0T0maTYadi8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=dQGgpISO; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="dQGgpISO" Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-42819654737so14980215e9.1 for ; Fri, 09 Aug 2024 06:16:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209360; x=1723814160; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HC3jHGsjdZENIhkaZnf9KhbQHlXlkHO+680dBNxyyNY=; b=dQGgpISOBVYhG21pCTlGvvG9W4gi2GC8NUUfAWX03NtxIECxc0Oo+mNt6mz757WSyi ErtLBkr+K+vtuBFMr7t2sruqfUbOyHXWrJ8B6NhCs0SHfXVIdnce8O+VvzAkLZdpDF+K NKoHJ7KmlgupjGhD6f9AZj9uCC+wBBCJM98OZ+ZWIXWIBcvTG3LIRLI9CgG51WNr4CAD lcRud9eo1vHmDzCftntK1h9X9do7xtnaxlq/UvXwhcOQRKSnwprfXfbzUMK1o4oQmWzF mP0nqxXAblt+RlpsnzvgWjYomlHsLAs4HeBvZ06CymS8/gR8CN+Js0bV/LvGc8fMzXgw XM6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209360; x=1723814160; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HC3jHGsjdZENIhkaZnf9KhbQHlXlkHO+680dBNxyyNY=; b=AvrhT7LRIR+k1sMi0X/Eu9MNnX2rwpqk1zrt3eKCOuWYR56KyufqWvKUAZXRhkTL1O jhh96Dd/APqHJ2BrtsY183wMmm8y3l2qJBETBb8UGS6WMNmjzXe3HVN7egpDdc34VbVK 6KTq9tx30nq1TuMlE0rKTk9CdVPKmmFXfQ4opIJk6ecvw/PGINsrrWhrksGltt+tAA6J mwM3Hd8ia/rfLhhRfVutqql0/mfaTmr4ReZge+OvFdu7orMAai/QMj3tKuNout5ojim6 8H45ovxN9x0gaQuxz6lpZtVdBYBiOcU/KzKtCIxv++Twgvi74b+7SAyIt2H7CTCY+JTc VMCw== X-Gm-Message-State: AOJu0Yytq+dqR6BCbgXeGFMnOdHsoW6BzZvBP9T3jtoNYYT9EUP/w6mS siyAUbKzAYId+JFlHTlL+X+r2hK5k9NxJK6craKv5IJXH3QOvAa3Of6L2+PLbd94BgAc98Pe244 SC9A= X-Google-Smtp-Source: AGHT+IEc73iMrGKwQUVryFXUYxjrirSmpfWGiigM84auD1QPv2kqKCqJhr3GXkbxTx7DWUTv6ae4XA== X-Received: by 2002:a05:600c:198a:b0:426:6e95:78d6 with SMTP id 5b1f17b1804b1-429c3a19090mr11497105e9.4.1723209359798; Fri, 09 Aug 2024 06:15:59 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:15:59 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 09/13] RDMA/rtrs-clt: Print request type for errors Date: Fri, 9 Aug 2024 15:15:34 +0200 Message-Id: <20240809131538.944907-10-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jack Wang Extend the output to print also the request type. Signed-off-by: Jack Wang Signed-off-by: Grzegorz Prajsner Signed-off-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 777f8e52ed7c..7c6d40380638 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -439,8 +439,10 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, req->con = NULL; if (errno) { - rtrs_err_rl(con->c.path, "IO request failed: error=%d path=%s [%s:%u] notify=%d\n", - errno, kobject_name(&clt_path->kobj), clt_path->hca_name, + rtrs_err_rl(con->c.path, + "IO %s request failed: error=%d path=%s [%s:%u] notify=%d\n", + req->dir == DMA_TO_DEVICE ? "write" : "read", errno, + kobject_name(&clt_path->kobj), clt_path->hca_name, clt_path->hca_port, notify); } From patchwork Fri Aug 9 13:15:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758802 Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5570D195B18 for ; Fri, 9 Aug 2024 13:16:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209364; cv=none; b=mn8pjC3RU+PtDq4BQWc4SnJKn5fuPEnvsXHGFeucXJ5uMOpB6X7gkrpo/qERkMHgid+MHgA+KTqWQ4pCeMad+u6/50qsunMAStS2WhI3lMpCB/VaTyc5tW/ULK+fv1gzWXRIHGMDwp2/WHhNZDnQ4wT498x/7Ua3a1eCTVzTT5A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209364; c=relaxed/simple; bh=bDW/dF13VUplmcG0FnCFuS1gT0xUyjb3t46RhgMKjUQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kchnNhWI2uXAXH6tX+Erah58DBsd8FXlix9dTsC05SCTAomAALlxVRE3U01XltjUPbFreUczz165udu3u1VESVechBIukVf7b69zr/R+iCJvtthO7SbJXcJnnhLQPhAhCW7L7oYSE6d7g//arV01+9Av+4hy965nQkmY86h9M3M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=YIMiDe5r; arc=none smtp.client-ip=209.85.208.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="YIMiDe5r" Received: by mail-lj1-f179.google.com with SMTP id 38308e7fff4ca-2eeb1ba040aso27889451fa.1 for ; Fri, 09 Aug 2024 06:16:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209360; x=1723814160; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LN/3NtEA8ChzCaWTJF8ZmgksgTpp42boQxGcCG9OcqE=; b=YIMiDe5rmxjHqGFBXiSQRESme7CoEO5T4zpkFwsF5Akt0isRt4ILLMhB+IFMGcONvo YV490nDCSEDSfP8IPgqdxuoWec159g1ki4zybH4hHNl/6zDH02csmyg+1AgOax65VUbc r1/5TkUTgEHIxCs2nX9ignWqnzFv6iNFQFXrr/I6jIIXCQ4IMvA4VaU6HXh19txiDH6w CxAlxOACVZlFz+VaJZ0JGTvz1UlPUzNrjj9o5wSnMpLk62Jwa9UuR+yxWdwqAz3EU3lZ DVioR5lmlbYD/YMKtHSGln8/u7MjtAQ1/oSfn62FYHTp8kQ4uVx0LYBi5ofX6gsoLiMF ixFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209360; x=1723814160; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LN/3NtEA8ChzCaWTJF8ZmgksgTpp42boQxGcCG9OcqE=; b=I/kX+NkyholqQE1T22Az/11Xlhdjnt+8J16Utr1fg//USoAUlzslmZKBCv6bb8aHoH 2ryByYA92BF/yrysnKVRmdiCe9f/1DjQGCCg6lYrqYAeVGzevjBDYxf1ifPK21rurphm QdNRzNGGArcxhWzl7GT6pqGJFIEz6+l2bIGrnC0UoaClTfLr6Vodyj/AeGEZ05oIWGZd F/D5s7ZhYg7eIpZ+AYKRljMrG/aG6U87FUcmPq+hUBhTAUIDaORkDxHg+vK0j5WP3blr cdemVLZrn8sTVGDj1udux1YeR2WilZBOVr5TQy7275nLMuPXDtwjVppd0vmEKb+OfGVa zpdw== X-Gm-Message-State: AOJu0YydsUPGdQJ9UglJhJQLuGUxb/3+g/pL7HZ0KXDiPhpQbblWt9tY uAvSEB0EZLjGiOiRaeVhr39xJvcCqU6XltxP0DXG8JL25tjr2eFJDRaK9kmReEwguVrAZwX7X4F AUuc= X-Google-Smtp-Source: AGHT+IEttW1/uGKUeVw42t5d5ekfysknQZ9MJbkkrqAFgAGNt9Fmek8l5eJy6OItypje7dXnaNpRYw== X-Received: by 2002:a2e:b5ca:0:b0:2ee:df26:9d4d with SMTP id 38308e7fff4ca-2f1a6d6f776mr11485201fa.47.1723209360488; Fri, 09 Aug 2024 06:16:00 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.15.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:16:00 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 10/13] RDMA/rtrs-srv: Avoid null pointer deref during path establishment Date: Fri, 9 Aug 2024 15:15:35 +0200 Message-Id: <20240809131538.944907-11-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 For RTRS path establishment, RTRS client initiates and completes con_num of connections. After establishing all its connections, the information is exchanged between the client and server through the info_req message. During this exchange, it is essential that all connections have been established, and the state of the RTRS srv path is CONNECTED. So add these sanity checks, to make sure we detect and abort process in error scenarios to avoid null pointer deref. Signed-off-by: Md Haris Iqbal Signed-off-by: Jack Wang Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index c7aec40d4934..fe8abbe0d2e9 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -935,12 +935,11 @@ static void rtrs_srv_info_req_done(struct ib_cq *cq, struct ib_wc *wc) if (err) goto close; -out: rtrs_iu_free(iu, srv_path->s.dev->ib_dev, 1); return; close: + rtrs_iu_free(iu, srv_path->s.dev->ib_dev, 1); close_path(srv_path); - goto out; } static int post_recv_info_req(struct rtrs_srv_con *con) @@ -991,6 +990,16 @@ static int post_recv_path(struct rtrs_srv_path *srv_path) q_size = SERVICE_CON_QUEUE_DEPTH; else q_size = srv->queue_depth; + if (srv_path->state != RTRS_SRV_CONNECTING) { + rtrs_err(s, "Path state invalid. state %s\n", + rtrs_srv_state_str(srv_path->state)); + return -EIO; + } + + if (!srv_path->s.con[cid]) { + rtrs_err(s, "Conn not set for %d\n", cid); + return -EIO; + } err = post_recv_io(to_srv_con(srv_path->s.con[cid]), q_size); if (err) { From patchwork Fri Aug 9 13:15:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758803 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEE1E195980 for ; Fri, 9 Aug 2024 13:16:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209364; cv=none; b=V8DuEVJoBir9URr6tc0odbUltL1eBVcGKA/yRf/d3LQXb3CzhsFViethB8cuLXPaauHwMUIpFc2WA0YK3NsHOV6Lmb/wmm37b137/4SNqUHmenztJIZE794NVy5GTMULHR5EOvQokdKLEkuKmdnveokEPAsv13K5euFHVYNDr/U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209364; c=relaxed/simple; bh=ad06e7JutVcxmMQ0BOmwpDAQmKkjLSLR/RzU0bbWwGM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WQEnNkYuOjwNSr3Wa0iQiqjTafKXf0ryD4fi8YOPEOwuBnBNX9x476Qby/eIu4VKB6s9rh7zUoo1Yy4sGg5VUDUS0a9Wid9xaIkMZXPJFlniW2p7aRPevivgDO/LrsZb/KOhZbVxD6EbQWOgafnjYR5UK543G1VSl/AtL8Zywog= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=bajxgn0Y; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="bajxgn0Y" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-428fb103724so17777515e9.1 for ; Fri, 09 Aug 2024 06:16:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209361; x=1723814161; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=84Ddqq+9UVuqSicaxcUJ99dvqDlCAHZBvWXLqs6sph8=; b=bajxgn0Yr9pdrhLRanDSayDOMBfK5NTN8JwN0SSRikNVNpiw0ycCDVLzIVH8k8Fu5x +gWRM4GcAitUkcmhTbNikFOsPkhol+dm+zfLeCmQeghxrTw7hHv33oY/zNFT4HbEu2Co nXn1kvsIs0vEGl6cN/f8nHIxH0uG7VKNS6HpUHwvdBVjg0uproVLAgM71YXeNqz20eBQ kRS2OFYSASuPWYa++fOKwnJSvLgAk/P+TyDgtRRgq7t2Tg8nVdvjLkDHjPed6tJ2lR0g 5fisFWlalbPzZZ/F1R8PaGuOP43NhuJiZhbZbnEoQE3rTauGNmFrSRTM7dPPiTGVF2yT JJgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209361; x=1723814161; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=84Ddqq+9UVuqSicaxcUJ99dvqDlCAHZBvWXLqs6sph8=; b=wXy7rwRcwbfayKQuMl2uW63vXR0GcX1wRtGHoL78jZ3Md6fhTSvhleBDpXXyEIHYl8 5Gi6jsF6OxMsEVgqb3Z+5t89jtZIMtbXJA+eCfnFp3eNytxVWTu1FixCKwa0lRJE3UdV OOO3WG/LRrODngz1jEbEDJhxQlbdfHyfJWrDEcvgdBhKIsvXyPBt8uBHDdPPGsYLl70U vTq1AZKIAjhtrLVkHteIkLX0LvuiOLHhlR65PpKfKrfQfvxbCCguuExv9ij1NTUszs3j 6hokZNjQPrngyl1RPqHGYRO+YYId2l8qMcRmokhUrEyXbxO8l8S/4TM2SGtg6YFsxFoF g24A== X-Gm-Message-State: AOJu0YxLs+tzNUvFgKV4VpUIJf1FmKeus0ZMFZOhosXbV8KtrGd9fwnF kQmC92QxwDdqVhsGR2QWmO2TLDmir29hKgXgQAKTgnEmnErAeOHccWHMLF0uN9kaLKS5itflvmf fPo4= X-Google-Smtp-Source: AGHT+IFEZzPEJw80hqASm7nrWVNaM8N/Qf9MOdXGDp3baMw8HWD/hRRj3wzJTfvx06isFl5ZTRjSxg== X-Received: by 2002:a05:600c:4583:b0:424:71f7:77f2 with SMTP id 5b1f17b1804b1-4290b8df20fmr40606135e9.16.1723209361171; Fri, 09 Aug 2024 06:16:01 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.16.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:16:00 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 11/13] RDMA/rtrs: register ib event handler Date: Fri, 9 Aug 2024 15:15:36 +0200 Message-Id: <20240809131538.944907-12-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Grzegorz Prajsner Use ib_register_event_handler() to register event handlers for both client and server side. For now, all those handlers do, is to print type of incoming event. Signed-off-by: Grzegorz Prajsner Signed-off-by: Jack Wang Signed-off-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 21 +++++++++++++++- drivers/infiniband/ulp/rtrs/rtrs-clt.h | 2 ++ drivers/infiniband/ulp/rtrs/rtrs-pri.h | 2 ++ drivers/infiniband/ulp/rtrs/rtrs-srv.c | 33 +++++++++++++++++++++++++- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 2 ++ 5 files changed, 58 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 7c6d40380638..230e5f6c8c90 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -3149,8 +3149,20 @@ int rtrs_clt_create_path_from_sysfs(struct rtrs_clt_sess *clt, return err; } +void rtrs_clt_ib_event_handler(struct ib_event_handler *handler, + struct ib_event *ibevent) +{ + pr_info("Handling event: %s (%d).\n", ib_event_msg(ibevent->event), + ibevent->event); +} + + static int rtrs_clt_ib_dev_init(struct rtrs_ib_dev *dev) { + INIT_IB_EVENT_HANDLER(&dev->event_handler, dev->ib_dev, + rtrs_clt_ib_event_handler); + ib_register_event_handler(&dev->event_handler); + if (!(dev->ib_dev->attrs.device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS)) { pr_err("Memory registrations not supported.\n"); @@ -3160,8 +3172,15 @@ static int rtrs_clt_ib_dev_init(struct rtrs_ib_dev *dev) return 0; } +static void rtrs_clt_ib_dev_deinit(struct rtrs_ib_dev *dev) +{ + ib_unregister_event_handler(&dev->event_handler); +} + + static const struct rtrs_rdma_dev_pd_ops dev_pd_ops = { - .init = rtrs_clt_ib_dev_init + .init = rtrs_clt_ib_dev_init, + .deinit = rtrs_clt_ib_dev_deinit }; static int __init rtrs_client_init(void) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.h b/drivers/infiniband/ulp/rtrs/rtrs-clt.h index 45dac15825f4..0f57759b3080 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.h @@ -212,6 +212,8 @@ int rtrs_clt_remove_path_from_sysfs(struct rtrs_clt_path *path, void rtrs_clt_set_max_reconnect_attempts(struct rtrs_clt_sess *clt, int value); int rtrs_clt_get_max_reconnect_attempts(const struct rtrs_clt_sess *clt); void free_path(struct rtrs_clt_path *clt_path); +void rtrs_clt_ib_event_handler(struct ib_event_handler *handler, + struct ib_event *ibevent); /* rtrs-clt-stats.c */ diff --git a/drivers/infiniband/ulp/rtrs/rtrs-pri.h b/drivers/infiniband/ulp/rtrs/rtrs-pri.h index ab25619261d2..ef29bd483b5a 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-pri.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-pri.h @@ -69,6 +69,7 @@ struct rtrs_ib_dev; struct rtrs_rdma_dev_pd_ops { int (*init)(struct rtrs_ib_dev *dev); + void (*deinit)(struct rtrs_ib_dev *dev); }; struct rtrs_rdma_dev_pd { @@ -84,6 +85,7 @@ struct rtrs_ib_dev { struct kref ref; struct list_head entry; struct rtrs_rdma_dev_pd *pool; + struct ib_event_handler event_handler; }; struct rtrs_con { diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index fe8abbe0d2e9..141f05a1c395 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -26,7 +26,10 @@ MODULE_LICENSE("GPL"); #define DEFAULT_SESS_QUEUE_DEPTH 512 #define MAX_HDR_SIZE PAGE_SIZE -static struct rtrs_rdma_dev_pd dev_pd; +static const struct rtrs_rdma_dev_pd_ops dev_pd_ops; +static struct rtrs_rdma_dev_pd dev_pd = { + .ops = &dev_pd_ops +}; const struct class rtrs_dev_class = { .name = "rtrs-server", }; @@ -2286,6 +2289,34 @@ static int check_module_params(void) return 0; } +void rtrs_srv_ib_event_handler(struct ib_event_handler *handler, + struct ib_event *ibevent) +{ + pr_info("Handling event: %s (%d).\n", ib_event_msg(ibevent->event), + ibevent->event); +} + +static int rtrs_srv_ib_dev_init(struct rtrs_ib_dev *dev) +{ + INIT_IB_EVENT_HANDLER(&dev->event_handler, dev->ib_dev, + rtrs_srv_ib_event_handler); + ib_register_event_handler(&dev->event_handler); + + return 0; +} + +static void rtrs_srv_ib_dev_deinit(struct rtrs_ib_dev *dev) +{ + ib_unregister_event_handler(&dev->event_handler); +} + + +static const struct rtrs_rdma_dev_pd_ops dev_pd_ops = { + .init = rtrs_srv_ib_dev_init, + .deinit = rtrs_srv_ib_dev_deinit +}; + + static int __init rtrs_server_init(void) { int err; diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.h b/drivers/infiniband/ulp/rtrs/rtrs-srv.h index 4924dde0a708..99cfd2e58ca1 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.h @@ -133,6 +133,8 @@ struct rtrs_srv_ib_ctx { extern const struct class rtrs_dev_class; void close_path(struct rtrs_srv_path *srv_path); +void rtrs_srv_ib_event_handler(struct ib_event_handler *handler, + struct ib_event *ibevent); static inline void rtrs_srv_update_rdma_stats(struct rtrs_srv_stats *s, size_t size, int d) From patchwork Fri Aug 9 13:15:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758804 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76A7C195B37 for ; Fri, 9 Aug 2024 13:16:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209365; cv=none; b=VRbfohT4qFTSQsu9BWm4dJ4DpRXt1kriY/IcB+OBG/kxyIjfU9jwr2nPEXUmWEbJ18WfIXX8TxZGqsBberT5QPG+Y1Xu4VtaL9exd9lun10ZGpCjy+IjGv492usC3PBhmxMnB5y4ET33NMOL0HV4eTQYGJ8/6wfk+k5YVCk1MvE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209365; c=relaxed/simple; bh=xLcPXGIQYb8rDHlaDSABgBgt2ojUJwEMbr395zl0B7I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PUyckA+4WLZICKmkWkKl1QEgkkDCnqbskZF56TgA496op5yiTSCTgYTcshiWf0/iNITMIk8wG+FHdkJurkNC6xM5kJ8Llgw9iElLHDOT6LW6DsNhRquY3uE26/MsRadfVqZcF5Zp1qtZj0MKx+JwP5UDGOJpkskSz5u/FX0Lwik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=cUb83A4r; arc=none smtp.client-ip=209.85.128.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="cUb83A4r" Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-428243f928fso19943135e9.0 for ; Fri, 09 Aug 2024 06:16:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209362; x=1723814162; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GQMKt+Z6oU8k718oVYzHw1t/snarDjAZbdeYqhgVzX8=; b=cUb83A4rAOiQm508KwkecCsqgkqjNglwgH+U0ZOKuCMDpTDoWsLRXfSKjjPPFb4lHD A2sZXbyGrBabdnWtsvqOt5qx+cMGjhGASMP/puAKKNlEr4xqIer4NjjL6QwbNKlP3/Vs f+uEo8uXAb4b+wo/kdgf7wj8oRRzM6BZJLW4icUa/Yj1wisDSvLfPQXQIwv5MTGPtlA1 HLHvW9Fux7ZgMdsMpbuNHz1+E6Px7eiLYYfYgXOq7lhmfqn6WGeXblvtrYbEq5r7CjIW aqXQx47q5p7oq4ZeAOoz/7JJso0Ky4tUvE+3esidn3l0ZWHr+q26799OAT7m0gfC3XyM 3Hqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209362; x=1723814162; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GQMKt+Z6oU8k718oVYzHw1t/snarDjAZbdeYqhgVzX8=; b=IzM39PxKyAOBCkmtcEjXFJXhE9h4EbcE62DJ66u2/aSxgPkJFegWML24vfKibX4hv3 r08xlrbPE1zhE2iAHgGAdb6Dts2TEHQQ6PROLrrL44tMnNQYgAFttnTS1fgB8l9cSpeF KQURsMcWUBFbg5AaArThozM72lVhtQeNRJsvA0qUb02CkVekX6Na9dnVTPfpodOkL6W6 ecnd1CFF02bcAhEOm03m5pCNgz893nYdheMZbPCgBX5zG9x3ZR7msfjshTuZuTjyO+D5 n7r0/0UZRKME/kwyvfQLaoStiLv7vdFYZk1z0oXiLONuA7KAfEDRz5e9Cbq1U9vofmus Hsvg== X-Gm-Message-State: AOJu0YwlTp5cDo59e/repJE5a7ppcc12xbX7YYFkzTgV9GYagjecR2Rh 8VTzdcBzCoECl8yuw9K6aNXcyPCVkWX9rjOOPPF3biCQg2iBuPR7FWOVsv04oXNmdnRCmCgiIdf vYK8= X-Google-Smtp-Source: AGHT+IG5/72HG33wWaeikkhgssEOQf1p+F7W/WPJQ+UU2bS7uNGoCmvWcSHFgvMBB+NONFW/g3fKRA== X-Received: by 2002:a05:600c:1d1d:b0:426:6ed5:fcb with SMTP id 5b1f17b1804b1-429c3a178fcmr14742575e9.4.1723209361795; Fri, 09 Aug 2024 06:16:01 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.16.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:16:01 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Grzegorz Prajsner Subject: [PATCH for-next 12/13] RDMA/rtrs-clt: Do local invalidate after write io completion Date: Fri, 9 Aug 2024 15:15:37 +0200 Message-Id: <20240809131538.944907-13-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jack Wang Switch local invalidate after write io completion avoid the chain usage of WR, this fixed the local protection error on LOCAL INVALIDATE WR. Signed-off-by: Jack Wang Signed-off-by: Md Haris Iqbal Signed-off-by: Grzegorz Prajsner --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 230e5f6c8c90..fb548d6a0aae 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -395,9 +395,9 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, /* * We are here to invalidate read/write requests * ourselves. In normal scenario server should - * send INV for all read requests, we do chained local - * invalidate for write requests, but - * we are here, thus two things could happen: + * send INV for all read requests, we do local + * invalidate for write requests ourselves, but + * we are here, thus three things could happen: * * 1. this is failover, when errno != 0 * and can_wait == 1, @@ -405,6 +405,9 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, * 2. something totally bad happened and * server forgot to send INV, so we * should do that ourselves. + * + * 3. write request finishes, we need to do local + * invalidate */ if (can_wait) { @@ -1085,7 +1088,6 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) int ret, count = 0; u32 imm, buf_id; struct ib_reg_wr rwr; - struct ib_send_wr inv_wr; struct ib_send_wr *wr = NULL; bool fr_en = false; @@ -1126,13 +1128,6 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) req->sg_cnt, req->dir); return ret; } - inv_wr = (struct ib_send_wr) { - .opcode = IB_WR_LOCAL_INV, - .wr_cqe = &req->inv_cqe, - .send_flags = IB_SEND_SIGNALED, - .ex.invalidate_rkey = req->mr->rkey, - }; - req->inv_cqe.done = rtrs_clt_inv_rkey_done; rwr = (struct ib_reg_wr) { .wr.opcode = IB_WR_REG_MR, .wr.wr_cqe = &fast_reg_cqe, @@ -1142,7 +1137,6 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) }; wr = &rwr.wr; fr_en = true; - refcount_inc(&req->ref); req->mr->need_inval = true; } /* @@ -1153,7 +1147,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) ret = rtrs_post_rdma_write_sg(req->con, req, rbuf, fr_en, count, req->usr_len + sizeof(*msg), - imm, wr, &inv_wr); + imm, wr, NULL); if (ret) { rtrs_err_rl(s, "Write request failed: error=%d path=%s [%s:%u]\n", From patchwork Fri Aug 9 13:15:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haris Iqbal X-Patchwork-Id: 13758805 Received: from mail-lj1-f170.google.com (mail-lj1-f170.google.com [209.85.208.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E787195B14 for ; Fri, 9 Aug 2024 13:16:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209366; cv=none; b=K4Nv2BbXF2t6jUXBC/gWMhVJO1TQLTyC2Zj4mN6+CjKIsU2R3RvLX0TY8btLFLY49dgS1jrdvACje/nnGnA1O3XnCOPlPq9VwgWJnZXUf8sgLkXpq6Y9r4hC6U2yHX9N2GVWu/Lwi+3ysYwf2ny76PpiVY5nIUBAWKYjwfS++ZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723209366; c=relaxed/simple; bh=NHOZPA7FQFWYBrrm4XDDP7jjZga37iVxrWesXRzefvo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=u0aLVTRuFFY9kJ8Lgy1Fec/LlTqTpIEdRld28/bISWji5DUyCHHf+lQW0vxTs12K8H2wwBlKFbtUWbVR8yfbJAE5jewAz///BB3mDs03Ll7tMT3irhibZHCGcn5fdKWWYNsLMPC7JTzu8IXsUkDLiAEfxLKnz0dG4FV9ihi87ds= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=Eu2Fg5+g; arc=none smtp.client-ip=209.85.208.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="Eu2Fg5+g" Received: by mail-lj1-f170.google.com with SMTP id 38308e7fff4ca-2f15e48f35bso20647241fa.0 for ; Fri, 09 Aug 2024 06:16:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1723209363; x=1723814163; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ekWst1Xo947LcQ5w8RGs9hy6QKMdjD2JEkl1IhxRBGY=; b=Eu2Fg5+gZMDScKcIXoc2atMpcYunjNH7Cn3EoEmFgQsWCS0jjjU5at93dARRKm0RXd 7tmK5Rr4kw5cy8QdGAzhQ6xa2BbES3h2hY8GNWADuQgc5wn0FtTKBIYy++Xgz9tnSbpr R8kF3thA1RCkrdnf84sJvGrowoe6AZW+H802Qa7JGkHOy16oqXThVFaFJ5BJHqPemBDz 64CcSPLlHWCN464891bJgxfsgEUXf2U+RqaZV0KkEyRZb+UFduWb5cRAzO9FDPKwnHXg 6nMuCv+CBfw1B/i/FbYGgfSJKQoqq5f5AccuQ6trCjiMNeOPn2XgBsxsq4cRHU+3XR86 7+Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723209363; x=1723814163; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ekWst1Xo947LcQ5w8RGs9hy6QKMdjD2JEkl1IhxRBGY=; b=MZ7k7O+SzsVX6TrI1Gb3G5U1R/m6yYrcHcFvh2ukL22NxVB2xyrYT7IomY/4YKWT3Z /WvqEU93qv6qetJiUtOERHuYBm8CvgXBJDWm5y9ySOtshnF6XRdqY+Gr3P6cfMrbOU3M XvlnhR7XdX3UNqNPHiBJ/1jkv7ojOZGyrmd28U/6B08HtCJiYvD9FMSUB+EDoz4JXwhW di3WAPt7VlIjyRJ2w39k85kDolPur6getF87cNsQyxR4F5dti90pO93LOXkA3pKsPa8v FHYi/bzFy5Rc4U4FWrjncptSOs9RXlFFOw4HK8sh5iQtFSzZH2xb/1oyvyfDsXTJpwn9 uUTw== X-Gm-Message-State: AOJu0Yx7rUm4PubzvXWMjTHW0MHT0Bc79S0JWDd0ZdrihkgHoB/4htEr W+d7g84xCiofLjngjSOD8UaKLoAWtMMTrc453YuJDGND7v4d9/3OKFDtZWqW4zdD0ck9/Vm8ShH esQQ= X-Google-Smtp-Source: AGHT+IEuOLWi7mFIp6IFy13DinFW+llzXDfxpgfP169ms/QzFtO4N0cdMMFaLzSYtI0docGmz5tM8Q== X-Received: by 2002:a2e:b8c8:0:b0:2f0:25dc:186d with SMTP id 38308e7fff4ca-2f1a6d0038dmr13766611fa.1.1723209362497; Fri, 09 Aug 2024 06:16:02 -0700 (PDT) Received: from lb01533.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4290c77f078sm78280625e9.37.2024.08.09.06.16.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 06:16:02 -0700 (PDT) From: Md Haris Iqbal To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Alexei Pastuchov , Grzegorz Prajsner Subject: [PATCH for-next 13/13] RDMA/rtrs-clt: Remove an extra space Date: Fri, 9 Aug 2024 15:15:38 +0200 Message-Id: <20240809131538.944907-14-haris.iqbal@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240809131538.944907-1-haris.iqbal@ionos.com> References: <20240809131538.944907-1-haris.iqbal@ionos.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jack Wang Signed-off-by: Jack Wang Signed-off-by: Alexei Pastuchov Signed-off-by: Grzegorz Prajsner Signed-off-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index fb548d6a0aae..71387811b281 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -1208,7 +1208,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req) ret = rtrs_map_sg_fr(req, count); if (ret < 0) { rtrs_err_rl(s, - "Read request failed, failed to map fast reg. data, err: %d\n", + "Read request failed, failed to map fast reg. data, err: %d\n", ret); ib_dma_unmap_sg(dev->ib_dev, req->sglist, req->sg_cnt, req->dir);