From patchwork Thu Jul 14 09:44:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 12917620 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1811CCA47B for ; Thu, 14 Jul 2022 09:45:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238243AbiGNJpC (ORCPT ); Thu, 14 Jul 2022 05:45:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238429AbiGNJoq (ORCPT ); Thu, 14 Jul 2022 05:44:46 -0400 Received: from out199-5.us.a.mail.aliyun.com (out199-5.us.a.mail.aliyun.com [47.90.199.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABBD72125F; Thu, 14 Jul 2022 02:44:43 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R601e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0VJIcptN_1657791854; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VJIcptN_1657791854) by smtp.aliyun-inc.com; Thu, 14 Jul 2022 17:44:38 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v2 1/6] net/smc: remove redundant dma sync ops Date: Thu, 14 Jul 2022 17:44:00 +0800 Message-Id: <1657791845-1060-2-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1657791845-1060-1-git-send-email-guwen@linux.alibaba.com> References: <1657791845-1060-1-git-send-email-guwen@linux.alibaba.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Guangguan Wang smc_ib_sync_sg_for_cpu/device are the ops used for dma memory cache consistency. Smc sndbufs are dma buffers, where CPU writes data to it and PCIE device reads data from it. So for sndbufs, smc_ib_sync_sg_for_device is needed and smc_ib_sync_sg_for_cpu is redundant as PCIE device will not write the buffers. Smc rmbs are dma buffers, where PCIE device write data to it and CPU read data from it. So for rmbs, smc_ib_sync_sg_for_cpu is needed and smc_ib_sync_sg_for_device is redundant as CPU will not write the buffers. Signed-off-by: Guangguan Wang --- net/smc/af_smc.c | 2 -- net/smc/smc_core.c | 22 ---------------------- net/smc/smc_core.h | 2 -- net/smc/smc_rx.c | 2 -- net/smc/smc_tx.c | 1 - 5 files changed, 29 deletions(-) diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 433bb5a7..9497a3b 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1226,7 +1226,6 @@ static int smc_connect_rdma(struct smc_sock *smc, goto connect_abort; } } - smc_rmb_sync_sg_for_device(&smc->conn); if (aclc->hdr.version > SMC_V1) { struct smc_clc_msg_accept_confirm_v2 *clc_v2 = @@ -2113,7 +2112,6 @@ static int smc_listen_rdma_reg(struct smc_sock *new_smc, bool local_first) if (smcr_lgr_reg_rmbs(conn->lnk, conn->rmb_desc)) return SMC_CLC_DECL_ERR_REGRMB; } - smc_rmb_sync_sg_for_device(&new_smc->conn); return 0; } diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index f40f6ed..1faa0cb 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -2290,14 +2290,6 @@ static int __smc_buf_create(struct smc_sock *smc, bool is_smcd, bool is_rmb) return 0; } -void smc_sndbuf_sync_sg_for_cpu(struct smc_connection *conn) -{ - if (!smc_conn_lgr_valid(conn) || conn->lgr->is_smcd || - !smc_link_active(conn->lnk)) - return; - smc_ib_sync_sg_for_cpu(conn->lnk, conn->sndbuf_desc, DMA_TO_DEVICE); -} - void smc_sndbuf_sync_sg_for_device(struct smc_connection *conn) { if (!smc_conn_lgr_valid(conn) || conn->lgr->is_smcd || @@ -2320,20 +2312,6 @@ void smc_rmb_sync_sg_for_cpu(struct smc_connection *conn) } } -void smc_rmb_sync_sg_for_device(struct smc_connection *conn) -{ - int i; - - if (!smc_conn_lgr_valid(conn) || conn->lgr->is_smcd) - return; - for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) { - if (!smc_link_active(&conn->lgr->lnk[i])) - continue; - smc_ib_sync_sg_for_device(&conn->lgr->lnk[i], conn->rmb_desc, - DMA_FROM_DEVICE); - } -} - /* create the send and receive buffer for an SMC socket; * receive buffers are called RMBs; * (even though the SMC protocol allows more than one RMB-element per RMB, diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h index 4cb03e9..c441dfe 100644 --- a/net/smc/smc_core.h +++ b/net/smc/smc_core.h @@ -513,10 +513,8 @@ void smc_rtoken_set(struct smc_link_group *lgr, int link_idx, int link_idx_new, __be32 nw_rkey_known, __be64 nw_vaddr, __be32 nw_rkey); void smc_rtoken_set2(struct smc_link_group *lgr, int rtok_idx, int link_id, __be64 nw_vaddr, __be32 nw_rkey); -void smc_sndbuf_sync_sg_for_cpu(struct smc_connection *conn); void smc_sndbuf_sync_sg_for_device(struct smc_connection *conn); void smc_rmb_sync_sg_for_cpu(struct smc_connection *conn); -void smc_rmb_sync_sg_for_device(struct smc_connection *conn); int smc_vlan_by_tcpsk(struct socket *clcsock, struct smc_init_info *ini); void smc_conn_free(struct smc_connection *conn); diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c index 338b9ef..00ad004 100644 --- a/net/smc/smc_rx.c +++ b/net/smc/smc_rx.c @@ -413,7 +413,6 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg, if (rc < 0) { if (!read_done) read_done = -EFAULT; - smc_rmb_sync_sg_for_device(conn); goto out; } } @@ -427,7 +426,6 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg, chunk_len_sum += chunk_len; chunk_off = 0; /* modulo offset in recv ring buffer */ } - smc_rmb_sync_sg_for_device(conn); /* update cursors */ if (!(flags & MSG_PEEK)) { diff --git a/net/smc/smc_tx.c b/net/smc/smc_tx.c index 805a546..ca0d5f5 100644 --- a/net/smc/smc_tx.c +++ b/net/smc/smc_tx.c @@ -246,7 +246,6 @@ int smc_tx_sendmsg(struct smc_sock *smc, struct msghdr *msg, size_t len) tx_cnt_prep); chunk_len_sum = chunk_len; chunk_off = tx_cnt_prep; - smc_sndbuf_sync_sg_for_cpu(conn); for (chunk = 0; chunk < 2; chunk++) { rc = memcpy_from_msg(sndbuf_base + chunk_off, msg, chunk_len);