From patchwork Wed Feb 14 10:36:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 10218371 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A7AC1602CB for ; Wed, 14 Feb 2018 10:33:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A3EF728D30 for ; Wed, 14 Feb 2018 10:33:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 97E4428F5E; Wed, 14 Feb 2018 10:33:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 05D7928D30 for ; Wed, 14 Feb 2018 10:33:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S967177AbeBNKdY (ORCPT ); Wed, 14 Feb 2018 05:33:24 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:41750 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S967151AbeBNKdX (ORCPT ); Wed, 14 Feb 2018 05:33:23 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w1EAVmJM100909; Wed, 14 Feb 2018 10:33:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2017-10-26; bh=8dVZaxvLVJVxkqZO4/MLwK8ZLEY8ThQmMinTElqM+2s=; b=EXmcaoLhC7SdmvhGwujbeq8V6dfxdrfM/qsPTQarMCxl+7VHk1Mh2elYIqHnDlNh5IpV CyKWV34olWC03cEIWOFPQfQciEAB+Ed8gYndWjXWYYGx1M9+Gfealkn/on03Xj9idXoy kQMJfzRdDHzUuZ5XdGMSnQ/o514fiNZiAcKlDxZ5fLT7pB/r5X9thWRtb0LzaK19AjBg 1mtAcsWSGMvsaCUpMbQ/ozFzUYJhiA5981SingACw4PHwZHOfBDFfec3HPLB/VI3bICl 3Z8QWv+IXnli08jwV25nr+/e9oemArkgBgZA+4Lglpci7oYXu3acsIVNlb8Wuq0nxLXk 6w== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2120.oracle.com with ESMTP id 2g4jw3860f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Feb 2018 10:33:17 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w1EAXHlv012142 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 14 Feb 2018 10:33:17 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w1EAXH0i009234; Wed, 14 Feb 2018 10:33:17 GMT Received: from office-bj2017.cn.oracle.com (/10.182.69.78) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 14 Feb 2018 02:33:16 -0800 From: Zhu Yanjun To: monis@mellanox.com, dledford@redhat.com, jgg@ziepe.ca, linux-rdma@vger.kernel.org, yuval.shaia@oracle.com Subject: [PATCHv3 1/1] IB/rxe: remove unnecessary skb_clone Date: Wed, 14 Feb 2018 05:36:40 -0500 Message-Id: <1518604600-4425-1-git-send-email-yanjun.zhu@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <20180213112124.GA6991@yuvallap> References: <20180213112124.GA6991@yuvallap> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8804 signatures=668670 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1802140125 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In send_atomic_ack function, it is not necessary to make a skb_clone. To gain better performance (high throughput and low latency), this skb_clone is removed. The following tests are made. server client --------- --------- |1.1.1.1|<----rxe-channel--->|1.1.1.2| --------- --------- On server: rping -s -a 1.1.1.1 -v -C 1000 -S 512 On client: rping -c -a 1.1.1.1 -v -C 1000 -S 512 The kernel config CONFIG_DEBUG_KMEMLEAK is enabled on both server and client. This test runs for several hours. There is no memory leak and the whole system can work well. As the above network, the following tests are made. Server: ibv_rc_pingpong -d rxe0 -g 1 Client: ibv_rc_pingpong -d rxe0 -g 1 1.1.1.1 The result on Server(10 tests are made). Before: Throughput is 137.07 Mbit/sec Latency is 517.76 usec/iter After: Throughput is 148.85 Mbit/sec Latency is 476.64 usec/iter The throughput is enhanced and the latency is reduced. CC: Srinivas Eeda CC: Junxiao Bi Signed-off-by: Zhu Yanjun --- V2-->V3: Fix typo errors. V1-->V2: 10 tests are made. From throughput and latency, the performance is better. --- drivers/infiniband/sw/rxe/rxe_resp.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index d37bb9b..2ebb07c 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -969,7 +969,6 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, int rc = 0; struct rxe_pkt_info ack_pkt; struct sk_buff *skb; - struct sk_buff *skb_copy; struct rxe_dev *rxe = to_rdev(qp->ibqp.device); struct resp_res *res; @@ -981,15 +980,6 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, goto out; } - skb_copy = skb_clone(skb, GFP_ATOMIC); - if (skb_copy) - rxe_add_ref(qp); /* for the new SKB */ - else { - pr_warn("Could not clone atomic response\n"); - rc = -ENOMEM; - goto out; - } - res = &qp->resp.resources[qp->resp.res_head]; free_rd_atomic_resource(qp, res); rxe_advance_resp_resource(qp); @@ -998,18 +988,16 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, memset((unsigned char *)SKB_TO_PKT(skb) + sizeof(ack_pkt), 0, sizeof(skb->cb) - sizeof(ack_pkt)); + refcount_inc(&skb->users); res->type = RXE_ATOMIC_MASK; res->atomic.skb = skb; res->first_psn = ack_pkt.psn; res->last_psn = ack_pkt.psn; res->cur_psn = ack_pkt.psn; - rc = rxe_xmit_packet(rxe, qp, &ack_pkt, skb_copy); - if (rc) { + rc = rxe_xmit_packet(rxe, qp, &ack_pkt, skb); + if (rc) pr_err_ratelimited("Failed sending ack\n"); - rxe_drop_ref(qp); - kfree_skb(skb_copy); - } out: return rc;