From patchwork Thu Apr 4 21:12:20 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roland Dreier X-Patchwork-Id: 2395111 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 3186BDF25A for ; Thu, 4 Apr 2013 21:12:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1763613Ab3DDVM0 (ORCPT ); Thu, 4 Apr 2013 17:12:26 -0400 Received: from na3sys010aog102.obsmtp.com ([74.125.245.72]:43997 "HELO na3sys010aog102.obsmtp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1763239Ab3DDVMZ (ORCPT ); Thu, 4 Apr 2013 17:12:25 -0400 Received: from mail-pa0-f71.google.com ([209.85.220.71]) (using TLSv1) by na3sys010aob102.postini.com ([74.125.244.12]) with SMTP ID DSNKUV3sud0tRF3a+ZRmhOIRokYsyEPmOxQ9@postini.com; Thu, 04 Apr 2013 14:12:25 PDT Received: by mail-pa0-f71.google.com with SMTP id fb10so2875539pad.2 for ; Thu, 04 Apr 2013 14:12:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google; h=x-received:x-received:sender:from:to:cc:subject:date:message-id :x-mailer; bh=wEdzSm0s86WDhDbuyWmpZ3rOxxz1dEgY+y2n6R/wJr8=; b=EoFR3ctAQ3gPCKALRAeJ+TfAHfsInDaPATTEZKm9b648i6HcmOHtSUdaTrVmJ10vW4 KIty23VWwemOjwk3rTJsaRT5pemIIEr+oKGL92hzinalTO54iQSacmJjn5JuVmXGkGk9 N84/jqbkp4O/CDPb7s00HpUo/aXfMYAlt6WkA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:x-received:sender:from:to:cc:subject:date:message-id :x-mailer:x-gm-message-state; bh=wEdzSm0s86WDhDbuyWmpZ3rOxxz1dEgY+y2n6R/wJr8=; b=kw/OZKvKEQkQWWaRgWyXfNrf+EyFQEM52XISi2Xq9vcxGljjX3iW479JQZE8UwWQSV OuE7ioQ6Mq6nLiq/HQvVFNcFrafQFFrqxXTwPq07JMKUPtthXvY9Zzv+onwLjxFU4i26 gVwuGt4y45jBXwbsIMdXyNyS6rhOljrIICT3RRSC7uKazeMDJ80DfF4Z93JaGOQ2vIbh 3FFdrqRZ/0cJpFgxCdBpZZ+W4wspRud8ftqeGa7f/iI/KAyfeJFvieTuPxxC+RdUhDPt rwQ127KYlElOXNM3PksDRWwojGrOmU5Z3VbsvyJbR/lu6LHAxgcXzn/AQzuRpa65woE0 8Gxg== X-Received: by 10.68.231.164 with SMTP id th4mr10621188pbc.198.1365109944791; Thu, 04 Apr 2013 14:12:24 -0700 (PDT) X-Received: by 10.68.231.164 with SMTP id th4mr10621179pbc.198.1365109944679; Thu, 04 Apr 2013 14:12:24 -0700 (PDT) Received: from roland-t410s.purestorage.com ([216.200.155.2]) by mx.google.com with ESMTPS id fn8sm604226pab.19.2013.04.04.14.12.23 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 04 Apr 2013 14:12:23 -0700 (PDT) From: Roland Dreier To: linux-rdma@vger.kernel.org, Markus Stockhausen Cc: Eric Dumazet Subject: [RFC/PATCH v2] IPoIB: Leave space in skb linear buffer for IP headers Date: Thu, 4 Apr 2013 14:12:20 -0700 Message-Id: <1365109940-22916-1-git-send-email-roland@kernel.org> X-Mailer: git-send-email 1.8.1.2 X-Gm-Message-State: ALoCoQmIF63fNb8O3NB4F2RkWxvU3nBnusXVOepWTlwNX/xejMPP11BcwW2FIokZpgShcUFmMOTLyi/dJV2XYLjpuScBXnHINuIH6bXfSpICWaCjWFGaj97iow2HWjTQijC5q2tjTmeYYqH8ihZIGuNdRZjdOEPGSAAtNzrqBci2Hrf6zq3S1hM= Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Roland Dreier Markus Stockhausen noticed that IPoIB was spending significant time doing memcpy() in __pskb_pull_tail(). He found that this is because his adapter reports a maximum MTU of 4K, which causes IPoIB datagram mode to receive all the actual data in a separate page in the fragment list. We're already allocating extra tailroom for the skb linear part, so we might as well use it. Cc: Eric Dumazet Reported-by: Markus Stockhausen Signed-off-by: Roland Dreier --- v2: Try to handle the case where we get all the data in the linear part of the skb and don't need the frag part at all. drivers/infiniband/ulp/ipoib/ipoib.h | 3 ++- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 17 ++++++----------- 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index eb71aaa..ab2cc4c 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -64,7 +64,8 @@ enum ipoib_flush_level { enum { IPOIB_ENCAP_LEN = 4, - IPOIB_UD_HEAD_SIZE = IB_GRH_BYTES + IPOIB_ENCAP_LEN, + /* add 128 bytes of tailroom for IP/TCP headers */ + IPOIB_UD_HEAD_SIZE = IB_GRH_BYTES + IPOIB_ENCAP_LEN + 128, IPOIB_UD_RX_SG = 2, /* max buffer needed for 4K mtu */ IPOIB_CM_MTU = 0x10000 - 0x10, /* padding to align header to 16 */ diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 2cfa76f..ecf4faf 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -109,11 +109,12 @@ static void ipoib_ud_skb_put_frags(struct ipoib_dev_priv *priv, struct sk_buff *skb, unsigned int length) { - if (ipoib_ud_need_sg(priv->max_ib_mtu)) { + if (ipoib_ud_need_sg(priv->max_ib_mtu) && + length > IPOIB_UD_HEAD_SIZE) { skb_frag_t *frag = &skb_shinfo(skb)->frags[0]; unsigned int size; /* - * There is only two buffers needed for max_payload = 4K, + * There are only two buffers needed for max_payload = 4K, * first buf size is IPOIB_UD_HEAD_SIZE */ skb->tail += IPOIB_UD_HEAD_SIZE; @@ -156,18 +157,12 @@ static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id) struct ipoib_dev_priv *priv = netdev_priv(dev); struct sk_buff *skb; int buf_size; - int tailroom; u64 *mapping; - if (ipoib_ud_need_sg(priv->max_ib_mtu)) { - buf_size = IPOIB_UD_HEAD_SIZE; - tailroom = 128; /* reserve some tailroom for IP/TCP headers */ - } else { - buf_size = IPOIB_UD_BUF_SIZE(priv->max_ib_mtu); - tailroom = 0; - } + buf_size = ipoib_ud_need_sg(priv->max_ib_mtu) ? + IPOIB_UD_HEAD_SIZE : IPOIB_UD_BUF_SIZE(priv->max_ib_mtu); - skb = dev_alloc_skb(buf_size + tailroom + 4); + skb = dev_alloc_skb(buf_size + 4); if (unlikely(!skb)) return NULL;