From patchwork Thu Feb 5 13:44:01 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Kazior X-Patchwork-Id: 5784701 Return-Path: X-Original-To: patchwork-linux-wireless@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id CB5389F336 for ; Thu, 5 Feb 2015 13:44:15 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id EDBB92021F for ; Thu, 5 Feb 2015 13:44:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 06B402011E for ; Thu, 5 Feb 2015 13:44:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757696AbbBENoG (ORCPT ); Thu, 5 Feb 2015 08:44:06 -0500 Received: from mail-wi0-f181.google.com ([209.85.212.181]:47137 "EHLO mail-wi0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757689AbbBENoD convert rfc822-to-8bit (ORCPT ); Thu, 5 Feb 2015 08:44:03 -0500 Received: by mail-wi0-f181.google.com with SMTP id fb4so10634308wid.2 for ; Thu, 05 Feb 2015 05:44:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tieto.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=zRIpI44gPYJBW4bE95i89ICbJzWwrMLq8XNOX2tIhHo=; b=VYGPeMIlcbrwfLDuTH67yQj3qRspfbbZxrcIddlhHIXUKOvLBheQjkuKBDTedFwasO NhBaeQoUUt7FfbiEolhxunnI9S8EaEGhsShPxL24UoUEhcoTKJtApu16Y1cE7RJWBcfb Tl/SQKQIPXx4+ncw5qpDT9w/nCq/Wr110Z//o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=zRIpI44gPYJBW4bE95i89ICbJzWwrMLq8XNOX2tIhHo=; b=QBJNkt+K7HBOpUjUiQgxN316/3lo5KMLrmnvkXmo9+6+Es+hpAoc2K1Ty+eaLblcPm Cr1EPbKGoYlZgqLBTiMWV64dyUXg6neYxsE79fQCZbNFHKpMzsbmvUp3THhQTFuaznXT 9jkZGTJNVKfttcoVkx28+6gLgn0SJPEOyLwhL0NJepS0qykMTOovjBpJjDJGxxvdyG6d Z+JJbHfGQ4KBq0MdCKH6k2+6KNSViT1dcOjLuAJBZ8XjFgCUy+RfLj5U2t57lo/wLe16 RwMkDH0UxnGWY0JpmzK1KfQYykh2M60oVodcFM4ar7GxdFW5TK2Tt/rzTmwvBcgNMxMf aaxA== X-Gm-Message-State: ALoCoQmDfayLYDZzd5J3Xz86XQSF3NiW++1fOWgISYMVDVMVZWAN6ZLZqhah9KF15IlKbxSETY+dg12byqazPTnO+dGRA2g/xGWB7EbMwjbF1vveM8XFHFg4lAzTKx3iNilvYYHemo9K MIME-Version: 1.0 X-Received: by 10.194.108.41 with SMTP id hh9mr8012832wjb.25.1423143841251; Thu, 05 Feb 2015 05:44:01 -0800 (PST) Received: by 10.27.101.138 with HTTP; Thu, 5 Feb 2015 05:44:01 -0800 (PST) In-Reply-To: <1423142342.31870.49.camel@edumazet-glaptop2.roam.corp.google.com> References: <1422537297.21689.15.camel@edumazet-glaptop2.roam.corp.google.com> <1422628835.21689.95.camel@edumazet-glaptop2.roam.corp.google.com> <1422903136.21689.114.camel@edumazet-glaptop2.roam.corp.google.com> <1422926330.21689.138.camel@edumazet-glaptop2.roam.corp.google.com> <1422973660.907.10.camel@edumazet-glaptop2.roam.corp.google.com> <1423051045.907.108.camel@edumazet-glaptop2.roam.corp.google.com> <1423053531.907.115.camel@edumazet-glaptop2.roam.corp.google.com> <1423055810.907.125.camel@edumazet-glaptop2.roam.corp.google.com> <1423056591.907.130.camel@edumazet-glaptop2.roam.corp.google.com> <1423084303.31870.15.camel@edumazet-glaptop2.roam.corp.google.com> <1423141038.31870.38.camel@edumazet-glaptop2.roam.corp.google.com> <1423142342.31870.49.camel@edumazet-glaptop2.roam.corp.google.com> Date: Thu, 5 Feb 2015 14:44:01 +0100 Message-ID: Subject: Re: Throughput regression with `tcp: refine TSO autosizing` From: Michal Kazior To: Eric Dumazet Cc: Neal Cardwell , linux-wireless , Network Development , eyalpe@dev.mellanox.co.il X-DomainID: tieto.com Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 5 February 2015 at 14:19, Eric Dumazet wrote: > On Thu, 2015-02-05 at 04:57 -0800, Eric Dumazet wrote: > >> The intention is to control the queues to the following : >> >> 1 ms of buffering, but limited to a configurable value. >> >> On a 40Gbps flow, 1ms represents 5 MB, which is insane. >> >> We do not want to queue 5 MB of traffic, this would destroy latencies >> for all concurrent flows. (Or would require having fq_codel or fq as >> packet schedulers, instead of default pfifo_fast) >> >> This is why having 1.5 ms delay between the transmit and TX completion >> is a problem in your case. I do get your point. But 1.5ms is really tough on Wi-Fi. Just look at this: ; ping 192.168.1.2 -c 3 PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. 64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=1.83 ms 64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=2.02 ms 64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=1.98 ms ; ping 192.168.1.2 -c 3 -Q 224 PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. 64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=0.939 ms 64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=0.906 ms 64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=0.946 ms This was run with no load so batching code in the driver itself should have no measurable effect. The channel was near-ideal: low noise floor, cabled rf, no other traffic. The lower latency ping is when 802.11 QoS Voice Access Category is used. I also get 400mbps instead of 250mbps in this case with 5 flows (net/master). Dealing with black box firmware blobs is a pain. > Note that TCP stack could detect when this happens, *if* ACK where > delivered before the TX completions, or when TX completion happens, > we could detect that the clone of the freed packet was freed. > > In my test, when I did "ethtool -C eth0 tx-usecs 1024 tx-frames 64", and > disabling GSO, TCP stack sends a bunch of packets (a bit less than 64), > blocks on tcp_limit_output_bytes. > > Then we receive 2 stretch ACKS after ~50 usec. > > TCP stack tries to push again some packets but blocks on > tcp_limit_output_bytes again. > > 1ms later, TX completion happens, tcp_wfree() is called, and TCP stack > push following ~60 packets. > > > TCP could eventually dynamically adjust the tcp_limit_output_bytes, > using a per flow dynamic value, but I would rather not add a kludge in > TCP stack only to deal with a possible bug in ath10k driver. > > niu has a similar issue and simply had to call skb_orphan() : > > drivers/net/ethernet/sun/niu.c:6669: skb_orphan(skb); Ok. I tried calling skb_orphan() right after I submit each Tx frame (similar to niu which does this in start_xmit): err_unmap_msdu: Now, with {net/master + ath10k GRO + the above} I get 620mbps on a single flow (even better then before). Wow. Does this look ok/safe as a solution to you? Micha? --- To unsubscribe from this list: send the line "unsubscribe linux-wireless" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html --- a/drivers/net/wireless/ath/ath10k/htt_tx.c +++ b/drivers/net/wireless/ath/ath10k/htt_tx.c @@ -564,6 +564,8 @@ int ath10k_htt_tx(struct ath10k_htt *htt, struct sk_buff *msdu) if (res) goto err_unmap_msdu; + skb_orphan(msdu); + return 0;