From patchwork Sun May 1 19:55:53 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 8989271 X-Patchwork-Delegate: kvalo@adurom.com Return-Path: X-Original-To: patchwork-ath10k@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 7E5D3BF29F for ; Sun, 1 May 2016 19:56:34 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7CBA32024C for ; Sun, 1 May 2016 19:56:33 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9FA092011B for ; Sun, 1 May 2016 19:56:32 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1awxTW-0001tP-D0; Sun, 01 May 2016 19:56:18 +0000 Received: from mail-qg0-x236.google.com ([2607:f8b0:400d:c04::236]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1awxTU-0001rU-NL for ath10k@lists.infradead.org; Sun, 01 May 2016 19:56:17 +0000 Received: by mail-qg0-x236.google.com with SMTP id w36so8039216qge.3 for ; Sun, 01 May 2016 12:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=bHRmLln1Y21fjDrhF++IFWY5AYAvAmvPXIynMWcyGNg=; b=GaUaaeltZT/Dsyqwb49WMqr10WXjf+yvW6eIzrUa8ynWp0FCVy27iM2SYYNAxLWL/m QqzEkiS5d4s25jJHTThBw1toaikIS/9LOQwGTXTLrPSTDiY2xXbQx6C8j9EfyzZip2X7 CxxeU+Fc6aiqSGPD4H5X8i7BxmkYtXYmOiUt6FD65HMvLgaQMkRRI6IcOfHiwYUPZMKJ xo7us1WWZ9GUvIGOh1SutznEHD5hx1BRjxGQ9CXdMNgXz4rmKOmIf3y+R+suU/QK89Ok BgaSn8yjYOdDzEgnEpk0MdLQxdqfLAvNcOF6kxOnz55PDq6ZEIAN6FmkmAuzvDJLfBhV 4+VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=bHRmLln1Y21fjDrhF++IFWY5AYAvAmvPXIynMWcyGNg=; b=QS3JnB1xLpDQLvrtIziDMYEzshndUzcCZf6KpUKkdkZYMr1HxVs19onx85sCp9z/lh D1hp4P28UL3b29jI5zYcqMiU5ixe+JV8Imxj/wsO/z6pjX5EUyrNxQmZHmQ35TLvA9ob y55tQ3I+9xE4wQ9PIi4r5US+2+cKDb4xvbo+51uUl776/u8a04ORSKcKXb7DOkBRgI3c IFUYp7EpBYvh+X+++mLh+Fw0/VzJXmjpFCHLU0tUfzrKg3jnUGvYK/8EEsk/zLe7XFSy e2fKaaVyDSd/5xOsXzLJCRKWM8EcFNZMUl7tQCEQRE8eDFAGZ5MrA4HMZyoNEl+1zlBC QGqA== X-Gm-Message-State: AOPr4FU2rCT6OUDJZ/RgEPolllxVPfaBG5y3ndUXiWZ2+FJONlQma48xTvUX4KlBVxCx1g== X-Received: by 10.141.7.84 with SMTP id j81mr29278173qhd.29.1462132555651; Sun, 01 May 2016 12:55:55 -0700 (PDT) Received: from [100.96.106.16] ([100.96.106.16]) by smtp.googlemail.com with ESMTPSA id z64sm8126147qkb.29.2016.05.01.12.55.54 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 01 May 2016 12:55:54 -0700 (PDT) Message-ID: <1462132553.5535.207.camel@edumazet-glaptop3.roam.corp.google.com> Subject: Re: [Codel] fq_codel_drop vs a udp flood From: Eric Dumazet To: Jonathan Morton Date: Sun, 01 May 2016 12:55:53 -0700 In-Reply-To: <1462128385.5535.200.camel@edumazet-glaptop3.roam.corp.google.com> References: <1462125592.5535.194.camel@edumazet-glaptop3.roam.corp.google.com> <865DA393-262D-40B6-A9D3-1B978CD5F6C6@gmail.com> <1462128385.5535.200.camel@edumazet-glaptop3.roam.corp.google.com> X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160501_125616_883382_7473F118 X-CRM114-Status: GOOD ( 15.18 ) X-Spam-Score: -2.7 (--) X-BeenThere: ath10k@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: make-wifi-fast@lists.bufferbloat.net, "codel@lists.bufferbloat.net" , Dave Taht , ath10k Sender: "ath10k" Errors-To: ath10k-bounces+patchwork-ath10k=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Sun, 2016-05-01 at 11:46 -0700, Eric Dumazet wrote: > Just drop half backlog packets instead of 1, (maybe add a cap of 64 > packets to avoid too big burts of kfree_skb() which might add cpu > spikes) and problem is gone. > I used following patch and it indeed solved the issue in my tests. (Not the DDOS case, but when few fat flows are really bad citizens) diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c index a5e420b3d4ab..0cb8699624bc 100644 --- a/net/sched/sch_fq_codel.c +++ b/net/sched/sch_fq_codel.c @@ -135,11 +135,11 @@ static inline void flow_queue_add(struct fq_codel_flow *flow, skb->next = NULL; } -static unsigned int fq_codel_drop(struct Qdisc *sch) +static unsigned int fq_codel_drop(struct Qdisc *sch, unsigned int max) { struct fq_codel_sched_data *q = qdisc_priv(sch); struct sk_buff *skb; - unsigned int maxbacklog = 0, idx = 0, i, len; + unsigned int maxbacklog = 0, idx = 0, i, len = 0; struct fq_codel_flow *flow; /* Queue is full! Find the fat flow and drop packet from it. @@ -153,15 +153,26 @@ static unsigned int fq_codel_drop(struct Qdisc *sch) idx = i; } } + /* As the search was painful, drop half bytes of this fat flow. + * Limit to max packets to not inflict too big latencies, + * as kfree_skb() might be quite expensive. + */ + maxbacklog >>= 1; + flow = &q->flows[idx]; - skb = dequeue_head(flow); - len = qdisc_pkt_len(skb); + for (i = 0; i < max;) { + skb = dequeue_head(flow); + len += qdisc_pkt_len(skb); + kfree_skb(skb); + i++; + if (len >= maxbacklog) + break; + } + sch->qstats.drops += i; + sch->qstats.backlog -= len; q->backlogs[idx] -= len; - sch->q.qlen--; - qdisc_qstats_drop(sch); - qdisc_qstats_backlog_dec(sch, skb); - kfree_skb(skb); - flow->dropped++; + sch->q.qlen -= i; + flow->dropped += i; return idx; } @@ -170,14 +181,14 @@ static unsigned int fq_codel_qdisc_drop(struct Qdisc *sch) unsigned int prev_backlog; prev_backlog = sch->qstats.backlog; - fq_codel_drop(sch); + fq_codel_drop(sch, 1U); return prev_backlog - sch->qstats.backlog; } static int fq_codel_enqueue(struct sk_buff *skb, struct Qdisc *sch) { struct fq_codel_sched_data *q = qdisc_priv(sch); - unsigned int idx, prev_backlog; + unsigned int idx, prev_backlog, prev_qlen; struct fq_codel_flow *flow; int uninitialized_var(ret); @@ -206,16 +217,15 @@ static int fq_codel_enqueue(struct sk_buff *skb, struct Qdisc *sch) return NET_XMIT_SUCCESS; prev_backlog = sch->qstats.backlog; - q->drop_overlimit++; - /* Return Congestion Notification only if we dropped a packet - * from this flow. - */ - if (fq_codel_drop(sch) == idx) - return NET_XMIT_CN; + prev_qlen = sch->q.qlen; + ret = fq_codel_drop(sch, 64U); + q->drop_overlimit += prev_qlen - sch->q.qlen; + + /* As we dropped packet(s), better let upper stack know this */ + qdisc_tree_reduce_backlog(sch, prev_qlen - sch->q.qlen, + prev_backlog - sch->qstats.backlog); - /* As we dropped a packet, better let upper stack know this */ - qdisc_tree_reduce_backlog(sch, 1, prev_backlog - sch->qstats.backlog); - return NET_XMIT_SUCCESS; + return ret == idx ? NET_XMIT_CN : NET_XMIT_SUCCESS; } /* This is the specific function called from codel_dequeue()