From patchwork Thu Sep 15 16:23:15 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shaohua Li X-Patchwork-Id: 9334189 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 423236089F for ; Thu, 15 Sep 2016 16:24:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37AB429A4C for ; Thu, 15 Sep 2016 16:24:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 294C029A4D; Thu, 15 Sep 2016 16:24:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8B59E29A47 for ; Thu, 15 Sep 2016 16:23:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755091AbcIOQXe (ORCPT ); Thu, 15 Sep 2016 12:23:34 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:33483 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754868AbcIOQXW (ORCPT ); Thu, 15 Sep 2016 12:23:22 -0400 Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id u8FGFCYk022570 for ; Thu, 15 Sep 2016 09:23:22 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=N79mEtKQcr4ROmM2RBXatyPaEq6WpdK3lNPVOkGNps8=; b=fdNhzJ6onFoBvNUtLg+R5hj2vDY+QJZOniDMixj0cEW7OyotcaL4pjpcvBw9vIg+QJ2+ ce2Hcy+ZKT4H5qomAcuSyxtkKqMZD9LLSNxgUL3x9B4Ns7jWugxb66j6Plgju6QuNmiu IzDF0/nsrX/R3xk45lzYQ3AcZpAaPepgxLM= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 25fww30vgx-4 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 15 Sep 2016 09:23:22 -0700 Received: from mx-out.facebook.com (192.168.52.123) by PRN-CHUB12.TheFacebook.com (192.168.16.22) with Microsoft SMTP Server (TLS) id 14.3.294.0; Thu, 15 Sep 2016 09:23:20 -0700 Received: from facebook.com (2401:db00:11:d0a2:face:0:39:0) by mx-out.facebook.com (10.223.100.97) with ESMTP id b6fb9e4a7b6011e6a8e524be0593f280-f85fba50 for ; Thu, 15 Sep 2016 09:23:20 -0700 Received: by devbig084.prn1.facebook.com (Postfix, from userid 11222) id AB5744800720; Thu, 15 Sep 2016 09:23:18 -0700 (PDT) From: Shaohua Li To: , CC: , , , , Subject: [PATCH V2 08/11] blk-throttle: detect completed idle cgroup Date: Thu, 15 Sep 2016 09:23:15 -0700 Message-ID: <9519a7a7ff79d702b9a4d6591734e17b4e39f92f.1473953743.git.shli@fb.com> X-Mailer: git-send-email 2.8.0.rc2 In-Reply-To: References: X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-09-15_08:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP cgroup could be assigned a limit, but doesn't dispatch enough IO, eg the cgroup is idle. When this happens, the cgroup doesn't hit its limit, so we can't move the state machine to higher level and all cgroups will be throttled to thier lower limit, so we waste bandwidth. Detecting idle cgroup is hard. This patch handles a simple case, a cgroup doesn't dispatch any IO. We ignore such cgroup's limit, so other cgroups can use the bandwidth. Signed-off-by: Shaohua Li --- block/blk-throttle.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index bc94086..e4713b1 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -144,6 +144,8 @@ struct throtl_grp { unsigned long last_check_time; + unsigned long last_dispatch_time[2]; + /* When did we start a new slice */ unsigned long slice_start[2]; unsigned long slice_end[2]; @@ -438,11 +440,14 @@ static void tg_update_has_rules(struct throtl_grp *tg) static void throtl_pd_online(struct blkg_policy_data *pd) { + struct throtl_grp *tg = pd_to_tg(pd); /* * We don't want new groups to escape the limits of its ancestors. * Update has_rules[] after a new group is brought online. */ - tg_update_has_rules(pd_to_tg(pd)); + tg_update_has_rules(tg); + tg->last_dispatch_time[READ] = jiffies; + tg->last_dispatch_time[WRITE] = jiffies; } static void blk_throtl_update_valid_limit(struct throtl_data *td) @@ -1604,6 +1609,12 @@ static bool throtl_upgrade_check_one(struct throtl_grp *tg) if (write_limit && sq->nr_queued[WRITE] && (!read_limit || sq->nr_queued[READ])) return true; + + if (time_after_eq(jiffies, + tg->last_dispatch_time[READ] + tg->td->throtl_slice) && + time_after_eq(jiffies, + tg->last_dispatch_time[WRITE] + tg->td->throtl_slice)) + return true; return false; } @@ -1684,6 +1695,9 @@ static bool throtl_downgrade_check_one(struct throtl_grp *tg) struct throtl_data *td = tg->td; unsigned long now = jiffies; + if (time_after_eq(now, tg->last_dispatch_time[READ] + td->throtl_slice) && + time_after_eq(now, tg->last_dispatch_time[WRITE] + td->throtl_slice)) + return false; /* * If cgroup is below high limit, consider downgrade and throttle other * cgroups @@ -1802,6 +1816,7 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, again: while (true) { + tg->last_dispatch_time[rw] = jiffies; if (tg->last_high_overflow_time[rw] == 0) tg->last_high_overflow_time[rw] = jiffies; throtl_downgrade_check(tg);