From patchwork Thu Dec 15 20:33:00 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shaohua Li X-Patchwork-Id: 9476927 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 54156607EE for ; Thu, 15 Dec 2016 20:38:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 477EF28787 for ; Thu, 15 Dec 2016 20:38:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C1AF287CD; Thu, 15 Dec 2016 20:38:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E58C228787 for ; Thu, 15 Dec 2016 20:38:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756333AbcLOUin (ORCPT ); Thu, 15 Dec 2016 15:38:43 -0500 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:60167 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1756281AbcLOUdL (ORCPT ); Thu, 15 Dec 2016 15:33:11 -0500 Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.17/8.16.0.17) with SMTP id uBFKUVnp003250 for ; Thu, 15 Dec 2016 12:33:11 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=kygWANTuM4BgW0XOXFv6+tLmx2z8Q91f/PfAs2gY1UI=; b=MHg5iI825Uw5lL9W4KPciaw6rlqIo1+iP/IRwIyK04aQvSFUWsSiaQ/dznW+DB6aQ91J nkHsptrfhGHT6TN7fUBaOf/wB8r4KGwpmctkdtGHY2RLmESfQ48xMOpGaqpUA+w+fQh6 PHFd0NuZeyLfPP0T0oTUkuIR+xGdSmce1ZY= Received: from mail.thefacebook.com ([199.201.64.23]) by m0089730.ppops.net with ESMTP id 27bvqb475d-3 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 15 Dec 2016 12:33:11 -0800 Received: from mx-out.facebook.com (192.168.52.123) by PRN-CHUB14.TheFacebook.com (192.168.16.24) with Microsoft SMTP Server (TLS) id 14.3.294.0; Thu, 15 Dec 2016 12:33:09 -0800 Received: from facebook.com (2401:db00:21:603d:face:0:19:0) by mx-out.facebook.com (10.103.99.99) with ESMTP id b0ea7124c30511e699290002c9dfb610-2d9f5a50 for ; Thu, 15 Dec 2016 12:33:09 -0800 Received: by devbig638.prn2.facebook.com (Postfix, from userid 11222) id 15C934860757; Thu, 15 Dec 2016 12:33:08 -0800 (PST) Smtp-Origin-Hostprefix: devbig From: Shaohua Li Smtp-Origin-Hostname: devbig638.prn2.facebook.com To: , CC: , , , Smtp-Origin-Cluster: prn2c22 Subject: [PATCH V5 09/17] blk-throttle: detect completed idle cgroup Date: Thu, 15 Dec 2016 12:33:00 -0800 Message-ID: <9c3254d92f47a5172e83fc1e84c09fca8243a1e7.1481833017.git.shli@fb.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: References: X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-12-15_14:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP cgroup could be assigned a limit, but doesn't dispatch enough IO, eg the cgroup is idle. When this happens, the cgroup doesn't hit its limit, so we can't move the state machine to higher level and all cgroups will be throttled to their lower limit, so we waste bandwidth. Detecting idle cgroup is hard. This patch handles a simple case, a cgroup doesn't dispatch any IO. We ignore such cgroup's limit, so other cgroups can use the bandwidth. Signed-off-by: Shaohua Li --- block/blk-throttle.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index cd10c65..a0ba961 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -148,6 +148,8 @@ struct throtl_grp { unsigned long last_check_time; + unsigned long last_dispatch_time[2]; + /* When did we start a new slice */ unsigned long slice_start[2]; unsigned long slice_end[2]; @@ -437,11 +439,14 @@ static void tg_update_has_rules(struct throtl_grp *tg) static void throtl_pd_online(struct blkg_policy_data *pd) { + struct throtl_grp *tg = pd_to_tg(pd); /* * We don't want new groups to escape the limits of its ancestors. * Update has_rules[] after a new group is brought online. */ - tg_update_has_rules(pd_to_tg(pd)); + tg_update_has_rules(tg); + tg->last_dispatch_time[READ] = jiffies; + tg->last_dispatch_time[WRITE] = jiffies; } static void blk_throtl_update_valid_limit(struct throtl_data *td) @@ -1582,6 +1587,12 @@ static bool throtl_tg_can_upgrade(struct throtl_grp *tg) if (write_limit && sq->nr_queued[WRITE] && (!read_limit || sq->nr_queued[READ])) return true; + + if (time_after_eq(jiffies, + tg->last_dispatch_time[READ] + tg->td->throtl_slice) && + time_after_eq(jiffies, + tg->last_dispatch_time[WRITE] + tg->td->throtl_slice)) + return true; return false; } @@ -1660,6 +1671,11 @@ static bool throtl_tg_can_downgrade(struct throtl_grp *tg) struct throtl_data *td = tg->td; unsigned long now = jiffies; + if (time_after_eq(now, tg->last_dispatch_time[READ] + + td->throtl_slice) && + time_after_eq(now, tg->last_dispatch_time[WRITE] + + td->throtl_slice)) + return false; /* * If cgroup is below low limit, consider downgrade and throttle other * cgroups @@ -1769,6 +1785,7 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, again: while (true) { + tg->last_dispatch_time[rw] = jiffies; if (tg->last_low_overflow_time[rw] == 0) tg->last_low_overflow_time[rw] = jiffies; throtl_downgrade_check(tg);