From patchwork Sat Mar 5 09:11:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770284 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 611AEC433EF for ; Sat, 5 Mar 2022 08:56:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230009AbiCEI5d (ORCPT ); Sat, 5 Mar 2022 03:57:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229977AbiCEI5c (ORCPT ); Sat, 5 Mar 2022 03:57:32 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE600254A92; Sat, 5 Mar 2022 00:56:42 -0800 (PST) Received: from kwepemi100005.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4K9dpd5r8hzdcJs; Sat, 5 Mar 2022 16:55:21 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi100005.china.huawei.com (7.221.188.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:40 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:40 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 01/11] block, bfq: add new apis to iterate bfq entities Date: Sat, 5 Mar 2022 17:11:55 +0800 Message-ID: <20220305091205.4188398-2-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The old and the new apis are the same currently, prepare to count root group into 'num_groups_with_pending_reqs'. The old apis will be used to iterate with root group's entity, and the new apis will be used to iterate without root group's entity. Signed-off-by: Yu Kuai --- block/bfq-iosched.h | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 3b83e3d1c2e5..d703492714e2 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -1037,9 +1037,20 @@ extern struct blkcg_policy blkcg_policy_bfq; #define for_each_entity_safe(entity, parent) \ for (; entity && ({ parent = entity->parent; 1; }); entity = parent) +#define is_root_entity(entity) \ + (entity->sched_data == NULL) + +#define for_each_entity_not_root(entity) \ + for (; entity && !is_root_entity(entity); entity = entity->parent) + +#define for_each_entity_not_root_safe(entity, parent) \ + for (; entity && !is_root_entity(entity) && \ + ({ parent = entity->parent; 1; }); entity = parent) #else /* CONFIG_BFQ_GROUP_IOSCHED */ +#define is_root_entity(entity) (false) + /* - * Next two macros are fake loops when cgroups support is not + * Next four macros are fake loops when cgroups support is not * enabled. I fact, in such a case, there is only one level to go up * (to reach the root group). */ @@ -1048,6 +1059,12 @@ extern struct blkcg_policy blkcg_policy_bfq; #define for_each_entity_safe(entity, parent) \ for (parent = NULL; entity ; entity = parent) + +#define for_each_entity_not_root(entity) \ + for (; entity ; entity = NULL) + +#define for_each_entity_not_root_safe(entity, parent) \ + for (parent = NULL; entity ; entity = parent) #endif /* CONFIG_BFQ_GROUP_IOSCHED */ struct bfq_queue *bfq_entity_to_bfqq(struct bfq_entity *entity); From patchwork Sat Mar 5 09:11:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770286 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 065E0C433F5 for ; Sat, 5 Mar 2022 08:56:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230378AbiCEI5e (ORCPT ); Sat, 5 Mar 2022 03:57:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229986AbiCEI5d (ORCPT ); Sat, 5 Mar 2022 03:57:33 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C120E254A99; Sat, 5 Mar 2022 00:56:43 -0800 (PST) Received: from kwepemi100001.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4K9dpf4S64zdcNH; Sat, 5 Mar 2022 16:55:22 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi100001.china.huawei.com (7.221.188.215) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:41 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:40 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 02/11] block, bfq: apply news apis where root group is not expected Date: Sat, 5 Mar 2022 17:11:56 +0800 Message-ID: <20220305091205.4188398-3-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org 'entity->sched_data' is set to parent group's sched_data, thus it's NULL for root group. And for_each_entity() is used widely to access 'entity->sched_data', thus aplly news apis if root group is not expected. Prepare to count root group into 'num_groups_with_pending_reqs'. Signed-off-by: Yu Kuai --- block/bfq-iosched.c | 2 +- block/bfq-iosched.h | 22 ++++++++-------------- block/bfq-wf2q.c | 10 +++++----- 3 files changed, 14 insertions(+), 20 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 69ddf6b0f01d..3bc7a7686aad 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -4393,7 +4393,7 @@ void bfq_bfqq_expire(struct bfq_data *bfqd, * service with the same budget. */ entity = entity->parent; - for_each_entity(entity) + for_each_entity_not_root(entity) entity->service = 0; } diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index d703492714e2..ddd8eff5c272 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -1024,25 +1024,22 @@ extern struct blkcg_policy blkcg_policy_bfq; /* - interface of the internal hierarchical B-WF2Q+ scheduler - */ #ifdef CONFIG_BFQ_GROUP_IOSCHED -/* both next loops stop at one of the child entities of the root group */ +/* stop at one of the child entities of the root group */ #define for_each_entity(entity) \ for (; entity ; entity = entity->parent) -/* - * For each iteration, compute parent in advance, so as to be safe if - * entity is deallocated during the iteration. Such a deallocation may - * happen as a consequence of a bfq_put_queue that frees the bfq_queue - * containing entity. - */ -#define for_each_entity_safe(entity, parent) \ - for (; entity && ({ parent = entity->parent; 1; }); entity = parent) - #define is_root_entity(entity) \ (entity->sched_data == NULL) #define for_each_entity_not_root(entity) \ for (; entity && !is_root_entity(entity); entity = entity->parent) +/* + * For each iteration, compute parent in advance, so as to be safe if + * entity is deallocated during the iteration. Such a deallocation may + * happen as a consequence of a bfq_put_queue that frees the bfq_queue + * containing entity. + */ #define for_each_entity_not_root_safe(entity, parent) \ for (; entity && !is_root_entity(entity) && \ ({ parent = entity->parent; 1; }); entity = parent) @@ -1050,16 +1047,13 @@ extern struct blkcg_policy blkcg_policy_bfq; #define is_root_entity(entity) (false) /* - * Next four macros are fake loops when cgroups support is not + * Next three macros are fake loops when cgroups support is not * enabled. I fact, in such a case, there is only one level to go up * (to reach the root group). */ #define for_each_entity(entity) \ for (; entity ; entity = NULL) -#define for_each_entity_safe(entity, parent) \ - for (parent = NULL; entity ; entity = parent) - #define for_each_entity_not_root(entity) \ for (; entity ; entity = NULL) diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index f8eb340381cf..c4cb935a615a 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -815,7 +815,7 @@ void bfq_bfqq_served(struct bfq_queue *bfqq, int served) bfqq->service_from_wr += served; bfqq->service_from_backlogged += served; - for_each_entity(entity) { + for_each_entity_not_root(entity) { st = bfq_entity_service_tree(entity); entity->service += served; @@ -1201,7 +1201,7 @@ static void bfq_deactivate_entity(struct bfq_entity *entity, struct bfq_sched_data *sd; struct bfq_entity *parent = NULL; - for_each_entity_safe(entity, parent) { + for_each_entity_not_root_safe(entity, parent) { sd = entity->sched_data; if (!__bfq_deactivate_entity(entity, ins_into_idle_tree)) { @@ -1270,7 +1270,7 @@ static void bfq_deactivate_entity(struct bfq_entity *entity, * is not the case. */ entity = parent; - for_each_entity(entity) { + for_each_entity_not_root(entity) { /* * Invoke __bfq_requeue_entity on entity, even if * already active, to requeue/reposition it in the @@ -1570,7 +1570,7 @@ struct bfq_queue *bfq_get_next_queue(struct bfq_data *bfqd) * We can finally update all next-to-serve entities along the * path from the leaf entity just set in service to the root. */ - for_each_entity(entity) { + for_each_entity_not_root(entity) { struct bfq_sched_data *sd = entity->sched_data; if (!bfq_update_next_in_service(sd, NULL, false)) @@ -1597,7 +1597,7 @@ bool __bfq_bfqd_reset_in_service(struct bfq_data *bfqd) * execute the final step: reset in_service_entity along the * path from entity to the root. */ - for_each_entity(entity) + for_each_entity_not_root(entity) entity->sched_data->in_service_entity = NULL; /* From patchwork Sat Mar 5 09:11:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770288 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2C5CC43217 for ; Sat, 5 Mar 2022 08:56:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230440AbiCEI5f (ORCPT ); Sat, 5 Mar 2022 03:57:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230384AbiCEI5e (ORCPT ); Sat, 5 Mar 2022 03:57:34 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0145F68C0; Sat, 5 Mar 2022 00:56:44 -0800 (PST) Received: from kwepemi500021.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4K9dpf71rMzdhK0; Sat, 5 Mar 2022 16:55:22 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi500021.china.huawei.com (7.221.188.245) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Sat, 5 Mar 2022 16:56:42 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:41 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 03/11] block, bfq: cleanup for __bfq_activate_requeue_entity() Date: Sat, 5 Mar 2022 17:11:57 +0800 Message-ID: <20220305091205.4188398-4-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Remove the parameter 'sd', which can be access by 'entity'. Just to make the code a litter cleaner. Signed-off-by: Yu Kuai --- block/bfq-wf2q.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index c4cb935a615a..e30da27f356d 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -1082,12 +1082,12 @@ static void __bfq_requeue_entity(struct bfq_entity *entity) } static void __bfq_activate_requeue_entity(struct bfq_entity *entity, - struct bfq_sched_data *sd, bool non_blocking_wait_rq) { struct bfq_service_tree *st = bfq_entity_service_tree(entity); - if (sd->in_service_entity == entity || entity->tree == &st->active) + if (entity->sched_data->in_service_entity == entity || + entity->tree == &st->active) /* * in service or already queued on the active tree, * requeue or reposition @@ -1119,14 +1119,11 @@ static void bfq_activate_requeue_entity(struct bfq_entity *entity, bool non_blocking_wait_rq, bool requeue, bool expiration) { - struct bfq_sched_data *sd; - for_each_entity(entity) { - sd = entity->sched_data; - __bfq_activate_requeue_entity(entity, sd, non_blocking_wait_rq); + __bfq_activate_requeue_entity(entity, non_blocking_wait_rq); - if (!bfq_update_next_in_service(sd, entity, expiration) && - !requeue) + if (!bfq_update_next_in_service(entity->sched_data, entity, + expiration) && !requeue) break; } } From patchwork Sat Mar 5 09:11:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53A3CC433EF for ; Sat, 5 Mar 2022 08:56:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230413AbiCEI5f (ORCPT ); Sat, 5 Mar 2022 03:57:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230353AbiCEI5e (ORCPT ); Sat, 5 Mar 2022 03:57:34 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EFCAC254A92; Sat, 5 Mar 2022 00:56:44 -0800 (PST) Received: from kwepemi100013.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4K9dph0hR6zdcRc; Sat, 5 Mar 2022 16:55:24 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi100013.china.huawei.com (7.221.188.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:43 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:42 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 04/11] block, bfq: move the increasement of 'num_groups_with_pending_reqs' to it's caller Date: Sat, 5 Mar 2022 17:11:58 +0800 Message-ID: <20220305091205.4188398-5-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Root group is not in service tree, thus __bfq_activate_entity() is not needed for root_group. This will simplify counting root group into 'num_groups_with_pending_reqs'. Signed-off-by: Yu Kuai --- block/bfq-wf2q.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index e30da27f356d..17f1d2c5b8dc 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -218,6 +218,19 @@ static bool bfq_no_longer_next_in_service(struct bfq_entity *entity) return false; } +static void bfq_update_groups_with_pending_reqs(struct bfq_entity *entity) +{ + if (!bfq_entity_to_bfqq(entity) && /* bfq_group */ + !entity->in_groups_with_pending_reqs) { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + struct bfq_data *bfqd = bfqg->bfqd; + + entity->in_groups_with_pending_reqs = true; + bfqd->num_groups_with_pending_reqs++; + } +} + #else /* CONFIG_BFQ_GROUP_IOSCHED */ static bool bfq_update_parent_budget(struct bfq_entity *next_in_service) @@ -230,6 +243,10 @@ static bool bfq_no_longer_next_in_service(struct bfq_entity *entity) return true; } +static void bfq_update_groups_with_pending_reqs(struct bfq_entity *entity) +{ +} + #endif /* CONFIG_BFQ_GROUP_IOSCHED */ /* @@ -984,19 +1001,6 @@ static void __bfq_activate_entity(struct bfq_entity *entity, entity->on_st_or_in_serv = true; } -#ifdef CONFIG_BFQ_GROUP_IOSCHED - if (!bfq_entity_to_bfqq(entity)) { /* bfq_group */ - struct bfq_group *bfqg = - container_of(entity, struct bfq_group, entity); - struct bfq_data *bfqd = bfqg->bfqd; - - if (!entity->in_groups_with_pending_reqs) { - entity->in_groups_with_pending_reqs = true; - bfqd->num_groups_with_pending_reqs++; - } - } -#endif - bfq_update_fin_time_enqueue(entity, st, backshifted); } @@ -1120,6 +1124,7 @@ static void bfq_activate_requeue_entity(struct bfq_entity *entity, bool requeue, bool expiration) { for_each_entity(entity) { + bfq_update_groups_with_pending_reqs(entity); __bfq_activate_requeue_entity(entity, non_blocking_wait_rq); if (!bfq_update_next_in_service(entity->sched_data, entity, From patchwork Sat Mar 5 09:11:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95D84C433EF for ; Sat, 5 Mar 2022 08:56:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231240AbiCEI5n (ORCPT ); Sat, 5 Mar 2022 03:57:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230496AbiCEI5h (ORCPT ); Sat, 5 Mar 2022 03:57:37 -0500 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 029B0F68C0; Sat, 5 Mar 2022 00:56:46 -0800 (PST) Received: from kwepemi500024.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4K9dkm3B9Sz1GDhL; Sat, 5 Mar 2022 16:52:00 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi500024.china.huawei.com (7.221.188.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Sat, 5 Mar 2022 16:56:43 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:43 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 05/11] block, bfq: count root group into 'num_groups_with_pending_reqs' Date: Sat, 5 Mar 2022 17:11:59 +0800 Message-ID: <20220305091205.4188398-6-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Root group is not counted into 'num_groups_with_pending_reqs' because 'entity->parent' is set to NULL for child entities, thus for_each_entity() can't access root group. This patch set root_group's entity to 'entity->parent' for child entities, this way root_group will be counted because for_each_entity() can access root_group in bfq_activate_requeue_entity(), Signed-off-by: Yu Kuai --- block/bfq-cgroup.c | 6 +++--- block/bfq-iosched.h | 3 ++- block/bfq-wf2q.c | 5 +++++ 3 files changed, 10 insertions(+), 4 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 420eda2589c0..6cd65b5e790d 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -436,7 +436,7 @@ void bfq_init_entity(struct bfq_entity *entity, struct bfq_group *bfqg) */ bfqg_and_blkg_get(bfqg); } - entity->parent = bfqg->my_entity; /* NULL for root group */ + entity->parent = &bfqg->entity; entity->sched_data = &bfqg->sched_data; } @@ -581,7 +581,7 @@ static void bfq_group_set_parent(struct bfq_group *bfqg, struct bfq_entity *entity; entity = &bfqg->entity; - entity->parent = parent->my_entity; + entity->parent = &parent->entity; entity->sched_data = &parent->sched_data; } @@ -688,7 +688,7 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, else if (bfqd->last_bfqq_created == bfqq) bfqd->last_bfqq_created = NULL; - entity->parent = bfqg->my_entity; + entity->parent = &bfqg->entity; entity->sched_data = &bfqg->sched_data; /* pin down bfqg and its associated blkg */ bfqg_and_blkg_get(bfqg); diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index ddd8eff5c272..4530ab8b42ac 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -1024,13 +1024,14 @@ extern struct blkcg_policy blkcg_policy_bfq; /* - interface of the internal hierarchical B-WF2Q+ scheduler - */ #ifdef CONFIG_BFQ_GROUP_IOSCHED -/* stop at one of the child entities of the root group */ +/* stop at root group */ #define for_each_entity(entity) \ for (; entity ; entity = entity->parent) #define is_root_entity(entity) \ (entity->sched_data == NULL) +/* stop at one of the child entities of the root group */ #define for_each_entity_not_root(entity) \ for (; entity && !is_root_entity(entity); entity = entity->parent) diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index 17f1d2c5b8dc..138a2950b841 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -1125,6 +1125,11 @@ static void bfq_activate_requeue_entity(struct bfq_entity *entity, { for_each_entity(entity) { bfq_update_groups_with_pending_reqs(entity); + + /* root group is not in service tree */ + if (is_root_entity(entity)) + break; + __bfq_activate_requeue_entity(entity, non_blocking_wait_rq); if (!bfq_update_next_in_service(entity->sched_data, entity, From patchwork Sat Mar 5 09:12:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E85A7C433FE for ; Sat, 5 Mar 2022 08:56:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231133AbiCEI5i (ORCPT ); Sat, 5 Mar 2022 03:57:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230468AbiCEI5h (ORCPT ); Sat, 5 Mar 2022 03:57:37 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02A44254AA0; Sat, 5 Mar 2022 00:56:46 -0800 (PST) Received: from kwepemi500023.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4K9dkm6vXQzbc6y; Sat, 5 Mar 2022 16:52:00 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi500023.china.huawei.com (7.221.188.76) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Sat, 5 Mar 2022 16:56:44 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:43 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 06/11] block, bfq: do not idle if only one cgroup is activated Date: Sat, 5 Mar 2022 17:12:00 +0800 Message-ID: <20220305091205.4188398-7-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Now that root group is counted into 'num_groups_with_pending_reqs', 'num_groups_with_pending_reqs > 0' is always true in bfq_asymmetric_scenario(). Thus change the condition to 'num_groups_with_pending_reqs > 1', so it's consistent without counting root group. On the other hand, with the following patches to only count groups(not ancestors) with pending requests, sync io can be handled concurrently if only one group has pending requests. Signed-off-by: Yu Kuai --- block/bfq-iosched.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 3bc7a7686aad..07027dc9dc4c 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -812,7 +812,7 @@ bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq) * much easier to maintain the needed state: * 1) all active queues have the same weight, * 2) all active queues belong to the same I/O-priority class, - * 3) there are no active groups. + * 3) there are one active groups at most. * In particular, the last condition is always true if hierarchical * support or the cgroups interface are not enabled, thus no state * needs to be maintained in this case. @@ -844,7 +844,7 @@ static bool bfq_asymmetric_scenario(struct bfq_data *bfqd, return varied_queue_weights || multiple_classes_busy #ifdef CONFIG_BFQ_GROUP_IOSCHED - || bfqd->num_groups_with_pending_reqs > 0 + || bfqd->num_groups_with_pending_reqs > 1 #endif ; } From patchwork Sat Mar 5 09:12:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13204C4332F for ; Sat, 5 Mar 2022 08:56:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231267AbiCEI5n (ORCPT ); Sat, 5 Mar 2022 03:57:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230476AbiCEI5h (ORCPT ); Sat, 5 Mar 2022 03:57:37 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DA392634F2; Sat, 5 Mar 2022 00:56:47 -0800 (PST) Received: from kwepemi500025.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4K9dkn55PHzbc1t; Sat, 5 Mar 2022 16:52:01 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi500025.china.huawei.com (7.221.188.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Sat, 5 Mar 2022 16:56:45 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:44 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 07/11] block, bfq: only count parent bfqg when bfqq is activated Date: Sat, 5 Mar 2022 17:12:01 +0800 Message-ID: <20220305091205.4188398-8-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Currently, bfqg will be counted into 'num_groups_with_pending_reqs' once it's child cgroup is activated, even if the group doesn't have any pending requests itself. For example, if we issue sync io in cgroup /root/c1/c2, root, c1 and c2 will all be counted into 'num_groups_with_pending_reqs', which makes it impossible to handle requests concurrently. This patch doesn't count the group that doesn't have any pending requests while it's child group has pending requests. Signed-off-by: Yu Kuai --- block/bfq-wf2q.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index 138a2950b841..db066ae35a71 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -1123,13 +1123,7 @@ static void bfq_activate_requeue_entity(struct bfq_entity *entity, bool non_blocking_wait_rq, bool requeue, bool expiration) { - for_each_entity(entity) { - bfq_update_groups_with_pending_reqs(entity); - - /* root group is not in service tree */ - if (is_root_entity(entity)) - break; - + for_each_entity_not_root(entity) { __bfq_activate_requeue_entity(entity, non_blocking_wait_rq); if (!bfq_update_next_in_service(entity->sched_data, entity, @@ -1640,6 +1634,7 @@ void bfq_activate_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq) { struct bfq_entity *entity = &bfqq->entity; + bfq_update_groups_with_pending_reqs(bfqq->entity.parent); bfq_activate_requeue_entity(entity, bfq_bfqq_non_blocking_wait_rq(bfqq), false, false); bfq_clear_bfqq_non_blocking_wait_rq(bfqq); From patchwork Sat Mar 5 09:12:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770290 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CD74C433F5 for ; Sat, 5 Mar 2022 08:56:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231208AbiCEI5m (ORCPT ); Sat, 5 Mar 2022 03:57:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230498AbiCEI5h (ORCPT ); Sat, 5 Mar 2022 03:57:37 -0500 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D9C1269A45; Sat, 5 Mar 2022 00:56:48 -0800 (PST) Received: from kwepemi500022.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4K9dkp4lYVz1GDMs; Sat, 5 Mar 2022 16:52:02 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi500022.china.huawei.com (7.221.188.64) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Sat, 5 Mar 2022 16:56:46 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:45 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 08/11] block, bfq: record how many queues have pending requests in bfq_group Date: Sat, 5 Mar 2022 17:12:02 +0800 Message-ID: <20220305091205.4188398-9-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Prepare to decrease 'num_groups_with_pending_reqs' earlier. bfqq will be inserted to weights_tree when new io is inserted to it, and bfqq will be removed from weights_tree when all the requests are completed. Thus use weights_tree insertion and removal to track how many queues have pending requests. Signed-off-by: Yu Kuai --- block/bfq-cgroup.c | 1 + block/bfq-iosched.c | 15 +++++++++++++++ block/bfq-iosched.h | 1 + 3 files changed, 17 insertions(+) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 6cd65b5e790d..58acaf14a91d 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -557,6 +557,7 @@ static void bfq_pd_init(struct blkg_policy_data *pd) */ bfqg->bfqd = bfqd; bfqg->active_entities = 0; + bfqg->num_entities_with_pending_reqs = 0; bfqg->rq_pos_tree = RB_ROOT; } diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 07027dc9dc4c..2a48c40b4f02 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -928,6 +928,13 @@ void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, inc_counter: bfqq->weight_counter->num_active++; bfqq->ref++; + +#ifdef CONFIG_BFQ_GROUP_IOSCHED + if (!entity->in_groups_with_pending_reqs) { + entity->in_groups_with_pending_reqs = true; + bfqq_group(bfqq)->num_entities_with_pending_reqs++; + } +#endif } /* @@ -944,6 +951,14 @@ void __bfq_weights_tree_remove(struct bfq_data *bfqd, return; bfqq->weight_counter->num_active--; + +#ifdef CONFIG_BFQ_GROUP_IOSCHED + if (bfqq->entity.in_groups_with_pending_reqs) { + bfqq->entity.in_groups_with_pending_reqs = false; + bfqq_group(bfqq)->num_entities_with_pending_reqs--; + } +#endif + if (bfqq->weight_counter->num_active > 0) goto reset_entity_pointer; diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 4530ab8b42ac..5d904851519c 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -940,6 +940,7 @@ struct bfq_group { struct bfq_entity *my_entity; int active_entities; + int num_entities_with_pending_reqs; struct rb_root rq_pos_tree; From patchwork Sat Mar 5 09:12:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770292 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 826C2C433FE for ; Sat, 5 Mar 2022 08:56:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230353AbiCEI5q (ORCPT ); Sat, 5 Mar 2022 03:57:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231159AbiCEI5j (ORCPT ); Sat, 5 Mar 2022 03:57:39 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23BC4269A77; Sat, 5 Mar 2022 00:56:48 -0800 (PST) Received: from kwepemi500020.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4K9dpl6Jk5zdcJs; Sat, 5 Mar 2022 16:55:27 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi500020.china.huawei.com (7.221.188.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Sat, 5 Mar 2022 16:56:46 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:46 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 09/11] block, bfq: move forward __bfq_weights_tree_remove() Date: Sat, 5 Mar 2022 17:12:03 +0800 Message-ID: <20220305091205.4188398-10-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Prepare to decrease 'num_groups_with_pending_reqs' earlier. Signed-off-by: Yu Kuai --- block/bfq-iosched.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 2a48c40b4f02..f221e9cab4d0 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -979,6 +979,19 @@ void bfq_weights_tree_remove(struct bfq_data *bfqd, { struct bfq_entity *entity = bfqq->entity.parent; + /* + * grab a ref to prevent bfqq to be freed in + * __bfq_weights_tree_remove + */ + bfqq->ref++; + + /* + * remove bfqq from weights tree first, so that how many queues have + * pending requests in parent bfqg is updated. + */ + __bfq_weights_tree_remove(bfqd, bfqq, + &bfqd->queue_weights_tree); + for_each_entity(entity) { struct bfq_sched_data *sd = entity->my_sched_data; @@ -1013,14 +1026,7 @@ void bfq_weights_tree_remove(struct bfq_data *bfqd, } } - /* - * Next function is invoked last, because it causes bfqq to be - * freed if the following holds: bfqq is not in service and - * has no dispatched request. DO NOT use bfqq after the next - * function invocation. - */ - __bfq_weights_tree_remove(bfqd, bfqq, - &bfqd->queue_weights_tree); + bfq_put_queue(bfqq); } /* From patchwork Sat Mar 5 09:12:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABA94C433EF for ; Sat, 5 Mar 2022 08:57:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231354AbiCEI5r (ORCPT ); Sat, 5 Mar 2022 03:57:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231185AbiCEI5m (ORCPT ); Sat, 5 Mar 2022 03:57:42 -0500 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86080254A92; Sat, 5 Mar 2022 00:56:49 -0800 (PST) Received: from kwepemi500026.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4K9dkr1Nncz1GDbv; Sat, 5 Mar 2022 16:52:04 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi500026.china.huawei.com (7.221.188.247) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Sat, 5 Mar 2022 16:56:47 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:46 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 10/11] block, bfq: decrease 'num_groups_with_pending_reqs' earlier Date: Sat, 5 Mar 2022 17:12:04 +0800 Message-ID: <20220305091205.4188398-11-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Currently 'num_groups_with_pending_reqs' won't be decreased when the group doesn't have any pending requests, while some child group still have pending requests. The decrement is delayed to when all the child groups doesn't have any pending requests. For example: 1) t1 issue sync io on root group, t2 and t3 issue sync io on the same child group. num_groups_with_pending_reqs is 2 now. 2) t1 stopped, num_groups_with_pending_reqs is still 2. io from t2 and t3 still can't be handled concurrently. Fix the problem by decreasing 'num_groups_with_pending_reqs' immediately upon the weights_tree removal of last bfqq of the group. Signed-off-by: Yu Kuai --- block/bfq-iosched.c | 56 +++++++++++++++------------------------------ block/bfq-iosched.h | 16 ++++++------- 2 files changed, 27 insertions(+), 45 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index f221e9cab4d0..119b64c9c1d9 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -970,6 +970,24 @@ void __bfq_weights_tree_remove(struct bfq_data *bfqd, bfq_put_queue(bfqq); } +static void decrease_groups_with_pending_reqs(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ +#ifdef CONFIG_BFQ_GROUP_IOSCHED + struct bfq_entity *entity = bfqq->entity.parent; + + /* + * The decrement of num_groups_with_pending_reqs is performed + * immediately when the last bfqq completes all the requests. + */ + if (!bfqq_group(bfqq)->num_entities_with_pending_reqs && + entity->in_groups_with_pending_reqs) { + entity->in_groups_with_pending_reqs = false; + bfqd->num_groups_with_pending_reqs--; + } +#endif +} + /* * Invoke __bfq_weights_tree_remove on bfqq and decrement the number * of active groups for each queue's inactive parent entity. @@ -977,8 +995,6 @@ void __bfq_weights_tree_remove(struct bfq_data *bfqd, void bfq_weights_tree_remove(struct bfq_data *bfqd, struct bfq_queue *bfqq) { - struct bfq_entity *entity = bfqq->entity.parent; - /* * grab a ref to prevent bfqq to be freed in * __bfq_weights_tree_remove @@ -991,41 +1007,7 @@ void bfq_weights_tree_remove(struct bfq_data *bfqd, */ __bfq_weights_tree_remove(bfqd, bfqq, &bfqd->queue_weights_tree); - - for_each_entity(entity) { - struct bfq_sched_data *sd = entity->my_sched_data; - - if (sd->next_in_service || sd->in_service_entity) { - /* - * entity is still active, because either - * next_in_service or in_service_entity is not - * NULL (see the comments on the definition of - * next_in_service for details on why - * in_service_entity must be checked too). - * - * As a consequence, its parent entities are - * active as well, and thus this loop must - * stop here. - */ - break; - } - - /* - * The decrement of num_groups_with_pending_reqs is - * not performed immediately upon the deactivation of - * entity, but it is delayed to when it also happens - * that the first leaf descendant bfqq of entity gets - * all its pending requests completed. The following - * instructions perform this delayed decrement, if - * needed. See the comments on - * num_groups_with_pending_reqs for details. - */ - if (entity->in_groups_with_pending_reqs) { - entity->in_groups_with_pending_reqs = false; - bfqd->num_groups_with_pending_reqs--; - } - } - + decrease_groups_with_pending_reqs(bfqd, bfqq); bfq_put_queue(bfqq); } diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 5d904851519c..9ec72bd24fc2 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -495,7 +495,7 @@ struct bfq_data { struct rb_root_cached queue_weights_tree; /* - * Number of groups with at least one descendant process that + * Number of groups with at least one process that * has at least one request waiting for completion. Note that * this accounts for also requests already dispatched, but not * yet completed. Therefore this number of groups may differ @@ -508,14 +508,14 @@ struct bfq_data { * bfq_better_to_idle(). * * However, it is hard to compute this number exactly, for - * groups with multiple descendant processes. Consider a group - * that is inactive, i.e., that has no descendant process with + * groups with multiple processes. Consider a group + * that is inactive, i.e., that has no process with * pending I/O inside BFQ queues. Then suppose that * num_groups_with_pending_reqs is still accounting for this - * group, because the group has descendant processes with some + * group, because the group has processes with some * I/O request still in flight. num_groups_with_pending_reqs * should be decremented when the in-flight request of the - * last descendant process is finally completed (assuming that + * last process is finally completed (assuming that * nothing else has changed for the group in the meantime, in * terms of composition of the group and active/inactive state of child * groups and processes). To accomplish this, an additional @@ -524,7 +524,7 @@ struct bfq_data { * we resort to the following tradeoff between simplicity and * accuracy: for an inactive group that is still counted in * num_groups_with_pending_reqs, we decrement - * num_groups_with_pending_reqs when the first descendant + * num_groups_with_pending_reqs when the last * process of the group remains with no request waiting for * completion. * @@ -532,12 +532,12 @@ struct bfq_data { * carefulness: to avoid multiple decrements, we flag a group, * more precisely an entity representing a group, as still * counted in num_groups_with_pending_reqs when it becomes - * inactive. Then, when the first descendant queue of the + * inactive. Then, when the last queue of the * entity remains with no request waiting for completion, * num_groups_with_pending_reqs is decremented, and this flag * is reset. After this flag is reset for the entity, * num_groups_with_pending_reqs won't be decremented any - * longer in case a new descendant queue of the entity remains + * longer in case a new queue of the entity remains * with no request waiting for completion. */ unsigned int num_groups_with_pending_reqs; From patchwork Sat Mar 5 09:12:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12770294 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA9C3C433F5 for ; Sat, 5 Mar 2022 08:57:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231223AbiCEI5s (ORCPT ); Sat, 5 Mar 2022 03:57:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231194AbiCEI5m (ORCPT ); Sat, 5 Mar 2022 03:57:42 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C897255FB1; Sat, 5 Mar 2022 00:56:50 -0800 (PST) Received: from kwepemi500018.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4K9dp84ZlQzBrhP; Sat, 5 Mar 2022 16:54:56 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by kwepemi500018.china.huawei.com (7.221.188.213) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:48 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Sat, 5 Mar 2022 16:56:47 +0800 From: Yu Kuai To: , , , CC: , , , , Subject: [PATCH -next 11/11] block, bfq: cleanup bfqq_group() Date: Sat, 5 Mar 2022 17:12:05 +0800 Message-ID: <20220305091205.4188398-12-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220305091205.4188398-1-yukuai3@huawei.com> References: <20220305091205.4188398-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Now that if bfqq is under root group, 'bfqq->entity.parent' is set to root group's entity instead of NULL, there is no point for the judgement in bfqq_group() anymore. Signed-off-by: Yu Kuai --- block/bfq-cgroup.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 58acaf14a91d..1fcb13e97cf0 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -307,11 +307,7 @@ static struct bfq_group *bfqg_parent(struct bfq_group *bfqg) struct bfq_group *bfqq_group(struct bfq_queue *bfqq) { - struct bfq_entity *group_entity = bfqq->entity.parent; - - return group_entity ? container_of(group_entity, struct bfq_group, - entity) : - bfqq->bfqd->root_group; + return container_of(bfqq->entity.parent, struct bfq_group, entity); } /*