From patchwork Wed Aug 7 14:17:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 11082247 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E15C1399 for ; Wed, 7 Aug 2019 14:18:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5DF1728A57 for ; Wed, 7 Aug 2019 14:18:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5C56228A60; Wed, 7 Aug 2019 14:18:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD98928A57 for ; Wed, 7 Aug 2019 14:18:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388296AbfHGOSK (ORCPT ); Wed, 7 Aug 2019 10:18:10 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:35041 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729814AbfHGOSK (ORCPT ); Wed, 7 Aug 2019 10:18:10 -0400 Received: by mail-wm1-f65.google.com with SMTP id l2so241741wmg.0 for ; Wed, 07 Aug 2019 07:18:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kkeCGHY0DrsSxD9DKRdcnC2Wr0bb+DymKOjtK802aV8=; b=s487BfnVB9DFRp3LPdjVxRcYZly5YMoGZEBCfc4Yk3Hs9KDNnZp/jkEHPUUrLPNVG3 22uvC0GjMLtxHzpnrmH7k6IQ5EMXulSzuzzEeZ+TMPpXRsM4Cy3q9X03+8x+lXnO8Q0W wvAGp6vk06yhqPgWh7GJ9aU0iNmuzyYJv9CwoZBkxC+7m14HlpyA5RmRFBLmj5qrwJ4F ThM4d4PIwnK7kYhw5T+tfJJO1O7m0Mps2/BVDGzUn/rxYMUwJTptf6n2oWzhBwxlGf8S XjbswJu8iiM2PFjuwp+Wn/jsK5ShQ/5zx1PodtxwKmgV0tUjPBAAxoAvlIEYsDuff5yz Zjjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kkeCGHY0DrsSxD9DKRdcnC2Wr0bb+DymKOjtK802aV8=; b=XkBx1HANh+CovTRjMK3/5SySfZXGQjjwsHVYHABaWYL9yTLszzJbb8cn4pZj24+QQo bg8+bxA0lhsYCpOLSTG/gNxfVYPDY88ePmEVyPGmmrcqCnbIDRhOFl9YG31Pi/2d3Tfq brHyKU09Y0R1YO3X+t1esQ/ctOZ7kd/szMRPNYDKGvbYUinp2WlljYLo7nsJg/3V5x4W zmKmCeO0hYsOlkW4wftsmJiq1gg8UctuPjWCM3M7ucjINzhF/DOn5oW6+qVRn7kAnTx4 gv0i8CrgdGYC9vFi4XHqP/dKNNwlZexYQqtOUrbFz4wUHtD7YfBJlPX3MkiVjID9UFZf /RUg== X-Gm-Message-State: APjAAAUhwBe2Y3pYNwTmhiJn7dgMAj530/Nj66sklYqePE4gyPp9vEsr 6vP3JkYYUnR6e2LF6I1CYyBp4g== X-Google-Smtp-Source: APXvYqxBikW160o/IgvXRE2/Rs7ztL0oE2L/3/w6NtjyBftI6H8FuCL2HWhs+cOtNBrQvERchRk0Hw== X-Received: by 2002:a1c:4d01:: with SMTP id o1mr204052wmh.55.1565187488110; Wed, 07 Aug 2019 07:18:08 -0700 (PDT) Received: from localhost.localdomain (88-147-66-140.dyn.eolo.it. [88.147.66.140]) by smtp.gmail.com with ESMTPSA id o7sm83472wmc.36.2019.08.07.07.18.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 Aug 2019 07:18:07 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, pavel@denx.de, Paolo Valente , Douglas Anderson Subject: [PATCH BUGFIX 1/2] block, bfq: reset last_completed_rq_bfqq if the pointed queue is freed Date: Wed, 7 Aug 2019 16:17:53 +0200 Message-Id: <20190807141754.3567-2-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190807141754.3567-1-paolo.valente@linaro.org> References: <20190807141754.3567-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Since commit 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O"), BFQ stores, in a per-device pointer last_completed_rq_bfqq, the last bfq_queue that had an I/O request completed. If some bfq_queue receives new I/O right after the last request of last_completed_rq_bfqq has been completed, then last_completed_rq_bfqq may be a waker bfq_queue. But if the bfq_queue last_completed_rq_bfqq points to is freed, then last_completed_rq_bfqq becomes a dangling reference. This commit resets last_completed_rq_bfqq if the pointed bfq_queue is freed. Fixes: 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O") Reported-by: Douglas Anderson Tested-by: Douglas Anderson Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 586fcfe227ea..b2009650afc2 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -1924,12 +1924,13 @@ static void bfq_add_request(struct request *rq) * confirmed no later than during the next * I/O-plugging interval for bfqq. */ - if (!bfq_bfqq_has_short_ttime(bfqq) && + if (bfqd->last_completed_rq_bfqq && + !bfq_bfqq_has_short_ttime(bfqq) && ktime_get_ns() - bfqd->last_completion < 200 * NSEC_PER_USEC) { if (bfqd->last_completed_rq_bfqq != bfqq && - bfqd->last_completed_rq_bfqq != - bfqq->waker_bfqq) { + bfqd->last_completed_rq_bfqq != + bfqq->waker_bfqq) { /* * First synchronization detected with * a candidate waker queue, or with a @@ -4808,6 +4809,9 @@ void bfq_put_queue(struct bfq_queue *bfqq) bfqq->bfqd->burst_size--; } + if (bfqq->bfqd && bfqq->bfqd->last_completed_rq_bfqq == bfqq) + bfqq->bfqd->last_completed_rq_bfqq = NULL; + kmem_cache_free(bfq_pool, bfqq); #ifdef CONFIG_BFQ_GROUP_IOSCHED bfqg_and_blkg_put(bfqg); From patchwork Wed Aug 7 14:17:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 11082249 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA9C4912 for ; Wed, 7 Aug 2019 14:18:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AAC70288DA for ; Wed, 7 Aug 2019 14:18:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A8C4528A61; Wed, 7 Aug 2019 14:18:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F30D28A63 for ; Wed, 7 Aug 2019 14:18:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388331AbfHGOSQ (ORCPT ); Wed, 7 Aug 2019 10:18:16 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:55906 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388284AbfHGOSM (ORCPT ); Wed, 7 Aug 2019 10:18:12 -0400 Received: by mail-wm1-f68.google.com with SMTP id f72so210889wmf.5 for ; Wed, 07 Aug 2019 07:18:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2fEGd1avRjzNgoZKRJo57rOsvWSouUmLScAruU7orM4=; b=CDf3h5luG1Il3ysYALeHWQ6nAmZqFgCk9fpa7U9ICPkY270+BhZcQd1bpfu9LpL/cW 01/f7VTeMqbP+MFgqN7O087injQ04LCr9x8jQzbG7vzOiXIkQmXy6B19cyzm+lte6gie kqfl9GXrDqh+UN78SCyYhcv1oQ7uZOpfHYTvW8HGDhfucQtMZxypnsAYiai/SXLIpEvB 8i0Mx/Roa1Tx3g1TdvMXc5BFGIGd8zDcU7tw206biQfAAWpwz77CIYp3lbJ/uApynJxu +FOOPcNPL+mQmuU14xJh/0aDwunKYcmCasNuIlg5DEkxVEUSZ6o3NlMT+TBiw6I03F/o qROQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2fEGd1avRjzNgoZKRJo57rOsvWSouUmLScAruU7orM4=; b=FgMVEt0BQn4Hxy4mAZXyUez9hcuKm/stkwYdUV/XwbVCyT7/ExRksyGlcGrwxYXgU+ oS1ljkb2SAUvpLIHSYK0E3y0Wff7yI3PXKia4i/xxP/3LiFixq3tyGaoLqdiYb2iDRmJ prJguQ2/ZYFlv+vC4WVUxkYK8d7HPGSFMCSudHVYZWw0c8lXjlLXOGYzo3M8pHJDyfYR ul6RxkR4plt0wLAuvvNeZSfBFbMK/DHyvkSgI/i8RQao8GyOXFf0vchD5qMDpK9gHRwB NbHPeLdUzjanma7Q7YExEmNanNsIFbPR3N6o54EoJQhzVCubW022DF0cGtJCir/9mqlS zZxw== X-Gm-Message-State: APjAAAXZbyukYe1VC2ga5H4psovXqXA0O0FNY+ZBKIwKIWADmuWlikp+ p5wFn/pRFvcpOND2+VoRPAVATg== X-Google-Smtp-Source: APXvYqz4Ie0/WbmHMaV9QT+7Q/YEhMJKUx3I61VhSuHO5eFrBxrNI+AyMPeybnzglU2yYvc/DTgnrA== X-Received: by 2002:a1c:9696:: with SMTP id y144mr194876wmd.73.1565187489338; Wed, 07 Aug 2019 07:18:09 -0700 (PDT) Received: from localhost.localdomain (88-147-66-140.dyn.eolo.it. [88.147.66.140]) by smtp.gmail.com with ESMTPSA id o7sm83472wmc.36.2019.08.07.07.18.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 Aug 2019 07:18:08 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, pavel@denx.de, Paolo Valente , Douglas Anderson Subject: [PATCH BUGFIX 2/2] block, bfq: move update of waker and woken list to queue freeing Date: Wed, 7 Aug 2019 16:17:54 +0200 Message-Id: <20190807141754.3567-3-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190807141754.3567-1-paolo.valente@linaro.org> References: <20190807141754.3567-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Since commit 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O"), every bfq_queue has a pointer to a waker bfq_queue and a list of the bfq_queues it may wake. In this respect, when a bfq_queue, say Q, remains with no I/O source attached to it, Q cannot be woken by any other bfq_queue, and cannot wake any other bfq_queue. Then Q must be removed from the woken list of its possible waker bfq_queue, and all bfq_queues in the woken list of Q must stop having a waker bfq_queue. Q remains with no I/O source in two cases: when the last process associated with Q exits or when such a process gets associated with a different bfq_queue. Unfortunately, commit 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O") performed the above updates only in the first case. This commit fixes this bug by moving these updates to when Q gets freed. This is a simple and safe way to handle all cases, as both the above events, process exit and re-association, lead to Q being freed soon, and because dangling references would come out only after Q gets freed (if no update were performed). Fixes: 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O") Reported-by: Douglas Anderson Tested-by: Douglas Anderson Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 44 +++++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 15 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index b2009650afc2..5f477501bb3d 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -4765,6 +4765,8 @@ static struct request *bfq_dispatch_request(struct blk_mq_hw_ctx *hctx) */ void bfq_put_queue(struct bfq_queue *bfqq) { + struct bfq_queue *item; + struct hlist_node *n; #ifdef CONFIG_BFQ_GROUP_IOSCHED struct bfq_group *bfqg = bfqq_group(bfqq); #endif @@ -4809,6 +4811,33 @@ void bfq_put_queue(struct bfq_queue *bfqq) bfqq->bfqd->burst_size--; } + /* + * bfqq does not exist any longer, so it cannot be woken by + * any other queue, and cannot wake any other queue. Then bfqq + * must be removed from the woken list of its possible waker + * queue, and all queues in the woken list of bfqq must stop + * having a waker queue. Strictly speaking, these updates + * should be performed when bfqq remains with no I/O source + * attached to it, which happens before bfqq gets freed. In + * particular, this happens when the last process associated + * with bfqq exits or gets associated with a different + * queue. However, both events lead to bfqq being freed soon, + * and dangling references would come out only after bfqq gets + * freed. So these updates are done here, as a simple and safe + * way to handle all cases. + */ + /* remove bfqq from woken list */ + if (!hlist_unhashed(&bfqq->woken_list_node)) + hlist_del_init(&bfqq->woken_list_node); + + /* reset waker for all queues in woken list */ + hlist_for_each_entry_safe(item, n, &bfqq->woken_list, + woken_list_node) { + item->waker_bfqq = NULL; + bfq_clear_bfqq_has_waker(item); + hlist_del_init(&item->woken_list_node); + } + if (bfqq->bfqd && bfqq->bfqd->last_completed_rq_bfqq == bfqq) bfqq->bfqd->last_completed_rq_bfqq = NULL; @@ -4839,9 +4868,6 @@ static void bfq_put_cooperator(struct bfq_queue *bfqq) static void bfq_exit_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq) { - struct bfq_queue *item; - struct hlist_node *n; - if (bfqq == bfqd->in_service_queue) { __bfq_bfqq_expire(bfqd, bfqq, BFQQE_BUDGET_TIMEOUT); bfq_schedule_dispatch(bfqd); @@ -4851,18 +4877,6 @@ static void bfq_exit_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq) bfq_put_cooperator(bfqq); - /* remove bfqq from woken list */ - if (!hlist_unhashed(&bfqq->woken_list_node)) - hlist_del_init(&bfqq->woken_list_node); - - /* reset waker for all queues in woken list */ - hlist_for_each_entry_safe(item, n, &bfqq->woken_list, - woken_list_node) { - item->waker_bfqq = NULL; - bfq_clear_bfqq_has_waker(item); - hlist_del_init(&item->woken_list_node); - } - bfq_put_queue(bfqq); /* release process reference */ }