From patchwork Sat Jan 22 11:10:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DD31C433F5 for ; Sat, 22 Jan 2022 11:11:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234114AbiAVLLe (ORCPT ); Sat, 22 Jan 2022 06:11:34 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:46296 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234155AbiAVLLd (ORCPT ); Sat, 22 Jan 2022 06:11:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849893; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kYSY5CHYNIA9hN8QVlij2GnIqK59IRJkhItz3wdmKVE=; b=dz2goOGe9YNdcgFe+oROnVakR9LUn3NJn/XODSkC5TO5jSoZlmdXkXh784W/frILhKwvzT SgNEVVGQtiMqlNHxLaSA7GuNvmtkmpDxNmelTH4XGau6Jq1QTIqvtDh9mW8ennjryqd09o Uu1xtNNaKCMbyIwJXtsB6WAX9ycehr8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-641-jpdpLSJKOjeU72gf5C5rmA-1; Sat, 22 Jan 2022 06:11:29 -0500 X-MC-Unique: jpdpLSJKOjeU72gf5C5rmA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3B9191083F60; Sat, 22 Jan 2022 11:11:28 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC64E7316D; Sat, 22 Jan 2022 11:11:22 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 01/13] block: declare blkcg_[init|exit]_queue in private header Date: Sat, 22 Jan 2022 19:10:42 +0800 Message-Id: <20220122111054.1126146-2-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Both two functions are used by block core code only, so move the declaration into private header of block layer. Signed-off-by: Ming Lei --- block/blk.h | 8 ++++++++ include/linux/blk-cgroup.h | 4 ---- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/block/blk.h b/block/blk.h index 8bd43b3ad33d..7b0f12260ae6 100644 --- a/block/blk.h +++ b/block/blk.h @@ -406,6 +406,14 @@ extern int blk_iolatency_init(struct request_queue *q); static inline int blk_iolatency_init(struct request_queue *q) { return 0; } #endif +#ifdef CONFIG_BLK_CGROUP +int blkcg_init_queue(struct request_queue *q); +void blkcg_exit_queue(struct request_queue *q); +#else +static inline int blkcg_init_queue(struct request_queue *q) { return 0; } +static inline void blkcg_exit_queue(struct request_queue *q) { } +#endif + struct bio *blk_next_bio(struct bio *bio, unsigned int nr_pages, gfp_t gfp); #ifdef CONFIG_BLK_DEV_ZONED diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index b4de2010fba5..d9dcd18d77f4 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -182,8 +182,6 @@ extern bool blkcg_debug_stats; struct blkcg_gq *blkg_lookup_slowpath(struct blkcg *blkcg, struct request_queue *q, bool update_hint); -int blkcg_init_queue(struct request_queue *q); -void blkcg_exit_queue(struct request_queue *q); /* Blkio controller policy registration */ int blkcg_policy_register(struct blkcg_policy *pol); @@ -637,8 +635,6 @@ static inline void blkcg_schedule_throttle(struct request_queue *q, bool use_mem static inline struct blkcg_gq *blkg_lookup(struct blkcg *blkcg, void *key) { return NULL; } static inline struct blkcg_gq *blk_queue_root_blkg(struct request_queue *q) { return NULL; } -static inline int blkcg_init_queue(struct request_queue *q) { return 0; } -static inline void blkcg_exit_queue(struct request_queue *q) { } static inline int blkcg_policy_register(struct blkcg_policy *pol) { return 0; } static inline void blkcg_policy_unregister(struct blkcg_policy *pol) { } static inline int blkcg_activate_policy(struct request_queue *q, From patchwork Sat Jan 22 11:10:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82BB8C433EF for ; Sat, 22 Jan 2022 11:11:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234161AbiAVLLx (ORCPT ); Sat, 22 Jan 2022 06:11:53 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44681 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234157AbiAVLLw (ORCPT ); Sat, 22 Jan 2022 06:11:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CpLLy27V5MbHuoHqIYVsmv/vaXJTkt3tPuaNGjzjjso=; b=VF5p26LB5BGmUk/JR0/4+FnOSolY7fqD0dMm3qmlYn0U8p+RjwNW5mZRwDvKIVmcmMbks/ H9MbbQQL+GwPofYENsuITisPakyzu6Yjb6VBx3OFV0Dh+eMi0VuRymGePH9blK/H0GWxue sa6settbe6xD7fMV5iRxOuXDZ1g0ou8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-164-covyJr92MAS458TWqOBzTw-1; Sat, 22 Jan 2022 06:11:49 -0500 X-MC-Unique: covyJr92MAS458TWqOBzTw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BD8B7814245; Sat, 22 Jan 2022 11:11:47 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id B167E6F116; Sat, 22 Jan 2022 11:11:31 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 02/13] block: move initialization of q->blkg_list into blkcg_init_queue Date: Sat, 22 Jan 2022 19:10:43 +0800 Message-Id: <20220122111054.1126146-3-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org q->blkg_list is only used by blkcg code, so move it into blkcg_init_queue. Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Reviewed-by: Bart Van Assche --- block/blk-cgroup.c | 2 ++ block/blk-core.c | 3 --- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 650f7e27989f..498753e2bb73 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1176,6 +1176,8 @@ int blkcg_init_queue(struct request_queue *q) bool preloaded; int ret; + INIT_LIST_HEAD(&q->blkg_list); + new_blkg = blkg_alloc(&blkcg_root, q, GFP_KERNEL); if (!new_blkg) return -ENOMEM; diff --git a/block/blk-core.c b/block/blk-core.c index 97f8bc8d3a79..2a400fa8cabd 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -475,9 +475,6 @@ struct request_queue *blk_alloc_queue(int node_id, bool alloc_srcu) timer_setup(&q->timeout, blk_rq_timed_out_timer, 0); INIT_WORK(&q->timeout_work, blk_timeout_work); INIT_LIST_HEAD(&q->icq_list); -#ifdef CONFIG_BLK_CGROUP - INIT_LIST_HEAD(&q->blkg_list); -#endif kobject_init(&q->kobj, &blk_queue_ktype); From patchwork Sat Jan 22 11:10:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FFCAC433EF for ; Sat, 22 Jan 2022 11:12:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234188AbiAVLMB (ORCPT ); Sat, 22 Jan 2022 06:12:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:23573 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234157AbiAVLMA (ORCPT ); Sat, 22 Jan 2022 06:12:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IhzZxghKLjv/eAyw20BiM/yJyRuvLKKix2OV+q1Q9hY=; b=LkLEgDOMDETXazdrP9H1K6l4TISL8fRJ/2D0VC23mH8I6tQHnTi/CLd1/GvlNuRzFnQWk/ UZN6XcFASdKt8vAa93YxvDqAU+wtM4xF5lsgpzlrgTUTg82S79x2V74lqSYZTUweb0BJfo X+xjAcsz4pSRsVOpvSElDcRJQiDRyy4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-673-VdJ4s97LPuO7HEw3K5CcjQ-1; Sat, 22 Jan 2022 06:11:56 -0500 X-MC-Unique: VdJ4s97LPuO7HEw3K5CcjQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B58BD1083F60; Sat, 22 Jan 2022 11:11:54 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 03059108BD2C; Sat, 22 Jan 2022 11:11:50 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 03/13] block: move blkcg initialization/destroy into disk allocation/release handler Date: Sat, 22 Jan 2022 19:10:44 +0800 Message-Id: <20220122111054.1126146-4-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org blkcg works on FS bio level, so it is reasonable to make both blkcg and gendisk sharing same lifetime. Meantime there won't be any FS IO when releasing disk, so safe to move blkcg initialization/destroy into disk allocation/release handler Long term, we can move blkcg into gendisk completely. Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig --- block/blk-core.c | 5 ----- block/blk-sysfs.c | 7 ------- block/genhd.c | 14 ++++++++++++++ 3 files changed, 14 insertions(+), 12 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 2a400fa8cabd..d9477191b303 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -495,17 +495,12 @@ struct request_queue *blk_alloc_queue(int node_id, bool alloc_srcu) PERCPU_REF_INIT_ATOMIC, GFP_KERNEL)) goto fail_stats; - if (blkcg_init_queue(q)) - goto fail_ref; - blk_queue_dma_alignment(q, 511); blk_set_default_limits(&q->limits); q->nr_requests = BLKDEV_DEFAULT_RQ; return q; -fail_ref: - percpu_ref_exit(&q->q_usage_counter); fail_stats: blk_free_queue_stats(q->stats); fail_split: diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index e20eadfcf5c8..6f326b44fb00 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -751,13 +751,6 @@ static void blk_exit_queue(struct request_queue *q) ioc_clear_queue(q); elevator_exit(q); } - - /* - * Remove all references to @q from the block cgroup controller before - * restoring @q->queue_lock to avoid that restoring this pointer causes - * e.g. blkcg_print_blkgs() to crash. - */ - blkcg_exit_queue(q); } /** diff --git a/block/genhd.c b/block/genhd.c index 626c8406f21a..b9b0db168ce1 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -1112,6 +1112,14 @@ static void disk_release(struct device *dev) disk_release_events(disk); kfree(disk->random); xa_destroy(&disk->part_tbl); + + /* + * Remove all references to @q from the block cgroup controller before + * restoring @q->queue_lock to avoid that restoring this pointer causes + * e.g. blkcg_print_blkgs() to crash. + */ + blkcg_exit_queue(disk->queue); + disk->queue->disk = NULL; blk_put_queue(disk->queue); iput(disk->part0->bd_inode); /* frees the disk */ @@ -1308,6 +1316,10 @@ struct gendisk *__alloc_disk_node(struct request_queue *q, int node_id, if (xa_insert(&disk->part_tbl, 0, disk->part0, GFP_KERNEL)) goto out_destroy_part_tbl; + /* todo: move blkcg into gendisk */ + if (blkcg_init_queue(q)) + goto out_erase_part0; + rand_initialize_disk(disk); disk_to_dev(disk)->class = &block_class; disk_to_dev(disk)->type = &disk_type; @@ -1320,6 +1332,8 @@ struct gendisk *__alloc_disk_node(struct request_queue *q, int node_id, #endif return disk; +out_erase_part0: + xa_erase(&disk->part_tbl, 0); out_destroy_part_tbl: xa_destroy(&disk->part_tbl); disk->part0->bd_disk = NULL; From patchwork Sat Jan 22 11:10:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BA0DC433EF for ; Sat, 22 Jan 2022 11:12:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234230AbiAVLMF (ORCPT ); Sat, 22 Jan 2022 06:12:05 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:40638 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234157AbiAVLME (ORCPT ); Sat, 22 Jan 2022 06:12:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849924; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LlF+wgG4ISXixuSFtCnbmre1aZwrkzCKFUpXClc+E00=; b=EvJ8xKv3rKLEjB0a3e5rauGs7M2FTKZMsJEQQ2VljyXU75LgUAiedhIZLR/SuqKbKBY48/ FZyc9DzeO9mlQnFgPnL5WilND6Wv7gtBNY9KxgiAN12mGwoxsmQd4NmqMeq3+TVuJ0i3cp m401AoQLxMYHW3LmOySXVhTM4RzExDE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-457-z72St53NPz-2-WbMBszyRQ-1; Sat, 22 Jan 2022 06:12:00 -0500 X-MC-Unique: z72St53NPz-2-WbMBszyRQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 08F9A814243; Sat, 22 Jan 2022 11:11:59 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A9E8752D6; Sat, 22 Jan 2022 11:11:57 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Laibin Qiu , Ming Lei Subject: [PATCH V2 04/13] block/wbt: fix negative inflight counter when remove scsi device Date: Sat, 22 Jan 2022 19:10:45 +0800 Message-Id: <20220122111054.1126146-5-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Laibin Qiu Now that we disable wbt by set WBT_STATE_OFF_DEFAULT in wbt_disable_default() when switch elevator to bfq. And when we remove scsi device, wbt will be enabled by wbt_enable_default. If it become false positive between wbt_wait() and wbt_track() when submit write request. The following is the scenario that triggered the problem. T1 T2 T3 elevator_switch_mq bfq_init_queue wbt_disable_default <= Set rwb->enable_state (OFF) Submit_bio blk_mq_make_request rq_qos_throttle <= rwb->enable_state (OFF) scsi_remove_device sd_remove del_gendisk blk_unregister_queue elv_unregister_queue wbt_enable_default <= Set rwb->enable_state (ON) q_qos_track <= rwb->enable_state (ON) ^^^^^^ this request will mark WBT_TRACKED without inflight add and will lead to drop rqw->inflight to -1 in wbt_done() which will trigger IO hung. Fix this by move wbt_enable_default() from elv_unregister to bfq_exit_queue(). Only re-enable wbt when bfq exit. Fixes: 76a8040817b4b ("blk-wbt: make sure throttle is enabled properly") Remove oneline stale comment, and kill one oneshot local variable. Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/linux-block/20211214133103.551813-1-qiulaibin@huawei.com/ Signed-off-by: Laibin Qiu --- block/bfq-iosched.c | 2 ++ block/elevator.c | 2 -- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 0c612a911696..36a66e97e3c2 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -7018,6 +7018,8 @@ static void bfq_exit_queue(struct elevator_queue *e) spin_unlock_irq(&bfqd->lock); #endif + wbt_enable_default(bfqd->queue); + kfree(bfqd); } diff --git a/block/elevator.c b/block/elevator.c index ec98aed39c4f..482df2a350fc 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -525,8 +525,6 @@ void elv_unregister_queue(struct request_queue *q) kobject_del(&e->kobj); e->registered = 0; - /* Re-enable throttling in case elevator disabled it */ - wbt_enable_default(q); } } From patchwork Sat Jan 22 11:10:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51EDFC433EF for ; Sat, 22 Jan 2022 11:12:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234242AbiAVLMM (ORCPT ); Sat, 22 Jan 2022 06:12:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21604 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234203AbiAVLMM (ORCPT ); Sat, 22 Jan 2022 06:12:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FPH7dGVIFzXjTeIzBsKDE0c39uig7yHFFZMv14xZNew=; b=ZB7DeHvjW+B4GCzDDYruNdnibnabBSBVNdr1KthWwiVchqlLohbgC7lC4A8dy3KqbxLn3V I6FwJNodTP6H3/T/33S32ybgxdWcT26yRHXC/eRI7Z1ilfxFfxU0Gm49XpI8lscJww/AHr ODwU3pZ4PJmEc/8nEwIxxQ8eKb1q29M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-414-5UCEbBjDNkKDyihBe5yZDw-1; Sat, 22 Jan 2022 06:12:07 -0500 X-MC-Unique: 5UCEbBjDNkKDyihBe5yZDw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A88F72F47; Sat, 22 Jan 2022 11:12:06 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 56A1B7B9D3; Sat, 22 Jan 2022 11:12:01 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 05/13] block: only account passthrough IO from userspace Date: Sat, 22 Jan 2022 19:10:46 +0800 Message-Id: <20220122111054.1126146-6-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Passthrough request from userspace has one active block_device/disk associated, so they can be accounted via rq->q->disk. For other passthrough request, there may not be disk/block_device for the queue, since either the queue has not a disk or the disk may be deleted already. Add flag of BLK_MQ_REQ_USER_IO for only accounting passthrough request from userspace. Signed-off-by: Ming Lei Signed-off-by: Christoph Hellwig --- block/blk-mq.c | 13 ++++++++++++- drivers/nvme/host/ioctl.c | 2 +- drivers/scsi/scsi_ioctl.c | 3 ++- include/linux/blk-mq.h | 2 ++ 4 files changed, 17 insertions(+), 3 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index a6d4780580fc..0d25cc5778c9 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -336,6 +336,17 @@ void blk_rq_init(struct request_queue *q, struct request *rq) } EXPORT_SYMBOL(blk_rq_init); +static inline bool blk_mq_io_may_account(struct blk_mq_alloc_data *data) +{ + if (!blk_op_is_passthrough(data->cmd_flags)) + return true; + + if (data->flags & BLK_MQ_REQ_USER_IO) + return true; + + return false; +} + static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data, struct blk_mq_tags *tags, unsigned int tag, u64 alloc_time_ns) { @@ -351,7 +362,7 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data, if (data->flags & BLK_MQ_REQ_PM) data->rq_flags |= RQF_PM; - if (blk_queue_io_stat(q)) + if (blk_queue_io_stat(q) && blk_mq_io_may_account(data)) data->rq_flags |= RQF_IO_STAT; rq->rq_flags = data->rq_flags; diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 22314962842d..f94afc38a6e3 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -66,7 +66,7 @@ static int nvme_submit_user_cmd(struct request_queue *q, void *meta = NULL; int ret; - req = nvme_alloc_request(q, cmd, 0); + req = nvme_alloc_request(q, cmd, BLK_MQ_REQ_USER_IO); if (IS_ERR(req)) return PTR_ERR(req); diff --git a/drivers/scsi/scsi_ioctl.c b/drivers/scsi/scsi_ioctl.c index e13fd380deb6..b262fe06dacc 100644 --- a/drivers/scsi/scsi_ioctl.c +++ b/drivers/scsi/scsi_ioctl.c @@ -440,7 +440,8 @@ static int sg_io(struct scsi_device *sdev, struct sg_io_hdr *hdr, fmode_t mode) ret = -ENOMEM; rq = scsi_alloc_request(sdev->request_queue, writing ? - REQ_OP_DRV_OUT : REQ_OP_DRV_IN, 0); + REQ_OP_DRV_OUT : REQ_OP_DRV_IN, + BLK_MQ_REQ_USER_IO); if (IS_ERR(rq)) return PTR_ERR(rq); req = scsi_req(rq); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index d319ffa59354..d2ad2ed11723 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -709,6 +709,8 @@ enum { BLK_MQ_REQ_RESERVED = (__force blk_mq_req_flags_t)(1 << 1), /* set RQF_PM */ BLK_MQ_REQ_PM = (__force blk_mq_req_flags_t)(1 << 2), + /* user IO request */ + BLK_MQ_REQ_USER_IO = (__force blk_mq_req_flags_t)(1 << 3), }; struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op, From patchwork Sat Jan 22 11:10:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BED07C433F5 for ; Sat, 22 Jan 2022 11:12:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234235AbiAVLMS (ORCPT ); Sat, 22 Jan 2022 06:12:18 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:34247 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234212AbiAVLMS (ORCPT ); Sat, 22 Jan 2022 06:12:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849937; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7SzlZ2qY9gfx7Y6rCDOZklcwjV2ZnHtReA0zi58Z5r8=; b=fA4FLMWCNR3FZgSL6q3ekFk1hLqOwyinWqMuILpwcQeqNhlsugOEchsRPWbfaTVA492MMG Axnxg5lDAfrpA3kgAa6JrEwgs2ykxSoYT/kkHDVrglY9owysamVAukGDTuVAYh0I6t3nfP 22EC+EDIMCqnSd+MLiRFtz1hBE3ZEv8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-413-EHdXdVtiO22OobtnEPxh-w-1; Sat, 22 Jan 2022 06:12:11 -0500 X-MC-Unique: EHdXdVtiO22OobtnEPxh-w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6036C1006AA3; Sat, 22 Jan 2022 11:12:10 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id AAF4F7B9E1; Sat, 22 Jan 2022 11:12:09 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 06/13] block: don't remove hctx debugfs dir from blk_mq_exit_queue Date: Sat, 22 Jan 2022 19:10:47 +0800 Message-Id: <20220122111054.1126146-7-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The queue's top debugfs dir is removed from blk_release_queue(), so all hctx's debugfs dirs are removed from there. Given blk_mq_exit_queue() is only called from blk_cleanup_queue(), it isn't necessary to remove hctx debugfs from blk_mq_exit_queue(). So remove it from blk_mq_exit_queue(). Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 0d25cc5778c9..66cc701921c1 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3454,7 +3454,6 @@ static void blk_mq_exit_hw_queues(struct request_queue *q, queue_for_each_hw_ctx(q, hctx, i) { if (i == nr_queue) break; - blk_mq_debugfs_unregister_hctx(hctx); blk_mq_exit_hctx(q, set, hctx, i); } } From patchwork Sat Jan 22 11:10:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8A8DC433EF for ; Sat, 22 Jan 2022 11:12:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234260AbiAVLMT (ORCPT ); Sat, 22 Jan 2022 06:12:19 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:57682 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234243AbiAVLMT (ORCPT ); Sat, 22 Jan 2022 06:12:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849938; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pZXy6AjZjPC6H/SXIyE5VyyrzC96iMDRLk5+OuIek/c=; b=QiIn4rfabcgdIlB+mxTua2WwWC8Ss+FfscKz4QY18CmWSXV9XTJVkfwBLS4uk+/kQlPA8X abRTMUiKurjQHECUe4/J2tcVfqYmN6uzLCCJ5JNGZzjJfWdUUjNGHO+/pDnYrgGDduQAIk wjRpNZxTFa9rV8RgNXyHZmwopZuWLoY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-100-VMuyXiscMfyyuedqexKhUA-1; Sat, 22 Jan 2022 06:12:15 -0500 X-MC-Unique: VMuyXiscMfyyuedqexKhUA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BFBEF2F47; Sat, 22 Jan 2022 11:12:13 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id EC14D7B9DC; Sat, 22 Jan 2022 11:12:12 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 07/13] block: move q_usage_counter release into blk_queue_release Date: Sat, 22 Jan 2022 19:10:48 +0800 Message-Id: <20220122111054.1126146-8-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org After blk_cleanup_queue() returns, disk may not be released yet, so probably bio may still be submitted and ->q_usage_counter may be touched, so far this way seems safe, but not good from API's viewpoint. Move the release ofq_usage_counter into blk_queue_release(). Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Reviewed-by: Bart Van Assche --- block/blk-core.c | 2 -- block/blk-sysfs.c | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index d9477191b303..bcb4d982cd80 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -341,8 +341,6 @@ void blk_cleanup_queue(struct request_queue *q) blk_mq_sched_free_rqs(q); mutex_unlock(&q->sysfs_lock); - percpu_ref_exit(&q->q_usage_counter); - /* @q is and will stay empty, shutdown and put */ blk_put_queue(q); } diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 6f326b44fb00..5f14fd333182 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -780,6 +780,8 @@ static void blk_release_queue(struct kobject *kobj) might_sleep(); + percpu_ref_exit(&q->q_usage_counter); + if (q->poll_stat) blk_stat_remove_callback(q, q->poll_cb); blk_stat_free_callback(q->poll_cb); From patchwork Sat Jan 22 11:10:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F30F4C433EF for ; Sat, 22 Jan 2022 11:12:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234246AbiAVLMa (ORCPT ); Sat, 22 Jan 2022 06:12:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:25181 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234254AbiAVLM0 (ORCPT ); Sat, 22 Jan 2022 06:12:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849946; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iWDZhCx4WfOjbjkCKSgQjjsWcEnBKzFq0pmqP92OyGQ=; b=CQ02tPt6eUA/8QlQf33tT/t5jeLgqsIPAkLGRNVxaSfOZR/JnpyTSG/pkwphdYKp4n9ywD kqVLBmJUA9vdhXD9VKPCwRTR1VefdQWgRoBatoSGC+u3T3fLy2amGBgMao/b0vGp6FCq/F qGBiTIc3Vau7TiC3l5nTRqn6JXCbklk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-657-0Qe6dymAMrSf663s74Rv1A-1; Sat, 22 Jan 2022 06:12:22 -0500 X-MC-Unique: 0Qe6dymAMrSf663s74Rv1A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 71A0D1006AA3; Sat, 22 Jan 2022 11:12:21 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id BC90770D3D; Sat, 22 Jan 2022 11:12:17 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 08/13] block: export __blk_mq_unfreeze_queue Date: Sat, 22 Jan 2022 19:10:49 +0800 Message-Id: <20220122111054.1126146-9-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org blk_mq_unfreeze_queue() is used by scsi when releasing disk, so not necessary to unfreeze into percpu mode, then the following blk_cleanup_queue doesn't need to freeze queue from percpu mode, and the implied RCU grace period may be avoided. Meantime move clearing QUEUE_FLAG_INIT_DONE into this API, so that when one disk is added, ->q_usage_counter can be switched to percpu mode again. Signed-off-by: Ming Lei --- block/blk-mq.c | 10 +++++++++- block/blk.h | 1 - block/genhd.c | 1 - include/linux/blk-mq.h | 1 + 4 files changed, 10 insertions(+), 3 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 66cc701921c1..d51b0aa2e4e4 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -215,11 +215,18 @@ void blk_mq_freeze_queue(struct request_queue *q) } EXPORT_SYMBOL_GPL(blk_mq_freeze_queue); +/* + * When 'force_atomic' is passed as true, this API is supposed to be + * called only in case that disk is removed or released. + */ void __blk_mq_unfreeze_queue(struct request_queue *q, bool force_atomic) { mutex_lock(&q->mq_freeze_lock); - if (force_atomic) + if (force_atomic) { + /* When new disk is added, switch to percpu mode */ + blk_queue_flag_clear(QUEUE_FLAG_INIT_DONE, q); q->q_usage_counter.data->force_atomic = true; + } q->mq_freeze_depth--; WARN_ON_ONCE(q->mq_freeze_depth < 0); if (!q->mq_freeze_depth) { @@ -228,6 +235,7 @@ void __blk_mq_unfreeze_queue(struct request_queue *q, bool force_atomic) } mutex_unlock(&q->mq_freeze_lock); } +EXPORT_SYMBOL_GPL(__blk_mq_unfreeze_queue); void blk_mq_unfreeze_queue(struct request_queue *q) { diff --git a/block/blk.h b/block/blk.h index 7b0f12260ae6..a038e25d8637 100644 --- a/block/blk.h +++ b/block/blk.h @@ -43,7 +43,6 @@ struct blk_flush_queue *blk_alloc_flush_queue(int node, int cmd_size, void blk_free_flush_queue(struct blk_flush_queue *q); void blk_freeze_queue(struct request_queue *q); -void __blk_mq_unfreeze_queue(struct request_queue *q, bool force_atomic); void blk_queue_start_drain(struct request_queue *q); int __bio_queue_enter(struct request_queue *q, struct bio *bio); bool submit_bio_checks(struct bio *bio); diff --git a/block/genhd.c b/block/genhd.c index b9b0db168ce1..5bd7bcd6246e 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -628,7 +628,6 @@ void del_gendisk(struct gendisk *disk) /* * Allow using passthrough request again after the queue is torn down. */ - blk_queue_flag_clear(QUEUE_FLAG_INIT_DONE, q); __blk_mq_unfreeze_queue(q, true); } diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index d2ad2ed11723..1645159d10f3 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -869,6 +869,7 @@ void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset, void blk_mq_tagset_wait_completed_request(struct blk_mq_tag_set *tagset); void blk_mq_freeze_queue(struct request_queue *q); void blk_mq_unfreeze_queue(struct request_queue *q); +void __blk_mq_unfreeze_queue(struct request_queue *q, bool force_atomic); void blk_freeze_queue_start(struct request_queue *q); void blk_mq_freeze_queue_wait(struct request_queue *q); int blk_mq_freeze_queue_wait_timeout(struct request_queue *q, From patchwork Sat Jan 22 11:10:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49744C433EF for ; Sat, 22 Jan 2022 11:12:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229938AbiAVLMh (ORCPT ); Sat, 22 Jan 2022 06:12:37 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34676 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234261AbiAVLMg (ORCPT ); Sat, 22 Jan 2022 06:12:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849955; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qxxh2m6PuB2dHP6HEE7eZbZqAl8fy6fCQ0SrIpnMil4=; b=bJqd3UH1G+i5OXAyPATOqLkkNj5eAGP9TfcI0d+6SewRzXXzUO4didO3uXhMRZjfyyDLeu jEEgx+8RjdpZPIgPBQrSz2v123nfv20nq8LtH8f0dL0Gpb3QGwBXJCAQLTxZMaU93c2r4y NNjI0zJ1zGOtjS+c9uKeqov/lsu4W7I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-208-3_Ai3tyWNYGEhkRQpEXpnA-1; Sat, 22 Jan 2022 06:12:32 -0500 X-MC-Unique: 3_Ai3tyWNYGEhkRQpEXpnA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 62D0C814243; Sat, 22 Jan 2022 11:12:31 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id C92847B9D3; Sat, 22 Jan 2022 11:12:24 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 09/13] scsi: force unfreezing queue into atomic mode Date: Sat, 22 Jan 2022 19:10:50 +0800 Message-Id: <20220122111054.1126146-10-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In scsi_disk_release() request queue is frozen for clearing disk->private_data, and there can't be any FS IO issued to this queue, and only private passthrough request will be handled, so force unfreezing queue into atomic mode. Signed-off-by: Ming Lei --- drivers/scsi/sd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 0e73c3f2f381..27f04c860f00 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -3670,7 +3670,7 @@ static void scsi_disk_release(struct device *dev) * in case multiple processes open a /dev/sd... node concurrently. */ blk_mq_freeze_queue(q); - blk_mq_unfreeze_queue(q); + __blk_mq_unfreeze_queue(q, true); disk->private_data = NULL; put_disk(disk); From patchwork Sat Jan 22 11:10:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720732 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E46FC433F5 for ; Sat, 22 Jan 2022 11:12:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234212AbiAVLMq (ORCPT ); Sat, 22 Jan 2022 06:12:46 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:25407 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234157AbiAVLMp (ORCPT ); Sat, 22 Jan 2022 06:12:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tqhve7JJhbDQBptqG2AcZsgYpgBMvqtn5u69ofC0wkI=; b=T+zWRfWH1HG8rEfHRLBwUD+h0KGM2lTDMI1KLR4uEXVUZIH5CmJxn1Y0ILeY2cLL1s0Wjx uJw0NAmsQO/rBfJH7BEyEzPzoKEKrUOAMW5xeJKHZBFpXD8q/JQiMaI2JZLZ+1oJqvtTwe hKUrKsOWm8KwNLE9ZfgyzlQBaY9mQlk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-639-9Rq3Ax0EPSiVOhMzJ1MIAA-1; Sat, 22 Jan 2022 06:12:39 -0500 X-MC-Unique: 9Rq3Ax0EPSiVOhMzJ1MIAA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5501918397B3; Sat, 22 Jan 2022 11:12:38 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8B1481081300; Sat, 22 Jan 2022 11:12:34 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 10/13] block: add helper of disk_release_queue for release queue data for disk Date: Sat, 22 Jan 2022 19:10:51 +0800 Message-Id: <20220122111054.1126146-11-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add one helper of disk_release_queue for releasing queue data for disk. Signed-off-by: Ming Lei --- block/genhd.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/block/genhd.c b/block/genhd.c index 5bd7bcd6246e..a86027619683 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -1085,6 +1085,20 @@ static const struct attribute_group *disk_attr_groups[] = { NULL }; +static void disk_release_queue(struct gendisk *disk) +{ + struct request_queue *q = disk->queue; + + blk_mq_cancel_work_sync(q); + + /* + * Remove all references to @q from the block cgroup controller before + * restoring @q->queue_lock to avoid that restoring this pointer causes + * e.g. blkcg_print_blkgs() to crash. + */ + blkcg_exit_queue(q); +} + /** * disk_release - releases all allocated resources of the gendisk * @dev: the device representing this disk @@ -1106,19 +1120,12 @@ static void disk_release(struct device *dev) might_sleep(); WARN_ON_ONCE(disk_live(disk)); - blk_mq_cancel_work_sync(disk->queue); + disk_release_queue(disk); disk_release_events(disk); kfree(disk->random); xa_destroy(&disk->part_tbl); - /* - * Remove all references to @q from the block cgroup controller before - * restoring @q->queue_lock to avoid that restoring this pointer causes - * e.g. blkcg_print_blkgs() to crash. - */ - blkcg_exit_queue(disk->queue); - disk->queue->disk = NULL; blk_put_queue(disk->queue); iput(disk->part0->bd_inode); /* frees the disk */ From patchwork Sat Jan 22 11:10:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6123CC433EF for ; Sat, 22 Jan 2022 11:12:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234273AbiAVLMs (ORCPT ); Sat, 22 Jan 2022 06:12:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:58267 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234157AbiAVLMr (ORCPT ); Sat, 22 Jan 2022 06:12:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849967; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+uWC3JQ3E8TC4lMrDyZZGJvPEZT304AfqmguiNRJ6m4=; b=SDG0Y/YEe4VoHevEBgigioJU8mARZ49OXQUMVExR4DSuwbQYtQo4r+KJVno8IFLseWLZvi LbAjPkl1UnhdrQzlnhZnSe9HJQK+txS21dv0a3dYmk4lTgj2S1RkvUo+7HgpNDhw3BOZId XcEpfC3TvID2NyNMtQU4B4/lfRyS2qY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-21-BTUij3deM8WgvB4okf_wWQ-1; Sat, 22 Jan 2022 06:12:43 -0500 X-MC-Unique: BTUij3deM8WgvB4okf_wWQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1303218397A7; Sat, 22 Jan 2022 11:12:42 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40DE27B9DC; Sat, 22 Jan 2022 11:12:40 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 11/13] block: move blk_exit_queue into disk_release Date: Sat, 22 Jan 2022 19:10:52 +0800 Message-Id: <20220122111054.1126146-12-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org There can't be FS IO in disk_release(), so move blk_exit_queue() there. We still need to freeze queue here since request is freed after bio is ended, meantime passthrough request relies on scheduler tags too, but either q->q_usage_counter is in atomic mode, such as scsi, or it has been drained already, such as most of other drivers, so the added queue freeze is pretty fast, and RCU grace period isn't supposed to be involved. disk can be released before or after queue is cleaned up, and we have to free scheduler request pool before blk_cleanup_queue returns, meantime the request pool has to be freed before exiting the scheduler. So move scheduler pool freeing into both the two functions. Signed-off-by: Ming Lei --- block/blk-mq.c | 3 +++ block/blk-sysfs.c | 16 ---------------- block/genhd.c | 39 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 41 insertions(+), 17 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index d51b0aa2e4e4..72ae9955cf27 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3101,6 +3101,9 @@ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags, struct blk_mq_tags *drv_tags; struct page *page; + if (list_empty(&tags->page_list)) + return; + if (blk_mq_is_shared_tags(set->flags)) drv_tags = set->shared_tags; else diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 5f14fd333182..e0f29b56e8e2 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -739,20 +739,6 @@ static void blk_free_queue_rcu(struct rcu_head *rcu_head) kmem_cache_free(blk_get_queue_kmem_cache(blk_queue_has_srcu(q)), q); } -/* Unconfigure the I/O scheduler and dissociate from the cgroup controller. */ -static void blk_exit_queue(struct request_queue *q) -{ - /* - * Since the I/O scheduler exit code may access cgroup information, - * perform I/O scheduler exit before disassociating from the block - * cgroup controller. - */ - if (q->elevator) { - ioc_clear_queue(q); - elevator_exit(q); - } -} - /** * blk_release_queue - releases all allocated resources of the request_queue * @kobj: pointer to a kobject, whose container is a request_queue @@ -786,8 +772,6 @@ static void blk_release_queue(struct kobject *kobj) blk_stat_remove_callback(q, q->poll_cb); blk_stat_free_callback(q->poll_cb); - blk_exit_queue(q); - blk_free_queue_stats(q->stats); kfree(q->poll_stat); diff --git a/block/genhd.c b/block/genhd.c index a86027619683..f1aef5d13afa 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -1085,11 +1085,48 @@ static const struct attribute_group *disk_attr_groups[] = { NULL }; +/* Unconfigure the I/O scheduler and dissociate from the cgroup controller. */ +static void blk_exit_queue(struct request_queue *q) +{ + /* + * Since the I/O scheduler exit code may access cgroup information, + * perform I/O scheduler exit before disassociating from the block + * cgroup controller. + */ + if (q->elevator) { + ioc_clear_queue(q); + + mutex_lock(&q->sysfs_lock); + blk_mq_sched_free_rqs(q); + elevator_exit(q); + mutex_unlock(&q->sysfs_lock); + } +} + static void disk_release_queue(struct gendisk *disk) { struct request_queue *q = disk->queue; - blk_mq_cancel_work_sync(q); + if (queue_is_mq(q)) { + blk_mq_cancel_work_sync(q); + + /* + * All FS bios have been done, however FS request may not + * be freed yet since we end bio before freeing request, + * meantime passthrough request replies on scheduler tags, + * so queue needs to be frozen here. + * + * Most of drivers release disk after blk_cleanup_queue() + * returns, and SCSI may release disk before calling + * blk_cleanup_queue, but request queue has been in atomic + * mode already, see scsi_disk_release(), so the following + * queue freeze is pretty fast, and RCU grace period isn't + * supposed to be involved. + */ + blk_mq_freeze_queue(q); + blk_exit_queue(q); + __blk_mq_unfreeze_queue(q, true); + } /* * Remove all references to @q from the block cgroup controller before From patchwork Sat Jan 22 11:10:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95F07C433EF for ; Sat, 22 Jan 2022 11:12:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230053AbiAVLMv (ORCPT ); Sat, 22 Jan 2022 06:12:51 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:58143 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234276AbiAVLMt (ORCPT ); Sat, 22 Jan 2022 06:12:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qirubjWy4NpcVODIc9rAZROQONujRbVp9oXqOYfiJYE=; b=N5JMxV94UMk5lAMhjuVOhE8QIax+bO9KpcCkcvIqtc4BeclYBPgBznEgdQhtYvkQN5P7Xx +nP/H8qXq+/7+eFagJW5i+BD5AtpbCr9BTKmUnEIJXgBCjYgeNAx9gjlz5GNwHozLTE0Yz UEys1aFos55DdgTCOsg7ZCUxoaMjKL4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-653-jlWM71YgOaKt1P9L5_8vIQ-1; Sat, 22 Jan 2022 06:12:46 -0500 X-MC-Unique: jlWM71YgOaKt1P9L5_8vIQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 18906814245; Sat, 22 Jan 2022 11:12:45 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 408711081300; Sat, 22 Jan 2022 11:12:43 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 12/13] block: move rq_qos_exit() into disk_release() Date: Sat, 22 Jan 2022 19:10:53 +0800 Message-Id: <20220122111054.1126146-13-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org There can't be FS IO in disk_release(), so it is safe to move rq_qos_exit() there. Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig --- block/genhd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/genhd.c b/block/genhd.c index f1aef5d13afa..2f0e92cdcf6d 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -622,7 +622,6 @@ void del_gendisk(struct gendisk *disk) blk_mq_freeze_queue_wait(q); - rq_qos_exit(q); blk_sync_queue(q); blk_flush_integrity(); /* @@ -1125,6 +1124,7 @@ static void disk_release_queue(struct gendisk *disk) */ blk_mq_freeze_queue(q); blk_exit_queue(q); + rq_qos_exit(q); __blk_mq_unfreeze_queue(q, true); } From patchwork Sat Jan 22 11:10:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12720735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BADCC433EF for ; Sat, 22 Jan 2022 11:12:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234277AbiAVLM6 (ORCPT ); Sat, 22 Jan 2022 06:12:58 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24176 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230228AbiAVLM6 (ORCPT ); Sat, 22 Jan 2022 06:12:58 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642849977; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G+NRXXc+hwDjuvMZtnwreB6eugOj8g0/jJ/eXBY93LM=; b=FuLc8ZE9VS9Vr+H2iyj+cEwBwNm+ToJlLdWK9Rcg0gWcmmhP+VxwwK2sXCR1PKsnRNizdw 7fE1T8Ru2W6q7NNEazMiL/8ekerY6jchG9exa8rwQQ2lmLs0nt2gls+Cq9beVxhNUUI/Sz MOREVcOF5CAmBGEN/OQeMxbV0Fg+Wjk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-222-dQCM9ynzOlCq1WlgVh7rGg-1; Sat, 22 Jan 2022 06:12:52 -0500 X-MC-Unique: dQCM9ynzOlCq1WlgVh7rGg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B281818397A7; Sat, 22 Jan 2022 11:12:51 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id F0E7D752CF; Sat, 22 Jan 2022 11:12:47 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Jens Axboe , "Martin K . Petersen" Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Ming Lei Subject: [PATCH V2 13/13] block: don't drain file system I/O on del_gendisk Date: Sat, 22 Jan 2022 19:10:54 +0800 Message-Id: <20220122111054.1126146-14-ming.lei@redhat.com> In-Reply-To: <20220122111054.1126146-1-ming.lei@redhat.com> References: <20220122111054.1126146-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Commit 8e141f9eb803 ("block: drain file system I/O on del_gendisk") is added for avoiding to reference of q->disk after disk is released, since disk release can be done before running into blk_cleanup_queue(), but it can't avoid the issue completely, and the approach is very fragile: 1) queue freezing can't drain FS I/O for bio based driver, so there is still inflight bios after disk is deleted, so blk-cgroup shutdown can't be moved to del_gendisk(); meantime elevator shutdown can't be moved to del_gendisk() because freezing queue isn't enough to drain io issue activities. So both block-cgroup and elevator code may still refer to q->disk somewhere. Same with q->disk referring for bio based driver. 2) the added flag of GD_DEAD may not be observed reliably in __bio_queue_enter() because queue freezing might not imply rcu grace period. Now we have moved blkcg, rqos and elevator shutdown code into disk_release() where it is guaranteed that no any FS IO is inflight. Meantime we only account passthrough IO issued from userspace, where q->disk is always valid. Then q->disk can be guaranteed to be referred before releasing disk. So not necessary to drain FS I/O in del_gendisk() any more. Signed-off-by: Ming Lei --- block/blk-core.c | 21 +++++++++------------ block/blk.h | 1 - block/genhd.c | 21 --------------------- include/linux/genhd.h | 1 - 4 files changed, 9 insertions(+), 35 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index bcb4d982cd80..2216360165c9 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -270,8 +270,10 @@ void blk_put_queue(struct request_queue *q) } EXPORT_SYMBOL(blk_put_queue); -void blk_queue_start_drain(struct request_queue *q) +void blk_set_queue_dying(struct request_queue *q) { + blk_queue_flag_set(QUEUE_FLAG_DYING, q); + /* * When queue DYING flag is set, we need to block new req * entering queue, so we call blk_freeze_queue_start() to @@ -283,12 +285,6 @@ void blk_queue_start_drain(struct request_queue *q) /* Make blk_queue_enter() reexamine the DYING flag. */ wake_up_all(&q->mq_freeze_wq); } - -void blk_set_queue_dying(struct request_queue *q) -{ - blk_queue_flag_set(QUEUE_FLAG_DYING, q); - blk_queue_start_drain(q); -} EXPORT_SYMBOL_GPL(blk_set_queue_dying); /** @@ -322,6 +318,9 @@ void blk_cleanup_queue(struct request_queue *q) blk_queue_flag_set(QUEUE_FLAG_DEAD, q); + /* for synchronous bio-based driver finish in-flight integrity i/o */ + blk_flush_integrity(); + blk_sync_queue(q); if (queue_is_mq(q)) { blk_mq_cancel_work_sync(q); @@ -381,10 +380,8 @@ int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags) int __bio_queue_enter(struct request_queue *q, struct bio *bio) { while (!blk_try_enter_queue(q, false)) { - struct gendisk *disk = bio->bi_bdev->bd_disk; - if (bio->bi_opf & REQ_NOWAIT) { - if (test_bit(GD_DEAD, &disk->state)) + if (blk_queue_dying(q)) goto dead; bio_wouldblock_error(bio); return -EBUSY; @@ -401,8 +398,8 @@ int __bio_queue_enter(struct request_queue *q, struct bio *bio) wait_event(q->mq_freeze_wq, (!q->mq_freeze_depth && blk_pm_resume_queue(false, q)) || - test_bit(GD_DEAD, &disk->state)); - if (test_bit(GD_DEAD, &disk->state)) + blk_queue_dying(q)); + if (blk_queue_dying(q)) goto dead; } diff --git a/block/blk.h b/block/blk.h index a038e25d8637..6adedf97cba2 100644 --- a/block/blk.h +++ b/block/blk.h @@ -43,7 +43,6 @@ struct blk_flush_queue *blk_alloc_flush_queue(int node, int cmd_size, void blk_free_flush_queue(struct blk_flush_queue *q); void blk_freeze_queue(struct request_queue *q); -void blk_queue_start_drain(struct request_queue *q); int __bio_queue_enter(struct request_queue *q, struct bio *bio); bool submit_bio_checks(struct bio *bio); diff --git a/block/genhd.c b/block/genhd.c index 2f0e92cdcf6d..90742679f949 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -569,8 +569,6 @@ EXPORT_SYMBOL(device_add_disk); */ void del_gendisk(struct gendisk *disk) { - struct request_queue *q = disk->queue; - might_sleep(); if (WARN_ON_ONCE(!disk_live(disk) && !(disk->flags & GENHD_FL_HIDDEN))) @@ -587,17 +585,8 @@ void del_gendisk(struct gendisk *disk) fsync_bdev(disk->part0); __invalidate_device(disk->part0, true); - /* - * Fail any new I/O. - */ - set_bit(GD_DEAD, &disk->state); set_capacity(disk, 0); - /* - * Prevent new I/O from crossing bio_queue_enter(). - */ - blk_queue_start_drain(q); - if (!(disk->flags & GENHD_FL_HIDDEN)) { sysfs_remove_link(&disk_to_dev(disk)->kobj, "bdi"); @@ -619,16 +608,6 @@ void del_gendisk(struct gendisk *disk) sysfs_remove_link(block_depr, dev_name(disk_to_dev(disk))); pm_runtime_set_memalloc_noio(disk_to_dev(disk), false); device_del(disk_to_dev(disk)); - - blk_mq_freeze_queue_wait(q); - - blk_sync_queue(q); - blk_flush_integrity(); - /* - * Allow using passthrough request again after the queue is torn down. - */ - __blk_mq_unfreeze_queue(q, true); - } EXPORT_SYMBOL(del_gendisk); diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 6906a45bc761..3e9f234495e4 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -108,7 +108,6 @@ struct gendisk { unsigned long state; #define GD_NEED_PART_SCAN 0 #define GD_READ_ONLY 1 -#define GD_DEAD 2 #define GD_NATIVE_CAPACITY 3 struct mutex open_mutex; /* open/close mutex */