From patchwork Sat Jan 4 13:25:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Le Moal X-Patchwork-Id: 13926253 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA4BF1B0430 for ; Sat, 4 Jan 2025 13:26:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735997160; cv=none; b=pYBor19lDyX07GcPRgQfsLS1IJpS8gLw/yAZprRQwV7g/Yua9p1arQH/UY5JQiqDu+xiyuXw2tXcSwUOB6l158MlT23MN4sm4vaqUIV68Alf52Wmt0Lum791yhoSe/ZfzI0umwloH177CWiWBeE/e11+fPdoLhZHDCSz33RNZPs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735997160; c=relaxed/simple; bh=E2AvZuaVVt3QyD9lOW0ANW+D6nr9wEq3tnQpouzbBVc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=t/YBqzrm/dhZhKBD9h/QPitsVBHy0lruxwVkgT6lO7SWgn/Uz4Fffr0MXh3PTz3RbWNJxlHUS12qFAejpIgswH8PgXluxlz6niWDTYGXN0jo/787Ya9VBVU4XkU8FHSb+bUD3xtWHCS2KVzKH/Baa6sOZm81BmGfvpb5Itu9vvU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=utxHbUAf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="utxHbUAf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5257FC4CED2; Sat, 4 Jan 2025 13:25:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735997160; bh=E2AvZuaVVt3QyD9lOW0ANW+D6nr9wEq3tnQpouzbBVc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=utxHbUAflQo/AnvMyQFHt1NXHhrF2orzzVfqTiHWN9YEWzhU48fmu4oZfXgPn4eCG jQRCVIl/RBrVhPWjG9DPAcPwa/LR67uVPRwCXp8BtyOFbIdsMwoOaCAY8MHv9kDMK9 Z7FU45euRTtGYs9H+A1g4AKk/VpwMcn2mg9SC8YNE0d5zjihS+tInn+OySMkcZ2Hiu Mv+gm7xKCtUVf5wi6qbqD1+PB4Q4JJldJHACG+nck3XgWzQMDEtFHzkZwAlRkviouF lqZl/6B1+ZNF0yATttiPOo30xu+QPyWmsldOMNOZuUsZ+waLcXfVCdzubHEdwSNPeS UGZmUgT5SF7zA== From: Damien Le Moal To: Jens Axboe , linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: Ming Lei , Nilay Shroff Subject: [PATCH 1/3] block: Fix sysfs queue freeze and limits lock order Date: Sat, 4 Jan 2025 22:25:20 +0900 Message-ID: <20250104132522.247376-2-dlemoal@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250104132522.247376-1-dlemoal@kernel.org> References: <20250104132522.247376-1-dlemoal@kernel.org> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 queue_attr_store() always freezes a device queue before calling the attribute store operation. For attributes that control queue limits, the store operation will also lock the queue limits with a call to queue_limits_start_update(). However, some drivers (e.g. SCSI sd) may need to issue commands to a device to obtain limit values from the hardware with the queue limits locked. This creates a potential ABBA deadlock situation if a user attempts to modify a limit (thus freezing the device queue) while the device driver starts a revalidation of the device queue limits. Avoid such deadlock by introducing the ->store_limit() operation in struct queue_sysfs_entry and use this operation for all attributes that modify the device queue limits through the QUEUE_RW_LIMIT_ENTRY() macro definition. queue_attr_store() is modified to call the ->store_limit() operation (if it is defined) without the device queue frozen. The device queue freeze for attributes defining the ->stor_limit() operation is moved to after the operation completes and is done only around the call to queue_limits_commit_update(). Cc: stable@vger.kernel.org # v6.9+ Signed-off-by: Damien Le Moal --- block/blk-sysfs.c | 123 ++++++++++++++++++++++++---------------------- 1 file changed, 64 insertions(+), 59 deletions(-) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 767598e719ab..4fc0020c73a5 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -24,6 +24,8 @@ struct queue_sysfs_entry { struct attribute attr; ssize_t (*show)(struct gendisk *disk, char *page); ssize_t (*store)(struct gendisk *disk, const char *page, size_t count); + ssize_t (*store_limit)(struct gendisk *disk, struct queue_limits *lim, + const char *page, size_t count); void (*load_module)(struct gendisk *disk, const char *page, size_t count); }; @@ -154,55 +156,46 @@ QUEUE_SYSFS_SHOW_CONST(write_same_max, 0) QUEUE_SYSFS_SHOW_CONST(poll_delay, -1) static ssize_t queue_max_discard_sectors_store(struct gendisk *disk, - const char *page, size_t count) + struct queue_limits *lim, const char *page, size_t count) { unsigned long max_discard_bytes; - struct queue_limits lim; ssize_t ret; - int err; ret = queue_var_store(&max_discard_bytes, page, count); if (ret < 0) return ret; - if (max_discard_bytes & (disk->queue->limits.discard_granularity - 1)) + if (max_discard_bytes & (lim->discard_granularity - 1)) return -EINVAL; if ((max_discard_bytes >> SECTOR_SHIFT) > UINT_MAX) return -EINVAL; - lim = queue_limits_start_update(disk->queue); - lim.max_user_discard_sectors = max_discard_bytes >> SECTOR_SHIFT; - err = queue_limits_commit_update(disk->queue, &lim); - if (err) - return err; - return ret; + lim->max_user_discard_sectors = max_discard_bytes >> SECTOR_SHIFT; + + return count; } static ssize_t -queue_max_sectors_store(struct gendisk *disk, const char *page, size_t count) +queue_max_sectors_store(struct gendisk *disk, struct queue_limits *lim, + const char *page, size_t count) { unsigned long max_sectors_kb; - struct queue_limits lim; ssize_t ret; - int err; ret = queue_var_store(&max_sectors_kb, page, count); if (ret < 0) return ret; - lim = queue_limits_start_update(disk->queue); - lim.max_user_sectors = max_sectors_kb << 1; - err = queue_limits_commit_update(disk->queue, &lim); - if (err) - return err; - return ret; + lim->max_user_sectors = max_sectors_kb << 1; + + return count; } -static ssize_t queue_feature_store(struct gendisk *disk, const char *page, +static ssize_t queue_feature_store(struct gendisk *disk, + struct queue_limits *lim, const char *page, size_t count, blk_features_t feature) { - struct queue_limits lim; unsigned long val; ssize_t ret; @@ -210,14 +203,10 @@ static ssize_t queue_feature_store(struct gendisk *disk, const char *page, if (ret < 0) return ret; - lim = queue_limits_start_update(disk->queue); if (val) - lim.features |= feature; + lim->features |= feature; else - lim.features &= ~feature; - ret = queue_limits_commit_update(disk->queue, &lim); - if (ret) - return ret; + lim->features &= ~feature; return count; } @@ -228,9 +217,10 @@ static ssize_t queue_##_name##_show(struct gendisk *disk, char *page) \ !!(disk->queue->limits.features & _feature)); \ } \ static ssize_t queue_##_name##_store(struct gendisk *disk, \ + struct queue_limits *lim, \ const char *page, size_t count) \ { \ - return queue_feature_store(disk, page, count, _feature); \ + return queue_feature_store(disk, lim, page, count, _feature); \ } QUEUE_SYSFS_FEATURE(rotational, BLK_FEAT_ROTATIONAL) @@ -267,9 +257,8 @@ static ssize_t queue_iostats_passthrough_show(struct gendisk *disk, char *page) } static ssize_t queue_iostats_passthrough_store(struct gendisk *disk, - const char *page, size_t count) + struct queue_limits *lim, const char *page, size_t count) { - struct queue_limits lim; unsigned long ios; ssize_t ret; @@ -277,15 +266,10 @@ static ssize_t queue_iostats_passthrough_store(struct gendisk *disk, if (ret < 0) return ret; - lim = queue_limits_start_update(disk->queue); if (ios) - lim.flags |= BLK_FLAG_IOSTATS_PASSTHROUGH; + lim->flags |= BLK_FLAG_IOSTATS_PASSTHROUGH; else - lim.flags &= ~BLK_FLAG_IOSTATS_PASSTHROUGH; - - ret = queue_limits_commit_update(disk->queue, &lim); - if (ret) - return ret; + lim->flags &= ~BLK_FLAG_IOSTATS_PASSTHROUGH; return count; } @@ -391,12 +375,10 @@ static ssize_t queue_wc_show(struct gendisk *disk, char *page) return sysfs_emit(page, "write through\n"); } -static ssize_t queue_wc_store(struct gendisk *disk, const char *page, - size_t count) +static ssize_t queue_wc_store(struct gendisk *disk, struct queue_limits *lim, + const char *page, size_t count) { - struct queue_limits lim; bool disable; - int err; if (!strncmp(page, "write back", 10)) { disable = false; @@ -407,14 +389,10 @@ static ssize_t queue_wc_store(struct gendisk *disk, const char *page, return -EINVAL; } - lim = queue_limits_start_update(disk->queue); if (disable) - lim.flags |= BLK_FLAG_WRITE_CACHE_DISABLED; + lim->flags |= BLK_FLAG_WRITE_CACHE_DISABLED; else - lim.flags &= ~BLK_FLAG_WRITE_CACHE_DISABLED; - err = queue_limits_commit_update(disk->queue, &lim); - if (err) - return err; + lim->flags &= ~BLK_FLAG_WRITE_CACHE_DISABLED; return count; } @@ -439,9 +417,16 @@ static struct queue_sysfs_entry _prefix##_entry = { \ .store = _prefix##_store, \ } +#define QUEUE_RW_LIMIT_ENTRY(_prefix, _name) \ +static struct queue_sysfs_entry _prefix##_entry = { \ + .attr = { .name = _name, .mode = 0644 }, \ + .show = _prefix##_show, \ + .store_limit = _prefix##_store, \ +} + QUEUE_RW_ENTRY(queue_requests, "nr_requests"); QUEUE_RW_ENTRY(queue_ra, "read_ahead_kb"); -QUEUE_RW_ENTRY(queue_max_sectors, "max_sectors_kb"); +QUEUE_RW_LIMIT_ENTRY(queue_max_sectors, "max_sectors_kb"); QUEUE_RO_ENTRY(queue_max_hw_sectors, "max_hw_sectors_kb"); QUEUE_RO_ENTRY(queue_max_segments, "max_segments"); QUEUE_RO_ENTRY(queue_max_integrity_segments, "max_integrity_segments"); @@ -457,7 +442,7 @@ QUEUE_RO_ENTRY(queue_io_opt, "optimal_io_size"); QUEUE_RO_ENTRY(queue_max_discard_segments, "max_discard_segments"); QUEUE_RO_ENTRY(queue_discard_granularity, "discard_granularity"); QUEUE_RO_ENTRY(queue_max_hw_discard_sectors, "discard_max_hw_bytes"); -QUEUE_RW_ENTRY(queue_max_discard_sectors, "discard_max_bytes"); +QUEUE_RW_LIMIT_ENTRY(queue_max_discard_sectors, "discard_max_bytes"); QUEUE_RO_ENTRY(queue_discard_zeroes_data, "discard_zeroes_data"); QUEUE_RO_ENTRY(queue_atomic_write_max_sectors, "atomic_write_max_bytes"); @@ -477,11 +462,11 @@ QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones"); QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones"); QUEUE_RW_ENTRY(queue_nomerges, "nomerges"); -QUEUE_RW_ENTRY(queue_iostats_passthrough, "iostats_passthrough"); +QUEUE_RW_LIMIT_ENTRY(queue_iostats_passthrough, "iostats_passthrough"); QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity"); QUEUE_RW_ENTRY(queue_poll, "io_poll"); QUEUE_RW_ENTRY(queue_poll_delay, "io_poll_delay"); -QUEUE_RW_ENTRY(queue_wc, "write_cache"); +QUEUE_RW_LIMIT_ENTRY(queue_wc, "write_cache"); QUEUE_RO_ENTRY(queue_fua, "fua"); QUEUE_RO_ENTRY(queue_dax, "dax"); QUEUE_RW_ENTRY(queue_io_timeout, "io_timeout"); @@ -494,10 +479,10 @@ static struct queue_sysfs_entry queue_hw_sector_size_entry = { .show = queue_logical_block_size_show, }; -QUEUE_RW_ENTRY(queue_rotational, "rotational"); -QUEUE_RW_ENTRY(queue_iostats, "iostats"); -QUEUE_RW_ENTRY(queue_add_random, "add_random"); -QUEUE_RW_ENTRY(queue_stable_writes, "stable_writes"); +QUEUE_RW_LIMIT_ENTRY(queue_rotational, "rotational"); +QUEUE_RW_LIMIT_ENTRY(queue_iostats, "iostats"); +QUEUE_RW_LIMIT_ENTRY(queue_add_random, "add_random"); +QUEUE_RW_LIMIT_ENTRY(queue_stable_writes, "stable_writes"); #ifdef CONFIG_BLK_WBT static ssize_t queue_var_store64(s64 *var, const char *page) @@ -693,9 +678,11 @@ queue_attr_store(struct kobject *kobj, struct attribute *attr, struct queue_sysfs_entry *entry = to_queue(attr); struct gendisk *disk = container_of(kobj, struct gendisk, queue_kobj); struct request_queue *q = disk->queue; + struct queue_limits lim = { }; ssize_t res; + int ret; - if (!entry->store) + if (!entry->store && !entry->store_limit) return -EIO; /* @@ -706,12 +693,30 @@ queue_attr_store(struct kobject *kobj, struct attribute *attr, if (entry->load_module) entry->load_module(disk, page, length); - blk_mq_freeze_queue(q); + if (entry->store) { + blk_mq_freeze_queue(q); + mutex_lock(&q->sysfs_lock); + res = entry->store(disk, page, length); + mutex_unlock(&q->sysfs_lock); + blk_mq_unfreeze_queue(q); + return res; + } + + lim = queue_limits_start_update(q); + mutex_lock(&q->sysfs_lock); - res = entry->store(disk, page, length); + res = entry->store_limit(disk, &lim, page, length); mutex_unlock(&q->sysfs_lock); + if (res < 0) { + queue_limits_cancel_update(q); + return res; + } + + blk_mq_freeze_queue(q); + ret = queue_limits_commit_update(disk->queue, &lim); blk_mq_unfreeze_queue(q); - return res; + + return ret ? ret : res; } static const struct sysfs_ops queue_sysfs_ops = { From patchwork Sat Jan 4 13:25:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Le Moal X-Patchwork-Id: 13926254 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2003D1B6CE3 for ; Sat, 4 Jan 2025 13:26:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735997162; cv=none; b=Bbwa2gp1Fq+djoaKMrYcrdKMP4EFbwZA45MqEpRxT8zn2ruavtd3iDTq69+TxO4jeRK6LA/tSff8siV1/8InZkQphIqaDdj8BNEbfbhSHocaS3dHRcLqUD6gR0oxVMW2zsXLk3ymfie6uwOxZhv//Y6FiVWxL+1zWXAlVJ2KCuc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735997162; c=relaxed/simple; bh=2am3FapwRA8t0IL3zu3UkrLfm2C8ws46MF9QDum4EY4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aPT0gH+mP7fIg0gFZm2u3eF9CCiFrG21qPaPH1WudePn6KoB1BGAuWV2cVgKdLPaJHn+ONgktFgm3zTUWu/pXJH4nr/8zY58+lNLPxUsIqXvqqwKyJmwXR8xMeoBCzshs3fkieH1ksSD2bVchXZdZnjQcsKhYW1OBV3EC/DKkb4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ets+EtRs; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ets+EtRs" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A5884C4CEDF; Sat, 4 Jan 2025 13:26:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735997161; bh=2am3FapwRA8t0IL3zu3UkrLfm2C8ws46MF9QDum4EY4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ets+EtRsm2yOvtZgSBvtRg44/RkWIIjTwc1QUHKg3FukZWGr5hBbeULLCDiyKvGzj gsePPpSPaHYYgJ5EpCVw++oj2bi844ygJONT4r+UbloUhMaKXGOmW3Ap6KxmLiy09T BFRInbln8ySSDa2eg8FozL/TjJokO/c90u17eueeWGfXCZnsNvpPmzUEFNqk3Dr+Vi cuZDU65qIQuP0XoREtRLc7++Ahe8HTgxQQX/Yb4VX/nGbph6O8Kmt5jjpp7op4GYWS QkUXnf53m9sfXBktIwjpJo16wa59D/PEFB3v2gQ6sD7vk1awUUBdTOjazidEqFIGN8 6qdB3ZkCxJC0A== From: Damien Le Moal To: Jens Axboe , linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: Ming Lei , Nilay Shroff Subject: [PATCH 2/3] block: Fix __blk_mq_update_nr_hw_queues() queue freeze and limits lock order Date: Sat, 4 Jan 2025 22:25:21 +0900 Message-ID: <20250104132522.247376-3-dlemoal@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250104132522.247376-1-dlemoal@kernel.org> References: <20250104132522.247376-1-dlemoal@kernel.org> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 __blk_mq_update_nr_hw_queues() freezes a device queues during operation, which also includes updating the BLK_FEAT_POLL feature flag for the device queues using queue_limits_start_update() and queue_limits_commit_update(). This call order thus creates an invalid ordering of a queue freeze and queue limit locking which can lead to a deadlock when the device driver must issue commands to probe the device when revalidating its limits. Avoid this issue by moving the update of the BLK_FEAT_POLL feature flag out of the main queue remapping loop to the end of __blk_mq_update_nr_hw_queues(), after the device queues have been unfrozen. Fixes: 8023e144f9d6 ("block: move the poll flag to queue_limits") Cc: stable@vger.kernel.org Signed-off-by: Damien Le Moal --- block/blk-mq.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 8ac19d4ae3c0..1f63067790c8 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -5021,8 +5021,6 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, fallback: blk_mq_update_queue_map(set); list_for_each_entry(q, &set->tag_list, tag_set_list) { - struct queue_limits lim; - blk_mq_realloc_hw_ctxs(set, q); if (q->nr_hw_queues != set->nr_hw_queues) { @@ -5036,13 +5034,6 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, set->nr_hw_queues = prev_nr_hw_queues; goto fallback; } - lim = queue_limits_start_update(q); - if (blk_mq_can_poll(set)) - lim.features |= BLK_FEAT_POLL; - else - lim.features &= ~BLK_FEAT_POLL; - if (queue_limits_commit_update(q, &lim) < 0) - pr_warn("updating the poll flag failed\n"); blk_mq_map_swqueue(q); } @@ -5059,6 +5050,22 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, list_for_each_entry(q, &set->tag_list, tag_set_list) blk_mq_unfreeze_queue(q); + list_for_each_entry(q, &set->tag_list, tag_set_list) { + struct queue_limits lim; + int ret; + + lim = queue_limits_start_update(q); + if (blk_mq_can_poll(set)) + lim.features |= BLK_FEAT_POLL; + else + lim.features &= ~BLK_FEAT_POLL; + blk_mq_freeze_queue(q); + ret = queue_limits_commit_update(q, &lim); + blk_mq_unfreeze_queue(q); + if (ret < 0) + pr_warn("updating the poll flag failed\n"); + } + /* Free the excess tags when nr_hw_queues shrink. */ for (i = set->nr_hw_queues; i < prev_nr_hw_queues; i++) __blk_mq_free_map_and_rqs(set, i); From patchwork Sat Jan 4 13:25:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Le Moal X-Patchwork-Id: 13926255 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 723341B0430 for ; Sat, 4 Jan 2025 13:26:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735997163; cv=none; b=BWVwvwIzJ0sZfKnXi3VBvOPyE03vQEKjlW0zioCLdJ0DL2ZhkX9QBZDDjkrR7YBGkscR4UJgR0ujSuUtkE99ngwgciBQFGquKpIx/53juUmICDo8op19U1I1sEbVodnBt5WcOgxwWrT6NMFms8v85j8NyaiqNNJE8T7PW4pdDp8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735997163; c=relaxed/simple; bh=COVE0bvwII+G60ByWk+rzuxwByQi7P/2T3NJasfjEeo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PgUG5bumXxJ1rVLNDpkZcDbIFE4+64OoKB227shO1VCVecg/Nmnl3dFWQ4sp8YtdrLSd+tpLd31f62G/cIhkqGtRKE7oehNtoVpZa+K/BNrDWJoUqGnMF8bsXD6ApdF67dpa1R/yO4AFBbmalToMLuSB3raviFAPi6M3U2RTSkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Z7YAIA0y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Z7YAIA0y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 058DBC4CED1; Sat, 4 Jan 2025 13:26:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735997163; bh=COVE0bvwII+G60ByWk+rzuxwByQi7P/2T3NJasfjEeo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z7YAIA0yjkNnWkkNU5CJThD0r/hOGucsvMxakh/NbL2tf7+2fhE6wXGm9bt4ISPJR nwwH+7AoFGdLr//bXhUkf7Bz7Xqshzh3oEmFu78I6dCFxtO/TqzJ/gBrJA/1YiN0Xf qChanWG2U73ONHEKYEbFhxhu2VFQBFnlSvQU69dTdnFmJC6jQqqDI4FKB11fRXOzC2 RcewRngEPM7BoRliPwgIWnbj5R1Kq/MtivgFWZ3tekVKFZoCmjLAmVEHhoDrD/UpBh 6S+kWywgZ62duvbsNYe73TpFp7DfftjcKhgVXEqDv+YAmQuy2BxeVgMsAWJYPq0w/n uw+O88tgm2gew== From: Damien Le Moal To: Jens Axboe , linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: Ming Lei , Nilay Shroff Subject: [PATCH 3/3] nvme: Fix queue freeze and limits lock order Date: Sat, 4 Jan 2025 22:25:22 +0900 Message-ID: <20250104132522.247376-4-dlemoal@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250104132522.247376-1-dlemoal@kernel.org> References: <20250104132522.247376-1-dlemoal@kernel.org> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Modify the functions nvme_update_ns_info_generic(), nvme_update_ns_info_block() and nvme_update_ns_info() to freeze a namespace queue using blk_mq_freeze_queue() after starting the queue limits update with queue_limits_start_update() so that the queue freezing is always done after obtaining the device queue limits lock (as per the block layer convention for sysfs attributes). Fixes: e6c9b130d681 ("nvme: use the atomic queue limits update API") Cc: stable@vger.kernel.org Signed-off-by: Damien Le Moal --- drivers/nvme/host/core.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index a970168a3014..8d4ae36c35fc 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -2128,8 +2128,8 @@ static int nvme_update_ns_info_generic(struct nvme_ns *ns, struct queue_limits lim; int ret; - blk_mq_freeze_queue(ns->disk->queue); lim = queue_limits_start_update(ns->disk->queue); + blk_mq_freeze_queue(ns->disk->queue); nvme_set_ctrl_limits(ns->ctrl, &lim); ret = queue_limits_commit_update(ns->disk->queue, &lim); set_disk_ro(ns->disk, nvme_ns_is_readonly(ns, info)); @@ -2177,12 +2177,13 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, goto out; } + lim = queue_limits_start_update(ns->disk->queue); + blk_mq_freeze_queue(ns->disk->queue); ns->head->lba_shift = id->lbaf[lbaf].ds; ns->head->nuse = le64_to_cpu(id->nuse); capacity = nvme_lba_to_sect(ns->head, le64_to_cpu(id->nsze)); - lim = queue_limits_start_update(ns->disk->queue); nvme_set_ctrl_limits(ns->ctrl, &lim); nvme_configure_metadata(ns->ctrl, ns->head, id, nvm, info); nvme_set_chunk_sectors(ns, id, &lim); @@ -2285,6 +2286,8 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_ns_info *info) struct queue_limits *ns_lim = &ns->disk->queue->limits; struct queue_limits lim; + lim = queue_limits_start_update(ns->head->disk->queue); + blk_mq_freeze_queue(ns->head->disk->queue); /* * queue_limits mixes values that are the hardware limitations @@ -2301,7 +2304,6 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_ns_info *info) * the splitting limits in to make sure we still obey possibly * lower limitations of other controllers. */ - lim = queue_limits_start_update(ns->head->disk->queue); lim.logical_block_size = ns_lim->logical_block_size; lim.physical_block_size = ns_lim->physical_block_size; lim.io_min = ns_lim->io_min;