From patchwork Thu Apr 10 13:30:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046512 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BAF878F4E for ; Thu, 10 Apr 2025 13:31:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291864; cv=none; b=AyORMQEaHm5iaDuQqb5bV0yuQwCGhbj9KWJ+a/MQWKzY9iFnCvYHJy371Yj5QcXugjiDfVGqqPIEXlQaXPaAfQ5xW86GITXP2urcUT30LIanL6idKSgjRqn2JDneCmXsxMQwfx+NpZQcOVgUivnUZFG9fOlwDdGsOnxso39OC+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291864; c=relaxed/simple; bh=bavUu1Lt/jALufmHuyOkgx+9KGl02TYwcyjDVjt+QC0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q0w2FkYc5GQtOdTwjDPjvIgtHNBV0o1pV01tbVvc4zH0/1kr1jlmKnl134/E+hVmFlXj456KgR6gT08Z+Q+A0arR1OIgx5LoT/+d00llksJdFvknKWNqI6Xu5n9YHSPLgYHiOxpeSE1ody85n3amgZGnNa8ier/0XCniX5NPqRQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=XJPLCJMK; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XJPLCJMK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291862; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3HInUZ1JgSSKLSxNX9tbFbkA6LH4NVwRZynu5inN9PA=; b=XJPLCJMK2JsdLqk2t2Tl2ZKLL9D5QA1YAYtVF5MPI3dYNm+szGwkREOJv6fNL0jsehu2N1 bMjmUwN4AVjvkurhNVTlZk1jVlzwvz0a2/RIpZLNGV7JgRhyEekX+rUraujqN30tbPn5dj Kiqf3zK5dY+hhPuDTJBnHnL2svd2Uv8= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-648-A3rwYZo0OiaegcekcDJp_g-1; Thu, 10 Apr 2025 09:30:57 -0400 X-MC-Unique: A3rwYZo0OiaegcekcDJp_g-1 X-Mimecast-MFC-AGG-ID: A3rwYZo0OiaegcekcDJp_g_1744291855 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C68E81955DCE; Thu, 10 Apr 2025 13:30:54 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 909361828A9E; Thu, 10 Apr 2025 13:30:52 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 01/15] block: don't call freeze queue in elevator_switch() and elevator_disable() Date: Thu, 10 Apr 2025 21:30:13 +0800 Message-ID: <20250410133029.2487054-2-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Both elevator_switch() and elevator_disable() are called from sysfs store and updating nr_hw_queue code paths only. And in the two code paths, queue has been frozen already, so don't call freeze queue in the two functions. Reviewed-by: Nilay Shroff Signed-off-by: Ming Lei --- block/elevator.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/block/elevator.c b/block/elevator.c index b4d08026b02c..5051a98dc08c 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -615,12 +615,11 @@ void elevator_init_mq(struct request_queue *q) */ int elevator_switch(struct request_queue *q, struct elevator_type *new_e) { - unsigned int memflags; int ret; + WARN_ON_ONCE(q->mq_freeze_depth == 0); lockdep_assert_held(&q->elevator_lock); - memflags = blk_mq_freeze_queue(q); blk_mq_quiesce_queue(q); if (q->elevator) { @@ -641,7 +640,6 @@ int elevator_switch(struct request_queue *q, struct elevator_type *new_e) out_unfreeze: blk_mq_unquiesce_queue(q); - blk_mq_unfreeze_queue(q, memflags); if (ret) { pr_warn("elv: switch to \"%s\" failed, falling back to \"none\"\n", @@ -653,11 +651,9 @@ int elevator_switch(struct request_queue *q, struct elevator_type *new_e) void elevator_disable(struct request_queue *q) { - unsigned int memflags; - + WARN_ON_ONCE(q->mq_freeze_depth == 0); lockdep_assert_held(&q->elevator_lock); - memflags = blk_mq_freeze_queue(q); blk_mq_quiesce_queue(q); elv_unregister_queue(q); @@ -668,7 +664,6 @@ void elevator_disable(struct request_queue *q) blk_add_trace_msg(q, "elv switch: none"); blk_mq_unquiesce_queue(q); - blk_mq_unfreeze_queue(q, memflags); } /* From patchwork Thu Apr 10 13:30:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046513 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15F8E270EDD for ; Thu, 10 Apr 2025 13:31:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291866; cv=none; b=ifxQjqNac6VNH3Ss9FlbdIMolfj9gzRl4CMb3TBcNoWq5GlH5IUpYxOP0PKiOlXVC8LtE6qrgr/VuzQ//xm+Or+8/yHTjR9v8bf4O4fsRJRS/IwEfHaAI7lJ8Fb9ifoKLQpvBu8cVwYCJfUJhOAwZyzFeM2Pm2lpOUr9WSKbVAE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291866; c=relaxed/simple; bh=W1UdxACutlB4PXXwGop6Bj2NlUg6zghyZyBRqP7SF+o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CZqTiyDLFx4nJL9J+0Iebjku4zeqAHlRNC9GPaDJEBOyST/IOQcDKo193eWnZDvSw/K/OXgjjuSgFtQHQ8IA597n0g/ZBS+k9Or9z0gkcF38K2v80gsNk6hXMU3BWaDs96Py1v/ekHfeEXUKdFaey6MSGrp8mAehuXO28rRTArs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cA4Qr/5M; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cA4Qr/5M" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291864; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EkMxNwGGiOx5QHMiE4ibpmnkcfdM3hpGrThHPVIPN10=; b=cA4Qr/5M8eM2Om9Ir/2Go2TM2sz1C0cOv4uK722ixvQ17WX/5mA+ylv0uFvlfDQUA3TLiX j+KomvLMxoGb7EIqbNk/uK/bVxr5DxTFU3Dg0EhqBBTLKOB9w5WwiJc88pIey8VrRas8K3 VCHQJ7Vqpcoy3dGR055J/Sz7r/Qw65M= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-132-6yON_z6VM4S2J0RfnmPUsw-1; Thu, 10 Apr 2025 09:31:00 -0400 X-MC-Unique: 6yON_z6VM4S2J0RfnmPUsw-1 X-Mimecast-MFC-AGG-ID: 6yON_z6VM4S2J0RfnmPUsw_1744291859 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7234E180025A; Thu, 10 Apr 2025 13:30:59 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4C8FA180175B; Thu, 10 Apr 2025 13:30:57 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 02/15] block: add two helpers for registering/un-registering sched debugfs Date: Thu, 10 Apr 2025 21:30:14 +0800 Message-ID: <20250410133029.2487054-3-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Add blk_mq_sched_reg_debugfs()/blk_mq_sched_unreg_debugfs() to clean up sched init/exit code a bit. Signed-off-by: Ming Lei --- block/blk-mq-sched.c | 42 +++++++++++++++++++++++++++--------------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index 109611445d40..f66abaa25430 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -436,6 +436,30 @@ static int blk_mq_init_sched_shared_tags(struct request_queue *queue) return 0; } +static void blk_mq_sched_reg_debugfs(struct request_queue *q) +{ + struct blk_mq_hw_ctx *hctx; + unsigned long i; + + mutex_lock(&q->debugfs_mutex); + blk_mq_debugfs_register_sched(q); + queue_for_each_hw_ctx(q, hctx, i) + blk_mq_debugfs_register_sched_hctx(q, hctx); + mutex_unlock(&q->debugfs_mutex); +} + +static void blk_mq_sched_unreg_debugfs(struct request_queue *q) +{ + struct blk_mq_hw_ctx *hctx; + unsigned long i; + + mutex_lock(&q->debugfs_mutex); + queue_for_each_hw_ctx(q, hctx, i) + blk_mq_debugfs_unregister_sched_hctx(hctx); + blk_mq_debugfs_unregister_sched(q); + mutex_unlock(&q->debugfs_mutex); +} + /* caller must have a reference to @e, will grab another one if successful */ int blk_mq_init_sched(struct request_queue *q, struct elevator_type *e) { @@ -469,10 +493,6 @@ int blk_mq_init_sched(struct request_queue *q, struct elevator_type *e) if (ret) goto err_free_map_and_rqs; - mutex_lock(&q->debugfs_mutex); - blk_mq_debugfs_register_sched(q); - mutex_unlock(&q->debugfs_mutex); - queue_for_each_hw_ctx(q, hctx, i) { if (e->ops.init_hctx) { ret = e->ops.init_hctx(hctx, i); @@ -484,10 +504,8 @@ int blk_mq_init_sched(struct request_queue *q, struct elevator_type *e) return ret; } } - mutex_lock(&q->debugfs_mutex); - blk_mq_debugfs_register_sched_hctx(q, hctx); - mutex_unlock(&q->debugfs_mutex); } + blk_mq_sched_reg_debugfs(q); return 0; @@ -526,11 +544,9 @@ void blk_mq_exit_sched(struct request_queue *q, struct elevator_queue *e) unsigned long i; unsigned int flags = 0; - queue_for_each_hw_ctx(q, hctx, i) { - mutex_lock(&q->debugfs_mutex); - blk_mq_debugfs_unregister_sched_hctx(hctx); - mutex_unlock(&q->debugfs_mutex); + blk_mq_sched_unreg_debugfs(q); + queue_for_each_hw_ctx(q, hctx, i) { if (e->type->ops.exit_hctx && hctx->sched_data) { e->type->ops.exit_hctx(hctx, i); hctx->sched_data = NULL; @@ -538,10 +554,6 @@ void blk_mq_exit_sched(struct request_queue *q, struct elevator_queue *e) flags = hctx->flags; } - mutex_lock(&q->debugfs_mutex); - blk_mq_debugfs_unregister_sched(q); - mutex_unlock(&q->debugfs_mutex); - if (e->type->ops.exit_sched) e->type->ops.exit_sched(e); blk_mq_sched_tags_teardown(q, flags); From patchwork Thu Apr 10 13:30:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046514 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE2E58633A for ; Thu, 10 Apr 2025 13:31:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291872; cv=none; b=sYAggPYgaGFIP5JfJOS/rE/YAao/ZQR0Crx+WkZXuJ/0XhqdlB7GJzUuY7b3/vFnP4Naq6fEMvtRk96dDY6sMNoTw/tzWoYkVBHZPfQQK83bqiZ3UPJtWMbcCrVmn7rrJ49kwbYemu5O+QHsTC+MKuem8XFuimCSgK1x2EFNb60= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291872; c=relaxed/simple; bh=tfSKemNfw6WoQTadT5wtSUDxFOMrG+XuReDHe6NKuto=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rVj3od9wHuhambJ18cn2KX2XaOun4TXuWzvLpVdEuqSfXx30aqly1Cmea8w8yG0zz1i0OB8WmPLXyZVYTXPB3TKPv6VG5K/L8X+AtlXAx7DmSlCG0MTprLCIjcX/OcM4hBR+ONk8zfG74lCgzxbbgNFSA8J5jfeQUL3gHtfESCw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=F4jW2hfn; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="F4jW2hfn" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pIYWmJnTkyV3sXs2FJyBB4Kxj3WbPutyEvPCf0slPFA=; b=F4jW2hfnJd6cx2SrdJJxSxLGFBm87yMNe1guqqsj4U8YUymNANxPi+6NG0Z3ZZPRo2B57R yj+VypHPDzXZToEyBrVFek0vmgDaWHRxXZiNKmTl4B1mYYi+CqE377NTB/ilEasPgPoa3Y Vj4mNvv9nHL7LgkTBBY+5YXWzussevw= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-26-oR4zlVMCPE2_ZEOs3uz1sA-1; Thu, 10 Apr 2025 09:31:06 -0400 X-MC-Unique: oR4zlVMCPE2_ZEOs3uz1sA-1 X-Mimecast-MFC-AGG-ID: oR4zlVMCPE2_ZEOs3uz1sA_1744291865 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 69E041800EC5; Thu, 10 Apr 2025 13:31:04 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0CC7E1956094; Thu, 10 Apr 2025 13:31:02 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 03/15] block: move sched debugfs register into elvevator_register_queue Date: Thu, 10 Apr 2025 21:30:15 +0800 Message-ID: <20250410133029.2487054-4-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 sched debugfs shares same lifetime with scheduler's kobject, and same lock(elevator lock), so move sched debugfs register/unregister into elevator_register_queue() and elevator_unregister_queue(). Then we needn't blk_mq_debugfs_register() for us to register sched debugfs any more. Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 12 ------------ block/blk-mq-sched.c | 7 ++----- block/elevator.c | 6 ++++++ block/elevator.h | 3 +++ 4 files changed, 11 insertions(+), 17 deletions(-) diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index 3421b5521fe2..c308699ded58 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -624,22 +624,10 @@ void blk_mq_debugfs_register(struct request_queue *q) debugfs_create_files(q->debugfs_dir, q, blk_mq_debugfs_queue_attrs); - /* - * blk_mq_init_sched() attempted to do this already, but q->debugfs_dir - * didn't exist yet (because we don't know what to name the directory - * until the queue is registered to a gendisk). - */ - if (q->elevator && !q->sched_debugfs_dir) - blk_mq_debugfs_register_sched(q); - - /* Similarly, blk_mq_init_hctx() couldn't do this previously. */ queue_for_each_hw_ctx(q, hctx, i) { if (!hctx->debugfs_dir) blk_mq_debugfs_register_hctx(q, hctx); - if (q->elevator && !hctx->sched_debugfs_dir) - blk_mq_debugfs_register_sched_hctx(q, hctx); } - if (q->rq_qos) { struct rq_qos *rqos = q->rq_qos; diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index f66abaa25430..14552c58c4e8 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -436,7 +436,7 @@ static int blk_mq_init_sched_shared_tags(struct request_queue *queue) return 0; } -static void blk_mq_sched_reg_debugfs(struct request_queue *q) +void blk_mq_sched_reg_debugfs(struct request_queue *q) { struct blk_mq_hw_ctx *hctx; unsigned long i; @@ -448,7 +448,7 @@ static void blk_mq_sched_reg_debugfs(struct request_queue *q) mutex_unlock(&q->debugfs_mutex); } -static void blk_mq_sched_unreg_debugfs(struct request_queue *q) +void blk_mq_sched_unreg_debugfs(struct request_queue *q) { struct blk_mq_hw_ctx *hctx; unsigned long i; @@ -505,7 +505,6 @@ int blk_mq_init_sched(struct request_queue *q, struct elevator_type *e) } } } - blk_mq_sched_reg_debugfs(q); return 0; @@ -544,8 +543,6 @@ void blk_mq_exit_sched(struct request_queue *q, struct elevator_queue *e) unsigned long i; unsigned int flags = 0; - blk_mq_sched_unreg_debugfs(q); - queue_for_each_hw_ctx(q, hctx, i) { if (e->type->ops.exit_hctx && hctx->sched_data) { e->type->ops.exit_hctx(hctx, i); diff --git a/block/elevator.c b/block/elevator.c index 5051a98dc08c..cf48613c6e62 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -459,6 +459,9 @@ int elv_register_queue(struct request_queue *q, bool uevent) lockdep_assert_held(&q->elevator_lock); + if (test_bit(ELEVATOR_FLAG_REGISTERED, &e->flags)) + return 0; + error = kobject_add(&e->kobj, &q->disk->queue_kobj, "iosched"); if (!error) { const struct elv_fs_entry *attr = e->type->elevator_attrs; @@ -472,6 +475,7 @@ int elv_register_queue(struct request_queue *q, bool uevent) if (uevent) kobject_uevent(&e->kobj, KOBJ_ADD); + blk_mq_sched_reg_debugfs(q); set_bit(ELEVATOR_FLAG_REGISTERED, &e->flags); } return error; @@ -486,6 +490,8 @@ void elv_unregister_queue(struct request_queue *q) if (e && test_and_clear_bit(ELEVATOR_FLAG_REGISTERED, &e->flags)) { kobject_uevent(&e->kobj, KOBJ_REMOVE); kobject_del(&e->kobj); + + blk_mq_sched_unreg_debugfs(q); } } diff --git a/block/elevator.h b/block/elevator.h index e4e44dfac503..80ff9b28a66f 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -182,4 +182,7 @@ extern struct request *elv_rb_find(struct rb_root *, sector_t); #define rq_entry_fifo(ptr) list_entry((ptr), struct request, queuelist) #define rq_fifo_clear(rq) list_del_init(&(rq)->queuelist) +void blk_mq_sched_reg_debugfs(struct request_queue *q); +void blk_mq_sched_unreg_debugfs(struct request_queue *q); + #endif /* _ELEVATOR_H */ From patchwork Thu Apr 10 13:30:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046515 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E593270EDD for ; Thu, 10 Apr 2025 13:31:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291877; cv=none; b=EwLnyw2FSOpEFbU/b/taCZ+QqTUymQ+++OOCIBQMdE75yHqj6mvOe26F8WxhPwv31+KL+5vdM+fksySC7BwGTxtnIBdvP/fbT5CvyVzcvM91EX+hbheepkFCbCsJI5QCBjqPJKB28kOkoDioCffwF6JZk7ngL4/LdbqcfPm9WUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291877; c=relaxed/simple; bh=vB94mJlJVNxRktJmKMzj1ehDRWHsgE+mw5WQGHr9uA0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RSm4H5t2zjnWnuZlYazexWZ4RQPl4tmfXcUDCSAJC16bAOeTImzXnlFqzTADR1M5TnYYk5+R2m7WI+/GLlCLN8LnU1OhD2Wj5zXdDSvHNaEzUqN4ckmSThSRM61QrdWXBC9o8pDC/fftkuVTbXR4nW5mYrxP4VjJ021+HchMG/g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=X4MiWT5V; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X4MiWT5V" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291874; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E6agpd/d3D4wyti/Vo36daRc/khYWwqjQIgEm97GzwE=; b=X4MiWT5ViPYV1lGXTuVK1DiryvGlNdR4WMlfn+BvOSj4LWGG9/oJgmqU7LGdOeVBRJwpH7 WqcHzdqin6z3qLCjgRDYb2wwmXhHA82rINZsNj0JwVoxaY6rM5R4NOIu1jfDjQj+nuLBCV JSNdCRWSY2py0SRmJp9JH/VYv8zTqqg= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-250-WjOQ-DTqOBuhgbuBkSfreQ-1; Thu, 10 Apr 2025 09:31:10 -0400 X-MC-Unique: WjOQ-DTqOBuhgbuBkSfreQ-1 X-Mimecast-MFC-AGG-ID: WjOQ-DTqOBuhgbuBkSfreQ_1744291869 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 405DD1954B32; Thu, 10 Apr 2025 13:31:09 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1F8901808882; Thu, 10 Apr 2025 13:31:07 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 04/15] block: prevent elevator switch during updating nr_hw_queues Date: Thu, 10 Apr 2025 21:30:16 +0800 Message-ID: <20250410133029.2487054-5-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 updating nr_hw_queues is usually used for error handling code, when it doesn't make sense to allow blk-mq elevator switching, since nr_hw_queues may change, and elevator tags depends on nr_hw_queues. Prevent elevator switch during updating nr_hw_queues by setting flag of BLK_MQ_F_UPDATE_HW_QUEUES, and use srcu to fail elevator switch during the period. Here elevator switch code is srcu reader of nr_hw_queues, and blk_mq_update_nr_hw_queues() is the writer. This way avoids lot of trouble. Reported-by: Shinichiro Kawasaki Closes: https://lore.kernel.org/linux-block/mz4t4tlwiqjijw3zvqnjb7ovvvaegkqganegmmlc567tt5xj67@xal5ro544cnc/ Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 1 + block/blk-mq.c | 19 ++++++++++++++++++- block/elevator.c | 12 +++++++++++- include/linux/blk-mq.h | 10 +++++++++- 4 files changed, 39 insertions(+), 3 deletions(-) diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index c308699ded58..27f984311bb7 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -180,6 +180,7 @@ static const char *const hctx_flag_name[] = { HCTX_FLAG_NAME(BLOCKING), HCTX_FLAG_NAME(TAG_RR), HCTX_FLAG_NAME(NO_SCHED_BY_DEFAULT), + HCTX_FLAG_NAME(UPDATE_HW_QUEUES), }; #undef HCTX_FLAG_NAME diff --git a/block/blk-mq.c b/block/blk-mq.c index d7a103dc258b..4b0707fb7ae3 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4785,12 +4785,16 @@ int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set) goto out_free_srcu; } + ret = init_srcu_struct(&set->update_nr_hwq_srcu); + if (ret) + goto out_cleanup_srcu; + ret = -ENOMEM; set->tags = kcalloc_node(set->nr_hw_queues, sizeof(struct blk_mq_tags *), GFP_KERNEL, set->numa_node); if (!set->tags) - goto out_cleanup_srcu; + goto out_cleanup_hwq_srcu; for (i = 0; i < set->nr_maps; i++) { set->map[i].mq_map = kcalloc_node(nr_cpu_ids, @@ -4819,6 +4823,8 @@ int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set) } kfree(set->tags); set->tags = NULL; +out_cleanup_hwq_srcu: + cleanup_srcu_struct(&set->update_nr_hwq_srcu); out_cleanup_srcu: if (set->flags & BLK_MQ_F_BLOCKING) cleanup_srcu_struct(set->srcu); @@ -5081,7 +5087,18 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues) { mutex_lock(&set->tag_list_lock); + /* + * Mark us in updating nr_hw_queues for preventing switching + * elevator + * + * Elevator switch code can _not_ acquire ->tag_list_lock + */ + set->flags |= BLK_MQ_F_UPDATE_HW_QUEUES; + synchronize_srcu(&set->update_nr_hwq_srcu); + __blk_mq_update_nr_hw_queues(set, nr_hw_queues); + + set->flags &= BLK_MQ_F_UPDATE_HW_QUEUES; mutex_unlock(&set->tag_list_lock); } EXPORT_SYMBOL_GPL(blk_mq_update_nr_hw_queues); diff --git a/block/elevator.c b/block/elevator.c index cf48613c6e62..7d7b77dd4341 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -718,9 +718,10 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, { char elevator_name[ELV_NAME_MAX]; char *name; - int ret; + int ret, idx; unsigned int memflags; struct request_queue *q = disk->queue; + struct blk_mq_tag_set *set = q->tag_set; /* * If the attribute needs to load a module, do it before freezing the @@ -732,6 +733,13 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, elv_iosched_load_module(name); + idx = srcu_read_lock(&set->update_nr_hwq_srcu); + + if (set->flags & BLK_MQ_F_UPDATE_HW_QUEUES) { + ret = -EBUSY; + goto exit; + } + memflags = blk_mq_freeze_queue(q); mutex_lock(&q->elevator_lock); ret = elevator_change(q, name); @@ -739,6 +747,8 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, ret = count; mutex_unlock(&q->elevator_lock); blk_mq_unfreeze_queue(q, memflags); +exit: + srcu_read_unlock(&set->update_nr_hwq_srcu, idx); return ret; } diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 8eb9b3310167..473871c760e1 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -527,6 +527,7 @@ struct blk_mq_tag_set { struct mutex tag_list_lock; struct list_head tag_list; struct srcu_struct *srcu; + struct srcu_struct update_nr_hwq_srcu; }; /** @@ -681,7 +682,14 @@ enum { */ BLK_MQ_F_NO_SCHED_BY_DEFAULT = 1 << 6, - BLK_MQ_F_MAX = 1 << 7, + /* + * True when updating nr_hw_queues is in-progress + * + * tag_set only flag, not usable for hctx + */ + BLK_MQ_F_UPDATE_HW_QUEUES = 1 << 7, + + BLK_MQ_F_MAX = 1 << 8, }; #define BLK_MQ_MAX_DEPTH (10240) From patchwork Thu Apr 10 13:30:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046517 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0234270EDD for ; Thu, 10 Apr 2025 13:31:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291888; cv=none; b=dtAgB8B6hWR/IpFUYXIcuVzoqn6uPxD5XmdBFKGxU7nZ96rjvYvS5N+zmhBXa8Y5Zw5J6mHDfkuVDxyS+mnmDspUU+cMEnW1u27taVxNGrCuW3GIMQ6yPbR7JP6Zmb37xN4eHFFLNHVQNYvSkXt/EaUTCJVO7KwuUqxv8Qec2ws= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291888; c=relaxed/simple; bh=bIItKvllyzkv/stnQfIRoG2izOGgwrCVHEYBbl+75sA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bR1XoDnVVPUoJTOMCNzDaWBVAuvlWnsdErvNbjXfKxUsKlePDa5xhvFMiEs5qYfB+hzzBA2Q1RyfognGt8vnlzAvLnR7+BEGc7SDssFwzk5KUCjIMxIrgsciaO1PMKCWsgYLbaoXjga2gEuP4aQwdUUMJA1QdL/AKkwRf15yTFU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=E1J9vDxX; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="E1J9vDxX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291885; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iSAjPXD943HbiwtD4xIES+89vR/waJNMHZahYJfS6Kw=; b=E1J9vDxX+6/kkOSKuZ8atj28GYwVPyFp1/xSbreroQQB9GzEoPqrPlTOSn8Jk2me3ffCqR AfSqEZlEwsnsRc9Yai4rGi0IEs5j1DT21qol3I8gjsbLoic4FWR3OfEoJ33sssbuJ8kzeV fUHK9EqgfXhN9PNO3uwDRzlxX0Iy4xw= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-672-TZl929hcPeCoXB1PeJr3-g-1; Thu, 10 Apr 2025 09:31:18 -0400 X-MC-Unique: TZl929hcPeCoXB1PeJr3-g-1 X-Mimecast-MFC-AGG-ID: TZl929hcPeCoXB1PeJr3-g_1744291874 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 06077180882E; Thu, 10 Apr 2025 13:31:14 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BC3EE180174E; Thu, 10 Apr 2025 13:31:12 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 05/15] block: simplify elevator reset for updating nr_hw_queues Date: Thu, 10 Apr 2025 21:30:17 +0800 Message-ID: <20250410133029.2487054-6-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 In blk_mq_update_nr_hw_queues(), nr_hw_queues may change, so elevator has to be reset after nr_hw_queues is changed. Now elevator switch isn't allowed during blk_mq_update_nr_hw_queues(), so we can simply call elevator_change() to reset elevator sched tags after nr_hw_queues is updated. Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 99 +++++------------------------------------------- block/blk.h | 4 +- block/elevator.c | 12 +++--- 3 files changed, 18 insertions(+), 97 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 4b0707fb7ae3..b7e3cd355e66 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4930,88 +4930,10 @@ int blk_mq_update_nr_requests(struct request_queue *q, unsigned int nr) return ret; } -/* - * request_queue and elevator_type pair. - * It is just used by __blk_mq_update_nr_hw_queues to cache - * the elevator_type associated with a request_queue. - */ -struct blk_mq_qe_pair { - struct list_head node; - struct request_queue *q; - struct elevator_type *type; -}; - -/* - * Cache the elevator_type in qe pair list and switch the - * io scheduler to 'none' - */ -static bool blk_mq_elv_switch_none(struct list_head *head, - struct request_queue *q) -{ - struct blk_mq_qe_pair *qe; - - qe = kmalloc(sizeof(*qe), GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY); - if (!qe) - return false; - - /* Accessing q->elevator needs protection from ->elevator_lock. */ - mutex_lock(&q->elevator_lock); - - if (!q->elevator) { - kfree(qe); - goto unlock; - } - - INIT_LIST_HEAD(&qe->node); - qe->q = q; - qe->type = q->elevator->type; - /* keep a reference to the elevator module as we'll switch back */ - __elevator_get(qe->type); - list_add(&qe->node, head); - elevator_disable(q); -unlock: - mutex_unlock(&q->elevator_lock); - - return true; -} - -static struct blk_mq_qe_pair *blk_lookup_qe_pair(struct list_head *head, - struct request_queue *q) -{ - struct blk_mq_qe_pair *qe; - - list_for_each_entry(qe, head, node) - if (qe->q == q) - return qe; - - return NULL; -} - -static void blk_mq_elv_switch_back(struct list_head *head, - struct request_queue *q) -{ - struct blk_mq_qe_pair *qe; - struct elevator_type *t; - - qe = blk_lookup_qe_pair(head, q); - if (!qe) - return; - t = qe->type; - list_del(&qe->node); - kfree(qe); - - mutex_lock(&q->elevator_lock); - elevator_switch(q, t); - /* drop the reference acquired in blk_mq_elv_switch_none */ - elevator_put(t); - mutex_unlock(&q->elevator_lock); -} - static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues) { struct request_queue *q; - LIST_HEAD(head); int prev_nr_hw_queues = set->nr_hw_queues; unsigned int memflags; int i; @@ -5029,15 +4951,6 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, list_for_each_entry(q, &set->tag_list, tag_set_list) blk_mq_freeze_queue_nomemsave(q); - /* - * Switch IO scheduler to 'none', cleaning up the data associated - * with the previous scheduler. We will switch back once we are done - * updating the new sw to hw queue mappings. - */ - list_for_each_entry(q, &set->tag_list, tag_set_list) - if (!blk_mq_elv_switch_none(&head, q)) - goto switch_back; - list_for_each_entry(q, &set->tag_list, tag_set_list) { blk_mq_debugfs_unregister_hctxs(q); blk_mq_sysfs_unregister_hctxs(q); @@ -5071,9 +4984,15 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, blk_mq_debugfs_register_hctxs(q); } -switch_back: - list_for_each_entry(q, &set->tag_list, tag_set_list) - blk_mq_elv_switch_back(&head, q); + list_for_each_entry(q, &set->tag_list, tag_set_list) { + const char *name = "none"; + + mutex_lock(&q->elevator_lock); + if (q->elevator && !blk_queue_dying(q)) + name = q->elevator->type->elevator_name; + __elevator_change(q, name, true); + mutex_unlock(&q->elevator_lock); + } list_for_each_entry(q, &set->tag_list, tag_set_list) blk_mq_unfreeze_queue_nomemrestore(q); diff --git a/block/blk.h b/block/blk.h index 006e3be433d2..0c3cc1af2525 100644 --- a/block/blk.h +++ b/block/blk.h @@ -319,8 +319,8 @@ bool blk_bio_list_merge(struct request_queue *q, struct list_head *list, bool blk_insert_flush(struct request *rq); -int elevator_switch(struct request_queue *q, struct elevator_type *new_e); -void elevator_disable(struct request_queue *q); +int __elevator_change(struct request_queue *q, const char *elevator_name, + bool force); void elevator_exit(struct request_queue *q); int elv_register_queue(struct request_queue *q, bool uevent); void elv_unregister_queue(struct request_queue *q); diff --git a/block/elevator.c b/block/elevator.c index 7d7b77dd4341..612fa2bdd40d 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -619,7 +619,7 @@ void elevator_init_mq(struct request_queue *q) * If switching fails, we are most likely running out of memory and not able * to restore the old io scheduler, so leaving the io scheduler being none. */ -int elevator_switch(struct request_queue *q, struct elevator_type *new_e) +static int elevator_switch(struct request_queue *q, struct elevator_type *new_e) { int ret; @@ -655,7 +655,7 @@ int elevator_switch(struct request_queue *q, struct elevator_type *new_e) return ret; } -void elevator_disable(struct request_queue *q) +static void elevator_disable(struct request_queue *q) { WARN_ON_ONCE(q->mq_freeze_depth == 0); lockdep_assert_held(&q->elevator_lock); @@ -675,7 +675,8 @@ void elevator_disable(struct request_queue *q) /* * Switch this queue to the given IO scheduler. */ -static int elevator_change(struct request_queue *q, const char *elevator_name) +int __elevator_change(struct request_queue *q, const char *elevator_name, + bool force) { struct elevator_type *e; int ret; @@ -690,7 +691,8 @@ static int elevator_change(struct request_queue *q, const char *elevator_name) return 0; } - if (q->elevator && elevator_match(q->elevator->type, elevator_name)) + if (!force && q->elevator && + elevator_match(q->elevator->type, elevator_name)) return 0; e = elevator_find_get(elevator_name); @@ -742,7 +744,7 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, memflags = blk_mq_freeze_queue(q); mutex_lock(&q->elevator_lock); - ret = elevator_change(q, name); + ret = __elevator_change(q, name, false); if (!ret) ret = count; mutex_unlock(&q->elevator_lock); From patchwork Thu Apr 10 13:30:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046516 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80F748633A for ; Thu, 10 Apr 2025 13:31:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291887; cv=none; b=Bf5X3DGzLHTFqUTb8FzOyJ6DCgqcQlioeMDiAT3TTZILFp4BFzru+/L5qs5GTI5qRDrdiKUP84BVXChDWsPcBobx7aFG6eMJG8pbVp2rT3og+NVmkch5//DBreRXh2qTMtR3R6CX/RTCGqdwcUTt2OhbQMI6cjoo956eVpsNs6Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291887; c=relaxed/simple; bh=yKoRJYYXrgSR4iQBbtWnOQyarXYrui4zmdZENlBIUMo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Wt9bUSoJBRuZQYPBNfwwykhsARYiWn/m69ZRF2Qk746j5p1LFk9X32vGxGctBL6abWKYVq981I1jeUg8DUo8vdqNXs+Q+C/IRgvt6lRCMy7r6N6UlztW6XOgMbakD2PAXNMhCiGu6LDgrJVNdHledBvOaJzIaVPLjg5qrRmNB84= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=K9knUBX/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="K9knUBX/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291884; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q6yhpuePknjGj4Q76z4Q/TzgSmmjJhZfiXg0Eft7+eg=; b=K9knUBX/etkaYelj5x9DUzkG8Q7mwL8SXI2t+9S5TmUVDrSjtleRk2qh4ft/cYcgJwVupW jhtGC2fvHpL4aEJLfjqA8OYo61hDZzg4ju0NG4fWz0OINDAkrAU6lBA4uJXVb3s3aAhmqO 7sKqdNhiVudj+xBNfJyJQjWFPzEbp+s= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-644-HE-8IBffN3CvoIJISHJonA-1; Thu, 10 Apr 2025 09:31:21 -0400 X-MC-Unique: HE-8IBffN3CvoIJISHJonA-1 X-Mimecast-MFC-AGG-ID: HE-8IBffN3CvoIJISHJonA_1744291878 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 51674180025C; Thu, 10 Apr 2025 13:31:18 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5EDBD180B486; Thu, 10 Apr 2025 13:31:16 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 06/15] block: add helper of elevator_change() Date: Thu, 10 Apr 2025 21:30:18 +0800 Message-ID: <20250410133029.2487054-7-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Add elevator_change() to simplify elv_iosched_store() a bit, and the new helper will be used for unifying all scheduler change. Signed-off-by: Ming Lei --- block/elevator.c | 52 +++++++++++++++++++++++++++++------------------- 1 file changed, 31 insertions(+), 21 deletions(-) diff --git a/block/elevator.c b/block/elevator.c index 612fa2bdd40d..e028d2ff9624 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -53,6 +53,8 @@ static LIST_HEAD(elv_list); */ #define rq_hash_key(rq) (blk_rq_pos(rq) + blk_rq_sectors(rq)) +static int elevator_change(struct request_queue *q, const char *name); + /* * Query io scheduler to see if the current process issuing bio may be * merged with rq. @@ -681,10 +683,6 @@ int __elevator_change(struct request_queue *q, const char *elevator_name, struct elevator_type *e; int ret; - /* Make sure queue is not in the middle of being removed */ - if (!blk_queue_registered(q)) - return -ENOENT; - if (!strncmp(elevator_name, "none", 4)) { if (q->elevator) elevator_disable(q); @@ -703,6 +701,29 @@ int __elevator_change(struct request_queue *q, const char *elevator_name, return ret; } +static int elevator_change(struct request_queue *q, const char *name) +{ + int ret, idx; + unsigned int memflags; + struct blk_mq_tag_set *set = q->tag_set; + + idx = srcu_read_lock(&set->update_nr_hwq_srcu); + + if (set->flags & BLK_MQ_F_UPDATE_HW_QUEUES) { + ret = -EBUSY; + goto exit; + } + + memflags = blk_mq_freeze_queue(q); + mutex_lock(&q->elevator_lock); + ret = __elevator_change(q, name, false); + mutex_unlock(&q->elevator_lock); + blk_mq_unfreeze_queue(q, memflags); +exit: + srcu_read_unlock(&set->update_nr_hwq_srcu, idx); + return ret; +} + static void elv_iosched_load_module(char *elevator_name) { struct elevator_type *found; @@ -718,12 +739,10 @@ static void elv_iosched_load_module(char *elevator_name) ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, size_t count) { + struct request_queue *q = disk->queue; char elevator_name[ELV_NAME_MAX]; char *name; - int ret, idx; - unsigned int memflags; - struct request_queue *q = disk->queue; - struct blk_mq_tag_set *set = q->tag_set; + int ret; /* * If the attribute needs to load a module, do it before freezing the @@ -735,22 +754,13 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, elv_iosched_load_module(name); - idx = srcu_read_lock(&set->update_nr_hwq_srcu); - - if (set->flags & BLK_MQ_F_UPDATE_HW_QUEUES) { - ret = -EBUSY; - goto exit; - } + /* Make sure queue is not in the middle of being removed */ + if (!blk_queue_registered(q)) + return -ENOENT; - memflags = blk_mq_freeze_queue(q); - mutex_lock(&q->elevator_lock); - ret = __elevator_change(q, name, false); + ret = elevator_change(q, name); if (!ret) ret = count; - mutex_unlock(&q->elevator_lock); - blk_mq_unfreeze_queue(q, memflags); -exit: - srcu_read_unlock(&set->update_nr_hwq_srcu, idx); return ret; } From patchwork Thu Apr 10 13:30:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046518 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F0798633A for ; Thu, 10 Apr 2025 13:31:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291894; cv=none; b=dsFCNzCgtRRJs3HgPPbvP4GjULqYK1YUuKkcENOWzMYRLpUHYXxw7C4XgXy4qKZ4/zQvmTy2a87skWsBvVFOa+KF0jBym1sa5Zn2wVNcVesTKvAdM9XAkM5Zre1C0dwL8JDu59cURCj7air+33voqFjyMrC6H/0UlxYuC/hTlHY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291894; c=relaxed/simple; bh=ys36xuRxasvFBQAggPOSKq0MJ+kOM4dkpKM1wXgaWKA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YDu0a//XDDVhhkmcMUaWSYZiXbLRrCw8BCOkuUWODYzyjlS1noHyyAciQrlLZigYlXCduo/ib09bd/r5EVlU63OCvUGASkiYJB+QemqvlG4jcPKqktJnV46ggCrFRRBzBlkFA0bHxAtKniPoCyyrH3s0LYtyfwC1g6V8k0/FC0E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=CXb3k+A6; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CXb3k+A6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291890; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F78dOB1IcPUXETUNoydupo0qY3sU/ZSPJ3VakeLk0+4=; b=CXb3k+A6hJPUiuHZ9gw+x1iVyAPcrDqVQgAwJX1p/fbAAv0glFAqh04baOrzpZg/8Y0qpN oZZkawZ47+KB3Lu/xmxc5A3yI6EoIFG55CwmQKviGuzICnAoLA0bwkV4LGhmbiyE6rJr7i M8NNeLefiOkhdOoRS+I90Ic6bQbr5CM= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-56-_CT1uXJ_PmGcachMhWwE_Q-1; Thu, 10 Apr 2025 09:31:26 -0400 X-MC-Unique: _CT1uXJ_PmGcachMhWwE_Q-1 X-Mimecast-MFC-AGG-ID: _CT1uXJ_PmGcachMhWwE_Q_1744291883 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0815C1801A06; Thu, 10 Apr 2025 13:31:23 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E767B1808882; Thu, 10 Apr 2025 13:31:21 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 07/15] block: move blk_unregister_queue() & device_del() after freeze wait Date: Thu, 10 Apr 2025 21:30:19 +0800 Message-ID: <20250410133029.2487054-8-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Move blk_unregister_queue() & device_del() after freeze wait, and prepare for unifying elevator switch. This way is just fine, since bdev has been unhashed at the beginning of del_gendisk(), both blk_unregister_queue() & device_del() are dealing with kobject & debugfs thing only. Signed-off-by: Ming Lei --- block/genhd.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block/genhd.c b/block/genhd.c index c2bd86cd09de..f426c13edf55 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -721,8 +721,6 @@ void del_gendisk(struct gendisk *disk) bdi_unregister(disk->bdi); } - blk_unregister_queue(disk); - kobject_put(disk->part0->bd_holder_dir); kobject_put(disk->slave_dir); disk->slave_dir = NULL; @@ -731,10 +729,12 @@ void del_gendisk(struct gendisk *disk) disk->part0->bd_stamp = 0; sysfs_remove_link(block_depr, dev_name(disk_to_dev(disk))); pm_runtime_set_memalloc_noio(disk_to_dev(disk), false); - device_del(disk_to_dev(disk)); blk_mq_freeze_queue_wait(q); + blk_unregister_queue(disk); + device_del(disk_to_dev(disk)); + blk_throtl_cancel_bios(disk); blk_sync_queue(q); From patchwork Thu Apr 10 13:30:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046519 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF9241F930 for ; Thu, 10 Apr 2025 13:31:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291895; cv=none; b=oPoktwg8ImaxqfQ9QjxpIeGOSS6UGXw1isY0CFtqnJhoQYtVBRa8G8smfU7wxO1TORvdsrAA63bG9I2vZDTiR88o8N9IOQX2JdZ4A1H09cBstBNXT7JgjlUzhEAP6uWdnYQRhHJ24/Bi0eDxzhBI3FnQSMj03l9YtQHd1SlQFXs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291895; c=relaxed/simple; bh=R7JM6IoWeIAw9tOs7WOhhuoQMfvBtFgKweDXdwAbERg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GTXl8FTZ+atlMpvTBFNbVUZ3PqhaRIsrI/GNNnKZ2PU+gkshrlszT9bFv9hArYHZPx6+plk6Uj5xz93+Jz7VWIT88ZcODdiuJ84wG0IZ5G+T/3aVjkEKPjyQTxPjHxQOO/vbJ/QiFw/f17clbO7fgWLXTB4tptdu0RVhHhSamIg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=XHpNd1eX; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XHpNd1eX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291892; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Usj3xW4j7dvSx1IAVd2APO2KC2STys+e2IkRdQ7SDgI=; b=XHpNd1eXi2y8gsvVntkT0dQz7zmSxQuuZiZCJW5ce7Zs/wZsw78MK1sdu6QD1Pr1CpYeJy UOH8jv/GorsqGF9PHTQT8OctREUHQphbUqrLVjckJZrYRJ04sTWLhGEbzBcK/Op7/9BPjV 3AsFQrB5EgP1phEWbnlq82/WAOPN0qU= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-478-2tNjr1z6NN-JdAS5q233xQ-1; Thu, 10 Apr 2025 09:31:29 -0400 X-MC-Unique: 2tNjr1z6NN-JdAS5q233xQ-1 X-Mimecast-MFC-AGG-ID: 2tNjr1z6NN-JdAS5q233xQ_1744291888 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AD1391801A00; Thu, 10 Apr 2025 13:31:27 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8ECAB1808882; Thu, 10 Apr 2025 13:31:26 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 08/15] block: add `struct elev_change_ctx` for unifying elevator change Date: Thu, 10 Apr 2025 21:30:20 +0800 Message-ID: <20250410133029.2487054-9-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Add `struct elev_change_ctx` and prepare for unifying elevator change, with this way, any input & output parameter can be provided & observed in top caller. This way also helps to move kobject & debugfs things out of ->elevator_lock. Signed-off-by: Ming Lei --- block/blk-mq.c | 10 +++++++--- block/blk.h | 4 ++-- block/elevator.c | 33 +++++++++++++++++++-------------- block/elevator.h | 7 +++++++ 4 files changed, 35 insertions(+), 19 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index b7e3cd355e66..608a74e3a87c 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4985,12 +4985,16 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, } list_for_each_entry(q, &set->tag_list, tag_set_list) { - const char *name = "none"; + struct elev_change_ctx ctx = { + .name = "none", + .force = 1, + .uevent = 1, + }; mutex_lock(&q->elevator_lock); if (q->elevator && !blk_queue_dying(q)) - name = q->elevator->type->elevator_name; - __elevator_change(q, name, true); + ctx.name = q->elevator->type->elevator_name; + __elevator_change(q, &ctx); mutex_unlock(&q->elevator_lock); } diff --git a/block/blk.h b/block/blk.h index 0c3cc1af2525..922a429b5363 100644 --- a/block/blk.h +++ b/block/blk.h @@ -12,6 +12,7 @@ #include "blk-crypto-internal.h" struct elevator_type; +struct elev_change_ctx; #define BLK_DEV_MAX_SECTORS (LLONG_MAX >> 9) #define BLK_MIN_SEGMENT_SIZE 4096 @@ -319,8 +320,7 @@ bool blk_bio_list_merge(struct request_queue *q, struct list_head *list, bool blk_insert_flush(struct request *rq); -int __elevator_change(struct request_queue *q, const char *elevator_name, - bool force); +int __elevator_change(struct request_queue *q, struct elev_change_ctx *ctx); void elevator_exit(struct request_queue *q); int elv_register_queue(struct request_queue *q, bool uevent); void elv_unregister_queue(struct request_queue *q); diff --git a/block/elevator.c b/block/elevator.c index e028d2ff9624..2bc1679dcd1f 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -53,7 +53,8 @@ static LIST_HEAD(elv_list); */ #define rq_hash_key(rq) (blk_rq_pos(rq) + blk_rq_sectors(rq)) -static int elevator_change(struct request_queue *q, const char *name); +static int elevator_change(struct request_queue *q, + struct elev_change_ctx *ctx); /* * Query io scheduler to see if the current process issuing bio may be @@ -621,7 +622,8 @@ void elevator_init_mq(struct request_queue *q) * If switching fails, we are most likely running out of memory and not able * to restore the old io scheduler, so leaving the io scheduler being none. */ -static int elevator_switch(struct request_queue *q, struct elevator_type *new_e) +static int elevator_switch(struct request_queue *q, struct elevator_type *new_e, + struct elev_change_ctx *ctx) { int ret; @@ -639,7 +641,7 @@ static int elevator_switch(struct request_queue *q, struct elevator_type *new_e) if (ret) goto out_unfreeze; - ret = elv_register_queue(q, true); + ret = elv_register_queue(q, ctx->uevent); if (ret) { elevator_exit(q); goto out_unfreeze; @@ -677,9 +679,9 @@ static void elevator_disable(struct request_queue *q) /* * Switch this queue to the given IO scheduler. */ -int __elevator_change(struct request_queue *q, const char *elevator_name, - bool force) +int __elevator_change(struct request_queue *q, struct elev_change_ctx *ctx) { + const char *elevator_name = ctx->name; struct elevator_type *e; int ret; @@ -689,19 +691,20 @@ int __elevator_change(struct request_queue *q, const char *elevator_name, return 0; } - if (!force && q->elevator && + if (!ctx->force && q->elevator && elevator_match(q->elevator->type, elevator_name)) return 0; e = elevator_find_get(elevator_name); if (!e) return -EINVAL; - ret = elevator_switch(q, e); + ret = elevator_switch(q, e, ctx); elevator_put(e); return ret; } -static int elevator_change(struct request_queue *q, const char *name) +static int elevator_change(struct request_queue *q, + struct elev_change_ctx *ctx) { int ret, idx; unsigned int memflags; @@ -716,7 +719,7 @@ static int elevator_change(struct request_queue *q, const char *name) memflags = blk_mq_freeze_queue(q); mutex_lock(&q->elevator_lock); - ret = __elevator_change(q, name, false); + ret = __elevator_change(q, ctx); mutex_unlock(&q->elevator_lock); blk_mq_unfreeze_queue(q, memflags); exit: @@ -724,7 +727,7 @@ static int elevator_change(struct request_queue *q, const char *name) return ret; } -static void elv_iosched_load_module(char *elevator_name) +static void elv_iosched_load_module(const char *elevator_name) { struct elevator_type *found; @@ -741,7 +744,9 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, { struct request_queue *q = disk->queue; char elevator_name[ELV_NAME_MAX]; - char *name; + struct elev_change_ctx ctx = { + .uevent = 1, + }; int ret; /* @@ -750,15 +755,15 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, * queue is the one for the device storing the module file. */ strscpy(elevator_name, buf, sizeof(elevator_name)); - name = strstrip(elevator_name); + ctx.name = strstrip(elevator_name); - elv_iosched_load_module(name); + elv_iosched_load_module(ctx.name); /* Make sure queue is not in the middle of being removed */ if (!blk_queue_registered(q)) return -ENOENT; - ret = elevator_change(q, name); + ret = elevator_change(q, &ctx); if (!ret) ret = count; return ret; diff --git a/block/elevator.h b/block/elevator.h index 80ff9b28a66f..86b4977cf772 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -123,6 +123,13 @@ struct elevator_queue #define ELEVATOR_FLAG_REGISTERED 0 #define ELEVATOR_FLAG_DISABLE_WBT 1 +/* Holding data for changing elevator */ +struct elev_change_ctx { + const char *name; + unsigned int force:1; + unsigned int uevent:1; +}; + /* * block elevator interface */ From patchwork Thu Apr 10 13:30:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046520 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34E568633A for ; Thu, 10 Apr 2025 13:31:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291900; cv=none; b=JrYa3yw0havGz85qqPmyFliyNRaMT0cKoE8cCsFsfE2AA1Vw9H33bVRRH7SDwqLurbXhbx3Ubmf7Yj7RewDz1LRMghmVPQWIQhluAyt9cZpC7PZ7F15UGtVU1PZ42u8mx1svSoOEcQmnMKGWJWNF4xL7jnCZXFPQF+zVOh9jjbk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291900; c=relaxed/simple; bh=vdEJFDcxGrpnjUG1It7djsmMU2zQFFKmKunGB0vqonA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hu+Wg45/DM6qRyezW1YSeVUdnbvZJYLZGHd7x12yKfWYxRGkn3nwnTTiVLSVvM7ecslcS5jTau5cIovJW0P+qYF6XR3clbRDMLz4TClvPC+hK/lObb8ih5Y4Oick7h+4WwBu1bFsRn9sz9G1KD05fWago2j2LYO3wDwDklmCaY4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=g9ffmJ2T; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="g9ffmJ2T" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291897; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rVjMXIjSo8E5kVtN6suOJX1JQdUEtPaI6atDmbgJDok=; b=g9ffmJ2TZPVZBRNT8FHalkw2BOzZYP91tdH92qyPliazQd4Umrxxe8SVcIlwV9VhCPK3ZD NPzyabOrXT2hpnQpjBJ7yT9WCaEbhpiX0JHTKYg5nzCSqA30kc+9Ilz3aH1twF/6A3Cj0s yJcbqgr909m2QwYv3OUHl2FF1AFhstM= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-36-GT2IqPFgMh-THUyXfd8zvg-1; Thu, 10 Apr 2025 09:31:34 -0400 X-MC-Unique: GT2IqPFgMh-THUyXfd8zvg-1 X-Mimecast-MFC-AGG-ID: GT2IqPFgMh-THUyXfd8zvg_1744291892 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BAB5319560B0; Thu, 10 Apr 2025 13:31:32 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 63B1D180174E; Thu, 10 Apr 2025 13:31:30 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 09/15] block: unifying elevator change Date: Thu, 10 Apr 2025 21:30:21 +0800 Message-ID: <20250410133029.2487054-10-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 elevator change is one well-define behavior: - tear down current elevator if it exists - setup new elevator It is supposed to cover any case for changing elevator, typically the following cases: - setup default elevator in add_disk() - switch to none in del_disk() - reset elevator in blk_mq_update_nr_hw_queues() - switch elevator in sysfs `store` elevator attribute This patch uses elevator_change() to cover all above cases: - every elevator switch is serialized with each other: add_disk/del_disk/ store elevator is serialized already, blk_mq_update_nr_hw_queues() uses srcu for syncing with the other three cases - for both add_disk()/del_disk(), queue freeze works at atomic mode or has been froze, so the freeze in elevator_change() won't add extra delay Signed-off-by: Ming Lei --- block/blk-sysfs.c | 18 ++++------- block/blk.h | 5 ++-- block/elevator.c | 76 +++++++++++++++++++++++++---------------------- block/genhd.c | 19 +----------- 4 files changed, 50 insertions(+), 68 deletions(-) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index a2882751f0d2..58c50709bc14 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -869,14 +869,8 @@ int blk_register_queue(struct gendisk *disk) if (ret) goto out_unregister_ia_ranges; + elevator_set_default(q); mutex_lock(&q->elevator_lock); - if (q->elevator) { - ret = elv_register_queue(q, false); - if (ret) { - mutex_unlock(&q->elevator_lock); - goto out_crypto_sysfs_unregister; - } - } wbt_enable_default(disk); mutex_unlock(&q->elevator_lock); @@ -902,8 +896,6 @@ int blk_register_queue(struct gendisk *disk) return ret; -out_crypto_sysfs_unregister: - blk_crypto_sysfs_unregister(disk); out_unregister_ia_ranges: disk_unregister_independent_access_ranges(disk); out_debugfs_remove: @@ -949,9 +941,11 @@ void blk_unregister_queue(struct gendisk *disk) blk_mq_sysfs_unregister(disk); blk_crypto_sysfs_unregister(disk); - mutex_lock(&q->elevator_lock); - elv_unregister_queue(q); - mutex_unlock(&q->elevator_lock); + if (q->elevator) { + blk_mq_quiesce_queue(q); + elevator_set_none(q); + blk_mq_unquiesce_queue(q); + } mutex_lock(&q->sysfs_lock); disk_unregister_independent_access_ranges(disk); diff --git a/block/blk.h b/block/blk.h index 922a429b5363..4626beedfdce 100644 --- a/block/blk.h +++ b/block/blk.h @@ -321,9 +321,8 @@ bool blk_bio_list_merge(struct request_queue *q, struct list_head *list, bool blk_insert_flush(struct request *rq); int __elevator_change(struct request_queue *q, struct elev_change_ctx *ctx); -void elevator_exit(struct request_queue *q); -int elv_register_queue(struct request_queue *q, bool uevent); -void elv_unregister_queue(struct request_queue *q); +void elevator_set_default(struct request_queue *q); +void elevator_set_none(struct request_queue *q); ssize_t part_size_show(struct device *dev, struct device_attribute *attr, char *buf); diff --git a/block/elevator.c b/block/elevator.c index 2bc1679dcd1f..7d2a56ef0be6 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -151,7 +151,7 @@ static void elevator_release(struct kobject *kobj) kfree(e); } -void elevator_exit(struct request_queue *q) +static void elevator_exit(struct request_queue *q) { struct elevator_queue *e = q->elevator; @@ -455,7 +455,7 @@ static const struct kobj_type elv_ktype = { .release = elevator_release, }; -int elv_register_queue(struct request_queue *q, bool uevent) +static int elv_register_queue(struct request_queue *q, bool uevent) { struct elevator_queue *e = q->elevator; int error; @@ -484,7 +484,7 @@ int elv_register_queue(struct request_queue *q, bool uevent) return error; } -void elv_unregister_queue(struct request_queue *q) +static void elv_unregister_queue(struct request_queue *q) { struct elevator_queue *e = q->elevator; @@ -560,60 +560,56 @@ EXPORT_SYMBOL_GPL(elv_unregister); * For single queue devices, default to using mq-deadline. If we have multiple * queues or mq-deadline is not available, default to "none". */ -static struct elevator_type *elevator_get_default(struct request_queue *q) +static bool use_default_elevator(struct request_queue *q) { if (q->tag_set->flags & BLK_MQ_F_NO_SCHED_BY_DEFAULT) - return NULL; + return false; if (q->nr_hw_queues != 1 && !blk_mq_is_shared_tags(q->tag_set->flags)) - return NULL; + return false; - return elevator_find_get("mq-deadline"); + return true; } /* * Use the default elevator settings. If the chosen elevator initialization * fails, fall back to the "none" elevator (no elevator). */ -void elevator_init_mq(struct request_queue *q) +void elevator_set_default(struct request_queue *q) { - struct elevator_type *e; - unsigned int memflags; + struct elev_change_ctx ctx = { }; int err; - WARN_ON_ONCE(blk_queue_registered(q)); - - if (unlikely(q->elevator)) + if (!queue_is_mq(q)) return; - e = elevator_get_default(q); - if (!e) + ctx.name = use_default_elevator(q) ? "mq-deadline" : "none"; + if (!q->elevator && !strcmp(ctx.name, "none")) return; + err = elevator_change(q, &ctx); + if (err < 0) + pr_warn("\"%s\" set elevator failed %d, " + "falling back to \"none\"\n", ctx.name, err); +} - /* - * We are called before adding disk, when there isn't any FS I/O, - * so freezing queue plus canceling dispatch work is enough to - * drain any dispatch activities originated from passthrough - * requests, then no need to quiesce queue which may add long boot - * latency, especially when lots of disks are involved. - * - * Disk isn't added yet, so verifying queue lock only manually. - */ - memflags = blk_mq_freeze_queue(q); - - blk_mq_cancel_work_sync(q); - - err = blk_mq_init_sched(q, e); +void elevator_set_none(struct request_queue *q) +{ + struct elev_change_ctx ctx = { + .name = "none", + .uevent = 1, + }; + int err; - blk_mq_unfreeze_queue(q, memflags); + if (!queue_is_mq(q)) + return; - if (err) { - pr_warn("\"%s\" elevator initialization failed, " - "falling back to \"none\"\n", e->elevator_name); - } + if (!q->elevator) + return; - elevator_put(e); + err = elevator_change(q, &ctx); + if (err < 0) + pr_warn("%s: set none elevator failed %d\n", __func__, err); } /* @@ -718,6 +714,16 @@ static int elevator_change(struct request_queue *q, } memflags = blk_mq_freeze_queue(q); + /* + * May be called before adding disk, when there isn't any FS I/O, + * so freezing queue plus canceling dispatch work is enough to + * drain any dispatch activities originated from passthrough + * requests, then no need to quiesce queue which may add long boot + * latency, especially when lots of disks are involved. + * + * Disk isn't added yet, so verifying queue lock only manually. + */ + blk_mq_cancel_work_sync(q); mutex_lock(&q->elevator_lock); ret = __elevator_change(q, ctx); mutex_unlock(&q->elevator_lock); diff --git a/block/genhd.c b/block/genhd.c index f426c13edf55..d7264546a178 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -416,12 +416,6 @@ int __must_check add_disk_fwnode(struct device *parent, struct gendisk *disk, */ if (disk->fops->submit_bio || disk->fops->poll_bio) return -EINVAL; - - /* - * Initialize the I/O scheduler code and pick a default one if - * needed. - */ - elevator_init_mq(disk->queue); } else { if (!disk->fops->submit_bio) return -EINVAL; @@ -565,11 +559,7 @@ int __must_check add_disk_fwnode(struct device *parent, struct gendisk *disk, if (disk->major == BLOCK_EXT_MAJOR) blk_free_ext_minor(disk->first_minor); out_exit_elevator: - if (disk->queue->elevator) { - mutex_lock(&disk->queue->elevator_lock); - elevator_exit(disk->queue); - mutex_unlock(&disk->queue->elevator_lock); - } + elevator_set_none(disk->queue); return ret; } EXPORT_SYMBOL_GPL(add_disk_fwnode); @@ -743,14 +733,7 @@ void del_gendisk(struct gendisk *disk) if (queue_is_mq(q)) blk_mq_cancel_work_sync(q); - blk_mq_quiesce_queue(q); - if (q->elevator) { - mutex_lock(&q->elevator_lock); - elevator_exit(q); - mutex_unlock(&q->elevator_lock); - } rq_qos_exit(q); - blk_mq_unquiesce_queue(q); /* * If the disk does not own the queue, allow using passthrough requests From patchwork Thu Apr 10 13:30:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046521 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74D6B28A40B for ; Thu, 10 Apr 2025 13:31:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291902; cv=none; b=lgzcTEtpayjRrbBmeS+Bk9bN1rxkAlHy5r4yoseKfZ6svDqQby8D1SikYLg3ctgFGr3Mk/sz3XlHMMZ3ibTlXBqUh+SVQPwTZyF9s4F9XU1kYXtiuCNzanxtYBVUxFH5mW2jpLV7rQ99M/2Pxh4HQUz+FJVEp9ryIFUUCxuEc30= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291902; c=relaxed/simple; bh=720h5vI4UjCgHiu/oDG2Y1EmVTVJSMg2oU7RN5i3+SA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uSqG1iCFmeZdIrFtnolOp2bbf947HIo7oIl5Uno9pOgLOMeb3IzhSMDxZVlq9icsTWwnfJ0lILvKptnC3tRa8CE5gWjmMAB1TU4iZ0WNznL4Gu7PXlZFAY4audtkmMFI9GsEgSyVsjiGgzeqxm+QNihqfM5mKjiTQbFRjPGeXfs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=PnUbRmga; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="PnUbRmga" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291899; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YHrLWHPyLHEITbCLh+w++Qah5C8kzi6YcdiT3KUsi/c=; b=PnUbRmgaQr28//abIkg3SWnOzs6lwkJvva5tdSyPZYo/abdqvtme1tT29V1nGCVvzIdWwS jtwbrw3DIrBcv+LNbXSsbxrw6mWf0K/ALOVIAGJvij2XAGJx/EVQs/WFJ3Y+ZnyWlG8jik X/jYL1BWZamolZ2UApLwnYGUVL3hNH4= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-421-7O3gLIvOPxWsq2vvaUpZMA-1; Thu, 10 Apr 2025 09:31:38 -0400 X-MC-Unique: 7O3gLIvOPxWsq2vvaUpZMA-1 X-Mimecast-MFC-AGG-ID: 7O3gLIvOPxWsq2vvaUpZMA_1744291897 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 25257195609E; Thu, 10 Apr 2025 13:31:37 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0AE2E1956094; Thu, 10 Apr 2025 13:31:35 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 10/15] block: pass elevator_queue to elv_register_queue & unregister_queue Date: Thu, 10 Apr 2025 21:30:22 +0800 Message-ID: <20250410133029.2487054-11-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Pass elevator_queue reference to elv_register_queue() & elv_unregister_queue(). No functional change, and prepare for moving the two out of elevator lock. Signed-off-by: Ming Lei --- block/elevator.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/block/elevator.c b/block/elevator.c index 7d2a56ef0be6..238b8d47cc2b 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -455,9 +455,10 @@ static const struct kobj_type elv_ktype = { .release = elevator_release, }; -static int elv_register_queue(struct request_queue *q, bool uevent) +static int elv_register_queue(struct request_queue *q, + struct elevator_queue *e, + bool uevent) { - struct elevator_queue *e = q->elevator; int error; lockdep_assert_held(&q->elevator_lock); @@ -484,10 +485,9 @@ static int elv_register_queue(struct request_queue *q, bool uevent) return error; } -static void elv_unregister_queue(struct request_queue *q) +static void elv_unregister_queue(struct request_queue *q, + struct elevator_queue *e) { - struct elevator_queue *e = q->elevator; - lockdep_assert_held(&q->elevator_lock); if (e && test_and_clear_bit(ELEVATOR_FLAG_REGISTERED, &e->flags)) { @@ -629,7 +629,7 @@ static int elevator_switch(struct request_queue *q, struct elevator_type *new_e, blk_mq_quiesce_queue(q); if (q->elevator) { - elv_unregister_queue(q); + elv_unregister_queue(q, q->elevator); elevator_exit(q); } @@ -637,7 +637,7 @@ static int elevator_switch(struct request_queue *q, struct elevator_type *new_e, if (ret) goto out_unfreeze; - ret = elv_register_queue(q, ctx->uevent); + ret = elv_register_queue(q, q->elevator, ctx->uevent); if (ret) { elevator_exit(q); goto out_unfreeze; @@ -662,7 +662,7 @@ static void elevator_disable(struct request_queue *q) blk_mq_quiesce_queue(q); - elv_unregister_queue(q); + elv_unregister_queue(q, q->elevator); elevator_exit(q); blk_queue_flag_clear(QUEUE_FLAG_SQ_SCHED, q); q->elevator = NULL; From patchwork Thu Apr 10 13:30:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046522 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B6F92857F1 for ; Thu, 10 Apr 2025 13:31:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291909; cv=none; b=Pkyy9WflBOagjFeGQqTI/FqedkUUD9V8EvI33sQUbHw5XU1ZTjtt3yNtJvUilLOqcxq+qsb3O/zhz3mq9hCsYiJWswmSLkdfLuV0Lv/kNeeeXGdX4LhBD9GSkkV6oaryi3pAHKf7A73ComJv2NE38r6UMAeSeT+a/+Bi2bCDh+w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291909; c=relaxed/simple; bh=sQzBRb6djE14fXxwyoanIqkwnJlxILhasCQNJ3/imu4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=R1vmmCcFSonCKX2Y0HgGgVXGD7kC6wRLKn5z6jvr11S3rbTKYCSekSjnMoJ+n51TQOCAxWeTdCjpQ6LGc2TTI/BibFkYJEElmGyyLf2RABac65hC+L5akWSU7Tl5ENY7S9BEsMFkljJYXJ8jEoT7indEajlpOSFfJq94nFBdLp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GDe0wmkJ; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GDe0wmkJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291907; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a29raZsIKbstEY4RZDUaY5m2siILLtua0KYgV7Zwkfs=; b=GDe0wmkJEu6mBhr9nm9zm4bM2FW53QI1Sa7EoKilQYJCAXEPUzLfseGiSxiv5k+H5lteZh X0M0Rsv+pUFnurRt54CaZ9rZnsxEDrhOgnZxTvdq3Bx8INStMSCwWRpk7xv+WyQ0+Zf76A Njjt3o69UZUyfoZVeWjUWxKeaV19eiA= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-532-rXAxqA58PsOpBQxGmTj_Tg-1; Thu, 10 Apr 2025 09:31:43 -0400 X-MC-Unique: rXAxqA58PsOpBQxGmTj_Tg-1 X-Mimecast-MFC-AGG-ID: rXAxqA58PsOpBQxGmTj_Tg_1744291901 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7E6061800361; Thu, 10 Apr 2025 13:31:41 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 880D5180174E; Thu, 10 Apr 2025 13:31:40 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 11/15] block: move elv_register[unregister]_queue out of elevator_lock Date: Thu, 10 Apr 2025 21:30:23 +0800 Message-ID: <20250410133029.2487054-12-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Move elv_register[unregister]_queue out of ->elevator_lock & queue freezing, so we can kill many lockdep warnings. elv_register[unregister]_queue() is serialized, and just dealing with sysfs/debugfs things, no need to be done with queue frozen. Signed-off-by: Ming Lei --- block/blk-mq.c | 9 ++++---- block/blk.h | 1 + block/elevator.c | 58 ++++++++++++++++++++++++++++++++++-------------- block/elevator.h | 5 +++++ 4 files changed, 52 insertions(+), 21 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 608a74e3a87c..7219b01764da 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4990,16 +4990,17 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, .force = 1, .uevent = 1, }; + int ret = -ENODEV; mutex_lock(&q->elevator_lock); if (q->elevator && !blk_queue_dying(q)) ctx.name = q->elevator->type->elevator_name; - __elevator_change(q, &ctx); + ret = __elevator_change(q, &ctx); mutex_unlock(&q->elevator_lock); - } - - list_for_each_entry(q, &set->tag_list, tag_set_list) blk_mq_unfreeze_queue_nomemrestore(q); + if (!ret) + WARN_ON_ONCE(elevator_change_done(q, &ctx)); + } memalloc_noio_restore(memflags); /* Free the excess tags when nr_hw_queues shrink. */ diff --git a/block/blk.h b/block/blk.h index 4626beedfdce..634cebd7a7b4 100644 --- a/block/blk.h +++ b/block/blk.h @@ -321,6 +321,7 @@ bool blk_bio_list_merge(struct request_queue *q, struct list_head *list, bool blk_insert_flush(struct request *rq); int __elevator_change(struct request_queue *q, struct elev_change_ctx *ctx); +int elevator_change_done(struct request_queue *q, struct elev_change_ctx *ctx); void elevator_set_default(struct request_queue *q); void elevator_set_none(struct request_queue *q); diff --git a/block/elevator.c b/block/elevator.c index 238b8d47cc2b..1cc640a9db3e 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -151,18 +151,24 @@ static void elevator_release(struct kobject *kobj) kfree(e); } -static void elevator_exit(struct request_queue *q) +static void __elevator_exit(struct request_queue *q) { struct elevator_queue *e = q->elevator; + lockdep_assert_held(&q->elevator_lock); + ioc_clear_queue(q); blk_mq_sched_free_rqs(q); mutex_lock(&e->sysfs_lock); blk_mq_exit_sched(q, e); mutex_unlock(&e->sysfs_lock); +} - kobject_put(&e->kobj); +static void elevator_exit(struct request_queue *q) +{ + __elevator_exit(q); + kobject_put(&q->elevator->kobj); } static inline void __elv_rqhash_del(struct request *rq) @@ -461,8 +467,6 @@ static int elv_register_queue(struct request_queue *q, { int error; - lockdep_assert_held(&q->elevator_lock); - if (test_bit(ELEVATOR_FLAG_REGISTERED, &e->flags)) return 0; @@ -488,8 +492,6 @@ static int elv_register_queue(struct request_queue *q, static void elv_unregister_queue(struct request_queue *q, struct elevator_queue *e) { - lockdep_assert_held(&q->elevator_lock); - if (e && test_and_clear_bit(ELEVATOR_FLAG_REGISTERED, &e->flags)) { kobject_uevent(&e->kobj, KOBJ_REMOVE); kobject_del(&e->kobj); @@ -629,19 +631,15 @@ static int elevator_switch(struct request_queue *q, struct elevator_type *new_e, blk_mq_quiesce_queue(q); if (q->elevator) { - elv_unregister_queue(q, q->elevator); - elevator_exit(q); + ctx->old = q->elevator; + __elevator_exit(q); } ret = blk_mq_init_sched(q, new_e); if (ret) goto out_unfreeze; - ret = elv_register_queue(q, q->elevator, ctx->uevent); - if (ret) { - elevator_exit(q); - goto out_unfreeze; - } + ctx->new = q->elevator; blk_add_trace_msg(q, "elv switch: %s", new_e->elevator_name); out_unfreeze: @@ -655,15 +653,16 @@ static int elevator_switch(struct request_queue *q, struct elevator_type *new_e, return ret; } -static void elevator_disable(struct request_queue *q) +static void elevator_disable(struct request_queue *q, + struct elev_change_ctx *ctx) { WARN_ON_ONCE(q->mq_freeze_depth == 0); lockdep_assert_held(&q->elevator_lock); blk_mq_quiesce_queue(q); - elv_unregister_queue(q, q->elevator); - elevator_exit(q); + ctx->old = q->elevator; + __elevator_exit(q); blk_queue_flag_clear(QUEUE_FLAG_SQ_SCHED, q); q->elevator = NULL; q->nr_requests = q->tag_set->queue_depth; @@ -672,6 +671,28 @@ static void elevator_disable(struct request_queue *q) blk_mq_unquiesce_queue(q); } +int elevator_change_done(struct request_queue *q, struct elev_change_ctx *ctx) +{ + int ret = 0; + + if (ctx->old) { + elv_unregister_queue(q, ctx->old); + kobject_put(&ctx->old->kobj); + } + if (ctx->new) { + ret = elv_register_queue(q, ctx->new, ctx->uevent); + if (ret) { + unsigned memflags = blk_mq_freeze_queue(q); + + mutex_lock(&q->elevator_lock); + elevator_exit(q); + mutex_unlock(&q->elevator_lock); + blk_mq_unfreeze_queue(q, memflags); + } + } + return 0; +} + /* * Switch this queue to the given IO scheduler. */ @@ -683,7 +704,7 @@ int __elevator_change(struct request_queue *q, struct elev_change_ctx *ctx) if (!strncmp(elevator_name, "none", 4)) { if (q->elevator) - elevator_disable(q); + elevator_disable(q, ctx); return 0; } @@ -728,6 +749,9 @@ static int elevator_change(struct request_queue *q, ret = __elevator_change(q, ctx); mutex_unlock(&q->elevator_lock); blk_mq_unfreeze_queue(q, memflags); + if (!ret) + ret = elevator_change_done(q, ctx); + exit: srcu_read_unlock(&set->update_nr_hwq_srcu, idx); return ret; diff --git a/block/elevator.h b/block/elevator.h index 86b4977cf772..87848fdc8a52 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -128,6 +128,11 @@ struct elev_change_ctx { const char *name; unsigned int force:1; unsigned int uevent:1; + + /* for unregistering old elevator */ + struct elevator_queue *old; + /* for registering new elevator */ + struct elevator_queue *new; }; /* From patchwork Thu Apr 10 13:30:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046523 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 401AD14658C for ; Thu, 10 Apr 2025 13:31:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291913; cv=none; b=t8l09jCgR1moznWxpSR46kohAtzJcq8qZHRoPPkVnGWRivYD7woLcnYrfTUGcBKBazn/OIqsxCINMQw18mdl4FRizDIkoAp6oM1BShSOEVvZIgEn3IJ3fWKkaCmCJUgc3wX+/0o+2gn3I06qtEb3niELrAzlck9op/jwWGlBMTA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291913; c=relaxed/simple; bh=zvqKWt+3xoyEHl+hQxKvpTEV0brKUKAKJYo6GsnHYkE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lFQjM4P6hOkGUQK8I7lCIY3ZTAVo8waZVql/QTPppIjqBPYYZ7wMLWXJ1bjgwCIhMFBbBji9/r4SmEYHVZTJE2/KdqFNUGaHc8z2uAHm4sZbiFcNY9fhOLeuKlhWqcSrV+IvQiOvV6XBku+NXMyZpfhzftEF/x3wsJu/Si6qjAM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AdnU1qQ0; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AdnU1qQ0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291911; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0tyDZz+LhXObHxXJHusskvh98XURw3zdwnCWMduKl+k=; b=AdnU1qQ0LIpAWxXfEDfSQu0BxH4POAs/Lgmq6yj1Q92HWpAb4z1KNEfDvzS7iWWEIv4Zqa DBJ7nvpv2b75DZBbqWK/+q0aj0xv6eW8ld/ro4+K/NeCXxDQvEyYlViMTkY/EGJ+rSRHOy 5Fq3VLq9bKE/5eDBqbQ279uc8bsklmw= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-438-mw6AmN2QP_O8QBfymRWR1Q-1; Thu, 10 Apr 2025 09:31:47 -0400 X-MC-Unique: mw6AmN2QP_O8QBfymRWR1Q-1 X-Mimecast-MFC-AGG-ID: mw6AmN2QP_O8QBfymRWR1Q_1744291906 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 36E3C180AF73; Thu, 10 Apr 2025 13:31:46 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1B8903001D15; Thu, 10 Apr 2025 13:31:44 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 12/15] block: move debugfs/sysfs register out of freezing queue Date: Thu, 10 Apr 2025 21:30:24 +0800 Message-ID: <20250410133029.2487054-13-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Move debugfs/sysfs register out of freezing queue in __blk_mq_update_nr_hw_queues(), so that the following lockdep dependency can be killed: #2 (&q->q_usage_counter(io)#16){++++}-{0:0}: #1 (fs_reclaim){+.+.}-{0:0}: #0 (&sb->s_type->i_mutex_key#3){+.+.}-{4:4}: //debugfs And registering/un-registering debugfs/sysfs does not require queue to be frozen. Signed-off-by: Ming Lei --- block/blk-mq.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 7219b01764da..0fb72a698d77 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4947,15 +4947,15 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, if (set->nr_maps == 1 && nr_hw_queues == set->nr_hw_queues) return; - memflags = memalloc_noio_save(); - list_for_each_entry(q, &set->tag_list, tag_set_list) - blk_mq_freeze_queue_nomemsave(q); - list_for_each_entry(q, &set->tag_list, tag_set_list) { blk_mq_debugfs_unregister_hctxs(q); blk_mq_sysfs_unregister_hctxs(q); } + memflags = memalloc_noio_save(); + list_for_each_entry(q, &set->tag_list, tag_set_list) + blk_mq_freeze_queue_nomemsave(q); + if (blk_mq_realloc_tag_set_tags(set, nr_hw_queues) < 0) goto reregister; @@ -4978,12 +4978,6 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, blk_mq_map_swqueue(q); } -reregister: - list_for_each_entry(q, &set->tag_list, tag_set_list) { - blk_mq_sysfs_register_hctxs(q); - blk_mq_debugfs_register_hctxs(q); - } - list_for_each_entry(q, &set->tag_list, tag_set_list) { struct elev_change_ctx ctx = { .name = "none", @@ -5003,6 +4997,12 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, } memalloc_noio_restore(memflags); +reregister: + list_for_each_entry(q, &set->tag_list, tag_set_list) { + blk_mq_sysfs_register_hctxs(q); + blk_mq_debugfs_register_hctxs(q); + } + /* Free the excess tags when nr_hw_queues shrink. */ for (i = set->nr_hw_queues; i < prev_nr_hw_queues; i++) __blk_mq_free_map_and_rqs(set, i); From patchwork Thu Apr 10 13:30:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046524 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB4CC155725 for ; Thu, 10 Apr 2025 13:31:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291920; cv=none; b=UggKE6fp80OOrMGTcHuK/0uZcR7juiPsbadPqmch9bBAx8cxmDQ57idD3iTb0ia/pz4vFoXG5KrnjuxbseLYF97qr0qwBUsQhxU8MEdLEbrWBoDzNwfBIxv/SKV3wgD53RhY3jtAj49aflLd5EbND/iaryEForVpBSirI1UxQ94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291920; c=relaxed/simple; bh=5Hvf/YMszIxRm8ZbOrrI4xqZ6200+EzXnKquIG8NtW8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SepEUiDkL4DcNR1qF8OdlKgP8s6iRIUpicm6hRd8Yv2Ce3x8ytU5BB69EELKXodTX3Bq73Isy8sMUSuf3pKaF1E32DHGJLTjWqxKjKIYI/0H2JqYi4db11NmCffAtTgDYpwdE4GwLjFW8rxvWn+WkdSweoT/T5/0YhnAtUYRSAo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=jNGsGMtk; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="jNGsGMtk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291917; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=133+/K49MltfKZhkXEIcW++J9FEWj1MGGw5FK8LNxEg=; b=jNGsGMtkOdeQxDE6nMg5mngrSCVnYgzS33ryv2nEeJhL47qHAIyFVa+BZWd/6KYT0xX3pr TmmcZoI8dFHWOwnKClm+oOw7ag+dLn17vM8wfI57IzuI5zF5hGsCah22KUwrYXvuMJvITJ lObHchPntjX0paURUgBIm+ZrVWus7rU= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-198-hpWabvu7NDGCtm7HL3wqzQ-1; Thu, 10 Apr 2025 09:31:52 -0400 X-MC-Unique: hpWabvu7NDGCtm7HL3wqzQ-1 X-Mimecast-MFC-AGG-ID: hpWabvu7NDGCtm7HL3wqzQ_1744291911 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F31CC1955BC6; Thu, 10 Apr 2025 13:31:50 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C282D1808882; Thu, 10 Apr 2025 13:31:49 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 13/15] block: remove several ->elevator_lock Date: Thu, 10 Apr 2025 21:30:25 +0800 Message-ID: <20250410133029.2487054-14-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Both blk_mq_map_swqueue() and blk_mq_realloc_hw_ctxs() are only called from queue initialization or updating nr_hw_queues code, in which elevator switch can't happen any more. So remove these ->elevator_lock uses. Signed-off-by: Ming Lei --- block/blk-mq.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 0fb72a698d77..812dfe759b89 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4095,8 +4095,6 @@ static void blk_mq_map_swqueue(struct request_queue *q) struct blk_mq_ctx *ctx; struct blk_mq_tag_set *set = q->tag_set; - mutex_lock(&q->elevator_lock); - queue_for_each_hw_ctx(q, hctx, i) { cpumask_clear(hctx->cpumask); hctx->nr_ctx = 0; @@ -4201,8 +4199,6 @@ static void blk_mq_map_swqueue(struct request_queue *q) hctx->next_cpu = blk_mq_first_mapped_cpu(hctx); hctx->next_cpu_batch = BLK_MQ_CPU_WORK_BATCH; } - - mutex_unlock(&q->elevator_lock); } /* @@ -4506,16 +4502,9 @@ static void __blk_mq_realloc_hw_ctxs(struct blk_mq_tag_set *set, } static void blk_mq_realloc_hw_ctxs(struct blk_mq_tag_set *set, - struct request_queue *q, bool lock) + struct request_queue *q) { - if (lock) { - /* protect against switching io scheduler */ - mutex_lock(&q->elevator_lock); - __blk_mq_realloc_hw_ctxs(set, q); - mutex_unlock(&q->elevator_lock); - } else { - __blk_mq_realloc_hw_ctxs(set, q); - } + __blk_mq_realloc_hw_ctxs(set, q); /* unregister cpuhp callbacks for exited hctxs */ blk_mq_remove_hw_queues_cpuhp(q); @@ -4547,7 +4536,7 @@ int blk_mq_init_allocated_queue(struct blk_mq_tag_set *set, xa_init(&q->hctx_table); - blk_mq_realloc_hw_ctxs(set, q, false); + blk_mq_realloc_hw_ctxs(set, q); if (!q->nr_hw_queues) goto err_hctxs; @@ -4962,7 +4951,7 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, fallback: blk_mq_update_queue_map(set); list_for_each_entry(q, &set->tag_list, tag_set_list) { - blk_mq_realloc_hw_ctxs(set, q, true); + blk_mq_realloc_hw_ctxs(set, q); if (q->nr_hw_queues != set->nr_hw_queues) { int i = prev_nr_hw_queues; From patchwork Thu Apr 10 13:30:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046525 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DCDE428A40C for ; Thu, 10 Apr 2025 13:32:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291924; cv=none; b=ZPzIcXtIWR5Io58DoZC9BaHj24xSHcTV3Sz0CiV8lwwUVGwBtC1PYaTciWNcKcUvA25NnFbGNrUr+vWtpBBhrwHN8h0Mbg3c55ROBOr42nmLr6OGk8fpmWwDRjUvw/ZYzQRpMoNP4cB7J8+cmunLMLTByMt2VdK2IqqUjWeyN3Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291924; c=relaxed/simple; bh=VB1M2mV5t8p1L3diMbdJnyvVVpKTN9mVy/v0Ym5r2Rc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KbcNcShg/16sPG375JmU0e0mcl4wABXi8ZqtNBt6jihK/GmkPcCQHJSplUzTx1E57fw5lQLmw1YliPtrf7I4Mxr3nlOviRc69LEjb2osd6SwxhccouZMVjbptmMaok2HS4Und+5Eoy2iyyUzEcSLIUAbow7VKTjsQAlSn0eJWa8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=HgYDMC2Z; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HgYDMC2Z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cymcRY9iDUYr6d17XwcfJiNspOFwnByYtZxgrOE3+hs=; b=HgYDMC2Z+ipcBlSPsxhde98oD6m4Xkf1k3C5Ltp/ooytcz0hNvHRkRzYZOXAZnFzNEPUzo 9yhKZAzM8LgQr5x+GqA2VRYdAif/Ntt3uCXd0xDXWZjXDxrToO9NDL2oLvU6U/+yKNDW/9 BDy0AmxFRNNk3DMdvOed45O0RM8jUz8= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-62-Cyb6U6YhNNW0pWNPjMdEMA-1; Thu, 10 Apr 2025 09:31:58 -0400 X-MC-Unique: Cyb6U6YhNNW0pWNPjMdEMA-1 X-Mimecast-MFC-AGG-ID: Cyb6U6YhNNW0pWNPjMdEMA_1744291915 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B5834180035C; Thu, 10 Apr 2025 13:31:55 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9A22C1955DCE; Thu, 10 Apr 2025 13:31:54 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 14/15] block: move hctx cpuhp add/del out of queue freezing Date: Thu, 10 Apr 2025 21:30:26 +0800 Message-ID: <20250410133029.2487054-15-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Move hctx cpuhp add/del out of queue freezing for not connecting freeze lock with cpuhp locks, then lockdep warning can be avoided. This way is safe because both needn't queue to be frozen and scheduler switch isn't allowed. Signed-off-by: Ming Lei --- block/blk-mq.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 812dfe759b89..d721e55c0844 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4951,7 +4951,7 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, fallback: blk_mq_update_queue_map(set); list_for_each_entry(q, &set->tag_list, tag_set_list) { - blk_mq_realloc_hw_ctxs(set, q); + __blk_mq_realloc_hw_ctxs(set, q); if (q->nr_hw_queues != set->nr_hw_queues) { int i = prev_nr_hw_queues; @@ -4990,6 +4990,9 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, list_for_each_entry(q, &set->tag_list, tag_set_list) { blk_mq_sysfs_register_hctxs(q); blk_mq_debugfs_register_hctxs(q); + + blk_mq_remove_hw_queues_cpuhp(q); + blk_mq_add_hw_queues_cpuhp(q); } /* Free the excess tags when nr_hw_queues shrink. */ From patchwork Thu Apr 10 13:30:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14046526 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E58DB188006 for ; Thu, 10 Apr 2025 13:32:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291927; cv=none; b=uazxnxMz+AWf8VSmT38vPIjt1unJ41BJwqHIawwxmchKA0JjcrB69Kxum5dRDaJ7e0Sl8dQ0W4F5ZitZscMQFjOc4FBr5+BpvcEbJfOtyFBAT5etnJBenl59/ZWpmZ8uDIGdv5k2Q1VMzpUpBVT1e/Mryj/bskWb2wmt0+EqS3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744291927; c=relaxed/simple; bh=k4XS5BBXWJoOZ9an0OYny3cUGfP2t9D0aCKxih9/Evw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ISA7L4umjB6CNOK6rtxUVcNOMpw0Pp65Do7L4rA8YmZHIaBlTdb9dkSGP8Pd2FqTuPqV/RqaYBcYOkNBhpCYjTtsle+hqe0Q5T/2gCm3KKbEBHhseVqRB1v3Cy0YBWTV7hhRKD9YjHrB5YZDDjaYLg9Mub6goYL69ndxL8bcVZE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fpPpYvf+; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fpPpYvf+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744291925; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BKHd7cs1y9A1ReQXiJleGLMIXRcUpiEW5ZZKSVQRBkk=; b=fpPpYvf+2mJmk6T0PI/xm9L/NQonbSv3Pu5RsLSiIyjoU2wGtoDF+fSfSd3Ot3igFvf96d UZzGqPOX6+28+hLbduk/vZfMvf12/xzjuVoZqPRlZGLQX+Cv5YMnN0FRemKJyBa+U477br vUmt8xK5trrrgMEZIeGgX/3gS3tjlwE= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-360-nMfwbdOrMvmt2UDFz5IrMA-1; Thu, 10 Apr 2025 09:32:01 -0400 X-MC-Unique: nMfwbdOrMvmt2UDFz5IrMA-1 X-Mimecast-MFC-AGG-ID: nMfwbdOrMvmt2UDFz5IrMA_1744291920 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 049D319560A2; Thu, 10 Apr 2025 13:32:00 +0000 (UTC) Received: from localhost (unknown [10.72.120.20]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 104A53001D0E; Thu, 10 Apr 2025 13:31:58 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Nilay Shroff , Shinichiro Kawasaki , =?utf-8?q?Thomas_Hellstr?= =?utf-8?q?=C3=B6m?= , Christoph Hellwig , Ming Lei Subject: [PATCH 15/15] block: move wbt_enable_default() out of queue freezing from scheduler's ->exit() Date: Thu, 10 Apr 2025 21:30:27 +0800 Message-ID: <20250410133029.2487054-16-ming.lei@redhat.com> In-Reply-To: <20250410133029.2487054-1-ming.lei@redhat.com> References: <20250410133029.2487054-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 scheduler's ->exit() is called with queue frozen and elevator lock is held, and wbt_enable_default() can't be called with queue frozen, otherwise the following lockdep warning is triggered: #6 (&q->rq_qos_mutex){+.+.}-{4:4}: #5 (&eq->sysfs_lock){+.+.}-{4:4}: #4 (&q->elevator_lock){+.+.}-{4:4}: #3 (&q->q_usage_counter(io)#3){++++}-{0:0}: #2 (fs_reclaim){+.+.}-{0:0}: #1 (&sb->s_type->i_mutex_key#3){+.+.}-{4:4}: #0 (&q->debugfs_mutex){+.+.}-{4:4}: Fix the issue by moving wbt_enable_default() out of bfq's exit(), and call it from elevator_change_done(). Signed-off-by: Ming Lei --- block/bfq-iosched.c | 2 +- block/elevator.c | 5 +++++ block/elevator.h | 1 + 3 files changed, 7 insertions(+), 1 deletion(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index abd80dc13562..18018c8cf84d 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -7211,7 +7211,7 @@ static void bfq_exit_queue(struct elevator_queue *e) blk_stat_disable_accounting(bfqd->queue); clear_bit(ELEVATOR_FLAG_DISABLE_WBT, &e->flags); - wbt_enable_default(bfqd->queue->disk); + set_bit(ELEVATOR_FLAG_ENABLE_WBT_ON_EXIT, &e->flags); kfree(bfqd); } diff --git a/block/elevator.c b/block/elevator.c index 1cc640a9db3e..9cf78db4d6a4 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -676,8 +676,13 @@ int elevator_change_done(struct request_queue *q, struct elev_change_ctx *ctx) int ret = 0; if (ctx->old) { + bool enable_wbt = test_bit(ELEVATOR_FLAG_ENABLE_WBT_ON_EXIT, + &ctx->old->flags); + elv_unregister_queue(q, ctx->old); kobject_put(&ctx->old->kobj); + if (enable_wbt) + wbt_enable_default(q->disk); } if (ctx->new) { ret = elv_register_queue(q, ctx->new, ctx->uevent); diff --git a/block/elevator.h b/block/elevator.h index 87848fdc8a52..7b66be5b8295 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -122,6 +122,7 @@ struct elevator_queue #define ELEVATOR_FLAG_REGISTERED 0 #define ELEVATOR_FLAG_DISABLE_WBT 1 +#define ELEVATOR_FLAG_ENABLE_WBT_ON_EXIT 2 /* Holding data for changing elevator */ struct elev_change_ctx {