From patchwork Mon Aug 21 09:56:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 13359294 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40A7FEE4993 for ; Mon, 21 Aug 2023 09:57:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232995AbjHUJ5h (ORCPT ); Mon, 21 Aug 2023 05:57:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232343AbjHUJ5g (ORCPT ); Mon, 21 Aug 2023 05:57:36 -0400 Received: from out-49.mta1.migadu.com (out-49.mta1.migadu.com [IPv6:2001:41d0:203:375::31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D2FFA9 for ; Mon, 21 Aug 2023 02:57:35 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1692611853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=HTQQLcZjy5SOyzVREu6p8o8uUAQapiAefMRwTkFEijM=; b=N3xaoeoc/ub7B7aT+bcMxfGMMOZ4MaMCof7Tj9Fam2czFP1dDEkSYNQZPY6gFgtIn5PgbU 4t6Xk1VOKTbfF7PROi3MAcfecOdsQ1dM3mtrRiDT1qHZ1da6wM1r2W0ahfSQW5CvfE8UmK Mn560Q2cidZVTJxkY1MukQ+s7n72wv4= From: chengming.zhou@linux.dev To: axboe@kernel.dk, hch@lst.de, ming.lei@redhat.com, bvanassche@acm.org Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhouchengming@bytedance.com Subject: [PATCH 1/3] blk-mq: fix tags leak when shrink nr_hw_queues Date: Mon, 21 Aug 2023 17:56:00 +0800 Message-ID: <20230821095602.70742-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Chengming Zhou Although we don't need to realloc set->tags[] when shrink nr_hw_queues, we need to free them. Or these tags will be leaked. How to reproduce: 1. mount -t configfs configfs /mnt 2. modprobe null_blk nr_devices=0 submit_queues=8 3. mkdir /mnt/nullb/nullb0 4. echo 1 > /mnt/nullb/nullb0/power 5. echo 4 > /mnt/nullb/nullb0/submit_queues 6. rmdir /mnt/nullb/nullb0 In step 4, will alloc 9 tags (8 submit queues and 1 poll queue), then in step 5, new_nr_hw_queues = 5 (4 submit queues and 1 poll queue). At last in step 6, only these 5 tags are freed, the other 4 tags leaked. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei --- block/blk-mq.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index f14b8669ac69..8af29a3b1400 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4386,9 +4386,13 @@ static int blk_mq_realloc_tag_set_tags(struct blk_mq_tag_set *set, int new_nr_hw_queues) { struct blk_mq_tags **new_tags; + int i; - if (set->nr_hw_queues >= new_nr_hw_queues) + if (set->nr_hw_queues >= new_nr_hw_queues) { + for (i = new_nr_hw_queues; i < set->nr_hw_queues; i++) + __blk_mq_free_map_and_rqs(set, i); goto done; + } new_tags = kcalloc_node(new_nr_hw_queues, sizeof(struct blk_mq_tags *), GFP_KERNEL, set->numa_node); From patchwork Mon Aug 21 09:56:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 13359295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8593EE49A5 for ; Mon, 21 Aug 2023 09:57:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234505AbjHUJ5y (ORCPT ); Mon, 21 Aug 2023 05:57:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234502AbjHUJ5x (ORCPT ); Mon, 21 Aug 2023 05:57:53 -0400 Received: from out-50.mta1.migadu.com (out-50.mta1.migadu.com [95.215.58.50]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41901ED for ; Mon, 21 Aug 2023 02:57:49 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1692611867; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m9OilRxGI20nC45Qzhn26jvVe/s07+OS9kKShML1tRI=; b=wW9g+0zD8xkhGCgoG44/Kl2S/T4TCRmzJH8wWm1V8Qucsr1OCG9BiPrdc9QzC9rynMpVKN BCARgSaSRDgqWrSviRVOh/rOUG0Wxw7IwbBclL7zCuleeQclKQowEG3AK7B/73aK3LM3WI bxpBOg6GNl1OqPQVGtzvuVR2ww5dxyM= From: chengming.zhou@linux.dev To: axboe@kernel.dk, hch@lst.de, ming.lei@redhat.com, bvanassche@acm.org Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhouchengming@bytedance.com Subject: [PATCH 2/3] blk-mq: delete redundant tagset map update when fallback Date: Mon, 21 Aug 2023 17:56:01 +0800 Message-ID: <20230821095602.70742-2-chengming.zhou@linux.dev> In-Reply-To: <20230821095602.70742-1-chengming.zhou@linux.dev> References: <20230821095602.70742-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Chengming Zhou When we increase nr_hw_queues fail, the fallback path will use blk_mq_update_queue_map() to clear and update all maps. Obviously, this line of update of HCTX_TYPE_DEFAULT only is not needed, so delete it. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei --- block/blk-mq.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 8af29a3b1400..8d02bafff331 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4737,7 +4737,6 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, __blk_mq_free_map_and_rqs(set, i); set->nr_hw_queues = prev_nr_hw_queues; - blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]); goto fallback; } blk_mq_map_swqueue(q); From patchwork Mon Aug 21 09:56:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 13359296 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 141B0EE49A5 for ; Mon, 21 Aug 2023 09:57:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234519AbjHUJ56 (ORCPT ); Mon, 21 Aug 2023 05:57:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234515AbjHUJ54 (ORCPT ); Mon, 21 Aug 2023 05:57:56 -0400 Received: from out-23.mta1.migadu.com (out-23.mta1.migadu.com [IPv6:2001:41d0:203:375::17]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A43BBE7 for ; Mon, 21 Aug 2023 02:57:52 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1692611871; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rkDR5NLi5YNWQzKxCk4+sjUxTF0L3YdBSik5mAIut1w=; b=l5ybMqIqiUClbR21w3/Cb97G6tbSdSA0tJhEWUpmZkiEDxXXMER3F22eTiWGLZSu7ydzZj G+2+giVlTmmnohHbywwXsoBMdpk50xMSC1JkoCxAiqJiBJ40sHF/xusFIClHDZAR8xWw8f rp9qYb2Z5/OJmezHY6xS73PByAWlHDM= From: chengming.zhou@linux.dev To: axboe@kernel.dk, hch@lst.de, ming.lei@redhat.com, bvanassche@acm.org Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhouchengming@bytedance.com Subject: [PATCH 3/3] blk-mq: prealloc tags when increase tagset nr_hw_queues Date: Mon, 21 Aug 2023 17:56:02 +0800 Message-ID: <20230821095602.70742-3-chengming.zhou@linux.dev> In-Reply-To: <20230821095602.70742-1-chengming.zhou@linux.dev> References: <20230821095602.70742-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Chengming Zhou Just like blk_mq_alloc_tag_set(), it's better to prepare all tags before using to map to queue ctxs in blk_mq_map_swqueue(), which now have to consider empty set->tags[]. The good point is that we can fallback easily if increasing nr_hw_queues fail, instead of just mapping to hctx[0] when fail in blk_mq_map_swqueue(). And the fallback path already has tags free & clean handling, so all is good. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei --- block/blk-mq.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index 8d02bafff331..84adb3657b30 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4404,6 +4404,16 @@ static int blk_mq_realloc_tag_set_tags(struct blk_mq_tag_set *set, sizeof(*set->tags)); kfree(set->tags); set->tags = new_tags; + + for (i = set->nr_hw_queues; i < new_nr_hw_queues; i++) { + if (!__blk_mq_alloc_map_and_rqs(set, i)) { + while (--i >= set->nr_hw_queues) + __blk_mq_free_map_and_rqs(set, i); + return -ENOMEM; + } + cond_resched(); + } + done: set->nr_hw_queues = new_nr_hw_queues; return 0;