From patchwork Thu May 9 17:01:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13659946 Received: from 008.lax.mailroute.net (008.lax.mailroute.net [199.89.1.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0BF7945A for ; Thu, 9 May 2024 17:02:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=199.89.1.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715274137; cv=none; b=UKf/zL3BRgZWUmf3eLDpxaLU0mBtl8Fd248HMfTYFP4P5uGrfKKd6aDRxahgiSIYuHdeMhDsByQjdj5YEqTeexlsnmjgSHrKK3lXkP32lXO6g9RRVqcQ3v9iAHCrsfmJOgaa+smZkgxBdpQeW+kpg97hMgQEvDm0yFJoSVICE8g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715274137; c=relaxed/simple; bh=E7BXl2V3/Opp4NtNtBuqMsiN7ZJ31jbkZhDWs9PH82s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=js5hxFhP3tVP9QyLCrdUN7EMbk3UmjQNn5K7sOmB/kbeHu9wzfbFjYx1MhKKk8tMugNaeGsm9GyrVg3wkr/UvhII+5bV6wlQjNr4JYSj6UffFQz/HMnZrg5m0SbJfWi9e8ou/3cBZadnFwoSwx8FpYcns5hXWPMl63jFP3r6bUg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=acm.org; spf=pass smtp.mailfrom=acm.org; dkim=pass (2048-bit key) header.d=acm.org header.i=@acm.org header.b=ZOnwVoNL; arc=none smtp.client-ip=199.89.1.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=acm.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=acm.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=acm.org header.i=@acm.org header.b="ZOnwVoNL" Received: from localhost (localhost [127.0.0.1]) by 008.lax.mailroute.net (Postfix) with ESMTP id 4VZyx31kHpz6Cnk8y; Thu, 9 May 2024 17:02:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h= content-transfer-encoding:mime-version:references:in-reply-to :x-mailer:message-id:date:date:subject:subject:from:from :received:received; s=mr01; t=1715274132; x=1717866133; bh=uJTF4 +vyTibGOAD6m1YeVB3bweRb8m2pAGp+fHDKSAM=; b=ZOnwVoNLRwApExMhgMXco JbnmVfJQiptR3Gu98A87IGYrYOUaKrp1bmXbfXRhc8OIdT4v+p0oSlDwofmxCzeX Ox8ZNY6IiJrB0coiN6T4lORbEiajlyRH4lt5sGGm7OX4io9HuJWHsP8dBaGuqRvf PcYSLUg9zuyKaX8ZqY5OzJDuUlYVIE9GXhxA3IRypX+PydmCH6aw7WBKGSP6cNV2 cO1J4R8xnnrKlncj9azobaI+0PkiPsq2fcQRtG1pjrsxg4Ur0L/ELMWynow+gj1P WXLd770vGzYHgOJD5NCRhpSA6P6xz293OUNy4nShHI+3+OYYLOKU/phPfqVui9n4 Q== X-Virus-Scanned: by MailRoute Received: from 008.lax.mailroute.net ([127.0.0.1]) by localhost (008.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP id 0jGfjXgnAc1u; Thu, 9 May 2024 17:02:12 +0000 (UTC) Received: from asus.hsd1.ca.comcast.net (c-73-231-117-72.hsd1.ca.comcast.net [73.231.117.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bvanassche@acm.org) by 008.lax.mailroute.net (Postfix) with ESMTPSA id 4VZywy5qbkz6Cnk8t; Thu, 9 May 2024 17:02:10 +0000 (UTC) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Zhiguo Niu Subject: [PATCH v2 1/2] block: Call .limit_depth() after .hctx has been set Date: Thu, 9 May 2024 10:01:48 -0700 Message-ID: <20240509170149.7639-2-bvanassche@acm.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240509170149.7639-1-bvanassche@acm.org> References: <20240509170149.7639-1-bvanassche@acm.org> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Call .limit_depth() after data->hctx has been set such that data->hctx can be used in .limit_depth() implementations. Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Zhiguo Niu Fixes: 07757588e507 ("block/mq-deadline: Reserve 25% of scheduler tags for synchronous requests") Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 9f677ea85a52..a6310a550b78 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -448,6 +448,10 @@ static struct request *__blk_mq_alloc_requests(struct blk_mq_alloc_data *data) if (data->cmd_flags & REQ_NOWAIT) data->flags |= BLK_MQ_REQ_NOWAIT; +retry: + data->ctx = blk_mq_get_ctx(q); + data->hctx = blk_mq_map_queue(q, data->cmd_flags, data->ctx); + if (q->elevator) { /* * All requests use scheduler tags when an I/O scheduler is @@ -469,13 +473,9 @@ static struct request *__blk_mq_alloc_requests(struct blk_mq_alloc_data *data) if (ops->limit_depth) ops->limit_depth(data->cmd_flags, data); } - } - -retry: - data->ctx = blk_mq_get_ctx(q); - data->hctx = blk_mq_map_queue(q, data->cmd_flags, data->ctx); - if (!(data->rq_flags & RQF_SCHED_TAGS)) + } else { blk_mq_tag_busy(data->hctx); + } if (data->flags & BLK_MQ_REQ_RESERVED) data->rq_flags |= RQF_RESV; From patchwork Thu May 9 17:01:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13659947 Received: from 008.lax.mailroute.net (008.lax.mailroute.net [199.89.1.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1D59C2ED for ; Thu, 9 May 2024 17:02:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=199.89.1.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715274140; cv=none; b=QzG6nxitjaSORydXjqGDzYVnw2x/rnuBKQytwUlMRF1lwepI1f8aoZn4AGREub/nW/5/iGip7k+voUUUgnXi3gTcVbHDJnRk9e134D5+AW6w63g0OvV6UgxDM5v1S9enpZumVbpVHOLH+kNKhXM6wrfYs+Lkx4rntBbJPrdxILs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715274140; c=relaxed/simple; bh=jhUZnInkrjo+riCu4qEmDel7/Cn84rpXdlBkKQXyCKc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rsFv5Xf/tEQ3RCBrjq2KCRqJz+AXYTIZcuSbxEYsg67cURY86sXay27o3VGO2G7IEUahgTg7U0qKhDx8siNcXhQB0znRUY/S98CNP596TQwHZd+U5Lg4iZ2qvh9ygOFEVdJZY63zRtcmHQWGYktsqDMwfj0opxR7S5sre83E1UU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=acm.org; spf=pass smtp.mailfrom=acm.org; dkim=pass (2048-bit key) header.d=acm.org header.i=@acm.org header.b=bP1jzvZ1; arc=none smtp.client-ip=199.89.1.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=acm.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=acm.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=acm.org header.i=@acm.org header.b="bP1jzvZ1" Received: from localhost (localhost [127.0.0.1]) by 008.lax.mailroute.net (Postfix) with ESMTP id 4VZyx63KRZz6Cnk8t; Thu, 9 May 2024 17:02:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h= content-transfer-encoding:mime-version:references:in-reply-to :x-mailer:message-id:date:date:subject:subject:from:from :received:received; s=mr01; t=1715274134; x=1717866135; bh=vfS5r xu/jVyPDLMfwcKKUVf4Usik4EvYrIok0TYMcWw=; b=bP1jzvZ1Z3TbQGSkeD/UI K3XWtg4mff/g81ggoEng+9hLt1k9TxCLMxHZojQz3ah3AvvAY6WSez1xq5rpXeIe 3x00yNWaTq5TuuiK05jBq+rOnE6yOBkfU30OJiJspjg4uAbltLTzs1roznD6Z229 V4d+HYyYDRVwjBS0P2Nrm46elr/Dj2ksOAPu3Kkh6xJsKEFYAi8IA77fHLrLK8wH xd5NAqtEMNAvnIpYfOEOUKiigIyGFgS8FLH89IL96ITrSHsUQj3H+yJXRBTtzrLl spfT4nqThcIWxjJkj2Jjedny1wiqPQDJhx5OOLsdCHf7U61i9b0ew7a5LWjzEHxi A== X-Virus-Scanned: by MailRoute Received: from 008.lax.mailroute.net ([127.0.0.1]) by localhost (008.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP id dJjVa4Aq2-WX; Thu, 9 May 2024 17:02:14 +0000 (UTC) Received: from asus.hsd1.ca.comcast.net (c-73-231-117-72.hsd1.ca.comcast.net [73.231.117.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bvanassche@acm.org) by 008.lax.mailroute.net (Postfix) with ESMTPSA id 4VZyx06DdCz6Cnk8s; Thu, 9 May 2024 17:02:12 +0000 (UTC) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Zhiguo Niu Subject: [PATCH v2 2/2] block/mq-deadline: Fix the tag reservation code Date: Thu, 9 May 2024 10:01:49 -0700 Message-ID: <20240509170149.7639-3-bvanassche@acm.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240509170149.7639-1-bvanassche@acm.org> References: <20240509170149.7639-1-bvanassche@acm.org> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The current tag reservation code is based on a misunderstanding of the meaning of data->shallow_depth. Fix the tag reservation code as follows: * By default, do not reserve any tags for synchronous requests because for certain use cases reserving tags reduces performance. See also Harshit Mogalapalli, [bug-report] Performance regression with fio sequential-write on a multipath setup, 2024-03-07 (https://lore.kernel.org/linux-block/5ce2ae5d-61e2-4ede-ad55-551112602401@oracle.com/) * Reduce min_shallow_depth to one because min_shallow_depth must be less than or equal any shallow_depth value. * Scale dd->async_depth from the range [1, nr_requests] to [1, bits_per_sbitmap_word]. Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Zhiguo Niu Fixes: 07757588e507 ("block/mq-deadline: Reserve 25% of scheduler tags for synchronous requests") Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig --- block/mq-deadline.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 94eede4fb9eb..acdc28756d9d 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -487,6 +487,20 @@ static struct request *dd_dispatch_request(struct blk_mq_hw_ctx *hctx) return rq; } +/* + * 'depth' is a number in the range 1..INT_MAX representing a number of + * requests. Scale it with a factor (1 << bt->sb.shift) / q->nr_requests since + * 1..(1 << bt->sb.shift) is the range expected by sbitmap_get_shallow(). + * Values larger than q->nr_requests have the same effect as q->nr_requests. + */ +static int dd_to_word_depth(struct blk_mq_hw_ctx *hctx, unsigned int qdepth) +{ + struct sbitmap_queue *bt = &hctx->sched_tags->bitmap_tags; + const unsigned int nrr = hctx->queue->nr_requests; + + return ((qdepth << bt->sb.shift) + nrr - 1) / nrr; +} + /* * Called by __blk_mq_alloc_request(). The shallow_depth value set by this * function is used by __blk_mq_get_tag(). @@ -503,7 +517,7 @@ static void dd_limit_depth(blk_opf_t opf, struct blk_mq_alloc_data *data) * Throttle asynchronous requests and writes such that these requests * do not block the allocation of synchronous requests. */ - data->shallow_depth = dd->async_depth; + data->shallow_depth = dd_to_word_depth(data->hctx, dd->async_depth); } /* Called by blk_mq_update_nr_requests(). */ @@ -513,9 +527,9 @@ static void dd_depth_updated(struct blk_mq_hw_ctx *hctx) struct deadline_data *dd = q->elevator->elevator_data; struct blk_mq_tags *tags = hctx->sched_tags; - dd->async_depth = max(1UL, 3 * q->nr_requests / 4); + dd->async_depth = q->nr_requests; - sbitmap_queue_min_shallow_depth(&tags->bitmap_tags, dd->async_depth); + sbitmap_queue_min_shallow_depth(&tags->bitmap_tags, 1); } /* Called by blk_mq_init_hctx() and blk_mq_init_sched(). */