From patchwork Tue May 22 17:46:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10419241 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B0FAF6032A for ; Tue, 22 May 2018 17:46:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8FDBE28F55 for ; Tue, 22 May 2018 17:46:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 80B8228F88; Tue, 22 May 2018 17:46:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 198D428F55 for ; Tue, 22 May 2018 17:46:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751343AbeEVRqP (ORCPT ); Tue, 22 May 2018 13:46:15 -0400 Received: from mail-io0-f193.google.com ([209.85.223.193]:32908 "EHLO mail-io0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751279AbeEVRqO (ORCPT ); Tue, 22 May 2018 13:46:14 -0400 Received: by mail-io0-f193.google.com with SMTP id e78-v6so19491210iod.0 for ; Tue, 22 May 2018 10:46:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=h90bgq7B1KEtpcra7zaPdjj+vzMhjFuoHihjWsPJSOM=; b=VQDB3nrXrIKeVRx0W5Wh9jzIdR4dgaqLIskmLA1wik5jt+o/NpPycD/NwjbwWUMPOd K32YvNc/Ey9mRMrZNko0BCXIgWCVGAdd0/CyRJsnlsuT8N1GVkzH8xWpK8jLh9EGJKS9 xZhyQ4d9xRcsHFLm5YruhXdciQYybnsCvtl8buDggdh6oddM3iTWKFWPjgypSBVzOP69 f0QwXv1HXtZTwiPm4XFFt3V5r3UM9/RkABIzg6Vca+FLMv/t/x/7ET8hhUJpkQKqI3ku ZG2vmYb6lMuxezSiG2PZU/LmkmHYm49pI2mn08tzIxRt0BnfgCiDavQ6VJ2towHDGC4N L3Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=h90bgq7B1KEtpcra7zaPdjj+vzMhjFuoHihjWsPJSOM=; b=L2O4gX6guz3NBSIXWSTXMQquHElW61rvYGjTFKs4AzJyn0qjsYQxzjVCAh7YG0a3tx L6hLHxk+wOVJSinidG0lZ/v5VcDvhMVORhYVQz9056N21J7e8yXy93B2tkMAqMEA08fU ccx8yoqE0+DTpuc4jIKKEiHNLepk3pyD30mUHKI4tAnowWfoWgLfDieUjdhagS36sNZe pSQC3KhVHLyqZqCzswUMhpZ9RqVPrgi4ITKM+hS4nxzjuE1NwaBZZfgJfXWvmWQBQOL+ z0IjyfeWrdtZk9e/Y/2nb+JwYOXtc1ycZe1GA4yOwZrgrdvL4gr5nI4blD7hjTao2caG yKGg== X-Gm-Message-State: ALKqPwek4zdhzxcjBm2OOn4TQGimC04SiiJPD/jU7cWeaHjvRqBBJbIN cMKT24kdU4vNlTFf/ppGIgxFrQ== X-Google-Smtp-Source: AB8JxZqQQ4OJd9Xa7HcOw3HISzQNTZo6FDl+stWuHJfcUcS0yPejQmwe39R0jkMAml/y0vQ1yPR89A== X-Received: by 2002:a6b:591:: with SMTP id 139-v6mr24113786iof.282.1527011173434; Tue, 22 May 2018 10:46:13 -0700 (PDT) Received: from [192.168.1.167] ([216.160.245.98]) by smtp.gmail.com with ESMTPSA id d16-v6sm9253778iob.74.2018.05.22.10.46.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 May 2018 10:46:12 -0700 (PDT) Subject: Re: [PATCH] block: kyber: make kyber more friendly with merging From: Jens Axboe To: =?UTF-8?Q?Holger_Hoffst=c3=a4tte?= , Jianchao Wang Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org References: <1527000509-2619-1-git-send-email-jianchao.w.wang@oracle.com> <163c5760-cf0e-faea-ee4e-ac5d688310fe@applied-asynchrony.com> <00e5dfd4-d3a2-b008-3d8b-390a788f61c9@kernel.dk> Message-ID: <7c93b85f-6a02-bdcf-5e92-a9533500df20@kernel.dk> Date: Tue, 22 May 2018 11:46:11 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: <00e5dfd4-d3a2-b008-3d8b-390a788f61c9@kernel.dk> Content-Language: en-US Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 5/22/18 10:20 AM, Jens Axboe wrote: > On 5/22/18 10:17 AM, Holger Hoffstätte wrote: >> On 05/22/18 16:48, Jianchao Wang wrote: >>> Currently, kyber is very unfriendly with merging. kyber depends >>> on ctx rq_list to do merging, however, most of time, it will not >>> leave any requests in ctx rq_list. This is because even if tokens >>> of one domain is used up, kyber will try to dispatch requests >>> from other domain and flush the rq_list there. >>> >>> To improve this, we setup kyber_ctx_queue (kcq) which is similar >>> with ctx, but it has rq_lists for different domain and build same >>> mapping between kcq and khd as the ctx & hctx. Then we could merge, >>> insert and dispatch for different domains separately. If one domain >>> token is used up, the requests could be left in the rq_list of >>> that domain and maybe merged with following io. >>> >>> Following is my test result on machine with 8 cores and NVMe card >>> INTEL SSDPEKKR128G7 >>> >>> fio size=256m ioengine=libaio iodepth=64 direct=1 numjobs=8 >>> seq/random >>> +------+---------------------------------------------------------------+ >>> |patch?| bw(MB/s) | iops | slat(usec) | clat(usec) | merge | >>> +----------------------------------------------------------------------+ >>> | w/o | 606/612 | 151k/153k | 6.89/7.03 | 3349.21/3305.40 | 0/0 | >>> +----------------------------------------------------------------------+ >>> | w/ | 1083/616 | 277k/154k | 4.93/6.95 | 1830.62/3279.95 | 223k/3k | >>> +----------------------------------------------------------------------+ >>> When set numjobs to 16, the bw and iops could reach 1662MB/s and 425k >>> on my platform. >>> >>> Signed-off-by: Jianchao Wang >> >> >> >> This looks great but prevents kyber from being built as module, >> which is AFAIK supposed to work (and works now): >> >> .. >> CC [M] block/kyber-iosched.o >> Building modules, stage 2. >> MODPOST 313 modules >> ERROR: "bio_attempt_back_merge" [block/kyber-iosched.ko] undefined! >> ERROR: "bio_attempt_front_merge" [block/kyber-iosched.ko] undefined! >> ERROR: "bio_attempt_discard_merge" [block/kyber-iosched.ko] undefined! >> ERROR: "blk_try_merge" [block/kyber-iosched.ko] undefined! >> ERROR: "blk_rq_merge_ok" [block/kyber-iosched.ko] undefined! >> .. >> >> It does build fine when compiled in, obviously. :) > > It's basically duplicating the contents of blk_attempt_plug_merge(). > I would suggest abstracting out the list loop and merge check > into a helper, that could then both be called from kyber and the > plug merge function. See attached, prep patch and yours rebased on top of it. Tested-by: Holger Hoffstätte From 82d8018412c12407fbfb134c7be5410ba5a76084 Mon Sep 17 00:00:00 2001 From: Jens Axboe Date: Tue, 22 May 2018 11:41:46 -0600 Subject: [PATCH 1/2] blk-mq: abstract out blk-mq-sched rq list iteration bio merge helper No functional changes in this patch, just a prep patch for utilizing this in an IO scheduler. Signed-off-by: Jens Axboe --- block/blk-mq-sched.c | 34 ++++++++++++++++++++++++---------- include/linux/blk-mq.h | 3 ++- 2 files changed, 26 insertions(+), 11 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index 25c14c58385c..b0f2c2a40a0c 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -268,19 +268,16 @@ bool blk_mq_sched_try_merge(struct request_queue *q, struct bio *bio, EXPORT_SYMBOL_GPL(blk_mq_sched_try_merge); /* - * Reverse check our software queue for entries that we could potentially - * merge with. Currently includes a hand-wavy stop count of 8, to not spend - * too much time checking for merges. + * Iterate list of requests and see if we can merge this bio with any + * of them. */ -static bool blk_mq_attempt_merge(struct request_queue *q, - struct blk_mq_ctx *ctx, struct bio *bio) +bool blk_mq_bio_list_merge(struct request_queue *q, struct list_head *list, + struct bio *bio) { struct request *rq; int checked = 8; - lockdep_assert_held(&ctx->lock); - - list_for_each_entry_reverse(rq, &ctx->rq_list, queuelist) { + list_for_each_entry_reverse(rq, list, queuelist) { bool merged = false; if (!checked--) @@ -305,13 +302,30 @@ static bool blk_mq_attempt_merge(struct request_queue *q, continue; } - if (merged) - ctx->rq_merged++; return merged; } return false; } +EXPORT_SYMBOL_GPL(blk_mq_bio_list_merge); + +/* + * Reverse check our software queue for entries that we could potentially + * merge with. Currently includes a hand-wavy stop count of 8, to not spend + * too much time checking for merges. + */ +static bool blk_mq_attempt_merge(struct request_queue *q, + struct blk_mq_ctx *ctx, struct bio *bio) +{ + lockdep_assert_held(&ctx->lock); + + if (blk_mq_bio_list_merge(q, &ctx->rq_list, bio)) { + ctx->rq_merged++; + return true; + } + + return false; +} bool __blk_mq_sched_bio_merge(struct request_queue *q, struct bio *bio) { diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index ebc34a5686dc..fb355173f3c7 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -259,7 +259,8 @@ void blk_mq_add_to_requeue_list(struct request *rq, bool at_head, void blk_mq_kick_requeue_list(struct request_queue *q); void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs); void blk_mq_complete_request(struct request *rq); - +bool blk_mq_bio_list_merge(struct request_queue *q, struct list_head *list, + struct bio *bio); bool blk_mq_queue_stopped(struct request_queue *q); void blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx); void blk_mq_start_hw_queue(struct blk_mq_hw_ctx *hctx); -- 2.7.4