From patchwork Wed Jun 16 13:05:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12325137 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89B6DC49361 for ; Wed, 16 Jun 2021 13:07:05 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A6D86135C for ; Wed, 16 Jun 2021 13:07:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A6D86135C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=tempfail smtp.mailfrom=dm-devel-bounces@redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623848824; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=KnM5isQukCikJIZkG5vtnFmFYgnrf4Pht1BrtBc0QNY=; b=bdDoEosbFpEPw1jWeWbibBsE4lY8Ym9OE5KU0/fBNjAA2dgUjKWAEXxfUez4WgQDpxW0lu pYrfCgO12L/NXMXmYmrs5cINTKOvBuczIhQxiPMYIrwS9rAw/mkOFyeHmY6KcLg4KOimiS hIID/KcbEcSsThsF4EcDcmimaxe62fg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-225-PWw9mS0xOyyTBRbapJ4Xig-1; Wed, 16 Jun 2021 09:07:02 -0400 X-MC-Unique: PWw9mS0xOyyTBRbapJ4Xig-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0E15AA40C0; Wed, 16 Jun 2021 13:06:58 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C1DB61002EF0; Wed, 16 Jun 2021 13:06:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 129C11809CAE; Wed, 16 Jun 2021 13:06:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 15GD6t3Y001412 for ; Wed, 16 Jun 2021 09:06:55 -0400 Received: by smtp.corp.redhat.com (Postfix) id 221275C1C5; Wed, 16 Jun 2021 13:06:55 +0000 (UTC) Received: from localhost (ovpn-12-48.pek2.redhat.com [10.72.12.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id E5C835C26D; Wed, 16 Jun 2021 13:06:10 +0000 (UTC) From: Ming Lei To: Jens Axboe , Mike Snitzer Date: Wed, 16 Jun 2021 21:05:30 +0800 Message-Id: <20210616130533.754248-2-ming.lei@redhat.com> In-Reply-To: <20210616130533.754248-1-ming.lei@redhat.com> References: <20210616130533.754248-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: dm-devel@redhat.com Cc: Chaitanya Kulkarni , Ming Lei , linux-block@vger.kernel.org, dm-devel@redhat.com, Jeffle Xu , Christoph Hellwig Subject: [dm-devel] [RFC PATCH 1/4] block: add helper of blk_queue_poll X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com There has been 3 users, and will be more, so add one such helper. Reviewed-by: Chaitanya Kulkarni Reviewed-by: Jeffle Xu Reviewed-by: Hannes Reinecke Signed-off-by: Ming Lei --- block/blk-core.c | 5 ++--- block/blk-sysfs.c | 4 ++-- drivers/nvme/host/core.c | 2 +- include/linux/blkdev.h | 1 + 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 531176578221..1e24c71c6738 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -835,7 +835,7 @@ static noinline_for_stack bool submit_bio_checks(struct bio *bio) } } - if (!test_bit(QUEUE_FLAG_POLL, &q->queue_flags)) + if (!blk_queue_poll(q)) bio->bi_opf &= ~REQ_POLLED; switch (bio_op(bio)) { @@ -1117,8 +1117,7 @@ int bio_poll(struct bio *bio, unsigned int flags) blk_qc_t cookie = READ_ONCE(bio->bi_cookie); int ret; - if (cookie == BLK_QC_T_NONE || - !test_bit(QUEUE_FLAG_POLL, &q->queue_flags)) + if (cookie == BLK_QC_T_NONE || !blk_queue_poll(q)) return 0; if (current->plug) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index f78e73ca6091..93dcf2dfaafd 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -422,13 +422,13 @@ static ssize_t queue_poll_delay_store(struct request_queue *q, const char *page, static ssize_t queue_poll_show(struct request_queue *q, char *page) { - return queue_var_show(test_bit(QUEUE_FLAG_POLL, &q->queue_flags), page); + return queue_var_show(blk_queue_poll(q), page); } static ssize_t queue_poll_store(struct request_queue *q, const char *page, size_t count) { - if (!test_bit(QUEUE_FLAG_POLL, &q->queue_flags)) + if (!blk_queue_poll(q)) return -EINVAL; pr_info_ratelimited("writes to the poll attribute are ignored.\n"); pr_info_ratelimited("please use driver specific parameters instead.\n"); diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index fe0b8da3de7f..e31c7704ef4d 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -1025,7 +1025,7 @@ static void nvme_execute_rq_polled(struct request_queue *q, { DECLARE_COMPLETION_ONSTACK(wait); - WARN_ON_ONCE(!test_bit(QUEUE_FLAG_POLL, &q->queue_flags)); + WARN_ON_ONCE(!blk_queue_poll(q)); rq->cmd_flags |= REQ_POLLED; rq->end_io_data = &wait; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 561b04117bd4..fc0ba0b80776 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -677,6 +677,7 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #define blk_queue_fua(q) test_bit(QUEUE_FLAG_FUA, &(q)->queue_flags) #define blk_queue_registered(q) test_bit(QUEUE_FLAG_REGISTERED, &(q)->queue_flags) #define blk_queue_nowait(q) test_bit(QUEUE_FLAG_NOWAIT, &(q)->queue_flags) +#define blk_queue_poll(q) test_bit(QUEUE_FLAG_POLL, &(q)->queue_flags) extern void blk_set_pm_only(struct request_queue *q); extern void blk_clear_pm_only(struct request_queue *q); From patchwork Wed Jun 16 13:05:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12325139 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AA10C48BE5 for ; Wed, 16 Jun 2021 13:07:10 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C756B6024A for ; Wed, 16 Jun 2021 13:07:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C756B6024A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=tempfail smtp.mailfrom=dm-devel-bounces@redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623848829; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=FJlMNoUhsdxjOSGwagQiwGux4ll5bFYAxHS/VKQwRuE=; b=DAhGub2oWbkgLHKWd8oXhbeINSZspeFTxze+zBFT1SqCI5EiQUmm2HihhjTsx0OHGYBFId SeFYmNb4hkPPBkXO4jZGLXwY1vbC3QaJSQdyu2bPiXuQIzgJ9VllFkokDbr7kwrpUMC6gs PPFBtxpvsMfHQKRl7r9RApC7U5iYUNM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-376-S9HPkXU3P92Xi8siumThjg-1; Wed, 16 Jun 2021 09:07:07 -0400 X-MC-Unique: S9HPkXU3P92Xi8siumThjg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5B7D1100C62A; Wed, 16 Jun 2021 13:07:00 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 408236060F; Wed, 16 Jun 2021 13:07:00 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 94FA346F59; Wed, 16 Jun 2021 13:06:59 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 15GD6wM1001422 for ; Wed, 16 Jun 2021 09:06:58 -0400 Received: by smtp.corp.redhat.com (Postfix) id 447AE60CC9; Wed, 16 Jun 2021 13:06:58 +0000 (UTC) Received: from localhost (ovpn-12-48.pek2.redhat.com [10.72.12.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7159160C13; Wed, 16 Jun 2021 13:06:57 +0000 (UTC) From: Ming Lei To: Jens Axboe , Mike Snitzer Date: Wed, 16 Jun 2021 21:05:31 +0800 Message-Id: <20210616130533.754248-3-ming.lei@redhat.com> In-Reply-To: <20210616130533.754248-1-ming.lei@redhat.com> References: <20210616130533.754248-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: dm-devel@redhat.com Cc: Ming Lei , linux-block@vger.kernel.org, dm-devel@redhat.com, Jeffle Xu , Christoph Hellwig Subject: [dm-devel] [RFC PATCH 2/4] block: add field of .bi_bio_drv_data to bio X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com After bio is submitted, bio->bi_next is used for IO merge for request based queue only. Reuse the filed for bio based driver for storing driver data, and name it as .bi_bio_drv_data. Signed-off-by: Ming Lei --- include/linux/blk_types.h | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 6e6c2af48d74..acb45213338c 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -220,7 +220,16 @@ typedef unsigned int blk_qc_t; * stacking drivers) */ struct bio { - struct bio *bi_next; /* request queue link */ + union { + /* request queue link */ + struct bio *bi_next; + + /* + * once bio is submitted to bio based queue, driver can use + * this field to store its own data + */ + void *bi_bio_drv_data; + }; struct block_device *bi_bdev; unsigned int bi_opf; /* bottom bits req flags, * top bits REQ_OP. Use From patchwork Wed Jun 16 13:05:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12325141 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0F1AC48BE8 for ; Wed, 16 Jun 2021 13:07:10 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6C7C5613BD for ; Wed, 16 Jun 2021 13:07:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C7C5613BD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=tempfail smtp.mailfrom=dm-devel-bounces@redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623848829; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=o+h4PPJwICc0yAB7a1SnpG8GbHLQ8pzRFWIdNrGFHbg=; b=btNVz5kVyUyJJ8bhwQ77cA5FMBWG7UedmageWvVy/p1wk9Y9DFE1zZ6b0wLOqUXB7HLKTP /emyq0BKOiGjuiEjpc9rHcRV0axpeZkAeyahrkhppSWebLFkKfmPn3gCrUu+9KjSrzbHHW FzBNBHnIizD4typ0IHN+mEXwpKVca0c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-260-2tQQsUb0Pd-8LT5Mhq7lrg-1; Wed, 16 Jun 2021 09:07:08 -0400 X-MC-Unique: 2tQQsUb0Pd-8LT5Mhq7lrg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6EE6980A5FE; Wed, 16 Jun 2021 13:07:02 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 462031002F12; Wed, 16 Jun 2021 13:07:02 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 1A76C46F5B; Wed, 16 Jun 2021 13:07:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 15GD70Fk001433 for ; Wed, 16 Jun 2021 09:07:01 -0400 Received: by smtp.corp.redhat.com (Postfix) id F104260CC9; Wed, 16 Jun 2021 13:07:00 +0000 (UTC) Received: from localhost (ovpn-12-48.pek2.redhat.com [10.72.12.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4723860C13; Wed, 16 Jun 2021 13:06:59 +0000 (UTC) From: Ming Lei To: Jens Axboe , Mike Snitzer Date: Wed, 16 Jun 2021 21:05:32 +0800 Message-Id: <20210616130533.754248-4-ming.lei@redhat.com> In-Reply-To: <20210616130533.754248-1-ming.lei@redhat.com> References: <20210616130533.754248-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: dm-devel@redhat.com Cc: Ming Lei , linux-block@vger.kernel.org, dm-devel@redhat.com, Jeffle Xu , Christoph Hellwig Subject: [dm-devel] [RFC PATCH 3/4] block: add ->poll_bio to block_device_operations X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Prepare for supporting IO polling for bio based driver. Add ->poll_bio callback so that bio driver can provide their own logic for polling bio. Signed-off-by: Ming Lei --- block/blk-core.c | 18 +++++++++++++----- block/genhd.c | 3 +++ include/linux/blkdev.h | 2 ++ 3 files changed, 18 insertions(+), 5 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 1e24c71c6738..a1552ec8d608 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1113,11 +1113,13 @@ EXPORT_SYMBOL(submit_bio); */ int bio_poll(struct bio *bio, unsigned int flags) { - struct request_queue *q = bio->bi_bdev->bd_disk->queue; + struct gendisk *disk = bio->bi_bdev->bd_disk; + struct request_queue *q = disk->queue; blk_qc_t cookie = READ_ONCE(bio->bi_cookie); int ret; - if (cookie == BLK_QC_T_NONE || !blk_queue_poll(q)) + if ((queue_is_mq(q) && cookie == BLK_QC_T_NONE) || + !blk_queue_poll(q)) return 0; if (current->plug) @@ -1125,10 +1127,16 @@ int bio_poll(struct bio *bio, unsigned int flags) if (blk_queue_enter(q, BLK_MQ_REQ_NOWAIT)) return 0; - if (WARN_ON_ONCE(!queue_is_mq(q))) - ret = 0; /* not yet implemented, should not happen */ - else + if (!queue_is_mq(q)) { + if (disk->fops->poll_bio) { + ret = disk->fops->poll_bio(bio, flags); + } else { + WARN_ON_ONCE(1); + ret = 0; + } + } else { ret = blk_mq_poll(q, cookie, flags); + } blk_queue_exit(q); return ret; } diff --git a/block/genhd.c b/block/genhd.c index 5f5628216295..042dfa5b3f79 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -471,6 +471,9 @@ static void __device_add_disk(struct device *parent, struct gendisk *disk, { int ret; + /* ->poll_bio is only for bio based driver */ + WARN_ON_ONCE(queue_is_mq(disk->queue) && disk->fops->poll_bio); + /* * The disk queue should now be all set with enough information about * the device for the elevator code to pick an adequate default diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index fc0ba0b80776..6da6fb120148 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1858,6 +1858,8 @@ static inline void blk_ksm_unregister(struct request_queue *q) { } struct block_device_operations { void (*submit_bio)(struct bio *bio); + /* ->poll_bio is for bio driver only */ + int (*poll_bio)(struct bio *bio, unsigned int flags); int (*open) (struct block_device *, fmode_t); void (*release) (struct gendisk *, fmode_t); int (*rw_page)(struct block_device *, sector_t, struct page *, unsigned int); From patchwork Wed Jun 16 13:05:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12325143 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E674CC48BE5 for ; Wed, 16 Jun 2021 13:07:12 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9E5F261359 for ; Wed, 16 Jun 2021 13:07:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E5F261359 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=tempfail smtp.mailfrom=dm-devel-bounces@redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623848831; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=3bojTQnGyaGN1RATtwVvaK8oPWV14alusR0ulWxbDxc=; b=a2LakPiYZe8QHyyvrzVK7Yj/fikpbhuOMKvr2JCkSTLHamGnd9eUF0UPkAtXVJB98734zz ZhBrObwlLN1uBWdCteSMMaT4gnA6jKH5pmL5yPwA3QIa7kNp0ZrMLtYFxOy05MXm/vVM0O 2xOheNOaWyv0ilN5hyecIWhu34HoJPc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-560-KYme40loPmmZM5dFufWh-g-1; Wed, 16 Jun 2021 09:07:09 -0400 X-MC-Unique: KYme40loPmmZM5dFufWh-g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3549E1927800; Wed, 16 Jun 2021 13:07:05 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 187D05D9DE; Wed, 16 Jun 2021 13:07:05 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id DA0F246F59; Wed, 16 Jun 2021 13:07:04 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 15GD73xM001447 for ; Wed, 16 Jun 2021 09:07:03 -0400 Received: by smtp.corp.redhat.com (Postfix) id EC4B560C4A; Wed, 16 Jun 2021 13:07:03 +0000 (UTC) Received: from localhost (ovpn-12-48.pek2.redhat.com [10.72.12.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3E15D60C13; Wed, 16 Jun 2021 13:07:02 +0000 (UTC) From: Ming Lei To: Jens Axboe , Mike Snitzer Date: Wed, 16 Jun 2021 21:05:33 +0800 Message-Id: <20210616130533.754248-5-ming.lei@redhat.com> In-Reply-To: <20210616130533.754248-1-ming.lei@redhat.com> References: <20210616130533.754248-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: dm-devel@redhat.com Cc: Ming Lei , linux-block@vger.kernel.org, dm-devel@redhat.com, Jeffle Xu , Christoph Hellwig Subject: [dm-devel] [RFC PATCH 4/4] dm: support bio polling X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Support bio(REQ_POLLED) polling in the following approach: 1) setup one list in instance of 'struct dm_io', adds every 'struct dm_target_io' instance cloned for current dm bio into this list; store the list in 1) into bio->bi_bio_drv_data 2) hold one refcnt on io->io_count after submitting this dm bio with REQ_POLLED 4) implement .poll_bio() callback, and iterate over the list in 1) and polled on each ->clone of 'dm_target_io' instance; call dec_pending() if all target ios are done in .poll_bio(). 4) enable QUEUE_FLAG_POLL if all underlying queues enable QUEUE_FLAG_POLL, which is based on Jeffle's previous patch. Signed-off-by: Ming Lei --- drivers/md/dm-table.c | 24 ++++++++++++++++++ drivers/md/dm.c | 59 ++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 79 insertions(+), 4 deletions(-) diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index ee47a332b462..b14b379442d2 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1491,6 +1491,12 @@ struct dm_target *dm_table_find_target(struct dm_table *t, sector_t sector) return &t->targets[(KEYS_PER_NODE * n) + k]; } +static int device_not_poll_capable(struct dm_target *ti, struct dm_dev *dev, + sector_t start, sector_t len, void *data) +{ + return !blk_queue_poll(bdev_get_queue(dev->bdev)); +} + /* * type->iterate_devices() should be called when the sanity check needs to * iterate and check all underlying data devices. iterate_devices() will @@ -1541,6 +1547,11 @@ static int count_device(struct dm_target *ti, struct dm_dev *dev, return 0; } +static int dm_table_supports_poll(struct dm_table *t) +{ + return !dm_table_any_dev_attr(t, device_not_poll_capable, NULL); +} + /* * Check whether a table has no data devices attached using each * target's iterate_devices method. @@ -2078,6 +2089,19 @@ void dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, dm_update_keyslot_manager(q, t); blk_queue_update_readahead(q); + + /* + * Check for request-based device is remained to + * dm_mq_init_request_queue()->blk_mq_init_allocated_queue(). + * For bio-based device, only set QUEUE_FLAG_POLL when all underlying + * devices supporting polling. + */ + if (__table_type_bio_based(t->type)) { + if (dm_table_supports_poll(t)) + blk_queue_flag_set(QUEUE_FLAG_POLL, q); + else + blk_queue_flag_clear(QUEUE_FLAG_POLL, q); + } } unsigned int dm_table_get_num_targets(struct dm_table *t) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 363f12a285ce..0a0e4a38f435 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -84,6 +84,7 @@ struct dm_target_io { struct dm_target *ti; unsigned target_bio_nr; unsigned *len_ptr; + struct list_head list; bool inside_dm_io; struct bio clone; }; @@ -99,6 +100,7 @@ struct dm_io { blk_status_t status; atomic_t io_count; struct bio *orig_bio; + struct list_head poll_head; unsigned long start_time; spinlock_t endio_lock; struct dm_stats_aux stats_aux; @@ -655,6 +657,11 @@ static struct dm_io *alloc_io(struct mapped_device *md, struct bio *bio) io->md = md; spin_lock_init(&io->endio_lock); + if (bio->bi_opf & REQ_POLLED) { + bio->bi_bio_drv_data = io; + INIT_LIST_HEAD(&io->poll_head); + } + start_io_acct(io); return io; @@ -692,6 +699,8 @@ static struct dm_target_io *alloc_tio(struct clone_info *ci, struct dm_target *t static void free_tio(struct dm_target_io *tio) { + list_del_init(&tio->list); + if (tio->inside_dm_io) return; bio_put(&tio->clone); @@ -936,10 +945,15 @@ static void dec_pending(struct dm_io *io, blk_status_t error) io_error = io->status; bio = io->orig_bio; end_io_acct(io); + free_io(md, io); - if (io_error == BLK_STS_DM_REQUEUE) + if (io_error == BLK_STS_DM_REQUEUE) { + /* not poll any more in case of requeue */ + if (bio->bi_opf & REQ_POLLED) + bio->bi_opf &= ~REQ_POLLED; return; + } if ((bio->bi_opf & REQ_PREFLUSH) && bio->bi_iter.bi_size) { /* @@ -1043,7 +1057,9 @@ static void clone_endio(struct bio *bio) up(&md->swap_bios_semaphore); } - free_tio(tio); + /* Any cloned bio submitted as POLLED, free them all after dm_io is done */ + if (list_empty(&tio->list)) + free_tio(tio); dec_pending(io, error); } @@ -1300,6 +1316,11 @@ static void __map_bio(struct dm_target_io *tio) struct dm_io *io = tio->io; struct dm_target *ti = tio->ti; + if (clone->bi_opf & REQ_POLLED) + list_add_tail(&tio->list, &io->poll_head); + else + INIT_LIST_HEAD(&tio->list); + clone->bi_end_io = clone_endio; /* @@ -1666,8 +1687,9 @@ static void __split_and_process_bio(struct mapped_device *md, } } - /* drop the extra reference count */ - dec_pending(ci.io, errno_to_blk_status(error)); + /* drop the extra reference count for non-POLLED bio */ + if (!(bio->bi_opf & REQ_POLLED)) + dec_pending(ci.io, errno_to_blk_status(error)); } static void dm_submit_bio(struct bio *bio) @@ -1707,6 +1729,34 @@ static void dm_submit_bio(struct bio *bio) dm_put_live_table(md, srcu_idx); } +static int dm_poll_bio(struct bio *bio, unsigned int flags) +{ + struct dm_io *io = bio->bi_bio_drv_data; + struct dm_target_io *tio; + + if (!(bio->bi_opf & REQ_POLLED) || !io) + return 0; + + list_for_each_entry(tio, &io->poll_head, list) + bio_poll(&tio->clone, flags); + + /* bio_poll holds the last reference */ + if (atomic_read(&io->io_count) == 1) { + /* free all target IOs submitted as POLLED */ + while (!list_empty(&io->poll_head)) { + struct dm_target_io *tio = + list_entry(io->poll_head.next, + struct dm_target_io, list); + free_tio(tio); + } + bio->bi_bio_drv_data = NULL; + dec_pending(io, 0); + return 1; + } + + return 0; +} + /*----------------------------------------------------------------- * An IDR is used to keep track of allocated minor numbers. *---------------------------------------------------------------*/ @@ -3121,6 +3171,7 @@ static const struct pr_ops dm_pr_ops = { static const struct block_device_operations dm_blk_dops = { .submit_bio = dm_submit_bio, + .poll_bio = dm_poll_bio, .open = dm_blk_open, .release = dm_blk_close, .ioctl = dm_blk_ioctl,