From patchwork Thu Aug 10 10:08:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikulas Patocka X-Patchwork-Id: 13349205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E45F3C001B0 for ; Thu, 10 Aug 2023 10:09:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232180AbjHJKJ3 (ORCPT ); Thu, 10 Aug 2023 06:09:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231994AbjHJKJ2 (ORCPT ); Thu, 10 Aug 2023 06:09:28 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71D0F83 for ; Thu, 10 Aug 2023 03:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691662122; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=s9AiNKrVX8hfRXAiH8rdKAY0xfsUsj0VMC9UfNMLLnw=; b=N/MlXP1ChCSbCRkJbnjLNh5BRU56uCqBNFXAXabWZLYWzqQQn66Ex63PBx9CjSbdl3LQv9 sI0kURSwSmb7TW7itdIU9sjq4W3HJc4NKNRxf8OBxwYXwru2co2xFKg3TH/hGa9s/oKEUq cdp1AoyrgRHnb5EsfoekvU6zBOcxrv8= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-578-0QMvhMtAPzartjXIrlA_MQ-1; Thu, 10 Aug 2023 06:08:41 -0400 X-MC-Unique: 0QMvhMtAPzartjXIrlA_MQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BB7723C0D848; Thu, 10 Aug 2023 10:08:40 +0000 (UTC) Received: from file1-rdu.file-001.prod.rdu2.dc.redhat.com (unknown [10.11.5.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B020940C6F4E; Thu, 10 Aug 2023 10:08:40 +0000 (UTC) Received: by file1-rdu.file-001.prod.rdu2.dc.redhat.com (Postfix, from userid 12668) id C53AC30B9C07; Thu, 10 Aug 2023 10:08:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by file1-rdu.file-001.prod.rdu2.dc.redhat.com (Postfix) with ESMTP id BABF33F7CF; Thu, 10 Aug 2023 12:08:39 +0200 (CEST) Date: Thu, 10 Aug 2023 12:08:39 +0200 (CEST) From: Mikulas Patocka To: Jens Axboe cc: Li Nan , Zdenek Kabelac , Christoph Hellwig , Chaitanya Kulkarni , linux-block@vger.kernel.org, dm-devel@redhat.com Subject: [PATCH v3 1/4] brd: use a switch statement in brd_submit_bio In-Reply-To: <2dacc73-854-e71c-1746-99b017401c9a@redhat.com> Message-ID: References: <2dacc73-854-e71c-1746-99b017401c9a@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Use a switch statement in brd_submit_bio, so that the code will look better when we add support for more bio operations. Signed-off-by: Mikulas Patocka --- drivers/block/brd.c | 43 ++++++++++++++++++++++++++----------------- 1 file changed, 26 insertions(+), 17 deletions(-) Index: linux-2.6/drivers/block/brd.c =================================================================== --- linux-2.6.orig/drivers/block/brd.c +++ linux-2.6/drivers/block/brd.c @@ -243,29 +243,38 @@ out: static void brd_submit_bio(struct bio *bio) { struct brd_device *brd = bio->bi_bdev->bd_disk->private_data; - sector_t sector = bio->bi_iter.bi_sector; + sector_t sector; struct bio_vec bvec; struct bvec_iter iter; - bio_for_each_segment(bvec, bio, iter) { - unsigned int len = bvec.bv_len; - int err; + switch (bio_op(bio)) { + case REQ_OP_READ: + case REQ_OP_WRITE: + sector = bio->bi_iter.bi_sector; + bio_for_each_segment(bvec, bio, iter) { + unsigned int len = bvec.bv_len; + int err; - /* Don't support un-aligned buffer */ - WARN_ON_ONCE((bvec.bv_offset & (SECTOR_SIZE - 1)) || - (len & (SECTOR_SIZE - 1))); + /* Don't support un-aligned buffer */ + WARN_ON_ONCE((bvec.bv_offset & (SECTOR_SIZE - 1)) || + (len & (SECTOR_SIZE - 1))); - err = brd_do_bvec(brd, bvec.bv_page, len, bvec.bv_offset, - bio->bi_opf, sector); - if (err) { - if (err == -ENOMEM && bio->bi_opf & REQ_NOWAIT) { - bio_wouldblock_error(bio); - return; + err = brd_do_bvec(brd, bvec.bv_page, len, bvec.bv_offset, + bio->bi_opf, sector); + if (err) { + if (err == -ENOMEM && bio->bi_opf & REQ_NOWAIT) { + bio_wouldblock_error(bio); + return; + } + bio_io_error(bio); + return; + } + sector += len >> SECTOR_SHIFT; } - bio_io_error(bio); - return; - } - sector += len >> SECTOR_SHIFT; + break; + default: + bio->bi_status = BLK_STS_NOTSUPP; + break; } bio_endio(bio); From patchwork Thu Aug 10 10:09:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikulas Patocka X-Patchwork-Id: 13349206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09A54C001DE for ; Thu, 10 Aug 2023 10:10:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231283AbjHJKKC (ORCPT ); Thu, 10 Aug 2023 06:10:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233946AbjHJKKB (ORCPT ); Thu, 10 Aug 2023 06:10:01 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89CF3E0 for ; Thu, 10 Aug 2023 03:09:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691662155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Zmj5nzJTSHzE8zf1I1zUcK3keiQbhtTkmdKSaDQ9f0U=; b=U01m/qope0QhaAepEpfyMRMm+1FJOhs9hYMbTqDO48W7AMHz3gMRXrhX8UeTmb4kMeD/9g bM52r1KZVKN6o+hdoxoY+yeaLaS/fDpz0o2CgbqKtGf5r76UEVlUMJDHXZggbXYrYinvzd XprJ0f0hbefxc/A3ezxSqaCr06AxwcM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-665-btY38AADN2yPlZumBh7MJw-1; Thu, 10 Aug 2023 06:09:12 -0400 X-MC-Unique: btY38AADN2yPlZumBh7MJw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3469A8DC661; Thu, 10 Aug 2023 10:09:12 +0000 (UTC) Received: from file1-rdu.file-001.prod.rdu2.dc.redhat.com (unknown [10.11.5.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2488DC15BAE; Thu, 10 Aug 2023 10:09:12 +0000 (UTC) Received: by file1-rdu.file-001.prod.rdu2.dc.redhat.com (Postfix, from userid 12668) id 17FD730B9C07; Thu, 10 Aug 2023 10:09:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by file1-rdu.file-001.prod.rdu2.dc.redhat.com (Postfix) with ESMTP id 14B933F7CF; Thu, 10 Aug 2023 12:09:12 +0200 (CEST) Date: Thu, 10 Aug 2023 12:09:11 +0200 (CEST) From: Mikulas Patocka To: Jens Axboe cc: Li Nan , Zdenek Kabelac , Christoph Hellwig , Chaitanya Kulkarni , linux-block@vger.kernel.org, dm-devel@redhat.com Subject: [PATCH v3 2/4] brd: extend the rcu regions to cover read and write In-Reply-To: <2dacc73-854-e71c-1746-99b017401c9a@redhat.com> Message-ID: <2ae7f7aa-add7-a7f5-e73e-443fc97a0d9@redhat.com> References: <2dacc73-854-e71c-1746-99b017401c9a@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This patch extends the rcu regions, so that lookup followed by a read or write of a page is done inside rcu read lock. This is needed for the following patch that enables discard. Signed-off-by: Mikulas Patocka --- drivers/block/brd.c | 8 ++++++++ 1 file changed, 8 insertions(+) Index: linux-2.6/drivers/block/brd.c =================================================================== --- linux-2.6.orig/drivers/block/brd.c +++ linux-2.6/drivers/block/brd.c @@ -150,23 +150,27 @@ static void copy_to_brd(struct brd_devic size_t copy; copy = min_t(size_t, n, PAGE_SIZE - offset); + rcu_read_lock(); page = brd_lookup_page(brd, sector); BUG_ON(!page); dst = kmap_atomic(page); memcpy(dst + offset, src, copy); kunmap_atomic(dst); + rcu_read_unlock(); if (copy < n) { src += copy; sector += copy >> SECTOR_SHIFT; copy = n - copy; + rcu_read_lock(); page = brd_lookup_page(brd, sector); BUG_ON(!page); dst = kmap_atomic(page); memcpy(dst, src, copy); kunmap_atomic(dst); + rcu_read_unlock(); } } @@ -182,6 +186,7 @@ static void copy_from_brd(void *dst, str size_t copy; copy = min_t(size_t, n, PAGE_SIZE - offset); + rcu_read_lock(); page = brd_lookup_page(brd, sector); if (page) { src = kmap_atomic(page); @@ -189,11 +194,13 @@ static void copy_from_brd(void *dst, str kunmap_atomic(src); } else memset(dst, 0, copy); + rcu_read_unlock(); if (copy < n) { dst += copy; sector += copy >> SECTOR_SHIFT; copy = n - copy; + rcu_read_lock(); page = brd_lookup_page(brd, sector); if (page) { src = kmap_atomic(page); @@ -201,6 +208,7 @@ static void copy_from_brd(void *dst, str kunmap_atomic(src); } else memset(dst, 0, copy); + rcu_read_unlock(); } } From patchwork Thu Aug 10 10:09:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikulas Patocka X-Patchwork-Id: 13349207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80100C001B0 for ; Thu, 10 Aug 2023 10:10:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229793AbjHJKKX (ORCPT ); Thu, 10 Aug 2023 06:10:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234271AbjHJKKX (ORCPT ); Thu, 10 Aug 2023 06:10:23 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D589ADF for ; Thu, 10 Aug 2023 03:09:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691662179; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PM01ZyE8/gNnllK6jnROSczVJzrXeN+XAqapj2EmtGQ=; b=HUGz1ng+A95WSfO0sfZLrJG8s2Kgq+waHv1x1Gi2HsRd+Io1GkbLsX3SsdvRHMQRspRyAe Qc17mCBlvv3WtHWPNr8jx+R31V9wrOZUIzOh9vEIOrvoWHvHRDfbqnMt8rcBAa9hs/20fQ okSDJgnK8xq/P/7OuiH/oe5Gmi6hjzY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-90-u5JDl1oEMEWMaFNBrPYM7Q-1; Thu, 10 Aug 2023 06:09:35 -0400 X-MC-Unique: u5JDl1oEMEWMaFNBrPYM7Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8ACB5185A78F; Thu, 10 Aug 2023 10:09:34 +0000 (UTC) Received: from file1-rdu.file-001.prod.rdu2.dc.redhat.com (unknown [10.11.5.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7D72E1121314; Thu, 10 Aug 2023 10:09:34 +0000 (UTC) Received: by file1-rdu.file-001.prod.rdu2.dc.redhat.com (Postfix, from userid 12668) id 3D5D330B9C07; Thu, 10 Aug 2023 10:09:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by file1-rdu.file-001.prod.rdu2.dc.redhat.com (Postfix) with ESMTP id 3C09B3F7CF; Thu, 10 Aug 2023 12:09:34 +0200 (CEST) Date: Thu, 10 Aug 2023 12:09:33 +0200 (CEST) From: Mikulas Patocka To: Jens Axboe cc: Li Nan , Zdenek Kabelac , Christoph Hellwig , Chaitanya Kulkarni , linux-block@vger.kernel.org, dm-devel@redhat.com Subject: [PATCH v3 3/4] brd: enable discard In-Reply-To: <2dacc73-854-e71c-1746-99b017401c9a@redhat.com> Message-ID: <10a61c-8c51-11f3-83b-dbffe688d68d@redhat.com> References: <2dacc73-854-e71c-1746-99b017401c9a@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This patch implements discard in the brd driver. We use RCU to free the page, so that if there are any concurrent readers or writes, they won't touch the page after it is freed. Calling "call_rcu" for each page is inefficient, so we attempt to batch multiple pages to a single "call_rcu" call. Note that we replace "BUG_ON(!page);" with "if (page) ..." in copy_to_brd - the page can't be NULL under normal circumstances, it can only be NULL if REQ_OP_WRITE races with REQ_OP_DISCARD. If these two bios race with each other on the same page, the result is undefined, so we can handle this race condition just by skipping the copying. Signed-off-by: Mikulas Patocka --- drivers/block/brd.c | 144 +++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 131 insertions(+), 13 deletions(-) Index: linux-2.6/drivers/block/brd.c =================================================================== --- linux-2.6.orig/drivers/block/brd.c +++ linux-2.6/drivers/block/brd.c @@ -46,6 +46,8 @@ struct brd_device { u64 brd_nr_pages; }; +static bool discard; + /* * Look up and return a brd's page for a given sector. */ @@ -100,6 +102,54 @@ static int brd_insert_page(struct brd_de return ret; } +struct free_page_batch { + struct rcu_head rcu; + struct list_head list; +}; + +static void brd_free_page_rcu(struct rcu_head *head) +{ + __free_page(container_of(head, struct page, rcu_head)); +} + +static void brd_free_pages_rcu(struct rcu_head *head) +{ + struct free_page_batch *batch = container_of(head, struct free_page_batch, rcu); + + while (!list_empty(&batch->list)) { + struct page *page = list_entry(batch->list.prev, struct page, lru); + + list_del(&page->lru); + + __free_page(page); + } + + kfree(batch); +} + +static void brd_free_page(struct brd_device *brd, sector_t sector, + struct free_page_batch **batch) +{ + struct page *page; + pgoff_t idx; + + idx = sector >> PAGE_SECTORS_SHIFT; + page = xa_erase(&brd->brd_pages, idx); + + if (page) { + BUG_ON(page->index != idx); + if (!*batch) { + *batch = kmalloc(sizeof(struct free_page_batch), GFP_NOIO); + if (unlikely(!*batch)) { + call_rcu(&page->rcu_head, brd_free_page_rcu); + return; + } + INIT_LIST_HEAD(&(*batch)->list); + } + list_add(&page->lru, &(*batch)->list); + } +} + /* * Free all backing store pages and xarray. This must only be called when * there are no other users of the device. @@ -152,11 +202,11 @@ static void copy_to_brd(struct brd_devic copy = min_t(size_t, n, PAGE_SIZE - offset); rcu_read_lock(); page = brd_lookup_page(brd, sector); - BUG_ON(!page); - - dst = kmap_atomic(page); - memcpy(dst + offset, src, copy); - kunmap_atomic(dst); + if (page) { + dst = kmap_atomic(page); + memcpy(dst + offset, src, copy); + kunmap_atomic(dst); + } rcu_read_unlock(); if (copy < n) { @@ -165,11 +215,11 @@ static void copy_to_brd(struct brd_devic copy = n - copy; rcu_read_lock(); page = brd_lookup_page(brd, sector); - BUG_ON(!page); - - dst = kmap_atomic(page); - memcpy(dst, src, copy); - kunmap_atomic(dst); + if (page) { + dst = kmap_atomic(page); + memcpy(dst, src, copy); + kunmap_atomic(dst); + } rcu_read_unlock(); } } @@ -248,6 +298,34 @@ out: return err; } +void brd_do_discard(struct brd_device *brd, struct bio *bio) +{ + struct free_page_batch *batch = NULL; + sector_t sector, len, front_pad; + + if (unlikely(!discard)) { + bio->bi_status = BLK_STS_NOTSUPP; + return; + } + + sector = bio->bi_iter.bi_sector; + len = bio_sectors(bio); + front_pad = -sector & (PAGE_SECTORS - 1); + sector += front_pad; + if (unlikely(len <= front_pad)) + return; + len -= front_pad; + len = round_down(len, PAGE_SECTORS); + while (len) { + brd_free_page(brd, sector, &batch); + sector += PAGE_SECTORS; + len -= PAGE_SECTORS; + cond_resched(); + } + if (batch) + call_rcu(&batch->rcu, brd_free_pages_rcu); +} + static void brd_submit_bio(struct bio *bio) { struct brd_device *brd = bio->bi_bdev->bd_disk->private_data; @@ -280,6 +358,9 @@ static void brd_submit_bio(struct bio *b sector += len >> SECTOR_SHIFT; } break; + case REQ_OP_DISCARD: + brd_do_discard(brd, bio); + break; default: bio->bi_status = BLK_STS_NOTSUPP; break; @@ -293,6 +374,40 @@ static const struct block_device_operati .submit_bio = brd_submit_bio, }; +static LIST_HEAD(brd_devices); +static struct dentry *brd_debugfs_dir; + +static void brd_set_discard_limits(struct brd_device *brd) +{ + struct request_queue *queue = brd->brd_disk->queue; + if (discard) { + queue->limits.discard_granularity = PAGE_SIZE; + blk_queue_max_discard_sectors(queue, round_down(UINT_MAX, PAGE_SECTORS)); + } else { + queue->limits.discard_granularity = 0; + blk_queue_max_discard_sectors(queue, 0); + } +} + +static int discard_set_bool(const char *val, const struct kernel_param *kp) +{ + struct brd_device *brd; + + int r = param_set_bool(val, kp); + if (r) + return r; + + list_for_each_entry(brd, &brd_devices, brd_list) + brd_set_discard_limits(brd); + + return 0; +} + +static const struct kernel_param_ops discard_ops = { + .set = discard_set_bool, + .get = param_get_bool, +}; + /* * And now the modules code and kernel interface. */ @@ -308,6 +423,10 @@ static int max_part = 1; module_param(max_part, int, 0444); MODULE_PARM_DESC(max_part, "Num Minors to reserve between devices"); +static bool discard = false; +module_param_cb(discard, &discard_ops, &discard, 0644); +MODULE_PARM_DESC(discard, "Support discard"); + MODULE_LICENSE("GPL"); MODULE_ALIAS_BLOCKDEV_MAJOR(RAMDISK_MAJOR); MODULE_ALIAS("rd"); @@ -326,9 +445,6 @@ __setup("ramdisk_size=", ramdisk_size); * The device scheme is derived from loop.c. Keep them in synch where possible * (should share code eventually). */ -static LIST_HEAD(brd_devices); -static struct dentry *brd_debugfs_dir; - static int brd_alloc(int i) { struct brd_device *brd; @@ -373,6 +489,8 @@ static int brd_alloc(int i) */ blk_queue_physical_block_size(disk->queue, PAGE_SIZE); + brd_set_discard_limits(brd); + /* Tell the block layer that this is not a rotational device */ blk_queue_flag_set(QUEUE_FLAG_NONROT, disk->queue); blk_queue_flag_set(QUEUE_FLAG_SYNCHRONOUS, disk->queue); From patchwork Thu Aug 10 10:10:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikulas Patocka X-Patchwork-Id: 13349208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B723C001B0 for ; Thu, 10 Aug 2023 10:11:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233024AbjHJKLC (ORCPT ); Thu, 10 Aug 2023 06:11:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232030AbjHJKLB (ORCPT ); Thu, 10 Aug 2023 06:11:01 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90ABE128 for ; Thu, 10 Aug 2023 03:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691662215; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kZSPj5/CzHhTNACKP6Hil6EI3Kv631Cd5foB+TFz2IM=; b=eLUviX68S0dwOj3XdxNy5j11TuFIwH1SNCn3LdJd11n57Igwh+75KFOMa6RwIC8/XaFIyz D1mijD5qG1gpt2x/M6tJu1jovnyYSAgPzyDOX9nv7FApVsEvde4AdeHN0AIEPqj4dcH5QE wnTfFZEncrCSLlYwpKPdEbgNtzo784Y= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-497-_w7CmM8GPw-HxTCVzK6t7A-1; Thu, 10 Aug 2023 06:10:12 -0400 X-MC-Unique: _w7CmM8GPw-HxTCVzK6t7A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B8CB91C31C50; Thu, 10 Aug 2023 10:10:11 +0000 (UTC) Received: from file1-rdu.file-001.prod.rdu2.dc.redhat.com (unknown [10.11.5.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A92C940C6F4E; Thu, 10 Aug 2023 10:10:11 +0000 (UTC) Received: by file1-rdu.file-001.prod.rdu2.dc.redhat.com (Postfix, from userid 12668) id F0E7930B9C07; Thu, 10 Aug 2023 10:10:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by file1-rdu.file-001.prod.rdu2.dc.redhat.com (Postfix) with ESMTP id AD8B53F7CF; Thu, 10 Aug 2023 12:10:10 +0200 (CEST) Date: Thu, 10 Aug 2023 12:10:09 +0200 (CEST) From: Mikulas Patocka To: Jens Axboe cc: Li Nan , Zdenek Kabelac , Christoph Hellwig , Chaitanya Kulkarni , linux-block@vger.kernel.org, dm-devel@redhat.com Subject: [PATCH v3 4/4] brd: implement write zeroes In-Reply-To: <2dacc73-854-e71c-1746-99b017401c9a@redhat.com> Message-ID: <19a1a124-cc21-9123-1ba1-2d7cadb9cdbe@redhat.com> References: <2dacc73-854-e71c-1746-99b017401c9a@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This patch implements REQ_OP_WRITE_ZEROES on brd. Write zeroes will free the pages just like discard, but the difference is that it writes zeroes to the preceding and following page if the range is not aligned on page boundary. Signed-off-by: Mikulas Patocka --- drivers/block/brd.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) Index: linux-2.6/drivers/block/brd.c =================================================================== --- linux-2.6.orig/drivers/block/brd.c +++ linux-2.6/drivers/block/brd.c @@ -301,7 +301,8 @@ out: void brd_do_discard(struct brd_device *brd, struct bio *bio) { struct free_page_batch *batch = NULL; - sector_t sector, len, front_pad; + bool zero_padding = bio_op(bio) == REQ_OP_WRITE_ZEROES; + sector_t sector, len, front_pad, end_pad; if (unlikely(!discard)) { bio->bi_status = BLK_STS_NOTSUPP; @@ -311,11 +312,22 @@ void brd_do_discard(struct brd_device *b sector = bio->bi_iter.bi_sector; len = bio_sectors(bio); front_pad = -sector & (PAGE_SECTORS - 1); + + if (zero_padding && unlikely(front_pad != 0)) + copy_to_brd(brd, page_address(ZERO_PAGE(0)), + sector, min(len, front_pad) << SECTOR_SHIFT); + sector += front_pad; if (unlikely(len <= front_pad)) return; len -= front_pad; - len = round_down(len, PAGE_SECTORS); + + end_pad = len & (PAGE_SECTORS - 1); + if (zero_padding && unlikely(end_pad != 0)) + copy_to_brd(brd, page_address(ZERO_PAGE(0)), + sector + len - end_pad, end_pad << SECTOR_SHIFT); + len -= end_pad; + while (len) { brd_free_page(brd, sector, &batch); sector += PAGE_SECTORS; @@ -359,6 +371,7 @@ static void brd_submit_bio(struct bio *b } break; case REQ_OP_DISCARD: + case REQ_OP_WRITE_ZEROES: brd_do_discard(brd, bio); break; default: @@ -383,9 +396,11 @@ static void brd_set_discard_limits(struc if (discard) { queue->limits.discard_granularity = PAGE_SIZE; blk_queue_max_discard_sectors(queue, round_down(UINT_MAX, PAGE_SECTORS)); + blk_queue_max_write_zeroes_sectors(queue, round_down(UINT_MAX, PAGE_SECTORS)); } else { queue->limits.discard_granularity = 0; blk_queue_max_discard_sectors(queue, 0); + blk_queue_max_write_zeroes_sectors(queue, 0); } } @@ -425,7 +440,7 @@ MODULE_PARM_DESC(max_part, "Num Minors t static bool discard = false; module_param_cb(discard, &discard_ops, &discard, 0644); -MODULE_PARM_DESC(discard, "Support discard"); +MODULE_PARM_DESC(discard, "Support discard and write zeroes"); MODULE_LICENSE("GPL"); MODULE_ALIAS_BLOCKDEV_MAJOR(RAMDISK_MAJOR);