From patchwork Tue Feb 18 21:48:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11389763 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 51CFC109A for ; Tue, 18 Feb 2020 21:49:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 31886206E2 for ; Tue, 18 Feb 2020 21:49:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BhnJmMzp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728010AbgBRVtH (ORCPT ); Tue, 18 Feb 2020 16:49:07 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:29758 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727996AbgBRVtD (ORCPT ); Tue, 18 Feb 2020 16:49:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582062542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MPuW8FJYXb4EyYv2Zcbckz83nVUeVWfBw8sxF/N5fjs=; b=BhnJmMzpRfRha//wsvmMtq3Cy2+0tC0Hycmec9YbRtaa8FcG8oj/b/GDM1UDlaeG11bCn9 XLn+cdtX8emLz+8sj4YQyZ/YX9jtojHZxYhvbYKXkMeHmIxjwdpcsANIWohYr9nxXVnS2I k1Pm5XCrtAh9JJt/Ca5blsvCfS3cPws= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-423-t2mG82F3PxyQsJxHcH8wnQ-1; Tue, 18 Feb 2020 16:48:57 -0500 X-MC-Unique: t2mG82F3PxyQsJxHcH8wnQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EF155802567; Tue, 18 Feb 2020 21:48:55 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 10EBE5C1B0; Tue, 18 Feb 2020 21:48:53 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 8E9362257D3; Tue, 18 Feb 2020 16:48:52 -0500 (EST) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, linux-nvdimm@lists.01.org, hch@infradead.org, dan.j.williams@intel.com Cc: dm-devel@redhat.com, vishal.l.verma@intel.com, vgoyal@redhat.com, Christoph Hellwig Subject: [PATCH v5 1/8] pmem: Add functions for reading/writing page to/from pmem Date: Tue, 18 Feb 2020 16:48:34 -0500 Message-Id: <20200218214841.10076-2-vgoyal@redhat.com> In-Reply-To: <20200218214841.10076-1-vgoyal@redhat.com> References: <20200218214841.10076-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This splits pmem_do_bvec() into pmem_do_read() and pmem_do_write(). pmem_do_write() will be used by pmem zero_page_range() as well. Hence sharing the same code. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Vivek Goyal --- drivers/nvdimm/pmem.c | 86 +++++++++++++++++++++++++------------------ 1 file changed, 50 insertions(+), 36 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 4eae441f86c9..075b11682192 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -136,9 +136,25 @@ static blk_status_t read_pmem(struct page *page, unsigned int off, return BLK_STS_OK; } -static blk_status_t pmem_do_bvec(struct pmem_device *pmem, struct page *page, - unsigned int len, unsigned int off, unsigned int op, - sector_t sector) +static blk_status_t pmem_do_read(struct pmem_device *pmem, + struct page *page, unsigned int page_off, + sector_t sector, unsigned int len) +{ + blk_status_t rc; + phys_addr_t pmem_off = sector * 512 + pmem->data_offset; + void *pmem_addr = pmem->virt_addr + pmem_off; + + if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) + return BLK_STS_IOERR; + + rc = read_pmem(page, page_off, pmem_addr, len); + flush_dcache_page(page); + return rc; +} + +static blk_status_t pmem_do_write(struct pmem_device *pmem, + struct page *page, unsigned int page_off, + sector_t sector, unsigned int len) { blk_status_t rc = BLK_STS_OK; bool bad_pmem = false; @@ -148,34 +164,25 @@ static blk_status_t pmem_do_bvec(struct pmem_device *pmem, struct page *page, if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) bad_pmem = true; - if (!op_is_write(op)) { - if (unlikely(bad_pmem)) - rc = BLK_STS_IOERR; - else { - rc = read_pmem(page, off, pmem_addr, len); - flush_dcache_page(page); - } - } else { - /* - * Note that we write the data both before and after - * clearing poison. The write before clear poison - * handles situations where the latest written data is - * preserved and the clear poison operation simply marks - * the address range as valid without changing the data. - * In this case application software can assume that an - * interrupted write will either return the new good - * data or an error. - * - * However, if pmem_clear_poison() leaves the data in an - * indeterminate state we need to perform the write - * after clear poison. - */ - flush_dcache_page(page); - write_pmem(pmem_addr, page, off, len); - if (unlikely(bad_pmem)) { - rc = pmem_clear_poison(pmem, pmem_off, len); - write_pmem(pmem_addr, page, off, len); - } + /* + * Note that we write the data both before and after + * clearing poison. The write before clear poison + * handles situations where the latest written data is + * preserved and the clear poison operation simply marks + * the address range as valid without changing the data. + * In this case application software can assume that an + * interrupted write will either return the new good + * data or an error. + * + * However, if pmem_clear_poison() leaves the data in an + * indeterminate state we need to perform the write + * after clear poison. + */ + flush_dcache_page(page); + write_pmem(pmem_addr, page, page_off, len); + if (unlikely(bad_pmem)) { + rc = pmem_clear_poison(pmem, pmem_off, len); + write_pmem(pmem_addr, page, page_off, len); } return rc; @@ -197,8 +204,12 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio) do_acct = nd_iostat_start(bio, &start); bio_for_each_segment(bvec, bio, iter) { - rc = pmem_do_bvec(pmem, bvec.bv_page, bvec.bv_len, - bvec.bv_offset, bio_op(bio), iter.bi_sector); + if (op_is_write(bio_op(bio))) + rc = pmem_do_write(pmem, bvec.bv_page, bvec.bv_offset, + iter.bi_sector, bvec.bv_len); + else + rc = pmem_do_read(pmem, bvec.bv_page, bvec.bv_offset, + iter.bi_sector, bvec.bv_len); if (rc) { bio->bi_status = rc; break; @@ -223,9 +234,12 @@ static int pmem_rw_page(struct block_device *bdev, sector_t sector, struct pmem_device *pmem = bdev->bd_queue->queuedata; blk_status_t rc; - rc = pmem_do_bvec(pmem, page, hpage_nr_pages(page) * PAGE_SIZE, - 0, op, sector); - + if (op_is_write(op)) + rc = pmem_do_write(pmem, page, 0, sector, + hpage_nr_pages(page) * PAGE_SIZE); + else + rc = pmem_do_read(pmem, page, 0, sector, + hpage_nr_pages(page) * PAGE_SIZE); /* * The ->rw_page interface is subtle and tricky. The core * retries on any error, so we can only invoke page_endio() in From patchwork Tue Feb 18 21:48:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11389757 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6285217F0 for ; Tue, 18 Feb 2020 21:49:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 43E1622B48 for ; Tue, 18 Feb 2020 21:49:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gko633RR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728000AbgBRVtE (ORCPT ); Tue, 18 Feb 2020 16:49:04 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:32845 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727976AbgBRVtC (ORCPT ); Tue, 18 Feb 2020 16:49:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582062540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tyVUZZb1YzuBSN0HxippMw48mL8qf75NwDgXuTB/Cgc=; b=gko633RRKPlGOoTApvTCJ209XszxISVfojJ/8xdwqrM6/JYA3xvxEwL9okhp95RJVR/kbA n4wr+Xba40LzZwDk1qX0VdUpyse1hiK3tGUE7A9K8rekxieztN7ieTHKii5kuj9LotFyPs DaXSmnF5HtB7WWa0JR07hmszPd5k+oc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-103-y2FBWsfmMyi3cnQC9K4Img-1; Tue, 18 Feb 2020 16:48:57 -0500 X-MC-Unique: y2FBWsfmMyi3cnQC9K4Img-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E7CD9802566; Tue, 18 Feb 2020 21:48:55 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 007E75D9E5; Tue, 18 Feb 2020 21:48:52 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 9429C2257D4; Tue, 18 Feb 2020 16:48:52 -0500 (EST) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, linux-nvdimm@lists.01.org, hch@infradead.org, dan.j.williams@intel.com Cc: dm-devel@redhat.com, vishal.l.verma@intel.com, vgoyal@redhat.com Subject: [PATCH v5 2/8] drivers/pmem: Allow pmem_clear_poison() to accept arbitrary offset and len Date: Tue, 18 Feb 2020 16:48:35 -0500 Message-Id: <20200218214841.10076-3-vgoyal@redhat.com> In-Reply-To: <20200218214841.10076-1-vgoyal@redhat.com> References: <20200218214841.10076-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Currently pmem_clear_poison() expects offset and len to be sector aligned. Atleast that seems to be the assumption with which code has been written. It is called only from pmem_do_bvec() which is called only from pmem_rw_page() and pmem_make_request() which will only passe sector aligned offset and len. Soon we want use this function from dax_zero_page_range() code path which can try to zero arbitrary range of memory with-in a page. So update this function to assume that offset and length can be arbitrary and do the necessary alignments as needed. nvdimm_clear_poison() seems to assume offset and len to be aligned to clear_err_unit boundary. But this is currently internal detail and is not exported for others to use. So for now, continue to align offset and length to SECTOR_SIZE boundary. Improving it further and to align it to clear_err_unit boundary is a TODO item for future. Signed-off-by: Vivek Goyal Signed-off-by: Vivek Goyal --- drivers/nvdimm/pmem.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 075b11682192..e72959203253 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -74,14 +74,28 @@ static blk_status_t pmem_clear_poison(struct pmem_device *pmem, sector_t sector; long cleared; blk_status_t rc = BLK_STS_OK; + phys_addr_t start_aligned, end_aligned; + unsigned int len_aligned; - sector = (offset - pmem->data_offset) / 512; + /* + * Callers can pass arbitrary offset and len. But nvdimm_clear_poison() + * expects memory offset and length to meet certain alignment + * restrction (clear_err_unit). Currently nvdimm does not export + * required alignment. So align offset and length to sector boundary + * before passing it to nvdimm_clear_poison(). + */ + start_aligned = ALIGN(offset, SECTOR_SIZE); + end_aligned = ALIGN_DOWN((offset + len), SECTOR_SIZE) - 1; + len_aligned = end_aligned - start_aligned + 1; + + sector = (start_aligned - pmem->data_offset) / 512; - cleared = nvdimm_clear_poison(dev, pmem->phys_addr + offset, len); - if (cleared < len) + cleared = nvdimm_clear_poison(dev, pmem->phys_addr + start_aligned, + len_aligned); + if (cleared < len_aligned) rc = BLK_STS_IOERR; if (cleared > 0 && cleared / 512) { - hwpoison_clear(pmem, pmem->phys_addr + offset, cleared); + hwpoison_clear(pmem, pmem->phys_addr + start_aligned, cleared); cleared /= 512; dev_dbg(dev, "%#llx clear %ld sector%s\n", (unsigned long long) sector, cleared, From patchwork Tue Feb 18 21:48:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11389751 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F9CA17F0 for ; Tue, 18 Feb 2020 21:49:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 615FD24654 for ; Tue, 18 Feb 2020 21:49:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RGjskMZb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727992AbgBRVtC (ORCPT ); Tue, 18 Feb 2020 16:49:02 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:30198 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726481AbgBRVtC (ORCPT ); Tue, 18 Feb 2020 16:49:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582062540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/xheq0Kv3ALRSOX+hpzwGoRJOYo9Awxk5MHohlOXv/Y=; b=RGjskMZbHnJvFDaxcAv8cu33rcf+Bkdd0OL0siWj1FZRaSWXjJKraCFjmq9zVGD8WgjB1q Mw9xs3gbVfy5lYrnUvnaIv84qZAJj5y6771c5gvZGKrZ8aT1y7etTTS4nokMcT9WhBDcKf Ai/7z+OURcH/VM3Og9v3AC32N3rmv4g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-394-SWHJk1X9MHKf6zVamgtOJA-1; Tue, 18 Feb 2020 16:48:57 -0500 X-MC-Unique: SWHJk1X9MHKf6zVamgtOJA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CA3CA190B2AA; Tue, 18 Feb 2020 21:48:55 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2B8891001B05; Tue, 18 Feb 2020 21:48:53 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 9BF7D2257D5; Tue, 18 Feb 2020 16:48:52 -0500 (EST) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, linux-nvdimm@lists.01.org, hch@infradead.org, dan.j.williams@intel.com Cc: dm-devel@redhat.com, vishal.l.verma@intel.com, vgoyal@redhat.com Subject: [PATCH v5 3/8] pmem: Enable pmem_do_write() to deal with arbitrary ranges Date: Tue, 18 Feb 2020 16:48:36 -0500 Message-Id: <20200218214841.10076-4-vgoyal@redhat.com> In-Reply-To: <20200218214841.10076-1-vgoyal@redhat.com> References: <20200218214841.10076-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Currently pmem_do_write() is written with assumption that all I/O is sector aligned. Soon I want to use this function in zero_page_range() where range passed in does not have to be sector aligned. Modify this function to be able to deal with an arbitrary range. Which is specified by pmem_off and len. Signed-off-by: Vivek Goyal Reviewed-by: Christoph Hellwig --- drivers/nvdimm/pmem.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index e72959203253..3c46e9e6d04c 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -168,15 +168,23 @@ static blk_status_t pmem_do_read(struct pmem_device *pmem, static blk_status_t pmem_do_write(struct pmem_device *pmem, struct page *page, unsigned int page_off, - sector_t sector, unsigned int len) + u64 pmem_off, unsigned int len) { blk_status_t rc = BLK_STS_OK; bool bad_pmem = false; - phys_addr_t pmem_off = sector * 512 + pmem->data_offset; - void *pmem_addr = pmem->virt_addr + pmem_off; - - if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) - bad_pmem = true; + phys_addr_t pmem_real_off = pmem_off + pmem->data_offset; + void *pmem_addr = pmem->virt_addr + pmem_real_off; + sector_t sector_start, sector_end; + unsigned nr_sectors; + + sector_start = DIV_ROUND_UP(pmem_off, SECTOR_SIZE); + sector_end = (pmem_off + len) >> SECTOR_SHIFT; + if (sector_end > sector_start) { + nr_sectors = sector_end - sector_start; + if (is_bad_pmem(&pmem->bb, sector_start, + nr_sectors << SECTOR_SHIFT)) + bad_pmem = true; + } /* * Note that we write the data both before and after @@ -195,7 +203,7 @@ static blk_status_t pmem_do_write(struct pmem_device *pmem, flush_dcache_page(page); write_pmem(pmem_addr, page, page_off, len); if (unlikely(bad_pmem)) { - rc = pmem_clear_poison(pmem, pmem_off, len); + rc = pmem_clear_poison(pmem, pmem_real_off, len); write_pmem(pmem_addr, page, page_off, len); } @@ -220,7 +228,7 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio) bio_for_each_segment(bvec, bio, iter) { if (op_is_write(bio_op(bio))) rc = pmem_do_write(pmem, bvec.bv_page, bvec.bv_offset, - iter.bi_sector, bvec.bv_len); + iter.bi_sector << SECTOR_SHIFT, bvec.bv_len); else rc = pmem_do_read(pmem, bvec.bv_page, bvec.bv_offset, iter.bi_sector, bvec.bv_len); @@ -249,7 +257,7 @@ static int pmem_rw_page(struct block_device *bdev, sector_t sector, blk_status_t rc; if (op_is_write(op)) - rc = pmem_do_write(pmem, page, 0, sector, + rc = pmem_do_write(pmem, page, 0, sector << SECTOR_SHIFT, hpage_nr_pages(page) * PAGE_SIZE); else rc = pmem_do_read(pmem, page, 0, sector, From patchwork Tue Feb 18 21:48:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11389749 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB2EB109A for ; Tue, 18 Feb 2020 21:49:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C94E722B48 for ; Tue, 18 Feb 2020 21:49:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ESiefP7T" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727987AbgBRVtC (ORCPT ); Tue, 18 Feb 2020 16:49:02 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:44676 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727978AbgBRVtC (ORCPT ); Tue, 18 Feb 2020 16:49:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582062541; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dWuQevadGbeWpwHhQNwK1IjmkMEDQ0KYV5P1Kw6/vBY=; b=ESiefP7TuRy3fmTPP3dpCI5Y/E7iMabFJxY7/ADldKYVaWr0CE6JAMg9bswBAu6YgX/9T/ OqTX3fbNb+xGiK7eUEhA7xNDs2+CcxU8IFbIctuJLHvZVlNxkrGLE/HRppZ4YqKrIB2BIc OqxVgau7ItrXyksCqksedKoboZ3iwi0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-177-pkhcM5VjO76REe6mdKaKbg-1; Tue, 18 Feb 2020 16:48:57 -0500 X-MC-Unique: pkhcM5VjO76REe6mdKaKbg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CBA6710CE783; Tue, 18 Feb 2020 21:48:55 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2846460BE1; Tue, 18 Feb 2020 21:48:53 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id A7FE52257D6; Tue, 18 Feb 2020 16:48:52 -0500 (EST) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, linux-nvdimm@lists.01.org, hch@infradead.org, dan.j.williams@intel.com Cc: dm-devel@redhat.com, vishal.l.verma@intel.com, vgoyal@redhat.com, Christoph Hellwig Subject: [PATCH v5 4/8] dax, pmem: Add a dax operation zero_page_range Date: Tue, 18 Feb 2020 16:48:37 -0500 Message-Id: <20200218214841.10076-5-vgoyal@redhat.com> In-Reply-To: <20200218214841.10076-1-vgoyal@redhat.com> References: <20200218214841.10076-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a dax operation zero_page_range, to zero a range of memory. This will also clear any poison in the range being zeroed. As of now, zeroing of up to one page is allowed in a single call. There are no callers which are trying to zero more than a page in a single call. Once we grow the callers which zero more than a page in single call, we can add that support. Primary reason for not doing that yet is that this will add little complexity in dm implementation where a range might be spanning multiple underlying targets and one will have to split the range into multiple sub ranges and call zero_page_range() on individual targets. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Vivek Goyal --- drivers/dax/super.c | 19 +++++++++++++++++++ drivers/nvdimm/pmem.c | 10 ++++++++++ include/linux/dax.h | 3 +++ 3 files changed, 32 insertions(+) diff --git a/drivers/dax/super.c b/drivers/dax/super.c index 0aa4b6bc5101..c912808bc886 100644 --- a/drivers/dax/super.c +++ b/drivers/dax/super.c @@ -344,6 +344,25 @@ size_t dax_copy_to_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, } EXPORT_SYMBOL_GPL(dax_copy_to_iter); +int dax_zero_page_range(struct dax_device *dax_dev, u64 offset, size_t len) +{ + if (!dax_alive(dax_dev)) + return -ENXIO; + + if (!dax_dev->ops->zero_page_range) + return -EOPNOTSUPP; + /* + * There are no callers that want to zero across a page boundary as of + * now. Once users are there, this check can be removed after the + * device mapper code has been updated to split ranges across targets. + */ + if (offset_in_page(offset) + len > PAGE_SIZE) + return -EIO; + + return dax_dev->ops->zero_page_range(dax_dev, offset, len); +} +EXPORT_SYMBOL_GPL(dax_zero_page_range); + #ifdef CONFIG_ARCH_HAS_PMEM_API void arch_wb_cache_pmem(void *addr, size_t size); void dax_flush(struct dax_device *dax_dev, void *addr, size_t size) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 3c46e9e6d04c..e17f9f56d6fe 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -304,6 +304,15 @@ static const struct block_device_operations pmem_fops = { .revalidate_disk = nvdimm_revalidate_disk, }; +static int pmem_dax_zero_page_range(struct dax_device *dax_dev, u64 offset, + size_t len) +{ + struct pmem_device *pmem = dax_get_private(dax_dev); + + return blk_status_to_errno(pmem_do_write(pmem, ZERO_PAGE(0), 0, offset, + len)); +} + static long pmem_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, void **kaddr, pfn_t *pfn) { @@ -335,6 +344,7 @@ static const struct dax_operations pmem_dax_ops = { .dax_supported = generic_fsdax_supported, .copy_from_iter = pmem_copy_from_iter, .copy_to_iter = pmem_copy_to_iter, + .zero_page_range = pmem_dax_zero_page_range, }; static const struct attribute_group *pmem_attribute_groups[] = { diff --git a/include/linux/dax.h b/include/linux/dax.h index 328c2dbb4409..93a663c26d6a 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -34,6 +34,8 @@ struct dax_operations { /* copy_to_iter: required operation for fs-dax direct-i/o */ size_t (*copy_to_iter)(struct dax_device *, pgoff_t, void *, size_t, struct iov_iter *); + /* zero_page_range: required operation. Zero range with-in a page */ + int (*zero_page_range)(struct dax_device *, u64, size_t); }; extern struct attribute_group dax_attribute_group; @@ -199,6 +201,7 @@ size_t dax_copy_from_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, size_t bytes, struct iov_iter *i); size_t dax_copy_to_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, size_t bytes, struct iov_iter *i); +int dax_zero_page_range(struct dax_device *dax_dev, u64 offset, size_t len); void dax_flush(struct dax_device *dax_dev, void *addr, size_t size); ssize_t dax_iomap_rw(struct kiocb *iocb, struct iov_iter *iter, From patchwork Tue Feb 18 21:48:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11389753 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E02292A for ; Tue, 18 Feb 2020 21:49:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E47FE24654 for ; Tue, 18 Feb 2020 21:49:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bwAmmk/t" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727993AbgBRVtD (ORCPT ); Tue, 18 Feb 2020 16:49:03 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:37835 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727979AbgBRVtC (ORCPT ); Tue, 18 Feb 2020 16:49:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582062541; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cnevXfGVEDsFBctQatqNzlbLF8HNLGY3ShXMHssDWiQ=; b=bwAmmk/tRY0QIF0C3xm6HscuO56L73fIUB2b0EsdS0AsMjWfVeT8zo1ZactK4sU5HjRdwQ ADhxNpmCEE6oBU8+oSZ5cBY8hElATqT5FUYUkfhlLirnaUUDhHydiGtDYUwuMUku6LF4ar 5jPIYHOFaVNdjWRFmgmVJSiPWd7jgKM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-405-_qRODWZyNFm7pCqRV7mn4w-1; Tue, 18 Feb 2020 16:48:57 -0500 X-MC-Unique: _qRODWZyNFm7pCqRV7mn4w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6AE20190B2AB; Tue, 18 Feb 2020 21:48:56 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4557D90089; Tue, 18 Feb 2020 21:48:56 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id AE6712257D7; Tue, 18 Feb 2020 16:48:52 -0500 (EST) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, linux-nvdimm@lists.01.org, hch@infradead.org, dan.j.williams@intel.com Cc: dm-devel@redhat.com, vishal.l.verma@intel.com, vgoyal@redhat.com, linux-s390@vger.kernel.org, Gerald Schaefer Subject: [PATCH v5 5/8] s390,dcssblk,dax: Add dax zero_page_range operation to dcssblk driver Date: Tue, 18 Feb 2020 16:48:38 -0500 Message-Id: <20200218214841.10076-6-vgoyal@redhat.com> In-Reply-To: <20200218214841.10076-1-vgoyal@redhat.com> References: <20200218214841.10076-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add dax operation zero_page_range for dcssblk driver. CC: linux-s390@vger.kernel.org Suggested-by: Christoph Hellwig Reviewed-by: Gerald Schaefer Signed-off-by: Vivek Goyal --- drivers/s390/block/dcssblk.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/drivers/s390/block/dcssblk.c b/drivers/s390/block/dcssblk.c index 63502ca537eb..331abab5d066 100644 --- a/drivers/s390/block/dcssblk.c +++ b/drivers/s390/block/dcssblk.c @@ -57,11 +57,28 @@ static size_t dcssblk_dax_copy_to_iter(struct dax_device *dax_dev, return copy_to_iter(addr, bytes, i); } +static int dcssblk_dax_zero_page_range(struct dax_device *dax_dev, u64 offset, + size_t len) +{ + long rc; + void *kaddr; + pgoff_t pgoff = offset >> PAGE_SHIFT; + unsigned page_offset = offset_in_page(offset); + + rc = dax_direct_access(dax_dev, pgoff, 1, &kaddr, NULL); + if (rc < 0) + return rc; + memset(kaddr + page_offset, 0, len); + dax_flush(dax_dev, kaddr + page_offset, len); + return 0; +} + static const struct dax_operations dcssblk_dax_ops = { .direct_access = dcssblk_dax_direct_access, .dax_supported = generic_fsdax_supported, .copy_from_iter = dcssblk_dax_copy_from_iter, .copy_to_iter = dcssblk_dax_copy_to_iter, + .zero_page_range = dcssblk_dax_zero_page_range, }; struct dcssblk_dev_info { From patchwork Tue Feb 18 21:48:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11389761 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFDE492A for ; Tue, 18 Feb 2020 21:49:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A80B122B48 for ; Tue, 18 Feb 2020 21:49:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Zn0kMNW0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728003AbgBRVtH (ORCPT ); Tue, 18 Feb 2020 16:49:07 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:21292 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727999AbgBRVtF (ORCPT ); Tue, 18 Feb 2020 16:49:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582062543; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ua5wH/Fz9lZbyzCxi5AMLPX3L0G/B+dweayL4T+G0Qg=; b=Zn0kMNW0e0Zl6sM+hA74zWnm7a0kqEtwi8IWfIhUe99I5VWrjvx6oMLrqq0bGOR2hv8hi+ iubX+VneOoTCCemIvYDZmQggE0eCItVAJj9XuHefo9buZ4Tbo5/3nQfCYOha7uwGC2dgL7 4bRWKnpOFRaZknIMVSGGHl0sPrL9SXA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-324-5diS3l_UO8ucu2yTve2aBg-1; Tue, 18 Feb 2020 16:49:00 -0500 X-MC-Unique: 5diS3l_UO8ucu2yTve2aBg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 11B06800D5A; Tue, 18 Feb 2020 21:48:59 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58F9D90F5F; Tue, 18 Feb 2020 21:48:56 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id B31B12257D8; Tue, 18 Feb 2020 16:48:52 -0500 (EST) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, linux-nvdimm@lists.01.org, hch@infradead.org, dan.j.williams@intel.com Cc: dm-devel@redhat.com, vishal.l.verma@intel.com, vgoyal@redhat.com Subject: [PATCH v5 6/8] dm,dax: Add dax zero_page_range operation Date: Tue, 18 Feb 2020 16:48:39 -0500 Message-Id: <20200218214841.10076-7-vgoyal@redhat.com> In-Reply-To: <20200218214841.10076-1-vgoyal@redhat.com> References: <20200218214841.10076-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patch adds support for dax zero_page_range operation to dm targets. Signed-off-by: Vivek Goyal --- drivers/md/dm-linear.c | 21 +++++++++++++++++++++ drivers/md/dm-log-writes.c | 19 +++++++++++++++++++ drivers/md/dm-stripe.c | 26 ++++++++++++++++++++++++++ drivers/md/dm.c | 31 +++++++++++++++++++++++++++++++ include/linux/device-mapper.h | 3 +++ 5 files changed, 100 insertions(+) diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index 8d07fdf63a47..03f99e6ad372 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -201,10 +201,30 @@ static size_t linear_dax_copy_to_iter(struct dm_target *ti, pgoff_t pgoff, return dax_copy_to_iter(dax_dev, pgoff, addr, bytes, i); } +static int linear_dax_zero_page_range(struct dm_target *ti, u64 offset, + size_t len) +{ + int ret; + struct linear_c *lc = ti->private; + struct block_device *bdev = lc->dev->bdev; + struct dax_device *dax_dev = lc->dev->dax_dev; + pgoff_t pgoff = offset >> PAGE_SHIFT; + unsigned page_offset = offset_in_page(offset); + sector_t dev_sector, sector = pgoff * PAGE_SECTORS; + + dev_sector = linear_map_sector(ti, sector); + ret = bdev_dax_pgoff(bdev, dev_sector, ALIGN(len, PAGE_SIZE), &pgoff); + if (ret) + return ret; + return dax_zero_page_range(dax_dev, (pgoff << PAGE_SHIFT) + page_offset, + len); +} + #else #define linear_dax_direct_access NULL #define linear_dax_copy_from_iter NULL #define linear_dax_copy_to_iter NULL +#define linear_dax_zero_page_range NULL #endif static struct target_type linear_target = { @@ -226,6 +246,7 @@ static struct target_type linear_target = { .direct_access = linear_dax_direct_access, .dax_copy_from_iter = linear_dax_copy_from_iter, .dax_copy_to_iter = linear_dax_copy_to_iter, + .dax_zero_page_range = linear_dax_zero_page_range, }; int __init dm_linear_init(void) diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c index 99721c76225d..f36ee223cb60 100644 --- a/drivers/md/dm-log-writes.c +++ b/drivers/md/dm-log-writes.c @@ -994,10 +994,28 @@ static size_t log_writes_dax_copy_to_iter(struct dm_target *ti, return dax_copy_to_iter(lc->dev->dax_dev, pgoff, addr, bytes, i); } +static int log_writes_dax_zero_page_range(struct dm_target *ti, u64 offset, + size_t len) +{ + int ret; + struct log_writes_c *lc = ti->private; + pgoff_t pgoff = offset >> PAGE_SHIFT; + unsigned page_offset = offset_in_page(offset); + sector_t sector = pgoff * PAGE_SECTORS; + + ret = bdev_dax_pgoff(lc->dev->bdev, sector, ALIGN(len, PAGE_SIZE), + &pgoff); + if (ret) + return ret; + return dax_zero_page_range(lc->dev->dax_dev, + (pgoff << PAGE_SHIFT) + page_offset, len); +} + #else #define log_writes_dax_direct_access NULL #define log_writes_dax_copy_from_iter NULL #define log_writes_dax_copy_to_iter NULL +#define log_writes_dax_zero_page_range NULL #endif static struct target_type log_writes_target = { @@ -1016,6 +1034,7 @@ static struct target_type log_writes_target = { .direct_access = log_writes_dax_direct_access, .dax_copy_from_iter = log_writes_dax_copy_from_iter, .dax_copy_to_iter = log_writes_dax_copy_to_iter, + .dax_zero_page_range = log_writes_dax_zero_page_range, }; static int __init dm_log_writes_init(void) diff --git a/drivers/md/dm-stripe.c b/drivers/md/dm-stripe.c index 63bbcc20f49a..f5e17284c615 100644 --- a/drivers/md/dm-stripe.c +++ b/drivers/md/dm-stripe.c @@ -360,10 +360,35 @@ static size_t stripe_dax_copy_to_iter(struct dm_target *ti, pgoff_t pgoff, return dax_copy_to_iter(dax_dev, pgoff, addr, bytes, i); } +static int stripe_dax_zero_page_range(struct dm_target *ti, u64 offset, + size_t len) +{ + int ret; + pgoff_t pgoff = offset >> PAGE_SHIFT; + unsigned page_offset = offset_in_page(offset); + sector_t dev_sector, sector = pgoff * PAGE_SECTORS; + struct stripe_c *sc = ti->private; + struct dax_device *dax_dev; + struct block_device *bdev; + uint32_t stripe; + + stripe_map_sector(sc, sector, &stripe, &dev_sector); + dev_sector += sc->stripe[stripe].physical_start; + dax_dev = sc->stripe[stripe].dev->dax_dev; + bdev = sc->stripe[stripe].dev->bdev; + + ret = bdev_dax_pgoff(bdev, dev_sector, ALIGN(len, PAGE_SIZE), &pgoff); + if (ret) + return ret; + return dax_zero_page_range(dax_dev, (pgoff << PAGE_SHIFT) + page_offset, + len); +} + #else #define stripe_dax_direct_access NULL #define stripe_dax_copy_from_iter NULL #define stripe_dax_copy_to_iter NULL +#define stripe_dax_zero_page_range NULL #endif /* @@ -486,6 +511,7 @@ static struct target_type stripe_target = { .direct_access = stripe_dax_direct_access, .dax_copy_from_iter = stripe_dax_copy_from_iter, .dax_copy_to_iter = stripe_dax_copy_to_iter, + .dax_zero_page_range = stripe_dax_zero_page_range, }; int __init dm_stripe_init(void) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index b89f07ee2eff..c87cabdf7f18 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1198,6 +1198,36 @@ static size_t dm_dax_copy_to_iter(struct dax_device *dax_dev, pgoff_t pgoff, return ret; } +static int dm_dax_zero_page_range(struct dax_device *dax_dev, u64 offset, + size_t len) +{ + struct mapped_device *md = dax_get_private(dax_dev); + pgoff_t pgoff = offset >> PAGE_SHIFT; + sector_t sector = pgoff * PAGE_SECTORS; + struct dm_target *ti; + int ret = -EIO; + int srcu_idx; + + ti = dm_dax_get_live_target(md, sector, &srcu_idx); + + if (!ti) + goto out; + if (WARN_ON(!ti->type->dax_zero_page_range)) { + /* + * ->zero_page_range() is mandatory dax operation. If we are + * here, something is wrong. + */ + dm_put_live_table(md, srcu_idx); + goto out; + } + ret = ti->type->dax_zero_page_range(ti, offset, len); + + out: + dm_put_live_table(md, srcu_idx); + + return ret; +} + /* * A target may call dm_accept_partial_bio only from the map routine. It is * allowed for all bio types except REQ_PREFLUSH, REQ_OP_ZONE_RESET, @@ -3199,6 +3229,7 @@ static const struct dax_operations dm_dax_ops = { .dax_supported = dm_dax_supported, .copy_from_iter = dm_dax_copy_from_iter, .copy_to_iter = dm_dax_copy_to_iter, + .zero_page_range = dm_dax_zero_page_range, }; /* diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index 475668c69dbc..b4ef5b07be74 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -141,6 +141,8 @@ typedef long (*dm_dax_direct_access_fn) (struct dm_target *ti, pgoff_t pgoff, long nr_pages, void **kaddr, pfn_t *pfn); typedef size_t (*dm_dax_copy_iter_fn)(struct dm_target *ti, pgoff_t pgoff, void *addr, size_t bytes, struct iov_iter *i); +typedef int (*dm_dax_zero_page_range_fn)(struct dm_target *ti, u64 offset, + size_t len); #define PAGE_SECTORS (PAGE_SIZE / 512) void dm_error(const char *message); @@ -195,6 +197,7 @@ struct target_type { dm_dax_direct_access_fn direct_access; dm_dax_copy_iter_fn dax_copy_from_iter; dm_dax_copy_iter_fn dax_copy_to_iter; + dm_dax_zero_page_range_fn dax_zero_page_range; /* For internal device-mapper use. */ struct list_head list; From patchwork Tue Feb 18 21:48:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11389755 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E42B392A for ; Tue, 18 Feb 2020 21:49:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C59AE22B48 for ; Tue, 18 Feb 2020 21:49:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="V68OgOWE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727998AbgBRVtE (ORCPT ); Tue, 18 Feb 2020 16:49:04 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:32646 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727983AbgBRVtD (ORCPT ); Tue, 18 Feb 2020 16:49:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582062541; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Df+aJmrWeo0wbVJ8nParNgumveMm7/SrswBWEMMxch4=; b=V68OgOWEL+3luxF5pByMPWrwxWjo/dKARvlEA6aE0jWB8fv+wKhyivCJU3y0WtvwPiZlTV 6W++D+e2exFFBpaOeXs8+14NpRHJ3BisODQtl7mUiDzrFxV6f9SRUGX6QuXDGORfSbz2aq z/TAlg+evsLMdoTou7/E16KYXdhsRco= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-323-m3ep5xkYMnChUTTFO9SX-A-1; Tue, 18 Feb 2020 16:48:57 -0500 X-MC-Unique: m3ep5xkYMnChUTTFO9SX-A-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7F5CA10CE785; Tue, 18 Feb 2020 21:48:56 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58E0F5C1B0; Tue, 18 Feb 2020 21:48:56 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id B85582257D9; Tue, 18 Feb 2020 16:48:52 -0500 (EST) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, linux-nvdimm@lists.01.org, hch@infradead.org, dan.j.williams@intel.com Cc: dm-devel@redhat.com, vishal.l.verma@intel.com, vgoyal@redhat.com, Christoph Hellwig Subject: [PATCH v5 7/8] dax,iomap: Start using dax native zero_page_range() Date: Tue, 18 Feb 2020 16:48:40 -0500 Message-Id: <20200218214841.10076-8-vgoyal@redhat.com> In-Reply-To: <20200218214841.10076-1-vgoyal@redhat.com> References: <20200218214841.10076-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Get rid of calling block device interface for zeroing in iomap dax zeroing path and use dax native zeroing interface instead. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Vivek Goyal --- fs/dax.c | 45 +++++++++------------------------------------ 1 file changed, 9 insertions(+), 36 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 35da144375a0..f8ae0a9984fa 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1038,48 +1038,21 @@ static vm_fault_t dax_load_hole(struct xa_state *xas, return ret; } -static bool dax_range_is_aligned(struct block_device *bdev, - unsigned int offset, unsigned int length) -{ - unsigned short sector_size = bdev_logical_block_size(bdev); - - if (!IS_ALIGNED(offset, sector_size)) - return false; - if (!IS_ALIGNED(length, sector_size)) - return false; - - return true; -} - int __dax_zero_page_range(struct block_device *bdev, struct dax_device *dax_dev, sector_t sector, unsigned int offset, unsigned int size) { - if (dax_range_is_aligned(bdev, offset, size)) { - sector_t start_sector = sector + (offset >> 9); - - return blkdev_issue_zeroout(bdev, start_sector, - size >> 9, GFP_NOFS, 0); - } else { - pgoff_t pgoff; - long rc, id; - void *kaddr; + pgoff_t pgoff; + long rc, id; - rc = bdev_dax_pgoff(bdev, sector, PAGE_SIZE, &pgoff); - if (rc) - return rc; + rc = bdev_dax_pgoff(bdev, sector, PAGE_SIZE, &pgoff); + if (rc) + return rc; - id = dax_read_lock(); - rc = dax_direct_access(dax_dev, pgoff, 1, &kaddr, NULL); - if (rc < 0) { - dax_read_unlock(id); - return rc; - } - memset(kaddr + offset, 0, size); - dax_flush(dax_dev, kaddr + offset, size); - dax_read_unlock(id); - } - return 0; + id = dax_read_lock(); + rc = dax_zero_page_range(dax_dev, (pgoff << PAGE_SHIFT) + offset, size); + dax_read_unlock(id); + return rc; } EXPORT_SYMBOL_GPL(__dax_zero_page_range); From patchwork Tue Feb 18 21:48:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11389765 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD10C92A for ; Tue, 18 Feb 2020 21:49:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8E7D124125 for ; Tue, 18 Feb 2020 21:49:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SG/l3cnr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728008AbgBRVtH (ORCPT ); Tue, 18 Feb 2020 16:49:07 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:22038 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727976AbgBRVtF (ORCPT ); Tue, 18 Feb 2020 16:49:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582062545; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m3A/bE5M304IMLJrwnJ9LyWGdI9w9dAyP1ptcoHPK04=; b=SG/l3cnr9hH/o5yzaDZ1mlj8a9/aS04lKijkMpFs5ziwOUq+R3KIA8fR3/u+5Gl2gVq/UO 1eCBLhz/K4pY2UcZsVI0pC1GOy21mUYJEI5QHVxbUG9Ep9tPeQzlFVYWuwwrNz4AkcpZrI grdOoLDB/86YPuI7FWkGSj7vVOXNvWk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-324-lKwINEcDMf6UbT6OcDik8A-1; Tue, 18 Feb 2020 16:49:00 -0500 X-MC-Unique: lKwINEcDMf6UbT6OcDik8A-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4C437101FC60; Tue, 18 Feb 2020 21:48:59 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5A09E19756; Tue, 18 Feb 2020 21:48:56 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id C717F2257DA; Tue, 18 Feb 2020 16:48:52 -0500 (EST) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, linux-nvdimm@lists.01.org, hch@infradead.org, dan.j.williams@intel.com Cc: dm-devel@redhat.com, vishal.l.verma@intel.com, vgoyal@redhat.com, Christoph Hellwig Subject: [PATCH v5 8/8] dax,iomap: Add helper dax_iomap_zero() to zero a range Date: Tue, 18 Feb 2020 16:48:41 -0500 Message-Id: <20200218214841.10076-9-vgoyal@redhat.com> In-Reply-To: <20200218214841.10076-1-vgoyal@redhat.com> References: <20200218214841.10076-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a helper dax_ioamp_zero() to zero a range. This patch basically merges __dax_zero_page_range() and iomap_dax_zero(). Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Vivek Goyal --- fs/dax.c | 12 ++++++------ fs/iomap/buffered-io.c | 9 +-------- include/linux/dax.h | 17 +++-------------- 3 files changed, 10 insertions(+), 28 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index f8ae0a9984fa..4be581009db5 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1038,23 +1038,23 @@ static vm_fault_t dax_load_hole(struct xa_state *xas, return ret; } -int __dax_zero_page_range(struct block_device *bdev, - struct dax_device *dax_dev, sector_t sector, - unsigned int offset, unsigned int size) +int dax_iomap_zero(loff_t pos, unsigned offset, unsigned size, + struct iomap *iomap) { pgoff_t pgoff; long rc, id; + sector_t sector = iomap_sector(iomap, pos & PAGE_MASK); - rc = bdev_dax_pgoff(bdev, sector, PAGE_SIZE, &pgoff); + rc = bdev_dax_pgoff(iomap->bdev, sector, PAGE_SIZE, &pgoff); if (rc) return rc; id = dax_read_lock(); - rc = dax_zero_page_range(dax_dev, (pgoff << PAGE_SHIFT) + offset, size); + rc = dax_zero_page_range(iomap->dax_dev, (pgoff << PAGE_SHIFT) + offset, + size); dax_read_unlock(id); return rc; } -EXPORT_SYMBOL_GPL(__dax_zero_page_range); static loff_t dax_iomap_actor(struct inode *inode, loff_t pos, loff_t length, void *data, diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 7c84c4c027c4..6f750da545e5 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -974,13 +974,6 @@ static int iomap_zero(struct inode *inode, loff_t pos, unsigned offset, return iomap_write_end(inode, pos, bytes, bytes, page, iomap, srcmap); } -static int iomap_dax_zero(loff_t pos, unsigned offset, unsigned bytes, - struct iomap *iomap) -{ - return __dax_zero_page_range(iomap->bdev, iomap->dax_dev, - iomap_sector(iomap, pos & PAGE_MASK), offset, bytes); -} - static loff_t iomap_zero_range_actor(struct inode *inode, loff_t pos, loff_t count, void *data, struct iomap *iomap, struct iomap *srcmap) @@ -1000,7 +993,7 @@ iomap_zero_range_actor(struct inode *inode, loff_t pos, loff_t count, bytes = min_t(loff_t, PAGE_SIZE - offset, count); if (IS_DAX(inode)) - status = iomap_dax_zero(pos, offset, bytes, iomap); + status = dax_iomap_zero(pos, offset, bytes, iomap); else status = iomap_zero(inode, pos, offset, bytes, iomap, srcmap); diff --git a/include/linux/dax.h b/include/linux/dax.h index 93a663c26d6a..7b681acfc522 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -13,6 +13,7 @@ typedef unsigned long dax_entry_t; struct iomap_ops; +struct iomap; struct dax_device; struct dax_operations { /* @@ -213,20 +214,8 @@ vm_fault_t dax_finish_sync_fault(struct vm_fault *vmf, int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index); int dax_invalidate_mapping_entry_sync(struct address_space *mapping, pgoff_t index); - -#ifdef CONFIG_FS_DAX -int __dax_zero_page_range(struct block_device *bdev, - struct dax_device *dax_dev, sector_t sector, - unsigned int offset, unsigned int length); -#else -static inline int __dax_zero_page_range(struct block_device *bdev, - struct dax_device *dax_dev, sector_t sector, - unsigned int offset, unsigned int length) -{ - return -ENXIO; -} -#endif - +int dax_iomap_zero(loff_t pos, unsigned offset, unsigned size, + struct iomap *iomap); static inline bool dax_mapping(struct address_space *mapping) { return mapping->host && IS_DAX(mapping->host);