From patchwork Wed Oct 14 03:03:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837137 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48AB214B5 for ; Wed, 14 Oct 2020 09:21:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1A07F214D8 for ; Wed, 14 Oct 2020 09:21:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="h2D+PCR8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730071AbgJNJVr (ORCPT ); Wed, 14 Oct 2020 05:21:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730978AbgJNJUj (ORCPT ); Wed, 14 Oct 2020 05:20:39 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF3F2C0F26E7; Tue, 13 Oct 2020 20:04:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=j56S9O9WrvJwFfp3jWhAG41igcUKnP5ChYtNlOLmqVw=; b=h2D+PCR8Gha6OcbuOB2GBMzhS0 gRUjdmkK723DckDKVFPqjMcduPz3oe6xBGPe5GSehH75/txbO92mgjSmEse+8HPoa2Yq379BRn+hZ pTGxv+R0ZsvJe6iqLMcrvzPfj30EaZi7NAk9yJqHDDHsJW2C1HFyOFN5z4lXPAdxU+et8/LLMvC6b /Mcy8Ya+xrko0fZggbRRzFLko98VDFJ5rvW3WBoyCPnCBSLvGbxKNWi6UOaopBgHMIYUCwZPjFE/E OTp/mwMIFyH6YptVXd+l1UgFZZeH/fgsHWihD/tk74FyBPkxdDeHXUbstUHmpZ28aisEklqjgGMLy 4oG3MciA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX55-0005i3-Ff; Wed, 14 Oct 2020 03:03:59 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 01/14] fs: Support THPs in vfs_dedupe_file_range Date: Wed, 14 Oct 2020 04:03:44 +0100 Message-Id: <20201014030357.21898-2-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org We may get tail pages returned from vfs_dedupe_get_page(). If we do, we have to call page_mapping() instead of dereferencing page->mapping directly. We may also deadlock trying to lock the page twice if they're subpages of the same THP, so compare the head pages instead. Signed-off-by: Matthew Wilcox (Oracle) --- fs/read_write.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/read_write.c b/fs/read_write.c index 19f5c4bf75aa..ead675fef582 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -1604,6 +1604,8 @@ static struct page *vfs_dedupe_get_page(struct inode *inode, loff_t offset) */ static void vfs_lock_two_pages(struct page *page1, struct page *page2) { + page1 = thp_head(page1); + page2 = thp_head(page2); /* Always lock in order of increasing index. */ if (page1->index > page2->index) swap(page1, page2); @@ -1616,6 +1618,8 @@ static void vfs_lock_two_pages(struct page *page1, struct page *page2) /* Unlock two pages, being careful not to unlock the same page twice. */ static void vfs_unlock_two_pages(struct page *page1, struct page *page2) { + page1 = thp_head(page1); + page2 = thp_head(page2); unlock_page(page1); if (page1 != page2) unlock_page(page2); @@ -1670,8 +1674,8 @@ static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff, * someone is invalidating pages on us and we lose. */ if (!PageUptodate(src_page) || !PageUptodate(dest_page) || - src_page->mapping != src->i_mapping || - dest_page->mapping != dest->i_mapping) { + page_mapping(src_page) != src->i_mapping || + page_mapping(dest_page) != dest->i_mapping) { same = false; goto unlock; } From patchwork Wed Oct 14 03:03:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837227 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA32914B5 for ; Wed, 14 Oct 2020 09:25:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9FE322222A for ; Wed, 14 Oct 2020 09:25:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="KjYOvQja" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730664AbgJNJYv (ORCPT ); Wed, 14 Oct 2020 05:24:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730649AbgJNJUB (ORCPT ); Wed, 14 Oct 2020 05:20:01 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26F40C0F26E9; Tue, 13 Oct 2020 20:04:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Fzh51vQNpYK9qDtm8DdSGaGA15CKG5lqyy79LzIykwE=; b=KjYOvQja/Iwuct+ojBrauVcVFx AxDAnkywo8nFPa+UzyHzrFSK8cpK50zKIJo5mc2Lkn4uXOfdssASd7f2vK5QZ2A2Em9jS6AH0C5FC q4eA7p84qUjFnY6ez4a9CO6Ao3Obj7lPz31pHvQxpsSXd38W2AuQW6RysUkD1xOLviU4QgPO/v0vy QVyWPtDCp2/Zp9sWQ9r7gUosW/VBQnfn4kpninNnAGchfyQPqgy1H5sGp/ZGoBvExla98nY1tCq8U AJjTXeo+/HXc9pH26lwtrVUh2cB7McLvU5g8jdj3ciXceDJrtncqAn8Jbh1STKhZta/UmTXamuMlL Uwi7ZsHA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX55-0005i8-Lw; Wed, 14 Oct 2020 03:03:59 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 02/14] fs: Make page_mkwrite_check_truncate thp-aware Date: Wed, 14 Oct 2020 04:03:45 +0100 Message-Id: <20201014030357.21898-3-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org If the page is compound, check the last index in the page and return the appropriate size. Change the return type to ssize_t in case we ever support pages larger than 2GB. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/pagemap.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 7dc6bf713d27..5083a0efafa8 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -983,22 +983,22 @@ static inline unsigned long dir_pages(struct inode *inode) * @page: the page to check * @inode: the inode to check the page against * - * Returns the number of bytes in the page up to EOF, + * Return: The number of bytes in the page up to EOF, * or -EFAULT if the page was truncated. */ -static inline int page_mkwrite_check_truncate(struct page *page, +static inline ssize_t page_mkwrite_check_truncate(struct page *page, struct inode *inode) { loff_t size = i_size_read(inode); pgoff_t index = size >> PAGE_SHIFT; - int offset = offset_in_page(size); + unsigned long offset = offset_in_thp(page, size); if (page->mapping != inode->i_mapping) return -EFAULT; /* page is wholly inside EOF */ - if (page->index < index) - return PAGE_SIZE; + if (page->index + thp_nr_pages(page) - 1 < index) + return thp_size(page); /* page is wholly past EOF */ if (page->index > index || !offset) return -EFAULT; From patchwork Wed Oct 14 03:03:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837155 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E4C115E6 for ; Wed, 14 Oct 2020 09:22:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 48332206DC for ; Wed, 14 Oct 2020 09:22:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="JtDbuy8/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728922AbgJNJW1 (ORCPT ); Wed, 14 Oct 2020 05:22:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731143AbgJNJVw (ORCPT ); Wed, 14 Oct 2020 05:21:52 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69B6FC0F26EA; Tue, 13 Oct 2020 20:04:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=zaip0vMsJG7AN9Csr/40q0m3O27lGYg2jcMuOG9Osec=; b=JtDbuy8/xiVmCy2QAhqgDwxUr0 sT6664qzppcyCFLAJGVc8Y89wkgGRTAZZJ2mEPQXExHsHocwSSx6IfyfF2FhdvBGgXd1qJwS3z11D iEKYLKzhw5utaoblG7UL0HyJCVJMvr8raUUZNCeH8XC5l3hN6milMSz0eiWA+aWQCAriDAyFvegJs ew8oZA8iWrcX3TveFeERKgYVZ+l6HJe0xkNrrKeFXzKqO7KncNwHIHbNjJHIvQ1CAcOFEzgQTkqq1 SGgkgyWqmfL2uHtkPa9rYRaLyUDUdsw9B9HlydFXqNTzTlvG8YuMUgnhYYbOi3Xm3WwuhB80ePWa8 3GPIRuFw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX55-0005iI-Th; Wed, 14 Oct 2020 03:03:59 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 03/14] iomap: Support THPs in BIO completion path Date: Wed, 14 Oct 2020 04:03:46 +0100 Message-Id: <20201014030357.21898-4-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org bio_for_each_segment_all() iterates once per regular sized page. Use bio_for_each_bvec_all() to iterate once per bvec and handle merged THPs ourselves, instead of teaching the block layer about THPs. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 62 ++++++++++++++++++++++++++++++------------ 1 file changed, 44 insertions(+), 18 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 3e1eb40a73fd..935468d79d9d 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -167,32 +167,45 @@ iomap_set_range_uptodate(struct page *page, unsigned off, unsigned len) SetPageUptodate(page); } -static void -iomap_read_page_end_io(struct bio_vec *bvec, int error) +static void iomap_finish_page_read(struct page *page, size_t offset, + size_t length, int error) { - struct page *page = bvec->bv_page; struct iomap_page *iop = to_iomap_page(page); if (unlikely(error)) { ClearPageUptodate(page); SetPageError(page); } else { - iomap_set_range_uptodate(page, bvec->bv_offset, bvec->bv_len); + iomap_set_range_uptodate(page, offset, length); } - if (!iop || atomic_sub_and_test(bvec->bv_len, &iop->read_bytes_pending)) + if (!iop || atomic_sub_and_test(length, &iop->read_bytes_pending)) unlock_page(page); } -static void -iomap_read_end_io(struct bio *bio) +static void iomap_finish_bvec_read(struct page *page, size_t offset, + size_t length, int error) +{ + while (length > 0) { + size_t count = min(thp_size(page) - offset, length); + + iomap_finish_page_read(page, offset, count, error); + + page += (offset + count) / PAGE_SIZE; + offset = 0; + length -= count; + } +} + +static void iomap_read_end_io(struct bio *bio) { - int error = blk_status_to_errno(bio->bi_status); + int i, error = blk_status_to_errno(bio->bi_status); struct bio_vec *bvec; - struct bvec_iter_all iter_all; - bio_for_each_segment_all(bvec, bio, iter_all) - iomap_read_page_end_io(bvec, error); + bio_for_each_bvec_all(bvec, bio, i) + iomap_finish_bvec_read(bvec->bv_page, bvec->bv_offset, + bvec->bv_len, error); + bio_put(bio); } @@ -1035,9 +1048,8 @@ vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops) } EXPORT_SYMBOL_GPL(iomap_page_mkwrite); -static void -iomap_finish_page_writeback(struct inode *inode, struct page *page, - int error, unsigned int len) +static void iomap_finish_page_write(struct inode *inode, struct page *page, + unsigned int len, int error) { struct iomap_page *iop = to_iomap_page(page); @@ -1053,6 +1065,20 @@ iomap_finish_page_writeback(struct inode *inode, struct page *page, end_page_writeback(page); } +static void iomap_finish_bvec_write(struct inode *inode, struct page *page, + size_t offset, size_t length, int error) +{ + while (length > 0) { + size_t count = min(thp_size(page) - offset, length); + + iomap_finish_page_write(inode, page, count, error); + + page += (offset + count) / PAGE_SIZE; + offset = 0; + length -= count; + } +} + /* * We're now finished for good with this ioend structure. Update the page * state, release holds on bios, and finally free up memory. Do not use the @@ -1070,7 +1096,7 @@ iomap_finish_ioend(struct iomap_ioend *ioend, int error) for (bio = &ioend->io_inline_bio; bio; bio = next) { struct bio_vec *bv; - struct bvec_iter_all iter_all; + int i; /* * For the last bio, bi_private points to the ioend, so we @@ -1082,9 +1108,9 @@ iomap_finish_ioend(struct iomap_ioend *ioend, int error) next = bio->bi_private; /* walk each page on bio, ending page IO on them */ - bio_for_each_segment_all(bv, bio, iter_all) - iomap_finish_page_writeback(inode, bv->bv_page, error, - bv->bv_len); + bio_for_each_bvec_all(bv, bio, i) + iomap_finish_bvec_write(inode, bv->bv_page, + bv->bv_offset, bv->bv_len, error); bio_put(bio); } /* The ioend has been freed by bio_put() */ From patchwork Wed Oct 14 03:03:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837167 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10B241130 for ; Wed, 14 Oct 2020 09:22:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD6C3206F4 for ; Wed, 14 Oct 2020 09:22:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="dAJRx/fT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726663AbgJNJW4 (ORCPT ); Wed, 14 Oct 2020 05:22:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731217AbgJNJWy (ORCPT ); Wed, 14 Oct 2020 05:22:54 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B251EC0F26EB; Tue, 13 Oct 2020 20:04:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=4iAeOZdU9QHnYt7ejF3PPD264zNgtL6+n5nU0rXZdys=; b=dAJRx/fTLFMzLRu/24u9ewIUv3 xP30B9BFUpSIa5/3fqHNpra5nwFkI5jcArT2lerdK1DYO6lO1qghDherQqwZ4rbi9eXvgZN0vV61Y ashI2qr1egjhnH51A9G/HzAWN4DvFa1iwZjwwipy2VHCGIpqI1Fg4zSPPFTGsZwMbEz9HsuVLZJNj Nje7S21oyHvQ6RXw8DWwE9iXveCTV75c0/MxTxSnRcf6SJfJk+zXjEMfkegeGDOHDUe2vjoy4CRty UV9zS58yT8WAyrgh2P7zEr7IdGhutz5C/4KuO3jpYGYe2eK0noE6y9ZWgA0Ao7vFLHhDIv/+9PtJI 18TjE90Q==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX56-0005iQ-5d; Wed, 14 Oct 2020 03:04:00 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 04/14] iomap: Support THPs in iomap_adjust_read_range Date: Wed, 14 Oct 2020 04:03:47 +0100 Message-Id: <20201014030357.21898-5-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Pass the struct page instead of the iomap_page so we can determine the size of the page. Use offset_in_thp() instead of offset_in_page() and use thp_size() instead of PAGE_SIZE. Convert the arguments to be size_t instead of unsigned int, in case pages ever get larger than 2^31 bytes. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 935468d79d9d..95ac66731297 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -82,16 +82,16 @@ iomap_page_release(struct page *page) /* * Calculate the range inside the page that we actually need to read. */ -static void -iomap_adjust_read_range(struct inode *inode, struct iomap_page *iop, - loff_t *pos, loff_t length, unsigned *offp, unsigned *lenp) +static void iomap_adjust_read_range(struct inode *inode, struct page *page, + loff_t *pos, loff_t length, size_t *offp, size_t *lenp) { + struct iomap_page *iop = to_iomap_page(page); loff_t orig_pos = *pos; loff_t isize = i_size_read(inode); unsigned block_bits = inode->i_blkbits; unsigned block_size = (1 << block_bits); - unsigned poff = offset_in_page(*pos); - unsigned plen = min_t(loff_t, PAGE_SIZE - poff, length); + size_t poff = offset_in_thp(page, *pos); + size_t plen = min_t(loff_t, thp_size(page) - poff, length); unsigned first = poff >> block_bits; unsigned last = (poff + plen - 1) >> block_bits; @@ -129,7 +129,7 @@ iomap_adjust_read_range(struct inode *inode, struct iomap_page *iop, * page cache for blocks that are entirely outside of i_size. */ if (orig_pos <= isize && orig_pos + length > isize) { - unsigned end = offset_in_page(isize - 1) >> block_bits; + unsigned end = offset_in_thp(page, isize - 1) >> block_bits; if (first <= end && last > end) plen -= (last - end) * block_size; @@ -253,7 +253,7 @@ iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data, struct iomap_page *iop = iomap_page_create(inode, page); bool same_page = false, is_contig = false; loff_t orig_pos = pos; - unsigned poff, plen; + size_t poff, plen; sector_t sector; if (iomap->type == IOMAP_INLINE) { @@ -263,7 +263,7 @@ iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data, } /* zero post-eof blocks as the page may be mapped */ - iomap_adjust_read_range(inode, iop, &pos, length, &poff, &plen); + iomap_adjust_read_range(inode, page, &pos, length, &poff, &plen); if (plen == 0) goto done; @@ -561,18 +561,19 @@ static int __iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, int flags, struct page *page, struct iomap *srcmap) { - struct iomap_page *iop = iomap_page_create(inode, page); loff_t block_size = i_blocksize(inode); loff_t block_start = pos & ~(block_size - 1); loff_t block_end = (pos + len + block_size - 1) & ~(block_size - 1); - unsigned from = offset_in_page(pos), to = from + len, poff, plen; + unsigned from = offset_in_page(pos), to = from + len; + size_t poff, plen; if (PageUptodate(page)) return 0; ClearPageError(page); + iomap_page_create(inode, page); do { - iomap_adjust_read_range(inode, iop, &block_start, + iomap_adjust_read_range(inode, page, &block_start, block_end - block_start, &poff, &plen); if (plen == 0) break; From patchwork Wed Oct 14 03:03:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837105 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 273E71130 for ; Wed, 14 Oct 2020 09:20:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0197620878 for ; Wed, 14 Oct 2020 09:20:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="G1zcZpij" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730759AbgJNJUJ (ORCPT ); Wed, 14 Oct 2020 05:20:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387823AbgJNJUG (ORCPT ); Wed, 14 Oct 2020 05:20:06 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02471C0F26EC; Tue, 13 Oct 2020 20:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=HCUdH2gZ2mH2ZOiD3F29fY5+kaO7fY88tsiSFo4riTw=; b=G1zcZpijkarteofqchFTXXIIQW vBzEil6I7JRxpZhCNqz47pCtJDP7jL4ydibo68z1pfHAPcz61TtjnPUJhrlcFcdIlC7bvUYxbzeh/ gJT6IDybpCuGYNxLXQjLqzgXPCUXNc+/rhDMz/EEzpn9lfb+5bqYrmycSOoFL53Mp4sDaMHYQIBDg KPppMqsoDEr3OPtV5qQUzC5jCog+ZAXl8qlkiSXQvHxwA+6vuPrkWEt2CChpDRoyQpKg0ZYufrNph 5NnMUCqhIq8sstn9ovmiKWm0Jkaz9Xm/8fxX0BQZFzsoatYJ/6tCa7pl6fM+vVThtLsNZsJI/iHWH hBn5sAuw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX56-0005ia-FO; Wed, 14 Oct 2020 03:04:00 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 05/14] iomap: Support THPs in invalidatepage Date: Wed, 14 Oct 2020 04:03:48 +0100 Message-Id: <20201014030357.21898-6-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org If we're punching a hole in a THP, we need to remove the per-page iomap data as the THP is about to be split and each page will need its own. This means that writepage can now come across a page with no iop allocated, so remove the assertion that there is already one, and just create one (with the uptodate bits set) if there isn't one. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 95ac66731297..4633ebd03a3f 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -60,6 +60,8 @@ iomap_page_create(struct inode *inode, struct page *page) iop = kzalloc(struct_size(iop, uptodate, BITS_TO_LONGS(nr_blocks)), GFP_NOFS | __GFP_NOFAIL); spin_lock_init(&iop->uptodate_lock); + if (PageUptodate(page)) + bitmap_fill(iop->uptodate, nr_blocks); attach_page_private(page, iop); return iop; } @@ -494,10 +496,14 @@ iomap_invalidatepage(struct page *page, unsigned int offset, unsigned int len) * If we are invalidating the entire page, clear the dirty state from it * and release it to avoid unnecessary buildup of the LRU. */ - if (offset == 0 && len == PAGE_SIZE) { + if (offset == 0 && len == thp_size(page)) { WARN_ON_ONCE(PageWriteback(page)); cancel_dirty_page(page); iomap_page_release(page); + } else if (PageTransHuge(page)) { + /* Punching a hole in a THP requires releasing the iop */ + WARN_ON_ONCE(!PageUptodate(page) && PageDirty(page)); + iomap_page_release(page); } } EXPORT_SYMBOL_GPL(iomap_invalidatepage); @@ -1363,14 +1369,13 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, struct writeback_control *wbc, struct inode *inode, struct page *page, u64 end_offset) { - struct iomap_page *iop = to_iomap_page(page); + struct iomap_page *iop = iomap_page_create(inode, page); struct iomap_ioend *ioend, *next; unsigned len = i_blocksize(inode); u64 file_offset; /* file offset of page */ int error = 0, count = 0, i; LIST_HEAD(submit_list); - WARN_ON_ONCE(i_blocks_per_page(inode, page) > 1 && !iop); WARN_ON_ONCE(iop && atomic_read(&iop->write_bytes_pending) != 0); /* @@ -1415,7 +1420,6 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, */ if (wpc->ops->discard_page) wpc->ops->discard_page(page); - ClearPageUptodate(page); unlock_page(page); goto done; } From patchwork Wed Oct 14 03:03:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8A1D15E6 for ; Wed, 14 Oct 2020 09:20:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B04DE214D8 for ; Wed, 14 Oct 2020 09:20:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="rC3J4Sbt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731101AbgJNJUu (ORCPT ); Wed, 14 Oct 2020 05:20:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39952 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731049AbgJNJUn (ORCPT ); Wed, 14 Oct 2020 05:20:43 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45A43C0F26ED; Tue, 13 Oct 2020 20:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=3K1TcA9ng9V8ipeBlpNGiMaylyWSGeQVhOpeVH2itWk=; b=rC3J4SbtqpkzQ1b036e2mLfvI4 oWvmxuQVRrYNkC8ODb4AFicqOCMYWUmy/RJoLyi/MH7gm4Sn6GFWC1ZpwP32WgHEFut8PS0plvbOf qUGBRvqQmkLmsmboFYn0MHCBAjgxKUHpCyfkzvjF+cqlluDaAFK7bCOCR1sTjy1nWq787sSFORN+x wolPiv9KqKkmY0g/2hEMkjFTeMBhGMsVi54Pbu56zHxYpZAcpIDxo/iHSiRU9on+eEgkZKsoKW8ok Faiw7eDvCChe/VIce3eURUenV8cPw7KmCuySJEZ84ZDQQdUxc3MJcKBj32K/jX3FIIJjtGWmgjTaV BwaQFfOw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX56-0005ii-Ng; Wed, 14 Oct 2020 03:04:00 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 06/14] iomap: Support THPs in iomap_is_partially_uptodate Date: Wed, 14 Oct 2020 04:03:49 +0100 Message-Id: <20201014030357.21898-7-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Factor iomap_range_uptodate() out of iomap_is_partially_uptodate() to use by iomap_readpage() later. iomap_is_partially_uptodate() can be called on a tail page by generic_file_buffered_read(), so handle that correctly. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 43 ++++++++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 18 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 4633ebd03a3f..4ea6c601a183 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -141,6 +141,24 @@ static void iomap_adjust_read_range(struct inode *inode, struct page *page, *lenp = plen; } +static bool iomap_range_uptodate(struct inode *inode, struct page *page, + size_t start, size_t len) +{ + struct iomap_page *iop = to_iomap_page(page); + size_t first = start >> inode->i_blkbits; + size_t last = (start + len - 1) >> inode->i_blkbits; + size_t i; + + VM_BUG_ON_PGFLAGS(!PageLocked(page), page); + if (!iop) + return false; + + for (i = first; i <= last; i++) + if (!test_bit(i, iop->uptodate)) + return false; + return true; +} + static void iomap_iop_set_range_uptodate(struct page *page, unsigned off, unsigned len) { @@ -446,26 +464,15 @@ int iomap_is_partially_uptodate(struct page *page, unsigned long from, unsigned long count) { - struct iomap_page *iop = to_iomap_page(page); - struct inode *inode = page->mapping->host; - unsigned len, first, last; - unsigned i; - - /* Limit range to one page */ - len = min_t(unsigned, PAGE_SIZE - from, count); + struct page *head = thp_head(page); + size_t len; - /* First and last blocks in range within page */ - first = from >> inode->i_blkbits; - last = (from + len - 1) >> inode->i_blkbits; + /* 'from' is relative to page, but the bitmap is relative to head */ + from += (page - head) * PAGE_SIZE; + /* Limit range to this page */ + len = min(thp_size(head) - from, count); - if (iop) { - for (i = first; i <= last; i++) - if (!test_bit(i, iop->uptodate)) - return 0; - return 1; - } - - return 0; + return iomap_range_uptodate(head->mapping->host, head, from, len); } EXPORT_SYMBOL_GPL(iomap_is_partially_uptodate); From patchwork Wed Oct 14 03:03:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837199 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B7E11130 for ; Wed, 14 Oct 2020 09:24:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7424B206F4 for ; Wed, 14 Oct 2020 09:24:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="WykgJeVC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730071AbgJNJYF (ORCPT ); Wed, 14 Oct 2020 05:24:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731165AbgJNJV4 (ORCPT ); Wed, 14 Oct 2020 05:21:56 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7154CC0F26EE; Tue, 13 Oct 2020 20:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=0MPgJlidjQUsAxPbCUuZp6ht/vrWL0vbWyhDYWDy180=; b=WykgJeVCo2jBQUR9e914mBBRiL t/lZ2mFkZxghyBxhOoN0gFG4v4rBCu0O8s+jjng4FgaJdj2EsMcCkCSkh+dr9VEj2ml19Jd+E4hC0 k0ZdVqKhuOx51oPCzYDDFk42yzL1+JCDjJv9Q2WG0rrLDeKgs66dLxPxhbn2kr1msHppQ+OYTROqV ZlMhXD/1ztzOIQpwfCmWcMTEGtVLWtatHUrKfNq9lh46GEbr7tO2hR4IhKm0BJNbZtLvPE9JsqQ/K eODQ+8dzCnkWwPNVForkhUZEQyL2kCn6wX5tSY96UCkylBbkGgdWExB+OkxMhemfbWANj/RQGCtOF ix1iQOyw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX56-0005ip-Uo; Wed, 14 Oct 2020 03:04:00 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 07/14] iomap: Support THPs in readpage Date: Wed, 14 Oct 2020 04:03:50 +0100 Message-Id: <20201014030357.21898-8-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org The VFS only calls readpage if readahead has encountered an error. Assume that any error requires the page to be split, and attempt to do so. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 4ea6c601a183..ca305fbaf811 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -343,15 +343,50 @@ iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data, return pos - orig_pos + plen; } +/* + * The page that was passed in has become Uptodate. This may be due to + * the storage being synchronous or due to a page split finding the page + * is actually uptodate. The page is still locked. + * Lift this into the VFS at some point. + */ +#define AOP_UPDATED_PAGE (AOP_TRUNCATED_PAGE + 1) + +static int iomap_split_page(struct inode *inode, struct page *page) +{ + struct page *head = thp_head(page); + bool uptodate = iomap_range_uptodate(inode, head, + (page - head) * PAGE_SIZE, PAGE_SIZE); + + iomap_page_release(head); + if (split_huge_page(page) < 0) { + unlock_page(page); + return AOP_TRUNCATED_PAGE; + } + if (!uptodate) + return 0; + SetPageUptodate(page); + return AOP_UPDATED_PAGE; +} + int iomap_readpage(struct page *page, const struct iomap_ops *ops) { struct iomap_readpage_ctx ctx = { .cur_page = page }; - struct inode *inode = page->mapping->host; + struct inode *inode = thp_head(page)->mapping->host; unsigned poff; loff_t ret; - trace_iomap_readpage(page->mapping->host, 1); + trace_iomap_readpage(inode, 1); + + if (PageTransCompound(page)) { + int status = iomap_split_page(inode, page); + if (status == AOP_UPDATED_PAGE) { + unlock_page(page); + return 0; + } + if (status) + return status; + } for (poff = 0; poff < PAGE_SIZE; poff += ret) { ret = iomap_apply(inode, page_offset(page) + poff, From patchwork Wed Oct 14 03:03:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837209 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DACD815E6 for ; Wed, 14 Oct 2020 09:24:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1C88206DD for ; Wed, 14 Oct 2020 09:24:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Nk0Ramlg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728292AbgJNJYg (ORCPT ); Wed, 14 Oct 2020 05:24:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39932 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730794AbgJNJUK (ORCPT ); Wed, 14 Oct 2020 05:20:10 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2C38C0F26EF; Tue, 13 Oct 2020 20:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=rdRSJzenCaIYGt1c8MjAxB4wKLLuGR+cw6SImpK8bI0=; b=Nk0RamlgvFMF3mfimqDPDv38YJ k1gSyEeWe3JCYqS7T7athwnlLgcyd0P2Fb9Po755SB01tnrMWAFTijZxeBzd4TW/PDxKxFit+H5ID hqjAdiDH71a2YRdx9q965OeHOYfhzmY/Hz1sjf3qcozLR9fNew4FybrFICoJig4+8JKH8jBbolkQI CXmKrCX7BrT/LwIZZ6Z4UD5/8g9CeVXm7g2FChVcECTAGu5lLACZ52E+Y/nnTXSm0/dVrXDbBL1bA HDCcjAK9gkkfF0b6I7L2QD+TPiSVl6U1Mgm0ncYdO3NQC82J2km+nJQjis4xaFJkRykAbVlPnOtX8 4nPVlDfw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX57-0005iw-6i; Wed, 14 Oct 2020 03:04:01 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 08/14] iomap: Support THPs in readahead Date: Wed, 14 Oct 2020 04:03:51 +0100 Message-Id: <20201014030357.21898-9-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Use offset_in_thp() instead of offset_in_page() and change how we estimate the number of bio_vecs we're going to need. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index ca305fbaf811..4ef02afaedc5 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -264,6 +264,16 @@ static inline bool iomap_block_needs_zeroing(struct inode *inode, pos >= i_size_read(inode); } +/* + * Estimate the number of vectors we need based on the current page size; + * if we're wrong we'll end up doing an overly large allocation or needing + * to do a second allocation, neither of which is a big deal. + */ +static unsigned int iomap_nr_vecs(struct page *page, loff_t length) +{ + return (length + thp_size(page) - 1) >> page_shift(page); +} + static loff_t iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data, struct iomap *iomap, struct iomap *srcmap) @@ -309,7 +319,7 @@ iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data, if (!is_contig || bio_full(ctx->bio, plen)) { gfp_t gfp = mapping_gfp_constraint(page->mapping, GFP_KERNEL); gfp_t orig_gfp = gfp; - int nr_vecs = (length + PAGE_SIZE - 1) >> PAGE_SHIFT; + int nr_vecs = iomap_nr_vecs(page, length); if (ctx->bio) submit_bio(ctx->bio); @@ -424,7 +434,8 @@ iomap_readahead_actor(struct inode *inode, loff_t pos, loff_t length, loff_t done, ret; for (done = 0; done < length; done += ret) { - if (ctx->cur_page && offset_in_page(pos + done) == 0) { + if (ctx->cur_page && + offset_in_thp(ctx->cur_page, pos + done) == 0) { if (!ctx->cur_page_in_bio) unlock_page(ctx->cur_page); put_page(ctx->cur_page); From patchwork Wed Oct 14 03:03:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837239 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9091B1130 for ; Wed, 14 Oct 2020 09:30:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6505E206F4 for ; Wed, 14 Oct 2020 09:30:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="abukv0v1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727826AbgJNJan (ORCPT ); Wed, 14 Oct 2020 05:30:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729574AbgJNJTh (ORCPT ); Wed, 14 Oct 2020 05:19:37 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECDCDC0F26F0; Tue, 13 Oct 2020 20:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=78jQva7SN0URAIlYak97f7/G/y+lO4R9hQAD5LqWZfM=; b=abukv0v1g/Y5sNHv1T8SRp7RLT 2S5W1gZUZIfNydJ4TCWEIlrzKkaM/3O3UM23uKmiN2gir7ANyxX1ey2ra2cmfmfZgzkoiK/aNfLp+ icIlf5blq+ln6nx68CNgDpx/WHFyJ4HYIjaWn/H+V3SKzU/uO3cX0y6PqSi9S7wdlahpPmDfufqSl brOCMrF/BB2pNmj3zPHnqgxi4W+UX55+Pux6nJ1dDDV1RtWeS9vbxQG96lB9iim18PVZYVN09Om+d VFGJqwolfbxukF80XjIg8zUZRHnxz+/tzMFGTffHBFsW5HQAJCodJYccYZymlCQ3//N6FuIZntpV+ p8bkrL9Q==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX57-0005j3-G0; Wed, 14 Oct 2020 03:04:01 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 09/14] iomap: Change iomap_write_begin calling convention Date: Wed, 14 Oct 2020 04:03:52 +0100 Message-Id: <20201014030357.21898-10-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Pass (up to) the remaining length of the extent to iomap_write_begin() and have it return the number of bytes that will fit in the page. That lets us copy more bytes per call to iomap_write_begin() if the page cache has already allocated a THP (and will in future allow us to pass a hint to the page cache that it should try to allocate a larger page if there are none in the cache). Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 61 +++++++++++++++++++++++------------------- 1 file changed, 33 insertions(+), 28 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 4ef02afaedc5..397795db3ce5 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -616,14 +616,14 @@ iomap_read_page_sync(loff_t block_start, struct page *page, unsigned poff, return submit_bio_wait(&bio); } -static int -__iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, int flags, - struct page *page, struct iomap *srcmap) +static ssize_t __iomap_write_begin(struct inode *inode, loff_t pos, + size_t len, int flags, struct page *page, struct iomap *srcmap) { loff_t block_size = i_blocksize(inode); loff_t block_start = pos & ~(block_size - 1); loff_t block_end = (pos + len + block_size - 1) & ~(block_size - 1); - unsigned from = offset_in_page(pos), to = from + len; + size_t from = offset_in_thp(page, pos); + size_t to = from + len; size_t poff, plen; if (PageUptodate(page)) @@ -658,12 +658,13 @@ __iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, int flags, return 0; } -static int -iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, unsigned flags, - struct page **pagep, struct iomap *iomap, struct iomap *srcmap) +static ssize_t iomap_write_begin(struct inode *inode, loff_t pos, loff_t len, + unsigned flags, struct page **pagep, struct iomap *iomap, + struct iomap *srcmap) { const struct iomap_page_ops *page_ops = iomap->page_ops; struct page *page; + size_t offset; int status = 0; BUG_ON(pos + len > iomap->offset + iomap->length); @@ -674,6 +675,8 @@ iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, unsigned flags, return -EINTR; if (page_ops && page_ops->page_prepare) { + if (len > UINT_MAX) + len = UINT_MAX; status = page_ops->page_prepare(inode, pos, len, iomap); if (status) return status; @@ -685,6 +688,10 @@ iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, unsigned flags, status = -ENOMEM; goto out_no_page; } + page = thp_head(page); + offset = offset_in_thp(page, pos); + if (len > thp_size(page) - offset) + len = thp_size(page) - offset; if (srcmap->type == IOMAP_INLINE) iomap_read_inline_data(inode, page, srcmap); @@ -694,11 +701,11 @@ iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, unsigned flags, status = __iomap_write_begin(inode, pos, len, flags, page, srcmap); - if (unlikely(status)) + if (status < 0) goto out_unlock; *pagep = page; - return 0; + return len; out_unlock: unlock_page(page); @@ -854,8 +861,10 @@ iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data, status = iomap_write_begin(inode, pos, bytes, 0, &page, iomap, srcmap); - if (unlikely(status)) + if (status < 0) break; + /* We may be partway through a THP */ + offset = offset_in_thp(page, pos); if (mapping_writably_mapped(inode->i_mapping)) flush_dcache_page(page); @@ -915,7 +924,6 @@ static loff_t iomap_unshare_actor(struct inode *inode, loff_t pos, loff_t length, void *data, struct iomap *iomap, struct iomap *srcmap) { - long status = 0; loff_t written = 0; /* don't bother with blocks that are not shared to start with */ @@ -926,25 +934,24 @@ iomap_unshare_actor(struct inode *inode, loff_t pos, loff_t length, void *data, return length; do { - unsigned long offset = offset_in_page(pos); - unsigned long bytes = min_t(loff_t, PAGE_SIZE - offset, length); struct page *page; + ssize_t bytes; - status = iomap_write_begin(inode, pos, bytes, + bytes = iomap_write_begin(inode, pos, length, IOMAP_WRITE_F_UNSHARE, &page, iomap, srcmap); - if (unlikely(status)) - return status; + if (bytes < 0) + return bytes; - status = iomap_write_end(inode, pos, bytes, bytes, page, iomap, + bytes = iomap_write_end(inode, pos, bytes, bytes, page, iomap, srcmap); - if (WARN_ON_ONCE(status == 0)) + if (WARN_ON_ONCE(bytes == 0)) return -EIO; cond_resched(); - pos += status; - written += status; - length -= status; + pos += bytes; + written += bytes; + length -= bytes; balance_dirty_pages_ratelimited(inode->i_mapping); } while (length); @@ -975,15 +982,13 @@ static s64 iomap_zero(struct inode *inode, loff_t pos, u64 length, struct iomap *iomap, struct iomap *srcmap) { struct page *page; - int status; - unsigned offset = offset_in_page(pos); - unsigned bytes = min_t(u64, PAGE_SIZE - offset, length); + ssize_t bytes; - status = iomap_write_begin(inode, pos, bytes, 0, &page, iomap, srcmap); - if (status) - return status; + bytes = iomap_write_begin(inode, pos, length, 0, &page, iomap, srcmap); + if (bytes < 0) + return bytes; - zero_user(page, offset, bytes); + zero_user(page, offset_in_thp(page, pos), bytes); mark_page_accessed(page); return iomap_write_end(inode, pos, bytes, bytes, page, iomap, srcmap); From patchwork Wed Oct 14 03:03:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837121 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0E301744 for ; Wed, 14 Oct 2020 09:21:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B9FD12222A for ; Wed, 14 Oct 2020 09:21:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="gFatcmt4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731096AbgJNJU5 (ORCPT ); Wed, 14 Oct 2020 05:20:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731051AbgJNJUn (ORCPT ); Wed, 14 Oct 2020 05:20:43 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 484ACC0F26F1; Tue, 13 Oct 2020 20:04:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Bpmz/m3iXeUk23W+8MMxf9WSox2EZIuxPCrXIX5P4lY=; b=gFatcmt45Yz9HQXA6+8PQzqSET 4Gz04kVe8F6sUR6iuXkTL46htFS73/cIwxf531hY/wkhpI+vGMIhhrSXen6fh41NGOKIY/tul7HQk lzVgf1FzIQnX+aAVlXuqPM5MpzBC4ZHCsROAI2kDldXwRy7Ijvx0nGMlqehv6E0oTKAaH+KAKjAxx kITWsrJuvbtX4TqD61FJLyzYzr3tmPngIGxy/oWTl7tZt37YKAa6GP19wEYyZp4LPlbxZPXch+nTx XvouAM8dl8+qnqjfa80i4r5g/g6bt8tdjlI5HBio5tJGImaRAkDzX1EXltww0gBwP1WLYLjvrfdkd vTMLibqw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX57-0005jA-Py; Wed, 14 Oct 2020 03:04:01 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 10/14] iomap: Handle THPs when writing to pages Date: Wed, 14 Oct 2020 04:03:53 +0100 Message-Id: <20201014030357.21898-11-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org If we come across a THP that is not uptodate when writing to the page cache, this must be due to a readahead error, so behave the same way as readpage and split it. Make sure to flush the right page after completing the write. We still only copy up to a page boundary, so there's no need to flush multiple pages at this time. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 397795db3ce5..0a1fe7d1a27c 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -682,12 +682,19 @@ static ssize_t iomap_write_begin(struct inode *inode, loff_t pos, loff_t len, return status; } +retry: page = grab_cache_page_write_begin(inode->i_mapping, pos >> PAGE_SHIFT, AOP_FLAG_NOFS); if (!page) { status = -ENOMEM; goto out_no_page; } + if (PageTransCompound(page) && !PageUptodate(page)) { + if (iomap_split_page(inode, page) == AOP_TRUNCATED_PAGE) { + put_page(page); + goto retry; + } + } page = thp_head(page); offset = offset_in_thp(page, pos); if (len > thp_size(page) - offset) @@ -724,6 +731,7 @@ iomap_set_page_dirty(struct page *page) struct address_space *mapping = page_mapping(page); int newly_dirty; + VM_BUG_ON_PGFLAGS(PageTail(page), page); if (unlikely(!mapping)) return !TestSetPageDirty(page); @@ -746,7 +754,9 @@ EXPORT_SYMBOL_GPL(iomap_set_page_dirty); static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len, size_t copied, struct page *page) { - flush_dcache_page(page); + size_t offset = offset_in_thp(page, pos); + + flush_dcache_page(page + offset / PAGE_SIZE); /* * The blocks that were entirely written will now be uptodate, so we @@ -761,7 +771,7 @@ static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len, */ if (unlikely(copied < len && !PageUptodate(page))) return 0; - iomap_set_range_uptodate(page, offset_in_page(pos), len); + iomap_set_range_uptodate(page, offset, len); iomap_set_page_dirty(page); return copied; } @@ -837,6 +847,10 @@ iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data, unsigned long bytes; /* Bytes to write to page */ size_t copied; /* Bytes copied from user */ + /* + * XXX: We don't know what size page we'll find in the + * page cache, so only copy up to a regular page boundary. + */ offset = offset_in_page(pos); bytes = min_t(unsigned long, PAGE_SIZE - offset, iov_iter_count(i)); @@ -867,7 +881,7 @@ iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data, offset = offset_in_thp(page, pos); if (mapping_writably_mapped(inode->i_mapping)) - flush_dcache_page(page); + flush_dcache_page(page + offset / PAGE_SIZE); copied = iov_iter_copy_from_user_atomic(page, i, offset, bytes); From patchwork Wed Oct 14 03:03:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837235 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 669F214B4 for ; Wed, 14 Oct 2020 09:30:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 32B4F206F4 for ; Wed, 14 Oct 2020 09:30:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="vj5QlohI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725984AbgJNJaY (ORCPT ); Wed, 14 Oct 2020 05:30:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730103AbgJNJTj (ORCPT ); Wed, 14 Oct 2020 05:19:39 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85117C0F26F2; Tue, 13 Oct 2020 20:04:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=4CizuPWSROnWzpNyUK4VUnX0ZFhzuX47JJGzIGersuE=; b=vj5QlohI/1Ko5rtIubd+FWW2Ud Y75jzewv2TNpOe0TCvDblxbt2bnQLFPAZuwheWl+QEvi8MNl3qPD3/Q0pDcvQ9LJzWqBnOwNmcVjA NmhHMZe7gGracSc7SgU1GumvEP0xT7VLtQ/GaGRmCjHLQfBCJbpQzka4p+SRrqYU0cs5nxnWe8ZhR 7TBes0NRdTe1L47Vz/sKIPGoD39z0TgNHf4AQoIjCjDSsMuoXlL6Sk2w0COX0zbT9nWAx4IcqFmOv nhGJOVByRL71PxP/HcOiIy46v3geKyhSy+tB1jLXdyKFTW9ZIs/+UtLvJtOzqMx4PwdlbkMbGKVro rq6GaoLw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX58-0005jH-21; Wed, 14 Oct 2020 03:04:02 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 11/14] iomap: Support THP writeback Date: Wed, 14 Oct 2020 04:03:54 +0100 Message-Id: <20201014030357.21898-12-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Use offset_in_thp() and similar helpers to submit THPs for writeback. This simplifies the logic in iomap_do_writepage() around handling the end of the file. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 0a1fe7d1a27c..38fd69ebd4cc 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -1394,7 +1394,7 @@ iomap_add_to_ioend(struct inode *inode, loff_t offset, struct page *page, { sector_t sector = iomap_sector(&wpc->iomap, offset); unsigned len = i_blocksize(inode); - unsigned poff = offset & (PAGE_SIZE - 1); + unsigned poff = offset_in_thp(page, offset); bool merged, same_page = false; if (!wpc->ioend || !iomap_can_add_to_ioend(wpc, offset, sector)) { @@ -1444,8 +1444,9 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, struct iomap_page *iop = iomap_page_create(inode, page); struct iomap_ioend *ioend, *next; unsigned len = i_blocksize(inode); - u64 file_offset; /* file offset of page */ + loff_t pos; int error = 0, count = 0, i; + int nr_blocks = i_blocks_per_page(inode, page); LIST_HEAD(submit_list); WARN_ON_ONCE(iop && atomic_read(&iop->write_bytes_pending) != 0); @@ -1455,20 +1456,20 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, * end of the current map or find the current map invalid, grab a new * one. */ - for (i = 0, file_offset = page_offset(page); - i < (PAGE_SIZE >> inode->i_blkbits) && file_offset < end_offset; - i++, file_offset += len) { + for (i = 0, pos = page_offset(page); + i < nr_blocks && pos < end_offset; + i++, pos += len) { if (iop && !test_bit(i, iop->uptodate)) continue; - error = wpc->ops->map_blocks(wpc, inode, file_offset); + error = wpc->ops->map_blocks(wpc, inode, pos); if (error) break; if (WARN_ON_ONCE(wpc->iomap.type == IOMAP_INLINE)) continue; if (wpc->iomap.type == IOMAP_HOLE) continue; - iomap_add_to_ioend(inode, file_offset, page, iop, wpc, wbc, + iomap_add_to_ioend(inode, pos, page, iop, wpc, wbc, &submit_list); count++; } @@ -1549,11 +1550,11 @@ iomap_do_writepage(struct page *page, struct writeback_control *wbc, void *data) { struct iomap_writepage_ctx *wpc = data; struct inode *inode = page->mapping->host; - pgoff_t end_index; u64 end_offset; loff_t offset; - trace_iomap_writepage(inode, page_offset(page), PAGE_SIZE); + VM_BUG_ON_PGFLAGS(PageTail(page), page); + trace_iomap_writepage(inode, page_offset(page), thp_size(page)); /* * Refuse to write the page out if we are called from reclaim context. @@ -1590,10 +1591,8 @@ iomap_do_writepage(struct page *page, struct writeback_control *wbc, void *data) * ---------------------------------^------------------| */ offset = i_size_read(inode); - end_index = offset >> PAGE_SHIFT; - if (page->index < end_index) - end_offset = (loff_t)(page->index + 1) << PAGE_SHIFT; - else { + end_offset = page_offset(page) + thp_size(page); + if (end_offset > offset) { /* * Check whether the page to write out is beyond or straddles * i_size or not. @@ -1605,7 +1604,8 @@ iomap_do_writepage(struct page *page, struct writeback_control *wbc, void *data) * | | Straddles | * ---------------------------------^-----------|--------| */ - unsigned offset_into_page = offset & (PAGE_SIZE - 1); + unsigned offset_into_page = offset_in_thp(page, offset); + pgoff_t end_index = offset >> PAGE_SHIFT; /* * Skip the page if it is fully outside i_size, e.g. due to a @@ -1636,7 +1636,7 @@ iomap_do_writepage(struct page *page, struct writeback_control *wbc, void *data) * memory is zeroed when mapped, and writes to that region are * not written out to the file." */ - zero_user_segment(page, offset_into_page, PAGE_SIZE); + zero_user_segment(page, offset_into_page, thp_size(page)); /* Adjust the end_offset to the end of file */ end_offset = offset; From patchwork Wed Oct 14 03:03:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837183 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC6381130 for ; Wed, 14 Oct 2020 09:23:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B7AF9206F4 for ; Wed, 14 Oct 2020 09:23:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ntibXZzm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730080AbgJNJXQ (ORCPT ); Wed, 14 Oct 2020 05:23:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726694AbgJNJXN (ORCPT ); Wed, 14 Oct 2020 05:23:13 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0C69C0F26F3; Tue, 13 Oct 2020 20:04:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=1e91Asfpb/o4Ps9/wGE3CU5stqf4CIBV6Nn0nrPVRoM=; b=ntibXZzmDHAR1Q5xYvRs9f6j8q V+UK0m3POBUBnFvAU5t3JdQqb6a8qPlC7Coi20lYHYuuo3HKxAgOu/mNQDLA3RuIiUfqNprN8Q9/f ZjFzbOyVtUYuSjaHtT9I8X3K3/BEeTeXFGD97TWytMlkWRr0X09+5inotIdVRbXP/f2vC9qTVjM4s XL8uQYGcqPRul2J8XcjOMhyWg50MZpzcj9hfl6kOEiZAyMXwP1B21DUOBvBHfRu2YPoOCTbPQgHU+ SaIszX8Kxy9TnAE12CDHCW2s9cEiu0obb5NP9Y5isBt0J8iAkAOLnfg8rqf6rakDtlPPXoxeCTO7R tBbbxhwA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX58-0005jO-9K; Wed, 14 Oct 2020 03:04:02 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH 12/14] iomap: Inline data shouldn't see THPs Date: Wed, 14 Oct 2020 04:03:55 +0100 Message-Id: <20201014030357.21898-13-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Assert that we're not seeing THPs in functions that read/write inline data, rather than zeroing out the tail. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- fs/iomap/buffered-io.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 38fd69ebd4cc..38b6b75e7639 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -247,6 +247,7 @@ iomap_read_inline_data(struct inode *inode, struct page *page, return; BUG_ON(page->index); + BUG_ON(PageCompound(page)); BUG_ON(size > PAGE_SIZE - offset_in_page(iomap->inline_data)); addr = kmap_atomic(page); @@ -782,6 +783,7 @@ static size_t iomap_write_end_inline(struct inode *inode, struct page *page, void *addr; WARN_ON_ONCE(!PageUptodate(page)); + BUG_ON(PageCompound(page)); BUG_ON(pos + copied > PAGE_SIZE - offset_in_page(iomap->inline_data)); flush_dcache_page(page); From patchwork Wed Oct 14 03:03:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837177 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 416F915E6 for ; Wed, 14 Oct 2020 09:23:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1123D20BED for ; Wed, 14 Oct 2020 09:23:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Hqnk7nD/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731173AbgJNJXO (ORCPT ); Wed, 14 Oct 2020 05:23:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730736AbgJNJWw (ORCPT ); Wed, 14 Oct 2020 05:22:52 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14B00C0F26F4; Tue, 13 Oct 2020 20:04:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=sU/fZV47iwWzgioCKRXJ5YfzflXRPUbnphxuAtGKKLU=; b=Hqnk7nD/NpsmPqpAxpI/NTmhdZ okkum/QWQxjmat6HRVOilaBBz+DBF43Mm3+Mz99+kL3C9NzzVr5YwWzl8ld9twDHVYXd6fWOefBsD 9t4NoSgsIW09ZHFMrlNbPP2beVE04RB/2P4201oEzGtwFCN0XHIv5MknHziksLxJto/R6dYXAINtD nA+eVJLMJUJ4pIjfUPjz2EIyojC4B8lHtiCkQ1PZjfT1zVpF5qSipBBCNgBbqcr7MIM6wbLK8uTt4 CPGIuBXGY1YND6XlfNGzBjluBLCwrDJvG5l+YrE5n0Dih0YXWdFErsc/Kkr4uPZ1prbmRIrJA3l0+ kjGk8B7g==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX58-0005jV-Ht; Wed, 14 Oct 2020 03:04:02 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 13/14] iomap: Handle tail pages in iomap_page_mkwrite Date: Wed, 14 Oct 2020 04:03:56 +0100 Message-Id: <20201014030357.21898-14-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org iomap_page_mkwrite() can be called with a tail page. If we are, operate on the head page, since we're treating the entire thing as a single unit and the whole page is dirtied at the same time. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 38b6b75e7639..c2540c85f629 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -1098,7 +1098,7 @@ iomap_page_mkwrite_actor(struct inode *inode, loff_t pos, loff_t length, vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops) { - struct page *page = vmf->page; + struct page *page = thp_head(vmf->page); struct inode *inode = file_inode(vmf->vma->vm_file); unsigned long length; loff_t offset; From patchwork Wed Oct 14 03:03:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11837151 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E230B15E6 for ; Wed, 14 Oct 2020 09:22:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC5EC206F4 for ; Wed, 14 Oct 2020 09:22:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="HPvt8zMz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728697AbgJNJWS (ORCPT ); Wed, 14 Oct 2020 05:22:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731148AbgJNJVw (ORCPT ); Wed, 14 Oct 2020 05:21:52 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63EE4C0F26F7; Tue, 13 Oct 2020 20:04:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ucyxaJpq5naE3hSGBf22G2ayK5tvBlA2NbkHxseQKlE=; b=HPvt8zMzCQyt5wKS2Ww7Ob+sZW E7D7PyVYARCgeOSDe8CmkA7OBFwi/FMzoqlYtKRbh/1CfzcOdjDdK7RXr3AomhCJZMmeejOGvTNv9 p3pKYIvE4cIRHLlZPBL6QxwI68394yopB3GtKEqoaITAe6ftDW+ZJJ1MhHF2w/AUPZzWgnytTZQdX xC7H1Yw0zewTn/Xh+FG4Q23u5D6FKuhZrhjjLxs67dRwieGwRgCHco1O8HalCHuniPZGN0ifrgMas KDOMUWhK8h8YGF3+OGu4+zZImILQte7C7e/SKoFD1YXLYd+Go30sw/2aikKIvkNdGfJSz5gJXrp+d pCF1P8nw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSX58-0005jc-Ry; Wed, 14 Oct 2020 03:04:02 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 14/14] xfs: Support THPs Date: Wed, 14 Oct 2020 04:03:57 +0100 Message-Id: <20201014030357.21898-15-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201014030357.21898-1-willy@infradead.org> References: <20201014030357.21898-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org There is one place which assumes the size of a page; fix it. Signed-off-by: Matthew Wilcox (Oracle) --- fs/xfs/xfs_aops.c | 2 +- fs/xfs/xfs_super.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 55d126d4e096..20968842b2f0 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -548,7 +548,7 @@ xfs_discard_page( if (error && !XFS_FORCED_SHUTDOWN(mp)) xfs_alert(mp, "page discard unable to remove delalloc mapping."); out_invalidate: - iomap_invalidatepage(page, 0, PAGE_SIZE); + iomap_invalidatepage(page, 0, thp_size(page)); } static const struct iomap_writeback_ops xfs_writeback_ops = { diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 71ac6c1cdc36..4b6e1cfc57a8 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -1840,7 +1840,7 @@ static struct file_system_type xfs_fs_type = { .init_fs_context = xfs_init_fs_context, .parameters = xfs_fs_parameters, .kill_sb = kill_block_super, - .fs_flags = FS_REQUIRES_DEV, + .fs_flags = FS_REQUIRES_DEV | FS_THP_SUPPORT, }; MODULE_ALIAS_FS("xfs");