From patchwork Tue Nov 29 11:22:45 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A . Shutemov" X-Patchwork-Id: 9451677 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9A25F60710 for ; Tue, 29 Nov 2016 11:33:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7F09C281A7 for ; Tue, 29 Nov 2016 11:33:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7128E281C3; Tue, 29 Nov 2016 11:33:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B4E6A281A7 for ; Tue, 29 Nov 2016 11:33:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757512AbcK2Lbp (ORCPT ); Tue, 29 Nov 2016 06:31:45 -0500 Received: from mga09.intel.com ([134.134.136.24]:51634 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757250AbcK2LXm (ORCPT ); Tue, 29 Nov 2016 06:23:42 -0500 Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga102.jf.intel.com with ESMTP; 29 Nov 2016 03:23:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,568,1473145200"; d="scan'208";a="10982365" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga002.jf.intel.com with ESMTP; 29 Nov 2016 03:23:31 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id C01CC929; Tue, 29 Nov 2016 13:23:12 +0200 (EET) From: "Kirill A. Shutemov" To: "Theodore Ts'o" , Andreas Dilger , Jan Kara , Andrew Morton Cc: Alexander Viro , Hugh Dickins , Andrea Arcangeli , Dave Hansen , Vlastimil Babka , Matthew Wilcox , Ross Zwisler , linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv5 17/36] fs: make block_read_full_page() be able to read huge page Date: Tue, 29 Nov 2016 14:22:45 +0300 Message-Id: <20161129112304.90056-18-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.10.2 In-Reply-To: <20161129112304.90056-1-kirill.shutemov@linux.intel.com> References: <20161129112304.90056-1-kirill.shutemov@linux.intel.com> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The approach is straight-forward: for compound pages we read out whole huge page. For huge page we cannot have array of buffer head pointers on stack -- it's 4096 pointers on x86-64 -- 'arr' is allocated with kmalloc() for huge pages. Signed-off-by: Kirill A. Shutemov --- fs/buffer.c | 22 +++++++++++++++++----- include/linux/buffer_head.h | 9 +++++---- include/linux/page-flags.h | 2 +- 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index d21771fcf7d3..090f7edfa6b7 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -871,7 +871,7 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, try_again: head = NULL; - offset = PAGE_SIZE; + offset = hpage_size(page); while ((offset -= size) >= 0) { bh = alloc_buffer_head(GFP_NOFS); if (!bh) @@ -1466,7 +1466,7 @@ void set_bh_page(struct buffer_head *bh, struct page *page, unsigned long offset) { bh->b_page = page; - BUG_ON(offset >= PAGE_SIZE); + BUG_ON(offset >= hpage_size(page)); if (PageHighMem(page)) /* * This catches illegal uses and preserves the offset: @@ -2280,11 +2280,13 @@ int block_read_full_page(struct page *page, get_block_t *get_block) { struct inode *inode = page->mapping->host; sector_t iblock, lblock; - struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; + struct buffer_head *arr_on_stack[MAX_BUF_PER_PAGE]; + struct buffer_head *bh, *head, **arr = arr_on_stack; unsigned int blocksize, bbits; int nr, i; int fully_mapped = 1; + VM_BUG_ON_PAGE(PageTail(page), page); head = create_page_buffers(page, inode, 0); blocksize = head->b_size; bbits = block_size_bits(blocksize); @@ -2295,6 +2297,11 @@ int block_read_full_page(struct page *page, get_block_t *get_block) nr = 0; i = 0; + if (PageTransHuge(page)) { + arr = kmalloc(sizeof(struct buffer_head *) * HPAGE_PMD_NR * + MAX_BUF_PER_PAGE, GFP_NOFS); + } + do { if (buffer_uptodate(bh)) continue; @@ -2310,7 +2317,9 @@ int block_read_full_page(struct page *page, get_block_t *get_block) SetPageError(page); } if (!buffer_mapped(bh)) { - zero_user(page, i * blocksize, blocksize); + zero_user(page + (i * blocksize / PAGE_SIZE), + i * blocksize % PAGE_SIZE, + blocksize); if (!err) set_buffer_uptodate(bh); continue; @@ -2336,7 +2345,7 @@ int block_read_full_page(struct page *page, get_block_t *get_block) if (!PageError(page)) SetPageUptodate(page); unlock_page(page); - return 0; + goto out; } /* Stage two: lock the buffers */ @@ -2358,6 +2367,9 @@ int block_read_full_page(struct page *page, get_block_t *get_block) else submit_bh(REQ_OP_READ, 0, bh); } +out: + if (arr != arr_on_stack) + kfree(arr); return 0; } EXPORT_SYMBOL(block_read_full_page); diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index fd4134ce9c54..f12f6293ed44 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -131,13 +131,14 @@ BUFFER_FNS(Meta, meta) BUFFER_FNS(Prio, prio) BUFFER_FNS(Defer_Completion, defer_completion) -#define bh_offset(bh) ((unsigned long)(bh)->b_data & ~PAGE_MASK) +#define bh_offset(bh) ((unsigned long)(bh)->b_data & ~hpage_mask(bh->b_page)) /* If we *know* page->private refers to buffer_heads */ -#define page_buffers(page) \ +#define page_buffers(__page) \ ({ \ - BUG_ON(!PagePrivate(page)); \ - ((struct buffer_head *)page_private(page)); \ + struct page *p = compound_head(__page); \ + BUG_ON(!PagePrivate(p)); \ + ((struct buffer_head *)page_private(p)); \ }) #define page_has_buffers(page) PagePrivate(page) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index a2bef9a41bcf..20b7684e9298 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -730,7 +730,7 @@ static inline void ClearPageSlabPfmemalloc(struct page *page) */ static inline int page_has_private(struct page *page) { - return !!(page->flags & PAGE_FLAGS_PRIVATE); + return !!(compound_head(page)->flags & PAGE_FLAGS_PRIVATE); } #undef PF_ANY