From patchwork Thu Sep 15 11:55:05 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A . Shutemov" X-Patchwork-Id: 9333435 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AE3C56089F for ; Thu, 15 Sep 2016 12:08:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9BDEB2969E for ; Thu, 15 Sep 2016 12:08:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 90DA3296A3; Thu, 15 Sep 2016 12:08:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA9B52969E for ; Thu, 15 Sep 2016 12:08:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934633AbcIOMIS (ORCPT ); Thu, 15 Sep 2016 08:08:18 -0400 Received: from mga03.intel.com ([134.134.136.65]:19304 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755773AbcIOLzp (ORCPT ); Thu, 15 Sep 2016 07:55:45 -0400 Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga103.jf.intel.com with ESMTP; 15 Sep 2016 04:55:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.30,339,1470726000"; d="scan'208";a="761104996" Received: from black.fi.intel.com ([10.237.72.56]) by FMSMGA003.fm.intel.com with ESMTP; 15 Sep 2016 04:55:40 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 140BE980; Thu, 15 Sep 2016 14:55:27 +0300 (EEST) From: "Kirill A. Shutemov" To: "Theodore Ts'o" , Andreas Dilger , Jan Kara , Andrew Morton Cc: Alexander Viro , Hugh Dickins , Andrea Arcangeli , Dave Hansen , Vlastimil Babka , Matthew Wilcox , Ross Zwisler , linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv3 23/41] fs: make block_read_full_page() be able to read huge page Date: Thu, 15 Sep 2016 14:55:05 +0300 Message-Id: <20160915115523.29737-24-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20160915115523.29737-1-kirill.shutemov@linux.intel.com> References: <20160915115523.29737-1-kirill.shutemov@linux.intel.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The approach is straight-forward: for compound pages we read out whole huge page. For huge page we cannot have array of buffer head pointers on stack -- it's 4096 pointers on x86-64 -- 'arr' is allocated with kmalloc() for huge pages. Signed-off-by: Kirill A. Shutemov --- fs/buffer.c | 22 +++++++++++++++++----- include/linux/buffer_head.h | 9 +++++---- include/linux/page-flags.h | 2 +- 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 9c8eb9b6db6a..2739f5dae690 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -870,7 +870,7 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, try_again: head = NULL; - offset = PAGE_SIZE; + offset = hpage_size(page); while ((offset -= size) >= 0) { bh = alloc_buffer_head(GFP_NOFS); if (!bh) @@ -1466,7 +1466,7 @@ void set_bh_page(struct buffer_head *bh, struct page *page, unsigned long offset) { bh->b_page = page; - BUG_ON(offset >= PAGE_SIZE); + BUG_ON(offset >= hpage_size(page)); if (PageHighMem(page)) /* * This catches illegal uses and preserves the offset: @@ -2239,11 +2239,13 @@ int block_read_full_page(struct page *page, get_block_t *get_block) { struct inode *inode = page->mapping->host; sector_t iblock, lblock; - struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; + struct buffer_head *arr_on_stack[MAX_BUF_PER_PAGE]; + struct buffer_head *bh, *head, **arr = arr_on_stack; unsigned int blocksize, bbits; int nr, i; int fully_mapped = 1; + VM_BUG_ON_PAGE(PageTail(page), page); head = create_page_buffers(page, inode, 0); blocksize = head->b_size; bbits = block_size_bits(blocksize); @@ -2254,6 +2256,11 @@ int block_read_full_page(struct page *page, get_block_t *get_block) nr = 0; i = 0; + if (PageTransHuge(page)) { + arr = kmalloc(sizeof(struct buffer_head *) * HPAGE_PMD_NR * + MAX_BUF_PER_PAGE, GFP_NOFS); + } + do { if (buffer_uptodate(bh)) continue; @@ -2269,7 +2276,9 @@ int block_read_full_page(struct page *page, get_block_t *get_block) SetPageError(page); } if (!buffer_mapped(bh)) { - zero_user(page, i * blocksize, blocksize); + zero_user(page + (i * blocksize / PAGE_SIZE), + i * blocksize % PAGE_SIZE, + blocksize); if (!err) set_buffer_uptodate(bh); continue; @@ -2295,7 +2304,7 @@ int block_read_full_page(struct page *page, get_block_t *get_block) if (!PageError(page)) SetPageUptodate(page); unlock_page(page); - return 0; + goto out; } /* Stage two: lock the buffers */ @@ -2317,6 +2326,9 @@ int block_read_full_page(struct page *page, get_block_t *get_block) else submit_bh(REQ_OP_READ, 0, bh); } +out: + if (arr != arr_on_stack) + kfree(arr); return 0; } EXPORT_SYMBOL(block_read_full_page); diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 006a8a42acfb..194a85822d5f 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -131,13 +131,14 @@ BUFFER_FNS(Meta, meta) BUFFER_FNS(Prio, prio) BUFFER_FNS(Defer_Completion, defer_completion) -#define bh_offset(bh) ((unsigned long)(bh)->b_data & ~PAGE_MASK) +#define bh_offset(bh) ((unsigned long)(bh)->b_data & ~hpage_mask(bh->b_page)) /* If we *know* page->private refers to buffer_heads */ -#define page_buffers(page) \ +#define page_buffers(__page) \ ({ \ - BUG_ON(!PagePrivate(page)); \ - ((struct buffer_head *)page_private(page)); \ + struct page *p = compound_head(__page); \ + BUG_ON(!PagePrivate(p)); \ + ((struct buffer_head *)page_private(p)); \ }) #define page_has_buffers(page) PagePrivate(page) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index a2bef9a41bcf..20b7684e9298 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -730,7 +730,7 @@ static inline void ClearPageSlabPfmemalloc(struct page *page) */ static inline int page_has_private(struct page *page) { - return !!(page->flags & PAGE_FLAGS_PRIVATE); + return !!(compound_head(page)->flags & PAGE_FLAGS_PRIVATE); } #undef PF_ANY