From patchwork Mon Jun 19 17:01:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Punit Agrawal X-Patchwork-Id: 9797051 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 390BC600C5 for ; Mon, 19 Jun 2017 17:35:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0F2581FE95 for ; Mon, 19 Jun 2017 17:35:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 03CF826B39; Mon, 19 Jun 2017 17:35:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AB89D1FE95 for ; Mon, 19 Jun 2017 17:35:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=onBJ++q4eLMWj6LkDJCE1Gdtoq/ZHp6s8ZccaLtN97c=; b=dU0caxFatZvXU5K7k/6iig9PfL AaCz1dYXa1D7019biXDx+Eu9O84ZMUT/j47WWftM2bpE+Sw447G/GAbeTzNnEimHPBk7kPeQX/dYs 0zjbnM4NjzOEg2d/sq3obiTmYQqHt4/DOadvacS2eWoEgQjqE9+qQUzEc5J+BAkGEemuVdFwZ94nk 5XU+uEYAAhSj5ZspYP7Q5hZ3qKR9rSzKN0NKvx/Q7uc4h58thhkWYgwyvj9NFqXWyl1D+Pxjat7rN Law+EEo49S1bLenKS5xl4gAp+MhBGsjKDTyrKzu2sItzYEB5MmdEnnErSjwa03KP1KIAzkiWXxCnI VHO4LioA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1dN0aV-0006Vy-QA; Mon, 19 Jun 2017 17:35:43 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1dN0LM-000637-PZ for linux-arm-kernel@bombadil.infradead.org; Mon, 19 Jun 2017 17:20:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+8pGhwUYFufQAXJx0ALLGmKsVrX4vn3K/lBh596VtBE=; b=B409f9F8U2m0jrRkMLGeZ1A5U /nqS4z212DouCJxIKQI9pvH7fhPlgL8chqNOg/DURPrIxKwjDevF1kqs/NswoNYEDFG1z0WdS7vza Ywv0QOJ3KdwrtidQbmttZnVAOytxveGjnfPAjhlmlMlf2mGsl69ryO5GGMvtIxw4PcyAswnMXdShR VlRjEeKFRJvWV4G6k625v8XRhym/FMFUH/UbnbG5CS/cpH8l39tZIkbLiDbq00+sKFZ4AtoWo/pAS GN6nvXfi5mLg5UFjg0HNBDzcR+mZopflDWAU4RFxmHJ2XBWBL/d3+Dkr/lASXqObUR35ec4q4x55B /Jt+NVqaA==; Received: from foss.arm.com ([217.140.101.70]) by merlin.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1dN04s-0002YG-EH for linux-arm-kernel@lists.infradead.org; Mon, 19 Jun 2017 17:03:04 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2E99E15AD; Mon, 19 Jun 2017 10:02:41 -0700 (PDT) Received: from localhost (e105922-lin.cambridge.arm.com [10.1.207.56]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F03A33F41F; Mon, 19 Jun 2017 10:02:40 -0700 (PDT) From: Punit Agrawal To: akpm@linux-foundation.org Subject: [PATCH v5 4/8] mm, gup: Ensure real head page is ref-counted when using hugepages Date: Mon, 19 Jun 2017 18:01:41 +0100 Message-Id: <20170619170145.25577-5-punit.agrawal@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170619170145.25577-1-punit.agrawal@arm.com> References: <20170619170145.25577-1-punit.agrawal@arm.com> X-ARM-No-Footer: FoSSMail X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, linux-arch@vger.kernel.org, Michal Hocko , steve.capper@arm.com, aneesh.kumar@linux.vnet.ibm.com, catalin.marinas@arm.com, Punit Agrawal , will.deacon@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kirill.shutemov@linux.intel.com, n-horiguchi@ah.jp.nec.com, linux-arm-kernel@lists.infradead.org, mike.kravetz@oracle.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP When speculatively taking references to a hugepage using page_cache_add_speculative() in gup_huge_pmd(), it is assumed that the page returned by pmd_page() is the head page. Although normally true, this assumption doesn't hold when the hugepage comprises of successive page table entries such as when using contiguous bit on arm64 at PTE or PMD levels. This can be addressed by ensuring that the page passed to page_cache_add_speculative() is the real head or by de-referencing the head page within the function. We take the first approach to keep the usage pattern aligned with page_cache_get_speculative() where users already pass the appropriate page, i.e., the de-referenced head. Apply the same logic to fix gup_huge_[pud|pgd]() as well. Signed-off-by: Punit Agrawal Cc: Steve Capper Cc: Michal Hocko Cc: "Kirill A. Shutemov" Cc: Aneesh Kumar K.V --- mm/gup.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index e74e0b5a0c7c..6bd39264d0e7 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1354,8 +1354,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, return __gup_device_huge_pmd(orig, addr, end, pages, nr); refs = 0; - head = pmd_page(orig); - page = head + ((addr & ~PMD_MASK) >> PAGE_SHIFT); + page = pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1363,6 +1362,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pmd_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0; @@ -1392,8 +1392,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, return __gup_device_huge_pud(orig, addr, end, pages, nr); refs = 0; - head = pud_page(orig); - page = head + ((addr & ~PUD_MASK) >> PAGE_SHIFT); + page = pud_page(orig) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1401,6 +1400,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pud_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0; @@ -1429,8 +1429,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, BUILD_BUG_ON(pgd_devmap(orig)); refs = 0; - head = pgd_page(orig); - page = head + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); + page = pgd_page(orig) + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1438,6 +1437,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pgd_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0;