From patchwork Mon Jun 5 12:51:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Punit Agrawal X-Patchwork-Id: 9766193 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9B76560353 for ; Mon, 5 Jun 2017 12:52:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 854C11FF60 for ; Mon, 5 Jun 2017 12:52:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 79B6D27D4D; Mon, 5 Jun 2017 12:52:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BA7441FF60 for ; Mon, 5 Jun 2017 12:52:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=K9rZ1WBGRfoFYdS/lnvaqOKY5OOEubb9tOATw57o97M=; b=SDc6tsI5RMzd53xZ0OlJ/AGrel eTsqpCOKBiHnHhEV3rU8/gH/1dztUQnE0c/PWbAqhtPN25Zzr5VogwZFAg3dZtzv8deIhZ7jvFYDq H4Cw4ajVd51ixRClyZVcnxDWPRWIstyJqNNGY6upCLJtA5m0T107kI3EDfbfchTAtDgIZRVMIfEpn op21Ic0SMe5mlwC/dHBRiN4EIW+/P0jzlUO3A84Ucm59AIPh1PsskTISdPS8JIULTqaCGiJia1e12 7e/gL8WAyBRJJ0eDNnQq0jicHP5s/RbPtdJQ1WCl65HD68NdsJJgmwXZEK3hDq6hMsNWCOSwCbipx c+hPdTsQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1dHrUb-00012B-EK; Mon, 05 Jun 2017 12:52:21 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1dHrUX-0000v0-7F for linux-arm-kernel@lists.infradead.org; Mon, 05 Jun 2017 12:52:19 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E72C52B; Mon, 5 Jun 2017 05:51:55 -0700 (PDT) Received: from localhost (e105922-lin.cambridge.arm.com [10.1.207.56]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B494D3F578; Mon, 5 Jun 2017 05:51:55 -0700 (PDT) From: Punit Agrawal To: akpm@linux-foundation.org Subject: [PATCH v4.1 4/8] mm, gup: Ensure real head page is ref-counted when using hugepages Date: Mon, 5 Jun 2017 13:51:12 +0100 Message-Id: <20170605125112.19530-1-punit.agrawal@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170524115409.31309-5-punit.agrawal@arm.com> References: <20170524115409.31309-5-punit.agrawal@arm.com> X-ARM-No-Footer: FoSSMail X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170605_055217_278868_C98401A4 X-CRM114-Status: GOOD ( 11.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, linux-arch@vger.kernel.org, Michal Hocko , steve.capper@arm.com, aneesh.kumar@linux.vnet.ibm.com, catalin.marinas@arm.com, Punit Agrawal , will.deacon@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, ynorov@caviumnetworks.com, kirill.shutemov@linux.intel.com, n-horiguchi@ah.jp.nec.com, linux-arm-kernel@lists.infradead.org, mike.kravetz@oracle.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP When speculatively taking references to a hugepage using page_cache_add_speculative() in gup_huge_pmd(), it is assumed that the page returned by pmd_page() is the head page. Although normally true, this assumption doesn't hold when the hugepage comprises of successive page table entries such as when using contiguous bit on arm64 at PTE or PMD levels. This can be addressed by ensuring that the page passed to page_cache_add_speculative() is the real head or by de-referencing the head page within the function. We take the first approach to keep the usage pattern aligned with page_cache_get_speculative() where users already pass the appropriate page, i.e., the de-referenced head. Apply the same logic to fix gup_huge_[pud|pgd]() as well. Signed-off-by: Punit Agrawal Cc: Steve Capper Cc: Michal Hocko Cc: "Kirill A. Shutemov" Cc: Aneesh Kumar K.V --- Hi Andrew, Please update the patch in your queue with this version. It fixes the ltp failures reported by Yury[0]. The original patch led to incorrect ref-count on certain pages due to taking a referencing on the following page in some instances. Should be fixed with this version. Thanks, Punit [0] http://lists.infradead.org/pipermail/linux-arm-kernel/2017-June/510318.html mm/gup.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index e74e0b5a0c7c..6bd39264d0e7 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1354,8 +1354,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, return __gup_device_huge_pmd(orig, addr, end, pages, nr); refs = 0; - head = pmd_page(orig); - page = head + ((addr & ~PMD_MASK) >> PAGE_SHIFT); + page = pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1363,6 +1362,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pmd_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0; @@ -1392,8 +1392,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, return __gup_device_huge_pud(orig, addr, end, pages, nr); refs = 0; - head = pud_page(orig); - page = head + ((addr & ~PUD_MASK) >> PAGE_SHIFT); + page = pud_page(orig) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1401,6 +1400,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pud_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0; @@ -1429,8 +1429,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, BUILD_BUG_ON(pgd_devmap(orig)); refs = 0; - head = pgd_page(orig); - page = head + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); + page = pgd_page(orig) + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1438,6 +1437,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pgd_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0;