From patchwork Wed Jul 25 16:18:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 10544281 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F5F817FD for ; Wed, 25 Jul 2018 16:19:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F1F272A409 for ; Wed, 25 Jul 2018 16:19:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E44DC2A5BD; Wed, 25 Jul 2018 16:19:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3338B2A409 for ; Wed, 25 Jul 2018 16:19:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B9E46B0006; Wed, 25 Jul 2018 12:19:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 069C66B0007; Wed, 25 Jul 2018 12:19:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4C806B0008; Wed, 25 Jul 2018 12:19:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 7EC576B0006 for ; Wed, 25 Jul 2018 12:19:21 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id b25-v6so3281940eds.17 for ; Wed, 25 Jul 2018 09:19:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=+fCFlF4eolsGb/kFDiQghaeCwGM1mxZGgO0SyG4u4Mg=; b=Mdoq2ZD9NjO109J5hah9WQF1VvCr+YRx0JE+PWmW7O+vPobFcaKgeSw4C9rghQOn5E LbNmTO5sYXet3hPKVsp/EeqCa3lAgNpGBr0l7Vif8yZSVT+mX0SkB1NpT2DSiEY/OwiY bSGW4O/WQSW/SY/myuNl+PSzePeGCSbOT0nQBwZSMFz4n4lq9UXRDBgQscHckVOvEG72 q31RKT0uVejH+/HY/WDwf3YgxV2KuAd0xajB/1P/yqnuV0wBz8+CYYqByUagKamwSe/S gxQyUED5oBhAh7J9zUmzO8mzm/Skwc/0D0maVX2K4Zs4pdIl64PI/XrWIusIrdj/CqNH W4fg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com X-Gm-Message-State: AOUpUlEsRZ6JayRlnHTHvfjAULHVCCZNwEeCqky2qUXpkklYx9CiYool cTHzfwIYf12ZUujaLrPOhf+2h+3HuEJxn4cMs95ZhQZ+yciB8LzbpIS4TuGehYMU6qYd+JY8sv2 ADa03BgYDpa7uesf495RV1xbJfKNtTaqhFrhoSC2QW1ysj68veLTEkOxpEtHvum0B7g== X-Received: by 2002:a50:9226:: with SMTP id i35-v6mr23306917eda.27.1532535561026; Wed, 25 Jul 2018 09:19:21 -0700 (PDT) X-Google-Smtp-Source: AAOMgpft3x5iAUE/EGyUHF1zHwT8v6nBwUNU9NumaZaPkPU2OHOX08PW/32ENZZnDV3tNosSpgOB X-Received: by 2002:a50:9226:: with SMTP id i35-v6mr23306857eda.27.1532535559908; Wed, 25 Jul 2018 09:19:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1532535559; cv=none; d=google.com; s=arc-20160816; b=oa5BT8P06g1Coq6/i3wlwzgyp5tXqvEYLkfhnl5/Sj8CTwth5h28pyLOl+oG2lUsQF LLkgmdauyxbJ0i7fGgxsgN0ea/rdHcWWM5HyHNGkRvxQoBVh94XL6fVmEZf5R7XBDCwt sY0jEYe54CpiKU0Z0czjclkrWYQz4UmxXk99iQGHEMhvhb5w4KTq2kMqu8a36HmcCTyC +jCr5mdQL6BA7LypOQKIhTeXi2+x7+BZliokYy+Ccy0MZgSc/UOwp0pc/1aw2+RRLtwv dTkQx517djBAbQwCeR0+w7SweFlIQDL/wEay2ROEcUgJQ4Mss9ss3Iob2PlaPkpoeZGR +PHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:arc-authentication-results; bh=+fCFlF4eolsGb/kFDiQghaeCwGM1mxZGgO0SyG4u4Mg=; b=thHyu9OtY3cYQXWH5JL2lPKRpM2fLpUK04tGmvCNRAv34lj8AMTSZcD1vPLbe7sJP5 aMpK9PzZe1VS0kkO7B3eE4MeJan2kZEAYdB0dRu6KwLNf2Ff4Ar3Ca9ruJ2wHsRnYklt Jgubsh2gMvSF+R1UwitHvM6haiSfFUzIKvXZUhs2UkIS8M5DVJ/uC4qDanatpdeBypzs B9lPRuXwBuWtsIToSL+0f9FGpRZXAj08Vl5LSA7pvhvVTi7do2psGlst1JBwoMYciW1W WWjwI3bIyFYoOt72pqImavb7HD7UmPpirSptAttfdfbvTvTZs7Jur2p4EuT2fde7GVIa oIxg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com. [148.163.158.5]) by mx.google.com with ESMTPS id n22-v6si36569edo.298.2018.07.25.09.19.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jul 2018 09:19:19 -0700 (PDT) Received-SPF: pass (google.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) client-ip=148.163.158.5; Authentication-Results: mx.google.com; spf=pass (google.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w6PGFio8037500 for ; Wed, 25 Jul 2018 12:19:18 -0400 Received: from e13.ny.us.ibm.com (e13.ny.us.ibm.com [129.33.205.203]) by mx0b-001b2d01.pphosted.com with ESMTP id 2keuhsk46t-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 25 Jul 2018 12:19:17 -0400 Received: from localhost by e13.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 25 Jul 2018 12:19:16 -0400 Received: from b01cxnp23033.gho.pok.ibm.com (9.57.198.28) by e13.ny.us.ibm.com (146.89.104.200) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 25 Jul 2018 12:19:12 -0400 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w6PGJBGv14418242 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 25 Jul 2018 16:19:12 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CB387AE05C; Wed, 25 Jul 2018 12:18:26 -0400 (EDT) Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 01B73AE063; Wed, 25 Jul 2018 12:18:23 -0400 (EDT) Received: from skywalker.ibmmodules.com (unknown [9.199.35.55]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 25 Jul 2018 12:18:23 -0400 (EDT) From: "Aneesh Kumar K.V" To: npiggin@gmail.com, benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au, Naoya Horiguchi Cc: linuxppc-dev@lists.ozlabs.org, linux-mm@kvack.org, "Aneesh Kumar K.V" Subject: [PATCH V2 1/6] powerpc/mm/book3s: Update pmd_present to look at _PAGE_PRESENT bit Date: Wed, 25 Jul 2018 21:48:58 +0530 X-Mailer: git-send-email 2.17.1 X-TM-AS-GCONF: 00 x-cbid: 18072516-0064-0000-0000-0000032F86F3 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009425; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000266; SDB=6.01065988; UDB=6.00547631; IPR=6.00843848; MB=3.00022321; MTD=3.00000008; XFM=3.00000015; UTC=2018-07-25 16:19:15 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18072516-0065-0000-0000-00003A114C57 Message-Id: <20180725161903.31257-1-aneesh.kumar@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-07-25_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807250174 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With this patch we use 0x8000000000000000UL (_PAGE_PRESENT) to indicate a valid pgd/pud/pmd entry. We also switch the p**_present() to look at this bit. With pmd_present, we have a special case. We need to make sure we consider a pmd marked invalid during THP split as present. Right now we clear the _PAGE_PRESENT bit during a pmdp_invalidate. Inorder to consider this special case we add a new pte bit _PAGE_INVALID (mapped to _RPAGE_SW0). This bit is only used with _PAGE_PRESENT cleared. Hence we are not really losing a pte bit for this special case. pmd_present is also updated to look at _PAGE_INVALID. Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/include/asm/book3s/64/hash.h | 5 +++++ arch/powerpc/include/asm/book3s/64/pgtable.h | 23 +++++++++++++++++--- arch/powerpc/mm/hash_utils_64.c | 6 ++--- arch/powerpc/mm/pgtable-book3s64.c | 2 +- 4 files changed, 29 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/hash.h b/arch/powerpc/include/asm/book3s/64/hash.h index 0387b155f13d..a371ac7c3183 100644 --- a/arch/powerpc/include/asm/book3s/64/hash.h +++ b/arch/powerpc/include/asm/book3s/64/hash.h @@ -16,6 +16,11 @@ #include #endif +/* Bits to set in a PMD/PUD/PGD entry valid bit*/ +#define HASH_PMD_VAL_BITS (0x8000000000000000UL) +#define HASH_PUD_VAL_BITS (0x8000000000000000UL) +#define HASH_PGD_VAL_BITS (0x8000000000000000UL) + /* * Size of EA range mapped by our pagetables. */ diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 676118743a06..fce9ce8781a0 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -44,6 +44,15 @@ #define _PAGE_PTE 0x4000000000000000UL /* distinguishes PTEs from pointers */ #define _PAGE_PRESENT 0x8000000000000000UL /* pte contains a translation */ +/* + * We need to mark a pmd pte invalid while splitting. We can do that by clearing the + * _PAGE_PRESENT bit. But then that will be taken as a swap pte. Inorder to differentiate + * between two use a SW field when invalidating. We don't add a special bit to indicate + * swap pte because that is also used for migration ptes, and we do back and forth between + * a valid pte entry and migration ptes. So any information in the software bits will be + * lost if we overload those bits. + */ +#define _PAGE_INVALID _RPAGE_SW0 /* * Top and bottom bits of RPN which can be used by hash @@ -859,8 +868,16 @@ static inline int pmd_none(pmd_t pmd) static inline int pmd_present(pmd_t pmd) { + /* + * A pmd is considerent present if _PAGE_PRESENT is set. + * We also need to consider the pmd present which is marked + * invalid during a split. Hence we look for _PAGE_INVALID + * if we find _PAGE_PRESENT cleared. + */ + if (pmd_raw(pmd) & cpu_to_be64(_PAGE_PRESENT | _PAGE_INVALID)) + return true; - return !pmd_none(pmd); + return false; } static inline int pmd_bad(pmd_t pmd) @@ -887,7 +904,7 @@ static inline int pud_none(pud_t pud) static inline int pud_present(pud_t pud) { - return !pud_none(pud); + return (pud_raw(pud) & cpu_to_be64(_PAGE_PRESENT)); } extern struct page *pud_page(pud_t pud); @@ -934,7 +951,7 @@ static inline int pgd_none(pgd_t pgd) static inline int pgd_present(pgd_t pgd) { - return !pgd_none(pgd); + return (pgd_raw(pgd) & cpu_to_be64(_PAGE_PRESENT)); } static inline pte_t pgd_pte(pgd_t pgd) diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c index 5a72e980e25a..7ce7fa5397d5 100644 --- a/arch/powerpc/mm/hash_utils_64.c +++ b/arch/powerpc/mm/hash_utils_64.c @@ -1002,9 +1002,9 @@ void __init hash__early_init_mmu(void) * 4k use hugepd format, so for hash set then to * zero */ - __pmd_val_bits = 0; - __pud_val_bits = 0; - __pgd_val_bits = 0; + __pmd_val_bits = HASH_PMD_VAL_BITS; + __pud_val_bits = HASH_PUD_VAL_BITS; + __pgd_val_bits = HASH_PGD_VAL_BITS; __kernel_virt_start = H_KERN_VIRT_START; __kernel_virt_size = H_KERN_VIRT_SIZE; diff --git a/arch/powerpc/mm/pgtable-book3s64.c b/arch/powerpc/mm/pgtable-book3s64.c index 4afbfbb64bfd..24346ab4cd37 100644 --- a/arch/powerpc/mm/pgtable-book3s64.c +++ b/arch/powerpc/mm/pgtable-book3s64.c @@ -106,7 +106,7 @@ pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, { unsigned long old_pmd; - old_pmd = pmd_hugepage_update(vma->vm_mm, address, pmdp, _PAGE_PRESENT, 0); + old_pmd = pmd_hugepage_update(vma->vm_mm, address, pmdp, _PAGE_PRESENT, _PAGE_INVALID); flush_pmd_tlb_range(vma, address, address + HPAGE_PMD_SIZE); /* * This ensures that generic code that rely on IRQ disabling