From patchwork Tue Mar 12 00:57:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 10848455 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF8B2139A for ; Tue, 12 Mar 2019 00:58:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A792A29401 for ; Tue, 12 Mar 2019 00:58:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A54C429403; Tue, 12 Mar 2019 00:58:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4BA5729414 for ; Tue, 12 Mar 2019 00:58:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=to3pfuCJn/YhJJf1qh4GDYA9m3I+dhXSUxgz8rwEaNg=; b=Axhvq8Ca0AxPMj tg8lx5iN5ge6S6vCKLgtGNgu6IBKB7qsyyImx3zpvQtehq99p08rV3Gocw5LkPfmRUCaSJC2txXgC KVvolaNFZTOz+Y36EGFcVrOTGxhZqjdubcNOV1vHMtM5Un8Kn9mEmgct/QKN8M6bfXRVU/FVDN1T1 oB7JYjZ0BjFZQDs7nfdieZogOtEzdw7lVHDQqEkBV+A4hV+juyHjkpkdtzfhFTcTAMsvccKGiK/Vp 0dKkAXuvigssexiKDMENnFFfx95JbTC7gz0OomlQtzBYE2wR/BL5aKtiBfma6PLKxrRBR/14cfOnV fwzjv7inmDT7z/oHWq8w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h3Vkm-0004ER-1t; Tue, 12 Mar 2019 00:58:48 +0000 Received: from mail-it1-x149.google.com ([2607:f8b0:4864:20::149]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1h3VkA-0003VE-71 for linux-arm-kernel@lists.infradead.org; Tue, 12 Mar 2019 00:58:11 +0000 Received: by mail-it1-x149.google.com with SMTP id 190so762395itl.7 for ; Mon, 11 Mar 2019 17:58:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=GuInaEquY9cKCo+/R3bLL1KFcpfX5BwU8EHfq1hu08M=; b=HBNdWh+/H/UQD3+xTCC5Gk1vTOFdi8buqG7IOzQCbqWRid+PcTT/T4iKPfX1Muy57H mA9gcEpPcKUv1DaJBopn5SU1wwyHnz+qqvQzVQiKuH4IuAZHw8WmPDHRTekZ4RoJuPoI k8fyIUxtuU7UBUmfFhSlmXIqDkwkNUo2QrkJLaPkIgWzkEwroQQwBNFou6cniBzdTUeU pguYiLIcrrFrKPeCm6lJ2yAvRHkaGHnYB9ACSfOKeP/GH7Q1zppaEI6Nk5aSRMS5Mt3C 9GOsSCQ8e1rnKd9uYUOK1sUbi9ceU8L2Dtpr15+FePDmyFg81CK03OWXDcm0/xCUaK9T fuqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=GuInaEquY9cKCo+/R3bLL1KFcpfX5BwU8EHfq1hu08M=; b=sMjUBMaHvYRu27Ehf+TBMFLDD60Ua2E4HlzZBZ/u6Qdvej7Y5ikVY3Vt/9PITI/1YO V4Xi7v2xIP2bxmeN9rDFRX9fyu/tm5vVvXm4f2wfIifELGxwwV8uQ/KYLAlJX6JVr1wc RRExw01AiahpVXIm0qrPPKdY6TECifJWCBJbo1Js/oi09Qf1KaEOa/ioBbOB9EXvCxLm xNj9QLyyfMFHox0ZviMez3Q2ZMkmbrK/ibQoXzVJIuOvw+ngZD4+yokLQ5TcjJjGZFiD 1AteAxpMFAD0xtOAOek/mjeo1BrbqMj30vAjyaidLy7nqX6ra1qJNbWH7SRiE6IFiNH+ RZAw== X-Gm-Message-State: APjAAAW8/8HnzrYBGfZ0A2sN/vDO94kwZ1Cv9lpT6i78LecYpD3nF569 Ygb9I3t2a4kPA8NH2aprjrkAaDCd4Lw= X-Google-Smtp-Source: APXvYqxhul1R8mnVTm2ANYck/U0sQICDmCd8jsCIgw/3U8ahE12sg7YNMdshwxSVxfvSc2y++gz3qAvURMY= X-Received: by 2002:a24:6b55:: with SMTP id v82mr545054itc.37.1552352288828; Mon, 11 Mar 2019 17:58:08 -0700 (PDT) Date: Mon, 11 Mar 2019 18:57:49 -0600 In-Reply-To: <20190312005749.30166-1-yuzhao@google.com> Message-Id: <20190312005749.30166-4-yuzhao@google.com> Mime-Version: 1.0 References: <20190310011906.254635-1-yuzhao@google.com> <20190312005749.30166-1-yuzhao@google.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928-goog Subject: [PATCH v4 4/4] arm64: mm: enable per pmd page table lock From: Yu Zhao To: Catalin Marinas , Will Deacon , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190311_175810_251381_787E84C5 X-CRM114-Status: GOOD ( 14.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Yu Zhao , Ard Biesheuvel , Peter Zijlstra , linux-kernel@vger.kernel.org, Nick Piggin , Jun Yao , linux-mm@kvack.org, "Aneesh Kumar K . V" , Chintan Pandya , Joel Fernandes , "Kirill A . Shutemov" , Andrew Morton , Laura Abbott , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Switch from per mm_struct to per pmd page table lock by enabling ARCH_ENABLE_SPLIT_PMD_PTLOCK. This provides better granularity for large system. I'm not sure if there is contention on mm->page_table_lock. Given the option comes at no cost (apart from initializing more spin locks), why not enable it now. We only do so when pmd is not folded, so we don't mistakenly call pgtable_pmd_page_ctor() on pud or p4d in pgd_pgtable_alloc(). Signed-off-by: Yu Zhao --- arch/arm64/Kconfig | 3 +++ arch/arm64/include/asm/pgalloc.h | 12 +++++++++++- arch/arm64/include/asm/tlb.h | 5 ++++- 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index cfbf307d6dc4..a3b1b789f766 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -872,6 +872,9 @@ config ARCH_WANT_HUGE_PMD_SHARE config ARCH_HAS_CACHE_LINE_SIZE def_bool y +config ARCH_ENABLE_SPLIT_PMD_PTLOCK + def_bool y if PGTABLE_LEVELS > 2 + config SECCOMP bool "Enable seccomp to safely compute untrusted bytecode" ---help--- diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 52fa47c73bf0..dabba4b2c61f 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -33,12 +33,22 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr) { - return (pmd_t *)__get_free_page(PGALLOC_GFP); + struct page *page; + + page = alloc_page(PGALLOC_GFP); + if (!page) + return NULL; + if (!pgtable_pmd_page_ctor(page)) { + __free_page(page); + return NULL; + } + return page_address(page); } static inline void pmd_free(struct mm_struct *mm, pmd_t *pmdp) { BUG_ON((unsigned long)pmdp & (PAGE_SIZE-1)); + pgtable_pmd_page_dtor(virt_to_page(pmdp)); free_page((unsigned long)pmdp); } diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h index 106fdc951b6e..4e3becfed387 100644 --- a/arch/arm64/include/asm/tlb.h +++ b/arch/arm64/include/asm/tlb.h @@ -62,7 +62,10 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp, unsigned long addr) { - tlb_remove_table(tlb, virt_to_page(pmdp)); + struct page *page = virt_to_page(pmdp); + + pgtable_pmd_page_dtor(page); + tlb_remove_table(tlb, page); } #endif