From patchwork Sun Apr 25 14:12:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12223329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56953C433B4 for ; Sun, 25 Apr 2021 14:16:03 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DAA5560232 for ; Sun, 25 Apr 2021 14:16:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DAA5560232 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2PgQEJRroA0A0/Pqo+XTB+Spqx/1mPyompvNcUBOHhk=; b=nchC3lPdVG3a/eR0byP2MbVv1 bNZiaf5XW7LhecWZZ5mkoUPdgWqlVqwocl+mni+CvwObhNeAdfFQHCrfanKuUXeT/9wJaH4V2ptqP WgkItrTQxCePa/QvIyq90UeXgcHFHB7oDVePLuNiQEYXApfgEhNtMsz10qLIgHlkg/Bw53f/9hK00 R0WO69N4r1Fy8wdXe7gz0YyfTjPmAhMBAAVyMUXufCGJNTyXTaorjurSKZmeFsuWjraGUmuT34B03 oTBXcELgE3Wacjf1XqLIE49A8bAdmxF4XOKtOAraJ2ZL8mJfcBbeMud77tunZtM39xKtQZQbl95IX Ud+ho69iw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lafWc-005iKq-Dv; Sun, 25 Apr 2021 14:14:20 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lafWF-005iHG-JN for linux-arm-kernel@desiato.infradead.org; Sun, 25 Apr 2021 14:13:55 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Rq5MdgbFwq5YnoQQmOm0iH5gEV1BaGEp7qhwvG8Ibyo=; b=aOVPcQEuWNh0y4zyDVkI8md18v OvniJbXmFMO7SPlgDOWcwp8P5W7X1wIv7DVKjjZQNjRhLbOC7+YNPvyflAK3gAhgnUan/mZzQAFsF hZCqx65RECjIEXx0ECwvtqjvvVCpawjGA2bqJ5GPcJo7JThusbA9VDU25bhuDh5ZGnAoV/RyZz29U F3IosrBNunpSbXYGxiWurU6Dk6NiBYmQsNGDWklfgDl0yGa1cfSi1e32mnn4NUcdgLUCuITTCrxSO YNslbkrPKqlApUVlAyS8dIyBLYy+vg+Pmk1KH+xe3g3AR9z5Sdfqpy1H+CMwZSP0c2GFrsENrIUz2 joJxwA7A==; Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lafWD-00FT5M-05 for linux-arm-kernel@lists.infradead.org; Sun, 25 Apr 2021 14:13:54 +0000 Received: by mail-pl1-x62f.google.com with SMTP id y1so11907399plg.11 for ; Sun, 25 Apr 2021 07:13:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Rq5MdgbFwq5YnoQQmOm0iH5gEV1BaGEp7qhwvG8Ibyo=; b=mV0xuKIeEI6j+1MXq1J6oreOpwP7pc5dWy2qVaG2QHH9RuVFsbKit3jHuna5FP7Vsa /hgxzFC/e/cQIzpjwYuKWG5HEaNbLoQWR+ttUNCokZlELF7D1fnBBmkyB77xJ8qtzOGA UOMOOLejM3eRGvrRaNjNQuQnWOxacsHgw8+jQD1zXhC3ANUkRcfOe3ji2hdw3O4cOZ0o EqAgrM4I227LvIeR1Ybb4Hw40Xzgo1d+uyh+YEHClIaKHeu2XM62ZDwuhCvo6P6PmhOS td3tKOu17vDHeLDsZHtTejK5sZQVhTqkOhKHu+EfsBAFgsMdEMFXCMBmNPkKZlsI2OZu ntSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Rq5MdgbFwq5YnoQQmOm0iH5gEV1BaGEp7qhwvG8Ibyo=; b=WEYlmTFNr4ZqJgp3IT57rKE2BcbmVnwAenWajY8WuMAH/EhGpqKZg/0Yc9sYarWbWJ Ch0FNF7BbJa22iuju24g6RBNQv18lYLbzVwCTHPC4Z+XTxf7HTzR43zgPzmi74N76SN8 oV61zS6LEmCilmIudCSGuYzsFdZS3gMo5Ebqfn17QxT95DlvoYy38zQxqldmZZIpEHda aR1MHSIlNz7EvQCjFbbO0DKx8vdE4gZCx6Jzm3NbfeSnvHKoESPRWDoiNXAmiUnxb1vX qek8NVntAc1Wb6wZ/Y6HUcYk5vA/QxpSYaMhZtaW20wLwVitPzAfsS0IT8bzxObPZSKe Okjw== X-Gm-Message-State: AOAM530vTEzEyWbSMJXGzeimj8nQdhJwhV7TT5dx5FcsHZj6DyPlmCEb +yeoJjV2+4Gn8uUNoZq/WbhVZl1obw== X-Google-Smtp-Source: ABdhPJwTeWXmjB8ZBQi5cLL8QK5euu4MxeHQtTtWnaYJ1mc2au6QXlfgc7+DDNNORNyTxlLbrBPeBA== X-Received: by 2002:a17:902:9008:b029:e6:f37a:2183 with SMTP id a8-20020a1709029008b02900e6f37a2183mr13662623plp.49.1619360030341; Sun, 25 Apr 2021 07:13:50 -0700 (PDT) Received: from x1pad.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id r32sm1007139pgm.49.2021.04.25.07.13.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Apr 2021 07:13:50 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Catalin Marinas , Will Deacon , Marc Zyngier , Kristina Martsenko , James Morse , Steven Price , Jonathan Cameron , Pavel Tatashin , Anshuman Khandual , Atish Patra , Mike Rapoport , Logan Gunthorpe , Mark Brown Subject: [PATCHv2 04/10] arm64/mm: enable __create_pgd_mapping() to run across different pgtable Date: Sun, 25 Apr 2021 22:12:58 +0800 Message-Id: <20210425141304.32721-5-kernelfans@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210425141304.32721-1-kernelfans@gmail.com> References: <20210425141304.32721-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210425_071353_062712_5E8B3F86 X-CRM114-Status: GOOD ( 12.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __create_pgd_mapping() is planned not only to be used in boot process, but also after fully bootup. The latter means several callers can run __create_pgd_mapping() concurrenlty. In this case, PUP/PMD/PTE fixmap should be not used, instead, the virtual addresss owned by each pgtable is used. Signed-off-by: Pingfan Liu Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier Cc: Kristina Martsenko Cc: James Morse Cc: Steven Price Cc: Jonathan Cameron Cc: Pavel Tatashin Cc: Anshuman Khandual Cc: Atish Patra Cc: Mike Rapoport Cc: Logan Gunthorpe Cc: Mark Brown To: linux-arm-kernel@lists.infradead.org --- rfc -> v2: change allocator return type back to phys_addr_t --- arch/arm64/mm/mmu_include.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/arch/arm64/mm/mmu_include.c b/arch/arm64/mm/mmu_include.c index 732e603fe3fc..ac8850fe6ce2 100644 --- a/arch/arm64/mm/mmu_include.c +++ b/arch/arm64/mm/mmu_include.c @@ -7,6 +7,7 @@ #define NO_BLOCK_MAPPINGS BIT(0) #define NO_CONT_MAPPINGS BIT(1) +#define NO_FIXMAP BIT(2) static bool pgattr_change_is_safe(u64 old, u64 new) { @@ -43,11 +44,14 @@ static bool pgattr_change_is_safe(u64 old, u64 new) } static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, - phys_addr_t phys, pgprot_t prot) + phys_addr_t phys, pgprot_t prot, int flags) { pte_t *ptep; - ptep = pte_set_fixmap_offset(pmdp, addr); + if (likely(!(flags & NO_FIXMAP))) + ptep = pte_set_fixmap_offset(pmdp, addr); + else + ptep = pte_offset_kernel(pmdp, addr); do { pte_t old_pte = READ_ONCE(*ptep); @@ -63,7 +67,8 @@ static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys += PAGE_SIZE; } while (ptep++, addr += PAGE_SIZE, addr != end); - pte_clear_fixmap(); + if (likely(!(flags & NO_FIXMAP))) + pte_clear_fixmap(); } static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, @@ -97,7 +102,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, (flags & NO_CONT_MAPPINGS) == 0) __prot = __pgprot(pgprot_val(prot) | PTE_CONT); - init_pte(pmdp, addr, next, phys, __prot); + init_pte(pmdp, addr, next, phys, __prot, flags); phys += next - addr; } while (addr = next, addr != end); @@ -112,7 +117,10 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, unsigned long next; pmd_t *pmdp; - pmdp = pmd_set_fixmap_offset(pudp, addr); + if (likely(!(flags & NO_FIXMAP))) + pmdp = pmd_set_fixmap_offset(pudp, addr); + else + pmdp = pmd_offset(pudp, addr); do { pmd_t old_pmd = READ_ONCE(*pmdp); @@ -139,7 +147,8 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys += next - addr; } while (pmdp++, addr = next, addr != end); - pmd_clear_fixmap(); + if (likely(!(flags & NO_FIXMAP))) + pmd_clear_fixmap(); } static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, @@ -215,7 +224,10 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, } BUG_ON(p4d_bad(p4d)); - pudp = pud_set_fixmap_offset(p4dp, addr); + if (likely(!(flags & NO_FIXMAP))) + pudp = pud_set_fixmap_offset(p4dp, addr); + else + pudp = pud_offset(p4dp, addr); do { pud_t old_pud = READ_ONCE(*pudp); @@ -244,7 +256,8 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, phys += next - addr; } while (pudp++, addr = next, addr != end); - pud_clear_fixmap(); + if (likely(!(flags & NO_FIXMAP))) + pud_clear_fixmap(); } static void __create_pgd_mapping(pgd_t *pgdir, unsigned int entries_cnt, phys_addr_t phys,