From patchwork Mon Nov 27 08:46:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 13469266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F40BC4167B for ; Mon, 27 Nov 2023 08:47:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E33256B0329; Mon, 27 Nov 2023 03:47:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DE2C86B032A; Mon, 27 Nov 2023 03:47:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5BA06B032B; Mon, 27 Nov 2023 03:47:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B1DFC6B0329 for ; Mon, 27 Nov 2023 03:47:32 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 897DAB525F for ; Mon, 27 Nov 2023 08:47:32 +0000 (UTC) X-FDA: 81503105544.19.D14B987 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf22.hostedemail.com (Postfix) with ESMTP id 9CF4BC0023 for ; Mon, 27 Nov 2023 08:47:30 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=g2lKJOFQ; spf=pass (imf22.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701074850; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CXU92qYOzo6aL9X8rG2XQuljOMNrDRKtELmWJz0kLI0=; b=F1tOaoFbP8vdzfbXRFyo/fB1upEEs1V/KUUN92zaewrvXX+O/V/uFQILP9OrxQtW+/JaUB 4sjHHPVyTfLyxTVKUpXA2HhyTxQgb1Md5+do4Ab2gYv2dZRGxn8PSf+1Rh+eJD2we51BFF AhrNR3xhnbVuMli2GnosIOQ1RLHjlKg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701074850; a=rsa-sha256; cv=none; b=xGqj0lbyntT8f0DPk2/PC8ukNc7lCqPoUaE5tpHhzV70Rr5RKuwdFEDFVPYlVOLUtTtLb2 KbPsNXB/LUNfb6DEk9/Jt0s8N+8UWp7r5K5S+FeSjUuymGV0mI1Q9xDWhnL0m1Hzjax8lh pxURlvzS9/PCqvkq6FklLfQA3utwKl8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=g2lKJOFQ; spf=pass (imf22.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-6cb66fbc63dso2699969b3a.0 for ; Mon, 27 Nov 2023 00:47:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1701074849; x=1701679649; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CXU92qYOzo6aL9X8rG2XQuljOMNrDRKtELmWJz0kLI0=; b=g2lKJOFQN0gRis8Wxsgi235SYDIb7g5qr45jJNfoFJ+QpmUq8aJEZ5xMI1iSELvBHH ma+TDNCArDggqEvaaiuMg1JalsIz0ihF5c07XJxjkGW9BtkRSjFXbkn+hmPRL7rbEH7o li6vPEF9VL0quXxm0LAKjFr4O6y8d2SDImcnC1Sub8OZs6WyUuW7EfJyWV9k9YSSMHKE tkGXlzk14gkgypQTnvL3lEHLMRaWV/yNHKZLc65HkN8xGaj2hgIt0/0Y7vqEAHMbn4kk LAh9aMC6o09PfiDk8q9oA4KkyTAwpVV6g3LqlIc8kGLisfTr8qGLM1QHNMW7RzR58fNJ czwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701074849; x=1701679649; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CXU92qYOzo6aL9X8rG2XQuljOMNrDRKtELmWJz0kLI0=; b=aCrTWQCnB7Ms7g7eSKD5VaESQTB92bwYAq5zhf08lhrgr/IZDKuULEnfOq9xlVv6c+ DQfr9xYPduHuvFrvVrVXdFoS8npZojIuSAMAah2HxoT4EitK4XeratNXlhwp8+n/SmAy hDYVBlPrUtU9W5M9z70v/09SN4W/qbGgz40QqwMiJ+4ofV+D1qtIBBQ02JXTFAA1aCTd LqnTJ5nPowrmzF/mPUJqICxhaOtYAzW6mTApbhS1bK8F2dAKO1tGOI+L3B2WC/HSDlIy E4yFjlj25djtvJl1IW+zGVYh0KkqGSod1cjhdvnktS5t2e6Zuyu1+uldDBF7asSdmMWZ RM4w== X-Gm-Message-State: AOJu0Yx4npiV9It06p+5jGm1udvqsMWabnPzEJ+RA5kGTzUppM6RfoDG 5wxCuqo1xfk+YIHJy5gSDIYnIA== X-Google-Smtp-Source: AGHT+IHXNermaEvQJyziObFn7H4znAJ2WmRDi/l1vX6FLLvpxZ6tTRgBpFuCL+ANq0CYCQTGl1dHCg== X-Received: by 2002:a05:6a00:878c:b0:6cb:cc23:f69f with SMTP id hk12-20020a056a00878c00b006cbcc23f69fmr18033800pfb.16.1701074849478; Mon, 27 Nov 2023 00:47:29 -0800 (PST) Received: from PXLDJ45XCM.bytedance.net ([139.177.225.230]) by smtp.gmail.com with ESMTPSA id e22-20020aa78c56000000b006c875abecbcsm6686932pfd.121.2023.11.27.00.47.26 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 27 Nov 2023 00:47:28 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, muchun.song@linux.dev, akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH 3/4] mm: hugetlb_vmemmap: move PageVmemmapSelfHosted() check to split_vmemmap_huge_pmd() Date: Mon, 27 Nov 2023 16:46:44 +0800 Message-Id: <20231127084645.27017-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231127084645.27017-1-songmuchun@bytedance.com> References: <20231127084645.27017-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: 8x4dtq73xp7uo3ygw963agopar1syxbo X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 9CF4BC0023 X-Rspam-User: X-HE-Tag: 1701074850-200350 X-HE-Meta: U2FsdGVkX1+Onc2JEMPO5Xy4gHimWSHtGeQl0oFuPvmy9QQOecoRgiJkzx1YtvF+fM3xISPMKDObDE/O5ty/4QwSIwmrlOA4bahTkOXmgYoK7oDMau6Smura2tcuncOTssaqk/Fu3E+hr2RLySXDHf+SVyZE8AM9HAgcEsi2hirvcoQMnonRJ72UIFT6FgmsZo7sdQStJE8ai0W6jTg7APSFa0kz5mo9slteHAZxF5+oA9RvBGIIXBg2k5CSaMOt+q6rWya+Ci7CZSRy2KfKx2kwUjnzuq8ySWqzPeOxiuz8l1kehC+305N0IKIN8uJtweFee7ZrjJ3LQA8Z8XqNm56LIekWksxd0nwlalz8CFReZKqeSKKcysDykJNeIZcIP5zDPFyqZQCuGl23CddVlv7kZjTpd839oZ87WjKHBgt+fPmL3xEsMgE5LRRK13UptPWk4mADpnypF9x81x9V+wPujt4BwYZhufziIZw2Lerc5I/D9diLVTLQ5N1ip2hVKg1ypevCKgi0PjqbglLvsDrqce73IB7KaPmnX92TL8U5qNytNZynjDiKfFHZFz0Zr36jg/+eRRrKRgQejAuzfRy+I+fGFsySeqB4/nlJ81MROenNrhF5YUcKFkuQnHnqK0083+76d4p+2VOTuumL/AdmC5ftRU7ZkiunUB5bStz9N2UK2ormGM8ag9QvE8SK4Phfk+CsBw9uD6iT70h/9n1N+F57AkGt6xJcO5iqUHT3RdxF+FSHsVgIaX4jw4WW5P75WOQ1j64973WIVqVCidy0rlLxpCXhwXsJO9AcZLHYJSen23uCbYDVyr5T1NqqDciT8jX4Q8mafaFx4PuvqYR0HxO5UQk6qs0wghoHnEnIkjlu0aX3xV89Zg4P7yYL6jC1gn1/mZLfkZBlZO4KLvytYcEU0Sva5swQDFZi5DRPFcwsttjFCCIjDSCjG480HwVqSFLS41WyIO+ISlU YUMEKVOO YWUumsMBYGpsV3D1O9PRN6gE4+Y6mRhzpXBdzjnaGbihEt+vH+0ftJW/8LzsvnnXIdsyotGg+87oKLxIO/tGW6pYWpQGihkpcdKw8DPdN6TO1E/JbozPD/xy2Og+gOV8sJQ4jcmJvQERUmlpzTu3kbqubkOpfDhk6/dLKFaJS/q21Ug/RLjr8hwXJiO6fucklVS5q/s5gL/b4YQODDjdP17w51xNX5bK+hkWLTE2Wlqda/xTKjAOlBVJyFNUr3U/qYVrLsM+9ofj/KqKnpwqYPbpZh2bolYzTP0iBolyLUJyMT4bKKBlO1QkBPT1AHB1aArZ3XlcFbLl8kSfh7HkzQxE2G89BbtFNBG/Oymo0svdZtpVAMDFQS7ZeqAmImwXfgkGn00g7aeBy/fGtTZPIHeyDcHa5n371+D3ktOJlDBiWU3RJOuEQKUofJWYrBH4BELK0d8wKhSsHSuE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To check a page whether it is self-hosted needs to traverse the page table (e.g. pmd_off_k()), however, we already have done this in the next calling of vmemmap_remap_range(). Moving PageVmemmapSelfHosted() check to vmemmap_pmd_entry() could simplify the code a bit. Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/hugetlb_vmemmap.c | 70 +++++++++++++++----------------------------- 1 file changed, 24 insertions(+), 46 deletions(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index ef14356855d13..ce920ca6c90ee 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -95,6 +95,7 @@ static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start, static int vmemmap_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long next, struct mm_walk *walk) { + int ret = 0; struct page *head; struct vmemmap_remap_walk *vmemmap_walk = walk->private; @@ -104,9 +105,30 @@ static int vmemmap_pmd_entry(pmd_t *pmd, unsigned long addr, spin_lock(&init_mm.page_table_lock); head = pmd_leaf(*pmd) ? pmd_page(*pmd) : NULL; + /* + * Due to HugeTLB alignment requirements and the vmemmap + * pages being at the start of the hotplugged memory + * region in memory_hotplug.memmap_on_memory case. Checking + * the vmemmap page associated with the first vmemmap page + * if it is self-hosted is sufficient. + * + * [ hotplugged memory ] + * [ section ][...][ section ] + * [ vmemmap ][ usable memory ] + * ^ | ^ | + * +--+ | | + * +------------------------+ + */ + if (unlikely(!vmemmap_walk->nr_walked)) { + struct page *page = head ? head + pte_index(addr) : + pte_page(ptep_get(pte_offset_kernel(pmd, addr))); + + if (PageVmemmapSelfHosted(page)) + ret = -ENOTSUPP; + } spin_unlock(&init_mm.page_table_lock); - if (!head) - return 0; + if (!head || ret) + return ret; return vmemmap_split_pmd(pmd, head, addr & PMD_MASK, vmemmap_walk); } @@ -524,50 +546,6 @@ static bool vmemmap_should_optimize(const struct hstate *h, const struct page *h if (!hugetlb_vmemmap_optimizable(h)) return false; - if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG)) { - pmd_t *pmdp, pmd; - struct page *vmemmap_page; - unsigned long vaddr = (unsigned long)head; - - /* - * Only the vmemmap page's vmemmap page can be self-hosted. - * Walking the page tables to find the backing page of the - * vmemmap page. - */ - pmdp = pmd_off_k(vaddr); - /* - * The READ_ONCE() is used to stabilize *pmdp in a register or - * on the stack so that it will stop changing under the code. - * The only concurrent operation where it can be changed is - * split_vmemmap_huge_pmd() (*pmdp will be stable after this - * operation). - */ - pmd = READ_ONCE(*pmdp); - if (pmd_leaf(pmd)) - vmemmap_page = pmd_page(pmd) + pte_index(vaddr); - else - vmemmap_page = pte_page(*pte_offset_kernel(pmdp, vaddr)); - /* - * Due to HugeTLB alignment requirements and the vmemmap pages - * being at the start of the hotplugged memory region in - * memory_hotplug.memmap_on_memory case. Checking any vmemmap - * page's vmemmap page if it is marked as VmemmapSelfHosted is - * sufficient. - * - * [ hotplugged memory ] - * [ section ][...][ section ] - * [ vmemmap ][ usable memory ] - * ^ | | | - * +---+ | | - * ^ | | - * +-------+ | - * ^ | - * +-------------------------------------------+ - */ - if (PageVmemmapSelfHosted(vmemmap_page)) - return false; - } - return true; }