From patchwork Mon Oct 31 22:25:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 13026424 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7371FA3744 for ; Mon, 31 Oct 2022 22:26:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E98E6B007D; Mon, 31 Oct 2022 18:26:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D4B56B0081; Mon, 31 Oct 2022 18:26:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25D9B6B0080; Mon, 31 Oct 2022 18:26:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 189586B007D for ; Mon, 31 Oct 2022 18:26:10 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DAA6F1C67B7 for ; Mon, 31 Oct 2022 22:26:09 +0000 (UTC) X-FDA: 80082678858.02.3C5C8DF Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf03.hostedemail.com (Postfix) with ESMTP id 8AA1F20021 for ; Mon, 31 Oct 2022 22:26:09 +0000 (UTC) Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29VMPLWL010654 for ; Mon, 31 Oct 2022 15:26:08 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3kh20w3kps-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 31 Oct 2022 15:26:08 -0700 Received: from twshared16963.27.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Mon, 31 Oct 2022 15:26:06 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id EE8A8F0CC5CC; Mon, 31 Oct 2022 15:25:56 -0700 (PDT) From: Song Liu To: , CC: , , , , , , , Song Liu Subject: [PATCH bpf-next v1 RESEND 5/5] x86: use register_text_tail_vm Date: Mon, 31 Oct 2022 15:25:41 -0700 Message-ID: <20221031222541.1773452-6-song@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221031222541.1773452-1-song@kernel.org> References: <20221031222541.1773452-1-song@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: kkdeh4EFLjSITqI1OkFr7pJHc8YjMLfW X-Proofpoint-ORIG-GUID: kkdeh4EFLjSITqI1OkFr7pJHc8YjMLfW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-31_21,2022-10-31_01,2022-06-22_01 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667255169; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=omE+zG+vjYJGN4/mSCchK3i4EgV63TFghVHMLql9mng=; b=Y2pIkgNIC6iPQuJ1L3LTeoNuKYUecFj390f32d7tH/CAXeYbNAqR+WBOelwpUA7ioWYC9/ mJ0y+n4tK75JaORGEtHoIUaXH7qXF2LkI/i2xPRlt05Sp9yM4/zTMa54ZO8d6JwRySSB13 02vzS7oHcS2c1J/knu/Ua4p+gqtYQSc= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of "prvs=2303bf8861=songliubraving@meta.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=2303bf8861=songliubraving@meta.com"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667255169; a=rsa-sha256; cv=none; b=fTToeJAjBm55tQJN+GxOv+n/M60sBt2lMNXBiN7t5AK558tSkskwHiXDqWp/1EwRRzq/Xn OSzDo6lhJgoblF9ZvzlPKv+U5tLUK/GeB0EUJuVLSgcGu7/m2k+AgnMTidvp1n8gq235/8 +ojCd20DdIr+xVpw4RDdnc70YpzQYZY= X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 8AA1F20021 Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of "prvs=2303bf8861=songliubraving@meta.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=2303bf8861=songliubraving@meta.com"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) X-Stat-Signature: k4nouw1kbwyxmph9wb7pof98kxj3aikc X-HE-Tag: 1667255169-647802 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allocate 2MB pages up to round_up(_etext, 2MB), and register memory [round_up(_etext, 4kb), round_up(_etext, 2MB)] with register_text_tail_vm so that we can use this part of memory for dynamic kernel text (BPF programs, etc.). Here is an example: [root@eth50-1 ~]# grep _etext /proc/kallsyms ffffffff82202a08 T _etext [root@eth50-1 ~]# grep bpf_prog_ /proc/kallsyms | tail -n 3 ffffffff8220f920 t bpf_prog_cc61a5364ac11d93_handle__sched_wakeup [bpf] ffffffff8220fa28 t bpf_prog_cc61a5364ac11d93_handle__sched_wakeup_new [bpf] ffffffff8220fad4 t bpf_prog_3bf73fa16f5e3d92_handle__sched_switch [bpf] [root@eth50-1 ~]# grep 0xffffffff82200000 /sys/kernel/debug/page_tables/kernel 0xffffffff82200000-0xffffffff82400000 2M ro PSE x pmd ffffffff82200000-ffffffff82400000 is a 2MB page, serving kernel text, and bpf programs. Signed-off-by: Song Liu --- arch/x86/include/asm/pgtable_64_types.h | 1 + arch/x86/mm/init_64.c | 4 +++- include/linux/vmalloc.h | 4 ++++ 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h index 04f36063ad54..c0f9cceb109a 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -101,6 +101,7 @@ extern unsigned int ptrs_per_p4d; #define PUD_MASK (~(PUD_SIZE - 1)) #define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT) #define PGDIR_MASK (~(PGDIR_SIZE - 1)) +#define PMD_ALIGN(x) (((unsigned long)(x) + (PMD_SIZE - 1)) & PMD_MASK) /* * See Documentation/x86/x86_64/mm.rst for a description of the memory map. diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 3f040c6e5d13..5b42fc0c6099 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1373,7 +1373,7 @@ void mark_rodata_ro(void) unsigned long start = PFN_ALIGN(_text); unsigned long rodata_start = PFN_ALIGN(__start_rodata); unsigned long end = (unsigned long)__end_rodata_hpage_align; - unsigned long text_end = PFN_ALIGN(_etext); + unsigned long text_end = PMD_ALIGN(_etext); unsigned long rodata_end = PFN_ALIGN(__end_rodata); unsigned long all_end; @@ -1414,6 +1414,8 @@ void mark_rodata_ro(void) (void *)rodata_end, (void *)_sdata); debug_checkwx(); + register_text_tail_vm(PFN_ALIGN((unsigned long)_etext), + PMD_ALIGN((unsigned long)_etext)); } int kern_addr_valid(unsigned long addr) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 9b2042313c12..7365cf9c4e7f 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -132,11 +132,15 @@ extern void vm_unmap_aliases(void); #ifdef CONFIG_MMU extern void __init vmalloc_init(void); extern unsigned long vmalloc_nr_pages(void); +void register_text_tail_vm(unsigned long start, unsigned long end); #else static inline void vmalloc_init(void) { } static inline unsigned long vmalloc_nr_pages(void) { return 0; } +void register_text_tail_vm(unsigned long start, unsigned long end) +{ +} #endif extern void *vmalloc(unsigned long size) __alloc_size(1);