From patchwork Mon Oct 31 21:58:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 13026401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27C38ECAAA1 for ; Mon, 31 Oct 2022 22:04:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5F006B0073; Mon, 31 Oct 2022 18:04:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A35256B0074; Mon, 31 Oct 2022 18:04:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AFA16B0075; Mon, 31 Oct 2022 18:04:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7DE9E6B0073 for ; Mon, 31 Oct 2022 18:04:46 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4C9DF80DCC for ; Mon, 31 Oct 2022 22:04:46 +0000 (UTC) X-FDA: 80082624972.22.E226F91 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf15.hostedemail.com (Postfix) with ESMTP id CF444A0015 for ; Mon, 31 Oct 2022 22:04:45 +0000 (UTC) Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29VLsknf031992 for ; Mon, 31 Oct 2022 15:04:45 -0700 Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3kjfyrmm88-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 31 Oct 2022 15:04:44 -0700 Received: from snc-exhub201.TheFacebook.com (2620:10d:c085:21d::7) by snc-exhub104.TheFacebook.com (2620:10d:c085:11d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Mon, 31 Oct 2022 15:04:42 -0700 Received: from twshared21592.39.frc1.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Mon, 31 Oct 2022 15:04:42 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 55E8CF0C8867; Mon, 31 Oct 2022 14:58:37 -0700 (PDT) From: Song Liu To: , CC: , , , , , , , , , Song Liu Subject: [PATCH bpf-next v1 0/5] vmalloc_exec for modules and BPF programs Date: Mon, 31 Oct 2022 14:58:29 -0700 Message-ID: <20221031215834.1615596-1-song@kernel.org> X-Mailer: git-send-email 2.30.2 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: BYPeCLmzmDfcWBxpRQ-h8w583tdb90ch X-Proofpoint-GUID: BYPeCLmzmDfcWBxpRQ-h8w583tdb90ch X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-31_21,2022-10-31_01,2022-06-22_01 ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of "prvs=2303bf8861=songliubraving@meta.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=2303bf8861=songliubraving@meta.com"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667253886; a=rsa-sha256; cv=none; b=KPzvyol8ttcO5seU1sGXqnJiB3tE36stbOk/XGv+ZeAnr9hkLjSCgPhWxpRyzqSTFicQ5V xGiqdWUSlfPOPlcuTv1LGlwTJQJrujNMmvUEB7XAA/bWZe583zRj65GtW1ancOfuyjbs3n OaDAE7YkefLboL/argZI4Pv35p+0Umg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667253885; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references; bh=w6JraVneCbKK1ahmiTJj/SN6ohfOlgloiLZHnLMhwqU=; b=6zPOdFDNTXgFRqgy7yo54D2APa/whFqFS47xbhzBofkit/J9RAaGYIuPdfjo70XqL6e+PQ qPO9MiHcejr9A4dsuq6tZxcIPWQAk/xn9LOaErHdt52HlwMOiAL3Wt9TpYKDoPOtmSgiEx uAZACuuMFCRt4K72tvJAhO7hzG2KsIc= X-Rspam-User: Authentication-Results: imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of "prvs=2303bf8861=songliubraving@meta.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=2303bf8861=songliubraving@meta.com"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) X-Rspamd-Server: rspam07 X-Stat-Signature: u433poayzh7pndqygky8gzp536tednu5 X-Rspamd-Queue-Id: CF444A0015 X-HE-Tag: 1667253885-791298 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This set enables bpf programs and bpf dispatchers to share huge pages with new API: vmalloc_exec() vfree_exec() vcopy_exec() The idea is similar to Peter's suggestion in [1]. vmalloc_exec() manages a set of PMD_SIZE RO+X memory, and allocates these memory to its users. vfree_exec() is used to free memory allocated by vmalloc_exec(). vcopy_exec() is used to update memory allocated by vmalloc_exec(). Memory allocated by vmalloc_exec() is RO+X, so this doesnot violate W^X. The caller has to update the content with text_poke like mechanism. Specifically, vcopy_exec() is provided to update memory allocated by vmalloc_exec(). vcopy_exec() also makes sure the update stays in the boundary of one chunk allocated by vmalloc_exec(). Please refer to patch 1/5 for more details of Patch 3/5 uses these new APIs in bpf program and bpf dispatcher. Patch 4/5 and 5/5 allows static kernel text (_stext to _etext) to share PMD_SIZE pages with dynamic kernel text on x86_64. This is achieved by allocating PMD_SIZE pages to roundup(_etext, PMD_SIZE), and then use _etext to roundup(_etext, PMD_SIZE) for dynamic kernel text. [1] https://lore.kernel.org/bpf/Ys6cWUMHO8XwyYgr@hirez.programming.kicks-ass.net/ [2] RFC v1: https://lore.kernel.org/linux-mm/20220818224218.2399791-3-song@kernel.org/T/ Changes RFC v2 => PATCH v1: 1. Add vcopy_exec(), which updates memory allocated by vmalloc_exec(). It also ensures vcopy_exec() is only used to update memory from one single vmalloc_exec() call. (Christoph Hellwig) 2. Add arch_vcopy_exec() and arch_invalidate_exec() as wrapper for the text_poke() like logic. 3. Drop changes for kernel modules and focus on BPF side changes. Changes RFC v1 => RFC v2: 1. Major rewrite of the logic of vmalloc_exec and vfree_exec. They now work fine with BPF programs (patch 1, 2, 4). But module side (patch 3) still need some work. Song Liu (5): vmalloc: introduce vmalloc_exec, vfree_exec, and vcopy_exec x86/alternative: support vmalloc_exec() and vfree_exec() bpf: use vmalloc_exec for bpf program and bpf dispatcher vmalloc: introduce register_text_tail_vm() x86: use register_text_tail_vm arch/x86/include/asm/pgtable_64_types.h | 1 + arch/x86/kernel/alternative.c | 12 + arch/x86/mm/init_64.c | 4 +- arch/x86/net/bpf_jit_comp.c | 23 +- include/linux/bpf.h | 3 - include/linux/filter.h | 5 - include/linux/vmalloc.h | 9 + kernel/bpf/core.c | 180 +----------- kernel/bpf/dispatcher.c | 11 +- mm/nommu.c | 7 + mm/vmalloc.c | 351 ++++++++++++++++++++++++ 11 files changed, 404 insertions(+), 202 deletions(-) --- 2.30.2