From patchwork Thu Jan 25 13:31:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13530822 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A6D5C47258 for ; Thu, 25 Jan 2024 13:32:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Et9CviQAj6+1NtQJJsQCHbP6lE9dQ8zdz+bOVHzmN/k=; b=0vUVtaPcWrL7T8 JM5Q8ULdeVfLaevm/mj3CN6o7P8fqV3R811kY44XcW2d1Qc4H7mPIn2rn2LS8LMa/gfyApfBLBYJp dQYlx92k2UXC2NIxF58UDiVsAm4D+Uf2vYWE7eWT6OqKp7LzglX+Z2HC5MgBQC2/LzTqZJXPMV06K bd4JauWPiopkn/v0utPJ8NvLnZHSK1yFz/vYyU5wyIo62eDB1FO1xCKIez+7+xuSTzqHsfXTBIvmC 34Kt0zwW7SP4SsKs8RmUgHzV1eLHIsBbSquasH9Pdt6Ww8thjDQuT8lGtEUrOszxEwPd+xIQjXTGJ a+Yg05Kv3/NUpkWk9g3Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rSzqO-000000003BO-0B7E; Thu, 25 Jan 2024 13:32:36 +0000 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rSzqF-0000000038A-3to8 for linux-arm-kernel@lists.infradead.org; Thu, 25 Jan 2024 13:32:29 +0000 Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-339208f5105so5584700f8f.1 for ; Thu, 25 Jan 2024 05:32:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706189546; x=1706794346; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FI4++Tri0XwJHgi/SwI3tIuIhoPqQHm+YqaSOpS3kwM=; b=kXtLRPmUcZKrjdNE5nsd1Z5iVIMH7Q2uIaUH/HB9rciO8vgt6vMA9W68twkGET6sZ9 ecm+0l9BMzXR0s/Wg8jRG2zXZvu/6mTUpUbipgMvnkiUlarvlBrF2tHzUPYD8eSFtT1v d8F/LDhGhPTtnh+QcwV3ZcIsLCjeHnOqtLed4cnZYGh5UZT2ip/emQHPk0OpL9oe8Wua Qt9V55Tnl6oglbK0qQKPfinKy9zE4ad+2XiXZDv8K/rEoj8L7HMHRZkEKuDMJ1rlgOi/ CkkCzPTsml1xFLFyUYET5WP3OmmoQHXPXiFfYaIOPGvEoN8RMUwdUtEjqQs+otqwuKbQ oUlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706189546; x=1706794346; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FI4++Tri0XwJHgi/SwI3tIuIhoPqQHm+YqaSOpS3kwM=; b=s1LYypkC+quKq8XT3CcAqvAxJKXGM3H+W01SQSHI+RUb/9qyFwwU5Tv4KUbl0uIDQe dx1tYmjt4Q9IoaeQzsGbRLl4M0edVcWlLTyW3e2mlw1Sx24bxzquJZBR2a0lsKaMETZA qdGA4g+rWvrwjKllU1dwfzv8WRhHyHc6yVhPzKcpbK+Wb/HvRurZ8/JGLSeQxgQkVOzP eZUGQs1TE0wzaT0z6Qu4SHlKxvhwPO1wrtC9MyL+V10N8Q6hdiCZ4zgNfZWDaqDdLhfX IIb0K1hfV9jKvHKKlLQr8XKKgSyA8th3s2Ova21LgngU40CE+AHUbPCCYh7FCYd0N6hM G5sg== X-Gm-Message-State: AOJu0YzUFb5+n+63wUGGKN7RkaR3Y2vbS8zJ/rl9AoIR73RraEF6hr6w 6HcSRqmOwup3GX1qfyNLL0IWHEoD++FBGhaH7ZTSeBblnq0yacPA X-Google-Smtp-Source: AGHT+IHQIZJsJdAU+2qb2txKeOEYR7L1CR8S/iMgbYTmFiK9T+jtJK46ukwppWquTcpNEz5Z/6XRtw== X-Received: by 2002:a05:6000:1757:b0:336:6920:d766 with SMTP id m23-20020a056000175700b003366920d766mr339879wrf.93.1706189546543; Thu, 25 Jan 2024 05:32:26 -0800 (PST) Received: from localhost (54-240-197-231.amazon.com. [54.240.197.231]) by smtp.gmail.com with ESMTPSA id t4-20020a5d42c4000000b003392c3141absm12418488wrr.1.2024.01.25.05.32.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Jan 2024 05:32:26 -0800 (PST) From: Puranjay Mohan To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, bpf@vger.kernel.org, kpsingh@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, xukuohai@huaweicloud.com Cc: puranjay12@gmail.com Subject: [PATCH bpf-next v7 1/2] arm64: patching: implement text_poke API Date: Thu, 25 Jan 2024 13:31:58 +0000 Message-Id: <20240125133159.85086-2-puranjay12@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240125133159.85086-1-puranjay12@gmail.com> References: <20240125133159.85086-1-puranjay12@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240125_053227_991267_E34EA8C0 X-CRM114-Status: GOOD ( 17.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The text_poke API is used to implement functions like memcpy() and memset() for instruction memory (RO+X). The implementation is similar to the x86 version. This will be used by the BPF JIT to write and modify BPF programs. There could be more users of this in the future. Signed-off-by: Puranjay Mohan --- arch/arm64/include/asm/patching.h | 2 + arch/arm64/kernel/patching.c | 80 +++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+) diff --git a/arch/arm64/include/asm/patching.h b/arch/arm64/include/asm/patching.h index 68908b82b168..587bdb91ab7a 100644 --- a/arch/arm64/include/asm/patching.h +++ b/arch/arm64/include/asm/patching.h @@ -8,6 +8,8 @@ int aarch64_insn_read(void *addr, u32 *insnp); int aarch64_insn_write(void *addr, u32 insn); int aarch64_insn_write_literal_u64(void *addr, u64 val); +void *aarch64_insn_set(void *dst, u32 insn, size_t len); +void *aarch64_insn_copy(void *dst, void *src, size_t len); int aarch64_insn_patch_text_nosync(void *addr, u32 insn); int aarch64_insn_patch_text(void *addrs[], u32 insns[], int cnt); diff --git a/arch/arm64/kernel/patching.c b/arch/arm64/kernel/patching.c index b4835f6d594b..5c2d34d890cf 100644 --- a/arch/arm64/kernel/patching.c +++ b/arch/arm64/kernel/patching.c @@ -105,6 +105,86 @@ noinstr int aarch64_insn_write_literal_u64(void *addr, u64 val) return ret; } +typedef void text_poke_f(void *dst, void *src, size_t patched, size_t len); + +static void *__text_poke(text_poke_f func, void *addr, void *src, size_t len) +{ + unsigned long flags; + size_t patched = 0; + size_t size; + void *waddr; + void *ptr; + int ret; + + raw_spin_lock_irqsave(&patch_lock, flags); + + while (patched < len) { + ptr = addr + patched; + size = min_t(size_t, PAGE_SIZE - offset_in_page(ptr), + len - patched); + + waddr = patch_map(ptr, FIX_TEXT_POKE0); + func(waddr, src, patched, size); + patch_unmap(FIX_TEXT_POKE0); + + if (ret < 0) { + raw_spin_unlock_irqrestore(&patch_lock, flags); + return NULL; + } + patched += size; + } + raw_spin_unlock_irqrestore(&patch_lock, flags); + + flush_icache_range((uintptr_t)addr, (uintptr_t)addr + len); + + return addr; +} + +static void text_poke_memcpy(void *dst, void *src, size_t patched, size_t len) +{ + copy_to_kernel_nofault(dst, src + patched, len); +} + +static void text_poke_memset(void *dst, void *src, size_t patched, size_t len) +{ + u32 c = *(u32 *)src; + + memset32(dst, c, len / 4); +} + +/** + * aarch64_insn_copy - Copy instructions into (an unused part of) RX memory + * @dst: address to modify + * @src: source of the copy + * @len: length to copy + * + * Useful for JITs to dump new code blocks into unused regions of RX memory. + */ +noinstr void *aarch64_insn_copy(void *dst, void *src, size_t len) +{ + /* A64 instructions must be word aligned */ + if ((uintptr_t)dst & 0x3) + return NULL; + + return __text_poke(text_poke_memcpy, dst, src, len); +} + +/** + * aarch64_insn_set - memset for RX memory regions. + * @dst: address to modify + * @insn: value to set + * @len: length of memory region. + * + * Useful for JITs to fill regions of RX memory with illegal instructions. + */ +noinstr void *aarch64_insn_set(void *dst, u32 insn, size_t len) +{ + if ((uintptr_t)dst & 0x3) + return NULL; + + return __text_poke(text_poke_memset, dst, &insn, len); +} + int __kprobes aarch64_insn_patch_text_nosync(void *addr, u32 insn) { u32 *tp = addr;