From patchwork Sat Nov 17 18:57:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10687615 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F0AAE14DB for ; Sat, 17 Nov 2018 18:57:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0A1529E21 for ; Sat, 17 Nov 2018 18:57:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D482329E42; Sat, 17 Nov 2018 18:57:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7465F29E21 for ; Sat, 17 Nov 2018 18:57:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=CX5VJwxKgCrXJZWxrZih8+0VYpQjLfUSnvYRf7Uqlrs=; b=Vzpx6IYkSfS8/0C3JQE/KaRiiq v79qc6gAOxuONTmqFfN6nRVQfC6fP/kYIBwolh5Mv2CWWX92zJLOap5RHyywEzxrh52qxIupN85KZ UjHJ5wyM2weDBX1MzQQaoZe64sc6X06AQ/26/3hHqzd/qsSwk6V+0a4TuPlApcoKgvuw2/uhFt5KJ x+uaZRS+Jds673pginIItjiHfONUMpCMZmxwZwRq+InrB6R1U5b/f2q9ARoOt8N8IM4yYURIG7/gV n5fXVv1Cu5Eigg3j6tHLMsjTXzFuu7pJVtaYXj5M3RHysRFCIRxkmpPClqlD27UHC0q+04Q9fZhAs Wvcg5eNA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gO5n0-0004gy-V6; Sat, 17 Nov 2018 18:57:54 +0000 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gO5md-0004Dv-H8 for linux-arm-kernel@lists.infradead.org; Sat, 17 Nov 2018 18:57:33 +0000 Received: by mail-pf1-x442.google.com with SMTP id q1so4935574pfi.5 for ; Sat, 17 Nov 2018 10:57:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vKtcwajq6uet7RwpHABNJRRfvhJ5RLh+7DwnrP6tkoU=; b=HCrZnyJKCfFnJaZK1DpCvziEmRstnj4rwYTbT92UFXz73rvqQ9ZSjTGDU8wRnGgNJ+ ExMscyjZDgE+uBnbk94MoYbWGi/Ppcc1XZG5FsvUrcfYFnni03qp7O8Pw8kD/mV3EtA8 4qJ0sf0q5nq3snS/fKcmV1/2jIO7ddsYaFvVw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vKtcwajq6uet7RwpHABNJRRfvhJ5RLh+7DwnrP6tkoU=; b=bJ0SFwggUi5+RRS0XlHNVyH27n1LfkKLT2Pn54tMiu2x2fpGdlnImLkxGlfe4UmZxv ryzXQ2ykmWR5rDMztbEhOIrQTZeNB02co+x6BiyrDj8PfKsCz8XtGlKKxoDsCZwoleyj 7E3+c5nNYC0MWb+rwbFw8YHSAACbsOhQDFFjKFLID5dWOzcxt2QJWzQ4+boAq1vt/zt/ JTj30iksGVMGbY7216TZQIVHpH+Q3WPvl2EXCaUyeT1IrV0Fb/l1hc0Tr4UN1wnSq/ML XY/Vhf/LyEpnJxPH4GEe7kPcQ3B8nlgOTsqZuncRzecDOBbp8YaU9gzVcD1+SWAbYJ93 4fmA== X-Gm-Message-State: AGRZ1gJSCPJxQdSAlDfOZudG/+0zDT3eX+YhKABSDQRch24eY7KGWcso l5CQMWf2rNR5RxG80S60b5VzEQ== X-Google-Smtp-Source: AJdET5ctCscFODEPbn/yoyf1Ns6z0NZ0bHOkoDOM6hIBx8Sdy8wtnp67zeIE7sjr1zu/AsWtPwDO4g== X-Received: by 2002:a62:cac4:: with SMTP id y65-v6mr16264821pfk.27.1542481040868; Sat, 17 Nov 2018 10:57:20 -0800 (PST) Received: from mba13.psav.com ([64.114.255.114]) by smtp.gmail.com with ESMTPSA id u76-v6sm49550745pfa.176.2018.11.17.10.57.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 10:57:20 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Subject: [PATCH 1/4] bpf: account for freed JIT allocations in arch code Date: Sat, 17 Nov 2018 10:57:12 -0800 Message-Id: <20181117185715.25198-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117185715.25198-1-ard.biesheuvel@linaro.org> References: <20181117185715.25198-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181117_105731_562765_1A9A49DC X-CRM114-Status: GOOD ( 14.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , linux-mips@linux-mips.org, Benjamin Herrenschmidt , Will Deacon , Alexei Starovoitov , Paul Mackerras , sparclinux@vger.kernel.org, Daniel Borkmann , Michael Ellerman , Catalin Marinas , James Hogan , Kees Cook , Arnd Bergmann , Jann Horn , linux-arm-kernel@lists.infradead.org, Eric Dumazet , Ard Biesheuvel , netdev@vger.kernel.org, Ralf Baechle , Paul Burton , Jessica Yu , Rick Edgecombe , linuxppc-dev@lists.ozlabs.org, "David S. Miller" MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Commit ede95a63b5e84 ("bpf: add bpf_jit_limit knob to restrict unpriv allocations") added a call to bpf_jit_uncharge_modmem() to the routine bpf_jit_binary_free() which is called from the __weak bpf_jit_free(). This function is overridden by arches, some of which do not call bpf_jit_binary_free() to release the memory, and so the released memory is not accounted for, potentially leading to spurious allocation failures. So replace the direct calls to module_memfree() in the arch code with calls to bpf_jit_binary_free(). Signed-off-by: Ard Biesheuvel --- arch/mips/net/bpf_jit.c | 2 +- arch/powerpc/net/bpf_jit_comp.c | 2 +- arch/powerpc/net/bpf_jit_comp64.c | 5 +---- arch/sparc/net/bpf_jit_comp_32.c | 2 +- 4 files changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/mips/net/bpf_jit.c b/arch/mips/net/bpf_jit.c index 4d8cb9bb8365..1b69897274a1 100644 --- a/arch/mips/net/bpf_jit.c +++ b/arch/mips/net/bpf_jit.c @@ -1264,7 +1264,7 @@ void bpf_jit_compile(struct bpf_prog *fp) void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) - module_memfree(fp->bpf_func); + bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); bpf_prog_unlock_free(fp); } diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c index d5bfe24bb3b5..a1ea1ea6b40d 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -683,7 +683,7 @@ void bpf_jit_compile(struct bpf_prog *fp) void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) - module_memfree(fp->bpf_func); + bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); bpf_prog_unlock_free(fp); } diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 50b129785aee..84c8f013a6c6 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -1024,11 +1024,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) /* Overriding bpf_jit_free() as we don't set images read-only. */ void bpf_jit_free(struct bpf_prog *fp) { - unsigned long addr = (unsigned long)fp->bpf_func & PAGE_MASK; - struct bpf_binary_header *bpf_hdr = (void *)addr; - if (fp->jited) - bpf_jit_binary_free(bpf_hdr); + bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); bpf_prog_unlock_free(fp); } diff --git a/arch/sparc/net/bpf_jit_comp_32.c b/arch/sparc/net/bpf_jit_comp_32.c index a5ff88643d5c..01bda6bc9e7f 100644 --- a/arch/sparc/net/bpf_jit_comp_32.c +++ b/arch/sparc/net/bpf_jit_comp_32.c @@ -759,7 +759,7 @@ cond_branch: f_offset = addrs[i + filter[i].jf]; void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) - module_memfree(fp->bpf_func); + bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); bpf_prog_unlock_free(fp); } From patchwork Sat Nov 17 18:57:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10687617 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D442F14DB for ; Sat, 17 Nov 2018 18:58:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C53CE29E21 for ; Sat, 17 Nov 2018 18:58:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B7F5B29E42; Sat, 17 Nov 2018 18:58:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3C77D29E21 for ; Sat, 17 Nov 2018 18:58:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=5y95HSKJUarjoXIIrll9ObSado/Wc8XS413hJeAKkG8=; b=qQ2jLhcKy52rpsNxIf5aksMw6p 1rc5D3L3aUcYmV1N7N2xkNTmNDKTlhBVvdRvCb/BNLrf7ySt2K9LcSetKhM8SsUMorHDkERJEZcGK SmmOWplWCz1vaRCF+trhOou5DNP+92eUqZSHRGdUTKl1xGWPYx5P5drIjBfOhDpV1j+UcCWBP1ez7 lMtAMGjUy+XjN3Tx3L2uqGp1tGs3yARXPJGrwfvXCuZohh7V9CpBHh+ijjChsCfN+XxRkd7LUXz85 TRry8UhIwOyduJAQT+DSTepaM+uJC9KIJla8NGtSDWaBow9W1/lm7lj4W59m0J4tR64jam7C7ZpQj giYH9f4g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gO5nA-0004vp-HM; Sat, 17 Nov 2018 18:58:04 +0000 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gO5me-0004E5-Re for linux-arm-kernel@lists.infradead.org; Sat, 17 Nov 2018 18:57:34 +0000 Received: by mail-pf1-x444.google.com with SMTP id w73so3603254pfk.10 for ; Sat, 17 Nov 2018 10:57:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=FFyK4Oz1QZNyPCsvoKn4MQd4VSSgFUXXhCBgp8QbhcQ=; b=cObMuJm7WoQyjusLl+MSm4ZIOr4K71j4aNd5tPxwLRhwvXT/sYcvL7RDfA86wcLwA1 MBgV1MZKVJe1BNu/nkLqb6788B61g2+j8XPg6DFJaa+9SwGmzpOHS6y6Xqbefq4fvVkj +/MH7MuuSDpYgCOZS+o5G38042o7Ps9gEKLjw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=FFyK4Oz1QZNyPCsvoKn4MQd4VSSgFUXXhCBgp8QbhcQ=; b=audtCOYgUkl+Aw4W6xT142nEyPWLKrBAIcoP4pDavx+jgTu6+sQ1M4mZBoBkezBRnA hPZmHgFPby4iaTFWAkluUs6RY0u3bq6aYWVAuewpfB7f3KQ7ldiJgTzx3B/K2n5HAkYN JvWiQRzs/ktyH+edHzdL9fHjInT+11tjS+ytr415GVAUYrvyDtSL316tdgQNKIuSFgrZ flAz4uROO8TWV0PyykXm+Ns8nuaKm2cDh31Gl5OA5UXkNj/b3v7mbV0SCYoc7LX3KXUW clhoB6rK5pFQTnXJHGJVjeDgQQ+f/xT21GygFUsNhRwiM5aJt/xPR+XIJwQyxJU8KDLf JoaA== X-Gm-Message-State: AGRZ1gIyCtVJU3xBoU9MjbxRlSkvBqyg3VRSxerZ2YNZltwCndaQ+DgE 69sUYgOS6pkrM1bXAK1nPCxWHQ== X-Google-Smtp-Source: AJdET5eTfTm7MZO1Ys5mfFPU5yjtvrYsgtZia1/NgQb951W2tIaAMrSSEf/Q1za/zZx27f//YDj5kQ== X-Received: by 2002:a62:da5a:: with SMTP id w26mr3755848pfl.106.1542481042106; Sat, 17 Nov 2018 10:57:22 -0800 (PST) Received: from mba13.psav.com ([64.114.255.114]) by smtp.gmail.com with ESMTPSA id u76-v6sm49550745pfa.176.2018.11.17.10.57.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 10:57:21 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Subject: [PATCH 2/4] net/bpf: refactor freeing of executable allocations Date: Sat, 17 Nov 2018 10:57:13 -0800 Message-Id: <20181117185715.25198-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117185715.25198-1-ard.biesheuvel@linaro.org> References: <20181117185715.25198-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181117_105732_889372_AE41F3F5 X-CRM114-Status: GOOD ( 15.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , linux-mips@linux-mips.org, Benjamin Herrenschmidt , Will Deacon , Alexei Starovoitov , Paul Mackerras , sparclinux@vger.kernel.org, Daniel Borkmann , Michael Ellerman , Catalin Marinas , James Hogan , Kees Cook , Arnd Bergmann , Jann Horn , linux-arm-kernel@lists.infradead.org, Eric Dumazet , Ard Biesheuvel , netdev@vger.kernel.org, Ralf Baechle , Paul Burton , Jessica Yu , Rick Edgecombe , linuxppc-dev@lists.ozlabs.org, "David S. Miller" MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP All arch overrides of the __weak bpf_jit_free() amount to the same thing: the allocated memory was never mapped read-only, and so it does not have to be remapped to read-write before being freed. So in preparation of permitting arches to serve allocations for BPF JIT programs from other regions than the module region, refactor the existing bpf_jit_free() implementations to use the shared code where possible, and only specialize the remap and free operations. Signed-off-by: Ard Biesheuvel --- arch/mips/net/bpf_jit.c | 7 ++----- arch/powerpc/net/bpf_jit_comp.c | 7 ++----- arch/powerpc/net/bpf_jit_comp64.c | 9 +++------ arch/sparc/net/bpf_jit_comp_32.c | 7 ++----- kernel/bpf/core.c | 15 +++++---------- 5 files changed, 14 insertions(+), 31 deletions(-) diff --git a/arch/mips/net/bpf_jit.c b/arch/mips/net/bpf_jit.c index 1b69897274a1..5696bd7dccc7 100644 --- a/arch/mips/net/bpf_jit.c +++ b/arch/mips/net/bpf_jit.c @@ -1261,10 +1261,7 @@ void bpf_jit_compile(struct bpf_prog *fp) kfree(ctx.offsets); } -void bpf_jit_free(struct bpf_prog *fp) +void bpf_jit_binary_free(struct bpf_binary_header *hdr) { - if (fp->jited) - bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); - - bpf_prog_unlock_free(fp); + module_memfree(hdr); } diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c index a1ea1ea6b40d..5b5ce4a1b44b 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -680,10 +680,7 @@ void bpf_jit_compile(struct bpf_prog *fp) return; } -void bpf_jit_free(struct bpf_prog *fp) +void bpf_jit_binary_free(struct bpf_binary_header *hdr) { - if (fp->jited) - bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); - - bpf_prog_unlock_free(fp); + module_memfree(hdr); } diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 84c8f013a6c6..f64f1294bd62 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -1021,11 +1021,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) return fp; } -/* Overriding bpf_jit_free() as we don't set images read-only. */ -void bpf_jit_free(struct bpf_prog *fp) +/* Overriding bpf_jit_binary_free() as we don't set images read-only. */ +void bpf_jit_binary_free(struct bpf_binary_header *hdr) { - if (fp->jited) - bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); - - bpf_prog_unlock_free(fp); + module_memfree(hdr); } diff --git a/arch/sparc/net/bpf_jit_comp_32.c b/arch/sparc/net/bpf_jit_comp_32.c index 01bda6bc9e7f..589950d152cc 100644 --- a/arch/sparc/net/bpf_jit_comp_32.c +++ b/arch/sparc/net/bpf_jit_comp_32.c @@ -756,10 +756,7 @@ cond_branch: f_offset = addrs[i + filter[i].jf]; return; } -void bpf_jit_free(struct bpf_prog *fp) +void bpf_jit_binary_free(struct bpf_binary_header *hdr) { - if (fp->jited) - bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); - - bpf_prog_unlock_free(fp); + module_memfree(hdr); } diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 1a796e0799ec..29f766dac203 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -646,25 +646,20 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr, return hdr; } -void bpf_jit_binary_free(struct bpf_binary_header *hdr) +void __weak bpf_jit_binary_free(struct bpf_binary_header *hdr) { - u32 pages = hdr->pages; - + bpf_jit_binary_unlock_ro(hdr); module_memfree(hdr); - bpf_jit_uncharge_modmem(pages); } -/* This symbol is only overridden by archs that have different - * requirements than the usual eBPF JITs, f.e. when they only - * implement cBPF JIT, do not set images read-only, etc. - */ -void __weak bpf_jit_free(struct bpf_prog *fp) +void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) { struct bpf_binary_header *hdr = bpf_jit_binary_hdr(fp); + u32 pages = hdr->pages; - bpf_jit_binary_unlock_ro(hdr); bpf_jit_binary_free(hdr); + bpf_jit_uncharge_modmem(pages); WARN_ON_ONCE(!bpf_prog_kallsyms_verify_off(fp)); } From patchwork Sat Nov 17 18:57:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10687611 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9435A13BF for ; Sat, 17 Nov 2018 18:57:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7562A29E21 for ; Sat, 17 Nov 2018 18:57:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 613DF29E42; Sat, 17 Nov 2018 18:57:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E8DF629E21 for ; Sat, 17 Nov 2018 18:57:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=teso88V4NgXdFkT2Hieh54bjXxo269Z7RIb+y4M8628=; b=iJZ54JeG08HWhJvt6QtZYRxtrv 9QhDYdNzAmtLfJduSEQiV4yCLbmDKpzJkTY69EOGcKhluXDR6hIsBzs/ttB2JeHIMmheIL009ReG2 Mo9HTLStpIPg7+0DcmnhcYKHfOHw3ErvL7rIVA5RoTjxh3ZfOxNo9349ahLsS3T9x0gYzbJFd3IT2 /VqTOgjw9F5LXdILOHzR/bXVyJNR+3p5TXiWrqj7PfHwGS4uPTbCDGdumNApPBPwfxqvypaLM3pyZ 7XEjIfOSwdmAQViMsER6OLfdlIuNmeCkHoWh5z4pcP1/cseU15qXr9SgnH1OM9ZrqMw+QpeysqGl1 AA/rv5UA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gO5mf-0004Hh-WB; Sat, 17 Nov 2018 18:57:34 +0000 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gO5mc-0004EH-Nx for linux-arm-kernel@lists.infradead.org; Sat, 17 Nov 2018 18:57:32 +0000 Received: by mail-pg1-x544.google.com with SMTP id z11so9398653pgu.0 for ; Sat, 17 Nov 2018 10:57:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=26kHMy6e9ZHNMFq26DfYcP5qT10rPVni7XnqYDLJuPI=; b=XYLhyO5mN3FVA4ReKitsYfMKGuHNDNlOhuo5CX3m1WdOPZeKdKBplc4oZZ6V6yFg5A hQ83SmcdvQq6M0spjr5rwWJUh9QdiW39oPX0Z2kNZVWF+j0XU6sZnr/m82angEa1AG65 GQ2t7WMlRjYOWyc5If1zGyUDez/kBmIctb+b4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=26kHMy6e9ZHNMFq26DfYcP5qT10rPVni7XnqYDLJuPI=; b=hBNjnTpjCA47veO+6djCNcXMsHHP7ojMJFDbubu01Whv7eXCjyqmbkcVojEwcWeQE3 V+sDSRdMdbAjV2U2YYR7qSZiknt4rmLRnP8agMFn9h9IoCSbr0jOS1KnEKdz/f4MFcsS ts4qhXiWqv6Is+bd1RoYKCpBqrbcH/fVURUdNK/EZwkGCyK+J7F+m9IfatI+y7sD7SKH zGZK0Zm4hrPIKznXv9TY6/syYkQra8L4QHkqXJr3f5KKNN/gxQl7cXW+eNUxb0X5F0aj glMhjw92ARFUWBaoywtrVYagmxl8rqKn+PaNbzmeupJmXuyHduO470beOOdXlQYhfkvA HM7A== X-Gm-Message-State: AGRZ1gLJa3OU6jxJG906AjfSqWchdVlpQ3qk0366XA9MITdH5r4dLM6G +d5IJ3cpaL1/ENWXwevtChikoA== X-Google-Smtp-Source: AJdET5dbJskaau0oeh2D+j2r5+z2XCOLI6qC1Gpj92MrCzlhVrhcUg2kk12XFX1xpT0h9FiUT1nBcw== X-Received: by 2002:a63:f241:: with SMTP id d1mr14569834pgk.2.1542481043353; Sat, 17 Nov 2018 10:57:23 -0800 (PST) Received: from mba13.psav.com ([64.114.255.114]) by smtp.gmail.com with ESMTPSA id u76-v6sm49550745pfa.176.2018.11.17.10.57.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 10:57:22 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Subject: [PATCH 3/4] bpf: add __weak hook for allocating executable memory Date: Sat, 17 Nov 2018 10:57:14 -0800 Message-Id: <20181117185715.25198-4-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117185715.25198-1-ard.biesheuvel@linaro.org> References: <20181117185715.25198-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181117_105730_779844_F37AF89D X-CRM114-Status: GOOD ( 15.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , linux-mips@linux-mips.org, Benjamin Herrenschmidt , Will Deacon , Alexei Starovoitov , Paul Mackerras , sparclinux@vger.kernel.org, Daniel Borkmann , Michael Ellerman , Catalin Marinas , James Hogan , Kees Cook , Arnd Bergmann , Jann Horn , linux-arm-kernel@lists.infradead.org, Eric Dumazet , Ard Biesheuvel , netdev@vger.kernel.org, Ralf Baechle , Paul Burton , Jessica Yu , Rick Edgecombe , linuxppc-dev@lists.ozlabs.org, "David S. Miller" MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP By default, BPF uses module_alloc() to allocate executable memory, but this is not necessary on all arches and potentially undesirable on some of them. So break out the module_alloc() call into a __weak function to allow it to be overridden in arch code. Signed-off-by: Ard Biesheuvel --- kernel/bpf/core.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 29f766dac203..156d6b96ac6c 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -609,6 +609,11 @@ static void bpf_jit_uncharge_modmem(u32 pages) atomic_long_sub(pages, &bpf_jit_current); } +void *__weak bpf_jit_alloc_exec(unsigned long size) +{ + return module_alloc(size); +} + struct bpf_binary_header * bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr, unsigned int alignment, @@ -626,7 +631,7 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr, if (bpf_jit_charge_modmem(pages)) return NULL; - hdr = module_alloc(size); + hdr = bpf_jit_alloc_exec(size); if (!hdr) { bpf_jit_uncharge_modmem(pages); return NULL; From patchwork Sat Nov 17 18:57:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10687619 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC1E613BF for ; Sat, 17 Nov 2018 18:58:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCE5C29E21 for ; Sat, 17 Nov 2018 18:58:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BDAAE29E42; Sat, 17 Nov 2018 18:58:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 54CBD29E21 for ; Sat, 17 Nov 2018 18:58:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=J47iypGP2syUrNM7KCKMfzUyFZmJ3UbVAQBcetsLPYM=; b=OMLmtFLEGgxZ+tGHZrGrNuea5N J6Wg567+sSLqLxKLM7WpTm8bDdbeTXkRB1GT+xfGcRjJbxfU3ZxSu1TzKm3XcXVqnG1002FGKxkhL 0vOHzg6XPJHqZT+jz76rm+3Qc/kvjwidcznjxehL7N0MfaWpOuDGX/f29rs6vQtLw6edgCnAmJGjC v1Dc/ivHnVYmycI3FkLYJQFwWU5BYmDJzlzpHbIXQGWdh5lRzouv8SoZpOShjm2vbEdQcPuD8UaBI h50U3jeijTt/sfT8m1PBA3Q+VgqPWhgBE5/8rJhCruZvvA28syt3lowPVnnkKkDxfvhr/qRz83o7r FlRSLNgw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gO5nL-0005A2-4E; Sat, 17 Nov 2018 18:58:15 +0000 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gO5mh-0004EI-BO for linux-arm-kernel@lists.infradead.org; Sat, 17 Nov 2018 18:57:36 +0000 Received: by mail-pg1-x542.google.com with SMTP id z10so12030454pgp.7 for ; Sat, 17 Nov 2018 10:57:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JmtMWaiD9FenKM3ns0kNxP2J4QR+RUeNI/5TJ/tQ45I=; b=KAW8uS2esdTjeBcDfFodpTuBiPuzrfOgpEvhmkdmizzqp7CMEjaFC2ORlF9MeL+yeA stdwlyxwK9cJAK+MAYm3DEfW4lRgMCiJRj3yewaXsrtQx5DmclC4R/K9qjVetVNcal+H pjN2zNK/65b6+lBFp6r24jInRxn3xDvrMAU8w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JmtMWaiD9FenKM3ns0kNxP2J4QR+RUeNI/5TJ/tQ45I=; b=i9YELnZ78yQT6uLl3DZZfICsQIVC94r69diawoCGtgZwW2MYUKmBw+HA0G2d9df/AF 1LV5Sup19JcSA8o8vt/Ky6ihLO+u+U8Y2Qb47h3k+TohySRgXctiiR9B/WR6V8E2wruB 4NewVYIlup+reU5t5eLkBpFUCozc7cZXj4jZnfCUIeXPx/UEUYn5Hkz0BAJZ/sckSVKG EdQ+Auy/ynNFvqZLF6g/zNUbB8Z6V7SvcQdthBFPOCrrAQzz6oPhxLzCR8Xp9wouFUm7 ID1s+PScHaRa1lCU7LCCx16CJPRfJ/f7qYrmksLy5I0DWzoldsaO79qWHqBNtIRMTKnj 2KrA== X-Gm-Message-State: AGRZ1gJyf2bkCbv87I+ZiUQZzafFRISBjSDvQaD306Zhlm0dcINCLaq4 K1jDBtBvgFqZR/K5+XOWgqRb+Q== X-Google-Smtp-Source: AJdET5epwL8opIfWAHhuSBY1hNn/GKz9erLG44ejL6V5gIz84eLRbi+P5B/LB3RozshYttdcdgtQpg== X-Received: by 2002:a62:ed09:: with SMTP id u9-v6mr16036733pfh.188.1542481044594; Sat, 17 Nov 2018 10:57:24 -0800 (PST) Received: from mba13.psav.com ([64.114.255.114]) by smtp.gmail.com with ESMTPSA id u76-v6sm49550745pfa.176.2018.11.17.10.57.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 10:57:23 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Subject: [PATCH 4/4] arm64/bpf: don't allocate BPF JIT programs in module memory Date: Sat, 17 Nov 2018 10:57:15 -0800 Message-Id: <20181117185715.25198-5-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117185715.25198-1-ard.biesheuvel@linaro.org> References: <20181117185715.25198-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181117_105735_414222_D6062E42 X-CRM114-Status: GOOD ( 12.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , linux-mips@linux-mips.org, Benjamin Herrenschmidt , Will Deacon , Alexei Starovoitov , Paul Mackerras , sparclinux@vger.kernel.org, Daniel Borkmann , Michael Ellerman , Catalin Marinas , James Hogan , Kees Cook , Arnd Bergmann , Jann Horn , linux-arm-kernel@lists.infradead.org, Eric Dumazet , Ard Biesheuvel , netdev@vger.kernel.org, Ralf Baechle , Paul Burton , Jessica Yu , Rick Edgecombe , linuxppc-dev@lists.ozlabs.org, "David S. Miller" MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The arm64 module region is a 128 MB region that is kept close to the core kernel, in order to ensure that relative branches are always in range. So using the same region for programs that do not have this restriction is wasteful, and preferably avoided. Now that the core BPF JIT code permits the alloc/free routines to be overridden, implement them by simple vmalloc_exec()/vfree() calls, which can be served from anywere. This also solves an issue under KASAN, where shadow memory is needlessly allocated for all BPF programs (which don't require KASAN shadow pages since they are not KASAN instrumented) Signed-off-by: Ard Biesheuvel --- arch/arm64/net/bpf_jit_comp.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index a6fdaea07c63..e0c702c2f682 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -940,3 +940,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) tmp : orig_prog); return prog; } + +void *bpf_jit_alloc_exec(unsigned long size) +{ + return vmalloc_exec(size); +} + +void bpf_jit_binary_free(struct bpf_binary_header *hdr) +{ + bpf_jit_binary_unlock_ro(hdr); + vfree(hdr); +}