From patchwork Mon Dec 9 17:31:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11279751 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85A5B930 for ; Mon, 9 Dec 2019 17:31:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6402F20828 for ; Mon, 9 Dec 2019 17:31:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="iAWygKjL"; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JG/Fdtoa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6402F20828 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bYazlit0M4yBFQMfwCbOmC5g6AIzkoA/91yiEDZt7b0=; b=iAWygKjLejcquJ 4hoH2PS4u2ts6HIuEKs4w3L3z8WI6Pemtg8nozMLt4GvFWq59cdsrtkY/NvMlUGcEiBxrJeHLlLLf BtaMkQ1CG0US6yapwIRQP4FsK6GO8kVcNlHNdkIbS53U3ICPgIRYR6s2ujewLbFLghmhRKNNsS5rY YRuNAFoeH9XA+/RI3WNIQdJ2xSX2bSzmG0QfLXsURXxbk7oI2LjeXiJ7t54jVlE0Z2t4cqYj3Ccu4 OntSYjmITM77hJCprZHn9uujXwXXB+dgKWsBx97D3Zswj4yKjCekqdbgU7EshvREPlaxpHwuCj/9s 45PnbU/wkcFLpQN9uczg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMt0-0004F9-C3; Mon, 09 Dec 2019 17:31:54 +0000 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMsx-0004D9-Q2 for linux-riscv@lists.infradead.org; Mon, 09 Dec 2019 17:31:53 +0000 Received: by mail-pl1-x644.google.com with SMTP id o8so6086507pls.5 for ; Mon, 09 Dec 2019 09:31:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bYazlit0M4yBFQMfwCbOmC5g6AIzkoA/91yiEDZt7b0=; b=JG/FdtoaT1PdHdeb1nQSG3n58INhw4WAwPx4fgf/iNqq6+YukonnT/Te2EfrJCYbwS BA839prSyM9HNh1zIq1Y6mtGIkk9Hb+UVfTCxDxX7jXyY2CwmcHbybwq+G5REvSl/X40 P9Rh91X5SyPVRrqmh9uS5XjMdub94pM7M0MD5oQtO/gb4wKcvuCcLgwXIQ2VOyXkAC1b q22mSGWXKWS6XeieReZpy5r+4wkmjv+esiy9JZkyXkCJRjbFGxTYAPCdvNaLN6VVjfK2 0Gua8XtVNUIyFq0HqEyNqTgudqrWWL9uHE+hGO7h/IaBfuYzgCqRMBV0akFCO4kKU77v 2YZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bYazlit0M4yBFQMfwCbOmC5g6AIzkoA/91yiEDZt7b0=; b=HxptjIYzifoCvi/RC6y+l+Al3c1rA/pRHct8zUxfNuP+YWTSVdd0AJV8Q0q6vz75Kp n09EUysSEF70+6sweplc4L/pipwfAxpMXuXeZCIy1qZt9HC001zhpGeMOkmchONV498H fhXV7U4cBI7zur3QGcroMYr5yaRvbV9G+OLO2FnNJCZMKy5jTxRgIvJv3oVXMDuCaIqz 0b9DCoAYe7BB1yLcgXsinGHdyHWW+L/m2yWDURyRw7JDOFH5h2AmLdnhON7rkkkLKXVY sgRhzS3JUfWYT+w3KdHmwY/Uq0Ra8RkGBxMPqxR7V73L/7mJdzDxfibBGlWumxyg80kf WBlQ== X-Gm-Message-State: APjAAAWXvKWtY4XD5p6fHcS10Om4nfuKPSK0jvNG7OiYmOOEarZyl6mp GV8Xg4SxILnxBuRKFJ4xn2Y= X-Google-Smtp-Source: APXvYqwrSC1AJl671lj66BCLyGpEOq1PPz3mWlePEUYhbgIUczUyAsdnUS2TdGA2AA3b+CVw6LGv2Q== X-Received: by 2002:a17:902:6502:: with SMTP id b2mr30307947plk.182.1575912710989; Mon, 09 Dec 2019 09:31:50 -0800 (PST) Received: from btopel-mobl.ger.intel.com ([192.55.55.41]) by smtp.gmail.com with ESMTPSA id d23sm54943pfo.176.2019.12.09.09.31.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 09:31:50 -0800 (PST) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: daniel@iogearbox.net, ast@kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 1/8] riscv, bpf: fix broken BPF tail calls Date: Mon, 9 Dec 2019 18:31:29 +0100 Message-Id: <20191209173136.29615-2-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191209173136.29615-1-bjorn.topel@gmail.com> References: <20191209173136.29615-1-bjorn.topel@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191209_093151_855541_65183F43 X-CRM114-Status: UNSURE ( 9.63 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:644 listed in] [list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (bjorn.topel[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-riscv@lists.infradead.org, bpf@vger.kernel.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org The BPF JIT incorrectly clobbered the a0 register, and did not flag usage of s5 register when BPF stack was being used. Fixes: 2353ecc6f91f ("bpf, riscv: add BPF JIT for RV64G") Signed-off-by: Björn Töpel Reviewed-by: Palmer Dabbelt Acked-by: Palmer Dabbelt --- arch/riscv/net/bpf_jit_comp.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/arch/riscv/net/bpf_jit_comp.c b/arch/riscv/net/bpf_jit_comp.c index 5451ef3845f2..1606ebd49666 100644 --- a/arch/riscv/net/bpf_jit_comp.c +++ b/arch/riscv/net/bpf_jit_comp.c @@ -120,6 +120,11 @@ static bool seen_reg(int reg, struct rv_jit_context *ctx) return false; } +static void mark_fp(struct rv_jit_context *ctx) +{ + __set_bit(RV_CTX_F_SEEN_S5, &ctx->flags); +} + static void mark_call(struct rv_jit_context *ctx) { __set_bit(RV_CTX_F_SEEN_CALL, &ctx->flags); @@ -596,7 +601,8 @@ static void __build_epilogue(u8 reg, struct rv_jit_context *ctx) emit(rv_addi(RV_REG_SP, RV_REG_SP, stack_adjust), ctx); /* Set return value. */ - emit(rv_addi(RV_REG_A0, RV_REG_A5, 0), ctx); + if (reg == RV_REG_RA) + emit(rv_addi(RV_REG_A0, RV_REG_A5, 0), ctx); emit(rv_jalr(RV_REG_ZERO, reg, 0), ctx); } @@ -1426,6 +1432,10 @@ static void build_prologue(struct rv_jit_context *ctx) { int stack_adjust = 0, store_offset, bpf_stack_adjust; + bpf_stack_adjust = round_up(ctx->prog->aux->stack_depth, 16); + if (bpf_stack_adjust) + mark_fp(ctx); + if (seen_reg(RV_REG_RA, ctx)) stack_adjust += 8; stack_adjust += 8; /* RV_REG_FP */ @@ -1443,7 +1453,6 @@ static void build_prologue(struct rv_jit_context *ctx) stack_adjust += 8; stack_adjust = round_up(stack_adjust, 16); - bpf_stack_adjust = round_up(ctx->prog->aux->stack_depth, 16); stack_adjust += bpf_stack_adjust; store_offset = stack_adjust - 8; From patchwork Mon Dec 9 17:31:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11279753 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3577F112B for ; Mon, 9 Dec 2019 17:32:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 112EC20828 for ; Mon, 9 Dec 2019 17:32:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="hGZ3mFEZ"; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hyiEarzz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 112EC20828 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FpE65aBNA+JRfVC7pbkzHG8/CovFVrJd6SUJWCtvEik=; b=hGZ3mFEZnigECU xjuz3pp0tdLxFVFXfGf8HB4ylM6VgpWHJNMUZgwRulaejxwEEUkmsVFC+2T7uRDGAFdJArzZ8FxR0 ClONZhiJAwJqsz26bqhhm8iLoyHfvTW03GFHfcJAPkbX/34Ut5IPvaA/RWQ8m/GLq7GbBMxLXpXAN CF0JSp3Xo9EKuHQ4iLrE8pVx14ejXAkRznCKDfnO/tpmWAssNskBR8EXton8SqH9hZyqfIaeng83F 0O5MKytXA651Wyw9q8QmammKFOMKfTB0EWmZ42cAadljc5vwqKtn4BGZO9br17KLb9Q7BsTcx7+3s j2gvaHBRL9pwBlwH/AQw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMt5-0004Js-Rm; Mon, 09 Dec 2019 17:31:59 +0000 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMt1-0004FF-Bx for linux-riscv@lists.infradead.org; Mon, 09 Dec 2019 17:31:58 +0000 Received: by mail-pg1-x544.google.com with SMTP id k25so7450160pgt.7 for ; Mon, 09 Dec 2019 09:31:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FpE65aBNA+JRfVC7pbkzHG8/CovFVrJd6SUJWCtvEik=; b=hyiEarzzZZv0ZuMGwMs7TdCkIsAPGnN4mkQbB2c4GWNr8p9IdB0/xGIA2UC6azt0X9 Nl3jXU7HRZmwdwUfz6OJ3a/JfQYSouDC2lnSauUz3yS1bf/7r0rmIWeYHPcOyhZV3xcG jGwTR3mosWTXjrG3TTQdGW2UoFK59H8sV+N6SWhS3qTj1EbAubLZNSrmqPY01DgF7OKd Fo6OtJlZLAZkV4/Vw2+pCsDy82A3GNTtjCB96y0M8t9hAt6D7RAb04yqjr0y9MHfBpve k2Cc2Qr5WOjI+zGRkvEgFX2FnQL3DQou2uVNb1Ec7ikN9OKrtU+9Eyuc2UGOlR9+ye1O TciQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FpE65aBNA+JRfVC7pbkzHG8/CovFVrJd6SUJWCtvEik=; b=IVBeLC2eQbA7dAHJKoxoOVUx5o3NbZjJ6Gwb9enQvd61vyCcNvOPAXGEzhn4z4p69F J2yOcr+A0KCA+ozXRlyVC4k0UkFYoFwM7VuJpZJyvnErDt/EBcpB7obs+z+bihYsM6P2 9Nm+aNTYPRGas6+Eimu2OMfcIdJ55/xRVXstfIqgrCT9Bb5Q+RY+oNsjXfw1D/BgyzVN BKpVSzbU5guFoBDjDGBADmdPjXtMV8Jr3VWUA7oQFVhk6bZA4Aph0HagnbKie4xp5gaf VZFzg+Sr7gbmzGWLoohVLE8mZBH0i3dxGHydBzFHKGvUOP2NVM0PgXFiMKcoNkHH1mXD uxUw== X-Gm-Message-State: APjAAAUbSpsoQq8IT/WWIyx2lUFUxvHRrvbhPeyhUTZCRf4DeDuJH04L LEm6MCtFYnGsNUZEPX2tk8c= X-Google-Smtp-Source: APXvYqxjk+MyMW4UEEeWkZhw1Bbzy/NgEIEXK1TuK8Sq1RlkB2DHr080EkHSzUVec/O25/t1D+W/IQ== X-Received: by 2002:a63:1a22:: with SMTP id a34mr18743924pga.403.1575912713780; Mon, 09 Dec 2019 09:31:53 -0800 (PST) Received: from btopel-mobl.ger.intel.com ([192.55.55.41]) by smtp.gmail.com with ESMTPSA id d23sm54943pfo.176.2019.12.09.09.31.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 09:31:53 -0800 (PST) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: daniel@iogearbox.net, ast@kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 2/8] riscv, bpf: add support for far branching Date: Mon, 9 Dec 2019 18:31:30 +0100 Message-Id: <20191209173136.29615-3-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191209173136.29615-1-bjorn.topel@gmail.com> References: <20191209173136.29615-1-bjorn.topel@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191209_093155_479051_D8F41CBE X-CRM114-Status: GOOD ( 16.21 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:544 listed in] [list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (bjorn.topel[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-riscv@lists.infradead.org, bpf@vger.kernel.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org This commit adds branch relaxation to the BPF JIT, and with that support for far (offset greater than 12b) branching. The branch relaxation requires more than two passes to converge. For most programs it is three passes, but for larger programs it can be more. Signed-off-by: Björn Töpel Reviewed-by: Luke Nelson --- arch/riscv/net/bpf_jit_comp.c | 352 ++++++++++++++++++---------------- 1 file changed, 188 insertions(+), 164 deletions(-) diff --git a/arch/riscv/net/bpf_jit_comp.c b/arch/riscv/net/bpf_jit_comp.c index 1606ebd49666..e2f6724b64f7 100644 --- a/arch/riscv/net/bpf_jit_comp.c +++ b/arch/riscv/net/bpf_jit_comp.c @@ -461,6 +461,11 @@ static u32 rv_amoadd_d(u8 rd, u8 rs2, u8 rs1, u8 aq, u8 rl) return rv_amo_insn(0, aq, rl, rs2, rs1, 3, rd, 0x2f); } +static u32 rv_auipc(u8 rd, u32 imm31_12) +{ + return rv_u_insn(imm31_12, rd, 0x17); +} + static bool is_12b_int(s64 val) { return -(1 << 11) <= val && val < (1 << 11); @@ -484,7 +489,7 @@ static bool is_32b_int(s64 val) static int is_12b_check(int off, int insn) { if (!is_12b_int(off)) { - pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n", + pr_err("bpf-jit: insn=%d 12b < offset=%d not supported yet!\n", insn, (int)off); return -1; } @@ -494,7 +499,7 @@ static int is_12b_check(int off, int insn) static int is_13b_check(int off, int insn) { if (!is_13b_int(off)) { - pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n", + pr_err("bpf-jit: insn=%d 13b < offset=%d not supported yet!\n", insn, (int)off); return -1; } @@ -504,7 +509,7 @@ static int is_13b_check(int off, int insn) static int is_21b_check(int off, int insn) { if (!is_21b_int(off)) { - pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n", + pr_err("bpf-jit: insn=%d 21b < offset=%d not supported yet!\n", insn, (int)off); return -1; } @@ -550,10 +555,13 @@ static void emit_imm(u8 rd, s64 val, struct rv_jit_context *ctx) emit(rv_addi(rd, rd, lower), ctx); } -static int rv_offset(int bpf_to, int bpf_from, struct rv_jit_context *ctx) +static int rv_offset(int insn, int off, struct rv_jit_context *ctx) { - int from = ctx->offset[bpf_from] - 1, to = ctx->offset[bpf_to]; + int from, to; + off++; /* BPF branch is from PC+1, RV is from PC */ + from = (insn > 0) ? ctx->offset[insn - 1] : 0; + to = (insn + off > 0) ? ctx->offset[insn + off - 1] : 0; return (to - from) << 2; } @@ -693,13 +701,6 @@ static void init_regs(u8 *rd, u8 *rs, const struct bpf_insn *insn, *rs = bpf_to_rv_reg(insn->src_reg, ctx); } -static int rv_offset_check(int *rvoff, s16 off, int insn, - struct rv_jit_context *ctx) -{ - *rvoff = rv_offset(insn + off, insn, ctx); - return is_13b_check(*rvoff, insn); -} - static void emit_zext_32_rd_rs(u8 *rd, u8 *rs, struct rv_jit_context *ctx) { emit(rv_addi(RV_REG_T2, *rd, 0), ctx); @@ -732,13 +733,122 @@ static void emit_sext_32_rd(u8 *rd, struct rv_jit_context *ctx) *rd = RV_REG_T2; } +/* return -1 or inverted cond */ +static int invert_bpf_cond(u8 cond) +{ + switch (cond) { + case BPF_JEQ: + return BPF_JNE; + case BPF_JGT: + return BPF_JLE; + case BPF_JLT: + return BPF_JGE; + case BPF_JGE: + return BPF_JLT; + case BPF_JLE: + return BPF_JGT; + case BPF_JNE: + return BPF_JEQ; + case BPF_JSGT: + return BPF_JSLE; + case BPF_JSLT: + return BPF_JSGE; + case BPF_JSGE: + return BPF_JSLT; + case BPF_JSLE: + return BPF_JSGT; + } + return -1; +} + +static void emit_bcc(u8 cond, u8 rd, u8 rs, int rvoff, + struct rv_jit_context *ctx) +{ + switch (cond) { + case BPF_JEQ: + emit(rv_beq(rd, rs, rvoff >> 1), ctx); + return; + case BPF_JGT: + emit(rv_bltu(rs, rd, rvoff >> 1), ctx); + return; + case BPF_JLT: + emit(rv_bltu(rd, rs, rvoff >> 1), ctx); + return; + case BPF_JGE: + emit(rv_bgeu(rd, rs, rvoff >> 1), ctx); + return; + case BPF_JLE: + emit(rv_bgeu(rs, rd, rvoff >> 1), ctx); + return; + case BPF_JNE: + emit(rv_bne(rd, rs, rvoff >> 1), ctx); + return; + case BPF_JSGT: + emit(rv_blt(rs, rd, rvoff >> 1), ctx); + return; + case BPF_JSLT: + emit(rv_blt(rd, rs, rvoff >> 1), ctx); + return; + case BPF_JSGE: + emit(rv_bge(rd, rs, rvoff >> 1), ctx); + return; + case BPF_JSLE: + emit(rv_bge(rs, rd, rvoff >> 1), ctx); + } +} + +static void emit_branch(u8 cond, u8 rd, u8 rs, int insn, int rvoff, + struct rv_jit_context *ctx) +{ + s64 upper, lower; + + if (is_13b_int(rvoff)) { + emit_bcc(cond, rd, rs, rvoff, ctx); + return; + } + + /* Adjust for jal */ + rvoff -= 4; + + /* Transform, e.g.: + * bne rd,rs,foo + * to + * beq rd,rs,<.L1> + * (auipc foo) + * jal(r) foo + * .L1 + */ + cond = invert_bpf_cond(cond); + if (is_21b_int(rvoff)) { + emit_bcc(cond, rd, rs, 8, ctx); + emit(rv_jal(RV_REG_ZERO, rvoff >> 1), ctx); + return; + } + + /* 32b No need for an additional rvoff adjustment, since we + * get that from the auipc at PC', where PC = PC' + 4. + */ + upper = (rvoff + (1 << 11)) >> 12; + lower = rvoff & 0xfff; + + emit_bcc(cond, rd, rs, 12, ctx); + emit(rv_auipc(RV_REG_T1, upper), ctx); + emit(rv_jalr(RV_REG_ZERO, RV_REG_T1, lower), ctx); +} + +static bool is_signed_bpf_cond(u8 cond) +{ + return cond == BPF_JSGT || cond == BPF_JSLT || + cond == BPF_JSGE || cond == BPF_JSLE; +} + static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, bool extra_pass) { bool is64 = BPF_CLASS(insn->code) == BPF_ALU64 || BPF_CLASS(insn->code) == BPF_JMP; + int s, e, rvoff, i = insn - ctx->prog->insnsi; struct bpf_prog_aux *aux = ctx->prog->aux; - int rvoff, i = insn - ctx->prog->insnsi; u8 rd = -1, rs = -1, code = insn->code; s16 off = insn->off; s32 imm = insn->imm; @@ -1006,7 +1116,7 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, /* JUMP off */ case BPF_JMP | BPF_JA: - rvoff = rv_offset(i + off, i, ctx); + rvoff = rv_offset(i, off, ctx); if (!is_21b_int(rvoff)) { pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n", i, rvoff); @@ -1019,194 +1129,96 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, /* IF (dst COND src) JUMP off */ case BPF_JMP | BPF_JEQ | BPF_X: case BPF_JMP32 | BPF_JEQ | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_zext_32_rd_rs(&rd, &rs, ctx); - emit(rv_beq(rd, rs, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JGT | BPF_X: case BPF_JMP32 | BPF_JGT | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_zext_32_rd_rs(&rd, &rs, ctx); - emit(rv_bltu(rs, rd, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JLT | BPF_X: case BPF_JMP32 | BPF_JLT | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_zext_32_rd_rs(&rd, &rs, ctx); - emit(rv_bltu(rd, rs, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JGE | BPF_X: case BPF_JMP32 | BPF_JGE | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_zext_32_rd_rs(&rd, &rs, ctx); - emit(rv_bgeu(rd, rs, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JLE | BPF_X: case BPF_JMP32 | BPF_JLE | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_zext_32_rd_rs(&rd, &rs, ctx); - emit(rv_bgeu(rs, rd, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JNE | BPF_X: case BPF_JMP32 | BPF_JNE | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_zext_32_rd_rs(&rd, &rs, ctx); - emit(rv_bne(rd, rs, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSGT | BPF_X: case BPF_JMP32 | BPF_JSGT | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_sext_32_rd_rs(&rd, &rs, ctx); - emit(rv_blt(rs, rd, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSLT | BPF_X: case BPF_JMP32 | BPF_JSLT | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_sext_32_rd_rs(&rd, &rs, ctx); - emit(rv_blt(rd, rs, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSGE | BPF_X: case BPF_JMP32 | BPF_JSGE | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_sext_32_rd_rs(&rd, &rs, ctx); - emit(rv_bge(rd, rs, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSLE | BPF_X: case BPF_JMP32 | BPF_JSLE | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_sext_32_rd_rs(&rd, &rs, ctx); - emit(rv_bge(rs, rd, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSET | BPF_X: case BPF_JMP32 | BPF_JSET | BPF_X: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - if (!is64) - emit_zext_32_rd_rs(&rd, &rs, ctx); - emit(rv_and(RV_REG_T1, rd, rs), ctx); - emit(rv_bne(RV_REG_T1, RV_REG_ZERO, rvoff >> 1), ctx); + rvoff = rv_offset(i, off, ctx); + if (!is64) { + s = ctx->ninsns; + if (is_signed_bpf_cond(BPF_OP(code))) + emit_sext_32_rd_rs(&rd, &rs, ctx); + else + emit_zext_32_rd_rs(&rd, &rs, ctx); + e = ctx->ninsns; + + /* Adjust for extra insns */ + rvoff -= (e - s) << 2; + } + + if (BPF_OP(code) == BPF_JSET) { + /* Adjust for and */ + rvoff -= 4; + emit(rv_and(RV_REG_T1, rd, rs), ctx); + emit_branch(BPF_JNE, RV_REG_T1, RV_REG_ZERO, i, rvoff, + ctx); + } else { + emit_branch(BPF_OP(code), rd, rs, i, rvoff, ctx); + } break; /* IF (dst COND imm) JUMP off */ case BPF_JMP | BPF_JEQ | BPF_K: case BPF_JMP32 | BPF_JEQ | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_zext_32_rd_t1(&rd, ctx); - emit(rv_beq(rd, RV_REG_T1, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JGT | BPF_K: case BPF_JMP32 | BPF_JGT | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_zext_32_rd_t1(&rd, ctx); - emit(rv_bltu(RV_REG_T1, rd, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JLT | BPF_K: case BPF_JMP32 | BPF_JLT | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_zext_32_rd_t1(&rd, ctx); - emit(rv_bltu(rd, RV_REG_T1, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JGE | BPF_K: case BPF_JMP32 | BPF_JGE | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_zext_32_rd_t1(&rd, ctx); - emit(rv_bgeu(rd, RV_REG_T1, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JLE | BPF_K: case BPF_JMP32 | BPF_JLE | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_zext_32_rd_t1(&rd, ctx); - emit(rv_bgeu(RV_REG_T1, rd, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JNE | BPF_K: case BPF_JMP32 | BPF_JNE | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_zext_32_rd_t1(&rd, ctx); - emit(rv_bne(rd, RV_REG_T1, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSGT | BPF_K: case BPF_JMP32 | BPF_JSGT | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_sext_32_rd(&rd, ctx); - emit(rv_blt(RV_REG_T1, rd, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSLT | BPF_K: case BPF_JMP32 | BPF_JSLT | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_sext_32_rd(&rd, ctx); - emit(rv_blt(rd, RV_REG_T1, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSGE | BPF_K: case BPF_JMP32 | BPF_JSGE | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_sext_32_rd(&rd, ctx); - emit(rv_bge(rd, RV_REG_T1, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSLE | BPF_K: case BPF_JMP32 | BPF_JSLE | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; - emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_sext_32_rd(&rd, ctx); - emit(rv_bge(RV_REG_T1, rd, rvoff >> 1), ctx); - break; case BPF_JMP | BPF_JSET | BPF_K: case BPF_JMP32 | BPF_JSET | BPF_K: - if (rv_offset_check(&rvoff, off, i, ctx)) - return -1; + rvoff = rv_offset(i, off, ctx); + s = ctx->ninsns; emit_imm(RV_REG_T1, imm, ctx); - if (!is64) - emit_zext_32_rd_t1(&rd, ctx); - emit(rv_and(RV_REG_T1, rd, RV_REG_T1), ctx); - emit(rv_bne(RV_REG_T1, RV_REG_ZERO, rvoff >> 1), ctx); + if (!is64) { + if (is_signed_bpf_cond(BPF_OP(code))) + emit_sext_32_rd(&rd, ctx); + else + emit_zext_32_rd_t1(&rd, ctx); + } + e = ctx->ninsns; + + /* Adjust for extra insns */ + rvoff -= (e - s) << 2; + + if (BPF_OP(code) == BPF_JSET) { + /* Adjust for and */ + rvoff -= 4; + emit(rv_and(RV_REG_T1, rd, RV_REG_T1), ctx); + emit_branch(BPF_JNE, RV_REG_T1, RV_REG_ZERO, i, rvoff, + ctx); + } else { + emit_branch(BPF_OP(code), rd, RV_REG_T1, i, rvoff, ctx); + } break; /* function call */ @@ -1557,6 +1569,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) { bool tmp_blinded = false, extra_pass = false; struct bpf_prog *tmp, *orig_prog = prog; + int pass = 0, prev_ninsns = 0, i; struct rv_jit_data *jit_data; struct rv_jit_context *ctx; unsigned int image_size; @@ -1596,15 +1609,25 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) prog = orig_prog; goto out_offset; } + for (i = 0; i < prog->len; i++) { + prev_ninsns += 32; + ctx->offset[i] = prev_ninsns; + } - /* First pass generates the ctx->offset, but does not emit an image. */ - if (build_body(ctx, extra_pass)) { - prog = orig_prog; - goto out_offset; + for (i = 0; i < 16; i++) { + pass++; + ctx->ninsns = 0; + if (build_body(ctx, extra_pass)) { + prog = orig_prog; + goto out_offset; + } + build_prologue(ctx); + ctx->epilogue_offset = ctx->ninsns; + build_epilogue(ctx); + if (ctx->ninsns == prev_ninsns) + break; + prev_ninsns = ctx->ninsns; } - build_prologue(ctx); - ctx->epilogue_offset = ctx->ninsns; - build_epilogue(ctx); /* Allocate image, now that we know the size. */ image_size = sizeof(u32) * ctx->ninsns; @@ -1619,6 +1642,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) /* Second, real pass, that acutally emits the image. */ ctx->insns = (u32 *)jit_data->image; skip_init_ctx: + pass++; ctx->ninsns = 0; build_prologue(ctx); @@ -1630,7 +1654,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) build_epilogue(ctx); if (bpf_jit_enable > 1) - bpf_jit_dump(prog->len, image_size, 2, ctx->insns); + bpf_jit_dump(prog->len, image_size, pass, ctx->insns); prog->bpf_func = (void *)ctx->insns; prog->jited = 1; From patchwork Mon Dec 9 17:31:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11279755 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 259B6112B for ; Mon, 9 Dec 2019 17:32:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0456820828 for ; Mon, 9 Dec 2019 17:32:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="kZVxgEkI"; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="I2wCeTTo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0456820828 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ldUAi30p4Fm9YFcB6XPp82qK2Gx4LpczH2It45srXeo=; b=kZVxgEkIYfLI/T 40fXu/X/2YzUV6SOwzxQbddRfC1xNVvDX0zNiTf/4tOtHkQph+LECo9pUwXlqgN3kfH1vn7HmpWHx UQ1aw4xA/Uoxz3MMOx9iLrKBMmrSvDPQC/4DkH+4vswCg0o1JrtyQc7XlbDJVWsRkG3mVaL4sx5/+ 8d7B4bi0Z5vlkw89K0CU4tVi7xc9ve8CPewxy+9T2WWzeIitOBUFmz4c261gJseR8s39PeqZn2e6L X4SaFbNI1PfULf0YIe52tPn1iF+5zEC+jsadVIcxZAdMkAAhwKV6xmi6cmbd0AosDR7ejQJPHooq+ 0AuoS2QnH01cGV3NR/gg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMt7-0004MC-Qk; Mon, 09 Dec 2019 17:32:01 +0000 Received: from mail-pf1-x443.google.com ([2607:f8b0:4864:20::443]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMt3-0004Hg-HP for linux-riscv@lists.infradead.org; Mon, 09 Dec 2019 17:31:59 +0000 Received: by mail-pf1-x443.google.com with SMTP id b19so7590114pfo.2 for ; Mon, 09 Dec 2019 09:31:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ldUAi30p4Fm9YFcB6XPp82qK2Gx4LpczH2It45srXeo=; b=I2wCeTTo+LKa1bhU4VfmVyMc9Cd30NbyeEDRCtVDaC5dxtYFpo1WOKYY95ZDtYBt77 iLg4li5iVJPHBpqiaviSog5MP82WrRWIN/U2NuUoXiE3R3G05In5bf47/BcNlaYuSw5r TY7FoFnUt8LsxJwW7Golo15Zr7pxD2JDCB/F7YFcStshkWcBbf14zYnOq9CyNkKkyUYL GQtBNYFSfB+mL85SqgoiKbPnOcD0xRd6re5WemLJ97anofCxlu2xl7NAKKtUT1ksbCmL TiLKe8bVYr+vQoalNaG2xu/HojCwUu3u1UYd2zktXNHybg6SAW1BJNBwyv8xjNIasv9G RMIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ldUAi30p4Fm9YFcB6XPp82qK2Gx4LpczH2It45srXeo=; b=ZymhZw2mKAStsuO+2xy7LZnP9/JYlZb1xLDJH8zMKKTNgQRBnNduXcomhL2fKuwKN9 tnZJu5oDjEHoEsnUu+vFEBVH5uZIV6dViUyjR9UB807KjwHMTQkyxRHF8dag0PaEgIeQ h4ErZ6gCv3adVQTYfAksA5tH6GN/xY3FzlDH1EHsI38OirHG+3OlTbsWTlocuNIwFPZb co3ZhqG5ZoddbZ4m1yjJY3WSP4zlkWSslxZ6rjEMeu5VqmAa8ZJh3drBrKbgj1Lsv+6H myStUNvBTYLuEbSTXF0qNEVcsGuuldPdBYhyXTgFzlROVNjWOgE1ov7IvRyX4EyP9pwr ub2w== X-Gm-Message-State: APjAAAXJUepl+8lwtllDEox5mH5A8bxnku83EfuTdGyxzuCHTU5VWPsX fH+aqeMuIOPsWrry8MUMBBg= X-Google-Smtp-Source: APXvYqwXA7UudhebZ3PjOLU7xMSYDqHPA3gpUTrpfjHhuIm1CEOoB/cMYEqKSVi2rCrmP7SQU/ckUQ== X-Received: by 2002:a63:e14b:: with SMTP id h11mr19248346pgk.297.1575912716328; Mon, 09 Dec 2019 09:31:56 -0800 (PST) Received: from btopel-mobl.ger.intel.com ([192.55.55.41]) by smtp.gmail.com with ESMTPSA id d23sm54943pfo.176.2019.12.09.09.31.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 09:31:55 -0800 (PST) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: daniel@iogearbox.net, ast@kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 3/8] riscv, bpf: add support for far jumps and exits Date: Mon, 9 Dec 2019 18:31:31 +0100 Message-Id: <20191209173136.29615-4-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191209173136.29615-1-bjorn.topel@gmail.com> References: <20191209173136.29615-1-bjorn.topel@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191209_093157_593708_5A6118A6 X-CRM114-Status: GOOD ( 11.12 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:443 listed in] [list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (bjorn.topel[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-riscv@lists.infradead.org, bpf@vger.kernel.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org This commit add support for far (offset > 21b) jumps and exits. Signed-off-by: Björn Töpel Reviewed-by: Luke Nelson --- arch/riscv/net/bpf_jit_comp.c | 37 ++++++++++++++++------------------- 1 file changed, 17 insertions(+), 20 deletions(-) diff --git a/arch/riscv/net/bpf_jit_comp.c b/arch/riscv/net/bpf_jit_comp.c index e2f6724b64f7..e9cc9832ac2c 100644 --- a/arch/riscv/net/bpf_jit_comp.c +++ b/arch/riscv/net/bpf_jit_comp.c @@ -506,16 +506,6 @@ static int is_13b_check(int off, int insn) return 0; } -static int is_21b_check(int off, int insn) -{ - if (!is_21b_int(off)) { - pr_err("bpf-jit: insn=%d 21b < offset=%d not supported yet!\n", - insn, (int)off); - return -1; - } - return 0; -} - static void emit_imm(u8 rd, s64 val, struct rv_jit_context *ctx) { /* Note that the immediate from the add is sign-extended, @@ -733,6 +723,21 @@ static void emit_sext_32_rd(u8 *rd, struct rv_jit_context *ctx) *rd = RV_REG_T2; } +static void emit_jump_and_link(u8 rd, int rvoff, struct rv_jit_context *ctx) +{ + s64 upper, lower; + + if (is_21b_int(rvoff)) { + emit(rv_jal(rd, rvoff >> 1), ctx); + return; + } + + upper = (rvoff + (1 << 11)) >> 12; + lower = rvoff & 0xfff; + emit(rv_auipc(RV_REG_T1, upper), ctx); + emit(rv_jalr(rd, RV_REG_T1, lower), ctx); +} + /* return -1 or inverted cond */ static int invert_bpf_cond(u8 cond) { @@ -1117,13 +1122,7 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, /* JUMP off */ case BPF_JMP | BPF_JA: rvoff = rv_offset(i, off, ctx); - if (!is_21b_int(rvoff)) { - pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n", - i, rvoff); - return -1; - } - - emit(rv_jal(RV_REG_ZERO, rvoff >> 1), ctx); + emit_jump_and_link(RV_REG_ZERO, rvoff, ctx); break; /* IF (dst COND src) JUMP off */ @@ -1261,9 +1260,7 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, break; rvoff = epilogue_offset(ctx); - if (is_21b_check(rvoff, i)) - return -1; - emit(rv_jal(RV_REG_ZERO, rvoff >> 1), ctx); + emit_jump_and_link(RV_REG_ZERO, rvoff, ctx); break; /* dst = imm64 */ From patchwork Mon Dec 9 17:31:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11279757 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 526A3930 for ; Mon, 9 Dec 2019 17:32:05 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 304F720828 for ; Mon, 9 Dec 2019 17:32:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WuzMQxfU"; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sRk23x/J" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 304F720828 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Q72cQM+3SI70k7ctcpb9LKDM8a7LAen8E8Vv2d4hTLo=; b=WuzMQxfUzXKsGW E2RnMQx8Sog0LpaWZSZLrfGmT6xWG008zU7sLvR0u5CCuW18/omfEKoKmlh5EVTcMGVZp41g/r6UD w/iD6JAssfF8zIru3QwM/ZmqKe4emvE5MkrZdeTW3lSH5Kb+SEJvg7XEUVFWS2v/LK/xD29WmMg7w 46+/IG8OUfGhvEa27sXORMXhnkL/zU1Gi6pAHLz4zZ915aEm6P5Rt/Ndbl2Oi3WfIxA1KSh1yCC3U US4uIORrePpWGVbA+N9FZb/dto107gESWOi76GFqb/hJf8nz3ZlJl06G/RIazTGO78aJjYmA9IyBQ 7a6438C0k8fS91j7g8qA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMt9-0004Om-Tp; Mon, 09 Dec 2019 17:32:03 +0000 Received: from mail-pf1-x441.google.com ([2607:f8b0:4864:20::441]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMt5-0004Jf-Us for linux-riscv@lists.infradead.org; Mon, 09 Dec 2019 17:32:01 +0000 Received: by mail-pf1-x441.google.com with SMTP id h14so7570464pfe.10 for ; Mon, 09 Dec 2019 09:31:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Q72cQM+3SI70k7ctcpb9LKDM8a7LAen8E8Vv2d4hTLo=; b=sRk23x/JpxVKbzZj54qVRUi2aIExQ0SRE7GqFLD/hMDxEVsFavaeMfS44RU6oiuXfb H0xi5LdEkduZ2iYCJYodCJF6Ing7Rv1IW8FMsyWVB2+sCdRzMHiUhKXmMg3xSMH6nHsX ayoDJZab2LVWXwXub2wg/AlQtXMlEvPjhk8JTyMAKviCRlRECgURo0yOcgepZ2IDOykr W3UsS2Gn8bgOe6SddLX7KSOBEYFK9IYdxMuaWv45tRXpfXQSmLLWywYTazar2mRObXn5 1loXAYvQ7kzXfJ7qWWqSuEMjX6mcN+A83MnD5l6sHk/M+WOUax9WPdqqaHvnilPlsfcj gSCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Q72cQM+3SI70k7ctcpb9LKDM8a7LAen8E8Vv2d4hTLo=; b=bCi+vUSOaFDC5inZaJsVAuqQ1T/wTlgLtRK2Z9l15oOUmXKYjnJTLB4vnHD1FiWAiu 91FlMt5n5AIjspDC8KxbvhS0euMxDsWRK8PjqlOXL9CdbotPKr/idMYznjojkBfoLmNy rMZlSl5Ku8UgA1waxAduJX/XP8ltL8W0kue0NtWL742gSbEwBswnqVGE3n5MVAOahdpv 9Q5JV8stjDOsLa5sSQGzMjkzBc+DhfAJ7aPnOKbZeLPyDabydkvs0mSvegEvz5hV3WWj RA+aQEbGM3Zxs8lXW07EuVxMcI7rMRySy24IBeupNyI1vicuYyabplTW4WbVzZnd3XWd oOcQ== X-Gm-Message-State: APjAAAVKJomFIw6bTxxLODBRgM9TlTBKbhJUigkxYFVnYXPfbonIyvq+ DxD1iTTZCQl2w7YsojnhSMI= X-Google-Smtp-Source: APXvYqxlMvkVuCdFoizkUk0kiP7e9NabguDeK3+5YeBoqX6bG5u2+aRmcgQwBwQtLnS2H8IY30slww== X-Received: by 2002:a63:d501:: with SMTP id c1mr19227970pgg.356.1575912719116; Mon, 09 Dec 2019 09:31:59 -0800 (PST) Received: from btopel-mobl.ger.intel.com ([192.55.55.41]) by smtp.gmail.com with ESMTPSA id d23sm54943pfo.176.2019.12.09.09.31.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 09:31:58 -0800 (PST) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: daniel@iogearbox.net, ast@kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 4/8] riscv, bpf: optimize BPF tail calls Date: Mon, 9 Dec 2019 18:31:32 +0100 Message-Id: <20191209173136.29615-5-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191209173136.29615-1-bjorn.topel@gmail.com> References: <20191209173136.29615-1-bjorn.topel@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191209_093200_037154_32676E6E X-CRM114-Status: UNSURE ( 9.66 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:441 listed in] [list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (bjorn.topel[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-riscv@lists.infradead.org, bpf@vger.kernel.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Remove one addi, and instead use the offset part of jalr. Signed-off-by: Björn Töpel --- arch/riscv/net/bpf_jit_comp.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/riscv/net/bpf_jit_comp.c b/arch/riscv/net/bpf_jit_comp.c index e9cc9832ac2c..cbcb33613d1d 100644 --- a/arch/riscv/net/bpf_jit_comp.c +++ b/arch/riscv/net/bpf_jit_comp.c @@ -562,7 +562,7 @@ static int epilogue_offset(struct rv_jit_context *ctx) return (to - from) << 2; } -static void __build_epilogue(u8 reg, struct rv_jit_context *ctx) +static void __build_epilogue(bool is_tail_call, struct rv_jit_context *ctx) { int stack_adjust = ctx->stack_size, store_offset = stack_adjust - 8; @@ -599,9 +599,11 @@ static void __build_epilogue(u8 reg, struct rv_jit_context *ctx) emit(rv_addi(RV_REG_SP, RV_REG_SP, stack_adjust), ctx); /* Set return value. */ - if (reg == RV_REG_RA) + if (!is_tail_call) emit(rv_addi(RV_REG_A0, RV_REG_A5, 0), ctx); - emit(rv_jalr(RV_REG_ZERO, reg, 0), ctx); + emit(rv_jalr(RV_REG_ZERO, is_tail_call ? RV_REG_T3 : RV_REG_RA, + is_tail_call ? 4 : 0), /* skip TCC init */ + ctx); } static void emit_zext_32(u8 reg, struct rv_jit_context *ctx) @@ -664,9 +666,8 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) if (is_12b_check(off, insn)) return -1; emit(rv_ld(RV_REG_T3, off, RV_REG_T2), ctx); - emit(rv_addi(RV_REG_T3, RV_REG_T3, 4), ctx); emit(rv_addi(RV_REG_TCC, RV_REG_T1, 0), ctx); - __build_epilogue(RV_REG_T3, ctx); + __build_epilogue(true, ctx); return 0; } @@ -1520,7 +1521,7 @@ static void build_prologue(struct rv_jit_context *ctx) static void build_epilogue(struct rv_jit_context *ctx) { - __build_epilogue(RV_REG_RA, ctx); + __build_epilogue(false, ctx); } static int build_body(struct rv_jit_context *ctx, bool extra_pass) From patchwork Mon Dec 9 17:31:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11279759 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 286A1112B for ; Mon, 9 Dec 2019 17:32:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 062ED20828 for ; Mon, 9 Dec 2019 17:32:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="SNJ45MAl"; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b7syrMdQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 062ED20828 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BNCw8ZLuRZFhPijRpMqXTUSJYcWT9YK5uJt+c/1wpxE=; b=SNJ45MAl3m6YRH kuhQpXHyLdMWjPOIkFNkDzv8SPMDnK0CslbQO8s0TaDGG4/FyJWiP01FPwXUhiLPzInEFbJ7mnUbj ZnpF4gitY+MuDMalAMzcb1qsoHpQRVTwK5Wa2CTwsup+lV2s3AnakYpVmS7bcup44C6KOLabfLDyF KByDzfw6sDoLue1X9LCcumzPOKKkKMKdvOVoit4WfyIRcR/zEmmtNOmPIGstaqGm49hXcplHK2BgM f2eMVnYrEowVG/+ZcGp/8wtPW9T0Bp+2iEiqzXcjbO5P6IsSGyvRB0dJLpEGyMFxauMwHSpzHp0eC jPJFjyo4fm55Mw8t5AHQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMtE-0004UE-Jw; Mon, 09 Dec 2019 17:32:08 +0000 Received: from mail-pj1-x1042.google.com ([2607:f8b0:4864:20::1042]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMt8-0004Mh-On for linux-riscv@lists.infradead.org; Mon, 09 Dec 2019 17:32:04 +0000 Received: by mail-pj1-x1042.google.com with SMTP id r67so6196310pjb.0 for ; Mon, 09 Dec 2019 09:32:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BNCw8ZLuRZFhPijRpMqXTUSJYcWT9YK5uJt+c/1wpxE=; b=b7syrMdQLRdmJqKeslHa0OrbiFCOHk5VEYemE7weyZnuuaYVznuyKpmWFBR8lyo6kP yGmjhggOh9emFV8hukxxUpWO1eii90aRClJVfM7b+bwsZBAxhOekqV+ukncrnYvhLtXM pCvumWBiO2ELk58wvN8WvIMTYRvRWOvF7GZyuRDYJ5Clqxuyyn1B3JCD4t6XxmJHrR1r Fz26XQj+ini66Hogzfc9faAaols2T66ku7hzmKlr4OHhgfu714PHeP2dFh6e5wlblZ3r 3e+yAs/fr3GQ5T0blwgwCAXTcETIIN3AHfPAqNQSVNCEk1FuZIk1YwLFzYD8KyLz7EEY 9xQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BNCw8ZLuRZFhPijRpMqXTUSJYcWT9YK5uJt+c/1wpxE=; b=VOfkS6ge6MbQzLHqrK7FbUhzwh2WRB/jHqjsbX1oc6T+K5Lb3tWbWG74rSWnm+AuCe /WiYUMb9GWx5vzgTmciFEysxj2pZDaXcIdYm+JtKyewDec15144vIpe2z4uepWb9rojx ubY97N/BuD6E6UjLmCtehTzWN+PmW0xhi8FLzBV52v3g9tQjC3jMxVPOreRrnHEmBMUK hTlLJmoypNXlYoM+5H4OoMg8wrKua/x7egPXEWPdRC3KvyW4YhGV26X8w+wjA0E01PUi bxQNb5u1udUs/8f3QOqtXAXAKqBpVQ/3UNlVBqMuP6s59kQU93KmwUIOQebZ+ogw1cPM Ue8Q== X-Gm-Message-State: APjAAAW3qqKL3hkbW9w+Fbcd0jXCDZiIgTdRiFjKdGA1TNhoDdCHvJVS ySfb+Ezx10N5bqvVniGw+O8= X-Google-Smtp-Source: APXvYqy9HFzdcebDinK4pQ1i8ARP6HChfcwpNNcw7Sjeo3SRHNx3mFUKRIy77RI0aIokZfEt2wuL+w== X-Received: by 2002:a17:902:8a97:: with SMTP id p23mr30685730plo.233.1575912721683; Mon, 09 Dec 2019 09:32:01 -0800 (PST) Received: from btopel-mobl.ger.intel.com ([192.55.55.41]) by smtp.gmail.com with ESMTPSA id d23sm54943pfo.176.2019.12.09.09.31.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 09:32:01 -0800 (PST) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: daniel@iogearbox.net, ast@kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 5/8] riscv, bpf: provide RISC-V specific JIT image alloc/free Date: Mon, 9 Dec 2019 18:31:33 +0100 Message-Id: <20191209173136.29615-6-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191209173136.29615-1-bjorn.topel@gmail.com> References: <20191209173136.29615-1-bjorn.topel@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191209_093202_823500_D999C25E X-CRM114-Status: UNSURE ( 9.08 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (bjorn.topel[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-riscv@lists.infradead.org, bpf@vger.kernel.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org This commit makes sure that the JIT images is kept close to the kernel text, so BPF calls can use relative calling with auipc/jalr or jal instead of loading the full 64-bit address and jalr. The BPF JIT image region is 128 MB before the kernel text. Signed-off-by: Björn Töpel --- arch/riscv/include/asm/pgtable.h | 4 ++++ arch/riscv/net/bpf_jit_comp.c | 13 +++++++++++++ 2 files changed, 17 insertions(+) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 7ff0ed4f292e..cc3f49415620 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -404,6 +404,10 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, #define VMALLOC_END (PAGE_OFFSET - 1) #define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) +#define BPF_JIT_REGION_SIZE (SZ_128M) +#define BPF_JIT_REGION_START (PAGE_OFFSET - BPF_JIT_REGION_SIZE) +#define BPF_JIT_REGION_END (VMALLOC_END) + /* * Roughly size the vmemmap space to be large enough to fit enough * struct pages to map half the virtual address space. Then diff --git a/arch/riscv/net/bpf_jit_comp.c b/arch/riscv/net/bpf_jit_comp.c index cbcb33613d1d..f7b1ae3a968f 100644 --- a/arch/riscv/net/bpf_jit_comp.c +++ b/arch/riscv/net/bpf_jit_comp.c @@ -1672,3 +1672,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) tmp : orig_prog); return prog; } + +void *bpf_jit_alloc_exec(unsigned long size) +{ + return __vmalloc_node_range(size, PAGE_SIZE, BPF_JIT_REGION_START, + BPF_JIT_REGION_END, GFP_KERNEL, + PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, + __builtin_return_address(0)); +} + +void bpf_jit_free_exec(void *addr) +{ + return vfree(addr); +} From patchwork Mon Dec 9 17:31:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11279761 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 887E8930 for ; Mon, 9 Dec 2019 17:32:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 666E020828 for ; Mon, 9 Dec 2019 17:32:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="fIq12OR8"; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dHqonQVZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 666E020828 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+poYiv+8PtDbY0kklYLAZ46MuOoXr5bUTyoS0TPIaGc=; b=fIq12OR8rI1qEs 55MmRBhOCg6udj4wuGjTBQyA04K4o04JpTc/h76kkm/3YSF9dUv0u+8V/ETmBlpucjDZg8nKOLNIz 9lRyj+Y2YuxXDEodKHF686OnCw4ezuuSMZE0zPuRWfJ9DnV6j25dyAwmlCh1HoDjHO1MZ4FAIj3aP PqJTYdTCDF6TWzkUDd0qoZ/y7hg0AS9j1zA5Yf6zUlGxJReJZTYnoOxpsoXA8kOTwabP+Twhbschw XOeOsCjI1s3d2Bs9HBsvKyLd837rfX6V0pbfLu6x+sjExIZbgflPyuG2Sqn/3mMxuAkVk2GUy8p84 TQHljnX0ZD+LSjFjxaqg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMtH-0004YB-VW; Mon, 09 Dec 2019 17:32:11 +0000 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMtB-0004RF-UN for linux-riscv@lists.infradead.org; Mon, 09 Dec 2019 17:32:07 +0000 Received: by mail-pf1-x444.google.com with SMTP id q8so7573268pfh.7 for ; Mon, 09 Dec 2019 09:32:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+poYiv+8PtDbY0kklYLAZ46MuOoXr5bUTyoS0TPIaGc=; b=dHqonQVZ3ujeSFRJMFtSGpE2eEwMEk45rbKxlpuuJmALXtzb+jILOwh6xuPC+D5KyN NZzpDTQIapqhXwCycBNG1+J5rVbbXoKHAdduigDRySfVkNxyFuVqN1xb7TIc2AzOrdKl aiorNkA4ycTHePbaALmZ+GE6KiAQHoTwCgB2KE+BFWdAXn3BtFQJkRqzkVo/NoBFIO8S 2FOnSdPLojp1MIarTbxJInL5vb3w/JWZIiBL9xmHuXjGACR1qRmK6oaSDPH2jnWns4XQ 36h5HQ0H/5wL3qwfjyOgJvAMoswfbDmWAUTueNFLXcQ37OkygjwjgVIN13djZeKbo8r9 chXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+poYiv+8PtDbY0kklYLAZ46MuOoXr5bUTyoS0TPIaGc=; b=AYlO6CelUCdIYRycNx2hIUm6YJ/B0oxEbONubVHk1S86CWx1v6brskpxIsYWSNUJ3t BsbYO4xHR977ccK2iIZ0IOOxOZZa9qYW/zDz5BGlliV/81F9aE9lxu8a6H15kBGrp48W xOJ3nNxa4NroccdT0+tIPDn1uzlSUXZ+jD6V/LnI7O4PfptibPoZkI6ymluga11kz9iD 0AcOPSXKHL5AH533MzlJB4sCfJ+8jCwHMPH8N8bEgACCXO7QRI73VW82PWVhlfS+FVGB MvOBMxp0HZX51vB1sHz72BgxiNVdHlClOt6U5XRrXVsmWTA0xcnQo/+wZUhjng3CWEqg 5nRQ== X-Gm-Message-State: APjAAAVf+9zCYKdxJLVSnhMGz/1a40nf6VHP6dRdfLRSy7NO42jSr/VM GaABYNVxIyuiRlv7lmYJP18= X-Google-Smtp-Source: APXvYqyGewL1ziYSiaxyGVrbvZGP4Pfkl6VVi14TKt9OHRmgsGBv1ENpkokaLMh/72xQ5D162f9Gqw== X-Received: by 2002:aa7:9af1:: with SMTP id y17mr30579052pfp.21.1575912724538; Mon, 09 Dec 2019 09:32:04 -0800 (PST) Received: from btopel-mobl.ger.intel.com ([192.55.55.41]) by smtp.gmail.com with ESMTPSA id d23sm54943pfo.176.2019.12.09.09.32.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 09:32:04 -0800 (PST) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: daniel@iogearbox.net, ast@kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 6/8] riscv, bpf: optimize calls Date: Mon, 9 Dec 2019 18:31:34 +0100 Message-Id: <20191209173136.29615-7-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191209173136.29615-1-bjorn.topel@gmail.com> References: <20191209173136.29615-1-bjorn.topel@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191209_093206_016882_4491BD26 X-CRM114-Status: GOOD ( 15.82 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:444 listed in] [list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (bjorn.topel[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-riscv@lists.infradead.org, bpf@vger.kernel.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Instead of using emit_imm() and emit_jalr() which can expand to six instructions, start using jal or auipc+jalr. Signed-off-by: Björn Töpel --- arch/riscv/net/bpf_jit_comp.c | 101 +++++++++++++++++++++------------- 1 file changed, 64 insertions(+), 37 deletions(-) diff --git a/arch/riscv/net/bpf_jit_comp.c b/arch/riscv/net/bpf_jit_comp.c index f7b1ae3a968f..260df0a87fd3 100644 --- a/arch/riscv/net/bpf_jit_comp.c +++ b/arch/riscv/net/bpf_jit_comp.c @@ -724,11 +724,12 @@ static void emit_sext_32_rd(u8 *rd, struct rv_jit_context *ctx) *rd = RV_REG_T2; } -static void emit_jump_and_link(u8 rd, int rvoff, struct rv_jit_context *ctx) +static void emit_jump_and_link(u8 rd, s64 rvoff, bool force_jalr, + struct rv_jit_context *ctx) { s64 upper, lower; - if (is_21b_int(rvoff)) { + if (rvoff && is_21b_int(rvoff) && !force_jalr) { emit(rv_jal(rd, rvoff >> 1), ctx); return; } @@ -848,6 +849,28 @@ static bool is_signed_bpf_cond(u8 cond) cond == BPF_JSGE || cond == BPF_JSLE; } +static int emit_call(bool fixed, u64 addr, struct rv_jit_context *ctx) +{ + s64 off = 0; + u64 ip; + u8 rd; + + if (addr && ctx->insns) { + ip = (u64)(long)(ctx->insns + ctx->ninsns); + off = addr - ip; + if (!is_32b_int(off)) { + pr_err("bpf-jit: target call addr %pK is out of range\n", + (void *)addr); + return -ERANGE; + } + } + + emit_jump_and_link(RV_REG_RA, off, !fixed, ctx); + rd = bpf_to_rv_reg(BPF_REG_0, ctx); + emit(rv_addi(rd, RV_REG_A0, 0), ctx); + return 0; +} + static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, bool extra_pass) { @@ -1123,7 +1146,7 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, /* JUMP off */ case BPF_JMP | BPF_JA: rvoff = rv_offset(i, off, ctx); - emit_jump_and_link(RV_REG_ZERO, rvoff, ctx); + emit_jump_and_link(RV_REG_ZERO, rvoff, false, ctx); break; /* IF (dst COND src) JUMP off */ @@ -1225,7 +1248,7 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, case BPF_JMP | BPF_CALL: { bool fixed; - int i, ret; + int ret; u64 addr; mark_call(ctx); @@ -1233,20 +1256,9 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, &fixed); if (ret < 0) return ret; - if (fixed) { - emit_imm(RV_REG_T1, addr, ctx); - } else { - i = ctx->ninsns; - emit_imm(RV_REG_T1, addr, ctx); - for (i = ctx->ninsns - i; i < 8; i++) { - /* nop */ - emit(rv_addi(RV_REG_ZERO, RV_REG_ZERO, 0), - ctx); - } - } - emit(rv_jalr(RV_REG_RA, RV_REG_T1, 0), ctx); - rd = bpf_to_rv_reg(BPF_REG_0, ctx); - emit(rv_addi(rd, RV_REG_A0, 0), ctx); + ret = emit_call(fixed, addr, ctx); + if (ret) + return ret; break; } /* tail call */ @@ -1261,7 +1273,7 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, break; rvoff = epilogue_offset(ctx); - emit_jump_and_link(RV_REG_ZERO, rvoff, ctx); + emit_jump_and_link(RV_REG_ZERO, rvoff, false, ctx); break; /* dst = imm64 */ @@ -1524,7 +1536,7 @@ static void build_epilogue(struct rv_jit_context *ctx) __build_epilogue(false, ctx); } -static int build_body(struct rv_jit_context *ctx, bool extra_pass) +static int build_body(struct rv_jit_context *ctx, bool extra_pass, int *offset) { const struct bpf_prog *prog = ctx->prog; int i; @@ -1536,12 +1548,12 @@ static int build_body(struct rv_jit_context *ctx, bool extra_pass) ret = emit_insn(insn, ctx, extra_pass); if (ret > 0) { i++; - if (ctx->insns == NULL) - ctx->offset[i] = ctx->ninsns; + if (offset) + offset[i] = ctx->ninsns; continue; } - if (ctx->insns == NULL) - ctx->offset[i] = ctx->ninsns; + if (offset) + offset[i] = ctx->ninsns; if (ret) return ret; } @@ -1569,8 +1581,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) struct bpf_prog *tmp, *orig_prog = prog; int pass = 0, prev_ninsns = 0, i; struct rv_jit_data *jit_data; + unsigned int image_size = 0; struct rv_jit_context *ctx; - unsigned int image_size; if (!prog->jit_requested) return orig_prog; @@ -1615,36 +1627,51 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) for (i = 0; i < 16; i++) { pass++; ctx->ninsns = 0; - if (build_body(ctx, extra_pass)) { + if (build_body(ctx, extra_pass, ctx->offset)) { prog = orig_prog; goto out_offset; } build_prologue(ctx); ctx->epilogue_offset = ctx->ninsns; build_epilogue(ctx); - if (ctx->ninsns == prev_ninsns) - break; + + if (ctx->ninsns == prev_ninsns) { + if (jit_data->header) + break; + + image_size = sizeof(u32) * ctx->ninsns; + jit_data->header = + bpf_jit_binary_alloc(image_size, + &jit_data->image, + sizeof(u32), + bpf_fill_ill_insns); + if (!jit_data->header) { + prog = orig_prog; + goto out_offset; + } + + ctx->insns = (u32 *)jit_data->image; + /* Now, when the image is allocated, the image + * can potentially shrink more (auipc/jalr -> + * jal). + */ + } prev_ninsns = ctx->ninsns; } - /* Allocate image, now that we know the size. */ - image_size = sizeof(u32) * ctx->ninsns; - jit_data->header = bpf_jit_binary_alloc(image_size, &jit_data->image, - sizeof(u32), - bpf_fill_ill_insns); - if (!jit_data->header) { + if (i == 16) { + pr_err("bpf-jit: image did not converge in <%d passes!\n", i); + bpf_jit_binary_free(jit_data->header); prog = orig_prog; goto out_offset; } - /* Second, real pass, that acutally emits the image. */ - ctx->insns = (u32 *)jit_data->image; skip_init_ctx: pass++; ctx->ninsns = 0; build_prologue(ctx); - if (build_body(ctx, extra_pass)) { + if (build_body(ctx, extra_pass, NULL)) { bpf_jit_binary_free(jit_data->header); prog = orig_prog; goto out_offset; From patchwork Mon Dec 9 17:31:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11279763 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C6383930 for ; Mon, 9 Dec 2019 17:32:16 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A4BA9208C3 for ; Mon, 9 Dec 2019 17:32:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="A3hF4I7p"; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="vOpqIJ6q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A4BA9208C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qigMm4EqiyANinCkFguk39Cl6HfqkkQUc4cgfeXf+Bk=; b=A3hF4I7pkaeqnz N2WczWHz39fcxLhLDyLgtIS08s4fz0BsX88tn+zhBU4k2cBZDyWrk2LEcrA8Q3arRfIpB2YswnoyY 135AladjWW//mnq0CXxDQfCm7GYpDdOQpPnbrjvTFB7U/7P1yYJqmHsLEyiXHknF06NK5sxHrSnyZ mItx4a86EsVzFwZOumM/7DL16mXiydD5sdH8SRHyZfet1GJT9gdNUKeAZJUTbfFuG47kCDfY4Ny2O P0JMm2ORseNHZHdPlyyxiQIHS20bEOYPWq8bUxP3cijzWKYyKJYsdBtVPwzyN8CHH0zXjGkOgIZX+ ELmdbQIpMtKWA1m8hw/g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMtL-0004bc-A8; Mon, 09 Dec 2019 17:32:15 +0000 Received: from mail-pj1-x1042.google.com ([2607:f8b0:4864:20::1042]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMtE-0004Tr-1L for linux-riscv@lists.infradead.org; Mon, 09 Dec 2019 17:32:09 +0000 Received: by mail-pj1-x1042.google.com with SMTP id z21so6166291pjq.13 for ; Mon, 09 Dec 2019 09:32:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qigMm4EqiyANinCkFguk39Cl6HfqkkQUc4cgfeXf+Bk=; b=vOpqIJ6q75D7Put1Ibs+aBj9GoAdDjQOk0kztitX4aNnyO/uS5BEGTuVOgzGw3NuBq JhC61Pb0KNlCTr4SsK7zAwkA3PhZlEAIDpK9DHRPY9T8ODSc9+HqQ7xpE8Ta/uMp8QUW qHTBgLILsfFr+3JqxiF26oCYVBtFk2gGrj72WTrXW3GF5KB0cxTJi0xpP/oy4drKf1vA ZxqCN+fhD3HZ716kGXZCnQqRhyvSGhQg8vPxDf4XLKKz8GX+t7jFrKyhuLlHH/z8klwd Y91XRDVJEquj55w7xMkcNWqqJnBOyzRGmO3lrm7tXbKsPFW1J+nIqrjeMe6b3wBgC90K Ogaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qigMm4EqiyANinCkFguk39Cl6HfqkkQUc4cgfeXf+Bk=; b=jgM/YJJ3VR392MCowmPfqRpZ/cKiFL6iAgwUs6/amEuSCe5nD06wzFOkW6FvrK9A2/ cUXFd2Zl9/loXpluSHW9A4TL0UWmFruWAlR68RqbydZTo7ZjAdqMhpxlRFoX3bp0enud YzCCZUZZqkj+1LbXzUJw0N6ltfriWXmCnjtpZQFTjZqBj9mQb2elz7Y9Fcxadyn0Mx14 BRtA0SQLOeUqS/MW8ZopbI4L+iwL0Be/jtNBGEJuCiOizJx4EIZB1aDrXt1ZnOuU2Des CEuHBLzSQk04ZhZ9mwy12Q2qvLSlxatu843CieDc4PvngZ7URdig8PF82pv/qT3p62U6 4BpQ== X-Gm-Message-State: APjAAAVkEPGKlM2sGYBBsu6ES99MJp0vx+4kf2xcUQVf087MGlYyfgRF d+Zgh9QdRI/NUADrVc8G0V4= X-Google-Smtp-Source: APXvYqyWxdQ0hJhMJ0rThrltp9ujToYcSdgDSDZcAlBK2cFpYYNKVR4Jr1SX1fsoeMJ7K9c2xhIJGA== X-Received: by 2002:a17:902:ac8b:: with SMTP id h11mr30749482plr.87.1575912727095; Mon, 09 Dec 2019 09:32:07 -0800 (PST) Received: from btopel-mobl.ger.intel.com ([192.55.55.41]) by smtp.gmail.com with ESMTPSA id d23sm54943pfo.176.2019.12.09.09.32.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 09:32:06 -0800 (PST) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: daniel@iogearbox.net, ast@kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 7/8] riscv, bpf: add missing uapi header for BPF_PROG_TYPE_PERF_EVENT programs Date: Mon, 9 Dec 2019 18:31:35 +0100 Message-Id: <20191209173136.29615-8-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191209173136.29615-1-bjorn.topel@gmail.com> References: <20191209173136.29615-1-bjorn.topel@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191209_093208_138486_44CC1B8D X-CRM114-Status: UNSURE ( 9.43 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (bjorn.topel[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-riscv@lists.infradead.org, bpf@vger.kernel.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Add missing uapi header the BPF_PROG_TYPE_PERF_EVENT programs by exporting struct user_regs_struct instead of struct pt_regs which is in-kernel only. Signed-off-by: Björn Töpel --- arch/riscv/include/uapi/asm/bpf_perf_event.h | 9 +++++++++ tools/include/uapi/asm/bpf_perf_event.h | 2 ++ 2 files changed, 11 insertions(+) create mode 100644 arch/riscv/include/uapi/asm/bpf_perf_event.h diff --git a/arch/riscv/include/uapi/asm/bpf_perf_event.h b/arch/riscv/include/uapi/asm/bpf_perf_event.h new file mode 100644 index 000000000000..6cb1c2823288 --- /dev/null +++ b/arch/riscv/include/uapi/asm/bpf_perf_event.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _UAPI__ASM_BPF_PERF_EVENT_H__ +#define _UAPI__ASM_BPF_PERF_EVENT_H__ + +#include + +typedef struct user_regs_struct bpf_user_pt_regs_t; + +#endif /* _UAPI__ASM_BPF_PERF_EVENT_H__ */ diff --git a/tools/include/uapi/asm/bpf_perf_event.h b/tools/include/uapi/asm/bpf_perf_event.h index 13a58531e6fa..39acc149d843 100644 --- a/tools/include/uapi/asm/bpf_perf_event.h +++ b/tools/include/uapi/asm/bpf_perf_event.h @@ -2,6 +2,8 @@ #include "../../arch/arm64/include/uapi/asm/bpf_perf_event.h" #elif defined(__s390__) #include "../../arch/s390/include/uapi/asm/bpf_perf_event.h" +#elif defined(__riscv) +#include "../../arch/riscv/include/uapi/asm/bpf_perf_event.h" #else #include #endif From patchwork Mon Dec 9 17:31:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11279765 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C952D930 for ; Mon, 9 Dec 2019 17:32:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 93B9B214AF for ; Mon, 9 Dec 2019 17:32:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="bFyJrnza"; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="uS7kguZb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93B9B214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JJx1gb7dyIux/wvtlAjUM4Ul67VDqBxG4kjHz1T/apg=; b=bFyJrnzawGwPbW lmy8lSSk7lbtHtqidIqEeaqF3SVJsnqepZwYGsZj5GKp+m789hd/KC0W2lPcjpqrknpO7pi0DGRAE ZeeltCAvzdXKLbVC1OVo3gt/DB4EXO1OGYiL7YWf/Qc59W7GauVVmolrSfXvw76l1oKTV7kal+Lrd PWBPhOPlVXpCnGQlmxN+DOkghhFSp3y2i8Tra/RbVixfezpnz1nlRRsJwzdq8RqciT0WPNX2jxpgs 43EkW9g7yYf12qLvoeqCmr4GY7SCwmFsAzHRrF3UZxPOCwOUu9mYGWdW4OVpaFaftliEjeaQXcFpk dNxbfjQA490LYNbFxUag==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMtN-0004ey-OC; Mon, 09 Dec 2019 17:32:17 +0000 Received: from mail-pj1-x1043.google.com ([2607:f8b0:4864:20::1043]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ieMtG-0004WZ-PN for linux-riscv@lists.infradead.org; Mon, 09 Dec 2019 17:32:12 +0000 Received: by mail-pj1-x1043.google.com with SMTP id o11so6180344pjp.9 for ; Mon, 09 Dec 2019 09:32:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JJx1gb7dyIux/wvtlAjUM4Ul67VDqBxG4kjHz1T/apg=; b=uS7kguZbewotiewe3eNEHiAYQ1NBG4BmutBfowVjnF3VSkL7gmTKFZI1qF19mjsr7U 5Ymr+omBiK4cAUBfSnV7+LszdqA+5WKxmsJQPsh1nWll88ySD6QpGmUEVspEOECCoazn ksCs2t465xw2OIIjFykCvuSH2f/l6AHW15B/bMSIXzYmku7qq5PQJxVBwFHHqEEEIJf2 lcBEpY62ERPEUolysJgD7ieIuyWWFH6st04PLD881a9iHhTgiq1ezMCQy+k7i9Kccdxs Xy0jnm0HdR+Ci0bYEumQQ43R2A8zgVWjnPvVyqCmlkjf9AE0SgiTdy2UDK2cx9hPyCgq qhnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JJx1gb7dyIux/wvtlAjUM4Ul67VDqBxG4kjHz1T/apg=; b=kbdyHsDmYmETwSEO90/8xnGHkg1Kvz5enwJz+kq1tNblOhlparYqwE7w5TipWHEVcw ZoRLS2QIHyzoYMT9aAKKEJ4nxU05fheOqvQqPFWIZ2joh/nW5eSkpqHtcRshBvZg8izK iEu9vmvmsmLwW9ryfvyupjTkcbusgz4T0beyuLJGsm9lPFLev9R0vMZeXTrODG2e48Rc vNB76osD9teVSpxSB6tvV1st28/YRVT+pHP2sOqIhyQrGpkjpmE2oHoyTJm7X/zwW7cS 36ImITyQBHi22uqAt7A5lgQo9DHBLDHu/O4MiKqSh7mjmb9DmlDwTuy/F986I9GuKXt+ 4PEA== X-Gm-Message-State: APjAAAUjYHz/yoLDpma12QufMn27LPNr8z4JXZ0s3iLh7c75zs4OlDZM zjjQDnc1jp959sv8VQ0Y0QA= X-Google-Smtp-Source: APXvYqy8i85CXpsfyv8dckCdsUg7eLteOhLSNdZVYk516MDZYXdwuPo9V61Bsf6Mk2ZN6V00qpmncQ== X-Received: by 2002:a17:90a:ac0e:: with SMTP id o14mr156393pjq.11.1575912729718; Mon, 09 Dec 2019 09:32:09 -0800 (PST) Received: from btopel-mobl.ger.intel.com ([192.55.55.41]) by smtp.gmail.com with ESMTPSA id d23sm54943pfo.176.2019.12.09.09.32.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 09:32:09 -0800 (PST) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: daniel@iogearbox.net, ast@kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 8/8] riscv, perf: add arch specific perf_arch_bpf_user_pt_regs Date: Mon, 9 Dec 2019 18:31:36 +0100 Message-Id: <20191209173136.29615-9-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191209173136.29615-1-bjorn.topel@gmail.com> References: <20191209173136.29615-1-bjorn.topel@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191209_093210_840139_AEF80920 X-CRM114-Status: UNSURE ( 6.87 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (bjorn.topel[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-riscv@lists.infradead.org, bpf@vger.kernel.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org RISC-V was missing a proper perf_arch_bpf_user_pt_regs macro for CONFIG_PERF_EVENT builds. Signed-off-by: Björn Töpel --- arch/riscv/include/asm/perf_event.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/riscv/include/asm/perf_event.h b/arch/riscv/include/asm/perf_event.h index aefbfaa6a781..0234048b12bc 100644 --- a/arch/riscv/include/asm/perf_event.h +++ b/arch/riscv/include/asm/perf_event.h @@ -82,4 +82,8 @@ struct riscv_pmu { int irq; }; +#ifdef CONFIG_PERF_EVENTS +#define perf_arch_bpf_user_pt_regs(regs) (struct user_regs_struct *)regs +#endif + #endif /* _ASM_RISCV_PERF_EVENT_H */