From patchwork Tue Nov 26 10:07:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261785 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15BDB6C1 for ; Tue, 26 Nov 2019 10:08:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EAAE220835 for ; Tue, 26 Nov 2019 10:08:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rA6x08ok" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727800AbfKZKIr (ORCPT ); Tue, 26 Nov 2019 05:08:47 -0500 Received: from mail-pl1-f193.google.com ([209.85.214.193]:41684 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727482AbfKZKIq (ORCPT ); Tue, 26 Nov 2019 05:08:46 -0500 Received: by mail-pl1-f193.google.com with SMTP id t8so7862979plr.8; Tue, 26 Nov 2019 02:08:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1Sl43iVtC4+YcjLflynrQtIj0zEs25bg62I/x2OEQu0=; b=rA6x08okN9S24McUdG2LHlghnPmhIHd24jmymmQaGlT2PAs6RXcIrFN4KgQ3vtEJQH ienFzUlPPNlyI9b9KsozGHG9jQpVMUrYON+jnaQvRyei7uhWiVfPBjDJK/c0EY88qnZp ZDkq7a+rI2J1MXnLhIb+/EQ22pDhhmb3BAM7x8wEHqFYQ02bE0rYvl/KjEMarmxBop+q p94+hXB+SsDfHHCvbR+iONGBx5fsJeIFS9XDNF/UJ7uD0+X9HjxSjqCZa8oiIpnB76Hg UUUoXmIcImh1gfbohC0t5i//haGFawSR5oYyXeA/IjzMZkOh/+g99F3i75HTqeit7+cx TveQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1Sl43iVtC4+YcjLflynrQtIj0zEs25bg62I/x2OEQu0=; b=lUXdYO1z8ShuHf00pGQ3lfGv76p2GfSL0c171Kgr4bhMpbfML5s4IlrqtgkGiK8kf0 znp/A5cd6LGhCTEqpAw3U3RFedFSeAp/A/V76nMEF7W1qq6LsdKJMd3zZYcK2Mk7G1d8 EBh/O2Fc4s5e8K3QlT44wfZGJMlIfac6KhsF9pKyhiSzcROIV7g+zpp5mWauqcLH0VTF oGAUAJzFcZUBBBYeK0QBacNeeJ1IVVkfJQMv2UGkEtDngTIXWtbTkm1jGYO9UDvriiAi NswRWo3THH/oEeEVV8GsDNcFf29j9Np4vN7UiKhdFMjiJcVHj5205XcPzRD1yhf4dMJQ 2MUQ== X-Gm-Message-State: APjAAAUT1eZcJA1LzvTGPo0+e6lvrqCOKbGpMvkdRgyGo6mJLwUgH/Ap It6Wr0q0VfOrNDRu5n2uwgQ= X-Google-Smtp-Source: APXvYqxA6tuNs0G2K7BYOi/UgwxlWXR+3qbxnD5Hrpm6hj4529qhKxDgIXplkY5qr6yeBIl6U2CJIw== X-Received: by 2002:a17:90a:5aa3:: with SMTP id n32mr5651673pji.97.1574762926051; Tue, 26 Nov 2019 02:08:46 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.08.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:08:45 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 01/18] bpf: introduce bpf_prog_offload_verifier_setup() Date: Tue, 26 Nov 2019 19:07:27 +0900 Message-Id: <20191126100744.5083-2-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang Background: This change was initiated from virtio_net XDP offload work. As per the implementation plan, a copy of original program with map fds from guest replaced with map fds from host needs to be offloaded to the host. To implement this fd replacement, insn_hook() must provide an insn with map fd intact. bpf_map and driver specific map data can be derived from map_fd. Since verifier calls all the offload callbacks after replacing map fds, it was difficult to implement virtio_net XDP offload feature. If virtio_net gets only one callback with original bpf program, it will get a chance to perform the fd replacement in its own copy of the program. Solution: Let's introduce a setup() callback in bpf_prog_offload_ops. It will be non mandetory. The verifier will call it just before replacing the map fds. Signed-off-by: Jason Wang Signed-off-by: Prashant Bhole --- include/linux/bpf.h | 1 + include/linux/bpf_verifier.h | 1 + kernel/bpf/offload.c | 14 ++++++++++++++ kernel/bpf/verifier.c | 6 ++++++ 4 files changed, 22 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 35903f148be5..1cdba120357c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -361,6 +361,7 @@ struct bpf_prog_offload_ops { struct bpf_insn *insn); int (*remove_insns)(struct bpf_verifier_env *env, u32 off, u32 cnt); /* program management callbacks */ + int (*setup)(struct bpf_prog *prog); int (*prepare)(struct bpf_prog *prog); int (*translate)(struct bpf_prog *prog); void (*destroy)(struct bpf_prog *prog); diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 26e40de9ef55..de7028e17c0d 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -418,6 +418,7 @@ static inline struct bpf_reg_state *cur_regs(struct bpf_verifier_env *env) return cur_func(env)->regs; } +int bpf_prog_offload_verifier_setup(struct bpf_prog *prog); int bpf_prog_offload_verifier_prep(struct bpf_prog *prog); int bpf_prog_offload_verify_insn(struct bpf_verifier_env *env, int insn_idx, int prev_insn_idx); diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 5b9da0954a27..04ca7a31d947 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -124,6 +124,20 @@ int bpf_prog_offload_init(struct bpf_prog *prog, union bpf_attr *attr) return err; } +int bpf_prog_offload_verifier_setup(struct bpf_prog *prog) +{ + struct bpf_prog_offload *offload; + int ret = 0; + + down_read(&bpf_devs_lock); + offload = prog->aux->offload; + if (offload && offload->offdev->ops->setup) + ret = offload->offdev->ops->setup(prog); + up_read(&bpf_devs_lock); + + return ret; +} + int bpf_prog_offload_verifier_prep(struct bpf_prog *prog) { struct bpf_prog_offload *offload; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a0482e1c4a77..94b43542439e 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -9737,6 +9737,12 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, env->allow_ptr_leaks = is_priv; + if (bpf_prog_is_dev_bound(env->prog->aux)) { + ret = bpf_prog_offload_verifier_setup(env->prog); + if (ret) + goto skip_full_check; + } + if (is_priv) env->test_state_freq = attr->prog_flags & BPF_F_TEST_STATE_FREQ; From patchwork Tue Nov 26 10:07:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261787 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C46E6C1 for ; Tue, 26 Nov 2019 10:08:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F0CD320835 for ; Tue, 26 Nov 2019 10:08:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="olv12Gp3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727810AbfKZKIv (ORCPT ); Tue, 26 Nov 2019 05:08:51 -0500 Received: from mail-pj1-f65.google.com ([209.85.216.65]:34853 "EHLO mail-pj1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727804AbfKZKIu (ORCPT ); Tue, 26 Nov 2019 05:08:50 -0500 Received: by mail-pj1-f65.google.com with SMTP id s8so8070834pji.2; Tue, 26 Nov 2019 02:08:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TORKveKiyyu6m2oHodE9IBcjkbwEar3Vt5jDpzgDylY=; b=olv12Gp3grnekpbckL4cgsuKfI3uibM/xyKgxKaXAgysBCG0gD1zfwzI4bdTig2QPa bYj2KhBwBQCyeYdYx2+f9ANR6ntDY1QlasYD6qxyDMTFGZj2uvm7ko8kE58GZXl1Gx83 yxZQsAPqwNrJ0vcBYS7ybC0BFUlHVtzRy09jDp4McRhur9+hRM3WTEt+lbKHstbWmP9z At4RGyUZXvbnGh/bg5mBCIRT9jJzXZ/XyUpQTxrGKC9T6fcorgnyhMn8Hm9GLjMjBr2J pd/ik9nqYGr+vwPtpw0IYIaT05AfrD53fQBnlXT6YI9axZ6zErZc8n6ZW5g2ZslzHp5l 2UEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TORKveKiyyu6m2oHodE9IBcjkbwEar3Vt5jDpzgDylY=; b=kURwev+ls/Jl6vD5eylZT2zdr/K0tnG0/15KNy/uaeiBX8J2BEDF8L5ybWo7T/cgDk 8zEDWZFsbkpOR1DFgVVcPcb0O3ccCShYgCo57NeQCWqndp+sUbAsu/Qq8zdX/CmpbRWl 7V3lDYlC0ugdDVZFRUZ2MPYPk6XNryrE7qdOzDoa9tAy2wTciZaapTGvO8bwNENis1vZ hn+5Th79MIoaVH7giT6asMI1mO8dZp/D3+KQgEDwXyeQggwOH1zYjOcc6V15VB1q2de0 Ygrm1NA0P1kdZk0Lon4aPCfJ//i2kAdfmso8FA96FAWAHzPE8Y7QluYCfu7oawzmT4sf TeTw== X-Gm-Message-State: APjAAAViS04I2IeK7zoccrYq9N72p/18Qtqiiz9yynvxUMYpqmBdqYi0 YcstfjJyOwiV1LJnkwI/a10= X-Google-Smtp-Source: APXvYqy7uBDP8l0aXZ/j5b2P/uoXgigesp5vSEQ3UrSFY/YqyNb+UCqonnG8Y2jEb1rpt6g0c9YHbg== X-Received: by 2002:a17:90a:c789:: with SMTP id gn9mr5574668pjb.99.1574762929845; Tue, 26 Nov 2019 02:08:49 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.08.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:08:49 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 02/18] net: core: rename netif_receive_generic_xdp() to do_generic_xdp_core() Date: Tue, 26 Nov 2019 19:07:28 +0900 Message-Id: <20191126100744.5083-3-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang In skb generic path, we need a way to run XDP program on skb but to have customized handling of XDP actions. netif_receive_generic_xdp will be more helpful in such cases than do_xdp_generic. This patch prepares netif_receive_generic_xdp() to be used as general purpose function by renaming it. Signed-off-by: Jason Wang Signed-off-by: Prashant Bhole --- net/core/dev.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index c7fc902ccbdc..5ae647b9914f 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4461,9 +4461,9 @@ static struct netdev_rx_queue *netif_get_rxqueue(struct sk_buff *skb) return rxqueue; } -static u32 netif_receive_generic_xdp(struct sk_buff *skb, - struct xdp_buff *xdp, - struct bpf_prog *xdp_prog) +static u32 do_xdp_generic_core(struct sk_buff *skb, + struct xdp_buff *xdp, + struct bpf_prog *xdp_prog) { struct netdev_rx_queue *rxqueue; void *orig_data, *orig_data_end; @@ -4610,7 +4610,7 @@ int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb) u32 act; int err; - act = netif_receive_generic_xdp(skb, &xdp, xdp_prog); + act = do_xdp_generic_core(skb, &xdp, xdp_prog); if (act != XDP_PASS) { switch (act) { case XDP_REDIRECT: From patchwork Tue Nov 26 10:07:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261789 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0B1D11393 for ; Tue, 26 Nov 2019 10:08:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DD9062084D for ; Tue, 26 Nov 2019 10:08:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LIZgXrZn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727793AbfKZKIy (ORCPT ); Tue, 26 Nov 2019 05:08:54 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:33157 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727817AbfKZKIy (ORCPT ); Tue, 26 Nov 2019 05:08:54 -0500 Received: by mail-pl1-f194.google.com with SMTP id ay6so7955187plb.0; Tue, 26 Nov 2019 02:08:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2bAHX0FXOgqRwz29p5ya1CXvQIHb2CLin8kXdVB/0KE=; b=LIZgXrZnUx86XNNP0wmHAcrNw9/JIZ8Z/K+g4m7VRaHAmOxfSZCyPu0eA26cNr146y o7pf4ACxMyylsjo6uGEODbNQv8BRy46PhL70owQw/xwCKefsVPclpEamrqGo72ehYh+9 v63wDQiT5CznqvW3e78uI6rd3/N+sk3gekTAk9Ai4rTTvwCGcYckKvWPtFjd9xcw1oOG 8R1NnTguMJFxf9tZ09BhOA6+IrzVhQlaOh9GYKE9/yZf9zK1SwVLEIg5oFO2RFCUDefS 13vpeM20b7IeXPytkqCO5Efyk0g/6IAoInedl5KOOFZshV6zSMntUF6MhlF1jpnMUsy2 W+Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2bAHX0FXOgqRwz29p5ya1CXvQIHb2CLin8kXdVB/0KE=; b=d6JOmqBcdUBiiHvSnCpFV6Bf+c2pPk1n120/c5higyNKPqXJDKPk9HuoJOB259SuVR AklUuIRgP95tEz7j+k9RbpBihlx+5jeDEOzuEz4arYQ+qTmYU4SjVXcLlhqc+JhIlBJG xtWHQVjUCXlWB3R7nS76fuMz9Z/hOnX8DM4Z5nvB5gFtsj3/hC7TJdK/IrpObrFNA4s2 oyskWcXsqGIh9wlJpD1TeGzAjI5TFCwcY7LkV9rvkPfeqU9jwNEi29+htMaSNGDk31UU 178ZNXC6+cCkTQN6ls+tQ76sricmUptjPtZ/s+johAyLqYQ3Tmg1fpiVjXBD/Dl0H60m iplw== X-Gm-Message-State: APjAAAUIczKZ2gLSz5nMbvq/uBAcpcQVNQhf0RYP5INQeg/eOJ4P5hqr j/21TcwWqDzT2YUO4XD5FjE= X-Google-Smtp-Source: APXvYqyvpP2N7hYAzxx2bd4M9VJ+Jp8Kjxs/E28fT/8KVvvGyoey0uyI2syTr1ITSFDcJg6CMebg+A== X-Received: by 2002:a17:902:7290:: with SMTP id d16mr32584601pll.340.1574762933671; Tue, 26 Nov 2019 02:08:53 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.08.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:08:53 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 03/18] net: core: export do_xdp_generic_core() Date: Tue, 26 Nov 2019 19:07:29 +0900 Message-Id: <20191126100744.5083-4-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang Let's export do_xdp_generic as a general purpose function. It will just run XDP program on skb but will not handle XDP actions. Signed-off-by: Jason Wang Signed-off-by: Prashant Bhole --- include/linux/netdevice.h | 2 ++ net/core/dev.c | 6 +++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 9e6fb8524d91..2b6317ac9795 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3648,6 +3648,8 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb); +u32 do_xdp_generic_core(struct sk_buff *skb, struct xdp_buff *xdp, + struct bpf_prog *xdp_prog); int netif_rx(struct sk_buff *skb); int netif_rx_ni(struct sk_buff *skb); int netif_receive_skb(struct sk_buff *skb); diff --git a/net/core/dev.c b/net/core/dev.c index 5ae647b9914f..d97c3f35e047 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4461,9 +4461,8 @@ static struct netdev_rx_queue *netif_get_rxqueue(struct sk_buff *skb) return rxqueue; } -static u32 do_xdp_generic_core(struct sk_buff *skb, - struct xdp_buff *xdp, - struct bpf_prog *xdp_prog) +u32 do_xdp_generic_core(struct sk_buff *skb, struct xdp_buff *xdp, + struct bpf_prog *xdp_prog) { struct netdev_rx_queue *rxqueue; void *orig_data, *orig_data_end; @@ -4574,6 +4573,7 @@ static u32 do_xdp_generic_core(struct sk_buff *skb, return act; } +EXPORT_SYMBOL_GPL(do_xdp_generic_core); /* When doing generic XDP we have to bypass the qdisc layer and the * network taps in order to match in-driver-XDP behavior. From patchwork Tue Nov 26 10:07:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261791 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 715F76C1 for ; Tue, 26 Nov 2019 10:09:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 52C9420862 for ; Tue, 26 Nov 2019 10:09:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dzBZCOna" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727829AbfKZKI6 (ORCPT ); Tue, 26 Nov 2019 05:08:58 -0500 Received: from mail-pj1-f67.google.com ([209.85.216.67]:43737 "EHLO mail-pj1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727733AbfKZKI6 (ORCPT ); Tue, 26 Nov 2019 05:08:58 -0500 Received: by mail-pj1-f67.google.com with SMTP id a10so8054624pju.10; Tue, 26 Nov 2019 02:08:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V/KNFyebQK7C43tEw7YnjUgOQj/Np7ljq5WcxHdrR0c=; b=dzBZCOnaWE7Ha7XRUgoI1TMso7rPaEIgzx/qbfK80VCj2jnRcx784yIM+tcJz7c1W4 SX24el6V4d3P89iqBPTFv2AmbDubMl9yoJqacCzuZhlnR3Bk050iQx1k6m6poXeq+bfl nCsgtrM1+U9l5Ye2fGXuufDnG0O2Gs/JqEuATOJ5p/18g7oqYF+NooT1+Mdardio8ul3 GV1l4Qd/F5C3N0Vmi+3MI6zR9VuP0jtaYWvbvIgF8HfBcTH1gioMVf0mIhU757jNqbji SVXIEm14A7LiPCXAPyyVnHOhayKjwJrvX2PGeac7T9Z21XmzwuLidD9Nq6gJQVhxwP8a f/wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V/KNFyebQK7C43tEw7YnjUgOQj/Np7ljq5WcxHdrR0c=; b=cejOk3ja3NqpeZUH/u+SlJRMKpwPIHXVwmQq1KF9PusatRovsbKMK8rUCRUDcC3SwU qVkOYfqVgUuMNTZs9O2JKAh2AVHBeqzWqHyB9dGSRTXjIS2+3fj4Pm6bmg2AqsSjOXoM bR5/UOaAYpPVQOrAc+Kb1IQWuizcgIqBednAWgsouAlRPqVqkgUqeV8LpUtxX2kE45RS 5qbnkNhKsY/w4vGreYly0CV3d9BPRVTi/VLCI3s6mPzt1o5Nj32qwhPgNFsVqnI6sOud T46fpG2dZZV0KVy6DJbhKj3po9JOl+zSsfXIN1qsDh74vrdn+QzfsGJrbgkrbK9/44sS +dUw== X-Gm-Message-State: APjAAAXIatkt2pDVJG0p35ypysy1IlQeNz48wbzMpWDTdDXSt0vh6seD YIziqVcbolf0U71psjMWAQ0= X-Google-Smtp-Source: APXvYqwzcOMViqGh/CDqZ6GTDBF1yWFhOG4mwf7CZlGyeqi/dlAtJKkC4MdApMy1ZQ7I10+ijRtVKw== X-Received: by 2002:a17:90a:d58e:: with SMTP id v14mr5946678pju.142.1574762937533; Tue, 26 Nov 2019 02:08:57 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.08.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:08:57 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 04/18] tuntap: check tun_msg_ctl type at necessary places Date: Tue, 26 Nov 2019 19:07:30 +0900 Message-Id: <20191126100744.5083-5-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org tun_msg_ctl is used by vhost_net to communicate with tuntap. We will introduce another type in soon. As a preparation this patch adds conditions to check tun_msg_ctl type at necessary places. Signed-off-by: Prashant Bhole --- drivers/net/tap.c | 7 +++++-- drivers/net/tun.c | 6 +++++- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/net/tap.c b/drivers/net/tap.c index 3ae70c7e6860..4df7bf00af66 100644 --- a/drivers/net/tap.c +++ b/drivers/net/tap.c @@ -1213,6 +1213,7 @@ static int tap_sendmsg(struct socket *sock, struct msghdr *m, struct tap_queue *q = container_of(sock, struct tap_queue, sock); struct tun_msg_ctl *ctl = m->msg_control; struct xdp_buff *xdp; + void *ptr = NULL; int i; if (ctl && (ctl->type == TUN_MSG_PTR)) { @@ -1223,8 +1224,10 @@ static int tap_sendmsg(struct socket *sock, struct msghdr *m, return 0; } - return tap_get_user(q, ctl ? ctl->ptr : NULL, &m->msg_iter, - m->msg_flags & MSG_DONTWAIT); + if (ctl && ctl->type == TUN_MSG_UBUF) + ptr = ctl->ptr; + + return tap_get_user(q, ptr, &m->msg_iter, m->msg_flags & MSG_DONTWAIT); } static int tap_recvmsg(struct socket *sock, struct msghdr *m, diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 683d371e6e82..1e436d9ec4e1 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -2529,6 +2529,7 @@ static int tun_sendmsg(struct socket *sock, struct msghdr *m, size_t total_len) struct tun_struct *tun = tun_get(tfile); struct tun_msg_ctl *ctl = m->msg_control; struct xdp_buff *xdp; + void *ptr = NULL; if (!tun) return -EBADFD; @@ -2560,7 +2561,10 @@ static int tun_sendmsg(struct socket *sock, struct msghdr *m, size_t total_len) goto out; } - ret = tun_get_user(tun, tfile, ctl ? ctl->ptr : NULL, &m->msg_iter, + if (ctl && ctl->type == TUN_MSG_UBUF) + ptr = ctl->ptr; + + ret = tun_get_user(tun, tfile, ptr, &m->msg_iter, m->msg_flags & MSG_DONTWAIT, m->msg_flags & MSG_MORE); out: From patchwork Tue Nov 26 10:07:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261793 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C0136C1 for ; Tue, 26 Nov 2019 10:09:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 426BA20862 for ; Tue, 26 Nov 2019 10:09:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QkhGdXqA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727836AbfKZKJC (ORCPT ); Tue, 26 Nov 2019 05:09:02 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:40630 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727841AbfKZKJC (ORCPT ); Tue, 26 Nov 2019 05:09:02 -0500 Received: by mail-pl1-f195.google.com with SMTP id f9so7935047plr.7; Tue, 26 Nov 2019 02:09:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JOqfaTjB9hR4SvgWUY3wyefTnNCZb7GyoEbULC6AJJo=; b=QkhGdXqAqq10xMSu+2CnyH8h3k9xRLREYvQrc7Kl+Q0+83MDkF+tuhBxvnRiUIrU5H eLqXuIZUtQQQafXXx5KJYDzw+cbyREKs1xo+xDhrhWrKOz2Hfe+KnD5BCeK8L6z7FoGB H8uL0SKa0QBz+PC6YnjRyHOBKKwWwEqH1f4T41+MgFl+QMvSklHCyhIdh8muKtDnfRAe wnppauQ8mL6ut7U+3hHZwdGgDP61KM8PE2qspO4rG1wmunzRjivSxjNu6ss3SQdisLdS 3CD2OlXrfUF8AsknNCn14UV/RDnRG4uS2zPqKLV30IbvCAr64AsnlplUmR+oN2N7ka5Y wSMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JOqfaTjB9hR4SvgWUY3wyefTnNCZb7GyoEbULC6AJJo=; b=kofofmyTNIoeFUm1qFrzSFX30jhzjAn8Twl3anLeGsOzJRiv2ApRnN8ifyoCvfasek 9QH9nQKAl3m07gBQmb2U3hFANqx5enkP1ahMXYumUX0if764y7IDlTAMHI4mnuki/LgM qkbTEexAKTCDCzYby0EPsblIWs24HdethA6nzUeBbfweBvOUCmhGBoWu5yG0CtDz/gdO sDbkr1nhoNAQtzJCy6UxRMp1Oj62SZWBaKCYVmZLqS3BA16RH06jFN+5VSIHqWp7rmDL cy7iioPhx+a9hftsvT2GbqIPH2GeiNGxh8G/aIm5Pvsx4Quvr81i0gG0k3Cqu61ekDhg lh0w== X-Gm-Message-State: APjAAAUc0Txte9Hoooxilso/urRvCY2d4vbI0tOzYDxa5RuRKG7EtSUR yxBO3LmZkystT8YNbv7cNBk= X-Google-Smtp-Source: APXvYqwPyv6+pc3DP4PZm1a3x96MwS0Kc6f03xdB6l2+mtJteT+8mFJPIbbpCn/sngodhdsZCCyQtA== X-Received: by 2002:a17:90b:f0c:: with SMTP id br12mr5745156pjb.67.1574762941343; Tue, 26 Nov 2019 02:09:01 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.08.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:00 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 05/18] vhost_net: user tap recvmsg api to access ptr ring Date: Tue, 26 Nov 2019 19:07:31 +0900 Message-Id: <20191126100744.5083-6-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Currently vhost_net directly accesses ptr ring of tap driver to fetch Rx packet pointers. In order to avoid it this patch modifies tap driver's recvmsg api to do additional task of fetching Rx packet pointers. A special struct tun_msg_ctl is already being passed via msg_control for tun Rx XDP batching. This patch extends tun_msg_ctl usage to send sub commands to recvmsg api. Now tun_recvmsg will handle commands to consume and unconsume packet pointers from ptr ring. This will be useful in implementation of virtio-net XDP offload feature, where packets will be XDP processed before they are passed to vhost_net. Signed-off-by: Prashant Bhole --- drivers/net/tap.c | 22 ++++++++++++++++++- drivers/net/tun.c | 24 ++++++++++++++++++++- drivers/vhost/net.c | 48 +++++++++++++++++++++++++++++++----------- include/linux/if_tun.h | 18 ++++++++++++++++ 4 files changed, 98 insertions(+), 14 deletions(-) diff --git a/drivers/net/tap.c b/drivers/net/tap.c index 4df7bf00af66..8635cdfd7aa4 100644 --- a/drivers/net/tap.c +++ b/drivers/net/tap.c @@ -1234,8 +1234,28 @@ static int tap_recvmsg(struct socket *sock, struct msghdr *m, size_t total_len, int flags) { struct tap_queue *q = container_of(sock, struct tap_queue, sock); - struct sk_buff *skb = m->msg_control; + struct tun_msg_ctl *ctl = m->msg_control; + struct sk_buff *skb = NULL; int ret; + + if (ctl) { + switch (ctl->type) { + case TUN_MSG_PKT: + skb = ctl->ptr; + break; + case TUN_MSG_CONSUME_PKTS: + return ptr_ring_consume_batched(&q->ring, + ctl->ptr, + ctl->num); + case TUN_MSG_UNCONSUME_PKTS: + ptr_ring_unconsume(&q->ring, ctl->ptr, ctl->num, + tun_ptr_free); + return 0; + default: + return -EINVAL; + } + } + if (flags & ~(MSG_DONTWAIT|MSG_TRUNC)) { kfree_skb(skb); return -EINVAL; diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 1e436d9ec4e1..4f28f2387435 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -2577,7 +2577,8 @@ static int tun_recvmsg(struct socket *sock, struct msghdr *m, size_t total_len, { struct tun_file *tfile = container_of(sock, struct tun_file, socket); struct tun_struct *tun = tun_get(tfile); - void *ptr = m->msg_control; + struct tun_msg_ctl *ctl = m->msg_control; + void *ptr = NULL; int ret; if (!tun) { @@ -2585,6 +2586,27 @@ static int tun_recvmsg(struct socket *sock, struct msghdr *m, size_t total_len, goto out_free; } + if (ctl) { + switch (ctl->type) { + case TUN_MSG_PKT: + ptr = ctl->ptr; + break; + case TUN_MSG_CONSUME_PKTS: + ret = ptr_ring_consume_batched(&tfile->tx_ring, + ctl->ptr, + ctl->num); + goto out; + case TUN_MSG_UNCONSUME_PKTS: + ptr_ring_unconsume(&tfile->tx_ring, ctl->ptr, + ctl->num, tun_ptr_free); + ret = 0; + goto out; + default: + ret = -EINVAL; + goto out_put_tun; + } + } + if (flags & ~(MSG_DONTWAIT|MSG_TRUNC|MSG_ERRQUEUE)) { ret = -EINVAL; goto out_put_tun; diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 1a2dd53caade..0f91b374a558 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -175,24 +175,44 @@ static void *vhost_net_buf_consume(struct vhost_net_buf *rxq) static int vhost_net_buf_produce(struct vhost_net_virtqueue *nvq) { + struct vhost_virtqueue *vq = &nvq->vq; + struct socket *sock = vq->private_data; struct vhost_net_buf *rxq = &nvq->rxq; + struct tun_msg_ctl ctl = { + .type = TUN_MSG_CONSUME_PKTS, + .ptr = (void *) rxq->queue, + .num = VHOST_NET_BATCH, + }; + struct msghdr msg = { + .msg_control = &ctl, + }; rxq->head = 0; - rxq->tail = ptr_ring_consume_batched(nvq->rx_ring, rxq->queue, - VHOST_NET_BATCH); + rxq->tail = sock->ops->recvmsg(sock, &msg, 0, 0); + if (WARN_ON_ONCE(rxq->tail < 0)) + rxq->tail = 0; + return rxq->tail; } static void vhost_net_buf_unproduce(struct vhost_net_virtqueue *nvq) { + struct vhost_virtqueue *vq = &nvq->vq; + struct socket *sock = vq->private_data; struct vhost_net_buf *rxq = &nvq->rxq; + struct tun_msg_ctl ctl = { + .type = TUN_MSG_UNCONSUME_PKTS, + .ptr = (void *) (rxq->queue + rxq->head), + .num = vhost_net_buf_get_size(rxq), + }; + struct msghdr msg = { + .msg_control = &ctl, + }; - if (nvq->rx_ring && !vhost_net_buf_is_empty(rxq)) { - ptr_ring_unconsume(nvq->rx_ring, rxq->queue + rxq->head, - vhost_net_buf_get_size(rxq), - tun_ptr_free); - rxq->head = rxq->tail = 0; - } + if (!vhost_net_buf_is_empty(rxq)) + sock->ops->recvmsg(sock, &msg, 0, 0); + + rxq->head = rxq->tail = 0; } static int vhost_net_buf_peek_len(void *ptr) @@ -1109,6 +1129,7 @@ static void handle_rx(struct vhost_net *net) .flags = 0, .gso_type = VIRTIO_NET_HDR_GSO_NONE }; + struct tun_msg_ctl ctl; size_t total_len = 0; int err, mergeable; s16 headcount; @@ -1166,8 +1187,11 @@ static void handle_rx(struct vhost_net *net) goto out; } busyloop_intr = false; - if (nvq->rx_ring) - msg.msg_control = vhost_net_buf_consume(&nvq->rxq); + if (nvq->rx_ring) { + ctl.type = TUN_MSG_PKT; + ctl.ptr = vhost_net_buf_consume(&nvq->rxq); + msg.msg_control = &ctl; + } /* On overrun, truncate and discard */ if (unlikely(headcount > UIO_MAXIOV)) { iov_iter_init(&msg.msg_iter, READ, vq->iov, 1, 1); @@ -1346,8 +1370,8 @@ static struct socket *vhost_net_stop_vq(struct vhost_net *n, mutex_lock(&vq->mutex); sock = vq->private_data; vhost_net_disable_vq(n, vq); - vq->private_data = NULL; vhost_net_buf_unproduce(nvq); + vq->private_data = NULL; nvq->rx_ring = NULL; mutex_unlock(&vq->mutex); return sock; @@ -1538,8 +1562,8 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd) } vhost_net_disable_vq(n, vq); - vq->private_data = sock; vhost_net_buf_unproduce(nvq); + vq->private_data = sock; r = vhost_vq_init_access(vq); if (r) goto err_used; diff --git a/include/linux/if_tun.h b/include/linux/if_tun.h index 5bda8cf457b6..bb94843e3829 100644 --- a/include/linux/if_tun.h +++ b/include/linux/if_tun.h @@ -11,8 +11,26 @@ #define TUN_XDP_FLAG 0x1UL +/* + * tun_msg_ctl types + */ + #define TUN_MSG_UBUF 1 #define TUN_MSG_PTR 2 +/* + * Used for passing a packet pointer from vhost to tun + */ +#define TUN_MSG_PKT 3 +/* + * Used for passing an array of pointer from vhost to tun. + * tun consumes packets from ptr ring and stores in pointer array. + */ +#define TUN_MSG_CONSUME_PKTS 4 +/* + * Used for passing an array of pointer from vhost to tun. + * tun consumes get pointer from array and puts back into ptr ring. + */ +#define TUN_MSG_UNCONSUME_PKTS 5 struct tun_msg_ctl { unsigned short type; unsigned short num; From patchwork Tue Nov 26 10:07:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261795 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 152656C1 for ; Tue, 26 Nov 2019 10:09:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E18932086A for ; Tue, 26 Nov 2019 10:09:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Dad2fXqh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727856AbfKZKJG (ORCPT ); Tue, 26 Nov 2019 05:09:06 -0500 Received: from mail-pj1-f66.google.com ([209.85.216.66]:38759 "EHLO mail-pj1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727817AbfKZKJG (ORCPT ); Tue, 26 Nov 2019 05:09:06 -0500 Received: by mail-pj1-f66.google.com with SMTP id f7so8070953pjw.5; Tue, 26 Nov 2019 02:09:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+QD4/vKzpUKJ7oAMMcWF4SBVUkQv/vIBx+F+3VPKSIM=; b=Dad2fXqh5PjlpIH3rO+thxIGofHNhYZUm+ZLETyge4e7luum6aCm1A3AI5ThMS310E R4Xj/8Eh+TByGj7pfweNNFqDHyqQKsaKv7ujNrigl64aOA5ORPDVuVwF8DBwlLQA/xsN 2ICt4DG1Q+3waVVvDBsMLEigv3rmbn1vXvQDRuRdH7w2/jXjlYjlC09XERU2kaL7p9Db MEY1/8o4irNqzMaJuU5BozHQ9JT/4oEIQ6655AN0p1nRUuGHXJHKBb/VYgmPs+Bzrcx6 VcfSImSIuac8mJWcGQMOg8yxKY4j1WneWZfogz3EYkU6dkoWEiuYjotAtqoP44jBGAOe Utcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+QD4/vKzpUKJ7oAMMcWF4SBVUkQv/vIBx+F+3VPKSIM=; b=ZYWwTqdKvklO7C6i+rApGBDs0vjl+G+6XH+Oz8/4J3NgyoutaBkJ3Eqk//ui4f5+69 kTBy4nM/O80jGL4Z3rCUu/MIi7ZASLgm/V93jIFltixH/1OHFRlq19AfACtp0rDIgvN8 JekU3EuaHZt+KnCJDjxTiMM4kwLkkCkq7nP2QFIKZXUDbNzNt4FOi85ggkgFcnjn6xvA peEQiaicajBEBmu3VvKOxrTzCQqHgNvPTDLq2pRAYeODTaj84KIhx8OF+Sesq8kpCUbk 5GlxuxwctQeOrvFhrEmwihTDvAOvOkapwlvCrXN0JXCMaYTZpnN4V4bHUOj+1pv8NqTd J2fg== X-Gm-Message-State: APjAAAUdbYb9uUvsZkYbLsnJBBrRBGnYxZDoyNB9LivMNokW86DYVtLA 1SuHXbYCNz5OuqgaLKFhkgo= X-Google-Smtp-Source: APXvYqxxHoIpfqZB5NCqWS+mIJcFEPH5D91ww1dQwvhntGuceZdTfPynWee8zZHRxK2/XD+OeSmwNA== X-Received: by 2002:a17:902:9891:: with SMTP id s17mr33484924plp.101.1574762945278; Tue, 26 Nov 2019 02:09:05 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:04 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 06/18] tuntap: remove usage of ptr ring in vhost_net Date: Tue, 26 Nov 2019 19:07:32 +0900 Message-Id: <20191126100744.5083-7-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Remove usage of ptr ring of tuntap in vhost_net and remove the functions exported from tuntap drivers to get ptr ring. Signed-off-by: Prashant Bhole --- drivers/net/tap.c | 13 ------------- drivers/net/tun.c | 13 ------------- drivers/vhost/net.c | 31 ++++--------------------------- include/linux/if_tap.h | 5 ----- include/linux/if_tun.h | 5 ----- 5 files changed, 4 insertions(+), 63 deletions(-) diff --git a/drivers/net/tap.c b/drivers/net/tap.c index 8635cdfd7aa4..6426501b8d0e 100644 --- a/drivers/net/tap.c +++ b/drivers/net/tap.c @@ -1298,19 +1298,6 @@ struct socket *tap_get_socket(struct file *file) } EXPORT_SYMBOL_GPL(tap_get_socket); -struct ptr_ring *tap_get_ptr_ring(struct file *file) -{ - struct tap_queue *q; - - if (file->f_op != &tap_fops) - return ERR_PTR(-EINVAL); - q = file->private_data; - if (!q) - return ERR_PTR(-EBADFD); - return &q->ring; -} -EXPORT_SYMBOL_GPL(tap_get_ptr_ring); - int tap_queue_resize(struct tap_dev *tap) { struct net_device *dev = tap->dev; diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 4f28f2387435..d078b4659897 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -3750,19 +3750,6 @@ struct socket *tun_get_socket(struct file *file) } EXPORT_SYMBOL_GPL(tun_get_socket); -struct ptr_ring *tun_get_tx_ring(struct file *file) -{ - struct tun_file *tfile; - - if (file->f_op != &tun_fops) - return ERR_PTR(-EINVAL); - tfile = file->private_data; - if (!tfile) - return ERR_PTR(-EBADFD); - return &tfile->tx_ring; -} -EXPORT_SYMBOL_GPL(tun_get_tx_ring); - module_init(tun_init); module_exit(tun_cleanup); MODULE_DESCRIPTION(DRV_DESCRIPTION); diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 0f91b374a558..2e069d1ef946 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -122,7 +122,6 @@ struct vhost_net_virtqueue { /* Reference counting for outstanding ubufs. * Protected by vq mutex. Writers must also take device mutex. */ struct vhost_net_ubuf_ref *ubufs; - struct ptr_ring *rx_ring; struct vhost_net_buf rxq; /* Batched XDP buffs */ struct xdp_buff *xdp; @@ -997,8 +996,9 @@ static int peek_head_len(struct vhost_net_virtqueue *rvq, struct sock *sk) int len = 0; unsigned long flags; - if (rvq->rx_ring) - return vhost_net_buf_peek(rvq); + len = vhost_net_buf_peek(rvq); + if (len) + return len; spin_lock_irqsave(&sk->sk_receive_queue.lock, flags); head = skb_peek(&sk->sk_receive_queue); @@ -1187,7 +1187,7 @@ static void handle_rx(struct vhost_net *net) goto out; } busyloop_intr = false; - if (nvq->rx_ring) { + if (!vhost_net_buf_is_empty(&nvq->rxq)) { ctl.type = TUN_MSG_PKT; ctl.ptr = vhost_net_buf_consume(&nvq->rxq); msg.msg_control = &ctl; @@ -1343,7 +1343,6 @@ static int vhost_net_open(struct inode *inode, struct file *f) n->vqs[i].batched_xdp = 0; n->vqs[i].vhost_hlen = 0; n->vqs[i].sock_hlen = 0; - n->vqs[i].rx_ring = NULL; vhost_net_buf_init(&n->vqs[i].rxq); } vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX, @@ -1372,7 +1371,6 @@ static struct socket *vhost_net_stop_vq(struct vhost_net *n, vhost_net_disable_vq(n, vq); vhost_net_buf_unproduce(nvq); vq->private_data = NULL; - nvq->rx_ring = NULL; mutex_unlock(&vq->mutex); return sock; } @@ -1468,25 +1466,6 @@ static struct socket *get_raw_socket(int fd) return ERR_PTR(r); } -static struct ptr_ring *get_tap_ptr_ring(int fd) -{ - struct ptr_ring *ring; - struct file *file = fget(fd); - - if (!file) - return NULL; - ring = tun_get_tx_ring(file); - if (!IS_ERR(ring)) - goto out; - ring = tap_get_ptr_ring(file); - if (!IS_ERR(ring)) - goto out; - ring = NULL; -out: - fput(file); - return ring; -} - static struct socket *get_tap_socket(int fd) { struct file *file = fget(fd); @@ -1570,8 +1549,6 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd) r = vhost_net_enable_vq(n, vq); if (r) goto err_used; - if (index == VHOST_NET_VQ_RX) - nvq->rx_ring = get_tap_ptr_ring(fd); oldubufs = nvq->ubufs; nvq->ubufs = ubufs; diff --git a/include/linux/if_tap.h b/include/linux/if_tap.h index 915a187cfabd..68fe366fb185 100644 --- a/include/linux/if_tap.h +++ b/include/linux/if_tap.h @@ -4,7 +4,6 @@ #if IS_ENABLED(CONFIG_TAP) struct socket *tap_get_socket(struct file *); -struct ptr_ring *tap_get_ptr_ring(struct file *file); #else #include #include @@ -14,10 +13,6 @@ static inline struct socket *tap_get_socket(struct file *f) { return ERR_PTR(-EINVAL); } -static inline struct ptr_ring *tap_get_ptr_ring(struct file *f) -{ - return ERR_PTR(-EINVAL); -} #endif /* CONFIG_TAP */ #include diff --git a/include/linux/if_tun.h b/include/linux/if_tun.h index bb94843e3829..f01a255e076d 100644 --- a/include/linux/if_tun.h +++ b/include/linux/if_tun.h @@ -44,7 +44,6 @@ struct tun_xdp_hdr { #if defined(CONFIG_TUN) || defined(CONFIG_TUN_MODULE) struct socket *tun_get_socket(struct file *); -struct ptr_ring *tun_get_tx_ring(struct file *file); bool tun_is_xdp_frame(void *ptr); void *tun_xdp_to_ptr(void *ptr); void *tun_ptr_to_xdp(void *ptr); @@ -58,10 +57,6 @@ static inline struct socket *tun_get_socket(struct file *f) { return ERR_PTR(-EINVAL); } -static inline struct ptr_ring *tun_get_tx_ring(struct file *f) -{ - return ERR_PTR(-EINVAL); -} static inline bool tun_is_xdp_frame(void *ptr) { return false; From patchwork Tue Nov 26 10:07:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261797 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8C171393 for ; Tue, 26 Nov 2019 10:09:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 99B5C2086A for ; Tue, 26 Nov 2019 10:09:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="T0I1EGJ6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727861AbfKZKJK (ORCPT ); Tue, 26 Nov 2019 05:09:10 -0500 Received: from mail-pg1-f195.google.com ([209.85.215.195]:40466 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727840AbfKZKJJ (ORCPT ); Tue, 26 Nov 2019 05:09:09 -0500 Received: by mail-pg1-f195.google.com with SMTP id e17so8744173pgd.7; Tue, 26 Nov 2019 02:09:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o7wWeRbc+tNqQMSjrOB4Aqcs6TIYjdNZ6xF+Z6+tUq0=; b=T0I1EGJ6YTSRSu/Y4XLTUBBYnqBS9LwqseER/BjzdXl8Q7YQ6BBhOS6hWPwJ+Fampz P7EI5RxePE3I3u7Y94Tky0n+nFE78T2UBuBrF7foiUu5ODFJMPS7xZL9Attd9J/fv2xk YC0mrtwZW0lEhmUYXkLjAnrzjGf0MLJJzOGH6wypDpR4PUbQQZUzKRmWBcG+SeZ9cW79 sSdTVoF36tW8MK07iZfRmuRxtHw8eS3BBMGKOv/L1m+aD66CZK50R2dRXBlpOSt96TbP YzvoGyvxaN5aONyAay/kSX0/Rb7Ga8eSzdNDD3OD5e5IZ1t55ny2HbvmNcgnnY5mr39+ 5zeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o7wWeRbc+tNqQMSjrOB4Aqcs6TIYjdNZ6xF+Z6+tUq0=; b=FEg0XJiyyQqIHsbT7FmgULPkXQPIu7pnXJrHus82fHrBIsAZkefbvddCiY0mV7cvl7 DxhqV+Nb9FqFnvTXs9Q1u8wH7+4Yd8dvh7UHFuAjMyNGX+BwsK4FmKku2g6wR7e6eD/E D02sd3InkOyPOYe0Ln/d8QL9FYv6pt4natI3j3ItA7A66bAOE7LvbvCIOvwpcpjREj9s 1hB9whgmBtGBc2LS7wCUVCOrD7rEpE33TCMQlQDE7Lg9ygPqUzIQaXIdas5GqvTs6XiQ s24DTxnz2vKwEU6iHKaxI8BBK4OhMoXoHtW5MNtvYefyY1Y7zkvws39SLI3b4n/Yj8zM 6n/Q== X-Gm-Message-State: APjAAAVEKBSRncaZNTtocJZwtwvReqxxj2KJ5IOX1oM3Ov+UE+zqyucI 1DcCew78vbcKj2WOH0Mbg10= X-Google-Smtp-Source: APXvYqwfaVLiFIs0iXc/+SHQsi0c/jFNHsDcCU5mOA5Ky5UmlUqcVt7e8bS0r/y9o05pZ3I6c6EV8A== X-Received: by 2002:a65:4342:: with SMTP id k2mr38951684pgq.63.1574762949130; Tue, 26 Nov 2019 02:09:09 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:08 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 07/18] tun: set offloaded xdp program Date: Tue, 26 Nov 2019 19:07:33 +0900 Message-Id: <20191126100744.5083-8-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang This patch introduces an ioctl way to set an offloaded XDP program to tun driver. This ioctl will be used by qemu to offload XDP program from virtio_net in the guest. Signed-off-by: Jason Wang Signed-off-by: Prashant Bhole --- drivers/net/tun.c | 19 ++++++++++++++----- include/uapi/linux/if_tun.h | 1 + 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/drivers/net/tun.c b/drivers/net/tun.c index d078b4659897..ecb49101b0b5 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -241,6 +241,7 @@ struct tun_struct { struct bpf_prog __rcu *xdp_prog; struct tun_prog __rcu *steering_prog; struct tun_prog __rcu *filter_prog; + struct tun_prog __rcu *offloaded_xdp_prog; struct ethtool_link_ksettings link_ksettings; }; @@ -2256,7 +2257,7 @@ static void tun_prog_free(struct rcu_head *rcu) { struct tun_prog *prog = container_of(rcu, struct tun_prog, rcu); - bpf_prog_destroy(prog->prog); + bpf_prog_put(prog->prog); kfree(prog); } @@ -2301,6 +2302,7 @@ static void tun_free_netdev(struct net_device *dev) security_tun_dev_free_security(tun->security); __tun_set_ebpf(tun, &tun->steering_prog, NULL); __tun_set_ebpf(tun, &tun->filter_prog, NULL); + __tun_set_ebpf(tun, &tun->offloaded_xdp_prog, NULL); } static void tun_setup(struct net_device *dev) @@ -3036,7 +3038,7 @@ static int tun_set_queue(struct file *file, struct ifreq *ifr) } static int tun_set_ebpf(struct tun_struct *tun, struct tun_prog **prog_p, - void __user *data) + void __user *data, int type) { struct bpf_prog *prog; int fd; @@ -3047,7 +3049,7 @@ static int tun_set_ebpf(struct tun_struct *tun, struct tun_prog **prog_p, if (fd == -1) { prog = NULL; } else { - prog = bpf_prog_get_type(fd, BPF_PROG_TYPE_SOCKET_FILTER); + prog = bpf_prog_get_type(fd, type); if (IS_ERR(prog)) return PTR_ERR(prog); } @@ -3345,11 +3347,18 @@ static long __tun_chr_ioctl(struct file *file, unsigned int cmd, break; case TUNSETSTEERINGEBPF: - ret = tun_set_ebpf(tun, &tun->steering_prog, argp); + ret = tun_set_ebpf(tun, &tun->steering_prog, argp, + BPF_PROG_TYPE_SOCKET_FILTER); break; case TUNSETFILTEREBPF: - ret = tun_set_ebpf(tun, &tun->filter_prog, argp); + ret = tun_set_ebpf(tun, &tun->filter_prog, argp, + BPF_PROG_TYPE_SOCKET_FILTER); + break; + + case TUNSETOFFLOADEDXDP: + ret = tun_set_ebpf(tun, &tun->offloaded_xdp_prog, argp, + BPF_PROG_TYPE_XDP); break; case TUNSETCARRIER: diff --git a/include/uapi/linux/if_tun.h b/include/uapi/linux/if_tun.h index 454ae31b93c7..21dbd8db2401 100644 --- a/include/uapi/linux/if_tun.h +++ b/include/uapi/linux/if_tun.h @@ -61,6 +61,7 @@ #define TUNSETFILTEREBPF _IOR('T', 225, int) #define TUNSETCARRIER _IOW('T', 226, int) #define TUNGETDEVNETNS _IO('T', 227) +#define TUNSETOFFLOADEDXDP _IOW('T', 228, int) /* TUNSETIFF ifr flags */ #define IFF_TUN 0x0001 From patchwork Tue Nov 26 10:07:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261799 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 838C51393 for ; Tue, 26 Nov 2019 10:09:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5BE6020871 for ; Tue, 26 Nov 2019 10:09:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TcQUjhhk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727872AbfKZKJO (ORCPT ); Tue, 26 Nov 2019 05:09:14 -0500 Received: from mail-pg1-f196.google.com ([209.85.215.196]:37051 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727840AbfKZKJN (ORCPT ); Tue, 26 Nov 2019 05:09:13 -0500 Received: by mail-pg1-f196.google.com with SMTP id b10so8756108pgd.4; Tue, 26 Nov 2019 02:09:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=B6QMMXb6Lc0tWH98wR58tzl2/7IOcJXsAUoOf1rCNto=; b=TcQUjhhkWtAHHBo7BGhoPxdH+OkQBuhIxzcHX5EgouRjPkgbXzarm0y1vo1NhR8gDf eG2Lo7ddCpKcjSeJMFMlcP8tJQ/55Jt0TGNgznW4/XjmDldCe92mvnBxFGEghuAeWujG 7zNia4tH4IGRLOy70KvJXM+j82tbJ8nUIicPouLS6TjkV6HD27P/W/h6R5O7MmSu2k+V 143iMJdLR6+6bv9luW2P50ryb4MAyiIUafw49CErl7oj+EMEJ7fXiv42whVv9WX7vO8P re3ksZIi50XxOCmuXJZr+0KLAm6i5NuOZT8Bux4mTYLARNU4OGVu3EBoIPihOhj+s8jc e7dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=B6QMMXb6Lc0tWH98wR58tzl2/7IOcJXsAUoOf1rCNto=; b=V8nrArMfNn9l63g2ccOY/4Y/ynfagcMRUL4IYpvkBHFEG5wgUWs0uJWX6DGNIqgEa7 61TsaQi3utWaD4pqwHJFAnar0zobwLYA+Gv+9UZJDy/zksTL1tDa7Qvy1k64CVYyC0kN bnPhFyDPmsfGT9Wx02bRDsnxtxx83WhwlmGp2RjPtLxfdJ+Ko63a3DYGdGk/t/+g5DvE ijUnkgQ2kbY4AxL/Fw+SXgOAl63Ffv6CLiK2WagArcXZuN3izPPCCt5VeR37UWvmRjCo iGM72w7xPq0WOKiR+7xdQS6Dqu/H1PsUhwbVtefetTid72itYwS/FIawBAXKEtNGo5QE sK6w== X-Gm-Message-State: APjAAAXRZjHKEs66usO4UK+Juo3xnc27mB4tspUwMhI4IXnN5E1zeDT/ 0I8M3ihaWHZWnb0Dk5l0YZc= X-Google-Smtp-Source: APXvYqzpyS68KcI5oBdWBrSJfaGRRiRmODA4vA4zTy1YjkKqKqQWp2z/hxWs5tczO+UmDNsD9QUaSw== X-Received: by 2002:a63:d14d:: with SMTP id c13mr18566161pgj.227.1574762953032; Tue, 26 Nov 2019 02:09:13 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:12 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 08/18] tun: run offloaded XDP program in Tx path Date: Tue, 26 Nov 2019 19:07:34 +0900 Message-Id: <20191126100744.5083-9-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org run offloaded XDP program as soon as packet is removed from the ptr ring. Since this is XDP in Tx path, the traditional handling of XDP actions XDP_TX/REDIRECT isn't valid. For this reason we call do_xdp_generic_core instead of do_xdp_generic. do_xdp_generic_core just runs the program and leaves the action handling to us. Signed-off-by: Prashant Bhole --- drivers/net/tun.c | 149 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 146 insertions(+), 3 deletions(-) diff --git a/drivers/net/tun.c b/drivers/net/tun.c index ecb49101b0b5..466ea69f00ee 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -131,6 +131,7 @@ struct tap_filter { /* MAX_TAP_QUEUES 256 is chosen to allow rx/tx queues to be equal * to max number of VCPUs in guest. */ #define MAX_TAP_QUEUES 256 +#define MAX_TAP_BATCH 64 #define MAX_TAP_FLOWS 4096 #define TUN_FLOW_EXPIRE (3 * HZ) @@ -2156,6 +2157,109 @@ static ssize_t tun_put_user(struct tun_struct *tun, return total; } +static struct sk_buff *tun_prepare_xdp_skb(struct sk_buff *skb) +{ + struct sk_buff *nskb; + + if (skb_shared(skb) || skb_cloned(skb)) { + nskb = skb_copy(skb, GFP_ATOMIC); + consume_skb(skb); + return nskb; + } + + return skb; +} + +static u32 tun_do_xdp_offload_generic(struct tun_struct *tun, + struct sk_buff *skb) +{ + struct tun_prog *xdp_prog; + struct xdp_buff xdp; + u32 act = XDP_PASS; + + xdp_prog = rcu_dereference(tun->offloaded_xdp_prog); + if (xdp_prog) { + skb = tun_prepare_xdp_skb(skb); + if (!skb) { + act = XDP_DROP; + kfree_skb(skb); + goto drop; + } + + act = do_xdp_generic_core(skb, &xdp, xdp_prog->prog); + switch (act) { + case XDP_TX: + /* + * Rx path generic XDP will be called in this path + */ + netif_receive_skb(skb); + break; + case XDP_PASS: + break; + case XDP_REDIRECT: + /* + * Since we are not handling this case yet, let's free + * skb here. In case of XDP_DROP/XDP_ABORTED, the skb + * was already freed in do_xdp_generic_core() + */ + kfree_skb(skb); + /* fall through */ + default: + bpf_warn_invalid_xdp_action(act); + /* fall through */ + case XDP_ABORTED: + trace_xdp_exception(tun->dev, xdp_prog->prog, act); + /* fall through */ + case XDP_DROP: + goto drop; + } + } + + return act; +drop: + this_cpu_inc(tun->pcpu_stats->tx_dropped); + return act; +} + +static u32 tun_do_xdp_offload(struct tun_struct *tun, struct tun_file *tfile, + struct xdp_frame *frame) +{ + struct tun_prog *xdp_prog; + struct tun_page tpage; + struct xdp_buff xdp; + u32 act = XDP_PASS; + int flush = 0; + + xdp_prog = rcu_dereference(tun->offloaded_xdp_prog); + if (xdp_prog) { + xdp.data_hard_start = frame->data - frame->headroom; + xdp.data = frame->data; + xdp.data_end = xdp.data + frame->len; + xdp.data_meta = xdp.data - frame->metasize; + + act = bpf_prog_run_xdp(xdp_prog->prog, &xdp); + switch (act) { + case XDP_PASS: + break; + case XDP_TX: + /* fall through */ + case XDP_REDIRECT: + /* fall through */ + default: + bpf_warn_invalid_xdp_action(act); + /* fall through */ + case XDP_ABORTED: + trace_xdp_exception(tun->dev, xdp_prog->prog, act); + /* fall through */ + case XDP_DROP: + xdp_return_frame_rx_napi(frame); + break; + } + } + + return act; +} + static void *tun_ring_recv(struct tun_file *tfile, int noblock, int *err) { DECLARE_WAITQUEUE(wait, current); @@ -2574,6 +2678,47 @@ static int tun_sendmsg(struct socket *sock, struct msghdr *m, size_t total_len) return ret; } +static int tun_consume_packets(struct tun_file *tfile, void **ptr_array, int n) +{ + struct tun_prog *xdp_prog; + struct xdp_frame *frame; + struct tun_struct *tun; + int i, num_ptrs; + int pkt_cnt = 0; + void *pkts[MAX_TAP_BATCH]; + void *ptr; + u32 act; + + if (unlikely(!tfile)) + return 0; + + if (n > MAX_TAP_BATCH) + n = MAX_TAP_BATCH; + + rcu_read_lock(); + tun = rcu_dereference(tfile->tun); + if (unlikely(!tun)) + return 0; + xdp_prog = rcu_dereference(tun->offloaded_xdp_prog); + + num_ptrs = ptr_ring_consume_batched(&tfile->tx_ring, pkts, n); + for (i = 0; i < num_ptrs; i++) { + ptr = pkts[i]; + if (tun_is_xdp_frame(ptr)) { + frame = tun_ptr_to_xdp(ptr); + act = tun_do_xdp_offload(tun, tfile, frame); + } else { + act = tun_do_xdp_offload_generic(tun, ptr); + } + + if (act == XDP_PASS) + ptr_array[pkt_cnt++] = ptr; + } + + rcu_read_unlock(); + return pkt_cnt; +} + static int tun_recvmsg(struct socket *sock, struct msghdr *m, size_t total_len, int flags) { @@ -2594,9 +2739,7 @@ static int tun_recvmsg(struct socket *sock, struct msghdr *m, size_t total_len, ptr = ctl->ptr; break; case TUN_MSG_CONSUME_PKTS: - ret = ptr_ring_consume_batched(&tfile->tx_ring, - ctl->ptr, - ctl->num); + ret = tun_consume_packets(tfile, ctl->ptr, ctl->num); goto out; case TUN_MSG_UNCONSUME_PKTS: ptr_ring_unconsume(&tfile->tx_ring, ctl->ptr, From patchwork Tue Nov 26 10:07:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261801 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 475856C1 for ; Tue, 26 Nov 2019 10:09:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 282E42086A for ; Tue, 26 Nov 2019 10:09:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ajEYLyOj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727733AbfKZKJS (ORCPT ); Tue, 26 Nov 2019 05:09:18 -0500 Received: from mail-pg1-f193.google.com ([209.85.215.193]:46241 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727883AbfKZKJR (ORCPT ); Tue, 26 Nov 2019 05:09:17 -0500 Received: by mail-pg1-f193.google.com with SMTP id k1so385521pga.13; Tue, 26 Nov 2019 02:09:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=A+zZHbHPcf2+kZRFz8qYInOcnfbpxMipg8cRARobJQk=; b=ajEYLyOj6rPflxN/F4j7gzmTTfjkzFiOUEnj4x/mlPjwT6HScKGEYBOt5supKzMBi7 YBKGt62f0nlaJKL+4zEw20xBeu1qYjK6d3SjEQUAl982WY7KJ01zVqk8DP3SnV/E/4fE CvaL6nj6eGp7V+momNglWpxUiWUrQvkycbmynW0f+hE5TfdRhAIloHm1ZaWraGj06zkS pbNZdnR7LBGBbDTgO7C2YbNJOL1telar0aLkM/XV6T7hkE7/6YBVYNdFfy3Jeey0Oa37 By6HHXTNu/fbLWGSKnYoHiZtOXRdndYXg/KtDhSuf3L0sVTRKHyC/x4qHts5RUV3kdo8 txrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=A+zZHbHPcf2+kZRFz8qYInOcnfbpxMipg8cRARobJQk=; b=Z0MAh/IRdgjbiwNIKT+0/KZtQ3z/fxJyiaRV3jo2FkUPhO3FUQTq33A73L8YSfqIRZ uu0l5OW5Qtt523uviBiNy2hjr3gqiBsgCCWMkvQWb9RDxIP6d1MWu29tavP4WgjyHtEg SdBQ6KRM1kyzbPk6gmxCf23q5d+qNcYh1K9O50QoZaKLmdgF+Nl2KTys3UNS9v5t6JJ4 S0BbHJz+DakpDW2Qp5VCEbHa9OaJjKFxKmYMmwNhGRgkABwqEEHMVsOePhNN8QULD6Rh +NX9c/6jsG7rf9Cfi2fuEF7RTbPtA2CQjk5FWAaGsVhuY2KCBpSU2mUKSCZ5Rn5ofzN8 6utQ== X-Gm-Message-State: APjAAAW0kNftyHVD71KFGJQDKl5sx4Lx7h6U2jc1E5okZKW7qmA0RgSn uET5mbneGKHK3yfO9BZTgxA= X-Google-Smtp-Source: APXvYqza/1KXHr4vy/bIqcXkAf21DPD3YjEsUIsSGZ5/9H4FB/QXnB77j4wEUyEgpjV4ork9q/KtvA== X-Received: by 2002:a63:b40d:: with SMTP id s13mr38800406pgf.215.1574762957081; Tue, 26 Nov 2019 02:09:17 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:16 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 09/18] tun: add a way to inject Tx path packet into Rx path Date: Tue, 26 Nov 2019 19:07:35 +0900 Message-Id: <20191126100744.5083-10-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In order to support XDP_TX from offloaded XDP program, we need a way to inject Tx path packet into Rx path. Let's modify the Rx path function tun_xdp_one() for this purpose. This patch adds a parameter to pass information whether packet has virtio_net header. When header isn't present, it is considered as XDP_TX'ed packet from offloaded program. Signed-off-by: Prashant Bhole --- drivers/net/tun.c | 35 ++++++++++++++++++++++++++++------- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 466ea69f00ee..8d6cdd3e5139 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -2221,6 +2221,13 @@ static u32 tun_do_xdp_offload_generic(struct tun_struct *tun, return act; } +static int tun_xdp_one(struct tun_struct *tun, + struct tun_file *tfile, + struct xdp_buff *xdp, int *flush, + struct tun_page *tpage, int has_hdr); + +static void tun_put_page(struct tun_page *tpage); + static u32 tun_do_xdp_offload(struct tun_struct *tun, struct tun_file *tfile, struct xdp_frame *frame) { @@ -2527,23 +2534,36 @@ static void tun_put_page(struct tun_page *tpage) static int tun_xdp_one(struct tun_struct *tun, struct tun_file *tfile, struct xdp_buff *xdp, int *flush, - struct tun_page *tpage) + struct tun_page *tpage, int has_hdr) { unsigned int datasize = xdp->data_end - xdp->data; - struct tun_xdp_hdr *hdr = xdp->data_hard_start; - struct virtio_net_hdr *gso = &hdr->gso; + struct tun_xdp_hdr *hdr; + struct virtio_net_hdr *gso; struct tun_pcpu_stats *stats; struct bpf_prog *xdp_prog; struct sk_buff *skb = NULL; + unsigned int headroom; u32 rxhash = 0, act; - int buflen = hdr->buflen; + int buflen; int err = 0; bool skb_xdp = false; struct page *page; + if (has_hdr) { + hdr = xdp->data_hard_start; + gso = &hdr->gso; + buflen = hdr->buflen; + } else { + /* came here from tun tx path */ + xdp->data_hard_start -= sizeof(struct xdp_frame); + headroom = xdp->data - xdp->data_hard_start; + buflen = datasize + headroom + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); + } + xdp_prog = rcu_dereference(tun->xdp_prog); if (xdp_prog) { - if (gso->gso_type) { + if (has_hdr && gso->gso_type) { skb_xdp = true; goto build; } @@ -2588,7 +2608,8 @@ static int tun_xdp_one(struct tun_struct *tun, skb_reserve(skb, xdp->data - xdp->data_hard_start); skb_put(skb, xdp->data_end - xdp->data); - if (virtio_net_hdr_to_skb(skb, gso, tun_is_little_endian(tun))) { + if (has_hdr && + virtio_net_hdr_to_skb(skb, gso, tun_is_little_endian(tun))) { this_cpu_inc(tun->pcpu_stats->rx_frame_errors); kfree_skb(skb); err = -EINVAL; @@ -2652,7 +2673,7 @@ static int tun_sendmsg(struct socket *sock, struct msghdr *m, size_t total_len) for (i = 0; i < n; i++) { xdp = &((struct xdp_buff *)ctl->ptr)[i]; - tun_xdp_one(tun, tfile, xdp, &flush, &tpage); + tun_xdp_one(tun, tfile, xdp, &flush, &tpage, true); } if (flush) From patchwork Tue Nov 26 10:07:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261803 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 91FC66C1 for ; Tue, 26 Nov 2019 10:09:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 72D8B20874 for ; Tue, 26 Nov 2019 10:09:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fru48dzl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727891AbfKZKJX (ORCPT ); Tue, 26 Nov 2019 05:09:23 -0500 Received: from mail-pg1-f194.google.com ([209.85.215.194]:37074 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727858AbfKZKJV (ORCPT ); Tue, 26 Nov 2019 05:09:21 -0500 Received: by mail-pg1-f194.google.com with SMTP id b10so8756297pgd.4; Tue, 26 Nov 2019 02:09:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=svo/DJZyqQpUo02aclcCc0QABc66WKSHZ6zQKD7a+Ow=; b=fru48dzlPmxdsRDObkYQrHO7JapBQH3c9E855CQRNtWbayYXM0dPryQmi5/whFOlpS GH/fVBCqvL0AWAPenfff2xRlbHbJeyN0FRY1MhHhuPk2ZlwRT9JdOfr9u4eKWjej1HTC q8G8loaTXtCX8EcOlrVw8hLTEh4GJrvM6aVuqV67oJNuisl5NmKblB7E+689x4Qa7LIa 4SQq9JNWkXDoaqj5RJYNpcE68Ha/YC5zbmY6IAUN1+QQ+4kJDzhyh+LTm5ZfOUaLxAhM sWkA++CI/tDwSaPETlK0YUOCfMRpR+0Q+b9L8Xpbo+zpd9pTLP+X50M8f7aw4NZ0ZvnG QC3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=svo/DJZyqQpUo02aclcCc0QABc66WKSHZ6zQKD7a+Ow=; b=nwb2W9BvkVhZ9X0MOfqyGcHkYYjKx/N90VDG1PHT1cHSPOktrdA3SpMP8uWL4UMdEx MPqDm6akNZe2eAaW5jADgjrCPMTfQx+zbsf/0zBXLkYyRSj/CidQjWPS4533W4GP42TN 3sK2rt4p5Cj3tagGA3JqkSElN/FxvfAM2995QnjDkCwm+CA2erOIgtYgxfCtECdlWbpp 5sOPWSPEBhXNd/u3bohA73EjNDRJ7DWRSK04Y8ZJkTQe7Dk+774iKAJGNZWvnT6higBs JpE4iUSfHX4xk6wfZx1uh6Iq3gHPhHRdolAWG3I2BSExlkoBtcsjCn2gAmbH3PmMTijU cCgQ== X-Gm-Message-State: APjAAAUNFXiNjtd1TJx8RcJUf8EvM7dPqi0oF+qe3m7OTwSJRaIDfPd+ f3Dxu+OnmI5698Me2Yhud5A= X-Google-Smtp-Source: APXvYqx1KJySu2VSUlGFNdx7ooxLmUOdCiiNdkl+RVxrLvkON5zLWT+2U/MjtFSUA7MS5Y/R8paLmg== X-Received: by 2002:a63:ff26:: with SMTP id k38mr39206249pgi.128.1574762960938; Tue, 26 Nov 2019 02:09:20 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:20 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 10/18] tun: handle XDP_TX action of offloaded program Date: Tue, 26 Nov 2019 19:07:36 +0900 Message-Id: <20191126100744.5083-11-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When offloaded program returns XDP_TX, we need to inject the packet in Rx path of tun. This patch injects such packets in Rx path using tun_xdp_one. Signed-off-by: Prashant Bhole --- drivers/net/tun.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 8d6cdd3e5139..084ca95358fe 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -2249,7 +2249,13 @@ static u32 tun_do_xdp_offload(struct tun_struct *tun, struct tun_file *tfile, case XDP_PASS: break; case XDP_TX: - /* fall through */ + tpage.page = NULL; + tpage.count = 0; + tun_xdp_one(tun, tfile, &xdp, &flush, &tpage, false); + tun_put_page(&tpage); + if (flush) + xdp_do_flush_map(); + break; case XDP_REDIRECT: /* fall through */ default: From patchwork Tue Nov 26 10:07:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261805 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ECBE36C1 for ; Tue, 26 Nov 2019 10:09:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CE07620891 for ; Tue, 26 Nov 2019 10:09:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mc9u7EWT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727903AbfKZKJ0 (ORCPT ); Tue, 26 Nov 2019 05:09:26 -0500 Received: from mail-pg1-f195.google.com ([209.85.215.195]:37087 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727841AbfKZKJZ (ORCPT ); Tue, 26 Nov 2019 05:09:25 -0500 Received: by mail-pg1-f195.google.com with SMTP id b10so8756377pgd.4; Tue, 26 Nov 2019 02:09:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EWBtU+IlOUXhhtmkIhJ52D/yxzedCQhAg5N/fljwEYM=; b=mc9u7EWTbaAvtKBfR1FxVgVgSAqvpWaqH4I+At8oqCl0wPs4cfCfmxX5bUF+7APNl4 Xmw2dKPJ0fdmby7ayXU98TCOKpU3tQ1wLifizZBS9IU8NwE5tsD4IoVzY5pYlcirUpf9 GML0jDXsPkYpnJe+/RALaLB8XlL6sdizXEjybBT7EgZxdovS1whnnDXgS67wftULUchR O3p4sPYpo+nunkY6U7wrAUIFDpc4+vzXWVNuD0tkqqUfIITHpRM0zfXFvhm5VUEnlf0k FecGp1HqFADGdhQP1ea4gIcNAQRIySeCIanS7L8aCvBvaHue4K/B4u8Yk6YJPLPoV0QZ x0AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EWBtU+IlOUXhhtmkIhJ52D/yxzedCQhAg5N/fljwEYM=; b=cw+bDVHiE+2qHMbMmaKvT9YDDz11I0bZ7hRBqEhSgCAlZaMKcKd8V6kl+oAMv9mv8A wfezoVKQSNr4IdlGlVHfV2XsPHvjbQc87iXl9EeQssMBQmTvrsWwaRWFb1OdjvjCdjbC 8q5YvwVo2gpWBUg6Z0DRVbBBYxaZSDXzEEy+MhAKxu+L127arCnxJF5QGo+uIuXLNlGw RFWPy0EagwNSPZM2M3uPjzANPBOuP9VGUHL6i2393aNMVH59IXcu0UqufoCErodm9gYw pvSyR5n/3ahe7nVHaTtgjevmATGQPPO2eqgTM16hiLrKNaqhIxS1Z4nbtTFQTYsuZ4HK ekfQ== X-Gm-Message-State: APjAAAVcEGThHZsNJTT7fftvRn7tHA2bl2VmZYc3dvozzG/WB9jREAt6 c2uL0+88HyGHq52Z0suhe0g= X-Google-Smtp-Source: APXvYqz49czbgi+kYj34fPF5hK5fvqdKlFHvat1AI4Q3wYxfqfWIPNVeAlyJ9wlfZ+jcUET4SjDAZg== X-Received: by 2002:a63:6b87:: with SMTP id g129mr29079841pgc.438.1574762964825; Tue, 26 Nov 2019 02:09:24 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:24 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 11/18] tun: run xdp prog when tun is read from file interface Date: Tue, 26 Nov 2019 19:07:37 +0900 Message-Id: <20191126100744.5083-12-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It handles the case when qemu performs read on tun using file operations. Signed-off-by: Prashant Bhole --- drivers/net/tun.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 084ca95358fe..639921c10e32 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -2318,8 +2318,10 @@ static ssize_t tun_do_read(struct tun_struct *tun, struct tun_file *tfile, struct iov_iter *to, int noblock, void *ptr) { + struct xdp_frame *frame; ssize_t ret; int err; + u32 act; tun_debug(KERN_INFO, tun, "tun_do_read\n"); @@ -2333,6 +2335,15 @@ static ssize_t tun_do_read(struct tun_struct *tun, struct tun_file *tfile, ptr = tun_ring_recv(tfile, noblock, &err); if (!ptr) return err; + + if (tun_is_xdp_frame(ptr)) { + frame = tun_ptr_to_xdp(ptr); + act = tun_do_xdp_offload(tun, tfile, frame); + } else { + act = tun_do_xdp_offload_generic(tun, ptr); + } + if (act != XDP_PASS) + return err; } if (tun_is_xdp_frame(ptr)) { From patchwork Tue Nov 26 10:07:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261807 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 555086C1 for ; Tue, 26 Nov 2019 10:09:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2D88920895 for ; Tue, 26 Nov 2019 10:09:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qNxv9rom" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727901AbfKZKJa (ORCPT ); Tue, 26 Nov 2019 05:09:30 -0500 Received: from mail-pl1-f193.google.com ([209.85.214.193]:43964 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727838AbfKZKJ3 (ORCPT ); Tue, 26 Nov 2019 05:09:29 -0500 Received: by mail-pl1-f193.google.com with SMTP id q16so3746383plr.10; Tue, 26 Nov 2019 02:09:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=04y2jMBz+TniBsSijixdhxAgItaCRW1Bg/M7TEdqtZw=; b=qNxv9romfmc3q9HrRe1SsyMGoaUYYBfCAqMUu5CAK2O3s3+yZxepf2025cJPH3wVaq YdPZyUS+KjlG/AyXxckpOaJIFC/CznF+s9mOw22e/xJsgsAaSfC+QzI6FDrXI81xPOtl jHxPnXyPvnymn3KeFZk8QSD+AqaskNH3wrtAVq4sk73gCruVUF8q2EL+osao+obGfuXa E4Yvk9grWBs1sV7Ecjvmb2aJXX2sdhUMcocV9tdygaC/9AnOyqH6hyLQHPQRz4EYZYed OROth7uyN6VTu0xFGEmoI2bu6lRMgCUIYSq4c1GTkIT6StqdUJfKPl1MlLtm9JNoEsMG eJtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=04y2jMBz+TniBsSijixdhxAgItaCRW1Bg/M7TEdqtZw=; b=WuOt0R5P+aHp0NvYE6cx4qY6SXX1uIYItAvDicP83V2CYKFZh+tvSlKrJpytXbveK3 Vw+xGU6pr5Gur6eEmCA7MSBoGsnh5rO27wBUpgOvAQN9S56/kFpzWLp/V4PFpgDGohbc DmJ7oiLB1uJ3ptLLNCGzNouRdzNRrCIhWVH8ZGaQz7SlekmfyRbWpB+FH6jXscp9cYOP c3q1iXAy+uPpn9Mu+J9I2n5DOzl5rZe8duKeV9Qh4HaOVU5233TQNeaeZ5xi97Tw2VMG FMLpFpcHCV18wRkDQD4oUR/9wIu/uqKwwwzH3tJaBlBDDsAMyUOjP5Hqqc8y/H4huEe5 KzEw== X-Gm-Message-State: APjAAAVAI1d9oMvZoF8JHT500prMlK0YcDxWxtlGuaziRh30HNhFdj+q rThK9ZAfEuPIWehLIwhNCwo= X-Google-Smtp-Source: APXvYqwRnyofw+ROJrA/0vdTheynnaSCjgx1858Q2XAMEgN8s0N0WdIa2wiPcUAvtMOS75RBHUMq6A== X-Received: by 2002:a17:902:758a:: with SMTP id j10mr34601161pll.29.1574762968666; Tue, 26 Nov 2019 02:09:28 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:28 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 12/18] virtio-net: store xdp_prog in device Date: Tue, 26 Nov 2019 19:07:38 +0900 Message-Id: <20191126100744.5083-13-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang This is a preparation for adding XDP offload support in virtio_net driver. By storing XDP program in virtionet_info will make it consistent with the offloaded program which will introduce in next patches. Signed-off-by: Jason Wang Co-developed-by: Prashant Bhole Signed-off-by: Prashant Bhole --- drivers/net/virtio_net.c | 62 ++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 37 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 4d7d5434cc5d..c8bbb1b90c1c 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -137,8 +137,6 @@ struct receive_queue { struct napi_struct napi; - struct bpf_prog __rcu *xdp_prog; - struct virtnet_rq_stats stats; /* Chain pages by the private ptr. */ @@ -229,6 +227,8 @@ struct virtnet_info { /* failover when STANDBY feature enabled */ struct failover *failover; + + struct bpf_prog __rcu *xdp_prog; }; struct padded_vnet_hdr { @@ -486,7 +486,6 @@ static int virtnet_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames, u32 flags) { struct virtnet_info *vi = netdev_priv(dev); - struct receive_queue *rq = vi->rq; struct bpf_prog *xdp_prog; struct send_queue *sq; unsigned int len; @@ -501,7 +500,7 @@ static int virtnet_xdp_xmit(struct net_device *dev, /* Only allow ndo_xdp_xmit if XDP is loaded on dev, as this * indicate XDP resources have been successfully allocated. */ - xdp_prog = rcu_dereference(rq->xdp_prog); + xdp_prog = rcu_dereference(vi->xdp_prog); if (!xdp_prog) return -ENXIO; @@ -649,7 +648,7 @@ static struct sk_buff *receive_small(struct net_device *dev, stats->bytes += len; rcu_read_lock(); - xdp_prog = rcu_dereference(rq->xdp_prog); + xdp_prog = rcu_dereference(vi->xdp_prog); if (xdp_prog) { struct virtio_net_hdr_mrg_rxbuf *hdr = buf + header_offset; struct xdp_frame *xdpf; @@ -798,7 +797,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev, stats->bytes += len - vi->hdr_len; rcu_read_lock(); - xdp_prog = rcu_dereference(rq->xdp_prog); + xdp_prog = rcu_dereference(vi->xdp_prog); if (xdp_prog) { struct xdp_frame *xdpf; struct page *xdp_page; @@ -2060,7 +2059,7 @@ static int virtnet_set_channels(struct net_device *dev, * also when XDP is loaded all RX queues have XDP programs so we only * need to check a single RX queue. */ - if (vi->rq[0].xdp_prog) + if (vi->xdp_prog) return -EINVAL; get_online_cpus(); @@ -2441,13 +2440,10 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, return -ENOMEM; } - old_prog = rtnl_dereference(vi->rq[0].xdp_prog); + old_prog = rtnl_dereference(vi->xdp_prog); if (!prog && !old_prog) return 0; - if (prog) - bpf_prog_add(prog, vi->max_queue_pairs - 1); - /* Make sure NAPI is not using any XDP TX queues for RX. */ if (netif_running(dev)) { for (i = 0; i < vi->max_queue_pairs; i++) { @@ -2457,11 +2453,8 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, } if (!prog) { - for (i = 0; i < vi->max_queue_pairs; i++) { - rcu_assign_pointer(vi->rq[i].xdp_prog, prog); - if (i == 0) - virtnet_restore_guest_offloads(vi); - } + rcu_assign_pointer(vi->xdp_prog, prog); + virtnet_restore_guest_offloads(vi); synchronize_net(); } @@ -2472,16 +2465,12 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, vi->xdp_queue_pairs = xdp_qp; if (prog) { - for (i = 0; i < vi->max_queue_pairs; i++) { - rcu_assign_pointer(vi->rq[i].xdp_prog, prog); - if (i == 0 && !old_prog) - virtnet_clear_guest_offloads(vi); - } + rcu_assign_pointer(vi->xdp_prog, prog); + if (!old_prog) + virtnet_clear_guest_offloads(vi); } for (i = 0; i < vi->max_queue_pairs; i++) { - if (old_prog) - bpf_prog_put(old_prog); if (netif_running(dev)) { virtnet_napi_enable(vi->rq[i].vq, &vi->rq[i].napi); virtnet_napi_tx_enable(vi, vi->sq[i].vq, @@ -2489,13 +2478,15 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, } } + if (old_prog) + bpf_prog_put(old_prog); + return 0; err: if (!prog) { virtnet_clear_guest_offloads(vi); - for (i = 0; i < vi->max_queue_pairs; i++) - rcu_assign_pointer(vi->rq[i].xdp_prog, old_prog); + rcu_assign_pointer(vi->xdp_prog, old_prog); } if (netif_running(dev)) { @@ -2514,13 +2505,11 @@ static u32 virtnet_xdp_query(struct net_device *dev) { struct virtnet_info *vi = netdev_priv(dev); const struct bpf_prog *xdp_prog; - int i; - for (i = 0; i < vi->max_queue_pairs; i++) { - xdp_prog = rtnl_dereference(vi->rq[i].xdp_prog); - if (xdp_prog) - return xdp_prog->aux->id; - } + xdp_prog = rtnl_dereference(vi->xdp_prog); + if (xdp_prog) + return xdp_prog->aux->id; + return 0; } @@ -2657,18 +2646,17 @@ static void virtnet_free_queues(struct virtnet_info *vi) static void _free_receive_bufs(struct virtnet_info *vi) { - struct bpf_prog *old_prog; + struct bpf_prog *old_prog = rtnl_dereference(vi->xdp_prog); int i; for (i = 0; i < vi->max_queue_pairs; i++) { while (vi->rq[i].pages) __free_pages(get_a_page(&vi->rq[i], GFP_KERNEL), 0); - - old_prog = rtnl_dereference(vi->rq[i].xdp_prog); - RCU_INIT_POINTER(vi->rq[i].xdp_prog, NULL); - if (old_prog) - bpf_prog_put(old_prog); } + + RCU_INIT_POINTER(vi->xdp_prog, NULL); + if (old_prog) + bpf_prog_put(old_prog); } static void free_receive_bufs(struct virtnet_info *vi) From patchwork Tue Nov 26 10:07:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261809 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C9ED1393 for ; Tue, 26 Nov 2019 10:09:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5D5A0208A3 for ; Tue, 26 Nov 2019 10:09:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tLp6iRQl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727911AbfKZKJd (ORCPT ); Tue, 26 Nov 2019 05:09:33 -0500 Received: from mail-pf1-f194.google.com ([209.85.210.194]:38164 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727838AbfKZKJd (ORCPT ); Tue, 26 Nov 2019 05:09:33 -0500 Received: by mail-pf1-f194.google.com with SMTP id c13so8942511pfp.5; Tue, 26 Nov 2019 02:09:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zx5Ey469MhuUUqEJoZa/8N8K8LFzbIEk0ZZ64tnsKp0=; b=tLp6iRQl3sTOSSAx/gYoWpZG6XJZLoT8Zsh4xTn5YujxIAIlwEJMfJD/rbDLDKunxB PPLhWNxdNgkLYho5QUCo4jvD2cNjRkGWG6TUcgyp8JAnzvNAS/wBqESuBabVuZ58ZXMA CjvxUg68D5TAnjrU3ucmi0vThEn2NdeL2t0OCC+t1mzws0+Uo2GF7nK9INOfwKCC+uvF 8tn+hq13m8b2koupUYky/h2w8ql+9nXEVhJybe8LqphIjRZMW5HRezUzTCS/F9spcCWo ReC2AEI2NuSoPd4v59sHgkAEbAfWOp9Mdh5J9/gpiK41Xs5JEe0O3S4M624X2enKwNJd lrrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zx5Ey469MhuUUqEJoZa/8N8K8LFzbIEk0ZZ64tnsKp0=; b=Xyn05KXHuGDZPa6IURd56s1C5eaPAbbjXfs1vmSAI8+VLzPLNKow4BhntED7Fy5tEn 5OkCH9TSMy6ka6NFhaak63Sn4nrzOHnhy/SYiX7yF6HOWGPppNduUZ92BwzEo6suTzRm 8pN1CeWD/Kw1fwz7QdriT4TSv5rxu9olgzYR8/XiQmKYGxabHIRZVm1ftHBK3ghVmmF4 WajGlVM7ZaddoNOfjup0joY2dwKQ6WRNSfsdYzz99xlM1R5QSBLmCQOtDUXXg6BsvFxO Zx7RjG9fNir27mnPyIHXVwpaBkHktGVNL0CWTvzFyOyVOzk6hVPmQXLQdNr+KLbDT/sA cnrA== X-Gm-Message-State: APjAAAW6roiPrPwApyyR2ktolOJ+R77qbT4pekbeZoe7YgKIAdpzQQXq 83ZPQWOm+k5pr/vPSbOrwGk= X-Google-Smtp-Source: APXvYqy5nSGYMuA9W/Y541xqtiKlrCKtpPV27lN87wYp1YRIe1p9W24OXy6SRFejVlg/PWYn3qdtdA== X-Received: by 2002:a63:6b46:: with SMTP id g67mr37703871pgc.371.1574762972460; Tue, 26 Nov 2019 02:09:32 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:32 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 13/18] virtio_net: use XDP attachment helpers Date: Tue, 26 Nov 2019 19:07:39 +0900 Message-Id: <20191126100744.5083-14-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Next patches will introduce virtio_net XDP offloading. In that case we need to manage offloaded and non-offload program. In order to make it consistent this patch introduces use of XDP attachment helpers. Signed-off-by: Prashant Bhole --- drivers/net/virtio_net.c | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index c8bbb1b90c1c..cee5c2b15c62 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -229,6 +229,8 @@ struct virtnet_info { struct failover *failover; struct bpf_prog __rcu *xdp_prog; + + struct xdp_attachment_info xdp; }; struct padded_vnet_hdr { @@ -2398,15 +2400,19 @@ static int virtnet_restore_guest_offloads(struct virtnet_info *vi) return virtnet_set_guest_offloads(vi, offloads); } -static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, - struct netlink_ext_ack *extack) +static int virtnet_xdp_set(struct net_device *dev, struct netdev_bpf *bpf) { unsigned long int max_sz = PAGE_SIZE - sizeof(struct padded_vnet_hdr); + struct netlink_ext_ack *extack = bpf->extack; struct virtnet_info *vi = netdev_priv(dev); + struct bpf_prog *prog = bpf->prog; struct bpf_prog *old_prog; u16 xdp_qp = 0, curr_qp; int i, err; + if (!xdp_attachment_flags_ok(&vi->xdp, bpf)) + return -EBUSY; + if (!virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) && (virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO4) || virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO6) || @@ -2478,8 +2484,7 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, } } - if (old_prog) - bpf_prog_put(old_prog); + xdp_attachment_setup(&vi->xdp, bpf); return 0; @@ -2501,26 +2506,13 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, return err; } -static u32 virtnet_xdp_query(struct net_device *dev) -{ - struct virtnet_info *vi = netdev_priv(dev); - const struct bpf_prog *xdp_prog; - - xdp_prog = rtnl_dereference(vi->xdp_prog); - if (xdp_prog) - return xdp_prog->aux->id; - - return 0; -} - static int virtnet_xdp(struct net_device *dev, struct netdev_bpf *xdp) { switch (xdp->command) { case XDP_SETUP_PROG: - return virtnet_xdp_set(dev, xdp->prog, xdp->extack); + return virtnet_xdp_set(dev, xdp); case XDP_QUERY_PROG: - xdp->prog_id = virtnet_xdp_query(dev); - return 0; + return xdp_attachment_query(&vi->xdp, xdp); default: return -EINVAL; } From patchwork Tue Nov 26 10:07:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261815 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A37001393 for ; Tue, 26 Nov 2019 10:09:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7B250208CC for ; Tue, 26 Nov 2019 10:09:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fRo5+9ZK" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727927AbfKZKJi (ORCPT ); Tue, 26 Nov 2019 05:09:38 -0500 Received: from mail-pg1-f196.google.com ([209.85.215.196]:38312 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727841AbfKZKJh (ORCPT ); Tue, 26 Nov 2019 05:09:37 -0500 Received: by mail-pg1-f196.google.com with SMTP id t3so8308787pgl.5; Tue, 26 Nov 2019 02:09:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L8/5aZaE5i5lcakkRBtwP3FriVl4GQ+xN5YrOjIUlCM=; b=fRo5+9ZKWTD3s5uJLU9TNtKkQVCRPVzYcxVxl6Lb9D5Sflcesxx6GY10RfAqq10k3L 2dKkTp08HPOBLdXS0wZZPqLpVXU51DY9oXE2b9DC9UxnCDjdvcM1IukuHoVC1n4l7jyy ic145rX5r2EoWdTs4R+LaZL+vi+pwlT7G0BG07erZKekmBETQ63fioUFdc3gK1FBFBVl 0TnAFzy1s0cYyK+PPgWGbwH9g275XIrgsW06CMWNnSW9E8TnlJnbcCnYK2GVgBWiFkNI 7otab7zMtDrWC6ZLW4+mk7e6VL4XojLzPtRhSIbukm9LRqIKdMUAAATJq1dn7YseW9QI I8lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L8/5aZaE5i5lcakkRBtwP3FriVl4GQ+xN5YrOjIUlCM=; b=TxGFq6/WDGnC6xCBdSmMRw7BOrpqy8GGzYxkNLo7bBLibZXie5XruAd0aHhZ3q2ZnT Zk7lYlRV09gjUzusoqtk1MlsWHlvIIxTNugjaxNVscjhr4JANGK4Z+lYH7/94VHI/EVT QbvS87ExJcuz99FPIhuvMq46chLhaLVr2pEevRXjRcblQY5sV0NFiEN4GY5oHu8fRvNX EfvdJAF5SCsnas/FAWcpRNaP2C7QatL0O0P5pGbe9hoEmXuGw1+FVJKMRD9NCoUsz3LE uvgRP8hRskeq6s+duSHg0IX5yH85bmE6j0S1Gsp0FIZJuWoBDnpriD6dkIh9YLs0D7Vc XT/Q== X-Gm-Message-State: APjAAAWTenG4hAP250zK0h8MZbzMaFuni5X4gV4PzDDeaO8Bjtptzlqm kSiZs+opJcthdtc4ndxdq18= X-Google-Smtp-Source: APXvYqwM6Ee33ApLWb/jq6ETiPN5BAkCEQ3rIFbqFuJ25Hzp+y84mxjBl6KWK1nKTl45ek26gmxnEQ== X-Received: by 2002:aa7:90d0:: with SMTP id k16mr40811463pfk.131.1574762976474; Tue, 26 Nov 2019 02:09:36 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:35 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 14/18] virtio_net: add XDP prog offload infrastructure Date: Tue, 26 Nov 2019 19:07:40 +0900 Message-Id: <20191126100744.5083-15-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang This patch prepares virtio_net of XDP offloading. It adds data structures, blank callback implementations for bpf_prog_offload_ops. It also implements ndo_init, ndo_uninit operations for setting up offload related data structures. Signed-off-by: Jason Wang Co-developed-by: Prashant Bhole Signed-off-by: Prashant Bhole --- drivers/net/virtio_net.c | 103 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index cee5c2b15c62..a1088d0114f2 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -229,8 +229,14 @@ struct virtnet_info { struct failover *failover; struct bpf_prog __rcu *xdp_prog; + struct bpf_prog __rcu *offload_xdp_prog; struct xdp_attachment_info xdp; + struct xdp_attachment_info xdp_hw; + + struct bpf_offload_dev *bpf_dev; + + struct list_head bpf_bound_progs; }; struct padded_vnet_hdr { @@ -258,6 +264,14 @@ static struct xdp_frame *ptr_to_xdp(void *ptr) return (struct xdp_frame *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); } +struct virtnet_bpf_bound_prog { + struct virtnet_info *vi; + struct bpf_prog *prog; + struct list_head list; + u32 len; + struct bpf_insn insnsi[0]; +}; + /* Converting between virtqueue no. and kernel tx/rx queue no. * 0:rx0 1:tx0 2:rx1 3:tx1 ... 2N:rxN 2N+1:txN 2N+2:cvq */ @@ -2506,13 +2520,63 @@ static int virtnet_xdp_set(struct net_device *dev, struct netdev_bpf *bpf) return err; } +static int virtnet_bpf_verify_insn(struct bpf_verifier_env *env, int insn_idx, + int prev_insn) +{ + return 0; +} + +static void virtnet_bpf_destroy_prog(struct bpf_prog *prog) +{ +} + +static int virtnet_xdp_set_offload(struct virtnet_info *vi, + struct netdev_bpf *bpf) +{ + return -EBUSY; +} + +static int virtnet_bpf_verifier_setup(struct bpf_prog *prog) +{ + return -ENOMEM; +} + +static int virtnet_bpf_verifier_prep(struct bpf_prog *prog) +{ + return 0; +} + +static int virtnet_bpf_translate(struct bpf_prog *prog) +{ + return 0; +} + +static int virtnet_bpf_finalize(struct bpf_verifier_env *env) +{ + return 0; +} + +static const struct bpf_prog_offload_ops virtnet_bpf_dev_ops = { + .setup = virtnet_bpf_verifier_setup, + .prepare = virtnet_bpf_verifier_prep, + .insn_hook = virtnet_bpf_verify_insn, + .finalize = virtnet_bpf_finalize, + .translate = virtnet_bpf_translate, + .destroy = virtnet_bpf_destroy_prog, +}; + static int virtnet_xdp(struct net_device *dev, struct netdev_bpf *xdp) { + struct virtnet_info *vi = netdev_priv(dev); switch (xdp->command) { case XDP_SETUP_PROG: return virtnet_xdp_set(dev, xdp); case XDP_QUERY_PROG: return xdp_attachment_query(&vi->xdp, xdp); + case XDP_SETUP_PROG_HW: + return virtnet_xdp_set_offload(vi, xdp); + case XDP_QUERY_PROG_HW: + return xdp_attachment_query(&vi->xdp_hw, xdp); default: return -EINVAL; } @@ -2559,7 +2623,46 @@ static int virtnet_set_features(struct net_device *dev, return 0; } +static int virtnet_bpf_init(struct virtnet_info *vi) +{ + int err; + + vi->bpf_dev = bpf_offload_dev_create(&virtnet_bpf_dev_ops, NULL); + err = PTR_ERR_OR_ZERO(vi->bpf_dev); + if (err) + return err; + + err = bpf_offload_dev_netdev_register(vi->bpf_dev, vi->dev); + if (err) + goto err_netdev_register; + + INIT_LIST_HEAD(&vi->bpf_bound_progs); + + return 0; + +err_netdev_register: + bpf_offload_dev_destroy(vi->bpf_dev); + return err; +} + +static int virtnet_init(struct net_device *dev) +{ + struct virtnet_info *vi = netdev_priv(dev); + + return virtnet_bpf_init(vi); +} + +static void virtnet_uninit(struct net_device *dev) +{ + struct virtnet_info *vi = netdev_priv(dev); + + bpf_offload_dev_netdev_unregister(vi->bpf_dev, vi->dev); + bpf_offload_dev_destroy(vi->bpf_dev); +} + static const struct net_device_ops virtnet_netdev = { + .ndo_init = virtnet_init, + .ndo_uninit = virtnet_uninit, .ndo_open = virtnet_open, .ndo_stop = virtnet_close, .ndo_start_xmit = start_xmit, From patchwork Tue Nov 26 10:07:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261817 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 07A616C1 for ; Tue, 26 Nov 2019 10:09:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D4014208D4 for ; Tue, 26 Nov 2019 10:09:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="D0UYZK0R" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727841AbfKZKJm (ORCPT ); Tue, 26 Nov 2019 05:09:42 -0500 Received: from mail-pg1-f193.google.com ([209.85.215.193]:39264 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727922AbfKZKJl (ORCPT ); Tue, 26 Nov 2019 05:09:41 -0500 Received: by mail-pg1-f193.google.com with SMTP id b137so6441846pga.6; Tue, 26 Nov 2019 02:09:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iF2Qj2HQU3QXOgxa/8KPlCCJG4r2E8XApomblPiSB8E=; b=D0UYZK0RE+Et5nv6Qca/F1vgfk1JQVgqSzdMnDmmjurIApyqdCxw6DeQsL8FYTA6u/ KwVzke5RnI0wtXSENtsBmA8wDFxTd6hG+npLFS8bRPb9ugm3/hVV5z8O1I9WDK95KvSE kH3jNxWjPH+jsjkKFHaRrfhsgjCVFwLsD7LNo1xglTVJoWhF1DhwV1ZhDxH5TPdVm/1r XDIiXt1wqIwi3UvE8SfYxp70qrphSnVs70SuuC1ugFHwVuJoBJppJ8Kw/nBjLSqSmkeJ uR01Y6/l83pYdXi8KDjWnm48kQRtavKWqtKuyKEs9AlkD1GFUuFo21izmbRfYwC46KKt Emag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iF2Qj2HQU3QXOgxa/8KPlCCJG4r2E8XApomblPiSB8E=; b=ca5kDjmlSZyQZvDtvnApy6FXPLL9i8IYrrBmu0nPBE/liTu2amYigwAjzV9uZJ+oAg tYN/XMfwqsTam25rFRQFFtX2xn/KriVsMPM6KDM7r1SGM79+4yA6CAMXwrQ4D1cBpKHg H8WgMgxu5BQLsKvnv3B3PZT4YPJ+2cEkBOP9fCv0CerlD2DV4OpFuT9k8Uv6OTZl/3S4 jwBHd+GlxDZ9WyQ5gib9xps++E2hOAe4Roxy8PTUbDWjiFatfW8adpENvyEZrYrmhXdH WBe/rCdWqkmXLhRbN3B9qUjOb9YbvGPV83JVSm1+UKGBQ0elcVxkabdzjlCL5WjyKk7R BQBw== X-Gm-Message-State: APjAAAWbfbLF8Vw9PIXkkQEtPvJjAkrp+5ceUk6c91q2AVyXxcJg5Vdz gsPyT6zCgs8ZHV7vg/YA2k4= X-Google-Smtp-Source: APXvYqz+uMctIG2ZIs56c5eSzst4/2rCLxB7JzbDXh2IKeUxkAbdP3247ZE0k9PRARnctTS5VQGjqQ== X-Received: by 2002:a63:e647:: with SMTP id p7mr38714558pgj.47.1574762980220; Tue, 26 Nov 2019 02:09:40 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:39 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 15/18] virtio_net: implement XDP prog offload functionality Date: Tue, 26 Nov 2019 19:07:41 +0900 Message-Id: <20191126100744.5083-16-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang This patch implements bpf_prog_offload_ops callbacks and adds handling for XDP_SETUP_PROG_HW. Handling of XDP_SETUP_PROG_HW involves setting up struct virtio_net_ctrl_ebpf_prog and appending program instructions to it. This control buffer is sent to Qemu with class VIRTIO_NET_CTRL_EBPF and command VIRTIO_NET_BPF_CMD_SET_OFFLOAD. The expected behavior from Qemu is that it should try to load the program in host os and report the status. It also adds restriction to have either driver or offloaded program at a time. This restriction can be removed later after proper testing. Signed-off-by: Jason Wang Co-developed-by: Prashant Bhole Signed-off-by: Prashant Bhole --- drivers/net/virtio_net.c | 114 +++++++++++++++++++++++++++++++- include/uapi/linux/virtio_net.h | 27 ++++++++ 2 files changed, 139 insertions(+), 2 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index a1088d0114f2..dddfbb2a2075 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -169,6 +169,7 @@ struct control_buf { u8 allmulti; __virtio16 vid; __virtio64 offloads; + struct virtio_net_ctrl_ebpf_prog prog_ctrl; }; struct virtnet_info { @@ -272,6 +273,8 @@ struct virtnet_bpf_bound_prog { struct bpf_insn insnsi[0]; }; +#define VIRTNET_EA(extack, msg) NL_SET_ERR_MSG_MOD((extack), msg) + /* Converting between virtqueue no. and kernel tx/rx queue no. * 0:rx0 1:tx0 2:rx1 3:tx1 ... 2N:rxN 2N+1:txN 2N+2:cvq */ @@ -2427,6 +2430,11 @@ static int virtnet_xdp_set(struct net_device *dev, struct netdev_bpf *bpf) if (!xdp_attachment_flags_ok(&vi->xdp, bpf)) return -EBUSY; + if (rtnl_dereference(vi->offload_xdp_prog)) { + VIRTNET_EA(bpf->extack, "program already attached in offload mode"); + return -EINVAL; + } + if (!virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) && (virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO4) || virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO6) || @@ -2528,17 +2536,114 @@ static int virtnet_bpf_verify_insn(struct bpf_verifier_env *env, int insn_idx, static void virtnet_bpf_destroy_prog(struct bpf_prog *prog) { + struct virtnet_bpf_bound_prog *state; + + state = prog->aux->offload->dev_priv; + list_del(&state->list); + kfree(state); +} + +static int virtnet_xdp_offload_check(struct virtnet_info *vi, + struct netdev_bpf *bpf) +{ + if (!bpf->prog) + return 0; + + if (!bpf->prog->aux->offload) { + VIRTNET_EA(bpf->extack, "xdpoffload of non-bound program"); + return -EINVAL; + } + if (bpf->prog->aux->offload->netdev != vi->dev) { + VIRTNET_EA(bpf->extack, "program bound to different dev"); + return -EINVAL; + } + + if (rtnl_dereference(vi->xdp_prog)) { + VIRTNET_EA(bpf->extack, "program already attached in driver mode"); + return -EINVAL; + } + + return 0; } static int virtnet_xdp_set_offload(struct virtnet_info *vi, struct netdev_bpf *bpf) { - return -EBUSY; + struct virtio_net_ctrl_ebpf_prog *ctrl; + struct virtnet_bpf_bound_prog *bound_prog = NULL; + struct virtio_device *vdev = vi->vdev; + struct bpf_prog *prog = bpf->prog; + void *ctrl_buf = NULL; + struct scatterlist sg; + int prog_len; + int err = 0; + + if (!xdp_attachment_flags_ok(&vi->xdp_hw, bpf)) + return -EBUSY; + + if (prog) { + if (prog->type != BPF_PROG_TYPE_XDP) + return -EOPNOTSUPP; + bound_prog = prog->aux->offload->dev_priv; + prog_len = prog->len * sizeof(bound_prog->insnsi[0]); + + ctrl_buf = kmalloc(GFP_KERNEL, sizeof(*ctrl) + prog_len); + if (!ctrl_buf) + return -ENOMEM; + ctrl = ctrl_buf; + ctrl->cmd = cpu_to_virtio32(vi->vdev, + VIRTIO_NET_BPF_CMD_SET_OFFLOAD); + ctrl->len = cpu_to_virtio32(vi->vdev, prog_len); + ctrl->gpl_compatible = cpu_to_virtio16(vi->vdev, + prog->gpl_compatible); + memcpy(ctrl->insns, bound_prog->insnsi, + prog->len * sizeof(bound_prog->insnsi[0])); + sg_init_one(&sg, ctrl_buf, sizeof(*ctrl) + prog_len); + } else { + ctrl = &vi->ctrl->prog_ctrl; + ctrl->cmd = cpu_to_virtio32(vi->vdev, + VIRTIO_NET_BPF_CMD_UNSET_OFFLOAD); + sg_init_one(&sg, ctrl, sizeof(*ctrl)); + } + + if (!virtnet_send_command(vi, VIRTIO_NET_CTRL_EBPF, + VIRTIO_NET_CTRL_EBPF_PROG, + &sg)) { + dev_warn(&vdev->dev, "Failed to set bpf offload prog\n"); + err = -EFAULT; + goto out; + } + + rcu_assign_pointer(vi->offload_xdp_prog, prog); + + xdp_attachment_setup(&vi->xdp_hw, bpf); + +out: + kfree(ctrl_buf); + return err; } static int virtnet_bpf_verifier_setup(struct bpf_prog *prog) { - return -ENOMEM; + struct virtnet_info *vi = netdev_priv(prog->aux->offload->netdev); + size_t insn_len = prog->len * sizeof(struct bpf_insn); + struct virtnet_bpf_bound_prog *state; + + state = kzalloc(sizeof(*state) + insn_len, GFP_KERNEL); + if (!state) + return -ENOMEM; + + memcpy(&state->insnsi[0], prog->insnsi, insn_len); + + state->vi = vi; + state->prog = prog; + state->len = prog->len; + + list_add_tail(&state->list, &vi->bpf_bound_progs); + + prog->aux->offload->dev_priv = state; + + return 0; } static int virtnet_bpf_verifier_prep(struct bpf_prog *prog) @@ -2568,12 +2673,17 @@ static const struct bpf_prog_offload_ops virtnet_bpf_dev_ops = { static int virtnet_xdp(struct net_device *dev, struct netdev_bpf *xdp) { struct virtnet_info *vi = netdev_priv(dev); + int err; + switch (xdp->command) { case XDP_SETUP_PROG: return virtnet_xdp_set(dev, xdp); case XDP_QUERY_PROG: return xdp_attachment_query(&vi->xdp, xdp); case XDP_SETUP_PROG_HW: + err = virtnet_xdp_offload_check(vi, xdp); + if (err) + return err; return virtnet_xdp_set_offload(vi, xdp); case XDP_QUERY_PROG_HW: return xdp_attachment_query(&vi->xdp_hw, xdp); diff --git a/include/uapi/linux/virtio_net.h b/include/uapi/linux/virtio_net.h index a3715a3224c1..0ea2f7910a5a 100644 --- a/include/uapi/linux/virtio_net.h +++ b/include/uapi/linux/virtio_net.h @@ -261,4 +261,31 @@ struct virtio_net_ctrl_mq { #define VIRTIO_NET_CTRL_GUEST_OFFLOADS 5 #define VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET 0 +/* + * Control XDP offloads offloads + * + * When guest wants to offload XDP program to the device, it calls + * VIRTIO_NET_CTRL_EBPF_PROG along with VIRTIO_NET_BPF_CMD_SET_OFFLOAD + * subcommands. When offloading is successful, the device runs offloaded + * XDP program for each packet before sending it to the guest. + * + * VIRTIO_NET_BPF_CMD_UNSET_OFFLOAD removes the the offloaded program from + * the device, if exists. + */ + +struct virtio_net_ctrl_ebpf_prog { + /* program length in bytes */ + __virtio32 len; + __virtio16 cmd; + __virtio16 gpl_compatible; + __u8 insns[0]; +}; + +#define VIRTIO_NET_CTRL_EBPF 6 + #define VIRTIO_NET_CTRL_EBPF_PROG 1 + +/* Commands for VIRTIO_NET_CTRL_EBPF_PROG */ +#define VIRTIO_NET_BPF_CMD_SET_OFFLOAD 1 +#define VIRTIO_NET_BPF_CMD_UNSET_OFFLOAD 2 + #endif /* _UAPI_LINUX_VIRTIO_NET_H */ From patchwork Tue Nov 26 10:07:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261819 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 546E76C1 for ; Tue, 26 Nov 2019 10:09:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 34C41208D4 for ; Tue, 26 Nov 2019 10:09:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="V/YUucEh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727941AbfKZKJp (ORCPT ); Tue, 26 Nov 2019 05:09:45 -0500 Received: from mail-pg1-f195.google.com ([209.85.215.195]:37143 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727919AbfKZKJo (ORCPT ); Tue, 26 Nov 2019 05:09:44 -0500 Received: by mail-pg1-f195.google.com with SMTP id b10so8756821pgd.4; Tue, 26 Nov 2019 02:09:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pUc8XgU3PRUIYnmGD1sk4dmdkx2zuWhcKZo2RCjAbj4=; b=V/YUucEhcvTyJgZSWsi5RwLlFSOkklklct0nM4joSGWaIR7na/no+LPxPiWVwQxxt3 MZTId0BfvRrcBB5cS/xNn6YkR/nR/a/oFs+qoKWNnr42ZGOZSgTJb1F9wGQ/O7rl4ool fqDK2tc6OlvGgzuoLOzoJUJQHilZeQPmXpSCsMF/iAGRQPq1estAj8ZW+DJ3zXLxs0I/ Q9QpB8OHiq3d1Lt0nQ7YBRWLpuxJLuJe9KhDCvCcZTqb9EKaythXaA5hLTcRTjhHTcg0 Zmy8SaIL9vFHoxTDHybOLMp6I58SLo01Hd5rnDgBZPghb4JF9K2no8TAUZxDYsM3NvAV RVqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pUc8XgU3PRUIYnmGD1sk4dmdkx2zuWhcKZo2RCjAbj4=; b=O9+qpQW0FhQD8rPBmhv4FycRYTSLSamB6YnyBj76qcm5GvWuNhlSy0ktfHNCjRMV4R 4M50zfTf2kDCb893TM8dPrNiaTBr/GdIYBMwuPWc6qNlRFjp8mTMoaZfxHNJBsgm8PwY 7CEd5Wg+kIM3WFOvYCpNPp0CgA7h/oWeZKhF55BusraV8zstB2M6x/wSSb013aC95Xvz ROH0XzJ02ZdJHxWr3OUBNqllVlcByqwxpKg78xI+3la1uS1zPYRVsbv2u5rWsOfoG2bD zVidoAdfK2N+Utj1M4y9sxxGPyybeR8j6NP6t9eJ8U7Z3DNsQ7YqrBhoPhp/HTzv/uGJ iY8A== X-Gm-Message-State: APjAAAXTBSXTeOkRSY1MmCbt5/OPR09c6s4EYr573H8a86PWUEhHoJPI 232w9TFST/afPiaQrc9jUmA= X-Google-Smtp-Source: APXvYqyXGcO2rx/7HpN4RNXvapDE7gQDbHTJMsH1+iqGmKXZYJEaN21RZ6AzcvVf5dFk+Y1nNp9XGQ== X-Received: by 2002:a63:6b87:: with SMTP id g129mr29081453pgc.438.1574762984061; Tue, 26 Nov 2019 02:09:44 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:43 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 16/18] bpf: export function __bpf_map_get Date: Tue, 26 Nov 2019 19:07:42 +0900 Message-Id: <20191126100744.5083-17-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang __bpf_map_get is necessary to get verify whether an fd corresponds to a bpf map, without adding a refcount on that map. After exporting it can be used by a kernel module. Signed-off-by: Jason Wang Signed-off-by: Prashant Bhole --- kernel/bpf/syscall.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index e3461ec59570..e524ab1e7c64 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -737,6 +737,7 @@ struct bpf_map *__bpf_map_get(struct fd f) return f.file->private_data; } +EXPORT_SYMBOL(__bpf_map_get); void bpf_map_inc(struct bpf_map *map) { From patchwork Tue Nov 26 10:07:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261825 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3D1031393 for ; Tue, 26 Nov 2019 10:09:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 15874208CE for ; Tue, 26 Nov 2019 10:09:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i+EvPSxf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727972AbfKZKJu (ORCPT ); Tue, 26 Nov 2019 05:09:50 -0500 Received: from mail-pj1-f68.google.com ([209.85.216.68]:46821 "EHLO mail-pj1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727763AbfKZKJt (ORCPT ); Tue, 26 Nov 2019 05:09:49 -0500 Received: by mail-pj1-f68.google.com with SMTP id a16so8057974pjs.13; Tue, 26 Nov 2019 02:09:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Zk3sW22S1vh0AAHBme1qJ6RZt3yGG+m+CCZCVNC9O+U=; b=i+EvPSxfehY7pdZKrtOH/92CceCXUXqxQNHLS8Dgi76SvXlg05ezIAKQ4FdC4oJRA+ vkIRYqqni/0dSQQeSIisOK9PmMqJEgS8HBbC5oAS1K4Uogo9Std2Szp8RPbhFubRLtWa GAXaGUhUjWJzY99ZSABBweLRJ3wy66M7ruI46vL6dB8E6ZUse7Ub9ld8rb+k1+hnefYS Lt/XYCNpBDHO399apLSYB4kp81Zz1DYQ9MVL5fQ+E6xSj57jgVaqCw1zZFeGxt2KCgKi KQ+8dAkH6CzVS39i6plLht441sTUmKWtdf7AhRHO7SrQC/HcuCz7Iur7n3eP/5PfybuL Oj3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zk3sW22S1vh0AAHBme1qJ6RZt3yGG+m+CCZCVNC9O+U=; b=WSfuNASXqicEzKBWmKKf8lfQ0HPMWvpycnEKQ3BMYdkEVQe7XmQlKuvxwnwk04I0Gh FpRVnZyVPqmwitc1vN5l6QWb+rlDl7Cc+XIpAUDi3gpi3+QszNPmAS8xgWVF5QeRK7OB MJ702lfW/W+4WaVJUMx2O9Fy8qOiJXIaLNmeQjkr9d5kZXiYEAVA9p/X/bkHA3m3nUCr tesOUNIXfsJ2SizsiFdcqNPW/DWQ8Qn/2WXj1aUZ6DnNxGNN/+1uMvltJoE3RiL7AIJ+ eqmrNIqnV2xBxfHSs1igsqc+++BbygnV9wEXMETCfDsXo2pXjVKn42NHbxnXQZ5vFMl8 SCRg== X-Gm-Message-State: APjAAAX31EE0jDPXGQtuyYf1trlBK9YCmngLqSBBsW/eGzKfDJah9Vhp UyOuRw6l6t7zD6AcK7MibOc= X-Google-Smtp-Source: APXvYqzFyFh+pbLegLoH9n9d2gmpHvn7dEVY5afqLnbUM7fE1SOq8IDt/3EeOo1AM4jKJu5rmRGdoA== X-Received: by 2002:a17:902:8f81:: with SMTP id z1mr32304035plo.228.1574762987931; Tue, 26 Nov 2019 02:09:47 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:47 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org, Prashant Bhole Subject: [RFC net-next 17/18] virtio_net: implment XDP map offload functionality Date: Tue, 26 Nov 2019 19:07:43 +0900 Message-Id: <20191126100744.5083-18-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jason Wang This patch implements: * Handling of BPF_OFFLOAD_MAP_ALLOC, BPF_OFFLOAD_MAP_FREE: Allocate driver specific map data structure. Set up struct virtio_net_ctrl_ebpf_map and send the control buffer to Qemu with class VIRTIO_NET_CTRL_EBPF, cmd VIRTIO_NET_CTRL_EBPF_MAP. The cmd in the control buffer is set to VIRTIO_NET_BPF_CMD_CREATE_MAP. The expected behavior from Qemu is that it should perform the action as per command and return the status (and map data). In case of create map command, Qemu should set the map_fd in the control buffer * bpf_map_dev_ops operations: Common map operations are implemented with use of above mentioned struct virtio_net_ctrl_ebpf_map. This control buffer has space for storing: key + key or key + value. * map_fd replacement in a copy of the program: Since map are created before the verification of program begins, we have map fds from the host side for each offloaded map when program verification begins. map fds in the copy of the program are replaced with map fds from host side. This copy of program is used for offloading. Signed-off-by: Jason Wang Co-developed-by: Prashant Bhole Signed-off-by: Prashant Bhole --- drivers/net/virtio_net.c | 241 ++++++++++++++++++++++++++++++++ include/uapi/linux/virtio_net.h | 23 +++ 2 files changed, 264 insertions(+) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index dddfbb2a2075..91a94b787c64 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -238,6 +238,7 @@ struct virtnet_info { struct bpf_offload_dev *bpf_dev; struct list_head bpf_bound_progs; + struct list_head map_list; }; struct padded_vnet_hdr { @@ -275,6 +276,13 @@ struct virtnet_bpf_bound_prog { #define VIRTNET_EA(extack, msg) NL_SET_ERR_MSG_MOD((extack), msg) +struct virtnet_bpf_map { + struct bpf_offloaded_map *offmap; + struct virtnet_info *vi; + struct virtio_net_ctrl_ebpf_map *ctrl; + struct list_head list; +}; + /* Converting between virtqueue no. and kernel tx/rx queue no. * 0:rx0 1:tx0 2:rx1 3:tx1 ... 2N:rxN 2N+1:txN 2N+2:cvq */ @@ -2528,6 +2536,19 @@ static int virtnet_xdp_set(struct net_device *dev, struct netdev_bpf *bpf) return err; } +static struct virtnet_bpf_map *virtnet_get_bpf_map(struct virtnet_info *vi, + struct bpf_map *map) +{ + struct virtnet_bpf_map *virtnet_map; + + list_for_each_entry(virtnet_map, &vi->map_list, list) { + if (&virtnet_map->offmap->map == map) + return virtnet_map; + } + + return NULL; +} + static int virtnet_bpf_verify_insn(struct bpf_verifier_env *env, int insn_idx, int prev_insn) { @@ -2623,11 +2644,194 @@ static int virtnet_xdp_set_offload(struct virtnet_info *vi, return err; } +static int virtnet_bpf_ctrl_map(struct bpf_offloaded_map *offmap, + int cmd, u8 *key, u8 *value, u64 flags, + u8 *out_key, u8 *out_value) +{ + struct virtio_net_ctrl_ebpf_map *ctrl; + struct virtnet_bpf_map *virtnet_map; + struct bpf_map *map = &offmap->map; + unsigned char *keyptr, *valptr; + struct virtnet_info *vi; + struct scatterlist sg; + + virtnet_map = offmap->dev_priv; + vi = virtnet_map->vi; + ctrl = virtnet_map->ctrl; + + keyptr = ctrl->buf; + valptr = ctrl->buf + ctrl->key_size; + + if (key) + memcpy(keyptr, key, map->key_size); + if (value) + memcpy(valptr, value, map->value_size); + + ctrl->cmd = cpu_to_virtio32(vi->vdev, cmd); + ctrl->flags = cpu_to_virtio64(vi->vdev, flags); + + sg_init_one(&sg, ctrl, sizeof(*ctrl) + ctrl->buf_len); + if (!virtnet_send_command(vi, VIRTIO_NET_CTRL_EBPF, + VIRTIO_NET_CTRL_EBPF_MAP, + &sg)) + return -EFAULT; + + if (out_key) + memcpy(out_key, valptr, map->key_size); + if (out_value) + memcpy(out_value, valptr, map->value_size); + return 0; +} + +static int virtnet_bpf_map_update_entry(struct bpf_offloaded_map *offmap, + void *key, void *value, u64 flags) +{ + return virtnet_bpf_ctrl_map(offmap, + VIRTIO_NET_BPF_CMD_UPDATE_ELEM, + key, value, flags, NULL, NULL); +} + +static int virtnet_bpf_map_delete_elem(struct bpf_offloaded_map *offmap, + void *key) +{ + return virtnet_bpf_ctrl_map(offmap, + VIRTIO_NET_BPF_CMD_DELETE_ELEM, + key, NULL, 0, NULL, NULL); +} + +static int virtnet_bpf_map_lookup_entry(struct bpf_offloaded_map *offmap, + void *key, void *value) +{ + return virtnet_bpf_ctrl_map(offmap, + VIRTIO_NET_BPF_CMD_LOOKUP_ELEM, + key, NULL, 0, NULL, value); +} + +static int virtnet_bpf_map_get_first_key(struct bpf_offloaded_map *offmap, + void *next_key) +{ + return virtnet_bpf_ctrl_map(offmap, + VIRTIO_NET_BPF_CMD_GET_FIRST, + NULL, NULL, 0, next_key, NULL); +} + +static int virtnet_bpf_map_get_next_key(struct bpf_offloaded_map *offmap, + void *key, void *next_key) +{ + if (!key) + return virtnet_bpf_map_get_first_key(offmap, next_key); + + return virtnet_bpf_ctrl_map(offmap, + VIRTIO_NET_BPF_CMD_GET_NEXT, + key, NULL, 0, next_key, NULL); +} + +static const struct bpf_map_dev_ops virtnet_bpf_map_ops = { + .map_get_next_key = virtnet_bpf_map_get_next_key, + .map_lookup_elem = virtnet_bpf_map_lookup_entry, + .map_update_elem = virtnet_bpf_map_update_entry, + .map_delete_elem = virtnet_bpf_map_delete_elem, +}; + +static int virtnet_bpf_map_alloc(struct virtnet_info *vi, + struct bpf_offloaded_map *offmap) +{ + struct virtnet_bpf_map *virtnet_map = NULL; + struct virtio_net_ctrl_ebpf_map *ctrl = NULL; + struct bpf_map *map = &offmap->map; + struct scatterlist sg; + int buf_len; + + if (map->map_type != BPF_MAP_TYPE_ARRAY && + map->map_type != BPF_MAP_TYPE_HASH) + goto err; + + virtnet_map = kzalloc(sizeof(*virtnet_map), GFP_KERNEL); + if (!virtnet_map) + goto err; + + /* allocate buffer size to fit + * - sizeof (struct virio_net_ctrl_map_buf) + * - key_size + * - max(key_size + value_size, key_size + key_size) + */ + buf_len = map->key_size; + buf_len += (map->key_size > map->value_size) ? + map->key_size : map->value_size; + ctrl = kzalloc(sizeof(*ctrl) + buf_len, GFP_KERNEL); + if (!ctrl) + goto err; + + ctrl->buf_len = cpu_to_virtio32(vi->vdev, buf_len); + ctrl->key_size = cpu_to_virtio32(vi->vdev, map->key_size); + ctrl->value_size = cpu_to_virtio32(vi->vdev, map->value_size); + ctrl->max_entries = cpu_to_virtio32(vi->vdev, map->max_entries); + ctrl->map_type = cpu_to_virtio32(vi->vdev, map->map_type); + ctrl->map_flags = 0; + ctrl->cmd = cpu_to_virtio32(vi->vdev, VIRTIO_NET_BPF_CMD_CREATE_MAP); + + sg_init_one(&sg, ctrl, sizeof(*ctrl) + ctrl->buf_len); + + if (!virtnet_send_command(vi, VIRTIO_NET_CTRL_EBPF, + VIRTIO_NET_CTRL_EBPF_MAP, + &sg)) { + dev_warn(&vi->vdev->dev, "Failed to create ebpf map\n"); + goto err; + } + + offmap->dev_ops = &virtnet_bpf_map_ops; + offmap->dev_priv = virtnet_map; + + virtnet_map->offmap = offmap; + virtnet_map->vi = vi; + virtnet_map->ctrl = ctrl; + + list_add_tail(&virtnet_map->list, &vi->map_list); + + return 0; +err: + kfree(virtnet_map); + kfree(ctrl); + return -EFAULT; +} + +static int virtnet_bpf_map_free(struct virtnet_info *vi, + struct bpf_offloaded_map *offmap) +{ + struct bpf_map *map = &offmap->map; + struct virtnet_bpf_map *virtnet_map = virtnet_get_bpf_map(vi, map); + struct virtio_net_ctrl_ebpf_map *ctrl = virtnet_map->ctrl; + struct scatterlist sg; + + if (!virtnet_map) + return -EINVAL; + + ctrl->cmd = cpu_to_virtio32(vi->vdev, VIRTIO_NET_BPF_CMD_FREE_MAP); + + sg_init_one(&sg, ctrl, sizeof(*ctrl) + ctrl->buf_len); + + if (!virtnet_send_command(vi, VIRTIO_NET_CTRL_EBPF, + VIRTIO_NET_CTRL_EBPF_MAP, + &sg)) { + dev_warn(&vi->vdev->dev, "Failed to free ebpf map\n"); + return -EFAULT; + } + + list_del(&virtnet_map->list); + kfree(virtnet_map->ctrl); + kfree(virtnet_map); + return 0; +} + static int virtnet_bpf_verifier_setup(struct bpf_prog *prog) { struct virtnet_info *vi = netdev_priv(prog->aux->offload->netdev); size_t insn_len = prog->len * sizeof(struct bpf_insn); struct virtnet_bpf_bound_prog *state; + struct virtnet_bpf_map *virtnet_map; + struct bpf_map *map; + struct fd mapfd; + int i, err = 0; state = kzalloc(sizeof(*state) + insn_len, GFP_KERNEL); if (!state) @@ -2639,11 +2843,43 @@ static int virtnet_bpf_verifier_setup(struct bpf_prog *prog) state->prog = prog; state->len = prog->len; + for (i = 0; i < state->len; i++) { + struct bpf_insn *insn = &state->insnsi[i]; + + if (insn->code != (BPF_LD | BPF_IMM | BPF_DW)) + continue; + + mapfd = fdget(insn->imm); + map = __bpf_map_get(mapfd); + if (IS_ERR(map)) { + pr_warn("fd %d is not pointing to valid bpf_map\n", + insn->imm); + err = -EINVAL; + goto err_replace; + } + + virtnet_map = virtnet_get_bpf_map(vi, map); + if (!virtnet_map) { + pr_warn("could not get a offloaded map fd %d\n", + insn->imm); + err = -EINVAL; + goto err_replace; + } + + /* Note: no need of virtio32_to_cpu */ + insn->imm = virtnet_map->ctrl->map_fd; + fdput(mapfd); + } + list_add_tail(&state->list, &vi->bpf_bound_progs); prog->aux->offload->dev_priv = state; return 0; + +err_replace: + kfree(state); + return err; } static int virtnet_bpf_verifier_prep(struct bpf_prog *prog) @@ -2676,6 +2912,10 @@ static int virtnet_xdp(struct net_device *dev, struct netdev_bpf *xdp) int err; switch (xdp->command) { + case BPF_OFFLOAD_MAP_ALLOC: + return virtnet_bpf_map_alloc(vi, xdp->offmap); + case BPF_OFFLOAD_MAP_FREE: + return virtnet_bpf_map_free(vi, xdp->offmap); case XDP_SETUP_PROG: return virtnet_xdp_set(dev, xdp); case XDP_QUERY_PROG: @@ -2747,6 +2987,7 @@ static int virtnet_bpf_init(struct virtnet_info *vi) goto err_netdev_register; INIT_LIST_HEAD(&vi->bpf_bound_progs); + INIT_LIST_HEAD(&vi->map_list); return 0; diff --git a/include/uapi/linux/virtio_net.h b/include/uapi/linux/virtio_net.h index 0ea2f7910a5a..1d330a0883ac 100644 --- a/include/uapi/linux/virtio_net.h +++ b/include/uapi/linux/virtio_net.h @@ -281,11 +281,34 @@ struct virtio_net_ctrl_ebpf_prog { __u8 insns[0]; }; +struct virtio_net_ctrl_ebpf_map { + __virtio32 buf_len; + __virtio32 cmd; + __virtio32 map_type; + __virtio32 key_size; + __virtio32 value_size; + __virtio32 max_entries; + __virtio32 map_flags; + __virtio32 map_fd; + __virtio64 flags; + __u8 buf[0]; +}; + #define VIRTIO_NET_CTRL_EBPF 6 #define VIRTIO_NET_CTRL_EBPF_PROG 1 + #define VIRTIO_NET_CTRL_EBPF_MAP 2 /* Commands for VIRTIO_NET_CTRL_EBPF_PROG */ #define VIRTIO_NET_BPF_CMD_SET_OFFLOAD 1 #define VIRTIO_NET_BPF_CMD_UNSET_OFFLOAD 2 +/* Commands for VIRTIO_NET_CTRL_EBPF_MAP */ +#define VIRTIO_NET_BPF_CMD_CREATE_MAP 1 +#define VIRTIO_NET_BPF_CMD_FREE_MAP 2 +#define VIRTIO_NET_BPF_CMD_UPDATE_ELEM 3 +#define VIRTIO_NET_BPF_CMD_LOOKUP_ELEM 4 +#define VIRTIO_NET_BPF_CMD_DELETE_ELEM 5 +#define VIRTIO_NET_BPF_CMD_GET_FIRST 6 +#define VIRTIO_NET_BPF_CMD_GET_NEXT 7 + #endif /* _UAPI_LINUX_VIRTIO_NET_H */ From patchwork Tue Nov 26 10:07:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Bhole X-Patchwork-Id: 11261827 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E3D466C1 for ; Tue, 26 Nov 2019 10:10:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C46A4208E3 for ; Tue, 26 Nov 2019 10:10:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hfC8Y0Gl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727949AbfKZKJz (ORCPT ); Tue, 26 Nov 2019 05:09:55 -0500 Received: from mail-pj1-f67.google.com ([209.85.216.67]:41234 "EHLO mail-pj1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727957AbfKZKJw (ORCPT ); Tue, 26 Nov 2019 05:09:52 -0500 Received: by mail-pj1-f67.google.com with SMTP id gc1so8062977pjb.8; Tue, 26 Nov 2019 02:09:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vjhBcNZlvW0lMvmwhzAhIEHJGNVVbcL/eLdp//zF+fg=; b=hfC8Y0GlPj/uXQPu3A2bgAU/XKe+wQFyKu/9eLipQh61lukaue4hfFzl2ix7VClJjZ eqSqR/bDgLg7pVcqACj11DMCuCv0gwXL7+xHhsNh5NwgCfARRwd9PlFnobWiTzP9n90N Cad2Cf8SKecEWov8eZLUH408k+NDKtsw7byKo/MzSxyDe560t1bn8KA8Nw44LvCzF9vm USGyVrg3TbYALODisuOSe3Ve8xKDQg/MXtM9pDiIBi8Zs54oRtnKLNOLXzCEtbzwdMLf iPJ/eSOXkY0IEK3bI3VUv0V9mmMlT1OQKz3zMG/tabHlx5zqAMJowhWzs1hYAg70KyV9 hpug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vjhBcNZlvW0lMvmwhzAhIEHJGNVVbcL/eLdp//zF+fg=; b=ANvlYWuBw57gqMne1aUVRRkDCxOm0XizLVOJAU2A/G+bqEN+lNP60fMKUFrZCUbCAZ /BgBYOHO6q59zyQA2Az0Q/ZN2MNWGi3/+afX7U+k1V5ZjJewFWFyxFiQ5uBwtBmI8MO1 +MYkdqZVrsIYml3vA8YRtv47XGKD4BldMIeTEVjUlKzcEB+oDa46ABBHF4McNI5NGlZL +4Wx0UMqqxhOOZJTl/5+SAqSIlnwHKdYsYh5skM74cTNFckgADFMX/OHf/49DyMuigC6 pk45g5fx4q0pnEN82Ri1C768JIKjRjmoaY0I1aIenmLEcUunKPsi8bSVbm863ZO+9Q9E W/7w== X-Gm-Message-State: APjAAAXKO9bccptlr4DhPUNkDyCciRwtvlb7eAG8c+wGLLC0+/ckIPC0 03e3U9o1KGtTX10oXQLUQkM= X-Google-Smtp-Source: APXvYqzqJGhF3OWpPSeJBTojUttgXbH4VXymhLmS8Ad3+dmUNyuhjPT2iweDPV8n33fEjuC8F6+rjQ== X-Received: by 2002:a17:90a:a612:: with SMTP id c18mr5725937pjq.49.1574762991777; Tue, 26 Nov 2019 02:09:51 -0800 (PST) Received: from localhost.localdomain ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id s24sm11848485pfh.108.2019.11.26.02.09.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 02:09:51 -0800 (PST) From: Prashant Bhole To: "David S . Miller" , "Michael S . Tsirkin" Cc: Prashant Bhole , Jason Wang , Alexei Starovoitov , Daniel Borkmann , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , netdev@vger.kernel.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [RFC net-next 18/18] virtio_net: restrict bpf helper calls from offloaded program Date: Tue, 26 Nov 2019 19:07:44 +0900 Message-Id: <20191126100744.5083-19-prashantbhole.linux@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191126100744.5083-1-prashantbhole.linux@gmail.com> References: <20191126100744.5083-1-prashantbhole.linux@gmail.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Since we are offloading this program to the host, some of the helper calls will not make sense. For example get_numa_node_id. Some helpers can not be used because we don't handle them yet. So let's allow a small set of helper calls for now. Signed-off-by: Prashant Bhole --- drivers/net/virtio_net.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 91a94b787c64..ab5be6b95bbd 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2549,6 +2549,25 @@ static struct virtnet_bpf_map *virtnet_get_bpf_map(struct virtnet_info *vi, return NULL; } +static int virtnet_bpf_check_helper_call(struct bpf_insn *insn) +{ + switch (insn->imm) { + case BPF_FUNC_map_lookup_elem: + case BPF_FUNC_map_update_elem: + case BPF_FUNC_map_delete_elem: + case BPF_FUNC_ktime_get_ns: + case BPF_FUNC_get_prandom_u32: + case BPF_FUNC_csum_update: + case BPF_FUNC_xdp_adjust_head: + case BPF_FUNC_xdp_adjust_meta: + case BPF_FUNC_xdp_adjust_tail: + case BPF_FUNC_strtol: + case BPF_FUNC_strtoul: + return 0; + } + return -EOPNOTSUPP; +} + static int virtnet_bpf_verify_insn(struct bpf_verifier_env *env, int insn_idx, int prev_insn) { @@ -2830,6 +2849,7 @@ static int virtnet_bpf_verifier_setup(struct bpf_prog *prog) struct virtnet_bpf_bound_prog *state; struct virtnet_bpf_map *virtnet_map; struct bpf_map *map; + u8 opcode, class; struct fd mapfd; int i, err = 0; @@ -2846,6 +2866,16 @@ static int virtnet_bpf_verifier_setup(struct bpf_prog *prog) for (i = 0; i < state->len; i++) { struct bpf_insn *insn = &state->insnsi[i]; + opcode = BPF_OP(insn->code); + class = BPF_CLASS(insn->code); + + if ((class == BPF_JMP || class == BPF_JMP32) && + opcode == BPF_CALL && insn->src_reg != BPF_PSEUDO_CALL) { + if (virtnet_bpf_check_helper_call(insn)) + return -EOPNOTSUPP; + continue; + } + if (insn->code != (BPF_LD | BPF_IMM | BPF_DW)) continue;