From patchwork Wed Jun 23 11:07:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339605 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6803C48BC2 for ; Wed, 23 Jun 2021 11:07:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9894961075 for ; Wed, 23 Jun 2021 11:07:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230028AbhFWLKF (ORCPT ); Wed, 23 Jun 2021 07:10:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:50978 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230102AbhFWLJv (ORCPT ); Wed, 23 Jun 2021 07:09:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446454; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ESlzoBrCtV1p5Bz5L61JWXgmG5edOzrWTpf13f6lwsQ=; b=T5FxS89tE53dO+G0s9WE0slZV2apFJJS0d3QVKU3m7EHk4eEiZzzT2Ywko6naiRMNyvlcj aqZ3x0s8Z5S/Uh2qI3/QuKll1hKQLVabupTCjvidqV4m7uDUOdwA/dxapGeELdoc1ncnnf sy8xZMgJvQJLjD1OZTh1VFPWChefsJs= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-122-7eYHg-vFP7ixUd6lUNXOKQ-1; Wed, 23 Jun 2021 07:07:30 -0400 X-MC-Unique: 7eYHg-vFP7ixUd6lUNXOKQ-1 Received: by mail-ej1-f70.google.com with SMTP id mh17-20020a170906eb91b0290477da799023so858272ejb.1 for ; Wed, 23 Jun 2021 04:07:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ESlzoBrCtV1p5Bz5L61JWXgmG5edOzrWTpf13f6lwsQ=; b=L7WBa2tqlgzvr/LTconvgZpHpVFze2wboxThX4Se8mmAq8v6o0vV284bKF+mZyAm8s c4O/ZdoyC99ggsp53Ms8w1yg9lB+77g19tkJ+B7XRCx1jz1F7a2x0gAOBRpodjj0jSBQ 2IXDCQJJ+NvhrnOog6VcOZ5sauTvy8HUZpbi7UeosKGOas8gHGhCfLaL9WIXXAFdgWRb qiDn3MBVu4EhrfIykbeymvTHgcAr0u2hLyhjbaESptIQwgHopeqaqSlNM2iwVqt7zSyN FcWu6Pyocr/Yfs2UPOUpzJw3D9KJpmXban1yhRaQAeuj8G8ddPuzwQlxoNqxZeVBM5+t 0NhQ== X-Gm-Message-State: AOAM5336G+LiU/0zkKFPFTkMHnwXHvoSSMDuXLIY9CxONdGSEVY1jBK9 DQu2LDF2SRuLs+ctfjJUVSxkMGWb4aOwI47YcaxkpWNwddk/6DEOPHCBSiKpyeUAvKLOnRawF/B hv6/TBzxlGGh+ X-Received: by 2002:a17:907:207c:: with SMTP id qp28mr9051769ejb.311.1624446449575; Wed, 23 Jun 2021 04:07:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwxUlmnTLWoQ9knMdWrbhkqZubUp9BhrNkGuD5ssEZcNVY6+xHag0CjUssqmpLH5HV7XgNf+Q== X-Received: by 2002:a17:907:207c:: with SMTP id qp28mr9051736ejb.311.1624446449212; Wed, 23 Jun 2021 04:07:29 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id g12sm512481edb.23.2021.06.23.04.07.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:28 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 030B31802B8; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v4 01/19] rcu: Create an unrcu_pointer() to remove __rcu from a pointer Date: Wed, 23 Jun 2021 13:07:09 +0200 Message-Id: <20210623110727.221922-2-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: "Paul E. McKenney" The xchg() and cmpxchg() functions are sometimes used to carry out RCU updates. Unfortunately, this can result in sparse warnings for both the old-value and new-value arguments, as well as for the return value. The arguments can be dealt with using RCU_INITIALIZER(): old_p = xchg(&p, RCU_INITIALIZER(new_p)); But a sparse warning still remains due to assigning the __rcu pointer returned from xchg to the (most likely) non-__rcu pointer old_p. This commit therefore provides an unrcu_pointer() macro that strips the __rcu. This macro can be used as follows: old_p = unrcu_pointer(xchg(&p, RCU_INITIALIZER(new_p))); Reported-by: Toke Høiland-Jørgensen Signed-off-by: Paul E. McKenney Signed-off-by: Toke Høiland-Jørgensen --- include/linux/rcupdate.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 9455476c5ba2..d7895b81264e 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -363,6 +363,20 @@ static inline void rcu_preempt_sleep_check(void) { } #define rcu_check_sparse(p, space) #endif /* #else #ifdef __CHECKER__ */ +/** + * unrcu_pointer - mark a pointer as not being RCU protected + * @p: pointer needing to lose its __rcu property + * + * Converts @p from an __rcu pointer to a __kernel pointer. + * This allows an __rcu pointer to be used with xchg() and friends. + */ +#define unrcu_pointer(p) \ +({ \ + typeof(*p) *_________p1 = (typeof(*p) *__force)(p); \ + rcu_check_sparse(p, __rcu); \ + ((typeof(*p) __force __kernel *)(_________p1)); \ +}) + #define __rcu_access_pointer(p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p) *__force)READ_ONCE(p); \ From patchwork Wed Jun 23 11:07:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339601 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65C0FC4743C for ; Wed, 23 Jun 2021 11:07:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3F91A6108E for ; Wed, 23 Jun 2021 11:07:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230103AbhFWLJv (ORCPT ); Wed, 23 Jun 2021 07:09:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39466 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230036AbhFWLJt (ORCPT ); Wed, 23 Jun 2021 07:09:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446452; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gCA//v5U2zdeKF30MPJkPlDU8nWyKsxnTIF8PhfpTLo=; b=dDlPByXZSbb6E6YCDuJB2bazeF0T8u0IOoh7/lXWfhX3c1dVRMFD5WD6u7TUibAJBEk15O kAmeOmDe080gZgaEM3dCXM5AHSofAPSUT1FPMCeWV99+jGcb2XN3S5JuGvG6eVNAL6HWw+ WuO1ynNUW7RIXmMVUI3tXrFTn74Rwyw= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-122-qawLYOH3PaOOjuCBmRWg6g-1; Wed, 23 Jun 2021 07:07:30 -0400 X-MC-Unique: qawLYOH3PaOOjuCBmRWg6g-1 Received: by mail-ed1-f71.google.com with SMTP id j15-20020a05640211cfb0290394f9de5750so532811edw.16 for ; Wed, 23 Jun 2021 04:07:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gCA//v5U2zdeKF30MPJkPlDU8nWyKsxnTIF8PhfpTLo=; b=ov0u6pIjh54xkU1qaOv8c+2PRh5E+hFFzvNtqzAYzQGCps68TGcS3zeSO9+gz6izvC z0NEDa2bW7wkU6Zyb41XVksG6iziQhAcTfNIgXQ4U8i23A3qKgXYPeKo5krHAyM5lEdq IGnq1dlJzVYbLB67U9x4h+rZMCxW25v+CvCUiV2P/G9DCmKqbw2wTlSthHS8WDf29mBB JaTIYAOxF/U4mDcet7giFkMrhcHCz2Fpg5wghJnUz7lXLxdAG3l+juW17We1XqQr2Jjw pVaa4Gaxkz78MtD6UiDuZDRmntzVSUFMGmZFVoj0Pgd/jMLGDIRAEzNt/xx4SZRUgBJV orxQ== X-Gm-Message-State: AOAM531XqAotNWHewp3KJ3VjBD8BBlKNsqHJzfht0XX40A0fx5PobAFK A3es3kICZtvHO+VBconBdZTpsadBUoluxklz1v4Nk8lNUIp5pEOqoQzjnyTy3qWyZ2NU7Raz6BE 6u+uMLlEH93c8 X-Received: by 2002:a05:6402:100e:: with SMTP id c14mr11312164edu.51.1624446449338; Wed, 23 Jun 2021 04:07:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxK5QVgpNKfKSNK2v/czWwASjaveoabV2SEAVqeVJeP9mCNaG4Ztjz6zIgdwue4LxXhpgF06A== X-Received: by 2002:a05:6402:100e:: with SMTP id c14mr11312118edu.51.1624446449004; Wed, 23 Jun 2021 04:07:29 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id cn10sm10177304edb.38.2021.06.23.04.07.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:28 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 07804180732; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v4 02/19] doc: Clarify and expand RCU updaters and corresponding readers Date: Wed, 23 Jun 2021 13:07:10 +0200 Message-Id: <20210623110727.221922-3-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: "Paul E. McKenney" This commit clarifies which primitives readers can use given that the corresponding updaters have made a specific choice. This commit also adds this information for the various RCU Tasks flavors. While in the area, it removes a paragraph that no longer applies in any straightforward manner. Signed-off-by: Paul E. McKenney Signed-off-by: Toke Høiland-Jørgensen --- Documentation/RCU/checklist.rst | 48 ++++++++++++++++++--------------- 1 file changed, 27 insertions(+), 21 deletions(-) diff --git a/Documentation/RCU/checklist.rst b/Documentation/RCU/checklist.rst index 1030119294d0..07f6cb8f674d 100644 --- a/Documentation/RCU/checklist.rst +++ b/Documentation/RCU/checklist.rst @@ -211,27 +211,33 @@ over a rather long period of time, but improvements are always welcome! of the system, especially to real-time workloads running on the rest of the system. -7. As of v4.20, a given kernel implements only one RCU flavor, - which is RCU-sched for PREEMPTION=n and RCU-preempt for PREEMPTION=y. - If the updater uses call_rcu() or synchronize_rcu(), - then the corresponding readers may use rcu_read_lock() and - rcu_read_unlock(), rcu_read_lock_bh() and rcu_read_unlock_bh(), - or any pair of primitives that disables and re-enables preemption, - for example, rcu_read_lock_sched() and rcu_read_unlock_sched(). - If the updater uses synchronize_srcu() or call_srcu(), - then the corresponding readers must use srcu_read_lock() and - srcu_read_unlock(), and with the same srcu_struct. The rules for - the expedited primitives are the same as for their non-expedited - counterparts. Mixing things up will result in confusion and - broken kernels, and has even resulted in an exploitable security - issue. - - One exception to this rule: rcu_read_lock() and rcu_read_unlock() - may be substituted for rcu_read_lock_bh() and rcu_read_unlock_bh() - in cases where local bottom halves are already known to be - disabled, for example, in irq or softirq context. Commenting - such cases is a must, of course! And the jury is still out on - whether the increased speed is worth it. +7. As of v4.20, a given kernel implements only one RCU flavor, which + is RCU-sched for PREEMPTION=n and RCU-preempt for PREEMPTION=y. + If the updater uses call_rcu() or synchronize_rcu(), then + the corresponding readers may use: (1) rcu_read_lock() and + rcu_read_unlock(), (2) any pair of primitives that disables + and re-enables softirq, for example, rcu_read_lock_bh() and + rcu_read_unlock_bh(), or (3) any pair of primitives that disables + and re-enables preemption, for example, rcu_read_lock_sched() and + rcu_read_unlock_sched(). If the updater uses synchronize_srcu() + or call_srcu(), then the corresponding readers must use + srcu_read_lock() and srcu_read_unlock(), and with the same + srcu_struct. The rules for the expedited RCU grace-period-wait + primitives are the same as for their non-expedited counterparts. + + If the updater uses call_rcu_tasks() or synchronize_rcu_tasks(), + then the readers must refrain from executing voluntary + context switches, that is, from blocking. If the updater uses + call_rcu_tasks_trace() or synchronize_rcu_tasks_trace(), then + the corresponding readers must use rcu_read_lock_trace() and + rcu_read_unlock_trace(). If an updater uses call_rcu_tasks_rude() + or synchronize_rcu_tasks_rude(), then the corresponding readers + must use anything that disables interrupts. + + Mixing things up will result in confusion and broken kernels, and + has even resulted in an exploitable security issue. Therefore, + when using non-obvious pairs of primitives, commenting is of + course a must. 8. Although synchronize_rcu() is slower than is call_rcu(), it usually results in simpler code. So, unless update performance is From patchwork Wed Jun 23 11:07:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339615 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63CA3C48BE5 for ; Wed, 23 Jun 2021 11:08:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4D0D161003 for ; Wed, 23 Jun 2021 11:08:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230393AbhFWLKS (ORCPT ); Wed, 23 Jun 2021 07:10:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48499 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230212AbhFWLJ4 (ORCPT ); Wed, 23 Jun 2021 07:09:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446459; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LUCuBp+kMAaiYhpJP43bLBOzgEXgjURSXqNOTfyeabU=; b=VWQt80RtVNXwPvukaon6UZmYZxuttdvGqFJ6Zu83iTc/IBWMTory0foFeSTa9pFl55KvOz CQ15IY4CqwTkuv3R/OaQ8ZzXL9wYdlno/B42hLWe1IbH730ZK+EEX60batRMIb3Ppx6DJB hOgzQrfPzdJfJWhNtFtLakpguSHKfuQ= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-74-NkKyRla2O7-tdDjjLQG48w-1; Wed, 23 Jun 2021 07:07:37 -0400 X-MC-Unique: NkKyRla2O7-tdDjjLQG48w-1 Received: by mail-ed1-f70.google.com with SMTP id j19-20020aa7c4130000b029039497d5cdbeso1088774edq.15 for ; Wed, 23 Jun 2021 04:07:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LUCuBp+kMAaiYhpJP43bLBOzgEXgjURSXqNOTfyeabU=; b=Kyg9Wq2WGW15imIDHb5sTvtAi0aZ7I9t//9/A0hm9pb00Bf0B/KSLLLU4CK8vngB2H Ione/5uBr5Vn0iAgEnke8dGB45it85Q2+sdNsFTlfyxKIawLY5FVHX+xyacC5DcLAEbe /gZ7X771/ksGPob6E7jr1JEt5MByFVBQzJAkE7Qku2al5r0EXx5YIkKRGNbjXylnjJM5 rpt2GFYiwKdj0EfgB34YQ+5P67CCz11+6l6TXSptcR0McTiGv+r0D4CSG/aIYlnFZ1kW GPluX33yI8DYUtL4QcCirvSrG96EY/0zrjpdOsMCn9cF+D8Md4lz6bU3c3YzUIL/GAPs dC/g== X-Gm-Message-State: AOAM532figUbi3qC1h/iL+fhfNyDJuIgT+bmSzcZz4gU43OpDcLK64bv cOGy1sZuyiRMRz1QejPyU2izLoK3EboPDFD8XXAl+L2zznH0XewLIQMHkZSCzgIUhaQWcBrwSkG Oc6/P/7TyelrI X-Received: by 2002:a05:6402:34d1:: with SMTP id w17mr9016405edc.167.1624446456599; Wed, 23 Jun 2021 04:07:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzryCgWTnTknnV/8acg2toeAqckr6zU4GlElUrLt6wlnzWfynQHUxYeQyBdepxK81ABJex3bg== X-Received: by 2002:a05:6402:34d1:: with SMTP id w17mr9016360edc.167.1624446456219; Wed, 23 Jun 2021 04:07:36 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id w2sm7152669ejn.118.2021.06.23.04.07.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:30 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 0CE81180733; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v4 03/19] doc: Give XDP as example of non-obvious RCU reader/updater pairing Date: Wed, 23 Jun 2021 13:07:11 +0200 Message-Id: <20210623110727.221922-4-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit gives an example of non-obvious RCU reader/updater pairing in the guise of the XDP feature in networking, which calls BPF programs from network-driver NAPI (softirq) context. Signed-off-by: Paul E. McKenney Signed-off-by: Toke Høiland-Jørgensen --- Documentation/RCU/checklist.rst | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/Documentation/RCU/checklist.rst b/Documentation/RCU/checklist.rst index 07f6cb8f674d..01cc21f17f7b 100644 --- a/Documentation/RCU/checklist.rst +++ b/Documentation/RCU/checklist.rst @@ -236,8 +236,15 @@ over a rather long period of time, but improvements are always welcome! Mixing things up will result in confusion and broken kernels, and has even resulted in an exploitable security issue. Therefore, - when using non-obvious pairs of primitives, commenting is of - course a must. + when using non-obvious pairs of primitives, commenting is + of course a must. One example of non-obvious pairing is + the XDP feature in networking, which calls BPF programs from + network-driver NAPI (softirq) context. BPF relies heavily on RCU + protection for its data structures, but because the BPF program + invocation happens entirely within a single local_bh_disable() + section in a NAPI poll cycle, this usage is safe. The reason + that this usage is safe is that readers can use anything that + disables BH when updaters use call_rcu() or synchronize_rcu(). 8. Although synchronize_rcu() is slower than is call_rcu(), it usually results in simpler code. So, unless update performance is From patchwork Wed Jun 23 11:07:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339611 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B8B4C49EA4 for ; Wed, 23 Jun 2021 11:07:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0110B61026 for ; Wed, 23 Jun 2021 11:07:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230048AbhFWLKN (ORCPT ); Wed, 23 Jun 2021 07:10:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43402 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230182AbhFWLJy (ORCPT ); Wed, 23 Jun 2021 07:09:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446457; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x+C1vvBih2DDvMDumUaYzwO9FAV6D2CItQN3YHn0G7w=; b=IufQU3pMOKy3QQK0oaA27O2JHZUOb5naU1M7MNd8PFUCG8LwPL1auuwPvcCfMq2XTnD34W GvKQUIJAIo4aTrV/UVXIPSe8q5Mix9F58FXWWdPegxkoT5n7Adix0zepN9Rjfp7jzcctnA sbTrTS10YqnYBjNcrH+KCXKRDqRpVsE= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-476-BKzeQ0jWNc6JGhZKcl2wSg-1; Wed, 23 Jun 2021 07:07:35 -0400 X-MC-Unique: BKzeQ0jWNc6JGhZKcl2wSg-1 Received: by mail-ed1-f70.google.com with SMTP id w1-20020a0564022681b0290394cedd8a6aso1089776edd.14 for ; Wed, 23 Jun 2021 04:07:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=x+C1vvBih2DDvMDumUaYzwO9FAV6D2CItQN3YHn0G7w=; b=d6zRUvXevdpPa803dJjKaXGq/A6myPT9zDziBnMox2omconpTYuQSwVSJTth7bOb7g w/UwaOAU9N04CkB2o3lxhBpfvdUYMHDxrdUbb5Zbrt0VN+ZzVTiHUeoSpBpL6LDYaYOK CAxOLVEqY6WhOeWjO+cDgi1iBbUvg3/6FAGPwd74XdWtKEOtDL7+9ly4iUumDcfpnrkA I8drYBJMTBbkIF1jEie50k2w1j/gd8udNCLSOJQV8aAYMnDVTUDmVApY/T9GvQSeAIMK tRONCTkMdPmZ9oKMNReq7zfyF/Dj+7uEoW/WkO3k7TIk2yuy3tXgY7mSA1Jubfx9DIUC hFGw== X-Gm-Message-State: AOAM5337WrEsxGrHsQmXbptECwPlmjFVrugm+rLewnA72FbxCemCC99R 3+LLnGW5LILAz9tQ9FCXTs2+Lsqc0CJyq9PTbO2c35zeTN3c9Wda8BDgpBRLb5Mtrgkd7UcvCcn C6mNdRLvBX62D X-Received: by 2002:a05:6402:3492:: with SMTP id v18mr11454462edc.130.1624446454587; Wed, 23 Jun 2021 04:07:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxmiCc4ZMRsI3PC3oqRFSBn7eYUhULqjssJglZJh0kuoMh4ieBNvP8AQu5xofhuWL8s70rGUA== X-Received: by 2002:a05:6402:3492:: with SMTP id v18mr11454434edc.130.1624446454395; Wed, 23 Jun 2021 04:07:34 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id b24sm1663362ejl.61.2021.06.23.04.07.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:29 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 1296E180734; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v4 04/19] bpf: allow RCU-protected lookups to happen from bh context Date: Wed, 23 Jun 2021 13:07:12 +0200 Message-Id: <20210623110727.221922-5-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net XDP programs are called from a NAPI poll context, which means the RCU reference liveness is ensured by local_bh_disable(). Add rcu_read_lock_bh_held() as a condition to the RCU checks for map lookups so lockdep understands that the dereferences are safe from inside *either* an rcu_read_lock() section *or* a local_bh_disable() section. While both bh_disabled and rcu_read_lock() provide RCU protection, they are semantically distinct, so we need both conditions to prevent lockdep complaints. This change is done in preparation for removing the redundant rcu_read_lock()s from drivers. Acked-by: Martin KaFai Lau Signed-off-by: Toke Høiland-Jørgensen --- kernel/bpf/hashtab.c | 21 ++++++++++++++------- kernel/bpf/helpers.c | 6 +++--- kernel/bpf/lpm_trie.c | 6 ++++-- 3 files changed, 21 insertions(+), 12 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 6f6681b07364..72c58cc516a3 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -596,7 +596,8 @@ static void *__htab_map_lookup_elem(struct bpf_map *map, void *key) struct htab_elem *l; u32 hash, key_size; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -989,7 +990,8 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1082,7 +1084,8 @@ static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1148,7 +1151,8 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1202,7 +1206,8 @@ static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1276,7 +1281,8 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key) u32 hash, key_size; int ret; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1311,7 +1317,8 @@ static int htab_lru_map_delete_elem(struct bpf_map *map, void *key) u32 hash, key_size; int ret; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 544773970dbc..e880f6bb6f28 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -28,7 +28,7 @@ */ BPF_CALL_2(bpf_map_lookup_elem, struct bpf_map *, map, void *, key) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return (unsigned long) map->ops->map_lookup_elem(map, key); } @@ -44,7 +44,7 @@ const struct bpf_func_proto bpf_map_lookup_elem_proto = { BPF_CALL_4(bpf_map_update_elem, struct bpf_map *, map, void *, key, void *, value, u64, flags) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return map->ops->map_update_elem(map, key, value, flags); } @@ -61,7 +61,7 @@ const struct bpf_func_proto bpf_map_update_elem_proto = { BPF_CALL_2(bpf_map_delete_elem, struct bpf_map *, map, void *, key) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return map->ops->map_delete_elem(map, key); } diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 1b7b8a6f34ee..423549d2c52e 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -232,7 +232,8 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) /* Start walking the trie from the root node ... */ - for (node = rcu_dereference(trie->root); node;) { + for (node = rcu_dereference_check(trie->root, rcu_read_lock_bh_held()); + node;) { unsigned int next_bit; size_t matchlen; @@ -264,7 +265,8 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) * traverse down. */ next_bit = extract_bit(key->data, node->prefixlen); - node = rcu_dereference(node->child[next_bit]); + node = rcu_dereference_check(node->child[next_bit], + rcu_read_lock_bh_held()); } if (!found) From patchwork Wed Jun 23 11:07:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339609 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1055DC4743C for ; Wed, 23 Jun 2021 11:07:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E77A061075 for ; Wed, 23 Jun 2021 11:07:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230102AbhFWLKK (ORCPT ); Wed, 23 Jun 2021 07:10:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55839 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230160AbhFWLJx (ORCPT ); Wed, 23 Jun 2021 07:09:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1tU7eRcW+hhp6Kr6NSPKlDkbh0X4RswUa63evZaKri4=; b=PgrP0Kr6pdEkKN+vKFdmSBitvOCKRr25iY/P7koF5aznPAAAZykRnSo9ByUZzuLLO0UHTB q6ZYDs1AQBldRigKPPvFs7ziVHXphj8fvCtfV0fnVdkhxdvVQvlkEbfASpgLybMz9b1crs klnTiuFaSwfz/ooeNC+TPxqrfmmy+Jo= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-440-06gv1EykOeOqWsRChJdySQ-1; Wed, 23 Jun 2021 07:07:34 -0400 X-MC-Unique: 06gv1EykOeOqWsRChJdySQ-1 Received: by mail-ed1-f71.google.com with SMTP id z5-20020a05640235c5b0290393974bcf7eso1116690edc.2 for ; Wed, 23 Jun 2021 04:07:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1tU7eRcW+hhp6Kr6NSPKlDkbh0X4RswUa63evZaKri4=; b=nYkDfOhMi0NIvC4xgYR3Iv0LVWdU/zamPSl0DoycxRsDnOMGw4ANwXCpy1ymgx/BWC yn3VdFO3G33v+h3sLAmaKNABvzLJFgr+/TVAmF6//URAhXFJRFY+KHeEZLmmZ7EwzDVA /3EpK6rReo6OZcQwUGNmS2tl/nnC/IrrOBEykNMTPEwgn9a9DN1dlMOGQTMtZaA+9tN4 Cyistxg2kS6PFIarnokytIQHZjEWwzGlWXYa8a/TO/CMygdiEJjc8Nf5QmwtxAgHqvuZ 3VMRU9tnazUnZiErH+8Y2xDnn8Di7xwGZEPFY22uWRAuFdKg+CYyiRcKzm4nYJNYj5kU oraA== X-Gm-Message-State: AOAM531VFg5+GqmWw9YZn7QEdJFkxAeyVV1UmwxTQbJr2dSJbzPrNpRH 6tCeLJNefOB0zRe6Fm89UzAtor2F8gIefv3K7uAODz5mpJ6QXsj0RccMO3EV26BmDMD42yxlyBS JNzunAgxpvwXh X-Received: by 2002:a05:6402:11c9:: with SMTP id j9mr11603255edw.89.1624446451303; Wed, 23 Jun 2021 04:07:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxuLzp0SI45BEfVrLjET686TlW0wARCP2KBxyQoxRe0SVoAArfXJVuJJ1QzU7jH/novBLR43w== X-Received: by 2002:a05:6402:11c9:: with SMTP id j9mr11603203edw.89.1624446450833; Wed, 23 Jun 2021 04:07:30 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id n2sm2269415edq.2.2021.06.23.04.07.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:29 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 184AC180735; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v4 05/19] xdp: add proper __rcu annotations to redirect map entries Date: Wed, 23 Jun 2021 13:07:13 +0200 Message-Id: <20210623110727.221922-6-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net XDP_REDIRECT works by a three-step process: the bpf_redirect() and bpf_redirect_map() helpers will lookup the target of the redirect and store it (along with some other metadata) in a per-CPU struct bpf_redirect_info. Next, when the program returns the XDP_REDIRECT return code, the driver will call xdp_do_redirect() which will use the information thus stored to actually enqueue the frame into a bulk queue structure (that differs slightly by map type, but shares the same principle). Finally, before exiting its NAPI poll loop, the driver will call xdp_do_flush(), which will flush all the different bulk queues, thus completing the redirect. Pointers to the map entries will be kept around for this whole sequence of steps, protected by RCU. However, there is no top-level rcu_read_lock() in the core code; instead drivers add their own rcu_read_lock() around the XDP portions of the code, but somewhat inconsistently as Martin discovered[0]. However, things still work because everything happens inside a single NAPI poll sequence, which means it's between a pair of calls to local_bh_disable()/local_bh_enable(). So Paul suggested[1] that we could document this intention by using rcu_dereference_check() with rcu_read_lock_bh_held() as a second parameter, thus allowing sparse and lockdep to verify that everything is done correctly. This patch does just that: we add an __rcu annotation to the map entry pointers and remove the various comments explaining the NAPI poll assurance strewn through devmap.c in favour of a longer explanation in filter.c. The goal is to have one coherent documentation of the entire flow, and rely on the RCU annotations as a "standard" way of communicating the flow in the map code (which can additionally be understood by sparse and lockdep). The RCU annotation replacements result in a fairly straight-forward replacement where READ_ONCE() becomes rcu_dereference_check(), WRITE_ONCE() becomes rcu_assign_pointer() and xchg() and cmpxchg() gets wrapped in the proper constructs to cast the pointer back and forth between __rcu and __kernel address space (for the benefit of sparse). The one complication is that xskmap has a few constructions where double-pointers are passed back and forth; these simply all gain __rcu annotations, and only the final reference/dereference to the inner-most pointer gets changed. With this, everything can be run through sparse without eliciting complaints, and lockdep can verify correctness even without the use of rcu_read_lock() in the drivers. Subsequent patches will clean these up from the drivers. [0] https://lore.kernel.org/bpf/20210415173551.7ma4slcbqeyiba2r@kafai-mbp.dhcp.thefacebook.com/ [1] https://lore.kernel.org/bpf/20210419165837.GA975577@paulmck-ThinkPad-P17-Gen-1/ Signed-off-by: Toke Høiland-Jørgensen --- include/linux/filter.h | 10 ++++----- include/net/xdp_sock.h | 2 +- kernel/bpf/cpumap.c | 13 +++++++---- kernel/bpf/devmap.c | 49 ++++++++++++++++++------------------------ net/core/filter.c | 28 ++++++++++++++++++++++++ net/xdp/xsk.c | 4 ++-- net/xdp/xsk.h | 4 ++-- net/xdp/xskmap.c | 29 ++++++++++++++----------- 8 files changed, 84 insertions(+), 55 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index c5ad7df029ed..b01e266dad9e 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -762,12 +762,10 @@ DECLARE_BPF_DISPATCHER(xdp) static __always_inline u32 bpf_prog_run_xdp(const struct bpf_prog *prog, struct xdp_buff *xdp) -{ - /* Caller needs to hold rcu_read_lock() (!), otherwise program - * can be released while still running, or map elements could be - * freed early while still having concurrent users. XDP fastpath - * already takes rcu_read_lock() when fetching the program, so - * it's not necessary here anymore. + + /* Driver XDP hooks are invoked within a single NAPI poll cycle and thus + * under local_bh_disable(), which provides the needed RCU protection + * for accessing map entries. */ return __BPF_PROG_RUN(prog, xdp, BPF_DISPATCHER_FUNC(xdp)); } diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h index 9c0722c6d7ac..fff069d2ed1b 100644 --- a/include/net/xdp_sock.h +++ b/include/net/xdp_sock.h @@ -37,7 +37,7 @@ struct xdp_umem { struct xsk_map { struct bpf_map map; spinlock_t lock; /* Synchronize map updates */ - struct xdp_sock *xsk_map[]; + struct xdp_sock __rcu *xsk_map[]; }; struct xdp_sock { diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index a1a0c4e791c6..480e936c54d0 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -74,7 +74,7 @@ struct bpf_cpu_map_entry { struct bpf_cpu_map { struct bpf_map map; /* Below members specific for map type */ - struct bpf_cpu_map_entry **cpu_map; + struct bpf_cpu_map_entry __rcu **cpu_map; }; static DEFINE_PER_CPU(struct list_head, cpu_map_flush_list); @@ -469,7 +469,7 @@ static void __cpu_map_entry_replace(struct bpf_cpu_map *cmap, { struct bpf_cpu_map_entry *old_rcpu; - old_rcpu = xchg(&cmap->cpu_map[key_cpu], rcpu); + old_rcpu = unrcu_pointer(xchg(&cmap->cpu_map[key_cpu], RCU_INITIALIZER(rcpu))); if (old_rcpu) { call_rcu(&old_rcpu->rcu, __cpu_map_entry_free); INIT_WORK(&old_rcpu->kthread_stop_wq, cpu_map_kthread_stop); @@ -551,7 +551,7 @@ static void cpu_map_free(struct bpf_map *map) for (i = 0; i < cmap->map.max_entries; i++) { struct bpf_cpu_map_entry *rcpu; - rcpu = READ_ONCE(cmap->cpu_map[i]); + rcpu = rcu_dereference_raw(cmap->cpu_map[i]); if (!rcpu) continue; @@ -562,6 +562,10 @@ static void cpu_map_free(struct bpf_map *map) kfree(cmap); } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__cpu_map_lookup_elem(struct bpf_map *map, u32 key) { struct bpf_cpu_map *cmap = container_of(map, struct bpf_cpu_map, map); @@ -570,7 +574,8 @@ static void *__cpu_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - rcpu = READ_ONCE(cmap->cpu_map[key]); + rcpu = rcu_dereference_check(cmap->cpu_map[key], + rcu_read_lock_bh_held()); return rcpu; } diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 2a75e6c2d27d..2f6bd75cd682 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -73,7 +73,7 @@ struct bpf_dtab_netdev { struct bpf_dtab { struct bpf_map map; - struct bpf_dtab_netdev **netdev_map; /* DEVMAP type only */ + struct bpf_dtab_netdev __rcu **netdev_map; /* DEVMAP type only */ struct list_head list; /* these are only used for DEVMAP_HASH type maps */ @@ -226,7 +226,7 @@ static void dev_map_free(struct bpf_map *map) for (i = 0; i < dtab->map.max_entries; i++) { struct bpf_dtab_netdev *dev; - dev = dtab->netdev_map[i]; + dev = rcu_dereference_raw(dtab->netdev_map[i]); if (!dev) continue; @@ -259,6 +259,10 @@ static int dev_map_get_next_key(struct bpf_map *map, void *key, void *next_key) return 0; } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__dev_map_hash_lookup_elem(struct bpf_map *map, u32 key) { struct bpf_dtab *dtab = container_of(map, struct bpf_dtab, map); @@ -410,15 +414,9 @@ static void bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) trace_xdp_devmap_xmit(bq->dev_rx, dev, sent, cnt - sent, err); } -/* __dev_flush is called from xdp_do_flush() which _must_ be signaled - * from the driver before returning from its napi->poll() routine. The poll() - * routine is called either from busy_poll context or net_rx_action signaled - * from NET_RX_SOFTIRQ. Either way the poll routine must complete before the - * net device can be torn down. On devmap tear down we ensure the flush list - * is empty before completing to ensure all flush operations have completed. - * When drivers update the bpf program they may need to ensure any flush ops - * are also complete. Using synchronize_rcu or call_rcu will suffice for this - * because both wait for napi context to exit. +/* __dev_flush is called from xdp_do_flush() which _must_ be signalled from the + * driver before returning from its napi->poll() routine. See the comment above + * xdp_do_flush() in filter.c. */ void __dev_flush(void) { @@ -433,9 +431,9 @@ void __dev_flush(void) } } -/* rcu_read_lock (from syscall and BPF contexts) ensures that if a delete and/or - * update happens in parallel here a dev_put won't happen until after reading - * the ifindex. +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. */ static void *__dev_map_lookup_elem(struct bpf_map *map, u32 key) { @@ -445,12 +443,14 @@ static void *__dev_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - obj = READ_ONCE(dtab->netdev_map[key]); + obj = rcu_dereference_check(dtab->netdev_map[key], + rcu_read_lock_bh_held()); return obj; } -/* Runs under RCU-read-side, plus in softirq under NAPI protection. - * Thus, safe percpu variable access. +/* Runs in NAPI, i.e., softirq under local_bh_disable(). Thus, safe percpu + * variable access, and map elements stick around. See comment above + * xdp_do_flush() in filter.c. */ static void bq_enqueue(struct net_device *dev, struct xdp_frame *xdpf, struct net_device *dev_rx, struct bpf_prog *xdp_prog) @@ -735,14 +735,7 @@ static int dev_map_delete_elem(struct bpf_map *map, void *key) if (k >= map->max_entries) return -EINVAL; - /* Use call_rcu() here to ensure any rcu critical sections have - * completed as well as any flush operations because call_rcu - * will wait for preempt-disable region to complete, NAPI in this - * context. And additionally, the driver tear down ensures all - * soft irqs are complete before removing the net device in the - * case of dev_put equals zero. - */ - old_dev = xchg(&dtab->netdev_map[k], NULL); + old_dev = unrcu_pointer(xchg(&dtab->netdev_map[k], NULL)); if (old_dev) call_rcu(&old_dev->rcu, __dev_map_entry_free); return 0; @@ -851,7 +844,7 @@ static int __dev_map_update_elem(struct net *net, struct bpf_map *map, * Remembering the driver side flush operation will happen before the * net device is removed. */ - old_dev = xchg(&dtab->netdev_map[i], dev); + old_dev = unrcu_pointer(xchg(&dtab->netdev_map[i], RCU_INITIALIZER(dev))); if (old_dev) call_rcu(&old_dev->rcu, __dev_map_entry_free); @@ -1031,10 +1024,10 @@ static int dev_map_notification(struct notifier_block *notifier, for (i = 0; i < dtab->map.max_entries; i++) { struct bpf_dtab_netdev *dev, *odev; - dev = READ_ONCE(dtab->netdev_map[i]); + dev = rcu_dereference(dtab->netdev_map[i]); if (!dev || netdev != dev->dev) continue; - odev = cmpxchg(&dtab->netdev_map[i], dev, NULL); + odev = unrcu_pointer(cmpxchg(&dtab->netdev_map[i], RCU_INITIALIZER(dev), NULL)); if (dev == odev) call_rcu(&dev->rcu, __dev_map_entry_free); diff --git a/net/core/filter.c b/net/core/filter.c index caa88955562e..0b7db5c70385 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -3922,6 +3922,34 @@ static const struct bpf_func_proto bpf_xdp_adjust_meta_proto = { .arg2_type = ARG_ANYTHING, }; +/* XDP_REDIRECT works by a three-step process, implemented in the functions + * below: + * + * 1. The bpf_redirect() and bpf_redirect_map() helpers will lookup the target + * of the redirect and store it (along with some other metadata) in a per-CPU + * struct bpf_redirect_info. + * + * 2. When the program returns the XDP_REDIRECT return code, the driver will + * call xdp_do_redirect() which will use the information in struct + * bpf_redirect_info to actually enqueue the frame into a map type-specific + * bulk queue structure. + * + * 3. Before exiting its NAPI poll loop, the driver will call xdp_do_flush(), + * which will flush all the different bulk queues, thus completing the + * redirect. + * + * Pointers to the map entries will be kept around for this whole sequence of + * steps, protected by RCU. However, there is no top-level rcu_read_lock() in + * the core code; instead, the RCU protection relies on everything happening + * inside a single NAPI poll sequence, which means it's between a pair of calls + * to local_bh_disable()/local_bh_enable(). + * + * The map entries are marked as __rcu and the map code makes sure to + * dereference those pointers with rcu_dereference_check() in a way that works + * for both sections that to hold an rcu_read_lock() and sections that are + * called from NAPI without a separate rcu_read_lock(). The code below does not + * use RCU annotations, but relies on those in the map code. + */ void xdp_do_flush(void) { __dev_flush(); diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index cd62d4ba87a9..996da915f520 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -749,7 +749,7 @@ static void xsk_unbind_dev(struct xdp_sock *xs) } static struct xsk_map *xsk_get_map_list_entry(struct xdp_sock *xs, - struct xdp_sock ***map_entry) + struct xdp_sock __rcu ***map_entry) { struct xsk_map *map = NULL; struct xsk_map_node *node; @@ -785,7 +785,7 @@ static void xsk_delete_from_maps(struct xdp_sock *xs) * might be updates to the map between * xsk_get_map_list_entry() and xsk_map_try_sock_delete(). */ - struct xdp_sock **map_entry = NULL; + struct xdp_sock __rcu **map_entry = NULL; struct xsk_map *map; while ((map = xsk_get_map_list_entry(xs, &map_entry))) { diff --git a/net/xdp/xsk.h b/net/xdp/xsk.h index edcf249ad1f1..a4bc4749faac 100644 --- a/net/xdp/xsk.h +++ b/net/xdp/xsk.h @@ -31,7 +31,7 @@ struct xdp_mmap_offsets_v1 { struct xsk_map_node { struct list_head node; struct xsk_map *map; - struct xdp_sock **map_entry; + struct xdp_sock __rcu **map_entry; }; static inline struct xdp_sock *xdp_sk(struct sock *sk) @@ -40,7 +40,7 @@ static inline struct xdp_sock *xdp_sk(struct sock *sk) } void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs, - struct xdp_sock **map_entry); + struct xdp_sock __rcu **map_entry); void xsk_clear_pool_at_qid(struct net_device *dev, u16 queue_id); int xsk_reg_pool_at_qid(struct net_device *dev, struct xsk_buff_pool *pool, u16 queue_id); diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index 9df75ea4a567..2e48d0e094d9 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -12,7 +12,7 @@ #include "xsk.h" static struct xsk_map_node *xsk_map_node_alloc(struct xsk_map *map, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { struct xsk_map_node *node; @@ -42,7 +42,7 @@ static void xsk_map_sock_add(struct xdp_sock *xs, struct xsk_map_node *node) } static void xsk_map_sock_delete(struct xdp_sock *xs, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { struct xsk_map_node *n, *tmp; @@ -124,6 +124,10 @@ static int xsk_map_gen_lookup(struct bpf_map *map, struct bpf_insn *insn_buf) return insn - insn_buf; } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__xsk_map_lookup_elem(struct bpf_map *map, u32 key) { struct xsk_map *m = container_of(map, struct xsk_map, map); @@ -131,12 +135,11 @@ static void *__xsk_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - return READ_ONCE(m->xsk_map[key]); + return rcu_dereference_check(m->xsk_map[key], rcu_read_lock_bh_held()); } static void *xsk_map_lookup_elem(struct bpf_map *map, void *key) { - WARN_ON_ONCE(!rcu_read_lock_held()); return __xsk_map_lookup_elem(map, *(u32 *)key); } @@ -149,7 +152,8 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags) { struct xsk_map *m = container_of(map, struct xsk_map, map); - struct xdp_sock *xs, *old_xs, **map_entry; + struct xdp_sock __rcu **map_entry; + struct xdp_sock *xs, *old_xs; u32 i = *(u32 *)key, fd = *(u32 *)value; struct xsk_map_node *node; struct socket *sock; @@ -179,7 +183,7 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, } spin_lock_bh(&m->lock); - old_xs = READ_ONCE(*map_entry); + old_xs = rcu_dereference_protected(*map_entry, lockdep_is_held(&m->lock)); if (old_xs == xs) { err = 0; goto out; @@ -191,7 +195,7 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, goto out; } xsk_map_sock_add(xs, node); - WRITE_ONCE(*map_entry, xs); + rcu_assign_pointer(*map_entry, xs); if (old_xs) xsk_map_sock_delete(old_xs, map_entry); spin_unlock_bh(&m->lock); @@ -208,7 +212,8 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, static int xsk_map_delete_elem(struct bpf_map *map, void *key) { struct xsk_map *m = container_of(map, struct xsk_map, map); - struct xdp_sock *old_xs, **map_entry; + struct xdp_sock __rcu **map_entry; + struct xdp_sock *old_xs; int k = *(u32 *)key; if (k >= map->max_entries) @@ -216,7 +221,7 @@ static int xsk_map_delete_elem(struct bpf_map *map, void *key) spin_lock_bh(&m->lock); map_entry = &m->xsk_map[k]; - old_xs = xchg(map_entry, NULL); + old_xs = unrcu_pointer(xchg(map_entry, NULL)); if (old_xs) xsk_map_sock_delete(old_xs, map_entry); spin_unlock_bh(&m->lock); @@ -231,11 +236,11 @@ static int xsk_map_redirect(struct bpf_map *map, u32 ifindex, u64 flags) } void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { spin_lock_bh(&map->lock); - if (READ_ONCE(*map_entry) == xs) { - WRITE_ONCE(*map_entry, NULL); + if (rcu_access_pointer(*map_entry) == xs) { + rcu_assign_pointer(*map_entry, NULL); xsk_map_sock_delete(xs, map_entry); } spin_unlock_bh(&map->lock); From patchwork Wed Jun 23 11:07:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339607 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EFD5C48BC2 for ; Wed, 23 Jun 2021 11:07:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7B0846108E for ; Wed, 23 Jun 2021 11:07:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230160AbhFWLKL (ORCPT ); Wed, 23 Jun 2021 07:10:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:51102 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230180AbhFWLJy (ORCPT ); Wed, 23 Jun 2021 07:09:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446456; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w/RD8INVqrZhxhRYQu3p0iItEELqlKBFxEmUhyTn65o=; b=DFYu4DMGN3RwHXbmP+9cbPQYkVulaUttcBP5F6xoMiL3WYv7xAUugigQMpBQKAy8KX2hEC llUb8OpUfp507cq1L+O4LMaYhS8VHNHtDc/Fva2hAZUrdRx8rdJj9s5Kxtap1PR6ozFZzV aDDmugyzuvxZfLWpYcWCjO7Vbj2b89A= Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-189-2SNfBwGuM8-bAruu28jlMg-1; Wed, 23 Jun 2021 07:07:35 -0400 X-MC-Unique: 2SNfBwGuM8-bAruu28jlMg-1 Received: by mail-ej1-f69.google.com with SMTP id p5-20020a17090653c5b02903db1cfa514dso839514ejo.13 for ; Wed, 23 Jun 2021 04:07:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w/RD8INVqrZhxhRYQu3p0iItEELqlKBFxEmUhyTn65o=; b=mv2Pya3FXQXQCy+sLuuDcmpM4xMcYnd3Tp+V2Bd8A77u/EldCei5HWv5LlbJIf0hsk a7jNANNVRnvVfI8pp/FrUzI4az+nEbuJDGL826fO2lpdwNUwvAinLcoIj8zOqBQe3Rv1 RZCpEQqWDhlP4pXUb307EWF2UUm8RLAeXNW92YXlCloRoWAaLhYnGJNComJSxYE63sUv gQbTg0myzVBo77dx4mKTZsZ1BEG0srcM2i5F9s7+dZ1/zhLgwP38PkavmMUWAMn64qn3 MnuCyy8IipPqx4tlDWAB/l01jgH5nIUS5zNGrTy5ZExnkRXxobSPOBUg3RP8FSES0rW2 x6hQ== X-Gm-Message-State: AOAM532dfjipiMefBgQWxIKe0zvF11KjmIKiJSwjwzD0qVsPE7/uPvDx X/AH/UPZwsvF9QgknF5ILQ6BhX7+HD5SvwNUTkeCodX2Q+epwyDI2sCa4Z3enB/kOHwzlPrL7h0 9cD0bUEj51Jk6 X-Received: by 2002:a05:6402:18f6:: with SMTP id x54mr11865257edy.53.1624446454210; Wed, 23 Jun 2021 04:07:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzkLnLptVgUW3clOL3nBx1FYnj3knCIVTef6R4LuM+oJs85AS2SKwdWmRQkxek9cThzk7dCeg== X-Received: by 2002:a05:6402:18f6:: with SMTP id x54mr11865234edy.53.1624446454079; Wed, 23 Jun 2021 04:07:34 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id z20sm7246260ejd.18.2021.06.23.04.07.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:29 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 1DB33180736; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v4 06/19] sched: remove unneeded rcu_read_lock() around BPF program invocation Date: Wed, 23 Jun 2021 13:07:14 +0200 Message-Id: <20210623110727.221922-7-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The rcu_read_lock() call in cls_bpf and act_bpf are redundant: on the TX side, there's already a call to rcu_read_lock_bh() in __dev_queue_xmit(), and on RX there's a covering rcu_read_lock() in netif_receive_skb{,_list}_internal(). With the previous patches we also amended the lockdep checks in the map code to not require any particular RCU flavour, so we can just get rid of the rcu_read_lock()s. Signed-off-by: Toke Høiland-Jørgensen --- net/sched/act_bpf.c | 2 -- net/sched/cls_bpf.c | 3 --- 2 files changed, 5 deletions(-) diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c index e48e980c3b93..e409a0005717 100644 --- a/net/sched/act_bpf.c +++ b/net/sched/act_bpf.c @@ -43,7 +43,6 @@ static int tcf_bpf_act(struct sk_buff *skb, const struct tc_action *act, tcf_lastuse_update(&prog->tcf_tm); bstats_cpu_update(this_cpu_ptr(prog->common.cpu_bstats), skb); - rcu_read_lock(); filter = rcu_dereference(prog->filter); if (at_ingress) { __skb_push(skb, skb->mac_len); @@ -56,7 +55,6 @@ static int tcf_bpf_act(struct sk_buff *skb, const struct tc_action *act, } if (skb_sk_is_prefetched(skb) && filter_res != TC_ACT_OK) skb_orphan(skb); - rcu_read_unlock(); /* A BPF program may overwrite the default action opcode. * Similarly as in cls_bpf, if filter_res == -1 we use the diff --git a/net/sched/cls_bpf.c b/net/sched/cls_bpf.c index 6e3e63db0e01..fa739efa59f4 100644 --- a/net/sched/cls_bpf.c +++ b/net/sched/cls_bpf.c @@ -85,8 +85,6 @@ static int cls_bpf_classify(struct sk_buff *skb, const struct tcf_proto *tp, struct cls_bpf_prog *prog; int ret = -1; - /* Needed here for accessing maps. */ - rcu_read_lock(); list_for_each_entry_rcu(prog, &head->plist, link) { int filter_res; @@ -131,7 +129,6 @@ static int cls_bpf_classify(struct sk_buff *skb, const struct tcf_proto *tp, break; } - rcu_read_unlock(); return ret; } From patchwork Wed Jun 23 11:07:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339613 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE7CAC4743C for ; Wed, 23 Jun 2021 11:08:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 95C1E61075 for ; Wed, 23 Jun 2021 11:08:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230351AbhFWLKO (ORCPT ); Wed, 23 Jun 2021 07:10:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55524 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230202AbhFWLJz (ORCPT ); Wed, 23 Jun 2021 07:09:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446457; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r8Kxje9mo0eA1OfdUdQS/iz/Q4P9zCSTZ9JCaTqfbwY=; b=LJ/LrixdY0DomJ1qdS+pxgzhaqOT9CEW3QPgifRTaeoZVKaIBnhZFij+ah/r7f/jxTFVA+ kLUCK/5s/5QAAbBDh0y4J63alDqtBb9vHu880OIZNaygFRrv9yTnccS5c2vT3V2nz1pbAi cVfcT6P/Sikq/Q6I599bWqwd9W4zVIU= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-598-04jS0yrxOUiTOFq0HlOBbg-1; Wed, 23 Jun 2021 07:07:36 -0400 X-MC-Unique: 04jS0yrxOUiTOFq0HlOBbg-1 Received: by mail-ed1-f70.google.com with SMTP id v8-20020a0564023488b0290393873961f6so1091162edc.17 for ; Wed, 23 Jun 2021 04:07:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r8Kxje9mo0eA1OfdUdQS/iz/Q4P9zCSTZ9JCaTqfbwY=; b=RCOM7Ksu3RgjVXrmElNFJb0FhqpeOtL5D8nx3qqztJ0r4SUvKVbuT96XVsczsjBRIY 84JBDunMU/lz/PogO8bz4pN7PYg+VN3SQV5t0GklFdEjUDAQ/IJ4xvYUGl/QbsW7VUQ3 mOu6GZw94SczpF3d1fNWI0ZGe0+MI2r2ONyTlehTlqbYQCbWsZBHdBwOwy/0gscrj1Yi TGdDZDCQBuUimtbQz3vsFed+qkbE7ETtMdQmd65EIeT/8cSayozFMJkDdIwUvcx8K2Za Es7q70uIonSVzMOGModchxB3o/u86gu0j/sUmTWNJcbb8lxGixJQQSemVlnbvp+D27Z0 r4XQ== X-Gm-Message-State: AOAM5330xDUZapjz54O5p1oAsYeAp+hzQN8WSpRd+EzRbScUYuPV5thd acYjPUXDNEgFB569xj+eHr6u2RbGOg9OkefZJi83MRgUHV5cSH6Y2m4iJD0nu07g2dcvp6GiSUJ t0TJKq7x1U+4g X-Received: by 2002:a17:907:2622:: with SMTP id aq2mr9352238ejc.48.1624446455072; Wed, 23 Jun 2021 04:07:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx9y32Eh+jTvhrwfrV+XjryvNhw+II0gK5l99fd1c7Gry3y4vk/TmSfoejTBO7gW/Bf7bP9qw== X-Received: by 2002:a17:907:2622:: with SMTP id aq2mr9352206ejc.48.1624446454670; Wed, 23 Jun 2021 04:07:34 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id c15sm10860553edu.19.2021.06.23.04.07.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:29 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 266CE180737; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Guy Tzalik , Saeed Bishara Subject: [PATCH bpf-next v4 07/19] ena: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:15 +0200 Message-Id: <20210623110727.221922-8-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The ena driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Guy Tzalik Cc: Saeed Bishara Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/amazon/ena/ena_netdev.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index 881f88754bf6..a4378b14af4c 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -385,7 +385,6 @@ static int ena_xdp_execute(struct ena_ring *rx_ring, struct xdp_buff *xdp) u64 *xdp_stat; int qid; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_bpf_prog); if (!xdp_prog) @@ -443,8 +442,6 @@ static int ena_xdp_execute(struct ena_ring *rx_ring, struct xdp_buff *xdp) ena_increase_stat(xdp_stat, 1, &rx_ring->syncp); out: - rcu_read_unlock(); - return verdict; } From patchwork Wed Jun 23 11:07:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339617 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E73E1C49EA4 for ; Wed, 23 Jun 2021 11:08:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D062D61164 for ; Wed, 23 Jun 2021 11:08:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230425AbhFWLKY (ORCPT ); Wed, 23 Jun 2021 07:10:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39917 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230222AbhFWLJ4 (ORCPT ); Wed, 23 Jun 2021 07:09:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446459; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6wgUfE3YuyYqn3UXIkFwKI8I46axO+CjBlTP2HaYNQM=; b=haKbaYHAcK/TVv1BrruVYLKOU3QcMtLT4GX/HISDznKw3zgOFs9rF0DUE2w2kX6vAylEsK Sk9Qdl9ENBc+ThSeDrYeB0u+aRAij7RcUtgr/HbyH2VAD13ksi81bVeHJSD/qKXftMZ2cW OxeFX+XV2PRLi8BS0l3k+zUL7elbalE= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-175-tW8ZibmTPkSnCWs8Rx2sdA-1; Wed, 23 Jun 2021 07:07:37 -0400 X-MC-Unique: tW8ZibmTPkSnCWs8Rx2sdA-1 Received: by mail-ed1-f72.google.com with SMTP id l9-20020a0564022549b0290394bafbfbcaso1114525edb.3 for ; Wed, 23 Jun 2021 04:07:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6wgUfE3YuyYqn3UXIkFwKI8I46axO+CjBlTP2HaYNQM=; b=rqLrg7qt542f5jxd9Vu+E6sfI6hffGrgVeLQu5hPl+xJAmuN0Uk4SlNf+by+/Zv0py GF5hx3hgA6RzTAWHFNFex/XGddl5fUV6mEUNcWBfUnb39Y7H/21wW9Hrmj4QaCKrE24b ZYn64p38oV+KoIK/F1Did6xPDNjFTPkKXU3mcBlXNj41dsWRjj4J68s8hnoWlo3U39T9 xXYWGdB8svl2mjsCuuxddBoSJiOwMvnqm0yjjgO9lJrY8njDmRZ0Vem2MlB0qqe+oc6V 8cUkFT38SDJX9rgWqIMVE77S0VWT/GXu9CFgMi0FMqP/rWBQN3q4829KDYODWfIALzJY i62g== X-Gm-Message-State: AOAM530Ush+D1lH1oRh8As5E4CSU8+tmLlPrlMaltS16k2tg9yHobAdv r41V51x6nyOEWhLNT1evjaRzoFjzSfy12Drw/z9l/9EzaloCGbB3r6N4IuowG9a0RB6dm4dMEQC RNqAVXcLFvja/ X-Received: by 2002:a17:906:244d:: with SMTP id a13mr9452854ejb.551.1624446455888; Wed, 23 Jun 2021 04:07:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyJuPR44sF0OT9/VupxhagYWjU/hFE+FqxoBPV+oB/0joRp0VUukfBXi80r+7q0PrJGZP+Gvw== X-Received: by 2002:a17:906:244d:: with SMTP id a13mr9452834ejb.551.1624446455697; Wed, 23 Jun 2021 04:07:35 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id n2sm13468522edi.32.2021.06.23.04.07.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:30 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 2D0B9180738; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Michael Chan Subject: [PATCH bpf-next v4 08/19] bnxt: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:16 +0200 Message-Id: <20210623110727.221922-9-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The bnxt driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Michael Chan Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c index ec9564e584e0..bee6e091a997 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c @@ -138,9 +138,7 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons, xdp_prepare_buff(&xdp, *data_ptr - offset, offset, *len, false); orig_data = xdp.data; - rcu_read_lock(); act = bpf_prog_run_xdp(xdp_prog, &xdp); - rcu_read_unlock(); tx_avail = bnxt_tx_avail(bp, txr); /* If the tx ring is not full, we must not update the rx producer yet From patchwork Wed Jun 23 11:07:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339621 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2357BC49EA7 for ; Wed, 23 Jun 2021 11:08:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0F58261185 for ; Wed, 23 Jun 2021 11:08:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230445AbhFWLK0 (ORCPT ); Wed, 23 Jun 2021 07:10:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:38189 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230236AbhFWLJ5 (ORCPT ); Wed, 23 Jun 2021 07:09:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2arA4F9H8piXSu2BaYZfrwo91oZXpALZLKMt5odcOCY=; b=RM0QewaZCDtUVr6mp4IlKWBtXVEtWd4JRiw+i2sKlCa8zcyAL0tSbtZxz8Cuxk+ZU4jJBC IyxY8+Gsx3FxzsJO92AHCoFuhSjaJgwgSy3lLxYaif7JYVdwqcOyKqy6Vkd4g++KfXq0pH vvUNnCWe4x1eC9XzOIL0lbIBJg0SeFg= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-132-BggyrzDGPGyuAr9EH4X8kQ-1; Wed, 23 Jun 2021 07:07:38 -0400 X-MC-Unique: BggyrzDGPGyuAr9EH4X8kQ-1 Received: by mail-ej1-f71.google.com with SMTP id g6-20020a1709064e46b02903f57f85ac45so835067ejw.15 for ; Wed, 23 Jun 2021 04:07:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2arA4F9H8piXSu2BaYZfrwo91oZXpALZLKMt5odcOCY=; b=XZ6iQhR6twaRusMzAVyt53xbvFMhznvLjjYXriUZit0kLIYBag74SrvHoODi5Ex0gj u/6MBLnzRqukLzIXK7hO4DQiN7WKbOeZwhEFg5eM24v6lRw5arraFmFET5HgtGXMho3m F2oJt2X4MEWNQMwlLQI0e9vYkXZqPSM0KadZ/ab2QqXhNP4ETo2+R/kmLs2Qk91AElXT HEBu1nBEg5GsCLWJfl6gqWCsYTzStXfMm/WFOMTG29dLJWi35C5vNIEWTPVkshtHLb/z LBJytklWsRLNwWya6EKtrMG07ao71e0JoA8ONKOmG7CXxzQGV+eK2GaEGHeMfAbqFL4d G/JA== X-Gm-Message-State: AOAM532bb30ca7ccaIEmOMdH575s1NYcz5juCISXQZ8SqWfj6nS2/edD q96oYbO1WqdA7jpPVMsLNDBWKKTxKXnZ3/VKgfrppCCtBFBymXNbVh/nnygEX8DXvfmK+5oNc8j 3AD/L/EPOunZb X-Received: by 2002:a17:906:190c:: with SMTP id a12mr9070040eje.491.1624446457193; Wed, 23 Jun 2021 04:07:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyfYOxCXGNl6WwCkAT61NO9Jw5DYitkekz8LmF8VlrNsuA0TOsWQBt9kYvTmODjjA976dI+eg== X-Received: by 2002:a17:906:190c:: with SMTP id a12mr9070029eje.491.1624446457054; Wed, 23 Jun 2021 04:07:37 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id jl21sm5514474ejc.42.2021.06.23.04.07.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:35 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 32D22180739; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Sunil Goutham , linux-arm-kernel@lists.infradead.org Subject: [PATCH bpf-next v4 09/19] thunderx: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:17 +0200 Message-Id: <20210623110727.221922-10-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The thunderx driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Sunil Goutham Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/cavium/thunder/nicvf_main.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c index c33b4e837515..e2b290135fd9 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c @@ -555,9 +555,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog, xdp_prepare_buff(&xdp, hard_start, data - hard_start, len, false); orig_data = xdp.data; - rcu_read_lock(); action = bpf_prog_run_xdp(prog, &xdp); - rcu_read_unlock(); len = xdp.data_end - xdp.data; /* Check if XDP program has changed headers */ From patchwork Wed Jun 23 11:07:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339619 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C184FC49EA5 for ; Wed, 23 Jun 2021 11:08:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B0267611AC for ; Wed, 23 Jun 2021 11:08:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230433AbhFWLKZ (ORCPT ); Wed, 23 Jun 2021 07:10:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48641 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230229AbhFWLJ5 (ORCPT ); Wed, 23 Jun 2021 07:09:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446459; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8n6XcLLXm/HC8/e7TDiaIKa4D1+079NdF0CtM9PlMq8=; b=L2zpNPwBkA2KUqOIW9hMKfMmx+LIkOG7Z0saVu27Z3tAUTk0xc3MMqMnZ6gf8Gb0WUbtO4 30rZLdTLae7odS+nUKq/m1jva/7prs+//Z5kmrPpO/Vm68dn+3c9QbVZ+Ih4K/kA5tiJ2T oxJmf1bDq9njJS74sramQ8De2fBGFLc= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-226-jyV1jZI1PLSvg-GPMjGrHg-1; Wed, 23 Jun 2021 07:07:38 -0400 X-MC-Unique: jyV1jZI1PLSvg-GPMjGrHg-1 Received: by mail-ed1-f71.google.com with SMTP id i19-20020a05640200d3b02903948b71f25cso1112926edu.4 for ; Wed, 23 Jun 2021 04:07:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8n6XcLLXm/HC8/e7TDiaIKa4D1+079NdF0CtM9PlMq8=; b=GrVGJ9Wb4QjWKNtS/MoEywrQ2KCDVQyqVOXWS2v7ixC76z0wBjmiyEb5t+GmZwiNY0 WoCRxu1GcGvWfUFBErlJlkhDFBd+s+4z46fk34S3DYCrhEHVmlGSp7TB+68yO9+g+JUG I9LL/4KRa4uPRMxk7G/bVJJu5MsK7KaH1J1mVEtmEqO6XYpu/6YwTlpZmVIEQXcBPQCm V719JXMMWaW7I/ck+wZqksVnINchEQXlgDmCRZ6XfTXP2+XMBGxHsc+4cNUK7xkFoLxb o3zCEJ5Gv5JDEjWBj41NeD5npNcBhQmEFpQ9VOXARhgDkRaf2Z6Dh33VCVsexcbSzAIt +SKA== X-Gm-Message-State: AOAM531vgUhxr7utON8gaSqUewQaU5bKOSvqvUd6vlzY16hDM9EmKmO1 KEYnEBx2bbLMYv+96rqmj62ugUlk0d0fWGuFcIvrX2dfZEYy6WyC7BuX75r8XXgoLJ/IcJhx07a E5p0iwnuimrk5 X-Received: by 2002:a05:6402:6d3:: with SMTP id n19mr11792337edy.248.1624446457003; Wed, 23 Jun 2021 04:07:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzkiD5H3LlCGb7IK6e0FT13/bnwywKl1w8bKqqDPQkSwyQvC9JZLaYqn51+rFHolYaIz4ezaw== X-Received: by 2002:a05:6402:6d3:: with SMTP id n19mr11792322edy.248.1624446456831; Wed, 23 Jun 2021 04:07:36 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id ck2sm4769159edb.72.2021.06.23.04.07.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:35 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 38CE818073A; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Madalin Bucur , Ioana Ciornei , Ioana Radulescu , Camelia Groza Subject: [PATCH bpf-next v4 10/19] freescale: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:18 +0200 Message-Id: <20210623110727.221922-11-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The dpaa and dpaa2 drivers have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Madalin Bucur Cc: Ioana Ciornei Cc: Ioana Radulescu Reviewed-by: Camelia Groza Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 8 +------- drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 3 --- 2 files changed, 1 insertion(+), 10 deletions(-) diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c index 177c020bf34a..e6826561cf11 100644 --- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c +++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c @@ -2558,13 +2558,9 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, u32 xdp_act; int err; - rcu_read_lock(); - xdp_prog = READ_ONCE(priv->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + if (!xdp_prog) return XDP_PASS; - } xdp_init_buff(&xdp, DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE, &dpaa_fq->xdp_rxq); @@ -2638,8 +2634,6 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, break; } - rcu_read_unlock(); - return xdp_act; } diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index 8433aa730c42..973352393bd4 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -352,8 +352,6 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, u32 xdp_act = XDP_PASS; int err, offset; - rcu_read_lock(); - xdp_prog = READ_ONCE(ch->xdp.prog); if (!xdp_prog) goto out; @@ -414,7 +412,6 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, ch->xdp.res |= xdp_act; out: - rcu_read_unlock(); return xdp_act; } From patchwork Wed Jun 23 11:07:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339639 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABA3AC49EA4 for ; Wed, 23 Jun 2021 11:14:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 948746102A for ; Wed, 23 Jun 2021 11:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230500AbhFWLQR (ORCPT ); Wed, 23 Jun 2021 07:16:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22080 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230205AbhFWLQG (ORCPT ); Wed, 23 Jun 2021 07:16:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qk5GBiM0tj1/P0/7gaLS65wrkya24n+AVH6Pdae+6a0=; b=RRut4kD/ofm4Cz56Ku9dhscPOp1VT3GK53d0zg1OnhaLI5PbUPmBHu1Wp0aeiWygzLHFz2 1noEPxhN9ltg8L7pSyXKnViPHVdiX5ePNx7HwEiQr0Avqhstgh9ugJDrexSK0MtwGpYud1 +MMyPlOgyClhRr3KZ02fusBQ5+UHNHE= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-244-EyFxPtafNhC4PWUB7T8Ovg-1; Wed, 23 Jun 2021 07:13:45 -0400 X-MC-Unique: EyFxPtafNhC4PWUB7T8Ovg-1 Received: by mail-ed1-f69.google.com with SMTP id t11-20020a056402524bb029038ffacf1cafso1118539edd.5 for ; Wed, 23 Jun 2021 04:13:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Qk5GBiM0tj1/P0/7gaLS65wrkya24n+AVH6Pdae+6a0=; b=JL/yb4DKu20IacmIUn/BxM6NfaQh292SSR37JJSVzrbhZjMaPJ99g4qxbBHWtUNA3w BqlQLFmAcXWNGjoP8pCIrP8jEAd2FSsRyssanE0hRaBLeluYue50a4v52QBN16s2H0a5 VC//OYu9DaPYhU9aLs9XzfHjrysDEerjhDoCLT8pBoDXoMQzJuagQPO3nDPUN4VV44zR bTEec18mPksknT5tRXLYgfCjLI6n7ib/CiskoXXH9DUXJQGTgTP16JLuEo/8dAG4AJlw tu0Y2TJ8/bBq2pOMaYa48m27YPGjdc0RfhCNh3W8o3C2M8sj3syZ/IC58GYQxirWDfA5 yAfg== X-Gm-Message-State: AOAM531kdlPU02AG6gEHhtwhFK4hwhFJnlny/petnyA5qBQIb9NG56c+ 1j80C607oUXn36nBnI95PZrtGhjOpwgP+qmTIDxXZb/QWyGraExgW2umnWeS4bwnFRgPNRo9clA 3LpNmFU2xoVau X-Received: by 2002:aa7:c50d:: with SMTP id o13mr11372715edq.9.1624446823855; Wed, 23 Jun 2021 04:13:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJztF5ae03ntoyILjp8pqHlp+iOfl1HNxJXBQz0XPbt2+lIDWho8ON30nAuFqqxYDRxnx9LSsA== X-Received: by 2002:aa7:c50d:: with SMTP id o13mr11372696edq.9.1624446823683; Wed, 23 Jun 2021 04:13:43 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id g8sm13567784edw.89.2021.06.23.04.13.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:13:43 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 3EFD618073B; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jesse Brandeburg , Tony Nguyen , intel-wired-lan@lists.osuosl.org Subject: [PATCH bpf-next v4 11/19] net: intel: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:19 +0200 Message-Id: <20210623110727.221922-12-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The Intel drivers all have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Jesse Brandeburg Cc: Tony Nguyen Cc: intel-wired-lan@lists.osuosl.org Tested-by: Jesper Dangaard Brouer # i40e Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 2 -- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 6 +----- drivers/net/ethernet/intel/ice/ice_txrx.c | 6 +----- drivers/net/ethernet/intel/ice/ice_xsk.c | 6 +----- drivers/net/ethernet/intel/igb/igb_main.c | 2 -- drivers/net/ethernet/intel/igc/igc_main.c | 7 ++----- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 -- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 6 +----- drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 2 -- 9 files changed, 6 insertions(+), 33 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index de70c16ef619..ae3a64b6f5f8 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -2298,7 +2298,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -2329,7 +2328,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) break; } xdp_out: - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c index 46d884417c63..8dca53b7daff 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c +++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c @@ -153,7 +153,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); /* NB! xdp_prog will always be !NULL, due to the fact that * this path is enabled by setting an XDP program. */ @@ -162,9 +161,7 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED; } switch (act) { @@ -184,7 +181,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) result = I40E_XDP_CONSUMED; break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index e2b4b29ea207..1a311e91fb6d 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -1129,15 +1129,11 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) xdp.frame_sz = ice_rx_frame_truesize(rx_ring, size); #endif - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + if (!xdp_prog) goto construct_skb; - } xdp_res = ice_run_xdp(rx_ring, &xdp, xdp_prog); - rcu_read_unlock(); if (!xdp_res) goto construct_skb; if (xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR)) { diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index faa7b8d96adb..d6da377f5ac3 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -463,7 +463,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) struct ice_ring *xdp_ring; u32 act; - rcu_read_lock(); /* ZC patch is enabled only when XDP program is set, * so here it can not be NULL */ @@ -473,9 +472,7 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED; } switch (act) { @@ -496,7 +493,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 038a9fd1af44..8a11b7e55326 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -8387,7 +8387,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -8420,7 +8419,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index ea998d2defa4..2b666a6ec989 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -2175,18 +2175,15 @@ static struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, struct bpf_prog *prog; int res; - rcu_read_lock(); - prog = READ_ONCE(adapter->xdp_prog); if (!prog) { res = IGC_XDP_PASS; - goto unlock; + goto out; } res = __igc_xdp_run_prog(adapter, prog, xdp); -unlock: - rcu_read_unlock(); +out: return ERR_PTR(-res); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index c5ec17d19c59..27d7467534e0 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -2199,7 +2199,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -2237,7 +2236,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index 91ad5b902673..ffbf8a694362 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -100,15 +100,12 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); act = bpf_prog_run_xdp(xdp_prog, xdp); if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED; } switch (act) { @@ -132,7 +129,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, result = IXGBE_XDP_CONSUMED; break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index ba2ed8a43d2d..fabada4ce315 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -1054,7 +1054,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter, struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -1079,7 +1078,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } From patchwork Wed Jun 23 11:07:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339631 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9631BC49EA6 for ; Wed, 23 Jun 2021 11:13:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8156161185 for ; Wed, 23 Jun 2021 11:13:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230288AbhFWLQD (ORCPT ); Wed, 23 Jun 2021 07:16:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29769 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230249AbhFWLQC (ORCPT ); Wed, 23 Jun 2021 07:16:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446825; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gmhJ1VlINXTEKM3djohmsywIadv1r5SFbuahHAWTyD8=; b=K3tgma1Mz+MO509rH/2n+uikGc/fjdA9A/JmOerMdSt2bTY/tolNhMRrbHv16oRvpelUvi WR603V9VUY34w/O9eMmYGYx2+VChUr7ZA1R4wFa1/pkQ20Rc2F7oYcKYgiBqfBHIIpFbZ5 CLkJnYpALlUEFe7sLdfWhBw2gJw/Or0= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-582-5ueRdsoDNxGZaGYKekMJ8w-1; Wed, 23 Jun 2021 07:13:43 -0400 X-MC-Unique: 5ueRdsoDNxGZaGYKekMJ8w-1 Received: by mail-ej1-f70.google.com with SMTP id p5-20020a17090653c5b02903db1cfa514dso845301ejo.13 for ; Wed, 23 Jun 2021 04:13:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gmhJ1VlINXTEKM3djohmsywIadv1r5SFbuahHAWTyD8=; b=qxIXusm59kQlYmY4oMNSXNR1AYDs9pkvIV6qRWMvksxRS0SFTd60aqLr6iBagcPI2G NmLwWHLHdiItrimkb3+QCZ5qjMhpCTytObThpCULDXN+uypCiEEqujevfoSYP2HB35Yn f5jjNB7uZZhCjc7P5o7ft2hV5P5v4ZMkSrM6xIQw3NaS6sAOmyS6+IXiQViG6KOPb346 hCiG7CLGjQkedKJYwi+8Q9L/GRkeAN0ft3YHexL1lhw1NvfyS6AQROWN8gJIRoS9VCBK 5MjnsPEB/PfpPn5hqk4B45KckfByAFr6q+/2ieOrVpNz6vaenI6Im9uWhzb80LZtb1xy eSXA== X-Gm-Message-State: AOAM532BZ7snIktZAGXsplk0eZU1BQUIs+N0HkLJkPUywxNolCsDK3k8 BLnndwtIRjsbe6Hb5kBO5Ay8PqTNMY1MXrNNN4B28f1grJezJLj+HDYyxw5DLv79e8bScrnPp5M Zdxft4bsq8xvI X-Received: by 2002:a17:906:244d:: with SMTP id a13mr9481667ejb.551.1624446822286; Wed, 23 Jun 2021 04:13:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwdQSSJcE+91z72FaynqtRNDzaWX4zKiT+FfIm+2jhIEcgvH1gQT2AERfxu2ILllE2rf+WhJA== X-Received: by 2002:a17:906:244d:: with SMTP id a13mr9481629ejb.551.1624446821923; Wed, 23 Jun 2021 04:13:41 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id aq21sm7624726ejc.83.2021.06.23.04.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:13:40 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 4C25318073C; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Thomas Petazzoni , Marcin Wojtas , Russell King Subject: [PATCH bpf-next v4 12/19] marvell: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:20 +0200 Message-Id: <20210623110727.221922-13-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The mvneta and mvpp2 drivers have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Thomas Petazzoni Cc: Marcin Wojtas Cc: Russell King Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/marvell/mvneta.c | 2 -- drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 4 ---- 2 files changed, 6 deletions(-) diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index 7d5cd9bc6c99..b9e5875b20bc 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -2370,7 +2370,6 @@ static int mvneta_rx_swbm(struct napi_struct *napi, /* Get number of received packets */ rx_todo = mvneta_rxq_busy_desc_num_get(pp, rxq); - rcu_read_lock(); xdp_prog = READ_ONCE(pp->xdp_prog); /* Fairness NAPI loop */ @@ -2448,7 +2447,6 @@ static int mvneta_rx_swbm(struct napi_struct *napi, xdp_buf.data_hard_start = NULL; sinfo.nr_frags = 0; } - rcu_read_unlock(); if (xdp_buf.data_hard_start) mvneta_xdp_put_buff(pp, rxq, &xdp_buf, &sinfo, -1); diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index b2259bf1d299..521ed3c1cfe9 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -3852,8 +3852,6 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, int rx_done = 0; u32 xdp_ret = 0; - rcu_read_lock(); - xdp_prog = READ_ONCE(port->xdp_prog); /* Get number of received packets and clamp the to-do */ @@ -3988,8 +3986,6 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, mvpp2_bm_pool_put(port, pool, dma_addr, phys_addr); } - rcu_read_unlock(); - if (xdp_ret & MVPP2_XDP_REDIR) xdp_do_flush_map(); From patchwork Wed Jun 23 11:07:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339637 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E6B6C48BE5 for ; Wed, 23 Jun 2021 11:14:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68A366102A for ; Wed, 23 Jun 2021 11:14:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230124AbhFWLQR (ORCPT ); Wed, 23 Jun 2021 07:16:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47193 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230302AbhFWLQE (ORCPT ); Wed, 23 Jun 2021 07:16:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446827; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c8O4Yimc+TSjNewaWd3nzsqZvk1ap2sDHxPur5LQuO8=; b=aQ7zXUPViRfdZBC+NloimgYK1k7AMUjLdsGxcWKHMOIzBfSAsigHaYt+qDhDo00m74Q4Rv 8EblEsMhIKRfJLjFg3pUQ3E5mrEMCztLJGAQeHnvRf2SmfAX11NpvCeYDoYzoRlTc0n1mo RJa2Pvip5wL5l5EfWyVVqNdksSrTK2Y= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-38-j6LQDQG6NHysmPaWEi-RbQ-1; Wed, 23 Jun 2021 07:13:45 -0400 X-MC-Unique: j6LQDQG6NHysmPaWEi-RbQ-1 Received: by mail-ed1-f70.google.com with SMTP id t11-20020a056402524bb029038ffacf1cafso1118559edd.5 for ; Wed, 23 Jun 2021 04:13:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c8O4Yimc+TSjNewaWd3nzsqZvk1ap2sDHxPur5LQuO8=; b=RLhSiAjMNjUjbJbIhWfTD4oWLfCDqqWsygo11oTs/iXE63S+T/DKDfNgYRXoNlKYXS Sqqhk73T9y3/S/AeTyOLUea6CQAVigXO+Dy3AlZcK5VFU90uzqILWOSwrpcWmCrdpK05 Z84EIjRlgBhdtVg2GbjzdASIoiAISB3me2KHYOCdZ0a7X5mrhV8fkgiLoXGpZr1U00Of qQaxhIM+rfcYwWZVd6k/PCxxDH1WMvWIMHKcyKvWH2YaZVE6uKJuQtKcy2fhfh1s8UQI MmZoG3241fv1Djb2DZjvGxgnkl/VAJWHmRaOFkybk/zn37gC6P4m/69hbgoe4ypVpzOk by+w== X-Gm-Message-State: AOAM533r1qRrHbmRyINTcpkIu1Bk038vlTu0oj/1r2yEuPyyAfWyexSQ QZRtE1gNX3tEtMRVh9wZevE9+fYXf0uCrUuJt6Qc3rnBIRyDzlJf3+8xC7gNer6esL/PFT+F3ux TKSGPUBthFRzU X-Received: by 2002:a50:fb14:: with SMTP id d20mr3291893edq.187.1624446824538; Wed, 23 Jun 2021 04:13:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzsSY5GdO8QnMV3L9SPl1VreJbS+DP/2ecehYnp9BW+UbIigCzyGSGeiKpi3x7jyvffXawbww== X-Received: by 2002:a50:fb14:: with SMTP id d20mr3291871edq.187.1624446824347; Wed, 23 Jun 2021 04:13:44 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id q20sm7334738ejb.71.2021.06.23.04.13.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:13:43 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 5654518073D; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Tariq Toukan Subject: [PATCH bpf-next v4 13/19] mlx4: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:21 +0200 Message-Id: <20210623110727.221922-14-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The mlx4 driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Also switch the RCU dereferences in the driver loop itself to the _bh variants. Cc: Tariq Toukan Reviewed-by: Tariq Toukan Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index e35e4d7ef4d1..3f08c14d0441 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -679,9 +679,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud ring = priv->rx_ring[cq_ring]; - /* Protect accesses to: ring->xdp_prog, priv->mac_hash list */ - rcu_read_lock(); - xdp_prog = rcu_dereference(ring->xdp_prog); + xdp_prog = rcu_dereference_bh(ring->xdp_prog); xdp_init_buff(&xdp, priv->frag_info[0].frag_stride, &ring->xdp_rxq); doorbell_pending = false; @@ -744,7 +742,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud /* Drop the packet, since HW loopback-ed it */ mac_hash = ethh->h_source[MLX4_EN_MAC_HASH_IDX]; bucket = &priv->mac_hash[mac_hash]; - hlist_for_each_entry_rcu(entry, bucket, hlist) { + hlist_for_each_entry_rcu_bh(entry, bucket, hlist) { if (ether_addr_equal_64bits(entry->mac, ethh->h_source)) goto next; @@ -899,8 +897,6 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud break; } - rcu_read_unlock(); - if (likely(polled)) { if (doorbell_pending) { priv->tx_cq[TX_XDP][cq_ring]->xdp_busy = true; From patchwork Wed Jun 23 11:07:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339635 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8967C4743C for ; Wed, 23 Jun 2021 11:14:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D383E6102A for ; Wed, 23 Jun 2021 11:14:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230480AbhFWLQQ (ORCPT ); Wed, 23 Jun 2021 07:16:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:28485 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230299AbhFWLQE (ORCPT ); Wed, 23 Jun 2021 07:16:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446826; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3OGKlT+K3Cpr2CAOizG3FJ/dNahuGZz8NpheYYFR6ws=; b=O2zr/BJnwqEjDCfiWkYbzi2E+ox7eU/rIyC1eKXJqIcJGNrZuzvgwJD6VbtLWMESYx0Kti gQMCGnCe0BOsIjMo2SUC57fzro1Yx6D58TcoM52zx48PbDp3If0hvi4cFHQ+hZfBPAGGDm zM6YdKQc2IAvduKSyRrP3cQajp5BOrA= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-30-zhkUatjYMHOJem7Plrv3Tg-1; Wed, 23 Jun 2021 07:13:45 -0400 X-MC-Unique: zhkUatjYMHOJem7Plrv3Tg-1 Received: by mail-ed1-f69.google.com with SMTP id r15-20020aa7da0f0000b02903946a530334so1089881eds.22 for ; Wed, 23 Jun 2021 04:13:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3OGKlT+K3Cpr2CAOizG3FJ/dNahuGZz8NpheYYFR6ws=; b=qMmk6HPoisvHm0ZMuZ48H/ta83PtmEqWUgSTLzvePQiBUrLy6LPtl9ZfdTFFvCH5zx 62TbGE72SbWl/LDpPWRY/QHnlkZ4YCuHkO1oRMTQGyfrhpKSKR9I47alvNhFQkuorswu FKQYRe5A+UZULu/ckPZ0qdEhT+jrUVL6Ep7kTqMQ49eh9UUR2Bd7FLI4bxczBUHOKNoU qHyUTRgPkMhT8pl/tdR0lhC500xChrrG9BJVB4DchKYt96B6RO6NvA1s4TOZIAglkoNJ tM/DaFFQdsTxX2+A8pP8wUF8lqdipI/OqgFukOcxWWFWiLhLxo4YD0YFgUn6xG5dz+7v TusQ== X-Gm-Message-State: AOAM533bSTy/tmuvkahSUCwAYBoZt8wb3KEvNa1YB0tfyzouwgKBmsPB wIOpC8OIDXL4aezpd7j9UvxHoM7xRvk2NNE5ZGBPa9VZfonluDS+NuRvOOlA6zGo7Bv1dKq3sLs L1Wy2JX8AEsuu X-Received: by 2002:a17:906:244d:: with SMTP id a13mr9481826ejb.551.1624446824262; Wed, 23 Jun 2021 04:13:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw71VQKEC/9KOOq6gTZpsMVmotp3bYZ3eOEuR03USdj7N4YBneTwSsmElAQRP2hdOY2F02acA== X-Received: by 2002:a17:906:244d:: with SMTP id a13mr9481811ejb.551.1624446824081; Wed, 23 Jun 2021 04:13:44 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id n2sm13477595edi.32.2021.06.23.04.13.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:13:43 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 61ACB18073E; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Simon Horman , oss-drivers@netronome.com Subject: [PATCH bpf-next v4 14/19] nfp: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:22 +0200 Message-Id: <20210623110727.221922-15-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The nfp driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. While this is not actually an issue for the nfp driver because it doesn't support XDP_REDIRECT (and thus doesn't call xdp_do_flush()), the rcu_read_lock() is still unneeded. And With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Simon Horman Cc: oss-drivers@netronome.com Reviewed-by: Simon Horman Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/netronome/nfp/nfp_net_common.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index eeb30680b4dc..5dfa4799c34f 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -1819,7 +1819,6 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) struct xdp_buff xdp; int idx; - rcu_read_lock(); xdp_prog = READ_ONCE(dp->xdp_prog); true_bufsz = xdp_prog ? PAGE_SIZE : dp->fl_bufsz; xdp_init_buff(&xdp, PAGE_SIZE - NFP_NET_RX_BUF_HEADROOM, @@ -2036,7 +2035,6 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) if (!nfp_net_xdp_complete(tx_ring)) pkts_polled = budget; } - rcu_read_unlock(); return pkts_polled; } From patchwork Wed Jun 23 11:07:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339627 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D214C49EA5 for ; Wed, 23 Jun 2021 11:13:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 48199610F7 for ; Wed, 23 Jun 2021 11:13:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230272AbhFWLQD (ORCPT ); Wed, 23 Jun 2021 07:16:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:54841 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230234AbhFWLQC (ORCPT ); Wed, 23 Jun 2021 07:16:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZRUA/Y9lIwzUgTw5uRoa39eJ4yIyIch6xpJH66I9Qdc=; b=cBV1dY+56IayfjMygaVM6XwcOpfakBGjMPq0+MyBVXYFJO1GFZqL4Y5tMv2uAyKa6N959V oyXd62GJ1aOLuBPoORa5NreVE9+Al8i59DZL89FFL5da7bvBgysE/dsr3hZFFN7I81HD4G hiRlxbVN8+xj6e6O48bWWc0Uh3gRtkk= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-490-khex4e86NEGqwEePj9nchA-1; Wed, 23 Jun 2021 07:13:43 -0400 X-MC-Unique: khex4e86NEGqwEePj9nchA-1 Received: by mail-ed1-f70.google.com with SMTP id ee28-20020a056402291cb0290394a9a0bfaeso1112493edb.6 for ; Wed, 23 Jun 2021 04:13:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZRUA/Y9lIwzUgTw5uRoa39eJ4yIyIch6xpJH66I9Qdc=; b=V9wdVlMyxJXa20gbRIO3rQF4ciQhRq98Ecu4+Bn+cAsn3GBa24GaKy3opQdiaSXrAo HJFHV57xR5vSiFoSAzE5Y5dsRkbMH8FW9ZEA00qHxYYymAnl9NHqr8Y4X0DA8vHZaJCC mMM8l+dZuFMuabUQFqdx2/HlDabyWTIgC/xKiFMKXbaCtB59XcAv2gOad1QhnX/rJPWZ tx+P/IaPrpRY8AxXsJzEUk/PJGiG4RZoLBOjpn5LwkPTBoO/Vjl8oOieN5+XrZSWtGMb TBtHA1sf14UFoyjgTzavmhdiq6JLFEf5OhmV9jsA2RF5J4KrALbkmAHOlBOfAAtYNpaL m68w== X-Gm-Message-State: AOAM532MjrGV8GjVpdaAwz6oPXJHEuNAkoQfhYjgR7aQc6dggYfsdY2N MmPJi9zUIIhrADwlqjxzAvx9r6xsn87JvNw68Am4LJEyOVW5/IzoNob0ovvZiiKVfyHGQWJjG04 kYpJmiTOM5Q61 X-Received: by 2002:a17:906:34cf:: with SMTP id h15mr9261315ejb.526.1624446821843; Wed, 23 Jun 2021 04:13:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwNBQa0CtVLyqSCtU7E+yLdrrdnxdFKoAi4uPGzq58jwGDY1F5UViDRBybswKmPs7D4pY6M7Q== X-Received: by 2002:a17:906:34cf:: with SMTP id h15mr9261282ejb.526.1624446821485; Wed, 23 Jun 2021 04:13:41 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id ml14sm3124574ejb.27.2021.06.23.04.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:13:40 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 6C06218073F; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Ariel Elior , GR-everest-linux-l2@marvell.com Subject: [PATCH bpf-next v4 15/19] qede: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:23 +0200 Message-Id: <20210623110727.221922-16-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The qede driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Ariel Elior Cc: GR-everest-linux-l2@marvell.com Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/qlogic/qede/qede_fp.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c index 8e150dd4f899..065e9004598e 100644 --- a/drivers/net/ethernet/qlogic/qede/qede_fp.c +++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c @@ -1089,13 +1089,7 @@ static bool qede_rx_xdp(struct qede_dev *edev, xdp_prepare_buff(&xdp, page_address(bd->data), *data_offset, *len, false); - /* Queues always have a full reset currently, so for the time - * being until there's atomic program replace just mark read - * side for map helpers. - */ - rcu_read_lock(); act = bpf_prog_run_xdp(prog, &xdp); - rcu_read_unlock(); /* Recalculate, as XDP might have changed the headers */ *data_offset = xdp.data - xdp.data_hard_start; From patchwork Wed Jun 23 11:07:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339629 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F828C49EA4 for ; Wed, 23 Jun 2021 11:13:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5CFE5611AD for ; Wed, 23 Jun 2021 11:13:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230267AbhFWLQC (ORCPT ); Wed, 23 Jun 2021 07:16:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47884 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230205AbhFWLQC (ORCPT ); Wed, 23 Jun 2021 07:16:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cOyh19f+n3Klc9n0Uf50USUZ/OAvnTAcZiRzl4mD+JA=; b=W9SyoSi8B1sOzdyy3LCQRfLDEX/RUOxpDwaPziCFQFg5J9jh0XnbA84m8kGMdedx7E3yMI CvTX8PRZGceRt6Pe7LxiaVmPPBz1FXSByJboZJuGm9H7xihcUSGnwloHlucWzaPcMEKkv9 DK+mpScFTtx9PlA6Tqx2SLoKBD8jnCE= Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-475-W0TcVJuyOAuxxoJRnO9oYA-1; Wed, 23 Jun 2021 07:13:43 -0400 X-MC-Unique: W0TcVJuyOAuxxoJRnO9oYA-1 Received: by mail-ej1-f69.google.com with SMTP id o12-20020a17090611ccb02904876d1b5532so851044eja.11 for ; Wed, 23 Jun 2021 04:13:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cOyh19f+n3Klc9n0Uf50USUZ/OAvnTAcZiRzl4mD+JA=; b=XFgnVuuCGNTAIlmGoPOfvvhgIsgPvNci8SnmfVfLqNxpeD5UVxgaoI84S/W/CYycUH VnG1qEj7SjfKula+MY1Nen7v9HA8umpXKVQAF8TPZTjCKa5SeoSiYzu5kSnUpEZ6s768 /DNiIlrxUEZx4UVsuykQD1lNKsyAgCTmk8gpvVhFGAl213qaVmqybJLzpCQssb1QEin/ qNwedIa9iKbPQOEy7m9clYfclAImVhD78BnsRpBF0phQeGqr7pwznQNg1VlGcTSraPtm dYlDXNluOGm144Tw9Mxsi8QLQwYkxKNTciXCaO7e7EM7R0RLn2ROXwTNDbGEG9T46JqC X6kw== X-Gm-Message-State: AOAM533VWWRGLpHvpjO4td82VloqYOxkiilr7rGCTWAgpgHvJQPFnGxm CvDcTKKDgVw/4dh9I/5MVv1UepAz+Lz34VBsng+mu8ePxPYwaalzJFCTfBvANQq41WzvlvN9hh1 g1ADEE0aWyWV0 X-Received: by 2002:a17:906:c108:: with SMTP id do8mr9602766ejc.74.1624446821903; Wed, 23 Jun 2021 04:13:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxFC4J4yF4C76IBXxeFdHx2yW30lP8pvvXj7t+qqzCoChKhiwz3GKGGMBaMUXSqFPorcLDoxw== X-Received: by 2002:a17:906:c108:: with SMTP id do8mr9602748ejc.74.1624446821693; Wed, 23 Jun 2021 04:13:41 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id bq1sm7200517ejb.66.2021.06.23.04.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:13:40 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 74594180740; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Edward Cree , Martin Habets Subject: [PATCH bpf-next v4 16/19] sfc: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:24 +0200 Message-Id: <20210623110727.221922-17-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The sfc driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Edward Cree Cc: Martin Habets Acked-by: Edward Cree Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/sfc/rx.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c index 17b8119c48e5..606750938b89 100644 --- a/drivers/net/ethernet/sfc/rx.c +++ b/drivers/net/ethernet/sfc/rx.c @@ -260,18 +260,14 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel, s16 offset; int err; - rcu_read_lock(); - xdp_prog = rcu_dereference(efx->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + xdp_prog = rcu_dereference_bh(efx->xdp_prog); + if (!xdp_prog) return true; - } rx_queue = efx_channel_get_rx_queue(channel); if (unlikely(channel->rx_pkt_n_frags > 1)) { /* We can't do XDP on fragmented packets - drop. */ - rcu_read_unlock(); efx_free_rx_buffers(rx_queue, rx_buf, channel->rx_pkt_n_frags); if (net_ratelimit()) @@ -296,7 +292,6 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel, rx_buf->len, false); xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp); - rcu_read_unlock(); offset = (u8 *)xdp.data - *ehp; From patchwork Wed Jun 23 11:07:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339623 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5856C49EAB for ; Wed, 23 Jun 2021 11:08:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B18B561166 for ; Wed, 23 Jun 2021 11:08:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230252AbhFWLK0 (ORCPT ); Wed, 23 Jun 2021 07:10:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29898 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230241AbhFWLJ7 (ORCPT ); Wed, 23 Jun 2021 07:09:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446461; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yDyNUpCpBTNbFxDC2WQuaaPWytT5QLCtZC1t66JXAlk=; b=HghLPu/6cLfRJgCvUyZzMOADyhneoMvrhonyjKx80rLTAMV6oU+GkJXwUsfia9QAri9Z0C elBNp6vGPEVoBoHgMMTHGVC0lcOAgCfkUqn6yE5YnkG4nTcsfDWn3PPBLnyMlPNJghQysu WhkjVVhuTq/GLcdXO5n3tktchYQfeDA= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-520-fhR4XsQCPGO1yRcJOOnsWQ-1; Wed, 23 Jun 2021 07:07:40 -0400 X-MC-Unique: fhR4XsQCPGO1yRcJOOnsWQ-1 Received: by mail-ej1-f71.google.com with SMTP id w13-20020a170906384db02903d9ad6b26d8so868253ejc.0 for ; Wed, 23 Jun 2021 04:07:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yDyNUpCpBTNbFxDC2WQuaaPWytT5QLCtZC1t66JXAlk=; b=JOhsKZkjRfAyVKbShHJTGIxM1CMV27Qnm570q5S6bwWnphrSgm0vuytZn4wiircp+q luVBrXqE3q0QYfB+1JlvZEVX9yAuoB1i+PRXUvtBZrW76C/XKBQO8cR4pRhd4c6clR+o rkslEFQOSWpjZxxQcVTjJCoEq6XQ6aMziXChraj3p6fAfQgNkS5+l3OedlZKN6bk8rtw /RxReMMqdcyeZ5yWYHIlLOqwA4LhGoRGe4BnI2nSgxwaXckfDbm0PKZWFPPFI94b3aIm 1yaosShK4vrbyUzhD1xmTD7YKlH30Zjrf4aC29+7YPAEXs9+yQplRRjMQDdJSaphZCZJ ojbw== X-Gm-Message-State: AOAM5328tNlg96wUTIyMfLxCK1MjyoIQyyf/VqFQE7zjC8mH0mgVXP3m CP+g7B75++Bb72KyW7pSQ46H5YlSLVOCQzIOmIWTe7A65welw1ZRl6tSQ3YEdbo4bZZSw30kttE TRpzSFoIXY+pP X-Received: by 2002:a17:906:841a:: with SMTP id n26mr9052465ejx.430.1624446458600; Wed, 23 Jun 2021 04:07:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyiCn+E0GqnflDNDr2cZUSBElhE9Z+SzSMhk9zf2y/FqN45sosWDLQuM6WRitS6qMnDZWcnwQ== X-Received: by 2002:a17:906:841a:: with SMTP id n26mr9052436ejx.430.1624446458285; Wed, 23 Jun 2021 04:07:38 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id j19sm3767903ejo.3.2021.06.23.04.07.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:07:37 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 804F0180741; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jassi Brar , Ilias Apalodimas Subject: [PATCH bpf-next v4 17/19] netsec: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:25 +0200 Message-Id: <20210623110727.221922-18-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The netsec driver has a rcu_read_lock()/rcu_read_unlock() pair around the full RX loop, covering everything up to and including xdp_do_flush(). This is actually the correct behaviour, but because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), it is also technically redundant. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep the rcu_read_lock() around anymore, so let's just remove it. Cc: Jassi Brar Cc: Ilias Apalodimas Acked-by: Ilias Apalodimas Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/socionext/netsec.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index dfc85cc68173..20d148c019d8 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -958,7 +958,6 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) xdp_init_buff(&xdp, PAGE_SIZE, &dring->xdp_rxq); - rcu_read_lock(); xdp_prog = READ_ONCE(priv->xdp_prog); dma_dir = page_pool_get_dma_dir(dring->page_pool); @@ -1069,8 +1068,6 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) } netsec_finalize_xdp_rx(priv, xdp_act, xdp_xmit); - rcu_read_unlock(); - return done; } From patchwork Wed Jun 23 11:07:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339633 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8C17C4743C for ; Wed, 23 Jun 2021 11:13:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 968A6610F7 for ; Wed, 23 Jun 2021 11:13:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230232AbhFWLQE (ORCPT ); Wed, 23 Jun 2021 07:16:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45507 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230205AbhFWLQD (ORCPT ); Wed, 23 Jun 2021 07:16:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446825; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=llJCIySvFOtrYZhTAdojGqVA7hNSjDHEmwwvlUzwRuo=; b=BN5m6m9d/5aK6ze/ACjEHaAt9YX4ZAcQUjh0nHmtWO2lmxxc5PmaBbKfup/gic0HPhgcwu sl/qflNG0JBmaImJwwVzGyAnEccb9xeRwoMsvjjCgqiezM63jNHcTrOrh/XaWXvNckKn78 8laCrxKHy5SRKCmp0eVAi7c8P/uMMt4= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-503-V1YKrT8tN62LSg3aVI019w-1; Wed, 23 Jun 2021 07:13:44 -0400 X-MC-Unique: V1YKrT8tN62LSg3aVI019w-1 Received: by mail-ed1-f72.google.com with SMTP id y18-20020a0564022712b029038ffac1995eso1112407edd.12 for ; Wed, 23 Jun 2021 04:13:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=llJCIySvFOtrYZhTAdojGqVA7hNSjDHEmwwvlUzwRuo=; b=KszepY4fjcIrVxDEwJqdZZdEyIWs0Q81Hw65AFcQ2ZSd+p5HhnkoYiU1GBetgae9Fa /P/Ox00GzMe4zbiYPGpD7Ung83sX8Ga90J2ndqMFlFgFKpJwcbNsOPFrwv9XDFdWkz5O /gBiaPpSbRxD/LuHoCyEWqUKFwxT4WCbq1Cfo5j/Z6ZUx/2KJ1uIh1NqLGpBpJumCCGw YkrjhEqeeH2XKIb7ETi2jfFrkNvuUlZHFo9Td0RZfKbVxU6KWmbTajfIGaUSWmW7XGa6 kcZHEurx3af/7n4omPD3eYg8bfFpPdwyHgCcqE1Djn6WdiQKRUQdwNpeRcpaH/m1ZKgX aTOQ== X-Gm-Message-State: AOAM531dMfrKBEjIucy4DU393Q/aOko5EK5QrDjj2rJOwMcmWFQfcgMv R4/PhRjCakboVsldAwSj0qN8/Vi3LRsNzc3HnVMQa6JmbC/H/2kPY6ORkUb/hjXA5sU2zFhrPhb gWkyOLJdGOCaA X-Received: by 2002:a05:6402:42d2:: with SMTP id i18mr11522743edc.168.1624446823371; Wed, 23 Jun 2021 04:13:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz8yFgTmKihKxpCubnh8O8F/QHFr0n5nAr6MMyZq4hXtQykYRJ1mSniQXOqaN4HV1YIOxxxAQ== X-Received: by 2002:a05:6402:42d2:: with SMTP id i18mr11522725edc.168.1624446823254; Wed, 23 Jun 2021 04:13:43 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id gx4sm3605409ejc.34.2021.06.23.04.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:13:40 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 872E3180742; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Giuseppe Cavallaro , Alexandre Torgue , Jose Abreu Subject: [PATCH bpf-next v4 18/19] stmmac: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:26 +0200 Message-Id: <20210623110727.221922-19-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The stmmac driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Giuseppe Cavallaro Cc: Alexandre Torgue Cc: Jose Abreu Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index bf9fe25fed69..08c4b999e1ba 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -4654,7 +4654,6 @@ static int stmmac_xdp_xmit_back(struct stmmac_priv *priv, return res; } -/* This function assumes rcu_read_lock() is held by the caller. */ static int __stmmac_xdp_run_prog(struct stmmac_priv *priv, struct bpf_prog *prog, struct xdp_buff *xdp) @@ -4696,17 +4695,14 @@ static struct sk_buff *stmmac_xdp_run_prog(struct stmmac_priv *priv, struct bpf_prog *prog; int res; - rcu_read_lock(); - prog = READ_ONCE(priv->xdp_prog); if (!prog) { res = STMMAC_XDP_PASS; - goto unlock; + goto out; } res = __stmmac_xdp_run_prog(priv, prog, xdp); -unlock: - rcu_read_unlock(); +out: return ERR_PTR(-res); } @@ -4976,10 +4972,8 @@ static int stmmac_rx_zc(struct stmmac_priv *priv, int limit, u32 queue) buf->xdp->data_end = buf->xdp->data + buf1_len; xsk_buff_dma_sync_for_cpu(buf->xdp, rx_q->xsk_pool); - rcu_read_lock(); prog = READ_ONCE(priv->xdp_prog); res = __stmmac_xdp_run_prog(priv, prog, buf->xdp); - rcu_read_unlock(); switch (res) { case STMMAC_XDP_PASS: From patchwork Wed Jun 23 11:07:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12339625 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 715E1C48BC2 for ; Wed, 23 Jun 2021 11:13:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 575B161166 for ; Wed, 23 Jun 2021 11:13:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230236AbhFWLQC (ORCPT ); Wed, 23 Jun 2021 07:16:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37026 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230225AbhFWLQB (ORCPT ); Wed, 23 Jun 2021 07:16:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624446824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=436OaxB7FEvDC8Cj4cGRRV7bgo1JzvFAvSGPF+E5hVA=; b=WYqNt7VrfMPudHS4vOBEDIOeKp9b+FM95dU/6Gtg2iiIPcgw68GcN7NTZFwZOjKuuvmlwz twaVR222m8MIXnlcJ54wzOsY68X0vicZDZw349Igm+QpCvw8W1O03TBYEdfkobcefPso4w UgQs4Emc5sLBdm6q9RTR/P9n+WYZNL4= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-540-X33T-IpTN-W1aOJRWfFOPQ-1; Wed, 23 Jun 2021 07:13:42 -0400 X-MC-Unique: X33T-IpTN-W1aOJRWfFOPQ-1 Received: by mail-ej1-f72.google.com with SMTP id 16-20020a1709063010b029037417ca2d43so857527ejz.5 for ; Wed, 23 Jun 2021 04:13:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=436OaxB7FEvDC8Cj4cGRRV7bgo1JzvFAvSGPF+E5hVA=; b=D2hVwbNkjpzmTdF71VwXncqTDvJYmlq5piaxG9cuI+rvjwte4CodVeD12KcMa9eHc+ E1zJYvP5Kz5udm9zsGh+JlA5lZ83P0vcnH8Gv79+focw8tmvJHtanDyUPLEWqhSucToP 5v+XYLHUHATig03CbVKU+EVMyG8hsJLo6IdORmrwgtkoS2/JcnGhjbeRL6FnyknDnngl qyB4VeRbu/LL3BFcjY0QSUjcYY0LjTN6wjrpjFvDVq52+9e1MUXhQtdhKHhjxBAbGDGu LfQLYotpKHQTDzRL7LDP5ZTydhQDiggg5w3pRgKTnGy75cLKtzKC1S0qQnmMZ1pst/rI NJ2g== X-Gm-Message-State: AOAM532N40ewja3wiACl17Ow67wi/aQzkIV6CQQu+OoskONHnmWaBq87 3qz7OGqUt9Z8zS1wFy++OD8ptszXAJ4JAhU2q12dLd7SFGde8jkmqtTHfvqQt8B3gpw5G86RgWL v17n0iqtbY52R X-Received: by 2002:a17:907:628d:: with SMTP id nd13mr9065697ejc.299.1624446821447; Wed, 23 Jun 2021 04:13:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzaB0sFOK4WAUNgBrAcENVFdpPOx4htt22G+HZurPvZsTao6jHLM32s7B6T27sxtDVUXTa1cg== X-Received: by 2002:a17:907:628d:: with SMTP id nd13mr9065675ejc.299.1624446821251; Wed, 23 Jun 2021 04:13:41 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id d7sm4046175edy.6.2021.06.23.04.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 04:13:40 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 8C7E0180743; Wed, 23 Jun 2021 13:07:28 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Grygorii Strashko , linux-omap@vger.kernel.org Subject: [PATCH bpf-next v4 19/19] net: ti: remove rcu_read_lock() around XDP program invocation Date: Wed, 23 Jun 2021 13:07:27 +0200 Message-Id: <20210623110727.221922-20-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210623110727.221922-1-toke@redhat.com> References: <20210623110727.221922-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The cpsw driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Grygorii Strashko Cc: linux-omap@vger.kernel.org Tested-by: Grygorii Strashko Reviewed-by: Grygorii Strashko Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/ti/cpsw_priv.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/ti/cpsw_priv.c b/drivers/net/ethernet/ti/cpsw_priv.c index 5862f0a4a975..ecc2a6b7e28f 100644 --- a/drivers/net/ethernet/ti/cpsw_priv.c +++ b/drivers/net/ethernet/ti/cpsw_priv.c @@ -1328,13 +1328,9 @@ int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp, struct bpf_prog *prog; u32 act; - rcu_read_lock(); - prog = READ_ONCE(priv->xdp_prog); - if (!prog) { - ret = CPSW_XDP_PASS; - goto out; - } + if (!prog) + return CPSW_XDP_PASS; act = bpf_prog_run_xdp(prog, xdp); /* XDP prog might have changed packet data and boundaries */ @@ -1378,10 +1374,8 @@ int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp, ndev->stats.rx_bytes += *len; ndev->stats.rx_packets++; out: - rcu_read_unlock(); return ret; drop: - rcu_read_unlock(); page_pool_recycle_direct(cpsw->page_pool[ch], page); return ret; }