From patchwork Wed May 24 15:33:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13254201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D5DAC7EE32 for ; Wed, 24 May 2023 15:33:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A60D900016; Wed, 24 May 2023 11:33:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1301D900007; Wed, 24 May 2023 11:33:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC411900016; Wed, 24 May 2023 11:33:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D9955900007 for ; Wed, 24 May 2023 11:33:49 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B7A1540A2C for ; Wed, 24 May 2023 15:33:49 +0000 (UTC) X-FDA: 80825543778.06.E1D4D46 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 1594E40019 for ; Wed, 24 May 2023 15:33:47 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bjhZvAul; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684942428; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hgbi0fncQFlNqYJvJIncCzqzDTHwL/0fY7UO/lvli4s=; b=QUH1z0ORzgtOQ8OAKVfijzg81ynOm0zVWBEjCTMROkbdrlX/rGLolqHvtApkVFiKzZeqeT x7un2WlOxaCsIgnY3ChfA6g9M4RrAMrHCJxuJjiWvQsFNan70YZEzLrZtaCMkDxNw8ezAd lhx/S+GzAaBuCyt4THFDOnEGjaDo65U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684942428; a=rsa-sha256; cv=none; b=PLFRcZtbFyu1Xhzdw7H2IF2OXi1JIAvIH6CK/CL2E//3GDT/v942PHlSESV1OfzfEOIZvg ZIzN0O9sFhGiZxV88EYxmD4wpcEK9Oed8n08tYhSMwBMiKLlE4TbeVC/t1B/q8uwizczMZ SiZ5OAT2mAqooT+T76cniVwDO2gyXNg= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bjhZvAul; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684942427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hgbi0fncQFlNqYJvJIncCzqzDTHwL/0fY7UO/lvli4s=; b=bjhZvAulGll7IJ/qVh+Q5ipODUMBSh9DoFt3h0jhJoCZ06vUTtVrAjFZZOYo7uU+44CXJM jNn1SW/kHaDLu/FJKy4TA2i5M3u3KyaaSfcsp5ZOR3ZDZGYeYiBB6Y+g64os+huIkgCphL 5knqKvpUIGZz1hRlPN/rVPJ4iRGumUE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-170-IhsFsMjsP6yLutFsBhfOSA-1; Wed, 24 May 2023 11:33:44 -0400 X-MC-Unique: IhsFsMjsP6yLutFsBhfOSA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6EDD280027F; Wed, 24 May 2023 15:33:43 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2DC4C1ED99; Wed, 24 May 2023 15:33:41 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 07/12] net: Clean up users of netdev_alloc_cache and napi_frag_cache Date: Wed, 24 May 2023 16:33:06 +0100 Message-Id: <20230524153311.3625329-8-dhowells@redhat.com> In-Reply-To: <20230524153311.3625329-1-dhowells@redhat.com> References: <20230524153311.3625329-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Stat-Signature: z94h9siopin3g9k5xgsoozzep5cbatwf X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 1594E40019 X-Rspam-User: X-HE-Tag: 1684942427-486341 X-HE-Meta: U2FsdGVkX19WL4SQkQPd6R7AZZv2pZD2Ilrp5BIgJeFkf8Yw2VvcqsPCDnleSv/27usjn73miQuzB0ygmdfq/Qweaz12WceULoNo5lm3F2C+wIPgCC1cKGyrtHHZHmIYHPzhTi9l1Ryk5l8bdGbqDEK8VMLa0wg+gilNRmjpc20szoUGKbliirSByYmBvLA4n3ehhGFTgJqJ5ydLEN2zhW/ku8UY9mvPshrI9HkgeQWYm0CBMT51BolitwGvML5/3f1t72NDbVLdB8t+xv6msGQ3ukYXOaubvbXSWN92jlEh/W1nTl9di06z469SAe7CSEGxMeBPnhFETTSCbbTfVcoerYpsm4Q8j5blT22rtPTOft00Eqgy2kuiElGzE/lgeiBUQQhI2iNZTSoSaHLkqwBOOU6aXeYrLU4ZdbXigxHuKLTPBDNK0tpEMm+sd5lBfRs5mfyqFQoEQhJD/FwhXXMXvKWyZeSqlA7to+HLXLbgGp3orhyRrb/SCZ6Vcp88MZ/lUORM5hZXT7jkkGAgESa+tUWphVVwaDPGfRBE+Xd4ObfPkeTIQRc7UCWVhaNmfPcJT0E4Bn+hni/mJvRkMm9wTlF10X8vdFbhbd5+W7H5gmbwBfAejZs4yhycPWf6ECBX9qmyJ4gJfothp+hPWtrSoC54d3VqCwu8wEWEH2s9PjmMOa9PKGVROkBlmXRI2ShjJUXthPOwvEK+bINmmFn2tnlljSobtFLR8Syz7CdulvTBIcOqrFhr8zLsNxI4FNjRp8rz/Po1ssgzW6xHtCyjnAcRtSiT2+KNUb+u5H0iF3007E0dUZGo0qmEJHco1LDG+gOYNPH8s6FSHr6Dgft1fvQvyxFPWF0zptcX7/wxDtozIRPO312JLdYdqfFvG9H7pXM+Wv2qsChgorDkRCszXMghk4fvuYqhgnvIa+OnDLQFqVUIgPFx1xRYc54kD/KVARt7SW/w4r1pKgT J4kRzTwE Ox+nQCXk0dQG45gfEQlE5JDLLkWKNlmrzM2vlJ7HOktcjq42geYYs6Gqn8BP1PlTx0p10q+T6ZcWRMNDYastrMmmFKdbc6aNyZ5w7nzdhSdbUX8tdF4jo8Bg8C3sPAhF8aTxTEyMeoaS7gDOaqI/1g0gafciIympKXNs+2BS46E4EREqZ6TElLswZMyZmMf0/kY91TLyTlhhysYTM6qnO9BWekxnNfKIiiKhWqvh1y3AZQ2S9A9Ly4aVkkeeT0DwjFzkQvm0fZaN3WZRDNl3CXZE2yQ9j7qDEdb1ZCLVPWL01Txn/04AAZCSpthwZ4dxxuzThdLuTn/gxH5sUoZFOLMtbIOl57AndPXT0DE37wmJUpLV7Ou5GepVhccs42VrCBvrTX72ziVQYm/wadKLrI9oUMqbR5uzhtr0iANoeZKpPzq3bcup5P0YjG4TihlAWxqwLyRHeFpIeoPJtqY5TAkHXU+YC2B1Cu4dSYttC2z/P1KJsxuuuFADaPw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The users of netdev_alloc_cache and napi_frag_cache don't need to take the bh lock around access to these fragment caches any more as the percpu handling is now done in page_frag_alloc_align(). Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: linux-mm@kvack.org --- include/linux/skbuff.h | 3 ++- net/core/skbuff.c | 29 +++++++++-------------------- 2 files changed, 11 insertions(+), 21 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 41b63e72c6c3..e11a765fe7fa 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -252,7 +252,8 @@ /* Maximum value in skb->csum_level */ #define SKB_MAX_CSUM_LEVEL 3 -#define SKB_DATA_ALIGN(X) ALIGN(X, SMP_CACHE_BYTES) +#define SKB_DATA_ALIGNMENT SMP_CACHE_BYTES +#define SKB_DATA_ALIGN(X) ALIGN(X, SKB_DATA_ALIGNMENT) #define SKB_WITH_OVERHEAD(X) \ ((X) - SKB_DATA_ALIGN(sizeof(struct skb_shared_info))) diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 225a16f3713f..c2840b0dcad9 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -291,27 +291,20 @@ void napi_get_frags_check(struct napi_struct *napi) void *napi_alloc_frag_align(unsigned int fragsz, unsigned int align) { - fragsz = SKB_DATA_ALIGN(fragsz); - + align = min_t(unsigned int, align, SKB_DATA_ALIGNMENT); return page_frag_alloc_align(&napi_frag_cache, fragsz, GFP_ATOMIC, align); } EXPORT_SYMBOL(napi_alloc_frag_align); void *netdev_alloc_frag_align(unsigned int fragsz, unsigned int align) { - void *data; - - fragsz = SKB_DATA_ALIGN(fragsz); - if (in_hardirq() || irqs_disabled()) { - data = page_frag_alloc_align(&netdev_alloc_cache, + align = min_t(unsigned int, align, SKB_DATA_ALIGNMENT); + if (in_hardirq() || irqs_disabled()) + return page_frag_alloc_align(&netdev_alloc_cache, fragsz, GFP_ATOMIC, align); - } else { - local_bh_disable(); - data = page_frag_alloc_align(&napi_frag_cache, + else + return page_frag_alloc_align(&napi_frag_cache, fragsz, GFP_ATOMIC, align); - local_bh_enable(); - } - return data; } EXPORT_SYMBOL(netdev_alloc_frag_align); @@ -709,15 +702,11 @@ struct sk_buff *__netdev_alloc_skb(struct net_device *dev, unsigned int len, if (sk_memalloc_socks()) gfp_mask |= __GFP_MEMALLOC; - if (in_hardirq() || irqs_disabled()) { + if (in_hardirq() || irqs_disabled()) data = page_frag_alloc(&netdev_alloc_cache, len, gfp_mask); - pfmemalloc = folio_is_pfmemalloc(virt_to_folio(data)); - } else { - local_bh_disable(); + else data = page_frag_alloc(&napi_frag_cache, len, gfp_mask); - pfmemalloc = folio_is_pfmemalloc(virt_to_folio(data)); - local_bh_enable(); - } + pfmemalloc = folio_is_pfmemalloc(virt_to_folio(data)); if (unlikely(!data)) return NULL;