From patchwork Wed Feb 2 01:12:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732480 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21259C433F5 for ; Wed, 2 Feb 2022 01:13:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243617AbiBBBNM (ORCPT ); Tue, 1 Feb 2022 20:13:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229994AbiBBBNK (ORCPT ); Tue, 1 Feb 2022 20:13:10 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1329BC061714 for ; Tue, 1 Feb 2022 17:13:10 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id oa14-20020a17090b1bce00b001b61aed4a03so4408985pjb.5 for ; Tue, 01 Feb 2022 17:13:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jZQROXOP0pza6cD/HAg54T+BPrYK4YdGCqA3Sg45Jw8=; b=RDBf+s6K0tMlVJLHBBbFtWPd3Ign7mH8rambp7zJDwcbXOD6rGamKQFFtuHcbxhm2n pW9NXo6PLUOF/pDg1hkZW3mu4bx0pgE2Pqhf8CAVs4cuthu1ATOxxyL0UYORwEwQ1yX9 gbr61c6OOy+OpmLXLeC+1rdr5Fuk7RYzpXGDA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jZQROXOP0pza6cD/HAg54T+BPrYK4YdGCqA3Sg45Jw8=; b=cLl9Bui+kma9pJopBzcU9+kaTQI5aeUg8vchGovR7YY0hH2A7Rswdr2v2+0wn2Tm5z H+dQITweCU4BXznMZ96O6dxguLsBoynUIuNw+aMzljcnPkuRCFxNJ4qDM2bQIJE+QC8U JAc7/P23ziTCm69TSfcGYdC3/xdBh6tnOY56W0spGpvDUnePUf9BEMn+9N2XN4TPe76r TYfD3CT3m7Ut3EfveJnF9vPGlgVMruh+qVII5Nsdv0ihEXw1+2vJuq9RHoaLnBiDcRZQ 6abtkStEpKb5qYhJ0n0s6EW9hPHLXZxTVJN5S6XHeICJuaY4ZwSYr1uLMLaKf35OvYEt WcTA== X-Gm-Message-State: AOAM53325Z1bz84HdvawnxH5HFAdhiOrsd2RNZYjZYQuj/Rlf9t1m4Fd t2D9Ty9gylhl+C4MGH4mpvgAluwBVTpMLQrUV2Bu0Yfp2oKC9M8HwVlz7wSRipyCRJXOmADfv74 3FPghwxim2uOtMltLXmJsG7bSADSOec7J6J00KlvqBRgjMthLdm0sGYK0LjEzdxAkW/Xd X-Google-Smtp-Source: ABdhPJzKA0dPqkMZSKJbqrEcG9Jg5+e5DtqDgH/RgjjZ650Q5zL///I99kL3PLV1bX2WMAEzticJFw== X-Received: by 2002:a17:902:8204:: with SMTP id x4mr28797680pln.18.1643764389130; Tue, 01 Feb 2022 17:13:09 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:08 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 01/10] page_pool: kconfig: Add flag for page pool stats Date: Tue, 1 Feb 2022 17:12:07 -0800 Message-Id: <1643764336-63864-2-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Control enabling / disabling page_pool_stats with a kernel config option. Option is defaulted to N. Signed-off-by: Joe Damato --- net/Kconfig | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/net/Kconfig b/net/Kconfig index 8a1f9d0..604b3eb 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -434,6 +434,18 @@ config NET_DEVLINK config PAGE_POOL bool +config PAGE_POOL_STATS + default n + bool "Page pool stats" + depends on PAGE_POOL + help + Enable page pool statistics to track allocations. Stats are exported + to the file /proc/net/page_pool_stat. Users can examine these + stats to better understand how their drivers and the kernel's + page allocator, and the page pool interact with each other. + + If unsure, say N. + config FAILOVER tristate "Generic failover module" help From patchwork Wed Feb 2 01:12:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732481 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31F87C433EF for ; Wed, 2 Feb 2022 01:13:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243621AbiBBBNN (ORCPT ); Tue, 1 Feb 2022 20:13:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229994AbiBBBNM (ORCPT ); Tue, 1 Feb 2022 20:13:12 -0500 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4750DC061714 for ; Tue, 1 Feb 2022 17:13:12 -0800 (PST) Received: by mail-pj1-x102e.google.com with SMTP id oa14-20020a17090b1bce00b001b61aed4a03so4409027pjb.5 for ; Tue, 01 Feb 2022 17:13:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5MPwYILWWDHRKPtSihChctU6rXVQMQ9CjUeZyzwHNXA=; b=qsDBl+ImIGCU5q7Gu7kENCYbyFg5c8Uq5cT+hZqUhVQmZ1UW5kguipJjyGQH4owHJS cpLup4UCa7YNFHasPmAlB/JcdXXdErJWWKybNH3XGUUANv/T09bqdlKLD9vygubD1wHj RC8j+ASDIMTcuS2J1WfYBrJV7Vi1eKi2GI6E4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5MPwYILWWDHRKPtSihChctU6rXVQMQ9CjUeZyzwHNXA=; b=pIAC8kiChOC2bXRu5J5KxT9XBzJoTGFRfivzEyp3vTycyI1IfHl86yyvBNM/Sz7jTF AeJgXNYrqw6AcoHKYLv3nsVoF+Gn8NTykpjR7B+lMqr7uh1wFCIyd+lTLx7zHgHDVZxd kceBBr+ZXPIWJwtyso+7Rj6IKk8KPt9Ci88T96Km0g80CIUjD1+sFWt3uA9pA9SGozcN yE1DDYf1dSjO6FpTiWS+oWeBoppdzb3sPUQclaFHj/GhkazgVoVkIMsC3FnqcBMvJlLH ++5tvl58upLpON1Xw2RIkHtZu2zFzvTdVYCbyrEC3gjaSSy9bHjf0y5rXexYGAiwS4Mh msMQ== X-Gm-Message-State: AOAM530w28GLvCsGG4Vnob8MURSPjN9QXjJeWJMmYSHURCEExUt6k/Jp 3/SGmGnZaWEF1uqzlwLMGw3PaF1edMOwRrI17L6QcqeOo4JOH06om30qC5i387i/dbQGEQ0R4vd XnRDMU5H8Skwt9W8q3JAdBBDh0VPbocD0qxWTTEqAl2xyLqR0OGb4DYggUE10PIZB1wXI X-Google-Smtp-Source: ABdhPJzLTDIGuxBDwuZeeL509TnDm450ROwO5S7iMtZhClC4AdUtZ5Gx1iqX1cFjCUeyIHT0XNlSMQ== X-Received: by 2002:a17:902:c40e:: with SMTP id k14mr28151740plk.103.1643764390679; Tue, 01 Feb 2022 17:13:10 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:10 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 02/10] page_pool: Add per-cpu page_pool_stats struct Date: Tue, 1 Feb 2022 17:12:08 -0800 Message-Id: <1643764336-63864-3-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org A per-cpu (empty) page_pool_stats struct has been added as a place holder. Signed-off-by: Joe Damato --- include/net/page_pool.h | 10 ++++++++++ net/core/page_pool.c | 5 +++++ 2 files changed, 15 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 79a8055..dae65f2 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -137,6 +137,16 @@ struct page_pool { u64 destroy_cnt; }; +#ifdef CONFIG_PAGE_POOL_STATS +/* + * stats for tracking page_pool events. + */ +struct page_pool_stats { +}; + +DECLARE_PER_CPU_ALIGNED(struct page_pool_stats, page_pool_stats); +#endif + struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp); static inline struct page *page_pool_dev_alloc_pages(struct page_pool *pool) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index bd62c01..7e33590 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -26,6 +26,11 @@ #define BIAS_MAX LONG_MAX +#ifdef CONFIG_PAGE_POOL_STATS +DEFINE_PER_CPU_ALIGNED(struct page_pool_stats, page_pool_stats); +EXPORT_PER_CPU_SYMBOL(page_pool_stats); +#endif + static int page_pool_init(struct page_pool *pool, const struct page_pool_params *params) { From patchwork Wed Feb 2 01:12:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732482 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CBF8C433F5 for ; Wed, 2 Feb 2022 01:13:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243626AbiBBBNO (ORCPT ); Tue, 1 Feb 2022 20:13:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243618AbiBBBNN (ORCPT ); Tue, 1 Feb 2022 20:13:13 -0500 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 038EEC061714 for ; Tue, 1 Feb 2022 17:13:13 -0800 (PST) Received: by mail-pl1-x636.google.com with SMTP id j16so16871604plx.4 for ; Tue, 01 Feb 2022 17:13:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=daCp29CfHqba5AzFqLjgXp71o/Cmax20BodJH4eahfI=; b=rCfcL+1/Iefyd2FrXdrcQsQIzJ+47YUFjUB1ORbXt19QFY+Hi8Vq689uM9TB+fyZtc mcXR1zGpVrNlUUsMMwyeh0C9UrSxPCrFbFp4tRnJcv5aE7y+8sl1scePmp2KYFEOknEG n3FgEbQ+sGHXhZD3QIh3rc6Bl8hR6/Gh+zlZ4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=daCp29CfHqba5AzFqLjgXp71o/Cmax20BodJH4eahfI=; b=Dw88/wId3d5yl4sqQAXocPhqHpK+fvaaiVDcsiAgEs/78AlRgxnxzJqv/3VicM0D33 pisgYkq/jVhrh6wjhm7JLRmRb4lIW6lY7OSwwZ8Ihcr3O0fioWdSIG+C38pEJvun6WHm EXB5AnmUBeYl9zBZdqxUABIypcjtP84pklAAHeTtPejtjea9XXh5/5KGE6/uEfqlVQbw 8b2C4QdhnZw6p+h7oDkgf2wauJ1s32oOYZhDs16VO585m7XmIa6p+SiMac1bOGxOAyA9 2NkXE/ITgWXLOQmssIZzvUihLOk7dKR54/U/ubNGkWJQc2n8w1IoWeTuZOARZF29a0SG yONA== X-Gm-Message-State: AOAM531RsPz8u29WF/OR0kYIHHlelUtbrc95iN1I2fHqlkLUl/1uvubL zar6/QeQ5FXJlkx7EP0l1keUOrf5NYChpLDIgIYLxBMMsFODF/Bg2y+bDVo4KQYPdeIM53nSZTE O57XyChmntslONDwG/r+WXAZnORJUsMwKPhiQ4yHBVi894JgNKIfUySaPcLVmYpuZkHwP X-Google-Smtp-Source: ABdhPJxYIc/L5PcsCZEH5mMcEf/+DqTB1uM6RrRc/tuGJm4oL8UoJPloCuGX2Bz7JL3YZffOB2Ffgg== X-Received: by 2002:a17:902:aa90:: with SMTP id d16mr28355867plr.89.1643764392117; Tue, 01 Feb 2022 17:13:12 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:11 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 03/10] page_pool: Add a macro for incrementing stats Date: Tue, 1 Feb 2022 17:12:09 -0800 Message-Id: <1643764336-63864-4-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add simple wrapper macro for incrementing page pool stats. This wrapper is intended to be used in softirq context. Signed-off-by: Joe Damato --- net/core/page_pool.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 7e33590..b1a2599 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -29,6 +29,15 @@ #ifdef CONFIG_PAGE_POOL_STATS DEFINE_PER_CPU_ALIGNED(struct page_pool_stats, page_pool_stats); EXPORT_PER_CPU_SYMBOL(page_pool_stats); + +#define page_pool_stat_alloc_inc(__stat) \ + do { \ + struct page_pool_stats *pps = this_cpu_ptr(&page_pool_stats); \ + pps->alloc.__stat++; \ + } while (0) + +#else +#define page_pool_stat_alloc_inc(stat) #endif static int page_pool_init(struct page_pool *pool, From patchwork Wed Feb 2 01:12:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732483 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4D76C433FE for ; Wed, 2 Feb 2022 01:13:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243629AbiBBBNP (ORCPT ); Tue, 1 Feb 2022 20:13:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243618AbiBBBNO (ORCPT ); Tue, 1 Feb 2022 20:13:14 -0500 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63A14C061714 for ; Tue, 1 Feb 2022 17:13:14 -0800 (PST) Received: by mail-pg1-x52b.google.com with SMTP id h125so29885pgc.3 for ; Tue, 01 Feb 2022 17:13:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5DfdEyld7StyWrFrpQh0H8+Y2uMglsDtEWNNoHSX1Uo=; b=ZEhyiEoV/fR+sTr6984edWTDMi2uhk43lxKsyNeXBxMz08Yd05wndjzxr97k316uFP XeydJTa86Tjg88VszGOl2rGSHgq6MM6bHFGDw47ox7nWch5l4jKyzJzQp9/KlEfl0bx8 OE3lWvxQfAm5RcBeNPWFWwZln5GDFqCM7hjY0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5DfdEyld7StyWrFrpQh0H8+Y2uMglsDtEWNNoHSX1Uo=; b=cGIz1vOJMJzGP/As9Jlrk8DuloQRkX7bOtFHLrawjby2u851SeWpt019jz5lLOGWUn hHqn1yoz0qbpQ8WQhAzRye9JKNLyNs+V28/OUQs4bSJZWxtjbRGHIIEu+WdZ9kuCN7lk MjzomkiVQLtJYTAUNu/kz4tVAOw0yk2n5JD+i+IXY+q7iyrNEWB8/Q7NpexDgSC7/8pB KAPM+5Sak3ZGC8i6N/b3S0FKBsEpowy7VakKrVXzC2rfoPOByCKJcbETeEFAMZiP0Nbi MyNpYhyua+VeVMCJVyDP68Y/xueMsNv4liUdb8AM0CZsOrftybAHG5uI/a7AlNboVGyP xGEg== X-Gm-Message-State: AOAM533fdf1v88FxwYjGJgj4XSigM5XT2rO6u7ocXLVEj7BqZN9egCQl oaPcVglw8JTHKoveJHBFftZhWVqgAu5vy1YS6Rv6tDQTGs7S1oJHzMQCfPTY3BHly5CIRzIWyr5 wd7733i+bZZIn2WiV89hior9p22nrDBRpQbalHRlkq05Iz3Za+yayeZzXZLzSO46iWbb7 X-Google-Smtp-Source: ABdhPJxK68SehvkETpaUFSob1E0Yg9F8bw8eTKsPtsy+F1i4zzEy4dUQSrD45Lk8RBebM0jzRWY3YQ== X-Received: by 2002:a63:451f:: with SMTP id s31mr23324271pga.114.1643764393553; Tue, 01 Feb 2022 17:13:13 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:13 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 04/10] page_pool: Add stat tracking fast path allocations Date: Tue, 1 Feb 2022 17:12:10 -0800 Message-Id: <1643764336-63864-5-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a counter to track successful fast-path allocations. Signed-off-by: Joe Damato --- include/net/page_pool.h | 3 +++ net/core/page_pool.c | 1 + 2 files changed, 4 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index dae65f2..96949ad 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -142,6 +142,9 @@ struct page_pool { * stats for tracking page_pool events. */ struct page_pool_stats { + struct { + u64 fast; /* fast path allocations */ + } alloc; }; DECLARE_PER_CPU_ALIGNED(struct page_pool_stats, page_pool_stats); diff --git a/net/core/page_pool.c b/net/core/page_pool.c index b1a2599..6f692d9 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -180,6 +180,7 @@ static struct page *__page_pool_get_cached(struct page_pool *pool) if (likely(pool->alloc.count)) { /* Fast-path */ page = pool->alloc.cache[--pool->alloc.count]; + page_pool_stat_alloc_inc(fast); } else { page = page_pool_refill_alloc_cache(pool); } From patchwork Wed Feb 2 01:12:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732484 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92B74C433EF for ; Wed, 2 Feb 2022 01:13:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243628AbiBBBNQ (ORCPT ); Tue, 1 Feb 2022 20:13:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243630AbiBBBNQ (ORCPT ); Tue, 1 Feb 2022 20:13:16 -0500 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECAA0C06173D for ; Tue, 1 Feb 2022 17:13:15 -0800 (PST) Received: by mail-pl1-x629.google.com with SMTP id z5so16838192plg.8 for ; Tue, 01 Feb 2022 17:13:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NTd7TBI6abPTmN3I9QjdcpJvF3jv9TVetLryoOk/zcc=; b=l/KwjDz1TCEY1XcxLRvSdICPQnUGqRb9sGPDGFUnwfdjegjgVkOGtMF0MkgLwOLfzE 4mVmvfpD74W32+diIdu8AzuC1/JKsQaSAl6wh05/+6Q4+catorp3+1CGNz2AqMeSi5Lo yBPmhel/BaVsxKEwOcMrK6GOQ3s043sqQ0g6A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NTd7TBI6abPTmN3I9QjdcpJvF3jv9TVetLryoOk/zcc=; b=sASscXQQvwCgHTBTmbJhMd1cnVGtYsraIQxwFgUsn+kgf/njgCgQGcYWQGSLETGs+C nhHWI4kRBShhiwk9CY/v3c1xSqVWwmYdk0uTrEKI0Si0IT8N8qNcri6evUNsIe/z1qf4 g5I00v9nveEnyKzfHsoQ5TZyhZ2NbtkizyGkDLY+OSIU2sraWVC25UoZMhpLPrb57kkB tb7fUn+4iNpJl6uIlbT4ZXyOurrXaIVA5znfC1VkG0CZZMsdnYqzftmKZw2yYXImWo4O ysmoLSnPSmXUO+ysmLIKzxRSrtr/dX63fI0GdyzDgk1iO6Jy0m6bSjKJAivC7nWqXrTs KlEQ== X-Gm-Message-State: AOAM533ZHYnubdErHC7KU+ff0YJ09ncyqHcBCw899ESTHpSi+i2LMqhu GojVHI+U2nqzgi6rFHuKUyD7+Gh9BJJ1iswQbq4IQ6OaexNuoKX1ocN3d6wdT+DIRbGplpknhHc 0ZFuRqAUpYxWkmhq9U6/uplvQOOMxaWbDh009s5CkZ7tSZ99WROogv+nAKo1PsGoIWgLf X-Google-Smtp-Source: ABdhPJzcAISH0g4wX6g/wgDoBnlAZqpUod3kQEC4mY1F9tXMkNJ1W4YRip2nknErbYwV7ssdiEn3kA== X-Received: by 2002:a17:90b:911:: with SMTP id bo17mr5358283pjb.165.1643764395072; Tue, 01 Feb 2022 17:13:15 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:14 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 05/10] page_pool: Add slow path order 0 allocation stat Date: Tue, 1 Feb 2022 17:12:11 -0800 Message-Id: <1643764336-63864-6-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Track order 0 allocations in the slow path which cause an interaction with the buddy allocator. Signed-off-by: Joe Damato --- include/net/page_pool.h | 1 + net/core/page_pool.c | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 96949ad..ab67e86 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -144,6 +144,7 @@ struct page_pool { struct page_pool_stats { struct { u64 fast; /* fast path allocations */ + u64 slow; /* slow-path order-0 allocations */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 6f692d9..554a40e 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -308,10 +308,12 @@ static struct page *__page_pool_alloc_pages_slow(struct page_pool *pool, } /* Return last page */ - if (likely(pool->alloc.count > 0)) + if (likely(pool->alloc.count > 0)) { page = pool->alloc.cache[--pool->alloc.count]; - else + page_pool_stat_alloc_inc(slow); + } else { page = NULL; + } /* When page just alloc'ed is should/must have refcnt 1. */ return page; From patchwork Wed Feb 2 01:12:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732485 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDA4DC433FE for ; Wed, 2 Feb 2022 01:13:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243636AbiBBBNS (ORCPT ); Tue, 1 Feb 2022 20:13:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243631AbiBBBNR (ORCPT ); Tue, 1 Feb 2022 20:13:17 -0500 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79ADFC061714 for ; Tue, 1 Feb 2022 17:13:17 -0800 (PST) Received: by mail-pj1-x1031.google.com with SMTP id y5-20020a17090aca8500b001b8127e3d3aso4419155pjt.3 for ; Tue, 01 Feb 2022 17:13:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1yfjqxRjSP8MT5gsQb3DyZFYqyWlknDY7YCco0qbtPE=; b=PkPJsuhGl6ADqWE1Fco3J+iMxYdhrFNnCz3/BXwPQC7/SZ3QaHEFOWzHEyBZDYJIzW bUcaY6AquWSQtvQjJMRWr10bs5tl+XJGkGBfrxhMCypJ7IGr/SQiaz4TrUi3mJypgZrC iHBqPOyWF6WK4h95/lSeUIpieFmqMf3Le9em0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1yfjqxRjSP8MT5gsQb3DyZFYqyWlknDY7YCco0qbtPE=; b=P9D3OSPC1cQCa0HxC44LHSkbq2LHY7ajVOgLl+8Q/GsA7YN2usM6p4tZ5DKISvTkPp IV+Q/ejJI9rtfa9ndaURXYGJhyUiU8Wbr5sP2B/D/UvcjQLGTv0L2LhKdWbS7G/AXpgD MLz4Exet97mdAZ096KdyJ2gvJuTmgsj4GXMyaVr66xvcGLwGNQpdql0Y+lcwSzVFBJKs Q0++lBb8Qw+CYvFQGb+pT94eg/coLA1U9YpX3+vnwj1IYtnfSNdTjiC8WcvLNON0PxfE mh1O4W/bHcY2vSIVa0//LwV9CaZnOrRVpN0LmQtOL82Y75FOY+UstXfWqhXCvo1b3eP/ Z3bg== X-Gm-Message-State: AOAM5322OPj6Dlabs+LpQscD5xVfAWa6at3IzqRZHUJuJR0STY+RRLAO bITTwNy1ANETSPzGLvL6Xiji/lSiGQJM8pNPDHVlWS0WTrYHvvWFZpOU1wOKjQ6iF6ebNPJKteo LT1+Z4Dgzw25CWGNrpcf5/v8Q3/4531LtRFkdKZxo26/cyHQgWIUZsY6Oz3wdf0BdUhYA X-Google-Smtp-Source: ABdhPJw+h5f8g7rz+kbNVGdPQzpmQ4AHC6n9TKHh+Rrbwcsjq5k/2ZSz710TFGY8SGQlVNcvB/8TNQ== X-Received: by 2002:a17:90b:3b46:: with SMTP id ot6mr5496954pjb.179.1643764396560; Tue, 01 Feb 2022 17:13:16 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:15 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 06/10] page_pool: Add slow path high order allocation stat Date: Tue, 1 Feb 2022 17:12:12 -0800 Message-Id: <1643764336-63864-7-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Track high order allocations in the slow path which cause an interaction with the buddy allocator. Signed-off-by: Joe Damato --- include/net/page_pool.h | 1 + net/core/page_pool.c | 1 + 2 files changed, 2 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index ab67e86..f59b8a9 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -145,6 +145,7 @@ struct page_pool_stats { struct { u64 fast; /* fast path allocations */ u64 slow; /* slow-path order-0 allocations */ + u64 slow_high_order; /* slow-path high order allocations */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 554a40e..24306d6 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -254,6 +254,7 @@ static struct page *__page_pool_alloc_page_order(struct page_pool *pool, return NULL; } + page_pool_stat_alloc_inc(slow_high_order); page_pool_set_pp_info(pool, page); /* Track how many pages are held 'in-flight' */ From patchwork Wed Feb 2 01:12:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732486 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23794C433EF for ; Wed, 2 Feb 2022 01:13:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243648AbiBBBNV (ORCPT ); Tue, 1 Feb 2022 20:13:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243640AbiBBBNS (ORCPT ); Tue, 1 Feb 2022 20:13:18 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB08FC06173D for ; Tue, 1 Feb 2022 17:13:18 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id m7so1777685pjk.0 for ; Tue, 01 Feb 2022 17:13:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=EZpuFdvqL88AwGUcDLPACkayKgEAfVa9VWvyXah0H48=; b=QIQzUc6LtwNh+fcBFc5+8IA9tEO5cmzJXKydYi1vKAl6FRLug0ridTNbC7Qms6zcy4 9h/mPBXdOPsmI3RxDl50UA3ANDilBMI4lOOJL5oCZyw+rnfK/oZOSskNeyNiFrPTj+q8 A9O2MSVaJq9WXFyCvL1X32rqxYj+Ai4CcpncQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=EZpuFdvqL88AwGUcDLPACkayKgEAfVa9VWvyXah0H48=; b=vt4+AvuwrD5kGtFM3hxluOK0Jon7NqyPTvwLJB2ZJAnbbWmD9thnFlOePFAPkfxNsh A47RLzxpnyGQEFaABsfHcVXY38oKOlclf2MMcxx9WDO3BuigiH5Mjtfx/HYp26Q1gAEH f0SqXfXwIsbgXgLADrj9oRQszXvXJncYor7Af7vRf2XB3H/gOC4dlblAoJOz3cuAkhxL q4KV5vTh9pKdr1FU0DnjjMeYJb8gR0J0Ula1KVgABNh9ZMxQXPGOsruQQpJmg+xOmcDV 5IUdBxwHpHMTOGvfGXdgT1akmsT/ws6w2IiQYaU1e5MGrOC2dHWYwoD2FCZdDZzxsAG9 pFcw== X-Gm-Message-State: AOAM5311eaFI+c6V20PiAAdyYyuIslRDoeEoWYlvpJyF+mWM5V6Zx5c0 dsfCNTNw3RnNbv32/G07jjilPbN5cY89mzaT1edVvlx3vAMD6eDvMAR3fbqucZKhUH22ozp0ZYs 5dkn7lZGGw/cZ1G52vobY/ZjhDedsLqgVt+VpcTYgbKtIl4YhlxOQYKII7bo76RANCKO7 X-Google-Smtp-Source: ABdhPJyZPNeizD5oS7CtGiKdT+Jhk9gCEEIhwdSo4laSIaW6LlWzeEljUfiMzy1i+lO302xfDTisAg== X-Received: by 2002:a17:90b:1c0e:: with SMTP id oc14mr5533501pjb.2.1643764398006; Tue, 01 Feb 2022 17:13:18 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:17 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 07/10] page_pool: Add stat tracking empty ring Date: Tue, 1 Feb 2022 17:12:13 -0800 Message-Id: <1643764336-63864-8-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stat tracking when the ptr ring is empty. When this occurs, the cache could not be refilled and a slow path allocation was forced. Signed-off-by: Joe Damato --- include/net/page_pool.h | 3 +++ net/core/page_pool.c | 4 +++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index f59b8a9..ed2bc73 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -146,6 +146,9 @@ struct page_pool_stats { u64 fast; /* fast path allocations */ u64 slow; /* slow-path order-0 allocations */ u64 slow_high_order; /* slow-path high order allocations */ + u64 empty; /* failed refills due to empty ptr ring, forcing + * slow path allocation + */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 24306d6..9d20b12 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -131,8 +131,10 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) int pref_nid; /* preferred NUMA node */ /* Quicker fallback, avoid locks when ring is empty */ - if (__ptr_ring_empty(r)) + if (__ptr_ring_empty(r)) { + page_pool_stat_alloc_inc(empty); return NULL; + } /* Softirq guarantee CPU and thus NUMA node is stable. This, * assumes CPU refilling driver RX-ring will also run RX-NAPI. From patchwork Wed Feb 2 01:12:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732487 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E59BC433F5 for ; Wed, 2 Feb 2022 01:13:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243618AbiBBBNX (ORCPT ); Tue, 1 Feb 2022 20:13:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243632AbiBBBNV (ORCPT ); Tue, 1 Feb 2022 20:13:21 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F385EC06173B for ; Tue, 1 Feb 2022 17:13:20 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id cq9-20020a17090af98900b001b8262fe2d5so981155pjb.0 for ; Tue, 01 Feb 2022 17:13:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QbO+mRA437kEIBKYkoV8UATS3Q+q6DqWwFp2zvezEIo=; b=dpm3DgcxuR1zI8VIs5K1qszDWyt7wE2tMPvffPa30dfpQXAcEDIrK1ONpbxNp1kid0 DJx78AW3I57LAnJWEApspcN53qlp11xvpx0IXYteq1zpQo8HcHsPmeaA3S7MFC7UOuL6 jTd/E0u9liS3wc3a36ovrn3WoFLazJl6C3q4k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QbO+mRA437kEIBKYkoV8UATS3Q+q6DqWwFp2zvezEIo=; b=7NfNjl56ni0vHz+CUT8cO3HTWynFC7F3OY6hH656TUXofn+L8ALa6/I6ErXGvTJH6h TCYpYD6JT1k2h1KsDSfOUrBJvE31Jnl80JlcbEeDv4JGUQpeXT4W+pg6r2ALr8jOVsbe 5x891oiU7mu6iFBY7/zHoKfaIKxT6z4WngWUxi4fOyXkLRuhOsFbGNaMbkKs9BZ/vga3 AUVqcKg5xADdiEz8xV3HcE3LZ0lOi7BCFiX6Gu13NfO2xqjlqB8NwHHk6C9yJyD3/P70 TrXHI9gkYKYzV27lHUWzdKZjjmvlHtSRnBNj0uqkHsHeSRWypc46HApNghZIOMnIpdsJ CBdQ== X-Gm-Message-State: AOAM530AuBRmHMhYjqyl2sgKO4sPYOKF9Ugw91LD9Vr4qrrsQHpixlnO mMEJQnunUYqn11jhYUYNY179ZPD5aAZF601X0cJ1SQnWiucNUzi0qw8CGJT4dYTNLJIYzklmUbx 9/vgrGM8J6lOir76VJXIZbxtp19xsg6Z3wcZSQizfIm9HnMcfyvAr+HAe8hffTpWOlFa5 X-Google-Smtp-Source: ABdhPJzKRxstrcnVBtdAMV43S7UcCalL1mvfxdxXdUxF0RGhH8ymQD1O0ga+57kH0oG+LPlMG3knuw== X-Received: by 2002:a17:902:8504:: with SMTP id bj4mr28919966plb.108.1643764399466; Tue, 01 Feb 2022 17:13:19 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:18 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 08/10] page_pool: Add stat tracking cache refill Date: Tue, 1 Feb 2022 17:12:14 -0800 Message-Id: <1643764336-63864-9-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stat tracking succesfull allocations which triggered a refill. Signed-off-by: Joe Damato --- include/net/page_pool.h | 1 + net/core/page_pool.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index ed2bc73..4991109 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -149,6 +149,7 @@ struct page_pool_stats { u64 empty; /* failed refills due to empty ptr ring, forcing * slow path allocation */ + u64 refill; /* allocations via successful refill */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 9d20b12..00adab5 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -167,8 +167,10 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) } while (pool->alloc.count < PP_ALLOC_CACHE_REFILL); /* Return last page */ - if (likely(pool->alloc.count > 0)) + if (likely(pool->alloc.count > 0)) { page = pool->alloc.cache[--pool->alloc.count]; + page_pool_stat_alloc_inc(refill); + } return page; } From patchwork Wed Feb 2 01:12:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732488 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02ED7C433FE for ; Wed, 2 Feb 2022 01:13:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243645AbiBBBNX (ORCPT ); Tue, 1 Feb 2022 20:13:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32892 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243652AbiBBBNW (ORCPT ); Tue, 1 Feb 2022 20:13:22 -0500 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E20EAC06173D for ; Tue, 1 Feb 2022 17:13:21 -0800 (PST) Received: by mail-pl1-x62b.google.com with SMTP id k17so16942748plk.0 for ; Tue, 01 Feb 2022 17:13:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=C8VUXPM7TXe0m5OvMX/WoLxUo7hUGvGMBbngPBOXhII=; b=w/essKQEqLTCNunGkh/PMXbvgmvgD9H2PdbruyRxvERCKgoASTDsdNObxydRAs2szn 28f7oCBKpgbUG2/ZOTO3Pn6eMFwHdFP910fzF4NeaPwNaOLhwp9d8ZTYJnlGaOj2FIch iKlQinKt4xhWcebluzW1fPge8soWC6cYoj5K0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=C8VUXPM7TXe0m5OvMX/WoLxUo7hUGvGMBbngPBOXhII=; b=Xs/mgwgCOs/EvRrhFcNeyBny+W8Vc0+sev20Gvbmys/ayOeYWbsgNcqgomP8e/ZDXO o5dAf1epk5QWG2ounFarYtjsH+Q3hk10E/0R2xntiLwfzN1st64eqD2Q/BjAmHwrXLdU NM0HabYpbmbGj+rpKxyejbnYWhiRmLukUERQLDLEzhMN7ShFA2aPed1ZT5C9xrDxUGRM P/TdtNV1LoEs86H2RWDpFCps4375W7USCdkR1hq4ELJ4ySyMtjRGffDBFNYhu3hQTya8 2LtrV1JlzszpZg5aaETwgk5xpOMiIZcAJQbxji014Zjfmy2pewcLUZ4jMvPE5BuQlXtR DD9g== X-Gm-Message-State: AOAM531lQ14wPkHk0LTbKfk2WhNcgX8gWpY9qeyqnc1bSnc/T754ZfVc /9KnYYVkowFYZFtYKX3OKf5e/npU8lgzpVnN/XGYarZ/lJt9Sk7uWej4lNvL6WBen7zJOsFockB aIZpQZcZk26GWRPMgoVqTzfY/MWLqzbddTIzLg+U9YP0brq5BBVkrTDUFcMAI5QZZijCN X-Google-Smtp-Source: ABdhPJxWOb8bUbV/7upSo7J0YgGpVuJVkVBQu/aIAqo+Jjnpra2ju7SqcaXwEMRc/+F5Rzuc37YMqg== X-Received: by 2002:a17:90b:1a92:: with SMTP id ng18mr5441566pjb.139.1643764401017; Tue, 01 Feb 2022 17:13:21 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:20 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 09/10] page_pool: Add a stat tracking waived pages Date: Tue, 1 Feb 2022 17:12:15 -0800 Message-Id: <1643764336-63864-10-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Track how often pages obtained from the ring cannot be added to the cache because of a NUMA mismatch. Signed-off-by: Joe Damato --- include/net/page_pool.h | 1 + net/core/page_pool.c | 1 + 2 files changed, 2 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 4991109..e411ef6 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -150,6 +150,7 @@ struct page_pool_stats { * slow path allocation */ u64 refill; /* allocations via successful refill */ + u64 waive; /* failed refills due to numa zone mismatch */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 00adab5..41725e3 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -161,6 +161,7 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) * This limit stress on page buddy alloactor. */ page_pool_return_page(pool, page); + page_pool_stat_alloc_inc(waive); page = NULL; break; } From patchwork Wed Feb 2 01:12:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12732489 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 348F6C433F5 for ; Wed, 2 Feb 2022 01:13:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243639AbiBBBNZ (ORCPT ); Tue, 1 Feb 2022 20:13:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243646AbiBBBNX (ORCPT ); Tue, 1 Feb 2022 20:13:23 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 915D4C06173E for ; Tue, 1 Feb 2022 17:13:23 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id a19so11455826pfx.4 for ; Tue, 01 Feb 2022 17:13:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=iQhK3QJDPOtx3ex2lFMNHo4XnsW4IlmYbsO1LGXKA2k=; b=OE+S8yOtgvhE7N+F8yc6DaUog1MmceemA18bRckGOGcXxxFpSXqKewt4HwhP1JaKbo hxHxF7epWxICw6FfBttBZaHVcafu2M57BcpoCn1DpCf2JX5UiHEq5gQ+/erVjEhPdFkA N3iB7AUhkvKKsHHjYZOZE86GwN6+qwCe/JS+c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=iQhK3QJDPOtx3ex2lFMNHo4XnsW4IlmYbsO1LGXKA2k=; b=W+k4/wdFVUbY2CZqUNaf3Z+mRbgzmBIQz3NQVf//YBKkDTxfSWI1NDVhNJkzmnlmIW eSq0DE/jfXw+4zjht64bcT4g8n5+tVqf7efy/IN29hDDeqIkUIsu+TAmsCUe83ZjTs1d ZL8KbK93wMbfbqgTrkq0J4XMMZHhcB4lDYIHczCqsXQq2v1KvSbYLstpFfhd/kK+LzLP dqgGKFWePt05MzE2F1cR29Gi2RW5tnAyIgpNNjRiFYD3fX42yJJH6bqUQNyG0cN7ahrq 9ouoGGD2tbetrK1watpKUgwi7uz07TeLy44yjdKRMtZbBEte8lLZ5tYoj+TFe+EYPZ3e C/pw== X-Gm-Message-State: AOAM532873Ez2RyucJbzl19J3cAsvrNwH5qSSY1l8j8GkAkko81P1r+l fdI7dlZ/paLRKhvKGGAI41niMuV1k/JPCdtXJgLLFB30CE/e6M6WvMH7s8NU90Yjx7bDMNYDQcv 0Ti2CWfSVqee8k1P4lSLVrV/QfRg6RvoomyXomrWXAEwLN4XCesV8Zjk4Srfq5a3PVYxA X-Google-Smtp-Source: ABdhPJyV2jjFIZKgCqgZcpXcGkB0kVhb4DzhXobw81O7ScaTNiQb1P0z4/F2+EzozsW34oCRe5kYLg== X-Received: by 2002:a63:396:: with SMTP id 144mr22546262pgd.288.1643764402570; Tue, 01 Feb 2022 17:13:22 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id a125sm15170025pfa.205.2022.02.01.17.13.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Feb 2022 17:13:21 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v3 10/10] net-procfs: Show page pool stats in proc Date: Tue, 1 Feb 2022 17:12:16 -0800 Message-Id: <1643764336-63864-11-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643764336-63864-1-git-send-email-jdamato@fastly.com> References: <1643764336-63864-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Per-cpu page pool allocation stats are exported in the file /proc/net/page_pool_stat allowing users to better understand the interaction between their drivers and kernel memory allocation. Signed-off-by: Joe Damato Reported-by: kernel test robot --- net/core/net-procfs.c | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/net/core/net-procfs.c b/net/core/net-procfs.c index 88cc0ad..3bc6e53 100644 --- a/net/core/net-procfs.c +++ b/net/core/net-procfs.c @@ -4,6 +4,10 @@ #include #include +#ifdef CONFIG_PAGE_POOL_STATS +#include +#endif + #define BUCKET_SPACE (32 - NETDEV_HASHBITS - 1) #define get_bucket(x) ((x) >> BUCKET_SPACE) @@ -310,6 +314,57 @@ static const struct seq_operations ptype_seq_ops = { .show = ptype_seq_show, }; +#ifdef CONFIG_PAGE_POOL_STATS +static struct page_pool_stats *page_pool_stat_get_online(loff_t *pos) +{ + struct page_pool_stats *pp_stat = NULL; + + while (*pos < nr_cpu_ids) { + if (cpu_online(*pos)) { + pp_stat = per_cpu_ptr(&page_pool_stats, *pos); + break; + } + + ++*pos; + } + + return pp_stat; +} + +static void *page_pool_seq_start(struct seq_file *seq, loff_t *pos) +{ + return page_pool_stat_get_online(pos); +} + +static void *page_pool_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + ++*pos; + return page_pool_stat_get_online(pos); +} + +static void page_pool_seq_stop(struct seq_file *seq, void *v) +{ +} + +static int page_pool_seq_show(struct seq_file *seq, void *v) +{ + struct page_pool_stats *pp_stat = v; + + seq_printf(seq, "%08llx %08llx %08llx %08llx %08llx %08llx %08llx\n", + seq->index, pp_stat->alloc.fast, + pp_stat->alloc.slow, pp_stat->alloc.slow_high_order, + pp_stat->alloc.empty, pp_stat->alloc.refill, pp_stat->alloc.waive); + return 0; +} + +static const struct seq_operations page_pool_seq_ops = { + .start = page_pool_seq_start, + .next = page_pool_seq_next, + .stop = page_pool_seq_stop, + .show = page_pool_seq_show, +}; +#endif + static int __net_init dev_proc_net_init(struct net *net) { int rc = -ENOMEM; @@ -326,6 +381,15 @@ static int __net_init dev_proc_net_init(struct net *net) if (wext_proc_init(net)) goto out_ptype; + +#ifdef CONFIG_PAGE_POOL_STATS + if (!proc_create_seq("page_pool_stat", 0444, net->proc_net, + &page_pool_seq_ops)) { + wext_proc_exit(net); + goto out_ptype; + } +#endif + rc = 0; out: return rc; @@ -342,6 +406,9 @@ static void __net_exit dev_proc_net_exit(struct net *net) { wext_proc_exit(net); +#ifdef CONFIG_PAGE_POOL_STATS + remove_proc_entry("page_pool_stat", net->proc_net); +#endif remove_proc_entry("ptype", net->proc_net); remove_proc_entry("softnet_stat", net->proc_net); remove_proc_entry("dev", net->proc_net);