From patchwork Sat Jan 29 23:38:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729691 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E3DBC433F5 for ; Sat, 29 Jan 2022 23:40:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353437AbiA2XkG (ORCPT ); Sat, 29 Jan 2022 18:40:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238951AbiA2XkF (ORCPT ); Sat, 29 Jan 2022 18:40:05 -0500 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41D66C061714 for ; Sat, 29 Jan 2022 15:40:05 -0800 (PST) Received: by mail-pg1-x532.google.com with SMTP id t32so8638947pgm.7 for ; Sat, 29 Jan 2022 15:40:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kAXoMksJwSvCPGdPeM29rSArb1Q2ozVaNB51iTgnTDA=; b=r884UGDy9y8Xn5nmGJVhyXu1J52ewolzs/A9nOBWnD2Op70O2KR0lSLb8bQsTFrZfZ +MsdmluQdRhguW1kggl8JfdUWF5S0psH0JgXoGJ4f3SsZf0j8ndoHirwwRWD6MjMwuE9 9xAa2SgB7DufOlEtsw235rTsV54m3Cy/cl58w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kAXoMksJwSvCPGdPeM29rSArb1Q2ozVaNB51iTgnTDA=; b=wj3Irefc6amrYAKWg6vln8hSOYYgCv2BziSTe6xnfg3rD9ogQ31BEy4xtYkxRhXFDT p4qjIFpwTnvIPD20iqO4zTI8NM6gb53t9mlYPbCEgisjMFwPqJv5XYnfywratrzu263z n2ps0P6pX8dk1uhdv6yOaC7JGKhbAuBgTpoz0JK9jVUxrWJCQoPeMHVqhOhnfVFtQYyu GshO8UDZbhWuJmoNNX3b84rmw4du0Q9lVk6XV3bUbv3s9fDSzFiY7bDkp7MUzdpSQQu3 OVFAIHeky/h9f/OKglXNB5vdS89qpe5NdhUGjoCktpenLNeHnQktA7o8wuRAQX1i/c/I yUAA== X-Gm-Message-State: AOAM530wsQ5v11oMvqkbwOcOA1JC9xoIrvrUZtUKWaZVcktgEBlfZDfY 63/f6uDHJSNGR+wx7obVl5AzHX5ezxDbiyB4/u8OSXU5Ulc3hQo3Yr2ORPeLfdeXL8S0UPyc6mR TytTS2xHkM7Xtt3Ik+RU7Q45cuWgaXPP65DVv9kBFwVRePKnhL8psZpAA2WR/KB8a6C29 X-Google-Smtp-Source: ABdhPJwQgMQKv6eZG9AtKv3Ce4SLWw6KkoH8JcQsdkmtDZeuF4Qq242M71oQT7CJw0dzRQ3k/HVKIw== X-Received: by 2002:a63:d318:: with SMTP id b24mr11789629pgg.255.1643499604337; Sat, 29 Jan 2022 15:40:04 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:03 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 01/10] page_pool: kconfig: Add flag for page pool stats Date: Sat, 29 Jan 2022 15:38:51 -0800 Message-Id: <1643499540-8351-2-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Control enabling / disabling page_pool_stats with a kernel config option. Option is defaulted to N. Signed-off-by: Joe Damato --- net/Kconfig | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/net/Kconfig b/net/Kconfig index 8a1f9d0..feeca42 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -434,6 +434,18 @@ config NET_DEVLINK config PAGE_POOL bool +config PAGE_POOL_STATS + default n + bool "Page pool stats" + depends on PAGE_POOL + help + Enable page pool statistics to track allocations. Stats are exported + to the file /proc/net/page_pool_stat. Users can examine these + stats to better understand how their drivers and the kernel's + page allocator, and the page pool interact with each other. + + If unsure, say N. + config FAILOVER tristate "Generic failover module" help From patchwork Sat Jan 29 23:38:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729692 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D375C433FE for ; Sat, 29 Jan 2022 23:40:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353449AbiA2XkH (ORCPT ); Sat, 29 Jan 2022 18:40:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353443AbiA2XkH (ORCPT ); Sat, 29 Jan 2022 18:40:07 -0500 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC015C06173B for ; Sat, 29 Jan 2022 15:40:06 -0800 (PST) Received: by mail-pg1-x531.google.com with SMTP id p125so8661258pga.2 for ; Sat, 29 Jan 2022 15:40:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5MPwYILWWDHRKPtSihChctU6rXVQMQ9CjUeZyzwHNXA=; b=FF74KuP08CXjsoy+nwd8IuDTjtAzGz2lcfDLbKkYLXFTjUQnWS3XZNt7xtEO0nhJme JktM6RLqT3qJhrdEp4JD8BPCSK1eNNUJGlk2bRBi+7Bu+8HPgoznnNynsdppWzWgYN3V MSo8Tc+/AkLhXkOQ2XdrUeZjPwKyjmU5t8OYY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5MPwYILWWDHRKPtSihChctU6rXVQMQ9CjUeZyzwHNXA=; b=OJM12PwDfdlvTlpBQRKrd9I+jtKn3xKdkdcFr0HxrpmQ5I32TJj/yHDDDsSVAyylFp +Gjh16LUUw6UQqOU1q3E48W/CSmqBouE16+C/DVFw2ttWc61pS2OPk9ukX9TjYRizmuv I72x30VBwiCL2Nf+LTMUzqwr0LoaIL2nnvg8Vye6B65VaCyXmaEV1bRGGUOh8HN7aanh WaGzUPXSr99Yq0LcXHYgWML2lmGpjt3VA4ouS3f9GssspMA5ELEZgJQ5DgSVklhjGFdF 59LIhUaT4kYWpjsQ5xpQI3K29ftBFcov6dh/2iCH469Ymp0GoWBYlJ0yeCiQF2WgDci+ GD4Q== X-Gm-Message-State: AOAM531hukZcRceBQoHCNOOUsR7yKWZy2o/dSqaSYIDkZrEUwYO09xz1 IVo/B/f3vJyx0aJdz4RlTk9qKSh/8ZEc4X6jlSwHbTlZQphiTnIniGQa2DsmKZ79SeCylAdgMwS ohSlYR1r4iDGNZAxcUOj84mme5jUDrbDgMbPR4xZnMA5Ulnzm/g7kDGu48+JjvaJIFzMD X-Google-Smtp-Source: ABdhPJwB3ft/EOwfj8qKzPdlMmZVcrfY+NK0gI9R9YloOj8NDKlepveymhxqeONPKpx2Jh+j4l4GBw== X-Received: by 2002:a63:512:: with SMTP id 18mr11807111pgf.432.1643499605845; Sat, 29 Jan 2022 15:40:05 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:05 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 02/10] page_pool: Add per-cpu page_pool_stats struct Date: Sat, 29 Jan 2022 15:38:52 -0800 Message-Id: <1643499540-8351-3-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org A per-cpu (empty) page_pool_stats struct has been added as a place holder. Signed-off-by: Joe Damato --- include/net/page_pool.h | 10 ++++++++++ net/core/page_pool.c | 5 +++++ 2 files changed, 15 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 79a8055..dae65f2 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -137,6 +137,16 @@ struct page_pool { u64 destroy_cnt; }; +#ifdef CONFIG_PAGE_POOL_STATS +/* + * stats for tracking page_pool events. + */ +struct page_pool_stats { +}; + +DECLARE_PER_CPU_ALIGNED(struct page_pool_stats, page_pool_stats); +#endif + struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp); static inline struct page *page_pool_dev_alloc_pages(struct page_pool *pool) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index bd62c01..7e33590 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -26,6 +26,11 @@ #define BIAS_MAX LONG_MAX +#ifdef CONFIG_PAGE_POOL_STATS +DEFINE_PER_CPU_ALIGNED(struct page_pool_stats, page_pool_stats); +EXPORT_PER_CPU_SYMBOL(page_pool_stats); +#endif + static int page_pool_init(struct page_pool *pool, const struct page_pool_params *params) { From patchwork Sat Jan 29 23:38:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729694 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFA13C433EF for ; Sat, 29 Jan 2022 23:40:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353445AbiA2XkK (ORCPT ); Sat, 29 Jan 2022 18:40:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353443AbiA2XkI (ORCPT ); Sat, 29 Jan 2022 18:40:08 -0500 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DB5DC061714 for ; Sat, 29 Jan 2022 15:40:08 -0800 (PST) Received: by mail-pg1-x52d.google.com with SMTP id h23so8629751pgk.11 for ; Sat, 29 Jan 2022 15:40:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=daCp29CfHqba5AzFqLjgXp71o/Cmax20BodJH4eahfI=; b=f15XspafqxG6ESLY/xvakwMwGbcXQLArGWMHhs+0zRzmEqQ+wvYZ60nTQKCxu0b3Yo wNwayIyU0GeGzYHV6G1YiGLtSi5xMlc3ciZTFHa7i27XYRyKCXUI8Jzlmt7SXjTN8xBp DSw5LpL2ujIf85D0KfePlcqkioeyirbWahiCE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=daCp29CfHqba5AzFqLjgXp71o/Cmax20BodJH4eahfI=; b=haJsczvC5ZqqefreGzV4GkZRPcpu6aoqJN/rj1bNPfzA0Ocb9l6tsBJonLYwKQ/p43 dsFpzNGXSPB2dTU8YyBJ2xlZWSSVaoWNNIMyoTl5aZEai1rNxL5w1QpFFE/QewmSeHMT fnsEn2t+7sd5GaGKR/X56bu729naIZslMezh7ZAgta5RB/6yirIncWxqm47shHgv+ERn aG5GJ92U5o7l0xvn/5ZLuxj0DUYRKgAlvpFwxNtHvmQcNpnyCXekxJKWqXLJ2z1+KFkP FOPtlQa+fa7FhTDlbg0gTw0YeMHIj+OaoWAlNWpsEZbfwUz//2KQqKfz9NjSONEaKscm gkpg== X-Gm-Message-State: AOAM533j3W8QIFgG4t4+cEXt/QmXLDNJUTHDF/5twuO6MQZzLi4F6H2C Pu4N/txvrUYgrt06oVBz+1nkzP0lNBQB8XCqlOS9pcIp7BVPapKCgZiC6Dzcii1U2kWERNrj9sO e7qoJWMshu/2e4hK47pYRg/at/Hbap1Zj+K0cjNRN6UAJnjr4T1pb4PXRRU6f77jUWOlQ X-Google-Smtp-Source: ABdhPJyb1kC2ExfMUhCrIFjZL0BS6l4hG6Iy9nNY4joqBB+tMMvPQTLn4T5Rv0m6/kXUhLyCs1BAZQ== X-Received: by 2002:a63:1f12:: with SMTP id f18mr11738293pgf.426.1643499607369; Sat, 29 Jan 2022 15:40:07 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:06 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 03/10] page_pool: Add a macro for incrementing stats Date: Sat, 29 Jan 2022 15:38:53 -0800 Message-Id: <1643499540-8351-4-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add simple wrapper macro for incrementing page pool stats. This wrapper is intended to be used in softirq context. Signed-off-by: Joe Damato --- net/core/page_pool.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 7e33590..b1a2599 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -29,6 +29,15 @@ #ifdef CONFIG_PAGE_POOL_STATS DEFINE_PER_CPU_ALIGNED(struct page_pool_stats, page_pool_stats); EXPORT_PER_CPU_SYMBOL(page_pool_stats); + +#define page_pool_stat_alloc_inc(__stat) \ + do { \ + struct page_pool_stats *pps = this_cpu_ptr(&page_pool_stats); \ + pps->alloc.__stat++; \ + } while (0) + +#else +#define page_pool_stat_alloc_inc(stat) #endif static int page_pool_init(struct page_pool *pool, From patchwork Sat Jan 29 23:38:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729693 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C556EC4332F for ; Sat, 29 Jan 2022 23:40:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353454AbiA2XkL (ORCPT ); Sat, 29 Jan 2022 18:40:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353451AbiA2XkK (ORCPT ); Sat, 29 Jan 2022 18:40:10 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4175C061741 for ; Sat, 29 Jan 2022 15:40:09 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id oa14-20020a17090b1bce00b001b61aed4a03so8673342pjb.5 for ; Sat, 29 Jan 2022 15:40:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5DfdEyld7StyWrFrpQh0H8+Y2uMglsDtEWNNoHSX1Uo=; b=gaESIkclZWnWP67b8XocjXdfrgEqYGcIF2PUxbuovu52gXsAU+ol5/vJg+wA4PBH7s QaUxhStioE9ukZgotQndKH1jQnYZ/xxu4W/BhZg2oiBQVhXB+ZKX+ck12Lhe58m5AQcu g+Po1IWMOTwGUirdyLm6TDz44oPQlrkXFYPu8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5DfdEyld7StyWrFrpQh0H8+Y2uMglsDtEWNNoHSX1Uo=; b=kRmY2YP/lk2HHryoxku/i29vSlg/lTHVa0IcZEbdEyeCgE7Ucda1M3bWtWA7Of6UbG mIpj62btUjx6fAAweQ/MsfhUtKOSB+o3I1uuDPMpuIwqNDusYkd0yGzJYtptk7YzwQ2e r2vw6LaVvozUd+9+2DLFcHQYlr0DaxxExR6+ZhRLmbI3/nHdCN7hSdNI7YPEc/259GzO UtzcW0GK4UVHmk38sLjRPARXKerDcsc+sWN59ckPfEdHBId1OUuUv3UmeYI+Z7LCQDX1 r4KgPxgpYwKEzYA7ZegZvPBw4wK54mT7Ef/tGynUggDyKjSQEi3x6ex8MFNy8hZoz3pE mdRA== X-Gm-Message-State: AOAM530Dr7qAmW6rlJ+RtPJdDMj2eb2+hJZBk6w0f1SNEvkmBq6nsdDD U9DMUqvKCKHJZIqnQrOKXVUMf5mFaDy/7C/IJgvVTpJOvS5f44t4XNf/mssBAd5zBsnpaMnUU+q TihbGWTZUVOVhFeGVUuQXS34CnwUFLPTiv63pTTejBx+SZtn/nbj1KY2peNfTb7uHIqbO X-Google-Smtp-Source: ABdhPJz1xIttczSbe4Nq5We3OJoNLXyut6taib8t5UQ+uAKlGeBCVNANLSaXqSaINNAKYO2QQ+Tsnw== X-Received: by 2002:a17:90a:601:: with SMTP id j1mr27292526pjj.192.1643499608831; Sat, 29 Jan 2022 15:40:08 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:08 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 04/10] page_pool: Add stat tracking fast path allocations Date: Sat, 29 Jan 2022 15:38:54 -0800 Message-Id: <1643499540-8351-5-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a counter to track successful fast-path allocations. Signed-off-by: Joe Damato --- include/net/page_pool.h | 3 +++ net/core/page_pool.c | 1 + 2 files changed, 4 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index dae65f2..96949ad 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -142,6 +142,9 @@ struct page_pool { * stats for tracking page_pool events. */ struct page_pool_stats { + struct { + u64 fast; /* fast path allocations */ + } alloc; }; DECLARE_PER_CPU_ALIGNED(struct page_pool_stats, page_pool_stats); diff --git a/net/core/page_pool.c b/net/core/page_pool.c index b1a2599..6f692d9 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -180,6 +180,7 @@ static struct page *__page_pool_get_cached(struct page_pool *pool) if (likely(pool->alloc.count)) { /* Fast-path */ page = pool->alloc.cache[--pool->alloc.count]; + page_pool_stat_alloc_inc(fast); } else { page = page_pool_refill_alloc_cache(pool); } From patchwork Sat Jan 29 23:38:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729695 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70B0CC433F5 for ; Sat, 29 Jan 2022 23:40:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353459AbiA2XkN (ORCPT ); Sat, 29 Jan 2022 18:40:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353450AbiA2XkL (ORCPT ); Sat, 29 Jan 2022 18:40:11 -0500 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EE25C061714 for ; Sat, 29 Jan 2022 15:40:11 -0800 (PST) Received: by mail-pj1-x1036.google.com with SMTP id s2-20020a17090ad48200b001b501977b23so14482226pju.2 for ; Sat, 29 Jan 2022 15:40:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NTd7TBI6abPTmN3I9QjdcpJvF3jv9TVetLryoOk/zcc=; b=l6+WAZ1J1v9m0go2oG3ydFUB9vs2JlU+N8bbFq7akudwLtdCKuS4HFcjwrOUFUbUGm 1d7khlrkm7ok3VivLRe5osmCuMedNF4vYI5xROoly71XiVEwbZnA6Dsw2HFJDvNi07xg hsWHE6Tw9EumLJWae+B7Ib59vsUK7w8xzeIc8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NTd7TBI6abPTmN3I9QjdcpJvF3jv9TVetLryoOk/zcc=; b=eyH92HSNVlSmyzCn2ma84SQy+3YmON/3pgaD4rUjtNi8AZgiU+AACpID5w7zae8g3Y 7kN5MfJd4FY98GEupK3/sbOEqmzgD35wGfAFcBRRXvuwVv0tmcHQ0HhR0OQ2WIxFxldw 1HfnLYOTZiz9h6lSEevn5GgNh7BJqC91AY1YamHKtdl/oAU4FKOq8+wfNvnE7GUc1BLA wzB8Z0KwPrJAIMdgvIk+pdvjGMHAuboPwux7vrslnGgN77rBIY/eTnwyZuBAdn/3XCFZ mZ5FWKD/eFb49eaWthpdS4+h/h3QxOycmElmHLqjLUF8LbfB1o6IvHiR99K9LAHPFGtp kb2A== X-Gm-Message-State: AOAM532b79dDFyl4ulhd1UgtxcBOmx0zZ3u8eRqo0UKBFgyw9fpTTTTZ 4MBOrbVfCHSKsXDmOThBmM9LrpIhPc6gQ8RuL8DpeO4apQQlwUAplfK/pXkxSuQwKlvJ5l7uW69 JWxUv/H8YH0ozM9Vl8XScCfxBeGzApRYcbKW6Ckc87DcLzIfO3N+zfhiwh6dyy+f67YsE X-Google-Smtp-Source: ABdhPJxBS71uCTS/7KDQjnP5KQLaQVhJ8bFMbshYbSP4uMBa84i8924bTYJkDp19EXyWPkWse5CvJQ== X-Received: by 2002:a17:90b:4b88:: with SMTP id lr8mr27032837pjb.166.1643499610430; Sat, 29 Jan 2022 15:40:10 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:09 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 05/10] page_pool: Add slow path order 0 allocation stat Date: Sat, 29 Jan 2022 15:38:55 -0800 Message-Id: <1643499540-8351-6-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Track order 0 allocations in the slow path which cause an interaction with the buddy allocator. Signed-off-by: Joe Damato --- include/net/page_pool.h | 1 + net/core/page_pool.c | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 96949ad..ab67e86 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -144,6 +144,7 @@ struct page_pool { struct page_pool_stats { struct { u64 fast; /* fast path allocations */ + u64 slow; /* slow-path order-0 allocations */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 6f692d9..554a40e 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -308,10 +308,12 @@ static struct page *__page_pool_alloc_pages_slow(struct page_pool *pool, } /* Return last page */ - if (likely(pool->alloc.count > 0)) + if (likely(pool->alloc.count > 0)) { page = pool->alloc.cache[--pool->alloc.count]; - else + page_pool_stat_alloc_inc(slow); + } else { page = NULL; + } /* When page just alloc'ed is should/must have refcnt 1. */ return page; From patchwork Sat Jan 29 23:38:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729696 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55FDEC433F5 for ; Sat, 29 Jan 2022 23:40:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353455AbiA2XkS (ORCPT ); Sat, 29 Jan 2022 18:40:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353457AbiA2XkM (ORCPT ); Sat, 29 Jan 2022 18:40:12 -0500 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5DFDC061714 for ; Sat, 29 Jan 2022 15:40:12 -0800 (PST) Received: by mail-pj1-x1036.google.com with SMTP id s61-20020a17090a69c300b001b4d0427ea2so14461116pjj.4 for ; Sat, 29 Jan 2022 15:40:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1yfjqxRjSP8MT5gsQb3DyZFYqyWlknDY7YCco0qbtPE=; b=Jld+fVN1MmC8B9Y5uVQxx943tmXn52z1JrUV/yXWVc+VefgOvQZPDYvJUICFlH+wFl J1LciNCZ0psL3O9j2/hZzA99hDjec5V3piqZm8xHYriqI4AtowmKDNil/i9oSLBC5wry NCo/2jkaW3GpsAEawemNQSDl8JKxVvbZYz6vA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1yfjqxRjSP8MT5gsQb3DyZFYqyWlknDY7YCco0qbtPE=; b=3B1eJ9QR2i/XSxzc6qB6Wyf0wuiBGtqdUL7ldYInvSYjwTK7YVZyTTYE52cY0xCScE NwLof3zkwFfU/Hhow9hQcuU+WYVzTAeddhHnrVILbdNsk4g/5qjzltvmGPshi11wO4gj RZkcR1k8EEvxdpkmddmouGlnsN6dmahfqReE7J/CTS9ys4FMMZX2S4WNiEJxrUGV8x31 ylKR+beHItvJ5X36syMOYu9ggCQ8IuDqIIQzKsDuX2rv4YdVAIGwaGPDF4nttuOmvxDs VEVwP0GnZaGKdzhmr5ZYCSX4vLy98OlPwoeU9qQIpuvCllpAsljWBiXGMZBr9LEfY11G oVKA== X-Gm-Message-State: AOAM5338yXQ0MEj2ISzX/vT1y3dyries06DRAerV2HeNdnWTV7OIJ8Mp 7NVl3OJRJxLrHDuwV/dX0q1LocIUWUQneYM3OKFbUlKqaqy64flNQVtPfq0oY5wukUzkIYTc3iF Xgz+EKbc45Au8ksoKx+F3IWKwnKox1w+FSEEPN+it5JvCqmZmYDc4vEWhMh2AIb9dHnUu X-Google-Smtp-Source: ABdhPJyMIGYD3PAkmLbq6qgfhvV51QSJBYVUyZee9IFRKYlCH9Qmq7bz1YMvr4ASzRZWap4xd/2G8w== X-Received: by 2002:a17:90b:2243:: with SMTP id hk3mr21308254pjb.181.1643499611787; Sat, 29 Jan 2022 15:40:11 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:11 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 06/10] page_pool: Add slow path high order allocation stat Date: Sat, 29 Jan 2022 15:38:56 -0800 Message-Id: <1643499540-8351-7-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Track high order allocations in the slow path which cause an interaction with the buddy allocator. Signed-off-by: Joe Damato --- include/net/page_pool.h | 1 + net/core/page_pool.c | 1 + 2 files changed, 2 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index ab67e86..f59b8a9 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -145,6 +145,7 @@ struct page_pool_stats { struct { u64 fast; /* fast path allocations */ u64 slow; /* slow-path order-0 allocations */ + u64 slow_high_order; /* slow-path high order allocations */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 554a40e..24306d6 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -254,6 +254,7 @@ static struct page *__page_pool_alloc_page_order(struct page_pool *pool, return NULL; } + page_pool_stat_alloc_inc(slow_high_order); page_pool_set_pp_info(pool, page); /* Track how many pages are held 'in-flight' */ From patchwork Sat Jan 29 23:38:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729697 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B11E9C433EF for ; Sat, 29 Jan 2022 23:40:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353466AbiA2XkT (ORCPT ); Sat, 29 Jan 2022 18:40:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353464AbiA2XkO (ORCPT ); Sat, 29 Jan 2022 18:40:14 -0500 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07450C06173B for ; Sat, 29 Jan 2022 15:40:14 -0800 (PST) Received: by mail-pg1-x533.google.com with SMTP id j10so8656211pgc.6 for ; Sat, 29 Jan 2022 15:40:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=EZpuFdvqL88AwGUcDLPACkayKgEAfVa9VWvyXah0H48=; b=l1dSfwiMGeC8i57oJdo5sV1jtrGods/IFM2x58eErut76zl2LT5HnS74V2ceFHxBI1 HtqLXXuJGzAx3kWnDpVF95N7U1FX/P1p5D8kXi/Arz6TS3+V6Rd5b9DgqvUN7PagDnFD PiWXxiInrYuGTZ8TGPY50NPI6kva5YEi5qhkA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=EZpuFdvqL88AwGUcDLPACkayKgEAfVa9VWvyXah0H48=; b=CyecRe2KhbjBtu1je/MiKNS4POLbZ+jUS+GTKFsNsNB5BjQIsic5eCqcwSGkXJU6gz VcekoEVRkHzwjeHOKUjZd5x6kvenkl66g14j8mHFlRvdWEZtSx/pTbukDIai7gCBHHHO nQ2Toj0RkLRuK8Df7lTxyVLRcNFy90lHqRJvf/SdZwrk17/WmlNvo22F1S9Oc0pNAxeU J9ZUzrWHucfM6yKG8dJ5ALZIStZLFmCdevtGYIZjFtZWQLJb5IMXoKc92nNjvahMAbGa 4p1/C4QUBfriW5yWyLtH2KFSp7mZ5yEV9d4z/phxLwRDNoPazT6hK6GiY518AIqff7r2 8VMQ== X-Gm-Message-State: AOAM531AUsm1oFTwRk0oBxkXaD6a8Q9qcsoJ6j1PU6jYILPYh1vbYzxN ptnKOJDOG+Yj+i7aswDjQkF8jCKRa14M88dRlEEpbUs1wrGR/IIL+HDNo/Op72co/LwG7vXlLLa Y2Wst6zbvlttZsX7QGUy/pGW8S3hWlVAPGAICwSa1UEA6/Aqa/xD6HZLyMeIUa0VCtyJ9 X-Google-Smtp-Source: ABdhPJyAo5kCL4EfYhPn480Jf+Q63EvJGRNM45zQGtK7JOm/a/9HkvP3BwvKezIMg8gyJvIgoMyH/Q== X-Received: by 2002:a63:1641:: with SMTP id 1mr11492139pgw.396.1643499613165; Sat, 29 Jan 2022 15:40:13 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:12 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 07/10] page_pool: Add stat tracking empty ring Date: Sat, 29 Jan 2022 15:38:57 -0800 Message-Id: <1643499540-8351-8-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stat tracking when the ptr ring is empty. When this occurs, the cache could not be refilled and a slow path allocation was forced. Signed-off-by: Joe Damato --- include/net/page_pool.h | 3 +++ net/core/page_pool.c | 4 +++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index f59b8a9..ed2bc73 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -146,6 +146,9 @@ struct page_pool_stats { u64 fast; /* fast path allocations */ u64 slow; /* slow-path order-0 allocations */ u64 slow_high_order; /* slow-path high order allocations */ + u64 empty; /* failed refills due to empty ptr ring, forcing + * slow path allocation + */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 24306d6..9d20b12 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -131,8 +131,10 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) int pref_nid; /* preferred NUMA node */ /* Quicker fallback, avoid locks when ring is empty */ - if (__ptr_ring_empty(r)) + if (__ptr_ring_empty(r)) { + page_pool_stat_alloc_inc(empty); return NULL; + } /* Softirq guarantee CPU and thus NUMA node is stable. This, * assumes CPU refilling driver RX-ring will also run RX-NAPI. From patchwork Sat Jan 29 23:38:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729698 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0ECDC433FE for ; Sat, 29 Jan 2022 23:40:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353458AbiA2XkU (ORCPT ); Sat, 29 Jan 2022 18:40:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353450AbiA2XkP (ORCPT ); Sat, 29 Jan 2022 18:40:15 -0500 Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77CD4C061741 for ; Sat, 29 Jan 2022 15:40:15 -0800 (PST) Received: by mail-pg1-x530.google.com with SMTP id e9so8673670pgb.3 for ; Sat, 29 Jan 2022 15:40:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dWVvR+duDBC2ViqiU0poNo4IacKATL/f4uhXaP1KcDU=; b=vDCTaCjzVNItlFSD0Cdk1R9P2yvfWqh2XavR8xBJz1NkWLOBgQqKlBVOFu3SzrJ9s6 J+U26zf6N35EpixJzAJXdwT5YktcoFSi6MZ1GW+s0LRru8YKCgN2T5I+tNj4NQf2R1Di 80ekE+1v+ygmJirr3+aciXzYd8X+CaSZZ2lxM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dWVvR+duDBC2ViqiU0poNo4IacKATL/f4uhXaP1KcDU=; b=inHML9P2jeK3tPTFY265zTybMS/8G3K+psJcLBXGZS96shiGTRXityjPKdtx8wTWWR b4dxfsiefQ/eeDYGC7pszF10Cf6xgGXktbL11+mhh53UyMB3lBKFe2oSsgqijnhpSPvq pp8DTrkRsSfH9iM8xxiN/2ktWhGXorNLvu9Milqy2WquQu3FumI18QqHtlxcb8Ym9ftA 5PDn46UeZoBP2mDvvQRLif6mN2fwCzX4N7obJ5SHa9sQtCldWATR5CyB8Yp9VidwbiRZ CrHtZowl9LnRkgFFWHOOV0oxMZoNXdK2xOVXSZpUCVWssj2Cb+3Dhq2A6yZMQSpF7Vy/ zkaA== X-Gm-Message-State: AOAM533SgRGJcbQJTTV3UDTqTUI9n5ja1eOH/A5EjeEc+Su52mIiFXBC I085dH+Wup9NK79aoxVTxfyWnSeOD3vILk/fTPkrnpW+YKTJ/0+C3+SpIeiah5gE4oEagccT55A 9ZEvnqUPAqdxw5S42vuvO9qObczklYW1coF6BWKlR6qM2IQThPjj9wWYIJbsfImuDBJAT X-Google-Smtp-Source: ABdhPJwCAEKNiby2vYMUg5GSdivJHWkvhT6YTDTgBuFSbtKFYtSLzmJYq0pVZXhJJivGZSHI7Unm+A== X-Received: by 2002:a63:5146:: with SMTP id r6mr11511954pgl.455.1643499614536; Sat, 29 Jan 2022 15:40:14 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:14 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 08/10] page_pool: Add stat tracking cache refill Date: Sat, 29 Jan 2022 15:38:58 -0800 Message-Id: <1643499540-8351-9-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stat tracking succesfull allocations which triggered a refill. Signed-off-by: Joe Damato --- include/net/page_pool.h | 1 + net/core/page_pool.c | 1 + 2 files changed, 2 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index ed2bc73..4991109 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -149,6 +149,7 @@ struct page_pool_stats { u64 empty; /* failed refills due to empty ptr ring, forcing * slow path allocation */ + u64 refill; /* allocations via successful refill */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 9d20b12..ffb68b8 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -185,6 +185,7 @@ static struct page *__page_pool_get_cached(struct page_pool *pool) page_pool_stat_alloc_inc(fast); } else { page = page_pool_refill_alloc_cache(pool); + page_pool_stat_alloc_inc(refill); } return page; From patchwork Sat Jan 29 23:38:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729700 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E68B3C433F5 for ; Sat, 29 Jan 2022 23:40:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353487AbiA2XkX (ORCPT ); Sat, 29 Jan 2022 18:40:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353479AbiA2XkS (ORCPT ); Sat, 29 Jan 2022 18:40:18 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFC16C061753 for ; Sat, 29 Jan 2022 15:40:16 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id s2-20020a17090ad48200b001b501977b23so14482346pju.2 for ; Sat, 29 Jan 2022 15:40:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dJc+WeEpXTCtUAsBZBL3YsLvPmU50z2QJlgbbdI6CRs=; b=RUNjwi2mmjE5/Hs+Z89naGLKYdXCUvkFdDCORVO47p/rVfC5Ff9/ptYTsNQcbbEo7G iI0XgJZRjeDqx8dsD1aF01jNN7uHsAEfl1uhYwK38lYvUYExD1HAgRRfZ5S1NF/VhSdb gVeLLYRfosAdoooIqhTci/fOz4UpK4l7AIBGw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dJc+WeEpXTCtUAsBZBL3YsLvPmU50z2QJlgbbdI6CRs=; b=m5lz0uXuLkHFDmjhMoJc/DthotnOUVbPasoLYz4kbSDHyPYilkFZWFqJOyEvVXqd+c Mu8IJ58+OTXUkhx9OKwkO0jBh38cR5FDPQZNVpx85JhQDmPuVyKMaPYgIZ/kMYWF9Cdu 4/UKyisw9d6k6iCv+BLRqwaCKqOLngG4PcTbxC/jXpZG703axE98+ZP9a3qVXvOe8iJN shDFYAPzpaVqmSchHsbMATZcGK0YPIv4WX2VlEpAuFq5CXaKNPttrPdz+HX8J77Bs9NU Dt33Gy+m1gsOyNRyrek7LJhrfGyS11MQk7Rvfugvo9ipuDvuxHqU/b/JXvWbkg6JxyD4 ZrwQ== X-Gm-Message-State: AOAM531DINDzT3LYSVD8hedfNJToyKepyBdfja/sTDniM76Wm3sVDTXg +wBDBUWJrOcbQK2HTLZPjrzMxSuPOgSIKPU86DdX2NUU0TK3Nyp8AE2fgsnS1Uhj1ncMiUuL2O9 AvnL+LIsGpV+E6HEHp+y9YkApr3iYoBm2qZQm1v1fUFhrs7iWGFz3dFTsvau4ewtibzyu X-Google-Smtp-Source: ABdhPJzR1u/twS9ESa31rcx4irMNdOuitUJvlWpFILQEsSRzlrdwqFr8yZczA4RLSixG4Lm1K/GwoA== X-Received: by 2002:a17:902:eccb:: with SMTP id a11mr15062818plh.121.1643499615908; Sat, 29 Jan 2022 15:40:15 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:15 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 09/10] page_pool: Add a stat tracking waived pages Date: Sat, 29 Jan 2022 15:38:59 -0800 Message-Id: <1643499540-8351-10-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Track how often pages obtained from the ring cannot be added to the cache because of a NUMA mismatch. Signed-off-by: Joe Damato --- include/net/page_pool.h | 1 + net/core/page_pool.c | 1 + 2 files changed, 2 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 4991109..e411ef6 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -150,6 +150,7 @@ struct page_pool_stats { * slow path allocation */ u64 refill; /* allocations via successful refill */ + u64 waive; /* failed refills due to numa zone mismatch */ } alloc; }; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index ffb68b8..c6f31c5 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -161,6 +161,7 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) * This limit stress on page buddy alloactor. */ page_pool_return_page(pool, page); + page_pool_stat_alloc_inc(waive); page = NULL; break; } From patchwork Sat Jan 29 23:39:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12729699 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59936C4332F for ; Sat, 29 Jan 2022 23:40:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353467AbiA2XkV (ORCPT ); Sat, 29 Jan 2022 18:40:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353460AbiA2XkS (ORCPT ); Sat, 29 Jan 2022 18:40:18 -0500 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A6A3C061714 for ; Sat, 29 Jan 2022 15:40:18 -0800 (PST) Received: by mail-pl1-x62f.google.com with SMTP id s6so3823461plg.12 for ; Sat, 29 Jan 2022 15:40:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=OcKHYlUmrKmGbdNWNifNKpXqnTivrjtnzcz+DoQJ60o=; b=KyRVYs86ghuRCzykjGMif3/NH6u2yT/LopkuWA0id1rzjh46XRKVki4w8NFcYxkoF5 WpLHmVywG109Dd/mKi21Dd4Z7d7WO47Ghuit5d+pqP3/BSBHeaA+P41GOsj3CaAvvf2F l4JMNy8/ZZs27ySRin7NjlFUK5anRwJvwFmTA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=OcKHYlUmrKmGbdNWNifNKpXqnTivrjtnzcz+DoQJ60o=; b=7a9zXvSe0YMqiG8QeFIpVSCe5FxXELvTfGvQTWQxoy/I82UtpAfoiNsMwfM+/AJNiV hifrzLGIUJVdscpcfCELm6kwmRIMkElFv4TkkqiQgv6UcrGnAimkq9vxs1tKHmxLK1KO 4A9EP5KzeYItEId5s2USbbTEcSsg1hDe7CdwYad+m6zXxsu6296lCkLTuQUnZXZXxY7P 9nckzGeS9Hvnioh8fL3syJ3HAm1yFbV3M2TtXp8kjHHJyFWFlXN0wc0vihmB1CxLSztp yGMvDAoXSi5gk9Nu0qL1ZuzI3YXpIO3aJmELhCS/mjHNfp0+cmN6nhR/SktV10HM+VFY zIGQ== X-Gm-Message-State: AOAM531LiGvOCyt9sLiGK81yux+vTXmS1C+eE3tKvP5vJypWpSqsekPx aR5K+of/bz8jChYRAfw2B5J/34Gnnrg/ne4FUxvGJ6MCtJNPxzXYmy6tykhq9b3wW1+zkqa4wZK OhqTym93usyCC9mj1Rapk5dPTYt8juvW6YlK99EL92xJcBX0gSZmC8YEMO3P0XIStSMjV X-Google-Smtp-Source: ABdhPJwcppAguYZSfiDPvakaJfC/d5PCCFjI9E98usHjo40Q9JBCeFYPikD5PbV0lid758z/vsnyjQ== X-Received: by 2002:a17:902:db0a:: with SMTP id m10mr14632255plx.92.1643499617265; Sat, 29 Jan 2022 15:40:17 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id gb5sm6573276pjb.16.2022.01.29.15.40.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Jan 2022 15:40:16 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org, kuba@kernel.org, ilias.apalodimas@linaro.org, davem@davemloft.net, hawk@kernel.org Cc: Joe Damato Subject: [net-next v2 10/10] net-procfs: Show page pool stats in proc Date: Sat, 29 Jan 2022 15:39:00 -0800 Message-Id: <1643499540-8351-11-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643499540-8351-1-git-send-email-jdamato@fastly.com> References: <1643499540-8351-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Per-cpu page pool allocation stats are exported in the file /proc/net/page_pool_stat allowing users to better understand the interaction between their drivers and kernel memory allocation. Signed-off-by: Joe Damato Reported-by: kernel test robot --- net/core/net-procfs.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 68 insertions(+), 1 deletion(-) diff --git a/net/core/net-procfs.c b/net/core/net-procfs.c index 88cc0ad..3d3f3e8 100644 --- a/net/core/net-procfs.c +++ b/net/core/net-procfs.c @@ -4,6 +4,10 @@ #include #include +#ifdef CONFIG_PAGE_POOL_STATS +#include +#endif + #define BUCKET_SPACE (32 - NETDEV_HASHBITS - 1) #define get_bucket(x) ((x) >> BUCKET_SPACE) @@ -310,6 +314,58 @@ static const struct seq_operations ptype_seq_ops = { .show = ptype_seq_show, }; +#ifdef CONFIG_PAGE_POOL_STATS +static struct page_pool_stats *page_pool_stat_get_online(loff_t *pos) +{ + struct page_pool_stats *pp_stat = NULL; + + while (*pos < nr_cpu_ids) { + if (cpu_online(*pos)) { + pp_stat = per_cpu_ptr(&page_pool_stats, *pos); + break; + } + + ++*pos; + } + + return pp_stat; +} + +static void *page_pool_seq_start(struct seq_file *seq, loff_t *pos) +{ + return page_pool_stat_get_online(pos); +} + +static void *page_pool_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + ++*pos; + return page_pool_stat_get_online(pos); +} + +static void page_pool_seq_stop(struct seq_file *seq, void *v) +{ +} + +static int page_pool_seq_show(struct seq_file *seq, void *v) +{ + struct page_pool_stats *pp_stat = v; + + seq_printf(seq, "%08llx %08llx %08llx %08llx %08llx %08llx %08llx %08llx\n", + seq->index, pp_stat->alloc.fast, + pp_stat->alloc.slow, pp_stat->alloc.slow_high_order, + pp_stat->alloc.empty, pp_stat->alloc.refill, + pp_stat->alloc.refill, pp_stat->alloc.waive); + return 0; +} + +static const struct seq_operations page_pool_seq_ops = { + .start = page_pool_seq_start, + .next = page_pool_seq_next, + .stop = page_pool_seq_stop, + .show = page_pool_seq_show, +}; +#endif + static int __net_init dev_proc_net_init(struct net *net) { int rc = -ENOMEM; @@ -323,12 +379,23 @@ static int __net_init dev_proc_net_init(struct net *net) if (!proc_create_net("ptype", 0444, net->proc_net, &ptype_seq_ops, sizeof(struct seq_net_private))) goto out_softnet; +#ifdef CONFIG_PAGE_POOL_STATS + if (!proc_create_seq("page_pool_stat", 0444, net->proc_net, + &page_pool_seq_ops)) + goto out_ptype; +#endif if (wext_proc_init(net)) - goto out_ptype; + goto out_page_pool; rc = 0; out: return rc; + +out_page_pool: +#ifdef CONFIG_PAGE_POOL_STATS + remove_proc_entry("page_pool_stat", net->proc_net); +#endif + out_ptype: remove_proc_entry("ptype", net->proc_net); out_softnet: