From patchwork Wed Jan 26 22:48:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12725902 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D7B8C433FE for ; Wed, 26 Jan 2022 22:51:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231355AbiAZWvx (ORCPT ); Wed, 26 Jan 2022 17:51:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231376AbiAZWvw (ORCPT ); Wed, 26 Jan 2022 17:51:52 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C26CCC06173B for ; Wed, 26 Jan 2022 14:51:52 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id y27so1031693pfa.0 for ; Wed, 26 Jan 2022 14:51:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=iQtOf45R3wZ6d5P0weFxN9wE1JYLlAX2zbCnJw9CZhM=; b=uZodQ9s9Q9ttLoSgg4ImwQeMBABj/XtrTosuhH1UzwQ+Xmxr3W6XJc1qN1tQiKF7AH JjDVUM1m9xZ207obycYRP3/Ze+Z48zuIdwsPxDzjqh5MU2LlmMworC5iaXOLPGcYx05/ XdF0XnaaUBmjDnsh1vNXQYHP87X+sUbMiPuTY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=iQtOf45R3wZ6d5P0weFxN9wE1JYLlAX2zbCnJw9CZhM=; b=hWFkvJXN61Ez74MGGh4CnenUXaUtI3n5KtKTI3e7elEJDQxRsqVVx1lWlBRB5t0UsU +DtWeHXNb4iVRjr9gQJvYc4bFIoBa6/fQQttevW3zn13UALb+r5iqJYaD1jPNo6WYdw2 kukrre/pXsABKZ4zL2S/RiGC8Ei5mMJQ8TcbjnHyM99J7lNk54oLJpEb5BKfgepjqjDk YK+i+OTGdtoHOx8NPtkzp3VtrMGIo7fz5gGde/alkojUuptJvo+f3Cc2ZGHGIJuLYSuu GVedyeobf3U7NEDsVGr6KbbWdcE19wcjOBrE/a1hnM7i1r7ue1kv8d8tIK0Dos42HM8V 9ZXw== X-Gm-Message-State: AOAM530FgA110eFtgMEYlJbO+7iTlJrvrooYkK80ZkdCzjaiY1m6AxQI mVnuqeqXC2paH8z4JR+BfAydpP0ykAE6Vfk+nxNenvonHA86UjogGhGdZCRhBQsl50JDFpsbqlo rjUyrD4MXhDhNkaga5dSabMUFb3ou0c5pTQVHHaLwKyjt9j1dQljJrwklJv4pBbZHslF3NBw= X-Google-Smtp-Source: ABdhPJw8cP0R8iktyV3pTZTIucneKY3NFslkyEYd8mQwH2ACJ8ZvYZyqR8vymH/y7i+DYspvmQjdiQ== X-Received: by 2002:a63:50f:: with SMTP id 15mr788419pgf.186.1643237511832; Wed, 26 Jan 2022 14:51:51 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id q15sm3793941pjj.19.2022.01.26.14.51.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Jan 2022 14:51:51 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: kuba@kernel.org, davem@davemloft.net, ilias.apalodimas@linaro.org, hawk@kernel.org, Joe Damato Subject: [PATCH 1/6] net: page_pool: Add alloc stats and fast path stat Date: Wed, 26 Jan 2022 14:48:15 -0800 Message-Id: <1643237300-44904-2-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643237300-44904-1-git-send-email-jdamato@fastly.com> References: <1643237300-44904-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stats structure with a an internal alloc structure for holding allocation path related stats. The alloc structure contains the stat 'fast'. This stat tracks fast path allocations. A static inline accessor function is exposed for accessing this stat. Signed-off-by: Joe Damato --- include/net/page_pool.h | 26 ++++++++++++++++++++++++++ net/core/page_pool.c | 1 + 2 files changed, 27 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 79a8055..3ae3dc4 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -71,6 +71,20 @@ struct pp_alloc_cache { struct page *cache[PP_ALLOC_CACHE_SIZE]; }; +/** + * stats for tracking page_pool events. + * + * accessor functions for these stats provided below. + * + * Note that it is the responsibility of the API consumer to ensure that + * the page_pool has not been destroyed while accessing stats fields. + */ +struct page_pool_stats { + struct { + u64 fast; /* fast path allocations */ + } alloc; +}; + struct page_pool_params { unsigned int flags; unsigned int order; @@ -86,6 +100,7 @@ struct page_pool_params { struct page_pool { struct page_pool_params p; + struct page_pool_stats ps; struct delayed_work release_dw; void (*disconnect)(void *); @@ -180,6 +195,12 @@ void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *), void page_pool_release_page(struct page_pool *pool, struct page *page); void page_pool_put_page_bulk(struct page_pool *pool, void **data, int count); + +static inline u64 page_pool_stats_get_fast(struct page_pool *pool) +{ + return pool->ps.alloc.fast; +} + #else static inline void page_pool_destroy(struct page_pool *pool) { @@ -199,6 +220,11 @@ static inline void page_pool_put_page_bulk(struct page_pool *pool, void **data, int count) { } + +static inline u64 page_pool_stats_get_fast(struct page_pool *pool) +{ + return 0; +} #endif void page_pool_put_page(struct page_pool *pool, struct page *page, diff --git a/net/core/page_pool.c b/net/core/page_pool.c index bd62c01..84c9566 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -166,6 +166,7 @@ static struct page *__page_pool_get_cached(struct page_pool *pool) if (likely(pool->alloc.count)) { /* Fast-path */ page = pool->alloc.cache[--pool->alloc.count]; + pool->ps.alloc.fast++; } else { page = page_pool_refill_alloc_cache(pool); } From patchwork Wed Jan 26 22:48:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12725903 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C612C433F5 for ; Wed, 26 Jan 2022 22:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231555AbiAZWvz (ORCPT ); Wed, 26 Jan 2022 17:51:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231597AbiAZWvy (ORCPT ); Wed, 26 Jan 2022 17:51:54 -0500 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5396FC06173B for ; Wed, 26 Jan 2022 14:51:54 -0800 (PST) Received: by mail-pg1-x531.google.com with SMTP id q75so641435pgq.5 for ; Wed, 26 Jan 2022 14:51:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nMhyZuz5FFHXx6SDIWuWD353Ivk0f2F3J9m8oxhLKhE=; b=gFacH/DvGsMLVHix1oTy7Hmcnj6Vq+NTq8Ao+Ul1QiTURmSY31geJXe5C3LxzkJigh AwyT/X9otwhOvh7eqghURSEDQmTYdbG8PTSfNMaAX7pVSYw9tFpo+J6AoBP4F51LelrF EyfwrU3i+5u9/TFHgcPz1cWdsyOuvcwmvP0is= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nMhyZuz5FFHXx6SDIWuWD353Ivk0f2F3J9m8oxhLKhE=; b=1rux8G7iMmsXvsmMy+uU/IXLF7e4k74T4s+3jH92+lvzdj28Nc+x2CQ+XJgROUfV4T 1zBw/H2y6N4w7faqHvzH3BfOMyZWkfe2hmOecwN8MTJkdwx2fEL7PLVnxSK2++tS35/S W1T93DTmCEIKIL5cSd/2e2FwVe/Zn2mqdXrhNea68XZ6JAJfXHOl4zQsq9a3dzKWtEq4 g83+Q+QM9wqESKSZkCM9JG61gEEpUhevdTvVExbE4ruF3hBHu7J2/rtYULSgL0o8xbDH W34CEXtEbm59/iePEWluZ1aJsxtb8ewuqArWdKM63bJC+RxQORFklIBHm5GON/ifMLIN tDpQ== X-Gm-Message-State: AOAM5320PkzyDzyfYmvMdnIpAyMUNwnFBQiZq/Fozqk7WJMMNR3Gxj4b qJwM0sacMNZRFLZ7VP/Ujq/rYRDeCFVM7YFDB9GcPrZZB2KocdWWs7fFJ3nmeNM2ovWI7t4Xsgi T+eXWPK9ra7y5zfwSNGYpLUKLuFxBik7Mx20Qdm6+ZYjBB+bXMJ8eenTIYuJoxiZ2Pu/8 X-Google-Smtp-Source: ABdhPJze0ZxXHOjMTsRUtq4X/VQGhmFqHLmTsaROLnTdEqmfo7g8ivgM7DlJXGTYCUjMDpY/lHWuMw== X-Received: by 2002:a05:6a00:244b:: with SMTP id d11mr429376pfj.49.1643237513424; Wed, 26 Jan 2022 14:51:53 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id q15sm3793941pjj.19.2022.01.26.14.51.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Jan 2022 14:51:52 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: kuba@kernel.org, davem@davemloft.net, ilias.apalodimas@linaro.org, hawk@kernel.org, Joe Damato Subject: [PATCH net-next 2/6] net: page_pool: Add a stat for the slow alloc path Date: Wed, 26 Jan 2022 14:48:16 -0800 Message-Id: <1643237300-44904-3-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643237300-44904-1-git-send-email-jdamato@fastly.com> References: <1643237300-44904-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stat, 'slow', for the slow allocation path. A static inline accessor function is exposed for accessing this stat. Signed-off-by: Joe Damato --- include/net/page_pool.h | 10 ++++++++++ net/core/page_pool.c | 6 ++++-- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 3ae3dc4..b5691ee 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -82,6 +82,7 @@ struct pp_alloc_cache { struct page_pool_stats { struct { u64 fast; /* fast path allocations */ + u64 slow; /* slow-path order-0 allocations */ } alloc; }; @@ -201,6 +202,10 @@ static inline u64 page_pool_stats_get_fast(struct page_pool *pool) return pool->ps.alloc.fast; } +static inline u64 page_pool_stats_get_slow(struct page_pool *pool) +{ + return pool->ps.alloc.slow; +} #else static inline void page_pool_destroy(struct page_pool *pool) { @@ -225,6 +230,11 @@ static inline u64 page_pool_stats_get_fast(struct page_pool *pool) { return 0; } + +static inline u64 page_pool_stats_get_slow(struct page_pool *pool) +{ + return 0; +} #endif void page_pool_put_page(struct page_pool *pool, struct page *page, diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 84c9566..9dbe721 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -294,10 +294,12 @@ static struct page *__page_pool_alloc_pages_slow(struct page_pool *pool, } /* Return last page */ - if (likely(pool->alloc.count > 0)) + if (likely(pool->alloc.count > 0)) { page = pool->alloc.cache[--pool->alloc.count]; - else + pool->ps.alloc.slow++; + } else { page = NULL; + } /* When page just alloc'ed is should/must have refcnt 1. */ return page; From patchwork Wed Jan 26 22:48:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12725904 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DB51C433F5 for ; Wed, 26 Jan 2022 22:51:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231376AbiAZWv5 (ORCPT ); Wed, 26 Jan 2022 17:51:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231392AbiAZWvz (ORCPT ); Wed, 26 Jan 2022 17:51:55 -0500 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1C84C06161C for ; Wed, 26 Jan 2022 14:51:55 -0800 (PST) Received: by mail-pl1-x636.google.com with SMTP id i1so878038pla.9 for ; Wed, 26 Jan 2022 14:51:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pToh2wvYgYPrpKDxpymUchJpPok83K1wHJc2NYkaSuE=; b=lGyLwI0Iv8Imyx1xxIMX9LmDAXxbPapefd2E4ssjwVVbN34T93amq/LbgoOdc5TvJq AlMYt8EihMMvL60Wp8LaIoZ/hUFcEev0P4X9KLh4W9v+0ERI79/iMBwRdNG3lbGyG5L/ AW6ZxtrwiKdhHsjpdD70In9PxwQ/3rQmqX/7M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pToh2wvYgYPrpKDxpymUchJpPok83K1wHJc2NYkaSuE=; b=4zNznP/BVWo5MMg2W5UASbzAYw1SbIgtVIL3tZUqkfhHNoCQ9RN/Hr/37xyQ7BEgbg YRgO08Sbhg1WEwFc1iWRpTSzqqhcqW5J4+lJ8nPhTupvv8u7YhsbNoWJR4w/MVOFg/U0 nmHsdw+wU8V0EfdcsaeeNK9zQbnBTygOx+HRxItnGKkvOh/JRq78svkD+vp8tqObiHz8 miMSMpdjf7iq/UlosmS1mGPNx4foLi9HPe3juNGyy5b0NtxVH9ksHhXF3kCFACEG1fxG 4ujmsJHm34yBA+0nEH1kHJ/0Xiy9H3/pkOH76h2IdVqL8oCqBGJNaMip5R6WMak5//O+ lhag== X-Gm-Message-State: AOAM531oiwXUw4ePIXqD38dE2fzE7PvgGWywUV1jGltVUTK82UXS0dKP DYdwpnvNqQK8y8F1LwhkdTuJF2FEh3z+Nh9Y76x1928t/c9vPmwcLMW/+M2nKBflU51Bk2O6CDe 4UroPLxd5VwxJ/tpuyzIVmtDEkukUQrV1+ho0QNMkhnmQgU4GoQ5jG/K/owZYEy74hq2P X-Google-Smtp-Source: ABdhPJyPXupHScrKo6BUjE2ocjfDzLH/c1V80tmF/a+D/iBR1dCRkL5GAOOaHRSEceyabt3w6hEtyg== X-Received: by 2002:a17:902:7848:: with SMTP id e8mr862292pln.90.1643237514868; Wed, 26 Jan 2022 14:51:54 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id q15sm3793941pjj.19.2022.01.26.14.51.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Jan 2022 14:51:54 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: kuba@kernel.org, davem@davemloft.net, ilias.apalodimas@linaro.org, hawk@kernel.org, Joe Damato Subject: [PATCH net-next 3/6] net: page_pool: Add a high order alloc stat Date: Wed, 26 Jan 2022 14:48:17 -0800 Message-Id: <1643237300-44904-4-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643237300-44904-1-git-send-email-jdamato@fastly.com> References: <1643237300-44904-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stat to track high order allocations in the slow path. A static inline function is exposed for accessing this stat. Signed-off-by: Joe Damato --- include/net/page_pool.h | 11 +++++++++++ net/core/page_pool.c | 1 + 2 files changed, 12 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index b5691ee..b024197 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -83,6 +83,7 @@ struct page_pool_stats { struct { u64 fast; /* fast path allocations */ u64 slow; /* slow-path order-0 allocations */ + u64 slow_high_order; /* slow-path high order allocations */ } alloc; }; @@ -206,6 +207,11 @@ static inline u64 page_pool_stats_get_slow(struct page_pool *pool) { return pool->ps.alloc.slow; } + +static inline u64 page_pool_stats_get_slow_high_order(struct page_pool *pool) +{ + return pool->ps.alloc.slow_high_order; +} #else static inline void page_pool_destroy(struct page_pool *pool) { @@ -235,6 +241,11 @@ static inline u64 page_pool_stats_get_slow(struct page_pool *pool) { return 0; } + +static inline u64 page_pool_stats_get_slow_high_order(struct page_pool *pool) +{ + return 0; +} #endif void page_pool_put_page(struct page_pool *pool, struct page *page, diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 9dbe721..3a4b912 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -240,6 +240,7 @@ static struct page *__page_pool_alloc_page_order(struct page_pool *pool, return NULL; } + pool->ps.alloc.slow_high_order++; page_pool_set_pp_info(pool, page); /* Track how many pages are held 'in-flight' */ From patchwork Wed Jan 26 22:48:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12725905 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E714C433EF for ; Wed, 26 Jan 2022 22:51:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231779AbiAZWv6 (ORCPT ); Wed, 26 Jan 2022 17:51:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229742AbiAZWv5 (ORCPT ); Wed, 26 Jan 2022 17:51:57 -0500 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39ED2C06174E for ; Wed, 26 Jan 2022 14:51:57 -0800 (PST) Received: by mail-pj1-x102e.google.com with SMTP id s61-20020a17090a69c300b001b4d0427ea2so5681543pjj.4 for ; Wed, 26 Jan 2022 14:51:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qPqcIeRdK7gzpY5MRRAybRjZA49mLbpIBffMOLgY4GY=; b=K91YmoyfRccw1UBHpYCrzF5VNEwAVJzTObalIy6PirCBPJRnjI69o9fpqV3Jqn8UrE Y50JyGxRmb3+Hq9o5A6Lq3BIPBvbBGp/cPLt9M5Z5MrQ64rlUwJIOVju7J4iaVL4L8HJ 2xH5xWotQ7iRCOVT1v+hK+q7nmwPVktT5WLAs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qPqcIeRdK7gzpY5MRRAybRjZA49mLbpIBffMOLgY4GY=; b=Rgugh3FjeBXAQLW8uA0WL2/5K4JNZADAVwoBtzuFu2kXVpNx5a5eMJT97ut8RHzvKv VrWZgWdoiEpdfT4P59+MpUPx6IXBMRi8X45gLKRwMr5K0DMf2qvdKE15TyOADu0TXP3j AMi750YrNYxxoiJw3S9tLi2ozRdZNgI91kUIbrpS1wPWJFa1CtAh28pCOYCrz1t6MK4O /uCpJYVwUqm/v6InpGF/XrcTZ93TQEVKKBpDicJznPjOqQK3SnMYq/CklncYhdgDhDHF dDpdvj85WsmKvp8yV+nAl2AjfLvC4QxlJSdUEUf5h8xoY5qw48RQfQvaRN0cUH3Nbwl9 AN4w== X-Gm-Message-State: AOAM530m/M/VIQUG5ensSp7+EY5OmgdWf1b8P34S9UhToFpNe+Jv6I2w XozPzJaE79fokJcTu+Y3DOxsmWdtgjT1jI7K3DS4mWuhw4vf8wVjdh02shcvAnUXaCx0bM7RQVc q03BVep8Od1BiVUqnGJV4AZXxTEBI++e6sc9vE2HVSSrmacJwRVziWkydnnZuxCLsUuR/ X-Google-Smtp-Source: ABdhPJzEUwKG6lSvbH62KBukEw+pLylbraJG1AK1iGKdDOolqRiA945M2Kr5f0vL863051UPRsKgdw== X-Received: by 2002:a17:903:32ce:: with SMTP id i14mr608610plr.66.1643237516340; Wed, 26 Jan 2022 14:51:56 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id q15sm3793941pjj.19.2022.01.26.14.51.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Jan 2022 14:51:55 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: kuba@kernel.org, davem@davemloft.net, ilias.apalodimas@linaro.org, hawk@kernel.org, Joe Damato Subject: [PATCH net-next 4/6] net: page_pool: Add stat tracking empty ring Date: Wed, 26 Jan 2022 14:48:18 -0800 Message-Id: <1643237300-44904-5-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643237300-44904-1-git-send-email-jdamato@fastly.com> References: <1643237300-44904-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stat tracking when the ptr ring is empty. A static inline wrapper function is exposed to access this stat. Signed-off-by: Joe Damato --- include/net/page_pool.h | 15 +++++++++++++++ net/core/page_pool.c | 4 +++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index b024197..92edf8e 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -84,6 +84,10 @@ struct page_pool_stats { u64 fast; /* fast path allocations */ u64 slow; /* slow-path order-0 allocations */ u64 slow_high_order; /* slow-path high order allocations */ + u64 empty; /* failed refills due to empty ptr ring, forcing + * slow path allocation + */ + } alloc; }; @@ -212,6 +216,12 @@ static inline u64 page_pool_stats_get_slow_high_order(struct page_pool *pool) { return pool->ps.alloc.slow_high_order; } + +static inline u64 page_pool_stats_get_empty(struct page_pool *pool) +{ + return pool->ps.alloc.empty; +} + #else static inline void page_pool_destroy(struct page_pool *pool) { @@ -246,6 +256,11 @@ static inline u64 page_pool_stats_get_slow_high_order(struct page_pool *pool) { return 0; } + +static inline u64 page_pool_stats_get_empty(struct page_pool *pool) +{ + return 0; +} #endif void page_pool_put_page(struct page_pool *pool, struct page *page, diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 3a4b912..0de9d58 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -117,8 +117,10 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) int pref_nid; /* preferred NUMA node */ /* Quicker fallback, avoid locks when ring is empty */ - if (__ptr_ring_empty(r)) + if (__ptr_ring_empty(r)) { + pool->ps.alloc.empty++; return NULL; + } /* Softirq guarantee CPU and thus NUMA node is stable. This, * assumes CPU refilling driver RX-ring will also run RX-NAPI. From patchwork Wed Jan 26 22:48:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12725906 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9351CC433F5 for ; Wed, 26 Jan 2022 22:52:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231392AbiAZWwC (ORCPT ); Wed, 26 Jan 2022 17:52:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231829AbiAZWv6 (ORCPT ); Wed, 26 Jan 2022 17:51:58 -0500 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD587C06173B for ; Wed, 26 Jan 2022 14:51:58 -0800 (PST) Received: by mail-pg1-x529.google.com with SMTP id p125so657662pga.2 for ; Wed, 26 Jan 2022 14:51:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kXjN8WFwgBGK9vxvsU06qWyzmb8YpuggPizN+CnYy/s=; b=nusDvUkuRA8UlDGQyEfEhdBV6QctFDrPVMbzEojRnJQuG6ffW/DxBWYYP3psxKREhH AxvUlIeKtg8UgmrtJWUFMCEbZ5LUEl3ukqoT6EYP0ujn8EYHuFKCFjoOpIbp3SL/EUyX qStPIIXCmmrwnZ4Jtn3ysTa3Jipph4qsDGFsw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kXjN8WFwgBGK9vxvsU06qWyzmb8YpuggPizN+CnYy/s=; b=bxh/IuD5WJ0UdkFDzbNqw7jpWvuC2v6Sygg+ya7r8r4EJWbo/6oKFCF5ZGet7XMOs3 FEDuUmjDTAKC4DrYBK+TAHzJxmsgi+B6zoN7/E+aDga68QUAXwQB4NDMBgJw5LMgogLY LIJYsSwtn2EjY/HqX7G7KjGY7r8n2S4txgnOdu9x2LaRkIa9wQzC6EDqFsgpl++sACvO pN/Y9llfb7KrE1LfVDBsSScPnWZTjj0nuKMdHkevWUXDjZ84oPsvk+OntRsVSU9Hd5wA IWeiSjsrYDHAfjWp0ZW7SrGZFrMMbQIKqBSoQUXUK0D7w6TmwbhDY5kHTXZi7GAtuD2H 7aLw== X-Gm-Message-State: AOAM532bq6g9vB4+Zw8SasNT/1pH53cycH9CdqC4qguD3Vfx8nFP+yB5 E2K5LPq0ixQGMS+xWOKQ5zbETkzhjC/fShq8gop2UKQwlscy0E8IwPHWfetU6DD2CPWueZ79Vut O31nDBRXKsvYoTrZk2GqV+jscstfYMDCoqrj7ux0ejmdpBWzsDefCR5oOUWZjvUteNoXb X-Google-Smtp-Source: ABdhPJzj0C8b2ZLXg2hgDSY/maWIJ3u1ure/4x28nHAcgnPRaut5SaOcpbPrE2ZH+o/WoEnT5UtJbg== X-Received: by 2002:a63:9044:: with SMTP id a65mr748155pge.594.1643237517895; Wed, 26 Jan 2022 14:51:57 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id q15sm3793941pjj.19.2022.01.26.14.51.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Jan 2022 14:51:57 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: kuba@kernel.org, davem@davemloft.net, ilias.apalodimas@linaro.org, hawk@kernel.org, Joe Damato Subject: [PATCH net-next 5/6] net: page_pool: Add stat tracking cache refills. Date: Wed, 26 Jan 2022 14:48:19 -0800 Message-Id: <1643237300-44904-6-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643237300-44904-1-git-send-email-jdamato@fastly.com> References: <1643237300-44904-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a stat tracking succesfull allocations which triggered a refill. A static inline wrapper is exposed for accessing this stat. Signed-off-by: Joe Damato --- include/net/page_pool.h | 11 ++++++++++- net/core/page_pool.c | 2 ++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 92edf8e..a68d05f 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -87,7 +87,7 @@ struct page_pool_stats { u64 empty; /* failed refills due to empty ptr ring, forcing * slow path allocation */ - + u64 refill; /* allocations via successful refill */ } alloc; }; @@ -222,6 +222,10 @@ static inline u64 page_pool_stats_get_empty(struct page_pool *pool) return pool->ps.alloc.empty; } +static inline u64 page_pool_stats_get_refill(struct page_pool *pool) +{ + return pool->ps.alloc.refill; +} #else static inline void page_pool_destroy(struct page_pool *pool) { @@ -261,6 +265,11 @@ static inline u64 page_pool_stats_get_empty(struct page_pool *pool) { return 0; } + +static inline u64 page_pool_stats_get_refill(struct page_pool *pool) +{ + return 0; +} #endif void page_pool_put_page(struct page_pool *pool, struct page *page, diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 0de9d58..15f4e73 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -171,6 +171,8 @@ static struct page *__page_pool_get_cached(struct page_pool *pool) pool->ps.alloc.fast++; } else { page = page_pool_refill_alloc_cache(pool); + if (likely(page)) + pool->ps.alloc.refill++; } return page; From patchwork Wed Jan 26 22:48:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 12725907 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 390DDC433FE for ; Wed, 26 Jan 2022 22:52:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231829AbiAZWwC (ORCPT ); Wed, 26 Jan 2022 17:52:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232008AbiAZWwA (ORCPT ); Wed, 26 Jan 2022 17:52:00 -0500 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8689FC061749 for ; Wed, 26 Jan 2022 14:52:00 -0800 (PST) Received: by mail-pl1-x62a.google.com with SMTP id y17so891119plg.7 for ; Wed, 26 Jan 2022 14:52:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3XamyOUKOhYOUIf/RQQ4uu6WCjRVfiMgJ0/RAAtZBoY=; b=K1VMLE5Pur93VHjsvualZ92NugEpEzbEXz6Rpayh6pQM55+6IaCthqPTgHM543FK6e 5h8JTRdw23CkAPR3xovJg3OJeQkQ17z82Mu8eVGtk+B8f64pCQy2LGTn8zG+98W+oOjg OVaVQfdJGmuDWIWl9xI/hW66OdBRyUE+9c2ME= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3XamyOUKOhYOUIf/RQQ4uu6WCjRVfiMgJ0/RAAtZBoY=; b=1tUH4BO3k6TNsNsd9z4jf1OvD0J1e8K6LpbbBv7iDI4CWnKOM8qToX4jYX0YI/Ilir SOUruFXMTntK6q2XF5Xo/XtOLYgW9gVB65CrMxDBBtgMIavk2hipTLqLUW7+ToaLoeKu zhpKd44HbbFb2G+3t8Q46ajGDw/5zxve/t90TCZEk+bzZU7u8a4Dx+RhJpFy35lR8wax Utk1irBA45vs87d2ffoeix9hJDKZFeJF7fYQb5vRZ/rGBbTYijZPmlLRiKI6AOm82uaz IoureADRKw87GytaDu6LHSgiJ866++agM4AgCuLluflCGIi+r539CmebNRD4mw6VPN+K 1S9w== X-Gm-Message-State: AOAM530iGgHB4QnM+OTbdDFvMJ2W5vPHSlOxPDkhJyVzK9HhYY0/MM7I 52EUdTEcqJ+BNj1Ipa9tS2tqNHDTXzYPSrhaI5vHH55+iv+ITRgLWRUs1lFTSUx4tz9ys7c9uVa DpuNYWVSYp4oBSyJhLwJ6ycIutCP72lAioPknc195Kc9We6Dj98/tVbWp8mRr8NG0uvEa X-Google-Smtp-Source: ABdhPJwomOY+A7Ps2AhNrNrqPaSHN1WQV6vVvvCkyOcjQjAz1Xb50wDlKHi8BoTrDMXp8tDWbghgkA== X-Received: by 2002:a17:903:1cd:: with SMTP id e13mr644704plh.158.1643237519389; Wed, 26 Jan 2022 14:51:59 -0800 (PST) Received: from localhost.localdomain (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id q15sm3793941pjj.19.2022.01.26.14.51.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Jan 2022 14:51:58 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: kuba@kernel.org, davem@davemloft.net, ilias.apalodimas@linaro.org, hawk@kernel.org, Joe Damato Subject: [PATCH 6/6] net: page_pool: Add a stat tracking waived pages. Date: Wed, 26 Jan 2022 14:48:20 -0800 Message-Id: <1643237300-44904-7-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643237300-44904-1-git-send-email-jdamato@fastly.com> References: <1643237300-44904-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Track how often pages obtained from the ring cannot be added to the cache because of a NUMA mismatch. A static inline wrapper is added for accessing this stat. Signed-off-by: Joe Damato --- include/net/page_pool.h | 11 +++++++++++ net/core/page_pool.c | 1 + 2 files changed, 12 insertions(+) diff --git a/include/net/page_pool.h b/include/net/page_pool.h index a68d05f..cf65d78 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -88,6 +88,7 @@ struct page_pool_stats { * slow path allocation */ u64 refill; /* allocations via successful refill */ + u64 waive; /* failed refills due to numa zone mismatch */ } alloc; }; @@ -226,6 +227,11 @@ static inline u64 page_pool_stats_get_refill(struct page_pool *pool) { return pool->ps.alloc.refill; } + +static inline u64 page_pool_stats_get_waive(struct page_pool *pool) +{ + return pool->ps.alloc.waive; +} #else static inline void page_pool_destroy(struct page_pool *pool) { @@ -270,6 +276,11 @@ static inline u64 page_pool_stats_get_refill(struct page_pool *pool) { return 0; } + +static inline u64 page_pool_stats_get_waive(struct page_pool *pool) +{ + return 0; +} #endif void page_pool_put_page(struct page_pool *pool, struct page *page, diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 15f4e73..7c4ae2e 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -147,6 +147,7 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) * This limit stress on page buddy alloactor. */ page_pool_return_page(pool, page); + pool->ps.alloc.waive++; page = NULL; break; }