From patchwork Fri Sep 6 07:36:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13793532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D60BDCE7AB8 for ; Fri, 6 Sep 2024 07:43:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF3296B0093; Fri, 6 Sep 2024 03:42:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A527E6B0095; Fri, 6 Sep 2024 03:42:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CC8D6B0096; Fri, 6 Sep 2024 03:42:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6E6996B0093 for ; Fri, 6 Sep 2024 03:42:59 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B18D7C1080 for ; Fri, 6 Sep 2024 07:42:58 +0000 (UTC) X-FDA: 82533522036.14.6908815 Received: from szxga07-in.huawei.com (szxga07-in.huawei.com [45.249.212.35]) by imf09.hostedemail.com (Postfix) with ESMTP id 429FA14000C for ; Fri, 6 Sep 2024 07:42:55 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.35 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725608527; a=rsa-sha256; cv=none; b=AycxlSTd1DFwrMCy4SqJx68674IU0ELBF22Yl6xjsEhNBN0namFiplo5TRKqwYwKVA7iZ9 St4+w/qskvGSwMHNR55wFxl97pVk7NKNOs74/tr68EjSotR8+UYi5rKdvBjhWzlsJnxkQc VVFlBLvWyRcN1z1PpfVCUcmbtJATCVc= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.35 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725608527; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v2/aZMoWgQtjCquiJJ1G8YfRSVBJSWDwe2hHP2fBdbE=; b=izm2hux6hJPSlCf1ktaBZD2N59ZMS2KcLkvEXJAEe2cVYssxGhUdSrmYJhy9i1ffSM+XwU P4CWNfq4mzUKWmUVnKCrpLqGqL4fSa8vC0y4PhISpxx8mpC51HXlaO4QiMqFk+sw0M2Voc 5yrZ4EKxnNkR7nUDfZBbFX2ydf/mKjc= Received: from mail.maildlp.com (unknown [172.19.88.163]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4X0Sqn6WX5z1S9vd; Fri, 6 Sep 2024 15:42:29 +0800 (CST) Received: from dggpemf200006.china.huawei.com (unknown [7.185.36.61]) by mail.maildlp.com (Postfix) with ESMTPS id 404E2180041; Fri, 6 Sep 2024 15:42:53 +0800 (CST) Received: from localhost.localdomain (10.90.30.45) by dggpemf200006.china.huawei.com (7.185.36.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Fri, 6 Sep 2024 15:42:53 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Alexander Duyck , Andrew Morton , Subject: [PATCH net-next v18 07/14] mm: page_frag: some minor refactoring before adding new API Date: Fri, 6 Sep 2024 15:36:39 +0800 Message-ID: <20240906073646.2930809-8-linyunsheng@huawei.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20240906073646.2930809-1-linyunsheng@huawei.com> References: <20240906073646.2930809-1-linyunsheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.90.30.45] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemf200006.china.huawei.com (7.185.36.61) X-Stat-Signature: kzhxwfuuhwm8erdmx558zsoirewwrs9a X-Rspamd-Queue-Id: 429FA14000C X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1725608575-921616 X-HE-Meta: U2FsdGVkX1+fOjOUOpBLD4RYu82t8m1VjDciD6t9Z1FpLLtoHK/hir2i0hO7oMHyRdMw2u9YR6M5jGoWLS0Y/rm+9AhUACZTAZx2Xg9CnvgylXN3EyicWZwK3pFp/eS+R6isB58zu7Zs/a63xpNPGqCDpsUOzE0YGdQR4W+NIdrPKWGLHuQAJ0v4qrUI/q50hFIEVwKtod7ohCMXnd8vmqyE/PFUtEfhqMXex6XXmVCuQiGlgxe2P3ahr4/3bvgV9899o6/Up5aDbHxeam7PV/Av5fJr/t7+ddPf2JjGNEb2mDiG6lClELeqR9k9xovrTxk6aVkDG+HG5HDYilaSAw3dUQj3m51gAgKfoLfjmK7uMW+bccMZhyl4ucwY39xnbisw7qIgC55kNCCAgfMr6oLH4xIO9oWVKOcVqvqmJu4MZvuCw4Q9IqhXnE7ty4qtl8Eqqx2B4IvKD8/ZoFi8ESmUeWmQVc2kRwexcoiR17NZ5fjuPj+seg9eXLZpDEQYdK2mdw9X7+hobzWEEnVO/PDieiYy4APhvMGuSjHvXOyTZsPcbvi0ZwwE539EJb3ixvrWKNzyl3NqWxYTu4hc42bR3B9lQtB77cVnCcuq0jg/0nR0vj1xPhgB9kGJYHCBh+A/O58gUv8biddUJyAGw2q5LVrC/tGlQosxIcSQkU9/oGgUd82cG0rXqZpo2hfBGtcuRCBhwNXm6lTWGbncFZkThrF2+FrELawThGSbtDBDTVFM3oKMfP8EAk6ooP8PK4zwYZIItTuKqLRv9ufY/HY/R4GwTWI6Vmm+bX2rnBXZzmW2nvYF1U/CVUGME/6LI/ooT+4y58cvyO+a8B8CiNfwAq7jeXlGEtEzIi3BqHc/3gyt3jD75NIdeNwLO62SjWskatZFzQVCNJO4L1PX473IHC0v9Nmo0rMCgeV+xDV3wG0ip32LJBTXCkidXdv+y6ZPgox7uki0tT1HS3W /pgE5L5M VIcBBjp1C/0B9mlmF6j1PbCegQZ6KBfjqS6dCmtHOYqBzBVwfGZWNe6e4rYcIxKDHM6mSN7InAomuuLHf91ocBToyqYG5h7RdR/WTh75pjkW5e3q58id/OFHK4l+kFOFlbM1zVtmnt3/gzaxRYkIiUTExytx0AIKACVEuo4ZevFwam8DOEy5SNjZGeIyoAGyvtMIGNjFYkghgdCkSwRWybOMj6JjPpP4JZc4rvMzF2pIGQGC4lzYDpefkBDqQ2/wJmjC5cEZv1zg9IUng8+NNq1y+s+pzh18JtNxK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Refactor common codes from __page_frag_alloc_va_align() to __page_frag_cache_prepare() and __page_frag_cache_commit(), so that the new API can make use of them. CC: Alexander Duyck Signed-off-by: Yunsheng Lin --- include/linux/page_frag_cache.h | 36 +++++++++++++++++++++++++++-- mm/page_frag_cache.c | 40 ++++++++++++++++++++++++++------- 2 files changed, 66 insertions(+), 10 deletions(-) diff --git a/include/linux/page_frag_cache.h b/include/linux/page_frag_cache.h index 75aaad6eaea2..b634e1338741 100644 --- a/include/linux/page_frag_cache.h +++ b/include/linux/page_frag_cache.h @@ -5,6 +5,7 @@ #include #include +#include #include #include @@ -41,8 +42,39 @@ static inline bool page_frag_cache_is_pfmemalloc(struct page_frag_cache *nc) void page_frag_cache_drain(struct page_frag_cache *nc); void __page_frag_cache_drain(struct page *page, unsigned int count); -void *__page_frag_alloc_align(struct page_frag_cache *nc, unsigned int fragsz, - gfp_t gfp_mask, unsigned int align_mask); +void *__page_frag_cache_prepare(struct page_frag_cache *nc, unsigned int fragsz, + struct page_frag *pfrag, gfp_t gfp_mask, + unsigned int align_mask); +unsigned int __page_frag_cache_commit_noref(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz); + +static inline unsigned int __page_frag_cache_commit(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + VM_BUG_ON(!nc->pagecnt_bias); + nc->pagecnt_bias--; + + return __page_frag_cache_commit_noref(nc, pfrag, used_sz); +} + +static inline void *__page_frag_alloc_align(struct page_frag_cache *nc, + unsigned int fragsz, gfp_t gfp_mask, + unsigned int align_mask) +{ + struct page_frag page_frag; + void *va; + + va = __page_frag_cache_prepare(nc, fragsz, &page_frag, gfp_mask, + align_mask); + if (unlikely(!va)) + return NULL; + + __page_frag_cache_commit(nc, &page_frag, fragsz); + + return va; +} static inline void *page_frag_alloc_align(struct page_frag_cache *nc, unsigned int fragsz, gfp_t gfp_mask, diff --git a/mm/page_frag_cache.c b/mm/page_frag_cache.c index cf9375a81a64..6f6e47bbdc8d 100644 --- a/mm/page_frag_cache.c +++ b/mm/page_frag_cache.c @@ -95,9 +95,31 @@ void __page_frag_cache_drain(struct page *page, unsigned int count) } EXPORT_SYMBOL(__page_frag_cache_drain); -void *__page_frag_alloc_align(struct page_frag_cache *nc, - unsigned int fragsz, gfp_t gfp_mask, - unsigned int align_mask) +unsigned int __page_frag_cache_commit_noref(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + unsigned int orig_offset; + + VM_BUG_ON(used_sz > pfrag->size); + VM_BUG_ON(pfrag->page != page_frag_encoded_page_ptr(nc->encoded_page)); + VM_BUG_ON(pfrag->offset + pfrag->size > + page_frag_cache_page_size(nc->encoded_page)); + + /* pfrag->offset might be bigger than the nc->offset due to alignment */ + VM_BUG_ON(nc->offset > pfrag->offset); + + orig_offset = nc->offset; + nc->offset = pfrag->offset + used_sz; + + /* Return true size back to caller considering the offset alignment */ + return nc->offset - orig_offset; +} +EXPORT_SYMBOL(__page_frag_cache_commit_noref); + +void *__page_frag_cache_prepare(struct page_frag_cache *nc, unsigned int fragsz, + struct page_frag *pfrag, gfp_t gfp_mask, + unsigned int align_mask) { unsigned long encoded_page = nc->encoded_page; unsigned int size, offset; @@ -119,6 +141,8 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, /* reset page count bias and offset to start of new frag */ nc->pagecnt_bias = PAGE_FRAG_CACHE_MAX_SIZE + 1; nc->offset = 0; + } else { + page = page_frag_encoded_page_ptr(encoded_page); } size = page_frag_cache_page_size(encoded_page); @@ -137,8 +161,6 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, return NULL; } - page = page_frag_encoded_page_ptr(encoded_page); - if (!page_ref_sub_and_test(page, nc->pagecnt_bias)) goto refill; @@ -153,15 +175,17 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, /* reset page count bias and offset to start of new frag */ nc->pagecnt_bias = PAGE_FRAG_CACHE_MAX_SIZE + 1; + nc->offset = 0; offset = 0; } - nc->pagecnt_bias--; - nc->offset = offset + fragsz; + pfrag->page = page; + pfrag->offset = offset; + pfrag->size = size - offset; return page_frag_encoded_page_address(encoded_page) + offset; } -EXPORT_SYMBOL(__page_frag_alloc_align); +EXPORT_SYMBOL(__page_frag_cache_prepare); /* * Frees a page fragment allocated out of either a compound or order 0 page.