From patchwork Fri Oct 18 10:53:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13841629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1F80D2FFF2 for ; Fri, 18 Oct 2024 11:00:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5786C6B009A; Fri, 18 Oct 2024 07:00:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 528826B009B; Fri, 18 Oct 2024 07:00:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FAAD6B009C; Fri, 18 Oct 2024 07:00:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 230506B009A for ; Fri, 18 Oct 2024 07:00:40 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 25B5F16197F for ; Fri, 18 Oct 2024 11:00:26 +0000 (UTC) X-FDA: 82686429420.06.41D4CEE Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf14.hostedemail.com (Postfix) with ESMTP id 79E10100030 for ; Fri, 18 Oct 2024 11:00:24 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf14.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729249092; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZLwScqOff4Jo3eMrfdcob/2g1898hiJbHK4CizR+rHI=; b=8XoklijZedHfwbFBajyND/25ubItu8iS+NJcf5KspicSpA5nt95PWaxYfAgVnjktIKuEtJ h2ngWeW0k4VzyVCEnAwiQqtfjkwPHRs/p7SFwQpE572E60PFi0l903vHoCpIhnP51l0tZH cVkPVXqfgkpEihciGEFwklLWe+Es9ww= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729249092; a=rsa-sha256; cv=none; b=CJR7PtOk3dFqeT3H45ETOzR6+ZVd9BJYAzoX+mFKI4NI2bKpL0sOoG99s00Q/4vQM16B27 mrl94fM2GLYEDlBgcaEiBtAgwYZ7TiSgnPqAq6u7BH2hw3KM/9s+sWpbkTvn/lB7Zh5gW8 5V/QZcr3WtNjagSXo48jvWwrKou1BUU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf14.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com Received: from mail.maildlp.com (unknown [172.19.162.254]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4XVMBj6pkMz10NlC; Fri, 18 Oct 2024 18:58:37 +0800 (CST) Received: from dggpemf200006.china.huawei.com (unknown [7.185.36.61]) by mail.maildlp.com (Postfix) with ESMTPS id 5A10E180113; Fri, 18 Oct 2024 19:00:33 +0800 (CST) Received: from localhost.localdomain (10.90.30.45) by dggpemf200006.china.huawei.com (7.185.36.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Fri, 18 Oct 2024 19:00:33 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Alexander Duyck , Andrew Morton , Subject: [PATCH net-next v22 10/14] mm: page_frag: introduce prepare/probe/commit API Date: Fri, 18 Oct 2024 18:53:47 +0800 Message-ID: <20241018105351.1960345-11-linyunsheng@huawei.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20241018105351.1960345-1-linyunsheng@huawei.com> References: <20241018105351.1960345-1-linyunsheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.90.30.45] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemf200006.china.huawei.com (7.185.36.61) X-Rspamd-Queue-Id: 79E10100030 X-Stat-Signature: gxdxdaa7zk84eyxthxxf6dsic1x5ianq X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1729249224-87836 X-HE-Meta: U2FsdGVkX18P0DF9j2YVc9OwciFKjP38z4vfmkg6zp2ffa6VhaKIuE1MBPjmI9e5kLxuWMZ6A8phAuE5Qy7rbKo6a8HnMvic9euL/km9AXOBqvjqTYg0JYhfizfRGSBQCZ7LGLBxn8HO2NZsQJMHMtha02MyNC767dYfUxENGIkzQwpb8MzplB+6pabbZhqYJ1bG8suwrWQBLQnm5Vqg7JUB4EmGmkk3UvyniDDgH11Wt6R6SmCNX/hlq4s4itu4yhaF1pTAjhRsfDSECa9UQmaIUt4jkIRg6W91Z9Ufd5hoFA7GZnlCSvLkMG/NLpeLhZT1K2L5RXNSkQ+eclTClOlitLJBRa9AG1hHTyrAZ64oqqcyWY86BtGSjA0vT6XsAvMamV3qJgPBgpUoBiwhOrZaAAbYRMI8sQ4y8RjLISBVZrfUAKAbcW21XYzvV3T3q+KfmPTEkKx29ZsLAxg/gSizNr/5P6G6UyntSh610Fy4B6oPeGrPNkMs3aDFn7Dek8ngZaCNQgxdWuvMo6d80j49TfxNHfqZUUArk4qV8Trp0hzu7dnWjV+JR/F6QANS/almeulJIlYUWwSMBbNBG8g8FhrzEM3a1R5KfTIjEHyd11NRA7NiOSFNsi9Nauk26kM/3C2VPjtTrNoUbzppMAqzKKn6zJn2PKoAvNZiRzLCbVgxB4J0tnxPIV02QGvGaqT106/wvugppDWnr+ZnHNAoRTKwTYi4/2gRioLNykswZmFv5D/S6WQdK+N1c3AtxesXGUKsrxKdau+BXoQL6QHne8+0lu9CB05VmLAruK42cjUCB5V+FTLHY26AX3XjLceXP3MPHKriU4qj9MmK+a0v9xXbqF928a9S+TMeez8hs6cRXzgwevr6CW519cmwL/lFkfA6ydQmHQESBNZ75H8xy/ORliyiHcO0NgtEEcdxBJRa+I27XXdVW3vOram91IZIvDfV7NydU+DYkeD YiwHHrAH zMEG9v8M6rSbj9U4eIOLWSsKjqfrbQEYs7XIGDm/nD+X8eZ+rFgFuufLz1/DnPjDTgSjjBrcjPVFGzgNJ07cJihbSrRW10dt4kvtd3NWCsPc3MguxOGeeNm5H8THWiAsC865H4yrNhT+V3x1jNFN4To73EnCi6n4wxTKw23GHLnkj3qXllyOy5zPNLjkfbBnqckNucfOHeR3gbykTVmVqxVIDXN3sbU70DsUMdd/gRIyJ2k0pQDKGODy4+ptCaDg7Lmpxf+9hO0DBs0nABt1mS2xxpqpA9UG40y+c X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are many use cases that need minimum memory in order for forward progress, but more performant if more memory is available or need to probe the cache info to use any memory available for frag caoleasing reason. Currently skb_page_frag_refill() API is used to solve the above use cases, but caller needs to know about the internal detail and access the data field of 'struct page_frag' to meet the requirement of the above use cases and its implementation is similar to the one in mm subsystem. To unify those two page_frag implementations, introduce a prepare API to ensure minimum memory is satisfied and return how much the actual memory is available to the caller and a probe API to report the current available memory to caller without doing cache refilling. The caller needs to either call the commit API to report how much memory it actually uses, or not do so if deciding to not use any memory. CC: Alexander Duyck Signed-off-by: Yunsheng Lin --- include/linux/page_frag_cache.h | 130 ++++++++++++++++++++++++++++++++ mm/page_frag_cache.c | 21 ++++++ 2 files changed, 151 insertions(+) diff --git a/include/linux/page_frag_cache.h b/include/linux/page_frag_cache.h index feed99d0cddb..1c0c11250b66 100644 --- a/include/linux/page_frag_cache.h +++ b/include/linux/page_frag_cache.h @@ -46,6 +46,10 @@ void *__page_frag_cache_prepare(struct page_frag_cache *nc, unsigned int fragsz, unsigned int __page_frag_cache_commit_noref(struct page_frag_cache *nc, struct page_frag *pfrag, unsigned int used_sz); +void *__page_frag_alloc_refill_probe_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + unsigned int align_mask); static inline unsigned int __page_frag_cache_commit(struct page_frag_cache *nc, struct page_frag *pfrag, @@ -88,6 +92,132 @@ static inline void *page_frag_alloc(struct page_frag_cache *nc, return __page_frag_alloc_align(nc, fragsz, gfp_mask, ~0u); } +static inline bool __page_frag_refill_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align_mask) +{ + if (unlikely(!__page_frag_cache_prepare(nc, fragsz, pfrag, gfp_mask, + align_mask))) + return false; + + __page_frag_cache_commit(nc, pfrag, fragsz); + return true; +} + +static inline bool page_frag_refill_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, unsigned int align) +{ + WARN_ON_ONCE(!is_power_of_2(align)); + return __page_frag_refill_align(nc, fragsz, pfrag, gfp_mask, -align); +} + +static inline bool page_frag_refill(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, gfp_t gfp_mask) +{ + return __page_frag_refill_align(nc, fragsz, pfrag, gfp_mask, ~0u); +} + +static inline bool __page_frag_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align_mask) +{ + return !!__page_frag_cache_prepare(nc, fragsz, pfrag, gfp_mask, + align_mask); +} + +static inline bool page_frag_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align) +{ + WARN_ON_ONCE(!is_power_of_2(align)); + return __page_frag_refill_prepare_align(nc, fragsz, pfrag, gfp_mask, + -align); +} + +static inline bool page_frag_refill_prepare(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask) +{ + return __page_frag_refill_prepare_align(nc, fragsz, pfrag, gfp_mask, + ~0u); +} + +static inline void *__page_frag_alloc_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align_mask) +{ + return __page_frag_cache_prepare(nc, fragsz, pfrag, gfp_mask, align_mask); +} + +static inline void *page_frag_alloc_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align) +{ + WARN_ON_ONCE(!is_power_of_2(align)); + return __page_frag_alloc_refill_prepare_align(nc, fragsz, pfrag, + gfp_mask, -align); +} + +static inline void *page_frag_alloc_refill_prepare(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask) +{ + return __page_frag_alloc_refill_prepare_align(nc, fragsz, pfrag, + gfp_mask, ~0u); +} + +static inline void *page_frag_alloc_refill_probe(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag) +{ + return __page_frag_alloc_refill_probe_align(nc, fragsz, pfrag, ~0u); +} + +static inline bool page_frag_refill_probe(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag) +{ + return !!page_frag_alloc_refill_probe(nc, fragsz, pfrag); +} + +static inline void page_frag_commit(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + __page_frag_cache_commit(nc, pfrag, used_sz); +} + +static inline void page_frag_commit_noref(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + __page_frag_cache_commit_noref(nc, pfrag, used_sz); +} + +static inline void page_frag_alloc_abort(struct page_frag_cache *nc, + unsigned int fragsz) +{ + VM_BUG_ON(fragsz > nc->offset); + + nc->pagecnt_bias++; + nc->offset -= fragsz; +} + void page_frag_free(void *addr); #endif diff --git a/mm/page_frag_cache.c b/mm/page_frag_cache.c index f55d34cf7d43..5ea4b663ab8e 100644 --- a/mm/page_frag_cache.c +++ b/mm/page_frag_cache.c @@ -112,6 +112,27 @@ unsigned int __page_frag_cache_commit_noref(struct page_frag_cache *nc, } EXPORT_SYMBOL(__page_frag_cache_commit_noref); +void *__page_frag_alloc_refill_probe_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + unsigned int align_mask) +{ + unsigned long encoded_page = nc->encoded_page; + unsigned int size, offset; + + size = PAGE_SIZE << encoded_page_decode_order(encoded_page); + offset = __ALIGN_KERNEL_MASK(nc->offset, ~align_mask); + if (unlikely(!encoded_page || offset + fragsz > size)) + return NULL; + + pfrag->page = encoded_page_decode_page(encoded_page); + pfrag->size = size - offset; + pfrag->offset = offset; + + return encoded_page_decode_virt(encoded_page) + offset; +} +EXPORT_SYMBOL(__page_frag_alloc_refill_probe_align); + void *__page_frag_cache_prepare(struct page_frag_cache *nc, unsigned int fragsz, struct page_frag *pfrag, gfp_t gfp_mask, unsigned int align_mask)