From patchwork Thu Nov 14 12:16:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13875062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7817FD65C7E for ; Thu, 14 Nov 2024 12:22:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A2E46B0095; Thu, 14 Nov 2024 07:22:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 02B466B0098; Thu, 14 Nov 2024 07:22:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E34376B0096; Thu, 14 Nov 2024 07:22:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C242F6B0093 for ; Thu, 14 Nov 2024 07:22:56 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 70BCCACFDE for ; Thu, 14 Nov 2024 12:22:56 +0000 (UTC) X-FDA: 82784613030.06.8F88304 Received: from szxga06-in.huawei.com (szxga06-in.huawei.com [45.249.212.32]) by imf27.hostedemail.com (Postfix) with ESMTP id C8FD74000D for ; Thu, 14 Nov 2024 12:22:08 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf27.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731586796; a=rsa-sha256; cv=none; b=5h7MXl7WkgfBhbMbawBeCMn0tk05lmEktHmapnHYkX9QpjvXjSvM7HrImxYsKlG01/cB93 HvgCPFqsbdN3AK3aVIS8mYBtNBXiSTmskw3q2Uljl7A9z06AJymG6N13n1jeLDawmCtxgm vwogZTciohVOOBMkWPh08Q6GgizbH1E= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf27.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731586796; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SU9hFE/IxapLxlkNs+ckNoGZnzqsXf/PqdlweufhXC8=; b=XI+pzH4DBiI5V04yK1auQhOeZDBspG/MRd7WeTbnVR7sGqOR/5ejdmDyulHgCsOpApM1Tk 3OgOeJ8QVp6k6nB6TTNQnwtp1YVAtXT9mrarCrDZ0UlMZ6FuzM5l51h0zHYsaMUROC53/Z FC9MmOEal39CYfak3XX6eaOxOBMTQnQ= Received: from mail.maildlp.com (unknown [172.19.88.234]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4Xpznf1Mz4z1yqVG; Thu, 14 Nov 2024 20:23:02 +0800 (CST) Received: from dggpemf200006.china.huawei.com (unknown [7.185.36.61]) by mail.maildlp.com (Postfix) with ESMTPS id 6E3CB1401F4; Thu, 14 Nov 2024 20:22:50 +0800 (CST) Received: from localhost.localdomain (10.90.30.45) by dggpemf200006.china.huawei.com (7.185.36.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 14 Nov 2024 20:22:50 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Alexander Duyck , Andrew Morton , Linux-MM , Jonathan Corbet , Subject: [PATCH net-next v1 05/10] mm: page_frag: introduce refill prepare & commit API Date: Thu, 14 Nov 2024 20:16:00 +0800 Message-ID: <20241114121606.3434517-6-linyunsheng@huawei.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20241114121606.3434517-1-linyunsheng@huawei.com> References: <20241114121606.3434517-1-linyunsheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.90.30.45] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemf200006.china.huawei.com (7.185.36.61) X-Stat-Signature: y7rfand5kx433ockc36pt1hhne358q5u X-Rspamd-Queue-Id: C8FD74000D X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1731586928-190307 X-HE-Meta: U2FsdGVkX18gG3xggpV7AXs6dDKEk4JDuInL06rvLHyzQUu57gXIwDtOq8dNiNjyiiZhezue1Qc1PUXpdzqx6GDG43QtucJqCMQrmiAAF8nVgVaiLtes2L2NGvJMy+3ct9P9WnKvQGZ6JvbF1F2eMyurcnKlNcH7fZHNzN518OaiyAx1S97nGVZqGbLotWc1r76bWgAgF6JCPDD5AHepPLDjKHYve9QaNUdph/tWPGwNzj9serlfZEca6TyoF3YXiFR/8R9Ixaxgz2DtcErzq5SLt9E4OKQHxvH6ZNFg+fnrxGNSqQmEwiLOi2YVs+SDsHDpK4FgWsuFb1k2cK6BbjLGDxF2FvuXOvKG5aCIgGNWiB8gY5NqHAQneiEuXBjY8RuBGFZYywOVZ+uHkwtNmtMYStKbqv+BDNisrs22hbvcWHIbRz2z3dD0f3i4US/i/EpMzaXImtu3jiZGgEilYhX0FR/QNXfrSey+79fprA17WbvD9ZVdWFZqvrGx3pr43eHwqBRxbNTWiZCkIkQ+boRcMbOsXscUWiabEPLj6Bg/VVeJfNU4N5bhn1njjeejZuI/48gycya+wJ8nuk2rBurqAc+YnGFnL4WHCD/TtfRxAaYL3N+eRCWFYL8sl6XxIPCUoTMni4cGe3kzulzSkZLDWOOSZPTHANtEt3cqCJFVM/NNXWxH5ui5WrmrVrb895kl9m55GnsGHJVqIBJhkIqp5eNL8HOhYI62pWl+5Vi+ekTgqyhiBFu8Gn+DmXdwrNTiZsYxBVbfJ7GlCBtWuMhvvDwqAatrH1WSUhdwRC4grtCn0QkaQ0ryMZlK3aZ9FmxtKMqeF/L11tkWybgsPfSdRNbDl46OISmydtxUR6bHDuPaSba6CiRoSwiyf2MFq0AahE82MDRXNQTgpDV5qp1iaRsXd2RcUQHCnEEZHx7TFGKChdIWwX73x4G3f8LXcOiBKTZ80DlrUHFKd2I le9aHTV2 1SjeGvUI7WVzQCGcA/QN/pZETpMqOR7bT45MShsS2XTWnamYWWzMXsgKSsMR8+K5qHHlX8Hne5atXuPMI/ojiPvyzlqlmv81BQp147/qe+p+uCPm6iFfKySusiLsh0AjWsiOf4SRSoE9/M5suIQHtroNTU1j4A+UzormMN+rYnD0N3SORtbjLQtO7htJ3cYFrgDdsgoAZf82FMjwRcX06rnb5Zb792vxjo+8hhyU4aTTy+Q6PamSibBP2v9Hj/82MTz3VP3j7N9oZB8g+UHYd9QsxhXkWVq2nakRTe+dOY7IZBm9azGECHEJJLWwovjpAO4hiL1kwh1EqPciRuUENmLgwRYuBfFcEMCbSSjHc9BrBOxwDZIc8j1pkWoGFcTlw870Q7IOA7AMmmfnVA+5McElhtg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently page_frag only have a alloc API which returns the virtual address of a fragment by a specific size. There are many use cases that need minimum memory in order for forward progress, but more performant if more memory is available, and expect to use the 'struct page' of the allocated fragment directly instead of the virtual address. Currently skb_page_frag_refill() API is used to solve the above use cases, but caller needs to know about the internal detail and access the data field of 'struct page_frag' to meet the requirement of the above use cases and its implementation is similar to the one in mm subsystem. To unify those two page_frag implementations, introduce a prepare API to ensure minimum memory is satisfied and return how much the actual memory is available to the caller. The caller needs to either call the commit API to report how much memory it actually uses, or not do so if deciding to not use any memory. CC: Alexander Duyck CC: Andrew Morton CC: Linux-MM Signed-off-by: Yunsheng Lin --- Documentation/mm/page_frags.rst | 43 ++++++++++++- include/linux/page_frag_cache.h | 110 ++++++++++++++++++++++++++++++++ 2 files changed, 152 insertions(+), 1 deletion(-) diff --git a/Documentation/mm/page_frags.rst b/Documentation/mm/page_frags.rst index 339e641beb53..4cfdbe7db55a 100644 --- a/Documentation/mm/page_frags.rst +++ b/Documentation/mm/page_frags.rst @@ -111,10 +111,18 @@ page is aligned according to the 'align/alignment' parameter. Note the size of the allocated fragment is not aligned, the caller needs to provide an aligned fragsz if there is an alignment requirement for the size of the fragment. +There is a use case that needs minimum memory in order for forward progress, but +more performant if more memory is available. By using the prepare and commit +related API, the caller calls prepare API to requests the minimum memory it +needs and prepare API will return the maximum size of the fragment returned. The +caller needs to either call the commit API to report how much memory it actually +uses, or not do so if deciding to not use any memory. + .. kernel-doc:: include/linux/page_frag_cache.h :identifiers: page_frag_cache_init page_frag_cache_is_pfmemalloc __page_frag_alloc_align page_frag_alloc_align page_frag_alloc - page_frag_alloc_abort + page_frag_alloc_abort __page_frag_refill_prepare_align + page_frag_refill_prepare_align page_frag_refill_prepare .. kernel-doc:: mm/page_frag_cache.c :identifiers: page_frag_cache_drain page_frag_free page_frag_alloc_abort_ref @@ -152,3 +160,36 @@ Allocation & freeing API ... page_frag_free(va); + + +Refill Preparation & committing API +----------------------------------- + +.. code-block:: c + + struct page_frag page_frag, *pfrag; + bool merge = true; + + pfrag = &page_frag; + if (!page_frag_refill_prepare(nc, 32U, pfrag, GFP_KERNEL)) + goto wait_for_space; + + copy = min_t(unsigned int, copy, pfrag->size); + if (!skb_can_coalesce(skb, i, pfrag->page, pfrag->offset)) { + if (i >= max_skb_frags) + goto new_segment; + + merge = false; + } + + copy = mem_schedule(copy); + if (!copy) + goto wait_for_space; + + if (merge) { + skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); + page_frag_refill_commit_noref(nc, pfrag, copy); + } else { + skb_fill_page_desc(skb, i, pfrag->page, pfrag->offset, copy); + page_frag_refill_commit(nc, pfrag, copy); + } diff --git a/include/linux/page_frag_cache.h b/include/linux/page_frag_cache.h index c3347c97522c..1e699334646a 100644 --- a/include/linux/page_frag_cache.h +++ b/include/linux/page_frag_cache.h @@ -140,6 +140,116 @@ static inline void *page_frag_alloc(struct page_frag_cache *nc, return __page_frag_alloc_align(nc, fragsz, gfp_mask, ~0u); } +/** + * __page_frag_refill_prepare_align() - Prepare refilling a page_frag with + * aligning requirement. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache need to be refilled + * @align_mask: the requested aligning requirement for the fragment + * + * Prepare refilling a page_frag from page_frag cache with aligning requirement. + * + * Return: + * True if prepare refilling succeeds, otherwise return false. + */ +static inline bool __page_frag_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align_mask) +{ + return !!__page_frag_cache_prepare(nc, fragsz, pfrag, gfp_mask, + align_mask); +} + +/** + * page_frag_refill_prepare_align() - Prepare refilling a page_frag with + * aligning requirement. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache needs to be refilled + * @align: the requested aligning requirement for the fragment + * + * WARN_ON_ONCE() checking for @align before prepare refilling a page_frag from + * page_frag cache with aligning requirement. + * + * Return: + * True if prepare refilling succeeds, otherwise return false. + */ +static inline bool page_frag_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align) +{ + WARN_ON_ONCE(!is_power_of_2(align)); + return __page_frag_refill_prepare_align(nc, fragsz, pfrag, gfp_mask, + -align); +} + +/** + * page_frag_refill_prepare() - Prepare refilling a page_frag. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache need to be refilled + * + * Prepare refilling a page_frag from page_frag cache. + * + * Return: + * True if refill succeeds, otherwise return false. + */ +static inline bool page_frag_refill_prepare(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask) +{ + return __page_frag_refill_prepare_align(nc, fragsz, pfrag, gfp_mask, + ~0u); +} + +/** + * page_frag_refill_commit - Commit a prepare refilling. + * @nc: page_frag cache from which to commit + * @pfrag: the page_frag to be committed + * @used_sz: size of the page fragment has been used + * + * Commit the actual used size for the refill that was prepared. + * + * Return: + * The true size of the fragment considering the offset alignment. + */ +static inline unsigned int page_frag_refill_commit(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + return __page_frag_cache_commit(nc, pfrag, used_sz); +} + +/** + * page_frag_refill_commit_noref - Commit a prepare refilling without taking + * refcount. + * @nc: page_frag cache from which to commit + * @pfrag: the page_frag to be committed + * @used_sz: size of the page fragment has been used + * + * Commit the prepare refilling by passing the actual used size, but not taking + * refcount. Mostly used for fragmemt coalescing case when the current fragment + * can share the same refcount with previous fragment. + * + * Return: + * The true size of the fragment considering the offset alignment. + */ +static inline unsigned int +page_frag_refill_commit_noref(struct page_frag_cache *nc, + struct page_frag *pfrag, unsigned int used_sz) +{ + return __page_frag_cache_commit_noref(nc, pfrag, used_sz); +} + void page_frag_free(void *addr); void page_frag_alloc_abort_ref(struct page_frag_cache *nc, void *va, unsigned int fragsz);