From patchwork Mon Sep 2 07:21:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B3B0CA0ED3 for ; Mon, 2 Sep 2024 07:16:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A51048D0084; Mon, 2 Sep 2024 03:16:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9FF538D002D; Mon, 2 Sep 2024 03:16:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 89F8D8D0084; Mon, 2 Sep 2024 03:16:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 67E6C8D002D for ; Mon, 2 Sep 2024 03:16:39 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 07CD1121A19 for ; Mon, 2 Sep 2024 07:16:39 +0000 (UTC) X-FDA: 82518940518.13.5290E7D Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf06.hostedemail.com (Postfix) with ESMTP id 69D88180008 for ; Mon, 2 Sep 2024 07:16:37 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=agYKSAil; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261323; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dVJxbLAsEPgItqfNqHAQC63pv9j1Kl1PzimEv9vqIR4=; b=2mP5eRgIPEqS5K/4+bYk/1ue38MQ2tk02woLQwOLHqY3JPdwDMQbVg+q7JfXOMBBLODNHi ZyEs0Sos/LWWj3lXCrD4yxx7LzCVQQ2/i0CN4DnGSTPlSvJWd8t3vXmpWerFQrGOADlMan 52QmVaDZA6O9s8gAZjbfhK/bpVFegWw= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=agYKSAil; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261323; a=rsa-sha256; cv=none; b=2N3VI3M256JdYET2D9nBnhhD3AUkmugeDdv8EORLm1crSJNUNSnp8XavGdMe5S/yIsjp7V 80yn7I0QizMUcncb81rypcU8tqjY/DVOQzayxZs8ppNP1GCp965THJGobKqxjrlZPvuEcY V1BDjqj2NTYKIuGUnkEr91VPFA1e480= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 7F772A41714; Mon, 2 Sep 2024 07:16:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5BAB1C4CEC2; Mon, 2 Sep 2024 07:16:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261396; bh=35oS5cqvsOoGfro8SlCHvjRjEwfOQy4xgC0vABlxtJk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=agYKSAilfuYO5J7eGOLpdxzw52rI9hxDd1DKce1cIpm9Fjt0aSVLplL3wmg3sHr/G HcnCqtqK+RRS9/TCwa5CYuM7L/NqpTsTB4pDnREzgFVbLECV8qmZhLyt0sh+yUJ0TS FRx3J5kf7s+dBjFhx4zq5oHD+lnChGZzvxxnvQkR9PX/G6oLAS0039chUbgygF+8tp 8ZPsA6lo80t4DbiB6RX9oxVQLDuR6S6ekvtWBhywuT7ZpbwpxXZEVZhDdCjGsMqA2d 2YOFTQU8dknUtnJQPcYzUM2Dpjx2Gq7vW03RUznWSZB8bJNXXeiZPGqYcGs17eJXul wy6Zp76/km4Ug== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 01/21] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Mon, 2 Sep 2024 15:21:12 +0800 Message-ID: <20240902072136.578720-2-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: ewod7e6bytcbxtk1bi3p7nupq67j5x1e X-Rspam-User: X-Rspamd-Queue-Id: 69D88180008 X-Rspamd-Server: rspam02 X-HE-Tag: 1725261397-724873 X-HE-Meta: U2FsdGVkX19dTIY7J975NA4akCnVCmiokCtow3aOGoVXYMlaof1GBFpA9BDaxOJP+gWWmwBbI0jvCoiRaR40op405kqmQFamEZbTGVo3Rwy2jZM5VmuupfsfYJW6KKJDe5wg5a5Bm+hMV7eoA1V4KBfBizpnSay5aMAIFNd0rKk09A0KsVKOzpdPgSRlDuqV8gmpYrY3ABoF9v9gq1gsiRprYB93YMxPrmoPnn7F8sBLoQ9PiYDmhglwCZAKMFgeFu9rzNRKenk1DL0zfYij8NUxb0Pw2uQmfXyWbbX2EEn3iEParT8AMBYLj+AgZN2CobUQCiowyXIhTuJYUd6YKAQ4+fziDb4TyHgdLvblNfc3iq9cRS/XTxZxSoGvXrbW3CiX1ffKGVdWvigufBPK9PS4SDQy4GQmoG0aK0pbVI29eJatrXnnXfn736bxbEfwsG5pVaalwUrkcWVmtLZBisIzo6LmFb1bMVeG9u1+81jJe6KcSpj9taKjkxaSC1pee36FSCzURIrtZjcjmlbSm33HkgDvqiM3vsEfWLuo85AJXw8Pm8MK7poW7mn/eYZW2F5/l/YysPU7TZCcBlkQxvxnC1lcPHDskdq0GcmvDRDpU48R11XZxbFEqIz4Efj8XqxgSHGw73xAa6hDkWFr53d47sZ7Qad+akINqRlVnTcwwCVVPYpyGYTRJRVjbgw2FvAjhpRbAuv01zKRoCJh0uIeTY8/cMleZA15hEMGYIIkleUAtNvcMa47RVPF6MQpbGY7yKnn+EPMvYK29mDYDFU+lDiq/rmjSrQovzrHHm8kV4/L4rIytf5Q2RKcTlQ5BlGkd/WLLvJ2fuxqLZSF+sjuWcUK664yUbWNFFldVhuI5+WGL42lSvFBpt8XKBNYny2scMQnnuh2uUaa5pLFqfyolvc9Nw98trbG505aXfZ5zvNDTjaiNVLR3NE677dp1mPA4yRxdnmz/+0KM2+ R4Qlabs1 29Etlo/InxwhtKFoZaVP00o8ad6yt013893hdLiD4F7a+2A/mc08db52ew5p6ano5wv8MCbchfCicuJrdl0qqLrNs91IuS3v/AKG1/hPxExOcq3YhE4tV/fiSH26N5wrN0Wnas5yitJGnB76sg5OKTHri+08P5sxmXG8d0lCmjJee+MzjiFyg36KCPsLlmBG8S+IR+HqvSX1kLgeXNnsOj8Qkc1CWOM7d+0vYnrLalrzOCf40NGam4QN/zTFFwaVBqWqXA+61OA1L9B6iLOqF5VJoAUzZtumZUBWU/m5g5xHUisl6WsMZjCA1ebJPXIjms1H1xjWEu43ff35uE+s7Eic26g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi The 1st patch introduces new memory descriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). And keep the memcg_data member, since as Yosry pointed out: "When the pages are freed, put_page() -> folio_put() -> __folio_put() will call mem_cgroup_uncharge(). The latter will call folio_memcg() (which reads folio->memcg_data) to figure out if uncharging needs to be done. There are also other similar code paths that will check folio->memcg_data. It is currently expected to be present for all folios. So until we have custom code paths per-folio type for allocation/freeing/etc, we need to keep folio->memcg_data present and properly initialized." Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 72 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 9 ++++--- 2 files changed, 77 insertions(+), 4 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..721ef8861131 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirectly used by page migration + * @mops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Points to the zspage this zpdesc is a part of + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly use by page migration + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + struct movable_operations *mops; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; + unsigned int first_obj_offset; + atomic_t _refcount; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, mops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); +#ifdef CONFIG_MEMCG +ZPDESC_MATCH(memcg_data, memcg_data); +#endif +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +/* Using folio conversion to skip compound_head checking */ +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 73a3ec5b21ad..5d3e27083c72 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -64,6 +64,7 @@ #include #include #include +#include "zpdesc.h" #define ZSPAGE_MAGIC 0x58 @@ -251,7 +252,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -446,7 +447,7 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page = zspage->first_page; + struct page *first_page = zpdesc_page(zspage->first_zpdesc); VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -939,7 +940,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index = 0; if (i == 0) { - zspage->first_page = page; + zspage->first_zpdesc = page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) @@ -1315,7 +1316,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle = handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index = handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); From patchwork Mon Sep 2 07:21:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF79CCA0ED3 for ; Mon, 2 Sep 2024 07:16:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E7658D0085; Mon, 2 Sep 2024 03:16:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 36F958D002D; Mon, 2 Sep 2024 03:16:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19B8E8D0085; Mon, 2 Sep 2024 03:16:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EBDBD8D002D for ; Mon, 2 Sep 2024 03:16:42 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 603C2818A8 for ; Mon, 2 Sep 2024 07:16:42 +0000 (UTC) X-FDA: 82518940644.17.677B95F Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf02.hostedemail.com (Postfix) with ESMTP id C7BCF80006 for ; Mon, 2 Sep 2024 07:16:40 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=omLdEnIe; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261296; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xNc0ifA+gYNWAchE22TUrjdFuU6tmPFfKLoSMIKZZ1I=; b=N7WqHFUnHLLsxI3XfwFagdU/eEBIXO9Ivuk9HwSU2te9Juwtie2LxBVdZFPdJy+DLClcok Pa3i7K4Z/AQF+VDlWb/Sz4I8XhoNlNPG4Ncn6gNJjg2C64I0iFiDN6Jsu4rei7n3OiSL1Q 3J8VDxhtoaEMUu4IXiCqwXjT/7pesBw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261296; a=rsa-sha256; cv=none; b=x12uqwUx+is7CBrbbvEPMvtL7FF2ZSuVvZo2lT+kJVCQ0ck5eFWd+PIfPsHq32KTj9pP33 VEpYOwZpXXiw7r/9GVGdP+HA/aQ57Fkou1P1J1qyEZQciOE101Z8kd7qbR52iI2O4Tef/3 TT93Od9zzg6wWYtiW4GksmjHyh/QJ1g= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=omLdEnIe; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 1340CA41717; Mon, 2 Sep 2024 07:16:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E224BC4CEC2; Mon, 2 Sep 2024 07:16:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261399; bh=uRakrlKF0QiEotK1S+ipaw/WhogyCqf9q/BFIecbO0w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=omLdEnIeCSt4q9xhdyugvCd7eKaYl3jjcrrGs/5B5FQqUj6Mo5C+3aXl5bnh88LRb Kf5fnjA7jl1SR+iGOEUAKiCBuhlX1I/qHaw1wslfDP8slOpfXwmt2YsRgTHwbbOb4d 4a6sOIw5yI4d2mi9KKiLXgYak6mKpr+3ra02wNFlO+5HtvQibAw1un3yWbZlePLsyT XoBvbOMXFe/4sBiEHiPWG6H+JEsRShAaEAXDfmzMxiN0u8coenLq398BsAGcojPe0L xuRvTQSzBgZiepFORp39R6M7m5ShGBV1PKoQqBUqM+k1aYTQp09JtmAGIYTKWGNOM6 nDfHfMcBz4kXw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 02/21] mm/zsmalloc: use zpdesc in trylock_zspage()/lock_zspage() Date: Mon, 2 Sep 2024 15:21:13 +0800 Message-ID: <20240902072136.578720-3-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C7BCF80006 X-Stat-Signature: e6apeuq1j5j9w8478kt4kibjbhxmq3bk X-Rspam-User: X-HE-Tag: 1725261400-359975 X-HE-Meta: U2FsdGVkX19HPJW36eSwv4rCx3d+NjYYvVzGqNmIVrlUA0V2CeXN0RweQKEDgjVW+ZI0A76HYBo6rEJNLewX5hYbp74L/FVDiXj8rO4tFAEdimQYtIqoYSJ3jdSjLblM8+inDC5UN/zDnSZCOfcc45awyev03l+RmvYTokxCF6DF4NFnaSdbldS4mvPVrTWCj77zMH2yeQz+d2jSU+8h/nD7M7ii8XANBfmlpK8frz/gm5SuE3pjZYX7/0XWwEUcXv/A5hxaC0nsuKFykcckcv1zx8hcAuyBxvOTLisq43L2mOuulsEkoSuujITLY1b2/jo+oyFXIzDN7SAEbO2+R3kHcewMTSVg/iC0iLD+mf4x9RrUpN6hEwvdfdA7xEWQoFVGkf/M6lfhrpqcBv17+jY5OMubSQfeOa6q2Sb1SzzmbxN9p+86HOVEks9xBRG3cyDRzogeYmUGTPrpFXsLTUlHtbO8RY2Knk61EC1WblJj8VDaIaMvUP/YDfnVADlVRdVPAO6OSSgIYcpj4wFOtzdZhWzMzpzYzl9Nz301xWDSzKhF+mQH/I1VMzXNb0OeVVyK0lS3Zl25OrpjnBz1n531hJbL4NFBKKXzgYcjxicbn6IfavGrm13o3dthOXZkkYO1IA4+aNYDH8F0KOOFNYTYK6YiDGxyNqCPY9G6ZD1O4svrLlyN2a9T3MZXEl0wTho9C0T8jPD+VPuft+r2PObD2z3B4an/YL/sXKrnv6sd18UA8a7xwiWuEUMaKyyjYVrVbiusnV5abuaXPxtnBgdRGr1G0Z4FgTvkKSvEpwMPSs6mV1Z+nm/5Q8hLX7ZrFmHx4oq6HLoUb6bGfRbZLXdbTRB13t1FcezPuZ0FI+1iouyxn4Xb0pGusnxD8b0Q7ryYQ9SH+qyE/2UeYUy1Rkluv7wf2Btn653Rdm8/9nk1vOMMjbw5AyOULRGURkUoVvqYh8F7puAjtoaqOKG bK6XJDEc xlgwUMTn92aycNAkB/MkNfECjygCoQecbfCxJxX0Jd+zTgHsPzO9x0bNVISNcb4DqA2rQqf94EawrjTQIgoJ6phEu+0LYzScF38aFbwY+521lCEGh/iAZYQEROx5COTxM8YV719M5WweA0unYAQVT6uCtr17t9RMKFn15TC9GXIW5dNq17d91xyHdPCc0Rkgx7V1QR/ugQ7nWK+twjqtZv6rr2YkHdoV5VVPRVwW2AvH0ZckaKcWxDqVkdM8iLzZRhLWNlRvlZaORyGtac21hUz3ESI+6dIYZB9DQn0coBUNUGiTiGeBKjvus5+hqFhcfJd78kHGbOR3AItol+Bwown2pnl1drMpt49IX5TQ03ZpN3YQ8v01es0O+Gg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi To use zpdesc in trylock_zspage()/lock_zspage() funcs, we add couple of helpers: zpdesc_lock()/zpdesc_unlock()/zpdesc_trylock()/zpdesc_wait_locked() and zpdesc_get()/zpdesc_put() for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. BTW, thanks Intel LKP found a build warning on the patch. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 721ef8861131..782b5ad67cda 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -69,4 +69,34 @@ static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d3e27083c72..9806b1629880 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -433,13 +433,17 @@ static __maybe_unused int is_first_page(struct page *page) return PagePrivate(page); } +static inline bool is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse += val; @@ -453,6 +457,14 @@ static inline struct page *get_first_page(struct zspage *zspage) return first_page; } +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc = zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffffff static inline unsigned int get_first_obj_offset(struct page *page) @@ -739,6 +751,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -808,11 +830,11 @@ static void reset_page(struct page *page) static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; - for (cursor = get_first_page(zspage); cursor != NULL; cursor = - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor = get_first_zpdesc(zspage); cursor != NULL; cursor = + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail = cursor; goto unlock; } @@ -820,9 +842,9 @@ static int trylock_zspage(struct zspage *zspage) return 1; unlock: - for (cursor = get_first_page(zspage); cursor != fail; cursor = - get_next_page(cursor)) - unlock_page(cursor); + for (cursor = get_first_zpdesc(zspage); cursor != fail; cursor = + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); return 0; } @@ -1651,7 +1673,7 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1663,24 +1685,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page = get_first_page(zspage); - if (trylock_page(page)) + zpdesc = get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } - curr_page = page; - while ((page = get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page = page; + curr_zpdesc = zpdesc; + while ((zpdesc = get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc = zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } From patchwork Mon Sep 2 07:21:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B30C4CD3427 for ; Mon, 2 Sep 2024 07:16:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34BA58D0086; Mon, 2 Sep 2024 03:16:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2FABA8D002D; Mon, 2 Sep 2024 03:16:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 127168D0086; Mon, 2 Sep 2024 03:16:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E37A18D002D for ; Mon, 2 Sep 2024 03:16:45 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8170C1619A1 for ; Mon, 2 Sep 2024 07:16:45 +0000 (UTC) X-FDA: 82518940770.06.22376EA Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf30.hostedemail.com (Postfix) with ESMTP id F09CC80011 for ; Mon, 2 Sep 2024 07:16:43 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Hwc3Rqz3; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261356; a=rsa-sha256; cv=none; b=nBRNMbiZyC8i8fQ/PEakP7ZB1hqal9SoKSzR255evqphiDHvRv4VowqgFLQaX6CrFTIG5l 3y2bQmi7Dvn1NVWjUWV7PhZ3uyM3s0dMD945/pbCE/pIaYgTw4vD6zTgnPuav55ZgFNfTr 8XuvCfjXXQwMz9F8oJpmSYIDB+Iq1/g= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Hwc3Rqz3; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261356; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+NRszuVfACi8AQX1aYZ4ZW5Y5ZU31/p/Vio5ThCTz1k=; b=XSTkzPPEVrVeysQjA99XIdPAo9LkFy7P6uPrSByh02c/ccMWTkbJAXwCqfpZvi62rIH3I1 3VMGbn57Lz60vATpV06QQQmGnNQlQGBKYU4L00wWxhbY2wjQYGMgwvibfc1H8fZqkj4lJk gaHyRxxJDyBdMLpuWvF8ZyCy0CxvOMg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 4ADFFA416D9; Mon, 2 Sep 2024 07:16:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 72301C4CEC9; Mon, 2 Sep 2024 07:16:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261403; bh=WINuZGeFx9tI8KdY19QN4OBBU7X+cSgJrgZBdb4m5I0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hwc3Rqz3ixvX3jviF7KczOLII9qkX8ppPQNRek7rHhU8uwwExWu9TBfPEG0Z+d9b5 d1gtCEaQNeEsMRzGfr7gUwCFA1q0jlki4TeEvvDrJaXFnyrwiK2qkSTY3NscwUMmFL V8YeGKG+79ICG94273EdMqDP7cIBali7OEv5KU2y8SpOdC0m5Pv93IVlaBBvAg8goK VDA0Cp+KGinMewjf5gDY3forURqjs1gT8nJ/EqzDzoFaz7AGVA7wePPX69MVBXyFVf 2/az9VA4ty2O5MCtt+zhpVcXthqY6Zq0BoyOOC85PHRjAG4EtRmvCtTYtqX4D5aW+P HNBJVQlZhwv/Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 03/21] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Mon, 2 Sep 2024 15:21:14 +0800 Message-ID: <20240902072136.578720-4-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: qg46fzi5ki9iq33jipdpno38xaperwbr X-Rspamd-Queue-Id: F09CC80011 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1725261403-222329 X-HE-Meta: U2FsdGVkX19KvtCt7Mphw4NWZ2i+wIZ9Y+cG/uS8rZM6ya+E/FNhCiKK8CbaH9FzTDw+l480GHb2+fdtIUfC8svLK3FpmMBYG4PWwTVWT3Qty1Mb3Afdr1NhhyLYFuOYHctd155fpncFumIsfZPFxfeQNBctrZxdXXZQmi8PesfLPMFYG5ej0Tm9Fr9mJXK0zojVUr00IQkk8JXIizpUa7t7UelLtLKPkl8OSBBbgqOkeMhZZKZ8ojKsJzVL62aFniju2XcP5WQ9MBvZ5g/LUOfCz4JqiLsrE7LruPEfai3Hc5JKi14/BNre0c8tNbkajfaysyeehelg89dz3vCGbd9/td3WU+fvNR1IypUmleQ2snYyvo8FozhludBJjxZBYUybTfNgHQzLpdBPE4HlzAiOynEDd8SYLptsfHXoqanXGb6r3psGIuGT5B59E+cixXhWCVTpQL9fJFlmISesTQYAeLC9LZB/B7QePBsxdM65U4WWOLIVDp4/fzH/oJXo6dZBCpLLAGPeusGvEw3Ey57ZZ7Y/O3zF00KGuVXZVvAks44IK+UziVf2s3P/7/tBrZr9ENEKTs6TNTSUrxxCOk9DudyHIep2BZPsE1TbuSD9W386+xzHmxxHZ2mBRrmsh4ffBOvKxHbrIRlOBQyzcr0IzI1UMPhTdu/Qq0LFCjDnOBolGHjpH9dmiSjHDDremsiXkKQsQqFGTBJkpS2CiyK3XBlMALDgWjNeg5HkNV4bRzxfKggpRM+KBx8H6LB2KusuGSZnrby/pzZKbwQXmB4rkzPNrDbLojMdpUngSwKk/AAjfZuqeapToE18kZR1BowmZ4dWncpwzjEnTRONmIRjR2KsGTSVY4GYx3NV+l+aiSmyevtUgNg1ZS7dstnGJ276xBc46gAuCEcwY9h244aL0tq+1TWB6GSCKGhBVjFvjgK8VFdh1MJFyVuoF6vN6QD6IL+KZOBsXWYTPdW YBAnYqYO RrGP2k7wQBXOyp4DBEckTJd9LNEGZT4/8UfPnBBkLpT/IkQynOM4/0Upm2b8P0spoZCXJuhB7B/KVDl4KlLEUwrMTI7DbkKvUE669Ggv09AguEaYp5PnBXXb6Vc0KxiFBV441p6+9+XXHWnEtLJWD3Oq0MtvxpLvVGfcz8lcttk8q5tVAQ7DHVrIL5c/XePGt27HMpd4Ti4Qo/+fZZnr816FU3Tq5wRJ/jr4RYmIo6ginxfXTbeLXHpJQeJRd1s2MBCUDw3staleZwh41UmeQJpdhYNYzkmqKIOsdx3bBzWDqWNdI4EgUM9X/RlAsfI/smYTEUhOG9VwAWOp6JoGyVDrjDXlL/um17TybSV2oks/tkwjKgTQkZOG3ag== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9806b1629880..1b49f74fd728 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -243,6 +243,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1054,7 +1059,7 @@ static inline void __zs_cpu_down(struct mapping_area *area) } static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1071,10 +1076,10 @@ static void *__zs_map_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy object to per-cpu buffer */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1082,7 +1087,7 @@ static void *__zs_map_object(struct mapping_area *area, } static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1101,10 +1106,10 @@ static void __zs_unmap_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy per-cpu buffer to object */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); @@ -1245,7 +1250,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, pages[1] = get_next_page(page); BUG_ON(!pages[1]); - ret = __zs_map_object(area, pages, off, class->size); + ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1280,7 +1285,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) pages[1] = get_next_page(page); BUG_ON(!pages[1]); - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); From patchwork Mon Sep 2 07:21:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786828 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8BA7CD13CF for ; Mon, 2 Sep 2024 07:16:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 609588D0087; Mon, 2 Sep 2024 03:16:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B8B98D002D; Mon, 2 Sep 2024 03:16:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 432E78D0087; Mon, 2 Sep 2024 03:16:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 24FD98D002D for ; Mon, 2 Sep 2024 03:16:50 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BEC31161949 for ; Mon, 2 Sep 2024 07:16:49 +0000 (UTC) X-FDA: 82518940938.01.3A21EF7 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id 0D9E314001F for ; Mon, 2 Sep 2024 07:16:47 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WTSddoPE; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261314; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nqCAVjVmea9H++VjFGP+WRGCIqqKCdFfoxEev7pkdXY=; b=jgNS6sml+OQOlG0eYgMtAZzx/J1fb/I3sjn/T82UtqerF7fmS8xMCycVqaY0r2c2Qs5Oev iPsKEvPsvB0eXJlbzRcNHRW0jMSfS8tinr2IysZXtE1ztsnUyseSufWyNE1tdXQugNrnss S71VxKSwWmwln0pOz5SKo7B9eDjewYU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261314; a=rsa-sha256; cv=none; b=QeljrJqYxhHVb+3mE1a4KlUt+f1IhxHhRWiOJ8FsFjotVbdp9OUMipHhEkg7sRY4K2bOZq jSD5e36BFXk0mQ0pZe97sLc1T+RYdCsWZJPIaBC02gAfsN0/fADl14QfuR6ymJZzm5LLRq LT77vc08xgOMXCCkemKiYomj5M1zrK0= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WTSddoPE; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 897B35C572A; Mon, 2 Sep 2024 07:16:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CFA11C4CEC2; Mon, 2 Sep 2024 07:16:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261406; bh=XsUVOcBfW2JeyjUAJrM1VbT2++Hcy1Zh8J3Xk4CJ6fs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WTSddoPEbD5kSFh8DW5pYD4ONuFe79WI4ETo4U7BQjI1bST4rPU5bcbMIuyNXKMZN bNkFla+us9idV8Q6bk/xzhSxvRVyFkHC1XBCvLLJiz2CDIof7H2AaHdKdiAlLaEokN FBf1IH1mbZtOu4IsLPkCiWx8lU8624B4xKqIwEZ4Rq2nEwz9qzoAjwwHyk+OQ9GX0Q w1KoJNH5pGaKuNymf+b1bHL7jZPtsXyOW1IZ95yYbVEvJW2K0fsiiB4sYRoL8ZKJkN Dluez4d3GOKyB9AfxW54urTAXehe81vQgiYWrnQ+gUOcMQx3OxbTm7WAbQH498x/4V QUbQiC8L55T9w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 04/21] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Mon, 2 Sep 2024 15:21:15 +0800 Message-ID: <20240902072136.578720-5-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 4ps47ph3giw88goygzfd5ujsgkq3xnzj X-Rspamd-Queue-Id: 0D9E314001F X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1725261407-609665 X-HE-Meta: U2FsdGVkX19dQRHNHBW4FK3iWzoNwsne3wFaIKtNYBnbJaOMku1tqJKED6L/3Rzw7hv6VaMxd8MySgsJnB4gqPBcLRXuHAs8uiOY38NniqrnbYXGDosKEsuk/mGvQycUxqjcgmTOdagIk0mLG17/OGf4I+ZwgyD2MTKKSy/FpwVuqc5KFXh/iHyj7hSRluB19TbSpa+IyglX5/Pm2hAIir6umL3s8Eyu6IcshhSuQtunMRZZRxg7cafvSS+m5AvSO2jNB8rkPp3gMHELI2oH7yJNJeXr412GwlMOkkSmxRB0WIW2NMdgdqz3F/thFjHuuLBelJ4SoNdZYPty/ZmPxIu6eN7C8o4TFNYUyZLiYW458scjJmx7A3O+GyB8FZ2v5xCSxhhUZGv2bRHkFR+TEm3zP0vw45KMJcSfBAsxe11B8w9oeNhf+KNRXHJl1yE0ZNW6ddyGUeMHfa6K/GUpj9BVbNHKPYxbBvxXLBIgly++Df7I9BoXNJApkLEv/VnPzgZBni3FHx7YCS435wBDPvovAVv5md4ldxusfJVY+2T9i0il9uaofN+Cq8AVzPe6smlQuRJEnvjuYZ/CzFCZ3JMx9rclgxwBrw9avwxcmgT04IAecdpJwiZBdnXTk38nS2ltnyE5hbcdEhCv4WK6SKwj7kdA5JVkU3ArMezjlTkxKTiQBu1X3plvlYm6vmMp7I09UgL8ytLlOWlmfYsfLsKVcFf1urvkK8yF+hCAeg9HSu5TbMQaG/cIizoPUsO1Iz8lGd4fE5RXjItSjRH0njKZzkioI41vfEFciubSkWMazT6k7Btxb5xybu7E8pbW/yc7F0CIzlSZqifCFmgm5gJxWO24FvY/kr8dGapGDA5OuAFZWjYkXvZSBiuUJ3kmrGa6zE20KeWR9wbqvwM0pH9v04ZasUDs3rIYfplMmA+yMEYV43dwAwsL997okA7pE1Mg00qJ+gyn73JpA4z lF8f1TGO RJjeM4i0KgVKGPjZa7ffw87pLY2dTpGGpw3WqfaHUJwVS2i0ig+webIUmSeMgY9/7g9BRF5S6nPpIxcpAC0snJDoFcA8fyB/qCd021Ec3zHbHXshBXkVoYCjQwFADUJSvDtK9tiilrt8w5A4Hxqff2F4TnMUTPiSAeMdP0kpYLmlPQHJ38R0/cPkMMivMLZfW5dO0eeUf/qmZSiLa3/ZI/V2oXz7jdNpvf7Wpvgiw1bqGFdcdNaxKfEIxVNJ/VyBi0YMl7QCP/pRFgGJ9ubmj9lfmpw31L2MIEV3GU9nY35Mb0RduBQq04oC4MzlpCQvhQwOFTUJPjgfd4C/l09iavirr4Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 782b5ad67cda..11083a1c2464 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -99,4 +99,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1b49f74fd728..a7e73c0de039 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -767,15 +767,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) } /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx = (obj & OBJ_INDEX_MASK); } @@ -1201,13 +1201,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; /* @@ -1220,8 +1220,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1240,17 +1240,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr = kmap_atomic(page); + area->vm_addr = zpdesc_kmap_atomic(zpdesc); ret = area->vm_addr + off; goto out; } /* this object spans two pages */ - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret = __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1262,7 +1262,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; @@ -1270,8 +1270,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) struct mapping_area *area; obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1279,13 +1279,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) if (off + class->size <= PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); @@ -1427,23 +1427,24 @@ static void obj_free(int class_size, unsigned long obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(f_page); + zspage = get_zspage(zpdesc_page(f_zpdesc)); - vaddr = kmap_atomic(f_page); + vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next = get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index = 0; + f_zpdesc->next = NULL; set_freeobj(zspage, f_objidx); kunmap_atomic(vaddr); @@ -1488,7 +1489,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1497,8 +1498,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, s_size = d_size = class->size; - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); s_off = offset_in_page(class->size * s_objidx); d_off = offset_in_page(class->size * d_objidx); @@ -1509,8 +1510,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size = PAGE_SIZE - d_off; - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); while (1) { size = min(s_size, d_size); @@ -1535,17 +1536,17 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (s_off >= PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page = get_next_page(s_page); - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_zpdesc = get_next_zpdesc(s_zpdesc); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); s_size = class->size - written; s_off = 0; } if (d_off >= PAGE_SIZE) { kunmap_atomic(d_addr); - d_page = get_next_page(d_page); - d_addr = kmap_atomic(d_page); + d_zpdesc = get_next_zpdesc(d_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); d_size = class->size - written; d_off = 0; } @@ -1784,7 +1785,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; From patchwork Mon Sep 2 07:21:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB0E9CD13CF for ; Mon, 2 Sep 2024 07:16:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 593F48D0088; Mon, 2 Sep 2024 03:16:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5441D8D002D; Mon, 2 Sep 2024 03:16:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BDAF8D0088; Mon, 2 Sep 2024 03:16:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 19D7E8D002D for ; Mon, 2 Sep 2024 03:16:53 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CBE5B1619A1 for ; Mon, 2 Sep 2024 07:16:52 +0000 (UTC) X-FDA: 82518941064.21.041B1EE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf05.hostedemail.com (Postfix) with ESMTP id 3BD27100008 for ; Mon, 2 Sep 2024 07:16:51 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=a24F2VQn; spf=pass (imf05.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261388; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XUKJ1MZc1dZkwnh27HwLVGPFWxYRzt9OnfExQkxqP5g=; b=bhxnMXLgydbt50I7wH21/BkFc4qfobBY0FFnUl4BeVABAeDn4W7fReQvy1WbDSG9LTEYI0 6h/Px0GbcmImckjRb7dF+dN9EbyjVUfODiURN47klUZIBbVAIm+pmIBoSSycSEdB0LKatK /ID0ZMMbpyOxfvbijbos2ywmutwGlQ0= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=a24F2VQn; spf=pass (imf05.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261388; a=rsa-sha256; cv=none; b=05F49hZ4gG5iLYJjaslVY+CUi71xja7LtRXtoJewffeVeMfh5ScXszCv6TtKm/Wbpy3vzZ lxz/+0lxdIJSqe4qmRYORjWpwcEJq6ng6ccFP7XILQZK02W9F3JEQpg2nR/An2l3bz7Co+ roazorhu0xpqTaLCI5J7Higuyq4vhsA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 118295C2700; Mon, 2 Sep 2024 07:16:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 61482C4CEC2; Mon, 2 Sep 2024 07:16:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261410; bh=8cxk8W9Xo69SGYBFRex8FDtZ09OrBx21iVwdBQgpsTo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a24F2VQnLxBeAfFgzkncMF3h1rCsz0+HZE0mPCQdzHFzcAAnrpDMYveicOyyg990A HmRfhw0p7bfJqGm+RryWkBgUWvMhBgTW/uBJbjvf7YLCMIMB0Vhm/l8hsaxn/+mhnw 7WmVG5f14hJSFBysA0DWgujyfNTa86E7yzPq54eu5mJsGTnETUbsPO3OMT5UMdw1Gg cz7Ui+tx7hmuxYk8vVOCa9FxXonF+j/ayOHUXTuIvnRQlVH2Xg9xmlOKZML4wtJLfF zwn28zh8Yv7cNLIzHsX8TjaUmmFlNH8WAh1EUFEf6gINm6luljXGdCM4Tb2JV9Vs6P eYoSK2APZFQcg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 05/21] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Mon, 2 Sep 2024 15:21:16 +0800 Message-ID: <20240902072136.578720-6-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 6ess51pdytp6t8umotmtak3mz4psra5y X-Rspamd-Queue-Id: 3BD27100008 X-Rspamd-Server: rspam11 X-HE-Tag: 1725261411-325754 X-HE-Meta: U2FsdGVkX194X6AOGhW4T+G9d7LdZzC8EePOJWD6aHGOJoj1KgCFlfdpu3AeeGs5ep6Uk/bcWvwxS5HENOHhYSeny9xEQBP9g3KUWCUjLVLxE6Q+zEWiLtj/Nht62IZo109Ht558LIc1rDKlrz1F6keVW8OXAUB1hwoNd9qFSgi3hW+ag8KKKwoNWV8cZpEQ+h9x8s92BuvkVN3aI6gfnCuDZMLQg4zdroTuFJnb6DxEcxx4z0DHvxZODd/lj4SK98YsJhMz5xldphD7fBSIvP/5cfllwt6yWNZaYVJ5RgYr0FO79Kv9DuehUb6A6l8PF/6noXNOnX9cnmAjncXLoWEgMdVey6Hv8x9h6/OCxyJpDg35x2RIk8hLO8nrR0DjPiUBC5Vr2K3b5ChDOIdY6vUC8IopxR62rk7bDXR3JNivoUUCbmZX0xza7v8/ypbwC4/wJYSI3KchqZXBZ/ppnrDv5NYWnYif3C2vtjxWl4V85GibxkgqFzdw57pWvVann4Gkbp0tGr3PYWoQbVTpq0OyUG8IDpUOIEQaDzJddka01aZiIM5iYPatp8tlIIBXEvBDlwld9ZfDNOx/uGEZsaWm87nJ7VnFvpQRWdaZKEco/QUYmUPWO4HS99rnKv2O+bajYqN9yOEnbRQNqxOS83z6VT/wEnEAelMY3JCuu/Dga4SzKpnMj2S9Hsg4bF4RT6wgPJ/lfaPX2uwBGtFK+wJgzbCBq0gTjyc5Ht9lo+WVsqChsKucb0tTF7xoKqK8vBuAfE472aypOp/tp5IdMiR3EtXfbv2WnWFS4QcwPJbVcYcDxKrfszCXdSNZl4+Me5DRKTYY93y62tbdmlUOy4pawPPs95z939FgAZ1xcP5iGSMQo1bOK/uY4Eak0a9OZsCcPx5+HGXLzIhGV9Taz46oyEjCHcIWK4d9SvIwLnuKxv+PaCflNFDuuHdehlhn8jIqjoEyr1qSbBoAlIT gnLFrX/p gK2L27kG+GYSU4I3B4Bg+i8+jbx6axIBoTHZ/+GvRhfaQxXzgWSfTUTQ6q7f9xSKvc/qV/HXGAkWx0E7H+iJOXm1MIArNPmo/VA1wnYsLrVLdhKuJftU+8P49ekQVPqJHe4qhmGwOhozrAl1MACZvUznhd27nr8fSedKdKe+04cUoHCuhozuN6gmLKO7bI1gMuOaNcUuaPnKw/w3BJntSOuDDGI/Uw8j7Nb+MWhx0yLr4LXH4LwIKP1kFI8km7lTXsbY/GClsbQeNaAowh0UIrCFz2TYFO8XffyU1O2oBW8E+SQdQweQe1kwe58A+l4JqVs8raTxVAHw+aX5xIytEE5toRc1upH9WybgrBx1IuLuvF1xLcWFynJWs/Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a7e73c0de039..664fff88a7f0 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1315,12 +1315,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; @@ -1328,14 +1328,14 @@ static unsigned long obj_malloc(struct zs_pool *pool, obj = get_freeobj(zspage); offset = obj * class->size; - nr_page = offset >> PAGE_SHIFT; + nr_zpdesc = offset >> PAGE_SHIFT; m_offset = offset_in_page(offset); - m_page = get_first_page(zspage); + m_zpdesc = get_first_zpdesc(zspage); - for (i = 0; i < nr_page; i++) - m_page = get_next_page(m_page); + for (i = 0; i < nr_zpdesc; i++) + m_zpdesc = get_next_zpdesc(m_zpdesc); - vaddr = kmap_atomic(m_page); + vaddr = zpdesc_kmap_atomic(m_zpdesc); link = (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1348,7 +1348,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(m_page, obj); + obj = location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); return obj; From patchwork Mon Sep 2 07:21:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786830 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1907CCD13CF for ; Mon, 2 Sep 2024 07:16:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DB378D0089; Mon, 2 Sep 2024 03:16:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 98BB88D002D; Mon, 2 Sep 2024 03:16:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E10A8D0089; Mon, 2 Sep 2024 03:16:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5D3858D002D for ; Mon, 2 Sep 2024 03:16:56 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 23FBF1A19AD for ; Mon, 2 Sep 2024 07:16:56 +0000 (UTC) X-FDA: 82518941232.03.97C8A0C Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf01.hostedemail.com (Postfix) with ESMTP id 8DF1340018 for ; Mon, 2 Sep 2024 07:16:54 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="oeTHe/G/"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf01.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261310; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SDkuZXpuqLVGZ9taIyAv3xfe0cRw4bHpZU2uD4KumLg=; b=VlK2uoN3oziMj5OipSS3XBIwrduznfMwSppudDbdQBm1H+J1nJNcS83DriU8cDiWy/ZF6d qLLrOqNoEaRwP2TWcTZ7UJop9hOtyrUpVQJJJ/URAf+9yabVTXdjD40fYES49oGtgGGMeA 9M1Yb9TOFD2+aQFsxfzsLq0AuGOx9HU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261310; a=rsa-sha256; cv=none; b=nxv1g7v+OPw75Mi1fhPbRMxYtaPYR/tCWch+ycMWReMDLVDsR6bAx9wUIDcHiyvcDxSdAi zvHeyufrjnOjGfBiP5QR+z/ttAiRm0f2jOt/tYKNIQ+hq4sZFQIZFkLaTy/mc0NA0CEqyj spkwWqet7J6wr0PnQViu7YaVR4IHqik= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="oeTHe/G/"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf01.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id E403DA41719; Mon, 2 Sep 2024 07:16:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C2D43C4CECA; Mon, 2 Sep 2024 07:16:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261413; bh=LtPxxmWpxhvwgd6evBIlBSmgc0+AxTDG0ppcncG0AUE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oeTHe/G/ry/99t6lVEMFyFRpCzzGHE4tmMQdoPie6y3EhRhkY0ZQ9axzBbdD0XDjP DusqDDUtP11Vi9AmxWicp9ESzmFmMArZFk5ta8D+vj+PcVoHoTOvclom6F9CNE2Q7x NqwNMZzRt9JsUw5dEEDcysjJhNRyxXPTvBFenOH81yRxmGnvRGU2LF/3KPmDe3Wtte Bep//k78yEXeZGCB12dLrYyMyvMsmeuk3xdkjYyg60AhADg1syXKU9jHxJGRmkA+Ko TLgnEDcV+YakWCfs/M4A6tOi/arCYKTRpuW890jGwlxCqTnDoM78WCg/un3uzb//Yk W3QSnOsWnKmcA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 06/21] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Mon, 2 Sep 2024 15:21:17 +0800 Message-ID: <20240902072136.578720-7-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8DF1340018 X-Stat-Signature: irpr48pan8pyqw7mzo3gots7jjkci1r1 X-Rspam-User: X-HE-Tag: 1725261414-596828 X-HE-Meta: U2FsdGVkX1/iIOgRN+5D68+G5DnHR2ogpqZF291HhW5bwdOl6kjvmVvRRtdsdjSAKT3f6tZcwq3f5XR5gY3g7FRQebv/Llky0Dphh/I2nh+s5N0v6qGJyu+sToFISEmMDzm/9596sEf018Ic9KBAgjclC/TjNjWNFMa1vo9WAXUgRwKXbg9Gtnw8C/R5PuNHPdvTHOCr1Mk3LsWsQm+cZXZYQSqUP1OoPgJZbm1OnFrRs47pMG67eC3IUC3+mI2Cf6M3CShzM9zXwu8ZXCaqG2ZuDwPqyspB+Ll91D5KczolriTkI3J2KRzKVmqjQtqGTprcq3088wtWUf2vtqnkkFSdCuACarmkPZ3ugRFVOEUFGzt0Y8Yqi0EWRbucnLvklM09DH9mMEvh1/iWzIuCZssJpo4mCephx0Xr77qK7NfRXdhIkUbjWv6RmaUeFxbz8xtjEyvLq7S24GKGUtNTrkaqF1gNf8s+7hQg9Vy6WOl+cYcsJqbXcxsakzPI2rg5MNl/WIXiq0XEuMychsnWnIo2hJ7K5mZI5bRq4tmPBXzF3Kb9Q2p6PsWDO9axz3pletg0WVYXZXLBvywtA0OL3+kVwg9+lWBOQh9HE4Uo+ntpa+1V16QyDAXVrWzNCwUnvngS+SPJYleTHWFz0TIvDI0My+ACdCmcWbRs6wXp7uOO4RWM59outQRWOZWXXeIoRy+dq+QpGwQiYQL3pdASO6DEAb399ThnMdZWjylqTE5NlEH1eXZBBoJ0FSvAimkkQ0fnq5YiYOTozp7OZMdxT6J6XrvqQlq+aR1Ul16aQjBzKKpqXMjkBBFbVEr6CNlFByfNDEW3WFcHwoabO/EHGsccM1EIPGc6mybzeUZY2BOwoFO/R19RlbVEW74lAuuLtIL+DRna1sdv/KFMLTOAFnPlJ1QyNoh+E2lCPm22eiRf/Glv0sDPALfybIGD0PAiH+/GCQBLr9qc5oK8LN9 1kww0N1R /1Fd3NnK4vv57FUmy3wdWJ78i+DmcQs6Q4WZE6zcbuMRMF2kLx1cLhjSj/4jP9+J8fjNLVYYH9J2oW3Q7Lf77DwBAOWMDWMHDxhlgmHegUze51DHfRmRiXbT5YW8eqNDKR+o+kygJvxE5eK0wosD+vHmlm42ikHOOzxEK/67vrnQkobMDXySlITWTYcnYAliMbs4f5lIJlJilVoSZy0kb6MrXjjYr74NTS5+QR0GQfNXXdNWQdlG1c5rjgBt9oC+PSsF/SXsYlZ6DExrBTQhsQuDAZIBVQs4Zuj3MxKZDp98Pv3PvK5iato1aYI8lQaMZmIDfgRyL3bsz86kttYiAcM4XgB/vaCaG2uIBHJ7EdSMS8LoKLyXf3VChc0pc1s6wqq//HbOatRRHQSM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chain() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 109 ++++++++++++++++++++++++++++++++------------------ 2 files changed, 76 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 11083a1c2464..3a65a7d494b7 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -108,4 +108,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 664fff88a7f0..6afc3e642e02 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -248,6 +248,35 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page = alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page = zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -947,35 +976,35 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) } static void create_page_chain(struct size_class *class, struct zspage *zspage, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page = NULL; - int nr_pages = class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc = NULL; + int nr_zpdescs = class->pages_per_zspage; /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index = 0; + for (i = 0; i < nr_zpdescs; i++) { + zpdesc = zpdescs[i]; + zpdesc->zspage = zspage; + zpdesc->next = NULL; if (i == 0) { - zspage->first_zpdesc = page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc = zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) SetZsHugePage(zspage); } else { - prev_page->index = (unsigned long)page; + prev_zpdesc->next = zpdesc; } - prev_page = page; + prev_zpdesc = zpdesc; } } @@ -987,7 +1016,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage = cache_alloc_zspage(pool, gfp); if (!zspage) @@ -997,25 +1026,25 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, migrate_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; - page = alloc_page(gfp); - if (!page) { + zpdesc = alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >= 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] = page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1746,26 +1775,28 @@ static void migrate_write_unlock(struct zspage *zspage) static const struct movable_operations zsmalloc_mops; static void replace_sub_page(struct size_class *class, struct zspage *zspage, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + unsigned int first_obj_offset; int idx = 0; - page = get_first_page(zspage); + zpdesc = get_first_zpdesc(zspage); do { - if (page == oldpage) - pages[idx] = newpage; + if (zpdesc == oldzpdesc) + zpdescs[idx] = newzpdesc; else - pages[idx] = page; + zpdescs[idx] = zpdesc; idx++; - } while ((page = get_next_page(page)) != NULL); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index = oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle = oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1838,7 +1869,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. From patchwork Mon Sep 2 07:21:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A348CD13CF for ; Mon, 2 Sep 2024 07:17:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EBB798D008A; Mon, 2 Sep 2024 03:17:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E6CBF8D002D; Mon, 2 Sep 2024 03:17:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE61A8D008A; Mon, 2 Sep 2024 03:17:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AA49D8D002D for ; Mon, 2 Sep 2024 03:17:00 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1726D419E5 for ; Mon, 2 Sep 2024 07:17:00 +0000 (UTC) X-FDA: 82518941400.22.DD1EAFE Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf23.hostedemail.com (Postfix) with ESMTP id 724DF140009 for ; Mon, 2 Sep 2024 07:16:58 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CAfKDYNX; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261372; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z9k8kc3Inzc/Q35w1e/KpFGxKa6k+uyrWulTPS8rXV0=; b=u5aPoM9P9ON9ZSuvypZJ2jUhI+njwwXZVkugd5sQj47cFh9kEhOaiPD1IVcdZR1V/t98h7 DiSESvqhaCjeMxrjp3W9/rGfy1NG6IkTXkuH3GAotFW3fvjQPV5uWiVteqYjHIvEVec+zJ H5pa6DZyuvr4F2VRPqIepU9QyOnV5nM= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CAfKDYNX; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261372; a=rsa-sha256; cv=none; b=xFt+rjXkdm3+HIE/uiKEH6FZocHC8RfhmiW8xXWDngTzBUgKreCEzuDm1RgpCpV8d9xnoU Zah5ao9xlq+WCwW+oHYc+sGKtMw1wJTVpGPlG/hDP7zxekTL3WH/lXgOvF1y3vfvRzNvup 58r4eVi17q2MEQ4z3bbTsODJwz1S7XY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 93BA5A41720; Mon, 2 Sep 2024 07:16:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 533AAC4CEC8; Mon, 2 Sep 2024 07:16:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261417; bh=Br29/5+PsdLeZ/2tyuIzR5WeYQvl1vCMxw0jQ4y7I0c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CAfKDYNXjkGUMjjB3LlqQwolWE5LfDVmqwftcV6wvGJkYybv1hqip4//3RUWCXBE0 YopBu9YKpW1ldrbR0e9ShcXXpwmaI6gIc/4ttV2oKrteRG4M1HKS65E20+U+R3o7yM OAltuYv3c4EPwZjcbEtD7NJKJ3LGTe3YNRi3Oa5DHNCXVhHT29BWWzkUcwAuyPHaUG t/uhkkC3MUTgkk3pxWxY+yC6uFQ5ms5l1hG+8G7ElOlF8KBnl1Qga5dNieECRmUhUk 4jwlkCj/prBwfMUZYq9Ht59LMc5RBEF9mj17eEC2zhUSdTCdi4syzKXAvjIeSQEr+b d/CkesoRyrHHg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 07/21] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Mon, 2 Sep 2024 15:21:18 +0800 Message-ID: <20240902072136.578720-8-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 724DF140009 X-Stat-Signature: 66zmbj1f3mkzou7d8undn3x73a8ufd5w X-HE-Tag: 1725261418-775244 X-HE-Meta: U2FsdGVkX19j2xWjCSC8idwGYT9GeUidy0ivoETuQwV0EM+ruIggzP9cFUL/8KqIiDQU2PogY6sAxI+AzJEvccqcwgyQ1DqQyum6o0+8FNP/cqKBcLNeapwM8JS0vC8kDE48LdBCL/o4bebHzWjjZIFNocn2fbywb75dIey16dW16SR+MAnBXx2S3VWqnfF28/QH82+3q+5QLvwn5jAZiS3deqiXw2i2sFVbVq/3+8eym7DmxNRy8FzTFQTY/nIPey8De2rF87U6DYo75Cf40aFbx4BSixye6ebO3czNatMp2K7BXdECxgEdhqU+RRuhiBreCLBW/XtfZurzPyS4rej7OoFTTjAztId7IVZoJVHztr5ONX3r/dKFTfwFQg1m5NlWzzd+fv7LX43Bzgjm63/ysNSFIcGIk+OMiOLwgKxyN36sKgdBdb8vx8ZcyTxwsT5Lw5KBz54IsU0fJea+Yq6BNeaqfYvbyHIeoQm4QeWrOrFgGqE5mrLIoOpJ2eoxYPodGb5h3jpcCmTAwd3Z2Ymgp4/R0V0BIHejKz7Fz6zqI+9jj7vNoFOehB3XEy0aZONvPRpsSY5oRpwj1fDHbzON4NyO6G3gJKrayCeqZNxG6ThItcXDR6GRzBhw42Vm1EoZg2teYSXnbnROM1jNdHpLlYCc8b1F/tJVyM5NmNLgPrgNnYO5xwq5oHCswP8Mf/ym5Zkafmnm8LPb/pTa7zf18DEa5LO3jq0vrOJJSFg4ALnXPYTXYbkYyRHMvzDYTPKlp0/AyP0JTJ9wLfnuo13yjsbVVw1ucecge9yKfy6/5KF2bKe10JeD1guFgKwyuWYqm1N45yXUfV79e7mMxdp+RpS4P9Ba6x5DqmyO24r/5tuWMBNFuFUD8LeOZaFd7hSZkBn0R9hAGTd1X6q17fIf04HqzLqtKQYBisGVSzDsGc6SnUOEfksSpJCu6vsyRnaVMIX2KdfPtylM/gD QvX7Uwok qB0HljST72d0w2QY8nyN/YFznrIUy5h5pB/zYcKvdKZoJiFgM5g92ilSg5yrPSj6y9H7OIK+/zpBZbFRGBqk7ib8KHW/WyNgVQwm9OgxGCbgogiXYRsUq1kqpdapLeoBBWQKgrCvcMX5y35TfrZBJX+im34k+45K6fwHiatDMOj4/8xr6HqmtqEpgHajCmIZo0fnjyrYCMSnEZ1Mt0rXmBN+8nWHTnVIR1hOh0Vf9Xf+9Zi4L29vIZ1Rl6CTkLwO00QfuRVGft0DLjN6HDWxCfZjuf96Cn2Hex5RZND/9LwXSfxvzlKt/1s9PFTg7s2d2vwjCxUPMaDj2ThLyuzQSdG4OgLysgBVi+i3XGgo2KLlRf0BySNyYIcWz9A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6afc3e642e02..46d8be5cb4b7 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -833,15 +833,15 @@ static unsigned long handle_to_obj(unsigned long handle) return *(unsigned long *)handle; } -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle = page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle = zpdesc->handle; } else handle = *(unsigned long *)obj; @@ -1590,18 +1590,18 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index = *obj_idx; unsigned long handle = 0; - void *addr = kmap_atomic(page); + void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); offset += class->size * index; while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; offset += class->size; @@ -1625,7 +1625,7 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, s_page, &obj_idx); + handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page = get_next_page(s_page); if (!s_page) @@ -1858,7 +1858,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); From patchwork Mon Sep 2 07:21:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786832 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24F1CCA0ED3 for ; Mon, 2 Sep 2024 07:17:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 853A38D008B; Mon, 2 Sep 2024 03:17:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8067F8D002D; Mon, 2 Sep 2024 03:17:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 67E9C8D008B; Mon, 2 Sep 2024 03:17:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 40A3C8D002D for ; Mon, 2 Sep 2024 03:17:03 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EBA86AB0AD for ; Mon, 2 Sep 2024 07:17:02 +0000 (UTC) X-FDA: 82518941484.05.761399F Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf19.hostedemail.com (Postfix) with ESMTP id 694E71A000D for ; Mon, 2 Sep 2024 07:17:01 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Tjz9s9qe; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261345; a=rsa-sha256; cv=none; b=b8ZnVKxE8kaEKDEDCDrl+sKb9PPtcvAc7ik9J7W1eU/1jT6TQbthgsCVfxCCBVVxJ9AQz+ Uu1Y6zgTXpmI/znRdF8TODnHLBlRq4OWkDqV568ITpryqV8j0MOwMjJ5UgcPincQKkwZWG 40C6SZ44+FrXL1fmeJvRo3CcgqRDtk4= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Tjz9s9qe; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x4TXNLnMsdl0fNmTeNSaIynFkauZj0U2N4u9NyBei54=; b=lg7AHZHXF+ToBnXiqVNxsQOnmdomi14a6ziALo8KQb2f8bLBUpmdLk5hAsfurkiybvLOiR i72CnoCAfcA/QURGYCi6w48NXzLEt7oXlDe+YAwPZnpZYgi6g3zFBZQHgE3B8x6bt5PzD8 LZ74ySvrzRLgS36FNdNVwASjXkJ/ZvE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id B0DC9A4171F; Mon, 2 Sep 2024 07:16:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3069C4CEC9; Mon, 2 Sep 2024 07:16:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261420; bh=PNe+PCWyOI4z40ocTHR27DhffZO/+69yc298dwmlSes=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Tjz9s9qeCkxm3K4mcUE5debLCDqO25YRhRFWJ/avcEDhpY4b8AFlwwm6cq1qeRVSM TGzZdBTjO0kyKpbF/Z7DIxsvEizBwlGCw9sJadLQCvWyAzoTTek19qzxsiyEkUhEak u0KMdnQ6O+BTfJOhcIz2D04qnh0DCeQ4IOyceUrOokKwmSLO7zYm58lT2aGv38l27n ISkuZaS3xI44QuXK7+MXr0ymkpKO43OncUVwMQJoGL/cZpb3Er7s5xJojvxNtbKHEN vMl9o5p/VP+DNo/4sqQvC5TySyWuu0k5MImEY96NjGNU+enX0pdUQJ91RPuVGMeLfg dviBzU7NaaG7w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 08/21] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Mon, 2 Sep 2024 15:21:19 +0800 Message-ID: <20240902072136.578720-9-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 694E71A000D X-Stat-Signature: 1qkm65p9wwxzi8xy6xta1qarutwjqk3z X-Rspam-User: X-HE-Tag: 1725261421-529507 X-HE-Meta: U2FsdGVkX1+R90Fb7MHUCZyqt80WDdZlDayODUyIwSUkBl5Kz1X5whMgYkd9nNPf4LIqB5N2ARbE9oYdsuvj08wZ/WHwgKay5GrsIjgt0pSmCE8iZnn+UuAmPbjkKcyzDb5u9+KuHsFyzaqGPxto6Nz0Ir2GDnJ887KaONufEEyyb8JRy+OhCMez62xF1mPDqNLGKiRS3eKheuhGB8Se5x0EQN+ZUxVb9jeO9gTSrbDaiw/dHQvzZdKQEOw45GdxPTto7IFv21nDwRg89hbvKGhhjgbf5vap4hkaJFm0Z8a6NRsp+wxsgexgbTqLTJpTMCCnxDvpU5SxTe2LVFo9GdpAgkNRQvCADAMaBibN2EYc1h9LwGmCIWSoOBlEyMDoXzzoPcDabRf/H8qqSPYZyhYgDL1grIGgvGbH05hKVIWvmYdBgtGi8X5whLj3X/nHSEZfCqS0hqVjPtxjb6KHWuCQnvgmPjdSEgBzU/TD74vqAo0TIxPmr2lV6N/LXUmOfBLm1D2pxEn5/5eWjSCreuG7OiEmfkVkgIAUXkWHi2JVkc78JOkgSLWFXBLIcP7g75BIKPZmwWztgzof69oz9fnF2ShmID55go2YBz/NyLUbOt5rC74nKU4oF+R588yNfFpT6veYaTzJDWVHI3hm1Zkb9nYLsJxNaSBdRwEh8QRZ/oeF8pyHnSgaKvQXXpuMYDKLdmREOy9mzzApU3y4faFIcCAaFGIs4xHzLJeStgpteoK5FTu7oAM4e1HjxON2ZZiY+cdmQhsO+ROme2jY+3H2FZgUY7tfgrkuDpXdgFsF0vSeoCbbz0lrRV3IBY5dR64vTULI+or3pnLL2rYhdKz2QX6pLcSK3BOny5HIDHjNvt0cEqBSoVn0iKe7Mv491AJDqnLcHy13EWTjEGvqiumDN0bxDXuBw2Cj2bNUAC4/OY9ZDit6T7MaQ6z5k/22JFb0D6asvqvkRBO1gd+ AOuCNaiT Obn561gF2SDKa4wzYPK+ir7B0scX2gXnfCgqpNCdu4L12Hcmvw9Cg7T1lJI7Yb0p+wITS5xDaJjTU+7656AdpbPzOgCPFFMSH0fwYZT7W9MPLXTNjLqKHqomUGklokruYlia2GI5dm1u3Lx9iPtth+ybU6zQAYBLQ63filQYQ7QoFj+sQ2ZuAiGup9deHeT48uI5K1+5TXYLm4s+xnMd1XxveM6JRK5IZNue6UCG3XUInI/CDav60zNEdS0Aov3Q2QstJmd/peH7D5gN7mdzP2+afvfNH/MxbNHLx7MU0lUFhfdazNxwlniaBol6U2aC01d3EwTRYss3owKWQvvt2QowikpKaD23DglNAG3wAt+U9Xx8BRNNjC2nChA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 46d8be5cb4b7..01c5be6f4c96 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -935,16 +935,16 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) { unsigned int freeobj = 1; unsigned long off = 0; - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); - vaddr = kmap_atomic(page); + vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); while ((off += class->size) < PAGE_SIZE) { @@ -957,8 +957,8 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page = get_next_page(page); - if (next_page) { + next_zpdesc = get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next = freeobj++ << OBJ_TAG_BITS; } else { /* @@ -968,7 +968,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) link->next = -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page = next_page; + zpdesc = next_zpdesc; off %= PAGE_SIZE; } From patchwork Mon Sep 2 07:21:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF6F2CD13CF for ; Mon, 2 Sep 2024 07:17:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 67F1E8D008C; Mon, 2 Sep 2024 03:17:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6078F8D002D; Mon, 2 Sep 2024 03:17:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A8888D008C; Mon, 2 Sep 2024 03:17:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 27AA08D002D for ; Mon, 2 Sep 2024 03:17:07 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D2989C1948 for ; Mon, 2 Sep 2024 07:17:06 +0000 (UTC) X-FDA: 82518941652.16.685768E Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id 436E4140011 for ; Mon, 2 Sep 2024 07:17:05 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WbsWZzPz; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261333; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+i7qlOLu9s6ZZFQdVUBkJ6vmPIqVVf8R13g8kxpIl2s=; b=A8asG16gpr8CjGNLoGk1e7gQlx/cvtok4j0uV9fqM8Zhql+LfIhDdd+aHgc1FaQ7lzavaz McUdVSaCzm5tsIRBakLrAG/SCrC2Gd8ul8y+CoquwMUbhEJZbc2QqCyxdP48Nfww3ovPBM b0Sk+r6ccXiqDLKh/Prf90GCWj7blyE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261333; a=rsa-sha256; cv=none; b=lG4Uue7mOgZ+yE+kS9wiu8sXoZUicPRMmX8BVPsMiiuzPooih4PNCa1LQdx7yDdQx38RuO dzXN+DCcypm93t+qjnqg9JxY2ySK0EprMbBLqkHQqsnyBGQFzz6iYUTx0BISf/JPHHZYr8 w/by0XJZdO1B7oVi3KQ+npS308J5/nw= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WbsWZzPz; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 18A7F5C574C; Mon, 2 Sep 2024 07:17:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 422A2C4CEC8; Mon, 2 Sep 2024 07:17:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261424; bh=sa8ddOMDXRaYhp/IOg2wSEnMgXawFHsto5brOpBu4jE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WbsWZzPzOYnq9VACFjLbHT0QtTboQMrs9Cl5BnEg42AvkulcCszPJZC4ZgWJ7VBIs aa6jHlKKy7UdoTM3zYo5H5sY1gxJVq8u6gEGeIGVjHyKyvlMx7Jw+fs2WXoRrqOR6k B3lHLvwe6ydPue5clUvlCNMVXV1PbfHaTfNFXvS5C78imrXR4D+wTnakGT8ZwdzMak H1fg402rc99rP4R//IJ5X02R3KMzo1XrNoEQMHSIEhTNiwT7e4352mPk+9CfuKWovi EMzxRlUou3vQ/ZS/xQiscEZrpsDV8+bThb50Mn1FvN2/rNaA8KQJEsw34L71nnOBAY L0XGUwN8m4lWw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 09/21] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Mon, 2 Sep 2024 15:21:20 +0800 Message-ID: <20240902072136.578720-10-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 436E4140011 X-Stat-Signature: qfs6ihms7dbpewr8jqzfem3j6qct195y X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1725261425-81404 X-HE-Meta: U2FsdGVkX1/4prjMmxVyHxiRK30YUz7YC/rEGTutmyqGsN/LMQDh0ZSq8VoVgRIUEqCZNF5T1RsxKTsnjFHCPGpbkG+OEXIZ7gwbRI5zzxISSMri3c7IRVkQs4s44K3HAK1HzfJPYh53KxDQVGhKOQyd/+YyuPz3eJ4a0LORksogp35CgidXfNLtDQ8YnGa0CykhsAG5gJmSp1njWPxtum4ueu/qo9SNU7E/vnC8lmb8XSUJuLNIqXP8LypV1QZ44xZCD2PKga2lzkP2F/HzNYuhwD4N3wEAHfnwXXqD0JfOOFPcZAGpxn44O2r29p/7j9z5kkHjkYBtMplv05kBmRh+iBusqVEMaRwo32PoHXYj+gJwrd7/G6J8XGndpra8FPc9MDr45esTp1+16wZ+LFnCRUVnFcHYMAhvY8ZFXUwhonsTz+beomfjJWMIpS9xdfzyTkdzCetAfqs7PuKcPSFr7Mf8XdQuqTuH5u5yeOB+xgNNRPg3MRejJqmGxriCRf7+WHsdySSMm4CTlndkKRMIzqyjDfYN4rdN4BN3YKCZvNaAYBgupAf8BZsIgmi3qWsKMJ0p/HLFeZHAHGe8YF/xtCgGyUQzpuF8AE1wG4sC4ATxQC6N22YSi43hGohn4U8kwHPsGclyq/4cflnkv6LczlMz0b+uMfvE2s69DgEumVWSKzCc6tgF2uhSUI6qZhTWEUOc9KM2Sz+GnXhVLScPcm/yZIeHS+X53trXUCqw+AgGs2/yQJ4Jkgiogxf69Pitgj1LlsH6p9B6MDEzCGk7AmXK+JfyiePrWF6M0eKt7CHEKRr9vSjdIqjI6zlmkX+V1JdfU03R2goQoOwc5mUbi/BqPys6kJf2BQi3xXuWrud2xd2vdsDVNuJAvBpaw9WsHOlTDS9CHRfeBkZilHZRxRuyA9oFpg5UgcpACAKXufxmLJ0KaliLrvr/B2no8i/VE98aogQ46xg17yb A3IE2czX Lt0kub7CJQorCaeQQIqVgZqOYdkXvOcqm6wjIsbL1pAl68Ez+3YwhtwWg32GgQzY3xQsU4FBvBXaLXvn5Q/MVMRme4qpovksg0mKarBcql+ZYxeXZFkzOvfHbfZgJ8wa24ozZkDdJol46GvFjs4ZdjGjW9qY/fPslA1qAo9ZOy8X4NlPKSfYCiY7c6wzS1AfupXIWVnyUHmSk3a2ZwHG2Lz0ziBNXHvS/Kxc7lTs9R2p5J7LUGMCfoIrnug4ylBKTZ6kNq83CYJKgr4iEUvWGOqjEuT7H+1C8Mug00738T3FjHG6K7V/XzUdr9omZKhoYa3NEofL+QiKt6gKRrp+NXJ6LN0X/SNwHkWklNW+1SON6yJl/kkCtNfHvlESNK279n39c X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 01c5be6f4c96..90d93cdc5df6 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -808,9 +808,9 @@ static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, *obj_idx = (obj & OBJ_INDEX_MASK); } -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); } /** @@ -1483,7 +1483,7 @@ static void obj_free(int class_size, unsigned long obj) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1497,8 +1497,8 @@ void zs_free(struct zs_pool *pool, unsigned long handle) */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage = get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage = get_zspage(zpdesc_page(f_zpdesc)); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); From patchwork Mon Sep 2 07:21:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786834 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BFDACD3427 for ; Mon, 2 Sep 2024 07:17:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A52008D008D; Mon, 2 Sep 2024 03:17:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D9998D002D; Mon, 2 Sep 2024 03:17:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 854638D008D; Mon, 2 Sep 2024 03:17:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 66CEE8D002D for ; Mon, 2 Sep 2024 03:17:10 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 29A08AB0AD for ; Mon, 2 Sep 2024 07:17:10 +0000 (UTC) X-FDA: 82518941820.26.CF90951 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf06.hostedemail.com (Postfix) with ESMTP id 8869C18000C for ; Mon, 2 Sep 2024 07:17:08 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rkUy6HuW; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261324; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5ZvvNvc5kYv4dXMGIZPc07KpLduPDQMRbytcG5AX0T4=; b=S+l3quA9CcMnv8l0S8z2ZGoLWoGbmt5jzfTwjMlAhxVIHn3SGwAwFaCc0AkZdfjUneCoIm lUcbl1d7oQhtu358O54vA6SVENeXqeqI5L/NcMWxDSVjnJMyj/MhIIluoNUjQASzfBe84s qKr9n83yb06T444Tbys95TE5jheabvA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261324; a=rsa-sha256; cv=none; b=7kdYlj0QrhoSXKt8+zfTahZbSl9+sTlgJgv5+yihzR6jaj2ZCRtR1bjh9i6k23FT/QP6ST Q+KAKy3sgnuB2d4vhU8FV7rp2FR+w6PsTxWwtR/Td7lOdRKf9lZ+5xPEH5D//m3vKih0Th ELmx/Uz8nWeHo9O28tJ/+T5denokiEc= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rkUy6HuW; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id C6501A41722; Mon, 2 Sep 2024 07:17:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8A34C4CEC8; Mon, 2 Sep 2024 07:17:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261427; bh=oCzUVJExiVdlJN3ac9k05YJBYpT4iWle/rruJLC4yF4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rkUy6HuWPoGU4zFzaR50FOgnKCKF9Tv5JOIFvN6EGC+Wo0ag94CXHRxJhF/CSJ7Qq cDZxI3rBPwkmIZ4vAujby84oFn+WBybmfq1QQgNg+9K/3edwYfXgwyuzm6jDI1bVyQ y3m4XjueWEln4Vb7BgU/ODrYz9R0Sfj9cnTbhQaG8sPmEE/xrMSmTsdr+lb1yds84+ eGBorMA2OYL0Idrh4/9Eol5VGZ1XidpF+qyqo4xROKcLJtzGYV0iU40pCRuQQ7OenL gSHTH5L8jd9BzD/QVi97ByljTvtkpMyZLlNKKpOfgytuIfhVQ0LiRbgPi/doqJx79o RJnhD2a/uTqEA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 10/21] mm/zsmalloc: add zpdesc_is_isolated()/zpdesc_zone() helper for zs_page_migrate() Date: Mon, 2 Sep 2024 15:21:21 +0800 Message-ID: <20240902072136.578720-11-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8869C18000C X-Stat-Signature: mzo7dkcci5qkag95ojhs85z1fwsdfmz6 X-Rspam-User: X-HE-Tag: 1725261428-667237 X-HE-Meta: U2FsdGVkX191yaddHdSY53nk3N4LF4MdmgRbC+11GANe9m25o0GfWXMnoHTIwkEF2Jytk9zuthI8MmvaQnJOJ8NWGwswpLimEksveR//350XECtJNXnuK3MgDNsH55h7pa6Zz64A8AhCmSWTqywX2Bpz9huMmuDvko8JiFFQjTWZykYoJf29NP+tbBX4pVg8zk+DpFWfWo0rRz+C2S/C5uuD2sJIiIcN5E/QnFxXD0lGmI9i7UJ4LnbIhDPOxhmJhxRKBPouH/PGwVgSucoJAYI4InX/fqnMJPx5ZaqQCVosKk/FOeuAJKcss+x4lldfXV7oqwtTCAZmjj4IQc0eR2x1XCVjU1vtUSc/3k3ter982oJWZ5Lu430FwWqtp3f9NzrchCPdgvpDmfntMRWJkpFHrggS6NTmufXu6RhtD7HNETcaPAMapF/Swk2AJme1pYdF+pvMWuOfvGnT3bn5QMphy3AOqJIV1jVDz3PpqZoJETwdkO6R5yaET7wbMhXJa2o6zEW3AimTdg9KB+pYpI5KxlJ597UGjccG65klNC1dZEbJgxO9KUgeQMy+Di3fkJMbKlJIUEMh8AXgyyE5itGOwHKqO6K50KWLzjyAh9nyWT5j4tIGQCU32nFBReMuDaGLKiVvYYx+YLAlPLqdAdhJqzwnT8yxvjFzm3lJuoW6ZldbthFC/chFVHZgmkamqB7MFcRR6AwJ3PrQqyExbhd0stxh2oDWPOVJUNsfsdxCpaVc+HRdsT4LX5LQdlxVvAiVBXauUNeLhKm2z8G6aFTNjFajRRn4hwaGaBtcV6Lr/UY3SbYXIGqSs3jZZMTmV/Cj+A7a4x6yxbIGx297VUCEy/ncQamlY1rSmiSIEk8jKOPH8KTm3lwTFIfzS0MmU5Tz1sqPwB65RdSXx5yxiJShDHHUmB5QHkQ3zHHjL/DCuh20UqTKVzJghPeIxME5ylGP18MHKHIcABLRtrb 1EMF3VSH wjCcxLRvbgh5wfM/5dT4a+jea3Iv/slrvr52nQYYZDlIaL++cufasY2fOiNO7V78ue5l8bWdBLs5ju+hz+v/WlceSbFteqh5ihqKWELM+yYM0ItK9MPEneRTDiHyZBj+cbBSq0C0gkHA2qZv77pHMEyoN9KXaiL+kxieAjvdj6NYg2GT6lEvI3lWdppmgf5ya4ANnKzmJ1XhtcvF+6jqQZfbROQc6ycsOcP3GXW9jlfEVxdhldmX/8MgeqkGcZrF3ic0ku7Eb2ZPk/qStz7CjWdr7P4jhIGKJa9vJ0WNvkJy7ZYa9WfsFFidDpoqL6YVQ06eRPPutmgY3U7OwJus2yEyDjJR1zw5DYacSjEVPz24CEETjHbrBCwFO4A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated()/zpdesc_zone() helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 3a65a7d494b7..4b42d8517fcb 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -114,4 +114,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 90d93cdc5df6..caa4a7883ab8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1817,19 +1817,21 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc = page_zpdesc(newpage); + struct zpdesc *zpdesc = page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zpdesc_page(zpdesc)); pool = zspage->pool; /* @@ -1846,30 +1848,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_atomic(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr = zpdesc_kmap_atomic(zpdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_atomic(newpage); + d_addr = zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1878,14 +1880,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } reset_page(page); - put_page(page); + zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; } From patchwork Mon Sep 2 07:21:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B072CA0ED3 for ; Mon, 2 Sep 2024 07:17:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A60C8D008E; Mon, 2 Sep 2024 03:17:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8525E8D002D; Mon, 2 Sep 2024 03:17:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CD888D008E; Mon, 2 Sep 2024 03:17:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4A8518D002D for ; Mon, 2 Sep 2024 03:17:14 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C478B41A1F for ; Mon, 2 Sep 2024 07:17:13 +0000 (UTC) X-FDA: 82518941946.16.2BBA7C6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id 2857F120007 for ; Mon, 2 Sep 2024 07:17:11 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tINHAOSK; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261338; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xXTasWYVXJ4bjehGHqBQUEnrhGISSsBrnqSgl9/TimU=; b=6uv9XdbTpBfMAaufFBBMYbNCHiE8OvOh2Ns25Bc8694RKC3htug7SODr42ip3f8vzpaRjb ENvgsYxdlZxTOXQSOrEmLEBolb4GIVwrJMBonU9FmPNS8n9UA2THcKS+vpbOz+4vsczZ1U WFgAmXtjuGbiqUQ+3oadeL/zgVd3G8c= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261338; a=rsa-sha256; cv=none; b=OGjVnnns7yP7vbl/WE1tYvjxZ3a2+oPAo2vVQlS2qBF3bDad435WCOcbljPLu10Hxpm1NY KVeImaqk72pXs47YWonReJbjdGdMJSjo7zE3PUsM4RrBHIQQYx4IqbTV71uGtbpOXx0Bop TufPGuGo9gs/NnCnQdCN85xawt8+KG8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tINHAOSK; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 0B0C65C5724; Mon, 2 Sep 2024 07:17:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 59271C4CEC2; Mon, 2 Sep 2024 07:17:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261431; bh=TqaE/5WJpT1xBdjh2dqdooKapipE1l2P0P7kGVoluZA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tINHAOSKO69ja64wk58zkCcRcnjShQs6ujw7E9y0QqjTQ2Y9taiVSVw1xtraDIUOF NzJ4/jDvLlYoO4zZIiSJqmsszuMAbnuyMTsOCbm3B+BJIsl+16BsZ+omQXqmXSglnm v0yWrwoutoQuD4ye6gqAqFgIJmwaWmBXfx+V26fOfvQcdN6QCRaB8Tubehki32vm70 NbcjUFnlub6UuwvekHSYnnKakVcPvNyKfOZ9EjH2oC/AFPoZAGVtaVXAdiHExx3WCi rQciGFPkmq1gCe+us9jvCXv6Fm8P71mQUUJ1oqY7Z2FfKKB2hmIgeUX36hKv70OFoS aL3Y2UJ4+oxiQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 11/21] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Mon, 2 Sep 2024 15:21:22 +0800 Message-ID: <20240902072136.578720-12-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2857F120007 X-Stat-Signature: 74u8p1ijujmgfweagqnckhgkoac749ji X-HE-Tag: 1725261431-588452 X-HE-Meta: U2FsdGVkX19+F5OwH5fZzO9tKztppGX3Ss7bHR6uZD1MVw0kwDAWNEHpEhIHRP+2it5NbyqCKf+UnseoatqrV4i6O9qKuzmWw4ul9pQ76XDLZPxBc3XL5gc0a4kn0X50v/S1JjQYBwNG6hlhgivjH3exiElQwZTqrcBIfdcZgdFw8gLOsrdR7uu4F/kh5/BX2KjLbUIVlSgqPsGz9t8VQTCMDxVJvElSnCq6mSE2CaWdagkbXdn3mMtJSZBDKQY3kYtkzSTkoXzeTo6HLfEIqe29Txk51do1OhGNjYwH3Zw7ZIjxk3IpWWeXmRmROQ6JgFTAqyPNmwE97zrwUIxNJXp0a5R8CRTArJTMHngrijs2zCI3oKr6bhmUzeGkz3Y4U82sUKeS1wxWuz1BZeZc8ew/T7de9pyUqo6/bKCfzmisIoGOUmTQ9dgtfU7MLARrE+2lMoUYwYMLb/1Nnm9oKiYMp+RSEOoqRrKWgh6XzaegtEPIFC2zubzhO3JflqCU4xezsun45WDwi+jCkxmQZ/UtYj4a3mwNph0Q2jCj0cjjh0JX9z2BqaHaCcCQIQCPXBkcTWrLmWfqSK8MR1knoboUtVSaQpIl5BtvilAf4Oq8BDRhKpddgDXUVX6xc8q2qhwlj/sgY5/yvfLRxoTy8HSwAUF8//MwbEzuXGVeu4/v3OoAt9Cu56G9KBQ22ZABjpL/udf1Gy6IEQ+Eu7ESK1qJE31mmwI7/6diVFhzvWGYwZMjMJ6Mn53xKkVgbNMgorn9Z0BDL8uJx8wZX4QFETmkiy4KXJuXIYq5J2OXw53yr9ESVuzFaBDcmYaYKLKC6/5nXk0iF3X6SM3jDVCwQ/glhZwHqxAO7yY37g60qG0FOdEgFH9fJJXu+HtQT4mhmRXrnv28fOWNkV5Y6zk0IuXjq6c8i3TYUct82zEPHO5IZ6IjeZ+d2YmLW7570jBmn+prr/9k+snhreLDZ57 XPjhDXPA FbU+E9eKnbz+d/zNt4Hz8eDSCdAFAUv4e9NUFT7ebG5CmPXGqkZM+D/uLZGA+AcZJVz7qunhkdrqeHCv1k+3s+GIKG5l8RGQcESezuQsSxjqND93wPFiSiDwFgDQ8jE8HQ9TQdJ5LaDmilBMYq2/+y2mb39x0Dsydx6l9uC1vJ0uVhmfh9K9O3DOLPo8jFFIm825bSHGdvG0gLimj5KWwKc0JRa69OBsSgMtxXbJhY9BDr8MI/y6exbDuOgkPC74K4KLfFAEzzeVA7v3/XJoBiLFHGJkHWce/lyBSRzJAs3RCxI/SXbCICMHiTFMWxttzWQmCMKlsT+taEPAZTTYqyezh93omiJZVXXfe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() which is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index caa4a7883ab8..b03bd7e07e98 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -853,12 +853,14 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, return true; } -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page = zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index = 0; + zpdesc->zspage = NULL; + zpdesc->next = NULL; __ClearPageZsmalloc(page); } @@ -897,7 +899,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next = get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1886,7 +1888,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, zpdesc_inc_zone_page_state(newzpdesc); } - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; From patchwork Mon Sep 2 07:21:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 147C6CD13CF for ; Mon, 2 Sep 2024 07:17:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 992B68D008F; Mon, 2 Sep 2024 03:17:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 93C718D002D; Mon, 2 Sep 2024 03:17:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DE248D008F; Mon, 2 Sep 2024 03:17:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5B7318D002D for ; Mon, 2 Sep 2024 03:17:17 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2490BAB0C6 for ; Mon, 2 Sep 2024 07:17:17 +0000 (UTC) X-FDA: 82518942114.29.7225E39 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf07.hostedemail.com (Postfix) with ESMTP id 90F9C40004 for ; Mon, 2 Sep 2024 07:17:15 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QibvnDvI; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7lgdYI7ZuE3aDl7X8xuB7X/wsruUE99T8odE8I/cP7g=; b=zUrkSMKRkC6Ak+41j6Olfggd0PEdx+0kJQI8DMfpdIr89ORLEAnYcYitLW8PR8RRj1ervo oqvCUUrGnJD4GUwTyKvAuwdu2LIeXETHuT2wvAUgfQ/tI4acHvLIpxe0uV9YJKsVN08UU4 EM4eD6e6JvU8yIbMPMgu4FN6/WUGB6Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261331; a=rsa-sha256; cv=none; b=yYFM2A/aDh347JJqHVcsihbbMmS0D4ErHcFgt7a0SnYXaJi6lxFapEZtWCOQtuylvi0JFv 2cWP6zFyD3X8oGm8k1IhNmSjxhTvIz1+dZEiXudssf5XKJlZE/1CCk2CSM8/5LMlnsX/pt u06I/VK2Q5HWJ5uiG/XKlMPO0K1fJss= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QibvnDvI; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id DFDB1A41732; Mon, 2 Sep 2024 07:17:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BA1A4C4CEC4; Mon, 2 Sep 2024 07:17:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261434; bh=V2EFOKt+6az/uMmwc5vttTlHqgkkiN0OISIMWGbt/wQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QibvnDvIwyvgilFEHEmKNY6ncRz9JForNYuHKUV/SasiGH8CLsNIj6aePH+ZrSs3z hUios/48n2louj3GjCRjC0Oawhx0jRpwnhG35lr8yyZApFk47hBbbHO6FkP2hMeGcl VnE8LOmNM1GyTTtZngN4I4ZEWpAG4b0JKq1MykngZEfVoGoJ8ucTz62ynqEVofLygl vd4Y/uW/yblxNd5EQIvkSlGfKW6yOalNJd/t4sSqw34G5Hamy4yimVdGIf54eP5o2e 4jkfeedFF91ZcBTI1Fn/R6LQhRNayDr4GqPp11hZJBPM/uFsHukJ3ZsF1e9RxkbWv3 vmzcC3wTHfggg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 12/21] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Mon, 2 Sep 2024 15:21:23 +0800 Message-ID: <20240902072136.578720-13-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 90F9C40004 X-Stat-Signature: kogkrnk7umnpsypbaemwu54apem8zwj1 X-Rspam-User: X-HE-Tag: 1725261435-132689 X-HE-Meta: U2FsdGVkX1/9bWyN1otU1VyMMfRMewxwCMWXxL1MIfKpUP6JvSKSxVhBsAzXH855fJ7Mnp6LO9Afkf/tXg54DHa9vaFsFg0yR2Ep2FnnA9HeUhyz70timKUWMgRiSuQi1Fck5ypXt7uwwir34dhEWNzWWA3wTUJIxPGXuDIR6JdxAG7dGwtJYAfq8TbhzTLIeKMSpRHw5aFvxzppLZ9vIC8hdSnoXC7pxLYMAOCOfNQO3edRzasF7pjAJxlt38PmjPQV5JI9cIlLQ8pfeIowakUpZdkwT49LhEIaDU8MJNk9h9nb0Lngs0l0e25fWXDgQhQCCJMJBe/9dYvCCs3UTUvGPb/oQh8YgoKupBjVsvR/MOeyG/WGOpeacw2rnJjIzwqwijL60l1uGfN+KGUQQ+WGp21au0PngHvg5sqIcgoFGYWnSPZCuVyO6Qw2BlG6fSd7Vu6Sc6mTTzU7vQNQa7M6U6aKq5k4DoZ+CmH07D1+xZrDWMiqEKayF8PnrretybTJ/DpMIC49qkcmmKr9ZWb00nU17SVcms9Wywl//R3KbsVBpDZXc+AWNdDMrcuh4UlzGNPgBLXHoW6hIfqupzbUgKiAi6FzJWw/r0cAATv5J6gFqOsBlZkYM9mT1zfQgW23vO7aUDulA+HMpyj7PYE7mD7ausozlFFhP01W/ZStOJOiKDRSjt/x8vOGAvrPxIT158jqoqmSmq5Nx/K41GeiHg4F2mIbi0jpPmeRHUdikNVscty0mxydCjmemveK//nQWOwEzV249qWMXshCPy2TrnAuAAaU/0w4CQaOyjxOnJTSu5bu2rEW+hkSXO0pkrgUkZuJM8pVr5iCxaJXqLb1xNOehxcphczQyE0KZ0b5cIhw6max1sXjKhFEqFgaLBsLNktRpm+gv7UQd9+QIYwrhbzyhfIErbIxRb1sNl4yjo0Wh1QjnFN+oYrti6p54CXRUMBc1tRezyU9lj9 JBeIP0Jr u94ksVAaSYkyj9NSAdJ8HNKKJEWRkl5uenH9kN1M8fwAJMnAm9H3rSoRyKOTSvf03nr+XbOkfNEY2WxZxMr1OkR4+6GE/oTbptZSwPf4x7Cgso4cpNUu2rrs680SDBlozUeuEcWk5ezrrkqkN74n004CsmookfZVXRvp9CjNZ23haLdgQlbA2/XZE3hG7J5iCvrWvQMjbc+1D3x8pMF/IfCegExT3Nplrqs3OpoK5LAunFbu5gU1LuyeMhw0vomSZYDWedxejZU3uJdxo/lEmWiY2JlH/M8GJFPVDBY/WUiH9XYc6Nv+YO8rNZ+GDe83eGpHRYkqWI2D6+7ylF+p0jSyWq42uzVDUkbIzcftpkIf0ZkWyqQqFCKWZZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 4b42d8517fcb..a1834d36ccfc 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -125,4 +125,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) return page_zone(zpdesc_page(zpdesc)); } +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index b03bd7e07e98..5a8fd5fbc962 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -888,23 +888,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; assert_spin_locked(&class->lock); VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); - next = page = get_first_page(zspage); + next = zpdesc = get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next = get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page = next; - } while (page != NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next = get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc = next; + } while (zpdesc != NULL); cache_free_zspage(pool, zspage); From patchwork Mon Sep 2 07:21:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60DC8CA0ED3 for ; Mon, 2 Sep 2024 07:17:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E47128D0090; Mon, 2 Sep 2024 03:17:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DD13E8D002D; Mon, 2 Sep 2024 03:17:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C231C8D0090; Mon, 2 Sep 2024 03:17:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9FF178D002D for ; Mon, 2 Sep 2024 03:17:20 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 61E681C6417 for ; Mon, 2 Sep 2024 07:17:20 +0000 (UTC) X-FDA: 82518942240.06.F8A714B Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf10.hostedemail.com (Postfix) with ESMTP id C799BC001C for ; Mon, 2 Sep 2024 07:17:18 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NPeK2NyV; spf=pass (imf10.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8gWzCjoezRQSWIxwCRcWSKNiYPU+Pq1XTxenr8y5cc8=; b=DNnsX6JkrzJCI8h/OfHXnNVCd1Zn8BxJ03ekuFygmKkz8ZGo0dlUZ9LzufCONxnfN28E/w UiCNSwiLSP+VZ5jpi6c4sB9L5PVLgMIQvo7y50aDuxU9rTbzcpTzsur8xFFPGdGRQJpva4 PvczZsPF2QfRGjqBY3eoox4edhcpUYU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261345; a=rsa-sha256; cv=none; b=1Hnx6ilNiyy5oNMGuu5jxDHki4OwaETiYcYyy/lcb7yYy12vvbx3BIoxu0OwHDl8SOhr+x r2ZyYi0ItEwtirSxDbJWdStna2mTAxIkIrbNXAm4oeQu0DOM0lGz8GzcpDu5gXZqAjvmB6 x/6Up8m2l60H3m3NuAGYTfyJlaBHnx0= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NPeK2NyV; spf=pass (imf10.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 23FADA41725; Mon, 2 Sep 2024 07:17:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B362C4CEC4; Mon, 2 Sep 2024 07:17:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261438; bh=XMY5jg562sPF99BloXKZJR4HKiDGqicXgdq+2ws9Iy0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NPeK2NyVAp/tWckPu9912WQybh4F4gsYJTs+6ho+SST7R9QNtr+Wb0XlHgJI0twbc 3bINCAfA5kKpMF2Q7PrN+D2xXHoHSQd9M2nsfbhEmvC/kMGUJtZQ40YI+NCbthEAvB 0XPZVe6lyFovwfqWR2rNwDjMaRrcBQisjgCtJWR3reZwYuJFQnCsYbgAbxvefF059f UZ/3lSpf7g0T4mVBjpcJUPdNbwoy2hnea6sYvK8VwRlZ8WNWuyv4oGjYdw/iHVS6Z9 2kAImmM3/NlKqL3WG288/KRhAfKLnwJPAbm9U7O9TIdjYTmFD4EvUA93+Y3WuOibN1 KTCCtWBhvfP2w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 13/21] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Mon, 2 Sep 2024 15:21:24 +0800 Message-ID: <20240902072136.578720-14-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: zzsszagabrpsg3tn8nfsg9cnxmtpero4 X-Rspamd-Queue-Id: C799BC001C X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1725261438-623583 X-HE-Meta: U2FsdGVkX1/6AQG7M4YRXdHiJ5QyKW8wNtnY3ScpWxR5uEcoNKxfK2RBrctBESa3PyQGXcO3Y6XzgIh6gMJDW4hQNaH6r8fjwnE0ySVduK13TtC0+Zz1LX4yDIgLt4qqp3wGzXrkvfszqMCV5eOYoF1jy1aASqKhx7Sv+Xfk9gdp6xkrtNZW8SkZM2bSbYxJkPCmt3h10q6fMVVlrDyyQaFlZOPAAcVRsgCSuQlIwnG/jplXnsCRIdHzPxpwskbHbumK972nUaIhkaNkS2FSLOtcPG4gIlfDJNAFRgiTk49MIJXUuIiF7/Z+0S96H+AhVUMKaOUrfl1+VV9KGJ7y46BJY+D+2dF/DXzS6T6A300GEvFXG78lHO1a6LTY5Wer+1OSipD8PT4C6He0aKNb7vsIZU5vSDr+q5OG+Vt16IXveyTOJzNBQy6aI3lZnTq/mW8mnk+viwLCMcKRBOmHh0cbIMXPgNjar405DOynfXBpu4B7nK/59VxCcUWQ41lBy1+nKWh0qSyyPhwuSLZ+4Jllc8YEtAdXAE4kd0OVKDQ/Q/QpKvEJtDenAu04/Pa/FaCtdBp61X+USc996Xrn0NhzoDqGJ3fE4qA2oNklYn05Q+MhlleNoLq1BKAz2yeZICdNGByn8/IMJJG5jAXNq1NonH1+bTDL4qDstB3JJoIfQvN8izyW2f3wL5cHJaKUwWqqgMeRNBjeLkzg8rkeg8BBSRPbIw01avpK5LfR56KOXV9ySFhIJ/VzcI9ixYFXeqih9JXMvAThQ1KhBiDJ70M8+c57Ag6WwOk1U9FHaEHy6rWtpSZgF7yQ0LxneyI3n4EcgrclM5gtd0dd042vrNlZxirqaIg9dwztkBCY9Mf9VLyLDeMEz2eiCLvK57ZByv1wl8SQ4Q7aJgmsHBl9gHrkuzJL2bCDh6WRQginn3FWC1g0PkYeYKSD3+EEi+3UteeKcleOSurvYFoOt+C wCunlr7P mYYY9c0MRLwSFZ+XBT8o7WZnuBENQ+0SxuNT+BuiHi9oTi5v2SoZ3JJpveDswKSHvNTRktadIyOmmaFOvK8LfuXCfRzXp3RPWrWDAuRHkRJAV+xqUNlWOMijjDFYQ0ak3xK2Wb0eTrQQHi516Sqg5jTw6WL8iP0iEj00YDbUBD9NBGICypE49rrauqS0ZNiRGxuftGH85giKVYbqH4Fs8KU5UAK7dZFN3XLmvFuCZkt6rRBYAmVpt1wdOfZ/wsXedKNFgXbYTxPGWHju/+Fke4MwaHoeWGRnKQZc/K9pPoJq3ya8oWDMCed2YrlnrnizaWIKQVc8l7OIgPnZWfXBy9DijEBd1L1Zx7jMyYcAnvVXJO1qvuW6RnsQbyAkWm6zMxkU8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5a8fd5fbc962..f94c3a6653c7 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -814,15 +814,15 @@ static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) } /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx) { unsigned long obj; - obj = page_to_pfn(page) << OBJ_INDEX_BITS; + obj = zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |= obj_idx & OBJ_INDEX_MASK; return obj; @@ -1379,7 +1379,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(zpdesc_page(m_zpdesc), obj); + obj = location_to_obj(m_zpdesc, obj); record_obj(handle, obj); return obj; @@ -1866,8 +1866,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj = (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } From patchwork Mon Sep 2 07:21:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786838 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 417C5CA0ED3 for ; Mon, 2 Sep 2024 07:17:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9BEB8D0091; Mon, 2 Sep 2024 03:17:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C24DC8D002D; Mon, 2 Sep 2024 03:17:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC37E8D0091; Mon, 2 Sep 2024 03:17:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8C47B8D002D for ; Mon, 2 Sep 2024 03:17:24 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4960D1A1949 for ; Mon, 2 Sep 2024 07:17:24 +0000 (UTC) X-FDA: 82518942408.06.6817453 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf04.hostedemail.com (Postfix) with ESMTP id A2B6540026 for ; Mon, 2 Sep 2024 07:17:22 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uTv2JYWK; spf=pass (imf04.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261368; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ngiJ3h5ouGc7MrS+IIK7pMitHaf15uQXmnYzxL428lQ=; b=a+UtSVlFIP8vu+EcU4llgcq8zFYzTZUYDPsADtX4dsmms7k2FpMRSwGlepOLwb5ZmWbQ5w 7GiiCp8bIid2MYKv2846czHhKWE4LsAQUC6Jt+oUYqw6++fbPBlpdKet9/7SrvDsGEMGYz v0ijd6QBRR13Qplv+6BIHqb0imGIKOk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uTv2JYWK; spf=pass (imf04.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261368; a=rsa-sha256; cv=none; b=UN4qjTSha6zTzeMruQzb3fsAhW3FVM8Cwq707OIT4t/KOcmNAXjZhif9o1xBu9t3D+CzS5 iP3BRP5ABAs4wRViQIpZqID6iEnBn0UnQ4w584eapoFRSZ51ed4Wv+jBq9RZhh2nmJZY7n XQ+nBFadMJmw/jpS7BvstfPk65p7L8U= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 84D3B5C55C5; Mon, 2 Sep 2024 07:17:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ACE76C4CEC2; Mon, 2 Sep 2024 07:17:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261441; bh=oMrU85lTLRKqEwjZvX+/RAn/IsBRgGpZpbsgEGeKZYk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uTv2JYWKBWs/eY3vQwV4b7d46Dw+8FnfttVYa627PvUXCjgj8YFZiXNE2cONgQ2vD JkgfML6hEagHjvBeo08sMYm5L9jLkSjtkTdRS7T2LkNHQcFsDx2bVCHMuvp3eznYve SC/wsNxQ0jT08WxCgk5l1VnUbaEx4JcQxRDMcaxevO06zXMJubQ+lKPSu3nErUeF92 QQGh6bVLa6es1W1dR3HomaK6ZvUGMjggVhm7u6kO/swf/AcMtavkx6lsFCqJXQU6Mi 1Vo67rxmdi0vb2Ei+Mo06rE6WDhBAFTKNqdPX1iPelhAwfZsjP4PMzXFTcnadVGpX5 LyF4L2q+hsbNQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 14/21] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Mon, 2 Sep 2024 15:21:25 +0800 Message-ID: <20240902072136.578720-15-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: ts15rpptuzjiptnzk5zhj3i3i6464iig X-Rspam-User: X-Rspamd-Queue-Id: A2B6540026 X-Rspamd-Server: rspam02 X-HE-Tag: 1725261442-383296 X-HE-Meta: U2FsdGVkX19ZotfMNVxdJR/g8jdl9HoZXz6AO0ZjU2ahy9HW3R6EjFKWfJNgcC9T1uLE4PjfjTLNOxCw9NYeRJnMKH+KPOr4WHl4QokbEn8aVM0fB6HBfZs67UWjI2ATpZg0p6WVFNG8SalZ1AY8XtNRlwhbDiyW/gOk6ZDW/cQwmWuMh05xAcEiw6/canjYSp+yEjyGp/BNMCF7CHfjIy594jC83FuFkJ4dy1BOapkyFWlBzMSCz7wsYC+OSAndLrQLlnJIrn5ev4R3CAfuntnIXyZawrVtvftnNBmumlF1H6Q0FSyQuO9m3Zvlo1ItbrmWLXIfSoWOWgV2FRfwqupKlwhoq6wvT0mwK19GUqsZJbGy0qmXOzMPXCugkzdwPFir36zzNYcx9MlDWCkDCpKMxy0i2Gb7NbcD98W35sW6o8ie7eRVdPQ6vZmVTlm6449ndCfTTP90xZzVM+8axlcH05T0OwAaJWXK7I7qIO2IPcMVuR51uvnyjz2HdIW6yoPwF4UBx2ei0XpoSIQuzaOxYiQbnL0j60iHLzc7cQqH9nkz0XHBmNlYVA0SceyDwKJT6sPghcelRDvhj/HFYdQ0/6HVRLrNT7MvDcYrAk8NZhCeqq0d7wMyJdt2GP8rN00g8+ylHQeMJWEo6hJQzx+qw8oMk0mb3UqHI47apywMvg86SnVn8b59OEnqikujVJrRlaLl4GhSGKSNd2JEZlOer6wZgn8xRFXliebmk+ewAVwNh766kvS9Mx1XX9dOFeQirx2j5ATietP8ONf4Y83N10qw5L+7MPwo20fUJJFQ4b4DqoIG+/A6vxI+z6rqjo/bKNPUgX20qOK3lMOXC9IcRJbLqsJ5CDPfa+G/s8NsdNTRXmHtloE8FWqZ0UhgYS8nXBozTRlmnpIebgwmT7iL9V8RYCGXgBqAkaqhNvSV+FdGnGjTHF4W1FR1w5TNvKKM8qzWydmV6Z8MDH1 y+LazXj3 nV/VuS9muSrBkQVKEvP7iknif/ULuPm8XKEPYK0o+n3x0MoCKayW2pSgh1eEl+VX8lqc4VpDApDxi0LozQ15UDVmjv+1sNrNRikd345CvA2pdHg/DVrK6+Qa8BYdhA0FD5e4KkOyEJNvof1hOhX5T6HveMe6vdJDg7MSdTMO5LD3VvbC9QnPyrUIh0YrSrxex9PapS/foFZPPW6kqFkZv1GebAhVHJaBxnF/nPZIrLbkKXQV0n1eVu0aljNl5pL+zKzc94aEX3RcUakyrlcfCeqxp0599yPH22EYS8ak8o/r+P1v1sffgQmVh2YP+u9ksdaHAS/lFFDbbuNskfnKoK2/X2tM67Gc56swbr+PnTZr/EfHTzZzakuLF6orCvOwUg4dh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index f94c3a6653c7..aeb8cab5767b 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1623,14 +1623,14 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx = 0; - struct page *s_page = get_first_page(src_zspage); + struct zpdesc *s_zpdesc = get_first_zpdesc(src_zspage); struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle = find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page = get_next_page(s_page); - if (!s_page) + s_zpdesc = get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx = 0; continue; From patchwork Mon Sep 2 07:21:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73562CA0ED3 for ; Mon, 2 Sep 2024 07:17:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 08B558D0092; Mon, 2 Sep 2024 03:17:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 03BE48D002D; Mon, 2 Sep 2024 03:17:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF9728D0092; Mon, 2 Sep 2024 03:17:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BE6C38D002D for ; Mon, 2 Sep 2024 03:17:27 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 80AC5161A32 for ; Mon, 2 Sep 2024 07:17:27 +0000 (UTC) X-FDA: 82518942534.28.783F352 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf16.hostedemail.com (Postfix) with ESMTP id F29F3180011 for ; Mon, 2 Sep 2024 07:17:25 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="GXjsatA/"; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261423; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ggXGhnl3S4mO4sP+Bui+R+LSd2kVi+0mgL3AwPYgeTo=; b=bsD2JKLOb9NHuIvOiiad50eZziTdqm8ea4KjZl+yDGDhoQxjy1+gOxO5rzntnHqpOqOEWJ lu2llDApfFvFSv2ZXEfFMqttJLovncm1EIXohPvPhM7WvfnqJMxIPf8Upm7pt6MEZX0Wnc bltXE8RXycOgEqzR1dMO5xjXOUJW/VY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="GXjsatA/"; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261423; a=rsa-sha256; cv=none; b=SmNDEr1qKwXKoD0tCXRhzMcF8kgIGIcQJ2R/bxJLP47KOTswBSNPvjPej5i1AcbnWyIBUl +mT8B6wEFN+gsFUK4XBG5JXIkXF0QDFuk54NGzRBkPzdBL7PWrRfUDrwpk9ByoFIL52YBs /EWDKALDzqxo5VFFzNdlF4T+y7tpldE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 373D1A41712; Mon, 2 Sep 2024 07:17:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3AC57C4CEC9; Mon, 2 Sep 2024 07:17:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261445; bh=5d5O3WmlEwe4HKiK+a7YzCdtF3HLgTzqKDcXKeTJtNE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GXjsatA/b+bhLB5tHwCAiwV9pwnZATA6bAxMquy1hdL0LTxEf+G8D8b14lHW4PpbA YCLIrtwj9QXxUugNM0EKIeD4C5K/78gOBane3bfpBjUBmWXw40jJbTHEZGGyShhWtE a+OLLpl14bHKuvWxt+6wkxNlOTZg0oeM/kzbgkP2bXfa5YfwkdT0bHCLWPzBBXBTYM bY9oCR7Z8pWjz7hOpbswP+6qh7eAVoHu9J7os3pkqWqR8j2vllzd0uD0RFTmr6iLW6 3TAImOXHbZFvxvvi0yyL8lHiQSwH7gMFC9Nx50eEFSXu2sFl51dpXJ+o5Ba5b5mgh4 PRgEY1VF82sGQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 15/21] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Mon, 2 Sep 2024 15:21:26 +0800 Message-ID: <20240902072136.578720-16-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: qizh7pqfoybf8gmwxq1qi4iijsak4rfb X-Rspamd-Queue-Id: F29F3180011 X-Rspamd-Server: rspam11 X-HE-Tag: 1725261445-185308 X-HE-Meta: U2FsdGVkX19WE0eoVCXXKWZEx+7e9jzx8ArPkUXaOwaR0+GW+pbuufNSDb1mRN4Rf+L5Owjk895ZUDnBCrJzwD8BhKM+Ur+pKPJUoo0UYz8VuIRDUU5pbLTLcnDcdJwMj37aJ1VKCfbNdRAJ4rCU3uTluzpEpbee4K+FkfrBHYvAbg4woSEe9vNVIuO/wtDbRGe8Fq+aoNHlSjKOxB0SzJVMalrYTj1IbPIJXrufcrpvLwbk7eWeJyT+FlddlQgqZkLLtv8z6VFSc666tvRk3v/2mi1F5eHV9iO2wdvfcKNCK0A96+yIBa+KfZpaJnG2BwPhdfjGCKgydjY03hM5GO+HzXXhk7uSP+ecnJ93tQtXv7S+ZovdC04aykfRX2y/LkDLo02uKYepa0pcM86j/x02PFsOd0KuI7gNwXhoEur5k1rFUg+3sq1bpn4E7t373//XjBFR8zal51WXgxnlqJASIm/HA/KX9AArvzC2sMzbhnjV8CQZ1R5+bXP8anuzSrdYqgrOXD8gRvEFrWTRALLr1ox/srM7hMVVUmwio8/ekjVWehi150ugr+PGvLqY3i5K6Va+KKs1bgOY+qCUpWi2lQ37D8GAa2dpqK5gC8kJSu0B0P4oWJqS6rvrlw3i4DnoLkqQ2PqZa4DM+Qj4Y35PXrhUw052vu2ugfEDkES3SX9uKbU5/pp9mQShhicAWl8eU0IUQU+SOcdistr78aSnrTJPJfKjdgW8w9QNdNhoL+Lm3lD918veOwSZWfU0oepfBdnzSt3Y/uc2yyyE3JE3+4KQ0XrwtR7G1y0tyv2lJ9YfNITj+p6QrL5lTndq8ppyNVem42ZqIF8npPD8BCpP9QZoBM9AP2n8S/CAYCrK2uhxbcZQS5uDHMUNUV3yBxt6W13ZRB66qz4OtehHl3DsN6XuTHI7RNAxjLbgiMwWxQ9NTa4bMxFcBHGypT83cO5OuvkzGWlfyZvayrJ s8/UH4Kj 9VMkRQC1Ho6m7BPsELL5SqaaFNzqiJJKexIdQsJpNm5mfttlS3+bSakD2PBPTm4o0DQNJCxvWLN6DlBaa3lbJ/BNl5VhQJp1J8J+7HS42OITPZ3sH/I6QEMeu9i7GpP28FwYlnS3GS0cQ041ZSaqZXfNgOoutFUZ1lOZVAKWox4EQ7/xXD7kcTHxeoig0qLTThdyw1Tb5LTuJZiqRxLlp3jFYwOwd8QmMNTLsm3TWeSBX4RGXa2iSmm5IrTgF+oCgZ5XaYM22K7xgyo/jo5ds7Rb4xdiWremu1fkD16q30+CZ286OR25VBjjSooIz3np2YkOZrMuPdhdjhQXdq/m8oOyIx2dw5+uQOy38exMYyvIRUX5Chg8SFTGEDw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index aeb8cab5767b..d9251450ec60 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -767,9 +767,9 @@ static int fix_fullness_group(struct size_class *class, struct zspage *zspage) return newfg; } -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage = (struct zspage *)page_private(page); + struct zspage *zspage = zpdesc->zspage; BUG_ON(zspage->magic != ZSPAGE_MAGIC); return zspage; @@ -777,7 +777,7 @@ static struct zspage *get_zspage(struct page *page) static struct page *get_next_page(struct page *page) { - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(page_zpdesc(page)); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -787,7 +787,7 @@ static struct page *get_next_page(struct page *page) static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -837,7 +837,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1252,7 +1252,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1302,7 +1302,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1466,7 +1466,7 @@ static void obj_free(int class_size, unsigned long obj) obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); @@ -1500,7 +1500,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1833,7 +1833,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); pool = zspage->pool; /* From patchwork Mon Sep 2 07:21:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786840 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DABFDCD13CF for ; Mon, 2 Sep 2024 07:17:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7430B8D0093; Mon, 2 Sep 2024 03:17:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F2E38D002D; Mon, 2 Sep 2024 03:17:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 594278D0093; Mon, 2 Sep 2024 03:17:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 37EF08D002D for ; Mon, 2 Sep 2024 03:17:32 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0F3DEAB0FC for ; Mon, 2 Sep 2024 07:17:31 +0000 (UTC) X-FDA: 82518942702.24.7732A57 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf09.hostedemail.com (Postfix) with ESMTP id 71EBC14001F for ; Mon, 2 Sep 2024 07:17:29 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ERPokKEV; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261402; a=rsa-sha256; cv=none; b=40+K6SKFZ8BTEEGfkyXs7QYfW5+Tyf51A07dtp7k6HP22raAGawCAE/yyiJQBolho0Uyt8 VI82O1USiPr7zr04Al5mHEYV99J+MypIFk/TNtXqu721gDKj0SCNaqUzgvP2h0qqlQF+57 /+GmMcejhLUxQHz8B0dGGhUAmt/SxXE= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ERPokKEV; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VIjtBmZVxCU4L7fmy1RUXq/PJdjiwZ5zuPh3lDEwewE=; b=ri9s8HHK3Nqw/CwdGVINcKk1ZWbWSxcBRH8Bv5Z3/l3BQ62VdcB+oeY4x4AC1JaibJeOds 33vj8/akxPHP8nr+y3BaW/URtUO0P/x1oggSQGNGbq9TDia/hPAaz4lfcnV1wND1Cn9jRM PRcAWtJZtZ0f23XBBEvzqxUw6MzoFfY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id BC97FA41188; Mon, 2 Sep 2024 07:17:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 99803C4CEC4; Mon, 2 Sep 2024 07:17:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261448; bh=JuDze2waT5auO9Z7Ml30M0i2Ht/5hJoDWbD0pOFWjSk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ERPokKEVB7btxb1uvOqV12Md3j6Hiln/4Oe6BMGQ0KkLLvi6CvWIloMINQAnte7U7 elqpjG8Xnkla09nh2xTF0/JOW/ym6N4MKk36KVw3BiAIRpJZ9pcJMC3UsMGtOdlnqI if720f3F4KWiQWy84ic+ZksiAvMwBi1dSI/sAKv7FheYygvZZ+iDYNfsHbex8/h9qJ y/OXCEYiHNDN6tbacLZFXUhsNGNqvdN/w6Y82L/yLEqJP6G69cqslsWgZsIh1UUNOB a8U1Xqnj4M60MqOk0doSkUH2avB5I+3G/EckGtiFPlz4Bh+ZpGMsQK7m4iw2GeKdmj 6MeoSLhIzMHxA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 16/21] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Mon, 2 Sep 2024 15:21:27 +0800 Message-ID: <20240902072136.578720-17-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: gktw7h93356e6tr5ocshphz98a15csp1 X-Rspamd-Queue-Id: 71EBC14001F X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1725261449-535032 X-HE-Meta: U2FsdGVkX18WVGRcW7VYhK093MY1iZhzpusUaqJ0J+SGgITxZJ7hK3YmVc7pfM1QjA9YieAuBk7OeSqRKi3GqjDwmLeeYnCZy/W7561lSJs4VmyR6qRKoZrcvrd8w/YlvDkjy0S6JeStfhq2DhWnSKEQoWyjCjvdzORhEpmo8io6tHLoDc0jAnQ4vYrf+tOOHHULoV9RiYFj/zCn1/M7zwtzUuyj94zU/m1gKP+mjohVF9GHSMz9dSFITY43L4WgxmkAHvRpwqcbE6tgH1si+2ZMPli+dstr/4xBwE6HYDRINUYfNVejYMnyMabZBMlfWJ+W35/5ocC5ot0FHtr7ZGw5Tj/CI6vZcXdFCJYNfrfeDfHGHATMHSV9AbYeN7i/xXYlIlq3xaAC5kzmX77OaormzLOGqfmmyeuvKGhzN4G0vuf4/fRTkrrfmg0sQ4lNZjtN7dQumgACmbdtAUDoM6q2+0fSq/2U+dtOsvuIOxtzN5XNt5jPbk7rRbNHq7jP66boEsgbfr63UudZ0kfSwvDS5HT3wPik2ZxnorjARgxBS2b3+29c/DULkZn60WkwYDvyho6ESP7cC8Qu0Kyyqkafc8fwcptcTAPIT/tNcGDVlPG+QNLh5xYWTCCsNej40H6TMg2SO6c8A05t3UBxfnCiwocrDlMDeKOtXbletCMM6QqPzKTdv12LD/f9TMeBB9rKHxTGv/YijqTdvPSvWQhWM4MX6W/0nATl6Mnzbon6y7D8F0z3/FNi/J2ltgtLoFBcMTcgN4sRFLJMXp92CdCxz0wW2qrW3jII/4AER3tOD+CxGgf2OYPx6IYjRSuJzr2wVtULVQ6iwCZ4Fa1GmZXXEP6yPW0s25XznAxRNmy9ZEyFFTMVdcBdxJ6izOwStH9K5YQtg9uRcRSsP4yHK45R2EJrlkarMOkEQ53tqStdIByu6laIpoqCnKt8Zm66g7+podfl+PsYPSTK0D5 egISBKHR WKmtEVselrl1WKmc/l5+IlnLuFgoRqD5zuWRTpkx9Uzxn6KT+X7tK+sjm6nqOjxEFYDc9t2rwCLPtIJjB0J+IscJDFnzy5fJuUQqfdG4DWvMHl+gkWAPBqAFaGXQhid3Gzh1C9C6a7qnol51FX3pE6ezoxYc4R/ml7eifVcUplce1p+jjy78IxD6cSySxH890nvZOaktJ3+0nsG3GB8WuYNYptMLJN2q33ZZMy0cv8IdsdTt0TmMfR8ET37HozoISmPtLnGDp/C1lPIfBYOT+jZLmvmLHYuZuuMH7G1BjPiyjaiY/UUW4pSTS/wpR2W61n47a6C5R5+i2KMmKkEzuftW5yg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs: get_next_page()/get_first_page()/is_first_page(). Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d9251450ec60..3b98daa63ece 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -462,11 +462,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area) = { .lock = INIT_LOCAL_LOCK(lock), }; -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static inline bool is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -483,14 +478,6 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) zspage->inuse += val; } -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page = zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc = zspage->first_zpdesc; @@ -775,16 +762,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage = get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage = get_zspage(zpdesc); @@ -1957,13 +1934,13 @@ static void init_deferred_free(struct zs_pool *pool) static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page = get_next_page(page)) != NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } From patchwork Mon Sep 2 07:21:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2E94CA0ED3 for ; Mon, 2 Sep 2024 07:17:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 324918D0094; Mon, 2 Sep 2024 03:17:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2AFDA8D002D; Mon, 2 Sep 2024 03:17:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 132928D0094; Mon, 2 Sep 2024 03:17:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E37A58D002D for ; Mon, 2 Sep 2024 03:17:34 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A5AF9C176E for ; Mon, 2 Sep 2024 07:17:34 +0000 (UTC) X-FDA: 82518942828.27.2B2826D Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf11.hostedemail.com (Postfix) with ESMTP id 170AA40010 for ; Mon, 2 Sep 2024 07:17:32 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=gQujTYAA; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261359; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5EFT3knBScgr8IG5LmtuktpfCJ67L+lmDkF3KSnfYtU=; b=NQ7exKMHqPKdI8aQ2ZY0SpKU862dJHmF7VGsCjwVPzfR/xX1BrGEroBvNxvB8WZLuVHA1k uv0+luS4GpwqfGUWO8FBRui/3ULlhBvlJJp63AE7iLJMShNxHdLonXCnGQTJgVfnAfRgN1 Utcl5EIfwjIDHjXz3VWwmAPYWmbMVWM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261359; a=rsa-sha256; cv=none; b=p5eQBevGW1KPLPmj4K8Qu9CV0b6it6jHXMTmkV+2CCCd7aUauAlnnq/mA1XP8MBVbkUDI2 q++MA9Lkb5xPqtV9Eg1kgk+u9hryDVKSI0AsuQCTZO3Fm1BFS8xeLnB8dR9q+BXOQDPKty qG0QPkMN8xnZVAliP0Ccxhgiw05NOQ0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=gQujTYAA; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 52D50A41746; Mon, 2 Sep 2024 07:17:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A176C4CEC4; Mon, 2 Sep 2024 07:17:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261452; bh=mUt+2jl57a1O3x/5Q9wSnZvk6+ZzUwzDrkQBYoscUAE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gQujTYAAUF38D+/6f3rqz71xwGDllZgvbzpatRIu87HIi30swSwSE/K0ycx4vmIfl oqE0HY3KHgIE50k9iKI7PrQXI8TUzZPYgpAjpYabGLaVvtX4eEVL8YXBVMy0OBYb5x DpNOctRFE/8cBs7OB5UGid0JG7sHC1jSAOTOdVaee2LWSE6ozbnSHn+RgjVD2KI3fo KrMqJoCwDNOUe+Uz+r2k/5fJzP2DdEmhbb2/VhBxuab4pmB5bfiXjdaRYI2KKOWJTX FPGJ3k9WoBD36mKBgspStCsQzlzxftZjTo8/Hx22ZT38TQFA0rdz/+7aYpDWxNCHVG 3JwYu5XumfIUA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 17/21] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Mon, 2 Sep 2024 15:21:28 +0800 Message-ID: <20240902072136.578720-18-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: qzfbgtfainy8ihinhmoigatpzwtuxjsp X-Rspamd-Queue-Id: 170AA40010 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1725261452-894064 X-HE-Meta: U2FsdGVkX19+bRUG7W9w8ab3FV+/q21rMPWNCagtWipehy6I5L26GNOVsrq7XgZB+t0ayC/6zu9x8wXFIZ1q8d6gSPdyXR8V5+MOIJ9Ik58YzsoKrK6Cak73sCsVuYcY+2v41sYvseTa+0Kgewxcb4kToorPKlq/tUUVJIOWGWGywXVmDLEaiLsQuc2kweDMrIoUEP3JXMxSGIGtJkDJy7/hpVt3pQYASy6Acccvk456tfKHgG8I+dfPl4+/h/9n5/NdoZXCgz4/ttAxqXrQzYZRoSLfHNEo3anOjT2IxgAyc0M6Ao4XhG/E4uNodHy7OXbc4FBLtM7R6qVKEvAF5hYOInwom8mdzlkmAJbiKz/tguzRFd6yaYYQTs/pUrn8vg2cznHV22G9ivgN2IzWm6BRvgnOVHnCzJ5kWO59mViTWtMdG6autZiDYjnbqtbamDsivJtjID69u/3EMqjO6ql3Rkam5Ey23VU0E73IWT/7/EfLh8dWG0PxZSb2qQ4VEiv6xsyy0SS5uiGthSMTDQB8E/6SU7eCV+cbW3DdVoYBSCbf7KNSu/cCWGdxiPechKDUSmz3vPlJYloDOvp5j+KsI9YPYpKa/mHsFbkIWXRSkR/efX7j7zYBxb0rCAO2ZggnGjmjRUBAFToxpCo/0FfIXCreJ6LYEQ6U24czeVOCAceGDRI21Dz2RglqGagPaLaKjaMFtLcu46ICn0nFFwKISMBXaKAYSCo5UEofuoglsUNGY/ZM5ScnIXhiJ+PyNzosTPeXJOFFSivv0kYoGIgc0KfzdCYWaruElOmQpkA59npYVc7eUVdNhCH6UB5jTsZoAfUygmt9dt0xu5nVSkd1E8Qo0ph6Xgn8U6Kha7lSzULdymg9JloEaaOf24ZelRuUlndkRzODnEcuheTsp+CU76uB7ez/XMgtdJqJ1g3sOdrTqIEWuivEpig1SuuHhSE/YAGLanZCUUIVDGm CYn2rHC6 zko+XqBKdtUOYP9CYXwOzOv7h1nrxPPG/Nj+ybtSeuDJYvoZy+RxiP9f3PD0AOB4F8nKqjxfodwCunA3eH2FlX/RP3SvD753B5wrbroBkMGpYz+lfXNhcDy+4T/nIXnE9c4RJb5F5QvQoiW62qVTDJIwPEA9vE15QSzBFyrjumJxdkgC/Az4dh0kIlY2T5pCXFwofvSVekGpLWtUNWmECOB472eqyKFlikmGpbMXnVyJu4n7bNmkoHEjNWKHTY0tl6D7isgra85BXtpBqdxtId7V8DqzZcnLwVEeTNyQmfbJruWzyYrbO4hlPm7K3j9TaJp6uf+201g9kWoOP8FxLcTUdQTNUVgxrad7wDgh/TZyr6fNUzey3IhrTNA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3b98daa63ece..839fe7eaca8a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -488,20 +488,20 @@ static struct zpdesc *get_first_zpdesc(struct zspage *zspage) #define FIRST_OBJ_PAGE_TYPE_MASK 0xffffff -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } -static inline void set_first_obj_offset(struct page *page, unsigned int offset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned int offset) { - /* With 24 bits available, we can support offsets into 16 MiB pages. */ - BUILD_BUG_ON(PAGE_SIZE > SZ_16M); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + /* With 16 bit available, we can support offsets into 64 KiB pages. */ + BUILD_BUG_ON(PAGE_SIZE > SZ_64K); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &= ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |= offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &= ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |= offset & FIRST_OBJ_PAGE_TYPE_MASK; } static inline unsigned int get_freeobj(struct zspage *zspage) @@ -921,7 +921,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) struct link_free *link; void *vaddr; - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); @@ -1576,7 +1576,7 @@ static unsigned long find_alloced_obj(struct size_class *class, unsigned long handle = 0; void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); offset += class->size * index; while (offset < PAGE_SIZE) { @@ -1771,8 +1771,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); create_page_chain(class, zspage, zpdescs); - first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset = get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle = oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1827,7 +1827,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); s_addr = zpdesc_kmap_atomic(zpdesc); /* From patchwork Mon Sep 2 07:21:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13BC4CD13CF for ; Mon, 2 Sep 2024 07:17:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93E7C8D0095; Mon, 2 Sep 2024 03:17:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F1BF8D002D; Mon, 2 Sep 2024 03:17:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78FA38D0095; Mon, 2 Sep 2024 03:17:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5578E8D002D for ; Mon, 2 Sep 2024 03:17:38 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0D6451419C6 for ; Mon, 2 Sep 2024 07:17:38 +0000 (UTC) X-FDA: 82518942996.08.881E7FF Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id 72CD21C0006 for ; Mon, 2 Sep 2024 07:17:36 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EXNl4KFK; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261409; a=rsa-sha256; cv=none; b=IsR+P4Hg8slThIx644ZBSBD62p0tYgtf5akbnZ/NNKMSxFCHLCJ8kl00QfwQ3Jd8XYhL7L NCMUmdmAG9mfnxbHuJS4cCx1BF1mbhPBgFFtGTkTXnRV9ndqhszZwPeAEa6AUCAmrYHVXk 704kEakWDvEmVbUazrZxUJUwgY9kEYI= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EXNl4KFK; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261409; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LBIeXBRTvXSCauqKCSkeR5P4f1GeDGT3ZnPzM5xR0Wg=; b=TZN8i4QhRYaNghlgMIoP83RpfgpsUC9p2vwXw4YnG/ZclpByj8PRY5O90Va0fYbsShuNhI 1F+SkiH2H+LjKscrF/V/bI9QpwoRvfBcZdjy8OUTqxNcLQO+iPP5YJKOl3ds9SjNOPS0qo J+rW9Kch6H1O6916tg96IQqaccYKWTM= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 4AB405C1002; Mon, 2 Sep 2024 07:17:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3294C4CEC4; Mon, 2 Sep 2024 07:17:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261455; bh=kXYRElA12DbhEH97D9heS99VK1p14zBkU2t+ZW1MGTY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EXNl4KFKTK0kmfyfg0PFMIaOD3FttgGZCH0MnmSSo5zAW04bquMAE2xDfbjMNNVIR xyOchytZAcoeIyzGiM6gh7DHnGBRVYsm0uukeKA7UiBdDmv3Bna/wVYMnlbvQ/wF2G mHIEqY6gnf8yj3FLghQIzw7pDazFPq9rtnwYI/Oxk7Lc9kLqjGKxucW3wXOgHl5Ox+ 1Oe3QJVe/XVTHUZlHu9WVghZq/41pyRXmUhbaLI+563RJskdW+XeKdRFKxczmafI+1 VJ3pKjjHnlgQRmX8cbEnLMbzcoVpRjFIEki8S7ZigdUbQW9rPHfj4lPtEWfk+s2mrS Zza1TaewLy57Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 18/21] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Mon, 2 Sep 2024 15:21:29 +0800 Message-ID: <20240902072136.578720-19-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: bmmktu1rrxwbr5jazk4cgmuozypihsyj X-Rspamd-Queue-Id: 72CD21C0006 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1725261456-163566 X-HE-Meta: U2FsdGVkX1/galL6AWs538wGoUrT2kVtJh1ugF2FWspl0XzfGKV3baAUYcDecqz/SUrlNA5zs7HJiNU1pYbf3I3/pkjnweUx+Yjp5QatHrp7np9slUqwguYCK8djyZej32EpVcuc24/amJB/IXd65bRhHDdn3QpuzuhegX/5ZooDvREzlDveH+4OtGZT3uktiXkstqW1lSamEKPcP0MBy0HgKUlE+5JCRme+jeJwXpzFpRPNaeg9+2j2rqKRlX5rLyJxtqGdgVA3WwV+WO5EDuc7/3byp9hZGSoNIs/5zT8kewqlB9bUGEc3vOywwHak8/ihCos+yJFAyP1OynigS3Pqy87tiqFgHwcLZ6r4CQ/UEDeUx1Kv7r/YwYB+AebOz8yGxOGCZjRr/ZPxewjSHk6sb3ptXdw3Yuc4Wxti12TuGtxnstuKW/UdsjvESGHXN8YYr2VrnY9jlKhf5Kz+k4ktUtPbRdV+COrTFvKkxpghsRGQA+IW9f1GRIunrkZNpyz0ldivr+lRTj66VAkMXRwP8R3P2wo96fPagwiw7yTWGIHUNjOlNxu4Jyo5AT/157TBoGrRM8q25GkEdPliobrVoHbmHtV1YQhE+uySZhVhUABLzpK4SNYDbOgP/J64xFFTjt2FLECcsePHraEszSAkXTCiG2lkX4oy3pORHSdDowykHp6j0In9wSaGBfc9jcilEUZgdPRU8KuDDaKXHz61zyOENmNj83pmWl6wg+75c3QaBY+dfZ85B6IMQLblc8sy9nLI7C5MzmNUAqAolQ4rPfjfmjRBA54AZvPiOYqfAO/SjBLQOxYJNiqYjODXnEXGKN1dCB/oc9J1fdyn+x3LQD7WMKi7gaVgr82Cbz2d7dzwyxrq7b2HHrGpdES3sXymQsSgGcrUlROsl3zx63dCK+ImR0qozC/H/DrCAU6mr3ofPpDBWZax4Bb697Miuvl4d0aZAtMlj6RoZLN E3+yIZFK PbPyJ727VpMsJ88fOlHgE7nDv/7XDletUHOyyDUuVQba0XPblj0ie1TM9OsBfT0GynMcWbW4NdlaIgzZavBX27gXPk1p8UWEt7ioRQFj21r3qPK+xQOwhlJS/KDOun2Xh/pgIfjfZULktye0/upu2KF+AQD6rfGC39HYyZ1xYWIzON78Mie4yGr2uu/FDnER8alPHbwF9VQkswSw3cjS3Xk7UyXBz8tMv4ZOWfI/KYBhmiWdSZ6suFFeigaY0ST8Tj2irSLC+0iMWJ6TeUIwBUa1uQBBslj3HgnsdJV7JA/XZoOrWvhkGu9/+WrGqvgcOFyXlSKcCf8d7mX2/rMJjZKJUrDD5GTBzTG1d X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index a1834d36ccfc..747a2d410a35 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -115,6 +115,11 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 839fe7eaca8a..107973a2a36e 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -834,7 +834,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page = zpdesc_page(zpdesc); - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; From patchwork Mon Sep 2 07:21:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0EF1CA0ED3 for ; Mon, 2 Sep 2024 07:17:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 632518D002D; Mon, 2 Sep 2024 03:17:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5BA0C6B008A; Mon, 2 Sep 2024 03:17:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 434778D002D; Mon, 2 Sep 2024 03:17:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1FDF96B0088 for ; Mon, 2 Sep 2024 03:17:42 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C80F5419E5 for ; Mon, 2 Sep 2024 07:17:41 +0000 (UTC) X-FDA: 82518943122.15.5850908 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf22.hostedemail.com (Postfix) with ESMTP id 43888C0006 for ; Mon, 2 Sep 2024 07:17:40 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rmCwtlM6; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261437; a=rsa-sha256; cv=none; b=u/G2L6uXoQpxWVZ/0qNBt4Vj7izthBFyWbzMBpXKyayd9dZDqpdoTxLzGj351J9HUrSeT6 w3FNn2z1NQaW1kAxO6jXQ2vPzpPwpCRzMadEav6hWL2JV7e716yh1PqJTQe+AeDqu3OQHn xXTH7qnojYIIB8EWYitIa5hAujZwf0c= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rmCwtlM6; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261437; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gfjn5uAhOLEypwo9zarT1J2Zt/g3So2BaRMmNkbhT24=; b=kdX6MDheQP4p8M3GdNm5wQQ2BeGiTRb/JEPLMaioa7Pk4FzBbhHAmSvcRc09mkAT4twrAw t6BaJofdC+xAVP85FGMlNJL81LIkptK1KoWIK5kMWE6C3twrW5qok841Wee7nGoiXThkHA 07pDUxCoGrvGYBObkFA9IKIPDik8p+g= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 89DA4A41733; Mon, 2 Sep 2024 07:17:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1DA3BC4CEC4; Mon, 2 Sep 2024 07:17:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261459; bh=qBTLwe3n68h/nHaRcrmhMxfrsTQQMe/QD9FAr8WVzS4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rmCwtlM67H383ue/d1A7wHWuRCZDakd/TZzSnpJeA4VFGc32hDbeOS8FTOWo5sRb0 ViqeWCYjWICvSaWY72rsoZUG4M6YX+PGvWQMmy3jIv7c7QCQS60yusdDIeSZK812FQ WLRmX14gVqvnz4Cdklb3OlLQTXo9nCCtLvzPXK8uSEjWqTwzP+fhJ4ututCYbSksjy f7Osa+8CG1OpTfqZwFyicoXReh1YIGFAQgIjmm0MeiZif5Ahk46Xc59XHm09Urv5Zk lBJVGSAjYGJy3i6OlV8Yy5/NI7Ll8uGLNkXK40cfZieLx7/rjrLINdxsuvXMIZaOKL uY/1WpeDffh0Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 19/21] mm/zsmalloc: introduce __zpdesc_clear/set_zsmalloc() Date: Mon, 2 Sep 2024 15:21:30 +0800 Message-ID: <20240902072136.578720-20-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 43888C0006 X-Rspamd-Server: rspam01 X-Stat-Signature: d5ergtueyzd47wwwsqqfpe49rpnqa1ni X-HE-Tag: 1725261460-105573 X-HE-Meta: U2FsdGVkX19zaXzHf5Q8jBoHNdDm82Lm6a50NkbgNBVtoBohCnBKCPPzGjMvQSrFbHI8XP6LTwMucHojQWURolUWNeot+Nwnd1tgVlynCHe1EaklQyJpeI0xT7XfGVp5jXrRzMim/RCxDsGQz6Edu+G80Ckfmx3bdLmc+KdT5P05DjZASSSSS3TngSj8dVaungAuvyFthvdxIu3iAsCoOhligLzwKb0d2y+PbD65zYH9ImlWayyyS8vnZMZy+E9kfz7iNGrucp++jwYu0Gz1NFYQyezyZplDGt7UQWioHNzvRQuZrQ7Rc8ut1LtpWKkl8XR6PvZEyYWs8fKDeNu+C3BDWx2cgK5LrrZqcAHs3VxkO9xYc/v5CYfysnBU/0/As9ckPIxqATROrqRAcfsMI/6Iu5P6NMuOYz0lX+qudRbHCNsLqeeT4plVFTFa8ZmW4IMjzGFhv28qYidJK02jUH+aSI92eXbYeFzRHnQXRlCuOR+nQNyZLkbWUwxsJcmkU8eRRH5upN6Fhik3xBH0LDWGVYVRquhbERjyPz5qe+SzLR5IlxMPmKL3Mys+kfcFuSFQePhMtSsRK9s8CYYRj/68+DXcx/UkmcEqyUkB5et7thIaHs5GVTSaZMOfIcDqTWx55aygpOglZ5zWgDxTlCg4G/eCSmoPgqvUv4ATvIePdPP/NJ9ueQ5Q98fiYPFGCe9qfhbnl7fn/FLOsvbrImQ9OsDfc5OWA1VSYggoLYiPLHttLrlLSRi085QjP3SChcSSRZJ1l5/o9pA+eoe2r7GawMHzT2QmqJ7dvltlv4IvEG3WXaR1idbUxva/vO8vqremQt30LnIn38L8oIxQ0V+yyh4UXeoOfT8pU3kkk2N5Tt9bxdJrJfoErvEKPdh+0DmC8Jc1wvmwQGoBEA18US3CtOtXVyl6B9Lw+TEib2fmeOfbdanVyCy0TP/fI/a7GO8KiaLj1l38+JbPN0w 6C1d3CRK p4GbDZVujt4VpGByKfOKCIG+Qg84rNI3LLDs1rwuvCt7CnxgJeS1of8KKrC/SUnbixkaXp24dTAGEzTXZWKFjZL4LezsNctWD9w1JPeM2sk/rE9avPpIueDcGOZAAnyF2MMCfFi/bdNwIGSwwB4MNfgRS8Lwk3anAk+m3+BkXCWI+x1tMbiTx0xo83x7WlkJrAsIDKVBg6PlBKyq3qzMsFPkng8jTDjBGoFWjc55XpPDakwk0WsrkFnLWu3ZjOxDTgDU8cXyAy7769//xGNC07wqyQ4xekHOxHzcMLTAiSceg7CKGF4Bnjw08mEFiU7x/wnBNjQUbd5XnCOTKVTHCQHlim51BX6LTaX+imIlCYujjttk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use them in callers. Signed-off-by: Alex Shi --- mm/zpdesc.h | 10 ++++++++++ mm/zsmalloc.c | 8 ++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 747a2d410a35..33f599081281 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -120,6 +120,16 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 107973a2a36e..411621f00a02 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -838,7 +838,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } static int trylock_zspage(struct zspage *zspage) @@ -1011,13 +1011,13 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, if (!zpdesc) { while (--i >= 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] = zpdesc; @@ -1807,7 +1807,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); From patchwork Mon Sep 2 07:21:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D39CCCD13CF for ; Mon, 2 Sep 2024 07:17:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E3948D0097; Mon, 2 Sep 2024 03:17:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 56C328D0096; Mon, 2 Sep 2024 03:17:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E64A8D0097; Mon, 2 Sep 2024 03:17:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1CD2A8D0096 for ; Mon, 2 Sep 2024 03:17:45 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D02C81C66D8 for ; Mon, 2 Sep 2024 07:17:44 +0000 (UTC) X-FDA: 82518943248.30.4CD5A60 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf04.hostedemail.com (Postfix) with ESMTP id 50EEA40021 for ; Mon, 2 Sep 2024 07:17:43 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eXaocgJo; spf=pass (imf04.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261416; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t263XprIsSxrYEhO7vf/eoDJGVhbUEdYsSqIyXFnhEY=; b=mmPSJsKVCUQiuWwyTf7EmOriVV4PYNkBfqzbJxwA3iDoUkhYz0YnZdeoNUbphHF3cKak+q Acb+ylYtuJ2Q3L5gEisjxjvuTn/hh9tZ9HLg2+yNbc/vmLEahhg6unj1LCfgSgV1V8+r5U ZFMrplkSxAfHMbaQdREqZs8lazF46LI= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eXaocgJo; spf=pass (imf04.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261416; a=rsa-sha256; cv=none; b=V9n8fCIwY+oXXVVeeOCtkvwxHw2YuwKF5ixo7OwW4aCQaUJ/ZV5iDFr47s+11tSni+cytv 0d1DyQ5754eB/6HGPUJPA6Hwi5iVQqI12DW8bMV8eDQz47Lg80ttPumL3QER8oG1JTA2g/ nkwpiP5L0STdxKSO7m6n5jIFObLoHpg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id A3D30A41710; Mon, 2 Sep 2024 07:17:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3AEAC4CECA; Mon, 2 Sep 2024 07:17:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261462; bh=CFikcVVBkO0fkQwhPbTnJD5oTr7539f6hYvWeSZonkA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eXaocgJoKA1eC/mSBuf6h0+NhqRTKodMvnfs6tgPCGpmDK4W4Jfjf9Czj01vbzrB/ DwYWluMcU0nesvSRU1Oqa1i029m3hnIFjyRTlL7TRvoigsU1Si26+BxCZc1SMjXIrF m8JNHWz09WGPalcymX1yQO9CKN4uFTcN5ZwiKrsNbby8/9FA6yhG34TCuT7gTWM6JZ 4eNIM4jaA9E2e8kdSXAVYxorYAyipYDD1upXQF8a+L4fO1c3qgzq4wHVID461UIOfs gualGxbb0eMWhlydjmBwg9c78Xyb5/cKmORueJ+b6iqsiOJdA1wuJN/Osv7wlfjcdS AHBIjdJyXo26Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 20/21] mm/zsmalloc: introduce zpdesc_clear_first() helper Date: Mon, 2 Sep 2024 15:21:31 +0800 Message-ID: <20240902072136.578720-21-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 50EEA40021 X-Stat-Signature: uo6kpf364ix3effy36us64eeaut4h1ja X-HE-Tag: 1725261463-3798 X-HE-Meta: U2FsdGVkX1/HjefVYL9QDAO4FS0Cf/N+UnhZm35oRXFX0doOGgA5OndDRjxXPrk4SG6YY40u55JHzCrVFWHAxYxarPZDZHyPnJTz/bHb5U592TZcozjRat0zL8bbzVuMFpYBKShdcv/YhT2j+jI2/gPR55lt1v5Aoi4c/oDIfsbbZ+9Gfuk9P7LxxS9DWsraDnAqK/gd6ag9bzDpHgeZcFll6yWAbvYAuV78qQX8XmZqB09i56BDd3XIoSJAg3Wt4xI2zZRNZ/7mwgAtPTLwjqiPyYyxxMiRh7Io8pPRha0EIUta31F6wrAZZziOT8ATrbP4YSwB+bXIN9WZAsNyv8crtCY2md8nRids3NPEpM0w+2N8cj1PavCk4RNCDlPXGIuoyTZRNrcaveq8XOHbZj2kLh1DkevAxCsbnS6zGWjQyXPhH8zIcvxLpdhvo/d6cXwkAzcw0YcIAStVm+NLxLXxCuEDLgJv//XA9UXFI96yEEzQ/o4gAM4EyIBaGbTQO7FwsI1xjdw8L3oSmwEgxy2hVPP25dnu9cMPGEII1AhJavrfCFNImvwxEkP/AfwezMSlJxsVmScdPSHc9wnlatRbhCyeXsTE7HNLlMNvtp49KMjzXCHmIiYK7K2ybOjRowhbWbS+eq62wKFzSUClz5hq6S0iIZWJG49E+7shSr9ZRvBA6z4GXeeO8CgujR6Wr67+fxXHzZB7nr4KtQPU8nm8vT39bQmPOsbduABuAZdXEXGlIQ0PjPMEkGp+hj1BtYXgczRe9iLjEWpXRpZDLUUS3DgPkFWsXGWYdHd3EZBzuSrGdYxpfU6W4En/nENH4+q/o4xLKGvB5uHSE761l2lrmlha4v7+Fh+9vaPWEXxbzLEvf+ookQuZZZTDJAcx0Ml3m4ASbI5r58wWUZ7TRX3LrOeIL2F1fP8RHtv+9PnCw1OEhrYH37XF2lLDz5FRBOcgwYJTgp/isgtrDfg FBkQjsEj o4hhcxAbcpke6PSSYcvqEQE4xk9D10ljD/VnW7GGUOWFztrTlqNSDOF49bUGOfSvJhYFKjdPsnoUfb883Ji1yRlTeS4vGwY8ZE+fX6MPw/MOniN9lXR9Vef6rMwnBN7m4W8kvOIL2jT/4JZj/I2otQsA0GdWyEgRWdJeNcJtA+BnFIiCrVOJa7IZ9T5TLRtXpnZcfM145CD3yINKBXGxPAbTsSnaaIRBHcI3X9Pd4jjoBH/FURxxyidS3PPAL08zY891Vd4SJl8jtrVUhxgNGKAUFlSH4HTclVfYLDhsBP3OumgOluu0plOD2Q7+O3zCIVspZ5McZLTu2R3+N/CJyhaJPnDga4JI2BuyT X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Like the zpdesc_set_first(), introduce zpdesc_clear_first() helper for ClearPagePrivate(), then clean up a 'struct page' usage in reset_zpdesc(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 411621f00a02..849f192d4937 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -253,6 +253,11 @@ static inline void zpdesc_set_first(struct zpdesc *zpdesc) SetPagePrivate(zpdesc_page(zpdesc)); } +static inline void zpdesc_clear_first(struct zpdesc *zpdesc) +{ + ClearPagePrivate(zpdesc_page(zpdesc)); +} + static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) { inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); @@ -832,10 +837,8 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, static void reset_zpdesc(struct zpdesc *zpdesc) { - struct page *page = zpdesc_page(zpdesc); - __zpdesc_clear_movable(zpdesc); - ClearPagePrivate(page); + zpdesc_clear_first(zpdesc); zpdesc->zspage = NULL; zpdesc->next = NULL; __zpdesc_clear_zsmalloc(zpdesc); From patchwork Mon Sep 2 07:21:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13786852 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0710CA0ED3 for ; Mon, 2 Sep 2024 07:17:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C1C78D0096; Mon, 2 Sep 2024 03:17:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 349866B00DB; Mon, 2 Sep 2024 03:17:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C4868D0096; Mon, 2 Sep 2024 03:17:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D3D1D6B00D9 for ; Mon, 2 Sep 2024 03:17:48 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 919B2161A99 for ; Mon, 2 Sep 2024 07:17:48 +0000 (UTC) X-FDA: 82518943416.16.EFB489F Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf18.hostedemail.com (Postfix) with ESMTP id EEB1E1C000F for ; Mon, 2 Sep 2024 07:17:46 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=luyybGbQ; spf=pass (imf18.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725261444; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cnSV+hbwD6b1KIQGP3skQkvHv4Q6qbPV3pqxOakahcQ=; b=0GzfKqshWV1NAjASWjUExEL+WHHjDF7ddYl23lYHu2KjmuKZjjLAFn5FQKHSB08T3Ja0T0 GeiWbrFzS/b50zLVn9olIVnhBJyMOK7w9zmJAlU0eEHbtIwQHPITvfaokPHEG4u22TNt+x QTqnIy1y5S4JmLwOWTlgRv9P9QGTKiM= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=luyybGbQ; spf=pass (imf18.hostedemail.com: domain of alexs@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725261444; a=rsa-sha256; cv=none; b=ExSjfn8VdX4P8s7bxCnQTpuOVa8vYNWeS1bJskeSCJYJF4j4gjwYu+n4dTSi3xoI1gAzdV OnqcSx9hxy4OiE1dBdNdArZp0S13ixfO5cboTBXXaFGY8qY7FQVa5/RZKxUuAGjSNL6jDz dES+mmjgpekQAtVIjydnI5gjMAMD404= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 33F00A4174D; Mon, 2 Sep 2024 07:17:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1060BC4CEC2; Mon, 2 Sep 2024 07:17:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261466; bh=8NP5E97AtP/kem4IUjjLi+FAumtWmwCjNYRbQZ4Zmwk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=luyybGbQP4t9sYIyrJgDKuirEOXTx1BKnq3ONcswGeFiP+L0aMFR8Iec24a+I+X6u DrKhRpC/OLMgaJziuSmQlKpnzOYKyvR/Vh1yQ7aDSjErHfjApKkVdb/q/2ag0XSK/B chTwLoZUDipz8l1hXp7+h3YMah7dJUleabEjOxfGT3ru9pVV5+oNWZ8b0EElUFt3q3 ZcD8H2E6p/WxIAyU7fXdmEmIjqSnXdOZHhVx6nBIht4bV2himfEqtGPEx2/cx23eoq is6MR0afprQ7E1x9B9EySSyJw7/VDVGw+49zdYW7k/vImFfc7UupAym3vgCuVPrMtk pjSj/AX+ltH6w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 21/21] mm/zsmalloc: update comments for page->zpdesc changes Date: Mon, 2 Sep 2024 15:21:32 +0800 Message-ID: <20240902072136.578720-22-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 5f33d7otopat769d1jn3gjsz4fjoqjtx X-Rspamd-Queue-Id: EEB1E1C000F X-Rspamd-Server: rspam11 X-HE-Tag: 1725261466-468259 X-HE-Meta: U2FsdGVkX19McLujw7Pl7TIZpgsYd0IvDJ2xzfPT+lWzMlbPuUS0apKmrzZ4Jr9nXhvc0n6KjldBrzrre0Flc6v/SlMXnYVoHMhM1xzLISPrSFtWqInF0Xi5RP7akR9zvNoAEa5jcxznQ3dOce79PdOQpVjRhjzhfJJuqueRNcN8X965kx5Vb1pqgQpP6U5xhOp5NAR0IUGU7OIIXoDi6qQzGzrzhwDGdVgOUjrpI2zDteDJTZnuWEznGusndEr+QAzBQD+RGmYUTyX9tfhsfxjYUjmHCkRnGmZiPZllwgVHG9/8066Rwn0fdN5KwAFgTVwDwva425PamFEfGfDPSFVLMzY8EZutxOZBgAj0cMvvzJVTwt6hu9ZZcJaL4Lk4yqrPmHhlxLenoyE7C/qsERoc1+kZHDCAthCQMgq98SPN9F8KsJbiziLEYIMrrOv73zE03lg11uzrNHCVMdojG/9eQ4iXCFK+qj+0ibVrzqEfQ/We3mD2T7nRk+Cc3Qf0ebBSBqN/4hZH1BUcWQ+mmAZn3ot/hma5dTwK9K9Oz8c3+5aLMQ+BBMmuhyeSY9PjO9rMnRHsdKtsVdaFyNvtrZuhG9s8h1Fi+rC12wJfe8U7ydLqwzZCi07Eoa0HDQKxD84qMtWbKYxLIy+e6l10t+hnf9JpjN6VG3UW+6NtVzxLjkb4s9woTovEVbNs5qgpMEhF/yXk+B4lISNoMdsrSCp+DFr/JgjOVWZWFAy+sX7xKLWpUlT4bxix/qv5TO3sL+WxNLGAPMNA6l0fOojfICHBIPCcTgrU33OKOEcEtqXaj7tuN+yoxIi0WBcfuu9AsAMZdsykFje8zl1EP6oXJIRAI7b39Dpo1rna35XSXMC1vg7OF0KZzgeLAsF/DKHCPhMt1Cvw4qiSK9i/K0wmFvwX9l7e7t4aa8NWTAhXe3yy387R3Pqt/fycZkBstIJMj9ki+Stpt6dJ8Zrl7p7 qWhLoH28 OvnYCTEC/5jIdUka3rUzh7MRBuGSB+pmHTyeOHBgoQxWIcwdbQX7GfFydmYFti1UNySHNkfZ3K3FNSaMspqcvDKLf+QpDwZ97OKy7Y/8gYS6TboDappoPfUZ37CarsnYM4dYB/L7DVD0RqUBtEwQ3a+vHHwDHvikltdVOKaQsRXXLnQH0FfwaYyjGEifiYP0vvdZvpYfODlTC5I+XkWctRA04EPTei+oj9QnCSIRCITdTrdXi4oJ+NuiS7CtnXFSGLp9V0nATgfBIhf4KiYcvIfI0iVgnI29PX3lZhpc8ATdLtwnYJ6BYeoyb0g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi After the page to zpdesc conversion, there still left few comments or function named with page not zpdesc, let's update the comments and rename function create_page_chain() as create_zpdesc_chain(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 61 ++++++++++++++++++++++++++------------------------- 1 file changed, 31 insertions(+), 30 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 849f192d4937..1e47cda05c83 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,20 +13,19 @@ /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component zpdescs of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: PGTY_zsmalloc, lower 24 bits locate the first object - * offset in a subpage of a zspage - * - * Usage of struct page flags: - * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page + * zpdesc->first_obj_offset: PGTY_zsmalloc, lower 24 bits locate the first + * object offset in a subpage of a zspage * + * Usage of struct zpdesc(page) flags: + * PG_private: identifies the first component zpdesc + * PG_lock: lock all component zpdescs for a zspage free, serialize with */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -191,7 +190,10 @@ struct size_class { */ int size; int objs_per_zspage; - /* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */ + /* + * Number of PAGE_SIZE sized zpdescs/pages to combine to + * form a 'zspage' + */ int pages_per_zspage; unsigned int index; @@ -900,7 +902,7 @@ static void free_zspage(struct zs_pool *pool, struct size_class *class, /* * Since zs_free couldn't be sleepable, this function cannot call - * lock_page. The page locks trylock_zspage got will be released + * lock_page. The zpdesc locks trylock_zspage got will be released * by __free_zspage. */ if (!trylock_zspage(zspage)) { @@ -957,7 +959,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) set_freeobj(zspage, 0); } -static void create_page_chain(struct size_class *class, struct zspage *zspage, +static void create_zpdesc_chain(struct size_class *class, struct zspage *zspage, struct zpdesc *zpdescs[]) { int i; @@ -966,9 +968,9 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, int nr_zpdescs = class->pages_per_zspage; /* - * Allocate individual pages and link them together as: - * 1. all pages are linked together using zpdesc->next - * 2. each sub-page point to zspage using zpdesc->zspage + * Allocate individual zpdescs and link them together as: + * 1. all zpdescs are linked together using zpdesc->next + * 2. each sub-zpdesc point to zspage using zpdesc->zspage * * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). @@ -1026,7 +1028,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1353,7 +1355,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, /* record handle in the header of allocated chunk */ link->handle = handle | OBJ_ALLOCATED_TAG; else - /* record handle to page->index */ + /* record handle to zpdesc->handle */ zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_atomic(vaddr); @@ -1443,7 +1445,6 @@ static void obj_free(int class_size, unsigned long obj) unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); zspage = get_zspage(f_zpdesc); @@ -1686,19 +1687,19 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) #ifdef CONFIG_COMPACTION /* * To prevent zspage destroy during migration, zspage freeing should - * hold locks of all pages in the zspage. + * hold locks of all component zpdesc in the zspage. */ static void lock_zspage(struct zspage *zspage) { struct zpdesc *curr_zpdesc, *zpdesc; /* - * Pages we haven't locked yet can be migrated off the list while we're + * Zpdesc we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock - * may no longer belong to the zspage. This means that we may wait for - * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * lock each zpdesc under migrate_read_lock(). Otherwise, the zpdesc we + * lock may no longer belong to the zspage. This means that we may wait + * for the wrong zpdesc to unlock, so we must take a reference to the + * zpdesc prior to waiting for it to unlock outside migrate_read_lock(). */ while (1) { migrate_read_lock(zspage); @@ -1773,7 +1774,7 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, idx++; } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); first_obj_offset = get_first_obj_offset(oldzpdesc); set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) @@ -1784,8 +1785,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, static bool zs_page_isolate(struct page *page, isolate_mode_t mode) { /* - * Page is locked so zspage couldn't be destroyed. For detail, look at - * lock_zspage in free_zspage. + * Page/zpdesc is locked so zspage couldn't be destroyed. For detail, + * look at lock_zspage in free_zspage. */ VM_BUG_ON_PAGE(PageIsolated(page), page); @@ -1812,7 +1813,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* We're committed, tell the world that this is a Zsmalloc page. */ __zpdesc_set_zsmalloc(newzpdesc); - /* The page is locked, so this pointer must remain valid */ + /* The zpdesc/page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); pool = zspage->pool; @@ -1885,7 +1886,7 @@ static const struct movable_operations zsmalloc_mops = { }; /* - * Caller should hold page_lock of all pages in the zspage + * Caller should hold zpdesc locks of all in the zspage * In here, we cannot use zspage meta data. */ static void async_free_zspage(struct work_struct *work)