From patchwork Thu Jul 13 04:20:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hyeonggon Yoo <42.hyeyoo@gmail.com> X-Patchwork-Id: 13311271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 167EEC001B0 for ; Thu, 13 Jul 2023 04:21:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A364F8D0005; Thu, 13 Jul 2023 00:21:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BFEA8E0001; Thu, 13 Jul 2023 00:21:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8396A8D0008; Thu, 13 Jul 2023 00:21:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 72DD48D0005 for ; Thu, 13 Jul 2023 00:21:17 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 37616400BD for ; Thu, 13 Jul 2023 04:21:17 +0000 (UTC) X-FDA: 81005288994.17.2B441DE Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf14.hostedemail.com (Postfix) with ESMTP id 539F7100009 for ; Thu, 13 Jul 2023 04:21:15 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="i+W/NFoh"; spf=pass (imf14.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689222075; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=T5CtvWZNM/arT1DeaLF8JN33bJjE1wRpWQ+2TM1opec=; b=EdeJVmBp5ksQ0uZG7911cE+3qmZj1MCVR7S+UOj/wRQIKJu/E2ZGzp00Fvm/tSs01LkDZH buwNsCOkXkSp1Grbf1ZYsPiu0MFtdsO9AE/DPMEJApgBn/KBT9KYzGVwv8jP7JjoG6pzey YEWpw5mLEFXytM+DhikbSYKqp2p8UhI= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="i+W/NFoh"; spf=pass (imf14.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689222075; a=rsa-sha256; cv=none; b=i3syHDjeGFK6mC4beRxqXly21GYib1aD83iclCZkYpDdUoVHRN51yiSXKDhhIyC8rzEN0V HEsrQZTKMgY7CLmRAZ6lkSWn7rkjslhc7qUogicWFxe/h5U0k9ZjcgmH9wIu+aXLHQM2EV BSMV7r3WdCT7b1Z0m/XPAMiA67xRktw= Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-6686708c986so318337b3a.0 for ; Wed, 12 Jul 2023 21:21:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689222074; x=1691814074; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=T5CtvWZNM/arT1DeaLF8JN33bJjE1wRpWQ+2TM1opec=; b=i+W/NFohhvOCvKpcJMP9qgQqqgnP2EXS+qk6DRkcwUUbYvBlyO+0QzIrnCeCsGQzeG cYvSyZ09QP9qbQQXIlMXW8GJC9lCe0pdff6yWizEWhJGTSkLi4FfqcLNSlrIhbSp8wXZ h0fGuGyJO0H/0xTNTKH5YJCZnp3LI+99cDo9UlXjpTtxpQq0Q7rVV340Iv1F+/Q8nHSh aOTaBulsJzrhGj0kdl84YslsnY5LBVi0yVEhoS5Yq9IlgP8hnE79x8MaMNWqPTBDzDR3 70+Pn244J5aXK8AMhsA4BKwrtQ+ItypDMjw2JTBzWvaf7nbZi/QZhOpx9etOSHV+dYxI Tr7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689222074; x=1691814074; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T5CtvWZNM/arT1DeaLF8JN33bJjE1wRpWQ+2TM1opec=; b=Vwz+e6v3HXMs4iYKP1ecpHriStyNTfbcofyGU/sZd9+/PJJm4YAGZLRUsrb2mOM8ZV WrJ/ICWxgiQILVVDpBiGEJAvopxS43sFVg06KVjJqv+/bgL6vFZthNrWs50jKPDyoa5f ugfxgp1w9N9Y55DyyQJXNftHoV2MzamMOsO5NYIZit0djMIwHoE9hvxuzsSuu3HD5pO2 ZJ9balYOrLFWQNCaPzSVBG3k2YJo+5gwlFbLcgFsDm55ixTvzCy/mMq7UDhVX0zy1fp5 Wulf9PsXsivWkkNHgDAKPE4yuqfBLUlAylDwm7X3EkDs1Q4KVmhGzVR3rR3XzCuBaIeF t/Yg== X-Gm-Message-State: ABy/qLZam95lMTO8+skqdqSUS3nIrGm9zQMqt90mcdpDLXMA1elMqVZ0 0DWQr5I7GkD8YR49uGnaLeI= X-Google-Smtp-Source: APBJJlHjxNeh9XNM9Xg49rVHdiHhgRhHwEZjBVTmC7d2ywHYqshe0HtbWG6bkIYM6hqzf+06oaFONQ== X-Received: by 2002:a05:6a00:851:b0:668:9fb6:b311 with SMTP id q17-20020a056a00085100b006689fb6b311mr723443pfk.32.1689222074040; Wed, 12 Jul 2023 21:21:14 -0700 (PDT) Received: from fedora.. ([1.245.179.104]) by smtp.gmail.com with ESMTPSA id u18-20020aa78392000000b006827c26f147sm4346601pfm.138.2023.07.12.21.21.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Jul 2023 21:21:13 -0700 (PDT) From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Minchan Kim , Sergey Senozhatsky Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Matthew Wilcox , Mike Rapoport , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [RFC PATCH v2 07/21] mm/zsmalloc: convert obj_to_location() and its users to use zsdesc Date: Thu, 13 Jul 2023 13:20:22 +0900 Message-ID: <20230713042037.980211-8-42.hyeyoo@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713042037.980211-1-42.hyeyoo@gmail.com> References: <20230713042037.980211-1-42.hyeyoo@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 539F7100009 X-Rspam-User: X-Stat-Signature: iwr4m58m1xdpyyk46wtiqzf58dgqwty7 X-Rspamd-Server: rspam01 X-HE-Tag: 1689222075-842196 X-HE-Meta: U2FsdGVkX1/9aVtEwJlwlqBd046mCzJ/7476Cs6i6uQ0hKBoCKwabkJrLdUVUtP2gyyHyUnJzegcBpVlfuWgbEys+URhBx/Wphjf7wyjKZj5Qb6pWIOHwqOwTcs3XBc0IVNWcbPhwSmEcRhHT2bEB1OFI+F2UBG5JmedwHO/rDhG/2CbzziIamaW9ZOZsWjxFMbEUOyX5dJ0p2pey7HLKtsdKre2c6jZ6Yr4A7wqB7VvB3igxPeWyJ2ReiD/byhj0FdTidEuHzZ14x0pA6rccNga5ltAqxkE7m4sOI5356I7sebXpybeVEyaI8RK+nikjucDZ5U166wXHWa5d7KOGPQ5EFk1v7fgtnnj33csrNysEtjWsR/J74ziXW5Bpk6+UWmwgYgLMRlulGMK90mVyyuP5pM9UzNvmRg+PtUsienTu+7BfTiO7qpwBm0orRQZO+dvwkp68UKg3B5qwwEmJNcoLi52S1iX6v9ZLfXOZXQkWEpfTHjelXbyq2+N9Jmp9TsY6Frf2oy6mWUY6UhRVea2eDrpMQMavlWVPeRlF7eTHQBL7dDMKqWUG8Y7HkHuR5KXxoMLPkAQwmfyTPsqX0GQUIOKE6XvXDIZF8FWn0/aRlnbj2BdL2yGdmBWVpWvGlA0PSd0B+U1hCD0wPa4t2KRtdBclR8ah8RCHK3I9yJ1Y5lIA/Ql3zMImSbIAUzMbN+f15MVFDePW9bK7aXeUJLzII1K/Kyg0EZgDrgcpaOBbjB0E9klGd1q3WY1vodMZA26ylFWjE/YIfUWCw0hRwgL/gfqDYOx0QHFiPA1dhmZzCTG8d1/ZwIYWW2mz4j4/uMagNXLQXuHIikOgyxvjA+D/w+VyUvlxIlrMIuReygh2eyQXcNxwntfw1IQVaWKvhGbSc2DXrOZc1T8BCNkCUY0ef0KTIIOLHlZFO93oSYkAIxN+3c32k/hfs9KQEuENtfWsm1n1+k8RusIFWs EUr6uFck hW9LS7CXxcp2F2YbEhXOq42dhFAUkgdWAWCVs4cjdGS6EGpcSJTNNLjQrRFJTlx/QE12UiStwtA1izt/0s1LvCYtIgZvVmueJWDXMYXqw9p91UnmJ1Q12MH932e0unNnBqPk7uQENUw1TpoZrR0dAtNIEZQ+GvI9e+xFbdnlyXhzq9I2cbQOzKWjudBInDdovOdWDLjKmFgnUlaxcC+X8DYGqtwOGMjvHEslqle4MCWC/KAWPDfXQUu1PQMtJrp8gTPOELKB4uFeAPwoC9LplTX7uvw5x37nj1CHpLDiPSItqgvbNkgZSlqvjv7CLwgwjs0sAG00W/MhorZHFfreV2xlAgnWrvpXE26WOMMFouzJ4/1tLlXiIdBxiZlePZQ7A612gMdAekPZUoEknQUiRRAxTnbxj+5UDGIxLuAAr2DnryO962AbuJMhRg+k7sLoYenXdmTuwWZHfIQP8zvzviTSobeP9cx3+nOHzQzwgTBWVPj8ifRCifXGAh0t9mqBhYQtjHjqD5pBRNiHTQEj5kwIDjxiAod8PYOFkCVeNFHu3v/Wv8nEl+sNeb4v9gnAsYrAuPH1QG+UI3vrpDjfK5gr5Qw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert obj_to_location() to take zsdesc and also convert its users to use zsdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 1 file changed, 38 insertions(+), 37 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 4c0563fce3d0..be9762a49237 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -860,16 +860,16 @@ static __maybe_unused struct zsdesc *get_next_zsdesc(struct zsdesc *zsdesc) } /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zsdesc: zsdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zsdesc **zsdesc, unsigned int *obj_idx) { obj >>= OBJ_TAG_BITS; - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zsdesc = pfn_zsdesc(obj >> OBJ_INDEX_BITS); *obj_idx = (obj & OBJ_INDEX_MASK); } @@ -1298,13 +1298,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zsdesc *zsdesc; unsigned long obj, off; unsigned int obj_idx; struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zsdesc *zsdescs[2]; void *ret; /* @@ -1317,8 +1317,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* It guarantees it can get zspage from handle safely */ spin_lock(&pool->lock); obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zsdesc, &obj_idx); + zspage = get_zspage(zsdesc_page(zsdesc)); /* * migration cannot move any zpages in this zspage. Here, pool->lock @@ -1337,17 +1337,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr = kmap_atomic(page); + area->vm_addr = zsdesc_kmap_atomic(zsdesc); ret = area->vm_addr + off; goto out; } /* this object spans two pages */ - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zsdescs[0] = zsdesc; + zsdescs[1] = get_next_zsdesc(zsdesc); + BUG_ON(!zsdescs[1]); - ret = __zs_map_object(area, (struct zsdesc **)pages, off, class->size); + ret = __zs_map_object(area, zsdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1359,7 +1359,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zsdesc *zsdesc; unsigned long obj, off; unsigned int obj_idx; @@ -1367,8 +1367,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) struct mapping_area *area; obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zsdesc, &obj_idx); + zspage = get_zspage(zsdesc_page(zsdesc)); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1376,13 +1376,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) if (off + class->size <= PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zsdesc *zsdescs[2]; - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zsdescs[0] = zsdesc; + zsdescs[1] = get_next_zsdesc(zsdesc); + BUG_ON(!zsdescs[1]); - __zs_unmap_object(area, (struct zsdesc **)pages, off, class->size); + __zs_unmap_object(area, zsdescs, off, class->size); } local_unlock(&zs_map_area.lock); @@ -1524,23 +1524,24 @@ static void obj_free(int class_size, unsigned long obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zsdesc *f_zsdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zsdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(f_page); + zspage = get_zspage(zsdesc_page(f_zsdesc)); - vaddr = kmap_atomic(f_page); + vaddr = zsdesc_kmap_atomic(f_zsdesc); link = (struct link_free *)(vaddr + f_offset); /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next = get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index = 0; + f_zsdesc->next = NULL; set_freeobj(zspage, f_objidx); kunmap_atomic(vaddr); @@ -1583,7 +1584,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zsdesc *s_zsdesc, *d_zsdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1592,8 +1593,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, s_size = d_size = class->size; - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zsdesc, &s_objidx); + obj_to_location(dst, &d_zsdesc, &d_objidx); s_off = offset_in_page(class->size * s_objidx); d_off = offset_in_page(class->size * d_objidx); @@ -1604,8 +1605,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size = PAGE_SIZE - d_off; - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_addr = zsdesc_kmap_atomic(s_zsdesc); + d_addr = zsdesc_kmap_atomic(d_zsdesc); while (1) { size = min(s_size, d_size); @@ -1630,17 +1631,17 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (s_off >= PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page = get_next_page(s_page); - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_zsdesc = get_next_zsdesc(s_zsdesc); + s_addr = zsdesc_kmap_atomic(s_zsdesc); + d_addr = zsdesc_kmap_atomic(d_zsdesc); s_size = class->size - written; s_off = 0; } if (d_off >= PAGE_SIZE) { kunmap_atomic(d_addr); - d_page = get_next_page(d_page); - d_addr = kmap_atomic(d_page); + d_zsdesc = get_next_zsdesc(d_zsdesc); + d_addr = zsdesc_kmap_atomic(d_zsdesc); d_size = class->size - written; d_off = 0; } @@ -1904,7 +1905,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zsdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle;