From patchwork Fri Aug 23 19:04:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13775728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3126C52D7C for ; Fri, 23 Aug 2024 19:05:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B51E36B027F; Fri, 23 Aug 2024 15:05:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADC266B0280; Fri, 23 Aug 2024 15:05:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CDC8800B4; Fri, 23 Aug 2024 15:05:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 60F566B027F for ; Fri, 23 Aug 2024 15:05:52 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0B1B9121CF4 for ; Fri, 23 Aug 2024 19:05:52 +0000 (UTC) X-FDA: 82484439744.02.0940B07 Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by imf16.hostedemail.com (Postfix) with ESMTP id 1C39E180009 for ; Fri, 23 Aug 2024 19:05:49 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JX0vYuef; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.219.46 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724439889; a=rsa-sha256; cv=none; b=vy3jyTxXAnz1+EtCvczqL4mPnxNX73oxPgs/hTIWy284JXZ1HOKIhifAzB6aOM2ET32A02 UQo3cwO75RDnHEAJ3Ioy/Ykf7hZKmO1lsfVvfMa9Ex0s859Rs8u8JxZs7KpF0nxkGHCVSk XvL3bKuWHjvIdoe3by3hbl31bw42giM= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JX0vYuef; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.219.46 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724439889; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KXz1WqDOSOf0lx3PDODMvA3lWYB2E2RmwkpOqwgzHOw=; b=sGus3kdkptKCEIE4OXVPytUMlxsmju4TdXrmH3boWEqL3hquuJS2QeKvWVL7TmpE0EXwWF OQv8fu+2yNbxImg5Q9x5ZwJZgK1vbCfh1tdTZTJ4wkQXw1qwGLCn4s5MlEMiQWFrSZiasK 69YV6XApfwvDCSQh/KMZ7pEsrEBkMdw= Received: by mail-qv1-f46.google.com with SMTP id 6a1803df08f44-6bf7f4a133aso11673786d6.2 for ; Fri, 23 Aug 2024 12:05:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724439949; x=1725044749; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KXz1WqDOSOf0lx3PDODMvA3lWYB2E2RmwkpOqwgzHOw=; b=JX0vYuefLCuJJHA9zuu/zd2jf+e1q6Ii1vbA63Of1UBCg3osINvQLYKslREbFBbRwW yAM7qOx1DhFhm2kTLXLlrrkehlNcp4FNUybnIfWW/4ieyu8i2KqSnQKA7eZV792NVaiR Ui8ELSg71GjuVzh8vLHFMEzGoqbkWqeAlyImE78xXhd8rino+UmGYU3SiBSRGbRMQgLs bBTE2KPRDYX+fap67vYinQEuoKy7dZL/3p7kY30IAnitUIJceWEq0tUuY3GfUnvqhudQ IlNb/crxrOq04qV3LyisKoZUkUa3J0zv3ZBZX/6xiTBucOKYmdJrcR7cbTQm5G5QCKwx 4bDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724439949; x=1725044749; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KXz1WqDOSOf0lx3PDODMvA3lWYB2E2RmwkpOqwgzHOw=; b=NxA+F97MJa3ZVVPIFHXziZ3ghFiA/y480owRU/s1YX1yMrmvPXJ7Igl4Xcndjq+ewd w2Ic8bpIfvVrCEtN060jxcwL7Sj8s5k/teMqmhMbpikBIQ5bBhLr2+O6A2cMVRcKG62b wigcC04L+c+mbcBWBZ3j5AUrqx46LxP1wLZZBfu4XQSQmZbu5EAUruE95L7D8RNPIWV4 lZEqYY0mJvp6fu8uU69m5RzFXzVv5LoY2IJtwuORqRfjP2rf4X0jGCWWWec2eJeEtiTW aoo2Rl7AFYQgCy0pye4qBeSohCcePbYpTMjHIh0mE8MC9gq/cLW3rpkL7vuOKDyv2DcE ASHw== X-Forwarded-Encrypted: i=1; AJvYcCUz4VT8dtHskIff608QuMgHeYseSC680SUBp3YHO/85CrMVFjjF7GMYy1lIJxhapwAlcmGevKsFzA==@kvack.org X-Gm-Message-State: AOJu0YzwDl2tjg6JGug1+4k1hEoX90cuQ2Yu4WcKtcQiPffdn229Uhco q9+fmvfZGa0eCcptodLCoOCCR36iHvlWT2zZHigZrrvjPFISapwm X-Google-Smtp-Source: AGHT+IE9Bd7WW/rU0Ll51OR3EAyq5JreQRHH8kzeghLjMl8faLxlPdzbZDil5qdNrBwCfKa0NMZlZA== X-Received: by 2002:a05:6214:3f83:b0:6c1:6b23:dc9 with SMTP id 6a1803df08f44-6c16dcbd32cmr35351236d6.37.1724439949011; Fri, 23 Aug 2024 12:05:49 -0700 (PDT) Received: from localhost (fwdproxy-ash-018.fbsv.net. [2a03:2880:20ff:12::face:b00c]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6c162d4c1c6sm21607346d6.46.2024.08.23.12.05.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Aug 2024 12:05:48 -0700 (PDT) From: Usama Arif To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, shakeel.butt@linux.dev, david@redhat.com, ying.huang@intel.com, hughd@google.com, willy@infradead.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Usama Arif , Andi Kleen Subject: [RESEND v8 2/2] mm: remove code to handle same filled pages Date: Fri, 23 Aug 2024 20:04:40 +0100 Message-ID: <20240823190545.979059-3-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240823190545.979059-1-usamaarif642@gmail.com> References: <20240823190545.979059-1-usamaarif642@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1C39E180009 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ab9iy3g5exchixikwhrp35xy7eqs6t5y X-HE-Tag: 1724439949-135344 X-HE-Meta: U2FsdGVkX1/tFNV9zsjBrZnmLstkZ/UfHYLB06tI1/9LgP5d4ID8GJCW97wX3SxW1A7pvl5N07gWP9xKPX2gTvk4B7vBNg1tNNdHt92hbkBPoSf5B3y02QCZlQtOj33zijaRNmyomIJwcqSQGGSk1p8bcyYaf/e16AnynCtBexbbmKnPHGXEeUjEvCDvEwtAMGf4/mK6AWF7ofiI5xp9ypOzPmjSrYiFSKu2onxASRSOdK2/ooQI4+ioS3fW0zzv41m+FrZK9MONA/iYX0HMDxqJxkJJSKXC9tfbmKqoohmqttTr9Yi1+jP8vd4CMZIKCIBMNxJ+2viOcRhVpwBk4rOYGpVP9D0IZeSyLhRFIMpq0ECxlg29hwUAIR20/Dt+WZ4Q9yxR+bT9ORYalc9PyIcfz2gVoLBxHO4VUDXGjBJ0vqhqtnfXDJirmZSJ4wvfQdEk4xJxWU4piQccakbKBFjR//GzrHxYDYRignqIzOib0Km9944G2F3UN57jcmsat9v4dYgMvnV0W9ezFE3tjgyrDsVcgH1x8OyAFHx+o5mk8vskGiQdYuwuXDYS9ZpLLQ8+rcAwIyJwIzO7nWxHyOnHUTKg7lveLzBtwfzv2dvbmNEp3NBODxSyhn7vX/kh7i1zcdaRt+oOEmmjOeLJWXB4xOIltQc1i4Xqm7HWYSImS5f/TZ4y0ba3wq2ISA8OAeBy4P7SiL9XFNgXeSIK0NsGJkurUzNd0CNEcpTziNEc50VvWjAaHdSLW8XPZjdKCEnz5dLtSIUnlMkObRzeCKDn0kViIoOFh/V7u20Qp6yFfM25BnkUS9i76py4lvJGJ/RSziiweSGpFaVDjqJj0i41F1q65eVEe5jFqwS1Rt2lME/DNNyyxTCTvMVTgFwimiuWHUxyqxdinkEMWZN3bgTpZKd+tUR0Nd5+VD4W2t3hcEo9VtZS5KGAO4nHrtY+BiGw3oXQFEzv+cin/6S YTACgqqm 911Cc5JZv+WTYst1vkAf4PzOcS4qKqVCYj4V8QrtxU799jfl0ndgrffLUIlvWXsX8PGl4TCyCcRY8FyTzC3ZfmG75DLqHxS8V1uT9jphgtq+ncRhhzr6aAbc1u5zGu3/dz5PhiG+uDu1F3GCVK62dQhYEZCopW9k1/b9pR0+jv4UVJJ2/w9BBfBO09iE2ILtmWS6Sl/ES6Ffepz4LK30UJ8jA/+pH57rDy/o1G7TbiMxire8lptAdu7+MupMr/bPAiNSFWmWQQPh7dkqXkqzCrJsnQm6hmYNE/9qOLA+Frk01TIf4Dze81UdKviygjy0kMwpQtQ3Z2bG4ZW6A1bbect8LhF4u44eCDpWHjc7w2LuLFP9ohTSsMOXkRK8es97zoYeqhcGSuG/nBZUSUgToETwhnKgDzkJBizrEoU30IsILCy2qhtGmXMjHWeojn9feHpgbBgeumVM5yvh4j7byuZze0TYljPmqJfqzjHhzUmCooMvu50MpqtMkqxRk7VhAdSF0EzQ1Qq9P4oQKClZoLa9wbmJR7OUFAqpVS7xjOvXXuQASZwKnbWq5L/JcwZsCfhO2uaT9r5L6DmtcqaHuJqg42D8flihNh1NqssTcSvtsX40= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With an earlier commit to handle zero-filled pages in swap directly, and with only 1% of the same-filled pages being non-zero, zswap no longer needs to handle same-filled pages and can just work on compressed pages. Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham Cc: Andi Kleen Cc: David Hildenbrand Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox (Oracle) Cc: Shakeel Butt Signed-off-by: Andrew Morton Acked-by: Johannes Weiner Signed-off-by: Usama Arif --- mm/zswap.c | 85 +++++------------------------------------------------- 1 file changed, 8 insertions(+), 77 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index df66ab102d27..449914ea9919 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -44,8 +44,6 @@ **********************************/ /* The number of compressed pages currently stored in zswap */ atomic_t zswap_stored_pages = ATOMIC_INIT(0); -/* The number of same-value filled pages currently stored in zswap */ -static atomic_t zswap_same_filled_pages = ATOMIC_INIT(0); /* * The statistics below are not protected from concurrent access for @@ -185,8 +183,7 @@ static struct shrinker *zswap_shrinker; * * swpentry - associated swap entry, the offset indexes into the red-black tree * length - the length in bytes of the compressed page data. Needed during - * decompression. For a same value filled page length is 0, and both - * pool and lru are invalid and must be ignored. + * decompression. * referenced - true if the entry recently entered the zswap pool. Unset by the * writeback logic. The entry is only reclaimed by the writeback * logic if referenced is unset. See comments in the shrinker @@ -202,10 +199,7 @@ struct zswap_entry { unsigned int length; bool referenced; struct zswap_pool *pool; - union { - unsigned long handle; - unsigned long value; - }; + unsigned long handle; struct obj_cgroup *objcg; struct list_head lru; }; @@ -801,13 +795,9 @@ static void zswap_entry_cache_free(struct zswap_entry *entry) */ static void zswap_entry_free(struct zswap_entry *entry) { - if (!entry->length) - atomic_dec(&zswap_same_filled_pages); - else { - zswap_lru_del(&zswap_list_lru, entry); - zpool_free(entry->pool->zpool, entry->handle); - zswap_pool_put(entry->pool); - } + zswap_lru_del(&zswap_list_lru, entry); + zpool_free(entry->pool->zpool, entry->handle); + zswap_pool_put(entry->pool); if (entry->objcg) { obj_cgroup_uncharge_zswap(entry->objcg, entry->length); obj_cgroup_put(entry->objcg); @@ -1277,11 +1267,6 @@ static unsigned long zswap_shrinker_count(struct shrinker *shrinker, * This ensures that the better zswap compresses memory, the fewer * pages we will evict to swap (as it will otherwise incur IO for * relatively small memory saving). - * - * The memory saving factor calculated here takes same-filled pages into - * account, but those are not freeable since they almost occupy no - * space. Hence, we may scale nr_freeable down a little bit more than we - * should if we have a lot of same-filled pages. */ return mult_frac(nr_freeable, nr_backing, nr_stored); } @@ -1416,42 +1401,6 @@ static void shrink_worker(struct work_struct *w) } while (zswap_total_pages() > thr); } -/********************************* -* same-filled functions -**********************************/ -static bool zswap_is_folio_same_filled(struct folio *folio, unsigned long *value) -{ - unsigned long *data; - unsigned long val; - unsigned int pos, last_pos = PAGE_SIZE / sizeof(*data) - 1; - bool ret = false; - - data = kmap_local_folio(folio, 0); - val = data[0]; - - if (val != data[last_pos]) - goto out; - - for (pos = 1; pos < last_pos; pos++) { - if (val != data[pos]) - goto out; - } - - *value = val; - ret = true; -out: - kunmap_local(data); - return ret; -} - -static void zswap_fill_folio(struct folio *folio, unsigned long value) -{ - unsigned long *data = kmap_local_folio(folio, 0); - - memset_l(data, value, PAGE_SIZE / sizeof(unsigned long)); - kunmap_local(data); -} - /********************************* * main API **********************************/ @@ -1463,7 +1412,6 @@ bool zswap_store(struct folio *folio) struct zswap_entry *entry, *old; struct obj_cgroup *objcg = NULL; struct mem_cgroup *memcg = NULL; - unsigned long value; VM_WARN_ON_ONCE(!folio_test_locked(folio)); VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); @@ -1496,13 +1444,6 @@ bool zswap_store(struct folio *folio) goto reject; } - if (zswap_is_folio_same_filled(folio, &value)) { - entry->length = 0; - entry->value = value; - atomic_inc(&zswap_same_filled_pages); - goto store_entry; - } - /* if entry is successfully added, it keeps the reference */ entry->pool = zswap_pool_current_get(); if (!entry->pool) @@ -1520,7 +1461,6 @@ bool zswap_store(struct folio *folio) if (!zswap_compress(folio, entry)) goto put_pool; -store_entry: entry->swpentry = swp; entry->objcg = objcg; entry->referenced = true; @@ -1569,13 +1509,9 @@ bool zswap_store(struct folio *folio) return true; store_failed: - if (!entry->length) - atomic_dec(&zswap_same_filled_pages); - else { - zpool_free(entry->pool->zpool, entry->handle); + zpool_free(entry->pool->zpool, entry->handle); put_pool: - zswap_pool_put(entry->pool); - } + zswap_pool_put(entry->pool); freepage: zswap_entry_cache_free(entry); reject: @@ -1638,10 +1574,7 @@ bool zswap_load(struct folio *folio) if (!entry) return false; - if (entry->length) - zswap_decompress(entry, folio); - else - zswap_fill_folio(folio, entry->value); + zswap_decompress(entry, folio); count_vm_event(ZSWPIN); if (entry->objcg) @@ -1744,8 +1677,6 @@ static int zswap_debugfs_init(void) zswap_debugfs_root, NULL, &total_size_fops); debugfs_create_atomic_t("stored_pages", 0444, zswap_debugfs_root, &zswap_stored_pages); - debugfs_create_atomic_t("same_filled_pages", 0444, - zswap_debugfs_root, &zswap_same_filled_pages); return 0; }