From patchwork Thu Feb 23 03:04:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13149800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07CA9C61DA4 for ; Thu, 23 Feb 2023 03:05:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 761CE6B0075; Wed, 22 Feb 2023 22:05:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7117C6B0078; Wed, 22 Feb 2023 22:05:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D9266B007D; Wed, 22 Feb 2023 22:05:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4D1606B0075 for ; Wed, 22 Feb 2023 22:05:01 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 144CE8102C for ; Thu, 23 Feb 2023 03:05:01 +0000 (UTC) X-FDA: 80497064802.11.1EECE5C Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf06.hostedemail.com (Postfix) with ESMTP id 3E77918000F for ; Thu, 23 Feb 2023 03:04:58 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=GqW5JVFu; spf=pass (imf06.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.49 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677121499; a=rsa-sha256; cv=none; b=Az80isvPwhaDl1m6btr1Bs6LyLHD7HMlDyCf9xJynrosA7mURL1aG24i8jWSd/RfGtAMP1 9N2Ns4mfTGaInhUePikOj7y57+y9eFGPrJ7KHfr6uxWXintZtzo5hOGVYQzFcrfMg8HMZ8 Q7smjPajerOlibIe73x3pK3m8exlvAU= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=GqW5JVFu; spf=pass (imf06.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.49 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677121499; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=asf6wyECn9s5Ay0jPFpgYQEaM+Qi9v9+4K9yS1D0J28=; b=Swz/9OT2bpyntoeyvneOfV2Bb1hsrmfnpsDht/BMdoYZIM4AAHv1RekjnDUG7tweVvbj5i llgFxRvhqBR+H8SBufbl+tzRoWTxRmDyudW8fm7RLLi1n12Nv1whMzyDYmepWPebW5WTBw 4EY6FzdCxKH/hFYgwd72yBBpcQijt+g= Received: by mail-pj1-f49.google.com with SMTP id u10so11332967pjc.5 for ; Wed, 22 Feb 2023 19:04:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=asf6wyECn9s5Ay0jPFpgYQEaM+Qi9v9+4K9yS1D0J28=; b=GqW5JVFuXU7tv2rHfd8Pt7M7iy6Q2n1a4GMuUZashCmIOIlVkOSJRYEUBSVwU5p6BN HmcFaeQO/ndJkTMhziVkwhsUDE+mw2yq2JVYc3K7T1mKO0Z7l704UPaTXezHLwhMagoV tHQAbr94XZ2e0HAwUdwmKFL5dxpzFuOJz7rGU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=asf6wyECn9s5Ay0jPFpgYQEaM+Qi9v9+4K9yS1D0J28=; b=oiy7G5JBplXojcsiPROXIITh/xxsaeLQ1+5eE7SJHee2XPLMzzgH7RR4eHxL5c9xyC xyupY43xHwJ2WkEszq1041Gc4NCDTWvAA/8SiFiCre7kajGFHw02YxxEOXNl2k8NV6af AF0s37DMbS4LSbV0wbyOmIzjutq0XdRkaHZcdV15GVCaUVLkOFRYPiodUfbgJnANl6qC KXblrdUbqidhTvo8HS7lzZmBqiql2R+02VRiNbBJ3aub1ayYHMft162V3SnTWRWzABH6 2V8bK3yHRO1ejbXTf53YD1uN/yu3scVnzPZ3d04/KB5heBSSuW+v4tJ7um4yyyNBH+Xe ykgw== X-Gm-Message-State: AO0yUKUPlst8LKaOIK6WvjNPQiYwho8qM8iwK4/0rOENl2sObqAp5N1e 0oYHHt6aKoPRuqreD9VcGy/PZShRpHnmvRcg X-Google-Smtp-Source: AK7set+EATz74Pic1+zOa/tByny2R+TyuhM/yQ3zojLfWsgEkDn4lxaa1dTV4OZ51+F+v4AZoFtXow== X-Received: by 2002:a17:902:d482:b0:19c:356f:e98f with SMTP id c2-20020a170902d48200b0019c356fe98fmr13032448plg.60.1677121497854; Wed, 22 Feb 2023 19:04:57 -0800 (PST) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:6de2:9e85:b508:57b8]) by smtp.gmail.com with ESMTPSA id jl21-20020a170903135500b0019926c77577sm608520plb.90.2023.02.22.19.04.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 19:04:57 -0800 (PST) From: Sergey Senozhatsky To: Minchan Kim , Andrew Morton Cc: Yosry Ahmed , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv2 0/6] zsmalloc: fine-grained fullness and new compaction algorithm Date: Thu, 23 Feb 2023 12:04:45 +0900 Message-Id: <20230223030451.543162-1-senozhatsky@chromium.org> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 3E77918000F X-Rspamd-Server: rspam01 X-Stat-Signature: 71e957mto7ajm6onwcman183duxj33gy X-HE-Tag: 1677121498-528715 X-HE-Meta: U2FsdGVkX1+WEqRVf1jz2YW0zpqaFSNRo+NCzM4qPqvTkcyDcNXXbhky6IkBERFz2YJznvcpcl3SpChpbi3wbqKzOI/zqpbwkVhFbL9XWy9NV3wlu13d+A/QoVs6RP5QCCWGwYp8t8RUQlg3v2sEvwiXVBkcf4a3axkkxVwWrPO1pcwYrVgmVwiDtajHEletKiMBY1D5nymKL+YAqTTMpINaAPhnpA4FHhP/X3NTqPXVL9BpHiPPFcwxvT1g5SMCkNw58vsP+vRHg7nzPebCW7fGmYT9GgklZpsrZHZAglzhUwJcLUWfixz2nAKi6JcLxiGTeZH9dmJIshRhZfdOgKfwd1wMPCAKVu3o0tE2rhcKCDERUF6mWF6EsTIjBRmiciStrNCyUdalh+dyPlx28bNalgbzfDM3CNBbELSRXfCqImqlEHCd9eVgEFut0quoeHten8iRmZw0CGLjOJqCJgqTsIC85p8NTuxe1Thjaq52yug1Ijq3mxRiTKw2ub5NWP4RJN5do1BMYAeOzj3NcuqG8S3oSJUIrrcMemNU7RPsczJ+u5GK68vZH1LzBNcRzRjTClRI6v0/W0IZneCNLGUu/VEXRED20FyXss8SUki1PLjkYgmKqOl7Y58Sdwnm0TUgcxXr4iJXpA7qlJjxPQkm/xQlrT2tg80Y4boYJagAwvNY8rObAR4+aRmO4pw6ViuiZIOkxYKMhmxCAzF5aS/l473a+Vua17tCQvzgnyrMgKqG/0Itr4RsXAJW0npSH29gyVUufARLCjUCwEvSKoNSmUL/E2XAW0+gFol4vv1HuDgFeGYOdR1M4+G7KXnPPI+bfWhIv2D521vR+b+ZZnSLbyQWZK1JfGe8NOlGqap62jHB3L2dEifGxCZo0CSd21+RjSoIA4e6+UmoAZkU+dp/lWGsajvTIahyzzobZKWcuvdRp88agPaza/aDuJZQQW9mMBgekohaPyXTgMB QD7fIQUO fhSHJf3xASOPK5nx0Qw5DUkC1F+T6xgKvF5vYCEHBHVhXFXemkt/jCWg8hk90J6AY+/HrRdJet82PdqrvgvL+p2DTHZcXKYwBl78V09PvIV62L9CQcysnxqrfUzwD6BdAA9kykCqUX+FhICMUZnru1p5XmPkyPznBNkHLRXQs5K5ST4UpS82OeKcJoFxCdZGF7nYEAknNcNHUX1QwrZu/06q4fCclCR7cClvTE2xSs8fHS5kJfQU3//MKG7bJ2ptAt5WVY10WXHsB4XHHikSMBpp3fyuCK0IREPpnTCbNOaRycvga0D1LOvcQgkshQaJ9dA6LrwgN4gY7PS/fovC3xyEhv/KKPcqmt8xp4p/ix+SO6KQvOcWzDaqsJmhL3q+ZnX2h X-Bogosity: Ham, tests=bogofilter, spamicity=0.000056, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, Existing zsmalloc page fullness grouping leads to suboptimal page selection for both zs_malloc() and zs_compact(). This patchset reworks zsmalloc fullness grouping/classification. Additinally it also implements new compaction algorithm that is expected to use CPU-cycles (as it potentially does fewer memcpy-s in zs_object_copy()). TEST ==== It's very challenging to reliably test this series. I ended up developing my own synthetic test that has 100% reproducibility. The test generates significan fragmentation (for each size class) and then performs compaction for each class individually and tracks the number of memcpy() in zs_object_copy(), so that we can compare the amount work compaction does on per-class basis. Total amount of work (zram mm_stat objs_moved) ---------------------------------------------- Old fullness grouping, old compaction algorithm: 323977 memcpy() in zs_object_copy(). Old fullness grouping, new compaction algorithm: 262944 memcpy() in zs_object_copy(). New fullness grouping, new compaction algorithm: 213978 memcpy() in zs_object_copy(). Per-class compaction memcpy() comparison (T-test) ------------------------------------------------- x Old fullness grouping, old compaction algorithm + Old fullness grouping, new compaction algorithm N Min Max Median Avg Stddev x 140 349 3513 2461 2314.1214 806.03271 + 140 289 2778 2006 1878.1714 641.02073 Difference at 95.0% confidence -435.95 +/- 170.595 -18.8387% +/- 7.37193% (Student's t, pooled s = 728.216) x Old fullness grouping, old compaction algorithm + New fullness grouping, new compaction algorithm N Min Max Median Avg Stddev x 140 349 3513 2461 2314.1214 806.03271 + 140 226 2279 1644 1528.4143 524.85268 Difference at 95.0% confidence -785.707 +/- 159.331 -33.9527% +/- 6.88516% (Student's t, pooled s = 680.132) Sergey Senozhatsky (6): zsmalloc: remove insert_zspage() ->inuse optimization zsmalloc: remove stat and fullness enums zsmalloc: fine-grained inuse ratio based fullness grouping zsmalloc: rework compaction algorithm zsmalloc: extend compaction statistics zram: show zsmalloc objs_moved stat in mm_stat Documentation/admin-guide/blockdev/zram.rst | 1 + drivers/block/zram/zram_drv.c | 5 +- include/linux/zsmalloc.h | 2 + mm/zsmalloc.c | 365 ++++++++++---------- 4 files changed, 188 insertions(+), 185 deletions(-)