From patchwork Thu Jan 30 04:42:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954300 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 102CBC0218A for ; Thu, 30 Jan 2025 04:45:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C1212800BD; Wed, 29 Jan 2025 23:45:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8794F2800B9; Wed, 29 Jan 2025 23:45:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 710EA2800BD; Wed, 29 Jan 2025 23:45:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 51DAC2800B9 for ; Wed, 29 Jan 2025 23:45:18 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E126C1A0889 for ; Thu, 30 Jan 2025 04:45:17 +0000 (UTC) X-FDA: 83062879074.15.B5C7654 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf30.hostedemail.com (Postfix) with ESMTP id 0FCEB8000B for ; Thu, 30 Jan 2025 04:45:15 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Lg2+Qz26; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf30.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.43 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738212316; a=rsa-sha256; cv=none; b=OkcR+bQcrv7YJz5LOXOpk01q0tMWxz4iKj1hAhFEmYDjREh5xDDEuax+krDYjHslWx5+ht f7lbozdTG6i5bce7cAl1GZss0R8LqRw+T2CuKLg8hMVOYFIe80FD2JFnobPymhjmrgdRbt P0+JMUl+XkI34iYJ7bxLbnkybT2qrO4= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Lg2+Qz26; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf30.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.43 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738212316; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qoanfGA+iKM2lkr04fEBeNmdyi5bnUfmaHtYhcyT7Yw=; b=BXn+WfshK5SnQ+xJb4uGC1megubgGoztcooIcL95PFMEUo1dClFf1TihalkUfUbbSe0Qjl RpUJoTgFCaWyYfxr32DSgRE9vA3RwYb5klQoEYnXf7uKtOOKk+XddheKP1acr/0tqd+60/ LEN/0J8fTzHa2HwNMtmIJifYIWLT5Bk= Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2f4448bf96fso454576a91.0 for ; Wed, 29 Jan 2025 20:45:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738212315; x=1738817115; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qoanfGA+iKM2lkr04fEBeNmdyi5bnUfmaHtYhcyT7Yw=; b=Lg2+Qz26jPG20CX64oJyAxZQGQzY4GspdE2Sxd67CKfoyeGoUkO01LUZJbOvLnQxP3 xJSfX+y45TXHGVmFkIlOCKlZVeghbS0KPfUp/xR2OF8Nb6AYnqUtyLc/24ROKLsh600y 1s67JnCciNGT9dWiyfTgFZHw8y2jkBYDHCznE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738212315; x=1738817115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qoanfGA+iKM2lkr04fEBeNmdyi5bnUfmaHtYhcyT7Yw=; b=HLJ/Lz/n5de1UsrRB0JwbuknX30kTLzeq4JEIpHA7S5sGOOrpjv1mpN572GIswBPSq 0OFUGyadtZ8PABOZEzjfkU3qFSC7PacppWZORDTXTq4MEyDQyE/GsfPBqqHy9Pd+qggD 5XNP8lq7fMxP+fTL88Kx8xc6CHHytIopOeUoB0+Di4lug15xOlFMV3IaaVCheXDqBVL+ jqEvwXzefJPO0V25jI32kO5XOxDA/9a0xcdm1HtviOSZOoBlPAMgszc6Bar8JUvX4+LA af4u9twS9n1qDBVORkaMyX+2WXedOfcMQyfyP7+oP5A/6TdzWCwXVK+7rYseMYbAxNrv +c4w== X-Forwarded-Encrypted: i=1; AJvYcCWAfIo4gIvT/i8fa6p9q6vobQebJbZRNl4fXhZtVLJaz/FOpux8tGU96ul/9ESH/Y3VkV+iLqHl5Q==@kvack.org X-Gm-Message-State: AOJu0YzwIToSqzxNeYLaURTMcKj4riBAQ/7mXkagxXNjV4EZTPF5VwUF tLkpBmE3vsDUwoD7DoIcrlC0XG/Xqiu5rm0MWiDPa0O7FYGeQmd4MReFtU6Euw== X-Gm-Gg: ASbGncvF8GhX9A1sZ3bdC0RSMchXgfgXlDFNkeLLBrbEtvaRv1NvGekWfqJbGkjuVnE DdLuExT4Mj9i5R/y9kmt/CCMdn2CRspesBaYVl2gtlgLOGSjiPuhcwAOnBycJo6mT8b5QvjQSmS vr4l9hweCUvCm4di7cW5R2V9ZAR/EDUdGMv/qCKoL57gWDr8yB3PZJKa5C8niiwjeUSpSDzTY++ YdbQAMJXGfKu2f1ok9f0ZPDqm5oMoBcAaCa7QFqXVh4ZnSQyv9m1e8O5teqOcqrjOAcFRY08D3m i8gezrshZjrQYfHJuQ== X-Google-Smtp-Source: AGHT+IF2UbQJQ89YEqryPhFxoeZX28FVlFBfWDCACr57M7BnO6sGVckgAr34ZiwemLJQw2ugQkpK1Q== X-Received: by 2002:a17:90b:5190:b0:2ee:ad18:b30d with SMTP id 98e67ed59e1d1-2f83abbd886mr7881534a91.6.1738212314683; Wed, 29 Jan 2025 20:45:14 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d794:9e7a:5186:857e]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f83bc98ae9sm3012541a91.10.2025.01.29.20.45.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 29 Jan 2025 20:45:14 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Yosry Ahmed Cc: Minchan Kim , Johannes Weiner , Nhat Pham , Uros Bizjak , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 2/7] zsmalloc: re-shuffle zs_pool members Date: Thu, 30 Jan 2025 13:42:45 +0900 Message-ID: <20250130044455.2642465-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130044455.2642465-1-senozhatsky@chromium.org> References: <20250130044455.2642465-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 0FCEB8000B X-Rspamd-Server: rspam10 X-Stat-Signature: 43uxzszyajhrdjbp6mw7ob39hdjhcmuh X-HE-Tag: 1738212315-917157 X-HE-Meta: U2FsdGVkX181+1XOihExfj5eP2Osiov5XARz1nQDFTpdJr+xsXTD5OHKlwpAkkfARG+41FRM56dzw5gsRp8ets38IZmvXA8L/APxRVcB+dsYx0KquA2XrTvLVcQ2i6vOAgcAGuz2MuzrrIQteZWtD5LETwKfS0hJh4VjCiU71LGGmi+2AHWCGBQp0gwsl0uhqDRrAyLRUF4BGoV93qGcnijDAM5F3jIJEaMsYxigR3J98X66ClK68sD/R7y+I6P3BaTtc/ON4P95HnlIuVlkfjiLae7XnK58h4aQTNKbn0ab45VSCF5vkXoAW17bVPXUH7+Fu/LGNsrk+mIh7CYzE5cALbMOWh+DH2jhIYKAgLJW6cvj82sNsz2m2bbOtO8PGmfI/te5hzDaKYCatK6PGbYay21EEG/E8hYj3xkHKzYBEHi1d7PjH929FNj6TQJbY0pzT/l3lIJHKq3Z8VD0ML68TQFu+me70nGiuQw+D/swkh7K5k1sEd5x2BKzVDFcv21B6oE0HuQ3Sk1F4aYnyy8rgmvLO1N/WgX1LUQobxDP4dSOPsx1x8pmgj4ihBx3R4kYW1pn8DA+Y0JLW/DxIam6sNwWEUwRAM8q5l6MHH1YMtAn4K885fGqp+qKaOj7AoG/FuSXY7l+jLgUSsKUloIuY1eaQJK8en7YA18Rr+TEy9KUMzfFoFpmyQXpMS6yoXXHctNiRL0MM8OFB/dlqEyoOZ9Q3IJclFOY78ifYvRkDYhyJAGWZ3BzI06gLwo6Gvt6z2oQXr/J7CwR/XieEFEv0ZRytmVNkJoNfPq4oDhv0u1DUahzdJGbfIA5lOAtn/qtFUMtq4S49ULfBXsUGI5h3zTDlD81ikd5I6ArEScF4lTNJQ4AT16iFsKTNtNq7PaisZTNxFMv5ZlkhO9UOSOr/yaB9uo8NrA+JkV5rIb5Q5Up5n9pslQIuzMtwR31/vESiA7YyRhHCsHYggq Yw0Pu/W/ rAVeV9SUzEpu2p53ZjGreHNurySW9lEOs0jF4e/1JOBsuXoyp9xdKZ0QP20rPzBo9+doJnfP2KzGMvTCMAgmkDtZoCWN4oeJqXN4/YoeGR/ZZtPslQUJHF05uXDh9vrgBb9Jf9ThEbG/bPLwiy93/+Jq5YNqwf6/geFKVNDphlzYU8TkLrv2IzP1zEv55Sbixddhl/ljvuoye6oHflwXo/d8YX9B05RJFnCg2TDadoIX1drBrvXXJgvrPvU1FhwR6+sKJQeU+fguce5VcLdZzEo/RlVruPOzBhUq3hrKm5WQS+MjyUQDi495Ir7NgrVIrMiQCuYEJhmueMNwV3XDWZ9KWvaPY3r9kokXdX2PNluGtP1g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is no need in having ->name at offset 0, it's never user and is being followed but a very large RO (after init) array of size classes, so each time we need to lock pool->migrate_lock (which happens relatively often) we need to jump over that RO region. Swap ->migrate_lock and ->name. Move ->pages_allocated is modified relatively often, move it up right after RO size_classes arrays. We modify ->compaction_in_progress and ->stats only from compaction (defragmentation), which can run only on one CPU at a time and, depending on the case, may or may not be common. Move both to down. Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 8972b4b56cd3..2280ea17796b 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -204,18 +204,15 @@ struct link_free { }; struct zs_pool { - const char *name; + /* protect page/zspage migration */ + rwlock_t migrate_lock; struct size_class *size_class[ZS_SIZE_CLASSES]; - struct kmem_cache *handle_cachep; - struct kmem_cache *zspage_cachep; atomic_long_t pages_allocated; - struct zs_pool_stats stats; - - /* Compact classes */ - struct shrinker *shrinker; + struct kmem_cache *handle_cachep; + struct kmem_cache *zspage_cachep; #ifdef CONFIG_ZSMALLOC_STAT struct dentry *stat_dentry; @@ -223,9 +220,14 @@ struct zs_pool { #ifdef CONFIG_COMPACTION struct work_struct free_work; #endif - /* protect page/zspage migration */ - rwlock_t migrate_lock; + const char *name; + + /* Compact classes */ + struct shrinker *shrinker; + /* Permit only once compaction at a time */ atomic_t compaction_in_progress; + /* compaction stats */ + struct zs_pool_stats stats; }; static void pool_write_unlock(struct zs_pool *pool)