From patchwork Mon Dec 2 18:41:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13891174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 998C6E69E81 for ; Mon, 2 Dec 2024 18:42:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E5BB6B008C; Mon, 2 Dec 2024 13:42:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 296B76B0092; Mon, 2 Dec 2024 13:42:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E8046B0093; Mon, 2 Dec 2024 13:42:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E21076B008C for ; Mon, 2 Dec 2024 13:42:15 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 915638025D for ; Mon, 2 Dec 2024 18:42:15 +0000 (UTC) X-FDA: 82850888148.17.7D23726 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) by imf27.hostedemail.com (Postfix) with ESMTP id 434084000E for ; Mon, 2 Dec 2024 18:41:58 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AJNrUOV4; spf=pass (imf27.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.215.169 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733164925; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zPi/40j9fg7YjPYY/QjOr+/MPbDnr/kKPaDCNQF5gDU=; b=8MjjyWk9j2idfqFumCIyltdqbu1vPC3z/LSaJDl4cZ0ibWnCV114IlXyOvUFmMJGgw8uQ5 dp0O8LuMOrjDAMmWGbajRB0NWyE+NQ3Z5mLw9H5U3eYXRhC1d7nK22MF2Zun+jKv+LPVuN Y3i0g0leESU8OHWh/Np1QhGNtk6NCpk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733164925; a=rsa-sha256; cv=none; b=bpQSrUax7lYx3R1dqP/eIkiQvXmCTxiVSTL/h1EFnI3QUfWDuN1+tcbFPEFaIbuepPEEz/ XIDL0U2Vi1y/MS4qLKjtvq7xgQqMfVhElJKJgaRScD4KxdYgtfr6uMRoXVxi2adyrSfOsg JMbzUBooqZrBwCP2/ztKOsRBeMV3wHg= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AJNrUOV4; spf=pass (imf27.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.215.169 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-7f43259d220so3013537a12.3 for ; Mon, 02 Dec 2024 10:42:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733164932; x=1733769732; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=zPi/40j9fg7YjPYY/QjOr+/MPbDnr/kKPaDCNQF5gDU=; b=AJNrUOV4jG2RuM/je6ABxA7uGYSnMM6wdPRNg9BDlSlHikVoEX/0UV1Fs4Pn/CA2hl D2CNZa6wEunLtD3cm1CjbB/SNoIEqxp7vNfTOC6yiE2lIb0BMx/94Bq+nJT68gMUv4kE i8pyBzOQ8EcEnmwl68Dw+Irrir8dlIqi4EvFebxcOSzBE586t6eNxqy2/+y+HNKbREn4 ey349yJ02vJ4kE5YVBsA3eWzA8fUPJYhVeCD9+uLfpOAacUAyy3/JRNoy/7EQSRYxHMr 5KUHobcZqhBdGxH9YxQPhPdktVY0CO44GUFbpyfU+lGwkDSlhKk69+vlAWiLUblTCB3x yHcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733164932; x=1733769732; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zPi/40j9fg7YjPYY/QjOr+/MPbDnr/kKPaDCNQF5gDU=; b=Ay3kfqrxbkDOQdrTKAhQnbvLoIFLAI9MFWPUiu1glwdKquuCR4P+YuEBFnp36OLj16 BVeJUlDr4ROgvAaUag2+pmzUBjGY7UVkYnLAGpcqgLd9NExsuuM9hrWzXOtiE1eRLyet RLkPhsX8KW+skxSb8fQBHcFlbvCmCKFaopc0Ng5R6NzqPv5C3QeNW0FitihEW6urMI46 LvwDJi9LR4gTPqOpmWuR/x7dLgg4qb74Y3Ct17W2mBti3caKCFDeU0qgTDgC+XNP6dcM 7N3/UkM/nWZGNTthr4o9phukkPAgY1AmE7MlkiSkAvAJDnIkrL3KF+iX/VP7sREZHwze /psQ== X-Gm-Message-State: AOJu0YzoFV3qD0S8KWM/+lZhZSaxcMKkqlYukrcKYfEPV/v8zSt/91wh L+b8rQel09wFd58kW2If/iriXVI1N48nyu1vzGG6++xsRjTu2JxPJMTBvPW7gBdL5A== X-Gm-Gg: ASbGncsfzT68AqMXHYI84yNBtwVQ/ISE6sLt96k1KwHxL03N5L47vEiX8do4qF38eof GyYJGnrKtiD6+a6noDyD+mtDpsJcNncDytdxwpObiWXBXOmEhFbnOft3xT1ZuaRRp8uJiWGlwf5 Ur+n0GeWTZ6dN/CIpEV/rQUkcH1MWuNKWzLCxwZv7w9FNq1nVg035wrpjRt8JU2Vs3a+W/qkZSC t/Wa1EizDwK5iL+fEMBS55h5Hb2ndWmePr+a9FaTX7/oOEWulMOLxd+2KHPsJ+sacRAFpU= X-Google-Smtp-Source: AGHT+IE35vF16FjRydL5iBcJInocVBicSo1ZjyULb2W4jf/KJQsy63zCROjVa8zaUbfc3UsLb+zrGQ== X-Received: by 2002:a05:6a20:7350:b0:1e0:c378:6f5b with SMTP id adf61e73a8af0-1e0e0b71f30mr31209157637.38.1733164931986; Mon, 02 Dec 2024 10:42:11 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.120]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417fb7d5sm8849781b3a.94.2024.12.02.10.42.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Dec 2024 10:42:11 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Hugh Dickins , "Huang, Ying" , Yosry Ahmed , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 1/4] mm, memcontrol: avoid duplicated memcg enable check Date: Tue, 3 Dec 2024 02:41:51 +0800 Message-ID: <20241202184154.19321-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241202184154.19321-1-ryncsn@gmail.com> References: <20241202184154.19321-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 434084000E X-Stat-Signature: 45iwekee9da5aa949x6yqgtn1ia9ccp3 X-Rspam-User: X-HE-Tag: 1733164918-711307 X-HE-Meta: U2FsdGVkX18VizJLDbVMKvVGccbrtFwBACG+wCS2A+h4d/+fjoaW/+wv43XhIYFJlPuui2dD7zeXE9E3FaNp5DUkaRfnaD135AqNuCn3P6LoZ70qn/hHesDGNCcc2BDParXGzVXweabK2ytkr46L6gabULcOwr9Y/5u41BJgQ5PBiiPZDO1mUohocy+4oWRSidK/HySTutsx3ZGndpsbxrTGhSaOxhGMp8dLNyr9qMwIziblBhheuBnY7YkJGsCwbJw50mawqaVNaCpjbzws4zQz+2lk1AVIK74GI1xFLz4TR0iFtBh0lv72Ehj7xLSMWSTbEMKtIM/hk9vTgTqxZUCzNjEK8RNeXwf03EEp5/x1LVDZOm8xeB60LItVDjbUWoJW7QYQq1+eta+hvgSZCXk1mK1Fvc04t53CAazfoafl4ewyPvJ0l1YdUUrbTM0n/E9xoyLd8BL1jNIpnNZrcvNIq1qhGn1paiktGmHA5DpEBH2dZzW6jAjHLCLV0M7QLtcbHeTlrOZQIRujDvPqz3z6gHLDRdyAg0wuAs5d1mnJa5chg9LFREjuV4UP3SJ+wKWndvMdF0rtryqwCWBBCYq6UA9nt6klZKdUz1mzY6kj0Do1uiczmWIIoHypjBqyo2RWN6tXQaCEbgXfY3eITYvdvZmEqIBwqFWfgLtCZQFxZJWdgF6Dg+gSyl7GM9J0FffdXMGdM/pBnwCe46oAgZKfWFhGZBRbRUdwiOmXaPluPkHyBba1INtBCnIHFNCBerFOxbbUXE1r5edCGi6UnQIsUqQgOp3vuIGMXZvq5heGbokb1h5mT3MYuoHjgyn8BnlCvSbG0h6LOQ8api55nFn66eGbgSZFNorEbkCCSjG8SDzTxatGUoRM4YHCsrfQVGTc9pYMqc3+2bj8m041BN1/ehHGIdijWrmcL1RrwSMFb3hs+KyrE7LbbHIJ7v8htbSfOYzeYsFr9hnfZ61 hgr/Emfi oWYkymk/a1D/rbOTHaCYiUq+N5d6mWnD61ptsUbshRy3L1fQ19oiotNk2j68CfakNbpSj6XuyubFG5CygGkuzwXY4PF9T2hbG50jbZiMbbWv2hKpQIBytQNJlUlNMPEcD8V2S3StFf4x4Dxp6jlcI3po+9AdM3TiSi0HQ/RwukXm8K7tIeDMtb7I4ryQdIpgtBxVi9QLI8Rq5VpaE651sliXNYMpKSRmuunw2yTFAmceX1cTpHNsiW1bKOEWOKfo2HekLq++VBda4LDyScnHqg3XWuhYCOEivOAn3KgQlESSqTDygcdShNtTfzZYac+ME0HpvCBTDx3u3CzKQCrG8CcSsbOMedHDWSrdXbU+VjOuD1Mz+Ou0BFIQZG/p9kgvkgcv3wl/aXRgxqJaBaB2Hjo/llyhNEc6NpQEEjTRgPHCPOKNNVuRGW4XDHwaaRqJlHU0+EOTwcC9YfatNNeRr2TBFrknr9HsLrzhk0rr0+3z0gmP4nRKrKC313XwkzVDPKWn1SxRAk3OYTEw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song mem_cgroup_uncharge_swap() implies a mem_cgroup_disabled() check, which is already checked by the caller here. Skip it by calling __mem_cgroup_uncharge_swap() directly. Signed-off-by: Kairui Song Reviewed-by: Yosry Ahmed Acked-by: Shakeel Butt Reviewed-by: Roman Gushchin Reviewed-by: Barry Song Acked-by: Chris Li --- mm/memcontrol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b3503d12aaf..d3d1eb506eee 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4615,7 +4615,7 @@ void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry, unsigned int nr_pages) * let's not wait for it. The page already received a * memory+swap charge, drop the swap entry duplicate. */ - mem_cgroup_uncharge_swap(entry, nr_pages); + __mem_cgroup_uncharge_swap(entry, nr_pages); } } From patchwork Mon Dec 2 18:41:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13891175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A48ECE69E81 for ; Mon, 2 Dec 2024 18:42:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30DE86B0093; Mon, 2 Dec 2024 13:42:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BD376B0095; Mon, 2 Dec 2024 13:42:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 139E96B0096; Mon, 2 Dec 2024 13:42:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E40E06B0093 for ; Mon, 2 Dec 2024 13:42:19 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A99A1A0201 for ; Mon, 2 Dec 2024 18:42:19 +0000 (UTC) X-FDA: 82850888946.18.15659F2 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf01.hostedemail.com (Postfix) with ESMTP id 69D5440012 for ; Mon, 2 Dec 2024 18:42:08 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=NZWB4nLO; spf=pass (imf01.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733164928; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bgfYlxq8M7pj/EQVq83uOmq930B/5yWrzKvSMuqYhjs=; b=4jpbsab+drBjZx5RPzXMEAp/4iarhDy89E+6n85M8ntZZ97H1NeN6pPNbsV7IMwM1KH5hB o4p0PiZbOBHy75Qbm6YtAhRp+DS875W/WWhwytb9SBpi5hOacOAb+Sstq01zXon0UiKexO AjNCO0hZTwPPXx3XO6sKnIOvV8QGIoo= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=NZWB4nLO; spf=pass (imf01.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733164928; a=rsa-sha256; cv=none; b=cdckfXGz8AASn9hBxml6j8K/ToM6RCASu8CvbdelsxCYwM5ITSCxWktVKvz4jL1ADqLPtG Wvcauxqg24FaH+ny9jTfwulabmDDcFoBX4OeBstCd2EMET86siy7H0J6p9J1Ig6PvUUrS/ 1es2ZL1ngOjDUwoSven3VSYdVV8a55c= Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-724ffe64923so4809263b3a.2 for ; Mon, 02 Dec 2024 10:42:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733164936; x=1733769736; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=bgfYlxq8M7pj/EQVq83uOmq930B/5yWrzKvSMuqYhjs=; b=NZWB4nLOzshbcOODb1uXvncniQWsLpLdU7VGyTaVv9zInozOsok4RFVchwXWPOdwuF 37Zjt8Eoy0RYeNkioarvCh9GFuOmmI5PXX49zvWGbvW59bMZ/eIGpMdvrMhhyZJ54eWo H1YvvhrlHdR1Ag0xuYhg33FOBEMXWNJVB/ndcvppXKR0IyiqpsZ/vQJ2PZzaxWSGbmdc TCzI5TQQ7BBQJ7tVIOGg2obsbB8YPDdBhT8mXDOppu+2U65GiLSA7gH9Gpf8g+xH0+DB K8/4T5UKygalHgI7Yms/Pdl56wRXzl6vcnTChYKJZ6PLUS5wmaqZGE1gMU7LQmwC9QiA Fnew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733164936; x=1733769736; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=bgfYlxq8M7pj/EQVq83uOmq930B/5yWrzKvSMuqYhjs=; b=qYY7SYhSDp4Wf2TW4446I3aB0kxjA2PV++MGhhSs81BCI7rsxTqD4KU/r1m9nbixT1 0PKeX65MTSj3DWR++wmfB2UFGLKckuVBlm5yCtiUJ8xVrjbY5iXTMuzp3LLMudVv5Uvq cG/YauqvkeIEJcIs1ggqJm6Sn+u2Uyu51gVVLGBTOfhoah9YrnxC4yXWL3i/NWOHzm1F u7tPST7p8i0XU+EEgf4/WMpe5WQsojkJkHSgNw3+U6P2l/YAQn09h34jl5ZXo0iwSHUm G1Z1MRAnuNUnfwVH3a3WW9/F0g5/+LxyFjPOFKlG+zebzQ0seAnA+aAPvZasTj1mUrsm mHJg== X-Gm-Message-State: AOJu0YwQR0dqvda5QgDCCai4RwGE5GRAY/V+tumgGcfccMpYKPy4MVNV o8Cr3SHWuBjHC8yoORlqeHfCb0eP5cyku0YgriTpXHYjGJeQXNNrWa1zmxSf2TpFhw== X-Gm-Gg: ASbGncuGGXfMsoLUngyF3aU5xSlb8KlxKBGxPep+PjxI6zUI9rQPwk8f0ZxnmC457FH sqJ4gwyLay8ELBb3/avJAdzyNo4thtOCOytnLHnfSagD2czRACtm56WX2/meXXn7WTqbk9uCRD2 8sYThH0Rhh+C6gMpyIUZgzbfm3Hhyce2s+M+65KM/AH1/XQTi9WvPpuOICev5EhdhmTUfExAwwr 9cMeMf4watH88LNH2JzFqn2MlPG39Zg081QFK0kme1fPY0zJPj7E7MRAN2w+0Rc7GsAoEc= X-Google-Smtp-Source: AGHT+IEeppgHPbYXo/4IKwmclLHEdajSYeynVAjocWeV2X5vj8HNw3SKD3ti5afx7GbXPWkD1M5KNQ== X-Received: by 2002:a05:6a00:1c8d:b0:725:3bd4:9b56 with SMTP id d2e1a72fcca58-7253bd49c85mr31675949b3a.5.1733164936073; Mon, 02 Dec 2024 10:42:16 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.120]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417fb7d5sm8849781b3a.94.2024.12.02.10.42.12 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Dec 2024 10:42:15 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Hugh Dickins , "Huang, Ying" , Yosry Ahmed , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 2/4] mm/swap_cgroup: remove swap_cgroup_cmpxchg Date: Tue, 3 Dec 2024 02:41:52 +0800 Message-ID: <20241202184154.19321-3-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241202184154.19321-1-ryncsn@gmail.com> References: <20241202184154.19321-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 69D5440012 X-Rspamd-Server: rspam12 X-Stat-Signature: e5n6crw8c6rd74fme3h14hb754sedawc X-Rspam-User: X-HE-Tag: 1733164928-606878 X-HE-Meta: U2FsdGVkX19o+1SGT3n7eH2Z9nP3EmhzWGw1hJDV1Mve63OZ93nyJRpPwxA7eLZ8pSquij6Lfjt1zAhJQvl/EHYn42njmyX4HgIyET5pyKN8VjtVguDVDsi4/bMCLy1x2VXuytQz8XPeQlaj4/lDQYewlp0Nwnmg7OFASGnmtoWHY3o6cE9RWionur7BDPt5oN2WgjjbKLT8/0fXFrOzAZ/VmuaZoV4ITfxNMqgHodUAbgd68iUMYSvCt77Ekt/35VF2sA/GoDRNkM90Crl8lXtsmFYKtascJ6xtbVq7Qk7PP9l9K5LXG5qA5E5uB4W9HLChWMOUeWJPcTpwmlZy2HMoZHiWEnaCedhJkMJ4qX3ItRDMwRwNtDI0C7DWPpNCZ+IUM3x/QbuR1dhCtuj4I+JRXYu+hOJGcMLFRzFt53SZbvTXpCyQHkahx7XEL+coQSa6XxUxFK6WUV/MH6TU1rh7E54bcloXxH9ZAwgoBA8HRS6k7DCpFUQGNQwFRm3dT9HZmZuUQqZrQYLjZhHxrSgPIuW9KcM/d0iUQxWmYpk0J3CJgS2nO6YiAH9FwvXiFcBDRxj2HAnbnjVa7rN9MEluMOQizGyfUVAD96r50McxpWhqISDCY44efSlLZxO961GPgOueR1mLlG1ZT4dALdArYaOnrGV7GAnIiYafYsi5lTXU+gdUAyr6hG6e7kmLJsgb6hLYnNaUJJGZXUQyC/kycWcDj7VT31s6GPalUoN2fheRByij+8r4vwaS7nAk0fokfpAmAx3pDvQbKbW1sjDhUU5l8YTAni68M/Tg6yDokvfwz4UgaFA7iuUftfVJPcpcRTsGGcpTvPc41c4+CAw0WhHF09mXtZ9BSY8+rxxjVAS9SzfsRv9smMOBEQCqqVnouBtnxwF/cCXSogKEporNpjUJZC40jqRRVJv7ONmUdR+XUw8L+iI43PE8XEa3YPsBZP4TcAfjdc7J4rz MMkLhN7A MaQYA/UwlCD4yB7yn7YEeK1UrVopqeYX1rzLLM3k6VlzyI4L3WIGQ1sxoSJvRR/37aizeHIQGEFEp/bJK29FrZ6FhXy65kUlhlUooXfvmlqaaVxCMwGPxqMovDHbps60aKcJWtE7+75zZqJQuDcDdpVtU8PAK/IYJkG3zxUwG9V0hvT1u4zBO61DG3phf9BfKGJtKTGc3KPSOr4755FEnNp7ZyJQof1yIM98uk5NOr+XfvmJ9SKLpBCPJbzAUlsBMKLhFJIpR4v78fkyuULRc4hW4Cxl3vZJaNa7CTuTTjfCHfovYoZ2eo3eLYX1ZD6YgbBT0QXgqp1+2qG61C4pcdY1RJdWB7R1ug3ucJIHUU8LNXbLaADOUT0bv/UaykQcGcHChQjTodVRNegj6Src8qituSphQEncoKqACYgP50IdLfKP3fq6FD2J5zejJrZdnyxsNb9Vo/ELRfCRiX7iaSt/AVMGGS0BjssHDrVHPF5c04QFDDd/s0fJUCyy+2/4yMfVM+yKwZ1504+HAdJz4/nolq6bfxfwbagCERDN8BPBLUb6aD4AlDiWCl0m4W2lCQlQw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song This function is never used after commit 6b611388b626 ("memcg-v1: remove charge move code"). Signed-off-by: Kairui Song Reviewed-by: Yosry Ahmed Acked-by: Shakeel Butt Reviewed-by: Roman Gushchin Acked-by: Chris Li --- include/linux/swap_cgroup.h | 2 -- mm/swap_cgroup.c | 29 ----------------------------- 2 files changed, 31 deletions(-) diff --git a/include/linux/swap_cgroup.h b/include/linux/swap_cgroup.h index ae73a87775b3..d521ad1c4164 100644 --- a/include/linux/swap_cgroup.h +++ b/include/linux/swap_cgroup.h @@ -6,8 +6,6 @@ #if defined(CONFIG_MEMCG) && defined(CONFIG_SWAP) -extern unsigned short swap_cgroup_cmpxchg(swp_entry_t ent, - unsigned short old, unsigned short new); extern unsigned short swap_cgroup_record(swp_entry_t ent, unsigned short id, unsigned int nr_ents); extern unsigned short lookup_swap_cgroup_id(swp_entry_t ent); diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c index f63d1aa072a1..1770b076f6b7 100644 --- a/mm/swap_cgroup.c +++ b/mm/swap_cgroup.c @@ -45,35 +45,6 @@ static struct swap_cgroup *lookup_swap_cgroup(swp_entry_t ent, return &ctrl->map[offset]; } -/** - * swap_cgroup_cmpxchg - cmpxchg mem_cgroup's id for this swp_entry. - * @ent: swap entry to be cmpxchged - * @old: old id - * @new: new id - * - * Returns old id at success, 0 at failure. - * (There is no mem_cgroup using 0 as its id) - */ -unsigned short swap_cgroup_cmpxchg(swp_entry_t ent, - unsigned short old, unsigned short new) -{ - struct swap_cgroup_ctrl *ctrl; - struct swap_cgroup *sc; - unsigned long flags; - unsigned short retval; - - sc = lookup_swap_cgroup(ent, &ctrl); - - spin_lock_irqsave(&ctrl->lock, flags); - retval = sc->id; - if (retval == old) - sc->id = new; - else - retval = 0; - spin_unlock_irqrestore(&ctrl->lock, flags); - return retval; -} - /** * swap_cgroup_record - record mem_cgroup for a set of swap entries * @ent: the first swap entry to be recorded into From patchwork Mon Dec 2 18:41:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13891176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B72BE69E81 for ; Mon, 2 Dec 2024 18:42:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9EA226B0096; Mon, 2 Dec 2024 13:42:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 99A4E6B0098; Mon, 2 Dec 2024 13:42:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83B976B0099; Mon, 2 Dec 2024 13:42:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 65B5B6B0096 for ; Mon, 2 Dec 2024 13:42:24 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D0C618025E for ; Mon, 2 Dec 2024 18:42:23 +0000 (UTC) X-FDA: 82850889030.24.F182BDE Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf16.hostedemail.com (Postfix) with ESMTP id 88CC6180010 for ; Mon, 2 Dec 2024 18:42:09 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=R7vR7Sza; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733164933; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1aeU1SawrmUs4i99mcy6Gmav34YptGDoV51/MymT7EU=; b=W2Dz6Tz4pooC4+LAtPlVh/4j27uRFVYAEwAMB4z94tyxMawLPIzCzPpan44vOYGNGU/TvQ aB3d0puoIULs9hjwsM+04N5azoQgFQy3cveoVCPZ6eJMVEMMgZBMbHQl003CW6Xa1aT7WK zuXka+urqlnXFQ2rNAotxOFF/QabXGA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733164933; a=rsa-sha256; cv=none; b=xsy7ss7gbP1MlbTFd7E6/9xLtdR8JhcaA73/W7uwxraYL2tp3fj9stfVqfR04E7BQizQo3 4assw82BfKcvf6jmj2UEUBUaWe55pEaPCw0yT5eA/NNy07uDe+nifXGhgQlOvy1IhMd/Ig zszj54cXQQfyb7hqweI814bL5SbAHHs= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=R7vR7Sza; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-724d57a9f7cso3895492b3a.3 for ; Mon, 02 Dec 2024 10:42:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733164940; x=1733769740; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=1aeU1SawrmUs4i99mcy6Gmav34YptGDoV51/MymT7EU=; b=R7vR7SzawNGkvYwvpmxta6R34AfZJuzXS0E2kGYrTyuut3h43a27/9NUsJFI4rxqgK V6EK67boApN5aJ0a5EhneFuyIwwjkRQjOIvY9alE1CfPOd+8RpHoEPxQhnnFqT7FzjHp +/QLjBljsUxmh8AGBxHtci1EgXf3K8DduBkQwgUhbrVSBIBUQQGb+U/B9yQZfNJIPGqM kVnTWAJd4UaZOYuy0V5Ig+zM/0kn8oRhz0nbJ109cL4IDzustx0/J2oOBAgm82jalI2k qbLS4r2KGo4stf8NXHtw6tfMGPBprPcZRBOO5gRBwdDwO+bKBURs2c1m/6m6/bk9lROB MRpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733164940; x=1733769740; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=1aeU1SawrmUs4i99mcy6Gmav34YptGDoV51/MymT7EU=; b=D4YHps3OKlgDdgjiwbjvqxj+Z0B/S0HZNnRUYCMYyGZ1XMS6m/u+I3rbaw9sRL1UNQ 5W2LluMzS2+9/m0FFtBLc27fwoM6SM89fdunAdA7TFlitNsJ9koQxnq/M29PP0KDpjAJ r9Rg6dKJJFzTqFE30w5afsXt79rnZ9R7L2hudhpW4W7Cm5q4mMBIpvKkPsFBvaZDoQmi 5f9ur8BK+mVKWeNgFo+Wp+46KFCyqqtUDIUq0QZySLYKQU1GF7x61O37ksi2YNti0r80 9Na5zVyP3pW7tNqrvIn8bPX2HzyG9csyP8vxSfMjqFpePUj+iB+t2X9we8Myx0APp1xY aR3w== X-Gm-Message-State: AOJu0YzTKF7ihlHDXkSWXf1CbeCK/+6dbtXZ+5Hgs0UB8DJbuuWuO2HY 9eCuoHk2kSDRmRZ3ui03l1WfyyFKS4KegNT0J54sV56uwnvkqJAWjWyXn6M6NiwpYg== X-Gm-Gg: ASbGncvx+emovRDi1edlz832EvYhMxL61BtZWSwUHEvl6bguEBSyVlk9mepOtjImlRa PpT+j+M8K20SWBEsXifQULl2XGBW1eb4t4Crln3hlQ28mqFy00ql/zvZ8GcvQLFl9b8wRNve//j xWsiVZABIJ5Seq4P0SY1Ri3czyc0+gfmoRL2jP56mMZByjcukDg7kSMwUSri/BTEA4NXeL252ke 3ZtJH1XC7uVmXge1RsdVU4WpkZgpMFvQKmvST6q40Z+CIeLTlefVJAxdubVs/ExTS9HR+o= X-Google-Smtp-Source: AGHT+IGfiyQ6y7KACpekf7bdygxRsz/ROBwam32XtlPd4Ct4vBnneUD3+Zdg4Z2+z0w6o72q8gMZxw== X-Received: by 2002:a05:6a00:1a8b:b0:724:fcff:505b with SMTP id d2e1a72fcca58-7253007218fmr32371685b3a.14.1733164940196; Mon, 02 Dec 2024 10:42:20 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.120]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417fb7d5sm8849781b3a.94.2024.12.02.10.42.16 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Dec 2024 10:42:19 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Hugh Dickins , "Huang, Ying" , Yosry Ahmed , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 3/4] mm/swap_cgroup: simplify swap cgroup definitions Date: Tue, 3 Dec 2024 02:41:53 +0800 Message-ID: <20241202184154.19321-4-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241202184154.19321-1-ryncsn@gmail.com> References: <20241202184154.19321-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 88CC6180010 X-Stat-Signature: 3x4dnthf6xhet5gqk6u5mzaajxehjb99 X-Rspam-User: X-HE-Tag: 1733164929-724962 X-HE-Meta: U2FsdGVkX19mrCw8AIeuj/N0UrFLXzfcPmXfoMr/gnaaenKEbfXLFxVycJUBflZQjTuz6cAq+ESGJT4R9UWGx4Mtd0dZc7Ynrf+ug8HkYgM730XfHdKR9DAodNh2Lxl0d/OuCBwVnWnxE5n7fwpV2XFGlG1jo4q/Y48mBw3GG7GgxOzGKM11tvIqDoYbWkUJS12eO/BAomnflq+UYelFzXzIo9TKjxTGJpTCSkYtPj5/KLv1PpU9A/YJWZPxG8EauyAOqtUJu3Wfg9LxPQgStSbg3ysK2J3t0cPAztESkQlYrU7MhnRlgEaBrw6Ut6k06ewh10V9j+QNTyxU3xnJYTB2Z23x8WBMRaEEktT50r9MncBvXfctmfoU2aRZaFJHBvguWgZctJg5VBxZRtzcp+J1k4h15zycczdGi071pICl+YgvAJx1ZaFIBPd+coJ/31MxW9TmhdAzam7p0oESHyQRsbOwvQCWEPG8VIjEvywDd+k9ywB558ep84rP+S7ji1rI0FeK9BSqQ2J2tkxczdRd2B3a7PgpFQAjT9mjqjOMClz7xYhRroApyAaueZuFQyj4erXgEGxB8pVK2bg0MX2mXh6OkT0pqlGgMjzNEZSCszqitFOVOZePo+ZWA7dmLg5bDjrPQugDKrlV6YNXbbvaiFhXw4urfjXh0IqHBDIqUB6xeuR59ZWLALhWu3vZEIXvezY0zfs5jMT/kAWib8RjernoKMVyHCDkHFJIZ+5EYkwaIMHtbOP+0TegRqnS0S/wCh4taOqSyRcs6GwW4jQoKZpDEuh5msh43Aabq3fVmdwpcTv6D1BiPKhCZji3bWKRnV+K/JWocBRqabYn3Zni61XaXsmiiEZGJ332AMzCrUjXnwDwiLyJBxWu1+2ChuECjna5+Di9FdsPvkxBvpxJw9lMBHwinH8CjayF02XH0XFKTwdqHTFHCthuSEfsLEydi4O+hXGwEQpxiew pxGs+HiZ ojUNbbxEChvBpI92tmpR5QMKdiEdBe8GaXXwt5qN9d3CrM/8Kb2L9kpCAs2pl4YM5CEn/MFaAOW5btoDwvpq+iNzTME43oT23j1Rsvg4hPS/APCs43KaIbCrXtk1Vf2co0T27czWdgF36EPav7p2QyHstYEfr8+Qq96CJwKkB7mD+5T24/bW0EbPX8b1ua6GRqChWqLyLw6wrPcsr1UHr84ec4IJYVKDfoSiUyfr9h7bYCDc/l4uiN9fEmV+Dbf8oNB+j3mYLhxbjeqMXaruTK5QBjoQyE2OOcRI5DSZ3Ns/WAO8GA/FIGxJ3Dg6ppBvHrbTJ7fnxpRCw6XsQnnIhAqXcC9rMmXHwWTL+IwI95TPOcRiXBIL1qcdBLo7b+LyHYHA1xRP3Kn1e1E0Pdw4af3yOxEt1ZvC4niIAxMH1v4eMsysMjgRmeNV7usMUZuVu5omPc3jcybo42C00Ak3R9SxkpnzdfecgT4AZLb972Cs5Dj4fYQmph2aLo/C7LW7mWEm+jtRRAsfgcQLXbtilQDVQ3Ff/Skc0T33IDDPyyJ+w7d3p7tJTz//wBMmEx5Bs4Z4j X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Remove the intermediate struct swap_cgroup, it just a unsigned short wrapper, simplify the code. Also zero the map on initialization to prevent unexpected behaviour as swap cgroup helpers are suppose to return 0 on error. Signed-off-by: Kairui Song --- mm/swap_cgroup.c | 45 +++++++++++++++++++-------------------------- 1 file changed, 19 insertions(+), 26 deletions(-) diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c index 1770b076f6b7..a76afdc3666a 100644 --- a/mm/swap_cgroup.c +++ b/mm/swap_cgroup.c @@ -12,14 +12,12 @@ struct swap_cgroup { }; struct swap_cgroup_ctrl { - struct swap_cgroup *map; + unsigned short *map; spinlock_t lock; }; static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAPFILES]; -#define SC_PER_PAGE (PAGE_SIZE/sizeof(struct swap_cgroup)) - /* * SwapCgroup implements "lookup" and "exchange" operations. * In typical usage, this swap_cgroup is accessed via memcg's charge/uncharge @@ -33,18 +31,6 @@ static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAPFILES]; * * TODO: we can push these buffers out to HIGHMEM. */ -static struct swap_cgroup *lookup_swap_cgroup(swp_entry_t ent, - struct swap_cgroup_ctrl **ctrlp) -{ - pgoff_t offset = swp_offset(ent); - struct swap_cgroup_ctrl *ctrl; - - ctrl = &swap_cgroup_ctrl[swp_type(ent)]; - if (ctrlp) - *ctrlp = ctrl; - return &ctrl->map[offset]; -} - /** * swap_cgroup_record - record mem_cgroup for a set of swap entries * @ent: the first swap entry to be recorded into @@ -58,20 +44,21 @@ unsigned short swap_cgroup_record(swp_entry_t ent, unsigned short id, unsigned int nr_ents) { struct swap_cgroup_ctrl *ctrl; - struct swap_cgroup *sc; + unsigned short *map; unsigned short old; unsigned long flags; pgoff_t offset = swp_offset(ent); pgoff_t end = offset + nr_ents; - sc = lookup_swap_cgroup(ent, &ctrl); + ctrl = &swap_cgroup_ctrl[swp_type(ent)]; + map = ctrl->map; spin_lock_irqsave(&ctrl->lock, flags); - old = sc->id; - for (; offset < end; offset++, sc++) { - VM_BUG_ON(sc->id != old); - sc->id = id; - } + old = map[offset]; + do { + VM_BUG_ON(map[offset] != old); + map[offset] = id; + } while (++offset != end); spin_unlock_irqrestore(&ctrl->lock, flags); return old; @@ -85,20 +72,26 @@ unsigned short swap_cgroup_record(swp_entry_t ent, unsigned short id, */ unsigned short lookup_swap_cgroup_id(swp_entry_t ent) { + struct swap_cgroup_ctrl *ctrl; + if (mem_cgroup_disabled()) return 0; - return lookup_swap_cgroup(ent, NULL)->id; + + ctrl = &swap_cgroup_ctrl[swp_type(ent)]; + pgoff_t offset = swp_offset(ent); + + return READ_ONCE(ctrl->map[offset]); } int swap_cgroup_swapon(int type, unsigned long max_pages) { - struct swap_cgroup *map; + void *map; struct swap_cgroup_ctrl *ctrl; if (mem_cgroup_disabled()) return 0; - map = vcalloc(max_pages, sizeof(struct swap_cgroup)); + map = vzalloc(max_pages * sizeof(unsigned short)); if (!map) goto nomem; @@ -117,7 +110,7 @@ int swap_cgroup_swapon(int type, unsigned long max_pages) void swap_cgroup_swapoff(int type) { - struct swap_cgroup *map; + void *map; struct swap_cgroup_ctrl *ctrl; if (mem_cgroup_disabled()) From patchwork Mon Dec 2 18:41:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13891177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B53BE69E81 for ; Mon, 2 Dec 2024 18:42:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AAE5F6B0099; Mon, 2 Dec 2024 13:42:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A5E216B009A; Mon, 2 Dec 2024 13:42:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D8316B009B; Mon, 2 Dec 2024 13:42:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6E1F46B0099 for ; Mon, 2 Dec 2024 13:42:28 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2706716026C for ; Mon, 2 Dec 2024 18:42:28 +0000 (UTC) X-FDA: 82850889282.05.86B79D7 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf12.hostedemail.com (Postfix) with ESMTP id DA0ED40010 for ; Mon, 2 Dec 2024 18:42:20 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=NWo4LASc; spf=pass (imf12.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733164937; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=n486kN9w2/p4kQRIxzmOz9BLiT1ewnt6ZG6JQNAAsl4=; b=T6Fvv5IFrJ4xL8Znapbr0p6eoCLfl5x4+KkmEo3bvgPWUjMyzrruk0O6vZBYA6sq/7L546 P2Kv9CQYnXDgeFkUDFbXUy0YphMoJovBw4E09cUFketJIxwoMVAPvzlmpywwFqA8x1rbLa CjreHdd4LEwC/LeH+JzH0BOLMvoBlj8= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=NWo4LASc; spf=pass (imf12.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733164937; a=rsa-sha256; cv=none; b=8AmTGj/WxED6AxbVildiN6bciC4knDM7hWz1cfw+8ZRUQhgncNF4E/eOCq3/RHKKQpKwyE 3mE6qytV4b1OGIEetY5ainhzrE1wjBOE6hXHeZk+SpYx1Ftt+oqXH9xz98vGM0c0mNMOzb TFnaOpEjs7VTcSCxkZ7r8e01MNzhdQo= Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-724f41d550cso2744172b3a.2 for ; Mon, 02 Dec 2024 10:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733164944; x=1733769744; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=n486kN9w2/p4kQRIxzmOz9BLiT1ewnt6ZG6JQNAAsl4=; b=NWo4LAScWyOVlAwR0vHxtv/2Ea/mR0hTZnmYYhBS8gbYaSuxZ33oCNMXhfY26K3Jzr DBtaXZw7Ma21SGHj3nY0QIhTVEiUex1h55+M6SjQLAQFlVEukls+ULUCaybJbEwsQgBr VuL0889cphZzOXdnXp9yWAouQig+Pjf2zMB01levzWaH45cNjFw3bb8oUVMjXi2l2kDK 3ssYB8thc5pyVN/TNKiYfhB2cjTEuF6HqzeB0QwzbiFmcdlxOioGsWdXENB+wcsFrT9/ YYSjKYQTFilWZDYjTAtgS2dq3DYLofmgF647G2DTWbfnH7l29QxCcfBXb18NCa0mLsil 2Nyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733164944; x=1733769744; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=n486kN9w2/p4kQRIxzmOz9BLiT1ewnt6ZG6JQNAAsl4=; b=hijnroH47FA6gyAnrzglKvt0OzdtiARO+KE/nSmIdiEGKZAkShSmN7zprSwlok+3O0 h1IEojqi5JFMtXudBkiu9wHAdrkBqi9C1efO7bbq2492nxInnX8kDsd3GRMaPjHoxPgU 4PEXdzhst/bbVaERf1106oAuIx2EXGH8SIS8Le9Vi9IPZSQ9XMHIAW16PdCgn2Fn8eLJ O3fXB+okKWA3yhDuEdCrDZkGD55/yFQHfLwpi3otyZ1spdksYDHBlJ3YjpMHE7NTgX7f z7QsPwznlspLiakiXvRcgVZowSLRLtNpPDVrS/YfCbh5HAz9HgcOY8+sTISc5qytr3Yf uFeA== X-Gm-Message-State: AOJu0Yy3KOAd5n8NDKkuJjCFxI3aBWm847vnkYEgORoe16kUTOacQ7LV eor2KMmAhNYtERDDgFjUzC+4v49SsS5zhXqTmFnmXDJpjCvZnmnyn3QDOI1Jluv1Fw== X-Gm-Gg: ASbGnctDWO/q0fiGCl9VPwWUOabEMZbYok0u/TuD8zGL4Nw6AO8aYnscQLC+yqUjOtJ aXrbUe7NRKWJN1j1jF0rr7X1IZqOs2LvJtw5C0tIkL6TnMHadcURdnkz9jQvzgkNzQsBu37jRcn rwJbMTVGGY2E3yAoiCaUwrDo02avb5QEopiD7f1BrN1zFMxGbv3okDcUkrrhgJRCfE76dFEaoeY 1FRRwVmaYuhVX8L7OH2/r6WJGaSg70drZEqSiOyhcCAUys6It9n3KzMRqUpUB9R59Tj0zY= X-Google-Smtp-Source: AGHT+IE6F7f6lJrUVeUD4BRVPb3PuBX80oqzE3+WCYBtL1H232mPA2ctT8z6mSlYDWpdAGJGBit8ww== X-Received: by 2002:a05:6a00:4f81:b0:71d:eb7d:20d5 with SMTP id d2e1a72fcca58-72530045b6emr37194340b3a.8.1733164944304; Mon, 02 Dec 2024 10:42:24 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.120]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417fb7d5sm8849781b3a.94.2024.12.02.10.42.20 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Dec 2024 10:42:23 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Hugh Dickins , "Huang, Ying" , Yosry Ahmed , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 4/4] mm, swap_cgroup: remove global swap cgroup lock Date: Tue, 3 Dec 2024 02:41:54 +0800 Message-ID: <20241202184154.19321-5-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241202184154.19321-1-ryncsn@gmail.com> References: <20241202184154.19321-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: DA0ED40010 X-Rspamd-Server: rspam12 X-Stat-Signature: 3i7j8b6cuxuymaaj1jywockz5kn1kbsx X-Rspam-User: X-HE-Tag: 1733164940-849934 X-HE-Meta: U2FsdGVkX18SZKBkPFkboQwMA+ILOOluIRRoerMLZauRqc16IuYKAKaNz84WN/ZrEieGMQs4MTxaJj6/67vNkuO8WDARpjwzd4na7XH+fBtuB+OeoL5FA9IIp30aavEbpmhOUmbx7csJlJ57caAKACPxUP5PwIcZdVQ0kp1NkaybLMw7s5RXvrhADCTNR/F4T7/0DOt5U/Y7sWCqLNoZoF+gle160RZZP0WrxtxDA3UeHFUgEB7paQG14x1AXT1ogew1k8SrmNYlV4TscgVPTNFY7f+1r/ZsPHphhMWOD0JKOaSiqs/SHshhuPF7xWDpCIBK2B5xONfbkbpcaSI2/NV7YidxBbaG0nRYqmJNu325y1gIU8c5g0ML5Y5HJsCqxerzn27Pi7IaQ1eOF1OLRjQ1FBqgakB1+snnkW+sDkdJoSDrFfxEC+SZE1W2fy9k4Hcmgn+IBppIfGIi5ZQfDPrNv6xB+zbMsTNI1stkZpBetXy77RcGSgyFNXayzplLPmvBIcuQnHQdWa/X/PeJgCNV4ggsLqJlo7y8JCjCvRtmcues/9T5LFx9aokhBFmKccbbMugSmubQh6NePj77jDzUzsN28Nh/CABrcVzEDEdT+tHMxEf+D8YfcAEys7bPxkToxOXSYkWl9GiOtmGTkKXoXbozrC4noSw6PeH2YYh1lq5eyTPab/qzK1WShLyrc8PPJzKlm2yWt9A8S9CGfHLDlaDPG/5lfMWOLoZSAgIi3fiEP5jslDVEowCcdM8PCFPSCo7taM11sdyzCK3Wr9pLrH8Y5lSYJYgfsSV7337MvkRhsT4ygi5hJzakYz+wJEaG8/4bjVbJsX+CIUdeYY8Rk3YSpLu8gwakNdHstZvMa3YorCt69cchC9eMwXRtDHfLnzW2Yp0M7IiEckDMLpB9RsSat9xJACqfadciNXtY/zuuNtt2NVCW4/j2REQ6VCsTDmAxozRaVTwIEdA lEnG3GT+ r/hIyiN/w9CFG5HunNvOXF5Mhl8qzk8mOzyupCTBQh8jW91eTftfCvIPjfk8WgZcHKC9Nh6zu3k6B5ASuBdRUxSjLdDH9RR9Ihw16Fy+E1c1OD8f779D4cgMUBrhhZ5j7+Pu+2EOmdBICmF+fEHYniEfuYXbSlH9M6gkVDnfL6vm3mdxwxdTlktwVHX59yG2P3ceGeYAF+jAjQBNQAamVxXzP5jBST0SU+3Tw2gNbKAI0XJArRDEjy182zh4SUimKeJ2xVfGYkvMgZu8NXwMtLR4FofFF3k2rYO0WpK6uGB7DnVZjPUw2YD/eunlNQDz9Ldiew1cVbszgPq7TOpa1Il3Zc8/1k0t+Ds7fIpaRyxqo/3NNaKO8ibZxOZtJB/476VnGX9EnGIEeO8IE0CJk0NdQz5RzsPrRj7+gGvxOkXOr53H+nYd1y+YDcfpBmJjdx2MEZqpRNJf2KthkRIUUiR3srgJz2BTEydu3LnQePDmV1Ks5tvrR4MzbyKouO4dcPSE0e6IViVYC3ErLK+IrKtPMm47fe2YUMeL/X7OMslGspvOpsjhHXCgFgs70KErH5ed7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song commit e9e58a4ec3b1 ("memcg: avoid use cmpxchg in swap cgroup maintainance") replaced the cmpxchg/xchg with a global irq spinlock because some archs doesn't support 2 bytes cmpxchg/xchg. Clearly this won't scale well. And as commented in swap_cgroup.c, this lock is not needed for map synchronization. Emulation of 2 bytes cmpxchg/xchg with atomic isn't hard, so implement it to get rid of this lock. Testing using 64G brd and build with build kernel with make -j96 in 1.5G memory cgroup using 4k folios showed below improvement (10 test run): Before this series: Sys time: 10730.08 (stdev 49.030728) Real time: 171.03 (stdev 0.850355) After this commit: Sys time: 9612.24 (stdev 66.310789), -10.42% Real time: 159.78 (stdev 0.577193), -6.57% With 64k folios and 2G memcg: Before this series: Sys time: 7626.77 (stdev 43.545517) Real time: 136.22 (stdev 1.265544) After this commit: Sys time: 6936.03 (stdev 39.996280), -9.06% Real time: 129.65 (stdev 0.880039), -4.82% Sequential swapout of 8G 4k zero folios (24 test run): Before this series: 5461409.12 us (stdev 183957.827084) After this commit: 5420447.26 us (stdev 196419.240317) Sequential swapin of 8G 4k zero folios (24 test run): Before this series: 19736958.916667 us (stdev 189027.246676) After this commit: 19662182.629630 us (stdev 172717.640614) Performance is better or at least not worse for all tests above. Signed-off-by: Kairui Song --- mm/swap_cgroup.c | 56 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 41 insertions(+), 15 deletions(-) diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c index a76afdc3666a..028f5e6be3f0 100644 --- a/mm/swap_cgroup.c +++ b/mm/swap_cgroup.c @@ -5,6 +5,15 @@ #include /* depends on mm.h include */ +#define ID_PER_UNIT (sizeof(atomic_t) / sizeof(unsigned short)) +struct swap_cgroup_unit { + union { + int raw; + atomic_t val; + unsigned short __id[ID_PER_UNIT]; + }; +}; + static DEFINE_MUTEX(swap_cgroup_mutex); struct swap_cgroup { @@ -12,8 +21,10 @@ struct swap_cgroup { }; struct swap_cgroup_ctrl { - unsigned short *map; - spinlock_t lock; + union { + struct swap_cgroup_unit *units; + unsigned short *map; + }; }; static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAPFILES]; @@ -31,6 +42,24 @@ static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAPFILES]; * * TODO: we can push these buffers out to HIGHMEM. */ +static unsigned short __swap_cgroup_xchg(void *map, + pgoff_t offset, + unsigned int new_id) +{ + unsigned int old_id; + struct swap_cgroup_unit *units = map; + struct swap_cgroup_unit *unit = &units[offset / ID_PER_UNIT]; + struct swap_cgroup_unit new, old = { .raw = atomic_read(&unit->val) }; + + do { + new.raw = old.raw; + old_id = old.__id[offset % ID_PER_UNIT]; + new.__id[offset % ID_PER_UNIT] = new_id; + } while (!atomic_try_cmpxchg(&unit->val, &old.raw, new.raw)); + + return old_id; +} + /** * swap_cgroup_record - record mem_cgroup for a set of swap entries * @ent: the first swap entry to be recorded into @@ -44,22 +73,19 @@ unsigned short swap_cgroup_record(swp_entry_t ent, unsigned short id, unsigned int nr_ents) { struct swap_cgroup_ctrl *ctrl; - unsigned short *map; - unsigned short old; - unsigned long flags; pgoff_t offset = swp_offset(ent); pgoff_t end = offset + nr_ents; + unsigned short old, iter; + unsigned short *map; ctrl = &swap_cgroup_ctrl[swp_type(ent)]; map = ctrl->map; - spin_lock_irqsave(&ctrl->lock, flags); - old = map[offset]; + old = READ_ONCE(map[offset]); do { - VM_BUG_ON(map[offset] != old); - map[offset] = id; + iter = __swap_cgroup_xchg(map, offset, id); + VM_BUG_ON(iter != old); } while (++offset != end); - spin_unlock_irqrestore(&ctrl->lock, flags); return old; } @@ -85,20 +111,20 @@ unsigned short lookup_swap_cgroup_id(swp_entry_t ent) int swap_cgroup_swapon(int type, unsigned long max_pages) { - void *map; + struct swap_cgroup_unit *units; struct swap_cgroup_ctrl *ctrl; if (mem_cgroup_disabled()) return 0; - map = vzalloc(max_pages * sizeof(unsigned short)); - if (!map) + units = vzalloc(DIV_ROUND_UP(max_pages, ID_PER_UNIT) * + sizeof(struct swap_cgroup_unit)); + if (!units) goto nomem; ctrl = &swap_cgroup_ctrl[type]; mutex_lock(&swap_cgroup_mutex); - ctrl->map = map; - spin_lock_init(&ctrl->lock); + ctrl->units = units; mutex_unlock(&swap_cgroup_mutex); return 0;