From patchwork Thu Feb 27 21:55:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13995242 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 598F5C197BF for ; Thu, 27 Feb 2025 21:56:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6909E280003; Thu, 27 Feb 2025 16:56:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 641B86B0092; Thu, 27 Feb 2025 16:56:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 475D4280003; Thu, 27 Feb 2025 16:56:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0A0A36B008C for ; Thu, 27 Feb 2025 16:56:01 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 99A2F4B7EC for ; Thu, 27 Feb 2025 21:56:01 +0000 (UTC) X-FDA: 83167082922.28.1795B7F Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf07.hostedemail.com (Postfix) with ESMTP id 99A6440004 for ; Thu, 27 Feb 2025 21:55:59 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OQjmPcK8; spf=pass (imf07.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740693359; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Rzxzwt2mUUhnKAxYxNpGGBXKdjzbS8+sU15pD45zOWk=; b=7ByAlXpC/ke28LzFeI6OIi83xR91MKLXh5UlZ5OfEMqkSZRfop9npbypiFGZRMSIRhiU+U TpTLi0OnzMTFDA1F1Nifluv0rS++VoZ9w3N/Gli9WAaKRmDSVM3s6GMNrEurgW9spUPRtQ M93nK22vovpLTlMLtaI0WOFt1nAUpfk= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OQjmPcK8; spf=pass (imf07.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740693359; a=rsa-sha256; cv=none; b=bZPDEQPPJ8jE+KJrucyHLUm3S35ELVJvmaz9yG2hdRdAGe6csuHtxZ4an+dBTKZ3PSB8j6 oFCFwX53xjsP2t+yJijjRlGQ6G6dRBQnfIIvpj69elj+fhsJtimvVAB+Wn2IzI0hM+7po+ glojBqehqIA9Z3gIdBTZf2MBhtr+0S8= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2232b12cd36so21595585ad.0 for ; Thu, 27 Feb 2025 13:55:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740693358; x=1741298158; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Rzxzwt2mUUhnKAxYxNpGGBXKdjzbS8+sU15pD45zOWk=; b=OQjmPcK8sJ0Oz1OWRrWHAxl4kcFF2Znny1wAS9A4s25plNjJO7BfgP4nOa7mQ9kUEc wPLCPRGmzG2DpqH17cx1BaNVaro9CyJRRjNSmvcXSLOTVCBstQEiQSzfupjb0n2iy11H 7tcepWd7tSrCx9Lk52Vjhh1Bf7pytERtJ3I0TR/RdqvU8siHiPenVwHs9T40UmgnD9Ek 55ZHP9bqZ933MCR7hch0x0MDMCkgNBVCZXotRpNDpQQFxFMTShrAJli3zf2u/7HQaYEj 93GAomUSZAtU97FTlpiOcxILtElK2+ciBS0/m8/r9WdQSbAekw+W2NfivBziUJi4sudG olSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740693358; x=1741298158; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Rzxzwt2mUUhnKAxYxNpGGBXKdjzbS8+sU15pD45zOWk=; b=pusXd8d8LnK5XYBOGFAPiXl7MruapidKIEa6K/2iIOzI6xdPu+qwwk9iN+VVES1tM6 qz2BU3fI6WwsiK+uruCOI/xDJ4QdtwvVIaDubQ5zp2ycG+45c7IfEADC0VdSG1kmshfp D9K35BDMj3NvYV3e/hXXeWadEU76Tcfr4fBSr1n/4aYoprk4elQp2bM67DVLHuvzMJtd kE1LKxu1RhH0qEZIxMJOMrgrwT1y1HwBwdR2KTKLo7vjwrCRnT28+bKrYBtPtr/qMTlE Hu8bkzHmU077w7W3mMr9vIkACrvcoZQB2N3H5WJzDMzTqmhrV4/atWBpR8f5RasLjTw0 ZY3A== X-Gm-Message-State: AOJu0YyGR9xcoLxVT05Y4wDCkkmzX6p9VrawSOlOaND2BSzkwK5Ujep0 5djyBiNiAJdXVF7lFM8mgQDgEHewW9zXjI9H2BkWbqMw42xAlg+X X-Gm-Gg: ASbGncsaA/GexG8itXzhHEEuk3GLRvbdDLNP0wGOU6hvE7RPnzkoYxu3WAn+rEI57aN YDcRZEY/9ZdiQdoSrRZJ3tTePVgYDWy9/AzIR9Glrao08pwS1y52bQynVsBMlhB6upQuwRgnb7u jm2/zvyx+BqozBoA0Lv0o1M5cyBX56rrHivL3LrYqrYVdjwF0JcaIm79cYg6sxTieR3i9TaoRXq svu1BxPJhGJ+CyGJqnAr6Do4Z1wFyff1+5LgxW7teGtH6bQsmhVeqsH4h2MgLs2Avzqddptwt/U xGl0kZ52Nn9oemx9kb2ZHwSUnZ0hZfxVFZK64YMj3ndypBebLuFSRg== X-Google-Smtp-Source: AGHT+IGmCjZRC3Jlpg0kd+gq47vBuCLSQ0CSc+pP3NgcWQ95MgHTry9WY8zWNTMUBXSWfW65jZLQfg== X-Received: by 2002:a05:6a00:b8e:b0:732:4c47:f807 with SMTP id d2e1a72fcca58-734ac4499d3mr1561671b3a.21.1740693358442; Thu, 27 Feb 2025 13:55:58 -0800 (PST) Received: from jpkobryn-fedora-PF5CFKNC.thefacebook.com ([2620:10d:c090:500::4:4d60]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-734a003eb65sm2301321b3a.149.2025.02.27.13.55.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2025 13:55:57 -0800 (PST) From: inwardvessel To: tj@kernel.org, shakeel.butt@linux.dev, yosryahmed@google.com, mhocko@kernel.org, hannes@cmpxchg.org, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 2/4 v2] cgroup: rstat lock indirection Date: Thu, 27 Feb 2025 13:55:41 -0800 Message-ID: <20250227215543.49928-3-inwardvessel@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250227215543.49928-1-inwardvessel@gmail.com> References: <20250227215543.49928-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 99A6440004 X-Stat-Signature: xdc8eo8n9o1j1e3wrg91n7if1srnohww X-HE-Tag: 1740693359-166728 X-HE-Meta: U2FsdGVkX18UTNBx4qzdmik/WqunnT3jUzBnKls+5LcXnhCWdOQz/3kJc0a1T4pHYF/lFAekEQwr29c0UvDJbpDX5ym5gRvwPirEy3wPFSi51t5qcnkLnZWjQKzzs30NnmDwFe+XrJdG3UKLFFiyQEyiD3K6nX9nDMoMTMhOlMZ2aSvKL03v+QCZmaKpUJ2rAoSQvBPIz0uZBOXjhsZ/NY8aMB6ZaJk6FTrCPMzGlmF//WQGOkIJwg7jeO49IcOYWxCyJuET4uAuUPaRIqmrkwsksnBFZMtrELa3GZN3LYiqbiEB6IIIym+OTfdRJaPLEYMkKK7arNTaCAIhe5MZBNdupwSWg9GqlxTC0OelEMCsNhbw0uOUUAvdCcGFhjf+en0X8pFsbqITG6993Q+gzar9L1TqFvE1i7dhZlNokyvP5UWnIHYZQ7dZudkBkT3nphRB6fmB4/0AgKE+FKw2w2Id2EvI/HFdG2r1k1jFEci80fcrw3laAhNeb27zuGgZFRtwd95W6g6dFhFlyx4LReg3r4apE3MdNkCfy91xR1r1AUieNH7dfHo3yKG3Gn9r9B81PtQLjfUO87lwGyMGhLOssmy64dIF7GXihc67P395/buahFwsVnK/oaAm/LF3BQwZoN3WC4ud6xaLpP9Ab22gXLD/p0qQMaZBIzYodDkJiLUgBu7Wdx1SgcmsECDagazJDfuTcqpBJmYNEX/7UpdjTjLOMNqE56Bs0THsOgooCtW336yNFo61juimKrwHtRVYNTrQFi0DvzU8oJ65PnxFayIqAFJSmUC87wNDmRT2a8Q/Rnl7vlfu0iAMO/GJNc5QZNvSy69DZDCQzorSPzr6JmG90HwIFdru7mxXAFlf8mRgBmRKTA/xs68u6FA8YSpDOYC2KQIEhK8hTo0oszzyEvhKTKMqNQSuj//Lz8B5Y0mPaQfdepLDnyAYbsZE33BLxFTuGFVoWqAwBEs 5fnOt+g5 UoZ7HqijvefxMabqWHI72Z3hJ2Byr1Lr/JPJeCwK7dGEitQeDoUdDEVA28NI610X+FZVH11RTewlvkP5aIpdiL3/46LhBjo8848+kbAXdj4ojhHGexlH+v7xAGAreu6EB4acRqXRf3EHRuN92g3rZZSeHx1BcrQ9ygJ+BzCUrE6XovuA0ATBMMwErrzBil/esPv2JxOWrIGLs+rvBhN6VymkFQLuDLq9Ez0R+JdRcz0p1YZB/wyl+1/NmavX+23I8GsFqn3IF+YlVV1T4FzxtwTuI+62SvdLuSXbEKwY1k/xkjHaiYn5L60lOqsGeafyGJO5cjTOVoqtc2NnAbcHmRk9pTpsecBI8fAaJMDJrYkr+y5BCvminaYIDfarB+n4Th5uy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000041, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: JP Kobryn Instead of accessing the target lock directly via global var, access it indirectly in the form of a new parameter. Also change the ordering of the parameters to be consistent with the related per-cpu locking function _cgroup_rstat_cpu_lock(). Reviewed-by: Shakeel Butt Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 41 ++++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 9976f9acd62b..88908ef9212d 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -277,7 +277,7 @@ __weak noinline void bpf_rstat_flush(struct cgroup *cgrp, __bpf_hook_end(); /* - * Helper functions for locking cgroup_rstat_lock. + * Helper functions for locking. * * This makes it easier to diagnose locking issues and contention in * production environments. The parameter @cpu_in_loop indicate lock @@ -285,29 +285,32 @@ __bpf_hook_end(); * value -1 is used when obtaining the main lock else this is the CPU * number processed last. */ -static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop) - __acquires(&cgroup_rstat_lock) +static inline void __cgroup_rstat_lock(spinlock_t *lock, + struct cgroup *cgrp, int cpu_in_loop) + __acquires(lock) { bool contended; - contended = !spin_trylock_irq(&cgroup_rstat_lock); + contended = !spin_trylock_irq(lock); if (contended) { trace_cgroup_rstat_lock_contended(cgrp, cpu_in_loop, contended); - spin_lock_irq(&cgroup_rstat_lock); + spin_lock_irq(lock); } trace_cgroup_rstat_locked(cgrp, cpu_in_loop, contended); } -static inline void __cgroup_rstat_unlock(struct cgroup *cgrp, int cpu_in_loop) - __releases(&cgroup_rstat_lock) +static inline void __cgroup_rstat_unlock(spinlock_t *lock, + struct cgroup *cgrp, int cpu_in_loop) + __releases(lock) { trace_cgroup_rstat_unlock(cgrp, cpu_in_loop, false); - spin_unlock_irq(&cgroup_rstat_lock); + spin_unlock_irq(lock); } /* see cgroup_rstat_flush() */ -static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) - __releases(&cgroup_rstat_lock) __acquires(&cgroup_rstat_lock) +static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css, + spinlock_t *lock) + __releases(lock) __acquires(lock) { struct cgroup *cgrp = css->cgroup; int cpu; @@ -328,11 +331,11 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) } /* play nice and yield if necessary */ - if (need_resched() || spin_needbreak(&cgroup_rstat_lock)) { - __cgroup_rstat_unlock(cgrp, cpu); + if (need_resched() || spin_needbreak(lock)) { + __cgroup_rstat_unlock(lock, cgrp, cpu); if (!cond_resched()) cpu_relax(); - __cgroup_rstat_lock(cgrp, cpu); + __cgroup_rstat_lock(lock, cgrp, cpu); } } } @@ -356,9 +359,9 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) might_sleep(); - __cgroup_rstat_lock(cgrp, -1); - cgroup_rstat_flush_locked(css); - __cgroup_rstat_unlock(cgrp, -1); + __cgroup_rstat_lock(&cgroup_rstat_lock, cgrp, -1); + cgroup_rstat_flush_locked(css, &cgroup_rstat_lock); + __cgroup_rstat_unlock(&cgroup_rstat_lock, cgrp, -1); } /** @@ -375,8 +378,8 @@ void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) struct cgroup *cgrp = css->cgroup; might_sleep(); - __cgroup_rstat_lock(cgrp, -1); - cgroup_rstat_flush_locked(css); + __cgroup_rstat_lock(&cgroup_rstat_lock, cgrp, -1); + cgroup_rstat_flush_locked(css, &cgroup_rstat_lock); } /** @@ -386,7 +389,7 @@ void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) { struct cgroup *cgrp = css->cgroup; - __cgroup_rstat_unlock(cgrp, -1); + __cgroup_rstat_unlock(&cgroup_rstat_lock, cgrp, -1); } int cgroup_rstat_init(struct cgroup_subsys_state *css)