From patchwork Thu Jun 27 21:18:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesper Dangaard Brouer X-Patchwork-Id: 13715054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2630BC2BD09 for ; Thu, 27 Jun 2024 21:18:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93EF66B0096; Thu, 27 Jun 2024 17:18:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8EECD6B0098; Thu, 27 Jun 2024 17:18:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B66D6B0099; Thu, 27 Jun 2024 17:18:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5E3E16B0096 for ; Thu, 27 Jun 2024 17:18:45 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D333AA1509 for ; Thu, 27 Jun 2024 21:18:44 +0000 (UTC) X-FDA: 82277932968.20.B0DFB5B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf16.hostedemail.com (Postfix) with ESMTP id 451E318001A for ; Thu, 27 Jun 2024 21:18:43 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=pkyRYQwe; spf=pass (imf16.hostedemail.com: domain of hawk@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=hawk@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719523115; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=zz3Zc6teVxuyhjcUKZVJM4+/QQ/o/g+ZirA+qV+GwsI=; b=gmTHBee+ZH9fyXGBOkqUCzgJtUMhdYEdIiL2J99fUqNavzsES4jUxLtfvyVo2pnEeu1TKM k/FA7s4va36pO/G8QpHIRdedBjErIpsc6qsQ/XmIJ96CFhBo7Qnq1mAUMrwMY1xODD6Eaa Xxbz1Z7pL0JSJK0MJkFKDPsu5ITGllc= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=pkyRYQwe; spf=pass (imf16.hostedemail.com: domain of hawk@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=hawk@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719523115; a=rsa-sha256; cv=none; b=B7NamJnTbp9uBQDXoYZWp16xyULnf4CLawHuquM4LITXPGllKpljXZKd8oIrMmjOPR2MgF H67v61Pux1btZvM0bnj+LzEw+4OmEtEPH4eQIvAHynjjFZiGFqQfCV9MXmZ9quA+T6upqG o5qyWU0lTcF1qMaXPHZfEiqrROw6agw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 364F761EE7; Thu, 27 Jun 2024 21:18:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5C82EC2BBFC; Thu, 27 Jun 2024 21:18:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719523121; bh=REm3K5MgPmRQpONdIW7zODAdzetyL88KjbhiskxYUbE=; h=Subject:From:To:Cc:Date:From; b=pkyRYQweScIoKsig8Cx8MbjTacXVSRPGcIEGoeD3unqvHPRg/t60L1fhyf9i23RUa 728VwKM2V4BVQaeT4grDh9MuO+z/bkex68PhC7wocdyfeUIitbBr/i0trodAKM7XBb i021/EsBdOGXLGlO6elkx8EbmOg3951IHGWPSEMplKmfAEdqSLt/YRhfoEf5jdFIim NcmviQ7QCsRyLW78MKkfGZoXPRZrAgUFa2mOLR5lVeQFspxY+g7kLDw5W9O7cPDyNr 1ly2VzW5LS8eFEMjlgKUV4N7wmwPAouFuXEZIz7riBNAaHzkbinkRFhXw8MQU347x2 f/OegnvAdIVLg== Subject: [PATCH V4 1/2] cgroup/rstat: Helper functions for locking expose trylock From: Jesper Dangaard Brouer To: tj@kernel.org, cgroups@vger.kernel.org, yosryahmed@google.com, shakeel.butt@linux.dev Cc: Jesper Dangaard Brouer , hannes@cmpxchg.org, lizefan.x@bytedance.com, longman@redhat.com, kernel-team@cloudflare.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Thu, 27 Jun 2024 23:18:37 +0200 Message-ID: <171952310959.1810550.17003659816794335660.stgit@firesoul> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 451E318001A X-Stat-Signature: in9n5h5payhc3fefg7wks9xh4rrq7rs7 X-HE-Tag: 1719523123-397096 X-HE-Meta: U2FsdGVkX19BcVmPTXGNWoy4B3vXb9QwAIIl1Lz96auR0J5OM8o3cUjo4ZGCiXYCv7btooyuD1Avd+tqUEW0JiirNNYk+QLEgyk0MM+z+ONxV1WysT3xfJaAEJ0BHZzSL1MLU+kI7rJ6HdD+09KIMZ5K6qKCIyi2c3TPP6B2ZRjAGRnYFX/rpyXTro4S2FeLj7NTeAN78ZTqVcyP35VMcfEXzgr+Epi7eXtL0cX4wHCu+m8H3t1+A690HhZvFd22fdewHV3BVuCMuOdbqbJM3yBlrEbL0y3zs/lXfQc/wCGTgrvx234wI854QHucs6v9cwlxcC2p983OOYapIkFbKPBLHkWoKcpFMTFK9iwX0MmgCowYq5MSAndc2S/VYONdmf3nlVLN+RYOpJ780VEzc2e8wbRwdSr/LiehfNWuBCz1enL9jWcKYTBSZz4Wgck+FG/LsIDll7ZAiIqEJUsEZUBphP0Uek9bHJlbVkxqp0ssM490IXifeTURUEyHCDIKeSbNZHYpa3PTqi6KEcaVkBK/kxAXVD1lE1EATj76gToTKNbe0nzmo9OxeIOrMKy2DDpz4FSsxRUX4vA2eR5KPsK2LO8Q7GPhEj7cuBgNRMYskyU8iJruXv/08qAAO93pZXyO5W3uehj+XuQ0JF4MMfja0JZn9zj+P7BrQlsV/w3GVwF7zHRkZLp3arkeNjGTi8Q3fJzq0mXEPAKSZ2GJTeRaqNGa4xvtCKSGoxp9tfsJlvG/hKmuLjVbmtANTjSP0e2vyCe9XfilO+/JIm9wJydDnSDut1R7Smm6n6juERAVgT9A01No/KxCW0tj/LPA1cnIq+dgnFRzDa59iP2cnJEFGB33Y+mz9J7ebwxvbJ0YwRuKrDGRydEvhb0TlLmllhpAr4RCHJYlTA6uEk750wILpWc1O5ZO/CnJlwjn426II8G3N5Yu3La+D2Sagyy5eJvn/iBYqGOihAxY/L6 2u6bXtlw 36hqJyk+dZmNV9CqVogoGtHlWRGQyCxk3Gt90TRv0kE93rjYNp8rQyS8b2wgunf7uQNIBps0vYw+6v+8JVsDU2XtUO1l5ShD3CipqX2VTlVJPMZ1GwfOFiPrTidP907fcK/rGyfoAdGgb8X3CMlrrwWs9XFqBrGwJ1aZa8JnCATfMyGig4y2nMYiFYo+cmOwjD0fp3sY86ffZDC2PUziqN6yskUYy5kLrtmnqmKwsMPQsb5MGWsYJlRRkBQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000610, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Signed-off-by: Jesper Dangaard Brouer --- kernel/cgroup/rstat.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index fb8b49437573..2a42be3a9bb3 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -279,17 +279,30 @@ __bpf_hook_end(); * value -1 is used when obtaining the main lock else this is the CPU * number processed last. */ -static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop) +static inline bool __cgroup_rstat_trylock(struct cgroup *cgrp, int cpu_in_loop) +{ + bool locked; + + locked = spin_trylock_irq(&cgroup_rstat_lock); + if (!locked) + trace_cgroup_rstat_lock_contended(cgrp, cpu_in_loop, !locked); + + return locked; +} + +static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop, + bool check_contention) __acquires(&cgroup_rstat_lock) { - bool contended; + bool locked = false; - contended = !spin_trylock_irq(&cgroup_rstat_lock); - if (contended) { - trace_cgroup_rstat_lock_contended(cgrp, cpu_in_loop, contended); + if (check_contention) + locked = __cgroup_rstat_trylock(cgrp, cpu_in_loop); + + if (!locked) spin_lock_irq(&cgroup_rstat_lock); - } - trace_cgroup_rstat_locked(cgrp, cpu_in_loop, contended); + + trace_cgroup_rstat_locked(cgrp, cpu_in_loop, !locked); } static inline void __cgroup_rstat_unlock(struct cgroup *cgrp, int cpu_in_loop) @@ -328,7 +341,7 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) __cgroup_rstat_unlock(cgrp, cpu); if (!cond_resched()) cpu_relax(); - __cgroup_rstat_lock(cgrp, cpu); + __cgroup_rstat_lock(cgrp, cpu, true); } } } @@ -348,9 +361,16 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) */ __bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) { + bool locked; + might_sleep(); - __cgroup_rstat_lock(cgrp, -1); + locked = __cgroup_rstat_trylock(cgrp, -1); + if (!locked) { + /* Opportunity to ongoing flush detection */ + __cgroup_rstat_lock(cgrp, -1, false); + } + cgroup_rstat_flush_locked(cgrp); __cgroup_rstat_unlock(cgrp, -1); } @@ -368,7 +388,7 @@ void cgroup_rstat_flush_hold(struct cgroup *cgrp) __acquires(&cgroup_rstat_lock) { might_sleep(); - __cgroup_rstat_lock(cgrp, -1); + __cgroup_rstat_lock(cgrp, -1, true); cgroup_rstat_flush_locked(cgrp); }