From patchwork Wed Jun 26 21:18:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesper Dangaard Brouer X-Patchwork-Id: 13713434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2FC0C27C4F for ; Wed, 26 Jun 2024 21:18:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E2DC56B008C; Wed, 26 Jun 2024 17:18:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DDCFC6B0095; Wed, 26 Jun 2024 17:18:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CCBBE6B0096; Wed, 26 Jun 2024 17:18:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B53656B008C for ; Wed, 26 Jun 2024 17:18:15 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 66068A1776 for ; Wed, 26 Jun 2024 21:18:15 +0000 (UTC) X-FDA: 82274302950.07.7C329F6 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf05.hostedemail.com (Postfix) with ESMTP id 4D7F7100004 for ; Wed, 26 Jun 2024 21:18:12 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="D/Wac0qx"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf05.hostedemail.com: domain of hawk@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=hawk@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719436680; a=rsa-sha256; cv=none; b=jn/Kkc17zoahnq5o4T2ZrZ28JEMid9qM/JaMVqbxMlSmOCYf3p30f7ntpAaAraNeQX+o6H 1Pr5UU0hgC0YK+AEMbcZvu6H8JnF4E4C2bPdmnF31FKFF9wAJQtkeszApFD0MlMjwf2knY duFBCv7op8BS5NQEPhe0PwH7XukQEEk= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="D/Wac0qx"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf05.hostedemail.com: domain of hawk@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=hawk@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719436680; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=zz3Zc6teVxuyhjcUKZVJM4+/QQ/o/g+ZirA+qV+GwsI=; b=a/4XAVbV7Fbi6O157zYrH2tXjEzZhKW2kglpM+2IsvdybE6tleOnaw2a6prCowHV62n9Ye wCsqQ7CRFdAJumYgsvd2k3AFuqtUYG1GeSIYu/nICwOHEFDflQ9sPa6Wj2jps5/cNIrfQh v/+7IVLWO3+5zwo/Sl7kmk0cou3G3+Q= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id CE677CE2C7F; Wed, 26 Jun 2024 21:18:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 949C1C116B1; Wed, 26 Jun 2024 21:18:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719436688; bh=REm3K5MgPmRQpONdIW7zODAdzetyL88KjbhiskxYUbE=; h=Subject:From:To:Cc:Date:From; b=D/Wac0qxevcEfjBcZDFu7IFQMj7jU4HtWpIqu0UnsE2ueDdQJM712AAu2CFpsPkzm V9OegwGAcqG45Q5Ikg0ns+qtqKoRSfvLPTmLJ7RyyZN9k9M+uLoz36FdZxoWvkN19v vi0C+cDTKdkTH9SnrCWh4zgaoEQF7kFY0Kz2vjBLPTDa80mnzwoSDTc/MRrHwMZacG t0scleUPQrexxc+1kfkPa6Df6yv8zo6x9f/gG1h/IsxuCT8GUkrD5ohnlbqO2KAaYh dQQujx64oJpmy1IymKwad5QAatk3MlbkfWOQevsY8XwoeJ73PJ6sMb4BLhebfwNb6S ex6hfIqtR+Ivg== Subject: [PATCH V3 1/2] cgroup/rstat: Helper functions for locking expose trylock From: Jesper Dangaard Brouer To: tj@kernel.org, cgroups@vger.kernel.org, yosryahmed@google.com, shakeel.butt@linux.dev Cc: Jesper Dangaard Brouer , hannes@cmpxchg.org, lizefan.x@bytedance.com, longman@redhat.com, kernel-team@cloudflare.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Wed, 26 Jun 2024 23:18:03 +0200 Message-ID: <171943667611.1638606.4158229160024621051.stgit@firesoul> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4D7F7100004 X-Stat-Signature: a9iof9354at7ggr6y785zcnz71bhr3xe X-Rspam-User: X-HE-Tag: 1719436692-987486 X-HE-Meta: U2FsdGVkX1/+qwq/8NUxoY5h1XeGzRrgAxeRjg920ZNIVbvdlZwDADt4uQ8xg3rV1jSYY7i7Gxnz+KpF738m+NeFjbH4yz6TaNQXs2jDJD8IMigjG8xhg6S+JCIPuvnhTkarNbwiGy5UraOpa1avhJ07i0/z8tPO/eKFoOt+SY0CkRCxu+bFCZWyxE/Qy3/iJiD0Pkh2TvbjP/B3QpfUbg5lhAFP6hSwSIklPsOkya6EIZYoeEh4M4CJbBdvSlUM2p9XLD7om0oomafl7o74u0b1ePW3g+bwHzpMfgT1QoQFkPqo3iwg1J3oxbl7ap5Y4ESESUTv0wOsqPaZez9UHy+gFw2PrRYVvz69Z1L8w1nF7dXKGNARaRZ0yo8NnAfgSlshJYrjdnaQmgh4sQCkdplSQpZ+tpgZ9pXmbflV7L0KadhFM5a9vq25X9FxNn9DznzlLin/my17QQQerdWetTnPXo+koAafG/W9df9pR7S7shhCD95f1u3fYgqqiTNzVCxEQRWzrS77Bsc60VOrmN7gcJOyXsk6rCn4UJO35Gi5dh1bHsvyZ9PmGjYuIRTONSPpzJo0bP/8KFwJCPIhsk7/J8hzpZnpWQRI8HsHDHIi6xcBQtYPu3fUlTYB5j/e1QH6d7fKMvvOoAB1h2pGxN2Mc5BlmnmnE6AVDtnFmv47sVlVzCYfb42YKn8jAufB+zUoiU7832zVdeUgnufAHuuVzeoRuY0i77JDUz8a48PslDzEw95eJ4/Gw5K+GylbXs1Jm/ZciVVf830ZKczVcBmkcbJ1B0DaqVW2yjWu5Y+LVh6WZFDFVfZO8a9xk0MhQ7cRLHsRpUUb0cc89tQSO4AkisUzeD5o10tW7lOPfRWMSYE5OSaX5ZdCWzNIpPVHCsJjwq/4vGTtRNwkMfAmdhKQk3tJtbkiX0Gu5JF5MTysaGfEgVZIkf4eqZBmzS3ysoaktrZjWvqV6mFhdvG +AJdEZVQ 2ymJKoTj2jneQJBTBz/FXIx1Q0811UFvRqk2nPELrenOnUE+fjQGMlwQBWJiWauzZABY4Yf02QZgQWdrtqX+IuRF3YyDES4l3UXtN4nKcCa7SNpTzX1nolaGmyk0yRoEPrGdJk0Mc5qRnDQvQ5QhtAqOmnVaZa/FN5dDac0uGBq4qmwTIV4N8oeHLqu0nQWeObYdFgWPtfimJucMlv2459kiybV4EKRYfgwduwaFcymc73LtmO+n9Pwhlbg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.017167, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Signed-off-by: Jesper Dangaard Brouer --- kernel/cgroup/rstat.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index fb8b49437573..2a42be3a9bb3 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -279,17 +279,30 @@ __bpf_hook_end(); * value -1 is used when obtaining the main lock else this is the CPU * number processed last. */ -static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop) +static inline bool __cgroup_rstat_trylock(struct cgroup *cgrp, int cpu_in_loop) +{ + bool locked; + + locked = spin_trylock_irq(&cgroup_rstat_lock); + if (!locked) + trace_cgroup_rstat_lock_contended(cgrp, cpu_in_loop, !locked); + + return locked; +} + +static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop, + bool check_contention) __acquires(&cgroup_rstat_lock) { - bool contended; + bool locked = false; - contended = !spin_trylock_irq(&cgroup_rstat_lock); - if (contended) { - trace_cgroup_rstat_lock_contended(cgrp, cpu_in_loop, contended); + if (check_contention) + locked = __cgroup_rstat_trylock(cgrp, cpu_in_loop); + + if (!locked) spin_lock_irq(&cgroup_rstat_lock); - } - trace_cgroup_rstat_locked(cgrp, cpu_in_loop, contended); + + trace_cgroup_rstat_locked(cgrp, cpu_in_loop, !locked); } static inline void __cgroup_rstat_unlock(struct cgroup *cgrp, int cpu_in_loop) @@ -328,7 +341,7 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) __cgroup_rstat_unlock(cgrp, cpu); if (!cond_resched()) cpu_relax(); - __cgroup_rstat_lock(cgrp, cpu); + __cgroup_rstat_lock(cgrp, cpu, true); } } } @@ -348,9 +361,16 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) */ __bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) { + bool locked; + might_sleep(); - __cgroup_rstat_lock(cgrp, -1); + locked = __cgroup_rstat_trylock(cgrp, -1); + if (!locked) { + /* Opportunity to ongoing flush detection */ + __cgroup_rstat_lock(cgrp, -1, false); + } + cgroup_rstat_flush_locked(cgrp); __cgroup_rstat_unlock(cgrp, -1); } @@ -368,7 +388,7 @@ void cgroup_rstat_flush_hold(struct cgroup *cgrp) __acquires(&cgroup_rstat_lock) { might_sleep(); - __cgroup_rstat_lock(cgrp, -1); + __cgroup_rstat_lock(cgrp, -1, true); cgroup_rstat_flush_locked(cgrp); }