From patchwork Tue Sep 17 14:34:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boqun Feng X-Patchwork-Id: 13806223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96097C3601A for ; Tue, 17 Sep 2024 14:35:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 755A26B0089; Tue, 17 Sep 2024 10:35:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7039C6B0092; Tue, 17 Sep 2024 10:35:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50ABA6B008C; Tue, 17 Sep 2024 10:35:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2EF586B0085 for ; Tue, 17 Sep 2024 10:35:03 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D64E9160538 for ; Tue, 17 Sep 2024 14:35:02 +0000 (UTC) X-FDA: 82574477244.16.16348BC Received: from mail-yb1-f170.google.com (mail-yb1-f170.google.com [209.85.219.170]) by imf17.hostedemail.com (Postfix) with ESMTP id AC20C40009 for ; Tue, 17 Sep 2024 14:35:00 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nSreRGDn; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.219.170 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726583691; a=rsa-sha256; cv=none; b=oW1RUcB0Svclp1474ylbd1aiEJhn/OGZ3Z0imsCmIwxj8+PNBanSLoQp5zQ6bYcXt2kMFA NgCH5r863GldoJrneX+cfh4loslAB/InE7O3w0nvZmFrFLCaS+YmaxRjb7EsHnRlDI6Aqw Qf7pEfRKNFcktxLZjQZ8/BFhdwNQPys= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nSreRGDn; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.219.170 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726583691; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GcfRMZ3iyMDLyZX4twHVpqBTdlI1aOiND6/+aYkmmFI=; b=DTUokpJm2OUBW1ORvE9VVjqf+x1iQlLJvrRieKDYyCPloDraHJ0XKcZuyjtxN/LFAhUcd4 CNPZYEELqvoQlat3Lck22jACNZjHILz8wXP9DFJJqCR9zaP1wqTZs2p16xHi7yO1Nbu3J8 WssA8g8U8Hw+fjiASLIBH/XbMMrvUqc= Received: by mail-yb1-f170.google.com with SMTP id 3f1490d57ef6-e1a9e4fa5aaso5496565276.2 for ; Tue, 17 Sep 2024 07:35:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726583700; x=1727188500; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:from:to:cc:subject :date:message-id:reply-to; bh=GcfRMZ3iyMDLyZX4twHVpqBTdlI1aOiND6/+aYkmmFI=; b=nSreRGDn437uvRl4CbHtsthTjgdOAe20YAbnJOErzLWDwMTr/eh48MsNJSW7e9sxup 6a0O6nQMuGSdm0HzQFyWpxxy2xN5+As6/kNDq0M9Ta9vNo1HqMphiH8JnkOtHI03md3c ynktDqEIU9DatDyXayW3QvVa0hcsroqU5M8YHyPYvPeZ0758wJZYHk0q3hrsEV9C/H/N 9XciIGkj/zgHPdMUjJd4M0Zzgw8i7S1ZmQl+Zuw7fpdiyvUBsf8MKRAjdnlK/oXyFirA 6t4mP4SfqnNTJKgKDDTI9/m1haBKGxXy2dM7T27JPQBDVbs0lMBHceqleZxbhpMJPdyy qdRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726583700; x=1727188500; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=GcfRMZ3iyMDLyZX4twHVpqBTdlI1aOiND6/+aYkmmFI=; b=hhY3WsV8f7NtlypJt+SYhi7BKnQbcC3WERT53Yjy0qwHQLYXeKPOOpWHb/0vrLDoZy TyjGn2k813c9+SbmazTJcxlWUB5tIe+FZ6xynnbjFlYnf2NMePVp09SEXoKxvwBbFb4y geLyT/Xg4zmMLPm6e6yvOHNb/En4H9sWJyft5k7D8liUO+ouJeC6ZZyjP82mssueHiZz fCgphBqSqwQj1XjVTBb5xKyHJl5XPaHZpdCRdzRojDyidHN0TI0NokwcNOUaStsiZton A5iXMr6oFYrbu37oE9rtkRWdxkIDhoreNrpifOidXG7EVn8fypsnMlu/hNHgayyTE1HA szJQ== X-Forwarded-Encrypted: i=1; AJvYcCViiVWpm9M2xFj5nTJGr88yf252x/4OQ9dtUiG/8FKkrMtek0E8Mu5UzT+HtvA8t6aJzbcpVV5Oyg==@kvack.org X-Gm-Message-State: AOJu0Ywor6tSe2r2Ao4XQxDX8MRZVDFZ9JDq2vSis3b7VVe3tnhOudMM hIHWChgEqZVnGmvE3e8Z7vI3iLmICwejkaVfG9MKvxutjlBal0LX X-Google-Smtp-Source: AGHT+IGuBl7w6kHeocxSsrq7wz1PL/D3xhcWK31fEFq1kkjp/eaeFPUIN6wONAeMPCU446DmR2QvTw== X-Received: by 2002:a05:6902:cc7:b0:e13:ceaa:59d9 with SMTP id 3f1490d57ef6-e1d9dc41795mr12290884276.41.1726583699724; Tue, 17 Sep 2024 07:34:59 -0700 (PDT) Received: from fauth1-smtp.messagingengine.com (fauth1-smtp.messagingengine.com. [103.168.172.200]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6c58c7da98bsm34819796d6.145.2024.09.17.07.34.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2024 07:34:59 -0700 (PDT) Received: from phl-compute-04.internal (phl-compute-04.phl.internal [10.202.2.44]) by mailfauth.phl.internal (Postfix) with ESMTP id 7E0841200070; Tue, 17 Sep 2024 10:34:58 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-04.internal (MEProxy); Tue, 17 Sep 2024 10:34:58 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrudekjedgjeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephffvvefufffkofgjfhgggfestdekredtredttden ucfhrhhomhepuehoqhhunhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrihhlrd gtohhmqeenucggtffrrghtthgvrhhnpeegleejiedthedvheeggfejveefjeejkefgveff ieeujefhueeigfegueehgeeggfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpegsohhquhhnodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhi thihqdeiledvgeehtdeigedqudejjeekheehhedvqdgsohhquhhnrdhfvghngheppehgmh grihhlrdgtohhmsehfihigmhgvrdhnrghmvgdpnhgspghrtghpthhtohepvdehpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehlihhnuhigqdhkvghrnhgvlhesvhhgvghrrd hkvghrnhgvlhdrohhrghdprhgtphhtthhopehrtghusehvghgvrhdrkhgvrhhnvghlrdho rhhgpdhrtghpthhtoheplhhinhhugidqmhhmsehkvhgrtghkrdhorhhgpdhrtghpthhtoh eplhhkmhhmsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepphgruhhlmhgt kheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepfhhrvgguvghrihgtsehkvghrnhgvlh drohhrghdprhgtphhtthhopehnvggvrhgrjhdruhhprgguhhihrgihsehkvghrnhgvlhdr ohhrghdprhgtphhtthhopehjohgvlhesjhhovghlfhgvrhhnrghnuggvshdrohhrghdprh gtphhtthhopehjohhshhesjhhoshhhthhrihhplhgvthhtrdhorhhg X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 17 Sep 2024 10:34:57 -0400 (EDT) From: Boqun Feng To: linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-mm@kvack.org, lkmm@vger.kernel.org Cc: "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long , Mark Rutland , Thomas Gleixner , Kent Overstreet , Linus Torvalds , Vlastimil Babka , maged.michael@gmail.com Subject: [RFC PATCH 3/4] refscale: Add benchmarks for percpu_ref Date: Tue, 17 Sep 2024 07:34:01 -0700 Message-ID: <20240917143402.930114-4-boqun.feng@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240917143402.930114-1-boqun.feng@gmail.com> References: <20240917143402.930114-1-boqun.feng@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 46iastom6bpggqg7aqz6km37noikfof7 X-Rspamd-Queue-Id: AC20C40009 X-Rspamd-Server: rspam02 X-HE-Tag: 1726583700-682198 X-HE-Meta: U2FsdGVkX1/48uisqpsVgPlcAqlddcFHQVOFpsHeoIjHwi0P+kIIhPDi+Zl4ovD4W97+w8HgcAgVP0bJBbosGWG8y/iPHZecO2FK8dmh4kKxx3xB5qIZ7pwy+o6SMuafJ8eFtNZzEXrbweHMWfYt6mll3ZlJoliZg7mTamy5+Lvj+1f7zI7U7koryH+XaQSGjqJECAJNzuj/zckNx0+XQ4zueT5eC0/ssr13uNyTffoRkeFrgu9nUv/s20KfXGk1jRTfzH5OZAIE2EYIoW8tfYb1cevirW6lMUyrb2RNKELFnj9gpynZ3u+c2Gho3TdkS75nwA7z/pkOZDT+CFfJDhiUeTEKha3G8qDnFrtEtNWgFwAPC4NvAEHp7Z1nOjDC80HFVg1IpGmAa/gUO/Qgwzu1RbxYUMAmygjp5gHkSrsXLm+ghuy6C17fXAbfGa3OCcbiVGLJY74ByRHrHZ9UfX7j3Qc6BuK9VWshITyXSzut3F9Ulg/OiQXogRwN9vZOYp2oh67OpbxVqlWpvWd674eFs1PkIyEfSflFTee5jXB+zKuFR+pzi6yEvEXnJ3tQdAEld7vnx6ukl6ixr+pLMp1cuo8AcW/WwngV9YGVjOD4k2bVaN+QuVJpvfZe7UtIMLHKpFjbqbkUm7hZoIYFnlVzm93vJnAfx+nDD6veomwIe0ZNGRCRhnBtcJcnhfvtLGBFUgnTqr/LU1eLxuBnCecWkT9cznbrpJBgRvczar8VkgXHHbXfmwZMZES94vR+uNC/utk7fJvkGeaPhjFz+ZTwSf5vPyZMYE2aOFjiyJ4DQFRE/RlIxv1/LoXB7H5lCY1JQm9h7bF25MxjJZ5f7Jtznvh1f8c2+CeRZ+0aBbxSW4Sf2WyadFtQOmbBpqbcY5NmVXV06FdfKlp+zEbeC4nJtpWBLFz78l7PjYyfpOfJVqRG8QA/SfuubD2hhZv02WSQTu5zHo2E5bZZ5h2 Oak7YDaL 8DcuhQPUbou8TcYzA4rT7YsuH9grKpHdxOtMS3d69A+I3XCuEsqERZHqiSbCGg7/4qULkP3Bk6VduW3lxxXxEExrYHOijiu3GWgtSotRINKPKu4xJxPItoRHnoLEYaq1je+bJOoKD2XnrhpQW/E+vZo47/qMeG+wAkRX41m6LaoCKyCboZ3xsoz9jUy3JHXw/j1s9uO7D+nWSIfq3qVnKJoRPt+j4QLL+hKF/SX9jysxAi27FGtAMPHmgrRB1MEpA/WDpxXU9z9S3xsvNkcb+otOp+CF9QetrAqwuxBi293R+lpC0l3q/W3uVybbADJRGLTVD5nhjkVWdzGQk3rGb3IJABaQC8vzt2WhasWZMeXqkg579p0746EjkZw5/NBVDT0owakAXqyWchWdDsVk5we26AGEijYk1G1cmGewsdBhIrXClxqV7Qu4sFcmk1icXOwKCEaXS79ScdrTdpHCgBrNKfBtuWJhAe4ux8kGZSlRjRaL5phtGEc1+gR6tpSBQgyrg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Benchmarks for percpu_ref are added to evaluate the reader side performance between percpu_ref and other refcounting mechanisms. Signed-off-by: Boqun Feng --- kernel/rcu/refscale.c | 50 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/refscale.c b/kernel/rcu/refscale.c index 7e76ae5159e6..97b73c980c5d 100644 --- a/kernel/rcu/refscale.c +++ b/kernel/rcu/refscale.c @@ -393,6 +393,54 @@ static struct ref_scale_ops hazptr_ops = { .name = "hazptr" }; +struct percpu_ref percpu_ref; + +static void percpu_ref_dummy(struct percpu_ref *ref) {} + +static bool percpu_ref_scale_init(void) +{ + int ret; + + ret = percpu_ref_init(&percpu_ref, percpu_ref_dummy, PERCPU_REF_INIT_ATOMIC, GFP_KERNEL); + percpu_ref_switch_to_percpu(&percpu_ref); + + return !ret; +} + +static void percpu_ref_scale_cleanup(void) +{ + percpu_ref_exit(&percpu_ref); +} + +static void percpu_ref_section(const int nloops) +{ + int i; + + for (i = nloops; i >= 0; i--) { + percpu_ref_get(&percpu_ref); + percpu_ref_put(&percpu_ref); + } +} + +static void percpu_ref_delay_section(const int nloops, const int udl, const int ndl) +{ + int i; + + for (i = nloops; i >= 0; i--) { + percpu_ref_get(&percpu_ref); + un_delay(udl, ndl); + percpu_ref_put(&percpu_ref); + } +} + +static struct ref_scale_ops percpu_ref_ops = { + .init = percpu_ref_scale_init, + .cleanup = percpu_ref_scale_cleanup, + .readsection = percpu_ref_section, + .delaysection = percpu_ref_delay_section, + .name = "percpu_ref" +}; + // Definitions for rwlock static rwlock_t test_rwlock; @@ -1158,7 +1206,7 @@ ref_scale_init(void) static struct ref_scale_ops *scale_ops[] = { &rcu_ops, &srcu_ops, RCU_TRACE_OPS RCU_TASKS_OPS &refcnt_ops, &rwlock_ops, &rwsem_ops, &lock_ops, &lock_irq_ops, &acqrel_ops, &clock_ops, &jiffies_ops, - &typesafe_ref_ops, &typesafe_lock_ops, &typesafe_seqlock_ops, &hazptr_ops, + &typesafe_ref_ops, &typesafe_lock_ops, &typesafe_seqlock_ops, &hazptr_ops, &percpu_ref_ops, }; if (!torture_init_begin(scale_type, verbose))