From patchwork Thu Dec 12 18:02:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uladzislau Rezki X-Patchwork-Id: 13905761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89578E77182 for ; Thu, 12 Dec 2024 18:02:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 178896B0083; Thu, 12 Dec 2024 13:02:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 10BB16B0089; Thu, 12 Dec 2024 13:02:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6D9C6B0088; Thu, 12 Dec 2024 13:02:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CC0566B0083 for ; Thu, 12 Dec 2024 13:02:17 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7875E160508 for ; Thu, 12 Dec 2024 18:02:17 +0000 (UTC) X-FDA: 82887075558.23.ECB6DDA Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by imf03.hostedemail.com (Postfix) with ESMTP id 56E2E20033 for ; Thu, 12 Dec 2024 18:02:04 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="mtY/LyDm"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.48 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734026512; a=rsa-sha256; cv=none; b=I0u4FPMf3ba0+wMGD8oRRosK4Uo9gBey3wrPXy0k2nphBT8I00dfMvMFKJtP+55KTJEpQY rFku/qVj62qptr/B8FAIInydiqp5f11pcvMzSCBwmQvJMr4MyLBpMCpz+SMwQSqkBcVdd9 iD3js0DFm4LiIm+mTlMkPskrPJhFuug= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="mtY/LyDm"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.48 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734026512; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tH4hkCLHUFdjR9ahFZD+4T/NQJlUp6AP/fKB9gG9Cew=; b=zf7GHy4Gwc6cR9SfO1upvt/lABBi7nVi9/AtgrfcfNCdPqZfITZ/OuPqNDQxfJVUhpxhap PS+uIOo13DAtpyE0oC/LPHffLNlwlF79QfWg1xWwxEbXZAuOD6HRylwkyb0BXZUnf9u8x5 qP3zM8wgrUaT3VdJoPsFBT9K4FIHLBE= Received: by mail-lf1-f48.google.com with SMTP id 2adb3069b0e04-53ff1f7caaeso1057065e87.0 for ; Thu, 12 Dec 2024 10:02:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734026534; x=1734631334; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tH4hkCLHUFdjR9ahFZD+4T/NQJlUp6AP/fKB9gG9Cew=; b=mtY/LyDmQ4DGeB9kfcy+qq46HUtaVd5ImKzv7G9AHEJZH8x1A/wyRRXpbhakntaC+C yLoelvnYzOtM9EjsRiQNy/ptE2Hoy5GjncKpjr+wyfPfgp4z137lonjA2exgj8Gvs5vg ArWAW8AZfB/b8+afyVAJxu34nIuadSjDJYBvqeQqJr07QClH3lB5gdOS+2bb8gJQgL7j hBRGNjEZf9IB4l8PDYgItezfTifd90/fhsykUrs/9fsPgQ87XX5fLQwiUe2gkf7QGE0Q gFEtT8/tFbb+FVqRveUMqXLvgK0w+N6X31LsvXoN9OwrTn3n1rFKLoczV0CXcNvSSZGG F9Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734026534; x=1734631334; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tH4hkCLHUFdjR9ahFZD+4T/NQJlUp6AP/fKB9gG9Cew=; b=Oo1tS9vOcjnuh39KiB2WBTtSv/15/s971M7bDhkcqGBIr37jH88yNp0+da8TJGonia eoTJQAzPtkPraBtfW6xdY/2CxHyaOJcmnSvyMdngU8Z97A18kRX01ywiO+mkgVe54HMv 5w5CH7ngQQHyykQa3iy1QEwPBc000KLJxYzwmM4Lwc8CLds0EmduPEFtG3ChN/333BuQ brUHGm0E8d4ZSsM3rVwFvA/mjuk2T5nRTuSaWFwXZ8soZgyOvjcqH8LV7evQun7pBu8H xEY5E9Dnjg4GZzFtvRAWYRC7itbNOYTGdOx+3nuWe4JHViNWrb8ieK8C8zPLy+AuNfAD 8rMQ== X-Gm-Message-State: AOJu0YxcwT6U4a4ekCcpuZoLYyDXYK7YgU0wgSzUexDZ9+lPbL8xwRJZ iIAJ7FB2gggAWxVppqHBPevNArtWgNKfR/74umD7w0uuXZKtWaq1da5xFw== X-Gm-Gg: ASbGnctdizwMneCWJJOz/xo1jBLAXgj/o7kqpQvMkHxJFdGUvcrXJ3GW2rW7oLAymwu A4MNeghuP9YaDLud+34r3r3Q/ta8oD+68rUQ9Rbz7H0dSZrQoDvuOv0MPrNwQ1SP5YL99jQ0L7/ b4UlC0rb/lF/8tZMHBihTi63n6Tb2wC/6XGoe8y55Xu8x6+BGpGh/BA/0IForHsEkCMmZKPK49A +ONXUxraRMkPtwhZc+lhwQoQMXvPHdRh3nfmBGouCImP42XXvJD X-Google-Smtp-Source: AGHT+IEKLvPMI5CiAsBGeGMYPXutOtLCJKZ20+8zR8pPilM981v0MVz8wxdnXFoxtX1lsxgDEQ77sQ== X-Received: by 2002:a05:6512:3b13:b0:53e:350a:7298 with SMTP id 2adb3069b0e04-54034111a86mr428178e87.25.1734026533208; Thu, 12 Dec 2024 10:02:13 -0800 (PST) Received: from pc638.lan ([2001:9b1:d5a0:a500:2d8:61ff:fec9:d743]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-53e39375d36sm1940645e87.7.2024.12.12.10.02.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2024 10:02:12 -0800 (PST) From: "Uladzislau Rezki (Sony)" To: linux-mm@kvack.org, "Paul E . McKenney" , Andrew Morton , Vlastimil Babka Cc: RCU , LKML , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Uladzislau Rezki , Oleksiy Avramchenko Subject: [PATCH v2 2/5] rcu/kvfree: Move some functions under CONFIG_TINY_RCU Date: Thu, 12 Dec 2024 19:02:05 +0100 Message-Id: <20241212180208.274813-3-urezki@gmail.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241212180208.274813-1-urezki@gmail.com> References: <20241212180208.274813-1-urezki@gmail.com> MIME-Version: 1.0 X-Stat-Signature: ntietxpenpr3pw8i8kd7bpfaadno3c3f X-Rspam-User: X-Rspamd-Queue-Id: 56E2E20033 X-Rspamd-Server: rspam08 X-HE-Tag: 1734026524-531753 X-HE-Meta: U2FsdGVkX1+4v3F42CB9ps7GcRlCWeu4KwDp8pqOygCExIvxK2CksaR8XS8kcc0mDHUHnfo8/KpamWpHittve/YrLjfCVzIxoPnlZ8DEqS9G9UqgRRwEi08GnEV8FMxVpeskjeJbJVCXaTR5VU4K2hdl64a+fz1lQkkYPKecXNp7jIdidsRj+IBCFWgl9QKa+5hngiOeebTzSvH0Fjy4k6oNca82VdPeZI6qH4KGCR7ZMf93tcqIfxrwlfwqkNzgOu2lmJ4qVpFgyhEV8iR1gFJYrFz+Xdm6i1SAOTpVpvCoIi+CH6c4e7QUtKKCj8phbHu7N9Z4DShYq2TL6lJX/XJkg2RUOLBCY1bSqmvLbQX3OIwEpNPFjAJkuwaexUqEAxdN5KWmvWGnbSSSx5Z8UhOmNCA6mEMCwKZwpJ0yde5zHt0Yaw94q7LvOzEJNDxgEahwRs3BP9wgwH5Ln/CVVmQ+NO7zFJBp7uwMInVDIhhMf6HCxlWW036XsqMS5zG9qxyksT0UemFyBZX5mX+BP/m3IO6ivy8vl7Omui/95IqJGFfPw1Q03O1tmvoaqAze1JSlGxOtggTCiUHHOYXsUjIcfO0njGZDxXFXuYCMp7HmtdiMdqCMYtd+FBrw2sOQW4Cu5BchDiLSrEGnYBgrhPvvR2P3c2MCr5aYbMeLbDCZmCyv95XW4XxsHEEBOcUzeo2G//wKQd3ZWr/pXv9Sm0qOI8+Jwh78NLkB16sVcduaIJS54i+NrfANw6UydpVNT1r8wHkAtaSwKonTGZ4dPxd3Gw6bNBSXj0On2qxc5Z3OmW3LgOFyh7gAYZu2o6b4JCVoqG462b/nALCHbKakoBLeyezIfpeA/DU96n71hEDSCCL47wUbNO8AFt0K5RStvUPmci+Xu1V22eJwlIkj8s5kZU33+He9eIS+ISvkoTVTSPWKLy4m60nj0PT4Bgos3hZsF0M/x2oOFTNNgJm +gEoga5+ FSaqijmpQlVrOgDcM5zMfv0KLkFuVGvI9NDQkNOns6654obps5H9isUPaAs7eNIMIJg39iqnY2e3Y/PiPUBgIWjklg399N9WnAqLanWxJ9fMjnwJQdkIg/T8N7OGY+K3+aNdkLjDTBaHmgcZkEK8GoDbEhozaoq6r380Lx7TIcHetIr+CTSEAEt1IGio0aGuPHxycLKVJWLGyFdUveHztDt3p4DO8vf+8XQqDTuES50SH/JE8HWxTa1vmWAVCtl/3sseJdRSj6ccGYa7Wn1bXVIBjYLJZLDaDSfCt87KMkfxjv7beKn6mgTjIXq9Rna2KDfSVwg6o3WBgSwjJbXNDX+aByIUyfft+O9Dy4b+j4+Oyv7SMhiN5woNBX1I03qTwxkwoN4rY8YysUS9BO9ISYHauqUbv0G8Wrk25SVp7HY74GNSGrPEMGjQ4l2q591vdPF3mZJl16T+B2Auy1v06daxJnbK4X4kiZAXPy1aAWLErilBwRU5IWtIofQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000005, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently when a tiny RCU is enabled, the tree.c file is not compiled, thus duplicating function names do not conflict with each other. Because of moving of kvfree_rcu() functionality to the SLAB, we have to reorder some functions and place them together under CONFIG_TINY_RCU macro definition. Therefore, those functions name will not conflict when a kernel is compiled for CONFIG_TINY_RCU flavor. Signed-off-by: Uladzislau Rezki (Sony) --- kernel/rcu/tree.c | 90 +++++++++++++++++++++++++---------------------- 1 file changed, 47 insertions(+), 43 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index e69b867de8ef..b3853ae6e869 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3653,16 +3653,6 @@ static void kfree_rcu_monitor(struct work_struct *work) schedule_delayed_monitor_work(krcp); } -static enum hrtimer_restart -schedule_page_work_fn(struct hrtimer *t) -{ - struct kfree_rcu_cpu *krcp = - container_of(t, struct kfree_rcu_cpu, hrtimer); - - queue_delayed_work(system_highpri_wq, &krcp->page_cache_work, 0); - return HRTIMER_NORESTART; -} - static void fill_page_cache_func(struct work_struct *work) { struct kvfree_rcu_bulk_data *bnode; @@ -3698,27 +3688,6 @@ static void fill_page_cache_func(struct work_struct *work) atomic_set(&krcp->backoff_page_cache_fill, 0); } -static void -run_page_cache_worker(struct kfree_rcu_cpu *krcp) -{ - // If cache disabled, bail out. - if (!rcu_min_cached_objs) - return; - - if (rcu_scheduler_active == RCU_SCHEDULER_RUNNING && - !atomic_xchg(&krcp->work_in_progress, 1)) { - if (atomic_read(&krcp->backoff_page_cache_fill)) { - queue_delayed_work(system_unbound_wq, - &krcp->page_cache_work, - msecs_to_jiffies(rcu_delay_page_cache_fill_msec)); - } else { - hrtimer_init(&krcp->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); - krcp->hrtimer.function = schedule_page_work_fn; - hrtimer_start(&krcp->hrtimer, 0, HRTIMER_MODE_REL); - } - } -} - // Record ptr in a page managed by krcp, with the pre-krc_this_cpu_lock() // state specified by flags. If can_alloc is true, the caller must // be schedulable and not be holding any locks or mutexes that might be @@ -3779,6 +3748,51 @@ add_ptr_to_bulk_krc_lock(struct kfree_rcu_cpu **krcp, return true; } +#if !defined(CONFIG_TINY_RCU) + +static enum hrtimer_restart +schedule_page_work_fn(struct hrtimer *t) +{ + struct kfree_rcu_cpu *krcp = + container_of(t, struct kfree_rcu_cpu, hrtimer); + + queue_delayed_work(system_highpri_wq, &krcp->page_cache_work, 0); + return HRTIMER_NORESTART; +} + +static void +run_page_cache_worker(struct kfree_rcu_cpu *krcp) +{ + // If cache disabled, bail out. + if (!rcu_min_cached_objs) + return; + + if (rcu_scheduler_active == RCU_SCHEDULER_RUNNING && + !atomic_xchg(&krcp->work_in_progress, 1)) { + if (atomic_read(&krcp->backoff_page_cache_fill)) { + queue_delayed_work(system_unbound_wq, + &krcp->page_cache_work, + msecs_to_jiffies(rcu_delay_page_cache_fill_msec)); + } else { + hrtimer_init(&krcp->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + krcp->hrtimer.function = schedule_page_work_fn; + hrtimer_start(&krcp->hrtimer, 0, HRTIMER_MODE_REL); + } + } +} + +void __init kfree_rcu_scheduler_running(void) +{ + int cpu; + + for_each_possible_cpu(cpu) { + struct kfree_rcu_cpu *krcp = per_cpu_ptr(&krc, cpu); + + if (need_offload_krc(krcp)) + schedule_delayed_monitor_work(krcp); + } +} + /* * Queue a request for lazy invocation of the appropriate free routine * after a grace period. Please note that three paths are maintained, @@ -3944,6 +3958,8 @@ void kvfree_rcu_barrier(void) } EXPORT_SYMBOL_GPL(kvfree_rcu_barrier); +#endif /* #if !defined(CONFIG_TINY_RCU) */ + static unsigned long kfree_rcu_shrink_count(struct shrinker *shrink, struct shrink_control *sc) { @@ -3985,18 +4001,6 @@ kfree_rcu_shrink_scan(struct shrinker *shrink, struct shrink_control *sc) return freed == 0 ? SHRINK_STOP : freed; } -void __init kfree_rcu_scheduler_running(void) -{ - int cpu; - - for_each_possible_cpu(cpu) { - struct kfree_rcu_cpu *krcp = per_cpu_ptr(&krc, cpu); - - if (need_offload_krc(krcp)) - schedule_delayed_monitor_work(krcp); - } -} - /* * During early boot, any blocking grace-period wait automatically * implies a grace period.