From patchwork Fri Jul 26 21:56:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13743186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67632C3DA4A for ; Fri, 26 Jul 2024 21:57:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F25856B0088; Fri, 26 Jul 2024 17:57:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED6026B0089; Fri, 26 Jul 2024 17:57:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9E196B008A; Fri, 26 Jul 2024 17:57:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BC3CB6B0088 for ; Fri, 26 Jul 2024 17:57:37 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id EE5EE1C057D for ; Fri, 26 Jul 2024 21:57:36 +0000 (UTC) X-FDA: 82383266112.17.EB10236 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id 4CE634000A for ; Fri, 26 Jul 2024 21:57:35 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=dWmrsooC; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722031018; a=rsa-sha256; cv=none; b=Dojhv2GVl57noXn6NChd3vuo2YJx8PZ5psxszrGgN85kGRQr5p16YemrorOqzXsHT1LJmL VfUYcnOBy5ZMNGWRaoyctbzRIMIVm7/Aatwa6Pkr85g+QTBqeWtmjzaedboDrC+yq30NZJ KGksvv6iBFQ/EMa+FA3hMjA4Gw6d7WI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=dWmrsooC; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722031018; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G+lSjvJQqRDoZKM4s7Tcf6sdPWaFThzlyfeuxJg7u60=; b=KxGbrEz69b3H5QxDdtlk/GiF5cbO91pQ1ab6RpGLe4nJYQErDKkhG+b+PNKxbTpjEqYSdi KAVbMnQjRGCahru8d9mM7/K/43oF66onEVnkvR7nDEAY2HALLkVj/ZNdzk7mM+EUmELMSP 4jn1sJ8LtkraSHPVjN+qcL+2iHbhea4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 2EDF86199B; Fri, 26 Jul 2024 21:57:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A1FDDC4AF09; Fri, 26 Jul 2024 21:57:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722031053; bh=MUWUbgWSkKV4GLdom7VkftM4X5LXDstggWbHIa2p29w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dWmrsooCdYxUdk/aCUzDpll4gueCMQsfkinYrKzAWc1v/GAaIsHjE3rm8GWfqufpy gKRyDSNOjiXVu5BGliAyGQJhFUrk1RQuDB0kNKtBzDasPdPZvrIqiCgBy2WKWw0dXy zwLgigU+SitEoGE7TrTGk6i+9RF0xdScyWgbLzJH5VQihFensb8LOZ/Astq125xDGo bUggEcJQVzgvcNRlDFR4LEap94c9Tg6FMyRYZqG70PLQLhhXB7cvyZa2XUx1ECWnPF KMrNN2lDQKT3EJi/PW5+2xQjAbSKxLd7az60jjUo0uD2e8O5hDCpQPdjuMG827Vb1O aQ0djG1ROPjcg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Andrew Morton , Kees Cook , Peter Zijlstra , Thomas Gleixner , Michal Hocko , Vlastimil Babka , linux-mm@kvack.org, "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Boqun Feng , Zqiang , rcu@vger.kernel.org Subject: [RFC PATCH 11/20] kthread: Make sure kthread hasn't started while binding it Date: Fri, 26 Jul 2024 23:56:47 +0200 Message-ID: <20240726215701.19459-12-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726215701.19459-1-frederic@kernel.org> References: <20240726215701.19459-1-frederic@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4CE634000A X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: huen16wthfyxsd96zs49tjbq5respc9i X-HE-Tag: 1722031055-517014 X-HE-Meta: U2FsdGVkX18uiazs19SM3/bsF3hgIElG7Z8m94oXSbizDvu/o5Wz2IRJrn5Pm2ifioXqiUg+JPgU7/n+xg+k6YNW+DDq5gdUGpuo/en1DttZKsqAk9UQUmjrESRPt4bh3Nawt2Tq+3NmEHHm4VmSXju+LBUL+riYs9G+uwV5R9C/CoXcdY9uR0q/WhYUrJ+Po0wU2iaGkMWAR7EFlGdM8Z9NnjUCG9/EPadVHepkVdi/r40v9fuOE4VmXq8DHG5eaiIMVw5NKDH7YxKGxYnMi8iLD6LA7PjCy/eEQORzYugaVrKcD1fPQekhE+kVjvyqYn1aCFzQ6XExn7TIEraXY4/exDQuhyYbDTlPHnCwPozpbKQXSsh6oNuVClYMcFTbBXrcDR0JqgKF+aATJpf+hNNl7TiPltHJGtM6pO3eE555/rRBXU1m2JjeAVu8MD++C8USzQCC2Y6iIrr4/Lzp5YJl/nrNZtFJH0CQitiTJ6DVXl0X7M2CASrw+WUMF7W1COjwg1p8kT6LCsqDqRyKs7dHuGbJn0QI9ia2QSI1lDb+3Aiy+mrAqhKrkRN63cu4qQVh6YipWUHq83uIjEXI6XiLIrgkZRtEwNzosFf9OsbTFc4QwhkWlM3m2LL6ISrrFMYca9cOEoXQEV5NMTum1LolhUSuVs7gnCjDBQrW7iESyLUZ5FIp/xzxyjvTV0cKsdD1Kg6j7DY+1RbIZ7hU80mlpdmFVUSgurYnY1Z8LpLjdzZX1gRdu276nNdJylKezpmgYNyNTvRYosYPZIWUyjeiibFd34qlFeen7o0rwzZUwxUg0U2545ffpoB4dXvQdKvo+fP5XrDTs/47p74ULk9hOlnk26499dqdvzuorVTgcsI+g3WMJwgmLhAUeKY7vWbE6x4kw5D0JSAQessWhxy4yqJjOgsmu4p2XgKosV4A1GSnpKhDQMlUZdTEEbyzfk5TDmAie+R/M56kPyj 5zK/adP4 +zX8pKKmqR3h5LVZhoSJ11tRocUCa3ULuvJ+8UDFuagSUKVS9afD2GJhCYV86Gn37YMVWcb5sCq1bKx3xefk+j+EQ7JjwKsLc5H9b/9txVAryAiwkkoC2aJjE+KrRmX4qtJCj2mUA+HfObwOfwNMZiu/GXaicufG1xHPZlheya4KXQnyX/mI3ggLtTcEnbXkvVZJQ3sDgpqylUbDujlr3s+mk6HxFgfqMPPkFLzhZKqLwT04ujLkN+s0cOe4Hr8Y7FDzpfnnsJqLwPjGT2KJTJafQEr2iP1xKa2dt/EwrC9o56e5luoHzwgaUPg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make sure the kthread is sleeping in the schedule_preempt_disabled() call before calling its handler when kthread_bind[_mask]() is called on it. This provides a sanity check verifying that the task is not randomly blocked later at some point within its function handler, in which case it could be just concurrently awaken, leaving the call to do_set_cpus_allowed() without any effect until the next voluntary sleep. Rely on the wake-up ordering to ensure that the newly introduced "started" field returns the expected value: TASK A TASK B ------ ------ READ kthread->started wake_up_process(B) rq_lock() ... rq_unlock() // RELEASE schedule() rq_lock() // ACQUIRE // schedule task B rq_unlock() WRITE kthread->started Similarly, writing kthread->started before subsequent voluntary sleeps will be visible after calling wait_task_inactive() in __kthread_bind_mask(), reporting potential misuse of the API. Upcoming patches will make further use of this facility. Signed-off-by: Frederic Weisbecker Acked-by: Vlastimil Babka --- kernel/kthread.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kernel/kthread.c b/kernel/kthread.c index f7be976ff88a..ecb719f54f7a 100644 --- a/kernel/kthread.c +++ b/kernel/kthread.c @@ -53,6 +53,7 @@ struct kthread_create_info struct kthread { unsigned long flags; unsigned int cpu; + int started; int result; int (*threadfn)(void *); void *data; @@ -382,6 +383,8 @@ static int kthread(void *_create) schedule_preempt_disabled(); preempt_enable(); + self->started = 1; + ret = -EINTR; if (!test_bit(KTHREAD_SHOULD_STOP, &self->flags)) { cgroup_kthread_ready(); @@ -540,7 +543,9 @@ static void __kthread_bind(struct task_struct *p, unsigned int cpu, unsigned int void kthread_bind_mask(struct task_struct *p, const struct cpumask *mask) { + struct kthread *kthread = to_kthread(p); __kthread_bind_mask(p, mask, TASK_UNINTERRUPTIBLE); + WARN_ON_ONCE(kthread->started); } /** @@ -554,7 +559,9 @@ void kthread_bind_mask(struct task_struct *p, const struct cpumask *mask) */ void kthread_bind(struct task_struct *p, unsigned int cpu) { + struct kthread *kthread = to_kthread(p); __kthread_bind(p, cpu, TASK_UNINTERRUPTIBLE); + WARN_ON_ONCE(kthread->started); } EXPORT_SYMBOL(kthread_bind); From patchwork Fri Jul 26 21:56:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13743192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AA25C3DA7F for ; Fri, 26 Jul 2024 21:57:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DEB366B0092; Fri, 26 Jul 2024 17:57:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D7A666B0096; Fri, 26 Jul 2024 17:57:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADE326B0093; Fri, 26 Jul 2024 17:57:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8C5D26B0092 for ; Fri, 26 Jul 2024 17:57:45 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3C519120175 for ; Fri, 26 Jul 2024 21:57:45 +0000 (UTC) X-FDA: 82383266490.11.2C2ECB3 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf24.hostedemail.com (Postfix) with ESMTP id EB31718000F for ; Fri, 26 Jul 2024 21:57:41 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=heNYtWqP; spf=pass (imf24.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722031012; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CW1A0sP7ya8kf/pccF0TH8tFOgIiSQElkEzym5WR3+0=; b=ku8gFg8LzfIaHzin2GYyiZtI0dzjV3mftgQ5u0No5+cusEtPQkdt4culnnhwE4Ij/nC2zt rVYZJJ9VmbwY5haia4Ab0Q6AsZD5GgO+BMakyFF3oVoarxrVX7RNIpaCxutrofFTCZogdH Dpc9QOdpne5+OrTTwIlh8N+g90mAFNM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722031012; a=rsa-sha256; cv=none; b=o6fWIirBTSGCX+VkLRLNyaQj9DCjXC8x3o2Tsp9x1btVULwSrfCNj4aE5gwPShlfreODB/ Zz2IQMTdg0XijyIBz11UCTF/w8CxGILAgue8WBA2k3d+4YL5e7o1mi0wIjvzETD5yIX6fR JD3uTydApWPPFko61NLq3YDoHAn2DLc= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=heNYtWqP; spf=pass (imf24.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 3985ECE1952; Fri, 26 Jul 2024 21:57:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4DCB0C4AF0B; Fri, 26 Jul 2024 21:57:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722031057; bh=ti4mfUgveAuqTAPPwXGYkPNNFuo7Vc9aOZIdnCtmsYQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=heNYtWqPzI279b23ynorkZtq9s+XtrGZuDGs5HGiedBJIbhqIYfZlNLZbVnJRlGWb OoIJj85ZxFgF5Pf0IoPAk1ly5fkb/U6UyuT5cW0GnVsadONHBpy7j2gDi7LQybe50+ I4fHIxGYUhytu7KJ87eKYGc87z8Mzg0xc3xy96B3iDivajwqofkWVWVJujy2Gx3XVF a2bFKPz4bsNe5kK7YlfbJU6R/0i7zEH59l/Er15+P8rBvDbO2OVJ0sx5U7dcnX1D2n wWZXB/LQV7ncN0jrEUJxS/KDWlj6z3hX0I+H0asvSQRZgwj5QmDkOnsHlYVNJPc5yU wB+175sFPcwyQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Andrew Morton , Kees Cook , Peter Zijlstra , Thomas Gleixner , Michal Hocko , Vlastimil Babka , linux-mm@kvack.org, "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Boqun Feng , Zqiang , rcu@vger.kernel.org Subject: [RFC PATCH 12/20] kthread: Implement preferred affinity Date: Fri, 26 Jul 2024 23:56:48 +0200 Message-ID: <20240726215701.19459-13-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726215701.19459-1-frederic@kernel.org> References: <20240726215701.19459-1-frederic@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EB31718000F X-Stat-Signature: echhoykdxgnq7buaaa88nkipjshdpf1n X-HE-Tag: 1722031061-716740 X-HE-Meta: U2FsdGVkX1/+CWIOCpzsE+/Itx88PDOg3fMZL+ePrgLX1waPyL4NpXS/6t+0NGreWAghYHMi9c3NlxPdA1/poRrqEn09qwTb2J1Yg4BNMczDLpXoYCbimT9JFaUPZQfzLYJ1R1pKej4NPzGG3q+W86j94HgHMDSuLK46reN/UzdxIIfhgorX50LZSnUSYe1hMfmYPhKTvmsJED4Bh+DnNDLL39GKGa73PaXmJ1Gb64DpMvJeGFb1tsmXJFcAG7Z7h2zEStREA+2IARchSSR5kzNT18xt2D9amkMBX6f28/wdWFHDi/gL/Ai+tzKrlndnnd0Ulrwt1EAO29G84TVSaHzr12uHukmttI47qO+vbKZRHZQUDQvSW1KB6nKGlyeTLE0TcHy6If7rh/wmNXXfUFQQINbMh/R2ona9FGjCf7fS6YqIUr9txT865PqSxb6GUutNEwinz590HM4ooswkIB1pBmiOxiQvIRPSWKj0/2IdNUAKPEXeRyBu4fwyDvugy3S/KEL6pZpdVb+idty6F3o0Bv2pxJZfAreZ/IEoJBg1Jz4UToqzRJUgo33U+hjUfUFLlcmkmDoQJ72UOeTNlwaVlvKc0+/2SRnPXKC0BKu2B9x9FrLtlnw8qq9568VOGxPwYXAcUNyLetVWFTR2b2fpUv7j0GIhAHp7zuv3A+6v4mbL9rgIF2vXIpRdJcOhodx/rjXO1CnDQlLfF2vnrSf/YfHIMHqsj80VLEeauv8RXUrbGNv3GIGWxtPQ1IPm9V2mmRD53x4sIa4FEZDGsk+hqZ6Z7RSqM3mPx+L/PAub/wv9+wOX4e2EutJ20vJvo2MRoxutUhUV2xYA32GNIJ74XWPpsfrAFfnE+v1ztMlHyAGoUwbGdnMINVUFV/rP1wRJllTtIL3qTzHqk5jD9TBGGbBJsky1H/7oRnqKxfM6sv5kXyzQxWe9GlrouFHiq+DXTPjRfc+HFQrxFEe yI7iNYPx T4fnyk96majneFGLDalOCpCdWBFVgZutFNLhZYKO6+t2LQmoSzIwpMFTBygOTV0dg01gWwxhChqMHvy9S++E2bKXJU6WVK+gXebSnmbuTpJ32KlqdHIj2sFzXtKy1ji/zPHoLQJaQBK/f53zypuzwk5z4xCv2/VaNmEQqtpZ69+Xk79WZ65lqg4Esy74c3ykAOXUJiAbH9uxJiEdZKCVlDx9ZymSWXT0jYI+M1XKhmRh2L/S2QpVRZtKfGLNMTIhUaugqGZwD2/LyZYUbQP97V6QVSyBwiJO5fYyEAY6Ss3St/zD8RhsShqjAhg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Affining kthreads follow either of three existing different patterns: 1) Per-CPU kthreads must stay affine to a single CPU and never execute relevant code on any other CPU. This is currently handled by smpboot code which takes care of CPU-hotplug operations. 2) Kthreads that _have_ to be affine to a specific set of CPUs and can't run anywhere else. The affinity is set through kthread_bind_mask() and the subsystem takes care by itself to handle CPU-hotplug operations. 3) Kthreads that have a _preferred_ affinity but that can run anywhere without breaking correctness. Userspace can overwrite the affinity. It is set manually like any other task and CPU-hotplug is supposed to be handled by the relevant subsystem so that the task is properly reaffined whenever a given CPU from the preferred affinity comes up or down. Also care must be taken so that the preferred affinity doesn't cross housekeeping cpumask boundaries. Currently the preferred affinity pattern has at least 4 identified users, with more or less success when it comes to handle CPU-hotplug operations and housekeeping cpumask. Provide an infrastructure to handle this usecase patter. A new kthread_affine_preferred() API is introduced, to be used just like kthread_bind_mask(), right after kthread creation and before the first wake up. The kthread is then affine right away to the cpumask passed through the API if it has online housekeeping CPUs. Otherwise it will be affine to all online housekeeping CPUs as a last resort. It is aware of CPU hotplug events such that: * When a housekeeping CPU goes up and is part of the preferred affinity of a given kthread, it is added to its applied affinity set (and possibly the default last resort online housekeeping set is removed from the set). * When a housekeeping CPU goes down while it was part of the preferred affinity of a kthread, it is removed from the kthread's applied affinity. The last resort is to affine the kthread to all online housekeeping CPUs. Signed-off-by: Frederic Weisbecker Acked-by: Vlastimil Babka --- include/linux/cpuhotplug.h | 1 + include/linux/kthread.h | 1 + kernel/kthread.c | 121 +++++++++++++++++++++++++++++++++++++ 3 files changed, 123 insertions(+) diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 7a5785f405b6..5c204bd0fed6 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -238,6 +238,7 @@ enum cpuhp_state { CPUHP_AP_WORKQUEUE_ONLINE, CPUHP_AP_RANDOM_ONLINE, CPUHP_AP_RCUTREE_ONLINE, + CPUHP_AP_KTHREADS_ONLINE, CPUHP_AP_BASE_CACHEINFO_ONLINE, CPUHP_AP_ONLINE_DYN, CPUHP_AP_ONLINE_DYN_END = CPUHP_AP_ONLINE_DYN + 40, diff --git a/include/linux/kthread.h b/include/linux/kthread.h index b11f53c1ba2e..30209bdf83a2 100644 --- a/include/linux/kthread.h +++ b/include/linux/kthread.h @@ -85,6 +85,7 @@ kthread_run_on_cpu(int (*threadfn)(void *data), void *data, void free_kthread_struct(struct task_struct *k); void kthread_bind(struct task_struct *k, unsigned int cpu); void kthread_bind_mask(struct task_struct *k, const struct cpumask *mask); +int kthread_affine_preferred(struct task_struct *p, const struct cpumask *mask); int kthread_stop(struct task_struct *k); int kthread_stop_put(struct task_struct *k); bool kthread_should_stop(void); diff --git a/kernel/kthread.c b/kernel/kthread.c index ecb719f54f7a..cfa6e1b8d933 100644 --- a/kernel/kthread.c +++ b/kernel/kthread.c @@ -35,6 +35,10 @@ static DEFINE_SPINLOCK(kthread_create_lock); static LIST_HEAD(kthread_create_list); struct task_struct *kthreadd_task; +static struct cpumask kthread_online_mask; +static LIST_HEAD(kthreads_hotplug); +static DEFINE_MUTEX(kthreads_hotplug_lock); + struct kthread_create_info { /* Information passed to kthread() from kthreadd. */ @@ -64,6 +68,9 @@ struct kthread { #endif /* To store the full name if task comm is truncated. */ char *full_name; + struct task_struct *task; + struct list_head hotplug_node; + struct cpumask *preferred_affinity; }; enum KTHREAD_BITS { @@ -124,6 +131,7 @@ bool set_kthread_struct(struct task_struct *p) init_completion(&kthread->parked); p->vfork_done = &kthread->exited; + kthread->task = p; p->worker_private = kthread; return true; } @@ -314,6 +322,16 @@ void __noreturn kthread_exit(long result) { struct kthread *kthread = to_kthread(current); kthread->result = result; + if (kthread->preferred_affinity) { + mutex_lock(&kthreads_hotplug_lock); + list_del(&kthread->hotplug_node); + /* Make sure the kthread never gets re-affined globally */ + set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_TYPE_KTHREAD)); + mutex_unlock(&kthreads_hotplug_lock); + + kfree(kthread->preferred_affinity); + kthread->preferred_affinity = NULL; + } do_exit(0); } EXPORT_SYMBOL(kthread_exit); @@ -779,6 +797,109 @@ int kthreadd(void *unused) return 0; } +static void kthread_fetch_affinity(struct kthread *k, struct cpumask *mask) +{ + cpumask_and(mask, k->preferred_affinity, &kthread_online_mask); + cpumask_and(mask, mask, housekeeping_cpumask(HK_TYPE_KTHREAD)); + if (cpumask_empty(mask)) + cpumask_copy(mask, housekeeping_cpumask(HK_TYPE_KTHREAD)); +} + +int kthread_affine_preferred(struct task_struct *p, const struct cpumask *mask) +{ + struct kthread *kthread = to_kthread(p); + cpumask_var_t affinity; + unsigned long flags; + int ret; + + if (!wait_task_inactive(p, TASK_UNINTERRUPTIBLE) || kthread->started) { + WARN_ON(1); + return -EINVAL; + } + + if (!zalloc_cpumask_var(&affinity, GFP_KERNEL)) + return -ENOMEM; + + kthread->preferred_affinity = kzalloc(sizeof(struct cpumask), GFP_KERNEL); + if (!kthread->preferred_affinity) { + ret = -ENOMEM; + goto out; + } + + mutex_lock(&kthreads_hotplug_lock); + cpumask_copy(kthread->preferred_affinity, mask); + list_add_tail(&kthread->hotplug_node, &kthreads_hotplug); + kthread_fetch_affinity(kthread, affinity); + + /* It's safe because the task is inactive. */ + raw_spin_lock_irqsave(&p->pi_lock, flags); + do_set_cpus_allowed(p, mask); + raw_spin_unlock_irqrestore(&p->pi_lock, flags); + + mutex_unlock(&kthreads_hotplug_lock); +out: + free_cpumask_var(affinity); + + return 0; +} + +static int kthreads_hotplug_update(void) +{ + cpumask_var_t affinity; + struct kthread *k; + int err = 0; + + if (list_empty(&kthreads_hotplug)) + return 0; + + if (!zalloc_cpumask_var(&affinity, GFP_KERNEL)) + return -ENOMEM; + + list_for_each_entry(k, &kthreads_hotplug, hotplug_node) { + if (WARN_ON_ONCE(!k->preferred_affinity)) { + err = -EINVAL; + break; + } + kthread_fetch_affinity(k, affinity); + set_cpus_allowed_ptr(k->task, affinity); + } + + free_cpumask_var(affinity); + + return err; +} + +static int kthreads_offline_cpu(unsigned int cpu) +{ + int ret = 0; + + mutex_lock(&kthreads_hotplug_lock); + cpumask_clear_cpu(cpu, &kthread_online_mask); + ret = kthreads_hotplug_update(); + mutex_unlock(&kthreads_hotplug_lock); + + return ret; +} + +static int kthreads_online_cpu(unsigned int cpu) +{ + int ret = 0; + + mutex_lock(&kthreads_hotplug_lock); + cpumask_set_cpu(cpu, &kthread_online_mask); + ret = kthreads_hotplug_update(); + mutex_unlock(&kthreads_hotplug_lock); + + return ret; +} + +static int kthreads_init(void) +{ + return cpuhp_setup_state(CPUHP_AP_KTHREADS_ONLINE, "kthreads:online", + kthreads_online_cpu, kthreads_offline_cpu); +} +early_initcall(kthreads_init); + void __kthread_init_worker(struct kthread_worker *worker, const char *name, struct lock_class_key *key) From patchwork Fri Jul 26 21:56:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13743191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3746EC3DA49 for ; Fri, 26 Jul 2024 21:57:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCAA46B008A; Fri, 26 Jul 2024 17:57:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B7A1E6B0092; Fri, 26 Jul 2024 17:57:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A41D36B0095; Fri, 26 Jul 2024 17:57:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 894586B008A for ; Fri, 26 Jul 2024 17:57:45 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 00CFE120184 for ; Fri, 26 Jul 2024 21:57:44 +0000 (UTC) X-FDA: 82383266490.21.2E0E5B8 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf28.hostedemail.com (Postfix) with ESMTP id BEE94C0006 for ; Fri, 26 Jul 2024 21:57:42 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jKF0qw8o; spf=pass (imf28.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722031014; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cP4RuhNJyr4ZKZ1KAeqILqsev/qbyujBCmDMEijOHM8=; b=onyRcyoGXKZxP1TFESbdbqsr7Sr4VBMGxW7fGO31+fZQZIFWcRlUKKpRq3pQ5mTSxD5u7M pJ+9fy0J5iHiT9FEGyob8o50v/GvHdZNJgQCyIvwzE8kkErQYWWPHQEAqubU49QP5ces/L AdYqqrMxNsXQgM1pnqMiYishUOEn+ZA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722031014; a=rsa-sha256; cv=none; b=tldJpxQet5xZhKi2jXLb+cihXfO2uBzow/vSuA7J7a7KyLWZ1fqiTo8wcMZ5WhsLNJeo3Q n2/0/G4ncV30m6QqPpSrrgaEOw45TsNGBVVCI64iKJZZ/f9vSqFedpI53bfCbAWQ648DcS /u+OU9pTQrm1cRWWjL+Sv3Z57SDfaVw= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jKF0qw8o; spf=pass (imf28.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 46ACDCE1951; Fri, 26 Jul 2024 21:57:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DFBB2C4AF0E; Fri, 26 Jul 2024 21:57:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722031059; bh=Rndu056GW69+sdbAKM3F/CfIuWiCWMVJcK3IHpS2rDo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jKF0qw8oA9DB/ULKFIjWqBbIOivCNKNnP+icJQt71lrPxYLykaWaAGlKghIpOZukx fb58LWldjmjvwQJ8wN4PuM2kjFU9SJrJ0EQAa5gu2Y6Y6vor0u0/bX79kSl5BiQouI izJpmQ21q4aeCG1vc1o7OLlVRd41dYSzwgoq6ZMqQj4V1yb4DnBw3ecF9leZKgYXsx vV6HUhfxUkd+67VNqXhahAUB071lXMNcM8UxLLi/e8aPVCHWvrijo96H21LzJW8fZY zUAm9K9yimYovCNXHgUrc0q0g0ZhyzvyFhcNRS66sqXsaIdUtmQc8VnJ1pN3JpEHYu zVLO0zCMmrnRA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Michal Hocko , Vlastimil Babka , Andrew Morton , linux-mm@kvack.org, Peter Zijlstra , Thomas Gleixner Subject: [RFC PATCH 13/20] mm: Make Kcompactd use kthread's preferred affinity Date: Fri, 26 Jul 2024 23:56:49 +0200 Message-ID: <20240726215701.19459-14-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726215701.19459-1-frederic@kernel.org> References: <20240726215701.19459-1-frederic@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: BEE94C0006 X-Stat-Signature: fuq63sbza1q3j9xkgfq1xe1b5yokcimg X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1722031062-433040 X-HE-Meta: U2FsdGVkX1+qzLv8KP8yYyJYoOtktoNy+sl86gCDc0ZwMxAkOVEGTnYjYDaOyIAXL5dcMCs41E047RjPL8fhZsPWCmAhRWMmmofViWkHmWvEGAT3bSnQzdBG1c4aCZdJaVfvg8D/KhIZwHNDWaml4nn7Zatmx+NI30sJaCWMWVcPXIvDL1r63xbzz7aOIlPr4tCxJwlWIbe3jEUq9gaz9i7CTy/+7HTv8VeJkD6BjH5eALv7JaSF4hWHlqkgTcJLQUJpTLiA5hS+2o/kjtRO0gzWK+ncNQMoJ9+lUpAAIqKwFfn0fhQGII88qny0jAXsXryVLbCD2KAPUS1HMQtW8vtXMYW5Bpd+CGWh48mMrc1i5J5erKD0h0F7z24e2U0J+UjUuDckXbzbvz6fIp+nIKz+CoLx0Io+s2kVvH8Sn4tTYx1JTGMqP091oQy26iSGll1pCflMUxMV9kwN9hOJM0DEmM0QYqEaeCcndCtvZirzALLFYoKbp2L4EDcH74sNFhQW6hxNC7EUQGc8l4gRHfCWed03HIxSt4G9KzCspzv/zKkHBUe0H/EV7hw+FAdWBhxYVzt5yklaPMnF1LBjFQYpbnk6kZ1DMh8AhlJBzHNf95knF5xQ21YH55f1sfLE7civC8qnmvPV1oGies0Kt9c5swsXSuEQAKMv/9sDpF6GGcB6PrMl4uJapEChLF1YU2l2G/9gEqms0jgfzJ1OSUMXkibAING7QMmw30I1Wv/beKhfsSpmBMIsNLy5AqbZJhArnjD47NeFvNoRwJ2+ZieuDtlnNutVWsmrLv77Cu02gWlALfWRVeZ4H0RhuWGGWWofG00yfMkuImoCogKi6jfh0sRfzso5o7ofpwUeW99ed6f91haYwpTCeBds05IatllWB/wcWV4VFueuae2VPkfR+z/LxDc+FHGl4/OrmGtCUHHaMomSqVgVp0Yr2qO1Lu/OSY55MZih+QTbWi+ COfHRaPh qozh6C+wle5oOKVfaae8qTQu4FZkllKHje26GE/uHJrLXMNzsIZFQpH3X+GoA8yTjbXNAK8ok4lGDhPMwWqsS4fCDGsUHTGLvuVJXlzbwYXTNz2m2KtEf5+o7qjW/uwS4mzdCW9+ZVdD/GkaBlElsUcOAy/zkVZxplAsrD6QXsN+V0OsP4WTqKizvDLoyYEzRKy+oU3tfZ+FNjWkzNRrw4cq7XmoZgn2AEI7aQZV62qUgdYM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now that kthreads have an infrastructure to handle preferred affinity against CPU hotplug and housekeeping cpumask, convert Kcompactd to use it instead of handling halfway all the constraints by itself. Signed-off-by: Frederic Weisbecker Acked-by: Vlastimil Babka --- mm/compaction.c | 43 +++---------------------------------------- 1 file changed, 3 insertions(+), 40 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 739b1bf3d637..64a6486f06e1 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -3179,15 +3179,9 @@ void wakeup_kcompactd(pg_data_t *pgdat, int order, int highest_zoneidx) static int kcompactd(void *p) { pg_data_t *pgdat = (pg_data_t *)p; - struct task_struct *tsk = current; long default_timeout = msecs_to_jiffies(HPAGE_FRAG_CHECK_INTERVAL_MSEC); long timeout = default_timeout; - const struct cpumask *cpumask = cpumask_of_node(pgdat->node_id); - - if (!cpumask_empty(cpumask)) - set_cpus_allowed_ptr(tsk, cpumask); - set_freezable(); pgdat->kcompactd_max_order = 0; @@ -3258,11 +3252,13 @@ void __meminit kcompactd_run(int nid) if (pgdat->kcompactd) return; - pgdat->kcompactd = kthread_run(kcompactd, pgdat, "kcompactd%d", nid); + pgdat->kcompactd = kthread_create(kcompactd, pgdat, "kcompactd%d", nid); if (IS_ERR(pgdat->kcompactd)) { pr_err("Failed to start kcompactd on node %d\n", nid); pgdat->kcompactd = NULL; } + kthread_affine_preferred(pgdat->kcompactd, cpumask_of_node(pgdat->node_id)); + wake_up_process(pgdat->kcompactd); } /* @@ -3279,30 +3275,6 @@ void __meminit kcompactd_stop(int nid) } } -/* - * It's optimal to keep kcompactd on the same CPUs as their memory, but - * not required for correctness. So if the last cpu in a node goes - * away, we get changed to run anywhere: as the first one comes back, - * restore their cpu bindings. - */ -static int kcompactd_cpu_online(unsigned int cpu) -{ - int nid; - - for_each_node_state(nid, N_MEMORY) { - pg_data_t *pgdat = NODE_DATA(nid); - const struct cpumask *mask; - - mask = cpumask_of_node(pgdat->node_id); - - if (cpumask_any_and(cpu_online_mask, mask) < nr_cpu_ids) - /* One of our CPUs online: restore mask */ - if (pgdat->kcompactd) - set_cpus_allowed_ptr(pgdat->kcompactd, mask); - } - return 0; -} - static int proc_dointvec_minmax_warn_RT_change(struct ctl_table *table, int write, void *buffer, size_t *lenp, loff_t *ppos) { @@ -3362,15 +3334,6 @@ static struct ctl_table vm_compaction[] = { static int __init kcompactd_init(void) { int nid; - int ret; - - ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, - "mm/compaction:online", - kcompactd_cpu_online, NULL); - if (ret < 0) { - pr_err("kcompactd: failed to register hotplug callbacks.\n"); - return ret; - } for_each_node_state(nid, N_MEMORY) kcompactd_run(nid); From patchwork Fri Jul 26 21:56:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13743193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91682C3DA49 for ; Fri, 26 Jul 2024 21:57:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF1F66B0093; Fri, 26 Jul 2024 17:57:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DA2D86B0095; Fri, 26 Jul 2024 17:57:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA4FD6B0096; Fri, 26 Jul 2024 17:57:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7BC186B0093 for ; Fri, 26 Jul 2024 17:57:47 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 29EE5A0169 for ; Fri, 26 Jul 2024 21:57:47 +0000 (UTC) X-FDA: 82383266574.25.BF679DF Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf03.hostedemail.com (Postfix) with ESMTP id 202A720021 for ; Fri, 26 Jul 2024 21:57:44 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EfaHtNlM; spf=pass (imf03.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722031063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W6L+GxnhanS+b0pfe5oO7GNFHU0ng/W46a1khHMCwF0=; b=zM6Op3X7lBq334dexXjcf2QlNihBfE2/FraLhSUlcSvhO1DVJa5hBWGxLfczPxjsBQ3y0o dj05rOsATjrlsIjnBBv4sUReb8GfOOEdiUtUwyevYJOnBILCTgcnCY1EoxO8RKjxoUr+Rl D9yGLVilESiTihOEFjOGzKuUAAmBYE0= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EfaHtNlM; spf=pass (imf03.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722031063; a=rsa-sha256; cv=none; b=duPOBqYSNFLawG8EG5STqzVbIwiEEXyxTvFOmSk267Gr6w5+8zUu6Y1gFiMS4LbxlJH2G4 KFthlMnMaT3maJ5LAxHR0iH2uDdh4WxNFjyGy/qLwXLKoTxJCkP/vpyCRFEPb+XhMRh4M4 ODTxuycn8MR6kexIjMn/YDfS45dLQwU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 9E7FACE195F; Fri, 26 Jul 2024 21:57:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4A550C4AF09; Fri, 26 Jul 2024 21:57:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722031062; bh=0SibR5bQwBKFRBO45aA49JEwgVAnlIbZLSKlUtRYW0s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EfaHtNlMZlMJahhsPYP3XpnKrqeJqsjfxxstfWQNrL9lYhrDvZtANMo274OaJx/WR DK06QKky13O60roB5A1PC+pKD3hZucY4vdLhblwxyPKDFOePStfUXAM/+oCpScsUF2 VQHCBe3uZD8wqdoKdHhmwfxBAD00C1S76vldsuXXYD/yILvkN86vgtKpOVHSbLjCIo ixfo2steYAN967ul7maL36yPU0W2mRJU0P8dmMoi6Vzt6t/ObrFIeg0/+AxrxQFv6U cmkFLHddcdM3frJXp9chc8owG7UMLfzeBceiOIUjFtLAUGktjLI922jIcKN8lmkBz4 /9K0TNVD224dw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Michal Hocko , Vlastimil Babka , linux-mm@kvack.org, Andrew Morton , Peter Zijlstra , Thomas Gleixner Subject: [RFC PATCH 14/20] mm: Allocate kcompactd on its node Date: Fri, 26 Jul 2024 23:56:50 +0200 Message-ID: <20240726215701.19459-15-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726215701.19459-1-frederic@kernel.org> References: <20240726215701.19459-1-frederic@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: nj98xybo4pc7md38bh3549rx3uzthd8p X-Rspamd-Queue-Id: 202A720021 X-Rspamd-Server: rspam11 X-HE-Tag: 1722031064-565728 X-HE-Meta: U2FsdGVkX188bJfYsfBUap1kxmyDpU5AiuNGuGL3uW0/oTCN4PQ6H37VBlETOvvDETWgSVglna97GkX5Gazytl64ghPuYTITZsL5QcnxjielTDMn7aCphk/7ADET9xL5EivaMu2YTTIJAG4MWeYRpIngf7dYo7DsYa8CiCBDblbB5Jeq78YvjI1+Vro4LFevClNWNxZBUSCT27hiPQUXXMqd+lPyOLgKpiQchyILKBG20ueCjPmSca5l1kxT9eWJkKNscUe/RiJNul02pIYYuI3Pw/xmMcEpqUPUlxAkh2CP0OGmWzRfhaQhaoMulsQVU5h7VDPY3Zb2z72ol8wRJQ9hFBPOkaQWZ4uu+iZg7IP4KtbXgxLmrnl+bZ85tSE6/ga/zxr0OaXCkaIJFzhswKXAl/bOrq4BWOcRQ9RC3UMW1isSgeIfKxpSOG2QQTCrrquGWyHP/63r3YJEW3KiFUtJ68UjYRMQfvd16ZHBfyyyz8YSxeu4rcsS4MFewy+TCVOt7+QhlcOWU9VMFR/qoWmBFxloWeRKzOETjFw1kmm6cZmd6HO6Vquii1VK6B3YfAV4K0/dElMl4nxToGznTLt/uJeBDlOzfZfleCoRnIgw9GQxpoaAMdXieJKqXDhU9JgMMUuNJQOc/nS/Soz2yqy07EqrbICu2Gn8da+5dXnynUMJzqI/W5aALxGNc5Z2klgFE7aHXuk/0zkrbPkdrgg+GN3n73Qv1XEKs29hQXhm4jpNcBcX3RCYCKGV+SeeR6ehcQnC44pO0N+7mSocZh4cI1Bt/f9q8ODa68VXwZewvV/mFvlTQeUuAcQ43wXSQrftoUXJowLnhS9UYO3DJ/vIlxvPEQwDwVFlM3b9hYj3UQEjbHuVPkQaR1rll4AGDAdagq2dadGBDNxYLIUakD/fuaKar2bXNPWcRpT5pdvAoTHdCYr14spd6cWe8P/Z/x/zCNEhZs5wO9VbyD4 5sEBIy2m FMBNqGKOtH9y3zhbPwHvaaKicCSc5gx23b08On4iqK7ni3u0FdW7bOegR9VVOHiE0XmSSOf6dAPRvV0WdkUHeQlQkLmBj8CehnAb75cw4SlPnAAf1cOoSmC75puPkLgzVbPqsmInOEGi2WsE+fVvInmEtlZN1V8nhIM4Ps18hTmvnoczVy8nCCMLEJUJVxGPqoEUvNbdy4eq+6ikRvs71ZO84ktgW9PnVuYW+dAPZSKPAKRI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kcompactd runs preferrably on a specific node. Allocate its task structure accordingly for better memory locality. Signed-off-by: Frederic Weisbecker Acked-by: Vlastimil Babka --- mm/compaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/compaction.c b/mm/compaction.c index 64a6486f06e1..dc4347c4e352 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -3252,7 +3252,7 @@ void __meminit kcompactd_run(int nid) if (pgdat->kcompactd) return; - pgdat->kcompactd = kthread_create(kcompactd, pgdat, "kcompactd%d", nid); + pgdat->kcompactd = kthread_create_on_node(kcompactd, pgdat, nid, "kcompactd%d", nid); if (IS_ERR(pgdat->kcompactd)) { pr_err("Failed to start kcompactd on node %d\n", nid); pgdat->kcompactd = NULL; From patchwork Fri Jul 26 21:56:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13743194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDB0CC3DA4A for ; Fri, 26 Jul 2024 21:57:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6EDE46B0096; Fri, 26 Jul 2024 17:57:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 64CE16B0098; Fri, 26 Jul 2024 17:57:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A3696B0099; Fri, 26 Jul 2024 17:57:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 262E46B0096 for ; Fri, 26 Jul 2024 17:57:50 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D62D4C0169 for ; Fri, 26 Jul 2024 21:57:49 +0000 (UTC) X-FDA: 82383266658.27.1E459B5 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf19.hostedemail.com (Postfix) with ESMTP id CE2A51A001C for ; Fri, 26 Jul 2024 21:57:47 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=iqgrHSmz; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722031027; a=rsa-sha256; cv=none; b=sHDT2dmo/9FWR+DowAjjce9/VeR5QVLeLG0Ef9zwT+wMlbK+OG3eGV498RX4SaPZbN+J79 Iq9vvbQ2RDsJYoC0Yjx5KadxtIfSHfIajqlO2VLjjoaRYg689Ike32+hzn7LBk2cEPJH5N Wc1D5JAh622g0pnDcnxCx31UzqPWR64= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=iqgrHSmz; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722031027; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2plyUun0w48eTm1VlsywmP9butfRi8OOoMDte60dZ3Q=; b=aJSqwXsc22daCUfFXiPg/wwBO6qQUPkqpnfqgErJDfGc0/1iI0ridycaDPlrJaxXgFd6Zt E/XViTFrk/gatjDTa+APGJEQ+KTSlv/om+YzMa6GDC/Ox1Ks70JGmv4RNTyK8Lr/M4mDVv kGrJPrPqttOx22eY7ixBFFQSbV+iDe4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 48A12CE1940; Fri, 26 Jul 2024 21:57:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A2B33C32782; Fri, 26 Jul 2024 21:57:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722031064; bh=gPMXvX+GlymTegeJ33I/1BOjjt7hvEufmzbmhRPnpxg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iqgrHSmz8YA7EVyJFrBUszMUuoULJREzG0WmIVQaPwZgshXH9zPEIRF+Y8MrjnWkM Zt3wRLuZMuDHLaUl7mHZVZJJebIAnhF3ewfY12xrp+pMS3rOrGHuwiCIrXgDtdgbpo fqrvDWzHarZ5T4rMmD8B0dQWcCrjj7zT4ofXxfLUT1/BcPwCJnP2kMJVcshvr4xbwW xdMDv2GNTaRv6yTwhplpRpErKEak9nfL5A3iL7FO8eM/NlgFldnD6GIsJ4MQTFVPMn svskNbqqQ3C0B6v8p/pu/8VrsrkvDTA2qxa0iiKGjB9B0mJJXmZ3Z1wJndxKQW2vdE GB5zT8LjtgUBg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Michal Hocko , Vlastimil Babka , linux-mm@kvack.org, Andrew Morton , Peter Zijlstra , Thomas Gleixner Subject: [RFC PATCH 15/20] mm: Make kswapd use kthread's preferred affinity Date: Fri, 26 Jul 2024 23:56:51 +0200 Message-ID: <20240726215701.19459-16-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726215701.19459-1-frederic@kernel.org> References: <20240726215701.19459-1-frederic@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: CE2A51A001C X-Stat-Signature: 1a161cxaw8jfh43p69fkou8sks77yrik X-Rspam-User: X-HE-Tag: 1722031067-472832 X-HE-Meta: U2FsdGVkX18UxjCABxjP+tFJbfvD8np6VKalCGS7LKAF10iQf+00LxphUJT8J5f1gx5M8vdxQJK6IjsbY7AEIjMXE8B9zaNAk95kmwknaNZtScxFydik8aBUoIVa54edjOG1nfBOv6HDrc/JS/zshOcqoBlRw4jiOOHQO17hOMx1+H4UjE2tpUwAl2JMjdh5lmxEDyFdBZyF+1P0YPewvX5by33vyOlIhHhsvYbzflo4DDfQMithDTag504cwdmmzGQcPImuMIzkFvfUM5QSDKiJqFOXZ5d7xsFsYwzXIygmw3ybdmfr7M8BtLlBnIKpH2zRkMPhwAK+GyMZnZOAFgADgjF/mxCmoyZBKu6MpkXPMyQRHii9CgDYsZ+sgsLE6kQIwBVBlukWoTkudxMFw5X12fZx+TN6RW7T3pMv0ee8EpCuA8FfpipmxxsQr8NbWeQbU9X7UzaqdGaKqNH0Ot9Eswfdh9Ppm3Iimo+a+L4cbaFQrKhX9+xWGpb/PfoKE3smeH3Cv1UPHowmp/c4s9LBJ/yHtLKhtbar0zBaWekeqjEqJMqkf1lNMQ6l2At1ave2748f3mIWrKJWNWSIPSsGiG+/0+j1qeZQYLiX0d56yYB2yeok5abfoU+WHXn/CUkkiVsTdjz/1TmgqfiBzJMX/lVMCI8N4aXlD6PTJBflBA8h5TziqMWwE/SCz1lGqVE1VQgKV3g74ldsj4yBu8zCXEINq14eZbi8ZlVUbplBl3LGhH8yIx1RNxTpHe4XT9bnMEJZRhnQkPzWMmHQZn6n3FDG8/J5hUgrFZdv2Ps+APIxTprjGS/fJ5aFqh+VCePzAgyWVowGjBoF3HK0pSvay1dRMpycC8qKNaRiqQpfJPiYFYegIUFg6rWHYD1KVfXeQfa+pxUJIA7A0yM+U4xX9tLeeWIZiVsECMO6qvts4iTRaJzbmosvmHkSYKUz7+cZeiS57fjvKVydL/D gieSVavM daCeswrLibvgkwb8S6rQcoryTg36yO4vE1Wdi9FBXILnvmZSWHH19iOSCZ+DRJQ7Qd+QpvYUwjCbTWQTvyn2K1EKVNg+12VQBcO559yRknAqJYyKexhrrTp0iygBbu3jQHPttRZjsm6hLqH755t7rKEqorA//lUjmGGC/Q/0AxRStcS2HcD/grq919VpxYNAX4jGVgITnewoP52VRtRkeX0F2fo2qFl154g4GLt0WGrQp7KWSbaCa4jg+26wUFsZ75hztuPEGOCNQyzc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now that kthreads have an infrastructure to handle preferred affinity against CPU hotplug and housekeeping cpumask, convert kswapd to use it instead of handling halfway all the constraints by itself. Signed-off-by: Frederic Weisbecker Acked-by: Vlastimil Babka --- mm/vmscan.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 2e34de9cd0d4..94359a893b4f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -7116,10 +7116,6 @@ static int kswapd(void *p) unsigned int highest_zoneidx = MAX_NR_ZONES - 1; pg_data_t *pgdat = (pg_data_t *)p; struct task_struct *tsk = current; - const struct cpumask *cpumask = cpumask_of_node(pgdat->node_id); - - if (!cpumask_empty(cpumask)) - set_cpus_allowed_ptr(tsk, cpumask); /* * Tell the memory management that we're a "memory allocator", @@ -7288,7 +7284,7 @@ void __meminit kswapd_run(int nid) pgdat_kswapd_lock(pgdat); if (!pgdat->kswapd) { - pgdat->kswapd = kthread_run(kswapd, pgdat, "kswapd%d", nid); + pgdat->kswapd = kthread_create(kswapd, pgdat, "kswapd%d", nid); if (IS_ERR(pgdat->kswapd)) { /* failure at boot is fatal */ pr_err("Failed to start kswapd on node %d，ret=%ld\n", @@ -7296,6 +7292,8 @@ void __meminit kswapd_run(int nid) BUG_ON(system_state < SYSTEM_RUNNING); pgdat->kswapd = NULL; } + kthread_affine_preferred(pgdat->kswapd, cpumask_of_node(pgdat->node_id)); + wake_up_process(pgdat->kswapd); } pgdat_kswapd_unlock(pgdat); } From patchwork Fri Jul 26 21:56:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13743195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59439C3DA49 for ; Fri, 26 Jul 2024 21:57:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B09706B0098; Fri, 26 Jul 2024 17:57:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A6A3C6B009A; Fri, 26 Jul 2024 17:57:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BD0E6B009B; Fri, 26 Jul 2024 17:57:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 61EC16B0098 for ; Fri, 26 Jul 2024 17:57:52 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0AB0FC0173 for ; Fri, 26 Jul 2024 21:57:52 +0000 (UTC) X-FDA: 82383266784.18.EB21B60 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf26.hostedemail.com (Postfix) with ESMTP id EA93A140014 for ; Fri, 26 Jul 2024 21:57:49 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HNJiSszU; spf=pass (imf26.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722031019; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N2wDk2Gcd9+BquURlVjDlsA41ANtNBXVgDegm9qEKUc=; b=2DWCSePiXEkmqwaLI2voeY32yNw1zyl0ef3OEb5N22JqYNjXIZisOmaCi0jN3btXhpFqpt qBEbt2NA3BeedzVHsofjp3d9TJelIJMlAOUCsTu+oOpsnZQO0gRPGvDawQ8lGVwlAm2a9v oacnl/Io1e7hylrbFq6faIhMkZYOVqo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722031019; a=rsa-sha256; cv=none; b=PF8Ci4TQl0YLbjyTgQuGiteuafQtUq4tfrmuynXIomtGRLE4ygKz0D8HGVmHJIxlCb4f9E lMVhdg2oS13OzasfTpYdCFHtcWe2hfAkOgprr6vVbj6wjM7gAL00BpHRZ1d2T/tRWIZeuI 4OI58DnEdEPu81Ycvcve216MMpEzvBs= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HNJiSszU; spf=pass (imf26.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 64454CE1953; Fri, 26 Jul 2024 21:57:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ECEAAC4AF09; Fri, 26 Jul 2024 21:57:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722031066; bh=xSTwda2GjtC4wyMIao/A441KBqg4odJRHj3hK2x1g+0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HNJiSszULYtzhaeEVutUpqJehUbmWxSB6jq8VWOWPfAZdbHMxbimJX7PAPIKb+rCX W52rkj7eh6tAwyR8729dwXxsywgY8xzA8VBFr+v2kTGuMnpKDvsv1NhQBNyZXe5t0+ 01bV5vKyjTeG1oCaskaQOhyWG0qKuvAVy5laatvcHYtQ5BlQE8ER+P3CHL5deVon20 /J3e2MFPcvAexLrmDbabvPFF2loz01wv2FpsqHyCNxkF/0bpaB4P0ISCX/bAd7HSex ZUsqxDXFzSeoxk0tsvawONxFpR1xpWqnzoXRtkGhxX2Kqohw7My6WB+nP6z0KMxwOp LSSFCfUmNrGAQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Michal Hocko , Vlastimil Babka , linux-mm@kvack.org, Andrew Morton , Peter Zijlstra , Thomas Gleixner Subject: [RFC PATCH 16/20] mm: Allocate kswapd on its node Date: Fri, 26 Jul 2024 23:56:52 +0200 Message-ID: <20240726215701.19459-17-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726215701.19459-1-frederic@kernel.org> References: <20240726215701.19459-1-frederic@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 65859e15csroqux8be4uhtk5d94twr3s X-Rspamd-Queue-Id: EA93A140014 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1722031069-261216 X-HE-Meta: U2FsdGVkX18rzQNEKaTO3uD027igt5YKftpNtWHtYb55vY6gtd7GZrxcCAbiJTnqvMTt2m/A9D5P9PIYitI1o+eF14IaJ/lug0ACgWEBRlHdOEfe1OEsQ47oiHUlxeMFKwjjLTWA0FNApuQvKfOi+DdDpSIOa7v7ZbjoMpLJcOc+7ST/8TGbWmL0XmSbuktdI9Ed4ElvzR3R4koTTW50MVlaHUk2KX/ZpqCT2NjbS3dc4cpj+vEWyigVxVoemQcb2tHE66cfqRR8Fk7XCf3hKdw4Jn/iEFcwk0ayPKNxGPl3co0a5Pfaon/w3i2+/buja1toNkz2iidtCnchwdLWEawgNGKd/Abj/6Jgp9t5K5ZKAZZShz0ismR5uPv+Pi/v4QO4OGoaajyLfvClqD6FYeZ3EOjNrXuKWgPVNoDfq1Ks/k93/9cCtVp8bDm9W7toWf8gX9Vwyu7WMwAA3Sndy/RIlndLbkU0lpNEjjjPR0lENTen3kPPvlE+7YQPHByfJkQX+f2aXQCTplZYe4VIE5nsWtmr/DNuxLiEPVBru/AcFVhSXm1QsYSFsKZnWSQHPpJqhGnCSqWwZQdBydqFMnltHI8JL3o+mZ0v5JdOkoShQieNwQu+7w0NHtEMwR7CtP6QjFKwJjJSkZ9UP29i6ZlSxj0uIyZWHfn1XN+MeHWkMMx7Z29pwxUFugM5VPWvpQ09hPPLsHl74Qryt2KNbVoyUmU+8pEW1ZopOcjymQH39L58pBWiRL5b2/wkQuVsmBRG/7R9df6jKEs70fXGqhTwquJ+t3I8W4W7Ls2EQoi9Fq9XKQVhfUdXlJECTOS4NRMzfCn7ozAyRcu/isbmgA2bRDgZq9SG7whX+lwWhIFK29M24eBF0YDU0Oa2+KHYoDvZ4M1HeC6lGTsufvXE9xVLbs9bBQhHnkcDZCTwKvQEWiSIrRyORyIHJ6cWRzk1MmoGMXO8x5qIIx3FVZ1 TD5SKk1z GAphRdO0rD5GF5yFZufTA+GufITMjyA73FhE+J0sIrIC48ABsoFKIlHKkt+EeT2tlHMnbidIRqnTpmDQEx6GcCxyCdfK3FI6Bfi6gORXPLnEO2NtaZCZw4+8ySSNV/y5pLXgsT6iuTFiYACr6R9cYkB4ptlh7DXE0zqhJorQP5Za6hkbVfPbxhYrVXB/e6iI9opnbyQmnzkI5HRUdr6zaHB+2i0Qg+oLwDAvDHEFHBaVyr19Ax/EG5IgIbtxdxy8aJ44JB777Fdup0vI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kswapd runs preferrably on a specific node. Allocate its task structure accordingly for better memory locality. Signed-off-by: Frederic Weisbecker Acked-by: Vlastimil Babka --- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 94359a893b4f..adf8c1e7e89d 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -7284,7 +7284,7 @@ void __meminit kswapd_run(int nid) pgdat_kswapd_lock(pgdat); if (!pgdat->kswapd) { - pgdat->kswapd = kthread_create(kswapd, pgdat, "kswapd%d", nid); + pgdat->kswapd = kthread_create_on_node(kswapd, pgdat, nid, "kswapd%d", nid); if (IS_ERR(pgdat->kswapd)) { /* failure at boot is fatal */ pr_err("Failed to start kswapd on node %d，ret=%ld\n",