From patchwork Wed Apr 21 10:51:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12215969 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57C8AC433ED for ; Wed, 21 Apr 2021 10:52:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D25E961447 for ; Wed, 21 Apr 2021 10:52:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D25E961447 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 57E756B0070; Wed, 21 Apr 2021 06:52:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 555346B0071; Wed, 21 Apr 2021 06:52:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CEFD6B0072; Wed, 21 Apr 2021 06:52:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 245206B0070 for ; Wed, 21 Apr 2021 06:52:04 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id D713C181AEF2A for ; Wed, 21 Apr 2021 10:52:03 +0000 (UTC) X-FDA: 78056059326.27.91D9760 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf16.hostedemail.com (Postfix) with ESMTP id 8CC3980192D8 for ; Wed, 21 Apr 2021 10:52:02 +0000 (UTC) Received: by mail-yb1-f202.google.com with SMTP id k5-20020a2524050000b02904e716d0d7b1so15203528ybk.0 for ; Wed, 21 Apr 2021 03:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=OxgBhA98k3tGthKxuSN/4brjg9G3Dacp3flVNK8rvYo=; b=kZCrucRDppiGDBPjAXIIXIo0MxE4/IyqxJJhXwKZnRZitxeGk0n/gU50Zw9DaWrpD+ dIpvk0jbhDyEE//vafgmDxLrgr2YyYm66YH+SiaJKfmQozMx5XcaeO5hf4eWzECEKjp0 DelqluZ8nf+mHKY0XVzdVwGk8ty9BSH78j0/6K/wFDufY5AfFSbPeh8A4xraPkTh886n 9Z4kUhLBAsqyKAVGsv02WcAeZV3sEBGaDgqRuEJcNLLQHQPf3JIiEw3lCSUgcKZvdDXG xeUD6XPpxzPQXIngH07/btqdDuRqSxVDtooRU6+D+Q1pVBCH9B7GNwaUSnOrJcrjCtc4 Zjng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=OxgBhA98k3tGthKxuSN/4brjg9G3Dacp3flVNK8rvYo=; b=oN4p76FYRKWRlScPK/hi29fwTr+7gp2IYaS19YU8lehophHFBweT+FF3dS2GTr7Vgl PNoQFmK5yoaV0r/GeCCVQrUHq+NBzUSsGsJxGXKNqv1DXbYqQY2/1zrO6SrIf57gc/tn HJoBnlgHF3ZDK+ezRIPYHAw2sngmj00N0n36sM7LJm0gJRXhueCYCzTQ19TQHveMTeKV VYvVOoDIEYVqnd8LY0fcSEYjbiROlJXd2dKEoHwxlcOhVGFXoe8MnWNRE2w0Ui+mddF7 mBI9i/x/p6QVLd6kBwWvapAAa5skgNXpnYWbKQ4DcgqVvA4ndB+337qECcHmQxniQ+AC FOpw== X-Gm-Message-State: AOAM531N1u0cBOpeC4CXypj8zbwoeVTdx74zbTOpYJkhQCft1+04+beY CVuRc3hBQ2/MMBLxY7tR0nZB4fr9gA== X-Google-Smtp-Source: ABdhPJxnxD3CB/9+5BCPqRdLBDuWnvcCjjRpgFvxXOW0RsfmOBJPVhGilgvJZ/ty/H3/94vwogLNxmc1JQ== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:c552:ee7c:6a14:80cc]) (user=elver job=sendgmr) by 2002:a25:8b86:: with SMTP id j6mr29548944ybl.470.1619002322474; Wed, 21 Apr 2021 03:52:02 -0700 (PDT) Date: Wed, 21 Apr 2021 12:51:30 +0200 In-Reply-To: <20210421105132.3965998-1-elver@google.com> Message-Id: <20210421105132.3965998-2-elver@google.com> Mime-Version: 1.0 References: <20210421105132.3965998-1-elver@google.com> X-Mailer: git-send-email 2.31.1.368.gbe11c130af-goog Subject: [PATCH v2 1/3] kfence: await for allocation using wait_event From: Marco Elver To: elver@google.com, akpm@linux-foundation.org Cc: glider@google.com, dvyukov@google.com, jannh@google.com, mark.rutland@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, hdanton@sina.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 8CC3980192D8 X-Stat-Signature: 418ag1uiixh77ijy8n4kbf7witb5f6qj Received-SPF: none (flex--elver.bounces.google.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from="<30gOAYAUKCOMJQaJWLTTLQJ.HTRQNSZc-RRPaFHP.TWL@flex--elver.bounces.google.com>"; helo=mail-yb1-f202.google.com; client-ip=209.85.219.202 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619002322-690153 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On mostly-idle systems, we have observed that toggle_allocation_gate() is a cause of frequent wake-ups, preventing an otherwise idle CPU to go into a lower power state. A late change in KFENCE's development, due to a potential deadlock [1], required changing the scheduling-friendly wait_event_timeout() and wake_up() to an open-coded wait-loop using schedule_timeout(). [1] https://lkml.kernel.org/r/000000000000c0645805b7f982e4@google.com To avoid unnecessary wake-ups, switch to using wait_event_timeout(). Unfortunately, we still cannot use a version with direct wake_up() in __kfence_alloc() due to the same potential for deadlock as in [1]. Instead, add a level of indirection via an irq_work that is scheduled if we determine that the kfence_timer requires a wake_up(). Fixes: 0ce20dd84089 ("mm: add Kernel Electric-Fence infrastructure") Signed-off-by: Marco Elver --- v2: * Replace kfence_timer_waiting with simpler waitqueue_active() check. --- lib/Kconfig.kfence | 1 + mm/kfence/core.c | 45 +++++++++++++++++++++++++++++---------------- 2 files changed, 30 insertions(+), 16 deletions(-) diff --git a/lib/Kconfig.kfence b/lib/Kconfig.kfence index 78f50ccb3b45..e641add33947 100644 --- a/lib/Kconfig.kfence +++ b/lib/Kconfig.kfence @@ -7,6 +7,7 @@ menuconfig KFENCE bool "KFENCE: low-overhead sampling-based memory safety error detector" depends on HAVE_ARCH_KFENCE && (SLAB || SLUB) select STACKTRACE + select IRQ_WORK help KFENCE is a low-overhead sampling-based detector of heap out-of-bounds access, use-after-free, and invalid-free errors. KFENCE is designed diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 768dbd58170d..235d726f88bc 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -587,6 +588,17 @@ late_initcall(kfence_debugfs_init); /* === Allocation Gate Timer ================================================ */ +#ifdef CONFIG_KFENCE_STATIC_KEYS +/* Wait queue to wake up allocation-gate timer task. */ +static DECLARE_WAIT_QUEUE_HEAD(allocation_wait); + +static void wake_up_kfence_timer(struct irq_work *work) +{ + wake_up(&allocation_wait); +} +static DEFINE_IRQ_WORK(wake_up_kfence_timer_work, wake_up_kfence_timer); +#endif + /* * Set up delayed work, which will enable and disable the static key. We need to * use a work queue (rather than a simple timer), since enabling and disabling a @@ -604,25 +616,13 @@ static void toggle_allocation_gate(struct work_struct *work) if (!READ_ONCE(kfence_enabled)) return; - /* Enable static key, and await allocation to happen. */ atomic_set(&kfence_allocation_gate, 0); #ifdef CONFIG_KFENCE_STATIC_KEYS + /* Enable static key, and await allocation to happen. */ static_branch_enable(&kfence_allocation_key); - /* - * Await an allocation. Timeout after 1 second, in case the kernel stops - * doing allocations, to avoid stalling this worker task for too long. - */ - { - unsigned long end_wait = jiffies + HZ; - - do { - set_current_state(TASK_UNINTERRUPTIBLE); - if (atomic_read(&kfence_allocation_gate) != 0) - break; - schedule_timeout(1); - } while (time_before(jiffies, end_wait)); - __set_current_state(TASK_RUNNING); - } + + wait_event_timeout(allocation_wait, atomic_read(&kfence_allocation_gate), HZ); + /* Disable static key and reset timer. */ static_branch_disable(&kfence_allocation_key); #endif @@ -729,6 +729,19 @@ void *__kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) */ if (atomic_read(&kfence_allocation_gate) || atomic_inc_return(&kfence_allocation_gate) > 1) return NULL; +#ifdef CONFIG_KFENCE_STATIC_KEYS + /* + * waitqueue_active() is fully ordered after the update of + * kfence_allocation_gate per atomic_inc_return(). + */ + if (waitqueue_active(&allocation_wait)) { + /* + * Calling wake_up() here may deadlock when allocations happen + * from within timer code. Use an irq_work to defer it. + */ + irq_work_queue(&wake_up_kfence_timer_work); + } +#endif if (!READ_ONCE(kfence_enabled)) return NULL; From patchwork Wed Apr 21 10:51:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12215971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 428E4C433B4 for ; Wed, 21 Apr 2021 10:52:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BF1AA613FB for ; Wed, 21 Apr 2021 10:52:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BF1AA613FB Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 46C226B0071; Wed, 21 Apr 2021 06:52:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4201C6B0072; Wed, 21 Apr 2021 06:52:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E5A96B0073; Wed, 21 Apr 2021 06:52:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id 143EE6B0071 for ; Wed, 21 Apr 2021 06:52:07 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C27A4362C for ; Wed, 21 Apr 2021 10:52:06 +0000 (UTC) X-FDA: 78056059452.07.E07D01F Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by imf17.hostedemail.com (Postfix) with ESMTP id 9427740002E1 for ; Wed, 21 Apr 2021 10:52:03 +0000 (UTC) Received: by mail-ej1-f74.google.com with SMTP id jl27-20020a17090775dbb029037ccdce96e6so5698492ejc.21 for ; Wed, 21 Apr 2021 03:52:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=S7gKkVqw6gJJUkQX3/NKZ5hEYF7HuP5mcybwHhcoGA8=; b=BKLqZcruEVs1tQhMgavhobMYH3pkurizK7K0prlTtT7UrKBFybRppZodySbtpHep4Q dahuNpp+xKHhDa+OyyRnw8e2sf/QI9FxUpG+uLlUXHrf2chSK+ctL7VnE3D26Z4yixzt aqrvmWwUE9nVzIqIDx963uaNsvyOloDReUzMky+nBXxdQBFjLZuyH05HxCwsqKB2cnQ3 4mGy0+AnDgPt4VsmVV67b2DM0GvFf9qXOzs3jKk56lYF0E6l2UPZ9yg+vySmVVouOi+d ZcuQyL1tcho+GmMD5X26B1MJaXgBGDD2AkS/iWonncxyQv+YGdXrwBVcfiHto03hxFSk 50Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=S7gKkVqw6gJJUkQX3/NKZ5hEYF7HuP5mcybwHhcoGA8=; b=l3HQNRFcFA6wgCi240C5qCanvudtC/PfnPkKOfUC3FBW5qMKmd3QptConZ+d8o5rLa CIzzJmOTIF7cwFexUX70WaZHsIWKLBFjv34/40aQwVNOE7jtXC8eo00gxZqz+z6JWxdq rR71cQwu0epLhwl5R42rUMr37uzlVLDF+CvnRDYni6DGBZmkpVwU/YZ1vubhKdOoVkyS KFLENzMG5z/zjpsYGlRzz+RMKtoSdfrdtbjVS399AHoIOfc7nPBkZAEXmvY3S7+DH3Wh 2r/ztaxayhwVJ7QQPb1j08MNwob9d9gKk1uOtiRFliEppZuPErqGBicSiHnopn+P7NFP t0wQ== X-Gm-Message-State: AOAM531ArpcXwneroK8D9roHvvH1p8i545WZ2MTEIJ+KopBEu2ImZa/j 0PTlZcCVqygYJs2XbK+DF02OFLLa8Q== X-Google-Smtp-Source: ABdhPJyjCmPDiQcNUfvf1eIuCw4ql8YdC3rYLPuX10VxgctCoYcEuZQUMCVPDf2UQQgpokCHE10ey+Upqw== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:c552:ee7c:6a14:80cc]) (user=elver job=sendgmr) by 2002:a17:906:3d41:: with SMTP id q1mr31542346ejf.282.1619002325029; Wed, 21 Apr 2021 03:52:05 -0700 (PDT) Date: Wed, 21 Apr 2021 12:51:31 +0200 In-Reply-To: <20210421105132.3965998-1-elver@google.com> Message-Id: <20210421105132.3965998-3-elver@google.com> Mime-Version: 1.0 References: <20210421105132.3965998-1-elver@google.com> X-Mailer: git-send-email 2.31.1.368.gbe11c130af-goog Subject: [PATCH v2 2/3] kfence: maximize allocation wait timeout duration From: Marco Elver To: elver@google.com, akpm@linux-foundation.org Cc: glider@google.com, dvyukov@google.com, jannh@google.com, mark.rutland@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, hdanton@sina.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 9427740002E1 X-Stat-Signature: hrweeb34jgetuhu39w4rsangsgn84wik Received-SPF: none (flex--elver.bounces.google.com>: No applicable sender policy available) receiver=imf17; identity=mailfrom; envelope-from="<31QOAYAUKCOYMTdMZOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--elver.bounces.google.com>"; helo=mail-ej1-f74.google.com; client-ip=209.85.218.74 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619002323-194433 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The allocation wait timeout was initially added because of warnings due to CONFIG_DETECT_HUNG_TASK=y [1]. While the 1 sec timeout is sufficient to resolve the warnings (given the hung task timeout must be 1 sec or larger) it may cause unnecessary wake-ups if the system is idle. [1] https://lkml.kernel.org/r/CADYN=9J0DQhizAGB0-jz4HOBBh+05kMBXb4c0cXMS7Qi5NAJiw@mail.gmail.com Fix it by computing the timeout duration in terms of the current sysctl_hung_task_timeout_secs value. Signed-off-by: Marco Elver --- mm/kfence/core.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 235d726f88bc..9742649f3f88 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -621,7 +622,16 @@ static void toggle_allocation_gate(struct work_struct *work) /* Enable static key, and await allocation to happen. */ static_branch_enable(&kfence_allocation_key); - wait_event_timeout(allocation_wait, atomic_read(&kfence_allocation_gate), HZ); + if (sysctl_hung_task_timeout_secs) { + /* + * During low activity with no allocations we might wait a + * while; let's avoid the hung task warning. + */ + wait_event_timeout(allocation_wait, atomic_read(&kfence_allocation_gate), + sysctl_hung_task_timeout_secs * HZ / 2); + } else { + wait_event(allocation_wait, atomic_read(&kfence_allocation_gate)); + } /* Disable static key and reset timer. */ static_branch_disable(&kfence_allocation_key); From patchwork Wed Apr 21 10:51:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12215973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FC3EC433ED for ; Wed, 21 Apr 2021 10:52:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1823D61447 for ; Wed, 21 Apr 2021 10:52:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1823D61447 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9764E6B0072; Wed, 21 Apr 2021 06:52:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 94CC26B0073; Wed, 21 Apr 2021 06:52:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 814E06B0074; Wed, 21 Apr 2021 06:52:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0087.hostedemail.com [216.40.44.87]) by kanga.kvack.org (Postfix) with ESMTP id 68CD86B0072 for ; Wed, 21 Apr 2021 06:52:10 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 2A2288248D51 for ; Wed, 21 Apr 2021 10:52:10 +0000 (UTC) X-FDA: 78056059620.03.DE9B7E8 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf30.hostedemail.com (Postfix) with ESMTP id DD9C2E000105 for ; Wed, 21 Apr 2021 10:51:54 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id l18-20020a0560000232b02901026f4b8548so12497952wrz.10 for ; Wed, 21 Apr 2021 03:52:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=958qgNOMMQHBKiudXbOhYcKipML0hl7ML/vSp0CTCqA=; b=e4RwU/x5IzvdYtTruaVeAg8cfueMN5BJshXP8pv9Q+HnHtMBceFhACbcGE3soTOgpc trfZnV8dj77WSwGo0mkof3682PcrXAe+cVxI+/m2NWIC1pub3IEPhtD1qmNQ7jjfPkCS T76lyZLUX2V04rWtEHo/QifvHBRgKzwvNsL0sd4zjsNvAmOr+HvyE1kX/KeinDOfBIgx ENPqRwZKBIE0UTLq9iWBGJje4ZMwFlI1DZog/3pA2vE55gaIkzs+mmQ196mp8PYqEcfq 2/b/UCfHKASERuiv1Os5WDJ1UVqNfafFoz+HPmG/8SG6cvVkPY8Ixp0S0ando9EZoGua Bg8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=958qgNOMMQHBKiudXbOhYcKipML0hl7ML/vSp0CTCqA=; b=JUuUopfzoAaNuGiIG9C7TQD8F/LSnHl4ZoVIQ5rgNnhbanFU5zwrePTulPqsBWOpVX WpkFoh1vOFjHMLOcYGRnzRw4Ugewmv6hhtp4WQlvj5Ijlojy4u0+IO5M3c0UWNRB2IvE fJEyhCO8MHwzbe2OpLncOpQwch1zHgIYk2OuZdP8t620IX/yOJb8QIRCYZ1X9FtTWqWa hrfnwwYvRVNOulk79h3xpHypAUiLh857KPXGENhqR62seSkr5zh4Ha5hVO9h1dk8Y64W 05fvTl2r0UK//KQykDV/4jSHvR2doSFh7qiIj/80wp74ElY/gPFBsjDmzZPgV0/WHbXx YZ5g== X-Gm-Message-State: AOAM533qpTRyQZHBsrS4zMksj9UpIzDl4FCRz5gyAf2x/wntFcXM3MW5 Wi7532C5bWiEjhTF8pKNWMkz3L3IYg== X-Google-Smtp-Source: ABdhPJwBXkyuJWMZa2OUSRCCtP5kyDXU8thulTzdIfrwDxtxhQDYmMHY0T5AhyrTNdE1RBS4CvoHbhoO6A== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:c552:ee7c:6a14:80cc]) (user=elver job=sendgmr) by 2002:a05:600c:4fd4:: with SMTP id o20mr9476694wmq.166.1619002328581; Wed, 21 Apr 2021 03:52:08 -0700 (PDT) Date: Wed, 21 Apr 2021 12:51:32 +0200 In-Reply-To: <20210421105132.3965998-1-elver@google.com> Message-Id: <20210421105132.3965998-4-elver@google.com> Mime-Version: 1.0 References: <20210421105132.3965998-1-elver@google.com> X-Mailer: git-send-email 2.31.1.368.gbe11c130af-goog Subject: [PATCH v2 3/3] kfence: use power-efficient work queue to run delayed work From: Marco Elver To: elver@google.com, akpm@linux-foundation.org Cc: glider@google.com, dvyukov@google.com, jannh@google.com, mark.rutland@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, hdanton@sina.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: DD9C2E000105 X-Stat-Signature: efgetmuomrq61zipowbijxxxrr17knmx Received-SPF: none (flex--elver.bounces.google.com>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from="<32AOAYAUKCOkPWgPcRZZRWP.NZXWTYfi-XXVgLNV.ZcR@flex--elver.bounces.google.com>"; helo=mail-wr1-f73.google.com; client-ip=209.85.221.73 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619002314-230399 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000217, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the power-efficient work queue, to avoid the pathological case where we keep pinning ourselves on the same possibly idle CPU on systems that want to be power-efficient [1]. [1] https://lwn.net/Articles/731052/ Signed-off-by: Marco Elver --- mm/kfence/core.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 9742649f3f88..e18fbbd5d9b4 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -636,7 +636,8 @@ static void toggle_allocation_gate(struct work_struct *work) /* Disable static key and reset timer. */ static_branch_disable(&kfence_allocation_key); #endif - schedule_delayed_work(&kfence_timer, msecs_to_jiffies(kfence_sample_interval)); + queue_delayed_work(system_power_efficient_wq, &kfence_timer, + msecs_to_jiffies(kfence_sample_interval)); } static DECLARE_DELAYED_WORK(kfence_timer, toggle_allocation_gate); @@ -665,7 +666,7 @@ void __init kfence_init(void) } WRITE_ONCE(kfence_enabled, true); - schedule_delayed_work(&kfence_timer, 0); + queue_delayed_work(system_power_efficient_wq, &kfence_timer, 0); pr_info("initialized - using %lu bytes for %d objects at 0x%p-0x%p\n", KFENCE_POOL_SIZE, CONFIG_KFENCE_NUM_OBJECTS, (void *)__kfence_pool, (void *)(__kfence_pool + KFENCE_POOL_SIZE));