From patchwork Tue Nov 19 15:34:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36336D44173 for ; Tue, 19 Nov 2024 15:36:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF4EB6B00A3; Tue, 19 Nov 2024 10:36:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AA5716B00A4; Tue, 19 Nov 2024 10:36:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 946166B00A6; Tue, 19 Nov 2024 10:36:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 780726B00A3 for ; Tue, 19 Nov 2024 10:36:25 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 0B674404EA for ; Tue, 19 Nov 2024 15:36:25 +0000 (UTC) X-FDA: 82803245784.02.3551D44 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf24.hostedemail.com (Postfix) with ESMTP id A9940180028 for ; Tue, 19 Nov 2024 15:36:16 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WB6MWHnN; spf=pass (imf24.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0gaKqKMKlmwETWNmgCjaiOWIlMUIKbL1+CgbCwZ0tnA=; b=oEbLFVs7zLXfOpNesbl9bYG/RGfHE79ItGVIQHOd/z+ALFRrv3nX8kcZE2sKopd4C44JWK B68Cv4qr+JYzkss0OhPJN5vWEYXuTQSlSjljD7IaWQ3l/AIbGIoWo9QeM84Ol0ftaEMkbG +QBVnaYiQiJf3QJ+/AStYJ3UeMHEyZ4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WB6MWHnN; spf=pass (imf24.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030401; a=rsa-sha256; cv=none; b=inmhCB3MwfORG69n5G1d1mlf07vhCMoaAyVpIC/UmrjHFKB1QATXFXdB0CUhfqT1IpjvpZ 7/OV/cPwjadW+C9nQdk39rmHm5r97RhUffXqEoPV58maw9GnITl68BCZC/6FpUiGDXFckX vqz3VQ2O4RTIxxe5pxNRnd56SYBDgkI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030582; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0gaKqKMKlmwETWNmgCjaiOWIlMUIKbL1+CgbCwZ0tnA=; b=WB6MWHnNcoxJMnVTpGYGqvNKvOTrW5ZXaCrAsWog1v/9uGHRrfrh3/eeAarcHsUvOiWUig 3+RBpXDZjKHCjHFT0yaWxAMe2cfrzSQco4W/q/Dr50uix7y754bvWGaRQ18H1ZBLRYOOJM ySwmM6xA44jk3t1r9ckutyRSeuIkR9c= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-533-9WS2Ju2dPyC3J89ZBnjLFA-1; Tue, 19 Nov 2024 10:36:15 -0500 X-MC-Unique: 9WS2Ju2dPyC3J89ZBnjLFA-1 X-Mimecast-MFC-AGG-ID: 9WS2Ju2dPyC3J89ZBnjLFA Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8A8FD1955F2B; Tue, 19 Nov 2024 15:36:07 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 70AC63003C80; Tue, 19 Nov 2024 15:35:51 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 01/15] objtool: Make validate_call() recognize indirect calls to pv_ops[] Date: Tue, 19 Nov 2024 16:34:48 +0100 Message-ID: <20241119153502.41361-2-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: A9940180028 X-Stat-Signature: fpkrswozfeau8n4ub49se15b9bagbyb5 X-Rspam-User: X-HE-Tag: 1732030576-580538 X-HE-Meta: U2FsdGVkX1/d+bHCDeqyLGrJVp4nuGwaW1qXPVnnpcIVFigrQPfxxeLGEk9yFIAF4wXkAPMcx9hvnK54gExRuBBlD1Q2vz/YZt2Xur0vF2EoPo89smtePdW0aWja3Qo6aRY8i3nm9aYZStBNPSiLWD6RJZDoGsUIdEOkGiPZYxG1DBK5dPZCrlJISmZmaHy1lxBWTl3pSSiSuzbcs1mh2BYnQkqs0Eq4cpuZcjG6Rc7jYQZSr25Ph0Vwd6mnBOXKaQDlymnhtmzCsiE9GmxrfnEXRUJ7pd/sxR3CJsE94VXcIeUpRN8EE6mioMwAJfBsQihAmGw5XqWwMKLiFM3IxRHXOX/s55D+Vpr5A1GxphpBoxdgKEySo+9KhoWkH75nT0euIev5d1adZJTB0kVaYp8XZjLF6zpOkEAuzvmBFBIGN3qxH+RUvg+Ad7j/xok1wUErdodr6JGePAKoXq/0VPR56a3napc1/HWSXnVLBMBj1ST4qTQi6sHRnRA+1AYMWVUL8Cl2A5FXoO4d/ustHxwpcsz547UYjs+uCh7929g21J3DImpFez8+pwb6q+6+aNBBBKpHgpidFg4fD03x0TjzyoPPTQ/oXJKGfVPE6n1aHQhI2JbskgtdHeu13uJNIjxb7rxMnAl4j3Rm6p+C3dbKiism6ZPEZj6lxfRI1emC91aV/CWX1hbKM0Bb3yvp6YBkuJEug2SLYcvMZ6JtiNz0AzVguzDyctS1LGn90FNeZcFsb7M8vG2e3lH8Qek/sZNXszwSUGteBRzJvbbjX7WTiEdLUFzYfWgg7x1VPrXWm7jwb/5u9rN/Hbpzk2h3RwC2dncTUYuRzu9nXXcCEvKGy7jFBxV5BZi3iSqtYv7UjCiPss6RJG9zXAt+vi/8I8dA8NyQFFDtcJEeIvSZSctoOGqHM5XROY6ahRuXhpjb9vt9NJFPe125Q6JylkQVpUU5sXy1OR2YJTMUV86 mFypTG6T NJ4V4hMLpIcTCISVIOm1MShNFZyFzeSPmFHa5zh96EalkXoa4kjXqwCOLpLMAliusi/86s8h277AnhDZNmsqD1rgjS/Os+MjKv9OVhiEIAU86Lg/TFb0narWITMfy8Oq9j8/kio1jtKPPlyoXV72eov6ojFf6MJyFtR0iYNMnF6zty5RdL+CBuY7XBamKigcss/TR5T97gLJFOdm7/WGrtydwkT8lBjfhsgDWBYv0AjiJezCSYOIdFd8wRXpVDzescAGoOYL9FIiS9Hwo4pIqs8rojRjAjMPuXA7trun+eD9v2vQMb58qrifQjfNsuzAreTpYzxAipi47iflqjiyDdFUJGVX2Z7azCVnUjOuXggviKBV/hleGZswedFlLbDho6uKcc4mtBOZ6nqW1b0Y35nmG9+ByDzUTTNOcDwPKDwB8M5heKHyKT9CurvEN7vhrVynrZX+YfI5iHAswzw5MmhateoFqBHAqZlIHrgb3xDj1oUAyW6f/CZXmuA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: call_dest_name() does not get passed the file pointer of validate_call(), which means its invocation of insn_reloc() will always return NULL. Make it take a file pointer. While at it, make sure call_dest_name() uses arch_dest_reloc_offset(), otherwise it gets the pv_ops[] offset wrong. Fabricating an intentional warning shows the change; previously: vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to {dynamic}() leaves .noinstr.text section now: vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to pv_ops[1]() leaves .noinstr.text section Signed-off-by: Valentin Schneider Acked-by: Josh Poimboeuf --- tools/objtool/check.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 6604f5d038aad..5f1d0f95fc04b 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -3448,7 +3448,7 @@ static inline bool func_uaccess_safe(struct symbol *func) return false; } -static inline const char *call_dest_name(struct instruction *insn) +static inline const char *call_dest_name(struct objtool_file *file, struct instruction *insn) { static char pvname[19]; struct reloc *reloc; @@ -3457,9 +3457,9 @@ static inline const char *call_dest_name(struct instruction *insn) if (insn_call_dest(insn)) return insn_call_dest(insn)->name; - reloc = insn_reloc(NULL, insn); + reloc = insn_reloc(file, insn); if (reloc && !strcmp(reloc->sym->name, "pv_ops")) { - idx = (reloc_addend(reloc) / sizeof(void *)); + idx = (arch_dest_reloc_offset(reloc_addend(reloc)) / sizeof(void *)); snprintf(pvname, sizeof(pvname), "pv_ops[%d]", idx); return pvname; } @@ -3538,17 +3538,20 @@ static int validate_call(struct objtool_file *file, { if (state->noinstr && state->instr <= 0 && !noinstr_call_dest(file, insn, insn_call_dest(insn))) { - WARN_INSN(insn, "call to %s() leaves .noinstr.text section", call_dest_name(insn)); + WARN_INSN(insn, "call to %s() leaves .noinstr.text section", + call_dest_name(file, insn)); return 1; } if (state->uaccess && !func_uaccess_safe(insn_call_dest(insn))) { - WARN_INSN(insn, "call to %s() with UACCESS enabled", call_dest_name(insn)); + WARN_INSN(insn, "call to %s() with UACCESS enabled", + call_dest_name(file, insn)); return 1; } if (state->df) { - WARN_INSN(insn, "call to %s() with DF set", call_dest_name(insn)); + WARN_INSN(insn, "call to %s() with DF set", + call_dest_name(file, insn)); return 1; } From patchwork Tue Nov 19 15:34:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74D8BD44167 for ; Tue, 19 Nov 2024 15:36:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 096386B008C; Tue, 19 Nov 2024 10:36:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 046C06B00AA; Tue, 19 Nov 2024 10:36:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E01E66B00A6; Tue, 19 Nov 2024 10:36:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C06406B0089 for ; Tue, 19 Nov 2024 10:36:43 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 79C1E1405BD for ; Tue, 19 Nov 2024 15:36:43 +0000 (UTC) X-FDA: 82803245868.15.E5DC0FE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 9EA1F14000B for ; Tue, 19 Nov 2024 15:36:08 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=icBZySGT; spf=pass (imf23.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030420; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XIn2epAo4ei/z52bV+OQ9JOU7pzaMsMZcOjGajJfL0A=; b=TtAG1fEevUy0eR7vQcqkBDSNkHvJJU6YFa/dAOcgpEy86BUj9mRkNLldoPtVTzfLHHOuDp 6yqLV14+3GrPJNJC4q0ZWVRts00M1/WUB991VJc2uPBBMhF2indvW+tVHgxuMX7+afNXeE 8HIBvCMDFLqgGyAx72/SubeL89Fp8lg= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=icBZySGT; spf=pass (imf23.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030420; a=rsa-sha256; cv=none; b=IPpRJ+wkt0D4eHnZE+6MBm46Gg3/X4wpUaPszeRgwYZ5TyVdSrnWUJHtAGv6Ofy9GmfD3y n93eNEM+BTR6wak7E/eDSMYlPOe3tLhcX0U+HD6tr6EWRO3lnYA8wSSe0Zh8QIypSn7K2t ppSmM8ib8Th/GLLe+qNI/Chfkahkh9A= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XIn2epAo4ei/z52bV+OQ9JOU7pzaMsMZcOjGajJfL0A=; b=icBZySGTRPZc2I4JP+m8YNd4oDU1365AzdmKJIzkyymUcm3miXMYNV37s5mIPxmsByogUL Q1s4ib+vWjdABWd5UWrXEml2fF0SWd2bcMjiVxjYySJkfqpgoL5ovMEWqxO41QdaB8AtX7 ixs1h4u2PNAwArDa9Rosq6W7fli9W0o= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-182-C9wx2u1EM12YXIqHhclsig-1; Tue, 19 Nov 2024 10:36:36 -0500 X-MC-Unique: C9wx2u1EM12YXIqHhclsig-1 X-Mimecast-MFC-AGG-ID: C9wx2u1EM12YXIqHhclsig Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0A1A61956046; Tue, 19 Nov 2024 15:36:24 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 06AD830001A0; Tue, 19 Nov 2024 15:36:07 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 02/15] objtool: Flesh out warning related to pv_ops[] calls Date: Tue, 19 Nov 2024 16:34:49 +0100 Message-ID: <20241119153502.41361-3-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 9EA1F14000B X-Stat-Signature: u8mf63aaed3hof1dnrupy6osy53o1uud X-Rspam-User: X-HE-Tag: 1732030568-128216 X-HE-Meta: U2FsdGVkX1/0/VlyHHcEEHwX2AvzLKDT20/KRBXLSLNmQbMVlf/9dXNLYmpfk6ieEEERw38bCjgzzj+ZsR1Q71ZMaWd2C02hHkwIBrbad9K4qRce/8ZGu+Ah14Y8iGfvaeiMzFKKkYU92H+bvthyenSdUU4RqCHqk2t6lWSWkQNOirjEQruUSlQmLgi8rcOOBdqjHPnDIBhMU8reBTbOjCxlEYfuMkrXmb9d/DSdWn3yHIDNzhgRUxUg5K1HjwKJxSNwz0TZDv4WBXTWvkM2WB1WPJNqMeJvsSsLLLFZlcdpLAXI8wDvI2k7o7KkKrEWHZoKdT8gFEERXOa4mg5Nt2bf+lHnipZTJT9pP4Vg/ap+QTYI3+4VepfYOM3FVvJWl+/yX4TIBXaE+bdc3usxQncsUvHbSt/fxDm2mHHHMDYf8OJGaottSQFRPPHn3p0OPEZm836TDaWNPjEeBF1ICR/884NGCGTLa7+qDsxYQCZNAhR9hSLNFlhcWKup79VLY3rCWb+/8aZCFncWxRitjCOYlieEnyUKnPZPVf7H7mi32GrcU5b4d1rMXpcK5l8Ng4DKgkkGiBUn0R/tDLLrayoT3je1BfBqwifWoiZ8nLrigJuAlgUhUoqOulDubrn5FSYOVgrXyZxUGbldGRrkD37AD1gdsmZmWLid0Qr4DpmRWWib9M+ysNFSpKHAjjaVoIWKQhNB/5ePNFT0Zgi3GoUMDilKXbuS8D1adpTwk8OJIEdQv6FYLp0+WZwaUYwkdFAlixZxAhyKZsjm6TZTE+Qe7dQJJvI9o4lL3Umzb47vKRCsY4F8OqSS11AxIFgT9OJDHUw8bvvo36H4/43OD9CldwIX6wewWQO/vRlkbuOMCUT9SDHE8q1CFkieGAdNzFKL3Ly/f1Yf92QRp7pdczzBX0aUalmq5i3gvP7u7hWI5uCT53xu593JajwE8ZsEvP+3GrAMx5a/eQZI1BO c+SWUEEd py/TnPg7x1o9ugb64b+79Ioh2daEDac9xkdpLJ3dObnotXFvv1+85voXjRaUY+cqhLYwT4AJCaYNpL7F7R/MF21puUk/b/Pw8Zppe+7djzvG0vOOhIRjXnVOaPnSZXuV4dmo3RINIvu/geN/SrKlfXgMS1+4tUl+qWpAnQw/Q3r7X+Y0pKP5nzrW/lojTNhS8HKtLAseq7HBr5SA4+OdZB46QUWgm+mz/lFS/cs5q0ZKqy19BznIw80WvcrYQ/0kK5bNO8m0ZIcD/Pm0P2mYg/bQwzjdsYPTl7wRZlDdvQK7rIQOx4b+X//3lCeuIp+FR1j3KVcjtBqrylixpizXsBey2gpmLlW/21+5GKVzbVAGd/ih5fsjsDWkxXJGFj+4aXTu+QqXS2j0rybG65Ek5lJhbuW/q9Oz3jrnOrTd/m1xrbhXYOQ+QSKw6QTqqm9JFxqmWbiIweClqaGoiYg8Ylk6qCQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: I had to look into objtool itself to understand what this warning was about; make it more explicit. Signed-off-by: Valentin Schneider Acked-by: Josh Poimboeuf --- tools/objtool/check.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 5f1d0f95fc04b..00e25492f5065 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -3486,7 +3486,7 @@ static bool pv_call_dest(struct objtool_file *file, struct instruction *insn) list_for_each_entry(target, &file->pv_ops[idx].targets, pv_target) { if (!target->sec->noinstr) { - WARN("pv_ops[%d]: %s", idx, target->name); + WARN("pv_ops[%d]: indirect call to %s() leaves .noinstr.text section", idx, target->name); file->pv_ops[idx].clean = false; } } From patchwork Tue Nov 19 15:34:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DF6FD44173 for ; Tue, 19 Nov 2024 15:36:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C588B6B00AB; Tue, 19 Nov 2024 10:36:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C06CC6B00AC; Tue, 19 Nov 2024 10:36:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA70A6B00AD; Tue, 19 Nov 2024 10:36:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8B0F16B00AB for ; Tue, 19 Nov 2024 10:36:51 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0E564AD970 for ; Tue, 19 Nov 2024 15:36:51 +0000 (UTC) X-FDA: 82803246330.18.64477D7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id A65E4C0007 for ; Tue, 19 Nov 2024 15:35:56 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UL1MQIqu; spf=pass (imf28.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030549; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vzSk9ZzswXx0ijJl+1BL1pcjLYKU11Wq/W2Kyfj2Pb8=; b=GX+Sg7armHrIt70tQ/PeqRe4zje8wEwIXiAfHmnWLz9+JuWXTJBxHiahJLjI3aDB5ZvO5I YHmOGHmYOy5FzMAQP0Y18+Vi9S+V0yHERglOX9mbePCxejHkG4nvPyXN2NIhlvvOCrpW9r HNoJ/VW2m4UxklKCqdwwLMZkggsPuFo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030549; a=rsa-sha256; cv=none; b=hWULwvWnsBP0bJS3evEXNT4deINnITl1OjHzF0MsNO/+Z/+PqlJdG9KzlCW3vKZBCH6jc6 ZZSNEii+HKUwPN69r9JYj3/w/TaFa1/JDJGt+XbKPAiJhomkFZ0t/2+KVSMQWX+vjj7FWU qT0iNDp1Fj8R1wL1IPjzf7tKSVHano0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UL1MQIqu; spf=pass (imf28.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030608; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vzSk9ZzswXx0ijJl+1BL1pcjLYKU11Wq/W2Kyfj2Pb8=; b=UL1MQIquosZYdn73IQw/SqV9MwjQxXFmiadO4JzRnLrM80/SzgZjSLaIrxqn5bVhGx04XJ CDhL495xUjrum2gCdWt0joa67B/mghnqzazsvpkEvpRUK9c2ohKHzwV7ueUCrY/CSM6sxw 1WJjL/eKqO0vELF91cHqGWLbxBJcB/o= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-301-mg-v22g6NKyICZb_as9TEg-1; Tue, 19 Nov 2024 10:36:46 -0500 X-MC-Unique: mg-v22g6NKyICZb_as9TEg-1 X-Mimecast-MFC-AGG-ID: mg-v22g6NKyICZb_as9TEg Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7B5951955F34; Tue, 19 Nov 2024 15:36:40 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 77F8D3003B71; Tue, 19 Nov 2024 15:36:24 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 03/15] sched/clock: Make sched_clock_running __ro_after_init Date: Tue, 19 Nov 2024 16:34:50 +0100 Message-ID: <20241119153502.41361-4-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Stat-Signature: 4z6t67s9ie68ubyx3f5dkhfhjfu84ehf X-Rspam-User: X-Rspamd-Queue-Id: A65E4C0007 X-Rspamd-Server: rspam02 X-HE-Tag: 1732030556-591917 X-HE-Meta: U2FsdGVkX18jg1YbmdaUIRygWgiRDKsiooi+Jd/AQw8tNdfSTvvCQ02lQc8n3U0ZKHO0GJpJGp3V+b39PRtGtWzFpWz9IVAB8pOc376y4li2N8OwPEYmYlE7jo/C9dOJ7WdcfreduAC48lR9V2q6QJrDPayrl31va4+AfxCXKMmIVEdzDG6D5jOSZqePNr9wFL34WopdCsTweHtOiRwqe6X41lweyDk4z60z+/2pigY5HM4DoKTAyQkTIvIhF+DMtfZbJeldDjkIzifW0KpiF3YQqK1ldV+q8/l3NdHe7p7sc1OTdp5Sbc3IXL5I4Uu6L+huoIe9UHPJUbQ9LCPeA9aQr7gQNOKt/aj6RxO6BXbY0QK6sVNOcEjRYmrqr0Zb44ShxxzkiFHMAkJyTWR8KjIAQVELCKIuASkrirsYRamcz5vATDyyig3HRbendGexqlp3DhHxq7mG+09TK6wblbY3wV9mfZY18dsqLg255cxfQbreKXcfcxycFv+YGHwmakHa9EfmWpMfTQGSIuCJJDjw1RDWAmB0I2p5Z+gL7Uf3aI853zGFbDOsvp1yccE3floa4ByzkDUcq4bdtjm1OPYmpLSo4xPjdRGhmRalJXjHtm+IkHPwjY+IUZNWJuakV6StXN9z08ffCJpgYl27r43OtrKRGnD5SWXOfhfQivaKTh1iw4KOfir+KFnOwoDmVVRAtbKsLrKaHWdvdAc8e8sqZEKCzM6bncW0/YJc7cJ+IwxA3QOrBiVjpldd0a06Dby9FCBdifYaL9dvMv8/g5eX9cKeKMig83SapTcE1GCWmNFMb1A/idEEQBO0fjoHk00MmbPPoM9nf3/b1p11Q5qPOpuf2w41Tan+nU1Eyoh6c7VqOXbQlIlschlnHSfyohlleJ+/kHewcUHnABxLdaOq2sav4QMdTtaxE5QMFNA4+4D/IZy3Z8VPRF9lEd9x6HsnFhWnUD2yQEIgbTZ jwGDBJpE Wwp00qzcijEAMBaBNJwDMp2fCui/ff9PL6a/mlpQPivyxMVvXLzXIJCqwlY9JW2+Vzs9s5thqwksG/NRbf1CeEy/0P9QNZn6yIkEKKB3ZtSnfTdxJfO4TxdlhkIxtqUD8az0w0RrWbAgAxLaJZARFD1QvFsxFu9BNIQsj5/HWY69Aj9gvPSY+YvYDd6CLl+BXCHthE/xOK0Vww9iTvzUiqNwwWUXecKx3LtnP82Y2GESXV8FkuaG29kkb80psA22Ajg/+qr/TY3LIk+ywwXefvHvm4WJIAJWs4dD5cnqNU0e5vkhPeHKqrU0h/NBbuU/5vEyftYBrcMcP1+jZijipSzataZhZZexS2IFDZFhYmwJac/9vK5vxTzZXv+Sg/eKVjyzQ0+O9ZwX9jfiLKoZXuZunx/CNuf5KWzNljqpW7FZDewykuYYDfsHl6qp5EKbV3JH7d5NfRVoKFjWidUq3TtSKEQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: sched_clock_running is only ever enabled in the __init functions sched_clock_init() and sched_clock_init_late(), and is never disabled. Mark it __ro_after_init. Signed-off-by: Valentin Schneider --- kernel/sched/clock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c index a09655b481402..200e5568b9894 100644 --- a/kernel/sched/clock.c +++ b/kernel/sched/clock.c @@ -66,7 +66,7 @@ notrace unsigned long long __weak sched_clock(void) } EXPORT_SYMBOL_GPL(sched_clock); -static DEFINE_STATIC_KEY_FALSE(sched_clock_running); +static DEFINE_STATIC_KEY_FALSE_RO(sched_clock_running); #ifdef CONFIG_HAVE_UNSTABLE_SCHED_CLOCK /* From patchwork Tue Nov 19 15:34:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CDB9D44167 for ; Tue, 19 Nov 2024 15:37:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D41566B00AD; Tue, 19 Nov 2024 10:37:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D18FA6B00AF; Tue, 19 Nov 2024 10:37:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB87B6B00B0; Tue, 19 Nov 2024 10:37:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A0BF86B00AD for ; Tue, 19 Nov 2024 10:37:08 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4A3A2C0567 for ; Tue, 19 Nov 2024 15:37:08 +0000 (UTC) X-FDA: 82803246540.10.E361367 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf06.hostedemail.com (Postfix) with ESMTP id C1A38180015 for ; Tue, 19 Nov 2024 15:36:30 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IZ1ztEqy; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030535; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9/zjMAAzjZ63CvtK65oto4Glzya8ix3n4q04/5M16qM=; b=O6SsdwvHIMpVJ3xJiAwm1VV31UzBzvxOknsHA+ldp32wPEWWpXU7KNfuOmBW1XFw1yrVDw yt8pTfrYVW96FxS1Ai93rTK3d2ax7qOBc5F7aFEsQGLgTE+aMG9R4hF0/pdnlmEwgBsPq4 6MphdTZuS/ioSpEwtfmwTxN4Uo5lHdg= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IZ1ztEqy; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030535; a=rsa-sha256; cv=none; b=Janc5yxdAYwJA/C/Pm46GIL6VYBtw4haCuUIk7dI9dKSR8HExpLvS05SXjw1ewRqadyp+w pkweQiPzXL9sM2P3FiDtBQZAfW3zqpXem3H3QVPbmSBjBPiCiv//Iy9XdfXRHMC/RE84KO DDVNWwrtCpFWgvUht0YedHfjio3KNOs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030625; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9/zjMAAzjZ63CvtK65oto4Glzya8ix3n4q04/5M16qM=; b=IZ1ztEqy7+gIO7UlsThLnZzckIHQzW4ij0sGD8snjOkq5709KKOoREf4Y3ra8vp0SopQTd ma7gXv7s5aYpnqYG9BkKoZrGqHcAonBU7I60sYSox9nKfbflMHT3TBC0mN1EWRXl/MYSfO SvL4sJIfO7cbueQDKp6Y35bMdITl7uw= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-668-nbrc9rTzMP-Bp1uCFPcarg-1; Tue, 19 Nov 2024 10:37:04 -0500 X-MC-Unique: nbrc9rTzMP-Bp1uCFPcarg-1 X-Mimecast-MFC-AGG-ID: nbrc9rTzMP-Bp1uCFPcarg Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C4EB41952D0D; Tue, 19 Nov 2024 15:36:56 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EC7F330001A2; Tue, 19 Nov 2024 15:36:40 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: "Paul E . McKenney" , Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 04/15] rcu: Add a small-width RCU watching counter debug option Date: Tue, 19 Nov 2024 16:34:51 +0100 Message-ID: <20241119153502.41361-5-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C1A38180015 X-Stat-Signature: 8m3d85wzgkmxjpf7x583qb855ttz9tj9 X-HE-Tag: 1732030590-523650 X-HE-Meta: U2FsdGVkX19NWeBQfZV70TxeNZFSChNyEe4RsN3XkSr3kP4ilqLjXhAfZqPGyNqD3T19csFUk14ABzL2DTPS+n0llXVzGKs9uUX3E9+6oF8eAsLx/2nSdVhvlodcJKfoxUupMFGd0s6rhezxBIJvyJofRbjaI3hm1QQ4MelPC0iEM0ZFpmdLKgmkK4GPhjuXIeJT1TpCCLN3uTqmY01A//juTZtzdUbzCy2Xp0V4pJZYgh7A+rHFrbXbBw+xqYQXWua1C9RY4os6yp8edsQSthUMntn86g2dXwJDVsVvnTkFUciORdNmz5gi9sn3G52oDUtrRZxnRNxgcznhe2Wz+Y1EYxjTf55A7ODTa3vG4eWj3yisiDw/ebvBaeExzfTTgNt8+XZO9fL1upx3TU7bHJVEQBn18dJkSd4wV+Z6ncAHQktt2cnPevRasifEThnmBsJXazFd0fpAhZGFtKR7mysjDY5e6Xl1C1GC1PCCNkpikI7kDePwKE70z+L8sJ1XVKnaYmLWU2eN2T10irlsm/8kNRwl4951cVM1yH5AdsJbbB9P95yRTyM4DtX7lAR5ArW/X9HJjO0gtU1d5oIV1/YlqWDZcppUKVxa1ARD/svPZD66YO0Ep4dAfi87AUIuLSsgkqMuJtr/+L/ePrZlzLqaI/M2z4pc/zNgt7CQPhSEeM6syG2tgTv+RXcqYA4QAkI3JcqSOk7bogjvoiqq5br1389Oimva/tMxR+6fGYo2zllkvAL3rLzd9HICVcceHGzPryqt1fR8+u13RNjX4WJDgRp8zXeSnQwUjD8vXI/oKS/eC+dBjpBLr4bN3xR8I7hNpYMVbviJkvQatW1CkgqL+Qhe4ToMHgVVQC2SUkuhC+Z5UfPFp3+IJpb0ktBG273VPBP5mstabAaFTLagKwwRs+mQ5kDlNjlX/yjMSM5nL7OSr5egSADp+jST9dcDvSGIrXnZ1EKvBNKIAIm DdQLZDpm LFH0rhE4Fb5qdmMfldTysG0oUevvK4m4bEaTuoJh4+7iua6wOrG9Bh1onCKYqeV96eGP5feFP/hGNN9+XYveUgzUkT9hkqd/CnoAUMkQ7t3005NARtBsoS3nKHkk0RhG89HbilqTSSXy5xrWeF8Pw6YOZcunrRJAEjqOVKu4e0QKDhK1NKb3wt3N0fJhaEpCutfblrKugrzZXKa1hYCph7Y4ESwNeoOx9x66vCJaZbBwBqTo6Cl3W03CW4+6RUDM0cqvXa4+UPPM0pRa5EZazK4hapwIvL1PoZ1hwN3P1sFq5x/FVPLAOvmspcyPPKGtvkpzRT1nUVNm7y11Ei3EpQbQdkYqHdYEGqTFGgPWfnmG6y3MJqcYFpDKr5C25/wmL9HXIobThdc99qZ/hJzVSJ+C2C46rWSc46d8u96M7IyI4iAy0+WcqO40tO6WiA6esPoOZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A later commit will reduce the size of the RCU watching counter to free up some bits for another purpose. Paul suggested adding a config option to test the extreme case where the counter is reduced to its minimum usable width for rcutorture to poke at, so do that. Make it only configurable under RCU_EXPERT. While at it, add a comment to explain the layout of context_tracking->state. Link: http://lore.kernel.org/r/4c2cb573-168f-4806-b1d9-164e8276e66a@paulmck-laptop Suggested-by: Paul E. McKenney Signed-off-by: Valentin Schneider --- include/linux/context_tracking_state.h | 44 ++++++++++++++++++++++---- kernel/rcu/Kconfig.debug | 14 ++++++++ 2 files changed, 51 insertions(+), 7 deletions(-) diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h index 7b8433d5a8efe..0b81248aa03e2 100644 --- a/include/linux/context_tracking_state.h +++ b/include/linux/context_tracking_state.h @@ -18,12 +18,6 @@ enum ctx_state { CT_STATE_MAX = 4, }; -/* Odd value for watching, else even. */ -#define CT_RCU_WATCHING CT_STATE_MAX - -#define CT_STATE_MASK (CT_STATE_MAX - 1) -#define CT_RCU_WATCHING_MASK (~CT_STATE_MASK) - struct context_tracking { #ifdef CONFIG_CONTEXT_TRACKING_USER /* @@ -44,9 +38,45 @@ struct context_tracking { #endif }; +/* + * We cram two different things within the same atomic variable: + * + * CT_RCU_WATCHING_START CT_STATE_START + * | | + * v v + * MSB [ RCU watching counter ][ context_state ] LSB + * ^ ^ + * | | + * CT_RCU_WATCHING_END CT_STATE_END + * + * Bits are used from the LSB upwards, so unused bits (if any) will always be in + * upper bits of the variable. + */ #ifdef CONFIG_CONTEXT_TRACKING +#define CT_SIZE (sizeof(((struct context_tracking *)0)->state) * BITS_PER_BYTE) + +#define CT_STATE_WIDTH bits_per(CT_STATE_MAX - 1) +#define CT_STATE_START 0 +#define CT_STATE_END (CT_STATE_START + CT_STATE_WIDTH - 1) + +#define CT_RCU_WATCHING_MAX_WIDTH (CT_SIZE - CT_STATE_WIDTH) +#define CT_RCU_WATCHING_WIDTH (IS_ENABLED(CONFIG_RCU_DYNTICKS_TORTURE) ? 2 : CT_RCU_WATCHING_MAX_WIDTH) +#define CT_RCU_WATCHING_START (CT_STATE_END + 1) +#define CT_RCU_WATCHING_END (CT_RCU_WATCHING_START + CT_RCU_WATCHING_WIDTH - 1) +#define CT_RCU_WATCHING BIT(CT_RCU_WATCHING_START) + +#define CT_STATE_MASK GENMASK(CT_STATE_END, CT_STATE_START) +#define CT_RCU_WATCHING_MASK GENMASK(CT_RCU_WATCHING_END, CT_RCU_WATCHING_START) + +#define CT_UNUSED_WIDTH (CT_RCU_WATCHING_MAX_WIDTH - CT_RCU_WATCHING_WIDTH) + +static_assert(CT_STATE_WIDTH + + CT_RCU_WATCHING_WIDTH + + CT_UNUSED_WIDTH == + CT_SIZE); + DECLARE_PER_CPU(struct context_tracking, context_tracking); -#endif +#endif /* CONFIG_CONTEXT_TRACKING */ #ifdef CONFIG_CONTEXT_TRACKING_USER static __always_inline int __ct_state(void) diff --git a/kernel/rcu/Kconfig.debug b/kernel/rcu/Kconfig.debug index 9b0b52e1836fa..8dc505d841f8d 100644 --- a/kernel/rcu/Kconfig.debug +++ b/kernel/rcu/Kconfig.debug @@ -168,4 +168,18 @@ config RCU_STRICT_GRACE_PERIOD when looking for certain types of RCU usage bugs, for example, too-short RCU read-side critical sections. + +config RCU_DYNTICKS_TORTURE + bool "Minimize RCU dynticks counter size" + depends on RCU_EXPERT + default n + help + This option controls the width of the dynticks counter. + + Lower values will make overflows more frequent, which will increase + the likelihood of extending grace-periods. This option sets the width + to its minimum usable value. + + This has no value for production and is only for testing. + endmenu # "RCU Debugging" From patchwork Tue Nov 19 15:34:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34407D44167 for ; Tue, 19 Nov 2024 15:37:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE1186B00B0; Tue, 19 Nov 2024 10:37:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B91266B00B3; Tue, 19 Nov 2024 10:37:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A322B6B00B5; Tue, 19 Nov 2024 10:37:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 854036B00B0 for ; Tue, 19 Nov 2024 10:37:22 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3DC27805DA for ; Tue, 19 Nov 2024 15:37:22 +0000 (UTC) X-FDA: 82803246120.03.C12783B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 0EB65A0013 for ; Tue, 19 Nov 2024 15:36:43 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OMSwBOYk; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030505; a=rsa-sha256; cv=none; b=gEkG7w2gFScYfSqmFoYZn0fvs3oa82vQhe7rmKur/+OZwBtq2nmm1zdvQbmUwu8pqymoso +tIO4itfDyAcQl5CaItWLoFUWnZbABE7kKUlTMUinSIc8JG/UGa4QqmOgZoOC6DTcyq6L5 WlWbgDJ4TbOAsmzdTQMHtK8ZwlJEq3c= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OMSwBOYk; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030505; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VTrWPsFjEj6kjl2SR0Q+Shqo2M3AIl3ViwWNRJPCXGk=; b=3rb1ApY/8g14tgVLMuVSS4lEUGRKho4JCaUW1B0e8gnTqE4BrGO4xQ3RZ/LPdSMSK8UY2Z UgoeEMNncTowCcW8glnX/2TKt55YTfVqlSZLKxEkyd1jdo/rL/kGvKAon88NQIwLLUXLQp 1YJB8ZGi7siMyLlyWCm1/Sx1Y3Ucoh8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VTrWPsFjEj6kjl2SR0Q+Shqo2M3AIl3ViwWNRJPCXGk=; b=OMSwBOYk9UfBrO+N24YsN89XVkIiEHAnH43u1SXAgwCtIKiKk75/rnyY/0+VZ9SIZVc0EZ N3VxTThDQaSFegferOI/ggk3ZN4QfzjC/Ep38e5GxQ/SKhP35ZhTfMaV/NHU9GYsSFKIvo RTtzJqlBTfIDpESsAUTqnOrJ3LdDJ2k= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-240-1AlVl6ygNXGoWGme969LHQ-1; Tue, 19 Nov 2024 10:37:17 -0500 X-MC-Unique: 1AlVl6ygNXGoWGme969LHQ-1 X-Mimecast-MFC-AGG-ID: 1AlVl6ygNXGoWGme969LHQ Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 188B01955DC3; Tue, 19 Nov 2024 15:37:12 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3D90930001A2; Tue, 19 Nov 2024 15:36:57 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: "Paul E . McKenney" , Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 05/15] rcutorture: Make TREE04 use CONFIG_RCU_DYNTICKS_TORTURE Date: Tue, 19 Nov 2024 16:34:52 +0100 Message-ID: <20241119153502.41361-6-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Queue-Id: 0EB65A0013 X-Stat-Signature: suexrjt9cocosiscpewcojkegahkn3jm X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1732030603-572264 X-HE-Meta: U2FsdGVkX186hKnnaH6a4DCdedBR63O8/uZIiD10nAP9kM+eMIUmXmxzp3x16xh/IoZjOwL6p7s2arHufRrWkdGISlFYd88UFC/1U6n/hoap2ZI2Cg9PlucHfn+ez5jBIeE1lvaSdrOr0jI20WoiU+XtctJ/UNUe1LxaUz6rlRmQnimWTBfsLdOZboDEBi09O6kVtjtn2jVs3P1ZAHs9ImCqhxOABTc0tFpJQVDo/hEQR6B6sTWx34rHPA35y6siQosTNQT2NIPQ0zFqdibY0mqpkUMIoGCejyNvDvK9AGhyvCC077oRlPQftEU5Sx3z8ydWy+FoncPz97SGCQLSF7/kmH93Kk9514N8sNrVwhtKUYi48k0uIqsSvCpefnqm8QxFe26+AovIgqdlZuxo/S0NO4e7TsCA5BuRHiimfEtHCxhClxeqKkRbKyy3v4n58nXAMqJphzdYP3zpLzD70dgwMLrTnxxr0a29m2i4+PLcWOOw/XnwwJa+INA7pO1QsUPNgYrSh0R5SWy4TftzVjQM39n1Kz1ahjPMkJXZzMWFLUDnD0K9MFtc2kpkwEQdMq5GpqxX1LWtdk1KqzqVamHrWokmybF8lLzUiimMo4iZHbHt05/rlKIo/QqaVbhblvLfgM13xFTD9Tbfj4z/L3wirjraYhdRdmk8cPu/JFX/jJnBBlouKXSZOCHedo7kKw4W8qpNRe0kNE1vAA9G65A2JfHLXnJKFWKa01wKjE4aT7MS4yo+Zrx85dCa0PXuReS0dSwOiMDIwnGgNTwxz0HoyDuX/XGqJIUIXsX6Tqigp/m+hADcqPwsoxQ39BFbTOebu54g4vrsisyjva1SAn5kZJjF15M7kSbNMe3mq83NvLiznvR3Tvz9zC4T0hGcQ0Zo4GM0J/P+/FA4dpdSb430ZM95G2ARgxPYhnsqRtTa2MXKFzsW2RZKrpUE/Hlt2pI4ptGxcWJJCEAnkTb V7TjfIJf nhbO3W0TsHjK6xiFCQshduo2e2nDuFnhKrwXtSuMrrH7uHFi+VuIcMmtvaaKk6FGnG+tkPSU5LT+lKJGvAgrFY/qdDSznhjGCKfzCsjAJCcuWpkQwhL/BD2oyeWFW31aAf1sNtDSHgX2pODA4GfkHc2dfFaw+ZxC2jWTWUTM7PxtvIzx/22irGopRAcD0jneXQvHrzG5dS6xRKB5pR3BhHe54F4AxS82WvUaK9NiZAYWvhH+VfN6deZurNt4bcSMrP87OBBVopuQCQyNYebuGfNYmCJrvvA+PCfVw7Jaf8GLW++c0h4mMJAfd/Hh1/houJ0aZ7mSE/XMEcg1JXLZXLDXqssvQTWccCN/QZmPRy3AbfsIaZPYFfGNVA8yi88Iv/osSO+3BdGGb3oUWcDwcmk2wgA3yA2w6ZZwBwbwwSU92lZY6sYitIC+6JhFt7Tt2CzkTEISZAxfayb04mIl3p2o7CrsTETuCtLxkfnl0h3cgxOg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We now have an RCU_EXPERT config for testing small-sized RCU dynticks counter: CONFIG_RCU_DYNTICKS_TORTURE. Modify scenario TREE04 to exercise to use this config in order to test a ridiculously small counter (2 bits). Link: http://lore.kernel.org/r/4c2cb573-168f-4806-b1d9-164e8276e66a@paulmck-laptop Suggested-by: Paul E. McKenney Signed-off-by: Valentin Schneider --- tools/testing/selftests/rcutorture/configs/rcu/TREE04 | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE04 b/tools/testing/selftests/rcutorture/configs/rcu/TREE04 index dc4985064b3ad..67caf4276bb01 100644 --- a/tools/testing/selftests/rcutorture/configs/rcu/TREE04 +++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE04 @@ -16,3 +16,4 @@ CONFIG_DEBUG_OBJECTS_RCU_HEAD=n CONFIG_RCU_EXPERT=y CONFIG_RCU_EQS_DEBUG=y CONFIG_RCU_LAZY=y +CONFIG_RCU_DYNTICKS_TORTURE=y From patchwork Tue Nov 19 15:34:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880132 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13BDED44167 for ; Tue, 19 Nov 2024 15:37:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A34276B00B5; Tue, 19 Nov 2024 10:37:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BEA76B00B8; Tue, 19 Nov 2024 10:37:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E96B6B00B9; Tue, 19 Nov 2024 10:37:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5D1286B00B5 for ; Tue, 19 Nov 2024 10:37:39 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 19C8D14059F for ; Tue, 19 Nov 2024 15:37:39 +0000 (UTC) X-FDA: 82803246624.09.52FF677 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf29.hostedemail.com (Postfix) with ESMTP id 891DF12000D for ; Tue, 19 Nov 2024 15:36:27 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=YcW6xxsd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf29.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030591; a=rsa-sha256; cv=none; b=z/hBArSFEx76vLeHZ3lYJ5/k8KtXV1q4qeYK7w1Gg4eBkanRVm0FaykvqGSEO03mnk+MlF 23dPpZvWE+QzxKqwOgSpzHHUQIpk4Pdfkx3TbHV+QI4xzt/c71/U0kU9o76939ZzmWM9Af Zd/FogBSh2XnKIObhgwYD5Gq6yKiiMQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=YcW6xxsd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf29.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030591; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wuJGo/SWDYxjpRhP1lHMjQsEt0PrcPcPyU5QylcJ498=; b=FNLBnOoZW0D59/Fboqwcu1KLdU+MCe7a8nxVU7sPnJnd/qtPl+97JZlgPQvEl9gbMm8YQg ewPBz8aq5ynvEMfmIVFObo53RPZEd4+L15vZxMnsbEZbv/8pEjxi0FzdfWHJ57qWqkAwbx zA03gM0JT01FV86l4ORLHwfhGlfysG4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030655; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wuJGo/SWDYxjpRhP1lHMjQsEt0PrcPcPyU5QylcJ498=; b=YcW6xxsdzDfWLYOzoZGGCwbUDf9UYpmgJMe9dvif+3qY60OAqlrjoAFQbPLPhLtsuxWDFv V3ClElQbm0+A7xaUKR2ejAZKy26BqV2peT1z0HMQlsi5xUgvqXVDWY95s8iMFcS5xnpqgk 7eU410cUYUMfK+aYr8b5MatAymSlxTQ= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-427-4V10RU0zMVukkEJohwz2Cw-1; Tue, 19 Nov 2024 10:37:32 -0500 X-MC-Unique: 4V10RU0zMVukkEJohwz2Cw-1 X-Mimecast-MFC-AGG-ID: 4V10RU0zMVukkEJohwz2Cw Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4177119773FC; Tue, 19 Nov 2024 15:37:27 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 859B830001A0; Tue, 19 Nov 2024 15:37:12 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 06/15] jump_label: Add forceful jump label type Date: Tue, 19 Nov 2024 16:34:53 +0100 Message-ID: <20241119153502.41361-7-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspam-User: X-Rspamd-Queue-Id: 891DF12000D X-Rspamd-Server: rspam11 X-Stat-Signature: 5szmc3kmhzppxxssx6rsmc4y9c6si3ur X-HE-Tag: 1732030587-704704 X-HE-Meta: U2FsdGVkX18lojUYavE9HKQMEHfATpGVJPNEv6eJUDQ7cMGZ1eORB7eFPbylKLZbGd1z9aV9MsPdrfXO2yaFg1D7oNJ2UXSSfhLu5G51Qwa2JNqt0eDDU9vfLJAJJXDRqsqieUql/mpeVCRPfnhyw07hr44q41H6DjanLqwfmDZX/0yZkkIHKMQeiPaBVSCpSQbh3iz2ePuwJMrk6svVpWI5DvNOpUdHslD8Zv03ERc5a2Y0lXv/7lVUyV/oI7J6VNdO2ohgzCLx+mTBGAlnbU2QceuP5aW3CwQDBKtO882jq94ew/YUEa6wq++pxmqNsTLNjM0PzXeoe7d2gV+shQD2ySmcKjHpLh/MZRT3O0iKnHf2bQegI6ag1YnWQPXSi4jzQzigOSNYsEK0XrEVpXuGog0CeBkOZy0lgsQdeyLEhNFEtF2FGMkN7JAgIsIkpHdg5svOE1KT+61yDzOuXHkrPUOXA4A9zeoHV1hU22SoP2rHeSL5ok2V549m0BcYKeICqwVy9zAjwJj58QUTNaEbnqFftxsrGqAeDgg5Zx0fr4aon24Z9gdt0NuXYR6P0PlhJwPbQeCEYadiRe8SRpYQbWGcucNUy7aucw0j+WcJgICL48+XtMUlglMamSZ2o4xWu03sXS+u7au6ZNZpRfaYJFT5TQ3z56IdnJGmIQ8yp7cDoAttJhpNph+WDbNHtdW1xh3nLzR4VW2HeIFzFs0QEDVELdZYXb460ztZFTty0IO1DEV6SRycTgAo4e9vlg5SmZwSN4qm9YJPpkQWs+qwT8q8bpt8tAOMSFS75Sombeks7cHNslid0PtMmZ+wbP3CIpJ1pnurURRFlzuQW1K9RdaVjSHYvwDiQH+21XRUStrOCau1zBYe5oYBOlnaDVuNma+AzU5G3ffEQdevrCe2cRskedAxVFP49qnKBWtlVFC22hNQT1T4pa1Z6d+fD3ITPSBWnmxKjq7GYak xIXU7WHX SyXv0VdZ6stQynl1thj6wzTV9Rn1yUnRThwSfPoDnXbsEPvFbL1pzXb7XYx+MrOLrC/sO7JXh1QdeqkVg5RVXUjbwD8MmEl7LZXUMeQZtikjzrrknJsfWCC52W02FS5Ns6wwBLjSaZyorU3m0cnVkAGKoPW6GdLAcSyWr2fNQaqrU/MrqL1pwDX2z1yAe92VlHLMx6E7pKfbjqr4Tj1Ple62xpiBkzzeLfLn4GsT6TbeHeZEAzSOJOixPaam1X9yaABTcKmk2RIVdfahGUX+iIjVrjwqPJ3J9gZlywEQMc+M/WPhbzzFzRMEBXk7GU8aK22UMMK92RLewBtHlI80X0MhY1G5naWJJSW8COPeiVhaKLXniS/hHyPRsyh73TIFUzkTbK7ydwsR6ajZ88BuG9xacsY27pcb4I4fUtyhH/mEHjpbQfC+PR72IiJirMleTmkHXrK9Pbg/l8O7IDepnVpgecw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Later commits will cause objtool to warn about non __ro_after_init static keys being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. Two such keys currently exist: mds_idle_clear and __sched_clock_stable, which can both be modified at runtime. As discussed at LPC 2024 during the realtime micro-conference, modifying these specific keys incurs additional interference (SMT hotplug) or can even be downright incompatible with NOHZ_FULL (unstable TSC). Suppressing the IPI associated with modifying such keys is thus a minor concern wrt NOHZ_FULL interference, so add a jump type that will be leveraged by both the kernel (to know not to defer the IPI) and objtool (to know not to generate the aforementioned warning). Signed-off-by: Valentin Schneider --- include/linux/jump_label.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index f5a2727ca4a9a..93e729545b941 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -200,7 +200,8 @@ struct module; #define JUMP_TYPE_FALSE 0UL #define JUMP_TYPE_TRUE 1UL #define JUMP_TYPE_LINKED 2UL -#define JUMP_TYPE_MASK 3UL +#define JUMP_TYPE_FORCEFUL 4UL +#define JUMP_TYPE_MASK 7UL static __always_inline bool static_key_false(struct static_key *key) { @@ -244,12 +245,15 @@ extern enum jump_label_type jump_label_init_type(struct jump_entry *entry); * raw value, but have added a BUILD_BUG_ON() to catch any issues in * jump_label_init() see: kernel/jump_label.c. */ -#define STATIC_KEY_INIT_TRUE \ - { .enabled = { 1 }, \ - { .type = JUMP_TYPE_TRUE } } -#define STATIC_KEY_INIT_FALSE \ - { .enabled = { 0 }, \ - { .type = JUMP_TYPE_FALSE } } +#define __STATIC_KEY_INIT(_true, force) \ + { .enabled = { _true }, \ + { .type = (_true ? JUMP_TYPE_TRUE : JUMP_TYPE_FALSE) | \ + (force ? JUMP_TYPE_FORCEFUL : 0UL)} } + +#define STATIC_KEY_INIT_TRUE __STATIC_KEY_INIT(true, false) +#define STATIC_KEY_INIT_FALSE __STATIC_KEY_INIT(false, false) +#define STATIC_KEY_INIT_TRUE_FORCE __STATIC_KEY_INIT(true, true) +#define STATIC_KEY_INIT_FALSE_FORCE __STATIC_KEY_INIT(false, true) #else /* !CONFIG_JUMP_LABEL */ @@ -369,6 +373,8 @@ struct static_key_false { #define STATIC_KEY_TRUE_INIT (struct static_key_true) { .key = STATIC_KEY_INIT_TRUE, } #define STATIC_KEY_FALSE_INIT (struct static_key_false){ .key = STATIC_KEY_INIT_FALSE, } +#define STATIC_KEY_TRUE_FORCE_INIT (struct static_key_true) { .key = STATIC_KEY_INIT_TRUE_FORCE, } +#define STATIC_KEY_FALSE_FORCE_INIT (struct static_key_false){ .key = STATIC_KEY_INIT_FALSE_FORCE, } #define DEFINE_STATIC_KEY_TRUE(name) \ struct static_key_true name = STATIC_KEY_TRUE_INIT @@ -376,6 +382,9 @@ struct static_key_false { #define DEFINE_STATIC_KEY_TRUE_RO(name) \ struct static_key_true name __ro_after_init = STATIC_KEY_TRUE_INIT +#define DEFINE_STATIC_KEY_TRUE_FORCE(name) \ + struct static_key_true name = STATIC_KEY_TRUE_FORCE_INIT + #define DECLARE_STATIC_KEY_TRUE(name) \ extern struct static_key_true name @@ -385,6 +394,9 @@ struct static_key_false { #define DEFINE_STATIC_KEY_FALSE_RO(name) \ struct static_key_false name __ro_after_init = STATIC_KEY_FALSE_INIT +#define DEFINE_STATIC_KEY_FALSE_FORCE(name) \ + struct static_key_false name = STATIC_KEY_FALSE_FORCE_INIT + #define DECLARE_STATIC_KEY_FALSE(name) \ extern struct static_key_false name From patchwork Tue Nov 19 15:34:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0C4DD44174 for ; Tue, 19 Nov 2024 15:37:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68ACA6B0085; Tue, 19 Nov 2024 10:37:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 616616B00BA; Tue, 19 Nov 2024 10:37:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 440256B00BB; Tue, 19 Nov 2024 10:37:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2100E6B0085 for ; Tue, 19 Nov 2024 10:37:54 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D2A32160592 for ; Tue, 19 Nov 2024 15:37:53 +0000 (UTC) X-FDA: 82803249018.16.F639EAD Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 79D801C0007 for ; Tue, 19 Nov 2024 15:36:16 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WxYnmFsT; spf=pass (imf21.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UFFvFWWnWyZvXw5qfQp0QkusU/cmiYrMNoaijrYT6Zw=; b=LBLgmPO/9D+65sCS5F8eLX/ag83lnvNFkY1o8S0Uzw8rTJl4G8DeD80h1bMeHTmroTyNwY JJxTIUhG/+JQC7X+CQEkB5jkxiqYoS3XRlsjlTdF0rdmFyp+5U0eS7p8TcD5kUsi6zKBpr 2+rEkvxo9G6/7bZ+Oa4gp5rzj/G7vDA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030522; a=rsa-sha256; cv=none; b=G2DrEETb2BVB54CzOFOZsGGewk0kYWUc6kB6q/2gmaLpM9lKglODpOc+iErTZHuMrvhS6E XeIfO1t2zxulh3/MVKyrKfthTbhNWxXXTHnHsj4WAlw8wQQoTBrej7zYCeycFe23uH3PEL oxxlxl+7FCXz4lkITkvCk3c9/JhsDjA= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WxYnmFsT; spf=pass (imf21.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030671; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UFFvFWWnWyZvXw5qfQp0QkusU/cmiYrMNoaijrYT6Zw=; b=WxYnmFsTh1VKnOyjeivpXhvhNB8Z2uuwFr6HuiWlB6+/dz81FTDXTl+JCsCaeKhuX/vyNO pDCP68CXG+h9CtY+QjaEYg02XYwBWoSAdOJgD4og2cDMbD+TkIH8sIeKq464jgq5Vwnbev E8UZsu6EApDTAx3laFJJwMst0HA1SK8= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-594-8NjzI6AZPs6iJgNCQczRug-1; Tue, 19 Nov 2024 10:37:49 -0500 X-MC-Unique: 8NjzI6AZPs6iJgNCQczRug-1 X-Mimecast-MFC-AGG-ID: 8NjzI6AZPs6iJgNCQczRug Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DF4C91979052; Tue, 19 Nov 2024 15:37:43 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B133F30001A0; Tue, 19 Nov 2024 15:37:27 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 07/15] x86/speculation/mds: Make mds_idle_clear forceful Date: Tue, 19 Nov 2024 16:34:54 +0100 Message-ID: <20241119153502.41361-8-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 79D801C0007 X-Stat-Signature: 4e7sid5u83415d8wrm1j6xh8wuabagqn X-HE-Tag: 1732030576-695202 X-HE-Meta: U2FsdGVkX19GL/Py8Tzn+tnCOPbv82d+EnjK7Yw0n+hAuuZQ08Jj7wsTDA5IOhLgu8t16GwUC3pK4S+nuukRK4TQAgbEzWGJmVLvix3QKTDGTFcsgehx8MscqJ4r5tS0Dju4b6OJkTZSCrVMbTOGvcmJn7tPacMzPxSw4Ke5dbXCXHyzmc6HtLefga2A8m9HvrtAXyg2FnvA4C9T/KQz+U8dtVCbDNpmXaaTxcV33sue/ntPZpA9E7JYHZrFMIO7Oh9aRGRDN1VbV52Rjk+jUUWuCNZ9Md2xXv1YN8ROOWhNDDIxLXBDPkwAv5tnIZOhlfN+W1LHbXXLYpcDCfPIebesXs2A9WxjSiKBEZNxtlP1RHL36cTZZYzRYsRSxVAspOeTXULiWnsI2n+oFbeEWlx/aT490FgAqyv4NFTPQJx7FNt/dy/GOjYhzMbQC3PCenpkaTgJOikXIOuU03Eg/JOwPpQar79f77itgiPRM/UQZtiwIm0cwtIg+ihMxkh66PWoKdUDsrKp+ErYtFdEKvWT5FpDcHy4df9HKJJ0W7UPTjvF6gVSz+oVmBvPJSzR9C9FAm/uhZFv5QRUacpbbJ5eOq5hIcB8o3d/CBVOFcq+0vy91RGDSCC4gyu5e53shotsFCVgM2m2Q6P9kga2OxqS9jOCIquJrFT1EcFlS6NIswmPu/vloyDTygLrji9/tYCrKNhJaSa7HaV459V4XlAtvEY9cuWqOSHlkCu1J+lv8Cul+azzDQJhgmqsyAMJSRE7hIC60w5rwd1q0Qzc1t27Kp6DN1dutzI1UuH40Ltimc5lnEylprvyffAySVi8Rc/58fXqyP8GUTKK3yF6qtWww2t+etvr/vwFtxPyMjjr3mdH8Lat/rwuo8JB8vKlJveBCL7W7zY42b7w4DpLPbtNTIBtFqE0jTqIu6O34jqOWnepzStQwEWBwg2crJ9tnCFeeXK2RETRme2Fu/J 1n+LCKwA bn7jE/6pJemgSgmLZ9webLFe7JdcTFu12bBWU7AD5MAVwYw6Kvaz3hIi2nVGUOwPtemfW47R5VsE05IfvwapO8o0WvLHVke5aNHIaC8AQ1HvbRQDDEbmDZCYyeNClQUOHB7eRF6zEvo+jykLkZLnvPcnGVGx0ocGWk9EXICr8wnzN8IpfnhI5JffWRkkBpKTk7gZ24ke5KlmDLRvOFsEISKmyDORSx+RvnubX87y2FnGPrCm+uaEl8pQ5Wqo9Oay9SI9OERYUa6KNNSB55iY+12xV6M0achgJUNdNkQOl7ADV5tlqL1HJKhkDRgecOht5OIBzB9Ng2fftf9tggVoz/cT4Om90b+q83LSG9BKRecA8XQtgLEXw1yUaHLg2GfvCHAnUHcs7dh+qVLOqHqc5H1g72FX0IeQb7ioQm5lmty85L+AA15D+JE/GadeQ9/z9CCMllozCeYMlI1U7zddfyuJ49Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Later commits will cause objtool to warn about non __ro_after_init static keys being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. mds_idle_clear is used in .noinstr code, and can be modified at runtime (SMT hotplug). Suppressing the text_poke_sync() IPI has little benefits for this key, as hotplug implies eventually going through takedown_cpu() -> stop_machine_cpuslocked() which is going to cause interference on all online CPUs anyway. Mark it as forceful to let the kernel know to always send the text_poke_sync() IPI for it, and to let objtool know not to warn about it. Signed-off-by: Valentin Schneider --- arch/x86/kernel/cpu/bugs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index 47a01d4028f60..dcc4d5d6e3b95 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -114,7 +114,7 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_ibpb); DEFINE_STATIC_KEY_FALSE(switch_mm_always_ibpb); /* Control MDS CPU buffer clear before idling (halt, mwait) */ -DEFINE_STATIC_KEY_FALSE(mds_idle_clear); +DEFINE_STATIC_KEY_FALSE_FORCE(mds_idle_clear); EXPORT_SYMBOL_GPL(mds_idle_clear); /* From patchwork Tue Nov 19 15:34:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC137D44167 for ; Tue, 19 Nov 2024 15:38:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 41A086B00BB; Tue, 19 Nov 2024 10:38:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A28E6B00BC; Tue, 19 Nov 2024 10:38:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F6B26B00BD; Tue, 19 Nov 2024 10:38:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EEC506B00BB for ; Tue, 19 Nov 2024 10:38:10 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AC6DEA05CA for ; Tue, 19 Nov 2024 15:38:10 +0000 (UTC) X-FDA: 82803250278.29.73091BE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf23.hostedemail.com (Postfix) with ESMTP id A9CF114000B for ; Tue, 19 Nov 2024 15:37:35 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Ksdh/qUf"; spf=pass (imf23.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030597; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lpSGcS2OHkivSptNEqV1yoUV7L+1Tpt77Idg0XadxAw=; b=ZwvlYDyHILWY3eyKImhvNt32RHw9nFeM+42tWtBt+Ytslnnfz0xXLYmUOjkhiPosZ3mP3H xBtV9nJkavFuX619RrxdXCiDXqjteBbekYBhFyAoXLp04PGE3s3I2SULNcqdAi5rDt+59r jXwQJR57HkTzh7EbBIPzvVLCt6rT4ns= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030597; a=rsa-sha256; cv=none; b=mHIErzda3Gx86NwPtO8Yg5twdhwWZlyG7Iy85UpxWGuB6WYPC5+GKXX11TsatXO4A/lgod Kgg6kYuWCvxBNghpkILcoysP490N8JeN8/znE+j7fMkYjBOUtOuvWfDpHWUnmzba574b4/ PCgLS1nhtkLyGB7fbgvrsNzXhVbUUPM= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Ksdh/qUf"; spf=pass (imf23.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030688; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lpSGcS2OHkivSptNEqV1yoUV7L+1Tpt77Idg0XadxAw=; b=Ksdh/qUfMGAu3JF60FBMQQA8j5wcLeTDikdgTdcG/3VfJ7rqNBrEJ3w6b83/WBkylTaf9p GJrpI8XeDKN4Qci7qUdjy2EHfJycO42eIA1roL9NZsOV6FOnAKmEdbSABiOKL/AFzDT5yF EgBcphIs5Mu7KHFkPluWl44QWo8zcnk= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-209-w8WZReDNPhuTuNl53MRXLw-1; Tue, 19 Nov 2024 10:38:05 -0500 X-MC-Unique: w8WZReDNPhuTuNl53MRXLw-1 X-Mimecast-MFC-AGG-ID: w8WZReDNPhuTuNl53MRXLw Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8F25E1955F41; Tue, 19 Nov 2024 15:37:59 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 59F3B30001A2; Tue, 19 Nov 2024 15:37:44 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 08/15] sched/clock, x86: Make __sched_clock_stable forceful Date: Tue, 19 Nov 2024 16:34:55 +0100 Message-ID: <20241119153502.41361-9-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Server: rspam10 X-Stat-Signature: xctqcbpixwmsg1zt3px6hekjyc6ukwpy X-Rspamd-Queue-Id: A9CF114000B X-Rspam-User: X-HE-Tag: 1732030655-843691 X-HE-Meta: U2FsdGVkX18+mGX0DY+R5XvwsHVRpomj8CBVdB0FTtA5SldrOxo1YwA3I3+lP4pmH+eqxurZTVcVk+pGSC2Ewolc8MdUHSwAOffAPx0syjIBcjB8sF5wJK4zjKH9iHQSvORwOtVZ608H1RyHVmZBc2sFIw3lQT8/pdivHb4CXpsR/pBiZMd2gO6lWTpV1vYAe+4AinNwoUTMSLs61i3d58z0W+wz7mhihlHsOsvBRKs2kiqq0hc3ytwUIRCE7DyoF/tPkR3sWEOpsfEnM6xf65xbCdlwpSr4YWTTRcQhuK11ESIqaTB0jT9l/7JunmeHHtayxmb6aKrjTwUmzf6Xat/tDDryivgVQQCXAhLfoBbHA5rbHIc2wCl/pI4dD7IpS/+3Eb21nyzjb7ErxaZJB2C0esyXfHM0BZrV6x/kAGoyR07cvftySF7GbxEBa6uHGg0l4JAU99D4TZP7l1yVaS0x0+Iv81s3nBikMXvaGKNasrc3G/B6PQ+3FKUUPpwocRsiITNSgGAJDrdDWOZEFdUZiENxfgFWXDSM72UqZ52cPRSySfFAlbf+OlwJlVt4X7klqcYbceRPkPbH5PNnc5xChaXXH+gSmcPdyXZs2AC+WpIyjjPopQWTHpR2OH2e6s4CFyOhTH76xVcQuSanLicA84GjLaVZ1PAKI/4xBVxkgVwN77x0Q1D/aNCEzFpx5WDDVmH6iU95XkW/mOreWyT+9LGTPfP3d6X/nVf9f1xyDOhe75ahwk1kvQfKunpROrvpVY8eoQlaqfHbfuzXClT/kAUTwwM1wk2aWyUt2zQqXIvhOSv8K3iKMPj5ZloNCTQgfnLX0MVh6/fOZ0KqiVxpEtY500lJJqQe+YJHMxeQJounbDVo6hBndj2jhVQTynQNX8eWeK89RGgpj8iC6scYdYYscKgfMi3TkAd7QW5LZmaLcGxxW+bCL60GDIvaepACJ2NL1KM2kQdlSEZ prqmdcsA N2BP7i/vnUFaley2ZM2jdo54xHHhJ01CQ4wvwl1yP7wWjZBEUQqFVg0PgNW0tHXM7dwwd9Lw4pvdk5B0PM5JJKFb/CkodzhHgQhwv/M4NdzdzF4CiHeNTC8eJwFojpUBd/mUPibLMNhpzTed2jvHfVM7It8s8qzJ/WlVVp5coo9NvT3P9v3hzuhlJQebgIQeswes3uOn/lSsm54GdWsB8d+ZFwrnxN2uELwzZ0YiNt6SpgTMP+0o/5fwsK3eA3hAsj1GhFGQC/75tnSnTKVWnZyogFY6K7AS4wqrQYDBvSMmcZRM4Si3ZyLKKLm6O8wcHZiR2rMA9IdfRu+3TKtS2ojxeYSswn1xOaNuw2fAB89ZTxsKdHOjEpFeq+aPeK7E4QmO4VGY0FmRJSeRZfgs8AN5uRtw/PfJkoLO8ymm/SYuFmyvgv0LxgoovRmBMAOg/DgcGWh2gHTNDx361cNZb4apWQw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Later commits will cause objtool to warn about non __ro_after_init static keys being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. __sched_clock_stable is used in .noinstr code, and can be modified at runtime (e.g. KVM module loading). Suppressing the text_poke_sync() IPI has little benefits for this key, as NOHZ_FULL is incompatible with an unstable TSC anyway. Mark it as forceful to let the kernel know to always send the text_poke_sync() IPI for it, and to let objtool know not to warn about it. Signed-off-by: Valentin Schneider --- kernel/sched/clock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c index 200e5568b9894..dc94b3717f5ce 100644 --- a/kernel/sched/clock.c +++ b/kernel/sched/clock.c @@ -76,7 +76,7 @@ static DEFINE_STATIC_KEY_FALSE_RO(sched_clock_running); * Similarly we start with __sched_clock_stable_early, thereby assuming we * will become stable, such that there's only a single 1 -> 0 transition. */ -static DEFINE_STATIC_KEY_FALSE(__sched_clock_stable); +static DEFINE_STATIC_KEY_FALSE_FORCE(__sched_clock_stable); static int __sched_clock_stable_early = 1; /* From patchwork Tue Nov 19 15:34:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6DD3D44167 for ; Tue, 19 Nov 2024 15:38:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68C0D6B00BD; Tue, 19 Nov 2024 10:38:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 614716B00BE; Tue, 19 Nov 2024 10:38:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 48DF76B00BF; Tue, 19 Nov 2024 10:38:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 27EDE6B00BD for ; Tue, 19 Nov 2024 10:38:31 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A1A53AD803 for ; Tue, 19 Nov 2024 15:38:30 +0000 (UTC) X-FDA: 82803249648.06.471A8FB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 5B4E31C001A for ; Tue, 19 Nov 2024 15:36:53 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RKXdZioW; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030524; a=rsa-sha256; cv=none; b=EtU8mo50ahSC4OrAYxyxi98+4ixZFSAHH/SCaQdm/1S9jSs7sBccczpKFdR9sv1EjI5dqW CzaGOlwalDROPkp93i8nTSVxqxP2XMsIcldKOokXGPXVQssnhTuRaT7VZwRJ2u6dl3DWHF 7Fm2gWdJDVLw8W9ELF0hc3A9vYD0uOk= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RKXdZioW; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030524; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bbbl+TXDo+t5kIfzQ14qJGOWt1RDAARb2PWsvxSVw1o=; b=I0lQMPJbUBPkDOBp04LkcXr1xMqAg1FIoV2C0c84Zkx6cca9qWu/3N70P9VVMnog7U/sQT 0lojZSL8O7rHO7yCF7WPcb0c5WLeHXRpnuYlVMsyQ7m0w+u2MoF2YnYruv/7PRJXToak31 /K3ImGKeFies80G8iK/1SKzsqCQCu2E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030708; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bbbl+TXDo+t5kIfzQ14qJGOWt1RDAARb2PWsvxSVw1o=; b=RKXdZioWopPX8+69sfm+KF6YiWqhAH58iynNIqC0/i/mf/aqLaBpwMCzD4IueDoTdI4tlP +jgwj12LeBdoHMvRrsboAn6ALSmXv1ATZXl7z1gnobmsHTna8QYte4/o7hsrzLoEfFR4Q3 a9MGpB95iEtzvfx7AdzU6t4CRAd/3j0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-593-1WZ3p9jrPQW6G0U5yxmRQQ-1; Tue, 19 Nov 2024 10:38:24 -0500 X-MC-Unique: 1WZ3p9jrPQW6G0U5yxmRQQ-1 X-Mimecast-MFC-AGG-ID: 1WZ3p9jrPQW6G0U5yxmRQQ Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E53F01952D05; Tue, 19 Nov 2024 15:38:14 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DDE233003B71; Tue, 19 Nov 2024 15:37:59 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Josh Poimboeuf , Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 09/15] objtool: Warn about non __ro_after_init static key usage in .noinstr Date: Tue, 19 Nov 2024 16:34:56 +0100 Message-ID: <20241119153502.41361-10-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Stat-Signature: cbfoq13fn1ffzqtq5ffhcpi6tmaefyop X-Rspamd-Queue-Id: 5B4E31C001A X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1732030613-94811 X-HE-Meta: U2FsdGVkX1/YQMpYDGDA9Si8/sfKwi1fOcF3WQy33mb5/tumtiRjKpju5jC/ZBSfFCLPpj4avVGG9k9+3ry7VurBgFwva05ycLaPVxbTYkt4tPITqOaJeMIQYxRBv6VLByZtUTeAgzHC9FZVywUXU8Z/f4jd3MEd4DvcGTQZvlqVBTIfvKkgmLvre199vrBcU1mmJ5ctbPY3Uctk2i9uWP2XAnOehZxCZA9a316SeYblqwh/IPdTkwef0FEdldG6+cLfamlsWeSXfxVlfqSj2V+bLnKvJvgJ6JlwodDhx/AnzG6t6O2uhqfnd7lOslOETdhUBtmYXMZLynHX5K/IbSNRdpeZ3lmjHzQsjtQfW9lgHHk7qJN2860FHu2kYDHh8wL/zzSVjla6J0yU75ScGtwUs7EoXge2fgZ0HzssmJecucp+z6w+Wlq3Nk5NAKfLXKU3kKvqf2QSWGhESYmTkfqaj4DNW0jVVr+oG6H/HzGBPOBADwtC+7f8cSWit5/LRdf5VyOI2AOoIeU9CunHwS9wIKLPUZtQEA2dUnuFbUWviQ2eXZoGx5wQ5K7Fvc9OgNTalUktcZKRazCC3EFMPNbaBDJat/ZVCpMuq/PQv4UP+bzEv1U93DiblrwafqiCphx8iTtPa9Pwv516MTt4sDZ2BWiXQJhgtg2dkDO8dzXNDYdT4eYAbe89zN8SjY62Hymc5a5gCuExLEHTmW7JwdNSeqxD+MdiFyzBNUQp7gTGdkDGhMt3Mv+PHgyC1Qll3NjTuudzGbab8/IvDKSNyIe5JbRCDOmZJ4AiMyla+/Ef3pS3KfcYTe2A66g0acsDNuFmREauf5KnB9gLCBI0xzCvIAUPSM4NFDcABPX/+bukanEjCGAxmtjI4KKRC0UOdKFOaLz5sQl340wCpHzMvzg8IzPa+D7WRBHLX6JBdMxmncgtne3aF3jTzPP8vcy+flUOOQkJfISUirdPxM/ h4LVIgBQ Yuct5fEptvczlqDwyCz0lSU8aE7Np2Wm8syrThm1t7l2jsaAO+P1OY/zXkGpEQv6wTyRjALA+OE7ZkxavXRzJg2b6DrmRolMk0po9FOOdkDpxsW22ZT0tkwYSDjBBTcwkJ5GyQiPTP3GKi35T1XUq8yMbAeRMn6FsmmrewvyaTaoAcry63hHpFJmFnseRfo4ahPRC9rsEKsRmehk96MUKAZerk6ySkVb5SvxC+Q+yxmdSgIccir25ezSrlp+EwITaIK30tSJUeTWowMfzRGgccW53NkjWkT4TZVrK1Qsd7IC6QZQ4ToY+X+38GwaIrLiXHY70/z8C/YsYJ/JgTFr0ePghBsh278f5WGRw7J6lSVZ0nuuG2BNuGMKvcKkc/7lTHFoj2codlAgblyP06+yIfHEEExTx94MgJpCBpPRjr2uZhM9miPoBPQxJCbH4ZuvCS/Y/ecHfD5v42l9N+qYzkPnjdM9ytdfStP4GrT5UI9tUuPyBO4Uip/6x1Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Later commits will disallow runtime-mutable text in .noinstr sections in order to safely defer instruction patching IPIs. All static keys used in .noinstr sections have now been checked as being either flagged as __ro_after_init, or as forceful static keys. Any occurrence of this new warning would be the result of a code change that will need looking at. Suggested-by: Josh Poimboeuf Signed-off-by: Valentin Schneider --- offset_of(static_key.type) and JUMP_TYPE_FORCEFUL would need to be shoved into a somewhat standalone header file that could be included by objtool itself. --- tools/objtool/Documentation/objtool.txt | 13 ++++++++ tools/objtool/check.c | 41 +++++++++++++++++++++++++ tools/objtool/include/objtool/check.h | 1 + tools/objtool/include/objtool/special.h | 2 ++ tools/objtool/special.c | 3 ++ 5 files changed, 60 insertions(+) diff --git a/tools/objtool/Documentation/objtool.txt b/tools/objtool/Documentation/objtool.txt index 7c3ee959b63c7..06fa285873387 100644 --- a/tools/objtool/Documentation/objtool.txt +++ b/tools/objtool/Documentation/objtool.txt @@ -447,6 +447,19 @@ the objtool maintainers. names and does not use module_init() / module_exit() macros to create them. +13. file.o: warning: func()+0x2a: non __ro_after_init static key "key" in .noinstr section + + This means that the noinstr function func() uses a static key that can be + modified at runtime. This is not allowed as noinstr functions rely on + containing stable instructions after init. + + Check whether the static key in question can really be modified at runtime, + and if it is only enabled during init then mark it as __ro_after_init. If it + genuinely needs to be modified at runtime: + + 1) Directly rely on the underlying atomic count of they key in the noinstr + functions. + 2) Mark the static key as forceful. If the error doesn't seem to make sense, it could be a bug in objtool. Feel free to ask the objtool maintainer for help. diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 00e25492f5065..c1fb02c326839 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2056,6 +2056,9 @@ static int add_special_section_alts(struct objtool_file *file) alt->next = orig_insn->alts; orig_insn->alts = alt; + if (special_alt->key_sym) + orig_insn->key_sym = special_alt->key_sym; + list_del(&special_alt->list); free(special_alt); } @@ -3605,6 +3608,41 @@ static int validate_return(struct symbol *func, struct instruction *insn, struct return 0; } +static bool static_key_is_forceful(struct symbol *key) +{ + if (!strcmp(key->sec->name, ".data")) { + unsigned long data_offset = key->sec->data->d_off; + unsigned long key_offset = key->sym.st_value; + char* data = key->sec->data->d_buf; + + /* + * offset_of(static_key.type) + * v + * v JUMP_TYPE_FORCEFUL + * v v + */ + return data[(key_offset + 8) - data_offset] & 0x4; + } + + return false; +} + +static int validate_static_key(struct instruction *insn, struct insn_state *state) +{ + if (state->noinstr && state->instr <= 0) { + if (static_key_is_forceful(insn->key_sym)) + return 0; + + if ((strcmp(insn->key_sym->sec->name, ".data..ro_after_init"))) { + WARN_INSN(insn, "non __ro_after_init static key \"%s\" in .noinstr section", + insn->key_sym->name); + return 1; + } + } + + return 0; +} + static struct instruction *next_insn_to_validate(struct objtool_file *file, struct instruction *insn) { @@ -3766,6 +3804,9 @@ static int validate_branch(struct objtool_file *file, struct symbol *func, if (handle_insn_ops(insn, next_insn, &state)) return 1; + if (insn->key_sym) + validate_static_key(insn, &state); + switch (insn->type) { case INSN_RETURN: diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/objtool/check.h index daa46f1f0965a..35dd21f8f41e1 100644 --- a/tools/objtool/include/objtool/check.h +++ b/tools/objtool/include/objtool/check.h @@ -77,6 +77,7 @@ struct instruction { struct symbol *sym; struct stack_op *stack_ops; struct cfi_state *cfi; + struct symbol *key_sym; }; static inline struct symbol *insn_func(struct instruction *insn) diff --git a/tools/objtool/include/objtool/special.h b/tools/objtool/include/objtool/special.h index 86d4af9c5aa9d..0e61f34fe3a28 100644 --- a/tools/objtool/include/objtool/special.h +++ b/tools/objtool/include/objtool/special.h @@ -27,6 +27,8 @@ struct special_alt { struct section *new_sec; unsigned long new_off; + struct symbol *key_sym; + unsigned int orig_len, new_len; /* group only */ }; diff --git a/tools/objtool/special.c b/tools/objtool/special.c index 097a69db82a0e..fefab2b471bf8 100644 --- a/tools/objtool/special.c +++ b/tools/objtool/special.c @@ -127,6 +127,9 @@ static int get_alt_entry(struct elf *elf, const struct special_entry *entry, return -1; } alt->key_addend = reloc_addend(key_reloc); + + reloc_to_sec_off(key_reloc, &sec, &offset); + alt->key_sym = find_symbol_by_offset(sec, offset & ~3); } return 0; From patchwork Tue Nov 19 15:34:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AD6DD44174 for ; Tue, 19 Nov 2024 15:38:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21F466B00AF; Tue, 19 Nov 2024 10:38:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A83E6B00C2; Tue, 19 Nov 2024 10:38:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F17016B00C3; Tue, 19 Nov 2024 10:38:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CF10A6B00AF for ; Tue, 19 Nov 2024 10:38:49 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8B3C8AD2E6 for ; Tue, 19 Nov 2024 15:38:49 +0000 (UTC) X-FDA: 82803251874.28.2D7F5BA Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 660A714002A for ; Tue, 19 Nov 2024 15:38:14 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hJEvkfIj; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030636; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=URgg3URVd8Hv8WB8p504Oe6mTtzB0H+jxnfnYfZO9Vc=; b=GOeUUM/eHyJHu4P3wto93Zro9ABQJL85eLOxPUEOn7dXSXAjJawT1e48pngEF2sJjBDQgQ cc5xnOn3n1v6hp/I7fJT6WUT2QZILKy/HNIxsbGCrubcH4emJmx+A13ufS+056TVPt0ClG PuDudRNuf8eeE/6MTUEw1rr1hq/QRg8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hJEvkfIj; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030636; a=rsa-sha256; cv=none; b=sD3lKL2FGYpEY7+pqt/TiQX7q6JQl960EJsuOaj51g+Q547XYSlX3tiC7AJqw5h022FBIi SsGsRw7TskVvsRP/TqWBmM/VNKtQRZxTa2lAhPpSfB2K1y863Lv2q/VRElzsb2u1cklomG u7WAihl/Kr6H5r5eaKAOWDvekfGdH+k= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030726; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=URgg3URVd8Hv8WB8p504Oe6mTtzB0H+jxnfnYfZO9Vc=; b=hJEvkfIj1aFxVDKl4OGFI+9GBa2QFeq2uVNCkoX15h6L/saE052u0/Vg+bSkKjRqHa0c7L xpe5/Lkz0xxAlOAA17qexPH8TkwrhxI8GL6aY1gVcazWj4N4SUV8d56/i2sVII8/M6PHoO 1H6z8MeyZizKo/TdYr0aJ4SU58OBcPQ= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-99-3C0haj_PPdCTmZEKdqoqww-1; Tue, 19 Nov 2024 10:38:43 -0500 X-MC-Unique: 3C0haj_PPdCTmZEKdqoqww-1 X-Mimecast-MFC-AGG-ID: 3C0haj_PPdCTmZEKdqoqww Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DC0AB193EF53; Tue, 19 Nov 2024 15:38:32 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 613EB30001A2; Tue, 19 Nov 2024 15:38:15 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 10/15] x86/alternatives: Record text_poke's of JUMP_TYPE_FORCEFUL labels Date: Tue, 19 Nov 2024 16:34:57 +0100 Message-ID: <20241119153502.41361-11-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 660A714002A X-Stat-Signature: hoqfzhonknsuiatxz1o1h1n8yq5uuuyg X-HE-Tag: 1732030694-936654 X-HE-Meta: U2FsdGVkX1/OCbqCXQR9QrFXg7Fs+PpyJ6SJQ3MpRZ5fZODVgsnnxpbmhLdTxLEtwhwRzYe4hXVWeYeiobxHGRQVek1IhqxWcLLVPf4H95y8Qs87B7OBZ62bWt0W6oKe/bSGcijL8Qk6kAl22S0EaMDQ8AUqvcEW6z7GUErpxjR35clNLOEMjRBtHw5A/Ke9LTj+Keup3CYVJsiqXd+FPte7NH85sny4IUzCNeulYgXmxPbsBXoJxTJflHhQ4r5Oa20gY0gqQ1FTPTue8LwzYQK3YQ40ly/MP2JrKNTVbY0EKuHGhMyvLxseTU9OG2wePrxMDaTN2zqLubiD4MTZXonFQAyqMIEctzjRkS1aPRLGPHOOxIikB+bMGAAE+rk5lyxK+fjbgdtrZGr8lS8+gF9pJ1A5x4P8bgzUHYSE2UKnPPAEoeRN5UQX421gQp0sk63mEmOY9Y8mH9z/TGXseNtC4ANnHg4scbc/ZXAxh8h1D5AZAAItEdJPVud7/zjNt7AwKXU9Xj5EX2LaeyqLdt/LuPR8Hd5Oz1iOx6GV2ynpGFd+VpM4oqAnnnrhHhoHlNk1L5AtuHpT1MILqiLOU5zzILzFO2tAjO+GlDGI0mlAcmfmGF0KpUa4uDq2488eE58VXPoDfLdvvVcrjhEXa59poYQxTGm6Nd7bub5zTAhXqvbFiPIIHtvJef8RlsQdesTif9bg5JQnTz31ge6GeaJEq0ETzUGp19djG71hjTKqV1ZC/RmcniCeqk8Vuzi67z8QEoXgt64eh9hb5LS/V9wOUxzxuHNE7XUJFZQ0IFuJxp3CQR0pThLaHESaP86+Q8uXrHy0YiYy6nLcelJCs4eRJEpsh2UdpHRldqy/web7G82orp/PUhxPkTh6imlgRJU8x6/0RQmnLodIiQih7gvnMJvZKtCVeP5svTnV1WBK05xyI76wQMIToWc5tqZf8b/hldDXQlNQm99GmhP sZPJNHY9 vVLuRjoSA8hrnBT9Y217wSNw+dxps2/IlmZwdls2sc9bwFclrTgbuV9juV6YueKlo3+X29cA5nnRq+eFve1O/p5i7to0tmkQxM+uJfugilHAanZVhPfuevwf99FP5JfVya3A/xH9XvSvmy041nYvqCZkxlWBLNyLFEYN0vCUPORlALgMg3oBmDNQ4Jc58NG2gU9OhuaIa0b4Uxvh2YDOnPDb7HCX3V7J7ixLw2Xlm3quMLQlxSNURlV/HZO52BebyCn/QvBWC+F9rhNRb8vNBMZC3PLqu0tk4aIyTtac0Y9fI3bXyLYD98h7d9rO7/OaVudo1HByxpfx2d4/p4YfjKz9p9yL+HbONV08Yrz4aYgHenO5tdN+hclJmUgOPqyxaKOfGMV+6yFrZt+kVf5efxaQCj0xAgsoA37fhGGT9pJzEUIBsfwhpkiFgc4FpmXAesD+LVOGoBMT0Qt6hZYTLNNcXqA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Forceful static keys are used in early entry code where it is unsafe to defer the sync_core() IPIs, and flagged as such via their ->type field. Record that information when creating a text_poke_loc. The text_poke_loc.old field is written to when first iterating a text_poke() entry, and as such can be (ab)used to store this information at the start of text_poke_bp_batch(). Signed-off-by: Valentin Schneider --- arch/x86/include/asm/text-patching.h | 12 ++++++++++-- arch/x86/kernel/alternative.c | 16 ++++++++++------ arch/x86/kernel/jump_label.c | 7 ++++--- 3 files changed, 24 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index 6259f1937fe77..e34de36cab61e 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -38,9 +38,17 @@ extern void *text_poke_copy(void *addr, const void *opcode, size_t len); extern void *text_poke_copy_locked(void *addr, const void *opcode, size_t len, bool core_ok); extern void *text_poke_set(void *addr, int c, size_t len); extern int poke_int3_handler(struct pt_regs *regs); -extern void text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate); +extern void __text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate, bool force_ipi); +static inline void text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate) +{ + __text_poke_bp(addr, opcode, len, emulate, false); +} -extern void text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate); +extern void __text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate, bool force_ipi); +static inline void text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate) +{ + __text_poke_queue(addr, opcode, len, emulate, false); +} extern void text_poke_finish(void); #define INT3_INSN_SIZE 1 diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index d17518ca19b8b..954c4c0f7fc58 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2098,7 +2098,10 @@ struct text_poke_loc { u8 opcode; const u8 text[POKE_MAX_OPCODE_SIZE]; /* see text_poke_bp_batch() */ - u8 old; + union { + u8 old; + u8 force_ipi; + }; }; struct bp_patching_desc { @@ -2385,7 +2388,7 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries } static void text_poke_loc_init(struct text_poke_loc *tp, void *addr, - const void *opcode, size_t len, const void *emulate) + const void *opcode, size_t len, const void *emulate, bool force_ipi) { struct insn insn; int ret, i = 0; @@ -2402,6 +2405,7 @@ static void text_poke_loc_init(struct text_poke_loc *tp, void *addr, tp->rel_addr = addr - (void *)_stext; tp->len = len; tp->opcode = insn.opcode.bytes[0]; + tp->force_ipi = force_ipi; if (is_jcc32(&insn)) { /* @@ -2493,14 +2497,14 @@ void text_poke_finish(void) text_poke_flush(NULL); } -void __ref text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate) +void __ref __text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate, bool force_ipi) { struct text_poke_loc *tp; text_poke_flush(addr); tp = &tp_vec[tp_vec_nr++]; - text_poke_loc_init(tp, addr, opcode, len, emulate); + text_poke_loc_init(tp, addr, opcode, len, emulate, force_ipi); } /** @@ -2514,10 +2518,10 @@ void __ref text_poke_queue(void *addr, const void *opcode, size_t len, const voi * dynamically allocated memory. This function should be used when it is * not possible to allocate memory. */ -void __ref text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate) +void __ref __text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate, bool force_ipi) { struct text_poke_loc tp; - text_poke_loc_init(&tp, addr, opcode, len, emulate); + text_poke_loc_init(&tp, addr, opcode, len, emulate, force_ipi); text_poke_bp_batch(&tp, 1); } diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index f5b8ef02d172c..e03a4f56b30fd 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -101,8 +101,8 @@ __jump_label_transform(struct jump_entry *entry, text_poke_early((void *)jump_entry_code(entry), jlp.code, jlp.size); return; } - - text_poke_bp((void *)jump_entry_code(entry), jlp.code, jlp.size, NULL); + __text_poke_bp((void *)jump_entry_code(entry), jlp.code, jlp.size, NULL, + jump_entry_key(entry)->type & JUMP_TYPE_FORCEFUL); } static void __ref jump_label_transform(struct jump_entry *entry, @@ -135,7 +135,8 @@ bool arch_jump_label_transform_queue(struct jump_entry *entry, mutex_lock(&text_mutex); jlp = __jump_label_patch(entry, type); - text_poke_queue((void *)jump_entry_code(entry), jlp.code, jlp.size, NULL); + __text_poke_queue((void *)jump_entry_code(entry), jlp.code, jlp.size, NULL, + jump_entry_key(entry)->type & JUMP_TYPE_FORCEFUL); mutex_unlock(&text_mutex); return true; } From patchwork Tue Nov 19 15:34:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B405DD44167 for ; Tue, 19 Nov 2024 15:39:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 505556B00C3; Tue, 19 Nov 2024 10:39:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B4D46B00C4; Tue, 19 Nov 2024 10:39:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3085F6B00C5; Tue, 19 Nov 2024 10:39:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 05AE06B00C3 for ; Tue, 19 Nov 2024 10:39:00 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BF8101401D5 for ; Tue, 19 Nov 2024 15:39:00 +0000 (UTC) X-FDA: 82803250656.27.D5B5C1A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf02.hostedemail.com (Postfix) with ESMTP id 8CCA98001E for ; Tue, 19 Nov 2024 15:37:23 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VDW8lpR8; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf02.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030672; a=rsa-sha256; cv=none; b=dc1LLVxCnram5V9Ht0BKeK+GTMWJZAc6UoeTCGkwdEEE40NbcWhkCO6wIePrnGXlo8v6nY RP9/22FPlf+49HZF/+gZtwin8M7GV35m65RsRZGw9YpTixNhO23vmur6+NUkglEa+5PFyl ae6tELAOfQLgHhyuUa0iUxeUrCE/Rnc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VDW8lpR8; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf02.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030672; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gBDFZv8Y0YCT5rYDqfeJn3PBG/oi2+BA7C4xITE08vE=; b=n+/ufvtEwYFrF+etVTvEa0LtXCv+QrnMknMF+ZNe3ZZD7JQEnGoxBP9B9TjY4QTu3K6Cco 0fqElUErAEgAtKEEhndaf2e0g6pPj4SQzEVp1GoD4DhGbqAlLekBAASBgnYJ/jzTJG9fuc twgDHY5A6FHlC+Q770xG2wqkffqFOhU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030738; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gBDFZv8Y0YCT5rYDqfeJn3PBG/oi2+BA7C4xITE08vE=; b=VDW8lpR8GrBIHpRLIQPuFjI6AMOKG/16UqCNNf7WnztkQc3Za+WL6/7lJZKwLn+aOM7uID u3nM81CdrZQvvuX2/wxjoTgJJ61mwCYf60+GZykMYSU5QaycDF0awnM0ReXzqOUJP4c86g UpgSA78VTJVVvvixergzBAsKdvt3r9w= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-146-YBGQR59uOBaV7Ovj9k6N-g-1; Tue, 19 Nov 2024 10:38:56 -0500 X-MC-Unique: YBGQR59uOBaV7Ovj9k6N-g-1 X-Mimecast-MFC-AGG-ID: YBGQR59uOBaV7Ovj9k6N-g Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 466291952D29; Tue, 19 Nov 2024 15:38:49 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 569B930001A2; Tue, 19 Nov 2024 15:38:33 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Nicolas Saenz Julienne , Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 11/15] context-tracking: Introduce work deferral infrastructure Date: Tue, 19 Nov 2024 16:34:58 +0100 Message-ID: <20241119153502.41361-12-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspam-User: X-Rspamd-Queue-Id: 8CCA98001E X-Rspamd-Server: rspam01 X-Stat-Signature: hppnwp4f1bfmbmorwhwz63fbs996bang X-HE-Tag: 1732030643-389865 X-HE-Meta: U2FsdGVkX1+Vu2wFfLQuB3OoOiJsm55Kyndslfbhi1HRkpjMWUjln/N6OVdu9HrUliwmfcGzYv5aiYdPJPL8PayiUTP50nQny4Z/92Sq+HfWkl3QlP4cJdxjms5V3tilcy+pvBATYxDHUBgCDTQ32yKgx8ngducQ28gEn8jEQIfgdntKDmtWStKs1MHtww6+uI0fUVt5npFV61MxR4jAvrQmhf0iCjIDkbh+r9+0sF+sykmnfKt2GfESy09qxkDCPBUymPNMRNHWbZY2T4qE2G3ZDUvB6ym5nyNbuRESGJ2RsMQHCB4reEa1YrleuVol7jmBMGO7wD/VPGYbSchROebRGBEdRntrac04jrxU2pVkNKU5FwmOY5on7QAOm2POkKmwq4Eaex8TCtm1d4M2PjmDUsiByxXUfFIfJOdS8Kb1D/uKrUS1GHjcBFpsfCyDWynAT7y7t8GsTgRyTwrS0uIB6s5jK3os3cY1BDOsRa71DCCSv6TM2N4p1Va5iLutxbqQOckKzLCw6PwRTiTDiWIMsGmINlUQdoXNYGTxGuoCCYosc9mMifgl00V401HeytwRoIS9JhnCwGkz77xjx4sQ+VRnyaR5AAwwrYnVlBg+SMI7ZZ67iNOuEqJlJx5BNbFJUeMd+iXVs5OAZdvwKX3n19znoDxlQhkTgP0FvJM9bEQ9des/iaqdOdnImU8vN5rJw5tFGi2PdilSr16i1Sxy0PdhjyEDllahpetbia567VxYN82eStepdG+PhVVoosZjNxQk/QtEwztajnuOC8qSyLIyD2yFieUZL8JqOvb9nsgL48h8cogdybZVXHbLDR4cfbGOMMCFETDduXVgE1YTvlBxnOlBU02w5HIyhK+cViaA1tL1GADMPG5rjadAGs045N6G3cyQy7ZWcj4vegGHCOk2BNZcPBzlLehS4C3/ivnvME9whA2cKkxEPvX0JrrGPuDFx0bI0cbwv9W 3t0j2dBw 6B0wrOozrKSzF9/ZB03qPHN0nOdHwGcwKai8LTZu2gP7y263w5XIE2s/AeYY1bgp1GtlRy8CiThawjQbahWdQ+9Fu0CaPqo/pdz7h426wtb5ojN4JkrGG3LXmHnXE7DljJ8yDkpI4cWRFIG0cnzvCFR+O0ZlPlo266tAW35z7emDwAKZ9IKC42z5TRqWdkabnmKMwxvrLLk3qcTv/HITvQoqZoxzsVxCrgFhovQg9sWxGtxB2M2LmhYqoFIyRwCr3CXCNctCe0Ajmj2XdAO8t7NucTDG5M1ItweET3b9CnRgKfnNjPyTe8jk1zhNglWAb72mTmgReHNU2a9KdZZ0FNZbVAI2Ew8B4bL8oYoDHDKqRvmev+fuUUU6wP73KYvRzvm/wGkxAcMRXLvQqzYaff7/LOUBCONiajd+5Kpnyusdazdn7RdkBV8D3GY9LqjoUV5YEljeXaAclC0ijFXvnzcjjZPITgYO7P6n22hpXc1V2S/ol6JvWDsa7WQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: smp_call_function() & friends have the unfortunate habit of sending IPIs to isolated, NOHZ_FULL, in-userspace CPUs, as they blindly target all online CPUs. Some callsites can be bent into doing the right, such as done by commit: cc9e303c91f5 ("x86/cpu: Disable frequency requests via aperfmperf IPI for nohz_full CPUs") Unfortunately, not all SMP callbacks can be omitted in this fashion. However, some of them only affect execution in kernelspace, which means they don't have to be executed *immediately* if the target CPU is in userspace: stashing the callback and executing it upon the next kernel entry would suffice. x86 kernel instruction patching or kernel TLB invalidation are prime examples of it. Reduce the RCU dynticks counter width to free up some bits to be used as a deferred callback bitmask. Add some build-time checks to validate that setup. Presence of CT_STATE_KERNEL in the ct_state prevents queuing deferred work. Later commits introduce the bit:callback mappings. Link: https://lore.kernel.org/all/20210929151723.162004989@infradead.org/ Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Valentin Schneider --- arch/Kconfig | 9 ++++ arch/x86/Kconfig | 1 + arch/x86/include/asm/context_tracking_work.h | 14 ++++++ include/linux/context_tracking.h | 21 +++++++++ include/linux/context_tracking_state.h | 30 ++++++++----- include/linux/context_tracking_work.h | 26 +++++++++++ kernel/context_tracking.c | 46 +++++++++++++++++++- kernel/time/Kconfig | 5 +++ 8 files changed, 140 insertions(+), 12 deletions(-) create mode 100644 arch/x86/include/asm/context_tracking_work.h create mode 100644 include/linux/context_tracking_work.h diff --git a/arch/Kconfig b/arch/Kconfig index bd9f095d69fa0..e7f3f797a34a4 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -912,6 +912,15 @@ config HAVE_CONTEXT_TRACKING_USER_OFFSTACK - No use of instrumentation, unless instrumentation_begin() got called. +config HAVE_CONTEXT_TRACKING_WORK + bool + help + Architecture supports deferring work while not in kernel context. + This is especially useful on setups with isolated CPUs that might + want to avoid being interrupted to perform housekeeping tasks (for + ex. TLB invalidation or icache invalidation). The housekeeping + operations are performed upon re-entering the kernel. + config HAVE_TIF_NOHZ bool help diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 16354dfa6d965..c703376dd326b 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -213,6 +213,7 @@ config X86 select HAVE_CMPXCHG_LOCAL select HAVE_CONTEXT_TRACKING_USER if X86_64 select HAVE_CONTEXT_TRACKING_USER_OFFSTACK if HAVE_CONTEXT_TRACKING_USER + select HAVE_CONTEXT_TRACKING_WORK if X86_64 select HAVE_C_RECORDMCOUNT select HAVE_OBJTOOL_MCOUNT if HAVE_OBJTOOL select HAVE_OBJTOOL_NOP_MCOUNT if HAVE_OBJTOOL_MCOUNT diff --git a/arch/x86/include/asm/context_tracking_work.h b/arch/x86/include/asm/context_tracking_work.h new file mode 100644 index 0000000000000..5bc29e6b2ed38 --- /dev/null +++ b/arch/x86/include/asm/context_tracking_work.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_CONTEXT_TRACKING_WORK_H +#define _ASM_X86_CONTEXT_TRACKING_WORK_H + +static __always_inline void arch_context_tracking_work(int work) +{ + switch (work) { + case CONTEXT_WORK_n: + // Do work... + break; + } +} + +#endif diff --git a/include/linux/context_tracking.h b/include/linux/context_tracking.h index af9fe87a09225..16a2eb7525f1f 100644 --- a/include/linux/context_tracking.h +++ b/include/linux/context_tracking.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -137,6 +138,26 @@ static __always_inline unsigned long ct_state_inc(int incby) return raw_atomic_add_return(incby, this_cpu_ptr(&context_tracking.state)); } +#ifdef CONTEXT_TRACKING_WORK +static __always_inline unsigned long ct_state_inc_clear_work(int incby) +{ + struct context_tracking *ct = this_cpu_ptr(&context_tracking); + unsigned long new, old, state; + + state = arch_atomic_read(&ct->state); + do { + old = state; + new = old & ~CONTEXT_WORK_MASK; + new += incby; + state = arch_atomic_cmpxchg(&ct->state, old, new); + } while (old != state); + + return new; +} +#else +#define ct_state_inc_clear_work(x) ct_state_inc(x) +#endif + static __always_inline bool warn_rcu_enter(void) { bool ret = false; diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h index 0b81248aa03e2..d1d37dbdf7195 100644 --- a/include/linux/context_tracking_state.h +++ b/include/linux/context_tracking_state.h @@ -5,6 +5,7 @@ #include #include #include +#include /* Offset to allow distinguishing irq vs. task-based idle entry/exit. */ #define CT_NESTING_IRQ_NONIDLE ((LONG_MAX / 2) + 1) @@ -39,16 +40,19 @@ struct context_tracking { }; /* - * We cram two different things within the same atomic variable: + * We cram up to three different things within the same atomic variable: * - * CT_RCU_WATCHING_START CT_STATE_START - * | | - * v v - * MSB [ RCU watching counter ][ context_state ] LSB - * ^ ^ - * | | - * CT_RCU_WATCHING_END CT_STATE_END + * CT_RCU_WATCHING_START CT_STATE_START + * | CT_WORK_START | + * | | | + * v v v + * MSB [ RCU watching counter ][ context work ][ context_state ] LSB + * ^ ^ ^ + * | | | + * | CT_WORK_END | + * CT_RCU_WATCHING_END CT_STATE_END * + * The [ context work ] region spans 0 bits if CONFIG_CONTEXT_WORK=n * Bits are used from the LSB upwards, so unused bits (if any) will always be in * upper bits of the variable. */ @@ -59,18 +63,24 @@ struct context_tracking { #define CT_STATE_START 0 #define CT_STATE_END (CT_STATE_START + CT_STATE_WIDTH - 1) -#define CT_RCU_WATCHING_MAX_WIDTH (CT_SIZE - CT_STATE_WIDTH) +#define CT_WORK_WIDTH (IS_ENABLED(CONFIG_CONTEXT_TRACKING_WORK) ? CONTEXT_WORK_MAX_OFFSET : 0) +#define CT_WORK_START (CT_STATE_END + 1) +#define CT_WORK_END (CT_WORK_START + CT_WORK_WIDTH - 1) + +#define CT_RCU_WATCHING_MAX_WIDTH (CT_SIZE - CT_WORK_WIDTH - CT_STATE_WIDTH) #define CT_RCU_WATCHING_WIDTH (IS_ENABLED(CONFIG_RCU_DYNTICKS_TORTURE) ? 2 : CT_RCU_WATCHING_MAX_WIDTH) -#define CT_RCU_WATCHING_START (CT_STATE_END + 1) +#define CT_RCU_WATCHING_START (CT_WORK_END + 1) #define CT_RCU_WATCHING_END (CT_RCU_WATCHING_START + CT_RCU_WATCHING_WIDTH - 1) #define CT_RCU_WATCHING BIT(CT_RCU_WATCHING_START) #define CT_STATE_MASK GENMASK(CT_STATE_END, CT_STATE_START) +#define CT_WORK_MASK GENMASK(CT_WORK_END, CT_WORK_START) #define CT_RCU_WATCHING_MASK GENMASK(CT_RCU_WATCHING_END, CT_RCU_WATCHING_START) #define CT_UNUSED_WIDTH (CT_RCU_WATCHING_MAX_WIDTH - CT_RCU_WATCHING_WIDTH) static_assert(CT_STATE_WIDTH + + CT_WORK_WIDTH + CT_RCU_WATCHING_WIDTH + CT_UNUSED_WIDTH == CT_SIZE); diff --git a/include/linux/context_tracking_work.h b/include/linux/context_tracking_work.h new file mode 100644 index 0000000000000..fb74db8876dd2 --- /dev/null +++ b/include/linux/context_tracking_work.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_CONTEXT_TRACKING_WORK_H +#define _LINUX_CONTEXT_TRACKING_WORK_H + +#include + +enum { + CONTEXT_WORK_n_OFFSET, + CONTEXT_WORK_MAX_OFFSET +}; + +enum ct_work { + CONTEXT_WORK_n = BIT(CONTEXT_WORK_n_OFFSET), + CONTEXT_WORK_MAX = BIT(CONTEXT_WORK_MAX_OFFSET) +}; + +#include + +#ifdef CONFIG_CONTEXT_TRACKING_WORK +extern bool ct_set_cpu_work(unsigned int cpu, unsigned int work); +#else +static inline bool +ct_set_cpu_work(unsigned int cpu, unsigned int work) { return false; } +#endif + +#endif diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c index 938c48952d265..37b094ea56fb6 100644 --- a/kernel/context_tracking.c +++ b/kernel/context_tracking.c @@ -72,6 +72,47 @@ static __always_inline void rcu_task_trace_heavyweight_exit(void) #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */ } +#ifdef CONFIG_CONTEXT_TRACKING_WORK +static noinstr void ct_work_flush(unsigned long seq) +{ + int bit; + + seq = (seq & CT_WORK_MASK) >> CT_WORK_START; + + /* + * arch_context_tracking_work() must be noinstr, non-blocking, + * and NMI safe. + */ + for_each_set_bit(bit, &seq, CONTEXT_WORK_MAX) + arch_context_tracking_work(BIT(bit)); +} + +bool ct_set_cpu_work(unsigned int cpu, unsigned int work) +{ + struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu); + unsigned int old; + bool ret = false; + + preempt_disable(); + + old = atomic_read(&ct->state); + /* + * Try setting the work until either + * - the target CPU has entered kernelspace + * - the work has been set + */ + do { + ret = atomic_try_cmpxchg(&ct->state, &old, old | (work << CT_WORK_START)); + } while (!ret && ((old & CT_STATE_MASK) != CT_STATE_KERNEL)); + + preempt_enable(); + return ret; +} +#else +static __always_inline void ct_work_flush(unsigned long work) { } +static __always_inline void ct_work_clear(struct context_tracking *ct) { } +#endif + /* * Record entry into an extended quiescent state. This is only to be * called when not already in an extended quiescent state, that is, @@ -88,7 +129,7 @@ static noinstr void ct_kernel_exit_state(int offset) * next idle sojourn. */ rcu_task_trace_heavyweight_enter(); // Before CT state update! - seq = ct_state_inc(offset); + seq = ct_state_inc_clear_work(offset); // RCU is no longer watching. Better be in extended quiescent state! WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && (seq & CT_RCU_WATCHING)); } @@ -100,7 +141,7 @@ static noinstr void ct_kernel_exit_state(int offset) */ static noinstr void ct_kernel_enter_state(int offset) { - int seq; + unsigned long seq; /* * CPUs seeing atomic_add_return() must see prior idle sojourns, @@ -108,6 +149,7 @@ static noinstr void ct_kernel_enter_state(int offset) * critical section. */ seq = ct_state_inc(offset); + ct_work_flush(seq); // RCU is now watching. Better not be in an extended quiescent state! rcu_task_trace_heavyweight_exit(); // After CT state update! WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !(seq & CT_RCU_WATCHING)); diff --git a/kernel/time/Kconfig b/kernel/time/Kconfig index 8ebb6d5a106be..04efc2b605823 100644 --- a/kernel/time/Kconfig +++ b/kernel/time/Kconfig @@ -186,6 +186,11 @@ config CONTEXT_TRACKING_USER_FORCE Say N otherwise, this option brings an overhead that you don't want in production. +config CONTEXT_TRACKING_WORK + bool + depends on HAVE_CONTEXT_TRACKING_WORK && CONTEXT_TRACKING_USER + default y + config NO_HZ bool "Old Idle dynticks config" help From patchwork Tue Nov 19 15:34:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61A2DD44167 for ; Tue, 19 Nov 2024 15:39:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF04F6B00C5; Tue, 19 Nov 2024 10:39:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E79906B00C6; Tue, 19 Nov 2024 10:39:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CCC076B00C7; Tue, 19 Nov 2024 10:39:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AB1CE6B00C5 for ; Tue, 19 Nov 2024 10:39:15 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 729BB1C6AAF for ; Tue, 19 Nov 2024 15:39:15 +0000 (UTC) X-FDA: 82803250656.09.42D5050 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf20.hostedemail.com (Postfix) with ESMTP id B856B1C0004 for ; Tue, 19 Nov 2024 15:38:10 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DPkzAr+4; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030570; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gHLzoO2lla+YiGjlL3mjjdCwCDcrDiVLGjLeux29GIU=; b=zxMneV3B0WEDhrlnYJnkc0ZWJX4bZGZm+x7Gxj5yYRx+0MFRKwnioOKvfmSjIkxnOmN3Ee CuMzeienJtKbP2Ir91nKEDxql2ACT47Mt+PEmXchUYFHmpMwHI2fcy3xnOZXTAbJVoXQru cowepMakE3ybvnfVVXr6UBXIoUKGKVM= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DPkzAr+4; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030570; a=rsa-sha256; cv=none; b=zZxIxmsCqUxsi6iB59hH5rD7jk8kw6pdipQQRezjXslEvr6mAl4UvISyELlje2tEq9+m/1 rI6W2IrFeOKy2jyA19ovPA/BVFTG/XV1GMSywSiNFBpoi33wA5LeGC3M+h+I+JE2GzUZAT B4gGUWgJ2s2T/cnRDC8SpErMBnkWzmg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030753; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gHLzoO2lla+YiGjlL3mjjdCwCDcrDiVLGjLeux29GIU=; b=DPkzAr+4ZJOsOw/RHn7CzJMEHypYk2olQYjM+dGzJQFRBHQdCt09OoTlsrSFAsxb8I/Xd8 EMXGAWf9HnNkD6HrYvrw//eDWDMJIx7nsLzzFoObGDbmFpsW4Mizia0Bk4RFVcY0K5wRSl O1VcxD6v2VVk0rJwp8J1ibMIALngnhk= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-467-Mc4eVWE3PlutRNpeiXlJfw-1; Tue, 19 Nov 2024 10:39:11 -0500 X-MC-Unique: Mc4eVWE3PlutRNpeiXlJfw-1 X-Mimecast-MFC-AGG-ID: Mc4eVWE3PlutRNpeiXlJfw Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0E26E1955EA3; Tue, 19 Nov 2024 15:39:06 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B578D3003B7E; Tue, 19 Nov 2024 15:38:49 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Peter Zijlstra , Nicolas Saenz Julienne , Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 12/15] context_tracking,x86: Defer kernel text patching IPIs Date: Tue, 19 Nov 2024 16:34:59 +0100 Message-ID: <20241119153502.41361-13-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B856B1C0004 X-Stat-Signature: 6yjmg8dpjet7x5o7f5y9hbeq3d1f935s X-Rspam-User: X-HE-Tag: 1732030690-645807 X-HE-Meta: U2FsdGVkX1+MqekvGeiZbstYUt1Oh4SIonG1qo5GKyK3uyqaXwdO45oPNllVNlz4lfnw6gVjY2AYSbgkAA0cZ8B/JCOZc9hrO68drKzxN9QCpmG/3I6HjMvUdYXEcKRuEBdYAfvH1dfXGa3kkpdZ2IhrwqvW713CXFInQsKFWzYbhA9XKYmq/56yVQ56O6FKKkGOm12VpoSVtwmmqEryKjsC+H0V0+rl5xXnlHz5PsorJQldafL1n9YjHYV1iAUdNa+ZLgR1N1qIzuogYmVX2W7mWAxV4jOUGHSCsYDp99g/5jUhjGyAaYo1VCnKvpaRAxeRsilT3+NLvcHjr/XRi8IhdnAm1lPkRNkWZU12Af7wIrUdDa6nw0eAQzgjG6LfTQqVH9NuZ2h4sy6GW9d8jjcRVDmBP17vvAZcoTV1YEtP6SgrfA9dRsacMnJXo4TN9IsrMNICpyuRVuAyJETSw72+viyDH7StkUFWgYCtNDpODmkUcmbkgK3qUpxvAeDmEMGGTBQ760Ot6Cq+SbQIoOI9tgO1+EFtjMurBaJh8etx0cUURtwbFY8ez1ibwsXYinqIUEDmLqo8I6YrphmGkBs0XPx5u7odPA4MgSHjVy67uD/SKzjW0Ml3AMYMEcYL2mG+aK2SydzcjeLIMnlli/gAi7BHQ7+1Y2+vO5l/RfEne5YuoTKhadQODhhIxTUVCdDIICGFpkB24zhkb8l0mo7wNsMBr7frNHrTctYzqnEkKBlSQiYrWbMPof8nIJegm4qY+a62F1D4LbSM7lTc8MfM91j+bLzbZ2PZ8WcoxF0b0t4Vrd3Nd04IYjucLT8nmwA+9XUsuhqmo0vHRWeZNpNipQYwL6g7rRwK86Ps5YMaNxKEJE25TI3u+pFP40nkEZG2VUMk3yucPdGkCo8qFskyXP4xJ9lHOQtHT2rOyMGG1Crl7znuJLlimY7XyCx3fpSjDvnO69PBTcB8gSn OADf7FQo oqX1wtW5E9NSS5nzNTV6at14/VE9rBaZM5OMYluLAx3TZoIRUm82I2mm+Qs2Cq9vpYi2uSWwRddWlV80W6PcS+3SCbH2imtxfEq6O8FuGrmBRuCAXSXLbv3hQkqb96NGbAsa3nw573GsdDxa6bzXzdzQ+oZ7lMG1kOM9tB9ZqEuojBblIqOioejH7P33I01h/l6GHpr5PmVag3TLPWwIzo+fn/ZC5mwLE20Ph76GRT3rPa+TXjmjhvkGs2Rv7kQyWeosChRIr2q1s/0gZo2JlbIscB8r2z5PB2RElrLsvl8vSD25RjBADSqi59HGOZ8bMjGMTn8MMxpNZnRuCrT/YvcAItLQ4g1Bf8efBscPeUJzYFIwyOLbYHihlIq0Rkb02lyO4VoNafS+krnq2RCpoMU8UVtlCc3andxL4pi6NFTWsfPrczJF4MVdnsmQ2OJifZObRToveSKpyrcqy3BiBBlXlTTovYYcon3xwoa+Z8dIiusfYfnSlmk0Nj2En7+xeVaWf+NnkFhU4ZV0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: text_poke_bp_batch() sends IPIs to all online CPUs to synchronize them vs the newly patched instruction. CPUs that are executing in userspace do not need this synchronization to happen immediately, and this is actually harmful interference for NOHZ_FULL CPUs. As the synchronization IPIs are sent using a blocking call, returning from text_poke_bp_batch() implies all CPUs will observe the patched instruction(s), and this should be preserved even if the IPI is deferred. In other words, to safely defer this synchronization, any kernel instruction leading to the execution of the deferred instruction sync (ct_work_flush()) must *not* be mutable (patchable) at runtime. This means we must pay attention to mutable instructions in the early entry code: - alternatives - static keys - all sorts of probes (kprobes/ftrace/bpf/???) The early entry code leading to ct_work_flush() is noinstr, which gets rid of the probes. Alternatives are safe, because it's boot-time patching (before SMP is even brought up) which is before any IPI deferral can happen. This leaves us with static keys. Any static key used in early entry code should be only forever-enabled at boot time, IOW __ro_after_init (pretty much like alternatives). Exceptions are marked as forceful and will always generate an IPI when flipped. Objtool is now able to point at static keys that don't respect this, and all static keys used in early entry code have now been verified as behaving like so. Leverage the new context_tracking infrastructure to defer sync_core() IPIs to a target CPU's next kernel entry. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Valentin Schneider --- arch/x86/include/asm/context_tracking_work.h | 6 ++-- arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 33 +++++++++++++++++--- arch/x86/kernel/kprobes/core.c | 4 +-- arch/x86/kernel/kprobes/opt.c | 4 +-- arch/x86/kernel/module.c | 2 +- include/linux/context_tracking_work.h | 4 +-- 7 files changed, 41 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/context_tracking_work.h b/arch/x86/include/asm/context_tracking_work.h index 5bc29e6b2ed38..2c66687ce00e2 100644 --- a/arch/x86/include/asm/context_tracking_work.h +++ b/arch/x86/include/asm/context_tracking_work.h @@ -2,11 +2,13 @@ #ifndef _ASM_X86_CONTEXT_TRACKING_WORK_H #define _ASM_X86_CONTEXT_TRACKING_WORK_H +#include + static __always_inline void arch_context_tracking_work(int work) { switch (work) { - case CONTEXT_WORK_n: - // Do work... + case CONTEXT_WORK_SYNC: + sync_core(); break; } } diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index e34de36cab61e..37344e95f738f 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -33,6 +33,7 @@ extern void apply_relocation(u8 *buf, const u8 * const instr, size_t instrlen, u */ extern void *text_poke(void *addr, const void *opcode, size_t len); extern void text_poke_sync(void); +extern void text_poke_sync_deferrable(void); extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern void *text_poke_copy(void *addr, const void *opcode, size_t len); extern void *text_poke_copy_locked(void *addr, const void *opcode, size_t len, bool core_ok); diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 954c4c0f7fc58..4ce224d927b03 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -2080,9 +2081,24 @@ static void do_sync_core(void *info) sync_core(); } +static bool do_sync_core_defer_cond(int cpu, void *info) +{ + return !ct_set_cpu_work(cpu, CONTEXT_WORK_SYNC); +} + +static void __text_poke_sync(smp_cond_func_t cond_func) +{ + on_each_cpu_cond(cond_func, do_sync_core, NULL, 1); +} + void text_poke_sync(void) { - on_each_cpu(do_sync_core, NULL, 1); + __text_poke_sync(NULL); +} + +void text_poke_sync_deferrable(void) +{ + __text_poke_sync(do_sync_core_defer_cond); } /* @@ -2257,6 +2273,8 @@ static int tp_vec_nr; static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries) { unsigned char int3 = INT3_INSN_OPCODE; + bool force_ipi = false; + void (*sync_fn)(void); unsigned int i; int do_sync; @@ -2291,11 +2309,18 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries * First step: add a int3 trap to the address that will be patched. */ for (i = 0; i < nr_entries; i++) { + /* + * Record that we need to send the IPI if at least one location + * in the batch requires it. + */ + force_ipi |= tp[i].force_ipi; tp[i].old = *(u8 *)text_poke_addr(&tp[i]); text_poke(text_poke_addr(&tp[i]), &int3, INT3_INSN_SIZE); } - text_poke_sync(); + sync_fn = force_ipi ? text_poke_sync : text_poke_sync_deferrable; + + sync_fn(); /* * Second step: update all but the first byte of the patched range. @@ -2357,7 +2382,7 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries * not necessary and we'd be safe even without it. But * better safe than sorry (plus there's not only Intel). */ - text_poke_sync(); + sync_fn(); } /* @@ -2378,7 +2403,7 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries } if (do_sync) - text_poke_sync(); + sync_fn(); /* * Remove and wait for refs to be zero. diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 72e6a45e7ec24..c2fd2578fd5fc 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -817,7 +817,7 @@ void arch_arm_kprobe(struct kprobe *p) u8 int3 = INT3_INSN_OPCODE; text_poke(p->addr, &int3, 1); - text_poke_sync(); + text_poke_sync_deferrable(); perf_event_text_poke(p->addr, &p->opcode, 1, &int3, 1); } @@ -827,7 +827,7 @@ void arch_disarm_kprobe(struct kprobe *p) perf_event_text_poke(p->addr, &int3, 1, &p->opcode, 1); text_poke(p->addr, &p->opcode, 1); - text_poke_sync(); + text_poke_sync_deferrable(); } void arch_remove_kprobe(struct kprobe *p) diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c index 36d6809c6c9e1..b2ce4d9c3ba56 100644 --- a/arch/x86/kernel/kprobes/opt.c +++ b/arch/x86/kernel/kprobes/opt.c @@ -513,11 +513,11 @@ void arch_unoptimize_kprobe(struct optimized_kprobe *op) JMP32_INSN_SIZE - INT3_INSN_SIZE); text_poke(addr, new, INT3_INSN_SIZE); - text_poke_sync(); + text_poke_sync_deferrable(); text_poke(addr + INT3_INSN_SIZE, new + INT3_INSN_SIZE, JMP32_INSN_SIZE - INT3_INSN_SIZE); - text_poke_sync(); + text_poke_sync_deferrable(); perf_event_text_poke(op->kp.addr, old, JMP32_INSN_SIZE, new, JMP32_INSN_SIZE); } diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index 837450b6e882f..00e71ad30c01d 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -191,7 +191,7 @@ static int write_relocate_add(Elf64_Shdr *sechdrs, write, apply); if (!early) { - text_poke_sync(); + text_poke_sync_deferrable(); mutex_unlock(&text_mutex); } diff --git a/include/linux/context_tracking_work.h b/include/linux/context_tracking_work.h index fb74db8876dd2..13fc97b395030 100644 --- a/include/linux/context_tracking_work.h +++ b/include/linux/context_tracking_work.h @@ -5,12 +5,12 @@ #include enum { - CONTEXT_WORK_n_OFFSET, + CONTEXT_WORK_SYNC_OFFSET, CONTEXT_WORK_MAX_OFFSET }; enum ct_work { - CONTEXT_WORK_n = BIT(CONTEXT_WORK_n_OFFSET), + CONTEXT_WORK_SYNC = BIT(CONTEXT_WORK_SYNC_OFFSET), CONTEXT_WORK_MAX = BIT(CONTEXT_WORK_MAX_OFFSET) }; From patchwork Tue Nov 19 15:35:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92C4BD44167 for ; Tue, 19 Nov 2024 15:39:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 220566B00C7; Tue, 19 Nov 2024 10:39:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1CF5C6B00C8; Tue, 19 Nov 2024 10:39:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3E246B00C9; Tue, 19 Nov 2024 10:39:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D2BA16B00C7 for ; Tue, 19 Nov 2024 10:39:32 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 96FBD140595 for ; Tue, 19 Nov 2024 15:39:32 +0000 (UTC) X-FDA: 82803253218.04.8269AD9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf05.hostedemail.com (Postfix) with ESMTP id 8180010000A for ; Tue, 19 Nov 2024 15:37:55 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=L7Unmtra; spf=pass (imf05.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030679; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XZhR8LJzRJ/NA3355q22R2um1DqV6KheeHOnZlXN8BQ=; b=xmLvkqkhhUQO04EHs9ZQZRmXShYfedsSdW4+VU5hXE9mY+Fx5fMtBpTUnEUtSOEMEM7Z3X NCLUMk91z4MeZi6YN+YNi2gybWNcrP3tY065iWXMIL+kLvhEZxbjurk2gi9hvYjsQkUp9F tWaiqrA3FkDw32u36wbwvd6IQkb6Co8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030679; a=rsa-sha256; cv=none; b=MnwLwAxeOrQuVeouYqKGlSmt6vM7LwXaUuSVWyLoTmlWznrBKwuf+8DxTCD6T1OHCuC+l4 QP9JOlDja6Rj6gpH6HRdJKheHdtTGh06SrTr6AiStFMGp6DYsTcz6m1UjU7aUW0eRd3F5E L28D2wEotlt7OIdO4E0eI0QJgR3EAFY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=L7Unmtra; spf=pass (imf05.hostedemail.com: domain of vschneid@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030770; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XZhR8LJzRJ/NA3355q22R2um1DqV6KheeHOnZlXN8BQ=; b=L7UnmtraAWzVIFxFLxJ3eS78EWFNfGgYnfQ3SsLeTALiEGwfPlNmXxATt5hboWRMqgY0uZ lvbXDO9pSuGo2li82Y8Ao4vyzqfDc3q4G9F7Nx50fRPtLsqnKqPkT8xhg7ehMJBT8Ueb3z 4Jaq9T3rqaNFHstrfX3DDDv+pZSpec8= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-620-EyVM_Q77Oy2qLzT--XqbYw-1; Tue, 19 Nov 2024 10:39:27 -0500 X-MC-Unique: EyVM_Q77Oy2qLzT--XqbYw-1 X-Mimecast-MFC-AGG-ID: EyVM_Q77Oy2qLzT--XqbYw Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 209A21955F2C; Tue, 19 Nov 2024 15:39:22 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7E38F30001A0; Tue, 19 Nov 2024 15:39:06 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 13/15] context_tracking,x86: Add infrastructure to defer kernel TLBI Date: Tue, 19 Nov 2024 16:35:00 +0100 Message-ID: <20241119153502.41361-14-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Server: rspam10 X-Stat-Signature: 191d7p8idsyrzuisrsouyu6e5yxa7sys X-Rspamd-Queue-Id: 8180010000A X-Rspam-User: X-HE-Tag: 1732030675-531152 X-HE-Meta: U2FsdGVkX19uobxVlpnl9PE4B+9gBwwXknNd7fn7tv3ATZWFhFt3ryO1V4pb+qZAAyaP6r0iyqs62MZ2AFplh5nqtj06amHOYTX9tgwTFM+nXtSbT0+pkRLSnSNtM/lAsd+rOOhLsZD9zVcn9q1vO2+3ZsAkxmGVC1rTEy+PdHtx9E3jKKpAx+st1O4jtHhln7Q4QNs7C1sShlsrloYewqQEdi4Vd2TwX2hKSAXG2JxZvqyG9lfr0zOwFDuAKH1yaP8hp26mbAjO/Ew688s+8SN3Ei39qksdUENdNsIdVWjPmEG9WolHSJXT3sv7gXdLcLK+m+I2mIK4Sa6Yod+7xl2obEErcLeRfMCiO9hGyjD3BgsFVjq7yTcKG/tk7HKuztnCtb7JSlw1mslkI7yd7FKUBe3KN+ayXLQCIddm24pQ2S7yLFjq96JFFR1h9xAlCGYeyyxYszsC7uNIoIZH2Z8kb5somtV1mvduU8AdSX2JGlsktMTlhPLC8P1cd2kIHw3VDTrgv793+k4NrSzMoVYvf20m1hntLYMm3pL6apNAucBjY42ssl3ZtaNKtFtCmqedIWagTfLHPtXT1XYFQCRo/+STsxLTz48tTRmaKKiZbrzrQ3e0Si7Shq7mdrYKLmxngsxcc6kWuGvqNwxGmpndlmT2U7DmQKEYbw/MADBchSL2wLsZ4sQ/5brGKHtxvLZ+mdokmkGCNrZmbtRBIVNsmp+rrLA47yG+e65zyR76WAXjFd21Ix3jzp+gEmRhOlvhxRGZWWpCw1GYyvG+XlN/GiS40TS2RNbC5xKCzBjK1uuJ0qULeUTtmDlKpL4+0mpcBsKiyIwlbTJ8ul4jw3o9D+3pMjIrigMPazCLbu8Vawwo9LKZ7JrOiOmqMXjiVFUHn3RzGH57aLDRcGDn+pusKSHYAd+zWj/mr78hhrBAV+3InQeiBXlXfBEia+pluTJeFY2QPAWEvcJtCS+ +zIxgSCP F7sIIdGcHqeSZl0kIYmpOgnUMnx+u/w1MVXGONpvaOLs3wcrU86nte6s+YxMV34kTVTUFhB7WySad5qxrOPBSkHkRCYyfHCxMdwtY5ikEiMA6lmRv+wnuuqwJyL52V2/VjOeFOhDf7X8ORrk/vYBoA4ryD9WWZT2HiREoyRYvFUy4REGcSWqS31aCGTQ/YVzpcqQ5s+Jk79Ng5ABm7dunu5ntBzoTRtsYgKrYKFKBsS4xQr8jL1oHuNa5CJqkehrPnaN8/JIYxsyrgUSb34/b+n2yTXJbkuOo8Slk3jScBRDnbfKQPmTW2g+eiB02vQ1EB9N33A5p9eU7FKagblmIihDoa4LGv49bCoEd05ctfq+PlTKsWqdp0zgutW1vvBQQaQNNBfP4siw06hiv5UbfinCbrYkZ/Bhc+eJzVF6E2K/64wUBZmgS5PFvtGwVzY7zr5+hv6uCMAIANele+W0hLsmc3w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Kernel TLB invalidation IPIs are a common source of interference on NOHZ_FULL CPUs. Given NOHZ_FULL CPUs executing in userspace are not accessing any kernel addresses, these invalidations do not need to happen immediately, and can be deferred until the next user->kernel transition. Add a minimal, noinstr-compliant variant of __flush_tlb_all() that doesn't try to leverage INVPCID. To achieve this: o Add a noinstr variant of native_write_cr4(), keeping native_write_cr4() as "only" __no_profile. o Make invalidate_user_asid() __always_inline XXX: what about paravirt? Should these be instead new operations made available through pv_ops.mmu.*? x86_64_start_kernel() uses __native_tlb_flush_global() regardless of paravirt, so I'm thinking the paravirt variants are more optimizations than hard requirements? Signed-off-by: Valentin Schneider --- arch/x86/include/asm/context_tracking_work.h | 4 +++ arch/x86/include/asm/special_insns.h | 1 + arch/x86/include/asm/tlbflush.h | 16 ++++++++++-- arch/x86/kernel/cpu/common.c | 6 ++++- arch/x86/mm/tlb.c | 26 ++++++++++++++++++-- include/linux/context_tracking_work.h | 2 ++ 6 files changed, 50 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/context_tracking_work.h b/arch/x86/include/asm/context_tracking_work.h index 2c66687ce00e2..9d4f021b5a45b 100644 --- a/arch/x86/include/asm/context_tracking_work.h +++ b/arch/x86/include/asm/context_tracking_work.h @@ -3,6 +3,7 @@ #define _ASM_X86_CONTEXT_TRACKING_WORK_H #include +#include static __always_inline void arch_context_tracking_work(int work) { @@ -10,6 +11,9 @@ static __always_inline void arch_context_tracking_work(int work) case CONTEXT_WORK_SYNC: sync_core(); break; + case CONTEXT_WORK_TLBI: + __flush_tlb_all_noinstr(); + break; } } diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h index aec6e2d3aa1d5..b97157a69d48e 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -74,6 +74,7 @@ static inline unsigned long native_read_cr4(void) return val; } +void native_write_cr4_noinstr(unsigned long val); void native_write_cr4(unsigned long val); #ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 69e79fff41b80..a653b5f47f0e6 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -18,6 +18,7 @@ DECLARE_PER_CPU(u64, tlbstate_untag_mask); void __flush_tlb_all(void); +void noinstr __flush_tlb_all_noinstr(void); #define TLB_FLUSH_ALL -1UL #define TLB_GENERATION_INVALID 0 @@ -418,9 +419,20 @@ static inline void cpu_tlbstate_update_lam(unsigned long lam, u64 untag_mask) #endif #endif /* !MODULE */ +#define __NATIVE_TLB_FLUSH_GLOBAL(suffix, cr4) \ + native_write_cr4##suffix(cr4 ^ X86_CR4_PGE); \ + native_write_cr4##suffix(cr4) +#define NATIVE_TLB_FLUSH_GLOBAL(cr4) __NATIVE_TLB_FLUSH_GLOBAL(, cr4) +#define NATIVE_TLB_FLUSH_GLOBAL_NOINSTR(cr4) __NATIVE_TLB_FLUSH_GLOBAL(_noinstr, cr4) + static inline void __native_tlb_flush_global(unsigned long cr4) { - native_write_cr4(cr4 ^ X86_CR4_PGE); - native_write_cr4(cr4); + NATIVE_TLB_FLUSH_GLOBAL(cr4); } + +static inline void __native_tlb_flush_global_noinstr(unsigned long cr4) +{ + NATIVE_TLB_FLUSH_GLOBAL_NOINSTR(cr4); +} + #endif /* _ASM_X86_TLBFLUSH_H */ diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index a5f221ea56888..a84bb8511650b 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -424,7 +424,7 @@ void native_write_cr0(unsigned long val) } EXPORT_SYMBOL(native_write_cr0); -void __no_profile native_write_cr4(unsigned long val) +noinstr void native_write_cr4_noinstr(unsigned long val) { unsigned long bits_changed = 0; @@ -442,6 +442,10 @@ void __no_profile native_write_cr4(unsigned long val) bits_changed); } } +void native_write_cr4(unsigned long val) +{ + native_write_cr4_noinstr(val); +} #if IS_MODULE(CONFIG_LKDTM) EXPORT_SYMBOL_GPL(native_write_cr4); #endif diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 86593d1b787d8..973a4ab3f53b3 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -256,7 +256,7 @@ static void choose_new_asid(struct mm_struct *next, u64 next_tlb_gen, * * See SWITCH_TO_USER_CR3. */ -static inline void invalidate_user_asid(u16 asid) +static __always_inline void invalidate_user_asid(u16 asid) { /* There is no user ASID if address space separation is off */ if (!IS_ENABLED(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION)) @@ -1198,7 +1198,7 @@ STATIC_NOPV void native_flush_tlb_global(void) /* * Flush the entire current user mapping */ -STATIC_NOPV void native_flush_tlb_local(void) +static noinstr void native_flush_tlb_local_noinstr(void) { /* * Preemption or interrupts must be disabled to protect the access @@ -1213,6 +1213,11 @@ STATIC_NOPV void native_flush_tlb_local(void) native_write_cr3(__native_read_cr3()); } +STATIC_NOPV void native_flush_tlb_local(void) +{ + native_flush_tlb_local_noinstr(); +} + void flush_tlb_local(void) { __flush_tlb_local(); @@ -1240,6 +1245,23 @@ void __flush_tlb_all(void) } EXPORT_SYMBOL_GPL(__flush_tlb_all); +void noinstr __flush_tlb_all_noinstr(void) +{ + /* + * This is for invocation in early entry code that cannot be + * instrumented. A RMW to CR4 works for most cases, but relies on + * being able to flip either of the PGE or PCIDE bits. Flipping CR4.PCID + * would require also resetting CR3.PCID, so just try with CR4.PGE, else + * do the CR3 write. + * + * XXX: this gives paravirt the finger. + */ + if (cpu_feature_enabled(X86_FEATURE_PGE)) + __native_tlb_flush_global_noinstr(this_cpu_read(cpu_tlbstate.cr4)); + else + native_flush_tlb_local_noinstr(); +} + void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) { struct flush_tlb_info *info; diff --git a/include/linux/context_tracking_work.h b/include/linux/context_tracking_work.h index 13fc97b395030..47d5ced39a43a 100644 --- a/include/linux/context_tracking_work.h +++ b/include/linux/context_tracking_work.h @@ -6,11 +6,13 @@ enum { CONTEXT_WORK_SYNC_OFFSET, + CONTEXT_WORK_TLBI_OFFSET, CONTEXT_WORK_MAX_OFFSET }; enum ct_work { CONTEXT_WORK_SYNC = BIT(CONTEXT_WORK_SYNC_OFFSET), + CONTEXT_WORK_TLBI = BIT(CONTEXT_WORK_TLBI_OFFSET), CONTEXT_WORK_MAX = BIT(CONTEXT_WORK_MAX_OFFSET) }; From patchwork Tue Nov 19 15:35:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7B48D44167 for ; Tue, 19 Nov 2024 15:39:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 491956B00C8; Tue, 19 Nov 2024 10:39:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 419676B00CA; Tue, 19 Nov 2024 10:39:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26D076B00CB; Tue, 19 Nov 2024 10:39:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F3B496B00C8 for ; Tue, 19 Nov 2024 10:39:45 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B1B031C6AB5 for ; Tue, 19 Nov 2024 15:39:45 +0000 (UTC) X-FDA: 82803252168.03.663DF8A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 2127914000F for ; Tue, 19 Nov 2024 15:39:08 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JJeT2Tyl; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030649; a=rsa-sha256; cv=none; b=3jEE1fKQipuQwra3fLzzaGeM8UJCiFAHCqhmZLWMaaeExjiDVvGiM/z8wg/TqKtjidw+wi 1Og/plXwm8t2D5PQLNaZU8vHb8+YF8v7AT14yLXWzyMCRizE5ti/sh0OxPytH4pEPFHUwK D0V/PHI1D7l4AWdXd0y2raojIBDWzIs= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JJeT2Tyl; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030649; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xImyrBZyvIgQkmg3c7kz+ISL+g7A1zTmv4nEmku3rgI=; b=wikH7IGpE2NsOtez4q/lsEiKbPCOeOOYPFVUbrokn37R7Y4vV2qJ+CMX0Sxh6u88FFDMph 4FdK4h+ix/rcho03ET0B4aSXo4mQwEC31aiHo4bsa6pn37F7YNOHKqaecjjHGGC8vM7GPC fyEQ0dL6Jw1vOipfHRyqQzOXSCMKCfo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xImyrBZyvIgQkmg3c7kz+ISL+g7A1zTmv4nEmku3rgI=; b=JJeT2Tyl0AB0TWw6cslgZoBG+I/YkoALTGILYVjDpfm1MKG2XHNktYy+U8S1ecXs2huUhU Ty9YZZHrFPpxEpTNqz67bOVNf3ZIaN7drJheODbTJvNQQIq3egdmB6ZNBCHN1tJp6kiOpM u/BzbgaDmlOt0sqQZYQsTWYZtxuQ5Xk= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-615-Nxl_Cjx6PWO_riG5D1782Q-1; Tue, 19 Nov 2024 10:39:41 -0500 X-MC-Unique: Nxl_Cjx6PWO_riG5D1782Q-1 X-Mimecast-MFC-AGG-ID: Nxl_Cjx6PWO_riG5D1782Q Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E9AB019560A2; Tue, 19 Nov 2024 15:39:36 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 901D830001A0; Tue, 19 Nov 2024 15:39:22 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 14/15] x86/mm, mm/vmalloc: Defer flush_tlb_kernel_range() targeting NOHZ_FULL CPUs Date: Tue, 19 Nov 2024 16:35:01 +0100 Message-ID: <20241119153502.41361-15-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Queue-Id: 2127914000F X-Stat-Signature: gzbc6m69hggy4uqahqsg4c3tpo19nrfg X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1732030748-719783 X-HE-Meta: U2FsdGVkX18Y88z4vWSRSlhn6wz6CV3pXsAagJ15cx8D+8IvtRKr6zX9XlNzkXgRyY1pLapPkzfa5p+tm6cwLkye4bT1ilf6cOlVsPeL8NWH5NR09NBpoOsG9epj9d0sIBGwJe9koyGblSjbcnDNLgZh28LtGGrmqIwYb/PHWHy+XpXB7B5oRM8aXCJeLsWC2noNuP4obTgsI8AmkaBJOLT7hkT0W9CRNbEHY84sFwBRRoxABjCmH5SkErkEZa8kI+v67418+F1o2+CZfkvSSPWBhxvvtIF1ogHbMJI/EtoKcXNsgGJeXbPY6BRMfLzmH9aE7dCD20MAk/6fXdBc6Chloj6AX1KReYT0Rm70B/nVUnSrqmPZmhHn3o0is9me6hK0wekl31rJ8IMIimvgBHwpPStMhAISy2ZE7WI9JGOZXwITbgIOJ4VLyby3XVVt6ix/Y1+rrTr13+7xsOOJ9Uf8IqnZpqPmfFLayHldM4rogHX6WtABtcXytNKjzz9kTN7t8kZgvuFZH/V5Bm5ljnuFbW0mgE418Mbr2M3BIjhtI0SsNca6TNohfBszkUCBG4enx8avNKd223yOwYaF3wTtloiWnxQG87BdTbQ8IaWIbY4Ed4yo22BIeXy99ecb7CjmRzzsGmvjaNmJuQiqCvfN0bduR9hTxAvN8eWaqBN3Szxdz5+/9vxCBWmzE3vzhvoxjqmZfNORsWny6b6OFyqPW+KZUmI771BX8df2/DCf9G+M/HdihyojWO2VSWVnlAosfU3CI472yUDucRCP7N9TH8mp8UCStcCR5ch4gr2ScYM1fMXGgw/feGO12PgO/Jr1Qp7zGHNWAc41YOKYvUAJcGGETlq/L6d7H6s1RzTr+Vc+R72tKWDE89ZPm4WyW2XpMyrJOwvfpnC/vmAjOmYsznEu2sOLXkt8X+NZgq47bGKC4nPRfmpO/s6fC4ZspgdKLNs+IXsqTmiYGTZ SICNmoBJ udnQjDQrMsogkm+s4iNJSrLv0pf+fdP05r1cYMXNMdW2LTR6tZplESgjmGtJvhNKdFrazUpan2OAyhnGIGEyVyhUBfmYg+ZDpcjwkcBPLP7lhO8mhEnFfy6HzGmoYN45fj6mL2LqVlALHN8aQurl8GFzOra8jY2cQLDfLFROcHirIC6LpSJm800Sj6GVNXI2n5JMCicqfQwIdiWGiYEPgMG7RcOUax27J02hlppm+qtLymtUozawTecWUejtT2PtxW7gn3JeyadG7F0hlR+WvR9lcbAxOjPjNJ5ZLWXhFTjNkFxJHEwHQ3eOo2cxfrv+NXiyQGz1aEWgfTWjWt5V1AZWpxD1VeWqLUW8x0SooA4ekQEHSi0CIT69fAiVhXbVW5vEECZvKuN/BAvDhzo0fj0yk6wkE+9niTypGGB51fxAXkXdjCU+5xx7GKwJAFa+Ngrp1IPw4GjCI0eI4xGagUhutYg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vunmap()'s issued from housekeeping CPUs are a relatively common source of interference for isolated NOHZ_FULL CPUs, as they are hit by the flush_tlb_kernel_range() IPIs. Given that CPUs executing in userspace do not access data in the vmalloc range, these IPIs could be deferred until their next kernel entry. Deferral vs early entry danger zone =================================== This requires a guarantee that nothing in the vmalloc range can be vunmap'd and then accessed in early entry code. Vmalloc uses are, as reported by vmallocinfo: $ cat /proc/vmallocinfo | awk '{ print $3 }' | sort | uniq __pci_enable_msix_range+0x32b/0x560 acpi_os_map_iomem+0x22d/0x250 bpf_prog_alloc_no_stats+0x34/0x140 fork_idle+0x79/0x120 gen_pool_add_owner+0x3e/0xb0 ? hpet_enable+0xbf/0x470 irq_init_percpu_irqstack+0x129/0x160 kernel_clone+0xab/0x3b0 memremap+0x164/0x330 n_tty_open+0x18/0xa0 pcpu_create_chunk+0x4e/0x1b0 pcpu_create_chunk+0x75/0x1b0 pcpu_get_vm_areas+0x0/0x1100 unpurged vp_modern_map_capability+0x140/0x270 zisofs_init+0x16/0x30 I've categorized these as: a) Device or percpu mappings For these to be unmapped, the device (or CPU) has to be removed and an eventual IRQ freed. Even if the IRQ isn't freed, context tracking entry happens before handling the IRQ itself, per irqentry_enter() usage. __pci_enable_msix_range() acpi_os_map_iomem() irq_init_percpu_irqstack() (not even unmapped when CPU is hot-unplugged!) memremap() n_tty_open() pcpu_create_chunk() pcpu_get_vm_areas() vp_modern_map_capability() b) CONFIG_VMAP_STACK fork_idle() & kernel_clone() vmalloc'd kernel stacks are AFAICT a safe example, as a task running in userspace needs to enter kernelspace to execute do_exit() before its stack can be vfree'd. c) Non-issues bpf_prog_alloc_no_stats() - early entry is noinstr, no BPF! hpet_enable() - hpet_clear_mapping() is only called if __init function fails, no runtime worries zisofs_init () - used for zisofs block decompression, that's way past context tracking entry d) I'm not sure, have a look? gen_pool_add_owner() - AIUI this is mainly for PCI / DMA stuff, which again I wouldn't expect to be accessed before context tracking entry. Changes ====== Blindly deferring any and all flush of the kernel mappings is a risky move, so introduce a variant of flush_tlb_kernel_range() that explicitly allows deferral. Use it for vunmap flushes. Note that while flush_tlb_kernel_range() may end up issuing a full flush (including user mappings), this only happens when reaching a invalidation range threshold where it is cheaper to do a full flush than to individually invalidate each page in the range via INVLPG. IOW, it doesn't *require* invalidating user mappings, and thus remains safe to defer until a later kernel entry. Signed-off-by: Valentin Schneider --- arch/x86/include/asm/tlbflush.h | 1 + arch/x86/mm/tlb.c | 23 +++++++++++++++++++--- mm/vmalloc.c | 35 ++++++++++++++++++++++++++++----- 3 files changed, 51 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index a653b5f47f0e6..d89345c85fa9c 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -249,6 +249,7 @@ extern void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, unsigned long end, unsigned int stride_shift, bool freed_tables); extern void flush_tlb_kernel_range(unsigned long start, unsigned long end); +extern void flush_tlb_kernel_range_deferrable(unsigned long start, unsigned long end); static inline void flush_tlb_page(struct vm_area_struct *vma, unsigned long a) { diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 973a4ab3f53b3..bf6ff16a1a523 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -1041,6 +1042,11 @@ static void do_flush_tlb_all(void *info) __flush_tlb_all(); } +static bool do_kernel_flush_defer_cond(int cpu, void *info) +{ + return !ct_set_cpu_work(cpu, CONTEXT_WORK_TLBI); +} + void flush_tlb_all(void) { count_vm_tlb_event(NR_TLB_REMOTE_FLUSH); @@ -1057,12 +1063,13 @@ static void do_kernel_range_flush(void *info) flush_tlb_one_kernel(addr); } -void flush_tlb_kernel_range(unsigned long start, unsigned long end) +static inline void +__flush_tlb_kernel_range(smp_cond_func_t cond_func, unsigned long start, unsigned long end) { /* Balance as user space task's flush, a bit conservative */ if (end == TLB_FLUSH_ALL || (end - start) > tlb_single_page_flush_ceiling << PAGE_SHIFT) { - on_each_cpu(do_flush_tlb_all, NULL, 1); + on_each_cpu_cond(cond_func, do_flush_tlb_all, NULL, 1); } else { struct flush_tlb_info *info; @@ -1070,13 +1077,23 @@ void flush_tlb_kernel_range(unsigned long start, unsigned long end) info = get_flush_tlb_info(NULL, start, end, 0, false, TLB_GENERATION_INVALID); - on_each_cpu(do_kernel_range_flush, info, 1); + on_each_cpu_cond(cond_func, do_kernel_range_flush, info, 1); put_flush_tlb_info(); preempt_enable(); } } +void flush_tlb_kernel_range(unsigned long start, unsigned long end) +{ + __flush_tlb_kernel_range(NULL, start, end); +} + +void flush_tlb_kernel_range_deferrable(unsigned long start, unsigned long end) +{ + __flush_tlb_kernel_range(do_kernel_flush_defer_cond, start, end); +} + /* * This can be used from process context to figure out what the value of * CR3 is without needing to do a (slow) __read_cr3(). diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 634162271c004..02838c515ce2c 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -467,6 +467,31 @@ void vunmap_range_noflush(unsigned long start, unsigned long end) __vunmap_range_noflush(start, end); } +#ifdef CONFIG_CONTEXT_TRACKING_WORK +/* + * !!! BIG FAT WARNING !!! + * + * The CPU is free to cache any part of the paging hierarchy it wants at any + * time. It's also free to set accessed and dirty bits at any time, even for + * instructions that may never execute architecturally. + * + * This means that deferring a TLB flush affecting freed page-table-pages (IOW, + * keeping them in a CPU's paging hierarchy cache) is akin to dancing in a + * minefield. + * + * This isn't a problem for deferral of TLB flushes in vmalloc, because + * page-table-pages used for vmap() mappings are never freed - see how + * __vunmap_range_noflush() walks the whole mapping but only clears the leaf PTEs. + * If this ever changes, TLB flush deferral will cause misery. + */ +void __weak flush_tlb_kernel_range_deferrable(unsigned long start, unsigned long end) +{ + flush_tlb_kernel_range(start, end); +} +#else +#define flush_tlb_kernel_range_deferrable(start, end) flush_tlb_kernel_range(start, end) +#endif + /** * vunmap_range - unmap kernel virtual addresses * @addr: start of the VM area to unmap @@ -480,7 +505,7 @@ void vunmap_range(unsigned long addr, unsigned long end) { flush_cache_vunmap(addr, end); vunmap_range_noflush(addr, end); - flush_tlb_kernel_range(addr, end); + flush_tlb_kernel_range_deferrable(addr, end); } static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, @@ -2265,7 +2290,7 @@ static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end, nr_purge_nodes = cpumask_weight(&purge_nodes); if (nr_purge_nodes > 0) { - flush_tlb_kernel_range(start, end); + flush_tlb_kernel_range_deferrable(start, end); /* One extra worker is per a lazy_max_pages() full set minus one. */ nr_purge_helpers = atomic_long_read(&vmap_lazy_nr) / lazy_max_pages(); @@ -2368,7 +2393,7 @@ static void free_unmap_vmap_area(struct vmap_area *va) flush_cache_vunmap(va->va_start, va->va_end); vunmap_range_noflush(va->va_start, va->va_end); if (debug_pagealloc_enabled_static()) - flush_tlb_kernel_range(va->va_start, va->va_end); + flush_tlb_kernel_range_deferrable(va->va_start, va->va_end); free_vmap_area_noflush(va); } @@ -2816,7 +2841,7 @@ static void vb_free(unsigned long addr, unsigned long size) vunmap_range_noflush(addr, addr + size); if (debug_pagealloc_enabled_static()) - flush_tlb_kernel_range(addr, addr + size); + flush_tlb_kernel_range_deferrable(addr, addr + size); spin_lock(&vb->lock); @@ -2881,7 +2906,7 @@ static void _vm_unmap_aliases(unsigned long start, unsigned long end, int flush) free_purged_blocks(&purge_list); if (!__purge_vmap_area_lazy(start, end, false) && flush) - flush_tlb_kernel_range(start, end); + flush_tlb_kernel_range_deferrable(start, end); mutex_unlock(&vmap_purge_lock); } From patchwork Tue Nov 19 15:35:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8070D44167 for ; Tue, 19 Nov 2024 15:40:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C9816B00CA; Tue, 19 Nov 2024 10:40:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 552116B00CC; Tue, 19 Nov 2024 10:40:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CC746B00CD; Tue, 19 Nov 2024 10:40:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 168CB6B00CA for ; Tue, 19 Nov 2024 10:40:04 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C5148A05E2 for ; Tue, 19 Nov 2024 15:40:03 +0000 (UTC) X-FDA: 82803253974.26.F9F723F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf02.hostedemail.com (Postfix) with ESMTP id A437980017 for ; Tue, 19 Nov 2024 15:38:26 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=i+kPm+bd; spf=pass (imf02.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030710; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F1GnqQSFPr8VITMqjCT6H9sq2glpq77hQWm9Vj7AfgE=; b=cwxIYE7qQF0cvIcyZeZhH80P5nnP2fODCKKzOkD8n5G9wNhzyptsZxyx4GsDu1y7FxlDD9 zeseIhOFSBW29EGbDbx0gLeMvmWVQ6AcnYARuJyzj/vN9J2Mcu33eW6/8Zlh8KFZ5Yvzok eh6i+0nv4r27w2uXw/POlqu95l3w0wE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030710; a=rsa-sha256; cv=none; b=UMlB+WEOwGVkbCrR9yZqTOAiD72NmIoaX1TppZBoClRQ/41QfsxY7Y+BbIaM6NkzS6l0bt GJ3k+KEpM0LXW+yTp2gx6uOG//OuzIOONK2AxA0mkE2CSpnMBd+0Rk0XbCRtl1KTZm6Tfk hVKOFcb7cq/RJZqa8EtSmAhfY0jRzc8= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=i+kPm+bd; spf=pass (imf02.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F1GnqQSFPr8VITMqjCT6H9sq2glpq77hQWm9Vj7AfgE=; b=i+kPm+bdLi//bO2J7FCX109ghA9e0RfOq9AxT78RRyblJtpgUCvMvdW3vG1xaqCYsRmJ6k 8ZiHLwyfcFYB13a4vU61CAZ/kDvxbfqqbCoS4VQGc6Ngto6nZgvpvMWflLefRWV96d4xWw mgQ3mDEN3847O7+16W8qPLvHBtBM9Os= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-207-92ypQAdDPJmP9uLfkEuaoA-1; Tue, 19 Nov 2024 10:39:57 -0500 X-MC-Unique: 92ypQAdDPJmP9uLfkEuaoA-1 X-Mimecast-MFC-AGG-ID: 92ypQAdDPJmP9uLfkEuaoA Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 15EC119541A0; Tue, 19 Nov 2024 15:39:52 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 63E1730001A0; Tue, 19 Nov 2024 15:39:37 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 15/15] context-tracking: Add a Kconfig to enable IPI deferral for NO_HZ_IDLE Date: Tue, 19 Nov 2024 16:35:02 +0100 Message-ID: <20241119153502.41361-16-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Server: rspam10 X-Stat-Signature: m3c1geewtcdjxyc6bqhyqwzui6htmt87 X-Rspamd-Queue-Id: A437980017 X-Rspam-User: X-HE-Tag: 1732030706-168159 X-HE-Meta: U2FsdGVkX1/53Y+0YxwtEyGXsKqCoMVcy0p7vzQkvpoNdCc3H7n5kgXgz7OJlyt+rLzX4I9iZXpCPDXw5ZxGJDTKg6nzku4UX2vWby8+YLGQNCgc0EI+3Gp3hIy60PV8memRuYkXV4OrdScGUZaXgESLYU+5oXLZYCg6YegOAwBYd/NkGvjpTAEHo5L3Yi28GENeaVqLKBYVRtbnej8YD8oM0cqiK86508Ci7/KobdliFeLPrQo+tuY1aJtvVR0eyaxU48zzgArGaXExI8XdO+/LWjaIeAyli2q8ilJ/ceWxF25HUbq9Y2Ok/LjxLxoKemOzamXV/Y/XLtpATZVd7lIE5D5z6KfWHDDqcYFBNtzmcew0noPE+j6pjnioBASGY5fnOAwHRIDfReLeAUhOOux4lMPt97R1ILWCVFrYdBFMnfw79XaOaop2HJh4iJPWn1E8mJ9CcqEHCC7d5Gi+XMn4x7WvcKOZRJrBZAYYgvicp2gaU/9FiZVf9YS2q/GbZ1JPVGQQSrjXQTFvxl3+HUbRWOIiD0WHPjXFN4NK+PbyM6hpwkiEufe/UUnJ/M2kZqOj6v9yChDSEvgSjyNdDnOpFmx2YEuOaHPCl0T20NhLvcAtPl49cAlTlB2M4HmTn5d/6JxENld26nf43TrHnApU0tfBDkbC5ElqdakbeO7rNqLHKRK9K7Q152ruTzmOXgTqHbD16qfHJX0CvDzyxP1DGDTNKwRLGljjtbTaQO9L4t0KtAj+AViaZNDT9UQLgnB75pV/mn8/MWUIhvOVzYSri9px7XFUVXqjUwLkNNwXEEFbopZxOWR945SoBbpE45zirwy9/9xOX2QAKXgPMdU7mASnUZi6/S8lhyGvTUyPH8JfboLfZ6fDCtSFEvOLqwkUfWd28+uXwSKK2pQiLCP4AkTUzHaSLURTCZNM1iOHrjMSVlA05QkXpoLNvCVb6oKCc1OmVO7qSQlqP5C oY7HjQ9v Q/G7jWNv9NXTJ4c37Mjc6t8frQVWTW40AXFzqY4HGWMLsGTD/hLslHCZAU9BKngRl5yqpz4rfHpGvZ2H1QdyDQwSIzfegMjzfqOmRmWPGDRpnqab+zcuN0NWlkSXB1OFms3+ERJIfFg11nvuCCN5dNKS41/sCChVjyVQXd4jqRdx0NDl703oHhQbcUaVtr+S1osyg+X8GCgirbao5Mv8HPJauVEaIyBCTw/yTFQ2soGr1XKzUJTHn3isMBO/7l0QBwTOedU5PlK5wS67cTm4FiUXIrAR+zXJi6pQgUKaOhH6m9uOI+9oFQw95ToYDznaS8ptk7KYtFSXKyYptci2jhwBjaNwuqSW44joeR06wWY42gR5qiSjvsL86e7SrVp9PH7jSsOglwW5LIszw8ZHtfC7UUNM0vYMXJyJdPSsztpkEZtnxuAE+G3/N2awsMLhlOA4HfakyFd85m61FxA9sR50lWw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With NO_HZ_IDLE, we get CONTEXT_TRACKING_IDLE, so we get these transitions: ct_idle_enter() ct_kernel_exit() ct_state_inc_clear_work() ct_idle_exit() ct_kernel_enter() ct_work_flush() With just CONTEXT_TRACKING_IDLE, ct_state_inc_clear_work() is just ct_state_inc() and ct_work_flush() is a no-op. However, making them be functional as if under CONTEXT_TRACKING_WORK would allow NO_HZ_IDLE to leverage IPI deferral to keep idle CPUs idle longer. Having this enabled for NO_HZ_IDLE is a different argument than for having it for NO_HZ_FULL (power savings vs latency/performance), but the backing mechanism is identical. Add a default-no option to enable IPI deferral with NO_HZ_IDLE. Signed-off-by: Valentin Schneider --- kernel/time/Kconfig | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/kernel/time/Kconfig b/kernel/time/Kconfig index 04efc2b605823..a3cb903a4e022 100644 --- a/kernel/time/Kconfig +++ b/kernel/time/Kconfig @@ -188,9 +188,23 @@ config CONTEXT_TRACKING_USER_FORCE config CONTEXT_TRACKING_WORK bool - depends on HAVE_CONTEXT_TRACKING_WORK && CONTEXT_TRACKING_USER + depends on HAVE_CONTEXT_TRACKING_WORK && (CONTEXT_TRACKING_USER || CONTEXT_TRACKING_WORK_IDLE) default y +config CONTEXT_TRACKING_WORK_IDLE + bool + depends on HAVE_CONTEXT_TRACKING_WORK && CONTEXT_TRACKING_IDLE && !CONTEXT_TRACKING_USER + default n + help + This option enables deferral of some IPIs when they are targeted at CPUs + that are idle. This can help keep CPUs idle longer, but induces some + extra overhead to idle <-> kernel transitions and to IPI sending. + + Say Y if the power improvements are worth more to you than the added + overheads. + + Say N otherwise. + config NO_HZ bool "Old Idle dynticks config" help