From patchwork Fri Oct 7 15:41:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13001239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1CA3C433FE for ; Fri, 7 Oct 2022 15:44:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=QkvH8AtES8Qr/fzeU2FuFOJvyGu7h8VVApY+euAyOw8=; b=kxahXeqohCkJzX pPF3Xf3iHFIc83jpgc7KsxfH/Qr64W9etjOz0OqKIKgpz7fVu+mUUWRSZPeEQJCHimLKzNkc0m/co z3lckBLi9OZseo2uyRPMarOHTYzEWzXFnh7WwDT36tXf45B+O7R5cZKS2eirsHKkUsd+BU7gVRS8b 2KK06bCkOpo3iWagwJ9csPJl6PiNFUEeSnJmEFBe3oPK/UpylRdZ7oLayaR2Yyu/Gn4MkMF7VsCH1 m1U9jDCvolxNwydZvNX7QkrYc0Y6ythZndRJKNLjggNF1KfSUWxrz6IVvGYsZr87aBCbF9SI/AMuX 2s+gahqj3cPYT0bDLyPA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ogpV3-009WnU-A2; Fri, 07 Oct 2022 15:42:57 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ogpUw-009Wht-CS for linux-arm-kernel@lists.infradead.org; Fri, 07 Oct 2022 15:42:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665157366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1lKCnBjxRPjgpNNPlEylbLIMLTQ3r/0aTk3SgXwAPQE=; b=JHnNtrD1sfkQfddb35HeepWu/fe2diZLLiggd/W1cbABp8s21Qh/Vl9wIwjcWx5oOfHEMK 7YCZRZWJugWFTwitCOcv0PriuWqOS0B3QVfUGyoDmC2ecGjcqhvg67eYUqR6ipUnzrKuEV Vh5vvTxgBcev+FSEF8JC/53CCBXkcy8= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-454-0QbmitfCMay8NrDLH32C2A-1; Fri, 07 Oct 2022 11:42:46 -0400 X-MC-Unique: 0QbmitfCMay8NrDLH32C2A-1 Received: by mail-wm1-f70.google.com with SMTP id b7-20020a05600c4e0700b003bde2d860d1so2837356wmq.8 for ; Fri, 07 Oct 2022 08:42:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1lKCnBjxRPjgpNNPlEylbLIMLTQ3r/0aTk3SgXwAPQE=; b=E7TIApX0wq0mJpw7V+cMvbu0ZGaXTb8iEhzmXscamyXRr6Ud4eoSV7ZNHEXj5cjbuv UU3ecuOuhT5N1hrr1bdcUH7QpIvjI2u1nbSKjnnFV93zmJ3HcPMhWmjDOaPwa2RCl5JK N8V912MRROdJRksSExfNKUN8I9SiYQArEZB47wiIphE3XlPurHaIAw1yBSs+mHPDOU46 NwUwtkXTNGAHHIJEST1on3AZOUp87pIUyBn+AuwpWHxbUUWIREb/hDQ9znGj1yXasaCI Bh+JAkAxC9XjDQKqTGHvgrAPBmSa2iXyKhX5QFQM7HRd91B/ghMwJHfW+dxN0sAll9Zj RNJQ== X-Gm-Message-State: ACrzQf1tYHW5HqGz4KIJh5PbTsRdbT23JMvSsrfO+rrqVcmmkXuHqzxN l6cvLpAXv4BpQ7pMvquoZTSBuBxqmjO4u75uSDeVZTpPzW4CyWIQ0KgFe2ulUbWFPmdrHlfUuut QFeFrK10AbaelTjESloqEHAFv2jt0ZPc/ONE= X-Received: by 2002:a5d:5010:0:b0:22a:cb71:9493 with SMTP id e16-20020a5d5010000000b0022acb719493mr3770672wrt.514.1665157361757; Fri, 07 Oct 2022 08:42:41 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7Tr05uM1rtU5aj/QWiT6cTirx56otcH3SojWjYuAUChmPHRHcJ6HatWEhyYipO7y3unWYaxQ== X-Received: by 2002:a5d:5010:0:b0:22a:cb71:9493 with SMTP id e16-20020a5d5010000000b0022acb719493mr3770632wrt.514.1665157361532; Fri, 07 Oct 2022 08:42:41 -0700 (PDT) Received: from vschneid.remote.csb ([149.71.65.94]) by smtp.gmail.com with ESMTPSA id e1-20020a5d5941000000b0022af865810esm2307021wri.75.2022.10.07.08.42.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Oct 2022 08:42:40 -0700 (PDT) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: "Paul E. McKenney" , Steven Rostedt , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , Guo Ren , "David S. Miller" Subject: [RFC PATCH 0/5] Generic IPI sending tracepoint Date: Fri, 7 Oct 2022 16:41:40 +0100 Message-Id: <20221007154145.1877054-1-vschneid@redhat.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221007_084250_507859_269307A1 X-CRM114-Status: GOOD ( 18.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Background ========== Detecting IPI *reception* is relatively easy, e.g. using trace_irq_handler_{entry,exit} or even just function-trace flush_smp_call_function_queue() for SMP calls. Figuring out their *origin*, is trickier as there is no generic tracepoint tied to e.g. smp_call_function(): o AFAIA x86 has no tracepoint tied to sending IPIs, only receiving them (cf. trace_call_function{_single}_entry()). o arm/arm64 do have trace_ipi_raise(), which gives us the target cpus but also a mostly useless string (smp_calls will all be "Function call interrupts"). o Other architectures don't seem to have any IPI-sending related tracepoint. I believe one reason those tracepoints used by arm/arm64 ended up as they were is because these archs used to handle IPIs differently from regular interrupts (the IRQ driver would directly invoke an IPI-handling routine), which meant they never showed up in trace_irq_handler_{entry, exit}. The trace_ipi_{entry,exit} tracepoints gave a way to trace IPI reception but those have become redundant as of: 56afcd3dbd19 ("ARM: Allow IPIs to be handled as normal interrupts") d3afc7f12987 ("arm64: Allow IPIs to be handled as normal interrupts") which gave IPIs a "proper" handler function used through generic_handle_domain_irq(), which makes them show up via trace_irq_handler_{entry, exit}. Changing stuff up ================= Per the above, it would make sense to reshuffle trace_ipi_raise() and move it into generic code. This also came up during Daniel's talk on Osnoise at the CPU isolation MC of LPC 2022 [1]. Now, to be useful, such a tracepoint needs to export: o targeted CPU(s) o calling context The only way to get the calling context with trace_ipi_raise() is to trigger a stack dump, e.g. $(trace-cmd -e ipi* -T echo 42). As for the targeted CPUs, the existing tracepoint does export them, albeit in cpumask form, which is quite inconvenient from a tooling perspective. For instance, as far as I'm aware, it's not possible to do event filtering on a cpumask via trace-cmd. Because of the above points, this is introducing a new tracepoint. Patches ======= This results in having trace events for: o smp_call_function*() o smp_send_reschedule() o irq_work_queue*() This is incomplete, just looking at arm64 there's more IPI types that aren't covered: IPI_CPU_STOP, IPI_CPU_CRASH_STOP, IPI_TIMER, IPI_WAKEUP, ... But it feels like a good starting point. Another thing worth mentioning is that depending on the callsite, the _RET_IP_ fed to the tracepoint is not always useful - generic_exec_single() doesn't tell you much about the actual callback being sent via IPI, so there might be value in exploding the single tracepoint into at least one variant for smp_calls. Links ===== [1]: https://youtu.be/5gT57y4OzBM?t=14234 Valentin Schneider (5): trace: Add trace_ipi_send_{cpu, cpumask} sched, smp: Trace send_call_function_single_ipi() smp: Add a multi-CPU variant to send_call_function_single_ipi() irq_work: Trace calls to arch_irq_work_raise() treewide: Rename and trace arch-definitions of smp_send_reschedule() arch/alpha/kernel/smp.c | 2 +- arch/arc/kernel/smp.c | 2 +- arch/arm/kernel/smp.c | 5 +---- arch/arm64/kernel/smp.c | 3 +-- arch/csky/kernel/smp.c | 2 +- arch/hexagon/kernel/smp.c | 2 +- arch/ia64/kernel/smp.c | 4 ++-- arch/loongarch/include/asm/smp.h | 2 +- arch/mips/include/asm/smp.h | 2 +- arch/openrisc/kernel/smp.c | 2 +- arch/parisc/kernel/smp.c | 4 ++-- arch/powerpc/kernel/smp.c | 4 ++-- arch/riscv/kernel/smp.c | 4 ++-- arch/s390/kernel/smp.c | 2 +- arch/sh/kernel/smp.c | 2 +- arch/sparc/kernel/smp_32.c | 2 +- arch/sparc/kernel/smp_64.c | 2 +- arch/x86/include/asm/smp.h | 2 +- arch/xtensa/kernel/smp.c | 2 +- include/linux/smp.h | 1 + include/trace/events/ipi.h | 27 +++++++++++++++++++++++++++ kernel/irq_work.c | 12 +++++++++++- kernel/sched/core.c | 7 +++++-- kernel/smp.c | 18 +++++++++++++++++- 24 files changed, 84 insertions(+), 31 deletions(-) --- 2.31.1