From patchwork Fri Dec 3 10:47:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 12694649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DA60C433EF for ; Fri, 3 Dec 2021 10:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/cxiO4K4z5LHHNn2bi28pCN8Zf+KhMWMez27jqXfIDQ=; b=jUTrKipcW+Gyxx HzIfxLff2ADZkcUoB3acxuu9rMQFIQAb+zFvIQks6lZwR2cyJo6ddWMGhSLPYwa7QgFN+lrcGKLYq +EaXNl4OMoRybIzOJ0a/V26Ivt5F2BQ6TLJeYeLKySMKmM4objZMvG/K1QZfiB11cSBvTwi19rvcI I3BwhZYJtO2wTjsMPu7e0x8e/f1s0sJ68MAivI7JH0gMA5pBAKpbo1VOl5BlrDPMRvx/lmbtwPHAN ze+QYBgefI9FTh5rvqVd3eYXVhxp6DgjFnoWdqS/UJ6DLmFtjeZjzBH1Cdidt77VbA74E3NDbNzn6 EQOABjZS7f39NkRd2Ztg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mt66h-00FEhn-Lv; Fri, 03 Dec 2021 10:47:59 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mt66K-00FEYx-B1 for linux-arm-kernel@lists.infradead.org; Fri, 03 Dec 2021 10:47:37 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BB3D91596; Fri, 3 Dec 2021 02:47:35 -0800 (PST) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 85F223F5A1; Fri, 3 Dec 2021 02:47:34 -0800 (PST) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: andre.przywara@arm.com, ardb@kernel.org, catalin.marinas@arm.com, james.morse@arm.com, joey.gouly@arm.com, mark.rutland@arm.com, suzuki.poulose@arm.com, will@kernel.org Subject: [PATCH 2/4] arm64: insn: wait for other CPUs before patching Date: Fri, 3 Dec 2021 10:47:21 +0000 Message-Id: <20211203104723.3412383-3-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211203104723.3412383-1-mark.rutland@arm.com> References: <20211203104723.3412383-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211203_024736_456190_D2082763 X-CRM114-Status: GOOD ( 13.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In aarch64_insn_patch_text_cb(), secondary CPUs wait for the master CPU to signal that patching is complete, but the master CPU doesn't wait for secondaries to enter the polling loop, and it's possible that patching starts while secondaries are still within the stop_machine logic. Let's fix this by adding a vaguely simple polling protocol where the boot CPU waits for secondaries to signal that they have entered the unpatchable stop function. We can use the arch_atomic_*() functions for this, as these are inlined and not instrumented. These will not be patched concurrently with this code executing. Signed-off-by: Mark Rutland Cc: Andre Przywara Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Suzuki K Poulose Cc: Will Deacon --- arch/arm64/kernel/patching.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/patching.c b/arch/arm64/kernel/patching.c index 771f543464e0..c0d51340c913 100644 --- a/arch/arm64/kernel/patching.c +++ b/arch/arm64/kernel/patching.c @@ -116,16 +116,17 @@ static int __kprobes aarch64_insn_patch_text_cb(void *arg) { int i, ret = 0; struct aarch64_insn_patch *pp = arg; + int num_cpus = num_online_cpus(); - /* The first CPU becomes master */ - if (atomic_inc_return(&pp->cpu_count) == 1) { + /* The last CPU becomes master */ + if (arch_atomic_inc_return(&pp->cpu_count) == num_cpus) { for (i = 0; ret == 0 && i < pp->insn_cnt; i++) ret = aarch64_insn_patch_text_nosync(pp->text_addrs[i], pp->new_insns[i]); /* Notify other processors with an additional increment. */ atomic_inc(&pp->cpu_count); } else { - while (atomic_read(&pp->cpu_count) <= num_online_cpus()) + while (arch_atomic_read(&pp->cpu_count) <= num_cpus) cpu_relax(); isb(); }