From patchwork Wed Mar 29 20:25:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: D Scott Phillips X-Patchwork-Id: 13193195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7E659C6FD18 for ; Wed, 29 Mar 2023 20:26:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aqUg2zkJcgx9cGuEFzVf/S2JxfG8Y3H+PcWGaFM317A=; b=WtLaz7vjVGE9o8 D3VAaEwomI6AoF6NDMXXHFnYpi0AC2mYct3k9e6A654E4kjCkp+maWG0fyoyAOUUWD0UILW9VggII c+Puf2nrWVzKvKPDlDux1Ta2YwYmHVeNhNVdNOAm7tLmixra7gx6Zr7JUiCVwQ31WaBvRYh2uvaov LuZnR/bAv05I4S77UMoQgGuqdzmaTRH6lGMw25diuaoxo/cR7GpnIP1E9/G+5CxqER7dH4IEOMCiu oxcMpw9TNvcUP3wC1B8/qRttIuajoeQ2cW0rLRx358E9liLzOOALT0n8gvqwYjU4VYJQ2XKiuYdiu x6qhhs2tYZZkInGdZrXA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1phcMp-001ljf-1u; Wed, 29 Mar 2023 20:25:59 +0000 Received: from mail-dm6nam11on2072c.outbound.protection.outlook.com ([2a01:111:f400:7eaa::72c] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1phcMe-001lgT-1M for linux-arm-kernel@lists.infradead.org; Wed, 29 Mar 2023 20:25:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ceNYu0OEoUUKMrDGoc/yr8VNcaLoRgaM5Q2iJnKs3L8BubPOMnFijNbk8Gb81sF/MxiVUk/EtmuvH9smRuCMF+mcWUkacg7ZDPKeWRt+BW5/MSvdolNI67MGiEnq4cY6lYQlJ43jDfb9Zg1XKokyokmaLx2/a3ZnWUjoDcmsGKgBDDe3F1XgwqYolAD9j1Y/FLH+32WlFVg+EvgMvdN8SfhY1Suq8729fU+IFSbmY/jPWb8RnPHvACKmd5q3pghXsF2phT+c6YziSnLXsNpju3kCI8myhfF42lRlF+WXI1ZZGzI6CT2RfRTPyHEIMoCsfIWEHNw+h7ByP9Vv1NORhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l+IchaXBHyX6IlEygs7fF2jvnMOjFc0NGPqru3NL4HE=; b=BkmyeGfYMy9OD/bF67hQxlyuDeaqSHVWLPYI9lWgXVAuansYHGaIQwfhg7OYN7aOISBjINTpWYdM+nwSq4BRjHVED2PkWIOuVRPZ/M+pI0P1QjeXilfTIpsqzuDhxJ0gEzm2g7W7helnjv4ZbrxVEm+lOfmXOdvfpblEWQZoAwcUOtIsjT97cfywGDjEOCh7p93BTGTmFy3ba7aG0Un4N2EoYoxB52ndfJeSQEiSaUuWFdybR9cMx3S3umAg4DDjDw1RSm4aMT+TKDr+/9CAgRafhmMk24eHvomv4hvjC2XJr51EF5tmhXYCIS7skNUN1dnn/ouyXXg483ny1g09vw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=l+IchaXBHyX6IlEygs7fF2jvnMOjFc0NGPqru3NL4HE=; b=XVXcxCBMflxsruaVl4VDsizk9q/UUBEKYDpdEKM9yw3GGBAsc7FZqMzGP85DJoFiTe/I0iSXbqM2z2UjjoHsJzQuW1orNB3MQG0q6auf9g0vi7juPymuhipIzAEZctz9fUuTpBeWYU68duxmIT1wV20+5QBGvqBIJC6DjdTsoqs= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from MWHPR0101MB2893.prod.exchangelabs.com (2603:10b6:301:33::25) by BL0PR01MB4260.prod.exchangelabs.com (2603:10b6:208:49::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.18; Wed, 29 Mar 2023 20:25:37 +0000 Received: from MWHPR0101MB2893.prod.exchangelabs.com ([fe80::6f92:8006:1d6d:4cec]) by MWHPR0101MB2893.prod.exchangelabs.com ([fe80::6f92:8006:1d6d:4cec%6]) with mapi id 15.20.6254.018; Wed, 29 Mar 2023 20:25:36 +0000 From: D Scott Phillips To: linux-arm-kernel@lists.infradead.org, James Morse Cc: Catalin Marinas , Will Deacon Subject: [PATCH v2 2/2] arm64: sdei: abort running SDEI handlers during crash Date: Wed, 29 Mar 2023 13:25:19 -0700 Message-Id: <20230329202519.6110-2-scott@os.amperecomputing.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230329202519.6110-1-scott@os.amperecomputing.com> References: <20230329202519.6110-1-scott@os.amperecomputing.com> X-ClientProxiedBy: CH2PR14CA0015.namprd14.prod.outlook.com (2603:10b6:610:60::25) To MWHPR0101MB2893.prod.exchangelabs.com (2603:10b6:301:33::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWHPR0101MB2893:EE_|BL0PR01MB4260:EE_ X-MS-Office365-Filtering-Correlation-Id: e0194c28-be20-46f0-b686-08db3093bfb5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CLO/a2KU8CNcHoQJ9YeX/FUKVxoESh3zh9FXXXTUh1R7WTLSFZAsR6Xl+UAcdpzWLSzOdSJ48vLzNog8Mse9RKF5iDydzLzBIwNE8Kih2z2H/kPnm0OlSIVnsWAkNuJ6rIbo4+r+Mm+J9Vp1IrmwSI22krdXvkyIJiPagRaUhgZ8UoVznqU3S7WjDEL5IIref5Wor8jBcacaHIjJKfGYs1d3cfQK1G2gqXhFacz8LWyiIRbvCKVwjCeV04xT9HFGX9HEVEx9PmcVe0bPkIOz+4D2UVqNFnGGYFeq/ITEKT8h7gUP5tlwY8kGkpxXXf9SDtWuKv3W0baUcX6QZJwIjpqhfTbU96Z4UKP2dyj0k7FxTYHrbHkVlVhAMqHAn5pM31J6XGu23Td6ddKMo6CcblFgQK9heyRdWUqEZkey+bU1np10YpEHbg9UhygqPJph1bos7SPZDZ3aamQLP9MeAkrqgrXpn6josUtnzOZiK3wtERxNbuxiIoreax/f6ZKaAgaRp9ZsQ63SkVNub/16DqUW6DVjP4LP4zyR+DxmXtODxdzaHAT32o2VIo7JrVGFFSb5BZ1CqcqIuFJFjj7K83AUfJD9hoOOQwuQg2FTlZQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR0101MB2893.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(136003)(396003)(39850400004)(376002)(366004)(346002)(451199021)(478600001)(966005)(316002)(38100700002)(38350700002)(54906003)(86362001)(83380400001)(2616005)(2906002)(6486002)(52116002)(5660300002)(8676002)(66556008)(66946007)(66476007)(6916009)(4326008)(6506007)(8936002)(6512007)(6666004)(41300700001)(186003)(1076003)(26005);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: R3ub+fQUHTIH4U+x8P/uD9/sHhUWXdKjKQ57Ki8TLcle0FTVq04eFPqy9tPITp+9lxtV+a5s+xCYpBbWJMmYEtZLtp3a3yOtPyf1mC2nvmhllsWY3whOW6u1rO6QIaP7b4P1FC6V8cPRqhW8xLvOIJDq9Iklh669x0uaziDGeX6p7OiQKWy0Vo/LqWNN0vX945ydeyBt1tGC0fVIUOObdv43MJpsb+tMGVjWJEJen/4HrEmC6Axa9ltSuX5OdVcl2E4VuSiGkWdCRbr2z7dIrupSMu3QyzirehtcxqVq0EsIZ03H8KNMoa8RH0Y+AntAfBAvppxfg1rH5GExNsj3aVYsnKtYOUnIUa3F50ZBN2Di7O9UyEx/fXxmglMiojdNAaVvrPpths8vIXgHoc7pRSeKU/C4743SQn1lGBMpgYRiDWqRfJbhkpVTL8KriUlahzP5QEl1Nj7U0paZtNE1dHS+h5qUWWywxwWYYftxlqhBvONdWrV3T2tRMtJB7Gx9/kibfSbSTaTJ9ZRlataSwv28zoYcmvoHB9gSVMNSU4xbtPXtU06G1glg4+9LyTWfa6/Pn6scxd1XxLMOUn0vxWgT3L7xmF4Wo4DMH6e35hUCvWwPS/nfpcjUnKOe0NTFXcgTSL0AWZElrcfio6bFFYzgcF8T391lFDc27vNaprgmEfM+3CCn1efaPyNpPmsxNq1+f9klPQHXXgFa0hEeKjqAiloGgso0vNPD/Je0Nt4Py/mAaVrEU/id/E16iMAH7VHM7v4wfajhESjuBY3LhSDlqWdIhkJm1ETHBqS35YG5NPtK+PidoJGLzet0mjDZXQZK+Gupall1f66md/BLNwNwRApef5M8kSYmvph0ibhpEY6U7tWrFSef022o1ZcEA2DF1tyIoW86pIH4D2g7HJ4xvwNdE4g1KDHvQo6XjvZ0KFiC+1SnFZek+4kqijJCNmuJ9H3ZIKWzhWbvJt0xSJcy62tEP5us7q43AdZbljlkicQGPJUEitUFO1FkYIMEkl3gGELRlgVElmwgfTsth24gZ6nbiA89fBd5tiT09uEVgjkjMgwHufE+P+XdED+HGM65PGrT4iY7uKwMV88hxgF5B57Uzj4WKKSyyuMFgvBA8fDBFcJMekQBTkq4u1Vm2T9ZSjrjafkG1CPkamxClx/sq5DcgaUC2XmIIhbnJ/IkH/hKRBIveF6FwnrPukrIASvDhUlD0Y/BojLpGkJaSoACrTfFZNh/dY/fWy7QdbB+inj9cOvYUdw8RLfTxAdojaBoNapf/w6cNLRHVitTxfFeYKuEPtVtSIo1dKFdRKR2fEUxpiUhXAnZ1cpngc5pLuMBJeorM+M1GHE6A+QVYA/HLfKfkf/KiDAUHLMmWwq/2uIvvuckQT/QaVvHxpU7fI+h+ogek6sCv2kSfmbQELw4DnumEWGeA4M6hHctNlFyLauq/1k1xqwHvUcXGEHH6jaKNevgazTOoEEDED8zFXSsRPNgCpo8yFIVMDD7j6JtrXecDJO41PLRU1JhdqmgCLas69a4ktcMHqQtPUCgH6QHuGjyQONxqssg0M7pyrrN1Psq1yhsN8g2VisCGG3TAAggFGj+F5U3bXI9eU4uAg== X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: e0194c28-be20-46f0-b686-08db3093bfb5 X-MS-Exchange-CrossTenant-AuthSource: MWHPR0101MB2893.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Mar 2023 20:25:36.2320 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: x9oasX1J7xCfaYEM+34cE+BQOtg7Aok5Cg0zIUeEXbl56qtV9VgRwJtufDVQDiCR1XMjaommmrysDyE66d8Ca5LYhisKujo2w51P0+yS92TU4mM0ZDDl326sSomX7Gu9 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR01MB4260 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230329_132548_462789_1A99BD64 X-CRM114-Status: GOOD ( 25.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Interrupts are blocked in SDEI context, per the SDEI spec: "The client interrupts cannot preempt the event handler." If we crashed in the SDEI handler-running context (as with ACPI's AGDI) then we need to clean up the SDEI state before proceeding to the crash kernel so that the crash kernel can have working interrupts. Track the active SDEI handler per-cpu so that we can COMPLETE_AND_RESUME the handler, discarding the interrupted context. Fixes: f5df26961853 ("arm64: kernel: Add arch-specific SDEI entry code and CPU masking") Signed-off-by: D Scott Phillips --- Hi James, I added the per-cpu active event info instead of the stack pointer comparison you suggested for the purpose of handling the case where a critical event handler interrupts a normal event handler. I actually started adding it to get at the interrupted stack pointer, but then realized I could just use the active event info itself. This does leave a small gap where a critical event could interrput a normal event before or after the setting or clearing of the per-cpu active info. I suppose it could be covered by adding in _end labels to __sdei_entry_trampoline, __sdei_exit_trampoline, and __sdei_asm_handler, and checking for the critical event's interrupted pc within those functions. I didn't do that here in the thinking that this is good enough, thoughts on that? Changes since v1: - Store the active SDEI event being handled per-cpu, use the per-cpu active handler information to know when to abort. - Add prints before attempting to abort sdei handlers. v1 Link: https://lore.kernel.org/linux-arm-kernel/20230204000851.3871-1-scott@os.amperecomputing.com/ arch/arm64/include/asm/sdei.h | 8 ++++++++ arch/arm64/kernel/entry.S | 24 ++++++++++++++++++++++++ arch/arm64/kernel/sdei.c | 3 +++ arch/arm64/kernel/smp.c | 27 +++++++++++++++++++++++---- 4 files changed, 58 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h index badb39510515..58bc9ebe4a6d 100644 --- a/arch/arm64/include/asm/sdei.h +++ b/arch/arm64/include/asm/sdei.h @@ -13,6 +13,11 @@ #include +#ifdef CONFIG_ARM_SDE_INTERFACE +DECLARE_PER_CPU(struct sdei_registered_event *, sdei_active_normal_event); +DECLARE_PER_CPU(struct sdei_registered_event *, sdei_active_critical_event); +#endif + /* Software Delegated Exception entry point from firmware*/ asmlinkage void __sdei_asm_handler(unsigned long event_num, unsigned long arg, unsigned long pc, unsigned long pstate); @@ -23,6 +28,9 @@ asmlinkage void __sdei_asm_entry_trampoline(unsigned long event_num, unsigned long pc, unsigned long pstate); +/* Abort a running handler. Context is discarded. */ +void sdei_handler_abort(void); + /* * The above entry point does the minimum to call C code. This function does * anything else, before calling the driver. diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 80b6f67a67d4..253755bb33d7 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -998,6 +998,12 @@ SYM_CODE_START(__sdei_asm_handler) ldrb w4, [x19, #SDEI_EVENT_PRIORITY] #endif + cbnz w4, 1f + adr_this_cpu dst=x5, sym=sdei_active_normal_event, tmp=x6 + b 2f +1: adr_this_cpu dst=x5, sym=sdei_active_critical_event, tmp=x6 +2: str x19, [x5] + #ifdef CONFIG_VMAP_STACK /* * entry.S may have been using sp as a scratch register, find whether @@ -1061,6 +1067,13 @@ SYM_CODE_START(__sdei_asm_handler) mov_q x3, SDEI_1_0_FN_SDEI_EVENT_COMPLETE_AND_RESUME csel x0, x2, x3, ls + ldrb w3, [x4, #SDEI_EVENT_PRIORITY] + cbnz w3, 1f + adr_this_cpu dst=x5, sym=sdei_active_normal_event, tmp=x6 + b 2f +1: adr_this_cpu dst=x5, sym=sdei_active_critical_event, tmp=x6 +2: str xzr, [x5] + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 alternative_if ARM64_UNMAP_KERNEL_AT_EL0 b 1f @@ -1075,4 +1088,15 @@ alternative_else_nop_endif #endif SYM_CODE_END(__sdei_asm_handler) NOKPROBE(__sdei_asm_handler) + +SYM_CODE_START(sdei_handler_abort) + mov_q x0, SDEI_1_0_FN_SDEI_EVENT_COMPLETE_AND_RESUME + adr x1, 1f + sdei_handler_exit + // exit the handler and jump to the next instruction. + // Exit will stomp x0-x17, PSTATE, ELR_ELx, and SPSR_ELx. +1: ret +SYM_CODE_END(sdei_handler_abort) +NOKPROBE(sdei_handler_abort) + #endif /* CONFIG_ARM_SDE_INTERFACE */ diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index be7f6ea49956..07fa9630f0ce 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -45,6 +45,9 @@ DEFINE_PER_CPU(unsigned long *, sdei_shadow_call_stack_normal_ptr); DEFINE_PER_CPU(unsigned long *, sdei_shadow_call_stack_critical_ptr); #endif +DEFINE_PER_CPU(struct sdei_registered_event *, sdei_active_normal_event); +DEFINE_PER_CPU(struct sdei_registered_event *, sdei_active_critical_event); + static void _free_sdei_stack(unsigned long * __percpu *ptr, int cpu) { unsigned long *p; diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index ffc5d76cf695..311430990175 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -1047,10 +1047,8 @@ void crash_smp_send_stop(void) * If this cpu is the only one alive at this point in time, online or * not, there are no stop messages to be sent around, so just back out. */ - if (num_other_online_cpus() == 0) { - sdei_mask_local_cpu(); - return; - } + if (num_other_online_cpus() == 0) + goto skip_ipi; cpumask_copy(&mask, cpu_online_mask); cpumask_clear_cpu(smp_processor_id(), &mask); @@ -1069,7 +1067,28 @@ void crash_smp_send_stop(void) pr_warn("SMP: failed to stop secondary CPUs %*pbl\n", cpumask_pr_args(&mask)); +skip_ipi: sdei_mask_local_cpu(); + +#ifdef CONFIG_ARM_SDE_INTERFACE + /* + * If the crash happened in an SDEI event handler then we need to + * finish the handler with the firmware so that we can have working + * interrupts in the crash kernel. + */ + if (__this_cpu_read(sdei_active_critical_event)) { + pr_warn("SMP: stopped CPUS from SDEI critical event handler " + "context, attempting to finish handler.\n"); + sdei_handler_abort(); + __this_cpu_write(sdei_active_critical_event, NULL); + } + if (__this_cpu_read(sdei_active_normal_event)) { + pr_warn("SMP: stopped CPUS from SDEI normal event handler " + "context, attempting to finish handler.\n"); + sdei_handler_abort(); + __this_cpu_write(sdei_active_normal_event, NULL); + } +#endif } bool smp_crash_stop_failed(void)