From patchwork Wed Apr 6 12:40:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12803654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CBDCC433F5 for ; Wed, 6 Apr 2022 15:40:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236088AbiDFPmQ (ORCPT ); Wed, 6 Apr 2022 11:42:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235977AbiDFPll (ORCPT ); Wed, 6 Apr 2022 11:41:41 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 243C43ED985 for ; Wed, 6 Apr 2022 05:58:30 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7AB20210F4; Wed, 6 Apr 2022 12:39:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649248797; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=9mbjXaGNcUpVUJZqcZeqthHEqPAYzBSqmaYcs1QWZfo=; b=kNwYOu8uOCnpjySP4lqsXYUtDZSFFUekxCLDCXMsT1cb64QbzEK1k6UAE87N8qmN7x4C/z DAxymzXqaVRpaLTwN2/VBkjRlDE/xsZgl+ZkQp33lr9+SNftCfmsDAsQDeEr87a0T99qZa Sewerfh9vZFFpHqJB9F8OjLXGO02mTg= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3738613A8E; Wed, 6 Apr 2022 12:39:57 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 90mGCx2KTWKgaAAAMHmgww (envelope-from ); Wed, 06 Apr 2022 12:39:57 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, jroedel@suse.de, varad.gautam@suse.com Subject: [kvm-unit-tests PATCH] x86: cstart64: Serialize ap_start64 with a spin lock Date: Wed, 6 Apr 2022 14:40:02 +0200 Message-Id: <20220406124002.13741-1-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org ap_start64 serves as the 64-bit entrypoint for APs during bringup. Since apic.c:apic_ops is not guarded against concurrent accesses, there exists a race between reset_apic(), enable_apic() and enable_x2apic() which results in APs crashing or getting blocked in various scenarios (eg, enabling x2apic while disabling xapic). The bug is rare with vcpu count < 32, but becomes easier to reproduce with vcpus > 64 and the following thunk: lib/x86/apic.c: void enable_apic(void) { - printf("enabling apic\n"); xapic_write(APIC_SPIV, 0x1ff); } Serialize the bringup code in ap_start64 to fix this. Signed-off-by: Varad Gautam --- x86/cstart64.S | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/x86/cstart64.S b/x86/cstart64.S index 7272452..238cebf 100644 --- a/x86/cstart64.S +++ b/x86/cstart64.S @@ -45,6 +45,9 @@ mb_boot_info: .quad 0 pt_root: .quad ptl4 +ap_lock: + .long 0 + .section .init .code32 @@ -188,12 +191,18 @@ save_id: retq ap_start64: +.retry: + xor %eax, %eax + lock btsl %eax, ap_lock + jc .retry call reset_apic load_tss call enable_apic call save_id call enable_x2apic sti + xor %eax, %eax + lock btr %eax, ap_lock nop lock incw cpu_online_count