From patchwork Wed Oct 19 08:47:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaxi Chen X-Patchwork-Id: 13011489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D260C433FE for ; Wed, 19 Oct 2022 08:57:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231947AbiJSI5D (ORCPT ); Wed, 19 Oct 2022 04:57:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231950AbiJSI40 (ORCPT ); Wed, 19 Oct 2022 04:56:26 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DEF652479; Wed, 19 Oct 2022 01:52:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666169565; x=1697705565; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aN1z8mG+6FtVyNG8T1B6yk4JgG7etoGpqzlwCguutHU=; b=RjHTlC5clSe6EgBqvr3QFVOpn22g9Q3pVRE4XFeBMMPBSJ41cUqWkASw /VzXldnn0tipfKpgqNHMTZHfkYNxgsAWUOw1YB49ks+mfR86r+PuH83pc u5VD9QrUZkuWjJwfTADviemQIdlN2/jLUFUhsQ0qWT/pCmfjoOwvgzwcw dezP8knaaeciF1iGq4dFwCBDybLs9k7rDuoxwE0c2vcjiqMKJNB7f5fSj PrCVvaNlB3F5xLmwJY98E5NXPG36y2N4UNOYrujE0s4xanujQwkSQL6zu ESZpHYiLZNTxBw3lc2EtT27HRcYDTqRpdMM5Bmpa53MmwPKkODHj/MSiS g==; X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="286065913" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="286065913" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 01:47:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="804195844" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="804195844" Received: from jiaxichen-precision-3650-tower.sh.intel.com ([10.239.159.75]) by orsmga005.jf.intel.com with ESMTP; 19 Oct 2022 01:47:39 -0700 From: Jiaxi Chen To: kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, ndesaulniers@google.com, alexandre.belloni@bootlin.com, peterz@infradead.org, jiaxi.chen@linux.intel.com, jpoimboe@kernel.org, chang.seok.bae@intel.com, pawan.kumar.gupta@linux.intel.com, babu.moger@amd.com, jmattson@google.com, sandipan.das@amd.com, tony.luck@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, fenghua.yu@intel.com, keescook@chromium.org, jane.malalane@citrix.com, nathan@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/6] x86: KVM: Enable CMPccXADD CPUID and expose it to guest Date: Wed, 19 Oct 2022 16:47:29 +0800 Message-Id: <20221019084734.3590760-2-jiaxi.chen@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> References: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org CMPccXADD is a new set of instructions in the latest Intel platform Sierra Forest. It includes a semaphore operation that can compare and add the operands if condition is met, which can improve database performance. The bit definition: CPUID.(EAX=7,ECX=1):EAX[bit 7] This patch enables this CPUID in the kernel feature bits and expose it to guest OS. Signed-off-by: Jiaxi Chen --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index ef4775c6db01..445626cb5779 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -308,6 +308,7 @@ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* AVX512 BFLOAT16 instructions */ +#define X86_FEATURE_CMPCCXADD (12*32+ 7) /* CMPccXADD instructions */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0) /* CLZERO instruction */ diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 7065462378e2..3f745f6fdc43 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -657,7 +657,7 @@ void kvm_set_cpu_caps(void) kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL_SSBD); kvm_cpu_cap_mask(CPUID_7_1_EAX, - F(AVX_VNNI) | F(AVX512_BF16) + F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) ); kvm_cpu_cap_mask(CPUID_D_1_EAX, From patchwork Wed Oct 19 08:47:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaxi Chen X-Patchwork-Id: 13011482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15A16C4332F for ; Wed, 19 Oct 2022 08:53:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231828AbiJSIxF (ORCPT ); Wed, 19 Oct 2022 04:53:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59220 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231627AbiJSIwf (ORCPT ); Wed, 19 Oct 2022 04:52:35 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B1D51FCF4; Wed, 19 Oct 2022 01:50:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666169409; x=1697705409; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yv4TuyK9cOpIJfyoOHskJ3JXY6oP4TpsI2dJ99hsGpM=; b=JJPLHN/RVD4iBpNRXI3ZnL1s6qd48JShNX/bpebI1Lp6iCH8T79EUqxj oZefYIjBqB6+cD7g0ykzLJzuu+HHdFS0gqHl9QGiMvRAjwVS6nQzesH2D Cr3QY51wIXITnmZh6K22ZXRuZ0hA+3u+ErhCvi7ZKYi2fohBuBe1//Z7R r47x4j4hHqGrawASFzxT90BtHFfP/LsJLFhQKGrhZs37dPdVicDqfotbJ TLvKrlkcUyPOGn+OBcZCzqVbivrLBBna1JWFTVc12Q6+tAIAKrIpjAVpv 46cYUdB/wvlGHutwf50iHr4ssSHOXfaqUZtvTxrOBRBA4EMZBDOgyS5Cq Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="307456596" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="307456596" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 01:47:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="804195881" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="804195881" Received: from jiaxichen-precision-3650-tower.sh.intel.com ([10.239.159.75]) by orsmga005.jf.intel.com with ESMTP; 19 Oct 2022 01:47:45 -0700 From: Jiaxi Chen To: kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, ndesaulniers@google.com, alexandre.belloni@bootlin.com, peterz@infradead.org, jiaxi.chen@linux.intel.com, jpoimboe@kernel.org, chang.seok.bae@intel.com, pawan.kumar.gupta@linux.intel.com, babu.moger@amd.com, jmattson@google.com, sandipan.das@amd.com, tony.luck@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, fenghua.yu@intel.com, keescook@chromium.org, jane.malalane@citrix.com, nathan@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/6] x86: KVM: Enable AMX-FP16 CPUID and expose it to guest Date: Wed, 19 Oct 2022 16:47:30 +0800 Message-Id: <20221019084734.3590760-3-jiaxi.chen@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> References: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "Chang S. Bae" Latest Intel platform Granite Rapids has introduced a new instruction - AMX-FP16, which performs dot-products of two FP16 tiles and accumulates the results into a packed single precision tile. This instruction adds FP16 capability and also allows a FP16 GPU trained model to run faster without loss of accuracy or added SW overhead. The bit definition: CPUID.(EAX=7,ECX=1):EAX[bit 21] This patch enables this CPUID in the kernel feature bits and expose it to guest OS. Signed-off-by: Chang S. Bae Signed-off-by: Jiaxi Chen --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 445626cb5779..9313240e3cdd 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -309,6 +309,7 @@ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* AVX512 BFLOAT16 instructions */ #define X86_FEATURE_CMPCCXADD (12*32+ 7) /* CMPccXADD instructions */ +#define X86_FEATURE_AMX_FP16 (12*32+21) /* AMX fp16 Support */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0) /* CLZERO instruction */ diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 3f745f6fdc43..d983ddb974ba 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -657,7 +657,7 @@ void kvm_set_cpu_caps(void) kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL_SSBD); kvm_cpu_cap_mask(CPUID_7_1_EAX, - F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) + F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) | F(AMX_FP16) ); kvm_cpu_cap_mask(CPUID_D_1_EAX, From patchwork Wed Oct 19 08:47:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaxi Chen X-Patchwork-Id: 13011484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E67B2C433FE for ; Wed, 19 Oct 2022 08:56:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231925AbiJSI4S (ORCPT ); Wed, 19 Oct 2022 04:56:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49268 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232187AbiJSIyz (ORCPT ); Wed, 19 Oct 2022 04:54:55 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F41588E7BC; Wed, 19 Oct 2022 01:51:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666169520; x=1697705520; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fJWzlcjG0YN6OU/ylWIG97jYiSOWaB5SlA8WriOTTHY=; b=UJfJuZ/K1hnorhUbhmBFFdrF+pmRqDA0Jy8yvYtgl9V9PeBkqVi4bmU1 hnyk3oQZDDZ0WXu4FcbcfxrJf0kvO8qTI3ZutaTI/WxMOApCC369tjb/o 5YHuU3/ePpdidW4wJvo4F1+eF2SpQSx6zogbAQf5oU8WPTo+AfGeFnpxD KwzWlsetLx45gnuZLJ3wsObnEgATYSn6YiaUWGljdL8TUUvw5iEhPL9GO mYZUSacyqaXRe6Vv/pBvenk9yolqQ/kxNOe+Xxkwu8dqda3b4qUsX8nbr 8TinfM0GiqRXyXr+9niXjsM4FBgoYfOwiucbOkSdWP5xx6R/xCtGgbQKC A==; X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="392649494" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="392649494" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 01:47:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="804195928" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="804195928" Received: from jiaxichen-precision-3650-tower.sh.intel.com ([10.239.159.75]) by orsmga005.jf.intel.com with ESMTP; 19 Oct 2022 01:47:50 -0700 From: Jiaxi Chen To: kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, ndesaulniers@google.com, alexandre.belloni@bootlin.com, peterz@infradead.org, jiaxi.chen@linux.intel.com, jpoimboe@kernel.org, chang.seok.bae@intel.com, pawan.kumar.gupta@linux.intel.com, babu.moger@amd.com, jmattson@google.com, sandipan.das@amd.com, tony.luck@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, fenghua.yu@intel.com, keescook@chromium.org, jane.malalane@citrix.com, nathan@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/6] x86: KVM: Enable AVX-IFMA CPUID and expose it to guest Date: Wed, 19 Oct 2022 16:47:31 +0800 Message-Id: <20221019084734.3590760-4-jiaxi.chen@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> References: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org AVX-IFMA is a new instruction in the latest Intel platform Sierra Forest. AVX-IFMA packed multiplies unsigned 52-bit integers and add the low/high 52-bit products to Qword Accumulators. The bit definition: CPUID.(EAX=7,ECX=1):EAX[bit 23] This patch enables this CPUID in the kernel feature bits and expose it to guest OS. Signed-off-by: Jiaxi Chen --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 9313240e3cdd..a682f646243f 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -310,6 +310,7 @@ #define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* AVX512 BFLOAT16 instructions */ #define X86_FEATURE_CMPCCXADD (12*32+ 7) /* CMPccXADD instructions */ #define X86_FEATURE_AMX_FP16 (12*32+21) /* AMX fp16 Support */ +#define X86_FEATURE_AVX_IFMA (12*32+23) /* Support for VPMADD52[H,L]UQ */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0) /* CLZERO instruction */ diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index d983ddb974ba..837bcd1373e5 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -657,8 +657,8 @@ void kvm_set_cpu_caps(void) kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL_SSBD); kvm_cpu_cap_mask(CPUID_7_1_EAX, - F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) | F(AMX_FP16) - ); + F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) | F(AMX_FP16) | + F(AVX_IFMA)); kvm_cpu_cap_mask(CPUID_D_1_EAX, F(XSAVEOPT) | F(XSAVEC) | F(XGETBV1) | F(XSAVES) | f_xfd From patchwork Wed Oct 19 08:47:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaxi Chen X-Patchwork-Id: 13011488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8E2CC433FE for ; Wed, 19 Oct 2022 08:56:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231960AbiJSI4k (ORCPT ); Wed, 19 Oct 2022 04:56:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231860AbiJSIzk (ORCPT ); Wed, 19 Oct 2022 04:55:40 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4DCB9B875; Wed, 19 Oct 2022 01:52:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666169533; x=1697705533; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mMORpyBJhxkfSV067Kavho36Iu4oPBvAncoY3FxSR6s=; b=lOlt412Vld3aJ4gvn8Y8sVJsRnDzLzFQuMHwsCfyY7NjbtZhfxBYLo0K G0tIB+hacNyyfuvIqVHct2KpHCVHdD/QXW6Tay1zA34asG6rmkqNWYgiy V3F0pACWE2FAujUxGMm7BnpUKzClEJ3nEtUJbUfCK1Hh96g2vok27zXLy iG2FNM628VsVDNFgpGQWkKhYLexKNIW0DoTDzhxCc4pXY4bPjhHaPepYc RhOuhh9XgnbDt27sibK/UtkjD/eyMe/iH3+4L/P7IosAGaJOFeRKHtreD PYSmWYfFmvDNobIsEtunvbq0/UOLEvbDNtTqu41JcdOl+hQCln+9qbOaV Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="392649505" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="392649505" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 01:48:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="804195982" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="804195982" Received: from jiaxichen-precision-3650-tower.sh.intel.com ([10.239.159.75]) by orsmga005.jf.intel.com with ESMTP; 19 Oct 2022 01:47:56 -0700 From: Jiaxi Chen To: kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, ndesaulniers@google.com, alexandre.belloni@bootlin.com, peterz@infradead.org, jiaxi.chen@linux.intel.com, jpoimboe@kernel.org, chang.seok.bae@intel.com, pawan.kumar.gupta@linux.intel.com, babu.moger@amd.com, jmattson@google.com, sandipan.das@amd.com, tony.luck@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, fenghua.yu@intel.com, keescook@chromium.org, jane.malalane@citrix.com, nathan@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/6] x86: KVM: Enable AVX-VNNI-INT8 CPUID and expose it to guest Date: Wed, 19 Oct 2022 16:47:32 +0800 Message-Id: <20221019084734.3590760-5-jiaxi.chen@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> References: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org AVX-VNNI-INT8 is a new set of instructions in the latest Intel platform Sierra Forest. It multiplies the individual bytes of two unsigned or unsigned source operands, then add and accumulate the results into the destination dword element size operand. This instruction allows for the platform to have superior AI capabilities. The bit definition: CPUID.(EAX=7,ECX=1):EDX[bit 4] This patch enables this CPUID in the kernel feature bits and expose it to guest OS. Since the CPUID involves a bit of EDX (EAX=7,ECX=1) which has not been enumerated yet, this patch adds CPUID_7_1_EDX to CPUID subleaves. At the same time, word 20 is newly-defined in CPU features for CPUID level 0x00000007:1 (EDX). Signed-off-by: Jiaxi Chen --- arch/x86/include/asm/cpufeature.h | 7 +++++-- arch/x86/include/asm/cpufeatures.h | 5 ++++- arch/x86/include/asm/disabled-features.h | 3 ++- arch/x86/include/asm/required-features.h | 3 ++- arch/x86/kernel/cpu/common.c | 1 + arch/x86/kvm/cpuid.c | 5 ++++- arch/x86/kvm/reverse_cpuid.h | 1 + 7 files changed, 19 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufeature.h index 1a85e1fb0922..d46b802930b0 100644 --- a/arch/x86/include/asm/cpufeature.h +++ b/arch/x86/include/asm/cpufeature.h @@ -32,6 +32,7 @@ enum cpuid_leafs CPUID_8000_0007_EBX, CPUID_7_EDX, CPUID_8000_001F_EAX, + CPUID_7_1_EDX, }; #define X86_CAP_FMT_NUM "%d:%d" @@ -94,8 +95,9 @@ extern const char * const x86_bug_flags[NBUGINTS*32]; CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 17, feature_bit) || \ CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 18, feature_bit) || \ CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 19, feature_bit) || \ + CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 20, feature_bit) || \ REQUIRED_MASK_CHECK || \ - BUILD_BUG_ON_ZERO(NCAPINTS != 20)) + BUILD_BUG_ON_ZERO(NCAPINTS != 21)) #define DISABLED_MASK_BIT_SET(feature_bit) \ ( CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 0, feature_bit) || \ @@ -118,8 +120,9 @@ extern const char * const x86_bug_flags[NBUGINTS*32]; CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 17, feature_bit) || \ CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 18, feature_bit) || \ CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 19, feature_bit) || \ + CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 20, feature_bit) || \ DISABLED_MASK_CHECK || \ - BUILD_BUG_ON_ZERO(NCAPINTS != 20)) + BUILD_BUG_ON_ZERO(NCAPINTS != 21)) #define cpu_has(c, bit) \ (__builtin_constant_p(bit) && REQUIRED_MASK_BIT_SET(bit) ? 1 : \ diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index a682f646243f..b2aa761ea110 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -13,7 +13,7 @@ /* * Defines x86 CPU feature bits */ -#define NCAPINTS 20 /* N 32-bit words worth of info */ +#define NCAPINTS 21 /* N 32-bit words worth of info */ #define NBUGINTS 1 /* N 32-bit bug flags */ /* @@ -423,6 +423,9 @@ #define X86_FEATURE_V_TSC_AUX (19*32+ 9) /* "" Virtual TSC_AUX */ #define X86_FEATURE_SME_COHERENT (19*32+10) /* "" AMD hardware-enforced cache coherency */ +/* Intel-defined CPU features, CPUID level 0x00000007:1 (EDX), word 20 */ +#define X86_FEATURE_AVX_VNNI_INT8 (20*32+ 4) /* Support for VPDPB[SU,UU,SS]D[,S] */ + /* * BUG word(s) */ diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index 33d2cd04d254..2db6929ca868 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -111,6 +111,7 @@ #define DISABLED_MASK17 0 #define DISABLED_MASK18 0 #define DISABLED_MASK19 0 -#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20) +#define DISABLED_MASK20 0 +#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 21) #endif /* _ASM_X86_DISABLED_FEATURES_H */ diff --git a/arch/x86/include/asm/required-features.h b/arch/x86/include/asm/required-features.h index aff774775c67..3b5522b3f051 100644 --- a/arch/x86/include/asm/required-features.h +++ b/arch/x86/include/asm/required-features.h @@ -98,6 +98,7 @@ #define REQUIRED_MASK17 0 #define REQUIRED_MASK18 0 #define REQUIRED_MASK19 0 -#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20) +#define REQUIRED_MASK20 0 +#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 21) #endif /* _ASM_X86_REQUIRED_FEATURES_H */ diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 3e508f239098..7c659127ed89 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1031,6 +1031,7 @@ void get_cpu_cap(struct cpuinfo_x86 *c) if (eax >= 1) { cpuid_count(0x00000007, 1, &eax, &ebx, &ecx, &edx); c->x86_capability[CPUID_7_1_EAX] = eax; + c->x86_capability[CPUID_7_1_EDX] = edx; } } diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 837bcd1373e5..b1b53a5c788a 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -660,6 +660,9 @@ void kvm_set_cpu_caps(void) F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) | F(AMX_FP16) | F(AVX_IFMA)); + kvm_cpu_cap_mask(CPUID_7_1_EDX, + F(AVX_VNNI_INT8)); + kvm_cpu_cap_mask(CPUID_D_1_EAX, F(XSAVEOPT) | F(XSAVEC) | F(XGETBV1) | F(XSAVES) | f_xfd ); @@ -913,9 +916,9 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) goto out; cpuid_entry_override(entry, CPUID_7_1_EAX); + cpuid_entry_override(entry, CPUID_7_1_EDX); entry->ebx = 0; entry->ecx = 0; - entry->edx = 0; } break; case 0xa: { /* Architectural Performance Monitoring */ diff --git a/arch/x86/kvm/reverse_cpuid.h b/arch/x86/kvm/reverse_cpuid.h index a19d473d0184..fb4f4bffad5c 100644 --- a/arch/x86/kvm/reverse_cpuid.h +++ b/arch/x86/kvm/reverse_cpuid.h @@ -48,6 +48,7 @@ static const struct cpuid_reg reverse_cpuid[] = { [CPUID_7_1_EAX] = { 7, 1, CPUID_EAX}, [CPUID_12_EAX] = {0x00000012, 0, CPUID_EAX}, [CPUID_8000_001F_EAX] = {0x8000001f, 0, CPUID_EAX}, + [CPUID_7_1_EDX] = { 7, 1, CPUID_EDX}, }; /* From patchwork Wed Oct 19 08:47:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaxi Chen X-Patchwork-Id: 13011483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7329C4332F for ; Wed, 19 Oct 2022 08:55:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231688AbiJSIzN (ORCPT ); Wed, 19 Oct 2022 04:55:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231941AbiJSIyL (ORCPT ); Wed, 19 Oct 2022 04:54:11 -0400 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30AD49620A; Wed, 19 Oct 2022 01:51:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666169475; x=1697705475; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+X6cm8tXEq53y0cLc6cRR4TvIr9g+2l/NPCkvPotupE=; b=aM8QV0DubBd1jl08CrpYJxL6nf55tub/CGuPNKAZb/i3eW4k73apdiWb zJc6I9XVZaaDrqKbmZoDDfm7tBxisD6VXTNF7KEsouPKdNa8+0kdsznSH tcWeDgg86GCPs70fM3H2xl28q675dbbuQ/5JNV/lxEI+kIiIHc7/p5iZx SQ0CuFuXpSxUAisG/8mcE2k/9t2IfbgUDftUb3McbESw8Z9sFkU8aX/iC +hDbts3fz2MDBpuCnny66hS0QZfdXvOc5k1UvU230cBK2FPWeLBciw+6g 7Lv4mMo9OFtGDcDRltlBkO+zgsutOl95hNStuEJqCnlBo2X9SXA2OKrIB Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="286739743" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="286739743" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 01:48:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="804196026" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="804196026" Received: from jiaxichen-precision-3650-tower.sh.intel.com ([10.239.159.75]) by orsmga005.jf.intel.com with ESMTP; 19 Oct 2022 01:48:02 -0700 From: Jiaxi Chen To: kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, ndesaulniers@google.com, alexandre.belloni@bootlin.com, peterz@infradead.org, jiaxi.chen@linux.intel.com, jpoimboe@kernel.org, chang.seok.bae@intel.com, pawan.kumar.gupta@linux.intel.com, babu.moger@amd.com, jmattson@google.com, sandipan.das@amd.com, tony.luck@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, fenghua.yu@intel.com, keescook@chromium.org, jane.malalane@citrix.com, nathan@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/6] x86: KVM: Enable AVX-NE-CONVERT CPUID and expose it to guest Date: Wed, 19 Oct 2022 16:47:33 +0800 Message-Id: <20221019084734.3590760-6-jiaxi.chen@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> References: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org AVX-NE-CONVERT is a new set of instructions in the latest Intel platform Sierra Forest, which can convert low precision floating point like BF16/FP16 to high precision floating point FP32. It can also convert FP32 elements to BF16. This instruction allows the platform to have improved AI capabilities and better compatibility. The bit definition: CPUID.(EAX=7,ECX=1):EDX[bit 5] This patch enables this CPUID in the kernel feature bits and expose it to guest OS. Signed-off-by: Jiaxi Chen --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index b2aa761ea110..4e0bbffedbb4 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -425,6 +425,7 @@ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EDX), word 20 */ #define X86_FEATURE_AVX_VNNI_INT8 (20*32+ 4) /* Support for VPDPB[SU,UU,SS]D[,S] */ +#define X86_FEATURE_AVX_NE_CONVERT (20*32+ 5) /* AVX NE CONVERT Instructions */ /* * BUG word(s) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index b1b53a5c788a..fcd00c68e546 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -661,7 +661,8 @@ void kvm_set_cpu_caps(void) F(AVX_IFMA)); kvm_cpu_cap_mask(CPUID_7_1_EDX, - F(AVX_VNNI_INT8)); + F(AVX_VNNI_INT8) | F(AVX_NE_CONVERT) + ); kvm_cpu_cap_mask(CPUID_D_1_EAX, F(XSAVEOPT) | F(XSAVEC) | F(XGETBV1) | F(XSAVES) | f_xfd From patchwork Wed Oct 19 08:47:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaxi Chen X-Patchwork-Id: 13011490 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 977A2C433FE for ; Wed, 19 Oct 2022 09:00:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229919AbiJSJAE (ORCPT ); Wed, 19 Oct 2022 05:00:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232319AbiJSI72 (ORCPT ); Wed, 19 Oct 2022 04:59:28 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6EBE1260F; Wed, 19 Oct 2022 01:54:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666169696; x=1697705696; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Xh50grAc2J9OL82QlhScNH50h2hDqC36SRcoKoSLDwA=; b=lCbh+6FxrCSqBrcKL2DjQzfd1gqBG4pkiH6CCZRZy33Pb1cf4JsxHbHm 1F+6UJO7lgMxsycSskJLtYnOXK5cWvfVt9kaacIQLkkTtsO1Xlp0/OFhA SgwUjCFy28XrFcAQ9mLOEj3V2bPCG7v6Ww96ye/LMGLzMIBFc5Fy/G8Hw k8aYZ8sLI0PpEeifBsPK318vhnQfJ9DzAy5Z2GnDy6gVg+x1G10omPj96 2vuJFu+xnVMLP/JO/wZMrTOMAd+wLMHL4ZKo4ORzLZTMKZBc4SCp9PicK zsSMYBlzrTpaCHhu/9ZhuVGHh4gp/IlEXpxvcj4uXmO/PGRxLBSv9wM72 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="370567294" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="370567294" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 01:48:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="804196060" X-IronPort-AV: E=Sophos;i="5.95,195,1661842800"; d="scan'208";a="804196060" Received: from jiaxichen-precision-3650-tower.sh.intel.com ([10.239.159.75]) by orsmga005.jf.intel.com with ESMTP; 19 Oct 2022 01:48:07 -0700 From: Jiaxi Chen To: kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, ndesaulniers@google.com, alexandre.belloni@bootlin.com, peterz@infradead.org, jiaxi.chen@linux.intel.com, jpoimboe@kernel.org, chang.seok.bae@intel.com, pawan.kumar.gupta@linux.intel.com, babu.moger@amd.com, jmattson@google.com, sandipan.das@amd.com, tony.luck@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, fenghua.yu@intel.com, keescook@chromium.org, jane.malalane@citrix.com, nathan@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/6] x86: KVM: Enable PREFETCHIT0/1 CPUID and expose it to guest Date: Wed, 19 Oct 2022 16:47:34 +0800 Message-Id: <20221019084734.3590760-7-jiaxi.chen@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> References: <20221019084734.3590760-1-jiaxi.chen@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Latest Intel platform Granite Rapids has introduced a new instruction - PREFETCHIT0/1, which moves code to memory (cache) closer to the processor depending on specific hints. The bit definition: CPUID.(EAX=7,ECX=1):EDX[bit 14] This patch enables this CPUID in the kernel feature bits and expose it to guest OS. Signed-off-by: Jiaxi Chen --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 4e0bbffedbb4..de8aa62bbdb1 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -426,6 +426,7 @@ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EDX), word 20 */ #define X86_FEATURE_AVX_VNNI_INT8 (20*32+ 4) /* Support for VPDPB[SU,UU,SS]D[,S] */ #define X86_FEATURE_AVX_NE_CONVERT (20*32+ 5) /* AVX NE CONVERT Instructions */ +#define X86_FEATURE_PREFETCHITI (20*32+14) /* PREFETCHIT0/1 Instructions */ /* * BUG word(s) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index fcd00c68e546..e73307732d10 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -661,7 +661,7 @@ void kvm_set_cpu_caps(void) F(AVX_IFMA)); kvm_cpu_cap_mask(CPUID_7_1_EDX, - F(AVX_VNNI_INT8) | F(AVX_NE_CONVERT) + F(AVX_VNNI_INT8) | F(AVX_NE_CONVERT) | F(PREFETCHITI) ); kvm_cpu_cap_mask(CPUID_D_1_EAX,