From patchwork Fri Jan 26 23:42:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533600 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D67EB5914E; Fri, 26 Jan 2024 23:37:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312245; cv=none; b=IlwzRLHrUNs5IUyWlyazOTJ8dQqOfUyyo4k7jDE3V5KxlFQl7F+pFUly4b2szxjx7l9X1im5WNAB9YqZkGLe+aqu3wizO6bvOQ94VWo5vtASmde7iO9GIj5xXXiAZNUziFm19p/Ol7jU9SvFeZmf88z64AWNXnUkjxycQ7dJzR8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312245; c=relaxed/simple; bh=zBgPFasqGZOjr8K1swC88xgZ1PNA6b0D2k/Kg52KNW0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ThUVKNuxnSOU8FfLTLnVBEHS7ChtJLeW2p7FuE/psfST32mZqORknE+3uUAXLMUkgMaYeHgIB6Ho7tZOqq/no7JVwq8XEhbyLAuDAZuMSorfaIgQHQyVri+uBHm9RCu1qsez5RDEJ4dIq5HlXzzxE3UHW1r9LlKgk+y3yeQJucg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=kDt3zOO1; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="kDt3zOO1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312244; x=1737848244; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zBgPFasqGZOjr8K1swC88xgZ1PNA6b0D2k/Kg52KNW0=; b=kDt3zOO1pXD2HdzS6OZGdsWbv0gSfjOMTcs5P/bIXJUCRQeuVM6arC7j CGX/63b8lMvYI8bbFKRsbrzB5mMhG7m7ywS+GPZ55eEVFaovpW4JPWUsI BuVPo9QMgiOsNb86IwP5k3ZX7qliDUlTGtYM91gvTcci3tfweIuL/Ikfy +KZPjYYtLfrXSofkDOsKjxEUYH4LIOVmJuS74Zmyr/Vo0U1d77AqU9SiR A+35gDXRBueaoBqHXRXTlTNpsn9WYW8Fp6Qv8pXNxw3nSsFaaiBHLGzW6 qyPhLHLgqe6TPY57Gdrzyh24sO9X7kDkORHCWRQCE3bv2bzD9q/l3J416 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990631" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990631" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:20 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290708" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290708" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:19 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 01/15] x86/irq: Move posted interrupt descriptor out of vmx code Date: Fri, 26 Jan 2024 15:42:23 -0800 Message-Id: <20240126234237.547278-2-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To prepare native usage of posted interrupt, move PID declaration out of VMX code such that they can be shared. Signed-off-by: Jacob Pan --- arch/x86/include/asm/posted_intr.h | 88 ++++++++++++++++++++++++++++ arch/x86/kvm/vmx/posted_intr.h | 93 +----------------------------- arch/x86/kvm/vmx/vmx.c | 1 + arch/x86/kvm/vmx/vmx.h | 2 +- 4 files changed, 91 insertions(+), 93 deletions(-) create mode 100644 arch/x86/include/asm/posted_intr.h diff --git a/arch/x86/include/asm/posted_intr.h b/arch/x86/include/asm/posted_intr.h new file mode 100644 index 000000000000..f0324c56f7af --- /dev/null +++ b/arch/x86/include/asm/posted_intr.h @@ -0,0 +1,88 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _X86_POSTED_INTR_H +#define _X86_POSTED_INTR_H + +#define POSTED_INTR_ON 0 +#define POSTED_INTR_SN 1 + +#define PID_TABLE_ENTRY_VALID 1 + +/* Posted-Interrupt Descriptor */ +struct pi_desc { + u32 pir[8]; /* Posted interrupt requested */ + union { + struct { + /* bit 256 - Outstanding Notification */ + u16 on : 1, + /* bit 257 - Suppress Notification */ + sn : 1, + /* bit 271:258 - Reserved */ + rsvd_1 : 14; + /* bit 279:272 - Notification Vector */ + u8 nv; + /* bit 287:280 - Reserved */ + u8 rsvd_2; + /* bit 319:288 - Notification Destination */ + u32 ndst; + }; + u64 control; + }; + u32 rsvd[6]; +} __aligned(64); + +static inline bool pi_test_and_set_on(struct pi_desc *pi_desc) +{ + return test_and_set_bit(POSTED_INTR_ON, (unsigned long *)&pi_desc->control); +} + +static inline bool pi_test_and_clear_on(struct pi_desc *pi_desc) +{ + return test_and_clear_bit(POSTED_INTR_ON, (unsigned long *)&pi_desc->control); +} + +static inline bool pi_test_and_clear_sn(struct pi_desc *pi_desc) +{ + return test_and_clear_bit(POSTED_INTR_SN, (unsigned long *)&pi_desc->control); +} + +static inline bool pi_test_and_set_pir(int vector, struct pi_desc *pi_desc) +{ + return test_and_set_bit(vector, (unsigned long *)pi_desc->pir); +} + +static inline bool pi_is_pir_empty(struct pi_desc *pi_desc) +{ + return bitmap_empty((unsigned long *)pi_desc->pir, NR_VECTORS); +} + +static inline void pi_set_sn(struct pi_desc *pi_desc) +{ + set_bit(POSTED_INTR_SN, (unsigned long *)&pi_desc->control); +} + +static inline void pi_set_on(struct pi_desc *pi_desc) +{ + set_bit(POSTED_INTR_ON, (unsigned long *)&pi_desc->control); +} + +static inline void pi_clear_on(struct pi_desc *pi_desc) +{ + clear_bit(POSTED_INTR_ON, (unsigned long *)&pi_desc->control); +} + +static inline void pi_clear_sn(struct pi_desc *pi_desc) +{ + clear_bit(POSTED_INTR_SN, (unsigned long *)&pi_desc->control); +} + +static inline bool pi_test_on(struct pi_desc *pi_desc) +{ + return test_bit(POSTED_INTR_ON, (unsigned long *)&pi_desc->control); +} + +static inline bool pi_test_sn(struct pi_desc *pi_desc) +{ + return test_bit(POSTED_INTR_SN, (unsigned long *)&pi_desc->control); +} + +#endif /* _X86_POSTED_INTR_H */ diff --git a/arch/x86/kvm/vmx/posted_intr.h b/arch/x86/kvm/vmx/posted_intr.h index 26992076552e..6b2a0226257e 100644 --- a/arch/x86/kvm/vmx/posted_intr.h +++ b/arch/x86/kvm/vmx/posted_intr.h @@ -1,98 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ #ifndef __KVM_X86_VMX_POSTED_INTR_H #define __KVM_X86_VMX_POSTED_INTR_H - -#define POSTED_INTR_ON 0 -#define POSTED_INTR_SN 1 - -#define PID_TABLE_ENTRY_VALID 1 - -/* Posted-Interrupt Descriptor */ -struct pi_desc { - u32 pir[8]; /* Posted interrupt requested */ - union { - struct { - /* bit 256 - Outstanding Notification */ - u16 on : 1, - /* bit 257 - Suppress Notification */ - sn : 1, - /* bit 271:258 - Reserved */ - rsvd_1 : 14; - /* bit 279:272 - Notification Vector */ - u8 nv; - /* bit 287:280 - Reserved */ - u8 rsvd_2; - /* bit 319:288 - Notification Destination */ - u32 ndst; - }; - u64 control; - }; - u32 rsvd[6]; -} __aligned(64); - -static inline bool pi_test_and_set_on(struct pi_desc *pi_desc) -{ - return test_and_set_bit(POSTED_INTR_ON, - (unsigned long *)&pi_desc->control); -} - -static inline bool pi_test_and_clear_on(struct pi_desc *pi_desc) -{ - return test_and_clear_bit(POSTED_INTR_ON, - (unsigned long *)&pi_desc->control); -} - -static inline bool pi_test_and_clear_sn(struct pi_desc *pi_desc) -{ - return test_and_clear_bit(POSTED_INTR_SN, - (unsigned long *)&pi_desc->control); -} - -static inline bool pi_test_and_set_pir(int vector, struct pi_desc *pi_desc) -{ - return test_and_set_bit(vector, (unsigned long *)pi_desc->pir); -} - -static inline bool pi_is_pir_empty(struct pi_desc *pi_desc) -{ - return bitmap_empty((unsigned long *)pi_desc->pir, NR_VECTORS); -} - -static inline void pi_set_sn(struct pi_desc *pi_desc) -{ - set_bit(POSTED_INTR_SN, - (unsigned long *)&pi_desc->control); -} - -static inline void pi_set_on(struct pi_desc *pi_desc) -{ - set_bit(POSTED_INTR_ON, - (unsigned long *)&pi_desc->control); -} - -static inline void pi_clear_on(struct pi_desc *pi_desc) -{ - clear_bit(POSTED_INTR_ON, - (unsigned long *)&pi_desc->control); -} - -static inline void pi_clear_sn(struct pi_desc *pi_desc) -{ - clear_bit(POSTED_INTR_SN, - (unsigned long *)&pi_desc->control); -} - -static inline bool pi_test_on(struct pi_desc *pi_desc) -{ - return test_bit(POSTED_INTR_ON, - (unsigned long *)&pi_desc->control); -} - -static inline bool pi_test_sn(struct pi_desc *pi_desc) -{ - return test_bit(POSTED_INTR_SN, - (unsigned long *)&pi_desc->control); -} +#include void vmx_vcpu_pi_load(struct kvm_vcpu *vcpu, int cpu); void vmx_vcpu_pi_put(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index e262bc2ba4e5..d4b6072344ed 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -67,6 +67,7 @@ #include "x86.h" #include "smm.h" #include "vmx_onhyperv.h" +#include "posted_intr.h" MODULE_AUTHOR("Qumranet"); MODULE_LICENSE("GPL"); diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index e3b0985bb74a..5ca8ab15b4f8 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -7,10 +7,10 @@ #include #include #include +#include #include "capabilities.h" #include "../kvm_cache_regs.h" -#include "posted_intr.h" #include "vmcs.h" #include "vmx_ops.h" #include "../cpuid.h" From patchwork Fri Jan 26 23:42:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533598 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0F0758AD6; Fri, 26 Jan 2024 23:37:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312245; cv=none; b=XFvZQCSTXI0vaPSvBzbsGpZjSWghmBVCCdxvSeQ5J6um3oHc1arV77OXyHSURj1ErTHg3dhAbF0Elr/N+m7NEKkgUdL8dPKPdWxX+lxOz/EY7lg4RwJMTc8GbfhDuPcNwWcP58EdPZLy6B2pvQ2eB5j4eftZ+QA3n3o03a7yWO0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312245; c=relaxed/simple; bh=0GX8HhoSt+Bowm5iZvkwyHIzLD8tLG+XNrU90AlqK/Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cIU3Zw3CDMIqCtNAT88WQ1Rg63mwfQ2AvyeT3ZLMCcjelurM/dv2QaabmQxlvezeeVIY9UbgDRDhwfElzDasnxYvniFZcU3HZVxkDIC6dAvDigBkOfM3qC56M2STPEGhFkoEhNDuvKkbAQQF5WLd7WaL/D2LL9nQjCz2lOfiQMU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=UVFqwKBB; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="UVFqwKBB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312243; x=1737848243; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0GX8HhoSt+Bowm5iZvkwyHIzLD8tLG+XNrU90AlqK/Q=; b=UVFqwKBB0HHHM9yNjwsGDBn92Q0uNqj5NO+RtPvV59UwG/ppJD9qsHCZ AeCs2B8PaiBdqLQ/7O3/i79EXDqXRPSM4aiH+apvYkeiagetMx5yiK4ly uHXOQI8hT+lwcJ3xRGuYveePVg3YUd01nSTSPJ0Fk6ZK26gsYVFwf601K 4yGmcQJrswKXkTA5KHKxxfjebgmoIgxs3yPhothIsxXCdzPvVb0bguShk EC0oOAENuG4t5BMlnTuCM5XA+RCYgOj2zksaoVX/1BRNYq7C/CWTlwydf KPyF/h3IS7pwSNqJgmAyqY9gM6jBCot3o4Gm26x4hEMOZSvwNMjHNlJoI A==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990643" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990643" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:20 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290711" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290711" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:20 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 02/15] x86/irq: Unionize PID.PIR for 64bit access w/o casting Date: Fri, 26 Jan 2024 15:42:24 -0800 Message-Id: <20240126234237.547278-3-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Make PIR field into u64 such that atomic xchg64 can be used without ugly casting. Suggested-by: Thomas Gleixner Signed-off-by: Jacob Pan --- arch/x86/include/asm/posted_intr.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/posted_intr.h b/arch/x86/include/asm/posted_intr.h index f0324c56f7af..acf237b2882e 100644 --- a/arch/x86/include/asm/posted_intr.h +++ b/arch/x86/include/asm/posted_intr.h @@ -9,7 +9,10 @@ /* Posted-Interrupt Descriptor */ struct pi_desc { - u32 pir[8]; /* Posted interrupt requested */ + union { + u32 pir[8]; /* Posted interrupt requested */ + u64 pir64[4]; + }; union { struct { /* bit 256 - Outstanding Notification */ From patchwork Fri Jan 26 23:42:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533599 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 140E95915C; Fri, 26 Jan 2024 23:37:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312245; cv=none; b=Z0NHwntXMf30RBTtZr6Pa6g1WQUazRIGJK4g+gWohPkCvGOz/Kwn8JWhL4BOwg10sJ2a+RV35NmOFlukLmD21px+SUZ8dYg4ofht81qngAKrbdaTAxzTsrJbncGd7Yl2jPOK3aL3SFwNbSkYh3YZvgzOpER16AMFSkXw606uLno= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312245; c=relaxed/simple; bh=/Fp/FdRCU1EQNKKT7uYQEIyHFwY63ZnDL0vny89H1Xg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VCCXGY4XdQ6iYFhyWeXy4/OGW5ni34LixwDSaJC0OlUGQmq0ejEWK985SNOA3aUkg/+32YkgVk/x2aoJf9L2tBvKQDoodgsyHgtbWL1zks0s/8ShQokFhuUEwfESTsb+Gj8cJ1YGfFz0dRrnnOBG0UhAhuzAyUekPs7laqudVeA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=f6vqIDvl; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="f6vqIDvl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312244; x=1737848244; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/Fp/FdRCU1EQNKKT7uYQEIyHFwY63ZnDL0vny89H1Xg=; b=f6vqIDvl2sr1zLH9K5cYsB1cIBYqZajZXAv3W08rC/KjVF0kVaKZaGDL huYWEgCT6eVOrP2fEi6sqXhDsdR9f9tABDK6vzA8N1Z28IZQkNDwNU/+H cHcG8BfftYZ64lWCbpupNiC9l3GlB52Zz3n/L/JqMFYO7bkfVcvrx+1Oj A7p0xLcu+mVyFFOU5epSChTk4Df0cusWFzJlGHC5wEldBEihiiPxCGsvJ CmZlFuAUmLOWjW6EQQhfx6Lh0m/Y27aS+ixUetizLJ86asLYfzynwQSPv 7AbcgeWH9UL+pPTeHNK0e/7MTVT75jSo/lW1qrPoq67v1kxKkOwVYG3WT g==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990656" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990656" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290714" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290714" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:20 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 03/15] x86/irq: Use bitfields exclusively in posted interrupt descriptor Date: Fri, 26 Jan 2024 15:42:25 -0800 Message-Id: <20240126234237.547278-4-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Thomas Gleixner Mixture of bitfields and types is weird and really not intuitive, remove types and use bitfields exclusively. Signed-off-by: Thomas Gleixner Signed-off-by: Jacob Pan --- arch/x86/include/asm/posted_intr.h | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/posted_intr.h b/arch/x86/include/asm/posted_intr.h index acf237b2882e..896b3462f3dd 100644 --- a/arch/x86/include/asm/posted_intr.h +++ b/arch/x86/include/asm/posted_intr.h @@ -16,17 +16,17 @@ struct pi_desc { union { struct { /* bit 256 - Outstanding Notification */ - u16 on : 1, + u64 on : 1, /* bit 257 - Suppress Notification */ - sn : 1, + sn : 1, /* bit 271:258 - Reserved */ - rsvd_1 : 14; + : 14, /* bit 279:272 - Notification Vector */ - u8 nv; + nv : 8, /* bit 287:280 - Reserved */ - u8 rsvd_2; + : 8, /* bit 319:288 - Notification Destination */ - u32 ndst; + ndst : 32; }; u64 control; }; From patchwork Fri Jan 26 23:42:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533603 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B95559144; Fri, 26 Jan 2024 23:37:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312247; cv=none; b=shsk1Fbyk+tY6WCczf2ZEf+6298rSQ2zTLd4gQETGUdjM6N/PZcW8P4FPUB/OFoVEGIOuHZnW+0aTZgl2IC1zKNN/yb6udNJ+1KTLpPVRkiTNsdgy9szjIRHXBcQJk57+Qv0Ir2/CYe24UlYlYfiBF+75MT1x5jya6CNMx38QKU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312247; c=relaxed/simple; bh=TiUyHPWp/CruDSgiWpJ9abORXg7ueQkxnjZwsyBxeWU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=q1K7qRMmBUcpTcpdYFJ/BO4rRHROcD080leQaPrhGG7aV0HShK8aFlyctAX3b5YdisHNs8lo/v+fya7iDhf8+fkceljbCV/TJxPY+nBcS1S1Tbldyz+Ym4CDdFf6e/ecv6Xc1SZM19Gg3lape9bss1N4E4usmJczvmps8av92E8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DE9ZUdGM; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DE9ZUdGM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312245; x=1737848245; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TiUyHPWp/CruDSgiWpJ9abORXg7ueQkxnjZwsyBxeWU=; b=DE9ZUdGMQ/IBVIsZi7to9I6IXbOkkwHBIiX5ltT9Pfll+r/wztVO8egh 1fjb/6+5seJvjReJNoxAzfsqHJIhClGlP5e5NJYIP5N4eqjHokCj/p4YQ 5zWfTwhmS+7XY8RZLu6CXISZMWr0IZ1DdvFya7oFHXMnreTHD6//Y2dG2 tyxHMn2tJ8utBGXkCNMsp/SaW55mr/PY0RTWyMP6uNBW5BSt69fKQSsPZ 67aZUKl0uxxpkXMXk4x40nNbVwoP3CsSGLLoMO3KGabMvto/oSjxKoDbp 7eAEov6VHWpcxDErkVSI8Q5g9AdcR4II4NLi1AtVHhY5ffa8yj8Ca5GRq A==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990661" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990661" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290722" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290722" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:20 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 04/15] x86/irq: Add a Kconfig option for posted MSI Date: Fri, 26 Jan 2024 15:42:26 -0800 Message-Id: <20240126234237.547278-5-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This option will be used to support delivering MSIs as posted interrupts. Interrupt remapping is required. Signed-off-by: Jacob Pan --- arch/x86/Kconfig | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5edec175b9bf..79f04ee2b91c 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -463,6 +463,17 @@ config X86_X2APIC If you don't know what to do here, say N. +config X86_POSTED_MSI + bool "Enable MSI and MSI-x delivery by posted interrupts" + depends on X86_X2APIC && X86_64 && IRQ_REMAP + help + This enables MSIs that are under interrupt remapping to be delivered as + posted interrupts to the host kernel. Interrupt throughput can + potentially be improved by coalescing CPU notifications during high + frequency bursts. + + If you don't know what to do here, say N. + config X86_MPPARSE bool "Enable MPS table" if ACPI default y From patchwork Fri Jan 26 23:42:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533601 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC2065A7A2; Fri, 26 Jan 2024 23:37:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312247; cv=none; b=fqhZRQ9rChwt3FzSpnZFchBVZNgrx6O8LrMlnW/L/3PbxqnggyT7RlNgQQARX7k+gIF97q0oUOfTWhLClFmH0EsXvoFM2vHb7aUsz2NdEwruKCDMcvOeybQWSG4plM+3xqZIdoQo9ZHfkxt+/Jjhf3wtReg57i9Cc0cOPfwDvJ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312247; c=relaxed/simple; bh=QJV30lNeix5myDOaVVsdK3RMyy1VcDvQ9cOpKGwrPwg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SbNrgXSESzomJSsD+2oeIZzRarAH+od2CCaXEiHhSN3YbtcQkZ3YvC3K5EC9G7mdpMASBaXX2PlBabghAZuhnyrWNMsFAksX9nCWIabEF5/sdcrawHqgPk9nbw3v6SZVzKqFrU8pnOr1PlF2cs7UKhm603z6yC7JZsUfn7nZO8w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=R0KGBdX9; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="R0KGBdX9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312246; x=1737848246; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QJV30lNeix5myDOaVVsdK3RMyy1VcDvQ9cOpKGwrPwg=; b=R0KGBdX9thnwx1KkeuOUNNwybbsWww76snbTLVeujKIj/aVeBIcRF9yz YLtLNBvpyhifmvCUnBZhECmsT0jUajSx6CakdR/Yhy9YblOB0E31g96RM +/4+ivpABHmFNMfRySda3wFJi+2gCw42xj7baAeOaBRTNGrFfTvEPLXXQ dm1UWP2UasNS7HHdBwZsk+c+BBY9yHMKRP6xT1WVZu2WvtRRzfrvsAQzK 6SLKSDunpCZ1l1msPj7YhtGvJ+XMG3MJAVsqdjJe5C/OELalHTvhRCSKb RDp8n5FcH3uSIk19WyQLqdUus0cIU1dPhTE4lvP8VO03WTHZw+niHNIRT Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990682" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990682" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290725" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290725" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:21 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 05/15] x86/irq: Reserve a per CPU IDT vector for posted MSIs Date: Fri, 26 Jan 2024 15:42:27 -0800 Message-Id: <20240126234237.547278-6-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When posted MSI is enabled, all device MSIs are multiplexed into a single notification vector. MSI handlers will be de-multiplexed at run-time by system software without IDT delivery. Signed-off-by: Jacob Pan --- arch/x86/include/asm/irq_vectors.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_vectors.h index 3a19904c2db6..08329bef5b1d 100644 --- a/arch/x86/include/asm/irq_vectors.h +++ b/arch/x86/include/asm/irq_vectors.h @@ -99,9 +99,16 @@ #define LOCAL_TIMER_VECTOR 0xec +/* + * Posted interrupt notification vector for all device MSIs delivered to + * the host kernel. + */ +#define POSTED_MSI_NOTIFICATION_VECTOR 0xeb #define NR_VECTORS 256 -#ifdef CONFIG_X86_LOCAL_APIC +#ifdef X86_POSTED_MSI +#define FIRST_SYSTEM_VECTOR POSTED_MSI_NOTIFICATION_VECTOR +#elif defined(CONFIG_X86_LOCAL_APIC) #define FIRST_SYSTEM_VECTOR LOCAL_TIMER_VECTOR #else #define FIRST_SYSTEM_VECTOR NR_VECTORS From patchwork Fri Jan 26 23:42:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533602 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3AA65A7AC; Fri, 26 Jan 2024 23:37:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312247; cv=none; b=XpPeUT3i5BmvrkxIsTn+zthLVe5Nui6FA6HlG2NMZruAwAKZ6dz89P+236m4Vet8/Tn2udhNtcOD5aQn9Z9X+lN+VRyiXtMzgolLWb6ogBpnaegSGWPQeUe3F747oz+g0IFDuIWjUnnZ/DsNLsYE922vmFmw+zp5L0194UrifjY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312247; c=relaxed/simple; bh=+jcfYcRgLrk4OhxkOXBPXFSpbQQeXvWOwimgSWB0zss=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TdFEiq7MrOSWCHW8d98PG8/9xgrpm/nnr1eJp8jH9qNBO/G/4AMiMWBXAjF9S+LbmnVTpU1PF3vhcISh3AeqEwUdJI97C+2KyNauCzXBKBGrpiLkn2hi7VrmOL0muCiwVDOBaV07Xr0DHu5nBOC+CEttqTgJRN63q3cD+RapmQQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nR8z5EQ+; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nR8z5EQ+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312246; x=1737848246; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+jcfYcRgLrk4OhxkOXBPXFSpbQQeXvWOwimgSWB0zss=; b=nR8z5EQ+GTHFx2l49XovDXyuCOKPzblDhxWmnJF8NCI+2U0tnSx1yH7x d/J9dsfp32mTczBdrqEA6sZ/h7z1pq9nSxIAOMtJYq8/qfWo7RmCcb4P8 tKW+XSeC6f8GBxgz+0WzkwvR3J0+h29RhcUoJ+Q3UNoleLZgH81x4u59e seGva7bVVMIpmwXPXdP7EsKfgD7tTJpsOfeCkV99KYA1UDv1wk46QjDTL aBJT32E3p8akCKyg85y+ZGefBTtB6Ng+zC0ohmBcIpvVhC0c/TyoJTg7Z APvA675h4DrPyTwazejFOk7YgMxPW4jSmoYRl0ZF7qgFQsfzMgXLG2NTd w==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990697" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990697" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290728" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290728" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:21 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 06/15] x86/irq: Set up per host CPU posted interrupt descriptors Date: Fri, 26 Jan 2024 15:42:28 -0800 Message-Id: <20240126234237.547278-7-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Thomas Gleixner To support posted MSIs, create a posted interrupt descriptor (PID) for each host CPU. Later on, when setting up IRQ CPU affinity, IOMMU's interrupt remapping table entry (IRTE) will point to the physical address of the matching CPU's PID. Each PID is initialized with the owner CPU's physical APICID as the destination. Signed-off-by: Thomas Gleixner Signed-off-by: Jacob Pan --- arch/x86/include/asm/hardirq.h | 3 +++ arch/x86/include/asm/posted_intr.h | 7 +++++++ arch/x86/kernel/cpu/common.c | 3 +++ arch/x86/kernel/irq.c | 16 ++++++++++++++++ 4 files changed, 29 insertions(+) diff --git a/arch/x86/include/asm/hardirq.h b/arch/x86/include/asm/hardirq.h index 66837b8c67f1..72c6a084dba3 100644 --- a/arch/x86/include/asm/hardirq.h +++ b/arch/x86/include/asm/hardirq.h @@ -48,6 +48,9 @@ typedef struct { DECLARE_PER_CPU_SHARED_ALIGNED(irq_cpustat_t, irq_stat); +#ifdef CONFIG_X86_POSTED_MSI +DECLARE_PER_CPU_ALIGNED(struct pi_desc, posted_interrupt_desc); +#endif #define __ARCH_IRQ_STAT #define inc_irq_stat(member) this_cpu_inc(irq_stat.member) diff --git a/arch/x86/include/asm/posted_intr.h b/arch/x86/include/asm/posted_intr.h index 896b3462f3dd..a36cc971ea13 100644 --- a/arch/x86/include/asm/posted_intr.h +++ b/arch/x86/include/asm/posted_intr.h @@ -88,4 +88,11 @@ static inline bool pi_test_sn(struct pi_desc *pi_desc) return test_bit(POSTED_INTR_SN, (unsigned long *)&pi_desc->control); } +#ifdef CONFIG_X86_POSTED_MSI +extern void intel_posted_msi_init(void); + +#else +static inline void intel_posted_msi_init(void) {}; + +#endif /* X86_POSTED_MSI */ #endif /* _X86_POSTED_INTR_H */ diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 0b97bcde70c6..9b6248e7c073 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -67,6 +67,7 @@ #include #include #include +#include #include "cpu.h" @@ -2253,6 +2254,8 @@ void cpu_init(void) barrier(); x2apic_setup(); + + intel_posted_msi_init(); } mmgrab(&init_mm); diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index 11761c124545..f6546f83d616 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -22,6 +22,8 @@ #include #include #include +#include +#include #define CREATE_TRACE_POINTS #include @@ -334,6 +336,20 @@ DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_kvm_posted_intr_nested_ipi) } #endif +#ifdef CONFIG_X86_POSTED_MSI + +/* Posted Interrupt Descriptors for coalesced MSIs to be posted */ +DEFINE_PER_CPU_ALIGNED(struct pi_desc, posted_interrupt_desc); + +void intel_posted_msi_init(void) +{ + struct pi_desc *pid = this_cpu_ptr(&posted_interrupt_desc); + + pid->nv = POSTED_MSI_NOTIFICATION_VECTOR; + pid->ndst = this_cpu_read(x86_cpu_to_apicid); +} +} +#endif /* X86_POSTED_MSI */ #ifdef CONFIG_HOTPLUG_CPU /* A cpu has been removed from cpu_online_mask. Reset irq affinities. */ From patchwork Fri Jan 26 23:42:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533605 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A90365BAE7; Fri, 26 Jan 2024 23:37:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312249; cv=none; b=jFaVeGS+m5CDLgnuAYB3uJvAQedi6r61T+5tTNlqsQJ7f+WK8O43pM9xcZVSpHwdC0cHjYO8OohG1uHN5gtLfNMJvf1Q14i2vYGr3fAPTXpUVj0L5wZWTkJF9t4uAHJZKuVI/wV5i5aboHAJlUfs9B2m5/zDlQYqL3IdP62yc2o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312249; c=relaxed/simple; bh=MqzYCPV34vdfoVMKAgtS0juqP6fLnnDqzn4F90K69Xk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=L9LIFjH97JfLrZeK47JqeCznRdwPI7D8SicVMdalyeL1P/RiOsfzpfMZKZY/uQrMa2Dm+bXne53fNPMkcxm30TBaE7hcQVWHUv5bGy7vBMtI979ZF1Gzl1yR6PdnCS2gd3b7xcX5YXUd7VIzyvpuVAQBXn2hhNAALPHwWS99twI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=J6tgjPIF; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="J6tgjPIF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312248; x=1737848248; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MqzYCPV34vdfoVMKAgtS0juqP6fLnnDqzn4F90K69Xk=; b=J6tgjPIFmZEEAu3lcIFffeQgtu8D++Cv4u8kZ853dPNWmQ6qxXQLXw+6 vaTQjmtiWbw1/k8HsuDPk7ZeJKgWjslKaN8JWY7qQ2NzogM5XmBvgybx3 BPXUuTYVd+60XLenrTBej+2Bb3DOy1jDBbbql2kW7q+RK4E5U1oeOOyhD hLCMg2HMjZEqP8oAl5tmFSuSDd4Q3Mlscy74P/nx05H8pCtEypadbKxGY RijMjjo22AdPN9RL9D1avMKJ2U9CeWyaqljl0njH9CR1GSP9owcJGpSBx /GCqIxlUIHjs5tR+wlDA/sa+EwsXcr77C9ikjTRQCEYNwO3Zt8IdYtUQM w==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990714" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990714" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290731" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290731" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:21 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 07/15] x86/irq: Add accessors for posted interrupt descriptors Date: Fri, 26 Jan 2024 15:42:29 -0800 Message-Id: <20240126234237.547278-8-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Posted interrupts are controlled by and pending interrupts are marked in the posted interrupt descriptor. The upcoming support for host side posted interrupts requires accessors to check for pending vectors. This patch adds a helper function to check individual vector status. Signed-off-by: Jacob Pan --- arch/x86/include/asm/posted_intr.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/include/asm/posted_intr.h b/arch/x86/include/asm/posted_intr.h index a36cc971ea13..eb939f630b02 100644 --- a/arch/x86/include/asm/posted_intr.h +++ b/arch/x86/include/asm/posted_intr.h @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ #ifndef _X86_POSTED_INTR_H #define _X86_POSTED_INTR_H +#include #define POSTED_INTR_ON 0 #define POSTED_INTR_SN 1 @@ -89,9 +90,26 @@ static inline bool pi_test_sn(struct pi_desc *pi_desc) } #ifdef CONFIG_X86_POSTED_MSI +/* + * Not all external vectors are subject to interrupt remapping, e.g. IOMMU's + * own interrupts. Here we do not distinguish them since those vector bits in + * PIR will always be zero. + */ +static inline bool pi_pending_this_cpu(unsigned int vector) +{ + struct pi_desc *pid = this_cpu_ptr(&posted_interrupt_desc); + + if (WARN_ON_ONCE(vector > NR_VECTORS || vector < FIRST_EXTERNAL_VECTOR)) + return false; + + return test_bit(vector, (unsigned long *)pid->pir); +} + extern void intel_posted_msi_init(void); #else +static inline bool pi_pending_this_cpu(unsigned int vector) { return false; } + static inline void intel_posted_msi_init(void) {}; #endif /* X86_POSTED_MSI */ From patchwork Fri Jan 26 23:42:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533604 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A8615BADD; Fri, 26 Jan 2024 23:37:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312249; cv=none; b=mkEYgeOxhCD+v5tvuff4TpXJrlYZo8JLQPuAeR7LVcTQs0zae2G0fTIkRdZKDOiDp8SN9ls/uMBsJsTY7vLIEKe12I1CyaaTPjDrIyYzYIC57Q4Hg+JBadOE4fT2+KJS/xB10sQ452EzCJXLWqFUI9iXH0BOPHBwdxKHrfwawVw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312249; c=relaxed/simple; bh=KVXuQfj6u+U1cGUQ3TC+vIVsCaW1LZRHuP8dLBZSJDg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=odbM7rJ+qmhE2hLkULgEbVpHZyJc5zP+elEgVURpkCwvLJNVJFLUxGFf+aTLQFCpwxd84EMOn/80NlTnfkQ832c59qxDYiLYl/Y4UVqOhb25y6pxSpCU8tacY60KRuU5SG7IPYDfjKSAvvinNiMPzooisT1gD2C1KpNAjy2WCD0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ZfxnOCn9; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ZfxnOCn9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312248; x=1737848248; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KVXuQfj6u+U1cGUQ3TC+vIVsCaW1LZRHuP8dLBZSJDg=; b=ZfxnOCn9I4TNcSMm+NXVzWD7vVL1fw+gKUz1p5j4Kl68v/HRzgNSdK5x 08/aSW894nMo6fWzfSxZl9tA3ioN2cUfn8TFEaQGpKfQuO6BtjeNv1P5w uiUGEQzIIPjRwwS0mHwHOJTPpLmlzjc+ZjAs2D4G1fbcwCb92qTCY/7aY nN3t0CRVNhL2/V9U+//IQct/P8t8HxCnRkfjJ7wTlxUAL/DGRYiO/Rpy6 rRzrn8yZHbzW+perk5C4xVt8vv3a3oZNsTyB1sUttPYst1THIxZRdwJSO JXbNZF4lksDiuOfKMcQDregCGhjYc6vvNAUA6V5lX/kPdWlc5bYpOYqXh g==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990715" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990715" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290734" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290734" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:21 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 08/15] x86/irq: Factor out calling ISR from common_interrupt Date: Fri, 26 Jan 2024 15:42:30 -0800 Message-Id: <20240126234237.547278-9-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Prepare for calling external IRQ handlers directly from the posted MSI demultiplexing loop. Extract the common code with common interrupt to avoid code duplication. Signed-off-by: Jacob Pan --- arch/x86/kernel/irq.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index f6546f83d616..1a1762baf85f 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -242,18 +242,10 @@ static __always_inline void handle_irq(struct irq_desc *desc, __handle_irq(desc, regs); } -/* - * common_interrupt() handles all normal device IRQ's (the special SMP - * cross-CPU interrupts have their own entry points). - */ -DEFINE_IDTENTRY_IRQ(common_interrupt) +static __always_inline void call_irq_handler(int vector, struct pt_regs *regs) { - struct pt_regs *old_regs = set_irq_regs(regs); struct irq_desc *desc; - /* entry code tells RCU that we're not quiescent. Check it. */ - RCU_LOCKDEP_WARN(!rcu_is_watching(), "IRQ failed to wake up RCU"); - desc = __this_cpu_read(vector_irq[vector]); if (likely(!IS_ERR_OR_NULL(desc))) { handle_irq(desc, regs); @@ -268,7 +260,20 @@ DEFINE_IDTENTRY_IRQ(common_interrupt) __this_cpu_write(vector_irq[vector], VECTOR_UNUSED); } } +} + +/* + * common_interrupt() handles all normal device IRQ's (the special SMP + * cross-CPU interrupts have their own entry points). + */ +DEFINE_IDTENTRY_IRQ(common_interrupt) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + /* entry code tells RCU that we're not quiescent. Check it. */ + RCU_LOCKDEP_WARN(!rcu_is_watching(), "IRQ failed to wake up RCU"); + call_irq_handler(vector, regs); set_irq_regs(old_regs); } From patchwork Fri Jan 26 23:42:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533606 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC7895BAF0; Fri, 26 Jan 2024 23:37:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312250; cv=none; b=n+ky2Vm3xPMwXddbTAh1RktdzhhSZj3TtK+JokHC7qH9l5VkAlREpd3RjYKWPTHnda+jn5dwaqqMVGGxnuXcAOPL6WBJny8D4ohu3NEU8JvfcPEaklH25u6/U5SR+BiM/JC/OZPVIIb96ZNyuiNvmOeX7MtP0cTTc1JoqqWtXTY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312250; c=relaxed/simple; bh=h4lxSMKyQxlseu4S2RwYjNDhOv7kmbRScOIMsH8Mhyk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dOYfKmAEiiRKlJNpd3qwDlbo39a3Yq0uBFiB45BTZEJgochiv/hqprIrP3T4a8J5xQmh/XSa9PAxvi9we3P8oMs6WBo86tn2nufJUYHnRRuQB4x6EBErwAxC4y5qSwTtmwA0vKIEhd4A7UtOMlbY3fp08JafWK+DpwlPNKRU0wI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=SQZILzcH; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="SQZILzcH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312248; x=1737848248; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=h4lxSMKyQxlseu4S2RwYjNDhOv7kmbRScOIMsH8Mhyk=; b=SQZILzcHIc8BBqlyYbjGWRfEsRuWoimeBxIZvOtLAYAwHjAH+X/uckWn kCPQHW5h2Cfhifpx89GQ2y7jpIJ6vnNiBVnqeDmBjN1YvA89jqfulf0B7 MmLKuDCpU/zWWNOZZdX1Q5l85I3QDfu89qt0XStEQh8dFFea/7e+alnrv EboBZeMZrt+eJsQXVkA/DbYw1HCYrobaC3IXHEJwEkBn6rs8eWC6fKJ76 JeW7Eg5BjJy+CwTBJn71/XiGWEXfmjC6mSpu/ebXyfxgx622FTkiA94w9 3skfK5WmBLmq6t7xmBSdSOQyoaLDdmz+qwOWP6dJvJ5X6e+yTX5BYPrqp Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990739" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990739" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290739" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290739" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:21 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 09/15] x86/irq: Install posted MSI notification handler Date: Fri, 26 Jan 2024 15:42:31 -0800 Message-Id: <20240126234237.547278-10-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 All MSI vectors are multiplexed into a single notification vector when posted MSI is enabled. It is the responsibility of the notification vector handler to demultiplex MSI vectors. In this handler, for each pending bit, MSI vector handlers are dispatched without IDT delivery. For example, the interrupt flow will change as follows: (3 MSIs of different vectors arrive in a a high frequency burst) BEFORE: interrupt(MSI) irq_enter() handler() /* EOI */ irq_exit() process_softirq() interrupt(MSI) irq_enter() handler() /* EOI */ irq_exit() process_softirq() interrupt(MSI) irq_enter() handler() /* EOI */ irq_exit() process_softirq() AFTER: interrupt /* Posted MSI notification vector */ irq_enter() atomic_xchg(PIR) handler() handler() handler() pi_clear_on() apic_eoi() irq_exit() process_softirq() Except for the leading MSI, CPU notifications are skipped/coalesced. For MSIs arrive at a low frequency, the demultiplexing loop does not wait for more interrupts to coalesce. Therefore, there's no additional latency other than the processing time. Signed-off-by: Jacob Pan --- arch/x86/include/asm/hardirq.h | 3 + arch/x86/include/asm/idtentry.h | 3 + arch/x86/kernel/idt.c | 3 + arch/x86/kernel/irq.c | 112 ++++++++++++++++++++++++++++++++ 4 files changed, 121 insertions(+) diff --git a/arch/x86/include/asm/hardirq.h b/arch/x86/include/asm/hardirq.h index 72c6a084dba3..6c8daa7518eb 100644 --- a/arch/x86/include/asm/hardirq.h +++ b/arch/x86/include/asm/hardirq.h @@ -44,6 +44,9 @@ typedef struct { unsigned int irq_hv_reenlightenment_count; unsigned int hyperv_stimer0_count; #endif +#ifdef CONFIG_X86_POSTED_MSI + unsigned int posted_msi_notification_count; +#endif } ____cacheline_aligned irq_cpustat_t; DECLARE_PER_CPU_SHARED_ALIGNED(irq_cpustat_t, irq_stat); diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 13639e57e1f8..83d4de75df34 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -648,6 +648,9 @@ DECLARE_IDTENTRY_SYSVEC(ERROR_APIC_VECTOR, sysvec_error_interrupt); DECLARE_IDTENTRY_SYSVEC(SPURIOUS_APIC_VECTOR, sysvec_spurious_apic_interrupt); DECLARE_IDTENTRY_SYSVEC(LOCAL_TIMER_VECTOR, sysvec_apic_timer_interrupt); DECLARE_IDTENTRY_SYSVEC(X86_PLATFORM_IPI_VECTOR, sysvec_x86_platform_ipi); +# ifdef CONFIG_X86_POSTED_MSI +DECLARE_IDTENTRY_SYSVEC(POSTED_MSI_NOTIFICATION_VECTOR, sysvec_posted_msi_notification); +# endif #endif #ifdef CONFIG_SMP diff --git a/arch/x86/kernel/idt.c b/arch/x86/kernel/idt.c index 660b601f1d6c..061a927367ec 100644 --- a/arch/x86/kernel/idt.c +++ b/arch/x86/kernel/idt.c @@ -163,6 +163,9 @@ static const __initconst struct idt_data apic_idts[] = { # endif INTG(SPURIOUS_APIC_VECTOR, asm_sysvec_spurious_apic_interrupt), INTG(ERROR_APIC_VECTOR, asm_sysvec_error_interrupt), +# ifdef CONFIG_X86_POSTED_MSI + INTG(POSTED_MSI_NOTIFICATION_VECTOR, asm_sysvec_posted_msi_notification), +# endif #endif }; diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index 1a1762baf85f..54ddf148f1ed 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -183,6 +183,13 @@ int arch_show_interrupts(struct seq_file *p, int prec) seq_printf(p, "%10u ", irq_stats(j)->kvm_posted_intr_wakeup_ipis); seq_puts(p, " Posted-interrupt wakeup event\n"); +#endif +#ifdef CONFIG_X86_POSTED_MSI + seq_printf(p, "%*s: ", prec, "PMN"); + for_each_online_cpu(j) + seq_printf(p, "%10u ", + irq_stats(j)->posted_msi_notification_count); + seq_puts(p, " Posted MSI notification event\n"); #endif return 0; } @@ -353,6 +360,111 @@ void intel_posted_msi_init(void) pid->nv = POSTED_MSI_NOTIFICATION_VECTOR; pid->ndst = this_cpu_read(x86_cpu_to_apicid); } + +/* + * De-multiplexing posted interrupts is on the performance path, the code + * below is written to optimize the cache performance based on the following + * considerations: + * 1.Posted interrupt descriptor (PID) fits in a cache line that is frequently + * accessed by both CPU and IOMMU. + * 2.During posted MSI processing, the CPU needs to do 64-bit read and xchg + * for checking and clearing posted interrupt request (PIR), a 256 bit field + * within the PID. + * 3.On the other side, the IOMMU does atomic swaps of the entire PID cache + * line when posting interrupts and setting control bits. + * 4.The CPU can access the cache line a magnitude faster than the IOMMU. + * 5.Each time the IOMMU does interrupt posting to the PIR will evict the PID + * cache line. The cache line states after each operation are as follows: + * CPU IOMMU PID Cache line state + * --------------------------------------------------------------- + *...read64 exclusive + *...lock xchg64 modified + *... post/atomic swap invalid + *...------------------------------------------------------------- + * + * To reduce L1 data cache miss, it is important to avoid contention with + * IOMMU's interrupt posting/atomic swap. Therefore, a copy of PIR is used + * to dispatch interrupt handlers. + * + * In addition, the code is trying to keep the cache line state consistent + * as much as possible. e.g. when making a copy and clearing the PIR + * (assuming non-zero PIR bits are present in the entire PIR), it does: + * read, read, read, read, xchg, xchg, xchg, xchg + * instead of: + * read, xchg, read, xchg, read, xchg, read, xchg + */ +static __always_inline inline bool handle_pending_pir(u64 *pir, struct pt_regs *regs) +{ + int i, vec = FIRST_EXTERNAL_VECTOR; + unsigned long pir_copy[4]; + bool handled = false; + + for (i = 0; i < 4; i++) + pir_copy[i] = pir[i]; + + for (i = 0; i < 4; i++) { + if (!pir_copy[i]) + continue; + + pir_copy[i] = arch_xchg(pir, 0); + handled = true; + } + + if (handled) { + for_each_set_bit_from(vec, pir_copy, FIRST_SYSTEM_VECTOR) + call_irq_handler(vec, regs); + } + + return handled; +} + +/* + * Performance data shows that 3 is good enough to harvest 90+% of the benefit + * on high IRQ rate workload. + */ +#define MAX_POSTED_MSI_COALESCING_LOOP 3 + +/* + * For MSIs that are delivered as posted interrupts, the CPU notifications + * can be coalesced if the MSIs arrive in high frequency bursts. + */ +DEFINE_IDTENTRY_SYSVEC(sysvec_posted_msi_notification) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + struct pi_desc *pid; + int i = 0; + + pid = this_cpu_ptr(&posted_interrupt_desc); + + inc_irq_stat(posted_msi_notification_count); + irq_enter(); + + /* + * Max coalescing count includes the extra round of handle_pending_pir + * after clearing the outstanding notification bit. Hence, at most + * MAX_POSTED_MSI_COALESCING_LOOP - 1 loops are executed here. + */ + while (++i < MAX_POSTED_MSI_COALESCING_LOOP) { + if (!handle_pending_pir(pid->pir64, regs)) + break; + } + + /* + * Clear outstanding notification bit to allow new IRQ notifications, + * do this last to maximize the window of interrupt coalescing. + */ + pi_clear_on(pid); + + /* + * There could be a race of PI notification and the clearing of ON bit, + * process PIR bits one last time such that handling the new interrupts + * are not delayed until the next IRQ. + */ + handle_pending_pir(pid->pir64, regs); + + apic_eoi(); + irq_exit(); + set_irq_regs(old_regs); } #endif /* X86_POSTED_MSI */ From patchwork Fri Jan 26 23:42:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533608 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A07F260267; Fri, 26 Jan 2024 23:37:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312251; cv=none; b=C1Z04LhT0KNOJCILdgUk+ZmF7pQwH5c97oX7SeBOfGZ/CVbrrFRUJlRfFvmMV6hy2nkpyZ/KFzZZhNUGmsgGqX15CqewV0MUV+MGXmIiXAFz2ufqzSc9abVy73zupz19Qb+TpRMnDLvlx4c5xGXFslbZ44tZsLB8nbCqENGKwUw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312251; c=relaxed/simple; bh=6ZuNeLIEAeBdSx49wXWrTgYdLXjN/iAwcc2wy4sRSo0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Mw3EGsON8JSXs6LuhJi92CFKse9jInHBYHULy+CsJsLZh0D+lxlKUAPsY//quT334vbI7fFzJGXRpm3ko/QKN3zYhQnjhzSG4e4qvUorhnxiGyury3N4hH9PzAqDgxn7o3D6TJPOm0fjuPi24QaB5GwHpbAbgEH4U8EfzBamiL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=J/DZw0zL; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="J/DZw0zL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312250; x=1737848250; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6ZuNeLIEAeBdSx49wXWrTgYdLXjN/iAwcc2wy4sRSo0=; b=J/DZw0zLgz6bSzhcUX0kTWTJUbGmhwRtA/VZegDH0GrY5oKqaOjPET7U 5mNlLRNwsXq5AUS4+Rv6GkN7FLtA6i133znDY5mzAaaSmE5rQ7+4NvwGj 626TkMc6S8xsHzT723QFAGmAcKD89nVx4CKdudg43mDUqZvj4qrBdVmXb a2BVWW358vuyzy09e0eIPL5YYXOzK19BzoKvgjuYAeBnMdi2G8+91nHQq B1WotXMkwj2lM6qRZzx8u/bsQ1mxO/sPfYlSEwcMx/ClXmvOKSt2kxzzD lOhlV6dRglDWXAEvqE+UzC/Tx83D29+PwECRZO5IT+B0hRaEXJXxnnUWv w==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990744" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990744" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290742" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290742" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:22 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 10/15] x86/irq: Factor out common code for checking pending interrupts Date: Fri, 26 Jan 2024 15:42:32 -0800 Message-Id: <20240126234237.547278-11-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Use a common function for checking pending interrupt vector in APIC IRR instead of duplicated open coding them. Additional checks for posted MSI vectors can then be contained in this function. Signed-off-by: Jacob Pan --- arch/x86/include/asm/apic.h | 11 +++++++++++ arch/x86/kernel/apic/vector.c | 5 ++--- arch/x86/kernel/irq.c | 5 ++--- 3 files changed, 15 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 9d159b771dc8..e9d8e554765c 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -492,6 +492,17 @@ static inline bool lapic_vector_set_in_irr(unsigned int vector) return !!(irr & (1U << (vector % 32))); } +static inline bool is_vector_pending(unsigned int vector) +{ + unsigned int irr; + + irr = apic_read(APIC_IRR + (vector / 32 * 0x10)); + if (irr & (1 << (vector % 32))) + return true; + + return false; +} + /* * Warm reset vector position: */ diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c index 185738c72766..9eec52925fa3 100644 --- a/arch/x86/kernel/apic/vector.c +++ b/arch/x86/kernel/apic/vector.c @@ -965,7 +965,7 @@ static void __vector_cleanup(struct vector_cleanup *cl, bool check_irr) lockdep_assert_held(&vector_lock); hlist_for_each_entry_safe(apicd, tmp, &cl->head, clist) { - unsigned int irr, vector = apicd->prev_vector; + unsigned int vector = apicd->prev_vector; /* * Paranoia: Check if the vector that needs to be cleaned @@ -979,8 +979,7 @@ static void __vector_cleanup(struct vector_cleanup *cl, bool check_irr) * fixup_irqs() was just called to scan IRR for set bits and * forward them to new destination CPUs via IPIs. */ - irr = check_irr ? apic_read(APIC_IRR + (vector / 32 * 0x10)) : 0; - if (irr & (1U << (vector % 32))) { + if (check_irr && is_vector_pending(vector)) { pr_warn_once("Moved interrupt pending in old target APIC %u\n", apicd->irq); rearm = true; continue; diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index 54ddf148f1ed..8e09d40ea928 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -472,7 +472,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_posted_msi_notification) /* A cpu has been removed from cpu_online_mask. Reset irq affinities. */ void fixup_irqs(void) { - unsigned int irr, vector; + unsigned int vector; struct irq_desc *desc; struct irq_data *data; struct irq_chip *chip; @@ -499,8 +499,7 @@ void fixup_irqs(void) if (IS_ERR_OR_NULL(__this_cpu_read(vector_irq[vector]))) continue; - irr = apic_read(APIC_IRR + (vector / 32 * 0x10)); - if (irr & (1 << (vector % 32))) { + if (is_vector_pending(vector)) { desc = __this_cpu_read(vector_irq[vector]); raw_spin_lock(&desc->lock); From patchwork Fri Jan 26 23:42:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533607 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0BC560268; Fri, 26 Jan 2024 23:37:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312251; cv=none; b=MIoMveoadXWzrlXR4IHD861Ucqet4jUn8PDfO4+u2rMdJgY3VdaMz5Dj9O49eB7U0OPGQqRP4KxO7zQBqnmIzUFYnzgVLpcMOGvnhX0qPIX+nRzSzqaPiPj3HqIRWCUbZgcv3DRCBjnuAxijhzftXIXXYzKLIvHGGP2NHo/ART4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312251; c=relaxed/simple; bh=5z7HUBzE7Ul+Bci9YbfIrBXJxLaOXS9r2ggCh19e6Ag=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hWslVEJxlPWavH7xH9pm6qax6ENH5Dhh4EklWS+nVzJos465Eug+aGuCotEXR5icnpisuCRc1laA+m9VuDcOQI1R8+6qjT0KJx5lG+t+HN7W30neztvC/jzMmjRnEKTvvohriF8qYD9ocR0yD9ads5k2VrmbY5ZAFmg0/HoRYng= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iaKSdoT7; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iaKSdoT7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312250; x=1737848250; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5z7HUBzE7Ul+Bci9YbfIrBXJxLaOXS9r2ggCh19e6Ag=; b=iaKSdoT7f1BgjB9CyiKX5BC1FM6Up2YfHvVQNXGifwIO6UMS12E7P7Jr HAiHpFysaoD7/V/q0mcuopQwTu1A5yBQUesUBb4KW6kcFYhZYi/5OAJpv 3SCVJee/97y9xq5fiTlbp1pwtoAlyAoA17yzNCpE6los58/pcDlOP35Mh ljNwB1HLbhBBaVXBfwd07fpffXHlap3hWJaQ2xgZl4btRkrdBGTFYbXsP wknz0fFYRWGEM7cbBWz+y+5jJ8tjNgoX8Ugtdy2tmABrd+krvbfwy/ppt /E9gvtgzaSu5thLCnD/5MTd8O9Grj/+PqIIwBFuzHIdr7PjGZ5Ap6E7E7 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990760" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990760" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290745" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290745" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:22 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 11/15] x86/irq: Extend checks for pending vectors to posted interrupts Date: Fri, 26 Jan 2024 15:42:33 -0800 Message-Id: <20240126234237.547278-12-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 During interrupt affinity change, it is possible to have interrupts delivered to the old CPU after the affinity has changed to the new one. To prevent lost interrupts, local APIC IRR is checked on the old CPU. Similar checks must be done for posted MSIs given the same reason. Consider the following scenario: Device system agent iommu memory CPU/LAPIC 1 FEEX_XXXX 2 Interrupt request 3 Fetch IRTE -> 4 ->Atomic Swap PID.PIR(vec) Push to Global Observable(GO) 5 if (ON*) i done;* else 6 send a notification -> * ON: outstanding notification, 1 will suppress new notifications If the affinity change happens between 3 and 5 in IOMMU, the old CPU's posted interrupt request (PIR) could have pending bit set for the vector being moved. Signed-off-by: Jacob Pan --- arch/x86/include/asm/apic.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index e9d8e554765c..6661fefdd49a 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -13,6 +13,7 @@ #include #include #include +#include #define ARCH_APICTIMER_STOPS_ON_C3 1 @@ -500,7 +501,7 @@ static inline bool is_vector_pending(unsigned int vector) if (irr & (1 << (vector % 32))) return true; - return false; + return pi_pending_this_cpu(vector); } /* From patchwork Fri Jan 26 23:42:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533609 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24D0A604B7; Fri, 26 Jan 2024 23:37:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312252; cv=none; b=dYLx3r0Cnj6Q6f7FWMjapy15OYuQywO3jIc2HFgRz3oRDSXHZq+fdvOantYCkquOkWYFfiBsypVoH8h0V4KsdSU7vlMnU8d/Llz8tk4VyTwVZV3kZEw4CYyXQz6q82fIHS+COMptN9V/9/3nRujWGHr1SbqKdodigfQmXeku9V8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312252; c=relaxed/simple; bh=+q6Ng6aZ31M15sxzv7j/zcb8dlMqz5ufayt8TldTJ+8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PeidSPfO4fAfIghnNstt7OyjkeVt/3ysO+pThEEgUcsIj0/KZrOmctXsUbtSjtv1W0MDZ18lUKePCUPxtZ2Y5ob62q2WsmglaWeNrtxY260SOnUCTBxvCI/X074YdrZjBOJmdMczMnvL8+V2O/BcWkmhEGzy6TMk/UQPtY1T6Qs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PnwmklOy; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PnwmklOy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312250; x=1737848250; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+q6Ng6aZ31M15sxzv7j/zcb8dlMqz5ufayt8TldTJ+8=; b=PnwmklOyRsNQFMKCREY5L7G5HmXmdT9QTqlxPGy0oXPTXfUfnanU/xDs KNkb12nNp07C08N536rZ7lWU6KGE5IVIBg7UnLU6WEukAbgRC4NLm9qLR xOnJu1a9JasgxaXgDycT3vdwwDF2AgvxYFKeGBC52EZFqP9uBPs4raPsP 3xlzeCVC3ilPUQ8GdLll4WnrPAhtiv5Bq6JABMO1G+dMwTEOvF4VWLfJ3 leIwgpS7kiKeFjLe48jD2It+Kz80gubstdFEJnD6+kirzuu7T9imTrvko XslizsZt1KbKdkMz1seR8OaTKc0LkpxreL5YzvNMdWrFCEgY9Rp+5p2/3 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990764" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990764" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290748" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290748" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:22 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 12/15] iommu/vt-d: Make posted MSI an opt-in cmdline option Date: Fri, 26 Jan 2024 15:42:34 -0800 Message-Id: <20240126234237.547278-13-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a command line opt-in option for posted MSI if CONFIG_X86_POSTED_MSI=y. Also introduce a helper function for testing if posted MSI is supported on the platform. Signed-off-by: Jacob Pan --- Documentation/admin-guide/kernel-parameters.txt | 1 + arch/x86/include/asm/irq_remapping.h | 11 +++++++++++ drivers/iommu/irq_remapping.c | 13 ++++++++++++- 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 31b3a25680d0..35feefc0bdb0 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2212,6 +2212,7 @@ no_x2apic_optout BIOS x2APIC opt-out request will be ignored nopost disable Interrupt Posting + posted_msi enable MSIs delivered as posted interrupts iomem= Disable strict checking of access to MMIO memory strict regions from userspace. diff --git a/arch/x86/include/asm/irq_remapping.h b/arch/x86/include/asm/irq_remapping.h index 7a2ed154a5e1..e46bde61029b 100644 --- a/arch/x86/include/asm/irq_remapping.h +++ b/arch/x86/include/asm/irq_remapping.h @@ -50,6 +50,17 @@ static inline struct irq_domain *arch_get_ir_parent_domain(void) return x86_vector_domain; } +#ifdef CONFIG_X86_POSTED_MSI +extern int enable_posted_msi; + +static inline bool posted_msi_supported(void) +{ + return enable_posted_msi && irq_remapping_cap(IRQ_POSTING_CAP); +} +#else +static inline bool posted_msi_supported(void) { return false; }; +#endif + #else /* CONFIG_IRQ_REMAP */ static inline bool irq_remapping_cap(enum irq_remap_cap cap) { return 0; } diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c index 83314b9d8f38..4047ac396728 100644 --- a/drivers/iommu/irq_remapping.c +++ b/drivers/iommu/irq_remapping.c @@ -24,6 +24,10 @@ int no_x2apic_optout; int disable_irq_post = 0; +#ifdef CONFIG_X86_POSTED_MSI +int enable_posted_msi; +#endif + static int disable_irq_remap; static struct irq_remap_ops *remap_ops; @@ -70,7 +74,14 @@ static __init int setup_irqremap(char *str) no_x2apic_optout = 1; else if (!strncmp(str, "nopost", 6)) disable_irq_post = 1; - +#ifdef CONFIG_X86_POSTED_MSI + else if (!strncmp(str, "posted_msi", 10)) { + if (disable_irq_post || disable_irq_remap) + pr_warn("Posted MSI not enabled due to conflicting options!"); + else + enable_posted_msi = 1; + } +#endif str += strcspn(str, ","); while (*str == ',') str++; From patchwork Fri Jan 26 23:42:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533612 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A74BE60886; Fri, 26 Jan 2024 23:37:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312255; cv=none; b=ewcZMyd6QV3wtetaDTAB9QdL/FMEtAFxdTpaW54IdWfsWtA3g7s26E+C+R9WClnGpGa2/ocesuu9qzzYRlravFto+v+JXsah+5//jSw1LN5gRGoBbpcUOV6BONeWbrRGfQRD4Cl00OU6Tf2LsXWqR3Qsh+YXjRWZBcjniLdez2s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312255; c=relaxed/simple; bh=tBMxqVMHu63G2rJ3nBVSuahDnZ+Xdu27rPnRP82vOHs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=V5nkXVJg/nbBQFcU7N5HfrCVRFB1tT4xzzEUTP8DT2KMZmXyIT50GHcKpK/SP8rz4emZ8kDshtq8S2F3iV+L61Xe3VEU6NYOd5KNcxwp1Zv+ahS9XtBhNNa8DCKeE1ep9isJPrP2Ss9Fg17r6rZXHUCHcyITj79OOfB1lOM9wtM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CoHQNaQk; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CoHQNaQk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312252; x=1737848252; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tBMxqVMHu63G2rJ3nBVSuahDnZ+Xdu27rPnRP82vOHs=; b=CoHQNaQk+fa7SE2Cgdtn3qi11C/B4/3FesyQXpnM6wqpVpUsTQkuTnIP gWryWQC49UwoFhw/cstBTyAc68hXdkzy/Jr9cioFJORAYUbD69tho8cHx nRuNdOM4ZfMZI6Kpqbu/ocOa6ZRpqum+ryaLO9vMtlifykIRf866m0k+m B7hvQlw8TCxx3EbT4zPcKi0gZrtW8JVDNxwBmQp7VMIwejKBR7JbU5zeI LOr6FhCFcYJQoio+d84cNcxSEyAf42sFv5v7Xb+ksjUNdKB/z1oP2YmCX wrrZrZ4twNKwD23sDrBL41CDuqfrWmFeVVqZeUmNAHoXVY6xUCcqg/6gM g==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990778" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990778" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290752" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290752" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:23 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 13/15] iommu/vt-d: Add an irq_chip for posted MSIs Date: Fri, 26 Jan 2024 15:42:35 -0800 Message-Id: <20240126234237.547278-14-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce a new irq_chip for posted MSIs, the key difference is in irq_ack where EOI is performed by the notification handler. When posted MSI is enabled, MSI domain/chip hierarchy will look like this example: domain: IR-PCI-MSIX-0000:50:00.0-12 hwirq: 0x29 chip: IR-PCI-MSIX-0000:50:00.0 flags: 0x430 IRQCHIP_SKIP_SET_WAKE IRQCHIP_ONESHOT_SAFE parent: domain: INTEL-IR-10-13 hwirq: 0x2d0000 chip: INTEL-IR-POST flags: 0x0 parent: domain: VECTOR hwirq: 0x77 chip: APIC Signed-off-by: Jacob Pan --- drivers/iommu/intel/irq_remapping.c | 46 +++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c index 566297bc87dd..fa719936b44e 100644 --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -1233,6 +1233,52 @@ static struct irq_chip intel_ir_chip = { .irq_set_vcpu_affinity = intel_ir_set_vcpu_affinity, }; +static void dummy(struct irq_data *d) +{ +} + +/* + * With posted MSIs, all vectors are multiplexed into a single notification + * vector. Devices MSIs are then dispatched in a demux loop where + * EOIs can be coalesced as well. + * + * "INTEL-IR-POST" IRQ chip does not do EOI on ACK, thus the dummy irq_ack() + * function. Instead EOI is performed by the posted interrupt notification + * handler. + * + * For the example below, 3 MSIs are coalesced into one CPU notification. Only + * one apic_eoi() is needed. + * + * __sysvec_posted_msi_notification() + * irq_enter(); + * handle_edge_irq() + * irq_chip_ack_parent() + * dummy(); // No EOI + * handle_irq_event() + * driver_handler() + * irq_enter(); + * handle_edge_irq() + * irq_chip_ack_parent() + * dummy(); // No EOI + * handle_irq_event() + * driver_handler() + * irq_enter(); + * handle_edge_irq() + * irq_chip_ack_parent() + * dummy(); // No EOI + * handle_irq_event() + * driver_handler() + * apic_eoi() + * irq_exit() + */ +static struct irq_chip intel_ir_chip_post_msi = { + .name = "INTEL-IR-POST", + .irq_ack = dummy, + .irq_set_affinity = intel_ir_set_affinity, + .irq_compose_msi_msg = intel_ir_compose_msi_msg, + .irq_set_vcpu_affinity = intel_ir_set_vcpu_affinity, +}; + static void fill_msi_msg(struct msi_msg *msg, u32 index, u32 subhandle) { memset(msg, 0, sizeof(*msg)); From patchwork Fri Jan 26 23:42:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533610 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4BE76089C; Fri, 26 Jan 2024 23:37:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312254; cv=none; b=TvxPzsKuFWcHwR/OOIczAIZeS0e9FZQzhOxMFXEj3Ixn4EAZiU++KUb9TQUX4PGdWPmgB+HpBukdpncI0qCp0vJ40u2rOdcZO4oVNabO6s0QKb0/L03botqpaL4y5iyqCHevELcMaE5tlgo+9wiN2U532IeS8t6iRUJ8bJI62+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312254; c=relaxed/simple; bh=RLEMYdVVEnLOqM5rupdyL8vv8OyQExpN9wq5K3uinZc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DMswAOFwrr3Hq9hlKBD8alT17miAORXamBLdCzt1YsCInEU6e4MPvHUB+8YhdQQQTgOYGPCb6jxrOZoRnW9GRlsLSB/EHnHkusx+MO8tDFzIc2ftt5VY4LMMW9+d0drHzclKUraF1SQPtPyJUGP3hmv0Ir0NgD0wm4pggIj3ynY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hLljP1ki; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hLljP1ki" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312252; x=1737848252; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RLEMYdVVEnLOqM5rupdyL8vv8OyQExpN9wq5K3uinZc=; b=hLljP1kichLqz/h6poVw4fiZ+htNkkGk+R2nYgfJY0zb4VWLWDxhWVja GKv7VH27O7Sjd6apVqIOBP/bHDAOQgSCae6Bn9iZlGC1ygJ9KfLyYrxlQ tXPx0Y0IKiGXYLqaz9FL7RA1zrkUhZTA54LGg/cBdmA/+TtvBw3JF0sHm 3dMO5pyjvG+vfDDnW0Cjh+JX9uokUL5JHvggQpYIYoKjXxH1SXiGUkWZ1 ugQWW2VPhojUuCdTyVgxW6iRbCZS1nahY95zJ+y2I5++azX7NgKcannMg 8gDz/5r1hvOBqS4RWhyz54COeR1n1ojkz6PYLOKGx+NUrVhP4AuPjDXrD Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990798" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990798" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290755" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290755" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:23 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 14/15] iommu/vt-d: Add a helper to retrieve PID address Date: Fri, 26 Jan 2024 15:42:36 -0800 Message-Id: <20240126234237.547278-15-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Thomas Gleixner Physical address of the Intel posted interrupt descriptor (PID) is needed when programming interrupt remapping table entry (IRTE) for posted mode. PID is per-CPU, this patch adds a helper function to retrieve the target CPU's PID address based on the effective affinity mask of the interrupt. Signed-off-by: Thomas Gleixner Signed-off-by: Jacob Pan --- v1: Added a warning if the effective affinity mask is not set up --- drivers/iommu/intel/irq_remapping.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c index fa719936b44e..01df65dca1d5 100644 --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -19,6 +19,7 @@ #include #include #include +#include #include "iommu.h" #include "../irq_remapping.h" @@ -1126,6 +1127,19 @@ struct irq_remap_ops intel_irq_remap_ops = { .enable_faulting = enable_drhd_fault_handling, }; +#ifdef CONFIG_X86_POSTED_MSI + +static phys_addr_t get_pi_desc_addr(struct irq_data *irqd) +{ + int cpu = cpumask_first(irq_data_get_effective_affinity_mask(irqd)); + + if (WARN_ON(cpu >= nr_cpu_ids)) + return 0; + + return __pa(per_cpu_ptr(&posted_interrupt_desc, cpu)); +} +#endif + static void intel_ir_reconfigure_irte(struct irq_data *irqd, bool force) { struct intel_ir_data *ir_data = irqd->chip_data; From patchwork Fri Jan 26 23:42:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13533611 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B04E60BA0; Fri, 26 Jan 2024 23:37:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312254; cv=none; b=H6cq/qQlUpXrsZPvT5l5cvk/+4qvDofyTlgz+/Z1IVHYV5sdSEoZRPmrIxLAN7Afc1AIkQ4XbEFiejJ1jOoyj2JGr/9TzZWrUpIoEtb+paUyDpaQu756mz63rB889m5TVJH3T3g+0Zc5OkStIU1pwK4jDV1uV3bDjC/6WPR4aTc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706312254; c=relaxed/simple; bh=qUZEQDsSP0ln9mv97zVLDbJmQBXB2/WJhtv0FZ6mUTU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dG3R1J4F7a0aJJMNp3ZpTSxo2155t9ueVL0AjrCJqXHWTrzkU6XsUvRqmHKYmtcrthm/FnzJOGJltmvELzqHWqHUiwbmJCVufPer1fWhVE6nEblWsePzKCzZI97r6OLQ/FBsKd8Uu+2s/ffePSPAIzXfpY2euDA6lK870bPSz+w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XaKjn1f5; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XaKjn1f5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706312252; x=1737848252; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qUZEQDsSP0ln9mv97zVLDbJmQBXB2/WJhtv0FZ6mUTU=; b=XaKjn1f5OhrVxhiUxq/dD5OXnR4Mo675W4tGybfFcmklW60TlDtbklnI AV4Hf0YyHSAW6qTqSoHtlEWxogMG3YkRe8FoX1ftc4HKgWqlD0BckzY8u uc8/FdfT8CIBIOjlOrU6FVsQd8ccN/WR+0AkCq7wmaA/R9sDcYSzR8X7R ywUUWFCPOqrVCYsrSSPvnj0jo6qaX/7k7wV3GUzjJwyGlH3SvPxU8QhbS IyrUhjh/IF5bgL4tRVO7OjunNufTSAF0cx3hSuWi3fpGCOMbpHA14NCbd A0XrD79XTGaSMGkty8yfs6Au6vqgWOUmJTXa2po5edStMT6bzPTjI7rDH g==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9990810" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9990810" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 15:37:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="821290758" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="821290758" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.116]) by orsmga001.jf.intel.com with ESMTP; 26 Jan 2024 15:37:24 -0800 From: Jacob Pan To: LKML , X86 Kernel , Peter Zijlstra , iommu@lists.linux.dev, Thomas Gleixner , "Lu Baolu" , kvm@vger.kernel.org, Dave Hansen , Joerg Roedel , "H. Peter Anvin" , "Borislav Petkov" , "Ingo Molnar" Cc: Paul Luse , Dan Williams , Jens Axboe , Raj Ashok , "Tian, Kevin" , maz@kernel.org, seanjc@google.com, "Robin Murphy" , Jacob Pan Subject: [PATCH 15/15] iommu/vt-d: Enable posted mode for device MSIs Date: Fri, 26 Jan 2024 15:42:37 -0800 Message-Id: <20240126234237.547278-16-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 With posted MSI feature enabled on the CPU side, iommu interrupt remapping table entries (IRTEs) for device MSI/x can be allocated, activated, and programed in posted mode. This means that IRTEs are linked with their respective PIDs of the target CPU. Handlers for the posted MSI notification vector will de-multiplex device MSI handlers. CPU notifications are coalesced if interrupts arrive at a high frequency. Excluding the following: - legacy devices IOAPIC, HPET (may be needed for booting, not a source of high MSIs) - VT-d's own IRQs (not remappable). Signed-off-by: Jacob Pan --- drivers/iommu/intel/irq_remapping.c | 55 ++++++++++++++++++++++++++--- 1 file changed, 51 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c index 01df65dca1d5..ac5f9e83943b 100644 --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -50,6 +50,7 @@ struct irq_2_iommu { u16 sub_handle; u8 irte_mask; enum irq_mode mode; + bool posted_msi; }; struct intel_ir_data { @@ -1119,6 +1120,14 @@ static void prepare_irte(struct irte *irte, int vector, unsigned int dest) irte->redir_hint = 1; } +static void prepare_irte_posted(struct irte *irte) +{ + memset(irte, 0, sizeof(*irte)); + + irte->present = 1; + irte->p_pst = 1; +} + struct irq_remap_ops intel_irq_remap_ops = { .prepare = intel_prepare_irq_remapping, .enable = intel_enable_irq_remapping, @@ -1138,6 +1147,34 @@ static phys_addr_t get_pi_desc_addr(struct irq_data *irqd) return __pa(per_cpu_ptr(&posted_interrupt_desc, cpu)); } + +static void intel_ir_reconfigure_irte_posted(struct irq_data *irqd) +{ + struct intel_ir_data *ir_data = irqd->chip_data; + struct irte *irte = &ir_data->irte_entry; + struct irte irte_pi; + u64 pid_addr; + + pid_addr = get_pi_desc_addr(irqd); + + if (!pid_addr) { + pr_warn("Failed to setup IRQ %d for posted mode", irqd->irq); + return; + } + + memset(&irte_pi, 0, sizeof(irte_pi)); + + /* The shared IRTE already be set up as posted during alloc_irte */ + dmar_copy_shared_irte(&irte_pi, irte); + + irte_pi.pda_l = (pid_addr >> (32 - PDA_LOW_BIT)) & ~(-1UL << PDA_LOW_BIT); + irte_pi.pda_h = (pid_addr >> 32) & ~(-1UL << PDA_HIGH_BIT); + + modify_irte(&ir_data->irq_2_iommu, &irte_pi); +} + +#else +static inline void intel_ir_reconfigure_irte_posted(struct irq_data *irqd) {} #endif static void intel_ir_reconfigure_irte(struct irq_data *irqd, bool force) @@ -1153,8 +1190,9 @@ static void intel_ir_reconfigure_irte(struct irq_data *irqd, bool force) irte->vector = cfg->vector; irte->dest_id = IRTE_DEST(cfg->dest_apicid); - /* Update the hardware only if the interrupt is in remapped mode. */ - if (force || ir_data->irq_2_iommu.mode == IRQ_REMAPPING) + if (ir_data->irq_2_iommu.posted_msi) + intel_ir_reconfigure_irte_posted(irqd); + else if (force || ir_data->irq_2_iommu.mode == IRQ_REMAPPING) modify_irte(&ir_data->irq_2_iommu, irte); } @@ -1208,7 +1246,7 @@ static int intel_ir_set_vcpu_affinity(struct irq_data *data, void *info) struct intel_ir_data *ir_data = data->chip_data; struct vcpu_data *vcpu_pi_info = info; - /* stop posting interrupts, back to remapping mode */ + /* stop posting interrupts, back to the default mode */ if (!vcpu_pi_info) { modify_irte(&ir_data->irq_2_iommu, &ir_data->irte_entry); } else { @@ -1334,6 +1372,11 @@ static void intel_irq_remapping_prepare_irte(struct intel_ir_data *data, break; case X86_IRQ_ALLOC_TYPE_PCI_MSI: case X86_IRQ_ALLOC_TYPE_PCI_MSIX: + if (posted_msi_supported()) { + prepare_irte_posted(irte); + data->irq_2_iommu.posted_msi = 1; + } + set_msi_sid(irte, pci_real_dma_dev(msi_desc_to_pci_dev(info->desc))); break; @@ -1421,7 +1464,11 @@ static int intel_irq_remapping_alloc(struct irq_domain *domain, irq_data->hwirq = (index << 16) + i; irq_data->chip_data = ird; - irq_data->chip = &intel_ir_chip; + if (posted_msi_supported() && + ((info->type == X86_IRQ_ALLOC_TYPE_PCI_MSI) || (info->type == X86_IRQ_ALLOC_TYPE_PCI_MSIX))) + irq_data->chip = &intel_ir_chip_post_msi; + else + irq_data->chip = &intel_ir_chip; intel_irq_remapping_prepare_irte(ird, irq_cfg, info, index, i); irq_set_status_flags(virq + i, IRQ_MOVE_PCNTXT); }