From patchwork Thu Aug 18 19:42:41 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Suthikulpanit, Suravee" X-Patchwork-Id: 9288941 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 94268600CB for ; Fri, 19 Aug 2016 01:19:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8533128A63 for ; Fri, 19 Aug 2016 01:19:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 79E9828A6A; Fri, 19 Aug 2016 01:19:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E6C8B28A63 for ; Fri, 19 Aug 2016 01:19:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754516AbcHSBTi (ORCPT ); Thu, 18 Aug 2016 21:19:38 -0400 Received: from mail-sn1nam01on0058.outbound.protection.outlook.com ([104.47.32.58]:6892 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754140AbcHSBTg (ORCPT ); Thu, 18 Aug 2016 21:19:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Q4ALgmW3eUfJM6MgjTLEBKa1m65mjSCzUHOk5XyAeqU=; b=WrzVAccdIXk/Rx/Xlab4YoOaWM5wizZkr9xSvoRGvj6tcN5qihkG8XtUgM0zcFtTTOJpuy+YO5eb5GET0UJwelQb4jecUbiAzdMcQ3SSWwRanteRty1aquh1bqw/UcJ4fmIMZaObzwVQ+hC8zTaCM+4I8n+NnRBXmK3/UVIkLQY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Suravee.Suthikulpanit@amd.com; Received: from localhost.localdomain (114.109.128.54) by MWHPR12MB1453.namprd12.prod.outlook.com (10.172.55.22) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.9; Thu, 18 Aug 2016 19:43:52 +0000 From: Suravee Suthikulpanit To: , , , CC: , , , Suravee Suthikulpanit Subject: [PART2 PATCH v6 09/12] iommu/amd: Enable vAPIC interrupt remapping mode by default Date: Thu, 18 Aug 2016 14:42:41 -0500 Message-ID: <1471549364-6672-10-git-send-email-Suravee.Suthikulpanit@amd.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1471549364-6672-1-git-send-email-Suravee.Suthikulpanit@amd.com> References: <1471549364-6672-1-git-send-email-Suravee.Suthikulpanit@amd.com> MIME-Version: 1.0 X-Originating-IP: [114.109.128.54] X-ClientProxiedBy: KL1PR0201CA0016.apcprd02.prod.outlook.com (10.167.53.154) To MWHPR12MB1453.namprd12.prod.outlook.com (10.172.55.22) X-MS-Office365-Filtering-Correlation-Id: 1ccdff9f-50e2-44ff-b080-08d3c79ffd04 X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1453; 2:Mi/Yer+4knVo3XOc3oCJ5XiGrrknfpEncJE/3hE5s+LXe/z1yAV7fGwdyfMftW6oUAzltx6aAPTcqLvBDaOcs69MIewoYN8rEfilXGci9HMI9j4xBzzLTzNwFtekTnUOUo6JZpmNSRBdiAlor2CBxlV//nyyRLpNxOLAmaYnbQW9IvGFA5/HXz81FQv1Fu3V; 3:tiwcponmczAVVMOODjbIvxkP5QdEKiJJNByV+bcefvp1pevvX4E4niyFqj9PUtSmutN0l0AUrwqQ1uco0qVrGh8CgrdOj6tDEMlHdyGn27ZSIiBFHifcs7lOerZ0P+l5; 25:DzWjcNReiPusyXDzU4H5m1VrwhacrIb1KAwgKaYKq01d/rkMYHhw9j7ABNojbnXzjKJHY3xRfMQBxm5/fRwZ6ioqT7YpP4zzbt2fwtvkonUfklEszXXMVeqReCqm6aiJUPRj2Kg8CeC9H/tqEWi9KIuePRfZ3s6Qojca4uAlcfS+g7aLskFQdL6LvZXpUlFS5HA4cQED/X9bvynzrepoBMwdYYTmjraPXz21a6IlNohOateBhlq7/YGXaOq7U++MLqsyTUD1Cq+s/VwqbCjZ0kYZxN28G+szN6uWb+/HSfdOz5jiZuzfcLpfgpofd1tk4Qn1LuhSU0Fahx0DP3b31S6dUe1fOgFijHCl/njsrayoQPugrW6BDVu2qtEhaVPKJs6MPembmfya7Nl/kupxrS6RrVXq4YQoELuhykZB03c= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:MWHPR12MB1453; X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1453; 31:M5UJpGZgg8ueN5we95stkvS1/jXxcjDjygW+sxjfS2qvuNimthJycWOaj8je7lM0OP6OFrgqnSnYt+gGGfshhXVZN/DJ9vjvw84FGHxETkdLiwGEtyti9p8UHaw42Kz52dQBfJjcsQ5JusRLKsfn7t7Bu4RefjnlrYhBqYBPwHa1e6J+jNkymeGeKMJw0XzSHqTU4JDwGXx57bgKDwaD8Zkge8FwdnZRq+HvuoWjCoE=; 20:UZMVPh1FkJOIBmiqZpASQ5kqU8Zv7XWp5HilXUX89l1eGg3Iox0tS1zs0NTA/OMPrtKVwD0t0GNPHGPO1IDXNdEbNy461vVlzk4STr4PUXaM/U9IkqqrvekL0N+V6F2xpXMw/qpembyt2ZJ909YXz5jc19PHtb/2tNcA/xelSlu6iEeuFj4Qg6tpd7/z7BNV6hNue7KdLzxwb1dFrHzZLXRcu69TzO+RKXfAchWUauMJWMjZDpAnNnTL4tX10JRNN+ZUOZLYzjMtMDZgCEzpwfXXUpFUEDesASr7M73adGUu4FHXmgH4Rm6arRErJsXWMG6PHXcAKANxiKW3F9RoOLaGMJPYbRPFgLpve6CdRfHqcZpTp4cVE1Zlhy3+tOStNbKPf1gCN9t6jdKj4wj2EhGsBtWwDxdf2xc+KKFuR6tVQ0Qz6E3+vaBmqCfpbJ2TzOZ9bcADYAznZ+5XiVtAz6jmXo4irllBO2PBLfq4LVcUGm7RQtarrL8HrWKSqAnF X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026); SRVR:MWHPR12MB1453; BCL:0; PCL:0; RULEID:; SRVR:MWHPR12MB1453; X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1453; 4:Hpy2VjV33NEhNr6P46JHPwvqEyeJ2t9dsEzdTgdI9o82ZwcriCwHrAq50A6qJtPLWh2q4jKXosWvwunZeMsc7ukeekTMuSnZOtH6sxjEWacNi/Xx/MknDVwvo4YZGT7Whdx0KHhrXIMMhs/wyMm6sVExaj5W6b3muMln6yzZg3XY5xvxYIM23oy9Uj/Q+8WhoMUWjdW+gH7gnkhGi09zWwLG9f7mY+0SWsq5PNOLBBuwUih87aje8I6nmjHvCzmJZUmlDx5damZsOlpDioX/kdKUHN+ckON7ZsY3hThSa0jfTUAToqXgFIfNMzqKSOOonA5u9pJ+iKJFrzWEx/bkgX7ibZn6S6THT0y+TDgHNkFBqFz5iu8f6aV0jgV4l4VHcDFyLSHx7T1JmNYw08f26mRpM9lFrpZEIHpAO/9LwvfC5ULTuSuWRthmPGyo9Rsq X-Forefront-PRVS: 0038DE95A2 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(7916002)(189002)(199003)(86362001)(42186005)(50226002)(19580395003)(586003)(68736007)(92566002)(3846002)(6116002)(575784001)(189998001)(105586002)(8676002)(81156014)(81166006)(5003940100001)(47776003)(5001770100001)(2950100001)(97736004)(48376002)(77096005)(50986999)(50466002)(106356001)(36756003)(2201001)(305945005)(19580405001)(76176999)(7846002)(7736002)(4326007)(5890100001)(2906002)(66066001)(229853001)(101416001); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR12MB1453; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR12MB1453; 23:h+wDFyIdwUjUvyx671s0zULWK2V1CZNCsAFyFHlzH?= =?us-ascii?Q?DKVQZYCjSw50ieZOmxTmD0iuQ3xVyxoMUI+eLl9O+rJFnclxyB+juUBEfDHj?= =?us-ascii?Q?D5P1IxhvaPX+BNl2qpl6yQ0htDIKdZJ9CJZ6gwYut0ckg7SBXtCwSkysugLi?= =?us-ascii?Q?Kayudhvmt7gxa7yTIwTKv7+abrM6oOxvLA+FHXIZLprukQckSzDfq9V+JTwc?= =?us-ascii?Q?Ddtpv/pIJnk1kDBox1KCxLUhPhCVKhXyAJ7qHF6F+NINRiHZJOonlMZ26IOy?= =?us-ascii?Q?Ddz5hjpFP6plUpCCbnTNsHpQ/IABaPYDo/vxjInj9ckK2FNmepHkPD7rO10R?= =?us-ascii?Q?AuPnLvCCcYTxttHT26KKc1bToecfwOqxzjdGHusGjdfDZrB21OTFy5s1gHiq?= =?us-ascii?Q?0/3o//Pv18xjlphC+dvnfYyy+J5kBk2GrmqIAR7ontSvbv1vVG8XU40CmRV2?= =?us-ascii?Q?e84hP0H0NpE1mcHM3bhvViFf3bvib8SM7yeoxDhxbmqmoW9vutG7mSVZOSWI?= =?us-ascii?Q?m5D6LXy4CpVPo79cGgmuYMUsnJmtA1bZtlv0CG18Ln4a7vHvoNYP4kRg+Xbn?= =?us-ascii?Q?ebT7JbJLy8fQD+oOffkjP9hAxy1+u+/2ubdgQYMKRNKBcPUgFp2rXR2RWkw3?= =?us-ascii?Q?SnQsAG1NzjEFb2BgBH3tTyTRMEebQpyMyWqIsQdPDQaiG6tL2UleKW6viHsD?= =?us-ascii?Q?Ro6BLjzNZfnqWWARWtdQD5rHsHXTik6jOGM8YLNu8SWio+Ek2z9RexxxFTRp?= =?us-ascii?Q?Nt+qXACt9EUjO7rBfDlMMzpXMPPUSMOoKBrzF26yW+tHU2DosMdpS99RwJXt?= =?us-ascii?Q?b5ZRDjOh4OZDV9Z69ywrzDOp0G6JCUMNxhUSaUR/2RDR4gOTF/jSe2HK5aBZ?= =?us-ascii?Q?mfm+LfWO8jBXas/PbsNzsNrGtBKyBYLbZncHDe687jK8Xels2qkf7nTDYVHE?= =?us-ascii?Q?SdZOkw/ROJFfncYmslKa+SsovboK5jY9/JFf3xoRPjLSTsmya4IU/oFryCbo?= =?us-ascii?Q?SV5dhVmxGMPCNvBmQ0M07jiIRWph/4UB/snTmmBnjTHg/v+WyLkW7WlPE0Y8?= =?us-ascii?Q?/QtwkK/MvarqWPQlZ2hEpQ9Qh/aIwIJGabtux47WylqNJSjujuF0N0ihfLHw?= =?us-ascii?Q?y/Aaz2PtxA=3D?= X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1453; 6:Oz+kVngYKpy7VJ2Tqj46HOyKKICOwpdUPRUFux3OGLBDn3s2lmgiMifMs432YsNM9FmBo6NJIqO8IvWnu8t3fLEfWblb2svBXG/vQxsvxZp2l9ZXwvMOIVwtoX1iYelvcaPXZQLt6KC7Jdv7NVBg/m5eA7MzPSi99xD8jINFpp86OUiTUFc2t+vh3WXBAeVQ57EsRX4B3SthPOQ3GPX4Volw2SXkJq/vi7+rfdUlvQiP3F7w7yfCy+/3hImC22LJP1709MSH0imKcx3833qq3orxuei8xX3cdFoMftzdGNwYujZ10s/3I+KQC8K29QNa0VIzGVPWhEh1vlGkEoechw==; 5:+JZ+8wd68EIO0+H7xfwD3dbRB9/DPCw/m+Bae4wrJ6q/Ro3g9kwEUO0eE5V/8x6Has0HwmcVTTXPrTNCe/+VYF6J1wljAsZAfIek/b4mdwmzSq5J2Gqx2rygDF9YqvyUA1BuFHwZewLrZr45k/dMqA==; 24:CcNxuWG5vUahJ3V5muwujzMRpMlFwGe1eK3//hhogOshGpJVt4G06AjaFEffWm1QmEcQ4jEQA9KxN4dM5Q63oFzcQ0lRNMMdgyPHCQ7f+Jg=; 7:hrIA9fFZAbBWHa8RcYBUC4KTQaSeiv2j14ejivQyuoNWhMEx0m1QrGT2P7jW/bysHjCD4tCtA6pGIu1J5g+3U+kNEZyoSC9PxyFS0eYjTMrp2VXcK6KaNWzmdM+HkSb1uoUmGxQ4vqT/eHGARfQVBugf7S1WFgbnVCFmmnLpXv8JbyENhYbQ/9nO/wHG+XHRA/agbAidFxbakIw1vEBs/uIjXlbD0gP+g0nRsJ5sy8/KblDfyz1viciqzvd0xTzX SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1453; 20:++JcqoOq4wmSSgTD8MraEsyXT1IWSvHHoIUDcYpnaWfrE2sfIRExk0nNflN3uDOWHgL0KnuSfo5jotrbjJbBCb9K5kqBsEP/eMSlb9jSDpC5FdcOpQAQFkCbwZaaD5/AgohFKR4M9iv4fbGsxET/OKSB3NFRrEZCoD2JMTTncxw4YREAylRqBxtmxNsdLg3RdZV/hxapfbXtRqzUFw4KgyFx9LP2rgO5kAIxuXgKCx6PXAz5/amNuBhCyMBqk/DY X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2016 19:43:52.3301 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1453 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Introduce struct iommu_dev_data.use_vapic flag, which IOMMU driver uses to determine if it should enable vAPIC support, by setting the ga_mode bit in the device's interrupt remapping table entry. Currently, it is enabled for all pass-through device if vAPIC mode is enabled. Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd_iommu.c | 44 +++++++++++++++++++++++++++++++++-------- drivers/iommu/amd_iommu_init.c | 12 ++++++++++- drivers/iommu/amd_iommu_types.h | 2 +- 3 files changed, 48 insertions(+), 10 deletions(-) diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 65cdfcd..9cfac8b 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -105,6 +105,7 @@ struct iommu_dev_data { bool pri_tlp; /* PASID TLB required for PPR completions */ u32 errata; /* Bitmap for errata to apply */ + bool use_vapic; /* Enable device to use vapic mode */ }; /* @@ -3211,6 +3212,12 @@ static void amd_iommu_detach_device(struct iommu_domain *dom, if (!iommu) return; +#ifdef CONFIG_IRQ_REMAP + if (AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir) && + (dom->type == IOMMU_DOMAIN_UNMANAGED)) + dev_data->use_vapic = 0; +#endif + iommu_completion_wait(iommu); } @@ -3236,6 +3243,15 @@ static int amd_iommu_attach_device(struct iommu_domain *dom, ret = attach_device(dev, domain); +#ifdef CONFIG_IRQ_REMAP + if (AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir)) { + if (dom->type == IOMMU_DOMAIN_UNMANAGED) + dev_data->use_vapic = 1; + else + dev_data->use_vapic = 0; + } +#endif + iommu_completion_wait(iommu); return ret; @@ -3986,7 +4002,7 @@ static void free_irte(u16 devid, int index) static void irte_prepare(void *entry, u32 delivery_mode, u32 dest_mode, - u8 vector, u32 dest_apicid) + u8 vector, u32 dest_apicid, int devid) { union irte *irte = (union irte *) entry; @@ -4000,13 +4016,14 @@ static void irte_prepare(void *entry, static void irte_ga_prepare(void *entry, u32 delivery_mode, u32 dest_mode, - u8 vector, u32 dest_apicid) + u8 vector, u32 dest_apicid, int devid) { struct irte_ga *irte = (struct irte_ga *) entry; + struct iommu_dev_data *dev_data = search_dev_data(devid); irte->lo.val = 0; irte->hi.val = 0; - irte->lo.fields_remap.guest_mode = 0; + irte->lo.fields_remap.guest_mode = dev_data ? dev_data->use_vapic : 0; irte->lo.fields_remap.int_type = delivery_mode; irte->lo.fields_remap.dm = dest_mode; irte->hi.fields.vector = vector; @@ -4060,11 +4077,14 @@ static void irte_ga_set_affinity(void *entry, u16 devid, u16 index, u8 vector, u32 dest_apicid) { struct irte_ga *irte = (struct irte_ga *) entry; + struct iommu_dev_data *dev_data = search_dev_data(devid); - irte->hi.fields.vector = vector; - irte->lo.fields_remap.destination = dest_apicid; - irte->lo.fields_remap.guest_mode = 0; - modify_irte_ga(devid, index, irte, NULL); + if (!dev_data || !dev_data->use_vapic) { + irte->hi.fields.vector = vector; + irte->lo.fields_remap.destination = dest_apicid; + irte->lo.fields_remap.guest_mode = 0; + modify_irte_ga(devid, index, irte, NULL); + } } #define IRTE_ALLOCATED (~1U) @@ -4207,7 +4227,7 @@ static void irq_remapping_prepare_irte(struct amd_ir_data *data, data->irq_2_irte.index = index + sub_handle; iommu->irte_ops->prepare(data->entry, apic->irq_delivery_mode, apic->irq_dest_mode, irq_cfg->vector, - irq_cfg->dest_apicid); + irq_cfg->dest_apicid, devid); switch (info->type) { case X86_IRQ_ALLOC_TYPE_IOAPIC: @@ -4407,6 +4427,14 @@ static int amd_ir_set_vcpu_affinity(struct irq_data *data, void *vcpu_info) struct amd_ir_data *ir_data = data->chip_data; struct irte_ga *irte = (struct irte_ga *) ir_data->entry; struct irq_2_irte *irte_info = &ir_data->irq_2_irte; + struct iommu_dev_data *dev_data = search_dev_data(irte_info->devid); + + /* Note: + * This device has never been set up for guest mode. + * we should not modify the IRTE + */ + if (!dev_data || !dev_data->use_vapic) + return 0; pi_data->ir_data = ir_data; diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c index 156ab4b..cd17136 100644 --- a/drivers/iommu/amd_iommu_init.c +++ b/drivers/iommu/amd_iommu_init.c @@ -146,7 +146,7 @@ struct ivmd_header { bool amd_iommu_dump; bool amd_iommu_irq_remap __read_mostly; -int amd_iommu_guest_ir; +int amd_iommu_guest_ir = AMD_IOMMU_GUEST_IR_VAPIC; static bool amd_iommu_detected; static bool __initdata amd_iommu_disabled; @@ -2019,6 +2019,11 @@ static void early_enable_iommus(void) iommu_enable(iommu); iommu_flush_all_caches(iommu); } + +#ifdef CONFIG_IRQ_REMAP + if (AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir)) + amd_iommu_irq_ops.capability |= (1 << IRQ_POSTING_CAP); +#endif } static void enable_iommus_v2(void) @@ -2044,6 +2049,11 @@ static void disable_iommus(void) for_each_iommu(iommu) iommu_disable(iommu); + +#ifdef CONFIG_IRQ_REMAP + if (AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir)) + amd_iommu_irq_ops.capability &= ~(1 << IRQ_POSTING_CAP); +#endif } /* diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h index 71d1d0e..596e43f 100644 --- a/drivers/iommu/amd_iommu_types.h +++ b/drivers/iommu/amd_iommu_types.h @@ -803,7 +803,7 @@ struct irte_ga { }; struct amd_irte_ops { - void (*prepare)(void *, u32, u32, u8, u32); + void (*prepare)(void *, u32, u32, u8, u32, int); void (*activate)(void *, u16, u16); void (*deactivate)(void *, u16, u16); void (*set_affinity)(void *, u16, u16, u8, u32);