From patchwork Thu Jul 25 13:32:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11058831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ADA5D746 for ; Thu, 25 Jul 2019 13:37:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9CD6228812 for ; Thu, 25 Jul 2019 13:37:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9087A28826; Thu, 25 Jul 2019 13:37:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C28432882C for ; Thu, 25 Jul 2019 13:37:18 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hqduZ-000802-Nz; Thu, 25 Jul 2019 13:35:59 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hqduY-0007zl-Fl for xen-devel@lists.xenproject.org; Thu, 25 Jul 2019 13:35:58 +0000 X-Inumbo-ID: 20649880-aee1-11e9-b269-83af57f464a0 Received: from m4a0039g.houston.softwaregrp.com (unknown [15.124.2.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 20649880-aee1-11e9-b269-83af57f464a0; Thu, 25 Jul 2019 13:35:55 +0000 (UTC) Received: FROM m4a0039g.houston.softwaregrp.com (15.120.17.146) BY m4a0039g.houston.softwaregrp.com WITH ESMTP; Thu, 25 Jul 2019 13:35:50 +0000 Received: from M4W0334.microfocus.com (2002:f78:1192::f78:1192) by M4W0334.microfocus.com (2002:f78:1192::f78:1192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Thu, 25 Jul 2019 13:32:42 +0000 Received: from NAM05-BY2-obe.outbound.protection.outlook.com (15.124.8.13) by M4W0334.microfocus.com (15.120.17.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Thu, 25 Jul 2019 13:32:42 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UoF/N2fX+/4bsYZgbJ4d17ZbEo8S2lg64GzN9wRW9u6RMrV0G57dboSA5h0zyv34BKd0bKvcOXfQKEDq5K5XmvoonAqIycVL+EeMBjM2rQc70Sufr+H/tFZkkz53AxqGj+cq3lju+3Z7Pc8RtRUFyy9Jk9I+RXp4Swo4xUq9OGWxax4U3b478uDqVC823nuPw79h7Ak6l8nonRj9fOeUStoalQDpDHBgF/7uljf5fGJvEh+/fEPIyXqbUBRt8dkDW+0CfABKJeO5CEahiaAKFvM+pxSbQxEB0WC7B37S2Iynqvptiufwjojs1rVHRAhMZNBVMXzvXsnOiZK0arIL/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=05BeA7+zlzlPgeC8wqnYLrDRW+B+4+k20mX6+v0Bj0s=; b=i9pUunk2ARQRtPmoLIGrlsKIdhEScDvmo8Rwh43jghUrI3PaFPGafMStKE/estVYUn2T0vksdcFwWWlo5Y+Lp59vTKOAzUhS5LamNilZDOWrdi0beFVs9U2tu6zUrP75mAIRkJ64mkmxxqYAiBHVPoxnxTIwaFCGWJ/+MG/VUu5TvB428bghRFRy1ynpZfZPzspoSIKwGC3gOOJ7+Pq5w8nGj53HsVFturQKqe0Klny/B3O2zZP/X4xoufTA03NVmujqvIa5w0dJDyLTsi3L2ddTDLOEjweZntGC345Vw9DzZP82hXH6UoMAB3saMQoiLQvZQ912OkgU8xAxIx18fg== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=suse.com;dmarc=pass action=none header.from=suse.com;dkim=pass header.d=suse.com;arc=none Received: from BY5PR18MB3394.namprd18.prod.outlook.com (10.255.139.95) by BY5PR18MB3297.namprd18.prod.outlook.com (10.255.163.221) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2115.10; Thu, 25 Jul 2019 13:32:40 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::a4f0:4f46:c076:f74b]) by BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::a4f0:4f46:c076:f74b%7]) with mapi id 15.20.2115.005; Thu, 25 Jul 2019 13:32:40 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH v4 09/12] AMD/IOMMU: enable x2APIC mode when available Thread-Index: AQHVQu1ukmUsOZ1yIkGpOa1CMLMQVw== Date: Thu, 25 Jul 2019 13:32:40 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LO2P265CA0033.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:61::21) To BY5PR18MB3394.namprd18.prod.outlook.com (2603:10b6:a03:194::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7060274d-8c77-4585-7153-08d7110490d9 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:BY5PR18MB3297; x-ms-traffictypediagnostic: BY5PR18MB3297: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0109D382B0 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(39860400002)(366004)(136003)(396003)(376002)(346002)(189003)(199004)(6436002)(53936002)(2906002)(386003)(6506007)(2501003)(86362001)(478600001)(5660300002)(64756008)(4326008)(66556008)(305945005)(6486002)(31696002)(486006)(25786009)(66066001)(6512007)(102836004)(80792005)(66946007)(5640700003)(14454004)(26005)(66446008)(7736002)(14444005)(446003)(52116002)(99286004)(76176011)(54906003)(6116002)(3846002)(31686004)(316002)(81166006)(81156014)(6916009)(8676002)(8936002)(68736007)(2616005)(66476007)(11346002)(256004)(71190400001)(71200400001)(36756003)(476003)(2351001)(186003); DIR:OUT; SFP:1102; SCL:1; SRVR:BY5PR18MB3297; H:BY5PR18MB3394.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: xUwRFQty3UbbHKOMFna1lOQvyshRu9ScUWcMF0XJbOhmikCF4SFIRO+SbuFUBUDANDlOW/EXScdQuRSTKdVWnDgNmLGUSbNjX/nHOaAz6Qg1tKV1t4e9WyZvD+dicFgMUKcAYERt+fiFdMkwvMerZG+BfLF9j+HaWkZfgqsVOxIGU2nFRQ/zzSZsTx9BiY7z4oaOiB/Yu/CQ766lpb1j7K+zBueCaqdBY4c2E5MMUJUKhnGvGkEm+Is+b17u+ecZPk5LuQ0GnzWlfTv/Gbtw5kLKDvSKx/BLh3iXlYoB5cZh+LEQ3o2PJLVAViHXmCHBqwH9odfYFUjLoyeNhT1ntWsNqsEC6FSopQfZm7LIrCzznIW9MaQxuCGCuI48qhMj32D1z6/4oPkQ0ZJulwxeN1YjXApeoE8/coJRfzFAX+o= Content-ID: <010D44378A2BB94BA192137589FD8D20@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 7060274d-8c77-4585-7153-08d7110490d9 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Jul 2019 13:32:40.5614 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR18MB3297 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH v4 09/12] AMD/IOMMU: enable x2APIC mode when available X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Brian Woods , Suravee Suthikulpanit Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP In order for the CPUs to use x2APIC mode, the IOMMU(s) first need to be switched into suitable state. The post-AP-bringup IRQ affinity adjustment is done also for the non- x2APIC case, matching what VT-d does. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper Acked-by: Brian Woods --- v4: Re-base. v3: Set GAEn (and other control register bits) earlier. Also clear the bits enabled here in amd_iommu_init_cleanup(). Re-base. Pass NULL CPU mask to set_{x2apic,msi}_affinity(). v2: Drop cpu_has_cx16 check. Add comment. --- TBD: Instead of the system_state check in iov_enable_xt() the function could also zap its own hook pointer, at which point it could also become __init. This would, however, require that either resume_x2apic() be bound to ignore iommu_enable_x2apic() errors forever, or that iommu_enable_x2apic() be slightly re-arranged to not return -EOPNOTSUPP when finding a NULL hook during resume. --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -833,6 +833,30 @@ static bool_t __init set_iommu_interrupt return 1; } +int iov_adjust_irq_affinities(void) +{ + const struct amd_iommu *iommu; + + if ( !iommu_enabled ) + return 0; + + for_each_amd_iommu ( iommu ) + { + struct irq_desc *desc = irq_to_desc(iommu->msi.irq); + unsigned long flags; + + spin_lock_irqsave(&desc->lock, flags); + if ( iommu->ctrl.int_cap_xt_en ) + set_x2apic_affinity(desc, NULL); + else + set_msi_affinity(desc, NULL); + spin_unlock_irqrestore(&desc->lock, flags); + } + + return 0; +} +__initcall(iov_adjust_irq_affinities); + /* * Family15h Model 10h-1fh erratum 746 (IOMMU Logging May Stall Translations) * Workaround: @@ -1046,7 +1070,7 @@ static void * __init allocate_ppr_log(st IOMMU_PPR_LOG_DEFAULT_ENTRIES, "PPR Log"); } -static int __init amd_iommu_init_one(struct amd_iommu *iommu) +static int __init amd_iommu_init_one(struct amd_iommu *iommu, bool intr) { if ( allocate_cmd_buffer(iommu) == NULL ) goto error_out; @@ -1057,7 +1081,7 @@ static int __init amd_iommu_init_one(str if ( iommu->features.flds.ppr_sup && !allocate_ppr_log(iommu) ) goto error_out; - if ( !set_iommu_interrupt_handler(iommu) ) + if ( intr && !set_iommu_interrupt_handler(iommu) ) goto error_out; /* To make sure that device_table.buffer has been successfully allocated */ @@ -1086,8 +1110,16 @@ static void __init amd_iommu_init_cleanu list_for_each_entry_safe ( iommu, next, &amd_iommu_head, list ) { list_del(&iommu->list); + + iommu->ctrl.ga_en = 0; + iommu->ctrl.xt_en = 0; + iommu->ctrl.int_cap_xt_en = 0; + if ( iommu->enabled ) disable_iommu(iommu); + else if ( iommu->mmio_base ) + writeq(iommu->ctrl.raw, + iommu->mmio_base + IOMMU_CONTROL_MMIO_OFFSET); deallocate_ring_buffer(&iommu->cmd_buffer); deallocate_ring_buffer(&iommu->event_log); @@ -1289,7 +1321,7 @@ static int __init amd_iommu_prepare_one( return 0; } -int __init amd_iommu_init(void) +int __init amd_iommu_prepare(bool xt) { struct amd_iommu *iommu; int rc = -ENODEV; @@ -1304,9 +1336,14 @@ int __init amd_iommu_init(void) if ( unlikely(acpi_gbl_FADT.boot_flags & ACPI_FADT_NO_MSI) ) goto error_out; + /* Have we been here before? */ + if ( ivhd_type ) + return 0; + rc = amd_iommu_get_supported_ivhd_type(); if ( rc < 0 ) goto error_out; + BUG_ON(!rc); ivhd_type = rc; rc = amd_iommu_get_ivrs_dev_entries(); @@ -1322,9 +1359,37 @@ int __init amd_iommu_init(void) rc = amd_iommu_prepare_one(iommu); if ( rc ) goto error_out; + + rc = -ENODEV; + if ( xt && (!iommu->features.flds.ga_sup || !iommu->features.flds.xt_sup) ) + goto error_out; + } + + for_each_amd_iommu ( iommu ) + { + /* NB: There's no need to actually write these out right here. */ + iommu->ctrl.ga_en |= xt; + iommu->ctrl.xt_en = xt; + iommu->ctrl.int_cap_xt_en = xt; } rc = amd_iommu_update_ivrs_mapping_acpi(); + + error_out: + if ( rc ) + { + amd_iommu_init_cleanup(); + ivhd_type = 0; + } + + return rc; +} + +int __init amd_iommu_init(bool xt) +{ + struct amd_iommu *iommu; + int rc = amd_iommu_prepare(xt); + if ( rc ) goto error_out; @@ -1350,7 +1415,12 @@ int __init amd_iommu_init(void) /* per iommu initialization */ for_each_amd_iommu ( iommu ) { - rc = amd_iommu_init_one(iommu); + /* + * Setting up of the IOMMU interrupts cannot occur yet at the (very + * early) time we get here when enabling x2APIC mode. Suppress it + * here, and do it explicitly in amd_iommu_init_interrupt(). + */ + rc = amd_iommu_init_one(iommu, !xt); if ( rc ) goto error_out; } @@ -1362,6 +1432,40 @@ error_out: return rc; } +int __init amd_iommu_init_interrupt(void) +{ + struct amd_iommu *iommu; + int rc = 0; + + for_each_amd_iommu ( iommu ) + { + struct irq_desc *desc; + + if ( !set_iommu_interrupt_handler(iommu) ) + { + rc = -EIO; + break; + } + + desc = irq_to_desc(iommu->msi.irq); + + spin_lock(&desc->lock); + ASSERT(iommu->ctrl.int_cap_xt_en); + set_x2apic_affinity(desc, &cpu_online_map); + spin_unlock(&desc->lock); + + set_iommu_event_log_control(iommu, IOMMU_CONTROL_ENABLED); + + if ( iommu->features.flds.ppr_sup ) + set_iommu_ppr_log_control(iommu, IOMMU_CONTROL_ENABLED); + } + + if ( rc ) + amd_iommu_init_cleanup(); + + return rc; +} + static void invalidate_all_domain_pages(void) { struct domain *d; --- a/xen/drivers/passthrough/amd/iommu_intr.c +++ b/xen/drivers/passthrough/amd/iommu_intr.c @@ -799,6 +799,35 @@ void *__init amd_iommu_alloc_intremap_ta return tb; } +bool __init iov_supports_xt(void) +{ + unsigned int apic; + + if ( !iommu_enable || !iommu_intremap ) + return false; + + if ( amd_iommu_prepare(true) ) + return false; + + for ( apic = 0; apic < nr_ioapics; apic++ ) + { + unsigned int idx = ioapic_id_to_index(IO_APIC_ID(apic)); + + if ( idx == MAX_IO_APICS ) + return false; + + if ( !find_iommu_for_device(ioapic_sbdf[idx].seg, + ioapic_sbdf[idx].bdf) ) + { + AMD_IOMMU_DEBUG("No IOMMU for IO-APIC %#x (ID %x)\n", + apic, IO_APIC_ID(apic)); + return false; + } + } + + return true; +} + int __init amd_setup_hpet_msi(struct msi_desc *msi_desc) { spinlock_t *lock; --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -170,7 +170,8 @@ static int __init iov_detect(void) if ( !iommu_enable && !iommu_intremap ) return 0; - if ( amd_iommu_init() != 0 ) + else if ( (init_done ? amd_iommu_init_interrupt() + : amd_iommu_init(false)) != 0 ) { printk("AMD-Vi: Error initialization\n"); return -ENODEV; @@ -184,6 +185,25 @@ static int __init iov_detect(void) return 0; } +static int iov_enable_xt(void) +{ + int rc; + + if ( system_state >= SYS_STATE_active ) + return 0; + + if ( (rc = amd_iommu_init(true)) != 0 ) + { + printk("AMD-Vi: Error %d initializing for x2APIC mode\n", rc); + /* -ENXIO has special meaning to the caller - convert it. */ + return rc != -ENXIO ? rc : -ENODATA; + } + + init_done = true; + + return 0; +} + int amd_iommu_alloc_root(struct domain_iommu *hd) { if ( unlikely(!hd->arch.root_table) ) @@ -557,11 +577,13 @@ static const struct iommu_ops __initcons .free_page_table = deallocate_page_table, .reassign_device = reassign_device, .get_device_group_id = amd_iommu_group_id, + .enable_x2apic = iov_enable_xt, .update_ire_from_apic = amd_iommu_ioapic_update_ire, .update_ire_from_msi = amd_iommu_msi_msg_update_ire, .read_apic_from_ire = amd_iommu_read_ioapic_from_ire, .read_msi_from_ire = amd_iommu_read_msi_from_ire, .setup_hpet_msi = amd_setup_hpet_msi, + .adjust_irq_affinities = iov_adjust_irq_affinities, .suspend = amd_iommu_suspend, .resume = amd_iommu_resume, .crash_shutdown = amd_iommu_crash_shutdown, @@ -571,4 +593,5 @@ static const struct iommu_ops __initcons static const struct iommu_init_ops __initconstrel _iommu_init_ops = { .ops = &_iommu_ops, .setup = iov_detect, + .supports_x2apic = iov_supports_xt, }; --- a/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h +++ b/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h @@ -48,8 +48,11 @@ int amd_iommu_detect_acpi(void); void get_iommu_features(struct amd_iommu *iommu); /* amd-iommu-init functions */ -int amd_iommu_init(void); +int amd_iommu_prepare(bool xt); +int amd_iommu_init(bool xt); +int amd_iommu_init_interrupt(void); int amd_iommu_update_ivrs_mapping_acpi(void); +int iov_adjust_irq_affinities(void); /* mapping functions */ int __must_check amd_iommu_map_page(struct domain *d, dfn_t dfn, @@ -93,6 +96,7 @@ void amd_iommu_flush_all_caches(struct a struct amd_iommu *find_iommu_for_device(int seg, int bdf); /* interrupt remapping */ +bool iov_supports_xt(void); int amd_iommu_setup_ioapic_remapping(void); void *amd_iommu_alloc_intremap_table( const struct amd_iommu *, unsigned long **);