From patchwork Wed Sep 22 14:36:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12510809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F36B0C433EF for ; Wed, 22 Sep 2021 14:37:01 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9B54F60FE6 for ; Wed, 22 Sep 2021 14:37:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9B54F60FE6 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.192862.343558 (Exim 4.92) (envelope-from ) id 1mT3Mi-0001rj-Sl; Wed, 22 Sep 2021 14:36:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 192862.343558; Wed, 22 Sep 2021 14:36:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3Mi-0001ra-Pe; Wed, 22 Sep 2021 14:36:52 +0000 Received: by outflank-mailman (input) for mailman id 192862; Wed, 22 Sep 2021 14:36:51 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3Mh-0001rN-RX for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:36:51 +0000 Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 8596ceb8-1bb2-11ec-b99e-12813bfff9fa; Wed, 22 Sep 2021 14:36:50 +0000 (UTC) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04lp2055.outbound.protection.outlook.com [104.47.13.55]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-34-LH5A588vO8K5uVEmRWeoig-1; Wed, 22 Sep 2021 16:36:48 +0200 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep 2021 14:36:46 +0000 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021 14:36:45 +0000 Received: from [10.156.60.236] (37.24.206.209) by FR3P281CA0027.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend Transport; Wed, 22 Sep 2021 14:36:45 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8596ceb8-1bb2-11ec-b99e-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1632321409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ym1bnxR0U4vds7SXc2ouMI0aswya4N0rxIF16d3/+iM=; b=AZ9WRWLKhHmdDZGSHs8yUY75HkL9HDup3+WemIZQJJ0FLReUbfv+Y+Mw5nCCsPWFxS0UeC qnpM/3k2RAAWTTtQwJIxiaVDSoMEfonwPpYHF9JQob1tWi6MiQ10Y1DN/9okiV0Bxi3YHO 69CMHFSts7DJeKZ1zytmAD+ZLX37DVk= X-MC-Unique: LH5A588vO8K5uVEmRWeoig-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Lh8Xe1fxqTsI5KT+N8xZgj8gKILY+h5wfo1Uh0bKBhkrsr++rcUxPOoYEAdo21gpZ3q5O6OYv9x2EN4fdHRTXxPF/HLhtPdtYFu/FCJIcOLd07a1zcTIPFgebYmGeX/WPs+qYoJ7gIuPJuQrwCfBHfujr6ngXuyEf1kClRspYL1TDf+/rVipIvovx4ItjufDdx10a0V0yhUkZqanRYncjG0x+W3Cwuszfm6kWR2UPtsehPhbnjNbsIS8h4bqpqlhq9X/q4A0ZTQ4/yqreURyBRcSqYL7u0prmNX40VPBAS4drTwwEtDb1p+zrhSA8pUKxQxir6oMvO3tfIyz9C0EYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Ym1bnxR0U4vds7SXc2ouMI0aswya4N0rxIF16d3/+iM=; b=CXvssv9xSVHt4LWK6I55TFga+dHaAn0Tpj4b39qaLcFp7k+UGKR5OjxmKnLPFfXsVLBUFqC4+QfZ0Abkl1VXQeBIXenBhCqnZoS7c17eKvEFggc3lLO7KJi8wrCuKqbt43s7Eb/cpU7k9Hz8Omzo1tGxCliQG3SxpocP47+JM9WEGjHh+KHXYdSVp9A14uEOW5RsAGmrYdIKKbj4kwZ9u84mfsZ4PH+hTvaLuqO2zncqJdMOCf1e96T1L/s2c+6RMDh7pGnIro9NWWgm5qri8CbD3tET7+fTFsUW48EM+9wchCM8UMlN8KAaNsSSfCYy3yZYGNhrrk7hkECye9OcrA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=suse.com; Subject: [PATCH v8 1/6] AMD/IOMMU: obtain IVHD type to use earlier From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant References: Message-ID: Date: Wed, 22 Sep 2021 16:36:45 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Language: en-US X-ClientProxiedBy: FR3P281CA0027.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::14) To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 19ac131c-b341-44fa-1e57-08d97dd66720 X-MS-TrafficTypeDiagnostic: VI1PR04MB7039: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XsmVvJo1yHaisw75CefUVryWMfUUYQv6QCQ+7ukYy+Br5/WHAzLmPuCP9UrtMMkbdXI7rF2bf16+2xplxKFCkGxS9KY6FBYQMVBFGzS58gSri05lNxcURJQeDdgsVv1x6Km1rAuWcwblmgM/w/cLJSq7OGzCcvWBeg2EOR8YHkdMtqYSVpMyOdGqWy0Dz49LsR+fejQtyVar0DuNrl2FqZFRrPjTLBkDAV5+WdvCecZLBBLNOKDFowy88K3iGHuZlAShp8yL4D8qO1QCYzK5nSgEhApT9111WQ5qCxwVELe9EGZ3wulSd/2KBoTP0W1JXK/mZwa61iLFRxX0HtEdrZVwsZWAlCNaymOg3ZajLWi8cudro3VogBlwUTrhLq+ikSYUmAD1L/xqY4HjRpZ/wlh31YLSCw7/AoOfwAodvEa+AttHl+CdX4rViqJ+lNRfCTpSqCw1Fl22+GbUdzNTId/VRWJZ8IfEE6YMyiIBB6WHmi48dv+BoTbA0OnPsuCXDWWXRywNMOp2pr5jZz9sHA0V4ZFsel8i0muTifK+Ap5YQ1IJ31x3E+3qjyDKn4VYCq/AU/K6LlK2BEGqVsljAGdTtwHzCvdDFPgsow1jdRphMMLQ73wCUYtxwBxncPUE4+2SXiqpo8kg7jW6NuLoFN0hK/fnqueKyNh/L3GXVvQT5Uvr2JRPp+QF0lMqQTE6sEKJTFAK3S59UXnPHvC5Zqlt1aPg7AqauD/nMGwU3Jw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(66476007)(4326008)(66946007)(83380400001)(38100700002)(31696002)(66556008)(316002)(5660300002)(54906003)(31686004)(2906002)(86362001)(16576012)(6486002)(186003)(8676002)(2616005)(956004)(36756003)(6916009)(8936002)(508600001)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?f+201ESdgsNlfI0JPwV2VClm1thR?= =?utf-8?q?MT/FBBqSn+DCaQXHXq9ynrecyHiSUq7yzFnk96NXUwOg/YN8KgtN9uRB7ME7pq66H?= =?utf-8?q?ZFx2M8XYS/PhLusmBKSvhoOrxXuezpzt20hSzwFxB03NgmvvJlx7/9iYDI+tm2skP?= =?utf-8?q?yRgFUnNQLpNRWnT8i2JNkoYeES0gmiiidRmOQu6KQfBbBOnXDqyDHscnuK0s7Jof8?= =?utf-8?q?34sG12g9Dmh0rbQN9JYJ6NC9OCUZ8stiX4RyVklbcbwW05edHRnQw5B5XfKv6eUck?= =?utf-8?q?1vuNqvm9CtJ0VM6fY0ODuCYBl6OZxw+fUU1fKSm+FjM6A062BEhrRvbAuj9MhX5CQ?= =?utf-8?q?oZiOWSCPAQ+U6ree3uJh4PKeZqr0/dcC9/P/JaPpUihd41gUiZBsdJwmbG3bOYqk8?= =?utf-8?q?2NgNNX++EJvtT1F68Qu16JC55iD1xZlwh2j9tNviPh/7KoQdFSNR60MCmUbsEyroQ?= =?utf-8?q?rGeCz0BE9njlRnFi2tXgs0yBDGcEjB4fKB0ZMtBzGBxNdcBj25fUQmZ898GDIe0wS?= =?utf-8?q?nQW5ZqWt3OWATNB9OrDfSDS5yx6TYoqf4hf55TPtBBvJGUvDx0yb7m8U3fMPqgM5P?= =?utf-8?q?HPENVp/5D+cfFJsEx+osEQISDuS6tXY0hR+acUbAQVwhzAX/zeE/pW13Bkfsw/jRZ?= =?utf-8?q?GSavcOFQJYlVO0k8oJUDsw+IjkE+WAWBQQPHhyopzyZ/eb2uEq8J4b4k3cKSSujyd?= =?utf-8?q?KHE5eD2kcvqypNJKZZntuPYXDqcR1WEcioqxY5wWdfwpg+9cnfsHKcj7uJWZCGh0K?= =?utf-8?q?WqUxfsaDhEMwrTbUqKTmTYR7u3tAeaTAPE5hDrxB25q0peDA0ROOWHaEI9HYpD94I?= =?utf-8?q?dP1m4zfiJGcJdH8iR16fhLGIb3oGLEIxIKMYI3h0VL9TS5+H/Kwvqq9Y5t4EkcKMy?= =?utf-8?q?lPBcHfiu4xIp8gtmxAStuA79ZIXPa5H33vhjTw0tgwrnvu19VvyN6yHScLldRh9NS?= =?utf-8?q?viPj84a59NZJDfkum5i4yyI2CwOoCiFHKmM8OrVFt0mROfh8kYu9NMkAsg50irGV5?= =?utf-8?q?rerVb3bOXSkeqSsEYIiY2xx864RwJr+vLsjNocBkHDyTV4Rrk+lmTXmLIu+V+WxUY?= =?utf-8?q?ekeR2mEmWPaKFV8ChUN6DKI/u61LDatFAALpFBbt4qd1i7YHPUc2tk+/0C7Q9gyrT?= =?utf-8?q?NvWTuoIQ9juTm9+EWsNVjivzw8FGceX9VYvNhQcb+Ie+EFcuSVQbCoTbiaAqXCQQ/?= =?utf-8?q?5ss0wGNglvXj6as7PQD5+teXt1Y9VwRNYDXFb78bB+yUcGbp8MpXsrw91zhCyqyfB?= =?utf-8?q?mdMEBPW9TDQalDhy?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 19ac131c-b341-44fa-1e57-08d97dd66720 X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:36:45.7146 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cyDf217/hxYrsNVQpZfLa3KgiemEcboMlTiQSQ8Cl/RwRWbYjCZWmBucnQZCw7BLMEwWgQqU/nWoaC5iClSOYA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039 Doing this in amd_iommu_prepare() is too late for it, in particular, to be used in amd_iommu_detect_one_acpi(), as a subsequent change will want to do. Moving it immediately ahead of amd_iommu_detect_acpi() is (luckily) pretty simple, (pretty importantly) without breaking amd_iommu_prepare()'s logic to prevent multiple processing. This involves moving table checksumming, as amd_iommu_get_supported_ivhd_type() -> get_supported_ivhd_type() will now be invoked before amd_iommu_detect_acpi() -> detect_iommu_acpi(). In the course of doing so stop open-coding acpi_tb_checksum(), seeing that we have other uses of this originally ACPI-private function elsewhere in the tree. Signed-off-by: Jan Beulich Reviewed-by: Paul Durrant --- v7: Move table checksumming. v5: New. --- a/xen/drivers/passthrough/amd/iommu_acpi.c +++ b/xen/drivers/passthrough/amd/iommu_acpi.c @@ -22,6 +22,8 @@ #include +#include + #include "iommu.h" /* Some helper structures, particularly to deal with ranges. */ @@ -1167,20 +1169,7 @@ static int __init parse_ivrs_table(struc static int __init detect_iommu_acpi(struct acpi_table_header *table) { const struct acpi_ivrs_header *ivrs_block; - unsigned long i; unsigned long length = sizeof(struct acpi_table_ivrs); - u8 checksum, *raw_table; - - /* validate checksum: sum of entire table == 0 */ - checksum = 0; - raw_table = (u8 *)table; - for ( i = 0; i < table->length; i++ ) - checksum += raw_table[i]; - if ( checksum ) - { - AMD_IOMMU_DEBUG("IVRS Error: Invalid Checksum %#x\n", checksum); - return -ENODEV; - } while ( table->length > (length + sizeof(*ivrs_block)) ) { @@ -1317,6 +1306,15 @@ get_supported_ivhd_type(struct acpi_tabl { size_t length = sizeof(struct acpi_table_ivrs); const struct acpi_ivrs_header *ivrs_block, *blk = NULL; + uint8_t checksum; + + /* Validate checksum: Sum of entire table == 0. */ + checksum = acpi_tb_checksum(ACPI_CAST_PTR(uint8_t, table), table->length); + if ( checksum ) + { + AMD_IOMMU_DEBUG("IVRS Error: Invalid Checksum %#x\n", checksum); + return -ENODEV; + } while ( table->length > (length + sizeof(*ivrs_block)) ) { --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -1398,15 +1398,9 @@ int __init amd_iommu_prepare(bool xt) goto error_out; /* Have we been here before? */ - if ( ivhd_type ) + if ( ivrs_bdf_entries ) return 0; - rc = amd_iommu_get_supported_ivhd_type(); - if ( rc < 0 ) - goto error_out; - BUG_ON(!rc); - ivhd_type = rc; - rc = amd_iommu_get_ivrs_dev_entries(); if ( !rc ) rc = -ENODEV; --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -179,9 +179,17 @@ static int __must_check amd_iommu_setup_ int __init acpi_ivrs_init(void) { + int rc; + if ( !iommu_enable && !iommu_intremap ) return 0; + rc = amd_iommu_get_supported_ivhd_type(); + if ( rc < 0 ) + return rc; + BUG_ON(!rc); + ivhd_type = rc; + if ( (amd_iommu_detect_acpi() !=0) || (iommu_found() == 0) ) { iommu_intremap = iommu_intremap_off; From patchwork Wed Sep 22 14:37:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12510811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66CA2C433EF for ; Wed, 22 Sep 2021 14:37:20 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 073AC611CA for ; Wed, 22 Sep 2021 14:37:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 073AC611CA Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.192865.343569 (Exim 4.92) (envelope-from ) id 1mT3N2-0002Y0-5t; Wed, 22 Sep 2021 14:37:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 192865.343569; Wed, 22 Sep 2021 14:37:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3N2-0002Xt-2q; Wed, 22 Sep 2021 14:37:12 +0000 Received: by outflank-mailman (input) for mailman id 192865; Wed, 22 Sep 2021 14:37:10 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3N0-0002Vd-2a for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:37:10 +0000 Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id f7eb2009-6179-48fb-8483-cfefde1d690f; Wed, 22 Sep 2021 14:37:08 +0000 (UTC) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-7-6qAaoySUPVKlzxJtqDs2lw-1; Wed, 22 Sep 2021 16:37:06 +0200 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep 2021 14:37:05 +0000 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021 14:37:05 +0000 Received: from [10.156.60.236] (37.24.206.209) by FR3P281CA0026.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend Transport; Wed, 22 Sep 2021 14:37:04 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f7eb2009-6179-48fb-8483-cfefde1d690f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1632321427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+/R2l8vt0G8tNtmvPjl7fZ9gXSFTbwJmfLkNFVNLpmQ=; b=UhWUHBgxNCSleBHMj3V20uOMsPZX+2Yg03bHwtvIQCaGgbhiqxSYgP6nKOQxdBnSg76wlh 2WX3i86FjqAVCnlXacwsTQjmKYcHhIEBAc2gPBEhoGBxzHLGJoJT2la5pDuhaUTuUrD6U9 v8XNA+35bY1IE39DuErdfyU320jFEk4= X-MC-Unique: 6qAaoySUPVKlzxJtqDs2lw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z66vj0047mTU5EESP4sULXQ1MS0tNDiLIZ+//m6n5PWRgvUIG+Q1WUaQewKNnIYaAnEyDx38+oUvm9sdslKOnqnMxjJYcfDol7VuH3bIEHJfxfhTihgw0dUD+YKuZ8ckXd7h+GcmE+vA9GYfmot4E3sjexdU0ECBACudaCG3oXY/2HocYmsxNnrkxImZJ0RNNKl1iCrI7EIeoD75q2jorDXEG4/Hm6xsRYBarL80Qs145D0Q4XAlDn5D4eTTuDq4Kb9/uYNkjMnu2T28O/Xk/xqQgK14zdMZKVltJBzsz9OUh9dsqr4Y2318/hFi0JDbxWtuYX+kVy4g1vJMvt8tgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+/R2l8vt0G8tNtmvPjl7fZ9gXSFTbwJmfLkNFVNLpmQ=; b=V/CEU36JtINRpARvKMBYbVRo7osmTGzMeBCn7recrtS0e/9sLuPQyo5MFibi9mNEDY7OyP2/9xWwYcP3+N0ZIRBV+QC771gf8t9ctXW1QpRGmP3NC6Y5y1G1rOSGRCmHzVVA5zWS+RnL1GKye5zPFzWnEm8h2HoXxmtmDsrHHmlHQgc31ljHbGggsf+D4AzNOB0UzDOrix77/qJUZvwWUDa11uFijvYKg5mpHcqYxPOT+ZDwWBnhBdzqsXl7w5mtxZqNQY//hH4HCuf9KGTo5taUT3jFmdMVrtw+Cw4I4U6Qkv9nDugo+9BIL9Z0fTMk/OpcwzA7gFfPLY/pmIMINA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=suse.com; Subject: [PATCH v8 2/6] AMD/IOMMU: improve (extended) feature detection From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant References: Message-ID: <4e2034ee-a074-a321-ccbb-9e36c74f0060@suse.com> Date: Wed, 22 Sep 2021 16:37:04 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Language: en-US X-ClientProxiedBy: FR3P281CA0026.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::17) To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c0f66931-a73b-4102-a62e-08d97dd672c7 X-MS-TrafficTypeDiagnostic: VI1PR04MB7024: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Rg/HDUgGphnXEsx061sn1RpxYlO11XJv14xpB6aKFSr7OzqtW5PG49fKEU+OZAdIUWv/QCFspjiBOG2nVhtd4aL0Kt5vmQdV2KW1EZullR5ThDUOlJET1/FaTXUMjhNUdxp3DoTExtxeWW2LA4qTekl4x5TGEhNganNQCngIXb2SszuOtH77AeaMyrZtzUXK7Bqu7VFwDlJHLtNYavhLvQGYfJ4EoPMam75DuqFuVNifGkhGBnMMLowxZG1HI4qgxGEERJlUiaqH7xtXQNf+M8+/q0JQAbTp3om2uDfhTOPzQ1BAjsta+HKbtpGw7kEcSkbokKcvmOmG3GL5BB97E5B0kl02bFE0PTUOb3OqrHlhW3gYEbaV30O+nIS2oPSHbl+Q8OpV2daNnQ0b+7YHKR7M75RU0zdDHSOkBwmSmHy+M2tJwOL7RkenqdIoVkyTsDMKh0qmvTwr50F1AgtpopwAmUeHQ41RN8qRatb5d3s7MBPhIhAUsOR89LZGA1yTn3WxboUeiWB9s/JdcYbBurXIPNCEaVnZ0lTmIGLEBMNPa4S/SqvLQ6Bw3f2L0Ky5bj71nQ/oxB0e+rS0/WBlj4bq1e//QMFK4ogPBuBSNQ5A/SVnEKvNJnPAgdUBhuaO19yEyT0IA5N7epWrzvAXECsUFFmlDeidCmWdvtc/Tkz1fEjJx+TMmD93fzi1t6Vh+xhnt5KFAKvGJBIjuwYoF0LMUUAAYSv4VDONEmUA/yY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?419jzr2EHPYuEeqDobvS4HxH1Q3i?= =?utf-8?q?cQetxE34U7Hd+eJdyWzNnWl3K57g+7Pfxxvh1Fnc2+hGHHtdxMhh0XJD/jKm/hl7t?= =?utf-8?q?7ZauTd03T2cCpZ/Ci/HeASDQ/6t7WbE2VGhClHqmiXh3DnpuOs1k/LLd55LiRNOT3?= =?utf-8?q?JYofQ10pMkXn23YxgzbSzt648wm5PGRRUuA1AIxzSyCQgsqvdrKlqiF331uSFrSdh?= =?utf-8?q?pbH3n8lbrG2HZxY2Mop27W0wJtchx/2Zt02iFddMPJOTLWRlNUkZUJBCRDdIm6GU9?= =?utf-8?q?eA9AtdS36zZZ5KLHyMfyV/VOy1oPY4JgByAW7hYR6TubmWzyNLBfjxlmBmVn9Hlpf?= =?utf-8?q?KSQTdyH2IM0LqLdE2AJqt+0NmAQ5Cq/mYKlZAhTySzO+Fe9BcD/GTKQ7OfBt2i/+G?= =?utf-8?q?Gsxpyc/z5FlfGC+tZGVg+5omm7LWVxRDnbdAjm9WIx+dENAjEQjeY1J+PaSVunReh?= =?utf-8?q?nuvGizvD/gi5kdU+iJjr+JVLYvZIzRgIIcKvzLeZvkzo9FS7phP9XsywN7mmUd6oG?= =?utf-8?q?rNPMzGaN2c751F5eFj8EJZlW1h9+FgGa4g9uCW+XDm9MuW1AqtqLinJ6mui8+U2fF?= =?utf-8?q?l3k1ysebuOnAwn+K9/2xPnmabj0fulAt25COEyw0Cwm84urBtLCnCQUXJLozJvPlQ?= =?utf-8?q?oXr/fDz0Z9DQMalg+fsAIMn4BNfi3QFHsFklyQVsvHTjHM/LXAkbTIZapl9XjQBzq?= =?utf-8?q?fkHRVIoMLJtGPbHPLWQBjZO14HUZQzfmFjO9V9VXb0LHITG8CBhe8qrOfvkUj23cd?= =?utf-8?q?myLSPOsdNL8b80VYan/NiAEmM91vty+ZttRcrVwet9jxxgpObBtmYw/s1rgAmCCb1?= =?utf-8?q?9IGak9UBmZ9dyw+1wB6g6kRfkL1iFhqnF8HcVOO8VLQKqOuyx+0dbat0iM6vPVs7k?= =?utf-8?q?I6R2FoZb+tt/KbiONfqSszVrQ+YSCpTM+RYNyz6B/DL5927hxiG6aYMttOlERME/R?= =?utf-8?q?A8EM4C+3tO6icPuMRJIEHvtwHv1EtpeMjZCEhRzZEKGP1uaoee7Mm4bOSfg5bEowA?= =?utf-8?q?oMPzqt2VOuA+rfNPblPZhdlMhBfgGMjqHaKA3SbqbgaDLADovCiiba3OgbYum1QJh?= =?utf-8?q?AdFzxuDYhHRScDVXuR8fSGKiu1wXN0d3BGuM65JIuWOrQx5RnfGz/w79xxbnrynE0?= =?utf-8?q?xow6PlG5v56/dpMVFg+UFeL6E+bmd67/jsi1+Uwp6Y0E0+NHxHNXq9bETSgWKEWPz?= =?utf-8?q?07WZ5G8B7smm1aWGhb7qEAYFhcpFW9EFVY4EiC5YN8wNtEcbLD4ZggM01PucIgeJC?= =?utf-8?q?SxAPuPzsq8mXsU/w?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: c0f66931-a73b-4102-a62e-08d97dd672c7 X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:37:05.2835 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KoeSJrzDEo4+5cvD0xN2/E8RnEJXjqVKUUuIl3cptCLyi5dKnjhy/sQdYijLMSbBVKpoKsTR5Sc4CUA/y0inIw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024 First of all the documentation is very clear about ACPI table data superseding raw register data. Use raw register data only if EFRSup is clear in the ACPI tables (which may still go too far). Additionally if this flag is clear, the IVRS type 11H table is reserved and hence may not be recognized. Furthermore propagate IVRS type 10H data into the feature flags recorded, as the full extended features field is available in type 11H only. Note that this also makes necessary to stop the bad practice of us finding a type 11H IVHD entry, but still processing the type 10H one in detect_iommu_acpi()'s invocation of amd_iommu_detect_one_acpi(). Note also that the features.raw check in amd_iommu_prepare_one() needs replacing, now that the field can also be populated by different means. Key IOMMUv2 availability off of IVHD type not being 10H, and then move it a function layer up, so that it would be set only once all IOMMUs have been successfully prepared. Signed-off-by: Jan Beulich Reviewed-by: Paul Durrant --- v7: Re-base. v5: New. --- a/xen/drivers/passthrough/amd/iommu.h +++ b/xen/drivers/passthrough/amd/iommu.h @@ -306,6 +306,7 @@ extern struct hpet_sbdf { } init; } hpet_sbdf; +extern unsigned int amd_iommu_acpi_info; extern int amd_iommu_min_paging_mode; extern void *shared_intremap_table; --- a/xen/drivers/passthrough/amd/iommu_acpi.c +++ b/xen/drivers/passthrough/amd/iommu_acpi.c @@ -1062,7 +1062,8 @@ static unsigned int __initdata nr_ivmd; static inline bool_t is_ivhd_block(u8 type) { return (type == ACPI_IVRS_TYPE_HARDWARE || - type == ACPI_IVRS_TYPE_HARDWARE_11H); + ((amd_iommu_acpi_info & ACPI_IVRS_EFR_SUP) && + type == ACPI_IVRS_TYPE_HARDWARE_11H)); } static inline bool_t is_ivmd_block(u8 type) @@ -1176,7 +1177,7 @@ static int __init detect_iommu_acpi(stru ivrs_block = (struct acpi_ivrs_header *)((u8 *)table + length); if ( table->length < (length + ivrs_block->length) ) return -ENODEV; - if ( ivrs_block->type == ACPI_IVRS_TYPE_HARDWARE && + if ( ivrs_block->type == ivhd_type && amd_iommu_detect_one_acpi(to_ivhd_block(ivrs_block)) != 0 ) return -ENODEV; length += ivrs_block->length; @@ -1316,6 +1317,9 @@ get_supported_ivhd_type(struct acpi_tabl return -ENODEV; } + amd_iommu_acpi_info = container_of(table, const struct acpi_table_ivrs, + header)->info; + while ( table->length > (length + sizeof(*ivrs_block)) ) { ivrs_block = (struct acpi_ivrs_header *)((u8 *)table + length); --- a/xen/drivers/passthrough/amd/iommu_detect.c +++ b/xen/drivers/passthrough/amd/iommu_detect.c @@ -60,14 +60,14 @@ void __init get_iommu_features(struct am const struct amd_iommu *first; ASSERT( iommu->mmio_base ); - if ( !iommu_has_cap(iommu, PCI_CAP_EFRSUP_SHIFT) ) + if ( !(amd_iommu_acpi_info & ACPI_IVRS_EFR_SUP) ) { - iommu->features.raw = 0; - return; - } + if ( !iommu_has_cap(iommu, PCI_CAP_EFRSUP_SHIFT) ) + return; - iommu->features.raw = - readq(iommu->mmio_base + IOMMU_EXT_FEATURE_MMIO_OFFSET); + iommu->features.raw = + readq(iommu->mmio_base + IOMMU_EXT_FEATURE_MMIO_OFFSET); + } /* Don't log the same set of features over and over. */ first = list_first_entry(&amd_iommu_head, struct amd_iommu, list); @@ -164,6 +164,42 @@ int __init amd_iommu_detect_one_acpi( iommu->cap_offset = ivhd_block->capability_offset; iommu->mmio_base_phys = ivhd_block->base_address; + if ( ivhd_type != ACPI_IVRS_TYPE_HARDWARE ) + iommu->features.raw = ivhd_block->efr_image; + else if ( amd_iommu_acpi_info & ACPI_IVRS_EFR_SUP ) + { + union { + uint32_t raw; + struct { + unsigned int xt_sup:1; + unsigned int nx_sup:1; + unsigned int gt_sup:1; + unsigned int glx_sup:2; + unsigned int ia_sup:1; + unsigned int ga_sup:1; + unsigned int he_sup:1; + unsigned int pas_max:5; + unsigned int pn_counters:4; + unsigned int pn_banks:6; + unsigned int msi_num_ppr:5; + unsigned int gats:2; + unsigned int hats:2; + }; + } attr = { .raw = ivhd_block->iommu_attr }; + + iommu->features.flds.xt_sup = attr.xt_sup; + iommu->features.flds.nx_sup = attr.nx_sup; + iommu->features.flds.gt_sup = attr.gt_sup; + iommu->features.flds.glx_sup = attr.glx_sup; + iommu->features.flds.ia_sup = attr.ia_sup; + iommu->features.flds.ga_sup = attr.ga_sup; + iommu->features.flds.pas_max = attr.pas_max; + iommu->features.flds.gats = attr.gats; + iommu->features.flds.hats = attr.hats; + } + else if ( list_empty(&amd_iommu_head) ) + AMD_IOMMU_DEBUG("EFRSup not set in ACPI table; will fall back to hardware\n"); + /* override IOMMU HT flags */ iommu->ht_flags = ivhd_block->header.flags; --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -29,6 +29,7 @@ static bool __initdata pci_init; static void do_amd_iommu_irq(void *data); static DECLARE_SOFTIRQ_TASKLET(amd_iommu_irq_tasklet, do_amd_iommu_irq, NULL); +unsigned int __read_mostly amd_iommu_acpi_info; unsigned int __read_mostly ivrs_bdf_entries; u8 __read_mostly ivhd_type; static struct radix_tree_root ivrs_maps; @@ -1375,9 +1376,6 @@ static int __init amd_iommu_prepare_one( get_iommu_features(iommu); - if ( iommu->features.raw ) - iommuv2_enabled = true; - return 0; } @@ -1419,6 +1417,9 @@ int __init amd_iommu_prepare(bool xt) has_xt = false; } + if ( ivhd_type != ACPI_IVRS_TYPE_HARDWARE ) + iommuv2_enabled = true; + for_each_amd_iommu ( iommu ) { /* NB: There's no need to actually write these out right here. */ --- a/xen/include/acpi/actbl2.h +++ b/xen/include/acpi/actbl2.h @@ -716,6 +716,9 @@ struct acpi_table_ivrs { /* Values for Info field above */ +#define ACPI_IVRS_EFR_SUP 0x00000001 /* extended feature support */ +#define ACPI_IVRS_PREBOOT_DMA_REMAP 0x00000002 /* pre-boot DMA remapping in use */ +#define ACPI_IVRS_GVA_SIZE 0x000000E0 /* 3 bits, guest VA size */ #define ACPI_IVRS_PHYSICAL_SIZE 0x00007F00 /* 7 bits, physical address size */ #define ACPI_IVRS_VIRTUAL_SIZE 0x003F8000 /* 7 bits, virtual address size */ #define ACPI_IVRS_ATS_RESERVED 0x00400000 /* ATS address translation range reserved */ From patchwork Wed Sep 22 14:37:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12510813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F290C433EF for ; Wed, 22 Sep 2021 14:37:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BEA58611C9 for ; Wed, 22 Sep 2021 14:37:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BEA58611C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.192870.343580 (Exim 4.92) (envelope-from ) id 1mT3NN-00039m-JX; Wed, 22 Sep 2021 14:37:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 192870.343580; Wed, 22 Sep 2021 14:37:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3NN-00039d-GT; Wed, 22 Sep 2021 14:37:33 +0000 Received: by outflank-mailman (input) for mailman id 192870; Wed, 22 Sep 2021 14:37:33 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3NN-00039P-3K for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:37:33 +0000 Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 9e2aad00-1bb2-11ec-b99e-12813bfff9fa; Wed, 22 Sep 2021 14:37:32 +0000 (UTC) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-26-XzJ_7qFmMF64tdWfjv1idA-1; Wed, 22 Sep 2021 16:37:30 +0200 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep 2021 14:37:29 +0000 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021 14:37:28 +0000 Received: from [10.156.60.236] (37.24.206.209) by PR2PR09CA0011.eurprd09.prod.outlook.com (2603:10a6:101:16::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:37:28 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9e2aad00-1bb2-11ec-b99e-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1632321451; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=73rBnD5E2FMoCGn+vOvsFjvD7EEVJfzmIbJDfM5L9dI=; b=M74c+d+aHzoTjuu9rAlta0Gd22luR8rqINPmPtk11DBXP5VlXyqcxwc/nBi0lO36VXox8c NPussEIkRJKZshbt4JGz0sCJkLY0XBjR3SmVQZTZvMkSIWZuqP5erXNnm35+37iUi8Mcn1 oR3EyE+x1EZVMePAAa1Lp00UPZKd4iQ= X-MC-Unique: XzJ_7qFmMF64tdWfjv1idA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n9ZvOcIjJjphRQ9dBFZK6l11mtyJRXLh1VrsY/ZDs9Bx87l4imURxLSYFy2zqJE7o46KqdXT/Kdp4jvwM3/BIyKUgn6JgKDgzaZ6yJWAsqGTtVY16hXGm6GWnfCIuABUMf8hTyPz5VlYKVDhBTOWZqPoN6di9PMsaGyq2hDrwObt7LOqji6OZzTwivTVnjWaqR/TDDJZUsiVlBR2dtEyMZouI2ay9GUraTB7jTsb+i8EplpGrVnK8i1emPQhVJ+RU2Jzf5wCNdNktfZn9LRgr5zzzR6WEZI23JGoekh9stE7RkPPBt2Miqmk5oILnuu4xqn2uRdIHTGFcKIPrN3f/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=73rBnD5E2FMoCGn+vOvsFjvD7EEVJfzmIbJDfM5L9dI=; b=QGaMDTiRfSVZNMsJDnAGFVAhQ4nZMkUZUDPpL63U4Z6x6uYsguq5U7h7lsgBqt3UBjp4zS2xlCAXwhN/fzht1bbqW5lVObvDnb++PZRJYpbwyfE/+Oju3Rc46+FpbanCWp+TVcck+GG16aZGxU3GR9t2qyIAw7fYFUOPqSzuTo9d6pOzhDyYSALyvzgVu9bl1FQZgqoHAMihofnQeyp7UfkXBdoWJB2rO5liAoh60MEN7Gc+NgTKcmkSr9DfT4ejtigqOAkq1CRkiyYiQXKzRsErwL+RPPwv6jOiu5ZOWaYm5N1bxxOZHApfq5mdDPsrKAbXergXT5Gpyp6OSX1TEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=suse.com; Subject: [PATCH v8 3/6] AMD/IOMMU: check IVMD ranges against host implementation limits From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant References: Message-ID: <4ab0e244-e5a3-4b76-cef4-01d43345369c@suse.com> Date: Wed, 22 Sep 2021 16:37:28 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Language: en-US X-ClientProxiedBy: PR2PR09CA0011.eurprd09.prod.outlook.com (2603:10a6:101:16::23) To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: db108d3a-2307-40d0-fa88-08d97dd680d6 X-MS-TrafficTypeDiagnostic: VI1PR04MB7024: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3044; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 19KpKtvyQpYV+LoHCW8lM0qI0MvTAxYiUf9T4jzeUgyjHZQ9AC2O1k0FVMbAFeTjDk9iEqrw03vEdxELOHTdclvn3bvvjrMJFMd6JLdYiIiigQ2pCuT/4uuAK9fAhNbFncvyjaAMVjR6NU9Qa2CrRLQJp+47Rdm6WGyvk+quYMErmoSh+wGolHwxxMTZliJlC7unAvafiOko08LZdsQk20zp82Kqrpi7dIOjdHr2JboBqiqeziE9agVntvkBatexAwkd74KWPpDD1fW3jo9tsMcQE6zLxd2iNe3nMK5oHp1+GYKna1GI7fcATQIGOqX4Y+/O3aUc7y2N18F+c9Gl1mmu0AJHDNYcBaLNmXb/D0VTwLS18fuE72gh/6RHTUC9sKFhL68H26HfIqtuQB+FCxf/IIpJNg0g+JNupyfbrppLTBQVBaJebHv9ePnJqrJvl6N09OenHozaVrHMZOEBlDj+DmRtizKZTfYEhbiayqOaZajrOqeQZduyp5+f3G6U2FXiuD21JQj/lXP8QTAeXK4nHD9r4dnmQVJTXww/0Df+2PXc/2R6PiWSiX2vpTWd9udwg4D1nN4mlT1djO4bW2XNADxYAFun9LqxjhqWR3BeHubn1Y8e+2sh7juSl/qchgQlKeiFBdcSOiYa/rLjxval0nQkefkSwspGCFUE6O2H3vaTPy6h0LciahyPVyPtPtVkwnGO5d3EpyE/jefVeuuiXtL7md5IjZCIA75gaz0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?fMOK7XBSmmx5b3RRXihEk2fwB7ez?= =?utf-8?q?ZeYLWvc8X56gvTDFLW3A9lqZg52ADmCmPJtgx7GPp/nG9ACI9A50v0M5dDGvQ/ahl?= =?utf-8?q?iIesuCXVbrK974goMjy10Xl9/8Moa2MPbAr97/XzN5WLT0NZuP2azErqNpOYunfwa?= =?utf-8?q?8EBYGgIcjZ5vVSahxGgiip+rd+1MHiWjTNe/0npwlLQqACwn9hawYK8dWcOLmbTCt?= =?utf-8?q?1tvieSQw6mO5eROT5VXd/ssyhXuX6QamiKNVl7NfToqQiAYle+fKNu1SJGM1jPYUQ?= =?utf-8?q?X3Hadi8JRlCULZmTjhqpWnWxaiygIn2vg5hR7N08AhBhtubtmAsR12/QI9KpW7F+r?= =?utf-8?q?WFLksfpymiT9l37jjBWqbeKEaNvcrbrcu8NvkSI+xBFUh46QqMtKBUsrxHrukIj+9?= =?utf-8?q?eI4GMTI4it8vBY1VKMczSU75l64nq9ZQLWg1qKB+3uqGReDCFLYk91c4/DCIyCAQQ?= =?utf-8?q?PWDsfgbL6ykfkvNBrScpQ85ALgDU6lPozIEp9rMN36j7EJ0DDm4UXIQCZXemLqu0o?= =?utf-8?q?gReGe3ubwDQWrqPBJXNWR5cBQ8KyoWKd3jCeh4y7lnATlu0KNMaQHFkklt8ZgtpXT?= =?utf-8?q?v/CHi5WsoQ4O0D2JZsO71c6khMxgb7whi+/+R2Cw0s5TX3glWgDeFRxGwnp1OQdve?= =?utf-8?q?DioS+AdVPLa5H83zj7N79DNqEldtlGQaDdHdkozX+unxfdxaCPCXxW2WggZ2WLg8t?= =?utf-8?q?vLMmuCry/WszCGkB483vmufMzDBQCv3WlWpabg/m/+5O23r+Dv5mzpnOLizuBJLE0?= =?utf-8?q?2RZz+N987zfGFR87k3qCUcnrPTiKRn7LzsHEEKRqY1arj0SRZ09RW2cl2AuunbkrV?= =?utf-8?q?3I3PWjS+lYXlRe45ybjp3lz989GORjPXl+foklLpc+XRSwMjIeR3liTTRPSk2Vk1u?= =?utf-8?q?Iakl2IdJyWsiz/Udv6mwZLCkjP04HnbPggjdWPTqo5qQGhJL6GmArEIHsbCIyxPgf?= =?utf-8?q?0cW8kMCX4WWvzcD8BioTd+3IR/PA9XHnaR4aginM62vE/S31IicyemCN+UpDFll0Y?= =?utf-8?q?+1f2X6oslVz4OTDq8DUXg61v3RR4iEpZ9Gxta3d2m7Z6+0ido12j/rZ4lTylKkUB6?= =?utf-8?q?6FqslR9IxkVxlfBFPvBWayvXBwFq/OnmL5itni/LI/DosZYVMForq83yC8tvhIVSJ?= =?utf-8?q?RXuyEAylFn7OaUtxqtjMUUIUjZS6yrD08VFI5sC2inE0qVP5GQTNH6pge2KH4sJ8/?= =?utf-8?q?deSZZka1JjFXaE9PhKKPZeGcF2P5b+v2jmfgqjsUT6Hvvej9iQT5YiKRhwfRSUf+X?= =?utf-8?q?baWEEXzmCemUGz0s?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: db108d3a-2307-40d0-fa88-08d97dd680d6 X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:37:28.8281 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XcUrxD8nX1+6P/kugBiZ4pQ8sQIoq2WCdOwQAqg3q2TGGuwYQMXHO542fLbQD8Oax6+DE9zyDJX/3NW5Fifuvg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024 When such ranges can't be represented as 1:1 mappings in page tables, reject them as presumably bogus. Note that when we detect features late (because of EFRSup being clear in the ACPI tables), it would be quite a bit of work to check for (and drop) out of range IVMD ranges, so IOMMU initialization gets failed in this case instead. Signed-off-by: Jan Beulich Reviewed-by: Paul Durrant --- v7: Re-base. v6: Re-base. v5: New. --- a/xen/drivers/passthrough/amd/iommu.h +++ b/xen/drivers/passthrough/amd/iommu.h @@ -307,6 +307,7 @@ extern struct hpet_sbdf { } hpet_sbdf; extern unsigned int amd_iommu_acpi_info; +extern unsigned int amd_iommu_max_paging_mode; extern int amd_iommu_min_paging_mode; extern void *shared_intremap_table; @@ -360,7 +361,7 @@ static inline int amd_iommu_get_paging_m while ( max_frames > PTE_PER_TABLE_SIZE ) { max_frames = PTE_PER_TABLE_ALIGN(max_frames) >> PTE_PER_TABLE_SHIFT; - if ( ++level > 6 ) + if ( ++level > amd_iommu_max_paging_mode ) return -ENOMEM; } --- a/xen/drivers/passthrough/amd/iommu_acpi.c +++ b/xen/drivers/passthrough/amd/iommu_acpi.c @@ -378,6 +378,7 @@ static int __init parse_ivmd_device_iomm static int __init parse_ivmd_block(const struct acpi_ivrs_memory *ivmd_block) { unsigned long start_addr, mem_length, base, limit; + unsigned int addr_bits; bool iw = true, ir = true, exclusion = false; if ( ivmd_block->header.length < sizeof(*ivmd_block) ) @@ -394,6 +395,17 @@ static int __init parse_ivmd_block(const AMD_IOMMU_DEBUG("IVMD Block: type %#x phys %#lx len %#lx\n", ivmd_block->header.type, start_addr, mem_length); + addr_bits = min(MASK_EXTR(amd_iommu_acpi_info, ACPI_IVRS_PHYSICAL_SIZE), + MASK_EXTR(amd_iommu_acpi_info, ACPI_IVRS_VIRTUAL_SIZE)); + if ( amd_iommu_get_paging_mode(PFN_UP(start_addr + mem_length)) < 0 || + (addr_bits < BITS_PER_LONG && + ((start_addr + mem_length - 1) >> addr_bits)) ) + { + AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not IOMMU addressable\n", + start_addr, start_addr + mem_length); + return 0; + } + if ( !e820_all_mapped(base, limit + PAGE_SIZE, E820_RESERVED) ) { paddr_t addr; --- a/xen/drivers/passthrough/amd/iommu_detect.c +++ b/xen/drivers/passthrough/amd/iommu_detect.c @@ -67,6 +67,9 @@ void __init get_iommu_features(struct am iommu->features.raw = readq(iommu->mmio_base + IOMMU_EXT_FEATURE_MMIO_OFFSET); + + if ( 4 + iommu->features.flds.hats < amd_iommu_max_paging_mode ) + amd_iommu_max_paging_mode = 4 + iommu->features.flds.hats; } /* Don't log the same set of features over and over. */ @@ -200,6 +203,10 @@ int __init amd_iommu_detect_one_acpi( else if ( list_empty(&amd_iommu_head) ) AMD_IOMMU_DEBUG("EFRSup not set in ACPI table; will fall back to hardware\n"); + if ( (amd_iommu_acpi_info & ACPI_IVRS_EFR_SUP) && + 4 + iommu->features.flds.hats < amd_iommu_max_paging_mode ) + amd_iommu_max_paging_mode = 4 + iommu->features.flds.hats; + /* override IOMMU HT flags */ iommu->ht_flags = ivhd_block->header.flags; --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -1376,6 +1376,13 @@ static int __init amd_iommu_prepare_one( get_iommu_features(iommu); + /* + * Late extended feature determination may cause previously mappable + * IVMD ranges to become unmappable. + */ + if ( amd_iommu_max_paging_mode < amd_iommu_min_paging_mode ) + return -ERANGE; + return 0; } --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -254,6 +254,7 @@ int amd_iommu_alloc_root(struct domain * return 0; } +unsigned int __read_mostly amd_iommu_max_paging_mode = 6; int __read_mostly amd_iommu_min_paging_mode = 1; static int amd_iommu_domain_init(struct domain *d) From patchwork Wed Sep 22 14:37:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12510815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C801C433F5 for ; Wed, 22 Sep 2021 14:38:06 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4453F60F26 for ; Wed, 22 Sep 2021 14:38:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4453F60F26 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.192878.343591 (Exim 4.92) (envelope-from ) id 1mT3Nj-0003nw-Sv; Wed, 22 Sep 2021 14:37:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 192878.343591; Wed, 22 Sep 2021 14:37:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3Nj-0003np-Pe; Wed, 22 Sep 2021 14:37:55 +0000 Received: by outflank-mailman (input) for mailman id 192878; Wed, 22 Sep 2021 14:37:54 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3Ni-0003nN-Lg for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:37:54 +0000 Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 6624fd67-275b-423e-b278-278fd68fde3a; Wed, 22 Sep 2021 14:37:53 +0000 (UTC) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2056.outbound.protection.outlook.com [104.47.12.56]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-5-HiJxDSOZNc2ADWDudpIfFg-1; Wed, 22 Sep 2021 16:37:51 +0200 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep 2021 14:37:49 +0000 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021 14:37:49 +0000 Received: from [10.156.60.236] (37.24.206.209) by AM3PR05CA0108.eurprd05.prod.outlook.com (2603:10a6:207:1::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:37:49 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6624fd67-275b-423e-b278-278fd68fde3a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1632321472; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J6zzTKAR8SOShvgrracy6yF82HzYmfEYILkFc9My6/I=; b=hEcNw9dxif5adIMTJ5Artx65HkYIYGNPKGpCktrd+Jrexk8BtwXMs61faakJA9wa3Gm6ED l1jnHyZ7eZuzy1umNAkvKfAki0uGz0YL0YKXH72xEVPJawuw7zLFb23k6MbLsJPlMG4X6J Ycu6NExkdiIu8h/Dt8GW3kNA6gDiaew= X-MC-Unique: HiJxDSOZNc2ADWDudpIfFg-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Rm0mX7OpBYkhrujWIydQ9pQhVMCd0bQ6IYFoUvrIIvy3T+zX+yNmhntYkuxM6Qzg2SlCfLUdAy6XGkPbHmZVa1ggzFiCtRp2ZZkO/RIOTrQFKcXMqBCDaTdpc9ZTiQ2cWnh2LH6URP1kF942qawtLX3iq4ObEMI0D+SOMOpsmT0Wn2GiYsaKYHHtKKfIZyeRUz+QrlGdCfVvFGOQjhmdIVuW/eckpEY2gMAa7ybf5euvAz/jhcpBxpzyPdqfhEizyVW+tuZ0oZtb4cgYuva1pSGIui6coznSYqRGFHyz7tfm625oL6GfkTyw9cnSn2oijoRzP96EaLcGAk0hjCKY6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=J6zzTKAR8SOShvgrracy6yF82HzYmfEYILkFc9My6/I=; b=Mv1WVTtb+Lyn5safSIFGTcTMCh7KECT1ChcIpqsgIcWdxOfpYeNCs5IWi73Gi9hNmpqhVHTlBG+xOQsyuEbRyaP+lym6XZEVvAlRc+rjYdClq5w2F0TT0tIi8gHo4MXA4/B4UQGGONZKCr+pskFxGpcEwJG/0ykbd1KGK/hlIVVmpsVUgmmcFDqjqCSV59chcia85dKrMBAIc5u5vZibKqZHbnJ1zXVFalYcan9OuU+K4hF8wYLAl6GFecs4sM2ulVtjjmn3yzmJPTqi97SHM0zbYRnMH51yMDA6tWKxeofkhz1c5CJza1RmuE8AkLAPrtU9KtVXqvhK5PMOmmeAbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=suse.com; Subject: [PATCH v8 4/6] AMD/IOMMU: respect AtsDisabled device flag From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant References: Message-ID: <14d1c1e1-d2d0-d041-e206-608e876d43a0@suse.com> Date: Wed, 22 Sep 2021 16:37:49 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Language: en-US X-ClientProxiedBy: AM3PR05CA0108.eurprd05.prod.outlook.com (2603:10a6:207:1::34) To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 27edfd07-73c3-4b02-7129-08d97dd68d34 X-MS-TrafficTypeDiagnostic: VI1PR04MB7024: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dVysFPWQSZjjxAhHRCon4xmzgG/jsUxjqvQk28UpbmDuAQf3Sb2rE9JMlu94j4LooCGY2nbga7REy93SxRZwNMkAdL0dkjOplQrfCk5FRqg25QgYkKXwo7G5T6DnVhxTZDgBz5MCjmYHcvGbZtMol1Zt0cG1wPy177YbrM8zi8YyKBy/pd9EibY58IKulSDKuxliPHo9VcpDXuyn/bwRbgApBMx8Z8MtI/g8whoYQGTvfcwJd84av1+Q4MilmO0PJ+wjckWeG06x6mvg73AvB6bBSGVTnvwHzCx6KdMaUew06qUVl3YiHrQCNIbM59dpNKeqOx/QEEg3C8uVyOT3Xr2eK0KkJWsYLPMKhghnCYaYj/cPSOh8mTVKAYC0jNkZYJm48X6OTCOGb74S5hxzo8ugRZ8VL7gMZfeUmSduZ3Nd0XpoVix4/35tyojMEp2tzq0KoAL9x7SNWWgT9/fFUBuPpzXq6M2Ir+s//AFWXnCZpdD8Ea0x4U06A3H066ooLG0jRTHOUDYhpWaOHpW9P42T86wq38f3xyPJJrUvbz06qwPAyzGt3/7ncD5BpmwqszBTloIc6OYaHEu0JwSLangk3llf8fDY1jcqWgEMPqBn/fVFIBVhvSGcnXrCwPIJ5yA1ET6LtTYXPO4BtZVPoe/1c5+zEH8DJid8iswF7qEmfJOWDUrQF4TZRv7D1t/5S9ETWrvHXgmnD8QQH566pUQTuMXqU6WAiktvpHsfKws= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?OntrebJob+emoPDOO4fS1Yn/2KE6?= =?utf-8?q?32zCU/shOYneyxRld5Js1zPn8SIK647SR7uHUiwSMGgmIX9Jnyuey3+KFugPc5VM+?= =?utf-8?q?VZq45UF11peolKus4wk+16bCbYDdd8hlV+7l1LAOiX0kVE9Y2DfMn4zTfPD5sGlp2?= =?utf-8?q?sVkGoh4B9+OSDzS+/mpolIQMtJ9aRsslIhR69A6JqCxbf1yug/1vLMuYiRF7aXxQT?= =?utf-8?q?eDFdcBpS62LiM2o4udBOyrEhNsmIzIPRrM8IW5WqsoCjeUGd8XAQhwRyk4iuQLNpN?= =?utf-8?q?S07SHTCjvUEP/KzV6ukHPlyLR0wFsSwFXXDa6dtgcTj5pzulA1kLsQyUgmHhekEuI?= =?utf-8?q?JpzpPmXNxg+GhUPcrGP8jja4Rj6AUPGb1bNcvyNOPIu4sKO+qWeM/SApHflPhKh++?= =?utf-8?q?Ao1SZiEk3s46/Jpv/cxc1Wfpx9o32elVJ/AubSRIxw4Ill5KXJHc8TOvFHnEf1ffV?= =?utf-8?q?GfOqQCVJD1H3pTSqUjUWQmQIfWXRg6DrZnsPpDu0fjnHi7GSbgE0tc8dvn2ZDIkWz?= =?utf-8?q?tCRKRaDohmSqtSYQBVzxwLKe8ZRoO+xRtOZvY5dNPD2MrVTKD+NqJGquMQAxfSrVw?= =?utf-8?q?mfdTZ6xkuabiVWBmNtHVQjPrpkMm2QqHJ9FeyPhLsddaDWPQbbjsdSwLQK751NPZO?= =?utf-8?q?NAKnijNcgUHu1h5QIStMMpXPAuEVpq+LCQA4NuLrlVCX85gEg0D6CBNQrZqTjV3J4?= =?utf-8?q?Cz3gNmDlv+amvZ45UDe4cTPodCezNPvawMJA2hmKRujItznSADSdfdqKfIGuhXwzU?= =?utf-8?q?LunLkh1XGn2WltL6EOYeIrtDHbt9Hw3V9o9SLRrDlwLNpwAMkgYKteqpVuaAZ+nAQ?= =?utf-8?q?eurVIuiH0KwbJbXq5tvbDfzv6J87roRSqgfOxB8JXiKBKtIdpmhRHzH8EwIOE0D4q?= =?utf-8?q?uVMpowou1ZJ2Ke4SErJc3m3I92WUtjJNq60OFEJaF1MuhnbPde+dh5rYVZzVyMh7W?= =?utf-8?q?1Wnl58gr5x2a8MIEQSr6XzYoOgHUHVwPWScwc7JZYzUgEjSmSJOv5jXXXjU6DRpiP?= =?utf-8?q?mRQkkOmkr7HYkVqFqnLszQrqgDAEK2X6hnsJKwtkK/3u6ZYoBFKscEACkXk68fr4a?= =?utf-8?q?hieG+D03GMBEyCnGC2aRcDsbeVeJTjsmSPfFA4a9ZS+iVxfEGyFGX+ufdG8aJNdwG?= =?utf-8?q?jgnjulFfP8A3S0wjutn707QQFws9CbBNgD2aUxMAeIZZG/FKDOjaDjF2NV17VhFb9?= =?utf-8?q?LnEC3HZE2k7dH4F55YPXccLN7l8a2XLHWl0Ko6ZZSK/BYGJBGBCi8t/GgEF/pxZf0?= =?utf-8?q?dU9h8z4/0VAziSif?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 27edfd07-73c3-4b02-7129-08d97dd68d34 X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:37:49.5913 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: d2fvotjU3HAu+neEK54noldsR/hVwwaZxWxqYqL0PFcoCE4nMS7n6UZFiWxiYLe9jpBqlRrC+Sz3l/5M8QjQTA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024 IVHD entries may specify that ATS is to be blocked for a device or range of devices. Honor firmware telling us so. While adding respective checks I noticed that the 2nd conditional in amd_iommu_setup_domain_device() failed to check the IOMMU's capability. Add the missing part of the condition there, as no good can come from enabling ATS on a device when the IOMMU is not capable of dealing with ATS requests. For actually using ACPI_IVHD_ATS_DISABLED, make its expansion no longer exhibit UB. Signed-off-by: Jan Beulich Reviewed-by: Paul Durrant --- TBD: As an alternative to adding the missing IOMMU capability check, we may want to consider simply using dte->i in the 2nd conditional in amd_iommu_setup_domain_device(). Note that while ATS enabling/disabling gets invoked without any locks held, the two functions should not be possible to race with one another for any individual device (or else we'd be in trouble already, as ATS might then get re-enabled immediately after it was disabled, with the DTE out of sync with this setting). --- v7: New. --- a/xen/drivers/passthrough/amd/iommu.h +++ b/xen/drivers/passthrough/amd/iommu.h @@ -120,6 +120,7 @@ struct ivrs_mappings { uint16_t dte_requestor_id; bool valid:1; bool dte_allow_exclusion:1; + bool block_ats:1; /* ivhd device data settings */ uint8_t device_flags; --- a/xen/drivers/passthrough/amd/iommu_acpi.c +++ b/xen/drivers/passthrough/amd/iommu_acpi.c @@ -55,8 +55,8 @@ union acpi_ivhd_device { }; static void __init add_ivrs_mapping_entry( - uint16_t bdf, uint16_t alias_id, uint8_t flags, bool alloc_irt, - struct amd_iommu *iommu) + uint16_t bdf, uint16_t alias_id, uint8_t flags, unsigned int ext_flags, + bool alloc_irt, struct amd_iommu *iommu) { struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(iommu->seg); @@ -66,6 +66,7 @@ static void __init add_ivrs_mapping_entr ivrs_mappings[bdf].dte_requestor_id = alias_id; /* override flags for range of devices */ + ivrs_mappings[bdf].block_ats = ext_flags & ACPI_IVHD_ATS_DISABLED; ivrs_mappings[bdf].device_flags = flags; /* Don't map an IOMMU by itself. */ @@ -499,7 +500,7 @@ static u16 __init parse_ivhd_device_sele return 0; } - add_ivrs_mapping_entry(bdf, bdf, select->header.data_setting, false, + add_ivrs_mapping_entry(bdf, bdf, select->header.data_setting, 0, false, iommu); return sizeof(*select); @@ -545,7 +546,7 @@ static u16 __init parse_ivhd_device_rang AMD_IOMMU_DEBUG(" Dev_Id Range: %#x -> %#x\n", first_bdf, last_bdf); for ( bdf = first_bdf; bdf <= last_bdf; bdf++ ) - add_ivrs_mapping_entry(bdf, bdf, range->start.header.data_setting, + add_ivrs_mapping_entry(bdf, bdf, range->start.header.data_setting, 0, false, iommu); return dev_length; @@ -580,7 +581,7 @@ static u16 __init parse_ivhd_device_alia AMD_IOMMU_DEBUG(" Dev_Id Alias: %#x\n", alias_id); - add_ivrs_mapping_entry(bdf, alias_id, alias->header.data_setting, true, + add_ivrs_mapping_entry(bdf, alias_id, alias->header.data_setting, 0, true, iommu); return dev_length; @@ -636,7 +637,7 @@ static u16 __init parse_ivhd_device_alia for ( bdf = first_bdf; bdf <= last_bdf; bdf++ ) add_ivrs_mapping_entry(bdf, alias_id, range->alias.header.data_setting, - true, iommu); + 0, true, iommu); return dev_length; } @@ -661,7 +662,8 @@ static u16 __init parse_ivhd_device_exte return 0; } - add_ivrs_mapping_entry(bdf, bdf, ext->header.data_setting, false, iommu); + add_ivrs_mapping_entry(bdf, bdf, ext->header.data_setting, + ext->extended_data, false, iommu); return dev_length; } @@ -708,7 +710,7 @@ static u16 __init parse_ivhd_device_exte for ( bdf = first_bdf; bdf <= last_bdf; bdf++ ) add_ivrs_mapping_entry(bdf, bdf, range->extended.header.data_setting, - false, iommu); + range->extended.extended_data, false, iommu); return dev_length; } @@ -800,7 +802,7 @@ static u16 __init parse_ivhd_device_spec AMD_IOMMU_DEBUG("IVHD Special: %pp variety %#x handle %#x\n", &PCI_SBDF2(seg, bdf), special->variety, special->handle); - add_ivrs_mapping_entry(bdf, bdf, special->header.data_setting, true, + add_ivrs_mapping_entry(bdf, bdf, special->header.data_setting, 0, true, iommu); switch ( special->variety ) --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -105,6 +105,7 @@ static int __must_check amd_iommu_setup_ int req_id, valid = 1, rc; u8 bus = pdev->bus; struct domain_iommu *hd = dom_iommu(domain); + const struct ivrs_mappings *ivrs_dev; if ( QUARANTINE_SKIP(domain) ) return 0; @@ -122,20 +123,18 @@ static int __must_check amd_iommu_setup_ req_id = get_dma_requestor_id(iommu->seg, PCI_BDF2(bus, devfn)); table = iommu->dev_table.buffer; dte = &table[req_id]; + ivrs_dev = &get_ivrs_mappings(iommu->seg)[req_id]; spin_lock_irqsave(&iommu->lock, flags); if ( !dte->v || !dte->tv ) { - const struct ivrs_mappings *ivrs_dev; - /* bind DTE to domain page-tables */ amd_iommu_set_root_page_table( dte, page_to_maddr(hd->arch.amd.root_table), domain->domain_id, hd->arch.amd.paging_mode, valid); /* Undo what amd_iommu_disable_domain_device() may have done. */ - ivrs_dev = &get_ivrs_mappings(iommu->seg)[req_id]; if ( dte->it_root ) { dte->int_ctl = IOMMU_DEV_TABLE_INT_CONTROL_TRANSLATED; @@ -146,6 +145,7 @@ static int __must_check amd_iommu_setup_ dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT); if ( pci_ats_device(iommu->seg, bus, pdev->devfn) && + !ivrs_dev->block_ats && iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) ) dte->i = ats_enabled; @@ -166,6 +166,8 @@ static int __must_check amd_iommu_setup_ ASSERT(pcidevs_locked()); if ( pci_ats_device(iommu->seg, bus, pdev->devfn) && + !ivrs_dev->block_ats && + iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) && !pci_ats_enabled(iommu->seg, bus, pdev->devfn) ) { if ( devfn == pdev->devfn ) --- a/xen/include/acpi/actbl2.h +++ b/xen/include/acpi/actbl2.h @@ -851,7 +851,7 @@ struct acpi_ivrs_device8b { /* Values for extended_data above */ -#define ACPI_IVHD_ATS_DISABLED (1<<31) +#define ACPI_IVHD_ATS_DISABLED (1u << 31) /* Type 72: 8-byte device entry */ From patchwork Wed Sep 22 14:38:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12510817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3CDAC433EF for ; Wed, 22 Sep 2021 14:38:25 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7A4F2611CA for ; Wed, 22 Sep 2021 14:38:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7A4F2611CA Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.192883.343602 (Exim 4.92) (envelope-from ) id 1mT3O5-0004M1-86; Wed, 22 Sep 2021 14:38:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 192883.343602; Wed, 22 Sep 2021 14:38:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3O5-0004Lr-3F; Wed, 22 Sep 2021 14:38:17 +0000 Received: by outflank-mailman (input) for mailman id 192883; Wed, 22 Sep 2021 14:38:15 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3O3-0004LN-L9 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:38:15 +0000 Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a290730d-b82a-4d9c-93cb-14917531d40b; Wed, 22 Sep 2021 14:38:14 +0000 (UTC) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-15-0PX2CnW2O9OHHYZEJFoY4g-1; Wed, 22 Sep 2021 16:38:12 +0200 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep 2021 14:38:11 +0000 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021 14:38:11 +0000 Received: from [10.156.60.236] (37.24.206.209) by AM0PR02CA0171.eurprd02.prod.outlook.com (2603:10a6:20b:28e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:38:10 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a290730d-b82a-4d9c-93cb-14917531d40b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1632321493; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L4OzGEj91QdJC5vdoESTAykpaiWdGj46EwoZ9sMMZn8=; b=eRVJHYuSPX7jhAb0P4sKFMEHSz0L1usuj0D6+tBXwciklhM4/GeWhUFJiR/wcI6XRIp4/6 6bi1xMl78S/RNyDFH2m+n9jPQMA/JqF6Qn90xeLKgZs92Xc7eaDugWIhFlgjq8ZxKyvynN wyKZe1Lv/v2igNNKG9F9RMK9GJIkYbw= X-MC-Unique: 0PX2CnW2O9OHHYZEJFoY4g-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fc/ksO43unMARgcWrNAODMghjoBwYq0onMQNlotNsMIfjU2jZUA+5pEfyyUepztttEAmCQa/U3kSXEg9EqxFJjTOnjofB+zQmibDftehe38kw0b3ogzLhgr5oLFo3BtX2XcezWzV32SYqXb5hgHklG63a1IrfI8aMfoJaUa22UBSeIpiGlB3lHUDh0tBOBbL5qw5Lwpao+oNS+J/HMsF4SEHaGKrxffLp1InGuD2b+TL5fLdW9IL0ZDCo006yGGaFw9vhnzTiccpN0zygFnQ/MlC6uGIMBDpxaHglV46RfR+/EOhXh3zC0xOChEjfeiY1ZIud+oEuFUz6BjqPMJ4Jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=L4OzGEj91QdJC5vdoESTAykpaiWdGj46EwoZ9sMMZn8=; b=lwUwJSZDUNVr5P2xHsnGjGYmzVxPWT4DBJ76dX6DxO6J0NwxlkOGwEzU4TEjVgvKyT9QSHXjPHJrK+nHJ1bz3Ow0NZQIxqfAXxsQAl2cOBjhk47dLaZlslmhZBctI1OKEvDQfp783HY5ZQQiC2FQ2A8ZrkRORyJ4LaynxQrW6mnNJoQHe4SLTne/0/IxDAUZtVYA1enYGYMH9BTlPJ3Sg6bJ2qNEbt8/hvbkGf7nNRYE58xmN5JOO7A8qi8MNBvuYSz46ZcTsuyJ5mBJFctRFn2KhGRfYqhO5pK4xrRmlaLWdWIG8zaR6I5p7MDVENQRN7qEOE6MzzOscF2bk/9p/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=suse.com; Subject: [PATCH v8 5/6] AMD/IOMMU: pull ATS disabling earlier From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant References: Message-ID: <34c081bd-ac6d-055c-e6c1-eeac62851389@suse.com> Date: Wed, 22 Sep 2021 16:38:10 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Language: en-US X-ClientProxiedBy: AM0PR02CA0171.eurprd02.prod.outlook.com (2603:10a6:20b:28e::8) To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4a0bd19c-454f-43f6-72d5-08d97dd69a0b X-MS-TrafficTypeDiagnostic: VI1PR04MB7024: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WHHKaRBBGCyTAzxb+ZeAonbYdx57SgjqKWPZgfh+06yjnPo7oafVQJVwtTZgOierleiIXe89jsyU7qMnOJUlOFa+sRKVNS4NZ9FT9g+sePZ2yfRaK7XkRjU93ESvhjelIHwE9Z2ppSXGaV3zz/mt5yCuTFDqB98UjInggXRERE+gUfY5vTShc8jzsbUZ23czTxXuFLY7dfS0EzIIOPZDc16OQiRcM+GmQMTivusumZUnngxCNLWsaS+p/NdW/+4q1uovk1RJiuIpUkEoAQ/2+h2MYOofkqonF0HutEksQSqh/Xnc1wb5ZxQ4UyjOQDKODQh/xIgh72l6hVrmGsw71NctKgsFmdTPuWa7UT+scOwTTNtz1hYNiYtmOSCM8Qg+TTCC8OhWe2Yhlx+S7zSX7LB/YXLzu/V3EhLPcY/ZGRp1qJUzy5Q3bYTWHi2HPW/1Z1Gtl+jgOmiwRouW16N56eJEE7vme5vARShrBlbSWlTRCFUKoz4QiQaRrm7PH7T6q6FJVlneGXuIPwQp/9pG0UGDZDy1jgQ93uRD3r0gr4ObpJ2QYukiw6gIg2fpCvlgpvRExiFBb8XB6mo7n0JhqmnAD+lvHHAXsG490DdDQ5v39NUHT7uSK6wR0UXciPbElhzaLZW18g6geXp0xL2KJkzcvGDupHvksemtYBRpgkPjACukRTUHL3S9jA/w+5vxLZVPVFsVsGtFUkjO4fiQz8VaziOL3AfFOu/slvBLuqc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?RcaI4u7r8itkZZbP3AtDGwUyIZ3T?= =?utf-8?q?ehHCKB9v+6ik5JWbO7j2hsx0/XyIUrCSnsCzP84jslJwv45PY8JxBMeDicuWLBndB?= =?utf-8?q?RxOg2W2DPtaVs039vpdciI6kUFqgnYpTazwPmZHltxJKwcvYFlrWq3gqy804bcWFB?= =?utf-8?q?07t2Krgq7nXVzwJ06azYESXBEPvFKLHzZUOIP8Y0BZdNfP1ZTSJzZ6P8fYtT9k/KC?= =?utf-8?q?03SHSdefbS5mDMtLMF2+Wlhs0MlRyMtJw8aqeqL7G3TyF1bNnXb5LyyoNuzZwuRnx?= =?utf-8?q?VGyaRC14y0WQGdz/mU0E2mnVZUY0q3rPIkxJKHf8D0RL0oeu8WUcc3ISxSr9y6Mg7?= =?utf-8?q?7WEGL6q9FdfAg79uzQNEkEvbo8IgBdIZE/PLFEsXopWH71sS3XFKKf2aOmetWw4ey?= =?utf-8?q?ckuThXNEuVPMkpP885S/mVrjwZ1u+0+Pu6DqmO1iVaYTYB3Ve+E9bNyasJl5MpYVF?= =?utf-8?q?AiFaDLWbugQIyeoawxLjeWBiKGoq3hUytO9lrB7NKOPr5juYX7hOomTpOXlMw6CP7?= =?utf-8?q?+fJvKgRL7uzBkAQ7gB1H5IBSQgFHF6Y2d63s0DFKAZAnylxbuD2+YyO+vQbmgB1fA?= =?utf-8?q?D5PGdQKQR94ZyFleB2C7BRXSf1tocKbBDc/MpWdwCUUX/EaAT7TSbubs9ovfhRtyt?= =?utf-8?q?Gu54bN7lMkJp+HrI9RMlhTB5D0eMNvG1Cc3/YcegYiyWTCPZnmBnM4epIzpHZaZej?= =?utf-8?q?sqEBIIMNZZl6HuQqMMxdOpyfuuvxzwS+ZmsjrDQgiJC/y8ZBEaAXlBktoOx3/QR5b?= =?utf-8?q?x9uSSYDOGHHf6JFNjuJfxe/QvLHRWRRP0LzonqBmp/scc7R+TbdB/IgIZ3di6B8/T?= =?utf-8?q?OCDFHFV0TfAzL3/sAxcXX2aBN0gjymX17/xgukFE4m3989RauVP37RFyHjgSsuc4g?= =?utf-8?q?0T4tqDf3MaUKMUctTvheSmS3UlJgUUtikYpE2IIjUd+/Uch03G0Xh14ipUKxKE1sb?= =?utf-8?q?nvcSmKnbIEwuNrMHBvGKVay2pvlNMjk96RpTbZXa8F97CJkoBcHsbJTa3WyNP4pYr?= =?utf-8?q?BJcFu5RGmw9UqZp+A6hAeJX32Pp2Ig6Pol06Gf2z0mdV58SwiU/Ue2cfxP22RUpg4?= =?utf-8?q?IVrVakUtKk33D0OaLKTOmPBFZeOUch4BEC8VuFA3PZgJODdftO6BU39QBOJVKHGP2?= =?utf-8?q?nAxEOoWBsIwIt8Tpf1cEEfm6tKL+U3JYsSvRGc4Az2Ju/4bOVysFOXDyCc7TG7FgD?= =?utf-8?q?6Ng2RnddjNUzPI90cwf6T9lqRAmJIvUlR3WMp6IoiT44S0i8i/gIJBiboW32N1RsO?= =?utf-8?q?g8J8wLCpAzGLksz/?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4a0bd19c-454f-43f6-72d5-08d97dd69a0b X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:38:11.1350 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KwwRN+RDChHsgpPzEKBSPp5kM7vIOcDMCggOsBmfEZx+e1lRjTZkua321RhAJmAkcoTR5Cvh6Y6tDNA5+QnZxA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024 Disabling should be done in the opposite order of enabling: ATS wants to be turned off before adjusting the DTE, just like it gets enabled only after the DTE was suitably prepared. Note that we want ATS to be disabled as soon as any of the DTEs involved in the handling of a device (including phantom devices) gets adjusted respectively. For this reason the "devfn == pdev->devfn" of the original conditional gets dropped. Signed-off-by: Jan Beulich Reviewed-by: Paul Durrant --- TBD: This points out that for phantom devices the ordering in amd_iommu_setup_domain_device() may also not be fully suitable: ATS would better be enabled on the device only after all involved DTEs have got prepared. This would be a less straightforward change, though. --- v8: New. --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -313,6 +313,12 @@ static void amd_iommu_disable_domain_dev if ( QUARANTINE_SKIP(domain) ) return; + ASSERT(pcidevs_locked()); + + if ( pci_ats_device(iommu->seg, bus, pdev->devfn) && + pci_ats_enabled(iommu->seg, bus, pdev->devfn) ) + disable_ats_device(pdev); + BUG_ON ( iommu->dev_table.buffer == NULL ); req_id = get_dma_requestor_id(iommu->seg, PCI_BDF2(bus, devfn)); table = iommu->dev_table.buffer; @@ -348,13 +354,6 @@ static void amd_iommu_disable_domain_dev } else spin_unlock_irqrestore(&iommu->lock, flags); - - ASSERT(pcidevs_locked()); - - if ( devfn == pdev->devfn && - pci_ats_device(iommu->seg, bus, devfn) && - pci_ats_enabled(iommu->seg, bus, devfn) ) - disable_ats_device(pdev); } static int reassign_device(struct domain *source, struct domain *target, From patchwork Wed Sep 22 14:38:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12510819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A222EC433F5 for ; Wed, 22 Sep 2021 14:38:49 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 35C7B60F26 for ; Wed, 22 Sep 2021 14:38:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 35C7B60F26 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.192892.343613 (Exim 4.92) (envelope-from ) id 1mT3OS-000549-Mb; Wed, 22 Sep 2021 14:38:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 192892.343613; Wed, 22 Sep 2021 14:38:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3OS-000542-J3; Wed, 22 Sep 2021 14:38:40 +0000 Received: by outflank-mailman (input) for mailman id 192892; Wed, 22 Sep 2021 14:38:40 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mT3OS-0004ys-8Q for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:38:40 +0000 Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 2b2ff7cc-b033-4d38-b88e-1507635c57dd; Wed, 22 Sep 2021 14:38:35 +0000 (UTC) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-37-E95VvMe0PpS4HE8kUjRD6A-1; Wed, 22 Sep 2021 16:38:33 +0200 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep 2021 14:38:32 +0000 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021 14:38:32 +0000 Received: from [10.156.60.236] (37.24.206.209) by AM3PR04CA0127.eurprd04.prod.outlook.com (2603:10a6:207::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:38:31 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2b2ff7cc-b033-4d38-b88e-1507635c57dd DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1632321514; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=04x1FN662zVsOx4AF8ENkQRIOoZIpNC+jLSRni5iedE=; b=X592sk0E9+CYPDVnDEE5dHo0LkNfOwQrWsIuSXx7WSo0sfeHiYXV1NjZ6/e2aUAfi+jfy8 HtAR27o2aeh4tGUl+W0t9hP9ioZp0CWjH5GMdmSRDS0V0Lb4Wn4A/te7brAXkE5uF43lBS Q6vGvHHJv5vB36ii12BeB68mK9yE+Wk= X-MC-Unique: E95VvMe0PpS4HE8kUjRD6A-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=miQD+jlnl/Wt42LIOFroovVzmLJbrBrSyKqHXs6ZZEj1K3Wo9R11YxzmC3WSAI57HNi7iaoOvi50yxeDzb1GAUkPov7+x6+CmxE1MayrvErSD1hpp2oZxx0JEABRADkEZmB8b/iN7L5OCMRXDR8/8HIpwm8UNTHlrJkx3gh5Jh286BYCM/br9/f34O/f1DXbAC8kGItZ8DoWi5DVJzQVECgcEVBTC7DbfTWpYNvv1xrZGGyvLEk9ISQt2LYvvrbUAvWm4c4/buO4tys4Zt4qz249R92tqN4gdzMOqB0udcrkeND3pD62ukdK+2kQ6oiDQffbGRwkmbFXr77jQ/qFEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=04x1FN662zVsOx4AF8ENkQRIOoZIpNC+jLSRni5iedE=; b=jxECi7dC9VTTmgtgNevgYWdIGAtE9FIDT0XoZTBLfU6CrbWDJrWFSDpJAaWss088JKtDmyKvHl2AhP6eeoQaKw32u1oy4X3D+tkRWWkcqK6TMh85JswdAwbeE0m69aWL8lqczLNOFBVPwtWtaMUR0ZKbLRODD1rl1sVOGENmBc6l79bxbyW/KDTJaqnMzoC5g2RuxFBGA6G2smy35jascfVTnA2sTyVM+n/JbuvePD0aS5ktnBiuJzb8d+LXo/Un7dJv0ruPXAYrdlFTaZQn6pt7aKKpaAdJFY28Ct9bMQ1vD+hPajO49FSnpEstFV7IR4JADHUS9EEZ6UZ7fa/UQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=suse.com; Subject: [PATCH v8 6/6] AMD/IOMMU: expose errors and warnings unconditionally From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant References: Message-ID: Date: Wed, 22 Sep 2021 16:38:31 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Language: en-US X-ClientProxiedBy: AM3PR04CA0127.eurprd04.prod.outlook.com (2603:10a6:207::11) To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 336a2ede-e6db-43f0-4424-08d97dd6a66c X-MS-TrafficTypeDiagnostic: VI1PR04MB7024: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:82; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4rIZEWo7cpE39OwVJ+uB7CrckO0bPNZv5ffJsjxsRRkmX0B7BPM7U2iadIAm8+T5f3c5d61kQGIdGVLQmd/FD9RzNmHptOjkUhU1TEng4z2cheIhhWpW3LV2k9W63HXEv1yZL9WfE/uTh2muiQuusnMNSfnLLzFJoVysYK9BQ0Zw3NdLFK5Gzjl8qFQszf+V6JHB+EB0qBIPkmhe4eqmHZsggwm+9HtF2pY6v1V1ET6Iqz+HYxOtrpbMZwTGAxwl80yYXIthJIEf5+Ym5cPFC4b8qKpgebF18zJMJtRXrTDpBY4FVwo5ekp1Bodp+fSAIYemW+9im7+vyaBw0FowqzMZcbPHCTSSIC+8NlJNJBHUpRXr0G+j6TmVCA6+I7Oq/6TpPnPt1GqyhTec1/Y8okuupyClh/Ir/JtWzFw+5N5GeArLwA4pmYVwQ/vbie6/d1hiHgG4BEXsIPur7YcyMzCtXXIH+6fFw7Mv2r0JKNlrvii6Ij992cVI3ITYA9Erea44FlGtkY6xPdIADoZav1HRGmQph2Ue1p4CDjTcgLPdzt2PsUsoz9cBL4HqGmr7ZDR1gTtNEnxopwqaLMcA9sD+bYhCOvEX5rqg2024V9psYyXPRs+uLL7rV82502ofJqhjDipZtrYTjR/fU6o/wtLvN5i3f8R4zL55Fa66teLUDi+nsvtcofmNJvZ12XjoNUf1rPYV7iCsViXy74WDnHwflZcv5ExRTMeMD48/r20= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(30864003)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?WDbyoLTgM9kAE8+0J3+S/Fk0R6qv?= =?utf-8?q?PAtxrGvj1W1onXMAmmIpKUYMi9Vmcw1anhmLURLhmtoQA5gVbeOQnlFmZ0Xtjvuy6?= =?utf-8?q?iVs1uDRTZtTJFoaI5kJk25czR0Fn2QJDVHDUfBQkenaQW9m8ixUtaUx5n5dd42fhg?= =?utf-8?q?yZdBbMwGiFm0ujDe1lVwOMObOjrkv8Lwah6LqMjMKRUzcbMGIFq4ErOovkrvXADyp?= =?utf-8?q?w3jp9Q9N6r2Q+IuPYIldlvOWJbh1ZdI8a3Lyl+THOTeaWbpHOTQG0LMND5pyJnKaK?= =?utf-8?q?6V3LbzPCbsPxv7k4b1rvExoaCKR1Fzpo9cUvl/RY0xLCTOUxCnWYh6Bn7z2KNlTYE?= =?utf-8?q?rH9sNYcdfc/DwNxHOWn9xxQQ6vqZOHXXgRAD8lgoNqQujpl6GxfsOjZQxpVT15m9E?= =?utf-8?q?cBeNSFDNNugv6gm+PgqJMT1uRHPJnD3pKOeUiAkloALTETi5vJ7wYpRUracODLg5V?= =?utf-8?q?Aq6igFYuBzVERqSsJH8kxprZ4uzUXj9yoT7pRizOweBQzyypf8snEKXxK/GPmF6hn?= =?utf-8?q?pTiPm9q+C7SpW8ibyFlZTdF9eU9Yzc0C4/JAVT2QcMFCnUt0/oBmfWzL/NmXukZw7?= =?utf-8?q?z72PF01rzLAEBIfF01Qp/KKOQrxXNXeQjWUSQZTWYdL+gBQkPrWKt1zjkjQgX5qAb?= =?utf-8?q?ClQ8xROc+u1fRprzFU2nA31O2jS26nSQvrce5GSTabC4mfqT+Q5uelKqTbh4JDdw3?= =?utf-8?q?3R3AhcfwOcO0bAQJUobpz6WCb6ekTg9jYE4ZN7Ue/beMs9u6rXj+XhaJE5d/FZ0IB?= =?utf-8?q?YWIOtsxMYes55FBJtkuX48R7SxB1EhNUhDo4ON3sj4mYZYtlMKPXN5KsV2d/GQ8NL?= =?utf-8?q?xvhf0SwhSxdyqNNFadPbRvqwRZJm7m34QTceEvm6YLPFerHjit0cQBcat4Cu9wylX?= =?utf-8?q?oMQptcVS1BlihjVp3qFtdfMKj9zbH9uhCM+xZkZ6UFS9gxcgtUQ2Tr/WQkMqb3/yK?= =?utf-8?q?tTh6hF4BFJR4WEVueSHmCchf6VYcyLbOtRv8GM6sBgRbetE7DDMCfOzW4ByfRwEsO?= =?utf-8?q?6usXnHKHcF96aV/saignuW5K+OlwTVz+XGIswYrVCidwto1bpPohYybGFJABCJVVV?= =?utf-8?q?USMHcTTPsnz0M7fscZzhKS/eCahf63KYFFg/2GPGvY1m8I1D3aMidwHnVNbHYoh+n?= =?utf-8?q?cwHmrrHkjwokS2kLSSv/PoQfWOWdzuZnA6GMCngF60kYdhgq3eVXGcYM8IACrsBK0?= =?utf-8?q?hro6VT9YFFKe5BLrkB7xPK5AZRDrtY15jdl+cnh8fF/L+zaJaRDV8aqUI/eCGg43M?= =?utf-8?q?23moT7KWNbVp/jzW?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 336a2ede-e6db-43f0-4424-08d97dd6a66c X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:38:31.9561 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: p5RF9mDBppMBR0c0nUUQb5mTsGRYkk8StNK2WNKBkkQovI5ijNzOJKwvojM5Z2G7gtCbqvNWfse7D1xl7zPJUg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024 Making these dependent upon "iommu=debug" isn't really helpful in the field. Where touching respective code anyway also make use of %pp and %pd. Requested-by: Andrew Cooper Signed-off-by: Jan Beulich Reviewed-by: Paul Durrant --- While I'm adding AMD_IOMMU_VERBOSE(), there aren't any uses for now. It's not really clear to me where to draw the boundary to AMD_IOMMU_DEBUG(). I didn't bother touching iommu_guest.c here. --- v8: New. --- a/xen/drivers/passthrough/amd/iommu.h +++ b/xen/drivers/passthrough/amd/iommu.h @@ -203,6 +203,18 @@ struct acpi_ivrs_hardware; #define DMA_32BIT_MASK 0x00000000ffffffffULL +#define AMD_IOMMU_ERROR(fmt, args...) \ + printk(XENLOG_ERR "AMD-Vi: Error: " fmt, ## args) + +#define AMD_IOMMU_WARN(fmt, args...) \ + printk(XENLOG_WARNING "AMD-Vi: Warning: " fmt, ## args) + +#define AMD_IOMMU_VERBOSE(fmt, args...) \ + do { \ + if ( iommu_verbose ) \ + printk(XENLOG_INFO "AMD-Vi: " fmt, ## args); \ + } while ( false ) + #define AMD_IOMMU_DEBUG(fmt, args...) \ do \ { \ --- a/xen/drivers/passthrough/amd/iommu_acpi.c +++ b/xen/drivers/passthrough/amd/iommu_acpi.c @@ -174,7 +174,7 @@ static int __init reserve_unity_map_for_ if ( unity_map->addr + unity_map->length > base && base + length > unity_map->addr ) { - AMD_IOMMU_DEBUG("IVMD Error: overlap [%lx,%lx) vs [%lx,%lx)\n", + AMD_IOMMU_ERROR("IVMD: overlap [%lx,%lx) vs [%lx,%lx)\n", base, base + length, unity_map->addr, unity_map->addr + unity_map->length); return -EPERM; @@ -248,7 +248,7 @@ static int __init register_range_for_dev iommu = find_iommu_for_device(seg, bdf); if ( !iommu ) { - AMD_IOMMU_DEBUG("IVMD Error: No IOMMU for Dev_Id %#x!\n", bdf); + AMD_IOMMU_ERROR("IVMD: no IOMMU for Dev_Id %#x\n", bdf); return -ENODEV; } req = ivrs_mappings[bdf].dte_requestor_id; @@ -318,7 +318,7 @@ static int __init parse_ivmd_device_sele bdf = ivmd_block->header.device_id; if ( bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVMD Error: Invalid Dev_Id %#x\n", bdf); + AMD_IOMMU_ERROR("IVMD: invalid Dev_Id %#x\n", bdf); return -ENODEV; } @@ -335,16 +335,14 @@ static int __init parse_ivmd_device_rang first_bdf = ivmd_block->header.device_id; if ( first_bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVMD Error: " - "Invalid Range_First Dev_Id %#x\n", first_bdf); + AMD_IOMMU_ERROR("IVMD: invalid Range_First Dev_Id %#x\n", first_bdf); return -ENODEV; } last_bdf = ivmd_block->aux_data; if ( (last_bdf >= ivrs_bdf_entries) || (last_bdf <= first_bdf) ) { - AMD_IOMMU_DEBUG("IVMD Error: " - "Invalid Range_Last Dev_Id %#x\n", last_bdf); + AMD_IOMMU_ERROR("IVMD: invalid Range_Last Dev_Id %#x\n", last_bdf); return -ENODEV; } @@ -367,7 +365,7 @@ static int __init parse_ivmd_device_iomm ivmd_block->aux_data); if ( !iommu ) { - AMD_IOMMU_DEBUG("IVMD Error: No IOMMU for Dev_Id %#x Cap %#x\n", + AMD_IOMMU_ERROR("IVMD: no IOMMU for Dev_Id %#x Cap %#x\n", ivmd_block->header.device_id, ivmd_block->aux_data); return -ENODEV; } @@ -384,7 +382,7 @@ static int __init parse_ivmd_block(const if ( ivmd_block->header.length < sizeof(*ivmd_block) ) { - AMD_IOMMU_DEBUG("IVMD Error: Invalid Block Length!\n"); + AMD_IOMMU_ERROR("IVMD: invalid block length\n"); return -ENODEV; } @@ -402,8 +400,8 @@ static int __init parse_ivmd_block(const (addr_bits < BITS_PER_LONG && ((start_addr + mem_length - 1) >> addr_bits)) ) { - AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not IOMMU addressable\n", - start_addr, start_addr + mem_length); + AMD_IOMMU_WARN("IVMD: [%lx,%lx) is not IOMMU addressable\n", + start_addr, start_addr + mem_length); return 0; } @@ -411,8 +409,8 @@ static int __init parse_ivmd_block(const { paddr_t addr; - AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not (entirely) in reserved memory\n", - base, limit + PAGE_SIZE); + AMD_IOMMU_WARN("IVMD: [%lx,%lx) is not (entirely) in reserved memory\n", + base, limit + PAGE_SIZE); for ( addr = base; addr <= limit; addr += PAGE_SIZE ) { @@ -423,7 +421,7 @@ static int __init parse_ivmd_block(const if ( e820_add_range(&e820, addr, addr + PAGE_SIZE, E820_RESERVED) ) continue; - AMD_IOMMU_DEBUG("IVMD Error: Page at %lx couldn't be reserved\n", + AMD_IOMMU_ERROR("IVMD: page at %lx couldn't be reserved\n", addr); return -EIO; } @@ -433,8 +431,7 @@ static int __init parse_ivmd_block(const RAM_TYPE_UNUSABLE)) ) continue; - AMD_IOMMU_DEBUG("IVMD Error: Page at %lx can't be converted\n", - addr); + AMD_IOMMU_ERROR("IVMD: page at %lx can't be converted\n", addr); return -EIO; } } @@ -448,7 +445,7 @@ static int __init parse_ivmd_block(const } else { - AMD_IOMMU_DEBUG("IVMD Error: Invalid Flag Field!\n"); + AMD_IOMMU_ERROR("IVMD: invalid flag field\n"); return -ENODEV; } @@ -471,7 +468,8 @@ static int __init parse_ivmd_block(const iw, ir, exclusion); default: - AMD_IOMMU_DEBUG("IVMD Error: Invalid Block Type!\n"); + AMD_IOMMU_ERROR("IVMD: unknown block type %#x\n", + ivmd_block->header.type); return -ENODEV; } } @@ -481,7 +479,7 @@ static u16 __init parse_ivhd_device_padd { if ( header_length < (block_length + pad_length) ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n"); return 0; } @@ -496,7 +494,7 @@ static u16 __init parse_ivhd_device_sele bdf = select->header.id; if ( bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf); return 0; } @@ -515,14 +513,13 @@ static u16 __init parse_ivhd_device_rang dev_length = sizeof(*range); if ( header_length < (block_length + dev_length) ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n"); return 0; } if ( range->end.header.type != ACPI_IVRS_TYPE_END ) { - AMD_IOMMU_DEBUG("IVHD Error: " - "Invalid Range: End_Type %#x\n", + AMD_IOMMU_ERROR("IVHD Error: invalid range: End_Type %#x\n", range->end.header.type); return 0; } @@ -530,16 +527,14 @@ static u16 __init parse_ivhd_device_rang first_bdf = range->start.header.id; if ( first_bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVHD Error: " - "Invalid Range: First Dev_Id %#x\n", first_bdf); + AMD_IOMMU_ERROR("IVHD: invalid range: First Dev_Id %#x\n", first_bdf); return 0; } last_bdf = range->end.header.id; if ( (last_bdf >= ivrs_bdf_entries) || (last_bdf <= first_bdf) ) { - AMD_IOMMU_DEBUG("IVHD Error: " - "Invalid Range: Last Dev_Id %#x\n", last_bdf); + AMD_IOMMU_ERROR("IVHD: invalid range: Last Dev_Id %#x\n", last_bdf); return 0; } @@ -561,21 +556,21 @@ static u16 __init parse_ivhd_device_alia dev_length = sizeof(*alias); if ( header_length < (block_length + dev_length) ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n"); return 0; } bdf = alias->header.id; if ( bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf); return 0; } alias_id = alias->used_id; if ( alias_id >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Alias Dev_Id %#x\n", alias_id); + AMD_IOMMU_ERROR("IVHD: invalid Alias Dev_Id %#x\n", alias_id); return 0; } @@ -597,14 +592,13 @@ static u16 __init parse_ivhd_device_alia dev_length = sizeof(*range); if ( header_length < (block_length + dev_length) ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n"); return 0; } if ( range->end.header.type != ACPI_IVRS_TYPE_END ) { - AMD_IOMMU_DEBUG("IVHD Error: " - "Invalid Range: End_Type %#x\n", + AMD_IOMMU_ERROR("IVHD: invalid range: End_Type %#x\n", range->end.header.type); return 0; } @@ -612,16 +606,14 @@ static u16 __init parse_ivhd_device_alia first_bdf = range->alias.header.id; if ( first_bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVHD Error: " - "Invalid Range: First Dev_Id %#x\n", first_bdf); + AMD_IOMMU_ERROR("IVHD: invalid range: First Dev_Id %#x\n", first_bdf); return 0; } last_bdf = range->end.header.id; if ( last_bdf >= ivrs_bdf_entries || last_bdf <= first_bdf ) { - AMD_IOMMU_DEBUG( - "IVHD Error: Invalid Range: Last Dev_Id %#x\n", last_bdf); + AMD_IOMMU_ERROR("IVHD: invalid range: Last Dev_Id %#x\n", last_bdf); return 0; } @@ -651,14 +643,14 @@ static u16 __init parse_ivhd_device_exte dev_length = sizeof(*ext); if ( header_length < (block_length + dev_length) ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n"); return 0; } bdf = ext->header.id; if ( bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf); return 0; } @@ -677,14 +669,13 @@ static u16 __init parse_ivhd_device_exte dev_length = sizeof(*range); if ( header_length < (block_length + dev_length) ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n"); return 0; } if ( range->end.header.type != ACPI_IVRS_TYPE_END ) { - AMD_IOMMU_DEBUG("IVHD Error: " - "Invalid Range: End_Type %#x\n", + AMD_IOMMU_ERROR("IVHD: invalid range: End_Type %#x\n", range->end.header.type); return 0; } @@ -692,16 +683,14 @@ static u16 __init parse_ivhd_device_exte first_bdf = range->extended.header.id; if ( first_bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVHD Error: " - "Invalid Range: First Dev_Id %#x\n", first_bdf); + AMD_IOMMU_ERROR("IVHD: invalid range: First Dev_Id %#x\n", first_bdf); return 0; } last_bdf = range->end.header.id; if ( (last_bdf >= ivrs_bdf_entries) || (last_bdf <= first_bdf) ) { - AMD_IOMMU_DEBUG("IVHD Error: " - "Invalid Range: Last Dev_Id %#x\n", last_bdf); + AMD_IOMMU_ERROR("IVHD: invalid range: Last Dev_Id %#x\n", last_bdf); return 0; } @@ -789,14 +778,14 @@ static u16 __init parse_ivhd_device_spec dev_length = sizeof(*special); if ( header_length < (block_length + dev_length) ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n"); return 0; } bdf = special->used_id; if ( bdf >= ivrs_bdf_entries ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf); + AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf); return 0; } @@ -844,12 +833,12 @@ static u16 __init parse_ivhd_device_spec { if ( ioapic_sbdf[idx].bdf == bdf && ioapic_sbdf[idx].seg == seg ) - AMD_IOMMU_DEBUG("IVHD Warning: Duplicate IO-APIC %#x entries\n", + AMD_IOMMU_WARN("IVHD: duplicate IO-APIC %#x entries\n", special->handle); else { - printk(XENLOG_ERR "IVHD Error: Conflicting IO-APIC %#x entries\n", - special->handle); + AMD_IOMMU_ERROR("IVHD: conflicting IO-APIC %#x entries\n", + special->handle); if ( amd_iommu_perdev_intremap ) return 0; } @@ -944,7 +933,7 @@ static int __init parse_ivhd_block(const if ( ivhd_block->header.length < hdr_size ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Block Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid block length\n"); return -ENODEV; } @@ -953,7 +942,7 @@ static int __init parse_ivhd_block(const ivhd_block->capability_offset); if ( !iommu ) { - AMD_IOMMU_DEBUG("IVHD Error: No IOMMU for Dev_Id %#x Cap %#x\n", + AMD_IOMMU_ERROR("IVHD: no IOMMU for Dev_Id %#x Cap %#x\n", ivhd_block->header.device_id, ivhd_block->capability_offset); return -ENODEV; @@ -1016,7 +1005,8 @@ static int __init parse_ivhd_block(const ivhd_block->header.length, block_length, iommu); break; default: - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device Type!\n"); + AMD_IOMMU_WARN("IVHD: unknown device type %#x\n", + ivhd_device->header.type); dev_length = 0; break; } @@ -1113,8 +1103,7 @@ static int __init parse_ivrs_table(struc if ( table->length < (length + ivrs_block->length) ) { - AMD_IOMMU_DEBUG("IVRS Error: " - "Table Length Exceeded: %#x -> %#lx\n", + AMD_IOMMU_ERROR("IVRS: table length exceeded: %#x -> %#lx\n", table->length, (length + ivrs_block->length)); return -ENODEV; @@ -1214,7 +1203,7 @@ static int __init get_last_bdf_ivhd( if ( ivhd_block->header.length < hdr_size ) { - AMD_IOMMU_DEBUG("IVHD Error: Invalid Block Length!\n"); + AMD_IOMMU_ERROR("IVHD: invalid block length\n"); return -ENODEV; } @@ -1261,7 +1250,8 @@ static int __init get_last_bdf_ivhd( dev_length = sizeof(ivhd_device->special); break; default: - AMD_IOMMU_DEBUG("IVHD Error: Invalid Device Type!\n"); + AMD_IOMMU_WARN("IVHD: unknown device type %#x\n", + ivhd_device->header.type); dev_length = 0; break; } @@ -1327,7 +1317,7 @@ get_supported_ivhd_type(struct acpi_tabl checksum = acpi_tb_checksum(ACPI_CAST_PTR(uint8_t, table), table->length); if ( checksum ) { - AMD_IOMMU_DEBUG("IVRS Error: Invalid Checksum %#x\n", checksum); + AMD_IOMMU_ERROR("IVRS: invalid checksum %#x\n", checksum); return -ENODEV; } @@ -1340,8 +1330,7 @@ get_supported_ivhd_type(struct acpi_tabl if ( table->length < (length + ivrs_block->length) ) { - AMD_IOMMU_DEBUG("IVRS Error: " - "Table Length Exceeded: %#x -> %#lx\n", + AMD_IOMMU_ERROR("IVRS: table length exceeded: %#x -> %#lx\n", table->length, (length + ivrs_block->length)); return -ENODEV; --- a/xen/drivers/passthrough/amd/iommu_cmd.c +++ b/xen/drivers/passthrough/amd/iommu_cmd.c @@ -291,8 +291,7 @@ void amd_iommu_flush_iotlb(u8 devfn, con if ( !iommu ) { - AMD_IOMMU_DEBUG("%s: Can't find iommu for %pp\n", - __func__, &pdev->sbdf); + AMD_IOMMU_WARN("can't find IOMMU for %pp\n", &pdev->sbdf); return; } --- a/xen/drivers/passthrough/amd/iommu_detect.c +++ b/xen/drivers/passthrough/amd/iommu_detect.c @@ -141,21 +141,21 @@ int __init amd_iommu_detect_one_acpi( if ( ivhd_block->header.length < sizeof(*ivhd_block) ) { - AMD_IOMMU_DEBUG("Invalid IVHD Block Length!\n"); + AMD_IOMMU_ERROR("invalid IVHD block length\n"); return -ENODEV; } if ( !ivhd_block->header.device_id || !ivhd_block->capability_offset || !ivhd_block->base_address) { - AMD_IOMMU_DEBUG("Invalid IVHD Block!\n"); + AMD_IOMMU_ERROR("invalid IVHD block\n"); return -ENODEV; } iommu = xzalloc(struct amd_iommu); if ( !iommu ) { - AMD_IOMMU_DEBUG("Error allocating amd_iommu\n"); + AMD_IOMMU_ERROR("cannot allocate amd_iommu\n"); return -ENOMEM; } --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -386,8 +386,8 @@ static void iommu_reset_log(struct amd_i if ( log_run ) { - AMD_IOMMU_DEBUG("Warning: Log Run bit %d is not cleared" - "before reset!\n", run_bit); + AMD_IOMMU_WARN("Log Run bit %d is not cleared before reset\n", + run_bit); return; } @@ -754,8 +754,8 @@ static bool_t __init set_iommu_interrupt pcidevs_unlock(); if ( !iommu->msi.dev ) { - AMD_IOMMU_DEBUG("IOMMU: no pdev for %pp\n", - &PCI_SBDF2(iommu->seg, iommu->bdf)); + AMD_IOMMU_WARN("no pdev for %pp\n", + &PCI_SBDF2(iommu->seg, iommu->bdf)); return 0; } @@ -799,7 +799,7 @@ static bool_t __init set_iommu_interrupt if ( ret ) { destroy_irq(irq); - AMD_IOMMU_DEBUG("can't request irq\n"); + AMD_IOMMU_ERROR("can't request irq\n"); return 0; } @@ -992,7 +992,7 @@ static void *__init allocate_buffer(unsi if ( buffer == NULL ) { - AMD_IOMMU_DEBUG("Error allocating %s\n", name); + AMD_IOMMU_ERROR("cannot allocate %s\n", name); return NULL; } @@ -1224,7 +1224,7 @@ static int __init alloc_ivrs_mappings(u1 ivrs_mappings = xzalloc_array(struct ivrs_mappings, ivrs_bdf_entries + 1); if ( ivrs_mappings == NULL ) { - AMD_IOMMU_DEBUG("Error allocating IVRS Mappings table\n"); + AMD_IOMMU_ERROR("cannot allocate IVRS Mappings table\n"); return -ENOMEM; } IVRS_MAPPINGS_SEG(ivrs_mappings) = seg; --- a/xen/drivers/passthrough/amd/iommu_intr.c +++ b/xen/drivers/passthrough/amd/iommu_intr.c @@ -377,8 +377,8 @@ void amd_iommu_ioapic_update_ire( iommu = find_iommu_for_device(seg, bdf); if ( !iommu ) { - AMD_IOMMU_DEBUG("Fail to find iommu for ioapic device id =" - " %04x:%04x\n", seg, bdf); + AMD_IOMMU_WARN("failed to find IOMMU for IO-APIC @ %04x:%04x\n", + seg, bdf); __io_apic_write(apic, reg, value); return; } @@ -747,8 +747,8 @@ bool __init iov_supports_xt(void) if ( !find_iommu_for_device(ioapic_sbdf[idx].seg, ioapic_sbdf[idx].bdf) ) { - AMD_IOMMU_DEBUG("No IOMMU for IO-APIC %#x (ID %x)\n", - apic, IO_APIC_ID(apic)); + AMD_IOMMU_WARN("no IOMMU for IO-APIC %#x (ID %x)\n", + apic, IO_APIC_ID(apic)); return false; } } @@ -765,14 +765,12 @@ int __init amd_setup_hpet_msi(struct msi if ( hpet_sbdf.init == HPET_NONE ) { - AMD_IOMMU_DEBUG("Failed to setup HPET MSI remapping." - " Missing IVRS HPET info.\n"); + AMD_IOMMU_ERROR("failed to setup HPET MSI remapping: missing IVRS HPET info\n"); return -ENODEV; } if ( msi_desc->hpet_id != hpet_sbdf.id ) { - AMD_IOMMU_DEBUG("Failed to setup HPET MSI remapping." - " Wrong HPET.\n"); + AMD_IOMMU_ERROR("failed to setup HPET MSI remapping: wrong HPET\n"); return -ENODEV; } --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -222,7 +222,7 @@ static int iommu_pde_from_dfn(struct dom table = iommu_alloc_pgtable(d); if ( table == NULL ) { - AMD_IOMMU_DEBUG("Cannot allocate I/O page table\n"); + AMD_IOMMU_ERROR("cannot allocate I/O page table\n"); unmap_domain_page(next_table_vaddr); return 1; } @@ -252,7 +252,7 @@ static int iommu_pde_from_dfn(struct dom table = iommu_alloc_pgtable(d); if ( table == NULL ) { - AMD_IOMMU_DEBUG("Cannot allocate I/O page table\n"); + AMD_IOMMU_ERROR("cannot allocate I/O page table\n"); unmap_domain_page(next_table_vaddr); return 1; } @@ -301,7 +301,7 @@ int amd_iommu_map_page(struct domain *d, if ( rc ) { spin_unlock(&hd->arch.mapping_lock); - AMD_IOMMU_DEBUG("Root table alloc failed, dfn = %"PRI_dfn"\n", + AMD_IOMMU_ERROR("root table alloc failed, dfn = %"PRI_dfn"\n", dfn_x(dfn)); domain_crash(d); return rc; @@ -310,7 +310,7 @@ int amd_iommu_map_page(struct domain *d, if ( iommu_pde_from_dfn(d, dfn_x(dfn), &pt_mfn, true) || !pt_mfn ) { spin_unlock(&hd->arch.mapping_lock); - AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n", + AMD_IOMMU_ERROR("invalid IO pagetable entry dfn = %"PRI_dfn"\n", dfn_x(dfn)); domain_crash(d); return -EFAULT; @@ -343,7 +343,7 @@ int amd_iommu_unmap_page(struct domain * if ( iommu_pde_from_dfn(d, dfn_x(dfn), &pt_mfn, false) ) { spin_unlock(&hd->arch.mapping_lock); - AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n", + AMD_IOMMU_ERROR("invalid IO pagetable entry dfn = %"PRI_dfn"\n", dfn_x(dfn)); domain_crash(d); return -EFAULT; --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -367,10 +367,8 @@ static int reassign_device(struct domain iommu = find_iommu_for_device(pdev->seg, bdf); if ( !iommu ) { - AMD_IOMMU_DEBUG("Fail to find iommu." - " %04x:%02x:%x02.%x cannot be assigned to dom%d\n", - pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn), - target->domain_id); + AMD_IOMMU_WARN("failed to find IOMMU: %pp cannot be assigned to %pd\n", + &pdev->sbdf, target); return -ENODEV; } @@ -484,8 +482,8 @@ static int amd_iommu_add_device(u8 devfn return 0; } - AMD_IOMMU_DEBUG("No iommu for %pp; cannot be handed to d%d\n", - &pdev->sbdf, pdev->domain->domain_id); + AMD_IOMMU_WARN("no IOMMU for %pp; cannot be handed to %pd\n", + &pdev->sbdf, pdev->domain); return -ENODEV; } @@ -527,9 +525,8 @@ static int amd_iommu_add_device(u8 devfn pdev->domain, ivrs_mappings[ivrs_mappings[bdf].dte_requestor_id].unity_map, 0) ) - AMD_IOMMU_DEBUG("%pd: unity mapping failed for %04x:%02x:%02x.%u\n", - pdev->domain, pdev->seg, pdev->bus, PCI_SLOT(devfn), - PCI_FUNC(devfn)); + AMD_IOMMU_WARN("%pd: unity mapping failed for %pp\n", + pdev->domain, &pdev->sbdf); return amd_iommu_setup_domain_device(pdev->domain, iommu, devfn, pdev); } @@ -547,7 +544,7 @@ static int amd_iommu_remove_device(u8 de iommu = find_iommu_for_device(pdev->seg, bdf); if ( !iommu ) { - AMD_IOMMU_DEBUG("Fail to find iommu. %pp cannot be removed from %pd\n", + AMD_IOMMU_WARN("failed to find IOMMU: %pp cannot be removed from %pd\n", &pdev->sbdf, pdev->domain); return -ENODEV; } @@ -560,9 +557,8 @@ static int amd_iommu_remove_device(u8 de if ( amd_iommu_reserve_domain_unity_unmap( pdev->domain, ivrs_mappings[ivrs_mappings[bdf].dte_requestor_id].unity_map) ) - AMD_IOMMU_DEBUG("%pd: unity unmapping failed for %04x:%02x:%02x.%u\n", - pdev->domain, pdev->seg, pdev->bus, PCI_SLOT(devfn), - PCI_FUNC(devfn)); + AMD_IOMMU_WARN("%pd: unity unmapping failed for %pp\n", + pdev->domain, &pdev->sbdf); if ( amd_iommu_perdev_intremap && ivrs_mappings[bdf].dte_requestor_id == bdf &&