From patchwork Mon Apr 14 02:02:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Jimenez X-Patchwork-Id: 14049591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B11E0C3601E for ; Mon, 14 Apr 2025 02:06:06 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u49BH-0007Qc-RC; Sun, 13 Apr 2025 22:04:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u49B7-0007Mc-U6 for qemu-devel@nongnu.org; Sun, 13 Apr 2025 22:04:05 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u49B6-0007Vs-4t for qemu-devel@nongnu.org; Sun, 13 Apr 2025 22:04:05 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53E1q2gU028952; Mon, 14 Apr 2025 02:03:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=S7Vmg 0Nh3f8VouO4CdZYlJ44Lpyiz65PHE/SJoifmXo=; b=GUW7/H/TJY/U3tKZHky5T yFuKiBhjPxYO5MgdAqV4qWhxw7rKEwa1I/cSBwDi4VgiTl3a33X6sYV2WMYClVsG i9QC2otgcJAN+Vx8huhN33WgWcH1wG0m+uQboCEORA2AHouMf0d0xco+gcuUW9Ho kyaQlwUHYfGrG4I2wNU9Fc29asF5noIRNiGH2pY/+X1u306ojs0daenhW8+V5AVJ eF1ANL6RRWBZvyNE0TE6lvShIT0Pa7jtayl6skrRRObjAUcIU8Pa4T8ml9MWO+IQ Bu3vWUmmUXFrLdA4Jv9ZoZELwclKk/V+nCrigER4R75RzfqdpS4jOhMYHJCFANWK g== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 460s2u007t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2025 02:03:49 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 53DN3TWr009163; Mon, 14 Apr 2025 02:03:48 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 460d3fvqua-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2025 02:03:48 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 53E22tYo035550; Mon, 14 Apr 2025 02:03:48 GMT Received: from alaljimee5bm-ol9-20250405.osdevelopmeniad.oraclevcn.com (alaljimee5bm-ol9-20250405.allregionaliads.osdevelopmeniad.oraclevcn.com [100.100.254.235]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 460d3fvqg0-15; Mon, 14 Apr 2025 02:03:47 +0000 From: Alejandro Jimenez To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, richard.henderson@linaro.org, eduardo@habkost.net, peterx@redhat.com, david@redhat.com, philmd@linaro.org, mst@redhat.com, marcel.apfelbaum@gmail.com, alex.williamson@redhat.com, vasant.hegde@amd.com, suravee.suthikulpanit@amd.com, santosh.shukla@amd.com, sarunkod@amd.com, Wei.Huang2@amd.com, joao.m.martins@oracle.com, boris.ostrovsky@oracle.com, alejandro.j.jimenez@oracle.com Subject: [PATCH 14/18] amd_iommu: Toggle address translation on device table entry invalidation Date: Mon, 14 Apr 2025 02:02:49 +0000 Message-ID: <20250414020253.443831-15-alejandro.j.jimenez@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250414020253.443831-1-alejandro.j.jimenez@oracle.com> References: <20250414020253.443831-1-alejandro.j.jimenez@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-13_12,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2502280000 definitions=main-2504140014 X-Proofpoint-GUID: xrbHagKSOEMxN8fsBLGcsjsrGURY8yUK X-Proofpoint-ORIG-GUID: xrbHagKSOEMxN8fsBLGcsjsrGURY8yUK Received-SPF: pass client-ip=205.220.165.32; envelope-from=alejandro.j.jimenez@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.266, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A guest must issue an INVALIDATE_DEVTAB_ENTRY command after changing a Device Table entry (DTE) e.g. after attaching a device and setting up its DTE. When intercepting this event, determine if the DTE has been configured for paging or not, and toggle the appropriate memory regions to allow DMA address translation for the address space if needed. Signed-off-by: Alejandro Jimenez --- hw/i386/amd_iommu.c | 68 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 67 insertions(+), 1 deletion(-) diff --git a/hw/i386/amd_iommu.c b/hw/i386/amd_iommu.c index 3bfa08419ffe..abdd67f6b12c 100644 --- a/hw/i386/amd_iommu.c +++ b/hw/i386/amd_iommu.c @@ -101,6 +101,8 @@ static void amdvi_sync_shadow_page_table_range(AMDVIAddressSpace *as, uint64_t *dte, hwaddr addr, uint64_t size, bool send_unmap); static void amdvi_address_space_unmap(AMDVIAddressSpace *as, IOMMUNotifier *n); +static void amdvi_address_space_sync(AMDVIAddressSpace *as); +static void amdvi_switch_address_space(AMDVIAddressSpace *amdvi_as); uint64_t amdvi_extended_feature_register(AMDVIState *s) { @@ -432,7 +434,15 @@ static void amdvi_completion_wait(AMDVIState *s, uint64_t *cmd) trace_amdvi_completion_wait(addr, data); } -/* log error without aborting since linux seems to be using reserved bits */ +/* + * A guest driver must issue the INVALIDATE_DEVTAB_ENTRY command to the IOMMU + * after changing a Device Table entry. We can use this fact to detect when a + * Device Table entry is created for a device attached to a paging domain and + * and enable the corresponding IOMMU memory region to allow for DMA + * translation if appropriate. + * + * log error without aborting since linux seems to be using reserved bits + */ static void amdvi_inval_devtab_entry(AMDVIState *s, uint64_t *cmd) { uint16_t devid = cpu_to_le16((uint16_t)extract64(cmd[0], 0, 16)); @@ -442,6 +452,62 @@ static void amdvi_inval_devtab_entry(AMDVIState *s, uint64_t *cmd) amdvi_log_illegalcom_error(s, extract64(cmd[0], 60, 4), s->cmdbuf + s->cmdbuf_head); } + + /* + * Convert the devid encoded in the command to a bus and devfn in + * order to retrieve the corresponding address space. + */ + uint8_t bus_num, devfn, dte_mode; + AMDVIAddressSpace *as; + uint64_t dte[4] = { 0 }; + IOMMUNotifier *n; + int ret; + + bus_num = PCI_BUS_NUM(devid); + devfn = devid & 0xff; + + /* + * The main buffer of size (AMDVIAddressSpace *) * (PCI_BUS_MAX) has already + * been allocated within AMDVIState, but must be careful to not access + * unallocated devfn. + */ + if (!s->address_spaces[bus_num] || !s->address_spaces[bus_num][devfn]) { + return; + } + as = s->address_spaces[bus_num][devfn]; + + ret = amdvi_as_to_dte(as, dte); + + if (!ret) { + dte_mode = (dte[0] >> AMDVI_DEV_MODE_RSHIFT) & AMDVI_DEV_MODE_MASK; + } + + if ((ret < 0) || (!ret && !dte_mode)) { + /* + * The DTE could not be retrieved, it is not valid, or it is not setup + * for paging. In either case, ensure that if paging was previously in + * use then switch to use the no_dma memory region, and invalidate all + * existing mappings. + */ + if (as->addr_translation) { + as->addr_translation = false; + + amdvi_switch_address_space(as); + + IOMMU_NOTIFIER_FOREACH(n, &as->iommu) { + amdvi_address_space_unmap(as, n); + } + } + } else if (!as->addr_translation) { + /* + * Installing a DTE that enables translation where it wasn't previously + * active. Activate the DMA memory region. + */ + as->addr_translation = true; + amdvi_switch_address_space(as); + amdvi_address_space_sync(as); + } + trace_amdvi_devtab_inval(PCI_BUS_NUM(devid), PCI_SLOT(devid), PCI_FUNC(devid)); }