From patchwork Mon Apr 22 15:52:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: CLEMENT MATHIEU--DRIF X-Patchwork-Id: 13638653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 517ECC4345F for ; Mon, 22 Apr 2024 15:54:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ryvyg-0007Gc-4U; Mon, 22 Apr 2024 11:53:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ryvyc-0007Eo-KR for qemu-devel@nongnu.org; Mon, 22 Apr 2024 11:53:07 -0400 Received: from smarthost4.eviden.com ([80.78.11.85]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ryvyZ-0007c1-14 for qemu-devel@nongnu.org; Mon, 22 Apr 2024 11:53:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=eviden.com; i=@eviden.com; q=dns/txt; s=mail; t=1713801183; x=1745337183; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=A//6ZkMyDQVH+O5DOpFjHEE8sA2/YPKnd/qwbDTuIZ4=; b=mjlOAiZRzeemq/D7wicXmj1Vi6KIWI9SRULYvXy6dhcClUWbCyd1uXRs Xb+eKEiMcUw/G4BpbL0ltlLprxH0AwHCwxUvmjY5E5zcInu4euKIdAZVA 0VBd7ACxsgpdHQ/4rUzEwLyxQ2fKYWlWRiFmVfNUAWIWFmjc1WKE18M4m vy+xFwDcxqXJBTTwNrFPjtHpUAx0qxMZMq8iC5IvwqcmVVPc8eVLDivnP /tOLQ/1wI43G7RNKctUL3/nODZEKW1o7oPMpx/rHHcczT32vsjrOZtYrA IZRJkKh6loRJqP5bCY7nXXPFrNp9ahlNOSL2MTXuJcbUUcJnb4w1ZuXab w==; X-IronPort-AV: E=Sophos;i="6.07,220,1708383600"; d="scan'208";a="11027673" X-MGA-submission: MDE2w0TDNpiD8qfN4OSreQ0o9Xf9QP4shRuEVXhdcX4Fw2ZFgB2lpunDlB3Ij+w+jYsM7/LpwVl+vtoHMSIvwDfu26Gf5iMLUYMSKzwV7JIqVIC/AwLcwdAmpr1BmM/fmzC9Rq5/psCTkhCY0xHwQwvKOM6in8BGhaMG1DL4mgFarg== Received: from mail-am0eur02lp2233.outbound.protection.outlook.com (HELO EUR02-AM0-obe.outbound.protection.outlook.com) ([104.47.11.233]) by smarthost4.eviden.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Apr 2024 17:52:57 +0200 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ogqn2IWFzIdVt27toFb8pGSjqflmcjztDm4JKpX0NrKVQl1lPqwmxt9cgES1xl05Y18fxxtOtVn3H6uLck0dHDgB6HyywjzBPMqWuTX9QcK6cOJJg6jySF7greKbiS+bFIDIffkxyB+AE3uY3ol6aXomTAfey2eMDETFoOTIzjf/UOxdw5f2Difd5lPKP7Vg6qYyf/kL//5HStcJYP2WkpjBOvmv1jWV8p/HS/y6uxkXSLjNTRAGKKXorvX1+/p6qO4URMKnwYyycdKTgrpzdNx+h/VHziB8n0NvNIO6EsHj3ke/ZGPl7Sj4IvkFfpdahHJA9joaGtIazL3LOWZW2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=A//6ZkMyDQVH+O5DOpFjHEE8sA2/YPKnd/qwbDTuIZ4=; b=M7fSYPNAo9I+shPTIX/mZS7IrZINK1qjF3wsKed/f3wylIi8eOw7Gv4lRXqZPvG1vAjtofmMtKrjw5/RwwDK8vC33+k8hQmhk9+0NnlqlXUhZHUVTA8zPkdXJaoFlkeMKNdhBWhv5AqNOQ4oswMUEbnknf0OTVvXvkbn/p7KM6Rfta2+iu6wPF2w8l3cNeWsObNLUxqkPhQa6Wzwb0YUcTze/wW7xl5Ez6E5n070CqiaIpwgm1g7jS9Lkyki4FI56OnL7UIPfD5f6mJaQw6hrePZghpEmAL4cQlCIV0pYZFESiCA+5J8epPbWx7uO45/a5hdr+y+/ilO6U1VpRKJ1g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=eviden.com; dmarc=pass action=none header.from=eviden.com; dkim=pass header.d=eviden.com; arc=none Received: from AM8PR07MB7602.eurprd07.prod.outlook.com (2603:10a6:20b:24b::7) by DU0PR07MB9626.eurprd07.prod.outlook.com (2603:10a6:10:31e::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7472.44; Mon, 22 Apr 2024 15:52:54 +0000 Received: from AM8PR07MB7602.eurprd07.prod.outlook.com ([fe80::fbd7:ca71:b636:6f9d]) by AM8PR07MB7602.eurprd07.prod.outlook.com ([fe80::fbd7:ca71:b636:6f9d%5]) with mapi id 15.20.7472.044; Mon, 22 Apr 2024 15:52:54 +0000 From: CLEMENT MATHIEU--DRIF To: "qemu-devel@nongnu.org" CC: "jasowang@redhat.com" , CLEMENT MATHIEU--DRIF Subject: [PATCH intel_iommu 4/7] intel_iommu: add support for first-stage translation Thread-Topic: [PATCH intel_iommu 4/7] intel_iommu: add support for first-stage translation Thread-Index: AQHalM0il2RzFQqe1USu5Fe1IhjH8A== Date: Mon, 22 Apr 2024 15:52:53 +0000 Message-ID: <20240422155236.129179-5-clement.mathieu--drif@eviden.com> References: <20240422155236.129179-1-clement.mathieu--drif@eviden.com> In-Reply-To: <20240422155236.129179-1-clement.mathieu--drif@eviden.com> Accept-Language: en-GB, fr-FR, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=eviden.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: AM8PR07MB7602:EE_|DU0PR07MB9626:EE_ x-ms-office365-filtering-correlation-id: 7cd75ea8-5305-411a-4d10-08dc62e44618 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: =?utf-8?q?73jeBNIk+7g42++w2G+qxAF+ZC+tH/Z?= =?utf-8?q?B8Q2SzLQ9dvaXxnXvOyYDKM+Sk91cyJ9WmV51XBc8uB+dqTKEPOwjN6g7DUETrEkl?= =?utf-8?q?h4pKAvDnwgewRwv3ti36sHlnehs+PxEVOh2AVQFzIUorw9wF5qXXDLVKYUTTug6Tk?= =?utf-8?q?5CVK2is8z9Tceq2XCydmKrutbyYYokUXTH/vYNsOuxraa7gAGlQfVFPEAMtxnVAGa?= =?utf-8?q?oJseunft78BM3HGrc4naxyIx6Z/ENpujOlguZarYb4/iQaS16lDsPZA/l4Gt9YVfo?= =?utf-8?q?SpSZ1QxVwVwigCYCySnViAIs47IWtbvCMdV+sVTZImnAp95EZOQsOgHCW9iXBDCsJ?= =?utf-8?q?rapbxkR/QVK2jNjtrctf49IlKzc0v6ml7D42EGdUe3fNa6VM+Jgp0RGO/MWyY9G+i?= =?utf-8?q?z3yyS0nsZ5AU5bgv6ZWE79XEEadO+ifaWyf5iEOyZo+6M6xE7ghCXxZ4bCw/59zLz?= =?utf-8?q?2AKioO13TFZtQEP4RNdAr7+qbwr+ex6ctmHBpf9JVYCJeHmdr/NLgI+puNhIdo2rM?= =?utf-8?q?KJO1kbKAvPkQNLn1u/YepuUax04UfdTpBc8Sq6NxSufIHbt+tPueEulrzj1Pd8tA9?= =?utf-8?q?np4X8ypyMUGDxnOo99hLTV5FiLrTkSOrjEq6hs6xVKvUQWpwNbpShtwFe1aZduLZe?= =?utf-8?q?O/Pxu+YLsojPHRPj9/6nYcATs2Sd/w8jpDbrrmdwns9SJ6D+kCSQKI1WDcGepgKL8?= =?utf-8?q?vwPbHX+cmWJ7AjSRDoAtJ8TwKEtyte74cjo9cLMhyA/wMb/8As3lnBQG4gMSjPrp9?= =?utf-8?q?Hmx4jqQit+g1DlDzShdMoyTJdxMmxeQ7eVTLH0FOTvJREXie2BbQLuPOSQ/30rucS?= =?utf-8?q?1cvR8QtbWhTvGNl5hSvRZKrT+pJBBvMHplOeEx8AdSdgUMiVBtE4tmTlKiPhO2FEn?= =?utf-8?q?QDt2otVilkihtaCfbObqQ3ere2DJHzsLj6iY8VG5eOIsJd1dM2mUw1uaCmDzkXEPh?= =?utf-8?q?onclT2XKsyU0KbHiIEuz7GxSEnFMp03MINKmVePth7mfanYglZhB1woT38Yzio1Bj?= =?utf-8?q?xpwpiZKDvp8Oh3FCDM9YR44Rhz4GtUNoT1vzYo1ocUHrLNPabHpgz02GMeX2C23Yp?= =?utf-8?q?z/b1RPksKNPa/bw+KBIuoWEJz0bgW0jnjyf7cAP+NrUNzIvpaa1nQtFr1y5QZO78C?= =?utf-8?q?k1wNXcGnVe0wKQaLSL2uwL6EHgF+I2aGjfPii4bQeI7PYZDk3vnSnfBuU/m/supn+?= =?utf-8?q?ySNxXteKlnWcg3zFjSYlU4+5nTJi2O3jrHgPbzm0brLrbf6rVmMxiNH0WuIDWcuYR?= =?utf-8?q?qbwRl2bH4ReLu?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM8PR07MB7602.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(1800799015)(366007)(376005)(38070700009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?2XOQpoKG1q9igqz8tJwM2Mf1nPsE?= =?utf-8?q?xrely2m1r1aVvW5MkTWmxJ2FY1Z+/KO9PiwRv3ZyFjMlaPaHPKPMIonNTGAWOMP8+?= =?utf-8?q?rKwehdlOkuvysmzMFAZwpGCDcXuxabLiAqdEYgB9IeActig4Yypiugk9ZyKa+35Ta?= =?utf-8?q?4YgVs746wb+bFmZy9gJvuAe7VPUjQuDF2FsnkV36BoObPnjyQEI8SJirrXvFjoG3W?= =?utf-8?q?0bvtdif7BbOPpidhus2zmgZvmAEI/HzC37QRQRJCbp+spnvFBL7a8hyExbr3FuL/e?= =?utf-8?q?kAXQ3Id/6G96ntJywTi4TO3jtEOoEAJqaG8LQqzmP03QwiL8257+O/rrxM1OpQc6K?= =?utf-8?q?lVyvhVsbqrRbl3cQTjZEfXycOaeSl253SLbGDzUN54G40V0gKMrlrbCDZpVD7tjtQ?= =?utf-8?q?dxmxeS4hbRB4jGbqiy7pZaqxxkbrWX+UWku3D9MAggCJ+/zweWXi7s3NOmflq2gBM?= =?utf-8?q?fgC3k6rV8Jx9btPA+1ThUF6IOlW4lzMOnXNk7UlzEOLMUKWs3ze2TSGmgdeILXAmH?= =?utf-8?q?MpLbH4diYjJxAQlwVnfUA8TJuad2oi9KisCgSDFknQPXbY2AeYrfnpdNrHYqMosEf?= =?utf-8?q?ykoQm/02X+7tLHLGf5BYg8ViRTJj5aGyYulOXivX2Db3fDZ9ZuTgjpBXG+azyH3tL?= =?utf-8?q?OiRhLhem0yZKI429QA7ZwGDPwQTvnS3Nmf14Umx422fGX4QTNsyrWYUjbMHu2xQkd?= =?utf-8?q?YMNdoVT4CAM3RTm7Mj9hjl1OQhTSBMAOMd+eEB0zHRROZZHyZJJCOpK6D8d37A8mT?= =?utf-8?q?GBLUMw7TjK8QWsmbGJVR3WXvTHsmcTKpdipsFvba+WuPcz9hIjv+BUVMyYtrXqxR3?= =?utf-8?q?1A3O915NolG7X5malrhySUFBqK8z1xXGoTtmMHjBfAOCvdKnUredlUiDnCAV+g2Dl?= =?utf-8?q?UgZO3k2+6CFlkbtn5JOh+84XgUEWkn343rxEv2FnhSZghf2eFRND32tF036Hxrypc?= =?utf-8?q?LG8MO9GpuG8mDCQ6eg1VjMqnz/Y3EEpIwLnNdV/r2qGbtF0D0CYKUJGqylb6zqcau?= =?utf-8?q?VfU067A1/09HL21PWZEtX0ZvY/rWZsVbZmE5Rf1oLfWj+u7XtJmH0ClnMKQa0Aji7?= =?utf-8?q?DEr9Q/qNpiTmjzHDAkrSk2L4ex/1Pz8cudFvUl3LCKYoHmAXvkhbAidsC3tS2x5pK?= =?utf-8?q?sP2HLdoSjQq0wsrIW72Rsac5WF5mzKjA983Tyiegdgn0nPt6ZqOODCokD9Efzskxr?= =?utf-8?q?6pR9IyOIeo/sgQRfE4xY6j/2jeNJUdiA0SR3mkzqfTpKPd4iuut7BmcNb4Ii9SNzg?= =?utf-8?q?KjC1N7bnE9w4HZWJgR6znd/TRMnYlEHWWjBo8WZyFHwnjCxsvK9k71W7MUPYDPTy1?= =?utf-8?q?+3FuMt9CFkfL76uvbtg8YZV81pSqyNjKsmpG1VF1qsn4Oqcg0XVVUCuml/XFlY6gB?= =?utf-8?q?eZTEuF7uhODOp/Z93aQXWD98oog/sq2vzkUlWmJ+wR/zjWuDj7FVCRICPj5a1huVY?= =?utf-8?q?5Y2u8Wj77pLjwZq8sOXy/IA12Heh72hI8Eh5hsVsWxzN5utpBS/FllUdMu5l1JHKl?= =?utf-8?q?FL8BvJEZ4khb/nHvjTcybDFXgUBMwd0oQNXU90moq4/6CGn/1Cynmm4=3D?= Content-ID: <717C8F90C9DBAB4DAE6921D2483745CC@eurprd07.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: eviden.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM8PR07MB7602.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7cd75ea8-5305-411a-4d10-08dc62e44618 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Apr 2024 15:52:53.3583 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7d1c7785-2d8a-437d-b842-1ed5d8fbe00a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: TSFiVvwbIrYVfzSeAJRYfPdEdOU9YmACt0vlr1t8CtqCsM6Szrt5+BUSC7R6JrObkhJSXJjBXUCnUiScYOyIB4ZksV9Grjq4uLJ+UR+7OH7GKkZ/M7/nQ59fBRTPF79r X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR07MB9626 Received-SPF: pass client-ip=80.78.11.85; envelope-from=clement.mathieu--drif@eviden.com; helo=smarthost4.eviden.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This translation mode will only be made available in scalable mode Signed-off-by: Clément Mathieu--Drif --- hw/i386/intel_iommu.c | 364 ++++++++++++++++++++++++++++----- hw/i386/intel_iommu_internal.h | 51 ++++- 2 files changed, 362 insertions(+), 53 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index ba545590b1..3b9f120dec 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -713,6 +713,21 @@ static uint64_t vtd_get_pte(dma_addr_t base_addr, uint32_t index) return pte; } +static MemTxResult vtd_set_flag_in_pte(dma_addr_t base_addr, uint32_t index, + uint64_t pte, uint64_t flag) +{ + assert(index < VTD_PT_ENTRY_NR); + if (pte & flag) { + return MEMTX_OK; + } + pte |= flag; + pte = cpu_to_le64(pte); + return dma_memory_write(&address_space_memory, + base_addr + index * sizeof(pte), + &pte, sizeof(pte), + MEMTXATTRS_UNSPECIFIED); +} + /* Given an iova and the level of paging structure, return the offset * of current level. */ @@ -730,11 +745,17 @@ static inline bool vtd_is_level_supported(IntelIOMMUState *s, uint32_t level) } /* Return true if check passed, otherwise false */ -static inline bool vtd_pe_type_check(X86IOMMUState *x86_iommu, +static inline bool vtd_pe_type_check(IntelIOMMUState *s, VTDPASIDEntry *pe) { + X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s); + switch (VTD_PE_GET_TYPE(pe)) { case VTD_SM_PASID_ENTRY_FLT: + if (!(s->ecap & VTD_ECAP_FLTS)) { + return false; + } + break; case VTD_SM_PASID_ENTRY_SLT: case VTD_SM_PASID_ENTRY_NESTED: break; @@ -784,6 +805,11 @@ static inline bool vtd_pe_present(VTDPASIDEntry *pe) return pe->val[0] & VTD_PASID_ENTRY_P; } +static inline bool vtd_fl_pte_present(uint64_t pte) +{ + return pte & VTD_FL_PTE_P; +} + static int vtd_get_pe_in_pasid_leaf_table(IntelIOMMUState *s, uint32_t pasid, dma_addr_t addr, @@ -791,7 +817,6 @@ static int vtd_get_pe_in_pasid_leaf_table(IntelIOMMUState *s, { uint32_t index; dma_addr_t entry_size; - X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s); index = VTD_PASID_TABLE_INDEX(pasid); entry_size = VTD_PASID_ENTRY_SIZE; @@ -805,7 +830,7 @@ static int vtd_get_pe_in_pasid_leaf_table(IntelIOMMUState *s, } /* Do translation type check */ - if (!vtd_pe_type_check(x86_iommu, pe)) { + if (!vtd_pe_type_check(s, pe)) { return -VTD_FR_PASID_TABLE_INV; } @@ -1027,6 +1052,34 @@ static inline bool vtd_iova_sl_range_check(IntelIOMMUState *s, return !(iova & ~(vtd_iova_limit(s, ce, aw, pasid) - 1)); } +/* Return true if IOVA is canonical, otherwise false. */ +static bool vtd_iova_fl_check_canonical(IntelIOMMUState *s, + uint64_t iova, VTDContextEntry *ce, + uint8_t aw, uint32_t pasid) +{ + uint64_t iova_limit = vtd_iova_limit(s, ce, aw, pasid); + uint64_t upper_bits_mask = ~(iova_limit - 1); + uint64_t upper_bits = iova & upper_bits_mask; + bool msb = ((iova & (iova_limit >> 1)) != 0); + return !( + (!msb && (upper_bits != 0)) || + (msb && (upper_bits != upper_bits_mask)) + ); +} + +/* Return the page table base address corresponding to the translation type. */ +static dma_addr_t vtd_pe_get_pgtbl_base(VTDPASIDEntry *pe) +{ + uint16_t pgtt = VTD_PE_GET_TYPE(pe); + if (pgtt == VTD_SM_PASID_ENTRY_FLT) { + return pe->val[2] & VTD_SM_PASID_ENTRY_PTPTR; + } else if (pgtt == VTD_SM_PASID_ENTRY_SLT) { + return pe->val[0] & VTD_SM_PASID_ENTRY_PTPTR; + } + + return 0; /* Not supported */ +} + static dma_addr_t vtd_get_iova_pgtbl_base(IntelIOMMUState *s, VTDContextEntry *ce, uint32_t pasid) @@ -1035,7 +1088,7 @@ static dma_addr_t vtd_get_iova_pgtbl_base(IntelIOMMUState *s, if (s->root_scalable) { vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid); - return pe.val[0] & VTD_SM_PASID_ENTRY_SLPTPTR; + return vtd_pe_get_pgtbl_base(&pe); } return vtd_ce_get_slpt_base(ce); @@ -1053,6 +1106,10 @@ static dma_addr_t vtd_get_iova_pgtbl_base(IntelIOMMUState *s, static uint64_t vtd_spte_rsvd[VTD_SPTE_RSVD_LEN]; static uint64_t vtd_spte_rsvd_large[VTD_SPTE_RSVD_LEN]; +#define VTD_FPTE_RSVD_LEN 5 +static uint64_t vtd_fpte_rsvd[VTD_FPTE_RSVD_LEN]; +static uint64_t vtd_fpte_rsvd_large[VTD_FPTE_RSVD_LEN]; + static bool vtd_slpte_nonzero_rsvd(uint64_t slpte, uint32_t level) { uint64_t rsvd_mask; @@ -1079,21 +1136,140 @@ static bool vtd_slpte_nonzero_rsvd(uint64_t slpte, uint32_t level) return slpte & rsvd_mask; } -/* Given the @iova, get relevant @slptep. @slpte_level will be the last level - * of the translation, can be used for deciding the size of large page. - */ -static int vtd_iova_to_slpte(IntelIOMMUState *s, VTDContextEntry *ce, - uint64_t iova, bool is_write, - uint64_t *slptep, uint32_t *slpte_level, - bool *reads, bool *writes, uint8_t aw_bits, - uint32_t pasid) +static bool vtd_flpte_nonzero_rsvd(uint64_t flpte, uint32_t level) +{ + uint64_t rsvd_mask; + assert(level < VTD_FPTE_RSVD_LEN); + assert(level); + + if ((level == VTD_FL_PD_LEVEL || level == VTD_FL_PDP_LEVEL) && + (flpte & VTD_PT_PAGE_SIZE_MASK)) { + /* large page */ + rsvd_mask = vtd_fpte_rsvd_large[level]; + } else { + rsvd_mask = vtd_fpte_rsvd[level]; + } + + return flpte & rsvd_mask; +} + +static int vtd_iova_to_pte_check_read_error(IntelIOMMUState *s, + VTDContextEntry *ce, uint64_t iova, + uint64_t pte, uint32_t pasid, + uint32_t level, uint16_t pgtt) +{ + if (pte == (uint64_t)-1) { + error_report_once("%s: detected read error on DMAR pte " + "(iova=0x%" PRIx64 ", pasid=0x%" PRIx32 ")", + __func__, iova, pasid); + if (level == vtd_get_iova_level(s, ce, pasid)) { + /* Invalid programming of context-entry */ + if (s->root_scalable) { + return pgtt == VTD_SM_PASID_ENTRY_FLT ? + -VTD_FR_FIRST_FSPE_ACCESS : + -VTD_FR_FIRST_SSPE_ACCESS; + } else { + return -VTD_FR_CONTEXT_ENTRY_INV; + } + } else { + if (s->root_scalable) { + return pgtt == VTD_SM_PASID_ENTRY_FLT ? + -VTD_FR_FSPE_ACCESS : + -VTD_FR_SSPE_ACCESS; + } else { + return -VTD_FR_PAGING_ENTRY_INV; + } + } + } + + return 0; +} + +static inline bool vtd_addr_in_interrup_range(hwaddr addr, uint64_t size) +{ + return !((addr > VTD_INTERRUPT_ADDR_LAST) || + (addr + size - 1 < VTD_INTERRUPT_ADDR_FIRST)); +} + +static int vtd_iova_to_pte_fl(IntelIOMMUState *s, VTDContextEntry *ce, + uint64_t iova, bool is_write, uint64_t *ptep, + uint32_t *pte_level, bool *reads, bool *writes, + uint8_t aw_bits, uint32_t pasid, dma_addr_t addr) +{ + uint32_t offset; + uint64_t pte; + uint64_t access_right_check = is_write ? VTD_FL_W : 0; + int ret; + + if (!vtd_iova_fl_check_canonical(s, iova, ce, aw_bits, pasid)) { + error_report_once("%s: detected non canonical IOVA (iova=0x%" PRIx64 "," + "pasid=0x%" PRIx32 ")", __func__, iova, pasid); + return -VTD_FR_FS_NON_CANONICAL; + } + + while (true) { + offset = vtd_iova_level_offset(iova, *pte_level); + pte = vtd_get_pte(addr, offset); + + ret = vtd_iova_to_pte_check_read_error(s, ce, iova, pte, + pasid, *pte_level, + VTD_SM_PASID_ENTRY_FLT); + if (ret != 0) { + return ret; + } + + if (!vtd_fl_pte_present(pte)) { + return -VTD_FR_FSPE_NOT_PRESENT; + } + + *reads = true; + *writes = (*writes) && (pte & VTD_FL_W); + + if (vtd_set_flag_in_pte(addr, offset, pte, VTD_FL_PTE_A) != MEMTX_OK) { + return -VTD_FR_FS_BIT_UPDATE_FAILED; + } + + if ((pte & access_right_check) != access_right_check) { + error_report_once("%s: detected pte permission error " + "(iova=0x%" PRIx64 ", level=0x%" PRIx32 ", " + "pte=0x%" PRIx64 ", write=%d, pasid=0x%" + PRIx32 ")", __func__, iova, *pte_level, + pte, is_write, pasid); + return is_write ? -VTD_FR_SM_WRITE : -VTD_FR_SM_READ; + } + if (vtd_flpte_nonzero_rsvd(pte, *pte_level)) { + error_report_once("%s: detected flpte reserved non-zero " + "iova=0x%" PRIx64 ", level=0x%" PRIx32 + "pte=0x%" PRIx64 ", pasid=0x%" PRIX32 ")", + __func__, iova, *pte_level, pte, pasid); + return -VTD_FR_PAGING_ENTRY_RSVD; + } + + if (vtd_is_last_pte(pte, *pte_level)) { + *ptep = pte; + break; + } + addr = vtd_get_pte_addr(pte, aw_bits); + (*pte_level)--; + } + + if (is_write && + (vtd_set_flag_in_pte(addr, offset, pte, VTD_FL_PTE_D) != MEMTX_OK)) { + return -VTD_FR_FS_BIT_UPDATE_FAILED; + } + + return 0; +} + +static int vtd_iova_to_pte_sl(IntelIOMMUState *s, VTDContextEntry *ce, + uint64_t iova, bool is_write, uint64_t *slptep, + uint32_t *pte_level, bool *reads, bool *writes, + uint8_t aw_bits, uint32_t pasid, dma_addr_t addr) { - dma_addr_t addr = vtd_get_iova_pgtbl_base(s, ce, pasid); - uint32_t level = vtd_get_iova_level(s, ce, pasid); uint32_t offset; uint64_t slpte; - uint64_t access_right_check; - uint64_t xlat, size; + uint64_t access_right_check = is_write ? VTD_SL_W : VTD_SL_R; + int ret; if (!vtd_iova_sl_range_check(s, iova, ce, aw_bits, pasid)) { error_report_once("%s: detected IOVA overflow (iova=0x%" PRIx64 "," @@ -1101,72 +1277,128 @@ static int vtd_iova_to_slpte(IntelIOMMUState *s, VTDContextEntry *ce, return -VTD_FR_ADDR_BEYOND_MGAW; } - /* FIXME: what is the Atomics request here? */ - access_right_check = is_write ? VTD_SL_W : VTD_SL_R; - while (true) { - offset = vtd_iova_level_offset(iova, level); + offset = vtd_iova_level_offset(iova, *pte_level); slpte = vtd_get_pte(addr, offset); - if (slpte == (uint64_t)-1) { - error_report_once("%s: detected read error on DMAR slpte " - "(iova=0x%" PRIx64 ", pasid=0x%" PRIx32 ")", - __func__, iova, pasid); - if (level == vtd_get_iova_level(s, ce, pasid)) { - /* Invalid programming of context-entry */ - return -VTD_FR_CONTEXT_ENTRY_INV; - } else { - return -VTD_FR_PAGING_ENTRY_INV; - } + ret = vtd_iova_to_pte_check_read_error(s, ce, iova, slpte, + pasid, *pte_level, + VTD_SM_PASID_ENTRY_SLT); + if (ret != 0) { + return ret; } *reads = (*reads) && (slpte & VTD_SL_R); *writes = (*writes) && (slpte & VTD_SL_W); - if (!(slpte & access_right_check)) { + if ((slpte & access_right_check) != access_right_check) { error_report_once("%s: detected slpte permission error " "(iova=0x%" PRIx64 ", level=0x%" PRIx32 ", " "slpte=0x%" PRIx64 ", write=%d, pasid=0x%" - PRIx32 ")", __func__, iova, level, + PRIx32 ")", __func__, iova, *pte_level, slpte, is_write, pasid); - return is_write ? -VTD_FR_WRITE : -VTD_FR_READ; + if (s->root_scalable) { + return is_write ? -VTD_FR_SM_WRITE : -VTD_FR_SM_READ; + } else { + return is_write ? -VTD_FR_WRITE : -VTD_FR_READ; + } } - if (vtd_slpte_nonzero_rsvd(slpte, level)) { + if (vtd_slpte_nonzero_rsvd(slpte, *pte_level)) { error_report_once("%s: detected splte reserve non-zero " "iova=0x%" PRIx64 ", level=0x%" PRIx32 "slpte=0x%" PRIx64 ", pasid=0x%" PRIX32 ")", - __func__, iova, level, slpte, pasid); + __func__, iova, *pte_level, slpte, pasid); return -VTD_FR_PAGING_ENTRY_RSVD; } - if (vtd_is_last_pte(slpte, level)) { + if (vtd_is_last_pte(slpte, *pte_level)) { *slptep = slpte; - *slpte_level = level; break; } addr = vtd_get_pte_addr(slpte, aw_bits); - level--; + (*pte_level)--; + } + + return 0; +} + +/* + * Given the @iova, get relevant @ptep. @pte_level will be the last level + * of the translation, can be used for deciding the size of large page. + */ +static int vtd_iova_to_pte(IntelIOMMUState *s, VTDContextEntry *ce, + uint64_t iova, bool is_write, + uint64_t *ptep, uint32_t *pte_level, + bool *reads, bool *writes, uint8_t aw_bits, + uint32_t pasid) +{ + dma_addr_t addr; + uint16_t pgtt; + uint64_t xlat, size; + VTDPASIDEntry pe; + int ret; + + /* FIXME: what is the Atomics request in access rights? */ + + if (s->root_scalable) { + vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid); + pgtt = VTD_PE_GET_TYPE(&pe); + *pte_level = VTD_PE_GET_LEVEL(&pe); + addr = vtd_pe_get_pgtbl_base(&pe); + switch (pgtt) { + case VTD_SM_PASID_ENTRY_FLT: + if (s->ecap & VTD_ECAP_FLTS) { + ret = vtd_iova_to_pte_fl(s, ce, iova, is_write, ptep, pte_level, + reads, writes, aw_bits, pasid, addr); + } else { + error_report_once("First-stage translation not supported : %d", + pgtt); + return -VTD_FR_INVALID_PGTT; + } + break; + case VTD_SM_PASID_ENTRY_SLT: + if (s->ecap & VTD_ECAP_SLTS) { + ret = vtd_iova_to_pte_sl(s, ce, iova, is_write, ptep, pte_level, + reads, writes, aw_bits, pasid, addr); + } else { + error_report_once("Second-stage translation not supported : %d", + pgtt); + return -VTD_FR_INVALID_PGTT; + } + break; + default: + error_report_once("Unsupported PGTT : %d", pgtt); + ret = -VTD_FR_INVALID_PGTT; + break; + } + } else { + *pte_level = vtd_ce_get_level(ce); + addr = vtd_ce_get_slpt_base(ce); + ret = vtd_iova_to_pte_sl(s, ce, iova, is_write, ptep, pte_level, + reads, writes, aw_bits, pasid, addr); } - xlat = vtd_get_pte_addr(*slptep, aw_bits); - size = ~vtd_pt_level_page_mask(level) + 1; + if (ret != 0) { + return ret; + } + + xlat = vtd_get_pte_addr(*ptep, aw_bits); + size = ~vtd_pt_level_page_mask(*pte_level) + 1; /* * From VT-d spec 3.14: Untranslated requests and translation * requests that result in an address in the interrupt range will be * blocked with condition code LGN.4 or SGN.8. */ - if ((xlat > VTD_INTERRUPT_ADDR_LAST || - xlat + size - 1 < VTD_INTERRUPT_ADDR_FIRST)) { + if (!vtd_addr_in_interrup_range(xlat, size)) { return 0; } else { error_report_once("%s: xlat address is in interrupt range " "(iova=0x%" PRIx64 ", level=0x%" PRIx32 ", " - "slpte=0x%" PRIx64 ", write=%d, " + "pte=0x%" PRIx64 ", write=%d, " "xlat=0x%" PRIx64 ", size=0x%" PRIx64 ", " "pasid=0x%" PRIx32 ")", - __func__, iova, level, slpte, is_write, + __func__, iova, *pte_level, *ptep, is_write, xlat, size, pasid); - return s->scalable_mode ? -VTD_FR_SM_INTERRUPT_ADDR : - -VTD_FR_INTERRUPT_ADDR; + return -VTD_INTERRUPT_RANGE_ERROR(s); } } @@ -1772,6 +2004,16 @@ static const bool vtd_qualified_faults[] = { [VTD_FR_CONTEXT_ENTRY_TT] = true, [VTD_FR_PASID_TABLE_INV] = false, [VTD_FR_SM_INTERRUPT_ADDR] = true, + [VTD_FR_INVALID_PGTT] = true, + [VTD_FR_FSPE_ACCESS] = true, + [VTD_FR_FSPE_NOT_PRESENT] = true, + [VTD_FR_FIRST_FSPE_ACCESS] = true, + [VTD_FR_SSPE_ACCESS] = true, + [VTD_FR_FIRST_SSPE_ACCESS] = true, + [VTD_FR_FS_NON_CANONICAL] = true, + [VTD_FR_SM_WRITE] = true, + [VTD_FR_SM_READ] = true, + [VTD_FR_FS_BIT_UPDATE_FAILED] = true, [VTD_FR_MAX] = false, }; @@ -1870,7 +2112,8 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, uint8_t bus_num = pci_bus_num(bus); VTDContextCacheEntry *cc_entry; uint64_t pte, page_mask; - uint32_t level, pasid = vtd_as->pasid; + uint32_t level = UINT32_MAX; + uint32_t pasid = vtd_as->pasid; uint16_t source_id = PCI_BUILD_BDF(bus_num, devfn); int ret_fr; bool is_fpd_set = false; @@ -1981,7 +2224,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, } } - ret_fr = vtd_iova_to_slpte(s, &ce, addr, is_write, &pte, &level, + ret_fr = vtd_iova_to_pte(s, &ce, addr, is_write, &pte, &level, &reads, &writes, s->aw_bits, pasid); if (ret_fr) { vtd_report_fault(s, -ret_fr, is_fpd_set, source_id, @@ -1990,6 +2233,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, } page_mask = vtd_pt_level_page_mask(level); + /* Exe is always set to 0 in the spec */ access_flags = IOMMU_ACCESS_FLAG(reads, writes); vtd_update_iotlb(s, source_id, vtd_get_domain_id(s, &ce, pasid), addr, pte, access_flags, level, pasid); @@ -2005,7 +2249,12 @@ error: vtd_iommu_unlock(s); entry->iova = 0; entry->translated_addr = 0; - entry->addr_mask = 0; + /* + * Set the mask for ATS (the range must be present even when the + * translation fails : PCIe rev 5 10.2.3.5) + */ + entry->addr_mask = (level != UINT32_MAX) ? + (~vtd_pt_level_page_mask(level)) : (~VTD_PAGE_MASK_4K); entry->perm = IOMMU_NONE; return false; } @@ -3187,6 +3436,8 @@ static IOMMUTLBEntry vtd_iommu_translate(IOMMUMemoryRegion *iommu, hwaddr addr, }; bool success; + /* IOMMUAccessFlags : IOMMU_EXEC and IOMMU_PRIV not supported */ + if (likely(s->dmar_enabled)) { success = vtd_do_iommu_translate(vtd_as, vtd_as->bus, vtd_as->devfn, addr, flag & IOMMU_WO, &iotlb); @@ -3989,6 +4240,21 @@ static void vtd_init(IntelIOMMUState *s) vtd_spte_rsvd_large[3] = VTD_SPTE_LPAGE_L3_RSVD_MASK(s->aw_bits, x86_iommu->dt_supported); + /* + * Rsvd field masks for fpte + */ + vtd_fpte_rsvd[0] = ~0ULL; + vtd_fpte_rsvd[1] = VTD_FPTE_PAGE_L1_RSVD_MASK(s->aw_bits); + vtd_fpte_rsvd[2] = VTD_FPTE_PAGE_L2_RSVD_MASK(s->aw_bits); + vtd_fpte_rsvd[3] = VTD_FPTE_PAGE_L3_RSVD_MASK(s->aw_bits); + vtd_fpte_rsvd[4] = VTD_FPTE_PAGE_L4_RSVD_MASK(s->aw_bits); + + vtd_fpte_rsvd_large[0] = ~0ULL; + vtd_fpte_rsvd_large[1] = ~0ULL; + vtd_fpte_rsvd_large[2] = VTD_FPTE_PAGE_L2_FS2MP_RSVD_MASK(s->aw_bits); + vtd_fpte_rsvd_large[3] = VTD_FPTE_PAGE_L3_FS1GP_RSVD_MASK(s->aw_bits); + vtd_fpte_rsvd_large[4] = ~0ULL; + if (s->scalable_mode || s->snoop_control) { vtd_spte_rsvd[1] &= ~VTD_SPTE_SNP; vtd_spte_rsvd_large[2] &= ~VTD_SPTE_SNP; diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 8d27b1c15b..ed61979934 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -195,6 +195,7 @@ #define VTD_ECAP_PASID (1ULL << 40) #define VTD_ECAP_SMTS (1ULL << 43) #define VTD_ECAP_SLTS (1ULL << 46) +#define VTD_ECAP_FLTS (1ULL << 47) /* CAP_REG */ /* (offset >> 4) << 24 */ @@ -215,6 +216,7 @@ #define VTD_CAP_CM (1ULL << 7) #define VTD_PASID_ID_SHIFT 20 #define VTD_PASID_ID_MASK ((1ULL << VTD_PASID_ID_SHIFT) - 1) +#define VTD_CAP_FS1GP (1ULL << 56) /* Supported Adjusted Guest Address Widths */ #define VTD_CAP_SAGAW_SHIFT 8 @@ -270,6 +272,10 @@ #define VTD_FRCD_PP(val) (((val) & 0x1ULL) << 31) #define VTD_FRCD_IR_IDX(val) (((val) & 0xffffULL) << 48) +#define VTD_INTERRUPT_RANGE_ERROR(s) ((s)->scalable_mode ? \ + VTD_FR_SM_INTERRUPT_ADDR : \ + VTD_FR_INTERRUPT_ADDR) + /* DMA Remapping Fault Conditions */ typedef enum VTDFaultReason { VTD_FR_RESERVED = 0, /* Reserved for Advanced Fault logging */ @@ -312,9 +318,21 @@ typedef enum VTDFaultReason { VTD_FR_IR_SID_ERR = 0x26, /* Invalid Source-ID */ VTD_FR_PASID_TABLE_INV = 0x58, /*Invalid PASID table entry */ + VTD_FR_INVALID_PGTT = 0x5b, /* SPT.4.2 Invalid PGTT */ + + VTD_FR_FSPE_ACCESS = 0x70, /* SFS.1 */ + VTD_FR_FSPE_NOT_PRESENT = 0x71, /* SFS.2 : Present (P) field is 0.*/ + VTD_FR_FIRST_FSPE_ACCESS = 0x73, /* SFS.4 */ + VTD_FR_SSPE_ACCESS = 0x78, /* SSS.1 */ + VTD_FR_FIRST_SSPE_ACCESS = 0x7b, /* SSS.4 */ + VTD_FR_FS_NON_CANONICAL = 0x80, /* SNG.1 : Address for FS not canonical.*/ + + VTD_FR_SM_WRITE = 0x85, /* SGN.6 */ + VTD_FR_SM_READ = 0x86, /* SGN.7 */ /* Output address in the interrupt address range for scalable mode */ - VTD_FR_SM_INTERRUPT_ADDR = 0x87, + VTD_FR_SM_INTERRUPT_ADDR = 0x87, /* SGN.8 */ + VTD_FR_FS_BIT_UPDATE_FAILED = 0x91, /* SFS.10 */ VTD_FR_MAX, /* Guard */ } VTDFaultReason; @@ -431,6 +449,23 @@ typedef union VTDInvDesc VTDInvDesc; (0x3ffff800ULL | ~(VTD_HAW_MASK(aw) | VTD_SL_IGN_COM | VTD_SL_TM)) : \ (0x3ffff800ULL | ~(VTD_HAW_MASK(aw) | VTD_SL_IGN_COM)) +#define VTD_FS_UPPER_IGNORED 0xfff0000000000000ULL +#define VTD_FPTE_PAGE_L1_RSVD_MASK(aw) (~(VTD_HAW_MASK(aw)) & \ + (~VTD_FS_UPPER_IGNORED)) +#define VTD_FPTE_PAGE_L2_RSVD_MASK(aw) (~(VTD_HAW_MASK(aw)) & \ + (~VTD_FS_UPPER_IGNORED)) +#define VTD_FPTE_PAGE_L3_RSVD_MASK(aw) (~(VTD_HAW_MASK(aw)) & \ + (~VTD_FS_UPPER_IGNORED)) +#define VTD_FPTE_PAGE_L3_FS1GP_RSVD_MASK(aw) ((0x3fffe000ULL | \ + ~(VTD_HAW_MASK(aw))) \ + & (~VTD_FS_UPPER_IGNORED)) +#define VTD_FPTE_PAGE_L2_FS2MP_RSVD_MASK(aw) ((0x1fe000ULL | \ + ~(VTD_HAW_MASK(aw))) \ + & (~VTD_FS_UPPER_IGNORED)) +#define VTD_FPTE_PAGE_L4_RSVD_MASK(aw) ((0x80ULL | \ + ~(VTD_HAW_MASK(aw))) \ + & (~VTD_FS_UPPER_IGNORED)) + /* Information about page-selective IOTLB invalidate */ struct VTDIOTLBPageInvInfo { uint16_t domain_id; @@ -514,9 +549,6 @@ typedef struct VTDRootEntry VTDRootEntry; #define VTD_SM_PASID_ENTRY_AW 7ULL /* Adjusted guest-address-width */ #define VTD_SM_PASID_ENTRY_DID(val) ((val) & VTD_DOMAIN_ID_MASK) -/* Second Level Page Translation Pointer*/ -#define VTD_SM_PASID_ENTRY_SLPTPTR (~0xfffULL) - /* Paging Structure common */ #define VTD_SM_PASID_ENTRY_PTPTR (~0xfffULL) #define VTD_PT_PAGE_SIZE_MASK (1ULL << 7) @@ -532,6 +564,14 @@ typedef struct VTDRootEntry VTDRootEntry; #define VTD_SL_PD_LEVEL 2 #define VTD_SL_PT_LEVEL 1 +/* First Level Paging Structure */ +#define VTD_FL_PDP_LEVEL 3 +#define VTD_FL_PD_LEVEL 2 +#define VTD_FL_PTE_P 0x1 +#define VTD_FL_PTE_A 0x20 +#define VTD_FL_PTE_D 0x40 +#define VTD_FL_PTE_EA 0x400 + /* Masks for Second Level Paging Entry */ #define VTD_SL_RW_MASK 3ULL #define VTD_SL_R 1ULL @@ -539,4 +579,7 @@ typedef struct VTDRootEntry VTDRootEntry; #define VTD_SL_IGN_COM 0xbff0000000000000ULL #define VTD_SL_TM (1ULL << 62) +/* Masks for First Level Paging Entry */ +#define VTD_FL_W (1ULL << 1) + #endif