From patchwork Fri Nov 29 07:43:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: CLEMENT MATHIEU--DRIF X-Patchwork-Id: 13888456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 67B1FD6EBE5 for ; Fri, 29 Nov 2024 07:48:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tGvfm-00083j-4l; Fri, 29 Nov 2024 02:44:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tGvfe-0007zT-7x for qemu-devel@nongnu.org; Fri, 29 Nov 2024 02:44:11 -0500 Received: from smarthost2.eviden.com ([80.78.11.83]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tGvfb-0002nM-0H for qemu-devel@nongnu.org; Fri, 29 Nov 2024 02:44:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=eviden.com; i=@eviden.com; q=dns/txt; s=mail; t=1732866247; x=1764402247; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=lKFYdcossLRHzL2m6cO8jzSLlEUPo1FKRfk/C2cEJBY=; b=JCigumQ7ryzPAgOhNtxGccSP0NG4R5pI9u49yyB5l9VlsXClZf1Wkk3X 34mXIkqvOyWyr1if9Ngr3mJHc/vwitmcNMo/YabpufKSgmFV2gjTHUrbx Gbntag19siczOOsHwdWIzQwIbWTQjRe3Nj2fH8aGC7E81jjB618EaMvoT ITZRr0Fx4YPMN8jVpWzMGOpPvrUQj8Dm+t5W3nTPr4VnvtkCjlRRzj/Vz FYPkoX6NvepJsm2v5Z25sjc+NDASLGHcsDASt6j4o4zBSaUbC+6hPIdCJ P/qb9XY7+v8YwvLz7AfdSFr+CYVPJYmjzi52a/n5zRKWbIYpp9f/2JxyL g==; X-IronPort-AV: E=Sophos;i="6.12,194,1728943200"; d="scan'208";a="25670811" X-MGA-submission: MDGo7LgHrwqw3jDPn6WpDqwxKaBmucBmi77fnmNbIxiLI5obCk6RsYBsPbqLlxt29aAeM33YFAlR6DqVFKuqzkOFJIdFsuKpOUbby8dIED2cBTghyAeM6RsgpP5ppek+N4wjcqwzf4eRdi1EZfXQpGHWOgUuJRLbQ1aok60N38cmAw== Received: from mail-norwayeastazlp17013077.outbound.protection.outlook.com (HELO OSPPR02CU001.outbound.protection.outlook.com) ([40.93.81.77]) by smarthost2.eviden.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2024 08:44:01 +0100 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=r4OAEbsHTQBwcuctCtA0HEY4P015KpHbwQYQMtMUU3a34g7TzcDaKLXHDU2oGBU9zffuWsx5ngRNRDWCJ2CV587k+ICZv2JWV4nTTvEmKvTakOKWTZG6V1N6pQ/wHB9oEz04z4OkWYGFYY+sU9j9TrWVCQkRnCctQCkqzpi1pw0NmzEOijZ1V4G6s59z1Yj6ZscNt8ZC5pGpaKp1SwpLNhiS7P05ILaweq6R8LghC+8Zgxzr5ttd/kuXYmyWhywFAiMU3M8baxjnBAeW85Lm1jdrC9/bOsz60/i+AKJp5hdyePnAutyd4eBVytKvWEdIQ49FbdGNY9YzhxJbhdxpKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lKFYdcossLRHzL2m6cO8jzSLlEUPo1FKRfk/C2cEJBY=; b=Wr+mKCsyhxUld+HBC6PCs84M9M9XxI68qWrM6oNzjxZ4BXaisSUL1efn612vxpPFyLat6wZSqYFfeodODXA54EftWNwZkmbzZFiE8rVXWvowuOMiylXTUrTHwVJAR60NeNSaK0GvdxTPgkhTd6tCnB5WphT7M5INP6lXgYA6HaqRMIujJNJRuHx9D/JkH4dGTTAOcQugVyE/xMBDGZfz/EDMvub2TDVbuyVpBVrLISZQywCTJhVGiOj+usg4qU9z0kAKnOnZ06iK6cLrY1ECVcxSKarIW6iJe47UsNjyU5ZNeivMK5pT9mNYGKpEEyu+8tE92iCT11D1ZIcvviN6SQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=eviden.com; dmarc=pass action=none header.from=eviden.com; dkim=pass header.d=eviden.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Eviden.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lKFYdcossLRHzL2m6cO8jzSLlEUPo1FKRfk/C2cEJBY=; b=IXZMD6gZ+vjtaS5HVrXC6rP9eesmWBjyCq241zSs85GPkh/4NHnVSx/3EuvzJgcMUkH45o1v48IUl+cAj2VLLI5nMQaGbPOlA9/wNaM5YRKaQzQz0n9lZybgbjUjufxAIohJSayvW/Q+CVt4cGl4nOZ1kbBOCudyHlKaOwr0oRDnn9EhlFR0lRWs6ttIG6fkb4b+/ojQyzcBWrHOvfsdAK4YbcgmH4a0y8N98G2zaF7BWFGOIbASjjPFSJLpZDltaWwOJOEwcSVzogy7OdalqROrGGp+tagREFvzWYmmi3qAAXb8RUqA+uqYtFsDS9zQxZdXEi78HdWZw/kvSk4SIg== Received: from AM8PR07MB7602.eurprd07.prod.outlook.com (2603:10a6:20b:24b::7) by VI1PR07MB6414.eurprd07.prod.outlook.com (2603:10a6:800:139::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.15; Fri, 29 Nov 2024 07:43:59 +0000 Received: from AM8PR07MB7602.eurprd07.prod.outlook.com ([fe80::fbd7:ca71:b636:6f9d]) by AM8PR07MB7602.eurprd07.prod.outlook.com ([fe80::fbd7:ca71:b636:6f9d%5]) with mapi id 15.20.8207.014; Fri, 29 Nov 2024 07:43:59 +0000 From: CLEMENT MATHIEU--DRIF To: "qemu-devel@nongnu.org" CC: "jasowang@redhat.com" , "zhenzhong.duan@intel.com" , "kevin.tian@intel.com" , "yi.l.liu@intel.com" , "joao.m.martins@oracle.com" , "peterx@redhat.com" , "mst@redhat.com" , "tjeznach@rivosinc.com" , "minwoo.im@samsung.com" , CLEMENT MATHIEU--DRIF Subject: [PATCH v1 13/19] atc: Generic ATC that can be used by PCIe devices that support SVM Thread-Topic: [PATCH v1 13/19] atc: Generic ATC that can be used by PCIe devices that support SVM Thread-Index: AQHbQjJzOocc57+pXUCiOlRFpHu04w== Date: Fri, 29 Nov 2024 07:43:58 +0000 Message-ID: <20241129074332.87535-14-clement.mathieu--drif@eviden.com> References: <20241129074332.87535-1-clement.mathieu--drif@eviden.com> In-Reply-To: <20241129074332.87535-1-clement.mathieu--drif@eviden.com> Accept-Language: en-GB, fr-FR, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=eviden.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: AM8PR07MB7602:EE_|VI1PR07MB6414:EE_ x-ms-office365-filtering-correlation-id: 7a8d3666-3ab7-4b63-ddeb-08dd104995e1 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|366016|376014|7416014|1800799024|38070700018; x-microsoft-antispam-message-info: =?utf-8?q?Za/ZHFMPH/rv1YsUatPXd4YP6UkRGHb?= =?utf-8?q?fYFYIRsX/dN6Jw3q+eumhTarMP6D4HdPGO8CgKk13bEeshvaqEGF1C+gTirSMJ/ow?= =?utf-8?q?9ufnl/siixQtRmezAeqsfW0c52NmRo4j7i3p+nBPx5a+xKhT5b1BQoAp1JrUcx0KN?= =?utf-8?q?9jJGQxNupbcLmQ8FNbRqQon5f6fqtJmBJB9wklS6UaCpeVs+7uQiEnxlqg9QKroLI?= =?utf-8?q?xfdcUvi58i6ZQZO78a51DjIMA6YGT4hi22QdWTGUdEV0HNfQX1BeBCixXI/r3cn2h?= =?utf-8?q?IRKGA403MJBFript9iYVrryrIpiSx03GAz5iqXyxs0koDCD5+vRcRRmH0uci1oNSQ?= =?utf-8?q?QSuX5jf/3khQnBwPigE/NAkD/Mx1M08Vo5GCmvtfMO3e+j2l2eG1eS+Eej3qhwV5I?= =?utf-8?q?9IjdzNccYkybQciP9N0dhVvg//rW/RWTSrLliQrCK4PKrE3BLdfI9ptkm4Eu3mbIQ?= =?utf-8?q?zKcJ3hqNNTgKGlceh2F/IqEnlbd4TEtaXSCSA5OFY5ba/dCML6QNKSrKkXooirSAG?= =?utf-8?q?DxYeyPwG0PzJ9dJMYxjnme2TnEHsydQvcGVVKebrV2CPiXs/JAIHgfEg3R0/oa3mi?= =?utf-8?q?cStSpYkxV6AwYAY39gvIfiE7+BjmNcJjG7eCU8biTxRWJNvEPb4gXJ+p7rMoA2sP0?= =?utf-8?q?go41eYzxd1O4MWUbE9VXxqhALNdKVwaht3fPyDBxmgJbLDGDc5M0cx+lB1nmvQsTf?= =?utf-8?q?2FBwA2VEsM8+5yp6BR9hQBMUNWKTVaynWrHZsJGSzZPbt4syotiyZLG6hCLT1GKoU?= =?utf-8?q?m2bp+ZOWYr8o9U2Q+4IuyjsG9DnbyMRrjCaAPE/KntXYcVjJevsaGTIDmjP4yw2O0?= =?utf-8?q?A9MJgkYZ1y+q6VFJN58GsQdZ7iTZf+hWECSn4+UYIDT6st9Z0PgZg6I2dSFQHn79L?= =?utf-8?q?yeG4pAyQEEVAjv0fQeaYYufi+LlzdHo/0x1F/eox7n0MNGyF1yxHqr6lwpbMHovzz?= =?utf-8?q?c8/n1JG9BdWSts0X7s7fmHR9Wh0LEcaEGxU621qRwvm70UgE/H5+5i4fJMRY6VIgN?= =?utf-8?q?gcVAAGovRnDoaPvQPwAxWVmABLSoCHOLjtrGf8ejUTUopv2/hQevVzEsi38GLbusT?= =?utf-8?q?FCop0ULbShnEXc52uoRET6SyC9e8OHxOmWggdjuLafB/SFSAuidouksBjqKeLp2Oa?= =?utf-8?q?ZGxxC/wRzlF0+6yoFaYLy/eJoU66HZpoEKF5E9w3rt4ujn2/lcWpCDwCaayvbwLeP?= =?utf-8?q?ni82SRBNZkuDiFO2ZSO0UNe7FWrTSt3K0NycC3cHgyjdL1fjFxX4ryeKsKiMYeAX+?= =?utf-8?q?MMftU7EhaZgJzh+XsbdK0eNd75gwRSg8QwImLqBrckZ0PvEwdbboLsMZm5BXMZWmH?= =?utf-8?q?jdfm6iLywa/xK+kvEHKD2gC/ZJn5TL4NJA=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM8PR07MB7602.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(376014)(7416014)(1800799024)(38070700018); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?cyIcgmmG4WSODey5P7vTZ36b0Ruz?= =?utf-8?q?sHOu9P5sXTna73xm8/b/1wuM9iwtQrwn9F3L6nRUVE1nFs881yJTd6fyA9cKgVx0A?= =?utf-8?q?WIr0SoVL3XT0Z60FDyVaVLIwQOa/UwrHxA5uX0r86ETuqBg1clqenHYAiPXmC2PTP?= =?utf-8?q?7f/3rJf+8PNS2F/Tfxt5G86lXlm/51MLUlIdqa9njRVfrc3CrDpMcCQydkagOm0lE?= =?utf-8?q?MZADi4YfERaotmCrwmtqIUTTA58B9Pai/uXg6+L71H5T0kRGM+RxC+V4IH/80fn+V?= =?utf-8?q?ltwm6gL6Y6f9OC3wXTO2TGi/DahBXFEQmHEuWAsbu9dJJaEaCEPUv1gssMenn/ixc?= =?utf-8?q?RDXYMiiLnxIBXG03Tp409MVIn+eT0KesStFNzzXFiU9Mh5yoTwuQ2iG1n3jUniaUc?= =?utf-8?q?txyKkeERiklYu7IfIH0o7vYxv1BsKXm5Rgmkz4RiPllcodNs7SioN+TH9VNUyesJd?= =?utf-8?q?Csn9epVt+y+t8p56XphCkhaQIPRWLEm2tAe2Tpi/u0cfRnCJR8ipjKb2PxTO+wNzs?= =?utf-8?q?wlrgbJbDVwmuQmqHy06ychg3v30DmsiIJg0iE2d9jmpQg7KwW607gKs1xeFc4rsbT?= =?utf-8?q?+j+jFnHUEgOSf8wwGdc+tE8vK+46HTUytZ4SA3dVwp/3U7JLeTXuLN84UJzzvD28q?= =?utf-8?q?IyBUT1qX8Cl99dxuC0SwP5iBBFBjkUYj9yXoiFO6uFY3EaVRBH74yRj5TwM+8RAL3?= =?utf-8?q?xom6b/ekRv04OjPotfhXH3cX2X3Caiwn6dggY29kFHfMyAzQqn3qXRel5oGlWLRav?= =?utf-8?q?ITXRlECzNHDJkwhOtPWo08lc6lcmN307EhL7vQ9/x3Pt5coUeID/UteVgud6auKOT?= =?utf-8?q?GDScgx7n1ebB6ULiGZ+EByCba59w0BPdQ1I93Kr3emNdldTeDb5ur9xS5v7ZIjmkQ?= =?utf-8?q?HjNNbZW20e3U6sG18dW16WkUAAXo16sWi+iQ4QlZ85JNWlbEx2p7WvNZRNJijG2Pr?= =?utf-8?q?WwpWho85+DqaH2YiBbyr7Tch7MCPhk2kbBdTStx03AUykCZrfcTrLy6G/Nde6p0kd?= =?utf-8?q?8B1HdW91Xn+zEZaJSY1arO+OJGGLsvLqrgygK/rmlOLygX6xRraw6C0aMtAOBpohK?= =?utf-8?q?TyMxFfwPpbdEwRWUlbOVxtiRF54AHXWll/FS/i2roTbNoIkaV2cZ0XFDVByMANccc?= =?utf-8?q?MPaglRXtXS12C59BfAlFGZYqzOZzmXeNbPYkIF1kZfOCLS0r86NKjiDyd0KRtmhGE?= =?utf-8?q?Bxxirs9QqcpsQf5l2MrRg/+ZPHqpDKUaw+TsrWGqso1FEDhptAqn3ylfmbHhkToRm?= =?utf-8?q?igBUXowc6Bj3/x69h3diXl/NNP3lMAUtMT/7pPSw3WpUMSJlX5wwolcoCFq1y+9gY?= =?utf-8?q?99Bru6VPZXmpoYbYETTRSint/sbU2ZKVkVcnmTZKCAWaKJOAM3B6iS5vs9ov4e+Ne?= =?utf-8?q?M0LlFR2OxIcOuu/X2x1EJNLFOn6WnDtwVujrrnwPr3Vhqvg/F8eVOtX5tAeJq4ntv?= =?utf-8?q?5SY2/UVBSL9Mj5u/dVUQ8zmGWp4+71T94shBz4eVWcEAxwzVJDz0Tjydf3XrBe2HC?= =?utf-8?q?jF7XoJSddoI53SfS+0gyeAz/p8BXlZgnYpMSRK/XYZSTkCgEag1R1rk=3D?= Content-ID: MIME-Version: 1.0 X-OriginatorOrg: eviden.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM8PR07MB7602.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7a8d3666-3ab7-4b63-ddeb-08dd104995e1 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Nov 2024 07:43:58.9325 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7d1c7785-2d8a-437d-b842-1ed5d8fbe00a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DPPJyJUi2EnELMUvViUz5yJV+o0PSt93rPi/qUoIwbACMEELdNj3Ba7UgPibb/ZTzb5hZ4TkLF6VtCruIL7egWWgQxpTCq5i1FzH/K+EOTkbjkvviYDjl7a43UdDr09y X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR07MB6414 Received-SPF: pass client-ip=80.78.11.83; envelope-from=clement.mathieu--drif@eviden.com; helo=smarthost2.eviden.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Clément Mathieu--Drif As the SVM-capable devices will need to cache translations, we provide an first implementation. This cache uses a two-level design based on hash tables. The first level is indexed by a PASID and the second by a virtual addresse. Signed-off-by: Clément Mathieu--Drif --- util/atc.c | 211 +++++++++++++++++++++++++++++++++++++++++++++++ util/atc.h | 117 ++++++++++++++++++++++++++ util/meson.build | 1 + 3 files changed, 329 insertions(+) create mode 100644 util/atc.c create mode 100644 util/atc.h diff --git a/util/atc.c b/util/atc.c new file mode 100644 index 0000000000..584ce045db --- /dev/null +++ b/util/atc.c @@ -0,0 +1,211 @@ +/* + * QEMU emulation of an ATC + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + + * You should have received a copy of the GNU General Public License along + * with this program; if not, see . + */ + +#include "util/atc.h" + + +#define PAGE_TABLE_ENTRY_SIZE 8 + +/* a pasid is hashed using the identity function */ +static guint atc_pasid_key_hash(gconstpointer v) +{ + return (guint)(uintptr_t)v; /* pasid */ +} + +/* pasid equality */ +static gboolean atc_pasid_key_equal(gconstpointer v1, gconstpointer v2) +{ + return v1 == v2; +} + +/* Hash function for IOTLB entries */ +static guint atc_addr_key_hash(gconstpointer v) +{ + hwaddr addr = (hwaddr)v; + return (guint)((addr >> 32) ^ (addr & 0xffffffffU)); +} + +/* Equality test for IOTLB entries */ +static gboolean atc_addr_key_equal(gconstpointer v1, gconstpointer v2) +{ + return (hwaddr)v1 == (hwaddr)v2; +} + +static void atc_address_space_free(void *as) +{ + g_hash_table_unref(as); +} + +/* return log2(val), or UINT8_MAX if val is not a power of 2 */ +static uint8_t ilog2(uint64_t val) +{ + uint8_t result = 0; + while (val != 1) { + if (val & 1) { + return UINT8_MAX; + } + + val >>= 1; + result += 1; + } + return result; +} + +ATC *atc_new(uint64_t page_size, uint8_t address_width) +{ + ATC *atc; + uint8_t log_page_size = ilog2(page_size); + /* number of bits each used to store all the intermediate indexes */ + uint64_t addr_lookup_indexes_size; + + if (log_page_size == UINT8_MAX) { + return NULL; + } + /* + * We only support page table entries of 8 (PAGE_TABLE_ENTRY_SIZE) bytes + * log2(page_size / 8) = log2(page_size) - 3 + * is the level offset + */ + if (log_page_size <= 3) { + return NULL; + } + + atc = g_new0(ATC, 1); + atc->address_spaces = g_hash_table_new_full(atc_pasid_key_hash, + atc_pasid_key_equal, + NULL, atc_address_space_free); + atc->level_offset = log_page_size - 3; + /* at this point, we know that page_size is a power of 2 */ + atc->min_addr_mask = page_size - 1; + addr_lookup_indexes_size = address_width - log_page_size; + if ((addr_lookup_indexes_size % atc->level_offset) != 0) { + goto error; + } + atc->levels = addr_lookup_indexes_size / atc->level_offset; + atc->page_size = page_size; + return atc; + +error: + g_free(atc); + return NULL; +} + +static inline GHashTable *atc_get_address_space_cache(ATC *atc, uint32_t pasid) +{ + return g_hash_table_lookup(atc->address_spaces, + (gconstpointer)(uintptr_t)pasid); +} + +void atc_create_address_space_cache(ATC *atc, uint32_t pasid) +{ + GHashTable *as_cache; + + as_cache = atc_get_address_space_cache(atc, pasid); + if (!as_cache) { + as_cache = g_hash_table_new_full(atc_addr_key_hash, + atc_addr_key_equal, + NULL, g_free); + g_hash_table_replace(atc->address_spaces, + (gpointer)(uintptr_t)pasid, as_cache); + } +} + +void atc_delete_address_space_cache(ATC *atc, uint32_t pasid) +{ + g_hash_table_remove(atc->address_spaces, (gpointer)(uintptr_t)pasid); +} + +int atc_update(ATC *atc, IOMMUTLBEntry *entry) +{ + IOMMUTLBEntry *value; + GHashTable *as_cache = atc_get_address_space_cache(atc, entry->pasid); + if (!as_cache) { + return -ENODEV; + } + value = g_memdup2(entry, sizeof(*value)); + g_hash_table_replace(as_cache, (gpointer)(entry->iova), value); + return 0; +} + +IOMMUTLBEntry *atc_lookup(ATC *atc, uint32_t pasid, hwaddr addr) +{ + IOMMUTLBEntry *entry; + hwaddr mask = atc->min_addr_mask; + hwaddr key = addr & (~mask); + GHashTable *as_cache = atc_get_address_space_cache(atc, pasid); + + if (!as_cache) { + return NULL; + } + + /* + * Iterate over the possible page sizes and try to find a hit + */ + for (uint8_t level = 0; level < atc->levels; ++level) { + entry = g_hash_table_lookup(as_cache, (gconstpointer)key); + if (entry && (mask == entry->addr_mask)) { + return entry; + } + mask = (mask << atc->level_offset) | ((1 << atc->level_offset) - 1); + key = addr & (~mask); + } + + return NULL; +} + +static gboolean atc_invalidate_entry_predicate(gpointer key, gpointer value, + gpointer user_data) +{ + IOMMUTLBEntry *entry = (IOMMUTLBEntry *)value; + IOMMUTLBEntry *target = (IOMMUTLBEntry *)user_data; + hwaddr target_mask = ~target->addr_mask; + hwaddr entry_mask = ~entry->addr_mask; + return ((target->iova & target_mask) == (entry->iova & target_mask)) || + ((target->iova & entry_mask) == (entry->iova & entry_mask)); +} + +void atc_invalidate(ATC *atc, IOMMUTLBEntry *entry) +{ + GHashTable *as_cache = atc_get_address_space_cache(atc, entry->pasid); + if (!as_cache) { + return; + } + g_hash_table_foreach_remove(as_cache, + atc_invalidate_entry_predicate, + entry); +} + +void atc_destroy(ATC *atc) +{ + g_hash_table_unref(atc->address_spaces); +} + +size_t atc_get_max_number_of_pages(ATC *atc, hwaddr addr, size_t length) +{ + hwaddr page_mask = ~(atc->min_addr_mask); + size_t result = (length / atc->page_size); + if ((((addr & page_mask) + length - 1) & page_mask) != + ((addr + length - 1) & page_mask)) { + result += 1; + } + return result + (length % atc->page_size != 0 ? 1 : 0); +} + +void atc_reset(ATC *atc) +{ + g_hash_table_remove_all(atc->address_spaces); +} diff --git a/util/atc.h b/util/atc.h new file mode 100644 index 0000000000..8be95f5cca --- /dev/null +++ b/util/atc.h @@ -0,0 +1,117 @@ +/* + * QEMU emulation of an ATC + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + + * You should have received a copy of the GNU General Public License along + * with this program; if not, see . + */ + +#ifndef UTIL_ATC_H +#define UTIL_ATC_H + +#include "qemu/osdep.h" +#include "exec/memory.h" + +typedef struct ATC { + GHashTable *address_spaces; /* Key : pasid, value : GHashTable */ + hwaddr min_addr_mask; + uint64_t page_size; + uint8_t levels; + uint8_t level_offset; +} ATC; + +/* + * atc_new: Create an ATC. + * + * Return an ATC or NULL if the creation failed + * + * @page_size: #PCIDevice doing the memory access + * @address_width: width of the virtual addresses used by the IOMMU (in bits) + */ +ATC *atc_new(uint64_t page_size, uint8_t address_width); + +/* + * atc_update: Insert or update an entry in the cache + * + * Return 0 if the operation succeeds, a negative error code otherwise + * + * The insertion will fail if the address space associated with this pasid + * has not been created with atc_create_address_space_cache + * + * @atc: the ATC to update + * @entry: the tlb entry to insert into the cache + */ +int atc_update(ATC *atc, IOMMUTLBEntry *entry); + +/* + * atc_create_address_space_cache: delare a new address space + * identified by a PASID + * + * @atc: the ATC to update + * @pasid: the pasid of the address space to be created + */ +void atc_create_address_space_cache(ATC *atc, uint32_t pasid); + +/* + * atc_delete_address_space_cache: delete an address space + * identified by a PASID + * + * @atc: the ATC to update + * @pasid: the pasid of the address space to be deleted + */ +void atc_delete_address_space_cache(ATC *atc, uint32_t pasid); + +/* + * atc_lookup: query the cache in a given address space + * + * @atc: the ATC to query + * @pasid: the pasid of the address space to query + * @addr: the virtual address to translate + */ +IOMMUTLBEntry *atc_lookup(ATC *atc, uint32_t pasid, hwaddr addr); + +/* + * atc_invalidate: invalidate an entry in the cache + * + * @atc: the ATC to update + * @entry: the entry to invalidate + */ +void atc_invalidate(ATC *atc, IOMMUTLBEntry *entry); + +/* + * atc_destroy: delete an ATC + * + * @atc: the cache to be deleted + */ +void atc_destroy(ATC *atc); + +/* + * atc_get_max_number_of_pages: get the number of pages a memory operation + * will access if all the pages concerned have the minimum size. + * + * This function can be used to determine the size of the result array to be + * allocated when issuing an ATS request. + * + * @atc: the cache + * @addr: start address + * @length: number of bytes accessed from addr + */ +size_t atc_get_max_number_of_pages(ATC *atc, hwaddr addr, size_t length); + +/* + * atc_reset: invalidates all the entries stored in the ATC + * + * @atc: the cache + */ +void atc_reset(ATC *atc); + +#endif diff --git a/util/meson.build b/util/meson.build index 5d8bef9891..f2dec01300 100644 --- a/util/meson.build +++ b/util/meson.build @@ -93,6 +93,7 @@ if have_block util_ss.add(files('hbitmap.c')) util_ss.add(files('hexdump.c')) util_ss.add(files('iova-tree.c')) + util_ss.add(files('atc.c')) util_ss.add(files('iov.c')) util_ss.add(files('nvdimm-utils.c')) util_ss.add(files('block-helpers.c'))