From patchwork Sat Mar 20 15:19:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sven Peter X-Patchwork-Id: 12152523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72EC8C433C1 for ; Sat, 20 Mar 2021 15:21:22 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E2F7261965 for ; Sat, 20 Mar 2021 15:21:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E2F7261965 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=svenpeter.dev Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:Reply-To:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID: Subject:Cc:From:To:Date:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Te5XvYYOB6J1QBGzdUIYHN4JGkJRbSaI3v+2TsySE5Q=; b=dBJIvLsC2pKziVkxQCPSz03hw tQz8YahAhgYEo92FLbOlEzwdw5nGUFlvH81dg3V6d3GpIzZ1sGVJHE+Rxlq8oswjtfpoLHy8jB3h1 Bc31RiZN0KGfUkPzPzoRyL2lfrNhl5gIQFgCInBBNr6ZOCt5J5BCVAP4gEnUdTjc/LtuIJaji1HeN IbAkrWNHulbqH5OEfB+ejAzvNjjVV1RY4qThT4MNdEzTrMDfGQB7ByYvkfZmPEtVZm2qGq/kfGXls LZFd8DfV2v5INDr8yDxLsdG/3aBHtFDoO8N8gHdKl49EVETMBGrhYCRlaaynxiVLEpbcGmP5QhllD it/Ik+OOg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lNdOX-008yIC-Dk; Sat, 20 Mar 2021 15:20:05 +0000 Received: from mail-40134.protonmail.ch ([185.70.40.134]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lNdOS-008yHb-Eu for linux-arm-kernel@lists.infradead.org; Sat, 20 Mar 2021 15:20:03 +0000 Date: Sat, 20 Mar 2021 15:19:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svenpeter.dev; s=protonmail3; t=1616253599; bh=UmBJ19FRNdQXRczXhAbcIfSvSxmkFnUO3Bq5JKvNzm4=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=fsOp8RQJauPiKI+aSzOXutFeZjwRHtDBIoMZCO44lpdoyPTgn38/PDhPAwXA9C4Eo qG13iUY5rRpS2WrrF4Z/67tO1l7tTZo6qR+6gJ/Ju2OG35nS5Ftt7dr9Rjxx647uLl eKue8xA3PtyNAi5seCk6ZGbdxbv+d9CdVl+OmMTlnr9Q0H4mhv6O3W1/AerEXroMRT +tXFhO/FXdC5YQdnBejMvFR+ZwMcE2+20nUgUDsbRF2WRXC8ZIi/UBlnVsaVVY5gpF MLpH5AILG+0axnQtDjgF7feZiGkA9qfILeWJJcoVgset9eC3HVz3GIF84ThxWV7OOI UzrJ19V3e8Prg== To: iommu@lists.linux-foundation.org From: Sven Peter Cc: Joerg Roedel , Will Deacon , Robin Murphy , Rob Herring , Arnd Bergmann , Hector Martin , Mark Kettenis , Marc Zyngier , Mohamed Mediouni , Stan Skowronek , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Sven Peter Subject: [PATCH 1/3] iommu: io-pgtable: add DART pagetable format Message-ID: <20210320151903.60759-2-sven@svenpeter.dev> In-Reply-To: <20210320151903.60759-1-sven@svenpeter.dev> References: <20210320151903.60759-1-sven@svenpeter.dev> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210320_152000_633385_1AE81BD5 X-CRM114-Status: GOOD ( 12.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sven Peter Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Apple's DART iommu uses a pagetable format that's very similar to the ones already implemented by io-pgtable.c. Add a new format variant to support the required differences. Signed-off-by: Sven Peter --- drivers/iommu/Kconfig | 13 +++++++ drivers/iommu/io-pgtable-arm.c | 70 ++++++++++++++++++++++++++++++++++ drivers/iommu/io-pgtable.c | 3 ++ include/linux/io-pgtable.h | 6 +++ 4 files changed, 92 insertions(+) -- 2.25.1 diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index 192ef8f61310..3c95c8524abe 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -39,6 +39,19 @@ config IOMMU_IO_PGTABLE_LPAE sizes at both stage-1 and stage-2, as well as address spaces up to 48-bits in size. +config IOMMU_IO_PGTABLE_APPLE_DART + bool "Apple DART Descriptor Format" + select IOMMU_IO_PGTABLE + select IOMMU_IO_PGTABLE_LPAE + depends on ARM64 || (COMPILE_TEST && !GENERIC_ATOMIC64) + help + Enable support for the Apple DART iommu pagetable format. + This format is a variant of the ARMv7/v8 Long Descriptor + Format specific to Apple's iommu found in their SoCs. + + Say Y here if you have a Apple SoC like the M1 which + contains DART iommus. + config IOMMU_IO_PGTABLE_LPAE_SELFTEST bool "LPAE selftests" depends on IOMMU_IO_PGTABLE_LPAE diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 87def58e79b5..18674469313d 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -127,6 +127,10 @@ #define ARM_MALI_LPAE_MEMATTR_IMP_DEF 0x88ULL #define ARM_MALI_LPAE_MEMATTR_WRITE_ALLOC 0x8DULL +/* APPLE_DART_PTE_PROT_NO_WRITE actually maps to ARM_LPAE_PTE_AP_RDONLY */ +#define APPLE_DART_PTE_PROT_NO_WRITE (1<<7) +#define APPLE_DART_PTE_PROT_NO_READ (1<<8) + /* IOPTE accessors */ #define iopte_deref(pte,d) __va(iopte_to_paddr(pte, d)) @@ -381,6 +385,17 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, { arm_lpae_iopte pte; +#ifdef CONFIG_IOMMU_IO_PGTABLE_APPLE_DART + if (data->iop.fmt == ARM_APPLE_DART) { + pte = 0; + if (!(prot & IOMMU_WRITE)) + pte |= APPLE_DART_PTE_PROT_NO_WRITE; + if (!(prot & IOMMU_READ)) + pte |= APPLE_DART_PTE_PROT_NO_READ; + return pte; + } +#endif + if (data->iop.fmt == ARM_64_LPAE_S1 || data->iop.fmt == ARM_32_LPAE_S1) { pte = ARM_LPAE_PTE_nG; @@ -1043,6 +1058,54 @@ arm_mali_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie) return NULL; } +#ifdef CONFIG_IOMMU_IO_PGTABLE_APPLE_DART +static struct io_pgtable * +apple_dart_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie) +{ + struct arm_lpae_io_pgtable *data; + + if (cfg->ias > 38) + return NULL; + if (cfg->oas > 36) + return NULL; + + if (!cfg->coherent_walk) + return NULL; + + cfg->pgsize_bitmap &= SZ_16K; + if (!cfg->pgsize_bitmap) + return NULL; + + data = arm_lpae_alloc_pgtable(cfg); + if (!data) + return NULL; + + /* + * the hardware only supports this specific three level pagetable layout with + * the first level being encoded into four hardware registers + */ + data->start_level = ARM_LPAE_MAX_LEVELS - 2; + data->pgd_bits = 13; + data->bits_per_level = 11; + + data->pgd = __arm_lpae_alloc_pages(ARM_LPAE_PGD_SIZE(data), GFP_KERNEL, + cfg); + if (!data->pgd) + goto out_free_data; + + cfg->apple_dart_cfg.pgd[0] = virt_to_phys(data->pgd); + cfg->apple_dart_cfg.pgd[1] = virt_to_phys(data->pgd + 0x4000); + cfg->apple_dart_cfg.pgd[2] = virt_to_phys(data->pgd + 0x8000); + cfg->apple_dart_cfg.pgd[3] = virt_to_phys(data->pgd + 0xc000); + + return &data->iop; + +out_free_data: + kfree(data); + return NULL; +} +#endif + struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = { .alloc = arm_64_lpae_alloc_pgtable_s1, .free = arm_lpae_free_pgtable, @@ -1068,6 +1131,13 @@ struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns = { .free = arm_lpae_free_pgtable, }; +#ifdef CONFIG_IOMMU_IO_PGTABLE_APPLE_DART +struct io_pgtable_init_fns io_pgtable_apple_dart_init_fns = { + .alloc = apple_dart_alloc_pgtable, + .free = arm_lpae_free_pgtable, +}; +#endif + #ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST static struct io_pgtable_cfg *cfg_cookie __initdata; diff --git a/drivers/iommu/io-pgtable.c b/drivers/iommu/io-pgtable.c index 6e9917ce980f..d86590b0673a 100644 --- a/drivers/iommu/io-pgtable.c +++ b/drivers/iommu/io-pgtable.c @@ -27,6 +27,9 @@ io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = { #ifdef CONFIG_AMD_IOMMU [AMD_IOMMU_V1] = &io_pgtable_amd_iommu_v1_init_fns, #endif +#ifdef CONFIG_IOMMU_IO_PGTABLE_APPLE_DART + [ARM_APPLE_DART] = &io_pgtable_apple_dart_init_fns, +#endif }; struct io_pgtable_ops *alloc_io_pgtable_ops(enum io_pgtable_fmt fmt, diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h index a4c9ca2c31f1..19d9b631d319 100644 --- a/include/linux/io-pgtable.h +++ b/include/linux/io-pgtable.h @@ -16,6 +16,7 @@ enum io_pgtable_fmt { ARM_V7S, ARM_MALI_LPAE, AMD_IOMMU_V1, + ARM_APPLE_DART, IO_PGTABLE_NUM_FMTS, }; @@ -136,6 +137,10 @@ struct io_pgtable_cfg { u64 transtab; u64 memattr; } arm_mali_lpae_cfg; + + struct { + u64 pgd[4]; + } apple_dart_cfg; }; }; @@ -250,5 +255,6 @@ extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns; extern struct io_pgtable_init_fns io_pgtable_amd_iommu_v1_init_fns; +extern struct io_pgtable_init_fns io_pgtable_apple_dart_init_fns; #endif /* __IO_PGTABLE_H */ From patchwork Sat Mar 20 15:20:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sven Peter X-Patchwork-Id: 12152525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B2B6C433C1 for ; Sat, 20 Mar 2021 15:21:43 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 77A8961951 for ; Sat, 20 Mar 2021 15:21:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 77A8961951 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=svenpeter.dev Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:Reply-To:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID: Subject:Cc:From:To:Date:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5LWAUPQs5pm4e83KfM8y1wrOIh7MWS2rhqNJloiJa6A=; b=gO9oMlmjKP7qWD/8BuqdrlhVK MCONGHkSgFOnkpjzJVU/Y4GR8Z+971eavpnVGrjNhNiusmgkPySPi9Em+UYBLFTDIMUrn1oy4ZdbR hVvFsNAlds8HpIpWQf0uWXp0sBnA+0vNGAsJHlj5IkL7ufn7+EZNhloQiACaRfDgr3DM6bYCuXV3C UrImSYU/SOja6QZIyGYVdnRIuT4WN0ViRMAw8A0LLHUwB5gF9G2LCAWCvXqXMc6x0R4SldFs8IAC1 UiLbcxwW8ZKUZryy52NBrzoou2nAxymtulZMyzCrQUKoZNyi93DqLk8JSabltf3jfcpMwiAp7L9tC efDkgCk5A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lNdOn-008yJb-7F; Sat, 20 Mar 2021 15:20:21 +0000 Received: from mail-40134.protonmail.ch ([185.70.40.134]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lNdOj-008yJ3-GY for linux-arm-kernel@lists.infradead.org; Sat, 20 Mar 2021 15:20:19 +0000 Date: Sat, 20 Mar 2021 15:20:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svenpeter.dev; s=protonmail3; t=1616253616; bh=GOagsrEw4SsLChHE8bqUY6XXHswLBJsU8M1xRpDR/Ds=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=KeSAi5xxj8JV35umMvuVdhLCnkXdNmc1FJg13DceRY9y+ke3LFGuKi940tD9QfctT 9cz7qsJouqCq2S7gqnwS2yysEs145gQe7Visufh2rB5j1KEpKoJxuY9dE5cxXueRGZ mwE95gf7pZaGSAeRxFX8dF4/O76TDSxwm/US4tOevxHYeZwjJpGXIZNQCTMI1I5dW+ Y6lUFwnyUnE5HzboyzKtYBXz+2V4UPOHmT4OnpMfANF5FosIDgQf2yshTj32Q0Wuoo 93iccrC3DxKRvbNZQkhPX0MIplCxahVX/bvRusvQjYuaaheeQVZWtcb6lzTIOD5tER xovotgbJ8Bb3A== To: iommu@lists.linux-foundation.org From: Sven Peter Cc: Joerg Roedel , Will Deacon , Robin Murphy , Rob Herring , Arnd Bergmann , Hector Martin , Mark Kettenis , Marc Zyngier , Mohamed Mediouni , Stan Skowronek , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Sven Peter Subject: [PATCH 2/3] dt-bindings: iommu: add DART iommu bindings Message-ID: <20210320151903.60759-3-sven@svenpeter.dev> In-Reply-To: <20210320151903.60759-1-sven@svenpeter.dev> References: <20210320151903.60759-1-sven@svenpeter.dev> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210320_152017_630477_48F1AD85 X-CRM114-Status: GOOD ( 13.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sven Peter Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DART (Device Address Resolution Table) is the iommu found on Apple ARM SoCs such as the M1. Signed-off-by: Sven Peter --- .../bindings/iommu/apple,t8103-dart.yaml | 82 +++++++++++++++++++ MAINTAINERS | 6 ++ 2 files changed, 88 insertions(+) create mode 100644 Documentation/devicetree/bindings/iommu/apple,t8103-dart.yaml -- 2.25.1 diff --git a/Documentation/devicetree/bindings/iommu/apple,t8103-dart.yaml b/Documentation/devicetree/bindings/iommu/apple,t8103-dart.yaml new file mode 100644 index 000000000000..2ec2d905a2ea --- /dev/null +++ b/Documentation/devicetree/bindings/iommu/apple,t8103-dart.yaml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/iommu/apple,t8103-dart.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Apple DART IOMMU Implementation + +maintainers: + - Sven Peter + +description: |+ + Apple SoCs may contain an implementation of their Device Address + Resolution Table which provides a mandatory layer of address + translations for various masters external to the CPU. + + Each DART instance is capable of handling up to 16 masters + mapped to up tp 16 different virtual address spaces with + page-level read/write access control flags. + + This DART IOMMU also raises interrupts in response to various + fault conditions. + +properties: + compatible: + const: apple,t8103-dart + + reg: + const: 1 + + interrupts: + const: 1 + + clocks: + maxItems: 1 + + '#iommu-cells': + const: 1 + description: + The number of the master connected to this DART. + +required: + - compatible + - reg + - '#iommu-cells' + - interrupts + +additionalProperties: false + +examples: + - |+ + dart1: dart1@382f80000 { + compatible = "apple,t8103-dart"; + reg = <0x3 0x82f80000 0x0 0x4000>; + interrupt-parent = <&aic>; + interrupts = ; + #iommu-cells = <1>; + }; + + master1 { + iommus = <&dart1 0>; + }; + + - |+ + usb_dart0: usb_dart0@382f80000 { + compatible = "apple,t8103-dart"; + reg = <0x3 0x82f80000 0x0 0x4000>; + interrupt-parent = <&aic>; + interrupts = ; + #iommu-cells = <1>; + }; + + usbdrd_dwc3_0: dwc3@382280000 { + compatible = "snps,dwc3"; + reg = <0x3 0x82280000 0x0 0x100000>; + interrupt-parent = <&aic>; + interrupts = ; + iommus = <&usb_dart0 1>; + #address-cells = <1>; + #size-cells = <0>; + dr_mode = "peripheral"; + }; diff --git a/MAINTAINERS b/MAINTAINERS index d5e4d93a536a..1f9a4f2de88b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1236,6 +1236,12 @@ L: linux-input@vger.kernel.org S: Odd fixes F: drivers/input/mouse/bcm5974.c +APPLE DART IOMMU DRIVER +M: Sven Peter +L: iommu@lists.linux-foundation.org +S: Maintained +F: Documentation/devicetree/bindings/iommu/apple,t8103-dart.yaml + APPLE SMC DRIVER M: Henrik Rydberg L: linux-hwmon@vger.kernel.org From patchwork Sat Mar 20 15:34:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sven Peter X-Patchwork-Id: 12152547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10140C433DB for ; Sat, 20 Mar 2021 15:36:28 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61DB361965 for ; Sat, 20 Mar 2021 15:36:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61DB361965 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=svenpeter.dev Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:Reply-To:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-ID:Subject:Cc:From:To:Date: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=ff9YukGYAHIWES0IOlklIXI+5YqYhZLyxU4d6NUr9w8=; b=Eb6ml5SFsHewaWHkh5HLyyPqIc oIn1brNpwBCq2fsAnocAA7A8s2kosA+4AAZXplk/RVhOr9tbJ3QxQJi1EccwL5fYO4sQnTWSi3b2D L8zbpdANoiIGgeFYdbtGluIZzNzqJZJZqfNVgNsumdKRFKjQRyN0yvxMkxzwzqw0OUdZiKlpF/3av RZYAWAZ0b7Q7iyLdiiZ+DKp3KXKlKylDowjuGFUd05cshNp8pq6HzlL0S9vb0VOHEwKM1F2jvzMap j6poZzk+nFeZyNWk7sNChnDhIAOMI+pLk7TLB/RIXUnaCa0sh4XDVoDkHcF5m2X9ZgHX9733xMjmg AB5zx1bQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lNdcs-008zTa-It; Sat, 20 Mar 2021 15:34:54 +0000 Received: from mail2.protonmail.ch ([185.70.40.22]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lNdcn-008zTB-6N for linux-arm-kernel@lists.infradead.org; Sat, 20 Mar 2021 15:34:51 +0000 Date: Sat, 20 Mar 2021 15:34:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svenpeter.dev; s=protonmail3; t=1616254486; bh=JTyt7legjbFhec6GFumJoMePKi1InCF9VN17NylsqRE=; h=Date:To:From:Cc:Reply-To:Subject:From; b=Nb8kmdt4B3a51am5iZsJpAi7WUIS++kbEc3Orbn+Moy2/MVoFvBQs0QlSzJtNV/+t l1br16fPNrPDXQfIicmG5rjfeVPVxJSIwmj1xLwP56PEmfb/0xeJlrrbNIb2EYeV0w YLfJuziguX5bA1WE4EnOHthTE5Vc5Udx9aVsdGCLQftHjivuryzwgRxxjxJliDIyv2 D/GVvFswygnOKcgx1HhJ4uLxMp5mr/+yH7W6dAT405Eo2qdHzLda2n9mGqYW9ACirX jRaqMxCQmpRxA5oDDUWNuxaL9z8cg2N6GKC3H4yiBCR0FcNofYMRByUPhxI84hqTbm K9/zpawux33ow== To: iommu@lists.linux-foundation.org From: Sven Peter Cc: Joerg Roedel , Will Deacon , Robin Murphy , Rob Herring , Arnd Bergmann , Hector Martin , Mark Kettenis , Marc Zyngier , Mohamed Mediouni , Stan Skowronek , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Sven Peter Subject: [PATCH 3/3] iommu: dart: Add DART iommu driver Message-ID: <20210320153419.61311-1-sven@svenpeter.dev> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210320_153449_613167_C9D8CBF3 X-CRM114-Status: GOOD ( 15.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sven Peter Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Apple's new SoCs use iommus for almost all peripherals. These Device Address Resolution Tables must be setup before these peripherals can act as DMA masters. Signed-off-by: Sven Peter --- MAINTAINERS | 1 + drivers/iommu/Kconfig | 13 + drivers/iommu/Makefile | 1 + drivers/iommu/apple-dart-iommu.c | 653 +++++++++++++++++++++++++++++++ 4 files changed, 668 insertions(+) create mode 100644 drivers/iommu/apple-dart-iommu.c -- 2.25.1 diff --git a/MAINTAINERS b/MAINTAINERS index 1f9a4f2de88b..7dcfce53dd04 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1241,6 +1241,7 @@ M: Sven Peter L: iommu@lists.linux-foundation.org S: Maintained F: Documentation/devicetree/bindings/iommu/apple,t8103-dart.yaml +F: drivers/iommu/apple-dart-iommu.c APPLE SMC DRIVER M: Henrik Rydberg diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index 3c95c8524abe..810bcb3ed414 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -262,6 +262,19 @@ config SPAPR_TCE_IOMMU Enables bits of IOMMU API required by VFIO. The iommu_ops is not implemented as it is not necessary for VFIO. +config IOMMU_APPLE_DART + tristate "Apple DART IOMMU Support" + depends on ARM64 || (COMPILE_TEST && !GENERIC_ATOMIC64) + select IOMMU_API + select IOMMU_IO_PGTABLE_APPLE_DART + help + Support for Apple DART (Device Address Resolution Table) IOMMUs + found in Apple ARM SoCs like the M1. + This IOMMU is required for most peripherals using DMA to access + the main memory. + + Say Y here if you are using an Apple SoC with a DART IOMMU. + # ARM IOMMU support config ARM_SMMU tristate "ARM Ltd. System MMU (SMMU) Support" diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 61bd30cd8369..5f21f0dfec6a 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -28,3 +28,4 @@ obj-$(CONFIG_S390_IOMMU) += s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) += hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o obj-$(CONFIG_IOMMU_SVA_LIB) += iommu-sva-lib.o +obj-$(CONFIG_IOMMU_APPLE_DART) += apple-dart-iommu.o diff --git a/drivers/iommu/apple-dart-iommu.c b/drivers/iommu/apple-dart-iommu.c new file mode 100644 index 000000000000..a642dbc22281 --- /dev/null +++ b/drivers/iommu/apple-dart-iommu.c @@ -0,0 +1,653 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Apple DART (Device Address Resolution Table) IOMMU driver + * + * Based on arm/arm-smmu/arm-ssmu.c and arm/arm-smmu-v3/arm-smmu-v3.c + * Copyright (C) 2013 ARM Limited + * Copyright (C) 2015 ARM Limited + * + * Copyright (C) 2021 The Asahi Linux Contributors + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define DART_MAX_DEVICES 16 +#define DART_MAX_DOMAINS 16 +#define DART_MAX_TTBR 4 + +#define DART_DOMAINS_ALL 0xffff + +#define DART_CONFIG 0x60 +#define DART_CONFIG_LOCK BIT(15) + +#define DART_ERROR 0x40 +#define DART_ERROR_DOMAIN_SHIFT 24 +#define DART_ERROR_DOMAIN_MASK 0xf +#define DART_ERROR_CODE_MASK 0xffffff +#define DART_ERROR_FLAG BIT(31) +#define DART_ERROR_READ_FAULT BIT(4) +#define DART_ERROR_WRITE_FAULT BIT(3) +#define DART_ERROR_NO_PTE BIT(2) +#define DART_ERROR_NO_PMD BIT(1) +#define DART_ERROR_NO_PGD BIT(0) + +#define DART_DOMAIN_SELECT 0x34 + +#define DART_DOMAIN_COMMAND 0x20 +#define DART_DOMAIN_COMMAND_BUSY BIT(2) +#define DART_DOMAIN_COMMAND_INVALIDATE BIT(20) + +#define DART_DOMAIN_COMMAND_BUSY_TIMEOUT 100 + +#define DART_DEVICE2DOMAIN_MAP 0x80 + +#define DART_ERROR_ADDR_HI 0x54 +#define DART_ERROR_ADDR_LO 0x50 + +#define DART_TCR(domain) (0x100 + 4 * (domain)) +#define DART_TCR_TRANSLATE_ENABLE BIT(7) +#define DART_TCR_BYPASS_ENABLE BIT(8) + +#define DART_TTBR(domain, idx) (0x200 + 16 * (domain) + 4 * (idx)) +#define DART_TTBR_VALID BIT(31) +#define DART_TTBR_SHIFT 12 + +struct apple_dart_domain; + +struct apple_dart { + struct device *dev; + void __iomem *regs; + + int irq; + struct clk_bulk_data *clks; + int num_clks; + + struct iommu_device iommu; + + struct apple_dart_domain *domains[DART_MAX_DOMAINS]; + + spinlock_t command_lock; + struct mutex domain_mutex; +}; + +struct apple_dart_domain { + struct apple_dart *dart; + + int domain_idx; + + struct io_pgtable_ops *pgtbl_ops; + + struct iommu_domain domain; +}; + +struct apple_dart_master { + struct apple_dart *dart; + u32 sid; +}; + +static struct platform_driver apple_dart_driver; +static const struct iommu_ops apple_dart_iommu_ops; + +static struct apple_dart_domain *to_dart_domain(struct iommu_domain *dom) +{ + return container_of(dom, struct apple_dart_domain, domain); +} + +static void apple_dart_hw_enable_translation(struct apple_dart *dart, + u16 domain) +{ + writel(DART_TCR_TRANSLATE_ENABLE, dart->regs + DART_TCR(domain)); +} + +static void apple_dart_hw_disable_translation(struct apple_dart *dart, + u16 domain) +{ + writel(0, dart->regs + DART_TCR(domain)); +} + +static void apple_dart_hw_set_pgd(struct apple_dart *dart, u16 domain, u16 idx, + phys_addr_t paddr) +{ + writel(DART_TTBR_VALID | (paddr >> DART_TTBR_SHIFT), + dart->regs + DART_TTBR(domain, idx)); +} + +static void apple_dart_hw_clear_pgd(struct apple_dart *dart, u16 domain, + u16 idx) +{ + writel(0, dart->regs + DART_TTBR(domain, idx)); +} + +static void apple_dart_hw_clear_all_pgds(struct apple_dart *dart, u16 domain) +{ + int i; + + for (i = 0; i < 4; ++i) + apple_dart_hw_clear_pgd(dart, domain, i); +} + +static int apple_dart_hw_domain_command(struct apple_dart *dart, + u16 domain_bitmap, u32 command) +{ + unsigned long flags; + int ret; + u32 command_reg; + + spin_lock_irqsave(&dart->command_lock, flags); + writel(domain_bitmap, dart->regs + DART_DOMAIN_SELECT); + writel(command, dart->regs + DART_DOMAIN_COMMAND); + + ret = readl_poll_timeout(dart->regs + DART_DOMAIN_COMMAND, command_reg, + !(command_reg & DART_DOMAIN_COMMAND_BUSY), 1, + DART_DOMAIN_COMMAND_BUSY_TIMEOUT); + spin_unlock_irqrestore(&dart->command_lock, flags); + + if (ret) { + dev_err(dart->dev, + "Timeout while waiting for busy flag to be cleared after issuing command %08x for domains %x\n", + command, domain_bitmap); + return ret; + } + + return 0; +} + +static int apple_dart_hw_invalidate_tlb_global(struct apple_dart *dart) +{ + return apple_dart_hw_domain_command(dart, DART_DOMAINS_ALL, + DART_DOMAIN_COMMAND_INVALIDATE); +} + +static int apple_dart_hw_invalidate_tlb_domain(struct apple_dart *dart, + u16 domain) +{ + return apple_dart_hw_domain_command(dart, 1 << domain, + DART_DOMAIN_COMMAND_INVALIDATE); +} + +static void apple_dart_hw_map_device_to_domain(struct apple_dart *dart, + u8 device, u8 domain) +{ + u32 val; + unsigned int reg = device / 4; + unsigned int shift = 8 * (device % 4); + unsigned int mask = 0xff << shift; + + val = readl(dart->regs + DART_DEVICE2DOMAIN_MAP + 4 * reg); + val &= ~mask; + val |= domain << shift; + writel(val, dart->regs + DART_DEVICE2DOMAIN_MAP + 4 * reg); +} + +static int apple_hw_dart_reset(struct apple_dart *dart) +{ + int domain; + u32 config; + + config = readl(dart->regs + DART_CONFIG); + if (config & DART_CONFIG_LOCK) { + dev_err(dart->dev, "DART is locked down until reboot: %08x\n", + config); + return -EINVAL; + } + + for (domain = 0; domain < DART_MAX_DOMAINS; ++domain) { + apple_dart_hw_disable_translation(dart, domain); + apple_dart_hw_clear_all_pgds(dart, domain); + } + + /* restore identity device to domain map */ + for (domain = 0; domain < DART_MAX_DOMAINS; ++domain) + apple_dart_hw_map_device_to_domain(dart, domain, domain); + + return apple_dart_hw_invalidate_tlb_global(dart); +} + +static void apple_dart_tlb_flush_all(void *cookie) +{ + struct apple_dart_domain *domain = cookie; + + apple_dart_hw_invalidate_tlb_domain(domain->dart, domain->domain_idx); +} + +static void apple_dart_tlb_flush_walk(unsigned long iova, size_t size, + size_t granule, void *cookie) +{ + struct apple_dart_domain *domain = cookie; + + apple_dart_hw_invalidate_tlb_domain(domain->dart, domain->domain_idx); +} + +static const struct iommu_flush_ops apple_dart_tlb_ops = { + .tlb_flush_all = apple_dart_tlb_flush_all, + .tlb_flush_walk = apple_dart_tlb_flush_walk, + .tlb_add_page = NULL, +}; + +static struct iommu_domain *apple_dart_domain_alloc(unsigned int type) +{ + struct apple_dart_domain *dart_domain; + + if (type != IOMMU_DOMAIN_DMA && type != IOMMU_DOMAIN_UNMANAGED) + return NULL; + + dart_domain = kzalloc(sizeof(*dart_domain), GFP_KERNEL); + if (!dart_domain) + return NULL; + + dart_domain->domain_idx = -1; + iommu_get_dma_cookie(&dart_domain->domain); + + return &dart_domain->domain; +} + +static void apple_dart_domain_free(struct iommu_domain *domain) +{ + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + struct apple_dart *dart = dart_domain->dart; + + if (dart && dart_domain->domain_idx >= 0) { + mutex_lock(&dart->domain_mutex); + apple_dart_hw_clear_all_pgds(dart_domain->dart, + dart_domain->domain_idx); + apple_dart_hw_disable_translation(dart_domain->dart, + dart_domain->domain_idx); + + apple_dart_hw_invalidate_tlb_domain(dart_domain->dart, + dart_domain->domain_idx); + + dart->domains[dart_domain->domain_idx] = NULL; + + mutex_unlock(&dart->domain_mutex); + } + + kfree(dart_domain); +} + +static struct apple_dart *apple_dart_get_by_fwnode(struct fwnode_handle *fwnode) +{ + struct device *dev = + driver_find_device_by_fwnode(&apple_dart_driver.driver, fwnode); + put_device(dev); + return dev ? dev_get_drvdata(dev) : NULL; +} + +static int apple_dart_finalise_domain(struct iommu_domain *domain, + struct apple_dart_master *cfg) +{ + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + struct apple_dart *dart = dart_domain->dart; + struct io_pgtable_cfg pgtbl_cfg; + int i, ret; + + mutex_lock(&dart->domain_mutex); + + /* sanity check to ensure this domain isn't already initialized */ + if (WARN_ON(dart_domain->domain_idx >= 0)) { + ret = -EINVAL; + goto out; + } + + dart_domain->domain_idx = cfg->sid; + + /* sanity check to ensure this index isn't already in use */ + if (WARN_ON(dart->domains[dart_domain->domain_idx] != NULL)) { + ret = -EINVAL; + goto out; + } + dart->domains[dart_domain->domain_idx] = dart_domain; + + pgtbl_cfg = (struct io_pgtable_cfg){ + .pgsize_bitmap = SZ_16K, + .ias = 32, + .oas = 36, + .coherent_walk = 1, + .tlb = &apple_dart_tlb_ops, + .iommu_dev = dart->dev, + }; + + dart_domain->pgtbl_ops = + alloc_io_pgtable_ops(ARM_APPLE_DART, &pgtbl_cfg, domain); + if (!dart_domain->pgtbl_ops) { + ret = -ENOMEM; + goto out_domain; + } + + for (i = 0; i < 4; ++i) { + apple_dart_hw_set_pgd(dart, dart_domain->domain_idx, i, + pgtbl_cfg.apple_dart_cfg.pgd[i]); + } + + apple_dart_hw_enable_translation(dart, dart_domain->domain_idx); + apple_dart_hw_invalidate_tlb_domain(dart, dart_domain->domain_idx); + + domain->pgsize_bitmap = pgtbl_cfg.pgsize_bitmap; + domain->geometry.aperture_end = (1UL << pgtbl_cfg.ias) - 1; + domain->geometry.force_aperture = true; + + ret = 0; + +out_domain: + if (ret) + dart->domains[dart_domain->domain_idx] = NULL; +out: + mutex_unlock(&dart->domain_mutex); + return ret; +} + +static int apple_dart_attach_dev(struct iommu_domain *domain, + struct device *dev) +{ + int ret; + struct apple_dart_master *cfg = dev_iommu_priv_get(dev); + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + + dart_domain->dart = cfg->dart; + + ret = apple_dart_finalise_domain(domain, cfg); + if (ret) + return ret; + + return 0; +} + +static struct iommu_device *apple_dart_probe_device(struct device *dev) +{ + struct apple_dart *dart = NULL; + struct apple_dart_master *cfg = NULL; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + if (!fwspec || fwspec->ops != &apple_dart_iommu_ops) + return ERR_PTR(-ENODEV); + + dart = apple_dart_get_by_fwnode(fwspec->iommu_fwnode); + if (!dart) + return ERR_PTR(-ENODEV); + + cfg = kzalloc(offsetof(struct apple_dart_master, sid), GFP_KERNEL); + if (!cfg) + return ERR_PTR(-ENOMEM); + + cfg->dart = dart; + cfg->sid = fwspec->ids[0]; + dev_iommu_priv_set(dev, cfg); + + device_link_add(dev, dart->dev, + DL_FLAG_PM_RUNTIME | DL_FLAG_AUTOREMOVE_SUPPLIER); + return &dart->iommu; +} + +static void apple_dart_release_device(struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + if (!fwspec || fwspec->ops != &apple_dart_iommu_ops) + return; + + iommu_fwspec_free(dev); +} + +static int apple_dart_of_xlate(struct device *dev, struct of_phandle_args *args) +{ + if (args->args_count != 1) { + dev_err(dev, "invalid #iommu-cells(%d) property for IOMMU\n", + args->args_count); + return -EINVAL; + } + + return iommu_fwspec_add_ids(dev, args->args, 1); +} + +static phys_addr_t apple_dart_iova_to_phys(struct iommu_domain *domain, + dma_addr_t iova) +{ + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + struct io_pgtable_ops *ops = dart_domain->pgtbl_ops; + + if (!ops) + return -ENODEV; + + return ops->iova_to_phys(ops, iova); +} + +static int apple_dart_map(struct iommu_domain *domain, unsigned long iova, + phys_addr_t paddr, size_t size, int prot, gfp_t gfp) +{ + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + struct io_pgtable_ops *ops = dart_domain->pgtbl_ops; + + if (!ops) + return -ENODEV; + + return ops->map(ops, iova, paddr, size, prot, gfp); +} + +static size_t apple_dart_unmap(struct iommu_domain *domain, unsigned long iova, + size_t size, struct iommu_iotlb_gather *gather) +{ + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + struct io_pgtable_ops *ops = dart_domain->pgtbl_ops; + + if (!ops) + return 0; + + return ops->unmap(ops, iova, size, gather); +} + +static void apple_dart_iotlb_sync(struct iommu_domain *domain, + struct iommu_iotlb_gather *gather) +{ + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + struct apple_dart *dart = dart_domain->dart; + + if (!dart) + return; + + apple_dart_hw_invalidate_tlb_domain(dart, dart_domain->domain_idx); +} + +static void apple_dart_iotlb_sync_map(struct iommu_domain *domain, + unsigned long iova, size_t size) +{ + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + struct apple_dart *dart = dart_domain->dart; + + if (!dart) + return; + + apple_dart_hw_invalidate_tlb_domain(dart, dart_domain->domain_idx); +} + +static void apple_dart_flush_iotlb_all(struct iommu_domain *domain) +{ + struct apple_dart_domain *dart_domain = to_dart_domain(domain); + struct apple_dart *dart = dart_domain->dart; + + if (!dart) + return; + + apple_dart_hw_invalidate_tlb_domain(dart, dart_domain->domain_idx); +} + +static struct iommu_group *apple_dart_device_group(struct device *dev) +{ + /* once we have PCI support this needs to use pci_device_group conditionally */ + return generic_device_group(dev); +} + +static const struct iommu_ops apple_dart_iommu_ops = { + .domain_alloc = apple_dart_domain_alloc, + .domain_free = apple_dart_domain_free, + .attach_dev = apple_dart_attach_dev, + .map = apple_dart_map, + .unmap = apple_dart_unmap, + .flush_iotlb_all = apple_dart_flush_iotlb_all, + .iotlb_sync = apple_dart_iotlb_sync, + .iotlb_sync_map = apple_dart_iotlb_sync_map, + .iova_to_phys = apple_dart_iova_to_phys, + .probe_device = apple_dart_probe_device, + .release_device = apple_dart_release_device, + .device_group = apple_dart_device_group, + .of_xlate = apple_dart_of_xlate, + .pgsize_bitmap = SZ_16K, +}; + +static irqreturn_t apple_dart_irq(int irq, void *dev) +{ + static DEFINE_RATELIMIT_STATE(rs, DEFAULT_RATELIMIT_INTERVAL, + DEFAULT_RATELIMIT_BURST); + struct apple_dart *dart = dev; + const char *fault_name = NULL; + u32 error = readl(dart->regs + DART_ERROR); + u32 error_code = error & DART_ERROR_CODE_MASK; + u32 addr_hi = readl(dart->regs + DART_ERROR_ADDR_HI); + u32 addr_lo = readl(dart->regs + DART_ERROR_ADDR_LO); + u8 domain_idx = + (error >> DART_ERROR_DOMAIN_SHIFT) & DART_ERROR_DOMAIN_MASK; + + if (!(error & DART_ERROR_FLAG)) + return IRQ_NONE; + + if (error_code & DART_ERROR_READ_FAULT) + fault_name = "READ FAULT"; + else if (error_code & DART_ERROR_WRITE_FAULT) + fault_name = "WRITE FAULT"; + else if (error_code & DART_ERROR_NO_PTE) + fault_name = "NO PTE FOR IOVA"; + else if (error_code & DART_ERROR_NO_PMD) + fault_name = "NO PMD FOR IOVA"; + else if (error_code & DART_ERROR_NO_PGD) + fault_name = "NO PGD FOR IOVA"; + + if (WARN_ON(fault_name == NULL)) + fault_name = "unknown"; + + if (__ratelimit(&rs)) { + dev_err(dart->dev, + "Apple DART translation fault: error status %08x [domain:%d code:%x (%s)] at address 0x%08x%08x", + error, domain_idx, error_code, fault_name, addr_hi, + addr_lo); + } + + writel(error, dart->regs + DART_ERROR); + return IRQ_HANDLED; +} + +static int apple_dart_probe(struct platform_device *pdev) +{ + int ret; + struct resource *res; + resource_size_t ioaddr; + struct apple_dart *dart; + struct device *dev = &pdev->dev; + + dart = devm_kzalloc(dev, sizeof(*dart), GFP_KERNEL); + if (!dart) + return -ENOMEM; + dart->dev = dev; + spin_lock_init(&dart->command_lock); + mutex_init(&dart->domain_mutex); + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (resource_size(res) < 0x4000) { + dev_err(dev, "MMIO region too small (%pr)\n", res); + return -EINVAL; + } + ioaddr = res->start; + dart->regs = devm_ioremap_resource(dev, res); + if (IS_ERR(dart->regs)) + return PTR_ERR(dart->regs); + + ret = devm_clk_bulk_get_all(dev, &dart->clks); + if (ret < 0) + return ret; + dart->num_clks = ret; + + ret = clk_bulk_prepare_enable(dart->num_clks, dart->clks); + if (ret) + return ret; + + ret = apple_hw_dart_reset(dart); + if (ret) + return ret; + + dart->irq = platform_get_irq(pdev, 0); + if (dart->irq < 0) + return -ENODEV; + + ret = devm_request_irq(dart->dev, dart->irq, apple_dart_irq, + IRQF_SHARED, "apple-dart fault", dart); + if (ret) + return ret; + + platform_set_drvdata(pdev, dart); + + ret = iommu_device_sysfs_add(&dart->iommu, dev, NULL, "apple-dart.%s", + dev_name(&pdev->dev)); + if (ret) + return ret; + + iommu_device_set_ops(&dart->iommu, &apple_dart_iommu_ops); + iommu_device_set_fwnode(&dart->iommu, dev->fwnode); + + ret = iommu_device_register(&dart->iommu); + if (ret) + return ret; + + ret = bus_set_iommu(dev->bus, &apple_dart_iommu_ops); + if (ret) + return ret; + + return 0; +} + +static int apple_dart_remove(struct platform_device *pdev) +{ + struct apple_dart *dart = platform_get_drvdata(pdev); + + iommu_device_unregister(&dart->iommu); + iommu_device_sysfs_remove(&dart->iommu); + + clk_bulk_disable(dart->num_clks, dart->clks); + clk_bulk_unprepare(dart->num_clks, dart->clks); + + return 0; +} + +static void apple_dart_shutdown(struct platform_device *pdev) +{ + apple_dart_remove(pdev); +} + +static const struct of_device_id apple_dart_of_match[] = { + { .compatible = "apple,t8103-dart", .data = NULL }, + {}, +}; +MODULE_DEVICE_TABLE(of, apple_dart_of_match); + +static struct platform_driver apple_dart_driver = { + .driver = { + .name = "apple-dart", + .of_match_table = apple_dart_of_match, + }, + .probe = apple_dart_probe, + .remove = apple_dart_remove, + .shutdown = apple_dart_shutdown, +}; +module_platform_driver(apple_dart_driver); + +MODULE_DESCRIPTION("IOMMU API for Apple's DART"); +MODULE_AUTHOR("Sven Peter "); +MODULE_LICENSE("GPL v2");