From patchwork Sat Mar 20 15:19:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sven Peter X-Patchwork-Id: 12152521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB6DFC433DB for ; Sat, 20 Mar 2021 15:21:14 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 425B061964 for ; Sat, 20 Mar 2021 15:21:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 425B061964 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=svenpeter.dev Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:Reply-To:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-ID:Subject:Cc:From:To:Date: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=wcOCrJpc8+Bj8kvKaXL/GPal/izaKaDltnsGESCLXjc=; b=k5IipKzzlIeA9bSxb7WUxqoyUl QT1EZPUT3LDFimNlKWJAQc2XV+qbAQa38n/x8Fr0Vg51hGkqForokIiluzfbdCC2A1Kc/nKI/lBD8 m1xCDQ9WKZtSJB0Vrk1QhOH5aNJxYt4HOjqoeTU7F/QKEcKSBUrp+LRyvW2s/NcpdkhrofYSRVlMK VpvHfiFUpoLKS9bYdYJ3YddAGvpGAwuVWM6HljCvms26n1J9JEs4lzNteB//FTjzMx0yEqAtCekg8 kPwGQZYxSTfDhHho+6gqYiqnvuTfYKbIbLYEwsZUejACTdV2RixxWJvjqbLo5ghS+uDMRM88CtuXh 297JN44g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lNdOE-008yHG-TM; Sat, 20 Mar 2021 15:19:47 +0000 Received: from mail-40131.protonmail.ch ([185.70.40.131]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lNdOA-008yGi-6z for linux-arm-kernel@lists.infradead.org; Sat, 20 Mar 2021 15:19:45 +0000 Date: Sat, 20 Mar 2021 15:19:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svenpeter.dev; s=protonmail3; t=1616253575; bh=rRb0CjHmraJyeX+g9eakf9Qm7lh28jtpNuhNTu/RlwY=; h=Date:To:From:Cc:Reply-To:Subject:From; b=RkCqzYspbvUCMMsy777eroY/bnpCj4qZJg+IEZvKlr2YFE7M6Hdw+mZpZWo5GcI7w 9x6vrc7I+OieqMiQ7QsdsLk3jQuSUSHY3Rn7JbcexUXVt4NCn/PsmjoXGH34vkQNkk 0CkWJHu8Yf6ztHIPbQU4AK4eqM3TCr4iUkDvbBM8HJAKYnHyVunlMDX0ZS9j56lRnw JURq8ktUnC6bscDInomvcHgpZdUpgqHFNMCxnP9ZqJDHd5bgllEDop1SXjEyQyeTdh Upi3/K2G3ZeBRcxeDz5irK1VM7VXyb7mWD77l++lETMSNooQXJyEuvcGwiURyqzls0 6Mn0vD7ohrQtw== To: iommu@lists.linux-foundation.org From: Sven Peter Cc: Joerg Roedel , Will Deacon , Robin Murphy , Rob Herring , Arnd Bergmann , Hector Martin , Mark Kettenis , Marc Zyngier , Mohamed Mediouni , Stan Skowronek , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH 0/3] Apple M1 DART IOMMU driver Message-ID: <20210320151903.60759-1-sven@svenpeter.dev> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210320_151942_746331_03FF786C X-CRM114-Status: GOOD ( 15.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sven Peter Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, After Hector's initial work [1] to bring up Linux on Apple's M1 it's time to bring up more devices. Most peripherals connected to the SoC are behind a iommu which Apple calls "Device Address Resolution Table", or DART for short [2]. Unfortunately, it only shares the name with PowerPC's DART. Configuring this iommu is mandatory if these peripherals require DMA access. This patchset implements initial support for this iommu. The hardware itself uses a pagetable format that's very similar to the one already implement in io-pgtable.c. There are some minor modifications, namely some details of the PTE format and that there are always three pagetable levels, which I've implement as a new format variant. I have mainly tested this with the USB controller in device mode which is compatible with Linux's dwc3 driver. Some custom PHY initialization (which is not yet ready or fully understood) is required though to bring up the ports, see e.g. my patches to our m1n1 bootloader [3,4]. If you want to test the same setup you will probably need that branch for now and add the nodes from the DT binding specification example to your device tree. Even though each DART instances could support up to 16 devices usually only a single device is actually connected. Different devices generally just use an entirely separate DART instance with a seperate MMIO range, IRQ, etc. I have just noticed today though that at least the USB DWC3 controller in host mode uses *two* darts at the same time. I'm not sure yet which parts seem to require which DART instance. This means that we might need to support devices attached to two iommus simultaneously and just create the same iova mappings. Currently this only seems to be required for USB according to Apple's Device Tree. I see two options for this and would like to get feedback before I implement either one: 1) Change #iommu-cells = <1>; to #iommu-cells = <2>; and use the first cell to identify the DART and the second one to identify the master. The DART DT node would then also take two register ranges that would correspond to the two DARTs. Both instances use the same IRQ and the same clocks according to Apple's device tree and my experiments. This would keep a single device node and the DART driver would then simply map iovas in both DARTs if required. 2) Keep #iommu-cells as-is but support iommus = <&usb_dart1a 1>, <&usb_dart1b 0>; instead. This would then require two devices nodes for the two DART instances and some housekeeping in the DART driver to support mapping iovas in both DARTs. I believe omap-iommu.c supports this setup but I will have to read more code to understand the details there and figure out how to implement this in a sane way. I currently prefer the first option but I don't understand enough details of the iommu system to actually make an informed decision. I'm obviously also open to more options :-) Best regards, Sven [1] https://lore.kernel.org/linux-arch/20210304213902.83903-1-marcan@marcan.st/ [2] https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DataMgmt/DataMgmt.html [3] https://github.com/svenpeter42/m1n1/commit/1e2661abf5ea2c820297b3ff591235c408d19a34 [4] https://github.com/svenpeter42/m1n1/tree/usb-uartproxy-console-wip