From patchwork Wed May 29 20:54:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967729 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1530992A for ; Wed, 29 May 2019 20:56:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 077A028A38 for ; Wed, 29 May 2019 20:56:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ED73C28A4B; Wed, 29 May 2019 20:56:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8736B28A38 for ; Wed, 29 May 2019 20:56:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726540AbfE2UzK (ORCPT ); Wed, 29 May 2019 16:55:10 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:55310 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726186AbfE2UzJ (ORCPT ); Wed, 29 May 2019 16:55:09 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id DDD9E607C3; Wed, 29 May 2019 20:55:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163307; bh=H1gJftayyhVU3QVkkOlFv2DhgJ/CrcDyVexTTU22x4A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lBBCCLNWLtMP6MTlifW0/hhWULMbqAHn8agLMArTbyVmXfLZY0mTYUbtDvXrhGJRB +rMjk9pU/Y5kmWw2na3O35JtABPJw1UFGa73g5+eW6WGZz0ZGgzD6SIKgOIL0poPBS VGANA337XjlLLCTgDmOZQgtPBQchIWcqJZaSBX0U= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id F21AB606DB; Wed, 29 May 2019 20:55:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163307; bh=H1gJftayyhVU3QVkkOlFv2DhgJ/CrcDyVexTTU22x4A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lBBCCLNWLtMP6MTlifW0/hhWULMbqAHn8agLMArTbyVmXfLZY0mTYUbtDvXrhGJRB +rMjk9pU/Y5kmWw2na3O35JtABPJw1UFGa73g5+eW6WGZz0ZGgzD6SIKgOIL0poPBS VGANA337XjlLLCTgDmOZQgtPBQchIWcqJZaSBX0U= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org F21AB606DB Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 01/16] iommu/arm-smmu: Allow client devices to select direct mapping Date: Wed, 29 May 2019 14:54:37 -0600 Message-Id: <1559163292-4792-2-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Some client devices want to directly map the IOMMU themselves instead of using the DMA domain. Allow those devices to opt in to direct mapping by way of a list of compatible strings. v3: use iommu_request_dm_for_dev() to set up a default identity domain for a group, per Robin Signed-off-by: Jordan Crouse --- drivers/iommu/arm-smmu.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 5e54cc0..7537639 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -1235,6 +1235,35 @@ static int arm_smmu_domain_add_master(struct arm_smmu_domain *smmu_domain, return 0; } +struct arm_smmu_client_match_data { + bool direct_mapping; +}; + +static const struct arm_smmu_client_match_data qcom_adreno = { + .direct_mapping = true, +}; + +static const struct arm_smmu_client_match_data qcom_mdss = { + .direct_mapping = true, +}; + +static const struct of_device_id arm_smmu_client_of_match[] = { + { .compatible = "qcom,adreno", .data = &qcom_adreno }, + { .compatible = "qcom,mdp4", .data = &qcom_mdss }, + { .compatible = "qcom,mdss", .data = &qcom_mdss }, + { .compatible = "qcom,sdm845-mdss", .data = &qcom_mdss }, + {}, +}; + +static const struct arm_smmu_client_match_data * +arm_smmu_client_data(struct device *dev) +{ + const struct of_device_id *match = + of_match_device(arm_smmu_client_of_match, dev); + + return match ? match->data : NULL; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret; @@ -1450,6 +1479,7 @@ static int arm_smmu_add_device(struct device *dev) struct arm_smmu_device *smmu; struct arm_smmu_master_cfg *cfg; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + const struct arm_smmu_client_match_data *client; int i, ret; if (using_legacy_binding) { @@ -1512,6 +1542,11 @@ static int arm_smmu_add_device(struct device *dev) device_link_add(dev, smmu->dev, DL_FLAG_PM_RUNTIME | DL_FLAG_AUTOREMOVE_SUPPLIER); + client = arm_smmu_client_data(dev); + + if (client && client->direct_mapping) + iommu_request_dm_for_dev(dev); + return 0; out_cfg_free: From patchwork Wed May 29 20:54:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967685 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 54879912 for ; Wed, 29 May 2019 20:55:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47E73219AC for ; Wed, 29 May 2019 20:55:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C5C028965; Wed, 29 May 2019 20:55:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC525219AC for ; Wed, 29 May 2019 20:55:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726752AbfE2UzN (ORCPT ); Wed, 29 May 2019 16:55:13 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:55540 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726743AbfE2UzL (ORCPT ); Wed, 29 May 2019 16:55:11 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id E1A5B60F3F; Wed, 29 May 2019 20:55:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163310; bh=ao1igd18u0fKnkzgWlolVx+hvnaOxQo8OJMI9HBKlt4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jlh8ZatgBdfUYzda6tEzyZqE8+ObhJJCrv0D3nqtlgo6/sgkZW4Y/P6mRepiAKDZC GDTsegKrIFttDfIugupT/2G5HHqnszcofccBhEt3zho2wBIoJ6GqXXsubRwurDHE6N GmImIVqA3uSa5iJGNrbOYBOLC+gFIlGLVE35TzVs= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 0B02F60E5C; Wed, 29 May 2019 20:55:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163310; bh=ao1igd18u0fKnkzgWlolVx+hvnaOxQo8OJMI9HBKlt4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jlh8ZatgBdfUYzda6tEzyZqE8+ObhJJCrv0D3nqtlgo6/sgkZW4Y/P6mRepiAKDZC GDTsegKrIFttDfIugupT/2G5HHqnszcofccBhEt3zho2wBIoJ6GqXXsubRwurDHE6N GmImIVqA3uSa5iJGNrbOYBOLC+gFIlGLVE35TzVs= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 0B02F60E5C Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, iommu@lists.linux-foundation.org, Joerg Roedel , linux-kernel@vger.kernel.org Subject: [PATCH v3 02/16] iommu: Add DOMAIN_ATTR_SPLIT_TABLES Date: Wed, 29 May 2019 14:54:38 -0600 Message-Id: <1559163292-4792-3-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a new domain attribute to enable split pagetable support for devices devices that support it. Signed-off-by: Jordan Crouse --- include/linux/iommu.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index a815cf6..a2f07cf 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -128,6 +128,7 @@ enum iommu_attr { DOMAIN_ATTR_FSL_PAMUV1, DOMAIN_ATTR_NESTING, /* two stages of translation */ DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE, + DOMAIN_ATTR_SPLIT_TABLES, DOMAIN_ATTR_MAX, }; From patchwork Wed May 29 20:54:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CCDB92A for ; Wed, 29 May 2019 20:55:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D894204C1 for ; Wed, 29 May 2019 20:55:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 41A1A289AA; Wed, 29 May 2019 20:55:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 26CDD219AC for ; Wed, 29 May 2019 20:55:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726791AbfE2UzS (ORCPT ); Wed, 29 May 2019 16:55:18 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:56136 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726743AbfE2UzS (ORCPT ); Wed, 29 May 2019 16:55:18 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 8F33C611CE; Wed, 29 May 2019 20:55:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163316; bh=Um9kzaiAFz17yY+QP3HgIrY+GTguTper/EK2qg9F5K4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LZafntDiSQQ8FCBCnAankr8L9FLpUoHb00D+gXcsjiAtPBkdG3JvxAb1rgl7ELDpJ BfzfSKHPF+cpCJuJgJWf9t8OrEkvAA6mfjEDMv95Z/D4gyy/rOdZuiBT59SsF/qiDg yPKw40gClOZX2fLjyG1q00a/8VK0trYaE+1oUOO8= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id C2C5061112; Wed, 29 May 2019 20:55:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163313; bh=Um9kzaiAFz17yY+QP3HgIrY+GTguTper/EK2qg9F5K4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RQQrUIjpbbv9S6wOQ9TNXmCpwD5ToKPvL1jnzPtlV6LMZYihGjeMx4RdHQq6EX3Jl TzTxEeCPmVeGDb3jfjA3afSIEWWeaY1JAUDEG7b+zRYIup2N6CLQhyQdKzcpfyf836 cF84wYrJ8TdQZJMdDmB7DmrkQwjV2HOwaxAnfeAA= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org C2C5061112 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Zhen Lei , Rob Herring , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 03/16] iommu/io-pgtable-arm: Add support for AARCH64 split pagetables Date: Wed, 29 May 2019 14:54:39 -0600 Message-Id: <1559163292-4792-4-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a new sub-format ARM_64_LPAE_SPLIT_S1 to create and set up split pagetables (TTBR0 and TTBR1). The initialization function sets up the correct va_size and sign extension bits and correctly programs the TCR registers. Split pagetable formats use their own own map/unmap wrappers to ensure that the correct pagetable is selected based on the incoming iova but most of the heavy lifting is common. v3: New patch taking most of the TTBR1 specific code out of arm-smmu Signed-off-by: Jordan Crouse --- drivers/iommu/io-pgtable-arm.c | 261 +++++++++++++++++++++++++++++++++++++---- drivers/iommu/io-pgtable.c | 1 + include/linux/io-pgtable.h | 2 + 3 files changed, 240 insertions(+), 24 deletions(-) diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 4e21efb..6ee333b9 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -129,7 +129,12 @@ #define ARM_LPAE_TCR_TG0_64K (1 << 14) #define ARM_LPAE_TCR_TG0_16K (2 << 14) +#define ARM_LPAE_TCR_TG1_4K (0 << 30) +#define ARM_LPAE_TCR_TG1_64K (1 << 30) +#define ARM_LPAE_TCR_TG1_16K (2 << 30) + #define ARM_LPAE_TCR_SH0_SHIFT 12 +#define ARM_LPAE_TCR_SH1_SHIFT 28 #define ARM_LPAE_TCR_SH0_MASK 0x3 #define ARM_LPAE_TCR_SH_NS 0 #define ARM_LPAE_TCR_SH_OS 2 @@ -137,6 +142,8 @@ #define ARM_LPAE_TCR_ORGN0_SHIFT 10 #define ARM_LPAE_TCR_IRGN0_SHIFT 8 +#define ARM_LPAE_TCR_ORGN1_SHIFT 26 +#define ARM_LPAE_TCR_IRGN1_SHIFT 24 #define ARM_LPAE_TCR_RGN_MASK 0x3 #define ARM_LPAE_TCR_RGN_NC 0 #define ARM_LPAE_TCR_RGN_WBWA 1 @@ -147,6 +154,7 @@ #define ARM_LPAE_TCR_SL0_MASK 0x3 #define ARM_LPAE_TCR_T0SZ_SHIFT 0 +#define ARM_LPAE_TCR_T1SZ_SHIFT 16 #define ARM_LPAE_TCR_SZ_MASK 0xf #define ARM_LPAE_TCR_PS_SHIFT 16 @@ -163,6 +171,14 @@ #define ARM_LPAE_TCR_PS_48_BIT 0x5ULL #define ARM_LPAE_TCR_PS_52_BIT 0x6ULL +#define ARM_LPAE_TCR_SEP_SHIFT 47 +#define ARM_LPAE_TCR_SEP_31 (0x0ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_35 (0x1ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_39 (0x2ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_41 (0x3ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_43 (0x4ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_UPSTREAM (0x7ULL << ARM_LPAE_TCR_SEP_SHIFT) + #define ARM_LPAE_MAIR_ATTR_SHIFT(n) ((n) << 3) #define ARM_LPAE_MAIR_ATTR_MASK 0xff #define ARM_LPAE_MAIR_ATTR_DEVICE 0x04 @@ -188,11 +204,12 @@ struct arm_lpae_io_pgtable { struct io_pgtable iop; int levels; + u32 sep; size_t pgd_size; unsigned long pg_shift; unsigned long bits_per_level; - void *pgd; + void *pgd[2]; }; typedef u64 arm_lpae_iopte; @@ -437,7 +454,8 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, arm_lpae_iopte pte; if (data->iop.fmt == ARM_64_LPAE_S1 || - data->iop.fmt == ARM_32_LPAE_S1) { + data->iop.fmt == ARM_32_LPAE_S1 || + data->iop.fmt == ARM_64_LPAE_SPLIT_S1) { pte = ARM_LPAE_PTE_nG; if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ)) pte |= ARM_LPAE_PTE_AP_RDONLY; @@ -478,11 +496,10 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, return pte; } -static int arm_lpae_map(struct io_pgtable_ops *ops, unsigned long iova, - phys_addr_t paddr, size_t size, int iommu_prot) +static int _arm_lpae_map(struct arm_lpae_io_pgtable *data, unsigned long iova, + phys_addr_t paddr, size_t size, int iommu_prot, + arm_lpae_iopte *ptep) { - struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); - arm_lpae_iopte *ptep = data->pgd; int ret, lvl = ARM_LPAE_START_LVL(data); arm_lpae_iopte prot; @@ -505,12 +522,39 @@ static int arm_lpae_map(struct io_pgtable_ops *ops, unsigned long iova, return ret; } +static int arm_lpae_split_map(struct io_pgtable_ops *ops, unsigned long iova, + phys_addr_t paddr, size_t size, int iommu_prot) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + unsigned long mask = 1UL << data->sep; + arm_lpae_iopte *ptep; + + if (iova & mask) { + ptep = data->pgd[1]; + iova &= (mask - 1); + } else + ptep = data->pgd[0]; + + return _arm_lpae_map(data, iova, paddr, size, iommu_prot, ptep); +} + +static int arm_lpae_map(struct io_pgtable_ops *ops, unsigned long iova, + phys_addr_t paddr, size_t size, int iommu_prot) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + + return _arm_lpae_map(data, iova, paddr, size, iommu_prot, data->pgd[0]); +} + static void __arm_lpae_free_pgtable(struct arm_lpae_io_pgtable *data, int lvl, arm_lpae_iopte *ptep) { arm_lpae_iopte *start, *end; unsigned long table_size; + if (!ptep) + return; + if (lvl == ARM_LPAE_START_LVL(data)) table_size = data->pgd_size; else @@ -540,7 +584,8 @@ static void arm_lpae_free_pgtable(struct io_pgtable *iop) { struct arm_lpae_io_pgtable *data = io_pgtable_to_data(iop); - __arm_lpae_free_pgtable(data, ARM_LPAE_START_LVL(data), data->pgd); + __arm_lpae_free_pgtable(data, ARM_LPAE_START_LVL(data), data->pgd[0]); + __arm_lpae_free_pgtable(data, ARM_LPAE_START_LVL(data), data->pgd[1]); kfree(data); } @@ -651,11 +696,28 @@ static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data, return __arm_lpae_unmap(data, iova, size, lvl + 1, ptep); } +static size_t arm_lpae_split_unmap(struct io_pgtable_ops *ops, + unsigned long iova, size_t size) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + unsigned long mask = 1UL << data->sep; + arm_lpae_iopte *ptep; + int lvl = ARM_LPAE_START_LVL(data); + + if (iova & mask) { + ptep = data->pgd[1]; + iova &= (mask - 1); + } else + ptep = data->pgd[0]; + + return __arm_lpae_unmap(data, iova, size, lvl, ptep); +} + static size_t arm_lpae_unmap(struct io_pgtable_ops *ops, unsigned long iova, size_t size) { struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); - arm_lpae_iopte *ptep = data->pgd; + arm_lpae_iopte *ptep = data->pgd[0]; int lvl = ARM_LPAE_START_LVL(data); if (WARN_ON(iova >= (1ULL << data->iop.cfg.ias))) @@ -664,11 +726,11 @@ static size_t arm_lpae_unmap(struct io_pgtable_ops *ops, unsigned long iova, return __arm_lpae_unmap(data, iova, size, lvl, ptep); } -static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops, - unsigned long iova) +static phys_addr_t _arm_lpae_iova_to_phys(struct arm_lpae_io_pgtable *data, + unsigned long iova, + arm_lpae_iopte *ptep) { - struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); - arm_lpae_iopte pte, *ptep = data->pgd; + arm_lpae_iopte pte; int lvl = ARM_LPAE_START_LVL(data); do { @@ -700,6 +762,31 @@ static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops, return iopte_to_paddr(pte, data) | iova; } + +static phys_addr_t arm_lpae_split_iova_to_phys(struct io_pgtable_ops *ops, + unsigned long iova) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + unsigned long mask = 1UL << data->sep; + arm_lpae_iopte *ptep; + + if (iova & mask) { + ptep = data->pgd[1]; + iova &= (mask - 1); + } else + ptep = data->pgd[0]; + + return _arm_lpae_iova_to_phys(data, iova, ptep); +} + +static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops, + unsigned long iova) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + + return _arm_lpae_iova_to_phys(data, iova, data->pgd[0]); +} + static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg) { unsigned long granule, page_sizes; @@ -779,6 +866,9 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg) pgd_bits = va_bits - (data->bits_per_level * (data->levels - 1)); data->pgd_size = 1UL << (pgd_bits + ilog2(sizeof(arm_lpae_iopte))); + data->pgd[0] = NULL; + data->pgd[1] = NULL; + data->iop.ops = (struct io_pgtable_ops) { .map = arm_lpae_map, .unmap = arm_lpae_unmap, @@ -788,8 +878,8 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg) return data; } -static struct io_pgtable * -arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) +static struct arm_lpae_io_pgtable * +_arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg) { u64 reg; struct arm_lpae_io_pgtable *data; @@ -847,8 +937,6 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) reg |= (64ULL - cfg->ias) << ARM_LPAE_TCR_T0SZ_SHIFT; - /* Disable speculative walks through TTBR1 */ - reg |= ARM_LPAE_TCR_EPD1; cfg->arm_lpae_s1_cfg.tcr = reg; /* MAIRs */ @@ -863,24 +951,143 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) cfg->arm_lpae_s1_cfg.mair[1] = 0; /* Looking good; allocate a pgd */ - data->pgd = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); - if (!data->pgd) + data->pgd[0] = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); + if (!data->pgd[0]) goto out_free_data; /* Ensure the empty pgd is visible before any actual TTBR write */ wmb(); /* TTBRs */ - cfg->arm_lpae_s1_cfg.ttbr[0] = virt_to_phys(data->pgd); + cfg->arm_lpae_s1_cfg.ttbr[0] = virt_to_phys(data->pgd[0]); cfg->arm_lpae_s1_cfg.ttbr[1] = 0; + return data; + +out_free_data: + kfree(data); + return NULL; +} + + +/* Allocate split pagetables */ +static struct io_pgtable * +arm_64_lpae_alloc_pgtable_split_s1(struct io_pgtable_cfg *cfg, void *cookie) +{ + u64 reg; + struct arm_lpae_io_pgtable *data; + u32 sep; + + /* Figure out what the sign extension bit should be */ + switch (cfg->ias) { + case 32: + case 36: + case 40: + case 42: + case 44: + sep = cfg->ias - 1; + /* Adjust the address size to account for the extension bit */ + cfg->ias--; + break; + case 48: + /* IAS of 48 is a special case, it has a dedicated bit */ + sep = 48; + break; + default: + return NULL; + } + + data = _arm_64_lpae_alloc_pgtable_s1(cfg); + if (!data) + return NULL; + + /* Add the TTBR1 settings */ + reg = cfg->arm_lpae_s1_cfg.tcr; + + /* TCR */ + reg |= (ARM_LPAE_TCR_SH_IS << ARM_LPAE_TCR_SH1_SHIFT) | + (ARM_LPAE_TCR_RGN_WBWA << ARM_LPAE_TCR_IRGN1_SHIFT) | + (ARM_LPAE_TCR_RGN_WBWA << ARM_LPAE_TCR_ORGN1_SHIFT); + + switch (ARM_LPAE_GRANULE(data)) { + case SZ_4K: + reg |= ARM_LPAE_TCR_TG1_4K; + break; + case SZ_16K: + reg |= ARM_LPAE_TCR_TG1_16K; + break; + case SZ_64K: + reg |= ARM_LPAE_TCR_TG1_64K; + break; + } + + reg |= (64ULL - cfg->ias) << ARM_LPAE_TCR_T1SZ_SHIFT; + + switch (sep) { + case 31: + reg |= ARM_LPAE_TCR_SEP_31; + break; + case 35: + reg |= ARM_LPAE_TCR_SEP_35; + break; + case 39: + reg |= ARM_LPAE_TCR_SEP_39; + break; + case 41: + reg |= ARM_LPAE_TCR_SEP_41; + break; + case 43: + reg |= ARM_LPAE_TCR_SEP_43; + break; + case 48: + reg |= ARM_LPAE_TCR_SEP_UPSTREAM; + break; + } + + cfg->arm_lpae_s1_cfg.tcr = reg; + + /* Allocate the TTBR1 pagetable */ + data->pgd[1] = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); + if (!data->pgd[1]) + goto out_free_data; + + /* Override the data ops with split table specific ops */ + data->iop.ops = (struct io_pgtable_ops) { + .map = arm_lpae_split_map, + .unmap = arm_lpae_split_unmap, + .iova_to_phys = arm_lpae_split_iova_to_phys, + }; + + /* + * remember the sign extension bit, we'll need it later to figure out + * which pagetable to use + */ + data->sep = sep; + + /* Ensure the empty pgd is visible before any actual TTBR write */ + wmb(); + + cfg->arm_lpae_s1_cfg.ttbr[1] = virt_to_phys(data->pgd[1]); return &data->iop; out_free_data: + __arm_lpae_free_pages(data->pgd[0], data->pgd_size, cfg); kfree(data); return NULL; } static struct io_pgtable * +arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) +{ + struct arm_lpae_io_pgtable *data; + + data = _arm_64_lpae_alloc_pgtable_s1(cfg); + if (!data) + return NULL; + + return &data->iop; +} + +static struct io_pgtable * arm_64_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *cfg, void *cookie) { u64 reg, sl; @@ -961,15 +1168,15 @@ arm_64_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *cfg, void *cookie) cfg->arm_lpae_s2_cfg.vtcr = reg; /* Allocate pgd pages */ - data->pgd = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); - if (!data->pgd) + data->pgd[0] = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); + if (!data->pgd[0]) goto out_free_data; /* Ensure the empty pgd is visible before any actual TTBR write */ wmb(); /* VTTBR */ - cfg->arm_lpae_s2_cfg.vttbr = virt_to_phys(data->pgd); + cfg->arm_lpae_s2_cfg.vttbr = virt_to_phys(data->pgd[0]); return &data->iop; out_free_data: @@ -1042,6 +1249,11 @@ struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = { .free = arm_lpae_free_pgtable, }; +struct io_pgtable_init_fns io_pgtable_arm_64_lpae_split_s1_init_fns = { + .alloc = arm_64_lpae_alloc_pgtable_split_s1, + .free = arm_lpae_free_pgtable, +}; + struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns = { .alloc = arm_64_lpae_alloc_pgtable_s2, .free = arm_lpae_free_pgtable, @@ -1096,9 +1308,9 @@ static void __init arm_lpae_dump_ops(struct io_pgtable_ops *ops) pr_err("cfg: pgsize_bitmap 0x%lx, ias %u-bit\n", cfg->pgsize_bitmap, cfg->ias); - pr_err("data: %d levels, 0x%zx pgd_size, %lu pg_shift, %lu bits_per_level, pgd @ %p\n", + pr_err("data: %d levels, 0x%zx pgd_size, %lu pg_shift, %lu bits_per_level, pgd @ %p %p\n", data->levels, data->pgd_size, data->pg_shift, - data->bits_per_level, data->pgd); + data->bits_per_level, data->pgd[0], data->pgd[1]); } #define __FAIL(ops, i) ({ \ @@ -1113,6 +1325,7 @@ static int __init arm_lpae_run_tests(struct io_pgtable_cfg *cfg) static const enum io_pgtable_fmt fmts[] = { ARM_64_LPAE_S1, ARM_64_LPAE_S2, + ARM_64_LPAE_SPLIT_S1, }; int i, j; diff --git a/drivers/iommu/io-pgtable.c b/drivers/iommu/io-pgtable.c index 5227cfd..58d0012 100644 --- a/drivers/iommu/io-pgtable.c +++ b/drivers/iommu/io-pgtable.c @@ -29,6 +29,7 @@ io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = { [ARM_32_LPAE_S1] = &io_pgtable_arm_32_lpae_s1_init_fns, [ARM_32_LPAE_S2] = &io_pgtable_arm_32_lpae_s2_init_fns, [ARM_64_LPAE_S1] = &io_pgtable_arm_64_lpae_s1_init_fns, + [ARM_64_LPAE_SPLIT_S1] = &io_pgtable_arm_64_lpae_split_s1_init_fns, [ARM_64_LPAE_S2] = &io_pgtable_arm_64_lpae_s2_init_fns, [ARM_MALI_LPAE] = &io_pgtable_arm_mali_lpae_init_fns, #endif diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h index 76969a5..821080c 100644 --- a/include/linux/io-pgtable.h +++ b/include/linux/io-pgtable.h @@ -10,6 +10,7 @@ enum io_pgtable_fmt { ARM_32_LPAE_S1, ARM_32_LPAE_S2, ARM_64_LPAE_S1, + ARM_64_LPAE_SPLIT_S1, ARM_64_LPAE_S2, ARM_V7S, ARM_MALI_LPAE, @@ -214,6 +215,7 @@ extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns; +extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_split_s1_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns; From patchwork Wed May 29 20:54:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 779DC15E6 for ; Wed, 29 May 2019 20:55:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B04F204C1 for ; Wed, 29 May 2019 20:55:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 697F5219AC; Wed, 29 May 2019 20:55:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A4802893B for ; Wed, 29 May 2019 20:55:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726636AbfE2Uz1 (ORCPT ); Wed, 29 May 2019 16:55:27 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:56274 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726787AbfE2UzT (ORCPT ); Wed, 29 May 2019 16:55:19 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 2BB0C61212; Wed, 29 May 2019 20:55:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163318; bh=n2zt0GanUIWLgrJz7fCZ/sI6rrlNw5KuRp/QXgNCtHw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i07xyBpW+ItVIUgetfisBNxNETW/8YdqkSOA/1K+BYUcH6NGATUnGlzXuOcei13RG Pd46vFoBmrBTNKh1brFPrK/A2gYNfEARPkMscsnpbPrkBzKZsyVuiP+2HTqGKsAE39 TDH/wZXGVErxssIWKOTBT8c3mbwAkIuMCm875OgA= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 29D3A61196; Wed, 29 May 2019 20:55:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163316; bh=n2zt0GanUIWLgrJz7fCZ/sI6rrlNw5KuRp/QXgNCtHw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kZvKvIeP81tykgOcfXH3sVPApm3sFC85c/7tkGXe1a5MFNhkk+42UCooFgk+5meM+ 0xgvg0tjAVTvAbqCdzV2LuLIHRl4Jpczd9i5o01dUQ+VADD2Xr2/MZCJBQjTNKPnm5 SaderNpmoB/EFBVrUYIcuNZUh+tG1mpG06kXFcZ0= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 29D3A61196 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 04/16] iommu/arm-smmu: Add support for DOMAIN_ATTR_SPLIT_TABLES Date: Wed, 29 May 2019 14:54:40 -0600 Message-Id: <1559163292-4792-5-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If DOMAIN_ATTR_SPLIT_TABLES is specified for a domain pass ARM_64_LPAE_SPLIT_S1 to io_pgtable_ops to allocate and initialize TTBR0 and TTBR1 pagetables. v3: Moved all the pagetable specific work into io-pgtable-arm in previous patch Signed-off-by: Jordan Crouse --- drivers/iommu/arm-smmu.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 7537639..33e6928 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -260,6 +260,7 @@ struct arm_smmu_domain { bool non_strict; struct mutex init_mutex; /* Protects smmu pointer */ spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ + u32 attributes; struct iommu_domain domain; }; @@ -835,7 +836,11 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, ias = smmu->va_size; oas = smmu->ipa_size; if (cfg->fmt == ARM_SMMU_CTX_FMT_AARCH64) { - fmt = ARM_64_LPAE_S1; + if (smmu_domain->attributes & + (1 << DOMAIN_ATTR_SPLIT_TABLES)) + fmt = ARM_64_LPAE_SPLIT_S1; + else + fmt = ARM_64_LPAE_S1; } else if (cfg->fmt == ARM_SMMU_CTX_FMT_AARCH32_L) { fmt = ARM_32_LPAE_S1; ias = min(ias, 32UL); @@ -1622,6 +1627,10 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain, case DOMAIN_ATTR_NESTING: *(int *)data = (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED); return 0; + case DOMAIN_ATTR_SPLIT_TABLES: + *(int *)data = !!(smmu_domain->attributes & + (1 << DOMAIN_ATTR_SPLIT_TABLES)); + return 0; default: return -ENODEV; } @@ -1662,6 +1671,11 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain, else smmu_domain->stage = ARM_SMMU_DOMAIN_S1; break; + case DOMAIN_ATTR_SPLIT_TABLES: + if (*((int *)data)) + smmu_domain->attributes |= + (1 << DOMAIN_ATTR_SPLIT_TABLES); + break; default: ret = -ENODEV; } From patchwork Wed May 29 20:54:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967689 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C899912 for ; Wed, 29 May 2019 20:55:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EDB0204C1 for ; Wed, 29 May 2019 20:55:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 219A8289B3; Wed, 29 May 2019 20:55:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BA12A288F1 for ; Wed, 29 May 2019 20:55:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726816AbfE2UzW (ORCPT ); Wed, 29 May 2019 16:55:22 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:56626 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726743AbfE2UzV (ORCPT ); Wed, 29 May 2019 16:55:21 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id C84B3609F3; Wed, 29 May 2019 20:55:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163320; bh=MNplseeIKAQ0I6ymvujag5Wo9kZnnfQJJGa0KJiYvTw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l6gTfJoSMrYl5TH3aNNHXFJTlII3aQimbOW5TFPGP4QOClPMcQYc3Vbdi1EmFuqbS Y/+UKJZhcr2YhF2n6tuhLiHyHvmqLMY/3I3KKnzN745qQWMmWt0ITWlJhLqL/kb6xz gEt0MYAd1ls7+XhPUzkXz4T7S2aNnaJdkpwTzTUI= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 316DD6133A; Wed, 29 May 2019 20:55:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163319; bh=MNplseeIKAQ0I6ymvujag5Wo9kZnnfQJJGa0KJiYvTw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kCF0pV8kDxPMcxpp+IyTex1gwg/kAHKVn4XAqB5esDSO93Xl+GZ/Y3itkDgvITbFW hsVAyZL2PjbfPvLGFb3/IQSk3xE/qZFsnXnioDQY2E/uCuCLzVdec4wzyCD0+mibcO paF/NvWuL6yXiI+I7DoQ//XSYK7/HmghFCVDIuNs= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 316DD6133A Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, iommu@lists.linux-foundation.org, Joerg Roedel , linux-kernel@vger.kernel.org Subject: [PATCH v3 05/16] iommu: Add DOMAIN_ATTR_PTBASE Date: Wed, 29 May 2019 14:54:41 -0600 Message-Id: <1559163292-4792-6-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add an attribute to return the base address of the pagetable. This is used by auxiliary domains from arm-smmu to return the address of the pagetable to the leaf driver so that it can set the appropriate pagetable through it's own means. Signed-off-by: Jordan Crouse --- include/linux/iommu.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index a2f07cf..49639b7 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -129,6 +129,7 @@ enum iommu_attr { DOMAIN_ATTR_NESTING, /* two stages of translation */ DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE, DOMAIN_ATTR_SPLIT_TABLES, + DOMAIN_ATTR_PTBASE, DOMAIN_ATTR_MAX, }; From patchwork Wed May 29 20:54:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967733 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EA2F9912 for ; Wed, 29 May 2019 20:56:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD27F204C1 for ; Wed, 29 May 2019 20:56:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D16692893B; Wed, 29 May 2019 20:56:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E418228965 for ; Wed, 29 May 2019 20:56:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726454AbfE2Uz0 (ORCPT ); Wed, 29 May 2019 16:55:26 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:57100 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726743AbfE2Uz0 (ORCPT ); Wed, 29 May 2019 16:55:26 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id CFC6A60A00; Wed, 29 May 2019 20:55:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163324; bh=6chReSEkevyOEDtSRSepdc5qrnqGOGKdxjwcEABL24Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FvcMAh/1Firmuvzj0ZCSwVjKCTArjYqcV0qHQj6XFLbMsDJlledodozqOrTqMi4HK HOxuZgpZV49FlCoulRdEUKEIuqrSIGy1DWjTnNwgKjNlEDA6CWt+btIxyQ+qbX5xIb lccFF7e0eBPpznfqvN+mAresPQiZqrzLpMhm6JWc= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 947D961515; Wed, 29 May 2019 20:55:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163322; bh=6chReSEkevyOEDtSRSepdc5qrnqGOGKdxjwcEABL24Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d5USjgZhLsCu/CcIgjmqin6bnpyeIQFXMMljimY56jQ7H7qj2YZRAIw/Lnv85+jJY 2WoD4QHTW2j80L9WeDMgPLIlYYbh5DI/IIUcOA+AHS7G8yw/ybq0ppdQY19XEFXM2S LUk0LmwpVHVqHBqYR++VLy3rFH5sljYo37kFdnas= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 947D961515 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 06/16] iommu/arm-smmu: Add auxiliary domain support for arm-smmuv2 Date: Wed, 29 May 2019 14:54:42 -0600 Message-Id: <1559163292-4792-7-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Support auxiliary domains for arm-smmu-v2 to initialize and support multiple pagetables for a single SMMU context bank. Since the smmu-v2 hardware doesn't have any built in support for switching the pagetable base it is left as an exercise to the caller to actually use the pagetable; aux domains in the IOMMU driver are only preoccupied with creating and managing the pagetable memory. Following is a pseudo code example of how a domain can be created /* Check to see if aux domains are supported */ if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { iommu = iommu_domain_alloc(...); if (iommu_aux_attach_device(domain, dev)) return FAIL; /* Save the base address of the pagetable for use by the driver iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); } Then 'domain' can be used like any other iommu domain to map and unmap iova addresses in the pagetable. The driver/hardware is used to switch the pagetable according to its own specific implementation. v3: Trivial update to reflect new pgtable ops situation Signed-off-by: Jordan Crouse --- drivers/iommu/arm-smmu.c | 125 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 110 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 33e6928..589da47 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -262,6 +262,8 @@ struct arm_smmu_domain { spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ u32 attributes; struct iommu_domain domain; + bool is_aux; + u64 ttbr0; }; struct arm_smmu_option_prop { @@ -803,6 +805,12 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, if (!(smmu->features & ARM_SMMU_FEAT_TRANS_S2)) smmu_domain->stage = ARM_SMMU_DOMAIN_S1; + /* Aux domains can only be created for stage-1 tables */ + if (smmu_domain->is_aux && smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + ret = -EINVAL; + goto out_unlock; + } + /* * Choosing a suitable context format is even more fiddly. Until we * grow some way for the caller to express a preference, and/or move @@ -850,6 +858,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, ias = min(ias, 32UL); oas = min(oas, 32UL); } + smmu_domain->tlb_ops = &arm_smmu_s1_tlb_ops; break; case ARM_SMMU_DOMAIN_NESTED: @@ -869,6 +878,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, ias = min(ias, 40UL); oas = min(oas, 40UL); } + if (smmu->version == ARM_SMMU_V2) smmu_domain->tlb_ops = &arm_smmu_s2_tlb_ops_v2; else @@ -878,23 +888,30 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, ret = -EINVAL; goto out_unlock; } - ret = __arm_smmu_alloc_bitmap(smmu->context_map, start, - smmu->num_context_banks); - if (ret < 0) - goto out_unlock; - cfg->cbndx = ret; - if (smmu->version < ARM_SMMU_V2) { - cfg->irptndx = atomic_inc_return(&smmu->irptndx); - cfg->irptndx %= smmu->num_context_irqs; - } else { - cfg->irptndx = cfg->cbndx; - } + /* + * Aux domains will use the same context bank assigned to the master + * domain for the device + */ + if (!smmu_domain->is_aux) { + ret = __arm_smmu_alloc_bitmap(smmu->context_map, start, + smmu->num_context_banks); + if (ret < 0) + goto out_unlock; - if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2) - cfg->vmid = cfg->cbndx + 1 + smmu->cavium_id_base; - else - cfg->asid = cfg->cbndx + smmu->cavium_id_base; + cfg->cbndx = ret; + if (smmu->version < ARM_SMMU_V2) { + cfg->irptndx = atomic_inc_return(&smmu->irptndx); + cfg->irptndx %= smmu->num_context_irqs; + } else { + cfg->irptndx = cfg->cbndx; + } + + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2) + cfg->vmid = cfg->cbndx + 1 + smmu->cavium_id_base; + else + cfg->asid = cfg->cbndx + smmu->cavium_id_base; + } pgtbl_cfg = (struct io_pgtable_cfg) { .pgsize_bitmap = smmu->pgsize_bitmap, @@ -917,11 +934,21 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, goto out_clear_smmu; } + /* Cache the TTBR0 for the aux domain */ + smmu_domain->ttbr0 = pgtbl_cfg.arm_lpae_s1_cfg.ttbr[0]; + /* Update the domain's page sizes to reflect the page table format */ domain->pgsize_bitmap = pgtbl_cfg.pgsize_bitmap; domain->geometry.aperture_end = (1UL << ias) - 1; domain->geometry.force_aperture = true; + /* + * aux domains don't use split tables or program the hardware so we're + * done setting it up + */ + if (smmu_domain->is_aux) + goto out; + /* Initialise the context bank with our page table cfg */ arm_smmu_init_context_bank(smmu_domain, &pgtbl_cfg); arm_smmu_write_context_bank(smmu, cfg->cbndx); @@ -939,6 +966,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, cfg->irptndx = INVALID_IRPTNDX; } +out: mutex_unlock(&smmu_domain->init_mutex); /* Publish page table ops for map/unmap */ @@ -962,6 +990,12 @@ static void arm_smmu_destroy_domain_context(struct iommu_domain *domain) if (!smmu || domain->type == IOMMU_DOMAIN_IDENTITY) return; + /* All we need to do for aux devices is destroy the pagetable */ + if (smmu_domain->is_aux) { + free_io_pgtable_ops(smmu_domain->pgtbl_ops); + return; + } + ret = arm_smmu_rpm_get(smmu); if (ret < 0) return; @@ -1242,14 +1276,17 @@ static int arm_smmu_domain_add_master(struct arm_smmu_domain *smmu_domain, struct arm_smmu_client_match_data { bool direct_mapping; + bool allow_aux_domain; }; static const struct arm_smmu_client_match_data qcom_adreno = { .direct_mapping = true, + .allow_aux_domain = true, }; static const struct arm_smmu_client_match_data qcom_mdss = { .direct_mapping = true, + .allow_aux_domain = false, }; static const struct of_device_id arm_smmu_client_of_match[] = { @@ -1269,6 +1306,55 @@ arm_smmu_client_data(struct device *dev) return match ? match->data : NULL; } +static bool arm_smmu_supports_aux(struct device *dev) +{ + const struct arm_smmu_client_match_data *data = + arm_smmu_client_data(dev); + + return (data && data->allow_aux_domain); +} + +static bool arm_smmu_dev_has_feat(struct device *dev, + enum iommu_dev_features feat) +{ + if (feat != IOMMU_DEV_FEAT_AUX) + return false; + + return arm_smmu_supports_aux(dev); +} + +static int arm_smmu_dev_enable_feat(struct device *dev, + enum iommu_dev_features feat) +{ + /* If supported aux domain support is always "on" */ + if (feat == IOMMU_DEV_FEAT_AUX && arm_smmu_supports_aux(dev)) + return 0; + + return -ENODEV; +} + +static int arm_smmu_dev_disable_feat(struct device *dev, + enum iommu_dev_features feat) +{ + return -EBUSY; +} + +/* Set up a new aux domain and create a new pagetable with the same + * characteristics as the master + */ +static int arm_smmu_aux_attach_dev(struct iommu_domain *domain, + struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu = fwspec_smmu(fwspec); + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + + smmu_domain->is_aux = true; + + /* No power is needed because aux domain doesn't touch the hardware */ + return arm_smmu_init_domain_context(domain, smmu); +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret; @@ -1631,6 +1717,11 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain, *(int *)data = !!(smmu_domain->attributes & (1 << DOMAIN_ATTR_SPLIT_TABLES)); return 0; + case DOMAIN_ATTR_PTBASE: + if (!smmu_domain->is_aux) + return -ENODEV; + *((u64 *)data) = smmu_domain->ttbr0; + return 0; default: return -ENODEV; } @@ -1741,7 +1832,11 @@ static struct iommu_ops arm_smmu_ops = { .capable = arm_smmu_capable, .domain_alloc = arm_smmu_domain_alloc, .domain_free = arm_smmu_domain_free, + .dev_has_feat = arm_smmu_dev_has_feat, + .dev_enable_feat = arm_smmu_dev_enable_feat, + .dev_disable_feat = arm_smmu_dev_disable_feat, .attach_dev = arm_smmu_attach_dev, + .aux_attach_dev = arm_smmu_aux_attach_dev, .map = arm_smmu_map, .unmap = arm_smmu_unmap, .flush_iotlb_all = arm_smmu_flush_iotlb_all, From patchwork Wed May 29 20:54:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967727 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7957E912 for ; Wed, 29 May 2019 20:56:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 694F1289A1 for ; Wed, 29 May 2019 20:56:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5D0BF2893B; Wed, 29 May 2019 20:56:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE3E9288F1 for ; Wed, 29 May 2019 20:56:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726876AbfE2Uzc (ORCPT ); Wed, 29 May 2019 16:55:32 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:57470 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726871AbfE2Uzb (ORCPT ); Wed, 29 May 2019 16:55:31 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 9554D61633; Wed, 29 May 2019 20:55:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163330; bh=B4EA6+W0bMBZWz+Img5LRozrErEZ8aBa+Kh0uCtpCTE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TcpYeA+J8rOgMttD2H3KglY6R7jLOe3K1UCH9fggHbm37SeKXGvlpRG3PHj1Amtse tg1kso1o1UcmG9bSW6+S996oydlPzSl+6tvffjQhBMLvh2iZCB/OcCZlkezQ6bn/Ta FSuHizi/9rAxav4AYWMWrNof19dPdqkHdvO7XXD8= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id C08B1613A7; Wed, 29 May 2019 20:55:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163326; bh=B4EA6+W0bMBZWz+Img5LRozrErEZ8aBa+Kh0uCtpCTE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NygVLsg7o0398FQypBvm8OZaapigH9X5AOxMFZdT6auiW/ETsNUtUV7r6kU3X2gl1 TADVCcURJb6BUnGC1JAtZSaK8/2loVlhRuteIWQWUHegvJJCq8jrhQzgdoOJY4FRQQ NTh2B7FxYdeI5aTT/fi1HfujDCisLEKiFF1h3aSU= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org C08B1613A7 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , Kees Cook , Wen Yang , Sharat Masetty , dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Rob Clark , David Airlie , Mamta Shukla , Daniel Vetter Subject: [PATCH v3 07/16] drm/msm/adreno: Enable 64 bit mode by default on a5xx and a6xx targets Date: Wed, 29 May 2019 14:54:43 -0600 Message-Id: <1559163292-4792-8-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A5XX and newer GPUs can be run in either 32 or 64 bit mode. The GPU registers and the microcode use 64 bit virtual addressing in either case but the upper 32 bits are ignored if the GPU is in 32 bit mode. There is no performance disadvantage to remaining in 64 bit mode even if we are only generating 32 bit addresses so switch over now to prepare for using addresses above 4G for targets that support them. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 14 ++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 14 ++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c index e5fcefa..43a2b4a 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c @@ -642,6 +642,20 @@ static int a5xx_hw_init(struct msm_gpu *gpu) REG_A5XX_RBBM_SECVID_TSB_TRUSTED_BASE_HI, 0x00000000); gpu_write(gpu, REG_A5XX_RBBM_SECVID_TSB_TRUSTED_SIZE, 0x00000000); + /* Put the GPU into 64 bit by default */ + gpu_write(gpu, REG_A5XX_CP_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_VSC_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_GRAS_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_RB_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_PC_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_HLSQ_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_VFD_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_VPC_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_UCHE_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_SP_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_TPL1_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A5XX_RBBM_SECVID_TSB_ADDR_MODE_CNTL, 0x1); + ret = adreno_hw_init(gpu); if (ret) return ret; diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 1f9f4b0..be39cf0 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -391,6 +391,20 @@ static int a6xx_hw_init(struct msm_gpu *gpu) REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_HI, 0x00000000); gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_SIZE, 0x00000000); + /* Turn on 64 bit addressing for all blocks */ + gpu_write(gpu, REG_A6XX_CP_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_VSC_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_GRAS_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_RB_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_PC_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_HLSQ_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_VFD_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_VPC_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_UCHE_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_SP_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_TPL1_ADDR_MODE_CNTL, 0x1); + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_ADDR_MODE_CNTL, 0x1); + /* enable hardware clockgating */ a6xx_set_hwcg(gpu, true); From patchwork Wed May 29 20:54:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967699 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE13615E6 for ; Wed, 29 May 2019 20:55:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FFCC289A1 for ; Wed, 29 May 2019 20:55:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 92BDD289AA; Wed, 29 May 2019 20:55:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37218289B3 for ; Wed, 29 May 2019 20:55:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726902AbfE2Uzf (ORCPT ); Wed, 29 May 2019 16:55:35 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:57702 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726896AbfE2Uzf (ORCPT ); Wed, 29 May 2019 16:55:35 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 8AD72616FE; Wed, 29 May 2019 20:55:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163334; bh=YbD1o3QTKsJc4Zlft7yYAtBneOymuecIq+BLGIcWNb8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ep2pnSl2ZoaT6WMCGII3T905YRQ401enAsz1ZMEeGvgla8cWuRUr5cjtxMM7Rxvq9 A4T4wQXhEr666vbzOh4M2SRFbqnBlzlY4mhDYf0w17cNoBDB+ZPZ8JfnPQl0hB62pY FBnI9wr+wc6SGvrQQcH4bgAK7NpApHYW++no8irw= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id A4616616B8; Wed, 29 May 2019 20:55:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163329; bh=YbD1o3QTKsJc4Zlft7yYAtBneOymuecIq+BLGIcWNb8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nYHYkyZJa+Lai5q8jqcgeBcDrEj6cQB62keJdOHVhbcNkhiNKBKQEn4nVWpyDFsui jsPKrwoJ57vFozh20pqRBWWpA9JbdrhdyZ8yqz+GPvvyQHnNA8qH5xmFUepULKgJWn 9XoPVBQKH3+h3Q8AT9R+CYggtVxjMUwIfmoarMy8= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org A4616616B8 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Rob Clark , David Airlie , Daniel Vetter Subject: [PATCH v3 08/16] drm/msm: Print all 64 bits of the faulting IOMMU address Date: Wed, 29 May 2019 14:54:44 -0600 Message-Id: <1559163292-4792-9-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When we move to 64 bit addressing for a5xx and a6xx targets we will start seeing pagefaults at larger addresses so format them appropriately in the log message for easier debugging. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/msm_iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c index 12bb54c..1926329 100644 --- a/drivers/gpu/drm/msm/msm_iommu.c +++ b/drivers/gpu/drm/msm/msm_iommu.c @@ -30,7 +30,7 @@ static int msm_fault_handler(struct iommu_domain *domain, struct device *dev, struct msm_iommu *iommu = arg; if (iommu->base.handler) return iommu->base.handler(iommu->base.arg, iova, flags); - pr_warn_ratelimited("*** fault: iova=%08lx, flags=%d\n", iova, flags); + pr_warn_ratelimited("*** fault: iova=%16lx, flags=%d\n", iova, flags); return 0; } From patchwork Wed May 29 20:54:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967703 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 869AA912 for ; Wed, 29 May 2019 20:55:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77661289A1 for ; Wed, 29 May 2019 20:55:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B06128A48; Wed, 29 May 2019 20:55:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D696728A38 for ; Wed, 29 May 2019 20:55:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726787AbfE2Uzk (ORCPT ); Wed, 29 May 2019 16:55:40 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:57810 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726903AbfE2Uzh (ORCPT ); Wed, 29 May 2019 16:55:37 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 82C6E6179C; Wed, 29 May 2019 20:55:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163335; bh=9XdIWFmX8TNK6D9uheFUQ+/bDecKzs/u3znPSS/ObVo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HA+8kVNYu6SeA3h2dBPTFplTgWSwCwtoBQVl8rbK7qTSWUNUZh/0cKoHqnKe+Qan4 Klau1mlPyVYAa7u6JFX5/NyIbpyxj1FMEcgz+vl6AjuXCSt1eP5nDgmAeoLIKfHmTN gxC2Rbk1Bm8KKa7NDJvjR8fmNbyo5ca/fvcKn5cw= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 02D6661709; Wed, 29 May 2019 20:55:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163332; bh=9XdIWFmX8TNK6D9uheFUQ+/bDecKzs/u3znPSS/ObVo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZIw/SGXpwPbuJOiCkJhsWdu1oKeEJFnCsV4tC7GqX8s0p3Cr6XL/GQysg1/y7XV/R Vd1kIjBqDu3BepHVYddFMHU/3K/ylWShAQaUkOn6PsviJhMHr1i0eoo6NhC+MpiJm4 +qt6s/Ak9Ou7UX4tdfEwRBlJtb7Ygs6HF4N5FEnA= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 02D6661709 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Rob Clark , David Airlie , Daniel Vetter Subject: [PATCH v3 09/16] drm/msm: Pass the MMU domain index in struct msm_file_private Date: Wed, 29 May 2019 14:54:45 -0600 Message-Id: <1559163292-4792-10-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Pass the index of the MMU domain in struct msm_file_private instead of assuming gpu->id throughout the submit path. This clears the way to change ctx->aspace to a per-instance pagetable. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/msm_drv.c | 2 ++ drivers/gpu/drm/msm/msm_drv.h | 1 + drivers/gpu/drm/msm/msm_gem.h | 1 + drivers/gpu/drm/msm/msm_gem_submit.c | 13 ++++++++----- drivers/gpu/drm/msm/msm_gpu.c | 5 ++--- 5 files changed, 14 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index 31deb87..4c51063 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -611,6 +611,7 @@ static void load_gpu(struct drm_device *dev) static int context_init(struct drm_device *dev, struct drm_file *file) { + struct msm_drm_private *priv = dev->dev_private; struct msm_file_private *ctx; ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); @@ -619,6 +620,7 @@ static int context_init(struct drm_device *dev, struct drm_file *file) msm_submitqueue_init(dev, ctx); + ctx->aspace = priv->gpu->aspace; file->driver_priv = ctx; return 0; diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h index e20e6b4..d9aa7ba 100644 --- a/drivers/gpu/drm/msm/msm_drv.h +++ b/drivers/gpu/drm/msm/msm_drv.h @@ -68,6 +68,7 @@ struct msm_file_private { rwlock_t queuelock; struct list_head submitqueues; int queueid; + struct msm_gem_address_space *aspace; }; enum msm_mdp_plane_property { diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 812d1b1..36aeb58 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -141,6 +141,7 @@ void msm_gem_free_work(struct work_struct *work); struct msm_gem_submit { struct drm_device *dev; struct msm_gpu *gpu; + struct msm_gem_address_space *aspace; struct list_head node; /* node in ring submit list */ struct list_head bo_list; struct ww_acquire_ctx ticket; diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index 1b68130..d3801bf 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -32,8 +32,9 @@ #define BO_PINNED 0x2000 static struct msm_gem_submit *submit_create(struct drm_device *dev, - struct msm_gpu *gpu, struct msm_gpu_submitqueue *queue, - uint32_t nr_bos, uint32_t nr_cmds) + struct msm_gpu *gpu, struct msm_gem_address_space *aspace, + struct msm_gpu_submitqueue *queue, uint32_t nr_bos, + uint32_t nr_cmds) { struct msm_gem_submit *submit; uint64_t sz = sizeof(*submit) + ((u64)nr_bos * sizeof(submit->bos[0])) + @@ -47,6 +48,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, return NULL; submit->dev = dev; + submit->aspace = aspace; submit->gpu = gpu; submit->fence = NULL; submit->cmd = (void *)&submit->bos[nr_bos]; @@ -160,7 +162,7 @@ static void submit_unlock_unpin_bo(struct msm_gem_submit *submit, struct msm_gem_object *msm_obj = submit->bos[i].obj; if (submit->bos[i].flags & BO_PINNED) - msm_gem_unpin_iova(&msm_obj->base, submit->gpu->aspace); + msm_gem_unpin_iova(&msm_obj->base, submit->aspace); if (submit->bos[i].flags & BO_LOCKED) ww_mutex_unlock(&msm_obj->base.resv->lock); @@ -264,7 +266,7 @@ static int submit_pin_objects(struct msm_gem_submit *submit) /* if locking succeeded, pin bo: */ ret = msm_gem_get_and_pin_iova(&msm_obj->base, - submit->gpu->aspace, &iova); + submit->aspace, &iova); if (ret) break; @@ -477,7 +479,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, } } - submit = submit_create(dev, gpu, queue, args->nr_bos, args->nr_cmds); + submit = submit_create(dev, gpu, ctx->aspace, queue, args->nr_bos, + args->nr_cmds); if (!submit) { ret = -ENOMEM; goto out_unlock; diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index bf4ee27..0a4c77f 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -684,7 +684,7 @@ static void retire_submit(struct msm_gpu *gpu, struct msm_ringbuffer *ring, struct msm_gem_object *msm_obj = submit->bos[i].obj; /* move to inactive: */ msm_gem_move_to_inactive(&msm_obj->base); - msm_gem_unpin_iova(&msm_obj->base, gpu->aspace); + msm_gem_unpin_iova(&msm_obj->base, submit->aspace); drm_gem_object_put(&msm_obj->base); } @@ -768,8 +768,7 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, /* submit takes a reference to the bo and iova until retired: */ drm_gem_object_get(&msm_obj->base); - msm_gem_get_and_pin_iova(&msm_obj->base, - submit->gpu->aspace, &iova); + msm_gem_get_and_pin_iova(&msm_obj->base, submit->aspace, &iova); if (submit->bos[i].flags & MSM_SUBMIT_BO_WRITE) msm_gem_move_to_active(&msm_obj->base, gpu, true, submit->fence); From patchwork Wed May 29 20:54:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967725 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA67692A for ; Wed, 29 May 2019 20:56:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC104288F1 for ; Wed, 29 May 2019 20:56:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C048528A38; Wed, 29 May 2019 20:56:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AB5A2288F1 for ; Wed, 29 May 2019 20:56:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726604AbfE2Uzp (ORCPT ); Wed, 29 May 2019 16:55:45 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:58108 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726944AbfE2Uzn (ORCPT ); Wed, 29 May 2019 16:55:43 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 1993861795; Wed, 29 May 2019 20:55:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163341; bh=YSKIV6DFCr7JB08CB8IZHjgCrJtdnzcRYTSNMf5lgv8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fbvQjTnihQExgjwE53rY+mjWvZ9cz464+UmqWCjHukvj8PfK7b3608wy3zx2bpOLY +3OJAiw1ncx1qDztVtZoG+W60lekuPzYd55vy0fQXocSbitfx3Ode3oucvZTnD3Jwi mjQF+KVyWCHCqRAiHD5erFZEV1t/0+cE1r1+6fIo= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 3633861795; Wed, 29 May 2019 20:55:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163337; bh=YSKIV6DFCr7JB08CB8IZHjgCrJtdnzcRYTSNMf5lgv8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mMU42Z0DjK6EOxT6wrMdqw+wv2PtWTVYWimS1JK4Y3L3YyDahGZi613pFXaDzYkjh BsCNhDXg1+HB1v+OX1QcEzQNm+hUH8XopPOfmfzTxyoZgsU2j65UVZz7q/dXZrhV7H CTXlx8x0xohDeYKssgM42RHb2zE7vYBHuUncYlzY= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 3633861795 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , Kees Cook , Thomas Zimmermann , Sharat Masetty , dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Rob Clark , David Airlie , Jonathan Marek , Mamta Shukla , Daniel Vetter Subject: [PATCH v3 10/16] drm/msm/gpu: Move address space setup to the GPU targets Date: Wed, 29 May 2019 14:54:46 -0600 Message-Id: <1559163292-4792-11-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Move the address space steup code out of the generic msm GPU code to to the individual GPU targets. This allows us to do target specific setup such as gpummu for a2xx or split pagetables and per-instance pagetables for newer a5xx and a6xx targets. All this is at the expense of duplicated code in some of the target files but I think it pays for itself in improved code flow and flexibility. v3: change NULl return to ERR_PTR in address space create functions Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 37 ++++++++++++++++------ drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 50 ++++++++++++++++++++++-------- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 51 +++++++++++++++++++++++-------- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 37 +++++++++++++++++++--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 37 +++++++++++++++++++--- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 7 ----- drivers/gpu/drm/msm/msm_gem.h | 1 + drivers/gpu/drm/msm/msm_gpu.c | 54 ++------------------------------- drivers/gpu/drm/msm/msm_gpu.h | 2 ++ 9 files changed, 173 insertions(+), 103 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a2xx_gpu.c b/drivers/gpu/drm/msm/adreno/a2xx_gpu.c index 1f83bc1..49241d0 100644 --- a/drivers/gpu/drm/msm/adreno/a2xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a2xx_gpu.c @@ -401,6 +401,30 @@ static struct msm_gpu_state *a2xx_gpu_state_get(struct msm_gpu *gpu) return state; } +static struct msm_gem_address_space * +a2xx_create_address_space(struct msm_gpu *gpu) +{ + struct msm_gem_address_space *aspace; + int ret; + + aspace = msm_gem_address_space_create_a2xx(&gpu->pdev->dev, gpu, + "gpu", SZ_16M, SZ_16M + 0xff * SZ_64K); + if (IS_ERR(aspace)) { + DRM_DEV_ERROR(gpu->dev->dev, + "No memory protection without MMU\n"); + return ERR_PTR(-ENXIO); + } + + ret = aspace->mmu->funcs->attach(aspace->mmu, NULL, 0); + if (ret) { + msm_gem_address_space_put(aspace); + return ERR_PTR(ret); + } + + return aspace; +} + + /* Register offset defines for A2XX - copy of A3XX */ static const unsigned int a2xx_register_offsets[REG_ADRENO_REGISTER_MAX] = { REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_BASE, REG_AXXX_CP_RB_BASE), @@ -429,6 +453,7 @@ static const struct adreno_gpu_funcs funcs = { #endif .gpu_state_get = a2xx_gpu_state_get, .gpu_state_put = adreno_gpu_state_put, + .create_address_space = a2xx_create_address_space, }, }; @@ -473,16 +498,8 @@ struct msm_gpu *a2xx_gpu_init(struct drm_device *dev) adreno_gpu->reg_offsets = a2xx_register_offsets; ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1); - if (ret) - goto fail; - - if (!gpu->aspace) { - dev_err(dev->dev, "No memory protection without MMU\n"); - ret = -ENXIO; - goto fail; - } - - return gpu; + if (!ret) + return gpu; fail: if (a2xx_gpu) diff --git a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c index c3b4bc6..33ab5e8 100644 --- a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c @@ -21,6 +21,7 @@ # include #endif +#include "msm_gem.h" #include "a3xx_gpu.h" #define A3XX_INT0_MASK \ @@ -433,6 +434,41 @@ static struct msm_gpu_state *a3xx_gpu_state_get(struct msm_gpu *gpu) return state; } +static struct msm_gem_address_space * +a3xx_create_address_space(struct msm_gpu *gpu) +{ + struct msm_gem_address_space *aspace; + struct iommu_domain *iommu; + int ret; + + iommu = iommu_domain_alloc(&platform_bus_type); + if (!iommu) { + DRM_DEV_ERROR(gpu->dev->dev, + "No memory protection without IOMMU\n"); + return ERR_PTR(-ENXIO); + } + + iommu->geometry.aperture_start = SZ_16M; + iommu->geometry.aperture_end = 0xffffffff; + + aspace = msm_gem_address_space_create(&gpu->pdev->dev, iommu, "gpu"); + if (IS_ERR(aspace)) { + iommu_domain_free(iommu); + DRM_DEV_ERROR(gpu->dev->dev, "failed to init mmu: %ld\n", + PTR_ERR(aspace)); + return aspace; + } + + ret = aspace->mmu->funcs->attach(aspace->mmu, NULL, 0); + if (ret) { + msm_gem_address_space_put(aspace); + return ERR_PTR(ret); + } + + return aspace; +} + + /* Register offset defines for A3XX */ static const unsigned int a3xx_register_offsets[REG_ADRENO_REGISTER_MAX] = { REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_BASE, REG_AXXX_CP_RB_BASE), @@ -461,6 +497,7 @@ static const struct adreno_gpu_funcs funcs = { #endif .gpu_state_get = a3xx_gpu_state_get, .gpu_state_put = adreno_gpu_state_put, + .create_address_space = a3xx_create_address_space, }, }; @@ -520,19 +557,6 @@ struct msm_gpu *a3xx_gpu_init(struct drm_device *dev) #endif } - if (!gpu->aspace) { - /* TODO we think it is possible to configure the GPU to - * restrict access to VRAM carveout. But the required - * registers are unknown. For now just bail out and - * limp along with just modesetting. If it turns out - * to not be possible to restrict access, then we must - * implement a cmdstream validator. - */ - DRM_DEV_ERROR(dev->dev, "No memory protection without IOMMU\n"); - ret = -ENXIO; - goto fail; - } - return gpu; fail: diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c index 18f9a8e..08a5729 100644 --- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c @@ -15,6 +15,8 @@ # include #endif +#include "msm_gem.h" + #define A4XX_INT0_MASK \ (A4XX_INT0_RBBM_AHB_ERROR | \ A4XX_INT0_RBBM_ATB_BUS_OVERFLOW | \ @@ -530,6 +532,41 @@ static int a4xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value) return 0; } +static struct msm_gem_address_space * +a4xx_create_address_space(struct msm_gpu *gpu) +{ + struct msm_gem_address_space *aspace; + struct iommu_domain *iommu; + int ret; + + iommu = iommu_domain_alloc(&platform_bus_type); + if (!iommu) { + DRM_DEV_ERROR(gpu->dev->dev, + "No memory protection without IOMMU\n"); + return ERR_PTR(-ENXIO); + } + + iommu->geometry.aperture_start = SZ_16M; + iommu->geometry.aperture_end = 0xffffffff; + + aspace = msm_gem_address_space_create(&gpu->pdev->dev, iommu, "gpu"); + if (IS_ERR(aspace)) { + iommu_domain_free(iommu); + DRM_DEV_ERROR(gpu->dev->dev, "failed to init mmu: %ld\n", + PTR_ERR(aspace)); + return aspace; + } + + ret = aspace->mmu->funcs->attach(aspace->mmu, NULL, 0); + if (ret) { + msm_gem_address_space_put(aspace); + return ERR_PTR(ret); + } + + return aspace; +} + + static const struct adreno_gpu_funcs funcs = { .base = { .get_param = adreno_get_param, @@ -547,6 +584,7 @@ static const struct adreno_gpu_funcs funcs = { #endif .gpu_state_get = a4xx_gpu_state_get, .gpu_state_put = adreno_gpu_state_put, + .create_address_space = a4xx_create_address_space, }, .get_timestamp = a4xx_get_timestamp, }; @@ -600,19 +638,6 @@ struct msm_gpu *a4xx_gpu_init(struct drm_device *dev) #endif } - if (!gpu->aspace) { - /* TODO we think it is possible to configure the GPU to - * restrict access to VRAM carveout. But the required - * registers are unknown. For now just bail out and - * limp along with just modesetting. If it turns out - * to not be possible to restrict access, then we must - * implement a cmdstream validator. - */ - DRM_DEV_ERROR(dev->dev, "No memory protection without IOMMU\n"); - ret = -ENXIO; - goto fail; - } - return gpu; fail: diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c index 43a2b4a..2f87c3e 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c @@ -1349,6 +1349,38 @@ static unsigned long a5xx_gpu_busy(struct msm_gpu *gpu) return (unsigned long)busy_time; } +static struct msm_gem_address_space * +a5xx_create_address_space(struct msm_gpu *gpu) +{ + struct msm_gem_address_space *aspace; + struct iommu_domain *iommu; + int ret; + + iommu = iommu_domain_alloc(&platform_bus_type); + if (!iommu) + return ERR_PTR(-ENXIO); + + iommu->geometry.aperture_start = 0x100000000ULL; + iommu->geometry.aperture_end = 0x1ffffffffULL; + + aspace = msm_gem_address_space_create(&gpu->pdev->dev, iommu, "gpu"); + if (IS_ERR(aspace)) { + iommu_domain_free(iommu); + DRM_DEV_ERROR(gpu->dev->dev, "failed to init mmu: %ld\n", + PTR_ERR(aspace)); + return aspace; + } + + ret = aspace->mmu->funcs->attach(aspace->mmu, NULL, 0); + if (ret) { + msm_gem_address_space_put(aspace); + return ERR_PTR(ret); + } + + msm_mmu_set_fault_handler(aspace->mmu, gpu, a5xx_fault_handler); + return aspace; +} + static const struct adreno_gpu_funcs funcs = { .base = { .get_param = adreno_get_param, @@ -1370,6 +1402,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_busy = a5xx_gpu_busy, .gpu_state_get = a5xx_gpu_state_get, .gpu_state_put = a5xx_gpu_state_put, + .create_address_space = a5xx_create_address_space, }, .get_timestamp = a5xx_get_timestamp, }; @@ -1416,7 +1449,6 @@ struct msm_gpu *a5xx_gpu_init(struct drm_device *dev) adreno_gpu->registers = a5xx_registers; adreno_gpu->reg_offsets = a5xx_register_offsets; - a5xx_gpu->lm_leakage = 0x4E001A; check_speed_bin(&pdev->dev); @@ -1427,9 +1459,6 @@ struct msm_gpu *a5xx_gpu_init(struct drm_device *dev) return ERR_PTR(ret); } - if (gpu->aspace) - msm_mmu_set_fault_handler(gpu->aspace->mmu, gpu, a5xx_fault_handler); - /* Set up the preemption specific bits and pieces for each ringbuffer */ a5xx_preempt_init(gpu); diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index be39cf0..3d70588 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -810,6 +810,38 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu) return (unsigned long)busy_time; } +static struct msm_gem_address_space * +a6xx_create_address_space(struct msm_gpu *gpu) +{ + struct msm_gem_address_space *aspace; + struct iommu_domain *iommu; + int ret; + + iommu = iommu_domain_alloc(&platform_bus_type); + if (!iommu) + return ERR_PTR(-ENXIO); + + iommu->geometry.aperture_start = 0x100000000ULL; + iommu->geometry.aperture_end = 0x1ffffffffULL; + + aspace = msm_gem_address_space_create(&gpu->pdev->dev, iommu, "gpu"); + if (IS_ERR(aspace)) { + iommu_domain_free(iommu); + DRM_DEV_ERROR(gpu->dev->dev, "failed to init mmu: %ld\n", + PTR_ERR(aspace)); + return aspace; + } + + ret = aspace->mmu->funcs->attach(aspace->mmu, NULL, 0); + if (ret) { + msm_gem_address_space_put(aspace); + return ERR_PTR(ret); + } + + msm_mmu_set_fault_handler(aspace->mmu, gpu, a6xx_fault_handler); + return aspace; +} + static const struct adreno_gpu_funcs funcs = { .base = { .get_param = adreno_get_param, @@ -832,6 +864,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_get = a6xx_gpu_state_get, .gpu_state_put = a6xx_gpu_state_put, #endif + .create_address_space = a6xx_create_address_space, }, .get_timestamp = a6xx_get_timestamp, }; @@ -874,9 +907,5 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) return ERR_PTR(ret); } - if (gpu->aspace) - msm_mmu_set_fault_handler(gpu->aspace->mmu, gpu, - a6xx_fault_handler); - return gpu; } diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c index 6f7f411..3ba7141 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c @@ -912,13 +912,6 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev, adreno_gpu->rev = config->rev; adreno_gpu_config.ioname = "kgsl_3d0_reg_memory"; - - adreno_gpu_config.va_start = SZ_16M; - adreno_gpu_config.va_end = 0xffffffff; - /* maximum range of a2xx mmu */ - if (adreno_is_a2xx(adreno_gpu)) - adreno_gpu_config.va_end = SZ_16M + 0xfff * SZ_64K; - adreno_gpu_config.nr_rings = nr_rings; adreno_get_pwrlevels(&pdev->dev, gpu); diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 36aeb58..fd67153 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -21,6 +21,7 @@ #include #include #include "msm_drv.h" +#include "msm_mmu.h" /* Additional internal-use only BO flags: */ #define MSM_BO_STOLEN 0x10000000 /* try to use stolen/splash memory */ diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index 0a4c77f..e8a14b0 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -20,7 +20,6 @@ #include "msm_mmu.h" #include "msm_fence.h" #include "msm_gpu_trace.h" -#include "adreno/adreno_gpu.h" #include #include @@ -812,51 +811,6 @@ static int get_clocks(struct platform_device *pdev, struct msm_gpu *gpu) return 0; } -static struct msm_gem_address_space * -msm_gpu_create_address_space(struct msm_gpu *gpu, struct platform_device *pdev, - uint64_t va_start, uint64_t va_end) -{ - struct msm_gem_address_space *aspace; - int ret; - - /* - * Setup IOMMU.. eventually we will (I think) do this once per context - * and have separate page tables per context. For now, to keep things - * simple and to get something working, just use a single address space: - */ - if (!adreno_is_a2xx(to_adreno_gpu(gpu))) { - struct iommu_domain *iommu = iommu_domain_alloc(&platform_bus_type); - if (!iommu) - return NULL; - - iommu->geometry.aperture_start = va_start; - iommu->geometry.aperture_end = va_end; - - DRM_DEV_INFO(gpu->dev->dev, "%s: using IOMMU\n", gpu->name); - - aspace = msm_gem_address_space_create(&pdev->dev, iommu, "gpu"); - if (IS_ERR(aspace)) - iommu_domain_free(iommu); - } else { - aspace = msm_gem_address_space_create_a2xx(&pdev->dev, gpu, "gpu", - va_start, va_end); - } - - if (IS_ERR(aspace)) { - DRM_DEV_ERROR(gpu->dev->dev, "failed to init mmu: %ld\n", - PTR_ERR(aspace)); - return ERR_CAST(aspace); - } - - ret = aspace->mmu->funcs->attach(aspace->mmu, NULL, 0); - if (ret) { - msm_gem_address_space_put(aspace); - return ERR_PTR(ret); - } - - return aspace; -} - int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, struct msm_gpu *gpu, const struct msm_gpu_funcs *funcs, const char *name, struct msm_gpu_config *config) @@ -929,12 +883,8 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, msm_devfreq_init(gpu); - gpu->aspace = msm_gpu_create_address_space(gpu, pdev, - config->va_start, config->va_end); - - if (gpu->aspace == NULL) - DRM_DEV_INFO(drm->dev, "%s: no IOMMU, fallback to VRAM carveout!\n", name); - else if (IS_ERR(gpu->aspace)) { + gpu->aspace = gpu->funcs->create_address_space(gpu); + if (IS_ERR(gpu->aspace)) { ret = PTR_ERR(gpu->aspace); goto fail; } diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index f2739cd..d4bf051 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -75,6 +75,8 @@ struct msm_gpu_funcs { int (*gpu_state_put)(struct msm_gpu_state *state); unsigned long (*gpu_get_freq)(struct msm_gpu *gpu); void (*gpu_set_freq)(struct msm_gpu *gpu, unsigned long freq); + struct msm_gem_address_space *(*create_address_space) + (struct msm_gpu *gpu); }; struct msm_gpu { From patchwork Wed May 29 20:54:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967713 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33AD2912 for ; Wed, 29 May 2019 20:55:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24403204C1 for ; Wed, 29 May 2019 20:55:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1892E288E4; Wed, 29 May 2019 20:55:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8437F2893B for ; Wed, 29 May 2019 20:55:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726741AbfE2Uzw (ORCPT ); Wed, 29 May 2019 16:55:52 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:58806 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726426AbfE2Uzv (ORCPT ); Wed, 29 May 2019 16:55:51 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id EEF2961893; Wed, 29 May 2019 20:55:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163350; bh=wBZB+LFIv8r0uTFu6MNzs2A3S+65mKQ9oZ1KI8mimLg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lYLYPQVETT4KRj48uDMXx//JHBH3/zmfNurE7CnleCXufjEIWMIYRAGe+88UggWou XR9rkw2ukAEjffcWLBMY7N2LbJNjz4xAamU1fVbI8sG62efuDhvgNct3o05/jRGPVa oT3umd0p2B37d4CGSFxyTgslWZSptdIon2QopDZs= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id A13386188E; Wed, 29 May 2019 20:55:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163340; bh=wBZB+LFIv8r0uTFu6MNzs2A3S+65mKQ9oZ1KI8mimLg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lRgOxT9EVeJWEkzqR48/Lagip5AY9QLHRg/itHQse7KN2tZWlBWOYM1M46xcdXv9T NQTaMd2APngD8tWkgsNyK8Qhv9U2aTqKfw0lF5Uwhd/7cscS7QixRcwpM32Ees32Cd Lb6OfAXicl0eWnlpOi7Lwc2ZJkR+bhmbK9B2Ei/c= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org A13386188E Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Rob Clark , David Airlie , Daniel Vetter Subject: [PATCH v3 11/16] drm/msm: Add support for IOMMU auxiliary domains Date: Wed, 29 May 2019 14:54:47 -0600 Message-Id: <1559163292-4792-12-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add support for creating a auxiliary domain from the IOMMU device to implement per-instance pagetables. Also add a helper function to return the pagetable base address (ttbr) and asid to the caller so that the GPU target code can set up the pagetable switch. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/msm_iommu.c | 97 +++++++++++++++++++++++++++++++++++++++++ drivers/gpu/drm/msm/msm_mmu.h | 4 ++ 2 files changed, 101 insertions(+) diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c index 1926329..adf9f18 100644 --- a/drivers/gpu/drm/msm/msm_iommu.c +++ b/drivers/gpu/drm/msm/msm_iommu.c @@ -21,9 +21,21 @@ struct msm_iommu { struct msm_mmu base; struct iommu_domain *domain; + u64 ttbr; + u32 asid; }; #define to_msm_iommu(x) container_of(x, struct msm_iommu, base) +/* + * The asid is currently unused for arm-smmu-v2 since all the pagetable + * switching does a TLBIALL but still assign a somewhat unique number per + * instance to leave open the possibility of being smarter about it + * + * Accepted range is 32 to 255 (starting at 32 gives a cushion for the asids + * assigned to the real context banks in the arm-smmu driver. + */ +static int msm_iommu_asid = 32; + static int msm_fault_handler(struct iommu_domain *domain, struct device *dev, unsigned long iova, int flags, void *arg) { @@ -34,6 +46,47 @@ static int msm_fault_handler(struct iommu_domain *domain, struct device *dev, return 0; } +static int msm_iommu_aux_attach(struct msm_mmu *mmu, const char * const *names, + int cnt) +{ + struct msm_iommu *iommu = to_msm_iommu(mmu); + int ret; + + /* Attach the aux device */ + ret = iommu_aux_attach_device(iommu->domain, mmu->dev); + if (ret) + return ret; + + /* Get the base address of the pagetable */ + ret = iommu_domain_get_attr(iommu->domain, DOMAIN_ATTR_PTBASE, + &iommu->ttbr); + if (ret) + return ret; + + /* + * Assign an asid for the instance even though the code doesn't + * currently support per-asid TLB invalidation. There isn't any + * protection on this so two instances could in theory end up with the + * same ASID but that would have very minor performance implications if + * per-ASID TLB invalidation were to be enabled in the future + */ + iommu->asid = msm_iommu_asid++; + + if (msm_iommu_asid > 0xff) + msm_iommu_asid = 32; + + return 0; +} + +static void msm_iommu_aux_detach(struct msm_mmu *mmu, const char * const *names, + int cnt) +{ + struct msm_iommu *iommu = to_msm_iommu(mmu); + + iommu->ttbr = 0; + iommu->asid = 0; +} + static int msm_iommu_attach(struct msm_mmu *mmu, const char * const *names, int cnt) { @@ -86,6 +139,50 @@ static const struct msm_mmu_funcs funcs = { .destroy = msm_iommu_destroy, }; +static const struct msm_mmu_funcs aux_funcs = { + .attach = msm_iommu_aux_attach, + .detach = msm_iommu_aux_detach, + .map = msm_iommu_map, + .unmap = msm_iommu_unmap, + .destroy = msm_iommu_destroy, +}; + +bool msm_iommu_get_ptinfo(struct msm_mmu *mmu, u64 *ttbr, u32 *asid) +{ + struct msm_iommu *iommu = to_msm_iommu(mmu); + + if (!iommu->ttbr) + return false; + + if (ttbr) + *ttbr = iommu->ttbr; + if (asid) + *asid = iommu->asid; + + return true; +} + + +struct msm_mmu *msm_iommu_new_instance(struct device *dev) +{ + struct msm_iommu *iommu; + + iommu = kzalloc(sizeof(*iommu), GFP_KERNEL); + if (!iommu) + return ERR_PTR(-ENOMEM); + + /* Create a new domain that will be attached as an aux domain */ + iommu->domain = iommu_domain_alloc(&platform_bus_type); + if (!iommu->domain) { + kfree(iommu); + return ERR_PTR(-ENOMEM); + } + + msm_mmu_init(&iommu->base, dev, &aux_funcs); + + return &iommu->base; +} + struct msm_mmu *msm_iommu_new(struct device *dev, struct iommu_domain *domain) { struct msm_iommu *iommu; diff --git a/drivers/gpu/drm/msm/msm_mmu.h b/drivers/gpu/drm/msm/msm_mmu.h index d21b266..f430903 100644 --- a/drivers/gpu/drm/msm/msm_mmu.h +++ b/drivers/gpu/drm/msm/msm_mmu.h @@ -46,6 +46,10 @@ static inline void msm_mmu_init(struct msm_mmu *mmu, struct device *dev, struct msm_mmu *msm_iommu_new(struct device *dev, struct iommu_domain *domain); struct msm_mmu *msm_gpummu_new(struct device *dev, struct msm_gpu *gpu); +struct msm_mmu *msm_iommu_new_instance(struct device *dev); + +bool msm_iommu_get_ptinfo(struct msm_mmu *mmu, u64 *ttbr, u32 *asid); + static inline void msm_mmu_set_fault_handler(struct msm_mmu *mmu, void *arg, int (*handler)(void *arg, unsigned long iova, int flags)) { From patchwork Wed May 29 20:54:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967717 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C460B92A for ; Wed, 29 May 2019 20:56:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B70B02893B for ; Wed, 29 May 2019 20:56:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ABA3628A38; Wed, 29 May 2019 20:56:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 905192893B for ; Wed, 29 May 2019 20:55:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726418AbfE2Uzu (ORCPT ); Wed, 29 May 2019 16:55:50 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:58594 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726944AbfE2Uzt (ORCPT ); Wed, 29 May 2019 16:55:49 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id BE6CC619B4; Wed, 29 May 2019 20:55:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163348; bh=9aXhSj9Phw8JHRWY+YFRT00OkAJ1xzvQFBspXjSWjck=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DlLTXI6D4ODG+V9UsMgaHCr2ZUr3pdKethOGTeAzd73+xCmiXfcfjSkiv6OjZqZgi Xfqfou1DZUy36SnkdB9GQttCDycxxsIhRXJjdn5f+F/p4AyhUsOmWS2ozWHwx7CDuZ 0hCIaouti6i/zeMGlpHEWdu0sztaSAdwLCBQttKg= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 4D0EB61893; Wed, 29 May 2019 20:55:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163344; bh=9aXhSj9Phw8JHRWY+YFRT00OkAJ1xzvQFBspXjSWjck=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eJqexVeSyscsIpxCzkF0HmshEvM+QN0sglQoNcW7Z8aL5ZWCNKEB9oRW2iMbYgyXM 9Y6D3h/4T6PzXLBdw4nEOiuLsTcEWER+bRD5EcrOLUOrOiwldI/HCkNwqkEjYaQHJZ zJtt+mbCrElkuDakahFKf/WM4Q05LYchCANkZQAY= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 4D0EB61893 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Rob Clark , David Airlie , Daniel Vetter Subject: [PATCH v3 12/16] drm/msm: Add a helper function for a per-instance address space Date: Wed, 29 May 2019 14:54:48 -0600 Message-Id: <1559163292-4792-13-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a helper function to create a GEM address space attached to an iommu auxiliary domain for a per-instance pagetable. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/msm_drv.h | 4 +++ drivers/gpu/drm/msm/msm_gem_vma.c | 53 +++++++++++++++++++++++---------------- 2 files changed, 36 insertions(+), 21 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h index d9aa7ba..1d4b45a 100644 --- a/drivers/gpu/drm/msm/msm_drv.h +++ b/drivers/gpu/drm/msm/msm_drv.h @@ -262,6 +262,10 @@ struct msm_gem_address_space * msm_gem_address_space_create_a2xx(struct device *dev, struct msm_gpu *gpu, const char *name, uint64_t va_start, uint64_t va_end); +struct msm_gem_address_space * +msm_gem_address_space_create_instance(struct device *dev, const char *name, + u64 va_start, u64 va_end); + int msm_register_mmu(struct drm_device *dev, struct msm_mmu *mmu); void msm_unregister_mmu(struct drm_device *dev, struct msm_mmu *mmu); diff --git a/drivers/gpu/drm/msm/msm_gem_vma.c b/drivers/gpu/drm/msm/msm_gem_vma.c index fcf7a83..0ee11b4 100644 --- a/drivers/gpu/drm/msm/msm_gem_vma.c +++ b/drivers/gpu/drm/msm/msm_gem_vma.c @@ -136,14 +136,12 @@ int msm_gem_init_vma(struct msm_gem_address_space *aspace, return 0; } - -struct msm_gem_address_space * -msm_gem_address_space_create(struct device *dev, struct iommu_domain *domain, - const char *name) +static struct msm_gem_address_space * +msm_gem_address_space_new(struct msm_mmu *mmu, const char *name, + u64 va_start, u64 va_end) { struct msm_gem_address_space *aspace; - u64 size = domain->geometry.aperture_end - - domain->geometry.aperture_start; + u64 size = va_end - va_start; aspace = kzalloc(sizeof(*aspace), GFP_KERNEL); if (!aspace) @@ -151,10 +149,9 @@ msm_gem_address_space_create(struct device *dev, struct iommu_domain *domain, spin_lock_init(&aspace->lock); aspace->name = name; - aspace->mmu = msm_iommu_new(dev, domain); + aspace->mmu = mmu; - drm_mm_init(&aspace->mm, (domain->geometry.aperture_start >> PAGE_SHIFT), - size >> PAGE_SHIFT); + drm_mm_init(&aspace->mm, (va_start >> PAGE_SHIFT), size >> PAGE_SHIFT); kref_init(&aspace->kref); @@ -162,24 +159,38 @@ msm_gem_address_space_create(struct device *dev, struct iommu_domain *domain, } struct msm_gem_address_space * +msm_gem_address_space_create(struct device *dev, struct iommu_domain *domain, + const char *name) +{ + struct msm_mmu *mmu = msm_iommu_new(dev, domain); + + if (IS_ERR(mmu)) + return ERR_CAST(mmu); + + return msm_gem_address_space_new(mmu, name, + domain->geometry.aperture_start, domain->geometry.aperture_end); +} + +struct msm_gem_address_space * msm_gem_address_space_create_a2xx(struct device *dev, struct msm_gpu *gpu, const char *name, uint64_t va_start, uint64_t va_end) { - struct msm_gem_address_space *aspace; - u64 size = va_end - va_start; + struct msm_mmu *mmu = msm_gpummu_new(dev, gpu); - aspace = kzalloc(sizeof(*aspace), GFP_KERNEL); - if (!aspace) - return ERR_PTR(-ENOMEM); + if (IS_ERR(mmu)) + return ERR_CAST(mmu); - spin_lock_init(&aspace->lock); - aspace->name = name; - aspace->mmu = msm_gpummu_new(dev, gpu); + return msm_gem_address_space_new(mmu, name, va_start, va_end); +} - drm_mm_init(&aspace->mm, (va_start >> PAGE_SHIFT), - size >> PAGE_SHIFT); +struct msm_gem_address_space * +msm_gem_address_space_create_instance(struct device *dev, const char *name, + u64 va_start, u64 va_end) +{ + struct msm_mmu *mmu = msm_iommu_new_instance(dev); - kref_init(&aspace->kref); + if (IS_ERR(mmu)) + return ERR_CAST(mmu); - return aspace; + return msm_gem_address_space_new(mmu, name, va_start, va_end); } From patchwork Wed May 29 20:54:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967723 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66B2392A for ; Wed, 29 May 2019 20:56:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59EF5288F1 for ; Wed, 29 May 2019 20:56:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4E702289A1; Wed, 29 May 2019 20:56:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFC6D288F1 for ; Wed, 29 May 2019 20:56:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726535AbfE2Uz7 (ORCPT ); Wed, 29 May 2019 16:55:59 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:59578 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726426AbfE2Uz6 (ORCPT ); Wed, 29 May 2019 16:55:58 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 82453618CE; Wed, 29 May 2019 20:55:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163357; bh=TB4M/rt699+7GpLoS9OHP7ZRv5MHTXpe7MAOFn1ir+A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FuwiQejBLS/4m99GWIgPpVK2e9XkORdqMKETMunlZZMy0qk3d4c0dLYh/wgAqMXtN X7Zy9ZzypsNXDpDzYQo86FNY+2kob5bLpkO95Bs2EGPQYeEMF77wycDCA7x1iiaugs nJSV/NN2Bd1UTzyHDREdjvmEF6NmblBKqaK21ZIM= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id E89656192E; Wed, 29 May 2019 20:55:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163347; bh=TB4M/rt699+7GpLoS9OHP7ZRv5MHTXpe7MAOFn1ir+A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OnhxSkrDBTnHxUssHyGW0vZs7PCS1z3HfaZYwlHPLcXL5/I/LzpKoBAQv3J/V27vP l8xO+iEqqfNVo4zst0gpmMyFqPhwglA9jAgEihn5VLAaY2KjtySe6VyhhUTILrjP4D 7mrSKsxib8vrOMrnO3Zmn2ONaTY6D0dRBvTR2dkc= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org E89656192E Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Rob Clark , David Airlie , Daniel Vetter Subject: [PATCH v3 13/16] drm/msm: Add support to create target specific address spaces Date: Wed, 29 May 2019 14:54:49 -0600 Message-Id: <1559163292-4792-14-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add support to create a GPU target specific address space for a context. For those targets that support per-instance pagetables they will return a new address space set up for the instance if possible otherwise just use the global device pagetable. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/msm_drv.c | 25 ++++++++++++++++++++++--- drivers/gpu/drm/msm/msm_gpu.h | 1 + 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index 4c51063..dd3eb30 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -609,6 +609,14 @@ static void load_gpu(struct drm_device *dev) mutex_unlock(&init_lock); } +static struct msm_gem_address_space *context_address_space(struct msm_gpu *gpu) +{ + if (!gpu->funcs->new_address_space) + return gpu->aspace; + + return gpu->funcs->new_address_space(gpu); +} + static int context_init(struct drm_device *dev, struct drm_file *file) { struct msm_drm_private *priv = dev->dev_private; @@ -618,9 +626,16 @@ static int context_init(struct drm_device *dev, struct drm_file *file) if (!ctx) return -ENOMEM; + ctx->aspace = context_address_space(priv->gpu); + if (IS_ERR(ctx->aspace)) { + int ret = PTR_ERR(ctx->aspace); + + kfree(ctx); + return ret; + } + msm_submitqueue_init(dev, ctx); - ctx->aspace = priv->gpu->aspace; file->driver_priv = ctx; return 0; @@ -636,8 +651,12 @@ static int msm_open(struct drm_device *dev, struct drm_file *file) return context_init(dev, file); } -static void context_close(struct msm_file_private *ctx) +static void context_close(struct msm_drm_private *priv, + struct msm_file_private *ctx) { + if (ctx->aspace != priv->gpu->aspace) + msm_gem_address_space_put(ctx->aspace); + msm_submitqueue_close(ctx); kfree(ctx); } @@ -652,7 +671,7 @@ static void msm_postclose(struct drm_device *dev, struct drm_file *file) priv->lastctx = NULL; mutex_unlock(&dev->struct_mutex); - context_close(ctx); + context_close(priv, ctx); } static irqreturn_t msm_irq(int irq, void *arg) diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index d4bf051..588d7ba 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -77,6 +77,7 @@ struct msm_gpu_funcs { void (*gpu_set_freq)(struct msm_gpu *gpu, unsigned long freq); struct msm_gem_address_space *(*create_address_space) (struct msm_gpu *gpu); + struct msm_gem_address_space *(*new_address_space)(struct msm_gpu *gpu); }; struct msm_gpu { From patchwork Wed May 29 20:54:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967739 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 794DD912 for ; Wed, 29 May 2019 20:56:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C0FE28A38 for ; Wed, 29 May 2019 20:56:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 603CD28A49; Wed, 29 May 2019 20:56:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 120D228A38 for ; Wed, 29 May 2019 20:56:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726453AbfE2U4V (ORCPT ); Wed, 29 May 2019 16:56:21 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:60772 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726038AbfE2U4V (ORCPT ); Wed, 29 May 2019 16:56:21 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 5E45D60FEB; Wed, 29 May 2019 20:55:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163380; bh=Q7ZQd878jM/ppuJ4uNWWJeXdoWzwLS/xcQibVXy6ISg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l0rkU5p6c78A9yyAOon8wO29jALyf4dfxO+tiGdkvP7bWZ9WFSJ10/Ip3jjwWHG0O RNnIdzTxZJG2kAhcjh19xc9ST7NdUeXIGAG/fCJWaDGLNPL1jlwLXXviGa1QDC08SY fI8vcgYLhZ1FY6UofJVUT+ULk3hYGm3QTB3H5bKo= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 7EAD460A63; Wed, 29 May 2019 20:55:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163351; bh=Q7ZQd878jM/ppuJ4uNWWJeXdoWzwLS/xcQibVXy6ISg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M2LzsUiEUpDhY8rHCdeqFQfHqXLFLPZdacqRZUXosKTXQTfzq5KShaUFbm1wW4qQn 40kWnbOGOvRIHp7Gp4FTcfbqjJM2Q4mdBpTAIAtYELW1ZbcY8Wo8WivQ0hEnzih6RA 0/QkDy4mNnOgEuNQzl2lUaH+cTbza0a1N23OSomg= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 7EAD460A63 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Rob Clark , David Airlie , Daniel Vetter Subject: [PATCH v3 14/16] drm/msm/gpu: Add ttbr0 to the memptrs Date: Wed, 29 May 2019 14:54:50 -0600 Message-Id: <1559163292-4792-15-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Targets that support per-instance pagetable switching will have to keep track of which pagetable belongs to each instance to be able to recover for preemption. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/msm_ringbuffer.h | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.h b/drivers/gpu/drm/msm/msm_ringbuffer.h index 6434ebb..493fa89 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.h +++ b/drivers/gpu/drm/msm/msm_ringbuffer.h @@ -40,6 +40,7 @@ struct msm_gpu_submit_stats { struct msm_rbmemptrs { volatile uint32_t rptr; volatile uint32_t fence; + volatile uint64_t ttbr0; volatile struct msm_gpu_submit_stats stats[MSM_GPU_SUBMIT_STATS_COUNT]; }; From patchwork Wed May 29 20:54:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967737 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48E5A15E6 for ; Wed, 29 May 2019 20:56:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B34D288CA for ; Wed, 29 May 2019 20:56:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3018528960; Wed, 29 May 2019 20:56:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 90FDF28965 for ; Wed, 29 May 2019 20:56:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726311AbfE2U4S (ORCPT ); Wed, 29 May 2019 16:56:18 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:60616 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726038AbfE2U4S (ORCPT ); Wed, 29 May 2019 16:56:18 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id DC915618F6; Wed, 29 May 2019 20:55:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163376; bh=8eb3TT7qBqZKcSLWpPylJf4TCDyFQpG0l2uLmE4Jm3w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZFOOvJJHvgpRUhO6cTXunqkO4r/0udV88jqpoSD3I/ylElt6Q8XiBTGTHEAuC6OYb rKbBOWjQ+Oj187Il6kCyWS4gszNShKYoOqqf/WUHEWBv8w+x+0OYzdSxLvoGtnDZV4 NkZuVeonVP+ZHMbRJmqpq9lQOTEtz4dPoFj2mSyc= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 0D94761A20; Wed, 29 May 2019 20:55:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163354; bh=8eb3TT7qBqZKcSLWpPylJf4TCDyFQpG0l2uLmE4Jm3w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mAEg4epFJbBDBMF194fxxqjbBJO0LX4xNb2jftwJYexCO50YVqtFW9U8a8s7D5CrM vzoMy4rYlVnnSKPxekgOfdHQiGQWjIpRy88CIkwaaH5VR4XrNuPSDn4SMR3s4jqR/3 bRjWLbFdR18Z/JTFd/9C3r3QqqOv0VAaLIgeWE80= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 0D94761A20 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , Sharat Masetty , dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Rob Clark , David Airlie , Daniel Vetter Subject: [PATCH v3 15/16] drm/msm/a6xx: Support per-instance pagetables Date: Wed, 29 May 2019 14:54:51 -0600 Message-Id: <1559163292-4792-16-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add support for per-instance pagetables for a6xx targets. Add support to handle split pagetables and create a new instance if the needed IOMMU support exists and insert the necessary PM4 commands to trigger a pagetable switch at the beginning of a user command. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 123 ++++++++++++++++++++++++++++++++-- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 + 2 files changed, 120 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 3d70588..1be8bfc 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -12,6 +12,62 @@ #define GPU_PAS_ID 13 +static void a6xx_set_pagetable(struct msm_gpu *gpu, struct msm_ringbuffer *ring, + struct msm_file_private *ctx) +{ + u64 ttbr; + u32 asid; + + if (!msm_iommu_get_ptinfo(ctx->aspace->mmu, &ttbr, &asid)) + return; + + ttbr = ttbr | ((u64) asid) << 48; + + /* Turn off protected mode */ + OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); + OUT_RING(ring, 0); + + /* Turn on APIV mode to access critical regions */ + OUT_PKT4(ring, REG_A6XX_CP_MISC_CNTL, 1); + OUT_RING(ring, 1); + + /* Make sure the ME is synchronized before staring the update */ + OUT_PKT7(ring, CP_WAIT_FOR_ME, 0); + + /* Execute the table update */ + OUT_PKT7(ring, CP_SMMU_TABLE_UPDATE, 4); + OUT_RING(ring, lower_32_bits(ttbr)); + OUT_RING(ring, upper_32_bits(ttbr)); + /* CONTEXTIDR is currently unused */ + OUT_RING(ring, 0); + /* CONTEXTBANK is currently unused */ + OUT_RING(ring, 0); + + /* + * Write the new TTBR0 to the preemption records - this will be used to + * reload the pagetable if the current ring gets preempted out. + */ + OUT_PKT7(ring, CP_MEM_WRITE, 4); + OUT_RING(ring, lower_32_bits(rbmemptr(ring, ttbr0))); + OUT_RING(ring, upper_32_bits(rbmemptr(ring, ttbr0))); + OUT_RING(ring, lower_32_bits(ttbr)); + OUT_RING(ring, upper_32_bits(ttbr)); + + /* Invalidate the draw state so we start off fresh */ + OUT_PKT7(ring, CP_SET_DRAW_STATE, 3); + OUT_RING(ring, 0x40000); + OUT_RING(ring, 1); + OUT_RING(ring, 0); + + /* Turn off APRIV */ + OUT_PKT4(ring, REG_A6XX_CP_MISC_CNTL, 1); + OUT_RING(ring, 0); + + /* Turn off protected mode */ + OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); + OUT_RING(ring, 1); +} + static inline bool _a6xx_check_idle(struct msm_gpu *gpu) { struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); @@ -89,6 +145,8 @@ static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, struct msm_ringbuffer *ring = submit->ring; unsigned int i; + a6xx_set_pagetable(gpu, ring, ctx); + get_stats_counter(ring, REG_A6XX_RBBM_PERFCTR_CP_0_LO, rbmemptr_stats(ring, index, cpcycles_start)); @@ -810,21 +868,77 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu) return (unsigned long)busy_time; } +static struct msm_gem_address_space *a6xx_new_address_space(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct msm_gem_address_space *aspace; + int ret; + + /* Return the default pagetable if per instance tables don't work */ + if (!a6xx_gpu->per_instance_tables) + return gpu->aspace; + + aspace = msm_gem_address_space_create_instance(&gpu->pdev->dev, "gpu", + 0x100000000ULL, 0x1ffffffffULL); + if (IS_ERR(aspace)) + return aspace; + + ret = aspace->mmu->funcs->attach(aspace->mmu, NULL, 0); + if (ret) { + /* -ENODEV means that aux domains aren't supported */ + if (ret == -ENODEV) + return gpu->aspace; + + return ERR_PTR(ret); + } + + return aspace; +} + static struct msm_gem_address_space * a6xx_create_address_space(struct msm_gpu *gpu) { + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct device *dev = &gpu->pdev->dev; struct msm_gem_address_space *aspace; struct iommu_domain *iommu; - int ret; + int ret, val = 1; + + a6xx_gpu->per_instance_tables = false; iommu = iommu_domain_alloc(&platform_bus_type); if (!iommu) return ERR_PTR(-ENXIO); - iommu->geometry.aperture_start = 0x100000000ULL; - iommu->geometry.aperture_end = 0x1ffffffffULL; + /* Try to enable split pagetables */ + if (iommu_domain_set_attr(iommu, DOMAIN_ATTR_SPLIT_TABLES, &val)) { + /* + * If split pagetables aren't available we won't be able to do + * per-instance pagetables so set up the global va space at our + * susual location + */ + iommu->geometry.aperture_start = 0x100000000ULL; + iommu->geometry.aperture_end = 0x1ffffffffULL; + } else { + /* + * If split pagetables are available then we might be able to do + * per-instance pagetables. Put the default va-space in TTBR1 to + * prepare + */ + iommu->geometry.aperture_start = 0xfffffff100000000ULL; + iommu->geometry.aperture_end = 0xffffff1ffffffffULL; + + /* + * If both split pagetables and aux domains are supported we can + * do per_instance pagetables + */ + a6xx_gpu->per_instance_tables = + iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX); + } - aspace = msm_gem_address_space_create(&gpu->pdev->dev, iommu, "gpu"); + aspace = msm_gem_address_space_create(dev, iommu, "gpu"); if (IS_ERR(aspace)) { iommu_domain_free(iommu); DRM_DEV_ERROR(gpu->dev->dev, "failed to init mmu: %ld\n", @@ -865,6 +979,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_put = a6xx_gpu_state_put, #endif .create_address_space = a6xx_create_address_space, + .new_address_space = a6xx_new_address_space, }, .get_timestamp = a6xx_get_timestamp, }; diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h index 6439955..b133aaa 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h @@ -21,6 +21,7 @@ struct a6xx_gpu { struct msm_ringbuffer *cur_ring; struct a6xx_gmu gmu; + bool per_instance_tables; }; #define to_a6xx_gpu(x) container_of(x, struct a6xx_gpu, base) From patchwork Wed May 29 20:54:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10967741 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD87A912 for ; Wed, 29 May 2019 20:56:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF6AF28960 for ; Wed, 29 May 2019 20:56:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C3A0628A48; Wed, 29 May 2019 20:56:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFFD728960 for ; Wed, 29 May 2019 20:56:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726362AbfE2U4k (ORCPT ); Wed, 29 May 2019 16:56:40 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:32936 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726043AbfE2U4k (ORCPT ); Wed, 29 May 2019 16:56:40 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id BAF2D6119F; Wed, 29 May 2019 20:56:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163398; bh=cUIPIGctA4wgg/uQMmcfCltYmcPMsZcXek/F57krbj8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hVA2FqlD5sSPHBF7dMppKb+Y6YGkkng9CJwtfwOrLvdIUDRhFC4VUVkSZZGFYCR4R 2RKjKkoq9ZwNgxbu1026tEESSvat34NI2etZ3CpqUt+mVezDubaloYby7Si8FYcPC1 S4eeRrI5yzZl0Xmlxs4hV6H5xPpBrtbJDG5i/IR0= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 7A5C661A64; Wed, 29 May 2019 20:55:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1559163359; bh=cUIPIGctA4wgg/uQMmcfCltYmcPMsZcXek/F57krbj8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P83qkCzwhSOwkMKxEoS1lLkD3rpGdJPj2h0iGI1atM/hUMjDunUuJfaue+901h6sp CxAh4OMZv2QDcIn/f0tsXhCWPMSNd1vXSXjAkuegx4bhcHTnjFDhLAJOKHgojYGf4V CTzYjxQ/tBj3uZe6ERCtsuL2IGVhuOzYhdSFjQww= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 7A5C661A64 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, hoegsberg@google.com, dianders@chromium.org, Sean Paul , Wen Yang , Thomas Zimmermann , Sharat Masetty , dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Rob Clark , David Airlie , Mamta Shukla , Daniel Vetter Subject: [PATCH v3 16/16] drm/msm/a5xx: Support per-instance pagetables Date: Wed, 29 May 2019 14:54:52 -0600 Message-Id: <1559163292-4792-17-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> References: <1559163292-4792-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add support for per-instance pagetables for 5XX targets. Create a support buffer for preemption to hold the SMMU pagetable information for a preempted ring, enable TTBR1 to support split pagetables and add the necessary PM4 commands to trigger a pagetable switch at the beginning of a user command. Signed-off-by: Jordan Crouse --- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 120 +++++++++++++++++++++++++++++- drivers/gpu/drm/msm/adreno/a5xx_gpu.h | 19 +++++ drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 70 +++++++++++++---- 3 files changed, 192 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c index 2f87c3e..fedd470 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c @@ -111,6 +111,59 @@ static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct msm_gem_submit *submit msm_gpu_retire(gpu); } +static void a5xx_set_pagetable(struct msm_gpu *gpu, struct msm_ringbuffer *ring, + struct msm_file_private *ctx) +{ + u64 ttbr; + u32 asid; + + if (!msm_iommu_get_ptinfo(ctx->aspace->mmu, &ttbr, &asid)) + return; + + ttbr = ttbr | ((u64) asid) << 48; + + /* Turn off protected mode */ + OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); + OUT_RING(ring, 0); + + /* Turn on APIV mode to access critical regions */ + OUT_PKT4(ring, REG_A5XX_CP_CNTL, 1); + OUT_RING(ring, 1); + + /* Make sure the ME is synchronized before staring the update */ + OUT_PKT7(ring, CP_WAIT_FOR_ME, 0); + + /* Execute the table update */ + OUT_PKT7(ring, CP_SMMU_TABLE_UPDATE, 3); + OUT_RING(ring, lower_32_bits(ttbr)); + OUT_RING(ring, upper_32_bits(ttbr)); + OUT_RING(ring, 0); + + /* + * Write the new TTBR0 to the preemption records - this will be used to + * reload the pagetable if the current ring gets preempted out. + */ + OUT_PKT7(ring, CP_MEM_WRITE, 4); + OUT_RING(ring, lower_32_bits(rbmemptr(ring, ttbr0))); + OUT_RING(ring, upper_32_bits(rbmemptr(ring, ttbr0))); + OUT_RING(ring, lower_32_bits(ttbr)); + OUT_RING(ring, upper_32_bits(ttbr)); + + /* Invalidate the draw state so we start off fresh */ + OUT_PKT7(ring, CP_SET_DRAW_STATE, 3); + OUT_RING(ring, 0x40000); + OUT_RING(ring, 1); + OUT_RING(ring, 0); + + /* Turn off APRIV */ + OUT_PKT4(ring, REG_A5XX_CP_CNTL, 1); + OUT_RING(ring, 0); + + /* Turn off protected mode */ + OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); + OUT_RING(ring, 1); +} + static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, struct msm_file_private *ctx) { @@ -126,6 +179,8 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, return; } + a5xx_set_pagetable(gpu, ring, ctx); + OUT_PKT7(ring, CP_PREEMPT_ENABLE_GLOBAL, 1); OUT_RING(ring, 0x02); @@ -1349,21 +1404,77 @@ static unsigned long a5xx_gpu_busy(struct msm_gpu *gpu) return (unsigned long)busy_time; } +static struct msm_gem_address_space *a5xx_new_address_space(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu); + struct msm_gem_address_space *aspace; + int ret; + + /* Return the default pagetable if per instance tables don't work */ + if (!a5xx_gpu->per_instance_tables) + return gpu->aspace; + + aspace = msm_gem_address_space_create_instance(&gpu->pdev->dev, + "gpu", 0x100000000ULL, 0x1ffffffffULL); + if (IS_ERR(aspace)) + return aspace; + + ret = aspace->mmu->funcs->attach(aspace->mmu, NULL, 0); + if (ret) { + /* -ENODEV means that aux domains aren't supported */ + if (ret == -ENODEV) + return gpu->aspace; + + return ERR_PTR(ret); + } + + return aspace; +} + static struct msm_gem_address_space * a5xx_create_address_space(struct msm_gpu *gpu) { + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu); + struct device *dev = &gpu->pdev->dev; struct msm_gem_address_space *aspace; struct iommu_domain *iommu; - int ret; + int ret, val = 1; + + a5xx_gpu->per_instance_tables = false; iommu = iommu_domain_alloc(&platform_bus_type); if (!iommu) return ERR_PTR(-ENXIO); - iommu->geometry.aperture_start = 0x100000000ULL; - iommu->geometry.aperture_end = 0x1ffffffffULL; + /* Try to enable split pagetables */ + if (iommu_domain_set_attr(iommu, DOMAIN_ATTR_SPLIT_TABLES, &val)) { + /* + * If split pagetables aren't available we won't be able to do + * per-instance pagetables so set up the global va space at our + * susual location + */ + iommu->geometry.aperture_start = 0x100000000ULL; + iommu->geometry.aperture_end = 0x1ffffffffULL; + } else { + /* + * If split pagetables are available then we might be able to do + * per-instance pagetables. Put the default va-space in TTBR1 to + * prepare + */ + iommu->geometry.aperture_start = 0xfffffff100000000ULL; + iommu->geometry.aperture_end = 0xfffffff1ffffffffULL; + + /* + * If both split pagetables and aux domains are supported we can + * do per_instance pagetables + */ + a5xx_gpu->per_instance_tables = + iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX); + } - aspace = msm_gem_address_space_create(&gpu->pdev->dev, iommu, "gpu"); + aspace = msm_gem_address_space_create(dev, iommu, "gpu"); if (IS_ERR(aspace)) { iommu_domain_free(iommu); DRM_DEV_ERROR(gpu->dev->dev, "failed to init mmu: %ld\n", @@ -1403,6 +1514,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_get = a5xx_gpu_state_get, .gpu_state_put = a5xx_gpu_state_put, .create_address_space = a5xx_create_address_space, + .new_address_space = a5xx_new_address_space, }, .get_timestamp = a5xx_get_timestamp, }; diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.h b/drivers/gpu/drm/msm/adreno/a5xx_gpu.h index 7d71860..82ceb9b 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.h +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.h @@ -45,6 +45,11 @@ struct a5xx_gpu { atomic_t preempt_state; struct timer_list preempt_timer; + struct a5xx_smmu_info *smmu_info; + struct drm_gem_object *smmu_info_bo; + uint64_t smmu_info_iova; + + bool per_instance_tables; }; #define to_a5xx_gpu(x) container_of(x, struct a5xx_gpu, base) @@ -132,6 +137,20 @@ struct a5xx_preempt_record { */ #define A5XX_PREEMPT_COUNTER_SIZE (16 * 4) +/* + * This is a global structure that the preemption code uses to switch in the + * pagetable for the preempted process - the code switches in whatever we + * after preempting in a new ring. + */ +struct a5xx_smmu_info { + uint32_t magic; + uint32_t _pad4; + uint64_t ttbr0; + uint32_t asid; + uint32_t contextidr; +}; + +#define A5XX_SMMU_INFO_MAGIC 0x3618CDA3UL int a5xx_power_init(struct msm_gpu *gpu); void a5xx_gpmu_ucode_init(struct msm_gpu *gpu); diff --git a/drivers/gpu/drm/msm/adreno/a5xx_preempt.c b/drivers/gpu/drm/msm/adreno/a5xx_preempt.c index 3d62310..1050409 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_preempt.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_preempt.c @@ -12,6 +12,7 @@ */ #include "msm_gem.h" +#include "msm_mmu.h" #include "a5xx_gpu.h" /* @@ -145,6 +146,15 @@ void a5xx_preempt_trigger(struct msm_gpu *gpu) a5xx_gpu->preempt[ring->id]->wptr = get_wptr(ring); spin_unlock_irqrestore(&ring->lock, flags); + /* Do read barrier to make sure we have updated pagetable info */ + rmb(); + + /* Set the SMMU info for the preemption */ + if (a5xx_gpu->smmu_info) { + a5xx_gpu->smmu_info->ttbr0 = ring->memptrs->ttbr0; + a5xx_gpu->smmu_info->contextidr = 0; + } + /* Set the address of the incoming preemption record */ gpu_write64(gpu, REG_A5XX_CP_CONTEXT_SWITCH_RESTORE_ADDR_LO, REG_A5XX_CP_CONTEXT_SWITCH_RESTORE_ADDR_HI, @@ -221,9 +231,10 @@ void a5xx_preempt_hw_init(struct msm_gpu *gpu) a5xx_gpu->preempt[i]->rbase = gpu->rb[i]->iova; } - /* Write a 0 to signal that we aren't switching pagetables */ + /* Tell the CP where to find the smmu_info buffer*/ gpu_write64(gpu, REG_A5XX_CP_CONTEXT_SWITCH_SMMU_INFO_LO, - REG_A5XX_CP_CONTEXT_SWITCH_SMMU_INFO_HI, 0); + REG_A5XX_CP_CONTEXT_SWITCH_SMMU_INFO_HI, + a5xx_gpu->smmu_info_iova); /* Reset the preemption state */ set_preempt_state(a5xx_gpu, PREEMPT_NONE); @@ -271,6 +282,34 @@ void a5xx_preempt_fini(struct msm_gpu *gpu) for (i = 0; i < gpu->nr_rings; i++) msm_gem_kernel_put(a5xx_gpu->preempt_bo[i], gpu->aspace, true); + + msm_gem_kernel_put(a5xx_gpu->smmu_info_bo, gpu->aspace, true); +} + +static int a5xx_smmu_info_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu); + struct a5xx_smmu_info *ptr; + struct drm_gem_object *bo; + u64 iova; + + if (!a5xx_gpu->per_instance_tables) + return 0; + + ptr = msm_gem_kernel_new(gpu->dev, sizeof(struct a5xx_smmu_info), + MSM_BO_UNCACHED, gpu->aspace, &bo, &iova); + + if (IS_ERR(ptr)) + return PTR_ERR(ptr); + + ptr->magic = A5XX_SMMU_INFO_MAGIC; + + a5xx_gpu->smmu_info_bo = bo; + a5xx_gpu->smmu_info_iova = iova; + a5xx_gpu->smmu_info = ptr; + + return 0; } void a5xx_preempt_init(struct msm_gpu *gpu) @@ -284,17 +323,22 @@ void a5xx_preempt_init(struct msm_gpu *gpu) return; for (i = 0; i < gpu->nr_rings; i++) { - if (preempt_init_ring(a5xx_gpu, gpu->rb[i])) { - /* - * On any failure our adventure is over. Clean up and - * set nr_rings to 1 to force preemption off - */ - a5xx_preempt_fini(gpu); - gpu->nr_rings = 1; - - return; - } + if (preempt_init_ring(a5xx_gpu, gpu->rb[i])) + goto fail; } - timer_setup(&a5xx_gpu->preempt_timer, a5xx_preempt_timer, 0); + if (a5xx_smmu_info_init(gpu)) + goto fail; + + timer_setup(&a5xx_gpu->preempt_timer, a5xx_preempt_timer, + (unsigned long) a5xx_gpu); + + return; +fail: + /* + * On any failure our adventure is over. Clean up and + * set nr_rings to 1 to force preemption off + */ + a5xx_preempt_fini(gpu); + gpu->nr_rings = 1; }