From patchwork Fri May 11 19:06:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10394985 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7B89760348 for ; Fri, 11 May 2018 19:09:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6806428F8B for ; Fri, 11 May 2018 19:09:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5CA1728F98; Fri, 11 May 2018 19:09:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B378D28F8B for ; Fri, 11 May 2018 19:09:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 77F986B06B0; Fri, 11 May 2018 15:09:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 72E9E6B06B1; Fri, 11 May 2018 15:09:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CFAB6B06B3; Fri, 11 May 2018 15:09:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ot0-f197.google.com (mail-ot0-f197.google.com [74.125.82.197]) by kanga.kvack.org (Postfix) with ESMTP id 2AA536B06B0 for ; Fri, 11 May 2018 15:09:52 -0400 (EDT) Received: by mail-ot0-f197.google.com with SMTP id a14-v6so4387352otf.1 for ; Fri, 11 May 2018 12:09:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=g9ub7OyH9/EaYWDH9cs6PAjhM66zbksX7HnMV/vk41o=; b=P0lffGaxXq/GbvIWQ84j7sZF90feXrXpMtEEgLWhrDKGTrqMIjxmN7vuSJbQ0fZDYc 07QGUtJLHmjdxT9Vbq/9Cgt4xUAKwjHcyQ5f9FiIUFlPa4cYpheAa7KNormHnzeeTzjv l4hU16sc0ywWqzVWkC8BbjtuC8eSVqIbE0rAEh0Ed9xar63sMa2OLpe1w4ASZSdtfkDr fEaq38JNxU6gY99R9NO5yj7EvHrZAmyl+tPTdhwTHVG8nRlDyB912dKnR9eBuhFYQB5h +8O5xsgdsulX4YQzGMMzfZlrVhum0wmqgDjLyCx9wBx4x/eShlwZcrrfe+hv2sNY8iy1 Wejw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jean-philippe.brucker@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=jean-philippe.brucker@arm.com X-Gm-Message-State: ALKqPweIGgR+hJTPygdI+roXtlIYTSss7djdvqiCpr3izmqRt8egOI0o o1ZtDw2UC+W8rMa/qTcYAFma/RqgrxAp9XuFaJd2w8B/7xyEk5LNheTWZwgr+4W2wh6TSJQtEkI DxDn1uUsMdlTwYQI/S/EvLkF7S5+FXt91/q/gw8SemCCRSwq6FyTSbldVLpzN/Q+Ggg== X-Received: by 2002:aca:3887:: with SMTP id f129-v6mr44670oia.14.1526065791932; Fri, 11 May 2018 12:09:51 -0700 (PDT) X-Google-Smtp-Source: AB8JxZq7gJ0BoYFxK5s+YwMXFCdSv7qkZpQv+eyR+Pmx62OnymqwVucJlK0CxZfRZKiGe+nvHYJx X-Received: by 2002:aca:3887:: with SMTP id f129-v6mr44634oia.14.1526065790992; Fri, 11 May 2018 12:09:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526065790; cv=none; d=google.com; s=arc-20160816; b=dcz0wQPXh6myWIaOmAVNxryn7VVLJCtxbDXME9Bm+12hzRdu8WZnljwlveysVBksyD CngGptMk2ZrijMzO3Ng+Z03JP91Fqs+IorGeNVhNWfFl7Nu3WOkbGSxrgZByT7gIX0ll bvZ6fcgtVISNkQVWEJJkTqoko8zBR5G00isNjPCIOP0eeIVGhud3ur03utjlIGwXBZoM 4XXJaEJ0kamrTtyqFNSpnxNpx+yhzXjfE07m2n/tboMoodVYQ7cgRrO8Yq64lXfN9beo u+QOcCmF3hhl+svXckQ0sUY0ClxgRBpXrEByB5dA+BpyfpKM4jEYkGxpuXFEuTxs0KsK n6Xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=g9ub7OyH9/EaYWDH9cs6PAjhM66zbksX7HnMV/vk41o=; b=fjz+Xalj6WRd63Z7Tkh83phw8oYbIUPephmbi7s5Qndm2gcIY3crLsrfwSEiRPrJlA ThPViJlujo5FrfD+G62pCCnIJrdGO4PWPWGIdOIqvrCqMPjqtsyFgzTnXsgZOrUwkVlK QWXo67R0mEzXGBURN3yi6DEQ0h/QB9xPTX6L7lbe73YlX/W5N/Cqtf5qCIeJYDXpywt6 UPC0YJRO29r/OCAKStBx801x3caod/yerGSt80Gsufmzi5Ll5mYmcdMmPMRC/u5NrKIK KNjLYtMwLW19BiUD3oebbYeFW6nLMHQgLQykFOmor6t16I2ubGBPiELqrUToM4Hpx48u DeMA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jean-philippe.brucker@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=jean-philippe.brucker@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id s92-v6si1329568ota.441.2018.05.11.12.09.50 for ; Fri, 11 May 2018 12:09:50 -0700 (PDT) Received-SPF: pass (google.com: domain of jean-philippe.brucker@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jean-philippe.brucker@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=jean-philippe.brucker@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8F2AB1993; Fri, 11 May 2018 12:09:50 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.210.33]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 511E53F23C; Fri, 11 May 2018 12:09:45 -0700 (PDT) From: Jean-Philippe Brucker To: linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-mm@kvack.org Cc: joro@8bytes.org, will.deacon@arm.com, robin.murphy@arm.com, alex.williamson@redhat.com, tn@semihalf.com, liubo95@huawei.com, thunder.leizhen@huawei.com, xieyisheng1@huawei.com, xuzaibo@huawei.com, ilias.apalodimas@linaro.org, jonathan.cameron@huawei.com, liudongdong3@huawei.com, shunyong.yang@hxt-semitech.com, nwatters@codeaurora.org, okaya@codeaurora.org, jcrouse@codeaurora.org, rfranz@cavium.com, dwmw2@infradead.org, jacob.jun.pan@linux.intel.com, yi.l.liu@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, baolu.lu@linux.intel.com, robdclark@gmail.com, christian.koenig@amd.com, bharatku@xilinx.com, rgummal@xilinx.com Subject: [PATCH v2 25/40] iommu/arm-smmu-v3: Add support for VHE Date: Fri, 11 May 2018 20:06:26 +0100 Message-Id: <20180511190641.23008-26-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker@arm.com> References: <20180511190641.23008-1-jean-philippe.brucker@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP ARMv8.1 extensions added Virtualization Host Extensions (VHE), which allow to run a host kernel at EL2. When using normal DMA, Device and CPU address spaces are dissociated, and do not need to implement the same capabilities, so VHE hasn't been used in the SMMU until now. With shared address spaces however, ASIDs are shared between MMU and SMMU, and broadcast TLB invalidations issued by a CPU are taken into account by the SMMU. TLB entries on both sides need to have identical exception level in order to be cleared with a single invalidation. When the CPU is using VHE, enable VHE in the SMMU for all STEs. Normal DMA mappings will need to use TLBI_EL2 commands instead of TLBI_NH, but shouldn't be otherwise affected by this change. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 16b08f2fb8ac..280a5d9be839 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -400,6 +401,8 @@ struct arm_smmu_cmdq_ent { #define CMDQ_OP_TLBI_NH_ASID 0x11 #define CMDQ_OP_TLBI_NH_VA 0x12 #define CMDQ_OP_TLBI_EL2_ALL 0x20 + #define CMDQ_OP_TLBI_EL2_ASID 0x21 + #define CMDQ_OP_TLBI_EL2_VA 0x22 #define CMDQ_OP_TLBI_S12_VMALL 0x28 #define CMDQ_OP_TLBI_S2_IPA 0x2a #define CMDQ_OP_TLBI_NSNH_ALL 0x30 @@ -519,6 +522,7 @@ struct arm_smmu_device { #define ARM_SMMU_FEAT_HYP (1 << 12) #define ARM_SMMU_FEAT_STALL_FORCE (1 << 13) #define ARM_SMMU_FEAT_VAX (1 << 14) +#define ARM_SMMU_FEAT_E2H (1 << 15) u32 features; #define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0) @@ -777,6 +781,7 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) cmd[1] |= FIELD_PREP(CMDQ_CFGI_1_RANGE, 31); break; case CMDQ_OP_TLBI_NH_VA: + case CMDQ_OP_TLBI_EL2_VA: cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_ASID, ent->tlbi.asid); cmd[1] |= FIELD_PREP(CMDQ_TLBI_1_LEAF, ent->tlbi.leaf); cmd[1] |= ent->tlbi.addr & CMDQ_TLBI_1_VA_MASK; @@ -792,6 +797,9 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) case CMDQ_OP_TLBI_S12_VMALL: cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); break; + case CMDQ_OP_TLBI_EL2_ASID: + cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_ASID, ent->tlbi.asid); + break; case CMDQ_OP_PRI_RESP: cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); @@ -1064,6 +1072,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, if (ste->s1_cfg) { struct iommu_pasid_table_cfg *cfg = &ste->s1_cfg->tables; + int strw = smmu->features & ARM_SMMU_FEAT_E2H ? + STRTAB_STE_1_STRW_EL2 : STRTAB_STE_1_STRW_NSEL1; BUG_ON(ste_live); dst[1] = cpu_to_le64( @@ -1074,7 +1084,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, #ifdef CONFIG_PCI_ATS FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | #endif - FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); + FIELD_PREP(STRTAB_STE_1_STRW, strw)); if (smmu->features & ARM_SMMU_FEAT_STALLS && !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) @@ -1325,7 +1335,8 @@ static void arm_smmu_tlb_inv_context(void *cookie) if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { if (unlikely(!smmu_domain->s1_cfg.cd0)) return; - cmd.opcode = CMDQ_OP_TLBI_NH_ASID; + cmd.opcode = smmu->features & ARM_SMMU_FEAT_E2H ? + CMDQ_OP_TLBI_EL2_ASID : CMDQ_OP_TLBI_NH_ASID; cmd.tlbi.asid = smmu_domain->s1_cfg.cd0->tag; cmd.tlbi.vmid = 0; } else { @@ -1352,7 +1363,8 @@ static void arm_smmu_tlb_inv_range_nosync(unsigned long iova, size_t size, if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { if (unlikely(!smmu_domain->s1_cfg.cd0)) return; - cmd.opcode = CMDQ_OP_TLBI_NH_VA; + cmd.opcode = smmu->features & ARM_SMMU_FEAT_E2H ? + CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; cmd.tlbi.asid = smmu_domain->s1_cfg.cd0->tag; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; @@ -1422,7 +1434,8 @@ static void arm_smmu_tlb_inv_ssid(void *cookie, int ssid, struct arm_smmu_domain *smmu_domain = cookie; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_cmdq_ent cmd = { - .opcode = CMDQ_OP_TLBI_NH_ASID, + .opcode = smmu->features & ARM_SMMU_FEAT_E2H ? + CMDQ_OP_TLBI_EL2_ASID : CMDQ_OP_TLBI_NH_ASID, .tlbi.asid = entry->tag, }; @@ -2446,7 +2459,11 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) writel_relaxed(reg, smmu->base + ARM_SMMU_CR1); /* CR2 (random crap) */ - reg = CR2_PTM | CR2_RECINVSID | CR2_E2H; + reg = CR2_PTM | CR2_RECINVSID; + + if (smmu->features & ARM_SMMU_FEAT_E2H) + reg |= CR2_E2H; + writel_relaxed(reg, smmu->base + ARM_SMMU_CR2); /* Stream table */ @@ -2594,8 +2611,11 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) if (reg & IDR0_MSI) smmu->features |= ARM_SMMU_FEAT_MSI; - if (reg & IDR0_HYP) + if (reg & IDR0_HYP) { smmu->features |= ARM_SMMU_FEAT_HYP; + if (cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN)) + smmu->features |= ARM_SMMU_FEAT_E2H; + } /* * The coherency feature as set by FW is used in preference to the ID