From patchwork Wed Feb 1 12:53:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 13124394 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F356CC05027 for ; Wed, 1 Feb 2023 14:10:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=p3dLEx5wa2qSIMXxoSMGswWObri1sibKkYFjvtTt+Ms=; b=4MQk6Nj74DuEpL aykDdPyHwFJDvDXuW/aevXtur/O1d8sd+eIzr9+/shoIv7QnzE/Tk8Q+JOsJmqRo9MUgOxaI1635A cEGhTsIaqv+hmzDl4eGhBEhFbRFuRvabLkh7Yu+wqd7FTKHKiL0ibXzaRaU7Bh3T3KCg4hYNO8FYG X5lZ/TOUeg5YA7aMuHweudJPikcw9tiChhvetV8nWGgCfbXUYE6U3FxBOwlRtWGpUgjZoxgXqvxtH El9/gQYd2K1v3nGVMq494shF+rqFoTkXNuiQFoVFwSCpf65GrNzoSDN7Y08hrFej+wXTstLLCoGbs XpHGC5QZC5Z9Gxrn4nwQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pNDoC-00CEql-Ok; Wed, 01 Feb 2023 14:09:58 +0000 Received: from mail-wr1-f41.google.com ([209.85.221.41]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pNCiE-00BnOl-4g for linux-arm-kernel@lists.infradead.org; Wed, 01 Feb 2023 12:59:43 +0000 Received: by mail-wr1-f41.google.com with SMTP id h12so17213097wrv.10 for ; Wed, 01 Feb 2023 04:59:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j6ny/GGJ/xSBkLuV4RcE+GC+n9HdkCFGxovAOHzkf4Y=; b=mXkKJ8GVi2o9hKXvO38ATeKbu/kQNts062SXg30ZfOt60zaBNySaqz9YhPtDU9dW6w AW1yaIveh1cDIXzFT25hr8nkVPVGEJmmUcziyjH8QnV1Ty9HQQpGZek4Wgq/YqwpEhbD eG7u9b+IfLjB3tlihWtJZMF5NBxRwCkUdrAFQF0NZd1cy/kLh0GceoWrOqtWiUNYgCmm uc1674z0i6LT9WQEBxR/UHtAzYZyAqYWjaF022HvtQuOg8HgJ/XwQ0P8xqPQa7NnEaGm OUhefUImf2fERRYo6Z/sVaX3gmmesQ+3UVBrGQjTmYoGKQt7YI3SpD3JTr/+XnxNRbe5 /JNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j6ny/GGJ/xSBkLuV4RcE+GC+n9HdkCFGxovAOHzkf4Y=; b=7aDyKV57w8zcgSaJyoYCGUOAgttjI4t30B1ysODr8wZJdZszWJORNw0CUDDlQS8f45 DjISMz8J3LQyVzBsKMMD9CJMny1+OdTTz/YwpDSjtwy23OFAvI/KCXUHY68Ddyxq6tnl vRZN9gB8yE863uqWL2o7gkLCyymMprHUkPUvvMurcEI5tot9QLc5QWzGGITIdpuRmacf bou/XDwZNMuJLaVRLrgS9KYKZzhAq0vZZyDkEwrwyhV74EBTgn494LhNrJWHEMhRIEZs +J+9IYU6ZEDFWcEUdpMolj0GaPGz2J6+/c5tJgneRafyf5vg+aQ7NkHLC4eMhLiVr1cX gL3A== X-Gm-Message-State: AO0yUKUOFsZdhjCqz3XStsGG/nmS5io1WWUbIyEtwpOPioTkt8eYChPR 8oXIiplUBF3uzyOutjkKMZxAEg== X-Google-Smtp-Source: AK7set/uX5ReOHShIxYjKCrFXO8QhEm90M+SO3bJfnKAiOUk5aZz9Jj3/ob0BmcC5+s7c5TD4BjyEg== X-Received: by 2002:a5d:6d8e:0:b0:2bf:da39:5628 with SMTP id l14-20020a5d6d8e000000b002bfda395628mr2693278wrs.41.1675256378313; Wed, 01 Feb 2023 04:59:38 -0800 (PST) Received: from localhost.localdomain (054592b0.skybroadband.com. [5.69.146.176]) by smtp.gmail.com with ESMTPSA id m15-20020a056000024f00b002bfae16ee2fsm17972811wrz.111.2023.02.01.04.59.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 04:59:37 -0800 (PST) From: Jean-Philippe Brucker To: maz@kernel.org, catalin.marinas@arm.com, will@kernel.org, joro@8bytes.org Cc: robin.murphy@arm.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, smostafa@google.com, dbrazdil@google.com, ryan.roberts@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev, Jean-Philippe Brucker Subject: [RFC PATCH 24/45] KVM: arm64: smmu-v3: Setup stream table Date: Wed, 1 Feb 2023 12:53:08 +0000 Message-Id: <20230201125328.2186498-25-jean-philippe@linaro.org> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230201125328.2186498-1-jean-philippe@linaro.org> References: <20230201125328.2186498-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230201_045942_219865_DB6004DF X-CRM114-Status: GOOD ( 18.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Map the stream table allocated by the host into the hypervisor address space. When the host mappings are finalized, the table is unmapped from the host. Depending on the host configuration, the stream table may have one or two levels. Populate the level-2 stream table lazily. Signed-off-by: Jean-Philippe Brucker --- include/kvm/arm_smmu_v3.h | 4 + arch/arm64/kvm/hyp/nvhe/iommu/arm-smmu-v3.c | 133 +++++++++++++++++++- 2 files changed, 136 insertions(+), 1 deletion(-) diff --git a/include/kvm/arm_smmu_v3.h b/include/kvm/arm_smmu_v3.h index da36737bc1e0..fc67a3bf5709 100644 --- a/include/kvm/arm_smmu_v3.h +++ b/include/kvm/arm_smmu_v3.h @@ -24,6 +24,10 @@ struct hyp_arm_smmu_v3_device { u32 cmdq_prod; u64 *cmdq_base; size_t cmdq_log2size; + u64 *strtab_base; + size_t strtab_num_entries; + size_t strtab_num_l1_entries; + u8 strtab_split; }; extern size_t kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count); diff --git a/arch/arm64/kvm/hyp/nvhe/iommu/arm-smmu-v3.c b/arch/arm64/kvm/hyp/nvhe/iommu/arm-smmu-v3.c index 36ee5724f36f..021bebebd40c 100644 --- a/arch/arm64/kvm/hyp/nvhe/iommu/arm-smmu-v3.c +++ b/arch/arm64/kvm/hyp/nvhe/iommu/arm-smmu-v3.c @@ -141,7 +141,6 @@ static int smmu_sync_cmd(struct hyp_arm_smmu_v3_device *smmu) return smmu_wait_event(smmu, smmu_cmdq_empty(smmu)); } -__maybe_unused static int smmu_send_cmd(struct hyp_arm_smmu_v3_device *smmu, struct arm_smmu_cmdq_ent *cmd) { @@ -153,6 +152,82 @@ static int smmu_send_cmd(struct hyp_arm_smmu_v3_device *smmu, return smmu_sync_cmd(smmu); } +__maybe_unused +static int smmu_sync_ste(struct hyp_arm_smmu_v3_device *smmu, u32 sid) +{ + struct arm_smmu_cmdq_ent cmd = { + .opcode = CMDQ_OP_CFGI_STE, + .cfgi.sid = sid, + .cfgi.leaf = true, + }; + + return smmu_send_cmd(smmu, &cmd); +} + +static int smmu_alloc_l2_strtab(struct hyp_arm_smmu_v3_device *smmu, u32 idx) +{ + void *table; + u64 l2ptr, span; + + /* Leaf tables must be page-sized */ + if (smmu->strtab_split + ilog2(STRTAB_STE_DWORDS) + 3 != PAGE_SHIFT) + return -EINVAL; + + span = smmu->strtab_split + 1; + if (WARN_ON(span < 1 || span > 11)) + return -EINVAL; + + table = kvm_iommu_donate_page(); + if (!table) + return -ENOMEM; + + l2ptr = hyp_virt_to_phys(table); + if (l2ptr & (~STRTAB_L1_DESC_L2PTR_MASK | ~PAGE_MASK)) + return -EINVAL; + + /* Ensure the empty stream table is visible before the descriptor write */ + wmb(); + + if ((cmpxchg64_relaxed(&smmu->strtab_base[idx], 0, l2ptr | span) != 0)) + kvm_iommu_reclaim_page(table); + + return 0; +} + +__maybe_unused +static u64 *smmu_get_ste_ptr(struct hyp_arm_smmu_v3_device *smmu, u32 sid) +{ + u32 idx; + int ret; + u64 l1std, span, *base; + + if (sid >= smmu->strtab_num_entries) + return NULL; + sid = array_index_nospec(sid, smmu->strtab_num_entries); + + if (!smmu->strtab_split) + return smmu->strtab_base + sid * STRTAB_STE_DWORDS; + + idx = sid >> smmu->strtab_split; + l1std = smmu->strtab_base[idx]; + if (!l1std) { + ret = smmu_alloc_l2_strtab(smmu, idx); + if (ret) + return NULL; + l1std = smmu->strtab_base[idx]; + if (WARN_ON(!l1std)) + return NULL; + } + + span = l1std & STRTAB_L1_DESC_SPAN; + idx = sid & ((1 << smmu->strtab_split) - 1); + if (!span || idx >= (1 << (span - 1))) + return NULL; + + base = hyp_phys_to_virt(l1std & STRTAB_L1_DESC_L2PTR_MASK); + return base + idx * STRTAB_STE_DWORDS; +} + static int smmu_init_registers(struct hyp_arm_smmu_v3_device *smmu) { u64 val, old; @@ -221,6 +296,58 @@ static int smmu_init_cmdq(struct hyp_arm_smmu_v3_device *smmu) return 0; } +static int smmu_init_strtab(struct hyp_arm_smmu_v3_device *smmu) +{ + u64 strtab_base; + size_t strtab_size; + u32 strtab_cfg, fmt; + int split, log2size; + + strtab_base = readq_relaxed(smmu->base + ARM_SMMU_STRTAB_BASE); + if (strtab_base & ~(STRTAB_BASE_ADDR_MASK | STRTAB_BASE_RA)) + return -EINVAL; + + strtab_cfg = readl_relaxed(smmu->base + ARM_SMMU_STRTAB_BASE_CFG); + if (strtab_cfg & ~(STRTAB_BASE_CFG_FMT | STRTAB_BASE_CFG_SPLIT | + STRTAB_BASE_CFG_LOG2SIZE)) + return -EINVAL; + + fmt = FIELD_GET(STRTAB_BASE_CFG_FMT, strtab_cfg); + split = FIELD_GET(STRTAB_BASE_CFG_SPLIT, strtab_cfg); + log2size = FIELD_GET(STRTAB_BASE_CFG_LOG2SIZE, strtab_cfg); + + smmu->strtab_split = split; + smmu->strtab_num_entries = 1 << log2size; + + switch (fmt) { + case STRTAB_BASE_CFG_FMT_LINEAR: + if (split) + return -EINVAL; + smmu->strtab_num_l1_entries = smmu->strtab_num_entries; + strtab_size = smmu->strtab_num_l1_entries * + STRTAB_STE_DWORDS * 8; + break; + case STRTAB_BASE_CFG_FMT_2LVL: + if (split != 6 && split != 8 && split != 10) + return -EINVAL; + smmu->strtab_num_l1_entries = 1 << max(0, log2size - split); + strtab_size = smmu->strtab_num_l1_entries * + STRTAB_L1_DESC_DWORDS * 8; + break; + default: + return -EINVAL; + } + + strtab_base &= STRTAB_BASE_ADDR_MASK; + smmu->strtab_base = smmu_take_pages(strtab_base, strtab_size); + if (!smmu->strtab_base) + return -EINVAL; + + /* Disable all STEs */ + memset(smmu->strtab_base, 0, strtab_size); + return 0; +} + static int smmu_init_device(struct hyp_arm_smmu_v3_device *smmu) { int ret; @@ -241,6 +368,10 @@ static int smmu_init_device(struct hyp_arm_smmu_v3_device *smmu) if (ret) return ret; + ret = smmu_init_strtab(smmu); + if (ret) + return ret; + return 0; }