From patchwork Thu Jun 11 12:54:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 11600069 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D22A7913 for ; Thu, 11 Jun 2020 13:02:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B1CE72078D for ; Thu, 11 Jun 2020 13:02:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1CE72078D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:37498 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jjMqf-000252-SU for patchwork-qemu-devel@patchwork.kernel.org; Thu, 11 Jun 2020 09:02:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46458) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjMd3-0002pW-00 for qemu-devel@nongnu.org; Thu, 11 Jun 2020 08:48:21 -0400 Received: from mga06.intel.com ([134.134.136.31]:46975) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjMcy-0007ze-Hn for qemu-devel@nongnu.org; Thu, 11 Jun 2020 08:48:20 -0400 IronPort-SDR: ToimgzC8gN6X+RRILTF06gRnYViOAQ5IWoJDqMWu7iPOdCkMM2u0do9oIVEephLh/75BLlvExx 28JfggGamzSw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jun 2020 05:48:01 -0700 IronPort-SDR: 4qfAmefgLYNTnN/vWx+pZwUgGdhdApYcHReu/f4iKic1P5cIMvE/vSomhjx9zYqNHp+CtJTF1J hyAeICU6WHnw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,499,1583222400"; d="scan'208";a="447911266" Received: from jacob-builder.jf.intel.com ([10.7.199.155]) by orsmga005.jf.intel.com with ESMTP; 11 Jun 2020 05:48:00 -0700 From: Liu Yi L To: qemu-devel@nongnu.org, alex.williamson@redhat.com, peterx@redhat.com Subject: [RFC v6 17/25] intel_iommu: sync IOMMU nesting cap info for assigned devices Date: Thu, 11 Jun 2020 05:54:16 -0700 Message-Id: <1591880064-30638-18-git-send-email-yi.l.liu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1591880064-30638-1-git-send-email-yi.l.liu@intel.com> References: <1591880064-30638-1-git-send-email-yi.l.liu@intel.com> Received-SPF: pass client-ip=134.134.136.31; envelope-from=yi.l.liu@intel.com; helo=mga06.intel.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/11 08:48:01 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe@linaro.org, kevin.tian@intel.com, yi.l.liu@intel.com, Yi Sun , kvm@vger.kernel.org, mst@redhat.com, jun.j.tian@intel.com, eric.auger@redhat.com, yi.y.sun@intel.com, Jacob Pan , pbonzini@redhat.com, hao.wu@intel.com, Richard Henderson , david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" For assigned devices, Intel vIOMMU which wants to build DMA protection based on physical IOMMU nesting paging should check the IOMMU nesting support in host side. The host will return IOMMU nesting cap info to user-space (e.g. VFIO returns IOMMU nesting cap info for nesting type IOMMU). vIOMMU needs to check: a) IOMMU model b) 1st-level page table supports c) address width d) pasid support This patch syncs the IOMMU nesting cap info when PCIe device (VFIO case) sets HostIOMMUContext to vIOMMU. If the host IOMMU nesting support is not compatible, vIOMMU should return failure to PCIe device. Cc: Kevin Tian Cc: Jacob Pan Cc: Peter Xu Cc: Yi Sun Cc: Paolo Bonzini Cc: Richard Henderson Signed-off-by: Liu Yi L --- hw/i386/intel_iommu.c | 105 +++++++++++++++++++++++++++++++++++++++++ hw/i386/intel_iommu_internal.h | 5 ++ include/hw/i386/intel_iommu.h | 4 ++ 3 files changed, 114 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 805801c..4a794f9 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -4099,6 +4099,82 @@ static int vtd_dev_get_iommu_attr(PCIBus *bus, void *opaque, int32_t devfn, return ret; } + +static bool vtd_check_nesting_info(IntelIOMMUState *s, + struct iommu_nesting_info_vtd *vtd) +{ + return !((s->aw_bits != vtd->addr_width) || + ((s->host_cap & vtd->cap_mask) != + (vtd->cap_reg & vtd->cap_mask)) || + ((s->host_ecap & vtd->ecap_mask) != + (vtd->ecap_reg & vtd->ecap_mask)) || + (VTD_GET_PSS(s->host_ecap) != (vtd->pasid_bits - 1))); +} + +/* Caller should hold iommu lock. */ +static bool vtd_sync_nesting_info(IntelIOMMUState *s, + struct iommu_nesting_info_vtd *vtd) +{ + uint64_t cap, ecap; + + if (s->cap_finalized) { + return vtd_check_nesting_info(s, vtd); + } + + if (s->aw_bits > vtd->addr_width) { + error_report("User aw-bits: %u > host address width: %u", + s->aw_bits, vtd->addr_width); + return false; + } + + cap = s->host_cap & vtd->cap_reg & vtd->cap_mask; + s->host_cap &= ~vtd->cap_mask; + s->host_cap |= cap; + + ecap = s->host_ecap & vtd->ecap_reg & vtd->ecap_mask; + s->host_ecap &= ~vtd->ecap_mask; + s->host_ecap |= ecap; + + if ((VTD_ECAP_PASID & s->host_ecap) && vtd->pasid_bits && + (VTD_GET_PSS(s->host_ecap) > (vtd->pasid_bits - 1))) { + s->host_ecap &= ~VTD_ECAP_PSS_MASK; + s->host_ecap |= VTD_ECAP_PSS(vtd->pasid_bits - 1); + } + return true; +} + +/* + * virtual VT-d which wants nested needs to check the host IOMMU + * nesting cap info behind the assigned devices. Thus that vIOMMU + * could bind guest page table to host. + */ +static bool vtd_check_iommu_ctx(IntelIOMMUState *s, + HostIOMMUContext *iommu_ctx) +{ + struct iommu_nesting_info *info = iommu_ctx->info; + struct iommu_nesting_info_vtd *vtd; + uint32_t minsz, size; + + if (IOMMU_PASID_FORMAT_INTEL_VTD != info->format) { + error_report("Format is not compatible for nesting!!!"); + return false; + } + + size = sizeof(*vtd); + minsz = endof(struct iommu_nesting_info, flags); + if (size > (info->size - minsz)) { + /* + * QEMU may have been using new linux-headers/iommu.h than + * kernel supports, hence fail it. + */ + error_report("IOMMU nesting cap is not compatible!!!"); + return false; + } + + vtd = (struct iommu_nesting_info_vtd *) &info->data; + return vtd_sync_nesting_info(s, vtd); +} + static int vtd_dev_set_iommu_context(PCIBus *bus, void *opaque, int devfn, HostIOMMUContext *iommu_ctx) @@ -4113,6 +4189,11 @@ static int vtd_dev_set_iommu_context(PCIBus *bus, void *opaque, vtd_iommu_lock(s); + if (!vtd_check_iommu_ctx(s, iommu_ctx)) { + vtd_iommu_unlock(s); + return -ENOENT; + } + vtd_dev_icx = vtd_bus->dev_icx[devfn]; assert(!vtd_dev_icx); @@ -4368,6 +4449,14 @@ static void vtd_init(IntelIOMMUState *s) s->ecap |= VTD_ECAP_SMTS | VTD_ECAP_SRS | VTD_ECAP_SLTS; } + if (!s->cap_finalized) { + s->host_cap = s->cap; + s->host_ecap = s->ecap; + } else { + s->cap = s->host_cap; + s->ecap = s->host_ecap; + } + vtd_reset_caches(s); /* Define registers with default values and bit semantics */ @@ -4501,6 +4590,12 @@ static bool vtd_decide_config(IntelIOMMUState *s, Error **errp) return true; } +static void vtd_refresh_capability_reg(IntelIOMMUState *s) +{ + vtd_set_quad(s, DMAR_CAP_REG, s->cap); + vtd_set_quad(s, DMAR_ECAP_REG, s->ecap); +} + static int vtd_machine_done_notify_one(Object *child, void *unused) { IntelIOMMUState *iommu = INTEL_IOMMU_DEVICE(x86_iommu_get_default()); @@ -4514,6 +4609,15 @@ static int vtd_machine_done_notify_one(Object *child, void *unused) vtd_panic_require_caching_mode(); } + vtd_iommu_lock(iommu); + iommu->cap = iommu->host_cap & iommu->cap; + iommu->ecap = iommu->host_ecap & iommu->ecap; + if (!iommu->cap_finalized) { + iommu->cap_finalized = true; + } + + vtd_refresh_capability_reg(iommu); + vtd_iommu_unlock(iommu); return 0; } @@ -4545,6 +4649,7 @@ static void vtd_realize(DeviceState *dev, Error **errp) QLIST_INIT(&s->vtd_as_with_notifiers); QLIST_INIT(&s->vtd_dev_icx_list); qemu_mutex_init(&s->iommu_lock); + s->cap_finalized = false; memset(s->vtd_as_by_bus_num, 0, sizeof(s->vtd_as_by_bus_num)); memory_region_init_io(&s->csrmem, OBJECT(s), &vtd_mem_ops, s, "intel_iommu", DMAR_REG_SIZE); diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 51c0833..dbadd66 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -196,9 +196,14 @@ #define VTD_ECAP_PT (1ULL << 6) #define VTD_ECAP_MHMV (15ULL << 20) #define VTD_ECAP_SRS (1ULL << 31) +#define VTD_ECAP_PSS(val) (((val) & 0x1fULL) << 35) +#define VTD_ECAP_PASID (1ULL << 40) #define VTD_ECAP_SMTS (1ULL << 43) #define VTD_ECAP_SLTS (1ULL << 46) +#define VTD_GET_PSS(val) (((val) >> 35) & 0x1f) +#define VTD_ECAP_PSS_MASK (0x1fULL << 35) + /* CAP_REG */ /* (offset >> 4) << 24 */ #define VTD_CAP_FRO (DMAR_FRCD_REG_OFFSET << 20) diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index 626c1cd..1aab882 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -284,6 +284,9 @@ struct IntelIOMMUState { uint64_t cap; /* The value of capability reg */ uint64_t ecap; /* The value of extended capability reg */ + uint64_t host_cap; /* The value of host capability reg */ + uint64_t host_ecap; /* The value of host ext-capability reg */ + uint32_t context_cache_gen; /* Should be in [1,MAX] */ GHashTable *iotlb; /* IOTLB */ @@ -310,6 +313,7 @@ struct IntelIOMMUState { uint64_t vccap; /* The value of vcmd capability reg */ uint64_t vcrsp; /* Current value of VCMD RSP REG */ + bool cap_finalized; /* Whether VTD capability finalized */ /* * iommu_lock protects below: * - per-IOMMU IOTLB caches