From patchwork Thu Oct 19 08:08:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 10016229 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B98D7600CC for ; Thu, 19 Oct 2017 09:24:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B92A9287B3 for ; Thu, 19 Oct 2017 09:24:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB1A5287EE; Thu, 19 Oct 2017 09:24:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 08A50286C1 for ; Thu, 19 Oct 2017 09:24:37 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e571u-0001u5-TR; Thu, 19 Oct 2017 09:22:18 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e55rV-0004AB-RO for xen-devel@lists.xenproject.org; Thu, 19 Oct 2017 08:07:29 +0000 Received: from [85.158.139.211] by server-2.bemta-5.messagelabs.com id 58/2B-23269-14D58E95; Thu, 19 Oct 2017 08:07:29 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrDIsWRWlGSWpSXmKPExsVywNxEW9ch9kW kwemN/Bbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8bjqWsZC14HVmy5/pSlgfGZeRcjJ4eQQKVE 07tN7CC2hACvxJFlM1ghbH+Jh6+fs0HUFEncuLSCqYuRC8heyChxtfcgWAObgLZE8433LCC2i ICSxL1Vk8GKmAV6mSQW3PgKlhAWCJL4cngxM4jNIqAq0famDaiZg4NXIEGiZ6UZxDI5iZvnOs FKeAUEJU7OfALWyilgJ3H32RaoI2wlnn37yQLSyiygLrF+nhBImFlAXqJ562zmCYyCs5B0z0K omoWkagEj8ypGjeLUorLUIl1DQ72kosz0jJLcxMwcXUMDU73c1OLixPTUnMSkYr3k/NxNjMCQ ZQCCHYwr250PMUpyMCmJ8soGvogU4kvKT6nMSCzOiC8qzUktPsQow8GhJMFbEAOUEyxKTU+tS MvMAUYPTFqCg0dJhNcTJM1bXJCYW5yZDpE6xajL0XHz7h8mIZa8/LxUKXFeMZAiAZCijNI8uB GwSL7EKCslzMsIdJQQT0FqUW5mCar8K0ZxDkYlYd4AkCk8mXklcJteAR3BBHQEuz3YESWJCCm pBsZevnn3cjlfd0960vLz/+MX39vNLyQ4Xp78qvF6n9qL5wxT2uxvpD6zM+g5kbZk999jsZ+n Cmpd8TogeJFP+kL+lcakHWwbS8x0F/KKxW86w7Ig+s6ZW22hnR31kwvZmxU62tVmfp+o8jYz7 9O8CFsOgVv/nT96Httw6eYitWPncmcvkgzQYT+qxFKckWioxVxUnAgAR+/QFN8CAAA= X-Env-Sender: yi.z.zhang@linux.intel.com X-Msg-Ref: server-4.tower-206.messagelabs.com!1508400446!107634910!1 X-Originating-IP: [192.55.52.43] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 20131 invoked from network); 19 Oct 2017 08:07:27 -0000 Received: from mga05.intel.com (HELO mga05.intel.com) (192.55.52.43) by server-4.tower-206.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 19 Oct 2017 08:07:27 -0000 Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP; 19 Oct 2017 01:07:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.43,400,1503385200"; d="scan'208";a="911443879" Received: from linux.intel.com ([10.54.29.200]) by FMSMGA003.fm.intel.com with ESMTP; 19 Oct 2017 01:07:24 -0700 Received: from dazhang1-ssd.sh.intel.com (unknown [10.239.48.55]) by linux.intel.com (Postfix) with ESMTP id 21BE9580354; Thu, 19 Oct 2017 01:07:21 -0700 (PDT) From: Zhang Yi To: xen-devel@lists.xenproject.org Date: Thu, 19 Oct 2017 16:08:00 +0800 Message-Id: <4347990d8bfaea22b8972db1003225bee5443b00.1508397860.git.yi.z.zhang@linux.intel.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 In-Reply-To: References: X-Mailman-Approved-At: Thu, 19 Oct 2017 09:22:18 +0000 Cc: kevin.tian@intel.com, tamas@tklengyel.com, wei.liu2@citrix.com, jun.nakajima@intel.com, rcojocaru@bitdefender.com, george.dunlap@eu.citrix.com, andrew.cooper3@citrix.com, ian.jackson@eu.citrix.com, Zhang Yi Z , jbeulich@suse.com Subject: [Xen-devel] [PATCH RFC 01/14] xen: vmx: Added EPT based Subpage Write Protection Doc. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Yi Z Signed-off-by: Zhang Yi Z --- docs/txt/misc/spp_xen.txt | 259 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 259 insertions(+) create mode 100644 docs/txt/misc/spp_xen.txt diff --git a/docs/txt/misc/spp_xen.txt b/docs/txt/misc/spp_xen.txt new file mode 100644 index 0000000..d84eca2 --- /dev/null +++ b/docs/txt/misc/spp_xen.txt @@ -0,0 +1,259 @@ +DRAFT: EPT-Based Sub-Page Protection (SPP) Design Doc for Xen +============================================================= + +1. Overview + +EPT-based Sub-Page Protection (SPP) capability to allow Virtual Machine +Monitors to specify write-protection for guest physical memory at a +sub-page (128 byte) granularity. When this capability is utilized, the +CPU enforces write-access permissions for sub-page regions of 4K pages +as specified by the VMM. + +2. Operation of SPP + +Sub-Page Protection Table (SPPT) is introduced to manage sub-page +write-access. + +SPPT is active when the "sub-page write protection" VM-execution control +is 1. SPPT looks up the guest physical addresses to derive a 64 bit +"sub-page permission" value containing sub-page write permissions. The +lookup from guest-physical addresses to the sub-page region permissions +is determined by a set of SPPT paging structures. + +When the "sub-page write protection" VM-execution control is 1, the SPPT +is used to lookup write permission bits for the 128 byte sub-page regions +containing in the 4KB guest physical page. EPT specifies the 4KB page +level privileges that software is allowed when accessing the guest +physical address, whereas SPPT defines the write permissions for software +at the 128 byte granularity regions within a 4KB page. Write accesses +prevented due to sub-page permissions looked up via SPPT are reported as +EPT violation VM exits. Similar to EPT, a logical processor uses SPPT to +lookup sub-page region write permissions for guest-physical addresses +only when those addresses are used to access memory. +______________________________________________________________________________ + +How SPP hardware works: +_______________________________________________________________________________ + +Guest write access --> GPA --> Walk EPT --> EPT leaf entry -┐ +┌-----------------------------------------------------------┘ +└-> if VMexec_control.spp && ept_leaf_entry.spp_bit (bit 61) + | + └-> --> EPT legacy behavior + | + | + └-> --> if ept_leaf_entry.writable + | + └-> --> Ignore SPP + | + └-> --> GPA --> Walk SPP 4-level table--┐ + | +┌------------<----------get-the-SPPT-point-from-VMCS-filed-----<------┘ +| +Walk SPP L4E table +| +└┐--> entry misconfiguration ------------>----------┐<----------------┐ + | | | +else | | + | | | + | ┌------------------SPP VMexit<-----------------┘ | + | | | + | └-> exit_qualification & sppt_misconfig --> sppt misconfig | + | | | + | └-> exit_qualification & sppt_miss --> sppt miss | + └--┐ | + | | +walk SPPT L3E--┐--> if-entry-misconfiguration------------>------------┘ + | | + else | + | | + | | + walk SPPT L2E --┐--> if-entry-misconfiguration-------->-------┘ + | | + else | + | | + | | + walk SPPT L1E --┐-> if-entry-misconfiguration--->----┘ + | + else + | + └-> if sub-page writable + └-> allow, write access + └-> disallow, EPT violation +______________________________________________________________________________ + +3. Interfaces + +* Feature enabling + +Add "spp_enable=1" to Xen Command line to enable SPP feature, default is off. + +* Get/Set sub-page write access permission + +New Xen HVM Hyper Call: + +`HVMOP_set_subpage`: +Set sub-pages write access bitmap corresponding to given gfn. + +```c +/* for Xen HVMOP_set_subpage */ +struct xen_hvm_subpage_t { + domid domid; + __u64 gfn; + __u32 access_map; /* sub-page write-access bitmap */ +}; + +#define HVMOP_set_subpage 26 + +xencall2(handle->xcall, __HYPERVISOR_hvm_op, HVMOP_set_subpage, + HYPERCALL_BUFFER_AS_ARG(arg)) +``` + +4. SPPT initialization + +* SPPT root page allocation + + SPPT is referenced via a 64-bit control field called "sub-page + protection table pointe" (SPPTP, encoding 0x2030) which contains a + 4K-align physical address. + + SPPT also has 4 level table as well as EPT. So, as EPT does, when Xen + loads mmu, we allocate a root page for SPPT L4 table. + +* EPT leaf entry SPP bit + + Set 0 to SPP bit to close SPP by default. + +5. Set/Get Sub-Page access bitmap for bunch of guest physical pages + +* To utilize SPP feature, system admin should Set a Sub-page access write via + SPP Xen hyper call `HVMOP_set_subpage`, which will prepared the flowing things. + + (1.Got the corresponding EPT leaf entry via the guest physical address. + (2.If it is a 4K page frame, flag the bit 61 to enable subpage protection on this page. + (3.Setup spp page structure, the page structure format is list following. + + Format of the SPPT L4E, L3E, L2E: + | Bit | Contents | + | :----- | :------------------------------------------------------------------------| + | 0 | Valid entry when set; indicates whether the entry is present | + | 11:1 | Reserved (0) | + | N-1:12 | Physical address of 4KB aligned SPPT LX-1 Table referenced by this entry | + | 51:N | Reserved (0) | + | 63:52 | Reserved (0) | + Note: N is the physical address width supported by the processor. X is the page level + + Format of the SPPT L1E: + | Bit | Contents | + | :---- | :---------------------------------------------------------------- | + | 0+2i | Write permission for i-th 128 byte sub-page region. | + | 1+2i | Reserved (0). | + Note: `0<=i<=31` + +* Sub-page write access bitmap setting pseudo-code: + +```c +static int p2m_set_subpage(struct domain *d, + struct xen_hvm_subpage_t *spp_info) +{ + gfn_t *gfns = spp_info->gfns; + u64 *access_map = spp_info->access_map; + + sanity_check(); + + /* SPP works when the page is unwritable */ + if (set_ept_leaf_level_unwritable(gfn) == success) + + if (p2m_set_spp_page_st(gfn) == success) + + success; + +} +``` + +User could get the subpage info via SPP Xen hyper call `HVMOP_get_subpage`. + +* Sub-page get subpage info pseudo-code: + +```c +static int p2m_get_subpage(struct domain *d + struct xen_hvm_subpage_t *spp_info) +{ + gfn_t *gfns = spp_info->gfns; + + sanity_check(gfn); + spp_info = p2m_get_spp_page_frame(gfn); +} +``` + +6. SPPT-induced vmexits + +* SPP VM exits + +Accesses using guest physical addresses may cause VM exits due to a SPPT +Misconfiguration or a SPPT Miss. + +A SPPT Misconfiguration vmexit occurs when, in the course of translating +a guest physical address, the logical proceesor encounters a leaf EPT +paging-structure entry mapping a 4KB page, with SPP enabled, during the +SPPT lookup, a SPPT paging-structure entry contains an unsupported +value. + +A SPPT Miss vmexit occurs during the SPPT lookup there is no SPPT +misconfiguration but any level of SPPT paging-structure entries are not +present. + +NOTE. SPPT misconfigurations and SPPT miss can occur only due to an +attempt to write memory with a guest physical address. + +* EPT violation vmexits due to SPPT + +EPT violations due to memory write accesses disallowed due to sub-page +protection permissions specified in the SPPT are reported via EPT +violation VM exits. + +7. SPPT-induced vmexits handling + +```c +#define EXIT_REASON_SPP 66 +vmx_vmexit_handler { + ... + [EXIT_REASON_SPP] = vmx_handle_spp, + ... +}; +``` +New exit qualification for SPPT-induced vmexits. + +| Bit | Contents | +| :---- | :---------------------------------------------------------------- | +| 10:0 | Reserved (0). | +| 11 | SPPT VM exit type. Set for SPPT Miss, cleared for SPPT Misconfig. | +| 12 | NMI unblocking due to IRET | +| 63:13 | Reserved (0) | + +* SPPT miss and misconfiguration + +SPP VMexit handler Pseudo-code: +```c +static int vmx_handle_spp() +{ + exit_qualification = vmcs_readl(EXIT_QUALIFICATION); + if (exit_qualification & SPP_EXIT_TYPE_BIT) { + /* SPPT Miss */ + } else { + /* SPPT Misconfig */ + WARN_ON(1); + } + return 0; +} +``` + +8. EPT violation vmexits due to SPPT + +While hardware walking the SPP page table, If the sub-page region write +permission bit is set, the write is allowed, else the write is disallowed +and results in an EPT violation. + +we need peek this case in EPT violation handler. + +