From patchwork Thu Oct 3 20:36:31 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 2985871 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 832E49F288 for ; Thu, 3 Oct 2013 20:42:49 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 872C0202FE for ; Thu, 3 Oct 2013 20:42:48 +0000 (UTC) Received: from casper.infradead.org (casper.infradead.org [85.118.1.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7DCBE20303 for ; Thu, 3 Oct 2013 20:42:47 +0000 (UTC) Received: from merlin.infradead.org ([2001:4978:20e::2]) by casper.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1VRpig-0000Ho-AT; Thu, 03 Oct 2013 20:41:58 +0000 Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1VRpdn-0008De-EO; Thu, 03 Oct 2013 20:36:55 +0000 Received: from mail-pd0-f169.google.com ([209.85.192.169]) by merlin.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1VRpde-0008BV-Ve for linux-arm-kernel@lists.infradead.org; Thu, 03 Oct 2013 20:36:48 +0000 Received: by mail-pd0-f169.google.com with SMTP id r10so3014572pdi.0 for ; Thu, 03 Oct 2013 13:36:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=hQUYzpah17sbZMm/JEaFWDUyu+olsIZ0a4/dXB+X5OA=; b=lEJmOMtEvtmr5X7xnNbsEyy0G2yfWQPqfl+2EhrdgoTFSamgiFiNpxbGa/J3fDXq77 Z960e9tgAveBP8YqyVBq1fgQgjpPoXjbBjihJ8mPAI4/CndQ4sukhO5frDh35CSIIAP1 nyUokA8mPsM/fCl63SLSPNABKtiMTWp/iMe+buZ8yR/U3CUDljCR0db5Y9ABdc//4zyC pDrdXxcDQ/mFnbEuiiyfLKnphuNH7gRyj5PH+qFhy5ZEdq60rzUD8VECWuPIOwPq1Asz v6MWGUdlVf/V6Cr4IxcAKLJHtW/igM97kwgwfgfWIcLQx4QfufVFHKdEiv2BRzn5ZM1I C0Bg== X-Gm-Message-State: ALoCoQlnWxfssa/EuFVuTxzfO1wo2savQtG47eCQHg4oGSrxcKYlzafCed11mjdE+bRXrBh5HlyJ X-Received: by 10.68.169.161 with SMTP id af1mr10598995pbc.22.1380832583375; Thu, 03 Oct 2013 13:36:23 -0700 (PDT) Received: from localhost.localdomain (c-67-169-181-221.hsd1.ca.comcast.net. [67.169.181.221]) by mx.google.com with ESMTPSA id ef10sm12856401pac.1.1969.12.31.16.00.00 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 03 Oct 2013 13:36:22 -0700 (PDT) From: Christoffer Dall To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v2 2/2] KVM: ARM: Transparent huge page (THP) support Date: Thu, 3 Oct 2013 13:36:31 -0700 Message-Id: <1380832591-4789-3-git-send-email-christoffer.dall@linaro.org> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1380832591-4789-1-git-send-email-christoffer.dall@linaro.org> References: <1380832591-4789-1-git-send-email-christoffer.dall@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20131003_163647_144999_B8ABFA9E X-CRM114-Status: GOOD ( 15.77 ) X-Spam-Score: -1.9 (-) Cc: Christoffer Dall , linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Support transparent huge pages in KVM/ARM and KVM/ARM64. The transparent_hugepage_adjust is not very pretty, but this is also how it's solved on x86 and seems to be simply an artifact on how THPs behave. This should eventually be shared across architectures if possible, but that can always be changed down the road. Signed-off-by: Christoffer Dall --- Changelog[v2]: - THP handling moved into separate patch. - Minor changes and clarified comment in transparent_hugepage_adjust from Marc Z's review. --- arch/arm/kvm/mmu.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index cab031b..0a856a0 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -42,7 +42,7 @@ static unsigned long hyp_idmap_start; static unsigned long hyp_idmap_end; static phys_addr_t hyp_idmap_vector; -#define kvm_pmd_huge(_x) (pmd_huge(_x)) +#define kvm_pmd_huge(_x) (pmd_huge(_x) || pmd_trans_huge(_x)) static void kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa) { @@ -576,6 +576,47 @@ out: return ret; } +static bool transparent_hugepage_adjust(pfn_t *pfnp, phys_addr_t *ipap) +{ + pfn_t pfn = *pfnp; + gfn_t gfn = *ipap >> PAGE_SHIFT; + + if (PageTransCompound(pfn_to_page(pfn))) { + unsigned long mask; + /* + * The address we faulted on is backed by a transparent huge + * page. However, because we map the compound huge page and + * not the individual tail page, we need to transfer the + * refcount to the head page. We have to be careful that the + * THP doesn't start to split while we are adjusting the + * refcounts. + * + * We are sure this doesn't happen, because mmu_notifier_retry + * was succesful and we are holding the mmu_lock, so if this + * THP is trying to split, it will be blocked in the mmu + * notifier before touching any of the pages, specifically + * before being able to call __split_huge_page_refcount(). + * + * We can therefore safely transfer the refcount from PG_tail + * to PG_head and switch the pfn from a tail page to the head + * page accordingly. + */ + mask = PTRS_PER_PMD - 1; + VM_BUG_ON((gfn & mask) != (pfn & mask)); + if (pfn & mask) { + *ipap &= PMD_MASK; + kvm_release_pfn_clean(pfn); + pfn &= ~mask; + kvm_get_pfn(pfn); + *pfnp = pfn; + } + + return true; + } + + return false; +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_memory_slot *memslot, unsigned long fault_status) @@ -632,6 +673,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, spin_lock(&kvm->mmu_lock); if (mmu_notifier_retry(kvm, mmu_seq)) goto out_unlock; + if (!hugetlb && !force_pte) + hugetlb = transparent_hugepage_adjust(&pfn, &fault_ipa); if (hugetlb) { pmd_t new_pmd = pfn_pmd(pfn, PAGE_S2);