From patchwork Thu Nov 9 14:17:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= X-Patchwork-Id: 10051037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 19EC0602D7 for ; Thu, 9 Nov 2017 14:17:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0CEF12AD32 for ; Thu, 9 Nov 2017 14:17:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F325B2AD5A; Thu, 9 Nov 2017 14:17:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82DDA2AD32 for ; Thu, 9 Nov 2017 14:17:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754785AbdKIORp (ORCPT ); Thu, 9 Nov 2017 09:17:45 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55580 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753849AbdKIORn (ORCPT ); Thu, 9 Nov 2017 09:17:43 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4723D2CE909; Thu, 9 Nov 2017 14:17:43 +0000 (UTC) Received: from flask (unknown [10.43.2.80]) by smtp.corp.redhat.com (Postfix) with SMTP id E28A66375A; Thu, 9 Nov 2017 14:17:33 +0000 (UTC) Received: by flask (sSMTP sendmail emulation); Thu, 09 Nov 2017 15:17:33 +0100 Date: Thu, 9 Nov 2017 15:17:33 +0100 From: Radim =?utf-8?B?S3LEjW3DocWZ?= To: Eduardo Valentin Cc: Paolo Bonzini , Matt Wilson , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, Peter Zijlstra , Waiman Long , kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, "Jan H . Schoenherr" , Anthony Liguori Subject: Re: [PATCHv3 1/1] locking/qspinlock/x86: Avoid test-and-set when PV_DEDICATED is set Message-ID: <20171109141732.GA20859@flask> References: <1509999980-10828-1-git-send-email-eduval@amazon.com> <20171108173651.GA3664@flask> <20171109085547.GA5107@u40b0340c692b58f6553c.ant.amazon.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20171109085547.GA5107@u40b0340c692b58f6553c.ant.amazon.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 09 Nov 2017 14:17:43 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP 2017-11-09 00:55-0800, Eduardo Valentin: > Hello, > > On Wed, Nov 08, 2017 at 06:36:52PM +0100, Radim Krčmář wrote: > > 2017-11-06 12:26-0800, Eduardo Valentin: > > > Currently, the existing qspinlock implementation will fallback to > > > test-and-set if the hypervisor has not set the PV_UNHALT flag. > > > > > > This patch gives the opportunity to guest kernels to select > > > between test-and-set and the regular queueu fair lock implementation > > > based on the PV_DEDICATED KVM feature flag. When the PV_DEDICATED > > > flag is not set, the code will still fall back to test-and-set, > > > but when the PV_DEDICATED flag is set, the code will use > > > the regular queue spinlock implementation. > > > > > > With this patch, when in autoselect mode, the guest will > > > use the default spinlock implementation based on host feature > > > flags as follows: > > > > > > PV_DEDICATED = 1, PV_UNHALT = anything: default is qspinlock > > > PV_DEDICATED = 0, PV_UNHALT = 1: default is pvqspinlock > > > PV_DEDICATED = 0, PV_UNHALT = 0: default is tas > > > > > > Cc: Paolo Bonzini > > > Cc: "Radim Krčmář" > > > Cc: Jonathan Corbet > > > Cc: Thomas Gleixner > > > Cc: Ingo Molnar > > > Cc: "H. Peter Anvin" > > > Cc: x86@kernel.org > > > Cc: Peter Zijlstra > > > Cc: Waiman Long > > > Cc: kvm@vger.kernel.org > > > Cc: linux-doc@vger.kernel.org > > > Cc: linux-kernel@vger.kernel.org > > > Cc: Jan H. Schoenherr > > > Cc: Anthony Liguori > > > Suggested-by: Matt Wilson > > > Signed-off-by: Eduardo Valentin > > > --- > > > V3: > > > - When PV_DEDICATED is set (1), qspinlock is selected, > > > regardless of the value of PV_UNHAULT. Suggested by Paolo Bonzini. > > > - Refreshed on top of tip/master. > > > V2: > > > - rebase on top of tip/master > > > > > > Documentation/virtual/kvm/cpuid.txt | 6 ++++++ > > > arch/x86/include/asm/qspinlock.h | 4 ++++ > > > arch/x86/include/uapi/asm/kvm_para.h | 1 + > > > arch/x86/kernel/kvm.c | 2 ++ > > > 4 files changed, 13 insertions(+) > > > > > > diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt > > > index 3c65feb..117066a 100644 > > > --- a/Documentation/virtual/kvm/cpuid.txt > > > +++ b/Documentation/virtual/kvm/cpuid.txt > > > @@ -54,6 +54,12 @@ KVM_FEATURE_PV_UNHALT || 7 || guest checks this feature bit > > > || || before enabling paravirtualized > > > || || spinlock support. > > > ------------------------------------------------------------------------------ > > > +KVM_FEATURE_PV_DEDICATED || 8 || guest checks this feature bit > > > + || || to determine if they run on > > > + || || dedicated vCPUs, allowing opti- > > > + || || mizations such as usage of > > > + || || qspinlocks. > > > +------------------------------------------------------------------------------ > > > KVM_FEATURE_CLOCKSOURCE_STABLE_BIT || 24 || host will warn if no guest-side > > > || || per-cpu warps are expected in > > > || || kvmclock. > > > diff --git a/arch/x86/include/asm/qspinlock.h b/arch/x86/include/asm/qspinlock.h > > > index 5e16b5d..de42694 100644 > > > --- a/arch/x86/include/asm/qspinlock.h > > > +++ b/arch/x86/include/asm/qspinlock.h > > > @@ -3,6 +3,8 @@ > > > #define _ASM_X86_QSPINLOCK_H > > > > > > #include > > > +#include > > > + > > > #include > > > #include > > > #include > > > @@ -58,6 +60,8 @@ static inline bool virt_spin_lock(struct qspinlock *lock) > > > if (!static_branch_likely(&virt_spin_lock_key)) > > > return false; > > > > > > + if (kvm_para_has_feature(KVM_FEATURE_PV_DEDICATED)) > > > + return false; > > > > Hm, every spinlock slowpath calls cpuid, which causes a VM exit, so I > > wouldn't expect it to be faster than the existing implementations. > > (Using the static key would be better.) > > > > How does this patch perform compared to user-forced qspinlock and hybrid > > pvqspinlock? > > This patch should have same effect as user-forced qspinlock. This is what I'm doubting, because the patch is adding about two thousand cycles to every spinlock-taken path. Doesn't this patch yield better results? > However, the key aspect > here is this patch gives a way for the host to instruct the guest to use qspinlock. > Even with Longman's patch which allows guest to select the spinlock implementation, > there should still be the auto-select mode. In such mode, PV_DEDICATED should > allow the host to get the guest to use qspinlock, without, the guest will fallback > to tas when PV_UNHALT == 0. I agree that a flag can be useful for certains setups. diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 3df743b60c80..d9225e48c11a 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -676,6 +676,12 @@ void __init kvm_spinlock_init(void) { if (!kvm_para_available()) return; + + if (kvm_para_has_feature(KVM_FEATURE_PV_DEDICATED)) { + static_branch_disable(&virt_spin_lock_key); + return; + } + /* Does host kernel support KVM_FEATURE_PV_UNHALT? */ if (!kvm_para_has_feature(KVM_FEATURE_PV_UNHALT)) return;