From patchwork Fri Jul 9 04:37:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 12366657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30C9AC07E9C for ; Fri, 9 Jul 2021 04:39:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 02EC86141A for ; Fri, 9 Jul 2021 04:39:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 02EC86141A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kJWbFLWsuDpQrvTDVyO+FlGzDhLPlWXJpUwQn7SmjrY=; b=l0ELCoiQQ3j+4D 0GPrFY72G7VotVYtSe+n4jSdZ/hiIBZZz7l7MD5iiPtJx1ymq2hlqkDZpDy/teau2L6PUsvZZ/ejf /m9aXiVKnu3NzkYKanC8w6qw2x1s1ZXR9KM6pk4ZLcmxd4nUL0+ZJv1+tpvLLL/5QDOL7JjVsrzHL fE0zwPOaCTUCkrKVrhr9FoLEpEbwBRBgWeonS2qXyLhqIyk4VhiASTXfyn/7DrK+pwZJkQytGtytS 4tBNZ1lSaDZqjhqqvyQM07vJOwgUArhOXTEPldw3PkD2JBR3enAP6kCPK5XZgYVQIKadtg+cLH+0y BeilzxaAufiKnAmf1FBg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m1iGr-000k68-O1; Fri, 09 Jul 2021 04:37:49 +0000 Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m1iGe-000k3f-Iv for linux-arm-kernel@lists.infradead.org; Fri, 09 Jul 2021 04:37:37 +0000 Received: by mail-pg1-x532.google.com with SMTP id w15so8690373pgk.13 for ; Thu, 08 Jul 2021 21:37:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eBdDu50tMNuB7A6waM/ZyWIMl2LOJYIC1/NpalUridM=; b=C655qxBGfJlYMHH9QBFLbhyK4uAmvdwzkJEVRuHMZB1mKz3O4MEfvXbS4szocXmsQx XIZA71qy/kDzskTcV6GbpzRzk8XHO4KV9K7Cf1kSB9kEBsdsJHCYou1sWIlHT4aU7d8n oO+1zOGotRjyCjPehfvMbg1+d41s4nllzJJ4o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eBdDu50tMNuB7A6waM/ZyWIMl2LOJYIC1/NpalUridM=; b=RIaHE94j7wMDTrTc77v6oq/wF0NisbdU/t+eGPTwx3JGfnl3EEJonbtWVloq3BuXpZ 8CcTDQBLfQHTVqS1nwZ305yJ/butRbJ4g9AZMG8368gx7GLPVQcQv0KXOsUDKupTj8K+ rcSm+uyt1LKlvAW/UKED3KBtn9DmC+MwxyWKtHdcmSHBjn0qxwssDQI23AABVNHvAm/I zVwJoHJwqilD4aB1+pT2Fe0EJHNjCYZDTQ1AbD5/XsiFf6LMmKiWMm2RHVTvd8skfG7h U1GTfgBpj+sx6r/fWvcuOl6UkdncwwWJ3ODzXSwd03J3ZCXEBOSBMKp/1Cd8Xr48dh7r BHqg== X-Gm-Message-State: AOAM531CF2vA7Fc+D9enHHEkFmKgxnjQlBeFqJxyGdLL7gNNgm8Gc/wr P9nT/fETIHmbZ/qKJT7IFSVV0g== X-Google-Smtp-Source: ABdhPJyU5pINJoX/AogKYgRXTY5MbyhKXColfNbeIbVpm0ISXAwe7bj9tLDDjqgSf+cAP3MHB+R6Wg== X-Received: by 2002:a63:4e4d:: with SMTP id o13mr35755196pgl.361.1625805455971; Thu, 08 Jul 2021 21:37:35 -0700 (PDT) Received: from senozhatsky.flets-east.jp ([2409:10:2e40:5100:a30d:fc4:7834:ee08]) by smtp.gmail.com with ESMTPSA id y4sm4420760pfa.14.2021.07.08.21.37.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jul 2021 21:37:35 -0700 (PDT) From: Sergey Senozhatsky To: Marc Zyngier , Will Deacon Cc: Suleiman Souhlal , Joel Fernandes , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, Sergey Senozhatsky Subject: [PATCHv2 1/4] arm64: smccc: Add SMCCC pv-vcpu-state function call IDs Date: Fri, 9 Jul 2021 13:37:10 +0900 Message-Id: <20210709043713.887098-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog In-Reply-To: <20210709043713.887098-1-senozhatsky@chromium.org> References: <20210709043713.887098-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210708_213736_702376_82021971 X-CRM114-Status: GOOD ( 10.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add the definitions of the SMCCC functions that will be used to paravirt VCPU state configuration. Signed-off-by: Sergey Senozhatsky --- include/linux/arm-smccc.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index 7d1cabe15262..dbf0d658e54a 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -177,6 +177,24 @@ ARM_SMCCC_OWNER_STANDARD, \ 0x53) +#define ARM_SMCCC_HV_PV_VCPU_STATE_FEATURES \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_STANDARD_HYP, \ + 0x60) + +#define ARM_SMCCC_HV_PV_VCPU_STATE_INIT \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_STANDARD_HYP, \ + 0x61) + +#define ARM_SMCCC_HV_PV_VCPU_STATE_RELEASE \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_STANDARD_HYP, \ + 0x62) + /* * Return codes defined in ARM DEN 0070A * ARM DEN 0070A is now merged/consolidated into ARM DEN 0028 C From patchwork Fri Jul 9 04:37:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 12366661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-22.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70010C11F66 for ; Fri, 9 Jul 2021 04:39:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 36C646141A for ; Fri, 9 Jul 2021 04:39:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36C646141A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L/RtS9/Ihp2+jWDcmYdgPG9RN3wGlE1ugMDV55ICzp0=; b=GxzKIfHyeMKWHJ +m39VDf7D+lQ84iNeTYlHnca/IySnbwkFmoIQynO9D3fRB1x6cODvXx23kq6ohGYsT6RixqgH06vm mwP+rzt4ly0OxbjC1GzyfCz9QOQqFGV5XYRl3SLik+yB8udXy7MuV2pcPpc2lmEE4oYpWGi195ygQ sbgS+OwztU9XfD86YSWADhGjzEiQeKznvXniKYSTBsD2KrgMNk+sCi7uN2MUN1KUFgUfT9JbKQi2D IsHOoShGfpmbV8rFjnobTx3sOuPqnBmkjm1v2RUzCjqg0yH6LfC8ZZJwWYwOrkEkM37RKbqWCzDXh azcZw1Lj98xwapirm+0g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m1iH1-000k7R-PU; Fri, 09 Jul 2021 04:37:59 +0000 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m1iGh-000k4H-CM for linux-arm-kernel@lists.infradead.org; Fri, 09 Jul 2021 04:37:41 +0000 Received: by mail-pj1-x1036.google.com with SMTP id oj10-20020a17090b4d8ab0290172f77377ebso5401848pjb.0 for ; Thu, 08 Jul 2021 21:37:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZZUGx6FUn/yCOexFnd62yjwoHeZYcMvsi4jsYnjIhcI=; b=mnqL5LKj1+FnamixOwkTSQeCEEsEDIiTJ4OrXzpL+MYCLTaIX8UceKo56jssGXfhya AiyKBgqPshL1jRfsrnva3VJkh9nIs5pzd7owYvCclj5HvzoDdtDpNgTYta2VlV97cVbq f719LMEdgFQMijyy0TdAzT9kq+olxmWAYTGWI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZZUGx6FUn/yCOexFnd62yjwoHeZYcMvsi4jsYnjIhcI=; b=k1pxGj/7Ayfqss1jIAmIdFqsiScZbszcaC/gDd528FhybWvCAJ5QSZoVXFkdNLHXUV 3amcOkYvJtvxBi6274WVKi8hWZ+aX4uFAqmUTlIUvRHzHS4zwYmmZiDAJnOWk9IgJU1g aOPWShd+ArYlROYOVogZ0vo7JMSnW70lQMM9hq583D8eI029IyG4OUYkhrjwgSTcpo0j 89AsmYKhMyhOyA4KL6HSZ3wfI2NORm5CiiTfV6ltugCvTUtoAfMMfCjOW9WHFyOkuvFq e6IN6Rsnufyt9XuCyOhJFrngdRnxS/1KjGK3NK7SPaXdzy20Kxn+j0njhUwr3h/ZMUyy 1icQ== X-Gm-Message-State: AOAM53136XJY53cfb2ymbL4eGDkt3woOOcZT6ARTP0tJj+NBtcoUtY55 ae+21uyOGNh+U1/STzn5CNAv6w== X-Google-Smtp-Source: ABdhPJy83yDQ64mKLzi/SYhx94sw/R1wHIjE+W52lxng461RsjxcZZAvhU+WYZGbVC1nt6g8LZkI2g== X-Received: by 2002:a17:903:18c:b029:125:b183:798f with SMTP id z12-20020a170903018cb0290125b183798fmr28860558plg.24.1625805458691; Thu, 08 Jul 2021 21:37:38 -0700 (PDT) Received: from senozhatsky.flets-east.jp ([2409:10:2e40:5100:a30d:fc4:7834:ee08]) by smtp.gmail.com with ESMTPSA id y4sm4420760pfa.14.2021.07.08.21.37.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jul 2021 21:37:38 -0700 (PDT) From: Sergey Senozhatsky To: Marc Zyngier , Will Deacon Cc: Suleiman Souhlal , Joel Fernandes , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, Sergey Senozhatsky Subject: [PATCHv2 2/4] arm64: add guest pvstate support Date: Fri, 9 Jul 2021 13:37:11 +0900 Message-Id: <20210709043713.887098-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog In-Reply-To: <20210709043713.887098-1-senozhatsky@chromium.org> References: <20210709043713.887098-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210708_213739_466990_4AEB4F99 X-CRM114-Status: GOOD ( 17.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org PV-vcpu-state is a per-CPU struct, which, for the time being, holds boolean `preempted' vCPU state. During the startup, given that host supports PV-state, each guest vCPU sends a pointer to its per-CPU variable to the host as a payload with the SMCCC HV call, so that host can update vCPU state when it puts or loads vCPU. This has impact on the guest's scheduler: [..] wake_up_process() try_to_wake_up() select_task_rq_fair() available_idle_cpu() vcpu_is_preempted() Some sched benchmarks data is available on the github page [0]. [0] https://github.com/sergey-senozhatsky/arm64-vcpu_is_preempted Signed-off-by: Sergey Senozhatsky --- arch/arm64/include/asm/paravirt.h | 19 +++++++ arch/arm64/kernel/paravirt.c | 94 +++++++++++++++++++++++++++++++ arch/arm64/kernel/smp.c | 4 ++ 3 files changed, 117 insertions(+) diff --git a/arch/arm64/include/asm/paravirt.h b/arch/arm64/include/asm/paravirt.h index 9aa193e0e8f2..a3f7665dff38 100644 --- a/arch/arm64/include/asm/paravirt.h +++ b/arch/arm64/include/asm/paravirt.h @@ -2,6 +2,11 @@ #ifndef _ASM_ARM64_PARAVIRT_H #define _ASM_ARM64_PARAVIRT_H +struct vcpu_state { + bool preempted; + u8 reserved[63]; +}; + #ifdef CONFIG_PARAVIRT #include @@ -20,8 +25,22 @@ static inline u64 paravirt_steal_clock(int cpu) int __init pv_time_init(void); +bool dummy_vcpu_is_preempted(unsigned int cpu); + +extern struct static_key pv_vcpu_is_preempted_enabled; +DECLARE_STATIC_CALL(pv_vcpu_is_preempted, dummy_vcpu_is_preempted); + +static inline bool paravirt_vcpu_is_preempted(unsigned int cpu) +{ + return static_call(pv_vcpu_is_preempted)(cpu); +} + +int __init pv_vcpu_state_init(void); + #else +#define pv_vcpu_state_init() do {} while (0) + #define pv_time_init() do {} while (0) #endif // CONFIG_PARAVIRT diff --git a/arch/arm64/kernel/paravirt.c b/arch/arm64/kernel/paravirt.c index 75fed4460407..d8fc46795d94 100644 --- a/arch/arm64/kernel/paravirt.c +++ b/arch/arm64/kernel/paravirt.c @@ -40,6 +40,11 @@ struct pv_time_stolen_time_region { static DEFINE_PER_CPU(struct pv_time_stolen_time_region, stolen_time_region); +static DEFINE_PER_CPU(struct vcpu_state, vcpus_states); +struct static_key pv_vcpu_is_preempted_enabled; + +DEFINE_STATIC_CALL(pv_vcpu_is_preempted, dummy_vcpu_is_preempted); + static bool steal_acc = true; static int __init parse_no_stealacc(char *arg) { @@ -165,3 +170,92 @@ int __init pv_time_init(void) return 0; } + +bool dummy_vcpu_is_preempted(unsigned int cpu) +{ + return false; +} + +static bool __vcpu_is_preempted(unsigned int cpu) +{ + struct vcpu_state *st; + + st = &per_cpu(vcpus_states, cpu); + return READ_ONCE(st->preempted); +} + +static bool has_pv_vcpu_state(void) +{ + struct arm_smccc_res res; + + /* To detect the presence of PV time support we require SMCCC 1.1+ */ + if (arm_smccc_1_1_get_conduit() == SMCCC_CONDUIT_NONE) + return false; + + arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, + ARM_SMCCC_HV_PV_VCPU_STATE_FEATURES, + &res); + + if (res.a0 != SMCCC_RET_SUCCESS) + return false; + return true; +} + +static int __pv_vcpu_state_hook(unsigned int cpu, int event) +{ + struct arm_smccc_res res; + struct vcpu_state *st; + + st = &per_cpu(vcpus_states, cpu); + arm_smccc_1_1_invoke(event, virt_to_phys(st), &res); + if (res.a0 != SMCCC_RET_SUCCESS) + return -EINVAL; + return 0; +} + +static int vcpu_state_init(unsigned int cpu) +{ + int ret = __pv_vcpu_state_hook(cpu, ARM_SMCCC_HV_PV_VCPU_STATE_INIT); + + if (ret) + pr_warn("Unable to ARM_SMCCC_HV_PV_STATE_INIT\n"); + return ret; +} + +static int vcpu_state_release(unsigned int cpu) +{ + int ret = __pv_vcpu_state_hook(cpu, ARM_SMCCC_HV_PV_VCPU_STATE_RELEASE); + + if (ret) + pr_warn("Unable to ARM_SMCCC_HV_PV_STATE_RELEASE\n"); + return ret; +} + +static int pv_vcpu_state_register_hooks(void) +{ + int ret; + + ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, + "hypervisor/arm/pvstate:starting", + vcpu_state_init, + vcpu_state_release); + if (ret < 0) + pr_warn("Failed to register CPU hooks\n"); + return 0; +} + +int __init pv_vcpu_state_init(void) +{ + int ret; + + if (!has_pv_vcpu_state()) + return 0; + + ret = pv_vcpu_state_register_hooks(); + if (ret) + return ret; + + static_call_update(pv_vcpu_is_preempted, __vcpu_is_preempted); + static_key_slow_inc(&pv_vcpu_is_preempted_enabled); + return 0; +} diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 6f6ff072acbd..20d42e0f2a99 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -50,6 +50,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -756,6 +757,9 @@ void __init smp_prepare_cpus(unsigned int max_cpus) numa_store_cpu_info(this_cpu); numa_add_cpu(this_cpu); + /* Init paravirt CPU state */ + pv_vcpu_state_init(); + /* * If UP is mandated by "nosmp" (which implies "maxcpus=0"), don't set * secondary CPUs present. From patchwork Fri Jul 9 04:37:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 12366663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95050C07E9E for ; Fri, 9 Jul 2021 04:39:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5C9DC61437 for ; Fri, 9 Jul 2021 04:39:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5C9DC61437 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZgtaWYcFUspNk3vBCxBmAZE+FqGV2ba4VKUbWesFDmE=; b=HXIBo9pVALhhms jIm6A8xW6xlGuFNvCKXxA3w52x8aPTiJTzJMx5n9MwpHL+FCYTXepRqbeneh+gAoFEIlfyWwFFkUS mTCiedvIpKqabkdcluyt3z2iC1GVe+0zkdj+ySFz15sLleDELYvZL7cZsgZf7daFJ6/hyRlC9cx3P /dF58/F2D2384xNDY6HPDj+tVmKJ27IAlaFmBqgEt7v/jfCgU02BZlgcZIVxMX8jOgbivAIV/qskj N8OlLmN7GkLDfLRDh9Dr4yyiT+gOuQDGcoZvWW5HCEpb4j79JixUWbuS24C5+OHOhmHG9kz9FMj0q Rk7L9IkhJEw4IWBQFZ6w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m1iHD-000k96-PX; Fri, 09 Jul 2021 04:38:11 +0000 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m1iGk-000k4t-4t for linux-arm-kernel@lists.infradead.org; Fri, 09 Jul 2021 04:37:43 +0000 Received: by mail-pj1-x102a.google.com with SMTP id l11so5072102pji.5 for ; Thu, 08 Jul 2021 21:37:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QP3/GYzOBKmilm7HuEJkuqubxHWg9l5vJdRHFzc5VNI=; b=RclDDj3+rehsUldw5TyhuwvHXTZwkgCQaCL0mXmrg3nr75emQHZsdQL/PMBpmd2FSB 1okB0a10syEhmCx3GptxPQ6XYeTuHpa4AESmoVEqtDp3+bLgb4ii01UctXQhRSyNL3s2 nQd9N40sCnQ4Tz9f+NrHySY4qVntzcKYSjVho= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QP3/GYzOBKmilm7HuEJkuqubxHWg9l5vJdRHFzc5VNI=; b=bIZ/UoeBjiwYu8LRpjm4t3kn4A6LmzQtuwi9CjGUuIU4xAdP7ykM3qrHdJmiqrKisK 7a7XqlmKENky71h3XaUpfs5A/tlVhtb4kD7oleqCUZ+JWbYou8l/dCA8ie8Nfu+nctEh +sNCwM9yPbcov+Ff97Dg6fQMD6N4n5S4wOWQ/wNlTCiszEAoAnJmAPdFb22888ETnWPP KzUTTTrWbEIlu60Hpt1MxuhhXHLCWAo6w2/d/YabnYMVx/yrM6iQrtenhB1okxFHqg4p rDo9d1NrBzvjQrGMG6qh6xuj/ZO2tuLgt+rp08vuaDn9C6eaa0fR2AmhhitGfChV9KRR TeIg== X-Gm-Message-State: AOAM5310g4s/0NuZ6eItuNRINAcKdcNnCDFPLcBby/Seu80sWZZRPvF9 ra/G5FZP3rOMdkSzFXsrOBEI2g== X-Google-Smtp-Source: ABdhPJw7ofI5jbueMZXenvbdrtCTJqUoJIWkbzb4yngHCY3TUfGWSBMgYm2T5M9G4xte0hL+XeYjmw== X-Received: by 2002:a17:902:bc47:b029:129:dd30:1c30 with SMTP id t7-20020a170902bc47b0290129dd301c30mr5497773plz.4.1625805461468; Thu, 08 Jul 2021 21:37:41 -0700 (PDT) Received: from senozhatsky.flets-east.jp ([2409:10:2e40:5100:a30d:fc4:7834:ee08]) by smtp.gmail.com with ESMTPSA id y4sm4420760pfa.14.2021.07.08.21.37.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jul 2021 21:37:41 -0700 (PDT) From: Sergey Senozhatsky To: Marc Zyngier , Will Deacon Cc: Suleiman Souhlal , Joel Fernandes , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, Sergey Senozhatsky Subject: [PATCHv2 3/4] arm64: do not use dummy vcpu_is_preempted() Date: Fri, 9 Jul 2021 13:37:12 +0900 Message-Id: <20210709043713.887098-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog In-Reply-To: <20210709043713.887098-1-senozhatsky@chromium.org> References: <20210709043713.887098-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210708_213742_232695_6EAD49B8 X-CRM114-Status: GOOD ( 11.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org vcpu_is_preempted() now can represent the actual state of the VCPU, so the scheduler can make better decisions when it picks the idle CPU to enqueue a task on. Signed-off-by: Sergey Senozhatsky --- arch/arm64/include/asm/spinlock.h | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/spinlock.h b/arch/arm64/include/asm/spinlock.h index 0525c0b089ed..1d579497e1b8 100644 --- a/arch/arm64/include/asm/spinlock.h +++ b/arch/arm64/include/asm/spinlock.h @@ -7,21 +7,23 @@ #include #include +#include /* See include/linux/spinlock.h */ #define smp_mb__after_spinlock() smp_mb() -/* - * Changing this will break osq_lock() thanks to the call inside - * smp_cond_load_relaxed(). - * - * See: - * https://lore.kernel.org/lkml/20200110100612.GC2827@hirez.programming.kicks-ass.net - */ #define vcpu_is_preempted vcpu_is_preempted -static inline bool vcpu_is_preempted(int cpu) + +#ifdef CONFIG_PARAVIRT +static inline bool vcpu_is_preempted(unsigned int cpu) +{ + return paravirt_vcpu_is_preempted(cpu); +} +#else +static inline bool vcpu_is_preempted(unsigned int cpu) { return false; } +#endif /* CONFIG_PARAVIRT */ #endif /* __ASM_SPINLOCK_H */ From patchwork Fri Jul 9 04:37:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 12366665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA363C07E99 for ; Fri, 9 Jul 2021 04:39:48 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AAF6161435 for ; Fri, 9 Jul 2021 04:39:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAF6161435 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w6qW6jOMCKLjx1DxL5sty16Mh6zBM2X//BHa7jlepvc=; b=4N8FFXnEAH/9Vy g7ILUnbEXc12idnLsbHytfow1IA80/iESDZlQGJXKFk/R26LRzC1gfP9TpkLoJPiWYsXtKsrDFglA +JkLJHqwiCu7H5wfFX6BrkXcxP8/HIfs/EQ8FpcRnDDL3OloNyON/e8eFCKAq1Fu5oLlNPOOxkNZq LH9A0mMnrTv+0jPFAq0HkOjlD5bCURtMfHtgde8a7wBM5/i8d03I84ZvSW8p3VOSbtm2e04FYp0Bd xBzT/g00Es+b8VGerw6aHXJHk+MJzW6j7nZOsKz0aD5DsPfvz0wiRt1KESmwTZZgh1alan8RBwzAC cIGatzXvWGM4tkc5Cv7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m1iHP-000kCl-HU; Fri, 09 Jul 2021 04:38:23 +0000 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m1iGm-000k5G-R7 for linux-arm-kernel@lists.infradead.org; Fri, 09 Jul 2021 04:37:46 +0000 Received: by mail-pg1-x534.google.com with SMTP id w15so8690639pgk.13 for ; Thu, 08 Jul 2021 21:37:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ETT27YTe1Df1R34anpclNh/mV2vuSiYHLPmA2jkljAE=; b=dmlECuSGzQyT5BPgzoazb9gJYXKcWH37icvNOgZw3JI5O2XU1fCzPX6W+JxiavMd5Y scIVQeiaRcZ1l5KpD5zrGJ+/WAobQlvoW4/EiobILmMS+XW8TD2yOCS+Kijs4IGU+OiX HH1Dnh7r/BPjoIvuPN/qVdzlL2Go2M62+fnlo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ETT27YTe1Df1R34anpclNh/mV2vuSiYHLPmA2jkljAE=; b=e4v0ZVOu2VbvIxgJj+Q8W5PN0hORq/AXHyXviX1bZFa+R1OQnUuR7dgWWTnk9TNAiK Qp77qRAVXnFNnPZeWqAuq77hhBtloNOTsOVYVuJW9trQR8jlVu0ZGZlU4+uuTYxNSNyu slCpSENozTux8yJ/wY5rX+IckhCF1FaEfdOEF1DiclXxCX+ThW4rvEyCLKJhs4NNUhs+ BrXXeJ7MVD6N6IvuZoXepy6ndb9I24rDlj+3leujuHSoVGVyI1bKRCRSXy8Y3T38dIeE +V0X9x9VNyVCOafXAeg5g1xztnC7ScZfe2Skx7KiBa9bJBCK+/wYYpT/iHd5G8dw+1v/ SEdA== X-Gm-Message-State: AOAM532vVA1Qpr8qjP+hDxl8KW7psNYqYN3BLZixnNdpMTYhRy9/HYKU pOJfU0chAAm131/BlkLZtrCIfA== X-Google-Smtp-Source: ABdhPJyUpwfKVXdRnNKLdaV06jZZYOCtJWZE+pQ7Yig5Okf20u5ZB19Pmb0K+S0mixoNDsX0ipuwgg== X-Received: by 2002:a63:1226:: with SMTP id h38mr35402819pgl.376.1625805464184; Thu, 08 Jul 2021 21:37:44 -0700 (PDT) Received: from senozhatsky.flets-east.jp ([2409:10:2e40:5100:a30d:fc4:7834:ee08]) by smtp.gmail.com with ESMTPSA id y4sm4420760pfa.14.2021.07.08.21.37.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jul 2021 21:37:43 -0700 (PDT) From: Sergey Senozhatsky To: Marc Zyngier , Will Deacon Cc: Suleiman Souhlal , Joel Fernandes , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, Sergey Senozhatsky Subject: [PATCHv2 4/4] arm64: add host pv-vcpu-state support Date: Fri, 9 Jul 2021 13:37:13 +0900 Message-Id: <20210709043713.887098-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog In-Reply-To: <20210709043713.887098-1-senozhatsky@chromium.org> References: <20210709043713.887098-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210708_213744_930487_B59B426F X-CRM114-Status: GOOD ( 23.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add PV-vcpu-state support bits to the host. Host uses the guest PV-state per-CPU pointers to update the VCPU state each time it kvm_arch_vcpu_load() or kvm_arch_vcpu_put() the VCPU, so that guest scheduler can become aware of the fact that not all VCPUs are always available. Currently guest scheduler on amr64 always assumes that all CPUs are available because vcpu_is_preempted() is not implemented on arm64. - schbench -t 3 -m 3 -p 4096 Latency percentiles (usec) BASE ================================================ 50.0th: 1 (3556427 samples) 75.0th: 13 (879210 samples) 90.0th: 15 (893311 samples) 95.0th: 18 (159594 samples) *99.0th: 118 (224187 samples) 99.5th: 691 (28555 samples) 99.9th: 7384 (23076 samples) min=1, max=104218 avg worker transfer: 25192.00 ops/sec 98.41MB/s PATCHED ================================================ 50.0th: 1 (3507010 samples) 75.0th: 13 (1635775 samples) 90.0th: 16 (901271 samples) 95.0th: 24 (281051 samples) *99.0th: 114 (255581 samples) 99.5th: 382 (33051 samples) 99.9th: 6392 (26592 samples) min=1, max=83877 avg worker transfer: 28613.39 ops/sec 111.77MB/s - perf bench sched all ops/sec BASE PATCHED ================================================ 33452 36485 33541 39405 33365 36858 33455 38047 33449 37866 33616 34922 33479 34388 33594 37203 33458 35363 33704 35180 Student's T-test N Min Max Median Avg Stddev base 10 33365 33704 33479 33511.3 100.92467 patched 10 34388 39405 36858 36571.7 1607.454 Difference at 95.0% confidence 3060.4 +/- 1070.09 9.13244% +/- 3.19321% (Student's t, pooled s = 1138.88) Signed-off-by: Sergey Senozhatsky --- arch/arm64/include/asm/kvm_host.h | 23 +++++++++++ arch/arm64/kvm/Makefile | 3 +- arch/arm64/kvm/arm.c | 3 ++ arch/arm64/kvm/hypercalls.c | 11 ++++++ arch/arm64/kvm/pv-vcpu-state.c | 64 +++++++++++++++++++++++++++++++ 5 files changed, 103 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/kvm/pv-vcpu-state.c diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 41911585ae0c..e782f4d0c916 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -381,6 +381,12 @@ struct kvm_vcpu_arch { u64 last_steal; gpa_t base; } steal; + + /* PV state of the VCPU */ + struct { + gpa_t base; + struct gfn_to_hva_cache ghc; + } vcpu_state; }; /* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ @@ -695,6 +701,23 @@ static inline bool kvm_arm_is_pvtime_enabled(struct kvm_vcpu_arch *vcpu_arch) return (vcpu_arch->steal.base != GPA_INVALID); } +int kvm_init_vcpu_state(struct kvm_vcpu *vcpu, gfn_t addr); +int kvm_release_vcpu_state(struct kvm_vcpu *vcpu); + +static inline void kvm_arm_vcpu_state_init(struct kvm_vcpu_arch *vcpu_arch) +{ + vcpu_arch->vcpu_state.base = GPA_INVALID; + memset(&vcpu_arch->vcpu_state.ghc, 0, sizeof(struct gfn_to_hva_cache)); +} + +static inline bool +kvm_arm_is_vcpu_state_enabled(struct kvm_vcpu_arch *vcpu_arch) +{ + return (vcpu_arch->vcpu_state.base != GPA_INVALID); +} + +void kvm_update_vcpu_preempted(struct kvm_vcpu *vcpu, bool preempted); + void kvm_set_sei_esr(struct kvm_vcpu *vcpu, u64 syndrome); struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr); diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 989bb5dad2c8..2a3ee82c6d90 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -12,7 +12,8 @@ obj-$(CONFIG_KVM) += hyp/ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \ $(KVM)/vfio.o $(KVM)/irqchip.o $(KVM)/binary_stats.o \ - arm.o mmu.o mmio.o psci.o perf.o hypercalls.o pvtime.o \ + arm.o mmu.o mmio.o psci.o perf.o hypercalls.o \ + pvtime.o pv-vcpu-state.o \ inject_fault.o va_layout.o handle_exit.o \ guest.o debug.o reset.o sys_regs.o \ vgic-sys-reg-v3.o fpsimd.o pmu.o \ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e9a2b8f27792..43e995c9fddb 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -332,6 +332,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) kvm_arm_reset_debug_ptr(vcpu); kvm_arm_pvtime_vcpu_init(&vcpu->arch); + kvm_arm_vcpu_state_init(&vcpu->arch); vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu; @@ -429,10 +430,12 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) if (vcpu_has_ptrauth(vcpu)) vcpu_ptrauth_disable(vcpu); kvm_arch_vcpu_load_debug_state_flags(vcpu); + kvm_update_vcpu_preempted(vcpu, false); } void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) { + kvm_update_vcpu_preempted(vcpu, true); kvm_arch_vcpu_put_debug_state_flags(vcpu); kvm_arch_vcpu_put_fp(vcpu); if (has_vhe()) diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c index 30da78f72b3b..95bcf86e0b6f 100644 --- a/arch/arm64/kvm/hypercalls.c +++ b/arch/arm64/kvm/hypercalls.c @@ -110,6 +110,9 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) case ARM_SMCCC_HV_PV_TIME_FEATURES: val[0] = SMCCC_RET_SUCCESS; break; + case ARM_SMCCC_HV_PV_VCPU_STATE_FEATURES: + val[0] = SMCCC_RET_SUCCESS; + break; } break; case ARM_SMCCC_HV_PV_TIME_FEATURES: @@ -139,6 +142,14 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) case ARM_SMCCC_TRNG_RND32: case ARM_SMCCC_TRNG_RND64: return kvm_trng_call(vcpu); + case ARM_SMCCC_HV_PV_VCPU_STATE_INIT: + if (kvm_init_vcpu_state(vcpu, smccc_get_arg1(vcpu)) == 0) + val[0] = SMCCC_RET_SUCCESS; + break; + case ARM_SMCCC_HV_PV_VCPU_STATE_RELEASE: + if (kvm_release_vcpu_state(vcpu) == 0) + val[0] = SMCCC_RET_SUCCESS; + break; default: return kvm_psci_call(vcpu); } diff --git a/arch/arm64/kvm/pv-vcpu-state.c b/arch/arm64/kvm/pv-vcpu-state.c new file mode 100644 index 000000000000..8496bb2a5966 --- /dev/null +++ b/arch/arm64/kvm/pv-vcpu-state.c @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include + +#include +#include + +#include + +int kvm_init_vcpu_state(struct kvm_vcpu *vcpu, gpa_t addr) +{ + struct kvm *kvm = vcpu->kvm; + int ret; + u64 idx; + + if (kvm_arm_is_vcpu_state_enabled(&vcpu->arch)) + return 0; + + idx = srcu_read_lock(&kvm->srcu); + ret = kvm_gfn_to_hva_cache_init(vcpu->kvm, + &vcpu->arch.vcpu_state.ghc, + addr, + sizeof(struct vcpu_state)); + srcu_read_unlock(&kvm->srcu, idx); + + if (!ret) + vcpu->arch.vcpu_state.base = addr; + return ret; +} + +int kvm_release_vcpu_state(struct kvm_vcpu *vcpu) +{ + if (!kvm_arm_is_vcpu_state_enabled(&vcpu->arch)) + return 0; + + kvm_arm_vcpu_state_init(&vcpu->arch); + return 0; +} + +void kvm_update_vcpu_preempted(struct kvm_vcpu *vcpu, bool preempted) +{ + struct kvm *kvm = vcpu->kvm; + u64 idx; + + if (!kvm_arm_is_vcpu_state_enabled(&vcpu->arch)) + return; + + /* + * This function is called from atomic context, so we need to + * disable page faults. kvm_write_guest_cached() will call + * might_fault(). + */ + pagefault_disable(); + /* + * Need to take the SRCU lock because kvm_write_guest_offset_cached() + * calls kvm_memslots(); + */ + idx = srcu_read_lock(&kvm->srcu); + kvm_write_guest_cached(kvm, &vcpu->arch.vcpu_state.ghc, + &preempted, sizeof(bool)); + srcu_read_unlock(&kvm->srcu, idx); + pagefault_enable(); +}