From patchwork Fri Mar 15 13:40:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 13593485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E7530C54E67 for ; Fri, 15 Mar 2024 13:40:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=XhTyE+OkjqbC8YPkcuI9cdaLCBO9nzFQR718HDulCEk=; b=DdkRKg3SSO2Szx k/+tJr4g73UxYiXYtJqvdIQw1KBLTakw5EPHIVVsEKtQKPYtxhQmRzhWHZkwUf8yRFfK8es0UrIRM fJUW+fancw+SQ+myLf/kCWd49x2cfnI2kbUS0FJ6cUZXdkEvzRrHWfxXfX+imrf4/836vyqpeMju9 6Zx8K/g5iVX5ZFolPJgWXlEvlQD9IwopGdfvWnrTL+XesUW+bp1iIME9wV5BqGyRNdRPXl86GwvO9 jBQNgkrJcjCJXsADsLpXxlmW41tGisfPcXS/uekA6iRxh/yED3CU0pk8nAJKCxRixnT7oArYvN9t8 t4Wn1ccTjS3C+cLQSFWA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rl7nD-00000000M2B-2b06; Fri, 15 Mar 2024 13:40:15 +0000 Received: from mail-ed1-x534.google.com ([2a00:1450:4864:20::534]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rl7nA-00000000M0u-3vmp for linux-riscv@lists.infradead.org; Fri, 15 Mar 2024 13:40:14 +0000 Received: by mail-ed1-x534.google.com with SMTP id 4fb4d7f45d1cf-568b7350397so417963a12.1 for ; Fri, 15 Mar 2024 06:40:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1710510011; x=1711114811; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=8GpF70CfX6/ZYz42S7wUpo7NgF/Xw34C1Kcm5Nkkfys=; b=Yx1kfUNXvlHOSfMMvM2rX11iYu1ljiEFqdjySRU8wA2JGvWpbqK3SajLBIl9vYkmMA I4p2+Yt2oY9WXI2z/MSjsImX+TkvXfzFUGR12JWEkGXKt/0YKQLjJdKZj32/NuizDC79 Wih/tSIy8NzeZl2ogRhO/EjACZDCjCVXqwUJk6lFc0RR3aIOfdVUArlBvsJEbL8IoaGo 4qCPuROdeJQddllaFmYmBJqYb2zzWVkixTwG3hBNZGC7X1diQgOKh++c0M2eqStEtOOR DPW+DfWht7VMxJnpVvQiYZyf8eZs6efJwE1d6rn3shSX/tW7vX0+RnClerIrB9zwYX/Z dTuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710510011; x=1711114811; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8GpF70CfX6/ZYz42S7wUpo7NgF/Xw34C1Kcm5Nkkfys=; b=dTsxW170QqVEbhmF57HM7G2b/5ETfzolpIlGw8MV9oWyHYnndCYIgxxw8BFPt5/pBY lqKk7c0Skak2g84NLMR+2IunGV7lABxW3JDQMyYxSj6n8MdEVPTSsgwT6Qp4s+POM1WF pvXz1DsV0ngf6r9HYN8DVqG8vKq0zNQ0ZWRcwEuJ2UA5t1Ixkd+dYBUPX0qrUPiOB8tS yz4dj2tpSJdWKShnK4wWzM8ve5m1DnIyFH0xtQGq1fh7+kjSO0AWz19gN/qaiomHc7X6 UpOKNbnjJhpE+3naDpu4Y2G5GDfCYRjpovwMrKBDy+NCBH4BqzgtlXAiAfKIZglDzELF VUvA== X-Gm-Message-State: AOJu0Yz5Yty+77Qa2IWM8yO8QRwNcj1zarHmXsVoQLusJX8YxroCxEeG 0i8cPIG2PVuvQXJpTt36LYjQeaDIPP4TCGCVi5MtjxpD92LgVdUH1v/jLMVuHfOu+teokp7ozTk w X-Google-Smtp-Source: AGHT+IEJ+GZq0aCYW5AFPWYR3CFedcyBB47DOlC8RkupvQ/yhTekijeSH8H6wrfvFKiGlJBuhDSdUw== X-Received: by 2002:a17:906:7c88:b0:a46:6d16:14e with SMTP id w8-20020a1709067c8800b00a466d16014emr1970461ejo.50.1710510011274; Fri, 15 Mar 2024 06:40:11 -0700 (PDT) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id hg25-20020a1709072cd900b00a469be48551sm40805ejc.45.2024.03.15.06.40.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Mar 2024 06:40:10 -0700 (PDT) From: Andrew Jones To: linux-riscv@lists.infradead.org, kvm-riscv@lists.infradead.org Cc: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, conor.dooley@microchip.com, anup@brainfault.org, atishp@atishpatra.org, christoph.muellner@vrull.eu, heiko@sntech.de, charlie@rivosinc.com, David.Laight@ACULAB.COM Subject: [PATCH 0/5] riscv: Apply Zawrs when available Date: Fri, 15 Mar 2024 14:40:10 +0100 Message-ID: <20240315134009.580167-7-ajones@ventanamicro.com> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240315_064013_006276_9BAF7635 X-CRM114-Status: GOOD ( 19.63 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Zawrs provides two instructions (wrs.nto and wrs.sto), where both are meant to allow the hart to enter a low-power state while waiting on a store to a memory location. The instructions also both wait an implementation-defined "short" duration (unless the implementation terminates the stall for another reason). The difference is that while wrs.sto will terminate when the duration elapses, wrs.nto, depending on configuration, will either just keep waiting or an ILL exception will be raised. Like wfi (and with the same {m,h}status bits to configure it), when wrs.nto is configured to raise exceptions it's expected that the higher privilege level will see the instruction was a wait instruction, do something, and then resume execution following the instruction. Currently, it's not expected that M-mode will configure and handle exceptions for timeouts (so it's expected that mstatus.TW=0), but KVM does configure exceptions for wfi (hstatus.VTW=1) and therefore also for wrs.nto. KVM does this for wfi since it's better to allow other tasks to be scheduled while a VCPU waits for an interrupt. For waits such as those where wrs.nto/sto would be used, which are typically locks, it is also a good idea for KVM to be involved, as it can attempt to schedule the lock holding VCPU. This series starts with Christoph's addition of riscv smp_cond_load* functions which apply wrs.sto when available. We then switch from wrs.sto to wrs.nto, add hwprobe support (since the instructions are also usable from usermode), and finally teach KVM about wrs.nto, allowing guests to see and use the Zawrs extension. We still don't have test results from hardware, and it's not possible to prove that using Zawrs is a win when testing on QEMU, not even when oversubscribing VCPUs to guests. However, it is possible to use KVM selftests to force a scenario where we can prove Zawrs does its job and does it well. [4] is a test which does this and, on my machine, without Zawrs it takes 16 seconds to complete and with Zawrs it takes 0.25 seconds. This series is based on kvm/queue and also available here [1]. In order to use QEMU for testing a build with [2] is needed. In order to enable guests to use Zawrs with KVM using kvmtool, the branch at [3] may be used. [1] https://github.com/jones-drew/linux/commits/riscv/zawrs-v1/ [2] https://lore.kernel.org/all/20240312152901.512001-2-ajones@ventanamicro.com/ [3] https://github.com/jones-drew/kvmtool/commits/riscv/zawrs/ [4] https://github.com/jones-drew/linux/commit/2e712b19b7bb78634199bf262e6a75e09e1c87d2 Thanks, drew Andrew Jones (4): riscv: Prefer wrs.nto over wrs.sto riscv: hwprobe: export Zawrs ISA extension KVM: riscv: Support guest wrs.nto KVM: riscv: selftests: Add Zawrs extension to get-reg-list test Christoph Müllner (1): riscv: Add Zawrs support for spinlocks Documentation/arch/riscv/hwprobe.rst | 4 + arch/riscv/Kconfig | 13 +++ arch/riscv/include/asm/barrier.h | 87 +++++++++++++++++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/asm/kvm_host.h | 1 + arch/riscv/include/uapi/asm/hwprobe.h | 1 + arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kernel/cpufeature.c | 1 + arch/riscv/kernel/sys_hwprobe.c | 1 + arch/riscv/kvm/vcpu.c | 1 + arch/riscv/kvm/vcpu_insn.c | 15 ++++ arch/riscv/kvm/vcpu_onereg.c | 2 + .../selftests/kvm/riscv/get-reg-list.c | 4 + 13 files changed, 132 insertions(+)