From patchwork Sun Sep 15 18:34:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Raphael Tiovalen X-Patchwork-Id: 13804879 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E99901C9DF3 for ; Sun, 15 Sep 2024 18:35:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726425310; cv=none; b=Xhkpqmy07y9sJzBWECAmpt1kJeLTSy/eQgkFrcfiMDMaJxnAQm+3KmjZ0lxe2W1VaXOMV9KxpN+AwtKJxH/IP/HI1CArNlU92djORMv4m5b3mTdP0FGIyFQdJANh7L5+ucf4NCUyCZ+MG96ygziFfc6UMx7zTwboresVuHK/CR0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726425310; c=relaxed/simple; bh=i+LrkneODal7A0KUNklvU3HueYmU2j5tLZ4SiaMqtrs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q6ORKz6i0FxuNY06B1tdo6WoeopiPGzvuIiN5NXs6j/fzu6JklmA3rKcv4Qj5DFakMqRYdKC6b8LCPwDeHcATf8BnzaZNtxLZK07nOxmWwTH5BROC/Dfcdxs/sqJuBgYj9ua2bzKv1Q3veGy2/rAH5HQbzW+Mni8eiNdACueRi8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kNVPwG5b; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kNVPwG5b" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-718f4fd89e5so3473379b3a.0 for ; Sun, 15 Sep 2024 11:35:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726425308; x=1727030108; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9LTSemqbVgb/sEkY5N+jEphbkR9WTsGrnAb88vyNg3M=; b=kNVPwG5bLURbqPitrxlLOwvSmRStV2wkwu+eu5JkPsmKhzpyKaS/1X3W6RbV1BTEnt WjbGiCjeOhRIr9M2OMEXvFozqDXa7ak2XrtfRcKZzaZhtGSS3Mt2SBM/PAEFy8ZRSbum 0v4KtVIYhLus0LC4OyZA/KFxuXWV3/5m/ZGH3QwwMvRWR7U4cgP0l/AmItw9IAJ1Ll8h FOEJMaO3BHzbxgEcN9+PKjEC9gOVrR+fdr1/V+fMjUbluxg974usUcMUZPUiAI1AKq+J B4U/K9XjR3i8uVm1VPAoXkezh2wlq3ajn3Wn9Tdd8urI7NPKIV4/7xYwrbUWfn7ZKCdO CJCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726425308; x=1727030108; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9LTSemqbVgb/sEkY5N+jEphbkR9WTsGrnAb88vyNg3M=; b=OM42cWItyZaxEPp5XxjD+UTKC9FdbSupau9kaM8aTsha2kOCYeuXzexBViAi8eZ5UK FbtFXDzyo5XNq/PzDSp2U3CDF2RG0Wzppa4uEDH6Qho7xZuUlBGn6QPQyN8vcyFE7024 +/u0OmIig8FrtuexYhC/nKmciozWxkGiNCHMdfLYYujpOpNnx7z03Uo6P6PErfU2kDK4 cQuoQ3IzoMZQAKU0kqCnGJCTW9/dZCMNqcAn37ZCTFZaYbUZGGHfTkkTnKyOgCth2aDf jJvHs5cQaiON0Nw9eEullvwbNOX0OwHNvXk6WfZW6bRAzQN+R4BATwsfnNg5VqyEwjiv eMOA== X-Gm-Message-State: AOJu0YzL/wVddNUNv8ZIbp1wITNfQPLRWGqxRGnoJ+20NYb1JVMcLjPi xJxhvQiQ7SOBkSu/yKjAEFtC//woZ4kgSmu797o9q3CA4K71kCscbqW5Iigl X-Google-Smtp-Source: AGHT+IEJYx8snMnEK1Dol2k51jwYmO3tN2vK5II1BkhfqIXuTsrf0eGkZO3AURKnOYqGhKP8TJpMLA== X-Received: by 2002:a05:6a00:b92:b0:717:8d81:e548 with SMTP id d2e1a72fcca58-7192606523bmr19627833b3a.1.1726425307601; Sun, 15 Sep 2024 11:35:07 -0700 (PDT) Received: from JRT-PC.. ([202.166.44.78]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7db498de2d5sm2358874a12.15.2024.09.15.11.35.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Sep 2024 11:35:06 -0700 (PDT) From: James Raphael Tiovalen To: kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: andrew.jones@linux.dev, atishp@rivosinc.com, cade.richard@berkeley.edu Subject: [kvm-unit-tests PATCH v4 1/3] riscv: Rewrite hartid_to_cpu in assembly Date: Mon, 16 Sep 2024 02:34:57 +0800 Message-ID: <20240915183459.52476-2-jamestiotio@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240915183459.52476-1-jamestiotio@gmail.com> References: <20240915183459.52476-1-jamestiotio@gmail.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Andrew Jones Some SBI HSM tests run without a stack being setup so they can't run C code. Those tests still need to know the corresponding cpuid for the hartid on which they are running. Give those tests hartid_to_cpu() by reimplementing it in assembly. Signed-off-by: Andrew Jones --- lib/riscv/asm-offsets.c | 5 +++++ lib/riscv/setup.c | 10 ---------- riscv/cstart.S | 23 +++++++++++++++++++++++ 3 files changed, 28 insertions(+), 10 deletions(-) diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c index a2a32438..6c511c14 100644 --- a/lib/riscv/asm-offsets.c +++ b/lib/riscv/asm-offsets.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-only #include #include +#include #include #include @@ -58,5 +59,9 @@ int main(void) OFFSET(SECONDARY_FUNC, secondary_data, func); DEFINE(SECONDARY_DATA_SIZE, sizeof(struct secondary_data)); + OFFSET(THREAD_INFO_CPU, thread_info, cpu); + OFFSET(THREAD_INFO_HARTID, thread_info, hartid); + DEFINE(THREAD_INFO_SIZE, sizeof(struct thread_info)); + return 0; } diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c index 495db041..f347ad63 100644 --- a/lib/riscv/setup.c +++ b/lib/riscv/setup.c @@ -43,16 +43,6 @@ uint64_t timebase_frequency; static struct mem_region riscv_mem_regions[NR_MEM_REGIONS + 1]; -int hartid_to_cpu(unsigned long hartid) -{ - int cpu; - - for_each_present_cpu(cpu) - if (cpus[cpu].hartid == hartid) - return cpu; - return -1; -} - static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused) { int cpu = nr_cpus++; diff --git a/riscv/cstart.S b/riscv/cstart.S index 8f269997..6784d5e1 100644 --- a/riscv/cstart.S +++ b/riscv/cstart.S @@ -109,6 +109,29 @@ halt: 1: wfi j 1b +/* + * hartid_to_cpu + * a0 is a hartid on entry + * returns the corresponding cpuid in a0 + */ +.balign 4 +.global hartid_to_cpu +hartid_to_cpu: + la t0, cpus + la t1, nr_cpus + lw t1, 0(t1) + li t2, 0 +1: bne t2, t1, 2f + li a0, -1 + ret +2: REG_L t3, THREAD_INFO_HARTID(t0) + bne a0, t3, 3f + lw a0, THREAD_INFO_CPU(t0) + ret +3: addi t0, t0, THREAD_INFO_SIZE + addi t2, t2, 1 + j 1b + .balign 4 .global secondary_entry secondary_entry: From patchwork Sun Sep 15 18:34:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Raphael Tiovalen X-Patchwork-Id: 13804880 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B15331C9ECF for ; Sun, 15 Sep 2024 18:35:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726425313; cv=none; b=jXtYThInIhVEPT6igXmY0/c3dL5/sIxEYsaWt9rcrDl5kRTdiA1ad4aYJ9BeE8NSdbz/OWSEuWD4vXwgLx7O1hZylpdFkMkUQUW/JoiAn8rXG7HKnc/SLm7anyIWV4++xNV4EtPzDYqbzZw6AbEcmpgyxNxHwIyktyYzMyTBq7Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726425313; c=relaxed/simple; bh=uxyOVT2We6RxS5pitF5CRwoCJA7t9IkhBPykbJPGItc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bZqunGjOQZD+lyPDNsxORLJYdoAI7n+zIYBYRLNSzkgZMtmjEYbqN9bumH8xhhJQ/F8HcH5i3hZlYY249XeQ4WsgYzqPdjqnVteX+JzexAXnPXuTi+IBOHYQaOrJF33yf3sBMrN8ZCa0hwP5hdW2DFQxa4FIzLtjvOWSWHCtcE0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ORMi8w+x; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ORMi8w+x" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-206e614953aso37824915ad.1 for ; Sun, 15 Sep 2024 11:35:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726425310; x=1727030110; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k8eKKClr8hTX6nllVRY4mm+euC3yi1At6P9UopPGjNo=; b=ORMi8w+xlhy1GTf6G64n4D/r4ARtT9InjrWko9VDopMQPI/kUDx9PONuBNti5+hxnO zMsEBJIb6mA0q0jl2S/jixhWw+fBjTxDseiPXX4cnSiOEcwsNrDfn+iqLKz91O8YJzul xFpoWmPwKVjBmMPF60gkoVZGXoGTq4yRI/OKVPQdVlcerQVtJvk2QKmHkQba9ikLDeuE Jtmvy0iuzlf4GyvQhBAID5t0A/AuxCrxRPskMXZkj+U79Py+ly1G6w53K7srL13REu4V FlioqOkpyKgA9P8zj5x1eu5kUnHv10KscdRcPvIO3+Co0vMIqHwAxD0bHMKQ7mTRaVPs wWpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726425310; x=1727030110; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k8eKKClr8hTX6nllVRY4mm+euC3yi1At6P9UopPGjNo=; b=RnYEe4A84Q+j3NsGIwCFYlkb/+ubs+5bmSN8IXY5iM48TXItjQtH6eIN0uecFqLpa8 vdK+BfF909SQYFSQ+n9PtPiizoAYnrrjaUIL0uQIXY2UgOWp9r2akJlTF/fBO9ZCJLrP hXDBCy8VP2Nfxnu9UWWHUipj4eVWIFpotp+s5Hr+bsNXRVpwwFZfpeBaUTv4t47acwoA ElUm00d1GBiYI8KoLS0gKd1SvRwhdxEqFie0wAirt/5PTRYhqOnf5dY5sIrP1QpiVe3t QiotzrDBDBYnQLbN2XnlqPDEMEtYgeIlzbeGcM8I00HncSiIVM/HrwcwttZXohvRSOB8 b19A== X-Gm-Message-State: AOJu0YzzuogoSMzIAy+FEmug3aIuDvfbvb0kKaqFa/9LhKl4RPcIG9bo f5USw+mLB4POvrhu4NA7vj0yGn5dyQX3Gj7C9N9XcRFZpv4r+neY6NdXJIb+ X-Google-Smtp-Source: AGHT+IERlGyinzWkYeJZs6CL7cbjZaj2sacdNUYLb4P0s//oRjUVqjuIKKypmeTYSCNVv0ctXgkQQw== X-Received: by 2002:a17:903:230e:b0:205:937f:3ac6 with SMTP id d9443c01a7336-2076e31f03fmr190893975ad.1.1726425310008; Sun, 15 Sep 2024 11:35:10 -0700 (PDT) Received: from JRT-PC.. ([202.166.44.78]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7db498de2d5sm2358874a12.15.2024.09.15.11.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Sep 2024 11:35:09 -0700 (PDT) From: James Raphael Tiovalen To: kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: andrew.jones@linux.dev, atishp@rivosinc.com, cade.richard@berkeley.edu, James Raphael Tiovalen Subject: [kvm-unit-tests PATCH v4 2/3] riscv: sbi: Provide entry point for HSM tests Date: Mon, 16 Sep 2024 02:34:58 +0800 Message-ID: <20240915183459.52476-3-jamestiotio@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240915183459.52476-1-jamestiotio@gmail.com> References: <20240915183459.52476-1-jamestiotio@gmail.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The HSM tests will need to test HSM start and resumption from HSM suspend. Provide an entry point written in assembly that doesn't use a stack for this. Results of the test are written to global per-hart arrays to be checked by the main SBI HSM test function. The started/resumed hart does its checks and then just loops until it gets a signal from the main SBI HSM test function to invoke HSM stop. Signed-off-by: James Raphael Tiovalen Co-developed-by: Andrew Jones Signed-off-by: Andrew Jones --- riscv/Makefile | 3 +- riscv/sbi-tests.h | 10 +++++++ riscv/sbi-asm.S | 71 +++++++++++++++++++++++++++++++++++++++++++++++ riscv/sbi.c | 5 ++++ 4 files changed, 88 insertions(+), 1 deletion(-) create mode 100644 riscv/sbi-tests.h create mode 100644 riscv/sbi-asm.S diff --git a/riscv/Makefile b/riscv/Makefile index 2ee7c5bb..4676d262 100644 --- a/riscv/Makefile +++ b/riscv/Makefile @@ -43,6 +43,7 @@ cflatobjs += lib/riscv/timer.o ifeq ($(ARCH),riscv32) cflatobjs += lib/ldiv32.o endif +cflatobjs += riscv/sbi-asm.o ######################################## @@ -80,7 +81,7 @@ CFLAGS += -mcmodel=medany CFLAGS += -std=gnu99 CFLAGS += -ffreestanding CFLAGS += -O2 -CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib +CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib -I $(SRCDIR)/riscv asm-offsets = lib/riscv/asm-offsets.h include $(SRCDIR)/scripts/asm-offsets.mak diff --git a/riscv/sbi-tests.h b/riscv/sbi-tests.h new file mode 100644 index 00000000..f5cc8635 --- /dev/null +++ b/riscv/sbi-tests.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _RISCV_SBI_TESTS_H_ +#define _RISCV_SBI_TESTS_H_ + +#define SBI_HSM_TEST_DONE (1 << 0) +#define SBI_HSM_TEST_HARTID_A1 (1 << 1) +#define SBI_HSM_TEST_SATP (1 << 2) +#define SBI_HSM_TEST_SIE (1 << 3) + +#endif /* _RISCV_SBI_TESTS_H_ */ diff --git a/riscv/sbi-asm.S b/riscv/sbi-asm.S new file mode 100644 index 00000000..f165f9da --- /dev/null +++ b/riscv/sbi-asm.S @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Helper assembly code routines for RISC-V SBI extension tests. + * + * Copyright (C) 2024, James Raphael Tiovalen + */ +#define __ASSEMBLY__ +#include + +#include "sbi-tests.h" + +.section .text + +/* + * sbi_hsm_check + * a0 and a1 are set by SBI HSM start/suspend + * s1 is the address of the results array + * Doesn't return. + * + * This function is only called from HSM start and on resumption + * from HSM suspend which means we can do whatever we like with + * all registers. So, to avoid complicated register agreements with + * other assembly functions called, we just always use the saved + * registers for anything that should be maintained across calls. + */ +#define RESULTS_ARRAY s1 +#define RESULTS_MAP s2 +#define CPU_INDEX s3 +.balign 4 +sbi_hsm_check: + li RESULTS_MAP, 0 + bne a0, a1, 1f + ori RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_HARTID_A1 +1: csrr t0, CSR_SATP + bnez t0, 2f + ori RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_SATP +2: csrr t0, CSR_SSTATUS + andi t0, t0, SR_SIE + bnez t0, 3f + ori RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_SIE +3: call hartid_to_cpu + mv CPU_INDEX, a0 + li t0, -1 + bne CPU_INDEX, t0, 5f +4: pause + j 4b +5: ori RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_DONE + add t0, RESULTS_ARRAY, CPU_INDEX + sb RESULTS_MAP, 0(t0) + la t1, sbi_hsm_stop_hart + add t1, t1, CPU_INDEX +6: lb t0, 0(t1) + pause + beqz t0, 6b + li a7, 0x48534d /* SBI_EXT_HSM */ + li a6, 1 /* SBI_EXT_HSM_HART_STOP */ + ecall +7: pause + j 7b + +.balign 4 +.global sbi_hsm_check_hart_start +sbi_hsm_check_hart_start: + la RESULTS_ARRAY, sbi_hsm_hart_start_checks + j sbi_hsm_check + +.balign 4 +.global sbi_hsm_check_non_retentive_suspend +sbi_hsm_check_non_retentive_suspend: + la RESULTS_ARRAY, sbi_hsm_non_retentive_hart_suspend_checks + j sbi_hsm_check diff --git a/riscv/sbi.c b/riscv/sbi.c index a7abc08c..d4dfd48e 100644 --- a/riscv/sbi.c +++ b/riscv/sbi.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -288,6 +289,10 @@ static void check_time(void) report_prefix_popn(2); } +unsigned char sbi_hsm_stop_hart[NR_CPUS]; +unsigned char sbi_hsm_hart_start_checks[NR_CPUS]; +unsigned char sbi_hsm_non_retentive_hart_suspend_checks[NR_CPUS]; + #define DBCN_WRITE_TEST_STRING "DBCN_WRITE_TEST_STRING\n" #define DBCN_WRITE_BYTE_TEST_BYTE ((u8)'a') From patchwork Sun Sep 15 18:34:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Raphael Tiovalen X-Patchwork-Id: 13804881 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D2F02F4A for ; Sun, 15 Sep 2024 18:35:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726425315; cv=none; b=vFPmzQ7GTsexDo7d9luC5h+sfTnHpj6dJXpREk+OmpAhG5tMXB9YTgjbS5DEtw0B50+wpojidOxaBODSOtBH/HlClPEDs1j+wvSQyVQE1eH/ID0KTgDt5SFnZ6Z7xez/vAciQV6xvW0c7gwqB2B9McZCU/wjk8h0vS2tv0GtPVQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726425315; c=relaxed/simple; bh=y5a/L3//Ix8/omuUUGPoTFGlxkSLNhdEc+C4z4Ab4aA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rRCPBt5DRhBfx3/ShvpaqBq36OyDJAP7RPLJcwQfwc3Hfd4XUZMwdKJFoHPcyTICSk8vl6fUPYUuxDdxGvOh1F3wSp2lb6SBaAKoG0UmHVVlNo/S88fXTqv65D2y8CzQUQRYZV/uQ28w7Uz4Cdgh+sEjFShhUW+Rlx/PFSKb2fc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IKqQT8Jr; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IKqQT8Jr" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-718f4fd89e5so3473412b3a.0 for ; Sun, 15 Sep 2024 11:35:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726425312; x=1727030112; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VecAxz5qXutxIU5JiDT/QHrCKYj1plrVx+KdpKdPfqM=; b=IKqQT8JreclJqdUgCmky1TTzjrabdW9o+qg4ttnPTsLXA6/KGwtN4CkiX+txNxznTW q7QeXvR4sBRYn/qnzxZCFzEM049EHFB8773MzFM2i7velw/jn5rd9CTtRGnKbwtmpYEv bh7BZDygJue0/T3gBQdxZu67H8sWVqSRE0NfEtHcEAOl2uGYu8fxiFp7fd3Qe0+fxOk/ e1Llh/bnLwuqhUqTMjN6rmNndVkgFyoxYDDNW94HO+/fQfbwv51YFnGsSi6D+yHA9q3g 3YuSRYTk0NAa74FwedkJICJswkUawp6dndWamgAZ+0Ee1E9Zpe3uuclf7prJxib9eehh HIIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726425312; x=1727030112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VecAxz5qXutxIU5JiDT/QHrCKYj1plrVx+KdpKdPfqM=; b=VUmnhodWX3DAmZVjranIbnp5WAKMgAfmvAMc6d4Og9Mko9kaK+lJzyOyEIN+xj4YK7 YEYcVif30t+3Nq0YIxtjW+y7M16NJ/CmWVqfAxhO6WOgDdqBFHWfi49Ba6Ng+1ANEnrG z5Rlew/+XSJyB3ujC8HhOdKTVzq+g1DLsbDO+voCeYfQa7Dge5NXBwX26V+CIrGnx5eG nap+4IHnsn8aT6aCu4/bHkTkPuCZ8V0XBtfQCIsWwkNs73oBRVdRYmdY0BjPZCKxZxT1 ShrPWWXfIzGtjwhZUaAr/P+lR0KvT+g46xtQfrKpDchshcWUslKkpsaJxK5DoiNObCGy L9lw== X-Gm-Message-State: AOJu0YzgmESAjwP/HJYiwyttDP0PzqBgdUcNY0Wq9Ec7O2aIF6vQE2pG gWt1580cIiy6ZMuaSsnrylpAIqHSRi+MyA3RZoB46XOe/KHsEACyarL6PPBx X-Google-Smtp-Source: AGHT+IFDDmBePeYnMyo0qqm/QAnZwsdW39acBN/r8mR93xLAcMWihKaLxbJTmisw9J9rL++RFGkA+w== X-Received: by 2002:a05:6a20:d523:b0:1d2:bbf2:4c0e with SMTP id adf61e73a8af0-1d2bbf24c8fmr6313164637.43.1726425312343; Sun, 15 Sep 2024 11:35:12 -0700 (PDT) Received: from JRT-PC.. ([202.166.44.78]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7db498de2d5sm2358874a12.15.2024.09.15.11.35.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Sep 2024 11:35:11 -0700 (PDT) From: James Raphael Tiovalen To: kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: andrew.jones@linux.dev, atishp@rivosinc.com, cade.richard@berkeley.edu, James Raphael Tiovalen Subject: [kvm-unit-tests PATCH v4 3/3] riscv: sbi: Add tests for HSM extension Date: Mon, 16 Sep 2024 02:34:59 +0800 Message-ID: <20240915183459.52476-4-jamestiotio@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240915183459.52476-1-jamestiotio@gmail.com> References: <20240915183459.52476-1-jamestiotio@gmail.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add some tests for all of the HSM extension functions. These tests ensure that the HSM extension functions follow the behavior as described in the SBI specification. Signed-off-by: James Raphael Tiovalen --- riscv/sbi.h | 10 + riscv/sbi.c | 561 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 571 insertions(+) create mode 100644 riscv/sbi.h diff --git a/riscv/sbi.h b/riscv/sbi.h new file mode 100644 index 00000000..e8625cb1 --- /dev/null +++ b/riscv/sbi.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _RISCV_SBI_H_ +#define _RISCV_SBI_H_ + +#define SBI_HSM_TEST_DONE (1 << 0) +#define SBI_HSM_TEST_SATP (1 << 1) +#define SBI_HSM_TEST_SIE (1 << 2) +#define SBI_HSM_TEST_HARTID_A1 (1 << 3) + +#endif /* _RISCV_SBI_H_ */ diff --git a/riscv/sbi.c b/riscv/sbi.c index d4dfd48e..fab0091b 100644 --- a/riscv/sbi.c +++ b/riscv/sbi.c @@ -6,6 +6,8 @@ */ #include #include +#include +#include #include #include #include @@ -16,11 +18,13 @@ #include #include #include +#include #include #include #include #include #include +#include #define HIGH_ADDR_BOUNDARY ((phys_addr_t)1 << 32) @@ -47,6 +51,11 @@ static struct sbiret sbi_dbcn_write_byte(uint8_t byte) return sbi_ecall(SBI_EXT_DBCN, SBI_EXT_DBCN_CONSOLE_WRITE_BYTE, byte, 0, 0, 0, 0, 0); } +static struct sbiret sbi_hart_suspend(uint32_t suspend_type, unsigned long resume_addr, unsigned long opaque) +{ + return sbi_ecall(SBI_EXT_HSM, SBI_EXT_HSM_HART_SUSPEND, suspend_type, resume_addr, opaque, 0, 0, 0); +} + static void split_phys_addr(phys_addr_t paddr, unsigned long *hi, unsigned long *lo) { *lo = (unsigned long)paddr; @@ -434,6 +443,557 @@ static void check_dbcn(void) report_prefix_popn(2); } +unsigned char sbi_hsm_stop_hart[NR_CPUS]; +unsigned char sbi_hsm_hart_start_checks[NR_CPUS]; +unsigned char sbi_hsm_non_retentive_hart_suspend_checks[NR_CPUS]; +cpumask_t sbi_hsm_started_hart_checks; +cpumask_t sbi_hsm_invalid_hartid_checks; +static bool hsm_timer_fired; +extern void sbi_hsm_check_hart_start(void); +extern void sbi_hsm_check_non_retentive_suspend(void); + +static void hsm_timer_irq_handler(struct pt_regs *regs) +{ + hsm_timer_fired = true; + timer_stop(); +} + +static void hsm_timer_setup(void) +{ + install_irq_handler(IRQ_S_TIMER, hsm_timer_irq_handler); + local_irq_enable(); + timer_irq_enable(); +} + +static void hsm_timer_teardown(void) +{ + timer_irq_disable(); + local_irq_disable(); + install_irq_handler(IRQ_S_TIMER, NULL); +} + +static void hart_empty_fn(void *data) {} + +static void hart_execute(void *data) +{ + struct sbiret ret; + unsigned long hartid = current_thread_info()->hartid; + int me = smp_processor_id(); + + ret = sbi_hart_start(hartid, virt_to_phys(&hart_empty_fn), 0); + + if (ret.error == SBI_ERR_ALREADY_AVAILABLE) + cpumask_set_cpu(me, &sbi_hsm_started_hart_checks); + + ret = sbi_hart_start(ULONG_MAX, virt_to_phys(&hart_empty_fn), 0); + + if (ret.error == SBI_ERR_INVALID_PARAM) + cpumask_set_cpu(me, &sbi_hsm_invalid_hartid_checks); +} + +static void hart_retentive_suspend(void *data) +{ + unsigned long hartid = current_thread_info()->hartid; + struct sbiret ret = sbi_hart_suspend(SBI_EXT_HSM_HART_SUSPEND_RETENTIVE, 0, 0); + + if (ret.error) + report_fail("failed to retentive suspend hart %ld (error=%ld)", hartid, ret.error); +} + +static void hart_non_retentive_suspend(void *data) +{ + unsigned long hartid = current_thread_info()->hartid; + + /* Set opaque as hartid so that we can check a0 == a1, ensuring that a0 is hartid and a1 is opaque */ + struct sbiret ret = sbi_hart_suspend(SBI_EXT_HSM_HART_SUSPEND_NON_RETENTIVE, + virt_to_phys(&sbi_hsm_check_non_retentive_suspend), hartid); + + report_fail("failed to non-retentive suspend hart %ld (error=%ld)", hartid, ret.error); +} + +static void hart_wait_on_status(unsigned long hartid, enum sbi_ext_hsm_sid status) +{ + struct sbiret ret = sbi_hart_get_status(hartid); + + while (!ret.error && ret.value == status && !hsm_timer_fired) { + cpu_relax(); + ret = sbi_hart_get_status(hartid); + } + + if (hsm_timer_fired) + report_info("timer fired while waiting on status %u for hart %ld", status, hartid); + else if (ret.error) + report_fail("got %ld while waiting on status %u for hart %ld\n", ret.error, status, hartid); +} + +static void check_hsm(void) +{ + struct sbiret ret; + unsigned long hartid; + cpumask_t secondary_cpus_mask, hsm_start, hsm_stop, hsm_suspend, hsm_resume, hsm_check; + int cpu, me = smp_processor_id(); + int max_cpu = getenv("SBI_HSM_MAX_CPU") ? strtol(getenv("SBI_HSM_MAX_CPU"), NULL, 0) : INT_MAX; + unsigned long hsm_timer_duration = getenv("SBI_HSM_TIMER_DURATION") + ? strtol(getenv("SBI_HSM_TIMER_DURATION"), NULL, 0) : 200000; + + max_cpu = MIN(max_cpu, nr_cpus - 1); + + cpumask_copy(&secondary_cpus_mask, &cpu_present_mask); + cpumask_clear_cpu(me, &secondary_cpus_mask); + for_each_cpu(cpu, &secondary_cpus_mask) + if (cpu > max_cpu) + cpumask_clear_cpu(cpu, &secondary_cpus_mask); + + report_prefix_push("hsm"); + + if (!sbi_probe(SBI_EXT_HSM)) { + report_skip("hsm extension not available"); + report_prefix_pop(); + return; + } + + report_prefix_push("hart_get_status"); + + hartid = current_thread_info()->hartid; + ret = sbi_hart_get_status(hartid); + + if (ret.error) { + report_fail("failed to get status of current hart (error=%ld)", ret.error); + report_prefix_popn(2); + return; + } else if (ret.value != SBI_EXT_HSM_STARTED) { + report_fail("current hart is not started (ret.value=%ld)", ret.value); + report_prefix_popn(2); + return; + } + + report_pass("status of current hart is started"); + + report_prefix_pop(); + + if (max_cpu + 1 < 2) { + report_skip("no other cpus to run the remaining hsm tests on"); + report_prefix_pop(); + return; + } + + /* This is necessary since we do not choose which cpu the boot hart will run on */ + if (me > max_cpu) + max_cpu++; + + report_prefix_push("hart_start"); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + /* Set opaque as hartid so that we can check a0 == a1, ensuring that a0 is hartid and a1 is opaque */ + ret = sbi_hart_start(hartid, virt_to_phys(&sbi_hsm_check_hart_start), hartid); + if (ret.error) { + report_fail("failed to start test hart %ld (error=%ld)", hartid, ret.error); + report_prefix_popn(2); + return; + } + } + + cpumask_clear(&hsm_start); + hsm_timer_setup(); + timer_start(hsm_timer_duration); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + hart_wait_on_status(hartid, SBI_EXT_HSM_STOPPED); + if (hsm_timer_fired) + break; + hart_wait_on_status(hartid, SBI_EXT_HSM_START_PENDING); + if (hsm_timer_fired) + break; + ret = sbi_hart_get_status(hartid); + if (ret.error) + report_info("hart %ld get status failed (error=%ld)", hartid, ret.error); + else if (ret.value != SBI_EXT_HSM_STARTED) + report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value); + else + cpumask_set_cpu(cpu, &hsm_start); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts started"); + report_prefix_popn(2); + return; + } + + report(cpumask_weight(&hsm_start) == max_cpu, "all secondary harts started"); + + cpumask_clear(&hsm_check); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + + while (!(READ_ONCE(sbi_hsm_hart_start_checks[cpu]) & SBI_HSM_TEST_DONE) && !hsm_timer_fired) + cpu_relax(); + + if (hsm_timer_fired) + break; + + if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_SATP)) + report_info("satp is not zero for test hart %ld", hartid); + else if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_SIE)) + report_info("sstatus.SIE is not zero for test hart %ld", hartid); + else if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_HARTID_A1)) + report_info("either a0 or a1 is not hartid for test hart %ld", hartid); + else + cpumask_set_cpu(cpu, &hsm_check); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts are done with checks"); + report_prefix_popn(2); + return; + } + + timer_stop(); + + report(cpumask_weight(&hsm_check) == max_cpu, + "all secondary harts have expected register values after hart start"); + + report_prefix_pop(); + + report_prefix_push("hart_stop"); + + memset(sbi_hsm_stop_hart, 1, sizeof(sbi_hsm_stop_hart)); + + cpumask_clear(&hsm_stop); + hsm_timer_fired = false; + timer_start(hsm_timer_duration); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED); + if (hsm_timer_fired) + break; + hart_wait_on_status(hartid, SBI_EXT_HSM_STOP_PENDING); + if (hsm_timer_fired) + break; + ret = sbi_hart_get_status(hartid); + if (ret.error) + report_info("hart %ld get status failed (error=%ld)", hartid, ret.error); + else if (ret.value != SBI_EXT_HSM_STOPPED) + report_info("hart %ld status is not 'stopped' (ret.value=%ld)", hartid, ret.value); + else + cpumask_set_cpu(cpu, &hsm_stop); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts stopped"); + report_prefix_popn(2); + return; + } + + timer_stop(); + + report(cpumask_weight(&hsm_stop) == max_cpu, "all secondary harts stopped"); + + /* Reset the stop flags so that we can reuse them after suspension tests */ + memset(sbi_hsm_stop_hart, 0, sizeof(sbi_hsm_stop_hart)); + + report_prefix_pop(); + + report_prefix_push("hart_start"); + + on_cpumask_async(&secondary_cpus_mask, hart_execute, NULL); + + cpumask_clear(&hsm_start); + hsm_timer_fired = false; + timer_start(hsm_timer_duration); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + hart_wait_on_status(hartid, SBI_EXT_HSM_STOPPED); + if (hsm_timer_fired) + break; + hart_wait_on_status(hartid, SBI_EXT_HSM_START_PENDING); + if (hsm_timer_fired) + break; + ret = sbi_hart_get_status(hartid); + if (ret.error) + report_info("hart %ld get status failed (error=%ld)", hartid, ret.error); + else if (ret.value != SBI_EXT_HSM_STARTED) + report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value); + else + cpumask_set_cpu(cpu, &hsm_start); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts started"); + report_prefix_popn(2); + return; + } + + report(cpumask_weight(&hsm_start) == max_cpu, "all secondary harts started"); + + while (cpumask_weight(&cpu_idle_mask) != max_cpu && !hsm_timer_fired) + cpu_relax(); + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts started"); + report_prefix_popn(2); + return; + } + + timer_stop(); + + report(cpumask_weight(&cpu_idle_mask) == max_cpu, + "all secondary harts successfully executed code after start"); + report(cpumask_weight(&cpu_online_mask) == max_cpu + 1, "all secondary harts online"); + report(cpumask_weight(&sbi_hsm_started_hart_checks) == max_cpu, + "all secondary harts are already started"); + report(cpumask_weight(&sbi_hsm_invalid_hartid_checks) == max_cpu, + "all secondary harts refuse to start with invalid hartid"); + + report_prefix_pop(); + + report_prefix_push("hart_suspend"); + + if (!sbi_probe(SBI_EXT_IPI)) { + hsm_timer_teardown(); + report_skip("skipping suspension tests since ipi extension is unavailable"); + report_prefix_popn(2); + return; + } + + on_cpumask_async(&secondary_cpus_mask, hart_retentive_suspend, NULL); + + cpumask_clear(&hsm_suspend); + hsm_timer_fired = false; + timer_start(hsm_timer_duration); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED); + if (hsm_timer_fired) + break; + hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING); + if (hsm_timer_fired) + break; + ret = sbi_hart_get_status(hartid); + if (ret.error) + report_info("hart %ld get status failed (error=%ld)", hartid, ret.error); + else if (ret.value != SBI_EXT_HSM_SUSPENDED) + report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value); + else + cpumask_set_cpu(cpu, &hsm_suspend); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts retentive suspended"); + report_prefix_popn(2); + return; + } + + timer_stop(); + + report(cpumask_weight(&hsm_suspend) == max_cpu, "all secondary harts retentive suspended"); + + ret = sbi_send_ipi_cpumask(&secondary_cpus_mask); + + if (!ret.error) { + cpumask_clear(&hsm_resume); + hsm_timer_fired = false; + timer_start(hsm_timer_duration); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED); + if (hsm_timer_fired) + break; + hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING); + if (hsm_timer_fired) + break; + ret = sbi_hart_get_status(hartid); + if (ret.error) + report_info("hart %ld get status failed (error=%ld)", hartid, ret.error); + else if (ret.value != SBI_EXT_HSM_STARTED) + report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value); + else + cpumask_set_cpu(cpu, &hsm_resume); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts retentive resumed"); + report_prefix_popn(2); + return; + } + + report(cpumask_weight(&hsm_resume) == max_cpu, "all secondary harts retentive resumed"); + + while (cpumask_weight(&cpu_idle_mask) != max_cpu && !hsm_timer_fired) + cpu_relax(); + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts retentive resumed"); + report_prefix_popn(2); + return; + } + + timer_stop(); + + report(cpumask_weight(&cpu_idle_mask) == max_cpu, + "all secondary harts successfully executed code after retentive suspend"); + report(cpumask_weight(&cpu_online_mask) == max_cpu + 1, + "all secondary harts online"); + } + + on_cpumask_async(&secondary_cpus_mask, hart_non_retentive_suspend, NULL); + + cpumask_clear(&hsm_suspend); + hsm_timer_fired = false; + timer_start(hsm_timer_duration); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED); + if (hsm_timer_fired) + break; + hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING); + if (hsm_timer_fired) + break; + ret = sbi_hart_get_status(hartid); + if (ret.error) + report_info("hart %ld get status failed (error=%ld)", hartid, ret.error); + else if (ret.value != SBI_EXT_HSM_SUSPENDED) + report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value); + else + cpumask_set_cpu(cpu, &hsm_suspend); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts non-retentive suspended"); + report_prefix_popn(2); + return; + } + + timer_stop(); + + report(cpumask_weight(&hsm_suspend) == max_cpu, "all secondary harts non-retentive suspended"); + + ret = sbi_send_ipi_cpumask(&secondary_cpus_mask); + + if (!ret.error) { + cpumask_clear(&hsm_resume); + hsm_timer_fired = false; + timer_start(hsm_timer_duration); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED); + if (hsm_timer_fired) + break; + hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING); + if (hsm_timer_fired) + break; + ret = sbi_hart_get_status(hartid); + if (ret.error) + report_info("hart %ld get status failed (error=%ld)", hartid, ret.error); + else if (ret.value != SBI_EXT_HSM_STARTED) + report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value); + else + cpumask_set_cpu(cpu, &hsm_resume); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts non-retentive resumed"); + report_prefix_popn(2); + return; + } + + report(cpumask_weight(&hsm_resume) == max_cpu, "all secondary harts non-retentive resumed"); + + cpumask_clear(&hsm_check); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + + while (!((READ_ONCE(sbi_hsm_non_retentive_hart_suspend_checks[cpu])) & SBI_HSM_TEST_DONE) + && !hsm_timer_fired) + cpu_relax(); + + if (hsm_timer_fired) + break; + + if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SATP)) + report_info("satp is not zero for test hart %ld", hartid); + else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SIE)) + report_info("sstatus.SIE is not zero for test hart %ld", hartid); + else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_HARTID_A1)) + report_info("either a0 or a1 is not hartid for test hart %ld", hartid); + else + cpumask_set_cpu(cpu, &hsm_check); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts are done with checks"); + report_prefix_popn(2); + return; + } + + timer_stop(); + + report(cpumask_weight(&hsm_check) == max_cpu, + "all secondary harts have expected register values after non-retentive resume"); + + report_prefix_pop(); + + report_prefix_push("hart_stop"); + + memset(sbi_hsm_stop_hart, 1, sizeof(sbi_hsm_stop_hart)); + + cpumask_clear(&hsm_stop); + hsm_timer_fired = false; + timer_start(hsm_timer_duration); + + for_each_cpu(cpu, &secondary_cpus_mask) { + hartid = cpus[cpu].hartid; + hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED); + if (hsm_timer_fired) + break; + hart_wait_on_status(hartid, SBI_EXT_HSM_STOP_PENDING); + if (hsm_timer_fired) + break; + ret = sbi_hart_get_status(hartid); + if (ret.error) + report_info("hart %ld get status failed (error=%ld)", hartid, ret.error); + else if (ret.value != SBI_EXT_HSM_STOPPED) + report_info("hart %ld status is not 'stopped' (ret.value=%ld)", hartid, ret.value); + else + cpumask_set_cpu(cpu, &hsm_stop); + } + + if (hsm_timer_fired) { + hsm_timer_teardown(); + report_fail("hsm timer fired before all secondary harts stopped after resumption"); + report_prefix_popn(2); + return; + } + + timer_stop(); + + report(cpumask_weight(&hsm_stop) == max_cpu, "all secondary harts stopped after resumption"); + } + + hsm_timer_teardown(); + report_prefix_popn(2); +} + int main(int argc, char **argv) { if (argc > 1 && !strcmp(argv[1], "-h")) { @@ -444,6 +1004,7 @@ int main(int argc, char **argv) report_prefix_push("sbi"); check_base(); check_time(); + check_hsm(); check_dbcn(); return report_summary();