From patchwork Wed Aug 18 00:12:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12442401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F77DC4338F for ; Wed, 18 Aug 2021 00:15:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5305D60EBC for ; Wed, 18 Aug 2021 00:15:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5305D60EBC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=j+GIOPOhyA5cXaSHsVXYmCz6zmT6AHprFxtDPwBioZI=; b=fIu5GiINkL+PME 5nx8j/uq2g/86umvJEx5abFDndt/fNZ6xk8PZt5xmBhYeswdUC7grVW/ib1QQGSnQEQVRosi1LjCj mK7oSwXC/ZTbMWSA3Xl88JAeo8hR2c1Th79hdfmtkMYR0/HOTR6Ia4Rz7luKTXT6gMB9Z7+BQAMSV 1JLyeD80XA6M2+mPZST7L1DBe4gYzqjvB+KiHSRmuQj83sK8oa2uLa4R4gfF69CsQDHc9muRwUeEL von0V93sV8wGtsfrzsrSyLBQbL6X2wXjh6lKnWhBsO96tMxX6FzX3ZjuW01KoSCpXQZxVSTvQVVfg kw7a97JSwPftBXtsDKog==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mG9Cy-003knr-4h; Wed, 18 Aug 2021 00:13:28 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mG9C5-003kWL-Tm for linux-arm-kernel@lists.infradead.org; Wed, 18 Aug 2021 00:12:35 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id d69-20020a25e6480000b02904f4a117bd74so940793ybh.17 for ; Tue, 17 Aug 2021 17:12:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=HGsy1AYzAaNzOmCNqthC1tCvP/rp7O5tcVnL7Zz8tBI=; b=sh+QVsG0ol48I0ldcdsbPdihtJRoNYZPe0D2QiktQlpO0cKb5nXByr86R+F1H8DvLa betvTNViADCpa2luSEIvXPH5Tm/3E/6pbWXlndcN5d2jGZuAgR9zTROn/eA+cgT/+AJq mUFR/B4LKLSlB7Pd9Px3yOreOSwSXzVr7/o1BEs7xvewKCyRqqCyuksorGhU6Nuc9cBN V/nM7cq6QqTmJEM+a/L8JscwoAxz6CA4MUNwpv1pJkJSU9IFycWQDD1jE95g5EcZOY6z phLtJF9xi4TR7XKdoU09nlD54YfMfylCa9FlWIHeq58yCT/f9K1eaok/zfAbCBch1u+n 7bEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=HGsy1AYzAaNzOmCNqthC1tCvP/rp7O5tcVnL7Zz8tBI=; b=gqHgy5Mf9iJfrlvlMh9AYBzCVPT7riiNuK5vvWqpvnRqPEGUmEo1u/mGBnYP7UUnif 5G2gBg5m13mH/CIoe95Tkc3wYL7dsA6tolEOULIGKZkEEW2ospMEpj4oGVCsFyAK+mgV athj/L/zk6qwaNAG2zFEgvcv0J/EAlDLCAd8cDchdopWlQmqeO25KrnoYnrRnEXbDsjN 4sB9HAc07CdbTZb869o+kG1klo0LIPIV8O6TXbEv6w0sRgvU9jHEv/k16VnN0HVzg/UR FrrQwRrMaSgb5/AEb7Sl4/Q7DoyNqyNppsRe2BQRBrpk3AaknV/rvI3+PSIDcYramur8 8G5g== X-Gm-Message-State: AOAM533EgP+p7nREZkJle5qrtjGk7uwCmPoX6Cfhkc+AYphR1f+zqnhB vsXZHFcZ0qcC5EJ/JnV/EzZbOWkpTpQ= X-Google-Smtp-Source: ABdhPJxKuNoi8BurtLgbuppXh+jBMCzbufP37yZkuLi1u+HzA7x/QRXt4AsLOGUGibA7aBEUElEO9ttIYL8= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:158e:55a:74dd:5197]) (user=seanjc job=sendgmr) by 2002:a25:db82:: with SMTP id g124mr7742228ybf.46.1629245552432; Tue, 17 Aug 2021 17:12:32 -0700 (PDT) Date: Tue, 17 Aug 2021 17:12:09 -0700 In-Reply-To: <20210818001210.4073390-1-seanjc@google.com> Message-Id: <20210818001210.4073390-5-seanjc@google.com> Mime-Version: 1.0 References: <20210818001210.4073390-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.rc1.237.g0d66db33f3-goog Subject: [PATCH 4/5] KVM: selftests: Add a test for KVM_RUN+rseq to detect task migration bugs From: Sean Christopherson To: Russell King , Catalin Marinas , Will Deacon , Guo Ren , Thomas Bogendoerfer , Michael Ellerman , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Oleg Nesterov , Steven Rostedt , Ingo Molnar , Thomas Gleixner , Peter Zijlstra , Andy Lutomirski , Mathieu Desnoyers , "Paul E. McKenney" , Boqun Feng , Paolo Bonzini , Shuah Khan Cc: Benjamin Herrenschmidt , Paul Mackerras , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org, Peter Foley , Shakeel Butt , Sean Christopherson , Ben Gardon X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210817_171234_025686_786B3940 X-CRM114-Status: GOOD ( 23.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a test to verify an rseq's CPU ID is updated correctly if the task is migrated while the kernel is handling KVM_RUN. This is a regression test for a bug introduced by commit 72c3c0fe54a3 ("x86/kvm: Use generic xfer to guest work function"), where TIF_NOTIFY_RESUME would be cleared by KVM without updating rseq, leading to a stale CPU ID and other badness. Signed-off-by: Sean Christopherson --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 3 + tools/testing/selftests/kvm/rseq_test.c | 131 ++++++++++++++++++++++++ 3 files changed, 135 insertions(+) create mode 100644 tools/testing/selftests/kvm/rseq_test.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index 0709af0144c8..6d031ff6b68e 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -47,6 +47,7 @@ /kvm_page_table_test /memslot_modification_stress_test /memslot_perf_test +/rseq_test /set_memory_region_test /steal_time /kvm_binary_stats_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 5832f510a16c..0756e79cb513 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -80,6 +80,7 @@ TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus TEST_GEN_PROGS_x86_64 += kvm_page_table_test TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test TEST_GEN_PROGS_x86_64 += memslot_perf_test +TEST_GEN_PROGS_x86_64 += rseq_test TEST_GEN_PROGS_x86_64 += set_memory_region_test TEST_GEN_PROGS_x86_64 += steal_time TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test @@ -92,6 +93,7 @@ TEST_GEN_PROGS_aarch64 += dirty_log_test TEST_GEN_PROGS_aarch64 += dirty_log_perf_test TEST_GEN_PROGS_aarch64 += kvm_create_max_vcpus TEST_GEN_PROGS_aarch64 += kvm_page_table_test +TEST_GEN_PROGS_aarch64 += rseq_test TEST_GEN_PROGS_aarch64 += set_memory_region_test TEST_GEN_PROGS_aarch64 += steal_time TEST_GEN_PROGS_aarch64 += kvm_binary_stats_test @@ -103,6 +105,7 @@ TEST_GEN_PROGS_s390x += demand_paging_test TEST_GEN_PROGS_s390x += dirty_log_test TEST_GEN_PROGS_s390x += kvm_create_max_vcpus TEST_GEN_PROGS_s390x += kvm_page_table_test +TEST_GEN_PROGS_s390x += rseq_test TEST_GEN_PROGS_s390x += set_memory_region_test TEST_GEN_PROGS_s390x += kvm_binary_stats_test diff --git a/tools/testing/selftests/kvm/rseq_test.c b/tools/testing/selftests/kvm/rseq_test.c new file mode 100644 index 000000000000..90ed535eded7 --- /dev/null +++ b/tools/testing/selftests/kvm/rseq_test.c @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: GPL-2.0-only +#define _GNU_SOURCE /* for program_invocation_short_name */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kvm_util.h" +#include "processor.h" +#include "test_util.h" + +#define VCPU_ID 0 + +static __thread volatile struct rseq __rseq = { + .cpu_id = RSEQ_CPU_ID_UNINITIALIZED, +}; + +#define RSEQ_SIG 0xdeadbeef + +static pthread_t migration_thread; +static cpu_set_t possible_mask; +static bool done; + +static void guest_code(void) +{ + for (;;) + GUEST_SYNC(0); +} + +static void sys_rseq(int flags) +{ + int r; + + r = syscall(__NR_rseq, &__rseq, sizeof(__rseq), flags, RSEQ_SIG); + TEST_ASSERT(!r, "rseq failed, errno = %d (%s)", errno, strerror(errno)); +} + +static void *migration_worker(void *ign) +{ + cpu_set_t allowed_mask; + int r, i, nr_cpus, cpu; + + CPU_ZERO(&allowed_mask); + + nr_cpus = CPU_COUNT(&possible_mask); + + for (i = 0; i < 20000; i++) { + cpu = i % nr_cpus; + if (!CPU_ISSET(cpu, &possible_mask)) + continue; + + CPU_SET(cpu, &allowed_mask); + + r = sched_setaffinity(0, sizeof(allowed_mask), &allowed_mask); + TEST_ASSERT(!r, "sched_setaffinity failed, errno = %d (%s)", errno, + strerror(errno)); + + CPU_CLR(cpu, &allowed_mask); + + usleep(10); + } + done = true; + return NULL; +} + +int main(int argc, char *argv[]) +{ + struct kvm_vm *vm; + u32 cpu, rseq_cpu; + int r; + + /* Tell stdout not to buffer its content */ + setbuf(stdout, NULL); + + r = sched_getaffinity(0, sizeof(possible_mask), &possible_mask); + TEST_ASSERT(!r, "sched_getaffinity failed, errno = %d (%s)", errno, + strerror(errno)); + + if (CPU_COUNT(&possible_mask) < 2) { + print_skip("Only one CPU, task migration not possible\n"); + exit(KSFT_SKIP); + } + + sys_rseq(0); + + /* + * Create and run a dummy VM that immediately exits to userspace via + * GUEST_SYNC, while concurrently migrating the process by setting its + * CPU affinity. + */ + vm = vm_create_default(VCPU_ID, 0, guest_code); + + pthread_create(&migration_thread, NULL, migration_worker, 0); + + while (!done) { + vcpu_run(vm, VCPU_ID); + TEST_ASSERT(get_ucall(vm, VCPU_ID, NULL) == UCALL_SYNC, + "Guest failed?"); + + cpu = sched_getcpu(); + rseq_cpu = READ_ONCE(__rseq.cpu_id); + + /* + * Verify rseq's CPU matches sched's CPU, and that sched's CPU + * is stable. This doesn't handle the case where the task is + * migrated between sched_getcpu() and reading rseq, and again + * between reading rseq and sched_getcpu(), but in practice no + * false positives have been observed, while on the other hand + * blocking migration while this thread reads CPUs messes with + * the timing and prevents hitting failures on a buggy kernel. + */ + TEST_ASSERT(rseq_cpu == cpu || cpu != sched_getcpu(), + "rseq CPU = %d, sched CPU = %d\n", rseq_cpu, cpu); + } + + pthread_join(migration_thread, NULL); + + kvm_vm_free(vm); + + sys_rseq(RSEQ_FLAG_UNREGISTER); + + return 0; +}