From patchwork Wed Apr 12 21:34:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BC19C7619A for ; Wed, 12 Apr 2023 21:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229746AbjDLVfV (ORCPT ); Wed, 12 Apr 2023 17:35:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229532AbjDLVfS (ORCPT ); Wed, 12 Apr 2023 17:35:18 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 975007A93 for ; Wed, 12 Apr 2023 14:35:16 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-54f8a3ded60so42058287b3.8 for ; Wed, 12 Apr 2023 14:35:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335316; x=1683927316; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/kXYf91+49LZ0JwpuqmoTf9tA1AbmxtwzhKhgKU0C/k=; b=VMzXtnNvKf2y7EtWP5a9jMGMHV6AhZQazpABnRuJhoLL/DhGR042EHMp9+y7Bjbauy YcuVlfZ7NxMgQ8cDodJGrSFSJoc+drBie83nQRyq+5r+/JZAEBSXa6+xNsWo6GqsXLlV YoNmFHo9NiDYLRl34+6TH8sUGpuN02NSKJEDxS9QvRaR9T25hl+V2R+hERVETVtweJsb brcFqh7l9QU2QkEiqp3mHA/SV2EY4+264uRjaPczvOldWcoBZwYWEzR1aIL75xV7eJ4b SDS0ZLj4BnzDcGSbSYJh822viTFjuVa+xJq1DDijd6MigmfOcN0bLKDriB/ZQXA/+Dkm 4oAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335316; x=1683927316; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/kXYf91+49LZ0JwpuqmoTf9tA1AbmxtwzhKhgKU0C/k=; b=JDFda4zZiLe4uUiplexCtImQLScj11ZnORXqo7XWqZGlrYv2yNiIBXa1aLRwfzlW2h v/CkDfHQCMsjMsgjnSjteseJ+Unmtzmk7Re6SDz6ec5kL0VM9K3ZhAiUY2IGvfts0lPZ 3Dq/USGpqHayao0zVu3EP2z1WL5zL2VS+Cv7V36L8Dcbs6HGiXtCSHF4FjcA/MXNhg4d SLPNHGWOAWib8M2Gpuc4dy80l+8LPv6wpA/2WfB98nT6GV++JK6Qe1/Sw8MFDUAJhJ0P zNh6+bAkSBdkTHNsO4rZwOwOj0ynobgN8BHwXlBmYiw1FOB1KkHy4Vz44ijqocuv0HfQ eHnw== X-Gm-Message-State: AAQBX9dtGOSfVyK3QIR6Mqz2vIq9IkWT0qNyw0l6mKsoiRe6kIa9hWCX r5kXwm/MKJUqsvoMAo5iLxOg2ToJN0D/rQ== X-Google-Smtp-Source: AKy350bm3qmH6tQx04VSo1u5wAk3QedosfoCcidK89MQsRENDOeokBjvXvZshnaGV/tbVkAHuy9T68uXnE4JFw== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:d60c:0:b0:b8f:54f5:89f9 with SMTP id n12-20020a25d60c000000b00b8f54f589f9mr40160ybg.0.1681335315919; Wed, 12 Apr 2023 14:35:15 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:49 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-2-amoorthy@google.com> Subject: [PATCH v3 01/22] KVM: selftests: Allow many vCPUs and reader threads per UFFD in demand paging test From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org At the moment, demand_paging_test does not support profiling/testing multiple vCPU threads concurrently faulting on a single uffd because (a) "-u" (run test in userfaultfd mode) creates a uffd for each vCPU's region, so that each uffd services a single vCPU thread. (b) "-u -o" (userfaultfd mode + overlapped vCPU memory accesses) simply doesn't work: the test tries to register the same memory to multiple uffds, causing an error. Add support for many vcpus per uffd by (1) Keeping "-u" behavior unchanged. (2) Making "-u -a" create a single uffd for all of guest memory. (3) Making "-u -o" implicitly pass "-a", solving the problem in (b). In cases (2) and (3) all vCPU threads fault on a single uffd. With multiple potentially multiple vCPU per UFFD, it makes sense to allow configuring the number reader threads per UFFD as well: add the "-r" flag to do so. Signed-off-by: Anish Moorthy Acked-by: James Houghton --- .../selftests/kvm/aarch64/page_fault_test.c | 4 +- .../selftests/kvm/demand_paging_test.c | 62 +++++++++---- .../selftests/kvm/include/userfaultfd_util.h | 18 +++- .../selftests/kvm/lib/userfaultfd_util.c | 86 +++++++++++++------ 4 files changed, 124 insertions(+), 46 deletions(-) diff --git a/tools/testing/selftests/kvm/aarch64/page_fault_test.c b/tools/testing/selftests/kvm/aarch64/page_fault_test.c index df10f1ffa20d9..3b6d228a9340d 100644 --- a/tools/testing/selftests/kvm/aarch64/page_fault_test.c +++ b/tools/testing/selftests/kvm/aarch64/page_fault_test.c @@ -376,14 +376,14 @@ static void setup_uffd(struct kvm_vm *vm, struct test_params *p, *pt_uffd = uffd_setup_demand_paging(uffd_mode, 0, pt_args.hva, pt_args.paging_size, - test->uffd_pt_handler); + 1, test->uffd_pt_handler); *data_uffd = NULL; if (test->uffd_data_handler) *data_uffd = uffd_setup_demand_paging(uffd_mode, 0, data_args.hva, data_args.paging_size, - test->uffd_data_handler); + 1, test->uffd_data_handler); } static void free_uffd(struct test_desc *test, struct uffd_desc *pt_uffd, diff --git a/tools/testing/selftests/kvm/demand_paging_test.c b/tools/testing/selftests/kvm/demand_paging_test.c index b0e1fc4de9e29..6c2253f4a64ef 100644 --- a/tools/testing/selftests/kvm/demand_paging_test.c +++ b/tools/testing/selftests/kvm/demand_paging_test.c @@ -77,9 +77,15 @@ static int handle_uffd_page_request(int uffd_mode, int uffd, copy.mode = 0; r = ioctl(uffd, UFFDIO_COPY, ©); - if (r == -1) { - pr_info("Failed UFFDIO_COPY in 0x%lx from thread %d with errno: %d\n", - addr, tid, errno); + /* + * With multiple vCPU threads fault on a single page and there are + * multiple readers for the UFFD, at least one of the UFFDIO_COPYs + * will fail with EEXIST: handle that case without signaling an + * error. + */ + if (r == -1 && errno != EEXIST) { + pr_info("Failed UFFDIO_COPY in 0x%lx from thread %d, errno = %d\n", + addr, tid, errno); return r; } } else if (uffd_mode == UFFDIO_REGISTER_MODE_MINOR) { @@ -89,9 +95,10 @@ static int handle_uffd_page_request(int uffd_mode, int uffd, cont.range.len = demand_paging_size; r = ioctl(uffd, UFFDIO_CONTINUE, &cont); - if (r == -1) { - pr_info("Failed UFFDIO_CONTINUE in 0x%lx from thread %d with errno: %d\n", - addr, tid, errno); + /* See the note about EEXISTs in the UFFDIO_COPY branch. */ + if (r == -1 && errno != EEXIST) { + pr_info("Failed UFFDIO_CONTINUE in 0x%lx, thread %d, errno = %d\n", + addr, tid, errno); return r; } } else { @@ -110,7 +117,9 @@ static int handle_uffd_page_request(int uffd_mode, int uffd, struct test_params { int uffd_mode; + bool single_uffd; useconds_t uffd_delay; + int readers_per_uffd; enum vm_mem_backing_src_type src_type; bool partition_vcpu_memory_access; }; @@ -133,7 +142,8 @@ static void run_test(enum vm_guest_mode mode, void *arg) struct timespec start; struct timespec ts_diff; struct kvm_vm *vm; - int i; + int i, num_uffds = 0; + uint64_t uffd_region_size; vm = memstress_create_vm(mode, nr_vcpus, guest_percpu_mem_size, 1, p->src_type, p->partition_vcpu_memory_access); @@ -146,10 +156,13 @@ static void run_test(enum vm_guest_mode mode, void *arg) memset(guest_data_prototype, 0xAB, demand_paging_size); if (p->uffd_mode) { - uffd_descs = malloc(nr_vcpus * sizeof(struct uffd_desc *)); + num_uffds = p->single_uffd ? 1 : nr_vcpus; + uffd_region_size = nr_vcpus * guest_percpu_mem_size / num_uffds; + + uffd_descs = malloc(num_uffds * sizeof(struct uffd_desc *)); TEST_ASSERT(uffd_descs, "Memory allocation failed"); - for (i = 0; i < nr_vcpus; i++) { + for (i = 0; i < num_uffds; i++) { struct memstress_vcpu_args *vcpu_args; void *vcpu_hva; void *vcpu_alias; @@ -160,8 +173,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) vcpu_hva = addr_gpa2hva(vm, vcpu_args->gpa); vcpu_alias = addr_gpa2alias(vm, vcpu_args->gpa); - prefault_mem(vcpu_alias, - vcpu_args->pages * memstress_args.guest_page_size); + prefault_mem(vcpu_alias, uffd_region_size); /* * Set up user fault fd to handle demand paging @@ -169,7 +181,8 @@ static void run_test(enum vm_guest_mode mode, void *arg) */ uffd_descs[i] = uffd_setup_demand_paging( p->uffd_mode, p->uffd_delay, vcpu_hva, - vcpu_args->pages * memstress_args.guest_page_size, + uffd_region_size, + p->readers_per_uffd, &handle_uffd_page_request); } } @@ -186,7 +199,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) if (p->uffd_mode) { /* Tell the user fault fd handler threads to quit */ - for (i = 0; i < nr_vcpus; i++) + for (i = 0; i < num_uffds; i++) uffd_stop_demand_paging(uffd_descs[i]); } @@ -206,14 +219,19 @@ static void run_test(enum vm_guest_mode mode, void *arg) static void help(char *name) { puts(""); - printf("usage: %s [-h] [-m vm_mode] [-u uffd_mode] [-d uffd_delay_usec]\n" - " [-b memory] [-s type] [-v vcpus] [-o]\n", name); + printf("usage: %s [-h] [-m vm_mode] [-u uffd_mode] [-a]\n" + " [-d uffd_delay_usec] [-r readers_per_uffd] [-b memory]\n" + " [-s type] [-v vcpus] [-o]\n", name); guest_modes_help(); printf(" -u: use userfaultfd to handle vCPU page faults. Mode is a\n" " UFFD registration mode: 'MISSING' or 'MINOR'.\n"); + printf(" -a: Use a single userfaultfd for all of guest memory, instead of\n" + " creating one for each region paged by a unique vCPU\n" + " Set implicitly with -o, and no effect without -u.\n"); printf(" -d: add a delay in usec to the User Fault\n" " FD handler to simulate demand paging\n" " overheads. Ignored without -u.\n"); + printf(" -r: Set the number of reader threads per uffd.\n"); printf(" -b: specify the size of the memory region which should be\n" " demand paged by each vCPU. e.g. 10M or 3G.\n" " Default: 1G\n"); @@ -231,12 +249,14 @@ int main(int argc, char *argv[]) struct test_params p = { .src_type = DEFAULT_VM_MEM_SRC, .partition_vcpu_memory_access = true, + .readers_per_uffd = 1, + .single_uffd = false, }; int opt; guest_modes_append_default(); - while ((opt = getopt(argc, argv, "hm:u:d:b:s:v:o")) != -1) { + while ((opt = getopt(argc, argv, "ahom:u:d:b:s:v:r:")) != -1) { switch (opt) { case 'm': guest_modes_cmdline(optarg); @@ -248,6 +268,9 @@ int main(int argc, char *argv[]) p.uffd_mode = UFFDIO_REGISTER_MODE_MINOR; TEST_ASSERT(p.uffd_mode, "UFFD mode must be 'MISSING' or 'MINOR'."); break; + case 'a': + p.single_uffd = true; + break; case 'd': p.uffd_delay = strtoul(optarg, NULL, 0); TEST_ASSERT(p.uffd_delay >= 0, "A negative UFFD delay is not supported."); @@ -265,6 +288,13 @@ int main(int argc, char *argv[]) break; case 'o': p.partition_vcpu_memory_access = false; + p.single_uffd = true; + break; + case 'r': + p.readers_per_uffd = atoi(optarg); + TEST_ASSERT(p.readers_per_uffd >= 1, + "Invalid number of readers per uffd %d: must be >=1", + p.readers_per_uffd); break; case 'h': default: diff --git a/tools/testing/selftests/kvm/include/userfaultfd_util.h b/tools/testing/selftests/kvm/include/userfaultfd_util.h index 877449c345928..92cc1f9ec0686 100644 --- a/tools/testing/selftests/kvm/include/userfaultfd_util.h +++ b/tools/testing/selftests/kvm/include/userfaultfd_util.h @@ -17,18 +17,30 @@ typedef int (*uffd_handler_t)(int uffd_mode, int uffd, struct uffd_msg *msg); +struct uffd_reader_args { + int uffd_mode; + int uffd; + useconds_t delay; + uffd_handler_t handler; + /* Holds the read end of the pipe for killing the reader. */ + int pipe; +}; + struct uffd_desc { int uffd_mode; int uffd; - int pipefds[2]; useconds_t delay; uffd_handler_t handler; - pthread_t thread; + uint64_t num_readers; + /* Holds the write ends of the pipes for killing the readers. */ + int *pipefds; + pthread_t *readers; + struct uffd_reader_args *reader_args; }; struct uffd_desc *uffd_setup_demand_paging(int uffd_mode, useconds_t delay, void *hva, uint64_t len, - uffd_handler_t handler); + uint64_t num_readers, uffd_handler_t handler); void uffd_stop_demand_paging(struct uffd_desc *uffd); diff --git a/tools/testing/selftests/kvm/lib/userfaultfd_util.c b/tools/testing/selftests/kvm/lib/userfaultfd_util.c index 92cef20902f1f..2723ee1e3e1b2 100644 --- a/tools/testing/selftests/kvm/lib/userfaultfd_util.c +++ b/tools/testing/selftests/kvm/lib/userfaultfd_util.c @@ -27,10 +27,8 @@ static void *uffd_handler_thread_fn(void *arg) { - struct uffd_desc *uffd_desc = (struct uffd_desc *)arg; - int uffd = uffd_desc->uffd; - int pipefd = uffd_desc->pipefds[0]; - useconds_t delay = uffd_desc->delay; + struct uffd_reader_args *reader_args = (struct uffd_reader_args *)arg; + int uffd = reader_args->uffd; int64_t pages = 0; struct timespec start; struct timespec ts_diff; @@ -44,7 +42,7 @@ static void *uffd_handler_thread_fn(void *arg) pollfd[0].fd = uffd; pollfd[0].events = POLLIN; - pollfd[1].fd = pipefd; + pollfd[1].fd = reader_args->pipe; pollfd[1].events = POLLIN; r = poll(pollfd, 2, -1); @@ -92,9 +90,9 @@ static void *uffd_handler_thread_fn(void *arg) if (!(msg.event & UFFD_EVENT_PAGEFAULT)) continue; - if (delay) - usleep(delay); - r = uffd_desc->handler(uffd_desc->uffd_mode, uffd, &msg); + if (reader_args->delay) + usleep(reader_args->delay); + r = reader_args->handler(reader_args->uffd_mode, uffd, &msg); if (r < 0) return NULL; pages++; @@ -110,7 +108,7 @@ static void *uffd_handler_thread_fn(void *arg) struct uffd_desc *uffd_setup_demand_paging(int uffd_mode, useconds_t delay, void *hva, uint64_t len, - uffd_handler_t handler) + uint64_t num_readers, uffd_handler_t handler) { struct uffd_desc *uffd_desc; bool is_minor = (uffd_mode == UFFDIO_REGISTER_MODE_MINOR); @@ -118,14 +116,26 @@ struct uffd_desc *uffd_setup_demand_paging(int uffd_mode, useconds_t delay, struct uffdio_api uffdio_api; struct uffdio_register uffdio_register; uint64_t expected_ioctls = ((uint64_t) 1) << _UFFDIO_COPY; - int ret; + int ret, i; PER_PAGE_DEBUG("Userfaultfd %s mode, faults resolved with %s\n", is_minor ? "MINOR" : "MISSING", is_minor ? "UFFDIO_CONINUE" : "UFFDIO_COPY"); uffd_desc = malloc(sizeof(struct uffd_desc)); - TEST_ASSERT(uffd_desc, "malloc failed"); + TEST_ASSERT(uffd_desc, "Failed to malloc uffd descriptor"); + + uffd_desc->pipefds = malloc(sizeof(int) * num_readers); + TEST_ASSERT(uffd_desc->pipefds, "Failed to malloc pipes"); + + uffd_desc->readers = malloc(sizeof(pthread_t) * num_readers); + TEST_ASSERT(uffd_desc->readers, "Failed to malloc reader threads"); + + uffd_desc->reader_args = malloc( + sizeof(struct uffd_reader_args) * num_readers); + TEST_ASSERT(uffd_desc->reader_args, "Failed to malloc reader_args"); + + uffd_desc->num_readers = num_readers; /* In order to get minor faults, prefault via the alias. */ if (is_minor) @@ -148,18 +158,32 @@ struct uffd_desc *uffd_setup_demand_paging(int uffd_mode, useconds_t delay, TEST_ASSERT((uffdio_register.ioctls & expected_ioctls) == expected_ioctls, "missing userfaultfd ioctls"); - ret = pipe2(uffd_desc->pipefds, O_CLOEXEC | O_NONBLOCK); - TEST_ASSERT(!ret, "Failed to set up pipefd"); - uffd_desc->uffd_mode = uffd_mode; uffd_desc->uffd = uffd; uffd_desc->delay = delay; uffd_desc->handler = handler; - pthread_create(&uffd_desc->thread, NULL, uffd_handler_thread_fn, - uffd_desc); - PER_VCPU_DEBUG("Created uffd thread for HVA range [%p, %p)\n", - hva, hva + len); + for (i = 0; i < uffd_desc->num_readers; ++i) { + int pipes[2]; + + ret = pipe2((int *) &pipes, O_CLOEXEC | O_NONBLOCK); + TEST_ASSERT(!ret, "Failed to set up pipefd %i for uffd_desc %p", + i, uffd_desc); + + uffd_desc->pipefds[i] = pipes[1]; + + uffd_desc->reader_args[i].uffd_mode = uffd_mode; + uffd_desc->reader_args[i].uffd = uffd; + uffd_desc->reader_args[i].delay = delay; + uffd_desc->reader_args[i].handler = handler; + uffd_desc->reader_args[i].pipe = pipes[0]; + + pthread_create(&uffd_desc->readers[i], NULL, uffd_handler_thread_fn, + &uffd_desc->reader_args[i]); + + PER_VCPU_DEBUG("Created uffd thread %i for HVA range [%p, %p)\n", + i, hva, hva + len); + } return uffd_desc; } @@ -167,19 +191,31 @@ struct uffd_desc *uffd_setup_demand_paging(int uffd_mode, useconds_t delay, void uffd_stop_demand_paging(struct uffd_desc *uffd) { char c = 0; - int ret; + int i, ret; - ret = write(uffd->pipefds[1], &c, 1); - TEST_ASSERT(ret == 1, "Unable to write to pipefd"); + for (i = 0; i < uffd->num_readers; ++i) { + ret = write(uffd->pipefds[i], &c, 1); + TEST_ASSERT( + ret == 1, "Unable to write to pipefd %i for uffd_desc %p", i, uffd); + } - ret = pthread_join(uffd->thread, NULL); - TEST_ASSERT(ret == 0, "Pthread_join failed."); + for (i = 0; i < uffd->num_readers; ++i) { + ret = pthread_join(uffd->readers[i], NULL); + TEST_ASSERT( + ret == 0, + "Pthread_join failed on reader thread %i for uffd_desc %p", i, uffd); + } close(uffd->uffd); - close(uffd->pipefds[1]); - close(uffd->pipefds[0]); + for (i = 0; i < uffd->num_readers; ++i) { + close(uffd->pipefds[i]); + close(uffd->reader_args[i].pipe); + } + free(uffd->pipefds); + free(uffd->readers); + free(uffd->reader_args); free(uffd); } From patchwork Wed Apr 12 21:34:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EF6DC77B6E for ; Wed, 12 Apr 2023 21:35:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229818AbjDLVfX (ORCPT ); Wed, 12 Apr 2023 17:35:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229624AbjDLVfS (ORCPT ); Wed, 12 Apr 2023 17:35:18 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7053E76BB for ; Wed, 12 Apr 2023 14:35:17 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id t66-20020a254645000000b00b74680a7904so13784621yba.15 for ; Wed, 12 Apr 2023 14:35:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335316; x=1683927316; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lgzcOmQmlQzsv3LCJGMDuq4A2EjoJUzVVB/w7/0z8vA=; b=cmkcdKVeLh2S6IKc1Cs9ade/x7sNiMRO13/K9q5EqMJZiT82Hj0Tc3sdzQggNYX8dR PAs4hdoWMmDdLzoDjQ7M87uV2FCLwoqhlSzmTaE9nosSkoIEVLQ6ioJmcPMrePaCsU82 sIvGgwk+i1JGAw9zJetzP9EqJe82sgH80BfYvDAqOTk39wgqc/t/l/6dHC2Jq+jyz4C+ KuahS5DBS1a45NRUNeRmgTBO/Bo06AQ/fCIt/iv782FVdmXUpqKyydZUUix0I4KFVw5i ImhGoHqY6ACtB6T7QYawlA6G8rTJBrbTsHK0Fr4rVjouyVTglQSIFpE3h879Sr5gJ9H1 EUuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335316; x=1683927316; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lgzcOmQmlQzsv3LCJGMDuq4A2EjoJUzVVB/w7/0z8vA=; b=M07sMVc+ma6oe6Gq58lhUDEaASCHtd3P2ZHz1N86bZjmXwUd+aEtN4qfW+a2LjsVqu ogmNA+IU3QOU6gkcapDgRwhTlsJjayogqlKHfMI7NeS5unaMiGAr1m4CGWJwQbdCLSRw SVulzTOkVvSAA0nWWm2H6ydJ5ZVo6KAUsSeqI0VxJZAk5pJi8+L1ZTwPzJmsRvNff6gJ UqOWuaPyYMXacO3wgHjCZZU1DCp10u+ZJ5mipAd7XQXEmf4WJlv+A1lqIWqIz/tmRWXX oO0F9OI2+18u7bYCUdf76YWxkPpQbbSzlp6/GeZ8NDNqp1xc3h60BvDRetMXCvc8ONIO ofNQ== X-Gm-Message-State: AAQBX9drjOMbH1OrJPL1ktquoWemxqgbdYoIfoxWUrL331Zxj9/xQfIB LxFlY4cSuenojG9ERtnd3cHf+uF3osTpvg== X-Google-Smtp-Source: AKy350ZaAH0mQGsm4PuPKgEeTdL7xMnGtRfw6DgIFwoSqkDwzWee/Vm+Xz5CI+v5GnOQo871ROv9QALXJwf7HA== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:76c6:0:b0:b8b:ee74:c9d4 with SMTP id r189-20020a2576c6000000b00b8bee74c9d4mr13128406ybc.12.1681335316750; Wed, 12 Apr 2023 14:35:16 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:50 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-3-amoorthy@google.com> Subject: [PATCH v3 02/22] KVM: selftests: Use EPOLL in userfaultfd_util reader threads and signal errors via TEST_ASSERT From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org With multiple reader threads POLLing a single UFFD, the test suffers from the thundering herd problem: performance degrades as the number of reader threads is increased. Solve this issue [1] by switching the the polling mechanism to EPOLL + EPOLLEXCLUSIVE. Also, change the error-handling convention of uffd_handler_thread_fn. Instead of just printing errors and returning early from the polling loop, check for them via TEST_ASSERT. "return NULL" is reserved for a successful exit from uffd_handler_thread_fn, ie one triggered by a write to the exit pipe. Performance samples generated by the command in [2] are given below. Num Reader Threads, Paging Rate (POLL), Paging Rate (EPOLL) 1 249k 185k 2 201k 235k 4 186k 155k 16 150k 217k 32 89k 198k [1] Single-vCPU performance does suffer somewhat. [2] ./demand_paging_test -u MINOR -s shmem -v 4 -o -r Signed-off-by: Anish Moorthy Acked-by: James Houghton --- .../selftests/kvm/demand_paging_test.c | 1 - .../selftests/kvm/lib/userfaultfd_util.c | 74 +++++++++---------- 2 files changed, 35 insertions(+), 40 deletions(-) diff --git a/tools/testing/selftests/kvm/demand_paging_test.c b/tools/testing/selftests/kvm/demand_paging_test.c index 6c2253f4a64ef..c729cee4c2055 100644 --- a/tools/testing/selftests/kvm/demand_paging_test.c +++ b/tools/testing/selftests/kvm/demand_paging_test.c @@ -13,7 +13,6 @@ #include #include #include -#include #include #include #include diff --git a/tools/testing/selftests/kvm/lib/userfaultfd_util.c b/tools/testing/selftests/kvm/lib/userfaultfd_util.c index 2723ee1e3e1b2..909ad69c1cb04 100644 --- a/tools/testing/selftests/kvm/lib/userfaultfd_util.c +++ b/tools/testing/selftests/kvm/lib/userfaultfd_util.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include "kvm_util.h" @@ -32,60 +33,55 @@ static void *uffd_handler_thread_fn(void *arg) int64_t pages = 0; struct timespec start; struct timespec ts_diff; + int epollfd; + struct epoll_event evt; + + epollfd = epoll_create(1); + TEST_ASSERT(epollfd >= 0, "Failed to create epollfd."); + + evt.events = EPOLLIN | EPOLLEXCLUSIVE; + evt.data.u32 = 0; + TEST_ASSERT(epoll_ctl(epollfd, EPOLL_CTL_ADD, uffd, &evt) == 0, + "Failed to add uffd to epollfd"); + + evt.events = EPOLLIN; + evt.data.u32 = 1; + TEST_ASSERT(epoll_ctl(epollfd, EPOLL_CTL_ADD, reader_args->pipe, &evt) == 0, + "Failed to add pipe to epollfd"); clock_gettime(CLOCK_MONOTONIC, &start); while (1) { struct uffd_msg msg; - struct pollfd pollfd[2]; - char tmp_chr; int r; - pollfd[0].fd = uffd; - pollfd[0].events = POLLIN; - pollfd[1].fd = reader_args->pipe; - pollfd[1].events = POLLIN; - - r = poll(pollfd, 2, -1); - switch (r) { - case -1: - pr_info("poll err"); - continue; - case 0: - continue; - case 1: - break; - default: - pr_info("Polling uffd returned %d", r); - return NULL; - } + r = epoll_wait(epollfd, &evt, 1, -1); + TEST_ASSERT(r == 1, + "Unexpected number of events (%d) from epoll, errno = %d", + r, errno); - if (pollfd[0].revents & POLLERR) { - pr_info("uffd revents has POLLERR"); - return NULL; - } + if (evt.data.u32 == 1) { + char tmp_chr; - if (pollfd[1].revents & POLLIN) { - r = read(pollfd[1].fd, &tmp_chr, 1); + TEST_ASSERT(!(evt.events & (EPOLLERR | EPOLLHUP)), + "Reader thread received EPOLLERR or EPOLLHUP on pipe."); + r = read(reader_args->pipe, &tmp_chr, 1); TEST_ASSERT(r == 1, - "Error reading pipefd in UFFD thread\n"); + "Error reading pipefd in uffd reader thread"); return NULL; } - if (!(pollfd[0].revents & POLLIN)) - continue; + TEST_ASSERT(!(evt.events & (EPOLLERR | EPOLLHUP)), + "Reader thread received EPOLLERR or EPOLLHUP on uffd."); r = read(uffd, &msg, sizeof(msg)); if (r == -1) { - if (errno == EAGAIN) - continue; - pr_info("Read of uffd got errno %d\n", errno); - return NULL; + TEST_ASSERT(errno == EAGAIN, + "Error reading from UFFD: errno = %d", errno); + continue; } - if (r != sizeof(msg)) { - pr_info("Read on uffd returned unexpected size: %d bytes", r); - return NULL; - } + TEST_ASSERT(r == sizeof(msg), + "Read on uffd returned unexpected number of bytes (%d)", r); if (!(msg.event & UFFD_EVENT_PAGEFAULT)) continue; @@ -93,8 +89,8 @@ static void *uffd_handler_thread_fn(void *arg) if (reader_args->delay) usleep(reader_args->delay); r = reader_args->handler(reader_args->uffd_mode, uffd, &msg); - if (r < 0) - return NULL; + TEST_ASSERT(r >= 0, + "Reader thread handler fn returned negative value %d", r); pages++; } From patchwork Wed Apr 12 21:34:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE459C77B6C for ; Wed, 12 Apr 2023 21:35:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229877AbjDLVf3 (ORCPT ); Wed, 12 Apr 2023 17:35:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229773AbjDLVfX (ORCPT ); Wed, 12 Apr 2023 17:35:23 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C7A87A84 for ; Wed, 12 Apr 2023 14:35:18 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id v67-20020a254846000000b00b8189f73e94so36715058yba.12 for ; Wed, 12 Apr 2023 14:35:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335317; x=1683927317; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tZP5BD9r0skQIBcVsEuwbsf0/jfPby4JFW+0o25mrlY=; b=DAuQvN09j6rlTcQ+SQvMQ+Srqk6zFEIzLM+R85LsZhahtQ9CqOj17Bl9EwDey3+OVk P61HT30gkple2nno/lQ6MMs2NiUU9yRsMw67CiJ06U4l7n5Y6XMTqmmU4enb4W3pHBmq nBmCLgs5DM0SwjG0Z4HWqplWgT+BrSEOhLpoMD5vE9qPZc9sZNjFbjyUlwuSOybbkUau /NL1HdHJ7Qj+Wnp9YdpsWuGUi7Qwd9lEr+7CEbZOMeleYsWX4yUNsC/N65vWzHpxrdJV Kg9I13GwIJsn6srNEau4xyl+XnBWmGtXj4V9QSu4M9kEdgzLrmgUSFyd/ElRA6N/5V6+ oT/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335317; x=1683927317; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tZP5BD9r0skQIBcVsEuwbsf0/jfPby4JFW+0o25mrlY=; b=TF1HaUv/pE3rAp+EyIgWIeR5X8OwDhYNVgIeqnRQ64bj5+riHR69uKv5BhipR0fgfu ZJDDO/Unrn/swjYfD+uEhOyYntyktvvfTtToqwTX+Q7zAE5aFMbfHOI0J/Uu9ZoVvD/b uxjz15ngcWnSCc+Ay0KdkJnB4IrIxoS72xWLVPtvSO+SC84wnJUWKvbrreZ7kV1MmzyH CuLT1qvVKaQGrQxrZxeo6880ihmN8bgkDMsmniPQaXhJA14VsmqhtIL9nr4QHJJG03Ux YmwOqIDgb0PPx+7ye8mOo2oQf8YC1ubjSwYfwaW6UdEUJDPg44wp33scw8qPO22hwtun 7OqA== X-Gm-Message-State: AAQBX9cmoqz0EX+CrZA3o18GP7+g1S4XNkb13YpqZprhlboxeOZKRlfh rHfosggU69MNaH1LugYoKTlAYak6fj53yA== X-Google-Smtp-Source: AKy350ad5sOlGC49a3Zq+AqLstAbTPejBd4itxZk6c/jHU4q2Zhim4pjqaakFyBZUvWX5WivnmgBjuGC8os9Ig== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:d20c:0:b0:b6b:6a39:949c with SMTP id j12-20020a25d20c000000b00b6b6a39949cmr326ybg.6.1681335317547; Wed, 12 Apr 2023 14:35:17 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:51 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-4-amoorthy@google.com> Subject: [PATCH v3 03/22] KVM: Allow hva_pfn_fast() to resolve read-only faults. From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org hva_to_pfn_fast() currently just fails for read-only faults, which is unnecessary. Instead, try pinning the page without passing FOLL_WRITE. This allows read-only faults to (potentially) be resolved without falling back to slow GUP. Suggested-by: James Houghton Signed-off-by: Anish Moorthy --- virt/kvm/kvm_main.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f40b72eb0e7bf..cf7d3de6f3689 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2479,7 +2479,7 @@ static inline int check_user_page_hwpoison(unsigned long addr) } /* - * The fast path to get the writable pfn which will be stored in @pfn, + * The fast path to get the pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. It's also the * only part that runs if we can in atomic context. */ @@ -2493,10 +2493,9 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * or the caller allows to map a writable pfn for a read fault * request. */ - if (!(write_fault || writable)) - return false; + unsigned int gup_flags = (write_fault || writable) ? FOLL_WRITE : 0; - if (get_user_page_fast_only(addr, FOLL_WRITE, page)) { + if (get_user_page_fast_only(addr, gup_flags, page)) { *pfn = page_to_pfn(page[0]); if (writable) From patchwork Wed Apr 12 21:34:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DB97C7619A for ; Wed, 12 Apr 2023 21:35:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229749AbjDLVfZ (ORCPT ); Wed, 12 Apr 2023 17:35:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229663AbjDLVfU (ORCPT ); Wed, 12 Apr 2023 17:35:20 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 399987A85 for ; Wed, 12 Apr 2023 14:35:19 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id j6-20020a255506000000b00b8ef3da4acfso10961149ybb.8 for ; Wed, 12 Apr 2023 14:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335318; x=1683927318; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9qcXR+cj62Nmz/QQQB2g9N2pbdYpcmmyV5nmt0T5Wjo=; b=jKd+t0UtWM9xdOiqGRYNFX92os/otTxaJl4Aq9bIi0v9GANiTqkZBL/I45CujgAQS6 MzK4MDvKOetboWsJ3AHeOfQd/NePDJlf65y7kIDGm3MVGQnsV8BAB2K34AdrUiC0Dhn3 ETjJ1PEuxflgPxc8u+TwmpiZ1VExPHDqjNtsatLtld6xdOsm/BMr4fHWr2dAljkvCJ9j t4SUTI6Jq4R2QOZv+CxYoQeTRoneT7k0B1dC9jqxfcJA9b5ur1FurjeI473lS5euST9Z FDKBAA7J0WRtJLH7ZUmGAUuavMkA+9wKoI/V/FPWkmO6R2om6cXKIV2IivywVuTD+ZpM j57A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335318; x=1683927318; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9qcXR+cj62Nmz/QQQB2g9N2pbdYpcmmyV5nmt0T5Wjo=; b=E1rKLsGj03TeFcJFLZmws0yQVpXx7PGyrlrT++2P4xRNADX5/18U2l+1WENUqfCIEr oT3XX3uZOE69hOXPGX4eZO0ikx7mVRmch6FwugliW+U0x1cTbLxTZIC8/rHfhjsUSXJm ey0jDy8LNIfDy/s5PG4z/6muaYWqbo1zeoy8nDYIKxNdf1BwME21CiDn7Y8ETox0giMN /btsCjWKGj/1sZgLhO4kW1PSUHWEZ+ncpzXSYg+aGt9KAZQ1u2P7P8Jqq6Q9VT0VWxbT Z7nb4vq2OwB2q0rPB4NDqMTwGZPbFda74euVLkF0tRkgyrtajac2jS55j2np/EaaQTgl zSKw== X-Gm-Message-State: AAQBX9cwR0w2dvtgSCJdRhT2tbYj8rDJ0+OXiqFggDhqoEBtRUdUrw6X wMAs1UFtdjBS7rfZ0urTszMHD9TjVngspg== X-Google-Smtp-Source: AKy350Zu8+mZvFG+GBSeyU7SYAPl8PpXVNqqRPXN8kTxiuOldRi6f9+m5WF52N2/PJJYK1MSzUO1lNI0nmUPUw== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:cb97:0:b0:b8f:52e6:69a8 with SMTP id b145-20020a25cb97000000b00b8f52e669a8mr753519ybg.13.1681335318559; Wed, 12 Apr 2023 14:35:18 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:52 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-5-amoorthy@google.com> Subject: [PATCH v3 04/22] KVM: x86: Set vCPU exit reason to KVM_EXIT_UNKNOWN at the start of KVM_RUN From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Give kvm_run.exit_reason a defined initial value on entry into KVM_RUN: other architectures (riscv, arm64) already use KVM_EXIT_UNKNOWN for this purpose, so copy that convention. This gives vCPUs trying to fill the run struct a mechanism to avoid overwriting already-populated data, albeit an imperfect one. Being able to detect an already-populated KVM run struct will prevent at least some bugs in the upcoming implementation of KVM_CAP_MEMORY_FAULT_INFO, which will attempt to fill the run struct whenever a vCPU fails a guest memory access. Without the already-populated check, KVM_CAP_MEMORY_FAULT_INFO could change kvm_run in any code paths which 1. Populate kvm_run for some exit and prepare to return to userspace 2. Access guest memory for some reason (but without returning -EFAULTs to userspace) 3. Finish the return to userspace set up in (1), now with the contents of kvm_run changed to contain efault info. Signed-off-by: Anish Moorthy --- arch/x86/kvm/x86.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 237c483b12301..ca73eb066af81 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10965,6 +10965,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) kvm_run->flags = 0; kvm_load_guest_fpu(vcpu); + kvm_run->exit_reason = KVM_EXIT_UNKNOWN; kvm_vcpu_srcu_read_lock(vcpu); if (unlikely(vcpu->arch.mp_state == KVM_MP_STATE_UNINITIALIZED)) { if (kvm_run->immediate_exit) { From patchwork Wed Apr 12 21:34:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C19EEC77B6E for ; Wed, 12 Apr 2023 21:35:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229908AbjDLVf2 (ORCPT ); Wed, 12 Apr 2023 17:35:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229778AbjDLVfW (ORCPT ); Wed, 12 Apr 2023 17:35:22 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 861467A9A for ; Wed, 12 Apr 2023 14:35:20 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-54fba72c1adso5263297b3.18 for ; Wed, 12 Apr 2023 14:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335319; x=1683927319; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XA7ZPxmHg4/s+cCGlrNdqftkAczUKyZiwPE0Cfk6uhc=; b=UMlPsaMe2jdeh3T/gz/9yNk1xOARgALvKqLtffWllfUu8xTIxk6QNi++BZqxvSTcGh E8/nbCIPgwNn5PPf9dY6ivUQVDL3QDfeGie4xPfltePK3Ajjce3yJTXFXCON0JmGEq4y xfih2yUMOBP5ilfWXDlWVn6JImXGodq658JHFrHbBHnF2TC2ZlUWINB59M9TXCUGj1wa cA/UrVOheh8CaRND3hBP8ZiPFXL2d7cJJI/0b0FDU/Bbq93oif2Auk4VvERlLJ/wUdrL Crq2wSQhMesKwWC4v8shkUrIQLoErj8MXAZIUZ155V3HOAhMsgLi3lMNhO2IEgjOocEh Lmbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335319; x=1683927319; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XA7ZPxmHg4/s+cCGlrNdqftkAczUKyZiwPE0Cfk6uhc=; b=kVGMACXNIEnooBqlLmsOTJqgv1rZ1FGkFjUGI2Nh8IQe+M+shJ9vwcQEjX6B1lUEyA zHCxi0XwEUl5Hr08y3hZemtKFvmQEgHFmGc3Wj6vBgUb48lwKu/xE1fGXShaNjCMqyD1 9CyoUK6pCbbb22P+Rvl/nSkozUZ+LeSuRwq1pN+21sk9EpKPWEWllW62gKNDs09V0WA5 U1S19+715Uk13Px0Iv+dxa52Ry6qhoDLAZ6Ilb8Ol5s5diORKW31n8G9DHfrT/qVEAL6 VpS0K7cU+QaFx4bDX4YU4Z4cxkFhwetOu40ZlLjW6cU/OsOxklpmM50bZF70CV3IlBOu knEQ== X-Gm-Message-State: AAQBX9eXh/2WFCN+zqq3UwUNz6PD/7x7ghy6PYWc5a4bCYg9UfKV8PJV 5pQT6OupTG81s3LY19jCkToHVRdcLO2vqA== X-Google-Smtp-Source: AKy350Yut9wS/+fWCKspx5L/4ZsPekYmGr9kxfTgK6RJrGbUmHYyDsLRxjDmjwTNPoBVgF5A+tmMyPisYta+Kw== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a81:ed11:0:b0:54f:a986:3f94 with SMTP id k17-20020a81ed11000000b0054fa9863f94mr10765ywm.3.1681335319419; Wed, 12 Apr 2023 14:35:19 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:53 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-6-amoorthy@google.com> Subject: [PATCH v3 05/22] KVM: Add KVM_CAP_MEMORY_FAULT_INFO From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org KVM_CAP_MEMORY_FAULT_INFO allows kvm_run to return useful information besides a return value of -1 and errno of EFAULT when a vCPU fails an access to guest memory. Add documentation, updates to the KVM headers, and a helper function (kvm_populate_efault_info) for implementing the capability. Besides simply filling the run struct, kvm_populate_efault_info takes two safety measures a. It tries to prevent concurrent fills on a single vCPU run struct by checking that the run struct being modified corresponds to the currently loaded vCPU. b. It tries to avoid filling an already-populated run struct by checking whether the exit reason has been modified since entry into KVM_RUN. Finally, mark KVM_CAP_MEMORY_FAULT_INFO as available on arm64 and x86, even though EFAULT annotation are currently totally absent. Picking a point to declare the implementation "done" is difficult because 1. Annotations will be performed incrementally in subsequent commits across both core and arch-specific KVM. 2. The initial series will very likely miss some cases which need annotation. Although these omissions are to be fixed in the future, userspace thus still needs to expect and be able to handle unannotated EFAULTs. Given these qualifications, just marking it available here seems the least arbitrary thing to do. Suggested-by: Sean Christopherson Signed-off-by: Anish Moorthy --- Documentation/virt/kvm/api.rst | 35 +++++++++++++++++++++++++++ arch/arm64/kvm/arm.c | 1 + arch/x86/kvm/x86.c | 1 + include/linux/kvm_host.h | 12 ++++++++++ include/uapi/linux/kvm.h | 16 +++++++++++++ tools/include/uapi/linux/kvm.h | 11 +++++++++ virt/kvm/kvm_main.c | 44 ++++++++++++++++++++++++++++++++++ 7 files changed, 120 insertions(+) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 48fad65568227..f174f43c38d45 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6637,6 +6637,18 @@ array field represents return values. The userspace should update the return values of SBI call before resuming the VCPU. For more details on RISC-V SBI spec refer, https://github.com/riscv/riscv-sbi-doc. +:: + + /* KVM_EXIT_MEMORY_FAULT */ + struct { + __u64 flags; + __u64 gpa; + __u64 len; /* in bytes */ + } memory_fault; + +Indicates a vCPU memory fault on the guest physical address range +[gpa, gpa + len). See KVM_CAP_MEMORY_FAULT_INFO for more details. + :: /* KVM_EXIT_NOTIFY */ @@ -7670,6 +7682,29 @@ This capability is aimed to mitigate the threat that malicious VMs can cause CPU stuck (due to event windows don't open up) and make the CPU unavailable to host or other VMs. +7.34 KVM_CAP_MEMORY_FAULT_INFO +------------------------------ + +:Architectures: x86, arm64 +:Parameters: args[0] - KVM_MEMORY_FAULT_INFO_ENABLE|DISABLE to enable/disable + the capability. +:Returns: 0 on success, or -EINVAL if unsupported or invalid args[0]. + +When enabled, EFAULTs "returned" by KVM_RUN in response to failed vCPU guest +memory accesses may be annotated with additional information. When KVM_RUN +returns an error with errno=EFAULT, userspace may check the exit reason: if it +is KVM_EXIT_MEMORY_FAULT, userspace is then permitted to read the 'memory_fault' +member of the run struct. + +The 'gpa' and 'len' (in bytes) fields describe the range of guest +physical memory to which access failed, i.e. [gpa, gpa + len). 'flags' is +currently always zero. + +NOTE: The implementation of this capability is incomplete. Even with it enabled, +userspace may receive "bare" EFAULTs (i.e. exit reason != +KVM_EXIT_MEMORY_FAULT) from KVM_RUN. These should be considered bugs and +reported to the maintainers. + 8. Other capabilities. ====================== diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index a43e1cb3b7e97..a932346b59f61 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -220,6 +220,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_VCPU_ATTRIBUTES: case KVM_CAP_PTP_KVM: case KVM_CAP_ARM_SYSTEM_SUSPEND: + case KVM_CAP_MEMORY_FAULT_INFO: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ca73eb066af81..0925678e741de 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4432,6 +4432,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_VAPIC: case KVM_CAP_ENABLE_CAP: case KVM_CAP_VM_DISABLE_NX_HUGE_PAGES: + case KVM_CAP_MEMORY_FAULT_INFO: r = 1; break; case KVM_CAP_EXIT_HYPERCALL: diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 90edc16d37e59..776f9713f3921 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -805,6 +805,8 @@ struct kvm { struct notifier_block pm_notifier; #endif char stats_id[KVM_STATS_NAME_SIZE]; + + bool fill_efault_info; }; #define kvm_err(fmt, ...) \ @@ -2277,4 +2279,14 @@ static inline void kvm_account_pgtable_pages(void *virt, int nr) /* Max number of entries allowed for each kvm dirty ring */ #define KVM_DIRTY_RING_MAX_ENTRIES 65536 +/* + * Attempts to set the run struct's exit reason to KVM_EXIT_MEMORY_FAULT and + * populate the memory_fault field with the given information. + * + * Does nothing if KVM_CAP_MEMORY_FAULT_INFO is not enabled. WARNs and does + * nothing if the exit reason is not KVM_EXIT_UNKNOWN, or if 'vcpu' is not + * the current running vcpu. + */ +inline void kvm_populate_efault_info(struct kvm_vcpu *vcpu, + uint64_t gpa, uint64_t len); #endif diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 4003a166328cc..bc73e8381a2bb 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -264,6 +264,7 @@ struct kvm_xen_exit { #define KVM_EXIT_RISCV_SBI 35 #define KVM_EXIT_RISCV_CSR 36 #define KVM_EXIT_NOTIFY 37 +#define KVM_EXIT_MEMORY_FAULT 38 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -505,6 +506,16 @@ struct kvm_run { #define KVM_NOTIFY_CONTEXT_INVALID (1 << 0) __u32 flags; } notify; + /* KVM_EXIT_MEMORY_FAULT */ + struct { + /* + * Indicates a memory fault on the guest physical address range + * [gpa, gpa + len). flags is always zero for now. + */ + __u64 flags; + __u64 gpa; + __u64 len; /* in bytes */ + } memory_fault; /* Fix the size of the union. */ char padding[256]; }; @@ -1184,6 +1195,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224 #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225 #define KVM_CAP_PMU_EVENT_MASKED_EVENTS 226 +#define KVM_CAP_MEMORY_FAULT_INFO 227 #ifdef KVM_CAP_IRQ_ROUTING @@ -2237,4 +2249,8 @@ struct kvm_s390_zpci_op { /* flags for kvm_s390_zpci_op->u.reg_aen.flags */ #define KVM_S390_ZPCIOP_REGAEN_HOST (1 << 0) +/* flags for KVM_CAP_MEMORY_FAULT_INFO */ +#define KVM_MEMORY_FAULT_INFO_DISABLE 0 +#define KVM_MEMORY_FAULT_INFO_ENABLE 1 + #endif /* __LINUX_KVM_H */ diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h index 4003a166328cc..5c57796364d65 100644 --- a/tools/include/uapi/linux/kvm.h +++ b/tools/include/uapi/linux/kvm.h @@ -264,6 +264,7 @@ struct kvm_xen_exit { #define KVM_EXIT_RISCV_SBI 35 #define KVM_EXIT_RISCV_CSR 36 #define KVM_EXIT_NOTIFY 37 +#define KVM_EXIT_MEMORY_FAULT 38 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -505,6 +506,16 @@ struct kvm_run { #define KVM_NOTIFY_CONTEXT_INVALID (1 << 0) __u32 flags; } notify; + /* KVM_EXIT_MEMORY_FAULT */ + struct { + /* + * Indicates a memory fault on the guest physical address range + * [gpa, gpa + len). flags is always zero for now. + */ + __u64 flags; + __u64 gpa; + __u64 len; /* in bytes */ + } memory_fault; /* Fix the size of the union. */ char padding[256]; }; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index cf7d3de6f3689..f3effc93cbef3 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1142,6 +1142,7 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname) spin_lock_init(&kvm->mn_invalidate_lock); rcuwait_init(&kvm->mn_memslots_update_rcuwait); xa_init(&kvm->vcpu_array); + kvm->fill_efault_info = false; INIT_LIST_HEAD(&kvm->gpc_list); spin_lock_init(&kvm->gpc_lock); @@ -4096,6 +4097,8 @@ static long kvm_vcpu_ioctl(struct file *filp, put_pid(oldpid); } r = kvm_arch_vcpu_ioctl_run(vcpu); + WARN_ON_ONCE(r == -EFAULT && + vcpu->run->exit_reason != KVM_EXIT_MEMORY_FAULT); trace_kvm_userspace_exit(vcpu->run->exit_reason, r); break; } @@ -4672,6 +4675,15 @@ static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm, return r; } + case KVM_CAP_MEMORY_FAULT_INFO: { + if (!kvm_vm_ioctl_check_extension_generic(kvm, cap->cap) + || (cap->args[0] != KVM_MEMORY_FAULT_INFO_ENABLE + && cap->args[0] != KVM_MEMORY_FAULT_INFO_DISABLE)) { + return -EINVAL; + } + kvm->fill_efault_info = cap->args[0] == KVM_MEMORY_FAULT_INFO_ENABLE; + return 0; + } default: return kvm_vm_ioctl_enable_cap(kvm, cap); } @@ -6173,3 +6185,35 @@ int kvm_vm_create_worker_thread(struct kvm *kvm, kvm_vm_thread_fn_t thread_fn, return init_context.err; } + +inline void kvm_populate_efault_info(struct kvm_vcpu *vcpu, + uint64_t gpa, uint64_t len) +{ + if (!vcpu->kvm->fill_efault_info) + return; + + preempt_disable(); + /* + * Ensure the this vCPU isn't modifying another vCPU's run struct, which + * would open the door for races between concurrent calls to this + * function. + */ + if (WARN_ON_ONCE(vcpu != __this_cpu_read(kvm_running_vcpu))) + goto out; + /* + * Try not to overwrite an already-populated run struct. + * This isn't a perfect solution, as there's no guarantee that the exit + * reason is set before the run struct is populated, but it should prevent + * at least some bugs. + */ + else if (WARN_ON_ONCE(vcpu->run->exit_reason != KVM_EXIT_UNKNOWN)) + goto out; + + vcpu->run->exit_reason = KVM_EXIT_MEMORY_FAULT; + vcpu->run->memory_fault.gpa = gpa; + vcpu->run->memory_fault.len = len; + vcpu->run->memory_fault.flags = 0; + +out: + preempt_enable(); +} From patchwork Wed Apr 12 21:34:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14F7BC77B6C for ; Wed, 12 Apr 2023 21:35:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229895AbjDLVf0 (ORCPT ); Wed, 12 Apr 2023 17:35:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229532AbjDLVfV (ORCPT ); Wed, 12 Apr 2023 17:35:21 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2F627DB0 for ; Wed, 12 Apr 2023 14:35:20 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id e199-20020a2550d0000000b00b8db8a4530dso15857153ybb.5 for ; Wed, 12 Apr 2023 14:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335320; x=1683927320; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Wl6iG0DOT7sovg486sG/qtkYJxnYf7LGgKWwCJBoMuk=; b=e6oJMJAXHkG1GgpSvcna+UUqOW6jAVvShZ5EdXAucj6cFMXDvMYAc7fhW6BNAkflBZ gm9HtIbwRa7Mr1HBJgRmG0sivVcTDY1okRzGulVzvhKujEyWGbAYRj86jyOiqogTK+HX zH8eZ3I8wLGkEHq1RFtbhG54176VhNBW3ppd1Ufcer0GgeLbG6KjSbVn/r4X6GGrQIQD A1FhT5EwCxIJqg6EK4njGOlZSUi/o8lzcPsHzbnG0kbKu5XN+8lpYIaw2cgMsvSarZ1I K4RUM2haY6aKr8ttVscTRXWARYqk68+ZtfFUhh2enJPcDEUR6B8Or5xBWdMFY2O3GXPG 6Qzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335320; x=1683927320; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Wl6iG0DOT7sovg486sG/qtkYJxnYf7LGgKWwCJBoMuk=; b=QXRQoGeExmaSNb8/VhQCBsgUIKg8/F/acrY4XtdJiD2VfwyVJKSnxabqA4PsFL+F2M tmAHH69LPmpQgUTKYPAI8F8zF4zX9FT5nCuMirNjTM0u4Agh09rJgxEUe7MVO5fGCoUX Jov8CYEZCIQVD+hxAKPWpRVEHxeeKppjIVaH0VpdTWXHk0hqgV/JeGnyBtpDGDV5JJys gTr5NWjeHuSbM1Rpe2BJRw3jcaxbtLLRx1kEo9kI2Wy5qkZd7wnKBeM9bInF7oKwoQVv DucDHmYLHaAmHlu5GP++g6e7mDz/q4rvciL9gIFNufgOUlcdBH037RDYBBOC/OZ1yRWS 4B+Q== X-Gm-Message-State: AAQBX9cqVth4TGxJ6oyLwQlFy17eoL9Uky7KYld0k1vFczMi289mzvkp ae3IodcVi6hjfpnO6L8jY2RG2QytBETMAg== X-Google-Smtp-Source: AKy350ZEHPQmI8hm7Ol6t1Lr3Upb/g72LKo6Tr5VDajGP6irllnJy24C2tTFtoEJLO01pwL6l2QifUQCzDsdNQ== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:40f:0:b0:b26:884:c35e with SMTP id 15-20020a25040f000000b00b260884c35emr3043114ybe.4.1681335320281; Wed, 12 Apr 2023 14:35:20 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:54 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-7-amoorthy@google.com> Subject: [PATCH v3 06/22] KVM: Add docstrings to __kvm_write_guest_page() and __kvm_read_guest_page() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The order of parameters in these function signature is a little strange, with "offset" actually applying to "gfn" rather than to "data". Add short comments to make things perfectly clear. Signed-off-by: Anish Moorthy --- virt/kvm/kvm_main.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f3effc93cbef3..63b4285d858d1 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2982,6 +2982,9 @@ static int next_segment(unsigned long len, int offset) return len; } +/* + * Copy 'len' bytes from guest memory at '(gfn * PAGE_SIZE) + offset' to 'data' + */ static int __kvm_read_guest_page(struct kvm_memory_slot *slot, gfn_t gfn, void *data, int offset, int len) { @@ -3083,6 +3086,9 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, } EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic); +/* + * Copy 'len' bytes from 'data' into guest memory at '(gfn * PAGE_SIZE) + offset' + */ static int __kvm_write_guest_page(struct kvm *kvm, struct kvm_memory_slot *memslot, gfn_t gfn, const void *data, int offset, int len) From patchwork Wed Apr 12 21:34:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAAE3C77B6C for ; Wed, 12 Apr 2023 21:35:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229936AbjDLVfd (ORCPT ); Wed, 12 Apr 2023 17:35:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229624AbjDLVfZ (ORCPT ); Wed, 12 Apr 2023 17:35:25 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C92E37EC2 for ; Wed, 12 Apr 2023 14:35:21 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-54f87e44598so46584897b3.5 for ; Wed, 12 Apr 2023 14:35:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335321; x=1683927321; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FCUsl1ZTxToHK35g7kMPjWEsQGVLI4o3jf0Ta0rdcNU=; b=GqYSDW8zsU2k4X6g1gBul0HwBunj4Kc5oYYOBMCg2HxZHahOe7VzzKcmgXGNk6Y76U ES2/HgE87JrE3Kj+DU1U7U7ZPB3OeWv2fQqzkndHu4nkh9tNJXo+8XA6i+dP16vd5jH6 yhhp5Xlpg/1WXEt2TCimkolDf+lDpG00WZd+4Rf7ToYnySMIrDySxVaaJY8qsIpPh+Xc F6NoX4FGulsoWmc57IaSRd8SkJ6BOC/oz9X5iq9fyC4RhUl/E/ELD4HBRCVFBKSZpISJ 5uLvNeMIJ+NcqD5ZUkiMlnPJ9ZP/M8N/9PrABFSOd2H7umgblTHbxO2YbtmUCYZAHR+e 8T1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335321; x=1683927321; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FCUsl1ZTxToHK35g7kMPjWEsQGVLI4o3jf0Ta0rdcNU=; b=doHqMVYqHbCSo2xb7nDBvAIT+KOFdHNhhtKoqM6OAxnMlUoJjOASHyrXmtvEWisA6F E2FavM7xhbZxs2JP3PNjXsfCkaIgrfniOZmY/ePU1YbmmptJddarj/ihQD0E7P4tl/p+ F1usu3qeL8W7n16tY86JppbDUMffTeLaRy7t8ZNHYt1N/o9Ft4l1jqoTobRkbTuSxxw+ W9jwCDKHO8VL0qDITzxFaRKXMFeT4rraVEt2uVhbNkq+ICFoMQt6eexO2Ai6BGUFdk5R KF+OSZs2vQe0UiQtvXuVTplgXWP42feghA0IoPI3olC1kT8rRZR9g4GLSkRfXuxcwAIf xLpg== X-Gm-Message-State: AAQBX9d5Yn72cLWbSj94Rl7lr+leVwnWgHt+tiLSNATXHRsF5BXQTxCi JaHX555MRzRTUFnznnjqUSymhFvU4B+yxg== X-Google-Smtp-Source: AKy350YICLmkH8ieQ0EkmaBManzdBLGb7Tvuk3VBxUWumq1jgSxExJBdj4wohiY0LLASz9PuLStOSQz2vaCbWA== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a05:690c:d0f:b0:545:f3ed:d251 with SMTP id cn15-20020a05690c0d0f00b00545f3edd251mr3435849ywb.1.1681335321139; Wed, 12 Apr 2023 14:35:21 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:55 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-8-amoorthy@google.com> Subject: [PATCH v3 07/22] KVM: Annotate -EFAULTs from kvm_vcpu_write_guest_page() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults from kvm_vcpu_write_guest_page() Signed-off-by: Anish Moorthy --- virt/kvm/kvm_main.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 63b4285d858d1..b29a38af543f0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3119,8 +3119,11 @@ int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, const void *data, int offset, int len) { struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); + int ret = __kvm_write_guest_page(vcpu->kvm, slot, gfn, data, offset, len); - return __kvm_write_guest_page(vcpu->kvm, slot, gfn, data, offset, len); + if (ret == -EFAULT) + kvm_populate_efault_info(vcpu, gfn * PAGE_SIZE + offset, len); + return ret; } EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest_page); From patchwork Wed Apr 12 21:34:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8399AC77B76 for ; Wed, 12 Apr 2023 21:35:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230009AbjDLVfb (ORCPT ); Wed, 12 Apr 2023 17:35:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229828AbjDLVfY (ORCPT ); Wed, 12 Apr 2023 17:35:24 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 129D37EEC for ; Wed, 12 Apr 2023 14:35:23 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-54f3e30726cso92487247b3.22 for ; Wed, 12 Apr 2023 14:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335322; x=1683927322; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2HEZpjwJAllfV0I5F7ADUYQU3lomGIeMIJyobWdtpuc=; b=lAmykGgc7/jSWWpiEUSSuOFZq53ucIIAMaKUFqmueRpj2EzlhWgnqQLfis3Ztcd4wi h5Zgje/Jxjcj2CcRv9fSMfzcrGEpsPUQ+X4OYlwUzTG9hZZgzZxEg4iERSVRsazSR4dQ mbJBRRiQGBkQ1VaXe8CftZCD+8LyveM0bff5wLsLok2+QNnHiJm2xDU1peod1tRA9Gme 8K+wJXLIyDSfDU2Liq5+LTeILHdZRIhAdBtJ4tBTagjbxyHGme3Bdyd46qOb/TRcpdXT elyMkDBA5mdJuuZYGfiFiCeYQ/U5/qRgk3o+DrSjOQIWmibO2PFVdAWi+0KxVip7dHLw hBDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335322; x=1683927322; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2HEZpjwJAllfV0I5F7ADUYQU3lomGIeMIJyobWdtpuc=; b=K7U9dSxZb6mzYS9IRrC/CWPZQQX4xQiYfQ+N+QiptZSHL2x7oi67UcydcvKCIbjv5G GohVKa2GmOhhh7dYZ+41omrsV0h2HZKT1F/gaBQ80y292iLslbKjfqtE+bzOU5tFk2f3 kRBzrgcZmzl9+PV0fIJV2xpQ2bbQ82XaoGdn1zkxyTqeU6j+vDvYBdXkuz4Dmjm3EZtc f3HBmeVM7GaIhI/DtdkT+ldAmehGaSD42B1IKkbxoDqOshHgE3gFSEMtYGcLl5o7fbT4 oEA7V9bXQD9mcBK2UVZMTbpCXg941NQT1w2+mxOP5y9kUJ53JgE8oD/RBYmxrXIIgUAo MNKw== X-Gm-Message-State: AAQBX9fWoKydd0dJmLX11ekeCRVelExPLajCx/6gl238J+h/cj8+q9Hh hJbpzmJXVcISpNqm4rQrn/Cd8wL3djau9A== X-Google-Smtp-Source: AKy350ZEcbjFijoKkL3xtGAA1xKzYaEJ45MS8KT6dNFR0HK9SeJ8kLYqa/GC16fyN32eT7P0BxcvFD5YNNEzLA== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:d78b:0:b0:b67:d295:d614 with SMTP id o133-20020a25d78b000000b00b67d295d614mr9968308ybg.12.1681335322385; Wed, 12 Apr 2023 14:35:22 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:56 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-9-amoorthy@google.com> Subject: [PATCH v3 08/22] KVM: Annotate -EFAULTs from kvm_vcpu_read_guest_page() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults generated by kvm_vcpu_read_guest_page(). Signed-off-by: Anish Moorthy --- virt/kvm/kvm_main.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b29a38af543f0..572adba9ad8ed 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3014,7 +3014,11 @@ int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, { struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); - return __kvm_read_guest_page(slot, gfn, data, offset, len); + int ret = __kvm_read_guest_page(slot, gfn, data, offset, len); + + if (ret == -EFAULT) + kvm_populate_efault_info(vcpu, gfn * PAGE_SIZE + offset, len); + return ret; } EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_page); From patchwork Wed Apr 12 21:34:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50233C7619A for ; Wed, 12 Apr 2023 21:35:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230020AbjDLVfc (ORCPT ); Wed, 12 Apr 2023 17:35:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229775AbjDLVfZ (ORCPT ); Wed, 12 Apr 2023 17:35:25 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 667157A85 for ; Wed, 12 Apr 2023 14:35:24 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-54bfc4e0330so213032637b3.3 for ; Wed, 12 Apr 2023 14:35:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335323; x=1683927323; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uHo5QQHeSuM2EtVp0WZkAVQAlrFLxra0Ir0r+Emw+q4=; b=sa+XBYOkX2d/Tu87yvD3UhP5nhC7As51Olcx+W/RSEY/FGJuNDurfGxnX//1FqsoM2 PJTt3/hUG7FrXNKJiVjFgxDx7oahgt4JlSXSeXB7SuChwWitdUEUlqC13KbMvt9pAe5b zArqJRTUKp7mKE6YJmU9VTtlAuOdBuED2zJNjYiJQGU4IdX1hTel0j5u28jgfcUoLPuM UcSNN40jmwOi2rQ3p+pwHCL+Lzc4bweXDCbWWTR7hgoE/VkKJ3GoZ07ZMgQq8SUMudLk dwcDtmojhM9j6Jpn97L/vupf06dWUHVzfFDZhSr5ygLVJGg8y2SUDVOmW1Ps1ad3Wrad 9zGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335323; x=1683927323; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uHo5QQHeSuM2EtVp0WZkAVQAlrFLxra0Ir0r+Emw+q4=; b=LjueViCjvuHITv8+eLW2IzALLY/aLgjBVMtartS4ZugfHj7TE42BsNAV1lHhL2+iL5 K1zXkhoCGkmCEaqC9f1iyYYnkXZVAUyQvN1V2DOFXoeGiHrd/qC0W/qG1Bt1UWv4VhFk 2Th6kyp4CVis2ejlXsDHjyHP4lBFttDMUlfgVHp5BFHcOg5lEldhz2XBcxI/ueT9w0zE 9Se8ut2h87wAty2vyCTJ9EzhprGEN3TOW8LQ/+Cg5wU9EtvEJT/ES516o/R0ZRJXk7oh SItd9lkOpvpmjbbU3UKnzpMJFP+P4lY5YlCGocC4OWan4eBR/UMG6WPx2MaER6lPVnVA +1LA== X-Gm-Message-State: AAQBX9egXHeUd4olRrXiiuhF468UAnyipAy/ECXo7jsh1dF2k4I/AtbD S7cFiIR9wQf8f4kp9XgIl3PTQO6v4Ez3/A== X-Google-Smtp-Source: AKy350Z2PC9zVB+t1rGgWRL74ikHOAYhj1EdKpabNbisfi4nEhLfjYbZhgRy3U2heOy2zhj+WgIR06d2qDQXMQ== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:6c07:0:b0:b8b:eea7:525a with SMTP id h7-20020a256c07000000b00b8beea7525amr9848148ybc.7.1681335323435; Wed, 12 Apr 2023 14:35:23 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:57 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-10-amoorthy@google.com> Subject: [PATCH v3 09/22] KVM: Annotate -EFAULTs from kvm_vcpu_map() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults generated by kvm_vcpu_map(). Signed-off-by: Anish Moorthy --- virt/kvm/kvm_main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 572adba9ad8ed..f3be5aa49829a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2843,8 +2843,10 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) #endif } - if (!hva) + if (!hva) { + kvm_populate_efault_info(vcpu, gfn * PAGE_SIZE, PAGE_SIZE); return -EFAULT; + } map->page = page; map->hva = hva; From patchwork Wed Apr 12 21:34:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A4ECC77B76 for ; Wed, 12 Apr 2023 21:35:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229998AbjDLVfe (ORCPT ); Wed, 12 Apr 2023 17:35:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229663AbjDLVf0 (ORCPT ); Wed, 12 Apr 2023 17:35:26 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CEBA83C0 for ; Wed, 12 Apr 2023 14:35:25 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 132-20020a250c8a000000b00b8f4e12dd57so1884659ybm.1 for ; Wed, 12 Apr 2023 14:35:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335324; x=1683927324; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=fXIZH8fY83vgfwH8POrr//89XoSgOMitG1Dp++mIiPs=; b=YisQpRnBOeAwg9ZX219UZQMhQxuyDfMdNDGwIM7kYemLzsSfZ/SGeZjelJTvKZTX01 h+3Mva+AsoeDLPkEZfXti0QYJCd0Gnxtx2iPS46KcZYJMbtNIpccXd1nkV7JjclKAgOT ZZgzQxYVQewWug5Hu/GM9rA5g0xIWgk9PTD6gv8RLkgLSKnFdVu0S2i0SZwpYwJwRWHr M4RMrS3cKuQowbs4mv/G49o/rXgk/U+bWQ1jnOSm37UF5tkUkoqwpaqgy25/jiVwm4S5 dK6aGhjypYDdse2/jDhX7vXjaJhm5k+2X10ISVVE0X24dQw/ZwqzdXc93uctIAQ/wiBy t+PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335324; x=1683927324; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fXIZH8fY83vgfwH8POrr//89XoSgOMitG1Dp++mIiPs=; b=aTO9V1orO3vKZuCIig4wTgLj5J9WhrhaEteUG0IrNOKz4U+yTi+/ItE3R9nyxYfYW5 qnCT+RacP1WHgOK2p4esDxiPOhUerGd5rxvyzwikOa5aCWfQn7Q3b41sv6WhmDcuL2gs G2H2+YDCEO2bJ8QAElB7KPTsAxSRAVaLfZ4moSSKfX90Y2sHYCMX0D7DItXcXAdCe82q QN+RlDUYk9CvLb3jB77stjuRSHqVUcVEN2iUT7Vg6Ec8BcN78ll5ZO1jfL6NgNXlTgHE W+Oe0uRTu1X9M9p8bMkEZJZj229CDCE7UyPX6x+hTIwUtwjbPA/CcQmIv7keAkYZPEXH HTtQ== X-Gm-Message-State: AAQBX9cmQB00v6xlJ4N85pfbF/SgN5Qj1NwqWTuJkFY/DR/XonZt1JV6 J+epPxnDe97NAlC3R3SVq42rWGyHE6lhqg== X-Google-Smtp-Source: AKy350bhXrkwYHL2oiwHoVZPq1a1kZqsww17x6UiZcbBFdt3mgWFNOOcT1fYEgc+pglz1McVDcJD24DYEi1ceQ== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a0d:ec02:0:b0:54c:2723:560d with SMTP id q2-20020a0dec02000000b0054c2723560dmr8663ywn.3.1681335324351; Wed, 12 Apr 2023 14:35:24 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:58 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-11-amoorthy@google.com> Subject: [PATCH v3 10/22] KVM: x86: Annotate -EFAULTs from kvm_mmu_page_fault() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults generated by kvm_mmu_page_fault(). Signed-off-by: Anish Moorthy --- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 144c5a01cd778..7391d1f75149d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5670,6 +5670,9 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 err return -EIO; } + if (r == -EFAULT) + kvm_populate_efault_info(vcpu, round_down(cr2_or_gpa, PAGE_SIZE), + PAGE_SIZE); if (r < 0) return r; if (r != RET_PF_EMULATE) From patchwork Wed Apr 12 21:34:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5FF7C7619A for ; Wed, 12 Apr 2023 21:35:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230031AbjDLVff (ORCPT ); Wed, 12 Apr 2023 17:35:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229897AbjDLVf0 (ORCPT ); Wed, 12 Apr 2023 17:35:26 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8D6C76BB for ; Wed, 12 Apr 2023 14:35:25 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-54f8d4f1ca1so37088367b3.20 for ; Wed, 12 Apr 2023 14:35:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335325; x=1683927325; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+2CX14Kj8/FFXl+IH0W/8FGtmxWGznSGRKdbhRDJLUc=; b=p8bqpn4Dk9Gz6MN4ywNEmQtagtzpLIxrNnIs/78XOR/BZCZsaubThr5jZlBe3A9/Bo xUGwI26ti8RR6P8GdBzrDgrFhOrTjkeh9AtMCg/Q9PFIi3EAVKhB8I8p0MNXBEhdx058 qHIdK5MhwqEdX2g5KKNjLJnb/mhhyuldqk8s0eAqYV4nAchHoT628Fnj3jVtkCgj0KSZ NQjg5DjX1iidMjq4DxkyjK7XFXry1UsIBkfsDw0DlcBJI06XomtB15gsVc+JjM+Pceog lgxYPlOh7M/77mtkNft0npiYroONt+2TYP1Jm1lm0UlYFodlTIf10FXcOnCumbISliIr sCqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335325; x=1683927325; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+2CX14Kj8/FFXl+IH0W/8FGtmxWGznSGRKdbhRDJLUc=; b=ChxqHyQZSKqz5TqsuXZaMTT/mL/PvWSVMVYgk1nQkB8bJhVIuHyHxZq1ySsjJZhTnO c71pjIvklr9VrjFhr48A+vwfg8vuTRgYQ1kdwXoA+undClk1ygOwfLeo6s4Qn97c2PMS tZQxYkByQmh8Wv3wTQb+bqnyy3wsZ5SDreq4cf1NZTr5swJ3t8ZX1HumBPqlMe8D0lzZ oJXgrrmj0BHNYWz+7ynHqoXxZ374jJOREIhy9gx79vv4hKGdknrEbUCr/JSW/d+3IUKE rn4dBpwsF3DHhbMTDaY9Ai5yDoab2/3+xOqvjw9CQHR5spwfFQlRjuRnhaYPjjKBK80x v5ig== X-Gm-Message-State: AAQBX9cz7DUKbXL4Mqtm+/3gfungVwJX0ATtM8tmp8vsR3ju9YsZBNvJ Xbxos6Ea35Bx2miZz2X+iNSEzuAL5mY/pA== X-Google-Smtp-Source: AKy350ZoVAtaYWZqyK/PKLeLB7f5y1zErH4zuwClH5USqCfQLe7jp2ooTV3DJsYLDFSWaoh1ky/XqCmcJ18/XA== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a0d:ec02:0:b0:54c:2723:560d with SMTP id q2-20020a0dec02000000b0054c2723560dmr8679ywn.3.1681335325180; Wed, 12 Apr 2023 14:35:25 -0700 (PDT) Date: Wed, 12 Apr 2023 21:34:59 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-12-amoorthy@google.com> Subject: [PATCH v3 11/22] KVM: x86: Annotate -EFAULTs from setup_vmgexit_scratch() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults generated by setup_vmgexit_scratch(). Signed-off-by: Anish Moorthy --- arch/x86/kvm/svm/sev.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index c25aeb550cd97..9ef121f71dc26 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -2683,6 +2683,7 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len) pr_err("vmgexit: kvm_read_guest for scratch area failed\n"); kvfree(scratch_va); + kvm_populate_efault_info(&svm->vcpu, scratch_gpa_beg, len); return -EFAULT; } From patchwork Wed Apr 12 21:35:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2724AC77B6C for ; Wed, 12 Apr 2023 21:35:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230092AbjDLVfh (ORCPT ); Wed, 12 Apr 2023 17:35:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229886AbjDLVf2 (ORCPT ); Wed, 12 Apr 2023 17:35:28 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C06C676A6 for ; Wed, 12 Apr 2023 14:35:26 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id v200-20020a252fd1000000b00b8f548a72bbso862051ybv.9 for ; Wed, 12 Apr 2023 14:35:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335326; x=1683927326; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=j57tzoPl16sC9ByxrSzcSJpXXHgYa+UwHfuGAFzOM3I=; b=hPL8324f0dvuI9qEQzWCYrweNFqgGmr995W+iPkc+3GRVTRuBih5b/Tk+omn/cNuoJ QJdd85nlJ+8gvBlpPMGC2W5MkTQLs/SUWLfnjqPap4MmVSGebtW35hwDOjZMMG8REtua /lSLjpb4oFMt1H05FLzLPNItsS8aB/fITd6xVpVMffErvEaAjrQWdlIKMqUdcPA/heHg VoN+2vk5kF95IclXeJ7opZl6vuf12QAxytef0yL5vmQDLvQICDV+IE0uA8hk6P6YyXbR 8T1OpFbeK2s2/ufzJ+ZIT92Ohgl6rhkbVKN2MsQE2rgAtc+yORcHLQWC9t+kfbYn1oae i8tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335326; x=1683927326; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=j57tzoPl16sC9ByxrSzcSJpXXHgYa+UwHfuGAFzOM3I=; b=M65Hp8Ox/1/Oyc8wiUCBg0vuVIUJR2zCQ/O/f5fiiTIbU8p2vjiIvA2jmsAUG9E0IR Dqh5zLVUEQkpEySBAMP5p+1qAT0QjMW94Jasy27hwvZchbS7nKGjLQeHoTgT1us4har7 HZTdS65DySfFO/QjMSPikH8sm+k2XD2czNdrYP+vQbFtjKEPuouWRSo/L6ZDu9z2e44W NbIysZgQVLxi5ntOSPSdOj2SO7/lorjmdlqcoUD7WQcecyV9m+IFlFqYKSmxnJ1VkhCJ V/A6TpoWLhAeoYUj4mtj9I2b3t7d6sJ/2k7pKNiDQE2PP7jOsm+Dw+Lk83u7YHsn92JW 8T1g== X-Gm-Message-State: AAQBX9cFtBc4kv5r3U9qZ6z8sEjLku8LBxYg1E9NHScp8hCg14HPfdDa 72jEAi/tk+RmS2IcLlgrG3IOczvqvqu/Bg== X-Google-Smtp-Source: AKy350YnELcVqu3F38lVRb+xr7cHeJqZPpveoy9dsreWoQEbFvuQJOfQLeL37xkM4q0cAPIFSLawdQnrHh266A== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a81:4319:0:b0:545:62cb:3bcf with SMTP id q25-20020a814319000000b0054562cb3bcfmr12058ywa.2.1681335326021; Wed, 12 Apr 2023 14:35:26 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:00 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-13-amoorthy@google.com> Subject: [PATCH v3 12/22] KVM: x86: Annotate -EFAULTs from kvm_handle_page_fault() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for -EFAULTs caused by kvm_handle_page_fault(). Signed-off-by: Anish Moorthy --- arch/x86/kvm/mmu/mmu.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 7391d1f75149d..937329bee654e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4371,8 +4371,11 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code, #ifndef CONFIG_X86_64 /* A 64-bit CR2 should be impossible on 32-bit KVM. */ - if (WARN_ON_ONCE(fault_address >> 32)) + if (WARN_ON_ONCE(fault_address >> 32)) { + kvm_populate_efault_info(vcpu, round_down(fault_address, PAGE_SIZE), + PAGE_SIZE); return -EFAULT; + } #endif vcpu->arch.l1tf_flush_l1d = true; From patchwork Wed Apr 12 21:35:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 164B5C77B6E for ; Wed, 12 Apr 2023 21:35:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230029AbjDLVfi (ORCPT ); Wed, 12 Apr 2023 17:35:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230017AbjDLVf3 (ORCPT ); Wed, 12 Apr 2023 17:35:29 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCE037ABC for ; Wed, 12 Apr 2023 14:35:27 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 184-20020a2515c1000000b009419f64f6afso13842887ybv.2 for ; Wed, 12 Apr 2023 14:35:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335327; x=1683927327; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=y3j9qgUxhtl7kPmvN8p0uLV24AtP48c1UXcpkvJrSL4=; b=znOhPv9b3IaUt2OaMla7s/u7LvnQoRk71ovHLPBcB8TaKOJZW4sc0PrZ/iCJOA3dpf tOqbawMve8/s5KX/HdPG1vEStSOt72Kmw4r6I2CRzt7aTV/loRc9v7u5y8AVoh1knwY0 v5sr8nQ1FxhezlnNZBhcPhVGpylmLUDVNSAM6t0u+jRxkU2Qg/s3CPvYEq2oX8PLWAtn tbkqa47HvyfkWxPETB5zM2l4G6tE6nbgMjc7G1eq5ig+UgsglZAiNB7MUcWCOP8OGif5 IfWnPeuEFjIxy8esEjfZTnnBq4y3XPxCzKgnWXWNH8KBXL9L5a1ZJgPd6rgKBK0JIMGp 6Blw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335327; x=1683927327; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=y3j9qgUxhtl7kPmvN8p0uLV24AtP48c1UXcpkvJrSL4=; b=dFhM9/ivuF5iH9qP/IHuuvwFj2W/SstW4Sf6L6Gkf9lP3k3kKo9mKkFQbG0Psph+te eimmy2+lEMWnX5EjGtDnu8JErRDYdWO3MHeE4IR9GoKbNGmWa29+aTo6yof8/LV5gAjV RByZyWAvyOMd+KacnTkpEojkYJQ0HGv3GcLksCestdHA1wXXdl0QHJjMgbRnRV4mXkY0 Y8kUPt7fTosSkb8VMu0U6Yocl0Xv5sNuzsbV5/66iE6RGG1BWNhk2G4kOpsZB7CX+Mt1 9F6XmFAF0AhbS2as/zRygrqILib6I73UABZVY7Kje2pbf3KsMGk1tvbN/ZX9Tkmeslo+ gWhQ== X-Gm-Message-State: AAQBX9fRPaaCrGS53Nah2Nf0cKQbochzrw4nihUnhT1NHP+0ckaroD7c qtmHYWfEB1E7SQSxXZYPrIMbfOjGVQNoVQ== X-Google-Smtp-Source: AKy350YMWdkMCSKq761pJsOgwQGGxFRMIrQTIZEfBjuIxIleVcBDx4zXgdOdDVSMo9Fr8whNhn/grob/2f4/dQ== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a05:690c:706:b0:545:5f92:f7ee with SMTP id bs6-20020a05690c070600b005455f92f7eemr16230ywb.2.1681335326987; Wed, 12 Apr 2023 14:35:26 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:01 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-14-amoorthy@google.com> Subject: [PATCH v3 13/22] KVM: x86: Annotate -EFAULTs from kvm_hv_get_assist_page() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults generated by kvm_hv_get_assist_page(). Signed-off-by: Anish Moorthy --- arch/x86/kvm/hyperv.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index b28fd020066f6..467fff271bc88 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -921,13 +921,21 @@ EXPORT_SYMBOL_GPL(kvm_hv_assist_page_enabled); int kvm_hv_get_assist_page(struct kvm_vcpu *vcpu) { + int ret = -EFAULT; struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); if (!hv_vcpu || !kvm_hv_assist_page_enabled(vcpu)) - return -EFAULT; + goto out; + + ret = kvm_read_guest_cached(vcpu->kvm, &vcpu->arch.pv_eoi.data, + &hv_vcpu->vp_assist_page, + sizeof(struct hv_vp_assist_page)); - return kvm_read_guest_cached(vcpu->kvm, &vcpu->arch.pv_eoi.data, - &hv_vcpu->vp_assist_page, sizeof(struct hv_vp_assist_page)); +out: + if (ret == -EFAULT) + kvm_populate_efault_info(vcpu, vcpu->arch.pv_eoi.data.gpa, + vcpu->arch.pv_eoi.data.len); + return ret; } EXPORT_SYMBOL_GPL(kvm_hv_get_assist_page); From patchwork Wed Apr 12 21:35:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 762B6C7619A for ; Wed, 12 Apr 2023 21:35:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230112AbjDLVfj (ORCPT ); Wed, 12 Apr 2023 17:35:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229994AbjDLVfa (ORCPT ); Wed, 12 Apr 2023 17:35:30 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D3338685 for ; Wed, 12 Apr 2023 14:35:28 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id l124-20020a252582000000b00b8f5572bcdaso526809ybl.13 for ; Wed, 12 Apr 2023 14:35:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335327; x=1683927327; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dhrsBiTuRjQtjQ4EZddQuAWvN8rC8eA79u9jMPngMl0=; b=pzM8+7lUWwg1ctdt2epdb63FhOV76MRpYAvlOaDzO4lPffbgXGSciySvkTLGMABUo5 4Ix+ciTJNGaKEFiCj5wXKtvT6Ld76sG3TFtk+fxp7YvW4cU6b8MuIBAUCI0slkvN9HyC qoL691NessVKP39louyP6+8hEG1m2ZzP+9/pg4vgCFSv8hrSWHLtxWIezGImYBdrqoeE 9ouBzPUcFw3az2ZP6o+SHnsVaHJjjShgRZVyCZvErV2v6Q143Z73O9BGkTpxm5DHKKlt ehC4CUaXrwW+RId5xZYKrp13x89+m+8HIZn1FgRcIsaIYjOODmDqahB9EnxcEdndyFLJ F8HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335327; x=1683927327; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dhrsBiTuRjQtjQ4EZddQuAWvN8rC8eA79u9jMPngMl0=; b=froTyeCgGy94lLEuEbC+T6RmLihXZHwKF4WthzPuq727AUhDp4kykAhI3RUZNo3v4X I6d3hfSBlsEW5DNZ3i0ULn2oTfu2QtIgu9u0ATHq5e0LkyCOHWGF0hBL80UvtcYisdB+ pDQ0DReA3f9zfY8NiyJH5K1SwnskHv2KB+3CbV1CO6qq53BTK6RL6UEtwWcNSNmdVNj6 OsFBH3wB7oDg7o37Xcusx+pYDuVKG0gpIADvS1b4hkOo5MR+9uWwbXiOBVkA38uU1VgR JTrv2EAjcV+RJgF5Zp1QaTxTJFAx5cYLH11CQVuAWfNk/dF0bv807Wsv+QfV0D8bMbyw KDWw== X-Gm-Message-State: AAQBX9dxlVS348yV26A+uh5zNseejbMPSda36sgGAuNANLhD02jKe8bp +2r8Yr2ncashxtN5mKGYB5u8vt7n+GeO2w== X-Google-Smtp-Source: AKy350Yy9fFTdXwRu581MJp8gUoz3r22sI/d6sx2YB+r1CcqrBU9VrdlOkwBv38wUmogzZeQ3SFDc3UDjxFb+w== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a05:690c:706:b0:545:5f92:f7ee with SMTP id bs6-20020a05690c070600b005455f92f7eemr16253ywb.2.1681335327843; Wed, 12 Apr 2023 14:35:27 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:02 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-15-amoorthy@google.com> Subject: [PATCH v3 14/22] KVM: x86: Annotate -EFAULTs from kvm_pv_clock_pairing() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults generated by kvm_pv_clock_pairing(). Signed-off-by: Anish Moorthy --- arch/x86/kvm/x86.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0925678e741de..3e9deab31e1c8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9589,8 +9589,10 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, ret = 0; if (kvm_write_guest(vcpu->kvm, paddr, &clock_pairing, - sizeof(struct kvm_clock_pairing))) + sizeof(struct kvm_clock_pairing))) { + kvm_populate_efault_info(vcpu, paddr, sizeof(struct kvm_clock_pairing)); ret = -KVM_EFAULT; + } return ret; } From patchwork Wed Apr 12 21:35:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BE8FC77B73 for ; Wed, 12 Apr 2023 21:35:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230114AbjDLVfk (ORCPT ); Wed, 12 Apr 2023 17:35:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229989AbjDLVfb (ORCPT ); Wed, 12 Apr 2023 17:35:31 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18EF37A93 for ; Wed, 12 Apr 2023 14:35:28 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 132-20020a250c8a000000b00b8f4e12dd57so1884845ybm.1 for ; Wed, 12 Apr 2023 14:35:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335328; x=1683927328; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=TLJM370BGXbRsW/E5uGIyc+ccmSwnIlQ6m8okICmE78=; b=wejiGHqZGIyNR3UMUoT0jh07Z2bqhz0x6OsEUUNrr/QJTEbD4p/twiQXgY7TdOeW84 8uYvht+OkX6pUYnlSv0QTlb+0rBH6xO0DMty619puVSkmD9LfcU7/ZJuGj5erof3tjgc tR4JbovReXeRwmjus6dOnaSqXAgkTQ2aiu1XDf9Iog5WoqmLBa3dEinf347k61B37MrW s0bisQNQP7raE75xNPpWge3XlaaSBqJazbGtqjePh5xq79nyekhI/XMw6rGKBWsnaTyN +9Nba0FpzQij+M4tnFOzOt0qWcslF0JfO/3Cbz43eB6yXUbXmmsoo8wxVnChKVOUPWgF Bieg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335328; x=1683927328; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TLJM370BGXbRsW/E5uGIyc+ccmSwnIlQ6m8okICmE78=; b=Hi1Uc5q0+mGgPUKFM85YOCam51FEaXBZd1XeSQ1HyY9nWqEODFGWPEiaMvKL7XhFTr nOz5x5ZcT4PTdrr+WU5r+0xSrrAptJHzHjEapxZwLgoHdbo96/Szp2ieMfqXO1n56Jyv 9aLwGTOv3WFgfM8HSB+j747ZSChqZSTwtTL0QC/QmCxHCxpZrPltO9tQ4edW6rTog4ZQ /d6Khx5yJzOHBXk7FjTP+bRXrYAiBvJdA/ipGZzGIK3cznTDdOpSYYIHsq4y92y2w0vF uYljO1dDLeOCb5ZFCGnf5aQGB6PKJOCzVN2QVQbzsGkEyBvrdG3ggLg41VxGqjhTNxlC gNpg== X-Gm-Message-State: AAQBX9edJQ51xzMAuGnMeLNAQBjmw1fzJaMXjXIVh4XrNuzQ4nZACdhf TE5ukz1888l76ijCQBZQVu52Dbq6yCauDg== X-Google-Smtp-Source: AKy350Zz+xHx97O82en66JkvO0/xWQmHy3qKXiGdMYTUAU8j1OkF3SFZDNZl0zwMBp52T3C8+gmB+jaY/ERp4A== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a81:a809:0:b0:544:bce8:980f with SMTP id f9-20020a81a809000000b00544bce8980fmr4950839ywh.6.1681335328669; Wed, 12 Apr 2023 14:35:28 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:03 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-16-amoorthy@google.com> Subject: [PATCH v3 15/22] KVM: x86: Annotate -EFAULTs from direct_map() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults generated by direct_map(). Since direct_map() traverses multiple levels of the shadow page table, it seems like there are actually two correct guest physical address ranges which could be provided. 1. A smaller range, more specific range, which potentially only corresponds to a part of what could not be mapped. start = gfn_round_for_level(fault->gfn, fault->goal_level) length = KVM_PAGES_PER_HPAGE(fault->goal_level) 2. The entire range which could not be mapped start = gfn_round_for_level(fault->gfn, fault->goal_level) length = KVM_PAGES_PER_HPAGE(fault->goal_level) Take the first approach, although it's possible the second is actually preferable. Signed-off-by: Anish Moorthy --- arch/x86/kvm/mmu/mmu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 937329bee654e..a965c048edde8 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3192,8 +3192,13 @@ static int direct_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) fault->req_level >= it.level); } - if (WARN_ON_ONCE(it.level != fault->goal_level)) + if (WARN_ON_ONCE(it.level != fault->goal_level)) { + gfn_t rounded_gfn = gfn_round_for_level(fault->gfn, fault->goal_level); + uint64_t len = KVM_PAGES_PER_HPAGE(fault->goal_level); + + kvm_populate_efault_info(vcpu, rounded_gfn, len); return -EFAULT; + } ret = mmu_set_spte(vcpu, fault->slot, it.sptep, ACC_ALL, base_gfn, fault->pfn, fault); From patchwork Wed Apr 12 21:35:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 581BBC7619A for ; Wed, 12 Apr 2023 21:35:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230099AbjDLVfm (ORCPT ); Wed, 12 Apr 2023 17:35:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbjDLVfb (ORCPT ); Wed, 12 Apr 2023 17:35:31 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F051A83D7 for ; Wed, 12 Apr 2023 14:35:29 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-54f87e44598so46588517b3.5 for ; Wed, 12 Apr 2023 14:35:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335329; x=1683927329; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=TMoObdMgGKK1Dm08+sln58kp5QBYq24ihR/AEV1awlI=; b=rh+/+zw3Za1gxuRrVQJEEjhupmhmknQ/4pC4YKJQPjw/IBRrbOGZRl6uPFNdSfQIAg rLRvo/QL7yxEQnzBbcuQBBulo8e4IqcIDuwrrNv3C0inbPr8YZJ3V2T0PMO7P/s2222w d01Y596uxKiYiIi8HMOCx+Gc6YGpSjv6bd5ixrhHyr4X9rQi/uIHGmOmrslt7LEJz9fo qX+bWNOFqUeXEe/rqEzR3O3uzwVf204tZ0Js1kWQJ/+NXNbBWyLxdRPaW1+XPkDRtZCw yW7FfsafpaPKaeAu5nHfz4+R6ula5hXzkCq8pZtMZha3Oq8Hmn5CgaruZT0WhQSxPx1A 7FJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335329; x=1683927329; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TMoObdMgGKK1Dm08+sln58kp5QBYq24ihR/AEV1awlI=; b=f5QbsukO8eW6HtIvGiQtDw/W3YcFsGeJpUkRMEJMM9JgSHfLURUJ+5ll7a5vuP46x8 Uvs/Ttzs4Y9f1AoSAbAiPBWpTkHECAMQgf74LpqnegpZaLNSWYnTKvFjfGzPf/hSNuA/ uFtb2Gj8sLXsM9qqnH7LLJDvV8/PdGgOpZcQRtnXw1GG0pOhW/b+RlCLoJBj4EhVx2cA D9m3Jeh51sGecYUw708MCa2uJkU9YKAKsM9g4jyIz7+7C0/cEx8sc28RBuG4Jiq+RkJc FMzBTnV3I7D3BllMLDqTW5753vG5IkKAJxzZEqFpxj7c0fOgmvVCnwfSl5JEQ1QVW32E pMUw== X-Gm-Message-State: AAQBX9dkEu5hpHRE0l/6SO1aadD57KCVT7RCBF4IPy85GzNGcRluJjSv BCXLxxzhW4PiOO8n2B1ZUQuERDbqV5Oj+A== X-Google-Smtp-Source: AKy350awPEdXfzeyIZrkWycnGmb1wsYa+P7TPbbAsrvpvFBbHnKQbFgfxIvBnw6PFjQaDHuHgaXdgnJo4I8u5w== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a0d:ec50:0:b0:54d:913b:c9e8 with SMTP id r16-20020a0dec50000000b0054d913bc9e8mr8999819ywn.5.1681335329588; Wed, 12 Apr 2023 14:35:29 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:04 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-17-amoorthy@google.com> Subject: [PATCH v3 16/22] KVM: x86: Annotate -EFAULTs from kvm_handle_error_pfn() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for efaults generated by kvm_handle_error_pfn(). Signed-off-by: Anish Moorthy --- arch/x86/kvm/mmu/mmu.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a965c048edde8..d83a3e1e3eff9 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3218,6 +3218,9 @@ static void kvm_send_hwpoison_signal(struct kvm_memory_slot *slot, gfn_t gfn) static int kvm_handle_error_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { + uint64_t rounded_gfn; + uint64_t fault_size; + if (is_sigpending_pfn(fault->pfn)) { kvm_handle_signal_exit(vcpu); return -EINTR; @@ -3236,6 +3239,10 @@ static int kvm_handle_error_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fa return RET_PF_RETRY; } + fault_size = KVM_HPAGE_SIZE(fault->goal_level); + rounded_gfn = round_down(fault->gfn * PAGE_SIZE, fault_size); + + kvm_populate_efault_info(vcpu, rounded_gfn, fault_size); return -EFAULT; } From patchwork Wed Apr 12 21:35:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1545AC77B6C for ; Wed, 12 Apr 2023 21:35:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230111AbjDLVfn (ORCPT ); Wed, 12 Apr 2023 17:35:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229624AbjDLVff (ORCPT ); Wed, 12 Apr 2023 17:35:35 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EE6E86BA for ; Wed, 12 Apr 2023 14:35:31 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-54f9df9ebc5so27948487b3.13 for ; Wed, 12 Apr 2023 14:35:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335330; x=1683927330; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=IWH4ZRYVMjQ/mFRBZ49V9jlN8+sDpTIhwIJQUsNqRO4=; b=a0WlNccitJbMxtr99iHu5o5qhkB8GZPfinmSqjqpd26vcPKVaA0f8FWlEXKgCaCozq lZcKVgjiloiKSP+94bh2Asa7Rzt96bF9dgcd7xvBR9IN2QY3eGcsL5+7G0vaHw7y+FCU YgnCKxCxaH1bVVo0XNIEPCMbmMyjUJbRPv+T/soClb8P7YAtwnOZugsxGgf8KiCyzHuL h/ds/bxsBNMyYc7iboS3PftQei5tre/9z7VYV0idgPUC9U9r2QqY5lg+8y0PAlsDn/wn KFRtSyB7hPb4UnfDKhIcEXaAML6W3xbjI47GD83yBNOdQOBq53dlKy9GgKj/qJPSYue4 vLUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335330; x=1683927330; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IWH4ZRYVMjQ/mFRBZ49V9jlN8+sDpTIhwIJQUsNqRO4=; b=RHGH1mj/te5XoxRqUpukryQ+Suoq6vP32A6qHGnWIPgMKDsXukbV0bcL/2zBqROTc4 Yhs53qt1bahcfqramGUCC2Kx2HGhBq65ZBs3jBspHyqG61o/P2d+hABQIkP627G3yO4U YFGwM4jwYAQWH3fntHLt7b9CRA+pdZreWbXFKT6ffPCC/dm5yo5oZFKR96X0CzSAQ/Qw TpJ7oN2/Yn4AINFe8son4V0l01vJ/NlnxS6bYu9lzII/hY9+HtMyv5wQiFa04p7sco3a BMt8T9kGuGbdh5bLFu/HJm16ErRUthYz1iQt0yJfogPbqq8adHKHHO5jksFKUNyBJiEe IaKA== X-Gm-Message-State: AAQBX9fsj2WIiyfeCiR66SXWU1EHj/1xWiHDd4mEBZeCC32we6V4sTUF SZDJnlv1QKdpUPr6nkSYym39LGXdz6ShVA== X-Google-Smtp-Source: AKy350ar/HxRwT7YIUsCVGv0rPiDiH7lT+T3hSEZTIraxERXKIZKg70zugZhlLgV06/iho9mUA0D+Oe1xt+ZXA== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:cc07:0:b0:b8b:fe5f:2eaa with SMTP id l7-20020a25cc07000000b00b8bfe5f2eaamr15871ybf.2.1681335330526; Wed, 12 Apr 2023 14:35:30 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:05 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-18-amoorthy@google.com> Subject: [PATCH v3 17/22] KVM: Introduce KVM_CAP_ABSENT_MAPPING_FAULT without implementation From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add documentation, memslot flags, useful helper functions, and the actual new capability itself. Memory fault exits on absent mappings are particularly useful for userfaultfd-based postcopy live migration. When many vCPUs fault on a single userfaultfd the faults can take a while to surface to userspace due to having to contend for uffd wait queue locks. Bypassing the uffd entirely by returning information directly to the vCPU exit avoids this contention and improves the fault rate. Suggested-by: James Houghton Signed-off-by: Anish Moorthy --- Documentation/virt/kvm/api.rst | 31 ++++++++++++++++++++++++++++--- include/linux/kvm_host.h | 7 +++++++ include/uapi/linux/kvm.h | 2 ++ tools/include/uapi/linux/kvm.h | 1 + virt/kvm/kvm_main.c | 3 +++ 5 files changed, 41 insertions(+), 3 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index f174f43c38d45..7967b9909e28b 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -1312,6 +1312,7 @@ yet and must be cleared on entry. /* for kvm_userspace_memory_region::flags */ #define KVM_MEM_LOG_DIRTY_PAGES (1UL << 0) #define KVM_MEM_READONLY (1UL << 1) + #define KVM_MEM_ABSENT_MAPPING_FAULT (1UL << 2) This ioctl allows the user to create, modify or delete a guest physical memory slot. Bits 0-15 of "slot" specify the slot id and this value @@ -1342,12 +1343,15 @@ It is recommended that the lower 21 bits of guest_phys_addr and userspace_addr be identical. This allows large pages in the guest to be backed by large pages in the host. -The flags field supports two flags: KVM_MEM_LOG_DIRTY_PAGES and -KVM_MEM_READONLY. The former can be set to instruct KVM to keep track of +The flags field supports three flags + +1. KVM_MEM_LOG_DIRTY_PAGES: can be set to instruct KVM to keep track of writes to memory within the slot. See KVM_GET_DIRTY_LOG ioctl to know how to -use it. The latter can be set, if KVM_CAP_READONLY_MEM capability allows it, +use it. +2. KVM_MEM_READONLY: can be set, if KVM_CAP_READONLY_MEM capability allows it, to make a new slot read-only. In this case, writes to this memory will be posted to userspace as KVM_EXIT_MMIO exits. +3. KVM_MEM_ABSENT_MAPPING_FAULT: see KVM_CAP_ABSENT_MAPPING_FAULT for details. When the KVM_CAP_SYNC_MMU capability is available, changes in the backing of the memory region are automatically reflected into the guest. For example, an @@ -7705,6 +7709,27 @@ userspace may receive "bare" EFAULTs (i.e. exit reason != KVM_EXIT_MEMORY_FAULT) from KVM_RUN. These should be considered bugs and reported to the maintainers. +7.35 KVM_CAP_ABSENT_MAPPING_FAULT +--------------------------------- + +:Architectures: None +:Returns: -EINVAL. + +The presence of this capability indicates that userspace may pass the +KVM_MEM_ABSENT_MAPPING_FAULT flag to KVM_SET_USER_MEMORY_REGION to cause KVM_RUN +to fail (-EFAULT) in response to page faults for which the userspace page tables +do not contain present mappings. Attempting to enable the capability directly +will fail. + +The range of guest physical memory causing the fault is advertised to userspace +through KVM_CAP_MEMORY_FAULT_INFO (if it is enabled). + +Userspace should determine how best to make the mapping present, then take +appropriate action. For instance, in the case of absent mappings this might +involve establishing the mapping for the first time via UFFDIO_COPY/CONTINUE or +faulting the mapping in using MADV_POPULATE_READ/WRITE. After establishing the +mapping, userspace can return to KVM to retry the previous memory access. + 8. Other capabilities. ====================== diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 776f9713f3921..2407fc1e52ab8 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2289,4 +2289,11 @@ static inline void kvm_account_pgtable_pages(void *virt, int nr) */ inline void kvm_populate_efault_info(struct kvm_vcpu *vcpu, uint64_t gpa, uint64_t len); + +static inline bool kvm_slot_fault_on_absent_mapping( + const struct kvm_memory_slot *slot) +{ + return slot->flags & KVM_MEM_ABSENT_MAPPING_FAULT; +} + #endif diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index bc73e8381a2bb..21df449e74648 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -102,6 +102,7 @@ struct kvm_userspace_memory_region { */ #define KVM_MEM_LOG_DIRTY_PAGES (1UL << 0) #define KVM_MEM_READONLY (1UL << 1) +#define KVM_MEM_ABSENT_MAPPING_FAULT (1UL << 2) /* for KVM_IRQ_LINE */ struct kvm_irq_level { @@ -1196,6 +1197,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225 #define KVM_CAP_PMU_EVENT_MASKED_EVENTS 226 #define KVM_CAP_MEMORY_FAULT_INFO 227 +#define KVM_CAP_ABSENT_MAPPING_FAULT 228 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h index 5c57796364d65..59219da95634c 100644 --- a/tools/include/uapi/linux/kvm.h +++ b/tools/include/uapi/linux/kvm.h @@ -102,6 +102,7 @@ struct kvm_userspace_memory_region { */ #define KVM_MEM_LOG_DIRTY_PAGES (1UL << 0) #define KVM_MEM_READONLY (1UL << 1) +#define KVM_MEM_ABSENT_MAPPING_FAULT (1UL << 2) /* for KVM_IRQ_LINE */ struct kvm_irq_level { diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f3be5aa49829a..7cd0ad94726df 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1525,6 +1525,9 @@ static int check_memory_region_flags(const struct kvm_userspace_memory_region *m valid_flags |= KVM_MEM_READONLY; #endif + if (kvm_vm_ioctl_check_extension(NULL, KVM_CAP_ABSENT_MAPPING_FAULT)) + valid_flags |= KVM_MEM_ABSENT_MAPPING_FAULT; + if (mem->flags & ~valid_flags) return -EINVAL; From patchwork Wed Apr 12 21:35:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 449F1C77B6E for ; Wed, 12 Apr 2023 21:35:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230033AbjDLVfo (ORCPT ); Wed, 12 Apr 2023 17:35:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230071AbjDLVfi (ORCPT ); Wed, 12 Apr 2023 17:35:38 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5865E7A84 for ; Wed, 12 Apr 2023 14:35:32 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id c67-20020a254e46000000b00b88f1fd158fso29214248ybb.17 for ; Wed, 12 Apr 2023 14:35:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335331; x=1683927331; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=oBWUDJru52jdEOiurtVr+EjXu6VveYkIMdfENGYuqF4=; b=5Ov5+ZRfva2c/yhjYr6uvNwVs19NXcF+87yc5OV+Twmw1QDkz0RDskvnZST3sAoaOa 2DLJh3OJcPj62W7lWGXrhBgFBBN2ym5EzdxJy5WNiNWrRhDCQ7O9J4sD3IdsSDHkNkXH IlWrqpFtk2bhPyl/fjbcuDlyAw9LBNJtjlC4/XKTUmyPx+21UfU5rIDVX+W5CyqAo7OS gCrK6y4+iCGyxDx3qAIagEau4i7Q7ALYo2KpLBHy09J7o1/ugwnLzj5u7JBFxpuMMCcF 35wLNLvY8wChWZBVKEsf6irGy49x6q/APUlPlzGWacJC0sJ+CAjiGjEWDZ7ty6c3fUT1 QQRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335331; x=1683927331; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oBWUDJru52jdEOiurtVr+EjXu6VveYkIMdfENGYuqF4=; b=GUzLXObp272/ZR1jf0t32P1mcbHrioqbzmrDhgLjj5Nz2NQM32rE/fYIRwkfMyweeJ 9GlYV7aO0uUg3159zgqCuXbN3koU64kWSYmIBV124zxGqE9MXU4A0Rrb8RmGgOsJMpW6 G3koattla0hkIeNItQhoALKl2icRyypjoYIqS1bMluflhERShHieK13DgTj+AXb4m1bO CtgJHGu8ylUnbwemipgUJDRmg30BVyhLBiAk1asg/L7qyN1D3ZjBPFpZK1J0XTcZNs1m EZSuFC1Ob72ISwyPkgAC0YBfA6GxYR6wxbxT/ju+TyeD8WTgPkq0FUpMPlITBT6a6y0K qxJA== X-Gm-Message-State: AAQBX9cSufaWNlDB5yvcmH6sDPIIZgJI07l9jgn1L2kdJ+qG7w8ifVBk vwHJYt72rbAjLhuTcNGPl1RxcsefkegUbQ== X-Google-Smtp-Source: AKy350Z1Hz16awnvahWD8mb/lMbNByADtMsMOxZyj1zA+7AE5/pqJYBUSB/Hd6Gwg+5ip516oAEqf4bZn9oW7Q== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a81:ae1d:0:b0:54f:84c0:93ff with SMTP id m29-20020a81ae1d000000b0054f84c093ffmr3294464ywh.5.1681335331269; Wed, 12 Apr 2023 14:35:31 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:06 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-19-amoorthy@google.com> Subject: [PATCH v3 18/22] KVM: x86: Implement KVM_CAP_ABSENT_MAPPING_FAULT From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the memslot flag is enabled, fail guest memory accesses for which fast-gup fails (ie, for which the mappings are not present). Suggested-by: James Houghton Signed-off-by: Anish Moorthy --- Documentation/virt/kvm/api.rst | 2 +- arch/x86/kvm/mmu/mmu.c | 17 ++++++++++++----- arch/x86/kvm/x86.c | 1 + 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 7967b9909e28b..452bbca800b15 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -7712,7 +7712,7 @@ reported to the maintainers. 7.35 KVM_CAP_ABSENT_MAPPING_FAULT --------------------------------- -:Architectures: None +:Architectures: x86 :Returns: -EINVAL. The presence of this capability indicates that userspace may pass the diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index d83a3e1e3eff9..4aef79b97c985 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4218,7 +4218,9 @@ void kvm_arch_async_page_ready(struct kvm_vcpu *vcpu, struct kvm_async_pf *work) kvm_mmu_do_page_fault(vcpu, work->cr2_or_gpa, 0, true, NULL); } -static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) +static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, + bool fault_on_absent_mapping) { struct kvm_memory_slot *slot = fault->slot; bool async; @@ -4251,9 +4253,12 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault } async = false; - fault->pfn = __gfn_to_pfn_memslot(slot, fault->gfn, false, false, &async, - fault->write, &fault->map_writable, - &fault->hva); + + fault->pfn = __gfn_to_pfn_memslot(slot, fault->gfn, + fault_on_absent_mapping, false, + fault_on_absent_mapping ? NULL : &async, + fault->write, &fault->map_writable, &fault->hva); + if (!async) return RET_PF_CONTINUE; /* *pfn has correct page already */ @@ -4287,7 +4292,9 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, fault->mmu_seq = vcpu->kvm->mmu_invalidate_seq; smp_rmb(); - ret = __kvm_faultin_pfn(vcpu, fault); + ret = __kvm_faultin_pfn(vcpu, fault, + likely(fault->slot) + && kvm_slot_fault_on_absent_mapping(fault->slot)); if (ret != RET_PF_CONTINUE) return ret; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3e9deab31e1c8..bc465cde7acf6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4433,6 +4433,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_ENABLE_CAP: case KVM_CAP_VM_DISABLE_NX_HUGE_PAGES: case KVM_CAP_MEMORY_FAULT_INFO: + case KVM_CAP_ABSENT_MAPPING_FAULT: r = 1; break; case KVM_CAP_EXIT_HYPERCALL: From patchwork Wed Apr 12 21:35:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F749C7619A for ; Wed, 12 Apr 2023 21:35:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230056AbjDLVfp (ORCPT ); Wed, 12 Apr 2023 17:35:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230085AbjDLVfl (ORCPT ); Wed, 12 Apr 2023 17:35:41 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2657883C0 for ; Wed, 12 Apr 2023 14:35:33 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-54f9df9ebc5so27949227b3.13 for ; Wed, 12 Apr 2023 14:35:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335332; x=1683927332; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=fW2lVHfchp9UgxlQ0eqx0Qnt2L2yZQNT4SVI95DvpSs=; b=RbgkvGcRLIPqOOdYTNLhCUUYpP5u2hdsGoV9tBjd/zi1IjMKBqiqA4sTOeL9bmoW81 3H6Qawu5hfjaOHdrZKYJjjAR5wQ054U3LXNw8S1hT7BxcGUsYoBb3c/5ycAOlZhoOdIt dgXzaicpEJwGYoy9qsWynR9W7h9/m4Ym8cnWfYm6bzs75oxhyzRFVeyTI27Ln42QTX+R ETbJEewsFQeSLQ5tUo7roL1hW8q9eBwsa+gSY8ErXEcwEVuBQJ6YFyvwi3NQ8sUBj7jR AoxcJZMbiDGNw5e335EAnuUNdSYzaovS6U4EKDLMkQBvIw6z8nn7Dxs2OReIS8YFbH7v 2ODA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335332; x=1683927332; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fW2lVHfchp9UgxlQ0eqx0Qnt2L2yZQNT4SVI95DvpSs=; b=XflZFsA6zefkBWkBKRH8RHF10TCG8HyEBI7IqVkqfev7q/N/uTjjpM6InVO6NIbfuM Tp6ba45xQXqtkLSfUWARJzkYVdxZZpWE6TYPg2u/+i87GGbSQjpyoJJyQ3GTSgo2aXtg 2/4bmFImlxNjFUpbL8JriDjrXQwQ/luhVOmGfnVi3Dxy6sta3qPPN82bIUntquebalVL Ba0O0isGonmHRlYwsjNBSl41lhKwjp9Lv8Kzw3mOOXvLfmUNzmi3rCD7zJqsd/ta7IEy KmQICrztvmEQzS0xDRZFY+LMKw9ozjSoDT3GQFXsQoeK6sWy34RzkWA3ktDk4D56zywg +wtg== X-Gm-Message-State: AAQBX9eTO7u1YIxOiFgel0NMJ2hr7RD+RLJTzQOiLfA//1J9SsZlJ+AN joh9+wE7P+fOzQ1fZ4sq9h590ThNIworhg== X-Google-Smtp-Source: AKy350YH5KZy2ouOBNOYXjh55lIcz9SLFEFUhOyUV8JCny7/1ROofC6xXNHF+9m2d578/vmRjDjHN/WG7mo03Q== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:30c2:0:b0:b8f:553a:ddfd with SMTP id w185-20020a2530c2000000b00b8f553addfdmr2164ybw.5.1681335332195; Wed, 12 Apr 2023 14:35:32 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:07 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-20-amoorthy@google.com> Subject: [PATCH v3 19/22] KVM: arm64: Annotate (some) -EFAULTs from user_mem_abort() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement KVM_CAP_MEMORY_FAULT_INFO for at least some -EFAULTs returned by user_mem_abort(). Other EFAULTs returned by this function come from before the guest physical address of the fault is calculated: leave those unannotated. Signed-off-by: Anish Moorthy --- arch/arm64/kvm/mmu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 7113587222ffe..d5ae636c26d62 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1307,8 +1307,11 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, kvm_send_hwpoison_signal(hva, vma_shift); return 0; } - if (is_error_noslot_pfn(pfn)) + if (is_error_noslot_pfn(pfn)) { + kvm_populate_efault_info(vcpu, round_down(gfn * PAGE_SIZE, vma_pagesize), + vma_pagesize); return -EFAULT; + } if (kvm_is_device_pfn(pfn)) { /* @@ -1357,6 +1360,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (kvm_vma_mte_allowed(vma)) { sanitise_mte_tags(kvm, pfn, vma_pagesize); } else { + kvm_populate_efault_info(vcpu, + round_down(gfn * PAGE_SIZE, vma_pagesize), vma_pagesize); ret = -EFAULT; goto out_unlock; } From patchwork Wed Apr 12 21:35:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00980C7619A for ; Wed, 12 Apr 2023 21:35:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230126AbjDLVfs (ORCPT ); Wed, 12 Apr 2023 17:35:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230116AbjDLVfq (ORCPT ); Wed, 12 Apr 2023 17:35:46 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48ADF7D9E for ; Wed, 12 Apr 2023 14:35:34 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-54f810e01f5so49505157b3.0 for ; Wed, 12 Apr 2023 14:35:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335333; x=1683927333; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=g4meCJ11bu24h/cx6LqrB3qAGYM0pv712osrkf0fzPk=; b=ksc3Uo2kb7fia25zCArzbWloYH/nh04oxzOH/JXw50ehdelb83OV7185J0dcX5CTUl SGpdnP8yVtEWCbN+YKQ1yw8N8CQUKFw5GeaZzxeaIm8mtKU8dAgwpAVgKJavw0O/BKRr aANPAgurlfqKrwO6NGZX4gGPSuyn7eKZfTRk38L/4yupoynBZqCFaqDAjRlkDb1iUMh5 pC0nfYaS0kiBXS6IZr2a1LpJBUYMVii2A3gBADS7dpYAr1eW2Ktxxidq4/+TMOHkiSL7 o8XlEyDWSf+taexDIlKlaYBbFOU+RE/VhORCsyo31dF0tN3prUOBff0E7D5keRnnyYrE Mj+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335333; x=1683927333; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=g4meCJ11bu24h/cx6LqrB3qAGYM0pv712osrkf0fzPk=; b=eq6Ojvku4F838nihZUO5K/9VowwAmrbr0BCi6kW+pvGvBjsc3nWLCWCAn5m51Jt7nJ txB1SXL5PiE3exJqmkPy6u7HBfLaO2jB/3CRYmxrykQFK70gz4sXszoWadB9j1Uss3BB FKcIQ58Eshqj4t7gUpILZACgtHaDmjA3oDQPI3ofy6gKIPyLwoXkKuGjfBHFXU8SlbR1 I/bjoGdQ2rm8r89nxCwwcpGVHgyW08PE8vj1o3Zjf/gtPfHSomhI2JgQjN2bLMW81aXP /GLJvuEBh45gkz64bPQdyrfflBV0uWBLBFXlhB92s7LOBGqUdmqcMN5qpim3zMTQI9xV ZWOw== X-Gm-Message-State: AAQBX9e+oYUpTo3OyogQYNiAIg8RvzfyLYjNr2g1qw0WDceMjQSXkEle qvqbwxweBMwT5KHZyTrGJAGimoF6R0tzuw== X-Google-Smtp-Source: AKy350ZG6iLC4X3Y7S8xfkDc/hZ3/74579TfJlfN5A+jNG8WvNeg+IGrAA5URu61LfHolsG/pk2OemadO/dyGw== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a81:c94d:0:b0:54b:fd28:c5ff with SMTP id c13-20020a81c94d000000b0054bfd28c5ffmr2814728ywl.3.1681335333086; Wed, 12 Apr 2023 14:35:33 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:08 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-21-amoorthy@google.com> Subject: [PATCH v3 20/22] KVM: arm64: Implement KVM_CAP_ABSENT_MAPPING_FAULT From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Return -EFAULT from user_mem_abort when the memslot flag is enabled and fast GUP fails to find a present mapping for the page. Signed-off-by: Anish Moorthy --- Documentation/virt/kvm/api.rst | 2 +- arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/mmu.c | 11 +++++++++-- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 452bbca800b15..47f728701aca4 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -7712,7 +7712,7 @@ reported to the maintainers. 7.35 KVM_CAP_ABSENT_MAPPING_FAULT --------------------------------- -:Architectures: x86 +:Architectures: x86, arm64 :Returns: -EINVAL. The presence of this capability indicates that userspace may pass the diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index a932346b59f61..c9666d7c6c4ff 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -221,6 +221,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_PTP_KVM: case KVM_CAP_ARM_SYSTEM_SUSPEND: case KVM_CAP_MEMORY_FAULT_INFO: + case KVM_CAP_ABSENT_MAPPING_FAULT: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index d5ae636c26d62..26b9485557056 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1206,6 +1206,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, unsigned long vma_pagesize, fault_granule; enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt; + bool exit_on_memory_fault = kvm_slot_fault_on_absent_mapping(memslot); fault_granule = 1UL << ARM64_HW_PGTABLE_LEVEL_SHIFT(fault_level); write_fault = kvm_is_write_fault(vcpu); @@ -1301,8 +1302,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, */ smp_rmb(); - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, - write_fault, &writable, NULL); + pfn = __gfn_to_pfn_memslot(memslot, gfn, exit_on_memory_fault, false, NULL, + write_fault, &writable, NULL); + + if (exit_on_memory_fault && pfn == KVM_PFN_ERR_FAULT) { + kvm_populate_efault_info(vcpu, + round_down(gfn * PAGE_SIZE, vma_pagesize), vma_pagesize); + return -EFAULT; + } if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; From patchwork Wed Apr 12 21:35:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6316C77B6C for ; Wed, 12 Apr 2023 21:35:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230133AbjDLVfq (ORCPT ); Wed, 12 Apr 2023 17:35:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230071AbjDLVfo (ORCPT ); Wed, 12 Apr 2023 17:35:44 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DE568A7E for ; Wed, 12 Apr 2023 14:35:34 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id h206-20020a2521d7000000b00b8f3681db1eso3993838ybh.11 for ; Wed, 12 Apr 2023 14:35:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335334; x=1683927334; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/Enih2OM3xwbE5JxqMI9mr7glLAzseUE52pRXsIPwpY=; b=LJSLx2GmrRRLif5bbLyYGFln1e+mukhuEhQ8NMHWImtHxszerqlY/NKn+zz1MUyL4g Rlbv9IpPwkTyJx8bnss02J6GJZz2JTwlgyaXgxqHU3tn4LkPOw11BCHr/ArqNWqOxIB2 hHqqC6FrtyMb61n7b+Ls90p9ALsXIFcnVJa/q40H84E0KOXzBt6kpUTjEjtE+ocWlBdY RNd/SE0HnxV79jWKOBVBGLq8MMReIJ4+vmT4abRVkyAkKHk+1t/UXey6drvtWNdrlN4i um/lqceHm0LhijaKXqX57GaVUClwr+zMEqAwGOYFMXAvgIRufjrICOxdPvn8Hhik5V3Q 3yeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335334; x=1683927334; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/Enih2OM3xwbE5JxqMI9mr7glLAzseUE52pRXsIPwpY=; b=Lfi/X/xgHqWCQZfxH9UGPnbgVLQeYIR6T6t5RYYJJYcNQQ2O4xrdadyilZWRb21QTI 3511oEFC3WFyeO2AGKb+SOXgmUCuNLkJlrZ0Vk6ueoKQ392S8GI3lre9+lg8KXvUAhir 49QahhFSrypBiQKEoZwvtX3k2qHRtd1iYWTt+nnxm5YnDXuaxatSMnvkphh7qyiLiy58 3uPXMg9+O9e9kuTrBUKbEgYiXSRn/ctRkuXGUSo2dR5JI+PK5DmvkzUoG6sX299s5hqt T3M1ufbJMX81UuMyDjGdjDXpEkD/jKOuU6I/lmitbmKsSC23FVzeFc9QSDKTaCe+9EVt ry2w== X-Gm-Message-State: AAQBX9fVYF6xmeHhQM/P1+thjPWJeg6R2GywAGhqeqBPc2qsQWyogo1Q VjTPDHzL9+h/1blE1Wk58wDDxbjbesQE4g== X-Google-Smtp-Source: AKy350ZFX3aP1qiu5losJdiyWVTYHetybhx9O9AbXDP6rP4+1JK6fQgdp+4BmbBKGbw0H30C9IftH48afs2buA== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:3354:0:b0:b8e:f726:d198 with SMTP id z81-20020a253354000000b00b8ef726d198mr5949388ybz.8.1681335334026; Wed, 12 Apr 2023 14:35:34 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:09 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-22-amoorthy@google.com> Subject: [PATCH v3 21/22] KVM: selftests: Add memslot_flags parameter to memstress_create_vm() From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Memslot flags aren't currently exposed to the tests, and are just always set to 0. Add a parameter to allow tests to manually set those flags. Signed-off-by: Anish Moorthy --- tools/testing/selftests/kvm/access_tracking_perf_test.c | 2 +- tools/testing/selftests/kvm/demand_paging_test.c | 4 ++-- tools/testing/selftests/kvm/dirty_log_perf_test.c | 2 +- tools/testing/selftests/kvm/include/memstress.h | 2 +- tools/testing/selftests/kvm/lib/memstress.c | 4 ++-- .../testing/selftests/kvm/memslot_modification_stress_test.c | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/tools/testing/selftests/kvm/access_tracking_perf_test.c b/tools/testing/selftests/kvm/access_tracking_perf_test.c index 3c7defd34f567..b51656b408b83 100644 --- a/tools/testing/selftests/kvm/access_tracking_perf_test.c +++ b/tools/testing/selftests/kvm/access_tracking_perf_test.c @@ -306,7 +306,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) struct kvm_vm *vm; int nr_vcpus = params->nr_vcpus; - vm = memstress_create_vm(mode, nr_vcpus, params->vcpu_memory_bytes, 1, + vm = memstress_create_vm(mode, nr_vcpus, params->vcpu_memory_bytes, 1, 0, params->backing_src, !overlap_memory_access); memstress_start_vcpu_threads(nr_vcpus, vcpu_thread_main); diff --git a/tools/testing/selftests/kvm/demand_paging_test.c b/tools/testing/selftests/kvm/demand_paging_test.c index c729cee4c2055..e84dde345edbc 100644 --- a/tools/testing/selftests/kvm/demand_paging_test.c +++ b/tools/testing/selftests/kvm/demand_paging_test.c @@ -144,8 +144,8 @@ static void run_test(enum vm_guest_mode mode, void *arg) int i, num_uffds = 0; uint64_t uffd_region_size; - vm = memstress_create_vm(mode, nr_vcpus, guest_percpu_mem_size, 1, - p->src_type, p->partition_vcpu_memory_access); + vm = memstress_create_vm(mode, nr_vcpus, guest_percpu_mem_size, + 1, 0, p->src_type, p->partition_vcpu_memory_access); demand_paging_size = get_backing_src_pagesz(p->src_type); diff --git a/tools/testing/selftests/kvm/dirty_log_perf_test.c b/tools/testing/selftests/kvm/dirty_log_perf_test.c index e9d6d1aecf89c..6c8749193cfa4 100644 --- a/tools/testing/selftests/kvm/dirty_log_perf_test.c +++ b/tools/testing/selftests/kvm/dirty_log_perf_test.c @@ -224,7 +224,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) int i; vm = memstress_create_vm(mode, nr_vcpus, guest_percpu_mem_size, - p->slots, p->backing_src, + p->slots, 0, p->backing_src, p->partition_vcpu_memory_access); pr_info("Random seed: %u\n", p->random_seed); diff --git a/tools/testing/selftests/kvm/include/memstress.h b/tools/testing/selftests/kvm/include/memstress.h index 72e3e358ef7bd..1cba965d2d331 100644 --- a/tools/testing/selftests/kvm/include/memstress.h +++ b/tools/testing/selftests/kvm/include/memstress.h @@ -56,7 +56,7 @@ struct memstress_args { extern struct memstress_args memstress_args; struct kvm_vm *memstress_create_vm(enum vm_guest_mode mode, int nr_vcpus, - uint64_t vcpu_memory_bytes, int slots, + uint64_t vcpu_memory_bytes, int slots, uint32_t slot_flags, enum vm_mem_backing_src_type backing_src, bool partition_vcpu_memory_access); void memstress_destroy_vm(struct kvm_vm *vm); diff --git a/tools/testing/selftests/kvm/lib/memstress.c b/tools/testing/selftests/kvm/lib/memstress.c index 5f1d3173c238c..7589b8cef6911 100644 --- a/tools/testing/selftests/kvm/lib/memstress.c +++ b/tools/testing/selftests/kvm/lib/memstress.c @@ -119,7 +119,7 @@ void memstress_setup_vcpus(struct kvm_vm *vm, int nr_vcpus, } struct kvm_vm *memstress_create_vm(enum vm_guest_mode mode, int nr_vcpus, - uint64_t vcpu_memory_bytes, int slots, + uint64_t vcpu_memory_bytes, int slots, uint32_t slot_flags, enum vm_mem_backing_src_type backing_src, bool partition_vcpu_memory_access) { @@ -207,7 +207,7 @@ struct kvm_vm *memstress_create_vm(enum vm_guest_mode mode, int nr_vcpus, vm_userspace_mem_region_add(vm, backing_src, region_start, MEMSTRESS_MEM_SLOT_INDEX + i, - region_pages, 0); + region_pages, slot_flags); } /* Do mapping for the demand paging memory slot */ diff --git a/tools/testing/selftests/kvm/memslot_modification_stress_test.c b/tools/testing/selftests/kvm/memslot_modification_stress_test.c index 9855c41ca811f..0b19ec3ecc9cc 100644 --- a/tools/testing/selftests/kvm/memslot_modification_stress_test.c +++ b/tools/testing/selftests/kvm/memslot_modification_stress_test.c @@ -95,7 +95,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) struct test_params *p = arg; struct kvm_vm *vm; - vm = memstress_create_vm(mode, nr_vcpus, guest_percpu_mem_size, 1, + vm = memstress_create_vm(mode, nr_vcpus, guest_percpu_mem_size, 1, 0, VM_MEM_SRC_ANONYMOUS, p->partition_vcpu_memory_access); From patchwork Wed Apr 12 21:35:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anish Moorthy X-Patchwork-Id: 13209571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF8A5C77B6C for ; Wed, 12 Apr 2023 21:35:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230071AbjDLVft (ORCPT ); Wed, 12 Apr 2023 17:35:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230125AbjDLVfs (ORCPT ); Wed, 12 Apr 2023 17:35:48 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC6678A65 for ; Wed, 12 Apr 2023 14:35:35 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id c193-20020a25c0ca000000b00b868826cdfeso31958054ybf.0 for ; Wed, 12 Apr 2023 14:35:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681335335; x=1683927335; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+hL850Rd83GOD7Fb7Sl6JHJBqFDv6kd6OoypTzhr3NY=; b=guJyA5tzVoB5kS7iUGJuz+18m14zr6T065El8C5rANc5xqFQeaE+xi5KmUQ2A2QRNl 3g5buW00HMdicxzoJbus8totxZBQUfl2vx5v0g8TIsy3fUBSLyvC3MmgoUmypWVgqk3L 5B78Zq6zVtiKInbWtb6ktCNAFFYofkWwZ0jAcIq557wH/09R/flDvO88TEzYn5WHln3G 01PEm9ylMHy0BUCeVSl0fuE6J3U1zbnvdxA0HGPZyZqkmN3rHG2EAZxJh1VHoK3Np4yH 8L8wk/75No/LQepz/DtpW06YVGT9dj0Zeg0k2ZDFF5rYwuEm26O3es/Rp1nsqmGks+By Hl5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681335335; x=1683927335; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+hL850Rd83GOD7Fb7Sl6JHJBqFDv6kd6OoypTzhr3NY=; b=ehVBqbVwNuTH3ivHoHjRhPHsv//y2ZL+CVmXGTQJmePTK86fpJQAuInMUlnRHApdh6 kqGHiVtxATm8JRfyDnccOy4wY3k3FjmUfWdbtW1D9PNWlxW9WmJu6j/fuzuk+uUybxiY mr/tC5vssONw6rAeksvVH/epTxrWTxAGfT9WCvsynYoNiXRuJ6Y/IX9dT2/8uVM28A65 HOM9iFNd0H/sCuZBn+5vD2a8AqZl46a7aKewOEItMlXYreI3HI7Fp81HAAZHkzi4uTCR mm4eah9hSGhy/GtxxwwfWp+PhF0/YFMND1/bbHaK/Gzqj5Vzz0nDPUTWYFShBmRoihPG cKgA== X-Gm-Message-State: AAQBX9doKyzQXrB/+LjkR3Nnwp1qYyWVDfa+fycskqg0Tm6YWsDg5zU4 j70dEPIjzDU7sz9JXKm/WjV5L/DcHyKD5A== X-Google-Smtp-Source: AKy350ZjNdVqnc8GlhJSkLTeTZGtqRVwFuFH67stO3WNwC3BmY9bH1qHUPGuHGupi6XIaXDMBwWwAVrhb+pqnQ== X-Received: from laogai.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:2c9]) (user=amoorthy job=sendgmr) by 2002:a25:cb97:0:b0:b8b:f597:f3e5 with SMTP id b145-20020a25cb97000000b00b8bf597f3e5mr496ybg.9.1681335334931; Wed, 12 Apr 2023 14:35:34 -0700 (PDT) Date: Wed, 12 Apr 2023 21:35:10 +0000 In-Reply-To: <20230412213510.1220557-1-amoorthy@google.com> Mime-Version: 1.0 References: <20230412213510.1220557-1-amoorthy@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230412213510.1220557-23-amoorthy@google.com> Subject: [PATCH v3 22/22] KVM: selftests: Handle memory fault exits in demand_paging_test From: Anish Moorthy To: pbonzini@redhat.com, maz@kernel.org Cc: oliver.upton@linux.dev, seanjc@google.com, jthoughton@google.com, amoorthy@google.com, bgardon@google.com, dmatlack@google.com, ricarkol@google.com, axelrasmussen@google.com, peterx@redhat.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Demonstrate a (very basic) scheme for supporting memory fault exits. From the vCPU threads: 1. Simply issue UFFDIO_COPY/CONTINUEs in response to memory fault exits, with the purpose of establishing the absent mappings. Do so with wake_waiters=false to avoid serializing on the userfaultfd wait queue locks. 2. When the UFFDIO_COPY/CONTINUE in (1) fails with EEXIST, assume that the mapping was already established but is currently absent [A] and attempt to populate it using MADV_POPULATE_WRITE. Issue UFFDIO_COPY/CONTINUEs from the reader threads as well, but with wake_waiters=true to ensure that any threads sleeping on the uffd are eventually woken up. A real VMM would track whether it had already COPY/CONTINUEd pages (eg, via a bitmap) to avoid calls destined to EEXIST. However, even the naive approach is enough to demonstrate the performance advantages of KVM_EXIT_MEMORY_FAULT. [A] In reality it is much likelier that the vCPU thread simply lost a race to establish the mapping for the page. Signed-off-by: Anish Moorthy Acked-by: James Houghton --- .../selftests/kvm/demand_paging_test.c | 209 +++++++++++++----- 1 file changed, 155 insertions(+), 54 deletions(-) diff --git a/tools/testing/selftests/kvm/demand_paging_test.c b/tools/testing/selftests/kvm/demand_paging_test.c index e84dde345edbc..668bd63d944e7 100644 --- a/tools/testing/selftests/kvm/demand_paging_test.c +++ b/tools/testing/selftests/kvm/demand_paging_test.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include "kvm_util.h" @@ -31,6 +32,57 @@ static uint64_t guest_percpu_mem_size = DEFAULT_PER_VCPU_MEM_SIZE; static size_t demand_paging_size; static char *guest_data_prototype; +static int num_uffds; +static size_t uffd_region_size; +static struct uffd_desc **uffd_descs; +/* + * Delay when demand paging is performed through userfaultfd or directly by + * vcpu_worker in the case of a KVM_EXIT_MEMORY_FAULT. + */ +static useconds_t uffd_delay; +static int uffd_mode; + + +static int handle_uffd_page_request(int uffd_mode, int uffd, uint64_t hva, + bool is_vcpu); + +static void madv_write_or_err(uint64_t gpa) +{ + int r; + void *hva = addr_gpa2hva(memstress_args.vm, gpa); + + r = madvise(hva, demand_paging_size, MADV_POPULATE_WRITE); + TEST_ASSERT(r == 0, + "MADV_POPULATE_WRITE on hva 0x%lx (gpa 0x%lx) fail, errno %i\n", + (uintptr_t) hva, gpa, errno); +} + +static void ready_page(uint64_t gpa) +{ + int r, uffd; + + /* + * This test only registers memslot 1 w/ userfaultfd. Any accesses outside + * the registered ranges should fault in the physical pages through + * MADV_POPULATE_WRITE. + */ + if ((gpa < memstress_args.gpa) + || (gpa >= memstress_args.gpa + memstress_args.size)) { + madv_write_or_err(gpa); + } else { + if (uffd_delay) + usleep(uffd_delay); + + uffd = uffd_descs[(gpa - memstress_args.gpa) / uffd_region_size]->uffd; + + r = handle_uffd_page_request(uffd_mode, uffd, + (uint64_t) addr_gpa2hva(memstress_args.vm, gpa), true); + + if (r == EEXIST) + madv_write_or_err(gpa); + } +} + static void vcpu_worker(struct memstress_vcpu_args *vcpu_args) { struct kvm_vcpu *vcpu = vcpu_args->vcpu; @@ -42,25 +94,36 @@ static void vcpu_worker(struct memstress_vcpu_args *vcpu_args) clock_gettime(CLOCK_MONOTONIC, &start); - /* Let the guest access its memory */ - ret = _vcpu_run(vcpu); - TEST_ASSERT(ret == 0, "vcpu_run failed: %d\n", ret); - if (get_ucall(vcpu, NULL) != UCALL_SYNC) { - TEST_ASSERT(false, - "Invalid guest sync status: exit_reason=%s\n", - exit_reason_str(run->exit_reason)); - } + while (true) { + /* Let the guest access its memory */ + ret = _vcpu_run(vcpu); + TEST_ASSERT(ret == 0 + || (errno == EFAULT + && run->exit_reason == KVM_EXIT_MEMORY_FAULT), + "vcpu_run failed: %d\n", ret); + if (ret != 0 && get_ucall(vcpu, NULL) != UCALL_SYNC) { + + if (run->exit_reason == KVM_EXIT_MEMORY_FAULT) { + ready_page(run->memory_fault.gpa); + continue; + } + + TEST_ASSERT(false, + "Invalid guest sync status: exit_reason=%s\n", + exit_reason_str(run->exit_reason)); + } - ts_diff = timespec_elapsed(start); - PER_VCPU_DEBUG("vCPU %d execution time: %ld.%.9lds\n", vcpu_idx, - ts_diff.tv_sec, ts_diff.tv_nsec); + ts_diff = timespec_elapsed(start); + PER_VCPU_DEBUG("vCPU %d execution time: %ld.%.9lds\n", vcpu_idx, + ts_diff.tv_sec, ts_diff.tv_nsec); + break; + } } -static int handle_uffd_page_request(int uffd_mode, int uffd, - struct uffd_msg *msg) +static int handle_uffd_page_request(int uffd_mode, int uffd, uint64_t hva, + bool is_vcpu) { pid_t tid = syscall(__NR_gettid); - uint64_t addr = msg->arg.pagefault.address; struct timespec start; struct timespec ts_diff; int r; @@ -71,56 +134,78 @@ static int handle_uffd_page_request(int uffd_mode, int uffd, struct uffdio_copy copy; copy.src = (uint64_t)guest_data_prototype; - copy.dst = addr; + copy.dst = hva; copy.len = demand_paging_size; - copy.mode = 0; + copy.mode = UFFDIO_COPY_MODE_DONTWAKE; - r = ioctl(uffd, UFFDIO_COPY, ©); /* - * With multiple vCPU threads fault on a single page and there are - * multiple readers for the UFFD, at least one of the UFFDIO_COPYs - * will fail with EEXIST: handle that case without signaling an - * error. + * With multiple vCPU threads and at least one of multiple reader threads + * or vCPU memory faults, multiple vCPUs accessing an absent page will + * almost certainly cause some thread doing the UFFDIO_COPY here to get + * EEXIST: make sure to allow that case. */ - if (r == -1 && errno != EEXIST) { - pr_info("Failed UFFDIO_COPY in 0x%lx from thread %d, errno = %d\n", - addr, tid, errno); - return r; - } + r = ioctl(uffd, UFFDIO_COPY, ©); + TEST_ASSERT(r == 0 || errno == EEXIST, + "Thread 0x%x failed UFFDIO_COPY on hva 0x%lx, errno = %d", + gettid(), hva, errno); } else if (uffd_mode == UFFDIO_REGISTER_MODE_MINOR) { + /* The comments in the UFFDIO_COPY branch also apply here. */ struct uffdio_continue cont = {0}; - cont.range.start = addr; + cont.range.start = hva; cont.range.len = demand_paging_size; + cont.mode = UFFDIO_CONTINUE_MODE_DONTWAKE; r = ioctl(uffd, UFFDIO_CONTINUE, &cont); - /* See the note about EEXISTs in the UFFDIO_COPY branch. */ - if (r == -1 && errno != EEXIST) { - pr_info("Failed UFFDIO_CONTINUE in 0x%lx, thread %d, errno = %d\n", - addr, tid, errno); - return r; - } + TEST_ASSERT(r == 0 || errno == EEXIST, + "Thread 0x%x failed UFFDIO_CONTINUE on hva 0x%lx, errno = %d", + gettid(), hva, errno); } else { TEST_FAIL("Invalid uffd mode %d", uffd_mode); } + /* + * If the above UFFDIO_COPY/CONTINUE fails with EEXIST, it will do so without + * waking threads waiting on the UFFD: make sure that happens here. + */ + if (!is_vcpu) { + struct uffdio_range range = { + .start = hva, + .len = demand_paging_size + }; + r = ioctl(uffd, UFFDIO_WAKE, &range); + TEST_ASSERT( + r == 0, + "Thread 0x%x failed UFFDIO_WAKE on hva 0x%lx, errno = %d", + gettid(), hva, errno); + } + ts_diff = timespec_elapsed(start); PER_PAGE_DEBUG("UFFD page-in %d \t%ld ns\n", tid, timespec_to_ns(ts_diff)); PER_PAGE_DEBUG("Paged in %ld bytes at 0x%lx from thread %d\n", - demand_paging_size, addr, tid); + demand_paging_size, hva, tid); return 0; } +static int handle_uffd_page_request_from_uffd(int uffd_mode, int uffd, + struct uffd_msg *msg) +{ + TEST_ASSERT(msg->event == UFFD_EVENT_PAGEFAULT, + "Received uffd message with event %d != UFFD_EVENT_PAGEFAULT", + msg->event); + return handle_uffd_page_request(uffd_mode, uffd, + msg->arg.pagefault.address, false); +} + struct test_params { - int uffd_mode; bool single_uffd; - useconds_t uffd_delay; int readers_per_uffd; enum vm_mem_backing_src_type src_type; bool partition_vcpu_memory_access; + bool memfault_exits; }; static void prefault_mem(void *alias, uint64_t len) @@ -137,15 +222,26 @@ static void prefault_mem(void *alias, uint64_t len) static void run_test(enum vm_guest_mode mode, void *arg) { struct test_params *p = arg; - struct uffd_desc **uffd_descs = NULL; struct timespec start; struct timespec ts_diff; struct kvm_vm *vm; - int i, num_uffds = 0; - uint64_t uffd_region_size; + int i; + uint32_t slot_flags = 0; + bool uffd_memfault_exits = uffd_mode && p->memfault_exits; + + if (uffd_memfault_exits) { + TEST_ASSERT(kvm_has_cap(KVM_CAP_ABSENT_MAPPING_FAULT) > 0, + "KVM does not have KVM_CAP_ABSENT_MAPPING_FAULT"); + slot_flags = KVM_MEM_ABSENT_MAPPING_FAULT; + } vm = memstress_create_vm(mode, nr_vcpus, guest_percpu_mem_size, - 1, 0, p->src_type, p->partition_vcpu_memory_access); + 1, slot_flags, p->src_type, p->partition_vcpu_memory_access); + + if (uffd_memfault_exits) { + vm_enable_cap(vm, + KVM_CAP_MEMORY_FAULT_INFO, KVM_MEMORY_FAULT_INFO_ENABLE); + } demand_paging_size = get_backing_src_pagesz(p->src_type); @@ -154,12 +250,12 @@ static void run_test(enum vm_guest_mode mode, void *arg) "Failed to allocate buffer for guest data pattern"); memset(guest_data_prototype, 0xAB, demand_paging_size); - if (p->uffd_mode) { + if (uffd_mode) { num_uffds = p->single_uffd ? 1 : nr_vcpus; uffd_region_size = nr_vcpus * guest_percpu_mem_size / num_uffds; uffd_descs = malloc(num_uffds * sizeof(struct uffd_desc *)); - TEST_ASSERT(uffd_descs, "Memory allocation failed"); + TEST_ASSERT(uffd_descs, "Failed to allocate memory of uffd descriptors"); for (i = 0; i < num_uffds; i++) { struct memstress_vcpu_args *vcpu_args; @@ -179,10 +275,10 @@ static void run_test(enum vm_guest_mode mode, void *arg) * requests. */ uffd_descs[i] = uffd_setup_demand_paging( - p->uffd_mode, p->uffd_delay, vcpu_hva, + uffd_mode, uffd_delay, vcpu_hva, uffd_region_size, p->readers_per_uffd, - &handle_uffd_page_request); + &handle_uffd_page_request_from_uffd); } } @@ -196,7 +292,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) ts_diff = timespec_elapsed(start); pr_info("All vCPU threads joined\n"); - if (p->uffd_mode) { + if (uffd_mode) { /* Tell the user fault fd handler threads to quit */ for (i = 0; i < num_uffds; i++) uffd_stop_demand_paging(uffd_descs[i]); @@ -211,7 +307,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) memstress_destroy_vm(vm); free(guest_data_prototype); - if (p->uffd_mode) + if (uffd_mode) free(uffd_descs); } @@ -220,7 +316,7 @@ static void help(char *name) puts(""); printf("usage: %s [-h] [-m vm_mode] [-u uffd_mode] [-a]\n" " [-d uffd_delay_usec] [-r readers_per_uffd] [-b memory]\n" - " [-s type] [-v vcpus] [-o]\n", name); + " [-w] [-s type] [-v vcpus] [-o]\n", name); guest_modes_help(); printf(" -u: use userfaultfd to handle vCPU page faults. Mode is a\n" " UFFD registration mode: 'MISSING' or 'MINOR'.\n"); @@ -231,6 +327,7 @@ static void help(char *name) " FD handler to simulate demand paging\n" " overheads. Ignored without -u.\n"); printf(" -r: Set the number of reader threads per uffd.\n"); + printf(" -w: Enable kvm cap for memory fault exits.\n"); printf(" -b: specify the size of the memory region which should be\n" " demand paged by each vCPU. e.g. 10M or 3G.\n" " Default: 1G\n"); @@ -250,29 +347,30 @@ int main(int argc, char *argv[]) .partition_vcpu_memory_access = true, .readers_per_uffd = 1, .single_uffd = false, + .memfault_exits = false, }; int opt; guest_modes_append_default(); - while ((opt = getopt(argc, argv, "ahom:u:d:b:s:v:r:")) != -1) { + while ((opt = getopt(argc, argv, "ahowm:u:d:b:s:v:r:")) != -1) { switch (opt) { case 'm': guest_modes_cmdline(optarg); break; case 'u': if (!strcmp("MISSING", optarg)) - p.uffd_mode = UFFDIO_REGISTER_MODE_MISSING; + uffd_mode = UFFDIO_REGISTER_MODE_MISSING; else if (!strcmp("MINOR", optarg)) - p.uffd_mode = UFFDIO_REGISTER_MODE_MINOR; - TEST_ASSERT(p.uffd_mode, "UFFD mode must be 'MISSING' or 'MINOR'."); + uffd_mode = UFFDIO_REGISTER_MODE_MINOR; + TEST_ASSERT(uffd_mode, "UFFD mode must be 'MISSING' or 'MINOR'."); break; case 'a': p.single_uffd = true; break; case 'd': - p.uffd_delay = strtoul(optarg, NULL, 0); - TEST_ASSERT(p.uffd_delay >= 0, "A negative UFFD delay is not supported."); + uffd_delay = strtoul(optarg, NULL, 0); + TEST_ASSERT(uffd_delay >= 0, "A negative UFFD delay is not supported."); break; case 'b': guest_percpu_mem_size = parse_size(optarg); @@ -295,6 +393,9 @@ int main(int argc, char *argv[]) "Invalid number of readers per uffd %d: must be >=1", p.readers_per_uffd); break; + case 'w': + p.memfault_exits = true; + break; case 'h': default: help(argv[0]); @@ -302,7 +403,7 @@ int main(int argc, char *argv[]) } } - if (p.uffd_mode == UFFDIO_REGISTER_MODE_MINOR && + if (uffd_mode == UFFDIO_REGISTER_MODE_MINOR && !backing_src_is_shared(p.src_type)) { TEST_FAIL("userfaultfd MINOR mode requires shared memory; pick a different -s"); }