From patchwork Sat Oct 28 00:38:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13439343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66D58C27C47 for ; Sat, 28 Oct 2023 00:38:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EFE8B6B0408; Fri, 27 Oct 2023 20:38:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EAD176B0409; Fri, 27 Oct 2023 20:38:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C62FA6B040A; Fri, 27 Oct 2023 20:38:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B1CDD6B0408 for ; Fri, 27 Oct 2023 20:38:37 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 847F21208B5 for ; Sat, 28 Oct 2023 00:38:37 +0000 (UTC) X-FDA: 81393009474.15.B909659 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf27.hostedemail.com (Postfix) with ESMTP id C3EE840009 for ; Sat, 28 Oct 2023 00:38:34 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ZEfV7tgQ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3CVg8ZQYKCNoOQNAJ7CKKCHA.8KIHEJQT-IIGR68G.KNC@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3CVg8ZQYKCNoOQNAJ7CKKCHA.8KIHEJQT-IIGR68G.KNC@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698453514; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DPstGiI3zb7PZzW8m1s/TtyE02RR3fMaWMy1rl5jbQE=; b=plqjY+nb5HRaoDmRQzUxnl4HdEILZi6lqfco9Z7ZYR9tUX14boaBRMx8Sei7m+XvzWhte6 uBcIzPP4/PPvICeKEZ97x5Lk1kVS2CXsI+bwvWShGUP3rYMvPLc0JZI7DqDHcxox3nbH2a /tMHW3ZUtINvafAfXrfczPaxHVObrFs= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ZEfV7tgQ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3CVg8ZQYKCNoOQNAJ7CKKCHA.8KIHEJQT-IIGR68G.KNC@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3CVg8ZQYKCNoOQNAJ7CKKCHA.8KIHEJQT-IIGR68G.KNC@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698453514; a=rsa-sha256; cv=none; b=7j+hQ0vcDoas9IGjLP8AesQiApQ4/4RCO9qGJDL+FuLAqv83hRe7UNy/+OBkRGSzlw81d6 TD/knlMXBN8C7Eut9A35ZbZaKX8u+3rSd8aP19blXArmEqkBGcLtqRghcalk6yz/UcgGR+ 3HvtDIMS/91aHWuFxJ71ztzz3JB0xVo= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-5a909b4e079so23417687b3.2 for ; Fri, 27 Oct 2023 17:38:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1698453514; x=1699058314; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DPstGiI3zb7PZzW8m1s/TtyE02RR3fMaWMy1rl5jbQE=; b=ZEfV7tgQjh/l52LTFaN02QcKO6dp+yCdoTZM9opuy9ZZr37xxkLOFytHEiIoY4y9aw UYVg57XO/i9ySGYHiOYEeNd49phtjnDFCIHpAlppP1Caqh9ZKU/+W2eSOt2zsM+4b1iK M6sQH/IJmdq/Xk005+v/e/+TbYnvFpAm36SGiUg4vJZFzz00PziSI2c+UzSZJlEqNkpt 2YH8xRoTHw7rKGZZ1zLPAj3s5fgcq7TM9cblhhdsJeCb1+ubyvvWuqJlMAk0rKiZk9NK CQZLaICo1BvvU3jzX04Ns4XG8o2Dswx3Mu4WLUhCAfhdwmAuN7xV+gBR2GTDwQw2j/TQ h09g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698453514; x=1699058314; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DPstGiI3zb7PZzW8m1s/TtyE02RR3fMaWMy1rl5jbQE=; b=FMTUtanmpKIa/h2H/F4Lg8mmjWZoyBQ4+puuev9ZOOxsb7vc1hSEnjPyFVy2/JTPQH nW1Mjg0tGfoHSL/OP6k3dkK62OJNS+xWX3JDJDFiZTk0/mcM0/CW522CX1subN+uA2Yq IIYEfhh+AO65Z7UeCy2Ppv3Dus7YqgwTeYN46AU8lZRUIp1S5m6cTowQPgPLyfYhR0fe 84q8tk8pJXTiu6lJ9xczGNOjo/tXzQo8LekUSiks5hE5m916KnDkbvaQFKuh5uiIQL9v FrhuxxT5wHAo4ZOHo3OnHetxXyScG8PXy6nA+VRZBybZy/YKEk9SHag+SxiZqNnhRLuy UkJg== X-Gm-Message-State: AOJu0YxNXYjfbt5WnYAt0+sLzGDnExoQvJLhHa6d/E8y7nKK9tefzV+j +V+/gswJ7u2UsDHYkZ6F0hUS3EOnxvM= X-Google-Smtp-Source: AGHT+IHkyagOv7vQjaKfa8YJr6CBe53G6nlWj5O4BlA1iQSMbHSpme2yM6964WYBoEz45GQ9XcUSJSde3/4= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:cba3:7e7f:79e4:fa57]) (user=surenb job=sendgmr) by 2002:a81:6d17:0:b0:59b:c6bb:babb with SMTP id i23-20020a816d17000000b0059bc6bbbabbmr87372ywc.6.1698453513868; Fri, 27 Oct 2023 17:38:33 -0700 (PDT) Date: Fri, 27 Oct 2023 17:38:15 -0700 In-Reply-To: <20231028003819.652322-1-surenb@google.com> Mime-Version: 1.0 References: <20231028003819.652322-1-surenb@google.com> X-Mailer: git-send-email 2.42.0.820.g83a721a137-goog Message-ID: <20231028003819.652322-6-surenb@google.com> Subject: [PATCH v4 5/5] selftests/mm: add UFFDIO_MOVE ioctl test From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, shuah@kernel.org, aarcange@redhat.com, lokeshgidra@google.com, peterx@redhat.com, david@redhat.com, hughd@google.com, mhocko@suse.com, axelrasmussen@google.com, rppt@kernel.org, willy@infradead.org, Liam.Howlett@oracle.com, jannh@google.com, zhangpeng362@huawei.com, bgeffon@google.com, kaleshsingh@google.com, ngeoffray@google.com, jdduke@google.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@android.com X-Rspamd-Queue-Id: C3EE840009 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 4jqi3n5i7es9yx1fuodi8zqd7r9ggkx8 X-HE-Tag: 1698453514-898508 X-HE-Meta: U2FsdGVkX18ypUCxQalo85h23NBgL21B/23jNYgTdj8+1mTheOVOUAazrxCpH90rm5OB078NW14X8YvYIxSJMzPtXAoyCY2dytMDBxRdJl9/lhcQlxhygxFI95KDCqrl5zV/erW8764n6xVY64Bj+soxHt+Z2aEHGZ8wlcphYbpBINcxb/ySKWi5kkt6KOn9Qbtmh/r27RNjGdXC5KsYyaypbaEsNNmx8Im2dW1lgoahsykAH3c9n3+ywW7N8Mug79qDj9rySyQVdoZBHNsTsGNfzhel7kKzaLGXn30FD/2OHvMKOLaEVdKvKMXSZ+ON2a9vHOjVKqxS9JFUVhn3kVmrUk4P7C4gP8XBVdYG0i10y9HuqYKEZWYhauF5N4yY/HDy8D3a06x3RUzNDRKwDVBhzt96ALgcgTl+J5BUxTzy+tn26vHmRGwGt3NEvZYpES/aD+AYu1I7IwHGMS/R0hKYRFHwm+et4E7anjdry+tUyATC9B+BMT6QjPuj8quNKDCqS40KhYStXwRpae87fOtNcJtwQLzQ3HRy4XsemoisGqHltNTLEGhdNJnHVak24T/Q5GMZ8y97PqMTHNISfrAM9/AjLWOie3HQkk4LMW3bGMUsPW4xN4yc020k3uQYWezvSFxjNClo6FMi62S+x8DMwNObxoGSIEYnyuv6U5Twv1zEUEfWn5y4iqPg7Mdlw+TnTYrHafYlbYeU3bYOeIRe+VbCpUHVbAEtXAK0jG85tGy1zhap+od9r7s5IAmuoUD48tFtF5rf/tT2ChfuqA6uHiptuPHNMbC4ai8A6tozlObXeaHQNKoMSZPmje+g5I7HT5+b0EPyX5Z/7HG19ByeEV7YUW/udZU0SXEDsyuDuGHI4/ACtq+od3qPMxClGGcZ8EuH6pTuWQcE+qv1FpEOgBWF9699qV1rNypzUH+Kkw+J9WYUgg+23/MAtieW9tMjcurth6QmRsvL3C6 Kmed+qC1 gjN5Jy/ctZV+B5EpJ2VDTO4vPTLknCPjvhQ/XK6yVjjyTvECd2cEFyhoh8FNftBhQBzvE4qV/rmZpBof+KD0FBcHyA3X0Sv0MyYyA1t0yvo9ZSY9rHCdU+t8wW7jEES/988w6VFb1dqldUEd+aT4qs52TBH486f9X0bhRIu2ennF9Hu3c/7Hu5G+yWcmhsfhPDuUV+WgXwRBU2fSoSISg+VNM98xeo5EfNU5EZKwuOMR4odnOcWhl/WZnAn/JWk9DHFcROsRzFnFV65s3g9oJ1WWqhoKPKjK7jhg8gEuRyxY9Mw3z6zXbVuS4YnJRFF9M4dr9DrFaxPkTp4Ju83e5feyF97sqwLcEp9oCuoTzufGGsqHh6qCGzZj+KMInGodYpz+JKnoQkpL4a2wcK7RPFyjfW7XhzUGNkAmFdHorDs/HzqPmYbhOpHulr4Wu6yy94ZtAUqokNIwWBLXrEw0dREsOPSFflBvw8e4MHPjUbevFXpkubAFPsHYvsFGgvw5tBrADhUU/I3jQ6CMTEP538DMt5bFOsx3epqaJY398Bp0Q8jDBAqnR2ztcUdNVMm6w6GoyWKam2fUJKsJL1Y/hXX3cszd/iPOiFak5ip3k3GcVa9WYHfnBvbzbJ0MJRsWNjJKTE0TepOANbNvPKAV2WblVyg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add tests for new UFFDIO_MOVE ioctl which uses uffd to move source into destination buffer while checking the contents of both after the move. After the operation the content of the destination buffer should match the original source buffer's content while the source buffer should be zeroed. Separate tests are designed for PMD aligned and unaligned cases because they utilize different code paths in the kernel. Signed-off-by: Suren Baghdasaryan --- tools/testing/selftests/mm/uffd-common.c | 24 ++++ tools/testing/selftests/mm/uffd-common.h | 1 + tools/testing/selftests/mm/uffd-unit-tests.c | 141 +++++++++++++++++++ 3 files changed, 166 insertions(+) diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c index 69e6653ad255..98957fd788d8 100644 --- a/tools/testing/selftests/mm/uffd-common.c +++ b/tools/testing/selftests/mm/uffd-common.c @@ -643,6 +643,30 @@ int copy_page(int ufd, unsigned long offset, bool wp) return __copy_page(ufd, offset, false, wp); } +int move_page(int ufd, unsigned long offset) +{ + struct uffdio_move uffdio_move; + + if (offset >= nr_pages * page_size) + err("unexpected offset %lu\n", offset); + uffdio_move.dst = (unsigned long) area_dst + offset; + uffdio_move.src = (unsigned long) area_src + offset; + uffdio_move.len = page_size; + uffdio_move.mode = UFFDIO_MOVE_MODE_ALLOW_SRC_HOLES; + uffdio_move.move = 0; + if (ioctl(ufd, UFFDIO_MOVE, &uffdio_move)) { + /* real retval in uffdio_move.move */ + if (uffdio_move.move != -EEXIST) + err("UFFDIO_MOVE error: %"PRId64, + (int64_t)uffdio_move.move); + wake_range(ufd, uffdio_move.dst, page_size); + } else if (uffdio_move.move != page_size) { + err("UFFDIO_MOVE error: %"PRId64, (int64_t)uffdio_move.move); + } else + return 1; + return 0; +} + int uffd_open_dev(unsigned int flags) { int fd, uffd; diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h index 19930fd6682b..c9526b2cb6b3 100644 --- a/tools/testing/selftests/mm/uffd-common.h +++ b/tools/testing/selftests/mm/uffd-common.h @@ -121,6 +121,7 @@ void wp_range(int ufd, __u64 start, __u64 len, bool wp); void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args); int __copy_page(int ufd, unsigned long offset, bool retry, bool wp); int copy_page(int ufd, unsigned long offset, bool wp); +int move_page(int ufd, unsigned long offset); void *uffd_poll_thread(void *arg); int uffd_open_dev(unsigned int flags); diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c index debc423bdbf4..89e9529ce941 100644 --- a/tools/testing/selftests/mm/uffd-unit-tests.c +++ b/tools/testing/selftests/mm/uffd-unit-tests.c @@ -1064,6 +1064,133 @@ static void uffd_poison_test(uffd_test_args_t *targs) uffd_test_pass(); } +static void uffd_move_handle_fault( + struct uffd_msg *msg, struct uffd_args *args) +{ + unsigned long offset; + + if (msg->event != UFFD_EVENT_PAGEFAULT) + err("unexpected msg event %u", msg->event); + + if (msg->arg.pagefault.flags & + (UFFD_PAGEFAULT_FLAG_WP | UFFD_PAGEFAULT_FLAG_MINOR | UFFD_PAGEFAULT_FLAG_WRITE)) + err("unexpected fault type %llu", msg->arg.pagefault.flags); + + offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst; + offset &= ~(page_size-1); + + if (move_page(uffd, offset)) + args->missing_faults++; +} + +static void uffd_move_test(uffd_test_args_t *targs) +{ + unsigned long nr; + pthread_t uffd_mon; + char c; + unsigned long long count; + struct uffd_args args = { 0 }; + + /* Prevent source pages from being mapped more than once */ + if (madvise(area_src, nr_pages * page_size, MADV_DONTFORK)) + err("madvise(MADV_DONTFORK) failure"); + + if (uffd_register(uffd, area_dst, nr_pages * page_size, + true, false, false)) + err("register failure"); + + args.handle_fault = uffd_move_handle_fault; + if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args)) + err("uffd_poll_thread create"); + + /* + * Read each of the pages back using the UFFD-registered mapping. We + * expect that the first time we touch a page, it will result in a missing + * fault. uffd_poll_thread will resolve the fault by moving source + * page to destination. + */ + for (nr = 0; nr < nr_pages; nr++) { + /* Check area_src content */ + count = *area_count(area_src, nr); + if (count != count_verify[nr]) + err("nr %lu source memory invalid %llu %llu\n", + nr, count, count_verify[nr]); + + /* Faulting into area_dst should move the page */ + count = *area_count(area_dst, nr); + if (count != count_verify[nr]) + err("nr %lu memory corruption %llu %llu\n", + nr, count, count_verify[nr]); + + /* Re-check area_src content which should be empty */ + count = *area_count(area_src, nr); + if (count != 0) + err("nr %lu move failed %llu %llu\n", + nr, count, count_verify[nr]); + } + + if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) + err("pipe write"); + if (pthread_join(uffd_mon, NULL)) + err("join() failed"); + + if (args.missing_faults != nr_pages || args.minor_faults != 0) + uffd_test_fail("stats check error"); + else + uffd_test_pass(); +} + +static int prevent_hugepages(void) +{ + /* This should be done before source area is populated */ + if (madvise(area_src, nr_pages * page_size, MADV_NOHUGEPAGE)) { + /* Ignore if CONFIG_TRANSPARENT_HUGEPAGE=n */ + if (errno != EINVAL) + return -errno; + } + return 0; +} + +struct uffd_test_case_ops uffd_move_test_case_ops = { + .post_alloc = prevent_hugepages, +}; + +#define ALIGN_UP(x, align_to) \ + (__typeof__(x))((((unsigned long)(x)) + ((align_to)-1)) & ~((align_to)-1)) + +static char *orig_area_src, *orig_area_dst; +static int pmd_align_areas(void) +{ + orig_area_src = area_src; + orig_area_dst = area_dst; + area_src = ALIGN_UP(area_src, page_size); + area_dst = ALIGN_UP(area_dst, page_size); + nr_pages--; + + return 0; +} + +static void pmd_restore_areas(void) +{ + area_src = orig_area_src; + area_dst = orig_area_dst; + nr_pages++; +} + +static int adjust_page_size(void) +{ + page_size = default_huge_page_size(); + nr_pages = UFFD_TEST_MEM_SIZE / page_size; + + return 0; +} + +struct uffd_test_case_ops uffd_move_pmd_test_case_ops = { + .pre_alloc = adjust_page_size, + .post_alloc = pmd_align_areas, + .pre_release = pmd_restore_areas, +}; + /* * Test the returned uffdio_register.ioctls with different register modes. * Note that _UFFDIO_ZEROPAGE is tested separately in the zeropage test. @@ -1141,6 +1268,20 @@ uffd_test_case_t uffd_tests[] = { .mem_targets = MEM_ALL, .uffd_feature_required = 0, }, + { + .name = "move", + .uffd_fn = uffd_move_test, + .mem_targets = MEM_ANON, + .uffd_feature_required = UFFD_FEATURE_MOVE, + .test_case_ops = &uffd_move_test_case_ops, + }, + { + .name = "move-pmd", + .uffd_fn = uffd_move_test, + .mem_targets = MEM_ANON, + .uffd_feature_required = UFFD_FEATURE_MOVE, + .test_case_ops = &uffd_move_pmd_test_case_ops, + }, { .name = "wp-fork", .uffd_fn = uffd_wp_fork_test,