From patchwork Thu Aug 10 14:29:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13349530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5710DC04A6A for ; Thu, 10 Aug 2023 14:30:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C36D56B007D; Thu, 10 Aug 2023 10:30:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC20D6B007E; Thu, 10 Aug 2023 10:30:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EA996B0080; Thu, 10 Aug 2023 10:30:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 90A026B007D for ; Thu, 10 Aug 2023 10:30:09 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D1E4F121041 for ; Thu, 10 Aug 2023 14:30:08 +0000 (UTC) X-FDA: 81108429696.07.555B6FD Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf22.hostedemail.com (Postfix) with ESMTP id C805FC000D for ; Thu, 10 Aug 2023 14:30:06 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691677806; a=rsa-sha256; cv=none; b=TwsZZUbMUY6MiCdLT2ZkN+L51Md+KYILzUCnyBYB+LGrBrE7ETmaDk1KPDmlDi3P8OTj2H siuRgU3SOhjxW5LGT/YOPOizE4OILVkaVVu35UUpx8lyEwoBdo0zwi8f6Fhn7kDVajxFo/ /dbViU+6K061jBa14dn/VXagV2sPzK4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691677806; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YAYuga94X0gYR3iPhGr2eIcuvk2ipwl7xIVV/NAaqzI=; b=CrO40ev5y8+AeHB5D48ge2OU6PU/+h+1UwMI+sRAw+z0a/Njjisylsq+w4RSGcwBSKHFqm o7VF6+Zj0CFXLqUJXgW5FEAAZe9FiOAxMA9RC+chDm6QsBjCwvGZqkSyl2YyuHC3jmaNao jKKQZTIDCVe0/quqneMjig/TEspsnz4= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 50B52150C; Thu, 10 Aug 2023 07:30:48 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 937D63F64C; Thu, 10 Aug 2023 07:30:03 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Matthew Wilcox , Yin Fengwei , David Hildenbrand , Yu Zhao , Catalin Marinas , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain , Itaru Kitayama , "Kirill A. Shutemov" Cc: Ryan Roberts , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 5/5] selftests/mm/cow: Add large anon folio tests Date: Thu, 10 Aug 2023 15:29:42 +0100 Message-Id: <20230810142942.3169679-6-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230810142942.3169679-1-ryan.roberts@arm.com> References: <20230810142942.3169679-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C805FC000D X-Stat-Signature: f3ueos48xna481fd91g8hp65ktuc8gj8 X-HE-Tag: 1691677806-233387 X-HE-Meta: U2FsdGVkX18ppuCvKe0wHip8pg8C/cpkCzw7LaQ/ltqmjRIEfAVYLcE8bHkSuLrSyqDKocBk6q9z0XPyyqczIegLDMmGLP/WilzDSQiSUXjDneLGObtM34TSrmtuX60BYB1mrLNk2nTvBC9LiEhqG5S/3dK6En7EV3f8u2X9G5QNHeamymYatSJRiRvMI+WX0B8tNQ5nRbUI+QFVmwNwRwTnKuwSY/L8KDl54qG1jgVaEHQE6qfGEVeICTLGqSTIcaKg3LcvqNh2s+olbc6NFFV0VqD6WM6B5f8XnNNFE8KGOCD1mMv7fiC+Xjx1xHSKCyGG4HMtxSQM21RVYvxNpWutzWaqrUo9SqK2YmP6cqipWinZ/r2lDX01op/9Ic5fpZfE93H0K7r67H1n/KPxIGIXpY+naF1OyF++vJrYKIi0wDAl6NHzdyURTN2qhI0Pw9gnts5b7S4F9ePE90B059Y0f7ys/5QhNgBcJ7lNrXxXZRyF/7zM7+HtQJLZZRTNLY97LgnvI8vbs9s7O7qUWrWJ2qaUnsYBUFDqbKyWHZJOmgeeDxmQhNgFD2kklEl+YYkHysc+VBe5/6gXCzZLNnMCstUaF706x/G7wLZUdCeSQf+mx7OGHFrOg0ifCWQcqOmJgTAM0YFurEhbPFBvhpffE0eqAh6wvMMc+shFjabYv4UEv3kOjNhdPGoFt5JmjpyJciXnWo3IKAJ+eJSU/z2ShXfSpPaZ6G1D75S/Z+9Id6ac1vTlDyvm9gT+9Xii8MW9awTze1lbtGg4wGvXAQ5O5d1W5UtR9B1q/nj6X3HeAZ6sr3OsML8JTZiJVg4mc64I34dWYJEkyCydYGqtnWyzVBsqct2fIpgqIUILcS7+XtJjNfEHw1A38lqRlowGorEPRyNzYPjTuLS5rQdyWXYnALgdkyVPJHjr8rPoqPrge22XWZ5FEgF89WryUg41xXla2rWOFlBS7LPWbt/ 8noNNQLi u5WzSjHxsO4noV7KVAunnD7BFAEFpkk4cYlmVF+JoJMF+FPLY2kXoKpxuAaPQkftsw+QvY3s4T83tLzZRymq0RPDWomkrtbAiSifbzY4PEEeFXszNUiWp90HqATuJe2mWfJrkHpL5LL0KFEBMaxP7E4/6H9xLJOA9hoN9IP/g2VBxjOvFLQWSVBShPVbs0ZdED/ElG1fju56dXgdlNwJEVjNhyV4nYd/51YYD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add tests similar to the existing THP tests, but which operate on memory backed by large anonymous folios, which are smaller than THP. This reuses all the existing infrastructure. If the test suite detects that large anonyomous folios are not supported by the kernel, the new tests are skipped. Signed-off-by: Ryan Roberts --- tools/testing/selftests/mm/cow.c | 111 +++++++++++++++++++++++++++++-- 1 file changed, 106 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/mm/cow.c b/tools/testing/selftests/mm/cow.c index 304882bf2e5d..932242c965a4 100644 --- a/tools/testing/selftests/mm/cow.c +++ b/tools/testing/selftests/mm/cow.c @@ -33,6 +33,7 @@ static size_t pagesize; static int pagemap_fd; static size_t thpsize; +static size_t lafsize; static int nr_hugetlbsizes; static size_t hugetlbsizes[10]; static int gup_fd; @@ -927,6 +928,42 @@ static void run_with_partial_shared_thp(test_fn fn, const char *desc) do_run_with_large(fn, LARGE_RUN_PARTIAL_SHARED, thpsize); } +static void run_with_laf(test_fn fn, const char *desc) +{ + ksft_print_msg("[RUN] %s ... with large anon folio\n", desc); + do_run_with_large(fn, LARGE_RUN_PTE, lafsize); +} + +static void run_with_laf_swap(test_fn fn, const char *desc) +{ + ksft_print_msg("[RUN] %s ... with swapped-out large anon folio\n", desc); + do_run_with_large(fn, LARGE_RUN_PTE_SWAPOUT, lafsize); +} + +static void run_with_single_pte_of_laf(test_fn fn, const char *desc) +{ + ksft_print_msg("[RUN] %s ... with single PTE of large anon folio\n", desc); + do_run_with_large(fn, LARGE_RUN_SINGLE_PTE, lafsize); +} + +static void run_with_single_pte_of_laf_swap(test_fn fn, const char *desc) +{ + ksft_print_msg("[RUN] %s ... with single PTE of swapped-out large anon folio\n", desc); + do_run_with_large(fn, LARGE_RUN_SINGLE_PTE_SWAPOUT, lafsize); +} + +static void run_with_partial_mremap_laf(test_fn fn, const char *desc) +{ + ksft_print_msg("[RUN] %s ... with partially mremap()'ed large anon folio\n", desc); + do_run_with_large(fn, LARGE_RUN_PARTIAL_MREMAP, lafsize); +} + +static void run_with_partial_shared_laf(test_fn fn, const char *desc) +{ + ksft_print_msg("[RUN] %s ... with partially shared large anon folio\n", desc); + do_run_with_large(fn, LARGE_RUN_PARTIAL_SHARED, lafsize); +} + static void run_with_hugetlb(test_fn fn, const char *desc, size_t hugetlbsize) { int flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB; @@ -1105,6 +1142,14 @@ static void run_anon_test_case(struct test_case const *test_case) run_with_partial_mremap_thp(test_case->fn, test_case->desc); run_with_partial_shared_thp(test_case->fn, test_case->desc); } + if (lafsize) { + run_with_laf(test_case->fn, test_case->desc); + run_with_laf_swap(test_case->fn, test_case->desc); + run_with_single_pte_of_laf(test_case->fn, test_case->desc); + run_with_single_pte_of_laf_swap(test_case->fn, test_case->desc); + run_with_partial_mremap_laf(test_case->fn, test_case->desc); + run_with_partial_shared_laf(test_case->fn, test_case->desc); + } for (i = 0; i < nr_hugetlbsizes; i++) run_with_hugetlb(test_case->fn, test_case->desc, hugetlbsizes[i]); @@ -1126,6 +1171,8 @@ static int tests_per_anon_test_case(void) if (thpsize) tests += 8; + if (lafsize) + tests += 6; return tests; } @@ -1680,15 +1727,74 @@ static int tests_per_non_anon_test_case(void) return tests; } +static size_t large_anon_folio_size(void) +{ + /* + * There is no interface to query this. But we know that it must be less + * than thpsize. So we map a thpsize area, aligned to thpsize offset by + * thpsize/2 (to avoid a hugepage being allocated), then touch the first + * page and see how many pages get faulted in. + */ + + int max_order = __builtin_ctz(thpsize); + size_t mmap_size = thpsize * 3; + char *mmap_mem = NULL; + int order = 0; + char *mem; + size_t offset; + int ret; + + /* For alignment purposes, we need 2.5x the requested size. */ + mmap_mem = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (mmap_mem == MAP_FAILED) + goto out; + + /* Align the memory area to thpsize then offset it by thpsize/2. */ + mem = (char *)(((uintptr_t)mmap_mem + thpsize) & ~(thpsize - 1)); + mem += thpsize / 2; + + /* We might get a bigger large anon folio when MADV_HUGEPAGE is set. */ + ret = madvise(mem, thpsize, MADV_HUGEPAGE); + if (ret) + goto out; + + /* Probe the memory to see how much is populated. */ + mem[0] = 0; + for (order = 0; order < max_order; order++) { + offset = (1 << order) * pagesize; + if (!pagemap_is_populated(pagemap_fd, mem + offset)) + break; + } + +out: + if (mmap_mem) + munmap(mmap_mem, mmap_size); + + if (order == 0) + return 0; + + return offset; +} + int main(int argc, char **argv) { int err; + gup_fd = open("/sys/kernel/debug/gup_test", O_RDWR); + pagemap_fd = open("/proc/self/pagemap", O_RDONLY); + if (pagemap_fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + pagesize = getpagesize(); thpsize = read_pmd_pagesize(); if (thpsize) ksft_print_msg("[INFO] detected THP size: %zu KiB\n", thpsize / 1024); + lafsize = large_anon_folio_size(); + if (lafsize) + ksft_print_msg("[INFO] detected large anon folio size: %zu KiB\n", + lafsize / 1024); nr_hugetlbsizes = detect_hugetlb_page_sizes(hugetlbsizes, ARRAY_SIZE(hugetlbsizes)); detect_huge_zeropage(); @@ -1698,11 +1804,6 @@ int main(int argc, char **argv) ARRAY_SIZE(anon_thp_test_cases) * tests_per_anon_thp_test_case() + ARRAY_SIZE(non_anon_test_cases) * tests_per_non_anon_test_case()); - gup_fd = open("/sys/kernel/debug/gup_test", O_RDWR); - pagemap_fd = open("/proc/self/pagemap", O_RDONLY); - if (pagemap_fd < 0) - ksft_exit_fail_msg("opening pagemap failed\n"); - run_anon_test_cases(); run_anon_thp_test_cases(); run_non_anon_test_cases();