From patchwork Tue Jan 24 02:11:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13113304 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61A0AC05027 for ; Tue, 24 Jan 2023 02:11:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CA1326B0071; Mon, 23 Jan 2023 21:11:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C29BC6B0072; Mon, 23 Jan 2023 21:11:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA5436B0074; Mon, 23 Jan 2023 21:11:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 912EA6B0071 for ; Mon, 23 Jan 2023 21:11:22 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 669B614043C for ; Tue, 24 Jan 2023 02:11:22 +0000 (UTC) X-FDA: 80388065604.27.2FEAB96 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf11.hostedemail.com (Postfix) with ESMTP id A5E4A40008 for ; Tue, 24 Jan 2023 02:11:20 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="Q2OJ7E/o"; spf=pass (imf11.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674526280; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=SCGlc7mWURATKW4pBp/mMbnvAjkNxvQ5RvlFZ+5SXrA=; b=A4bn8VJ9vxee6dmVRgUpZLp2yc8zhaq/EIifSlBobXEIqaAYc/OvcAu3gWzbW43nWo781R zgU42Vza5xz8YNKvhsoQYQeVIkv386S+q66cOD/URfCsUwA5SMEj7QD0nXA98J1HfLKvHf BuuhKbhLjTMYcZYx85zelknHOxLxKtE= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="Q2OJ7E/o"; spf=pass (imf11.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674526280; a=rsa-sha256; cv=none; b=svYHickAfNcZL4+mWywyxfNH4ZOLOmJmkRXsTVx9Z2zj/fxNnf2PwO7947Tt8rCKKHn2LX QFL58EKRsNuTbSwqof80oObkSrjUUed7cMLVVd6qZ95epHkVJVPSBQCFZ/fL2xzUDCm7Rm 3VJ1ClBpi0cNVmMQt4+L6e6gIM5WMHc= Received: by mail-pg1-f174.google.com with SMTP id e10so10388550pgc.9 for ; Mon, 23 Jan 2023 18:11:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=SCGlc7mWURATKW4pBp/mMbnvAjkNxvQ5RvlFZ+5SXrA=; b=Q2OJ7E/ovDJZRJ1/WrC7I1yuDO3Hb6jh2iai0Os4K2WdyHqq0h1cH5JrkPYY0LyrTY 7pubiTyoZItmkh0TVFAVAGmXnvHqYco6E/bJ9SAJWFGJV+UzGBvkYmb/zpo5/BffZziZ JaUWdDXTBbvbtQ1aNpDtvOl58OyifVwiKon0KX67c66ujtxL6CwZEzUbjnw8bOnoGm74 8k3YxA5iHDZF2LERPK0k7Ti8XPJH2AN7w1s1gLDWa+ggmjT+8d+vp02GVoi90UC4ndJK Op15iN5zK7IgomVvrth+xB+TALm5s/EdbhomubldZ9LTG+Ta/mwoayP7RSJmJgZfUz5E 2wOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SCGlc7mWURATKW4pBp/mMbnvAjkNxvQ5RvlFZ+5SXrA=; b=8Du1UGClB6wZPlsA+wcapUkXMkUYrdaWXkgOtfxGy1LmJN5t/M/71SmBRv/XtlSOlt vxYiU1p4y7dUOSeBd118xNz2Rnt3jcguwb+Wt1ONxipoB2p5Lh42M4CZ0EpDKFNXtWy1 UJhb8shsNHbRurtj7ihDqMAO4uyPoIPjlvLIEgCLF9MY15bb1uLA2+JXCm0FotXIeLpq psMazqcCzIqINGgNzT9fgD7Gk0pPNvcvqVopMHXQMiAANr2bWZS9lQDCOyl9VtVkyH3k ddIS9IzCLSJCYQqG+NL4zrv6Spu5MvDAQCuRcaRGWDRikGBkIiosxXlNjpgOhob8ZwHH 5baQ== X-Gm-Message-State: AFqh2krCL9Hl08/S8NUGaB+qL0azxORuApj/mHxYjgB7rt++LZiaSl1M 1gK6GY203HARMqqySGCQEVs= X-Google-Smtp-Source: AMrXdXssaRcK8VwNrNmgxgUxK3PVNJ9Gzeiag04Vn7nr3eYgvMh+HvBnpxQv18Cho9tj1oOKoXLqNQ== X-Received: by 2002:a62:e406:0:b0:581:7430:aba with SMTP id r6-20020a62e406000000b0058174300abamr28478685pfh.10.1674526279369; Mon, 23 Jan 2023 18:11:19 -0800 (PST) Received: from localhost (fwdproxy-prn-020.fbsv.net. [2a03:2880:ff:14::face:b00c]) by smtp.gmail.com with ESMTPSA id z67-20020a626546000000b0058bacd6c4e8sm254143pfb.207.2023.01.23.18.11.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Jan 2023 18:11:18 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, linux-api@vger.kernel.org, kernel-team@meta.com Subject: [PATCH v7 0/3] cachestat: a new syscall for page cache state of files Date: Mon, 23 Jan 2023 18:11:15 -0800 Message-Id: <20230124021118.154078-1-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A5E4A40008 X-Rspam-User: X-Stat-Signature: ngx3b1kscunp7uxuux6ojkgg8yh6skg3 X-HE-Tag: 1674526280-156837 X-HE-Meta: U2FsdGVkX1/a3TTfrOLaIXKu2ZjdKXKjzg6O6WfvkptUxokCPkBalu4iiJq6qvdRHZYE3y913asc3reJkPiymBBo3hEn/z6spXQvKEavrsDZP2fQzIy2o8HVcaGT+Uxpwa1UMeB5h14PNBkTACHdER69L6a7eamQexW1eTX7bcXr/2JkPSJM45FYEWJ8vwBHvBSYAOGaRx7ttYjq9Zh1uchQwSP/FrX9xVLDei+3f3VWmPok3JZrBJ3tUR03DFaEFam1bIVTTxnugFzcgdhhhl3C6yldYoAqCN3uRhRAzaE6bfOX0N15c3G7QzOsDBno1BxkwFzlp5u4EZOpa6Q9rbrJCLdUNYvDQqe3MYi0rXydOfeoeULB3BuSBo6iSz0HE0y9sA614xVHr055Oiv9E4UzdsX+p1eXtumXYkB8ZZxoidndKmuV0p/P0xQR/kh69sFGDbl2oGFGeKowcS0Z2mHMR910unG19wLUgbhRWDZ58bHIbNu+umQDqYcUiHfNcoDY+Kn7IAGqu6UTSwlq5HaxkRG2p8gXeXyzjHy1iefOFckB12W+dgxXMFVVI5jMZCm7tJVn82ZyZS7G0fWBew1n0NSB64HjiZEYIdVYw/45cE+WEDWhvQz8OX5sC2cyb9CQ1KcMWu8yzpCYEU6UZq8Q6JkQBDjQno173O+wo+LnipHJIJ18OCcfopcEfySWIfYjTQOtEUj1iwkQKftS4ztzS/FrY/Cq7EjPkeejsdHOIJZ7jQQ8s74NPyKRSwZ1VpYvow1wsw7pFeSMUxIXASOwL8ykZR2uDlJ9wRpkXYf5d5lytInWVV9M5WqPfFciE12ibnpxHd4nAerSDBNFUc73pYlg6HMEKfGHa4FyNFlas65gUIwuYIhkgPIVMZMr9PnKQrPvbhTJ8nHXTc5Oh0J0UhO95Cspf3Yl9UL8RBls7RxHJehraSGUDKy8sTyya8lb4gIeprG3d5lnLOD hlxAggNa 5y9HN9O0RT4TiuFX5tJeyIn/0K1vJ8S22x+AlzQxIsOnTYYTQevHaXVxQJ3wPS4IuDX+yRwPT5XpZtYPWbUaeqjNGmi9y+i9u8aKxQ6aDXYVus9tBjfe2WtlvR1zem2VJtBuTnWhQ3ygquQ9t7iwd4KLqA/GmaBwEqj3CCzNMhDaRLrGzTGiAXBslpl7UKkwiELQjpuCuXmz2P1cF52HD3RvH7E7iZIDO/kGxesMymGemsJqc9rVTD7or1jklCcB+r80IzcEv5HyZ3FauCrcW/O5dA9jPBCgjOniHHDrPHPfFZWG01GHjiu+BWxk87ijzKSWCrKsjSGmiaKOSMDVvn+L3xFCqKmVzI+l6pB7Iq0rdYAJaADV3XivvMnI+37NMb4GUHYiRBJd7fzDSD+TQd6ttzXYSQ+LuTURHvskL88NXWVvl6pShdJRVZksUhrChTlbONY4+Ev6cX+Xx3JbOFcAXISB+7SBBWp9lWkcSYZXtbAL+Kk1anu8MLIFPKe5uBOjaS88lt+sM2h+ZRC5OurISkQuquofDiBPgtQ45bsMSw4s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Changelog: v7: * Fix and use lru_gen_test_recent (suggested by Brian Foster) (patch 2) * Small formatting and organizational fixes v6: * Add a missing fdput() (suggested by Brian Foster) (patch 2) * Replace cstat_size with cstat_version (suggested by Brian Foster) (patch 2) * Add conditional resched to the xas walk. (suggested by Hillf Danton) (patch 2) v5: * Separate first patch into its own series. (suggested by Andrew Morton) * Expose filemap_cachestat() to non-syscall usage (patch 2) (suggested by Brian Foster). * Fix some build errors from last version. (patch 2) * Explain eviction and recent eviction in the draft man page and documentation (suggested by Andrew Morton). (patch 2) v4: * Refactor cachestat and move it to mm/filemap.c (patch 3) (suggested by Brian Foster) * Remove redundant checks (!folio, access_ok) (patch 3) (suggested by Matthew Wilcox and Al Viro) * Fix a bug in handling multipages folio. (patch 3) (suggested by Matthew Wilcox) * Add a selftest for shmem files, which can be used to test huge pages (patch 4) (suggested by Johannes Weiner) v3: * Fix some minor formatting issues and build errors. * Add the new syscall entry to missing architecture syscall tables. (patch 3). * Add flags argument for the syscall. (patch 3). * Clean up the recency refactoring (patch 2) (suggested by Yu Zhao) * Add the new Kconfig (CONFIG_CACHESTAT) to disable the syscall. (patch 3) (suggested by Josh Triplett) v2: * len == 0 means query to EOF. len < 0 is invalid. (patch 3) (suggested by Brian Foster) * Make cachestat extensible by adding the `cstat_size` argument in the syscall (patch 3) There is currently no good way to query the page cache state of large file sets and directory trees. There is mincore(), but it scales poorly: the kernel writes out a lot of bitmap data that userspace has to aggregate, when the user really doesn not care about per-page information in that case. The user also needs to mmap and unmap each file as it goes along, which can be quite slow as well. This series of patches introduces a new system call, cachestat, that summarizes the page cache statistics (number of cached pages, dirty pages, pages marked for writeback, evicted pages etc.) of a file, in a specified range of bytes. It also include a selftest suite that tests some typical usage This interface is inspired by past discussion and concerns with fincore, which has a similar design (and as a result, issues) as mincore. Relevant links: https://lkml.indiana.edu/hypermail/linux/kernel/1302.1/04207.html https://lkml.indiana.edu/hypermail/linux/kernel/1302.1/04209.html For comparison with mincore, I ran both syscalls on a 2TB sparse file: Using mincore: real 0m37.510s user 0m2.934s sys 0m34.558s Using cachestat: real 0m0.009s user 0m0.000s sys 0m0.009s This series should be applied on top of: workingset: fix confusion around eviction vs refault container https://lkml.org/lkml/2023/1/4/1066 This series consist of 3 patches: Nhat Pham (3): workingset: refactor LRU refault to expose refault recency check cachestat: implement cachestat syscall selftests: Add selftests for cachestat MAINTAINERS | 7 + arch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/ia64/kernel/syscalls/syscall.tbl | 1 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + include/linux/fs.h | 3 + include/linux/swap.h | 1 + include/linux/syscalls.h | 4 + include/uapi/asm-generic/unistd.h | 5 +- include/uapi/linux/mman.h | 9 + init/Kconfig | 10 + kernel/sys_ni.c | 1 + mm/filemap.c | 154 +++++++++++ mm/workingset.c | 142 ++++++---- tools/testing/selftests/Makefile | 1 + tools/testing/selftests/cachestat/.gitignore | 2 + tools/testing/selftests/cachestat/Makefile | 8 + .../selftests/cachestat/test_cachestat.c | 260 ++++++++++++++++++ 27 files changed, 573 insertions(+), 47 deletions(-) create mode 100644 tools/testing/selftests/cachestat/.gitignore create mode 100644 tools/testing/selftests/cachestat/Makefile create mode 100644 tools/testing/selftests/cachestat/test_cachestat.c base-commit: 1440f576022887004f719883acb094e7e0dd4944 prerequisite-patch-id: 171a43d333e1b267ce14188a5beaea2f313787fb