From patchwork Tue Sep 24 15:05:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sistare X-Patchwork-Id: 13811012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C01ACF9C6B for ; Tue, 24 Sep 2024 15:05:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B77D26B00A7; Tue, 24 Sep 2024 11:05:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B27BA6B00A8; Tue, 24 Sep 2024 11:05:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A16896B00AA; Tue, 24 Sep 2024 11:05:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7FCC16B00A7 for ; Tue, 24 Sep 2024 11:05:48 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CE630141BD8 for ; Tue, 24 Sep 2024 15:05:47 +0000 (UTC) X-FDA: 82599956334.17.E10DBD1 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf23.hostedemail.com (Postfix) with ESMTP id 81C02140032 for ; Tue, 24 Sep 2024 15:05:44 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=EDqk+Xsk; dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf23.hostedemail.com: domain of steven.sistare@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=steven.sistare@oracle.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727190247; a=rsa-sha256; cv=none; b=wLL2wrWjrWNCnpLzcOxBQ/+qd47VAZJueMtdoxiHILti9P80p0cgrVoBDjWsffjXKhSmhy oezjy8fzQqMkLhC6KAzmQ/l7JmqdKtsa7SOEUMTGodnfnjGw02+aTu8OyHQGkKCV+53IWq ITV9KKFTyTkgKVy8Vc36RrCdozy+QSU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=EDqk+Xsk; dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf23.hostedemail.com: domain of steven.sistare@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=steven.sistare@oracle.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727190247; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references:dkim-signature; bh=Snn0P7WbP6pr/ETPqI5fIvp4ze/88hhLSVY7T7UPeqg=; b=Y209k8sqyCmYzt4uIq4/Pui5+zqt4WP5/KcVw40sde5/CMK7yq39RLBb5k+py7mq3LwjW1 xqjeJZh9XLMXzkY2SQGt9w3dEvqVLOnJiNXNjQDcOcQeME335Fjm75Kqme7nyu6Npx6MZJ iN17HpP0m3XxANZxBZJZr6IBk/jcevw= Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48OEQV8X000920; Tue, 24 Sep 2024 15:05:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h= from:to:cc:subject:date:message-id; s=corp-2023-11-20; bh=Snn0P7 WbP6pr/ETPqI5fIvp4ze/88hhLSVY7T7UPeqg=; b=EDqk+XskqKn4wYbQUiwOSz wPxZK6xie4mjWmg3zmpXIKXeJ6s2icW9XkIorkQCks5Sn9KcRpM/fPWiR24hwDD6 9ApvAbsPRy40KwojYT5CInDpAejHimAoAH6Unj0Edj2XRfaZZziyPmrQwUGUOwYl atWigB3qLzUjRdYQyyb69iYRku6BTG+RjFOPa1NB7yszrixBCCkFroloUbE4F2+S UzYul2wSj9eK/mltzeL5m8enTi7DCKVC6S4SaQGLOPsz0perYTWrSmQeCFKEBbY6 4ax71M4y6cz1vBvDHFxb0wmw9W5PfDOe468KzrxCjs8MKSdfg2URBBZPm8npn/VA == Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 41smx359qx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 24 Sep 2024 15:05:35 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 48OEjVO7029624; Tue, 24 Sep 2024 15:05:34 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 41smk9be3h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 24 Sep 2024 15:05:34 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 48OEuXG8023995; Tue, 24 Sep 2024 15:05:34 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 41smk9be22-1; Tue, 24 Sep 2024 15:05:34 +0000 From: Steve Sistare To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Jason Gunthorpe , Matthew Wilcox , Steve Sistare Subject: [PATCH V2] mm/gup: folio_split_user_page_pin Date: Tue, 24 Sep 2024 08:05:32 -0700 Message-Id: <1727190332-385657-1-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-24_02,2024-09-24_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 bulkscore=0 suspectscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2408220000 definitions=main-2409240108 X-Proofpoint-ORIG-GUID: 9h6vJ-WHV0b90WOJ57NuZNxs7a5OgfZa X-Proofpoint-GUID: 9h6vJ-WHV0b90WOJ57NuZNxs7a5OgfZa X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 81C02140032 X-Stat-Signature: 489uaushjx5x3f1qxnh9zqng9fpcd9ex X-Rspam-User: X-HE-Tag: 1727190344-606954 X-HE-Meta: U2FsdGVkX19G/YJKd1e0PCe5/iC0uWU6Vo0goN2SRqyMXHfAzkfRNhtLDBnt44QFAjEuYnf0e8B2uQlp8EExLJ1FIIYtMwBtCQNvsJB0iGmBrgayD599H5JtSSl4MrG97rWypRfrVq8qGcwMBEDfDEw6Nf/0jMqHaOXs//cCRrYMbEhKvDyTG1yV66kdNmi6fvZgcHuJ4+5NeGYkt9uQE7lHMA+Cos+Lr/CYBWuWf4NAuLMA38z41OCXTwA/gZCY0ZYKDiz5q6eluaCguzKvqTY/1urqPvfhqOX0svZ6Dr6BVEfZn28sDWHEYp47llcpQKZOmIub2hb0Cd0RfIu9qXucUOcdbVSinemA/7GdNm42MH4ltUih+2K25AlJvccJJ07Hb7Tzx42HPJpBk+xV3vxfX/YdmOzhpw/++pn7IpgSA/qOMWUyfTkZ02Bd6JHVmE4thc9NipxmOeI52FvaTepnL3ct+G4i0vghpushVdtsVPuq0xgHU1o1mlsaxxHTnzNBSXZ7fz/M1xKY4GvJ/ngEcZ7MjY5yVIwpURW3uQHMTY98f27T9/XgNNVbeSSdjppcAe43zs7Ad8IcgHzqI42vLKa73chaSkd/RgD3ta5/bFDiDw6T1hrJ13lFAFE0mz3iIOkKZeVO2BVTle7O5gid7KhccxL1sdeCtbgWEi6/4HdA4SVTkkC3pkNRcPBsiYuUCY0lHsbatybRPycEGQHNX/zC6ackU1lWEm2FsMoH8GrES03Z2lc8YS1jg/JiZsguPGJIV7p2mwu48B0gdsDKaz5RjXlnScWztCri6+t80ibag/8GVFuEOFrN5vqk9EIb9c0jJjKXAf7OsAMQXJaZtrxZVLZZGjTNdzJy51StUzemNMXSw65SRmKPyGCe0RROleZUQnPq5qGUuQHc8Y3K5XNggts3UCzeu4DBo5eUGZ7LKJFNx8NgdC6G+DlXpZK4rmjJ+/F0YCHPMEg T5vHKWSC zxybMztlj10KiZ7EGy20tT+/E7CBe/2NdRTe7eCjKREzj7IPa2eCRMzw44RKBoKUD3jK2vfct3X0vSNZqrOu5+dDiaDHoogRx6Vlxsxp721swfJIBtSTeQStJbXXliRJE6RyGVjLSAaeFoXGcGjp1Ted1fZVLlCpBZGSy5kmyQZDH724Q08RguIh1H6BQKYtOUydwIFh+8jSymrswCHsPNBoEcI9KaQFciNonhCQZ3rhnLaURg56uQWHJcIQQBQElCOhq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Export a function that repins a high-order folio at small-page granularity. This allows any range of small pages within the folio to be unpinned later. For example, pages pinned via memfd_pin_folios and modified by folio_split_user_page_pin could be unpinned via unpin_user_page(s). Suggested-by: Jason Gunthorpe Signed-off-by: Steve Sistare Reviewed-by: Jason Gunthorpe --- In V2 this has been renamed from repin_folio_unhugely, but is otherwise unchanged from V1. --- --- include/linux/mm.h | 1 + mm/gup.c | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 13bff7c..b0b572d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2521,6 +2521,7 @@ long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, struct folio **folios, unsigned int max_folios, pgoff_t *offset); +void folio_split_user_page_pin(struct folio *folio, unsigned long npages); int get_user_pages_fast(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages); diff --git a/mm/gup.c b/mm/gup.c index fcd602b..94ee79dd 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3733,3 +3733,23 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, return ret; } EXPORT_SYMBOL_GPL(memfd_pin_folios); + +/** + * folio_split_user_page_pin() - split the pin on a high order folio + * @folio: the folio to split + * @npages: The new number of pages the folio pin reference should hold + * + * Given a high order folio that is already pinned, adjust the reference + * count to allow unpin_user_page_range() and related to be called on a + * the folio. npages is the number of pages that will be passed to a + * future unpin_user_page_range(). + */ +void folio_split_user_page_pin(struct folio *folio, unsigned long npages) +{ + if (!folio_test_large(folio) || is_huge_zero_folio(folio) || + npages == 1) + return; + atomic_add(npages - 1, &folio->_refcount); + atomic_add(npages - 1, &folio->_pincount); +} +EXPORT_SYMBOL_GPL(folio_split_user_page_pin);