From patchwork Tue Jun 11 01:04:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Yang X-Patchwork-Id: 13692726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50FCCC27C4F for ; Tue, 11 Jun 2024 01:04:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AE35C6B00A2; Mon, 10 Jun 2024 21:04:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A922F6B00A3; Mon, 10 Jun 2024 21:04:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 932EC6B00A4; Mon, 10 Jun 2024 21:04:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 756966B00A2 for ; Mon, 10 Jun 2024 21:04:37 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 00841810AC for ; Tue, 11 Jun 2024 01:04:36 +0000 (UTC) X-FDA: 82216812594.19.D5F2B7F Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf19.hostedemail.com (Postfix) with ESMTP id 305841A0006 for ; Tue, 11 Jun 2024 01:04:35 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="I9ua/Uei"; spf=pass (imf19.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718067875; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=o/e1voOxQLnVD8oRgm9L6T4MRE4ZONIiEombO/Pd7l0=; b=d4KQOu2LVhyGkEneBTOrDMJORMgElsdyHgGCcfkzgukdl8FkI3VXSHeJeqDTIORME+vKwd lwWj2CZKB/SNPOGMPGVkmgkPcSm4VMpvqr4NQ9wuyYGFIQwlPJw15wpMQb+RW9VSYVXdI9 eEObTCDb6ke32jyWgJmrrjp1vpNNrpY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718067875; a=rsa-sha256; cv=none; b=QN/Mj89raecgoCty0W23UkdWb7ufHXyR8B4HYZ1fI7FfFMngZpxQnKtRjlIsUjfqO8UOAh ukJ5CEH9i1xI7+ZyaGSB0jQGlERyVBjX+Tc3Nw/hSuPNSjUxewEBlzKcfeCTTyHqybK0+g 1Y9pPtuky50Vbw148zFXEJvtHgRrvaE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="I9ua/Uei"; spf=pass (imf19.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-7042882e741so1944068b3a.2 for ; Mon, 10 Jun 2024 18:04:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718067874; x=1718672674; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=o/e1voOxQLnVD8oRgm9L6T4MRE4ZONIiEombO/Pd7l0=; b=I9ua/UeixY+ZZCP+VpOGdmK+ZKAGOVpLpfHQym6p8W04nfns1wN10sTtV0pKtAx5c/ T1NIVUawCcyZ1f3dOm6zMzeEfW3gVC1Z2YBISFSQ4vQW/NuqDiFQATz3Ad5Br5jPD8hM Fpou+23o+azj+zH6Q+NC7+DBUW0KAgN4CWzK8R1+002bJhSmavbmTRZr6pnmrn9MF4Tg dfXPGaHrQX01no6CGyuKrsw19X67BnGwXiRI9iuQdN/g1PE+BApwqdA6F2TwPjYbGT0w lxfzdNV4ECaNrcWt8H0sTtzdlPGvj8NIzK1FqbueROJ2YOcXHcRF7AGePsQLTRueVmwg IuBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718067874; x=1718672674; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=o/e1voOxQLnVD8oRgm9L6T4MRE4ZONIiEombO/Pd7l0=; b=r5vwgmpzvQlQLRYzm7LD0bRVl2Zeraa3aejk0keeN5IhPo2kXkvvU55nPOnWkVUgho Xf+ztNvjJX0u/5MQFIxMfIXze26KiQ5tTEGuKmBd9L0Ei9C4/mt5s7aFeB8LP/xxZT1L S0DGqH1I4je4mXLBVc9qV1p0M0sZaYRi1CL1LU7TXcdHS2/3IzAvqS7XwZ/sbvIkFPog VFVNP9Evd2tPCltOdBUIKmw6VHf/eGCkKXTxx8a9IHug7XFub9RvCmGaSVOfcQqIheM+ Q0BshLguDco6XKaMLoj2dmdOUyF/avfOsT/2E1gcnoub4tmI+wgeHgiY9dHorE86PwJi L0rQ== X-Forwarded-Encrypted: i=1; AJvYcCXVV9o6+eaamctis3IUoFO2yfnTnwY1ubjb3XDLcAGsfesHhZJvQQ5ODEsqLc1AOw93ee2bK1cd/2I7JYSdDxdi8mw= X-Gm-Message-State: AOJu0Yx39Th6hGiBt7CKBDWuGKVJZYGufKGuTo1NCuL46Com/iqyC9tY DYcT6V+QEWF04yV5JyxzAQOpXhF3/y9d8a0bYFYl58TkVmqTDyn0 X-Google-Smtp-Source: AGHT+IFiG0Bki8T01+0oqaiEQJxC+MT42/SLLey1eFmNRn2EjSlxQmSPt8ZWivilxFsnLAYY4kbI/w== X-Received: by 2002:a05:6a20:6a20:b0:1b5:cf26:ecfd with SMTP id adf61e73a8af0-1b5cf2704e2mr7248815637.24.1718067873837; Mon, 10 Jun 2024 18:04:33 -0700 (PDT) Received: from LancedeMBP.lan.lan ([2403:2c80:6::304c]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70410b976a9sm6153772b3a.147.2024.06.10.18.04.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 18:04:33 -0700 (PDT) From: Lance Yang To: akpm@linux-foundation.org Cc: ryan.roberts@arm.com, david@redhat.com, 21cnbao@gmail.com, baolin.wang@linux.alibaba.com, ziy@nvidia.com, fengwei.yin@intel.com, ying.huang@intel.com, libang.li@antgroup.com, willy@infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Lance Yang Subject: [PATCH v3 1/1] mm/mlock: implement folio_mlock_step() using folio_pte_batch() Date: Tue, 11 Jun 2024 09:04:18 +0800 Message-Id: <20240611010418.70797-1-ioworker0@gmail.com> X-Mailer: git-send-email 2.33.1 MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Stat-Signature: f36x7jc8bpe57ho3dndpa5aswwabsuce X-Rspamd-Queue-Id: 305841A0006 X-Rspam-User: X-HE-Tag: 1718067875-168656 X-HE-Meta: U2FsdGVkX18HM4Mm528iWEELOhxo96LO/1uPKYxqMKz3RDCXA5UIROLkMg0MS+4FT697+B4tdqH1MLuCIkY74vZqSvy3c+dkrAsZj0KGxFbiW4CBKHFrtnH6Zb50iO3zULLpayrjgT+roMnKpWeVtd/J9Q3b+aDfSCICGtE4yRU2hcLSkl3LHJegnY4clRYZpr6qVAzp1R2elpFhdJDRDR4J9Utxa869tveym4jPqI0SGBKuaeSmIGziPPrrDVqKOf7VMAy7Bo42RvCcEEbJHezAphIg1m6lxDzOaqsfyWC4kADiL6L7nI2y2xemAkr3ciaAaejDySDTDH2FLuJVJsfcKo/U84rqcw6Xscv60f2tazaJEVnzyAT/HcNHH+FXigwWwyN+FOiBQwGlCus2BLnWPkemOcNOocT2ZlSLg4vr125b9eEqJFwm/juavHLvVgClAyvw4Qwaw7aWQpC0rJKFDd8LK/AMMNREZkmjg7ZeOgWbH0bQfXlaOnhtcwHVTiQJupG87k4DWJLPCnPJwGtTAvJd4EgUsBehdBN/RL2DhXLQOWnmTUKY/YDjsobQ9RUCByM/z2ZDg2jwMgEZEpuA3hzW9j0jCOBzunry4ljqmMOesM7m+Zc+1mkOsm+4wZCjc7qN1Vw3IoMdkBpXr1mF9NqavMqAu5vkMJwgPBrzodvMZakGPqcaKoJQBzpXuy+xaSUMInzLkc+atXUCE0jrFCFOE3AuduUxtzVOnlG61WrDxgP6b0HHRvoItJ6wR5J/OATYnYGJPR88f540S28yf5wGnicp7z+wmxtTO3IvMNh/n2+AqS46Z21unnYl9EECGRbP8TnLqfjfWnMeZBUB2Ur2ZXdKmPrWK5Ek2qbZolf2FiazQdYCxhu8LaLNgrVc56XrOT+Z/E0YzgNniohLolwfxriKlLOVUW4Q+ZNILJjmJBFISPZ93BFZNGi5Qp+BPjJNeDt9+UOVDdX udFFUYbs KZf0XJXS12FXZX7m8JUenc0Jn2TF5ELBay5z3IAUZ9jy9fwVB4/TylYjIs9QlgKT+x22ItAPIJVsFjUbsd4mbV+YRkj3QOLuU3q1Xh7xk1Fl1gztIdcNKvmieMslbY51LNl9aWFyeK3xrlg16Ds7+5Fivrh7jNLgOdsjJeTwne1co4qB433LojB+JnCEl6Mle8UdiGBBWun/qFSUE60UHQ/HOLTypSjKgXqUciu/5GFwm2dR77+rDJKpm7AqZYVdf10jtwY99JBU7RGEPbbL5sXSncremZ25FtdlDtKg+aK3qgTXyTHnBcEA+LXaZeTwbpWR0MO3+D0NpVvrX9dlJrWIfukh30Wkh7jsluLDOBkBjec0NgiUbD+mYucsKqwfk/07lAleOlI7/gNPAE5mib7Ki9vH/mGt9z6waaf/jte/ThE0ou+nEXW227e+uAizCyBNrMjoUI9q6++xgavlRH0vL+Fg/ZsiPfjUVxArTz0LqgoKQ0gyO35fxs1mofl+PJjCZYM+NiN58V3HQmhqUzYtYp89v1amOl8yZDWY3oP9nPeUXwB4RRlzU+PCxKdRDF+VE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's make folio_mlock_step() simply a wrapper around folio_pte_batch(), which will greatly reduce the cost of ptep_get() when scanning a range of contptes. Acked-by: David Hildenbrand Reviewed-by: Baolin Wang Suggested-by: Barry Song <21cnbao@gmail.com> Suggested-by: Matthew Wilcox Signed-off-by: Lance Yang --- v2 -> v3: - Rebased to mm/mm-unstable - https://lore.kernel.org/linux-mm/20240603140745.83880-1-ioworker0@gmail.com/ v1 -> v2: - Remove the likely() hint (per Matthew) - Keep type declarations at the beginning of the function (per Matthew) - Make a minimum change (per Barry) - Pick RB from Baolin - thanks! - Pick AB from David - thanks! - https://lore.kernel.org/linux-mm/20240603033118.76457-1-ioworker0@gmail.com/ mm/mlock.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/mm/mlock.c b/mm/mlock.c index 30b51cdea89d..52d6e401ad67 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -307,26 +307,15 @@ void munlock_folio(struct folio *folio) static inline unsigned int folio_mlock_step(struct folio *folio, pte_t *pte, unsigned long addr, unsigned long end) { - unsigned int count, i, nr = folio_nr_pages(folio); - unsigned long pfn = folio_pfn(folio); + const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; + unsigned int count = (end - addr) >> PAGE_SHIFT; pte_t ptent = ptep_get(pte); if (!folio_test_large(folio)) return 1; - count = pfn + nr - pte_pfn(ptent); - count = min_t(unsigned int, count, (end - addr) >> PAGE_SHIFT); - - for (i = 0; i < count; i++, pte++) { - pte_t entry = ptep_get(pte); - - if (!pte_present(entry)) - break; - if (pte_pfn(entry) - pfn >= nr) - break; - } - - return i; + return folio_pte_batch(folio, addr, pte, ptent, count, fpb_flags, NULL, + NULL, NULL); } static inline bool allow_mlock_munlock(struct folio *folio,