From patchwork Wed Sep 21 06:06:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 12983240 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F4052ECAAD8 for ; Wed, 21 Sep 2022 06:07:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CB9A6B0075; Wed, 21 Sep 2022 02:07:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 754B7940008; Wed, 21 Sep 2022 02:07:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CF2E940007; Wed, 21 Sep 2022 02:07:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 4D9376B0075 for ; Wed, 21 Sep 2022 02:07:00 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2A2E9C06FE for ; Wed, 21 Sep 2022 06:07:00 +0000 (UTC) X-FDA: 79935059400.21.471875F Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf28.hostedemail.com (Postfix) with ESMTP id 879E6C000C for ; Wed, 21 Sep 2022 06:06:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663740419; x=1695276419; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vMOxlk+IH0y/jJD/DO08c91l+llx8vnqZ8FEqWsnej4=; b=m+YMZEJA4Et195RUlu9WpNWq7bHU9pVayHONR0AwDaZdFFvFVmRb/IJI DTn4Pm2PW5U4sIJK8TnRKpppEY0+j6DTtqLREUjUmhOdrsMVEyiPBTXGB 22QEzVdM3KuErVO37s/kayGCWEMUZgCQ9gutb6EOSWCmcKsAEgVqqE32E tpycN1djDqJsqsjVWhgTvJaZ3oHvhhJPmwlo50Xyu9E5Po4CzY6uvDLVm VfgCpFZQCVV8nEoXkNkPs0qcKA+UZcrgs4eMljX27i0BQfnlUtik0xAgQ BJOSbjNHyMWtUiag8utm8TAZJY9a5rPxUP8A7bL+WK0H/pWE3FxhYlo05 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10476"; a="282956841" X-IronPort-AV: E=Sophos;i="5.93,332,1654585200"; d="scan'208";a="282956841" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Sep 2022 23:06:59 -0700 X-IronPort-AV: E=Sophos;i="5.93,332,1654585200"; d="scan'208";a="649913874" Received: from yhuang6-mobl2.sh.intel.com ([10.238.5.245]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Sep 2022 23:06:56 -0700 From: Huang Ying To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , Huang Ying , Zi Yan , Yang Shi , Baolin Wang , Oscar Salvador , Matthew Wilcox Subject: [RFC 3/6] mm/migrate_pages: restrict number of pages to migrate in batch Date: Wed, 21 Sep 2022 14:06:13 +0800 Message-Id: <20220921060616.73086-4-ying.huang@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220921060616.73086-1-ying.huang@intel.com> References: <20220921060616.73086-1-ying.huang@intel.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663740419; a=rsa-sha256; cv=none; b=AfDQvg7oz6UymHU1icNxOFHnBsQymy9tPukPqASn2/hYAZkvV5GEDSRdf6LH9MqV5uSjUe VWgpJjpTJ6Dv949NoCMJ25PUNI5LaLHDUkBO0/mUVSXQFHJhWeN54r7jX2FDZh7zojKJpw ZQVyxImi/dGLTlI/xFXETLwJtx61fnA= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=m+YMZEJA; spf=pass (imf28.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663740419; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6sI578zPO31SGuIvnfv9ruk/rXq1w1XVZD9mdppxNr0=; b=oG+FN19tDDZJEzyCRp3628MtOuCPXFNXWN7y+F5Kktsb//TG7g3ACsRlcct/Qj03igyVUN xmzFrOQVjgh7z+WZRJA9aESf1RwTdGPlh9+i1g8k7p2KkK4JSQxOHQyrrGjMVxCf3ZqvXv RdfXlLfv+1p+f8KEWNEOge/2K0f1yDk= X-Rspamd-Server: rspam06 X-Rspam-User: X-Stat-Signature: 6zggqsw95b17fuf75dhh8kmx8tpwbmra X-Rspamd-Queue-Id: 879E6C000C Authentication-Results: imf28.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=m+YMZEJA; spf=pass (imf28.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com X-HE-Tag: 1663740419-734486 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a preparation patch to batch the page unmapping and moving for the normal pages and THP. If we had batched the page unmapping, all pages to be migrated would be unmapped before copying the contents and flags of the pages. If the number of pages that were passed to migrate_pages() was too large, too many pages would be unmapped. Then, the execution of their processes would be stopped for too long time. For example, migrate_pages() syscall will call migrate_pages() with all pages of a process. To avoid this possible issue, in this patch, we restrict the number of pages to be migrated to be no more than HPAGE_PMD_NR. That is, the influence is at the same level of THP migration. Signed-off-by: "Huang, Ying" Cc: Zi Yan Cc: Yang Shi Cc: Baolin Wang Cc: Oscar Salvador Cc: Matthew Wilcox --- mm/migrate.c | 93 +++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 67 insertions(+), 26 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 4a81e0bfdbcd..1077af858e36 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1439,32 +1439,7 @@ static inline int try_split_thp(struct page *page, struct list_head *split_pages return rc; } -/* - * migrate_pages - migrate the pages specified in a list, to the free pages - * supplied as the target for the page migration - * - * @from: The list of pages to be migrated. - * @get_new_page: The function used to allocate free pages to be used - * as the target of the page migration. - * @put_new_page: The function used to free target pages if migration - * fails, or NULL if no special handling is necessary. - * @private: Private data to be passed on to get_new_page() - * @mode: The migration mode that specifies the constraints for - * page migration, if any. - * @reason: The reason for page migration. - * @ret_succeeded: Set to the number of normal pages migrated successfully if - * the caller passes a non-NULL pointer. - * - * The function returns after 10 attempts or if no pages are movable any more - * because the list has become empty or no retryable pages exist any more. - * It is caller's responsibility to call putback_movable_pages() to return pages - * to the LRU or free list only if ret != 0. - * - * Returns the number of {normal page, THP, hugetlb} that were not migrated, or - * an error code. The number of THP splits will be considered as the number of - * non-migrated THP, no matter how many subpages of the THP are migrated successfully. - */ -int migrate_pages(struct list_head *from, new_page_t get_new_page, +static int migrate_pages_batch(struct list_head *from, new_page_t get_new_page, free_page_t put_new_page, unsigned long private, enum migrate_mode mode, int reason, unsigned int *ret_succeeded) { @@ -1709,6 +1684,72 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, return rc; } +/* + * migrate_pages - migrate the pages specified in a list, to the free pages + * supplied as the target for the page migration + * + * @from: The list of pages to be migrated. + * @get_new_page: The function used to allocate free pages to be used + * as the target of the page migration. + * @put_new_page: The function used to free target pages if migration + * fails, or NULL if no special handling is necessary. + * @private: Private data to be passed on to get_new_page() + * @mode: The migration mode that specifies the constraints for + * page migration, if any. + * @reason: The reason for page migration. + * @ret_succeeded: Set to the number of normal pages migrated successfully if + * the caller passes a non-NULL pointer. + * + * The function returns after 10 attempts or if no pages are movable any more + * because the list has become empty or no retryable pages exist any more. + * It is caller's responsibility to call putback_movable_pages() to return pages + * to the LRU or free list only if ret != 0. + * + * Returns the number of {normal page, THP, hugetlb} that were not migrated, or + * an error code. The number of THP splits will be considered as the number of + * non-migrated THP, no matter how many subpages of the THP are migrated successfully. + */ +int migrate_pages(struct list_head *from, new_page_t get_new_page, + free_page_t put_new_page, unsigned long private, + enum migrate_mode mode, int reason, unsigned int *pret_succeeded) +{ + int rc, rc_gether = 0; + int ret_succeeded, ret_succeeded_gether = 0; + int nr_pages; + struct page *page; + LIST_HEAD(pagelist); + LIST_HEAD(ret_pages); + +again: + nr_pages = 0; + list_for_each_entry(page, from, lru) { + nr_pages += compound_nr(page); + if (nr_pages > HPAGE_PMD_NR) + break; + } + if (nr_pages > HPAGE_PMD_NR) + list_cut_before(&pagelist, from, &page->lru); + else + list_splice_init(from, &pagelist); + rc = migrate_pages_batch(&pagelist, get_new_page, put_new_page, private, + mode, reason, &ret_succeeded); + ret_succeeded_gether += ret_succeeded; + list_splice_tail_init(&pagelist, &ret_pages); + if (rc == -ENOMEM) { + rc_gether = rc; + goto out; + } + rc_gether += rc; + if (!list_empty(from)) + goto again; +out: + if (pret_succeeded) + *pret_succeeded = ret_succeeded_gether; + list_splice(&ret_pages, from); + + return rc_gether; +} + struct page *alloc_migration_target(struct page *page, unsigned long private) { struct folio *folio = page_folio(page);