From patchwork Wed Sep 21 06:06:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 12983237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9858C32771 for ; Wed, 21 Sep 2022 06:06:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D96C86B0072; Wed, 21 Sep 2022 02:06:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D4711940007; Wed, 21 Sep 2022 02:06:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C35996B0074; Wed, 21 Sep 2022 02:06:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B57696B0072 for ; Wed, 21 Sep 2022 02:06:54 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 829E51C5C54 for ; Wed, 21 Sep 2022 06:06:54 +0000 (UTC) X-FDA: 79935059148.05.28EB512 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf02.hostedemail.com (Postfix) with ESMTP id E83CD8000E for ; Wed, 21 Sep 2022 06:06:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663740413; x=1695276413; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=xsZ2Mvy4tpdwu8/Jy1B6jf7NAn4Nf2Vm//ir3UuH1OY=; b=C81HvUrWXU3qYAJku/ms0SsYiidAQ6wQnQ47vCDwI8ajUWBeOwxsZmgK oIfmGiOf/3IMGZrpmMWsUPLd+ZtT47hvBkf5gWVQ06a9DInFjZAsJCKop rpC1dR3mrqsUyWK24rY8jkesjnuE+kVoxHTPYk1MyblyN85KQpygbk65O nvUlj+QcG/rBr/2y2yEUjaVSYRIdDPemsioUAu71Ir4eKB36JIfexWRNj P2i7G5TKCoXCMwrwQQy4sCXJH6TlUkKyknh3Di75bIMCilLPvv6PrjqQn Vc8XedmGeS9pL6PxLDHcqTdmntoVciF3Fas7sDpoVkB0hiBljl4wPUZbh w==; X-IronPort-AV: E=McAfee;i="6500,9779,10476"; a="282956807" X-IronPort-AV: E=Sophos;i="5.93,332,1654585200"; d="scan'208";a="282956807" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Sep 2022 23:06:50 -0700 X-IronPort-AV: E=Sophos;i="5.93,332,1654585200"; d="scan'208";a="649913832" Received: from yhuang6-mobl2.sh.intel.com ([10.238.5.245]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Sep 2022 23:06:48 -0700 From: Huang Ying To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , "Huang, Ying" , Zi Yan , Yang Shi , Baolin Wang , Oscar Salvador , Matthew Wilcox Subject: [RFC 0/6] migrate_pages(): batch TLB flushing Date: Wed, 21 Sep 2022 14:06:10 +0800 Message-Id: <20220921060616.73086-1-ying.huang@intel.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663740414; a=rsa-sha256; cv=none; b=D4kFEuxj9ls85bLxjRwlXmJpYXdkjVHluR7xxwh4W0sNoLPf7B9+w7aqcgeeolrrNZTI9/ IyjgC4a4Re655dwKhbqsToFlCUHT901neEibGjN5GJHG4sl/74+Oa4A/0nFYvbrdGVDprD EfGCTnQmJKXzP1IKmD8FjBVgIHZdVtw= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=C81HvUrW; spf=pass (imf02.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663740414; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=co6TtTB9rdOAxaI5a0pa1GKU6dt6GmTUvZnZasRsuiQ=; b=Q42Q58HYOorDoq7Ygm2tmqRUkke24XeHwFpDBhbYD3sbKNcq8d2v6mBmsPdcJ4D1nRY0A6 yN6W655z4AP7FbMuBDySmbEUTo7OL3sFUtjLNBuzrDV341M+1QtcggAF0G4YpTB7zu6tcG RrRGuwwWmaF/R2R+9n1j2gLzOlW0vGU= X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E83CD8000E Authentication-Results: imf02.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=C81HvUrW; spf=pass (imf02.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Stat-Signature: 5cgoj1ek5sika7yqqqt95ykfbzyubehb X-HE-Tag: 1663740412-991761 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Huang, Ying" Now, migrate_pages() migrate pages one by one, like the fake code as follows, for each page unmap flush TLB copy restore map If multiple pages are passed to migrate_pages(), there are opportunities to batch the TLB flushing and copying. That is, we can change the code to something as follows, for each page unmap for each page flush TLB for each page copy for each page restore map The total number of TLB flushing IPI can be reduced considerably. And we may use some hardware accelerator such as DSA to accelerate the page copying. So in this patch, we refactor the migrate_pages() implementation and implement the TLB flushing batching. Base on this, hardware accelerated page copying can be implemented. If too many pages are passed to migrate_pages(), in the naive batched implementation, we may unmap too many pages at the same time. The possibility for a task to wait for the migrated pages to be mapped again increases. So the latency may be hurt. To deal with this issue, the max number of pages be unmapped in batch is restricted to no more than HPAGE_PMD_NR. That is, the influence is at the same level of THP migration. We use the following test to measure the performance impact of the patchset, On a 2-socket Intel server, - Run pmbench memory accessing benchmark - Run `migratepages` to migrate pages of pmbench between node 0 and node 1 back and forth. With the patch, the TLB flushing IPI reduces 99.1% during the test and the number of pages migrated successfully per second increases 291.7%. This patchset is based on v6.0-rc5 and the following patchset, [PATCH -V3 0/8] migrate_pages(): fix several bugs in error path https://lore.kernel.org/lkml/20220817081408.513338-1-ying.huang@intel.com/ The migrate_pages() related code is converting to folio now. So this patchset cannot apply recent akpm/mm-unstable branch. This patchset is used to check the basic idea. If it is OK, I will rebase the patchset on top of folio changes. Best Regards, Huang, Ying