From patchwork Wed Jan 22 17:43:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 11346193 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C08C2139A for ; Wed, 22 Jan 2020 17:44:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 83BF724683 for ; Wed, 22 Jan 2020 17:44:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PXQ9nF6e" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 83BF724683 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A8F016B026D; Wed, 22 Jan 2020 12:44:02 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A66606B026E; Wed, 22 Jan 2020 12:44:02 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A2756B026F; Wed, 22 Jan 2020 12:44:02 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id 84D186B026D for ; Wed, 22 Jan 2020 12:44:02 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 2FC2E2C1F for ; Wed, 22 Jan 2020 17:44:02 +0000 (UTC) X-FDA: 76405993524.24.bear60_6108e4a9ee30 X-Spam-Summary: 2,0,0,aeee78fc6fb011a8,d41d8cd98f00b204,alexander.duyck@gmail.com,:kvm@vger.kernel.org:mst@redhat.com:linux-kernel@vger.kernel.org:willy@infradead.org:mhocko@kernel.org::akpm@linux-foundation.org:mgorman@techsingularity.net:vbabka@suse.cz:yang.zhang.wz@gmail.com:nitesh@redhat.com:konrad.wilk@oracle.com:david@redhat.com:pagupta@redhat.com:riel@surriel.com:lcapitulino@redhat.com:dave.hansen@intel.com:wei.w.wang@intel.com:aarcange@redhat.com:pbonzini@redhat.com:dan.j.williams@intel.com:alexander.h.duyck@linux.intel.com:osalvador@suse.de,RULES_HIT:41:152:355:379:960:966:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1544:1593:1594:1605:1711:1730:1747:1777:1792:2194:2196:2197:2198:2199:2200:2201:2202:2393:2553:2559:2562:2693:2731:2739:2895:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4037:4118:4250:4321:4385:4401:4423:4605:5007:6119:6261:6653:6742:7576:7903:8660:8957:9010:9121:9413:10004:11026:11473:11658:11914 :12043:1 X-HE-Tag: bear60_6108e4a9ee30 X-Filterd-Recvd-Size: 7909 Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Wed, 22 Jan 2020 17:44:01 +0000 (UTC) Received: by mail-pg1-f179.google.com with SMTP id k25so3893765pgt.7 for ; Wed, 22 Jan 2020 09:44:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=yW1O0jjXB9z8bWi4RkuBpy5DqO+Vy5mEgFaRwUh2ELE=; b=PXQ9nF6eD9JqXqGbBcHCAnoNaPt6TbCdFAlSKlRh3x/clFanOB7zXHzp/j/btaJeQe sAdDm0NRgY2EXlZYVSmb4vDeib4pDcZ5cuXq4PNBrrbvaPxAac/m1TWqdx+JJcJYtLZu oKVRDrOH6f1kb3Gzp6ZQxwD4FYFshRc82FwTHwQNY5w4TbuHBrK+0MnMkkVY15xhdK67 bXmQe0Sj2+r9z+Vb0aQrbIk+XV8VvgMXUnwIM3/3hS0aOAZesFR8y54xkDUZ96H8zFMH PjI0Ubd8/EOVV3jQmqRBehTB2bgxZIXhHLIjQfdYK9313sUtY4VY8gganAVhavYMnhY6 EUeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=yW1O0jjXB9z8bWi4RkuBpy5DqO+Vy5mEgFaRwUh2ELE=; b=gcUfh0uV6vf26SENOBx5MmotybrsvmZiMte5U0ZQ1/1vRY7U9PKbMfBwScspfF8Vbx Q4X+hozA33kmHsEZ+2O3EeEBhmuZ4WPz7RTMdMs6UW4a5qb4gXuSCr2wwFQvFBZCVz3w nQDvfx7j1yFoWjpdV+69Nzy69xaLQUfOgOimkjOIyYQOD0oaTjHaPoF2bbarA+54IOBY ktGyjgxL6LEDM+miIf5soj4J6/shDmdGKuJBN4DlgO7wiioSA89ZIQCw5sWDoMJigfD8 hJoZBW1vHk5iubRE82YjN2KdioGiEyudUka6RZeC0ulOPO5L99fd+GwansRBCBfJSq1S pxTQ== X-Gm-Message-State: APjAAAXvmDROXG1+QbuIY2W9x+7Wgy03RZnBZbgQrG/pleUZoT9P5KuG rlh8ZOE7bquXZmGUgI44vt4= X-Google-Smtp-Source: APXvYqz0A9CG8xZ7h+97UwPy5e048YsSkZo8foaSUuWR8QkXhlxpnBzoYJBxR+QAKI+++O7kQfKYsw== X-Received: by 2002:a63:c748:: with SMTP id v8mr11843658pgg.451.1579715040621; Wed, 22 Jan 2020 09:44:00 -0800 (PST) Received: from localhost.localdomain ([2001:470:b:9c3:9e5c:8eff:fe4f:f2d0]) by smtp.gmail.com with ESMTPSA id a10sm4093429pjq.8.2020.01.22.09.43.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Jan 2020 09:44:00 -0800 (PST) Subject: [PATCH v16.1 8/9] mm/page_reporting: Add budget limit on how many pages can be reported per pass From: Alexander Duyck To: kvm@vger.kernel.org, mst@redhat.com, linux-kernel@vger.kernel.org, willy@infradead.org, mhocko@kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, mgorman@techsingularity.net, vbabka@suse.cz Cc: yang.zhang.wz@gmail.com, nitesh@redhat.com, konrad.wilk@oracle.com, david@redhat.com, pagupta@redhat.com, riel@surriel.com, lcapitulino@redhat.com, dave.hansen@intel.com, wei.w.wang@intel.com, aarcange@redhat.com, pbonzini@redhat.com, dan.j.williams@intel.com, alexander.h.duyck@linux.intel.com, osalvador@suse.de Date: Wed, 22 Jan 2020 09:43:59 -0800 Message-ID: <20200122174359.6142.21993.stgit@localhost.localdomain> In-Reply-To: <20200122173040.6142.39116.stgit@localhost.localdomain> References: <20200122173040.6142.39116.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alexander Duyck In order to keep ourselves from reporting pages that are just going to be reused again in the case of heavy churn we can put a limit on how many total pages we will process per pass. Doing this will allow the worker thread to go into idle much more quickly so that we avoid competing with other threads that might be allocating or freeing pages. The logic added here will limit the worker thread to no more than one sixteenth of the total free pages in a given area per list. Once that limit is reached it will update the state so that at the end of the pass we will reschedule the worker to try again in 2 seconds when the memory churn has hopefully settled down. Again this optimization doesn't show much of a benefit in the standard case as the memory churn is minmal. However with page allocator shuffling enabled the gain is quite noticeable. Below are the results with a THP enabled version of the will-it-scale page_fault1 test showing the improvement in iterations for 16 processes or threads. Without: tasks processes processes_idle threads threads_idle 16 8283274.75 0.17 5594261.00 38.15 With: tasks processes processes_idle threads threads_idle 16 8767010.50 0.21 5791312.75 36.98 Signed-off-by: Alexander Duyck --- include/linux/page_reporting.h | 1 + mm/page_reporting.c | 33 ++++++++++++++++++++++++++++++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 32355486f572..3b99e0ec24f2 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -5,6 +5,7 @@ #include #include +/* This value should always be a power of 2, see page_reporting_cycle() */ #define PAGE_REPORTING_CAPACITY 32 struct page_reporting_dev_info { diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 6885e74c2367..3bbd471cfc81 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -114,6 +114,7 @@ void __page_reporting_notify(void) struct list_head *list = &area->free_list[mt]; unsigned int page_len = PAGE_SIZE << order; struct page *page, *next; + long budget; int err = 0; /* @@ -125,12 +126,39 @@ void __page_reporting_notify(void) spin_lock_irq(&zone->lock); + /* + * Limit how many calls we will be making to the page reporting + * device for this list. By doing this we avoid processing any + * given list for too long. + * + * The current value used allows us enough calls to process over a + * sixteenth of the current list plus one additional call to handle + * any pages that may have already been present from the previous + * list processed. This should result in us reporting all pages on + * an idle system in about 30 seconds. + * + * The division here should be cheap since PAGE_REPORTING_CAPACITY + * should always be a power of 2. + */ + budget = DIV_ROUND_UP(area->nr_free, PAGE_REPORTING_CAPACITY * 16); + /* loop through free list adding unreported pages to sg list */ list_for_each_entry_safe(page, next, list, lru) { /* We are going to skip over the reported pages. */ if (PageReported(page)) continue; + /* + * If we fully consumed our budget then update our + * state to indicate that we are requesting additional + * processing and exit this list. + */ + if (budget < 0) { + atomic_set(&prdev->state, PAGE_REPORTING_REQUESTED); + next = page; + break; + } + /* Attempt to pull page from list and place in scatterlist */ if (*offset) { if (!__isolate_free_page(page, order)) { @@ -146,7 +174,7 @@ void __page_reporting_notify(void) } /* - * Make the first non-processed page in the free list + * Make the first non-reported page in the free list * the new head of the free list before we release the * zone lock. */ @@ -162,6 +190,9 @@ void __page_reporting_notify(void) /* reset offset since the full list was reported */ *offset = PAGE_REPORTING_CAPACITY; + /* update budget to reflect call to report function */ + budget--; + /* reacquire zone lock and resume processing */ spin_lock_irq(&zone->lock);