From patchwork Mon Sep 16 11:07:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dev Jain X-Patchwork-Id: 13805279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBD71C3ABB2 for ; Mon, 16 Sep 2024 11:08:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4732A6B0093; Mon, 16 Sep 2024 07:08:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F9226B0095; Mon, 16 Sep 2024 07:08:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24C306B0096; Mon, 16 Sep 2024 07:08:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 03C686B0093 for ; Mon, 16 Sep 2024 07:08:17 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8644C14033B for ; Mon, 16 Sep 2024 11:08:17 +0000 (UTC) X-FDA: 82570327434.20.463606E Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 03E0A10000B for ; Mon, 16 Sep 2024 11:08:15 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726484786; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DKmFxZ7weftyY68zIWFg14sI4661+BB6jesUl0LX+DE=; b=cQhpE1+BFrPbGvJGg+sRWuHVHcrN9odnAopOqbvNQXCFvVLyur5jAREcOR+rMCTp35P+NW 1dYHcS7GyUcFIYU4EIo/XYtsIBXo6vA+/yB6ZVEHTmYyZYDnUpUuyYsHElOvXc/ioV7xcN eSiSLrySbSO3FJnnBPeObE6vGTC9jUo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726484786; a=rsa-sha256; cv=none; b=BCaxkyknxmc7SjTnvkCWNAcqRnuQIsOmFiFjU7z6zG4UYsXaW924XxbFeG258MzuxjUVmM rTQqSA1MJmpDWKYmQbMkvUNnIGBtWE23795xvfjA/ra3tjxa4xPoJBGGf/VYIgZtLc7asN XpwBK46PSi4Kwtf1va7+3LmeV7UjXCE= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7F8571516; Mon, 16 Sep 2024 04:08:44 -0700 (PDT) Received: from e116581.blr.arm.com (e116581.arm.com [10.162.42.11]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8E6013F64C; Mon, 16 Sep 2024 04:08:10 -0700 (PDT) From: Dev Jain To: akpm@linux-foundation.org, david@redhat.com, willy@infradead.org Cc: ryan.roberts@arm.com, anshuman.khandual@arm.com, baohua@kernel.org, hughd@google.com, ioworker0@gmail.com, wangkefeng.wang@huawei.com, baolin.wang@linux.alibaba.com, gshan@redhat.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Dev Jain Subject: [PATCH v2 2/2] mm: Compute first_set_pte to eliminate evaluating redundant ranges Date: Mon, 16 Sep 2024 16:37:54 +0530 Message-Id: <20240916110754.1236200-3-dev.jain@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240916110754.1236200-1-dev.jain@arm.com> References: <20240916110754.1236200-1-dev.jain@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 03E0A10000B X-Stat-Signature: bgmihqrpzixqqogbdddibx81y4h7n65s X-HE-Tag: 1726484895-42278 X-HE-Meta: U2FsdGVkX1+63KugMnhKtSiTcw/FUrKnZWDuYX/ZX5yaqEWXkJn9OY7feEgnFzHECt6YzGu283EmEM+gIA4r96AlMWM2lWnF3zHCQuNf2Re1cJAL35fRJ7azShF1oygKfUPLTugDwCx9lul3T8nRCsy7rZbIeQjyEuR+lQHVn3wWZu8xgxD+3PcOLvgnDN9RBY/ncOCkS7E2jGBr54VomaANcJ7rLQkbi9l8v7FP0K7vqjdYod4Zbl3OrI2lgw3orC/U/qDM9pDVE3oaOg0JCQ6pD2hYJQJ3lfGOVfSZI9eJuK4pPVPtj0d2BISLoMHUBc81eGTswNvZ0cSU2aaSZaVBmHzD1yHdiM7XeNUnFMa/TiZ0ElHzJRLnBz6dFWKZhwk9JmRKaFKiEvkdP27c7DFyDOg197jdrzjciq19bCZr+PVjJh7tDknKhdc8rFnN1ZxP3NAVKTcayfH7RfjCOqu1pY7Kvv/UcaGpkFVCHBo7bJAbDz+q32vb0g98g9LlbEZgh5wyILM/7gmskU9qWvlTFkJvGRGlynBUfIL+Y0ACYY/U8gF5Qd68wRzn8GeMZ1A0mlxLdPclfUxrtWMW/Gf6lnLwCFDv7dE7HJDWcJj6dsrj+2Sj31xIcwTTfisoYdDTlZUGTyqLuGjFIOydHdJ+yq+kLHZv2r5t6bUCHAYwFpKw17oVdBVRJqkYTZAPQT8ANmN2UDxK+D/pigrXXYP/nofZW+vIL1IbFNXUUdNWsmQqnsvs+nNo7O1Cj1k+vpFIzk5Dzob1iAxX2R0/ztFIAle+QBk6F98GOvDnqkizxrhUXLEv+ACoIUPiNurjqudl32TAESD54VzCkIIT08BClEHhEElL1Z7iCvhcArk1onK0NbxhpEMrzOxzv2LHgOW9rmBAAZlg8RiHDb/kJHIoobHbjlNtItDkNxdgKmg6W7no6eU79hAt3lSdWf2WvuQvgFZiE74/JrlHQaF tc6nsJ+9 HpH3G/KV5FvJeXST/WaQyAMiKoxWHvBlQAgfIeXOdJ+3fNOLRHFMc9t0I8oUAMnLHZhiuV9+BJPRGZsaruUXEPtbcbgBTW9r0bo5qEUKKF8QIjAjnnrryHylKZtWjge4E4YTFRg6J1asi7P70Ed6ImOmSfGAy/1lmgTT0DwVXwpYcE5TWH4BtpSUS9QUXG/ZoMGIB1yxWyMkHhz8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For an mTHP allocation, we need to check, for every order, whether for that order, we have enough number of contiguous PTEs empty. Instead of iterating the while loop for every order, use some information, which is the first set PTE found, from the previous iteration to eliminate some cases. The key to understanding the correctness of the patch is that the ranges we want to examine form a strictly decreasing sequence of nested intervals. Suggested-by: Ryan Roberts Signed-off-by: Dev Jain --- mm/memory.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 8bb1236de93c..e81c6abe09ce 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4633,10 +4633,11 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; #ifdef CONFIG_TRANSPARENT_HUGEPAGE + pte_t *first_set_pte = NULL, *align_pte, *pte; unsigned long orders; struct folio *folio; unsigned long addr; - pte_t *pte; + int max_empty; gfp_t gfp; int order; @@ -4671,8 +4672,23 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) order = highest_order(orders); while (orders) { addr = ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - if (pte_range_none(pte + pte_index(addr), 1 << order) == 1 << order) + align_pte = pte + pte_index(addr); + + /* Range to be scanned known to be empty */ + if (align_pte + (1 << order) <= first_set_pte) + break; + + /* Range to be scanned contains first_set_pte */ + if (align_pte <= first_set_pte) + goto repeat; + + /* align_pte > first_set_pte, so need to check properly */ + max_empty = pte_range_none(align_pte, 1 << order); + if (max_empty == 1 << order) break; + + first_set_pte = align_pte + max_empty; +repeat: order = next_order(&orders, order); }