From patchwork Fri Aug 9 16:08:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13758919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC222C3DA4A for ; Fri, 9 Aug 2024 16:09:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 412036B00A0; Fri, 9 Aug 2024 12:09:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3BF876B00A1; Fri, 9 Aug 2024 12:09:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1EC056B00A2; Fri, 9 Aug 2024 12:09:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F25026B00A0 for ; Fri, 9 Aug 2024 12:09:26 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A6A8F412A6 for ; Fri, 9 Aug 2024 16:09:26 +0000 (UTC) X-FDA: 82433191932.02.9635CC4 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 855421A0026 for ; Fri, 9 Aug 2024 16:09:24 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AIkLRe78; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723219710; a=rsa-sha256; cv=none; b=xJtOf3lKG51d3vUYilqm3hMTXsRR+Rj6V/Mdf07bjy/NV+0upSS2cp8Ja35JvS4CNDF+FY YM0zjZ2DvUIxLNjzxg8mXDZhngJ06MauO3vCIo0uyxvjzMQ2NmA4CmYCyh0fylyxhUawsJ q54ODlmi5o+XrsnIMz6j1zWikpJGI8w= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AIkLRe78; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723219710; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pOevS25kl1+p5C9bnayjnL4IWCIfturozNfzs94vCz8=; b=GYSfXKV6XgvKePto0hn8uYOn2riTiAbG2BaC0V/2XrKOBLukf3x+Ecmc1npcXBbk8WdkBT nH/YeLe2eQcoKJ71HUVnovaZT4FHBHxUA3O0l+tfoCspWdHJ8gx0eobnkM6AEwMfsGC4w0 1rDEzQkZhh6+ztwP2F1bCzh6UgPY75E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723219763; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pOevS25kl1+p5C9bnayjnL4IWCIfturozNfzs94vCz8=; b=AIkLRe78Zxz80FdBVVF94Ok0iKX6PlppauloyrnCDhoRbT+1guSOkX9X42aP94brCJe12a NoqdEgbelGqZckfmJHl7mqnvKHmIQJjnd3miBCiJfR1Az3sUNGJUYHjyQuZRJrblqsWaBC 2AXDbRkHHI9NIA3Z6gxYJaHz4YaPMDQ= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-442-ATh9A-zFM0-5lWbJvGWAIQ-1; Fri, 09 Aug 2024 12:09:22 -0400 X-MC-Unique: ATh9A-zFM0-5lWbJvGWAIQ-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-6b7678caf7dso6678686d6.1 for ; Fri, 09 Aug 2024 09:09:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723219761; x=1723824561; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pOevS25kl1+p5C9bnayjnL4IWCIfturozNfzs94vCz8=; b=bvL9K7lvwYzhOHLVBT6QOBj4F0YSXlX+9O/+vsG4cfJNq1EDTtajILzw+opvS0hxOY 8p4P0VR9RM5OTySEszH3yrjLbLI9/KVEcbbd+8+GjXyO+Slcf5y0ScG7eJuQnleBevx7 +SQlM8uE/hMY3BnYAvYwnpLoR8Z512MS3wRT+cgRtXBrvnaW24KxFnOLHLwFlzlNR45R K96fzbB+myugMWwtswlzDlaPUm2ZQCsiNtY/WljIpuc40kPMHhz+TkHu/hQc/RPFZeaf bEln3cSCe8bbsysDYEBL7PICwXLhxLZXW7zHn/74zZmmr4vtxnqd+ugZAcHz7cIYXZS6 8uCg== X-Gm-Message-State: AOJu0YwhtDUYYya5v2MlrMMPwwJjTGO+WtqEiNmi/80IL9P8acm3V0wv MeVu9rghHwz9+QftZkGWhheU4SbpjTsYom2cXoYj6MNjDvoqn3NlrCWbnj7zArTQzQntJfF3NNF qo5TRBir60vPGsXqJPBsb4hiR0WDnPDOAGK5eWNiQZuXmIV9YHhwxL33oaYlER5CFr6tPLLFSJG Lwh9WW/Iez1CMa+RM3nJTu3fSXINFHlTZv X-Received: by 2002:a05:622a:1a8b:b0:44f:89e3:d119 with SMTP id d75a77b69052e-453126babcemr11885761cf.12.1723219761733; Fri, 09 Aug 2024 09:09:21 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGHUujWoHeQHvUCs2OAJSjyQdbxlWd9FX1yi8fb3RZKwv0O5abV6g6G64GeuNGe2uyjoJD48A== X-Received: by 2002:a05:622a:1a8b:b0:44f:89e3:d119 with SMTP id d75a77b69052e-453126babcemr11885331cf.12.1723219761262; Fri, 09 Aug 2024 09:09:21 -0700 (PDT) Received: from x1n.redhat.com (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-451c870016csm22526741cf.19.2024.08.09.09.09.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Aug 2024 09:09:20 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Sean Christopherson , Oscar Salvador , Jason Gunthorpe , Axel Rasmussen , linux-arm-kernel@lists.infradead.org, x86@kernel.org, peterx@redhat.com, Will Deacon , Gavin Shan , Paolo Bonzini , Zi Yan , Andrew Morton , Catalin Marinas , Ingo Molnar , Alistair Popple , Borislav Petkov , David Hildenbrand , Thomas Gleixner , kvm@vger.kernel.org, Dave Hansen , Alex Williamson , Yan Zhao Subject: [PATCH 03/19] mm: Mark special bits for huge pfn mappings when inject Date: Fri, 9 Aug 2024 12:08:53 -0400 Message-ID: <20240809160909.1023470-4-peterx@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240809160909.1023470-1-peterx@redhat.com> References: <20240809160909.1023470-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 855421A0026 X-Stat-Signature: fay3uthgbzk5ni4njriyreznqwzxihim X-Rspam-User: X-HE-Tag: 1723219764-553049 X-HE-Meta: U2FsdGVkX18xAymM046ybIm6d2q9j0dLlMmxGcys9lHDLErnE+o8kOlKRHPbgyYknBodjkkEdXTJP7KNyPUdPfg6jI8uvPJVQRACQMrNVV8UiePyWNFnLsMJIRPQ9x5NM46DEPZGtoYoz7PASqFtS2VRBbUHSPDpxMXHA9dTv3llia9G+VtkeApbtmrt0NbmjMyAfYxPFcgBxB6iK6Ye5e4uO79RNzPLfxLRoh0tdZS9GZxgS/mY/dKu1ciwaAFvzUxGstm5BKV2UQH0T2/8GdswoCeiRS23tIBRvII7Lpsue+KJRgJL89Z4SZayeC2rPQnY9EEJle5togWBx8NZYXtsuoxHYvKXNlICsuyAfH3JIkoBYuMS26VFpv0smCZA9hzgf2NhDvkoCUeSOVCxv22tCQYoaSgUrv5dfuy2iYmsVuOV+Bxf/VDlsAvrYKLrAdhGjoT/Bd48gSwJY6jl9sfmKk3qzuxG6DafrstASALbFYcGkmQH+YaVJKQ31lyj9yem78Y/VYQs37BxiPOTmVahJugOEEB+9AqAld3mYOnKQYWCQc4kjhDYQOfKjW2l34niUWhIU1knvq9ipK73oVDrEY2Cqf6KrdTkhkIOlQZK9uql1B0/O2PpO8pn9BvTvjhJf/daDs5wHjh89VAmfomIi17QU3InwrIrj3nSZzsjC+fI3TCf0dzObFnphvHu9Hp6LW2+DfkkGEeJqqZEtOmq3pW3Zxmbu0OYmIKIxcq+ORAaVfZzoRUz6VtI8X88D+QPZmlZqD0KNUttrIUTsB1hjY1YSpZtrXN91rUrRyGrmHj+oLn25hxNdiWC4Mp5ts6ywWANt3p9mwjMNCRaECBaxsPUcsXryKB7tIkKS6DsxsBBQEk7vGv2d6jFlQsO3kfKXBarpheLV2vEWgsi56UXg99a0h7eYsKflFb59NOFLhz/OVHvkCyEFg/gb1Hqg+92sXyTeMJ7DOTLNRj XIEd4y5r 072BuNrGHF920Op+xazCdD/XbJczyWjly8soJiQQoSI/jJnjuqGUUM5weJVRKZMNdyWMSQy5f+b28BI0x3wokZkfmg/x/d3BPpDZ+YZutkVyIdNZy38n4iPe01Q0allLCKS+JMyPJkrJHIT4cWFDXvTp4iGAUQEHJkoJa/hNaM+kz28sj89fFk61zWfEPKbxO5cd5+fRcu1qrreg6am2MfCdBmKLoKSlsG+9b+qjfHfWRhFyGNiyfjjf2S2SoKeeMw8yHeod/MjDYyN1zGtKe460DHJxnvy0YgaxDDDwMqX7HnV2Xn7wx0BAC4q7Gms/jhTPnaPn7H33ECgbNaFSWoDpfZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We need these special bits to be around to enable gup-fast on pfnmaps. Mark properly for !devmap case, reflecting that there's no page struct backing the entry. Signed-off-by: Peter Xu Reviewed-by: Jason Gunthorpe --- mm/huge_memory.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 39c401a62e87..e95b3a468aee 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1162,6 +1162,8 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, entry = pmd_mkhuge(pfn_t_pmd(pfn, prot)); if (pfn_t_devmap(pfn)) entry = pmd_mkdevmap(entry); + else + entry = pmd_mkspecial(entry); if (write) { entry = pmd_mkyoung(pmd_mkdirty(entry)); entry = maybe_pmd_mkwrite(entry, vma); @@ -1258,6 +1260,8 @@ static void insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr, entry = pud_mkhuge(pfn_t_pud(pfn, prot)); if (pfn_t_devmap(pfn)) entry = pud_mkdevmap(entry); + else + entry = pud_mkspecial(entry); if (write) { entry = pud_mkyoung(pud_mkdirty(entry)); entry = maybe_pud_mkwrite(entry, vma);