From patchwork Tue Mar 30 21:35:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FCF3C433F8 for ; Tue, 30 Mar 2021 21:28:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0DD9F619D3 for ; Tue, 30 Mar 2021 21:28:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232813AbhC3V1b (ORCPT ); Tue, 30 Mar 2021 17:27:31 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:60806 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232606AbhC3V0z (ORCPT ); Tue, 30 Mar 2021 17:26:55 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULP80x122942; Tue, 30 Mar 2021 21:25:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=4xJD9dBWZTGkr9GFOZ1rhfq9w/F6myJVw9nCCu4BHdo=; b=x7KT/KmElZkSmMt9Xh83PLS/vlYmvyJlzUDJhbSnwusBME1b+vCuN1zs5zhT89TChMhQ GoCZmkT6MugnpZoT7iv3Fz6ZfjESgPZ+y4m+qdiRhuxTyHWp/drDNEeplh7BtMicH/aC GgQxH0+v4JnrjbX/GGN1874+n6Pkj9FhtniNH0wX9AYnF7SexCWDvGcq2D8Rb78t95gx ZKN0Me5Cl3Me6SdAhfvIYiGs+tj9Ik1pZFFUWHpabasP81r8n9krOs0ULdYoqZSfyf6s DzMKmb00DRG0B0gLICluF8WLE71ppStKR23HYJHt6JG8sfytaJjpmjYM8bE8dGQdEQuv 6g== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 37mafv0810-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:08 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULBSwq065456; Tue, 30 Mar 2021 21:25:07 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2175.outbound.protection.outlook.com [104.47.55.175]) by userp3030.oracle.com with ESMTP id 37mabkbbjw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OirDWEi8VWEriZN1uRvAU5pUl9YmPntfh+HWYrn11SOLG9J+/iemp3SfHeQruR+5TA/QpOAACAk4ze0lhmcP1EyL/XAU0udfak4KSEwrFBiUtIeVd/lrEUBB3+AUaeyGodXo4tuimwa/D1cgncP4eQkkn5bkvK4bw8Q/6ZEuElGrCITSK+IrLjDOrcC713M+aiNomMXDpgpJRieD6C4EP5Dt6lZjOgNBlSyoMUZpIQsWazprkPRA6WhYieJZ0RUZLPWwxJfYvE3neETvNQSacg9NQMlsEcSBoEAd/szqgsaOOYUcwNOv0n9SFWBYuUXbWlxdWVm2dwEr4/u3xTm20w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4xJD9dBWZTGkr9GFOZ1rhfq9w/F6myJVw9nCCu4BHdo=; b=GOdeUdMSyII7Cs2i6hNPekPCSatdI4eyIlLol9qhy9t8NMHiRtkxUdHBc5yWGF3AEkxRiwDlHBBS58i0uRJoH6mwURIuWQQEgH361fraSV5SimFSCvo96RiMeMTMFbO3EZ5l5JTa0b0Umw+F4ucWQs9Qn6Ce88aCdlhP7qSV+qgpEnRO9jCpMMflZpADXrTt0xOIMrOiiOl+l8U6bnsW8gAxlx+ecpRAnGhm1HYWNMOEbczDCQ4RuPd3ELGZ+28QBv9PsL7GRPPH8I/HJSbryClYIdIvjw2PNouYSIWqVuFuojAkZnHDZduHA4QbGjK41HqptdshbwDgLs+Dt/dL0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4xJD9dBWZTGkr9GFOZ1rhfq9w/F6myJVw9nCCu4BHdo=; b=x+XDOGbxB0HtZf/T5MHs3eOKq7k85WdF3zJic7LYO2Riwe1Q4+ApIXW27DZekYSjmra5x8mqhaLGogSGxRlLVpC1eTqWfK90Nvmp4hPFt6jugd6HNnGnAxy9u+eTknDd9pKXVir9/ghnUS/uraLz+T5/tWjqu1DAngj3JveEVBs= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3120.namprd10.prod.outlook.com (2603:10b6:208:122::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:25:04 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:04 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 01/43] mm: add PKRAM API stubs and Kconfig Date: Tue, 30 Mar 2021 14:35:36 -0700 Message-Id: <1617140178-8773-2-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:00 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 479d6f06-0fa4-420d-5ef3-08d8f3c2489c X-MS-TrafficTypeDiagnostic: MN2PR10MB3120: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zmNxxGMDVb833n+1r8H87LN53U9hnUk0AiVCYHK0V4+ueKvUaP+GxT7gUuBMkZ5w4PGbrx1B0vBNiPdc4fOoNFCnfyLPvw8ipMxHiZCupTmh5ku3D/P4NJXMH7PPxZzTbWk3yTqPB/KmvCN5zTLOItfXlpMaGr/JKbff2E0hyi+IalLO3ThAebMdUjVBT8X6OM8A8QU/nCNa8OZr9WB17IGP19fDLdd/M5ygiQ6r4WseaC0lbUiEEXyUsQz8sjhZk7GTVnw90TKwzFkLoINjNo//IHt4wFJ+vZcCc6ypVUPrQBpzV7SvDuQnWH3TzNuvIKtATBJUHDV40aCYxONtRTQYPLX3iqh4bf14FPYrxla31abMCR5f+twZayRN0DJCLxN58tzZyqCN7jkX9uUtNbKjlJxWhmqIOxSd05DQ0P5d+3bZMlvbsGnZjrW09sh2uXKxspWchYv9U5g9MG+8Wn8OmPmZAbt+pybRT5eqezB+4ELSuY/7SGEJSxgr8agNwknZ9FRKT99ULGdKEPA5VNCrQQCwdugdUjqrbtsF/x7KnOhztsSU4Zg17WVDhC8/MXTQFU7VaXSq24yt/HpQYN1Qackgp5G5kXzda4GflX8ltLwwf6X3BBxrkJl9SdAeqqPOOHvKiO23UYN6gIFAnw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(366004)(376002)(346002)(396003)(52116002)(316002)(186003)(6486002)(16526019)(5660300002)(2616005)(956004)(6666004)(8676002)(66556008)(44832011)(478600001)(83380400001)(7406005)(38100700001)(86362001)(7696005)(30864003)(66476007)(66946007)(4326008)(2906002)(26005)(7416002)(36756003)(8936002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: hEr9bVWsUdduhJN90F2Mc8Drii01BtIjPCeeCwddo/OmDIostBeWaTGKOqZkY+YeD66C5rjwh7hm5g8g+n+OWlJwBQ7S3EoUGoqSZjg4QOmv/nWptqqI6C7F/U0oeKHPmMXu0tN2jzTaor4liAk40bH4S+XR0VRKQuyje2ulOJdnBNdUMCTelwXERNm/0kZ8feBZoQBeYkzOAtb2bHKqEWBpow9+kEbRFOflc6notPAB5YTftEQPAx0D4j8aDr+ZjPt9hHhZKKBx9HrnbdSt/Cia7i+1RzBa4ykFpvrviwGO0+ptl21eEt1oQoGuCp8L9vDE8bNPQ0yaY6sqVD7A7rXbpOVeq0hnnEEhuyIXlnW8O6AqGATHz7xSzHFCGybMTbTyhCflvSlwPBbPscCTZzar6M4+zlYzedvzqreMB+4ZghRoDoPDLdSeyFbPktN9CwWjO/5M+r5VdA/S4ZxGxd9fWgnfngbOXTIlFkouhVsuttc2+1Ex33zwgucucuagBgElBCwknk8fFWzzzGteOuHfuGL5XLFc/r9YWDwBB0pwppCnYz5KlDA6Sh4nmHOanDyf9JuZBsrnOt8/QznzmBTJir0S9yGUgspRmP6aw8TEuvFY+9i+sDxxuXA7Pdhf2XrqqxnXca7/Hl24vu9Lwockp/jw0aNla7s59FmoPzYTQLUJ+JQpvSQvzXOW6zrzKb/N9Tv/XXNiMUPTRTlpYYWXjFqDV4ZNe9t85q9sdR4XC16eVhkHIGodEIp0Brjl5wg6DfeoDDUiO4ESgVleDlGZJG/OvOPaVhFPuNDkQSi4497SuIvHjf1t45Cs3NVJlaGcsDH1zXA/L4odfKwB4wwRj41mQdxjdjkr9XfcMeufc8jUWxklhw/rwmhrKD/plPWCVZIMMQSzteoxVJTbNS2tecNJ38EErOlCO10g7b2beN1igjVjkrveZI9ZcyuU2CbHDjvNt5fv6fRfvyOA4KhCXpUGuHxKCYiV8510vEEgPCowAyAPvOsVlZ1J2fAmM9apTrp8yL3lSmLVRAANbHs6mDBdYUNzGJ+uYgfxS6ihrcnsO4lXXguWLEqzEkX9L1keGUqSS3BFB52VgsmOcvqyOc81ndChdKYnFrh04QBjg6U6aRJpEGS9nIhCCCO3Qg3hVJTcr2yGNtPkcTIqcptIvizD85u4ujK4umxMcAnoHszPCHDqhBSBgJMKRxn9zovjMCxjfCep9eqlkDLQNyZ/zQnkVgRhX4cI2cyYvx15teBPPfG+Uq24T0Z2GktejLZXmuzeFNivYRzdMnXN1TSnJHk5+6Z/Ta5W6KMvxLw3PQwUN+oren8P0t3buVMJ X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 479d6f06-0fa4-420d-5ef3-08d8f3c2489c X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:04.1387 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YW8slpzheQ8YtgY7O3M1ilN1VBn+qiqMgafAZg/wgBjb5EL0V+c3MEhhiOYYzNorPvk0dvG56D17Y01eesv6Nm4RwT3an19ieV7xxpwhUD4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3120 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300155 X-Proofpoint-ORIG-GUID: Iaf0wUPbZLWZXVChLsPXaYEnFKXLlCmc X-Proofpoint-GUID: Iaf0wUPbZLWZXVChLsPXaYEnFKXLlCmc X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Preserved-across-kexec memory or PKRAM is a method for saving memory pages of the currently executing kernel and restoring them after kexec boot into a new one. This can be utilized for preserving guest VM state, large in-memory databases, process memory, etc. across reboot. While DRAM-as-PMEM or actual persistent memory could be used to accomplish these things, PKRAM provides the latency of DRAM with the flexibility of dynamically determining the amount of memory to preserve. The proposed API: * Preserved memory is divided into nodes which can be saved or loaded independently of each other. The nodes are identified by unique name strings. A PKRAM node is created when save is initiated by calling pkram_prepare_save(). A PKRAM node is removed when load is initiated by calling pkram_prepare_load(). See below * A node is further divided into objects. An object represents closely coupled data in the form of a grouping of pages and/or a stream of byte data. For example, the pages and attributes of a file. After initiating an operation on a PKRAM node, PKRAM objects are initialized for saving or loading by calling pkram_prepare_save_obj() or pkram_prepare_load_obj(). * For saving/loading data from a PKRAM node/object instances of the pkram_stream and pkram_access structs are used. pkram_stream tracks the node and object being operated on while pkram_access tracks the data type and position within an object. The pkram_stream struct is initialized by calling pkram_prepare_save() or pkram_prepare_load() and then pkram_prepare_save_obj() or pkram_prepare_load_obj(). Once a pkram_stream is fully initialized, a pkram_access struct is initialized for each data type associated with the object. After save or load of a data type for the object is complete, pkram_finish_access() is called. After save or load is complete for the object, pkram_finish_save_obj() or pkram_finish_load_obj() must be called followed by pkram_finish_save() or pkram_finish_load() when save or load is completed for the node. If an error occurred during save, the saved data and the PKRAM node may be freed by calling pkram_discard_save() instead of pkram_finish_save(). * Both page data and byte data can separately be streamed to a PKRAM object. pkram_save_file_page() and pkram_load_file_page() are used to stream page data while pkram_write() and pkram_read() are used to stream byte data. A sequence of operations for saving/loading data from PKRAM would look like: * For saving data to PKRAM: /* create a PKRAM node and do initial stream setup */ pkram_prepare_save() /* create a PKRAM object associated with the PKRAM node and complete stream initialization */ pkram_prepare_save_obj() /* save data to the node/object */ PKRAM_ACCESS(pa_pages,...) PKRAM_ACCESS(pa_bytes,...) pkram_save_file_page(pa_pages,...)[,...] /* for file pages */ pkram_write(pa_bytes,...)[,...] /* for a byte stream */ pkram_finish_access(pa_pages) pkram_finish_access(pa_bytes) pkram_finish_save_obj() /* commit the save or discard and delete the node */ pkram_finish_save() /* on success, or pkram_discard_save() * ... in case of error */ * For loading data from PKRAM: /* remove a PKRAM node from the list and do initial stream setup */ pkram_prepare_load() /* Remove a PKRAM object from the node and complete stream initializtion for loading data from it. */ pkram_prepare_load_obj() /* load data from the node/object */ PKRAM_ACCESS(pa_pages,...) PKRAM_ACCESS(pa_bytes,...) pkram_load_file_page(pa_pages,...)[,...] /* for file pages */ pkram_read(pa_bytes,...)[,...] /* for a byte stream */ */ pkram_finish_access(pa_pages) pkram_finish_access(pa_bytes) /* free the object */ pkram_finish_load_obj() /* free the node */ pkram_finish_load() Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 47 +++++++++++++ mm/Kconfig | 9 +++ mm/Makefile | 1 + mm/pkram.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 236 insertions(+) create mode 100644 include/linux/pkram.h create mode 100644 mm/pkram.c diff --git a/include/linux/pkram.h b/include/linux/pkram.h new file mode 100644 index 000000000000..a575da2d6c79 --- /dev/null +++ b/include/linux/pkram.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_PKRAM_H +#define _LINUX_PKRAM_H + +#include +#include +#include + +/** + * enum pkram_data_flags - definition of data types contained in a pkram obj + * @PKRAM_DATA_none: No data types configured + */ +enum pkram_data_flags { + PKRAM_DATA_none = 0x0, /* No data types configured */ +}; + +struct pkram_stream; +struct pkram_access; + +#define PKRAM_NAME_MAX 256 /* including nul */ + +int pkram_prepare_save(struct pkram_stream *ps, const char *name, + gfp_t gfp_mask); +int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags); + +void pkram_finish_save(struct pkram_stream *ps); +void pkram_finish_save_obj(struct pkram_stream *ps); +void pkram_discard_save(struct pkram_stream *ps); + +int pkram_prepare_load(struct pkram_stream *ps, const char *name); +int pkram_prepare_load_obj(struct pkram_stream *ps); + +void pkram_finish_load(struct pkram_stream *ps); +void pkram_finish_load_obj(struct pkram_stream *ps); + +#define PKRAM_ACCESS(name, stream, type) \ + struct pkram_access name + +void pkram_finish_access(struct pkram_access *pa, bool status_ok); + +int pkram_save_file_page(struct pkram_access *pa, struct page *page); +struct page *pkram_load_file_page(struct pkram_access *pa, unsigned long *index); + +ssize_t pkram_write(struct pkram_access *pa, const void *buf, size_t count); +size_t pkram_read(struct pkram_access *pa, void *buf, size_t count); + +#endif /* _LINUX_PKRAM_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 24c045b24b95..ea8242c91728 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -872,4 +872,13 @@ config MAPPING_DIRTY_HELPERS config KMAP_LOCAL bool +config PKRAM + bool "Preserved-over-kexec memory storage" + default n + help + This option adds the kernel API that enables saving memory pages of + the currently executing kernel and restoring them after a kexec in + the newly booted one. This can be utilized for speeding up reboot by + leaving process memory and/or FS caches in-place. + endmenu diff --git a/mm/Makefile b/mm/Makefile index 72227b24a616..ab3a724769b5 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -120,3 +120,4 @@ obj-$(CONFIG_MEMFD_CREATE) += memfd.o obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += mapping_dirty_helpers.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o +obj-$(CONFIG_PKRAM) += pkram.o diff --git a/mm/pkram.c b/mm/pkram.c new file mode 100644 index 000000000000..59e4661b2fb7 --- /dev/null +++ b/mm/pkram.c @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include + +/** + * Create a preserved memory node with name @name and initialize stream @ps + * for saving data to it. + * + * @gfp_mask specifies the memory allocation mask to be used when saving data. + * + * Returns 0 on success, -errno on failure. + * + * After the save has finished, pkram_finish_save() (or pkram_discard_save() in + * case of failure) is to be called. + */ +int pkram_prepare_save(struct pkram_stream *ps, const char *name, gfp_t gfp_mask) +{ + return -ENOSYS; +} + +/** + * Create a preserved memory object and initialize stream @ps for saving data + * to it. + * + * Returns 0 on success, -errno on failure. + * + * After the save has finished, pkram_finish_save_obj() (or pkram_discard_save() + * in case of failure) is to be called. + */ +int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) +{ + return -ENOSYS; +} + +/** + * Commit the object started with pkram_prepare_save_obj() to preserved memory. + */ +void pkram_finish_save_obj(struct pkram_stream *ps) +{ + BUG(); +} + +/** + * Commit the save to preserved memory started with pkram_prepare_save(). + * After the call, the stream may not be used any more. + */ +void pkram_finish_save(struct pkram_stream *ps) +{ + BUG(); +} + +/** + * Cancel the save to preserved memory started with pkram_prepare_save() and + * destroy the corresponding preserved memory node freeing any data already + * saved to it. + */ +void pkram_discard_save(struct pkram_stream *ps) +{ + BUG(); +} + +/** + * Remove the preserved memory node with name @name and initialize stream @ps + * for loading data from it. + * + * Returns 0 on success, -errno on failure. + * + * After the load has finished, pkram_finish_load() is to be called. + */ +int pkram_prepare_load(struct pkram_stream *ps, const char *name) +{ + return -ENOSYS; +} + +/** + * Remove the next preserved memory object from the stream @ps and + * initialize stream @ps for loading data from it. + * + * Returns 0 on success, -errno on failure. + * + * After the load has finished, pkram_finish_load_obj() is to be called. + */ +int pkram_prepare_load_obj(struct pkram_stream *ps) +{ + return -ENOSYS; +} + +/** + * Finish the load of a preserved memory object started with + * pkram_prepare_load_obj() freeing the object and any data that has not + * been loaded from it. + */ +void pkram_finish_load_obj(struct pkram_stream *ps) +{ + BUG(); +} + +/** + * Finish the load from preserved memory started with pkram_prepare_load() + * freeing the corresponding preserved memory node and any data that has + * not been loaded from it. + */ +void pkram_finish_load(struct pkram_stream *ps) +{ + BUG(); +} + +/** + * Finish the data access to or from the preserved memory node and object + * associated with pkram stream access @pa. The access must have been + * initialized with PKRAM_ACCESS(). + */ +void pkram_finish_access(struct pkram_access *pa, bool status_ok) +{ + BUG(); +} + +/** + * Save file page @page to the preserved memory node and object associated + * with pkram stream access @pa. The stream must have been initialized with + * pkram_prepare_save() and pkram_prepare_save_obj() and access initialized + * with PKRAM_ACCESS(). + * + * Returns 0 on success, -errno on failure. + */ +int pkram_save_file_page(struct pkram_access *pa, struct page *page) +{ + return -ENOSYS; +} + +/** + * Load the next page from the preserved memory node and object associated + * with pkram stream access @pa. The stream must have been initialized with + * pkram_prepare_load() and pkram_prepare_load_obj() and access initialized + * with PKRAM_ACCESS(). + * + * If not NULL, @index is initialized with the preserved mapping offset of the + * page loaded. + * + * Returns the page loaded or NULL if the node is empty. + * + * The page loaded has its refcount incremented. + */ +struct page *pkram_load_file_page(struct pkram_access *pa, unsigned long *index) +{ + return NULL; +} + +/** + * Copy @count bytes from @buf to the preserved memory node and object + * associated with pkram stream access @pa. The stream must have been + * initialized with pkram_prepare_save() and pkram_prepare_save_obj() + * and access initialized with PKRAM_ACCESS(); + * + * On success, returns the number of bytes written, which is always equal to + * @count. On failure, -errno is returned. + */ +ssize_t pkram_write(struct pkram_access *pa, const void *buf, size_t count) +{ + return -ENOSYS; +} + +/** + * Copy up to @count bytes from the preserved memory node and object + * associated with pkram stream access @pa to @buf. The stream must have been + * initialized with pkram_prepare_load() and pkram_prepare_load_obj() and + * access initialized PKRAM_ACCESS(). + * + * Returns the number of bytes read, which may be less than @count if the node + * has fewer bytes available. + */ +size_t pkram_read(struct pkram_access *pa, void *buf, size_t count) +{ + return 0; +} From patchwork Tue Mar 30 21:35:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0C57C433FC for ; Tue, 30 Mar 2021 21:28:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 56FBE6196C for ; Tue, 30 Mar 2021 21:28:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232842AbhC3V1e (ORCPT ); Tue, 30 Mar 2021 17:27:34 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50816 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232614AbhC3V05 (ORCPT ); Tue, 30 Mar 2021 17:26:57 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPCo4145310; Tue, 30 Mar 2021 21:25:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=Vf2+P//blblnbGasvOJNQrK5gTSuHjVLlcP/vpjWUaU=; b=fnSM/zFJTQ6Mmg6wdLZyg/cpTjVEaSaDUSpgU/W/FIOJL3Nmn6MiFPbbsIOJ6TUz14Ah jsoMMCipAMPI8FyhmHUWgH6GnzS4WvsaXqlIBcr0uo5txabLtUNZKVfoeuTCu6U265Ms loleFHVrlRY7/Znboo7T//Dz0Kkvl8cZQSOy0MwMiB6ghYVTMVgkY9VoXtqSsWd2BaAG F/LSWThaTBGyku2cqDHZMSMV9ZvuixiIHCO2CChEbEWL1zvN+Cp2xRcOA5q7h7Ftp6hu 0D6A5ZTt6obsbOpnVHuR8pF5H4aNnrgFAudYujR5YW358fWG5FGHJepybwtYRnF0kMWR MQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 37mad9r8dj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:12 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOnTX184142; Tue, 30 Mar 2021 21:25:11 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2173.outbound.protection.outlook.com [104.47.55.173]) by aserp3020.oracle.com with ESMTP id 37mac7u3tk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:11 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=abZeMY/d7PKe2eglcqbMO5q3GS6EFRsE59MjbCZZoQJnXB/B8QUX4YCiUpUQxooKNQUSlQtkaksD9+FXPCZy69yQHLoCoyMTUk9uMYGqGvXaw/Whj1+ahbfHnvJaelv4CEsHDNrnyHQ3o5/GVQogVOz/Wto9FvepkspFad3Q1GiH4wvO27qs6VqUrw9c//65NMvdZpJwy/r3UwsPmh5D3Id/AlbEgwzWi/O7sFX2TpUrDLBhF0ZM3+cM03CDOG+VJQybxrfawwajbCK8S5fjaAv6iEsx+eMK6SntMPD/YbARwyqFZVrNRsEv1hnltwqvHliQqllp4Sy9TnS89zbibQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Vf2+P//blblnbGasvOJNQrK5gTSuHjVLlcP/vpjWUaU=; b=a/M103EdwCNm6/+Xcc5Gr0AWP9FT62Iovw9J4hHuGZmc093O3CIxMoKNYG4NrNxVl1AvUA7TAcCd0a00nIaQiA935Syf+wUHox8lHZ4A+5CWiYqGvQFBNaRXT5hb7+BrwCEApKFPkNfBIZ0R6KmWMv8r3bRNt9aYULo5m12UWMGzGMBY5geYWCwbzohQQn5wzPeV1B56FG8AxKROs66HiKrQo/f/xpzF30lnlUK9BUUvx3BsEhOSXbvvZLnn+N6PNlf/Qp/JugJmKZ6SsSjDxxlowO4iZ+9Kt1EBhpZupwp32JPWL8PW7P1hjiATkrYg9ySruKbnjtc79y5O6C+tyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Vf2+P//blblnbGasvOJNQrK5gTSuHjVLlcP/vpjWUaU=; b=fdAUq0t43618EDfGIXEypCchQ8T6SYZ6lBAFQigpHF4+1YwQ3BTmjZ9gyXu+/P0cv6dRTldSe+POwyUfHix8TNdsxzbyW+5Ko0veGw64axHDmy51EtR2XFW4wOD3287wAYSGw36FiXgz5GcDRAVnTpXVaCDYpx+XqbYFf/RgB1o= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3120.namprd10.prod.outlook.com (2603:10b6:208:122::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:25:08 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:08 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 02/43] mm: PKRAM: implement node load and save functions Date: Tue, 30 Mar 2021 14:35:37 -0700 Message-Id: <1617140178-8773-3-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:04 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 04d6137b-6f76-42e7-02ef-08d8f3c24b1f X-MS-TrafficTypeDiagnostic: MN2PR10MB3120: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DYlF3OqMu5RtbmKD2rIG1RW68YQ+3Fc06VmpZBAiaeMxjsFFUkg2fMOcgdjHTzu8rB4IqUkx6nZcBVvpNJGDR82SiDnM5pNKn3ElOZoiZiA+iq+NMe4A3T1C5tHLXVg9aVQJA15dV9o4dmoLEX4pnTrcLjEnmhQ71/Gdjcbf3yjSZ8M2JorhGRHuGWfSm9ETl60G8EnIzRJkZ7POW9lhzldWcwXL+yASUGHh9e9Tm21tgcJ7jDkaKugYCdZ1WYv4KEM41caWFs/fPugyCeFC10N5p1e4sTZTqus+Frz7Ru7iSOYp9zomRdfM5+an/+uDUDALP3O2NIkF1dyLehrf7BXi6X13MsbJqFAt6jeopl/pIYp3YCAJKTGB7QyIuBMW4ldVT5nQrLu223uDLaEvA/MmoxDOm5/6CX95CmgkAcUpvVxTuOUfUCjEx7ltkA5v9arr2KQ5fsVQojU8wTU2yYwDRVUcFXynp6Q62M2tMAfljTbALHAP8R/sqFFtlLS+Lije7OH8cqGz4jiD0kND2PUBXJfIZ+XU1nwPocQ/fK6mbyp+Db85wE59t/6fZEPbmBDSSBfWBx7W34iq/iB8lkmYKC5cPLmu7jOPeMtYH/HRpXc0vIJjHr4KsRPQ3tvPJLKK2rMWD76xgMdF7qNmCbgPUXHtgn2v6Xzg5+N3fja2qGIdvaKVGxdWvKPNUCs/ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(366004)(376002)(346002)(396003)(52116002)(316002)(186003)(6486002)(16526019)(5660300002)(2616005)(956004)(6666004)(8676002)(66556008)(44832011)(478600001)(83380400001)(7406005)(38100700001)(86362001)(7696005)(66476007)(66946007)(4326008)(2906002)(26005)(7416002)(36756003)(8936002)(134885004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: zJxwS9HiYyHMdrKuRmK59Gd45ShubpZlA52qktc8NNGNcTgrKXOnTENM+6i/wccGKfRRYS148Kh6oRTahFCY+4aZaaaQRpCydQ1FWRJ8SZslTtv//+kRQ+iVnTiAt+7wOuoeywifwQ9c7/mIB7o9yWN8WZz7DLaGmjmkP31PJexp/ioFf4bMV2SDj4yX15QKtqbr/9PIa5h838YpsxlNN98X9zVfvht1ndMzFka2b/urUfB0d6K/Y6EtDerkTRRC3D2CMKqgoOcMcq5DCRKLovN3ItaWiM11EVboPMwomGDS9PeKVc88i1cMAj/cKUK1GayEedJOpyPOtKbRtRByjcfrskhOiPFBf6USpNVFBGiOarZOMg1Xu8tsHrd7DP8c96UjZKJMjNGWLbSZm1pDxC1ou90CvheXUnLs4gDfUdhsRrYmYWCZ3zl6Uh+qKYwIhEcWX2AQZHniYxBxq72/IiRtD01nfwAQmI8t2Ap6bFMlcst4LnUTyHDVw9HcidYnJDq4/HTLf0mOdbFoKN5mwyJJyOza/UIQXAcifGtcPWPNrx1wGCjxb5qaG8hUykobaPTApAUwJJs2ats4dqfbpaDhXMnnTG3bMNOAfXJNjMFxUnFEgCyniZaycMx0wkI2gA6c5n0r/ysldE7pm75CCXl/LsAlMGq3V097o3xwsP5NfHylx6HHcL8Pz6lHYDTJBuI1vr5s5fcOuUaZ1UkrrULEcw11e3aEVnG6UpgcOsx9zD6kp3LfGllx/Ygw5h8hHQ323zSOkZWgj87AqBRPNXF80L/A5dtl4XR6UyASMeR4O3Io74f/Cby0tPgyF7eXwiJgTW/vi2pzoZA0xpX2QUHWHsGl7xDTBQhc+ZWUtkJjDVfViROHiJybmmCoP1YkRiXr2w0J+PF0XAvCZA9dZc9mFf8eNFp7JY+UixdIBz8pAnm98JrQjV5RH+P7GaQNU3VJ+2XxEWos1uhYKM3kA+cWBgjQTWvqX6Ujc5Ke9phHvm7tdp3WQs6y4gZvNQPRUBDmkYfu1Qpt+TT9qXGxApW8mlR0YAOw9uaRb1yIPDzXezOOfeDgRDBcOouuhy9CgsYjia2C7j44NnfSF66wF8kyCZ3LKah3E3dauaX7/qqtfNPwjdNmiGSj2f2Vp+jDsCmbzN0Niqj4XI/FBpy5edXl/1boqCifVK2hdd1Ehmr+vMjVv0y7g14SZpvytCTdLPRWdiTA+1WBY9sDfgMylS5Y/aJHPK6MdqY0Gkl2ljiQCEUMtjZvZ0GNjTlQqrUpDDq7I8wnt4FF01HGm+fI1TnsR58FdWtQtD7zl6SMXMH2cuWtHzw2eevBFg6vYZQl X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 04d6137b-6f76-42e7-02ef-08d8f3c24b1f X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:08.4132 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uuz81d8cXglpzvaU+wVP5EFoY1bFVsLIyncjDlwr//SoCfn32dFk1QavSm0QTevT285AxzlDgNYRq9KVfComYSmlgZKJe6Tlg4Tde723mc8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3120 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: kE4H91EIaDeftWTmgNuk0X_566WalKes X-Proofpoint-GUID: kE4H91EIaDeftWTmgNuk0X_566WalKes X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Preserved memory is divided into nodes which can be saved and loaded independently of each other. PKRAM nodes are kept on a list and identified by unique names. Whenever a save operation is initiated by calling pkram_prepare_save(), a new node is created and linked to the list. When the save operation has been committed by calling pkram_finish_save(), the node becomes loadable. A load operation can be then initiated by calling pkram_prepare_load() which deletes the node from the list and prepares the corresponding stream for loading data from it. After the load has been finished, the pkram_finish_load() function must be called to free the node. Nodes are also deleted when a save operation is discarded, i.e. pkram_discard_save() is called instead of pkram_finish_save(). Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 8 ++- mm/pkram.c | 148 ++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 150 insertions(+), 6 deletions(-) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index a575da2d6c79..01055a876450 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -6,6 +6,8 @@ #include #include +struct pkram_node; + /** * enum pkram_data_flags - definition of data types contained in a pkram obj * @PKRAM_DATA_none: No data types configured @@ -14,7 +16,11 @@ enum pkram_data_flags { PKRAM_DATA_none = 0x0, /* No data types configured */ }; -struct pkram_stream; +struct pkram_stream { + gfp_t gfp_mask; + struct pkram_node *node; +}; + struct pkram_access; #define PKRAM_NAME_MAX 256 /* including nul */ diff --git a/mm/pkram.c b/mm/pkram.c index 59e4661b2fb7..21976df6e0ea 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -2,16 +2,85 @@ #include #include #include +#include #include +#include #include +#include #include +/* + * Preserved memory is divided into nodes that can be saved or loaded + * independently of each other. The nodes are identified by unique name + * strings. + * + * The structure occupies a memory page. + */ +struct pkram_node { + __u32 flags; + + __u8 name[PKRAM_NAME_MAX]; +}; + +#define PKRAM_SAVE 1 +#define PKRAM_LOAD 2 +#define PKRAM_ACCMODE_MASK 3 + +static LIST_HEAD(pkram_nodes); /* linked through page::lru */ +static DEFINE_MUTEX(pkram_mutex); /* serializes open/close */ + +static inline struct page *pkram_alloc_page(gfp_t gfp_mask) +{ + return alloc_page(gfp_mask); +} + +static inline void pkram_free_page(void *addr) +{ + free_page((unsigned long)addr); +} + +static inline void pkram_insert_node(struct pkram_node *node) +{ + list_add(&virt_to_page(node)->lru, &pkram_nodes); +} + +static inline void pkram_delete_node(struct pkram_node *node) +{ + list_del(&virt_to_page(node)->lru); +} + +static struct pkram_node *pkram_find_node(const char *name) +{ + struct page *page; + struct pkram_node *node; + + list_for_each_entry(page, &pkram_nodes, lru) { + node = page_address(page); + if (strcmp(node->name, name) == 0) + return node; + } + return NULL; +} + +static void pkram_stream_init(struct pkram_stream *ps, + struct pkram_node *node, gfp_t gfp_mask) +{ + memset(ps, 0, sizeof(*ps)); + ps->gfp_mask = gfp_mask; + ps->node = node; +} + /** * Create a preserved memory node with name @name and initialize stream @ps * for saving data to it. * * @gfp_mask specifies the memory allocation mask to be used when saving data. * + * Error values: + * %ENAMETOOLONG: name len >= PKRAM_NAME_MAX + * %ENOMEM: insufficient memory available + * %EEXIST: node with specified name already exists + * * Returns 0 on success, -errno on failure. * * After the save has finished, pkram_finish_save() (or pkram_discard_save() in @@ -19,7 +88,34 @@ */ int pkram_prepare_save(struct pkram_stream *ps, const char *name, gfp_t gfp_mask) { - return -ENOSYS; + struct page *page; + struct pkram_node *node; + int err = 0; + + if (strlen(name) >= PKRAM_NAME_MAX) + return -ENAMETOOLONG; + + page = pkram_alloc_page(gfp_mask | __GFP_ZERO); + if (!page) + return -ENOMEM; + node = page_address(page); + + node->flags = PKRAM_SAVE; + strcpy(node->name, name); + + mutex_lock(&pkram_mutex); + if (!pkram_find_node(name)) + pkram_insert_node(node); + else + err = -EEXIST; + mutex_unlock(&pkram_mutex); + if (err) { + pkram_free_page(node); + return err; + } + + pkram_stream_init(ps, node, gfp_mask); + return 0; } /** @@ -50,7 +146,12 @@ void pkram_finish_save_obj(struct pkram_stream *ps) */ void pkram_finish_save(struct pkram_stream *ps) { - BUG(); + struct pkram_node *node = ps->node; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); + + smp_wmb(); + node->flags &= ~PKRAM_ACCMODE_MASK; } /** @@ -60,7 +161,15 @@ void pkram_finish_save(struct pkram_stream *ps) */ void pkram_discard_save(struct pkram_stream *ps) { - BUG(); + struct pkram_node *node = ps->node; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); + + mutex_lock(&pkram_mutex); + pkram_delete_node(node); + mutex_unlock(&pkram_mutex); + + pkram_free_page(node); } /** @@ -69,11 +178,36 @@ void pkram_discard_save(struct pkram_stream *ps) * * Returns 0 on success, -errno on failure. * + * Error values: + * %ENOENT: node with specified name does not exist + * %EBUSY: save to required node has not finished yet + * * After the load has finished, pkram_finish_load() is to be called. */ int pkram_prepare_load(struct pkram_stream *ps, const char *name) { - return -ENOSYS; + struct pkram_node *node; + int err = 0; + + mutex_lock(&pkram_mutex); + node = pkram_find_node(name); + if (!node) { + err = -ENOENT; + goto out_unlock; + } + if (node->flags & PKRAM_ACCMODE_MASK) { + err = -EBUSY; + goto out_unlock; + } + pkram_delete_node(node); +out_unlock: + mutex_unlock(&pkram_mutex); + if (err) + return err; + + node->flags |= PKRAM_LOAD; + pkram_stream_init(ps, node, 0); + return 0; } /** @@ -106,7 +240,11 @@ void pkram_finish_load_obj(struct pkram_stream *ps) */ void pkram_finish_load(struct pkram_stream *ps) { - BUG(); + struct pkram_node *node = ps->node; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_LOAD); + + pkram_free_page(node); } /** From patchwork Tue Mar 30 21:35:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89A59C4345D for ; Tue, 30 Mar 2021 21:28:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5150061A0B for ; Tue, 30 Mar 2021 21:28:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232853AbhC3V1f (ORCPT ); Tue, 30 Mar 2021 17:27:35 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50814 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232611AbhC3V05 (ORCPT ); Tue, 30 Mar 2021 17:26:57 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULP9x2145305; Tue, 30 Mar 2021 21:25:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=anR3JuhOf2/kaCECJEGMHmimb81ITGwiAVS+xEPtF6M=; b=dOfVSRZLmdgwabn2pZONARFU7FGHUM1HEcW14B8nVrBZ0FJ3oyc1cZt9q2ogKAZPL5uS YS1r6BV5AWdO+5iA4ObYSQFu7OChbl4Ij+ihHU31j8LVGql83n7t66Y35qmBeSWvcKCI FC3T/DQX+AHL/Iq+H2zbbfgt0TJsliiZbalu8bh7uPbBvCS/xH49zxP6m5FL2A+IhhS3 EvZUbNfc0w190TwRTq+Y8FJYD3IxL1mq492QDSu94x7vPO3DVlrN7UDATxGEW86+fQQc Sh2YsyF7He7gcjKcp2ZgTkrclqmdOSVPhhvxZcGu1DiMjcdfzxJUdBiSq5NSv6I3+hls tQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 37mad9r8ds-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:17 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOblx149856; Tue, 30 Mar 2021 21:25:16 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2172.outbound.protection.outlook.com [104.47.55.172]) by userp3020.oracle.com with ESMTP id 37mac4kc6r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SMnBEyPyoyYn7HLnk2jOaisP6aZJs1SiOaKRl5iHlleYfruE9R/f6dc+WscDmaBoc7Dbvm1nXEmtZpHMNWaOx/yCyFL7i55jzXOwojf6dD4IY7PHI0X73l1OPruYsKCww+T5ji/S6b/xczpqCCEO32ZGyk/bd1w+rkEAQGTXeV4a1nulxcYbXh7Vug/m7ZBslljlepot1SPtlzewVmt4LUKK3p9x/YTnNzY8TGX8euHLezEUmPQB5XbwjBSPM8xlvmVyd5Spi5SZFR2JG4TeBPjRzDm2HPvY6g253rYQ2Omgf7K08BU4Jh38XDsgX+ckkIij7eDs4mPugCptDmTQ4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=anR3JuhOf2/kaCECJEGMHmimb81ITGwiAVS+xEPtF6M=; b=dRwnIRdaVvkWBMHlwi8lYHpZxVJ0a0V7ial8IYPmWzOEtUsFToFEEAOBnBDbAAeEL6oy3ypnHha9Y7GE/Hu6RbGRG7awXI3se+a6W5v1UfBCVpTWv6srkddXooV5fa1SYzLpSuHA4omAivcU/wT5JJYimqHg27sBJWGlwxJ9kuEytZIwJvdRB124qknE5cSB6P0UBcMl+lXB5BvHNbFW0T2olWcDvYYzpOpaQUmW4SwInUT1QeK2ZK41glJvymEfTNXkIwvqoQqkorj4XETQPpPid49UQKQV6CU6z+10O1Bhs0aSiSNoyUFNeu9Siwn/I3ZIFX659cWDifjSyYM+Mw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=anR3JuhOf2/kaCECJEGMHmimb81ITGwiAVS+xEPtF6M=; b=RKYdKXCKO/teL6ersbZb/sgruguN1kC6tUVSlkKzFnqt39lZMYUx7thnpBmYuNO724hUtEA2Rl9h8spIwA+mL34yP/luscSwKPUpi5+HrqB8Vp79ovN0bNAjgcpxNwgg7dLoIo8GgsPpRnkuMDydStcmDczOltm6Gh2IYlqasyU= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3120.namprd10.prod.outlook.com (2603:10b6:208:122::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:25:12 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:12 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 03/43] mm: PKRAM: implement object load and save functions Date: Tue, 30 Mar 2021 14:35:38 -0700 Message-Id: <1617140178-8773-4-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 84624cf4-e76b-473d-946c-08d8f3c24db2 X-MS-TrafficTypeDiagnostic: MN2PR10MB3120: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qK1u7bhbLzniP866Jt/ifhzhVfWcUUDu8KAefS7+eYVjmZSILyCnhJKjsxkS+nb/X0fLrLBR7akgpjtSYwb1bIgwrrDcpUa9won4DJnmfZGz4tHbdsyuxOAnjo4EmT7sUhtScVS3Z4ZM/FdFjGRrPIeQ4mnSWt5gDxbCCprkfW/AdFp8sqyhM9Mv0CiZfslG5iIpplJiyc/PBgiCsuLBFUdTJ82728l0uMggFqNmWGC9di2WN8VeTCqTx5t3p4HQ7ru6WGah4wXbCX4C76GFECDtskLo2mF5EatceBkJEdgaFGiIfuYapMHSXmsZ0glipCzNPEQrBC7F146nMQQAMU4fToAPDvKxLxPKtNSCZaVPnMi0nQiCYWFGUhFQTM+oy4UYCZRACDqA+5QOomD+5/Zkz0rh/3bVLQQ5OIFniSOELmJN7KhrS2/p1OvKy/JoB2jH9bynCSXKACbjjzk5QSSb5eDQOtEhtaDp5L0qIav0gDpPLe9yuFsJC+XGQ74JlSTME5UIgw54K1H8dZxEieO+H2N4hgJharYhQCGrHrk94HAHeCCgMo404yMwfiPDSbladsA6V4dNJc6ZuL4YijU+1o3SCZZ7H9pn7oWpr+W5FtA7tLYkLBPwUIJ/ia4JNO/IuwoCY5wSmM/8V68wrbXndmBij73K09AlCJD2LKAS8HMEckoONhi/i8WXVPyi X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(366004)(376002)(346002)(396003)(52116002)(316002)(186003)(6486002)(16526019)(5660300002)(2616005)(956004)(6666004)(8676002)(66556008)(44832011)(478600001)(83380400001)(7406005)(38100700001)(86362001)(7696005)(66476007)(66946007)(4326008)(2906002)(26005)(7416002)(36756003)(8936002)(134885004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: mUdEfLqk1UcJ4n+VNMunRH1K6bg4BCk4dI1J0JJKK3IBdziwcVE0yu3Tu51Fb1CWgeM4mnD4xSMemxXhEfMqkx1bj5QzlumgWvJ5XtwDXU9zomN7WIDWIhkwN6A/xnIXtL9TNXU73C3RsAD33sm80sSMSxmQnA+M07h4NI6nRVlDcg/yI2iCWAVs68t98qhn6OJL616qOxXyG2iMZ5rDE8HbGW43qQEB0tZJy/CsoCNQ/ZRyHJ+cQ6oDekt7VISQRxOTnjPeFZNfE2HDwJHtl00+UFSKkn0y7aaFOVlWFKP1ggLBGQFAgrTCtCa8f8YbvZSc0atLlL7Ldzmcu4w9nK4xVSzkmjte7gzCx6rIHXRtdES0WKSKT7QMAwdQl1BbF6F8amiq/web+O08867LxSoUanXqP2debfgWVfl9APwT4vx8k2aUPulplxd5ArzesiYth875K21NueHlVolZzaaTUmJ11CUoUV7EwFYMAXWPw9yl7F/1yZeTd/FpzMmEs3aLci9Bhkb16FNAvD+fVB2Y2V5EKe2C0LsEAA5mmi7RscKE6tFCHYfyr2RUI/iu4eDO8PviiIxPT5NM3cG2nbITW1VFKyArdZiCNmw7d8TlkrgLAZBexmcJ/LG7XWTcNo9yWYPxQx2FncelF+C21USe4dR5/8Xr1oqQR5j0HGgMWKP4+j1Xm/iLIKiQJCuk8t0MpltCR4ZdtZqFKXDXC38uYFKFddJ2wI1o9Rw6wyoFP6iOiGkYKczUSb+2J+SpywV07QMrE63pOOYsgyd1X7J8KNVI7D3QSQNvgynxyRi/8PPIaSiJgtaEgJQY04kYgCTEExQdRs0XB1MH9ybeG8PpoodzhaegPRd4Iyq6nt0sg3OmAp88+vEOrrW4tO00VC7K9Ps+HB2t2t/eu+r57mt9xI8421rLg0xQ+cm15N9kXW9MoOxupJEboEGCxSwbj54kAb8aZpUAkEkh+r67H8+mUtFCHzbRXav8RVbJIkDionmnYALyobHlruJR4A8K/nrb76Ns13Md3imS4r9sFZG6vbRxKMJf0F8Le6hPCxo5MhI7a9De8RYlJgVeH7Sqn/rTs4xgD7CBKGVFXBUDfJRSqcNsromD8OvOVwv8UIAHLeT39nRUORPWrF/PBVOtK4Q06CiH1bgF8nA++qInQciZg8403w6D1lxefQ0Wpc5PaUd2yTJkEZI2e7ro7tMpC04MTkkB9Xp2SqyTrkeys2hIdUS0lchtqF/tXFsLaE01OxMW0lqaLVT7H6JQxn3tPQWBTS3VVACtxgCpbDRPESGNa99R5xsq7+grAjLF1ctMRdFHqRg7o2+yjaUBuaEH X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 84624cf4-e76b-473d-946c-08d8f3c24db2 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:12.7388 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QKXDpl8Almwx6/03PHwtomxWUmCVmgG5Aqx0OVLHudtK2mtwU+DsD67OupkHQ8TSKzsfcnXfbyJduY5BWQak5bLoIeIvTKF0+6oQOCq+fuI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3120 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: UfBCMuY7yXcBH0vh78GwREoMME8RVcAi X-Proofpoint-GUID: UfBCMuY7yXcBH0vh78GwREoMME8RVcAi X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org PKRAM nodes are further divided into a list of objects. After a save operation has been initiated for a node, a save operation for an object associated with the node is initiated by calling pkram_prepare_save_obj(). A new object is created and linked to the node. The save operation for the object is committed by calling pkram_finish_save_obj(). After a load operation has been initiated, pkram_prepare_load_obj() is called to delete the next object from the node and prepare the corresponding stream for loading data from it. After the load of object has been finished, pkram_finish_load_obj() is called to free the object. Objects are also deleted when a save operation is discarded. Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 2 ++ mm/pkram.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 70 insertions(+), 4 deletions(-) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index 01055a876450..a4d55af392c0 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -7,6 +7,7 @@ #include struct pkram_node; +struct pkram_obj; /** * enum pkram_data_flags - definition of data types contained in a pkram obj @@ -19,6 +20,7 @@ enum pkram_data_flags { struct pkram_stream { gfp_t gfp_mask; struct pkram_node *node; + struct pkram_obj *obj; }; struct pkram_access; diff --git a/mm/pkram.c b/mm/pkram.c index 21976df6e0ea..7c977c5982f8 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -6,9 +6,14 @@ #include #include #include +#include #include #include +struct pkram_obj { + __u64 obj_pfn; /* points to the next object in the list */ +}; + /* * Preserved memory is divided into nodes that can be saved or loaded * independently of each other. The nodes are identified by unique name @@ -18,6 +23,7 @@ */ struct pkram_node { __u32 flags; + __u64 obj_pfn; /* points to the first obj of the node */ __u8 name[PKRAM_NAME_MAX]; }; @@ -62,6 +68,21 @@ static struct pkram_node *pkram_find_node(const char *name) return NULL; } +static void pkram_truncate_node(struct pkram_node *node) +{ + unsigned long obj_pfn; + struct pkram_obj *obj; + + obj_pfn = node->obj_pfn; + while (obj_pfn) { + obj = pfn_to_kaddr(obj_pfn); + obj_pfn = obj->obj_pfn; + pkram_free_page(obj); + cond_resched(); + } + node->obj_pfn = 0; +} + static void pkram_stream_init(struct pkram_stream *ps, struct pkram_node *node, gfp_t gfp_mask) { @@ -124,12 +145,31 @@ int pkram_prepare_save(struct pkram_stream *ps, const char *name, gfp_t gfp_mask * * Returns 0 on success, -errno on failure. * + * Error values: + * %ENOMEM: insufficient memory available + * * After the save has finished, pkram_finish_save_obj() (or pkram_discard_save() * in case of failure) is to be called. */ int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) { - return -ENOSYS; + struct pkram_node *node = ps->node; + struct pkram_obj *obj; + struct page *page; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); + + page = pkram_alloc_page(ps->gfp_mask | __GFP_ZERO); + if (!page) + return -ENOMEM; + obj = page_address(page); + + if (node->obj_pfn) + obj->obj_pfn = node->obj_pfn; + node->obj_pfn = page_to_pfn(page); + + ps->obj = obj; + return 0; } /** @@ -137,7 +177,9 @@ int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) */ void pkram_finish_save_obj(struct pkram_stream *ps) { - BUG(); + struct pkram_node *node = ps->node; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); } /** @@ -169,6 +211,7 @@ void pkram_discard_save(struct pkram_stream *ps) pkram_delete_node(node); mutex_unlock(&pkram_mutex); + pkram_truncate_node(node); pkram_free_page(node); } @@ -216,11 +259,26 @@ int pkram_prepare_load(struct pkram_stream *ps, const char *name) * * Returns 0 on success, -errno on failure. * + * Error values: + * %ENODATA: Stream @ps has no preserved memory objects + * * After the load has finished, pkram_finish_load_obj() is to be called. */ int pkram_prepare_load_obj(struct pkram_stream *ps) { - return -ENOSYS; + struct pkram_node *node = ps->node; + struct pkram_obj *obj; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_LOAD); + + if (!node->obj_pfn) + return -ENODATA; + + obj = pfn_to_kaddr(node->obj_pfn); + node->obj_pfn = obj->obj_pfn; + + ps->obj = obj; + return 0; } /** @@ -230,7 +288,12 @@ int pkram_prepare_load_obj(struct pkram_stream *ps) */ void pkram_finish_load_obj(struct pkram_stream *ps) { - BUG(); + struct pkram_node *node = ps->node; + struct pkram_obj *obj = ps->obj; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_LOAD); + + pkram_free_page(obj); } /** @@ -244,6 +307,7 @@ void pkram_finish_load(struct pkram_stream *ps) BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_LOAD); + pkram_truncate_node(node); pkram_free_page(node); } From patchwork Tue Mar 30 21:35:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43496C433E4 for ; Tue, 30 Mar 2021 21:28:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1CFEA61A03 for ; Tue, 30 Mar 2021 21:28:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232789AbhC3V1Z (ORCPT ); Tue, 30 Mar 2021 17:27:25 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:60782 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232495AbhC3V0y (ORCPT ); Tue, 30 Mar 2021 17:26:54 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOe4T122849; Tue, 30 Mar 2021 21:25:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=AxqWT1rxHcU/XXNYUpQ7s9IYct8dM1BqesJ/k3PzW/g=; b=QHl2GQI+nsi42x0JX4AArt0mZ31DluWQjYcxLtAxr0nvyqzrml9ED0+tIjwjFDB9h75a Dj1yUJbSX18jwYej+OEZGK+6QZeCTeB8cf0DZCbEYqiEOBXD4Scmw81MxcH5dAtHgXmo 31SnuozpGKs4zZ9GYDFDaXotayQM5CNShwr1AySc/dpe2FwV7JtQutw5ypXE7ZvW+cDh 6FQQ8Ymab5hEOLS4GmKB2xFhKpE3EIKgRVYIwemroe26bw3DDgzrsBse+8+GRovP3WRv Knx3cxsYLbzdlWJGdKpaURxF/clWeczzdoKTltI6SmK496LXWyZhth6P9/RLKSmme3cl 0w== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2130.oracle.com with ESMTP id 37mafv081b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:20 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOmgq183961; Tue, 30 Mar 2021 21:25:20 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2176.outbound.protection.outlook.com [104.47.58.176]) by aserp3020.oracle.com with ESMTP id 37mac7u3x1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:20 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iDJ8jGpN3LFyWzgJuBsQ261ZrdAU3q28tPL7yZ2RFrOeDDzpwPRfAw1pfvo7Xwd0g1GxpEdaGJqpOtjeTVfHr7pNZHy3MXd8r1DblH6ORadpAcvZON9MGKMrSSQPDsJCQ+hbSQip1qXGNVfc2sRbbRfGJO1VZAa0dvitpDKIt2/CPiQ1xILdZmrUmoj1MIvtQcyW3CfiSx5rjOXHQROiPLSoFjvcjchQlgwUx9tk0r8f4GLY4zL8tXfCSdW3/dEyHRGqZmDjEMu6v9c5kk2ekZgxWiXtj22FUcYjC4U97U9Og/uggr9pHL4fN2skMuB22aVJmxyQfG3ov65Zy6Lf1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AxqWT1rxHcU/XXNYUpQ7s9IYct8dM1BqesJ/k3PzW/g=; b=MuthlqeaVTRJ24s5/RnIqWu3JbjSyseai2goRcFBp0hUYvEF0HE5AfNPCIyqRxPe2Jynid/kKaRsui9/HSzIPAqc+ODcfVm0t7giBnOgN8QJ1s0U50H4FnEhRa8uiadZQlJDYUhJ5p+Zpyeob7+IJUiq4Zf82nV7r+MDExfRyyRIezZWylCc7mLDrfjnEM5gnMjzhYz91pnCPagMiIt+Ke6Y91G2FvyN1+00W0gvL977mBTc4Tssbc7J5MbQDqhzacVZLv6I0viMEPBVwuJQ3kpwC16nbgwtrCwVBPWoSAcYNs81wFb4aquV1BMlbLMv3U4rKbZIY2eNZkpXKaaG0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AxqWT1rxHcU/XXNYUpQ7s9IYct8dM1BqesJ/k3PzW/g=; b=GkmZFhzQ6of8T3OoYOJzKG9pqNl0A02WKCHAKyr0EjqM30Vz5BNlbA0sO/xig8cE4xQMJa8YvVfhvPMVCo1ul/x0rJTqg8vEa9JxGpZ36ibk6Uwp3xcJPd6gc/hbLwforgFSiPN16xtEgFXHBddLJaPluOOxOASKhaJYh1vzalQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3120.namprd10.prod.outlook.com (2603:10b6:208:122::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:25:17 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:17 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 04/43] mm: PKRAM: implement page stream operations Date: Tue, 30 Mar 2021 14:35:39 -0700 Message-Id: <1617140178-8773-5-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:12 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b155224a-f0c4-4133-dc48-08d8f3c25043 X-MS-TrafficTypeDiagnostic: MN2PR10MB3120: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xqyr0Ig/r1lLnhDSIHtcP97++NlP50F5v7uv9dH/ziMveZ4ZQp8kILQ22T6UbgrwTWlhzF7mxg6EqXDeZf7JxgUi1Uq/skUChd0b55I1ZMG3Ytn+rUmr/GX+M6RwOCrX7lDtTj5J16yj0ybt8Z0plcZpEt6h0spSgNQ1NlMyPY+axDMaiZZC0rrWe1UsSzSsS1FteUCjl9+tZHX/j7AaZWIxB0DitvCg8kxkwCqfoYCW5of7leZVlHcGKndT1GggK6UtN9g4VPItJLNEH2iIiKVm9BacP50fX9rpbL9PACDs+PjZFTfZyAh8graT+zn5LpesFwlXpv6+NgCwohFi0MtVDxtfpnId+nefotYNeRg+intc+uiviOdN+efPW6xSZKW3zbEHtpnK91eqTJY2DN2CoQO4kHVlFXgOb9htS0FC/tvMgpSc4uKvN3Qv/Jm88Q1psDRJYGCAsm6q+AsqI/yDYster/4QmWYfcLYBDJs79nqf+C0MXeWaK6q2F62EXd2Hy22XaQESIYExTSzmNDd3MmSkKqvJoK2nPde7zN3mPw8QdDhO16h1VsBQMojPV7+b2bQcDAXx32dc8FQxIaC/Zt+JBqfks3UoyJMpAXd10qOyg+9kCbiljW1YHbJxEJzFFgme1eAW6rjXtz0Pl3yMMAAc8FsKabVhk4uJlAQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(366004)(376002)(346002)(396003)(52116002)(316002)(186003)(6486002)(16526019)(5660300002)(2616005)(956004)(6666004)(8676002)(66556008)(44832011)(478600001)(83380400001)(7406005)(38100700001)(86362001)(7696005)(30864003)(66476007)(66946007)(4326008)(2906002)(26005)(7416002)(36756003)(8936002)(21314003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: j4lMNZD9ALExZXGOG7DTi/OQ8kKcQZozG3Ki1JnjQClGVqL996hkLlQEi94PpIXR4vajHKVWLwXcoIaw49OT0bCeOoLmLwfXw9M2xlbruyelJoLR7tS9FPOTlPVQm7PTeKB4HclvWvyfXOUxfNTLzi7mB06W0DSJHfi8dM1682cmI4HoRAcelt7YDMOo4KcHSKqUdCp6q7OEQb/XMYHZoqHSg5BNNw+2sbsMZpVbmVojHR00cijpo7pZFQGztK88MmROchsH1R+wFnMF4MB/RrYiFGwW1Wxg5i2CFnxnfPigCk4Hni/R0izajhSu+qiONPtmNiud0z+UHUEBsJ71b2ktSIGNpMXz3s/mFQGTcWvmIDG8PkuVG/ekOw1MXhyk2ROQuZLnfKKBY0zEGPOkklSlTFPPBxGEJvlU15IyFMTpZzZHzoHaNeZcQKQL9Wroc/nhhENpiJS5rhQjw+46d+xCHB/zXXn/r+pfHMo2c9pbHaIjjv/hAkZtz8wB9uqUekJokOI0Z2IoHLrsir1Zwopy54hX7Ky6fQnSJCrfW+vruX9U2Q3CYZ2lkzRFkOEuWdLgVk0GYu+h9iBQEwVAhtbAHprI72hkxbJ3Rt1okC3uPevEpcK2D8bC7DznZ0QXAI9vG9tuSGW8QUzoR+xJzY+Rm9mH+wHgGDopnI6YlrneBwiXLFp0pmMa3BycpFAFCsC4k7W0mUaVx3PIXy+xZW0HcB2qvX+kv/WGKw2goE8wufowNQKu8JN13YKUIGmdAe0jZkRttbzduEfxSGuAR4QG71gb+LQeXnbatjhGIi7+eYMJK6m/alyMtQsNtYbiKwAKHnVoQSwi4fjRHiLFIcVash/8OwihapUmVbSnOshfE+NVhLIKfpUXVGCvv+j0KmBBqYlnOtGEbK4JriQDS29s1wwLgDpil5UPwglVrXBVSSkuj6PecMMp6ecqJlswwLHbRjqHxoR1/GPwKQz3oXgLzJjDkY3eckD7/qIo0mkYLmGmCQWbstiKc7u6wr4//sBIFy0TYNVlDjY6Z8076kITpKg2+ObwNCTnBocjtOyDSJO5AinDN/cpjKvPQE/yZpc85OfbbFBcqobePu7BzvE1jPPJ67eHzRDCDxukdF9KJDE1bfb6tsNgVQLFIWbrtUmQ1xdU1uPrbjh+hj6deJAlfGvjA0sVl6D81nhyyZHYatxXi/6JSKUkKe1lPZ8CMvFIVnCi6YVhQwD68RlTPxiRHNdoYPN23160Y1bR5vQdE8SQ0E2xSwAsphQgqrPPwgrwaMzwB6jx/UhpvVkSCUUEcHNna145Xq+epUuylgafeT7XmY2VkybwsauJl6KW X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b155224a-f0c4-4133-dc48-08d8f3c25043 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:17.0253 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ipqDJOMz3CCh+vJADVSqBkJfGtmYS8LQXUs0HOVIxGPsZOhJF4XLHu7fJ1fXQ2oGE4BzPG0i60Frv0azVYShnewXNhurh8g9wMi1CMOXNRk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3120 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: W7j8EXabPEzyqVZ-WpFAhKeiVGDRRETF X-Proofpoint-GUID: W7j8EXabPEzyqVZ-WpFAhKeiVGDRRETF X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Using the pkram_save_file_page() function, one can populate PKRAM objects with in-memory pages which can later be loaded using the pkram_load_file_page() function. Saving a page to PKRAM is accomplished by recording its pfn and mapping index and incrementing its refcount so that it will not be freed after the last user puts it. Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 42 +++++++- mm/pkram.c | 282 +++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 317 insertions(+), 7 deletions(-) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index a4d55af392c0..9d8a6fd96dd9 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -8,22 +8,47 @@ struct pkram_node; struct pkram_obj; +struct pkram_link; /** * enum pkram_data_flags - definition of data types contained in a pkram obj * @PKRAM_DATA_none: No data types configured + * @PKRAM_DATA_pages: obj contains file page data */ enum pkram_data_flags { - PKRAM_DATA_none = 0x0, /* No data types configured */ + PKRAM_DATA_none = 0x0, /* No data types configured */ + PKRAM_DATA_pages = 0x1, /* Contains file page data */ +}; + +struct pkram_data_stream { + /* List of link pages to add/remove from */ + __u64 *head_link_pfnp; + __u64 *tail_link_pfnp; + + struct pkram_link *link; /* current link */ + unsigned int entry_idx; /* next entry in link */ }; struct pkram_stream { gfp_t gfp_mask; struct pkram_node *node; struct pkram_obj *obj; + + __u64 *pages_head_link_pfnp; + __u64 *pages_tail_link_pfnp; +}; + +struct pkram_pages_access { + unsigned long next_index; }; -struct pkram_access; +struct pkram_access { + enum pkram_data_flags dtype; + struct pkram_stream *ps; + struct pkram_data_stream pds; + + struct pkram_pages_access pages; +}; #define PKRAM_NAME_MAX 256 /* including nul */ @@ -41,8 +66,19 @@ int pkram_prepare_save(struct pkram_stream *ps, const char *name, void pkram_finish_load(struct pkram_stream *ps); void pkram_finish_load_obj(struct pkram_stream *ps); +#define PKRAM_PDS_INIT(name, stream, type) { \ + .head_link_pfnp=(stream)->type##_head_link_pfnp, \ + .tail_link_pfnp=(stream)->type##_tail_link_pfnp, \ + } + +#define PKRAM_ACCESS_INIT(name, stream, type) { \ + .dtype = PKRAM_DATA_##type, \ + .ps = (stream), \ + .pds = PKRAM_PDS_INIT(name, stream, type), \ + } + #define PKRAM_ACCESS(name, stream, type) \ - struct pkram_access name + struct pkram_access name = PKRAM_ACCESS_INIT(name, stream, type) void pkram_finish_access(struct pkram_access *pa, bool status_ok); diff --git a/mm/pkram.c b/mm/pkram.c index 7c977c5982f8..9c42db66d022 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #include #include #include @@ -10,8 +11,39 @@ #include #include +#include "internal.h" + + +/* + * Represents a reference to a data page saved to PKRAM. + */ +typedef __u64 pkram_entry_t; + +#define PKRAM_ENTRY_FLAGS_SHIFT 0x5 +#define PKRAM_ENTRY_FLAGS_MASK 0x7f + +/* + * Keeps references to data pages saved to PKRAM. + * The structure occupies a memory page. + */ +struct pkram_link { + __u64 link_pfn; /* points to the next link of the object */ + __u64 index; /* mapping index of first pkram_entry_t */ + + /* + * the array occupies the rest of the link page; if the link is not + * full, the rest of the array must be filled with zeros + */ + pkram_entry_t entry[0]; +}; + +#define PKRAM_LINK_ENTRIES_MAX \ + ((PAGE_SIZE-sizeof(struct pkram_link))/sizeof(pkram_entry_t)) + struct pkram_obj { - __u64 obj_pfn; /* points to the next object in the list */ + __u64 pages_head_link_pfn; /* the first pages link of the object */ + __u64 pages_tail_link_pfn; /* the last pages link of the object */ + __u64 obj_pfn; /* points to the next object in the list */ }; /* @@ -19,6 +51,10 @@ struct pkram_obj { * independently of each other. The nodes are identified by unique name * strings. * + * References to data pages saved to a preserved memory node are kept in a + * singly-linked list of PKRAM link structures (see above), the node has a + * pointer to the head of. + * * The structure occupies a memory page. */ struct pkram_node { @@ -68,6 +104,41 @@ static struct pkram_node *pkram_find_node(const char *name) return NULL; } +static void pkram_truncate_link(struct pkram_link *link) +{ + struct page *page; + pkram_entry_t p; + int i; + + for (i = 0; i < PKRAM_LINK_ENTRIES_MAX; i++) { + p = link->entry[i]; + if (!p) + continue; + page = pfn_to_page(PHYS_PFN(p)); + put_page(page); + } +} + +static void pkram_truncate_links(unsigned long link_pfn) +{ + struct pkram_link *link; + + while (link_pfn) { + link = pfn_to_kaddr(link_pfn); + pkram_truncate_link(link); + link_pfn = link->link_pfn; + pkram_free_page(link); + cond_resched(); + } +} + +static void pkram_truncate_obj(struct pkram_obj *obj) +{ + pkram_truncate_links(obj->pages_head_link_pfn); + obj->pages_head_link_pfn = 0; + obj->pages_tail_link_pfn = 0; +} + static void pkram_truncate_node(struct pkram_node *node) { unsigned long obj_pfn; @@ -76,6 +147,7 @@ static void pkram_truncate_node(struct pkram_node *node) obj_pfn = node->obj_pfn; while (obj_pfn) { obj = pfn_to_kaddr(obj_pfn); + pkram_truncate_obj(obj); obj_pfn = obj->obj_pfn; pkram_free_page(obj); cond_resched(); @@ -83,6 +155,83 @@ static void pkram_truncate_node(struct pkram_node *node) node->obj_pfn = 0; } +static void pkram_add_link(struct pkram_link *link, struct pkram_data_stream *pds) +{ + __u64 link_pfn = page_to_pfn(virt_to_page(link)); + + if (!*pds->head_link_pfnp) { + *pds->head_link_pfnp = link_pfn; + *pds->tail_link_pfnp = link_pfn; + } else { + struct pkram_link *tail = pfn_to_kaddr(*pds->tail_link_pfnp); + + tail->link_pfn = link_pfn; + *pds->tail_link_pfnp = link_pfn; + } +} + +static struct pkram_link *pkram_remove_link(struct pkram_data_stream *pds) +{ + struct pkram_link *link; + + if (!*pds->head_link_pfnp) + return NULL; + + link = pfn_to_kaddr(*pds->head_link_pfnp); + *pds->head_link_pfnp = link->link_pfn; + if (!*pds->head_link_pfnp) + *pds->tail_link_pfnp = 0; + else + link->link_pfn = 0; + + return link; +} + +static struct pkram_link *pkram_new_link(struct pkram_data_stream *pds, gfp_t gfp_mask) +{ + struct pkram_link *link; + struct page *link_page; + + link_page = pkram_alloc_page((gfp_mask & GFP_RECLAIM_MASK) | + __GFP_ZERO); + if (!link_page) + return NULL; + + link = page_address(link_page); + pkram_add_link(link, pds); + pds->link = link; + pds->entry_idx = 0; + + return link; +} + +static void pkram_add_link_entry(struct pkram_data_stream *pds, struct page *page) +{ + struct pkram_link *link = pds->link; + pkram_entry_t p; + short flags = 0; + + p = page_to_phys(page); + p |= ((flags & PKRAM_ENTRY_FLAGS_MASK) << PKRAM_ENTRY_FLAGS_SHIFT); + link->entry[pds->entry_idx] = p; + pds->entry_idx++; +} + +static int pkram_next_link(struct pkram_data_stream *pds, struct pkram_link **linkp) +{ + struct pkram_link *link; + + link = pkram_remove_link(pds); + if (!link) + return -ENODATA; + + pds->link = link; + pds->entry_idx = 0; + *linkp = link; + + return 0; +} + static void pkram_stream_init(struct pkram_stream *ps, struct pkram_node *node, gfp_t gfp_mask) { @@ -159,6 +308,9 @@ int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); + if (flags & ~PKRAM_DATA_pages) + return -EINVAL; + page = pkram_alloc_page(ps->gfp_mask | __GFP_ZERO); if (!page) return -ENOMEM; @@ -168,6 +320,10 @@ int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) obj->obj_pfn = node->obj_pfn; node->obj_pfn = page_to_pfn(page); + if (flags & PKRAM_DATA_pages) { + ps->pages_head_link_pfnp = &obj->pages_head_link_pfn; + ps->pages_tail_link_pfnp = &obj->pages_tail_link_pfn; + } ps->obj = obj; return 0; } @@ -275,8 +431,17 @@ int pkram_prepare_load_obj(struct pkram_stream *ps) return -ENODATA; obj = pfn_to_kaddr(node->obj_pfn); + if (!obj->pages_head_link_pfn) { + WARN_ON(1); + return -EINVAL; + } + node->obj_pfn = obj->obj_pfn; + if (obj->pages_head_link_pfn) { + ps->pages_head_link_pfnp = &obj->pages_head_link_pfn; + ps->pages_tail_link_pfnp = &obj->pages_tail_link_pfn; + } ps->obj = obj; return 0; } @@ -293,6 +458,7 @@ void pkram_finish_load_obj(struct pkram_stream *ps) BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_LOAD); + pkram_truncate_obj(obj); pkram_free_page(obj); } @@ -318,7 +484,41 @@ void pkram_finish_load(struct pkram_stream *ps) */ void pkram_finish_access(struct pkram_access *pa, bool status_ok) { - BUG(); + if (status_ok) + return; + + if (pa->ps->node->flags == PKRAM_SAVE) + return; + + if (pa->pds.link) + pkram_truncate_link(pa->pds.link); +} + +/* + * Add file page to a PKRAM obj allocating a new PKRAM link if necessary. + */ +static int __pkram_save_page(struct pkram_access *pa, struct page *page, + unsigned long index) +{ + struct pkram_data_stream *pds = &pa->pds; + struct pkram_link *link = pds->link; + + if (!link || pds->entry_idx >= PKRAM_LINK_ENTRIES_MAX || + index != pa->pages.next_index) { + link = pkram_new_link(pds, pa->ps->gfp_mask); + if (!link) + return -ENOMEM; + + pa->pages.next_index = link->index = index; + } + + get_page(page); + + pkram_add_link_entry(pds, page); + + pa->pages.next_index++; + + return 0; } /** @@ -328,10 +528,80 @@ void pkram_finish_access(struct pkram_access *pa, bool status_ok) * with PKRAM_ACCESS(). * * Returns 0 on success, -errno on failure. + * + * Error values: + * %ENOMEM: insufficient amount of memory available + * + * Saving a page to preserved memory is simply incrementing its refcount so + * that it will not get freed after the last user puts it. That means it is + * safe to use the page as usual after it has been saved. */ int pkram_save_file_page(struct pkram_access *pa, struct page *page) { - return -ENOSYS; + struct pkram_node *node = pa->ps->node; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); + + BUG_ON(PageCompound(page)); + + return __pkram_save_page(pa, page, page->index); +} + +static struct page *__pkram_prep_load_page(pkram_entry_t p) +{ + struct page *page; + short flags; + + flags = (p >> PKRAM_ENTRY_FLAGS_SHIFT) & PKRAM_ENTRY_FLAGS_MASK; + page = pfn_to_page(PHYS_PFN(p)); + + return page; +} + +/* + * Extract the next page from preserved memory freeing a PKRAM link if it + * becomes empty. + */ +static struct page *__pkram_load_page(struct pkram_access *pa, unsigned long *index) +{ + struct pkram_data_stream *pds = &pa->pds; + struct pkram_link *link = pds->link; + struct page *page; + pkram_entry_t p; + int ret; + + if (!link) { + ret = pkram_next_link(pds, &link); + if (ret) + return NULL; // XXX return error value? + + if (index) + pa->pages.next_index = link->index; + } + + BUG_ON(pds->entry_idx >= PKRAM_LINK_ENTRIES_MAX); + + p = link->entry[pds->entry_idx]; + BUG_ON(!p); + + page = __pkram_prep_load_page(p); + + if (index) { + *index = pa->pages.next_index; + pa->pages.next_index++; + } + + /* clear to avoid double free (see pkram_truncate_link()) */ + link->entry[pds->entry_idx] = 0; + + pds->entry_idx++; + if (pds->entry_idx >= PKRAM_LINK_ENTRIES_MAX || + !link->entry[pds->entry_idx]) { + pds->link = NULL; + pkram_free_page(link); + } + + return page; } /** @@ -349,7 +619,11 @@ int pkram_save_file_page(struct pkram_access *pa, struct page *page) */ struct page *pkram_load_file_page(struct pkram_access *pa, unsigned long *index) { - return NULL; + struct pkram_node *node = pa->ps->node; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_LOAD); + + return __pkram_load_page(pa, index); } /** From patchwork Tue Mar 30 21:35:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92309C433E5 for ; Tue, 30 Mar 2021 21:28:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6BC4C6196C for ; Tue, 30 Mar 2021 21:28:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232662AbhC3V12 (ORCPT ); Tue, 30 Mar 2021 17:27:28 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:60774 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232491AbhC3V0y (ORCPT ); Tue, 30 Mar 2021 17:26:54 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPOZn123006; Tue, 30 Mar 2021 21:25:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=9KJx/AvLkVuTG95Nz9S7Rml6tSTK5D5P6CG5yLsuZdY=; b=sckAMlJlOto+DjOfTm1TeqhIe3N1CQl6wBlChEyUJEsSvv9RGI/vwv0JoccwfIrrgpUu EUbBw1YceXSlfApm41nZ585sCQGAA0mIAh9VAVSGe+xbuFICNjW9WVvDLaTxWPlxfKRe 3c7n9NMnPMcO9b+7C1SkSb+NJMzL1ohdseLRaEilwFN7Y+TPpPP/kS/LhJbKt1zOf/vk HPSCc+Rq7fFhTsuKVAQnmFCsOVxRJ05ZpjzBLS/rZHPr2g1ZLHc4zxDJhaxXXlviBCvn NmYFsXb8c/ks4bEctQdfx8kM3j2VQ3dSSSlyEH6lt4UqAN6QL8+Vff+Z+ls/0eK3+JXf zA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2130.oracle.com with ESMTP id 37mafv081e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:24 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOmHf183931; Tue, 30 Mar 2021 21:25:24 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2174.outbound.protection.outlook.com [104.47.58.174]) by aserp3020.oracle.com with ESMTP id 37mac7u3yf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DnVCjXHA7zw/J/hz9kgdMnEX5cslK3Kq4iZ3Ky+hqK21E66VOtrqoKWgVazC8gscyc11QkcR958qFa7PSwlO1F8xMydNLd4ULu8GfigdDV2zYo9MQma8hBIwTsuIlR6bx9M5JhFYAgClkv2tpWEz6Z8Xfqb70C70rz4/pXVS+oEUAfnHF92J6Eex8YcDs6OET/sFoAS7pyIos1x3bUTSWCAkGIJsqPrSuLTMhxb6H+j4odJGuKyNMuhPKnXnucorMm+88wdbbq2pwiSVhwrHzCFaxNaqbeGWLrYjvBPvShUrcgs4jBNqXYj5rPL5EVZ4soeV7MH8y0cpI5R+r37MNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9KJx/AvLkVuTG95Nz9S7Rml6tSTK5D5P6CG5yLsuZdY=; b=DzJV3r/nIAaavy3RNN7eAlgzlaAXIKlAn9y1AJ2OOtMTn9KqCTPHF839gSSi7VKrXYvmxoglNPYdoExvcEql1lqcnHhtGB74HKVOPW12sNm46FsYN93phsca1z6QE19H/QHCXzxJcwYkySbpuiYTPAPlefYXNw2oDaDDqnZkKENpkEejLY8uEL69N56BYCAoZb83GXQHHPhPvtIjC0F3OBFaVGcKcI6x/3SBQ2GKg0tT6ZovYirgI68IDl3SWoliGaQvLgc9dk/pKVlyicRVPlcJ/hR0mHZLQAgm66PuNdg1dP01FMYwALl/KQh7/kp+T1PWRd8Vtf6w3WVqZNILGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9KJx/AvLkVuTG95Nz9S7Rml6tSTK5D5P6CG5yLsuZdY=; b=UR+T/+dNFBlkljLkyrxFhtVC/wg8otMnUuPwTMWWbsx0RlbTm4Y+t5tmg/gPOGRTeNTO59kB1xVK6tlbWCzrRlBwteRy4KL8cVPpOUtfPIUXJBSCWZ87klDJx4Z518Kc2f1JRd8kKMyDwjPxf/s7CK+YKvWqjORLP4Q9Y01V+QE= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3120.namprd10.prod.outlook.com (2603:10b6:208:122::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:25:21 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:21 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 05/43] mm: PKRAM: support preserving transparent hugepages Date: Tue, 30 Mar 2021 14:35:40 -0700 Message-Id: <1617140178-8773-6-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7da638c5-3afb-45e3-afa9-08d8f3c252cb X-MS-TrafficTypeDiagnostic: MN2PR10MB3120: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: srKziVRZBVYXXKXAz9YlwY1e+E5gtEcphN71s15JQY/sbFV55Ii0bKHw94EmHMlJ1U9s/CaTSSuTormn0W8i+F+4iFOZfJ3WnX8UBQOJM2LaRe1DSyCMkW2uqR1Z18pcZCcAhUFxREN7k9pc5Um2oxajdpwmUKWV1IygHnNg6g+ja5ms2FICAd2kWuTd4Xez8eDr4XBXPA8QyMz6Ad/fnqGmjrBaTIMb3KvU/j2kl6JkqjEPm4BDznhXKdr2yrmhOF6UeAyvyGpubIHaW6fERKxxfEXh/7Wu2qV4C86jVw+bxjiaY8lVE+IRcnKiG+JnDnfo5/n1WWS6WtyISK0mx9isrsdMvFzrhuqMDQD1cGEVuziPJFxjUfV8DdqwzH1AIWqhznuNpMQ/FishcSkX/lyDpDdYeuMFbocYfxU9s1QWvTijM05+TwbUYU6jqSq/XDL5Pa90qhKVVdYTr9lbc1aszwTRZ/KukXjeCIYz90wsvACS72p6ktJcSEQflYke16qQg97meXuiCadAeDAVGGdVMTQc9jx94LXJgfN/MXT7djHWFNkN/SFkMS2Q0Yv8zSAYjVqE2BQPUV6zb3RqWgUADOKICZvbfiiYevlUogQCrSGS3wpiapDcHZUJbMs+KhUj+VbuRn9pwu7SOqELGA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(366004)(376002)(346002)(396003)(52116002)(316002)(186003)(6486002)(16526019)(5660300002)(2616005)(956004)(6666004)(8676002)(66556008)(44832011)(478600001)(83380400001)(7406005)(38100700001)(86362001)(7696005)(66476007)(66946007)(4326008)(2906002)(26005)(7416002)(36756003)(8936002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: aKQ5yCPIQoDErcrOAYYtwRGvWsp0EDoHnHJZ5ix12by3wo1td8Cn0hGjdxhkEtt2VheoEM81loSqOt5Aj/diy/cdGJ4dWh0SCmtXj9zAolMWuWXBFOOVG7vR0zEHu0uaNAszgTBZevhEMMyV6YV7MSCOxA28wjUCHEKMkkIv0sXpV/C+6j+jshCyPGIHAqRQB/Hk3P+qCsH5tmwOMz4H1LfbfG0joKdHAnFsMhpCRQaeyAG0WSgTSl8JSLLcpXxCUuKNUNgEctg/+0WXwjjMTTzNDA4lX8Txxk6lpL2NS8Nr3jgLJZkYGQPfTuHqafnmIPFpCNxfJ2doiWdpB/yXg2eFgMa2ORfLLbdrN/xTF2UBwvP3LfCphU2ViDLnfExKkRXFQx/7GBja8duE3luIs93T+T7zXgYoR6beTipTRq9fY4EhG02qrqjA0GQDJNse3g1H3ETHf+Q/xqBbrFLIbZpUbGtCMAjVtdaOKan989qiDyM25IQZ/BjGR9t5xFvr3rLjp00Tkr1WOGHTPMpfbQJiUAxmh4gHcfNe7eVfwLvXdCSWIxwtQXN0ADy0UNmNfVNUFvdx3WyWrxp13ouar25l0UuVJYzbIYuCyh84MBwsiq4uk7xNSG4/04TzfC8W4GZvUjHf95yXlyvccVMlSOMHIhADopr0v7GxVf1vokEU1j8hEY80lOnoQLYRNSJOUwiq/iUJslkZeUgIiDMdamrGnZIjtIpHGv9489lsFA+RbQofvFwNZtmOej8qWcIa+kACYxWVjAiAVOdUfWWLmMoiHP4DUEonXQ8Y0ZoD3/sgXbDuiB3a6XrJA1t9C9EE1fTywkOZvK94mS3CCi+V4sSeqYMUiC+vMj5QelQE1gGT+uWk74dIJI3piOZlV8igI9RRACJof2UPmeifCP+kST0XIwx2EPApIzdSyCnwf3+U/HVB2JEFYgGHbeVNMOjozdoqZecpymIyGy88kFPku+m+aYQmMeuHG2MWwXiFKQUKr/AyZUlnve/z354FyC691ISSppX8rw0Bifmh2ayhxHF4kUznfVBYgNNTrRKVcn5OLDkTHvHs+TLcUnEMvFLH6s7035sHve/jCiinBeWeVhOh6PGWBZt5YoF5uaLEnAbK+su7BkXZaJ627F9l6nTsDdjQhv3M+Gv628ejS7r+EawpXH9O+LkJwIwIQ7qMbiesDPLBiNV3DR3QIo+JyhzilpoSbekQ/nVkUIe2nI6RG+rTvcKT6ktO+Scg6hzfsJbh7xg2va6+ZWvxXK3Ev3FAry5WmhNHZAANjTLqeX9tH1e3T9YLJy9pkc88W2BFtHOH8jMujc+258n9wGUSzFsB X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7da638c5-3afb-45e3-afa9-08d8f3c252cb X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:21.2949 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UMtv2ipe9iQvNd+S+Xyb+Y871Mv31zjx9rNU0fsHaU/6G5dAOTHGJNCcR4LSSm4tOJhJos/LeEE6Ai/2DTW7T4KrijdFjCSBOcp/EQthmzg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3120 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: 60D6oIWZKJa9H0rka--lxzY85z_1zNud X-Proofpoint-GUID: 60D6oIWZKJa9H0rka--lxzY85z_1zNud X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Support preserving a transparent hugepage by recording the page order and a flag indicating it is a THP. Use these values when the page is restored to reconstruct the THP. Signed-off-by: Anthony Yznaga --- mm/pkram.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/mm/pkram.c b/mm/pkram.c index 9c42db66d022..da44a6060c5f 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -21,6 +21,9 @@ #define PKRAM_ENTRY_FLAGS_SHIFT 0x5 #define PKRAM_ENTRY_FLAGS_MASK 0x7f +#define PKRAM_ENTRY_ORDER_MASK 0x1f + +#define PKRAM_PAGE_TRANS_HUGE 0x1 /* page is a transparent hugepage */ /* * Keeps references to data pages saved to PKRAM. @@ -211,7 +214,11 @@ static void pkram_add_link_entry(struct pkram_data_stream *pds, struct page *pag pkram_entry_t p; short flags = 0; + if (PageTransHuge(page)) + flags |= PKRAM_PAGE_TRANS_HUGE; + p = page_to_phys(page); + p |= compound_order(page); p |= ((flags & PKRAM_ENTRY_FLAGS_MASK) << PKRAM_ENTRY_FLAGS_SHIFT); link->entry[pds->entry_idx] = p; pds->entry_idx++; @@ -516,7 +523,7 @@ static int __pkram_save_page(struct pkram_access *pa, struct page *page, pkram_add_link_entry(pds, page); - pa->pages.next_index++; + pa->pages.next_index += compound_nr(page); return 0; } @@ -542,19 +549,24 @@ int pkram_save_file_page(struct pkram_access *pa, struct page *page) BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); - BUG_ON(PageCompound(page)); - return __pkram_save_page(pa, page, page->index); } static struct page *__pkram_prep_load_page(pkram_entry_t p) { struct page *page; + int order; short flags; flags = (p >> PKRAM_ENTRY_FLAGS_SHIFT) & PKRAM_ENTRY_FLAGS_MASK; page = pfn_to_page(PHYS_PFN(p)); + if (flags & PKRAM_PAGE_TRANS_HUGE) { + order = p & PKRAM_ENTRY_ORDER_MASK; + prep_compound_page(page, order); + prep_transhuge_page(page); + } + return page; } @@ -588,7 +600,7 @@ static struct page *__pkram_load_page(struct pkram_access *pa, unsigned long *in if (index) { *index = pa->pages.next_index; - pa->pages.next_index++; + pa->pages.next_index += compound_nr(page); } /* clear to avoid double free (see pkram_truncate_link()) */ From patchwork Tue Mar 30 21:35:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5846DC433FB for ; Tue, 30 Mar 2021 21:28:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 407F6619D6 for ; Tue, 30 Mar 2021 21:28:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232827AbhC3V1c (ORCPT ); Tue, 30 Mar 2021 17:27:32 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49758 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232549AbhC3V0z (ORCPT ); Tue, 30 Mar 2021 17:26:55 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULNsLD130340; Tue, 30 Mar 2021 21:25:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=UUYO3z/elouuXXsWyOd9Gmz3+wWbtLHinkjbPkMqato=; b=MaFtLmlRz6ddxkE1+hJpM4l5EM4wVNEDbkBG+ngVA+H/8bL7FSAcVcNibavEh56/MmEp emP+t5Vri7uwWORBH8J/SWirOd3NM+UnioUabJ5ACDrqCZFn67Jzw/WqqtwG38teBACM YCHDzlcGRFCFwZntI1kyTBG8pAIrSEq6FQgya0nj6RqFIiTCCqv2KlEKwFjuF5SSt7e1 6/phio8p0T8Fe47jSCj+wp5CH1Acolbg3EhMzBGkLVxeqnjCluKbi4yo1KPKue5dq+tx xXJmsvMqLsYPk0DMbSDvBA/rnBLGLV3VfjnCsVlj2tv0UYGdnKLhiMS6QH+ZQrd3PR4p ng== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 37mabqr8ms-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:29 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOax3149709; Tue, 30 Mar 2021 21:25:29 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2169.outbound.protection.outlook.com [104.47.56.169]) by userp3020.oracle.com with ESMTP id 37mac4kchx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:28 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hjs7DI4/tSUXc09pV92PK6eeXc7/N1IQj2iQ5rw/s/frR5Nppg7Aw38OqM8ifBPnN68xhShtGFtmkn029N7UZbWIGHwyyipp6pz0pvkjoaEAsF6SICWEZP6rnjWZg4wjK56mj2eZvaDvKmlPWc+n2OsZ+kHwhJKqXgzgHaUmJdCAWwHzk/7hiu8PB7omi5C2ekAjwh5ipeAfy1pN5yU21SMo1UfkqpWWZ7z+SoINZzePwwlm8jPj9Z2psOlQmQ3Ny7myheJFAMS/X4ecSd7EiW41eL8jGlvdKzip+IRSaDFXLBstVoJUgd02nvsrSlYqwXWTieZyw+y53YlaREJODQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UUYO3z/elouuXXsWyOd9Gmz3+wWbtLHinkjbPkMqato=; b=kx0rl6MW6X3hK9JExicb517RMM9r6zcA4L7OXBkD/46YyUMXyMy9VmZj8QYERkIVWS7wBvj2pLgAeFNG05/JwD6iZQhsqwGnGkIxBS5yrnnPujndNwyyBzvWtAEgNylBPCpmL8cPhov5d30f1QE5WVSUf29db2NyeViGBsE7o64JktNBy5cG3FtoPD34Z+hpmCtfTHcKx7ZsMWJJaxhadIqpVIGlkVrbnhynXx2Xfvzg6Y/0m/z8xcrJp3KNmQJV809TwBvR/B14cHoGaf0Mz8r/E3HWidIV8fEUi7go6uecfoeXGnmwpuSIRPuuin5sttVpvgvwuLQPvkoHs102cA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UUYO3z/elouuXXsWyOd9Gmz3+wWbtLHinkjbPkMqato=; b=PQ6d/gD6OoRalBUrrqq8IlKVik86JoJyxCnEYcW8R9vfGAHqKIxOKvunoDJ4mQRhTZOxi/GAevXwFflp7AZiRIuPT2tntENty+HhMr9U0ooM11mHD/Udn5SBH5xeHWhVkqlUk0VyRp2uGTgdyHw3H0b9flCCqQbUZ3NVZmjWDbY= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3357.namprd10.prod.outlook.com (2603:10b6:208:12e::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:25:25 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:25 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 06/43] mm: PKRAM: implement byte stream operations Date: Tue, 30 Mar 2021 14:35:41 -0700 Message-Id: <1617140178-8773-7-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:21 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a23616cd-b4f3-4d86-2787-08d8f3c25559 X-MS-TrafficTypeDiagnostic: MN2PR10MB3357: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4125; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Q4nlL9xoE6KNeGmLZMvvWXj7Hccq6ZZqa43WpKoZuGSWGt/orQTvblAEcnddUg16q0+GZQYAYmq9QflJZ7FJfIowDJrzfD/L3c0mJIhXdDfQrb41h9BsO3rq0o14ahVyg+AUC8Jr94a3iwmBjkF/8MXOsK0bJlvICXwlSDzsdXofi3dfzgYEhNHlsDl8usfDwfK8Wz9AeUmeOYfRTzCY0c6xByCWV6a1wlEF0LAZrBKwzQFTPfqQ94fzP9GWOiTM8gbX8je9aXA1JI62AEFqK44/e8P0qRYxwrUreCtpol5v20iJotA8+bk95FgDXCiqHCTT98LbBSR0xrs2MZXUpKazI4wdNYYePdkuPJx7QC2pLMhFZqzg7co8cxXnQTGlCHSRA0sgMYQmhLbjOy0HR+rVMHq2IuO831xZmEFAQBQ56phQGCipoI4EC3Gbv6bzG0wAVDoy5o56Az+QjcR5c36CX6kxoPTdlBEYKwQaRm7ibjT9oDw6LwLVPFjyJT6j/asuD3kjrF5SLtglAAhqi+AG9xiYh91MyFnz3OT0t8Ki38rps9cpzWtCopGeali3Lkq2s7D4m3Iptk+Ojd7N0PLKcpseWSruCe1HanyKQP6j6jjJ55atiw6KBQdE8LTTAlMQd9V/zre2EGFJBJlSBw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(39860400002)(366004)(396003)(2616005)(5660300002)(956004)(6666004)(4326008)(16526019)(66946007)(186003)(44832011)(86362001)(478600001)(7406005)(26005)(38100700001)(8936002)(52116002)(83380400001)(8676002)(7696005)(2906002)(66476007)(66556008)(36756003)(6486002)(316002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: iouGMKabyhkA65Od1i9GabpQj+LeKT4THNDuW7SnADXVyrMN2Q+DhoFn3d+iz7XdcVLKbFM7LtndxvLpjC0xIjgoDC9thC5bDE6L3S2kSA73rTXhjbhvEAxCtDBA2X8XjluUkf9e7XxxCzfujLU/58lo2DyzlGOjUc+bW54Psw07iH+anlcWHWu5/r0bUFUCTnEZPLfzmWgCmaOCqbX3a7E+wyDPyuwlHXn75XOwEPaBrR7s2rrdigFM5nL6Y/xB60G68NxH6FVRXep+zwe6SiEY+p7xRIcb8HjIH0xttsI7oJiHMe9+CK7IqZ6gdg/B+c8PwZlY1sWSFMeMXgdyOcxkwdNtjT/0EM4OG7Uj/I6KDRkVxSXqDauaBxdanpVWWoiwSMb3JASNep4locdibasrJyJNThrpMxvobtBkykPdAaBROHKMgCyoXtHVeI9o9BddvRIEMfo+5cekASYhLOLM8Uc7X7EfoGHilqYdEkWBZizz/wOQCIW81NjQqmV0RNWKlYBA8Z7RjA5jBDPQm1KYRXdm+suefMGTjkpb3eoFqBc/rLl6hbCYi2YpHSGwE4KWDMHCZWthqiERBKuMcLvbJlTAuAdkUzXHorr4m+8kmckZIbEnbtFY24AnPgXDfLhKgw/lcIAl0SWTI6nQNe+c+R26bK/isf57+MD4od981g95v5wgnwGJkZgnjsCgquHffN+jOMUlBwluvkz0S3rNetilzBQQZOy7z69WLAfcbR3EGMRh3wcz3HxjiaUBTxVVkE/ZIVkAsq+ytwr56QWQzklsc2gqG70T9hvATNV+45uxYVDskX3wkpFTKcCwxhHeeu6ZQDzw4Z4Uwiz52TwfHRl99k0nvcIsNqCuxrGskkf6RFy/rzkxEWV7p/tGwUXnS277hLyTitm7nup3EfKsnIynty/SMiKUx9u62qwOttEsD8LdIXdZCJ2lFOnu2IJXk+Dduk1YWqMmyzwoZa605FXUHh8ZY5bBkMlilf2MYlTk8Q6V4henJrDquJvP0JyDmXgfY5w2pDiImezSPTDPyqDet1x+xgOl2Z8kh52/m/k08Xgy6W408Zv7gdfsp9Bu5qMsG+iYsZt+WodlEG5zi0O4/kvdGhjZ9jOOkRm26My+tewkQ9qZZKStblRzKF1iJyYMA0d7riDcA5Shx7MOX47Po7g+Vp1oCicvke71E1Ffp8Pgmq6+oLdOOKHdqpHomcjyuaAsYTkN/emYR2RoUOR6U+VTykBN/Rn0wex3XiNM0CsRPP1uHO2aCpaWGinptQFfimKAOg6J/+0cjNQGfn34W4X1gi2654imbSpBqghOvXMjTBo4s5mWOAHi X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a23616cd-b4f3-4d86-2787-08d8f3c25559 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:25.5945 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gzl7k54IevJzuMMPUt6dUN9L6ASuUKM5w9sN5vJFJ7X+QoavisAtMhPvc70WZN2fGgxNO3ga50R2UFI5bx9Z/I1rMTFL7E7Abiur2Hp9VBM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3357 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: rpdJ9_KxTJgStd7Go_TTy_GHTW9ym2jb X-Proofpoint-GUID: rpdJ9_KxTJgStd7Go_TTy_GHTW9ym2jb X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patch adds the ability to save an arbitrary byte streams to a a PKRAM object using pkram_write() to be restored later using pkram_read(). Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 11 +++++ mm/pkram.c | 123 ++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 130 insertions(+), 4 deletions(-) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index 9d8a6fd96dd9..4f95d4fb5339 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -14,10 +14,12 @@ * enum pkram_data_flags - definition of data types contained in a pkram obj * @PKRAM_DATA_none: No data types configured * @PKRAM_DATA_pages: obj contains file page data + * @PKRAM_DATA_bytes: obj contains byte data */ enum pkram_data_flags { PKRAM_DATA_none = 0x0, /* No data types configured */ PKRAM_DATA_pages = 0x1, /* Contains file page data */ + PKRAM_DATA_bytes = 0x2, /* Contains byte data */ }; struct pkram_data_stream { @@ -36,18 +38,27 @@ struct pkram_stream { __u64 *pages_head_link_pfnp; __u64 *pages_tail_link_pfnp; + + __u64 *bytes_head_link_pfnp; + __u64 *bytes_tail_link_pfnp; }; struct pkram_pages_access { unsigned long next_index; }; +struct pkram_bytes_access { + struct page *data_page; /* current page */ + unsigned int data_offset; /* offset into current page */ +}; + struct pkram_access { enum pkram_data_flags dtype; struct pkram_stream *ps; struct pkram_data_stream pds; struct pkram_pages_access pages; + struct pkram_bytes_access bytes; }; #define PKRAM_NAME_MAX 256 /* including nul */ diff --git a/mm/pkram.c b/mm/pkram.c index da44a6060c5f..d81af26c9a66 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #include #include #include @@ -46,6 +47,9 @@ struct pkram_link { struct pkram_obj { __u64 pages_head_link_pfn; /* the first pages link of the object */ __u64 pages_tail_link_pfn; /* the last pages link of the object */ + __u64 bytes_head_link_pfn; /* the first bytes link of the object */ + __u64 bytes_tail_link_pfn; /* the last bytes link of the object */ + __u64 data_len; /* byte data size */ __u64 obj_pfn; /* points to the next object in the list */ }; @@ -140,6 +144,11 @@ static void pkram_truncate_obj(struct pkram_obj *obj) pkram_truncate_links(obj->pages_head_link_pfn); obj->pages_head_link_pfn = 0; obj->pages_tail_link_pfn = 0; + + pkram_truncate_links(obj->bytes_head_link_pfn); + obj->bytes_head_link_pfn = 0; + obj->bytes_tail_link_pfn = 0; + obj->data_len = 0; } static void pkram_truncate_node(struct pkram_node *node) @@ -315,7 +324,7 @@ int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); - if (flags & ~PKRAM_DATA_pages) + if (flags & ~(PKRAM_DATA_pages | PKRAM_DATA_bytes)) return -EINVAL; page = pkram_alloc_page(ps->gfp_mask | __GFP_ZERO); @@ -331,6 +340,10 @@ int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) ps->pages_head_link_pfnp = &obj->pages_head_link_pfn; ps->pages_tail_link_pfnp = &obj->pages_tail_link_pfn; } + if (flags & PKRAM_DATA_bytes) { + ps->bytes_head_link_pfnp = &obj->bytes_head_link_pfn; + ps->bytes_tail_link_pfnp = &obj->bytes_tail_link_pfn; + } ps->obj = obj; return 0; } @@ -438,7 +451,7 @@ int pkram_prepare_load_obj(struct pkram_stream *ps) return -ENODATA; obj = pfn_to_kaddr(node->obj_pfn); - if (!obj->pages_head_link_pfn) { + if (!obj->pages_head_link_pfn && !obj->bytes_head_link_pfn) { WARN_ON(1); return -EINVAL; } @@ -449,6 +462,10 @@ int pkram_prepare_load_obj(struct pkram_stream *ps) ps->pages_head_link_pfnp = &obj->pages_head_link_pfn; ps->pages_tail_link_pfnp = &obj->pages_tail_link_pfn; } + if (obj->bytes_head_link_pfn) { + ps->bytes_head_link_pfnp = &obj->bytes_head_link_pfn; + ps->bytes_tail_link_pfnp = &obj->bytes_tail_link_pfn; + } ps->obj = obj; return 0; } @@ -499,6 +516,9 @@ void pkram_finish_access(struct pkram_access *pa, bool status_ok) if (pa->pds.link) pkram_truncate_link(pa->pds.link); + + if ((pa->dtype == PKRAM_DATA_bytes) && (pa->bytes.data_page)) + pkram_free_page(page_address(pa->bytes.data_page)); } /* @@ -552,6 +572,22 @@ int pkram_save_file_page(struct pkram_access *pa, struct page *page) return __pkram_save_page(pa, page, page->index); } +static int __pkram_bytes_save_page(struct pkram_access *pa, struct page *page) +{ + struct pkram_data_stream *pds = &pa->pds; + struct pkram_link *link = pds->link; + + if (!link || pds->entry_idx >= PKRAM_LINK_ENTRIES_MAX) { + link = pkram_new_link(pds, pa->ps->gfp_mask); + if (!link) + return -ENOMEM; + } + + pkram_add_link_entry(pds, page); + + return 0; +} + static struct page *__pkram_prep_load_page(pkram_entry_t p) { struct page *page; @@ -646,10 +682,53 @@ struct page *pkram_load_file_page(struct pkram_access *pa, unsigned long *index) * * On success, returns the number of bytes written, which is always equal to * @count. On failure, -errno is returned. + * + * Error values: + * %ENOMEM: insufficient amount of memory available */ ssize_t pkram_write(struct pkram_access *pa, const void *buf, size_t count) { - return -ENOSYS; + struct pkram_node *node = pa->ps->node; + struct pkram_obj *obj = pa->ps->obj; + size_t copy_count, write_count = 0; + void *addr; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); + + while (count > 0) { + if (!pa->bytes.data_page) { + gfp_t gfp_mask = pa->ps->gfp_mask; + struct page *page; + int err; + + page = pkram_alloc_page((gfp_mask & GFP_RECLAIM_MASK) | + __GFP_HIGHMEM | __GFP_ZERO); + if (!page) + return -ENOMEM; + err = __pkram_bytes_save_page(pa, page); + if (err) { + pkram_free_page(page_address(page)); + return err; + } + pa->bytes.data_page = page; + pa->bytes.data_offset = 0; + } + + copy_count = min_t(size_t, count, PAGE_SIZE - pa->bytes.data_offset); + addr = kmap_atomic(pa->bytes.data_page); + memcpy(addr + pa->bytes.data_offset, buf, copy_count); + kunmap_atomic(addr); + + buf += copy_count; + obj->data_len += copy_count; + pa->bytes.data_offset += copy_count; + if (pa->bytes.data_offset >= PAGE_SIZE) + pa->bytes.data_page = NULL; + + write_count += copy_count; + count -= copy_count; + } + return write_count; } /** @@ -663,5 +742,41 @@ ssize_t pkram_write(struct pkram_access *pa, const void *buf, size_t count) */ size_t pkram_read(struct pkram_access *pa, void *buf, size_t count) { - return 0; + struct pkram_node *node = pa->ps->node; + struct pkram_obj *obj = pa->ps->obj; + size_t copy_count, read_count = 0; + char *addr; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_LOAD); + + while (count > 0 && obj->data_len > 0) { + if (!pa->bytes.data_page) { + struct page *page; + + page = __pkram_load_page(pa, NULL); + if (!page) + break; + pa->bytes.data_page = page; + pa->bytes.data_offset = 0; + } + + copy_count = min_t(size_t, count, PAGE_SIZE - pa->bytes.data_offset); + if (copy_count > obj->data_len) + copy_count = obj->data_len; + addr = kmap_atomic(pa->bytes.data_page); + memcpy(buf, addr + pa->bytes.data_offset, copy_count); + kunmap_atomic(addr); + + buf += copy_count; + obj->data_len -= copy_count; + pa->bytes.data_offset += copy_count; + if (pa->bytes.data_offset >= PAGE_SIZE || !obj->data_len) { + put_page(pa->bytes.data_page); + pa->bytes.data_page = NULL; + } + + read_count += copy_count; + count -= copy_count; + } + return read_count; } From patchwork Tue Mar 30 21:35:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76F8CC433DB for ; Tue, 30 Mar 2021 21:28:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B7F3619FB for ; Tue, 30 Mar 2021 21:28:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232836AbhC3V1d (ORCPT ); Tue, 30 Mar 2021 17:27:33 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50812 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232624AbhC3V05 (ORCPT ); Tue, 30 Mar 2021 17:26:57 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULP4lH145287; Tue, 30 Mar 2021 21:25:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=XvhIVW1YeC3SfNdVXSE6ELE9IxQAzoSdaTuSqk4akz4=; b=DiY+ZxAssM0WGlPPytqDLZkcN2RCb4VdQqHXdc+TTQa88pLYH2/95P2VVWzdKnlk+3+U wZf+RlhYyAakT1/VpoMqzcT5IC/+oQwwoTb7/BDgHXYl7XlrY5Wke6bSRux1pEdbsqTn TwXpLlEPhrA3TSceiqbAif6nIg82D9BxhmDqmZ68HEnuUxFc1yok5N+A/oUpmFnx5c00 jOzEOGC4qOHDAFhvX9YXH3rmsxtpQiIDiRKe2NvuaiIgwSM6UEGW2HDWq2dOXtdvKFbr 5bV6uL8A122yxtM12JRbsBDrBg9n+XRjk9cAEHuSz0wGHfCMWLXzkBYj9LYYtctZDJfy rw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2130.oracle.com with ESMTP id 37mad9r8eb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:38 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPYUh124923; Tue, 30 Mar 2021 21:25:37 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2173.outbound.protection.outlook.com [104.47.56.173]) by aserp3030.oracle.com with ESMTP id 37mabnk6wj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mO1I4GGO3P/dZbG4QmxkdkGYWxI3Qp3veweN4SqLgTyTQqulGwo99CHhRLXFJuSkc8Cn/28mAVFlorBOlq2SeVT/7b8j6aY3mC2hUycy0OMfVhRaDXmHj5da5H6cKZL5sY9VXIcyPV8UPbZL4nOYTh9+mU/edu3wFtiE61Z1uKzvmMKnjzGfsBQAcQ/T7D4vbQsZSlraaGuBpsbSdGbcT605B2kP/ZxNj0M/7mx+M2MsOK9xf8vgmFjHOUrPrM0oJYKD3MPHzrc1UeSK6qD+1g3Vc+IJ7p41Snrkmyha/Ll1ysqNyvj2DSAM4wgZqOlQdWK+dYq0KZiZF6EdMk/ogQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XvhIVW1YeC3SfNdVXSE6ELE9IxQAzoSdaTuSqk4akz4=; b=L13g1nfF+upp44adpu7d51hkr2VurQt6pyMPkxDNNjtDiZlKkajnVnejQ/mAw2jfILxJAAwvwvDXDlfY9X53//HynWKJVl7hXxvkCW+QWOejuTMNSiHYwDR9S/pceUj20XiEHiIoTKVVCjn2Fk0PwaM8JeR9vLvUbJyfolD3XKR1mj+Tqny6F6rz4ACGoucEC61uIQKiF51WGmzlRTD7vAQmrlHNMKl0n1SaAueQ3+qIPFpHXUryhP3XmuEhUBDREolikXNwPE8JRjWnGVIGggaE0tbLu5YihPoLfjNRqzauSUghZTnOa667DF+O/pEfooU7oexNKc46zVbRNMgUyw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XvhIVW1YeC3SfNdVXSE6ELE9IxQAzoSdaTuSqk4akz4=; b=psm0brCwe2IiwwUOjtjTuTwYf8wcfBd4ihtWnui5oYcOKDWdWQHufg3Gss5ldBk6G80g+QISRgb9Nw6TbRoNnystgzXNb6yJ2tYrZjBIeW5nTzWTXgHG36FqRUg2yffsqX5KKTvsyP29orBiI4ERcJw5OQ56L/A+5LwQOa+al2w= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3357.namprd10.prod.outlook.com (2603:10b6:208:12e::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:25:30 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:30 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 07/43] mm: PKRAM: link nodes by pfn before reboot Date: Tue, 30 Mar 2021 14:35:42 -0700 Message-Id: <1617140178-8773-8-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:25 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8cbab8f2-e3c5-476c-47ba-08d8f3c257f3 X-MS-TrafficTypeDiagnostic: MN2PR10MB3357: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3631; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VFx8a+KrpvLWopgfain6CYKS10TMOy//BycVXo5CacmIuHSeIJQOANkD6NzH9+ENC1ixlIeQbaCNVMtyb9TqFbHeyzGxR1b94RxIhnY3tp+n+pzYc0CEhv8pLPI+EtT5vDpGeJtStz4XTxv0fqQ9H2/4rxxFlhFez/NZ5yt6VSoMHsW28y1vqJh6Br47MiTrEa7GRk+3CgH46hFQtbEV+2ra2i+nKA1nbnE0QOCW+AXqXr5FJmNCdOkMF1gckpH+Fdqyq0teLW//4CNuEUP/NUWmNXjxrX2XKlz7UsJZETo4eDJwSJb+/oGJFLTPXovyVuu4a9PatwSiLzDrTWn6o2X5p/hBYefV2JYWI06PXcs3yXc/7UEDdpJq+RpKlBsme/cQMLpnL4Eze6PnT7SGYs0v7/Exo1JqOb61Llr3rv7vHY2YptIfuti4OIQqWpqGg9Ed+BiUQotmrz2ni7ONyRHwjnYAl+IZ/UMJ+CFb0lAbDfBAS8h55LaJ195v3Rbk4URWdBXoAPHYZFRV7+swKNymO26N+FhrewqrgwNGTA8p4z9HiuVh6JvLkemRN8NFmQxsL74pAsX4G9yOYPSSNjJ/OFwOW+GBVdejV1nYnW8CMH9zOOVQYenjBSOb6fmhJLS8e3LE2AVwdZQJiyRCEg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(39860400002)(366004)(396003)(2616005)(5660300002)(956004)(6666004)(4326008)(16526019)(66946007)(186003)(44832011)(86362001)(478600001)(7406005)(26005)(38100700001)(8936002)(52116002)(83380400001)(8676002)(7696005)(2906002)(66476007)(66556008)(36756003)(6486002)(316002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 9Vlr0/2PgkPpUU0qcqde04DLi47hAuKHl8LJQYKfwe4a1CM1I/yWz/RUDZIgpWcvmzmn7kys2H6kL2WcpMuzesco+g2YLMFahcsr8NhFjrDAKUfiFY0+reT9V7xOu2vdDDHg/Tnn/u1qOIIL0udtHK9dzcjY6T6M4JrxOCCAfktHS+HQbWouvlX3Tauixtj6kIQ1Ym+532LDVdx3B363c6f81zIxqR7Iif7KGbkG+yhp9iew6lwa7CC8nTrLPfqS5V+wIagBE+AC+zWVPWPC2LyRPInbnNzWyNMCDkv3zAeyNUtmjpkVUtAGnd0jDo7skhXEornxtWk+jXHgXPsfKMmFK2lX0cUbCE2+hwBOWGVOgYinPxnW6bTCDtJLzLjSwV6lQP9XkN8U5QyUntkhsQh730RF0o41ebHAh8gQadGn/N+HiCxwcCJi709g+AyU2Mrlh0wBKIfuxH+55crPIBdA/ZgwUXO4B6il5Mc0Mb0LDffDJKOTcY12wpL8L3ICfyr2BmpKn9OBhIJQddBpN3CmOoGIzFkmBsRwAMrY6oSiiP9qrAvs8XrMgR2nGRPIocGD6KKm+dluIaoG6PLE7N5uZqotofNwk3DNZ6g4ys34921Im95gpzBG+kHeJl4afT60DRXBNv3KFQQwURuN5zBqGUI3RcPQC6oe3dfRT0tRKehSxiqI1BaaEiVC2zsV9Eom9mmyP37c18NQ1iXxPRg7e1pOz/5zTfjVEYBsrT34f1ni9sDFguMgCaWFvscXgIC5t/tI5OBoMXz1Q9qRcRRIQZCWIropg6vKecc10AtorHPqwHhPgXra46oqOKJd62wlice1zcWyUt+PiztTiHr+mseh+KGUk6s5+wb0B6u4uG6p3c++jmZ7F8KZIVUAmi3+tIDIJTBL/esuMbcfJXyCV1HgwY3h4BgnYwYRuLTEiQaNJ6QRv4huXoB2sH40NBjTtZYz/ux0Pw7eHVeCj9pvoozFklYlhh6DLh8iplMu6PphxlEX8F9wpMT+d/0+Dt3938utLQvvp8t7H5FJ+l4AGfKmSf875sN14vDjHsTfLYFYuuX+b4hiN64nTknuhZo04OZy+BzbGFNKHrdQGYQKkusvHOHhg2m0t1XFYqbwk76Ffi9KehEcPgd5R3dxZrsAaLgk6Elfs+U1DHNI7nctachqqmj21+lkQdtvKZaBkyIeL39cWZVkemnDHTmbsA1aCZ9dMCqRWCZ0wYuzxcr7YG79X0Sy91wkC/zTI/T5sBf11omCim5ctQYcDRy6DYS7p3Mk2lBEwEWHZCA2yA+S7gIZMtYiD0MZ2OJMxtCYdm0uZDb18P2612WDiepc X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8cbab8f2-e3c5-476c-47ba-08d8f3c257f3 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:29.9480 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pSV0MmixcpdxD2rcgUAE+cDe0j25Ud4Hq6BRreNeAmElVkVB5yTjxpoR6geQQp+ZGFhvLqfoI1ufaB1ejnWukH1l0TIpQH8tH95HAQ0rsAw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3357 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: My3yf6FMdkUK03pJEZ61I8lRZOQsyJqW X-Proofpoint-GUID: My3yf6FMdkUK03pJEZ61I8lRZOQsyJqW X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Since page structs are used for linking PKRAM nodes and cleared on boot, organize all PKRAM nodes into a list singly-linked by pfns before reboot to facilitate restoring the node list in the new kernel. Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- mm/pkram.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/mm/pkram.c b/mm/pkram.c index d81af26c9a66..975f200aef38 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -2,12 +2,16 @@ #include #include #include +#include #include #include #include #include +#include #include +#include #include +#include #include #include #include @@ -62,11 +66,15 @@ struct pkram_obj { * singly-linked list of PKRAM link structures (see above), the node has a * pointer to the head of. * + * To facilitate data restore in the new kernel, before reboot all PKRAM nodes + * are organized into a list singly-linked by pfn's (see pkram_reboot()). + * * The structure occupies a memory page. */ struct pkram_node { __u32 flags; __u64 obj_pfn; /* points to the first obj of the node */ + __u64 node_pfn; /* points to the next node in the node list */ __u8 name[PKRAM_NAME_MAX]; }; @@ -75,6 +83,10 @@ struct pkram_node { #define PKRAM_LOAD 2 #define PKRAM_ACCMODE_MASK 3 +/* + * For convenience sake PKRAM nodes are kept in an auxiliary doubly-linked list + * connected through the lru field of the page struct. + */ static LIST_HEAD(pkram_nodes); /* linked through page::lru */ static DEFINE_MUTEX(pkram_mutex); /* serializes open/close */ @@ -780,3 +792,41 @@ size_t pkram_read(struct pkram_access *pa, void *buf, size_t count) } return read_count; } + +/* + * Build the list of PKRAM nodes. + */ +static void __pkram_reboot(void) +{ + struct page *page; + struct pkram_node *node; + unsigned long node_pfn = 0; + + list_for_each_entry_reverse(page, &pkram_nodes, lru) { + node = page_address(page); + if (WARN_ON(node->flags & PKRAM_ACCMODE_MASK)) + continue; + node->node_pfn = node_pfn; + node_pfn = page_to_pfn(page); + } +} + +static int pkram_reboot(struct notifier_block *notifier, + unsigned long val, void *v) +{ + if (val != SYS_RESTART) + return NOTIFY_DONE; + __pkram_reboot(); + return NOTIFY_OK; +} + +static struct notifier_block pkram_reboot_notifier = { + .notifier_call = pkram_reboot, +}; + +static int __init pkram_init(void) +{ + register_reboot_notifier(&pkram_reboot_notifier); + return 0; +} +module_init(pkram_init); From patchwork Tue Mar 30 21:35:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21592C433DB for ; Tue, 30 Mar 2021 21:28:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 879FB619D0 for ; Tue, 30 Mar 2021 21:28:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232784AbhC3V1Z (ORCPT ); Tue, 30 Mar 2021 17:27:25 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:60778 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232492AbhC3V0y (ORCPT ); Tue, 30 Mar 2021 17:26:54 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPchp123355; Tue, 30 Mar 2021 21:25:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=zbgaaTx5IAoRKIXeSokkLnV7PhXDVAA2+4m3frQC38Y=; b=fAuArSYhZ/0njqM0/xhwEQmh4ItI4T4pbNiTvshbbXDTwzmuELrfdc3mZVUZbOVKEdIY 3JQcP3t4QDs/96l289WlXmgig3acWTAP4fI9sgN1W6TmIszKh5cJAVXMJ7KbLW3AoKID vDpSegCwuYFuQNUYszcZwZcyJkQWoQ9RQil9y192C38tzGw5t6QKGgzw443ph+W4HJRF 01H8G5YyAT23Arx4syAKL9KMU/bu/AirtRS+li3Rl66mUWW0YJBx0R6PQnkG+FmXYZVF OJKS1SVIWiXBX8kVljcAkk+n/ZM3GAFHYWVa0G85LC4Linr0fBJdWmpIQ5Qs+OA/tFhS vw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2130.oracle.com with ESMTP id 37mafv081t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:38 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOnNn184112; Tue, 30 Mar 2021 21:25:38 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101]) by aserp3020.oracle.com with ESMTP id 37mac7u434-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bgNL0vl7ZldyPsNdB1PMN2PD6styoePsPX056HiO6tn6p3orkTqm9zZJvc2a4yptdCGdZ3hs6Un73rcuoO1noJ7W1KFYcaRTqBzFRRxT0kw93uCCU4etc1lxTyzTu8nUO5h5FuJlcWD/RIT/Lz1Um0sXBvUJFSE3/4YsTJGBxUossGiqsNTMZBkmQCmOIXEY2ecGtMQVsp6SDnseU+kTDPS+IPr5duRoGSwOX/e87nT1nAkhrCzI5ml99rK0nze66MRi5mHiJ18c2y2MS22lN1LJFHTWAITix9YKTxywfdk+DP5fB1vkqxPsyt27KMr7JJRQhYbRETn26ECnjgmo0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zbgaaTx5IAoRKIXeSokkLnV7PhXDVAA2+4m3frQC38Y=; b=IQwBNOHz+251A5TaIpgy6CdJcSpx8eCVxM/V2pltt/geuQYPF/WGBuuRJAlNDMuyDn1xsxIlF6wCiSOqFDcFRGaZTs1HdQGu8yJitoHUwqG3Glfc9lurrZsWgvCAuaNf1m3WbddjaYHi4akSH8gXiz8a+6APDiEAp9M15whauO85xZYyZcxd9uljTsuY/wLtMAlNTSaCnSPpo13xycMAYNbkv91u9pcdnzKq1aJdOY7HcVZIVOGzr/pLD/JsylOBddwUrLsbrBS/FSaUzJU9Jguyc/c+OSu9PZNHzBZncpUj422P6tRfB1mrrGEEcpwoy3Vmk4rET6vgqtMizaJH0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zbgaaTx5IAoRKIXeSokkLnV7PhXDVAA2+4m3frQC38Y=; b=Q6uLLZpuUzxxUJ7Mlk07EnsQc0Vm79PB4ggFWzMNAP4Xw7N1FuYNLDiZGtjTtGZG/pFrAIxZMN9ZT+072RpQLyo49iLqt5UC+aGxu+Yay0zB0euZSuyka3b9RMf7W5UQsNqC1dkJC6/YbcneSS+RsFzh8mm3n44EI77x3t7H7x4= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:25:34 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:34 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 08/43] mm: PKRAM: introduce super block Date: Tue, 30 Mar 2021 14:35:43 -0700 Message-Id: <1617140178-8773-9-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:30 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 11bc6d82-2e77-4ce4-eef8-08d8f3c25a85 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FjAuyE4Tm6FVyUJ6CUgKkl8pjEnBVJhEq/E+VavbW6RwkjSdNyRWKG02F3r/mAqE000M37E7bdJhgnxRIem7UHwvBZy+GnlmurTNeM1DDn1qFHXdFOEfpLIIcfR5T5zEe/DGXRw6aMtmcwYcjVSa9AXJ8kYFF5vze1fmJ1rkXsSVyf77LtQJT9oyZ07PCb6nEAAB1WEN8arnvztE20cFAc4dc0kM3qnfsvgEYStbvl696+vSvp/2du0WNtbSmPFoQ5JpFz+nRjiImnkTe/MM/Pct0R4G/Dd9WiSPZL5wlywr10EUhlU13wceww+dZ/hKZ3Bi3YyVi/HVyQ7CQl+sc3uWO45JFy5EaLEBiXJYmz87aOw8rgRlqMb/EdarVWFaLNjJtlzRzZmwv6ooTWVSbr4Afq6xaUnc5yW75rTbh3dQ9Ni39QfkB4uFghPaL8s2hDIoVWX6ijBnypvCrF7xxIH7siiKYhcjGVDAfgJxtjXc1/p8p0vKb6/kms5/UNMwFxv4i0CDF0z2QzT9n1o87VvT8+v4c6SOPqlwQ14yJU0DJD0NmLuvTnyi12BVe2D62UzD04qnMbDpNQ3zXEhPpMOQa+AIUeAdxKMAyPMWbOEOiIhD1gmZB2Tvz93dgm2Xn6mBrtEfVcgvJXaTn2+q7J0I0IR+lTyfcEPhRay7LQE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(83380400001)(2616005)(134885004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: xef5jTi0Ql4g5jbOdmnrSP7ufNtB4VWJoprUt/wKxjSs2v1HU1/az6wnwiiFYeiJ1m/yHteh7s+PuCAVc+FtgtRJRt4bJ+LR/JIjeI3d5BE3wAZaTrsqX5ubkICqkO5bNcRTqfOeyExtvm19r0s3wzlmkpMJlI+UOT3WA8OdXNo3hhlWiEOKbo1CZiydpMgGBAVyGTFdaL0p+27ajqkvdasPeXqoEkD5DTRfdMyR35M9bOO5f98RBn8O3ZwRyyazjXO2TAlLkhLm0+V4H5qXMouVS84cHWgiFQ1ozVSR7LFWsNseNzzlt+h1X3O3T192T23psidgnBcZYvwP6F2KDTTZIBL0zd6Ew6Ai9k/CVBMlrnTzT0B6Zg9Ur1C69UKtluMmLFHP23oNaS+nSOTZnJXUGp/tzNQY+tEsqPQFzyzTx8KzBc4W5fixSakw8RzaUZhUnFKt0lkFNM8MbHSB5lk6ft0tN/5TO+vzaE8cniCDVekGj4KalEirNLpg4brCTyhy3AAQvC/FtZOkCNpiafFpUZeuABQAx3RZ+Uf4/hh9GClJl12fdzGhQHk3vaFJAKDa3HBsv89G16u4MmX0LXT+WDAthNceNmz6GBvGz5A3+WB3BLSREleTx/ufbeP3b/FvLgXYV+mIBB9asmTYWmjAc9ZQ5LMvs7sHQoemRZA31xnCnSsZApuwxCXCE97hkgQnGbbQG/iQ7ZtT+Z7fwnEYYHPGKqE6Nv45PgQKVVcQeTZqFALONX+Ub34a9/w19Tpk+S1RJ0V1qv08cRHfoe+Wn7QdGCl8RZg3ulYto8oN3mZDOFcKMzZ5oYxOO0GrXhHQPiT9jDH6kQu1b+DYo3RkdSl4SVZxVqIABYO7swdFoUbWjZOXxni9jZmRiWXfZ71hNE6Zrvp0CY5H3Jj202J/SrI/OoSfRmpuXo4WoR7l9cGwmxklTfMTwQYI9FGGEbN60KVgH/P2jtwww8FjTAWGrnIvJ3Yg5Em2B7YjF0q0JkYB1iaGrMo/b9hjdKyxadNzQggsAHINb0TAVKuD0ORToOm8v/OV2hAp1IU+bOne/ssOhTtoU+ul7m4+xQpUUsweEaluc+2lDA2zMCMXZNdHAtU2Z09nK4cmrt7Ft11veqiOdIEqtSTl69+YqZgR+nx/REIA/W/ydUQ3edMxh7RvpvIWVX8BldPmIad487NoVlUuNCu3dqrE3fzrgvEoeHzvVrm3xRlhaC6Sgr8NPLGIfsJ+D8AevQkwZHQgdgD14/oM4oVxbNnVdBLKpuLSblwQXyIeMMqSoxbQiIVD0lMEkmwQzYpmKuLYBtYDIsdWO14f5uye4ZP1QogWzbui X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11bc6d82-2e77-4ce4-eef8-08d8f3c25a85 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:34.2756 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /hccKn7CWbUJNMdnFma1E+85dLoZxHwH1755RMzlX0AwHQIAY/W2ByyV4A+b4kbXiVjkVyXyZTKre6Agi7u2d0l6r807mZZpel7cN20exvo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: eIYn0ncoyUl9EbS0Gl3kLgYIW0fZg6LE X-Proofpoint-GUID: eIYn0ncoyUl9EbS0Gl3kLgYIW0fZg6LE X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The PKRAM super block is the starting point for restoring preserved memory. By providing the super block to the new kernel at boot time, preserved memory can be reserved and made available to be restored. To point the kernel to the location of the super block, one passes its pfn via the 'pkram' boot param. For that purpose, the pkram super block pfn is exported via /sys/kernel/pkram. If none is passed, any preserved memory will not be kept, and a new super block will be allocated. Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- mm/pkram.c | 102 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 100 insertions(+), 2 deletions(-) diff --git a/mm/pkram.c b/mm/pkram.c index 975f200aef38..2809371a9aec 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -5,15 +5,18 @@ #include #include #include +#include #include #include #include #include #include +#include #include #include #include #include +#include #include #include "internal.h" @@ -84,12 +87,38 @@ struct pkram_node { #define PKRAM_ACCMODE_MASK 3 /* + * The PKRAM super block contains data needed to restore the preserved memory + * structure on boot. The pointer to it (pfn) should be passed via the 'pkram' + * boot param if one wants to restore preserved data saved by the previously + * executing kernel. For that purpose the kernel exports the pfn via + * /sys/kernel/pkram. If none is passed, preserved memory if any will not be + * preserved and a new clean page will be allocated for the super block. + * + * The structure occupies a memory page. + */ +struct pkram_super_block { + __u64 node_pfn; /* first element of the node list */ +}; + +static unsigned long pkram_sb_pfn __initdata; +static struct pkram_super_block *pkram_sb; + +/* * For convenience sake PKRAM nodes are kept in an auxiliary doubly-linked list * connected through the lru field of the page struct. */ static LIST_HEAD(pkram_nodes); /* linked through page::lru */ static DEFINE_MUTEX(pkram_mutex); /* serializes open/close */ +/* + * The PKRAM super block pfn, see above. + */ +static int __init parse_pkram_sb_pfn(char *arg) +{ + return kstrtoul(arg, 16, &pkram_sb_pfn); +} +early_param("pkram", parse_pkram_sb_pfn); + static inline struct page *pkram_alloc_page(gfp_t gfp_mask) { return alloc_page(gfp_mask); @@ -275,6 +304,7 @@ static void pkram_stream_init(struct pkram_stream *ps, * @gfp_mask specifies the memory allocation mask to be used when saving data. * * Error values: + * %ENODEV: PKRAM not available * %ENAMETOOLONG: name len >= PKRAM_NAME_MAX * %ENOMEM: insufficient memory available * %EEXIST: node with specified name already exists @@ -290,6 +320,9 @@ int pkram_prepare_save(struct pkram_stream *ps, const char *name, gfp_t gfp_mask struct pkram_node *node; int err = 0; + if (!pkram_sb) + return -ENODEV; + if (strlen(name) >= PKRAM_NAME_MAX) return -ENAMETOOLONG; @@ -410,6 +443,7 @@ void pkram_discard_save(struct pkram_stream *ps) * Returns 0 on success, -errno on failure. * * Error values: + * %ENODEV: PKRAM not available * %ENOENT: node with specified name does not exist * %EBUSY: save to required node has not finished yet * @@ -420,6 +454,9 @@ int pkram_prepare_load(struct pkram_stream *ps, const char *name) struct pkram_node *node; int err = 0; + if (!pkram_sb) + return -ENODEV; + mutex_lock(&pkram_mutex); node = pkram_find_node(name); if (!node) { @@ -809,6 +846,13 @@ static void __pkram_reboot(void) node->node_pfn = node_pfn; node_pfn = page_to_pfn(page); } + + /* + * Zero out pkram_sb completely since it may have been passed from + * the previous boot. + */ + memset(pkram_sb, 0, PAGE_SIZE); + pkram_sb->node_pfn = node_pfn; } static int pkram_reboot(struct notifier_block *notifier, @@ -816,7 +860,8 @@ static int pkram_reboot(struct notifier_block *notifier, { if (val != SYS_RESTART) return NOTIFY_DONE; - __pkram_reboot(); + if (pkram_sb) + __pkram_reboot(); return NOTIFY_OK; } @@ -824,9 +869,62 @@ static int pkram_reboot(struct notifier_block *notifier, .notifier_call = pkram_reboot, }; +static ssize_t show_pkram_sb_pfn(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + unsigned long pfn = pkram_sb ? PFN_DOWN(__pa(pkram_sb)) : 0; + + return sprintf(buf, "%lx\n", pfn); +} + +static struct kobj_attribute pkram_sb_pfn_attr = + __ATTR(pkram, 0444, show_pkram_sb_pfn, NULL); + +static struct attribute *pkram_attrs[] = { + &pkram_sb_pfn_attr.attr, + NULL, +}; + +static struct attribute_group pkram_attr_group = { + .attrs = pkram_attrs, +}; + +/* returns non-zero on success */ +static int __init pkram_init_sb(void) +{ + unsigned long pfn; + struct pkram_node *node; + + if (!pkram_sb) { + struct page *page; + + page = pkram_alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) { + pr_err("PKRAM: Failed to allocate super block\n"); + return 0; + } + pkram_sb = page_address(page); + } + + /* + * Build auxiliary doubly-linked list of nodes connected through + * page::lru for convenience sake. + */ + pfn = pkram_sb->node_pfn; + while (pfn) { + node = pfn_to_kaddr(pfn); + pkram_insert_node(node); + pfn = node->node_pfn; + } + return 1; +} + static int __init pkram_init(void) { - register_reboot_notifier(&pkram_reboot_notifier); + if (pkram_init_sb()) { + register_reboot_notifier(&pkram_reboot_notifier); + sysfs_update_group(kernel_kobj, &pkram_attr_group); + } return 0; } module_init(pkram_init); From patchwork Tue Mar 30 21:35:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADD38C4345E for ; Tue, 30 Mar 2021 21:28:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 78130619E6 for ; Tue, 30 Mar 2021 21:28:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232877AbhC3V1g (ORCPT ); Tue, 30 Mar 2021 17:27:36 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:42886 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232641AbhC3V1A (ORCPT ); Tue, 30 Mar 2021 17:27:00 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOMhL011475; Tue, 30 Mar 2021 21:25:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=ys/+06gv3pRz4k4hL35/QbiXotjBjyeNpQ6nps7tU34=; b=SOL77G6IO+HlfCTKPrNCCwOwu0OQQ+tCkdrGjSJS3MYQDWuxR+c+CD7057xnyhlJRQxa I//0gqirhBpmFyxpnBE5l/kzObEcogOBAq23YnpuFH6jJqQBLs4WtINcxzpvYFfvfRBW gtIR5F0c1h+sx32PAxNqJdRIunrP4t4CfaQWdvBR5V0nmQUpX9OtCpfqIVOAP+NT7hVi NqIMXVuemdulMXLtns5eNhj+pocJtE5VLoeFNeclyLAnVjV7yPQnmlokHI+5hTFt9hYi WEvNlm7bvpWbzO0r6LUlbbb6LveNe7Tlp9FXhGPCp/pr1ON2RRS6f7MQFF2PILYerxd7 LA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 37mab3g8vh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:42 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULObm5149856; Tue, 30 Mar 2021 21:25:42 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by userp3020.oracle.com with ESMTP id 37mac4kcx1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:41 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KIYHuYJudj/qLQ8ieSJmxHSL8/pwAa44oaQ0P/IrvrXN0KQY/91Um1R+3HqaWOqOzVUpb3pB7q1QHbi+e21eUAB9jiZl0N5WaOa+Ph+T3RbUG7nIb264vwrqcJtJ3ElVZ3iNZVfaK4BvgIyOgpR2xCi0GV0otYNJ7Z2itcIjLUyxs0+ru764MR5YR9Pd/KvA5XqfYuK/USFOo8teIL9Hx9+bS5SGu+T0wdi4IOh1lr0tXCHjluhRTY8QkE+Ket1PNzgvmjKu+3GFL6YwVcKo6ENjzfH0dSUbmHh81TqMmcZIIIl5iDIaZAqxHsqgtTyB+Wztmr7X3aAK1pQ+DCq5IQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ys/+06gv3pRz4k4hL35/QbiXotjBjyeNpQ6nps7tU34=; b=Yk3LAZWsSMEVQFXnx1DlgGVKG1gr4velt4s/IYyW+NwOhHwMKSNFERrF0z644YGPHs7q8TyrLFSJxfp2Xl2KHcBZq+udQxn43hMjRbQlIeC9bqiMbTxSO3p+X8yWkPSeExoBkfA+Bs0X2bgWgmesgL/BeSvUeNWKyA2mBsIJ127Ox9XZdoxfESrnlpl+u2rL+qAuEDTkvGg/cDfoyRQoiHezYXfwCwxGLdV4oMKWXp0jjaCRIMvoID8Z24NsoKjWM0KGdmJCAYUKLMuIEv8PQhx4sDMLClsUPclGhtgHHEVJfeNxfY19clhu5XgPd2Cfx6Peeo4Ci21e519/zPjaxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ys/+06gv3pRz4k4hL35/QbiXotjBjyeNpQ6nps7tU34=; b=Dmb+fpi4wD4zuw66vcQxomkjC+MafBL2cW5E1Sz0yitzCpdLLm9VcSCx6EkF9jzF/4k2lfMkbO07egJpuK0eCrKQ2aq4yqAxFNEerHN2s09IxY1dblmeN/NqKLOPwfJiyTGn2TiIc+wmLXqMpOMZj00ZmK2e8N4VS0lsBbPhzfY= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:25:38 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:38 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 09/43] PKRAM: track preserved pages in a physical mapping pagetable Date: Tue, 30 Mar 2021 14:35:44 -0700 Message-Id: <1617140178-8773-10-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:34 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d7791852-43e6-48d0-8074-08d8f3c25d19 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1468; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yb2UGdWaxWxic4lHS+gcxx0xNGi82a5rCCZQAFc6a8IDHLgOhpPYGvnKkXLeJ3vKW7SRPA8o2HI1J3PnfOonMWOkwcySi5utN0F2Xic+cAwK0yzCOyfzgCM5GpExaP2e0HRg63CLHsIRZZ9UnfHn7uJ7CtDBrPYp6ydcSpJxEqER12cPustewuWZnMIjURpOsbHdPRI+PIlYFgU6Psp1aqwUz1KLFVDzSgayCqEQCxhaGtsPnfbBoSB8+7nK4Ur55eK+F+/6/x3KPVoAquUBPiFltv1TE/iaV5tR92c9QLFtwNli8SNgRd5r4dRYQqQTmrhBwa+V8AszrT4z058LmPqm8pu8XwfgjXggfCQMLqCchOJGRzVx3m4eURHU8+OXZjWoI2Aiji/rF919Ue+IVl56uZ9/XAyMieAn5pNXjykZ2tZzWl/Qo5H10MhGX/PxFS/CqTRXVE0Jt7QZ8N1h6s0eLXXvsoZdR74nkkvnVJYs6+gaGACwyUy6PKvOUoLxYN5R0LAHigiBkHWoT7aYa3u9+/nrT41GGcsfBHSOVlDJicA5njr54FvTiP3X0cQfm73BmNhEFqAQzQDPDzOXyJfHfxp63tPAJI+AVX4u6b4c8cb+aJwU3kikdR42qfT2BkMHASMoGmlHA1z+fZB38A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(30864003)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(83380400001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: SiCYrmtcuaEa/F1G9UIGKcik9TEWpmFKLLjl7V4qnsutI0T1wlQjMQgU8+/86UPwVqxglV0MX0DvX0CZAAuNtK1GyQLWKz0CwITTiyU9I+iJ2Y093ZNIarI6DbaNX2za4oNHXzDtorBTdiL3/Vc9ntqcNUnIBBCag/Bye2rRPyem/JlDJoM97b3NkfzufRziHR++Bhz2jENoN4qgrGpjrbr3NmbPxpXP6RVt636a3OJSvwMVbTmZrzt0WhixnXTkuO2KY/t2M78d9UmT3gMVnYYOBFjFcgI1kjMRScJ4XledXXjsJ+IsOJV25Z/tDqCblSCLtX1SBzlfL4qIHTDyathFDuW3+jExr+tv8BEnIo3AJ0ex/kCp26hrROcjLBhn0zEWPZtIZcV8e9ofJsxrJUKC4ab5IOhfiW1t7qz6ckuBkD0cqeWMhLB5L6a8Yco40M1+17K2V1U1bWkRLQuyPuPqlT6RE36LfBe0VTRqDEd+seTqz4MJPoKO8+2Cmv31PRWQDx0FwlQ0RZIz6cFLZvm6f9AwuOoMgMq/Q6LD4lDGyn4Go15FLV6j8dW/E2GJb0E/ghV09Z5HR97szQYphUibnLyy8UbRHm/c1im3v7dab5B3j5a6QOdKktgtFAppM8errTga9AI1/+NX8ggxZrJozIsgkPtCdF+J/3fC6RWCDIjzwWt97X7s07tXFWdZ8t/gOUP1R4c2lefnwdb0vwBnaRNYRyGaAItHQbnKrz5mcKMmKdgL3OsM/6MNcyYmH59BpnXZn9qCVblretk6QROCCGN1Kz1feHZJE8sA2LQUvEkOMClOOXeH8WsTehMIgxDPFNhEsMvVzNUBDXLySpWG967BRgi1XinKEXbbjZlKel22rpVXgl4LNdlsu3GkvbqhSaiMQQPKZ7G4Z82tRL9pIw4AiSGhyrFCoYWUtp3CwUKsERojlqPCzZXB3EW0stztv05b/kPrN9C1mas1MMzaIgXq2Q1eJH4KqKq+k4j/5SLutfk3VSmWGI1k1kjUn9VGZKL0vxJSWWc4LWW4Wfc3EdQcDJUxBtZrNnZ517tcAQ0DxrMNdDtj2u2eYH+1QOFA9oBEaU20rIMjt1LT4HKVQlqVKXLJjcD4SDDH63L5gYgq5IBDp1Tdq+gx/mrXB5oyBL8mDQpr9GqhvVdZt5UOBb112ljF31j823XxYYp/RTHbJudtgQHePBZpcVp8YyP6WC5iEe3Jarxp2vbp4WaiO11peOMnl7RV1Peb7a1KCBaZMKshQ5IJEH3BPIxh0aRTuQ8tpvj9/ySb+CVBQUYMk5syj8KTq5loW2gVA2aQI1NQE4kSww/yeHcEoBki X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: d7791852-43e6-48d0-8074-08d8f3c25d19 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:38.5002 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ddLMi5sK2RptwMUGvC9c98IB8rpCCp4KFGJ28/r7Kblwolw+JFZH2G6t+ETixPr3/HoxTFnIPDYj7xsNKMGtM1YAInLmr+pYOdfrY1DXjTE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: PM7D4WN6LX8NRO0AsWT9UFEq5MOfWWPV X-Proofpoint-GUID: PM7D4WN6LX8NRO0AsWT9UFEq5MOfWWPV X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 phishscore=0 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Later patches in this series will need a way to efficiently identify physically contiguous ranges of preserved pages independent of their virtual addresses. To facilitate this all pages to be preserved across kexec are added to a pseudo identity mapping pagetable. The pagetable makes use of the existing architecture definitions for building a memory mapping pagetable except that a bitmap is used to represent the presence or absence of preserved pages at the PTE level. Signed-off-by: Anthony Yznaga --- mm/Makefile | 2 +- mm/pkram.c | 30 +++- mm/pkram_pagetable.c | 376 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 404 insertions(+), 4 deletions(-) create mode 100644 mm/pkram_pagetable.c diff --git a/mm/Makefile b/mm/Makefile index ab3a724769b5..f5c0dd0a3707 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -120,4 +120,4 @@ obj-$(CONFIG_MEMFD_CREATE) += memfd.o obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += mapping_dirty_helpers.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o -obj-$(CONFIG_PKRAM) += pkram.o +obj-$(CONFIG_PKRAM) += pkram.o pkram_pagetable.o diff --git a/mm/pkram.c b/mm/pkram.c index 2809371a9aec..a9e6cd8ca084 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -103,6 +103,9 @@ struct pkram_super_block { static unsigned long pkram_sb_pfn __initdata; static struct pkram_super_block *pkram_sb; +extern int pkram_add_identity_map(struct page *page); +extern void pkram_remove_identity_map(struct page *page); + /* * For convenience sake PKRAM nodes are kept in an auxiliary doubly-linked list * connected through the lru field of the page struct. @@ -121,11 +124,24 @@ static int __init parse_pkram_sb_pfn(char *arg) static inline struct page *pkram_alloc_page(gfp_t gfp_mask) { - return alloc_page(gfp_mask); + struct page *page; + int err; + + page = alloc_page(gfp_mask); + if (page) { + err = pkram_add_identity_map(page); + if (err) { + __free_page(page); + page = NULL; + } + } + + return page; } static inline void pkram_free_page(void *addr) { + pkram_remove_identity_map(virt_to_page(addr)); free_page((unsigned long)addr); } @@ -163,6 +179,7 @@ static void pkram_truncate_link(struct pkram_link *link) if (!p) continue; page = pfn_to_page(PHYS_PFN(p)); + pkram_remove_identity_map(page); put_page(page); } } @@ -615,10 +632,15 @@ static int __pkram_save_page(struct pkram_access *pa, struct page *page, int pkram_save_file_page(struct pkram_access *pa, struct page *page) { struct pkram_node *node = pa->ps->node; + int err; BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); - return __pkram_save_page(pa, page, page->index); + err = __pkram_save_page(pa, page, page->index); + if (!err) + err = pkram_add_identity_map(page); + + return err; } static int __pkram_bytes_save_page(struct pkram_access *pa, struct page *page) @@ -652,6 +674,8 @@ static struct page *__pkram_prep_load_page(pkram_entry_t p) prep_transhuge_page(page); } + pkram_remove_identity_map(page); + return page; } @@ -898,7 +922,7 @@ static int __init pkram_init_sb(void) if (!pkram_sb) { struct page *page; - page = pkram_alloc_page(GFP_KERNEL | __GFP_ZERO); + page = alloc_page(GFP_KERNEL | __GFP_ZERO); if (!page) { pr_err("PKRAM: Failed to allocate super block\n"); return 0; diff --git a/mm/pkram_pagetable.c b/mm/pkram_pagetable.c new file mode 100644 index 000000000000..9c5443bd7686 --- /dev/null +++ b/mm/pkram_pagetable.c @@ -0,0 +1,376 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +//#include +#include + +static pgd_t *pkram_pgd; +static DEFINE_SPINLOCK(pkram_pgd_lock); + +#define set_p4d(p4dp, p4d) WRITE_ONCE(*(p4dp), (p4d)) + +#define PKRAM_PTE_BM_BYTES (PTRS_PER_PTE / BITS_PER_BYTE) +#define PKRAM_PTE_BM_MASK (PAGE_SIZE / PKRAM_PTE_BM_BYTES - 1) + +static pmd_t make_bitmap_pmd(unsigned long *bitmap) +{ + unsigned long val; + + val = __pa(ALIGN_DOWN((unsigned long)bitmap, PAGE_SIZE)); + val |= (((unsigned long)bitmap & ~PAGE_MASK) / PKRAM_PTE_BM_BYTES); + + return __pmd(val); +} + +static unsigned long *get_bitmap_addr(pmd_t pmd) +{ + unsigned long val, off; + + val = pmd_val(pmd); + off = (val & PKRAM_PTE_BM_MASK) * PKRAM_PTE_BM_BYTES; + + val = (val & PAGE_MASK) + off; + + return __va(val); +} + +int pkram_add_identity_map(struct page *page) +{ + unsigned long paddr; + unsigned long *bitmap; + unsigned int index; + struct page *pg; + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + + if (!pkram_pgd) { + spin_lock(&pkram_pgd_lock); + if (!pkram_pgd) { + pg = alloc_page(GFP_ATOMIC|__GFP_ZERO); + if (!pg) + goto nomem; + pkram_pgd = page_address(pg); + } + spin_unlock(&pkram_pgd_lock); + } + + paddr = __pa(page_address(page)); + pgd = pkram_pgd; + pgd += pgd_index(paddr); + if (pgd_none(*pgd)) { + spin_lock(&pkram_pgd_lock); + if (pgd_none(*pgd)) { + pg = alloc_page(GFP_ATOMIC|__GFP_ZERO); + if (!pg) + goto nomem; + p4d = page_address(pg); + set_pgd(pgd, __pgd(__pa(p4d))); + } + spin_unlock(&pkram_pgd_lock); + } + p4d = p4d_offset(pgd, paddr); + if (p4d_none(*p4d)) { + spin_lock(&pkram_pgd_lock); + if (p4d_none(*p4d)) { + pg = alloc_page(GFP_ATOMIC|__GFP_ZERO); + if (!pg) + goto nomem; + pud = page_address(pg); + set_p4d(p4d, __p4d(__pa(pud))); + } + spin_unlock(&pkram_pgd_lock); + } + pud = pud_offset(p4d, paddr); + if (pud_none(*pud)) { + spin_lock(&pkram_pgd_lock); + if (pud_none(*pud)) { + pg = alloc_page(GFP_ATOMIC|__GFP_ZERO); + if (!pg) + goto nomem; + pmd = page_address(pg); + set_pud(pud, __pud(__pa(pmd))); + } + spin_unlock(&pkram_pgd_lock); + } + pmd = pmd_offset(pud, paddr); + if (pmd_none(*pmd)) { + spin_lock(&pkram_pgd_lock); + if (pmd_none(*pmd)) { + if (PageTransHuge(page)) { + set_pmd(pmd, pmd_mkhuge(*pmd)); + spin_unlock(&pkram_pgd_lock); + goto done; + } + bitmap = bitmap_zalloc(PTRS_PER_PTE, GFP_ATOMIC); + if (!bitmap) + goto nomem; + set_pmd(pmd, make_bitmap_pmd(bitmap)); + } else { + BUG_ON(pmd_large(*pmd)); + bitmap = get_bitmap_addr(*pmd); + } + spin_unlock(&pkram_pgd_lock); + } else { + BUG_ON(pmd_large(*pmd)); + bitmap = get_bitmap_addr(*pmd); + } + + index = pte_index(paddr); + BUG_ON(test_bit(index, bitmap)); + set_bit(index, bitmap); + smp_mb__after_atomic(); + if (bitmap_full(bitmap, PTRS_PER_PTE)) + set_pmd(pmd, pmd_mkhuge(*pmd)); +done: + return 0; +nomem: + return -ENOMEM; +} + +void pkram_remove_identity_map(struct page *page) +{ + unsigned long *bitmap; + unsigned long paddr; + unsigned int index; + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + + /* + * pkram_pgd will be null when freeing metadata pages after a reboot + */ + if (!pkram_pgd) + return; + + paddr = __pa(page_address(page)); + pgd = pkram_pgd; + pgd += pgd_index(paddr); + if (pgd_none(*pgd)) { + WARN_ONCE(1, "PKRAM: %s: no pgd for 0x%lx\n", __func__, paddr); + return; + } + p4d = p4d_offset(pgd, paddr); + if (p4d_none(*p4d)) { + WARN_ONCE(1, "PKRAM: %s: no p4d for 0x%lx\n", __func__, paddr); + return; + } + pud = pud_offset(p4d, paddr); + if (pud_none(*pud)) { + WARN_ONCE(1, "PKRAM: %s: no pud for 0x%lx\n", __func__, paddr); + return; + } + pmd = pmd_offset(pud, paddr); + if (pmd_none(*pmd)) { + WARN_ONCE(1, "PKRAM: %s: no pmd for 0x%lx\n", __func__, paddr); + return; + } + if (PageTransHuge(page)) { + BUG_ON(!pmd_large(*pmd)); + pmd_clear(pmd); + return; + } + + if (pmd_large(*pmd)) { + spin_lock(&pkram_pgd_lock); + if (pmd_large(*pmd)) + set_pmd(pmd, __pmd(pte_val(pte_clrhuge(*(pte_t *)pmd)))); + spin_unlock(&pkram_pgd_lock); + } + + bitmap = get_bitmap_addr(*pmd); + index = pte_index(paddr); + clear_bit(index, bitmap); + smp_mb__after_atomic(); + + spin_lock(&pkram_pgd_lock); + if (!pmd_none(*pmd) && bitmap_empty(bitmap, PTRS_PER_PTE)) { + pmd_clear(pmd); + spin_unlock(&pkram_pgd_lock); + bitmap_free(bitmap); + } else { + spin_unlock(&pkram_pgd_lock); + } +} + +struct pkram_pg_state { + int (*range_cb)(unsigned long base, unsigned long size, void *private); + unsigned long start_addr; + unsigned long curr_addr; + unsigned long min_addr; + unsigned long max_addr; + void *private; + bool tracking; +}; + +#define pgd_none(a) (pgtable_l5_enabled() ? pgd_none(a) : p4d_none(__p4d(pgd_val(a)))) + +static int note_page(struct pkram_pg_state *st, unsigned long addr, bool present) +{ + if (!st->tracking && present) { + if (addr >= st->max_addr) + return 1; + /* + * addr can be < min_addr if the page straddles the + * boundary + */ + st->start_addr = max(addr, st->min_addr); + st->tracking = true; + } else if (st->tracking) { + unsigned long base, size; + int ret; + + /* Continue tracking if upper bound has not been reached */ + if (present && addr < st->max_addr) + return 0; + + addr = min(addr, st->max_addr); + + base = st->start_addr; + size = addr - st->start_addr; + st->tracking = false; + + ret = st->range_cb(base, size, st->private); + + if (addr == st->max_addr) + return 1; + else + return ret; + } + + return 0; +} + +static int walk_pte_level(struct pkram_pg_state *st, pmd_t addr, unsigned long P) +{ + unsigned long *bitmap; + int present; + int i, ret; + + bitmap = get_bitmap_addr(addr); + for (i = 0; i < PTRS_PER_PTE; i++) { + unsigned long curr_addr = P + i * PAGE_SIZE; + + if (curr_addr < st->min_addr) + continue; + present = test_bit(i, bitmap); + ret = note_page(st, curr_addr, present); + if (ret) + break; + } + + return ret; +} + +static int walk_pmd_level(struct pkram_pg_state *st, pud_t addr, unsigned long P) +{ + pmd_t *start; + int i, ret; + + start = (pmd_t *)pud_page_vaddr(addr); + for (i = 0; i < PTRS_PER_PMD; i++, start++) { + unsigned long curr_addr = P + i * PMD_SIZE; + + if (curr_addr + PMD_SIZE <= st->min_addr) + continue; + if (!pmd_none(*start)) { + if (pmd_large(*start)) + ret = note_page(st, curr_addr, true); + else + ret = walk_pte_level(st, *start, curr_addr); + } else + ret = note_page(st, curr_addr, false); + if (ret) + break; + } + + return ret; +} + +static int walk_pud_level(struct pkram_pg_state *st, p4d_t addr, unsigned long P) +{ + pud_t *start; + int i, ret; + + start = (pud_t *)p4d_page_vaddr(addr); + for (i = 0; i < PTRS_PER_PUD; i++, start++) { + unsigned long curr_addr = P + i * PUD_SIZE; + + if (curr_addr + PUD_SIZE <= st->min_addr) + continue; + if (!pud_none(*start)) { + if (pud_large(*start)) + ret = note_page(st, curr_addr, true); + else + ret = walk_pmd_level(st, *start, curr_addr); + } else + ret = note_page(st, curr_addr, false); + if (ret) + break; + } + + return ret; +} + +static int walk_p4d_level(struct pkram_pg_state *st, pgd_t addr, unsigned long P) +{ + p4d_t *start; + int i, ret; + + if (PTRS_PER_P4D == 1) + return walk_pud_level(st, __p4d(pgd_val(addr)), P); + + start = (p4d_t *)pgd_page_vaddr(addr); + for (i = 0; i < PTRS_PER_P4D; i++, start++) { + unsigned long curr_addr = P + i * P4D_SIZE; + + if (curr_addr + P4D_SIZE <= st->min_addr) + continue; + if (!p4d_none(*start)) { + if (p4d_large(*start)) + ret = note_page(st, curr_addr, true); + else + ret = walk_pud_level(st, *start, curr_addr); + } else + ret = note_page(st, curr_addr, false); + if (ret) + break; + } + + return ret; +} + +void pkram_walk_pgt(struct pkram_pg_state *st, pgd_t *pgd) +{ + pgd_t *start = pgd; + int i, ret = 0; + + for (i = 0; i < PTRS_PER_PGD; i++, start++) { + unsigned long curr_addr = i * PGDIR_SIZE; + + if (curr_addr + PGDIR_SIZE <= st->min_addr) + continue; + if (!pgd_none(*start)) + ret = walk_p4d_level(st, *start, curr_addr); + else + ret = note_page(st, curr_addr, false); + if (ret) + break; + } +} + +void pkram_find_preserved(unsigned long start, unsigned long end, void *private, int (*callback)(unsigned long base, unsigned long size, void *private)) +{ + struct pkram_pg_state st = { + .range_cb = callback, + .min_addr = start, + .max_addr = end, + .private = private, + }; + + if (!pkram_pgd) + return; + + pkram_walk_pgt(&st, pkram_pgd); +} From patchwork Tue Mar 30 21:35:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E2A0C4332E for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 87D8C619E7 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232896AbhC3V1i (ORCPT ); Tue, 30 Mar 2021 17:27:38 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50866 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232636AbhC3V1A (ORCPT ); Tue, 30 Mar 2021 17:27:00 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULP4ob145272; Tue, 30 Mar 2021 21:25:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=sK0+kX48zyKrc5O2jB5SJhTBEc3qU76zrUCMJUFl03Y=; b=xHhR6d6nQsLVaxBawJD5zMHg1xZreANZojqmD2mUUb6npmTtWt+yTcoyz+HoVrkWIttx uXJPDPRmgrYg8fpeg6BxfLImmd2Sn5tqlP+cWauVjALUn9POWoWwYCaOja2cCXf6gyp8 M+cbrpaSRsu/mFsnHiC0UvG5BmpSsCvmhkwxWK7cEEmhsz/+Im4rP6y99YYeBPrxLUEU er5F4PI4FtXSpOoYHLogD7flMFVkGqOlN1WkBHdI0KZ+oPBvMt2db3jzGnLuvB0RQKQm vt1H+pOI66W4xttL071bh8pQTFv1oFkfA4omugUd8Uj3fyde3+o/DRJvGZ4mHVP37Z96 sw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 37mad9r8en-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:46 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOasw149688; Tue, 30 Mar 2021 21:25:45 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2102.outbound.protection.outlook.com [104.47.58.102]) by userp3020.oracle.com with ESMTP id 37mac4kd23-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eFKUdrzP6VSwPriPv6mUQFJXns6GWGNM0V43jpZCElbOv+tTGjfaNiBHsV1GfH/Gu8GYwqvog5RpY6xrJCnKmZbRxGXbIFFt+fqctNLGkloxRt4AWYOAx5Sl+oSIsdJcA18z9hIyu3W8tXaLTOBskhJ5ewrDXL+cqV9Td0Mp7c4f03zFv0DqcbB2msaRVtC7wdjvX3GoZr3zxv6NdoTXTR/nuvEL0RhbrlvSi8Kf3Pjeo350Az3d7ojyfF4p+WFNIYhvLdFOtxdzfrJu4NVGeWTEMVMV8LekQzpXWEAIgejRzKfYOBPJ3OTnY30sM339OEmNdVsufHZ9KzVGl4BVDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sK0+kX48zyKrc5O2jB5SJhTBEc3qU76zrUCMJUFl03Y=; b=SUckduC53kdsmvRAR2pdpdVThl8oKiY0clG4jXQl0Ctuc0BW9f+aTGOscM7xZzgXUbskWMzn21bxj0Nvo7vRARZoZo7pll7VyOeoWCW5ewMTnG75mfnCew83NHO/ww9JBj8Khp2zzwuhfQCSBVjcFOZiEcyNSw7190Pqup60p9hyIoz8PHcFPwwdbA8TgVu56shjFAF1HXR1k77KrBf3oLmheHBBuSfZK1AgPjVmisDppMm10VgZgiYDLP9tY13V6uNbsT5DmHm2UtknZk4kMrXUQcGtnjJX08VbbXUBcnbABRfIE/SRGSm7Tolq1AX7I6ie/7qV8+GByIEpj3FkbQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sK0+kX48zyKrc5O2jB5SJhTBEc3qU76zrUCMJUFl03Y=; b=FVoanjandHOy3oIvNB3bPsR6CbqjHx7K00TawVWBV+pKetYL5d9rC5kIKhAciiEjrqmP3Y0UbJbRz550oVsrOo3pjXo69ktBm0m4yym1YBOWfrXhZI0XNk1UxTlBjMidFHS5ywx5ggmuzmRUQ/AMPJ7nBUkLhlcb7IToCY154KM= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:25:42 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:42 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 10/43] PKRAM: pass a list of preserved ranges to the next kernel Date: Tue, 30 Mar 2021 14:35:45 -0700 Message-Id: <1617140178-8773-11-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:38 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 83c2a063-b13c-4c74-5cc4-08d8f3c25f9d X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3173; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pC9EMlPEER2Pr2zpc+tiQnmD/d4d/bOTojvE5qlTGIq12L/Nfth/OCtStGM9RVrUpMAUI0LiH+BMZMmNr9NurOul/Yh7g+AP7oMt0QK3XKIbeERfPzqJtXBvlj/BRjr831NzWLmGLyZz+aF3JcYwtB2PF4Y/5YYP0RzxVzDaMpylhRhVtR1fvf2x0VEk+qUufVPKhTHxjvNkDGPxsxyDMy9FDHjBgrrDhEVmq7iP/4HcgRiTB95HbssdJf/ZEGD22zVoUtWBgvjgTadYeTQZtCouwzdlrshiOwGZOOD6VSYhSx93BkHSIquVsHt37cTT7J2UF7ANm46J21C++R0fVxzfSHII9Mmvn5AGFcIXl+0W192qfHI/06SA+XnYmHcz8WZZ0JjR5XlH7LFXse2rdW1XeS8RKz0AyB8S5tdmPYJnK+gzN0J+vvl4a45/3nO69UfMcode4Bg79KxUaAQ1l03rsV0hlbZ+tYAQG0zpur/C44IjU9xpiX6Pb6xE56z9S99cPNKk6nFAVxfM7jP3QHiNSPRGnTXsFFhX9zN136nHrHakx/sgTocYHosCKMU4hN0VcU+ZH7J6jDRxV2xPLkNWgVEnxZtmENuqbH/qsAM5WEn+mA1Aqjl6i2d5SCgpo5yNWpLkoNWcO91+Ua1HuQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(83380400001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: I2fl42ClRvZJRdAWpmpQh2iiOg+UPP82EqZVErQAj8BHuL3IE9fsl//D7uZ/xFUBiSNxC896axp4eSej34Bxk6hwKxbRqdYrgspUCs/j/XFXEIKC3sSd+MatYK5D5T+QKMdSJRdPei+GgrH+Divjo687PpksmATjmF0uatpO1g3f0UIZOcUYzZE6PogY5jOKasToUvaFdjYDLR8de7H0iI0EdvQoYodUpFR23/3nXz+JzzxPv6qSLIK51tL2OPUaRUc4PKoOI5fp4vdTqHF8zqNlYpPF1Ty22jyh1sXkcsKv36YCU83uNUxvZeQX3ExthT9iWBxyO5oyfbWVp89efUVSW2v//ExjK2AZIL18BWpcX2krU/FNO61+GzuNyA8OU2PFA8n3qOi1dH7XV1srRpekm1D1TJ7xJfZsno8cBobDBcQAlIuRlGnnJV/QpumwMwVi3B6KTwGL093dLxseJSsHpNSs78QCduta6IXlCkAuCSAudu8Ct5Z5V4KrxTNwBsPSI29QVoZPXDHjtshX8QPnr6lbMBjQFgf2DkI/9jGIYDT5ElPxjpx/hHoXBAweoaLMm0WycLIqLM45wIe8D/SsC71oX0TGK0dbndJ6dd0YX3pzjgxayeEXEyuJP5f6YBZtfXkEENOD3Dm2Dw+G76Gn9bBPJ5MrZrj7WNW1AnJLb+41KOWXTvDCzg5PR8YdKYhK+Q9NJhO73oB52vUYp6xhqnSbu9gNRIV9lttFXxGRZyWBjQvYJFKL4Hw43A8475dcG01PvjGkOOih41VB6AuMfVfyTMA9imogiFUrBZkMKTB/0V8KVxrfi/Hcc5X0WEwmGUM92SORNaFEKkloVlYOrJJSj+HZsPASgjw4am4ysvb95WehYb0QpzQ+WswqfI6qx+GAdv0dJkmLZv6Q4lERVO6twnjTvotoD6+XYg5+hvg7U3YixxR2Ulif7K0pLW2j6yPXhofh5TBz1uozjscHc8I3QQMSKa9fhP4oioixO2rrU/lM9ZBN2NDM9lFN32jFJMttVKv9g/hG7AWdIrA6+N/UXewvYXzOD7YAVcE+h6go2Azq/1F3pJ3rKW/aJkWwaOSsyOTRgZuv/+twNAwEkzg5PYVmw+VFRdHmxzDzmO89UqSNDCyc5OxA57jejElm+L4lI3muBVgVdbvzzNpsHV+iJd81NXPM2SIF/KJLUSb9rqN34xsSSFTR+rynjuj7wgc8ymPxjSIoeMwxNpUw5e22DixBjnM0AwASW+HUC7WWQyfE9Smt1duWSZ+OBNA+AYJXjxHc8nx2lTBzgK5/pv+PzeRfBRJP2KGwWQIibQpiufHWwDnqgdx4KB3f X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 83c2a063-b13c-4c74-5cc4-08d8f3c25f9d X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:42.7757 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YOZ0qbY8YHHG0lleKe3U0aWcg4g6rUT8ck1nFYiAEUv94sCc0cZVsgp7xtIMtwPEMo+glilq3s44tj68iVCOgpMBVdR1whJ7Zcf7h2WZCG4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: PQ_3p6rZNQQPFETrM5UOYB9qLkQavGnj X-Proofpoint-GUID: PQ_3p6rZNQQPFETrM5UOYB9qLkQavGnj X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org In order to build a new memblock reserved list during boot that includes ranges preserved by the previous kernel, a list of preserved ranges is passed to the next kernel via the pkram superblock. The ranges are stored in ascending order in a linked list of pages. A more complete memblock list is not prepared to avoid possible conflicts with changes in a newer kernel and to avoid having to allocate a contiguous range larger than a page. Signed-off-by: Anthony Yznaga --- mm/pkram.c | 183 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 176 insertions(+), 7 deletions(-) diff --git a/mm/pkram.c b/mm/pkram.c index a9e6cd8ca084..4cfa236a4126 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -86,6 +86,20 @@ struct pkram_node { #define PKRAM_LOAD 2 #define PKRAM_ACCMODE_MASK 3 +struct pkram_region { + phys_addr_t base; + phys_addr_t size; +}; + +struct pkram_region_list { + __u64 prev_pfn; + __u64 next_pfn; + + struct pkram_region regions[0]; +}; + +#define PKRAM_REGIONS_LIST_MAX \ + ((PAGE_SIZE-sizeof(struct pkram_region_list))/sizeof(struct pkram_region)) /* * The PKRAM super block contains data needed to restore the preserved memory * structure on boot. The pointer to it (pfn) should be passed via the 'pkram' @@ -98,13 +112,20 @@ struct pkram_node { */ struct pkram_super_block { __u64 node_pfn; /* first element of the node list */ + __u64 region_list_pfn; + __u64 nr_regions; }; +static struct pkram_region_list *pkram_regions_list; +static int pkram_init_regions_list(void); +static unsigned long pkram_populate_regions_list(void); + static unsigned long pkram_sb_pfn __initdata; static struct pkram_super_block *pkram_sb; extern int pkram_add_identity_map(struct page *page); extern void pkram_remove_identity_map(struct page *page); +extern void pkram_find_preserved(unsigned long start, unsigned long end, void *private, int (*callback)(unsigned long base, unsigned long size, void *private)); /* * For convenience sake PKRAM nodes are kept in an auxiliary doubly-linked list @@ -862,21 +883,48 @@ static void __pkram_reboot(void) struct page *page; struct pkram_node *node; unsigned long node_pfn = 0; + unsigned long rl_pfn = 0; + unsigned long nr_regions = 0; + int err = 0; - list_for_each_entry_reverse(page, &pkram_nodes, lru) { - node = page_address(page); - if (WARN_ON(node->flags & PKRAM_ACCMODE_MASK)) - continue; - node->node_pfn = node_pfn; - node_pfn = page_to_pfn(page); + if (!list_empty(&pkram_nodes)) { + err = pkram_add_identity_map(virt_to_page(pkram_sb)); + if (err) { + pr_err("PKRAM: failed to add super block to pagetable\n"); + goto done; + } + list_for_each_entry_reverse(page, &pkram_nodes, lru) { + node = page_address(page); + if (WARN_ON(node->flags & PKRAM_ACCMODE_MASK)) + continue; + node->node_pfn = node_pfn; + node_pfn = page_to_pfn(page); + } + err = pkram_init_regions_list(); + if (err) { + pr_err("PKRAM: failed to init regions list\n"); + goto done; + } + nr_regions = pkram_populate_regions_list(); + if (IS_ERR_VALUE(nr_regions)) { + err = nr_regions; + pr_err("PKRAM: failed to populate regions list\n"); + goto done; + } + rl_pfn = page_to_pfn(virt_to_page(pkram_regions_list)); } +done: /* * Zero out pkram_sb completely since it may have been passed from * the previous boot. */ memset(pkram_sb, 0, PAGE_SIZE); - pkram_sb->node_pfn = node_pfn; + if (!err && node_pfn) { + pkram_sb->node_pfn = node_pfn; + pkram_sb->region_list_pfn = rl_pfn; + pkram_sb->nr_regions = nr_regions; + } } static int pkram_reboot(struct notifier_block *notifier, @@ -952,3 +1000,124 @@ static int __init pkram_init(void) return 0; } module_init(pkram_init); + +static int count_region_cb(unsigned long base, unsigned long size, void *private) +{ + unsigned long *nr_regions = (unsigned long *)private; + + (*nr_regions)++; + return 0; +} + +static unsigned long pkram_count_regions(void) +{ + unsigned long nr_regions = 0; + + pkram_find_preserved(0, PHYS_ADDR_MAX, &nr_regions, count_region_cb); + + return nr_regions; +} + +/* + * To faciliate rapidly building a new memblock reserved list during boot + * with the addition of preserved memory ranges a regions list is built + * before reboot. + * The regions list is a linked list of pages with each page containing an + * array of preserved memory ranges. The ranges are stored in each page + * and across the list in address order. A linked list is used rather than + * a single contiguous range to mitigate against the possibility that a + * larger, contiguous allocation may fail due to fragmentation. + * + * Since the pages of the regions list must be preserved and the pkram + * pagetable is used to determine what ranges are preserved, the list pages + * must be allocated and represented in the pkram pagetable before they can + * be populated. Rather than recounting the number of regions after + * allocating pages and repeating until a precise number of pages are + * are allocated, the number of pages needed is estimated. + */ +static int pkram_init_regions_list(void) +{ + struct pkram_region_list *rl; + unsigned long nr_regions; + unsigned long nr_lpages; + struct page *page; + + nr_regions = pkram_count_regions(); + + nr_lpages = DIV_ROUND_UP(nr_regions, PKRAM_REGIONS_LIST_MAX); + nr_regions += nr_lpages; + nr_lpages = DIV_ROUND_UP(nr_regions, PKRAM_REGIONS_LIST_MAX); + + for (; nr_lpages; nr_lpages--) { + page = pkram_alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) + return -ENOMEM; + rl = page_address(page); + if (pkram_regions_list) { + rl->next_pfn = page_to_pfn(virt_to_page(pkram_regions_list)); + pkram_regions_list->prev_pfn = page_to_pfn(page); + } + pkram_regions_list = rl; + } + + return 0; +} + +struct pkram_regions_priv { + struct pkram_region_list *curr; + struct pkram_region_list *last; + unsigned long nr_regions; + int idx; +}; + +static int add_region_cb(unsigned long base, unsigned long size, void *private) +{ + struct pkram_regions_priv *priv; + struct pkram_region_list *rl; + int i; + + priv = (struct pkram_regions_priv *)private; + rl = priv->curr; + i = priv->idx; + + if (!rl) { + WARN_ON(1); + return 1; + } + + if (!i) + priv->last = priv->curr; + + rl->regions[i].base = base; + rl->regions[i].size = size; + + priv->nr_regions++; + i++; + if (i == PKRAM_REGIONS_LIST_MAX) { + u64 next_pfn = rl->next_pfn; + + if (next_pfn) + priv->curr = pfn_to_kaddr(next_pfn); + else + priv->curr = NULL; + + i = 0; + } + priv->idx = i; + + return 0; +} + +static unsigned long pkram_populate_regions_list(void) +{ + struct pkram_regions_priv priv = { .curr = pkram_regions_list }; + + pkram_find_preserved(0, PHYS_ADDR_MAX, &priv, add_region_cb); + + /* + * Link the first node to the last populated one. + */ + pkram_regions_list->prev_pfn = page_to_pfn(virt_to_page(priv.last)); + + return priv.nr_regions; +} From patchwork Tue Mar 30 21:35:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 053FFC433DB for ; Tue, 30 Mar 2021 21:29:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DB171619CB for ; Tue, 30 Mar 2021 21:28:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232659AbhC3V21 (ORCPT ); Tue, 30 Mar 2021 17:28:27 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:51588 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232636AbhC3V1y (ORCPT ); Tue, 30 Mar 2021 17:27:54 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPGju145350; Tue, 30 Mar 2021 21:25:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=YqvghoD5svYRKsQeQ1oYy1SsBLqv8CMnuVd9yKWpWRs=; b=pMr0nHEfZI0+bqUiZeOGBhnFYDRLv+ZCrkgctsYWCThM+fDfwxxaiCD2zMzhyswPHJCU vrg5StC5N6mcKH5baQae5P3GTS2h8eoH9qoTxIExjfkW4HjZFJAG8TWCupkndNYjzdr+ h1CRhJROvFmWQXDA7FIzhOd3NKcWZFfxRxidfD/wJgEfwDBssXf/ZM2Bcyj2mrb1zEHn bzm9qZ6b0M6Vxamn+MBiQWi2G8U7aBM8LWJsWA8nFuMVZ+jioXitgZDNUUHwzOo8TVNY LobnZye19NwmiAS5TGwf7zZkBeAjxw91gqRomzxuKNUhx1mu1W4v2/PTfQH/LMcqwY5V gg== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 37mad9r8ev-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:51 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOmI5184039; Tue, 30 Mar 2021 21:25:50 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2104.outbound.protection.outlook.com [104.47.58.104]) by aserp3020.oracle.com with ESMTP id 37mac7u47e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VFhufleDNH8PQPoyUSVwjzwx4bS/bRr7OUaIz6XFloo5zhoMnB2quT3VUV8GADoZnkqSD3eGAUbYW8htOjv9fYd48N3XyFF2UIrK8nfFk3d9p4I/uBhbIpBEOvJchNSA4Geb5LLbutFYaSkgyemEA4jZkEt/h1femlxJPVOKaH9BlOR/T7C9o3UELPBwhDKHjmsX9y91h2PQCBv2wLapk4vXl/Ej1vD5eO6vTIh94t6sZ4D0FBslKnokYswJJPPEiF5A7Pd1q5CFy5F6qbLv0eTfBxOQSfOBmnHKlF6slY4nUAkMFAoDd5WuDG3yUqiDXMbL4g79A1PtVzTQihyrfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YqvghoD5svYRKsQeQ1oYy1SsBLqv8CMnuVd9yKWpWRs=; b=co39iqGaS2XuNGvTkFN8Qf7zFQSNwmQlk8l8wkEvwlUuwoji0Bg+lk7Nz5Z7K/PxB3vphpCaHX1i1Eor/UCnaeADfZzshVMl6KO/e6uOL3+MnJaS9oIODmvg5cUx+LsXw2//VDV39v27xeiSZC6UkuOnD1lAMtzVBrdbwy2HxP1o9grHp7DPVNXPZqPMVIuF87chWwuBHZI+diMLvFvz61rDrrknT2FYLPfiM37ioINw46ayuQULdp3rKBdQmVqPto/UvYRoeJr/PBdLNmuJ6eT75CmEC7xZnoN+pskT/9t5Wlw3px49JzxeecQzH1mehEZcJWPX/wLhPNNIblvXVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YqvghoD5svYRKsQeQ1oYy1SsBLqv8CMnuVd9yKWpWRs=; b=Q+acBETG24T18065kDDQZkSHYF/xidUcs/LfYyFxgwM8mVU+drJgswg3S/YLwSzznj/+5sQmgKrMibCqoSJ9OMwNCJ6IYTLfk4eyO5X19wgoPyOb3thtDqGQVCdp4/mA5Jareq+R88rXUvvUv6d8Jyx/FaTCOaWBHpGr/2Gft5s= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:25:47 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:47 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 11/43] PKRAM: prepare for adding preserved ranges to memblock reserved Date: Tue, 30 Mar 2021 14:35:46 -0700 Message-Id: <1617140178-8773-12-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:42 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3fa27c22-6c96-4f6c-54b3-08d8f3c26249 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cem83xutk/D6Annw7qyahQcZxzM78bunpmZ38SyZ4urHvAZtE7lprNTNJAH1wusVYCGQIvNifg+p0cY6x22YXy1AeS+v05NM4X42EQLtC1DLEsdPu82ue+yBvirExkSK6hksVaMT2iGJGelIH8x1cFXGsOwZCDlNNmr8PhvFxchj4YY2+VY3DS03YudGoENAn0uHcjOe4NHWFLVecoQVTp75fyPAoqtL4LSm/T1+EUjtSKdDkRs+/mM2pQmXVAywLFdnkpZtZaZpWCsTl68cQ32zHU+iMMTMTe6QxWYBvuMek8zp6xSFUKCn3xwi1xfwFnlHU2rHpzeeH44mf9aO24mCuGU6g7/GCI3YBJUanEOOjrVefmNoJQ7M4lD5hONh3ZGbtuIhnQRauuu6Tf7SErjKDNmntNKhV6woHGnZ2dQ3A0mnxrd7LagRFxvqRG7Q7Gzkl6kQMu+3pFhkhFUAGFKcjCVyAHWPr5coX3nOOEQxvEVndOKUPvLaTD5dpayXKKEh3N/51gVR9THdj2O33kelmTGfxlcdTceh3vr09AcFHnPh9QTLgB4ertIlf3u8BQJECWk6JyXQ3HoH2cspVDtKEUHqgk4PyHlFreFcTs5xyUgfohgEmUXUVqIZ271BX91j7uA5BS4tji71+pxhlQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(66556008)(316002)(83380400001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: mwg0cQ0j0FDEZw1mVtAmOf5kKZxy4BYWpY+DAoeeLBo/4jkr/dvLQthUJJ9Cx1FqAT76vmCmmosWkraxEVQEX6yllY2HVUAUfJkLB4pWGm773UgEBymft3CoOI01RYKvKTolUAw2P9VfpqqZOuV92S1PUqBJUsN7VkjdsaZornf6Dd7ZMKfw0PFuMj8933kib6tsu4YlyQWqIQJyDQVNqHIjJ2XNCxPY+SrHko4B2MCvk7vQaZDziq9m7/tiMw0MiXssG4UDh2IikaZjGRWh/yx3Y1A9MCKGOb8M6rqMRjKBY/DFKJRfXRotjVt1mG5/VnhjW+TSOGZzv6MytdDhmf6MOcfaodELkejUdeujLArNSW6VsB6sG5chtT2ZODC5mIr2qGrhnVSvxr2F1iGyocumKMX0vSUG/e8cmHHC4JMxsnHQZ3Z9dsbbEQKimPBm4SrPmVEQvkZc3zyYz6ZkGy5WeEALyw2iKsEL5LeGeEzSJjC9Y6jj8IM+DONPZNl+ISRYP9uXXYlxN2IiJp8uDMmeAI+KA/8spwU1Dzj8cuXUiiWvlNXJw/kTxz7nYnCPLV23J90JJQm2Cv82L/MO2Na90EIp985WVFMG9xvztJQopHXCiMSEHmOJqLX7t+Zaq0vrLePABoPBGGblYA+4AsDRbZoJs8PwZxDRga9YQoDXXgAvw7/myu8EbPmGubWQm9D7b32BcB8ENJZIHPgzjO92k/PkGKBeeFDgxFe5EJKyrVnp1vaoI3Du3KrWqpmHQW4ATkoE21WG8jOFz3itHDq90bwizOW7t+nP3NJ8TdC0nrT7vL+qISRLHAF0o55yeU/eb9pO0b66M8IXE8tnfB0xsCQW4S7BysB4zd/9vNk6IkQoGYo6zCjLH3jh9WsFjiL8vbi7FTPZL2D1Su0y0UYsDe4z66kMZti6pSHq+TzV7GVBY07rj0kD66hYjYoHwCcH6AJUKgOONUvCBIYCQ/Jwtsy4uXbXP9CcaRHlAlBsiJ0F53aJsSThgRUtYd9+4ZKTELPoBBDg/OeYcCPVEuYp3qOANdt5tx/noqDy/S7fAKS5w25H0O1IX6CA0Nx6qNOMLLAK8cK6wyFHoJMy72j2/KBnF3rTde8TgfftDYeHASbvIPO01hzfbr4bzibPfYwoGMK4C+yAdP7EZtpGlVn8v5QgT/eKL/hAJa2s8oL9VHH6AAq59H+EMpLQs+XtICFMD1Uv294J2MBnFtfGmGduEAw1LaoBCtY6pmeF+Ac6Sxs/6ROfcNoa4LfpHhRwH3I7omHjlDrxctmD+p9eaXDCs8Zd8XMe4l3rGW05UBEldLQtNyCeSKpFhKvgfaYO X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3fa27c22-6c96-4f6c-54b3-08d8f3c26249 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:47.4891 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CVhVVJ18svfI3t83mVZD/eEZ2H1L3XcV/xkpEOLK7G1bxZ4usQInAEqtHrYCUtftJpGCuMVG7MThC+wumHt9meX4eW4GI9qSgZjBrWk9xMw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: zFz7AVL2HY6fF-orVHxIb7fjRV8vBO6M X-Proofpoint-GUID: zFz7AVL2HY6fF-orVHxIb7fjRV8vBO6M X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Calling memblock_reserve() repeatedly to add preserved ranges is inefficient and risks clobbering preserved memory if the memblock reserved regions array must be resized. Instead, calculate the size needed to accomodate the preserved ranges, find a suitable range for a new reserved regions array that does not overlap any preserved range, and populate it with a new, merged regions array. Signed-off-by: Anthony Yznaga --- mm/pkram.c | 241 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 241 insertions(+) diff --git a/mm/pkram.c b/mm/pkram.c index 4cfa236a4126..b4a14837946a 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include @@ -1121,3 +1122,243 @@ static unsigned long pkram_populate_regions_list(void) return priv.nr_regions; } + +struct pkram_region *pkram_first_region(struct pkram_super_block *sb, struct pkram_region_list **rlp, int *idx) +{ + WARN_ON(!sb); + WARN_ON(!sb->region_list_pfn); + + if (!sb || !sb->region_list_pfn) + return NULL; + + *rlp = pfn_to_kaddr(sb->region_list_pfn); + *idx = 0; + + return &(*rlp)->regions[0]; +} + +struct pkram_region *pkram_next_region(struct pkram_region_list **rlp, int *idx) +{ + struct pkram_region_list *rl = *rlp; + int i = *idx; + + i++; + if (i >= PKRAM_REGIONS_LIST_MAX) { + if (!rl->next_pfn) { + pr_err("PKRAM: %s: no more pkram_region_list pages\n", __func__); + return NULL; + } + rl = pfn_to_kaddr(rl->next_pfn); + *rlp = rl; + i = 0; + } + *idx = i; + + if (rl->regions[i].size == 0) + return NULL; + + return &rl->regions[i]; +} + +struct pkram_region *pkram_first_region_topdown(struct pkram_super_block *sb, struct pkram_region_list **rlp, int *idx) +{ + struct pkram_region_list *rl; + + WARN_ON(!sb); + WARN_ON(!sb->region_list_pfn); + + if (!sb || !sb->region_list_pfn) + return NULL; + + rl = pfn_to_kaddr(sb->region_list_pfn); + if (!rl->prev_pfn) { + WARN_ON(1); + return NULL; + } + rl = pfn_to_kaddr(rl->prev_pfn); + + *rlp = rl; + + *idx = (sb->nr_regions - 1) % PKRAM_REGIONS_LIST_MAX; + + return &rl->regions[*idx]; +} + +struct pkram_region *pkram_next_region_topdown(struct pkram_region_list **rlp, int *idx) +{ + struct pkram_region_list *rl = *rlp; + int i = *idx; + + if (i == 0) { + if (!rl->prev_pfn) + return NULL; + rl = pfn_to_kaddr(rl->prev_pfn); + *rlp = rl; + i = PKRAM_REGIONS_LIST_MAX - 1; + } else + i--; + + *idx = i; + + return &rl->regions[i]; +} + +/* + * Use the pkram regions list to find an available block of memory that does + * not overlap with preserved pages. + */ +phys_addr_t __init find_available_topdown(phys_addr_t size) +{ + phys_addr_t hole_start, hole_end, hole_size; + struct pkram_region_list *rl; + struct pkram_region *r; + phys_addr_t addr = 0; + int idx; + + hole_end = memblock.current_limit; + r = pkram_first_region_topdown(pkram_sb, &rl, &idx); + + while (r) { + hole_start = r->base + r->size; + hole_size = hole_end - hole_start; + + if (hole_size >= size) { + addr = memblock_find_in_range(hole_start, hole_end, + size, PAGE_SIZE); + if (addr) + break; + } + + hole_end = r->base; + r = pkram_next_region_topdown(&rl, &idx); + } + + if (!addr) + addr = memblock_find_in_range(0, hole_end, size, PAGE_SIZE); + + return addr; +} + +int __init pkram_create_merged_reserved(struct memblock_type *new) +{ + unsigned long cnt_a; + unsigned long cnt_b; + long i, j, k; + struct memblock_region *r; + struct memblock_region *rgn; + struct pkram_region *pkr; + struct pkram_region_list *rl; + int idx; + unsigned long total_size = 0; + unsigned long nr_preserved = 0; + + cnt_a = memblock.reserved.cnt; + cnt_b = pkram_sb->nr_regions; + + i = 0; + j = 0; + k = 0; + + pkr = pkram_first_region(pkram_sb, &rl, &idx); + if (!pkr) + return -EINVAL; + while (i < cnt_a && j < cnt_b && pkr) { + r = &memblock.reserved.regions[i]; + rgn = &new->regions[k]; + + if (r->base + r->size <= pkr->base) { + *rgn = *r; + i++; + } else if (pkr->base + pkr->size <= r->base) { + rgn->base = pkr->base; + rgn->size = pkr->size; + memblock_set_region_node(rgn, MAX_NUMNODES); + + nr_preserved += (rgn->size >> PAGE_SHIFT); + pkr = pkram_next_region(&rl, &idx); + j++; + } else { + pr_err("PKRAM: unexpected overlap:\n"); + pr_err("PKRAM: reserved: base=%pa,size=%pa,flags=0x%x\n", &r->base, &r->size, (int)r->flags); + pr_err("PKRAM: pkram: base=%pa,size=%pa\n", &pkr->base, &pkr->size); + return -EBUSY; + } + total_size += rgn->size; + k++; + } + + while (i < cnt_a) { + r = &memblock.reserved.regions[i]; + rgn = &new->regions[k]; + + *rgn = *r; + + total_size += rgn->size; + i++; + k++; + } + while (j < cnt_b && pkr) { + rgn = &new->regions[k]; + rgn->base = pkr->base; + rgn->size = pkr->size; + memblock_set_region_node(rgn, MAX_NUMNODES); + + nr_preserved += (rgn->size >> PAGE_SHIFT); + total_size += rgn->size; + pkr = pkram_next_region(&rl, &idx); + j++; + k++; + } + + WARN_ON(cnt_a + cnt_b != k); + new->cnt = cnt_a + cnt_b; + new->total_size = total_size; + + return 0; +} + +/* + * Reserve pages that belong to preserved memory. This is accomplished by + * merging the existing reserved ranges with the preserved ranges into + * a new, sufficiently sized memblock reserved array. + * + * This function should be called at boot time as early as possible to prevent + * preserved memory from being recycled. + */ +int __init pkram_merge_with_reserved(void) +{ + struct memblock_type new; + unsigned long new_max; + phys_addr_t new_size; + phys_addr_t addr; + int err; + + /* + * Need space to insert one more range into memblock.reserved + * without memblock_double_array() being called. + */ + if (memblock.reserved.cnt == memblock.reserved.max) { + WARN_ONCE(1, "PKRAM: no space for new memblock list\n"); + return -ENOMEM; + } + + new_max = memblock.reserved.max + pkram_sb->nr_regions; + new_size = PAGE_ALIGN(sizeof (struct memblock_region) * new_max); + + addr = find_available_topdown(new_size); + if (!addr || memblock_reserve(addr, new_size)) + return -ENOMEM; + + new.regions = __va(addr); + new.max = new_max; + err = pkram_create_merged_reserved(&new); + if (err) + return err; + + memblock.reserved.cnt = new.cnt; + memblock.reserved.max = new.max; + memblock.reserved.total_size = new.total_size; + memblock.reserved.regions = new.regions; + + return 0; +} From patchwork Tue Mar 30 21:35:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 955C2C433EC for ; Tue, 30 Mar 2021 21:28:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E82E6196C for ; Tue, 30 Mar 2021 21:28:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232801AbhC3V1a (ORCPT ); Tue, 30 Mar 2021 17:27:30 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49766 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232607AbhC3V0z (ORCPT ); Tue, 30 Mar 2021 17:26:55 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOrrd130897; Tue, 30 Mar 2021 21:25:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=8LARW2khMGLrdTOnD9jJjXqihyUkWdkeHGpsKP/eAiY=; b=xP6hmp6RhrpHZ6/Fw2fUfIkhDD4VWBi7IqyyNUFVUlWspsLltlsfioAgmZ6xVmEC2ZDa ee2dD1D1t9jeraEix2dIqz2U3KAFZ9H5piq26npTDVOgCe6GXerQfC/ZiEBJRR+1faA2 jq8NUJHWRwKKkO2XzIgx6OwJvccHNumenQLWcapqu5BnTb5trAt5KnEZnNnY2GpjOMNk u4D30kuU49q8Ai88goUsNDcBl8fenfF8lHWHxBqxddMwNWKcH9elxLwVXg3ui4k0caaR Gb/vpASOhJkz6tBw8xgxfJqRgQP1hsxMMSJiNKBHnWdNJUk6d8CljeWEiFByJ2ZJRFjw cQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 37mabqr8nb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:55 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPSoe105808; Tue, 30 Mar 2021 21:25:54 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2104.outbound.protection.outlook.com [104.47.58.104]) by userp3030.oracle.com with ESMTP id 37mabkbc4f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:54 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NxuM+i28qYu0lNC13truAhC/2Qha4JHQuNzftYRWia2m7DC0s8Sx9lNAV7lkLDiVecH/HmQPezLXaNmaSfZZWsG/XW9+Qq2xGKh5Pn6wljCiZUZMbwdYXV3279zBsyhWIgTdCDBusITd/zK7SNcCCxemYTy/ftSvK5ynXLYcdIeaQqzRtC45DL3UOC0qHIxubjgS6ksUEt1b9Gc86yamSiAdGVFKcv0G8nH9PZskLKXvrs+U67ZMiqgmbYg6v1IEcKlsV2vMbRkRcJ708Xso2W1DaYi/IxGfssFx0gHsg7W2pMkM4dtIAucmePl6N9ru0qRpdWtNC0z+9URR8p9/aA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8LARW2khMGLrdTOnD9jJjXqihyUkWdkeHGpsKP/eAiY=; b=IDy5E3XQo4hGcen0In+smA3eQECinW5Z/OT3givZyQJRb0M+p97NNBB8T4w6dCznls0AZRps80/RWvM++XoM2quKgeOibro45pPKumlOScCv4m3zSS3SWBcaep/Q1FBxD1KTyL2ULp7FevLm5kfZASns/fY4nOEy4fe/IHOlKCkPYAt1xh+LybkttudeBnpdPd1OtXIYKXhqsMGnaeIiSSbAnTSKKDjfRGWXzE+kzr8KQVq2NT+viDb9NfiWQNw35N9T/Tw+8+IiAmlw+JDAewExh3Xa/fBZGhqbo6IUOaVjpiRiSjuyq4kO5Tq+nwWEFlQWXIDCs/b4i0AAlPEirA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8LARW2khMGLrdTOnD9jJjXqihyUkWdkeHGpsKP/eAiY=; b=aI5RWvYOjlCWyj3tiAiARTrKTx/0DbCu3KC4dZ1hly5yyziz9GvQwzovd+puwQE3z1SdfF6cNiEghe1xS48+eXSmg43di3s1gjLYjxZNOBEgTL4kTdUGdO3mPkYSMtbe2DvhXgvKp8ZrGTjRpddp0rrQCwvciOn789ClxvoXBbI= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:25:52 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:52 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 12/43] mm: PKRAM: reserve preserved memory at boot Date: Tue, 30 Mar 2021 14:35:47 -0700 Message-Id: <1617140178-8773-13-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:47 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4175c792-06fa-40e4-7b68-08d8f3c26503 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dLhUmqWbA7mp1AiI4HKY/AJeK+7xXN9d+sIBOEvgztsvgrbgzKMORpFbEAkjXdbKbvOCHtfzExztUrtU0YXThKp/opRU9fdxos11ORrJd4x/B8OcnuIZTGJ6Sam6BSCA8SFREyoTim9AKQGw0nHRZQGtqhc6cPNGgu6zd2z8ZaZ9SoBubyWWxUZC3w6DMaUHxWOHGDjn/v0moCV8gflt+uWc4p0EgnapLe96T3Aav3p2i/T3NrsUhCox8kQMON6td3bbtUAe7CeO4jcireheod3smG0cXOaF/TmzBg6dtNP5SXyunD5DQ8IDiVKImsB9LH0Ze7lPPE1sJGQYZtNWr38ZhRx4WP8d+iIfg7O+dDumkaMEq+JpI2/Uxw1Lwv2O8PF3FmLS678vFICo6baLGqCMhDGZgpa5DrnY6/Lf8dctmMJUzLZYBxca2v5llv67mfvXP0pEff+OVraE7gmLfDwrQS+MOxzZdih8pGPjDmQZ9DO2wtn+vFOcez55zxDG5++lfcu4IlJP40ekMG7Wf5agd3nfjqXhSkBY69y01DPIaoNfatQdJDiws9MIDIqVKK77O+U78cbm4FFNBmSluOPeoafDTBT1uIJa6qazyfF4R3wPznoedrfKUbvbByxcYoHemtmNOFnsO47YG8rs2A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(66556008)(316002)(83380400001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: CcnFwkkKQulQ3VBc4ngTXXRwG2fWco7C/jWX4v3c/bDt9XMjSDZ/8svRebb9bRSeBcAXdnCRSFl6qK881oOAvW5DYvUOqGLzAStF8xLeVpOGF2Czbsayhc5KUhFYWMewraV3gffT46mi7lajG+xNevnTLEeKqpwGMG+ztMg0ZNhZuSqwMUc4BTS6b02en/OaEoJBtPr1+nzl+3F+OAaIAE4ZJ+d2/t4ZpDMqrTp8nnmuk21qnZO3C5IfDouelQXRXjTHvJQSBSC/c1idSpDAABvV6dFiKYCrYztKSk+7vTNsbL1gwTXGusQPjWrS7WLo9tqqO8D9Mgqpc9j8Wf7ea06fxzGUeGcdiXZwWao115+ODOXTtk4puABr7eHkUjlyVYoZ7d+bxWCsjEGpUAfq40lPYqjrCAe0IZ+W2s8df3mFAdZfk5IZoI7rVKjpZ1AwgWspYYiX0jfBrpsUeZSLdQB1YMABkPo/nB/V/LENuZS35WIoc8E6agZmcHv2hn9/mEqoMQWp5W7JPKydDYfLCy8E+KVa/Z4pXxht120KPZCw/2sWdl6y2wz3JOa/UZjKYXmgelD/7gpan8sCwl5MiXQ/aFfE8cc4U3Ymv7zZ5Xhp4a/97ICoXVKjwbdS7wLliKtS6Rss3c4a/GeCABCM8iM0qokRP3WcYi7lH9UwrkkRNQsNSPSNr0Pt2xsfXB7WaS0NMq/RdruBnAiEwYQyRMtKYTJr/fxzQrfetOBRwSuOMoalNp2m1IPy1P4lKB4FDefTAWH7etYIF7qrCX9VFgrvKcvO9cCX9+9U+R5OuCANM+fbyhpgPsXeNemmdT14QwJVc5twnuiXK6zY52VckX9pIUumtGfTqi3ibKxfKuVRNhCqVT07teEHPcDPKJO4cv2AbWzK+jBxXLdyG/IpRNRgqtKGAUKeGqWLHFah5SGEcFGiTLpWllzOGKDyIlUGv0TN1rXCTdz6a37lPnH6UEcK1WJvxeUfBPlD2yh2uAynr27zVOCitCPth2w9IktJElmqi/K162WbuKyknrq0BP4LiizjftH5cBv1zns3znIj9UeNBhwWa6oKxfYazMlvlG/PfeyqS83aYCFZx9/3RCOD2XFKzrg+i2dBo9JmyMfokfHolga58ji3Wg+ZRAtt8qfRY0jUWP043cR4EOwoxmot+92e9d1L1pKOJz2vgZq9RQ11jpYpssQvuf+LL1ZxDc8SAZf9il4ld4mdemC52LklTCchy+Vz/eTBAGxNJ+MzXQV03D/Rgjs0mDduTWUajUuI2zAus6MmTJ3wqB0QL7R9o6NDxUriLEH4B8WwIWnlBlPcEXlnUBXqsyvZEkNR X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4175c792-06fa-40e4-7b68-08d8f3c26503 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:51.8436 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SKIO5g6PcEdfDdL/jA1B20nLeNenRtaBpW4PrsefM1b4Fn+HK0ltPylv0BzFHe3GZSQP1G+c66gW0Z4aHPIdR6pdxGFtracTOFLa0mik1gc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: LuvO8YRODka0zaFDEapQ0Ff8V0Jyr2VW X-Proofpoint-GUID: LuvO8YRODka0zaFDEapQ0Ff8V0Jyr2VW X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Keep preserved pages from being recycled during boot by adding them to the memblock reserved list during early boot. If memory reservation fails (e.g. a region has already been reserved), all preserved pages are dropped. Signed-off-by: Anthony Yznaga --- arch/x86/kernel/setup.c | 3 ++ arch/x86/mm/init_64.c | 2 ++ include/linux/pkram.h | 8 ++++++ mm/pkram.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 87 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index d883176ef2ce..fbd85964719d 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -1146,6 +1147,8 @@ void __init setup_arch(char **cmdline_p) initmem_init(); dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT); + pkram_reserve(); + if (boot_cpu_has(X86_FEATURE_GBPAGES)) hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT); diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index b5a3fa4033d3..8efb2fb2a88b 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -1293,6 +1294,7 @@ void __init mem_init(void) after_bootmem = 1; x86_init.hyper.init_after_bootmem(); + totalram_pages_add(pkram_reserved_pages); /* * Must be done after boot memory is put on freelist, because here we * might set fields in deferred struct pages that have not yet been diff --git a/include/linux/pkram.h b/include/linux/pkram.h index 4f95d4fb5339..8d3d780d9fe1 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -99,4 +99,12 @@ int pkram_prepare_save(struct pkram_stream *ps, const char *name, ssize_t pkram_write(struct pkram_access *pa, const void *buf, size_t count); size_t pkram_read(struct pkram_access *pa, void *buf, size_t count); +#ifdef CONFIG_PKRAM +extern unsigned long pkram_reserved_pages; +void pkram_reserve(void); +#else +#define pkram_reserved_pages 0UL +static inline void pkram_reserve(void) { } +#endif + #endif /* _LINUX_PKRAM_H */ diff --git a/mm/pkram.c b/mm/pkram.c index b4a14837946a..03731bb6af26 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -135,6 +135,8 @@ struct pkram_super_block { static LIST_HEAD(pkram_nodes); /* linked through page::lru */ static DEFINE_MUTEX(pkram_mutex); /* serializes open/close */ +unsigned long __initdata pkram_reserved_pages; + /* * The PKRAM super block pfn, see above. */ @@ -144,6 +146,59 @@ static int __init parse_pkram_sb_pfn(char *arg) } early_param("pkram", parse_pkram_sb_pfn); +static void * __init pkram_map_meta(unsigned long pfn) +{ + if (pfn >= max_low_pfn) + return ERR_PTR(-EINVAL); + return pfn_to_kaddr(pfn); +} + +int pkram_merge_with_reserved(void); +/* + * Reserve pages that belong to preserved memory. + * + * This function should be called at boot time as early as possible to prevent + * preserved memory from being recycled. + */ +void __init pkram_reserve(void) +{ + int err = 0; + + if (!pkram_sb_pfn) + return; + + pr_info("PKRAM: Examining preserved memory...\n"); + + /* Verify that nothing else has reserved the pkram_sb page */ + if (memblock_is_region_reserved(PFN_PHYS(pkram_sb_pfn), PAGE_SIZE)) { + err = -EBUSY; + goto out; + } + + pkram_sb = pkram_map_meta(pkram_sb_pfn); + if (IS_ERR(pkram_sb)) { + err = PTR_ERR(pkram_sb); + goto out; + } + /* An empty pkram_sb is not an error */ + if (!pkram_sb->node_pfn) { + pkram_sb = NULL; + goto done; + } + + err = pkram_merge_with_reserved(); +out: + if (err) { + pr_err("PKRAM: Reservation failed: %d\n", err); + WARN_ON(pkram_reserved_pages > 0); + pkram_sb = NULL; + return; + } + +done: + pr_info("PKRAM: %lu pages reserved\n", pkram_reserved_pages); +} + static inline struct page *pkram_alloc_page(gfp_t gfp_mask) { struct page *page; @@ -163,6 +218,11 @@ static inline struct page *pkram_alloc_page(gfp_t gfp_mask) static inline void pkram_free_page(void *addr) { + /* + * The page may have the reserved bit set since preserved pages + * are reserved early in boot. + */ + ClearPageReserved(virt_to_page(addr)); pkram_remove_identity_map(virt_to_page(addr)); free_page((unsigned long)addr); } @@ -201,6 +261,11 @@ static void pkram_truncate_link(struct pkram_link *link) if (!p) continue; page = pfn_to_page(PHYS_PFN(p)); + /* + * The page may have the reserved bit set since preserved pages + * are reserved early in boot. + */ + ClearPageReserved(page); pkram_remove_identity_map(page); put_page(page); } @@ -684,14 +749,20 @@ static int __pkram_bytes_save_page(struct pkram_access *pa, struct page *page) static struct page *__pkram_prep_load_page(pkram_entry_t p) { struct page *page; - int order; + int i, order; short flags; flags = (p >> PKRAM_ENTRY_FLAGS_SHIFT) & PKRAM_ENTRY_FLAGS_MASK; + order = p & PKRAM_ENTRY_ORDER_MASK; page = pfn_to_page(PHYS_PFN(p)); + for (i = 0; i < (1 << order); i++) { + struct page *pg = page + i; + + ClearPageReserved(pg); + } + if (flags & PKRAM_PAGE_TRANS_HUGE) { - order = p & PKRAM_ENTRY_ORDER_MASK; prep_compound_page(page, order); prep_transhuge_page(page); } @@ -1311,6 +1382,7 @@ int __init pkram_create_merged_reserved(struct memblock_type *new) } WARN_ON(cnt_a + cnt_b != k); + pkram_reserved_pages = nr_preserved; new->cnt = cnt_a + cnt_b; new->total_size = total_size; From patchwork Tue Mar 30 21:35:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D1ACC433E3 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 62232619D6 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232901AbhC3V1k (ORCPT ); Tue, 30 Mar 2021 17:27:40 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49838 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232659AbhC3V1B (ORCPT ); Tue, 30 Mar 2021 17:27:01 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULNwtB130351; Tue, 30 Mar 2021 21:26:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=eeHURyy6hU/3vomeUrsOPdCqibZVdiCRAeYgF62OlGM=; b=QSrWre0+MIaGCYOU7+DJTMuBtk1cCPaiJRKhxW1tqrX3bcQBF+TTbbU6t/6iOBzNDPnj vztSlfCHq4Txv0PJ2GZJHIYVQGir71OFWZKpbOTIh7d5YKsVQh5mOqqYWR2JxmGmEb1y umPXNnmXMS0kHClYO5KAsVsaTT+k+JBedfDQZaYJ9eaiUoNbWZWdtjX+nzp5l/ZOqdtD ReUAHbyz2lHs7Gs+euUUmgl7ViQTSQAFJd/UBIn8rPVunIG0ccbIY2Dyg+W0ri3rgF57 YJjtXKoSRtR9a2r+IfFHDLTlRzUu14/s4NGce5RkSNwX+OOjwc7LOaVhvfTmyldrn+3g HQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 37mabqr8ng-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:59 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOabo149644; Tue, 30 Mar 2021 21:25:59 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2104.outbound.protection.outlook.com [104.47.58.104]) by userp3020.oracle.com with ESMTP id 37mac4kdf6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:25:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UKUSpQ+EW9A912TSdp4d10GvOF1n0NrPiThg2sVY+dbNLuwiKEWZf61yJ5CsnO/f7FgMLlyLMXzdBo3MWcN/IBBHjsBWl5X3u9hgel62nLuhoHErN6OFvHSkv+OHalzowOcogkPVzBf48V9aXT3rXBl6PfV3wSZ53ANimY971lPXu2QMAvPcnHZ8ezU3hVf9pxWHVqxmeOa6RCtehm87QO9sB/I+ZJoeY2/vViC8trW7xOTWS3AdnlIzQOdzFydvITlsiJXtu3e6U2TFET4C9sEJe3iSmOi03fWnSiuVGWAzdPdKeZJka3ShaXQ0bfxvJJwK0IpSv5MfqdrTWP5VUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eeHURyy6hU/3vomeUrsOPdCqibZVdiCRAeYgF62OlGM=; b=MZUg1q1nmj74OSKK94hp4a8fJIkI68RqDgUy5b8qkJcWsVCP2duQT6fuIKC4G/xYggH4YZYn+OT1geQy1pcxqq5JcpoeMwyxxlbw0h6q0PWaAPahige2a1anRv9VTr9PY78GOkdxv5ZWNxEtuTKiGkcb3ilYfXgg41HLff7BpoqOywOoexmyUzkdxSbwwdF8JaG1QVCOplSgWO6fFlI4lefpKU0C9CrWCpX5ezo9X8xRcdNBhm+SxTp50nvuN4NfrlkCbLq9QNlBxtnPQwWGVeQBtdNzcEkuZJJGJRXPfB7o2nBo5QOZ+QYEN6P7oESJDR5XUzDzGX2U7zrIwyNUkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eeHURyy6hU/3vomeUrsOPdCqibZVdiCRAeYgF62OlGM=; b=F9rBjMUrlIgbbeJbR4vWmLqfFwoe7LDUAT1NvP3k9u2EaPV1xrZGQ6T8ZXNEzAVemW0U1yJiq3bQ8sF8lKz1bbKJK1eIbELHRvrN33r81Ds+j1Q8hE4ENS7VLqtwLdDf6l736aZvYKsyXG6PUaC2lga/4BAysMsjigeGVgqCuTs= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:25:56 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:25:56 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 13/43] PKRAM: free the preserved ranges list Date: Tue, 30 Mar 2021 14:35:48 -0700 Message-Id: <1617140178-8773-14-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:52 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 23a80e4c-b305-4445-a0bb-08d8f3c26792 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3044; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1PE7Yw5MNyJHxnEnAc5AjN82fZVNWb8BOpGZY3xByCbLkBlvYkUL41j5GH7jbsIYdNVDww039qzbXdrTHkJQ5ob5OHW1TYQ6qbvQIYiXuMwCtulC8Z7b4CNHojeypotssNnEDyu4ACWFaL9pRtPsrcGm+W20zcaAlYmluc4tiGO69qsE6erZFY4wiGQDNNWXufxMfnpjswbpdqpEprx2WwRM7HcHs2B5PxuPp67momuixcxseeFRUuD4HjuVSlNMygnftmAF1OyGeOG55ty/rzskENTt3ugy7bdbwn3Td4jiRRywJZmHbCfNhimMoggD8GLTgenkFTYdSe3ZdbyU/sXEdX0j8eqt7h0v/z+nDPNpRriHGQyl4WQdov1LSMOwO9yQJiM2JluPF38aZFE23Bg+WTjxMMbYAIsOs252WyHRdtNb5GSjebgEJrSaAm2AhSei9GH6O7zRHXBxl/NIqcQbdLNwoOB9/+5D6RnpxEVpMJFOyZ0XfAs90JD+qhtfWCYWNs8uzZmVeQ8fTv2zh8/oInyB0ru23Jx5Hl770rS1Tk9YklgLl5VoZ6uAvE4+YmLNgP69LF7TQpDEaM9ja2IGgwS7FTq5wBq3A3k/+vw2MP8QBSiD/NWxsPbA1OVtOaWF3b2YZeewDVMB+iTYAw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: eKzezh391nQqclYQXW7aLROI7q3saKoon9xplGDNOp4OKubJkOm5uGvnUR0T1y54hNKInZlK5j39AGVKq1jSyy3SEl7/CHG5cTONUB/CqTPLio27BNu+5AIHaXgB86G3QfeJsJt28OmU92TcbjVFwfBm6QGFNvSIpXVCVFjiMIrQKvCMdQgNcBgPiS+HvzQwokOFH3tLbZetSd/lEGW0mG01VW/s4H7+RL8ZgS1SC14eDHbla1IOCI597987ghDQI6YBsm8GBYO7jO1TPR00qEd1HM1+3x0uFW2mbaJVA43aknChzVY90MIxCrsvut2gXJlNrynIK0gFOb4CKnKy0CHOl3yk5250askx8CcYT1DvImBAqJLI8MRh/8wYdiobjQpIYOIXz8X4NlN0ge4DZU6rc9pGcJpuWrqhSl6v7rjCzeW9gPLIhuVyjXBbTbepESZwLI5ogblD8B9YBm5MbI6SePeu0TbNZLNbU+nk63/NlLVCo7OjLAlfjop7FaG7sl5PLiZM9P763JPMTVes/mzHk0Ad6nxzHD+V6DRI2Wqy85y0HWo4XO2NQeaadPvW4FEDWyiDyKkSdxxgnPuUzAdxqBIB7u1mX+ZNWv+KJx35spU6prKTdMXx/IIxtWpsWGxxnXCh7Z+ce+fQLmHAgG8fMJ3w1jVSGQ6RYzNm+/H0SvWNx6hmyWZfw87QGV3t451i8u779U7oAVdORcU3/EhoMWKKzX0NAJ41JJ5TDSAp66qioZ4BRIEbP1pcRuyl++rEkcOLAYY6Jg8bI+ssNisDJbPVM4a6vt8Zvpk2ik14U23NMGQc9A33jeSzPe5MJn4s9TPOuJJakJauRgNAqHrlqqryrBYFb39HWhnc3dnPqQK04e9PWv4v/KmqqaxafBTpN96QgnAR51GvHnG6cUDrB9j2v4Ya3GA8HBvztZSIYKjhSq1j6ZGKN8VXV7Vcu/oNFw7cWAgbiv65xiILsvquP5SwLSgu6X3pwAKNLyhKSycLQ5BXW5kN+LGjSZghemHg/BVo3LAhH2WDLjXthI8Aa6Wko9CzSViwgLYhoofha9A1j9LUaZKOms988wGtoUwZeakVGw3NCxBPwBx90PIdg7Y5j10AI7d9VxPpZhv5H9QrMxfxqcg1KGCf58h+LhtgiEbs7VrJLwdisuf4ZIKq2bS+Rhzouu3rcXFCAJRi3VD61gscuKnVWgeNcVT9vV0M559KQfgTLvteJSn/KDmlzRBH2DnGXGOeGxoOcgAA8vEH73KmmfYuLCwSCHokTwflsVuCVtqrRmdAoEpReRZmKhvDnQD+racFoNHwe8kS4WVkoV1uPR0b3U/mCN90 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 23a80e4c-b305-4445-a0bb-08d8f3c26792 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:25:56.1311 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ifa1kZL4zsZfIJkd/5wl6ZsDyg18LpL0kpGle0UYcTzWwESp0KWVwxzlopp3H2ivz9BBJHpzUvgG0vDj7B22qcAeUxOcBqSYWyjEim/F8gE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: Pe5kuM0osdDCsxlB-Tn1S0aXPdH9LAEe X-Proofpoint-GUID: Pe5kuM0osdDCsxlB-Tn1S0aXPdH9LAEe X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Free the pages used to pass the preserved ranges to the new boot. Signed-off-by: Anthony Yznaga --- arch/x86/mm/init_64.c | 1 + include/linux/pkram.h | 2 ++ mm/pkram.c | 20 ++++++++++++++++++++ 3 files changed, 23 insertions(+) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 8efb2fb2a88b..69bd71996b8b 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1294,6 +1294,7 @@ void __init mem_init(void) after_bootmem = 1; x86_init.hyper.init_after_bootmem(); + pkram_cleanup(); totalram_pages_add(pkram_reserved_pages); /* * Must be done after boot memory is put on freelist, because here we diff --git a/include/linux/pkram.h b/include/linux/pkram.h index 8d3d780d9fe1..c2099a4f2004 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -102,9 +102,11 @@ int pkram_prepare_save(struct pkram_stream *ps, const char *name, #ifdef CONFIG_PKRAM extern unsigned long pkram_reserved_pages; void pkram_reserve(void); +void pkram_cleanup(void); #else #define pkram_reserved_pages 0UL static inline void pkram_reserve(void) { } +static inline void pkram_cleanup(void) { } #endif #endif /* _LINUX_PKRAM_H */ diff --git a/mm/pkram.c b/mm/pkram.c index 03731bb6af26..dab6657080bf 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -1434,3 +1434,23 @@ int __init pkram_merge_with_reserved(void) return 0; } + +void __init pkram_cleanup(void) +{ + struct pkram_region_list *rl; + unsigned long next_pfn; + + if (!pkram_sb || !pkram_reserved_pages) + return; + + next_pfn = pkram_sb->region_list_pfn; + + while (next_pfn) { + struct page *page = pfn_to_page(next_pfn); + + rl = pfn_to_kaddr(next_pfn); + next_pfn = rl->next_pfn; + __free_pages_core(page, 0); + pkram_reserved_pages--; + } +} From patchwork Tue Mar 30 21:35:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BFA1C43445 for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E70D619D4 for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233060AbhC3V2J (ORCPT ); Tue, 30 Mar 2021 17:28:09 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50272 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232955AbhC3V1s (ORCPT ); Tue, 30 Mar 2021 17:27:48 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULODtq130422; Tue, 30 Mar 2021 21:26:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=dt+SOSW+qvDjw9EGYi/aMPFOqeWbXNmaL/IUeUDEBl0=; b=sxfMV2gK8EFvU5Ca2XMOD4ZSV7GNe1szMgEcivKtXSUMJzChHsBRZC7yP0GEf6ajD09O kLG0kmekrmNkP2Xbj5dQXCPI77ZmFJyMJI7MB8CVJuTTsCtnU33svOKEie9r7faOHk+e yaKmACbGRhvZLhhNQy5GfaJULu1hBB4eBtuZKPi/TjKPDwUTjDw7SD7VURcZX1/TP98H P6m0zJMiljvbltkZvJ7mx0k+jAvwa7x4by95vCETQMjTax/JeJhQZldiYu7sfA9FK+1x G0WbZSJHJ7jwf6kZa2XdYw9adWVCl8GZmvHmIgzNgIbxh8+Hchs2VPqxaHhXkXMilcIT 7A== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 37mabqr8nn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:04 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPSjf105821; Tue, 30 Mar 2021 21:26:03 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2105.outbound.protection.outlook.com [104.47.58.105]) by userp3030.oracle.com with ESMTP id 37mabkbc8d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TajIFc8467NYNapoImDZZoOci8TtpbhhhHICXfLfd4tEmSd8BGm3E1cccrRqn/nvui+a2mGoFpuJcgPMaQHL1PDm8hag21cKSeLTCKJmct97ZGLlmKsaL2vSEiMFthaD+f5sXRVX5TLr4hpN0bts2eSBHAVxtiQRNLJZVjlB7E//k7TbqiWF2SojPCHWFiOfTbXF/Tdx84gS9c0EH23PInQ8udS2lc2ddX/3Nkx4YHPXfxEZYHAhFlRrzAHl7FRAuMHhjX3U/AYNS0aUejRZHRMzDR9Com0hCWz8VyxlNsHGkpLLUjxrdcZl+BUeDpgRb5lhPMJoZFxhBqTLUq5wPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dt+SOSW+qvDjw9EGYi/aMPFOqeWbXNmaL/IUeUDEBl0=; b=meo4NdmbZjWFz2xffsrAduIsH8VX6C+jW2LVr8ySNPESLezVvjzLq39UmvlVwR0iPHQc3/24xVMVB8qopS0WGGxGok/4/guSBy/xu1lOH8dyHXnruENap6s688mB/g4t+Cp1OL6UkGLrYf5hI1pMOdpflG3hA5jzbCf7epmIFHfgeEmTRSRNa8EiGvq7n7/gdzVhsbCRS60mgc04a76kdKxNPctVdCFpRdjIS53RYuo7TGLKRizhgxEBAI4UoAoHNq3q76EcoxgUtxYkH1nxyUoKFvAfwvraYx2DZfYj+dYGjCteD8QUnNDRjIJ+1tbrrqwRus72UilEG4W930Xg6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dt+SOSW+qvDjw9EGYi/aMPFOqeWbXNmaL/IUeUDEBl0=; b=erLFKn51C18b2dYYlJhDP6k6miQSeNLbWuLGr8q4W624/ir7EBYQL25NPUsL6MQqc2PF8y9jufJTwlzu2l9FeOAUCpmIvNM6Lx4lLA7zoV4RtF+cZ47wdE1NnOnx8Js4uZtIyFWaI+Ab6ghHjnCwbEd/kKd1smLqOzNEcaMYTZQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:26:00 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:00 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 14/43] PKRAM: prevent inadvertent use of a stale superblock Date: Tue, 30 Mar 2021 14:35:49 -0700 Message-Id: <1617140178-8773-15-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:25:56 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 14222a83-9557-4da9-593c-08d8f3c26a26 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TgRLE1Yo2yqsravy1bOac6QeOOvVW2tUsh0IHd25Z3AHz5eGQwCwthf1wUlj9OJYc1SRpYoM7W3dOVD1c6o9xkE5MDG9azzPwhQZFeIv1XHoHqy09L/+KssKI1mC7giefSBtJG5w3Y8uJLppQ+8FwwRZ2IHXbyqhEYmmc8DgPLxOErOjCm+E3JFnLlQsUvbShYIa/CONWL8G2t2JYRpn8Bvm08uny9y9KjbRxgzidS+Q9DyMAZjXnpK/DTQF6Ean0cwA8Ow88YqYjz+nDJIZ6HUHfiP3dBD1YUK6m/cmb2ja3fUhHD2v1VhmH3tJLo/lZcTZSluGDba5vsqsJU56SZX0Te4WSAKnRLLwfuerjMCdV7WBpbyhr3q29XmMSrKVpXLyoKujw5wQ2K6UfUC9jE5khnwl4Bz9Tc3bfxORhJvcMthU4MBlPI+C/09Axh/1J+OFgheKs5xp2dN3+wKeZzScbChzN4G3E6nFGNJH/YytA2nmQZMZrGoV936n9sOcQZuGztyxZXPttfjve+9Hhm/NjamB6bTT+7BgD4da07i6MSgV9JT+LZIbnlxu67HdRohP/X2ZoA+6ztj2E0Op/OFUrC0XxBwBX2qN11ktfwl4Qj3xDWnfIV9aQwJCkNa9X2qRmxoGpkW89et2aeP2iA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(83380400001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 1dBGyObA1efgYcXbkoAxhpm2DfdBDE8+UZOP0lnslGvOjYSofYx3kcAqYASZ90RDw3tV5Gq1wDKXiZN7dp0uVJPSUUw9n+5+E6vDYTcyNDUbJTqWizCDxzun3ZAlN/ZtTJS/fjjZB4uwT4MFsJ6Xhh5GQy+y2rtW0XoRoQLt8WQE3F2ZoZmHOG+Q3Ir1+VkWf4YqkC4C5mpq7DPuofAuumat1zqE7iM9sn4VHUYraCBLQX4HxuXJ0e6kz1dk6HXkBmKZ///4V/yhxjeKob3L+P+x3FC0GdlVXQ5oWphD5QtkFBx8klJeD5GFJ1TMJVc3YVQG1hqZ4FUW3PEobZQfxdZbU9cfzERR9sbZ1XxqpO5QtNwskPfmHSeG/GWHaJa4BJoyyT9/Ew0/AAXuGjm/IrjNu6kQ0DckKPEvxnzfj8ovRZam0ZzBNIAaQNSYgeW224fh4BIihfVlv5vfD2cnDn+O8ZzB4wkbnJs1d+kIirS80BoPi35OEXxr41aCPU/02PJ9ARuipOFbTpNNFXSY3jyy3W371P9KHCTXXYwFPnkX0B/Aab4LoJpZU+XXJy6A9rGt3rD0JRYBMWOBQcT8VBW7eYh3nByTRyhW4mmBLJ6Vm9t6JroV8YEkJQ5+GzQyErX65wbU+8VkmxE2biIvj3Y5/FD3rsSLznwdNvuCfmlYG9o4oLSbxp5cXgiqjuBbEmQIVhjYHNsO4Vas5yyEbkuLOc2N9W5Ru9S+FRKX61whPqHLXJLo03FqJ/vTDfmY9j19RihxqBmQIp03/ZqnUBKdW3qKPsNUtWXv5a2WMAnjfsoEWy13U7eVRjY3nuV0zKVlY4hROZVSa9Zm1rJNuTBuCtI4x1exNCNjD/xRNeaBAM2x2WkD9RM2hsaeXOx6Yy04BwbvNd6CDEtVMGLLS48DZrr5NsAI4zGtKPKJpk+16Iap3NacGV5neKp2bmbuVYNEu+UBoNqcWv1PHts+w+8vkYziEq4huRm74pd2UvJaKcmujhcDuhgMLEIF29T1+ODddFduLsSMrqNj8kgqB9UwtSD3Gtw/TIVRi7DNnELHyDnsDA7KLQDtTOfYioS/lzh36sAZGQ6vdJI5KmXujugnIHNICoQBsl+Q6/Guz0FCPgSYP2q2ycWLFNrq0ODSxhXMaHsqZClMjEkRZZAN96BOxI1RLvp4ZEkG3j88CBMEWODXL4t4fdJwsNzjVmZHv7NHcA/OAWhH+MScHquVUn1FlDRw/PJmYEipK8Mo9Qs5+K+HXnndOMF3YoONzDX2J0mwNEZmECKZkVekhGxoY1UJWUgwXhBjwAnk45LLH7a3xicZFpTomIv82YKcLCMT X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 14222a83-9557-4da9-593c-08d8f3c26a26 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:00.4527 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eqA8PVSMvy/lgup76xFu6yCJc3WA9Q9jmDDNdqDorIPF5YvWhy7EfSWQMSlpJqrOOceA0EmKmvz9s57u3XEf/gCaPz/8Q+TrRhjGe2USx14= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: xGdJ571s29Ex_rL6pWeLSAxpiCct7jYm X-Proofpoint-GUID: xGdJ571s29Ex_rL6pWeLSAxpiCct7jYm X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org When pages have been saved to be preserved by the current boot, set a magic number on the super block to be validated by the next kernel. Signed-off-by: Anthony Yznaga --- mm/pkram.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/pkram.c b/mm/pkram.c index dab6657080bf..8670d1633a9d 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -22,6 +22,7 @@ #include "internal.h" +#define PKRAM_MAGIC 0x706B726D /* * Represents a reference to a data page saved to PKRAM. @@ -112,6 +113,8 @@ struct pkram_region_list { * The structure occupies a memory page. */ struct pkram_super_block { + __u32 magic; + __u64 node_pfn; /* first element of the node list */ __u64 region_list_pfn; __u64 nr_regions; @@ -180,6 +183,11 @@ void __init pkram_reserve(void) err = PTR_ERR(pkram_sb); goto out; } + if (pkram_sb->magic != PKRAM_MAGIC) { + pr_err("PKRAM: invalid super block\n"); + err = -EINVAL; + goto out; + } /* An empty pkram_sb is not an error */ if (!pkram_sb->node_pfn) { pkram_sb = NULL; @@ -993,6 +1001,7 @@ static void __pkram_reboot(void) */ memset(pkram_sb, 0, PAGE_SIZE); if (!err && node_pfn) { + pkram_sb->magic = PKRAM_MAGIC; pkram_sb->node_pfn = node_pfn; pkram_sb->region_list_pfn = rl_pfn; pkram_sb->nr_regions = nr_regions; From patchwork Tue Mar 30 21:35:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04EEDC433DB for ; Tue, 30 Mar 2021 21:27:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C2972619D0 for ; Tue, 30 Mar 2021 21:27:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232776AbhC3V1X (ORCPT ); Tue, 30 Mar 2021 17:27:23 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49756 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232503AbhC3V0y (ORCPT ); Tue, 30 Mar 2021 17:26:54 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULNj1o130310; Tue, 30 Mar 2021 21:26:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=AMXtTm3E0dn7+dTav4N7vCwJiggNJoJ6HvREteOIM8g=; b=SG/d9zVrC9H+wcodhUKy3I8642yg7a4tO/ZOfiHT7EmqMTpiBJLNE3ttOD4kPqNp0vvj gKu+Ph4d8Gy04WWXINRkEfi+qwtiwkcYnnJ7ddL2Vecfjbk6J+zF1H/UH2MYG2PEwwxh Ew0TuunckqA0N+HuqcxByIiiIqifFIiMuIlpKdeoG4gv8mw78ndkkQRTjU3snyXyl3Rz +wi4R05yw9dH3MAL9nvZS2W0V3TMxF/5poliiTJmAKTZHTWGL2MRDWFRhcS/QEtmO+v4 Rq6UARU35e3jO5oOSLNSmVC3ZM3vCMuOL7ZPzIGIXGw0VJyTdooMoOyuW2Gi6NCvxkX1 Fw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 37mabqr8ns-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:09 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPSoj105808; Tue, 30 Mar 2021 21:26:08 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2103.outbound.protection.outlook.com [104.47.58.103]) by userp3030.oracle.com with ESMTP id 37mabkbcad-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JAw6wRdNV39R6eRIGGec9FetM4AZziA6WZ8tWn3IwmsOrQi2WmIqVPNzhq9v4Nvq2eQh8aag8zpSKwsaAnx3cIlEYBtr/2N0FEbWT5BwG8OVz1AHDPEeBYtisihn82ldi1uqPNNxbzNTx0NyzvUyxSY7GpzlcEl7wSnLlSgyk54ANMgDotL1N3tI2DHgm8W1T9EkAxrwwregNq06Yd9jR1LyDMp/BsWaUFHE3/6vyQc1ViOE/xCbMnr2SIdNnMsWH0t/am4B1RwdiCHVviHEjuICVzDYDB/LCRYyv8LIKnsLCp6URGZDn7MIYSortsscDsT2WV4UALYhUYA76Ab59w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AMXtTm3E0dn7+dTav4N7vCwJiggNJoJ6HvREteOIM8g=; b=jMKTdu9y1pjnt89WQzOV2+MgG6LgNROIZ8HyR7AiQyKXITm3IxZRxczfqcQQxFyR7Rkq6hOvOqkBBJ6owKwJhKfkRhOgg5w+Ajp1FlxwZ5geSNEJkDn9gfzmcaqXQFXQ/VCz2gocw8o7zWzAP/cHaZ/5A6ikpw184jzZ7FIABAdGFDEzOjJJ7izBW/AGM/wcEliFGMHHiDLZbxX6SE+S2icd49UAY93NeKUk9U+RD8+zEaCtXykxBei9WF/t12LEUw+cZbHr2mMKpEMIqovNrI+Mcc0xTrvNeAIQ6tt5IuZ8eu1/L6TbTUE046qqgKeOHbGnECvKeLVizOFDjnVP6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AMXtTm3E0dn7+dTav4N7vCwJiggNJoJ6HvREteOIM8g=; b=fpwhHvHOntMqkm1BVDnfS5/T+xZklo8ddo/pjqrdxEM/35f3uMNOZkIzq3jbWE75lRID00l7iAWoQVOMiTqOPvt0zudbseduRQ/hNTuk0SYDpNsteTvKG2HTj5iz2AMcz0zknI+f53StD4IOPsCwpy9xR7eeZ08P8JwvT8kZkbg= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:26:05 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:05 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 15/43] PKRAM: provide a way to ban pages from use by PKRAM Date: Tue, 30 Mar 2021 14:35:50 -0700 Message-Id: <1617140178-8773-16-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:00 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9a97ad74-0825-459f-2788-08d8f3c26cb9 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: B8TltwCHssxUBgT2wbjjhhQUpZmth0E0q0wv/PSBnJEPTA5bHAFJUgXJOi8EhFNRt26VeXdTvtSwLVEZFLF5uIxHf3ztsM9zlMXZF4jb9uegslju4ag7jpzv+mAz4imNXWb9YJ3VrBhJ5P8X7hi/mFmtZt7QaMsgx97hbvxIjYQTQUNcM6qVgM3ZvYEPXcZNAEnRu48DDF9ZpjnTTNW4rUWlAPEfwvY02mmtSnSgUwT7etc2prgaTPrMb3qlf22c+nbViPKyVbTE7D/PMQxTLP93YSUa1V2WFi+250uCpUXrB6NZHsNIWeeT1yuX11D2uKFZTiqRQ7s0UWgNxWnIvlHWkfp+ZbId2I1ciW8PJwgA6879j6P4yvDkGNw5FlCwrxivtJvmVu760nyNSr39DOWFgaGXeD6Np1BL7Fe2r4GZgZscOMNgZC4qnBhhxXEiiJYu+SYbBlw9OI4kVlB1arb1hivTe1VB2DRsTOq6L6fjrxLMu8TYC73y8SwenS1KEj+SgW4zRAEou4abg8MtBnP0IF3WjtNl/P570pYzDBeGHQjx2Z7S2GuuZVEdh+ZLnzGvK799QJhtnpsvbxeV+VBzqCeCacMO2kqXDas3IDCuJQxODST4UHS1KiP0mtVgz1uEWgT2qXy9tztA3GlW8vaifabgo2RrdNvyppTvdJg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(83380400001)(2616005)(43043002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 0PuWjNRcsA9GwmSUcqzt43RbsPaBLMbr3e77P+eaUf7j68Qg53VTe9+x0x4uH5+Smt/3ecObKVYqdPOaQuboYAOqTjOQ+4waPyR41FolkMcUHWCY6iCtQR0Wt09nEAkTcrVHnbR3DEP77f/I2X/pCwnYPaPFF5BaWYcX9WB/6KJ2lw9V9rqcMdVR85Bm0XHiPACmAa75M1A155Y8XIVjgwpv+GZ63YHpvvUw8dKj5E0ZUWXq9rvwLuSaNGUvcxECcpnqACcACnNeqdgDTRTeJanK1hYHLg5mIPv6WM6SkKVom+mXquHduETKYiN1t1JB73KmciIT0Y6z4d7pvuFctNQPdJOBwOnMc1MUgCNByNFZUHhvclPvJD7JLcdrCGfT/Kl60oI3H2sFQ6FA9H8npkVeT8Yk10nnwYg5ejeKppvuEVzKTp9mElVbf7YawBALV6trOu27SkX05EOsEkuJqnZut6fthQ9ZUMACZyK9SAS/M1O56+lCTMY372al8rsbalYZAtXjGbPMVG9j+mDOZgDRV4gZcQOf4JTE2WONUblRDBgkGNSbeoODK7BXywh6nyTvHVIWUYwotigY9ZdZUKC8KvNoqVFuBpQPFgSHJKqe1PgiW3UoooA4YgDduXYNyAThGa1167fdNCMXfh7tZvbkevIkFwfPL9Y5t6IxQCJAeCsPP9+PZ0HjHDUCFWC2LNLrw3/grJn8xajusoT26lOsLOEreRRb6PIX7FpaKWDl2IqScf/6qonwKlXsgoqvvDJfKHOmJqaoQlb5leBcoQn/4TKR6r9vpKpwtvNhjPRo0oCeVh/iaZmoIpAE+f6pIFAMbkRmvkwrDRImaZovm9NyvtWp3F1GsA+yhfJI2LjzzIC0Sy2qJPP+coobuGmxwHFxCIl2aaVHjOLXkXgG3/zwGC9yg9cKP23SOG8aTbNm8u1AA3eHA2MmSixHbSVlEoY64OFZRstFYZBEQfEnd/g3QLbDsFXN08a++CQu5vPSyZr+eUKo2hgS+oCp1BdZ/+hYIHmM9FBWE9QDqYPSgyekvEqs7G4sl4Klf/moSnMkzyQgXdRsV+MJbhopJ8P+v3gBehjMoUfkyd5gKH2AXaE4CyaB1rQ8Tvxc3+5fhMz31e+bUWpMbpEIvRO87RzAwyXUo/4xCgUvgfh2eENd44JVlpVYmkHcXLvOhOYbaZtVD/MSZR59/U11+2TeGXJT1JGGoBaZp0oAbKIiNFNpkqq7nKaf2nqySMmyKsq6sIhcIqC92bN5sUAKlwiRPXKbegKT59bif8KbSxcc0cPk8VCv8FKFsIAP/61zcFFC98A04QIvbHkG8Pawgk9mtU/X X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9a97ad74-0825-459f-2788-08d8f3c26cb9 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:04.8002 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UOzFZuN1J1q6ZplvWq6PvaMvwAYM9dWzCrZ/CTCamOniUATPQ9veyEPRQDaOWQej6/nCyyUQ/2H57fJjzv02P51I+ULk4AFdVAZsPOWnrHA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: 31NP70dNym8jOVnYu71GidvBiy2sTXFV X-Proofpoint-GUID: 31NP70dNym8jOVnYu71GidvBiy2sTXFV X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Not all memory ranges can be used for saving preserved over-kexec data. For example, a kexec kernel may be loaded before pages are preserved. The memory regions where the kexec segments will be copied to on kexec must not contain preserved pages or else they will be clobbered. Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 2 + mm/pkram.c | 205 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 207 insertions(+) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index c2099a4f2004..97a7c2ac44a9 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -103,10 +103,12 @@ int pkram_prepare_save(struct pkram_stream *ps, const char *name, extern unsigned long pkram_reserved_pages; void pkram_reserve(void); void pkram_cleanup(void); +void pkram_ban_region(unsigned long start, unsigned long end); #else #define pkram_reserved_pages 0UL static inline void pkram_reserve(void) { } static inline void pkram_cleanup(void) { } +static inline void pkram_ban_region(unsigned long start, unsigned long end) { } #endif #endif /* _LINUX_PKRAM_H */ diff --git a/mm/pkram.c b/mm/pkram.c index 8670d1633a9d..d15be75c1032 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -141,6 +141,28 @@ struct pkram_super_block { unsigned long __initdata pkram_reserved_pages; /* + * For tracking a region of memory that PKRAM is not allowed to use. + */ +struct banned_region { + unsigned long start, end; /* pfn, inclusive */ +}; + +#define MAX_NR_BANNED (32 + MAX_NUMNODES * 2) + +static unsigned int nr_banned; /* number of banned regions */ + +/* banned regions; arranged in ascending order, do not overlap */ +static struct banned_region banned[MAX_NR_BANNED]; +/* + * If a page allocated for PKRAM turns out to belong to a banned region, + * it is placed on the banned_pages list so subsequent allocation attempts + * do not encounter it again. The list is shrunk when system memory is low. + */ +static LIST_HEAD(banned_pages); /* linked through page::lru */ +static DEFINE_SPINLOCK(banned_pages_lock); +static unsigned long nr_banned_pages; + +/* * The PKRAM super block pfn, see above. */ static int __init parse_pkram_sb_pfn(char *arg) @@ -207,12 +229,116 @@ void __init pkram_reserve(void) pr_info("PKRAM: %lu pages reserved\n", pkram_reserved_pages); } +/* + * Ban pfn range [start..end] (inclusive) from use in PKRAM. + */ +void pkram_ban_region(unsigned long start, unsigned long end) +{ + int i, merged = -1; + + /* first try to merge the region with an existing one */ + for (i = nr_banned - 1; i >= 0 && start <= banned[i].end + 1; i--) { + if (end + 1 >= banned[i].start) { + start = min(banned[i].start, start); + end = max(banned[i].end, end); + if (merged < 0) + merged = i; + } else + /* + * Regions are arranged in ascending order and do not + * intersect so the merged region cannot jump over its + * predecessors. + */ + BUG_ON(merged >= 0); + } + + i++; + + if (merged >= 0) { + banned[i].start = start; + banned[i].end = end; + /* shift if merged with more than one region */ + memmove(banned + i + 1, banned + merged + 1, + sizeof(*banned) * (nr_banned - merged - 1)); + nr_banned -= merged - i; + return; + } + + /* + * The region does not intersect with an existing one; + * try to create a new one. + */ + if (nr_banned == MAX_NR_BANNED) { + pr_err("PKRAM: Failed to ban %lu-%lu: " + "Too many banned regions\n", start, end); + return; + } + + memmove(banned + i + 1, banned + i, + sizeof(*banned) * (nr_banned - i)); + banned[i].start = start; + banned[i].end = end; + nr_banned++; +} + +static void pkram_show_banned(void) +{ + int i; + unsigned long n, total = 0; + + pr_info("PKRAM: banned regions:\n"); + for (i = 0; i < nr_banned; i++) { + n = banned[i].end - banned[i].start + 1; + pr_info("%4d: [%08lx - %08lx] %ld pages\n", + i, banned[i].start, banned[i].end, n); + total += n; + } + pr_info("Total banned: %ld pages in %d regions\n", + total, nr_banned); +} + +/* + * Returns true if the page may not be used for storing preserved data. + */ +static bool pkram_page_banned(struct page *page) +{ + unsigned long epfn, pfn = page_to_pfn(page); + int l = 0, r = nr_banned - 1, m; + + epfn = pfn + compound_nr(page) - 1; + + /* do binary search */ + while (l <= r) { + m = (l + r) / 2; + if (epfn < banned[m].start) + r = m - 1; + else if (pfn > banned[m].end) + l = m + 1; + else + return true; + } + return false; +} + static inline struct page *pkram_alloc_page(gfp_t gfp_mask) { struct page *page; + LIST_HEAD(list); + unsigned long len = 0; int err; page = alloc_page(gfp_mask); + while (page && pkram_page_banned(page)) { + len++; + list_add(&page->lru, &list); + page = alloc_page(gfp_mask); + } + if (len > 0) { + spin_lock(&banned_pages_lock); + nr_banned_pages += len; + list_splice(&list, &banned_pages); + spin_unlock(&banned_pages_lock); + } if (page) { err = pkram_add_identity_map(page); if (err) { @@ -235,6 +361,53 @@ static inline void pkram_free_page(void *addr) free_page((unsigned long)addr); } +static void __banned_pages_shrink(unsigned long nr_to_scan) +{ + struct page *page; + + if (nr_to_scan <= 0) + return; + + while (nr_banned_pages > 0) { + BUG_ON(list_empty(&banned_pages)); + page = list_first_entry(&banned_pages, struct page, lru); + list_del(&page->lru); + __free_page(page); + nr_banned_pages--; + nr_to_scan--; + if (!nr_to_scan) + break; + } +} + +static unsigned long +banned_pages_count(struct shrinker *shrink, struct shrink_control *sc) +{ + return nr_banned_pages; +} + +static unsigned long +banned_pages_scan(struct shrinker *shrink, struct shrink_control *sc) +{ + int nr_left = nr_banned_pages; + + if (!sc->nr_to_scan || !nr_left) + return nr_left; + + spin_lock(&banned_pages_lock); + __banned_pages_shrink(sc->nr_to_scan); + nr_left = nr_banned_pages; + spin_unlock(&banned_pages_lock); + + return nr_left; +} + +static struct shrinker banned_pages_shrinker = { + .count_objects = banned_pages_count, + .scan_objects = banned_pages_scan, + .seeks = DEFAULT_SEEKS, +}; + static inline void pkram_insert_node(struct pkram_node *node) { list_add(&virt_to_page(node)->lru, &pkram_nodes); @@ -709,6 +882,31 @@ static int __pkram_save_page(struct pkram_access *pa, struct page *page, return 0; } +static int __pkram_save_page_copy(struct pkram_access *pa, struct page *page) +{ + int nr_pages = compound_nr(page); + pgoff_t index = page->index; + int i, err; + + for (i = 0; i < nr_pages; i++, index++) { + struct page *p = page + i; + struct page *new; + + new = pkram_alloc_page(pa->ps->gfp_mask); + if (!new) + return -ENOMEM; + + copy_highpage(new, p); + err = __pkram_save_page(pa, new, index); + if (err) { + pkram_free_page(page_address(new)); + return err; + } + } + + return 0; +} + /** * Save file page @page to the preserved memory node and object associated * with pkram stream access @pa. The stream must have been initialized with @@ -731,6 +929,10 @@ int pkram_save_file_page(struct pkram_access *pa, struct page *page) BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); + /* if page is banned, relocate it */ + if (pkram_page_banned(page)) + return __pkram_save_page_copy(pa, page); + err = __pkram_save_page(pa, page, page->index); if (!err) err = pkram_add_identity_map(page); @@ -968,6 +1170,7 @@ static void __pkram_reboot(void) int err = 0; if (!list_empty(&pkram_nodes)) { + pkram_show_banned(); err = pkram_add_identity_map(virt_to_page(pkram_sb)); if (err) { pr_err("PKRAM: failed to add super block to pagetable\n"); @@ -1054,6 +1257,7 @@ static int __init pkram_init_sb(void) page = alloc_page(GFP_KERNEL | __GFP_ZERO); if (!page) { pr_err("PKRAM: Failed to allocate super block\n"); + __banned_pages_shrink(ULONG_MAX); return 0; } pkram_sb = page_address(page); @@ -1076,6 +1280,7 @@ static int __init pkram_init(void) { if (pkram_init_sb()) { register_reboot_notifier(&pkram_reboot_notifier); + register_shrinker(&banned_pages_shrinker); sysfs_update_group(kernel_kobj, &pkram_attr_group); } return 0; From patchwork Tue Mar 30 21:35:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFBC8C43332 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9B568619F5 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232907AbhC3V1k (ORCPT ); Tue, 30 Mar 2021 17:27:40 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50878 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232645AbhC3V1B (ORCPT ); Tue, 30 Mar 2021 17:27:01 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULP4tQ145269; Tue, 30 Mar 2021 21:26:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=amsaAxr5dRgEtn6kKRVtAsbBOsljUsCMyrQefSwuQPI=; b=M7AbEzWrp6HBBqoUDLpsC9AZh4t2bdueXly7L6J1m+YejrIOxnyiWr21U0/YKXgfsLi4 bpjq1gHVOfqim/bDNfNYhthNKrwJ9sESRnu+aJ677B5kVodfhLErNC7v+QvnucZ4NiE0 zoX7U5oKa7+ONf7zn5piyC7LbIybJWRGByFOtAEbPpWW+urhG6rFyZ+jj+osquQj1GHf LQcadYC30VSpgD7T3GSaCqabJzvUp1vRtzjWIkS7IqCdbuvX/rcZNvOn225iUwn+adMV S8HsBMvoAxCQeABehQrLr3DWJA5TSrwKPkvCn9LLR3zQJTXaoWfjTihCX21PWJbVf6t5 dQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2130.oracle.com with ESMTP id 37mad9r8ff-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:12 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPZ9k124987; Tue, 30 Mar 2021 21:26:11 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3030.oracle.com with ESMTP id 37mabnk79b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:11 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cE20CugVIxWqIMx75UOtft367YQYgQXiYgCzbDK0Qd1mDxxZdBH1B9P8qXOpv4sqXk7LTxliTNGut/kXuAQX6RYW6AecYJBTOlzRSRwcT/XsF8baiz8YqkjRfhCjyrJ29kEjVXiPbX18rfB3OJzJlQomabyPuwUmJTMYKLXFiKswe8BXSC0g5l2xIRycxxW5vnrVDvwmf3mEGnDLh121JezKAUElusjoaaR9y88yVLywGe2wtr1Rev/bsBB+ejNHeYyCgZnOVmYxHQsE92kAOu+iS6UBot4+qSnvgIA0d/4QOT02NUusXVxPguIZe6GkZD+6RDO10GYqSPmZtN8q1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=amsaAxr5dRgEtn6kKRVtAsbBOsljUsCMyrQefSwuQPI=; b=WxUEfxmozzQiSfdTteiSH68Goq52QV1Ze2nqVA3t2+lRB2m3nFtRs2Au5mYo0/3Qp7+U52Ybr0Yff1lqSzeOO3QqjYTDmcJj3tLnTdborzQ6Tx3ixZqascVVQ0ZSJXxQUNVbRcM2c2C8kbVVw/JYSyFxNbhETjAET8w6Wh/5yX+0C7Vx9m4tdFITu3d868ljbNvAWGRmU+JyZLA1JEqvx89qpcWGhHAfknKlSihGvdQA9xrzHOZb6TG5x+92Vn1qyd1Gs9ccw1h6jWRdSdragWxd25IoEJgtg4nfwXowSl7S8B8B4DRZam1x4z87Ip5iqEvc0DcI9SRiMpM+fqAFQA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=amsaAxr5dRgEtn6kKRVtAsbBOsljUsCMyrQefSwuQPI=; b=a+jjZAz0RTjG2B/U+rLdzwQuLJV4nEj9mBi+jOB5ruqhZ7R502oiEwVbGALvCj6sUUBq75qphmePAQWMOz34JOH4CMPcB67UGgly9sD0pKk+/bfCLROxnzjG5Rett0+ralckLKLBWhqi+oqhrtj+OASR3k2NPIGAQOvTtK89mTQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:26:09 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:09 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 16/43] kexec: PKRAM: prevent kexec clobbering preserved pages in some cases Date: Tue, 30 Mar 2021 14:35:51 -0700 Message-Id: <1617140178-8773-17-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:04 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b8113672-aa3c-473d-c672-08d8f3c26f46 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1091; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7YLL0W7hKvOnyCdBf8XC+KqSjjpAik+qhhOhEn10bo1iPzZdIjrxERb06h/jU2noCkeD3O3v1OSYpIMIZmLSFvMyVl1HcZxgdb7zy+rsGo36ckyPFhX8NxaGQZeL8744aKGKpR3sg+yKp3ZIGi30uIMaK/W7+/eNBFPYnktdWOthvYOqy4cOrgudSnIeRBeDFJznDN0xRCPOGMnDWN7duCqHov3c1BscVBgKWoA0dCGr6P9Sflm5myr8UJ3wgzt9CRCLE0oby+7K77vpm/PYClmQLUZEmmGL5+ev+PlrNn3/RmsPJxgp5bbvp/4C4qPUOtmDpR0oX4cZvXc2pRRr6b1NbQdbReMvuPpO2nMsKaWLIcU3GmzeQCBPqoQMOrmd734ootM3Bru+IjRsOJ37UpWQKuP8FHFto9tWIwgWS/j7ygvdiCNk6Y3xePT3h6/qfGV3APx46s+O27xehmq1q0HwlNbIHAfdWivx95GcDdj9ldZmLFLIOBKUThFLOstdjXuSRuF4QIYZv5W9ai6leAWSzxCNs1nTB/oezm2UF/uJtgm5ViLYQqh3j1K01hjckt8lJWFmRm3EabicvGdP5EFC3hdBmQWSdFOrfz0+uvuDD9AJkOI3xK9YSEU8BXqcoajNoylu1uTYhWUXkxpopg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(83380400001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: MCZc88onvvCfIBPEFPgfTeWY6uShWuUv6BtQ1UPDHINtW2w0MFlMUSY8K8JKF9R9XAu7kO6E4zl8J2rMagAjaf/Gw4c8hq/tTn416be+VA0jz2srxs5na2gJl96wy5faKvgmjh+fEO4zkRBJis27Cmy/pSg3Jtb0MsC9E40G9XEbQuiS6fMT6EAtlAWyPH8sKBfTLASgUyiJLRVFl3LHvZ965V8qdX9FXkYW5PmFTMS+2rE3oYds/5whqIh+elpMl1Ei1apmy7p4gCdENCLgqwQA8mN2NsNwDNDRlfZLsbB17RxKa7nZtXb91wQ+w4lGpZGqZVNZTddjvlsnyolOq41NslkydxT4QCuYzUf4rjXEAVcKctzPUCgRKHvdXlvhwbz6J/4D/HiGnkkmqr3fyvFX5Sj089WXgPZ9HwKok7qtIgB5ChMXVJQvR62TSxxx4qpPBJ/AAdEZuftrIl58/3YCi9YX5JSSdC/S0gtqZ8XkjE2IcRDsoUBrkiKv0fqRSYlOjUBs79aESJ1r7QzaVZeMTb/mAozZphOtodhSAsErZneLL0TyO0c51S1ITZT/poNnoMTSvPvsH31aOAaq6aTRt3YPqg0Zpc+WnBLMusjxgGpsMxq9kPJR512caGedfC4EWZGcw4o/cjk6x7JuAs4M/ZC0loMISNcjwGgKwVL2dkOS3dapMTqD11i8Qiqn/C+xnOP0VIekZghNJAV8kJrMamkz08HI1+TMchAXQ1Q6wc30w0a28pw9f+QkAxgoIvaSBSdjpSO6fLfj236dWwp+RW7ARlNkNI2sBp5V9im133wZbGRS4Sp9UxS91QJ2vq8MCZz0Z7WDWUYimJ3ePpmeBu2SX50/AWSH0FLmLOm/8ud3VKZ3Gco+dkzj9laq/Q8cdnyDwblhrxnRzhYfM2uUApz0/yHKKJTkiSH5h91MwGIsFmSGlu1fqL10D8K1vEJ1jukp+w9z7doxNNJ9hKoUigrFbvYwbcRp5YbAn2E9Dj+mm60g6vGHznNNHywZNzjT3wRUXlUc+oy3TnyartvV4Gf7J/CTE95kX14aWjesyvmx14yYwKDcIo4gjyVrM2dWyOW5AnfpZS9qa9zAHE9oend/bRS883S3xXWC4uUsLlMHURV3vxpGGfbn+N8Jk8pXsaUgU52f20S3Hsx9G3wfUFiWtr1Ku3+YjLmZpNtpo+NFJF3J5j/EYVJcBTzAkW181y+RN6XKMQgH1nlrOHdkPX4AVSXW9wzaa72mt1Lke/M+4R063zSI8aaRXhGLIUblgs/wuPvKvfdr9fhLTI85LujhbirjVFcWNWPZpHtO2KABrBvA8znnXEKmdTlU X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b8113672-aa3c-473d-c672-08d8f3c26f46 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:09.0818 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: I07Cb8muMdtBCuk8/IVsLe8n/G7PdLB3+n9QYsZcemhW+ZpyGOq9KSpiisiSozylkEhmRsq/aqnk+OsMgS98k9tXHJBZRHQ9qanO6INTicw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: Xw1lkZmopDx25j6OqeXY_BLeqaWPte_9 X-Proofpoint-GUID: Xw1lkZmopDx25j6OqeXY_BLeqaWPte_9 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org When loading a kernel for kexec, dynamically update the list of physical ranges that are not to be used for storing preserved pages with the ranges where kexec segments will be copied to on reboot. This ensures no pages preserved after the new kernel has been loaded will reside in these ranges on reboot. Not yet handled is the case where pages have been preserved before a kexec kernel is loaded. This will be covered by a later patch. Signed-off-by: Anthony Yznaga --- kernel/kexec.c | 9 +++++++++ kernel/kexec_file.c | 10 ++++++++++ 2 files changed, 19 insertions(+) diff --git a/kernel/kexec.c b/kernel/kexec.c index c82c6c06f051..826c8fb824d8 100644 --- a/kernel/kexec.c +++ b/kernel/kexec.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "kexec_internal.h" @@ -163,6 +164,14 @@ static int do_kexec_load(unsigned long entry, unsigned long nr_segments, if (ret) goto out; + for (i = 0; i < nr_segments; i++) { + unsigned long mem = image->segment[i].mem; + size_t memsz = image->segment[i].memsz; + + if (memsz) + pkram_ban_region(PFN_DOWN(mem), PFN_UP(mem + memsz) - 1); + } + /* Install the new kernel and uninstall the old */ image = xchg(dest_image, image); diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index 5c3447cf7ad5..1ec47a3c60dd 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -27,6 +27,8 @@ #include #include #include +#include + #include "kexec_internal.h" static int kexec_calculate_store_digests(struct kimage *image); @@ -429,6 +431,14 @@ void kimage_file_post_load_cleanup(struct kimage *image) if (ret) goto out; + for (i = 0; i < image->nr_segments; i++) { + unsigned long mem = image->segment[i].mem; + size_t memsz = image->segment[i].memsz; + + if (memsz) + pkram_ban_region(PFN_DOWN(mem), PFN_UP(mem + memsz) - 1); + } + /* * Free up any temporary buffers allocated which are not needed * after image has been loaded From patchwork Tue Mar 30 21:35:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2945FC4345A for ; Tue, 30 Mar 2021 21:28:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB2CD619D3 for ; Tue, 30 Mar 2021 21:28:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232859AbhC3V1g (ORCPT ); Tue, 30 Mar 2021 17:27:36 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:60872 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232628AbhC3V06 (ORCPT ); Tue, 30 Mar 2021 17:26:58 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPchq123355; Tue, 30 Mar 2021 21:26:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=dZRe5gfZ46LFF432m3qLr28Xysmg6HLZTWPgjewoNqc=; b=sG6cmxA6GoZd9L1qTnYXmZ+YM2bUvTRgKnkfn0pDPyBXLYGEI4iMyeuAFttonhF2Rny8 i3K5KmJxTQBnGBpKnEX7BUcAgb8wqEXy5VfjsrM9gWFc+xkl/e2eACwaIrB09H5xn/9d lgvv+f0h1NytXOYuk5gvg8snOdb/lheAz6nwCclEo0gdmR6ZF/uLQ3t3jFyoKZSaz/GG iaYf0fnpyu3ypVYC9c1ot/wJF9n5uJdW0UvMT8pMaLghGc48i9nLKd+uJ8mFduXluf89 euKy6XmH4LCCuV6+RhJfGTtKHwzCqXHw/j10Mp+CO7xj1oPzb5nKsQga+pDGvW4l2UUR RQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2130.oracle.com with ESMTP id 37mafv082f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:16 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPZae124960; Tue, 30 Mar 2021 21:26:16 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2103.outbound.protection.outlook.com [104.47.58.103]) by aserp3030.oracle.com with ESMTP id 37mabnk7ag-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IPM3b9ZaEMP4Jg8Uge7O0htQi4ZK64dR049VxHRLl2lt7g8uqda6J2U94SuQeVdSz3tYAFyDYvUTYypGRmO0IxGQLO3N2TxPF2vfjrRUQb3FFGrlvyAc2efQXFVdx6jAGXs5n3fJCtWVfcYQtCt7CIEzH4Ah+xIBecl7KxfReVJnv1cl3zZQUr+ZFl8ECjfipV2RH34tqxDHqdGlgk7oDZb9BfehdR9F+UJ8djSGZAdWtlsH0n4fZ7HRW+a2CaMUOoA8AbzdItZ4n3dU7vD7lBmo7WsHnjUeIB5GS+FOvsXX8SzbmLEWIuPx/WzRT3/H9TY4nrdUQF6JWOZX8vu/hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dZRe5gfZ46LFF432m3qLr28Xysmg6HLZTWPgjewoNqc=; b=Sh0IAnQiPnV2lb6sgvK+4GZQJ2cXPZ+tInziF8VAv0qnq+8NYg2X3B0FSsU1AQuCWQBDSvQbZptKnodduIImyC5bn9vlxMhE0g+7JXS+AgUu7zQr+HvbdI29Mh4DQjabOYAxSMCeWgh8VZFj5dL5+ZYK6BkglF8Gjvme3E5I604VwAVCMkDA8IBNXi4aH2QVYfknRl2BKoDMrmELnZntaBHkvRalUzpmB3pSwHekWM/52GkwIc8xNBPEtZ3njSwJeuRsDoXN1QYVsCElps3haWjPlGpSodYVeQpA5UKhhsuxmv4K4Z7fxxwfQK71tL79LKT0AZ5VU6QjOl8ApAAjqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dZRe5gfZ46LFF432m3qLr28Xysmg6HLZTWPgjewoNqc=; b=uVwXTC+j+3cG/9ajXElcj9Y1R4UuFmxZ+9VvZVUBHLLEJuacEvWxxOfGt/lBpqKH5QPwp3lMs5xSRp4wqLzLEYlUj9+5bT8buUmI/ssszUuJIh9H4hGnHlTLObmffhU2oBiaAat9vgCNq5/1Nfj6VsWRhAPizHehOadr/MLL+s8= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:26:13 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:13 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 17/43] PKRAM: provide a way to check if a memory range has preserved pages Date: Tue, 30 Mar 2021 14:35:52 -0700 Message-Id: <1617140178-8773-18-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:09 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d042e3f7-f446-43af-c4c0-08d8f3c271d5 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F83kyNQ3xJw0JHlek3GoRwHVCVyRo7FQwiq9/EOAMzURpzzyrahmy37shdNvQdjAQutQnsQLnx1NgD05Wjk1PthWa59r7MNWgDTzpgES6Nn9Nqk1O+269SLDw1t5wboFIAzkqbikvzv9KT+APHJrDWXWoqXnKn1FLEOCd4cfJ00pdrYkrXB1KOR9tQ8nTol5Rbd4sOYzMp3iqVY84ImupnY/FfvTaRsYZy0hkIlhaDAmUUYwGQnNWQREGq44LttexEAY1w5OH6EbSxEwlFg8zfsHc0RZXjb4ZJVQ3chxgKkhiHwgUTgoG5NFqLuK60wRJ5B8sBl3C7CoPTl+vjis/TOhSELT5yUUeqVOS+6VGaLZIrkssBVPDWtxz8zhxnUoaYj8yZ7MsL2VfZrK+nYzI2r1yO4u2dpPW94IXmYdzfHYIa0kaxKofJLptE22ole86eeS9heZ22+gOBi5S3hhyiar2xpG8kPI/j8lWmn6w/wBHXfiGeP0IoZDjySUtWG7Oft6AwEa2D2lXFljpwv0mXQiYPEDK2hvIMXoKt/3GxAtcKbhNY3uIipEOjAU8rLVq4Ryn79cj44Yu0N/2c9MI5J4KhWtYPg9fuwNMh5pCoOzCKGWUaLIXCePGoWsGN5/ZStb4gGjLmEat/SxD3+8gs2sDSf+FFyOJKwmdswI4sA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(2616005)(43043002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: xdb/G9NWT8K8GuG59FWc03pHOr+ORZ3FM3v8rLTNXDzHQr2+tYq6kgzrP5Uj4teAjMkGWMLheirY8VXKGt6LTffjNeWBKNm7sPAnLnegKrzHZf1u7oxYgfLYc0GMQAtrawXg21NhbCO/tY4dNtS0vUYNLqjzmtKeIWaXGh2GdIWLKLXrE+7IbRoS9ZtGEIc6VpAyMaYSqE1gpkBrGqBgiN2hUh+0wUj5Dus/DpZnamevkKnEDbcluEWlZz6rY0WZvpZuWu43VtMx1qjdFpRXyObjXJItFNl2tYI9ExjoourZ2vyE1s1Ce2+8B+i4Fb+bBbPuHRrwL24D9ZxtdolKWfP0kI44Ju4BPwrYS3iRXP8eRIYkPftcmmmQLqRhtWjd9MzhjXd5/yat3fcj6YLfdhPHHo3qmeWHiFtPTW8LWfkO0H60sJCp3DS5x3ob5mEdeBYleVCYCq/wDT46Lb+lFwMCxXsTokshfXnp3Nv+0yVMLzmkQjbCjzAWPbMbK1HeDRsdGs8UomrCJdzwZsFtSdpnFYX7eLtYyid2uFgGIk9Puv5vT/3fMuWFAhYFmNvOe/59TnK/AdFxHq4OFPxB02EU1GMNNsb1FDoylTQjzczaUvRY6PJGusggPcZ0mimcwUCgf3J6s7HqFEuJL3cnG0htqwfzBmX0YaKT8dXi8u/p4WYYGUef/iq2IXDXQj8fGfJIaLj0ZFk4w9hsRRvCCCaPx/4FqS+LvpAup76H1kk1kyCCyJqsJ9SQRpinqoAb7P3xRuQo/CxMmZ/I3B8CxQWtQHXzlDi40x9yikre3nXSnwq/un429umOOGKWnjcGfiEUSv9CkD9ur1oxaows7ZYNY08X3pUoyzBi3pDkhNM0E9GIxUHabe7xluh+GCVPR28VmjUP4HG8QJCLhvymcowD3bl9p0hOIjAHxga+6k3K/apC0qJmQDn3nkTivprYso14U/kPhid+9BdvkeLZm+vbh40ri4/Cmj93ZvbCD7MO34z65aZQ5Be5O/AFvVwSUYrHXAz6t65A31SVJB0Z9a+ppbv2mwXAZCAjoO4DLqc7qo9o9ZSo6A8HnDqIty6EOFX4HXDVjPdogNZ+L3Fyvx6esQXA/6xU2q6QBNrmSIUJsZF1tDVv0vV57C9bFGL/H5MAxGzd8V8xKFoDiCoK4CMDBTyxaBHIVYwv42C/LeswKEou2cJpKoBweMo0Jr2Ykh00B42QyPQ6onTgT4Wta4FZnk0OzS8/LlK6GfiJ9AsPAM0Va9x556JPvd2ffXmp8RSWu8W9Sf0qq1sF3Zl1FTJHiFVCXIH1cjbAHnvobYYXX0PI6ewBREXiPZf3j3je X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: d042e3f7-f446-43af-c4c0-08d8f3c271d5 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:13.3584 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C17YwZSnkmCjf1BhxH90gEkHDbdJn5PLHtBCwYozrUJslo6bRgjZ8nV2Pd7OklZnKrGCFBpCIzFnXK3g0f9GFx8pcEMCZ7VpOHibXbDpVr0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: SgAlfM8_FwgFBGMyyYTp0xEuY7rLn9bf X-Proofpoint-GUID: SgAlfM8_FwgFBGMyyYTp0xEuY7rLn9bf X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org When a kernel is loaded for kexec the address ranges where the kexec segments will be copied to may conflict with pages already set to be preserved. Provide a way to determine if preserved pages exist in a specified range. Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 2 ++ mm/pkram.c | 20 ++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index 97a7c2ac44a9..977cf45a1bcf 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -104,11 +104,13 @@ int pkram_prepare_save(struct pkram_stream *ps, const char *name, void pkram_reserve(void); void pkram_cleanup(void); void pkram_ban_region(unsigned long start, unsigned long end); +int pkram_has_preserved_pages(unsigned long start, unsigned long end); #else #define pkram_reserved_pages 0UL static inline void pkram_reserve(void) { } static inline void pkram_cleanup(void) { } static inline void pkram_ban_region(unsigned long start, unsigned long end) { } +static inline int pkram_has_preserved_pages(unsigned long start, unsigned long end) { return 0; } #endif #endif /* _LINUX_PKRAM_H */ diff --git a/mm/pkram.c b/mm/pkram.c index d15be75c1032..dcf84ba785a7 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -1668,3 +1668,23 @@ void __init pkram_cleanup(void) pkram_reserved_pages--; } } + +static int has_preserved_pages_cb(unsigned long base, unsigned long size, void *private) +{ + int *has_preserved = (int *)private; + + *has_preserved = 1; + return 1; +} + +/* + * Check whether the memory range [start, end) contains preserved pages. + */ +int pkram_has_preserved_pages(unsigned long start, unsigned long end) +{ + int has_preserved = 0; + + pkram_find_preserved(start, end, &has_preserved, has_preserved_pages_cb); + + return has_preserved; +} From patchwork Tue Mar 30 21:35:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F916C43462 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37FF8619D4 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232882AbhC3V1h (ORCPT ); Tue, 30 Mar 2021 17:27:37 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49828 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232640AbhC3V1A (ORCPT ); Tue, 30 Mar 2021 17:27:00 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULNs0f130344; Tue, 30 Mar 2021 21:26:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=TqYQOsMOO41jCmvDvp7/h3hU+25HSuj3RbvPOn1iPac=; b=M5mt9jswrUqP/UBmzvnbI9F92h6+wcQZEy90DnWlSpDP1NOT80kkifuY4PODWEmSw7Yc POsOxu2wsdXyJ7mZOxDgz4uXlXf/BYOyJo7gUNyPpfEDwjrsLMN2f+k2keewMcoLUcYY rr6K/s8eNvHLSZzALuudBmy9gqxxcEgIdFJ1kTDTjTwKXzBgNc+RyprtPlyJrduRzSuk lFpCA2VHSL32uM9jbTKK4b9+yFHyCwP02MTf0rySin6Fe8DxiqpXztwg//ynS0ia+Y/4 FAyhhO1VuDyK1WI9U1dUi42GF388YzBqqOjU65whq/V85gNhT2dzAR3IaOQKkeK27t2R mA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 37mabqr8p2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:21 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULObBP149810; Tue, 30 Mar 2021 21:26:20 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2105.outbound.protection.outlook.com [104.47.58.105]) by userp3020.oracle.com with ESMTP id 37mac4ke2v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:20 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ix5H+GrtrJ5NWN1hx4a6kjfYidCgWndHNe9UvO5HIt4nkEce/N0+3E1zAd23kHLOuEGuf+9srzGkuyp4fpp6n4Uqw8gWxcxYXWLR7Aj9v4Q38OqVQ7hzTAQdKRENO7SBCjPOjXqBmOZHskQPgHMWdL6qBUNy8a1rsSCzqUbf8eNknlZdPpdWdKogoPTtjGmOYbvO6Y3TL7/UX4qMfH4M0YxA59f0eD3C69I83XXhg0Q1rjp3bKaUefF05jOoeZg9WM8ZC+yhf2fWFrsko66QZkgnuk7V0NQR4n8G3krdvxX+hQR57oTeRpEiKCqwtiP1mo/YIlmrSUedMJzIoJ0lVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TqYQOsMOO41jCmvDvp7/h3hU+25HSuj3RbvPOn1iPac=; b=flPzo9u+yzB8Xi8wajFjFUjIFLSRsGQORr1IvBzRKLOfq2wZ22cXNGe4Yia3uD0G0UZgtnexougL6xKfQfW3ISkMJvM8Dp22sEuNIu1OUSdRjCJZcWDima21WdmNtEnKsWcc/q7y2cSup/SwL/ftwjA5wxBW5V0lvC3taN+Vs1WLvErz7iGFgNiWWc186ogrvehQSOsYwVynHoh2sVgl+83iad7o2pjpSiTSSzU5Rh4ufJlXOcIPM6hftZG2HxvRgGRVrxC+nFqtLHytjwyXPEMFWluFwkDDCq4C7bBTmtpPc3RHVvBQyPRNVaHmhH0TZX/LtaqoGs6AWPKXRQmEWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TqYQOsMOO41jCmvDvp7/h3hU+25HSuj3RbvPOn1iPac=; b=XDn+y5FyqbhjHE8m97Ti33i4ALLXKshXOrKKsKJVeArk7sHIQjeO5uN6X+LWBXOCIPOvqPvOd29p1HnO3KTMB+aQTk4eDp4ZNATzms4r9/TOkdDlUD7z1pBULSU/KcB2PpwrumDuANjcfNIEiz78iHdnuuQQH/zr7Ex38dkcRe8= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3613.namprd10.prod.outlook.com (2603:10b6:208:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.29; Tue, 30 Mar 2021 21:26:17 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:17 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 18/43] kexec: PKRAM: avoid clobbering already preserved pages Date: Tue, 30 Mar 2021 14:35:53 -0700 Message-Id: <1617140178-8773-19-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:13 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: db216ab1-8c19-4d5c-2265-08d8f3c27466 X-MS-TrafficTypeDiagnostic: MN2PR10MB3613: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NHBppfrhlXzHZZYHnD/QhOxMceqsQboPIIr4Z6hraPIIGzvvG2aKvuRxXTM5zE6xZ5tLft7RLbcPXNiYoD/pRD5ha+6s+5hFHoF2RodrN6QRQU0NWbJobDQAosTjrUgBn3kfsSNkLOm3fzs50yPjZwPVHUH4/xdETU1Q5NvXqk0dIpJ5OZvp/nW44sY7SInCNQkBQvKbRtA3TNmsY5nz0BXsHqd3nTjk1cbwQEk4urL3A3IRa4bCdbXpzWItaQxeBD9NcxX9MvEDdq88jCaGA4VVgOQfgk+pd0ix3KV7b2ebB1iaFbPxXdYB2XOoUJSA6jZO1VYd5wTM4VobYAgXZyJEzsCCH+P2WI4GLEmufdb2r5+HtEDMyzvkOpzHusAg84rrpY2yN6Z7U5eIWuh69qxkPY/yb5IavZHN/P6uFCxXWFXvY7vdhDk3B8dFvxObnvRWemD62Ova6jGqxE6vN2Z7E5bt2wlX86td2IDI0B0cNXYJ7QoccowOMf263X0N6M1iS47CorGXEhDiTRar1OPahl4n7SyMeHoB2dgshfeSAtft99Jap96VXGhnyv9MSisU9Z5IiWj82XADjffEHS45Pt6Ape/FSKji4H08S5ARQBEyuv7KQaXDyTnjghATHhBvht6ypuy+OAqaNlDM1w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(26005)(44832011)(5660300002)(8936002)(4326008)(7406005)(52116002)(956004)(186003)(7416002)(7696005)(6486002)(66476007)(2906002)(16526019)(66946007)(478600001)(38100700001)(36756003)(86362001)(6666004)(66556008)(316002)(83380400001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 8FUbDs3YhWwwUkSiyNFYP6RRBgou/MiQ0ri/huH6fa9zL5cBjyHsf/6jFzUuQtl/jlKT17qszfAJoZBjkL+U1A3IdGLe5/FCffjLvkPLSoIUXb47XVnsncdmuuob17686x+8bbt++zaHBhYSHtOpjBugncHKjxEPmNn6hgR/pnmBlaxDk5g5gKdnB4aLf64Yga7/qQq+D/LiB+xL1E+5hB7Woos0xaj+QrR73/DCw59al7FneD6sKbfqAWQUJol0SnDUy5hXpV1U6opX8KxRt5/AjUYAxGHtiJXUCIsROKpB9BSYPgxhrCbaEI2ajBDSPFm/j/+xb0OL8oIBqHtFe+MfJtz3foT455Oo3Ar60nNTve0dhoEIyUC44b67btY89+HXZrGCCu1Wp+ADGF9d5X6inwJJbkM+5RRIOCI/DZ0mrEVabfNDGKdLFNMJy3TlCMoqUmVLNT0H2+Eoh+Vn/skl0W6N8D+18xGpULQUC9w4dKJ2gMw10v7S8Dy6+DHrukKe4GM4zMLdYKvEZ4uH+9Z9KkBtJC3/BRUuIer/VqIX20EwPzWIizMCnB7njiU+JtGjwtuupl9umVIbuW4KuccgW70Ex6KHBjmDzHf4OcOMHH53YAl5W96bqZif2zulCbHmL4oMz5ckKSCtsSW7Iho+XqcUVO95555SyQxGzFQ+4RqEVUZK8al/0AAid+HyN9hMvKo3oRqoDMopOowBqCUDdgFX/ofZWU3xo/n+fBtzO7nHfLmL+nR8C2ENr81aApG7lD0uGYxv4SPuqWe/YncdbfREg2GgQrMkWYLSXwSwOsz9vwrUZpj/kOYW+Dyc+t2/673LXGYk3U2LcQyvrL/mtBijlE0Kxi5sIbSZxgI5ssM71uDazQEa2NJugnvEc0oAB29FCGriGzzaVF+wpSMJjQOjksOxZD6x9obkJBDEHBhB6WZhxV+ZFqfjNQKiEoNgzYjl/NY04VlYFDaV5NFzsSor4XeVTxfHUS4B9CcoXiCCyf4vK/UH86q4ZBhJRDqqLTCP6XjU5dtMFTmTFP7xAvo25Qyy0G5t+CdLRIXS+yjK8edZhJYGf4HX9WWCaU3RhhELBLwseGqZCJ+oqhBKnrIHJLfGTqcFynyQnTT6Ybmb3jaRKwrDFmYV/d1Z26wJarUL0+0Yrru6Vdrs7L1+f+UPTv8AJBwX6ZYnn9AQXUzRWXFN9PY+x8//a9pxvuc7irhBmW/1jBVswzj0APX26BChG7U8WoYyBWecAoOETFEY/5N3VnZWP4hm5YPK7nUbT+i4iMjx3Ip8AAnYg/RElO8i/jWzOzWm/Xh8iYgdYr3PHHuz7TA5IY/nT+7x X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: db216ab1-8c19-4d5c-2265-08d8f3c27466 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:17.6489 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JAP8vukwnKNZHLwXyyzb7FFycp9YUKBtVWUFTYgh/nW/wll/FP4dK+nP/t1sJNsVk+Z5B6EbHrEbHnFV8f/uERhZ+5yTekTSWyuRTbAQiU8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3613 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: 5DBEe8JZBf-F4VqmMycDIcibzkt4Vii1 X-Proofpoint-GUID: 5DBEe8JZBf-F4VqmMycDIcibzkt4Vii1 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Ensure destination ranges of the kexec segments do not overlap with any kernel pages marked to be preserved across kexec. For kexec_load, return EADDRNOTAVAIL if overlap is detected. For kexec_file_load, skip ranges containing preserved pages when seaching for available ranges to use. Signed-off-by: Anthony Yznaga --- kernel/kexec_core.c | 3 +++ kernel/kexec_file.c | 5 +++++ 2 files changed, 8 insertions(+) diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index a0b6780740c8..fda4abb865ff 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include @@ -175,6 +176,8 @@ int sanity_check_segment_list(struct kimage *image) return -EADDRNOTAVAIL; if (mend >= KEXEC_DESTINATION_MEMORY_LIMIT) return -EADDRNOTAVAIL; + if (pkram_has_preserved_pages(mstart, mend)) + return -EADDRNOTAVAIL; } /* Verify our destination addresses do not overlap. diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index 1ec47a3c60dd..94109bcdbeff 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -516,6 +516,11 @@ static int locate_mem_hole_bottom_up(unsigned long start, unsigned long end, continue; } + if (pkram_has_preserved_pages(temp_start, temp_end + 1)) { + temp_start = temp_start - PAGE_SIZE; + continue; + } + /* We found a suitable memory range */ break; } while (1); From patchwork Tue Mar 30 21:35:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFE2DC43333 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C6A7D61A23 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232912AbhC3V1l (ORCPT ); Tue, 30 Mar 2021 17:27:41 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49990 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232634AbhC3V1S (ORCPT ); Tue, 30 Mar 2021 17:27:18 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULNipW130302; Tue, 30 Mar 2021 21:26:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=yDLRLnFTPQdctAMZcnaA/yaqg3MDXIWikseBX1dUNEY=; b=DgsQHnDDCS6pEshRFyMtvkwK3Wa0oBdlzobJFg1w+FCLaMY5brkw4jeaXwAA+b0n+Ew1 Oc/GDsy2U3G0FEkHcI7O9GEALMEN06cQfZak8HgrxSIdOKXDVLYAqEv68nOkLZpvPe2v 6dqg/TnLPC8gtWnP4WW762EbnB5u0gVSVO34idsfQLSy9mKLV1ix7ZUlkNIk9be/cxnx MJoVI8wpvok0a2pJHAPNJnt3SWi0a2OO0T0l7nvVbbiOm9j1QFgvCcuSaeNV8XSrIFRl JbLEfiLJ/HlAEdu5A50dBlVlX34qirLur2pOCiSsn0AFs94Hw8H31ygoy8OTlx6eOM2T Sg== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 37mabqr8p6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:26 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOnOa184080; Tue, 30 Mar 2021 21:26:26 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2106.outbound.protection.outlook.com [104.47.55.106]) by aserp3020.oracle.com with ESMTP id 37mac7u4jc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F9aIyh+rzc8JG5NcLpgVyBk0+dmxAkmNEidhBIFjLAIiyNiPj0xTGplcPM6Z8rDFLCnCP0zbELWP1zdiuWCLyiPt+gJ9YIVT7EZzG/hFFX6kNJevQ6fNZujn33DyJJmGh1TeTCePQpwNtzTzLG1JBk/W7pe1gYbjr0RZnk7T/7Bj8PylmMZbfOetHtxQUd7dA/pyROrUXihIaeoeIpbRxYair54yr6or0Kc75yeD4efbwUehMNcw1pMXBw80Pxue8CH7rgmSIdqjEBOeSswbkoi6UoaoXaphJXMMEh85DoS4cFm7cTy6WRqXq8J/wWMdZjhvMVC2Xp0p2zIQVOh6Ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yDLRLnFTPQdctAMZcnaA/yaqg3MDXIWikseBX1dUNEY=; b=NokgA0AwQQZPdYMsVQIYdI7pCYtRTKH/qP6zq2mt+tL7yP44dGiIID4Et+zpg0RVAztycob6b9Dw2ocf7DXKEuz7LFp/tjRgsZgrlrEPRbXKF/qdkf7lcssXACu6kZ7eA3f+cvG2GxhJFCnkpZqDsHzNuC0z73O5onN2bPH4ZntZ0BQiK/sVisf2ZpCf0dJweooctxpFgHgxUOyj9xALalzyPZHxPJdLVnAn2kx6E+RtX1tURHouqBDkLU/yKK4TqPwRQUjdSmU/6FXyQYKN1Fy9nUqfT0ozCDRCZTjwbhFlc6+EUoZcJHSoUQaHfhGOS2TcH9AJITeQHeAGZj2wMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yDLRLnFTPQdctAMZcnaA/yaqg3MDXIWikseBX1dUNEY=; b=cpiWQRYER8WlZKIptvBBW5ajuylUdnkkiNfuAytYDCYNNpgtQoehPy3klXpMtgM1CkIgmD+4vDZB3hhC1C8EgveS7bFunDe96+F5iH34tlqJdj85CVxX+gTwZcZLCujwXpjKb586bwosxnnUTt0ht1fhMtWfcXjdw+ZYNexi9qs= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:22 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:22 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 19/43] mm: PKRAM: allow preserved memory to be freed from userspace Date: Tue, 30 Mar 2021 14:35:54 -0700 Message-Id: <1617140178-8773-20-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e0fe7fd5-ae02-4534-9153-08d8f3c276f7 X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1284; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PrdhDpxsUyxEkpnXeE8OhuoScRXECCtbqVJiYrESeCo0cfVsoxYmH9MP2sPUT474FhW7tSkcoRkQI+vp8bHbEJXsOjjphF7oU7Me3xpQUd6/veknLq84+6Btt82i+NQhBBi29b1UkYmwFMMhmga4IktFjh4JmcStVoWyoQbV9GsC35g4J4uVYwicr0gaRrJDH6hcyS2P9tFKggC43qrWkmaI1mwSdf8k+jFuTI83FLmwKAo1huC+lFxoigDp2vDuMOA1wETPOtObOJWGoV1VgsE23X4sBGz0EiRksyFoAi0ezsqJJQ9C6//TvM9AbuEiK//yVENHtHdchhEWle14HEBqEgAHMzBwjooEBFaVTV0YqBkK87NZp1drVUC+CJjsR55SPkj+NeCMhLwLFXX2k+KUmdm7zqKalNrbxbbR7yNETwjnCUFE+CGrIZQLJ3fub86D9AbV3cdhlRx6EQhK7McDydGmqYfshERAz6ur5VQllGgWUwDeAmPjlcfsCzG09XPpuX8R8EQBS07wppxTnphDcjmeYjxMrd98POeRO1voR6EPI1CQOu3DRxo/irH2WGdRnVrE7YQhLDgAwxsJJbk7tzL3bLUVQMjLuUqIGZdmWRJZ7QJD3dMLy28eg5rapeYSa4vb1Iiaq8zDLwIsCQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: zGgcDEKHqfeVCT6VOZzkY6o7Q/fGlOllFxgjhXVxYC/1UsKgvTjjTR/d1YlmezhecaPUG3qGttqpq1Ty9yQCSaMPyHFX1soNRESq5DCim+r52YmKQy30ZhIFqS0EnoX9MnuO0cz+lApazd6tubMLHLqs8ZEozAo7584lpWyy5PqJWIi9eaPo0U/mmGbegw8rT1rS8rXrMpXzqm9HA0B9T1YsS77RL4u6xh/XybYUd5p9MUCqPsU/5jJxceBbjfeCG0/bbE/2iEw1CK153wfl4r9aGVjq89u88BdR+H8Z688Ca7j6CJZ1KvlrCmNf4F5b+7DzYX64Um34HG1l4SEX5W1fiOgTiMZPjmnEhreoi4VEOG2kEj5SWWmOODKh4YyWfGCQTag+zHmcdPXyI0rgk/FloyzzIsFpmgoLFLp3nze4xrNMUvGE31pCOumGEmMyHkyNCcF5XRnlGilXeuVtqQSr57VCmIU0vof5O/6+h/7bx9PFdy9QXWPL6sxeFdQXp1f4WPZxsiJouM3EEUSdnVRjvskFdp6MfaYWrogUaNd5RdgguAHrUzlF2aZjOGfHkixZft/PBZFpfTNVzDKxGuvacxyEIdMKQZGpjrOKp9LMutjn2kZz8yLi2xpgl6sXKpXyEcCmLXuh0MUkbSZZL04yBDv3i+xVLuMjyveycI4WsSUwnkVaBBHX/UjjaZbUIiWwMsW8bz129ASShWiSCnNPOxhe7VxR8DC7xT+gr+mD6HagljJrImhyEO5wL51dWn84RjDnkKnjUCrF414MtT7eHpNbZ5Rbmhbd7ZhPpEIV8D/IJdR+Ip3OGMLK/ZF+hQrTA1AE9MTkNJ2T9NpmTdezWHyN1TYjeoQbX0cawAJOLhqesxcwAiUMGpJpyyil+cKyRblrV71yVMuO+awQYGheBDSvGJuwOGpMO+8PwIkPDbyzU1Ewg0UKS5POpab43Z5B0v9p2YiSGmPL9T6Kgypl9+aUNPLR9Gj30aqAcMidSDGhXHpXMJn9KQmDGBOBBvrBUfC7TXGDzGNAi0W86lHhyedNEiiHDX9sJ2e3jVNy7ughbmBKpV+tYxFdvIEHP6dJl8nf6dDvCTzCcmWFVKLcEDkOG5wChojzx0uid1JcbLihdDxG/vGfR/Bzvy+NxsDB/4wbfgdtwbIlMSOke8EVMtyg4bPevP7+lmLMMo7ENXQ00Wqm4RWfiCTHpT2yV4H28t/BHGnafQIOV358WkMK/w957Rn+Bxs94F3gFJM51JOVJ+S5KmDHGLUFIfHbZNJp56kBQUof5GPFKX8dToECca0NqpKvE9XZAqRpJxFJR5guZPuLJszhY3qcdIRp X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e0fe7fd5-ae02-4534-9153-08d8f3c276f7 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:21.9715 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TjI0o54wg1wET4H4pvLmJOFxn9DvnTgp++5XWTRmE/7EHsrdO/He/08Z3hnGMUYO54Bd6PAmaC4mY0LgnDHmId/vxePsT3oAz7DXLwMJeV8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: pMc0HMVeS0xDmcIluKiakYKWipZMKCXj X-Proofpoint-GUID: pMc0HMVeS0xDmcIluKiakYKWipZMKCXj X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org To free all space utilized for preserved memory, one can write 0 to /sys/kernel/pkram. This will destroy all PKRAM nodes that are not currently being read or written. Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- mm/pkram.c | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/mm/pkram.c b/mm/pkram.c index dcf84ba785a7..8700fd77dc67 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -493,6 +493,32 @@ static void pkram_truncate_node(struct pkram_node *node) node->obj_pfn = 0; } +/* + * Free all nodes that are not under operation. + */ +static void pkram_truncate(void) +{ + struct page *page, *tmp; + struct pkram_node *node; + LIST_HEAD(dispose); + + mutex_lock(&pkram_mutex); + list_for_each_entry_safe(page, tmp, &pkram_nodes, lru) { + node = page_address(page); + if (!(node->flags & PKRAM_ACCMODE_MASK)) + list_move(&page->lru, &dispose); + } + mutex_unlock(&pkram_mutex); + + while (!list_empty(&dispose)) { + page = list_first_entry(&dispose, struct page, lru); + list_del(&page->lru); + node = page_address(page); + pkram_truncate_node(node); + pkram_free_page(node); + } +} + static void pkram_add_link(struct pkram_link *link, struct pkram_data_stream *pds) { __u64 link_pfn = page_to_pfn(virt_to_page(link)); @@ -1233,8 +1259,19 @@ static ssize_t show_pkram_sb_pfn(struct kobject *kobj, return sprintf(buf, "%lx\n", pfn); } +static ssize_t store_pkram_sb_pfn(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + int val; + + if (kstrtoint(buf, 0, &val) || val) + return -EINVAL; + pkram_truncate(); + return count; +} + static struct kobj_attribute pkram_sb_pfn_attr = - __ATTR(pkram, 0444, show_pkram_sb_pfn, NULL); + __ATTR(pkram, 0644, show_pkram_sb_pfn, store_pkram_sb_pfn); static struct attribute *pkram_attrs[] = { &pkram_sb_pfn_attr.attr, From patchwork Tue Mar 30 21:35:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3803C433E8 for ; Tue, 30 Mar 2021 21:29:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8F55A6196C for ; Tue, 30 Mar 2021 21:29:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233218AbhC3V2c (ORCPT ); Tue, 30 Mar 2021 17:28:32 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:33406 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233061AbhC3V2J (ORCPT ); Tue, 30 Mar 2021 17:28:09 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPOZp123006; Tue, 30 Mar 2021 21:26:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=VB/rLeMWS58+QfyQWQSylpbvo0MGkJg4DhLXEixjI74=; b=VWzwP7QHWl8aJ2vVXEbP67wt4oG6dQl8KBBBjMbae1BShJtmCFZOAtPn9T0yDKwXCk4H dhYjMeNYr2G07GSrJBYAv29BmmsQZblLk+jdlc6d7uCuHnvSCB5YhK0hTE1UdQYziCLA 6k2j0qqpwcuX/5JcO6j9dRhIpuiv3dCD4+bfO2j7dt8z8Akh7IOAgCYt4KMw49swp6/S vd6JG6evgApq10DkTJy1CIE0BJY8HQROMoevX2XONRdj69p956ICrJJ54zlIeh6H1yN3 EsZN0EeaWVsEzs7N1AuEk+aOMMAGQLkRK72w2MY5DZvE3g2XuCioo/pkSaZqRrjaAgt0 Aw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 37mafv082v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:30 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPSE7105894; Tue, 30 Mar 2021 21:26:29 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2105.outbound.protection.outlook.com [104.47.55.105]) by userp3030.oracle.com with ESMTP id 37mabkbch1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:29 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ll7EMiIq71Pg29Yi9h9EGYmUOqufBeYItRi4iijhWyl/mjxpJ0O3HOToBBhN8x8uA6vsXuAC/Zir3yhPoijJ/B+7Ou/I9XA0CnGXWhbL5Ne9YpMsDg3qJWuUePSqkcfgqQAgVkBjPWoGJvu6W96LXc3ReJSbqtxeADnwL92s56bLF2veEpFHF/uYMrBAy1dKFWtcXIeaxUdxOiyylBYKxD9NEJPG2UmjLV8LLxZl6yUsYVhkLgZCiYzpz3Umz8QqYuIy/1i+QsP+2xtmv3HoGlcOntJNon4KKXxQ1qA0u9kvJmBttdfVeETVRvWsJQ7endd9CIX+W+sfrlcrhdP+ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VB/rLeMWS58+QfyQWQSylpbvo0MGkJg4DhLXEixjI74=; b=hvF/OBct6VIQZHBC8RQNIGnY4u29MnkaTAV8OWW9lPMUD0EgOnzRzUn/T2AgkXPncD0FUA+tWD7vKWceV+JRjDAWveYqX4ssfgloCCdLaXgpddmNa1WPgZqw+PrqNgMkWpmCe0OJSglxl2uWvvRXAMhfnSSAqjxqmpFGDtQqDPuvreb4RnLGhYamdhcYJ8tHIpVGMi4wL8mHlxnvKyvJ5OYKspznkebSOpP7Td/qiCoDH34TPKtUIdIwGRN0tn+xMA1HFHTY3jGtROvAJRg8dc3ZrDyYgmVi3kbzoIQFUzqAf0+hKucQFlb0KlgZ4SBUbmhrKWee74vWtpMp5nrSDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VB/rLeMWS58+QfyQWQSylpbvo0MGkJg4DhLXEixjI74=; b=LlA7K8vkkrVmZDSJqFsnYZwYKyNThxscjAiV95w/1mCQxs9zj2piU3cXU1by8iXIGpze2D4R6b9QtmJw6fTxL9zJ97+RtBtpfrT3EM/OBAvz9tCabhGIuzjZyBgCaOSfeMSQkN1J7OpLJiHU7rbvHbYwtlLO60A6A9YXJyj263Y= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:26 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:26 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 20/43] PKRAM: disable feature when running the kdump kernel Date: Tue, 30 Mar 2021 14:35:55 -0700 Message-Id: <1617140178-8773-21-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:22 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 694d7e04-2424-42f9-cb28-08d8f3c27997 X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3044; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cscEe9RqMCZ5Bk5eXmFWlUnzGgxgXMqA7SRpa39rNzkwKQgIzt2yozl6N3HI/q/487CHqHDzCV+goJ/hFfB14NfKAmxZOTCCeK6d22rRcZVRwg8Y9UpSilYeMBhftircYOn700OcJAUv8pnEOCcLZ8ai0Vuaq4FHjaGVmXZlhZsktrydjWexJhFBaUTMJkLRRuyW9j0sqPkRqINmyAwDhmF5iR27x+eubZDlEih6asT/fhId2Oef6RaV22ObJ1F9MDjP3w/QRLWwadrBHVjBTaPyOH7yZsZqtOCL8iFQ3jEhUIq+440pE+hZM7YLd/27/3jw6VNOIIYzVQJLxyl2jrCuBWBYVWFuaXAJsOfOG4LOKh1xZBTvXT5+hx82krXBTO0eDvBbvq5hEex2nJ8lkVtTsMaWTPdEkTVfTV51VfK8fb54CS0KD/wEwcNyc+bvfSvxN5GzK+FUBiwhtLfzguyANne4NkfBE90tFAujpH3LvXJVYraOZEmBXYA5LVoz6JlPC9JE0wxTiIZU5eTPqDNaaGIo6s4iNv/RtqH8LlJuZ271i74VEcXmoVyGTGBy/Sb0/CMsllc5jSuk+7jhkWofPlWiv/cdc4UJscViesM8tzz/Wejk/d9rA9S4VPzg/gboRnUS6+/OkVBwQ04H4Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 90xYl74I1NhfZOaneujZl42FoE/NCf/6pXzgSvXLQHN1XHgT2Sq6e1lteoaxthnZUSBCIUABWRoUjFIYl1owth0Jak11uhGFZhpsRtm6cob2dd89or3d/IH+BSsXgmPndyNgrN5BQHc5ICCJOK6Q+8Iq9NVuUzS89P/nIkG/vXoKa7X1gG2F38tM9+Qv4E7aW3HVk74CBoL0+H9hnwdSVtDepqnsmVF8djxhcgJVg3T3a+kx74kFOpLMsPKG1TSnCp1VDZlY/35tgPV6RMbVdlNZ4wkJFkFNVyy+TJDSDH+qUcBwSVbSqyVssnMtquTtuN7gIzjfAPbax0vb1h2graizSjcaWD/TLMhHXtULty0NDj2P4R8nYDgeZ88gqHew6Nlaz8oWgiyrFPs6Db4xVX1e4b9f+2wkxxWrqC6LvViGbVIZNl0DIMDoUw/veHAmwzah8xCbTNbvb1xxxTVzptTY6J7TJwwsV6le4Z5y/IHtAU0kJ2fp6Mj6sVYkwBMT0StHmcnq/jGGpFWYfWogkjzHkiyDCnCQN/iGK6pmVQmNe17771OLorik24njRj/EbvBqqOkwJArdedmEPEmhBf/Ja3UYXayGBSfEqUljimwkny6G5i9cGwRD+vQ7IchMrKKeccPTmXxSbJaS8eXs5flj0utSIPq1VYUo9/BEA9JBJngf0vtGsjo3tXsdPJCC8c/XH5XHqWJFeB6GlG7PadUw4xJUX2CcO+KR3JYPKqRC8b6o/r6HAYHedo87JfzpFOqs3otWJm3h1VJ2rJ3fnFjcPqETKuuI2yoF0TmVzcIHGawdJX9Qe0aHsQ8/Rk1jREPUJdjayQGWJKdfi0wID9omAoAjuUf0B+h1UHkkMqpJzupESFwfHUd/T+JSyhIQzleRnJkSAQPaY2fleFs9qWVsMdmDc/qKnSs8eyR3qqpF/nhzTRATrabp77kh1C5CFzVaGMQb8eRilxd4O3WzUXREv+p/tITdbEKG+2m95hFFukH3rZQcjOuNqeo/Kk4SiwSwEaLnhHlAl5U6ofyV66IyWZ0o2X0R/Op0AmYPgTP3cShqKqY/Ws/4iO1qrf8fiIjW4+5Z9SH1LDB6o/LMy+F2zMko07/ukeGrNYjCHBG0/si8eo/4W/Bv1liQ7MBlZBeK5/LojcdJAZ6iHXy1AQ48svuwC1b70JNQUipb0OHZtgsHFzQBi2AsfqmTpUgqsef21HWg1LFfJQYc2Xo954RBtnIWnTfcPqdEIPiuBoXtRjmN245VWps9kVhxMBUU7OCtavyQEgBOyTYADvZh7+PM6VHvOdUTEd8uLL45Kyd1cvE0ceJGYtHlUzLy6VJF X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 694d7e04-2424-42f9-cb28-08d8f3c27997 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:26.3710 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Vfin1igwlWt9wiJ4v1BKopZoQBjjZduZC2nMyMxN2YZ+SpDweDKPxb+gHJwkmUpOzALThrdKmpilpRXGMbzI/S1yl24jEiW2yUD2oSRj1jM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: Z_PvSgSATERsXMqEhJpVmcS7dQd3zSG3 X-Proofpoint-GUID: Z_PvSgSATERsXMqEhJpVmcS7dQd3zSG3 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The kdump kernel should not preserve or restore pages. Signed-off-by: Anthony Yznaga --- mm/pkram.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/mm/pkram.c b/mm/pkram.c index 8700fd77dc67..aea069cc49be 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -1,4 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include #include #include @@ -189,7 +190,7 @@ void __init pkram_reserve(void) { int err = 0; - if (!pkram_sb_pfn) + if (!pkram_sb_pfn || is_kdump_kernel()) return; pr_info("PKRAM: Examining preserved memory...\n"); @@ -286,6 +287,9 @@ static void pkram_show_banned(void) int i; unsigned long n, total = 0; + if (is_kdump_kernel()) + return; + pr_info("PKRAM: banned regions:\n"); for (i = 0; i < nr_banned; i++) { n = banned[i].end - banned[i].start + 1; @@ -1315,7 +1319,7 @@ static int __init pkram_init_sb(void) static int __init pkram_init(void) { - if (pkram_init_sb()) { + if (!is_kdump_kernel() && pkram_init_sb()) { register_reboot_notifier(&pkram_reboot_notifier); register_shrinker(&banned_pages_shrinker); sysfs_update_group(kernel_kobj, &pkram_attr_group); From patchwork Tue Mar 30 21:35:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9FB4C433E1 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D5D3F619D4 for ; Tue, 30 Mar 2021 21:28:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232918AbhC3V1m (ORCPT ); Tue, 30 Mar 2021 17:27:42 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:51210 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232700AbhC3V1U (ORCPT ); Tue, 30 Mar 2021 17:27:20 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULP4EA145279; Tue, 30 Mar 2021 21:26:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=5gs3kt58j9UHE+if8NBT4rT44pX3k6PV7VfaMeNhiWw=; b=cqvkyZE4IhinoTwc7pt0xiI1K0bYprXLSttf7cvCemG9SzXQlWtuAF+QWCvTICBZd4iX JG95kTeVZg7LRR+WBNYlaSAzs5NCGKmOrPHuiDmpF7pPS246cShX/00+MSypHWzL9bmL Iuq3nCiu0pmHuEWOD33YhmikiMHOcTFO3VeIyoMVjRGCiSGJoSsrlW2XBcxR/MkqsGCe u2IZ4RAVnrUGTahinKNRegXw58HDN0NmumclCWgKMDrWqKkv7YvNG7PKe/05mTXlX6th TA0Dx2avnI75DdCmVKi0KLKZCBiv53cYtDBsWslDL8I89s3NRwpB+RdUjGbMJifMkyPX Fg== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 37mad9r8g6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:34 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOmHt183931; Tue, 30 Mar 2021 21:26:33 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2103.outbound.protection.outlook.com [104.47.55.103]) by aserp3020.oracle.com with ESMTP id 37mac7u4nw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FHuCN3PChu9hQWew/mcj3ugwv2uZ0BdGqa7NinqLrfJZn/6X7w2oH/l0MmAkn5AoRlOEzWyC/nZe/IG+sIc+kDnxrhwoZ7U9aQUcHvw4XrU1gv/THGBlw3mAU5K4aEVAMfzA01cdkkSV2UEjWfNfPhDTRPgqjtrpl9+U4QX5SWcEfZ2LVMenYguGcSPn6vLS9nqP4/aBfIYRPTiRbnfNXt5FXbvmewT/9b76/VZuUhVWSJaVWGbzigtP0Uj02Cuq0KcC0oBZRZJoKlI6UCHvEn8ND2VCesMygmv4GNE5alxh2gZetv1NdYXlBfjJ/3fZVhVLo8WPwLtBMYBUu+cvXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5gs3kt58j9UHE+if8NBT4rT44pX3k6PV7VfaMeNhiWw=; b=XKp2mCpw8nLcOfBsKpw97N3TcdgJL+Sho3DGKT7MgU6o96xlRU5WRDnIfhIc6etfsZ83p3IUYUqC+U+lGyrP08MyyDPfaioRL3UPOdOk4PeQhNT6T1fs0mJoQOOm1qPSkzmvBXPXwxQ+H0AvtyEObXCNdHyUt4JHOQJTimaoRKKfiadMUMryFZ7oX4IeQGIYjX4Q8XHYCx1tWMYGUqI7wrGD8cKciJumwNXbfyWgjooV7DHKLpB7P4mMYYF68X1juaSJh+AIp4ScAnRJcMexQyeJU3IuG+t6194FKkXiD15ylUx05xUGCwQohyx98x9P65H1NWPstBgWSwtHM0glNA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5gs3kt58j9UHE+if8NBT4rT44pX3k6PV7VfaMeNhiWw=; b=NXEEMxPP4G5vT9nsmqj+S+m0SNYcQARtOhuaEaNGLAhraSDG64py2TGpMyBqD9lTOAKmXDZOtyN1Dhx++ONdvfMv9P9wDJTYtngV1md/bTsLUJ7wXETakhsqq1oiF8fjGeMveVN2Hhc52k6wSBBtwnNV+Zh+hI2xKQy11+K3/tQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:30 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:30 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 21/43] x86/KASLR: PKRAM: support physical kaslr Date: Tue, 30 Mar 2021 14:35:56 -0700 Message-Id: <1617140178-8773-22-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:26 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4ca0fedb-bfa2-42e4-4f3b-08d8f3c27c31 X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2043; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9HZHZeizw7fGjde36AqABFq5MkOakfXE1/Uvg6KbBOk3M8TEJNhdFucyJ2YB72ecLeJHD1em2xoRVztOhITSVN6mnLAmjCHSM553GWgP89v2Y+u56IpMvOU2sAmvCFB+7SGOqi7tZBSehMdyHHpVAKJ8GhwGSq55AHESs7BjZ+LetNl80bmbdRfa+CNc3N1u1KHfto6LxlaXUUDG7m3HBOQDg4UZIu45z8JE+vtDKSG1CZ/gYglmBsECc/0I3LuhdsDfrNA3/yVKjpBSN0MD2kE61nJIfW7L/1RlpFREd6VaWRALkkQYsnrqMBu0zoH9XJahbfyxr0TP3gd3aV27uyh7QV8JoN12n0Y1QwN/b/4yad7xRSunDy6kPw3SKl0huXzhKlM8P1bE8rhxyvzog9+l1qos8uBuKkrtAbe2sWw2LPCt7DQodh5ou5anRhNNpP0dG+2uKeN0hMCowoWjz+oEsW+wziGyb2OwhI53rjJ4a2XpsQTHZnVspQqunfq3xWKjiZ7WB9Aid/Vrb/2aKck1Tt3akMl71C70VSG42YTVnXoJl+fFFKkxLgLbwVAGVYzEUMBDH6K7cK9zoCJQ+Ym0upnUFLmHeZRk5UQf2SnlXFuwBjZdBq6JlaN1r46EMxPaL05K78W+8XCzkD7c/A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: xkgSBQsdp+OWdaNLjlR0zWKuROZQzqHjAK4ETlGY72ojit/WtQS/8bruVXEm66BF+mPsE0u/gtmBSNvt8HTZ20NR4H5bPZmQ9L8w4UBykfTrtUhoQk73juxnZ9uWQl+Idi2SWjt+4upXHiPtB0ZQDKboyEIAeJRw+u0SBomKXcmV9HCs7+4usP8IM8nx1PnVm6y6Dlq/eKN/IRoNabP3bCMz1si8yGNZsUlRXiG3vlT3ugyXQGC70nyqYM7xh1sO+4+DRysePNSLo5wwgOUIe8wCh2A6kfdHL2uWuNo12+vwgdH2OENv9aqg1JE0SdWdGMw9bIr56n5vMBLju8oxRpK7iYbdoU43Q+udcnyeKAq5NkzBxF4CHsgAcwpQ7lWzpFRe5kOcbU5VJ3J6DINJjmaCaQdfgKNnhbi+NQzXJKZ2DxNORJi0SmnPo0fdnF7s/hNJEBs0QWNbUJaZiKidg1p+IxZK+9EjzOnFsy3MOerQIOvwZMQrBVn8J2pxxtE+MiD07og723/k124UDOaERq9RGgGPQmiJxG2tSy0NSl09awhuvinQS3OPZV5fxu5GgVeP/SIuY0nePURxHR/4cg7Iz/DrArsBVdGFDmBlSyWFL3uMiKEh7TyP/vIZ+LyW1b83umFgyaptwUxGKmY1KPyuemWpxmfbdRzKDijB+ArmnUfO9WHQSWOSFZUFjOdt5E+nL80Ea8Dqf4Dg+HWFU0kGshhwmn5mE1tfQeDnHLv9kzBTqyg5joL0zgAcLvXL6gomBh2INumGkDu7VOBxcXv8be9+okZr5jiZTHD2lCcob1QpjYue45hFKIErCXzT7zwDiyYbReZXabXDjv5dVPR6nzAlRHOPzxVN8L0qqcRZm49VHAWGP9n37KO8GEEMs/i6K8IjYagVQNCEcLBCnnKWL3SsCWF7IGT45DX2mVhucfBbVNGVI2zDoY7pTY5MBn3DixRzLFeU9dI09PCYarAkWqIz4rVc/1PbWQfW82YDc9eib/auEff/Ry0KIDKX9vKoGUy5PC+cdZyoNizDkohK6Lk+gLnfGzpOMFyYyHFY1g5WO6+akPwl8b6Qy7KjRJtsrdMqyS/tOGLgcVBMmSrvyqcI94vId4C6qosFtcbu749t3fIZDRARfP8w/+LFuXI6u4RNl7Vvdp3ayHCgOTOUMWoPTaiWesU8xi9xHZG01fmC2qOgRBlodBaW5kNBhs3tMVf5WRwoWXbpf8J1/csDtvlIR+4cSPoS6wsiAKFE6wycYT0f3fHWY415wtrl53X3Di0k4ES78Osida/pqfm+6jH+1pBBxjWD/lJclzhdtOJqzT8S9fwENoEqylS5 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4ca0fedb-bfa2-42e4-4f3b-08d8f3c27c31 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:30.7415 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cDashneOTBdN/vrdiqbxGCCf25XO7nrV7Yt0WLm3Hdj3f8fM2NS2cXB59kRiDkJx9cMAyxJP1le68Fy2ko5/PbkmsFbHQ63vHTDxzsqXDFA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: B8bb_ymWzzcEVCO5qZuiU-DZA_fkOWZ_ X-Proofpoint-GUID: B8bb_ymWzzcEVCO5qZuiU-DZA_fkOWZ_ X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Avoid regions of memory that contain preserved pages when computing slots used to select where to put the decompressed kernel. Signed-off-by: Anthony Yznaga --- arch/x86/boot/compressed/Makefile | 3 ++ arch/x86/boot/compressed/kaslr.c | 10 +++- arch/x86/boot/compressed/misc.h | 10 ++++ arch/x86/boot/compressed/pkram.c | 109 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 130 insertions(+), 2 deletions(-) create mode 100644 arch/x86/boot/compressed/pkram.c diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index e0bc3988c3fa..ef27d411b641 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -93,6 +93,9 @@ ifdef CONFIG_X86_64 vmlinux-objs-y += $(obj)/mem_encrypt.o vmlinux-objs-y += $(obj)/pgtable_64.o vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/sev-es.o +ifdef CONFIG_RANDOMIZE_BASE + vmlinux-objs-$(CONFIG_PKRAM) += $(obj)/pkram.o +endif endif vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c index b92fffbe761f..a007363a7698 100644 --- a/arch/x86/boot/compressed/kaslr.c +++ b/arch/x86/boot/compressed/kaslr.c @@ -440,6 +440,7 @@ static bool mem_avoid_overlap(struct mem_vector *img, struct setup_data *ptr; u64 earliest = img->start + img->size; bool is_overlapping = false; + struct mem_vector avoid; for (i = 0; i < MEM_AVOID_MAX; i++) { if (mem_overlaps(img, &mem_avoid[i]) && @@ -453,8 +454,6 @@ static bool mem_avoid_overlap(struct mem_vector *img, /* Avoid all entries in the setup_data linked list. */ ptr = (struct setup_data *)(unsigned long)boot_params->hdr.setup_data; while (ptr) { - struct mem_vector avoid; - avoid.start = (unsigned long)ptr; avoid.size = sizeof(*ptr) + ptr->len; @@ -479,6 +478,12 @@ static bool mem_avoid_overlap(struct mem_vector *img, ptr = (struct setup_data *)(unsigned long)ptr->next; } + if (pkram_has_overlap(img, &avoid) && (avoid.start < earliest)) { + *overlap = avoid; + earliest = overlap->start; + is_overlapping = true; + } + return is_overlapping; } @@ -840,6 +845,7 @@ void choose_random_location(unsigned long input, return; } + pkram_init(); boot_params->hdr.loadflags |= KASLR_FLAG; if (IS_ENABLED(CONFIG_X86_32)) diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index 901ea5ebec22..f8232ffd8141 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -116,6 +116,16 @@ static inline void console_init(void) { } #endif +#ifdef CONFIG_PKRAM +void pkram_init(void); +int pkram_has_overlap(struct mem_vector *entry, struct mem_vector *overlap); +#else +static inline void pkram_init(void) { } +static inline int pkram_has_overlap(struct mem_vector *entry, + struct mem_vector *overlap); +{ return 0; } +#endif + void set_sev_encryption_mask(void); #ifdef CONFIG_AMD_MEM_ENCRYPT diff --git a/arch/x86/boot/compressed/pkram.c b/arch/x86/boot/compressed/pkram.c new file mode 100644 index 000000000000..60380f074c3f --- /dev/null +++ b/arch/x86/boot/compressed/pkram.c @@ -0,0 +1,109 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "misc.h" + +#define PKRAM_MAGIC 0x706B726D + +struct pkram_super_block { + __u32 magic; + + __u64 node_pfn; + __u64 region_list_pfn; + __u64 nr_regions; +}; + +struct pkram_region { + phys_addr_t base; + phys_addr_t size; +}; + +struct pkram_region_list { + __u64 prev_pfn; + __u64 next_pfn; + + struct pkram_region regions[0]; +}; + +#define PKRAM_REGIONS_LIST_MAX \ + ((PAGE_SIZE-sizeof(struct pkram_region_list))/sizeof(struct pkram_region)) + +static u64 pkram_sb_pfn; +static struct pkram_super_block *pkram_sb; + +void pkram_init(void) +{ + struct pkram_super_block *sb; + char arg[32]; + + if (cmdline_find_option("pkram", arg, sizeof(arg)) > 0) { + if (kstrtoull(arg, 16, &pkram_sb_pfn) != 0) + return; + } else + return; + + sb = (struct pkram_super_block *)(pkram_sb_pfn << PAGE_SHIFT); + if (sb->magic != PKRAM_MAGIC) { + debug_putstr("PKRAM: invalid super block\n"); + return; + } + + pkram_sb = sb; +} + +static struct pkram_region *pkram_first_region(struct pkram_super_block *sb, struct pkram_region_list **rlp, int *idx) +{ + if (!sb || !sb->region_list_pfn) + return NULL; + + *rlp = (struct pkram_region_list *)(sb->region_list_pfn << PAGE_SHIFT); + *idx = 0; + + return &(*rlp)->regions[0]; +} + +static struct pkram_region *pkram_next_region(struct pkram_region_list **rlp, int *idx) +{ + struct pkram_region_list *rl = *rlp; + int i = *idx; + + i++; + if (i >= PKRAM_REGIONS_LIST_MAX) { + if (!rl->next_pfn) { + debug_putstr("PKRAM: no more pkram_region_list pages\n"); + return NULL; + } + rl = (struct pkram_region_list *)(rl->next_pfn << PAGE_SHIFT); + *rlp = rl; + i = 0; + } + *idx = i; + + if (rl->regions[i].size == 0) + return NULL; + + return &rl->regions[i]; +} + +int pkram_has_overlap(struct mem_vector *entry, struct mem_vector *overlap) +{ + struct pkram_region_list *rl; + struct pkram_region *r; + int idx; + + r = pkram_first_region(pkram_sb, &rl, &idx); + + while (r) { + if (r->base + r->size <= entry->start) { + r = pkram_next_region(&rl, &idx); + continue; + } + if (r->base >= entry->start + entry->size) + return 0; + + overlap->start = r->base; + overlap->size = r->size; + return 1; + } + + return 0; +} From patchwork Tue Mar 30 21:35:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E83FDC432C3 for ; Tue, 30 Mar 2021 21:28:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF08C6196C for ; Tue, 30 Mar 2021 21:28:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232923AbhC3V1n (ORCPT ); Tue, 30 Mar 2021 17:27:43 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43112 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232735AbhC3V1X (ORCPT ); Tue, 30 Mar 2021 17:27:23 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOpH2011615; Tue, 30 Mar 2021 21:26:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=YBKqO6Kx+Py25onm/mOZ3eSPATqB2W3PO6LpaRw3oII=; b=fbYtdKja6TrrQl5ReDicXI9cPnZAmilTBO96wqt/zEZMbl+7mmluKDqCi+1aj+quejkW kKeN8fVjhdtsMnPyzmku3XdMWsg7SfvFoH1t07ys40KokFPTZ9N/Xj3oU9VCgXFfDwK1 GKWd7N9ATb6SZJdea9qQHVJN63msNLwYbEuc++OqobyGxTCUOUCGd1gyMmW/qj5o15dp 3mfMOx5t1iTcq9IoCTPbSIvFtd/YtRenRxDyf0XUTkrojDpJ6QfGG8ZYZC7Nj80R3PQQ 1pDsQD4GvoVhl1VvwpIKknEB2s/0HzwTNP9GkmKrCCrJEmEGGHXny/x72/qyZTNLHWJj YA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2120.oracle.com with ESMTP id 37mab3g8x9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:38 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPSCV105888; Tue, 30 Mar 2021 21:26:38 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2104.outbound.protection.outlook.com [104.47.55.104]) by userp3030.oracle.com with ESMTP id 37mabkbcm1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fAEFID0p4Xb3YzDO31ZpGboq9p6x6uOFB4bYMiMsj3v11y9CElnWJUGmyzRzzmUmjdhWc7yKe4Zx0dOuiHpE/TgGjujmg3dcR/scXzXbkWZ/N+CVFTTeVZuc7FLz6NdzdpunXQf251wBfWMRalSB0W18KMLsZld98I0cknknoyhsfq5bcAHeojbq5ol2dG/pOXYLOu8O0vGxMRSAhCcdU1CPgZsF0nQXnIPwJMW6WS93R/Sow3IYSjjGQ0UIECDfhfibTcJ8Q97Swcux/FDKfWummZG1vF8pTFBvgV1SPLGHkfXvlCmHy13D7jyJgcoXaPSw5yrABFcQeyEXgqCfhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YBKqO6Kx+Py25onm/mOZ3eSPATqB2W3PO6LpaRw3oII=; b=PW54++o4wZI5T5ZtBm1AWTxTu9kYYGUo41VkuT8TX4iJWbPBAJp8ZmiCN04H+nRWieP2APJadHpKLKMGruPAshlwpRxTED+6V+tN61ZpjJgT+ReOY7SyHwtywKZkm7VX+MzbPDuyisQa2SdMDzWAJS395z2TYOnSQ4OpRfcpDI8jPLta3Zotw4n+tI+kgTcb1s+0E3fvif8Qmr2Spu42rXjzXu9xo3fD6Ti/6gdaxnELEQK/czld2yTzMCsiIb4qAfhz7ma8QFMRvc7BMa8X3itPszBoX6LLclUQ3+bmJp7NSI+15RFCN2ic4A6pZEXnMfIj8Urtbe+DGylePxp0UQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YBKqO6Kx+Py25onm/mOZ3eSPATqB2W3PO6LpaRw3oII=; b=h0Qk7RaaA7/MH88q13mk2TcsYf0wIOgZ7ignkJJfMU4UFhhlUCqOA+QcO7iQVjf8sfPiCp+ArOG3KgaXwmQkzUBaGD6sOkWntwvCarEXyk5Z8Js/rUk7I+vc9OmpuMVdC6LipXRC5eRSgohfBGJvwmCxum7YnC68COUnlF+Ifa0= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:35 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:35 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 22/43] x86/boot/compressed/64: use 1GB pages for mappings Date: Tue, 30 Mar 2021 14:35:57 -0700 Message-Id: <1617140178-8773-23-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:30 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a9c3f303-e67a-4ef8-9b9c-08d8f3c27ec0 X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4c1Sux/vcaNySDgicY6EUOa0TwPMT3PgL+M8zbgWFqmaWw8xhpmaPl7ZMdsff9Qr/315bGzA2/bHeEhCGwNoKY4rIb1n1Q2EUaGVTS+Xkmz52OBTVzEYbm2X1+qDx0tkBLgt8yEBG/uCyHf48oKtpoShhTXzL9PCai8QmYknK/NequQMJNGFmOp9Ka/NLtvEH5Nr0Iw7Zft8dQib3f1otfA1UeS9Xgj+y3MD6GYdvkwJtC3tfeItjk1rdFgRv19+JPfLHa/XUahTmIGvjMfWfeCXfVl9I5/hhVbZRuO17HiRvmvU13kLhC60OVT+FCSk08kX7Nq+p4+DfG9imSbuXyUXY/jFS2liNZZgyu6JMqD945zMZJoxp45VT+AhlO1ADsWegl2oRqjd9xzx1WrpHbmuL4CI63rkcZW1heu3NXl0fCS6anHCcCuRoDDmbkfivuebizrVOuCpX6D4Y6GOvNfHzDdyI5aiPNNcxJDM1NjoBeqQVlnwG53MvOd6/AKzfR/jRGt2dXRLF/cqJCDS8orSoy1x9vTvL3MNjk6hCa2l4/TgeDBMlvLlQ5mS2Mj/8E9SJH19fuiDK8BPywYv0V8VX/lhuxr+ExAb5cOn1QLEz4E+OQfmLWaOochMF7KuhjX36iALNaL+Vnj1d7GbbA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 4jQ05CVbj5B00CojA9dhvXU3syqH4rCtyBCLplhAYbxiI9FWTPBFITNl5H+/X+uNIe9Smi9UQDKPakPHghgO+t72RioFeOdsqptcfxCunkFDq1d12ffDmDB6RS5U3YPKfh/Z1ek4VcGquc/wrYkmbQQLrGSu7U2Y42NvmMB+qF1MMbEuBsQ0o2LDJ0v3w9SM06ooIHgVZGL0rzu+1xE/A+NO0mJ6/aUH1W9fqXL2KWXL/wGuqTzBqOfBDMNRCgaAQLez/6VTN0O9ce1MzxtOny9iXLrO7xKIvt/dKE8DyPCFVZD+RnSO2nmRRWHIOA/osNMxUvfJvkW9UyDH7EPTxxd1Zivw9NA6oKUnZMR8gE/IEjdTLXCJZk8VCpyC2RWykYyndR4bAX56an769jg9mjf1mZFbN7UXq5nq8vzqFCNsT0SspMID0YRJXqSRVoYcrQbplFLmjG0+9EC+LZsb/23s8/VxDMg8N47ghcrV/a2OVeF24pA/3FRaXdwKnYJc8Xn/zrsGk16x4byWIjF/I+Zd/Mjbya+edXjIx0bNelLDWTgtQTpebD/pyKhIftoPnx1VG+tuBtqoJ6k+V9YAgDbdriMKDQKa1+Q8vjidBGfHFudu302gJ1fkqxaKGrFwdc9/LcqpOJCqLuZteor8+gNNMViBzdOh3CzUXU4FCrivDhYfBqLiXwgDsbFQW7KRSjnAlJGobm++WtVRrX8drPaRo3j+01OlMOzVuRoMXAIs8FQaTImCojj0K9QqMy0GCFnEu9sInnqOZ7P8BXXNzsAdu2pbITg8C7CMBujczNlKv8GpQVAp+jQPlQG1KvloSg141QFM1PDsMHQ1VIx8RcHXkYYZpuyY+ZK+xNQtXeXqHji3aTQcFm4Nv0M0qs88V9cdwyBf9sJ1NKTCb9lZ4ATizAhD0yBilwv6OZypadQhjli8KGLQ3dDuJnKTXeenHHosUUADFQi8fvXMQY7U7Fe32V5A6QDpJ94VR10Hdelal2rOq9xWpdNeKyoR+SVa+tTjL+/iWDjUQpSTkQ7tZ4vY6cqwY3SytKI2GgWK6cElgX6FaivmAxc8m6bA9/iUEvQWLQ3jFueQRbVOa3CpIHT1tXwSblPcCtrFJa0InXNPQ/2n5O4LeXpgTSkoXNVbxFowvc7P2OqQMQ3ZCydMny7ulxQZFc0UiWBHaB4Byjut3q5oi+az8uKlHuvSUBS7nXLq2RqE53WT/xWlCB/vLtDFEV+hwhHpNG4fBkb3aIrP2ZyJX078Ndij3ZdKTGlFJBv4HsBNc+ESFHGtUifzw+zfx2M1u+t96sT9xOSP34eLVBjEk4hccXwHn/cg+//p X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a9c3f303-e67a-4ef8-9b9c-08d8f3c27ec0 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:35.0221 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nwAHgAHOCXXBPVIbNDCeLsN/qYVGykYFmyMzjwRbf45eb4K0cLFI6/yt2E1iuLTIC6udAb4kLERlSunJlaQ9adTiVorRVuL+6V9BsRfhNbA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: j7RGzK128KWgKUwJZCfOSoqy471JPzep X-Proofpoint-GUID: j7RGzK128KWgKUwJZCfOSoqy471JPzep X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 phishscore=0 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org pkram kaslr code can incur multiple page faults when it walks its preserved ranges list called via mem_avoid_overlap(). The multiple faults can easily end up using up the small number of pages available to be allocated for page table pages. This patch hacks things so that mappings are 1GB which results in the need for far fewer page table pages. As is this breaks AMD SEV-ES which expects the mappings to be 2M. This could possibly be fixed by updating split code to split 1GB page if the aren't any other issues with using 1GB mappings. Signed-off-by: Anthony Yznaga --- arch/x86/boot/compressed/ident_map_64.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/boot/compressed/ident_map_64.c b/arch/x86/boot/compressed/ident_map_64.c index f7213d0943b8..6ff02da4cc1a 100644 --- a/arch/x86/boot/compressed/ident_map_64.c +++ b/arch/x86/boot/compressed/ident_map_64.c @@ -95,8 +95,8 @@ static void add_identity_map(unsigned long start, unsigned long end) int ret; /* Align boundary to 2M. */ - start = round_down(start, PMD_SIZE); - end = round_up(end, PMD_SIZE); + start = round_down(start, PUD_SIZE); + end = round_up(end, PUD_SIZE); if (start >= end) return; @@ -119,6 +119,7 @@ void initialize_identity_maps(void *rmode) mapping_info.context = &pgt_data; mapping_info.page_flag = __PAGE_KERNEL_LARGE_EXEC | sme_me_mask; mapping_info.kernpg_flag = _KERNPG_TABLE; + mapping_info.direct_gbpages = true; /* * It should be impossible for this not to already be true, @@ -329,8 +330,8 @@ void do_boot_page_fault(struct pt_regs *regs, unsigned long error_code) ghcb_fault = sev_es_check_ghcb_fault(address); - address &= PMD_MASK; - end = address + PMD_SIZE; + address &= PUD_MASK; + end = address + PUD_SIZE; /* * Check for unexpected error codes. Unexpected are: From patchwork Tue Mar 30 21:35:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99BBBC433E2 for ; Tue, 30 Mar 2021 21:28:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60588619D1 for ; Tue, 30 Mar 2021 21:28:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233014AbhC3V15 (ORCPT ); Tue, 30 Mar 2021 17:27:57 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:32894 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232750AbhC3V1X (ORCPT ); Tue, 30 Mar 2021 17:27:23 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPNHP123003; Tue, 30 Mar 2021 21:26:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=HcxrisHJNwMExf/6GDpp3V1z8lwkTPHuB9X7hKqwaM4=; b=vpxaSNbJ04gbCyBPF9HgFdczaY0g2X4y9uxH1CkGCR1ZkK5FgrWHYdSQL+0G1o6U9A4L GLtT6DR1Ri19KT5OUEs5r7dKfm+FznNhbJV1d4Q31Gg0BxySczR8H9oU6y/F0K/mQjKd U1w3e+WEXQf0ac6GEFhsZAXyAegLfPIW+zapREbsH71gDjk9uGFZq85k5eSxhot7Bgwf QVgng/Whvu6ksuP2s2tu7Rc1mthMSsYrgAgdgxBpe3BKwgGAlP+xN9rvoLEfHao2qa/S 6h49KfBjm2EDZYCojigC8F+5KPHEmuVeMkUOd2bYo2t8Kk75bmAuLd8MUkJSnfVxbRI/ nw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 37mafv0834-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:43 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPSF0105832; Tue, 30 Mar 2021 21:26:42 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2105.outbound.protection.outlook.com [104.47.55.105]) by userp3030.oracle.com with ESMTP id 37mabkbcn7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:42 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T5ZRT+xC2PqaFyzvEM5DDFDfyytRqUj8WRW8O4W8NEktbzdcpnS+twS0WQpTX9TxDpWAxr0zty/mKVJE6UOGVGDEqRJo0US2dvDJ7+7Yxy3XJexbbAqSMHMbW6ibfk71Xbl2ObtDYzLU5JqQcrQFGDr+n24w9Ntwyrch8hxn2tWm+qJAN5cLuA7415dJhIORrdyJ29d7nipFfNQN9pmo1R+JEU3RDomSAGTJLrBBrj/amEoXEssxrs0tKAS3cI+snl+6gAE+QDzJVSkKACCqed3DR3Uom9TXDY2IVk6nQIpuzDJuqdnQd6qyyf75LacCa3yf1tOJEQNQmIy7elss5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HcxrisHJNwMExf/6GDpp3V1z8lwkTPHuB9X7hKqwaM4=; b=FKX3w+g+2XA0zP2JNVTE0LRA3Z65QoJ6SU/H3JWZ/pHu20paXkT06RbahEM8er8Eg02766RNQ0YDgfhMJ4gtowqXjYWp+z9O3P+UU3+wo5I3z2GDDgZAVGAOD1+MCECpQVHaJhSwxx/u/8h3W3ukwoGBgyjSMOt8jpb9Q96T8MfLJfjBQ3M4AaIirsDJJoA4S8uwQzt/Cyf8f+CSiOk5PkDD9n48+ROlWR40KZ5sE9UKOYvgVe8N8xds5zx939HsjQRY7xnu0N9oX2wrn9jZmDiQR1Of1ao6L5p2SsshO2b/9hrOSsgyFL2H3UdffL7NWBhCSTKaXzsFBfaWzvptHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HcxrisHJNwMExf/6GDpp3V1z8lwkTPHuB9X7hKqwaM4=; b=lRWTU9GT/tke+Fxy6NNT/40OombVb4bE32B/sDDb651Qf/LZZWaewW3f1MxBovq2vR+9ShXgV0zIu4M9HvDqGhh+gVHpxAo/wUheIjX3IOGUqv+Prxjq8eX181kRlG8RZkJ1QxvOkxA8pgpF61fENSwdUul28E3BcJA5I4gy3F0= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:39 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:39 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 23/43] mm: shmem: introduce shmem_insert_page Date: Tue, 30 Mar 2021 14:35:58 -0700 Message-Id: <1617140178-8773-24-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:35 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 77817601-b231-4393-4bca-08d8f3c2814f X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:565; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +tGswkQLs6YU2oQixQMoXLtWctdtmxi9FiDSJHZ64p3HjVeVOg2J4UII2cqUCJnoFzrCTiy6aM4gm/jUGOCldzlLn2BbM0LZJvkSCGpchYRqmfmD+KwbQ4i9+4OXsvb/SxnOI3wi+pSRkqkAsRJK/QvySvcHbPkyeJxHijmIW/g1c8Do/FWRL3fS1IX7ouX7Y6knwgocZx59ddP9U6W4j6c2h2jQDO14n1L9ID2Zgc4otEexVj+Srq9g2j66yZTtefQfrruCNqockds0YZF3a/40kFJ3FawYTOhzfgUPqln5cDhPdNd1xBw10I3L7R6Z81tcBLKKUkptwqMxjGRkSeN+B5XjzANYbGzcFg2QnrZpivU5ZY9amqISVNMjipjVZ/lRtyGeWqmQr/IT6vs8rLEBgv5q7S4vf3zPZRh5mQe1g1w4R6CjcJ0uFn1BU/cFrycSsAQ/sVjEYcNSS50vKac2tpgUFpC+mgjExMXQxmX1mD58oYKCKTSloGneZau6aR8lgIgukNGYxBbgrG2YGldLNgaCjaI923WG1j9bEs09tLWKd+nyONRAIK2PU9hatc+Cw9G6HFBFRr/xTCHQrTS67wHgJTB8E0Z7heyzxTqFgpRQwH2riJeZQuJaT2n6hzxUFR6LduDlnNx/yhhYXQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: pUHzou8cLL7/F394BGoKtNuYtGmDdyVkKYzh6lXTo70QolTadnilEYNEloaTvgET/QMQq06b1bVf7rX64KBx0AUicArdYlVZ12mFfnKY5uXOHuNcKw0LRfk78bCEmTxJOqNNevxyHtPJxSwCBmNdt3Vqujbi4q6EXiPPF605ypCJvxZXaUHjriqWZPLQTaWOE3DiK9gVwdBOkhRD5uraqz0sjd9cMsjYvfYryNuTC/k+0Mo2gVSsSppKvYKii2bPbLFd/ASzbiblHSjm9cMDPHLIsVz/N9TtuKJDW+l4nNFc+c+jTIp/Qzifxru2EciI8mQw84FalhJjgmqImXUVSlNoiTVh3mR3I1aGS1PK74/7pb+bkop7ExpQuWD69jNSFQE6fErwt3DAqvoMzwvzwdNUMIHoxt2f/R76nH2iwGtkMDKyRK/qyXp58Qvhtjodq5ycUCOeLIK/wO4YP8HYLZPTt1ZR1fWbDs1ZtBX2n27ooKYgsGWpjD7WNjd6r/77FeAZzBWx/PodD1UdGx3Y5kAVXhbUnYptLOFRJQ212wErFQLqssNdi2dRcCyiOHS5bSAg2RX/ECQMiP8LCkV8b4Jlb77sMEAS3AZxvvsT8/T5OgrktEKpf+QScKEB3Y5D0l8ZDFl63FkwQBFy5SSsIQB4OF+8KFH0Pygj9zJghh4uIdWSgc97r0aU2GyW+oUPlSBr2SsEzIIkFCRUvJylCExVmGxASaWGqs9uwb8qOg2T8O75KS7BWyoqDJM3EfG01heQ+PVnSjQ50LdxvAswxjJisx8oA6mckQVHvQ/CZbcBMvsi7FvEMHgdcXcfCUBv0lYVxtSlHYsoNI/7W7HVPcC2nWtu5KwE895tTqryCB25UIOT2Ci/tHue3xKkfD4xknvZPUca/SDM8E9OQD6JymX+xgnXcjv/eAQKawvUTfKmY3bXSKc+f48fouHe4YlGcaksnHwEOFa9TiQk15Zz2LoiY2R4T48fUUnxLV+IFoeU+gqcOcJSp0tjoXqiw/z30aG+JTwUZSrwTwYOCL+Bz1ONhokAZN5+XmglcVohM3BMpxo8phfF2yD2exMlJDcddCqkkM6Jhg32mKNC8vpDttUjKhdfxwr2/GJeHUSlS+HyGWAnOY5aHIwrpSNLuLJDab2Lpj8jl5li6/1rzJymi9hAiD2XbwVVoEs5UjhYAx3hNHgY6/+kqMPN0JwTfaRhIF2t3PwzE5mTWODhVsbrCW7CZ+jJfoxOAYz5qJx97gX9PDBW6iaqr3+OoWoQovi9K0NcMzA4gwHqzecnWwQC2SEi9jJ+1j2ogLUDED+lYggEkKjyqvMpVGM5L9/z/1Pd X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 77817601-b231-4393-4bca-08d8f3c2814f X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:39.3106 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /ju3TOPG4cYJtIETQ0GvjFtHioeyehDOJ5ANK3lnCUh1f0QEZZQEiMPoYdppHvIBeuRBakqYEgEKPf4d/Sf5OzeNmuh8I+CHNk/I/3GzKg8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: 9vmOTQlOURLpy9Dd4IiOIVEZT7tso4_M X-Proofpoint-GUID: 9vmOTQlOURLpy9Dd4IiOIVEZT7tso4_M X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The function inserts a page into a shmem file at a specified offset. The page can be a regular PAGE_SIZE page or a transparent huge page. If there is something at the offset (page or swap), the function fails. The function will be used by the next patch. Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- include/linux/shmem_fs.h | 3 ++ mm/shmem.c | 77 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index d82b6f396588..3f0dd95efd46 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -103,6 +103,9 @@ enum sgp_type { extern int shmem_getpage(struct inode *inode, pgoff_t index, struct page **pagep, enum sgp_type sgp); +extern int shmem_insert_page(struct mm_struct *mm, struct inode *inode, + pgoff_t index, struct page *page); + static inline struct page *shmem_read_mapping_page( struct address_space *mapping, pgoff_t index) { diff --git a/mm/shmem.c b/mm/shmem.c index b2db4ed0fbc7..60e4f0ad23b9 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -755,6 +755,83 @@ static void shmem_delete_from_page_cache(struct page *page, void *radswap) BUG_ON(error); } +int shmem_insert_page(struct mm_struct *mm, struct inode *inode, pgoff_t index, + struct page *page) +{ + struct address_space *mapping = inode->i_mapping; + struct shmem_inode_info *info = SHMEM_I(inode); + struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb); + gfp_t gfp = mapping_gfp_mask(mapping); + int err; + int nr; + pgoff_t hindex = index; + bool on_lru = PageLRU(page); + + if (index > (MAX_LFS_FILESIZE >> PAGE_SHIFT)) + return -EFBIG; + + nr = thp_nr_pages(page); +retry: + err = 0; + if (!shmem_inode_acct_block(inode, nr)) + err = -ENOSPC; + if (err) { + int retry = 5; + + /* + * Try to reclaim some space by splitting a huge page + * beyond i_size on the filesystem. + */ + while (retry--) { + int ret; + + ret = shmem_unused_huge_shrink(sbinfo, NULL, 1); + if (ret == SHRINK_STOP) + break; + if (ret) + goto retry; + } + goto failed; + } + + if (!on_lru) { + __SetPageLocked(page); + __SetPageSwapBacked(page); + } else { + lock_page(page); + } + + hindex = round_down(index, nr); + __SetPageReferenced(page); + + err = shmem_add_to_page_cache(page, mapping, hindex, + NULL, gfp & GFP_RECLAIM_MASK, mm); + if (err) + goto out_unlock; + + if (!on_lru) + lru_cache_add(page); + + spin_lock(&info->lock); + info->alloced += nr; + inode->i_blocks += BLOCKS_PER_PAGE << thp_order(page); + shmem_recalc_inode(inode); + spin_unlock(&info->lock); + + flush_dcache_page(page); + SetPageUptodate(page); + set_page_dirty(page); + + unlock_page(page); + return 0; + +out_unlock: + unlock_page(page); + shmem_inode_unacct_blocks(inode, nr); +failed: + return err; +} + /* * Remove swap entry from page cache, free the swap and its page cache. */ From patchwork Tue Mar 30 21:35:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 341D3C433F7 for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 02D4D619F6 for ; Tue, 30 Mar 2021 21:28:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233036AbhC3V2D (ORCPT ); Tue, 30 Mar 2021 17:28:03 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43282 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232903AbhC3V1l (ORCPT ); Tue, 30 Mar 2021 17:27:41 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULONSq011485; Tue, 30 Mar 2021 21:26:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=iIHVwFGw8Hz4nPO94qn6qgFMWA5G5Zwf49ErfeTVY9E=; b=JTpiO6DMbvoKG8eguxDIhXRyMsonYM+E16hk2ZYy/qgvxEVcLukBoZmz+ylDo1DA0ZWR IlQCTKuM2+JTedKeld8UYu4+eD1eRo/ej1QJVmoSOEWbhQ9coivvdRGvSGK+fz0KPUcM AFANppaXbT3VcoobTtWot2zuM/rml7qBdsjPqsr8MSxqPzzVw4ZH1gEz+txEtsLUn2UH jdjz5AF7QmOj6HaCIiHeysIlJlQwdI1qRXsQaQvwjPBF7lsIcuiTA+C7LrD070OOjoK2 +WgwLN5pToKlJme91LM/ln5KLVzpru9/uiDdpBg2ldBWrP7I1EpnthLTAjmW+SMho8MV 0g== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 37mab3g8xk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:47 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPZ6i125105; Tue, 30 Mar 2021 21:26:47 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2100.outbound.protection.outlook.com [104.47.55.100]) by aserp3030.oracle.com with ESMTP id 37mabnk7kd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:46 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YjH8gSdCgRie1mmCVSZnw+udxcwwTMFq80xLq6EJSTabhW0ovlY+oC+14rbV/o4cDzHTlaNdvF+FU03w+JaUYWy+cBx9ADehz+CG5gs6qtb+l23jQNDbqJL01FANQsZXl78QuHBSggvyErDeX6gw6nwksS7mqVGqbjLfArAKiUacsHn1LMnQTMiQMsFDCALNS688y83o8qO67Aa+L6BFfy8dkNzCZoy1bX9UdlDmxaplQ1EId1x8lZkrUivkBKGungNLZMzO67a4oqA2gFrL61gyS2LuvRiolw7jBUQXu2T39TM4GWeLRCaE6O1Cnrx644Y/7O6h8666F8ANmh3xKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iIHVwFGw8Hz4nPO94qn6qgFMWA5G5Zwf49ErfeTVY9E=; b=fa3AvMdmCmqzy6nIJzcjsLJtU+j4mTjQwUD9AGc5EtqjSdSJpMV4FN8QQB7rViCjV1BiSnrG5fAEmvYRFw9oaDm7h2ROiD4umRtyWRODtHlzKk54JtefDC9epNPpwJMXSlMiZ3TRICNjvv3dF/PCu6ST0wrZng/UwzKU/SwrMWudDiMpfyt6b6f0CXYrQk7jriegmx/tsmdwVtZvEpnvnui2n/v7xW4LCKIFxT4X6aAVTJZymDkZHvVFpNsmU9/WUz8tIKZmDOXy33Bm7iRc3smwxgXXGaovTlmcriJq2utpV2exxDJRNt7bNC6N2HyvLyNsq2pjcYvuPag0hE/+rw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iIHVwFGw8Hz4nPO94qn6qgFMWA5G5Zwf49ErfeTVY9E=; b=CEjGhAoJh55zK0CubDhGEgtPDu0WULptoLN7HOismRTgbbJMF4RUXY5tVKrzbEo+C2GpHNbKlvVMDMzkaegkVX+V5N1Fm39ApUcctvJXsgm4J5t4gyvsBzZSk+n8iXSJG/kSdSy+raQokT/6kIn417Z3TbZFWRKxtpj+C4NZBJs= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:43 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:43 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 24/43] mm: shmem: enable saving to PKRAM Date: Tue, 30 Mar 2021 14:35:59 -0700 Message-Id: <1617140178-8773-25-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:39 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e7886cb2-de28-43b2-cc99-08d8f3c283d8 X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:245; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2XDtgVOcaCtewfyPd3nD5Zh8/ZEkKJ24ET60qIW5re+QNVLE7aukJsMqDCikOjLbEkKXrOcXFAkUaqOK/JWQkk9cY8qXz5mqejAY5CSrl8xfjl7RUhdWSErZei3UdtxIODRWhXTll0pF56JavF69+ld4WhD5D9dFpxmSWWV46qy6USt1J8fSho5eI3b2zideFwVGh03x03qjTgUAfqgI7R//bPhZhF1O/CdAJWurLOVUvD2lvKpVXSjpnd3PoLRHLRmpWjn7wA5iSZrm87Jle8ybxnnKDcgUtSPLu8kKqvEjgHh5xFiRriarRb4M9TPhRxC7tByNQ+OdNKHygrM6x9dS3QkexhJ6ZGKdI2lIaCDc/RnFqqLRn6NrtG281z8gGiXnbHvTk+77Jc2uL01s4YqZUrAAHOlY4Qi13mS7khh3NrmPCAuuGHnlSnUEjKJSOmJ/gnq4BT77uNvDCHQNcaPrbEugE0lkWx+eMmBB7nnHjVTgIUJQKFyp8lzQk1arf/HuRZMHhbUWWu60MYHvpbiawtRElbf4LdKevAP0oo5NS3ZX4m8xcrNAP7GiZ/2263LYI34soFPJ/mNhsa6GmiP139ixPA5bL2vNUd7hQwP9a4H+MbPUZMaJEILVNmjW1DOZEANh+FcTHGqMuIqyFw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(30864003)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: p+HWCO/tdpLOxZDV/aTkIb2lwjmuPLrR9HzHMwEVwZgXzUhEpxwFHWvTqRkopWu9OYX3EO7yDSCPhGKiJXU0tIaDEXTTjjrmHrBp0zlns9gmRv2h5RHRSHdyaP8MlRKFZrSC66UaxYIBuXwTJ6cPkeLLdonmOOQCZ3cOM2kEiDgA/pXUS3HSS1WobyaK5Ne/zNvXIHaSqZzummHnOgKm3UtMZZtSlwH+a1FhvtOHLwHvdkG+XMlyBum8jIVTPWOusiqAzk/KQCI+7YTPdttWRhCgo1ETfLEVbL3CSVpnj4zqXgoTUdxJTwZp0IrIfmLO7rgyo5mGACFReO6rpOtghmwDkJTgvAHFMtUEZYTH9CpVz5O5rZw/ncXRrGL8KngOXFKgVpqlpGjnnwp4eDf6auBFqD7hvBmZycnPJ+AmeyFaSwLkP0H3LN0BOt9IODBFCims7NNNZlpwgVKIter1qcu+YyHcmcW5DFHFB4wlmW2s+2JilKJpNbDUgEZN4n7SoIaPxGJj4Xl9mNyqA9K99RT4H2UzBwrkVEIFTdnGTJmZLIiFo56agFnX2S3riZ1B4lEs3sfDIzYfwIYWB9qF+MqfCRFy1UkAMmpeLzvnmW+kQFcUoceO4nvEnGEALsBUkaqeQyiNMpON44QvdZUrKlmzn2Hj2U+Pelvih6L6IAHovju7vvOMPYmYNvEJNlZTxo9tPGDIKyhr0tovRKvU21el/jzoJlzv12BshnepelwDktC68wXDVJUewGpUiEmNVwcXjdw2S+4UKcGPthNBXeubJuij9oGVKzi62jGtPZvrqmH0iz7FSlWez1imLGm2TX5KlIIU/Lt/6z9A2X5R21D0gCev4UL0W6j3yNnb4ve5bzFGDJ4x1ek2IGLloQs6l0Sos+cyM7e7PJRjhc0r97sb7SEwqU6M50RENA6jJ8/7CCQ0+FuO1pJIUL9BdanQKqQ0cUSC/xyrj60958E/A34+xALqovpKoqjNfFWzYjYMSw99Tze6WQ5RQZ775fiEgK9qYKay6CnF7Tojb4AihzCU/Fh/cYfLn2f60fw84lWR9Kg4o4ayelfM6CEf5s9TryZhQ3YeI6tNFfAQsgmfovNR6PmSvqbuTr2rNkwToMcwWagiyuI5vMK7CcIgAiw4t2jRXSlAErXdDzoo6PIACtl4Jwn4TwOix8Of1rMt7ISVoC7jpJBzfCOrPGE9U50Ke/S898ADIThRX41xhz71Fw2Gz5aQWLvXEabkicamVW2g6VHbFF6NRcwxDyTXeW+208JFIneHSHrxiQizenyoDI4xXgnpwuu9nYtYeO6srSBH588HLDApz5RAiysTm/VB X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7886cb2-de28-43b2-cc99-08d8f3c283d8 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:43.5073 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PV7BCEcSF5NeEKmW7S8qRqF2tHrSzrYDMctgvqAcbFOpqM5GpdFIb2aarZ8+zwiAcQIdMO4CIAPypBpXA/xH+RH+c3GuoFZOQvyr2PMLXJc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: OCI1vSQ_yZwsHEH_uu2XogfVI8mhSB1V X-Proofpoint-GUID: OCI1vSQ_yZwsHEH_uu2XogfVI8mhSB1V X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 phishscore=0 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patch illustrates how the PKRAM API can be used for preserving tmpfs. Two options are added to tmpfs: The 'pkram=' option specifies the PKRAM node to load/save the filesystem tree from/to. The 'preserve' option initiates preservation of a read-only filesystem tree. If the 'pkram=' options is passed on mount, shmem will look for the corresponding PKRAM node and load the FS tree from it. If the 'pkram=' options was passed on mount and the 'preserve' option is passed on remount and the filesystem is read-only, shmem will save the FS tree to the PKRAM node. A typical usage scenario looks like: # mount -t tmpfs -o pkram=mytmpfs none /mnt # echo something > /mnt/smth # mount -o remount ro,preserve /mnt # mount -t tmpfs -o pkram=mytmpfs none /mnt # cat /mnt/smth Each FS tree is saved into a PKRAM node, and each file is saved into a PKRAM object. A byte stream written to the object is used for saving file metadata (name, permissions, etc) while the page stream written to the object accommodates file content pages and their offsets. This implementation serves as a demonstration and therefore is simplified: it supports only regular files in the root directory without multiple hard links, and it does not save swapped out files and aborts if any are found. However, it can be elaborated to fully support tmpfs. Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- include/linux/shmem_fs.h | 24 +++ mm/Makefile | 2 +- mm/shmem.c | 64 ++++++++ mm/shmem_pkram.c | 385 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 474 insertions(+), 1 deletion(-) create mode 100644 mm/shmem_pkram.c diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index 3f0dd95efd46..78149d702a62 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -26,6 +26,11 @@ struct shmem_inode_info { struct inode vfs_inode; }; +#define SHMEM_PKRAM_NAME_MAX 128 +struct shmem_pkram_info { + char name[SHMEM_PKRAM_NAME_MAX]; +}; + struct shmem_sb_info { unsigned long max_blocks; /* How many blocks are allowed */ struct percpu_counter used_blocks; /* How many are allocated */ @@ -43,6 +48,8 @@ struct shmem_sb_info { spinlock_t shrinklist_lock; /* Protects shrinklist */ struct list_head shrinklist; /* List of shinkable inodes */ unsigned long shrinklist_len; /* Length of shrinklist */ + struct shmem_pkram_info *pkram; + bool preserve; /* PKRAM-enabled data is preserved */ }; static inline struct shmem_inode_info *SHMEM_I(struct inode *inode) @@ -106,6 +113,23 @@ extern int shmem_getpage(struct inode *inode, pgoff_t index, extern int shmem_insert_page(struct mm_struct *mm, struct inode *inode, pgoff_t index, struct page *page); +#ifdef CONFIG_PKRAM +extern int shmem_parse_pkram(const char *str, struct shmem_pkram_info **pkram); +extern void shmem_show_pkram(struct seq_file *seq, struct shmem_pkram_info *pkram, + bool preserve); +extern int shmem_save_pkram(struct super_block *sb); +extern void shmem_load_pkram(struct super_block *sb); +extern int shmem_release_pkram(struct super_block *sb); +#else +static inline int shmem_parse_pkram(const char *str, + struct shmem_pkram_info **pkram) { return 1; } +static inline void shmem_show_pkram(struct seq_file *seq, + struct shmem_pkram_info *pkram, bool preserve) { } +static inline int shmem_save_pkram(struct super_block *sb) { return 0; } +static inline void shmem_load_pkram(struct super_block *sb) { } +static inline int shmem_release_pkram(struct super_block *sb) { return 0; } +#endif + static inline struct page *shmem_read_mapping_page( struct address_space *mapping, pgoff_t index) { diff --git a/mm/Makefile b/mm/Makefile index f5c0dd0a3707..a4e9dd5545df 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -120,4 +120,4 @@ obj-$(CONFIG_MEMFD_CREATE) += memfd.o obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += mapping_dirty_helpers.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o -obj-$(CONFIG_PKRAM) += pkram.o pkram_pagetable.o +obj-$(CONFIG_PKRAM) += pkram.o pkram_pagetable.o shmem_pkram.o diff --git a/mm/shmem.c b/mm/shmem.c index 60e4f0ad23b9..c1c5760465f2 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -111,16 +111,20 @@ struct shmem_options { unsigned long long blocks; unsigned long long inodes; struct mempolicy *mpol; + struct shmem_pkram_info *pkram; kuid_t uid; kgid_t gid; umode_t mode; bool full_inums; + bool preserve; int huge; int seen; #define SHMEM_SEEN_BLOCKS 1 #define SHMEM_SEEN_INODES 2 #define SHMEM_SEEN_HUGE 4 #define SHMEM_SEEN_INUMS 8 +#define SHMEM_SEEN_PKRAM 16 +#define SHMEM_SEEN_PRESERVE 32 }; #ifdef CONFIG_TMPFS @@ -3441,6 +3445,8 @@ enum shmem_param { Opt_uid, Opt_inode32, Opt_inode64, + Opt_pkram, + Opt_preserve, }; static const struct constant_table shmem_param_enums_huge[] = { @@ -3462,6 +3468,8 @@ enum shmem_param { fsparam_u32 ("uid", Opt_uid), fsparam_flag ("inode32", Opt_inode32), fsparam_flag ("inode64", Opt_inode64), + fsparam_string("pkram", Opt_pkram), + fsparam_flag_no("preserve", Opt_preserve), {} }; @@ -3545,6 +3553,22 @@ static int shmem_parse_one(struct fs_context *fc, struct fs_parameter *param) ctx->full_inums = true; ctx->seen |= SHMEM_SEEN_INUMS; break; + case Opt_pkram: + if (IS_ENABLED(CONFIG_PKRAM)) { + kfree(ctx->pkram); + if (shmem_parse_pkram(param->string, &ctx->pkram)) + goto bad_value; + ctx->seen |= SHMEM_SEEN_PKRAM; + break; + } + goto unsupported_parameter; + case Opt_preserve: + if (IS_ENABLED(CONFIG_PKRAM)) { + ctx->preserve = result.boolean; + ctx->seen |= SHMEM_SEEN_PRESERVE; + break; + } + goto unsupported_parameter; } return 0; @@ -3641,6 +3665,41 @@ static int shmem_reconfigure(struct fs_context *fc) err = "Current inum too high to switch to 32-bit inums"; goto out; } + if (ctx->seen & SHMEM_SEEN_PRESERVE) { + if (!sbinfo->pkram && !(ctx->seen & SHMEM_SEEN_PKRAM)) { + err = "Cannot set preserve/nopreserve. Not enabled for PKRAM"; + goto out; + } + if (ctx->preserve && !(fc->sb_flags & SB_RDONLY)) { + err = "Cannot preserve. Filesystem must be read-only"; + goto out; + } + } + + if (ctx->pkram) { + kfree(sbinfo->pkram); + sbinfo->pkram = ctx->pkram; + } + + if (ctx->seen & SHMEM_SEEN_PRESERVE) { + int error; + + if (!sbinfo->preserve && ctx->preserve) { + error = shmem_save_pkram(fc->root->d_sb); + if (error) { + err = "Failed to preserve"; + goto out; + } + sbinfo->preserve = true; + } else if (sbinfo->preserve && !ctx->preserve) { + error = shmem_release_pkram(fc->root->d_sb); + if (error) { + err = "Failed to unpreserve"; + goto out; + } + sbinfo->preserve = false; + } + } if (ctx->seen & SHMEM_SEEN_HUGE) sbinfo->huge = ctx->huge; @@ -3714,6 +3773,7 @@ static int shmem_show_options(struct seq_file *seq, struct dentry *root) seq_printf(seq, ",huge=%s", shmem_format_huge(sbinfo->huge)); #endif shmem_show_mpol(seq, sbinfo->mpol); + shmem_show_pkram(seq, sbinfo->pkram, sbinfo->preserve); return 0; } @@ -3726,6 +3786,7 @@ static void shmem_put_super(struct super_block *sb) free_percpu(sbinfo->ino_batch); percpu_counter_destroy(&sbinfo->used_blocks); mpol_put(sbinfo->mpol); + kfree(sbinfo->pkram); kfree(sbinfo); sb->s_fs_info = NULL; } @@ -3780,6 +3841,8 @@ static int shmem_fill_super(struct super_block *sb, struct fs_context *fc) sbinfo->huge = ctx->huge; sbinfo->mpol = ctx->mpol; ctx->mpol = NULL; + sbinfo->pkram = ctx->pkram; + ctx->pkram = NULL; spin_lock_init(&sbinfo->stat_lock); if (percpu_counter_init(&sbinfo->used_blocks, 0, GFP_KERNEL)) @@ -3809,6 +3872,7 @@ static int shmem_fill_super(struct super_block *sb, struct fs_context *fc) sb->s_root = d_make_root(inode); if (!sb->s_root) goto failed; + shmem_load_pkram(sb); return 0; failed: diff --git a/mm/shmem_pkram.c b/mm/shmem_pkram.c new file mode 100644 index 000000000000..904b1b861ce5 --- /dev/null +++ b/mm/shmem_pkram.c @@ -0,0 +1,385 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct file_header { + __u32 mode; + kuid_t uid; + kgid_t gid; + __u32 namelen; + __u64 size; + __u64 atime; + __u64 mtime; + __u64 ctime; +}; + +int shmem_parse_pkram(const char *str, struct shmem_pkram_info **pkram) +{ + struct shmem_pkram_info *new; + size_t len; + + len = strlen(str); + if (!len || len >= SHMEM_PKRAM_NAME_MAX) + return 1; + new = kzalloc(sizeof(*new), GFP_KERNEL); + if (!new) + return 1; + strcpy(new->name, str); + *pkram = new; + return 0; +} + +void shmem_show_pkram(struct seq_file *seq, struct shmem_pkram_info *pkram, bool preserve) +{ + if (pkram) { + seq_printf(seq, ",pkram=%s", pkram->name); + seq_printf(seq, ",%s", preserve ? "preserve" : "nopreserve"); + } +} + +static int shmem_pkram_name(char *buf, size_t bufsize, + struct shmem_sb_info *sbinfo) +{ + if (snprintf(buf, bufsize, "shmem-%s", sbinfo->pkram->name) >= bufsize) + return -ENAMETOOLONG; + return 0; +} + +static int save_page(struct page *page, struct pkram_access *pa) +{ + int err = 0; + + if (page) + err = pkram_save_file_page(pa, page); + + return err; +} + +static int save_file_content(struct pkram_stream *ps, struct address_space *mapping) +{ + PKRAM_ACCESS(pa, ps, pages); + struct pagevec pvec; + unsigned long start, end; + int err = 0; + int i; + + start = 0; + end = DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); + pagevec_init(&pvec); + for ( ; ; ) { + pvec.nr = find_get_pages_range(mapping, &start, end, + PAGEVEC_SIZE, pvec.pages); + if (!pvec.nr) + break; + for (i = 0; i < pagevec_count(&pvec); ) { + struct page *page = pvec.pages[i]; + + lock_page(page); + BUG_ON(page->mapping != mapping); + err = save_page(page, &pa); + if (PageCompound(page)) { + start = page->index + compound_nr(page); + i += compound_nr(page); + } else { + i++; + } + + unlock_page(page); + if (err) + break; + } + pagevec_release(&pvec); + if (err || (start > end)) + break; + cond_resched(); + } + + pkram_finish_access(&pa, err == 0); + return err; +} + +static int save_file(struct dentry *dentry, struct pkram_stream *ps) +{ + PKRAM_ACCESS(pa_bytes, ps, bytes); + struct inode *inode = dentry->d_inode; + umode_t mode = inode->i_mode; + struct file_header hdr; + ssize_t ret; + int err; + + if (WARN_ON_ONCE(!S_ISREG(mode))) + return -EINVAL; + if (WARN_ON_ONCE(inode->i_nlink > 1)) + return -EINVAL; + + hdr.mode = mode; + hdr.uid = inode->i_uid; + hdr.gid = inode->i_gid; + hdr.namelen = dentry->d_name.len; + hdr.size = i_size_read(inode); + hdr.atime = timespec64_to_ns(&inode->i_atime); + hdr.mtime = timespec64_to_ns(&inode->i_mtime); + hdr.ctime = timespec64_to_ns(&inode->i_ctime); + + + ret = pkram_write(&pa_bytes, &hdr, sizeof(hdr)); + if (ret < 0) { + err = ret; + goto out; + } + ret = pkram_write(&pa_bytes, dentry->d_name.name, dentry->d_name.len); + if (ret < 0) { + err = ret; + goto out; + } + + err = save_file_content(ps, inode->i_mapping); +out: + pkram_finish_access(&pa_bytes, err == 0); + return err; +} + +static int save_tree(struct super_block *sb, struct pkram_stream *ps) +{ + struct dentry *dentry, *root = sb->s_root; + int err = 0; + + inode_lock(d_inode(root)); + spin_lock(&root->d_lock); + list_for_each_entry(dentry, &root->d_subdirs, d_child) { + if (d_unhashed(dentry) || !dentry->d_inode) + continue; + dget(dentry); + spin_unlock(&root->d_lock); + + err = pkram_prepare_save_obj(ps, PKRAM_DATA_pages|PKRAM_DATA_bytes); + if (!err) + err = save_file(dentry, ps); + if (!err) + pkram_finish_save_obj(ps); + spin_lock(&root->d_lock); + dput(dentry); + if (err) + break; + } + spin_unlock(&root->d_lock); + inode_unlock(d_inode(root)); + + return err; +} + +int shmem_save_pkram(struct super_block *sb) +{ + struct shmem_sb_info *sbinfo = sb->s_fs_info; + struct pkram_stream ps; + char *buf; + int err = -ENOMEM; + + if (!sbinfo || !sbinfo->pkram || is_kdump_kernel()) + return 0; + + buf = (void *)__get_free_page(GFP_KERNEL); + if (!buf) + goto out; + + err = shmem_pkram_name(buf, PAGE_SIZE, sbinfo); + if (!err) + err = pkram_prepare_save(&ps, buf, GFP_KERNEL); + if (err) + goto out_free_buf; + + err = save_tree(sb, &ps); + if (err) + goto out_discard_save; + + pkram_finish_save(&ps); + goto out_free_buf; + +out_discard_save: + pkram_discard_save(&ps); +out_free_buf: + free_page((unsigned long)buf); +out: + if (err) + pr_err("SHMEM: PKRAM save failed: %d\n", err); + + return err; +} + +static int load_file_content(struct pkram_stream *ps, struct address_space *mapping) +{ + PKRAM_ACCESS(pa, ps, pages); + unsigned long index; + struct page *page; + int err = 0; + + do { + page = pkram_load_file_page(&pa, &index); + if (!page) + break; + + err = shmem_insert_page(current->mm, mapping->host, index, page); + put_page(page); + cond_resched(); + } while (!err); + + pkram_finish_access(&pa, err == 0); + return err; +} + +static int load_file(struct dentry *parent, struct pkram_stream *ps, + char *buf, size_t bufsize) +{ + PKRAM_ACCESS(pa_bytes, ps, bytes); + struct dentry *dentry; + struct inode *inode; + struct file_header hdr; + size_t ret; + umode_t mode; + int namelen; + int err = -EINVAL; + + ret = pkram_read(&pa_bytes, &hdr, sizeof(hdr)); + if (ret != sizeof(hdr)) + goto out; + + mode = hdr.mode; + namelen = hdr.namelen; + if (!S_ISREG(mode) || namelen > bufsize) + goto out; + if (pkram_read(&pa_bytes, buf, namelen) != namelen) + goto out; + + inode_lock_nested(d_inode(parent), I_MUTEX_PARENT); + + dentry = lookup_one_len(buf, parent, namelen); + if (IS_ERR(dentry)) { + err = PTR_ERR(dentry); + goto out_unlock; + } + + err = vfs_create(&init_user_ns, parent->d_inode, dentry, mode, NULL); + dput(dentry); /* on success shmem pinned it */ + if (err) + goto out_unlock; + + inode = dentry->d_inode; + inode->i_mode = mode; + inode->i_uid = hdr.uid; + inode->i_gid = hdr.gid; + inode->i_atime = ns_to_timespec64(hdr.atime); + inode->i_mtime = ns_to_timespec64(hdr.mtime); + inode->i_ctime = ns_to_timespec64(hdr.ctime); + i_size_write(inode, hdr.size); + + err = load_file_content(ps, inode->i_mapping); +out_unlock: + inode_unlock(d_inode(parent)); +out: + pkram_finish_access(&pa_bytes, err == 0); + return err; +} + +static int load_tree(struct super_block *sb, struct pkram_stream *ps, + char *buf, size_t bufsize) +{ + int err; + + do { + err = pkram_prepare_load_obj(ps); + if (err) { + if (err == -ENODATA) + err = 0; + break; + } + err = load_file(sb->s_root, ps, buf, PAGE_SIZE); + pkram_finish_load_obj(ps); + } while (!err); + + return err; +} + +void shmem_load_pkram(struct super_block *sb) +{ + struct shmem_sb_info *sbinfo = sb->s_fs_info; + struct pkram_stream ps; + char *buf; + int err = -ENOMEM; + + if (!sbinfo->pkram) + return; + + buf = (void *)__get_free_page(GFP_KERNEL); + if (!buf) + goto out; + + err = shmem_pkram_name(buf, PAGE_SIZE, sbinfo); + if (!err) + err = pkram_prepare_load(&ps, buf); + if (err) { + if (err == -ENOENT) + err = 0; + goto out_free_buf; + } + + err = load_tree(sb, &ps, buf, PAGE_SIZE); + + pkram_finish_load(&ps); +out_free_buf: + free_page((unsigned long)buf); +out: + if (err) + pr_err("SHMEM: PKRAM load failed: %d\n", err); +} + +int shmem_release_pkram(struct super_block *sb) +{ + struct shmem_sb_info *sbinfo = sb->s_fs_info; + struct pkram_stream ps; + char *buf; + int err = -ENOMEM; + + if (!sbinfo->pkram) + return 0; + + buf = (void *)__get_free_page(GFP_KERNEL); + if (!buf) + goto out; + + err = shmem_pkram_name(buf, PAGE_SIZE, sbinfo); + if (!err) + err = pkram_prepare_load(&ps, buf); + if (err) { + if (err == -ENOENT) + err = 0; + goto out_free_buf; + } + + pkram_finish_load(&ps); +out_free_buf: + free_page((unsigned long)buf); +out: + if (err) + pr_err("SHMEM: PKRAM load failed: %d\n", err); + + return err; +} From patchwork Tue Mar 30 21:36:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DC2CC433E8 for ; Tue, 30 Mar 2021 21:29:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 19C9C619FA for ; Tue, 30 Mar 2021 21:29:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232611AbhC3V3G (ORCPT ); Tue, 30 Mar 2021 17:29:06 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50678 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233252AbhC3V2h (ORCPT ); Tue, 30 Mar 2021 17:28:37 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULO0Nh130359; Tue, 30 Mar 2021 21:26:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=Ht/hcmXkFVk5Xsg6Wh3LNQplzl18papsAaDctd4NMw8=; b=B3P53XYHXBqt3TMT6JMzEIsflEE6jzSBQA8IGt0+7OGOrrT460Mhb8L7Hfi3Gmf/aey4 4+xfDD262nT33uXpgOaqJOjy++JGQVYlryRFDh4tol9ICxHgIJMLwimnZek/sWvdHxXF ubRznpPfwqijnXsqQtVcX/oKs4i/7z/SFpI3QfTAU6iLqSxO1/G1ajY+AYdXPwYC2+8K /Mhx8ubR0D1LAN4YXLJxWuPsfxUIB9PgzRLVv2W5lhB/M5yjxwAjFnjWFpphXJyc3nrm fAAxBA0sEyfVEu7s8OKAoRFzw+INnQENOBAU9gAGpXIIVWNKj2drioJekk6FlQe6KRvA lw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 37mabqr8pq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:51 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPRpp105741; Tue, 30 Mar 2021 21:26:50 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2102.outbound.protection.outlook.com [104.47.55.102]) by userp3030.oracle.com with ESMTP id 37mabkbcr7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WY6L/nhxD1V2hCe0TFnVJ+7G2t6XvKgLeq7mRQWVfw7Y6Q2hy2lLEUg70cj5LUKA6RrqCDJWcOX7ZP3oVjYY+ysZ0wvlftjDXQh6Pbyxwq/XZghOSICXcmJNtChlRVmrWPULJ4fBkzbOB2mfcHFlvF/vY8p7TAOJLhv6Frb1zyw702dVd5AfeMQspkc3EWPtyOglshh2CKxUA5OSZk4aVyEg96Ux0uqYTUeCneqLGkLAN8cYFWZN6xYuA8VlTIlJJrcOniDGOMg3JUmOaTuNFp7z+ncgGFS/nY33vSArK49wcyyOsKyt2Pt9HMUSpsjBOEBYKO9JzgsaxgstBvb6zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ht/hcmXkFVk5Xsg6Wh3LNQplzl18papsAaDctd4NMw8=; b=lcCmUPokj1xoJL2zoMBbkrFQYWCWrVKkLAni5VIgaJvgUN0+wZH/PZs8F1lFLiWKk/jeN1cwS71dI7kkSGa6NaiJYCXNMoZiojrOF7dtCwKcWJZwf8ZFvQs4VTQYWDxdQ+z3gRMNmTd+/wsTGABSXvg89/7zRWXcKJavg8bwlIa1GSLImkma4hLGygsdwlgwMXAMsBtZ8RPe7KBo08NkMQGUQQcezk1tF5+Bo6LdOe4N+6gzKgu/HWJhjmblpOTww0VKHIyjo26AJHcqnqKk7ZAbyE/5cSHFJmL3v8++k/oywcmqTnRDpdD3uYSWo9K/9m72M7y+aWWqI0hFneCD/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ht/hcmXkFVk5Xsg6Wh3LNQplzl18papsAaDctd4NMw8=; b=U91gtlwwESPy+U8riPbruEUXXEfc7pJlGALW4AzIqcoHYA9qqI+TZWkhDhDDjZKKZi+84qfMxv8Aijistv47uV2YxkdUntErsAvPKNMox/m5MOQh0ubE0EIIXWpXYbMk43rFurQ9UEA0dAmfIwP4OVrEk5bewiJRZNa10KhfoKQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:48 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:48 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 25/43] mm: shmem: prevent swapping of PKRAM-enabled tmpfs pages Date: Tue, 30 Mar 2021 14:36:00 -0700 Message-Id: <1617140178-8773-26-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:43 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: eadf0edc-4eac-40a5-09c6-08d8f3c28663 X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5236; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: B/PDxGMwVnlG7BOn+lor3oCxBfJ689HdrfGAVxVCsWSuoX+GB6zj7Pdlb3p8IAwJuIsVwssV5E3X6SwZIoKShfmY57QiX2v+aTBfVK73CXVgmj/+aACpWqM0VD2D7hs1FHbuRRsY8e/SBdR44n+EKDMqE31iT0pge9vWltjq+Jeb8Fd1VxPJ6CQrJaw+G8JCue44xhMWi4v5jkB88b2v06M+qDmh0zGO1iOyXyYePUX9oV6zABXG42udhbx4aaMmKaup1tnA67DSvxmf9Aca5NWfFjqG671h5ilrehldROueytGGSPG8cYWjgshBf7fc7qjfYCEdMLGulRuAfKpMFsUk64Cjdfk/L2GOqFh+BMlvdb+5JqO+xv7nyOdTrClnXDamjY5mweXCOnaE5rSVsSZ235McZjdu8eGzzL3tOpOSKVFbeFOY6TBP+x7EIlxtG9M0vEBvN2hNq3ak2OPrBOw+GSIPMfzRvL9cySCiVmyZT6if+9esZpuPLmDhM9UWLKKpf73KCvbsE9Okp1VH8Ed2yglifG2e1bKEvgizfNQlUX5WHzz+xtiYnihL/6C5F5MpRhXrCsjthpbcGCqXT2iG2vXLzu/OLRU1aDxAT7gABGr/6A/yt//CArcySct4la87re5RYAq71U/kyUrA0A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(4744005)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: UzXgEaSYcN/83m5e0XLJV9BbC9MH0wkGy8YIxsKYq37LgfhCzVC2XreTudLlwP2MQIOx2H8d4+FewMvIplKEOfByuPzGH6A4lQj/pna7ruEEH0VjaA+ZCMpXKwnkLPq39D72LTZO1qJ2CV2dGFFxhm9cDNCd9HU+4UofCHCTjBKUGqmDPLzOrLskDN0d1rxF+A1uIZCglSwHuErUz4quX+ERN/u7FzQDzNU1rErJ8QylIdnfbwswvaRXJJzH962XSVJQDtDFkU767N/hDseObQlN/1Bazoopkejt4OvqpiR0LmZ7MPgBHLf6pQzYYXKcR32+uZzm75hhXb0wdaFVoxu2QfPgQ8EmeTEUXCBnbNw+NYLlMlejCrDUsHxYdHGda0Q8cqzPNHc+IFdNLV/EJo9Rm7pKnW/3G4t5sN8z2KWJVjNywSX6JfauW5SAZ/c7boQ42ix7hB1kavo1KegrT1ZV+VviG0agHwGBcplTN5MomLF4y2pS/GiWef5H6pykXG7FTXEJBeiyLswj6lzvt4HkpEYsWrM5xnSZ3uY6I9dtkD++KZGTLaIv5KUNK6YK3MOM+NO4sdCGtGo0PjrDeMm7WTAMfNT67NZb1zOuru2tlFX6ER10nePziGWm5xGZFo2lxuV58wWbarb8DIoNs/Hz+BI3to5QVqH13fX9XKcXixEJ1x5TvlnhfRZWrfhDKrX2vtpJlJjqlL95uI7CRPO2B3qGBcQ/5kSvOho9txL8MRXarzogcGf3xLWj8d7XvlwilJ5Y3GloTX1TzCnAd0V2owjC1Q/sRn2BVpQp76qR4XbA7tTq6yefjwHhoUoFji+1mjMoOdfPUOYCw9oi+8SRQbNgv/BFx8Wbl+GpEwpIQYP7QwdZlUS2UFIOfqP/6uTYn1fnaftHQh9jWBaRlXSNgoDrIsoxNIPeKJ1ad0CQE3HmvyRvgVWcg6WcX4xjXF/nrmxtpsWfe2FkQk5ByBx0JsK/fAUi1evkheVrlNc07JvRzT1Y9WcElxcpJW4V/QY+ItQRmrudC993KF9rOQC3XqgQNHFUjQj73DNuRCfmbCYmefskrprAgQ98HrGDwL4ZlyKaU7sUYiQogRD8X+IOvzTu57kBSONrFEMiVhTkcsD+O3Tv+e0DmziGyGQ3wFKyasPBJTzzlUclFP1HxBEYruPZeN/ejeq3lEU0h9x19ousF6n3GIA9IqH7FbSgDoUaKHnq5vksBVnPcbxjTlBYFcldxvEOEIG40+k2GMFDJevMDF1goSTxfFsdWXA7Fps9yhKHamVYKINQrpezdMvqU0dP1R324NGrvqKXBHAROzbz4Qlf/FgOvbJfVlAg X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: eadf0edc-4eac-40a5-09c6-08d8f3c28663 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:47.8416 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Qdt/hmZY7+RvVxOHqzbfGbND8KMk/2DaQemfxcdYLd3HkTRMs1IuJ944e1vW5KU9y6c3FN5LsdLjKbFriQe6ah7CKUZ/Wt6rVnG+kEnNydU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: 0odOQw-sJZWu2EsDzUQTn_53PPDuOgpD X-Proofpoint-GUID: 0odOQw-sJZWu2EsDzUQTn_53PPDuOgpD X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Work around the limitation that shmem pages must be in memory in order to be preserved by preventing them from being swapped out in the first place. Do this by marking shmem pages associated with a PKRAM node as unevictable. Signed-off-by: Anthony Yznaga --- mm/shmem.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index c1c5760465f2..8dfe80aeee97 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2400,6 +2400,8 @@ static struct inode *shmem_get_inode(struct super_block *sb, const struct inode INIT_LIST_HEAD(&info->swaplist); simple_xattrs_init(&info->xattrs); cache_no_acl(inode); + if (sbinfo->pkram) + mapping_set_unevictable(inode->i_mapping); switch (mode & S_IFMT) { default: From patchwork Tue Mar 30 21:36:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AF57C433EB for ; Tue, 30 Mar 2021 21:28:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B1476196C for ; Tue, 30 Mar 2021 21:28:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233026AbhC3V2C (ORCPT ); Tue, 30 Mar 2021 17:28:02 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:33014 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232871AbhC3V1g (ORCPT ); Tue, 30 Mar 2021 17:27:36 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPG4H122973; Tue, 30 Mar 2021 21:26:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=zgl5LgrNn6hrsz19f6IfEd781HdZViILBnwp/lvaJNw=; b=Ggj4Sl7peRP4HD9C6ln2YwNbEoYvgxtyxvQr+ZoVWfBDRKTIWyBRceMaW3WHQqXsfMZH RVsnvJuJF+pPgOAIjw0BvLX8UFBGgrBs0EtNU9obZWJWnYzngCrqc/taeg/0cvtmycgH IFQ7DPMzJdoh/2Mnp0YcC9ipJb4HAu14O1XBjCSmaVIDC1QCaE5v3VR6EONSGo+cds72 L0tF8SE4SytWEF+QVLBzossnhox4T9FXTuH3S5XX+UL5OZtQ1xzNMIlnNARhn4uTozlR yxM7ueH4PWiy/gIEEA9dL1iWvBCjDJzE/iS0GWfB3Sg7sdnWl7Wd23WnI3umeIEgloTz TA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 37mafv083j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:56 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOaCC149646; Tue, 30 Mar 2021 21:26:55 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2107.outbound.protection.outlook.com [104.47.55.107]) by userp3020.oracle.com with ESMTP id 37mac4kf36-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e+ZOe3pVxovc8FztkmfUaVyPC4mxy+v4cLnDmIKn5uWaZqJGttXLPV0we/x+BzbAZQmMs3C6fn2z8GEZk8RZKmcZ+GujlYOxOA6fJSOorvarJ+7RHwGN4rpAxoE70Mpqg5ag61XeZ7gNEZyW53slswdRlAuYh1Igt13nphR7hI8Bk+frgpn14pHunvJslbqbBtkf8fvnKnsAXG/Ah+UMuDWOmbGk8PMJlJ1PDgnBzWpC6oeJnOTeVtO6yGsVcS7z3p+jjK869zCkO41iXqTEw5J7YXB0CPSggiyzs6tG7jnKEO3knjAPSFGmw6mjpVuj95sWBXMjfrDrZsZhRjzpzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zgl5LgrNn6hrsz19f6IfEd781HdZViILBnwp/lvaJNw=; b=Hob+qkCdiAUOF5X80u2H9SdJk8+y1gAAbpF3JlwpcRjgG5YApeDS8uZDJ//sx78RsTSQPiXgkyZYEtQPmRqDh26qraowfGmsq16wN4n+I+PoEnTDrAeU6dz2pHy+ycS5JzeSELP6IFasrwf2foOOnn0V6el68IkJIVWXIyos1pAYNPnX/Pd/cEfZQmpai4B/oWzCHQPNxEExbVCzdeMlsa/3eYh7KXS7kYXhqqUEsBGSNSSbj7mm8unMgy8mBmM3KGaEvDjYqpIKRh03l2r+wbA1NS0K0/+dchoIQvfvhUzrmJJM9S548MyDYDStjBkTKPQZvsmWFWSVoo6TdiiB0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zgl5LgrNn6hrsz19f6IfEd781HdZViILBnwp/lvaJNw=; b=xRxXPSNarLAM/OPCDkG9wJkmKnFRkDbNjhkSzMFeM8uMWvoVVlBB4Dql2jK5L30ua12UzwXmbSPRZjkVkcAiIpSD4kScoSe0HbgDMWSB4a3rz2hL3JD5+pMQy5fTgZBX3TNAsvOOW0bcVL6wBKBO1i9FRnoGEdgueXreuRC+R/o= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:52 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:52 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 26/43] mm: shmem: specify the mm to use when inserting pages Date: Tue, 30 Mar 2021 14:36:01 -0700 Message-Id: <1617140178-8773-27-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:48 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f2c7844c-986f-49ee-eaba-08d8f3c288fb X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3044; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Hlqp344PKo4sYXS2yNszx+14cwSa7oZCuIqq/OMHnNNsyb20B3wgcFcaxdgfw1Khbin2Vux2Vc+vRMcSosQubuIcK7yY7aHRY1uiUDBLaVutvb01h1jsywYJQfCHjxUwFFKp2Wd5sn2DbWRHNDlNOdNlUIMqwwDcxCg70YbdQ+nO0TCaeohb4uaNdbokOJ5C1eapx/qsn2VsyVFlzAiHaKrpaW6LIWDJ93hlI0SGapjbv7WgtacogiDFUFfK0iKtf8dBLAxfqo/TvOXpvMCyVLAlGinpusJUMSqwQXMGtZk8pqQ6qgz84eA/cl7qKvPIf6MQ+c/eKr9rQu+DROTRWxnfMzzcKBYSxjxIUGzKuTBuqpY9ZWt/VGzFuyJyy4GcI87uKeEaKP1M0Ck0eiCXKSxNieH8G8gVo/xRNkEERnLEjxhnq9Qb3HtkIQD347rJr+JK10xXfk6ClIsPW4CmhEyYSaRqcX/p695fsV/h2rbPdSynvCdVrHFdigzRZu40EYKYOWmqfaw7fBp8mOTPMx+5QlY2zLEzeOISKdcUchkpnI9hOFnIkDqUbMFULNF9QZJAT4AECbc5H2ZuVSIU5D6VPKlpvjZpX4nc4f+LD9k3sOH+u6KfIyg2RmYG8u3PNcqGoWgu3Z0T2CeS2MRhEQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: /DsgAuT1t0NVzHO18b1CdFRZaKX/5yDScZNes80qkNInMVgqsHp2GULpxIC+V6IlrsuPhAYjn5liOqwwF+V1UM7/haPpQHOj7dIn+JVuwMcSvAoYKMyOmX+kGY9K6yq7stzQp+SAYaBdXVBoeeUk/SGnu/KcxbFvS8aoIttuTeE1mlJ9rbdvs4+HBvs5UJx2DssmHDFw89E3F0+uK2M7xJYwaExT8ltVeYSFBKyfIFBAm5j5DcE3t8ECwhy4yfsvt9JYeEHKhL8FQXKdIGYXF06qDmSbD/GUly8dGJeaQuJMKAl4x4g/YSx537ZhT1g4j0Cic0KCprtarz4SL+rgq5Cooz72+NGM+oXIo7a2JONEtIw43PScYqa5iyStGZ2KxwtWrfj4LEMn2+qXas8AUPDdO/1n7r0WSwAXsTiOsTyVRvfTtHchk4rGjGCrbV5/9isevFb21pgZ6kjE+ITZkFmF3Mg7eaFRZRF6tOuF9TodQLitzUTvA7lN0/LUN1z63d8ezu1SPV6i5ZlqVovwMCDC4iWQ5z8LOkRtdModqVPXLeJKFYWa4jpXlfxidMUFmoHo5gPQiXLHxnuxx2fwvXOSdYmpo8gzJ3O2bzrADdDVZh/RM5t6jvi69miy4KVNYj/0DqD4N9CTvnDomg0viTUTFOIlyJTJuO8eR9ex4XIyPsj6isRF1LGLXKktwzM+I0UaKLmegAzDqa6QGzGNSK2NoW60i/s9/U6mrjkNxwqLsaifdoVMbgWMFYhHvCoSZ5XmaVPMmhWt3sU1SXJzWro+IbnGkVuipourKVYd/qvZAcGRdEjF3pWczyJTbJ1H3//oaZUMCavAIEIA/2eJY0opQq69fPlBiqmKZFtjjwuLHN8lCqrhGgSZMly1TMPOGe+WoZap/FbqZJsKulOkvMVoq2wwqUf4weMoxZncBa9lcUK1uvrf2T5bjaF1l1FvOVw20jrbXfmgUxj6XGXUDTepzmlf0REmtMPImq2xtcRMNwyBRqSmev4bxw1AmhGrEIexXa4kTcU8C3mF7Lka14taP0zU4DJ3b2oHZ9qn5yb6dLw9g6aMx67a+XXpO8wRu0t1UJOoWgbbHAb0AbY0h0am94kdePsgn91Mlr67V9KeECUDAtrECwhTFwTenDDg8kxSlG91dJTyz284IIupICDIBS/apDzxQlTIoVmo8JHNfnwwN4IcWgtXGParfTsItl1o+/Mu6M+jIZM3//c8QZ4yFAeEHb9Ea8EogXO9d+f9Ij5hmuB12xxEW+Dt4GFMQPJxLGkMhWaPubpdxmqX04JN+YHMi3bWBaxzUER273DcQXxM8x7uf4h8FOfBgKg6 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f2c7844c-986f-49ee-eaba-08d8f3c288fb X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:52.2289 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8o+aL4pFbZSf4ZWrirThTWt3NB4v9FahqCV38mvXbzGTUv8K7i1zc6dnOq/9lfgFENI/G1S2OMPsZc9z8OoxR+AYvb+TN448PeXP8XKrtmg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: wwqvilEskCM1PZahcPdqtOy2JDn09BbF X-Proofpoint-GUID: wwqvilEskCM1PZahcPdqtOy2JDn09BbF X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Explicitly specify the mm to pass to shmem_insert_page() when the pkram_stream is initialized rather than use the mm of the current thread. This will allow for multiple kernel threads to target the same mm when inserting pages in parallel. Signed-off-by: Anthony Yznaga --- mm/shmem_pkram.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/shmem_pkram.c b/mm/shmem_pkram.c index 904b1b861ce5..8682b0c002c0 100644 --- a/mm/shmem_pkram.c +++ b/mm/shmem_pkram.c @@ -225,7 +225,7 @@ int shmem_save_pkram(struct super_block *sb) return err; } -static int load_file_content(struct pkram_stream *ps, struct address_space *mapping) +static int load_file_content(struct pkram_stream *ps, struct address_space *mapping, struct mm_struct *mm) { PKRAM_ACCESS(pa, ps, pages); unsigned long index; @@ -237,7 +237,7 @@ static int load_file_content(struct pkram_stream *ps, struct address_space *mapp if (!page) break; - err = shmem_insert_page(current->mm, mapping->host, index, page); + err = shmem_insert_page(mm, mapping->host, index, page); put_page(page); cond_resched(); } while (!err); @@ -291,7 +291,7 @@ static int load_file(struct dentry *parent, struct pkram_stream *ps, inode->i_ctime = ns_to_timespec64(hdr.ctime); i_size_write(inode, hdr.size); - err = load_file_content(ps, inode->i_mapping); + err = load_file_content(ps, inode->i_mapping, current->mm); out_unlock: inode_unlock(d_inode(parent)); out: From patchwork Tue Mar 30 21:36:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90C3CC43381 for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 69475619D1 for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233052AbhC3V2G (ORCPT ); Tue, 30 Mar 2021 17:28:06 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43386 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232611AbhC3V1t (ORCPT ); Tue, 30 Mar 2021 17:27:49 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOjpx011609; Tue, 30 Mar 2021 21:27:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=ETuALoFduXRIGxrnolIc0B04gfH9EfZlHyE4qLXkimg=; b=NIiekFRLqRpDo9VCL0UqJayJUrICywilxplFn/mUgi9PogETP16jcsYIq/64qqxSQI8E IrwH/2H9ZfHcPjXzigg7pOkEqmzRY0luTQ1fKoyH26RyO5bGsRYS9RfuHhi6nfByrUxh Swmk8zQk0uOWY1vCHZZvO8+xYjx3RA52Ld08vC0tNMhdV44GrkwxoptdylxtBk7sdDrs qfuWGzg2lJMyTYFAL3t84gIGWqMeS+5IpqiwK61vfPdSiUIwBVqgFOFBQmCl95LDdilw OL12ZWd0tBgZ7uSyPANqeyHY9EtcJTmMD0EalBwlETwQDJmrj+Q8tEMdcYJCEIvh1J3C WQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 37mab3g8y1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:59 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOaIw149715; Tue, 30 Mar 2021 21:26:59 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2102.outbound.protection.outlook.com [104.47.55.102]) by userp3020.oracle.com with ESMTP id 37mac4kf7g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:26:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T6QO96squBbAJSuX7C4MZ6SjUwX4PNcC7c9VCziCD2wMHPblty2eeyFvuDdP8rFERz1P97VlI5Qr2eSTG4Iub2Oq8rxZ0i+NESjgSCKk+6XARzPtrqh3O7ZdRrCvxz0d+U/jtcnjIkNS8kSkTELJA9Q7GQtpXIgDACLNIi6qb2zOMTJL++1FHSRbtkrtxRvqagrdrm1Qe+yKO6W6bEh/WLr3wg5aHEI/dsdYx9gbyFweooiJWJTHCKrRQhxHvRmM+fUMfQuCFUMjBJbRjPV0oi+VP8KjqDaa8SYej2hch4Df8BwkMlSPV9695KRiX7IRiFqRP9G//wmAevdD/KKL4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ETuALoFduXRIGxrnolIc0B04gfH9EfZlHyE4qLXkimg=; b=ZkOygGOVisaKfzGzW2aaYK7u3ADTxmEtGujbfRfPYblfhKZaVLZeWhlLdcwdVpyOh0eZBSIUMgCpBvY+cEHbpCsa5Nt/7qcHX973c0SN+CoW8eov3SRD8TazJAG9tnN5FkDZCBN/7o9R9WpTsFg66NZ37O7QPfrMCgQpFM0bbZsmQTJuYzEHzDeWAVyUlCRw6L+BZ0Q84yOKbFG7kQalaHs127CI9W8mwsw6auHicDaRXdoz/Rb+x8zy+lQXhlkS2JLIYjCw7OfYcFVf8UiaceiAvxo3vwOmuMXOgBMUUHP3q0TeHaCv59VPK8nAPCTOKLyC1Rl02zmwKu4Kdq8piA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ETuALoFduXRIGxrnolIc0B04gfH9EfZlHyE4qLXkimg=; b=zIFBjWbeotmDfwU3bEd3j8MqDRmCIN9SnqwxQ68Nw1QpHU0tjmmFbUpYxFMksMo7NCJJX+QVJZX9RENrZB/T6UMI8pxVfgZnTyTBU+NecuBlyP9IC3TPh+IbbNIpyaJTo5cF3e00o345ADroXVOBuaO4hmKCPL00ei5Hnp7jtI4= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3600.namprd10.prod.outlook.com (2603:10b6:208:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 30 Mar 2021 21:26:56 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:26:56 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 27/43] mm: shmem: when inserting, handle pages already charged to a memcg Date: Tue, 30 Mar 2021 14:36:02 -0700 Message-Id: <1617140178-8773-28-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:52 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 60c9fb9a-9ce1-424d-10de-08d8f3c28b99 X-MS-TrafficTypeDiagnostic: MN2PR10MB3600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1148; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QvPH2ja5qrv58hmLQE7Wx3kQAw7judrTiSyyLuug0Gi2MDPLWwI7tNfQdfvdv7hq7ZcANdlXW3TMsHReCIsvHnS560CUz1Zew0eowqu73R6O/x89m1jiWdu/VERtY0I9o6L+DW4fAK2mUbRbLJ3JbOT/HlaA1+aacL0oh/qonx999XAxgu2OSDIPYu/QrsGtX+JT4GBePtJJ3UFj8Y7oKROKbxil5++yCZ7IBpGaJD4Zo7P7FeVwZ+vaxpNbF91ZA/mHg+1OjEoaFgRv69gFEbaV0nai28x9E5HkYeXZKvByk7Lv+8bVgl9ffKzscotfQ4ylSeLHzFEijQ5lgf4EDQZGp/j7+x1wOHCtDFSeF08LyOb8ilGSB0/OAx1H0kGdbfpd3AoFA8rpSaysNClMCvUAtvPUMhFptk9/wJ83mPm0piuSzetyjMPRs8mEEtNkLpY6eOoJLGqm+eCtciR0wE5YK9AptIY26aCtd90+i3/AtltSqifp3Ft8wDLEKEOKEHAXRn/x+xJN1Vya9cBpjh06QE9T8vFbhqhbwrcmlmwArPGoo14FTQ5Myr/wFrtGab89Q3ev7WIPX6pbjhL4NUoLgh2nhEUnPHZoMC3WP6lhh1d+8e7KNkkjPVS3Kbe+X48K/O+aF3BjGx79igBOSw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(366004)(39860400002)(396003)(6666004)(7696005)(316002)(36756003)(44832011)(8936002)(2616005)(956004)(5660300002)(16526019)(186003)(4326008)(6486002)(86362001)(38100700001)(83380400001)(26005)(66476007)(7406005)(7416002)(66556008)(8676002)(478600001)(52116002)(2906002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: evsms6gqdy3s152iHzpEQ+Mt84fn8+VZ6czyydKHq1kgw1gF6Dn1EDsWVu3TaHy8iNw9U/OG7YLtQP8ABIBx7YjIhBElPuC5+nQvkP9XX6MitAOnGyLAJm4Lk70PUZO+pyLGlGEMscgaGtXewnnt2JWCe4fzXswjrxH9rfda0rigqe3fPlFAOORnuWXUxeyZifk68IpiX5DYEIaI3qd+KaQQM1QcN5+a+dXfQdoNqmSMmREtOp2KY2U6aop9jWqkWG/BjGpRPTUSGHt5itsyZPl+nE6c/DKflNcJBeWXdvFmxB2c94jgH98AsxFh45hBJVelfg48NDciO3VGJK1x37vC3o60DRP0otA434h65dkDPPaR/AsHAvBZpR9JnlnLYizqoVdx3zmCRu5Fu4+SDx8sBOnhhRe3szGShHhYekY3NRxCM5EnXqeDAjNqNVpQ2wBKTqB6An/HgKDfOKoopux26i5PN692y31zkjBIaHpLR4OOYWaRpSMhcnXN+oU5Sov+GAhUmmFj/pIrMy90F1MF+if8NkHeh6a+Q+o6uI4ZW6zexIUlQTAMM17Yn9i93k1kg4+gtOtLfAnoL/Upnrp9XHqDvaN1wNoUpAALRwG8ZYbDh5Myge07pUy5CUgVQTI08EsMLBHBhWXR5t/bP7rWHuNKphQ6cbPr635Y58AX619Og3q39j7kS/mxmrTVkAh0H8GOQbcQh0ktkNNtwRCHB+gKw/oMxz48q6j1oyrdB+sQm1Jqtl8spFMt4l+DdUNmz6nn27xad8tpmcUh4NHdUwq5TahvA2pt5efoiRWW+vT1fSdvC1n0bn4bmZnOKFsmWLsRM4/Dmh916TDVFPt7wC1LOCp0gYwZytCoTE/n7dCr6WpqHnW5/IlfHgvFbPBE8kK14YVqbctWYbyg4u0/PBA7YZq4u2HVGA9+/qnK7HX6fgBwhGPWPzs+W5+Gg87lu+WSEt4gcH1OVW0Gl1/VYw3vSzlprS1eJwQFeUBrde8P+R+E7TNCRevS6oh/Wqujo6hncaP6eW/MmlfWRv3+L9/JOhCyQ+2o1mUXgFA5i6mav863KpCBGTbnlGjSVpaKNHi2Ktz8xJoBpUEBsNNT2CoLPvX96WCTdILWuJ3MAB172WKGsZOJUuxTXl17TTfcBklQD/5X4bmK0oPFAPKEDY8yg0PV3Tfuq7OXhYwwkRhWJA2eDWbro6o4Y2esyhTuel+u2UyeGbuTfJTZDSJYbtUQf4HSdnPgLGLrqoudyrz08yQL4byL/zjM/oy1Y33qexRe6MlV7pt/5BpeZuzkmFWfW/I5HgjyrmTSK2NTdP31xSC7Gw86I3IceI/G X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 60c9fb9a-9ce1-424d-10de-08d8f3c28b99 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:26:56.6053 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zwAl6JGCUJ3Y81DVRX2Y3HNZZuiFNcSmDrgPy8bYAx/qcgIvOKHlK60Q+QxSu8ZSQz2WQayIJz54N6hBAyl0O6i2xtCt6fWvqXFqbxBaG30= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: 5ZKc5wafv4K_xPnnBbmbNutToVZsJ3iX X-Proofpoint-GUID: 5ZKc5wafv4K_xPnnBbmbNutToVZsJ3iX X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 phishscore=0 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org If shmem_insert_page() is called to insert a page that was preserved using PKRAM on the current boot (i.e. preserved page is restored without an intervening kexec boot), the page will still be charged to a memory cgroup because it is never freed. Don't try to charge it again. Signed-off-by: Anthony Yznaga --- mm/shmem.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 8dfe80aeee97..44cc158ab34d 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -671,7 +671,7 @@ static inline bool is_huge_enabled(struct shmem_sb_info *sbinfo) static int shmem_add_to_page_cache(struct page *page, struct address_space *mapping, pgoff_t index, void *expected, gfp_t gfp, - struct mm_struct *charge_mm) + struct mm_struct *charge_mm, bool skipcharge) { XA_STATE_ORDER(xas, &mapping->i_pages, index, compound_order(page)); unsigned long i = 0; @@ -688,7 +688,7 @@ static int shmem_add_to_page_cache(struct page *page, page->mapping = mapping; page->index = index; - if (!PageSwapCache(page)) { + if (!skipcharge && !PageSwapCache(page)) { error = mem_cgroup_charge(page, charge_mm, gfp); if (error) { if (PageTransHuge(page)) { @@ -770,6 +770,7 @@ int shmem_insert_page(struct mm_struct *mm, struct inode *inode, pgoff_t index, int nr; pgoff_t hindex = index; bool on_lru = PageLRU(page); + bool ischarged = page_memcg(page) ? true : false; if (index > (MAX_LFS_FILESIZE >> PAGE_SHIFT)) return -EFBIG; @@ -809,7 +810,8 @@ int shmem_insert_page(struct mm_struct *mm, struct inode *inode, pgoff_t index, __SetPageReferenced(page); err = shmem_add_to_page_cache(page, mapping, hindex, - NULL, gfp & GFP_RECLAIM_MASK, mm); + NULL, gfp & GFP_RECLAIM_MASK, + mm, ischarged); if (err) goto out_unlock; @@ -1829,7 +1831,7 @@ static int shmem_swapin_page(struct inode *inode, pgoff_t index, error = shmem_add_to_page_cache(page, mapping, index, swp_to_radix_entry(swap), gfp, - charge_mm); + charge_mm, false); if (error) goto failed; @@ -2009,7 +2011,7 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, error = shmem_add_to_page_cache(page, mapping, hindex, NULL, gfp & GFP_RECLAIM_MASK, - charge_mm); + charge_mm, false); if (error) goto unacct; lru_cache_add(page); @@ -2500,7 +2502,7 @@ static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, goto out_release; ret = shmem_add_to_page_cache(page, mapping, pgoff, NULL, - gfp & GFP_RECLAIM_MASK, dst_mm); + gfp & GFP_RECLAIM_MASK, dst_mm, false); if (ret) goto out_release; From patchwork Tue Mar 30 21:36:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A5C6C433F2 for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1F4DA619FC for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233045AbhC3V2F (ORCPT ); Tue, 30 Mar 2021 17:28:05 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50270 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232953AbhC3V1s (ORCPT ); Tue, 30 Mar 2021 17:27:48 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULNipb130302; Tue, 30 Mar 2021 21:27:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=h9M1QscLLjk4WbpjAPe7cGX4t+n3aWpsMfiMaeRoEkU=; b=xUm2yJ+PUSZFOs1US3yOFPmhiYbH4daGe8lDTBo0u5i4Ddw+GJG6OJzOFPF5SfW2fQ0s L60aaZG0EDG5MPO/Z9I6YEmE75xpUq4lMaltkt9DYnX3Keg0/aKIOAvHcFhXoiOtOT5E ANSVCcUFA/Ga9rkcuwNR0D7dAiJoMTApLb6pjln7HWyeN5gjFxCjZQZNrU+C6LR+MsON NaimmkTrgjOyYdrWVi1wpXGjAnt7y4ynMmMTzddzJosX+g/9zqYQtqOSHsjNUCvmpNe1 5FiVX8+j6Njjk41ElRUbUxlej7jn1w4kDOTwP7YXB7IeLbUAHcTqg4ETDPKTDMruAk0e xA== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 37mabqr8q7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:04 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPZCG125027; Tue, 30 Mar 2021 21:27:04 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2102.outbound.protection.outlook.com [104.47.55.102]) by aserp3030.oracle.com with ESMTP id 37mabnk7ru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VPVGaD/gLe91pBvE8wmWg6GvIiYYnOtvO3w1fGPo9SoeG1bLXXx8VyKguUoNkrBIlA+f2aNsgoyKH7zjH56PGCAh8n0u4beFrn1CMF5ZZ1iMf17qqHmkPUbGp1Goo6RdWJNz/jS3lkXOfKt1mq7nK0AfUJVlqcQojjEKx4HmLeCPzYICDLJJQRZmrJOPbuNPwV6AoMm4tsfmTAiDRCn9CYj05I4EB84WwYDNx+5fmwzPXkjTkS9ZuiJSL6KBB2tFeV8MOTdHJffM/E+ZtaNhZx1Q/Nf/4nIqouPjRLgvixkqpAnTZe3ato8PIj7SRVdXkH1RBLkBNxhqg6pDFvxF9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h9M1QscLLjk4WbpjAPe7cGX4t+n3aWpsMfiMaeRoEkU=; b=XK/7ZvR5y90EDiWomrEVHEzfbsYv3mQloFGJCw1qvspUDkGdx6OndTLSoUBTYv9XEqLFquSTJr/+SJJs+KYShPQPVvt7fnQ4KmUOOoljOO79C4/QuphM+wddxdxhGs89h+VtQHIZ25w9p7Kw2Sj4l1BZozR8hqYNhsGwutBs4yKgVWQDEa8yJQN3XvyuGYZAn3aOaIuwdpFmrdDtlVCoqNBwY/YRd+RetgOjpvTqWfx/6Spr9EqLg7inlY3K6Vd/ZnEb951Qp5Sz0gQHT+fJ1ks9VLQyMbkMPG9aaMT+Q79kVz1KFEEhkF7KN+s7W3tz6qUabTWEcNCE7G6qnnrLmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h9M1QscLLjk4WbpjAPe7cGX4t+n3aWpsMfiMaeRoEkU=; b=sgcQYoTkD5cJ06Y+ULGrxxmacTRxyQabmTXizUGkegi1wUprG5B6V4qxUQsmJurkR7e8l982auoUEA3U+2oH+DI3cGqptWJ2YIaftjQNvGgkzNpTfFc/a5T6bm3vBwhxsjgFU2CuQk2US2KEEvBQTavJRCn4u6xxjWzZCgCnR2Y= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:01 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:01 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 28/43] x86/mm/numa: add numa_isolate_memblocks() Date: Tue, 30 Mar 2021 14:36:03 -0700 Message-Id: <1617140178-8773-29-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:26:56 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0837dc0b-dfc6-4fbb-fbc1-08d8f3c28e24 X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Kv7gMLgjefEV6Z22xxu9YD0v9/HhkmXnOiMzsUDdZ38U/fGCvS09D+pPKcF4bRGEa67hf0yPrDPqguZnIaPsekHy2VLDmNUIjcJ7+lf0xOHl3NY/yal72C3buzPH9bKCnjlNWO2TH/9Nf5C/UBbuVOQmQjJues2APGPJTl404aQ2t78ZBCGmoVq60J6JOB5MSO0BeaeC1ZKpu3KRCy8H+QDLrU35fbqBkhjwM/WyF5VG+4/TpFNaN5HtjIhRVjzLLmAQSWLnxkIjVMGe3udWajbysSXUew5weIm3bjkKN/fotVk6GD5ucHaOVMLfF23lT2qy5UD1fG5+7d22MZwY0lloPYZKPP7k/ogtOxtMYmTbb3mggyjLelnYVFiLbOsFyGNucZ4e47RTLdMQx8bXaDBayx3UKHaz33OHfxwDGyIj538bl7STN351SQiL/STQg2pKlf6cB6/93swCOghGZP50Q98S8r4hPZEu7LXeVsXbMhH/MxgKm5QGv8APMW8nBzT5ATx546iINi5Oqt+j2BAm5uSCmTulyn3qjuPRO6mwM7I2Ug4EY1Ub+P2nXzOx7iLbWkN+vEJKCtFY+DqJyaixUvWGSuzsIJ+wyc4KYwbEJ0JUvGUxQb6VVJ8kLXV6q3Eh1BIDViL6Amgpl0pRCQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(346002)(376002)(956004)(2906002)(66476007)(16526019)(8676002)(316002)(2616005)(7406005)(8936002)(44832011)(83380400001)(38100700001)(6486002)(26005)(66946007)(66556008)(7416002)(186003)(6666004)(4326008)(86362001)(52116002)(36756003)(5660300002)(478600001)(7696005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: qpCQmMFasH8/9o4rHIiwB2+X95wGOCTmcdGPg3Z5nS7tPIV0V5rPQgbH6Ix9DgZI23QgSlIzDzK87uElUhvpLRXZRpiZhsTOkGoLmjclqgieps4gnYSsbIRm5IKFJhuJxflDPSJRrwoSmH2mAH0SIx49dTO9UyKgsibfalqjatwR70TbWA4R4vcEAqfZp+CDGpqEUCinZAfkXRBlUywG5pDWR5Y9cPIYmSXuQoJDcHbfBpH+A7tjVwU3a/rMUie0Pl6GtAnK1L0aNYteJWvlFdYeLvtYRRIA8UyPtdCcP8VhqpSnajrKgY1D6L46DAq4Km8e3xOm9waBuRbx4nW1pPN6OLm9GtNSw+zEbx0XCgXixQmyKMMQ6dE0oGJ7M+g6qUO/Njn3d2ycAmxroItsaVMMnrsg6xVVv/qVrbXxBwxW7rPiSGRpXmZLmThOliLjjCbGxcsA6momZIhC8GECP20dAdK7jIuGjN4IJD+KMwhinDb3qdaHtpNH9MOaZPVGgGJcn3kGiHuMoZUMut4S2v9NDtmUUBteQJs/BHBqaXQZLFB0EHB4KMULlFZuXqif0QfHzqyeofdi4JOOFRUA699ouL5W2AxP8an1WBtBpRUyy1PKEcXlVNAfTUtMyAoIuk8ApiHsMDs0lLn0Sw1S5+tOyxPcHF3O2LL1W/IdITmVPB1hDjn6y7VVrqAxOjO96CBfIoKIxR/kawOXxlE68W0FSdm47B2Z82fitVtXL928MnId8+s3Wy0XBEgYpE0OEg3pV5iApZxtvXenF+oUiB4gz1vKc9+wg2t+biR8MUKA3M5UtKroONEJeE6foyNkuXVPN9uwX2bsrYJV9X3Z+iLQYUT6mGLYpwttdgn7sGQ1F7uzN9rLAITJOSgNbDfvpl8wcVAwTXXDiPwGjcHkov+4XLbQrcckn6pYu/PBL3ALlo/1sWJ4dmSOqI06uEwFyrmx9fsWHUVkbtpI/LZrvTjtiUSo+S/ZqKRa4swpZEUu4fmLD4EhKSimDUoi6ObSy9tr/6FU08edhpnPbId7m1afuCyiLi0d8KnkFhl0+jRVGUzU1WYRxD336KQMwBapgUvPG3ZpmxNcxlE2GD/cRtj9KWxIKv9pa5C4aiuVZKB08CfaC/j9I/q7LEzgOsqtZ9VXAb4h0cEiDaCtbanFQjdN3oq5xnxjiNXIAB13pD99YizWnc4HNPVzEaAP89+yjo09CsqrpGmMTSLFAbPE9BEB/7xqPvcgl82SLlSG+1zapKSV+eQ8IIiRMFceB0bH8AQE8WRADvELr9Z98f3iSzHopv5RVUdl/4swaYx4QFs9tJxAbzV2avlVx8Rhrehx X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0837dc0b-dfc6-4fbb-fbc1-08d8f3c28e24 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:00.9077 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rH2Exzj5EMON0Ji56RP2Fs+VlxJR3gcSJCq9sRDbMlW/wVlB1Di27WcxhU1S4eT3+JZe9UtwUvMF54yvAdPHyAUYyor8/o5TvV0xRq7+o0I= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: No3OIjo6MZsQ_2HJXAsOgactNtqt76Ck X-Proofpoint-GUID: No3OIjo6MZsQ_2HJXAsOgactNtqt76Ck X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Provide a way for a caller external to numa to ensure memblocks in the memblock reserved list do not cross node boundaries and have a node id assigned to them. This will be used by PKRAM to ensure initialization of page structs for preserved pages can be deferred and multithreaded efficiently. Signed-off-by: Anthony Yznaga --- arch/x86/include/asm/numa.h | 4 ++++ arch/x86/mm/numa.c | 32 ++++++++++++++++++++------------ 2 files changed, 24 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/numa.h b/arch/x86/include/asm/numa.h index e3bae2b60a0d..632b5b6d8cb3 100644 --- a/arch/x86/include/asm/numa.h +++ b/arch/x86/include/asm/numa.h @@ -41,6 +41,7 @@ static inline void set_apicid_to_node(int apicid, s16 node) } extern int numa_cpu_node(int cpu); +extern void __init numa_isolate_memblocks(void); #else /* CONFIG_NUMA */ static inline void set_apicid_to_node(int apicid, s16 node) @@ -51,6 +52,9 @@ static inline int numa_cpu_node(int cpu) { return NUMA_NO_NODE; } +static inline void numa_isolate_memblocks(void) +{ +} #endif /* CONFIG_NUMA */ #ifdef CONFIG_X86_32 diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c index 5eb4dc2b97da..dd85098f9d72 100644 --- a/arch/x86/mm/numa.c +++ b/arch/x86/mm/numa.c @@ -473,6 +473,25 @@ static bool __init numa_meminfo_cover_memory(const struct numa_meminfo *mi) return true; } +void __init numa_isolate_memblocks(void) +{ + int i; + + /* + * Iterate over all memory known to the x86 architecture, + * and use those ranges to set the nid in memblock.reserved. + * This will split up the memblock regions along node + * boundaries and will set the node IDs as well. + */ + for (i = 0; i < numa_meminfo.nr_blks; i++) { + struct numa_memblk *mb = numa_meminfo.blk + i; + int ret; + + ret = memblock_set_node(mb->start, mb->end - mb->start, &memblock.reserved, mb->nid); + WARN_ON_ONCE(ret); + } +} + /* * Mark all currently memblock-reserved physical memory (which covers the * kernel's own memory ranges) as hot-unswappable. @@ -491,19 +510,8 @@ static void __init numa_clear_kernel_node_hotplug(void) * used by the kernel, but those regions are not split up * along node boundaries yet, and don't necessarily have their * node ID set yet either. - * - * So iterate over all memory known to the x86 architecture, - * and use those ranges to set the nid in memblock.reserved. - * This will split up the memblock regions along node - * boundaries and will set the node IDs as well. */ - for (i = 0; i < numa_meminfo.nr_blks; i++) { - struct numa_memblk *mb = numa_meminfo.blk + i; - int ret; - - ret = memblock_set_node(mb->start, mb->end - mb->start, &memblock.reserved, mb->nid); - WARN_ON_ONCE(ret); - } + numa_isolate_memblocks(); /* * Now go over all reserved memblock regions, to construct a From patchwork Tue Mar 30 21:36:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0AF5C4345B for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F2BC619D1 for ; Tue, 30 Mar 2021 21:28:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233068AbhC3V2K (ORCPT ); Tue, 30 Mar 2021 17:28:10 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:33174 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232974AbhC3V1u (ORCPT ); Tue, 30 Mar 2021 17:27:50 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULP0hM122922; Tue, 30 Mar 2021 21:27:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=qeWsMypjQJE5WJDxavYLccIioQ1C4MVe9l+DC4ES6oU=; b=ssy7RMsi/f1g4LL7Ht1jDPvcd0F+DnCfKUebZUbt+HiyRsqN2CWw6dNmsiwi/zbgE2VW CtGT27lzBiYTwLYd32yNBQrLADOXafeHzErzOnNFqhvtzLiqWdJ7xTWe0qRoh6HbOWWl WsO/ESCYSQn07sw3BYagmlAdJ5DY/AxnKQsfOuyhZxtLVNjwdlPgu/k1nmzQfp+bu/A2 QbqdefKgtDBHeUJ2NPke/4V/06iyRw0w+PUkZkCTHFFOT9dtCehvof8Smpeh9wHk0A4p 2jjaqiaWya+vS+NdJ3UKTj514tbliIfH8Ca1TItxbDmEfXmoZZK8IlLTeztSzwCipfD/ nA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 37mafv0840-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:10 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPR2R105742; Tue, 30 Mar 2021 21:27:09 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2104.outbound.protection.outlook.com [104.47.55.104]) by userp3030.oracle.com with ESMTP id 37mabkbcyb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZcNIihEWT5XS22I/vGYCmnTFAeSme1bNvHQGbj5z7wcfc79ouyxgNulkCIf5j+Hdq+z0ew/GbM9wuHHq+ELvozgehur5xOj5WFDiGqndLblbhXWrIVwJPpxYYIqokBNP80ebQF6n/htwLNAQ9tUg+/p9VDef8pUPrQ6oCkEwjF1+LJp3817Z1RXwmBmF1WmqmsKk45RFmxCQk3vkX8Ryn8YSwzUXQWbTzPWF6lffIOJ2sKRzU90BNVMNBylQ3uySN2pEr++7LrauON77MmtwDcCbcnoOLvoA3nhGGDOlQkABkNwTBo/5FUU7qhCY0c4H4zBuiKDdrCQezG/FaHWtZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qeWsMypjQJE5WJDxavYLccIioQ1C4MVe9l+DC4ES6oU=; b=MDiICrOrzDJiVoOoTbsGNgEaTo96RtEh81r0cetC5veJB+9todjchix4WQ/gmSCHNfMmIFDnZbSNX8kdF/CGuOo6J0fCMf49RBZY21decHy1+YHNPl5Y5Fgz7Tcqozx/rG3yETG1Cthq+A+A7IcxMzSWWxTEUNXTYSfjJGVcVaYi/QJ31Tn1sZuQN+X0hykW/sr+KJdlUMz3KX8p1u3IxfjVpgJfv2jqpMllqq0ls2e7CEp9iaTKsRyA2AmxNTP5B/xMXBklBYRwshDGDhbFWIGhF/CMIsjc9ed3yRtox4gMzxbjzrmLpclARTFYexvbWg3mqUjOm5Zwu2y++yfTjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qeWsMypjQJE5WJDxavYLccIioQ1C4MVe9l+DC4ES6oU=; b=qM/vejop5I2rVN5uJOzc0woKQGuSqC5kxt1NWE58Yo5y0dALt0Mj+/71/79POgvVegDPLuWQIfGGbKGHOal3LNJ1MQF9WsAyFCJ2/9223ALzCucSOhFK9QHYAV6+qBWem/fKeZexdr5UwAmcvaw+o8KUpru8/WaFgMst1sIObuc= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:05 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:05 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 29/43] PKRAM: ensure memblocks with preserved pages init'd for numa Date: Tue, 30 Mar 2021 14:36:04 -0700 Message-Id: <1617140178-8773-30-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:01 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ac83f2be-0e33-489d-86b1-08d8f3c290b8 X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: waZvgu4QA0MAWVX6D3VkK8Dpn4dFEhzVFzHgyyL2q7HaWMsCH7XShKSMbpbwYVtmsV78m/14EsV76qqqI7oCdJvGaUhBCov10rSvSmNvI6WmU0yKHdRCNbcHFMCj0xLPe4ycpmWlEBq1wtB8hPhL7T5bjfxjnAGz1Hdnb0UFvpq0Bb6WYilldBqM/z8pvGTuFfQDD/zZmv8bMFFEYLTZv/hEgOocTPZ5cuuGsoPRLhvRW1o1pYiVZ/N8u8gVArO0dL9DIrVxpyW24zM/jJLc2T4+ZkgKn8klWp4KGOuHAq70GPsv1/IItZZHkpzOxdX+gajSFVXOWu60ZHUX97g8V4NZdXkL1l1gwhJET1dhrRiFhOniGylZUA7XyHvKU8z1AOL4/0q3KXEOdedpG3Tk1twT8VpMLyRbIZWmO/joT+/r2qTabLs8A0MBpA2pItvTxiR+zp/tls6OWYx8CE/iY+7X8nsROuQ306AlElCbf/Y00mwTzaioMKKGC+i6AwXgedE9w3mttWITbhA/SthyMP4YeVwqaggAG7fYj3o+xAlTtLcLbocOZ2jna3gC8HxbVq2NBaBkik6tH7oUWA1zmP/G5gSLqEnSZnsEeSH4wPYuKp8upU1t6sgA8hF9tDbkdH16NHrrXXaa5cvMGPyKHQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(346002)(376002)(956004)(2906002)(66476007)(4744005)(16526019)(8676002)(316002)(2616005)(7406005)(8936002)(44832011)(83380400001)(38100700001)(6486002)(26005)(66946007)(66556008)(7416002)(186003)(4326008)(86362001)(52116002)(36756003)(5660300002)(478600001)(7696005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: ceqSPrILLk/4oVtRQcfNTngPkFrstVD0O8yeTDEJEchY3qyi7Nngey1EbqiNti2zw2hRWstBfqbH4VYd+uKFqp1E3j6c8hlusLD2/vgZsUIjTSARia2JXVHCE9L2+pBQOADtYDZ+pzmX/TFIv6SKeWD309KjmssAj8HE/qv92RiVmxKAG2INCxBusVCVTc3juT6HN89p7EuQQhmR8dKlpoDj0KrUwkHn/pRmESQ401GPsp/iOAXpxpDUp9LmVpOmBNRMfm9TBrzFAOvpvZRIjpGbhfEK9MWckfaP5cgQW21/3A01UlJjZxwkepryJDDzBzBB6Zuo7BnuAu4CtCluyHrdIm9FDcjS7B1eXZihF+ajI5EhrP49EQCiDcnQCx/xEtDp/eEpxhzY9LHQvntlO/Zl0pQmfjQ8RgLI/TIMSai/GUYs91FeKTrwrLUhLvhCH7ncnDPA6R2iq0QIUmGuEogeCLxYOHYjzgH2sdvexFba2NU1Zr21f+2Q4/50PFoubuk0dwxepiWeeSkJw08bn3l5i5Zm9xfpp9B6nYOt1fVvEEtxwAvWRFsQh0QHl4/DL9n5sv4CH4Vi34Zx3flkxj38VLajaG2NjFFZX92bjPGBAdY4oyzrwd+a8AOU4PbMZ2mi2IY61bxP3gPPUjMHbYTnzQGabctVP0sQg1nmz8LzImXi2ndCnsrzS91Kp603WZs/5CdVmTfPhRVOSdPNRtOka0pAsyjSNU189m7HtbkbBgtEqTZ4G9/JYG9kRpZzn+Y18yiPiCHzwva83OAX0HxWVHuy0gTBEbjJSNnOcuQgUeyIqWnije6wT/RK+PuZmmOB64Hth1jS3X2kjFHTpFg7jU7tDYUtFFFzxnojkB/tJ/P+nGv9/BYAwhEtZqqCpFdeqLRAMXNbAhT9OawTEcHslAUjhdurGQGLTgZ4LaW789Ut65B1zLc9NgWdwumRmbZohlBP6rgyvIMJd92RqTlpa+fs4dZhQOZbyjHSaDETCVbvYsWx75NAnk3D3X46+riGVW3B8xcdsm304UFkwQ9o3pA6/6AuSys0rNprkNH5IG1pQVLsTNRAeC6daO2filAPLtNALIEY6ZmMPvf2ruIjRhXyFfeV63wqdDPvjF5OyQUYdv80ekxEWm+F3s6UJqjqHVlNPSJu2CYmSI+jvnDvg8znuL9CC1o/yorMmsWrGMhPUrSGOpreokOrKkas/mnyD4CtdzeLkTeerVN8E47caqO8rB8w0vguisxM2UdXmJqmQjdViOjQG0Km0aLYDUNaacKKd/Ny6+YtusFreRGPMsssSub6JIUIK09NBB7+6XRUL7LROzMJHqcQeCb6 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ac83f2be-0e33-489d-86b1-08d8f3c290b8 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:05.1591 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: g9uaacnonPXPusJgIFgxG6zVg6v6rbwOvnNblZi3acnJR1ZXF3inhQ6sAdplTBsPC8aHvWo1yPTnbJoBFHT8UuLEnxKgOJUZFx/1Bak1wX0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: mtrY_P5IPbIHk2O2PiRKiuAgFw6QSQ61 X-Proofpoint-GUID: mtrY_P5IPbIHk2O2PiRKiuAgFw6QSQ61 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org In order to facilitate fast initialization of page structs for preserved pages, memblocks with preserved pages must not cross numa node boundaries and must have a node id assigned to them. Signed-off-by: Anthony Yznaga --- mm/pkram.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/pkram.c b/mm/pkram.c index aea069cc49be..b8d6b549fa6c 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -21,6 +21,7 @@ #include #include +#include #include "internal.h" #define PKRAM_MAGIC 0x706B726D @@ -226,6 +227,14 @@ void __init pkram_reserve(void) return; } + /* + * Fix up the reserved memblock list to ensure the + * memblock regions are split along node boundaries + * and have a node ID set. This will allow the page + * structs for the preserved pages to be initialized + * more efficiently. + */ + numa_isolate_memblocks(); done: pr_info("PKRAM: %lu pages reserved\n", pkram_reserved_pages); } From patchwork Tue Mar 30 21:36:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C2D7C433E3 for ; Tue, 30 Mar 2021 21:29:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60A7A6196C for ; Tue, 30 Mar 2021 21:29:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233200AbhC3V2b (ORCPT ); Tue, 30 Mar 2021 17:28:31 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:51790 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233034AbhC3V2E (ORCPT ); Tue, 30 Mar 2021 17:28:04 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULQ0SJ145826; Tue, 30 Mar 2021 21:27:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=h/B1IVNXrZ1tOavAPYPL5lfgzoSvevvaYjown85Jdf4=; b=sEmEYLEOSB2eK0tgqiP25aOBz5/4oHsGRviOHUNcRXFpNIafm/jqUMqqAS+6CS2myUZI lZ8Uk2L4ORjXD7yMBikjTfEZfzuoRWe9HkuoP7tQ2uTY3Q+MfJL8zXnOAfstCLqn6lsF 3T9Gff2V8sxY3M0C0XCSvChiYFWZn56C92ao4QYTPz7W3yC3MWqQJcEmQUpqjolM23HC 7lHFOc0D9ALF7iVVSetbpKMD+JRM4DH7qVSLNMF1zA8AgK2DxdI+pkUZIT+uURHQmhsX bF8NYNmc4N1LxuqdiVLltHUo6I0v8wfdtLjXALQZr1354/6iaFhEz0O3uO+xrWmA0xeX +g== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 37mad9r8h9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:12 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPSbs105924; Tue, 30 Mar 2021 21:27:12 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2104.outbound.protection.outlook.com [104.47.55.104]) by userp3030.oracle.com with ESMTP id 37mabkbd08-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MDGtJPkEb4jlTuWKNvTtqzoWGXRagOm2GevqP6FX5OeeyNq/7l+MGg6qJxzqsg7aP5QyUlxzfYvW20wYhHWWKyBZnrpMTaGtS8XNgeuO6Vhp03t5FDRUhT3x29yu2lzKTvJ+1/MpL46pGlIMBMofZPtt37AYOlrN1g70/DhZ4zgeCdiVZwrInYtw/lLY7c62dwdZwtUym0HFU2f4beaI9EBWO5MUrGDk4s5hQpVNPpeNtbqYLt5egAs/hF1+Aw8VuEri+hT9if77uMGKLATLtUTuiWd/vNgIX9KtorfI1land+NU091SxLWo24CKpyrE9KRlONo9qGjxnLCrgNNawg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h/B1IVNXrZ1tOavAPYPL5lfgzoSvevvaYjown85Jdf4=; b=eQDS/2QXMojiehU6ENEceLC7Zh8ZIMBkRVN04PfHFOC1E5z8MpXhBsdn8XFwmmncEVsVoODjnmCipc8jSQdFN8hB7sTuH+pEPTpk/1VDGicEPIrV4pmJiERqECK0rIuWlLH+9TeOowaWSOXG22d5wOzY4nk9IvjzapV0QZuNzD0i3hBnQI21faa7LqU3H3CNDqXCUrBXnXvXupesf8s463ysxPuY/QCsK1WAd7yOEji8bHYw8Vwe5DJt23jKKtGn6N+XvdOoqCyY6mBY0exwqQC5WGWmTldWEb9B5mDm1IfF0UQWtqqjWo+BmCdX8L8VGfLVYdoFE/jJeszixS65Gw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h/B1IVNXrZ1tOavAPYPL5lfgzoSvevvaYjown85Jdf4=; b=X/+3VZ+ul2SJSibVDiK9pXWuACFOgW+6MZ4NdCMdpNLDuj9wfyWXIiJjxg+s2G4n7C2gzHvj6DHEYUCl65+IOnsOj6uOf0bGD5IE+CUsi2GtPVtAn0/VL8i3dSq8jJCsvs5XITPj+yTOmCczQuT84rRxVYbxY9xrqECg+mXaj0I= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:09 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:09 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 30/43] memblock: PKRAM: mark memblocks that contain preserved pages Date: Tue, 30 Mar 2021 14:36:05 -0700 Message-Id: <1617140178-8773-31-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:05 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d691f5c6-3b36-4679-71c7-08d8f3c29344 X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YhReXnxPgfP1TzqPJBGTGVO+x4PSyB+cORTisG/QF6H+YRIGmur6uKqA76ksK4yNPhGZjKqstAX5CcjcTnW6Gro98FFRJUKCvjkonVzLXu/5qVb2489ElPPlbQ1wRroByr6e6wAgr+Xi4Xv4m2ORcLG6o4+bSOvGJQeXKbKZ63LJq/vBgXXa3povI/6oQaXb01jhhr5lCDxDsHK9HbdAi6e/MG6F+CWhgZrJWfGXEPQnl1ii+5M7143kRCiRryDwHBYE64JKM9lqtozfhjDJZbBHXN4a4Alaon1A7n/5+RhzW95YCHBFS/3mgryYHytEgujUrwbadYEzR90jM2QByGusOHxaoo85HGtVPOrxsf75BEJ/j4gKGFGrioPQId4v3WC6NLSCo7ZIAolost7DHkCiSQC7ehgaskD+tVrEt6P4r3ZiuLd0fOM+1ppGxBbhdVzzdEs+Z6qU0ROm+rG227LeeQe3FRqPG1sz+ajLDdmadQFwNfPzxSRiPs9MN+7WwHdbMnpuYRw0kO8fLIYHyS9zbPs9tIoSbRgD/9nLIkPSUzNApFbZEFrUmgb2FCbkCpvqIO4/h1w52KhI85udX+pgVsWoGkA+UkryxMLV7VIgd/8K0tYM2MqYuMPODP9FB7FvRmK7+GyXbSJku3K3tg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(346002)(376002)(956004)(2906002)(66476007)(16526019)(8676002)(316002)(2616005)(7406005)(8936002)(44832011)(83380400001)(38100700001)(6486002)(26005)(66946007)(66556008)(7416002)(186003)(4326008)(86362001)(52116002)(36756003)(5660300002)(478600001)(7696005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: B+gHDkJChqsAllPqar+wwNO4a3qv+smcGlKlLU3mF1sEK+yTG0ntJOWmwRiltMnAp2OdWnZ1IYHgHWCIZeJ2VZsd1D5qYHYORQUTai/NSPO1ycjoA3IeOkz3AGhoVh74kg2gmyHXeldZz9oCLozwF6sf8iEvEVWbZY9AJyOylRlKgyrmcrbxWcPfYaTznICZW7U2F0YozkI94+83pvwGTbnjtWT5EaRCYqflUEWBf7hZlxVpsuCM8IXAJtVjQoOOfAMYEZpUbRfM8h2Nk34QSPcrxhVwmf6K2IYLJrn73/SA2EcHq7ZMiBaQ2EcX8D0tq80PptLsWNjQ9Q1NuuQqGDoOX3rPWLbQ7nWXw6EVEY0hF33ofUbO/bqsvkGW4wUDtD3il9LDYBaPJl+OVQGLDqktcIpzdkl5ERSEciK9OCqcGsG6PelRZy+YB2RD48qpl6N8sgDQVaq8ohnKWNCZARIQO4C69+uwGyL1g465xMuq1Q7d944eSnqfJk7jjeJ1KHz5WI8iwdU/aSWpDdvirsw/kt+CyPT/iKgy38qvtXARyzMrp4sIkOrGv4eAeYPuAiFcvXUvw4lb0poabdQ0o4Jqo/69CYL6Ew7TC1lTOFMWYAB8TWHCMxRry17oeFrJdvp9DGEM2olEYENhAYEZQjgxfIBCY3grHNPrYgr4vSEv5NGpKLA1XjnWLqrqT867Bg9iKFNPLe7pF1qOIta46SI4IB8FikJ3dFdCINkfKZseo8KwUS9niID9tSq6qYiGHeIEy4HHi46pO+8o59QK4hMfrYkkdGheGES8BbCKYIeYBrYWA4oSS4K//JNA7+8fCaq2HW2kZJfeNYTkoDdb33q9OltWiB938fBWX5IGnb6D2EAKDKzsSWgFQED0RKKiSaqa2Rws8ST7qQSteWhoBg0dgXY9CHZKbDav6BPDjc9nEq4A2UMaUs6zVeRX14bC8PK3PrxAQWEeTHSF0yCyKB/FDUnxCA24j91eXiMIFcgslm+tgd2MRE18/EElz/wdS8wtlyhP56IDcxAm4d0fD5rTKK45vngn+gywgA1Kh1G3qscU152JkfR45TiWFvYXMODl47WolKAGI2lunOKIyRBi4D3ZnWcf0UEzjBjIHhOuZ2f7SEKxwQg0+ijeL+/bIC1aLCdhC+V1zWTNAdgMinoq3o0Sqby2V3h0eJC3pEwFmSNOVfTAtjSnsd7SkbtF6ebYyj9v2jWValvYNRgVzpnqx1WJBzd8M1RHkGV055EbasuBMXlfUInTkF9z9wpKu0pauND4bIHd+p6xeKkqGFfRntbqPHCp0ymykmFZTdSxPtMSfF18nIMcraUKgn/L X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: d691f5c6-3b36-4679-71c7-08d8f3c29344 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:09.4795 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /ukU4hnqe0r+rMIv7qE+SWLBE6zWuVyAJCTsQkSHmOqDgxrixueTJptFhiJQu1VNpXLlCWEIdEimUIqxYJnUOTAZBAMoB/ANO7svWYsT2Y0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: vjRxetZm2Z7TMBnOp4p4piIPyhwcci8w X-Proofpoint-GUID: vjRxetZm2Z7TMBnOp4p4piIPyhwcci8w X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org To support deferred initialization of page structs for preserved pages, separate memblocks containing preserved pages by setting a new flag when adding them to the memblock reserved list. Signed-off-by: Anthony Yznaga --- include/linux/memblock.h | 6 ++++++ mm/pkram.c | 2 ++ 2 files changed, 8 insertions(+) diff --git a/include/linux/memblock.h b/include/linux/memblock.h index d13e3cd938b4..39c53d08d9f7 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -37,6 +37,7 @@ enum memblock_flags { MEMBLOCK_HOTPLUG = 0x1, /* hotpluggable region */ MEMBLOCK_MIRROR = 0x2, /* mirrored region */ MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct mapping */ + MEMBLOCK_PRESERVED = 0x8, /* preserved pages region */ }; /** @@ -248,6 +249,11 @@ static inline bool memblock_is_nomap(struct memblock_region *m) return m->flags & MEMBLOCK_NOMAP; } +static inline bool memblock_is_preserved(struct memblock_region *m) +{ + return m->flags & MEMBLOCK_PRESERVED; +} + int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn, unsigned long *end_pfn); void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn, diff --git a/mm/pkram.c b/mm/pkram.c index b8d6b549fa6c..08144c18d425 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -1607,6 +1607,7 @@ int __init pkram_create_merged_reserved(struct memblock_type *new) } else if (pkr->base + pkr->size <= r->base) { rgn->base = pkr->base; rgn->size = pkr->size; + rgn->flags = MEMBLOCK_PRESERVED; memblock_set_region_node(rgn, MAX_NUMNODES); nr_preserved += (rgn->size >> PAGE_SHIFT); @@ -1636,6 +1637,7 @@ int __init pkram_create_merged_reserved(struct memblock_type *new) rgn = &new->regions[k]; rgn->base = pkr->base; rgn->size = pkr->size; + rgn->flags = MEMBLOCK_PRESERVED; memblock_set_region_node(rgn, MAX_NUMNODES); nr_preserved += (rgn->size >> PAGE_SHIFT); From patchwork Tue Mar 30 21:36:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B6D0C433E8 for ; Tue, 30 Mar 2021 21:30:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EEF6B619CF for ; Tue, 30 Mar 2021 21:30:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233483AbhC3VaM (ORCPT ); Tue, 30 Mar 2021 17:30:12 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:44076 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233357AbhC3V3Q (ORCPT ); Tue, 30 Mar 2021 17:29:16 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOvvL011652; Tue, 30 Mar 2021 21:27:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=NHGH6ndUHZ5DHk3h7VkhiG92ASmxYIQKyhSSVSB4ZGM=; b=VX4lGUu8LQcppxTZeBeB22Vtd36LymbkIr8DDZlqAHhR2V/Ss/PvmLIX33f/y+gPbCk3 5envP0bpjN0JKknyGnR0R4f83M7uw5jWIvDVzxaeTnEAhy2YHD6qMRlmBX9wtmAiPe4Q kqzwVzTkFbCaIDnh2lnoppMQ6QOp7E3whdJelS1u7oUSinZ/jUj2YT4DXwvoFctUmv26 1/ZEXYG0VdrNBr0IrqZyg82j5Ndk/iDlNVnNnB46GMhRyqmH1G6SiD+gKDi/Cq8R5Cis yP74IuKMeWmkNrGcUV7FQvcJr2/e/Mz6iX5n1M6iqqEN+seHXbPi4800cptYBHB1hOHI pg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 37mab3g8yd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:18 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPZFq125048; Tue, 30 Mar 2021 21:27:17 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2108.outbound.protection.outlook.com [104.47.55.108]) by aserp3030.oracle.com with ESMTP id 37mabnk7w3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c04z8XXuleftJYRZmcRB9p25by3iokCCnVa3EuOFauAe7RunDvHYeXfVx+bwDiCgSq3vv5NFeTm8dvJv2zFXTbu2mC+G3iURa9GhDqviMqX3UvK1Kiu6d8JpPsf48Gf2tvTG0Y8DXEa3hRngKuhSZCgkYgzT9fHg6fRD8TVBfp1HmK9gqictkgMk0+bQyumJfKH7EiICPH4QgmyFO4MeOkl0k/ASOzbeO8GQ7up/zDJl435iN+W6l1JZgbwDivRQx1ZOHGJERGF7XWXh2s8+ILfoYtJmzaeu+ntCKm7bP7yPTaD3g4lf2Cf9h8NfNqwT1szw1N1EXMfVJtEpjKY95g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NHGH6ndUHZ5DHk3h7VkhiG92ASmxYIQKyhSSVSB4ZGM=; b=PcrjsZ2cJWjksr81qtVtnl+A6eaZdnbvZQDWIFIIwfjDeiKBVHbfBPU61qw9EWVkfSvnP4m3Ux9qe5eEzmelK3/UdyYINDfbhH+oNhIkNlLJSI7JbVG1EeC73blETbi6RUEmy9wfxyqPQxsIV3ductU61uOpDrt5e8lnX13JEGYInDvvKjls7k9uWzlBQszH4OiC6XTHbUx7416AUe8DAs+WmZu6BhOpMkTEB0CwGwkb8eM4F/h7ya2JjYGlG7FDVGNPGJRUtvAPjXscfzWSkyXpirHXgya1zsVNR+gi5o+cfh3sWLE3rrIAi1gJltV0OV75YPsfHo9uXGPEy22EyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NHGH6ndUHZ5DHk3h7VkhiG92ASmxYIQKyhSSVSB4ZGM=; b=tfj+7CqI6CoufZNknPgQaI3QYkLYOIC0f5uG85BwO/YRkiZj49c6P8230kBr55YqKOoXFWuzrC3c7/PSs9MJf90l9YFljZV0PHjfyeBUW4cAlCaw2NdPl32XZ/5/L48ld2S02o4iBDafooem+sn6LIZVBjMO4oaqMCX/J6C7HRE= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:14 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:14 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 31/43] memblock, mm: defer initialization of preserved pages Date: Tue, 30 Mar 2021 14:36:06 -0700 Message-Id: <1617140178-8773-32-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:09 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f8acbf99-9e5c-4d72-0901-08d8f3c295e4 X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:923; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rXGz90M7GEoBjJp0YmpV0C6DeLzrGiGiWavsTQHYpR54oDUbjGoJ/UJEIbGCKB21JFOJ1+fAJW5rU4E4XJYcww7z4Yjx+urXzQ7+r/7PmDrI6e9FLl0aEQzmH2w1FrAjkEQ+thytvt87e4BEQ8WGOMTl/TccrFkF42QmmmtA8cHoMkpmlrK97/epJU2dvQm84hhNFIEFiIPKhMJjU+qpMQhHymvGja5HJR8aImB4HU5Htu7q0yi1buaWYy6xOXSNtI3ke/VYWApcqy5on6YbGrYSDVe/DhFSFHr6ZgXF1iHmrwHnZQcD6xEq/7yMQMZZQtj7mgeXo5GNUM8R636dCQMrLc9rLw+gu92VAEpO1fUR6dye6R+bW9+v3VEqkI4uwTPaPr9trzCzODMrFPnpiEToS8yr65ya+u8j1nHwDQ6DBiGIOYl3N1ATi3jKAGvie8c1sUy6aDdUGIse+VpTFyQLWNbnpP6RYwMwGDHfWCF3XeWCoWcEytpl7FbgSwR9U9w3Ne2UAMnZwfhBi/XBGp1GHdh9w5VEOwKNMsJ2I/1rRNOD7HbHKwTXmzxPGhPiFzrT2XcCTij3pYSMMKz+mj02dvlDUCdM8M1Sm8mcgW4mVCJopuVKCR3QeyQvlli/Hcd1dOn7erOilfxeMns/tw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(346002)(376002)(956004)(2906002)(66476007)(16526019)(8676002)(316002)(2616005)(7406005)(8936002)(44832011)(83380400001)(38100700001)(6486002)(26005)(66946007)(66556008)(7416002)(186003)(6666004)(4326008)(86362001)(52116002)(36756003)(5660300002)(478600001)(7696005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 7cj/HhsOgT2nxjIFgpq5E/GFlkNEmnnqysNktHsswiLgy2Li61vKoFSlnp7hkETe+ROFB47nqrxdPcEbMN1VFqn++Fq/35KgvZucmRcLYdfO9KctQquzpUnnHTeENmpd3nsG+Xol8rQoT7DnyIUvm//ORRtCM7crAfeYdIRF35cv3L756LI2I3n3Qzajbas2BeZq2VjDGRn/mvjZ+8YjoIerbCNESsby3iPApIqpedeoRfp//iX4IwiDL6GxgQztn9x9tSiuE4myocgDKMxyTwv93xyHJOCTU6zUQimVVoyn/DlESma0co9J4tTKUmMTV0gmCRcZVw1At2LsAwLrVq5WMR/OrulRNZLeojTnVpY7/Uj62zVa8kGnMliXNl5RcOYpESefxULSeMeKR/0isucVqMCz4OY96OEZnTZ/iCFYcWVZ/S/6DyTOip7wcXGtDuo1Rdf2w8jI1FZNfA1vLZYoYtBNGXPEMdltRKdV4sFbNtG3NPPUIRhRfNABzf4F3IOyC2yUU/uqvQ28ZGHNPElQP/H/2/dRsHhVVGEqa6Gvh4LG2XgtG9jJy7N+24XviWzKSUQR3Noq3di8EbELh1li4qTHGj03/lX+lilC9B2ghnxySu+bBEsovztOXKa050S0K+Qv0EXdYAPGA4fCYP5Evxv7ybgKs5WPP77nVCGh81xlGvF9kHbOp7mgMG4eOf7p0vajnP81Xvkg2bhj5Wgw5mdcems3in6igJB+87zdb3Aoj2qijNUIBOAGHedhfLFVfoG0wJykxEfrXQKBew3UI9w1OXGUIa8ql4fQc2ad+fN/bNvzf86tupbpVGaRV/m8lP3xQrZJ1f8H6LcJj6MToryuZ1Ti1F2Fw1lWfb+l+j9ix6ExK0xt2ika+e3mjRhFNYSC1iu78WmDExKZycVtfQyeJtAedpLqlRBJwYTAEGGcDXlJmoxaIKaYOq8BBLMv+Wje7rA5h5Nvk6mci7FDXsiGP0lYSuqqPkCF+q/TsqNFQxUDbgZDtP0qitXt2PyExw5FLxixuWX34y9rg3tTXaXUN42el6w0NWwmmZayPWjxyeXRQWoX/EkgA1AXLc2/0UrNLDLkiNU8rY1K4yv0KqtyJv6tmGN9O4NhAsYeRc1y4nrqZz1idqwhhHCpF2pEC14lxE+X8Jpy6W0qchcMQWo05Bf/lwAUc2opsblyCmx3RVufJrRoG8pWptfIYXYZwhTzBGyWQrNqw9d3xglQKYIctu/8ZCouZHxsWIZiN4vGwgsLFAx1Lyw6BsY5T/FaSbxi+KM3dAiKXuAUnz3BemRushMa98If60e6JcaLxpDedUHQLQZCiX7gcTAA X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f8acbf99-9e5c-4d72-0901-08d8f3c295e4 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:14.5824 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oc2w91Nc4yDtfRtNwUAxJJuO7itB3FX7FVCzz5dkdq6r5YURA1bNPFST+3nHrsiTAT4QhhgdQGPoSr9F3ur7iPZkL2Aljt4BGuJTf3pgyKY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: Horf0USydyvf5IQz3rkxBKd0oRJMI9V2 X-Proofpoint-GUID: Horf0USydyvf5IQz3rkxBKd0oRJMI9V2 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 phishscore=0 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Preserved pages are represented in the memblock reserved list, but page structs for pages in the reserved list are initialized early while boot is single threaded which means that a large number of preserved pages can impact boot time. To mitigate, defer initialization of preserved pages by skipping them when other reserved pages are initialized and initializing them later with a separate kernel thread. Signed-off-by: Anthony Yznaga --- arch/x86/mm/init_64.c | 1 - include/linux/mm.h | 2 +- mm/memblock.c | 11 +++++++++-- mm/page_alloc.c | 55 +++++++++++++++++++++++++++++++++++++++++++-------- 4 files changed, 57 insertions(+), 12 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 69bd71996b8b..8efb2fb2a88b 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1294,7 +1294,6 @@ void __init mem_init(void) after_bootmem = 1; x86_init.hyper.init_after_bootmem(); - pkram_cleanup(); totalram_pages_add(pkram_reserved_pages); /* * Must be done after boot memory is put on freelist, because here we diff --git a/include/linux/mm.h b/include/linux/mm.h index 64a71bf20536..2a93b2a6ec8d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2337,7 +2337,7 @@ extern unsigned long free_reserved_area(void *start, void *end, extern void adjust_managed_page_count(struct page *page, long count); extern void mem_init_print_info(const char *str); -extern void reserve_bootmem_region(phys_addr_t start, phys_addr_t end); +extern void reserve_bootmem_region(phys_addr_t start, phys_addr_t end, int nid); /* Free the reserved page into the buddy system, so it gets managed. */ static inline void free_reserved_page(struct page *page) diff --git a/mm/memblock.c b/mm/memblock.c index afaefa8fc6ab..461ea0f85495 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -2007,11 +2007,18 @@ static unsigned long __init free_low_memory_core_early(void) unsigned long count = 0; phys_addr_t start, end; u64 i; + struct memblock_region *r; memblock_clear_hotplug(0, -1); - for_each_reserved_mem_range(i, &start, &end) - reserve_bootmem_region(start, end); + for_each_reserved_mem_region(r) { + if (IS_ENABLED(CONFIG_DEFERRED_STRUCT_PAGE_INIT) && memblock_is_preserved(r)) + continue; + + start = r->base; + end = r->base + r->size; + reserve_bootmem_region(start, end, NUMA_NO_NODE); + } /* * We need to use NUMA_NO_NODE instead of NODE_DATA(0)->node_id diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cfc72873961d..999fcc8fe907 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -72,6 +72,7 @@ #include #include #include +#include #include #include @@ -1475,15 +1476,18 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn, } #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT -static void __meminit init_reserved_page(unsigned long pfn) +static void __meminit init_reserved_page(unsigned long pfn, int nid) { pg_data_t *pgdat; - int nid, zid; + int zid; - if (!early_page_uninitialised(pfn)) - return; + if (nid == NUMA_NO_NODE) { + if (!early_page_uninitialised(pfn)) + return; + + nid = early_pfn_to_nid(pfn); + } - nid = early_pfn_to_nid(pfn); pgdat = NODE_DATA(nid); for (zid = 0; zid < MAX_NR_ZONES; zid++) { @@ -1495,7 +1499,7 @@ static void __meminit init_reserved_page(unsigned long pfn) __init_single_page(pfn_to_page(pfn), pfn, zid, nid); } #else -static inline void init_reserved_page(unsigned long pfn) +static inline void init_reserved_page(unsigned long pfn, int nid) { } #endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */ @@ -1506,7 +1510,7 @@ static inline void init_reserved_page(unsigned long pfn) * marks the pages PageReserved. The remaining valid pages are later * sent to the buddy page allocator. */ -void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end) +void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end, int nid) { unsigned long start_pfn = PFN_DOWN(start); unsigned long end_pfn = PFN_UP(end); @@ -1515,7 +1519,7 @@ void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end) if (pfn_valid(start_pfn)) { struct page *page = pfn_to_page(start_pfn); - init_reserved_page(start_pfn); + init_reserved_page(start_pfn, nid); /* Avoid false-positive PageTail() */ INIT_LIST_HEAD(&page->lru); @@ -2008,6 +2012,35 @@ static int __init deferred_init_memmap(void *data) return 0; } +#ifdef CONFIG_PKRAM +static int __init deferred_init_preserved(void *dummy) +{ + unsigned long start = jiffies; + unsigned long nr_pages = 0; + struct memblock_region *r; + phys_addr_t spa, epa; + int nid; + + for_each_reserved_mem_region(r) { + if (!memblock_is_preserved(r)) + continue; + + spa = r->base; + epa = r->base + r->size; + nid = memblock_get_region_node(r); + + reserve_bootmem_region(spa, epa, nid); + nr_pages += ((epa - spa) >> PAGE_SHIFT); + } + + pr_info("initialised %lu preserved pages in %ums\n", nr_pages, + jiffies_to_msecs(jiffies - start)); + + pgdat_init_report_one_done(); + return 0; +} +#endif /* CONFIG_PKRAM */ + /* * If this zone has deferred pages, try to grow it by initializing enough * deferred pages to satisfy the allocation specified by order, rounded up to @@ -2107,6 +2140,10 @@ void __init page_alloc_init_late(void) /* There will be num_node_state(N_MEMORY) threads */ atomic_set(&pgdat_init_n_undone, num_node_state(N_MEMORY)); +#ifdef CONFIG_PKRAM + atomic_inc(&pgdat_init_n_undone); + kthread_run(deferred_init_preserved, NULL, "pgdatainit_preserved"); +#endif for_each_node_state(nid, N_MEMORY) { kthread_run(deferred_init_memmap, NODE_DATA(nid), "pgdatinit%d", nid); } @@ -2114,6 +2151,8 @@ void __init page_alloc_init_late(void) /* Block until all are initialised */ wait_for_completion(&pgdat_init_all_done_comp); + pkram_cleanup(); + /* * The number of managed pages has changed due to the initialisation * so the pcpu batch and high limits needs to be updated or the limits From patchwork Tue Mar 30 21:36:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D803C433E1 for ; Tue, 30 Mar 2021 21:29:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EA63F619CF for ; Tue, 30 Mar 2021 21:28:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233118AbhC3V23 (ORCPT ); Tue, 30 Mar 2021 17:28:29 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:33356 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233038AbhC3V2E (ORCPT ); Tue, 30 Mar 2021 17:28:04 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPTdN123025; Tue, 30 Mar 2021 21:27:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=v4aKNqr6h6mbPJap5xgsm98rRBX6spVn3y1s4HGYwxw=; b=uHhTjxfkhabtES0WCndF0Tl3/vgIf+yWZg/NZWYJEKGk576w2VkEgfE0DMmYn/PfMO+e MEMInYG5Vg8YLci5BVt8HzEgMldMaD+BvKF9i7pC+qEATWv1J3UCStADWG0+tiVs8x7e /QFP/UDfdFHp67BlsYWPMnH3ShkYvb5UJzTlPPPO+zSA6mjnD9DvNt9FRnpBjdAdP7vR 5KWpFeBgvOgYIf7VY0+SVc9y6RibNil8Ay+uKi0c7Y7SnxgNiuVM5SaHA96jaBixiUYj crc+VzGhzVKeHbCbwI3g+48DBf7czsLQbeTkExGE/jhBc2033OkhcExoExH4bi2XLZRS TA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2130.oracle.com with ESMTP id 37mafv084g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:22 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOlEe183911; Tue, 30 Mar 2021 21:27:22 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2107.outbound.protection.outlook.com [104.47.55.107]) by aserp3020.oracle.com with ESMTP id 37mac7u57j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=E37sln1YcQligx1p0HZmFGCQ4DgCqgejxCx4beitFDTg/nDdIxV3LEgCt+edrWEyzeC7hTjfo4ak0l/enIPeAfbj+zpp7xRvG8aKHUhod0qM4mB1neJASnStdLkdDy4KY2R4NFxnWPIZuCxYavHGwxVLQmxaeMlEsegCZw+puDjS/a9HrsQD/6UX7IODBllJrSoL2YVoObBd7/k5+ZPsRE7p6OD5G9NvFcOD8LGBZS6SeuYGiPcfUPR+xcwO3pC+FuT0BTpathTNHmVs0+lV/2BPCEhoihA/HGyMPHlByDg2USencUtFOSZPSSPvL1aEVLVziFQGtJvTeUD1xlSUgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v4aKNqr6h6mbPJap5xgsm98rRBX6spVn3y1s4HGYwxw=; b=TRy/XAksuZUUZxVrDP93/CWb3cBvkg933LMqLMCri98W55mi2qYCIjcZmh00YisZG3FMwY2DofAPt2lHZ1IxGhBk/SBxw6Um1I675zi2MAH48W+HUDefboWMx2pkvaYV6mlnBvEdxLwBkqJrECvpd23KM/y20SKlbwUTD+AT3B0iMq7Qg6BqDg+jd1a1A6e9Tzny6XLFMckN/Kg2Ud0+Is+lFW7RErsslv6P6SY3E+5vHyKPj1KSjfEJwpuREMCTgIDAwrwmcCwJKbc3N288oLHXLsS2evVNy+nJHADRlDaE6WVZVIqUmM+GEd4dD/5VZYGRHMEe9f1yr/PX7IzTPQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v4aKNqr6h6mbPJap5xgsm98rRBX6spVn3y1s4HGYwxw=; b=Q7B63SbalXIgjafKegNv/iI41vB/9H6fQp0fzOUPJD+dczm3UI7x0BPgAGXoXU8dwbfutSFI8m9QGM02awakvO91jFNg2w4TDI8nKemIctU++WmXBqzQsFFsxjazo1ElHRZm5MktRjeTN0kO5XRpjJ+YOvTyyEFemFkhcezUNKQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:19 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:19 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 32/43] shmem: preserve shmem files a chunk at a time Date: Tue, 30 Mar 2021 14:36:07 -0700 Message-Id: <1617140178-8773-33-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:14 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f8c43d14-9128-477e-d56b-08d8f3c298e5 X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gvo7lyHuuIXIIe1qWDdizLIbK9zx+GioIRkjmQB9UeAn6M1kneNpA9C4bUpKpHRks4vPrCB0n90zHPtLfBCfI1q5rLV+wUibEgBnlwppmmlKlc0MB3+tYvlPlRpDS7GNB0tdmdmL+/dQFVF+uv12HNWN5cHVlZVvZUzQ/l3E2UPpTE/9xeZp1CIiHhmt6nnTI/e9xtjgPp9CZIBgJvdpxxYn0LyiTrgelZQaizBTBOhOpQElgniSdcxaiA+SGEd3xnjie9oszmS353uI3gbArJ1O85zJZMRGDD1WrdrF0o4+g4/8bCdWHSKegMpdh1bq0Ir0JZuq23D4/43yPF7OoPrN9wEaoKQDfi9l72XB/AYYd6lqlbiej3YwcPNi4lXFzrVUeVc6h81xXMENcA3gpmozJvI6jvqFTvpvHXMVgufAMaJZOufw0PLBEO2w0iyVy+5XIHsKy1RP/MIqggNSkYkJalLa496hZpQQh//Ygq/ZVufcktlUv4Q1qa5CqtA7YUzLV+nv5W6cox6mXbn24aBR7jCm607ZCBDduqc0B68qeco15QryeuAxEAq5WOqYz0SdqlL4l3HN63NweOoA+a6tim58tHX997DXG2nJGOUg9Dh+kPD4jAroQdSOZBUEMvRzXMpKfyzuYT+/jNNTSg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(346002)(376002)(956004)(2906002)(66476007)(16526019)(8676002)(316002)(2616005)(7406005)(8936002)(44832011)(83380400001)(38100700001)(6486002)(26005)(66946007)(66556008)(7416002)(186003)(6666004)(4326008)(86362001)(52116002)(36756003)(5660300002)(478600001)(7696005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 0lWEkjh7OIxFueF3uRv6OC9n/BznyxvEpMioJmfWSGs/5rBb3quti4tihvnGBYpyE+57atv0wXu/E02K4fZ4u5rM7pUnDspX0XchLk+y5FOWq3Lrbo1gzi6+WMaZMPUWItr0Id4ENYm5pCec9/DO2j36+oTGUS1/NXhN0WsfJTozyeQeuGAghKX/YPBRWgy5TM3SY60iB6SWH3Dzt1Ts49GxCtrXyJpAT4m/oHRsgY+eygDU9nSl7aSC+H7AYOr1E2GN84EcnmbVQTPnafvx0xPH9dfMUwSbM4aU3UU/fKPs28+C63FB9T5OMUQDuKPWSbnKbfAEZ24NtCHhhAnq2F1XQSrOFUuR0YVCSpjPxdjlgz5Na+GzDozYElInRC/HT2mzoLdtsOqvN7qDvJ2QBjI/W374EuZqO2uKLQqHpyLPOwH8+mHDKYIzTNrAR4AK89kmaC3PpXBj2ZuuJwmROZ/JWeJQeppQ6clqDabxUAgDZJIY/b6Vt9PNMgXU2D4nNSMDrA862p9QAUpKJY5F7/mqKyB21C8WTwz3gsdpeuSndW9U7ZcCaRjVQeHGbljfFB+cFnLUOScMBd9CtgH1REafHuPSbLYLXXabmzq/ygW3wUG5qp1M0om+ojN5ANdYFkozIxxkCf+Yrb/uc26Rouw1DQzxtnj3ehW4D1vPyeS40DkAEC/2SGTlXIQHfUb++bUizaNIb1pbyQ/dMX/MaySl4RLnx5XxUUz989wzyChLqsSOceQQMPreKE508pBSDXCbDu4urP3VGTxygjE6hnVCcPCDDwK7YNUoY9wpPNXblJmK6AzVU8Hnn4l1DXW8BBw99V/Jsb9+tQKMEYndnAEqpAs8BL0n+rjs9/Vh2cluvFRpWauV5HVNQGkBvWgMhGsZ7h/ptr7PHlkm/rDDyeVpkAWejLN1tUjylZ+ED9HdBuGpkTe31uUTsIzQuASLhcpAQnGqnE9LxeLXadjm7eiVfoPgbs/mb0nHw1+XfI8lUT16li/aSlUh+o3K7ggFIySVsuqfnZynzbjIFyyt3goxX7jnFsWrpv8UsxUe0d94/B0XHE0U+d1ptsf1wnIEfciKuqWbIr6f/8PcwBfFi2bfOU8VieMJAlrpVHEjz+rpDQr79rYsWZprquej/M2JHtVMUKKrkqshQzTugsQZuWKl8P4TmMe3VTBr+slsGA1RLJIQAC27V89+sKkXJF/MKI1i97B59kRF+TtLmFUeapwZ4fUdIllIRvtiIH+ElAh+wmzfEYPJ79FWD0PccWHYd3z50XjdUe72HwgjdyowzwNbvKD62EbLjuk4Qn6Cd2xoghJiExZSWO2ND39M/asX X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f8c43d14-9128-477e-d56b-08d8f3c298e5 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:18.9088 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: I6NXyjsqO9feeZ7dgNmEGj6KaQvnopnUba1TOi3AQ2yb+HfHHdNOVXrzBqCgtnP9yqgS2586VdF43GhHejSUCwhf+tbPaY4phy+4hMbAMYk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: knRDIKqlIw1uoWbdPLSxBR9rvW_buTlb X-Proofpoint-GUID: knRDIKqlIw1uoWbdPLSxBR9rvW_buTlb X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org To prepare for multithreading the work to preserve a shmem file, divide the work into subranges of the total index range of the file. The chunk size is a rather arbitrary 256k indices. Signed-off-by: Anthony Yznaga --- mm/shmem_pkram.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 57 insertions(+), 7 deletions(-) diff --git a/mm/shmem_pkram.c b/mm/shmem_pkram.c index 8682b0c002c0..e52722b3a709 100644 --- a/mm/shmem_pkram.c +++ b/mm/shmem_pkram.c @@ -74,16 +74,14 @@ static int save_page(struct page *page, struct pkram_access *pa) return err; } -static int save_file_content(struct pkram_stream *ps, struct address_space *mapping) +static int save_file_content_range(struct pkram_access *pa, + struct address_space *mapping, + unsigned long start, unsigned long end) { - PKRAM_ACCESS(pa, ps, pages); struct pagevec pvec; - unsigned long start, end; int err = 0; int i; - start = 0; - end = DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); pagevec_init(&pvec); for ( ; ; ) { pvec.nr = find_get_pages_range(mapping, &start, end, @@ -95,7 +93,7 @@ static int save_file_content(struct pkram_stream *ps, struct address_space *mapp lock_page(page); BUG_ON(page->mapping != mapping); - err = save_page(page, &pa); + err = save_page(page, pa); if (PageCompound(page)) { start = page->index + compound_nr(page); i += compound_nr(page); @@ -113,10 +111,62 @@ static int save_file_content(struct pkram_stream *ps, struct address_space *mapp cond_resched(); } - pkram_finish_access(&pa, err == 0); return err; } +struct shmem_pkram_arg { + struct pkram_stream *ps; + struct address_space *mapping; + struct mm_struct *mm; + atomic64_t next; +}; + +unsigned long shmem_pkram_max_index_range = 512 * 512; + +static int get_save_range(unsigned long max, atomic64_t *next, unsigned long *start, unsigned long *end) +{ + unsigned long index; + + index = atomic64_fetch_add(shmem_pkram_max_index_range, next); + if (index >= max) + return -ENODATA; + + *start = index; + *end = index + shmem_pkram_max_index_range - 1; + + return 0; +} + +static int do_save_file_content(struct pkram_stream *ps, + struct address_space *mapping, + atomic64_t *next) +{ + PKRAM_ACCESS(pa, ps, pages); + unsigned long start, end, max; + int ret; + + max = DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); + + do { + ret = get_save_range(max, next, &start, &end); + if (!ret) + ret = save_file_content_range(&pa, mapping, start, end); + } while (!ret); + + if (ret == -ENODATA) + ret = 0; + + pkram_finish_access(&pa, ret == 0); + return ret; +} + +static int save_file_content(struct pkram_stream *ps, struct address_space *mapping) +{ + struct shmem_pkram_arg arg = { ps, mapping, NULL, ATOMIC64_INIT(0) }; + + return do_save_file_content(arg.ps, arg.mapping, &arg.next); +} + static int save_file(struct dentry *dentry, struct pkram_stream *ps) { PKRAM_ACCESS(pa_bytes, ps, bytes); From patchwork Tue Mar 30 21:36:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B9E8C433EC for ; Tue, 30 Mar 2021 21:29:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2BB2C619D8 for ; Tue, 30 Mar 2021 21:29:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233227AbhC3V2d (ORCPT ); Tue, 30 Mar 2021 17:28:33 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50410 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233058AbhC3V2J (ORCPT ); Tue, 30 Mar 2021 17:28:09 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOG4R130445; Tue, 30 Mar 2021 21:27:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=S5mxt1NIEyrRs7BS4f3y+XZwhYNVLCfqk6ZBT9heJ/w=; b=ftpagAZcp4SiMqaFdJ7S3YQJfC25BEDCzVQdVldIWCPt6V4YjBkujRy8BLT/y15r0rO0 fBmlMvJXD1puHCFNISXp2pH0THfjl2W/UpjrgqR+dt9nOlFNdNKXG8j+5L4/e3tnu7Bc 6EbmWBWTwVcnBVb6kQeuE27rb7n2jRpl23qQY2Fv9mFkJuOotQbl6E9xwFuHWhCpLJyD Kyb43hjoGu8+5v8lp8/BeR1F2IHnEoKQqe48FlVAFq0bkcVDcoZ7pH6BZGhyAPm/RJ8W Ypm096wLTUq7uhIp2dPu/+EMH7stqiT3HmRlDtHB6CwzDz0k3XGY+hXuflW1hcL+uSeW Bw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 37mabqr8qm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:26 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOnW5184074; Tue, 30 Mar 2021 21:27:26 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1nam02lp2050.outbound.protection.outlook.com [104.47.36.50]) by aserp3020.oracle.com with ESMTP id 37mac7u59j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:25 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A2BTvqLp1yhE5MbWaj03ZQkzTsfx35X3FMrg9VAlfZ2LYY1dF7oQvbE6k6VQT8Mtyfblc+zYmE/40g7Tw+qC3Al967+J4oZwFfA/KFKOyQcRiHQ3lHadMd7wrMnmlCuxsBqhAdGVxFxKfWGeHOrTD7pVQBFbPpN1RgeNEaA3+wy5zn6I3Z9dXlzcD/JaWowe3InjbDOj+QP5D+wWAb1YLHL+nEGNLMhu+XEYMzw4y3mqdJxYvlgP9BRZVfDjV4M3yen8Lzxwww1Ll8LcKF3yEOYbd61eA8S0zwcwddckuCw8QFP7aXTolXtvvZG8wNbNtSXi7sr0rOt6wE+nno+Pwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S5mxt1NIEyrRs7BS4f3y+XZwhYNVLCfqk6ZBT9heJ/w=; b=UUcx0DGO6ewhNhdkrJ6DboafCJ1XuqEUXWuWd9Ef1bBINIMUL7BZurJzYjOAgMeI0ZetX0teKDJyBk+J66Mzp4q2M5mDWrJQq5I3YaZHkbikjY2pBBYJlxsWDphgcpMS+v2lNLnbnM5KQ6GmGvblyDSi4oIWrSs9jA3VaIQuKq2DXgKo5b5utsy1UdHCEcFXyRQDU+Jf/banO8FdGf+EIjJQvHaraY+XfltYWngPaFrItWieF8NtmP/CD+iu2MngCKJ1RTWqSIopPA/Oz8nI6EfokbB2kCdfZlbAISZ5z3AWVCWF+rj+o6FPanI4zePVL488XD+YH6ULEQDWBWFVtQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S5mxt1NIEyrRs7BS4f3y+XZwhYNVLCfqk6ZBT9heJ/w=; b=HeHPyDdayOhpmTePn5O2bcH5jtQmfTggq4d/JGECOY/3Sa4eMOz65vDrw3sizK8fJpmRdR8Dcp3TjxFbdnNpSSUyCppv1GDKf0T93dTosz/6eUw5eWPqtMIVpcKzEsSQRIKEgGGQVwA+2Gni5q3JHPH6rOoaNk04avq9SpOgQ4w= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:24 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:24 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 33/43] PKRAM: atomically add and remove link pages Date: Tue, 30 Mar 2021 14:36:08 -0700 Message-Id: <1617140178-8773-34-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:19 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 59b955ca-22b9-4a44-a88e-08d8f3c29bca X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1169; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nehKIMHqflSPc5GbX/hVXOw3+BWvJNJJQHxQ2sO6zIWi6YKrmwzBEZ3H2FcUNlL7ABnC6v7B+xaU0GpbJw3Bvhohzuls9ekRm9lFi79AvNrfC8GIYdS0Yo726elgDECRYfaJuqP6mreyahxXYye2iM7LVLSDAEdo030RKX/BMEVs0Z7NOSVhCjMU3JBJs/Uv/9W+etpSSev2mkYpkvGACuM3zJxutHAL9ssudUacNEw5SuF+Danh8RU4Fn3NILQxmKH+hz97xBtJh9dnkAlAW/oPHtwPbeLlgRtEH4zxMHxTSQahqlH7AJbd9jI68muxCauKodJxin62SSf8cqDb8P/OVcfe6+7+LnFnol6RK7luu+LXqNJCWAotTB2W+5/532Edm15repOKs7v6W6IettVVdZDrOwhniztJmmh3fwwYBMjoJZtaYn+0dKk82KMZtNmwvzROHDJdh5pSooPK6JY0UndflMGe04rAhU+misgk4pLteXDsOrMaHd9fm0TSBC44viGqsoFRMSR6h/0NQjF7fOo1t6ywaEJsPdcEO9afygZeiOkT7m4BrutgYLJEIJwSJu+DfBaVKL+gQq0RJDkS2w8B/sExiZbNmYHRO+f5SgqNQBikVj1K4kZZRECfsK8uG3YHX5qujFw3vBqD3A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(346002)(136003)(39860400002)(366004)(6666004)(4326008)(86362001)(52116002)(186003)(478600001)(36756003)(5660300002)(7416002)(26005)(66946007)(66556008)(7696005)(2616005)(16526019)(8676002)(316002)(2906002)(66476007)(956004)(38100700001)(6486002)(7406005)(8936002)(44832011)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: bx2ZzxssyvqZcOPsXQVTmegxiQvA6zoABOf8cd7fOdZxZuKrI0VY79kAaDLwtxgU4ELsf7goIEIluaW0Yz1Mmt3gp0iPGBtC8yeKQXy2DtlCJuNO8gYD2kLGpS/6FE0RIyy4NZsOhjfN6Sj5jf6v/3vJNrJB3DHdVfsbBOxtYfmNg6Q87duupE6EG5Q1ppdpSiBdWbslTnN9Ou5atqyykzJVN/cYIOc+uLdJ0t5G97L/UaAmqUFXWKE8UgdFrI4rSoMB8KZZIxHdrh30Pq63QWjRH1LikGdDYun8Mz77cF81hjVV/z3dyD64R/Wi3zRTV5EOedHwmoPLvixpHAY2v40SqKWJX+Al9VnMod0aVZjVQOY+JFROU3HTmhY3fJUaLrxktdvbOJU7GFOIGgBcPHS8kJUIL7PJ9nmhWjUosVyG5HojTgFQvhIpb/xIyq/Ha10rFhEaLnTsUDdqQjgEi+cEYCgdl6l+pRzQjGNr99+L3Ha/w9/EpAvcQYR3kfA+axRNw3cU+iFiiMeL+7Vas7qJQavdb8PmN8ZH++q2FFShZm9kH8gkQWAXDCYp6LG3/eehVgkIeR4IwLggO1ApNrmMMe4uD1HGS1je/GDl6nkBK3cUNg1oiu61XInwmmhDf62E6HlrALB6olDSqi9yYij1p6nnS9d5pWSzFKURL2Zfmh/KkqM+q1Y1Gs0X8Hnm8HNa+zWlBnUlwjaiTax3SaZsHqvpIPQ4cRynXZ5bZHdKluEvK6ZVSP7vAv3/BAcWoXWGTHBvH3TXoQGwhaD+8ECr+bJFd3d0QXiPZT/ZkkEmvPMkkFZYKP5KlTBEC14//ySU3+j/7vMHgE176y0Yziw73I6LXDrFAOS97xcVpOPzrrelSjT3mJvBPeGXe/E8CLzjgSWExMf7KSNjoqIAV0VMj6MG1agmG0CBE4u+IOCfXgE8SkWNROAyhC8uoJJjEYH8xON3ZlvQ2lYiF4QobDkUrlMYqq5zZ63oKANN9AugD1IALZYmWXvI+CyfmfV3/F0XMQxrQfWookm6hpBB6g50IeFavHw3y/WPQrfs7DdCD4Yhh6sUpgQBSiyd6v9Ec1LlKM7oRlBwfJzD6Iwb0uCmM5uy5W94TFWaR3s5aVKvujpVssP/B0NpcXg+9UbpwoF+nl10DEV4RQ+xiSS7aeFp9givMepEohsitJJA0skWjTSY3G83qSs9DizUmog9YAEZhdcTtKSKxCu3xoqEbry4via5jaqefKHpvMTOdjSmC2sh4ZVnvE1IYBMYde70iKn66AXVE9vLpBCa0rHbvPslGI31yVmH28X2egD4ZIKh5UeJpLiMpIRhaUH3fE5E X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 59b955ca-22b9-4a44-a88e-08d8f3c29bca X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:23.7559 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eR8Uivv1NrltwMOKHX+QJeYYwqHh7bjDruvpFZqGU3nCkum3VPHNf9ZUEASNnW5JiPbbsw9DTlU3fTxuJCRonX0OYdBbKvBb1Sn2VueEBOI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: lVqEfo8D4jLu3BAYlthptuDqwnbws6vH X-Proofpoint-GUID: lVqEfo8D4jLu3BAYlthptuDqwnbws6vH X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add and remove pkram_link pages from a pkram_obj atomically to prepare for multithreading. Signed-off-by: Anthony Yznaga --- mm/pkram.c | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/mm/pkram.c b/mm/pkram.c index 08144c18d425..382ccf6f789f 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -535,33 +535,42 @@ static void pkram_truncate(void) static void pkram_add_link(struct pkram_link *link, struct pkram_data_stream *pds) { __u64 link_pfn = page_to_pfn(virt_to_page(link)); + __u64 *tail = pds->tail_link_pfnp; + __u64 tail_pfn; - if (!*pds->head_link_pfnp) { + do { + tail_pfn = *tail; + } while (cmpxchg64(tail, tail_pfn, link_pfn) != tail_pfn); + + if (!tail_pfn) { *pds->head_link_pfnp = link_pfn; - *pds->tail_link_pfnp = link_pfn; } else { - struct pkram_link *tail = pfn_to_kaddr(*pds->tail_link_pfnp); + struct pkram_link *prev_tail = pfn_to_kaddr(tail_pfn); - tail->link_pfn = link_pfn; - *pds->tail_link_pfnp = link_pfn; + prev_tail->link_pfn = link_pfn; } } static struct pkram_link *pkram_remove_link(struct pkram_data_stream *pds) { - struct pkram_link *link; + __u64 *head = pds->head_link_pfnp; + __u64 head_pfn = *head; - if (!*pds->head_link_pfnp) - return NULL; + while (head_pfn) { + struct pkram_link *link = pfn_to_kaddr(head_pfn); - link = pfn_to_kaddr(*pds->head_link_pfnp); - *pds->head_link_pfnp = link->link_pfn; - if (!*pds->head_link_pfnp) - *pds->tail_link_pfnp = 0; - else - link->link_pfn = 0; + if (cmpxchg64(head, head_pfn, link->link_pfn) == head_pfn) { + if (!*head) + *pds->tail_link_pfnp = 0; + else + link->link_pfn = 0; + return link; + } - return link; + head_pfn = *head; + } + + return NULL; } static struct pkram_link *pkram_new_link(struct pkram_data_stream *pds, gfp_t gfp_mask) From patchwork Tue Mar 30 21:36:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E65DC433EB for ; Tue, 30 Mar 2021 21:29:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3FCAB6196C for ; Tue, 30 Mar 2021 21:29:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233255AbhC3V2h (ORCPT ); Tue, 30 Mar 2021 17:28:37 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:33500 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233125AbhC3V2W (ORCPT ); Tue, 30 Mar 2021 17:28:22 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPOZr123006; Tue, 30 Mar 2021 21:27:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=O9v53orIHWVFA1t6PmlxKgAQW3yTqWCLxZiYmn97ZuU=; b=CGTjQF+Q/1/OBf5CjVdoeaL1RTUkR7Id+JWZAW56Xg30WsQVzJzmKsxnbZ/W9A5ZfXlI jILcTP9twgjc2n1nYtJjtw+cHeOxsWAxDextN0aLQOFCPPUv4sN992hMvF2cD+aOiMXb q7CY3Up8yg2Rm6pTuNtFwComwRShTh+zPhX8II8T/Fu4JRWLose/gM2cPZUoWjKDed3l HyU5A5CSDPgtULVxVYi732DboztuyumKOQg3Pd0O+FSlvVrkH1ZC/uvIrgXiX5uSPxMI 9VRMhOO3E5vXF43EpGAorEpm5Eyscuvfcq0hhQz0kOoHlH7zVMi4fn+mbZbrzdXBR8iB 8g== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 37mafv084p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:31 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOaCI149646; Tue, 30 Mar 2021 21:27:30 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1nam02lp2056.outbound.protection.outlook.com [104.47.36.56]) by userp3020.oracle.com with ESMTP id 37mac4kg35-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dfkdpaSjZM6RIzLgs3S46Hg5PPjaxtZiNfvlj6qbtd35PM8kBWubErF5dgpK07yvA8TGtoXSvSmP3ERjuPmiLa9oFgiJWqC0b+ZAShFvbg8rYFVxt63Ed6ITa/TKL+4Z+khUgdtqRpd/QjB+qzlBPJDw6l3unyJ4DGwVOKXrK2DJH1OQodT1bnzLQ/qQ4gLnfus0Y6pEHZOpf4u0cShhWs6ihiFs1Y4G7x2QFBzjUdFxTjWdA0Xq5XI3GXyRpQfAtElOfU9lnCDAztDdI8UZsTf5KJrAjInb2vLVTDVhYP+85KNG0GCf+lIB22zVu+D2l2bgnE9UapSMntSsf9KSqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O9v53orIHWVFA1t6PmlxKgAQW3yTqWCLxZiYmn97ZuU=; b=Qm/Lo12QMZqOn2vPeIG+P2zxwfZVgUzWCJr6o3g0DC7p+HMRE55//6jfa0m6pxiEHv+nHYEfp3xIFrwJAz9FNDc4ri6NeH4xqfUSp97oI6b0C0I6Ga7gmDONpRSxn6+cVS6jZkMR5EmAVMA3ub1VfO+l3A5FTWXGpo32rXkdtmftjqsvABxZVqqguOaHm3RRhsQDtEPO+yPOfoIJ3nF4a49+RTBX0hxD3JcLjx1+xQDtvya8yjxaoYCBHjzKR+ev9StyIutfO+gbZBSqp2T4VZPJrB49hjfZjGGT6DS8KNa5alNJKb9sO7F9mAksColJsynv3BYA2V4+pGVP2C4SRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O9v53orIHWVFA1t6PmlxKgAQW3yTqWCLxZiYmn97ZuU=; b=vY+8PIq7zd2XwB2RFhaezVEgmSWAkFUS9M5gWg4Ci5admvH3ERwYsujcdpuNNOdQgXq9GDY0ne3O6Jjdq1hbJytLcsNRll4ei7h9I9vHP1wWQDeXQ7muEClMXvo3DAbpXgFYZbKKFrCQNUY1hpBMQu/ue/wE9XNztSXHe2wuh9Y= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:28 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:28 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 34/43] shmem: PKRAM: multithread preserving and restoring shmem pages Date: Tue, 30 Mar 2021 14:36:09 -0700 Message-Id: <1617140178-8773-35-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:23 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e5208b1b-b954-451f-3045-08d8f3c29e57 X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2512; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Crl+KjBU0BGmpJaLt1bVcgmKAhrXorItGUvhvG4ToYPpqtL9OET9zvTfBeFeou+QzwsxBnvEBJLWpd9eUb3lpysWObfFxyQUKmKobEeCGMwz58ZWHM8pyKkLr7AcMOJZwZI4RBlWOhNKLvJc/+TX5fUGiE/U7GRZIvE3vvMcrmvLMSk1DoxInerxgvaXVnpuzF3GtoKylTNF8xBDWojNLTsPsH/YumtExKavVpVslxE+/1ZaI31gwC77KZuZiyc9aaKUJFhK+OK+szAjgN/BQsP2z3p9PNCS11MCfLQsaj5lyzyFUMNqejG8kg3iWtNKDC5Oc3JdE/phA1uZGfG6cNtw5WKPpq+8uzSNeGbYaAF9FNk+ZHWeBjlsal7R5SKi5M1LSdQrrOCzfzy4JNaTafsgjhy4kKWKP2O+VDp+KALc7sCSYy3RBfsadkgiE6yQq2oTARFUcA5f/O79tw0OXyUC/NuqiTc3DmWg71DQ8BNhUWEP29rHsSio+7Qecf6jdZZBf/Rv0EnREdjWbMflCa+0ejhSTYIzG1ivPt7b9+Jubd1CAsi1pBjGVuArGqrtFLwhCWZeBN4Zen5Nh0aRX1A2Dh1YDFl43HhL+aLrEjIbaRdE7El65EcEkQsgoxENZAJaKQw4++V74SptEwAEfg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(346002)(136003)(39860400002)(366004)(6666004)(4326008)(86362001)(52116002)(186003)(478600001)(36756003)(5660300002)(7416002)(26005)(66946007)(66556008)(7696005)(2616005)(16526019)(8676002)(316002)(2906002)(66476007)(956004)(38100700001)(6486002)(7406005)(8936002)(44832011)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: xfLrXmkQncPnVFrxt4kMdivBqHo7YnvxsNl3+n0mBq/gb1MYjMf5owjilYKRyi/OqSV84CarHL0d3LQRgRBdO37D3K8yl7tOaaUT5qfbQa6BhH6JPeTriRHDJxCdnl8kwXNVdlvHWH1mskb7IRgAvGALvRP2/8jMopWb4A9otGBCkLSYs/ob/wxsd1l1KV6s4TgOKP4AhXXVQ09yc1MQZkhB3fZtBalW098fHEIKbxXywHHik5nwJjg7sJZpN2HXF47BzXpcE7wgUhICPO5ify1yWeV8bVAJRMjdp745/R9utORovvN9Y07vA+sd7/SWE8DhG85aClsh2oeVzjfvThSNOpR1XFaUyA69rXpHPfw5c2ySHtf/Rt/kSoTBztMTOQLSOYnAX81WY2EuSS+VW1Yvi2ZYJLq5uWFvQymM9HKuSIGmT40vLSEeqMyreIPFvmwWYcB/tDKY7jw0nWLUQPZxMI1ccjKJ0qQq/Nv55uMdkewb7htLKuWxiZdKhW7vmyvNuJCwmykgYyxL45O0XCxSpEipMby/sZN2AuuLq5zQHrkDDqYAgdE5ZXU8m6RbxYU/87I8kSWvjzR4b7XVYaEXWgExjGuyfN1rau+t67bjeXL78HakjfIpZkaBfGkqPpxG/VgYE3pyzVNySubS70JxZp1YthgWhSIWWVCoqKSYuf+wpTejifS0Pktrni2Ejaxss6yED4CrLf4KNyt4Fjo0u9Eiv79VD7S0NdX67JJSS1aehlZKrHYNa6xNRJPGvQwFhfNUNeMkbeusu3Ipu4Qzt8GPJmsfsCKlMxehbS/N3H2s17l6cEPViDEPqrnjttC3z75+vSeXkUaI7UCAGyN2cbMFOjHZuYhh0i3IXjEJO7m4hAVmHPAk3gGhQ3ACLk+vtr3wj/XAwG/3qwsFioNcTubZljcYuFFJdhSqXyPe9qD4jRnvzj3NTkTiW2JchhlnSaH2aHpnJ1T6hV8IaAbrBHbRJSUKKB8CvIFm44IN90vw8zlFddQEaQRAknL30zD+sGhVDOsa7+QeWjskGFmw8acJGa6fMF3I27wG+25Rzq4XG6RUanf915xL+hXm1gqFckf1f9O9+XuhoDqpn4xjy0AzlpLimFQN9xwH2eLNs76zaMoNrV1MX9Eywqmj+k8ug/U/NojM+bbdwhTt6zCyhEgWDClq049NkYbSByT8eHLoKyN3avKBRXJveKb9X5tV8eppwkQz9W9TvHAj/BhDnTOUff8uCFLn5lqK3UlO3gastAJh8LGMW5F733MIHQ4dtn+3V+y21cFhn1adTP9HZ5fmyCRd9jMljwrJIG1sK9F1o1NeZYVqDBZVbO0E X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e5208b1b-b954-451f-3045-08d8f3c29e57 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:28.0123 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: F+Aneg7LPZmZkwsnhN9dZGCDtD6R/LgH2qFbKZCYKXfM7uIYPgCVEvFI98q1eTOmMWsqWbu9QUyGOs05tYgpmPrb3vEf5hhOqGS+uA/YnaI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: oDW_VUMVb_UCVg6X3iKbZPfCVZqVSO45 X-Proofpoint-GUID: oDW_VUMVb_UCVg6X3iKbZPfCVZqVSO45 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Improve performance by multithreading the work to preserve and restore shmem pages. When preserving pages each thread saves non-overlapping ranges of a file to a pkram_obj until all pages are preserved. When restoring pages each thread loads pages using a local pkram_access. Signed-off-by: Anthony Yznaga --- mm/shmem_pkram.c | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 89 insertions(+), 5 deletions(-) diff --git a/mm/shmem_pkram.c b/mm/shmem_pkram.c index e52722b3a709..354c2b58962c 100644 --- a/mm/shmem_pkram.c +++ b/mm/shmem_pkram.c @@ -115,6 +115,7 @@ static int save_file_content_range(struct pkram_access *pa, } struct shmem_pkram_arg { + int *error; struct pkram_stream *ps; struct address_space *mapping; struct mm_struct *mm; @@ -137,6 +138,16 @@ static int get_save_range(unsigned long max, atomic64_t *next, unsigned long *st return 0; } +/* Completion tracking for save_file_content_thr() threads */ +static atomic_t pkram_save_n_undone; +static DECLARE_COMPLETION(pkram_save_all_done_comp); + +static inline void pkram_save_report_one_done(void) +{ + if (atomic_dec_and_test(&pkram_save_n_undone)) + complete(&pkram_save_all_done_comp); +} + static int do_save_file_content(struct pkram_stream *ps, struct address_space *mapping, atomic64_t *next) @@ -160,11 +171,40 @@ static int do_save_file_content(struct pkram_stream *ps, return ret; } -static int save_file_content(struct pkram_stream *ps, struct address_space *mapping) +static int save_file_content_thr(void *data) { - struct shmem_pkram_arg arg = { ps, mapping, NULL, ATOMIC64_INIT(0) }; - - return do_save_file_content(arg.ps, arg.mapping, &arg.next); + struct shmem_pkram_arg *arg = data; + int ret; + + ret = do_save_file_content(arg->ps, arg->mapping, &arg->next); + if (ret && !*arg->error) + *arg->error = ret; + + pkram_save_report_one_done(); + return 0; +} + +static int shmem_pkram_max_threads = 16; + +static int save_file_content(struct pkram_stream *ps, struct address_space *mapping) + { + int err = 0; + struct shmem_pkram_arg arg = { &err, ps, mapping, NULL, ATOMIC64_INIT(0) }; + unsigned int thr, nr_threads; + + nr_threads = num_online_cpus() - 1; + nr_threads = clamp_val(shmem_pkram_max_threads, 1, nr_threads); + + if (nr_threads == 1) + return do_save_file_content(arg.ps, arg.mapping, &arg.next); + + atomic_set(&pkram_save_n_undone, nr_threads); + for (thr = 0; thr < nr_threads; thr++) + kthread_run(save_file_content_thr, &arg, "pkram_save%d", thr); + + wait_for_completion(&pkram_save_all_done_comp); + + return err; } static int save_file(struct dentry *dentry, struct pkram_stream *ps) @@ -275,7 +315,17 @@ int shmem_save_pkram(struct super_block *sb) return err; } -static int load_file_content(struct pkram_stream *ps, struct address_space *mapping, struct mm_struct *mm) +/* Completion tracking for load_file_content_thr() threads */ +static atomic_t pkram_load_n_undone; +static DECLARE_COMPLETION(pkram_load_all_done_comp); + +static inline void pkram_load_report_one_done(void) +{ + if (atomic_dec_and_test(&pkram_load_n_undone)) + complete(&pkram_load_all_done_comp); +} + +static int do_load_file_content(struct pkram_stream *ps, struct address_space *mapping, struct mm_struct *mm) { PKRAM_ACCESS(pa, ps, pages); unsigned long index; @@ -296,6 +346,40 @@ static int load_file_content(struct pkram_stream *ps, struct address_space *mapp return err; } +static int load_file_content_thr(void *data) +{ + struct shmem_pkram_arg *arg = data; + int ret; + + ret = do_load_file_content(arg->ps, arg->mapping, arg->mm); + if (ret && !*arg->error) + *arg->error = ret; + + pkram_load_report_one_done(); + return 0; +} + +static int load_file_content(struct pkram_stream *ps, struct address_space *mapping, struct mm_struct *mm) +{ + int err = 0; + struct shmem_pkram_arg arg = { &err, ps, mapping, mm }; + unsigned int thr, nr_threads; + + nr_threads = num_online_cpus() - 1; + nr_threads = clamp_val(shmem_pkram_max_threads, 1, nr_threads); + + if (nr_threads == 1) + return do_load_file_content(ps, mapping, mm); + + atomic_set(&pkram_load_n_undone, nr_threads); + for (thr = 0; thr < nr_threads; thr++) + kthread_run(load_file_content_thr, &arg, "pkram_load%d", thr); + + wait_for_completion(&pkram_load_all_done_comp); + + return err; +} + static int load_file(struct dentry *parent, struct pkram_stream *ps, char *buf, size_t bufsize) { From patchwork Tue Mar 30 21:36:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2B86C433C1 for ; Tue, 30 Mar 2021 21:29:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B810D619CB for ; Tue, 30 Mar 2021 21:29:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233353AbhC3V3A (ORCPT ); Tue, 30 Mar 2021 17:29:00 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52128 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233143AbhC3V2X (ORCPT ); Tue, 30 Mar 2021 17:28:23 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPew9145444; Tue, 30 Mar 2021 21:27:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=CHORIdLfCPoGUB2QPADa32PvM2G6NHIFWdfQ0TveQQU=; b=lSzAch0n0ovKymIV7ic0Vd6nZDPP1vU+Ez6xEGdDGaZDO9dpgkxerbbuXQgjGXqF/Fmi BS3bpVpZssTDiwuJ7F983Zas68PckA/khAAKFF5uKtm2u8oWe4OXp72U1zZL7WEwD3cS J3NKKbWDQexYRRblKmhGlZhdTc4EOH5lIo2+eIWwN+awf3lz4WY39la5wwDdSG4FZIB5 c/UWK/z1aoo3FcZOMmGXUlHdcRNc9WBFJ85pAvKbb4LamdYltQN1dKmT2B/bf6Rx99RK nt6gBX0uzvh2Idv1N0/G0k5m6vQMkowz6fXmE/rXLjVp7QSBINHx7Gx0PK3WEBvkoPQ7 JQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 37mad9r8j5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:35 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULObuU149800; Tue, 30 Mar 2021 21:27:35 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1nam02lp2055.outbound.protection.outlook.com [104.47.36.55]) by userp3020.oracle.com with ESMTP id 37mac4kg81-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LPohtHxgUcDHU88dSFhhLqp38HB8+3/ho925GgYBwWXTPCDV3f8T6e7kPJELJNJ5eztlc3gVcfsJhund/7jbpTbeiED8bosJJUDUQaH698UcdvDV1I93KlhJg5NAY4+q6eqdoqA0IGVLZjpTQl6o8gEU/I4t2dyW1J6/D/tlLVW7D+f8T/uUbKWSgD9ybZtS9rMWDtTTYvTYKpqffkbQQME08dDFDttwrefqkcPIz0KUZ92kJ99VQHKqjNjyUwIIrlS/kL85rIHlNxpL139Co5IU36/Xr7/1nNQ453raWQxVCoe8TgUs2Isds2vrRT4srUX4P8T/Y0WYLTS1bxCMFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CHORIdLfCPoGUB2QPADa32PvM2G6NHIFWdfQ0TveQQU=; b=DL9JmLA/AMiBUuCkPYkjU51818/8SbAvlPU3QfLwI5TGnAWLtWWWyiBtN73DD7uhfUj46plgjhN+lW0EK3eRul7vsE4jDh3SW6exTB1mYTD/6KtSZigz+OIeW4PokNxNpdofAu2oMETXlh8aUh+pyslkrRYkioEWFIFLTHvWOMIVThX+rMriao1DtfM+p4AJc/ucLMZYVIrnMpBwQGdq/FuynE0DlIMXM7FjBJLWCIuZxSzMmJw9VA7NmJlE9FlR4FKi/2UyphlgWfyGfvMPNHTZPRqYtlnaSeQu7S0aWFgO4BIUizPPfLFSfveEuRROnIz7ua0kIybT287QbHmbWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CHORIdLfCPoGUB2QPADa32PvM2G6NHIFWdfQ0TveQQU=; b=L0OkgYkcoqo5Dy5w7C6osyBLFgJDph3/IpLou1AHaiinzI/7QoYPiLWQoi9q1r1sY/MqFKWXfcRoLRwx2BXTmWGeO9HsnMYg2cFb7tuIu24AN7BuEUBaB1xyHybRd6Ew5zo7CbmqFVd99B+7t+aVbAHWYqAe7wRG9lY1FK5NlCQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:32 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:32 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 35/43] shmem: introduce shmem_insert_pages() Date: Tue, 30 Mar 2021 14:36:10 -0700 Message-Id: <1617140178-8773-36-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:28 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6ab9694c-bc9e-4cba-42ec-08d8f3c2a0e3 X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4303; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: y7UGSXgsaxIpkzunGXORvTr8msLifmhlziAVogyIUAlcIlB+b0+UZuIWWytkcKQQqBs0hdq+olErry6EmnDMFcFREVx0FeIiS6/qXM1WfOHIaEkpU2ERda4hroX8PtCZnHYb1u2JB7LJFS5E2a5hvkecHiNxmg//o7Lm49D9hTejVp7rYxeoQozuK286ljGAHctRWrsiVqjb4fpBLflwT/tzkSuQI6djCs5duuhxGxEAPh/Vy6h5OG1hWFW20KzNNd14ATc36JB4/6IX2XS72lMgzrBrK0tPkQit/8PvBa02jymjrc0yvzl5mzbcj5feCUK8vWoZrixgOmyaCBu1qE9+IoM3Py1NFxya1jmmW1bDuVNkl12GsphiWY0WQ6ec69dWfdp1xXYRSmAL96P2njR/6HjALdAhtOC+scJKehaFJfVCF4j9IS5ddEsO37bg5dtSKTeVkrYRZzlxlESnSKbtaAQ569vQf9LUlw/0Aooza0ZvoZi3090BBP6EMBFL2G+EsCaVEZxMVuPGLuUpB3NVlfT38hX+jW6YZi/DX1LSQc0sJYR7zGvcFjoh93EdB/x0Q0MGJRuVNz39FYCGwjIFkJdX0R6Ohylbu+5bpezoRpyuvjVz5mt8Zura+8IubCFAjPSXgOR8m2hklUIFJQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(346002)(136003)(39860400002)(366004)(6666004)(4326008)(86362001)(52116002)(186003)(478600001)(36756003)(5660300002)(7416002)(26005)(66946007)(66556008)(7696005)(2616005)(16526019)(8676002)(316002)(2906002)(66476007)(956004)(38100700001)(6486002)(7406005)(8936002)(44832011)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: aZ/K7nJa8xSiKyNs1QgNNg17qBVLr5UX9+D6qOxU5z5tyN2Xm3ln6+7KE1gDTlbO01TeNksMcmtx48I1ZfU1gTb0XSO6ycGqBhEVt9uksM3vK71VgT6zEpTnkjFEcxlCMIaEV9W87BVd0meUG+wmREj0gzO2EP+55KKQw5mKu3i4qO2PIqZgFpnm5dEGlZPkoH6JsZoiiqB5rAD3UHZ64RszMyPm43ZhNJUSXyjVtqASCkwGZzBeoomJZbSqC8Da/nvpR3eTiF4GfZ3nFw2GrrlsuLz9iDGppzwpqWnQG9UJ0ZAB5Rv5AGMYRJxlQ4+wV6ZokY8GG30JNln8VXJaNZfQtITiOfNiNsVzD6oz3FVg2VOumRTlT3mwOhO6DiUJoYiBVODVUWOh2pmmlVoQV60Ylx/9Gr4x49wKu7QfYOea0JTTQHQn4TxcCeynlJhtje2oehjAoFdiJRsACHGkC/HLfuGYlmPM/yUriOG/oz021Nk/mIpzRq2mLIgig7VO3CbtdfcozYvTo6L7EIeDG/nkIJWTQ3ykuaYvzZaN0+67H2OGYCJpNyCyiFkPRbIe8KZcHcDmQzdCZ3sq/Cd3UBCr674OWcXPgipr6hkofDyys+YAXElQmXWOnzA3zpeXk5uOgtXThWMeitS94uD4W338YjqS4hOTN3XPY1DtyTCirL38SH1eXz5VRVudcvd5HzbIaIRQYfcPg9zm6ZO4EGUVtHvVhqwwG9Owx9rpBfnfHrSxg0SDwfBHYz0kVmE2KvcqNo0YoxCjpbomn638CX5MbNzIdBSNCHYj/KVWR9nciQUKUJGR2xM9PLo/sMt3u6MGB3weAlBUfxxHwwiLWM4pRed008fUkc1g+Eo60+wo9EHf6P7ydhhHTHhKpqWqP5ya5vBqEtExSf5VXoIDjhEOqFIRD12AfdnvIxuNmmvQN+ewtwrvWf1fHO2QkdrdU7d5zrgAiudgkwz9imCnnBwppY7nEVdkUlMwiOh+o4oQ/fpEa21m4EBQAt0z+fSweXP5MiuFnEo+1Qb5vV/7VckbUpvORohU5a5WeyfGXuPa8kd8MXoKWl/rPfjdFZcOtHUaOJIjm07e1GbfdfeN/IyxgnzDPZfRlW/dHAujtyAGl4TLtNZ9sSQHsVqn6FnZH2/qxmsV++ed6RR29QPmyJsNyb6HgniVcFAEfatroVrKDTYKJzkAsPeN1UXMCsjQZo8+1mljxTr1t5DAwEhyVU2yXfW7xDXlR4DcZVCpvZqSfpbZ7h/NSeEbI0dLI74uGOtL17XkPsRjyQRyZEAPeaFGJ88bLFNu60gDJrP4J5dEB+lJuLM5e/6kKTMxuGfm X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6ab9694c-bc9e-4cba-42ec-08d8f3c2a0e3 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:32.3057 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IicIbPPRW4J+EmIrw5PGQGBzx/rKTQg0CsKyP0IRP7fho1gi4KorWwcI2iQqtxFiijw5LJs5MOguVHmOThc4+wfqRSiD9mW1bXaaJBcWCoQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: WphkaB2C82bsjcrfB4h-jP-eI1MzZSTG X-Proofpoint-GUID: WphkaB2C82bsjcrfB4h-jP-eI1MzZSTG X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Calling shmem_insert_page() to insert one page at a time does not scale well when multiple threads are inserting pages into the same shmem segment. This is primarily due to the locking needed when adding to the pagecache and LRU but also due to contention on the shmem_inode_info lock. To address the shmem_inode_info lock and prepare for future optimizations, introduce shmem_insert_pages() which allows a caller to pass an array of pages to be inserted into a shmem segment. Signed-off-by: Anthony Yznaga --- include/linux/shmem_fs.h | 3 +- mm/shmem.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+), 1 deletion(-) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index 78149d702a62..bc116c4fe145 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -112,7 +112,8 @@ extern int shmem_getpage(struct inode *inode, pgoff_t index, extern int shmem_insert_page(struct mm_struct *mm, struct inode *inode, pgoff_t index, struct page *page); - +extern int shmem_insert_pages(struct mm_struct *mm, struct inode *inode, + pgoff_t index, struct page *pages[], int npages); #ifdef CONFIG_PKRAM extern int shmem_parse_pkram(const char *str, struct shmem_pkram_info **pkram); extern void shmem_show_pkram(struct seq_file *seq, struct shmem_pkram_info *pkram, diff --git a/mm/shmem.c b/mm/shmem.c index 44cc158ab34d..c3fa72061d8a 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -838,6 +838,99 @@ int shmem_insert_page(struct mm_struct *mm, struct inode *inode, pgoff_t index, return err; } +int shmem_insert_pages(struct mm_struct *charge_mm, struct inode *inode, + pgoff_t index, struct page *pages[], int npages) +{ + struct address_space *mapping = inode->i_mapping; + struct shmem_inode_info *info = SHMEM_I(inode); + struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb); + gfp_t gfp = mapping_gfp_mask(mapping); + int i, err; + int nr = 0; + + for (i = 0; i < npages; i++) + nr += thp_nr_pages(pages[i]); + + if (index + nr - 1 > (MAX_LFS_FILESIZE >> PAGE_SHIFT)) + return -EFBIG; + +retry: + err = 0; + if (!shmem_inode_acct_block(inode, nr)) + err = -ENOSPC; + if (err) { + int retry = 5; + + /* + * Try to reclaim some space by splitting a huge page + * beyond i_size on the filesystem. + */ + while (retry--) { + int ret; + + ret = shmem_unused_huge_shrink(sbinfo, NULL, 1); + if (ret == SHRINK_STOP) + break; + if (ret) + goto retry; + } + goto failed; + } + + for (i = 0; i < npages; i++) { + if (!PageLRU(pages[i])) { + __SetPageLocked(pages[i]); + __SetPageSwapBacked(pages[i]); + } else { + lock_page(pages[i]); + } + + __SetPageReferenced(pages[i]); + } + + for (i = 0; i < npages; i++) { + bool ischarged = page_memcg(pages[i]) ? true : false; + + err = shmem_add_to_page_cache(pages[i], mapping, index, + NULL, gfp & GFP_RECLAIM_MASK, + charge_mm, ischarged); + if (err) + goto out_release; + + index += thp_nr_pages(pages[i]); + } + + spin_lock(&info->lock); + info->alloced += nr; + inode->i_blocks += BLOCKS_PER_PAGE * nr; + shmem_recalc_inode(inode); + spin_unlock(&info->lock); + + for (i = 0; i < npages; i++) { + if (!PageLRU(pages[i])) + lru_cache_add(pages[i]); + + flush_dcache_page(pages[i]); + SetPageUptodate(pages[i]); + set_page_dirty(pages[i]); + + unlock_page(pages[i]); + } + + return 0; + +out_release: + while (--i >= 0) + delete_from_page_cache(pages[i]); + + for (i = 0; i < npages; i++) + unlock_page(pages[i]); + + shmem_inode_unacct_blocks(inode, nr); +failed: + return err; +} + /* * Remove swap entry from page cache, free the swap and its page cache. */ From patchwork Tue Mar 30 21:36:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E273DC433DB for ; Tue, 30 Mar 2021 21:30:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C16E6619CB for ; Tue, 30 Mar 2021 21:30:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233392AbhC3V3h (ORCPT ); Tue, 30 Mar 2021 17:29:37 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52270 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233283AbhC3V2l (ORCPT ); Tue, 30 Mar 2021 17:28:41 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULP4EC145279; Tue, 30 Mar 2021 21:27:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=GZb4CKpxRwLNdd15u2UORaRNMHAWzMb1KyTaA8tHhy4=; b=mqRWvebD0I6wpyEJqjoYSO8ruKRCQ2WYAgSR7o2rieUPFL73AlwFyiDhp+Nsxokp/Imh 3ncEXPYbHcwOEDa6ZxM6IoocCjFw7GEQmCupqbf1xG7Ts+L7Xpj/++kRQ5KDYM8TCk7c 5BzUa/PyACJGKlUxAASVTzHrOX3nM97c11OWl9J7QRTv73MeVL34bWOIcndQvpxJ4Xai i3c8GfmK1bEIV+A7Ex/DNmb+fvkZEbzSJ6ra00Mjlv0SA3zyt+noK95y8xoB98TU+y8D MpQRWK9BUamAmjF1aI+vNp/FQ7f4FiKVHK/Ic457Kh0liVFRyyuFWGvgD4wZtRrxvJPu vQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 37mad9r8j8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:39 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPRHc105743; Tue, 30 Mar 2021 21:27:39 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2042.outbound.protection.outlook.com [104.47.66.42]) by userp3030.oracle.com with ESMTP id 37mabkbd9u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cpI7jMc0Uc59Tx4VVJWOS2ZbVjg+g6Bei4PNGcCGVu1f60bEPUiIy9pQnk/QCq6OmrM9o8/uY4d+D0Mqx29mGsTXS3e+mpwi8fEroLtqn+1DYlBMMlSRRTiRutfmiA2sMATCAacJgBuGIhBZqBzTxKFUQPsanblezIsvTkRSm+ke21WpKLDw2mZavGjrSYjf6jvpI9+7m3cAapcy+aA54C+RtLf70N+aCdMG0mBk5ZODiURD8JiaSciPIxzrOh6kdNvNVrpv5hH6Re41oOz5feWiXSjk5TX5URnD097KxZvnnxhNmUR+lpVx6Jwhrv3umsZsRCME3XiSVMIm7+QaJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GZb4CKpxRwLNdd15u2UORaRNMHAWzMb1KyTaA8tHhy4=; b=IIWNU6qCAlygsm08rB+7/Ed+wSdU8SaGDu0nAfL2YEQNtCl4Su0qfoNc2ebopyhguXHBMF+zaz0K8BY9Z0Wx6T13JXnRp6XmvwMqQAxx3gNG/fZ4Lygmid8D1ZExaWKI2KwhuxiOMcrnwyYsKEnN8qYmGk1TZjAecFVL+rq+Cr0lis1PeLD8v0QRR+6ldCxklJT2GceMVTA6VjrG9XJhxgzkJC/hbuz53iiuiDVY4opV5j2yyWgfMvpOxmw8snAvZ8rpXlkDornU5tCEvXZ3sCMPabp7l9jvo/3CfoliYRR5DoBAO4XPFvM5cHDERMJZBQwrWK6+bgIcSfWoOxa5jA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GZb4CKpxRwLNdd15u2UORaRNMHAWzMb1KyTaA8tHhy4=; b=KVzuITmvTMKqMWq2Lo+tOy0nMXa2xnpi41r3QpF0KuC2KazTOpaq+v1JfCTX53FH1PkZylSkDeqkKoTkJ26MMNz3oZy09Pf7M6OBCuXXVkzRCE88PJhAaPg+yJE7qgoYf29jVeYKP6iiqI2R+VqA8A7ioRwgqdycdOtQzJ57EKE= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by BLAPR10MB5265.namprd10.prod.outlook.com (2603:10b6:208:325::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:36 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:36 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 36/43] PKRAM: add support for loading pages in bulk Date: Tue, 30 Mar 2021 14:36:11 -0700 Message-Id: <1617140178-8773-37-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:32 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 24a46131-2667-4eca-bd58-08d8f3c2a375 X-MS-TrafficTypeDiagnostic: BLAPR10MB5265: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ze00x5J0RiFSrqhD7dupwIb+f6ZKYyOFV3N+LvLIFVqoCG1dITM/qHKqfghXQhMPt2/79wPVTt+lO8/Ynb8OzkFMRLyDu/gsgPXhZn0pjTqik8J1WPHbIV8bnArwxDmdMPXQxSYhWpsuPY7wRp1+n02JgF4JTy/stdbxFAM++zsGOxyacPB7ceI2Gd5DX9fP8LfmeHwJSCLw8u/vpQ5LibS6E9bv6PzZ47O/+8i5RWLHuQPO1o5Plhz+uj1lQCAfR4cENh9T6On9zI4DHXbhH4c44s2DBAum36WW9LZ5NlY56QFxRvMlvxRA6AQVcEsBCF3rk5TRCcNi1eyNMBqKBOA3qQpUH/CYpe6EM3yIY1DwwtMI4iUey7Hznq4K4t6fFeCAlEyai92iknWRG046ZuaUVklZsNkIfTBRUWCcXXaaXTJRYieopK7IOi6ku2YcoGc4adyNaWKfzAahM8uHv2oBDWtgAml5M3xMu3xOIXR2Q/uUvjt9SjMwEHPRNj9dqZMTOsJNJYtV3a0TuzlBVsOxfY2rCcSToTdZkeEAm/k14zMZsL86rN6Ta598k6GTCPAnr5N2TCVJt8sfauwtyRWy2up0wXNQ00uuOgtRWNBIsflyazJ1z2Shh26JgVNeAXN/t0In/58pGJhVDM3nqg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(346002)(376002)(39860400002)(26005)(956004)(52116002)(186003)(38100700001)(36756003)(4326008)(8676002)(44832011)(2906002)(316002)(66476007)(2616005)(7696005)(478600001)(8936002)(86362001)(6666004)(83380400001)(7416002)(66946007)(5660300002)(16526019)(6486002)(7406005)(66556008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: KVjdA1n0AV1WT4Gc9zYDX932WmIDW4OxC/dMk1b/ZEo+yW8GNbJDQvCV6tMLDtUew+84kmjj5HhojABgL6+TDMD7YJAuW3n//FUkmuJLmZW5u2N/7o+NPm75GeAa+CpcebQ2cdbQD199rfWBWlnwiQaATF4SuatK5wYwPRxTSD/vrcqUoD9ZIFjp6Nuf93nLuCswGwSPrbB7OflpfaFX982uOV20KyWViUT4YnA+U8P79Ve4BMuka/qm/pKbKaEhr7JwsNeiJvRA6asuosOeK+UoZW0KIIdnK6adCVjjozjid6q5xcCkG2EFsaebyUk8tSoT8Dk0xXFcEb/YHnPZcA4NY8KVPb2ZGjCzh7Tdha0jYkgnDt6DNOQrHodneKPmNA/kIb2HJ2JdQbW6mBFzOJYsttuI6T5ycoi11CQo7M5vuNPRSpFdEvSwqmA0kuk1qsGircgPfSGx1ohTvD3CWgrPdYwkCZKVjw8fu1O41DWQh62ncYcdprsSBDhYj9Gxgwu+KVqUiSIVle4TKJ6ob4KvkO8S9crINQS7rXkUK3IjIUI2Wtn7CU2NsLOAud7wiWKFx/usrU8vDFcyM/AeS/e45s4nmAwAuWmv2k00frGZPNksMEtZEgouHGhio2ZFxzhrMnoZCLfJBZmGe4JZN3vej2zQRg38Gu9R4G9h+D7SxStNZFaUPD7rpLSOf97r2WmwhBPHD1VIFJAkp7m1ux5PiLKqNTgT50KaxS0Mv2jiPRMEYVK/wxaHY8rhQw1MYTqjSoE7qTwLEv2jRurgEOFZPfyugayEFHqZjbo3Yj12JX3d9+rTLB9D/9dffGT+TY14JRijEl65CiYpmAXmJr7M0nW6xyJZPNY4sIQsza8pleSSuJqrPfjlVshPHeDpvuAnAlBEhVPyp8hpCnWt2ZHvl61WSbkf+XzG+snAWxLq0jiyJ8doMPohYllsfbk8E2+WPX12txBCTt57216AD6NBXRsWlauux5fuCcfL2Hg+aQOKht/ClaQ7v4xTKA9LHiP1FUQgmcS70yP2sRnytA175oSK8pm4vRTdkxFxnzzI4uXrkZ/sHMqb38fcyi9f0d3Yikp1d69ONwJs7rRkbWnV9SH5lJbUlS7VOxXdswwezJGvgRVsrVYQMNHcGgdjJrTcm2X4JFcYMn2sk5h4dbQKk4nb/jrkoiIvp+UagRkfxSrr945MLY6rKAIHUaS5GMSEEUpHVpQegmX15nvR8fUik4Jno/r1c4yeSbkF9WUWw4gHqw/GDdUGA+OLrgoDR7mhkrvv9K8I45ZaQ3qSS9jCfcnMzXdvjsJUS8cOVU/SC49yk4Y14ziK3Hxzuc3P X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 24a46131-2667-4eca-bd58-08d8f3c2a375 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:36.6421 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lx6J6g8oVLew/mQ8rJaDTzvnXWq/48/0rAOX9+LEMTljwfQjPw5etuyKfCHmYIAB0PMLZzJjvJnqf5wjVmT5chP31MsYhY5/736jHrAzTAE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5265 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: N85R9sVWvAIJVK2_WzLDZcyc4B5qvzPL X-Proofpoint-GUID: N85R9sVWvAIJVK2_WzLDZcyc4B5qvzPL X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Implement a new API function, pkram_load_file_pages(), to support loading pages in bulk. A caller provided buffer not smaller than PKRAM_PAGES_BUFSIZE is populated with pages pointers that are contiguous by their original mapping index values. The number of pages in the buffer and the mapping index of the first page are provided to the caller. Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 4 ++++ mm/pkram.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index 977cf45a1bcf..ca46e5eafe71 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -96,6 +96,10 @@ int pkram_prepare_save(struct pkram_stream *ps, const char *name, int pkram_save_file_page(struct pkram_access *pa, struct page *page); struct page *pkram_load_file_page(struct pkram_access *pa, unsigned long *index); +#define PKRAM_PAGES_BUFSIZE PAGE_SIZE + +int pkram_load_file_pages(struct pkram_access *pa, struct page *pages[], unsigned int *nr_pages, unsigned long *index); + ssize_t pkram_write(struct pkram_access *pa, const void *buf, size_t count); size_t pkram_read(struct pkram_access *pa, void *buf, size_t count); diff --git a/mm/pkram.c b/mm/pkram.c index 382ccf6f789f..b63b2a3958e7 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -1099,6 +1099,52 @@ struct page *pkram_load_file_page(struct pkram_access *pa, unsigned long *index) } /** + * Load pages from the preserved memory node and object associated with + * pkram stream access @pa. The stream must have been initialized with + * pkram_prepare_load() and pkram_prepare_load_obj() and access initialized + * with PKRAM_ACCESS(). + * The page entries of a single pkram_link are processed, and @pages is + * populated with the page pointers. @nr_pages is set to the number of + * pages, and @index is set to the mapping index of the first page. + * + * Returns 0 if one or more pages are loaded or -ENODATA if there are no + * pages to load. + * + * The pages loaded have an incremented refcount either because the page + * was initialized with a refcount of 1 at boot or because the page was + * subsequently preserved which increased the refcount. + */ +int pkram_load_file_pages(struct pkram_access *pa, struct page *pages[], unsigned int *nr_pages, unsigned long *index) +{ + struct pkram_data_stream *pds = &pa->pds; + struct pkram_link *link; + int nr_entries = 0; + int i, ret; + + ret = pkram_next_link(pds, &link); + if (ret) + return ret; + + for (i = 0; i < PKRAM_LINK_ENTRIES_MAX; i++) { + unsigned long p = link->entry[i]; + + if (!p) + break; + + pages[i] = __pkram_prep_load_page(p); + nr_entries++; + } + + *nr_pages = nr_entries; + *index = link->index; + + pkram_free_page(link); + pds->link = NULL; + + return 0; +} + +/** * Copy @count bytes from @buf to the preserved memory node and object * associated with pkram stream access @pa. The stream must have been * initialized with pkram_prepare_save() and pkram_prepare_save_obj() From patchwork Tue Mar 30 21:36:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B229C433E4 for ; Tue, 30 Mar 2021 21:29:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EEE79619CB for ; Tue, 30 Mar 2021 21:29:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233173AbhC3V3B (ORCPT ); Tue, 30 Mar 2021 17:29:01 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:33516 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233166AbhC3V2Z (ORCPT ); Tue, 30 Mar 2021 17:28:25 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULObGB122825; Tue, 30 Mar 2021 21:27:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=v2KigxXcZE6xlJHQ8aE8ca0rwLVEP3O2wfEC7HbsMsc=; b=AkK8TuFiPJv0jHSk4n2XebcD44o3b65WbuJMMXQG1+tPElHSzlED3i+1IiaY/VDtlENu ddACSfleBxr2XJJsoSAhn36QVSHa1CltgoSeekLA0TgaShzOO0ZNHpwalGvMsgNKlSSs NkUPFW0OL8HtQ5Fs0Pfpb9I40WQ5Xj1vgSZrhMvCrIUVN5lsSUukY43hhosFM0X+H1kc +T+yOYecqzawllcpgfsm+XeTuFT+YfsD/Rszf/5lGu6SZ5WUaFfOp0sjzdhhA8GDIahm qdsag+XgDF8DUZ9krm1kQRH4OxPjZg/YcPu4qZb3HUvzxYvZ76oXsHohofcBz5w8bfXv yw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 37mafv084y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:44 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOZw2149611; Tue, 30 Mar 2021 21:27:43 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2047.outbound.protection.outlook.com [104.47.66.47]) by userp3020.oracle.com with ESMTP id 37mac4kgfr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Lx72bby8grU4JYVJckmq7gEECSGcbZMjIT01rs/10+otuUetNhI3dCHeLUGlfRdcUk++Dph1GAH4mypzfQwt2PO5llYOeMSCr7CPYG8F4mXt8zCnGrNl1d8QTsA/hoPSsvm8MroLzyczkY3M6XX0zJh2BdcDkFVLh2r1LJIUYOHhsnJdW8xiM8gAF6Usvu1U0gmIDcTHWACtsg0SRudxldl55bt9Xdc0Vmtodc6gBU/dfwc6Hmrw4XDNdKxfFoD4HJkiBIboSa8D+wxHRCs5a/ZT38ba0bKjyOUtFwTWdCwYZTIpnwSwbR8kF+zHhyNGqr2g0zmqSXmyNd9xPNZa5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v2KigxXcZE6xlJHQ8aE8ca0rwLVEP3O2wfEC7HbsMsc=; b=bar5iiuoypRROsFMFUNzzsTUdEmgaecKkJzvcGexMSyAjSMKZVgeYmtt2+Ux9FjkbP9wDDXn0KEwihynPl2BekCHFgF0Z8rdOEotgwe6tHT9kpIdEMKa6I1fJ7sUZ7bNpinc3JISTaqtcN6YOMOODLv5+k/ujfJSi1tgH8ISv9ZABwCF8cHenL7/0sMbD9PpI3znAFi6RKy04Bm2JUm4oJhRhYYTt4b2Gx3pLxZs6LSSyjrnYD5QkOuqxFS2Q11lZk1GgzD/pkiPK0CIlhgf7A9eS0GEJPTO8Yt1XongpZJuDYT6DFoM62iqWY2TC4mXuJI3Eht93Ot5zY+kjTeHHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v2KigxXcZE6xlJHQ8aE8ca0rwLVEP3O2wfEC7HbsMsc=; b=G/NSXdIBzxKG7dyPVZDlBpKV3+7TD2osDYN2fqQFo35JL+spDTFncJ+vrfj+JhYZAFGMe2J04obUXwJWL/+Aa3uAOkDHFADrCqFUjjO99FYsSEt0ucsfMYkuVHmQtWOYOQ6MN68RGdnJZpvjBuOj5JiITUG8i0tKZviWHq7ljhQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by BLAPR10MB5265.namprd10.prod.outlook.com (2603:10b6:208:325::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:41 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:41 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 37/43] shmem: PKRAM: enable bulk loading of preserved pages into shmem Date: Tue, 30 Mar 2021 14:36:12 -0700 Message-Id: <1617140178-8773-38-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:36 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4d5dfe61-72fe-468d-335c-08d8f3c2a60a X-MS-TrafficTypeDiagnostic: BLAPR10MB5265: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1122; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oaqzA23g1sHDTw1RWvfLsh93CZni+lpTjwCS2rW4KXqKfVJ40qwWgrgBXejrbj/ab16qaeuOzDbGfI0KlywnmbKvA5FmqQniUgFLtfn4yWqwMtXlGSOyfp74R4Ty/LPmEhsXSQ5ZVvKhFRExBHLnLbfGLTdfOZQrgXktOZg/VKy5dpyNHZGfexli1ENo23oivUNrgiPRJiK7JKSKM45ugbAaAPDUWwSEoyYi9i3Hj9L95F4NUzoN1YM3Z7gLuyNlR5Kz4+uvaOMyxq02Xt1HO4f/h7crHIpwy4FTyVzi2zYeTNCblMOR9WPSoMn7NE9wpDsYV57PEuBfLOY4Wrfo7RbdIIdiXFPlPjZ8Bqg3zS2IAY0XY6tGZucgqX9AJyFWe+n0VWqJGpJonslLCA7bIgx/naP0h+92xCcMRpRVIwocyEh7v7BGcSDQz0Lq16Hvp1obJl8TERnSfr0e1eM8Jczt6DwzLeR+wh7sC+JY/kktxuXCfK/CEB6UofCszjL6IBjeFu7Ft3KaRwTZOoXAcIP3pc/qsFjUvtdgqCZ2pZ7+WyWMblWDFwZLLEyHZDglKIuwLJLwb/CiW4wrDD2dMT7VUUDEQaEWzLoqBJJxLqE2Hz1DBrH3rHsPlUa2pXZvUKLOg+9lJGe0brOro6/NdA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(346002)(376002)(39860400002)(26005)(956004)(52116002)(186003)(38100700001)(36756003)(4326008)(8676002)(44832011)(2906002)(316002)(66476007)(2616005)(7696005)(478600001)(8936002)(86362001)(6666004)(83380400001)(7416002)(66946007)(5660300002)(16526019)(6486002)(7406005)(66556008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: N9EVpIVg77lhR38oyow4xGf926FI59UZgXkx2TyPq48ldarLcQ8JRZCasn64lx1czalPGjTayajc3Lo1MDyS+WeHNndq/MKCP03/fxsjPOi3Y+PMboc8RY+3jguZHGdkN88rhtooy9vnAncc5xxLjj/wBwEGYZ7jCtqrVYGrVVU8dgll59DkdG+myxO9h+olAttJ70s1B2m+WuxfYzReTZGZxQDGdp2a97wQ20Ma1xJhHRFGhNj/tMmCv9wmToP4LwvIUIa0eg/bZcjnqoVoW/18i8JqchEy2cS6+YpNbWFPAQL/OlekdUzTAhxj+GL8VXDiYMakVQX55KGgP1iRPg5o7OHLLvx5WGkJZRiz2FFQqJ4xV8nCaEKuex8c1PYAMYFRuwsPA/rhVlw5FEvwOey3BUBxaRadXl4JQ39NJDp45YQOcLHRhkTk3gadJQRgaECFG+xVte6PUIiltfBA+3YFmIELOu2+lOZGF4d+NhMZEEZGhTZf62KugFOwqNe2+g2uVrPIaWCSsKjA4NB+AGBOF+G6oIB9O71jwdUASxcbeCBYQpaZwJJv3+t4Fc37qWx+yUy9939US8ctCcclErD4W1SqbbmNRs72vNrS5EuBsa5GBrq7zkn0AnsFgB0BIpgLm0/3fTHcxYSGTVdyCItoJ56oZqJz/vjj55rInAlhB5xY8dafm55ydWmNmhD/WL8fX3jOLi1BxfAGyAXweYi/OiXfn5ff/YxXZvCBs9UcjTpIW8A4/3Yy8RX4AG+L+EFL07He1tpRAsh83UBlM4khxCQT4sI06LSM087rm0uqLvQO43eLEz/gS/kXwOc1lnVJ5dOIPvUoHcTbCSuhgC/Q9XyGji/eh1o065UKxKL8AirtiSmcrGFDTRuNO1fD/uKhufDv+d+a+UmWfC6799/bfh74H6SLNgb4CqwIlJHsOd3agHLHjPYC52eMvVXCjBPBX6oed6INNYKA1tcJviMsGERFZUp4hitVpzxO0yNZ//UYzlOUSMZbMWUhdL8p3340PGK/UQfdR5Sz8Lbr0L4/VZq4mKVehGK9yf/x54U4Q/qUqImc8NftYJNJeZZSeAQP+bDhiQ8rNXSNY4zxNVa3TrreAO9MzOEl0JWPH2i2WVGMVBfnAvC+/T32aT3XDow9vTo47GTCw6jgxfpbBxBr6l5MA8rdY/G1h1iCQGvmtvOcFXc11zV60iMd7Qh61d1G6p1dTq0IWrpnIgEDKMUIh+R9J9k+ATjQt6K0rtXViW5Ihht+vxculzxRp/bN9/K/H77CYsFd+edITunBujbAIt3YQtMf//p51fqw1P8rrO6AAvBDp7XuGS58RRiO X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d5dfe61-72fe-468d-335c-08d8f3c2a60a X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:41.0285 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: EDSMHUQvsM10K6U275XDFiCedDIDxbBBgLGuae5kD/YTS+COD3xO/tRNuBOxtlL3QRXOtb0ka2c93i/akR/EOt8PArGWdbpzP/iaWjUAcgc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5265 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: def8N68OmN_oEfauHp9oC2IQSk7O-f2M X-Proofpoint-GUID: def8N68OmN_oEfauHp9oC2IQSk7O-f2M X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Make use of new interfaces for loading and inserting preserved pages into a shmem file in bulk. Signed-off-by: Anthony Yznaga --- mm/shmem_pkram.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/mm/shmem_pkram.c b/mm/shmem_pkram.c index 354c2b58962c..24a1ebb4af59 100644 --- a/mm/shmem_pkram.c +++ b/mm/shmem_pkram.c @@ -328,20 +328,31 @@ static inline void pkram_load_report_one_done(void) static int do_load_file_content(struct pkram_stream *ps, struct address_space *mapping, struct mm_struct *mm) { PKRAM_ACCESS(pa, ps, pages); + struct page **pages; + unsigned int nr_pages; unsigned long index; - struct page *page; - int err = 0; + int i, err; + + pages = kzalloc(PKRAM_PAGES_BUFSIZE, GFP_KERNEL); + if (!pages) + return -ENOMEM; do { - page = pkram_load_file_page(&pa, &index); - if (!page) + err = pkram_load_file_pages(&pa, pages, &nr_pages, &index); + if (err) { + if (err == -ENODATA) + err = 0; break; + } + + err = shmem_insert_pages(mm, mapping->host, index, pages, nr_pages); - err = shmem_insert_page(mm, mapping->host, index, page); - put_page(page); + for (i = 0; i < nr_pages; i++) + put_page(pages[i]); cond_resched(); } while (!err); + kfree(pages); pkram_finish_access(&pa, err == 0); return err; } From patchwork Tue Mar 30 21:36:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1EA7C433C1 for ; Tue, 30 Mar 2021 21:30:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9D134619C7 for ; Tue, 30 Mar 2021 21:30:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233259AbhC3V3g (ORCPT ); Tue, 30 Mar 2021 17:29:36 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43838 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233273AbhC3V2j (ORCPT ); Tue, 30 Mar 2021 17:28:39 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULQVT2012650; Tue, 30 Mar 2021 21:27:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=gS3vG8u54ctfaUP8I95zzdfhlydj+cI0QRKyrDCE0ws=; b=l9/qd24WwoIYMhviHj746zW45j6qpTni5G2PFlQljJsAJQMHI267hETuOWCshulFzamF lGiJghFbqjix/A1hs6gB2QSqIlpQ45omznFDBU4E4qsI+1UjAhlLoa2btoSxG+n720Am zarrDxsb8kU4HyaD+hlSlYMLLpiSz0AvOkpYXDmMYK1/caiWCdDw7sgLzJobrMhe7tQE v9aOxUWV/TLZa/Yag+R4L2q83nANkQFf4FLKV+SNobY4pEnu56UfBo+2QNsNyZphbbU/ pjvIHdeC3tr1JS6T+LaVelatrCwnYV5ArY6ZRfm2P83zuu5aB0MhpSAaXbS8+toQaNlW BA== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 37mab3g90v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:50 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPZ71125105; Tue, 30 Mar 2021 21:27:49 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2048.outbound.protection.outlook.com [104.47.66.48]) by aserp3030.oracle.com with ESMTP id 37mabnk883-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:49 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VAd6h5Z8farIW7kDvlo1a7/fvIoPKXulEQNKGhoAPoeylt/6D8ShfZu5P1bSd/VRIq+vzacPxxdIsyKt3w2E1X6n6WhkNMe1RjdIdtimtuus6cjVRdA+VcHeGXDv0w0PebQpiwt6cVocNi5DrvL/SWda+EILLVrhEM8dpRHkExFGZI9dluICPHLk0pUOIC/yqfUi42czPW87w6Nf8BD4iFn7oCY2ezpoiHFdyo5TPsKaadwz8mK/PTA29v6zXequhdHfIBE2b1i5Mz6Iyoo62S38+xUgdBS5iv4V0eB18d3TR7RP16YA4kQErdPXeP+KIB8lzKadq9ln2DI0Wt7vSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gS3vG8u54ctfaUP8I95zzdfhlydj+cI0QRKyrDCE0ws=; b=W3gBoiyDJsgDaLBnjxqiPaTLiYr8rCCs0iipNoez3XDnBSJyTGc7r6JEdCIUdJR1iRiS8kBnYQu10joa3d58k2icJ7zT/kNVBM0qlTQ/zS94lcgpu2ynNr8+YlEKfzaqaV5dafMmxr4f3QQEXjsiWfK9RSeX3GK456ZVWMxrJDJbjmv+Pp0cOUSH1V3LmHbm8iX2aS4Rcdzq8SEddk65KsU2N/Hw43t3wxlqI47oDYRgR6/83amvMxlRckrC5qXAkuxdGoOccSdfMaUuvCc8+gBTN+vhnskyHpOZmU1pvnemudJblWLdtI6DA7/ulLaVevv1CyOlGfk5HHksdcVylg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gS3vG8u54ctfaUP8I95zzdfhlydj+cI0QRKyrDCE0ws=; b=n2zYWj00cHRvDyzUsHQTGNpAKlMYE6MzjWqfxjY3AYSTnb+o2vk05eDG3PNZUnjiDITlrkCu6MJ1Q04rQXAkB0in1/nbsRxe2YI+JeNsE8UbnIb5rZ71uwvJ53g4qAA+FhP68KMBgpNY6sbrgjqa+JL+yprbXuDEe6XPWj/IoOo= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by BLAPR10MB5265.namprd10.prod.outlook.com (2603:10b6:208:325::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:45 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:45 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 38/43] mm: implement splicing a list of pages to the LRU Date: Tue, 30 Mar 2021 14:36:13 -0700 Message-Id: <1617140178-8773-39-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:41 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9026485b-54f8-49ef-2632-08d8f3c2a8ad X-MS-TrafficTypeDiagnostic: BLAPR10MB5265: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MnJIItaxKp8tjkoUCon9IrN3svugaZizax9s/07rSN3Loi1o/JHCIgto+muTPLCjJ3njqBuPFvA9qX4jlAeOScVoTWvwpveaNWXjSdVvIR2fhZqL7AhOipCJEcbq6PvsuIgYdVWj97F9aChT5pLnUKjQJl9gVk7ZcHVwDEkssMsxHkzxch0Z8AWYC+qV+zWni1f8wXOYJgan6ezXHxHp5xKpe6UGkzXky9etBiM7FwCv4M2gEeKUKNvaqjJPEyA2ElmYLJ1doNzNKjgc4Pongl2h1YYTeOUGA68Y9e9ayYCX0Fb8XS3HexELXgXGyPT16FTWQmhQuaBJ/deQuAQ2JHQYsQ9m3JcPmFiwuwlYpXsqX2iXwq6uQ0g4JjnPXEUkqEDGuXEeCNHR161S9Ho5macfM+xW5aTaFHM0p5VjrCUrcoZJiKyscR2MN4SFuY+BnPqemH2+SJauMFUcY70PcYhWNCLG4NLnE6Jegq5sXCDyay0wOv/uUx+x0+JrX6euyrcNFrhgQpaO0mzZpLp0jXp/RNe58m2547AenwKQcIQpxfmviTrnuu83/iUIpakGqQrp0PWx59OSlrSICmIMaegcpRCg3+LJk872GdQZOsHv3Ql0Ib/15U5dZuurZfXAl+uY342xs+qL8maSvryLhjHg+rCu4O/nUF6YSHYyXAk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(346002)(376002)(39860400002)(26005)(956004)(52116002)(186003)(38100700001)(36756003)(4326008)(8676002)(44832011)(2906002)(316002)(66476007)(2616005)(7696005)(478600001)(8936002)(86362001)(6666004)(83380400001)(7416002)(66946007)(5660300002)(16526019)(6486002)(7406005)(66556008)(21314003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: XRF6Zxwa/GKcSbl355G5fZESpiN3H2ftDoha65OHq9NJvD3YlLMUAdeeOkCw+/IcLrMEHIlBPiD2OCTvq2TpQEFm5+4q2d2P4qjHt48DrfJkJFuE09XLw2FOS1Wp4RViPyvlCUYMVutZcbpP9MiWgduCd32Khtuf1RSjMqzMuQw6ZIGlmYeeDaVt3K9+OmAM9qCwSMF/savrp+LkVVRangdqs7vMvrVJXzIbcwKK+/kg4x8hRtAJP0ZRHW4ADSqMZ1dMhSq6oRDFva90Yg2qCtuLLByOxIftnkHVApC6uJuqQQFvRhyI9WyMWYBqQadhxvG8rFfDJPuD/idNtcvHklUj/YwN08cEnUfXBKzb8OqBBzhhdSxFJ1IOWXxtNyS/8hFtaDcCkQn8v4N4woARwjoZHHE1eX9DZhcfO3tAOcSYFDjGJ4vrksnw9dbTPKUacgNS44pgZqDcUZVt07fOUjKEN+azp6otQARG1TZoDX/NxZev2XpkRIwB1QW8GOu0tEqI/JzkceJSeyPM8ZZRwgCk2Hfhbb7uU5VyxOunlR4UZdzudye4iK3jK0z+/CVXCTNhuS3P7sFbN/QiZV/BYGCGY3qIVTuQYJ2AGYnCKbcFg5/toNVWnajGMzMfv2Tl2cc21qXGP4SMV8QKJTbsGzg1Sp4jt2QoPhd+WLNVhMNev+cIgGR9FhXT+nLjlvSWlUgrVqeVr1TdyzYl6/y82sHIgO1FUbxf/rqZ74G/U+lSOi733RYaEpxPTae7GhY+LrrzOqYpZsE6YoH9NZN70e3a+4cAqULFLxOZWy/ukucW5h3VI5Rns/8E3vPj3x8oyJ2AwvNjJ6ksfw4PUufSChxr1Merp82GjssIH0CHyzQ3MuJ/V37GgqdqKO+1UT4Dn0G4mi/pftnIHFp3H1m1Qzoga6ZM4EGBiZM7RikSXF5a9wZcBEojtQ9gX5GqW/WmKgaPjxZiNwkwXBlNc1/9yOMChOVAOt3ozNvaDzmbTO7GSkVzMiRSUqQLXg6gpMSLEP3bkVRNo9OFlHvAm8VhZucF/5QFnZEpvkymiLa4fOjGdhCG0G7uz34YDC67GAg4m7r9Ge0/rzOaL/Rl3B5/51QB0C9sNajw+pdFKwUQmuzeET3qOSUy7RZ/WbrTaLLpOJM/TJEQ3ispgqv/YfskDZMYROK1jLHQPHy8Ghx06Zx1IBHgzWKItn5YZImKpRbpexkdflD0p9bwCYAdcr/bBacmnLAzTDbCd1eOZ4yH+zRIK/Vox2hqY2dowhafSA6RgS+ArHFOera/yqZIi01vMh7V10xTtoI27/IReVJzh8yzKdlTI+r5q5D8WldLTscL X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9026485b-54f8-49ef-2632-08d8f3c2a8ad X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:45.3579 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KQUtAJogR7awilxkKBv/hFhx7u4AIJt0m6FEuvWhkQpLn9TkKjZ8zE9Yyi4kD4VAvPjdZlu6hPCHfVHVNcKnbymCpTaE53fdSkd76nvdoFs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5265 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: aLCqfneXsEiwzpXEDaj3edm0I6pdUiFB X-Proofpoint-GUID: aLCqfneXsEiwzpXEDaj3edm0I6pdUiFB X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 phishscore=0 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Considerable contention on the LRU lock happens when multiple threads are used to insert pages into a shmem file in parallel. To alleviate this provide a way for pages to be added to the same LRU to be staged so that they can be added by splicing lists and updating stats once with the lock held. For now only unevictable pages are supported. Signed-off-by: Anthony Yznaga --- include/linux/swap.h | 13 ++++++++ mm/swap.c | 86 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 99 insertions(+) diff --git a/include/linux/swap.h b/include/linux/swap.h index 4cc6ec3bf0ab..254c9c8d71d0 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -351,6 +351,19 @@ extern void lru_note_cost(struct lruvec *lruvec, bool file, extern void lru_cache_add_inactive_or_unevictable(struct page *page, struct vm_area_struct *vma); +struct lru_splice { + struct list_head splice; + struct list_head *lru_head; + struct lruvec *lruvec; + enum lru_list lru; + unsigned long nr_pages[MAX_NR_ZONES]; + unsigned long pgculled; +}; +#define LRU_SPLICE_INIT(name) { .splice = LIST_HEAD_INIT(name.splice) } +#define LRU_SPLICE(name) \ + struct lru_splice name = LRU_SPLICE_INIT(name) +extern void lru_splice_add(struct page *page, struct lru_splice *splice); +extern void add_splice_to_lru_list(struct lru_splice *splice); /* linux/mm/vmscan.c */ extern unsigned long zone_reclaimable_pages(struct zone *zone); diff --git a/mm/swap.c b/mm/swap.c index 31b844d4ed94..a1db6a748608 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -200,6 +200,92 @@ int get_kernel_page(unsigned long start, int write, struct page **pages) } EXPORT_SYMBOL_GPL(get_kernel_page); +/* + * Update stats and move accumulated pages from an lru_splice to the lru. + */ +void add_splice_to_lru_list(struct lru_splice *splice) +{ + struct lruvec *lruvec = splice->lruvec; + enum lru_list lru = splice->lru; + unsigned long flags = 0; + int zid; + + if (list_empty(&splice->splice)) + return; + + spin_lock_irqsave(&lruvec->lru_lock, flags); + for (zid = 0; zid < MAX_NR_ZONES; zid++) { + if (splice->nr_pages[zid]) + update_lru_size(lruvec, lru, zid, splice->nr_pages[zid]); + } + count_vm_events(UNEVICTABLE_PGCULLED, splice->pgculled); + list_splice_init(&splice->splice, splice->lru_head); + spin_unlock_irqrestore(&lruvec->lru_lock, flags); +} + +static void add_page_to_lru_splice(struct page *page, struct lru_splice *splice, + struct lruvec *lruvec, enum lru_list lru) +{ + if (list_empty(&splice->splice)) { + int zid; + + splice->lruvec = lruvec; + splice->lru_head = &lruvec->lists[lru]; + splice->lru = lru; + for (zid = 0; zid < MAX_NR_ZONES; zid++) + splice->nr_pages[zid] = 0; + splice->pgculled = 0; + } + + BUG_ON(splice->lruvec != lruvec); + BUG_ON(splice->lru_head != &lruvec->lists[lru]); + + list_add(&page->lru, &splice->splice); + splice->nr_pages[page_zonenum(page)] += thp_nr_pages(page); +} + +/* + * Similar in functionality to __pagevec_lru_add_fn() but here the page is + * being added to an lru_splice and the LRU lock is not held. + */ +static void page_lru_splice_add(struct page *page, struct lru_splice *splice, struct lruvec *lruvec) +{ + enum lru_list lru; + int was_unevictable = TestClearPageUnevictable(page); + int nr_pages = thp_nr_pages(page); + + VM_BUG_ON_PAGE(PageLRU(page), page); + /* XXX only supports unevictable pages at the moment */ + VM_BUG_ON_PAGE(was_unevictable, page); + + SetPageLRU(page); + smp_mb__after_atomic(); + + lru = LRU_UNEVICTABLE; + ClearPageActive(page); + SetPageUnevictable(page); + if (!was_unevictable) + splice->pgculled += nr_pages; + + add_page_to_lru_splice(page, splice, lruvec, lru); + trace_mm_lru_insertion(page); +} + +void lru_splice_add(struct page *page, struct lru_splice *splice) +{ + struct lruvec *lruvec; + + VM_BUG_ON_PAGE(PageActive(page) && PageUnevictable(page), page); + VM_BUG_ON_PAGE(PageLRU(page), page); + + get_page(page); + lruvec = mem_cgroup_page_lruvec(page, page_pgdat(page)); + if (lruvec != splice->lruvec) + add_splice_to_lru_list(splice); + page_lru_splice_add(page, splice, lruvec); + put_page(page); +} + static void pagevec_lru_move_fn(struct pagevec *pvec, void (*move_fn)(struct page *page, struct lruvec *lruvec)) { From patchwork Tue Mar 30 21:36:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D0CFC433C1 for ; Tue, 30 Mar 2021 21:31:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5AE5F619CF for ; Tue, 30 Mar 2021 21:31:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233285AbhC3Van (ORCPT ); Tue, 30 Mar 2021 17:30:43 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52884 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233428AbhC3V3o (ORCPT ); Tue, 30 Mar 2021 17:29:44 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPewA145444; Tue, 30 Mar 2021 21:27:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=KQY8NusArg565ekWV72k5w1pkevrHjUNDXmRkiTni1c=; b=hBPe2WVY7LyopRfu6LZoQhKJiacygeyvwi1A5q3hZhkxO9NQTk7QyLZuctOycSoJzanE HtfWJiGxx/J3de2svH7/AwDpoQT2ZNai8l9e2sdLboo81VScJEoxmRir+3rvFp0o+a27 Ah2lr+a8q3eGyI0aaeubwm6x84zTfXZb5T3oOykXj12OvqL2uf1JJ/vrhF3g15ex1/iD mofJpbfiWTnzD46vOY2BEemGYAnWGcsC/LP+fjpGMUicMOMoVlStiR+58j3qkPWjF7UI he8tlPI3/msZDQonsGTrr9Ii1151SLTy2nmOkeLZtjMNUnGvMC+yT97Zb11v/3FKx4j7 mg== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 37mad9r8jj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:54 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOnJM184071; Tue, 30 Mar 2021 21:27:53 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2049.outbound.protection.outlook.com [104.47.66.49]) by aserp3020.oracle.com with ESMTP id 37mac7u5jq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EYEf/aFmg6fBJCpz/vT0BVQK5hwbO+J2WffPsmWwTmNSwIejJ/E7dYGL8srjVkds6FJ4eB6LIomNc9EpMLKwiAn9aK19HenF+Yvx7hvnfsWujxAByZ0nYkBXRvS9M92RTKAxt8t3QPs3chZ7pIaBatj3O9vvvrpOzMwH7ls9ICnryB6JXOebJDZHcpVfBr6hje1khAiaWHQQWhb6KS3uAOL0SeVZSjT0nvMqw1fzZx7VItHlbYqjl4fpcvC5TTn98tr1m2WDJZ6xmrwlC0+hdh0u7FilxhfGgTOCo21+bk0L4j6Iw58J7M/cGG2e+9cEk2JuHG9EsrCIUEMlXZVHNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KQY8NusArg565ekWV72k5w1pkevrHjUNDXmRkiTni1c=; b=jTESfLSMspKFERHCfJD8irOv+LDFJmB6ifb2vYWn+89PCabrHHou6kuS7PbiToJRSfr+yEFOP3Lp0ejyiRQm/kDYjj6HkkX7KwpSqd1y5zUqlD2duNrKQtPDNGauCfpEiuP75l8N7pxBdpb0bEdyvvvlJU/lc1WjnYT3lUrViC9nd8e4u9P67nH9YSNOQwx6P4ShbctVxnGez+f3P15PQd/JYSEFzJOZIh4Lb9fopIHFX46nPUelwhpZNq+iGup06KXBCD8fBkm2g2EJQu9XYb+dmmIqWSft+eCnmpv9Z2HSiBvXLvqbBHCRMkX+eafB9kSMt+18yYgkMbnBPcYMKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KQY8NusArg565ekWV72k5w1pkevrHjUNDXmRkiTni1c=; b=Mcu3X6dKHJ+Sd5toLgfgIPNFe1CJGmSafyjofcOAXjn9PTbRovDSDSPxYoDfRM2/gvQKw8Lw9EnjvobN8Rn8aT3yjTwl4lT8OqMf7EKNvnowGUSUIx8+ZV7yJwdbYetZkE7S5mkggdyLXDuzePBtwupKGC/JebZejdu8GqGndls= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by BLAPR10MB5265.namprd10.prod.outlook.com (2603:10b6:208:325::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:49 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:49 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 39/43] shmem: optimize adding pages to the LRU in shmem_insert_pages() Date: Tue, 30 Mar 2021 14:36:14 -0700 Message-Id: <1617140178-8773-40-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:45 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 54ba4375-49ee-4901-904f-08d8f3c2ab35 X-MS-TrafficTypeDiagnostic: BLAPR10MB5265: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3631; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TLZ457ebP33KpY29CyXgVRqd5+EwwJ9SEfvlTMzNp7kOdKdglw99bSEkLM0ne0vFzm/z2WesNE44PJ2/1XF+aaKR0/nUsdDGdlTokxjT2sWMiJbiwMrfzTLMkQJwNa3aItLIP/EDVnCFCktyy61dxeXbNYpi6M66mwx81Ar9aMcC7x3qPZsRBUzgMkTwRGubmJGNqSMd9ClM41VN/EIWCwMZGPBJ49OkA4fxEP9MSySrjvIvbBGPNy+3IHiXpa9dhlBOB/UbMNMy8Topsl3yDzdPNxI7E2zzOcKpXhWFnMDuYu3QzcZZVk6RUlO4iTJL1qaPM2V64h4icJ/77lgQ6Uu3OpItmV7gwOaftBv1+Mle+wocUgrmiz6DUZvu6kOdfgEfQ15Qn+Hp/ZBLYpFDFVXZTKONapCQ9zuS6UkMVNieZUaeg1iPeceSzOzqwwmH+uKgNVeiBJjU53qnLcZesDbT7gkOk84BwhgxVQB5cXrGqSjuKzuCisWblvc45ZKhl3mATbWV5x8GITgIC4gmyj69BsySnJw7v8/IlX97+OlhfOVF4yhjr5hugL/Xe2gZsGqQSwDxdh5CFC9pKlekgIN2XYRx685K+wwcaDIlgd0PhIQmNUxDAG/3STtGf5MGjZZQuj7RTWmWcQOeiGC9Eg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(346002)(376002)(39860400002)(26005)(956004)(52116002)(186003)(38100700001)(36756003)(4326008)(8676002)(44832011)(2906002)(316002)(66476007)(2616005)(7696005)(478600001)(8936002)(86362001)(6666004)(83380400001)(7416002)(66946007)(5660300002)(16526019)(6486002)(7406005)(66556008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: dBnoJoljv1nhW3uHqnS94kMNn5IaPhkAa1fFnNUkr66vRLcLorN3WaoJKQ9wdbWL2dOaSwOYGOsIO6JRyBGXaDmqZg7N9qEEyjK75FfaD/Pc0taq6xew5VsLcLo/4hXo/tFPMOo1TnZWxW38AjMF563qHDx8WQQo2WXneF8nXrFa58Vt/93lbRFGBhRnnZ0qJrSQhGAlxFFGXMUHi0DGX7X8pNNs23Rh1AXJXQdNiLrNV+1Q5j+Z1zMoXOVyCoV8k/DQ7c0+C1GXmWexspUENBXTbhjKZHfbt21SOvh3yp1b4+EItNXgR7h7+9jgxN4Sry5mpMDCp8Vh6vayiRaB0oAXIbf7VUSbisYq8j4q3FdCY6NHHqvllDyD7dhye6O6Z/riPQV8eBdSFuDtvRv799HT+fimh5YAXS7SBZJQkQWEnLK9zOKvEfREDG6wyaj685x5G4xgVCkxMSdFdarb+9FjQYd81LGd3w8YwvjKptu1j1RBJz9YJ3hXqaGeAEhiDU5xTvIsL5/QRNc7CYaDnTIXBXDuQQek9xvmf6vHljJSl229VbcD0l/YJsciQewXtJQX2/nWYir1I+d+QEmQ74Kgu4w+MVoeONh+13/ahfdYoPC/Rq/y/rmR4Z1j81C3cwWjQE9DdvkLVXqSRgN9o6Ukr0c62DI5DwzIi9KNa1OJ+2o9LkO1w8NPlbbZ4fpJOO+swAq6G2u55mMEuwyBhlBhGG5iIwCi9ExbafgApb/pgqce0cfpnC+WgafR13QWeo1N7g69kfxmNKs8wp0G/rPXM9tKY54dfqgOILR0QPzwLhhRcIuI5PA6I9Ftlmajoa3/o9kyXUmqZbXP8mMnYCHuHS6ylizE9+v9ZfhGI7ohXSH7dNFRwv4BjYnQRB9eFvEDwQr384wzEFft5fzRtF+i/MTIWwBNVUe/zwx4KX3QjjrtJcLlz0UM2LeGAiqG0G6xarbgy2k4rVBlJsqsBGLZa8B5ZVVqOTJ5SPHf+zGyQLBMJS4/qMYX3oN7XXKBcgYdO4+cbQ67QhhMiX51z4m/YqxKrMB1hM4Bs6S8UO+ty8NfqycSa2RKfoKyPW5gbFYJYZteVvFSLkotrZAXDjZrFRbL7c/ZQLs76QGKlbVK69j+RU713LSAhlDqfpIAcQSL/xr72sVe8CQthLZZaJRmv8LQRipDki4htv05TyXe9KRtN+HcuIGMjef+dARlRZVdlJOi7Azby8CXbzwSFsJI1/RqEBIAqxC4zR8sVal2u0I8q1hag4tb8Xmm8SyhjfCzvsWh8RMhlTAkWVw+B3N38UVKfSh3mTMcyGjupPYcJmADh7YtyTRsl0GhqoGT X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 54ba4375-49ee-4901-904f-08d8f3c2ab35 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:49.6213 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PnFa+B0jNgY9U2JD2js4JYdeXPQRWTvAyHPPwrj6BRrnaagNBmN0XZIp3a09TUfzIkLRz3aJ3T2l9D6O0wlNCPckpOOF3iDVb4bGTS+0674= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5265 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: E-2LpQrFvFE-HrWb5ArFHEtRigBhOqSe X-Proofpoint-GUID: E-2LpQrFvFE-HrWb5ArFHEtRigBhOqSe X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Reduce LRU lock contention when inserting shmem pages by staging pages to be added to the same LRU and adding them en masse. Signed-off-by: Anthony Yznaga --- mm/shmem.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index c3fa72061d8a..63299da75166 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -845,6 +845,7 @@ int shmem_insert_pages(struct mm_struct *charge_mm, struct inode *inode, struct shmem_inode_info *info = SHMEM_I(inode); struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb); gfp_t gfp = mapping_gfp_mask(mapping); + LRU_SPLICE(splice); int i, err; int nr = 0; @@ -908,7 +909,7 @@ int shmem_insert_pages(struct mm_struct *charge_mm, struct inode *inode, for (i = 0; i < npages; i++) { if (!PageLRU(pages[i])) - lru_cache_add(pages[i]); + lru_splice_add(pages[i], &splice); flush_dcache_page(pages[i]); SetPageUptodate(pages[i]); @@ -917,6 +918,8 @@ int shmem_insert_pages(struct mm_struct *charge_mm, struct inode *inode, unlock_page(pages[i]); } + add_splice_to_lru_list(&splice); + return 0; out_release: From patchwork Tue Mar 30 21:36:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20020C433E4 for ; Tue, 30 Mar 2021 21:30:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F20D3619D4 for ; Tue, 30 Mar 2021 21:30:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233408AbhC3V3j (ORCPT ); Tue, 30 Mar 2021 17:29:39 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52280 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233287AbhC3V2m (ORCPT ); Tue, 30 Mar 2021 17:28:42 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPHwI145355; Tue, 30 Mar 2021 21:27:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=MfqUNVuTZH17wEzrzvOmISttF92wGuYIZhlq/gCT0PM=; b=oEiWCD0oJbRxTK1qzWGzMGr9wwVhX8qX9EnQj1JdmBKiPRId35l3eqRygr9OcUQz/8Yz 0F2f+TIO02fnueVt5PTBp8igeupNR4XjckTcA91bfRUaB4908XZM+TimaPXM2iin26xn 9T8gz0RGAzDFnbbnJrwryUjki06KT6E/27MuzdPmwOKocvTe6dhUdVI30nOhL6LsD/j1 x51fEFy5tOrySJ77mDXu9Sw2x7Rz/EQA1bV2fgb3RJ3uLX18NqShU27JNhnLDdscUCA+ bs5uC9ElyDjwc6+hbPJuz8Ge4pGHHqen6gJevZETUXarFa/hIfrK4I0qFfYvM40/YDAG Mg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 37mad9r8jn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:56 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOa1V149731; Tue, 30 Mar 2021 21:27:56 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2043.outbound.protection.outlook.com [104.47.66.43]) by userp3020.oracle.com with ESMTP id 37mac4kgkj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:27:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JLrmciJi2i1/MUPuqsQuO0QUcVvHtMJC2zlk0FJoBFVR6srFzhobF7Aj69XLnDzy1Cgv29Whdc6sH5IB0HAZXOzkLiswnncLG2l8yZhW/TXKOsSZLKSu/D+RPbk6G/IWT55OqDP+0yDQ8JiIHYMeh+JIuldLiaKy+/HhI5CQ7eK+dz1nsTX/Kfpe064lmDiFQkrA3on6qBX2Ijnz7cuuQGFzA1/WmHwhjzqDFiIWMRpDJyzeyspDt6qHgwDbxFQsUg45YhDs3RSPIR8UlXN0bIuSYmZtxASpGkvBkVIDqzeRFvpuBps2fqVZEQFsq/YQnDrX0oyzpMIJwIJz5ypjEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MfqUNVuTZH17wEzrzvOmISttF92wGuYIZhlq/gCT0PM=; b=VnV7NRVvm22T5bklS/yjIfsZTYHwwR7pV6cyaf87/HdewIG2/wlqj6FtvXEXpmq5qufyraxJS1tlwmtYGaN2Q8NdnyK73DZwPPTemQlV2y9ZgWIyYpf3l4DAAfSZ3H+ZJ7jQMLVZafMzNXT6xyhEAkxMPovjBWaEyDONfPsXFJkWhorw0YgoOdxiVnE749lRLFfagJjLcZVwjoH6dbZlCve4h/KUqZ4uH/RPCF1brxuNhAicgTpwFmmmlyPJ2zX5t6mrWQwjnFochrPfmTmiU0+oeQ9xuKaD/AKJr+IMCj667nX2ffyxnNHporPU12rYIVcmzvpr/K7EdBC7L8DlUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MfqUNVuTZH17wEzrzvOmISttF92wGuYIZhlq/gCT0PM=; b=fuiXK7as7vXE8QbRQH03DVtN+fxjx0nPIicGkjHtN6bV0zvY5uGMXO734G2lheakNLv/gsOBu9KlsaDBAg9v2l5myP/sujAgBh4oklBudTB2wixYMc/7rXe9xizzIsbynS9YeU+qx8Bp9fNbiOSNYvhqWNS/WdfwH3XCl6ZT7U4= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by BLAPR10MB5265.namprd10.prod.outlook.com (2603:10b6:208:325::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:54 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:54 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 40/43] shmem: initial support for adding multiple pages to pagecache Date: Tue, 30 Mar 2021 14:36:15 -0700 Message-Id: <1617140178-8773-41-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ba1fe498-4abc-4f85-e360-08d8f3c2add0 X-MS-TrafficTypeDiagnostic: BLAPR10MB5265: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5Iey2YuNQvFEH+OfRYKCcI/wdKSG4Y0lPKgi6RuSiihWJ9jZDtGH9YHI46lOeVssLo+hPJ55crOQLp1Fy7+Ds4nNRxJeS8+wO5nDdbznvo4zmJM4QxanZSxAt3GEhi11PLN6Ap8BriJtszui/7fCEXMRxayWzg7BXBQcjFxpyJVhoNyx205u7prA0o5j85AUpCenB2BB5lPM1qyvZyEGS4cgYb9xIVNWGvgHanv0QBCIYpkkLFTOFn4rzCFEjftshwCC+KRlu1QNxyx1XzSuR09HBJLvzCUJ+cRNUU8/be5xbKQIh8b9Vyk9B/RrDGR5ylT9XXkA0pvU4CvmsfaHoiMW0xzLF36SqlEusBBYa+1dZ/gOGBwHaY5Ddt1pzf+cOJZd/xvF/hkuefLW6i0zSUq7Yu04y6JrKrO5JHcbZlGltPee6QqxCyqmtuOIIx/FpQ/QR8lCojW+a+eIhlkGqoINs4x0JDX5Ah0uVKI3ko1vn+4WPJd+ex4KLeLu4MxKEJsLehmhszOGQUD+GHRjOFI43kSIiXUXDE3SmnUog1lP8lujmrVE7Nfdac6gSZLC1umgt9qEN+raGuBbqpU24cwoQQRJ+eAgsjg281eYMMvQzrT4hvQyGIhYWABz8+QnE8BOp3aL9lUKk5LBwdW3fw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(346002)(376002)(39860400002)(26005)(956004)(52116002)(186003)(38100700001)(36756003)(4326008)(8676002)(44832011)(2906002)(316002)(66476007)(2616005)(7696005)(478600001)(8936002)(86362001)(6666004)(83380400001)(7416002)(66946007)(5660300002)(16526019)(6486002)(7406005)(66556008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: nRI8UzovRGKm74IwWNHVtw61Zq32B36pQwRn4CwKxs5k0pO05HcV4QFPWCPsY8gBbm5BWySitXTbt3viEtDOtATQe3soFFc3VfJwbAia+FV+QL0g/KPr2zIoeLsQy2BIi3iRd4Yk1d7HAN+64WCvIahAx0BpEjdE+jHQYBuh7OdbVsWkLPI75WpKA9E3T/fAwq4n6C72Pozs0O244ZGXw38vq4mJiD/nkFG8A9A1t+krngNagbVrqodjPz4/WH14hKbZbUt9UcWvPh7sOgEEMwwL5ZRF34ZFCRKKWijJOUspq6EwV8IYEUwmegydJOudt5fTgTEbJzpiw1Gh9lyBG4QGqUICwC2hG8ip3ZndxU60DG4MiuRPxuZPFhjU+JqWFW2Pzp0j+kwOBkq5t+B4BGsnEXo4UDDrkqY0OJgkG2yY+XG2SSYnSjXnOah++oAWKenGooHAOr/ymU7FQsuwbLgaeFVEhkJGYNN83nNZ4ejyFuhwUm7x48GfuVqgj54PtNh2nvvIeX++0Sc0ldywBDlNd0lE+qpQsOmlVHCymivSD4rlOfy2m3CgxJSILIfzoMxkX0z2kxw5IoKYEsy1zVmsThEMwEHU4ECI953ifGfCrvnIS6Tu7fSLXA0MYvbGJEnomTZaKhxcegmw/RWUReGUgr/wZoZKtT2iP6aYJmmxGxsvt+9e0ohHz/K9QyeumH84kHu3tEg57XIJBFZsW/QOdZL7dTUwQ3L7W63as0rDgLBsxYqPEiI5f+0dSgmsCtD2ogUZiw+w6fdhHMeo6z8yOwY0uA+h5H0je9uUMQ0XUv303qMtTgDEo+AH1uySofp9/G3THfAVL0hmpZgFm3d7ZFeIm12JaP3s7RnChvM7B3aCJdxf24p7TKZfUW8r0f/1UBRrfg49Qi/pnBnJxh2S63ot8p/n5Z1xRqbaz1No9Suqop9EWFvvqpbYTZDwLRQKNFoqx99ciY2MOGPUYjRAE3kgumi2UYy1DH+x5uQTT9KzwJuWTMKLTv9bUgVI0HmBp1k3opwMSMHAfmyTT70A+iML3/8MsRKpwomICS5YQ7GKKCDaaw3cEDUABUsY/UiRQM6tAxUER/WmLsf+ECyeqXRvKL60msn9co9Trx8A1eDBf/T2OoYsWxUIkO2TSt3R+fFPNS7rDhnyEaDH5t9irr+CsO2RlAfLdSwDgKZ67ebENdQT4y0rQqH+pwdaxMaLlL6dPNJtfqeoKDfQ1yPDWYfC4EOl8xLPpC0voej7DTNOXFCqcGm3OEiwmdbjcG8xdAEf7I4qPqZdf+MDwUVzBKCgxyIgk5B0HHnmM3CQFJ2ZylTv+4/k3sqDggYK X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ba1fe498-4abc-4f85-e360-08d8f3c2add0 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:54.0057 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /2qpNosNOoY1nGg/7PzeTm6ZIILF5w5/3d/VdluFmrpX6ILJG9abIDfj5GHN49mmD6iRB18zIW20WX2RExUU1dS0pUW3WGIUPJMXogU8V+A= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5265 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: WbnzDWW1Fov0WSyMuYuZkGF5hBKIsxtl X-Proofpoint-GUID: WbnzDWW1Fov0WSyMuYuZkGF5hBKIsxtl X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org shmem_insert_pages() currently loops over the array of pages passed to it and calls shmem_add_to_page_cache() for each one. Prepare for adding pages to the pagecache in bulk by adding and using a shmem_add_pages_to_cache() call. For now it just iterates over an array and adds pages individually, but improvements in performance when multiple threads are adding to the same pagecache are achieved by calling a new shmem_add_to_page_cache_fast() function that does not check for conflicts and drops the xarray lock before updating stats. Signed-off-by: Anthony Yznaga --- mm/shmem.c | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 108 insertions(+), 15 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 63299da75166..f495af51042e 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -738,6 +738,74 @@ static int shmem_add_to_page_cache(struct page *page, return error; } +static int shmem_add_to_page_cache_fast(struct page *page, + struct address_space *mapping, + pgoff_t index, gfp_t gfp, + struct mm_struct *charge_mm, bool skipcharge) +{ + XA_STATE_ORDER(xas, &mapping->i_pages, index, thp_order(page)); + unsigned long nr = thp_nr_pages(page); + unsigned long i = 0; + int error; + + VM_BUG_ON_PAGE(PageTail(page), page); + VM_BUG_ON_PAGE(index != round_down(index, nr), page); + VM_BUG_ON_PAGE(!PageLocked(page), page); + VM_BUG_ON_PAGE(!PageSwapBacked(page), page); + + page_ref_add(page, nr); + page->mapping = mapping; + page->index = index; + + if (!skipcharge && !PageSwapCache(page)) { + error = mem_cgroup_charge(page, charge_mm, gfp); + if (error) { + if (PageTransHuge(page)) { + count_vm_event(THP_FILE_FALLBACK); + count_vm_event(THP_FILE_FALLBACK_CHARGE); + } + goto error; + } + } + cgroup_throttle_swaprate(page, gfp); + + do { + xas_lock_irq(&xas); + xas_create_range(&xas); + if (xas_error(&xas)) + goto unlock; +next: + xas_store(&xas, page); + if (++i < nr) { + xas_next(&xas); + goto next; + } + mapping->nrpages += nr; + xas_unlock(&xas); + if (PageTransHuge(page)) { + count_vm_event(THP_FILE_ALLOC); + __inc_node_page_state(page, NR_SHMEM_THPS); + } + __mod_lruvec_page_state(page, NR_FILE_PAGES, nr); + __mod_lruvec_page_state(page, NR_SHMEM, nr); + local_irq_enable(); + break; +unlock: + xas_unlock_irq(&xas); + } while (xas_nomem(&xas, gfp)); + + if (xas_error(&xas)) { + error = xas_error(&xas); + goto error; + } + + return 0; +error: + page->mapping = NULL; + page_ref_sub(page, nr); + return error; +} + /* * Like delete_from_page_cache, but substitutes swap for page. */ @@ -759,6 +827,41 @@ static void shmem_delete_from_page_cache(struct page *page, void *radswap) BUG_ON(error); } +static int shmem_add_pages_to_cache(struct page *pages[], int npages, + struct address_space *mapping, + pgoff_t start, gfp_t gfp, + struct mm_struct *charge_mm) +{ + pgoff_t index = start; + int i, err; + + i = 0; + while (i < npages) { + if (PageTransHuge(pages[i])) { + err = shmem_add_to_page_cache_fast(pages[i], mapping, index, gfp, charge_mm, page_memcg(pages[i]) ? true : false); + if (err) + goto out_release; + index += thp_nr_pages(pages[i]); + i++; + continue; + } + + err = shmem_add_to_page_cache_fast(pages[i], mapping, index, gfp, charge_mm, page_memcg(pages[i]) ? true : false); + if (err) + goto out_release; + index++; + i++; + } + return 0; + +out_release: + while (i < npages) { + delete_from_page_cache(pages[i]); + i--; + } + return err; +} + int shmem_insert_page(struct mm_struct *mm, struct inode *inode, pgoff_t index, struct page *page) { @@ -889,17 +992,10 @@ int shmem_insert_pages(struct mm_struct *charge_mm, struct inode *inode, __SetPageReferenced(pages[i]); } - for (i = 0; i < npages; i++) { - bool ischarged = page_memcg(pages[i]) ? true : false; - - err = shmem_add_to_page_cache(pages[i], mapping, index, - NULL, gfp & GFP_RECLAIM_MASK, - charge_mm, ischarged); - if (err) - goto out_release; - - index += thp_nr_pages(pages[i]); - } + err = shmem_add_pages_to_cache(pages, npages, mapping, index, + gfp & GFP_RECLAIM_MASK, charge_mm); + if (err) + goto out_unlock; spin_lock(&info->lock); info->alloced += nr; @@ -922,10 +1018,7 @@ int shmem_insert_pages(struct mm_struct *charge_mm, struct inode *inode, return 0; -out_release: - while (--i >= 0) - delete_from_page_cache(pages[i]); - +out_unlock: for (i = 0; i < npages; i++) unlock_page(pages[i]); From patchwork Tue Mar 30 21:36:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4839C433DB for ; Tue, 30 Mar 2021 21:30:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A5B64619F0 for ; Tue, 30 Mar 2021 21:30:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233319AbhC3VaJ (ORCPT ); Tue, 30 Mar 2021 17:30:09 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43948 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233331AbhC3V2z (ORCPT ); Tue, 30 Mar 2021 17:28:55 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOjq1011609; Tue, 30 Mar 2021 21:28:02 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=StS8zsAa0vS/yJW7ncnSFIShqW2WyW3ks/4y0oh9XnE=; b=yDaZu6T3voG18G+W3UL6lYZqiDnmZEAVxkp23w+G4nrQD+jqVrHZq+iNyxvV8cb9tqnv UWjDKlK0VDX2BrxnyKaMN9mrfqPrP2+xg8FEvsYHonulePoeaUZOK/PFOYuRekGpsFGn mcfpIyVDbCV0qts/wiGucXfUMZidRnDeESyegSH46skHxIa+7NN2wrgJi5rgApAsCVBD //xAaxOnKWXlhWmhAJHl/b6inUIictd/RpZTu4OBjjRwTr8iOe5QYEt3fVYxpiuJB5AR FyHQ7B2na5CoK1U0wMo6WzaYpmjOpDePcPpP/nauBCISIk+YkYJD9liJ26EW5pdGK++z XA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 37mab3g91g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:28:02 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULOmlg183986; Tue, 30 Mar 2021 21:28:01 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2103.outbound.protection.outlook.com [104.47.55.103]) by aserp3020.oracle.com with ESMTP id 37mac7u5nv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:28:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VSRlm3iCB3WuAZvRoMHtpXeUjLve6vGteF6OvM39UuIvd9PNu/KEmMrEhBhGPmyrLXzfF128SZaB2+FGOx4vWaWKsjTpb1f8X4FKbTwZI7QhJCmJWcW+L8fR1hLcB5BQpP7/Z62mvItlsVbcuG4FPRY5Jhz/KQtZdm+hJ5gClNNQyusHftLqf2x1lfE0NLvJpUjfmCZaWilpHJq9l0gPSzGu95Vf0aAf0xDdnviPOKXjEpsOUrOCN50r0MtKM051s3FCFHq8eSm2jTObZYE4y2kRG4tle9cmc4/MUdKZEPDePtoyqNhzf7pp8ZR8YSqfovsp5zH1s4JzSh4k136Fow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=StS8zsAa0vS/yJW7ncnSFIShqW2WyW3ks/4y0oh9XnE=; b=VpSRymwyb/n6qHei3DxS6OVvKkeSgs12ZbeDGjUYDm104bUsgM5D98sWNwbvyX4Kv6txUDmX1LeY8/3L0N/E1TEzzFZCQlCkw2ykkSmEMKGwI5Ra8DASAqk8uyTbks84YwiSJm0qHS8gxB7cfvr9Vi8jbIUhnCjnouVzYSXjEBwkKNKv4bOAVvQa0YJsjyy3ATDENHLBHRJvBHDHN6MimfVcz33g/IPaYalJE3tq3YFGcnU5cu8i8J0ZUdDPUTHO4fCKc82zm5Y2wNN+Tf4D+EusjWm1yL28fZOMsUj2KhLoSDZ7s0an8uPPrWxibYJXUOieNhjyAr2AV2Kpu/t/vQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=StS8zsAa0vS/yJW7ncnSFIShqW2WyW3ks/4y0oh9XnE=; b=jcEQedXq5pZkYWWBQclfxyy4ETw2cOKPYjjopbOffmbIp2SmLBS0TDpL5KCiDQy815W1D2lWsxy2SQxYnQLJ9l/atXWqQQF0uA4g1LTA13FWGQCNi+6/FNTrcIq7VEv3AWYY8Nj1BmyrapH8CvOPnhRLW8fsKMam96nLAcBRjec= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by MN2PR10MB3679.namprd10.prod.outlook.com (2603:10b6:208:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:27:58 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:27:58 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 41/43] XArray: add xas_export_node() and xas_import_node() Date: Tue, 30 Mar 2021 14:36:16 -0700 Message-Id: <1617140178-8773-42-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:54 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 720da738-137f-4934-b7ba-08d8f3c2b078 X-MS-TrafficTypeDiagnostic: MN2PR10MB3679: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: APIWRvuKH+oXWT697yGj10JyMKQHD4MQmRBUKuO1EaIH+SKzg+r3qIHpXC/ExLd9QWqvYMltD3B5/rumzrg2cSH98bQGpd+rykSIxunFNAD7l9n3zGPAtOVB/CNXuKB9bLAQfa31fcrvpdTKpiLRo3S2NIeIn9diG9K9V4+du1uNeKW9YHdOyF32HM+Yza8rLthNKl4sQWnQjmhK86yK3R8BMhT3ZHs0jZB55jdF4K08VqDdyVl2ZUI4I4LavRc3BnpQacnqrfhvl0b855sL6ao57VsjeQLdS0TQ71o00OrpabgiBQQwvIwmLXKpdxwJHOSVXR7Ub0Ap+ctG9EUJo1GejLBiKzosmhJL9ogvKOzBumCgqB6rQFP9tiC6k9ax03BsrJZHrZakAKV/l//0DIG4aklxXvmIAXXhPCDvlEu9sjan9sFLapbN5G26Bn6ngYmhV9tzEkYtWGBS88QxuvXoQcxrLVEuu5naCLnBfMPdWXQ/8P2YCvCKp2dwNjKDhzZBMGrp/NMpj6RGK2fQ4dp2/DEzbDKJqjfqRhmVsxVc13yaiqEKEFaW5HeCmh+3lxDeNcDhpTUmzxM2NRpbrLoF5TXVLdgW7ES5IrIYvsGJFDQXStVdGXB2pgD2jewpLKyf99fP+lAWc2fl9oDytg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(346002)(136003)(39860400002)(366004)(6666004)(4326008)(86362001)(52116002)(186003)(478600001)(36756003)(5660300002)(7416002)(26005)(66946007)(66556008)(7696005)(2616005)(16526019)(8676002)(316002)(2906002)(66476007)(956004)(38100700001)(6486002)(7406005)(8936002)(44832011)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: S6jq4r6EP8JZ70qjeRB6agOM2dIojs28TI5Fi+ISRtF/dy62J4fMwj6x2uv+9Uj8AzyKojUVUqkJr6PO5AbvMdPErV/woe/4DkZXPaLrpMjkfc/iIU+6EgkBe+iHBihD1yeu1rxOXqQ26OXCFTZ3/F/7j/i1UbaXplt4KKwnl+4/k2d07Vyl8EemeST7jGKpTtGTmaIMktAK88olS8/bMz21cQ0msxNrZ1ErtW2dt1U4POPybM57Wq9xqdawlbHTsTAyJsZDyCxRgq/u50Tg7N/B6BcnZVV4TzZqMBdkyOXK/iqpyG56Kh6ObNLXt7EAfymV3KkePX+TLrdn40cMmGNR69plsTUr8Fon7mlxDa0cy30t5Wlsj5jc5YuJjVmlfzajsELq9HvQqKTGTa1vBnl6GyxCwBkt0XrxRZSJaXwER5RnEKT/PNsNDiNZchjUnukAgvUtf0iDSGBq/IjwPYugckdFxfPjKXZgYNCLyVMy+OZ+t0DZHc49P0c3aTw2fIHXUW31S3QdR2lDwbAOTxT5lz6UObP5VCHEY/fYWCmEPYUeJQbxOmBa2HI7CVtJozF69Mv3O4meFUt/VoYcXlyugA+XUaHJNeHRdpry3y/HbF8ZfthS4DMS/VPC3npiBuJ0rtbNCnXxnh53Vsxl2eJhGkRQeEZ4rqF+KGPoRsmArEuBzWceOi4PtqJ4iZXE30tVDETiPqxlHp6l/nTl62u1rwcOnG1Ml3T4ju5b3vNCZH4HcPivV9u3DXKhHQeZq6Zfna4jDNCNpkLFzMDQJDDW2m1YUzTipcAQC6kZmJxafMMXeMYRvweQfxD5g07yRJXhcslNABl1XlQsNu2caX3tPM+n6h8wYc7UMBpiZic3sEFtm0IenG0HzsqsGrNhlBByw7w1FjiTSEy14TYL+mBDNEVyEiwo1TWzt+CPOk2eu0K/XL6UZSnuPGxEIZK+rQGiNEcbUt7YoR2FF2qKX7bEgofHe/r0VQHRNtWcP8qtxr0EYXl6vkYfj6IygB09oefOT44bg5bKmlM5/PNvRcnwVLqeKRSvWdtqvZsI5LojnoKxdMJ16gxzeh2W1uklgmWMjO+6gl2XqBexthgluau/ZyYp2Ubihmm5Zm5LT/Bv28w9fxCybiMfpBMomc5AVcTYMP/3c/IRyIIfspfNdu4aouVPbUg1q0vYJzpsm1TqKl0r+BlJKqD6xYR7M9NyaO3NC9DPuKLE39iLcNQmmhriwFLe5LAe/dbi6R94ZLLmPC5V8bwJSag8wDX5ECZ8aWvxeUSkMwsGc+p5JqUOJtmsoOhshda53JdmBLVgE+28XE/VtFHKNzPn6RU9/FCd X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 720da738-137f-4934-b7ba-08d8f3c2b078 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:27:58.4440 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: q8KPoOfps9JiFF4NEjORrvVP9mjsbsVQ6Chrj9Kfsmjt81/DCkXb9qy6N0TdbsRYjhltHCOq9x+ilXqccGmIUxBF1Cb33adMHFqyyucSDik= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3679 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: 5sId7tjDJ2EdacY3ZifYoHTKYQaaeI4i X-Proofpoint-GUID: 5sId7tjDJ2EdacY3ZifYoHTKYQaaeI4i X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 phishscore=0 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Contention on the xarray lock when multiple threads are adding to the same xarray can be mitigated by providing a way to add entries in bulk. Allow a caller to allocate and populate an xarray node outside of the target xarray and then only take the xarray lock long enough to import the node into it. Signed-off-by: Anthony Yznaga --- Documentation/core-api/xarray.rst | 8 +++ include/linux/xarray.h | 2 + lib/test_xarray.c | 45 +++++++++++++++++ lib/xarray.c | 100 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 155 insertions(+) diff --git a/Documentation/core-api/xarray.rst b/Documentation/core-api/xarray.rst index a137a0e6d068..12ec59038fc8 100644 --- a/Documentation/core-api/xarray.rst +++ b/Documentation/core-api/xarray.rst @@ -444,6 +444,14 @@ called each time the XArray updates a node. This is used by the page cache workingset code to maintain its list of nodes which contain only shadow entries. +xas_export_node() is used to remove and return a node from an XArray +while xas_import_node() is used to add a node to an XArray. Together +these can be used, for example, to reduce lock contention when multiple +threads are updating an XArray by allowing a caller to allocate and +populate a node outside of the target XArray in a local XArray, export +the node, and then take the target XArray lock just long enough to import +the node. + Multi-Index Entries ------------------- diff --git a/include/linux/xarray.h b/include/linux/xarray.h index 92c0160b3352..1eda38cbe020 100644 --- a/include/linux/xarray.h +++ b/include/linux/xarray.h @@ -1506,6 +1506,8 @@ static inline bool xas_retry(struct xa_state *xas, const void *entry) void xas_pause(struct xa_state *); void xas_create_range(struct xa_state *); +struct xa_node *xas_export_node(struct xa_state *xas); +void xas_import_node(struct xa_state *xas, struct xa_node *node); #ifdef CONFIG_XARRAY_MULTI int xa_get_order(struct xarray *, unsigned long index); diff --git a/lib/test_xarray.c b/lib/test_xarray.c index 8294f43f4981..9cca0921cf9b 100644 --- a/lib/test_xarray.c +++ b/lib/test_xarray.c @@ -1765,6 +1765,50 @@ static noinline void check_destroy(struct xarray *xa) #endif } +static noinline void check_export_import_1(struct xarray *xa, + unsigned long index, unsigned int order) +{ + int xa_shift = order + XA_CHUNK_SHIFT - (order % XA_CHUNK_SHIFT); + XA_STATE(xas, xa, index); + struct xa_node *node; + unsigned long i; + + xa_store_many_order(xa, index, xa_shift); + + xas_lock(&xas); + xas_set_order(&xas, index, xa_shift); + node = xas_export_node(&xas); + xas_unlock(&xas); + + XA_BUG_ON(xa, !xa_empty(xa)); + + do { + xas_lock(&xas); + xas_set_order(&xas, index, xa_shift); + xas_import_node(&xas, node); + xas_unlock(&xas); + } while (xas_nomem(&xas, GFP_KERNEL)); + + for (i = index; i < index + (1UL << xa_shift); i++) + xa_erase_index(xa, i); + + XA_BUG_ON(xa, !xa_empty(xa)); +} + +static noinline void check_export_import(struct xarray *xa) +{ + unsigned int order; + unsigned int max_order = IS_ENABLED(CONFIG_XARRAY_MULTI) ? 12 : 1; + + for (order = 0; order < max_order; order += XA_CHUNK_SHIFT) { + int xa_shift = order + XA_CHUNK_SHIFT; + unsigned long j; + + for (j = 0; j < XA_CHUNK_SIZE; j++) + check_export_import_1(xa, j << xa_shift, order); + } +} + static DEFINE_XARRAY(array); static int xarray_checks(void) @@ -1797,6 +1841,7 @@ static int xarray_checks(void) check_workingset(&array, 0); check_workingset(&array, 64); check_workingset(&array, 4096); + check_export_import(&array); printk("XArray: %u of %u tests passed\n", tests_passed, tests_run); return (tests_run == tests_passed) ? 0 : -EINVAL; diff --git a/lib/xarray.c b/lib/xarray.c index 5fa51614802a..58d58333f0d0 100644 --- a/lib/xarray.c +++ b/lib/xarray.c @@ -510,6 +510,30 @@ static void xas_delete_node(struct xa_state *xas) xas_shrink(xas); } +static void xas_unlink_node(struct xa_state *xas) +{ + struct xa_node *node = xas->xa_node; + struct xa_node *parent; + + parent = xa_parent_locked(xas->xa, node); + xas->xa_node = parent; + xas->xa_offset = node->offset; + + if (!parent) { + xas->xa->xa_head = NULL; + xas->xa_node = XAS_BOUNDS; + return; + } + + parent->slots[xas->xa_offset] = NULL; + parent->count--; + XA_NODE_BUG_ON(parent, parent->count > XA_CHUNK_SIZE); + + xas_update(xas, parent); + + xas_delete_node(xas); +} + /** * xas_free_nodes() - Free this node and all nodes that it references * @xas: Array operation state. @@ -1690,6 +1714,82 @@ static void xas_set_range(struct xa_state *xas, unsigned long first, } /** + * xas_export_node() - remove and return a node from an XArray + * @xas: XArray operation state + * + * The range covered by @xas must be aligned to and cover a single node + * at any level of the tree. + * + * Return: On success, returns the removed node. If the range is invalid, + * returns %NULL and sets -EINVAL in @xas. Otherwise returns %NULL if the + * node does not exist. + */ +struct xa_node *xas_export_node(struct xa_state *xas) +{ + struct xa_node *node; + + if (!xas->xa_shift || xas->xa_sibs) { + xas_set_err(xas, -EINVAL); + return NULL; + } + + xas->xa_shift -= XA_CHUNK_SHIFT; + + if (!xas_find(xas, xas->xa_index)) + return NULL; + node = xas->xa_node; + xas_unlink_node(xas); + node->parent = NULL; + + return node; +} + +/** + * xas_import_node() - add a node to an XArray + * @xas: XArray operation state + * @node: The node to add + * + * The range covered by @xas must be aligned to and cover a single node + * at any level of the tree. No nodes should already exist within the + * range. + * Sets an error in @xas if the range is invalid or xas_create() fails + */ +void xas_import_node(struct xa_state *xas, struct xa_node *node) +{ + struct xa_node *parent = NULL; + void __rcu **slot = &xas->xa->xa_head; + int count = 0; + + if (!xas->xa_shift || xas->xa_sibs) { + xas_set_err(xas, -EINVAL); + return; + } + + if (xas->xa_index || xa_head_locked(xas->xa)) { + xas_set_order(xas, xas->xa_index, node->shift + XA_CHUNK_SHIFT); + xas_create(xas, true); + + if (xas_invalid(xas)) + return; + + parent = xas->xa_node; + } + + if (parent) { + slot = &parent->slots[xas->xa_offset]; + node->offset = xas->xa_offset; + count++; + } + + RCU_INIT_POINTER(node->parent, parent); + node->array = xas->xa; + + rcu_assign_pointer(*slot, xa_mk_node(node)); + + update_node(xas, parent, count, 0); +} + +/** * xa_store_range() - Store this entry at a range of indices in the XArray. * @xa: XArray. * @first: First index to affect. From patchwork Tue Mar 30 21:36:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E326FC433E4 for ; Tue, 30 Mar 2021 21:30:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C3E0C619D8 for ; Tue, 30 Mar 2021 21:30:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233103AbhC3VaK (ORCPT ); Tue, 30 Mar 2021 17:30:10 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50814 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233320AbhC3V2y (ORCPT ); Tue, 30 Mar 2021 17:28:54 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULNsLF130340; Tue, 30 Mar 2021 21:28:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=W+2R26l/jbQHknYkilwgK0LZFItVSlSZ+UXGymf9uJM=; b=txt5SPlN1eDtVstaB+GVMVo9RJhry++cPXEjDowMs2dPT/C62xfmWeXBsgHPj2AHwuvv +uI9JbdlHmRzSUC+mIX1RfvhbaUBmyyaNFa1QbV0p+1oNwoM/I9FXObgCGREGlb/l9Vq Dg1ouJFInqmx2jFg1qq2rz1NI1un1CFArVGVu4VWbloGcjJWSwCATs+Sq+fqcDDUdGcg U9D1P0Cjb+K2hWKFfajlbDeqW3u7P924ib1DNEfEQPlLuiOPQqfpzcU1fWwsHuc7qKAM OyvkRIEWtqeZaeoJIGqpWWx81Hxky3mudz6oWLVvFZWGIkiTXP69kPzM2YVCrzMuFe4j IA== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 37mabqr8rw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:28:07 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPZb0124960; Tue, 30 Mar 2021 21:28:06 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2106.outbound.protection.outlook.com [104.47.58.106]) by aserp3030.oracle.com with ESMTP id 37mabnk8gf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:28:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TsPjDv1VMCV4qNQIw79EDl4hcMAXXEpDGzJmttoCF5EUF5i3mO0e/bEWOWqUMl1oOwAZ2D3kJvMQ6GFk7er3jbpvlwiKf4l6ZyISE0eud1NmC8uTN85/lm2Rf9KfIxV/FWpClC6QdXf0LQSn3DYz8QRpt2b0E7pjT64o7wbYun9zQNpbdJCV/q/JSHAMIjKbDnnyu/DPFM+6sG6jGq+H89HbJxyUWAQx134eFemXAmUoJAXG4+NPkwUNlyXBgD98GmlwiIjQkVau9g0ZUiWl67HXQSWt97gX0T0jxQ7xNBq6NvejBHpvEHsBdC6g/A67bNt2hnzJxP0VxixCVlVaaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W+2R26l/jbQHknYkilwgK0LZFItVSlSZ+UXGymf9uJM=; b=mwiylNCkOluAyc7mFLaXHCNE6yudxQXmxgceChvA/3pvI+MLBrxFFa07HxNFkKyj4QqHEDLFRQRoPxMKarkoV/nSAHqpj8GKATXusVztLbDgc1sqib5LOYjENO0WaukhcsPmzGpmn/qkMkTA0QmT92//DPmrzJO2/cAxGZAPtD+eE2v+pngBbm36sGMa2ommFzPzt1hZohMHUbwmpw9YqAK6odD9RfOcHkxpNaslpU6rtHbvtOEu0w37+EPVPwCA3AV8qnNXLe+qPLg0fdsg9MsLPezwWWHX9/Vrdttog7McBZ4cWpnQlpMGuY7KQbtBbuX4Dpp22xUGcaIrhsfZ2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W+2R26l/jbQHknYkilwgK0LZFItVSlSZ+UXGymf9uJM=; b=KkouIs0/g7IFiU9V2G0ermIa92u/o1taKxlwn3fYe5fssq/QFb4EER5hJLbspirUoEhp5BNCOmxW1dUV+C8EdXS8wNjSjhkLCfUM+GYYS2KlSL/1q37FX0Z7E1uEjlxUzjiA0dfCEkd6HQOfwplCsHzcZD10Beuyd3pm9HVkG7g= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by BLAPR10MB5265.namprd10.prod.outlook.com (2603:10b6:208:325::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:28:02 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:28:02 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 42/43] shmem: reduce time holding xa_lock when inserting pages Date: Tue, 30 Mar 2021 14:36:17 -0700 Message-Id: <1617140178-8773-43-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:27:58 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2968816c-fbad-40c1-a7d9-08d8f3c2b30b X-MS-TrafficTypeDiagnostic: BLAPR10MB5265: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Zl3A8O/dAkiKli2SgzMfqMMmurty71GgTXHimiE2DEJ0BFWLBlCdAieDS93Bl7vmtzVRVXIjJdeySlQxztUdhhldIuePDdvleQO+Ee7OQYCYIqypqMHH1B0iYr3eJmrQCvxwlsKrM8L/5iYjXAR/TOI91ufcrskBoRFZ1apynPOP2I1h8W5D185telCFzMp19/K/G81KKmCKajCp+cyriOUJ2YKxAjh6H6EbI5+zmMSMaW1qbu7Irni4ZKAWdkxxQK1V0T51Te8KiUjtxM/CMQWfMsSM6CHQd+IEkNXI1E6LwtO5JdBF0fV/rhav3bJaWEiuiDLV7PUtKvFawkYahASV0caJwDec71NDhZ+TbpKKJH5+4FIV3/pGHBxNs6XbM+/d1QPJVU/jIgTvELNZjYlTh+aAFxNr2kQj25thZGwjEJOreLDJEaEOUQtFy1DAKetpMCG2x1KTXbpQRBBuxxCYMm+ATmT2Hg7OISyLFCtbst8p44N3wDUWR/U/tMDCd3cuGECk2meZ2QFSV3NnmCzgqNjhhLNZtPai5ZXuRIY2gVBT9bSVXm0vQTJ9wPQPhZV7jmVAdF6iUzk13x7TGIimDn2IzIaaCZ+M2Qxbj3fQNnZvOAPtiV8QgRG4WQflNvsM0qdRLrWOJ+lWrnxXyA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(346002)(376002)(39860400002)(26005)(956004)(52116002)(186003)(38100700001)(36756003)(4326008)(8676002)(44832011)(2906002)(316002)(66476007)(2616005)(7696005)(478600001)(8936002)(86362001)(6666004)(83380400001)(7416002)(66946007)(5660300002)(16526019)(6486002)(7406005)(66556008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: U1uXoOr8ojdDFkag3sklQ3PfJ7XE/YT9Ywts+DuiLeNaKxbDFcJbceeqwMYJT/8FWJA5MJ0fhuRUdTFY7RfW/GnGsFYfXXNLEoVDy6N9mSCvl+Y3FhJjMFYOe+w+V0e4fGamE+ewtnT8c6AGc5ZtxML7XWnXl6QZ1FwIZZY+JYfHqpEs6rDvrdFPBpaDEothgI6wiazZVJzSh9Cq7jtzwqcJxwFO0+6+jshmNod5UQGfvd6H1+YOKiLesJ3l0x/rnVcaOX5dC8t9j1xVBtTCprXlCBYQzwsFo0kjNz/p3FOpULe2twf9gr+84/zkEC3L3a2HibQ64LJhJUkKbKUNDs0xbb1DxwwVtaVdi0d0W5UY6y2CoXSL1EciMI1kcQHcUrD+Emi+2PlqqFhz9R7iqI94VcaA+2EkCqO4sukoeE2g1ZKamPq9CdiydLJgOVsRAqcS/TgQUSLod2Sz3QBirfJWFEVVkeGsdCeOqxRtQVp/h9pIo5cRlrEHVNtvgEI7fzCSQ1CyPSxhCF7+CrcLS4Ce/8etjw4Vu8c4oodw9zMcz/idqPEJlj7tae/j+QCM34nprisS/Gd9ntiE0zu2O54SlZvriD79mG89mBoajD7GyDH8kdwiIRwrKMxrc6VCnZsfNVFQ63fz83Xx5KOrVuyMUXuhoL9Pd34Q2hHKR+HEB4fsHUWU6i1Q/aEQ+Ph8mFXWg0x4QbQJp3ZO/iXmGydfIggJVQ/4Rv9H38cIRI4GYZ5wkiwbKK5lUQ9EAVPJYBq2mIXju1DkkDh26XH6iaeblrEN+sK9Lo8+VFfuTqt0MZVQjLQclag5+R3Ng12+YL1XRev9nTftSd0iZvTxP2OwoI1p/iTo3AyPwD5SI9nf7XXYUCjbuLgNdUxxo9EpmjlvhyOKgwRHe7EpYhlRA6KOqO44Kvn6jiIlevWBzqHy4PgVTSHcFtiJzDhNF2yYH5rwXWRB9dsLoJ0Ea/oh8C0nrrB5yUpmEMf9PsdeJzrw9866JSvoQ/sqlzbxqxSJXPqCn9pE9aCbh9cbl9KqYyxs08MIjKRpjo7tWgOoMR8SKPTg3P4K3nTvntGCxe1+A2wH55ktCxvOPJcdGSXddlpjkgKW6nZfImP8aLYbdQH1LW9D2ySZ52LdSbcnLnDLiXCQTwWSr/te+oCIApGRgcFbKkBZKUl08yCAmBpWvf43XaNnQgiWmpOUMHk5LaDSMv+UQW34O4TqCOgwi9HD6poTNBrdCtvPa3hx72CGi+y73OIXux6wk0IpCcEideg43DbevXtxf97RnC35VkBjZk8qUO1tFYtyNYP8tR8a85l4a7/8kSJl5vDcmWzqU22J X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2968816c-fbad-40c1-a7d9-08d8f3c2b30b X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:28:02.7614 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TvHTgmyMOGLTu8z6eBYEXfwjhYXVJyhn53hzcenuA6hEAbUnUeCixQ4PaVp08pdNSoXabtgVCeyF+Y1lScCOmqbZrldMpcbwUnBNYnsDlEo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5265 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: 8CgPjS41eBgvje_ZLWd54FkTPChxd2ux X-Proofpoint-GUID: 8CgPjS41eBgvje_ZLWd54FkTPChxd2ux X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Rather than adding one page at a time to the page cache and taking the page cache xarray lock each time, where possible add pages in bulk by first populating an xarray node outside of the page cache before taking the lock to insert it. When a group of pages to be inserted will fill an xarray node, add them to a local xarray, export the xarray node, and then take the lock on the page cache xarray and insert the node. Signed-off-by: Anthony Yznaga --- mm/shmem.c | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 156 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index f495af51042e..a7c23b43b57f 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -827,17 +827,149 @@ static void shmem_delete_from_page_cache(struct page *page, void *radswap) BUG_ON(error); } +static int shmem_add_aligned_to_page_cache(struct page *pages[], int npages, + struct address_space *mapping, + pgoff_t index, gfp_t gfp, int order, + struct mm_struct *charge_mm) +{ + int xa_shift = order + XA_CHUNK_SHIFT - (order % XA_CHUNK_SHIFT); + XA_STATE_ORDER(xas, &mapping->i_pages, index, xa_shift); + struct xarray xa_tmp; + /* + * Specify order so xas_create_range() only needs to be called once + * to allocate the entire range. This guarantees that xas_store() + * will not fail due to lack of memory. + * Specify index == 0 so the minimum necessary nodes are allocated. + */ + XA_STATE_ORDER(xas_tmp, &xa_tmp, 0, xa_shift); + unsigned long nr = 1UL << order; + struct xa_node *node; + int i, error; + + if (npages * nr != 1 << xa_shift) { + WARN_ONCE(1, "npages (%d) not aligned to xa_shift\n", npages); + return -EINVAL; + } + if (!IS_ALIGNED(index, 1 << xa_shift)) { + WARN_ONCE(1, "index (%lu) not aligned to xa_shift\n", index); + return -EINVAL; + } + + for (i = 0; i < npages; i++) { + bool skipcharge = page_memcg(pages[i]) ? true : false; + + VM_BUG_ON_PAGE(PageTail(pages[i]), pages[i]); + VM_BUG_ON_PAGE(!PageLocked(pages[i]), pages[i]); + VM_BUG_ON_PAGE(!PageSwapBacked(pages[i]), pages[i]); + + page_ref_add(pages[i], nr); + pages[i]->mapping = mapping; + pages[i]->index = index + (i * nr); + + if (!skipcharge && !PageSwapCache(pages[i])) { + error = mem_cgroup_charge(pages[i], charge_mm, gfp); + if (error) { + if (PageTransHuge(pages[i])) { + count_vm_event(THP_FILE_FALLBACK); + count_vm_event(THP_FILE_FALLBACK_CHARGE); + } + goto error; + } + } + cgroup_throttle_swaprate(pages[i], gfp); + } + + xa_init(&xa_tmp); + do { + xas_lock(&xas_tmp); + xas_create_range(&xas_tmp); + if (xas_error(&xas_tmp)) + goto unlock; + for (i = 0; i < npages; i++) { + int j = 0; +next: + xas_store(&xas_tmp, pages[i]); + if (++j < nr) { + xas_next(&xas_tmp); + goto next; + } + if (i < npages - 1) + xas_next(&xas_tmp); + } + xas_set_order(&xas_tmp, 0, xa_shift); + node = xas_export_node(&xas_tmp); +unlock: + xas_unlock(&xas_tmp); + } while (xas_nomem(&xas_tmp, gfp)); + + if (xas_error(&xas_tmp)) { + error = xas_error(&xas_tmp); + i = npages - 1; + goto error; + } + + do { + xas_lock_irq(&xas); + xas_import_node(&xas, node); + if (xas_error(&xas)) + goto unlock1; + mapping->nrpages += nr * npages; + xas_unlock(&xas); + for (i = 0; i < npages; i++) { + __mod_lruvec_page_state(pages[i], NR_FILE_PAGES, nr); + __mod_lruvec_page_state(pages[i], NR_SHMEM, nr); + if (PageTransHuge(pages[i])) { + count_vm_event(THP_FILE_ALLOC); + __inc_node_page_state(pages[i], NR_SHMEM_THPS); + } + } + local_irq_enable(); + break; +unlock1: + xas_unlock_irq(&xas); + } while (xas_nomem(&xas, gfp)); + + if (xas_error(&xas)) { + error = xas_error(&xas); + goto error; + } + + return 0; +error: + while (i != 0) { + pages[i]->mapping = NULL; + page_ref_sub(pages[i], nr); + i--; + } + return error; +} + static int shmem_add_pages_to_cache(struct page *pages[], int npages, struct address_space *mapping, pgoff_t start, gfp_t gfp, struct mm_struct *charge_mm) { pgoff_t index = start; - int i, err; + int i, j, err; i = 0; while (i < npages) { if (PageTransHuge(pages[i])) { + if (IS_ALIGNED(index, 4096) && i+8 <= npages) { + for (j = 1; j < 8; j++) { + if (!PageTransHuge(pages[i+j])) + break; + } + if (j == 8) { + err = shmem_add_aligned_to_page_cache(&pages[i], 8, mapping, index, gfp, HPAGE_PMD_ORDER, charge_mm); + if (err) + goto out_release; + index += HPAGE_PMD_NR * 8; + i += 8; + continue; + } + } + err = shmem_add_to_page_cache_fast(pages[i], mapping, index, gfp, charge_mm, page_memcg(pages[i]) ? true : false); if (err) goto out_release; @@ -846,11 +978,29 @@ static int shmem_add_pages_to_cache(struct page *pages[], int npages, continue; } - err = shmem_add_to_page_cache_fast(pages[i], mapping, index, gfp, charge_mm, page_memcg(pages[i]) ? true : false); - if (err) - goto out_release; - index++; - i++; + for (j = 1; i + j < npages; j++) { + if (PageTransHuge(pages[i + j])) + break; + } + + while (j > 0) { + if (IS_ALIGNED(index, 64) && j >= 64) { + err = shmem_add_aligned_to_page_cache(&pages[i], 64, mapping, index, gfp, 0, charge_mm); + if (err) + goto out_release; + index += 64; + i += 64; + j -= 64; + continue; + } + + err = shmem_add_to_page_cache_fast(pages[i], mapping, index, gfp, charge_mm, page_memcg(pages[i]) ? true : false); + if (err) + goto out_release; + index++; + i++; + j--; + } } return 0; From patchwork Tue Mar 30 21:36:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 12173885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7CF3C433E1 for ; Tue, 30 Mar 2021 21:30:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 972F861A21 for ; Tue, 30 Mar 2021 21:30:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232802AbhC3VaH (ORCPT ); Tue, 30 Mar 2021 17:30:07 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50808 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232636AbhC3V2y (ORCPT ); Tue, 30 Mar 2021 17:28:54 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULNs0k130344; Tue, 30 Mar 2021 21:28:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=DSGc7PTDZdYXBU2D0khCKJfHd3zVnsm62bMxQv2PtyI=; b=DYOujjE9TSqj28AzJ6onyeR4SlGuObEMBmWCb1PG+lvHDDMMPCnrt1kyRboxEMaBJWry p/A85dbbz6P7rwYFaaSUO3+QFsqwhWwFNhOYg9Y8GB7BiLecN33+L9jWJ76E7frcltM1 8R6diKMUcb2xw+3g7ayJ30q83wcu4JbYnYYIsuFanZx6Fth8p5BVqvOom/gcCwEqjCCu ftEdU9aOpr2lk+qcFC4bxJPwLcMtqSpuUZzuGG8TVKDs3NC7a2lzswRwyvyZAAwdECmb AY3Ud10ejOL81TRmFOETNIs+fFoxfvU377twTYSqh+L8PKcHzLtu7ONF4kpWXyjYKXne Bw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 37mabqr8s2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:28:10 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12ULPZb2124960; Tue, 30 Mar 2021 21:28:09 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2040.outbound.protection.outlook.com [104.47.66.40]) by aserp3030.oracle.com with ESMTP id 37mabnk8j3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 Mar 2021 21:28:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QJGecd626mc2Sg1N/nTh/TvHl8Zbkj77pXPvag5XTzHI+Aybh5FZsmbDS1Yl9G9Gf2ZvdX+8YDmYbNeK14OXk5yBehQKvKHkEvV6vgb+6z3yt2W2w+lQ3uSMMsSOEfaOoEfklBrPU0HEQYekXHZc2xCMMkPEHuV3g8KF8Akh7Ql3Xn9dVzsmZ8QI59Ht8TdCEXxJXgqk8nYpOwqg4eyCSR9eWpBL38pWzeFc3IybcvS/ptkOu2FATrcGLwdoSKnFyvzbno3HEkbzBeYXqmMqS3quZIbQ6KYNHLp9IrWHWE2PCzP2+njux36q1botXtEr9oWN4aT1KAuyeg+dLEvchg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DSGc7PTDZdYXBU2D0khCKJfHd3zVnsm62bMxQv2PtyI=; b=hkp49QV3QL1lPx5ZAIXu6tl02tRs9IXqUz8gmTpGaenQss4KrZsbb7utyXAG4dk33zFyqwN9w4P6gWb1gmL16779QuwA3u3q+y3Xa/kn2XMtrkuiGKlx7wVPjpji3i7eTJWyuSRtmWNaIeXH1nhjgV7NcDc6b/2TkJMSgxahMnDgi+FffbGl58UBGB5OYEIxOpOUEaoyqo1XK2XffgygSynykJLRGi8c3wv14tc5GMKz3+C+APTawFe5fLoVMttARUHIJ1nlT7PqIn3u17AAV3R0Nf5FwGQXtqAb8vZukPxUFQhaDBsRi5aEZbZDubiHMRxMvvlkcmoy0ahKXI0Daw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DSGc7PTDZdYXBU2D0khCKJfHd3zVnsm62bMxQv2PtyI=; b=zvE/kMrmF0LUlDTfZRycqNhPxiq/M398+re9ciBfNPKSeaiOoWCzhWRlxJz0qLZPG3KvMIzn+dh5wWVZ/m748/eZpkKpHmFU+whlR+9Z15ojjVmL0b5BrS4o5GGJmAMiWTg6iaEa/etfpubF9TgRYW4qtQ+vHlSj/ar1Fqxlf9Y= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) by BLAPR10MB5265.namprd10.prod.outlook.com (2603:10b6:208:325::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26; Tue, 30 Mar 2021 21:28:07 +0000 Received: from MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254]) by MN2PR10MB3533.namprd10.prod.outlook.com ([fe80::cc79:c40e:430f:e254%4]) with mapi id 15.20.3977.033; Tue, 30 Mar 2021 21:28:07 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, keescook@chromium.org, ardb@kernel.org, nivedita@alum.mit.edu, jroedel@suse.de, masahiroy@kernel.org, nathan@kernel.org, terrelln@fb.com, vincenzo.frascino@arm.com, martin.b.radev@gmail.com, andreyknvl@google.com, daniel.kiper@oracle.com, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, bhe@redhat.com, rminnich@gmail.com, ashish.kalra@amd.com, guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, iamjoonsoo.kim@lge.com, vbabka@suse.cz, alex.shi@linux.alibaba.com, david@redhat.com, richard.weiyang@gmail.com, vdavydov.dev@gmail.com, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, daniel.m.jordan@oracle.com, steven.sistare@oracle.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC v2 43/43] PKRAM: improve index alignment of pkram_link entries Date: Tue, 30 Mar 2021 14:36:18 -0700 Message-Id: <1617140178-8773-44-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> References: <1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) To MN2PR10MB3533.namprd10.prod.outlook.com (2603:10b6:208:118::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ca-qasparc-x86-2.us.oracle.com (148.87.23.8) by BYAPR11CA0099.namprd11.prod.outlook.com (2603:10b6:a03:f4::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Tue, 30 Mar 2021 21:28:02 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a0a8d607-5eee-4e19-2be3-08d8f3c2b59a X-MS-TrafficTypeDiagnostic: BLAPR10MB5265: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2043; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: U/QQ2mFjJdy9NMFDoLwwnFyudM5JzqRl07czgKAZQgitQ1g/RTdMQvK3Q02wZriGnUgfDkeAGP4WVCSBQQ2+kmzSG1umRHYOJ9Nz+AgzQpza/ylxtOoVIgIfmjHfA4KUKia0z3i/yO0c+2WZsmIyxlF1YPFZfj0B5oRpUx/FLU/V2SuPvSh6mYCSuV5X9RP6S4tPLNdnsR7xLb/iwkxkI0cPPvXQGZXDfAd3RYDN2KMDofqH/zDXluphojrDdVlVl7V+418d+VqGjdNoBaGGjihSScDgFrot33QJNl/qgEmJZmam7aOK51zZHXDGMyISzS84PmH5XQgdWBR/bXPN13zKCfjduwhkEpE7+UgyF4Ftpk9JW6tesBGPHK8fHxB3j4ozoNEBxWpsqsXQlIrsxF/DmXjOk7EHm+UJIwSnXlSNFcjXMoUu0ZNphefRTrpg6Rmc54LLzrFhAAI/KiqZhm9pu8Yw31Cse7er8/yD+oBHfOaaYZ6hJFppwUrVUe6nTj7e/LaLGrPzDYOqN3nUgnRN/eHISHufsqcoKbg6secCPwer2Ceardgie42/iuvl1k2wXy2bEHhRan00g7mOFttj/j3W/XbSCXGDsZTN3FX9GGai8om99LnPcJnVlajzFRu6w0IRLJsYKJ6iq6+Kpw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB3533.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(346002)(376002)(39860400002)(26005)(956004)(52116002)(186003)(38100700001)(36756003)(4326008)(8676002)(44832011)(2906002)(316002)(66476007)(2616005)(7696005)(478600001)(8936002)(86362001)(6666004)(83380400001)(7416002)(66946007)(5660300002)(16526019)(6486002)(7406005)(66556008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: qeYOuW09fv9SPRyrmJk0JkLH6S5er7zJmNH7MqnS7oQKgKOdy1GiSjDpzJilIjUaBJyjIWf03Lhs2+QpRWhvM3ZRo2DTxG/IKPx8MKEwIsTwIpkTLEJidiS58PFoQYrnIt/4LhAjj29U7JeilHWGTS7eTxjfPB2cxn2qRZM9IKxonVMuOyIeP+1sjcjx/5qkLKxjzAqfA+2o97NJ2oyNizxKcIhrdrtxuIcHJja5dghl0ic1MM8D/7aYDinWoRVAPjGRev/g1jaw9tkr6NUvFmJrPiAs22wQ0kLkwFOA9YzAimHD7SjFu5dLNn5pE1Gj5EMb+3ZGXvGeL5qObkKI0WSKFmRWZ5Mhg08jMksUkc9nB1jXnATNq+qssEHME882ropMSn010Ba/Pd7xgYxiIwyliCH2z2R8cfzQ1EpEk5GwbgNcRAgJC42sPDBXE7m0BTR9L9f7WIx1emqRWjXDi4H8gcmUCO+X9ckudYNXD+UzkK7SEbwPgEOpjPbC/XkhDzSlQifYroPbHhikJZLPW6unQGwVIOhl7DkFNg3Eq7RMGG3g7tdbI7d2bD8Ov8fL+nIegG9p+qteaNAENsOJf6DCtHTFI6gwR262Y0+7URPxBMqKvYr1J9WSiOm4jrAswQrpMnP++65aTiW9mM6va1UxCjAVmoL8girVjxGesRv5K8Oit9drUiugd01J6OO5yCvF2Ds6MZSaDzwdo5KVc0kaWHFUv2r4WAunAL9A+Sz/a2SQnVdyazxLGylPZej4QgsrJ+gTS48vOZaSOLYdgnhKrtRCesFqc8O6PQCYtSXAb7k3KZlC5yLAqHC/gfMdbtsz2uRY5q6adnmuA9zp0eztqjpBcIzeSKdTI76TaZze8PI0C6t0rD/F1SYVtpBRbvUADtOzGdNWgXWE3MQAlKFEoSiviKEiVC8u5bjaXL6/OVq/T2IVUZRli8R5NRYYnGCFgkHbM6/CzGH5VusRwtnACq9jrzJtpWFzPh5jaMHQnu4UrVUaW+X/S6zqzIsjHbw4uUg0dGY4+ZLjt5TGmg1h32o31S6etICxBVlIVWebOdnfkYidWAJ1XrJNkMEizfjfaTHvYB31ipfshMQZ7h1zcpBYRgRNBUjrFeZ/KlKEtFZl2amVd6s12wxMjefvSnlZNsUw+iNAj9ew0robRTQsVQ9D071Pwe9RoUHePhHZ0ypuX7NmZWBiAx8i5ri+JPJNNL8SKV0TakkUYQKk6oy5DuOpxBYGlvIiF+nPc5TtsnsN6nvNXHHtY2K345afOfdl3NevBuWIyjZtCQiB4QP6NqMTPVJwYfZ7z5p2Qg/SXs8l/UL9idT7NZtenNoP X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a0a8d607-5eee-4e19-2be3-08d8f3c2b59a X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB3533.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2021 21:28:07.0639 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6c0Vc660uRerXcRqD49W4MZvBtUBknhiafKQGZhr51i/42P1Fr3OD+WCniVjMBm3Z0Q5Je8aV5rdCYNEDbXVU2rK+ISd2XsKTU4EWpv4OLs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5265 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 X-Proofpoint-ORIG-GUID: BVK1iEZHg5ovKmvclLpOhn-Rb5_GIDS1 X-Proofpoint-GUID: BVK1iEZHg5ovKmvclLpOhn-Rb5_GIDS1 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9939 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103300000 definitions=main-2103300156 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org To take advantage of optimizations when adding pages to the page cache via shmem_insert_pages(), improve the likelihood that the pages array passed to shmem_insert_pages() starts on an aligned index. Do this when preserving pages by starting a new pkram_link page when the current page is aligned and the next aligned page will not fit on the pkram_link page. Signed-off-by: Anthony Yznaga --- mm/pkram.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/mm/pkram.c b/mm/pkram.c index b63b2a3958e7..3f43809c8a85 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -911,9 +911,20 @@ static int __pkram_save_page(struct pkram_access *pa, struct page *page, { struct pkram_data_stream *pds = &pa->pds; struct pkram_link *link = pds->link; + int align, align_cnt; + + if (PageTransHuge(page)) { + align = 1 << (HPAGE_PMD_ORDER + XA_CHUNK_SHIFT - (HPAGE_PMD_ORDER % XA_CHUNK_SHIFT)); + align_cnt = align >> HPAGE_PMD_ORDER; + } else { + align = XA_CHUNK_SIZE; + align_cnt = XA_CHUNK_SIZE; + } if (!link || pds->entry_idx >= PKRAM_LINK_ENTRIES_MAX || - index != pa->pages.next_index) { + index != pa->pages.next_index || + (IS_ALIGNED(index, align) && + (pds->entry_idx + align_cnt > PKRAM_LINK_ENTRIES_MAX))) { link = pkram_new_link(pds, pa->ps->gfp_mask); if (!link) return -ENOMEM;