From patchwork Sat Mar 19 06:28:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12786092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D06DC433FE for ; Sat, 19 Mar 2022 06:30:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242325AbiCSGbz (ORCPT ); Sat, 19 Mar 2022 02:31:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242320AbiCSGbh (ORCPT ); Sat, 19 Mar 2022 02:31:37 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EC1F2D5A38; Fri, 18 Mar 2022 23:30:05 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 22J1hWKI001709; Sat, 19 Mar 2022 06:29:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=nSEyfhezoAwlNyvhBjZH9FdND1rQ/6a4aYmOeSPmj0s=; b=SwObJ1fvIhv+6LgWZHxU+5jGRB8hjLHF6yMBxVss7WaMsSVRaVDu9Qhfu2YfLtmYcuOM 0mFV6kJBhs7ALaK4YXvuL7ylsqngyiUGY3Bb94OEZlQXYgzDF/zZti3zUk/kJnuY0l4i wKXhgK37RHdaE2kDqnrMosMMHRA0exGg+kXV/1eCD+Vb4m76O7PwRsnL6J8TO4tKCH5U hM1JdIR42fnsJX6wCWLrXxfbdrLm4RZ1houIEudGCK05zDe2uNZTCAVj9ETVpulWqDJx ZgzVmalH8J1XcYa7XM5Wk9WkrFzvnDitWay+HRAdEBvtJlcMgDgBf61JrXp+5mfnPzZN 7w== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3ew5s0g5k1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:10 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 22J6MkAJ028031; Sat, 19 Mar 2022 06:29:08 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2044.outbound.protection.outlook.com [104.47.56.44]) by userp3020.oracle.com with ESMTP id 3ew8mfrwjn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FftQFY9KhE2S8hSfnqZNoWS3LWFO/9Erx5IxGPJ3NszF8tsa+qvuXEdJj+AUGkQ0v3diIxelFENbCKRx8qoOFxcLPxQ9egv9suL8+Eqlhvc+Afw12PKjjmJWJGNW0f2BNadEV6Nq1XVYZY9ojl1fhGFzVyt1HgMeAh0UcEQYb2g8wczgOh7nj9kN6r+JMs9pRNM0Nlhpux2BL/UeTFU2JakfUMCJDRVHzufeTd19myXFK0tsFTdokJ3J4Bo4OwOR0LdfsUMlpqxvL3WfobPLy8dxidNOMTJh2nWYrnePySK3HkywK8XsIiqRbmBvDqI2ZubggBGmT6WQq9qU8aX8Kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nSEyfhezoAwlNyvhBjZH9FdND1rQ/6a4aYmOeSPmj0s=; b=U8spj0KAvtL1Kfkn3qnZn+M1GeKkjM6+Eyl232YykQ+wct/RTuYg7zxugGCr3Q0djAjnqVXT7ZXE9jJT6bDe42PZKJjf94O/iiWoBQELPOE+rgZ9c9A9oPiCZXcIVP3OtNY5fj0HF1FSkJYcjz2qFP8I3DhGCv3rsVcus0FM4w3++4DbLL+gJ8rBxEMNnsejFZrXK/cGXteivzG9Lb+SPxjkYbXUHxR1/0cQx59c/j6cTHsOM36rMVJzR1esC52umVeRUgW/LaRyOz/1HimcyFnORFqGMSukGakSbKoxxiV/zvgjUKBGuCAyqtW+HQLQRuibCor9eSDZejBT3Rs0gw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nSEyfhezoAwlNyvhBjZH9FdND1rQ/6a4aYmOeSPmj0s=; b=kjZGA1/Yz9512345bpkexLBeqv5W/HEw2Ot+dTL4jly97qkBlfMwWal84G//w4n2aQ+l2TAtY9nOW7KD5kMoL++8tBxUh/hLXUQt810MtSl9+8TAqPWzfblhUMh22ZfMdKIcbY10nRWlfmUoQTmlx2KF2maC1/WoO7r2S1J1mdc= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by CY4PR10MB1798.namprd10.prod.outlook.com (2603:10b6:903:126::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.17; Sat, 19 Mar 2022 06:29:06 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336%8]) with mapi id 15.20.5081.019; Sat, 19 Mar 2022 06:29:06 +0000 From: Jane Chu To: david@fromorbit.com, djwong@kernel.org, dan.j.williams@intel.com, hch@infradead.org, vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org Subject: [PATCH v6 1/6] x86/mm: fix comment Date: Sat, 19 Mar 2022 00:28:28 -0600 Message-Id: <20220319062833.3136528-2-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220319062833.3136528-1-jane.chu@oracle.com> References: <20220319062833.3136528-1-jane.chu@oracle.com> X-ClientProxiedBy: BY5PR16CA0025.namprd16.prod.outlook.com (2603:10b6:a03:1a0::38) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b9caddc3-7e76-46d1-809e-08da0971c4bb X-MS-TrafficTypeDiagnostic: CY4PR10MB1798:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N5bGlc+erw/EVNfrGAuQsEybCZ3gcdK69SDlfloAagjkZnFIrVgrcGv2mLxIvuuwOaica3k2TVXwvQ2oKomCKSKdhiHfQydvQ4caQ7ZtT1tS/1X3q8EtBMGHAOiwYahakTJNhBLb+NDRRVJi/Uzccm0THsE4mub5enIDM3ntXv58bSxyE8pMY7MBnG343x42zbTAXlqBwYSdb4B7faucg3OlGW0xIULRb/pGkvUmyKRLETck4fzX6xO/vRtKBxngEDux2gxeckUa2WVCheWXDIK1oUwfhAXVM0mA5hJaxOIvjQzyx2I2mLtSXixzdN4EGXDkOWlpXy7Q1pO3e4v6iZKSJ5LO9zN0GsxtUnVMxtdCXm68RCj/FLFYOSjawgx7ZUnxVBCedybNo5/ZpOpCTUoNxlmqokFbal4slS0aN9zRl3wPgAw/Xwl08+O42iPw3yLDIeiIQM/d+x/TCbnS2eyvwkPgcOB4LoOX9hL49NY3LUC0eWBVgggDOt3HgMeSQ+PmdkuFO7IHW6/P2+F20JMcR+evlmIvFDdGkAMiy9w+3/2r3y/wxI2btBs0jLnkLWYOmGK2B/iWNuopNDKESet9zwzG1PecVcxxSPcgNjKe/DTeHl9rMD58XifusdGli/GWmLvwDcsjaXMvecK0ZAjmDi3YSsAaJMqEBcqaE/HmBoTVngA2FHSiIbBNMhRg9Wsd8fMJtPc09WS4J17CIg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(52116002)(6512007)(66476007)(8676002)(83380400001)(66556008)(316002)(6506007)(2906002)(38100700002)(66946007)(508600001)(44832011)(5660300002)(8936002)(7416002)(4744005)(36756003)(1076003)(6486002)(86362001)(186003)(2616005)(921005)(6666004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mwYwvJZo19hXF07BhyUXxGivvBuBW0S+2vhfW2Gzn1c48V+6gu3IFpzhbk+aen3Af7rJQgq+IszkMR4LTUfllaILZWyZNEDC+W0nTS9tucHPU7Sybh9RJqE/cFPULaVDDromGdJ6jf3bA6vGgrpavB5Cc6hHu+fokfNBTI27GH/D6qmTfpc/1TfHbj3o3plTtiDQxg3pKs+IQM/w31QCSA1rrgqmt6OtUXeLVlUvlln1a9OLGSuSsm9H3jYCp0JgqRrXJDfmqIHQxLxM+VyQ6wh1tHzmqpb8FYh+/EfgwPRIGB6bzDSn4YBZFww5YLY89Suv93pmfmpBmoXPadsDtmT4BRqKwgZBxkj4k1yQy0MIMAys8UzneQNYKiz+fP8F8mFvgysAuMQVsCXTmhN4B/7KinjekED3nv+yrktPNImC9BBjByTnoi125mtwEseei2XIehBKi6Ifrb60Ex3VDUVdU9n0CeWQVKM6V0aXrLvq0MT3hIlD6Ofl6HVqTW74ig85RpY2YQHE0w3ghA+TW/BIVPreCQz1sG43pZ92eu0ZNpdjq99nfQ1vRy3uf5s4ABdPH5kZonWGO9XEkbKnAS53VziJdkULsWTbGhm1c7MUyMWaxDDBYIA3RfEP9QuSvo36XxtScskxzh7Xr3/LvpkJOPi1T8kzE0gZBB4Iq/7CBjNKpTNqjwNChwHIHLs6KHs+JXT298fyJqy9G/JaAJPTRhO1Hdyy7pR3pfDA9dcDZU8k0E36VjfNcnrHZ/Cjr6RHJtY6Ffjv6PR9Jko8qR4PzBHMNQ0UHHVhx/l36b6nPVZRgJSVaKkzZAc5PlSISFCFNT8LvfAC7JQe2GCXawZGYtkm3R/ZaB+URoN+4/rypJ9T6lqmWlvBtM9babLfFXUl/Fw05OojyP7hnrHWig6RB78MprVL3Pb9hA2xOJ3g7A/yLWHyN5JhXU2AGGZ4jK1t5XNF2rT9xvKIpjDfwb+qVidAa6pRTZAROvlcSjcGJcI/90RLW9NhAcXeiBQM+ALB+myyYD0WNHBkQc7/r+IbGtiMEQDLDgwKoy8ZnQu8s4s+CVTKMcPE+K86pkbGXce5e81BtH1swhkn7CY602WHba4jm0puWEjiYrT7KQ2ix7u1kopHhs7/+/E8d2aNZ5i6bFphJav9vG1D5m7UTIHwlQZGptTdaKb0IBO7p0+kKOkkWGdeZm6CptGr481KR8C0dfmJ/bSM99+G8YTzVAuQ4AD3uOvjq89EkPDYOnQPpSA0UdUvW5aDHnSC9CZNeSzZN2nMZ/TBSHR8JClMMXpiRHQh91TMt5STMi4LaeLR6tsH3D2rfiXgznSn/bMK4IWBesrkjHcdDyrkYKCigpQKmsMj7RYjufg4oSsE62T8GcGmWAPymAd5EMj8rk0A7wi3XcpzFf+QD5ten1zai35Ixaf7/CnwVcGgYJZ1qy+HNzeYKlVIIrKn8Il1t3ouBeYa4OA3df3hnMJRYcVMNUo/toqb3519Qq1Oz+d4CeI= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b9caddc3-7e76-46d1-809e-08da0971c4bb X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2022 06:29:06.6495 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bQQsYdQeD6pUtDnOf4J+sI3axJsdWuvKFK8pzJTMVHu4WgQyWozFWqfJla0inFnkA2sUTPJS9dzo31vfLsuQSw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR10MB1798 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10290 signatures=694221 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0 malwarescore=0 mlxlogscore=826 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2203190038 X-Proofpoint-GUID: ilkLvOq1iKrCtLQGpAotQwpPApzqyRw0 X-Proofpoint-ORIG-GUID: ilkLvOq1iKrCtLQGpAotQwpPApzqyRw0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org There is no _set_memory_prot internal helper, while coming across the code, might as well fix the comment. Signed-off-by: Jane Chu Reviewed-by: Christoph Hellwig --- arch/x86/mm/pat/set_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index b4072115c8ef..042cfac6272b 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -1816,7 +1816,7 @@ static inline int cpa_clear_pages_array(struct page **pages, int numpages, } /* - * _set_memory_prot is an internal helper for callers that have been passed + * __set_memory_prot is an internal helper for callers that have been passed * a pgprot_t value from upper layers and a reservation has already been taken. * If you want to set the pgprot to a specific page protocol, use the * set_memory_xx() functions. From patchwork Sat Mar 19 06:28:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12786091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05544C433EF for ; Sat, 19 Mar 2022 06:30:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242331AbiCSGbj (ORCPT ); Sat, 19 Mar 2022 02:31:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242293AbiCSGbX (ORCPT ); Sat, 19 Mar 2022 02:31:23 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7DC11AA8EB; Fri, 18 Mar 2022 23:30:02 -0700 (PDT) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 22J3vT3G008826; Sat, 19 Mar 2022 06:29:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=WQKHDFQRLfFyRpEZMEyBctbWgbBoX4FGjzPfRUszb30=; b=t1R1skXFl0/py/dFvPrVZIuN/L88yN/JhbV11zNNKItaHWe3d2xVKxJw+kJJe3klFSd3 DU6LbVFHv/ov8urretgjBcjxySjeJf/1ya6MGugutAF5M39MEp2XzNSAFnj1FeDdbioD Iz2PxgfvZD5NfKTCFdSdNON4O49agES8F67p5y4nceuNZRnQisfkl0X7GRvH19yZtlQK NHK6r7pd/fdBxf0/FrU9nH3rUnrSu3zb3jS8r6dSeMw+pcANjlRM7tzQiR9HMriLuEhl t1zbZLV5K69NDA/1AfOPiaMdVbZoOJsKKxPQOyQbOLH9fH5JRnLE3phGkZXS0G8D0HEM fA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80] (may be forged)) by mx0b-00069f02.pphosted.com with ESMTP id 3ew7qt0354-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:14 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 22J6NKgK011462; Sat, 19 Mar 2022 06:29:13 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2045.outbound.protection.outlook.com [104.47.56.45]) by userp3030.oracle.com with ESMTP id 3ew49r0aqy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Jc/JfKrIre9Fz+46SPaOPsf1VDhrGsVne8S4b6ECJ8ANbdrfDk9d+fZBCYlIZh5iBFTmWbtjNSxE8Qi0UAPgaw2TLKG87Y6JRBroWvsykpsUZglrLPoNIQj7VfSmxuaZ4cuEmZLG2rRxaJWUcveOixPv/SjurnxaPNfjzzfXtXRj59MZ6F+oBhm4ISVCAuhy5S8mjsJR/OxZx4JJTf5A68JhX6Yj9bCYmNR90G611qT+c8bSgxbiVkbU4/MV6KPMfTTJb5j8Ertg0JMSslai4lXzX7qawDp5UlZjMkyGTSzGIDFIoe6ihhjgDWr8zLhu5n2QcNkV23odwueD4wkuBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WQKHDFQRLfFyRpEZMEyBctbWgbBoX4FGjzPfRUszb30=; b=oYxPXSa/gF4hbSgvzO+mTgPZnQsU7sviNHA01EF2HaUdpyYAkk2mYfTsRYW90yGLBpSIXcLFAQLEM+jJ+oyT3FpFrBfXyqCVn+d4HY2EMsybYvzJOsLVXDcxNZo8R5/phiAIEtkmlnwOpPtg52R/FLOQt+eFw+LG/2/lkwuWRgWzQHB8LXBbtcBWLq5wsZl2shYX+gyJ+ElP5K1eMUFAKUQgTwT3TveOmlOTCrwVaumILD6hXZhgEeKWJZv6p06jzmZyTPRa9eg0qVQEyQfrl5VVDy6AMyPm8Vz0wvDHCuyJHRJLuejr96tMx1ibm/kmpBpvlgx+vE1hZOPawFQSUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WQKHDFQRLfFyRpEZMEyBctbWgbBoX4FGjzPfRUszb30=; b=jOkggDncy4ABO/2FGl/PThoP2QZOiitPoYj+l58y2VUrNvvjoA3m/GBrybgDZz39dmIA1fHprCWxH/hGxBecZE63lzKGTKIwNGhMnfmej8Filk/5ZySL4i1j6/W24OoTm1kPS6fGMSA7rzMl65KLYzmideML9Cl2HkuwN8aTDDs= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by CY4PR10MB1798.namprd10.prod.outlook.com (2603:10b6:903:126::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.17; Sat, 19 Mar 2022 06:29:11 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336%8]) with mapi id 15.20.5081.019; Sat, 19 Mar 2022 06:29:11 +0000 From: Jane Chu To: david@fromorbit.com, djwong@kernel.org, dan.j.williams@intel.com, hch@infradead.org, vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org Subject: [PATCH v6 2/6] x86/mce: relocate set{clear}_mce_nospec() functions Date: Sat, 19 Mar 2022 00:28:29 -0600 Message-Id: <20220319062833.3136528-3-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220319062833.3136528-1-jane.chu@oracle.com> References: <20220319062833.3136528-1-jane.chu@oracle.com> X-ClientProxiedBy: BY5PR16CA0025.namprd16.prod.outlook.com (2603:10b6:a03:1a0::38) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6e4c18c2-78bc-4be2-def2-08da0971c786 X-MS-TrafficTypeDiagnostic: CY4PR10MB1798:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dJzPb5HF6sq8LUjs1TWk9/2M/m32buQlX4rTuFYsjI+StLR6OVnNA26XiwQs9kf0+l6Hn2R8HbZHvJnndfhY3BFGdaHfyHG941TFlezBvqsHMN/wBw3pwOqXy/+SYBMfGftbYV8Ltrn3rn+h1bz/zY3qbAFhtck+6Q9aakaZiHdbDId5rMhBGHvOb8GEcPrwP5eLVkvPFIkY6c6i3SojPboJUZV62NyEGfVD2OJbZbXl4oSc27u5kLFT9GsMJUWoIyFI1t9glpBRwKBmx2L9uVy110sEX8cCd2hCTp4l1kUZtNaAT4oBjqz4SH2nUOxbnT5ibZMCxjEWXPVtdiXInJPX8OiQWD0nJF2DI8HCEapB13/zhbpI2vQK01Lkh2gKGGFkV4Io9ErC0sGnHvRrH/AfBUWOK9+QJrA+lJ4wcaNgpth4IWMoZ6/iXxTU+RODm580relGjEYyWOFDp2cWtqGvj/9bXZzHSgNpX+w2QqW+T/rYPqXP6wl8a0b+uQ46LQVw5GIwnPImP+PQAqQz4vxk5KcnyfCMKxVeHxxF9WX2wRna2o1UrUaiMtTQC8yAlc67RD3XcAxL7yW+MZqSOkPpeIflFg4IZ2RJ8IR6OtYMQEuKLMnYEVN5J56DfiD1ZCmlJOlyGrjt6FvOVfsxz5V3mRhutg3ih4/zvGzN+lzcaxM0KdELNOIMeljbYo0CvbnrS0nAwTDBSYYa3USsmQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(52116002)(6512007)(66476007)(8676002)(83380400001)(66556008)(316002)(6506007)(2906002)(38100700002)(66946007)(508600001)(44832011)(5660300002)(8936002)(7416002)(36756003)(1076003)(6486002)(86362001)(186003)(2616005)(921005)(6666004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: PrMeU8AxzAbXdLKkzUKZ/8otCZ2GB72QbcGg53PtAuPzSPTOF368LZzh3Q2op210pajmrzz9wNlIrUZKzmc+R+TDGMnk4HKMJ2E+jIJ2lBUOLo6WdSCgU9Kjhg82GBmgxXxaKAe9lB5qMmQ1Rv36xTLoTuOqIxLu02ZBj8lenqE2DXGlYbNyuq/ApYQk5hjA24dJaEGadr3gSGK9z+MwQ2duhNSJXlrY0HuJiDFnEJRk4xMS/7GD2gEt3XZzjn2JLGbwCkGRkqRH6cZJinxxbVDx44nFNIg4qv/4AOdGnUQZHs4EFa+qlQz0yPz4y7bub5I/kxIzD3I85WgHJzsGXlUmpBACcKxM+nFMT3/IgEK3lSbN/kFygCrK+9G0PIROlwO4DmgPlr27L3FNcm4aAeQEwP68eTfk29iZf++oN9xErVpueMriQlv5HO7I80D0o33ca3+K7/t7L/jtYxJIt1d/F4c+vto4W0N3JfLgabb2wEsvk31b5XL58h3aIPIO6oaNvey51Vfof5enqeg7mTheY8zQ65x9orQu2tzGcy3qYmo6gWt+RWurUYIlKc3WhkC9FpjFJn9QzsJfyDX/iLXsYyWljCJ8wcx51bdxaqSiq/ixGrdkXzrl+RKod3VPl7VZg46wHAoVNWqCTMsLCUmOY/OnZ7DqWyQ/klMzQxl3y6d8wyFJCu/i7pYNU5wD3swXf6ZUSvyHzYHelMURRis/t4Dk0rEDFMFYhHMWaT5tdfGqROrJFderoOa+2xt+Cao1www2cid9j7ExdYy7U6Vc9W2PpKQAmFiL4YIE98zq/mwtLdqt5CgQvOLN3Z12DEj52hWZCFn2XCi30ZIy0z/edX2ww3knPx0M1gGHkSNKzu9lBDkT9lK8f5MALPbUCLlZU80kTBTfcs0q5pBu0o/CgTxMRHkVVOKCdJkqbY25TuzGUJeU1Abi0kIRGs7J/3QkMgpAz0ozRr9hxjXJFDBJ/GHtsXcpHA+any+z8JDlPXWACtSV7/WkHjwbk3GcHfrXKiMBu5KiyqnIsgySEc06oja8pR2K+tisk1fBN8GkX9DC97mg22Md8gRbNiHtPfkz8vH5xqEqnn5t7QnRBStO4xdEMV73i4ZGEvcSlFsV5GOruhkaOFRX25ZAcJOXWbqzqp4p9EsFtztA93qD1jm86XHlKPNVprIFE4zYfc3C+HbAkjYTvUXnMerVABtsNuhFAz2n4wAg/sPk9ClauqlstyOYUZuk9HTmaNTrmRjqhX3yOsjlCG69uXnizJcgpA2oKKTO0ciNwXbDiiPvo/dLVLjLRD30+ihtRKkoZ6s+BQmMF+Z+bc+ANnwP/xj7ft7gW3BWzVVrPtHLgLE+EKMCuWEA7UwvlkiSMEWvw4juW17TXcrnSPBNsK1eFNAnE/Pb7KVi0Zcw9/axFNHr8iRJTyEtntCtiKmKNkr6TbYKKDHjmKxOY793OtOGM/UIJivXTIVrcausGnCqhKMPf2imhP6yTX86xdeePr/9a5Y= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6e4c18c2-78bc-4be2-def2-08da0971c786 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2022 06:29:11.1211 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C+ii/Utbdphpt5o61LBD6WW0BbzQpKItzZvYxtL4774C7KwqJY0FAScfZJADsdM0QKId5kbqoKlcMZH0CuOlmg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR10MB1798 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10290 signatures=694221 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxscore=0 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2203190038 X-Proofpoint-GUID: DE_shnLN9dgY0rLACOAvIoz8_7VtjjNW X-Proofpoint-ORIG-GUID: DE_shnLN9dgY0rLACOAvIoz8_7VtjjNW Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Relocate the twin mce functions to arch/x86/mm/pat/set_memory.c file where they belong. Signed-off-by: Jane Chu Reported-by: kernel test robot Reported-by: kernel test robot --- arch/x86/include/asm/set_memory.h | 52 ------------------------------- arch/x86/mm/pat/set_memory.c | 48 ++++++++++++++++++++++++++++ include/linux/set_memory.h | 9 +++--- 3 files changed, 53 insertions(+), 56 deletions(-) diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h index ff0f2d90338a..648be0bd20df 100644 --- a/arch/x86/include/asm/set_memory.h +++ b/arch/x86/include/asm/set_memory.h @@ -88,56 +88,4 @@ void notify_range_enc_status_changed(unsigned long vaddr, int npages, bool enc); extern int kernel_set_to_readonly; -#ifdef CONFIG_X86_64 -/* - * Prevent speculative access to the page by either unmapping - * it (if we do not require access to any part of the page) or - * marking it uncacheable (if we want to try to retrieve data - * from non-poisoned lines in the page). - */ -static inline int set_mce_nospec(unsigned long pfn, bool unmap) -{ - unsigned long decoy_addr; - int rc; - - /* SGX pages are not in the 1:1 map */ - if (arch_is_platform_page(pfn << PAGE_SHIFT)) - return 0; - /* - * We would like to just call: - * set_memory_XX((unsigned long)pfn_to_kaddr(pfn), 1); - * but doing that would radically increase the odds of a - * speculative access to the poison page because we'd have - * the virtual address of the kernel 1:1 mapping sitting - * around in registers. - * Instead we get tricky. We create a non-canonical address - * that looks just like the one we want, but has bit 63 flipped. - * This relies on set_memory_XX() properly sanitizing any __pa() - * results with __PHYSICAL_MASK or PTE_PFN_MASK. - */ - decoy_addr = (pfn << PAGE_SHIFT) + (PAGE_OFFSET ^ BIT(63)); - - if (unmap) - rc = set_memory_np(decoy_addr, 1); - else - rc = set_memory_uc(decoy_addr, 1); - if (rc) - pr_warn("Could not invalidate pfn=0x%lx from 1:1 map\n", pfn); - return rc; -} -#define set_mce_nospec set_mce_nospec - -/* Restore full speculative operation to the pfn. */ -static inline int clear_mce_nospec(unsigned long pfn) -{ - return set_memory_wb((unsigned long) pfn_to_kaddr(pfn), 1); -} -#define clear_mce_nospec clear_mce_nospec -#else -/* - * Few people would run a 32-bit kernel on a machine that supports - * recoverable errors because they have too much memory to boot 32-bit. - */ -#endif - #endif /* _ASM_X86_SET_MEMORY_H */ diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 042cfac6272b..9abc6077d768 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -1925,6 +1925,54 @@ int set_memory_wb(unsigned long addr, int numpages) } EXPORT_SYMBOL(set_memory_wb); +#ifdef CONFIG_X86_64 +/* + * Prevent speculative access to the page by either unmapping + * it (if we do not require access to any part of the page) or + * marking it uncacheable (if we want to try to retrieve data + * from non-poisoned lines in the page). + */ +int set_mce_nospec(unsigned long pfn, bool unmap) +{ + unsigned long decoy_addr; + int rc; + + /* SGX pages are not in the 1:1 map */ + if (arch_is_platform_page(pfn << PAGE_SHIFT)) + return 0; + /* + * We would like to just call: + * set_memory_XX((unsigned long)pfn_to_kaddr(pfn), 1); + * but doing that would radically increase the odds of a + * speculative access to the poison page because we'd have + * the virtual address of the kernel 1:1 mapping sitting + * around in registers. + * Instead we get tricky. We create a non-canonical address + * that looks just like the one we want, but has bit 63 flipped. + * This relies on set_memory_XX() properly sanitizing any __pa() + * results with __PHYSICAL_MASK or PTE_PFN_MASK. + */ + decoy_addr = (pfn << PAGE_SHIFT) + (PAGE_OFFSET ^ BIT(63)); + + if (unmap) + rc = set_memory_np(decoy_addr, 1); + else + rc = set_memory_uc(decoy_addr, 1); + if (rc) + pr_warn("Could not invalidate pfn=0x%lx from 1:1 map\n", pfn); + return rc; +} +EXPORT_SYMBOL(set_mce_nospec); + +/* Restore full speculative operation to the pfn. */ +int clear_mce_nospec(unsigned long pfn) +{ + return set_memory_wb((unsigned long) pfn_to_kaddr(pfn), 1); +} +EXPORT_SYMBOL(clear_mce_nospec); + +#endif + int set_memory_x(unsigned long addr, int numpages) { if (!(__supported_pte_mask & _PAGE_NX)) diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index f36be5166c19..d6263d7afb55 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -42,20 +42,21 @@ static inline bool can_set_direct_map(void) #endif #endif /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */ -#ifndef set_mce_nospec +#ifdef CONFIG_X86_64 +int set_mce_nospec(unsigned long pfn, bool unmap); +int clear_mce_nospec(unsigned long pfn); +#else static inline int set_mce_nospec(unsigned long pfn, bool unmap) { return 0; } -#endif - -#ifndef clear_mce_nospec static inline int clear_mce_nospec(unsigned long pfn) { return 0; } #endif + #ifndef CONFIG_ARCH_HAS_MEM_ENCRYPT static inline int set_memory_encrypted(unsigned long addr, int numpages) { From patchwork Sat Mar 19 06:28:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12786090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18E70C4332F for ; Sat, 19 Mar 2022 06:30:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242297AbiCSGbV (ORCPT ); Sat, 19 Mar 2022 02:31:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242285AbiCSGbR (ORCPT ); Sat, 19 Mar 2022 02:31:17 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F69E1AA8D7; Fri, 18 Mar 2022 23:29:57 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 22J1uTjd027608; Sat, 19 Mar 2022 06:29:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=8UcVsB4d7UZhFK1koBLr+fdQ5rM0pGrtEYfEFcXwq18=; b=O4tC+vKSBs2XHTtXRoR3EnqbyLb6laWyaNXdaOoHi+vEoHkx/0p/ysr9t0A2EznFqg+u HlMr60KME3r/bVh6h/b6yJ+uZEDsleombdTfVWk7x1htioQTBQHNRaaPkhADYxY2mOS8 PfyYvcLXUsmJ7q6z+FQLpzKgiaPCMVFhgMccswNK0BvRFlvZUl6Ws8fgIhuqcAbH8kAM H/oWiUVWERo6HmAK5nOq6cByKI6IPbcM6DSL70H8c5IrGFRRQZKhAwxseaiylWQDuRHH I1JMjnJHl0bdLciUDl3WlD+nVLKil81balmdUMw3DJFFi7sodeawY7WtoEe5Ml5hkwvk 1Q== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3ew5y1r5c0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:20 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 22J6M8Ng027457; Sat, 19 Mar 2022 06:29:19 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2044.outbound.protection.outlook.com [104.47.56.44]) by userp3020.oracle.com with ESMTP id 3ew8mfrwkm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SxGdYzVp90hoTSAx7kLzWgpFOuRSWp1YANeRAFAv6egEDEUncwZdUdnp2Su1JVdkAuwC1dfdfGIxSyDWofauw7HJheMeMnlCa6Jr4n3tuR1QoTxDy5faPvFtfHxOFsJSdelTZ7TTKX1Cy1P5qsW0sP3jtmcO4AZTM3QH5tWLDtpqXi0d1ZL/+dGkzQZrMY2mo1eA6/KCzJ+XU/+PdDfRNF2Z5nJ89XagTF+VGZz/aTH6JnnSr/knal3CyxpKQs2BwIi07uwasXZVrhl09PbM/kwTmjPZFQA1Sw0OxFYPvk4lrBm5GVRByp/T66HbIEeB748C6gppwygK9BLYpXeC3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8UcVsB4d7UZhFK1koBLr+fdQ5rM0pGrtEYfEFcXwq18=; b=kDoLh3Xd988gw+yCFV0Z97OF9IqSL9g994EVBthNoa45elCOKOpVTSgZCJawIYIjqFSN7eFIr8hCRmL2RL/scK17QYJ2/W/CDdfgqXRgiVQyU4GfZ9ZjK+RIQX85pC9pZ2crMbSssfxtWuVv1Mt38FoKuP5eZ+eHeEcEyLmnAFpxyjz4SKppjzFOb1tZma1HZzMj8R1XzpqDZYIaLJ5rSWd+EP2k0WsjPUpUZOOhqeqCRkwW7Lvd90a6qg5LFok+7MWhsPW+FypsK5LN7vh9wHDD8x5A9AZp0uD3pWE6S+cAq7Li1kpzy+qlrdsaoxOmkpoET7Ga8WKvebV5tQvAyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8UcVsB4d7UZhFK1koBLr+fdQ5rM0pGrtEYfEFcXwq18=; b=kz4bYJp8CFNnCHPU4NAkJKu5npy6XeMdxA8WKRobD3UNbLQyWgG8hnsxk0zIpppSVVwQCISch4VXmB6UKDLAX5udkTtertqTm7EgrgNHJmTjW6W2AZa7o3zvzHp9uksElucazXqk5N57+ehC0yb41iHB3zWBsPQoJN446RvUpes= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by CY4PR10MB1798.namprd10.prod.outlook.com (2603:10b6:903:126::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.17; Sat, 19 Mar 2022 06:29:17 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336%8]) with mapi id 15.20.5081.019; Sat, 19 Mar 2022 06:29:17 +0000 From: Jane Chu To: david@fromorbit.com, djwong@kernel.org, dan.j.williams@intel.com, hch@infradead.org, vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org Subject: [PATCH v6 3/6] mce: fix set_mce_nospec to always unmap the whole page Date: Sat, 19 Mar 2022 00:28:30 -0600 Message-Id: <20220319062833.3136528-4-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220319062833.3136528-1-jane.chu@oracle.com> References: <20220319062833.3136528-1-jane.chu@oracle.com> X-ClientProxiedBy: BY5PR16CA0025.namprd16.prod.outlook.com (2603:10b6:a03:1a0::38) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cc0170ff-0df4-4825-82da-08da0971cadc X-MS-TrafficTypeDiagnostic: CY4PR10MB1798:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: boM44YyIcnKPVPXYOi4Cd3P99cPpPUxid68zP7VdyHHKBmR5Z1yuRfsE+uP0D9uwIDhR/eyHZsjcY600anM+OCG5JQOZnwHZjqrlwuafZjEifDzEWPbT1It2yHirLSbFPGoJYWRuxfhCLJENfhI2ENqxKfQZvon18yQkJsUuxPdDNVhTIKvCn+G5651oS0mZTVw4EEnD2gaT5l5P8I4zJ8MPuw7Mnc0FLMe48opVV00qdWZClLRw2OcFGMxp9q0x/TyxVUbxpyUtL7pT0hkkK9iSaGNcc4LUDGxsMvcnVk46rvVy8e6iCVHWYrCM3SohtWZocoEpI2N0pVsvdJQiTaAS4KCYbmmOk8asYkecUfJbNM7v+baVB8apRv6cbiULxFVK9qaoVDrgFj8vVgc55AdiIe8BaTo18JjoyHCTDOpjyIR6XWuyC7yq3NajVW6Hgj2tBgSMDRiFi6QjQk8NLYc1NTuVvrJSIkDHO7s0ZDMuDDeiQyxbUObcCwq9U6rpQkB1vp9xtM4wjXSlvkU/LLwWB1v6lnqckdboT8MTTjMK3gU7I9GqtE4gWySXzaZSXU35KW1oPCOF88WpfHAEbPzpnarv4fAFggG4UurUIj/uDCoz/HEqbifZfhFhbIo4LzzmdyKiNDW7+sP3C9iy+/8UxO74r2sLWClINmJCSkJZlZ36nRTxCEWKQO0+3qorjQyrzraCEO1/Wjuo/DGD6BqMxJyA1hfcHeJCTylE8NCwP/jqS7mps16RDDSls5IydeYEnVNeglDeXIUbCZ6uqw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(52116002)(6512007)(66476007)(8676002)(83380400001)(66556008)(316002)(6506007)(2906002)(38100700002)(66946007)(508600001)(44832011)(966005)(5660300002)(8936002)(7416002)(36756003)(1076003)(6486002)(86362001)(186003)(2616005)(921005)(6666004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: d6MmD1lFSZrIGH5rew5M6omIiM927Tu2Sr5Nt3h4u0zTy8MlERFPUZhdp/wFPjczuwW4ctCgySaesAyjA1xVvdpXoCBVD8wFuqD8Ok4EJBEjlJV4uhJ5hbQRgG+2LMOpH7OGgDJuYTsc5y9k4pIxEFES9p2NfZjkj9QwRgTIuRABUMgbtDf1jmqaKFoFRyV9zL5yhLaUezbAEMZLlvdUxvHJ5s5BP0Vq4ipyaM1+eK0NipsOnBu85a26iQ5Ei1fKivmd3vqSSPmOm1QqwaJQ9NM3mBBDr62FofPhdHWolf9j7SVVU5E0J3lKPrzoVmMLt2qT0L2W6cKWgD2BgNh4DpNyN+c7Ot+RWRdAkN1N7ukwXk1ETzYoPh2BfNf+9e3nONIJwu7HQCE+hAo+ln4W6Iiep4lKwQv/YzcLOOGgd7hLztK7xyRh4N8sXZ8yU/EdEdC2C1a5iEr3cI72v4+L24HawSpxwKDvoHGe+v4rNwNF5hGkyZ9dAsLBg+Zp3/VBwknlX8I4KsTG5r3skuZvIl/+WssXJgzkH0RBg8LdGjX6CxZyv22UrLVnDIlIdPIA4RTnWgz6PCeKbvDnZoPdOVjrhn2hvT6+fmxVfwEhKAz1qCfaQhTjTuweUw97p2ltvP1qYF711odEdNPqBvE6EmBM9mqd2PvjARNd8Kbz8re7Ehkdw/jNZ7gCbauiJDgXjDSKmdfMppaRsrFoTuCCk7xV2N7vaJMzXvw8lHXX4FoijMqvHASu8mRn8UTMbQaC1UvSKL+gIgSbDw2bbd2C6yLW97fNMnpqp69H1Z7P/4wmDdB1ufYqBt5AUH/gdMyUdb6bhQgq+wINbIVLOOva8Uho+pZez0b9vL6dWq6kJ6Vwl8mV+MCSVPn6MzZghJj/V+CjFoAGflxvYVLvk3WcqFYbtZr/HB2uXzaGTIamcouKcUsc+UMDVMuRqKGOYrdvBFaDTs/ByFxyFZuy7ZxSW0QJP9QMbD3El4YKzL5nneatJ6TKFKZe0ucTMgA4Z5UbS2QHEHWcOrdvhZz1UIrHRZ7dCfgLBPHCe8pNLPB2m+C8ETyYEFYYdlkCqzdeVPf3gDmxzV1N+NO9M7myLLxe4YaD+B3Zsi8FASa3csAaDeDlv8+rt0DulJZf4tF48czT2Pa6VHo0UzajeItARyonVLgsq4yCTqyN7zaSmPXhrBsoTV2iS4SPVdM4PEzo526Y105mFpXrAYop9Ce47w0iEY7gfCcZdWRnUYsJ3A8c711QgDHR634RlFmSRYMw2Ac4GhDtdJCutJ09zuJjJ/Tcj6adeydWW7epWIySZU83SSb2bzb2NyIMA6ClgD9BYqeN7VR3o2oEevJBVs7vfpK2yWZC24v9wOLnF/Yku7hKj4KZIB1VxJI5sL+Z88E8ZAHINC1ZD3jG4eLcV4bL7qr/zdzOLnXlrfhA9HnfcA/VCmmm5SnhEzDg7qBBYnuiPPfjfbwTLC15H7qj7PQ/AR4PBoRKmgw6UY4NTEiNtM3IhP4= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: cc0170ff-0df4-4825-82da-08da0971cadc X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2022 06:29:16.9824 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3vmMtnUI39iMVlam3Dsvcrp/oXczj6ZCvUl3Ki1TtP/RNFoPgkrNAsgeisW31geMrnXpKR4DM8e9LBWiEeYefg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR10MB1798 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10290 signatures=694221 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2203190038 X-Proofpoint-GUID: rGe-jUCbnLULhNV454fh_8CjQ8bcRNa- X-Proofpoint-ORIG-GUID: rGe-jUCbnLULhNV454fh_8CjQ8bcRNa- Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Mark poisoned page as not present, and to reverse the 'np' effect, restate the _PAGE_PRESENT bit. Please refer to discussions here for reason behind the decision. https://lore.kernel.org/all/CAPcyv4hrXPb1tASBZUg-GgdVs0OOFKXMXLiHmktg_kFi7YBMyQ@mail.gmail.com/ Now since poisoned page is marked as not-present, in order to avoid writing to a 'np' page and trigger kernel Oops, also fix pmem_do_write(). Fixes: 284ce4011ba6 ("x86/memory_failure: Introduce {set, clear}_mce_nospec()") Signed-off-by: Jane Chu --- arch/x86/kernel/cpu/mce/core.c | 6 +++--- arch/x86/mm/pat/set_memory.c | 21 +++++++++------------ drivers/nvdimm/pmem.c | 31 +++++++------------------------ include/linux/set_memory.h | 4 ++-- 4 files changed, 21 insertions(+), 41 deletions(-) diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 5818b837fd4d..8d12739f283d 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -613,7 +613,7 @@ static int uc_decode_notifier(struct notifier_block *nb, unsigned long val, pfn = mce->addr >> PAGE_SHIFT; if (!memory_failure(pfn, 0)) { - set_mce_nospec(pfn, whole_page(mce)); + set_mce_nospec(pfn); mce->kflags |= MCE_HANDLED_UC; } @@ -1297,7 +1297,7 @@ static void kill_me_maybe(struct callback_head *cb) ret = memory_failure(p->mce_addr >> PAGE_SHIFT, flags); if (!ret) { - set_mce_nospec(p->mce_addr >> PAGE_SHIFT, p->mce_whole_page); + set_mce_nospec(p->mce_addr >> PAGE_SHIFT); sync_core(); return; } @@ -1321,7 +1321,7 @@ static void kill_me_never(struct callback_head *cb) p->mce_count = 0; pr_err("Kernel accessed poison in user space at %llx\n", p->mce_addr); if (!memory_failure(p->mce_addr >> PAGE_SHIFT, 0)) - set_mce_nospec(p->mce_addr >> PAGE_SHIFT, p->mce_whole_page); + set_mce_nospec(p->mce_addr >> PAGE_SHIFT); } static void queue_task_work(struct mce *m, char *msg, void (*func)(struct callback_head *)) diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 9abc6077d768..747614c3dd83 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -1925,14 +1925,14 @@ int set_memory_wb(unsigned long addr, int numpages) } EXPORT_SYMBOL(set_memory_wb); +static int _set_memory_present(unsigned long addr, int numpages) +{ + return change_page_attr_set(&addr, numpages, __pgprot(_PAGE_PRESENT), 0); +} + #ifdef CONFIG_X86_64 -/* - * Prevent speculative access to the page by either unmapping - * it (if we do not require access to any part of the page) or - * marking it uncacheable (if we want to try to retrieve data - * from non-poisoned lines in the page). - */ -int set_mce_nospec(unsigned long pfn, bool unmap) +/* Prevent speculative access to a page by marking it not-present */ +int set_mce_nospec(unsigned long pfn) { unsigned long decoy_addr; int rc; @@ -1954,10 +1954,7 @@ int set_mce_nospec(unsigned long pfn, bool unmap) */ decoy_addr = (pfn << PAGE_SHIFT) + (PAGE_OFFSET ^ BIT(63)); - if (unmap) - rc = set_memory_np(decoy_addr, 1); - else - rc = set_memory_uc(decoy_addr, 1); + rc = set_memory_np(decoy_addr, 1); if (rc) pr_warn("Could not invalidate pfn=0x%lx from 1:1 map\n", pfn); return rc; @@ -1967,7 +1964,7 @@ EXPORT_SYMBOL(set_mce_nospec); /* Restore full speculative operation to the pfn. */ int clear_mce_nospec(unsigned long pfn) { - return set_memory_wb((unsigned long) pfn_to_kaddr(pfn), 1); + return _set_memory_present((unsigned long) pfn_to_kaddr(pfn), 1); } EXPORT_SYMBOL(clear_mce_nospec); diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 58d95242a836..30c71a68175b 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -158,36 +158,19 @@ static blk_status_t pmem_do_write(struct pmem_device *pmem, struct page *page, unsigned int page_off, sector_t sector, unsigned int len) { - blk_status_t rc = BLK_STS_OK; - bool bad_pmem = false; phys_addr_t pmem_off = sector * 512 + pmem->data_offset; void *pmem_addr = pmem->virt_addr + pmem_off; - if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) - bad_pmem = true; + if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) { + blk_status_t rc = pmem_clear_poison(pmem, pmem_off, len); - /* - * Note that we write the data both before and after - * clearing poison. The write before clear poison - * handles situations where the latest written data is - * preserved and the clear poison operation simply marks - * the address range as valid without changing the data. - * In this case application software can assume that an - * interrupted write will either return the new good - * data or an error. - * - * However, if pmem_clear_poison() leaves the data in an - * indeterminate state we need to perform the write - * after clear poison. - */ + if (rc != BLK_STS_OK) + pr_warn_ratelimited("%s: failed to clear poison\n", __func__); + return rc; + } flush_dcache_page(page); write_pmem(pmem_addr, page, page_off, len); - if (unlikely(bad_pmem)) { - rc = pmem_clear_poison(pmem, pmem_off, len); - write_pmem(pmem_addr, page, page_off, len); - } - - return rc; + return BLK_STS_OK; } static void pmem_submit_bio(struct bio *bio) diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index d6263d7afb55..cde2d8687a7b 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -43,10 +43,10 @@ static inline bool can_set_direct_map(void) #endif /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */ #ifdef CONFIG_X86_64 -int set_mce_nospec(unsigned long pfn, bool unmap); +int set_mce_nospec(unsigned long pfn); int clear_mce_nospec(unsigned long pfn); #else -static inline int set_mce_nospec(unsigned long pfn, bool unmap) +static inline int set_mce_nospec(unsigned long pfn) { return 0; } From patchwork Sat Mar 19 06:28:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12786089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39989C433EF for ; Sat, 19 Mar 2022 06:30:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242290AbiCSGbU (ORCPT ); Sat, 19 Mar 2022 02:31:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242286AbiCSGbT (ORCPT ); Sat, 19 Mar 2022 02:31:19 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECA241AA8DC; Fri, 18 Mar 2022 23:29:57 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 22J2rVeV012459; Sat, 19 Mar 2022 06:29:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=Ut4oHXjVSH/7RAcqVXqEp+FP3kXraCZgOnBL9mk/Y7o=; b=R9UsWQuf2O0+ijNf48EE5fREi8uQlkS4jTk1zpQsU6TPaQx5AwGs0nj2M8PUf0b+XnEA utZ8XOPiKGCsNBW3EMefGheColr/RPSkgoKPJyBCfDonDy2eWgYMsu8FOqIaJQ1Jv0/w C221lPn9lmf38yY+/zarIjfpk6OZ4BDjLQVO61xcAovGnPqjNdrFNR+45XeVbH41QXpO NsiAIr4OFYWQV9RwJYi/iMa+UIDKAJZNQNKOg95F2ww6n1h+qmj4IUgEAtUWZb8dCc8T keWX383rleirHuoKuc6X+b+SHRrjI63Lb42sp5Rui4nFJio545HiIZequ8VGM88n9Pq8 +g== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80] (may be forged)) by mx0b-00069f02.pphosted.com with ESMTP id 3ew6ss04bs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:25 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 22J6M4PG010196; Sat, 19 Mar 2022 06:29:24 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2045.outbound.protection.outlook.com [104.47.56.45]) by userp3030.oracle.com with ESMTP id 3ew49r0ar4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qr7OJvUM2exvLhiJ1Hg1Trh4iRISUb6PijzOXC6FmEWx+7fLyjIGTyb9+yXpZwefbQrog9vlXXGdcVa4FqbFLLGoVj/PkVyVp6GKfG+AyBq/6YPQgWzu40XkTM+tUt/VAwSJrZwd5aake8jYxLrUd6O9WJtcvmFfvseL7cPaY9VCPNPIFmipKUYUwgkK+fmGU1opTiN9xJIPlacTm03jh3XyyMRErsYe4Ia16UYAcNCtF3aQAnYoZgHVQ11klh8Husu03CUqpKxdX6mEGFyq55VWaE/gekjvXpPoSBG1yqTO5r3FjkByRH/NyYd4bpotLXoAPuWIVDRGyLskzKz0hA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ut4oHXjVSH/7RAcqVXqEp+FP3kXraCZgOnBL9mk/Y7o=; b=cLoPIH8VNPMwCD5X2MFvKZMFvhzhty9VK3V0vgLWrTKt3BbooucFVdAT4x1QDtjmtePhjFnOqC+OYEt4tEezIXbDUZUi/g6sFj9PlOBUQBiEF7TBPlT9sZMGNEu1X5YkpG0ww2JCFWXhGo8vvaP9/0/3PaMs2Ig0na7JmEe7CZ28CNBlQsb8o5r0s3ZaORfV6y5tM0IpfTTaH0oqwNi3KVwUx5PhCxPQkd8OVwK7aTI32LZHY/cu6uS4dgt1GsFTmsHg7Qwxvm44aCd3p6tdtXheJfGUmNDDOcfoXUlfsDMKj+MgJNePeuJZLduZzx/OLLzuCnXhQKgV0pQtJodJkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ut4oHXjVSH/7RAcqVXqEp+FP3kXraCZgOnBL9mk/Y7o=; b=CkKGg30PhherkvWzj0kWTSfEbxJktsDi0cmDtKTOb1KbejI+C8vnFaEx/DcXKCPkgbUDG+gs2nRmgn4HGB4UMMM0FzWppIVgMHVy2YbvniqqaWp1yexeR/m4Qk8+8RWiKwVYkTHjJFlXH3u6PNhqZEQYzrVRV6QwQjbRJBcqA34= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by CY4PR10MB1798.namprd10.prod.outlook.com (2603:10b6:903:126::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.17; Sat, 19 Mar 2022 06:29:21 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336%8]) with mapi id 15.20.5081.019; Sat, 19 Mar 2022 06:29:21 +0000 From: Jane Chu To: david@fromorbit.com, djwong@kernel.org, dan.j.williams@intel.com, hch@infradead.org, vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org Subject: [PATCH v6 4/6] dax: add DAX_RECOVERY flag and .recovery_write dev_pgmap_ops Date: Sat, 19 Mar 2022 00:28:31 -0600 Message-Id: <20220319062833.3136528-5-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220319062833.3136528-1-jane.chu@oracle.com> References: <20220319062833.3136528-1-jane.chu@oracle.com> X-ClientProxiedBy: BY5PR16CA0025.namprd16.prod.outlook.com (2603:10b6:a03:1a0::38) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 26f8cb4b-a26b-4fdc-58a4-08da0971cde3 X-MS-TrafficTypeDiagnostic: CY4PR10MB1798:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sQViXqR7pe1C2jMdnGa2/sA2M97P+TP/F8KeDrhWLDAqfUn2/0zyXbluwnHXTU8eaVsb+TeGWBXVLw07agi6E7Ii0vItMzu/mE4md3kIXgUarj9hTw3wVaVtVOPStqXmy0MidzN4bQdRuDFMLfdp1B29r+y5cQ1K4RfeRz0rETcS37g6ycpFFYOkRVJ3NtIzvzCHWWuL9z1sAfTSiDDgIoB3vV6f2CDNR0YXPD8Ka8uk4Q/P6COd5KtND+PiDXnXfu07j8kap/5cJGU5NNhxN1iWvQJALpBfC38M/v434RM3UXqODsFgRY9vFrQTRfjd7MjQibKg9bxgf21FJdJ3AGVCysM5rzBcFHjpXJWokNHK4hG/BZBX2aQexqCeYm3vbImjIkmrBLk+uJmt5LYm9i350KDSeTQGlPTwwEKyCkrIErJnf3QXB957TkuLcf5pgX9i+QBGBSVXuKfijFR2GaF5t8eIXig8TaYx1yiVtYOBxYXkwgrmuIbrB9fc4WPtkw/w31x9lZ73MhoWhFjezmITbywU53RKpS/lHCD3yMszlD7/3HbC7e5o6IIXTWF/NjQ7Kgz7rHmBvXAqNXAkJv6IeA2KR++J9sIfwHNv5rFX/dlGU8NnoQJVDZmiOlnNA+Pn9k6GcT2QGO4yZn8vcGhegV+zFxYRhGBl3Lq5xTMWFgp+WU7kQy/CnP3d9K1QkO/MCYDmTakXKXt9HkuH0ttb/XH4dWG+wKyH8B2mCa/fV2aeXJ+sowpUpkp4bwgI X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(52116002)(6512007)(66476007)(8676002)(83380400001)(66556008)(316002)(6506007)(2906002)(38100700002)(66946007)(508600001)(44832011)(5660300002)(8936002)(7416002)(30864003)(36756003)(1076003)(6486002)(86362001)(186003)(2616005)(921005)(6666004)(142923001)(21314003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: wYxJVdqnb1n6Ef5Ah/0j70SQZ8ZIvRvf7lCXzp4ncruLHqwSOQc01nb4n3GZbzgrDqI7UlbPF32K7D/GR24Enfa33U+bLgl8ntDEx7wrVoZj8MwqOhztK38iiNNXOO4YY6x1RpMdjLlU5bJs1qkr8I5aSF8UxH0Vo8snoBHHuvEOl7lggqnm55hUV+sAmMTlpyvmLqSbOSODNq/eszi9i7ix2y/TI6/Eru6iAZgQDtqdvhZ9N1uSC4ZqSggf4T4F82jeH83HZTtQbWeZ/GRhyIe5Xy8jpWYPXqHcXzWzP2FHVLWVc7jQBuw7+d7+J5ChPkqSc1NBDMlTy9WqYQSYFXbBhvBmXBd6tmHU81tTLg57Rd2V07VVi46p/j5MlIbit/Whu03mA6TU92DcXJP9k5TdQs5bM/CSf+BHcUGudVW2FjEzErnpxCDgsg/kvpq9Ociws8CdlGkzv/URuJkT60MVAfqr9P0nPqhu/yjpmA9gv2/t+xefUZVHbi5WlRXUaXlm9GcyLx8Pgke50A/eeHNq0aBDsBjsUdbXxgGKYNtUBXgIm2Gz4b2udRcXG6VryDOUMHPXdhX+9joJ7qZmgKPiVEpVUC66CFcTjobLD84pWWPgkMvv7OVvE+mntUHlhvtpp82l5RkESLmnfrMhNva2V3Ru9vbFZLk3CKhMu9P78hMRNpHC3p9lyQi6JM+lYtaFgh4LKCGzUnxa5wMGfl1LU2ryxX2zBOoC0KF/WAXJXOJh3oDtOf8bCy216wb2I5HcKYj1SrHQgDJNNEWB8EjhHpCHFtZQLwvm9lqQeeXr51jvGJACtmPDCs2+vv0AYSdludp5wmbgADSDeXblHSup9dM2l7LMWk+BUV0+5PhePJUTj/5zRe+a3N/cfIXYinIv+5Bf33w8Mk11r3rTqSgtYPr3N/HJaMz/kcb82JelLeerK4tuBiiyLM7zkJ1RsOQ45mhy0kFXnnhG1HxHFrjDXcknzszP6VYlzjygGyRjm0oVLylhfmJ95FtB/gahJp5+3hfYZwBK8i6PZ/tybs9DUjLWhTyVzfWkypRhOzAipwF8Rj+BKLJiINRquJFNaKK36ZUpiFJkBOen1cNrINtmdvcpjYNsdBBmiGTtpXKfDmgJiuh89N0douNcDRYz9sylDbTA4hxVtKZz9+0tzwPy0uUg4GsV45jsf+MKEB5L/fJeLLkO8DlXdWRi/c8N3/jFMqFk+rFS1K9QSUZ367UqQiU5gWK4+Zy4w2kBJYU6m68tyqUosU/U78L0v7vYwZj3Yq/TZ+9gfiRzmgfWTlE47PryLZqmrA6B4yr2B7ddxIMBqGAvYEKADTDuNB42sDi4C8YW8qjWXUc9BErnrdqOjvox++/NkXb8Hqp56UdKRR9bxoisLQvaNgrmosexF/QEaUwqn509NjrRdNzA62L4iTtdmZ6e1xt7l2cF5ismythQlR+ZUS4ix+YOA4t8wI3NU7xDznFxybW0xtBuS6r4pdk47TekvRbgCrTe9H0= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 26f8cb4b-a26b-4fdc-58a4-08da0971cde3 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2022 06:29:21.7935 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Yl1k8zS5yHq14U2bcjZpWcAqxJMd28ZFSBh1uEG5XdzUUZtxQEazjfQEJY+4nRxVibcq+FmpT2+fS2wAbZDHTA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR10MB1798 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10290 signatures=694221 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxscore=0 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2203190038 X-Proofpoint-ORIG-GUID: JHkyksVwKGWf9gs4k1HWGeWjN8DeM5ky X-Proofpoint-GUID: JHkyksVwKGWf9gs4k1HWGeWjN8DeM5ky Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Introduce DAX_RECOVERY flag to dax_direct_access(). The flag is not set by default in dax_direct_access() such that the helper does not translate a pmem range to kernel virtual address if the range contains uncorrectable errors. When the flag is set, the helper ignores the UEs and return kernel virtual adderss so that the caller may get on with data recovery via write. Also introduce a new dev_pagemap_ops .recovery_write function. The function is applicable to FSDAX device only. The device page backend driver provides .recovery_write function if the device has underlying mechanism to clear the uncorrectable errors on the fly. Signed-off-by: Jane Chu Reported-by: kernel test robot Reported-by: kernel test robot --- drivers/dax/super.c | 23 +++++++++++++++++++++-- drivers/md/dm-linear.c | 4 ++-- drivers/md/dm-log-writes.c | 5 +++-- drivers/md/dm-stripe.c | 4 ++-- drivers/md/dm-target.c | 2 +- drivers/md/dm-writecache.c | 5 +++-- drivers/md/dm.c | 5 +++-- drivers/nvdimm/pmem.c | 27 +++++++++++++++++++++++---- drivers/nvdimm/pmem.h | 2 +- drivers/s390/block/dcssblk.c | 4 ++-- fs/dax.c | 32 ++++++++++++++++++++++++++------ fs/fuse/dax.c | 4 ++-- include/linux/dax.h | 12 +++++++++--- include/linux/device-mapper.h | 2 +- include/linux/memremap.h | 7 +++++++ tools/testing/nvdimm/pmem-dax.c | 2 +- 16 files changed, 107 insertions(+), 33 deletions(-) diff --git a/drivers/dax/super.c b/drivers/dax/super.c index e3029389d809..c0f0c8f980b1 100644 --- a/drivers/dax/super.c +++ b/drivers/dax/super.c @@ -28,6 +28,7 @@ struct dax_device { void *private; unsigned long flags; const struct dax_operations *ops; + struct dev_pagemap *pgmap; }; static dev_t dax_devt; @@ -116,6 +117,7 @@ enum dax_device_flags { * @dax_dev: a dax_device instance representing the logical memory range * @pgoff: offset in pages from the start of the device to translate * @nr_pages: number of consecutive pages caller can handle relative to @pfn + * @flags: by default 0, set to DAX_RECOVERY to kick start dax recovery * @kaddr: output parameter that returns a virtual address mapping of pfn * @pfn: output parameter that returns an absolute pfn translation of @pgoff * @@ -123,7 +125,7 @@ enum dax_device_flags { * pages accessible at the device relative @pgoff. */ long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, - void **kaddr, pfn_t *pfn) + int flags, void **kaddr, pfn_t *pfn) { long avail; @@ -136,7 +138,7 @@ long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, if (nr_pages < 0) return -EINVAL; - avail = dax_dev->ops->direct_access(dax_dev, pgoff, nr_pages, + avail = dax_dev->ops->direct_access(dax_dev, pgoff, nr_pages, flags, kaddr, pfn); if (!avail) return -ERANGE; @@ -193,6 +195,18 @@ int dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff, } EXPORT_SYMBOL_GPL(dax_zero_page_range); +size_t dax_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *iter) +{ + struct dev_pagemap *pgmap = dax_dev->pgmap; + + if (!pgmap || !pgmap->ops->recovery_write) + return -EIO; + return pgmap->ops->recovery_write(pgmap, pgoff, addr, bytes, + (void *)iter); +} +EXPORT_SYMBOL_GPL(dax_recovery_write); + #ifdef CONFIG_ARCH_HAS_PMEM_API void arch_wb_cache_pmem(void *addr, size_t size); void dax_flush(struct dax_device *dax_dev, void *addr, size_t size) @@ -247,6 +261,11 @@ void set_dax_nomc(struct dax_device *dax_dev) set_bit(DAXDEV_NOMC, &dax_dev->flags); } EXPORT_SYMBOL_GPL(set_dax_nomc); +void set_dax_pgmap(struct dax_device *dax_dev, struct dev_pagemap *pgmap) +{ + dax_dev->pgmap = pgmap; +} +EXPORT_SYMBOL_GPL(set_dax_pgmap); bool dax_alive(struct dax_device *dax_dev) { diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index 1b97a11d7151..bfd8895317c0 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -173,11 +173,11 @@ static struct dax_device *linear_dax_pgoff(struct dm_target *ti, pgoff_t *pgoff) } static long linear_dax_direct_access(struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, int flags, void **kaddr, pfn_t *pfn) { struct dax_device *dax_dev = linear_dax_pgoff(ti, &pgoff); - return dax_direct_access(dax_dev, pgoff, nr_pages, kaddr, pfn); + return dax_direct_access(dax_dev, pgoff, nr_pages, flags, kaddr, pfn); } static int linear_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c index 139b09b06eda..8ee8a9f5b161 100644 --- a/drivers/md/dm-log-writes.c +++ b/drivers/md/dm-log-writes.c @@ -912,11 +912,12 @@ static struct dax_device *log_writes_dax_pgoff(struct dm_target *ti, } static long log_writes_dax_direct_access(struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, int flags, + void **kaddr, pfn_t *pfn) { struct dax_device *dax_dev = log_writes_dax_pgoff(ti, &pgoff); - return dax_direct_access(dax_dev, pgoff, nr_pages, kaddr, pfn); + return dax_direct_access(dax_dev, pgoff, nr_pages, flags, kaddr, pfn); } static int log_writes_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, diff --git a/drivers/md/dm-stripe.c b/drivers/md/dm-stripe.c index e566115ec0bb..af043d93ef53 100644 --- a/drivers/md/dm-stripe.c +++ b/drivers/md/dm-stripe.c @@ -317,11 +317,11 @@ static struct dax_device *stripe_dax_pgoff(struct dm_target *ti, pgoff_t *pgoff) } static long stripe_dax_direct_access(struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, int flags, void **kaddr, pfn_t *pfn) { struct dax_device *dax_dev = stripe_dax_pgoff(ti, &pgoff); - return dax_direct_access(dax_dev, pgoff, nr_pages, kaddr, pfn); + return dax_direct_access(dax_dev, pgoff, nr_pages, flags, kaddr, pfn); } static int stripe_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, diff --git a/drivers/md/dm-target.c b/drivers/md/dm-target.c index 64dd0b34fcf4..24b1e5628f3a 100644 --- a/drivers/md/dm-target.c +++ b/drivers/md/dm-target.c @@ -142,7 +142,7 @@ static void io_err_release_clone_rq(struct request *clone, } static long io_err_dax_direct_access(struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, int flags, void **kaddr, pfn_t *pfn) { return -EIO; } diff --git a/drivers/md/dm-writecache.c b/drivers/md/dm-writecache.c index 4f31591d2d25..58af379107ec 100644 --- a/drivers/md/dm-writecache.c +++ b/drivers/md/dm-writecache.c @@ -286,7 +286,8 @@ static int persistent_memory_claim(struct dm_writecache *wc) id = dax_read_lock(); - da = dax_direct_access(wc->ssd_dev->dax_dev, offset, p, &wc->memory_map, &pfn); + da = dax_direct_access(wc->ssd_dev->dax_dev, offset, p, 0, + &wc->memory_map, &pfn); if (da < 0) { wc->memory_map = NULL; r = da; @@ -309,7 +310,7 @@ static int persistent_memory_claim(struct dm_writecache *wc) do { long daa; daa = dax_direct_access(wc->ssd_dev->dax_dev, offset + i, p - i, - NULL, &pfn); + 0, NULL, &pfn); if (daa <= 0) { r = daa ? daa : -EINVAL; goto err3; diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 997ace47bbd5..6b6509bfaa1a 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1001,7 +1001,8 @@ static struct dm_target *dm_dax_get_live_target(struct mapped_device *md, } static long dm_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, int flags, void **kaddr, + pfn_t *pfn) { struct mapped_device *md = dax_get_private(dax_dev); sector_t sector = pgoff * PAGE_SECTORS; @@ -1019,7 +1020,7 @@ static long dm_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, if (len < 1) goto out; nr_pages = min(len, nr_pages); - ret = ti->type->direct_access(ti, pgoff, nr_pages, kaddr, pfn); + ret = ti->type->direct_access(ti, pgoff, nr_pages, flags, kaddr, pfn); out: dm_put_live_table(md, srcu_idx); diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 30c71a68175b..7cdaa279beca 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -238,11 +238,11 @@ static int pmem_rw_page(struct block_device *bdev, sector_t sector, /* see "strong" declaration in tools/testing/nvdimm/pmem-dax.c */ __weak long __pmem_direct_access(struct pmem_device *pmem, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, int flags, void **kaddr, pfn_t *pfn) { resource_size_t offset = PFN_PHYS(pgoff) + pmem->data_offset; - if (unlikely(is_bad_pmem(&pmem->bb, PFN_PHYS(pgoff) / 512, + if (!flags && unlikely(is_bad_pmem(&pmem->bb, PFN_PHYS(pgoff) / 512, PFN_PHYS(nr_pages)))) return -EIO; @@ -277,11 +277,12 @@ static int pmem_dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff, } static long pmem_dax_direct_access(struct dax_device *dax_dev, - pgoff_t pgoff, long nr_pages, void **kaddr, pfn_t *pfn) + pgoff_t pgoff, long nr_pages, int flags, void **kaddr, + pfn_t *pfn) { struct pmem_device *pmem = dax_get_private(dax_dev); - return __pmem_direct_access(pmem, pgoff, nr_pages, kaddr, pfn); + return __pmem_direct_access(pmem, pgoff, nr_pages, flags, kaddr, pfn); } static const struct dax_operations pmem_dax_ops = { @@ -289,6 +290,17 @@ static const struct dax_operations pmem_dax_ops = { .zero_page_range = pmem_dax_zero_page_range, }; +static size_t pmem_recovery_write(struct dev_pagemap *pgmap, pgoff_t pgoff, + void *addr, size_t bytes, void *iter) +{ + struct pmem_device *pmem = pgmap->owner; + + dev_warn(pmem->bb.dev, "%s: not yet implemented\n", __func__); + + /* XXX more later */ + return 0; +} + static ssize_t write_cache_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -349,6 +361,10 @@ static void pmem_release_disk(void *__pmem) blk_cleanup_disk(pmem->disk); } +static const struct dev_pagemap_ops pmem_pgmap_ops = { + .recovery_write = pmem_recovery_write, +}; + static int pmem_attach_disk(struct device *dev, struct nd_namespace_common *ndns) { @@ -380,6 +396,8 @@ static int pmem_attach_disk(struct device *dev, rc = nvdimm_setup_pfn(nd_pfn, &pmem->pgmap); if (rc) return rc; + if (nd_pfn->mode == PFN_MODE_PMEM) + pmem->pgmap.ops = &pmem_pgmap_ops; } /* we're attaching a block device, disable raw namespace access */ @@ -464,6 +482,7 @@ static int pmem_attach_disk(struct device *dev, } set_dax_nocache(dax_dev); set_dax_nomc(dax_dev); + set_dax_pgmap(dax_dev, &pmem->pgmap); if (is_nvdimm_sync(nd_region)) set_dax_synchronous(dax_dev); rc = dax_add_host(dax_dev, disk); diff --git a/drivers/nvdimm/pmem.h b/drivers/nvdimm/pmem.h index 59cfe13ea8a8..823eeffa7298 100644 --- a/drivers/nvdimm/pmem.h +++ b/drivers/nvdimm/pmem.h @@ -27,7 +27,7 @@ struct pmem_device { }; long __pmem_direct_access(struct pmem_device *pmem, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn); + long nr_pages, int flag, void **kaddr, pfn_t *pfn); #ifdef CONFIG_MEMORY_FAILURE static inline bool test_and_clear_pmem_poison(struct page *page) diff --git a/drivers/s390/block/dcssblk.c b/drivers/s390/block/dcssblk.c index d614843caf6c..c3fbf500868f 100644 --- a/drivers/s390/block/dcssblk.c +++ b/drivers/s390/block/dcssblk.c @@ -32,7 +32,7 @@ static int dcssblk_open(struct block_device *bdev, fmode_t mode); static void dcssblk_release(struct gendisk *disk, fmode_t mode); static void dcssblk_submit_bio(struct bio *bio); static long dcssblk_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn); + long nr_pages, int flags, void **kaddr, pfn_t *pfn); static char dcssblk_segments[DCSSBLK_PARM_LEN] = "\0"; @@ -927,7 +927,7 @@ __dcssblk_direct_access(struct dcssblk_dev_info *dev_info, pgoff_t pgoff, static long dcssblk_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, int flags, void **kaddr, pfn_t *pfn) { struct dcssblk_dev_info *dev_info = dax_get_private(dax_dev); diff --git a/fs/dax.c b/fs/dax.c index cd03485867a7..9f7efd0a1dd9 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -722,7 +722,7 @@ static int copy_cow_page_dax(struct vm_fault *vmf, const struct iomap_iter *iter int id; id = dax_read_lock(); - rc = dax_direct_access(iter->iomap.dax_dev, pgoff, 1, &kaddr, NULL); + rc = dax_direct_access(iter->iomap.dax_dev, pgoff, 1, 0, &kaddr, NULL); if (rc < 0) { dax_read_unlock(id); return rc; @@ -1013,7 +1013,7 @@ static int dax_iomap_pfn(const struct iomap *iomap, loff_t pos, size_t size, long length; id = dax_read_lock(); - length = dax_direct_access(iomap->dax_dev, pgoff, PHYS_PFN(size), + length = dax_direct_access(iomap->dax_dev, pgoff, PHYS_PFN(size), 0, NULL, pfnp); if (length < 0) { rc = length; @@ -1123,7 +1123,7 @@ static int dax_memzero(struct dax_device *dax_dev, pgoff_t pgoff, void *kaddr; long ret; - ret = dax_direct_access(dax_dev, pgoff, 1, &kaddr, NULL); + ret = dax_direct_access(dax_dev, pgoff, 1, 0, &kaddr, NULL); if (ret > 0) { memset(kaddr + offset, 0, size); dax_flush(dax_dev, kaddr + offset, size); @@ -1240,15 +1240,27 @@ static loff_t dax_iomap_iter(const struct iomap_iter *iomi, const size_t size = ALIGN(length + offset, PAGE_SIZE); pgoff_t pgoff = dax_iomap_pgoff(iomap, pos); ssize_t map_len; + int flags, recov; void *kaddr; + long nrpg; if (fatal_signal_pending(current)) { ret = -EINTR; break; } - map_len = dax_direct_access(dax_dev, pgoff, PHYS_PFN(size), - &kaddr, NULL); + recov = 0; + flags = 0; + nrpg = PHYS_PFN(size); + map_len = dax_direct_access(dax_dev, pgoff, nrpg, flags, + &kaddr, NULL); + if ((map_len == -EIO) && (iov_iter_rw(iter) == WRITE)) { + flags |= DAX_RECOVERY; + map_len = dax_direct_access(dax_dev, pgoff, nrpg, + flags, &kaddr, NULL); + if (map_len > 0) + recov++; + } if (map_len < 0) { ret = map_len; break; @@ -1260,7 +1272,10 @@ static loff_t dax_iomap_iter(const struct iomap_iter *iomi, if (map_len > end - pos) map_len = end - pos; - if (iov_iter_rw(iter) == WRITE) + if (recov) + xfer = dax_recovery_write(dax_dev, pgoff, kaddr, + map_len, iter); + else if (iov_iter_rw(iter) == WRITE) xfer = dax_copy_from_iter(dax_dev, pgoff, kaddr, map_len, iter); else @@ -1271,6 +1286,11 @@ static loff_t dax_iomap_iter(const struct iomap_iter *iomi, length -= xfer; done += xfer; + if (recov && (xfer == (ssize_t) -EIO)) { + pr_warn("dax_recovery_write failed\n"); + ret = -EIO; + break; + } if (xfer == 0) ret = -EFAULT; if (xfer < map_len) diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c index 182b24a14804..e22c3ca9fdce 100644 --- a/fs/fuse/dax.c +++ b/fs/fuse/dax.c @@ -1241,8 +1241,8 @@ static int fuse_dax_mem_range_init(struct fuse_conn_dax *fcd) INIT_DELAYED_WORK(&fcd->free_work, fuse_dax_free_mem_worker); id = dax_read_lock(); - nr_pages = dax_direct_access(fcd->dev, 0, PHYS_PFN(dax_size), NULL, - NULL); + nr_pages = dax_direct_access(fcd->dev, 0, PHYS_PFN(dax_size), 0, + NULL, NULL); dax_read_unlock(id); if (nr_pages < 0) { pr_debug("dax_direct_access() returned %ld\n", nr_pages); diff --git a/include/linux/dax.h b/include/linux/dax.h index 9fc5f99a0ae2..50bf4b0fb9b6 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -14,14 +14,17 @@ struct iomap_ops; struct iomap_iter; struct iomap; +/* Flag to communicate for DAX recovery operation */ +#define DAX_RECOVERY 0x1 + struct dax_operations { /* * direct_access: translate a device-relative * logical-page-offset into an absolute physical pfn. Return the * number of pages available for DAX at that pfn. */ - long (*direct_access)(struct dax_device *, pgoff_t, long, - void **, pfn_t *); + long (*direct_access)(struct dax_device *dax_dev, pgoff_t pgoff, + long nr_pages, int flags, void **kaddr, pfn_t *pfn); /* * Validate whether this device is usable as an fsdax backing * device. @@ -40,6 +43,8 @@ void dax_write_cache(struct dax_device *dax_dev, bool wc); bool dax_write_cache_enabled(struct dax_device *dax_dev); bool dax_synchronous(struct dax_device *dax_dev); void set_dax_synchronous(struct dax_device *dax_dev); +size_t dax_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, + size_t bytes, struct iov_iter *i); /* * Check if given mapping is supported by the file / underlying device. */ @@ -91,6 +96,7 @@ static inline bool daxdev_mapping_supported(struct vm_area_struct *vma, void set_dax_nocache(struct dax_device *dax_dev); void set_dax_nomc(struct dax_device *dax_dev); +void set_dax_pgmap(struct dax_device *dax_dev, struct dev_pagemap *pgmap); struct writeback_control; #if defined(CONFIG_BLOCK) && defined(CONFIG_FS_DAX) @@ -178,7 +184,7 @@ static inline void dax_read_unlock(int id) bool dax_alive(struct dax_device *dax_dev); void *dax_get_private(struct dax_device *dax_dev); long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, - void **kaddr, pfn_t *pfn); + int flags, void **kaddr, pfn_t *pfn); size_t dax_copy_from_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, size_t bytes, struct iov_iter *i); size_t dax_copy_to_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index b26fecf6c8e8..8cdfd7566a38 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -146,7 +146,7 @@ typedef int (*dm_busy_fn) (struct dm_target *ti); * >= 0 : the number of bytes accessible at the address */ typedef long (*dm_dax_direct_access_fn) (struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn); + long nr_pages, int flags, void **kaddr, pfn_t *pfn); typedef int (*dm_dax_zero_page_range_fn)(struct dm_target *ti, pgoff_t pgoff, size_t nr_pages); diff --git a/include/linux/memremap.h b/include/linux/memremap.h index 1fafcc38acba..a495e3c4c5fc 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -77,6 +77,13 @@ struct dev_pagemap_ops { * the page back to a CPU accessible page. */ vm_fault_t (*migrate_to_ram)(struct vm_fault *vmf); + + /* + * Used for FS DAX device only. For synchronous recovery from DAX media + * encountering Uncorrectable Error. + */ + size_t (*recovery_write)(struct dev_pagemap *pgmap, pgoff_t pgoff, + void *addr, size_t bytes, void *iter); }; #define PGMAP_ALTMAP_VALID (1 << 0) diff --git a/tools/testing/nvdimm/pmem-dax.c b/tools/testing/nvdimm/pmem-dax.c index af19c85558e7..287db5e3e293 100644 --- a/tools/testing/nvdimm/pmem-dax.c +++ b/tools/testing/nvdimm/pmem-dax.c @@ -8,7 +8,7 @@ #include long __pmem_direct_access(struct pmem_device *pmem, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, int flags, void **kaddr, pfn_t *pfn) { resource_size_t offset = PFN_PHYS(pgoff) + pmem->data_offset; From patchwork Sat Mar 19 06:28:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12786093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56BFAC4332F for ; Sat, 19 Mar 2022 06:30:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242397AbiCSGb6 (ORCPT ); Sat, 19 Mar 2022 02:31:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242312AbiCSGbx (ORCPT ); Sat, 19 Mar 2022 02:31:53 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E9E92D638F; Fri, 18 Mar 2022 23:30:10 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 22J3CLUq001925; Sat, 19 Mar 2022 06:29:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=m3d2i1c4LKmCJX7BC/CCYxj8s31yz5Gw7yMTH6eEnzo=; b=sa/lQOvPAec8XwpVYYr/YNCvRrfI2uLN8kNwyUBwck1OO5+GjMHgU9kvhn02hPo+HEc4 NT3LxLkkkZ1wQw4wYk9RgYasqB08RwsufApWfS6bXKgbom3puwKurtlfQvPdvUcKN5TC 2G5ZUgjlfQLAr80IMzElrCYRq/XfhpmeFRd9lXfDJqWQTeTYuFcQ5D1yuVHFL5Xupvdh 4YOaRGgtSFivXjnnAZE11pAIirhX0o/EoWs8O1ZFbS75WdDcCfwI2W/NkwRbWbyrqNHP ZhFAsbI19eTPaQb/An/Xtr9gjbNu4PPcjf53ofVBewuOijRVuGp+vE79ThnGNam/ygnS rA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3ew72a844w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:29 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 22J6MkAK028031; Sat, 19 Mar 2022 06:29:27 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2049.outbound.protection.outlook.com [104.47.56.49]) by userp3020.oracle.com with ESMTP id 3ew8mfrwm7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZNBzYct6uDlzKC7KBh4Gm25Kf8ixJJWps+3WlcYN4BjFYkQKZpGTFOXmRqvi393nih9Wx6RyMU3YjJiYDId7lllAdoYI9cqRE0NBXEEV2CEqlLSSOP/KKSmTXdufNI3QM20HA+JYsoMTcmNKY+P5GqzELGkpG3w5nA0GJ/vL4O2syHXRZ3qA4Ns8nGz1prtJQunlpRIOfe4kcCxg9ysg+sBJxih5JwafwdHCL8D/CwLndL0xaRxMAbCJ395Rrrjk/lLagHlILGwFI8rlDP/bNcRmjpujzGuhdjKcqFQKT/xm+8dJGQgl9EcTR4ZK7s+3FQ7xlpslUPq+DDdVaiBJWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m3d2i1c4LKmCJX7BC/CCYxj8s31yz5Gw7yMTH6eEnzo=; b=gEnVn3cHMqQhSUmvDjLSW9Z+wLj+N2PkoCTPXXMI9S+v8RfPKu6p3zLAGZnbowFQwzW/CKRmYxpx5IVekITXOu5xZG8emJPKZDxEdTiSPz3s+JPQZQE36e+OJIpZ+L8pg/kzcBoIE8ReeoxdN0WZESl+SQmY4cv8AlPHaV2zFieRmXIr+LVkL81AEIQXG4kMoqG9tF9fxJLKV3plyY3JkEVsLkidc8R9f2A9gEda3Q86+u+aDth/lHYwCQA85jnR8s1d1MvphwykP+Y1dkXX5V4NrbeT4oNh6M3JHwsHVFnJGnbSMMxw9YG9Nrr0dVDTPAX10b6FsGQIrHnSu92Q2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m3d2i1c4LKmCJX7BC/CCYxj8s31yz5Gw7yMTH6eEnzo=; b=I0NXcg0y/ZZj4fnCPir631wvOd2ps2+9FAfyAjglN27/5jIEwHRpfGHphLeUKK7n7DmCeqUBqVFs2Qb+a9SwS05Ix9QseKgoN3JA/wDx4ZVxC8s35c361jX40jtJJqesO8l4jv/1IiYJU4ogA+uMZJw8fyn+9OMlBI9Jg7pLyEk= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by CY4PR10MB1798.namprd10.prod.outlook.com (2603:10b6:903:126::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.17; Sat, 19 Mar 2022 06:29:25 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336%8]) with mapi id 15.20.5081.019; Sat, 19 Mar 2022 06:29:25 +0000 From: Jane Chu To: david@fromorbit.com, djwong@kernel.org, dan.j.williams@intel.com, hch@infradead.org, vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org Subject: [PATCH v6 5/6] pmem: refactor pmem_clear_poison() Date: Sat, 19 Mar 2022 00:28:32 -0600 Message-Id: <20220319062833.3136528-6-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220319062833.3136528-1-jane.chu@oracle.com> References: <20220319062833.3136528-1-jane.chu@oracle.com> X-ClientProxiedBy: BY5PR16CA0025.namprd16.prod.outlook.com (2603:10b6:a03:1a0::38) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ff731b21-4de3-4ac7-3e2e-08da0971d03a X-MS-TrafficTypeDiagnostic: CY4PR10MB1798:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: paAS/Y84TQ6hkW/JLc+m6KtLnhML5FxwrxxrKRIwkz3s3QD2cH7FUQ34B37y9QA6uowbWeFyWwWyEEHMuo1N8cBcUtN2i1FUwCoj0LFcxBVZA7UQigovBpnAwIX1P+XOwzv01AY/cQ3Li04pGU0fGSdDDKid8+ty1yjwVqi/wrykn0UZRDM/biaKUmCEaDpAfxRMM9/gB6PfOZdeGnCDM8njOuiEj49fhCDscmqCKV8zmtkM8qZaFRInbG7RUEDNxjd4XtGVtxxEdPYgLbIQfp1lfqEn8M0C/nbPCUEZJnjyylsWG927stwqdru6NZbVs3hJrCWUZ6FJdf8JiGjlnFighqMU/UNNrqJgkaQV07jn+T92F+fXgwUj/EHx18ykFPnYAU39NxFSjRpMT4gLvt4UXaKpdffqnXCt+6yLqcHPWx8Vm251c8e4zkLk4iGN9akQOq+gch5iWM9fUyY0ZxjxOYBNkXHqz6+Vc87T3BRwUqtsp42PCTaYgnPvhMfBzNo34tTVstwk1xVMDeo9lLdsXQL7JyFu2xSI4CKqrs79bUrDuxhvGWQUYrijPwWntae4dpsjpbNs2e1N+p9CJcWmnLjrDyvogZ7b15Dex9QmHv9ZSRPUeJzcJG3PcIvpULV1OzIuq5GVHm6MSh0LLdRonCkEukZqvD7df1Hl39WU2U0jM4pMb3yDPb7CFIN0 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(52116002)(6512007)(66476007)(8676002)(83380400001)(66556008)(316002)(6506007)(2906002)(38100700002)(66946007)(508600001)(44832011)(5660300002)(8936002)(7416002)(36756003)(1076003)(6486002)(86362001)(186003)(2616005)(921005)(6666004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: I0R42Bw60kqrRT3eBvL7jFBpeDP5VMw/+WfxZkI36Itr8WvaCnEmNkmXhUUoroYiwSgXuhxIC0d8QQ0sV9trOLg3mhxjhUQcue6Cv3E98vOeScL0EMkZ3GeZHUZ1sw3EPNaQMAcdWaLnj+fEA6AF5rkC813TPrpe+eAZJg8NP7FGgmh2hAv+1raCjJrKiEhMleyYXVN4k6VsIAmUbhCqynoht8avt7H0InszopkeAwrtiTsB1shEGkHd5yXGvR2V47fmJNUoH8LSFCELtpuaMrniwyUL4oyrznblSADv+LDyAA0kEJTzx2AuUZyRp4ztkk/n5zn5gKj1AThp0k5qB2NUvNfZqnveBrGFVNx4/6kWCGiR4c3bpW7H3RaR6fokAHbG1fK2L9Li1cp0d+/FzfXhUc26S0/bl3iYw7Am5VVK9INAJCiAEgUZwy4hYF50rsriw9IFWu/qficSNkQcbDwp6P0ANajx9PvBhcrN78SzBiZ5Pj0nx+q+qn3SGfeZ4ZzNAnG8GkBTOlGUAFCLfMRLHYZ0WHG2Z/otH3M0UKoFog83Rh+o9433AYcHgiXQ3T+Fbm/Jkx+2RUiUU2DB2wYBTomDDH0o0v6aZGGQ0q+C/5Nbn/9fXSZe5uh5rnTLCI6zqZtsvR3OQyWNtIHzsGFItN0PkjuJwX7T6Bl0e86wb223KbVLqBU8t1Vc18rALlRPvqRa2r56NQSP1dUTOmlDfm+LqdJIsSgjLTFSy8CzE76E8KMh+yF1y9QDky2bj339pKtO3nu0sDCepHC+9ow86lycjT/j4hzwlZznWigp3SexGAdH0/SJQG3TcbAfc7f1jaQCDu1jXln8x4hQERTamndjq4mcahCpZ5naKueiZKf/Sko1SadSlLpbPURAeJSBu3PWsazsWsONF3S6MKTi0EcRGBnVp+iS2Z/R9+lU/M2hpdMPjbzej085wyTnb8T1EaNitn5EVCKv1egukm65I7ou25uoiZ+t9WqbmZ6mKLhTJs4MmHU9aq+P9wXxUEja9LJqjpGQq05uy5bms1FfD5FqWUJxbJxDbgpQxEsimS7iD2vq+aPR1qsnA/Uj1y1H/ns0tdLhOUeDraw5qEPa4ESaraGv8Hy1CVrytVNSMNtBHXaZ2kT6iKBuvs6K4b9igp9LRwlFDwy13d8AjyUTbcMxTGKgEpF4aNFqWkrt7CyTYRYtWZATi0dvyipk3cXzdoGlEyRK57wsf5c0g73Z9K1W2tRat3g5ca6AaUoNQWWeSz/YzENYrf8L07W1rjdJs1sIODTHzNrrfF75sTEyEbHQ8sZCdyneDEBW2qPNsxktqXIMLoTdYeS37Tped77TlL67OZPdFuBlJhUYvhfb1s8qJoEbZ/fN6+v84IVy/2jsvVYH6mqCpwl7zYxs5+EeeCmhX7aEcSr8znnFZGpQBHAOrvYpF2+YU/Vdgffp7j5g52/hJbnS1vaf5v9vS8pBZxAR/eQoMbAyi4d5fQ== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ff731b21-4de3-4ac7-3e2e-08da0971d03a X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2022 06:29:25.8278 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: B4TH/Ph8H0gSf0ZPbrq1zZQuxEtX6sSIRVssH74hWAnaKpLwvY1nphUmlvTiAabUibZ7L1dU9LDs5fHbwIESew== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR10MB1798 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10290 signatures=694221 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2203190038 X-Proofpoint-GUID: 4oZ6RGTnhXNXOFF_gMavcjLIr8l2ndsq X-Proofpoint-ORIG-GUID: 4oZ6RGTnhXNXOFF_gMavcjLIr8l2ndsq Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Refactor the pmem_clear_poison() in order to share common code later. Signed-off-by: Jane Chu --- drivers/nvdimm/pmem.c | 81 ++++++++++++++++++++++++++++++------------- 1 file changed, 56 insertions(+), 25 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 7cdaa279beca..18f357fbef69 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -45,10 +45,27 @@ static struct nd_region *to_region(struct pmem_device *pmem) return to_nd_region(to_dev(pmem)->parent); } -static void hwpoison_clear(struct pmem_device *pmem, - phys_addr_t phys, unsigned int len) +static phys_addr_t to_phys(struct pmem_device *pmem, phys_addr_t offset) { + return pmem->phys_addr + offset; +} + +static sector_t to_sect(struct pmem_device *pmem, phys_addr_t offset) +{ + return (offset - pmem->data_offset) / 512; +} + +static phys_addr_t to_offset(struct pmem_device *pmem, sector_t sector) +{ + return sector * 512 + pmem->data_offset; +} + +static void clear_hwpoison(struct pmem_device *pmem, phys_addr_t offset, + unsigned int len) +{ + phys_addr_t phys = to_phys(pmem, offset); unsigned long pfn_start, pfn_end, pfn; + unsigned int blks = len / 512; /* only pmem in the linear map supports HWPoison */ if (is_vmalloc_addr(pmem->virt_addr)) @@ -67,33 +84,47 @@ static void hwpoison_clear(struct pmem_device *pmem, if (test_and_clear_pmem_poison(page)) clear_mce_nospec(pfn); } + + dev_dbg(to_dev(pmem), "%#llx clear %u sector%s\n", + (unsigned long long) to_sect(pmem, offset), blks, + blks > 1 ? "s" : ""); } -static blk_status_t pmem_clear_poison(struct pmem_device *pmem, - phys_addr_t offset, unsigned int len) +static void clear_bb(struct pmem_device *pmem, sector_t sector, long blks) { - struct device *dev = to_dev(pmem); - sector_t sector; + badblocks_clear(&pmem->bb, sector, blks); + if (pmem->bb_state) + sysfs_notify_dirent(pmem->bb_state); +} + +static blk_status_t __pmem_clear_poison(struct pmem_device *pmem, + phys_addr_t offset, unsigned int len, + unsigned int *blks) +{ + phys_addr_t phys = to_phys(pmem, offset); long cleared; - blk_status_t rc = BLK_STS_OK; - - sector = (offset - pmem->data_offset) / 512; - - cleared = nvdimm_clear_poison(dev, pmem->phys_addr + offset, len); - if (cleared < len) - rc = BLK_STS_IOERR; - if (cleared > 0 && cleared / 512) { - hwpoison_clear(pmem, pmem->phys_addr + offset, cleared); - cleared /= 512; - dev_dbg(dev, "%#llx clear %ld sector%s\n", - (unsigned long long) sector, cleared, - cleared > 1 ? "s" : ""); - badblocks_clear(&pmem->bb, sector, cleared); - if (pmem->bb_state) - sysfs_notify_dirent(pmem->bb_state); - } + blk_status_t rc; + cleared = nvdimm_clear_poison(to_dev(pmem), phys, len); + *blks = cleared / 512; + rc = (cleared < len) ? BLK_STS_IOERR : BLK_STS_OK; + if (cleared <= 0 || *blks == 0) + return rc; + + clear_hwpoison(pmem, offset, cleared); arch_invalidate_pmem(pmem->virt_addr + offset, len); + return rc; +} + +static blk_status_t pmem_clear_poison(struct pmem_device *pmem, + phys_addr_t offset, unsigned int len) +{ + unsigned int blks; + blk_status_t rc; + + rc = __pmem_clear_poison(pmem, offset, len, &blks); + if (blks > 0) + clear_bb(pmem, to_sect(pmem, offset), blks); return rc; } @@ -143,7 +174,7 @@ static blk_status_t pmem_do_read(struct pmem_device *pmem, sector_t sector, unsigned int len) { blk_status_t rc; - phys_addr_t pmem_off = sector * 512 + pmem->data_offset; + phys_addr_t pmem_off = to_offset(pmem, sector); void *pmem_addr = pmem->virt_addr + pmem_off; if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) @@ -158,7 +189,7 @@ static blk_status_t pmem_do_write(struct pmem_device *pmem, struct page *page, unsigned int page_off, sector_t sector, unsigned int len) { - phys_addr_t pmem_off = sector * 512 + pmem->data_offset; + phys_addr_t pmem_off = to_offset(pmem, sector); void *pmem_addr = pmem->virt_addr + pmem_off; if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) { From patchwork Sat Mar 19 06:28:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12786094 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B7FEC433F5 for ; Sat, 19 Mar 2022 06:30:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242337AbiCSGcK (ORCPT ); Sat, 19 Mar 2022 02:32:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242341AbiCSGby (ORCPT ); Sat, 19 Mar 2022 02:31:54 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7FFD2D63A2; Fri, 18 Mar 2022 23:30:14 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 22J2rfog012483; Sat, 19 Mar 2022 06:29:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=a9t3qV04wjqQfuW8AG5CjaMGqxglyHCbSG6pjMyLSYg=; b=kPMdDbPbiDYQoxyDiflmZn1Wit5SefhxY90GSIeBOa2+Go5QvBjYtOVRyw/NxU1TzI+N Ja10bq4ZUso6ZAPnvza5IGBul4k15E+pb/fUayyz0/OE7kXamptlczxSe5P7FAL3Jcwy YYR5oBcgy+Nsa5jlHbTQe/tzGWrnkH5CrhEPqF/EkblOKP7jzX5ekJhmssckqqdCM2YA oFdX5xk2/GHoNP/R5Z1/nUWlDIPPivTxr1EttN7pKrgeZeqhXpVBaR6nK9ADUkPrTWFM hiEkqtGaw/4UyxeFKGvHl4/pUthqQ0QRnR46/40NTlbj3Ya2smHo/7FiJ1MzgfeIwyLe Xg== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3ew6ss04bu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:32 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 22J6MHPA187254; Sat, 19 Mar 2022 06:29:31 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2044.outbound.protection.outlook.com [104.47.56.44]) by aserp3020.oracle.com with ESMTP id 3ew6yyhedt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 19 Mar 2022 06:29:31 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kw4oIhHDwV2meL+R/YJJAzp2HHidcLndLmfBJBRQqyIDv99/KOQOqL5nzOz6Q4wJyfXC/LLQ5Hkyt6fhDHQD4FqbZjxbGMOWLy4MrD2t1LzROGSM8eOshtx2Gs8YHIZJtJMC3/OHWI371LmC5kWm/W/aZRK+rVkgGpBVCi7tpWITQWI6dfOCmWsUtOJClcDewPvbpPdYsqaY2mXV3tonwGXJGBj59EjqVdF184+g8FhE9gfgC7x5weIHXuu6YbNNDVRjtvRLBM+w3cJR5MhjfCzTorfBsSUrpDBI8x+MUkyYYgML8zUYgO/wibm9Hul9QiRIGI6tyOm1KU9G1no9Fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a9t3qV04wjqQfuW8AG5CjaMGqxglyHCbSG6pjMyLSYg=; b=YZNjIhmphIvwiUGlOoHn/Hzr986razHYc31kR1e/KE6t1jLcDz3UfICgsaxbJvIGJQuUN87xZGNwpIYsdxo1oUEaJriMSOuNtvifXQTJiZpYAfca8823CzDxxvX/KKSjvCq/dul1pu4hnV2X2wAb1aGO720Ctzfo5J60CkQYXvyc1xFNPH0uXWNV/pTJhM44oD6aN5Snfyve4q671U2leoFRmBxDIthGYiWPXrolqsCX//DexFQPmO2+c7zLJqVPhJSnJZv54n/L/DV0lgVo7pFbffVLp97zWcrvn8dBIbtnJLav21UgRg2Xe3TXXku1ljH5ALI4yXjxSauRgQ2jsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a9t3qV04wjqQfuW8AG5CjaMGqxglyHCbSG6pjMyLSYg=; b=qRh43LUGKpSiLQ6MEbSs+lpGLkb2ARSVHLG16Rv2wdz3GEa/J9R3gu5yu5HOXyvlu2xW6YA7wvOpeAFZ3SwF2dgy3546ooEsxwGH8+CJj/d+37XVTxv52Ke8g7sizzmsOWDt4BTYiSnliSRsW1FszSSwFkf/tcVzlPMzV//pMjw= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by CY4PR10MB1798.namprd10.prod.outlook.com (2603:10b6:903:126::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.17; Sat, 19 Mar 2022 06:29:29 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::2092:8e36:64c0:a336%8]) with mapi id 15.20.5081.019; Sat, 19 Mar 2022 06:29:29 +0000 From: Jane Chu To: david@fromorbit.com, djwong@kernel.org, dan.j.williams@intel.com, hch@infradead.org, vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org Subject: [PATCH v6 6/6] pmem: implement pmem_recovery_write() Date: Sat, 19 Mar 2022 00:28:33 -0600 Message-Id: <20220319062833.3136528-7-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220319062833.3136528-1-jane.chu@oracle.com> References: <20220319062833.3136528-1-jane.chu@oracle.com> X-ClientProxiedBy: BY5PR16CA0025.namprd16.prod.outlook.com (2603:10b6:a03:1a0::38) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 33974118-ddd2-4a72-9db4-08da0971d2a8 X-MS-TrafficTypeDiagnostic: CY4PR10MB1798:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ciAR7p9FLiDZ3s3welcDtJrS/WCCBJjenYczq3x3GDRSbF7GFsz/o/rqagM22Q30NfdfcqCUmrm3DfXCvMN/IzpMdHj7H04of2osJxnPSx4fl8xqLT+K4fKBEAcff7KLSwn4D6IAv+T5vbjm146bBXSKLHeia2oy5y6CkopOKNn+qAKIFhL1obvAs0G9isLl8Jjo29ia37tToKb3UgA11v8/CqPG9K7Lsy1s+Mva48UWNnqSKmKCou/CPjoN1M9V2kTTogl8iYZ/+pqtL0vdH218MeBJ346aDxaVSgv4UHyU/14ttMzY9OiF7KerkTSmEtLW4zLXwIWjQdfEvCjHhxZhjYJI4omZtdRtSUvAFS2nM8GeR5OIs3HUtLHmIhe6hdGu4iFPHklby99IGsa1Faz2jHpGfPshPLHqS35ZfoL7xOz/LruZjoYJFiM7cgkbPAEJWEiD3ahbVobAHWoxb51NGnnbx9jlZKDxcE56ynZNDwrZx8RTidg5x8wfcnXwFpoj9wCZkjhIb+wDS9AlsVJ5tt4mKr7PUaFHwWB3bNvC6QD1aDvOfy5PHZGdbPtsAs6syKrLxImxmnz0ZyN6hnEdbYeGRs917zT881A7TG8oE/0+lOkYW2BlHC3UdQHStc+b0/D0jtl7pg57idrYHeXL0/8v0p5GaCxrpnsMHb5wGl/TW0JmqkjQQFSYkl2DsS7LzwNHBhTPDyYvKupeOQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(52116002)(6512007)(66476007)(8676002)(83380400001)(66556008)(316002)(6506007)(2906002)(38100700002)(66946007)(508600001)(44832011)(5660300002)(8936002)(7416002)(36756003)(1076003)(6486002)(86362001)(186003)(2616005)(921005)(6666004)(21314003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dek/ppI6IhNcKYqVP4C/gJzE2kBloNsgjm3t17ShlGTGCI4LYTd3fWt734R2GbFjCzHPFkWflK00LNr9yvNwtUQU8XAXAKdM1KgbZ/+kjLK45qCMP8XWLOM4MsSKnhC/N1OOUgVKNCkx8lF13lHVA2MBVuEpqKx0Nu4rAiufxwGqqFS2HgI17y5nSmCyt961CGAkk4/dYkEmSvPHdlGg1GkrB1atasQQTT0JIKmi+Yz7Mw5/1fOYgOdIN+Mh2rv5UwNQY7t7ESDKxlFnCdOeWD/F6xCBwRxeE3n3wPWA17s4yxHDUTVsICtTAs8zMBJNZt77+aiMHn5M2YQRPPmx2Kx0VLTcDN1jHTOq70xx32ek/WGlh1wt6exPSL08Pm8nfjDZCRWHM3c2NYMG4heG7XLx5Zf/NgNubnKuXmEwXuSQuTUgxuy5+HjMAuLJvIVrxx97jsas8PV3VGqLT3K5UBwZjSokN33oWWrardkUfQcb8OwMHmYRmgbIhjXL8nVs4Dmjkrjtj5jCqqj/qJPS5nVetUNl36CKuDsh7su2riedRF1Hn/gLtn52jhj5JSw7w8K1nXyuS8CIezWeBczXRaXIDTqtHmqoVzIUYK1DzueWkhmtkOWCj2QwCdmOdHMwyKv0GKAi0gpb3euIIPuWRL3TRpOwnDEBuDXKa2s15JH181SQm+uGmLwYM5xYvH0/E8rce25+AQ7mI/xN6T8RtMoJzJSh3KXD4wD8p9qXmNxN1wn+iQNi2TermtIaCLpOBjf/7ZK2ApegsuYtdyjgKcprFyC7FacqiVl9LcPXa/COkaTsc1lX9gW3lXmX0b6DnVA8vduyxZhfuU3F+wY6kU96/mdSB/QQS6xhB4iWWZBdSFvwuxGBXakviDn4t00gEgaVUTnzGtIslUrVgEYQuzrfnhKDrzLvhmncMcagZwH4IVq2ekw1q8iKUuMaLB5CTe/PoNAN/H29TlwXjBcBYAsjzdIuMzNxpzTgT2lChr00R6QdMN86DzlWPWcI6xNZmdr/H2inb4Sl8Cr888J6AkLHCCXLODXG0WOiDlNl/C+6Oq7KGPv8r2oVTzLaG+x3vHcmFeKOGFNgXtXenW1CVUDU+iLtSApZ18Y76f8eUeBuLdlEENJeJZFo1tdohQytnnpTXnabZqbudQxwItgLn9f7uW1tMFIg2zm9n8iZNuhXU6+7tRnwqvUTYSiOIsQBFJRtPyq5jNcinrazXHsuL7MU7aCJpvqBnuBzS2I8if6Yps6tbHh3UZa7LmSwBPOt7dYmW6WcaYmq1exxubZ8XGqfatQRV3JuUOxLs58Rznf2E62EltswcY6ALqDLfUWNoIOM/2AiD9QAns7/jRtR0VE0b9ozWDwIUgxGr6b7uqEfjTZzHM7P76QhNWyturdCTdTAyIzzR97QHmpYRgDiPA993zsA+jn8EWricDHVir3Lfr3/+BlrYjPLWNWrQ8apEOCFzYdW9en67fUX512BEA== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 33974118-ddd2-4a72-9db4-08da0971d2a8 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2022 06:29:29.7504 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zGqecdbL3+HzzpFFfHZeJKFe1/WMx/JqhHEhivfmwTKA9yMkH24+wdJhnAr7mN9i2+HQFLaDqZRclhaZphTcFQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR10MB1798 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10290 signatures=694221 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 malwarescore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2203190038 X-Proofpoint-ORIG-GUID: 3KeeVu4cox7pGNRbO06qvrn__SyZQjiZ X-Proofpoint-GUID: 3KeeVu4cox7pGNRbO06qvrn__SyZQjiZ Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The recovery write thread started out as a normal pwrite thread and when the filesystem was told about potential media error in the range, filesystem turns the normal pwrite to a dax_recovery_write. The recovery write consists of clearing media poison, clearing page HWPoison bit, reenable page-wide read-write permission, flush the caches and finally write. A competing pread thread will be held off during the recovery process since data read back might not be valid, and this is achieved by clearing the badblock records after the recovery write is complete. Competing recovery write threads are serialized by pmem device level .recovery_lock. Signed-off-by: Jane Chu --- drivers/nvdimm/pmem.c | 49 ++++++++++++++++++++++++++++++++++++++++--- drivers/nvdimm/pmem.h | 1 + 2 files changed, 47 insertions(+), 3 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 18f357fbef69..c4fbd9426075 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -321,15 +321,57 @@ static const struct dax_operations pmem_dax_ops = { .zero_page_range = pmem_dax_zero_page_range, }; +/* + * The recovery write thread started out as a normal pwrite thread and + * when the filesystem was told about potential media error in the + * range, filesystem turns the normal pwrite to a dax_recovery_write. + * + * The recovery write consists of clearing media poison, clearing page + * HWPoison bit, reenable page-wide read-write permission, flush the + * caches and finally write. A competing pread thread will be held + * off during the recovery process since data read back might not be + * valid, and this is achieved by clearing the badblock records after + * the recovery write is complete. Competing recovery write threads + * are serialized by pmem device level .recovery_lock. + */ static size_t pmem_recovery_write(struct dev_pagemap *pgmap, pgoff_t pgoff, void *addr, size_t bytes, void *iter) { struct pmem_device *pmem = pgmap->owner; + size_t olen, len, off; + phys_addr_t pmem_off; + struct device *dev = pmem->bb.dev; + unsigned int blks; - dev_warn(pmem->bb.dev, "%s: not yet implemented\n", __func__); + off = (unsigned long)addr & ~PAGE_MASK; + len = PFN_PHYS(PFN_UP(off + bytes)); + if (!is_bad_pmem(&pmem->bb, PFN_PHYS(pgoff) / 512, len)) + return _copy_from_iter_flushcache(addr, bytes, iter); - /* XXX more later */ - return 0; + /* + * Not page-aligned range cannot be recovered. This should not + * happen unless something else went wrong. + */ + if (off || !(PAGE_ALIGNED(bytes))) { + dev_warn(dev, "Found poison, but addr(%p) or bytes(%#lx) not page aligned\n", + addr, bytes); + return (size_t) -EIO; + } + + mutex_lock(&pmem->recovery_lock); + pmem_off = PFN_PHYS(pgoff) + pmem->data_offset; + if (__pmem_clear_poison(pmem, pmem_off, len, &blks) != BLK_STS_OK) { + dev_warn(dev, "pmem dimm poison clearing failed\n"); + mutex_unlock(&pmem->recovery_lock); + return (size_t) -EIO; + } + + olen = _copy_from_iter_flushcache(addr, bytes, iter); + if (blks > 0) + clear_bb(pmem, to_sect(pmem, pmem_off), blks); + + mutex_unlock(&pmem->recovery_lock); + return olen; } static ssize_t write_cache_show(struct device *dev, @@ -520,6 +562,7 @@ static int pmem_attach_disk(struct device *dev, if (rc) goto out_cleanup_dax; dax_write_cache(dax_dev, nvdimm_has_cache(nd_region)); + mutex_init(&pmem->recovery_lock); pmem->dax_dev = dax_dev; rc = device_add_disk(dev, disk, pmem_attribute_groups); diff --git a/drivers/nvdimm/pmem.h b/drivers/nvdimm/pmem.h index 823eeffa7298..bad7d1b05691 100644 --- a/drivers/nvdimm/pmem.h +++ b/drivers/nvdimm/pmem.h @@ -24,6 +24,7 @@ struct pmem_device { struct dax_device *dax_dev; struct gendisk *disk; struct dev_pagemap pgmap; + struct mutex recovery_lock; }; long __pmem_direct_access(struct pmem_device *pmem, pgoff_t pgoff,