From patchwork Fri Apr 22 22:45:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12824264 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 764A62F59 for ; Fri, 22 Apr 2022 22:46:19 +0000 (UTC) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23MKYfT9024754; Fri, 22 Apr 2022 22:45:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=hTwNcoiHOD6AK8RdXhOLZVt6JHbSXa99IWo9CBHeCDQ=; b=yOVqzispV3Sx4r3tmU9+gHxQkJvC8Q+GHl6UJzMcmXwKEy+Hsz3Nq4Zxiz0ZC36apR4J FwpLjYOvS2b4BOE1S6nHNeI9diMCXHxS/ViUF/APFYj9H2ID4NEgLf1dpZQCUqQ6RoXP WdOLJCR4TNgV+2+Wtnl7PYMhJnqE+Ch6Z/ivnJIYHy4BkNVNeRsjl4Vv57PTzavoyW7w M2IyJ6izDTrvUdA3ZBwl3iPKXCEyj+8hUt5XT48BdJ3gAEB7PDN17PXn7Qxw2ObJozg2 RYZwAc8tBwMoMOMCEZWAXcqKGEagw6AsQg3FrF9x6biSz4tH8Wvsv8qb6wwRckq9QRFh CQ== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com with ESMTP id 3ffnp9qdhb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:45:52 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23MMewWS005311; Fri, 22 Apr 2022 22:45:50 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2174.outbound.protection.outlook.com [104.47.57.174]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3ffm8bpst3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:45:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Yh1VDvf6uCaaG9FxnezJTYinTqynG3BKjjNh4LoRVy22jMFrB04CLC+mT3D+3ag6p6j21yIcDIXekoubkl2n7dcYM+66uQG2AhupK273cHVubHAUDT25yGn16rlIkglFuBcZcLBXEoFH7Wgo7J7fAG8KxLerPFam//AD4GzbHEJZpWX9IX+vahxaCbIbH0g/Ma43MzgcKPQMDOJsgzqCdz+hyFA3nRgoTlZ94Poxr2QiAkNUlVA+UmdLSrcCo7gRVE6qDErNSAQY5q8WP41e03bPiOdiNhHIA4c01y6HcLlmMtKVUStzDv26haPeJYPlgZqUem7vJQVyOvnJ/nc9BQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hTwNcoiHOD6AK8RdXhOLZVt6JHbSXa99IWo9CBHeCDQ=; b=UepK+oaHs5f4B31nIz46B/gshbTHrKxtZp6AUa8s8uhhc3AjcaC21a3VyPyMJ1BkkJldMr73ep0qE+WSzi8ude0+pPdDy3nwl5QPC3bWYYe3TF1vwNRH49v0BVkZ1QEKKRL+HsvfeF0sI/0AEYFJzxLjxN0aUZM8TgxkyihKj8H8N729eN1kFqhSYU9CJT67hg3ftiWn8VoXSh7hUtAZ1EwqPWdmv7mY1sc38WBi2rT+IW4IJNbdEXFCluwffsuNRx90h76g9nqc7LfOopA6gETJiUQw81diEzUQ6ZhvcuUTnYwo4iR4QGuyNxNUHxudGoOnBWZ3skgDiqi8g6XiBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hTwNcoiHOD6AK8RdXhOLZVt6JHbSXa99IWo9CBHeCDQ=; b=b5SkkkMywHHmLhGHn4TqQwW/xdiFv6D6hQOKRLO0XrwLHguyOSSP2EJvHcsBb0lXnakutA8pS51cY72AcfIADR4btIURbZVML0vxJH/QuL0s5rwoCFWu+3DkqCm1qjtXzRKlteoxUaYUhi1NypGYJAToltnj+jkaUUnVF0gHXnY= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by BYAPR10MB2550.namprd10.prod.outlook.com (2603:10b6:a02:b1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Fri, 22 Apr 2022 22:45:48 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e%8]) with mapi id 15.20.5186.015; Fri, 22 Apr 2022 22:45:48 +0000 From: Jane Chu To: dan.j.williams@intel.com, bp@alien8.de, hch@infradead.org, dave.hansen@intel.com, peterz@infradead.org, luto@kernel.org, david@fromorbit.com, djwong@kernel.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, x86@kernel.org Cc: vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com Subject: [PATCH v9 1/7] acpi/nfit: rely on mce->misc to determine poison granularity Date: Fri, 22 Apr 2022 16:45:02 -0600 Message-Id: <20220422224508.440670-2-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220422224508.440670-1-jane.chu@oracle.com> References: <20220422224508.440670-1-jane.chu@oracle.com> X-ClientProxiedBy: SN1PR12CA0057.namprd12.prod.outlook.com (2603:10b6:802:20::28) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a8aa2a83-9940-4263-57c7-08da24b1d844 X-MS-TrafficTypeDiagnostic: BYAPR10MB2550:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LO5z5Wm/1zJjg85z3zmIpfAq/rfTrZNxTHMCb9bgtIyyhYz/XBIf3knCn/3q8K/R+bUQId183VwdwE+tZzinwkbfoLvsbf1Q81UJ4gMBhzt6iUitqw4NpKJBWbiHQrA2UXPEYAm89dIZCRIQGXQ6ZLGlfWC+gvXq7iYkw5PItOoMxjGb1ptzGXGS9Q3vBcRAYHnF4D5Ron04EvC7e/GxxAcrWN3o1m+4fCforbI4awSRy30bDnOhHXJ0ogrfwpqV3g5XKqd9VjjSoLmvxzvqKF9gTulu3hpU+t7BJ7ScjJMWU/bjlSJGKovE4tNYIRomfId8kZ2O9tev3UNNnsvVsiTFJbT/pAF1dwaoX7cZVy5GPM/AEORsPljaSKttcIC0iYyPOCcwcgw3Pi508edvUqqbDa+w2rUwV0T8+10IOtJAbe8KXCWR7bqQeWL6gEgZ9acc4soqGbs52X8IYefvoVcpc63NJ/4KlEitPIZRxxhPGKtAss2AR6YrBFcw6Ghu2BK2F3O7/H3is/UL9devtJ298zKaexPQh79hYDlRSos6AbnBUtFYpVZXaQubX0UChm7vwCXfInF3vZO5ev13uUwoh9Rxxt8fyFtj3FnYc3PaWqN1ecqpWK0hxY0/BD8VoHWhCI+3XyNwviofeGZ1Gj2kWfF/bYC2OBhbEXfUrzNEiFhDLTTjwQG084gPHStgf0NYEZ8gf72cJk0PH6Kz7A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6506007)(6666004)(6486002)(8936002)(508600001)(52116002)(83380400001)(36756003)(316002)(66556008)(66476007)(2906002)(8676002)(4326008)(66946007)(38100700002)(2616005)(86362001)(5660300002)(44832011)(186003)(6512007)(1076003)(7416002)(921005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: LI4ZXwYs8rmI88QdHZ/A4yj/ti26/5UzBRULKA4P1hbjNKbrqiRzhM7835YJh/JOCSOmQmvyGI2ilrkgMLswyhMiVtz+2vt6fM5fSwJJIV9qLZiZ6tb8g4fd1F+V2LKvVqEGmu25/nEFSMAMu9y/Wmo6+N8rkgKy8eQLUr0YyEFLbiSiVVIyOYOSGrfwkunLcDKnNClUDnMbnKFPGdaeTkGZqQqZUQ6ptm7UwsttGefYyUcAz5S4S7ieAJY5p52jbp+YYxtFvn0dbdhlH4uswAOfo4lR67ju/+80pR7kG/3Li0fk/2A+Tf7FRYZJig1uPl9Q+RDQb4RWk8Irozs+FJmHNjYn5gcmF/cojGl/W9BEiXciAEbyM7SKrJBaokJKxzVXC7critMozjm7euF7IjRFBn7/dBESKWAGKFy7nhpxDQcJ9/StDUKLHNYqjNgq3FiPOOmy4dFQx2i4CEanGp/+7S3+4A5c8ku7k2o+D2mpERNRFyD3yJxVaqv1OVv+Yp3p4rv2dtr8l0JRpgSMM7QgNN5nmTUNSI1NhhL4NQoPDnflodgsUGStZNdICVWvLjCleK61+69V8WhBMDjugWL6I/5dir8aRqlP4v2I8idptBgHcRGq7ckU46CK6ebx0A1buNRUK62CqK1qlWCUbGNWhKQ5jPnCJ7vrhlQj+Xw0pu+uVY7M2AzvBz0UtzKVmvEFDt3Y4LdBBxWoMvKol0wIy2be9mcu7ea5gH2BtYJKGqQOYqWG/a5oCretnWREv83RStO1Fy0NS2X7Pwra9p7jF/MMQOo/LtLyspz6IZs2IZMtVhw0ul+lfGuVMrkKIo+C9diDsg0wQmCyoeGofJEj+lTaSXOPTvVSJTwqcXaTYo+QxAbdtRkrm27yPmUfAkCHjyB3PAWjW3OQYoNjoj0jRx9dU6jnCJwPgEUCe4xBv28ovCEvrlSe7vfThsXS/iBH8iAdeYQ4YJ9XhQF06PphXANN043HnL0pmpvupR8CcONf5LIc2W0rGAWirwMY9vdF6baJIaw+iq5skHhMaF0RIC6lnEacKUCY1gwRMhSG782P344QmobFCMQQ4xOA1hO7yMaPnfN2vym6oS9WTFvFOTd+atclv8IzB49SP+hlp2gjGhBdU8oqIZGO/P2NKZLonFucILNU9BCZkQiaVYvkodBrw3AvPsd5td6K1CSnQ5sp+9QwP2IQ1fY0j6sPYoZj6r5zuZpMmwygFD6tgMg8TWr7/17sC7Jq77Pmpczs/B5jtfLJ/6sA1gfZEXm0YE3KmWWLDosiVX/N6vlldqG4aIvg3FAKmi2RqFjvk2bFMA9qMQitNna1W6xkKdQ9vBF82kTaQV27917vhJ2Cx7CnSHrq53iw6wGdCMgxtm2zcXHwk/LY+kexdWPTjPgZ2NdzoTn/fNbC+yuRsvbh1dPfhZSfBnH+YmJxKV0oG+lUX0Q9wiZTkcEMie61J97cbGyyhl8uLhyS0vHrGHXmKExvwi8o+/YHlhWzHxWdA3k8ZJoDiD0g4o9Rng96Cwiwz/6c2LIv5pUyOWISEWW6i+j7x7mteH94rBj44d33pe2rWxVQKoscjxPYxm+43D67ARDPTln+RbBefA07u7JE2f8fhMm044hEXqHXyzrfChlPHC1TLVb6vU/fSIspVV6hNR1Gm07RTLRAAGzSx4i2Tojk0xsqLIqoj589/lbOmXOw5vU/dLRYjjlklDUZBlJ7BWH92eGMlDe3MjZUpgaoT1aM2gvuEgK++Y4b/xptGw0= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a8aa2a83-9940-4263-57c7-08da24b1d844 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 22:45:48.4858 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ECkFmRlDT6XwM9BRZMMzx2IFxkDGTRFFzjjn8D+jn04uxcNCyQs3fYEFkpY1JQ5zzC1OPTkUoHBtoEpzT+Ialw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2550 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-22_07:2022-04-22,2022-04-22 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 bulkscore=0 suspectscore=0 mlxscore=0 spamscore=0 mlxlogscore=999 phishscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204220095 X-Proofpoint-ORIG-GUID: Y2MmNMVRgnjk7fj0aoz3OJRLn1hv-Pxj X-Proofpoint-GUID: Y2MmNMVRgnjk7fj0aoz3OJRLn1hv-Pxj nfit_handle_mec() hardcode poison granularity at L1_CACHE_BYTES. Instead, let the driver rely on mce->misc register to determine the poison granularity. Suggested-by: Dan Williams Reviewed-by: Dan Williams Signed-off-by: Jane Chu --- drivers/acpi/nfit/mce.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/acpi/nfit/mce.c b/drivers/acpi/nfit/mce.c index ee8d9973f60b..d48a388b796e 100644 --- a/drivers/acpi/nfit/mce.c +++ b/drivers/acpi/nfit/mce.c @@ -32,6 +32,7 @@ static int nfit_handle_mce(struct notifier_block *nb, unsigned long val, */ mutex_lock(&acpi_desc_lock); list_for_each_entry(acpi_desc, &acpi_descs, list) { + unsigned int align = 1UL << MCI_MISC_ADDR_LSB(mce->misc); struct device *dev = acpi_desc->dev; int found_match = 0; @@ -63,8 +64,7 @@ static int nfit_handle_mce(struct notifier_block *nb, unsigned long val, /* If this fails due to an -ENOMEM, there is little we can do */ nvdimm_bus_add_badrange(acpi_desc->nvdimm_bus, - ALIGN(mce->addr, L1_CACHE_BYTES), - L1_CACHE_BYTES); + ALIGN_DOWN(mce->addr, align), align); nvdimm_region_notify(nfit_spa->nd_region, NVDIMM_REVALIDATE_POISON); From patchwork Fri Apr 22 22:45:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12824266 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19E893D71 for ; Fri, 22 Apr 2022 22:46:30 +0000 (UTC) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23MK9YMg013493; Fri, 22 Apr 2022 22:46:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=IHhSB9rC/ITJAjPIc6g1lJg0i8M4cMw7xJTu55a52Q8=; b=lVtX8U+EePzCM82K/eaS8xfHeJ7qL7ItGspIIdNLajokZPRvvVi7+PfLUGY9BRjYYnKy GAWUpfVVdGv6WCw1mfejoRm2lfOTe+WC8BlJHuzHDQAYTH28/XnDebKnVUrnraSRfUTh PG4cJCTRi7CULdogjmL/THfg9mEDblZ4DPwlGIENYH9NwdXBOYDsRZAsIR9BLgDjpVAT SqY26wd+ZIw8oUiMsEF/2NusXSmAaCVRt/ywyuRDSuX5aUDdkfVin88g+9NSGXEe8XiN 2UGszaaiD9p1TNltL2AbzmS9hexBIoLXRG1cFn7Lk4qaeK/W1H60mlWJgXNcJE4nvPqk pA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3ffndtqxcg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:07 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23MMg0Ws008543; Fri, 22 Apr 2022 22:46:06 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2174.outbound.protection.outlook.com [104.47.57.174]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3ffm8e9mg9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U2x/Bl/x0abmLrex6k/Gls3TaYtQe0bPzFPw/zK4BdEMDd2HmAPfR7g+JBfWTGrndo1QdNEc10EUbHdz8Rsi7+vC7cWeWQr4IKgSpiUboQsr3EtX1Zl8lSxd08Jkagx66X8qds4+Aw5jKNQX41l/r8RjN9OFBQVkPHG3V3wz0ZlAW/m/EzCdq/wnzRwDuMPLwx9X90vOgjqNM9En5VvQ9+5gDoNkr1SHOyiEdpQOGI7iBJAs6tEwqD/2ZBQqaxYugC4/iGkFv0VxLFuXc5VNCoiGx7Ghmithtwu5mHyWdraAfKCIawb0BxcVHKS3wkQ89IvUj2qHMsxKFmGjKDri2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IHhSB9rC/ITJAjPIc6g1lJg0i8M4cMw7xJTu55a52Q8=; b=hhMmuvbAyjeeXMEqlxWlLb1C8aQqLDIRD6Sv1GLDeDJ5m6wXXohtoCZA1dprXbEZzqyS64NdEWvi9QM1cY5EJvgLjYHjcZbYE6WbHNHUzEobGJwuu/24iH0Nf/382ZXdHmfezK+E3ehLAiwZGWG93+0adXZ4XlH7GOeTxftMz1QnN9fZxgYI1IlQjzeVD05G8ELW7jPI2wxSDGh1S4OuYoyNGf/4BxoO8AzUpfKaFj1tZh6nybtMJODSXt0cqaLG7h/hQ11KH5cDVoAv3yiXSsnPDUmQT/cx8sZkuyCTttDei503eAh9Grf22h+TpSla2yRx9eLrJ/jflNIg+GPhtA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IHhSB9rC/ITJAjPIc6g1lJg0i8M4cMw7xJTu55a52Q8=; b=blG16PPcIbXVem/cVDoN3nWKyyG6FkR13VUtzT1O8zq5oajrVUM8gbXBAuPmFlYkM5OxsTV1lZzel83HWqz7ASrtb4Ei0y0Q2yGlX0cbmDzR+KhTjfcco0y57WbKEHQBKPTFUfpnHhvoI8XC5wv0Iv8ZtdW2G01P/R/R1/aXHBw= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by BYAPR10MB2550.namprd10.prod.outlook.com (2603:10b6:a02:b1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Fri, 22 Apr 2022 22:46:04 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e%8]) with mapi id 15.20.5186.015; Fri, 22 Apr 2022 22:46:03 +0000 From: Jane Chu To: dan.j.williams@intel.com, bp@alien8.de, hch@infradead.org, dave.hansen@intel.com, peterz@infradead.org, luto@kernel.org, david@fromorbit.com, djwong@kernel.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, x86@kernel.org Cc: vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com Subject: [PATCH v9 2/7] x86/mce: relocate set{clear}_mce_nospec() functions Date: Fri, 22 Apr 2022 16:45:03 -0600 Message-Id: <20220422224508.440670-3-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220422224508.440670-1-jane.chu@oracle.com> References: <20220422224508.440670-1-jane.chu@oracle.com> X-ClientProxiedBy: SN1PR12CA0057.namprd12.prod.outlook.com (2603:10b6:802:20::28) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f35f2d23-539f-4561-9ac3-08da24b1e17d X-MS-TrafficTypeDiagnostic: BYAPR10MB2550:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4We+QocWOwZALHDH0dZz7sw+Sn/8fQEDe+shYQW0tdJ8ws7Wd9y3eEpOxfjKu7h4f9mrLudvMS0fzZuQDLmjvfuAYE5I853iGNv0p+/jdqnJfMm5vAI052R62fGsvP8bjmHoXGsPsBbr75lL+Vso+mV0RPcNnRCsgF7UnceU8k+tCmWrnAQN3qnw25ku3tkopKEk835lYwzAp34//tPehOw0AiTYyxU7zUIwPCMDkzNScLl6HUD+HmE7WXyIwVoNY8LTRiEDz54cwaO8gSZoGkal5Q9Kr4D9UvEtot7YaqwhyVs3YVTeNFizR612vS8dmCGZtaw9rgnLjP0KpfdKofbgW4rQNICPPmCYCofRlcaEf/O8ARBQBWFiEJVwVN3zLzreavX9kT0XtB1AWRGxNFoSX+cavuEfsEj8aOTAbAMNXEzO+m6+L13wlS17ExWulY+ApDaK2bfkQ3H0wOnNjfgJJQxKecE9eoZivE5mGuxavVBs5Ck6iI6D+AJM4IthQLohNRB1JfLhc1az8rbCQCM0yEyyB97kb1Fn/FkjDzENvTPr/Tag1ebNRCz9z+a7PRoPCrf2sY10fjrkhyxEoR+VSygUW2Vs3HAmJrOLk4zWi4jw/mpIcz6ZyJ+cYzeAkn7tYpzbu7DdbUkMeyS+Gb37IYuKELl/8TB7J1jn/8WMqQcS8B+u5tlztmMqZEKM X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6506007)(6486002)(8936002)(508600001)(52116002)(83380400001)(36756003)(316002)(66556008)(66476007)(2906002)(8676002)(4326008)(66946007)(38100700002)(2616005)(86362001)(5660300002)(44832011)(186003)(6512007)(1076003)(7416002)(921005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: id/VrT+U1SyDhRZydrWy7QJ/dDIKreY2u1anPyLWhr00KaBcyZmkVkcNrIt3UuqhvuVO1HLVRBqRUdAONuRKOCf89oKMuA4z7IQHPRtMY2SjCCu1M+T21KaEzZMin4oGpRh/irKYNeqKU2Uz9cRqqHXAocKtMlNDtz5tRON5pcocrR+vP8cFCfM2b1RoTkdGTxzN56QdeOg6y4BSUL412PWa2Stsv3HW5rYp5Ul17Alw33DwvOzyBu5zc9MV6cm0/z+fuhLeFE97+FYAz8t+mP5PWaouIaUTAE6xBg4LCYnJScpTFSIELW3wKFPk+okT8yE+q2Abxn2H5ApwzDugrnwjTKTeav7nxcaakHeDnuWcC6MNUCxAajhKrI7KI+8ghP2pP2GA4BplkWLnKEA5YylqFMXpk8QKCRZ7TcKrPg30ln56txvk/2BnDNaooMNegQjbLTxgrVQJUX9VWMm4E6SVlBIksz9mhgBoB9HHRSJWHQqi5UJ920MZLEtXZxRwS7HgPalyX3oY9MHvVTEJsytgKqMZCQTaGyuBD4zgy3ekE9mhoBAaSilRIuRxAe5ECvc6PVdXparTQRwNL/D2AtFYIlW/w2rJtUe+tZGGPq/dkGkHVyLOxvulVUNwe7eraVyeFadwGHFOXIRdcuaZ85n3kOkX8kWhrga/JfSlcyxiyhdy95/02qd991lznmaKj53AxgEpyt1ftpugWSDkVqzfTmwhUedmdmBifRWvjO7/k39dkwMyeHBszA055BOYkp12P4prlel9T0j+PhxfiQ0o/s8dNwUS5XWSOG473U3eveYNEfVjUFWBHP4UobckSuXXFLfhmsUbWtl1Bn72TTK2Cqo/oGksoeyTMvj+AWPcoI3qjVDLXEUnIBk2jH8X/vZzaAmjq/PGvCjpHJ5eZCptVcJn3kMn/tTSzGM82jVt3+A8OdwkBfhUud6SZtxAyaooxrMA4a+FSaiseqImVqSm8m5DC4JNG05EjWB19FNZImx3nY7wJ7/GGVjVIARIO6PLDKyhMwLX3UKjXBkfx5p5hTTYcLdi4qsXQd8210KWlyFfgKP8PGb2L8meOtiNABYKh6RGDchzN/OvwF8BdXrbqOoIkNN8mKR7wqb8btlGh/GhRLo1OmZbixbs0MOSJNMNl+szASy7cRkbo2aXWykH39NA/TkpJEHBiADu7q0grn/4E/QPXZbdb7LM/IHVd14Eiz50Vwtnd0riTXgKZQjLdG4AgDPouRZQHoM+jgv+GUErpDYnP7fiiOTO+XLsp3cFesg2gsLCs0ktxFCZ9cY3rdrefs5OYEP70vI9mo2O1uyYEermsY7FkoSwWszqNqwmTgDY9xhDFPlg3BwT5ioEb2LME+mJM9cJbYXK8QpGuw1I78LxjxNloMTcholdOMq2//u4FloDTFYDLwMfB/5al3H1UQj8rjwGnYu92BvmjzGz9yNiq2fGaQA0FHfoHzxlpb1AjpHv0IrDFcTtSbIO/X/B/mztYdkfEWabup9nrEzUcmmCuN1nzBkhsGN1ONzmElPuS/FtQwGR7xxDlttNT40ZgtSJvtY/hvhOLm88yWan/2De2Kjj8D4/22T41aDhb1tV8sdnqi/4kOLLXB0YGULYCSwMGufdT//Csy8zTAZHtXzv3R2OCR8h1oNm/lmIwNO7nZWMzLUVGd3kGmpn5yWCGdv0aWLYhzZKaBcbT0A9OMVwAssdtamHR0IT7OvDNAVhtSaSpScgr8wgHkGDU492qry5zwIz0b5ontk= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f35f2d23-539f-4561-9ac3-08da24b1e17d X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 22:46:03.8669 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QuaaULRiDxBRRgvkBWiJJp7E5K1qnqVrzQPJNlV90nTicGlTz6I+z//S8z1DXGz7vFDp+MKrGySgLPixGi+e7Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2550 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-22_07:2022-04-22,2022-04-22 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 malwarescore=0 suspectscore=0 spamscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204220095 X-Proofpoint-ORIG-GUID: UfUbv9M8Bs3zm5Soq_hiqXyIelEP5Wy_ X-Proofpoint-GUID: UfUbv9M8Bs3zm5Soq_hiqXyIelEP5Wy_ Relocate the twin mce functions to arch/x86/mm/pat/set_memory.c file where they belong. While at it, fixup a function name in a comment. Reviewed-by: Christoph Hellwig Reviewed-by: Dan Williams Signed-off-by: Jane Chu Acked-by: Borislav Petkov --- arch/x86/include/asm/set_memory.h | 52 ------------------------------- arch/x86/mm/pat/set_memory.c | 49 ++++++++++++++++++++++++++++- include/linux/set_memory.h | 8 ++--- 3 files changed, 52 insertions(+), 57 deletions(-) diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h index 78ca53512486..b45c4d27fd46 100644 --- a/arch/x86/include/asm/set_memory.h +++ b/arch/x86/include/asm/set_memory.h @@ -86,56 +86,4 @@ bool kernel_page_present(struct page *page); extern int kernel_set_to_readonly; -#ifdef CONFIG_X86_64 -/* - * Prevent speculative access to the page by either unmapping - * it (if we do not require access to any part of the page) or - * marking it uncacheable (if we want to try to retrieve data - * from non-poisoned lines in the page). - */ -static inline int set_mce_nospec(unsigned long pfn, bool unmap) -{ - unsigned long decoy_addr; - int rc; - - /* SGX pages are not in the 1:1 map */ - if (arch_is_platform_page(pfn << PAGE_SHIFT)) - return 0; - /* - * We would like to just call: - * set_memory_XX((unsigned long)pfn_to_kaddr(pfn), 1); - * but doing that would radically increase the odds of a - * speculative access to the poison page because we'd have - * the virtual address of the kernel 1:1 mapping sitting - * around in registers. - * Instead we get tricky. We create a non-canonical address - * that looks just like the one we want, but has bit 63 flipped. - * This relies on set_memory_XX() properly sanitizing any __pa() - * results with __PHYSICAL_MASK or PTE_PFN_MASK. - */ - decoy_addr = (pfn << PAGE_SHIFT) + (PAGE_OFFSET ^ BIT(63)); - - if (unmap) - rc = set_memory_np(decoy_addr, 1); - else - rc = set_memory_uc(decoy_addr, 1); - if (rc) - pr_warn("Could not invalidate pfn=0x%lx from 1:1 map\n", pfn); - return rc; -} -#define set_mce_nospec set_mce_nospec - -/* Restore full speculative operation to the pfn. */ -static inline int clear_mce_nospec(unsigned long pfn) -{ - return set_memory_wb((unsigned long) pfn_to_kaddr(pfn), 1); -} -#define clear_mce_nospec clear_mce_nospec -#else -/* - * Few people would run a 32-bit kernel on a machine that supports - * recoverable errors because they have too much memory to boot 32-bit. - */ -#endif - #endif /* _ASM_X86_SET_MEMORY_H */ diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index abf5ed76e4b7..978cf5bd2ab6 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -1816,7 +1816,7 @@ static inline int cpa_clear_pages_array(struct page **pages, int numpages, } /* - * _set_memory_prot is an internal helper for callers that have been passed + * __set_memory_prot is an internal helper for callers that have been passed * a pgprot_t value from upper layers and a reservation has already been taken. * If you want to set the pgprot to a specific page protocol, use the * set_memory_xx() functions. @@ -1925,6 +1925,53 @@ int set_memory_wb(unsigned long addr, int numpages) } EXPORT_SYMBOL(set_memory_wb); +/* + * Prevent speculative access to the page by either unmapping + * it (if we do not require access to any part of the page) or + * marking it uncacheable (if we want to try to retrieve data + * from non-poisoned lines in the page). + */ +int set_mce_nospec(unsigned long pfn, bool unmap) +{ + unsigned long decoy_addr; + int rc; + + if (!IS_ENABLED(CONFIG_64BIT)) + return 0; + + /* SGX pages are not in the 1:1 map */ + if (arch_is_platform_page(pfn << PAGE_SHIFT)) + return 0; + /* + * We would like to just call: + * set_memory_XX((unsigned long)pfn_to_kaddr(pfn), 1); + * but doing that would radically increase the odds of a + * speculative access to the poison page because we'd have + * the virtual address of the kernel 1:1 mapping sitting + * around in registers. + * Instead we get tricky. We create a non-canonical address + * that looks just like the one we want, but has bit 63 flipped. + * This relies on set_memory_XX() properly sanitizing any __pa() + * results with __PHYSICAL_MASK or PTE_PFN_MASK. + */ + decoy_addr = (pfn << PAGE_SHIFT) + (PAGE_OFFSET ^ BIT(63)); + + if (unmap) + rc = set_memory_np(decoy_addr, 1); + else + rc = set_memory_uc(decoy_addr, 1); + if (rc) + pr_warn("Could not invalidate pfn=0x%lx from 1:1 map\n", pfn); + return rc; +} + +/* Restore full speculative operation to the pfn. */ +int clear_mce_nospec(unsigned long pfn) +{ + return set_memory_wb((unsigned long) pfn_to_kaddr(pfn), 1); +} +EXPORT_SYMBOL_GPL(clear_mce_nospec); + int set_memory_x(unsigned long addr, int numpages) { if (!(__supported_pte_mask & _PAGE_NX)) diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index f36be5166c19..683a6c3f7179 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -42,14 +42,14 @@ static inline bool can_set_direct_map(void) #endif #endif /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */ -#ifndef set_mce_nospec +#ifdef CONFIG_X86_64 +int set_mce_nospec(unsigned long pfn, bool unmap); +int clear_mce_nospec(unsigned long pfn); +#else static inline int set_mce_nospec(unsigned long pfn, bool unmap) { return 0; } -#endif - -#ifndef clear_mce_nospec static inline int clear_mce_nospec(unsigned long pfn) { return 0; From patchwork Fri Apr 22 22:45:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12824265 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 634F633EA for ; Fri, 22 Apr 2022 22:46:27 +0000 (UTC) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23MKYfTE024754; Fri, 22 Apr 2022 22:46:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=5i4EI8sOaKYGMRiOgdA9u2QARmpRrMcnmSxVoY2LkP4=; b=o42ZWv7FyijEytwOduq6MN31/lLnucfsyGAtN3YXYsumbC90SjQtgbvFuJsehcGqm8Cz z9AOMo2+Ao7X5iWaMq7m2qt5kv6lhr0fwYTuiGRr7P2sWvO3m6MD3FgWp9jz/bU5n59G QBfoyxGtlri9u8x0EeezzFpg1pa1TfFRZPLj6FonoHdYPuWrCQlIK+xTHWeCTgfUAGw5 S4oqbcn08kCFeZBjRhrT5KiyqN1nlKIaLSwl7QXdQKFC1DyvC184G6brOXOjZXIRdy9+ CyXlO7stty4FfweIZh7QgegE70ovtFugAS8HfoVgO86sP8v8/iPNdvkyFFSdHM5Bxd9B dA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3ffnp9qdhw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:11 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23MMeow6007324; Fri, 22 Apr 2022 22:46:10 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2168.outbound.protection.outlook.com [104.47.57.168]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fk3awf5rt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jluAnWkr8fi2PqjjvnKXsrrWeIhXNg1cIbR7gTi1b1Ond7nt0W6XXkCyUEgwlCU6E3Jetj+Z9/AmbDLz6RUbzziKyAg7PexfBXoHwULxc6Y3+tuVxYsGr2KucnEu87kHC0RcMbhFoly2Ga/2Q0ayaofzCFkUP2PtjqhR5yDEeSO86n6pgBF7uZGBM3DUEqwyqjUbXoABu/juBhpPKHmATI13qNYBz0dDdtI+oeyFTGIf+azeMp5CNEcItAThh5TOqNibMeXBsmB7IdumEf5dbjbFMFn7Eb/gZrxoB7bbuIYiwl65UsXqVb/RgxVIBTTgfpDvp/gqCgYwVeYBMVDI2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5i4EI8sOaKYGMRiOgdA9u2QARmpRrMcnmSxVoY2LkP4=; b=iqmwzZOq39FHh87Y0W0Zp/oSQZeSqYS2DBU0REqmWMEC3JVaGrchcj7v2yZx7XJ1lMKM7Ys6djICtkOS8oX4mTFCP+d7Ee5C9yTjMd6ljBlhykjz6nRJsWIrY3+yTAi9vQXe2mgKGm2bZoSwg18mGPjBHBgCnFk7Tb/Crtbc+H6xkqzIcM7Mb2AUY0RtleT06DXCweAZDTstpObm6b3O+U5aRGJ5XldE/6x/cIgTzUqIg4Lr+8mCevyy+jRPQGYaQ0kQn1Ho4JfzSDPypaaPXGQKaXkY6Gnjqx/ZsR5Wpfitn/hABPFLPx7k9/bp9NDm/J0ClSnXoQuVVj16KhWUJw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5i4EI8sOaKYGMRiOgdA9u2QARmpRrMcnmSxVoY2LkP4=; b=SOcm6tvhKj2pnB9of0kbDYlTAQX105FLnDmvoQbW/h7nVoG9AYXiKBLe1EYjA7VwJ4LsF0W6t5KNSV5puvDRoOnnVR5WduWwLgYpIenDpLSnJhhgRY0PRXJbZls6Nw/g/yoNeukN1uODiXkL06pYAgoRajDk/0PNo2MqZ0EWMU0= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by BYAPR10MB2550.namprd10.prod.outlook.com (2603:10b6:a02:b1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Fri, 22 Apr 2022 22:46:08 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e%8]) with mapi id 15.20.5186.015; Fri, 22 Apr 2022 22:46:08 +0000 From: Jane Chu To: dan.j.williams@intel.com, bp@alien8.de, hch@infradead.org, dave.hansen@intel.com, peterz@infradead.org, luto@kernel.org, david@fromorbit.com, djwong@kernel.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, x86@kernel.org Cc: vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com Subject: [PATCH v9 3/7] mce: fix set_mce_nospec to always unmap the whole page Date: Fri, 22 Apr 2022 16:45:04 -0600 Message-Id: <20220422224508.440670-4-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220422224508.440670-1-jane.chu@oracle.com> References: <20220422224508.440670-1-jane.chu@oracle.com> X-ClientProxiedBy: SN1PR12CA0057.namprd12.prod.outlook.com (2603:10b6:802:20::28) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 42d3d145-73c4-4cf1-ce75-08da24b1e41b X-MS-TrafficTypeDiagnostic: BYAPR10MB2550:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q2czr/Ny49kg38MYO4uc2+8AhMIQ+gMQ+QH2+Zo8lJbdIGeUYcX2jpoO2A5OzakCyh6TUsINcfaMsnv42bPO/KiAKXhfKMswv3k8hrGDe2/ey595IOfSINmCDxQCckhNglD4Y4ByX94JvyxnKCy2otC17XTMhE1xk4ygterNI6gNNcIH+UGNn0v0892OrLLl+7oNAOF00GndL8mPhMI3sWHOoGQujb2u2OGkfzjVNKyllsJYZvLL3nqIcRHmVjDK5zPA06ayPplW8iN+4MB7jOIomy9bZGJcGYAiSXBVh7i7XQltcrNICG2m2+TI0il73QsHdM86keVgPyaXnoh9TSBEN8wiYHjKpxzgbj3FtQGaoow8xNNyYevUbOmArLsQGoSRM5JLWIwukrJZAWOF2xuWLEcF9g0N7scaPjW//Ah4Fm/3EKezfdBhM5JB3hq/3ENFrvlMY9tqIUENFbaRTIDJHGMYbHahwfdLIEAZD5DuQuGrItmB9TLckAbtyVg9UQ9aTk68izemiAf6wsfw2/EBxR1XW6z7xeceEyW44FvCcmZznIACBT+ACLXmwZ9vTtE/EVetHZSyvSGXES5v0jzlfTpyNaY7+Vf+jSQW03WnSy49+0m59Mol0nap1QUal0s+IANL1Ph56vgPv2MU0lC2AL918ymFSRO6u5qKdo5kxUSds0JdnhC98K3MGOBq6xhgl1YW6rsKlA3CU/QW8SzDMxkcvEqOvlQDN1C6uSpTlylLypVzqWrLF+yMHrUvcBNkcXgnTPvDAYG2ZUUOOw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6506007)(6666004)(6486002)(8936002)(508600001)(966005)(52116002)(83380400001)(36756003)(316002)(66556008)(66476007)(2906002)(8676002)(4326008)(66946007)(38100700002)(2616005)(86362001)(5660300002)(44832011)(186003)(6512007)(1076003)(7416002)(921005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: tQDP6ygUGGVY3lsOF1Exdr5KAVlVXhWfO+DohmT3ra8tt6Ry3Xk72w9YmfZ1xR3rT0Xp3Ps5JMr72l5zLYLeGJkHCJjzZdvMLXhiofsIOyy1kPHuocfQ+FPCK3u5+dzYiPmf9dMAXb/6BpSrTet+DJI3utNNixcj1OTwJAut4l7XX3/eUTuwuxiLQGuUu2BiSIuK2/Eu9yxWBiAkArk/GGzZ3yUl0Tsdjo6KC2J0l3ezgwN3l+bIb3Q1lADNSIHqKHXzkrvCQduRa4GMP1D0Ma2XNbPMKtNq1sFxJtlChB3vME8O1Ygp+lmbpuWtVAc0RrWqApLbqZ0ma4FB0nesa61wFv1XpQca+MszMT+uPRG2qd0fCQd9zFd7ND7FpOyuZpEyDGvIaQtq/8xY0e+CwtmEpHNOnJSCFzbxWZhDOHGOA4bjpKPvzMSQy7h9rqZcN5T/O1UZcZxXDZa4xw8zINNvUiwP6g8eWvZcnIfIBv+3K7p4Xt1Jph7973Nrv2iLeXch+PcBcTjsMw3IqPXJ+paMpaxcMHEGFzG4jIjfpWluiGxr3yGZi3Uy6FEvO+q2TH2NQ6ixrf1kq1LK540d5dc3QOM5GB6uk8SwoVgJAf+IRXAm6lcCcJzUZ00fRjza17k6UMnSIIbiaviMiKzgS1JksXFIwaXEh/c9H3LC0M9t8QUhqt5H4rYmzD9AWbrC6Qd9rNntvA8HodyHhEFZJnFQx4oqYhB0wEZF0e9oASlN86ncMh2YUupv3YxDvokXHooLaOKZQoWBntTPu3teFqRVSaw6+G15y0/G1jU38uLBBDh13L8rnB/YHzEmqcmpWmF6wCkXb9LtLyMunasnMmpoFtupbmCk8urjrEjKQW4V54Dcurrm5h4V7UymCspnpLTCyD0uZ2ste/NleFnlBKFzYvHRjOYDQjAYw3u/skS3wbahAg5GAtN8z23ImG0/NDnnFTXflOsKXeTMcMQlqhmZDgO0p13/V7p7BnyIjIekErQ9/D4VisnDBpTiOvkAr8aE2rvZF7NMrOq0cR3rh4DeqsUaXFZlXIezeOhdj2yq7UPq4H98Xepl8xB+RZaFlhpije4U091UG4+0g03GwC0ES8hNZBCHlDCPms0m1Y6sEgHVmJvgbZOLpWYoY+r8fxuMwHfkVad+gzG+u6srD7MO+1fUFpFZZKILks2U+Z0qiwtTS9DPqNSL1hRRbgXHXnVe2XkwsOPSVzRUAl72cIURnz0GQkkpiyl4PtM/T/WA7+HCO6o0UnRGoC1xe4OJt509XkNaYa6/n+M1ljqiVAB7+gYyAj+fw2PowspptJMrDrEmfGv0UN8eWcpOkYTdSIrEXCj3AbIOuYswvgrwD74KcCbgkzUm351sp9l7cRZLyO8JrH3u5AyJNcbYbgAfj7m4yXI6VPQ4lb6rLsY/9+qF50MfIgA9uqRy9bigpt2E9+pRSu7OabE7Vc37Cc67RzY9wYvrpUn0StU5oGgxZDEA0QoWTXctlEcbpLFUz9C8A5N6QxxY8ErtUHgqc9pubWblWoAlJR4pt8TMGIf90Qz6TbQiNs+FD86tNjmoCdYTf8yhJJgy8/Apiw+spmfwt18T1IKn5twGQ9ph8nLcVjeKJSfB6bn1XY+Q4hp/eKR+fdAl7NlppPvYxVW5ApmehuQuZUIxTsi2w6NHmuRDRwbObCes8r1OPsrawFow4dBzgcUI9nKi/XbTR6Rx0i3jVZRq/EHcxwyZoXxt6EbL8c5nMMm/uU0cQhgtOVlnECU= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 42d3d145-73c4-4cf1-ce75-08da24b1e41b X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 22:46:08.2435 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 81wFhIi42M3MF1NItnJFifEdAi9RqflKJusbDcDAVkm00tRJVbSXmTOSpeLpqK8yhByQTXj2f3bejx1V8cTNdw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2550 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-22_07:2022-04-22,2022-04-22 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204220095 X-Proofpoint-ORIG-GUID: ltflpv_vZac7y6wylVbHcJpzJSNlSvE6 X-Proofpoint-GUID: ltflpv_vZac7y6wylVbHcJpzJSNlSvE6 The set_memory_uc() approach doesn't work well in all cases. As Dan pointed out when "The VMM unmapped the bad page from guest physical space and passed the machine check to the guest." "The guest gets virtual #MC on an access to that page. When the guest tries to do set_memory_uc() and instructs cpa_flush() to do clean caches that results in taking another fault / exception perhaps because the VMM unmapped the page from the guest." Since the driver has special knowledge to handle NP or UC, mark the poisoned page with NP and let driver handle it when it comes down to repair. Please refer to discussions here for more details. https://lore.kernel.org/all/CAPcyv4hrXPb1tASBZUg-GgdVs0OOFKXMXLiHmktg_kFi7YBMyQ@mail.gmail.com/ Now since poisoned page is marked as not-present, in order to avoid writing to a not-present page and trigger kernel Oops, also fix pmem_do_write(). Fixes: 284ce4011ba6 ("x86/memory_failure: Introduce {set, clear}_mce_nospec()") Reviewed-by: Christoph Hellwig Reviewed-by: Dan Williams Signed-off-by: Jane Chu --- arch/x86/kernel/cpu/mce/core.c | 6 +++--- arch/x86/mm/pat/set_memory.c | 23 +++++++++++------------ drivers/nvdimm/pmem.c | 30 +++++++----------------------- include/linux/set_memory.h | 4 ++-- 4 files changed, 23 insertions(+), 40 deletions(-) diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 981496e6bc0e..fa67bb9d1afe 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -579,7 +579,7 @@ static int uc_decode_notifier(struct notifier_block *nb, unsigned long val, pfn = mce->addr >> PAGE_SHIFT; if (!memory_failure(pfn, 0)) { - set_mce_nospec(pfn, whole_page(mce)); + set_mce_nospec(pfn); mce->kflags |= MCE_HANDLED_UC; } @@ -1316,7 +1316,7 @@ static void kill_me_maybe(struct callback_head *cb) ret = memory_failure(p->mce_addr >> PAGE_SHIFT, flags); if (!ret) { - set_mce_nospec(p->mce_addr >> PAGE_SHIFT, p->mce_whole_page); + set_mce_nospec(p->mce_addr >> PAGE_SHIFT); sync_core(); return; } @@ -1342,7 +1342,7 @@ static void kill_me_never(struct callback_head *cb) p->mce_count = 0; pr_err("Kernel accessed poison in user space at %llx\n", p->mce_addr); if (!memory_failure(p->mce_addr >> PAGE_SHIFT, 0)) - set_mce_nospec(p->mce_addr >> PAGE_SHIFT, p->mce_whole_page); + set_mce_nospec(p->mce_addr >> PAGE_SHIFT); } static void queue_task_work(struct mce *m, char *msg, void (*func)(struct callback_head *)) diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 978cf5bd2ab6..e3a5e55f3e08 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -1925,13 +1925,8 @@ int set_memory_wb(unsigned long addr, int numpages) } EXPORT_SYMBOL(set_memory_wb); -/* - * Prevent speculative access to the page by either unmapping - * it (if we do not require access to any part of the page) or - * marking it uncacheable (if we want to try to retrieve data - * from non-poisoned lines in the page). - */ -int set_mce_nospec(unsigned long pfn, bool unmap) +/* Prevent speculative access to a page by marking it not-present */ +int set_mce_nospec(unsigned long pfn) { unsigned long decoy_addr; int rc; @@ -1956,19 +1951,23 @@ int set_mce_nospec(unsigned long pfn, bool unmap) */ decoy_addr = (pfn << PAGE_SHIFT) + (PAGE_OFFSET ^ BIT(63)); - if (unmap) - rc = set_memory_np(decoy_addr, 1); - else - rc = set_memory_uc(decoy_addr, 1); + rc = set_memory_np(decoy_addr, 1); if (rc) pr_warn("Could not invalidate pfn=0x%lx from 1:1 map\n", pfn); return rc; } +static int set_memory_present(unsigned long *addr, int numpages) +{ + return change_page_attr_set(addr, numpages, __pgprot(_PAGE_PRESENT), 0); +} + /* Restore full speculative operation to the pfn. */ int clear_mce_nospec(unsigned long pfn) { - return set_memory_wb((unsigned long) pfn_to_kaddr(pfn), 1); + unsigned long addr = (unsigned long) pfn_to_kaddr(pfn); + + return set_memory_present(&addr, 1); } EXPORT_SYMBOL_GPL(clear_mce_nospec); diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 58d95242a836..4aa17132a557 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -158,36 +158,20 @@ static blk_status_t pmem_do_write(struct pmem_device *pmem, struct page *page, unsigned int page_off, sector_t sector, unsigned int len) { - blk_status_t rc = BLK_STS_OK; - bool bad_pmem = false; phys_addr_t pmem_off = sector * 512 + pmem->data_offset; void *pmem_addr = pmem->virt_addr + pmem_off; - if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) - bad_pmem = true; + if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) { + blk_status_t rc = pmem_clear_poison(pmem, pmem_off, len); + + if (rc != BLK_STS_OK) + return rc; + } - /* - * Note that we write the data both before and after - * clearing poison. The write before clear poison - * handles situations where the latest written data is - * preserved and the clear poison operation simply marks - * the address range as valid without changing the data. - * In this case application software can assume that an - * interrupted write will either return the new good - * data or an error. - * - * However, if pmem_clear_poison() leaves the data in an - * indeterminate state we need to perform the write - * after clear poison. - */ flush_dcache_page(page); write_pmem(pmem_addr, page, page_off, len); - if (unlikely(bad_pmem)) { - rc = pmem_clear_poison(pmem, pmem_off, len); - write_pmem(pmem_addr, page, page_off, len); - } - return rc; + return BLK_STS_OK; } static void pmem_submit_bio(struct bio *bio) diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 683a6c3f7179..369769ce7399 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -43,10 +43,10 @@ static inline bool can_set_direct_map(void) #endif /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */ #ifdef CONFIG_X86_64 -int set_mce_nospec(unsigned long pfn, bool unmap); +int set_mce_nospec(unsigned long pfn); int clear_mce_nospec(unsigned long pfn); #else -static inline int set_mce_nospec(unsigned long pfn, bool unmap) +static inline int set_mce_nospec(unsigned long pfn) { return 0; } From patchwork Fri Apr 22 22:45:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12824267 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C544333F1 for ; Fri, 22 Apr 2022 22:46:28 +0000 (UTC) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23MIYhRW019753; Fri, 22 Apr 2022 22:46:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=KgEw+0HVjksrzULZki0AbI/A03vuojycw3zI4n7x6/4=; b=JHMW8pRQQUYlu2Obqgj2b2Xg3NgifKVWgwoS8FR5G442TkQGdvbZikWdIekJN9DbjWLy SsPnXjsMRLUPIHymIR/UJ5wK8X1y7Zjvoux0dPsj8R7S5TImudo48H4LbCG1mMS/Y7AE V98DOc4dBt7jmrritTDLOcE9jb6uOyGOPlQgntwiGbBWt64OSgKD1JpIe0sJ8SxAJRg6 8OFHC04YKgMlyzH6u4EPFmWd8uVbtDFGIJQ1u71qPiEGbeW5D1rf5uqwj34g0pF0x/u6 l/a7ajqtCSNDOxByWROd9kwxSTRRWcJMjYuKqgxlimFlKcajvMGf0q6tGBmk6MmCcBVR cQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3ffmd1fx92-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:15 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23MMeovn007264; Fri, 22 Apr 2022 22:46:14 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2170.outbound.protection.outlook.com [104.47.57.170]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fk3awf5sd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j6irOy5f/Oi2ikKOTHvqcfARM5kmfnCMN0rV7V+ychThVel27Xjd3FvgiaRIjXF8mHky2Boybp3Zeh2uh2XzZiqFKsUqTXfOIAwXaGjATUjswx25KewlDU5sxGFxd5fSGEyC5c4LXps0YQms/dn02feQZ1bEn2TLMBbHxf6bt8F9MKUwGdvx/0uKUZLb4x569VpyRt84uKy9OYwzNWgZTo6d6SmV/kq7f+OlofMpt39wL9ZkfMsBv1dVvcjRUS4KsdsnzEICNKuGOex/YJbwbRhNSQ8UaqVtuw+oHui/TIePIJxM2SZ/OMMcxpqgIWXG/cE8IE5vzYgfCN6F91R9vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KgEw+0HVjksrzULZki0AbI/A03vuojycw3zI4n7x6/4=; b=AfBnRd8hzgafEOcdBT9qII6mmwadTapm1MMDMIdg4zhtvFMu25gJ5cdW9HF5px3TqXU3RZanDd0jGJnFNfxISI15LBpJlKS8cnWEVMEZSPafcJo9VT/SEEy3GPZawmkiFqJWy3UEPEiadaUftdNsyBafWrjBvAThdqEQ/e648eLWgAumAoRURtHRcJh3oYa1di4D3nBMsstGeZgzRprXIYyeD353w+rRQLtdZOupDjx0NgVwiCxg4ex9qGy6dE/QVFVnn5Sfz6L2Cx+SgpP/Jms6T0M97Ac+Ia2+lcYPpJinwW5W4oViVStGUaXGLdph1awPfncO1M633oj6f0KrHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KgEw+0HVjksrzULZki0AbI/A03vuojycw3zI4n7x6/4=; b=vd5TnFL22Mi0AZCS3mY+z2MdjJKN6mEPZYTUrBS++5pjClOUEHs0n5Az3M468nFOl/GOFFkya6Dqxh8Lv1q7LjI4IFjmCm7hv5qAs56Fk1cdfbjyrx6Z520JKV8rAgLljP7pxe50LKAlR1QHHimGtLr3ioMNU5D3ue29ntJEfGs= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by BYAPR10MB2550.namprd10.prod.outlook.com (2603:10b6:a02:b1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Fri, 22 Apr 2022 22:46:11 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e%8]) with mapi id 15.20.5186.015; Fri, 22 Apr 2022 22:46:11 +0000 From: Jane Chu To: dan.j.williams@intel.com, bp@alien8.de, hch@infradead.org, dave.hansen@intel.com, peterz@infradead.org, luto@kernel.org, david@fromorbit.com, djwong@kernel.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, x86@kernel.org Cc: vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com Subject: [PATCH v9 4/7] dax: introduce DAX_RECOVERY_WRITE dax access mode Date: Fri, 22 Apr 2022 16:45:05 -0600 Message-Id: <20220422224508.440670-5-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220422224508.440670-1-jane.chu@oracle.com> References: <20220422224508.440670-1-jane.chu@oracle.com> X-ClientProxiedBy: SN1PR12CA0057.namprd12.prod.outlook.com (2603:10b6:802:20::28) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f7b6010d-abe5-4e8d-0673-08da24b1e645 X-MS-TrafficTypeDiagnostic: BYAPR10MB2550:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: M94V3gNL2iescFr8lokSDHrzUJOung8HjMMp4QWlm9H9crLv2KuTOBgPFlCq7QMhTt1UKNcJyO8P4Ww/3KZI6ikPdzC4/gjGXLGkh8frYwUOpBz+io17FIh2lcsUFvGcEInsrbyt3S0zdnBQWcQZDjUzUMtDZZtSG5h4RZ/wxztzPNLWZK9ewD+ZD9tI31/uc2WSJBnyQw7QltyACCc6nZkEXcVeL7S0HnHMiBaprWQgIWL84aiZgdkiM7RJIRvvqdAu9nYjvkAas4wCF38/XxI0IB0wtYlro4UF6P1LwMDEK8V9r4C2l6UetOdpRJ171QdU6GePXG3QRP9beWt3+nfmRkVg4rmc+d1xzWxmFnyefGKNEiX9i50PY7zfyGpOUlTRAWMt7peeR7tLIfXwjXnRK+P2xwEXEopscM3Cli9k7Qw4TCwYCPk/Xi/GHiQULwI5KCNR4klAEkGYJUNy1PNpup1uE58IfCpoku1xOpWwSlYZSGn04AAmG4rUYfw0YlRQX7KR7MyQI1VeDonDSjeZl1b4BdTxBJOv79Pvua5QnRCDOWn5yKQRRMw1IOeIVZ08zjp4rvzFvgFujsv/TCULhNLgLQjsDzHw9CHmdSg3rQ9j4bFIoY6JG6GXirQqxLwftN8aCiGIV4JGaI9InjXNaVtkgjExh0LbSVQ7xQ9TzZ1Ng19ecJLr7z/saFVt X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6506007)(6666004)(6486002)(8936002)(508600001)(52116002)(83380400001)(36756003)(316002)(66556008)(66476007)(2906002)(8676002)(4326008)(66946007)(38100700002)(2616005)(86362001)(5660300002)(30864003)(44832011)(186003)(6512007)(1076003)(7416002)(921005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: u7X3gEZNfk9YdGvgoL74bnv7XosT6gQgzmc5jGBbjwUJvO37rI+JBGLAupwTYl9K6pQN6JEc+t49Z/lhfb0r3MrVaOnvb/l2dOdiGSqkO6GMm+iOaUkZMeMRJP8D9+SWRL1BL60uvdH56wD2d3hGBgF0UOfNzO0IW6gS0Fdhiq3QvlAbZWegtZ4foYmaey6udJ3iRF8d+m2TkDNyYM+6Ue6l/lysBOtdLIp0+4eNB9q4M4fJrrfX8ZQFLvCVqMCq5uexTL2GQx0Rls1EO/WWmz9UZtNOhw5sTGTOE1xtsxplnYGBeliCx+xsf9yL+mlAi0xya0Pj46wSBcnpdmQcOx63eATAf0Ap1wL5A3eQCseP8/Zj6hyHLYARKHv2jePBALcYJp8TI7LtND/c5FIuX1nY0/l3uTYzPYCW2XhXHabvV9Un6uBB7l9PS+jxk0x/DRo5SfAeI7aWR8UPuDKWZNNevrLm6uVerWm5445U+yWt9b2L7Nn9agBuWU41F+O6Yz2UHjLbG5b/UMOGG13mpJDjWtE/2+RnAq3Oh7YruDM4j2jpGZUmqlwgYaiRx4JehbcEWdjs4n/t+r8jMZuBaIAFSGTlEbqfEY/Kzyp1fLS+lncAvqPxFUaAmY67/rXwKjcEtgJPOTdiD8+wYm5FDYIeldKt4siCyHzKNCg/yDTYQylzjFi97IaRq3/5YTD5fWNKcCGBYYToIJZ9B36aI+S1epxPydELxR13+gDvXAcAEYHErrVDUeRSh5/9Z/zc6XcY7/2C1gSzwTw78thRYA7mIXQkNCe3nt9xeU2+aef8Z9ag01J0ei3+MkxnuUJ0XpSbuLuUl40AT/fKOHe0wS4hKwSGgmT0ApTueocwE1YGugEfXXwCa8PBupL7iKvh0mEg7oOEDIji6oQqh6/CtocWPGi+pm2FMgOD515pRQ7BEK6fsAsbcDQPsuzwJ52qQbgAMrQ3ALYM/FasW4S71pOUWDR76Go8nX1rjYjZQEcGLuKbm1/Zy+OwoDj+6RcUREFsiLJz7cG091LWKu+AVA+I2AMlL+D7hsdi7X0QOoCbDk0k2j7DMlgEg+giYIhOAxWXomSH1ePYaVj+akUw6iq7CXKx3Akc6YXTanurBSRoFDyjWdQ+CX3gJcmshI26/vPGJue/jOMQFv975io2Vrx1GkQv21WjWoUt6ofDO+May7sPk2ah0g0ME0/78IjF4xyujL5ntdkBNShvMMqzvYwr4H2V2L/7kuWrAqmNiRQ2fFEsxU8hfw3homx0ROfLenNpNM6Ecn9JgZMQzYfsPnpUXz2te0lKJYTFryxBVxLJ74480qgkkVX/+FeFXS1CHOaSWFxbxz+3+whV3+qtYit9ADnUPtsl6zlbr5uzL6FKnz73/VQOs8FqSN+KNVu5V6zoh4HpVFeeGLc9fisCgpJo400ibbJ2oNYl1UnJPR7lF574Z8y4+dlfgiCHtWzIgEWONqhaJrC5Eb70ryUnbYlbwKDsjpJPjun0ttn+WDFzSmvyL6UiO3kta0K1iVfQHz2Fnzpdl0iWgcoFh2cDNzUYeHUk0Fy5MzBNSMMNddputYxTANW0p6pcUSM53j6DE8UrU1g8argFsPXfPbqTVY9CkP8/dSOv5MwFYbLnk6Pjt3Fp9Xrbfz2gUwr/SZSHA0uHDRr18XoHfcdGmi+dWB3QfsWmHmnz3u6u0M1bDU2Ua6CEOccdYbvWikGR1ne0p/SKXdBYar3QVhJx8WU6lY4TD84geyjsDQFxQbFHoQ8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f7b6010d-abe5-4e8d-0673-08da24b1e645 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 22:46:11.8534 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lRxzdE8Qjetb9e3SiCpKotSQkfbn7CJN0a6w7eu3GjKCHssUsPpSDictK5YJo81eCkVAtjtSlVUjD4YbjNGqWg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2550 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-22_07:2022-04-22,2022-04-22 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204220095 X-Proofpoint-ORIG-GUID: dfRJQj5Op81y4lX6p8x7wGY5IhQEyAlt X-Proofpoint-GUID: dfRJQj5Op81y4lX6p8x7wGY5IhQEyAlt Up till now, dax_direct_access() is used implicitly for normal access, but for the purpose of recovery write, dax range with poison is requested. To make the interface clear, introduce enum dax_access_mode { DAX_ACCESS, DAX_RECOVERY_WRITE, } where DAX_ACCESS is used for normal dax access, and DAX_RECOVERY_WRITE is used for dax recovery write. Suggested-by: Dan Williams Signed-off-by: Jane Chu Reviewed-by: Christoph Hellwig --- drivers/dax/super.c | 5 +++-- drivers/md/dm-linear.c | 5 +++-- drivers/md/dm-log-writes.c | 5 +++-- drivers/md/dm-stripe.c | 5 +++-- drivers/md/dm-target.c | 4 +++- drivers/md/dm-writecache.c | 7 ++++--- drivers/md/dm.c | 5 +++-- drivers/nvdimm/pmem.c | 8 +++++--- drivers/nvdimm/pmem.h | 5 ++++- drivers/s390/block/dcssblk.c | 9 ++++++--- fs/dax.c | 9 +++++---- fs/fuse/dax.c | 4 ++-- include/linux/dax.h | 9 +++++++-- include/linux/device-mapper.h | 4 +++- tools/testing/nvdimm/pmem-dax.c | 3 ++- 15 files changed, 56 insertions(+), 31 deletions(-) diff --git a/drivers/dax/super.c b/drivers/dax/super.c index 0211e6f7b47a..5405eb553430 100644 --- a/drivers/dax/super.c +++ b/drivers/dax/super.c @@ -117,6 +117,7 @@ enum dax_device_flags { * @dax_dev: a dax_device instance representing the logical memory range * @pgoff: offset in pages from the start of the device to translate * @nr_pages: number of consecutive pages caller can handle relative to @pfn + * @mode: indicator on normal access or recovery write * @kaddr: output parameter that returns a virtual address mapping of pfn * @pfn: output parameter that returns an absolute pfn translation of @pgoff * @@ -124,7 +125,7 @@ enum dax_device_flags { * pages accessible at the device relative @pgoff. */ long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, - void **kaddr, pfn_t *pfn) + enum dax_access_mode mode, void **kaddr, pfn_t *pfn) { long avail; @@ -138,7 +139,7 @@ long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, return -EINVAL; avail = dax_dev->ops->direct_access(dax_dev, pgoff, nr_pages, - kaddr, pfn); + mode, kaddr, pfn); if (!avail) return -ERANGE; return min(avail, nr_pages); diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index 76b486e4d2be..13e263299c9c 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -172,11 +172,12 @@ static struct dax_device *linear_dax_pgoff(struct dm_target *ti, pgoff_t *pgoff) } static long linear_dax_direct_access(struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn) { struct dax_device *dax_dev = linear_dax_pgoff(ti, &pgoff); - return dax_direct_access(dax_dev, pgoff, nr_pages, kaddr, pfn); + return dax_direct_access(dax_dev, pgoff, nr_pages, mode, kaddr, pfn); } static int linear_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c index c9d036d6bb2e..06bdbed65eb1 100644 --- a/drivers/md/dm-log-writes.c +++ b/drivers/md/dm-log-writes.c @@ -889,11 +889,12 @@ static struct dax_device *log_writes_dax_pgoff(struct dm_target *ti, } static long log_writes_dax_direct_access(struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn) { struct dax_device *dax_dev = log_writes_dax_pgoff(ti, &pgoff); - return dax_direct_access(dax_dev, pgoff, nr_pages, kaddr, pfn); + return dax_direct_access(dax_dev, pgoff, nr_pages, mode, kaddr, pfn); } static int log_writes_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, diff --git a/drivers/md/dm-stripe.c b/drivers/md/dm-stripe.c index c81d331d1afe..77d72900e997 100644 --- a/drivers/md/dm-stripe.c +++ b/drivers/md/dm-stripe.c @@ -315,11 +315,12 @@ static struct dax_device *stripe_dax_pgoff(struct dm_target *ti, pgoff_t *pgoff) } static long stripe_dax_direct_access(struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn) { struct dax_device *dax_dev = stripe_dax_pgoff(ti, &pgoff); - return dax_direct_access(dax_dev, pgoff, nr_pages, kaddr, pfn); + return dax_direct_access(dax_dev, pgoff, nr_pages, mode, kaddr, pfn); } static int stripe_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, diff --git a/drivers/md/dm-target.c b/drivers/md/dm-target.c index 64dd0b34fcf4..8cd5184e62f0 100644 --- a/drivers/md/dm-target.c +++ b/drivers/md/dm-target.c @@ -10,6 +10,7 @@ #include #include #include +#include #define DM_MSG_PREFIX "target" @@ -142,7 +143,8 @@ static void io_err_release_clone_rq(struct request *clone, } static long io_err_dax_direct_access(struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn) { return -EIO; } diff --git a/drivers/md/dm-writecache.c b/drivers/md/dm-writecache.c index 5630b470ba42..d74c5a7a0ab4 100644 --- a/drivers/md/dm-writecache.c +++ b/drivers/md/dm-writecache.c @@ -286,7 +286,8 @@ static int persistent_memory_claim(struct dm_writecache *wc) id = dax_read_lock(); - da = dax_direct_access(wc->ssd_dev->dax_dev, offset, p, &wc->memory_map, &pfn); + da = dax_direct_access(wc->ssd_dev->dax_dev, offset, p, DAX_ACCESS, + &wc->memory_map, &pfn); if (da < 0) { wc->memory_map = NULL; r = da; @@ -308,8 +309,8 @@ static int persistent_memory_claim(struct dm_writecache *wc) i = 0; do { long daa; - daa = dax_direct_access(wc->ssd_dev->dax_dev, offset + i, p - i, - NULL, &pfn); + daa = dax_direct_access(wc->ssd_dev->dax_dev, offset + i, + p - i, DAX_ACCESS, NULL, &pfn); if (daa <= 0) { r = daa ? daa : -EINVAL; goto err3; diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 3c5fad7c4ee6..8258676a352f 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1093,7 +1093,8 @@ static struct dm_target *dm_dax_get_live_target(struct mapped_device *md, } static long dm_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn) { struct mapped_device *md = dax_get_private(dax_dev); sector_t sector = pgoff * PAGE_SECTORS; @@ -1111,7 +1112,7 @@ static long dm_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, if (len < 1) goto out; nr_pages = min(len, nr_pages); - ret = ti->type->direct_access(ti, pgoff, nr_pages, kaddr, pfn); + ret = ti->type->direct_access(ti, pgoff, nr_pages, mode, kaddr, pfn); out: dm_put_live_table(md, srcu_idx); diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 4aa17132a557..47f34c50f944 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -239,7 +239,8 @@ static int pmem_rw_page(struct block_device *bdev, sector_t sector, /* see "strong" declaration in tools/testing/nvdimm/pmem-dax.c */ __weak long __pmem_direct_access(struct pmem_device *pmem, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn) { resource_size_t offset = PFN_PHYS(pgoff) + pmem->data_offset; @@ -278,11 +279,12 @@ static int pmem_dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff, } static long pmem_dax_direct_access(struct dax_device *dax_dev, - pgoff_t pgoff, long nr_pages, void **kaddr, pfn_t *pfn) + pgoff_t pgoff, long nr_pages, enum dax_access_mode mode, + void **kaddr, pfn_t *pfn) { struct pmem_device *pmem = dax_get_private(dax_dev); - return __pmem_direct_access(pmem, pgoff, nr_pages, kaddr, pfn); + return __pmem_direct_access(pmem, pgoff, nr_pages, mode, kaddr, pfn); } static const struct dax_operations pmem_dax_ops = { diff --git a/drivers/nvdimm/pmem.h b/drivers/nvdimm/pmem.h index 1f51a2361429..392b0b38acb9 100644 --- a/drivers/nvdimm/pmem.h +++ b/drivers/nvdimm/pmem.h @@ -8,6 +8,8 @@ #include #include +enum dax_access_mode; + /* this definition is in it's own header for tools/testing/nvdimm to consume */ struct pmem_device { /* One contiguous memory region per device */ @@ -28,7 +30,8 @@ struct pmem_device { }; long __pmem_direct_access(struct pmem_device *pmem, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn); + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn); #ifdef CONFIG_MEMORY_FAILURE static inline bool test_and_clear_pmem_poison(struct page *page) diff --git a/drivers/s390/block/dcssblk.c b/drivers/s390/block/dcssblk.c index d614843caf6c..8d0d0eaa3059 100644 --- a/drivers/s390/block/dcssblk.c +++ b/drivers/s390/block/dcssblk.c @@ -32,7 +32,8 @@ static int dcssblk_open(struct block_device *bdev, fmode_t mode); static void dcssblk_release(struct gendisk *disk, fmode_t mode); static void dcssblk_submit_bio(struct bio *bio); static long dcssblk_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn); + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn); static char dcssblk_segments[DCSSBLK_PARM_LEN] = "\0"; @@ -50,7 +51,8 @@ static int dcssblk_dax_zero_page_range(struct dax_device *dax_dev, long rc; void *kaddr; - rc = dax_direct_access(dax_dev, pgoff, nr_pages, &kaddr, NULL); + rc = dax_direct_access(dax_dev, pgoff, nr_pages, DAX_ACCESS, + &kaddr, NULL); if (rc < 0) return rc; memset(kaddr, 0, nr_pages << PAGE_SHIFT); @@ -927,7 +929,8 @@ __dcssblk_direct_access(struct dcssblk_dev_info *dev_info, pgoff_t pgoff, static long dcssblk_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn) { struct dcssblk_dev_info *dev_info = dax_get_private(dax_dev); diff --git a/fs/dax.c b/fs/dax.c index 67a08a32fccb..ef3103107104 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -721,7 +721,8 @@ static int copy_cow_page_dax(struct vm_fault *vmf, const struct iomap_iter *iter int id; id = dax_read_lock(); - rc = dax_direct_access(iter->iomap.dax_dev, pgoff, 1, &kaddr, NULL); + rc = dax_direct_access(iter->iomap.dax_dev, pgoff, 1, DAX_ACCESS, + &kaddr, NULL); if (rc < 0) { dax_read_unlock(id); return rc; @@ -1013,7 +1014,7 @@ static int dax_iomap_pfn(const struct iomap *iomap, loff_t pos, size_t size, id = dax_read_lock(); length = dax_direct_access(iomap->dax_dev, pgoff, PHYS_PFN(size), - NULL, pfnp); + DAX_ACCESS, NULL, pfnp); if (length < 0) { rc = length; goto out; @@ -1122,7 +1123,7 @@ static int dax_memzero(struct dax_device *dax_dev, pgoff_t pgoff, void *kaddr; long ret; - ret = dax_direct_access(dax_dev, pgoff, 1, &kaddr, NULL); + ret = dax_direct_access(dax_dev, pgoff, 1, DAX_ACCESS, &kaddr, NULL); if (ret > 0) { memset(kaddr + offset, 0, size); dax_flush(dax_dev, kaddr + offset, size); @@ -1247,7 +1248,7 @@ static loff_t dax_iomap_iter(const struct iomap_iter *iomi, } map_len = dax_direct_access(dax_dev, pgoff, PHYS_PFN(size), - &kaddr, NULL); + DAX_ACCESS, &kaddr, NULL); if (map_len < 0) { ret = map_len; break; diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c index d7d3a7f06862..10eb50cbf398 100644 --- a/fs/fuse/dax.c +++ b/fs/fuse/dax.c @@ -1241,8 +1241,8 @@ static int fuse_dax_mem_range_init(struct fuse_conn_dax *fcd) INIT_DELAYED_WORK(&fcd->free_work, fuse_dax_free_mem_worker); id = dax_read_lock(); - nr_pages = dax_direct_access(fcd->dev, 0, PHYS_PFN(dax_size), NULL, - NULL); + nr_pages = dax_direct_access(fcd->dev, 0, PHYS_PFN(dax_size), + DAX_ACCESS, NULL, NULL); dax_read_unlock(id); if (nr_pages < 0) { pr_debug("dax_direct_access() returned %ld\n", nr_pages); diff --git a/include/linux/dax.h b/include/linux/dax.h index 9fc5f99a0ae2..3f1339bce3c0 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -14,6 +14,11 @@ struct iomap_ops; struct iomap_iter; struct iomap; +enum dax_access_mode { + DAX_ACCESS, + DAX_RECOVERY_WRITE, +}; + struct dax_operations { /* * direct_access: translate a device-relative @@ -21,7 +26,7 @@ struct dax_operations { * number of pages available for DAX at that pfn. */ long (*direct_access)(struct dax_device *, pgoff_t, long, - void **, pfn_t *); + enum dax_access_mode, void **, pfn_t *); /* * Validate whether this device is usable as an fsdax backing * device. @@ -178,7 +183,7 @@ static inline void dax_read_unlock(int id) bool dax_alive(struct dax_device *dax_dev); void *dax_get_private(struct dax_device *dax_dev); long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, - void **kaddr, pfn_t *pfn); + enum dax_access_mode mode, void **kaddr, pfn_t *pfn); size_t dax_copy_from_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, size_t bytes, struct iov_iter *i); size_t dax_copy_to_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index c2a3758c4aaa..acdedda0d12b 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -20,6 +20,7 @@ struct dm_table; struct dm_report_zones_args; struct mapped_device; struct bio_vec; +enum dax_access_mode; /* * Type of table, mapped_device's mempool and request_queue @@ -146,7 +147,8 @@ typedef int (*dm_busy_fn) (struct dm_target *ti); * >= 0 : the number of bytes accessible at the address */ typedef long (*dm_dax_direct_access_fn) (struct dm_target *ti, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn); + long nr_pages, enum dax_access_mode node, void **kaddr, + pfn_t *pfn); typedef int (*dm_dax_zero_page_range_fn)(struct dm_target *ti, pgoff_t pgoff, size_t nr_pages); diff --git a/tools/testing/nvdimm/pmem-dax.c b/tools/testing/nvdimm/pmem-dax.c index af19c85558e7..dcc328eba811 100644 --- a/tools/testing/nvdimm/pmem-dax.c +++ b/tools/testing/nvdimm/pmem-dax.c @@ -8,7 +8,8 @@ #include long __pmem_direct_access(struct pmem_device *pmem, pgoff_t pgoff, - long nr_pages, void **kaddr, pfn_t *pfn) + long nr_pages, enum dax_access_mode mode, void **kaddr, + pfn_t *pfn) { resource_size_t offset = PFN_PHYS(pgoff) + pmem->data_offset; From patchwork Fri Apr 22 22:45:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12824268 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC9093D72 for ; Fri, 22 Apr 2022 22:46:31 +0000 (UTC) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23MK5ruh024809; Fri, 22 Apr 2022 22:46:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=bGy+hL3VpQwyFjDqp7k0HE1+Sa45J4xFT+aUmE/87F8=; b=dlqQJumwPyfXq5EtWxw9EaDEqHUalVcnOhHO41mWFgh2qE+jZkaFPIW5UOFXOngCK2bJ KatTEaPkaqann5MGfbDBYpF77fcrZt5X+XQANlcyJzPzEaowRNkxw7QcTlBtVmd/8Gde 3oxZknt8LNRYYHcysz+Dc9IdiBBF2p1/llph3nkrOQ0Diw6bD7QZwchlBvKCbLffG9Gc OP+V2RvIGocdjKZrcqF+YFR9kIB3kD4w2WMtNi0n91RiK0t/oBGXVcnQVChNeN7uNw2J SJOBRmfQronB0qrTLBdnd0jICO6ZwFdTRVgiPFt8Q1vXylsvLyXLW/xZemAYQfV/TbG4 sg== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3ffnp9qdj1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:18 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23MMg0iZ008597; Fri, 22 Apr 2022 22:46:18 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2177.outbound.protection.outlook.com [104.47.57.177]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3ffm8e9my2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V0vbfwYWhkOT94or1nF224PonITrkzBydjkMOA8/hjTrMMsZ4TMZ9pm8wrQUJ8N8QEzH5A6m2gyqTNdMNokVKHNNjWi+uJ/TDQLfkuH+HFQwC69vicLoFRPV9FR1e+NReh59kl6kVr5p0uadTQPXguTtbg7V7r3xFLx+krksl9wjqKQyd9jRwXnQgokqYiFlQfbFRgJKhl7ug0iV4hWeIFsfV10HLDZpQoxJsj+qGnYFtnvpScdRbeLEcFbK6GLjfZD1Q4Ro08n5aBm6tckMiNmLGFJDwvm4cLXKwlD5r+6m4sOOQp2xgVnCKg0cwa0FD4tB304XyupFQmgnaIRSGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bGy+hL3VpQwyFjDqp7k0HE1+Sa45J4xFT+aUmE/87F8=; b=c8q7nhed3VaX/lANjgz3O7ctuZYsqu5F3vDcU62MRz6HoGXbf5lkL171JA2UPNQyMDDv7Y044vouTCPRvgL9aSPfE6CAX/VYoYtnFYBAmIrQPg8EOelY+s3mOigF0Fy2K9lw64ZyXCIuiklak1gAkKbXqsGBggLGf+kEU2FeBHWoBq4wSvCRwZNzzhMBWI3jOqeCrlZtlVtNui3wnRGGWlRdQsHYfjFn/+geC3dY+JghNWtNsABVJmQzHFMHK6mGC8sOXH3aE35wj6jwlQIYmljl5Pjvo8sOKsQLVZQGvcj7SnOjbNPSJBkS2KBEv44GXIxZsv+sV6MDy5/hbRU3Wg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bGy+hL3VpQwyFjDqp7k0HE1+Sa45J4xFT+aUmE/87F8=; b=XBkSjxrwLJSFtQTwgWY9DJBmHRu5Z0UA4LGfrExafaY3ysT+O3GCSuC4OsW5t8eHdT6GrkAno4i+cnXBzyTAPCv3NQBLPqJKyzRgJxeU/OEjP+93GL2cZI2voAs6fWK+/2LLfKZTAmKNXGsbOksBqANyx9oD7owDCwbNvg3jZmM= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by BYAPR10MB2550.namprd10.prod.outlook.com (2603:10b6:a02:b1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Fri, 22 Apr 2022 22:46:15 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e%8]) with mapi id 15.20.5186.015; Fri, 22 Apr 2022 22:46:15 +0000 From: Jane Chu To: dan.j.williams@intel.com, bp@alien8.de, hch@infradead.org, dave.hansen@intel.com, peterz@infradead.org, luto@kernel.org, david@fromorbit.com, djwong@kernel.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, x86@kernel.org Cc: vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com Subject: [PATCH v9 5/7] dax: add .recovery_write dax_operation Date: Fri, 22 Apr 2022 16:45:06 -0600 Message-Id: <20220422224508.440670-6-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220422224508.440670-1-jane.chu@oracle.com> References: <20220422224508.440670-1-jane.chu@oracle.com> X-ClientProxiedBy: SN1PR12CA0057.namprd12.prod.outlook.com (2603:10b6:802:20::28) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 65cef020-ed96-499c-5d2c-08da24b1e8a9 X-MS-TrafficTypeDiagnostic: BYAPR10MB2550:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Z8ytzmDG+MrIOpbJPVi6YDrNqSWsmcPkZwxZvz/WZ636pehQ5lezrKvuEU0W7ZUpeHelwZMclXN971XahMJ7IPwd43lPnzD3hjpo+4NVWKvdZ5lT5S2aLXPE189uJoNVgHTCCiJf/9yCPaWjawZtViEugaMBB2F7FT1x7QTeNDljsoEGCpkzuKlUN8nLlDqU7gpoqe5R6FxHy7Q9Ouxo8uYQWQWBbDaPNfsRFKToaGmC+sEEdFUVM8JYx3xdKbi243t42Jja4jP6G6ix4I2lbibtpmG/Pk6RPmFWiNSybT+V/0HuSscQC+aDRI0nrs2rFJAjxS5+SxOCKC2EqNlCrTjfsFXEWOe/ks9rfLZjwQw9ZJgSVD98J/cDshRKuV9xNClznSBHx+kBPJf8LLeP3YzRsWKhJpXo+7Ns9XsNvRbm04D3fGLxPEHyRmryAPchGacAnDHabOHJzXXgNyzeeBFqIPs5YuDFbKJ/CK+euRpGG2bFeKop8+OX3+LGNhKhEFFzlmWMzra1vYSKBb2GQwwHpUDxuD4i4hZGWWcAYvzw4FlUe8MVTkOKoOt7Zt+D5URJeai7zKebZmicQqpoidRsRC3Enk32jcgeZP3OC+CHK9Xa1iqFWIhySfS6BzpYycE9q/LsQncSxPysmKTvgEXLXvRCL1NuagSer/emzmYXJpi21+fFAw+E+Tz4nuRfVonpL5FPd+gwwD2bru7cme6iYYhTcnwBew24YpptqnM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6506007)(6666004)(6486002)(8936002)(508600001)(52116002)(83380400001)(36756003)(316002)(66556008)(66476007)(2906002)(8676002)(4326008)(66946007)(38100700002)(2616005)(86362001)(5660300002)(30864003)(44832011)(186003)(6512007)(1076003)(7416002)(921005)(142923001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3yYzgiBmJAPfBXidt6x+QNWyKzo2Q787ipcak7CR2fD240D7MShAPCTY7sgPBS/3W0xZ8RSrJ+NxJvIfmGhCYH8m/HyiNLgU+rvDcDU10uz+SqrqKvaxRqhH0+Yz0z11wRd+bHEA4CiHGjLMx0mdZtbUoQeyb7mZqLZyXn5UKQH+3zgfnARy6eMXyua/RoIWR6xqF0mgmI0ehTwKGN8zvaszUK/n4QUSvTxPWgP/88ixJmmyHhos1tpK3dURl0Gi/KoC/F+cDVlBpJpX96FOTFW+8E64ztrDDtnZVHp5Z1KEeaQL4imSsP4YuTkTIBt5emC25uPy7bgZDW3e+7VKnGoUlKOF6IsaD2kRIkPCqCX7IWaEkxryjpuNfkRImUJpJY9j/RNWyBtpJdKTHTyHTQJXc9y7WJnpGPmw6R126yrlpD60kU3iAw1kaHqW0/DmGFv+/taN1I18xtTbNGglAfeGROJ6bZOI06x9qivKr9MXCooyjmTPjQ229HU3A3PaQI3CVG2+CaCwnmb/dVbejxWPvKGmjoOGY3435uf4BUBTd5oJfAmLljN0jSfIBL+o9BSHinQVEvKmbjTB4rAHLzEU/Vzno7qKwQooLXkrTqtf+N9tqsuXrxsGZFqfw4taQbU3AgBxuDRdAwrqFCdRcSNOidm7aqhkBklpyHKkaREhXqtCKJsX6xcUzZggiTfAKt+qvhYr5/JnQ9tyjqInxUUQTIoxEmILAONCxZbnjvUpeWDHWYMrh7qiDCByilLR7VYkBm/y4mhVBvKh0AsSaxQ40HdorY3/O+/IKYCAcftaP1qun2nSWp5fX7q9qkVCzCAV/m3d1+d6ViSYkwilkEEYgtHtXoHBsf323Rbemkt/6LLjjne5WAI+1bPS1GwJgP/HV0iKdWPobYWbimm3Xl4/oRtgYxH4HGNGq+CYPulkLIQSxVKovPobpCLDqDCbywc0ht6vMw7gcF1CLQoREXmD3xjxLDRBcaQGMjmtjlWlxzkrnJgbyc/ESlsc57a71DQGiHX8eceDuaGOmGACg+3NyXwsmUYpW2QoyBJdRO3vSfsNWpg9BL1N09iyWYFGoIpj+eqnaq38a2gcHHBMmrIaqAjd+eToGZhmBpBh7x5pslwgglt1XJkCVxgoyW6l/YI31/aNR4OyWAXTfW2S5ufn+dGRtMnmGIwS+aBAx7qteFhMY0NVzsi0JNKWcUuPXGkmZkbMuZikFbkaWqa6kUHTZ2BUvBBXf3f/teNf1+OzHqlHdgAiZQOf4fzhzOt7qL63CYio2Xinfm51rH4NSGZWcv3kx3mXYAYHQoSWQPwJfCp3RHWWxcblK699eUXz5J9awp2XwA1KOoU2K2qgOKHvkjEbDWRpEM2HTfTBlJ6NQY2koKMzb2pJTcy7jU+4UbivYpOa9zEjg2XAnH//m+AfzCErOYNSnXYAxOCKtylirq85vxQ//CItUb2rqJo1IXYzRS/qLb4P2irYRpVGgRNtcUTGOuyi513l7bmeSrNdTIoyCup57kwX7pCdciPekQizkFX+Y2HxKdQDtVVlax+ryGgB7SMZAMle1s2FFKSiYr5Ru+NwuBoMhKIuETxi0du5MEDR9rcWO6Yi1bNUyZXg4U8zanmZISkXckeY8mLtEAf5ByCV439f2s8iP3cPKzwEpu46GENTAsVxQ6ViFjM2x40Y+dfR+yhKdbWjBBeRNsxHgNAFDHuTmxSAkJzuMTByXxWy/HNX8jgC+pmvT/1ZJOf4IGrr2g2H4a6yFH8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65cef020-ed96-499c-5d2c-08da24b1e8a9 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 22:46:15.8664 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: M9FcuQPvZMn3ztDf4vhTnuDXket5UbQipLiNnAlpqtL83n/iL5HXK4I/Q6XJ/Y7zCg493DLk+t/o+XjGVKrb0w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2550 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-22_07:2022-04-22,2022-04-22 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 malwarescore=0 suspectscore=0 spamscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204220095 X-Proofpoint-ORIG-GUID: AzwBVj0_VnkthZxxEALWhBkMZtmPFdzb X-Proofpoint-GUID: AzwBVj0_VnkthZxxEALWhBkMZtmPFdzb Introduce dax_recovery_write() operation. The function is used to recover a dax range that contains poison. Typical use case is when a user process receives a SIGBUS with si_code BUS_MCEERR_AR indicating poison(s) in a dax range, in response, the user process issues a pwrite() to the page-aligned dax range, thus clears the poison and puts valid data in the range. Reviewed-by: Christoph Hellwig Signed-off-by: Jane Chu --- drivers/dax/super.c | 9 +++++++++ drivers/md/dm-linear.c | 10 ++++++++++ drivers/md/dm-log-writes.c | 10 ++++++++++ drivers/md/dm-stripe.c | 10 ++++++++++ drivers/md/dm.c | 20 ++++++++++++++++++++ drivers/nvdimm/pmem.c | 7 +++++++ fs/dax.c | 13 ++++++++++++- include/linux/dax.h | 13 +++++++++++++ include/linux/device-mapper.h | 9 +++++++++ 9 files changed, 100 insertions(+), 1 deletion(-) diff --git a/drivers/dax/super.c b/drivers/dax/super.c index 5405eb553430..50a08b2ec247 100644 --- a/drivers/dax/super.c +++ b/drivers/dax/super.c @@ -195,6 +195,15 @@ int dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff, } EXPORT_SYMBOL_GPL(dax_zero_page_range); +size_t dax_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *iter) +{ + if (!dax_dev->ops->recovery_write) + return 0; + return dax_dev->ops->recovery_write(dax_dev, pgoff, addr, bytes, iter); +} +EXPORT_SYMBOL_GPL(dax_recovery_write); + #ifdef CONFIG_ARCH_HAS_PMEM_API void arch_wb_cache_pmem(void *addr, size_t size); void dax_flush(struct dax_device *dax_dev, void *addr, size_t size) diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index 13e263299c9c..cdf48bc8c5b0 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -188,9 +188,18 @@ static int linear_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, return dax_zero_page_range(dax_dev, pgoff, nr_pages); } +static size_t linear_dax_recovery_write(struct dm_target *ti, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *i) +{ + struct dax_device *dax_dev = linear_dax_pgoff(ti, &pgoff); + + return dax_recovery_write(dax_dev, pgoff, addr, bytes, i); +} + #else #define linear_dax_direct_access NULL #define linear_dax_zero_page_range NULL +#define linear_dax_recovery_write NULL #endif static struct target_type linear_target = { @@ -208,6 +217,7 @@ static struct target_type linear_target = { .iterate_devices = linear_iterate_devices, .direct_access = linear_dax_direct_access, .dax_zero_page_range = linear_dax_zero_page_range, + .dax_recovery_write = linear_dax_recovery_write, }; int __init dm_linear_init(void) diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c index 06bdbed65eb1..22739dccdd17 100644 --- a/drivers/md/dm-log-writes.c +++ b/drivers/md/dm-log-writes.c @@ -905,9 +905,18 @@ static int log_writes_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, return dax_zero_page_range(dax_dev, pgoff, nr_pages << PAGE_SHIFT); } +static size_t log_writes_dax_recovery_write(struct dm_target *ti, + pgoff_t pgoff, void *addr, size_t bytes, struct iov_iter *i) +{ + struct dax_device *dax_dev = log_writes_dax_pgoff(ti, &pgoff); + + return dax_recovery_write(dax_dev, pgoff, addr, bytes, i); +} + #else #define log_writes_dax_direct_access NULL #define log_writes_dax_zero_page_range NULL +#define log_writes_dax_recovery_write NULL #endif static struct target_type log_writes_target = { @@ -925,6 +934,7 @@ static struct target_type log_writes_target = { .io_hints = log_writes_io_hints, .direct_access = log_writes_dax_direct_access, .dax_zero_page_range = log_writes_dax_zero_page_range, + .dax_recovery_write = log_writes_dax_recovery_write, }; static int __init dm_log_writes_init(void) diff --git a/drivers/md/dm-stripe.c b/drivers/md/dm-stripe.c index 77d72900e997..baa085cc67bd 100644 --- a/drivers/md/dm-stripe.c +++ b/drivers/md/dm-stripe.c @@ -331,9 +331,18 @@ static int stripe_dax_zero_page_range(struct dm_target *ti, pgoff_t pgoff, return dax_zero_page_range(dax_dev, pgoff, nr_pages); } +static size_t stripe_dax_recovery_write(struct dm_target *ti, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *i) +{ + struct dax_device *dax_dev = stripe_dax_pgoff(ti, &pgoff); + + return dax_recovery_write(dax_dev, pgoff, addr, bytes, i); +} + #else #define stripe_dax_direct_access NULL #define stripe_dax_zero_page_range NULL +#define stripe_dax_recovery_write NULL #endif /* @@ -470,6 +479,7 @@ static struct target_type stripe_target = { .io_hints = stripe_io_hints, .direct_access = stripe_dax_direct_access, .dax_zero_page_range = stripe_dax_zero_page_range, + .dax_recovery_write = stripe_dax_recovery_write, }; int __init dm_stripe_init(void) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 8258676a352f..5374c8aba2d6 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1147,6 +1147,25 @@ static int dm_dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff, return ret; } +static size_t dm_dax_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *i) +{ + struct mapped_device *md = dax_get_private(dax_dev); + sector_t sector = pgoff * PAGE_SECTORS; + struct dm_target *ti; + int srcu_idx; + long ret = 0; + + ti = dm_dax_get_live_target(md, sector, &srcu_idx); + if (!ti || !ti->type->dax_recovery_write) + goto out; + + ret = ti->type->dax_recovery_write(ti, pgoff, addr, bytes, i); +out: + dm_put_live_table(md, srcu_idx); + return ret; +} + /* * A target may call dm_accept_partial_bio only from the map routine. It is * allowed for all bio types except REQ_PREFLUSH, REQ_OP_ZONE_* zone management @@ -3151,6 +3170,7 @@ static const struct block_device_operations dm_rq_blk_dops = { static const struct dax_operations dm_dax_ops = { .direct_access = dm_dax_direct_access, .zero_page_range = dm_dax_zero_page_range, + .recovery_write = dm_dax_recovery_write, }; /* diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 47f34c50f944..e5e288135af7 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -287,9 +287,16 @@ static long pmem_dax_direct_access(struct dax_device *dax_dev, return __pmem_direct_access(pmem, pgoff, nr_pages, mode, kaddr, pfn); } +static size_t pmem_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *i) +{ + return 0; +} + static const struct dax_operations pmem_dax_ops = { .direct_access = pmem_dax_direct_access, .zero_page_range = pmem_dax_zero_page_range, + .recovery_write = pmem_recovery_write, }; static ssize_t write_cache_show(struct device *dev, diff --git a/fs/dax.c b/fs/dax.c index ef3103107104..a1e4b45cbf55 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1240,6 +1240,7 @@ static loff_t dax_iomap_iter(const struct iomap_iter *iomi, const size_t size = ALIGN(length + offset, PAGE_SIZE); pgoff_t pgoff = dax_iomap_pgoff(iomap, pos); ssize_t map_len; + bool recovery = false; void *kaddr; if (fatal_signal_pending(current)) { @@ -1249,6 +1250,13 @@ static loff_t dax_iomap_iter(const struct iomap_iter *iomi, map_len = dax_direct_access(dax_dev, pgoff, PHYS_PFN(size), DAX_ACCESS, &kaddr, NULL); + if (map_len == -EIO && iov_iter_rw(iter) == WRITE) { + map_len = dax_direct_access(dax_dev, pgoff, + PHYS_PFN(size), DAX_RECOVERY_WRITE, + &kaddr, NULL); + if (map_len > 0) + recovery = true; + } if (map_len < 0) { ret = map_len; break; @@ -1260,7 +1268,10 @@ static loff_t dax_iomap_iter(const struct iomap_iter *iomi, if (map_len > end - pos) map_len = end - pos; - if (iov_iter_rw(iter) == WRITE) + if (recovery) + xfer = dax_recovery_write(dax_dev, pgoff, kaddr, + map_len, iter); + else if (iov_iter_rw(iter) == WRITE) xfer = dax_copy_from_iter(dax_dev, pgoff, kaddr, map_len, iter); else diff --git a/include/linux/dax.h b/include/linux/dax.h index 3f1339bce3c0..e7b81634c52a 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -35,6 +35,12 @@ struct dax_operations { sector_t, sector_t); /* zero_page_range: required operation. Zero page range */ int (*zero_page_range)(struct dax_device *, pgoff_t, size_t); + /* + * recovery_write: recover a poisoned range by DAX device driver + * capable of clearing poison. + */ + size_t (*recovery_write)(struct dax_device *dax_dev, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *iter); }; #if IS_ENABLED(CONFIG_DAX) @@ -45,6 +51,8 @@ void dax_write_cache(struct dax_device *dax_dev, bool wc); bool dax_write_cache_enabled(struct dax_device *dax_dev); bool dax_synchronous(struct dax_device *dax_dev); void set_dax_synchronous(struct dax_device *dax_dev); +size_t dax_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *i); /* * Check if given mapping is supported by the file / underlying device. */ @@ -92,6 +100,11 @@ static inline bool daxdev_mapping_supported(struct vm_area_struct *vma, { return !(vma->vm_flags & VM_SYNC); } +static inline size_t dax_recovery_write(struct dax_device *dax_dev, + pgoff_t pgoff, void *addr, size_t bytes, struct iov_iter *i) +{ + return 0; +} #endif void set_dax_nocache(struct dax_device *dax_dev); diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index acdedda0d12b..47a01c7cffdf 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -152,6 +152,14 @@ typedef long (*dm_dax_direct_access_fn) (struct dm_target *ti, pgoff_t pgoff, typedef int (*dm_dax_zero_page_range_fn)(struct dm_target *ti, pgoff_t pgoff, size_t nr_pages); +/* + * Returns: + * != 0 : number of bytes transferred + * 0 : recovery write failed + */ +typedef size_t (*dm_dax_recovery_write_fn)(struct dm_target *ti, pgoff_t pgoff, + void *addr, size_t bytes, struct iov_iter *i); + void dm_error(const char *message); struct dm_dev { @@ -201,6 +209,7 @@ struct target_type { dm_io_hints_fn io_hints; dm_dax_direct_access_fn direct_access; dm_dax_zero_page_range_fn dax_zero_page_range; + dm_dax_recovery_write_fn dax_recovery_write; /* For internal device-mapper use. */ struct list_head list; From patchwork Fri Apr 22 22:45:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12824269 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 255C13D72 for ; Fri, 22 Apr 2022 22:46:36 +0000 (UTC) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23MKCxRk009531; Fri, 22 Apr 2022 22:46:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=euu/6zi3HJi+so0p84tl62liECR+yv7a3vF73FxYsBM=; b=q8HXBrJbjIr168CrxVZKWKA+g7ObxPmoryyKmPbK4WArSdfZTb5OUsMjKqN8lLpSh0k4 G5WQGWpyPjjbLR1a4rq0EqtRGgouRoF1j3NVlo2nlMjzZfkc8Hnk7K9WJTw7/8akDh3z 2UV0FAdXum8SakfFGGqd7kqHvja0iGg1zABhP0aP8QuFz0yU9hDZoO3eadoCG0P3NeVB NgGqOszhp2P+lfgHp06PVCRUadIlj9UlMJass0WDOfouSGYp6saic7cUJy15Y8NrnPZS 12bcNMeL20fKNGqQDBsg6Q/NSS40u2R61TNw+8EKtLxeNiO3GmGlxVUQnurYqtDOBtfu RQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3ffmk2yvfk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:23 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23MMg0fU008607; Fri, 22 Apr 2022 22:46:23 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2177.outbound.protection.outlook.com [104.47.57.177]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3ffm8e9nby-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cEIpVrxCoefF+0YH068vTqGJImqD5xd/z5LRpFO1SDXaWrFwx/6UzxQmUg79VVPLLClCGCuiUIqeitE7dokaVK8eUR43vhzLKLqF/TOffyjqWCVjMKhnAih9Kx6/fWTtKgc9Ww1PZhvFecLRzzRUUz1zL8zSeQJMPJUBEiivmSSXwk9HLlhSUBGXa+XjzZOml9VNjzwPr8QF9/ecGwCd/XeltD58VymhB5oyKCkY5WfhwLbCpQ6sKxhmABsHqm6Sj4U2LykU99fSM18cxlENKPkAffQQF3UCkEXs9z3/0gxi9Xq3MLw2gW+LBMDWx37rFPhO7gf0NL84k5mcmpwFYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=euu/6zi3HJi+so0p84tl62liECR+yv7a3vF73FxYsBM=; b=WPuVjFeyPYuEyJ8rZ/AU1dwCZOf1dZYAA5n/CtBRYwyuiHrDSKtlUKkU/E8rxxRBTtiQhFkz3ugkt8NFK5Yfp8vVVSdm+BphR1aIWZo/IERbGBM+Mdw/1B7fh4pfJkhIdwXmMZ+pA9f4Kz7TSFbDUqtCwryV5hHdhn1tjghFqMBV1piun4Etz+A5lfsIMVrb85kGcFNncfYRbaTo+eVTWAzsqulro8C6IzPnv8sXh2qjelJPZAvVfPKdCCpWnlgdBqf1pEycYUcmAGGD5BUnYUeD3W9rzOs3LtIx8y/1J8Mpb+5FoBm30dNsU7EhFnnONk9rnq/QIJqkfDHIsMDpLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=euu/6zi3HJi+so0p84tl62liECR+yv7a3vF73FxYsBM=; b=yj9T1BQpiCP+LI0cy2ZnyaXPNZh/eYuE1Pq09WhcCweApGTuXUuk6gVMKVFvs1MPvmR1OXRyAwoBeN5ynIOmOV8IPg3Xo1YtSq5BMZarpcXQSzNztNcwDBq8Kr+NKS8iwgxFxcMWvGc+GPj1w9qz39u8Yd/vAbJjEF6r99R+4fE= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by BYAPR10MB2550.namprd10.prod.outlook.com (2603:10b6:a02:b1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Fri, 22 Apr 2022 22:46:20 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e%8]) with mapi id 15.20.5186.015; Fri, 22 Apr 2022 22:46:20 +0000 From: Jane Chu To: dan.j.williams@intel.com, bp@alien8.de, hch@infradead.org, dave.hansen@intel.com, peterz@infradead.org, luto@kernel.org, david@fromorbit.com, djwong@kernel.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, x86@kernel.org Cc: vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com Subject: [PATCH v9 6/7] pmem: refactor pmem_clear_poison() Date: Fri, 22 Apr 2022 16:45:07 -0600 Message-Id: <20220422224508.440670-7-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220422224508.440670-1-jane.chu@oracle.com> References: <20220422224508.440670-1-jane.chu@oracle.com> X-ClientProxiedBy: SN1PR12CA0057.namprd12.prod.outlook.com (2603:10b6:802:20::28) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 69c9a4c8-6450-44ec-1462-08da24b1eb58 X-MS-TrafficTypeDiagnostic: BYAPR10MB2550:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iRfdYfj2xRTdl+0P4S12zNC15GON3VjK3AAyS6cZ2IH1/1oRQdRKLNhEtXisO6EB9lHU1EniKuAM/lXfH51LZ2QaXUf5QopSeqNjbI0FAU+APbXZjOyzSafAIaIgOLG0qU+FZWjIxRXsVo7NEZN5rKgaNzi+pXZ1g7Em+zrFW2oB2ZqPv5C2DnvwTsHAXVToEuc+QMjHTOIUqyEH2GQPPYFZKzycd1hQxjVivKSOKiB3f3ZZwiG+4HDStL71oRJjwVz4vGXsDk5NB+y+bGrQFg29HSK5oxB5HFT87RbqIo6nQw23s0sgtEqmJWYZcQ3jQk9MOhE7vWqdDbyd3j1Kd0TCx0KjNuOBjN4zjPITjertTWiiQJIVzQTFzJW8CLZ4ElXy3mK8O08OAsYtKl0XvIm3ojJ9j4yj0tFgekGXBo18h3JCHIReEwl+unV3FYxmOvBFzK0ghwJ+0ABh7IXvCwpRvE0LUxaReAimLLjhXA20dh02jzOhjQw0oFljeXQhuZwPJgi+6j8T0noh0uHyNzLgjHZ2ER8LU1HGtcbqHmZxR652nmmsJygtJnNfA75+2+nDdkLiYBNwWS6wD1ReKHtIu/XDE1+PW4RQgWYVkOMkbtbJWYIRKSkwHz82QRgVb2CLd/KXCXlCXr7XLmR9oU6Yw7Q9dsh9+52WJIeJLdmM3XHZRKOMpcR5A0J5TcdP X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6506007)(6666004)(6486002)(8936002)(508600001)(52116002)(83380400001)(36756003)(316002)(66556008)(66476007)(2906002)(8676002)(4326008)(66946007)(38100700002)(2616005)(86362001)(5660300002)(44832011)(186003)(6512007)(1076003)(7416002)(921005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: A6gJWyor6kSPBn7cT2M5kU0w9eHXoSTMWdAg+FYu+44RuimmwxKVV7BglxP+DqksY5OLTcZWkoNNLwO5ub/arecoQfNGxOLjbL8Z78wSAg9c+tl2eDGFMj/7dE44yup4qCVMzFBUAk2rR+tHVif44IezDhanVw1aPHzFUjpZlfTFJNsBkB0UOnT9m3r5MMXtp9k3FVcAvbVVehnPXk7AcuI+B8W2Dq5Ch0k1cgt/LmHQrdXqYSEyxX6yAC3ttOyKTNEM4MIL6tqo5D5Y+Xh/MkHAZaTbjZ+LBz62vCgD3OEvv7H4zqEOdcqNXF0Z+FNeJAtAEIbfHUe4vs2Os1qO09buHPhoYR3wzBx+sJpmeVnJhxB3Qx/zadpmIDKH6reMz1WJDO1sTRAk35vtG+Urllcw2Vve91uC0BMJNkZJE44pWFmQVd+/Sc6wc5KE/VGdh1myoiTMIJAU8DFu1qW15/9tBlwHrrk0wtzDIR8KEhadVidH2E1AQ9haUzpAZVNs9Wug0+5tQRjkTr7eMt4PCJGXERzWPBQskBh/1sRdsy5noZBUBVcBvD7jCkeZwXnHHOezRBn+gkJMyHjzFwNrzvp1F9ND1U7ybhtgwzmi3D0C09kuNrooPZ+BZ4mgrBEXQHJLQXWlTLMtJReTnFO8ZV5xOGPANAzuYJ2zV8cY8OfD2jEBsMT8mL8UZW6RzpNc0hB86n+9xV+xHqV1Ap15vdD3L7wt1alA1946G3BtJ5yKPAL9vQ3aJGOOkA9LiivpwopybJnxiq+1roZMPVb+H2T9tXJO2dHWKOK9LTevPa4uZ9oMhQHtf4Itc/wZaemLZ5pZOjXO+h3D6xI2f1mEL8BjznCOscnxCdmGcDlXuX/QEAFpqskAQr/I8SGSTxmMn10fUYQ8I849dL7bgRCje4mJKO9yDlOpZSgMMNwF7lar9NftYtWXivfeCspj0PKWpU1y++VJZw4LYP6YJ04EqaRIcZcsOwgCk50hIyP5cf5J26zMzVPkRSd88/PfMch2x46XS6rR+xU5om0legREeQUT6HZB9ti7L92PUAYwEcFlscGwoOwdgRk8c8eUH3BLc/JNoV9FE+T+j/OR/RZNg26iY3myiN9pNlfKd0fuNPQ+fQCXElaxUlkdUobjVGyPZtEvx3wf9ofsGKpaia5btubZQNe+SoNLvHNT9ceV9B0nETTbB+xBPgQ5KPSUdlYwUCibJtFtZfi+chu8rJleirYr71D82esEGMS1v4GwyWpk3b0jyPlP1mC3vMvdGx37Xit2dGCKSaym5+CNMrg5JIeQioUkdltMR+2xHbWA1ieqyWr+kcb74VdHFj/sHotfvQnN4iwLmEgBM2YBd2qe9Pe2NvEEtoEy26IvYAN4viQlHFPMagf6xB9lgaOROKT8TKEKB6fqGy9K8XArCsiJ7WYzF7kfjbE9cYEuX59DtgsAm4QyN9AgVuniFWaqyZduQYpX+1gcz1motCriKmKTnOZJC7iwLF5t7+lhHYIL7aJ9e6XcDIo1J6bJ47mrQpeFHmfqx2viqE209wNGCx9pAOYVVFPivQxqbAKllLWiDF0fLmTcHDOebLZwxqa/ZPHros3scbVRTu2JCym+933s9Sq6GHVRwUHT4ty7DrLWaE/f35EKTU19OTWDW92/UCu7Kqp0iRR9tabMc0yLYKU6X8szUtqubognnMizpcZn1R4eRDSYAfLG71gPEfZWM232mCLoXtv3vHGpGVoskvs5eesA+QoOcTAYbFOd7sxOMJI= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 69c9a4c8-6450-44ec-1462-08da24b1eb58 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 22:46:20.3982 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wLLzXUWY7uu8I5YjPFe7TLVb6c5RuyGr/ccfrGr0lI2tfPt70mgqHRtMrqT1nLYV5IgG7T+LXeElXgtKDnFMhw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2550 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-22_07:2022-04-22,2022-04-22 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 malwarescore=0 suspectscore=0 spamscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204220095 X-Proofpoint-GUID: zT3EP53qVeuwS7X29LISdzp6MV5I4y-H X-Proofpoint-ORIG-GUID: zT3EP53qVeuwS7X29LISdzp6MV5I4y-H Refactor the pmem_clear_poison() function such that the common shared code between the typical write path and the recovery write path is factored out. Reviewed-by: Christoph Hellwig Reviewed-by: Dan Williams Signed-off-by: Jane Chu --- drivers/nvdimm/pmem.c | 73 ++++++++++++++++++++++++++++--------------- 1 file changed, 48 insertions(+), 25 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index e5e288135af7..0961625dfa05 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -45,9 +45,25 @@ static struct nd_region *to_region(struct pmem_device *pmem) return to_nd_region(to_dev(pmem)->parent); } -static void hwpoison_clear(struct pmem_device *pmem, - phys_addr_t phys, unsigned int len) +static phys_addr_t to_phys(struct pmem_device *pmem, phys_addr_t offset) { + return pmem->phys_addr + offset; +} + +static sector_t to_sect(struct pmem_device *pmem, phys_addr_t offset) +{ + return (offset - pmem->data_offset) >> SECTOR_SHIFT; +} + +static phys_addr_t to_offset(struct pmem_device *pmem, sector_t sector) +{ + return (sector << SECTOR_SHIFT) + pmem->data_offset; +} + +static void pmem_mkpage_present(struct pmem_device *pmem, phys_addr_t offset, + unsigned int len) +{ + phys_addr_t phys = to_phys(pmem, offset); unsigned long pfn_start, pfn_end, pfn; /* only pmem in the linear map supports HWPoison */ @@ -69,33 +85,40 @@ static void hwpoison_clear(struct pmem_device *pmem, } } -static blk_status_t pmem_clear_poison(struct pmem_device *pmem, - phys_addr_t offset, unsigned int len) +static void pmem_clear_bb(struct pmem_device *pmem, sector_t sector, long blks) { - struct device *dev = to_dev(pmem); - sector_t sector; - long cleared; - blk_status_t rc = BLK_STS_OK; + if (blks == 0) + return; + badblocks_clear(&pmem->bb, sector, blks); + if (pmem->bb_state) + sysfs_notify_dirent(pmem->bb_state); +} - sector = (offset - pmem->data_offset) / 512; +static long __pmem_clear_poison(struct pmem_device *pmem, + phys_addr_t offset, unsigned int len) +{ + phys_addr_t phys = to_phys(pmem, offset); + long cleared = nvdimm_clear_poison(to_dev(pmem), phys, len); - cleared = nvdimm_clear_poison(dev, pmem->phys_addr + offset, len); - if (cleared < len) - rc = BLK_STS_IOERR; - if (cleared > 0 && cleared / 512) { - hwpoison_clear(pmem, pmem->phys_addr + offset, cleared); - cleared /= 512; - dev_dbg(dev, "%#llx clear %ld sector%s\n", - (unsigned long long) sector, cleared, - cleared > 1 ? "s" : ""); - badblocks_clear(&pmem->bb, sector, cleared); - if (pmem->bb_state) - sysfs_notify_dirent(pmem->bb_state); + if (cleared > 0) { + pmem_mkpage_present(pmem, offset, cleared); + arch_invalidate_pmem(pmem->virt_addr + offset, len); } + return cleared; +} - arch_invalidate_pmem(pmem->virt_addr + offset, len); +static blk_status_t pmem_clear_poison(struct pmem_device *pmem, + phys_addr_t offset, unsigned int len) +{ + long cleared = __pmem_clear_poison(pmem, offset, len); - return rc; + if (cleared < 0) + return BLK_STS_IOERR; + + pmem_clear_bb(pmem, to_sect(pmem, offset), cleared >> SECTOR_SHIFT); + if (cleared < len) + return BLK_STS_IOERR; + return BLK_STS_OK; } static void write_pmem(void *pmem_addr, struct page *page, @@ -143,7 +166,7 @@ static blk_status_t pmem_do_read(struct pmem_device *pmem, sector_t sector, unsigned int len) { blk_status_t rc; - phys_addr_t pmem_off = sector * 512 + pmem->data_offset; + phys_addr_t pmem_off = to_offset(pmem, sector); void *pmem_addr = pmem->virt_addr + pmem_off; if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) @@ -158,7 +181,7 @@ static blk_status_t pmem_do_write(struct pmem_device *pmem, struct page *page, unsigned int page_off, sector_t sector, unsigned int len) { - phys_addr_t pmem_off = sector * 512 + pmem->data_offset; + phys_addr_t pmem_off = to_offset(pmem, sector); void *pmem_addr = pmem->virt_addr + pmem_off; if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) { From patchwork Fri Apr 22 22:45:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 12824270 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED6E43D72 for ; Fri, 22 Apr 2022 22:46:41 +0000 (UTC) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23MKCxRl009531; Fri, 22 Apr 2022 22:46:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=9vzBUP3NYrMycr6T6RziUA5yEk+8IxuWyFBQzxdQVPs=; b=tEZ36LU2P4fo8ktrrq2shoV6pu1Xj7UVrGExCA3EOrbVwWN6WFDEN7y8O0xNTRv5YFmt EfIiYNNFUXJYmOut1+/TERD/CyJ7NkCEUuq2QWQnFef1blK1q+hYUPAj7xAQ+197lhvJ 84GmCFQEQXZ1iTSUCCSzAmwb4iRZ5pYAhipvfs7LvZULi7Dy6mjckP0UjNo/87bkaoh7 KS38zR1eVxSo33yDcwq3EIolykIlBFgx9ELAGpIx0P/5HSN5Y1cdn2lb1S88yuam4+QQ CxSNlU23momMztsUVs91gVROIt3HG+wehUgiknfuARh3cpXFenkLDwtkTEPztwxElVOy tg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3ffmk2yvfm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:28 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23MMeocr007341; Fri, 22 Apr 2022 22:46:27 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2176.outbound.protection.outlook.com [104.47.57.176]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fk3awf5ut-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Apr 2022 22:46:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b8eYsUidtu7Oe87b+RSWToQ+8s9urznpky9P0HN1rTp5sTmqGG6My7Ez35Bk3/nsh7yYqHjnnoIWgjAhaaugAH2AEUVlZL93xNF6QNh0FwbkI7NwdpRi0lWWzPdB9PNmI7BC3yyU8FAqueM4ktHYZT35N4QayPhbfszAkFSI69uyhDgbRoEhZSMHYlgZiHWASrfYgVWRpAja1l0rYXcz8LApYTWqCRffLQ4PB5zNH1WAPio4lSG7ZVt6xJ6cHtaIXvoKE+yPq7WPxxJSAl2BXUFvhMwlpol6s7InBd3oMPgjyTwVCbLI4HrnvZh9b6xX5zIdlNbP+b5pP6iqYQR4Sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9vzBUP3NYrMycr6T6RziUA5yEk+8IxuWyFBQzxdQVPs=; b=Pu23dMS9beJvCBB7btcKPM+7N2HwaYIVP54Ts8kpKcxgELExcvUcDrRk4V0qHJ+6RU8buh6qENqZHWzh7EXyancHsN3+jM8T/ZrdcAjuqickazqVuGzfX71RseQ8AhwhfA0hH2l5fDeef+GVApJwHS66v88Iq5bu39G7EqaooIhVL4W14EqI1oakVeK4Ju/jq5fmyL7SFMzavGTau8yk1Eri3sbNSd2xR/gKQ2pRV++X8cEafLzmGQIq7ELcPQNyq0bykWcudl01FMdXbd61b3w+DWGAzPeBVAT8AiF5avxzdBJa57pFnTyov8NR+MzhHdq5sCYvc3GwSNrBbPnyGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9vzBUP3NYrMycr6T6RziUA5yEk+8IxuWyFBQzxdQVPs=; b=jMWlarNdoLJTJAGpaj256nikt5H438IQQTMbMygKjZQEZvoOxBlFm/db1tzwC6YUiKFRNw/zvIt/FWr5hrKLy/cFtc1hbP+sBGhyhRKYXq+6vmNRWeeHEG96jV8V/XIHwSzq+pnC7FxylBBGjTfCOlJeVNtShYOMJLIrPLef/0o= Received: from SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) by BYAPR10MB2550.namprd10.prod.outlook.com (2603:10b6:a02:b1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Fri, 22 Apr 2022 22:46:24 +0000 Received: from SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e]) by SJ0PR10MB4429.namprd10.prod.outlook.com ([fe80::1c44:15ca:b5c2:603e%8]) with mapi id 15.20.5186.015; Fri, 22 Apr 2022 22:46:24 +0000 From: Jane Chu To: dan.j.williams@intel.com, bp@alien8.de, hch@infradead.org, dave.hansen@intel.com, peterz@infradead.org, luto@kernel.org, david@fromorbit.com, djwong@kernel.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, x86@kernel.org Cc: vishal.l.verma@intel.com, dave.jiang@intel.com, agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, ira.weiny@intel.com, willy@infradead.org, vgoyal@redhat.com Subject: [PATCH v9 7/7] pmem: implement pmem_recovery_write() Date: Fri, 22 Apr 2022 16:45:08 -0600 Message-Id: <20220422224508.440670-8-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20220422224508.440670-1-jane.chu@oracle.com> References: <20220422224508.440670-1-jane.chu@oracle.com> X-ClientProxiedBy: SN1PR12CA0057.namprd12.prod.outlook.com (2603:10b6:802:20::28) To SJ0PR10MB4429.namprd10.prod.outlook.com (2603:10b6:a03:2d1::14) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 271b81fa-46b6-4b73-1b75-08da24b1ede0 X-MS-TrafficTypeDiagnostic: BYAPR10MB2550:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 25uy3vsQXgPasuMAz9Yk/XCyb8KG93PnoIDlEiDUGG9z/yzHc40QVgK8U+QqJbYgbEnU/phPFCWUNHUImivRV8XVR1mcMPnN719avwZtoGc6qsNrZ6rK2hWhYtulZx8yWKOynQni9ZwCr0SSZrQddZ3NPHTSi3CBqNRq2tDJJasv/GS0uNDv52+0wY+exdlO+qhypD32vM9uQeiNj3R0fsNfBQ0rvnoGAz0s+/4D1wM7Xr+/2N6c17v9zfF2ocbb2EM9b4Sh60WB2+Hr5IZBmPhODQTWQ/ew+zS87exe+mMeEu6qfvb/vtnvScmRVznaTjPJioQqNCzPT2nyErWxhNb/DPN5vgBC8PyBuC1Vu8kx57h6xHeEGXrvV5C2/QPiouF3ha02S46JIRVWVUVMcPWbGsJvBQVm6VmHdXw8WnLshC4LC4UGLNnFdxc+SUZJkiXrFhm6xmOtdz9IGAHP7lOFlGCMo4BGEW02bSo+jt8YY0rDemdfArQeQFqtngpcA6ib6DWXI1+eHqnvXRLD5D/u43Y8MoZQB1656PYV1+3sQLveV0VHMWbI/lX1Pdu0uHIMHGHHpcuy8ShEIrKAxGLI7/o+kagA4QfzieklZ0QFnv1JDnNzLcGGICsBgmEivq7hxFf73A4b/dVYYBcM8AIGMzg9Zm9vAh3hDNOO0r55HG3DOZDRew0yOFQNw5+w X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4429.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6506007)(6666004)(6486002)(8936002)(508600001)(52116002)(83380400001)(36756003)(316002)(66556008)(66476007)(2906002)(8676002)(4326008)(66946007)(38100700002)(2616005)(86362001)(5660300002)(44832011)(186003)(6512007)(1076003)(7416002)(921005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: szyhE4mqZjdpMHs1RiPFhqckN3Ba27VgtEtiMcdL/8VLpmzy0pJMGELnDvlR/bYovwgrjxQwKlYmq5oDz/nQBA/7BON2S2c+2aAQxaFym1ayqj0YrcrGxnVsDyRtMvBINw9+gv0pR4Z+3Q1iGRdGis67+IPThbCqS3W3yUlEK8sodILqOUr7+YNfQP9+niKeWxm1BQVd3aaHAaQiXJCXMh0JUWo3Mm56pC1vTxLSc3xDmF96nhEguI7emCR/Cjpf5L5KcL3LCkHd95Vho64TzzOmMrS0SqsoZlpI6hDXrCNZ3y/fi6u9pXpSsppFBIqgURRTaJ4h9swLb7JVRDFsGK1VwyOUVkOy6oepkW9SwGaWZeGFktH3ecepcO4cKb4YtB2kQ1rEvsvrobMe1X0SOmaTJFwbkr5U20yPqoc3kpYIWOmabw2Cd5tsYf/MaDAGYmGjJi67sjjlEV/cEbAA6PE6BSneCkrFTu3SSvP4MjICyPtMb0iigGp+UKj2RzovUiyMAtLFOel3Ue8qAa/H58csc708bP1TaHvlkS1rA9b7+LZpp2eGRpuS+q5rLVbg7dadJLkaBRX3fKp58TnJvvdaYD9Y+tn+j3g9PWvOedTrCLeOhMOSOTpTn+xMn4KybFpuqdtMaMXjzyVcWC5KJBp8hJp2IqzYgX4k2xI/iNICBwJGN8MbjDW8d/FGHG3SL53jMANuk57STj92iPAhqjI7OhswbeM1gzJZk1E42bZTIenSShtDVKFoSmPitZjBTlur5jjxJWMDcAhrO+cHPOWg0xurYec2DhLcqFrvio6E+z5a3Wvjw5Kdo7vM5t3gZTtBSWSQBk/6oZK2ycc07J8JqySkNgkXVbWMu+TVzVAFRhILlQqG+0/ieTyMvtS0c3OAXYTa+haDgHrKXBjHec7zR3edUhCU25M+oXxDI+RGIsQak8LO/F7sj76wPXw5x8JQLMq3M6I8EVFLbcdnZV9O7DaJdizTotGfpbvj/4Zfc8YstOX/8+K5j6DOh+9vxyk4LaWUKdslMbXHWLgl9zftzGX8faAcpMXnELWKLAZFaif0mp/PYStunlkpe9CpIRPzYWRhOowlY53kUf3wB4DHKRd9i/dtvEb3frpFNIGjw94Xdt9szs2F3ql2P/Kg+vznWTJ3g8zbEfYUISAqUlGFNMOu0Vjh10eFlUg1CPkV3kF/wkADErc4eRhzQAW+rSudppj1vl6QnypA9O1Df224qufT9NmuWFiHLRnifTH9KuXR+xRWkA6VTIVphUH7XhBrxiS2YNemEIhK1XmqrxiwGOWWWYqGK2TkPKio/Zv0OPCY8Q7z8q2B1i7YTI/UZAhCQ4IAr+iActg4prhqaxoX/mD83O4fZGpYu2EzJS1iaxcORmksaXLw5K8oyUcWZ/Qx1mzR+LYdY97HocDndBFJReipCbmErBrSYBdXDewP+80s9Qvlqz3g3jaDAQgTw4QaoyWQAsAjwAgw3tXfve+0r2J5j9v4BiFhT4t2hRPkZqDWyCYcOFVlTEsV+hyiubj9OROQpCbjUv6Wrpmm8EaewY0l1aZBIzHWp8FQZuVYipk1+jgD8fDtyB/GIG3UqU7uTPunxArBPZV7tBnSO8egO0PbQzFwjKuIiFhvZf7nfw3eCZCWdRWi2XV+Ynp5hxNzdle7PpJvk7wbQV+L8vqU+t7sZAnmJUC4AxoVWhr9oHIxBENRuCN0+P+FOl6d/keYEOEv9TtZ5S446fjP1CpZ0rZx0ma3iTNBmxx1cwA= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 271b81fa-46b6-4b73-1b75-08da24b1ede0 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 22:46:24.5981 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RAx1Ne9G1tvHOd652FbJtHEGWYlHtArVOyhcGrMQJ5XVe4X582p+aExqjituzGq1W8csSWQA6Eheg0VrzZLp/g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2550 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-22_07:2022-04-22,2022-04-22 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204220095 X-Proofpoint-GUID: 72kJ_8G3j_TQco9eRrdR7IHSjCkHhy8j X-Proofpoint-ORIG-GUID: 72kJ_8G3j_TQco9eRrdR7IHSjCkHhy8j The recovery write thread started out as a normal pwrite thread and when the filesystem was told about potential media error in the range, filesystem turns the normal pwrite to a dax_recovery_write. The recovery write consists of clearing media poison, clearing page HWPoison bit, reenable page-wide read-write permission, flush the caches and finally write. A competing pread thread will be held off during the recovery process since data read back might not be valid, and this is achieved by clearing the badblock records after the recovery write is complete. Competing recovery write threads are already serialized by writer lock held by dax_iomap_rw(). Signed-off-by: Jane Chu Reviewed-by: Christoph Hellwig --- drivers/nvdimm/pmem.c | 87 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 79 insertions(+), 8 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 0961625dfa05..91ce7b2b2ada 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -266,21 +266,43 @@ __weak long __pmem_direct_access(struct pmem_device *pmem, pgoff_t pgoff, pfn_t *pfn) { resource_size_t offset = PFN_PHYS(pgoff) + pmem->data_offset; - - if (unlikely(is_bad_pmem(&pmem->bb, PFN_PHYS(pgoff) / 512, - PFN_PHYS(nr_pages)))) - return -EIO; + sector_t sector = PFN_PHYS(pgoff) >> SECTOR_SHIFT; + unsigned int num = PFN_PHYS(nr_pages) >> SECTOR_SHIFT; + struct badblocks *bb = &pmem->bb; + sector_t first_bad; + int num_bad; if (kaddr) *kaddr = pmem->virt_addr + offset; if (pfn) *pfn = phys_to_pfn_t(pmem->phys_addr + offset, pmem->pfn_flags); + if (bb->count && + badblocks_check(bb, sector, num, &first_bad, &num_bad)) { + long actual_nr; + + if (mode != DAX_RECOVERY_WRITE) + return -EIO; + + /* + * Set the recovery stride is set to kernel page size because + * the underlying driver and firmware clear poison functions + * don't appear to handle large chunk(such as 2MiB) reliably. + */ + actual_nr = PHYS_PFN( + PAGE_ALIGN((first_bad - sector) << SECTOR_SHIFT)); + dev_dbg(pmem->bb.dev, "start sector(%llu), nr_pages(%ld), first_bad(%llu), actual_nr(%ld)\n", + sector, nr_pages, first_bad, actual_nr); + if (actual_nr) + return actual_nr; + return 1; + } + /* - * If badblocks are present, limit known good range to the - * requested range. + * If badblocks are present but not in the range, limit known good range + * to the requested range. */ - if (unlikely(pmem->bb.count)) + if (bb->count) return nr_pages; return PHYS_PFN(pmem->size - pmem->pfn_pad - offset); } @@ -310,10 +332,59 @@ static long pmem_dax_direct_access(struct dax_device *dax_dev, return __pmem_direct_access(pmem, pgoff, nr_pages, mode, kaddr, pfn); } +/* + * The recovery write thread started out as a normal pwrite thread and + * when the filesystem was told about potential media error in the + * range, filesystem turns the normal pwrite to a dax_recovery_write. + * + * The recovery write consists of clearing media poison, clearing page + * HWPoison bit, reenable page-wide read-write permission, flush the + * caches and finally write. A competing pread thread will be held + * off during the recovery process since data read back might not be + * valid, and this is achieved by clearing the badblock records after + * the recovery write is complete. Competing recovery write threads + * are already serialized by writer lock held by dax_iomap_rw(). + */ static size_t pmem_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff, void *addr, size_t bytes, struct iov_iter *i) { - return 0; + struct pmem_device *pmem = dax_get_private(dax_dev); + size_t olen, len, off; + phys_addr_t pmem_off; + struct device *dev = pmem->bb.dev; + long cleared; + + off = offset_in_page(addr); + len = PFN_PHYS(PFN_UP(off + bytes)); + if (!is_bad_pmem(&pmem->bb, PFN_PHYS(pgoff) >> SECTOR_SHIFT, len)) + return _copy_from_iter_flushcache(addr, bytes, i); + + /* + * Not page-aligned range cannot be recovered. This should not + * happen unless something else went wrong. + */ + if (off || !PAGE_ALIGNED(bytes)) { + dev_dbg(dev, "Found poison, but addr(%p) or bytes(%#lx) not page aligned\n", + addr, bytes); + return 0; + } + + pmem_off = PFN_PHYS(pgoff) + pmem->data_offset; + cleared = __pmem_clear_poison(pmem, pmem_off, len); + if (cleared > 0 && cleared < len) { + dev_dbg(dev, "poison cleared only %ld out of %lu bytes\n", + cleared, len); + return 0; + } + if (cleared < 0) { + dev_dbg(dev, "poison clear failed: %ld\n", cleared); + return 0; + } + + olen = _copy_from_iter_flushcache(addr, bytes, i); + pmem_clear_bb(pmem, to_sect(pmem, pmem_off), cleared >> SECTOR_SHIFT); + + return olen; } static const struct dax_operations pmem_dax_ops = {