From patchwork Mon Feb 14 09:51:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Markus_Bl=C3=B6chl?= X-Patchwork-Id: 12745350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5F01C4332F for ; Mon, 14 Feb 2022 10:13:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346018AbiBNKNu (ORCPT ); Mon, 14 Feb 2022 05:13:50 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:44706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345407AbiBNKNI (ORCPT ); Mon, 14 Feb 2022 05:13:08 -0500 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60047.outbound.protection.outlook.com [40.107.6.47]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CABB65780; Mon, 14 Feb 2022 01:51:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LQhrK68ikLxBZuSMAwhuCcI9r34kChWj574fpyNm1axrIFEOIXpqw/6Yav4SX213GXJ/kIIk5qnnaRaGoqsXOtJigIBXaXuUWeSEmuWHYOQwj8DzQ7y+eVEq74AtHcg2aLU4PbuQcGX/Da+/Ab/gywNri2byiJkb6NXx64avUz5wI98Ot4eBu4Dwyxvt1Sj0gbHRppwvaG8aZGXgMzidUngYSuel9BSENRn6ab72FR1Rebe6KVkscakLmbAVgxhlT1qtiUIMBr6wHX9jTq2MdNJbiqY5z5dJl1zs5bFfMbc2e9+veopgcsnCzW+xjPXlbGc5AnndBFIcbht+y1XhXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gsQ7L8rQ+j7uFSbvBR+XBVMOjuIRXFeibMuQAS5P3I4=; b=bxAmt2zchBVvfMVs/VB4j+bIZ4VNcFIWhBdLp405grPHfox4JieOkqxwaBsMc3jt9TGtveIXaSoXREmLXI0onbZsO03HSIXxnKHeLvheTzZAC3A52BDrJ4EtJK1WmG/uTpmn7Cu3524PHz54hSpuDJst3KTfAGI8DfR8k91Du0xG/fGoef6zieGmi9Tp3xaMhiIC9nqcDpI6Q8XSW52B1eixBxkIIuHsZX2vHbaK6yuCzMDKKRuBSNx4lfIVfBdLM1aO2vRTK8zdWJiP1l39ducFVfvu0lHvQb3zZG39hGOHJi4F9Z6HAx2xTxCs48/TxQ6db2UZDf42fi75ILjRHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipetronik.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gsQ7L8rQ+j7uFSbvBR+XBVMOjuIRXFeibMuQAS5P3I4=; b=GxqKGLT6ddfR0b6EzkEExPkOdJpQgUVcGXaEx88wKLLhl9bmrwRz0S363QB7SU9iuHpHXR9KjprogTi4OfoirJTC5pqcbECIEvss0Rxuz3g3WAlWhz6M5iPXekEjEVuKeHxswPaO2ZhV+TF59rX6Rg6jaklOtlH1LO15/J0pztPTgdY9x4+wFbZvdtKkv0dzpCVo0oUUqXzY68K2B1qahguB5K8Ru+qNhTDmpK/eVNKpV8GO5cttfV9Yu2QjkCVi1redF3xqxM741daA3X90Z82fqPHsjN2P8QIoE9FdOUJ15RWZ4pWfT6rSksOXgqGkSFUESXcbc2F1LrKx2WNuSA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ipetronik.com; Received: from AM9P193MB1572.EURP193.PROD.OUTLOOK.COM (2603:10a6:20b:307::12) by PAXP193MB1933.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:1c8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.12; Mon, 14 Feb 2022 09:51:09 +0000 Received: from AM9P193MB1572.EURP193.PROD.OUTLOOK.COM ([fe80::f127:41d:4d91:9dc0]) by AM9P193MB1572.EURP193.PROD.OUTLOOK.COM ([fe80::f127:41d:4d91:9dc0%7]) with mapi id 15.20.4975.019; Mon, 14 Feb 2022 09:51:09 +0000 Date: Mon, 14 Feb 2022 10:51:07 +0100 From: Markus =?utf-8?q?Bl=C3=B6chl?= To: Keith Busch , Jens Axboe Cc: Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Stefan Roese Subject: [RFC PATCH] nvme: prevent hang on surprise removal of NVMe disk Message-ID: <20220214095107.3t5en5a3tosaeoo6@ipetronik.com> Content-Disposition: inline X-ClientProxiedBy: AS8P189CA0013.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:31f::22) To AM9P193MB1572.EURP193.PROD.OUTLOOK.COM (2603:10a6:20b:307::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e2d5602a-d433-455c-8c2d-08d9ef9f86b8 X-MS-TrafficTypeDiagnostic: PAXP193MB1933:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZtwbfTg+BHk4anzJV88Pzd+jtxvYsg14hLiSqTJHZA+LOmLvxSmTxCnmOscLOIPd07Wep0CwFZ4tpsB4epn9f1fM+v+SqkFp7DXIJBnjuECBOpEVhaBA/EQTOdx4sGA0/idooFfY+mNedmzN34tHFrLx0SDjIt9NSyMAKk9oklsuc9ZvytNV74bi9VcFivFvpcLomlyY3MD6HezOLMZWqJYDMnDcX1ur68tHNhboLXVjOz/XwnfQHEn9zmZxX76DUEhQx3axj8M+Tr2IFnIaFY19tyuyrRCSCBKYIAeUfY8sRyUkItc8CBH2ZT7hbyVWp6/1v8IqeRAx0UlJX+oCV+PP7Hdx/IaPqL/bzB/ZK+kOwPo1Xli9W9bloxfqengxTBYGJFZ47dHnVJxK4DYOLloZNT5pQuRZ74BrSU7u/s5xLp7lw8Fq6l42zwLIULD3vq14LyYVZDjTBa2yv0yr7fBGn31e97l9iFhX5nRG8AxLaerhyVswkZjdtSiWx/wz24t9AsNwrfwBKlXws4C+ZujOhVsxQMNGKP57zNYnhVJHSObMyAGGmwv+7CxJ26J1MRLO0zbyNxkiqg+E3rz/lhx3t40HR2WV111fpjJYHrZUkYOu4HcK9HvdyYco0HBO2KmGJvkOjdR1sO4n34Cykg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9P193MB1572.EURP193.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230001)(366004)(83380400001)(8936002)(5660300002)(6506007)(2906002)(1076003)(186003)(36756003)(2616005)(66574015)(66946007)(54906003)(110136005)(508600001)(6486002)(6512007)(38100700002)(316002)(86362001)(8676002)(4326008)(66556008)(52116002)(66476007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?rMon09l31ilTVaNdGxPCa7rQE7Mn?= =?utf-8?q?1+W2Kt7k719yOK0JfQhUelHoFw0oriO2ciZxGwe7bARMBoXV3B/7YxUQtbp3FVnLz?= =?utf-8?q?R3Qdbg8e3DXAV0QxWYRlYpJV0UMp4EDnEO7NHfKubFpzvH4dOjvZw9tET9G3AJAJr?= =?utf-8?q?sNoiAqgQEd3fzgycZUsn6yIXM93uVFOfmwv8B+TQ0fmZaMxAltLpHBPHGguEwx2MN?= =?utf-8?q?OiegHW6uRJhia4KNlv2TdEUUOm6ZeV52/I/a5CjbfY6rqBUYBqEr9s61Se3hY1tne?= =?utf-8?q?PBY/RPpKH7yV3ZnGhcwYyeaz6q6FTsfOkPdGfAt/WL9JvCHm3vfMKZZcc7uIwCE2b?= =?utf-8?q?/i3nkA8DzjhuC8cD/6He8newQcZsKDc2/3fhP/aqASA5Bbn6eAB4Yvt4nzqsF0yUY?= =?utf-8?q?1+aDG1pye+/5wd4foRVz3GruoT2alm69KCqw5CXzKui+Feq6lrOG7I3T//owfyA76?= =?utf-8?q?8p+s646Hw9gdgkyEy5pUEf5aF2ec2sK99052R/NF7f/yjluXfOfn0zVTxOeTPl33K?= =?utf-8?q?XZyVXLNZxLqwDKisyKZ2lI3GWdCCcvVWK1Ad0yg6dwSSZps9PMOVxRzv+L/i+7rEN?= =?utf-8?q?YeySI/3At2KoBAJlBoj4hrrcb+VKu4UjMWPJWSc33/9xvk5Bh9jT9ef97+2iA8QPZ?= =?utf-8?q?PLZ46jbnU2nuw4g3h+arMzIeeWLgWCAgAoWUlznWMaYvZKPd3h2S08G8UOB6MtAeu?= =?utf-8?q?CvNuW2Y9CmGJdnHn6IDIM7j6vpr65LFwHY9yhZOrx7EkQLKEw09Wfx5RlEcKaED05?= =?utf-8?q?EDlWRoAZ0z+W8/xdsLDiRZ8AnUvGMJhn96rn67vTIS2MITQFvHUJSbWvmnHH/4ccM?= =?utf-8?q?sixO1wBgw3iQI15qYJ6m4ot2PaoRfLsFJ9ZTqKdL919whBt916wDwUwwYGsjfY0IV?= =?utf-8?q?vW0OUb/OiRKWVRe9sbvNKVAmgirILguaaL2MgYwAMhAULYMKY+CfAa8jkhoF9egdM?= =?utf-8?q?O4wM1GY5mr92f73vsbXHmth8dNFI7Z0mwlEGeIAdZNFTXpS7hAfejVYSwGVTtipnS?= =?utf-8?q?Y+akAo1mXHGsQkGzTpnRgbmg/NsxvZfEb5GG09z5S3AWfuRuhkWnD+aoAF/V4kxDf?= =?utf-8?q?L5YcBhU/7xYZfvJJxwBaT7YlZUkTd4bL994vfPXg2ywnaFiWUEnaCAG2vmmO4h2iT?= =?utf-8?q?TSfevWcWVL+wIP1F8fhgcSOtdbT+PPi8pq18Dwx+w/U8TwPcfCks0nS/qtJ4UzM25?= =?utf-8?q?EwfImrHXdYmzWbfUu+jA8qQd5vMyFW9bge9oPKZOiC9eMiIcp10DYny9bkm6IJGDv?= =?utf-8?q?soVWcOPxTLC0SJBkFMtAnDuX+mlTKlAfw+g0Qvv4U9AVKH2/JKnXJGIk+Q6vS1poj?= =?utf-8?q?NkDZn6Q84yuTLBLE5KJz482IS98MiLsKzSOaer6A9w0HKLOxlkZjbJjL6JefHKVHY?= =?utf-8?q?qgrzrxQDQz8e8l0wpUCPF3wfrVMNe9CmTEPBfx802VOWRgFIOn5emh4DdmfBckjQ4?= =?utf-8?q?0Mep26VuLz2ufcK3FRd/G9CLuS6aOV2zzmRPU9fNqbgDn5BVxUgWoRV0hilzswJnr?= =?utf-8?q?RiRZFBSapuvKFepg+2OVx8yS0CX/eP9XvMQvboVcKMARDNYwtjbKmF713bxUjoxLt?= =?utf-8?q?HGz0efL3xk+frfJ721YKyBBsQETPVa3rEDTB/6Ri5c505Z+gpemT8afNSaEBPHyLc?= =?utf-8?q?yw5y2jiwBBQtdQ9d+eJkc7VfUffenE0A=3D=3D?= X-OriginatorOrg: ipetronik.com X-MS-Exchange-CrossTenant-Network-Message-Id: e2d5602a-d433-455c-8c2d-08d9ef9f86b8 X-MS-Exchange-CrossTenant-AuthSource: AM9P193MB1572.EURP193.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 09:51:09.1215 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 982fe058-7d80-4936-bdfa-9bed4f9ae127 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: M9z+Lb2VDXu5Q/XRBcf2uF0dvAq/mK5x7Ei8cLJt2xxxky/moWsR3gR6YAIg+eFKOrjGlfoUCzKrA3kaU07Md3woYz5gahBPcqVofcIf0zg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXP193MB1933 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org After the surprise removal of a mounted NVMe disk the pciehp task reliably hangs forever with a trace similar to this one: INFO: task irq/43-pciehp:64 blocked for more than 120 seconds. Call Trace: __bio_queue_enter blk_mq_submit_bio submit_bio_noacct submit_bio_wait blkdev_issue_flush ext4_sync_fs sync_filesystem fsync_bdev delete_partition blk_drop_partitions del_gendisk nvme_ns_remove nvme_remove_namespaces nvme_remove pci_device_remove __device_release_driver device_release_driver pci_stop_bus_device pci_stop_and_remove_bus_device pciehp_unconfigure_device pciehp_disable_slot pciehp_handle_presence_or_link_change pciehp_ist I observed this with 5.15.5 from debian bullseye-backports and confirmed with 5.17.0-rc3 but previous kernels may be affected as well. I read that del_gendisk() prevents any new I/O only after flushing and dropping all partitions. But in case of a surprise removal any new blocking I/O must be prevented first. I assume that nvme_set_queue_dying() is supposed to do that. Is there any other mechanism in place which should achieve this? Unfortunately I am not very familiar with the blk_mq infrastructure so any comments and suggestions are very welcome. Best regards, Markus Signed-off-by: Markus Blöchl --- drivers/nvme/host/core.c | 2 ++ 1 file changed, 2 insertions(+) base-commit: f1baf68e1383f6ed93eb9cff2866d46562607a43 diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 961a5f8a44d2..0654cbe9b80e 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -4573,6 +4573,8 @@ static void nvme_set_queue_dying(struct nvme_ns *ns) if (test_and_set_bit(NVME_NS_DEAD, &ns->flags)) return; + set_bit(GD_DEAD, &ns->disk->state); + blk_set_queue_dying(ns->queue); nvme_start_ns_queue(ns);