From patchwork Wed Sep 27 03:12:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 9972999 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6E91860365 for ; Wed, 27 Sep 2017 03:12:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 57E9328F11 for ; Wed, 27 Sep 2017 03:12:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4C1DC28F46; Wed, 27 Sep 2017 03:12:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C2B528F11 for ; Wed, 27 Sep 2017 03:12:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031622AbdI0DMw (ORCPT ); Tue, 26 Sep 2017 23:12:52 -0400 Received: from esa5.hgst.iphmx.com ([216.71.153.144]:42481 "EHLO esa5.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031618AbdI0DMu (ORCPT ); Tue, 26 Sep 2017 23:12:50 -0400 X-IronPort-AV: E=Sophos;i="5.42,442,1500912000"; d="scan'208";a="54599958" Received: from mail-by2nam01lp0183.outbound.protection.outlook.com (HELO NAM01-BY2-obe.outbound.protection.outlook.com) ([216.32.181.183]) by ob1.hgst.iphmx.com with ESMTP; 27 Sep 2017 11:12:49 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector1-wdc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=pScyB41jyiII4udxaDFMTHwvAMH3qVE0GAt98/ngjOM=; b=EGnML/xcT4gXlivANJ3DwITz4OB+HL5FC3za+LQqPMwrQLBX5/9nWdxgTVa9hYgN+2sTEADW740XGni7Fu2oV9uiz2MPQHNLFBS4VedpdGcphCn8OjJPulax8uU6AJE3kDUukHfd3osLJH46F+SLYwlKNwU1yj8TbleLP/terFM= Received: from CY1PR0401MB1536.namprd04.prod.outlook.com (10.163.19.154) by CY1PR0401MB1535.namprd04.prod.outlook.com (10.163.19.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Wed, 27 Sep 2017 03:12:47 +0000 Received: from CY1PR0401MB1536.namprd04.prod.outlook.com ([10.163.19.154]) by CY1PR0401MB1536.namprd04.prod.outlook.com ([10.163.19.154]) with mapi id 15.20.0077.011; Wed, 27 Sep 2017 03:12:47 +0000 From: Bart Van Assche To: "ming.lei@redhat.com" CC: "linux-block@vger.kernel.org" , "jthumshirn@suse.de" , "linux-raid@vger.kernel.org" , "hch@lst.de" , "martin.petersen@oracle.com" , "axboe@kernel.dk" , "oleksandr@natalenko.name" , "hare@suse.com" , "shli@kernel.org" Subject: Re: [PATCH v4 1/7] md: Make md resync and reshape threads freezable Thread-Topic: [PATCH v4 1/7] md: Make md resync and reshape threads freezable Thread-Index: AQHTNj0L6OhPutAcAUGjdBMzsA5U76LHBbcAgAA5DgCAAATRAIAAzK6A Date: Wed, 27 Sep 2017 03:12:47 +0000 Message-ID: <1506481915.2822.9.camel@wdc.com> References: <20170925202924.16603-1-bart.vanassche@wdc.com> <20170925202924.16603-2-bart.vanassche@wdc.com> <20170926111747.GA16794@ming.t460p> <1506436926.3787.24.camel@wdc.com> <20170926145919.GC31449@ming.t460p> In-Reply-To: <20170926145919.GC31449@ming.t460p> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [104.153.224.167] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CY1PR0401MB1535; 20:Jb3pQuLV1SeigNXk+qmNwtZ9h5eouBrMYUJkK/xKiYr18nvSrGQU5ggGz2/rFale/daexQfrGf6aJ0LpY6r6ilmreSZZOvJZy3vCCbPx5caIpi0YMfCrylQjlS3GjPoXDwVCo5wGEQ2sVifKrsVYMKat8k1fQwitFCZt5KxpMys= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 447d3aca-9cf6-4f77-7600-08d50555a11c x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:CY1PR0401MB1535; x-ms-traffictypediagnostic: CY1PR0401MB1535: wdcipoutbound: EOP-TRUE x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(100000703101)(100105400095)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123564025)(20161123562025)(20161123555025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR0401MB1535; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR0401MB1535; x-forefront-prvs: 04433051BF x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39860400002)(346002)(376002)(199003)(377424004)(51914003)(24454002)(189002)(66066001)(81166006)(6436002)(86362001)(6246003)(478600001)(68736007)(2900100001)(229853002)(8676002)(3280700002)(4326008)(81156014)(33646002)(189998001)(72206003)(101416001)(54906003)(103116003)(316002)(93886005)(5640700003)(77096006)(6486002)(6916009)(97736004)(2950100002)(5660300001)(36756003)(2501003)(2351001)(76176999)(99286003)(2906002)(106356001)(53936002)(6506006)(14454004)(105586002)(50986999)(6512007)(305945005)(25786009)(54356999)(8936002)(3660700001)(6116002)(3846002)(102836003)(7736002)(7416002); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR0401MB1535; H:CY1PR0401MB1536.namprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2017 03:12:47.5211 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB1535 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Tue, 2017-09-26 at 22:59 +0800, Ming Lei wrote: > On Tue, Sep 26, 2017 at 02:42:07PM +0000, Bart Van Assche wrote: > > On Tue, 2017-09-26 at 19:17 +0800, Ming Lei wrote: > > > Just test this patch a bit and the following failure of freezing task > > > is triggered during suspend: [ ... ] > > > > What kernel version did you start from and which patches were applied on top of > > that kernel? Only patch 1/7 or all seven patches? What storage configuration did > > It is v4.14-rc1+, and top commit is 8d93c7a43157, with all your 7 patches > applied. > > > you use in your test and what command(s) did you use to trigger suspend? > > Follows my pm test script: > > #!/bin/sh > > echo check > /sys/block/md127/md/sync_action > > mkfs.ext4 -F /dev/md127 > > mount /dev/md0 /mnt/data > > dd if=/dev/zero of=/mnt/data/d1.img bs=4k count=128k& > > echo 9 > /proc/sys/kernel/printk > echo devices > /sys/power/pm_test > echo mem > /sys/power/state > > wait > umount /mnt/data > > Storage setting: > > sudo mdadm --create /dev/md/test /dev/sda /dev/sdb --level=1 --raid-devices=2 > both /dev/sda and /dev/sdb are virtio-scsi. Thanks for the detailed reply. I have been able to reproduce the freeze failure you reported. The output of SysRq-t learned me that the md reboot notifier was waiting for the frozen md sync thread and that this caused the freeze failure. So I have started testing the patch below instead of the patch at the start of this e-mail thread: Subject: [PATCH] md: Stop resync and reshape upon system freeze Some people use the md driver on laptops and use the suspend and resume functionality. Since it is essential that submitting of new I/O requests stops before a hibernation image is created, interrupt the md resync and reshape actions if the system is being frozen. Note: the resync and reshape will restart after the system is resumed and a message similar to the following will appear in the system log: md: md0: data-check interrupted. --- drivers/md/md.c | 34 ++++++++++++++++++++-------------- 1 file changed, 20 insertions(+), 14 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 08fcaebc61bd..1e9d50f7345e 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -66,6 +66,7 @@ #include #include #include +#include #include #include "md.h" @@ -8103,6 +8104,12 @@ void md_allow_write(struct mddev *mddev) } EXPORT_SYMBOL_GPL(md_allow_write); +static bool md_sync_interrupted(struct mddev *mddev) +{ + return test_bit(MD_RECOVERY_INTR, &mddev->recovery) || + freezing(current); +} + #define SYNC_MARKS 10 #define SYNC_MARK_STEP (3*HZ) #define UPDATE_FREQUENCY (5*60*HZ) @@ -8133,6 +8140,8 @@ void md_do_sync(struct md_thread *thread) return; } + set_freezable(); + if (mddev_is_clustered(mddev)) { ret = md_cluster_ops->resync_start(mddev); if (ret) @@ -8184,7 +8193,7 @@ void md_do_sync(struct md_thread *thread) mddev->curr_resync = 2; try_again: - if (test_bit(MD_RECOVERY_INTR, &mddev->recovery)) + if (md_sync_interrupted(mddev)) goto skip; for_each_mddev(mddev2, tmp) { if (mddev2 == mddev) @@ -8208,7 +8217,7 @@ void md_do_sync(struct md_thread *thread) * be caught by 'softlockup' */ prepare_to_wait(&resync_wait, &wq, TASK_INTERRUPTIBLE); - if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery) && + if (!md_sync_interrupted(mddev) && mddev2->curr_resync >= mddev->curr_resync) { if (mddev2_minor != mddev2->md_minor) { mddev2_minor = mddev2->md_minor; @@ -8335,8 +8344,7 @@ void md_do_sync(struct md_thread *thread) sysfs_notify(&mddev->kobj, NULL, "sync_completed"); } - while (j >= mddev->resync_max && - !test_bit(MD_RECOVERY_INTR, &mddev->recovery)) { + while (j >= mddev->resync_max && !md_sync_interrupted(mddev)) { /* As this condition is controlled by user-space, * we can block indefinitely, so use '_interruptible' * to avoid triggering warnings. @@ -8348,7 +8356,7 @@ void md_do_sync(struct md_thread *thread) &mddev->recovery)); } - if (test_bit(MD_RECOVERY_INTR, &mddev->recovery)) + if (md_sync_interrupted(mddev)) break; sectors = mddev->pers->sync_request(mddev, j, &skipped); @@ -8362,7 +8370,7 @@ void md_do_sync(struct md_thread *thread) atomic_add(sectors, &mddev->recovery_active); } - if (test_bit(MD_RECOVERY_INTR, &mddev->recovery)) + if (md_sync_interrupted(mddev)) break; j += sectors; @@ -8394,7 +8402,7 @@ void md_do_sync(struct md_thread *thread) last_mark = next; } - if (test_bit(MD_RECOVERY_INTR, &mddev->recovery)) + if (md_sync_interrupted(mddev)) break; /* @@ -8427,8 +8435,7 @@ void md_do_sync(struct md_thread *thread) } } pr_info("md: %s: %s %s.\n",mdname(mddev), desc, - test_bit(MD_RECOVERY_INTR, &mddev->recovery) - ? "interrupted" : "done"); + md_sync_interrupted(mddev) ? "interrupted" : "done"); /* * this also signals 'finished resyncing' to md_stop */ @@ -8436,8 +8443,7 @@ void md_do_sync(struct md_thread *thread) wait_event(mddev->recovery_wait, !atomic_read(&mddev->recovery_active)); if (!test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) && - !test_bit(MD_RECOVERY_INTR, &mddev->recovery) && - mddev->curr_resync > 3) { + !md_sync_interrupted(mddev) && mddev->curr_resync > 3) { mddev->curr_resync_completed = mddev->curr_resync; sysfs_notify(&mddev->kobj, NULL, "sync_completed"); } @@ -8446,7 +8452,7 @@ void md_do_sync(struct md_thread *thread) if (!test_bit(MD_RECOVERY_CHECK, &mddev->recovery) && mddev->curr_resync > 3) { if (test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) { - if (test_bit(MD_RECOVERY_INTR, &mddev->recovery)) { + if (md_sync_interrupted(mddev)) { if (mddev->curr_resync >= mddev->recovery_cp) { pr_debug("md: checkpointing %s of %s.\n", desc, mdname(mddev)); @@ -8461,7 +8467,7 @@ void md_do_sync(struct md_thread *thread) } else mddev->recovery_cp = MaxSector; } else { - if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery)) + if (!md_sync_interrupted(mddev)) mddev->curr_resync = MaxSector; rcu_read_lock(); rdev_for_each_rcu(rdev, mddev) @@ -8483,7 +8489,7 @@ void md_do_sync(struct md_thread *thread) BIT(MD_SB_CHANGE_PENDING) | BIT(MD_SB_CHANGE_DEVS)); spin_lock(&mddev->lock); - if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery)) { + if (!md_sync_interrupted(mddev)) { /* We completed so min/max setting can be forgotten if used. */ if (test_bit(MD_RECOVERY_REQUESTED, &mddev->recovery)) mddev->resync_min = 0;