From patchwork Fri Nov 3 04:21:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: piaojun X-Patchwork-Id: 10039501 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D105D602DA for ; Fri, 3 Nov 2017 04:23:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B6871292A4 for ; Fri, 3 Nov 2017 04:23:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A9BBA2945C; Fri, 3 Nov 2017 04:23:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 79D7A292A4 for ; Fri, 3 Nov 2017 04:23:27 +0000 (UTC) Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id vA34NCaC006788 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 3 Nov 2017 04:23:12 GMT Received: from oss.oracle.com (oss-old-reserved.oracle.com [137.254.22.2]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id vA34N7Qk009223 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 3 Nov 2017 04:23:08 GMT Received: from localhost ([127.0.0.1] helo=lb-oss.oracle.com) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1eATVb-0005ZL-Pf; Thu, 02 Nov 2017 21:23:07 -0700 Received: from userv0021.oracle.com ([156.151.31.71]) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1eATVX-0005Z8-Mo for ocfs2-devel@oss.oracle.com; Thu, 02 Nov 2017 21:23:05 -0700 Received: from userp2040.oracle.com (userp2040.oracle.com [156.151.31.90]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id vA34N3Qt009006 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 3 Nov 2017 04:23:03 GMT Received: from pps.filterd (userp2040.oracle.com [127.0.0.1]) by userp2040.oracle.com (8.16.0.21/8.16.0.21) with SMTP id vA34LmMX035568 for ; Fri, 3 Nov 2017 04:23:03 GMT Authentication-Results: oracle.com; spf=pass smtp.mailfrom=piaojun@huawei.com Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by userp2040.oracle.com with ESMTP id 2e0c71ra1d-1 (version=TLSv1 cipher=RC4-SHA bits=128 verify=NOT) for ; Fri, 03 Nov 2017 04:23:02 +0000 Received: from 172.30.72.59 (EHLO DGGEMS411-HUB.china.huawei.com) ([172.30.72.59]) by dggrg05-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DKH95043; Fri, 03 Nov 2017 12:22:15 +0800 (CST) Received: from [10.177.253.249] (10.177.253.249) by smtp.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.361.1; Fri, 3 Nov 2017 12:21:13 +0800 From: piaojun To: "akpm@linux-foundation.org" , Mark Fasheh , Joel Becker , "srinivas.eeda@oracle.com" Message-ID: <59FBEEB5.3040605@huawei.com> Date: Fri, 3 Nov 2017 12:21:09 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 X-Originating-IP: [10.177.253.249] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A090202.59FBEEF7.00D5, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: bb7c4fcd32e3a3178d5a1bf20597e95b X-CLX-Shades: MLX X-CLX-Response: 1TFkXGxMaEQpMehcdGhEKWU0XZ2ZyEQpZSRcHGx4dcR4GBx0ZdwYYGhoGGRp CGgYaBhkdBh0acR4QGncGGgYaBhoGGgYaBhpxGhAadwYaEQpZXhdobnkRCklGF0VYS0lGT3VaWE VOX0leQ0VEGXVPSxEKQ04XXXJtbG1kTWR1ckwfSX5baF5MY20TEkNhTHVHZl1HB3MRClhcFx8EG gQbGxMHG0gaThhOS08FGxoEGxoaBB4SBBsQGx4aHxoRCl5ZF3hncHpoEQpNXBcdHhEKTFoXaGlC a3kRCk1OF2gRCkNaFx4fBBgeEwQYGxgEGxMbEQpCXhcbEQpEXhccEQpESRcfEQpCRhdgU24eQxl zZ1hGZhEKQlwXGhEKQkUXZlxse3BkYnoSfEMRCkJOF2xCSFlTGk1leHgdEQpCTBdvSxkSEkRZeV sbXxEKQmwXYwVCUmZAYl5ae1IRCkJAF2hIcHxJe359HmtBEQpCWBdifW95AU8YGXBwexEKWlgXG xEKcGcXb2EcWWdSH3oSbBgQGRoRCnBoF2Via1leaxxPBUhAEBoRCnBoF2xQWm4SAX8cfxNjEBoR CnBoF3obYmZGbmBsAUFvEBoRCnBoF254fHpBBURkGHkYEBoRCnBoF25LWktdZElnWRxTEBoRCnB sF2FJeUN6c0l4Z2xiEBoRCm1+FxoRClhNF0sRIA== X-PDR: PASS X-ServerName: szxga05-in.huawei.com X-Proofpoint-SPF-Result: pass X-Proofpoint-SPF-Record: v=spf1 ip4:45.249.212.32 ip4:45.249.212.35 ip4:119.145.14.93 ip4:58.251.152.93 ip4:194.213.3.17 ip4:206.16.17.72 ip4:45.249.212.255 ip4:45.249.212.187/29 ip4:45.249.212.191 ~all X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8703 signatures=668611 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=70 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=190 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1711030051 Cc: "ocfs2-devel@oss.oracle.com" Subject: [Ocfs2-devel] [PATCH v2] ocfs2/dlm: wait for dlm recovery done when migrating all lockres X-BeenThere: ocfs2-devel@oss.oracle.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ocfs2-devel-bounces@oss.oracle.com Errors-To: ocfs2-devel-bounces@oss.oracle.com X-Source-IP: userv0021.oracle.com [156.151.31.71] X-Virus-Scanned: ClamAV using ClamSMTP wait for dlm recovery done when migrating all lockres in case of new lockres to be left after leaving dlm domain. NodeA NodeB NodeC umount and migrate all lockres node down do recovery for NodeB and collect a new lockres form other live nodes leave domain but the new lockres remains mount and join domain request for the owner of the new lockres, but all the other nodes said 'NO', so NodeC decide to the owner, and send do assert msg to other nodes. other nodes receive the msg and found two masters exist. at last cause BUG in dlm_assert_master_handler() -->BUG(); Fixes: bc9838c4d44a ("dlm: allow dlm do recovery during shutdown") Signed-off-by: Jun Piao Reviewed-by: Alex Chen Reviewed-by: Yiwen Jiang Acked-by: Changwei Ge --- fs/ocfs2/dlm/dlmcommon.h | 1 + fs/ocfs2/dlm/dlmdomain.c | 14 ++++++++++++++ fs/ocfs2/dlm/dlmrecovery.c | 13 ++++++++++--- 3 files changed, 25 insertions(+), 3 deletions(-) diff --git a/fs/ocfs2/dlm/dlmcommon.h b/fs/ocfs2/dlm/dlmcommon.h index e9f3705..999ab7d 100644 --- a/fs/ocfs2/dlm/dlmcommon.h +++ b/fs/ocfs2/dlm/dlmcommon.h @@ -140,6 +140,7 @@ struct dlm_ctxt u8 node_num; u32 key; u8 joining_node; + u8 migrate_done; /* set to 1 means node has migrated all lockres */ wait_queue_head_t dlm_join_events; unsigned long live_nodes_map[BITS_TO_LONGS(O2NM_MAX_NODES)]; unsigned long domain_map[BITS_TO_LONGS(O2NM_MAX_NODES)]; diff --git a/fs/ocfs2/dlm/dlmdomain.c b/fs/ocfs2/dlm/dlmdomain.c index e1fea14..98a8f56 100644 --- a/fs/ocfs2/dlm/dlmdomain.c +++ b/fs/ocfs2/dlm/dlmdomain.c @@ -461,6 +461,18 @@ static int dlm_migrate_all_locks(struct dlm_ctxt *dlm) cond_resched_lock(&dlm->spinlock); num += n; } + + if (!num) { + if (dlm->reco.state & DLM_RECO_STATE_ACTIVE) { + mlog(0, "%s: perhaps there are more lock resources need to " + "be migrated after dlm recovery\n", dlm->name); + ret = -EAGAIN; + } else { + mlog(0, "%s: we won't do dlm recovery after migrating all lockres", + dlm->name); + dlm->migrate_done = 1; + } + } spin_unlock(&dlm->spinlock); wake_up(&dlm->dlm_thread_wq); @@ -2052,6 +2064,8 @@ static struct dlm_ctxt *dlm_alloc_ctxt(const char *domain, dlm->joining_node = DLM_LOCK_RES_OWNER_UNKNOWN; init_waitqueue_head(&dlm->dlm_join_events); + dlm->migrate_done = 0; + dlm->reco.new_master = O2NM_INVALID_NODE_NUM; dlm->reco.dead_node = O2NM_INVALID_NODE_NUM; diff --git a/fs/ocfs2/dlm/dlmrecovery.c b/fs/ocfs2/dlm/dlmrecovery.c index 74407c6..c4cf682 100644 --- a/fs/ocfs2/dlm/dlmrecovery.c +++ b/fs/ocfs2/dlm/dlmrecovery.c @@ -423,12 +423,11 @@ void dlm_wait_for_recovery(struct dlm_ctxt *dlm) static void dlm_begin_recovery(struct dlm_ctxt *dlm) { - spin_lock(&dlm->spinlock); + assert_spin_locked(&dlm->spinlock); BUG_ON(dlm->reco.state & DLM_RECO_STATE_ACTIVE); printk(KERN_NOTICE "o2dlm: Begin recovery on domain %s for node %u\n", dlm->name, dlm->reco.dead_node); dlm->reco.state |= DLM_RECO_STATE_ACTIVE; - spin_unlock(&dlm->spinlock); } static void dlm_end_recovery(struct dlm_ctxt *dlm) @@ -456,6 +455,13 @@ static int dlm_do_recovery(struct dlm_ctxt *dlm) spin_lock(&dlm->spinlock); + if (dlm->migrate_done) { + mlog(0, "%s: no need do recovery after migrating all lockres\n", + dlm->name); + spin_unlock(&dlm->spinlock); + return 0; + } + /* check to see if the new master has died */ if (dlm->reco.new_master != O2NM_INVALID_NODE_NUM && test_bit(dlm->reco.new_master, dlm->recovery_map)) { @@ -490,12 +496,13 @@ static int dlm_do_recovery(struct dlm_ctxt *dlm) mlog(0, "%s(%d):recovery thread found node %u in the recovery map!\n", dlm->name, task_pid_nr(dlm->dlm_reco_thread_task), dlm->reco.dead_node); - spin_unlock(&dlm->spinlock); /* take write barrier */ /* (stops the list reshuffling thread, proxy ast handling) */ dlm_begin_recovery(dlm); + spin_unlock(&dlm->spinlock); + if (dlm->reco.new_master == dlm->node_num) goto master_here;