From patchwork Tue Nov 12 14:08:22 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Goldwyn Rodrigues X-Patchwork-Id: 3172771 Return-Path: X-Original-To: patchwork-ocfs2-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 87ACA9F68F for ; Tue, 12 Nov 2013 14:08:49 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 29AB9204CF for ; Tue, 12 Nov 2013 14:08:48 +0000 (UTC) Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2E3B82047B for ; Tue, 12 Nov 2013 14:08:43 +0000 (UTC) Received: from acsinet21.oracle.com (acsinet21.oracle.com [141.146.126.237]) by userp1040.oracle.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.1) with ESMTP id rACE8YiH007773 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 12 Nov 2013 14:08:35 GMT Received: from oss.oracle.com (oss-external.oracle.com [137.254.96.51]) by acsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id rACE8XvN007489 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 12 Nov 2013 14:08:34 GMT Received: from localhost ([127.0.0.1] helo=oss.oracle.com) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1VgEdt-0001PZ-Mc; Tue, 12 Nov 2013 06:08:33 -0800 Received: from acsinet22.oracle.com ([141.146.126.238]) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1VgEdl-0001PD-DS for ocfs2-devel@oss.oracle.com; Tue, 12 Nov 2013 06:08:25 -0800 Received: from userp1020.oracle.com (userp1020.oracle.com [156.151.31.79]) by acsinet22.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id rACE8Oks020017 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 12 Nov 2013 14:08:25 GMT Received: from mx2.suse.de (cantor2.suse.de [195.135.220.15]) by userp1020.oracle.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.1) with ESMTP id rACE8NgR002168 for ; Tue, 12 Nov 2013 14:08:24 GMT Received: from relay1.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 4B657A71A2; Tue, 12 Nov 2013 15:08:23 +0100 (CET) Date: Tue, 12 Nov 2013 08:08:22 -0600 From: Goldwyn Rodrigues To: ocfs2-devel@oss.oracle.com Message-ID: <20131112140817.GA3955@shrek.lan> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Flow-Control-Info: class=Pass-to-MM reputation=ipRisk-All ip=195.135.220.15 ct-class=T1 ct-vol1=0 ct-vol2=4 ct-vol3=4 ct-risk=36 ct-spam1=0 ct-spam2=47 ct-bulk=36 rcpts=1 size=7035 X-SPF-Info: NONE::cantor2.suse.de X-Sendmail-CM-Score: 0.00% X-Sendmail-CM-Analysis: v=2.1 cv=R4w6R7hX c=1 sm=1 tr=0 a=uEuDQZVrWKuLCe7byFjfVg==:117 a=uEuDQZVrWKuLCe7byFjfVg==:17 a=LcaDllckn3IA:10 a=_Ek0adKVqJ8A:10 a=7SFtLa4N3QQA:10 a=kj9zAlcOel0A:10 a=1xl6szHlWSYA:10 a=yPCof4ZbAAAA:8 a=iox4zFpeAAAA:8 a=-hFPRZmgrtxIgggP6i IA:9 a=CjuIK1q_8ugA:10 a=7DSvI1NPTFQA:10 a=n9GBPR9yFnkA:10 X-Sendmail-CT-Classification: not spam X-Sendmail-CT-RefID: str=0001.0A090208.52823658.007C:SCFSTAT13898897, ss=1, re=-4.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 Cc: mfasheh@suse.com, dan.carpenter@oracle.com Subject: [Ocfs2-devel] [PATCH 6/6] Use the new DLM operation callbacks while requesting new lockspace X-BeenThere: ocfs2-devel@oss.oracle.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ocfs2-devel-bounces@oss.oracle.com Errors-To: ocfs2-devel-bounces@oss.oracle.com X-Source-IP: acsinet21.oracle.com [141.146.126.237] X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Attempt to use the new DLM operations. If it is not supported, use the traditional ocfs2_controld. To exchange ocfs2 versioning, we use the LVB of the version dlm lock. It first attempts to take the lock in EX mode (non-blocking). If successful (which means it is the first mount), it writes the version number and downconverts to PR lock. If it is unsuccessful, it reads the version from the lock. If this becomes the standard (with o2cb as well), it could simplify userspace tools to check if the filesystem is mounted on other nodes. Dan: Since ocfs2_protocol_version are two u8 values, the additional checks with LONG* don't make sense. Signed-off-by: Dan Carpenter Signed-off-by: Goldwyn Rodrigues --- fs/ocfs2/stack_user.c | 119 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 95 insertions(+), 24 deletions(-) diff --git a/fs/ocfs2/stack_user.c b/fs/ocfs2/stack_user.c index d33dd05..ec6f406 100644 --- a/fs/ocfs2/stack_user.c +++ b/fs/ocfs2/stack_user.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include "stackglue.h" @@ -122,6 +123,7 @@ struct ocfs2_live_connection { struct dlm_lksb oc_version_lksb; char oc_lvb[DLM_LVB_LEN]; struct completion oc_sync_wait; + wait_queue_head_t oc_wait; }; struct ocfs2_control_private { @@ -218,7 +220,7 @@ static int ocfs2_live_connection_attach(struct ocfs2_cluster_connection *conn, mutex_lock(&ocfs2_control_lock); c->oc_conn = conn; - if (atomic_read(&ocfs2_control_opened)) + if ((c->oc_type == NO_CONTROLD) || atomic_read(&ocfs2_control_opened)) list_add(&c->oc_list, &ocfs2_live_connection_list); else { printk(KERN_ERR @@ -897,6 +899,48 @@ static int version_unlock(struct ocfs2_cluster_connection *conn) return sync_unlock(conn, &lc->oc_version_lksb, VERSION_LOCK); } +static int get_protocol_version(struct ocfs2_cluster_connection *conn) +{ + int ret; + struct ocfs2_live_connection *lc = conn->cc_private; + struct ocfs2_protocol_version pv; + + running_proto.pv_major = + ocfs2_user_plugin.sp_max_proto.pv_major; + running_proto.pv_minor = + ocfs2_user_plugin.sp_max_proto.pv_minor; + + lc->oc_version_lksb.sb_lvbptr = lc->oc_lvb; + ret = version_lock(conn, DLM_LOCK_EX, + DLM_LKF_VALBLK|DLM_LKF_NOQUEUE); + if (!ret) { + conn->cc_version.pv_major = running_proto.pv_major; + conn->cc_version.pv_minor = running_proto.pv_minor; + version_to_lvb(&running_proto, lc->oc_lvb); + version_lock(conn, DLM_LOCK_PR, DLM_LKF_CONVERT|DLM_LKF_VALBLK); + } else if (ret == -EAGAIN) { + ret = version_lock(conn, DLM_LOCK_PR, DLM_LKF_VALBLK); + if (ret) + goto out; + lvb_to_version(lc->oc_lvb, &pv); + + if (pv.pv_minor < 0) { + ret = -ERANGE; + goto out; + } + if ((pv.pv_major != running_proto.pv_major) || + (pv.pv_minor > running_proto.pv_minor)) { + ret = -EINVAL; + goto out; + } + + conn->cc_version.pv_major = pv.pv_major; + conn->cc_version.pv_minor = pv.pv_minor; + } +out: + return ret; +} + static void user_recover_prep(void *arg) { } @@ -925,6 +969,7 @@ static void user_recover_done(void *arg, struct dlm_slot *slots, } lc->oc_our_slot = our_slot; + wake_up(&lc->oc_wait); } const struct dlm_lockspace_ops ocfs2_ls_ops = { @@ -933,11 +978,21 @@ const struct dlm_lockspace_ops ocfs2_ls_ops = { .recover_done = user_recover_done, }; +static int user_cluster_disconnect(struct ocfs2_cluster_connection *conn) +{ + version_unlock(conn); + dlm_release_lockspace(conn->cc_lockspace, 2); + conn->cc_lockspace = NULL; + ocfs2_live_connection_drop(conn->cc_private); + conn->cc_private = NULL; + return 0; +} + static int user_cluster_connect(struct ocfs2_cluster_connection *conn) { dlm_lockspace_t *fsdlm; struct ocfs2_live_connection *lc; - int rc; + int rc, ops_rv; BUG_ON(conn == NULL); @@ -947,11 +1002,44 @@ static int user_cluster_connect(struct ocfs2_cluster_connection *conn) goto out; } - lc->oc_type = WITH_CONTROLD; + init_waitqueue_head(&lc->oc_wait); + init_completion(&lc->oc_sync_wait); + atomic_set(&lc->oc_this_node, 0); + conn->cc_private = lc; + lc->oc_type = NO_CONTROLD; + + rc = dlm_new_lockspace(conn->cc_name, conn->cc_cluster_name, + DLM_LSFL_FS, DLM_LVB_LEN, + &ocfs2_ls_ops, conn, &ops_rv, &fsdlm); + if (rc) + goto out; + + if (ops_rv == -EOPNOTSUPP) { + lc->oc_type = WITH_CONTROLD; + printk(KERN_NOTICE "ocfs2: You seem to be using an older " + "version of dlm_controld and/or ocfs2-tools." + " Please consider upgrading.\n"); + } else if (ops_rv) { + rc = ops_rv; + goto out; + } + conn->cc_lockspace = fsdlm; + rc = ocfs2_live_connection_attach(conn, lc); if (rc) goto out; + if (lc->oc_type == NO_CONTROLD) { + rc = get_protocol_version(conn); + if (rc) { + printk(KERN_ERR "ocfs2: Could not determine" + " locking version\n"); + user_cluster_disconnect(conn); + goto out; + } + wait_event(lc->oc_wait, (atomic_read(&lc->oc_this_node) > 0)); + } + /* * running_proto must have been set before we allowed any mounts * to proceed. @@ -959,40 +1047,20 @@ static int user_cluster_connect(struct ocfs2_cluster_connection *conn) if (fs_protocol_compare(&running_proto, &conn->cc_version)) { printk(KERN_ERR "Unable to mount with fs locking protocol version " - "%u.%u because the userspace control daemon has " - "negotiated %u.%u\n", + "%u.%u because negotiated protocol is %u.%u\n", conn->cc_version.pv_major, conn->cc_version.pv_minor, running_proto.pv_major, running_proto.pv_minor); rc = -EPROTO; ocfs2_live_connection_drop(lc); lc = NULL; - goto out; } - rc = dlm_new_lockspace(conn->cc_name, NULL, DLM_LSFL_FS, DLM_LVB_LEN, - NULL, NULL, NULL, &fsdlm); - if (rc) { - ocfs2_live_connection_drop(lc); - lc = NULL; - goto out; - } - - conn->cc_private = lc; - conn->cc_lockspace = fsdlm; out: if (rc && lc) kfree(lc); return rc; } -static int user_cluster_disconnect(struct ocfs2_cluster_connection *conn) -{ - dlm_release_lockspace(conn->cc_lockspace, 2); - conn->cc_lockspace = NULL; - ocfs2_live_connection_drop(conn->cc_private); - conn->cc_private = NULL; - return 0; -} static int user_cluster_this_node(struct ocfs2_cluster_connection *conn, unsigned int *this_node) @@ -1002,8 +1070,11 @@ static int user_cluster_this_node(struct ocfs2_cluster_connection *conn, if (lc->oc_type == WITH_CONTROLD) rc = ocfs2_control_get_this_node(); + else if (lc->oc_type == NO_CONTROLD) + rc = atomic_read(&lc->oc_this_node); else rc = -EINVAL; + if (rc < 0) return rc;