From patchwork Thu Nov 1 09:03:36 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zheng" X-Patchwork-Id: 1683601 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id ACABDDF264 for ; Thu, 1 Nov 2012 09:03:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756658Ab2KAJDo (ORCPT ); Thu, 1 Nov 2012 05:03:44 -0400 Received: from mga01.intel.com ([192.55.52.88]:13445 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756700Ab2KAJDm (ORCPT ); Thu, 1 Nov 2012 05:03:42 -0400 Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP; 01 Nov 2012 02:03:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.80,692,1344236400"; d="scan'208";a="241265471" Received: from zyan5-mobl.sh.intel.com ([10.239.36.138]) by fmsmga001.fm.intel.com with ESMTP; 01 Nov 2012 02:03:40 -0700 From: "Yan, Zheng" To: sage@inktank.com, ceph-devel@vger.kernel.org Cc: "Yan, Zheng" Subject: [PATCH 1/2] mds: Don't acquire replica object's versionlock Date: Thu, 1 Nov 2012 17:03:36 +0800 Message-Id: <1351760618-19874-2-git-send-email-zheng.z.yan@intel.com> X-Mailer: git-send-email 1.7.11.7 In-Reply-To: <1351760618-19874-1-git-send-email-zheng.z.yan@intel.com> References: <1351760618-19874-1-git-send-email-zheng.z.yan@intel.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org From: "Yan, Zheng" Both CInode and CDentry's versionlocks are of type LocalLock. Acquiring LocalLock in replica object is useless and problematic. For example, if two requests try acquiring a replica object's versionlock, the first request succeeds, the second request is added to wait queue. Later when the first request finishes, MDCache::request_drop_foreign_locks() finds the lock's parent is non-auth, it skips waking requests in the wait queue. So the second request hangs. Signed-off-by: Yan, Zheng --- src/mds/Locker.cc | 6 ++++++ src/mds/Server.cc | 25 ++++++++++--------------- 3 files changed, 17 insertions(+), 16 deletions(-) diff --git a/src/mds/Locker.cc b/src/mds/Locker.cc index e033bbe..6474743 100644 --- a/src/mds/Locker.cc +++ b/src/mds/Locker.cc @@ -196,6 +196,7 @@ bool Locker::acquire_locks(MDRequest *mdr, // augment xlock with a versionlock? if ((*p)->get_type() == CEPH_LOCK_DN) { CDentry *dn = (CDentry*)(*p)->get_parent(); + assert(dn->is_auth()); if (xlocks.count(&dn->versionlock)) continue; // we're xlocking the versionlock too; don't wrlock it! @@ -213,6 +214,8 @@ bool Locker::acquire_locks(MDRequest *mdr, if ((*p)->get_type() > CEPH_LOCK_IVERSION) { // inode version lock? CInode *in = (CInode*)(*p)->get_parent(); + if (!in->is_auth()) + continue; if (mdr->is_master()) { // master. wrlock versionlock so we can pipeline inode updates to journal. wrlocks.insert(&in->versionlock); @@ -3899,6 +3902,7 @@ void Locker::local_wrlock_grab(LocalLock *lock, Mutation *mut) dout(7) << "local_wrlock_grab on " << *lock << " on " << *lock->get_parent() << dendl; + assert(lock->get_parent()->is_auth()); assert(lock->can_wrlock()); assert(!mut->wrlocks.count(lock)); lock->get_wrlock(mut->get_client()); @@ -3911,6 +3915,7 @@ bool Locker::local_wrlock_start(LocalLock *lock, MDRequest *mut) dout(7) << "local_wrlock_start on " << *lock << " on " << *lock->get_parent() << dendl; + assert(lock->get_parent()->is_auth()); if (lock->can_wrlock()) { assert(!mut->wrlocks.count(lock)); lock->get_wrlock(mut->get_client()); @@ -3942,6 +3947,7 @@ bool Locker::local_xlock_start(LocalLock *lock, MDRequest *mut) dout(7) << "local_xlock_start on " << *lock << " on " << *lock->get_parent() << dendl; + assert(lock->get_parent()->is_auth()); if (!lock->can_xlock_local()) { lock->add_waiter(SimpleLock::WAIT_WR|SimpleLock::WAIT_STABLE, new C_MDS_RetryRequest(mdcache, mut)); return false; diff --git a/src/mds/Server.cc b/src/mds/Server.cc index 4642a13..45c890a 100644 --- a/src/mds/Server.cc +++ b/src/mds/Server.cc @@ -5204,25 +5204,20 @@ void Server::handle_client_rename(MDRequest *mdr) wrlocks.insert(&straydn->get_dir()->inode->nestlock); } - // xlock versionlock on srci if remote? - // this ensures it gets safely remotely auth_pinned, avoiding deadlock; - // strictly speaking, having the slave node freeze the inode is - // otherwise sufficient for avoiding conflicts with inode locks, etc. - if (!srcdn->is_auth() && srcdnl->is_primary()) // xlock versionlock on srci if there are any witnesses - xlocks.insert(&srci->versionlock); - // xlock versionlock on dentries if there are witnesses. // replicas can't see projected dentry linkages, and will get // confused if we try to pipeline things. if (!witnesses.empty()) { - if (srcdn->is_projected()) - xlocks.insert(&srcdn->versionlock); - if (destdn->is_projected()) - xlocks.insert(&destdn->versionlock); - // also take rdlock on all ancestor dentries for destdn. this ensures that the - // destdn can be traversed to by the witnesses. - for (int i=0; i<(int)desttrace.size(); i++) - xlocks.insert(&desttrace[i]->versionlock); + // take xlock on all projected dentries for srcdn and destdn. this ensures + // that the srcdn and destdn can be traversed to by the witnesses. + for (int i= 0; i<(int)srctrace.size(); i++) { + if (srctrace[i]->is_auth() && srctrace[i]->is_projected()) + xlocks.insert(&srctrace[i]->versionlock); + } + for (int i=0; i<(int)desttrace.size(); i++) { + if (desttrace[i]->is_auth() && desttrace[i]->is_projected()) + xlocks.insert(&desttrace[i]->versionlock); + } } // we need to update srci's ctime. xlock its least contended lock to do that...