From patchwork Tue Jan 20 12:41:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 5669031 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 5753AC058D for ; Tue, 20 Jan 2015 12:41:59 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 76FB720306 for ; Tue, 20 Jan 2015 12:41:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 76B9220131 for ; Tue, 20 Jan 2015 12:41:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754204AbbATMlz (ORCPT ); Tue, 20 Jan 2015 07:41:55 -0500 Received: from mail-wi0-f176.google.com ([209.85.212.176]:53976 "EHLO mail-wi0-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751241AbbATMlr (ORCPT ); Tue, 20 Jan 2015 07:41:47 -0500 Received: by mail-wi0-f176.google.com with SMTP id em10so5153584wid.3 for ; Tue, 20 Jan 2015 04:41:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=poCxVk+dRrPhlBBYbjaT3+EkGdfhOnoe8Ty3w2+iqwA=; b=XoWnrmnMQFRkHpII7SA6stZEVN68S2XUa42BpEulCaL4bJ65LiOMESgxroCKg1cmSN LgkUffqTmw+jEw75Et3LwQ8OJkdemQuxC3sitl5uuA33uST2jh3KNWZmoim+pMpPGjO2 OMFLZwk6lxmzc9+/iX3rc44DrWGcvLpR/VnSlRKaaS9Zq7wlA91FEQXbRVg0qfWPTzCG gL6H6Z1nWABDIV5sgYLpsNT38M76CvN122nXWCKpEk/x/SLuUfK5QtPt5H0mxcUPba3t CXpieE2aupLchgI3zLv+6hzLWDSrvAQZCuFd4ETKCOlPwrqd0E1EfaSoEMaPhAwQatm9 njrQ== X-Gm-Message-State: ALoCoQnXY0hDeC9Ygs1k0urNiFwk61JW02i9Id0fkK3EKyDOun7kv9kPfOatutDcpIp2lK/dTfon X-Received: by 10.180.210.167 with SMTP id mv7mr45581446wic.78.1421757706027; Tue, 20 Jan 2015 04:41:46 -0800 (PST) Received: from localhost.localdomain ([109.110.67.201]) by mx.google.com with ESMTPSA id ck7sm21136704wjb.13.2015.01.20.04.41.44 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Jan 2015 04:41:45 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Cc: Alex Elder Subject: [PATCH 1/3] rbd: fix rbd_dev_parent_get() when parent_overlap == 0 Date: Tue, 20 Jan 2015 15:41:07 +0300 Message-Id: <1421757669-38796-2-git-send-email-idryomov@redhat.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1421757669-38796-1-git-send-email-idryomov@redhat.com> References: <1421757669-38796-1-git-send-email-idryomov@redhat.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The comment for rbd_dev_parent_get() said * We must get the reference before checking for the overlap to * coordinate properly with zeroing the parent overlap in * rbd_dev_v2_parent_info() when an image gets flattened. We * drop it again if there is no overlap. but the "drop it again if there is no overlap" part was missing from the implementation. This lead to absurd parent_ref values for images with parent_overlap == 0, as parent_ref was incremented for each img_request and virtually never decremented. Fix this by leveraging the fact that refresh path calls rbd_dev_v2_parent_info() under header_rwsem and use it for read in rbd_dev_parent_get(), instead of messing around with atomics. Get rid of barriers in rbd_dev_v2_parent_info() while at it - I don't see what they'd pair with now and I suspect we are in a pretty miserable situation as far as proper locking goes regardless. Cc: stable@vger.kernel.org # 3.11+ Signed-off-by: Ilya Dryomov Reviewed-by: Josh Durgin Reviewed-by: Alex Elder --- drivers/block/rbd.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 31fa00f0d707..2990a1c75159 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -2098,32 +2098,26 @@ static void rbd_dev_parent_put(struct rbd_device *rbd_dev) * If an image has a non-zero parent overlap, get a reference to its * parent. * - * We must get the reference before checking for the overlap to - * coordinate properly with zeroing the parent overlap in - * rbd_dev_v2_parent_info() when an image gets flattened. We - * drop it again if there is no overlap. - * * Returns true if the rbd device has a parent with a non-zero * overlap and a reference for it was successfully taken, or * false otherwise. */ static bool rbd_dev_parent_get(struct rbd_device *rbd_dev) { - int counter; + int counter = 0; if (!rbd_dev->parent_spec) return false; - counter = atomic_inc_return_safe(&rbd_dev->parent_ref); - if (counter > 0 && rbd_dev->parent_overlap) - return true; - - /* Image was flattened, but parent is not yet torn down */ + down_read(&rbd_dev->header_rwsem); + if (rbd_dev->parent_overlap) + counter = atomic_inc_return_safe(&rbd_dev->parent_ref); + up_read(&rbd_dev->header_rwsem); if (counter < 0) rbd_warn(rbd_dev, "parent reference overflow"); - return false; + return counter > 0; } /* @@ -4238,7 +4232,6 @@ static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) */ if (rbd_dev->parent_overlap) { rbd_dev->parent_overlap = 0; - smp_mb(); rbd_dev_parent_put(rbd_dev); pr_info("%s: clone image has been flattened\n", rbd_dev->disk->disk_name); @@ -4284,7 +4277,6 @@ static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) * treat it specially. */ rbd_dev->parent_overlap = overlap; - smp_mb(); if (!overlap) { /* A null parent_spec indicates it's the initial probe */