From patchwork Fri Mar 1 23:27:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10836215 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 577091575 for ; Fri, 1 Mar 2019 23:27:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 451A62F050 for ; Fri, 1 Mar 2019 23:27:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 399B92F0AF; Fri, 1 Mar 2019 23:27:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CFE7E2F050 for ; Fri, 1 Mar 2019 23:27:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726232AbfCAX1V (ORCPT ); Fri, 1 Mar 2019 18:27:21 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:41850 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725958AbfCAX1V (ORCPT ); Fri, 1 Mar 2019 18:27:21 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x21NP8WJ123288 for ; Fri, 1 Mar 2019 23:27:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=talph3KKQze7s4I5lWnwtF6MHRON37zrgEIm40ervkM=; b=pvIa69CXdO2Vi+CAZdS9g9oVGJLATjier6TXOBOp+i8PULwwXcJbmqhSMluxC6qkNJew zLwbRAsxsmu9BbjLxpekuRTTavBrE1wXFWcXVwrjN3h2ctq2Z9x2Fji/ITW/OiRcQaip fl0M7NAzTrrHxoR4vQ3P8PtCgzicWAMmLaWE+MTrwU0DRivFZdA0+Blh/H5823mcHiHp i9hk9CQL9cXaBs6awSm4fnDd1iHzt+Vp0jeogiHgTNYWQYuP1rpHfIxk+E4o3cIejIDs NltpMbzWnkrB2uiSrM7Z3c8c/yT8ZgW8pMZpOe7sVvGR6EkbbPbgOZLRERpoj8JgWl24 zg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2130.oracle.com with ESMTP id 2qtupetehw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 01 Mar 2019 23:27:20 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x21NREbp006084 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 1 Mar 2019 23:27:15 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x21NREBK007595 for ; Fri, 1 Mar 2019 23:27:14 GMT Received: from localhost (/10.159.229.175) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 01 Mar 2019 15:27:14 -0800 Subject: [PATCH 04/23] xfs_scrub_all: walk the lsblk device/fs hierarchy correctly From: "Darrick J. Wong" To: darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Fri, 01 Mar 2019 15:27:13 -0800 Message-ID: <155148283348.16677.13557626525564289346.stgit@magnolia> In-Reply-To: <155148280859.16677.6057998944865066232.stgit@magnolia> References: <155148280859.16677.6057998944865066232.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9182 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=15 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=826 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903010161 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Back when I was designing xfs_scrub_all, I naïvely assumed that the emitted output would always list physical storage before the virtual devices stacked atop it. However, this is not actually true when one omits the "NAME" column, which is crucial to forcing the output (json or otherwise) to capture the block device hierarchy. If the assumption is violated, the program crashes with a python exception. To fix this, force the hierarchal json output and restructure the discovery routines to walk the json object that we receive, from the top (physical devices) downwards to wherever there are live xfs filesystems. Signed-off-by: Darrick J. Wong --- scrub/xfs_scrub_all.in | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/scrub/xfs_scrub_all.in b/scrub/xfs_scrub_all.in index c4e9899d..5b76b49a 100644 --- a/scrub/xfs_scrub_all.in +++ b/scrub/xfs_scrub_all.in @@ -28,9 +28,21 @@ def DEVNULL(): def find_mounts(): '''Map mountpoints to physical disks.''' + def find_xfs_mounts(bdev, fs, lastdisk): + '''Attach lastdisk to each fs found under bdev.''' + if bdev['fstype'] == 'xfs' and bdev['mountpoint'] is not None: + mnt = bdev['mountpoint'] + if mnt in fs: + fs[mnt].add(lastdisk) + else: + fs[mnt] = set([lastdisk]) + if 'children' not in bdev: + return + for child in bdev['children']: + find_xfs_mounts(child, fs, lastdisk) fs = {} - cmd=['lsblk', '-o', 'KNAME,TYPE,FSTYPE,MOUNTPOINT', '-J'] + cmd=['lsblk', '-o', 'NAME,KNAME,TYPE,FSTYPE,MOUNTPOINT', '-J'] result = subprocess.Popen(cmd, stdout=subprocess.PIPE) result.wait() if result.returncode != 0: @@ -38,18 +50,12 @@ def find_mounts(): sarray = [x.decode(sys.stdout.encoding) for x in result.stdout.readlines()] output = ' '.join(sarray) bdevdata = json.loads(output) + # The lsblk output had better be in disks-then-partitions order for bdev in bdevdata['blockdevices']: - if bdev['type'] in ('disk', 'loop'): - lastdisk = bdev['kname'] - if bdev['fstype'] == 'xfs': - mnt = bdev['mountpoint'] - if mnt is None: - continue - if mnt in fs: - fs[mnt].add(lastdisk) - else: - fs[mnt] = set([lastdisk]) + lastdisk = bdev['kname'] + find_xfs_mounts(bdev, fs, lastdisk) + return fs def kill_systemd(unit, proc):