From patchwork Thu Mar 14 21:04:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853605 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF14A17E6 for ; Thu, 14 Mar 2019 21:04:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC45A2A23B for ; Thu, 14 Mar 2019 21:04:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D83882A30E; Thu, 14 Mar 2019 21:04:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7DFF52A791 for ; Thu, 14 Mar 2019 21:04:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727765AbfCNVE4 (ORCPT ); Thu, 14 Mar 2019 17:04:56 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50918 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVEz (ORCPT ); Thu, 14 Mar 2019 17:04:55 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwvZE095467; Thu, 14 Mar 2019 21:04:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=talph3KKQze7s4I5lWnwtF6MHRON37zrgEIm40ervkM=; b=rnBC9IsJvEfA9gjxR3J0xN6aY4/sZGaGi6oliRpGKXVJlwbDAq0cJhz0LJeK5TOXeaJW as0dlxS3ZPdLODUClNcWLNRuK/40Di8Z72tgXTAfaTHEdC0m60uJrGBOFdNw9r0Ta6ZN GkAI+LKQrLwXz+zSM7mS8acevsb+EEhXxpx7f2PZIaYLm6kQKu0YzjKFp3BdMMz97ev8 cTLMuz97sD8msw/SuO877om9y3IzBHZlleycUAdadJyyTlKisnhNt65WhR3doRMseJYs KgD+X9lmkzvaBj1DZwWk2vSnwpVbGQezKmIOBrtQPqy8MKeMNmYmvnsT5Xc+fuiYMmvY Pw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2r44wukfgc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:53 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4qQD032764 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:52 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL4pnO019767; Thu, 14 Mar 2019 21:04:52 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:51 +0000 Subject: [PATCH 10/36] xfs_scrub_all: walk the lsblk device/fs hierarchy correctly From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:04:51 -0700 Message-ID: <155259749135.31886.17642733861513368966.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=9 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=865 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Back when I was designing xfs_scrub_all, I naïvely assumed that the emitted output would always list physical storage before the virtual devices stacked atop it. However, this is not actually true when one omits the "NAME" column, which is crucial to forcing the output (json or otherwise) to capture the block device hierarchy. If the assumption is violated, the program crashes with a python exception. To fix this, force the hierarchal json output and restructure the discovery routines to walk the json object that we receive, from the top (physical devices) downwards to wherever there are live xfs filesystems. Signed-off-by: Darrick J. Wong --- scrub/xfs_scrub_all.in | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/scrub/xfs_scrub_all.in b/scrub/xfs_scrub_all.in index c4e9899d..5b76b49a 100644 --- a/scrub/xfs_scrub_all.in +++ b/scrub/xfs_scrub_all.in @@ -28,9 +28,21 @@ def DEVNULL(): def find_mounts(): '''Map mountpoints to physical disks.''' + def find_xfs_mounts(bdev, fs, lastdisk): + '''Attach lastdisk to each fs found under bdev.''' + if bdev['fstype'] == 'xfs' and bdev['mountpoint'] is not None: + mnt = bdev['mountpoint'] + if mnt in fs: + fs[mnt].add(lastdisk) + else: + fs[mnt] = set([lastdisk]) + if 'children' not in bdev: + return + for child in bdev['children']: + find_xfs_mounts(child, fs, lastdisk) fs = {} - cmd=['lsblk', '-o', 'KNAME,TYPE,FSTYPE,MOUNTPOINT', '-J'] + cmd=['lsblk', '-o', 'NAME,KNAME,TYPE,FSTYPE,MOUNTPOINT', '-J'] result = subprocess.Popen(cmd, stdout=subprocess.PIPE) result.wait() if result.returncode != 0: @@ -38,18 +50,12 @@ def find_mounts(): sarray = [x.decode(sys.stdout.encoding) for x in result.stdout.readlines()] output = ' '.join(sarray) bdevdata = json.loads(output) + # The lsblk output had better be in disks-then-partitions order for bdev in bdevdata['blockdevices']: - if bdev['type'] in ('disk', 'loop'): - lastdisk = bdev['kname'] - if bdev['fstype'] == 'xfs': - mnt = bdev['mountpoint'] - if mnt is None: - continue - if mnt in fs: - fs[mnt].add(lastdisk) - else: - fs[mnt] = set([lastdisk]) + lastdisk = bdev['kname'] + find_xfs_mounts(bdev, fs, lastdisk) + return fs def kill_systemd(unit, proc):