From patchwork Thu Mar 14 21:03:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853587 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 538A16C2 for ; Thu, 14 Mar 2019 21:03:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BEF72A710 for ; Thu, 14 Mar 2019 21:03:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A35C2A7A7; Thu, 14 Mar 2019 21:03:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96FBB2A7A9 for ; Thu, 14 Mar 2019 21:03:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727625AbfCNVD4 (ORCPT ); Thu, 14 Mar 2019 17:03:56 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:49784 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVD4 (ORCPT ); Thu, 14 Mar 2019 17:03:56 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwqgN095361; Thu, 14 Mar 2019 21:03:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=jQ4x/kmEGo8n/vc83NDm3EAKpuiA4+Bs9ApeLePX2VQ=; b=GYk9hInWwX0TpOR/eaVpb2JibTYQ0V66yB6gGY5PaPk0WskGK0QK85PUJtMC5Q3rW8dO mZaQJbRmwCB6bPkd4UzKu8JFfgA4pPMx1tD1e6AWfXB1RShs69WDC0FV/rXmgBCkIM2W k41Vq4qbx1z63kevLGn5CeL53Nd3jbmV1HYdDVjfM/TYCWWR3F8BW1pKpoe04H8JKjdP sJwgtfniK21QBGpqWf6KX0DGCiBY/ph5O9Bq9et0BslAr5qeGeO5tnq5H1eNlWv1RbfH y4mjB4+XBc7tayLRL39uUmvyF2P37ECRECqGL8f8P3W7Xnrq2YO8kNMkq5G6PyHSWjm0 ow== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2r44wukfcf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:03:51 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL3pBG025470 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:03:51 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL3nwB019113; Thu, 14 Mar 2019 21:03:50 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:03:49 +0000 Subject: [PATCH 01/36] libxfs: fix repair deadlock due to failed inode flushes. From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Arkadiusz Miskiewicz , Dave Chinner Date: Thu, 14 Mar 2019 14:03:49 -0700 Message-ID: <155259742930.31886.16675099707553706623.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Dave Chinner If inode_item_done() fails to flush an inode after we've grabbed a reference to the underlying buffer during a transaction commit, we fail to put the buffer and hence leak it. We then deadlock on the next lookup ofthe inode buffer as it is still locked and no-one owns it. To fix it, put the buffer on error so that it gets unlocked and can be recovered appropriately in a later phase of repair. Reported-by: Arkadiusz Miskiewicz Fixes: d15188a1ec14 ("xfs: rework the inline directory verifiers") Signed-off-by: Dave Chinner Reviewed-by: Darrick J. Wong Signed-off-by: Darrick J. Wong Reviewed-by: Allison Henderson --- libxfs/trans.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/libxfs/trans.c b/libxfs/trans.c index 46ff8b4a..10a35dd4 100644 --- a/libxfs/trans.c +++ b/libxfs/trans.c @@ -824,8 +824,10 @@ _("Transaction block reservation exceeded! %u > %u\n"), /* * Transaction commital code follows (i.e. write to disk in libxfs) + * + * XXX (dgc): should failure to flush the inode (e.g. due to uncorrected + * corruption) result in transaction commit failure w/ EFSCORRUPTED? */ - static void inode_item_done( xfs_inode_log_item_t *iip) @@ -856,17 +858,24 @@ inode_item_done( return; } + /* + * Flush the inode and disassociate it from the transaction regardless + * of whether the flush succeed or not. If we fail the flush, make sure + * we still release the buffer reference we currently hold. + */ bp->b_log_item = iip; error = libxfs_iflush_int(ip, bp); + ip->i_transp = NULL; /* disassociate from transaction */ + bp->b_log_item = NULL; /* remove log item */ + bp->b_transp = NULL; /* remove xact ptr */ + if (error) { fprintf(stderr, _("%s: warning - iflush_int failed (%d)\n"), progname, error); + libxfs_putbuf(bp); return; } - ip->i_transp = NULL; /* disassociate from transaction */ - bp->b_log_item = NULL; /* remove log item */ - bp->b_transp = NULL; /* remove xact ptr */ libxfs_writebuf(bp, 0); #ifdef XACT_DEBUG fprintf(stderr, "flushing dirty inode %llu, buffer %p\n", From patchwork Thu Mar 14 21:03:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853589 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE29F6C2 for ; Thu, 14 Mar 2019 21:04:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A76A529DEC for ; Thu, 14 Mar 2019 21:04:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A5D4A2A23B; Thu, 14 Mar 2019 21:04:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 030B529DEC for ; Thu, 14 Mar 2019 21:04:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727715AbfCNVEC (ORCPT ); Thu, 14 Mar 2019 17:04:02 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:49514 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVEC (ORCPT ); Thu, 14 Mar 2019 17:04:02 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwjNm102525; Thu, 14 Mar 2019 21:03:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=aMkAGiQOwuihC2Pnxq7lPRfbety0U2FGLU/CExfdSew=; b=FjxCDS0jGf4BIHH1QtD8BbK4XyelBvqq1pk0lElcbSDCDg9eOMzZTst2AobbJzJusTKn BcnZdO2KDQeClcO6KzInjAuC2fhcd5yDUB8Hd/5+oL+0tejEqANWSe1qoh1BsUijD3aH MT+7B7tMNZ+wVtbK6kwia47s41lEG7MNfg/zNY5uUHFVS9RmWlgANQPY5aniBsHDw2tJ 7lQiEgX/QZ/TudFyKaAviYx6jSxC9PofKdo58bN/rvsCzXMMDMCJQsuGAa5ZQXbkAJGy iOs3XN6O2yrRZhbka9pRsI5II5lVY3q/gwnbJED/AxMuAwO+IUXrrmlFxBH7RRzKTF9W Dw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2r464ruea6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:03:57 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL3uA8000561 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:03:56 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL3utc032508; Thu, 14 Mar 2019 21:03:56 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:03:56 +0000 Subject: [PATCH 02/36] configure: use sys/xattr.h for fsetxattr detection From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Christoph Hellwig Date: Thu, 14 Mar 2019 14:03:55 -0700 Message-ID: <155259743543.31886.18053722203157328922.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong The only user of fsetxattr and HAVE_FSETXATTR is fsr, which includes sys/xattr.h (from libc). However, the m4 macro to detect fsetxattr support requires attr/xattr.h (from libattr). libattr dropped xattr.h last year, so update the check. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig Reviewed-by: Allison Henderson --- m4/package_libcdev.m4 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4 index 7ee4acdd..2c0c72ce 100644 --- a/m4/package_libcdev.m4 +++ b/m4/package_libcdev.m4 @@ -223,14 +223,14 @@ AC_DEFUN([AC_HAVE_FLS], ]) # -# Check if we have a fsetxattr call (Mac OS X) +# Check if we have a fsetxattr call # AC_DEFUN([AC_HAVE_FSETXATTR], [ AC_CHECK_DECL([fsetxattr], have_fsetxattr=yes, [], [#include - #include ] + #include ] ) AC_SUBST(have_fsetxattr) ]) From patchwork Thu Mar 14 21:04:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853591 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44F856C2 for ; Thu, 14 Mar 2019 21:04:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EA0D2A06A for ; Thu, 14 Mar 2019 21:04:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2CFFD2A296; Thu, 14 Mar 2019 21:04:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 580B62A06A for ; Thu, 14 Mar 2019 21:04:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727412AbfCNVEH (ORCPT ); Thu, 14 Mar 2019 17:04:07 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:49658 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727749AbfCNVEG (ORCPT ); Thu, 14 Mar 2019 17:04:06 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwu2n133203; Thu, 14 Mar 2019 21:04:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=vbLW49NfJhtCMdWhHL0ish6SYotHE/B/EBD8sqyr0uA=; b=UtqSl/orV9puTwP1nS5Z+vYHlfBcU/KUCYZLu0DqIaF3LVVi+mtygG40WBT//6l0vKje GKXMbdOzit9mUXdICRzc3C6KFp8b9NqkvIKqOQyKT0vLC/1muGJdCwCAfQYnlnCx5huj 0Ae1ZTLoGBHYDjIA4WDd8DhCGaBW1KnnQ4rOhOOY/qxsgG/PnPxA9xYcivBYBnVuGZ46 S1ohfPSNitlL1YyN08WRsXFFFFji5qFZMkLePPkw1tWXT3aofNPdAQZRfu14XTJY5J93 0Pi6Az3RWj5R3t3lRyb+/1YV4TSDogp8fzgmzoJ/9BQdZymyZiixgHibj0HvXRBHc4Ya hQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2r430f3xgu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:03 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL42ow001386 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:02 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL42Ha002530; Thu, 14 Mar 2019 21:04:02 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:02 +0000 Subject: [PATCH 03/36] configure.ac: fix alignment of features From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:04:01 -0700 Message-ID: <155259744173.31886.6475707140088498779.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Fix the alignment of the feature options in the --help screen. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen Reviewed-by: Allison Henderson --- configure.ac | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/configure.ac b/configure.ac index ccc6e292..5a4e55ad 100644 --- a/configure.ac +++ b/configure.ac @@ -28,82 +28,82 @@ if test "${BUILD_CFLAGS+set}" != "set"; then fi AC_ARG_ENABLE(shared, -[ --enable-shared=[yes/no] Enable use of shared libraries [default=yes]],, +[ --enable-shared=[yes/no] Enable use of shared libraries [default=yes]],, enable_shared=yes) AC_SUBST(enable_shared) AC_ARG_ENABLE(gettext, -[ --enable-gettext=[yes/no] Enable alternate language support [default=yes]],, +[ --enable-gettext=[yes/no] Enable alternate language support [default=yes]],, enable_gettext=yes) AC_SUBST(enable_gettext) AC_ARG_ENABLE(blkid, -[ --enable-blkid=[yes/no] Enable use of block device id library [default=yes]],, +[ --enable-blkid=[yes/no] Enable use of block device id library [default=yes]],, enable_blkid=yes) AC_SUBST(enable_blkid) AC_ARG_ENABLE(readline, -[ --enable-readline=[yes/no] Enable readline command editing [default=no]], +[ --enable-readline=[yes/no] Enable readline command editing [default=no]], test $enable_readline = yes && libreadline="-lreadline", enable_readline=no) AC_SUBST(libreadline) AC_SUBST(enable_readline) AC_ARG_ENABLE(editline, -[ --enable-editline=[yes/no] Enable editline command editing [default=no]], +[ --enable-editline=[yes/no] Enable editline command editing [default=no]], test $enable_editline = yes && libeditline="-ledit", enable_editline=no) AC_SUBST(libeditline) AC_SUBST(enable_editline) AC_ARG_ENABLE(termcap, -[ --enable-termcap=[yes/no] Enable terminal capabilities library [default=no]], +[ --enable-termcap=[yes/no] Enable terminal capabilities library [default=no]], test $enable_termcap = yes && libtermcap="-ltermcap",) AC_SUBST(libtermcap) AC_ARG_ENABLE(lib64, -[ --enable-lib64=[yes/no] Enable lib64 support [default=yes]],, +[ --enable-lib64=[yes/no] Enable lib64 support [default=yes]],, enable_lib64=yes) AC_SUBST(enable_lib64) AC_ARG_ENABLE(librt, -[ --enable-librt=[yes/no] Enable librt support [default=yes]],, +[ --enable-librt=[yes/no] Enable librt support [default=yes]],, enable_librt=yes) AC_SUBST(enable_librt) # Enable UBSAN; set enable_ubsan=probe below to enable autoprobe. AC_ARG_ENABLE(ubsan, -[ --enable-ubsan=[yes/no] Enable Undefined Behavior Sanitizer (UBSAN) [default=no]],, +[ --enable-ubsan=[yes/no] Enable Undefined Behavior Sanitizer (UBSAN) [default=no]],, enable_ubsan=no) AC_SUBST(enable_ubsan) # Enable ADDRSAN; set enable_addrsan=probe below to enable autoprobe. AC_ARG_ENABLE(addrsan, -[ --enable-addrsan=[yes/no] Enable Address Sanitizer (ADDRSAN) [default=no]],, +[ --enable-addrsan=[yes/no] Enable Address Sanitizer (ADDRSAN) [default=no]],, enable_addrsan=no) AC_SUBST(enable_addrsan) # Enable THREADSAN; set enable_threadsan=probe to enable autoprobe. AC_ARG_ENABLE(threadsan, -[ --enable-threadsan=[yes/no] Enable Thread Sanitizer (THREADSAN) [default=no]],, +[ --enable-threadsan=[yes/no] Enable Thread Sanitizer (THREADSAN) [default=no]],, enable_threadsan=no) AC_SUBST(enable_threadsan) AC_ARG_ENABLE(lto, -[ --enable-lto=[yes/no] Enable link time optimization (LTO) [default=no]],, +[ --enable-lto=[yes/no] Enable link time optimization (LTO) [default=no]],, enable_lto=no) AC_SUBST(enable_lto) # Enable xfs_scrub build AC_ARG_ENABLE(scrub, -[ --enable-scrub=[yes/no] Enable build of xfs_scrub utility [default=yes]],, +[ --enable-scrub=[yes/no] Enable build of xfs_scrub utility [default=yes]],, enable_scrub=yes) AC_SUBST(enable_scrub) # Enable libicu for xfs_scrubbing of malicious unicode sequences in names AC_ARG_ENABLE(libicu, -[ --enable-libicu=[yes/no] Enable Unicode name scanning in xfs_scrub (libicu) [default=probe]],, +[ --enable-libicu=[yes/no] Enable Unicode name scanning in xfs_scrub (libicu) [default=probe]],, enable_libicu=probe) # From patchwork Thu Mar 14 21:04:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853593 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D98966C2 for ; Thu, 14 Mar 2019 21:04:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C277E29DEC for ; Thu, 14 Mar 2019 21:04:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C0A812A1AD; Thu, 14 Mar 2019 21:04:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64B0529DEC for ; Thu, 14 Mar 2019 21:04:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727749AbfCNVEM (ORCPT ); Thu, 14 Mar 2019 17:04:12 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:49704 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727736AbfCNVEL (ORCPT ); Thu, 14 Mar 2019 17:04:11 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwfNn102491; Thu, 14 Mar 2019 21:04:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=qxVwmFY7h5ezN4AMS+55JQ1cpeURqD/MGs72f/XO5RI=; b=VeGPIGiaXMBx4ClLhSxw4Xgx1hor7VGLKbaH6UhJVVftiKgvHsezb5GXl6mw0s5AhAu2 N3bXo9pDawlu3TX5/nkfIQpGTGWku+1q9RRBydJJvoTwXXxxwomG8up1X632nZUWRgxT J/nAHUyFSovo83hMWK/MWuS7pvdBinfZiNXVqMwR1ECml9kd+CIfNzoCV14eBJwGMURq l96i9lbGoLoHhBnDF6wxxAdavMCWaO6OQET9OQLNaNMr+3Yw7aREpifv9chSzzWja619 +D+zkMVmf2aLXmBzmOufK51WOyNabfx5Fk0Gqm1wOeJorfnGvsAeskwbn0TmVsHcAV36 zA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2r464ruebe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:09 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL49jf026718 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:09 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL484Z019337; Thu, 14 Mar 2019 21:04:09 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:08 +0000 Subject: [PATCH 04/36] debian: drop dangling libhandle.a symlinks in xfslibs-dev From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:04:07 -0700 Message-ID: <155259744781.31886.15637750109545213350.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=910 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong We don't ship static libhandle libraries anymore, so make sure we drop the symlink. Fixes: ec1cf08dbeb2d ("Several updates to use more modern Debian packaging") Signed-off-by: Darrick J. Wong Reviewed-by: Nathan Scott Reviewed-by: Allison Henderson --- debian/rules | 1 + 1 file changed, 1 insertion(+) diff --git a/debian/rules b/debian/rules index 4c50654c..80dff4bb 100755 --- a/debian/rules +++ b/debian/rules @@ -80,6 +80,7 @@ binary-arch: checkroot built install -D -m 0755 debian/local/initramfs.hook debian/xfsprogs/usr/share/initramfs-tools/hooks/xfs rmdir debian/xfslibs-dev/usr/share/doc/xfsprogs rm -f debian/xfslibs-dev/lib/libhandle.la + rm -f debian/xfslibs-dev/lib/libhandle.a rm -fr debian/xfslibs-dev/usr/lib dh_installdocs dh_installchangelogs From patchwork Thu Mar 14 21:04:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853595 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C6CEB17E6 for ; Thu, 14 Mar 2019 21:04:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF7C72A1E1 for ; Thu, 14 Mar 2019 21:04:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ADF862A24A; Thu, 14 Mar 2019 21:04:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 602AA2A3D4 for ; Thu, 14 Mar 2019 21:04:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727760AbfCNVEX (ORCPT ); Thu, 14 Mar 2019 17:04:23 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:49980 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727625AbfCNVEX (ORCPT ); Thu, 14 Mar 2019 17:04:23 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwe4S132780; Thu, 14 Mar 2019 21:04:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=xCe/th0p0Scbdj7gcJSZDA0C6bXfAwZMvfZI+shc1Mw=; b=16oNdb5u0goLX3CBXplzMH5xCfVrjLv33XxKqOxr7Ybx1bY8AXwVrc6mYolCljIuQVQJ F4gXIp/HO/w0cPqJSx6olp2+g3FxOrz//gk4YTHRrvm4GGHnoGWdHEFDkzed8CJ5C1T6 RVwrLx8RlHxjyP4DT8aSa29Kc2y5c+67rKb6D1ewtG6J6yaCHxv08eXy2W982qxlkVC+ YFLw+gcdfbJVFyfUExviNwsFqRiq0tAmx7LmPSTZZavvQ1alhUUXB9h16QZN5Cxv3YTu GnIcHmM9tiMrM58YNwEMhPCQ3+llM7nnZC6V2t+S3DNKaZv+KEgjgk1OaQlQn8Me9mmY 8A== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2130.oracle.com with ESMTP id 2r430f3xj0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:21 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4FEm014696 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:15 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4EVK002636; Thu, 14 Mar 2019 21:04:15 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:14 +0000 Subject: [PATCH 05/36] debian: don't bypass top level Makefile when building subdirs From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:04:14 -0700 Message-ID: <155259745436.31886.466306969009991047.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong The top level Makefile does some processing to set build environment variables (Q and CHECK_CMD). debian/rules uses -C to build subdirs directly, which bypases this feature of the top-level makefile, which causes more build spew than necessary (because Q never gets set to quiet the build). Since the top level makefile can be used to build the subdirs debian/rules cares about, drop the -C and build subdirs via the top level Makefile to quiet the build. Signed-off-by: Darrick J. Wong Reviewed-by: Nathan Scott Reviewed-by: Allison Henderson --- debian/rules | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/debian/rules b/debian/rules index 80dff4bb..7c2a8b3c 100755 --- a/debian/rules +++ b/debian/rules @@ -51,7 +51,7 @@ dibuild: $(MAKE) -C $$dir NODEP=1 install-headers; \ done; \ for dir in include libxfs libxcmd libfrog mkfs; do \ - $(MAKE) -C $$dir; \ + $(MAKE) $$dir; \ done; \ mv mkfs/mkfs.xfs mkfs/mkfs.xfs-$(bootpkg); \ $(MAKE) distclean; \ From patchwork Thu Mar 14 21:04:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853597 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B34C1874 for ; Thu, 14 Mar 2019 21:04:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 144F02A393 for ; Thu, 14 Mar 2019 21:04:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 12C422A56C; Thu, 14 Mar 2019 21:04:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B80092A393 for ; Thu, 14 Mar 2019 21:04:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727625AbfCNVEY (ORCPT ); Thu, 14 Mar 2019 17:04:24 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:49928 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727616AbfCNVEY (ORCPT ); Thu, 14 Mar 2019 17:04:24 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwrno102596; Thu, 14 Mar 2019 21:04:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=9RCT+vZ4LHGltOx07QR2gv0KFtbNHD4EcZNxCaWTFA8=; b=KvZx0u36nH3k8HeXaIwj5Fjw8CX1YUmw+QN3oREh8Y+8djP3bPcbmnHoP4M7rr5zBBph jLkjgxgN/E28EssGTzdPOqt5n63AvqI0JDbflBgLH5SxTxibzP5zBHMANAVZA/+8bNdM hQF6sx7ImR5sOFtIzPq49rbOmiWv85FbUoWkf9SsIoQD/SmDeCs/ijBsWCKjKj+8C/mp TmoWxyWuel3DqWqtfqhvn00MEyEfgwLSmZUXSj5kdCogntlJSYCCf4/4sveXEjF3obOr m/Vs4JQa4z6DcrMJ7DXFm2AdFaJUge2LehKIrBc4DivP2vAHkGC7xPCswzKS2yN/hLlg wg== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2r464ruec7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:21 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4LUJ002696 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:21 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4L1U002660; Thu, 14 Mar 2019 21:04:21 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:20 +0000 Subject: [PATCH 06/36] debian: enable parallel make From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:04:20 -0700 Message-ID: <155259746044.31886.13633380455067163247.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Use parallel make to speed up dpkg builds. Signed-off-by: Darrick J. Wong Reviewed-by: Nathan Scott --- debian/rules | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/debian/rules b/debian/rules index 7c2a8b3c..e8509fb3 100755 --- a/debian/rules +++ b/debian/rules @@ -2,6 +2,11 @@ export DH_VERBOSE=1 +ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS))) + NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS))) + PMAKEFLAGS += -j$(NUMJOBS) +endif + package = xfsprogs develop = xfslibs-dev bootpkg = xfsprogs-udeb @@ -30,7 +35,7 @@ build-arch: built build-indep: built built: dibuild config @echo "== dpkg-buildpackage: build" 1>&2 - $(MAKE) default + $(MAKE) $(PMAKEFLAGS) default touch built config: .census @@ -38,7 +43,7 @@ config: .census @echo "== dpkg-buildpackage: configure" 1>&2 $(checkdir) AUTOHEADER=/bin/true dh_autoreconf - $(options) $(MAKE) include/platform_defs.h + $(options) $(MAKE) $(PMAKEFLAGS) include/platform_defs.h touch .census dibuild: @@ -48,10 +53,10 @@ dibuild: $(diopts) $(MAKE) include/platform_defs.h; \ mkdir -p include/xfs; \ for dir in include libxfs; do \ - $(MAKE) -C $$dir NODEP=1 install-headers; \ + $(MAKE) $(PMAKEFLAGS) -C $$dir NODEP=1 install-headers; \ done; \ for dir in include libxfs libxcmd libfrog mkfs; do \ - $(MAKE) $$dir; \ + $(MAKE) $(PMAKEFLAGS) $$dir; \ done; \ mv mkfs/mkfs.xfs mkfs/mkfs.xfs-$(bootpkg); \ $(MAKE) distclean; \ From patchwork Thu Mar 14 21:04:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853599 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0596E1515 for ; Thu, 14 Mar 2019 21:04:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E264F2988B for ; Thu, 14 Mar 2019 21:04:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C37F429DEC; Thu, 14 Mar 2019 21:04:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 541D229DEC for ; Thu, 14 Mar 2019 21:04:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727616AbfCNVEo (ORCPT ); Thu, 14 Mar 2019 17:04:44 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:50348 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVEo (ORCPT ); Thu, 14 Mar 2019 17:04:44 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwhw3102507; Thu, 14 Mar 2019 21:04:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=9KPBiGvUZVixIRVgF2LvW0yiPvbdeR2i06Z+SATxAQ4=; b=tRgqr1zXbll5Hj7FLshYEYULkx1DE1q5vFFtuIqUFRU7fTkAmaIicUJ4LFzgz4LpTlH3 0S3CpQ0nkJJlNvH4315mwOSxe7K3uKvF4Q4eNqBLSTzpty/ET6qr6NTFMS7dPrDsUwAq cqcgYOC8uMwSgN9S9LJGMcI3pwHN5KqwZng7NPlPwsUnYBoyoMl4BogPHmVRJ6wp3Atu s0cGir0Eedj+ow5+wafRNw7W1v1nZHiyv2QKIATTOcnGaCaOdkjqISoaKIJ8OWUl+RUL Qn6nRTYQG4ImQSq8KYj+VMroQimv09/JgdNqWJZRG9ocpKJSW03Qg6dLKxCJtakZlmWp 1w== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2r464ruede-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:39 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4Xu1015850 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:33 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4XuU006597; Thu, 14 Mar 2019 21:04:33 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:33 +0000 Subject: [PATCH 07/36] xfs_io: actually check copy file range helper return values From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Anna Schumaker , Christoph Hellwig Date: Thu, 14 Mar 2019 14:04:26 -0700 Message-ID: <155259746648.31886.15984241616376190830.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong We need to check the return value of copy_src_filesize and copy_dst_truncate because either could return -1 due to fstat/ftruncate failure. Fixes: 628e112afdd98c5 ("xfs_io: implement 'copy_range' command") Signed-off-by: Darrick J. Wong Reviewed-by: Anna Schumaker Reviewed-by: Christoph Hellwig --- io/copy_file_range.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/io/copy_file_range.c b/io/copy_file_range.c index 4e2969c9..d069e5bb 100644 --- a/io/copy_file_range.c +++ b/io/copy_file_range.c @@ -120,11 +120,24 @@ copy_range_f(int argc, char **argv) return 0; if (src == 0 && dst == 0 && len == 0) { - len = copy_src_filesize(fd); - copy_dst_truncate(); + off64_t sz; + + sz = copy_src_filesize(fd); + if (sz < 0 || (unsigned long long)sz > SIZE_MAX) { + ret = 1; + goto out; + } + len = sz; + + ret = copy_dst_truncate(); + if (ret < 0) { + ret = 1; + goto out; + } } ret = copy_file_range_cmd(fd, &src, &dst, len); +out: close(fd); return ret; } From patchwork Thu Mar 14 21:04:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A18D17E6 for ; Thu, 14 Mar 2019 21:04:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 734762A288 for ; Thu, 14 Mar 2019 21:04:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 71D2D2A792; Thu, 14 Mar 2019 21:04:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1CCBA2A288 for ; Thu, 14 Mar 2019 21:04:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727760AbfCNVEv (ORCPT ); Thu, 14 Mar 2019 17:04:51 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:50544 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVEv (ORCPT ); Thu, 14 Mar 2019 17:04:51 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwrt9132866; Thu, 14 Mar 2019 21:04:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=UPKVKFfnMxgC/XwefkeVOSa8THdxJnGNaDKVYZwnEsU=; b=Bon1BfzMpf4rOXGFl8O1Ts7yZ8unIelTUGnPUj3vAYe//6cjfTvPg2AmS8nezLtfQ2RB 4UEB7BJo54y0VoDJxaM07/4kXBDQrsXWZ9S8k3QJanhsW87rcdoFWkVYYlmOIvq1oYpR 3H4E5127PlKgEuQUkc72Jhqivngxt57mmVRmcB7C+6aomSZMnmxmqMwUlKCqr69hB3GI Qi2FTgdlincZOV+Zs+QY9TPbU0JPK124j24kX3HuIcxfIaJD6DYSfmx32OdYtgfHbr/0 VxGn3IUdGRHoDG/eQ7CTsqwmq7nmhLazaZDXw0dCtNqaadS84DO6Lzo2KNePE4Uee6wg JA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2r430f3xkn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:46 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4dZE028130 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:40 GMT Received: from abhmp0020.oracle.com (abhmp0020.oracle.com [141.146.116.26]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL4dhv000709; Thu, 14 Mar 2019 21:04:39 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:39 +0000 Subject: [PATCH 08/36] xfs_io: statx -r should print attributes_mask From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Christoph Hellwig Date: Thu, 14 Mar 2019 14:04:39 -0700 Message-ID: <155259747924.31886.11402658884761183760.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong We're dumping the raw structure, so we ought to dump everything, including the attributes_mask field. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig Reviewed-by: Eric Sandeen --- io/stat.c | 4 ++++ io/statx.h | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/io/stat.c b/io/stat.c index 64662b43..517be66e 100644 --- a/io/stat.c +++ b/io/stat.c @@ -6,6 +6,9 @@ * Portions of statx support written by David Howells (dhowells@redhat.com) */ +/* Try to pick up statx definitions from the system headers. */ +#include + #include "command.h" #include "input.h" #include "init.h" @@ -272,6 +275,7 @@ dump_raw_statx(struct statx *stx) printf("stat.ino = %llu\n", (unsigned long long)stx->stx_ino); printf("stat.size = %llu\n", (unsigned long long)stx->stx_size); printf("stat.blocks = %llu\n", (unsigned long long)stx->stx_blocks); + printf("stat.attributes_mask = 0x%llx\n", (unsigned long long)stx->stx_attributes_mask); printf("stat.atime.tv_sec = %lld\n", (long long)stx->stx_atime.tv_sec); printf("stat.atime.tv_nsec = %d\n", stx->stx_atime.tv_nsec); printf("stat.btime.tv_sec = %lld\n", (long long)stx->stx_btime.tv_sec); diff --git a/io/statx.h b/io/statx.h index 4e4b31ee..4f40eaa1 100644 --- a/io/statx.h +++ b/io/statx.h @@ -107,7 +107,7 @@ struct statx { __u64 stx_ino; /* Inode number */ __u64 stx_size; /* File size */ __u64 stx_blocks; /* Number of 512-byte blocks allocated */ - __u64 __spare1[1]; + __u64 stx_attributes_mask; /* Mask to show what's supported in stx_attributes */ /* 0x40 */ struct statx_timestamp stx_atime; /* Last access time */ struct statx_timestamp stx_btime; /* File creation time */ From patchwork Thu Mar 14 21:04:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853601 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11DC41515 for ; Thu, 14 Mar 2019 21:04:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC0E62A23B for ; Thu, 14 Mar 2019 21:04:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EA8462A296; Thu, 14 Mar 2019 21:04:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7D4192A78F for ; Thu, 14 Mar 2019 21:04:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727749AbfCNVEu (ORCPT ); Thu, 14 Mar 2019 17:04:50 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50808 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVEt (ORCPT ); Thu, 14 Mar 2019 17:04:49 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKxAYV095905; Thu, 14 Mar 2019 21:04:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Wbqvu2QM4X7NkMqPlxnvXVXbQdsBAmDNw+d19t616gA=; b=U8PbQcfCRupFqTcTWnvg4DFnfM/v/3P9vpnd4QCnGgANKsi3PWqAbz7f00WCk/jfaBzE 9mDTKMNz6XFQurbkeVcqQWVbyUs1x2IaylFJbbuzBsn1ZmiizpPigW99r/LDn2ql5SC8 b8IXYwtVoEYEfXUKnOh+99dq3BIGlV/QJK15elatjNlZjsZciKUSNTjtopjgwx6g7yeL f21YB/ckUiKDI6/1zLViCYrQhTB9WWTK21my2aOpTNl4H+x+gtwEnCuF8w4tcogktTbT 2PEcm82TICTC8DugKw7jhM3z9huuGDyBnFYwcmyBYJ8acuwd++qm3V1DU1Z0gq9oMcIB Jg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2r44wukfft-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:47 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4kIZ016440 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:46 GMT Received: from abhmp0020.oracle.com (abhmp0020.oracle.com [141.146.116.26]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4jn8003042; Thu, 14 Mar 2019 21:04:46 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:45 +0000 Subject: [PATCH 09/36] xfs_io: don't walk off the end of argv in fzero_f From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:04:45 -0700 Message-ID: <155259748531.31886.6568534900832262149.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong The fzero_f function doesn't check that there are enough non-switch parameters to supply offset and length arguments to fallocate. As a result, we can walk off the end of the argv array and crash. A secondary problem is that we don't use getopt to detect the -k, which is not how most xfs_io commands work. Therefore, use getopt to detect the -k argument and rewire the offset and length interpretation code to check optind and use argv correctly. This bug is trivially reproduced by "xfs_io -c 'fzero -k 0' /some/file". Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- io/prealloc.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/io/prealloc.c b/io/prealloc.c index 9a372bae..6d452354 100644 --- a/io/prealloc.c +++ b/io/prealloc.c @@ -285,18 +285,24 @@ fzero_f( { xfs_flock64_t segment; int mode = FALLOC_FL_ZERO_RANGE; - int index = 1; + int c; - if (strncmp(argv[index], "-k", 3) == 0) { - mode |= FALLOC_FL_KEEP_SIZE; - index++; + while ((c = getopt(argc, argv, "k")) != EOF) { + switch (c) { + case 'k': + mode |= FALLOC_FL_KEEP_SIZE; + break; + default: + command_usage(&fzero_cmd); + } } + if (optind != argc - 2) + return command_usage(&fzero_cmd); - if (!offset_length(argv[index], argv[index + 1], &segment)) + if (!offset_length(argv[optind], argv[optind + 1], &segment)) return 0; - if (fallocate(file->fd, mode, - segment.l_start, segment.l_len)) { + if (fallocate(file->fd, mode, segment.l_start, segment.l_len)) { perror("fallocate"); return 0; } From patchwork Thu Mar 14 21:04:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853605 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF14A17E6 for ; Thu, 14 Mar 2019 21:04:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC45A2A23B for ; Thu, 14 Mar 2019 21:04:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D83882A30E; Thu, 14 Mar 2019 21:04:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7DFF52A791 for ; Thu, 14 Mar 2019 21:04:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727765AbfCNVE4 (ORCPT ); Thu, 14 Mar 2019 17:04:56 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50918 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVEz (ORCPT ); Thu, 14 Mar 2019 17:04:55 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwvZE095467; Thu, 14 Mar 2019 21:04:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=talph3KKQze7s4I5lWnwtF6MHRON37zrgEIm40ervkM=; b=rnBC9IsJvEfA9gjxR3J0xN6aY4/sZGaGi6oliRpGKXVJlwbDAq0cJhz0LJeK5TOXeaJW as0dlxS3ZPdLODUClNcWLNRuK/40Di8Z72tgXTAfaTHEdC0m60uJrGBOFdNw9r0Ta6ZN GkAI+LKQrLwXz+zSM7mS8acevsb+EEhXxpx7f2PZIaYLm6kQKu0YzjKFp3BdMMz97ev8 cTLMuz97sD8msw/SuO877om9y3IzBHZlleycUAdadJyyTlKisnhNt65WhR3doRMseJYs KgD+X9lmkzvaBj1DZwWk2vSnwpVbGQezKmIOBrtQPqy8MKeMNmYmvnsT5Xc+fuiYMmvY Pw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2r44wukfgc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:53 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4qQD032764 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:52 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL4pnO019767; Thu, 14 Mar 2019 21:04:52 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:51 +0000 Subject: [PATCH 10/36] xfs_scrub_all: walk the lsblk device/fs hierarchy correctly From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:04:51 -0700 Message-ID: <155259749135.31886.17642733861513368966.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=9 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=865 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Back when I was designing xfs_scrub_all, I naïvely assumed that the emitted output would always list physical storage before the virtual devices stacked atop it. However, this is not actually true when one omits the "NAME" column, which is crucial to forcing the output (json or otherwise) to capture the block device hierarchy. If the assumption is violated, the program crashes with a python exception. To fix this, force the hierarchal json output and restructure the discovery routines to walk the json object that we receive, from the top (physical devices) downwards to wherever there are live xfs filesystems. Signed-off-by: Darrick J. Wong --- scrub/xfs_scrub_all.in | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/scrub/xfs_scrub_all.in b/scrub/xfs_scrub_all.in index c4e9899d..5b76b49a 100644 --- a/scrub/xfs_scrub_all.in +++ b/scrub/xfs_scrub_all.in @@ -28,9 +28,21 @@ def DEVNULL(): def find_mounts(): '''Map mountpoints to physical disks.''' + def find_xfs_mounts(bdev, fs, lastdisk): + '''Attach lastdisk to each fs found under bdev.''' + if bdev['fstype'] == 'xfs' and bdev['mountpoint'] is not None: + mnt = bdev['mountpoint'] + if mnt in fs: + fs[mnt].add(lastdisk) + else: + fs[mnt] = set([lastdisk]) + if 'children' not in bdev: + return + for child in bdev['children']: + find_xfs_mounts(child, fs, lastdisk) fs = {} - cmd=['lsblk', '-o', 'KNAME,TYPE,FSTYPE,MOUNTPOINT', '-J'] + cmd=['lsblk', '-o', 'NAME,KNAME,TYPE,FSTYPE,MOUNTPOINT', '-J'] result = subprocess.Popen(cmd, stdout=subprocess.PIPE) result.wait() if result.returncode != 0: @@ -38,18 +50,12 @@ def find_mounts(): sarray = [x.decode(sys.stdout.encoding) for x in result.stdout.readlines()] output = ' '.join(sarray) bdevdata = json.loads(output) + # The lsblk output had better be in disks-then-partitions order for bdev in bdevdata['blockdevices']: - if bdev['type'] in ('disk', 'loop'): - lastdisk = bdev['kname'] - if bdev['fstype'] == 'xfs': - mnt = bdev['mountpoint'] - if mnt is None: - continue - if mnt in fs: - fs[mnt].add(lastdisk) - else: - fs[mnt] = set([lastdisk]) + lastdisk = bdev['kname'] + find_xfs_mounts(bdev, fs, lastdisk) + return fs def kill_systemd(unit, proc): From patchwork Thu Mar 14 21:04:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853607 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9C4241515 for ; Thu, 14 Mar 2019 21:05:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83C172A2A0 for ; Thu, 14 Mar 2019 21:05:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 786D32A794; Thu, 14 Mar 2019 21:05:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C6BA2A2A0 for ; Thu, 14 Mar 2019 21:05:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727625AbfCNVFB (ORCPT ); Thu, 14 Mar 2019 17:05:01 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:50680 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVFA (ORCPT ); Thu, 14 Mar 2019 17:05:00 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwjQH102520; Thu, 14 Mar 2019 21:04:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=yyymBHEtM2Fbs7+p2t1cPuOPUXuu5ZNAExXYbOdwEBU=; b=yrffxOP4oXhGENtKPT3NtDHOksh/I4CkcbnLSSGO0oVyAvGTihQHvTVHdhER2KHvUgou +T2OgJn6YcwzDuGB41X5ggt6OFFM71QIA5S9bANVY24kdxsQiUA0y33LWqwk1Sg+zHm/ 21G/jQliKpi5s38JFvpmhWWz2daR/sUidrUccuvWniCVLqfSbQvQK03poS+CHbaToB+s dk9rO1rQ1VN7l7DAkQTzVls1DjGD00k7AD3k1oTLvxP9iacQgujYZPQtjNBKf9fYiE9H p++Xs01/V2ycomW9paWSTpogbh+GXxjgcbsw1aotn/i+Hn2ILWGEgAuYYxicfQLTUhkL jA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2r464ruefb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:58 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL4vYr029259 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:04:58 GMT Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL4vLV000800; Thu, 14 Mar 2019 21:04:57 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:04:57 +0000 Subject: [PATCH 11/36] xfs_scrub_all.timer: activate after most of the system is up From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:04:57 -0700 Message-ID: <155259749738.31886.13894651790702916465.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=9 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=934 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong We really don't want the xfs_scrub_all timer triggering while the system is booting up because not all the mounts will have finished, networking might not be up for reporting, and slowing down bootup annoys people. Therefore, delay the xfs_scrub_all service's activation until after the system has started all the big pieces it's going to start. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/xfs_scrub_all.service.in | 1 + 1 file changed, 1 insertion(+) diff --git a/scrub/xfs_scrub_all.service.in b/scrub/xfs_scrub_all.service.in index 66f82fc7..b1b80da4 100644 --- a/scrub/xfs_scrub_all.service.in +++ b/scrub/xfs_scrub_all.service.in @@ -2,6 +2,7 @@ Description=Online XFS Metadata Check for All Filesystems ConditionACPower=true Documentation=man:xfs_scrub_all(8) +After=paths.target multi-user.target network.target network-online.target systemd-networkd.service NetworkManager.service connman.service [Service] Type=oneshot From patchwork Thu Mar 14 21:05:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853609 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 97C5817E6 for ; Thu, 14 Mar 2019 21:05:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7F4D529DEC for ; Thu, 14 Mar 2019 21:05:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7DBFF2A30E; Thu, 14 Mar 2019 21:05:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E9BC2A79B for ; Thu, 14 Mar 2019 21:05:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727412AbfCNVFI (ORCPT ); Thu, 14 Mar 2019 17:05:08 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:51186 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727985AbfCNVFH (ORCPT ); Thu, 14 Mar 2019 17:05:07 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwuoc095452; Thu, 14 Mar 2019 21:05:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=F3nve9a9otUdCGhNveVCT7V8l/1YFNvIYgABm8e5ebE=; b=0IJrpmQZCj5kGeAq+LyPDJnkh4SjaWZT1NB2N9xilxBREmPonmnHiXvKD6gV5tOawv7C r4olRF2xa4Fdb/xjzI1g5CFDDKU38hC1Anq+p4eFi3Ipsg5Zdos0hM7+ueR2Wd9YFU36 fs8QmdDSNL3uXJifmhrmAYvXhkk9BXM2FHWohHppJtwBNAHeHVTckRsH5OgLtAXj/Bx/ vwOcJ4GQsqXH7Du1MBAPcYjviZU3Hp/OZxAx4TjmDtYsGIZp/e1KUC3Hwha6gYtavviH jt+pS+JaXmBX6pHbLGnF2yROjmIlHTpiYLQyBgxGEZrgokIRQ/Wx4/oK7W9ywfekqjFN FQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2r44wukfj4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:05 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL54iu029748 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:04 GMT Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL547W019968; Thu, 14 Mar 2019 21:05:04 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:03 +0000 Subject: [PATCH 12/36] xfs_scrub: rename the global nr_threads From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:03 -0700 Message-ID: <155259750338.31886.12656336161988116486.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Various functions have nr_threads local variables that shadow the global one. Since the global one forces the number of threads we use, change its name to remove this ambiguity and reflect what it really does. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/common.c | 4 ++-- scrub/disk.c | 4 ++-- scrub/xfs_scrub.c | 6 +++--- scrub/xfs_scrub.h | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/scrub/common.c b/scrub/common.c index 78afc4bf..c877c7c8 100644 --- a/scrub/common.c +++ b/scrub/common.c @@ -231,8 +231,8 @@ unsigned int scrub_nproc( struct scrub_ctx *ctx) { - if (nr_threads) - return nr_threads; + if (force_nr_threads) + return force_nr_threads; return ctx->nr_io_threads; } diff --git a/scrub/disk.c b/scrub/disk.c index 7daa508e..dd109533 100644 --- a/scrub/disk.c +++ b/scrub/disk.c @@ -75,8 +75,8 @@ unsigned int disk_heads( struct disk *disk) { - if (nr_threads) - return nr_threads; + if (force_nr_threads) + return force_nr_threads; return __disk_heads(disk); } diff --git a/scrub/xfs_scrub.c b/scrub/xfs_scrub.c index b8138000..71fc274f 100644 --- a/scrub/xfs_scrub.c +++ b/scrub/xfs_scrub.c @@ -133,7 +133,7 @@ unsigned int bg_mode; int nproc; /* Number of threads we're allowed to use. */ -unsigned int nr_threads; +unsigned int force_nr_threads; /* Verbosity; higher values print more information. */ bool verbose; @@ -589,7 +589,7 @@ main( } break; case 'b': - nr_threads = 1; + force_nr_threads = 1; bg_mode++; break; case 'C': @@ -659,7 +659,7 @@ main( perror("nr_threads"); usage(); } - nr_threads = x; + force_nr_threads = x; } if (optind != argc - 1) diff --git a/scrub/xfs_scrub.h b/scrub/xfs_scrub.h index a961d8fd..a459e4b5 100644 --- a/scrub/xfs_scrub.h +++ b/scrub/xfs_scrub.h @@ -10,7 +10,7 @@ extern char *progname; #define _PATH_PROC_MOUNTS "/proc/mounts" -extern unsigned int nr_threads; +extern unsigned int force_nr_threads; extern unsigned int bg_mode; extern unsigned int debug; extern int nproc; From patchwork Thu Mar 14 21:05:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853611 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 873AB17E6 for ; Thu, 14 Mar 2019 21:05:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D82A29E7B for ; Thu, 14 Mar 2019 21:05:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B0342A24A; Thu, 14 Mar 2019 21:05:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 15E7C2A78A for ; Thu, 14 Mar 2019 21:05:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727760AbfCNVFN (ORCPT ); Thu, 14 Mar 2019 17:05:13 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:50920 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727749AbfCNVFN (ORCPT ); Thu, 14 Mar 2019 17:05:13 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwqLK102588; Thu, 14 Mar 2019 21:05:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Yf4cp8/bOdGwF9rVbp/DETVmC4BagKq7/Lrfx1zF9lE=; b=vL+xZY+Wvk0N4INmr5d7l008zmrF4lXKtjKGjssxR2YqWSgntTfFSxRFa97si/LMnnBq MK9xXReWOWCDmk1mJfNHsg9U9VEC3Wi61Q3fRhjyoQDtf9gU0T/S7FQJ1a0KH/NEUkJT 6pnEH9JcDFNRA49xgt9jrJhdQ6MtjGzsVjjC0YqjAD+2GmYDInF8jrSFZQAuDZWAfhiQ ray0o16D77DoEt635JnFQCcH2Vdacx1UmStRYpxhdAK4iw83Z1G/k8unfDz+0pZO9zv+ yEAM/AvFO36j92u05ZbBKvod1aNSDtjb9gBVLjDFMZnr4s/xqITPBqlJWVqoeZ7GnKxe 6Q== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2r464ruege-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:10 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5AbP006014 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:10 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5Aaj006984; Thu, 14 Mar 2019 21:05:10 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:09 +0000 Subject: [PATCH 13/36] xfs_scrub: use datadev parallelization estimates for thread count From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:09 -0700 Message-ID: <155259750953.31886.7244219282258067422.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong During phases 2-5, xfs_scrub should estimate the level of parallelization possible on the data device to determine the number of threads spawned to scrub filesystem metadata, not just blindly using the number of CPUs. This avoids flooding non-rotational storage with random reads, which totally destroys performance and makes scrub runtimes higher. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/phase1.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/scrub/phase1.c b/scrub/phase1.c index 2113014b..6b472147 100644 --- a/scrub/phase1.c +++ b/scrub/phase1.c @@ -109,13 +109,6 @@ _("Must be root to run scrub.")); return false; } - ctx->nr_io_threads = nproc; - if (verbose) { - fprintf(stdout, _("%s: using %d threads to scrub.\n"), - ctx->mntpoint, scrub_nproc(ctx)); - fflush(stdout); - } - if (!platform_test_xfs_fd(ctx->mnt_fd)) { str_info(ctx, ctx->mntpoint, _("Does not appear to be an XFS filesystem!")); @@ -193,6 +186,13 @@ _("Unable to find realtime device path.")); return false; } + ctx->nr_io_threads = disk_heads(ctx->datadev); + if (verbose) { + fprintf(stdout, _("%s: using %d threads to scrub.\n"), + ctx->mntpoint, scrub_nproc(ctx)); + fflush(stdout); + } + if (ctx->fsinfo.fs_log) { ctx->logdev = disk_open(ctx->fsinfo.fs_log); if (error) { From patchwork Thu Mar 14 21:05:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853613 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EFAF91515 for ; Thu, 14 Mar 2019 21:05:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D6B742A791 for ; Thu, 14 Mar 2019 21:05:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D52EC2A794; Thu, 14 Mar 2019 21:05:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 327412A791 for ; Thu, 14 Mar 2019 21:05:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727991AbfCNVFT (ORCPT ); Thu, 14 Mar 2019 17:05:19 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:51082 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727366AbfCNVFT (ORCPT ); Thu, 14 Mar 2019 17:05:19 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwkAk102528; Thu, 14 Mar 2019 21:05:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Zth1NGGSsCVwHjnGePldTcS74IF04vfaJ48uMpMQ23E=; b=pQlDC30EqceOlKVyhBxlC6URY10vcJMhmu8W59mokMjbVuz/+yUlqD+MoZQNR9BRFbs4 q0st+Lmj8WXv/K1Z/2SNH/mweP2lc9qJBN4CgEXAHTyrrqx9gzwe5s0kWGkFzAmYrshM y3LtUCbsGDfVo37gRPJMGQi36SJA021H2sf3SPI6DhiVuGLooJ+9/NTLYoMx3si+2FWy GJf1KuZrOK6l74uGv3PuS9BhvXnwooPy8L/l3CYP9/ayE1jXKDMFyCUdUkMhoGGdKhmm hHSFsJd1T37wQ7kedTZu0kSrv+R8XSeqwsDKVGCx78JjVg2doNj9yL+4cu1h85BBovga 8A== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2r464rueh7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:17 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5GoU018694 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:16 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL5GMt001065; Thu, 14 Mar 2019 21:05:16 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:15 +0000 Subject: [PATCH 14/36] xfs_scrub: don't expose internal pool state From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:15 -0700 Message-ID: <155259751554.31886.1451967052557022474.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In xfs_scrub, the read/verify pool tries to coalesce the media verification requests into a smaller number of large IOs. There's no need to force callers to keep track of this internal state, so just move all that into read_verify.c. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/phase6.c | 23 +++++++---------------- scrub/read_verify.c | 40 ++++++++++++++++++++++++++++++++++------ scrub/read_verify.h | 16 ++++------------ 3 files changed, 45 insertions(+), 34 deletions(-) diff --git a/scrub/phase6.c b/scrub/phase6.c index ead48d77..fe121769 100644 --- a/scrub/phase6.c +++ b/scrub/phase6.c @@ -9,7 +9,6 @@ #include #include "handle.h" #include "path.h" -#include "ptvar.h" #include "workqueue.h" #include "xfs_scrub.h" #include "common.h" @@ -290,7 +289,6 @@ xfs_report_verify_errors( struct xfs_verify_extent { struct read_verify_pool *readverify; - struct ptvar *rvstate; struct bitmap *d_bad; /* bytes */ struct bitmap *r_bad; /* bytes */ }; @@ -424,13 +422,13 @@ xfs_check_rmap( /* Schedule the read verify command for (eventual) running. */ disk = xfs_dev_to_disk(ctx, map->fmr_device); - read_verify_schedule_io(ve->readverify, ptvar_get(ve->rvstate), disk, - map->fmr_physical, map->fmr_length, ve); + read_verify_schedule_io(ve->readverify, disk, map->fmr_physical, + map->fmr_length, ve); out: /* Is this the last extent? Fire off the read. */ if (map->fmr_flags & FMR_OF_LAST) - read_verify_force_io(ve->readverify, ptvar_get(ve->rvstate)); + read_verify_force_io(ve->readverify); return true; } @@ -450,16 +448,10 @@ xfs_scan_blocks( struct xfs_verify_extent ve; bool moveon; - ve.rvstate = ptvar_init(scrub_nproc(ctx), sizeof(struct read_verify)); - if (!ve.rvstate) { - str_errno(ctx, ctx->mntpoint); - return false; - } - moveon = bitmap_init(&ve.d_bad); if (!moveon) { str_errno(ctx, ctx->mntpoint); - goto out_ve; + goto out; } moveon = bitmap_init(&ve.r_bad); @@ -469,7 +461,8 @@ xfs_scan_blocks( } ve.readverify = read_verify_pool_init(ctx, ctx->geo.blocksize, - xfs_check_rmap_ioerr, disk_heads(ctx->datadev)); + xfs_check_rmap_ioerr, disk_heads(ctx->datadev), + scrub_nproc(ctx)); if (!ve.readverify) { moveon = false; str_info(ctx, ctx->mntpoint, @@ -489,7 +482,6 @@ _("Could not create media verifier.")); bitmap_free(&ve.r_bad); bitmap_free(&ve.d_bad); - ptvar_free(ve.rvstate); return moveon; out_pool: @@ -498,8 +490,7 @@ _("Could not create media verifier.")); bitmap_free(&ve.r_bad); out_dbad: bitmap_free(&ve.d_bad); -out_ve: - ptvar_free(ve.rvstate); +out: return moveon; } diff --git a/scrub/read_verify.c b/scrub/read_verify.c index 75cb53ca..b5774736 100644 --- a/scrub/read_verify.c +++ b/scrub/read_verify.c @@ -7,6 +7,7 @@ #include #include #include +#include "ptvar.h" #include "workqueue.h" #include "path.h" #include "xfs_scrub.h" @@ -36,22 +37,40 @@ /* Tolerate 64k holes in adjacent read verify requests. */ #define RVP_IO_BATCH_LOCALITY (65536) +struct read_verify { + void *io_end_arg; + struct disk *io_disk; + uint64_t io_start; /* bytes */ + uint64_t io_length; /* bytes */ +}; + struct read_verify_pool { struct workqueue wq; /* thread pool */ struct scrub_ctx *ctx; /* scrub context */ void *readbuf; /* read buffer */ struct ptcounter *verified_bytes; + struct ptvar *rvstate; /* combines read requests */ read_verify_ioerr_fn_t ioerr_fn; /* io error callback */ size_t miniosz; /* minimum io size, bytes */ }; -/* Create a thread pool to run read verifiers. */ +/* + * Create a thread pool to run read verifiers. + * + * @miniosz is the minimum size of an IO to expect (in bytes). + * @ioerr_fn will be called when IO errors occur. + * @nproc is the maximum number of verify requests that may be sent to a disk + * at any given time. + * @submitter_threads is the number of threads that may be sending verify + * requests at any given time. + */ struct read_verify_pool * read_verify_pool_init( struct scrub_ctx *ctx, size_t miniosz, read_verify_ioerr_fn_t ioerr_fn, - unsigned int nproc) + unsigned int nproc, + unsigned int submitter_threads) { struct read_verify_pool *rvp; bool ret; @@ -71,14 +90,20 @@ read_verify_pool_init( rvp->miniosz = miniosz; rvp->ctx = ctx; rvp->ioerr_fn = ioerr_fn; + rvp->rvstate = ptvar_init(submitter_threads, + sizeof(struct read_verify)); + if (rvp->rvstate == NULL) + goto out_counter; /* Run in the main thread if we only want one thread. */ if (nproc == 1) nproc = 0; ret = workqueue_create(&rvp->wq, (struct xfs_mount *)rvp, nproc); if (ret) - goto out_counter; + goto out_rvstate; return rvp; +out_rvstate: + ptvar_free(rvp->rvstate); out_counter: ptcounter_free(rvp->verified_bytes); out_buf: @@ -101,6 +126,7 @@ void read_verify_pool_destroy( struct read_verify_pool *rvp) { + ptvar_free(rvp->rvstate); ptcounter_free(rvp->verified_bytes); free(rvp->readbuf); free(rvp); @@ -186,16 +212,17 @@ _("Could not queue read-verify work.")); bool read_verify_schedule_io( struct read_verify_pool *rvp, - struct read_verify *rv, struct disk *disk, uint64_t start, uint64_t length, void *end_arg) { + struct read_verify *rv; uint64_t req_end; uint64_t rv_end; assert(rvp->readbuf); + rv = ptvar_get(rvp->rvstate); req_end = start + length; rv_end = rv->io_start + rv->io_length; @@ -229,12 +256,13 @@ read_verify_schedule_io( /* Force any stashed IOs into the verifier. */ bool read_verify_force_io( - struct read_verify_pool *rvp, - struct read_verify *rv) + struct read_verify_pool *rvp) { + struct read_verify *rv; bool moveon; assert(rvp->readbuf); + rv = ptvar_get(rvp->rvstate); if (rv->io_length == 0) return true; diff --git a/scrub/read_verify.h b/scrub/read_verify.h index 38f1cd1a..1e7fd83f 100644 --- a/scrub/read_verify.h +++ b/scrub/read_verify.h @@ -16,21 +16,13 @@ typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx, struct read_verify_pool *read_verify_pool_init(struct scrub_ctx *ctx, size_t miniosz, read_verify_ioerr_fn_t ioerr_fn, - unsigned int nproc); + unsigned int nproc, unsigned int submitter_threads); void read_verify_pool_flush(struct read_verify_pool *rvp); void read_verify_pool_destroy(struct read_verify_pool *rvp); -struct read_verify { - void *io_end_arg; - struct disk *io_disk; - uint64_t io_start; /* bytes */ - uint64_t io_length; /* bytes */ -}; - -bool read_verify_schedule_io(struct read_verify_pool *rvp, - struct read_verify *rv, struct disk *disk, uint64_t start, - uint64_t length, void *end_arg); -bool read_verify_force_io(struct read_verify_pool *rvp, struct read_verify *rv); +bool read_verify_schedule_io(struct read_verify_pool *rvp, struct disk *disk, + uint64_t start, uint64_t length, void *end_arg); +bool read_verify_force_io(struct read_verify_pool *rvp); uint64_t read_verify_bytes(struct read_verify_pool *rvp); #endif /* XFS_SCRUB_READ_VERIFY_H_ */ From patchwork Thu Mar 14 21:05:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853615 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C17941515 for ; Thu, 14 Mar 2019 21:05:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A6EC42A798 for ; Thu, 14 Mar 2019 21:05:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9ADB82A79B; Thu, 14 Mar 2019 21:05:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF9902A799 for ; Thu, 14 Mar 2019 21:05:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727843AbfCNVF2 (ORCPT ); Thu, 14 Mar 2019 17:05:28 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:51196 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727366AbfCNVF2 (ORCPT ); Thu, 14 Mar 2019 17:05:28 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwnZ8102560; Thu, 14 Mar 2019 21:05:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=fpMw72hpNg0tNBq0EK5RxStR0D9XlngjiDzccHgzFLw=; b=k9StRfCFC64dj+iVvtRx+wTjqfQRZokyfXo0DNiGbuXHXM0n49QjIzZU5gTciM/GhXTd M2N44pGvmrgqmCihIIJ4O92dIt5qZyFwI+Bpg8uPlarU8R91eoXdIx6GcYHOQfW6vVMi fV+etNm2EzcBxaM8iVUd6QPQyg6ChAf+xCESUe91xyst9VIyZYOdzE9kEKzOEsZlD2FQ 3s+w5h6STtGswDmyD2/6Rn5PjhFcd0CHqFN9mIWYiQXbtbxEtwmlcrsPzI9zwXpcF2zJ 1mntx4Ux7OGTepbsD74HntKN1EkxjZioyn06GKUjFiMuMbVlb32tfgGfBWoAqLmjwLET zQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2r464ruehv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:25 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5O54003277 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:24 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5NJq003322; Thu, 14 Mar 2019 21:05:23 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:23 +0000 Subject: [PATCH 15/36] xfs_scrub: one read/verify pool per disk From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:21 -0700 Message-ID: <155259752166.31886.13004303237464337185.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Simplify the read/verify pool code further by creating one pool per disk. This enables us to tailor the concurrency levels of each disk to that specific disk so that if we have a mixed hdd/ssd environment we don't flood the hdd with a lot of requests. Signed-off-by: Darrick J. Wong --- scrub/phase6.c | 110 ++++++++++++++++++++++++++++++++++++--------------- scrub/read_verify.c | 29 ++++++------- scrub/read_verify.h | 10 +++-- 3 files changed, 98 insertions(+), 51 deletions(-) diff --git a/scrub/phase6.c b/scrub/phase6.c index fe121769..ccb795ab 100644 --- a/scrub/phase6.c +++ b/scrub/phase6.c @@ -33,18 +33,29 @@ * and report the paths of the now corrupt files. */ +/* Verify disk blocks with GETFSMAP */ + +struct xfs_verify_extent { + struct read_verify_pool *rvp_data; + struct read_verify_pool *rvp_log; + struct read_verify_pool *rvp_realtime; + struct bitmap *d_bad; /* bytes */ + struct bitmap *r_bad; /* bytes */ +}; + /* Find the fd for a given device identifier. */ -static struct disk * -xfs_dev_to_disk( - struct scrub_ctx *ctx, - dev_t dev) +static struct read_verify_pool * +xfs_dev_to_pool( + struct scrub_ctx *ctx, + struct xfs_verify_extent *ve, + dev_t dev) { if (dev == ctx->fsinfo.fs_datadev) - return ctx->datadev; + return ve->rvp_data; else if (dev == ctx->fsinfo.fs_logdev) - return ctx->logdev; + return ve->rvp_log; else if (dev == ctx->fsinfo.fs_rtdev) - return ctx->rtdev; + return ve->rvp_realtime; abort(); } @@ -285,14 +296,6 @@ xfs_report_verify_errors( return xfs_scan_all_inodes(ctx, xfs_report_verify_inode, &vei); } -/* Verify disk blocks with GETFSMAP */ - -struct xfs_verify_extent { - struct read_verify_pool *readverify; - struct bitmap *d_bad; /* bytes */ - struct bitmap *r_bad; /* bytes */ -}; - /* Report an IO error resulting from read-verify based off getfsmap. */ static bool xfs_check_rmap_error_report( @@ -393,7 +396,9 @@ xfs_check_rmap( void *arg) { struct xfs_verify_extent *ve = arg; - struct disk *disk; + struct read_verify_pool *rvp; + + rvp = xfs_dev_to_pool(ctx, ve, map->fmr_device); dbg_printf("rmap dev %d:%d phys %"PRIu64" owner %"PRId64 " offset %"PRIu64" len %"PRIu64" flags 0x%x\n", @@ -420,19 +425,32 @@ xfs_check_rmap( /* XXX: Filter out directory data blocks. */ /* Schedule the read verify command for (eventual) running. */ - disk = xfs_dev_to_disk(ctx, map->fmr_device); - - read_verify_schedule_io(ve->readverify, disk, map->fmr_physical, - map->fmr_length, ve); + read_verify_schedule_io(rvp, map->fmr_physical, map->fmr_length, ve); out: /* Is this the last extent? Fire off the read. */ if (map->fmr_flags & FMR_OF_LAST) - read_verify_force_io(ve->readverify); + read_verify_force_io(rvp); return true; } +/* Wait for read/verify actions to finish, then return # bytes checked. */ +static uint64_t +clean_pool( + struct read_verify_pool *rvp) +{ + uint64_t ret; + + if (!rvp) + return 0; + + read_verify_pool_flush(rvp); + ret += read_verify_bytes(rvp); + read_verify_pool_destroy(rvp); + return ret; +} + /* * Read verify all the file data blocks in a filesystem. Since XFS doesn't * do data checksums, we trust that the underlying storage will pass back @@ -445,7 +463,7 @@ bool xfs_scan_blocks( struct scrub_ctx *ctx) { - struct xfs_verify_extent ve; + struct xfs_verify_extent ve = { NULL }; bool moveon; moveon = bitmap_init(&ve.d_bad); @@ -460,21 +478,43 @@ xfs_scan_blocks( goto out_dbad; } - ve.readverify = read_verify_pool_init(ctx, ctx->geo.blocksize, - xfs_check_rmap_ioerr, disk_heads(ctx->datadev), + ve.rvp_data = read_verify_pool_init(ctx, ctx->datadev, + ctx->geo.blocksize, xfs_check_rmap_ioerr, scrub_nproc(ctx)); - if (!ve.readverify) { + if (!ve.rvp_data) { moveon = false; str_info(ctx, ctx->mntpoint, -_("Could not create media verifier.")); +_("Could not create data device media verifier.")); goto out_rbad; } + if (ctx->logdev) { + ve.rvp_log = read_verify_pool_init(ctx, ctx->logdev, + ctx->geo.blocksize, xfs_check_rmap_ioerr, + scrub_nproc(ctx)); + if (!ve.rvp_log) { + moveon = false; + str_info(ctx, ctx->mntpoint, + _("Could not create log device media verifier.")); + goto out_datapool; + } + } + if (ctx->rtdev) { + ve.rvp_realtime = read_verify_pool_init(ctx, ctx->rtdev, + ctx->geo.blocksize, xfs_check_rmap_ioerr, + scrub_nproc(ctx)); + if (!ve.rvp_realtime) { + moveon = false; + str_info(ctx, ctx->mntpoint, + _("Could not create realtime device media verifier.")); + goto out_logpool; + } + } moveon = xfs_scan_all_spacemaps(ctx, xfs_check_rmap, &ve); if (!moveon) - goto out_pool; - read_verify_pool_flush(ve.readverify); - ctx->bytes_checked += read_verify_bytes(ve.readverify); - read_verify_pool_destroy(ve.readverify); + goto out_rtpool; + ctx->bytes_checked += clean_pool(ve.rvp_data); + ctx->bytes_checked += clean_pool(ve.rvp_log); + ctx->bytes_checked += clean_pool(ve.rvp_realtime); /* Scan the whole dir tree to see what matches the bad extents. */ if (!bitmap_empty(ve.d_bad) || !bitmap_empty(ve.r_bad)) @@ -484,8 +524,14 @@ _("Could not create media verifier.")); bitmap_free(&ve.d_bad); return moveon; -out_pool: - read_verify_pool_destroy(ve.readverify); +out_rtpool: + if (ve.rvp_realtime) + read_verify_pool_destroy(ve.rvp_realtime); +out_logpool: + if (ve.rvp_log) + read_verify_pool_destroy(ve.rvp_log); +out_datapool: + read_verify_pool_destroy(ve.rvp_data); out_rbad: bitmap_free(&ve.r_bad); out_dbad: diff --git a/scrub/read_verify.c b/scrub/read_verify.c index b5774736..4a9b91f2 100644 --- a/scrub/read_verify.c +++ b/scrub/read_verify.c @@ -50,6 +50,7 @@ struct read_verify_pool { void *readbuf; /* read buffer */ struct ptcounter *verified_bytes; struct ptvar *rvstate; /* combines read requests */ + struct disk *disk; /* which disk? */ read_verify_ioerr_fn_t ioerr_fn; /* io error callback */ size_t miniosz; /* minimum io size, bytes */ }; @@ -57,19 +58,18 @@ struct read_verify_pool { /* * Create a thread pool to run read verifiers. * + * @disk is the disk we want to verify. * @miniosz is the minimum size of an IO to expect (in bytes). * @ioerr_fn will be called when IO errors occur. - * @nproc is the maximum number of verify requests that may be sent to a disk - * at any given time. * @submitter_threads is the number of threads that may be sending verify * requests at any given time. */ struct read_verify_pool * read_verify_pool_init( struct scrub_ctx *ctx, + struct disk *disk, size_t miniosz, read_verify_ioerr_fn_t ioerr_fn, - unsigned int nproc, unsigned int submitter_threads) { struct read_verify_pool *rvp; @@ -89,6 +89,7 @@ read_verify_pool_init( goto out_buf; rvp->miniosz = miniosz; rvp->ctx = ctx; + rvp->disk = disk; rvp->ioerr_fn = ioerr_fn; rvp->rvstate = ptvar_init(submitter_threads, sizeof(struct read_verify)); @@ -97,7 +98,8 @@ read_verify_pool_init( /* Run in the main thread if we only want one thread. */ if (nproc == 1) nproc = 0; - ret = workqueue_create(&rvp->wq, (struct xfs_mount *)rvp, nproc); + ret = workqueue_create(&rvp->wq, (struct xfs_mount *)rvp, + disk_heads(disk)); if (ret) goto out_rvstate; return rvp; @@ -150,17 +152,16 @@ read_verify( rvp = (struct read_verify_pool *)wq->wq_ctx; while (rv->io_length > 0) { len = min(rv->io_length, RVP_IO_MAX_SIZE); - dbg_printf("diskverify %d %"PRIu64" %zu\n", rv->io_disk->d_fd, - rv->io_start, len); - sz = disk_read_verify(rv->io_disk, rvp->readbuf, + dbg_printf("diskverify %d %"PRIu64" %zu\n", rvp->disk->d_fd, rv->io_start, len); + sz = disk_read_verify(rvp->disk, rvp->readbuf, rv->io_start, + len); if (sz < 0) { dbg_printf("IOERR %d %"PRIu64" %zu\n", - rv->io_disk->d_fd, - rv->io_start, len); + rvp->disk->d_fd, rv->io_start, len); /* IO error, so try the next logical block. */ len = rvp->miniosz; - rvp->ioerr_fn(rvp->ctx, rv->io_disk, rv->io_start, len, + rvp->ioerr_fn(rvp->ctx, rvp->disk, rv->io_start, len, errno, rv->io_end_arg); } @@ -184,11 +185,11 @@ read_verify_queue( bool ret; dbg_printf("verify fd %d start %"PRIu64" len %"PRIu64"\n", - rv->io_disk->d_fd, rv->io_start, rv->io_length); + rvp->disk->d_fd, rv->io_start, rv->io_length); tmp = malloc(sizeof(struct read_verify)); if (!tmp) { - rvp->ioerr_fn(rvp->ctx, rv->io_disk, rv->io_start, + rvp->ioerr_fn(rvp->ctx, rvp->disk, rv->io_start, rv->io_length, errno, rv->io_end_arg); return true; } @@ -212,7 +213,6 @@ _("Could not queue read-verify work.")); bool read_verify_schedule_io( struct read_verify_pool *rvp, - struct disk *disk, uint64_t start, uint64_t length, void *end_arg) @@ -231,7 +231,7 @@ read_verify_schedule_io( * reporting is the same, and the two extents are close, * we can combine them. */ - if (rv->io_length > 0 && disk == rv->io_disk && + if (rv->io_length > 0 && end_arg == rv->io_end_arg && ((start >= rv->io_start && start <= rv_end + RVP_IO_BATCH_LOCALITY) || (rv->io_start >= start && @@ -244,7 +244,6 @@ read_verify_schedule_io( return read_verify_queue(rvp, rv); /* Stash the new IO. */ - rv->io_disk = disk; rv->io_start = start; rv->io_length = length; rv->io_end_arg = end_arg; diff --git a/scrub/read_verify.h b/scrub/read_verify.h index 1e7fd83f..5fabe5e0 100644 --- a/scrub/read_verify.h +++ b/scrub/read_verify.h @@ -8,6 +8,7 @@ struct scrub_ctx; struct read_verify_pool; +struct disk; /* Function called when an IO error happens. */ typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx, @@ -15,13 +16,14 @@ typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx, int error, void *arg); struct read_verify_pool *read_verify_pool_init(struct scrub_ctx *ctx, - size_t miniosz, read_verify_ioerr_fn_t ioerr_fn, - unsigned int nproc, unsigned int submitter_threads); + struct disk *disk, size_t miniosz, + read_verify_ioerr_fn_t ioerr_fn, + unsigned int submitter_threads); void read_verify_pool_flush(struct read_verify_pool *rvp); void read_verify_pool_destroy(struct read_verify_pool *rvp); -bool read_verify_schedule_io(struct read_verify_pool *rvp, struct disk *disk, - uint64_t start, uint64_t length, void *end_arg); +bool read_verify_schedule_io(struct read_verify_pool *rvp, uint64_t start, + uint64_t length, void *end_arg); bool read_verify_force_io(struct read_verify_pool *rvp); uint64_t read_verify_bytes(struct read_verify_pool *rvp); From patchwork Thu Mar 14 21:05:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853617 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF0E917E6 for ; Thu, 14 Mar 2019 21:05:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A97D82A24E for ; Thu, 14 Mar 2019 21:05:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A803F2A7AB; Thu, 14 Mar 2019 21:05:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 57B622A24E for ; Thu, 14 Mar 2019 21:05:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727388AbfCNVFe (ORCPT ); Thu, 14 Mar 2019 17:05:34 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:51296 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727366AbfCNVFd (ORCPT ); Thu, 14 Mar 2019 17:05:33 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwtAU102632; Thu, 14 Mar 2019 21:05:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=K5sMzxySs0XG7290aEJjjIBFxnryZu0HEpPADNhTox8=; b=tWOF0GIeoMXSPSHxBP8v4CJtU/CGChH+ZjduWiJCqZE0krYXFICGFCj35WXC40WAzFU4 o2UZ5EnxBTU9Z6GznST5ok2yg6mGzgwO02mbAVxJjUTnYpq/5kKQujQZ+b8+x2Dflz0A jfaY+GmxJzfSWTKrgctXcKZzA121DHSoL1bYAfImFkHRB/RWsemhRXGeF6AvzsLLZSQM 9HSeMK//uJytbw7PESZzSENs+O67VIYYkFg0w/SwtaMaZ8BvKgoeedR37clFFCSbbuk+ pPA881h8g+PvgJpPPiYMi54jzXurgGwPm++0ByTXnRE5SAzb/joXuXpkML2pEj3ONl+L lw== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2r464ruej6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:31 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5Uw2019658 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:30 GMT Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL5Tfs020159; Thu, 14 Mar 2019 21:05:30 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:29 +0000 Subject: [PATCH 16/36] xfs_scrub: don't close mnt_fd when mnt_fd open fails From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:29 -0700 Message-ID: <155259752927.31886.10173767197190539382.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=976 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong If we fail to open the mountpoint during phase 1 of scrub, don't bother trying to close the file descriptor since it's silly to spray error messages about that. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/phase1.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/scrub/phase1.c b/scrub/phase1.c index 6b472147..04a5f4a9 100644 --- a/scrub/phase1.c +++ b/scrub/phase1.c @@ -59,9 +59,11 @@ xfs_cleanup_fs( if (ctx->datadev) disk_close(ctx->datadev); fshandle_destroy(); - error = close(ctx->mnt_fd); - if (error) - str_errno(ctx, _("closing mountpoint fd")); + if (ctx->mnt_fd >= 0) { + error = close(ctx->mnt_fd); + if (error) + str_errno(ctx, _("closing mountpoint fd")); + } fs_table_destroy(); return true; From patchwork Thu Mar 14 21:05:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853619 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 639E717E6 for ; Thu, 14 Mar 2019 21:05:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4C22D2A7A6 for ; Thu, 14 Mar 2019 21:05:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4899C2A790; Thu, 14 Mar 2019 21:05:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C87172A23B for ; Thu, 14 Mar 2019 21:05:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727749AbfCNVFj (ORCPT ); Thu, 14 Mar 2019 17:05:39 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:51438 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727366AbfCNVFj (ORCPT ); Thu, 14 Mar 2019 17:05:39 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwvMg102645; Thu, 14 Mar 2019 21:05:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=1Y+TgiHXC4enoSZLJFqz6aZ+nZOyO9wBGg3LPdfcmSw=; b=LdBpQnHwKmZXYtXsEyiZnMpM5cFepb9C/B4vWgKnAcFavUBMCVdf/uyl94tnuGue2u/F J7TXeQhhefLqHRHRQFzVVM7+1zsnyTxrnHlJCqB7Jjv4saOhU/LPdVjE01LvzZ1VPgia bQVfVfeYJtxGkSNopZKOkuLklRWToWit/TE9S3t0h1FsFuYXXbobFXSanEvNuPR/YoMH 6DZFbh1heeK3QB4jGzgTbXxqMJEfbqDxgX0RsXdq2R0QvQQdp3sKQCS1rnYyGdnwdCXG +iMTVdQ5kQBILgE6lfjisK6qwNpvqYo2V0TRlxkjkL/Cu+DrcRidNgBq9IahjYN739GE Dw== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2r464ruejg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:36 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5ZcF019905 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:36 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL5ZqV001182; Thu, 14 Mar 2019 21:05:35 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:35 +0000 Subject: [PATCH 17/36] xfs_scrub: check label for misleading characters From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:35 -0700 Message-ID: <155259753535.31886.7933186379056372168.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Make sure that we can retrieve the label and that it doesn't contain anything potentially misleading. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/phase5.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ scrub/unicrash.c | 24 ++++++++++++++++++++++++ scrub/unicrash.h | 5 +++++ 3 files changed, 83 insertions(+) diff --git a/scrub/phase5.c b/scrub/phase5.c index 6ffcec2d..49886e6f 100644 --- a/scrub/phase5.c +++ b/scrub/phase5.c @@ -11,6 +11,7 @@ #ifdef HAVE_LIBATTR # include #endif +#include #include "handle.h" #include "list.h" #include "path.h" @@ -282,6 +283,55 @@ xfs_scrub_connections( return *pmoveon ? 0 : XFS_ITERATE_INODES_ABORT; } +#ifndef FS_IOC_GETFSLABEL +# define FSLABEL_MAX 256 +# define FS_IOC_GETFSLABEL _IOR(0x94, 49, char[FSLABEL_MAX]) +#endif /* FS_IOC_GETFSLABEL */ + +/* + * Check the filesystem label for Unicode normalization problems or misleading + * sequences. + */ +static bool +xfs_scrub_fs_label( + struct scrub_ctx *ctx) +{ + char label[FSLABEL_MAX]; + struct unicrash *uc = NULL; + bool moveon = true; + int error; + + moveon = unicrash_fs_label_init(&uc, ctx); + if (!moveon) + return false; + + /* Retrieve label; quietly bail if we don't support that. */ + error = ioctl(ctx->mnt_fd, FS_IOC_GETFSLABEL, &label); + if (error) { + if (errno != EOPNOTSUPP && errno != ENOTTY) { + moveon = false; + perror(ctx->mntpoint); + } + goto out; + } + + /* Ignore empty labels. */ + if (label[0] == 0) + goto out; + + /* Otherwise check for weirdness. */ + if (uc) + moveon = unicrash_check_fs_label(uc, ctx->mntpoint, label); + else + moveon = xfs_scrub_check_name(ctx, ctx->mntpoint, + _("filesystem label"), label); + if (!moveon) + goto out; +out: + unicrash_free(uc); + return moveon; +} + /* Check directory connectivity. */ bool xfs_scan_connections( @@ -296,6 +346,10 @@ _("Filesystem has errors, skipping connectivity checks.")); return true; } + moveon = xfs_scrub_fs_label(ctx); + if (!moveon) + return false; + ret = xfs_scan_all_inodes(ctx, xfs_scrub_connections, &moveon); if (!ret) moveon = false; diff --git a/scrub/unicrash.c b/scrub/unicrash.c index a95fc305..121eedbc 100644 --- a/scrub/unicrash.c +++ b/scrub/unicrash.c @@ -465,6 +465,15 @@ unicrash_xattr_init( is_only_root_writable(bstat)); } +/* Initialize the collision detector for a filesystem label. */ +bool +unicrash_fs_label_init( + struct unicrash **ucp, + struct scrub_ctx *ctx) +{ + return unicrash_init(ucp, ctx, false, 16, true); +} + /* Free the crash detector. */ void unicrash_free( @@ -698,3 +707,18 @@ unicrash_check_xattr_name( return __unicrash_check_name(uc, descr, _("extended attribute"), attrname, 0); } + +/* + * Check the fs label for unicode normalization problems or misleading bits. + */ +bool +unicrash_check_fs_label( + struct unicrash *uc, + const char *descr, + const char *label) +{ + if (!uc) + return true; + return __unicrash_check_name(uc, descr, _("filesystem label"), + label, 0); +} diff --git a/scrub/unicrash.h b/scrub/unicrash.h index 7d7276a8..85fcabc6 100644 --- a/scrub/unicrash.h +++ b/scrub/unicrash.h @@ -17,17 +17,22 @@ bool unicrash_dir_init(struct unicrash **ucp, struct scrub_ctx *ctx, struct xfs_bstat *bstat); bool unicrash_xattr_init(struct unicrash **ucp, struct scrub_ctx *ctx, struct xfs_bstat *bstat); +bool unicrash_fs_label_init(struct unicrash **ucp, struct scrub_ctx *ctx); void unicrash_free(struct unicrash *uc); bool unicrash_check_dir_name(struct unicrash *uc, const char *descr, struct dirent *dirent); bool unicrash_check_xattr_name(struct unicrash *uc, const char *descr, const char *attrname); +bool unicrash_check_fs_label(struct unicrash *uc, const char *descr, + const char *label); #else # define unicrash_dir_init(u, c, b) (true) # define unicrash_xattr_init(u, c, b) (true) +# define unicrash_label_init(u, c) (true) # define unicrash_free(u) do {(u) = (u);} while (0) # define unicrash_check_dir_name(u, d, n) (true) # define unicrash_check_xattr_name(u, d, n) (true) +# define unicrash_check_fs_label(u, d, n) (true) #endif /* HAVE_LIBICU */ #endif /* XFS_SCRUB_UNICRASH_H_ */ From patchwork Thu Mar 14 21:05:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853621 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6A0D81515 for ; Thu, 14 Mar 2019 21:05:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 51EC62A78A for ; Thu, 14 Mar 2019 21:05:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 504EE2A791; Thu, 14 Mar 2019 21:05:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D1C152A78A for ; Thu, 14 Mar 2019 21:05:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727915AbfCNVFu (ORCPT ); Thu, 14 Mar 2019 17:05:50 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52078 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727366AbfCNVFu (ORCPT ); Thu, 14 Mar 2019 17:05:50 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwmq2095332; Thu, 14 Mar 2019 21:05:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=5EVQQmNeXAFWgDGC2nf11b4QjXqlFVibvUxgUiYJkjA=; b=1IhZ46N2zDxxZ2Cz2gh4mA9DMqPlPA1XsmWygdt8QN0V2hL1vTs9D49bDGI7GDSKWOWa mJKSbkcVIvCzxYbYkkh12uz8tPlaASC0Isim/rgDxOOyri1XSv6LRcI5zBj+RyXlMQBv f5nVNZ9LwD7cBlw2MYmEuoYVrNDARJmPuRFNR+Ia7frDwBII81Bi6N4a/JBfAYPlJJLB UNcfYh68QT7+QVKhA9p/7gJ1wsBAy2R3b4fkXtdA6gGgqxPjhj1PEFzn9Rk4nPRqxYDy LJ/zSbPVCYEZo3eQBAz/QtGTd8r+FfKI6cUUxNV2CjZLGSiaZOenzUnoiWjN91OErMjs VQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2r44wukfnh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:47 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5fH1004436 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:41 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5ffG007260; Thu, 14 Mar 2019 21:05:41 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:41 +0000 Subject: [PATCH 18/36] scrub: fix Makefile targets which depend on builddefs From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:41 -0700 Message-ID: <155259754139.31886.15219987362146132076.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=9 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add Makefile dependencies for targets that require variables set in builddefs. Signed-off-by: Darrick J. Wong --- scrub/Makefile | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/scrub/Makefile b/scrub/Makefile index 6e155c2c..bbcfe338 100644 --- a/scrub/Makefile +++ b/scrub/Makefile @@ -3,7 +3,8 @@ # TOPDIR = .. -include $(TOPDIR)/include/builddefs +builddefs=$(TOPDIR)/include/builddefs +include $(builddefs) # On linux we get fsmap from the system or define it ourselves # so include this based on platform type. If this reverts to only @@ -103,27 +104,27 @@ LDIRT = $(XFS_SCRUB_ALL_PROG) *.service *.cron default: depend $(LTCOMMAND) $(XFS_SCRUB_ALL_PROG) $(OPTIONAL_TARGETS) -xfs_scrub_all: xfs_scrub_all.in +xfs_scrub_all: xfs_scrub_all.in $(builddefs) @echo " [SED] $@" $(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \ -e "s|@pkg_version@|$(PKG_VERSION)|g" \ -e "s|@scrub_args@|$(XFS_SCRUB_ARGS)|g" < $< > $@ $(Q)chmod a+x $@ -phase5.o unicrash.o xfs.o: $(TOPDIR)/include/builddefs +phase5.o unicrash.o xfs.o: $(builddefs) include $(BUILDRULES) install: $(INSTALL_SCRUB) -%.service: %.service.in +%.service: %.service.in $(builddefs) @echo " [SED] $@" $(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \ -e "s|@scrub_args@|$(XFS_SCRUB_ARGS)|g" \ -e "s|@pkg_lib_dir@|$(PKG_LIB_DIR)|g" \ -e "s|@pkg_name@|$(PKG_NAME)|g" < $< > $@ -%.cron: %.cron.in +%.cron: %.cron.in $(builddefs) @echo " [SED] $@" $(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" < $< > $@ From patchwork Thu Mar 14 21:05:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853623 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E3B617E6 for ; Thu, 14 Mar 2019 21:05:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 27DD12A031 for ; Thu, 14 Mar 2019 21:05:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 266E42A78F; Thu, 14 Mar 2019 21:05:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BBCEB2A031 for ; Thu, 14 Mar 2019 21:05:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727366AbfCNVFv (ORCPT ); Thu, 14 Mar 2019 17:05:51 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52090 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727899AbfCNVFv (ORCPT ); Thu, 14 Mar 2019 17:05:51 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwusJ095446; Thu, 14 Mar 2019 21:05:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=fwdLU4ScWZw439LCp0b1Ow/Mz9NfVWWv8cI5wphfPcA=; b=HmCNlVtj+cSx7cOKc4wdkegNMGEdk8KYOa3PrXVGRN3T6hUCm9nF/AKFOBMQRsUH0nuk /882pbX/AM9pQfw3Yr0Aw0WlJhg4LTqhsB87MsUC/XR2+tsZ2Q4szdt6e72uLaoT1GzT J7d1YPJTvUGkNGJ5VpeYaeYSEUpQeYaS+FPcViRXA3VZKcfneSIpAziFkdT+bE8hl8Qi laQXZaOA+l/hnjvXUH7HwZCm/41D1GoBbfMI2o5i3TXJCuzjd+jzTjNl025REkymZTC6 Kmgh/segT8i3qN+4MzXZxCt1Nf9U7EnBfHdk8yoXyjxBVi2NZlJppDC8TlUD/w7wQFBh BQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2r44wukfnp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:48 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5mmJ000362 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:48 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5l5k007302; Thu, 14 Mar 2019 21:05:47 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:47 +0000 Subject: [PATCH 19/36] mkfs: validate extent size hint parameters From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:47 -0700 Message-ID: <155259754740.31886.13886644609473939892.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Validate extent and cow extent size hints that are passed to mkfs so that we avoid formatting a filesystem that will never mount. Signed-off-by: Darrick J. Wong --- mkfs/xfs_mkfs.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c index d1387ddf..9e1c6ec5 100644 --- a/mkfs/xfs_mkfs.c +++ b/mkfs/xfs_mkfs.c @@ -2202,6 +2202,66 @@ validate_rtextsize( ASSERT(cfg->rtextblocks); } +/* Validate the incoming extsize hint as if we were a file. */ +static void +validate_extsize_hint( + struct xfs_mount *mp, + struct cli_params *cli) +{ + xfs_failaddr_t fa; + bool rt; + uint16_t flags = 0; + + rt = (cli->fsx.fsx_xflags & XFS_DIFLAG_RTINHERIT); + + if (cli->fsx.fsx_xflags & XFS_DIFLAG_EXTSZINHERIT) + flags |= XFS_DIFLAG_EXTSIZE; + + fa = libxfs_inode_validate_extsize(mp, cli->fsx.fsx_extsize, S_IFREG, + flags); + if (rt && fa == NULL) + fa = libxfs_inode_validate_extsize(mp, cli->fsx.fsx_extsize, + S_IFREG, flags | XFS_DIFLAG_REALTIME); + if (fa == NULL) + return; + + if (rt && mp->m_sb.sb_rextsize > 1) + fprintf(stderr, + _("illegal extent size hint %lld, must be less than %u and a multiple of %u.\n"), + (long long)cli->fsx.fsx_extsize, + min(MAXEXTLEN, mp->m_sb.sb_agblocks / 2), + mp->m_sb.sb_rextsize); + else + fprintf(stderr, + _("illegal extent size hint %lld, must be less than %u.\n"), + (long long)cli->fsx.fsx_extsize, + min(MAXEXTLEN, mp->m_sb.sb_agblocks / 2)); + usage(); +} + +/* Validate the incoming CoW extsize hint as if we were a file. */ +static void +validate_cowextsize_hint( + struct xfs_mount *mp, + struct cli_params *cli) +{ + xfs_failaddr_t fa; + uint64_t flags2 = 0; + + if (cli->fsx.fsx_xflags & FS_XFLAG_COWEXTSIZE) + flags2 |= XFS_DIFLAG2_COWEXTSIZE; + fa = libxfs_inode_validate_cowextsize(mp, cli->fsx.fsx_cowextsize, + S_IFREG, 0, flags2); + if (fa == NULL) + return; + + fprintf(stderr, +_("illegal CoW extent size hint %lld, must be less than %u.\n"), + (long long)cli->fsx.fsx_cowextsize, + min(MAXEXTLEN, mp->m_sb.sb_agblocks / 2)); + usage(); +} + /* * Validate the configured stripe geometry, or is none is specified, pull * the configuration from the underlying device. @@ -3945,6 +4005,10 @@ main( finish_superblock_setup(&cfg, mp, sbp); + /* Validate the extent size hints now that @mp is fully set up. */ + validate_extsize_hint(mp, &cli); + validate_cowextsize_hint(mp, &cli); + /* Print the intended geometry of the fs. */ if (!quiet || dry_run) { struct xfs_fsop_geom geo; From patchwork Thu Mar 14 21:05:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853625 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C348917E6 for ; Thu, 14 Mar 2019 21:05:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA8AC283B2 for ; Thu, 14 Mar 2019 21:05:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A91432A790; Thu, 14 Mar 2019 21:05:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 55C40283B2 for ; Thu, 14 Mar 2019 21:05:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727935AbfCNVF5 (ORCPT ); Thu, 14 Mar 2019 17:05:57 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52194 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727625AbfCNVF4 (ORCPT ); Thu, 14 Mar 2019 17:05:56 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwuoh095452; Thu, 14 Mar 2019 21:05:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=6xHrwya6Xf4x+5xVR9m8GJa49gQncdFMAfo56f/gza8=; b=bBF0QJyQ3D7qRj/O1Cjx2zMxQGfYYzm01DhYRjsky+a6ToMOovHm3QLg1nP9kSHmjvAI 3q5kl98LwWAngFdkIL9Ja53JXr2bOgwy+4PcMGOgQ2oG24Ytyplfrwe9ZkBgk2bVBaOL PSu86L7DlioDW1vzB41985tVcpUC4c9QdXL2ZiBJ0E5r68KhBcj4W/PobCW2+wJqbGta xDBV1W1p8WcP4o7K0IZ2uBmZ7/ItAxpqnrFjU+DUkJIAAuE0DU5rdKaI9iKsR8t8nlIZ zJtXe7gAGy8HwLbKmu7IWo3ReLqzZON4BuJv9lQflpZm/55yPYBonmpf5Z5mSya1FfW0 Ag== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2r44wukfp1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:54 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5sXA000761 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:05:54 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5rkb003561; Thu, 14 Mar 2019 21:05:54 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:53 +0000 Subject: [PATCH 20/36] xfs_db: fix finobt record decoding when sparse inodes enabled From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:53 -0700 Message-ID: <155259755345.31886.7689089384611545119.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Use the sparse inobt record field decoder (inobt_spcrc_hfld) to decode finobt records when sparse inodes are enabled. Otherwise, xfs_db prints out bogus things like: recs[1] = [startino,freecount,free] 1:[214720,16429,0xfffffffffff80000] There can never be 16429 records in an inode btree record; instead it should print: recs[1] = [startino,holemask,count,freecount,free] 1:[214720,0,64,45,0xfffffffffff80000] Signed-off-by: Darrick J. Wong --- db/type.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/db/type.c b/db/type.c index f5f65042..d42935a2 100644 --- a/db/type.c +++ b/db/type.c @@ -107,7 +107,7 @@ static const typ_t __typtab_crc[] = { { TYP_SYMLINK, "symlink", handle_struct, symlink_crc_hfld, &xfs_symlink_buf_ops, XFS_SYMLINK_CRC_OFF }, { TYP_TEXT, "text", handle_text, NULL, NULL, TYP_F_NO_CRC_OFF }, - { TYP_FINOBT, "finobt", handle_struct, inobt_crc_hfld, + { TYP_FINOBT, "finobt", handle_struct, inobt_spcrc_hfld, &xfs_inobt_buf_ops, XFS_BTREE_SBLOCK_CRC_OFF }, { TYP_NONE, NULL } }; From patchwork Thu Mar 14 21:05:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C13D1515 for ; Thu, 14 Mar 2019 21:06:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 128552A031 for ; Thu, 14 Mar 2019 21:06:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0E9E52A199; Thu, 14 Mar 2019 21:06:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEC7A2A77B for ; Thu, 14 Mar 2019 21:06:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727625AbfCNVGD (ORCPT ); Thu, 14 Mar 2019 17:06:03 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:51884 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727003AbfCNVGD (ORCPT ); Thu, 14 Mar 2019 17:06:03 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwe1S102479; Thu, 14 Mar 2019 21:06:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=1Wk4Q4LxuD4W1Vg6UQH3vJ3wku1kpaiAl3mr8kdejqQ=; b=VGy/jHeZvfeSgjUjRPYvBsHCDyChQMPNMXx/IFgIyWmk2/EWjyWBTsjk8evUey87D+jC 5zuqQ18KLiITMDxq/UTwQae072JuwDVe/Qtpjak/9yT0XrdANZvW/LMUHB/WnreZ8N2g IFmaQvmt3AA8BhK46qllL6vzJgyOLlMiP223XRstwMBg0JBYHEVYQNEjtP+17MZ0EWTY R8Rc9ov2hJm6bfqybWnMt9Dx5qfyWicWISbr+Nrg6+6VKICEcZzh/tioRqoF7daK21EF oN/t/ksGicO8by2B1VlXFgOELfPTDPLiTmY9YIGYMo3GBSXnIVlXTFhRIlRVjIslPAw4 DQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2r464ruem0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:00 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL60j3005499 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:00 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL5xDN007444; Thu, 14 Mar 2019 21:05:59 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:05:59 +0000 Subject: [PATCH 21/36] xfs_db: use TYP_FINOBT for finobt metadump From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:05:59 -0700 Message-ID: <155259755947.31886.6074285457193991395.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Use the correct xfs_db type for dumping free inode btree blocks. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- db/metadump.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/db/metadump.c b/db/metadump.c index 6ecd5685..3cce3012 100644 --- a/db/metadump.c +++ b/db/metadump.c @@ -2607,7 +2607,7 @@ copy_inodes( levels = be32_to_cpu(agi->agi_free_level); finobt = 1; - if (!scan_btree(agno, root, levels, TYP_INOBT, &finobt, + if (!scan_btree(agno, root, levels, TYP_FINOBT, &finobt, scanfunc_ino)) return 0; } From patchwork Thu Mar 14 21:06:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853629 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7799E1515 for ; Thu, 14 Mar 2019 21:06:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 60C512A296 for ; Thu, 14 Mar 2019 21:06:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5F4962A710; Thu, 14 Mar 2019 21:06:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 00AEB2A78A for ; Thu, 14 Mar 2019 21:06:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727146AbfCNVGO (ORCPT ); Thu, 14 Mar 2019 17:06:14 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52564 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727003AbfCNVGO (ORCPT ); Thu, 14 Mar 2019 17:06:14 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwnUU095338; Thu, 14 Mar 2019 21:06:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=aoW1C4NW7S0VXVHh6CFkHwfULuV/Vy83hMeoY8gpj98=; b=XeNNzvxu/L+lJ0Bn1XaoL81SrkH5f4xESsk1mV+A8VdhTq5reRcqPDLoPHzhK4R3ZEwD AT2eZM7wugUCRcwDF5epbd05X9d2ieZKZL8PXz98KWw57xznyGxMMXm+Oc9zffbYI46d pt/Z/rqhmlp7Brb7czqFzd5zN2u+sMf/PLeOmuZnxUbz9udQzCcE9MgxieIu90L9bc1M 4Can4xYq5XJJATva8LyOFkw0yEmpePdTMTOQONN/9sPnHkpYT47Ff6xmCWngOgq3T7KH VOnUXtLwYIhdfeq4q1KkGiQMIu8xCWVWyqRai3YMnGq7a09YZFE8MJXfUZ90Trp7hAx+ DA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2r44wukfq6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:11 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL66bd006067 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:06 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL65Te007530; Thu, 14 Mar 2019 21:06:05 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:05 +0000 Subject: [PATCH 22/36] xfs_info: use findmnt to handle mounted block devices From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:06:05 -0700 Message-ID: <155259756550.31886.4299958077869009509.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Use findmnt to determine if the passed-in argument is associated with a mount point, and if so, use spaceman to query the mounted filesystem. Signed-off-by: Darrick J. Wong --- debian/control | 2 +- spaceman/xfs_info.sh | 14 +++++++++++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/debian/control b/debian/control index f4f807b0..0b3205f5 100644 --- a/debian/control +++ b/debian/control @@ -8,7 +8,7 @@ Standards-Version: 4.0.0 Homepage: https://xfs.wiki.kernel.org/ Package: xfsprogs -Depends: ${shlibs:Depends}, ${misc:Depends}, python3:any +Depends: ${shlibs:Depends}, ${misc:Depends}, python3:any, util-linux Provides: fsck-backend Suggests: xfsdump, acl, attr, quota Breaks: xfsdump (<< 3.0.0) diff --git a/spaceman/xfs_info.sh b/spaceman/xfs_info.sh index ecf17f61..70978164 100755 --- a/spaceman/xfs_info.sh +++ b/spaceman/xfs_info.sh @@ -24,11 +24,19 @@ set -- extra "$@" shift $OPTIND case $# in 1) - if [ -b "$1" ] || [ -f "$1" ]; then - xfs_db -p xfs_info -c "info" $OPTS "$1" + arg="$1" + + # See if we can map the arg to a loop device + loopdev="$(losetup -n -O NAME -j "${arg}" 2> /dev/null)" + test -n "${loopdev}" && arg="${loopdev}" + + # If we find a mountpoint for the device, do a live query; + # otherwise try reading the fs with xfs_db. + if mountpt="$(findmnt -f -n -o TARGET "${arg}" 2> /dev/null)"; then + xfs_spaceman -p xfs_info -c "info" $OPTS "${mountpt}" status=$? else - xfs_spaceman -p xfs_info -c "info" $OPTS "$1" + xfs_db -p xfs_info -c "info" $OPTS "${arg}" status=$? fi ;; From patchwork Thu Mar 14 21:06:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CE9AD1874 for ; Thu, 14 Mar 2019 21:06:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B70AC2A2A0 for ; Thu, 14 Mar 2019 21:06:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B58E42A78B; Thu, 14 Mar 2019 21:06:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F33F2A2A0 for ; Thu, 14 Mar 2019 21:06:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727003AbfCNVGO (ORCPT ); Thu, 14 Mar 2019 17:06:14 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:52084 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727102AbfCNVGO (ORCPT ); Thu, 14 Mar 2019 17:06:14 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwnZD102560; Thu, 14 Mar 2019 21:06:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=i+A+TH1hAcAzWf6IcwDEb6ubBQtS4SpiUiLBYoRRkRQ=; b=5NtsTfpZ8ybhKRsG1iod5BUK9oHYegJ6yeeupFYcLd9kiShS4akIz1iSwnKmHhI2vbrK gZCvCia+8I1XKluRpv211iNHdUVjR5Rf9lbk+qUhZVskbbZYuD6XYhd9zwRmzK5x482Z D5ncqwsjHHwgYzcb0YwTILMzDO3/SaU51AAipbetJyJxYUvz13+svU5LPEaDwyGA6022 lrTWTE/yvYmlj33WC10XqMasZD2ZM+lYqiMK+WAiEZRcSF9Yd62qjxqtZxUAiPnjdcW/ itL1MQfrAd43ZTg3D1JevucB+ofKiebooVYkyzDevvmQl4DbLOQdCDMawPA3Pb46VA70 /w== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2r464ruemw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:12 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6CTL006327 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:12 GMT Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL6BBJ001618; Thu, 14 Mar 2019 21:06:11 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:11 +0000 Subject: [PATCH 23/36] xfs_repair: reinitialize the root directory nlink correctly From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:06:11 -0700 Message-ID: <155259757149.31886.6730311002359408147.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In mk_root_dir, we reinitialize the root directory inode with a link count of 1. This differs from mkfs parseproto, which initializes the root to have a link count of 2. The nlink discrepancy in repair is caught and corrected during phase 7, but this is unnecessary since we should set it properly in the first place. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- repair/phase6.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repair/phase6.c b/repair/phase6.c index 9477bc25..8a50b350 100644 --- a/repair/phase6.c +++ b/repair/phase6.c @@ -891,7 +891,7 @@ mk_root_dir(xfs_mount_t *mp) ip->i_d.di_format = XFS_DINODE_FMT_EXTENTS; ip->i_d.di_aformat = XFS_DINODE_FMT_EXTENTS; - set_nlink(VFS_I(ip), 1); /* account for . */ + set_nlink(VFS_I(ip), 2); /* account for . and .. */ times = XFS_ICHGTIME_CHG | XFS_ICHGTIME_MOD; if (ip->i_d.di_version == 3) { From patchwork Thu Mar 14 21:06:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE5A21515 for ; Thu, 14 Mar 2019 21:06:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A68E02A1E1 for ; Thu, 14 Mar 2019 21:06:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A4E632A23B; Thu, 14 Mar 2019 21:06:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 983AA2A78A for ; Thu, 14 Mar 2019 21:06:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727102AbfCNVGV (ORCPT ); Thu, 14 Mar 2019 17:06:21 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:52206 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727204AbfCNVGV (ORCPT ); Thu, 14 Mar 2019 17:06:21 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKweoW102482; Thu, 14 Mar 2019 21:06:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=8wkppLVQGDoo/u8MdwOgAOwNDrxcETBS1C4+5kvrnpk=; b=A8FJCJnIH8hUAMBbTyZFpRNpKl2ctq2xSdMC4omAx5Ej59Mdyr9rD3qDTbPmlhIcOMLW VJl1u63HRn9FO/fIVMWvYFv8usDm38aT/tR3dUa9L3dnxmB+uK2z72ScOcjbyyktXc7l MPT7fZER8H88TR6ZTEvT1GK8AL2TaP81wTZ/r5IE9ZYrpdQ49jNcliQbEcG9SGvrgn3W f8Q0kXtP2kJPVdNJCjbPVW6kPor7rlxKmtb/BX96G1K8wWigvGahOhPH0IB7wlNfluB4 fjB7g/p1Kg4a31ag51XNozRKB5abfZvt4WIY39qo/OUwbYu3uYh6urPM5pJE9b9JWWcM kA== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2r464ruen6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:18 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6IYE009703 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:18 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL6Ihm020466; Thu, 14 Mar 2019 21:06:18 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:17 +0000 Subject: [PATCH 24/36] xfs_repair: bump the irec on-disk nlink when adding lost+found From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:06:17 -0700 Message-ID: <155259757750.31886.15831311797503004248.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong We increment the nlink of the root directory inode when creating a "lost+found" directory during phase 6, but we don't update the irec copy of the root dir nlink. This normally gets papered over by phase 7, but this can fail badly if: 1) The root directory had an entry to a busted subdirectory, so that root directory will have nlink == 3, but in the ino_tree, counted_nlinks == 2 and disk_nlinks == 3. 2) Phase 6 creates lost+found to root the files that were in the busted directory, we'll set nlink = 4 and counted_nlinks = 3. The correct nlink is 3 ('.', '..', 'lost+found'), not 4. 3) During phase 7, we see that counted_nlinks == disk_nlinks and so we totally fail to correct the on-disk inode. 4) A subsequent run of xfs_repair complains about the nlink being 4 instead of 3. To fix this, we have to adjust the irec's disk_nlinks in step 2 so that phase 7 seeds that counted_nlinks < disk_nlinks and resets nlink to counted_nlinks. This can be reproduced somewhat frequently by xfs/117. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- repair/phase6.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/repair/phase6.c b/repair/phase6.c index 8a50b350..194cfdbf 100644 --- a/repair/phase6.c +++ b/repair/phase6.c @@ -1019,6 +1019,7 @@ mk_orphanage(xfs_mount_t *mp) */ set_inode_used(irec, ino_offset); add_inode_ref(irec, ino_offset); + add_inode_reached(irec, ino_offset); /* * now that we know the transaction will stay around, @@ -1037,14 +1038,14 @@ mk_orphanage(xfs_mount_t *mp) /* * bump up the link count in the root directory to account - * for .. in the new directory + * for .. in the new directory, and update the irec copy of the + * on-disk nlink so we don't fail the link count check later. */ inc_nlink(VFS_I(pip)); - add_inode_ref(find_inode_rec(mp, - XFS_INO_TO_AGNO(mp, mp->m_sb.sb_rootino), - XFS_INO_TO_AGINO(mp, mp->m_sb.sb_rootino)), 0); - - + irec = find_inode_rec(mp, XFS_INO_TO_AGNO(mp, mp->m_sb.sb_rootino), + XFS_INO_TO_AGINO(mp, mp->m_sb.sb_rootino)); + add_inode_ref(irec, 0); + set_inode_disk_nlinks(irec, 0, get_inode_disk_nlinks(irec, 0) + 1); libxfs_trans_log_inode(tp, pip, XFS_ILOG_CORE); libxfs_dir_init(tp, ip, pip); @@ -1056,7 +1057,6 @@ mk_orphanage(xfs_mount_t *mp) } libxfs_irele(ip); libxfs_irele(pip); - add_inode_reached(irec,ino_offset); return(ino); } From patchwork Thu Mar 14 21:06:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853635 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 690441515 for ; Thu, 14 Mar 2019 21:06:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 513C02A296 for ; Thu, 14 Mar 2019 21:06:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4F99F2A785; Thu, 14 Mar 2019 21:06:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C84D283B2 for ; Thu, 14 Mar 2019 21:06:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727309AbfCNVGb (ORCPT ); Thu, 14 Mar 2019 17:06:31 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:52566 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727204AbfCNVGb (ORCPT ); Thu, 14 Mar 2019 17:06:31 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwsN7132890; Thu, 14 Mar 2019 21:06:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=NmKM3sWy/DiXf3cwgkc3XlhymGRlCSSBQCGeFzl0p18=; b=eIVnb6OeRFqRi4P8pc9xev/LCd84HJIgT2B8FoyZ16fbTaHbKN9caErp5YgyIwL5X8rN BKlQ5lMX8ip8X8BRN7ffR445QwIjqUDQcsVwBqJnUCtLo0UjdWZE66YNM0js2lUOk08b XcUAT3VVc5kDO/laSl14f69k8ZhjYvvdDzPOZROWzpphxWX6kAHWjL4HnBhJcvTuSyUf 16Go/9ZfkcgnGZw5mGfnA6AT6lNQLPqCM9AuepTWrOpoOhkEQRGaoIig4gvHqcuRA0rB LlOTPl/ls0veqUhrPGi+tsnIQafejcUxQ1ZFApfdMZjmR/27KKBXr0MqT0FYAcWTMyjC yQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2r430f3xuc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:29 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6Oub006697 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:24 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL6Ool001658; Thu, 14 Mar 2019 21:06:24 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:23 +0000 Subject: [PATCH 25/36] xfs_repair: fix uninitialized variable warnings From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:06:23 -0700 Message-ID: <155259758358.31886.2042156413647033104.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Fix some uninitialized variable warnings because ASSERT disappears if DEBUG is not defined. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- repair/dinode.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/repair/dinode.c b/repair/dinode.c index f670bf87..c0a56daa 100644 --- a/repair/dinode.c +++ b/repair/dinode.c @@ -1176,8 +1176,8 @@ process_quota_inode( struct xfs_buf *bp; xfs_filblks_t dqchunklen; uint dqperchunk; - int quota_type; - char *quota_string; + int quota_type = 0; + char *quota_string = NULL; xfs_dqid_t dqid; xfs_fileoff_t qbno; int i; From patchwork Thu Mar 14 21:06:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853637 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1723F1515 for ; Thu, 14 Mar 2019 21:06:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0084A283B2 for ; Thu, 14 Mar 2019 21:06:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F2FF329DEC; Thu, 14 Mar 2019 21:06:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83CD0283B2 for ; Thu, 14 Mar 2019 21:06:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727467AbfCNVGd (ORCPT ); Thu, 14 Mar 2019 17:06:33 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:52372 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727204AbfCNVGd (ORCPT ); Thu, 14 Mar 2019 17:06:33 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwsKk102611; Thu, 14 Mar 2019 21:06:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=mmlJmCHsAlyJE4wwIJEF9kUQrHBhx20IRYv5jhuSajU=; b=ePDdzLk/Mt9UcnoIOhx6pl/yjiTig58yUEC1EOt76hdZCH039e+JEnmY446QTrRCotXR nL9QYzEl7lnRRQWLR2mbU3ysHQPI4lXh1QCODN5uLUmRmxsZNmE8AatgaZbLDn1VCNsF vBp2CaKjb0QaVc+omMhLJVk60Yn1LDj3Od0zItVZrOVV7Psb9UDlANB6DRtTm5LT5+qh B/yRzogpUo+btOLglRywHBrQgN1/BEm5BbIXfzsEbL4IsgW/fpbTqcOjZ+D/rFv7SZ0C JY/CWBNG2SKrnbpjnzxrz7R+i9LARCwuTeFKGrKbcDhbt5ZvvU1bdWJi14VdCR3LrOgs RQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2r464rueny-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:30 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6UOP010187 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:30 GMT Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6UqU007730; Thu, 14 Mar 2019 21:06:30 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:29 +0000 Subject: [PATCH 26/36] xfs_repair: refactor buffer ops assignments during phase 5 From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:06:29 -0700 Message-ID: <155259758961.31886.4826767055567611572.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Refactor the buffer ops assignments in phase 5 to use a helper function to determine the correct buf_ops instead of open-coding them. Signed-off-by: Darrick J. Wong --- repair/phase5.c | 34 ++++++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/repair/phase5.c b/repair/phase5.c index 367c3ab9..0c53096f 100644 --- a/repair/phase5.c +++ b/repair/phase5.c @@ -615,6 +615,24 @@ calculate_freespace_cursor(xfs_mount_t *mp, xfs_agnumber_t agno, return(extra_blocks); } +/* Map btnum to buffer ops for the types that need it. */ +static const struct xfs_buf_ops * +btnum_to_ops( + xfs_btnum_t btnum) +{ + switch (btnum) { + case XFS_BTNUM_BNO: + case XFS_BTNUM_CNT: + return &xfs_allocbt_buf_ops; + case XFS_BTNUM_INO: + case XFS_BTNUM_FINO: + return &xfs_inobt_buf_ops; + default: + ASSERT(0); + return NULL; + } +} + static void prop_freespace_cursor(xfs_mount_t *mp, xfs_agnumber_t agno, bt_status_t *btree_curs, xfs_agblock_t startblock, @@ -625,6 +643,7 @@ prop_freespace_cursor(xfs_mount_t *mp, xfs_agnumber_t agno, xfs_alloc_ptr_t *bt_ptr; xfs_agblock_t agbno; bt_stat_level_t *lptr; + const struct xfs_buf_ops *ops = btnum_to_ops(btnum); ASSERT(btnum == XFS_BTNUM_BNO || btnum == XFS_BTNUM_CNT); @@ -675,7 +694,7 @@ prop_freespace_cursor(xfs_mount_t *mp, xfs_agnumber_t agno, /* * initialize block header */ - lptr->buf_p->b_ops = &xfs_allocbt_buf_ops; + lptr->buf_p->b_ops = ops; bt_hdr = XFS_BUF_TO_BLOCK(lptr->buf_p); memset(bt_hdr, 0, mp->m_sb.sb_blocksize); libxfs_btree_init_block(mp, lptr->buf_p, btnum, level, @@ -723,6 +742,7 @@ build_freespace_tree(xfs_mount_t *mp, xfs_agnumber_t agno, extent_tree_node_t *ext_ptr; bt_stat_level_t *lptr; xfs_extlen_t freeblks; + const struct xfs_buf_ops *ops = btnum_to_ops(btnum); ASSERT(btnum == XFS_BTNUM_BNO || btnum == XFS_BTNUM_CNT); @@ -754,7 +774,7 @@ build_freespace_tree(xfs_mount_t *mp, xfs_agnumber_t agno, /* * initialize block header */ - lptr->buf_p->b_ops = &xfs_allocbt_buf_ops; + lptr->buf_p->b_ops = ops; bt_hdr = XFS_BUF_TO_BLOCK(lptr->buf_p); memset(bt_hdr, 0, mp->m_sb.sb_blocksize); libxfs_btree_init_block(mp, lptr->buf_p, btnum, i, 0, agno, 0); @@ -781,7 +801,7 @@ build_freespace_tree(xfs_mount_t *mp, xfs_agnumber_t agno, /* * block initialization, lay in block header */ - lptr->buf_p->b_ops = &xfs_allocbt_buf_ops; + lptr->buf_p->b_ops = ops; bt_hdr = XFS_BUF_TO_BLOCK(lptr->buf_p); memset(bt_hdr, 0, mp->m_sb.sb_blocksize); libxfs_btree_init_block(mp, lptr->buf_p, btnum, 0, 0, agno, 0); @@ -990,6 +1010,7 @@ prop_ino_cursor(xfs_mount_t *mp, xfs_agnumber_t agno, bt_status_t *btree_curs, xfs_inobt_ptr_t *bt_ptr; xfs_agblock_t agbno; bt_stat_level_t *lptr; + const struct xfs_buf_ops *ops = btnum_to_ops(btnum); level++; @@ -1038,7 +1059,7 @@ prop_ino_cursor(xfs_mount_t *mp, xfs_agnumber_t agno, bt_status_t *btree_curs, /* * initialize block header */ - lptr->buf_p->b_ops = &xfs_inobt_buf_ops; + lptr->buf_p->b_ops = ops; bt_hdr = XFS_BUF_TO_BLOCK(lptr->buf_p); memset(bt_hdr, 0, mp->m_sb.sb_blocksize); libxfs_btree_init_block(mp, lptr->buf_p, btnum, @@ -1130,6 +1151,7 @@ build_ino_tree(xfs_mount_t *mp, xfs_agnumber_t agno, xfs_inobt_rec_t *bt_rec; ino_tree_node_t *ino_rec; bt_stat_level_t *lptr; + const struct xfs_buf_ops *ops = btnum_to_ops(btnum); xfs_agino_t count = 0; xfs_agino_t freecount = 0; int inocnt; @@ -1160,7 +1182,7 @@ build_ino_tree(xfs_mount_t *mp, xfs_agnumber_t agno, * initialize block header */ - lptr->buf_p->b_ops = &xfs_inobt_buf_ops; + lptr->buf_p->b_ops = ops; bt_hdr = XFS_BUF_TO_BLOCK(lptr->buf_p); memset(bt_hdr, 0, mp->m_sb.sb_blocksize); libxfs_btree_init_block(mp, lptr->buf_p, btnum, i, 0, agno, 0); @@ -1188,7 +1210,7 @@ build_ino_tree(xfs_mount_t *mp, xfs_agnumber_t agno, /* * block initialization, lay in block header */ - lptr->buf_p->b_ops = &xfs_inobt_buf_ops; + lptr->buf_p->b_ops = ops; bt_hdr = XFS_BUF_TO_BLOCK(lptr->buf_p); memset(bt_hdr, 0, mp->m_sb.sb_blocksize); libxfs_btree_init_block(mp, lptr->buf_p, btnum, 0, 0, agno, 0); From patchwork Thu Mar 14 21:06:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853639 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 693C21515 for ; Thu, 14 Mar 2019 21:06:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 53C012A1E1 for ; Thu, 14 Mar 2019 21:06:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 524022A78B; Thu, 14 Mar 2019 21:06:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFCE02A1E1 for ; Thu, 14 Mar 2019 21:06:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727754AbfCNVGk (ORCPT ); Thu, 14 Mar 2019 17:06:40 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:52504 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727204AbfCNVGk (ORCPT ); Thu, 14 Mar 2019 17:06:40 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwfSg102492; Thu, 14 Mar 2019 21:06:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=PSPMEj2mvwAAQaPe+agldV3INknysvOvL/0L12fIHDo=; b=EQF3Xgq/mbz9/xFGMdH+5glXg4X1CE9gpgnoKjGj5xXXFAsXFH9jy7T6jJrS4rwCqOQA X9jz++2yJ72WLPCOa7hu1YhFLKEE8lT/1AWshSNjbI0Tc7zofBp3mPTPOFRFtAvgYBrq EdGL3iHJED/rg1tghGxhFMyxvQg0Y6Pb+2Jqa42s9YxSo0WT1i4UzTsLWDDheijjwtl7 Ovf7lsqkGMS+SYN2H8bjDtwDGSsBLrp4zzHtmiml1TIuXk2lzXR0IFOOUwgTag0ecgoh 8pfAmN1n4MjW1tjuDDFm88QbP97imFRDLUl1oXjXwvI+XFPkpiGBKagkSklZg2I+6TuU fQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2r464ruepf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:37 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6asW007296 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:36 GMT Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6aZI003777; Thu, 14 Mar 2019 21:06:36 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:35 +0000 Subject: [PATCH 27/36] xfs_repair: pass ops through during scan From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:06:35 -0700 Message-ID: <155259759563.31886.2004896919962409807.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Pass the buffer ops through scan_sbtree so that we detect finobt blocks properly and we don't have to keep switching on magic numbers for the free space btrees. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- repair/scan.c | 39 ++++++++++++++++----------------------- 1 file changed, 16 insertions(+), 23 deletions(-) diff --git a/repair/scan.c b/repair/scan.c index 12ca3144..edc375a9 100644 --- a/repair/scan.c +++ b/repair/scan.c @@ -59,7 +59,8 @@ scan_sbtree( int suspect, int isroot, uint32_t magic, - void *priv), + void *priv, + const struct xfs_buf_ops *ops), int isroot, uint32_t magic, void *priv, @@ -80,7 +81,7 @@ scan_sbtree( } (*func)(XFS_BUF_TO_BLOCK(bp), nlevels - 1, root, agno, suspect, - isroot, magic, priv); + isroot, magic, priv, ops); libxfs_putbuf(bp); } @@ -543,7 +544,8 @@ scan_allocbt( int suspect, int isroot, uint32_t magic, - void *priv) + void *priv, + const struct xfs_buf_ops *ops) { struct aghdr_cnts *agcnts = priv; const char *name; @@ -750,20 +752,8 @@ _("%s freespace btree block claimed (state %d), agno %d, bno %d, suspect %d\n"), * as possible. */ if (agbno != 0 && verify_agbno(mp, agno, agbno)) { - switch (magic) { - case XFS_ABTB_CRC_MAGIC: - case XFS_ABTB_MAGIC: - scan_sbtree(agbno, level, agno, suspect, - scan_allocbt, 0, magic, priv, - &xfs_allocbt_buf_ops); - break; - case XFS_ABTC_CRC_MAGIC: - case XFS_ABTC_MAGIC: - scan_sbtree(agbno, level, agno, suspect, - scan_allocbt, 0, magic, priv, - &xfs_allocbt_buf_ops); - break; - } + scan_sbtree(agbno, level, agno, suspect, scan_allocbt, + 0, magic, priv, ops); } } } @@ -926,7 +916,8 @@ scan_rmapbt( int suspect, int isroot, uint32_t magic, - void *priv) + void *priv, + const struct xfs_buf_ops *ops) { const char *name = "rmap"; int i; @@ -1205,7 +1196,7 @@ _("%s rmap btree block claimed (state %d), agno %d, bno %d, suspect %d\n"), if (agbno != 0 && verify_agbno(mp, agno, agbno)) { scan_sbtree(agbno, level, agno, suspect, scan_rmapbt, 0, - magic, priv, &xfs_rmapbt_buf_ops); + magic, priv, ops); } } @@ -1229,7 +1220,8 @@ scan_refcbt( int suspect, int isroot, uint32_t magic, - void *priv) + void *priv, + const struct xfs_buf_ops *ops) { const char *name = "refcount"; int i; @@ -1423,7 +1415,7 @@ _("extent (%u/%u) len %u claimed, state is %d\n"), if (agbno != 0 && verify_agbno(mp, agno, agbno)) { scan_sbtree(agbno, level, agno, suspect, scan_refcbt, 0, - magic, priv, &xfs_refcountbt_buf_ops); + magic, priv, ops); } } out: @@ -1935,7 +1927,8 @@ scan_inobt( int suspect, int isroot, uint32_t magic, - void *priv) + void *priv, + const struct xfs_buf_ops *ops) { struct aghdr_cnts *agcnts = priv; int i; @@ -2091,7 +2084,7 @@ _("inode btree block claimed (state %d), agno %d, bno %d, suspect %d\n"), be32_to_cpu(pp[i]))) scan_sbtree(be32_to_cpu(pp[i]), level, agno, suspect, scan_inobt, 0, magic, priv, - &xfs_inobt_buf_ops); + ops); } } From patchwork Thu Mar 14 21:06:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853643 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3950C17E6 for ; Thu, 14 Mar 2019 21:06:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1330C2A790 for ; Thu, 14 Mar 2019 21:06:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 08CAA2A786; Thu, 14 Mar 2019 21:06:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9986B2A790 for ; Thu, 14 Mar 2019 21:06:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727412AbfCNVG5 (ORCPT ); Thu, 14 Mar 2019 17:06:57 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:52710 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727204AbfCNVG5 (ORCPT ); Thu, 14 Mar 2019 17:06:57 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwkAn102528; Thu, 14 Mar 2019 21:06:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=27j8r3X7WTSV8p27qq2M89LPHbb0XBStKYVUAL0nLco=; b=rSFBNfR5P+utftuaGvTeFXdUC6Ur6qKi8vTW1JzXDqWnMx3EWYLZNhoV/e6HzOqkqDZt ydxawlkIgNl3XIN3ER+hh/Ex3Ts+gWsIQcgv1AGugTAtSPP/OJRh/X3Fjc4hrC7OHxHV H03kOWmua8KMq3P07oM2SoVNvzamNgpBBYrHiiBCYpZ6AsCWmfCBX8468LA1LiRBsboG AKTVdo4L40D8+5SugiiIFoa9Yap/XpzZcb5wQclGM9n0zDiiLMlGO/bilspxlGLtxjmj dmP9JrJ1iJ4jftRXr5flHgFnp1QMjrgPJjsxmmA/BkclP10iaZ8nI88aQN133MjpIPJw 9g== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2r464rueqa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:52 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6kUO007637 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:46 GMT Received: from abhmp0004.oracle.com (abhmp0004.oracle.com [141.146.116.10]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL6jRk020602; Thu, 14 Mar 2019 21:06:45 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:45 +0000 Subject: [PATCH 28/36] libfrog: hoist bitmap out of scrub From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:06:41 -0700 Message-ID: <155259760163.31886.7233821089132244184.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Move the bitmap code to libfrog so that we can use bitmaps in xfs_repair. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- include/bitmap.h | 24 +++ libfrog/Makefile | 1 libfrog/bitmap.c | 393 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ scrub/Makefile | 2 scrub/bitmap.c | 393 ------------------------------------------------------ scrub/bitmap.h | 24 --- 6 files changed, 418 insertions(+), 419 deletions(-) create mode 100644 include/bitmap.h create mode 100644 libfrog/bitmap.c delete mode 100644 scrub/bitmap.c delete mode 100644 scrub/bitmap.h diff --git a/include/bitmap.h b/include/bitmap.h new file mode 100644 index 00000000..e29a4335 --- /dev/null +++ b/include/bitmap.h @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright (C) 2018 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + */ +#ifndef LIBFROG_BITMAP_H_ +#define LIBFROG_BITMAP_H_ + +struct bitmap { + pthread_mutex_t bt_lock; + struct avl64tree_desc *bt_tree; +}; + +bool bitmap_init(struct bitmap **bmap); +void bitmap_free(struct bitmap **bmap); +bool bitmap_set(struct bitmap *bmap, uint64_t start, uint64_t length); +bool bitmap_iterate(struct bitmap *bmap, + bool (*fn)(uint64_t, uint64_t, void *), void *arg); +bool bitmap_test(struct bitmap *bmap, uint64_t start, + uint64_t len); +bool bitmap_empty(struct bitmap *bmap); +void bitmap_dump(struct bitmap *bmap); + +#endif /* LIBFROG_BITMAP_H_ */ diff --git a/libfrog/Makefile b/libfrog/Makefile index dbff9596..f5a0539b 100644 --- a/libfrog/Makefile +++ b/libfrog/Makefile @@ -12,6 +12,7 @@ LT_AGE = 0 CFILES = \ avl64.c \ +bitmap.c \ convert.c \ crc32.c \ fsgeom.c \ diff --git a/libfrog/bitmap.c b/libfrog/bitmap.c new file mode 100644 index 00000000..aecdba0d --- /dev/null +++ b/libfrog/bitmap.c @@ -0,0 +1,393 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright (C) 2018 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + */ +#include "xfs.h" +#include +#include +#include +#include +#include "platform_defs.h" +#include "avl64.h" +#include "list.h" +#include "bitmap.h" + +/* + * Space Efficient Bitmap + * + * Implements a space-efficient bitmap. We use an AVL tree to manage + * extent records that tell us which ranges are set; the bitmap key is + * an arbitrary uint64_t. The usual bitmap operations (set, clear, + * test, test and set) are supported, plus we can iterate set ranges. + */ + +#define avl_for_each_range_safe(pos, n, l, first, last) \ + for (pos = (first), n = pos->avl_nextino, l = (last)->avl_nextino; pos != (l); \ + pos = n, n = pos ? pos->avl_nextino : NULL) + +#define avl_for_each_safe(tree, pos, n) \ + for (pos = (tree)->avl_firstino, n = pos ? pos->avl_nextino : NULL; \ + pos != NULL; \ + pos = n, n = pos ? pos->avl_nextino : NULL) + +#define avl_for_each(tree, pos) \ + for (pos = (tree)->avl_firstino; pos != NULL; pos = pos->avl_nextino) + +struct bitmap_node { + struct avl64node btn_node; + uint64_t btn_start; + uint64_t btn_length; +}; + +static uint64_t +extent_start( + struct avl64node *node) +{ + struct bitmap_node *btn; + + btn = container_of(node, struct bitmap_node, btn_node); + return btn->btn_start; +} + +static uint64_t +extent_end( + struct avl64node *node) +{ + struct bitmap_node *btn; + + btn = container_of(node, struct bitmap_node, btn_node); + return btn->btn_start + btn->btn_length; +} + +static struct avl64ops bitmap_ops = { + extent_start, + extent_end, +}; + +/* Initialize a bitmap. */ +bool +bitmap_init( + struct bitmap **bmapp) +{ + struct bitmap *bmap; + + bmap = calloc(1, sizeof(struct bitmap)); + if (!bmap) + return false; + bmap->bt_tree = malloc(sizeof(struct avl64tree_desc)); + if (!bmap->bt_tree) { + free(bmap); + return false; + } + + pthread_mutex_init(&bmap->bt_lock, NULL); + avl64_init_tree(bmap->bt_tree, &bitmap_ops); + *bmapp = bmap; + + return true; +} + +/* Free a bitmap. */ +void +bitmap_free( + struct bitmap **bmapp) +{ + struct bitmap *bmap; + struct avl64node *node; + struct avl64node *n; + struct bitmap_node *ext; + + bmap = *bmapp; + avl_for_each_safe(bmap->bt_tree, node, n) { + ext = container_of(node, struct bitmap_node, btn_node); + free(ext); + } + free(bmap->bt_tree); + *bmapp = NULL; +} + +/* Create a new bitmap extent node. */ +static struct bitmap_node * +bitmap_node_init( + uint64_t start, + uint64_t len) +{ + struct bitmap_node *ext; + + ext = malloc(sizeof(struct bitmap_node)); + if (!ext) + return NULL; + + ext->btn_node.avl_nextino = NULL; + ext->btn_start = start; + ext->btn_length = len; + + return ext; +} + +/* Set a region of bits (locked). */ +static bool +__bitmap_set( + struct bitmap *bmap, + uint64_t start, + uint64_t length) +{ + struct avl64node *firstn; + struct avl64node *lastn; + struct avl64node *pos; + struct avl64node *n; + struct avl64node *l; + struct bitmap_node *ext; + uint64_t new_start; + uint64_t new_length; + struct avl64node *node; + bool res = true; + + /* Find any existing nodes adjacent or within that range. */ + avl64_findranges(bmap->bt_tree, start - 1, start + length + 1, + &firstn, &lastn); + + /* Nothing, just insert a new extent. */ + if (firstn == NULL && lastn == NULL) { + ext = bitmap_node_init(start, length); + if (!ext) + return false; + + node = avl64_insert(bmap->bt_tree, &ext->btn_node); + if (node == NULL) { + free(ext); + errno = EEXIST; + return false; + } + + return true; + } + + assert(firstn != NULL && lastn != NULL); + new_start = start; + new_length = length; + + avl_for_each_range_safe(pos, n, l, firstn, lastn) { + ext = container_of(pos, struct bitmap_node, btn_node); + + /* Bail if the new extent is contained within an old one. */ + if (ext->btn_start <= start && + ext->btn_start + ext->btn_length >= start + length) + return res; + + /* Check for overlapping and adjacent extents. */ + if (ext->btn_start + ext->btn_length >= start || + ext->btn_start <= start + length) { + if (ext->btn_start < start) { + new_start = ext->btn_start; + new_length += ext->btn_length; + } + + if (ext->btn_start + ext->btn_length > + new_start + new_length) + new_length = ext->btn_start + ext->btn_length - + new_start; + + avl64_delete(bmap->bt_tree, pos); + free(ext); + } + } + + ext = bitmap_node_init(new_start, new_length); + if (!ext) + return false; + + node = avl64_insert(bmap->bt_tree, &ext->btn_node); + if (node == NULL) { + free(ext); + errno = EEXIST; + return false; + } + + return res; +} + +/* Set a region of bits. */ +bool +bitmap_set( + struct bitmap *bmap, + uint64_t start, + uint64_t length) +{ + bool res; + + pthread_mutex_lock(&bmap->bt_lock); + res = __bitmap_set(bmap, start, length); + pthread_mutex_unlock(&bmap->bt_lock); + + return res; +} + +#if 0 /* Unused, provided for completeness. */ +/* Clear a region of bits. */ +bool +bitmap_clear( + struct bitmap *bmap, + uint64_t start, + uint64_t len) +{ + struct avl64node *firstn; + struct avl64node *lastn; + struct avl64node *pos; + struct avl64node *n; + struct avl64node *l; + struct bitmap_node *ext; + uint64_t new_start; + uint64_t new_length; + struct avl64node *node; + int stat; + + pthread_mutex_lock(&bmap->bt_lock); + /* Find any existing nodes over that range. */ + avl64_findranges(bmap->bt_tree, start, start + len, &firstn, &lastn); + + /* Nothing, we're done. */ + if (firstn == NULL && lastn == NULL) { + pthread_mutex_unlock(&bmap->bt_lock); + return true; + } + + assert(firstn != NULL && lastn != NULL); + + /* Delete or truncate everything in sight. */ + avl_for_each_range_safe(pos, n, l, firstn, lastn) { + ext = container_of(pos, struct bitmap_node, btn_node); + + stat = 0; + if (ext->btn_start < start) + stat |= 1; + if (ext->btn_start + ext->btn_length > start + len) + stat |= 2; + switch (stat) { + case 0: + /* Extent totally within range; delete. */ + avl64_delete(bmap->bt_tree, pos); + free(ext); + break; + case 1: + /* Extent is left-adjacent; truncate. */ + ext->btn_length = start - ext->btn_start; + break; + case 2: + /* Extent is right-adjacent; move it. */ + ext->btn_length = ext->btn_start + ext->btn_length - + (start + len); + ext->btn_start = start + len; + break; + case 3: + /* Extent overlaps both ends. */ + ext->btn_length = start - ext->btn_start; + new_start = start + len; + new_length = ext->btn_start + ext->btn_length - + new_start; + + ext = bitmap_node_init(new_start, new_length); + if (!ext) + return false; + + node = avl64_insert(bmap->bt_tree, &ext->btn_node); + if (node == NULL) { + errno = EEXIST; + return false; + } + break; + } + } + + pthread_mutex_unlock(&bmap->bt_lock); + return true; +} +#endif + +#ifdef DEBUG +/* Iterate the set regions of this bitmap. */ +bool +bitmap_iterate( + struct bitmap *bmap, + bool (*fn)(uint64_t, uint64_t, void *), + void *arg) +{ + struct avl64node *node; + struct bitmap_node *ext; + bool moveon = true; + + pthread_mutex_lock(&bmap->bt_lock); + avl_for_each(bmap->bt_tree, node) { + ext = container_of(node, struct bitmap_node, btn_node); + moveon = fn(ext->btn_start, ext->btn_length, arg); + if (!moveon) + break; + } + pthread_mutex_unlock(&bmap->bt_lock); + + return moveon; +} +#endif + +/* Do any bitmap extents overlap the given one? (locked) */ +static bool +__bitmap_test( + struct bitmap *bmap, + uint64_t start, + uint64_t len) +{ + struct avl64node *firstn; + struct avl64node *lastn; + + /* Find any existing nodes over that range. */ + avl64_findranges(bmap->bt_tree, start, start + len, &firstn, &lastn); + + return firstn != NULL && lastn != NULL; +} + +/* Is any part of this range set? */ +bool +bitmap_test( + struct bitmap *bmap, + uint64_t start, + uint64_t len) +{ + bool res; + + pthread_mutex_lock(&bmap->bt_lock); + res = __bitmap_test(bmap, start, len); + pthread_mutex_unlock(&bmap->bt_lock); + + return res; +} + +/* Are none of the bits set? */ +bool +bitmap_empty( + struct bitmap *bmap) +{ + return bmap->bt_tree->avl_firstino == NULL; +} + +#ifdef DEBUG +static bool +bitmap_dump_fn( + uint64_t startblock, + uint64_t blockcount, + void *arg) +{ + printf("%"PRIu64":%"PRIu64"\n", startblock, blockcount); + return true; +} + +/* Dump bitmap. */ +void +bitmap_dump( + struct bitmap *bmap) +{ + printf("BITMAP DUMP %p\n", bmap); + bitmap_iterate(bmap, bitmap_dump_fn, NULL); + printf("BITMAP DUMP DONE\n"); +} +#endif diff --git a/scrub/Makefile b/scrub/Makefile index bbcfe338..882da8fd 100644 --- a/scrub/Makefile +++ b/scrub/Makefile @@ -32,7 +32,6 @@ endif endif # scrub_prereqs HFILES = \ -bitmap.h \ common.h \ counter.h \ disk.h \ @@ -49,7 +48,6 @@ vfs.h \ xfs_scrub.h CFILES = \ -bitmap.c \ common.c \ counter.c \ disk.c \ diff --git a/scrub/bitmap.c b/scrub/bitmap.c deleted file mode 100644 index aecdba0d..00000000 --- a/scrub/bitmap.c +++ /dev/null @@ -1,393 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Copyright (C) 2018 Oracle. All Rights Reserved. - * Author: Darrick J. Wong - */ -#include "xfs.h" -#include -#include -#include -#include -#include "platform_defs.h" -#include "avl64.h" -#include "list.h" -#include "bitmap.h" - -/* - * Space Efficient Bitmap - * - * Implements a space-efficient bitmap. We use an AVL tree to manage - * extent records that tell us which ranges are set; the bitmap key is - * an arbitrary uint64_t. The usual bitmap operations (set, clear, - * test, test and set) are supported, plus we can iterate set ranges. - */ - -#define avl_for_each_range_safe(pos, n, l, first, last) \ - for (pos = (first), n = pos->avl_nextino, l = (last)->avl_nextino; pos != (l); \ - pos = n, n = pos ? pos->avl_nextino : NULL) - -#define avl_for_each_safe(tree, pos, n) \ - for (pos = (tree)->avl_firstino, n = pos ? pos->avl_nextino : NULL; \ - pos != NULL; \ - pos = n, n = pos ? pos->avl_nextino : NULL) - -#define avl_for_each(tree, pos) \ - for (pos = (tree)->avl_firstino; pos != NULL; pos = pos->avl_nextino) - -struct bitmap_node { - struct avl64node btn_node; - uint64_t btn_start; - uint64_t btn_length; -}; - -static uint64_t -extent_start( - struct avl64node *node) -{ - struct bitmap_node *btn; - - btn = container_of(node, struct bitmap_node, btn_node); - return btn->btn_start; -} - -static uint64_t -extent_end( - struct avl64node *node) -{ - struct bitmap_node *btn; - - btn = container_of(node, struct bitmap_node, btn_node); - return btn->btn_start + btn->btn_length; -} - -static struct avl64ops bitmap_ops = { - extent_start, - extent_end, -}; - -/* Initialize a bitmap. */ -bool -bitmap_init( - struct bitmap **bmapp) -{ - struct bitmap *bmap; - - bmap = calloc(1, sizeof(struct bitmap)); - if (!bmap) - return false; - bmap->bt_tree = malloc(sizeof(struct avl64tree_desc)); - if (!bmap->bt_tree) { - free(bmap); - return false; - } - - pthread_mutex_init(&bmap->bt_lock, NULL); - avl64_init_tree(bmap->bt_tree, &bitmap_ops); - *bmapp = bmap; - - return true; -} - -/* Free a bitmap. */ -void -bitmap_free( - struct bitmap **bmapp) -{ - struct bitmap *bmap; - struct avl64node *node; - struct avl64node *n; - struct bitmap_node *ext; - - bmap = *bmapp; - avl_for_each_safe(bmap->bt_tree, node, n) { - ext = container_of(node, struct bitmap_node, btn_node); - free(ext); - } - free(bmap->bt_tree); - *bmapp = NULL; -} - -/* Create a new bitmap extent node. */ -static struct bitmap_node * -bitmap_node_init( - uint64_t start, - uint64_t len) -{ - struct bitmap_node *ext; - - ext = malloc(sizeof(struct bitmap_node)); - if (!ext) - return NULL; - - ext->btn_node.avl_nextino = NULL; - ext->btn_start = start; - ext->btn_length = len; - - return ext; -} - -/* Set a region of bits (locked). */ -static bool -__bitmap_set( - struct bitmap *bmap, - uint64_t start, - uint64_t length) -{ - struct avl64node *firstn; - struct avl64node *lastn; - struct avl64node *pos; - struct avl64node *n; - struct avl64node *l; - struct bitmap_node *ext; - uint64_t new_start; - uint64_t new_length; - struct avl64node *node; - bool res = true; - - /* Find any existing nodes adjacent or within that range. */ - avl64_findranges(bmap->bt_tree, start - 1, start + length + 1, - &firstn, &lastn); - - /* Nothing, just insert a new extent. */ - if (firstn == NULL && lastn == NULL) { - ext = bitmap_node_init(start, length); - if (!ext) - return false; - - node = avl64_insert(bmap->bt_tree, &ext->btn_node); - if (node == NULL) { - free(ext); - errno = EEXIST; - return false; - } - - return true; - } - - assert(firstn != NULL && lastn != NULL); - new_start = start; - new_length = length; - - avl_for_each_range_safe(pos, n, l, firstn, lastn) { - ext = container_of(pos, struct bitmap_node, btn_node); - - /* Bail if the new extent is contained within an old one. */ - if (ext->btn_start <= start && - ext->btn_start + ext->btn_length >= start + length) - return res; - - /* Check for overlapping and adjacent extents. */ - if (ext->btn_start + ext->btn_length >= start || - ext->btn_start <= start + length) { - if (ext->btn_start < start) { - new_start = ext->btn_start; - new_length += ext->btn_length; - } - - if (ext->btn_start + ext->btn_length > - new_start + new_length) - new_length = ext->btn_start + ext->btn_length - - new_start; - - avl64_delete(bmap->bt_tree, pos); - free(ext); - } - } - - ext = bitmap_node_init(new_start, new_length); - if (!ext) - return false; - - node = avl64_insert(bmap->bt_tree, &ext->btn_node); - if (node == NULL) { - free(ext); - errno = EEXIST; - return false; - } - - return res; -} - -/* Set a region of bits. */ -bool -bitmap_set( - struct bitmap *bmap, - uint64_t start, - uint64_t length) -{ - bool res; - - pthread_mutex_lock(&bmap->bt_lock); - res = __bitmap_set(bmap, start, length); - pthread_mutex_unlock(&bmap->bt_lock); - - return res; -} - -#if 0 /* Unused, provided for completeness. */ -/* Clear a region of bits. */ -bool -bitmap_clear( - struct bitmap *bmap, - uint64_t start, - uint64_t len) -{ - struct avl64node *firstn; - struct avl64node *lastn; - struct avl64node *pos; - struct avl64node *n; - struct avl64node *l; - struct bitmap_node *ext; - uint64_t new_start; - uint64_t new_length; - struct avl64node *node; - int stat; - - pthread_mutex_lock(&bmap->bt_lock); - /* Find any existing nodes over that range. */ - avl64_findranges(bmap->bt_tree, start, start + len, &firstn, &lastn); - - /* Nothing, we're done. */ - if (firstn == NULL && lastn == NULL) { - pthread_mutex_unlock(&bmap->bt_lock); - return true; - } - - assert(firstn != NULL && lastn != NULL); - - /* Delete or truncate everything in sight. */ - avl_for_each_range_safe(pos, n, l, firstn, lastn) { - ext = container_of(pos, struct bitmap_node, btn_node); - - stat = 0; - if (ext->btn_start < start) - stat |= 1; - if (ext->btn_start + ext->btn_length > start + len) - stat |= 2; - switch (stat) { - case 0: - /* Extent totally within range; delete. */ - avl64_delete(bmap->bt_tree, pos); - free(ext); - break; - case 1: - /* Extent is left-adjacent; truncate. */ - ext->btn_length = start - ext->btn_start; - break; - case 2: - /* Extent is right-adjacent; move it. */ - ext->btn_length = ext->btn_start + ext->btn_length - - (start + len); - ext->btn_start = start + len; - break; - case 3: - /* Extent overlaps both ends. */ - ext->btn_length = start - ext->btn_start; - new_start = start + len; - new_length = ext->btn_start + ext->btn_length - - new_start; - - ext = bitmap_node_init(new_start, new_length); - if (!ext) - return false; - - node = avl64_insert(bmap->bt_tree, &ext->btn_node); - if (node == NULL) { - errno = EEXIST; - return false; - } - break; - } - } - - pthread_mutex_unlock(&bmap->bt_lock); - return true; -} -#endif - -#ifdef DEBUG -/* Iterate the set regions of this bitmap. */ -bool -bitmap_iterate( - struct bitmap *bmap, - bool (*fn)(uint64_t, uint64_t, void *), - void *arg) -{ - struct avl64node *node; - struct bitmap_node *ext; - bool moveon = true; - - pthread_mutex_lock(&bmap->bt_lock); - avl_for_each(bmap->bt_tree, node) { - ext = container_of(node, struct bitmap_node, btn_node); - moveon = fn(ext->btn_start, ext->btn_length, arg); - if (!moveon) - break; - } - pthread_mutex_unlock(&bmap->bt_lock); - - return moveon; -} -#endif - -/* Do any bitmap extents overlap the given one? (locked) */ -static bool -__bitmap_test( - struct bitmap *bmap, - uint64_t start, - uint64_t len) -{ - struct avl64node *firstn; - struct avl64node *lastn; - - /* Find any existing nodes over that range. */ - avl64_findranges(bmap->bt_tree, start, start + len, &firstn, &lastn); - - return firstn != NULL && lastn != NULL; -} - -/* Is any part of this range set? */ -bool -bitmap_test( - struct bitmap *bmap, - uint64_t start, - uint64_t len) -{ - bool res; - - pthread_mutex_lock(&bmap->bt_lock); - res = __bitmap_test(bmap, start, len); - pthread_mutex_unlock(&bmap->bt_lock); - - return res; -} - -/* Are none of the bits set? */ -bool -bitmap_empty( - struct bitmap *bmap) -{ - return bmap->bt_tree->avl_firstino == NULL; -} - -#ifdef DEBUG -static bool -bitmap_dump_fn( - uint64_t startblock, - uint64_t blockcount, - void *arg) -{ - printf("%"PRIu64":%"PRIu64"\n", startblock, blockcount); - return true; -} - -/* Dump bitmap. */ -void -bitmap_dump( - struct bitmap *bmap) -{ - printf("BITMAP DUMP %p\n", bmap); - bitmap_iterate(bmap, bitmap_dump_fn, NULL); - printf("BITMAP DUMP DONE\n"); -} -#endif diff --git a/scrub/bitmap.h b/scrub/bitmap.h deleted file mode 100644 index e9746a12..00000000 --- a/scrub/bitmap.h +++ /dev/null @@ -1,24 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Copyright (C) 2018 Oracle. All Rights Reserved. - * Author: Darrick J. Wong - */ -#ifndef XFS_SCRUB_BITMAP_H_ -#define XFS_SCRUB_BITMAP_H_ - -struct bitmap { - pthread_mutex_t bt_lock; - struct avl64tree_desc *bt_tree; -}; - -bool bitmap_init(struct bitmap **bmap); -void bitmap_free(struct bitmap **bmap); -bool bitmap_set(struct bitmap *bmap, uint64_t start, uint64_t length); -bool bitmap_iterate(struct bitmap *bmap, - bool (*fn)(uint64_t, uint64_t, void *), void *arg); -bool bitmap_test(struct bitmap *bmap, uint64_t start, - uint64_t len); -bool bitmap_empty(struct bitmap *bmap); -void bitmap_dump(struct bitmap *bmap); - -#endif /* XFS_SCRUB_BITMAP_H_ */ From patchwork Thu Mar 14 21:06:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853641 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6759417E6 for ; Thu, 14 Mar 2019 21:06:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4CAFB29E7B for ; Thu, 14 Mar 2019 21:06:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4AED42A784; Thu, 14 Mar 2019 21:06:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D16F29E7B for ; Thu, 14 Mar 2019 21:06:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727749AbfCNVG4 (ORCPT ); Thu, 14 Mar 2019 17:06:56 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:52934 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727412AbfCNVGz (ORCPT ); Thu, 14 Mar 2019 17:06:55 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwdUL132771; Thu, 14 Mar 2019 21:06:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=SpoO6AT32xwovdvNyCdgHgpf05ozpoq3YCmN/FzZ8Ws=; b=jRbQy4bT8m5WUDjBspFHBYtX/n9Sj6m4AUbd+rc2W2Z+qa8vFXD9nHYk0fgb9ruRFoME Fd7vsRs1wlMSxmLng9WRMwN+5Vs5hVTFo2LMlTt6GpExc1ns+6HnG8dMV+iifuxQEOMo MfXmokJA+GXlGUjeladuM1QtSBxeevbLBXBHftTKdOafK42al5+B1ZtkXRwV4wkaLIox FU+NtyUMu/Huu35GtF7naBLJQB3zMRq4/Uo3Zc3lNWvtnoqLDJjwE8dU4/38o7nefT2Q JRBfUWSOx7kI9xzHw8Fo7rd70wR3dPh4g1u7IrEw1MvfW7wDpHFigm3DuAHVjXiI8zhI jQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2r430f3xvk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:53 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6qUB011000 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:52 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6plV003930; Thu, 14 Mar 2019 21:06:52 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:51 +0000 Subject: [PATCH 29/36] xfs_repair: correctly account for free space btree shrinks when fixing freelist From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:06:51 -0700 Message-ID: <155259761133.31886.7944119751393420937.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong When we fix the freelist at the end of build_agf_agfl in phase 5 of repair, we need to create incore rmap records for the blocks that get added to the AGFL. We can't let the regular freelist fixing code use the regular on-disk rmapbt update code because the rmapbt isn't fully set up yet. Unfortunately, the original code fails to account for the fact that the free space btrees can shrink when we allocate blocks to fix the freelist; those blocks are also put on the freelist, but there are already incore rmaps for all the free space btree blocks. We must not create (redundant) incore rmaps for those blocks. If we do, repair fails with a complaint that rebuilding the rmapbt failed during phase 5. xfs/137 on a 1k block size occasionally triggers this bug. To fix the problem, construct a bitmap of all OWN_AG blocks that we know about before traversing the AGFL, and only create new incore rmaps for those AGFL blocks that are not already tracked in the bitmap. Signed-off-by: Darrick J. Wong --- repair/rmap.c | 54 ++++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 44 insertions(+), 10 deletions(-) diff --git a/repair/rmap.c b/repair/rmap.c index d0156f9d..19cceca3 100644 --- a/repair/rmap.c +++ b/repair/rmap.c @@ -12,6 +12,7 @@ #include "dinode.h" #include "slab.h" #include "rmap.h" +#include "bitmap.h" #undef RMAP_DEBUG @@ -450,15 +451,16 @@ rmap_store_ag_btree_rec( struct xfs_buf *agflbp = NULL; struct xfs_trans *tp; __be32 *agfl_bno, *b; + struct xfs_ag_rmap *ag_rmap = &ag_rmaps[agno]; + struct bitmap *own_ag_bitmap = NULL; int error = 0; if (!xfs_sb_version_hasrmapbt(&mp->m_sb)) return 0; /* Release the ar_rmaps; they were put into the rmapbt during p5. */ - free_slab(&ag_rmaps[agno].ar_rmaps); - error = init_slab(&ag_rmaps[agno].ar_rmaps, - sizeof(struct xfs_rmap_irec)); + free_slab(&ag_rmap->ar_rmaps); + error = init_slab(&ag_rmap->ar_rmaps, sizeof(struct xfs_rmap_irec)); if (error) goto err; @@ -478,19 +480,50 @@ rmap_store_ag_btree_rec( * rmap, we only need to add rmap records for AGFL blocks past * that point in the AGFL because those blocks are a result of a * no-rmap no-shrink freelist fixup that we did earlier. + * + * However, some blocks end up on the AGFL because the free space + * btrees shed blocks as a result of allocating space to fix the + * freelist. We already created in-core rmap records for the free + * space btree blocks, so we must be careful not to create those + * records again. Create a bitmap of already-recorded OWN_AG rmaps. */ + error = init_slab_cursor(ag_rmap->ar_raw_rmaps, rmap_compare, &rm_cur); + if (error) + goto err; + if (!bitmap_init(&own_ag_bitmap)) { + error = -ENOMEM; + goto err_slab; + } + while ((rm_rec = pop_slab_cursor(rm_cur)) != NULL) { + if (rm_rec->rm_owner != XFS_RMAP_OWN_AG) + continue; + if (!bitmap_set(own_ag_bitmap, rm_rec->rm_startblock, + rm_rec->rm_blockcount)) { + error = EFSCORRUPTED; + goto err_slab; + } + } + free_slab_cursor(&rm_cur); + + /* Create rmaps for any AGFL blocks that aren't already rmapped. */ agfl_bno = XFS_BUF_TO_AGFL_BNO(mp, agflbp); - b = agfl_bno + ag_rmaps[agno].ar_flcount; + b = agfl_bno + ag_rmap->ar_flcount; while (*b != cpu_to_be32(NULLAGBLOCK) && b - agfl_bno < libxfs_agfl_size(mp)) { - error = rmap_add_ag_rec(mp, agno, be32_to_cpu(*b), 1, - XFS_RMAP_OWN_AG); - if (error) - goto err; + xfs_agblock_t agbno; + + agbno = be32_to_cpu(*b); + if (!bitmap_test(own_ag_bitmap, agbno, 1)) { + error = rmap_add_ag_rec(mp, agno, agbno, 1, + XFS_RMAP_OWN_AG); + if (error) + goto err; + } b++; } libxfs_putbuf(agflbp); agflbp = NULL; + bitmap_free(&own_ag_bitmap); /* Merge all the raw rmaps into the main list */ error = rmap_fold_raw_recs(mp, agno); @@ -498,8 +531,7 @@ rmap_store_ag_btree_rec( goto err; /* Create cursors to refcount structures */ - error = init_slab_cursor(ag_rmaps[agno].ar_rmaps, rmap_compare, - &rm_cur); + error = init_slab_cursor(ag_rmap->ar_rmaps, rmap_compare, &rm_cur); if (error) goto err; @@ -542,6 +574,8 @@ rmap_store_ag_btree_rec( err: if (agflbp) libxfs_putbuf(agflbp); + if (own_ag_bitmap) + bitmap_free(&own_ag_bitmap); return error; } From patchwork Thu Mar 14 21:06:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853645 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E478A1515 for ; Thu, 14 Mar 2019 21:07:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC3BA2A1E1 for ; Thu, 14 Mar 2019 21:07:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CAC582A23B; Thu, 14 Mar 2019 21:07:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A3A42A296 for ; Thu, 14 Mar 2019 21:07:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727760AbfCNVHI (ORCPT ); Thu, 14 Mar 2019 17:07:08 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:53594 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727204AbfCNVHH (ORCPT ); Thu, 14 Mar 2019 17:07:07 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwfks095309; Thu, 14 Mar 2019 21:07:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=U9x2grLXIwFAZk9VosXftNJ02YbP7UP9ZHhDmdGIKkg=; b=EGJvS9DTEmozpejufKmkspV9/h5j82dVrD7C3XhstN3cIFoKrvjlhS7cZYO1C31fQm15 4r05OioUd2NkR6K6B+8SQYO7fHdlCH1D6KtchY3FJ41vqSGGh7LEj5tJIGLnbTECQyGa Gk6VO010/EzN8okcHN6NyBaRH/j5NCCad9bRaJ6TIwX7NGLGTF3ie9XdE8nJWwe/U6gx 5J1Y0Q+lpKKPw9TfKLgxMTK1yswca8SFhumNft1SNfLn5Da5Hemq3MR6CMb/AfqcAz86 5K7UKpuLZ1yEuSo3+esIwM6tGu0Uk6KOgKgi9gK7Ev0WmUqie02O5hcIdv1mTJ9WAqSy wg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2r44wukftq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:04 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL6we8008321 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:06:58 GMT Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL6wN5020649; Thu, 14 Mar 2019 21:06:58 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:06:57 +0000 Subject: [PATCH 30/36] libxfs: retain ifork_ops when flushing inode From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Arkadiusz Miskiewicz Date: Thu, 14 Mar 2019 14:06:57 -0700 Message-ID: <155259761740.31886.12775146464846669280.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Retain the ifork ops used to validate the inode so that we can use the same one to iflush it. xfs_repair phase 6 can use multiple transactions to fix various inode problems, which means that the inode might not be fully fixed when each transaction commits. This can be a particular problem if there's a shortform directory with both invalid directory entries and incorrect i8count. Phase 3 will set the parent inode to "0" to signal to phase 6 that it needs to reset the parent and i8count, but phase 6 starts a transaction to junk the bad entries which fail to commit because the parent is invalid: fixing i8count in inode 69022994673 Invalid inode number 0x0 xfs_dir_ino_validate: XFS_ERROR_REPORT Metadata corruption detected at 0x464eb0, inode 0x10121750f1 data fork xfs_repair: warning - iflush_int failed (-117) And thus the inode fixes never get written out. Reported-by: Arkadiusz Miskiewicz Signed-off-by: Darrick J. Wong --- include/xfs_inode.h | 1 + libxfs/rdwr.c | 1 + libxfs/util.c | 2 +- 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/include/xfs_inode.h b/include/xfs_inode.h index 79ec3a2d..e1e8b430 100644 --- a/include/xfs_inode.h +++ b/include/xfs_inode.h @@ -51,6 +51,7 @@ typedef struct xfs_inode { xfs_fsize_t i_size; /* in-memory size */ const struct xfs_dir_ops *d_ops; /* directory ops vector */ + struct xfs_ifork_ops *i_fork_ops; /* fork verifiers */ struct inode i_vnode; } xfs_inode_t; diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c index a00360e7..69d5abb2 100644 --- a/libxfs/rdwr.c +++ b/libxfs/rdwr.c @@ -1391,6 +1391,7 @@ libxfs_iget( return error; } + ip->i_fork_ops = ifork_ops; if (!libxfs_inode_verify_forks(ip, ifork_ops)) { libxfs_irele(ip); return -EFSCORRUPTED; diff --git a/libxfs/util.c b/libxfs/util.c index bd414043..0799f965 100644 --- a/libxfs/util.c +++ b/libxfs/util.c @@ -420,7 +420,7 @@ libxfs_iflush_int(xfs_inode_t *ip, xfs_buf_t *bp) VFS_I(ip)->i_version++; /* Check the inline fork data before we write out. */ - if (!libxfs_inode_verify_forks(ip, &xfs_default_ifork_ops)) + if (!libxfs_inode_verify_forks(ip, ip->i_fork_ops)) return -EFSCORRUPTED; /* From patchwork Thu Mar 14 21:07:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853647 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B72FB17E6 for ; Thu, 14 Mar 2019 21:07:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A062B2A77B for ; Thu, 14 Mar 2019 21:07:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9ECF02A780; Thu, 14 Mar 2019 21:07:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 51FB42A790 for ; Thu, 14 Mar 2019 21:07:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727843AbfCNVHM (ORCPT ); Thu, 14 Mar 2019 17:07:12 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:53336 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727204AbfCNVHM (ORCPT ); Thu, 14 Mar 2019 17:07:12 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwpFC132843; Thu, 14 Mar 2019 21:07:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=MyLC/QkcsVswqlbcEJZIObOa7tHjyBtEUZXj8Ko8vUg=; b=k1zFk3FuXtL9JXLR/LExcDUdhJ2L6oHHwDVftuJcSy8oa6iqZn/gMPs5T2dhyxhqm5oD Z2UOF4ASvwhTYKFQvu9NAV3ZFlLIRhHPs3w5sSH86mIIxTVf8PZbAlVj3c+U2RcwJ2jT pchnp+j6aKSUARbXqTVGe2v5G7AEScHvVhwU4YSa6noo+KG65i+TcjgAuN6c+RvGjaE8 7NdGuWe+bLL6mANo7hZ6Y+9TwO23QjdrzIzcwllg3uxI3ZXKRWkofpLlJ0SqPv9dvpmb s7mpoOYDw8hNH7Ymsy+741sgqrR/cZaQ8KmBsqMuuxZ/JkPF5dVpcowuetuR/g/+3ShF 0g== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2r430f3xxv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:10 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL74uZ011706 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:04 GMT Received: from abhmp0004.oracle.com (abhmp0004.oracle.com [141.146.116.10]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL74iA007968; Thu, 14 Mar 2019 21:07:04 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:07:03 +0000 Subject: [PATCH 31/36] libxfs: refactor the open-coded libxfs_trans_bjoin calls From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:07:03 -0700 Message-ID: <155259762358.31886.5732556054840025852.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Refactor open-coded bjoin code to use libxfs_trans_bjoin. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- libxfs/trans.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/libxfs/trans.c b/libxfs/trans.c index 10a35dd4..7bdd1544 100644 --- a/libxfs/trans.c +++ b/libxfs/trans.c @@ -663,13 +663,9 @@ libxfs_trans_get_buf_map( fprintf(stderr, "trans_get_buf buffer %p, transaction %p\n", bp, tp); #endif - xfs_buf_item_init(bp, tp->t_mountp); + libxfs_trans_bjoin(tp, bp); bip = bp->b_log_item; bip->bli_recur = 0; - xfs_trans_add_item(tp, (xfs_log_item_t *)bip); - - /* initialize b_transp so we can find it incore */ - bp->b_transp = tp; return bp; } @@ -701,13 +697,9 @@ libxfs_trans_getsb( fprintf(stderr, "trans_get_sb buffer %p, transaction %p\n", bp, tp); #endif - xfs_buf_item_init(bp, mp); + libxfs_trans_bjoin(tp, bp); bip = bp->b_log_item; bip->bli_recur = 0; - xfs_trans_add_item(tp, (xfs_log_item_t *)bip); - - /* initialize b_transp so we can find it incore */ - bp->b_transp = tp; return bp; } @@ -758,13 +750,9 @@ libxfs_trans_read_buf_map( fprintf(stderr, "trans_read_buf buffer %p, transaction %p\n", bp, tp); #endif - xfs_buf_item_init(bp, tp->t_mountp); + xfs_trans_bjoin(tp, bp); bip = bp->b_log_item; bip->bli_recur = 0; - xfs_trans_add_item(tp, (xfs_log_item_t *)bip); - - /* initialise b_transp so we can find it incore */ - bp->b_transp = tp; done: *bpp = bp; return 0; From patchwork Thu Mar 14 21:07:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853649 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0707F1874 for ; Thu, 14 Mar 2019 21:07:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4BE22A199 for ; Thu, 14 Mar 2019 21:07:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E33562A791; Thu, 14 Mar 2019 21:07:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9832F2A199 for ; Thu, 14 Mar 2019 21:07:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727204AbfCNVHN (ORCPT ); Thu, 14 Mar 2019 17:07:13 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:53112 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727832AbfCNVHM (ORCPT ); Thu, 14 Mar 2019 17:07:12 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwi9O102514; Thu, 14 Mar 2019 21:07:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=B9xepTNSmKTjjt2VM1lVrh/OHUM1mmDIMjrR0jpRczc=; b=QCzScao0UtAtfYcEGATEMfcv2TnMMdeGG1yuC7brWcX4Z14oSe8bkXSJ1OyDK0zK1vMj e11Kd3U/BCv7ZNBzj1ta2/gSruQJVtloULyOUbDGhs6mzazZ30t3JbOh6xfSlR1+sN5B UwylAAgXS0O9QPjG/hMKNMQ5ArtuTtiKnoX55mevwmiwZqSm9RDA8Ru8tZsxdMdZfz2r x2ID9tz+uSD78hguCPR9snhaSKrrwBM9TCd9BHSmbb31O06iWrolzault9OpfPpcp2ci s1Uv5wkklqqouiNg6Ymd37crKYJmjLvIuEBLhJNKd3Echq3cUFaoCqj1qj7e/w4eln2Q gg== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2r464ruesv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:10 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7AqJ011998 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:10 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7A2l004077; Thu, 14 Mar 2019 21:07:10 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:07:09 +0000 Subject: [PATCH 32/36] libxfs: refactor buffer item release code From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:07:09 -0700 Message-ID: <155259762965.31886.1477262840183695209.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Refactor the buffer item release code into a helper, which we will use in subsequent patches to make the buffer log item lifetime match the kernel equivalents. Signed-off-by: Darrick J. Wong --- libxfs/trans.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/libxfs/trans.c b/libxfs/trans.c index 7bdd1544..295e687e 100644 --- a/libxfs/trans.c +++ b/libxfs/trans.c @@ -540,6 +540,16 @@ libxfs_trans_ordered_buf( return ret; } +void +xfs_buf_item_put( + struct xfs_buf_log_item *bip) +{ + struct xfs_buf *bp = bip->bli_buf; + + bp->b_log_item = NULL; + kmem_zone_free(xfs_buf_item_zone, bip); +} + void libxfs_trans_brelse( xfs_trans_t *tp, @@ -881,7 +891,6 @@ buf_item_done( bp = bip->bli_buf; ASSERT(bp != NULL); - bp->b_log_item = NULL; /* remove log item */ bp->b_transp = NULL; /* remove xact ptr */ hold = (bip->bli_flags & XFS_BLI_HOLD); @@ -896,8 +905,7 @@ buf_item_done( bip->bli_flags &= ~XFS_BLI_HOLD; else libxfs_putbuf(bp); - /* release the buf item */ - kmem_zone_free(xfs_buf_item_zone, bip); + xfs_buf_item_put(bip); } static void From patchwork Thu Mar 14 21:07:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853651 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 498FC1515 for ; Thu, 14 Mar 2019 21:07:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3247129DEC for ; Thu, 14 Mar 2019 21:07:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 252F12A031; Thu, 14 Mar 2019 21:07:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD36729DEC for ; Thu, 14 Mar 2019 21:07:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727877AbfCNVHT (ORCPT ); Thu, 14 Mar 2019 17:07:19 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:53832 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727832AbfCNVHT (ORCPT ); Thu, 14 Mar 2019 17:07:19 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwnUW095338; Thu, 14 Mar 2019 21:07:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=PMMsYwFEDWYSc6KNBYej0XCTCXFAtNWCIX5gtm1apkY=; b=hqPCUKa8EsTY3GGShq5Y9TO8x35RAfbMxiaifn543zl6PkA9hQNBraSf6ekxduf5PMEh 8mMGjuO4FpRTFg4Eug4pjGi/xWvkItaRJwjntME/tNDbjJm3nBHHqA4HDetwQvpKQmiV QLntPPdKz0isA4RnjRR+bqgw1pHSpeZYW+AJvmyT9H/Oku8g14pH5vH/ro9tJaRlAz+H wALscD6M09onqyEIFSsWWdyqQ336bloE+1ck3LJAJbFOUSOnYr9j9Umudtz5jz7OuTJV XZMhBNhcUnhq8qzX/BheCKlWbeea/ci9SemDqnA1i2qmkur2mJ8uUryAi2ptpgpOI1NF Tw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2r44wukfuh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:17 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7GRT009118 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:16 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7GvL008074; Thu, 14 Mar 2019 21:07:16 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:07:16 +0000 Subject: [PATCH 33/36] libxfs: don't touch buffer log item pointer when flushing inode log item From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:07:15 -0700 Message-ID: <155259763568.31886.8011442389519021944.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=841 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong When we're flushing an inode log item, it is not necessary to mess with the inode cluster buffer's log item because the iflush code paths pass the inode log item directly. The unconditional reset causes us to leak buffer log items. Signed-off-by: Darrick J. Wong --- libxfs/trans.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/libxfs/trans.c b/libxfs/trans.c index 295e687e..bd7bbc69 100644 --- a/libxfs/trans.c +++ b/libxfs/trans.c @@ -861,10 +861,8 @@ inode_item_done( * of whether the flush succeed or not. If we fail the flush, make sure * we still release the buffer reference we currently hold. */ - bp->b_log_item = iip; error = libxfs_iflush_int(ip, bp); ip->i_transp = NULL; /* disassociate from transaction */ - bp->b_log_item = NULL; /* remove log item */ bp->b_transp = NULL; /* remove xact ptr */ if (error) { From patchwork Thu Mar 14 21:07:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853653 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DCA921515 for ; Thu, 14 Mar 2019 21:07:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C489229DEC for ; Thu, 14 Mar 2019 21:07:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B8BA929E96; Thu, 14 Mar 2019 21:07:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 66D8E29DEC for ; Thu, 14 Mar 2019 21:07:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727890AbfCNVH0 (ORCPT ); Thu, 14 Mar 2019 17:07:26 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:53626 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727832AbfCNVHZ (ORCPT ); Thu, 14 Mar 2019 17:07:25 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwrtK132866; Thu, 14 Mar 2019 21:07:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=2tMS7JHlSE+BRUHvAHtlSakq1/C2N1uDVbPQJB8oIpI=; b=hWte5pdUW+AYVRQ/h4IcBQwppm4LudVZz+Y24uraxbzdsezEie/jb9kaqRbjFlqFdXmS byYvO55uGY/YPlxch4GGyDpZEbzbIbG/TIn6hVH7Z7ryJgwatcjxJiVIfecuOEq237e0 jdHv7uuFYVsouTrSnJWt7gCOX5+P+Jk+/bmaxEcRJH4Z/xjx5dZ2qFsC8PI/xi8BaCKn baVt/08ewSpuYdUK9PwIS/8E7Kxv+98HQmPwNNKYt1+FpNzOlS0AL3Y12D/7vxQOPTcy HKbH4fWFy5qkLs7OQ/SHh6Yu7jFjascR+NmoztOWRe/rGKfdgCRwYCI0HRk3gxMqzK65 8Q== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2r430f3xyk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:23 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7M7q012513 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:22 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2EL7Mgc020874; Thu, 14 Mar 2019 21:07:22 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:07:22 +0000 Subject: [PATCH 34/36] libxfs: fix buffer log item lifetime weirdness From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:07:21 -0700 Message-ID: <155259764175.31886.6428125727142055154.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=702 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In xfsprogs, the lifetime of xfs_buf log items doesn't match the kernel because we keep them around after comitting or cancelling transactions. This is confusing, so change the lifetime to be consistent. Worse yet, if an inode cluster buffer gets bjoined to a transaction (e.g. someone called xfs_trans_read_buf) we'll leak it when flushing an inode core back to that buffer. Signed-off-by: Darrick J. Wong --- libxfs/trans.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libxfs/trans.c b/libxfs/trans.c index bd7bbc69..2ceb9838 100644 --- a/libxfs/trans.c +++ b/libxfs/trans.c @@ -580,6 +580,7 @@ libxfs_trans_brelse( xfs_trans_del_item(&bip->bli_item); if (bip->bli_flags & XFS_BLI_HOLD) bip->bli_flags &= ~XFS_BLI_HOLD; + xfs_buf_item_put(bip); bp->b_transp = NULL; libxfs_putbuf(bp); } @@ -939,6 +940,7 @@ buf_item_unlock( hold = bip->bli_flags & XFS_BLI_HOLD; bip->bli_flags &= ~XFS_BLI_HOLD; + xfs_buf_item_put(bip); if (!hold) libxfs_putbuf(bp); } From patchwork Thu Mar 14 21:07:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853655 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5994F1874 for ; Thu, 14 Mar 2019 21:07:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 422B929E7B for ; Thu, 14 Mar 2019 21:07:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3636C29DEC; Thu, 14 Mar 2019 21:07:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CEBD429DEC for ; Thu, 14 Mar 2019 21:07:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727832AbfCNVHh (ORCPT ); Thu, 14 Mar 2019 17:07:37 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:54172 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727366AbfCNVHh (ORCPT ); Thu, 14 Mar 2019 17:07:37 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwlPq095329; Thu, 14 Mar 2019 21:07:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=qG6fA622RSt/osajklYcB3EuBAt93j18pHNBfl2pYCQ=; b=fHJe5nlPEWeq5vhQVushjcOaaBbQ4i6ee1MIhIqE8h/cMrOcIN/ilNYfdvycDo6cXsb4 z4S8ZSFqUMgZjsqBLjQ5W1lGT9ahkw3i4BOu4T8kNM8QsFHkkTmh/SWm8DyKBrxDsX0t WsGVQVS9vii5dGwReXOX+krpw9bMGoWpATSnJvMIKiSH7MvJFclkqqmFkLlPqEdym1au mfJq9AT+D3BD2786PAp4M2lpf1OfrCMTizzLqUBIVQMAk2oM+j4LRg2Zp0WK3tOksJsc Iw/7rDeyahpHOXRkC0rmZMl/VEbKxRfFHFcRcWf6InHOzHwusqKr7+KfkwHkY9TR8iN0 Dw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2r44wukfvd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:34 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7ShB009733 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:29 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7SYq004147; Thu, 14 Mar 2019 21:07:28 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:07:28 +0000 Subject: [PATCH 35/36] libxfs: shorten inode item lifetime From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:07:27 -0700 Message-ID: <155259764777.31886.5969876849048391381.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=823 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Shorten the inode item lifetime so that we only keep them around while the inode is joined with a transaction. Signed-off-by: Darrick J. Wong --- libxfs/rdwr.c | 4 +--- libxfs/trans.c | 19 ++++++++++++++++--- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c index 69d5abb2..fd8c6591 100644 --- a/libxfs/rdwr.c +++ b/libxfs/rdwr.c @@ -1429,9 +1429,7 @@ void libxfs_irele( struct xfs_inode *ip) { - if (ip->i_itemp) - kmem_zone_free(xfs_ili_zone, ip->i_itemp); - ip->i_itemp = NULL; + ASSERT(ip->i_itemp == NULL); libxfs_idestroy(ip); kmem_zone_free(xfs_inode_zone, ip); } diff --git a/libxfs/trans.c b/libxfs/trans.c index 2ceb9838..ce199654 100644 --- a/libxfs/trans.c +++ b/libxfs/trans.c @@ -820,6 +820,16 @@ _("Transaction block reservation exceeded! %u > %u\n"), tp->t_flags |= (XFS_TRANS_SB_DIRTY | XFS_TRANS_DIRTY); } +void +xfs_inode_item_put( + struct xfs_inode_log_item *iip) +{ + struct xfs_inode *ip = iip->ili_inode; + + ip->i_itemp = NULL; + kmem_zone_free(xfs_ili_zone, iip); +} + /* * Transaction commital code follows (i.e. write to disk in libxfs) @@ -844,7 +854,7 @@ inode_item_done( if (!(iip->ili_fields & XFS_ILOG_ALL)) { ip->i_transp = NULL; /* disassociate from transaction */ iip->ili_flags = 0; /* reset all flags */ - return; + goto free; } /* @@ -854,7 +864,7 @@ inode_item_done( if (error) { fprintf(stderr, _("%s: warning - imap_to_bp failed (%d)\n"), progname, error); - return; + goto free; } /* @@ -870,7 +880,7 @@ inode_item_done( fprintf(stderr, _("%s: warning - iflush_int failed (%d)\n"), progname, error); libxfs_putbuf(bp); - return; + goto free; } libxfs_writebuf(bp, 0); @@ -878,6 +888,8 @@ inode_item_done( fprintf(stderr, "flushing dirty inode %llu, buffer %p\n", ip->i_ino, bp); #endif +free: + xfs_inode_item_put(iip); } static void @@ -955,6 +967,7 @@ inode_item_unlock( ip->i_transp = NULL; iip->ili_flags = 0; + xfs_inode_item_put(iip); } /* Detach and unlock all of the items in a transaction */ From patchwork Thu Mar 14 21:07:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10853657 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EA54F17E6 for ; Thu, 14 Mar 2019 21:07:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D07C129DEC for ; Thu, 14 Mar 2019 21:07:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C4A7D29E96; Thu, 14 Mar 2019 21:07:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EF2C29DEC for ; Thu, 14 Mar 2019 21:07:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727894AbfCNVHj (ORCPT ); Thu, 14 Mar 2019 17:07:39 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:53848 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727366AbfCNVHj (ORCPT ); Thu, 14 Mar 2019 17:07:39 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2EKwsNC132890; Thu, 14 Mar 2019 21:07:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=iF73ka7hLYvkMICkEGwAtdCd7wEAsrfcPPBTKKgePjg=; b=086Df8C1YAKx3P7CUReTvAwpN3dLRat24aBoCo/KuFT9xvKNCPVSBj2p01JQjCJ071m3 PSYHNZ6OHtM0/6iuB/5ZIIzHmy7yQKv/ShERPWelMVvpyRaAuRp11KHNztZYqOWU18io 4JJyLfd1y4q8erYeGhhAQOGF7jRynauZGhH7QtTy7bI2H4JK4fXRgCcWm/JnpkCBAary O6TGlBTugZIY2XoWwQRtLLmPfVivDd1/7AfpDqQv1p7F8M2dW8HaK9kDQZi9et/WnWOL cP9GgPHBPe1WIHQM/LabL4xDIi9ExU410fTa/AL34qjG0Re9Dq0x+VK0MTxd0JCGJDMs 7A== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2130.oracle.com with ESMTP id 2r430f3y09-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:36 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7a7h025389 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Mar 2019 21:07:36 GMT Received: from abhmp0004.oracle.com (abhmp0004.oracle.com [141.146.116.10]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2EL7ZV9004286; Thu, 14 Mar 2019 21:07:35 GMT Received: from localhost (/10.145.178.102) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Mar 2019 21:07:35 +0000 Subject: [PATCH 36/36] libxfs: remove libxfs_trans_iget From: "Darrick J. Wong" To: sandeen@sandeen.net, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Thu, 14 Mar 2019 14:07:33 -0700 Message-ID: <155259765381.31886.8973943242633989393.stgit@magnolia> In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140143 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong libxfs_trans_iget no longer has a counterpart in the kernel. Remove it and make the xfs_iget/xfs_trans_ijoin usage consistent throughout xfsprogs. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- include/xfs_trans.h | 2 -- libxfs/libxfs_api_defs.h | 1 - libxfs/trans.c | 35 ----------------------------------- libxfs/util.c | 4 +++- repair/phase6.c | 21 +++++++++++++++------ repair/phase7.c | 4 ++-- 6 files changed, 20 insertions(+), 47 deletions(-) diff --git a/include/xfs_trans.h b/include/xfs_trans.h index 9e44d18e..e6bb74c4 100644 --- a/include/xfs_trans.h +++ b/include/xfs_trans.h @@ -98,8 +98,6 @@ void xfs_defer_cancel(struct xfs_trans *); struct xfs_buf *libxfs_trans_getsb(struct xfs_trans *, struct xfs_mount *, int); -int libxfs_trans_iget(struct xfs_mount *, struct xfs_trans *, xfs_ino_t, - uint, uint, struct xfs_inode **); void libxfs_trans_ijoin(struct xfs_trans *, struct xfs_inode *, uint); void libxfs_trans_ijoin_ref(struct xfs_trans *, struct xfs_inode *, int); void libxfs_trans_log_inode (struct xfs_trans *, struct xfs_inode *, diff --git a/libxfs/libxfs_api_defs.h b/libxfs/libxfs_api_defs.h index e10d78cd..bb0f07bd 100644 --- a/libxfs/libxfs_api_defs.h +++ b/libxfs/libxfs_api_defs.h @@ -31,7 +31,6 @@ #define xfs_trans_del_item libxfs_trans_del_item #define xfs_trans_get_buf libxfs_trans_get_buf #define xfs_trans_getsb libxfs_trans_getsb -#define xfs_trans_iget libxfs_trans_iget #define xfs_trans_ichgtime libxfs_trans_ichgtime #define xfs_trans_ijoin libxfs_trans_ijoin #define xfs_trans_init libxfs_trans_init diff --git a/libxfs/trans.c b/libxfs/trans.c index ce199654..d6785e20 100644 --- a/libxfs/trans.c +++ b/libxfs/trans.c @@ -336,41 +336,6 @@ libxfs_trans_cancel( return; } -int -libxfs_trans_iget( - xfs_mount_t *mp, - xfs_trans_t *tp, - xfs_ino_t ino, - uint flags, - uint lock_flags, - xfs_inode_t **ipp) -{ - int error; - xfs_inode_t *ip; - xfs_inode_log_item_t *iip; - - if (tp == NULL) - return libxfs_iget(mp, tp, ino, lock_flags, ipp, - &xfs_default_ifork_ops); - - error = libxfs_iget(mp, tp, ino, lock_flags, &ip, - &xfs_default_ifork_ops); - if (error) - return error; - ASSERT(ip != NULL); - - if (ip->i_itemp == NULL) - xfs_inode_item_init(ip, mp); - iip = ip->i_itemp; - xfs_trans_add_item(tp, (xfs_log_item_t *)(iip)); - - /* initialize i_transp so we can find it incore */ - ip->i_transp = tp; - - *ipp = ip; - return 0; -} - void libxfs_trans_ijoin( xfs_trans_t *tp, diff --git a/libxfs/util.c b/libxfs/util.c index 0799f965..2e3b9d51 100644 --- a/libxfs/util.c +++ b/libxfs/util.c @@ -262,7 +262,8 @@ libxfs_ialloc( } ASSERT(*ialloc_context == NULL); - error = xfs_trans_iget(tp->t_mountp, tp, ino, 0, 0, &ip); + error = libxfs_iget(tp->t_mountp, tp, ino, 0, &ip, + &xfs_default_ifork_ops); if (error != 0) return error; ASSERT(ip != NULL); @@ -376,6 +377,7 @@ libxfs_ialloc( /* * Log the new values stuffed into the inode. */ + xfs_trans_ijoin(tp, ip, 0); xfs_trans_log_inode(tp, ip, flags); *ipp = ip; return 0; diff --git a/repair/phase6.c b/repair/phase6.c index 194cfdbf..28e633de 100644 --- a/repair/phase6.c +++ b/repair/phase6.c @@ -535,7 +535,8 @@ mk_rbmino(xfs_mount_t *mp) if (i) res_failed(i); - error = -libxfs_trans_iget(mp, tp, mp->m_sb.sb_rbmino, 0, 0, &ip); + error = -libxfs_iget(mp, tp, mp->m_sb.sb_rbmino, 0, &ip, + &xfs_default_ifork_ops); if (error) { do_error( _("couldn't iget realtime bitmap inode -- error - %d\n"), @@ -572,6 +573,7 @@ mk_rbmino(xfs_mount_t *mp) /* * commit changes */ + libxfs_trans_ijoin(tp, ip, 0); libxfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); error = -libxfs_trans_commit(tp); if (error) @@ -634,7 +636,8 @@ fill_rbmino(xfs_mount_t *mp) if (error) res_failed(error); - error = -libxfs_trans_iget(mp, tp, mp->m_sb.sb_rbmino, 0, 0, &ip); + error = -libxfs_iget(mp, tp, mp->m_sb.sb_rbmino, 0, &ip, + &xfs_default_ifork_ops); if (error) { do_error( _("couldn't iget realtime bitmap inode -- error - %d\n"), @@ -645,6 +648,7 @@ fill_rbmino(xfs_mount_t *mp) /* * fill the file one block at a time */ + libxfs_trans_ijoin(tp, ip, 0); nmap = 1; error = -libxfs_bmapi_write(tp, ip, bno, 1, 0, 1, &map, &nmap); if (error || nmap != 1) { @@ -703,7 +707,8 @@ fill_rsumino(xfs_mount_t *mp) if (error) res_failed(error); - error = -libxfs_trans_iget(mp, tp, mp->m_sb.sb_rsumino, 0, 0, &ip); + error = -libxfs_iget(mp, tp, mp->m_sb.sb_rsumino, 0, &ip, + &xfs_default_ifork_ops); if (error) { do_error( _("couldn't iget realtime summary inode -- error - %d\n"), @@ -714,6 +719,7 @@ fill_rsumino(xfs_mount_t *mp) /* * fill the file one block at a time */ + libxfs_trans_ijoin(tp, ip, 0); nmap = 1; error = -libxfs_bmapi_write(tp, ip, bno, 1, 0, 1, &map, &nmap); if (error || nmap != 1) { @@ -775,7 +781,8 @@ mk_rsumino(xfs_mount_t *mp) if (i) res_failed(i); - error = -libxfs_trans_iget(mp, tp, mp->m_sb.sb_rsumino, 0, 0, &ip); + error = -libxfs_iget(mp, tp, mp->m_sb.sb_rsumino, 0, &ip, + &xfs_default_ifork_ops); if (error) { do_error( _("couldn't iget realtime summary inode -- error - %d\n"), @@ -812,6 +819,7 @@ mk_rsumino(xfs_mount_t *mp) /* * commit changes */ + libxfs_trans_ijoin(tp, ip, 0); libxfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); error = -libxfs_trans_commit(tp); if (error) @@ -875,7 +883,8 @@ mk_root_dir(xfs_mount_t *mp) if (i) res_failed(i); - error = -libxfs_trans_iget(mp, tp, mp->m_sb.sb_rootino, 0, 0, &ip); + error = -libxfs_iget(mp, tp, mp->m_sb.sb_rootino, 0, &ip, + &xfs_default_ifork_ops); if (error) { do_error(_("could not iget root inode -- error - %d\n"), error); } @@ -900,7 +909,7 @@ mk_root_dir(xfs_mount_t *mp) times |= XFS_ICHGTIME_CREATE; } libxfs_trans_ichgtime(tp, ip, times); - + libxfs_trans_ijoin(tp, ip, 0); libxfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); /* diff --git a/repair/phase7.c b/repair/phase7.c index c2a60a93..c2996470 100644 --- a/repair/phase7.c +++ b/repair/phase7.c @@ -32,8 +32,7 @@ update_inode_nlinks( error = -libxfs_trans_alloc(mp, &M_RES(mp)->tr_remove, nres, 0, 0, &tp); ASSERT(error == 0); - error = -libxfs_trans_iget(mp, tp, ino, 0, 0, &ip); - + error = -libxfs_iget(mp, tp, ino, 0, &ip, &xfs_default_ifork_ops); if (error) { if (!no_modify) do_error( @@ -67,6 +66,7 @@ update_inode_nlinks( if (!dirty) { libxfs_trans_cancel(tp); } else { + libxfs_trans_ijoin(tp, ip, 0); libxfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); /* * no need to do a bmap finish since From patchwork Wed Mar 20 19:34:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862523 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4217A17E9 for ; Wed, 20 Mar 2019 19:34:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E030B2980D for ; Wed, 20 Mar 2019 19:34:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D313B29C07; Wed, 20 Mar 2019 19:34:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E711528CCF for ; Wed, 20 Mar 2019 19:34:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726115AbfCTTe0 (ORCPT ); Wed, 20 Mar 2019 15:34:26 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:59222 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726006AbfCTTe0 (ORCPT ); Wed, 20 Mar 2019 15:34:26 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KJXdpx151318; Wed, 20 Mar 2019 19:34:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=M/242KQ1vnxsy+gdJW/2RK4ClB1jyT6OZmbg5uCRZuk=; b=4wffDnm8uOax2uugVL5CeojqbSpIi0e5cgYW24J5mBHsO8BJt3HwA3kcDUDTbVFVdSFb jveoSLPGHyePCaSumXwc1sxmV8nKUjK86fQVHNL1SWtfOAUOpke+snItlX9vlyQ8EFjs N9Cs9FQKbYwt5tKOPImXzNxqSAFVwtdRD6KiuT+u6Ljzn8ctn1HZu0C1wxcprY8HbD8b lDuUs9IhlvNNfY8lbAFDN08wX4Is29sMKtVl7DRjPL+kXvXdDawhMRXqWvusIltMh+OG ybMfLDYjiibehP9HXjKOQHtO/S12gNCz+Xh/Dq8A8Pi0AN3wJ34WU5+hY8u4mFEyUQsH 4g== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2r8pnew2y7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:34:23 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJYH5B030867 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:34:17 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJYGib000318; Wed, 20 Mar 2019 19:34:17 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 12:34:16 -0700 Date: Wed, 20 Mar 2019 12:34:17 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 37/36] xfs_scrub: include unicrash.h in unicrash.c Message-ID: <20190320193417.GA1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200142 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Fix broken function prototype checking in unicrash.c by actually including its header. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/unicrash.c | 1 + 1 file changed, 1 insertion(+) diff --git a/scrub/unicrash.c b/scrub/unicrash.c index 121eedbc..824b10f0 100644 --- a/scrub/unicrash.c +++ b/scrub/unicrash.c @@ -16,6 +16,7 @@ #include "path.h" #include "xfs_scrub.h" #include "common.h" +#include "unicrash.h" /* * Detect Unicode confusable names in directories and attributes. From patchwork Wed Mar 20 19:34:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862525 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16E471390 for ; Wed, 20 Mar 2019 19:34:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE4BA29E0D for ; Wed, 20 Mar 2019 19:34:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E290D29E0F; Wed, 20 Mar 2019 19:34:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 874052980D for ; Wed, 20 Mar 2019 19:34:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726403AbfCTTey (ORCPT ); Wed, 20 Mar 2019 15:34:54 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:57686 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726006AbfCTTey (ORCPT ); Wed, 20 Mar 2019 15:34:54 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KJXhUI120982; Wed, 20 Mar 2019 19:34:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=2C/5BX0vcQKQBaTNPUodfrRN2owvPsGBDTUB2gTzjO8=; b=VIJ2UiHTHNxDI6DwrvPnUJxLgcliDuA9LJmHIrDJyEXM89MIdtpk9misv1sjeJvjdaMQ 0EI9LldpdP0HFwai3J+iUkMTR2otVVJWR9SLM9l09PecRWefHbei5pWzr8nmoZv/MzTJ FHW6MwZh0GFQRKCCH4XvuDK4FDe8eHZnMMntmDRu2E1dMCCw6wxAq8PP5oHUcI9ISRvC mfYY6WbPihrXJI6Gm5v7xc7mCVO2lt+DxB80fixVEhaNfyrnA6arxqqduSDFyOIkUiTo Z9rZe6WESuCe2TXv8+aGs5s/XMHuBbR1Pl73uPSNZdpYhIuyK6nAeRLGDUXc0HsDwGuy FQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2r8ssrmqjj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:34:50 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJYiKp025260 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:34:44 GMT Received: from abhmp0014.oracle.com (abhmp0014.oracle.com [141.146.116.20]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJYhda019918; Wed, 20 Mar 2019 19:34:44 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 12:34:43 -0700 Date: Wed, 20 Mar 2019 12:34:44 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 38/36] xfs_io: don't read garbage stack contents if INUMBERS goes nuts Message-ID: <20190320193444.GB1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200142 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In theory INUMBERS will never return an ocount of zero, but on the off chance it ever does we'll negative index the igroup array and return stack contents for an inode number. Don't do that. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- io/open.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/io/open.c b/io/open.c index f5fbd2c4..a406ea54 100644 --- a/io/open.c +++ b/io/open.c @@ -698,6 +698,9 @@ get_last_inode(void) lastgrp = ocount; } + if (lastgrp == 0) + return 0; + lastgrp--; /* The last inode number in use */ From patchwork Wed Mar 20 19:35:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862527 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3A4D1390 for ; Wed, 20 Mar 2019 19:35:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B6FF128E34 for ; Wed, 20 Mar 2019 19:35:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ABC3D29E0B; Wed, 20 Mar 2019 19:35:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2BAE829E7D for ; Wed, 20 Mar 2019 19:35:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726006AbfCTTfu (ORCPT ); Wed, 20 Mar 2019 15:35:50 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:49484 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726115AbfCTTfu (ORCPT ); Wed, 20 Mar 2019 15:35:50 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KJY1gX116953; Wed, 20 Mar 2019 19:35:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=0IQo/AQCSSOkwS5rzmd6/Pkt3f7DF04CpyJTzLoVUnc=; b=iZEwQ53onikRDD7atslOUq4KWCNkzp2/m0T68Qsxe0fJlI9bhPjSYHNBjR3rHqVqExWY c0HI/UPNAyBYbn39D8Wc2DJ/tXjQBufZZoTUYwhoAfoRxBRw9PwSWf38MmLADDcYZ5NE s5DAYIx9n1NW0dX8IlvJ3V9PgdKJpJSeIYSazhrZ3gMBpJhSH8I+cxBzmPtOkI/IREYj iQvOkII6Gr8ae+nyW1iyu9GMYGrd2B3eFRyrmhA+VLYU/Ryvu5JCTgybFNAPxH8Bg/Fo +RkTlkaR6oIrPcVPHnRir5NEfxD1g7DQ1Tzkahd8MZucTJr6w3x2IBl67hWkAtcgy2bB KA== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2r8rjuvw9m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:35:47 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJZfaS012943 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:35:41 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJZfpb000988; Wed, 20 Mar 2019 19:35:41 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 12:35:41 -0700 Date: Wed, 20 Mar 2019 12:35:42 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 39/36] misc: fix strncpy length complaints Message-ID: <20190320193542.GC1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200142 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Fix a number of complaints about feeding sizeof(dest) directly to strncpy. We do this by declaring the char arrays to be one larger than necessary and subtracting one, to ensure that we never overfill the buffer. Signed-off-by: Darrick J. Wong --- mkfs/xfs_mkfs.c | 13 +++++++++++-- quota/edit.c | 9 ++++++--- 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c index 9e1c6ec5..e87c692c 100644 --- a/mkfs/xfs_mkfs.c +++ b/mkfs/xfs_mkfs.c @@ -3251,8 +3251,17 @@ finish_superblock_setup( struct xfs_mount *mp, struct xfs_sb *sbp) { - if (cfg->label) - strncpy(sbp->sb_fname, cfg->label, sizeof(sbp->sb_fname)); + if (cfg->label) { + size_t label_len; + + /* + * Labels are null terminated unless the string fits exactly + * in the label field, so assume sb_fname is zeroed and then + * do a memcpy because the destination isn't a normal C string. + */ + label_len = min(sizeof(sbp->sb_fname), strlen(cfg->label)); + memcpy(sbp->sb_fname, cfg->label, label_len); + } sbp->sb_dblocks = cfg->dblocks; sbp->sb_rblocks = cfg->rtblocks; diff --git a/quota/edit.c b/quota/edit.c index b10a5b34..f9938b8a 100644 --- a/quota/edit.c +++ b/quota/edit.c @@ -368,8 +368,7 @@ restore_file( uint type) { char buffer[512]; - char devbuffer[512]; - char *dev = NULL; + char dev[512]; uint mask; int cnt; uint32_t id; @@ -377,7 +376,11 @@ restore_file( while (fgets(buffer, sizeof(buffer), fp) != NULL) { if (strncmp("fs = ", buffer, 5) == 0) { - dev = strncpy(devbuffer, buffer+5, sizeof(devbuffer)); + /* + * Copy the device name to dev, strip off the trailing + * newline, and move on to the next line. + */ + strncpy(dev, buffer + 5, sizeof(dev) - 1); dev[strlen(dev) - 1] = '\0'; continue; } From patchwork Wed Mar 20 19:36:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862529 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BB8117E9 for ; Wed, 20 Mar 2019 19:36:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E31128EB1 for ; Wed, 20 Mar 2019 19:36:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1301829E79; Wed, 20 Mar 2019 19:36:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B483B29E7A for ; Wed, 20 Mar 2019 19:36:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726123AbfCTTgK (ORCPT ); Wed, 20 Mar 2019 15:36:10 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:49770 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726115AbfCTTgK (ORCPT ); Wed, 20 Mar 2019 15:36:10 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KJXwvi116892; Wed, 20 Mar 2019 19:36:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=iokh9A77+xtshpBK9zkXh70Kd0j9EQx7f5b7rZWCp6Y=; b=Ea0486MLwvL3OOIkY0ODeL9ux9RPrxnc8A7cJsgxaMWIPYZmvRs7+/4k41RjajyUk/iE XJ9utaJmaVnLulzHBdWEp2DMOZNozRAft+EfM5JX1wfCoZVid7Jk0/gGLhDgcylHzLBP HoPTInbvb2LaB21P7RXpgEE5htLC04po4lsPsZhxiDwjpkSBKcOXhpHeOrGe+XOeqAFI ieI1om6mEwoRM5LXM3vltBOMDfcM1UNKvg9bR8SLLquSwusENvs/XMdhyhl01qpoJ8ym 9I8UBiJDM6oel/3js4hG0YdmoeH8Mcr5TsJWtuZ2kQOQyFr7U+tyFZEpsY0ETqbFFJMH xw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2r8rjuvwba-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:36:08 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJa7m9028966 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:36:07 GMT Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJa78Y001165; Wed, 20 Mar 2019 19:36:07 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 12:36:07 -0700 Date: Wed, 20 Mar 2019 12:36:08 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 40/36] xfs_io: fix label parsing and validation Message-ID: <20190320193608.GD1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200142 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong When we're trying to set a new label, check the length to make sure we won't overflow the label size, and size label[] so that we can use strncpy without static checker complaints. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen Reviewed-by: Eric Sandeen --- io/label.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/io/label.c b/io/label.c index 602ece89..72e07964 100644 --- a/io/label.c +++ b/io/label.c @@ -40,7 +40,7 @@ label_f( { int c; int error; - char label[FSLABEL_MAX]; + char label[FSLABEL_MAX + 1]; if (argc == 1) { memset(label, 0, sizeof(label)); @@ -54,7 +54,13 @@ label_f( label[0] = '\0'; break; case 's': - strncpy(label, optarg, sizeof(label)); + if (strlen(optarg) > FSLABEL_MAX) { + errno = EINVAL; + error = 1; + goto out; + } + strncpy(label, optarg, sizeof(label) - 1); + label[sizeof(label) - 1] = 0; break; default: return command_usage(&label_cmd); From patchwork Wed Mar 20 19:36:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862531 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 130691390 for ; Wed, 20 Mar 2019 19:36:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E8A7E29C39 for ; Wed, 20 Mar 2019 19:36:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DCE0929BFC; Wed, 20 Mar 2019 19:36:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 81F0B29C39 for ; Wed, 20 Mar 2019 19:36:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726538AbfCTTgh (ORCPT ); Wed, 20 Mar 2019 15:36:37 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50164 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726239AbfCTTgh (ORCPT ); Wed, 20 Mar 2019 15:36:37 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KJXvKv116878; Wed, 20 Mar 2019 19:36:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=oy3CEvx2YdTbJL87yWlZoFxw1ikSOh4dFkANT/7Horc=; b=a5BBYu1vd4RysqvzRccYRoBa5YIhqVUshpbfwszZhE5ZpAnvpcDZq33qT+CuYtvzXFJy hPb6Bqte24bWxSfKDhhTUOxqx0FHe+muFrlqYdI/6p4dtuYqKzHZDldCJE9OAf9rRudE WtSq6OHqOW44UCJZK6iK8YwZ9fnKEf6D4jl4/HjNXVX28VHtEY00Mgr9MEp3efavxU84 66Wa2qKEYQbraAQQkAxvupDpfluOqODtrsfqaWZeIjCOpFDhhdXVPaL3vMMXNms99Fkp C5dNICtgTszohsH78a18mhUiL0CmMkho3HUqpSoczfdQsH1SDyDLkm8lfKepZoQdgan4 rQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2r8rjuvwdg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:36:35 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJaUYS004163 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:36:30 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJaT8Z001434; Wed, 20 Mar 2019 19:36:29 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 12:36:29 -0700 Date: Wed, 20 Mar 2019 12:36:30 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 41/36] xfs_repair: better cli option parameter checking Message-ID: <20190320193630.GE1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=797 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200142 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong For the xfs_repair options that require a parameter, check that the user actually provided one instead of segfaulting. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- repair/xfs_repair.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c index 2d333d50..9657503f 100644 --- a/repair/xfs_repair.c +++ b/repair/xfs_repair.c @@ -233,10 +233,16 @@ process_args(int argc, char **argv) if (max_mem_specified) do_abort( _("-o bhash option cannot be used with -m option\n")); + if (!val) + do_abort( + _("-o bhash requires a parameter\n")); libxfs_bhash_size = (int)strtol(val, NULL, 0); bhash_option_used = 1; break; case AG_STRIDE: + if (!val) + do_abort( + _("-o ag_stride requires a parameter\n")); ag_stride = (int)strtol(val, NULL, 0); break; case FORCE_GEO: @@ -247,6 +253,9 @@ process_args(int argc, char **argv) force_geo = 1; break; case PHASE2_THREADS: + if (!val) + do_abort( + _("-o phase2_threads requires a parameter\n")); phase2_threads = (int)strtol(val, NULL, 0); break; default: @@ -262,6 +271,9 @@ process_args(int argc, char **argv) switch (getsubopt(&p, c_opts, &val)) { case CONVERT_LAZY_COUNT: + if (!val) + do_abort( + _("-c lazycount requires a parameter\n")); lazy_count = (int)strtol(val, NULL, 0); convert_lazy_count = 1; break; From patchwork Wed Mar 20 19:36:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862533 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D80531390 for ; Wed, 20 Mar 2019 19:37:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B74A228E55 for ; Wed, 20 Mar 2019 19:37:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ABC3329C21; Wed, 20 Mar 2019 19:37:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59D1A2980D for ; Wed, 20 Mar 2019 19:37:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726239AbfCTThB (ORCPT ); Wed, 20 Mar 2019 15:37:01 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:59568 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726006AbfCTThA (ORCPT ); Wed, 20 Mar 2019 15:37:00 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KJXjAt121017; Wed, 20 Mar 2019 19:36:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=jJ4AvlBEMW64If6sOLoGfhMHWPE/b5DZBp7l6Oo73SY=; b=sgDaKZjEincZEjh1BSK+QYzVxMRKq/fnlmtSrkeI2INLO/j6EbU8jkVZnK7nVnfjXx4C PMEFHE450/kbjESXKt4OoM7Et5UD3mU4WNlrDPDbw3q6Hb1GZMLjOuWeoigqIUwNI7l/ QHKm/fok1JX0E2KrImOAYD7terMLuJOOhxguczneCK1WpBHzr4GCGypY6UPzPqkemAkD 4c33wde3GTwLpeBYRYZC4Qiux2iPDJJXH2WIEDfFVysdvt4wdGecJ0REjcFSiBvQ3nqM UyEJ08s3qCKCsVNldMp8wHWweKd4NvkfLVy6fZIgEXhsayyee46FpjWTUhcf4fvu0/WT hA== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2r8ssrmqw0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:36:58 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJawJd031770 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:36:58 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJaw6P001589; Wed, 20 Mar 2019 19:36:58 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 12:36:57 -0700 Date: Wed, 20 Mar 2019 12:36:58 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 42/36] xfs_db: refactor metadump handling of multi-fsb objects Message-ID: <20190320193658.GF1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200142 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Separate the multi-fsb object dispatch from actual dir block processing so that we can implement symlink handling correctly as a multi-fsb file. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- db/metadump.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/db/metadump.c b/db/metadump.c index 3cce3012..83a257d0 100644 --- a/db/metadump.c +++ b/db/metadump.c @@ -1911,7 +1911,7 @@ static struct bbmap mfsb_map; static int mfsb_length; static int -process_multi_fsb_objects( +process_multi_fsb_dir( xfs_fileoff_t o, xfs_fsblock_t s, xfs_filblks_t c, @@ -1921,14 +1921,6 @@ process_multi_fsb_objects( char *dp; int ret = 0; - switch (btype) { - case TYP_DIR2: - break; - default: - print_warning("bad type for multi-fsb object %d", btype); - return -EINVAL; - } - while (c > 0) { unsigned int bm_len; @@ -1987,6 +1979,23 @@ process_multi_fsb_objects( return ret; } +static int +process_multi_fsb_objects( + xfs_fileoff_t o, + xfs_fsblock_t s, + xfs_filblks_t c, + typnm_t btype, + xfs_fileoff_t last) +{ + switch (btype) { + case TYP_DIR2: + return process_multi_fsb_dir(o, s, c, btype, last); + default: + print_warning("bad type for multi-fsb object %d", btype); + return -EINVAL; + } +} + /* inode copy routines */ static int process_bmbt_reclist( From patchwork Wed Mar 20 19:37:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862535 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 687421390 for ; Wed, 20 Mar 2019 19:37:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B20829E77 for ; Wed, 20 Mar 2019 19:37:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3FDFF29C39; Wed, 20 Mar 2019 19:37:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E160D29C07 for ; Wed, 20 Mar 2019 19:37:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726239AbfCTThY (ORCPT ); Wed, 20 Mar 2019 15:37:24 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:33576 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725988AbfCTThY (ORCPT ); Wed, 20 Mar 2019 15:37:24 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KJXfk9151362; Wed, 20 Mar 2019 19:37:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=mmtIbdcL1q+KMCdH+vKbzbLxlnAhgXsLXLYY0AQjwho=; b=MGJV/D1M8mJyyB5YKU8I19cc0d27IHna9S0tYp6d/hPLf/8jOipaC7sb3VpZUGlj261i 0FdDqNzvucg5MyOHTr2nfe7vBcCtEZxTXvCKzyLwGrTFyBZctl+Crfoe9EqAsXT7gRwl P6FMdcxpgWKPl9AhzlBsfOEFZ5XdHSCpt47YPwZYL0nj3in3rivxWqOrvSrcGTJhrkD1 /BiXoml3VzZ+j7CZyZ5cffOSKxXJ2XItl9g1dicVI4lcqCmDrmHhDu10bDTSowuaPISH 7sv8shVuldV283u5MnB8MzBfgHINydKOkPnMMPstTHXmvbi2zWeUuxe+eSE362lMtIJY YQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2r8pnew3ev-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:37:22 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJbLes007113 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:37:21 GMT Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJbLo5001793; Wed, 20 Mar 2019 19:37:21 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 12:37:20 -0700 Date: Wed, 20 Mar 2019 12:37:22 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 43/36] xfs_db: refactor multi-fsb object detection decision making Message-ID: <20190320193721.GG1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200142 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Pull the "is this a multi-fsb object" decision into a separate function that we can keep close to the actual multi-fsb object dispatcher. We will soon make the machinery more complex so we do this to avoid having a big hairy if statement. Signed-off-by: Darrick J. Wong --- db/metadump.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/db/metadump.c b/db/metadump.c index 83a257d0..57216291 100644 --- a/db/metadump.c +++ b/db/metadump.c @@ -1979,6 +1979,16 @@ process_multi_fsb_dir( return ret; } +static bool +is_multi_fsb_object( + struct xfs_mount *mp, + typnm_t btype) +{ + if (btype == TYP_DIR2 && mp->m_dir_geo->fsbcount > 1) + return true; + return false; +} + static int process_multi_fsb_objects( xfs_fileoff_t o, @@ -2011,6 +2021,7 @@ process_bmbt_reclist( xfs_fileoff_t last; xfs_agnumber_t agno; xfs_agblock_t agbno; + bool is_multi_fsb = is_multi_fsb_object(mp, btype); int error; if (btype == TYP_DATA) @@ -2074,7 +2085,7 @@ process_bmbt_reclist( } /* multi-extent blocks require special handling */ - if (btype != TYP_DIR2 || mp->m_dir_geo->fsbcount == 1) { + if (!is_multi_fsb) { error = process_single_fsb_objects(o, s, c, btype, last); } else { error = process_multi_fsb_objects(o, s, c, btype, last); From patchwork Wed Mar 20 19:37:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862537 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40FAD17E9 for ; Wed, 20 Mar 2019 19:37:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2274029E7A for ; Wed, 20 Mar 2019 19:37:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 16E4B29E80; Wed, 20 Mar 2019 19:37:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AAA7529C39 for ; Wed, 20 Mar 2019 19:37:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726067AbfCTTh6 (ORCPT ); Wed, 20 Mar 2019 15:37:58 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:34048 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726023AbfCTTh5 (ORCPT ); Wed, 20 Mar 2019 15:37:57 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KJXlpU151444; Wed, 20 Mar 2019 19:37:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=6w9pzHK8j7OJIj1effsHjh5uIanK+aW9UiV9NN7xKqo=; b=BCN1Khww6aDjJimFLlgV/f6JTQyweXEN8+lyOdb4nnoMOGJNfeIl0uxL0z0a3NJlS3h+ 6aAxXClFXirrbxrApusEFbUw6kMKdR7hXHv4CRuKKIPRP5u25hhlbrqmHWrkFdLbQB8u BoyMC/c0Vj/DSactOiFVpkm04oGmDXwnnq/4WMXNh8ej5KGDa7sfEGvTWIwxi0+vKBuq 0PxL0CCZmI1ap+sFkCavOGmHXjc4EjKbeMesZY/0zGLBnga8jumi8MJWZM+ryQT8aUNQ JueWqB3QNuwkVE2XqAwnQGVwKF0GtVEm3u2h/lrwqKtqqAGr/fFiKqgD0rywgkXrQhK0 Sg== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2r8pnew3hd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:37:55 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2KJbnkg008245 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 19:37:49 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2KJbnTh009107; Wed, 20 Mar 2019 19:37:49 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 12:37:49 -0700 Date: Wed, 20 Mar 2019 12:37:50 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 44/36] xfs_db: metadump should handle symlinks properly Message-ID: <20190320193750.GH1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200142 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Remote symlink target blocks are multi-fsb objects on XFS v5 filesystems because we only write one rmt header per data fork extent. For fs blocksize >= 2048 we never have more than one block and therefore nobody noticed, but for blocksize == 1024 this is definitely not true and leads to metadump spraying error messages about symlink block crc errors. Therefore, reformulate the symlink metadump into a multi-fsb dump function. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- db/metadump.c | 48 +++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 41 insertions(+), 7 deletions(-) diff --git a/db/metadump.c b/db/metadump.c index 57216291..e0dd463f 100644 --- a/db/metadump.c +++ b/db/metadump.c @@ -1638,11 +1638,39 @@ process_dir_data_block( } } -static void +static int process_symlink_block( - char *block) + xfs_fileoff_t o, + xfs_fsblock_t s, + xfs_filblks_t c, + typnm_t btype, + xfs_fileoff_t last) { - char *link = block; + struct bbmap map = { + .nmaps = 1, + .b = { + { + .bm_bn = XFS_FSB_TO_DADDR(mp, s), + .bm_len = XFS_FSB_TO_BB(mp, c), + } + }, + }; + char *link; + int ret = 0; + + push_cur(); + set_cur(&typtab[btype], 0, 0, DB_RING_IGN, &map); + if (!iocur_top->data) { + xfs_agnumber_t agno = XFS_FSB_TO_AGNO(mp, s); + xfs_agblock_t agbno = XFS_FSB_TO_AGBNO(mp, s); + + print_warning("cannot read %s block %u/%u (%llu)", + typtab[btype].name, agno, agbno, s); + if (stop_on_read_error) + ret = -1; + goto out_pop; + } + link = iocur_top->data; if (xfs_sb_version_hascrc(&(mp)->m_sb)) link += sizeof(struct xfs_dsymlink_hdr); @@ -1660,6 +1688,12 @@ process_symlink_block( if (zlen < mp->m_sb.sb_blocksize) memset(link + linklen, 0, zlen); } + + iocur_top->need_crc = 1; + ret = write_buf(iocur_top); +out_pop: + pop_cur(); + return ret; } #define MAX_REMOTE_VALS 4095 @@ -1879,10 +1913,6 @@ process_single_fsb_objects( } iocur_top->need_crc = 1; break; - case TYP_SYMLINK: - process_symlink_block(dp); - iocur_top->need_crc = 1; - break; case TYP_ATTR: process_attr_block(dp, o); iocur_top->need_crc = 1; @@ -1986,6 +2016,8 @@ is_multi_fsb_object( { if (btype == TYP_DIR2 && mp->m_dir_geo->fsbcount > 1) return true; + if (btype == TYP_SYMLINK) + return true; return false; } @@ -2000,6 +2032,8 @@ process_multi_fsb_objects( switch (btype) { case TYP_DIR2: return process_multi_fsb_dir(o, s, c, btype, last); + case TYP_SYMLINK: + return process_symlink_block(o, s, c, btype, last); default: print_warning("bad type for multi-fsb object %d", btype); return -EINVAL; From patchwork Wed Mar 20 20:03:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862555 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38F0A1708 for ; Wed, 20 Mar 2019 20:03:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 173D728B65 for ; Wed, 20 Mar 2019 20:03:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 05A4728B94; Wed, 20 Mar 2019 20:03:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A51B228B65 for ; Wed, 20 Mar 2019 20:03:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726303AbfCTUDi (ORCPT ); Wed, 20 Mar 2019 16:03:38 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:45548 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726067AbfCTUDh (ORCPT ); Wed, 20 Mar 2019 16:03:37 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KK275R140514; Wed, 20 Mar 2019 20:03:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=+TsI6l0kKg3X9z4AhmcCtFZp+s5uQeTsQkx0wmhb2Qs=; b=L9WJrGRJa0mt2vbw49nk3UajZ5lKB0gnLiST1ke8JAWttam279EQl4kGd6ZG+tRvBUp9 h7QIRp4HsPfsPdFMa+nhfO/rHaRWj69kCTsM9l9NBy1IJsN1IObp8MRjEuE9K/JemyZ2 p09RGXOJKLxeI9AUCi9OFh2dlpxa0YNVJONEV7hO5TOIlFpvtFhxO6IU0vigw9kplUul Np74FQKjy5EPMUd/jUK1zLeI4JeH2fKuPqCkt1SQGDWGqgTqAz0dsfIlh0Or6Ni2rnmE l9WpwdIv4uVx9jJKjl0cJYk6hq2FK8AugC8R9IvnoVGkFDY3s4HwvxYrcTGDucrOg0Fg YA== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2r8rjuw1ch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 20:03:35 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2KK3UCE010202 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 20:03:30 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2KK3T7c026808; Wed, 20 Mar 2019 20:03:30 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 13:03:29 -0700 Date: Wed, 20 Mar 2019 13:03:30 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 45/36] xfs_scrub: rename confusing structure Message-ID: <20190320200330.GI1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200145 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Rename xfs_verify_extent to media_verify_state so that the name more accurately reflects what we're storing in the structure. Signed-off-by: Darrick J. Wong --- scrub/phase6.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/scrub/phase6.c b/scrub/phase6.c index 5e2b9ee3..2b4f6ff1 100644 --- a/scrub/phase6.c +++ b/scrub/phase6.c @@ -35,7 +35,7 @@ /* Verify disk blocks with GETFSMAP */ -struct xfs_verify_extent { +struct media_verify_state { struct read_verify_pool *rvp_data; struct read_verify_pool *rvp_log; struct read_verify_pool *rvp_realtime; @@ -47,7 +47,7 @@ struct xfs_verify_extent { static struct read_verify_pool * xfs_dev_to_pool( struct scrub_ctx *ctx, - struct xfs_verify_extent *ve, + struct media_verify_state *ve, dev_t dev) { if (dev == ctx->fsinfo.fs_datadev) @@ -348,7 +348,7 @@ xfs_check_rmap_ioerr( { struct fsmap keys[2]; char descr[DESCR_BUFSZ]; - struct xfs_verify_extent *ve = arg; + struct media_verify_state *ve = arg; struct bitmap *tree; dev_t dev; bool moveon; @@ -395,7 +395,7 @@ xfs_check_rmap( struct fsmap *map, void *arg) { - struct xfs_verify_extent *ve = arg; + struct media_verify_state *ve = arg; struct read_verify_pool *rvp; rvp = xfs_dev_to_pool(ctx, ve, map->fmr_device); @@ -463,7 +463,7 @@ bool xfs_scan_blocks( struct scrub_ctx *ctx) { - struct xfs_verify_extent ve = { NULL }; + struct media_verify_state ve = { NULL }; bool moveon; moveon = bitmap_init(&ve.d_bad); From patchwork Wed Mar 20 20:04:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862557 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A610F17E0 for ; Wed, 20 Mar 2019 20:04:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8741929A70 for ; Wed, 20 Mar 2019 20:04:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7A76A29C48; Wed, 20 Mar 2019 20:04:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 207B329A70 for ; Wed, 20 Mar 2019 20:04:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726366AbfCTUEr (ORCPT ); Wed, 20 Mar 2019 16:04:47 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:57114 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726023AbfCTUEr (ORCPT ); Wed, 20 Mar 2019 16:04:47 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KK4KER177284; Wed, 20 Mar 2019 20:04:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=y1Y1oH6YDRc+Ry2r7po0+IhJlFi4ns2v1f3KgUVfZ8g=; b=MkQokyj8pCFNVEAZ65jIpoS6eRC4lG3scSh2lVaKpQMfHGPLhrSvzBAMHRLAko4xqxTk 1diFTposgTsr4vZ1BUU8azzfdEIyKnkscbU4oMiJkQ6YsfaKRjsbPpad5H47u7AZzwws c+2j8vrTD0DHHossRsEIXlu94TPJ0AD7LYcf3wMfDJdSUCVPB3kFYF5QLWgyqHnUkY24 jaamzzFTEY2tEHgkDMDZLS1C0FQeKzdcBB4DYOw8KlWEYRe0NwAY43oE2M3X3Qm8Yn/E 3WSQAXxMoYiCxseJqyGNqFTH0soMx3khUSKJdlM9u5zB8W9JtSRPLD2KIwzjhoA8cHvf gw== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2r8pnew7g9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 20:04:44 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2KK4hGB029001 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 20:04:44 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2KK4hFw009029; Wed, 20 Mar 2019 20:04:43 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 13:04:43 -0700 Date: Wed, 20 Mar 2019 13:04:44 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 46/36] xfs_scrub: remove pointless xfs_verify_error_info struct Message-ID: <20190320200444.GJ1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200145 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong The xfs_verify_error_info structure is a strict subset of media_verify_info so just pass that around. Signed-off-by: Darrick J. Wong --- scrub/phase6.c | 20 +++++--------------- 1 file changed, 5 insertions(+), 15 deletions(-) diff --git a/scrub/phase6.c b/scrub/phase6.c index 2b4f6ff1..f173f248 100644 --- a/scrub/phase6.c +++ b/scrub/phase6.c @@ -111,11 +111,6 @@ xfs_decode_special_owner( /* Routines to translate bad physical extents into file paths and offsets. */ -struct xfs_verify_error_info { - struct bitmap *d_bad; /* bytes */ - struct bitmap *r_bad; /* bytes */ -}; - /* Report if this extent overlaps a bad region. */ static bool xfs_report_verify_inode_bmap( @@ -127,7 +122,7 @@ xfs_report_verify_inode_bmap( struct xfs_bmap *bmap, void *arg) { - struct xfs_verify_error_info *vei = arg; + struct media_verify_state *vei = arg; struct bitmap *bmp; /* Only report errors for real extents. */ @@ -277,23 +272,18 @@ xfs_report_verify_dirent( static bool xfs_report_verify_errors( struct scrub_ctx *ctx, - struct bitmap *d_bad, - struct bitmap *r_bad) + struct media_verify_state *ve) { - struct xfs_verify_error_info vei; bool moveon; - vei.d_bad = d_bad; - vei.r_bad = r_bad; - /* Scan the directory tree to get file paths. */ moveon = scan_fs_tree(ctx, xfs_report_verify_dir, - xfs_report_verify_dirent, &vei); + xfs_report_verify_dirent, ve); if (!moveon) return false; /* Scan for unlinked files. */ - return xfs_scan_all_inodes(ctx, xfs_report_verify_inode, &vei); + return xfs_scan_all_inodes(ctx, xfs_report_verify_inode, ve); } /* Report an IO error resulting from read-verify based off getfsmap. */ @@ -518,7 +508,7 @@ _("Could not create data device media verifier.")); /* Scan the whole dir tree to see what matches the bad extents. */ if (!bitmap_empty(ve.d_bad) || !bitmap_empty(ve.r_bad)) - moveon = xfs_report_verify_errors(ctx, ve.d_bad, ve.r_bad); + moveon = xfs_report_verify_errors(ctx, &ve); bitmap_free(&ve.r_bad); bitmap_free(&ve.d_bad); From patchwork Wed Mar 20 20:05:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10862559 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AEF513B5 for ; Wed, 20 Mar 2019 20:05:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C4E829B8F for ; Wed, 20 Mar 2019 20:05:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1018E29C49; Wed, 20 Mar 2019 20:05:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9736629B8F for ; Wed, 20 Mar 2019 20:05:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726437AbfCTUFQ (ORCPT ); Wed, 20 Mar 2019 16:05:16 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:57036 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726023AbfCTUFQ (ORCPT ); Wed, 20 Mar 2019 16:05:16 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2KK2IFO145438; Wed, 20 Mar 2019 20:05:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=7AvlhRtoEAKwdNHZoHq08aYi7e3ogLv2ncXf1IqLGGc=; b=L0DJxa/nGtQCvDyKRsKf6mbZBuv4a3UF1Q0eOghX1/LeeKkCu5OZPwwju8h4kxh6YzvT 0mh3cma/rO/y7SP4ew6tCjBjUyuq8uHvGqKWXSYai7YGNqAQx1gQVyII3YfhoQGA/wbE Fr0YCvEk/OQbVyfJpxwyDfiGYrfgHTWuuxn1cnH0P+kMdVy4AW74JRn4pFdQ8eoZ5dJA nP4hqjWv5YmdeYN+nr9K/LypMbvq1NcpU4Xnv87WzBK6Uv5OZTRPqucrJKQdqJxWzJBd N61tdJV+3SlvNmsoED9eycgRZ0+eoMAO1L5uBYoidXhJZZtuXxL9R1rqBgN7FsFBrZoQ Pg== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2r8ssrmv2h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 20:05:13 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2KK5CnM030878 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2019 20:05:13 GMT Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2KK5CGM027978; Wed, 20 Mar 2019 20:05:12 GMT Received: from localhost (/10.159.247.212) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Mar 2019 13:05:12 -0700 Date: Wed, 20 Mar 2019 13:05:13 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 47/36] xfs_scrub: remove xfs_ prefixes from structure names Message-ID: <20190320200513.GK1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9201 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903200145 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Drop the xfs_ prefix from scrub's struct definitions since they're private to the program. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/phase5.c | 8 +++---- scrub/phase7.c | 66 ++++++++++++++++++++++++++++---------------------------- 2 files changed, 37 insertions(+), 37 deletions(-) diff --git a/scrub/phase5.c b/scrub/phase5.c index 49886e6f..1743119d 100644 --- a/scrub/phase5.c +++ b/scrub/phase5.c @@ -119,12 +119,12 @@ xfs_scrub_scan_dirents( #ifdef HAVE_LIBATTR /* Routines to scan all of an inode's xattrs for name problems. */ -struct xfs_attr_ns { +struct attrns_decode { int flags; const char *name; }; -static const struct xfs_attr_ns attr_ns[] = { +static const struct attrns_decode attr_ns[] = { {0, "user"}, {ATTR_ROOT, "system"}, {ATTR_SECURE, "secure"}, @@ -141,7 +141,7 @@ xfs_scrub_scan_fhandle_namespace_xattrs( const char *descr, struct xfs_handle *handle, struct xfs_bstat *bstat, - const struct xfs_attr_ns *attr_ns) + const struct attrns_decode *attr_ns) { struct attrlist_cursor cur; char attrbuf[XFS_XATTR_LIST_MAX]; @@ -202,7 +202,7 @@ xfs_scrub_scan_fhandle_xattrs( struct xfs_handle *handle, struct xfs_bstat *bstat) { - const struct xfs_attr_ns *ns; + const struct attrns_decode *ns; bool moveon = true; for (ns = attr_ns; ns->name; ns++) { diff --git a/scrub/phase7.c b/scrub/phase7.c index 504a6927..0c3202e4 100644 --- a/scrub/phase7.c +++ b/scrub/phase7.c @@ -16,7 +16,7 @@ /* Phase 7: Check summary counters. */ -struct xfs_summary_counts { +struct summary_counts { unsigned long long dbytes; /* data dev bytes */ unsigned long long rbytes; /* rt dev bytes */ unsigned long long next_phys; /* next phys bytes we see? */ @@ -26,13 +26,13 @@ struct xfs_summary_counts { /* Record block usage. */ static bool xfs_record_block_summary( - struct scrub_ctx *ctx, - const char *descr, - struct fsmap *fsmap, - void *arg) + struct scrub_ctx *ctx, + const char *descr, + struct fsmap *fsmap, + void *arg) { - struct xfs_summary_counts *counts; - unsigned long long len; + struct summary_counts *counts; + unsigned long long len; counts = ptvar_get((struct ptvar *)arg); if (fsmap->fmr_device == ctx->fsinfo.fs_logdev) @@ -67,12 +67,12 @@ xfs_record_block_summary( /* Add all the summaries in the per-thread counter */ static bool xfs_add_summaries( - struct ptvar *ptv, - void *data, - void *arg) + struct ptvar *ptv, + void *data, + void *arg) { - struct xfs_summary_counts *total = arg; - struct xfs_summary_counts *item = data; + struct summary_counts *total = arg; + struct summary_counts *item = data; total->dbytes += item->dbytes; total->rbytes += item->rbytes; @@ -88,27 +88,27 @@ xfs_add_summaries( */ bool xfs_scan_summary( - struct scrub_ctx *ctx) + struct scrub_ctx *ctx) { - struct xfs_summary_counts totalcount = {0}; - struct ptvar *ptvar; - unsigned long long used_data; - unsigned long long used_rt; - unsigned long long used_files; - unsigned long long stat_data; - unsigned long long stat_rt; - uint64_t counted_inodes = 0; - unsigned long long absdiff; - unsigned long long d_blocks; - unsigned long long d_bfree; - unsigned long long r_blocks; - unsigned long long r_bfree; - unsigned long long f_files; - unsigned long long f_free; - bool moveon; - bool complain; - int ip; - int error; + struct summary_counts totalcount = {0}; + struct ptvar *ptvar; + unsigned long long used_data; + unsigned long long used_rt; + unsigned long long used_files; + unsigned long long stat_data; + unsigned long long stat_rt; + uint64_t counted_inodes = 0; + unsigned long long absdiff; + unsigned long long d_blocks; + unsigned long long d_bfree; + unsigned long long r_blocks; + unsigned long long r_bfree; + unsigned long long f_files; + unsigned long long f_free; + bool moveon; + bool complain; + int ip; + int error; /* Flush everything out to disk before we start counting. */ error = syncfs(ctx->mnt_fd); @@ -117,7 +117,7 @@ xfs_scan_summary( return false; } - ptvar = ptvar_init(scrub_nproc(ctx), sizeof(struct xfs_summary_counts)); + ptvar = ptvar_init(scrub_nproc(ctx), sizeof(struct summary_counts)); if (!ptvar) { str_errno(ctx, ctx->mntpoint); return false; From patchwork Tue Mar 26 17:18:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10871699 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31D9F1669 for ; Tue, 26 Mar 2019 17:18:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1ADBE28B79 for ; Tue, 26 Mar 2019 17:18:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0E9B228B8A; Tue, 26 Mar 2019 17:18:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 01AE028B66 for ; Tue, 26 Mar 2019 17:18:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726175AbfCZRSY (ORCPT ); Tue, 26 Mar 2019 13:18:24 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:45636 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728314AbfCZRSY (ORCPT ); Tue, 26 Mar 2019 13:18:24 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2QHEXAm034351; Tue, 26 Mar 2019 17:18:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=U40QSwDjonhLVUHdt7ExZXAl98csQrH3WqqZbMQdQqw=; b=cNAKrnWW8NWAdjmh4+fh8d49hG1XjmDnYed8EE1dmddxNOWz0sJ/gWm1X6TQkxyYLGkA O3tOxtdGXbFPgumhqMtjnXkM6ofU96z9DSmBM6qGSQcLTcLwflehajS8DE9x1Nr13wdw B0PlaAjnzGOAyMnFcRFGtjZNAVMcDdHQDcGzc+V4/xO3LO5BiHQhnPBfWhUJNL9aLJb6 R7COP3FBJzoQhhnGtCNYzojrz2wW87HiCHaVbKzDeMVwtTLqogLYCld7FnWwLUuHfsVO 2uiTMQ+UT7GqRJKaeW4+tLB0l808O+ehHdSgpRkvOsImBS3a+f2h1r9oI+BkFa1NsxC9 7w== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2re6g13tsn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 Mar 2019 17:18:21 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2QHILP2008126 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 Mar 2019 17:18:21 GMT Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2QHIK73018048; Tue, 26 Mar 2019 17:18:21 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 26 Mar 2019 10:18:20 -0700 Date: Tue, 26 Mar 2019 10:18:19 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 48/36] mkfs: don't use DIFLAG values for fsx_xflags Message-ID: <20190326171819.GW1183@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9207 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=723 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903260120 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong mkfs shouldn't be using DIFLAG values where FS_XFLAG values are required. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- mkfs/xfs_mkfs.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c index e87c692c..19ad1c90 100644 --- a/mkfs/xfs_mkfs.c +++ b/mkfs/xfs_mkfs.c @@ -1451,15 +1451,15 @@ data_opts_parser( break; case D_RTINHERIT: if (getnum(value, opts, subopt)) - cli->fsx.fsx_xflags |= XFS_DIFLAG_RTINHERIT; + cli->fsx.fsx_xflags |= FS_XFLAG_RTINHERIT; break; case D_PROJINHERIT: cli->fsx.fsx_projid = getnum(value, opts, subopt); - cli->fsx.fsx_xflags |= XFS_DIFLAG_PROJINHERIT; + cli->fsx.fsx_xflags |= FS_XFLAG_PROJINHERIT; break; case D_EXTSZINHERIT: cli->fsx.fsx_extsize = getnum(value, opts, subopt); - cli->fsx.fsx_xflags |= XFS_DIFLAG_EXTSZINHERIT; + cli->fsx.fsx_xflags |= FS_XFLAG_EXTSZINHERIT; break; case D_COWEXTSIZE: cli->fsx.fsx_cowextsize = getnum(value, opts, subopt); @@ -2212,9 +2212,9 @@ validate_extsize_hint( bool rt; uint16_t flags = 0; - rt = (cli->fsx.fsx_xflags & XFS_DIFLAG_RTINHERIT); + rt = (cli->fsx.fsx_xflags & FS_XFLAG_RTINHERIT); - if (cli->fsx.fsx_xflags & XFS_DIFLAG_EXTSZINHERIT) + if (cli->fsx.fsx_xflags & FS_XFLAG_EXTSZINHERIT) flags |= XFS_DIFLAG_EXTSIZE; fa = libxfs_inode_validate_extsize(mp, cli->fsx.fsx_extsize, S_IFREG, From patchwork Mon Apr 1 16:43:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10880291 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0B8914DE for ; Mon, 1 Apr 2019 16:43:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9EC520416 for ; Mon, 1 Apr 2019 16:43:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9E24028640; Mon, 1 Apr 2019 16:43:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9281728632 for ; Mon, 1 Apr 2019 16:43:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727074AbfDAQnY (ORCPT ); Mon, 1 Apr 2019 12:43:24 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:57496 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727021AbfDAQnY (ORCPT ); Mon, 1 Apr 2019 12:43:24 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x31Gg3ui108391; Mon, 1 Apr 2019 16:43:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=kyeAGpv0EUs5d1uwfNFnp3BFRG+58i6hJV+JiruA0XY=; b=4D2ZxjTFQQJe6Nlq4ALjAvNSgrftImKZajCysY70jxApI9J0XRxyVORBFOwxdBaTAy9v 2ajVKQ5MLM2IkkPNvcnT/jDzjt7ZplHQcUm/8zHlCtdNsXB34WKxiPjTz1ySisCMIU2x waE8+V8OTZ4Sm6vFLaZsSQZ2AsIlkjv8Ti8XauxKLQQhPgLuMwt4MOw40KzBXj1+Rstp Xnt7KCqhZdoYBpWxMxbgtDC3i2wnAjE4q8QssC+rMfN45L1IZ0sFMtPaeQNmXXD6wpMb EfVM3wsnEsYrYwVS6zHlECpzWFHZKZSZOuk+J5AOWnTXXbF0YGQJvGncGmepJack/Cg9 LQ== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2rhyvt08dq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 01 Apr 2019 16:43:21 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x31GhJKe008602 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 1 Apr 2019 16:43:20 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x31GhI7S030680; Mon, 1 Apr 2019 16:43:19 GMT Received: from localhost (/67.161.8.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 01 Apr 2019 09:43:18 -0700 Date: Mon, 1 Apr 2019 09:43:17 -0700 From: "Darrick J. Wong" To: sandeen@sandeen.net Cc: linux-xfs@vger.kernel.org Subject: [PATCH 49/36] xfs_scrub: remove redundant function declarations Message-ID: <20190401164317.GA5147@magnolia> References: <155259742281.31886.17157720770696604377.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <155259742281.31886.17157720770696604377.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9214 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904010108 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Remove a redundant function declaration. Signed-off-by: Darrick J. Wong Reviewed-by: Eric Sandeen --- scrub/scrub.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/scrub/scrub.h b/scrub/scrub.h index 34c3dd30..e6e3f16f 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -25,8 +25,6 @@ bool xfs_scrub_ag_metadata(struct scrub_ctx *ctx, xfs_agnumber_t agno, struct xfs_action_list *alist); bool xfs_scrub_fs_metadata(struct scrub_ctx *ctx, struct xfs_action_list *alist); -enum check_outcome xfs_repair_metadata(struct scrub_ctx *ctx, int fd, - struct action_item *aitem, unsigned int flags); bool xfs_can_scrub_fs_metadata(struct scrub_ctx *ctx); bool xfs_can_scrub_inode(struct scrub_ctx *ctx);