From patchwork Fri Oct 23 06:32:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Allison Henderson X-Patchwork-Id: 11852285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38D0061C for ; Fri, 23 Oct 2020 06:33:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 083E422460 for ; Fri, 23 Oct 2020 06:33:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="TkUkNNKf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S372802AbgJWGdY (ORCPT ); Fri, 23 Oct 2020 02:33:24 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:37774 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S372798AbgJWGdX (ORCPT ); Fri, 23 Oct 2020 02:33:23 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 09N6PkDt007918 for ; Fri, 23 Oct 2020 06:33:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=UDtl0ParNPWzPnR894DAyF+vkDldffMz9fEDk1KZo1Y=; b=TkUkNNKfK+euAeAb1GYP2oEzgBPpda74idQS8dkKIutCax5jogQWrp4tntlum4BRNI/9 imItb96hDjY1pEiKWrtEoIZXD0ZiekE6BVs+Gg2fdvI7si1UygaSSOwTCJY8iO7pfyIv RgOAuMEKQQUuMbNJljL9TvrB5jwaKU4l1YBlXiSAjUwlrTr3qwv0bJ5Pb7RVyjPICpbH DBmXuQpQobw30kwZ71PUuPfCFtSmPmvnrlqHLQaygqwr72vCdUieCyiwTfyc+9um+QBs dypZhp86oSK9xKfszyrr5gxhe9lzlBikDBGV+R+kPFcuoxOb/SAyX3Ol3lQ7TVwvhhYu 8w== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 347p4b9du8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Fri, 23 Oct 2020 06:33:20 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 09N6Q1hK123276 for ; Fri, 23 Oct 2020 06:33:19 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3020.oracle.com with ESMTP id 34ak1aqy8w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 23 Oct 2020 06:33:19 +0000 Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 09N6XI0s009371 for ; Fri, 23 Oct 2020 06:33:18 GMT Received: from localhost.localdomain (/67.1.244.254) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 22 Oct 2020 23:33:18 -0700 From: Allison Henderson To: linux-xfs@vger.kernel.org Subject: [PATCH v13 07/14] xfsprogs: Set up infastructure for deferred attribute operations Date: Thu, 22 Oct 2020 23:32:59 -0700 Message-Id: <20201023063306.7441-8-allison.henderson@oracle.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201023063306.7441-1-allison.henderson@oracle.com> References: <20201023063306.7441-1-allison.henderson@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9782 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 mlxscore=0 phishscore=0 spamscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2010230045 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9782 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 priorityscore=1501 clxscore=1015 malwarescore=0 mlxscore=0 bulkscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 mlxlogscore=999 impostorscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2010230045 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Allison Collins Currently attributes are modified directly across one or more transactions. But they are not logged or replayed in the event of an error. The goal of delayed attributes is to enable logging and replaying of attribute operations using the existing delayed operations infrastructure. This will later enable the attributes to become part of larger multi part operations that also must first be recorded to the log. This is mostly of interest in the scheme of parent pointers which would need to maintain an attribute containing parent inode information any time an inode is moved, created, or removed. Parent pointers would then be of interest to any feature that would need to quickly derive an inode path from the mount point. Online scrub, nfs lookups and fs grow or shrink operations are all features that could take advantage of this. This patch adds two new log item types for setting or removing attributes as deferred operations. The xfs_attri_log_item logs an intent to set or remove an attribute. The corresponding xfs_attrd_log_item holds a reference to the xfs_attri_log_item and is freed once the transaction is done. Both log items use a generic xfs_attr_log_format structure that contains the attribute name, value, flags, inode, and an op_flag that indicates if the operations is a set or remove. Signed-off-by: Allison Henderson --- libxfs/defer_item.c | 132 ++++++++++++++++++++++++++++++++++++++++++++++++ libxfs/libxfs_priv.h | 1 + libxfs/xfs_attr.c | 7 ++- libxfs/xfs_attr.h | 19 +++++++ libxfs/xfs_defer.c | 1 + libxfs/xfs_defer.h | 2 + libxfs/xfs_format.h | 5 ++ libxfs/xfs_log_format.h | 43 +++++++++++++++- libxfs/xfs_types.h | 1 + 9 files changed, 205 insertions(+), 6 deletions(-) diff --git a/libxfs/defer_item.c b/libxfs/defer_item.c index b18182e..ab21173 100644 --- a/libxfs/defer_item.c +++ b/libxfs/defer_item.c @@ -16,10 +16,14 @@ #include "xfs_trans.h" #include "xfs_bmap.h" #include "xfs_alloc.h" +#include "xfs_attr_item.h" #include "xfs_rmap.h" #include "xfs_refcount.h" #include "xfs_bmap.h" #include "xfs_inode.h" +#include "xfs_da_format.h" +#include "xfs_da_btree.h" +#include "xfs_attr.h" /* Dummy defer item ops, since we don't do logging. */ @@ -112,6 +116,134 @@ const struct xfs_defer_op_type xfs_extent_free_defer_type = { .cancel_item = xfs_extent_free_cancel_item, }; +static inline struct xfs_attrd_log_item *ATTRD_ITEM(struct xfs_log_item *lip) +{ + return container_of(lip, struct xfs_attrd_log_item, attrd_item); +} + +/* + * Log an ATTRI it to the ATTRD when the attr op is done. An attr operation + * may be a set or a remove. Note that the transaction is marked dirty + * regardless of whether the operation succeeds or fails to support the + * ATTRI/ATTRD lifecycle rules. + */ +int +xfs_trans_attr( + struct xfs_delattr_context *dac, + struct xfs_attrd_log_item *attrdp, + struct xfs_buf **leaf_bp, + uint32_t op_flags) +{ + struct xfs_da_args *args = dac->da_args; + int error; + + error = xfs_qm_dqattach_locked(args->dp, 0); + if (error) + return error; + + switch (op_flags) { + case XFS_ATTR_OP_FLAGS_SET: + args->op_flags |= XFS_DA_OP_ADDNAME; + error = xfs_attr_set_iter(dac, leaf_bp); + break; + case XFS_ATTR_OP_FLAGS_REMOVE: + ASSERT(XFS_IFORK_Q((args->dp))); + error = xfs_attr_remove_iter(dac); + break; + default: + error = -EFSCORRUPTED; + break; + } + + /* + * Mark the transaction dirty, even on error. This ensures the + * transaction is aborted, which: + * + * 1.) releases the ATTRI and frees the ATTRD + * 2.) shuts down the filesystem + */ + args->trans->t_flags |= XFS_TRANS_DIRTY; + + return error; +} + +/* Get an ATTRI. */ +static struct xfs_log_item * +xfs_attr_create_intent( + struct xfs_trans *tp, + struct list_head *items, + unsigned int count, + bool sort) +{ + return NULL; +} + +/* Abort all pending ATTRs. */ +STATIC void +xfs_attr_abort_intent( + struct xfs_log_item *intent) +{ +} + +/* Get an ATTRD so we can process all the attrs. */ +static struct xfs_log_item * +xfs_attr_create_done( + struct xfs_trans *tp, + struct xfs_log_item *intent, + unsigned int count) +{ + return NULL; +} + +/* Process an attr. */ +STATIC int +xfs_attr_finish_item( + struct xfs_trans *tp, + struct xfs_log_item *done, + struct list_head *item, + struct xfs_btree_cur **state) +{ + struct xfs_attr_item *attr; + int error; + struct xfs_delattr_context *dac; + + attr = container_of(item, struct xfs_attr_item, xattri_list); + dac = &attr->xattri_dac; + + /* + * Always reset trans after EAGAIN cycle + * since the transaction is new + */ + dac->da_args->trans = tp; + + error = xfs_trans_attr(dac, ATTRD_ITEM(done), &dac->leaf_bp, + attr->xattri_op_flags); + if (error != -EAGAIN) + kmem_free(attr); + + return error; +} + +/* Cancel an attr */ +STATIC void +xfs_attr_cancel_item( + struct list_head *item) +{ + struct xfs_attr_item *attr; + + attr = container_of(item, struct xfs_attr_item, xattri_list); + kmem_free(attr); +} + +const struct xfs_defer_op_type xfs_attr_defer_type = { + .max_items = 1, + .create_intent = xfs_attr_create_intent, + .abort_intent = xfs_attr_abort_intent, + .create_done = xfs_attr_create_done, + .finish_item = xfs_attr_finish_item, + .cancel_item = xfs_attr_cancel_item, +}; + /* * AGFL blocks are accounted differently in the reserve pools and are not * inserted into the busy extent list. diff --git a/libxfs/libxfs_priv.h b/libxfs/libxfs_priv.h index bd724c3..130cef4 100644 --- a/libxfs/libxfs_priv.h +++ b/libxfs/libxfs_priv.h @@ -499,6 +499,7 @@ void __xfs_buf_mark_corrupt(struct xfs_buf *bp, xfs_failaddr_t fa); #define xfs_trans_reserve_quota_nblks(t,i,b,n,f) (0) #define xfs_trans_unreserve_quota_nblks(t,i,b,n,f) ((void) 0) #define xfs_qm_dqattach(i) (0) +#define xfs_qm_dqattach_locked(i,b) (0) #define uuid_copy(s,d) platform_uuid_copy((s),(d)) #define uuid_equal(s,d) (platform_uuid_compare((s),(d)) == 0) diff --git a/libxfs/xfs_attr.c b/libxfs/xfs_attr.c index 0cee4b17..ea054be 100644 --- a/libxfs/xfs_attr.c +++ b/libxfs/xfs_attr.c @@ -24,6 +24,7 @@ #include "xfs_quota_defs.h" #include "xfs_trans_space.h" #include "xfs_trace.h" +#include "xfs_attr_item.h" /* * xfs_attr.c @@ -59,8 +60,6 @@ STATIC int xfs_attr_node_hasname(xfs_da_args_t *args, STATIC int xfs_attr_fillstate(xfs_da_state_t *state); STATIC int xfs_attr_refillstate(xfs_da_state_t *state); STATIC int xfs_attr_leaf_try_add(struct xfs_da_args *args, struct xfs_buf *bp); -STATIC int xfs_attr_set_iter(struct xfs_delattr_context *dac, - struct xfs_buf **leaf_bp); int xfs_inode_hasattr( @@ -142,7 +141,7 @@ xfs_attr_get( /* * Calculate how many blocks we need for the new attribute, */ -STATIC int +int xfs_attr_calc_size( struct xfs_da_args *args, int *local) @@ -327,7 +326,7 @@ xfs_attr_set_args( * to handle this, and recall the function until a successful error code is * returned. */ -STATIC int +int xfs_attr_set_iter( struct xfs_delattr_context *dac, struct xfs_buf **leaf_bp) diff --git a/libxfs/xfs_attr.h b/libxfs/xfs_attr.h index 501f9df..5b4a1ca 100644 --- a/libxfs/xfs_attr.h +++ b/libxfs/xfs_attr.h @@ -247,6 +247,7 @@ enum xfs_delattr_state { #define XFS_DAC_DEFER_FINISH 0x01 /* finish the transaction */ #define XFS_DAC_NODE_RMVNAME_INIT 0x02 /* xfs_attr_node_removename init */ #define XFS_DAC_LEAF_ADDNAME_INIT 0x04 /* xfs_attr_leaf_addname init*/ +#define XFS_DAC_DELAYED_OP_INIT 0x08 /* delayed operations init*/ /* * Context used for keeping track of delayed attribute operations @@ -254,6 +255,9 @@ enum xfs_delattr_state { struct xfs_delattr_context { struct xfs_da_args *da_args; + /* Used by delayed attributes to hold leaf across transactions */ + struct xfs_buf *leaf_bp; + /* Used in xfs_attr_rmtval_set_blk to roll through allocating blocks */ struct xfs_bmbt_irec map; xfs_dablk_t lblkno; @@ -267,6 +271,18 @@ struct xfs_delattr_context { enum xfs_delattr_state dela_state; }; +/* + * List of attrs to commit later. + */ +struct xfs_attr_item { + struct xfs_delattr_context xattri_dac; + uint32_t xattri_op_flags;/* attr op set or rm */ + + /* used to log this item to an intent */ + struct list_head xattri_list; +}; + + /*======================================================================== * Function prototypes for the kernel. *========================================================================*/ @@ -282,11 +298,14 @@ int xfs_attr_get_ilocked(struct xfs_da_args *args); int xfs_attr_get(struct xfs_da_args *args); int xfs_attr_set(struct xfs_da_args *args); int xfs_attr_set_args(struct xfs_da_args *args); +int xfs_attr_set_iter(struct xfs_delattr_context *dac, + struct xfs_buf **leaf_bp); int xfs_has_attr(struct xfs_da_args *args); int xfs_attr_remove_args(struct xfs_da_args *args); int xfs_attr_remove_iter(struct xfs_delattr_context *dac); bool xfs_attr_namecheck(const void *name, size_t length); void xfs_delattr_context_init(struct xfs_delattr_context *dac, struct xfs_da_args *args); +int xfs_attr_calc_size(struct xfs_da_args *args, int *local); #endif /* __XFS_ATTR_H__ */ diff --git a/libxfs/xfs_defer.c b/libxfs/xfs_defer.c index 34c1019..c6e45e2 100644 --- a/libxfs/xfs_defer.c +++ b/libxfs/xfs_defer.c @@ -174,6 +174,7 @@ static const struct xfs_defer_op_type *defer_op_types[] = { [XFS_DEFER_OPS_TYPE_RMAP] = &xfs_rmap_update_defer_type, [XFS_DEFER_OPS_TYPE_FREE] = &xfs_extent_free_defer_type, [XFS_DEFER_OPS_TYPE_AGFL_FREE] = &xfs_agfl_free_defer_type, + [XFS_DEFER_OPS_TYPE_ATTR] = &xfs_attr_defer_type, }; static void diff --git a/libxfs/xfs_defer.h b/libxfs/xfs_defer.h index 6b2ca58..ed65927 100644 --- a/libxfs/xfs_defer.h +++ b/libxfs/xfs_defer.h @@ -18,6 +18,7 @@ enum xfs_defer_ops_type { XFS_DEFER_OPS_TYPE_RMAP, XFS_DEFER_OPS_TYPE_FREE, XFS_DEFER_OPS_TYPE_AGFL_FREE, + XFS_DEFER_OPS_TYPE_ATTR, XFS_DEFER_OPS_TYPE_MAX, }; @@ -62,5 +63,6 @@ extern const struct xfs_defer_op_type xfs_refcount_update_defer_type; extern const struct xfs_defer_op_type xfs_rmap_update_defer_type; extern const struct xfs_defer_op_type xfs_extent_free_defer_type; extern const struct xfs_defer_op_type xfs_agfl_free_defer_type; +extern const struct xfs_defer_op_type xfs_attr_defer_type; #endif /* __XFS_DEFER_H__ */ diff --git a/libxfs/xfs_format.h b/libxfs/xfs_format.h index 5a183b6..11e9fc5 100644 --- a/libxfs/xfs_format.h +++ b/libxfs/xfs_format.h @@ -565,6 +565,11 @@ static inline bool xfs_sb_version_hasreflink(struct xfs_sb *sbp) (sbp->sb_features_ro_compat & XFS_SB_FEAT_RO_COMPAT_REFLINK); } +static inline bool xfs_sb_version_hasdelattr(struct xfs_sb *sbp) +{ + return false; +} + /* * Inode btree block counter. We record the number of inobt and finobt blocks * in the AGI header so that we can skip the finobt walk at mount time when diff --git a/libxfs/xfs_log_format.h b/libxfs/xfs_log_format.h index e3400c9..6c5bf51 100644 --- a/libxfs/xfs_log_format.h +++ b/libxfs/xfs_log_format.h @@ -117,7 +117,11 @@ struct xfs_unmount_log_format { #define XLOG_REG_TYPE_CUD_FORMAT 24 #define XLOG_REG_TYPE_BUI_FORMAT 25 #define XLOG_REG_TYPE_BUD_FORMAT 26 -#define XLOG_REG_TYPE_MAX 26 +#define XLOG_REG_TYPE_ATTRI_FORMAT 27 +#define XLOG_REG_TYPE_ATTRD_FORMAT 28 +#define XLOG_REG_TYPE_ATTR_NAME 29 +#define XLOG_REG_TYPE_ATTR_VALUE 30 +#define XLOG_REG_TYPE_MAX 30 /* * Flags to log operation header @@ -240,6 +244,8 @@ typedef struct xfs_trans_header { #define XFS_LI_CUD 0x1243 #define XFS_LI_BUI 0x1244 /* bmbt update intent */ #define XFS_LI_BUD 0x1245 +#define XFS_LI_ATTRI 0x1246 /* attr set/remove intent*/ +#define XFS_LI_ATTRD 0x1247 /* attr set/remove done */ #define XFS_LI_TYPE_DESC \ { XFS_LI_EFI, "XFS_LI_EFI" }, \ @@ -255,7 +261,9 @@ typedef struct xfs_trans_header { { XFS_LI_CUI, "XFS_LI_CUI" }, \ { XFS_LI_CUD, "XFS_LI_CUD" }, \ { XFS_LI_BUI, "XFS_LI_BUI" }, \ - { XFS_LI_BUD, "XFS_LI_BUD" } + { XFS_LI_BUD, "XFS_LI_BUD" }, \ + { XFS_LI_ATTRI, "XFS_LI_ATTRI" }, \ + { XFS_LI_ATTRD, "XFS_LI_ATTRD" } /* * Inode Log Item Format definitions. @@ -860,4 +868,35 @@ struct xfs_icreate_log { __be32 icl_gen; /* inode generation number to use */ }; +/* + * Flags for deferred attribute operations. + * Upper bits are flags, lower byte is type code + */ +#define XFS_ATTR_OP_FLAGS_SET 1 /* Set the attribute */ +#define XFS_ATTR_OP_FLAGS_REMOVE 2 /* Remove the attribute */ +#define XFS_ATTR_OP_FLAGS_TYPE_MASK 0x0FF /* Flags type mask */ + +/* + * This is the structure used to lay out an attr log item in the + * log. + */ +struct xfs_attri_log_format { + uint16_t alfi_type; /* attri log item type */ + uint16_t alfi_size; /* size of this item */ + uint32_t __pad; /* pad to 64 bit aligned */ + uint64_t alfi_id; /* attri identifier */ + xfs_ino_t alfi_ino; /* the inode for this attr operation */ + uint32_t alfi_op_flags; /* marks the op as a set or remove */ + uint32_t alfi_name_len; /* attr name length */ + uint32_t alfi_value_len; /* attr value length */ + uint32_t alfi_attr_flags;/* attr flags */ +}; + +struct xfs_attrd_log_format { + uint16_t alfd_type; /* attrd log item type */ + uint16_t alfd_size; /* size of this item */ + uint32_t __pad; /* pad to 64 bit aligned */ + uint64_t alfd_alf_id; /* id of corresponding attrd */ +}; + #endif /* __XFS_LOG_FORMAT_H__ */ diff --git a/libxfs/xfs_types.h b/libxfs/xfs_types.h index 397d947..860cdd2 100644 --- a/libxfs/xfs_types.h +++ b/libxfs/xfs_types.h @@ -11,6 +11,7 @@ typedef uint32_t prid_t; /* project ID */ typedef uint32_t xfs_agblock_t; /* blockno in alloc. group */ typedef uint32_t xfs_agino_t; /* inode # within allocation grp */ typedef uint32_t xfs_extlen_t; /* extent length in blocks */ +typedef uint32_t xfs_attrlen_t; /* attr length */ typedef uint32_t xfs_agnumber_t; /* allocation group number */ typedef int32_t xfs_extnum_t; /* # of extents in a file */ typedef int16_t xfs_aextnum_t; /* # extents in an attribute fork */