[35/39] xfs: convert log vector chain to use list heads

Message ID	20210603052240.171998-36-david@fromorbit.com (mailing list archive)
State	Accepted
Headers	show Return-Path: <linux-xfs-owner@kernel.org> From: Dave Chinner <david@fromorbit.com> To: linux-xfs@vger.kernel.org Subject: [PATCH 35/39] xfs: convert log vector chain to use list heads Date: Thu, 3 Jun 2021 15:22:36 +1000 Message-Id: <20210603052240.171998-36-david@fromorbit.com> In-Reply-To: <20210603052240.171998-1-david@fromorbit.com> References: <20210603052240.171998-1-david@fromorbit.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk
Series	xfs: CIL and log optimisations \| expand [00/39,v5] xfs: CIL and log optimisations [01/39] xfs: log stripe roundoff is a property of the log [02/39] xfs: separate CIL commit record IO [03/39] xfs: remove xfs_blkdev_issue_flush [04/39] xfs: async blkdev cache flush [05/39] xfs: CIL checkpoint flushes caches unconditionally [06/39] xfs: remove need_start_rec parameter from xlog_write() [07/39] xfs: journal IO cache flush reductions [08/39] xfs: Fix CIL throttle hang when CIL space used going backwards [09/39] xfs: xfs_log_force_lsn isn't passed a LSN [10/39] xfs: AIL needs asynchronous CIL forcing [11/39] xfs: CIL work is serialised, not pipelined [12/39] xfs: factor out the CIL transaction header building [13/39] xfs: only CIL pushes require a start record [14/39] xfs: embed the xlog_op_header in the unmount record [15/39] xfs: embed the xlog_op_header in the commit record [16/39] xfs: log tickets don't need log client id [17/39] xfs: move log iovec alignment to preparation function [18/39] xfs: reserve space and initialise xlog_op_header in item formatting [19/39] xfs: log ticket region debug is largely useless [20/39] xfs: pass lv chain length into xlog_write() [21/39] xfs: introduce xlog_write_single() [22/39] xfs:_introduce xlog_write_partial() [23/39] xfs: xlog_write() no longer needs contwr state [24/39] xfs: xlog_write() doesn't need optype anymore [25/39] xfs: CIL context doesn't need to count iovecs [26/39] xfs: use the CIL space used counter for emptiness checks [27/39] xfs: lift init CIL reservation out of xc_cil_lock [28/39] xfs: rework per-iclog header CIL reservation [29/39] xfs: introduce per-cpu CIL tracking structure [30/39] xfs: implement percpu cil space used calculation [31/39] xfs: track CIL ticket reservation in percpu structure [32/39] xfs: convert CIL busy extents to per-cpu [33/39] xfs: Add order IDs to log items in CIL [34/39] xfs: convert CIL to unordered per cpu lists [35/39] xfs: convert log vector chain to use list heads [36/39] xfs: move CIL ordering to the logvec chain [37/39] xfs: avoid cil push lock if possible [38/39] xfs: xlog_sync() manually adjusts grant head space [39/39] xfs: expanding delayed logging design with background material

Hi Dave, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on xfs-linux/for-next] [also build test WARNING on linus/master v5.13-rc4 next-20210602] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Dave-Chinner/xfs-CIL-and-log-optimisations/20210603-134113 base: https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git for-next config: arm-randconfig-r023-20210603 (attached as .config) compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project d8e0ae9a76a62bdc6117630d59bf9967ac9bb4ea) reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # install arm cross compiling tool for clang build # apt-get install binutils-arm-linux-gnueabi # https://github.com/0day-ci/linux/commit/b9844caa1f30dfae6f4e78de2b70126ba2898eda git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Dave-Chinner/xfs-CIL-and-log-optimisations/20210603-134113 git checkout b9844caa1f30dfae6f4e78de2b70126ba2898eda # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com> All warnings (new ones prefixed by >>): >> fs/xfs/xfs_log_cil.c:836:31: warning: suggest braces around initialization of subobject [-Wmissing-braces] struct xfs_log_vec lvhdr = { NULL }; ^~~~ { } include/linux/stddef.h:8:14: note: expanded from macro 'NULL' #define NULL ((void *)0) ^~~~~~~~~~~ fs/xfs/xfs_log_cil.c:1550:30: error: invalid operands to binary expression ('void' and 'int') if (xlog_cil_pcp_hpadd(cil) < 0) { ~~~~~~~~~~~~~~~~~~~~~~~ ^ ~ 1 warning and 1 error generated. vim +836 fs/xfs/xfs_log_cil.c 08a813aea7434e Dave Chinner 2021-06-03 806 71e330b593905e Dave Chinner 2010-05-21 807 /* c7cc296ddd1f6d Christoph Hellwig 2020-03-20 808 * Push the Committed Item List to the log. c7cc296ddd1f6d Christoph Hellwig 2020-03-20 809 * c7cc296ddd1f6d Christoph Hellwig 2020-03-20 810 * If the current sequence is the same as xc_push_seq we need to do a flush. If c7cc296ddd1f6d Christoph Hellwig 2020-03-20 811 * xc_push_seq is less than the current sequence, then it has already been a44f13edf0ebb4 Dave Chinner 2010-08-24 812 * flushed and we don't need to do anything - the caller will wait for it to a44f13edf0ebb4 Dave Chinner 2010-08-24 813 * complete if necessary. a44f13edf0ebb4 Dave Chinner 2010-08-24 814 * c7cc296ddd1f6d Christoph Hellwig 2020-03-20 815 * xc_push_seq is checked unlocked against the sequence number for a match. c7cc296ddd1f6d Christoph Hellwig 2020-03-20 816 * Hence we can allow log forces to run racily and not issue pushes for the c7cc296ddd1f6d Christoph Hellwig 2020-03-20 817 * same sequence twice. If we get a race between multiple pushes for the same c7cc296ddd1f6d Christoph Hellwig 2020-03-20 818 * sequence they will block on the first one and then abort, hence avoiding c7cc296ddd1f6d Christoph Hellwig 2020-03-20 819 * needless pushes. c7cc296ddd1f6d Christoph Hellwig 2020-03-20 820 */ c7cc296ddd1f6d Christoph Hellwig 2020-03-20 821 static void c7cc296ddd1f6d Christoph Hellwig 2020-03-20 822 xlog_cil_push_work( c7cc296ddd1f6d Christoph Hellwig 2020-03-20 823 struct work_struct *work) 71e330b593905e Dave Chinner 2010-05-21 824 { 61691dab98fb92 Dave Chinner 2021-06-03 825 struct xfs_cil_ctx *ctx = 61691dab98fb92 Dave Chinner 2021-06-03 826 container_of(work, struct xfs_cil_ctx, push_work); 61691dab98fb92 Dave Chinner 2021-06-03 827 struct xfs_cil *cil = ctx->cil; c7cc296ddd1f6d Christoph Hellwig 2020-03-20 828 struct xlog *log = cil->xc_log; 71e330b593905e Dave Chinner 2010-05-21 829 struct xfs_log_vec *lv; 71e330b593905e Dave Chinner 2010-05-21 830 struct xfs_cil_ctx *new_ctx; 71e330b593905e Dave Chinner 2010-05-21 831 struct xlog_in_core *commit_iclog; 964c4f5053f0cd Dave Chinner 2021-06-03 832 int num_iovecs = 0; 964c4f5053f0cd Dave Chinner 2021-06-03 833 int num_bytes = 0; 71e330b593905e Dave Chinner 2010-05-21 834 int error = 0; 81be7bf64fac48 Dave Chinner 2021-06-03 835 struct xlog_cil_trans_hdr thdr; 71e330b593905e Dave Chinner 2010-05-21 @836 struct xfs_log_vec lvhdr = { NULL }; 71e330b593905e Dave Chinner 2010-05-21 837 xfs_lsn_t commit_lsn; 4c2d542f2e7865 Dave Chinner 2012-04-23 838 xfs_lsn_t push_seq; 395ce53dd09e23 Dave Chinner 2021-06-03 839 struct bio bio; 395ce53dd09e23 Dave Chinner 2021-06-03 840 DECLARE_COMPLETION_ONSTACK(bdev_flush); ed3d678659f2f3 Dave Chinner 2021-06-03 841 bool push_commit_stable; 71e330b593905e Dave Chinner 2010-05-21 842 61691dab98fb92 Dave Chinner 2021-06-03 843 new_ctx = xlog_cil_ctx_alloc(); 71e330b593905e Dave Chinner 2010-05-21 844 new_ctx->ticket = xlog_cil_ticket_alloc(log); 71e330b593905e Dave Chinner 2010-05-21 845 71e330b593905e Dave Chinner 2010-05-21 846 down_write(&cil->xc_ctx_lock); 71e330b593905e Dave Chinner 2010-05-21 847 4bb928cdb900d0 Dave Chinner 2013-08-12 848 spin_lock(&cil->xc_push_lock); 4c2d542f2e7865 Dave Chinner 2012-04-23 849 push_seq = cil->xc_push_seq; 4c2d542f2e7865 Dave Chinner 2012-04-23 850 ASSERT(push_seq <= ctx->sequence); ed3d678659f2f3 Dave Chinner 2021-06-03 851 push_commit_stable = cil->xc_push_commit_stable; ed3d678659f2f3 Dave Chinner 2021-06-03 852 cil->xc_push_commit_stable = false; 71e330b593905e Dave Chinner 2010-05-21 853 0e7ab7efe77451 Dave Chinner 2020-03-24 854 /* 1ac67445c59a8e Dave Chinner 2021-06-03 855 * As we are about to switch to a new, empty CIL context, we no longer 1ac67445c59a8e Dave Chinner 2021-06-03 856 * need to throttle tasks on CIL space overruns. Wake any waiters that 1ac67445c59a8e Dave Chinner 2021-06-03 857 * the hard push throttle may have caught so they can start committing 1ac67445c59a8e Dave Chinner 2021-06-03 858 * to the new context. The ctx->xc_push_lock provides the serialisation 1ac67445c59a8e Dave Chinner 2021-06-03 859 * necessary for safely using the lockless waitqueue_active() check in 1ac67445c59a8e Dave Chinner 2021-06-03 860 * this context. 1ac67445c59a8e Dave Chinner 2021-06-03 861 */ 1ac67445c59a8e Dave Chinner 2021-06-03 862 if (waitqueue_active(&cil->xc_push_wait)) c7f87f3984cfa1 Dave Chinner 2020-06-16 863 wake_up_all(&cil->xc_push_wait); 0e7ab7efe77451 Dave Chinner 2020-03-24 864 4c2d542f2e7865 Dave Chinner 2012-04-23 865 /* 4c2d542f2e7865 Dave Chinner 2012-04-23 866 * Check if we've anything to push. If there is nothing, then we don't 4c2d542f2e7865 Dave Chinner 2012-04-23 867 * move on to a new sequence number and so we have to be able to push 4c2d542f2e7865 Dave Chinner 2012-04-23 868 * this sequence again later. 4c2d542f2e7865 Dave Chinner 2012-04-23 869 */ b8fab3c5759918 Dave Chinner 2021-06-03 870 if (test_bit(XLOG_CIL_EMPTY, &cil->xc_flags)) { 4c2d542f2e7865 Dave Chinner 2012-04-23 871 cil->xc_push_seq = 0; 4bb928cdb900d0 Dave Chinner 2013-08-12 872 spin_unlock(&cil->xc_push_lock); a44f13edf0ebb4 Dave Chinner 2010-08-24 873 goto out_skip; 4c2d542f2e7865 Dave Chinner 2012-04-23 874 } 4c2d542f2e7865 Dave Chinner 2012-04-23 875 a44f13edf0ebb4 Dave Chinner 2010-08-24 876 cf085a1b5d2214 Joe Perches 2019-11-07 877 /* check for a previously pushed sequence */ 61691dab98fb92 Dave Chinner 2021-06-03 878 if (push_seq < ctx->sequence) { 8af3dcd3c89aef Dave Chinner 2014-09-23 879 spin_unlock(&cil->xc_push_lock); df806158b0f6eb Dave Chinner 2010-05-17 880 goto out_skip; 8af3dcd3c89aef Dave Chinner 2014-09-23 881 } 8af3dcd3c89aef Dave Chinner 2014-09-23 882 8af3dcd3c89aef Dave Chinner 2014-09-23 883 /* 8af3dcd3c89aef Dave Chinner 2014-09-23 884 * We are now going to push this context, so add it to the committing 8af3dcd3c89aef Dave Chinner 2014-09-23 885 * list before we do anything else. This ensures that anyone waiting on 8af3dcd3c89aef Dave Chinner 2014-09-23 886 * this push can easily detect the difference between a "push in 8af3dcd3c89aef Dave Chinner 2014-09-23 887 * progress" and "CIL is empty, nothing to do". 8af3dcd3c89aef Dave Chinner 2014-09-23 888 * 8af3dcd3c89aef Dave Chinner 2014-09-23 889 * IOWs, a wait loop can now check for: 8af3dcd3c89aef Dave Chinner 2014-09-23 890 * the current sequence not being found on the committing list; 8af3dcd3c89aef Dave Chinner 2014-09-23 891 * an empty CIL; and 8af3dcd3c89aef Dave Chinner 2014-09-23 892 * an unchanged sequence number 8af3dcd3c89aef Dave Chinner 2014-09-23 893 * to detect a push that had nothing to do and therefore does not need 8af3dcd3c89aef Dave Chinner 2014-09-23 894 * waiting on. If the CIL is not empty, we get put on the committing 8af3dcd3c89aef Dave Chinner 2014-09-23 895 * list before emptying the CIL and bumping the sequence number. Hence 8af3dcd3c89aef Dave Chinner 2014-09-23 896 * an empty CIL and an unchanged sequence number means we jumped out 8af3dcd3c89aef Dave Chinner 2014-09-23 897 * above after doing nothing. 8af3dcd3c89aef Dave Chinner 2014-09-23 898 * 8af3dcd3c89aef Dave Chinner 2014-09-23 899 * Hence the waiter will either find the commit sequence on the 8af3dcd3c89aef Dave Chinner 2014-09-23 900 * committing list or the sequence number will be unchanged and the CIL 8af3dcd3c89aef Dave Chinner 2014-09-23 901 * still dirty. In that latter case, the push has not yet started, and 8af3dcd3c89aef Dave Chinner 2014-09-23 902 * so the waiter will have to continue trying to check the CIL 8af3dcd3c89aef Dave Chinner 2014-09-23 903 * committing list until it is found. In extreme cases of delay, the 8af3dcd3c89aef Dave Chinner 2014-09-23 904 * sequence may fully commit between the attempts the wait makes to wait 8af3dcd3c89aef Dave Chinner 2014-09-23 905 * on the commit sequence. 8af3dcd3c89aef Dave Chinner 2014-09-23 906 */ 8af3dcd3c89aef Dave Chinner 2014-09-23 907 list_add(&ctx->committing, &cil->xc_committing); 8af3dcd3c89aef Dave Chinner 2014-09-23 908 spin_unlock(&cil->xc_push_lock); df806158b0f6eb Dave Chinner 2010-05-17 909 71e330b593905e Dave Chinner 2010-05-21 910 /* 395ce53dd09e23 Dave Chinner 2021-06-03 911 * The CIL is stable at this point - nothing new will be added to it 395ce53dd09e23 Dave Chinner 2021-06-03 912 * because we hold the flush lock exclusively. Hence we can now issue 395ce53dd09e23 Dave Chinner 2021-06-03 913 * a cache flush to ensure all the completed metadata in the journal we 395ce53dd09e23 Dave Chinner 2021-06-03 914 * are about to overwrite is on stable storage. 395ce53dd09e23 Dave Chinner 2021-06-03 915 */ 395ce53dd09e23 Dave Chinner 2021-06-03 916 xfs_flush_bdev_async(&bio, log->l_mp->m_ddev_targp->bt_bdev, 395ce53dd09e23 Dave Chinner 2021-06-03 917 &bdev_flush); 395ce53dd09e23 Dave Chinner 2021-06-03 918 4da22b91f10349 Dave Chinner 2021-06-03 919 xlog_cil_pcp_aggregate(cil, ctx); 4da22b91f10349 Dave Chinner 2021-06-03 920 3bf44a60c84eae Dave Chinner 2021-06-03 921 list_sort(NULL, &ctx->log_items, xlog_cil_order_cmp); 3bf44a60c84eae Dave Chinner 2021-06-03 922 while (!list_empty(&ctx->log_items)) { 71e330b593905e Dave Chinner 2010-05-21 923 struct xfs_log_item *item; 71e330b593905e Dave Chinner 2010-05-21 924 3bf44a60c84eae Dave Chinner 2021-06-03 925 item = list_first_entry(&ctx->log_items, 71e330b593905e Dave Chinner 2010-05-21 926 struct xfs_log_item, li_cil); b9844caa1f30df Dave Chinner 2021-06-03 927 lv = item->li_lv; 71e330b593905e Dave Chinner 2010-05-21 928 list_del_init(&item->li_cil); 08a813aea7434e Dave Chinner 2021-06-03 929 item->li_order_id = 0; 71e330b593905e Dave Chinner 2010-05-21 930 item->li_lv = NULL; 964c4f5053f0cd Dave Chinner 2021-06-03 931 b9844caa1f30df Dave Chinner 2021-06-03 932 num_iovecs += lv->lv_niovecs; 964c4f5053f0cd Dave Chinner 2021-06-03 933 /* we don't write ordered log vectors */ 964c4f5053f0cd Dave Chinner 2021-06-03 934 if (lv->lv_buf_len != XFS_LOG_VEC_ORDERED) 964c4f5053f0cd Dave Chinner 2021-06-03 935 num_bytes += lv->lv_bytes; b9844caa1f30df Dave Chinner 2021-06-03 936 b9844caa1f30df Dave Chinner 2021-06-03 937 list_add_tail(&lv->lv_list, &ctx->lv_chain); b9844caa1f30df Dave Chinner 2021-06-03 938 71e330b593905e Dave Chinner 2010-05-21 939 } 71e330b593905e Dave Chinner 2010-05-21 940 71e330b593905e Dave Chinner 2010-05-21 941 /* 61691dab98fb92 Dave Chinner 2021-06-03 942 * Switch the contexts so we can drop the context lock and move out 71e330b593905e Dave Chinner 2010-05-21 943 * of a shared context. We can't just go straight to the commit record, 71e330b593905e Dave Chinner 2010-05-21 944 * though - we need to synchronise with previous and future commits so 71e330b593905e Dave Chinner 2010-05-21 945 * that the commit records are correctly ordered in the log to ensure 71e330b593905e Dave Chinner 2010-05-21 946 * that we process items during log IO completion in the correct order. 71e330b593905e Dave Chinner 2010-05-21 947 * 71e330b593905e Dave Chinner 2010-05-21 948 * For example, if we get an EFI in one checkpoint and the EFD in the 71e330b593905e Dave Chinner 2010-05-21 949 * next (e.g. due to log forces), we do not want the checkpoint with 71e330b593905e Dave Chinner 2010-05-21 950 * the EFD to be committed before the checkpoint with the EFI. Hence 71e330b593905e Dave Chinner 2010-05-21 951 * we must strictly order the commit records of the checkpoints so 71e330b593905e Dave Chinner 2010-05-21 952 * that: a) the checkpoint callbacks are attached to the iclogs in the 71e330b593905e Dave Chinner 2010-05-21 953 * correct order; and b) the checkpoints are replayed in correct order 71e330b593905e Dave Chinner 2010-05-21 954 * in log recovery. 71e330b593905e Dave Chinner 2010-05-21 955 * 71e330b593905e Dave Chinner 2010-05-21 956 * Hence we need to add this context to the committing context list so 71e330b593905e Dave Chinner 2010-05-21 957 * that higher sequences will wait for us to write out a commit record 71e330b593905e Dave Chinner 2010-05-21 958 * before they do. f876e44603ad09 Dave Chinner 2014-02-27 959 * 015de1f284aa49 Dave Chinner 2021-06-03 960 * xfs_log_force_seq requires us to mirror the new sequence into the cil f876e44603ad09 Dave Chinner 2014-02-27 961 * structure atomically with the addition of this sequence to the f876e44603ad09 Dave Chinner 2014-02-27 962 * committing list. This also ensures that we can do unlocked checks f876e44603ad09 Dave Chinner 2014-02-27 963 * against the current sequence in log forces without risking f876e44603ad09 Dave Chinner 2014-02-27 964 * deferencing a freed context pointer. 71e330b593905e Dave Chinner 2010-05-21 965 */ 4bb928cdb900d0 Dave Chinner 2013-08-12 966 spin_lock(&cil->xc_push_lock); 61691dab98fb92 Dave Chinner 2021-06-03 967 xlog_cil_ctx_switch(cil, new_ctx); 4bb928cdb900d0 Dave Chinner 2013-08-12 968 spin_unlock(&cil->xc_push_lock); 71e330b593905e Dave Chinner 2010-05-21 969 up_write(&cil->xc_ctx_lock); 71e330b593905e Dave Chinner 2010-05-21 970 71e330b593905e Dave Chinner 2010-05-21 971 /* 71e330b593905e Dave Chinner 2010-05-21 972 * Build a checkpoint transaction header and write it to the log to 71e330b593905e Dave Chinner 2010-05-21 973 * begin the transaction. We need to account for the space used by the 71e330b593905e Dave Chinner 2010-05-21 974 * transaction header here as it is not accounted for in xlog_write(). b9844caa1f30df Dave Chinner 2021-06-03 975 * Add the lvhdr to the head of the lv chain we pass to xlog_write() so b9844caa1f30df Dave Chinner 2021-06-03 976 * it gets written into the iclog first. 71e330b593905e Dave Chinner 2010-05-21 977 */ 81be7bf64fac48 Dave Chinner 2021-06-03 978 xlog_cil_build_trans_hdr(ctx, &thdr, &lvhdr, num_iovecs); 964c4f5053f0cd Dave Chinner 2021-06-03 979 num_bytes += lvhdr.lv_bytes; b9844caa1f30df Dave Chinner 2021-06-03 980 list_add(&lvhdr.lv_list, &ctx->lv_chain); 71e330b593905e Dave Chinner 2010-05-21 981 395ce53dd09e23 Dave Chinner 2021-06-03 982 /* 395ce53dd09e23 Dave Chinner 2021-06-03 983 * Before we format and submit the first iclog, we have to ensure that 395ce53dd09e23 Dave Chinner 2021-06-03 984 * the metadata writeback ordering cache flush is complete. 395ce53dd09e23 Dave Chinner 2021-06-03 985 */ 395ce53dd09e23 Dave Chinner 2021-06-03 986 wait_for_completion(&bdev_flush); 395ce53dd09e23 Dave Chinner 2021-06-03 987 81be7bf64fac48 Dave Chinner 2021-06-03 988 /* 81be7bf64fac48 Dave Chinner 2021-06-03 989 * The LSN we need to pass to the log items on transaction commit is the 81be7bf64fac48 Dave Chinner 2021-06-03 990 * LSN reported by the first log vector write, not the commit lsn. If we 81be7bf64fac48 Dave Chinner 2021-06-03 991 * use the commit record lsn then we can move the tail beyond the grant 81be7bf64fac48 Dave Chinner 2021-06-03 992 * write head. 81be7bf64fac48 Dave Chinner 2021-06-03 993 */ b9844caa1f30df Dave Chinner 2021-06-03 994 error = xlog_write(log, &ctx->lv_chain, ctx->ticket, &ctx->start_lsn, b9844caa1f30df Dave Chinner 2021-06-03 995 NULL, num_bytes); b9844caa1f30df Dave Chinner 2021-06-03 996 b9844caa1f30df Dave Chinner 2021-06-03 997 /* b9844caa1f30df Dave Chinner 2021-06-03 998 * Take the lvhdr back off the lv_chain as it should not be passed b9844caa1f30df Dave Chinner 2021-06-03 999 * to log IO completion. b9844caa1f30df Dave Chinner 2021-06-03 1000 */ b9844caa1f30df Dave Chinner 2021-06-03 1001 list_del(&lvhdr.lv_list); 71e330b593905e Dave Chinner 2010-05-21 1002 if (error) 7db37c5e6575b2 Dave Chinner 2011-01-27 1003 goto out_abort_free_ticket; 71e330b593905e Dave Chinner 2010-05-21 1004 71e330b593905e Dave Chinner 2010-05-21 1005 /* 71e330b593905e Dave Chinner 2010-05-21 1006 * now that we've written the checkpoint into the log, strictly 71e330b593905e Dave Chinner 2010-05-21 1007 * order the commit records so replay will get them in the right order. 71e330b593905e Dave Chinner 2010-05-21 1008 */ 71e330b593905e Dave Chinner 2010-05-21 1009 restart: 4bb928cdb900d0 Dave Chinner 2013-08-12 1010 spin_lock(&cil->xc_push_lock); 71e330b593905e Dave Chinner 2010-05-21 1011 list_for_each_entry(new_ctx, &cil->xc_committing, committing) { ac983517ec5941 Dave Chinner 2014-05-07 1012 /* ac983517ec5941 Dave Chinner 2014-05-07 1013 * Avoid getting stuck in this loop because we were woken by the ac983517ec5941 Dave Chinner 2014-05-07 1014 * shutdown, but then went back to sleep once already in the ac983517ec5941 Dave Chinner 2014-05-07 1015 * shutdown state. ac983517ec5941 Dave Chinner 2014-05-07 1016 */ ac983517ec5941 Dave Chinner 2014-05-07 1017 if (XLOG_FORCED_SHUTDOWN(log)) { ac983517ec5941 Dave Chinner 2014-05-07 1018 spin_unlock(&cil->xc_push_lock); ac983517ec5941 Dave Chinner 2014-05-07 1019 goto out_abort_free_ticket; ac983517ec5941 Dave Chinner 2014-05-07 1020 } ac983517ec5941 Dave Chinner 2014-05-07 1021 71e330b593905e Dave Chinner 2010-05-21 1022 /* 71e330b593905e Dave Chinner 2010-05-21 1023 * Higher sequences will wait for this one so skip them. ac983517ec5941 Dave Chinner 2014-05-07 1024 * Don't wait for our own sequence, either. 71e330b593905e Dave Chinner 2010-05-21 1025 */ 71e330b593905e Dave Chinner 2010-05-21 1026 if (new_ctx->sequence >= ctx->sequence) 71e330b593905e Dave Chinner 2010-05-21 1027 continue; 71e330b593905e Dave Chinner 2010-05-21 1028 if (!new_ctx->commit_lsn) { 71e330b593905e Dave Chinner 2010-05-21 1029 /* 71e330b593905e Dave Chinner 2010-05-21 1030 * It is still being pushed! Wait for the push to 71e330b593905e Dave Chinner 2010-05-21 1031 * complete, then start again from the beginning. 71e330b593905e Dave Chinner 2010-05-21 1032 */ 4bb928cdb900d0 Dave Chinner 2013-08-12 1033 xlog_wait(&cil->xc_commit_wait, &cil->xc_push_lock); 71e330b593905e Dave Chinner 2010-05-21 1034 goto restart; 71e330b593905e Dave Chinner 2010-05-21 1035 } 71e330b593905e Dave Chinner 2010-05-21 1036 } 4bb928cdb900d0 Dave Chinner 2013-08-12 1037 spin_unlock(&cil->xc_push_lock); 71e330b593905e Dave Chinner 2010-05-21 1038 81be7bf64fac48 Dave Chinner 2021-06-03 1039 error = xlog_commit_record(log, ctx->ticket, &commit_iclog, &commit_lsn); dd401770b0ff68 Dave Chinner 2020-03-25 1040 if (error) dd401770b0ff68 Dave Chinner 2020-03-25 1041 goto out_abort_free_ticket; dd401770b0ff68 Dave Chinner 2020-03-25 1042 81be7bf64fac48 Dave Chinner 2021-06-03 1043 xfs_log_ticket_ungrant(log, ctx->ticket); 71e330b593905e Dave Chinner 2010-05-21 1044 89ae379d564c5d Christoph Hellwig 2019-06-28 1045 spin_lock(&commit_iclog->ic_callback_lock); 1858bb0bec612d Christoph Hellwig 2019-10-14 1046 if (commit_iclog->ic_state == XLOG_STATE_IOERROR) { 89ae379d564c5d Christoph Hellwig 2019-06-28 1047 spin_unlock(&commit_iclog->ic_callback_lock); 71e330b593905e Dave Chinner 2010-05-21 1048 goto out_abort; 89ae379d564c5d Christoph Hellwig 2019-06-28 1049 } 89ae379d564c5d Christoph Hellwig 2019-06-28 1050 ASSERT_ALWAYS(commit_iclog->ic_state == XLOG_STATE_ACTIVE || 89ae379d564c5d Christoph Hellwig 2019-06-28 1051 commit_iclog->ic_state == XLOG_STATE_WANT_SYNC); 89ae379d564c5d Christoph Hellwig 2019-06-28 1052 list_add_tail(&ctx->iclog_entry, &commit_iclog->ic_callbacks); 89ae379d564c5d Christoph Hellwig 2019-06-28 1053 spin_unlock(&commit_iclog->ic_callback_lock); 71e330b593905e Dave Chinner 2010-05-21 1054 71e330b593905e Dave Chinner 2010-05-21 1055 /* 71e330b593905e Dave Chinner 2010-05-21 1056 * now the checkpoint commit is complete and we've attached the 71e330b593905e Dave Chinner 2010-05-21 1057 * callbacks to the iclog we can assign the commit LSN to the context 71e330b593905e Dave Chinner 2010-05-21 1058 * and wake up anyone who is waiting for the commit to complete. 71e330b593905e Dave Chinner 2010-05-21 1059 */ 4bb928cdb900d0 Dave Chinner 2013-08-12 1060 spin_lock(&cil->xc_push_lock); 71e330b593905e Dave Chinner 2010-05-21 1061 ctx->commit_lsn = commit_lsn; eb40a87500ac2f Dave Chinner 2010-12-21 1062 wake_up_all(&cil->xc_commit_wait); 4bb928cdb900d0 Dave Chinner 2013-08-12 1063 spin_unlock(&cil->xc_push_lock); 71e330b593905e Dave Chinner 2010-05-21 1064 bde964f7e822cc Dave Chinner 2021-06-03 1065 /* bde964f7e822cc Dave Chinner 2021-06-03 1066 * If the checkpoint spans multiple iclogs, wait for all previous 80d287de7b49ed Dave Chinner 2021-06-03 1067 * iclogs to complete before we submit the commit_iclog. In this case, 80d287de7b49ed Dave Chinner 2021-06-03 1068 * the commit_iclog write needs to issue a pre-flush so that the 80d287de7b49ed Dave Chinner 2021-06-03 1069 * ordering is correctly preserved down to stable storage. bde964f7e822cc Dave Chinner 2021-06-03 1070 */ bde964f7e822cc Dave Chinner 2021-06-03 1071 spin_lock(&log->l_icloglock); 80d287de7b49ed Dave Chinner 2021-06-03 1072 if (ctx->start_lsn != commit_lsn) { bde964f7e822cc Dave Chinner 2021-06-03 1073 xlog_wait_on_iclog(commit_iclog->ic_prev); 80d287de7b49ed Dave Chinner 2021-06-03 1074 spin_lock(&log->l_icloglock); 80d287de7b49ed Dave Chinner 2021-06-03 1075 commit_iclog->ic_flags |= XLOG_ICL_NEED_FLUSH; bde964f7e822cc Dave Chinner 2021-06-03 1076 } bde964f7e822cc Dave Chinner 2021-06-03 1077 80d287de7b49ed Dave Chinner 2021-06-03 1078 /* 80d287de7b49ed Dave Chinner 2021-06-03 1079 * The commit iclog must be written to stable storage to guarantee 80d287de7b49ed Dave Chinner 2021-06-03 1080 * journal IO vs metadata writeback IO is correctly ordered on stable 80d287de7b49ed Dave Chinner 2021-06-03 1081 * storage. ed3d678659f2f3 Dave Chinner 2021-06-03 1082 * ed3d678659f2f3 Dave Chinner 2021-06-03 1083 * If the push caller needs the commit to be immediately stable and the ed3d678659f2f3 Dave Chinner 2021-06-03 1084 * commit_iclog is not yet marked as XLOG_STATE_WANT_SYNC to indicate it ed3d678659f2f3 Dave Chinner 2021-06-03 1085 * will be written when released, switch it's state to WANT_SYNC right ed3d678659f2f3 Dave Chinner 2021-06-03 1086 * now. 80d287de7b49ed Dave Chinner 2021-06-03 1087 */ 80d287de7b49ed Dave Chinner 2021-06-03 1088 commit_iclog->ic_flags |= XLOG_ICL_NEED_FUA; ed3d678659f2f3 Dave Chinner 2021-06-03 1089 if (push_commit_stable && commit_iclog->ic_state == XLOG_STATE_ACTIVE) ed3d678659f2f3 Dave Chinner 2021-06-03 1090 xlog_state_switch_iclogs(log, commit_iclog, 0); 80d287de7b49ed Dave Chinner 2021-06-03 1091 xlog_state_release_iclog(log, commit_iclog); 80d287de7b49ed Dave Chinner 2021-06-03 1092 spin_unlock(&log->l_icloglock); c7cc296ddd1f6d Christoph Hellwig 2020-03-20 1093 return; 71e330b593905e Dave Chinner 2010-05-21 1094 71e330b593905e Dave Chinner 2010-05-21 1095 out_skip: 71e330b593905e Dave Chinner 2010-05-21 1096 up_write(&cil->xc_ctx_lock); 71e330b593905e Dave Chinner 2010-05-21 1097 xfs_log_ticket_put(new_ctx->ticket); 71e330b593905e Dave Chinner 2010-05-21 1098 kmem_free(new_ctx); c7cc296ddd1f6d Christoph Hellwig 2020-03-20 1099 return; 71e330b593905e Dave Chinner 2010-05-21 1100 7db37c5e6575b2 Dave Chinner 2011-01-27 1101 out_abort_free_ticket: 81be7bf64fac48 Dave Chinner 2021-06-03 1102 xfs_log_ticket_ungrant(log, ctx->ticket); 71e330b593905e Dave Chinner 2010-05-21 1103 out_abort: 12e6a0f449d585 Christoph Hellwig 2020-03-20 1104 ASSERT(XLOG_FORCED_SHUTDOWN(log)); 12e6a0f449d585 Christoph Hellwig 2020-03-20 1105 xlog_cil_committed(ctx); 4c2d542f2e7865 Dave Chinner 2012-04-23 1106 } 4c2d542f2e7865 Dave Chinner 2012-04-23 1107 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

diff --git a/fs/xfs/xfs_log.c b/fs/xfs/xfs_log.c index 77d9ea7daf26..5511c5de6b78 100644 --- a/fs/xfs/xfs_log.c +++ b/fs/xfs/xfs_log.c @@ -848,6 +848,9 @@ xlog_write_unmount_record( .lv_niovecs = 1, .lv_iovecp = &reg, }; + LIST_HEAD(lv_chain); + INIT_LIST_HEAD(&vec.lv_list); + list_add(&vec.lv_list, &lv_chain); BUILD_BUG_ON((sizeof(struct xlog_op_header) + sizeof(struct xfs_unmount_log_format)) != @@ -863,7 +866,7 @@ xlog_write_unmount_record( */ if (log->l_targ != log->l_mp->m_ddev_targp) blkdev_issue_flush(log->l_targ->bt_bdev); - return xlog_write(log, &vec, ticket, NULL, NULL, reg.i_len); + return xlog_write(log, &lv_chain, ticket, NULL, NULL, reg.i_len); } /* @@ -1581,13 +1584,16 @@ xlog_commit_record( .lv_iovecp = &reg, }; int error; + LIST_HEAD(lv_chain); + INIT_LIST_HEAD(&vec.lv_list); + list_add(&vec.lv_list, &lv_chain); if (XLOG_FORCED_SHUTDOWN(log)) return -EIO; /* account for space used by record data */ ticket->t_curr_res -= reg.i_len; - error = xlog_write(log, &vec, ticket, lsn, iclog, reg.i_len); + error = xlog_write(log, &lv_chain, ticket, lsn, iclog, reg.i_len); if (error) xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR); return error; @@ -2118,6 +2124,7 @@ xlog_print_trans( */ static struct xfs_log_vec * xlog_write_single( + struct list_head *lv_chain, struct xfs_log_vec *log_vector, struct xlog_ticket *ticket, struct xlog_in_core *iclog, @@ -2134,7 +2141,9 @@ xlog_write_single( iclog->ic_state == XLOG_STATE_WANT_SYNC); ptr = iclog->ic_datap + *log_offset; - for (lv = log_vector; lv; lv = lv->lv_next) { + for (lv = log_vector; + !list_entry_is_head(lv, lv_chain, lv_list); + lv = list_next_entry(lv, lv_list)) { /* * If the entire log vec does not fit in the iclog, punt it to * the partial copy loop which can handle this case. @@ -2163,6 +2172,8 @@ xlog_write_single( *data_cnt += reg->i_len; } } + if (list_entry_is_head(lv, lv_chain, lv_list)) + lv = NULL; ASSERT(*len == 0 || lv); return lv; } @@ -2208,6 +2219,7 @@ xlog_write_get_more_iclog_space( static struct xfs_log_vec * xlog_write_partial( struct xlog *log, + struct list_head *lv_chain, struct xfs_log_vec *log_vector, struct xlog_ticket *ticket, struct xlog_in_core **iclogp, @@ -2347,7 +2359,10 @@ xlog_write_partial( * the caller so it can go back to fast path copying. */ *iclogp = iclog; - return lv->lv_next; + lv = list_next_entry(lv, lv_list); + if (list_entry_is_head(lv, lv_chain, lv_list)) + return NULL; + return lv; } /* @@ -2393,14 +2408,14 @@ xlog_write_partial( int xlog_write( struct xlog *log, - struct xfs_log_vec *log_vector, + struct list_head *lv_chain, struct xlog_ticket *ticket, xfs_lsn_t *start_lsn, struct xlog_in_core **commit_iclog, uint32_t len) { struct xlog_in_core *iclog = NULL; - struct xfs_log_vec *lv = log_vector; + struct xfs_log_vec *lv; int record_cnt = 0; int data_cnt = 0; int error = 0; @@ -2422,14 +2437,16 @@ xlog_write( if (start_lsn) *start_lsn = be64_to_cpu(iclog->ic_header.h_lsn); + lv = list_first_entry_or_null(lv_chain, struct xfs_log_vec, lv_list); while (lv) { - lv = xlog_write_single(lv, ticket, iclog, &log_offset, + lv = xlog_write_single(lv_chain, lv, ticket, iclog, &log_offset, &len, &record_cnt, &data_cnt); if (!lv) break; - lv = xlog_write_partial(log, lv, ticket, &iclog, &log_offset, - &len, &record_cnt, &data_cnt); + lv = xlog_write_partial(log, lv_chain, lv, ticket, &iclog, + &log_offset, &len, &record_cnt, + &data_cnt); if (IS_ERR_OR_NULL(lv)) { error = PTR_ERR_OR_ZERO(lv); break; diff --git a/fs/xfs/xfs_log.h b/fs/xfs/xfs_log.h index af54ea3f8c90..b4ad0e37a0c5 100644 --- a/fs/xfs/xfs_log.h +++ b/fs/xfs/xfs_log.h @@ -9,7 +9,7 @@ struct xfs_cil_ctx; struct xfs_log_vec { - struct xfs_log_vec *lv_next; /* next lv in build list */ + struct list_head lv_list; /* CIL lv chain ptrs */ int lv_niovecs; /* number of iovecs in lv */ struct xfs_log_iovec *lv_iovecp; /* iovec array */ struct xfs_log_item *lv_item; /* owner */ diff --git a/fs/xfs/xfs_log_cil.c b/fs/xfs/xfs_log_cil.c index 3ca9fce57584..65910fca652f 100644 --- a/fs/xfs/xfs_log_cil.c +++ b/fs/xfs/xfs_log_cil.c @@ -73,6 +73,7 @@ xlog_cil_ctx_alloc(void) INIT_LIST_HEAD(&ctx->committing); INIT_LIST_HEAD(&ctx->busy_extents); INIT_LIST_HEAD(&ctx->log_items); + INIT_LIST_HEAD(&ctx->lv_chain); INIT_WORK(&ctx->push_work, xlog_cil_push_work); return ctx; } @@ -274,6 +275,7 @@ xlog_cil_alloc_shadow_bufs( lv = kmem_alloc_large(buf_size, KM_NOFS); memset(lv, 0, xlog_cil_iovec_space(niovecs)); + INIT_LIST_HEAD(&lv->lv_list); lv->lv_item = lip; lv->lv_size = buf_size; if (ordered) @@ -289,7 +291,6 @@ xlog_cil_alloc_shadow_bufs( else lv->lv_buf_len = 0; lv->lv_bytes = 0; - lv->lv_next = NULL; } /* Ensure the lv is set up according to ->iop_size */ @@ -416,7 +417,6 @@ xlog_cil_insert_format_items( if (lip->li_lv && shadow->lv_size <= lip->li_lv->lv_size) { /* same or smaller, optimise common overwrite case */ lv = lip->li_lv; - lv->lv_next = NULL; if (ordered) goto insert; @@ -583,14 +583,14 @@ xlog_cil_insert_items( static void xlog_cil_free_logvec( - struct xfs_log_vec *log_vector) + struct list_head *lv_chain) { struct xfs_log_vec *lv; - for (lv = log_vector; lv; ) { - struct xfs_log_vec *next = lv->lv_next; + while (!list_empty(lv_chain)) { + lv = list_first_entry(lv_chain, struct xfs_log_vec, lv_list); + list_del_init(&lv->lv_list); kmem_free(lv); - lv = next; } } @@ -689,7 +689,7 @@ xlog_cil_committed( spin_unlock(&ctx->cil->xc_push_lock); } - xfs_trans_committed_bulk(ctx->cil->xc_log->l_ailp, ctx->lv_chain, + xfs_trans_committed_bulk(ctx->cil->xc_log->l_ailp, &ctx->lv_chain, ctx->start_lsn, abort); xfs_extent_busy_sort(&ctx->busy_extents); @@ -700,7 +700,7 @@ xlog_cil_committed( list_del(&ctx->committing); spin_unlock(&ctx->cil->xc_push_lock); - xlog_cil_free_logvec(ctx->lv_chain); + xlog_cil_free_logvec(&ctx->lv_chain); if (!list_empty(&ctx->busy_extents)) xlog_discard_busy_extents(mp, ctx); @@ -780,7 +780,6 @@ xlog_cil_build_trans_hdr( lvhdr->lv_niovecs = 2; lvhdr->lv_iovecp = &hdr->lhdr[0]; lvhdr->lv_bytes = hdr->lhdr[0].i_len + hdr->lhdr[1].i_len; - lvhdr->lv_next = ctx->lv_chain; tic->t_curr_res -= lvhdr->lv_bytes; } @@ -920,25 +919,23 @@ xlog_cil_push_work( xlog_cil_pcp_aggregate(cil, ctx); list_sort(NULL, &ctx->log_items, xlog_cil_order_cmp); - while (!list_empty(&ctx->log_items)) { struct xfs_log_item *item; item = list_first_entry(&ctx->log_items, struct xfs_log_item, li_cil); + lv = item->li_lv; list_del_init(&item->li_cil); item->li_order_id = 0; - if (!ctx->lv_chain) - ctx->lv_chain = item->li_lv; - else - lv->lv_next = item->li_lv; - lv = item->li_lv; item->li_lv = NULL; - num_iovecs += lv->lv_niovecs; + num_iovecs += lv->lv_niovecs; /* we don't write ordered log vectors */ if (lv->lv_buf_len != XFS_LOG_VEC_ORDERED) num_bytes += lv->lv_bytes; + + list_add_tail(&lv->lv_list, &ctx->lv_chain); + } /* @@ -975,9 +972,12 @@ xlog_cil_push_work( * Build a checkpoint transaction header and write it to the log to * begin the transaction. We need to account for the space used by the * transaction header here as it is not accounted for in xlog_write(). + * Add the lvhdr to the head of the lv chain we pass to xlog_write() so + * it gets written into the iclog first. */ xlog_cil_build_trans_hdr(ctx, &thdr, &lvhdr, num_iovecs); num_bytes += lvhdr.lv_bytes; + list_add(&lvhdr.lv_list, &ctx->lv_chain); /* * Before we format and submit the first iclog, we have to ensure that @@ -991,8 +991,14 @@ xlog_cil_push_work( * use the commit record lsn then we can move the tail beyond the grant * write head. */ - error = xlog_write(log, &lvhdr, ctx->ticket, &ctx->start_lsn, NULL, - num_bytes); + error = xlog_write(log, &ctx->lv_chain, ctx->ticket, &ctx->start_lsn, + NULL, num_bytes); + + /* + * Take the lvhdr back off the lv_chain as it should not be passed + * to log IO completion. + */ + list_del(&lvhdr.lv_list); if (error) goto out_abort_free_ticket; diff --git a/fs/xfs/xfs_log_priv.h b/fs/xfs/xfs_log_priv.h index d3bf3b367370..071367a96d8d 100644 --- a/fs/xfs/xfs_log_priv.h +++ b/fs/xfs/xfs_log_priv.h @@ -221,7 +221,7 @@ struct xfs_cil_ctx { atomic_t space_used; /* aggregate size of regions */ struct list_head busy_extents; /* busy extents in chkpt */ struct list_head log_items; /* log items in chkpt */ - struct xfs_log_vec *lv_chain; /* logvecs being pushed */ + struct list_head lv_chain; /* logvecs being pushed */ struct list_head iclog_entry; struct list_head committing; /* ctx committing list */ struct work_struct discard_endio_work; @@ -477,7 +477,7 @@ xlog_write_adv_cnt(void **ptr, int *len, int *off, size_t bytes) void xlog_print_tic_res(struct xfs_mount *mp, struct xlog_ticket *ticket); void xlog_print_trans(struct xfs_trans *); -int xlog_write(struct xlog *log, struct xfs_log_vec *log_vector, +int xlog_write(struct xlog *log, struct list_head *lv_chain, struct xlog_ticket *tic, xfs_lsn_t *start_lsn, struct xlog_in_core **commit_iclog, uint32_t len); int xlog_commit_record(struct xlog *log, struct xlog_ticket *ticket, diff --git a/fs/xfs/xfs_trans.c b/fs/xfs/xfs_trans.c index bc72826d1f97..0f8300adb12d 100644 --- a/fs/xfs/xfs_trans.c +++ b/fs/xfs/xfs_trans.c @@ -735,7 +735,7 @@ xfs_log_item_batch_insert( void xfs_trans_committed_bulk( struct xfs_ail *ailp, - struct xfs_log_vec *log_vector, + struct list_head *lv_chain, xfs_lsn_t commit_lsn, bool aborted) { @@ -750,7 +750,7 @@ xfs_trans_committed_bulk( spin_unlock(&ailp->ail_lock); /* unpin all the log items */ - for (lv = log_vector; lv; lv = lv->lv_next ) { + list_for_each_entry(lv, lv_chain, lv_list) { struct xfs_log_item *lip = lv->lv_item; xfs_lsn_t item_lsn; diff --git a/fs/xfs/xfs_trans_priv.h b/fs/xfs/xfs_trans_priv.h index 3004aeac9110..fc8667c728e3 100644 --- a/fs/xfs/xfs_trans_priv.h +++ b/fs/xfs/xfs_trans_priv.h @@ -18,7 +18,8 @@ void xfs_trans_add_item(struct xfs_trans *, struct xfs_log_item *); void xfs_trans_del_item(struct xfs_log_item *); void xfs_trans_unreserve_and_mod_sb(struct xfs_trans *tp); -void xfs_trans_committed_bulk(struct xfs_ail *ailp, struct xfs_log_vec *lv, +void xfs_trans_committed_bulk(struct xfs_ail *ailp, + struct list_head *lv_chain, xfs_lsn_t commit_lsn, bool aborted); /* * AIL traversal cursor.

[35/39] xfs: convert log vector chain to use list heads

Commit Message

Comments

Patch