From patchwork Fri Feb 24 19:53:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Foster X-Patchwork-Id: 9591115 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 970CC60578 for ; Fri, 24 Feb 2017 19:53:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89F2A2875C for ; Fri, 24 Feb 2017 19:53:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7EF882894A; Fri, 24 Feb 2017 19:53:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 209A128949 for ; Fri, 24 Feb 2017 19:53:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751515AbdBXTxa (ORCPT ); Fri, 24 Feb 2017 14:53:30 -0500 Received: from mx1.redhat.com ([209.132.183.28]:56404 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751513AbdBXTxW (ORCPT ); Fri, 24 Feb 2017 14:53:22 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EADB68048F; Fri, 24 Feb 2017 19:53:22 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-20.bos.redhat.com [10.18.41.20]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1OJrM3V029199; Fri, 24 Feb 2017 14:53:22 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id 3A66F1228ED; Fri, 24 Feb 2017 14:53:21 -0500 (EST) From: Brian Foster To: linux-xfs@vger.kernel.org Cc: Dave Chinner , Martin Svec Subject: [PATCH 3/3] [RFC] xfs: release buffer list after quotacheck buf reset Date: Fri, 24 Feb 2017 14:53:21 -0500 Message-Id: <1487966001-63263-4-git-send-email-bfoster@redhat.com> In-Reply-To: <1487966001-63263-1-git-send-email-bfoster@redhat.com> References: <1487966001-63263-1-git-send-email-bfoster@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 24 Feb 2017 19:53:22 +0000 (UTC) Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP XXX: This patch is broken and should not be merged. This purpose of this patch is to consider the notion of reducing quotacheck memory consumption requirements via earlier release of the buffer list. The intent is to allow dquot and buffer reclaim to operate as normal during the subsequent bulkstat (dqadjust) and dquot flush walk. In turn, this allows quotacheck to complete in environments with tight memory constraints. As it is, this approach has several unresolved tradeoffs/problems: - The change is limited in effectiveness to situations where the total dquot allocation requirements are the difference between OOM and a successful mount. OOM is still possible as we still allocate the entire dquot buffer space. - In limited but non-OOM situations, this can cause quotacheck to take significantly longer than normal. On a filesystem with ~4 million inodes and ~600k project quotas, a quotacheck typically runs for ~50s in a local vm limited to 1GB RAM. With this change, the same quotacheck requires anywhere from ~9-12m in my tests. - By releasing the buffer list and allowing reclaim to do some of the work, quotacheck has lost serialization against I/O completion of the adjusted dquots. This creates the potential for corruption if the filesystem crashes after quotacheck completion status has been logged but before dquot I/O has fully completed. IOWs, release of the buffer list as such requires the addition of a new, so far undefined post-quotacheck serialization sequence. One option here may be a post delwri submit walk of in-core dquots purely to cycle the flush locks, the idea being that dquots can't be fully reclaimed until the flush lock is unlocked and thus buffer I/O has completed. Another option may be a post-quotacheck xfs_wait_buftarg(). - This patch leads to some apparent buffer refcounting issues (unrelated to the pushbuf bits). I reproduce _XBF_DELWRI_Q buffer release time asserts and unmount hangs that need to be tracked down and resolved. It is not yet clear whether these are generic problems or require further changes in the quotacheck implementation to deal with the fact that quotacheck can no longer assume exclusive access to the dquot buffers. --- fs/xfs/xfs_qm.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c index 3815ed3..b072495 100644 --- a/fs/xfs/xfs_qm.c +++ b/fs/xfs/xfs_qm.c @@ -1267,10 +1267,9 @@ xfs_qm_flush_one( * cycle the flush lock. */ if (!xfs_dqflock_nowait(dqp)) { - /* buf is pinned in-core by delwri list */ - DEFINE_SINGLE_BUF_MAP(map, dqp->q_blkno, - mp->m_quotainfo->qi_dqchunklen); - bp = _xfs_buf_find(mp->m_ddev_targp, &map, 1, 0, NULL); + bp = xfs_buf_read(mp->m_ddev_targp, dqp->q_blkno, + mp->m_quotainfo->qi_dqchunklen, 0, + &xfs_dquot_buf_ops); if (!bp) { error = -EINVAL; goto out_unlock; @@ -1351,6 +1350,10 @@ xfs_qm_quotacheck( flags |= XFS_PQUOTA_CHKD; } + error = xfs_buf_delwri_submit(&buffer_list); + if (error) + goto error_return; + do { /* * Iterate thru all the inodes in the file system,