From patchwork Fri May 27 16:58:35 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 9138757 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 50BFC6075C for ; Fri, 27 May 2016 16:58:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4308727B33 for ; Fri, 27 May 2016 16:58:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 37B3E28159; Fri, 27 May 2016 16:58:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D48527B33 for ; Fri, 27 May 2016 16:58:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756248AbcE0Q6q (ORCPT ); Fri, 27 May 2016 12:58:46 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:34319 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754539AbcE0Q6o (ORCPT ); Fri, 27 May 2016 12:58:44 -0400 Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.11/8.16.0.11) with SMTP id u4RGtnmY003508 for ; Fri, 27 May 2016 09:58:43 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : subject : date : message-id : mime-version : content-type; s=facebook; bh=nVzVuTeDaxWoN7zOMp2DHg0G4zTNK/40psFphJzXY3U=; b=jEK5WtuobqMUvTNa5oBa8K3a0BY3Tu4oFDq2c4VnsHE/eqlJRGKNaLnwdSNzi5SptNkH 9iHXlrHD+KkBJRUAPm3WrgnAuAbGF71YVTWVEuF9j9NS8FTyQpQ2Xy9+zK4O2GmFArH9 uXOzcesWEG82C6hTi5D/q1PFt8ZWAqB4X2I= Received: from maileast.thefacebook.com ([199.201.65.23]) by m0089730.ppops.net with ESMTP id 236rfagx01-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 27 May 2016 09:58:43 -0700 Received: from na01-bn1-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.25) with Microsoft SMTP Server (TLS) id 14.3.294.0; Fri, 27 May 2016 12:58:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hK8U8Q+TEPqUMPAiJJaYeET68rQkal9haopq2rkk9mo=; b=HkqMC0XPQKTYLaJqbJKL4HmM8XLuymrxYgj0B+4oYdXzYJIY2iZKiOIvxRLQ1l+RB5QUPm5WStBf6EkXjDpHPqzyn73elVznMbxznCexTvuXo9xkH4rG7KmcKZiuLkuAcByPMuiwQGKsUQ41EDZ18z14r3qBSnYrnKOI2hO8r3k= Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=none action=none header.from=fb.com; Received: from localhost (107.15.72.49) by BY2PR15MB0839.namprd15.prod.outlook.com (10.164.171.133) with Microsoft SMTP Server (TLS) id 15.1.506.9; Fri, 27 May 2016 16:58:40 +0000 From: Josef Bacik To: , Subject: [PATCH V2] Btrfs: change how we calculate the global block rsv Date: Fri, 27 May 2016 12:58:35 -0400 Message-ID: <1464368315-3046-1-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 X-Originating-IP: [107.15.72.49] X-ClientProxiedBy: DM2PR03CA0029.namprd03.prod.outlook.com (10.141.96.28) To BY2PR15MB0839.namprd15.prod.outlook.com (10.164.171.133) X-MS-Office365-Filtering-Correlation-Id: 6fa1e289-d773-489a-30b8-08d386502741 X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0839; 2:lPNQ01isHLN57ubuSRLhkdAufUpEiHiIo8fL6QKTIRK4ZQ4waSCLAmZD1wn3cXSk+off0DYJGSTFnHsyH1OwBykNLFP36k6sWj14k9kh8Ip+u67ahKszqEM7egktC6FCSFnKJwArqRMeDa2w0IRbL1OEnYNMuf47L+uPMiDrQOnmWkwbrDaYyJzb8IRwpPyX; 3:zkvZEHUwzM8qbXRpIWmvplhGArNW94N35DQ+Yc9ZFekGS3tJWC/xoZ5964XDt0x+iDq1KWGF+8bIfb8fAZ8t7Zn8GUaHZ/6UX8qf3yBI81mjhQV443oAesjLIdjQqQWc X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR15MB0839; X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0839; 25:Ejxwyt++2ZDsMGb9uojV8EjNAI/xmilCh1p6FJatc6dh4lRNZF6FWr8g+LFOmv63xG//jt3awI39IE729BJT1u5Ik6mS9oTAH8mFl3suD7QFNI1tFMjqHamcl25FZcrLKwQuw6h5ETT6gXwkyF032J1g2Ic/jIiRzo/Dq5PmHgMqyXxp+1EbUY5VBMFAt9q+ODSiRRsSkUoQpsREnXGvyg5Ro4GqtaDVu+E8ZpnegyHTTBGZFXJDjbOlZprK0lsJB+RVykqX8wHpfDglkm9ncV2hr2b2H1ZKQKxqoKz8JwQuKfpdw+Jfx23d+5ipZzIPrguf1v01+1fDSx7DOGxbV7RtMakBGnfEigTLqh1C1azphI6ufY66wwSpV9QTrMQ0O6N0GDZrLgOn/Nod/W4ff241nUAaJphafiveNdxhXBDD23v/ha+QS3+HBioKpoXEePrnlPij+Ymqy3XynHddcLZ+XRfYrWVWOtKKM7tvAF1UJ8OP0oZlRjL0kaYJeK3g2e+/b02anmohWuZFCXoAwR6h5C1E2zAZqBQ9a8RaqrW0QJQoZMcmwpCoFz7sgAcT0lbCOw2kN/0clViT2bAoSLPSje3mvREX/Ax+klyn5J73g8PPC3lvTq2qMWmuYH8tsfnd8lStihaWhfnwx3kWwVmmqW/9TpUHuiNfAH+k+oaL/ZU7BJIgHP3hahNQ2xGZOqnFs5kEE8eyd/jRbuQO1JhL7ClynFPiuj+DGiXWoPo=; 20:7IADWRHwQNkM29TJZPWhekKnUkqDtj1vM4AfCcegzt1RaFhbxiJM45Sf+uoyouUse+2Nj2dtJreXdMnGpCzI7vg3f4mW/ga3fk+yFBM+qv0Ydf7XDpGlCApLGY4pF9Kpu7+ImI0DMwwO/Ro88Z2kyWoPe+fhX/Lb3XhHsi77ung= X-LD-Processed: 8ae927fe-1255-47a7-a2af-5f3a069daaa2,ExtAddr X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(67672495146484); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001); SRVR:BY2PR15MB0839; BCL:0; PCL:0; RULEID:; SRVR:BY2PR15MB0839; X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0839; 4:+9FICXEnGZRcaJAYW+KQkpvVxUUz3V6y7G6JQ9BVKMehQEiUMDQ9BbFSjhY3PSPIENg4VRUo4zKN7wIowVvi8fRc9sqTXT9lZpWU0rq3BDU7ATxVTv14A/Jnkxrx9+A10LcSKO1JV+V6qJecBknXIIxi/K13phKSVNJ+A5LPV9iEKpQVu4RFEOBmcJ50faeYcNGSBHFhtmP4W8RoqhVULWpikpBmKrcXv7p+n/hIP457xUDLV7eESWrG0yYHdyx1NkOnEX8OtyXjpK5uzg+bhNqbcS6HHo/VPOnNm7rY5b6FEZx5ZsVoZ0GE2Avc4AIqNhWByjNo46+7oIwOOSQV+vrS3D30nlFEz0eTirGe1ewCxvzGv1MwljYrEoft+gbMlPSN0hRPHUp08gbEbdtRT99gWm1NYBjR7USDtTDXF38= X-Forefront-PRVS: 09555FB1AD X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6069001)(6009001)(48376002)(50466002)(77096005)(19580405001)(19580395003)(36756003)(107886002)(189998001)(5008740100001)(92566002)(5001770100001)(2906002)(66066001)(586003)(47776003)(6116002)(3846002)(50226002)(76506005)(5003940100001)(42186005)(5004730100002)(229853001)(50986999)(86362001)(450100001)(33646002)(81166006)(8676002)(551934003)(4720700001); DIR:OUT; SFP:1102; SCL:1; SRVR:BY2PR15MB0839; H:localhost; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0839; 23:cPEG0r0QLik3e/zFXk86tYMrviROMB2olv9v9RGGfNPIWJt2iMikCwsHIgvuVhfcw0Qnh+cFQtUNmVFMXBWZp4DCttzhJFQJmaFAd5aCKX2YbfWVfc96EwIH/hJh/QuTECoOAvB3bobyG8BxnkZqWzwPzPi0QsMIcxsixy9yEgR6+gDYRFTYCFvdAYup0Fz7SZ2fd9OPRiHhauTHKdLJfDmPxlcy39HhUIgHyuzPIX/Q1myd6tGR9xr7bnOXlTmjYkW+rcVyBbPB9c4dLOZdidtLoEZlHv94vInwjJtOKweXG89OLOQ+0hHt88Ag2CTbzbfdSTXvD7tN9BRrgTm2bphh0A8KIvqw5GKmydvE7xNKbIukjlmpWhMTA+VeL4lHN+IfQMygBEP+0G9kXRqYJhxISuCwTtcWrWzPLzgERDXbitaWWSimXrkV6a/u8kJJPWibkSLBTmcPL/ElOJVoxfn0QGrIbInWTL0/1Ij49N2L5FoP/xjAQE3R8j6FZ+HfZWHXVFY/2IpEm688euRAE38XdRJpa/RAufLog5T8wt1tECWdCphVrapXWUzvF0Lbw/YuFI2iePQ7AYhy4xar2S8k8hACBkuE9n1zjfWSFkg/dVZuYW1MAvhjYCGq9KJLHxeQMVjqcvnpqq92arVYdGt6yLbMA7tvidEkQwZI0yKDBCETeRxxP85h678W04vJXrL9kqc+jXvEXzpFeSHH/6R2gIyLypEuGUafWMbCz87zuc38ge2Hlmq/ZMMAQo83n3N/IU+0MiUTSz0PzgI0x5ALX0d2enWncFL1INn+aI+M1NflbNwN0PCgROmuJK72+ucsWPt33xmd/c+uZ89bWYgZkuJ/jkM4+VdJE0H8sglcPJjuv7AE9MgACdrviwKXiSSykghjPp+LZEwdtI2pCnW1yKQbVcYvWwh3rE9Qh RUwfrUEaD/EQwhaRuLzeUiI X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0839; 5:nVAxDx4FbJipELMfqqE0RiZcSj+YCGyOn2j531WoTbiRbc1T7X92/YotKB+b9LIKuKlAkwtr7v1jWS1sk3V6kWI46pHsOpP7hKk0DmtZvr7yI+PI60iPdDc9mnUXESZqvcNUYEyPvwe0UDWlf9wbyQ==; 24:QwsmYwdrLxpWmuSm/mlZV/Tf8l/MAi2keJhXKISmZ7SQBfcCVsnGo2JH9MpIcHSiVGAHIQmGl+4lAcb2WW82hy9/lgkVmgVI3/mYv8FfHj4=; 7:YwgPa01+pInyLEZ0wBBqYNJ3KqQTArGh9gh7c2WbZwyH560W6T9A1gObX7AmFn5l+Db72gS7f/KCKTkRc21//PvOSvmLNwf2MfTE/P6oJPzdMogh0jRKaaQDewPnJpsG5F/CfJIcJTJrPjv4+SHQpCKWwaHAJYbGYXZ28g89PIzh/0pV925PmNk3aSZKtg0G; 20:Zlk627AkHRQRTqfZoffEoMjDBrTV4psTpB28kaUolfpGaH3OQ1Gqkj1ePfIlUDmHCcmwIALJCEvg4UPZRSFh4o/DaZUr6minxyzrMSxiun1RnnRiDNC7hilFkVWqQnxqJv+FdznQcqm487CzPbdF6x4CtoEqSCKgy175oDCcca8= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2016 16:58:40.0961 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR15MB0839 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-05-27_10:, , signatures=0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Traditionally we've calculated the global block rsv by guessing how much of the metadata used amount was the extent tree, and then taking the data size and figuring out how large the csum tree would have to be to hold that much data. This is imprecise and falls down on MIXED file systems as we can't trust the data used amount. This resulted in failures for xfstests generic/333 because it creates lots of clones, which explodes out the extent tree. Our global reserve calculations were woefully inaccurate in this case which meant we got into a situation where we did not have enough reserved to do our work. We know we only use the global block rsv for the extent, csum, and root trees, so just get the bytes used for these trees and use that as the basis of our global reserve. Since these are not reference counted trees the bytes_used value will be accurate. This fixed the transaction aborts seen with generic/333. Thanks, Signed-off-by: Josef Bacik --- V1->V2: Make sure to set a minimum global rsv size. fs/btrfs/extent-tree.c | 45 +++++++++------------------------------------ 1 file changed, 9 insertions(+), 36 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 1213a59..43bc9f5 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5550,48 +5550,21 @@ void btrfs_block_rsv_release(struct btrfs_root *root, num_bytes); } -/* - * helper to calculate size of global block reservation. - * the desired value is sum of space used by extent tree, - * checksum tree and root tree - */ -static u64 calc_global_metadata_size(struct btrfs_fs_info *fs_info) -{ - struct btrfs_space_info *sinfo; - u64 num_bytes; - u64 meta_used; - u64 data_used; - int csum_size = btrfs_super_csum_size(fs_info->super_copy); - - sinfo = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_DATA); - spin_lock(&sinfo->lock); - data_used = sinfo->bytes_used; - spin_unlock(&sinfo->lock); - - sinfo = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); - spin_lock(&sinfo->lock); - if (sinfo->flags & BTRFS_BLOCK_GROUP_DATA) - data_used = 0; - meta_used = sinfo->bytes_used; - spin_unlock(&sinfo->lock); - - num_bytes = (data_used >> fs_info->sb->s_blocksize_bits) * - csum_size * 2; - num_bytes += div_u64(data_used + meta_used, 50); - - if (num_bytes * 3 > meta_used) - num_bytes = div_u64(meta_used, 3); - - return ALIGN(num_bytes, fs_info->extent_root->nodesize << 10); -} - static void update_global_block_rsv(struct btrfs_fs_info *fs_info) { struct btrfs_block_rsv *block_rsv = &fs_info->global_block_rsv; struct btrfs_space_info *sinfo = block_rsv->space_info; u64 num_bytes; - num_bytes = calc_global_metadata_size(fs_info); + /* + * The global block rsv is based on the size of the extent tree, the + * checksum tree and the root tree. If the fs is empty we want to set + * it to a minimal amount for safety. + */ + num_bytes = btrfs_root_used(&fs_info->extent_root->root_item) + + btrfs_root_used(&fs_info->csum_root->root_item) + + btrfs_root_used(&fs_info->tree_root->root_item); + num_bytes = max_t(u64, num_bytes, SZ_16M); spin_lock(&sinfo->lock); spin_lock(&block_rsv->lock);