From patchwork Tue Mar 26 17:45:54 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 2342311 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 3D03FE00E6 for ; Tue, 26 Mar 2013 17:46:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754213Ab3CZRp7 (ORCPT ); Tue, 26 Mar 2013 13:45:59 -0400 Received: from dkim1.fusionio.com ([66.114.96.53]:57362 "EHLO dkim1.fusionio.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755808Ab3CZRp5 (ORCPT ); Tue, 26 Mar 2013 13:45:57 -0400 Received: from mx1.fusionio.com (unknown [10.101.1.160]) by dkim1.fusionio.com (Postfix) with ESMTP id 304F07C04E0 for ; Tue, 26 Mar 2013 11:45:57 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fusionio.com; s=default; t=1364319957; bh=DgrzO6bkUDrn/jDq/lCQZXuBN9kXFGHbMmSfTLRNniw=; h=Date:From:To:CC:Subject:References:In-Reply-To; b=Q9dYcoWfDecXPJ5IwdQTsrQ5VD1Xi9R5NklQ2w9E0EdMY0XLFpp743rgCTCyE72Ud Mv4sz8sw6xNIFtTYSlDKL6R/QWw5pPg6gGh6xmn7dt4v9TpMRUIMb4pTgQ0SQRFoCd L5hv79rtWzx9u4umAwGZCdND2G5bqH+vUmS5rM0Q= X-ASG-Debug-ID: 1364319956-03d6a52abef7440001-6jHSXT Received: from mail1.int.fusionio.com (mail1.int.fusionio.com [10.101.1.21]) by mx1.fusionio.com with ESMTP id GAdqbI4ld1mP4aok (version=TLSv1 cipher=AES128-SHA bits=128 verify=NO); Tue, 26 Mar 2013 11:45:56 -0600 (MDT) X-Barracuda-Envelope-From: JBacik@fusionio.com Received: from localhost (98.26.82.158) by mail.fusionio.com (10.101.1.19) with Microsoft SMTP Server (TLS) id 8.3.83.0; Tue, 26 Mar 2013 11:45:55 -0600 Date: Tue, 26 Mar 2013 13:45:54 -0400 From: Josef Bacik To: Stefan Priebe CC: Josef Bacik , Chris Mason , "linux-btrfs@vger.kernel.org" Subject: Re: No space left on device (28) Message-ID: <20130326174554.GB28030@localhost.localdomain> X-ASG-Orig-Subj: Re: No space left on device (28) References: <20130325201426.GG1955@localhost.localdomain> <51515226.9010509@profihost.ag> <20130326125319.GH1955@localhost.localdomain> <51519ABB.4030207@profihost.ag> <20130326133053.GK1955@localhost.localdomain> <5151A75C.6080703@profihost.ag> <20130326144426.GL1955@localhost.localdomain> <5151B8AF.60900@profihost.ag> <20130326152553.GN1955@localhost.localdomain> <5151CA87.3050801@profihost.ag> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <5151CA87.3050801@profihost.ag> User-Agent: Mutt/1.5.21 (2011-07-01) X-Barracuda-Connect: mail1.int.fusionio.com[10.101.1.21] X-Barracuda-Start-Time: 1364319956 X-Barracuda-Encrypted: AES128-SHA X-Barracuda-URL: http://10.101.1.180:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at fusionio.com X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.126309 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On Tue, Mar 26, 2013 at 10:19:19AM -0600, Stefan Priebe wrote: > HI, > > > Am 26.03.2013 16:25, schrieb Josef Bacik: > > On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: > >> Hi, > >> Am 26.03.2013 15:44, schrieb Josef Bacik: > >>>>>> Am 26.03.2013 13:53, schrieb Josef Bacik: > >>>>>> no - it's just mounted with mount -o noatime > >>>>>> > >>>>>> :~# cat /proc/mounts | grep btrfs > >>>>>> /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 > >>>>>> > >>>>> > >>>>> Ok I think I see what's going on. Can you try this patch and see if it fixes > >>>>> it? Thanks, > >>>> > >>>> It still does not fix the problem. > >>>> > >>>> The rsync output looks like this so it does not work for file a but then > >>>> continues on c d e, ... > >>>> > >>>> sync -av --progress /backup/ /mnt/ > >>>> sending incremental file list > >>>> .etc_openvpn/ipp.txt > >>>> 229 100% 3.99kB/s 0:00:00 (xfer#2, to-check=1009/1196) > >>>> .etc_openvpn/openvpn-status.log > >>>> 360 100% 6.28kB/s 0:00:00 (xfer#3, to-check=1007/1196) > >>>> rsync: rename "/mnt/.etc_openvpn/.ipp.txt.t9lucX" -> > >>>> ".etc_openvpn/ipp.txt": No space left on device (28) > >>>> .log/ > >>>> .log/UcliEvt.log > >>>> 104188 100% 147.67kB/s 0:00:00 (xfer#4, to-check=1131/2700) > >>>> .log/auth.log > >>>> 15211522 100% 2.97MB/s 0:00:04 (xfer#5, to-check=1105/2700) > >>>> .log/auth.log.1 > >>>> 19431424 61% 7.35MB/s 0:00:01 > >>>> > >>>> the dmesg output looks like this: > >>>> [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush > >>>> 2, flush_state 7 dumping space info > >>>> [ 551.323694] space_info 4 has 6439526400 free, is full > >>>> [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, > >>>> reserved=49152, may_use=6438453248, readonly=65536 > >>>> > >>> > >>> Ok so then this is probably it, let me know if it helps. Thanks, > >> > >> OK it now has copied a lot of files (170) without an error all were very > >> small. > >> > > > > Welp progress is good. Throw this into the mix and go again, it's just adding > > some more debugging so I can make sure I'm going down the right rabbit hole. > > Thanks, > > Output is now: > [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush > 2, flush_state 7 dumping space info > [ 9587.527392] dumping block rsv 2, size 0 reserved 0 > [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 > [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 > [ 9587.646958] space_info 4 has 6439428096 free, is full > [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, > reserved=45056, may_use=6438453248, readonly=65536 > [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush > 2, flush_state 7 dumping space info > [ 9587.727000] dumping block rsv 2, size 0 reserved 0 > [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 > [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 > [ 9587.839935] space_info 4 has 6439428096 free, is full > [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, > reserved=45056, may_use=6438354944, readonly=65536 > Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Josef diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 1cf810a..ac415cf7 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4471,7 +4471,12 @@ static void update_global_block_rsv(struct btrfs_fs_info *fs_info) spin_lock(&sinfo->lock); spin_lock(&block_rsv->lock); - block_rsv->size = num_bytes; + /* + * Limit the global block rsv to 512mb, we have infrastructure in place + * to throttle reservations if we start getting low on global block rsv + * space. + */ + block_rsv->size = min_t(u64, num_bytes, 512 * 1024 * 1024); num_bytes = sinfo->bytes_used + sinfo->bytes_pinned + sinfo->bytes_reserved + sinfo->bytes_readonly +