Recent issues with btrfs fi df

Commit Message

Austin S. Hemmelgarn Dec. 5, 2014, 12:19 p.m. UTC

On 2014-12-05 02:42, Satoru Takeuchi wrote:
> Hi Austin,
>
> (2014/12/04 23:31), Austin S Hemmelgarn wrote:
>> I've recently noticed on some of my systems, that btrfs fi df
>> doesn't consistently show all of the chunk types.
>> I'll occasionally not see the GlobalReserve, or even anything
>> but System,
>
> For one Btrfs file system, "how inconsistent" is the same even if
> time passes? In other word,
>
>   a) Once "GlobalReserve" becomes to be not shown, it keep as is
>      after tha, or
>   b) Oneday "GlobalReserver" disappeared. Howevert it appear again
>      at the next day or so.
>
In general, once it changes the first time, things don't seem to change
again afterwards.
>> although the behavior seems to be consistent for
>> a given filesystem.
>
> Did you confirm the following things for your Btrfs file system?
>
>   a) btrfs scrub finishes without any problem, and
>   b) dmesg doesn't show any suspicious message.
>
Scrub and dmesg both look fine, I've also run btrfsck in no-op mode and
that doesn't report any errors either.
>> I'm using btrfs-progs 3.17.1 and kernel 3.17.4
>> with grsecurity patches (although with much of the grsec stuff disabled)
>> on all such systems.  I'd be happy to provide kernel .config or other
>> information for debugging on request.
>
> Could you tell me the following information, if possible?
>
>   - mkfs options and mount options
In both cases that I currently have access to, I created the fs with:
mkfs.btrfs -O extref,skinny-metadata,no-holes -F <device-name>
and the mount option strings for the devices in question are:
noatime,space_cache,ssd,autodefrag
for /
and:
noatime,sync,nosuid,nodev,noexec,compress=zlib,ssd,space_cache,autodefrag for
/boot
>   - The output of btrfs fi df
For /, I get:
  Data, single: total=43.00GiB, used=40.76GiB
  System, DUP: total=32.00MiB, used=16.00KiB
  Metadata, DUP: total=1.50GiB, used=1.05GiB
For /boot, I get:
  System, single: total=32.00MiB, used=4.00KiB
>   - .config
I've attached a gzipped copy.
>   - Any possible trigger to cause this problem
There aren't any that I know of.
>   - Btrfs specific operations, for example weekly btrfs scrub
I run scrub weekly, and balance and fstrim as needed.
>   - Do you have any system which works fine and uses a kernel
>     without grsecurity patches?
Yes, although said system has exclusively multi-device filesystems,
while the affected one is all single device filesystems.
>
> In addition, running one of your system with
>
>   - upstream kernel without grsecurity, and
>   - btrfs file system with which btrfs fi df works correctly,
>
I've got a recovery environment built using buildroot that is based on
the same kernel version without grsec patches, I'll reboot into that and
see what it says.
> can help to distinguish whether the problem comes from
> upstream kernel (of course it includes btrfs) or grsecurity.
> I'm not sure about grsecurity. however, I have encountered
> many problems caused by security modules.
I do have one other local kernel patch that I use, I've attached that as
well, although it should have no effect whatsoever on the fs code.

Comments

Austin S. Hemmelgarn Dec. 5, 2014, 12:35 p.m. UTC | #1

On 2014-12-05 07:19, Austin S Hemmelgarn wrote:
> On 2014-12-05 02:42, Satoru Takeuchi wrote:
>> Hi Austin,
>>
>> (2014/12/04 23:31), Austin S Hemmelgarn wrote:
>>> I've recently noticed on some of my systems, that btrfs fi df
>>> doesn't consistently show all of the chunk types.
>>> I'll occasionally not see the GlobalReserve, or even anything
>>> but System,
>>
>> For one Btrfs file system, "how inconsistent" is the same even if
>> time passes? In other word,
>>
>>    a) Once "GlobalReserve" becomes to be not shown, it keep as is
>>       after tha, or
>>    b) Oneday "GlobalReserver" disappeared. Howevert it appear again
>>       at the next day or so.
>>
> In general, once it changes the first time, things don't seem to change
> again afterwards.
>>> although the behavior seems to be consistent for
>>> a given filesystem.
>>
>> Did you confirm the following things for your Btrfs file system?
>>
>>    a) btrfs scrub finishes without any problem, and
>>    b) dmesg doesn't show any suspicious message.
>>
> Scrub and dmesg both look fine, I've also run btrfsck in no-op mode and
> that doesn't report any errors either.
>>> I'm using btrfs-progs 3.17.1 and kernel 3.17.4
>>> with grsecurity patches (although with much of the grsec stuff disabled)
>>> on all such systems.  I'd be happy to provide kernel .config or other
>>> information for debugging on request.
>>
>> Could you tell me the following information, if possible?
>>
>>    - mkfs options and mount options
> In both cases that I currently have access to, I created the fs with:
> mkfs.btrfs -O extref,skinny-metadata,no-holes -F <device-name>
> and the mount option strings for the devices in question are:
> noatime,space_cache,ssd,autodefrag
> for /
> and:
> noatime,sync,nosuid,nodev,noexec,compress=zlib,ssd,space_cache,autodefrag for
> /boot
>>    - The output of btrfs fi df
> For /, I get:
>    Data, single: total=43.00GiB, used=40.76GiB
>    System, DUP: total=32.00MiB, used=16.00KiB
>    Metadata, DUP: total=1.50GiB, used=1.05GiB
> For /boot, I get:
>    System, single: total=32.00MiB, used=4.00KiB
>>    - .config
> I've attached a gzipped copy.
>>    - Any possible trigger to cause this problem
> There aren't any that I know of.
>>    - Btrfs specific operations, for example weekly btrfs scrub
> I run scrub weekly, and balance and fstrim as needed.
>>    - Do you have any system which works fine and uses a kernel
>>      without grsecurity patches?
> Yes, although said system has exclusively multi-device filesystems,
> while the affected one is all single device filesystems.
>>
>> In addition, running one of your system with
>>
>>    - upstream kernel without grsecurity, and
>>    - btrfs file system with which btrfs fi df works correctly,
>>
> I've got a recovery environment built using buildroot that is based on
> the same kernel version without grsec patches, I'll reboot into that and
> see what it says.
OK, so it definitely appears to be a kernel issue, as btrfs fi df 
reports everything correctly when used from the recovery environment, 
both with the copy of btrfs-progs in the recovery environment, and the 
copy from the root filesystem of the affected system.  I'm going to try 
to bisect down to what option in my kernel config is actually causing 
this, although it may be next week before I can actually do so.
>> can help to distinguish whether the problem comes from
>> upstream kernel (of course it includes btrfs) or grsecurity.
>> I'm not sure about grsecurity. however, I have encountered
>> many problems caused by security modules.
> I do have one other local kernel patch that I use, I've attached that as
> well, although it should have no effect whatsoever on the fs code.
>
>

Patch

diff -rU3 linux-3.16.3-gentoo/include/uapi/linux/vt.h /usr/src/linux-3.16.3-gentoo/include/uapi/linux/vt.h
--- linux-3.16.3-gentoo/include/uapi/linux/vt.h	2014-10-01 14:20:45.855383160 -0400
+++ /usr/src/linux-3.16.3-gentoo/include/uapi/linux/vt.h	2014-08-03 18:25:02.000000000 -0400
@@ -7,8 +7,8 @@ 
  * resizing).
  */
 #define MIN_NR_CONSOLES 1       /* must be at least 1 */
-#define MAX_NR_CONSOLES	63	/* serial lines start at 64 */
-#define MAX_NR_USER_CONSOLES 63	/* must be root to allocate above this */
+#define MAX_NR_CONSOLES	15	/* serial lines start at 64 */
+#define MAX_NR_USER_CONSOLES 15	/* must be root to allocate above this */
 		/* Note: the ioctl VT_GETSTATE does not work for
 		   consoles 16 and higher (since it returns a short) */