From patchwork Thu May 30 00:49:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Rosenberg X-Patchwork-Id: 10968021 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41A5A15E6 for ; Thu, 30 May 2019 00:53:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3335727F60 for ; Thu, 30 May 2019 00:53:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 27D7028A1D; Thu, 30 May 2019 00:53:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B6AC428632 for ; Thu, 30 May 2019 00:53:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727202AbfE3AxU (ORCPT ); Wed, 29 May 2019 20:53:20 -0400 Received: from mail-pf1-f201.google.com ([209.85.210.201]:33631 "EHLO mail-pf1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726859AbfE3AxU (ORCPT ); Wed, 29 May 2019 20:53:20 -0400 Received: by mail-pf1-f201.google.com with SMTP id f1so3285013pfb.0 for ; Wed, 29 May 2019 17:53:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=1zLLzunRacIsvbkfTaWpC4CoBmiOmCh3qurXkU7Ik8U=; b=h8FHABKxdlBN53cFTYJ5d0UiBL7PihQINuldUv4FTl0bNIxkuNBt/0Vhf2uJoL5oHc ntnxQeBd3+C8qEPYr5NVptJ+q35h8Vb6fgT/+ZS105INBTY18oRGfX0xwxo1L6VPNRA5 c22OcQrAt10dKzRjCahiaTgdDbZ/vBceEd0fjRMjFH/6+q0u9FQKKJf/x+fsZdfIY04A Vc4TLPxz0LbwxUj10798jCsJzwt35n5zYTHpIofjFuIp+AqXXRE63vKbMXk1C/H9oUB6 yHBPeUJEjTFKYiZzaX0gOwoHu4F0VQMZ+O+jfseDir1gYMx+VP4rwUkz9NYSSctfnSkb q/KQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=1zLLzunRacIsvbkfTaWpC4CoBmiOmCh3qurXkU7Ik8U=; b=tTY+XmignH0WTkiKjokVJNZrlZ7UsVfWyX9yYKty186yolfr3d4Gd58OuAj/Sj8mjM hRIpU04FlATnZRG0R2nUJ5yyNRpEtfWHJtYNmfN6DBD5HJ0xDxYismEWzONjrFp+U2et SUQQT736/RemSgFjy8RvulaOxxd4ycu0TBiA1P04Vj4c3oo2fuYFqR8U+p8cX5aZ4jCF WdREWKkqduX1pM+Nt0HazZueMglBXfTgHbdKo59w26KYk8Vxdm+h/XYn8sI5SzEKizLK SNbIcY+0EQDdAwAvSh9MIptJPEqzfusK9bDSclY1X/BNLuz8yNH2kV9egbcDL+LBzFTT kd4A== X-Gm-Message-State: APjAAAX+gCk6CIruIfK7k7ZOrqFq6BkmQI2cA2End4OEDnIkkNvGErOs 7R2LFP3+I0TmCwCakMqzXYOV1Owa4uY= X-Google-Smtp-Source: APXvYqyldr1oA4bszRUF74gO+ABbvOdB1x8BeqSdaTU/ourXVmDB1Vf2L9oUISnK4Rb3ipD8ZuMVYmiiBMg= X-Received: by 2002:a63:1622:: with SMTP id w34mr958542pgl.45.1559177599505; Wed, 29 May 2019 17:53:19 -0700 (PDT) Date: Wed, 29 May 2019 17:49:03 -0700 In-Reply-To: <20190530004906.261170-1-drosen@google.com> Message-Id: <20190530004906.261170-2-drosen@google.com> Mime-Version: 1.0 References: <20190530004906.261170-1-drosen@google.com> X-Mailer: git-send-email 2.22.0.rc1.257.g3120a18244-goog Subject: [PATCH v3 1/4] f2fs: Lower threshold for disable_cp_again From: Daniel Rosenberg To: Jaegeuk Kim , Chao Yu , Jonathan Corbet , linux-f2fs-devel@lists.sourceforge.net Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@android.com, Daniel Rosenberg Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The existing threshold for allowable holes at checkpoint=disable time is too high. The OVP space contains reserved segments, which are always in the form of free segments. These must be subtracted from the OVP value. The current threshold is meant to be the maximum value of holes of a single type we can have and still guarantee that we can fill the disk without failing to find space for a block of a given type. If the disk is full, ignoring current reserved, which only helps us, the amount of unused blocks is equal to the OVP area. Of that, there are reserved segments, which must be free segments, and the rest of the ovp area, which can come from either free segments or holes. The maximum possible amount of holes is OVP-reserved. Now, consider the disk when mounting with checkpoint=disable. We must be able to fill all available free space with either data or node blocks. When we start with checkpoint=disable, holes are locked to their current type. Say we have H of one type of hole, and H+X of the other. We can fill H of that space with arbitrary typed blocks via SSR. For the remaining H+X blocks, we may not have any of a given block type left at all. For instance, if we were to fill the disk entirely with blocks of the type with fewer holes, the H+X blocks of the opposite type would not be used. If H+X > OVP-reserved, there would be more holes than could possibly exist, and we would have failed to find a suitable block earlier on, leading to a crash in update_sit_entry. If H+X <= OVP-reserved, then the holes end up effectively masked by the OVP region in this case. Signed-off-by: Daniel Rosenberg Reviewed-by: Chao Yu --- fs/f2fs/segment.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 1a83115284b93..ec59cbd0e661d 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -876,7 +876,9 @@ void f2fs_dirty_to_prefree(struct f2fs_sb_info *sbi) int f2fs_disable_cp_again(struct f2fs_sb_info *sbi) { struct dirty_seglist_info *dirty_i = DIRTY_I(sbi); - block_t ovp = overprovision_segments(sbi) << sbi->log_blocks_per_seg; + int ovp_hole_segs = + (overprovision_segments(sbi) - reserved_segments(sbi)); + block_t ovp_holes = ovp_hole_segs << sbi->log_blocks_per_seg; block_t holes[2] = {0, 0}; /* DATA and NODE */ struct seg_entry *se; unsigned int segno; @@ -891,10 +893,10 @@ int f2fs_disable_cp_again(struct f2fs_sb_info *sbi) } mutex_unlock(&dirty_i->seglist_lock); - if (holes[DATA] > ovp || holes[NODE] > ovp) + if (holes[DATA] > ovp_holes || holes[NODE] > ovp_holes) return -EAGAIN; if (is_sbi_flag_set(sbi, SBI_CP_DISABLED_QUICK) && - dirty_segments(sbi) > overprovision_segments(sbi)) + dirty_segments(sbi) > ovp_hole_segs) return -EAGAIN; return 0; } From patchwork Thu May 30 00:49:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Rosenberg X-Patchwork-Id: 10968019 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C78C715E6 for ; Thu, 30 May 2019 00:53:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8CA0285B8 for ; Thu, 30 May 2019 00:53:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AD42728A72; Thu, 30 May 2019 00:53:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 656BA285B8 for ; Thu, 30 May 2019 00:53:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727230AbfE3AxY (ORCPT ); Wed, 29 May 2019 20:53:24 -0400 Received: from mail-yb1-f202.google.com ([209.85.219.202]:42670 "EHLO mail-yb1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727221AbfE3AxX (ORCPT ); Wed, 29 May 2019 20:53:23 -0400 Received: by mail-yb1-f202.google.com with SMTP id 126so3369310ybw.9 for ; Wed, 29 May 2019 17:53:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=d6X4/k5B5WeQgyIvpn8Af86NqovFLF22qTA1Y2V4Fe8=; b=Ax2EXMsdAnoEHIrqzhP0Cv23VlEaDoSoO1OWqhvxtcuBXmnYYLbiiBAZQlVXUUtGgA i6w+G4d54N8mfV0D6KY3vURUd55d8xA1pwqYEpTzjK68EmxEOCRJVDhq648b4GXT1jGV 3YrMacqCMm6wtkWM9ach4dG7xfETTseNfVw9UM/pxIHth0Ji7tonp1P9tLbqoqqtynbi DiENSJNzIdEipIykCMGLgC1Nz8YKxon3Gno9T6HkP2TBj6bY9GFxNm0etPi64anLqTnl aFpqLIUEQL5Pswsuq9dgOrW+1PKzioUiwtMYbgzR/a96OB4IOOCPkunrKKKeoQnMXWxu 4Y7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=d6X4/k5B5WeQgyIvpn8Af86NqovFLF22qTA1Y2V4Fe8=; b=axBNnbLIEUR2rK75bzv433TaWhcLUhohoWRo1BONgkAlgA1qg3WDjQ3dy+BOHlfmAB YgAFKXQ+1a/8VOhyN0xO9YNX9u5F+pAeyopCeRBoqBMY7k82q2jZO7Y0I/+6TlmEJvR2 9wbngMxWTr2x6OAhGGQLwYlk1aKSHc7BuqVrMaNXsTJdpjmF/sqh1DLIKFZwYQF/EJPN 3dCN8OTfENoaTXNblUV3IOWfOMnFWLhf8DUIP2lEqnZgSdvh3/QKLPIOsveM3e4d7pms p0UC1ZJpOK1M7yzd2HWeRiLbmqEV8lwFiYbUYbN7YtvSHT5KuDCmsjvLAZx2HgPUwetW MySQ== X-Gm-Message-State: APjAAAWKUDIYWa0kZX05g28aufrZb/PeNG8T55nyKQcjJDIcgzMFhZxc 0ZUP11NNNSPdCPzYOivMLUvn6IkMMiQ= X-Google-Smtp-Source: APXvYqzY+oI/v3047+rLfAdqlDZ9Z79kMZvQRPmWfGPVRdyfWVyQ3mLlfGRyfTAwrOQom+6CDZKVd7elYDo= X-Received: by 2002:a81:300c:: with SMTP id w12mr555304yww.57.1559177603199; Wed, 29 May 2019 17:53:23 -0700 (PDT) Date: Wed, 29 May 2019 17:49:04 -0700 In-Reply-To: <20190530004906.261170-1-drosen@google.com> Message-Id: <20190530004906.261170-3-drosen@google.com> Mime-Version: 1.0 References: <20190530004906.261170-1-drosen@google.com> X-Mailer: git-send-email 2.22.0.rc1.257.g3120a18244-goog Subject: [PATCH v3 2/4] f2fs: Fix root reserved on remount From: Daniel Rosenberg To: Jaegeuk Kim , Chao Yu , Jonathan Corbet , linux-f2fs-devel@lists.sourceforge.net Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@android.com, Daniel Rosenberg Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On a remount, you can currently set root reserved if it was not previously set. This can cause an underflow if reserved has been set to a very high value, since then root reserved + current reserved could be greater than user_block_count. inc_valid_block_count later subtracts out these values from user_block_count, causing an underflow. Signed-off-by: Daniel Rosenberg Reviewed-by: Chao Yu --- fs/f2fs/super.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index 912e2619d581b..359fd68509d16 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -213,7 +213,8 @@ void f2fs_msg(struct super_block *sb, const char *level, const char *fmt, ...) static inline void limit_reserve_root(struct f2fs_sb_info *sbi) { - block_t limit = (sbi->user_block_count << 1) / 1000; + block_t limit = min((sbi->user_block_count << 1) / 1000, + sbi->user_block_count - sbi->reserved_blocks); /* limit is 0.2% */ if (test_opt(sbi, RESERVE_ROOT) && From patchwork Thu May 30 00:49:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Rosenberg X-Patchwork-Id: 10968015 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E2F5A15E6 for ; Thu, 30 May 2019 00:53:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D459727F60 for ; Thu, 30 May 2019 00:53:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C554F287E9; Thu, 30 May 2019 00:53:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7629728A68 for ; Thu, 30 May 2019 00:53:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727248AbfE3Ax2 (ORCPT ); Wed, 29 May 2019 20:53:28 -0400 Received: from mail-qk1-f202.google.com ([209.85.222.202]:52400 "EHLO mail-qk1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727249AbfE3Ax1 (ORCPT ); Wed, 29 May 2019 20:53:27 -0400 Received: by mail-qk1-f202.google.com with SMTP id v80so3451711qkb.19 for ; Wed, 29 May 2019 17:53:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=pFDT61UYE6+362eUPJJuMi+49iBrHpoUZVksNkDJA2o=; b=tq0C+xLpxnScJrE7KGvLRNW1EvyW7w8fxZLYhMG38iqb6B65iu+pJNUwkSmSw9Wsl3 /9PNywbClVW9JA9DR/Dh1tSwsF6mMdgLrRvRMdWic+A+8I5sxRkeLEB1Ob2MAWWJm9jL 6ev53ORFgvokwK4cHk17Y7Ckh2RQmG74ba2tYgemlhSg7VkheKlADWDbsqwFIwOUSizD ZYBWe2VO+7gH+pPgHLcdOlhkOX4cS9lzTtfup83L3evOB0ATtVcM/T1mPq4hDcSUF5Uc u/Y86bsvfsu8KvdNmkA/2KSKxTBI0UsTGqIlC4MAGZr2YM/9PfAUtAnGcsXxF7iTVQkS YuCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=pFDT61UYE6+362eUPJJuMi+49iBrHpoUZVksNkDJA2o=; b=WSK2pxAhUc9yubZmyGFzsAkngy8stBhE7yA1KY/puiUD4HO0c5ZsLyxkZBmYaqyTAB jt2LKrGupmjBNskNTQ0byRmmousNMFIKfpW4DL5k+V1LYs+xCRDyMWOGQkUvdknf64cW zwl3CBbz5c3TL7boyOAXL6L2JZ1+M6BE2n0ehyeYXz1s5rcSUZaImk5QAMeGQAXOTYpJ vPHIn6/2LavnnIjF1DQ8dgtK6xamWZ1WwcT/uJ3/HVdSmc1SaGxeX8FjMKoTeW2VQkyE CAmMgoUzpOwzqi/GprMK60J7CmqnaUo6sE50Gw8pRfCCkFCT7SdFri8k9id0VMCudc1Z WYxQ== X-Gm-Message-State: APjAAAWn7gf6Y9Xa/gAGjxXB4mAgwuGwjvvaR1rbjpgXg6y494g5zdNp zaaz0+1fq2g2SFm8evjsx6qAWAReOjc= X-Google-Smtp-Source: APXvYqzdUz6c4pIj7GBI7bppXtBGbR717H7HlhysYDM55NbncVD1VFEeIv2JijSIJKEof8AzW57Fcs/wZys= X-Received: by 2002:a37:660b:: with SMTP id a11mr723548qkc.342.1559177606029; Wed, 29 May 2019 17:53:26 -0700 (PDT) Date: Wed, 29 May 2019 17:49:05 -0700 In-Reply-To: <20190530004906.261170-1-drosen@google.com> Message-Id: <20190530004906.261170-4-drosen@google.com> Mime-Version: 1.0 References: <20190530004906.261170-1-drosen@google.com> X-Mailer: git-send-email 2.22.0.rc1.257.g3120a18244-goog Subject: [PATCH v3 3/4] f2fs: Fix accounting for unusable blocks From: Daniel Rosenberg To: Jaegeuk Kim , Chao Yu , Jonathan Corbet , linux-f2fs-devel@lists.sourceforge.net Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@android.com, Daniel Rosenberg Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Fixes possible underflows when dealing with unusable blocks. Signed-off-by: Daniel Rosenberg Reviewed-by: Chao Yu --- fs/f2fs/f2fs.h | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index 9b3d9977cd1ef..a39cc4ffeb4b1 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -1769,8 +1769,12 @@ static inline int inc_valid_block_count(struct f2fs_sb_info *sbi, if (!__allow_reserved_blocks(sbi, inode, true)) avail_user_block_count -= F2FS_OPTION(sbi).root_reserved_blocks; - if (unlikely(is_sbi_flag_set(sbi, SBI_CP_DISABLED))) - avail_user_block_count -= sbi->unusable_block_count; + if (unlikely(is_sbi_flag_set(sbi, SBI_CP_DISABLED))) { + if (avail_user_block_count > sbi->unusable_block_count) + avail_user_block_count = 0; + else + avail_user_block_count -= sbi->unusable_block_count; + } if (unlikely(sbi->total_valid_block_count > avail_user_block_count)) { diff = sbi->total_valid_block_count - avail_user_block_count; if (diff > *count) @@ -1970,7 +1974,7 @@ static inline int inc_valid_node_count(struct f2fs_sb_info *sbi, struct inode *inode, bool is_inode) { block_t valid_block_count; - unsigned int valid_node_count; + unsigned int valid_node_count, user_block_count; int err; if (is_inode) { @@ -1997,10 +2001,11 @@ static inline int inc_valid_node_count(struct f2fs_sb_info *sbi, if (!__allow_reserved_blocks(sbi, inode, false)) valid_block_count += F2FS_OPTION(sbi).root_reserved_blocks; + user_block_count = sbi->user_block_count; if (unlikely(is_sbi_flag_set(sbi, SBI_CP_DISABLED))) - valid_block_count += sbi->unusable_block_count; + user_block_count -= sbi->unusable_block_count; - if (unlikely(valid_block_count > sbi->user_block_count)) { + if (unlikely(valid_block_count > user_block_count)) { spin_unlock(&sbi->stat_lock); goto enospc; } From patchwork Thu May 30 00:49:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Rosenberg X-Patchwork-Id: 10968017 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E1E1933 for ; Thu, 30 May 2019 00:53:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D97C27F60 for ; Thu, 30 May 2019 00:53:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2217928A1D; Thu, 30 May 2019 00:53:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1A693287E9 for ; Thu, 30 May 2019 00:53:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727275AbfE3Axd (ORCPT ); Wed, 29 May 2019 20:53:33 -0400 Received: from mail-pg1-f201.google.com ([209.85.215.201]:56385 "EHLO mail-pg1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727267AbfE3Ax3 (ORCPT ); Wed, 29 May 2019 20:53:29 -0400 Received: by mail-pg1-f201.google.com with SMTP id r191so1108013pgr.23 for ; Wed, 29 May 2019 17:53:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=b42+AR7n6zOPWsedWMtmn/sNL2bilZfayJ20H0mS03Q=; b=txWr4P1wbKm2FDUVQCS9xHw85QmuQ+d9AXAzLP9TAUEmv5bgOYmu4DxmOkRJDSgNz5 tqvAMIIVpapauzkILdHC+DcBc6Gp3EAX+JO29PzF//Y3YJRDc5qDxROcS7RB18Upu9R8 uaA8nck5xCmnmJS3jZop675bjEFxWy8rIPvqFBHyTbtwD6WZu2zdB/PVyYkjajxyj5T3 wvKJatIp0Sfe+NC5wiAeFmD+4utrqf5OmZ7StTbYOnz9OD7WcIlrzfRQP9heYdM8GPX2 mtggiIuV7VgZzsPV72KTS1zyQStuapjwy9sRuISjPcfSSpkWhX+mB1NrzGVJxmiamBs/ Ldhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=b42+AR7n6zOPWsedWMtmn/sNL2bilZfayJ20H0mS03Q=; b=esoprpDn99uGY1S3uhjvoVTTmsYtsWDBDo40RSNUGDDzLNPTNoAxjGrlWYfMjDn4ef NTWwb4CJHohg685OGZ+OL/6yrGSnerHm1I2rywiSZokE6b+YQkPefDSOFTu5x5n+ivX8 COj/5tVtY/X38TJRLQFAxcXxtxbIwHOkIoTR9nCZuBU+yvU5WFd/6CLJ2r7iL/bDwZ0q oVhxeL6cbVj7MDQQFyCpmjQH4ryq1TDWqwXAvOG/EvPSKycrTut6LEwQ5V/lQMDBg5iH M90Y1F0pAbK/Zi3WbLXsgP4ca2C1r08T76uqo2PXFSDMvRNCGM5v/kn4D/7OOwJUnuHP s1BA== X-Gm-Message-State: APjAAAWw7wuaWLI2iTZfHDSlcI7rMrdsEun/dZ4yRCPIPEHHbLYeq8fl 0wD401FmV5OGzXDD19aBV00MFdupaiQ= X-Google-Smtp-Source: APXvYqwq0u5fhrdHqmMQ0Jh9SDtVUwviQfew5VAVlFxx7HDPXhJGxaOzz8HP7Z/WXPlMifwcJjsHN58tHw8= X-Received: by 2002:a63:e645:: with SMTP id p5mr987859pgj.4.1559177608833; Wed, 29 May 2019 17:53:28 -0700 (PDT) Date: Wed, 29 May 2019 17:49:06 -0700 In-Reply-To: <20190530004906.261170-1-drosen@google.com> Message-Id: <20190530004906.261170-5-drosen@google.com> Mime-Version: 1.0 References: <20190530004906.261170-1-drosen@google.com> X-Mailer: git-send-email 2.22.0.rc1.257.g3120a18244-goog Subject: [PATCH v3 4/4] f2fs: Add option to limit required GC for checkpoint=disable From: Daniel Rosenberg To: Jaegeuk Kim , Chao Yu , Jonathan Corbet , linux-f2fs-devel@lists.sourceforge.net Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@android.com, Daniel Rosenberg Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This extends the checkpoint option to allow checkpoint=disable:%u[%] This allows you to specify what how much of the disk you are willing to lose access to while mounting with checkpoint=disable. If the amount lost would be higher, the mount will return -EAGAIN. This can be given as a percent of total space, or in blocks. Currently, we need to run garbage collection until the amount of holes is smaller than the OVP space. With the new option, f2fs can mark space as unusable up front instead of requiring garbage collection until the number of holes is small enough. Signed-off-by: Daniel Rosenberg Reviewed-by: Chao Yu --- Documentation/ABI/testing/sysfs-fs-f2fs | 8 ++++ Documentation/filesystems/f2fs.txt | 19 +++++++- fs/f2fs/f2fs.h | 6 ++- fs/f2fs/segment.c | 17 +++++-- fs/f2fs/super.c | 59 ++++++++++++++++--------- fs/f2fs/sysfs.c | 16 +++++++ 6 files changed, 99 insertions(+), 26 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-fs-f2fs b/Documentation/ABI/testing/sysfs-fs-f2fs index 91822ce258317..dca326e0ee3e1 100644 --- a/Documentation/ABI/testing/sysfs-fs-f2fs +++ b/Documentation/ABI/testing/sysfs-fs-f2fs @@ -243,3 +243,11 @@ Description: - Del: echo '[h/c]!extension' > /sys/fs/f2fs//extension_list - [h] means add/del hot file extension - [c] means add/del cold file extension + +What: /sys/fs/f2fs//unusable +Date April 2019 +Contact: "Daniel Rosenberg" +Description: + If checkpoint=disable, it displays the number of blocks that are unusable. + If checkpoint=enable it displays the enumber of blocks that would be unusable + if checkpoint=disable were to be set. diff --git a/Documentation/filesystems/f2fs.txt b/Documentation/filesystems/f2fs.txt index 66aca042988ee..bebd1be3ba495 100644 --- a/Documentation/filesystems/f2fs.txt +++ b/Documentation/filesystems/f2fs.txt @@ -214,11 +214,22 @@ fsync_mode=%s Control the policy of fsync. Currently supports "posix", non-atomic files likewise "nobarrier" mount option. test_dummy_encryption Enable dummy encryption, which provides a fake fscrypt context. The fake fscrypt context is used by xfstests. -checkpoint=%s Set to "disable" to turn off checkpointing. Set to "enable" +checkpoint=%s[:%u[%]] Set to "disable" to turn off checkpointing. Set to "enable" to reenable checkpointing. Is enabled by default. While disabled, any unmounting or unexpected shutdowns will cause the filesystem contents to appear as they did when the filesystem was mounted with that option. + While mounting with checkpoint=disabled, the filesystem must + run garbage collection to ensure that all available space can + be used. If this takes too much time, the mount may return + EAGAIN. You may optionally add a value to indicate how much + of the disk you would be willing to temporarily give up to + avoid additional garbage collection. This can be given as a + number of blocks, or as a percent. For instance, mounting + with checkpoint=disable:100% would always succeed, but it may + hide up to all remaining free space. The actual space that + would be unusable can be viewed at /sys/fs/f2fs//unusable + This space is reclaimed once checkpoint=enable. ================================================================================ DEBUGFS ENTRIES @@ -396,6 +407,12 @@ Files in /sys/fs/f2fs/ current_reserved_blocks This shows # of blocks currently reserved. + unusable If checkpoint=disable, this shows the number of + blocks that are unusable. + If checkpoint=enable it shows the number of blocks + that would be unusable if checkpoint=disable were + to be set. + ================================================================================ USAGE ================================================================================ diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index a39cc4ffeb4b1..11ef20fedfe72 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -136,6 +136,9 @@ struct f2fs_mount_info { int alloc_mode; /* segment allocation policy */ int fsync_mode; /* fsync policy */ bool test_dummy_encryption; /* test dummy encryption */ + block_t unusable_cap; /* Amount of space allowed to be + * unusable when disabling checkpoint + */ }; #define F2FS_FEATURE_ENCRYPT 0x0001 @@ -3085,7 +3088,8 @@ bool f2fs_issue_discard_timeout(struct f2fs_sb_info *sbi); void f2fs_clear_prefree_segments(struct f2fs_sb_info *sbi, struct cp_control *cpc); void f2fs_dirty_to_prefree(struct f2fs_sb_info *sbi); -int f2fs_disable_cp_again(struct f2fs_sb_info *sbi); +block_t f2fs_get_unusable_blocks(struct f2fs_sb_info *sbi); +int f2fs_disable_cp_again(struct f2fs_sb_info *sbi, block_t unusable); void f2fs_release_discard_addrs(struct f2fs_sb_info *sbi); int f2fs_npages_for_summary_flush(struct f2fs_sb_info *sbi, bool for_ra); void f2fs_allocate_new_segments(struct f2fs_sb_info *sbi); diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index ec59cbd0e661d..33bf07222f99f 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -873,13 +873,14 @@ void f2fs_dirty_to_prefree(struct f2fs_sb_info *sbi) mutex_unlock(&dirty_i->seglist_lock); } -int f2fs_disable_cp_again(struct f2fs_sb_info *sbi) +block_t f2fs_get_unusable_blocks(struct f2fs_sb_info *sbi) { - struct dirty_seglist_info *dirty_i = DIRTY_I(sbi); int ovp_hole_segs = (overprovision_segments(sbi) - reserved_segments(sbi)); block_t ovp_holes = ovp_hole_segs << sbi->log_blocks_per_seg; + struct dirty_seglist_info *dirty_i = DIRTY_I(sbi); block_t holes[2] = {0, 0}; /* DATA and NODE */ + block_t unusable; struct seg_entry *se; unsigned int segno; @@ -893,7 +894,17 @@ int f2fs_disable_cp_again(struct f2fs_sb_info *sbi) } mutex_unlock(&dirty_i->seglist_lock); - if (holes[DATA] > ovp_holes || holes[NODE] > ovp_holes) + unusable = holes[DATA] > holes[NODE] ? holes[DATA] : holes[NODE]; + if (unusable > ovp_holes) + return unusable - ovp_holes; + return 0; +} + +int f2fs_disable_cp_again(struct f2fs_sb_info *sbi, block_t unusable) +{ + int ovp_hole_segs = + (overprovision_segments(sbi) - reserved_segments(sbi)); + if (unusable > F2FS_OPTION(sbi).unusable_cap) return -EAGAIN; if (is_sbi_flag_set(sbi, SBI_CP_DISABLED_QUICK) && dirty_segments(sbi) > ovp_hole_segs) diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index 359fd68509d16..7d64e97611141 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -136,7 +136,10 @@ enum { Opt_alloc, Opt_fsync, Opt_test_dummy_encryption, - Opt_checkpoint, + Opt_checkpoint_disable, + Opt_checkpoint_disable_cap, + Opt_checkpoint_disable_cap_perc, + Opt_checkpoint_enable, Opt_err, }; @@ -195,7 +198,10 @@ static match_table_t f2fs_tokens = { {Opt_alloc, "alloc_mode=%s"}, {Opt_fsync, "fsync_mode=%s"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, - {Opt_checkpoint, "checkpoint=%s"}, + {Opt_checkpoint_disable, "checkpoint=disable"}, + {Opt_checkpoint_disable_cap, "checkpoint=disable:%u"}, + {Opt_checkpoint_disable_cap_perc, "checkpoint=disable:%u%%"}, + {Opt_checkpoint_enable, "checkpoint=enable"}, {Opt_err, NULL}, }; @@ -772,22 +778,30 @@ static int parse_options(struct super_block *sb, char *options) "Test dummy encryption mount option ignored"); #endif break; - case Opt_checkpoint: - name = match_strdup(&args[0]); - if (!name) - return -ENOMEM; - - if (strlen(name) == 6 && - !strncmp(name, "enable", 6)) { - clear_opt(sbi, DISABLE_CHECKPOINT); - } else if (strlen(name) == 7 && - !strncmp(name, "disable", 7)) { - set_opt(sbi, DISABLE_CHECKPOINT); - } else { - kvfree(name); + case Opt_checkpoint_disable_cap_perc: + if (args->from && match_int(args, &arg)) return -EINVAL; - } - kvfree(name); + if (arg < 0 || arg > 100) + return -EINVAL; + if (arg == 100) + F2FS_OPTION(sbi).unusable_cap = + sbi->user_block_count; + else + F2FS_OPTION(sbi).unusable_cap = + (sbi->user_block_count / 100) * arg; + set_opt(sbi, DISABLE_CHECKPOINT); + break; + case Opt_checkpoint_disable_cap: + if (args->from && match_int(args, &arg)) + return -EINVAL; + F2FS_OPTION(sbi).unusable_cap = arg; + set_opt(sbi, DISABLE_CHECKPOINT); + break; + case Opt_checkpoint_disable: + set_opt(sbi, DISABLE_CHECKPOINT); + break; + case Opt_checkpoint_enable: + clear_opt(sbi, DISABLE_CHECKPOINT); break; default: f2fs_msg(sb, KERN_ERR, @@ -1410,8 +1424,8 @@ static int f2fs_show_options(struct seq_file *seq, struct dentry *root) seq_printf(seq, ",alloc_mode=%s", "reuse"); if (test_opt(sbi, DISABLE_CHECKPOINT)) - seq_puts(seq, ",checkpoint=disable"); - + seq_printf(seq, ",checkpoint=disable:%u", + F2FS_OPTION(sbi).unusable_cap); if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_POSIX) seq_printf(seq, ",fsync_mode=%s", "posix"); else if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT) @@ -1440,6 +1454,7 @@ static void default_options(struct f2fs_sb_info *sbi) set_opt(sbi, EXTENT_CACHE); set_opt(sbi, NOHEAP); clear_opt(sbi, DISABLE_CHECKPOINT); + F2FS_OPTION(sbi).unusable_cap = 0; sbi->sb->s_flags |= SB_LAZYTIME; set_opt(sbi, FLUSH_MERGE); set_opt(sbi, DISCARD); @@ -1468,6 +1483,7 @@ static int f2fs_disable_checkpoint(struct f2fs_sb_info *sbi) struct cp_control cpc; int err = 0; int ret; + block_t unusable; if (s_flags & SB_RDONLY) { f2fs_msg(sbi->sb, KERN_ERR, @@ -1495,7 +1511,8 @@ static int f2fs_disable_checkpoint(struct f2fs_sb_info *sbi) goto restore_flag; } - if (f2fs_disable_cp_again(sbi)) { + unusable = f2fs_get_unusable_blocks(sbi); + if (f2fs_disable_cp_again(sbi, unusable)) { err = -EAGAIN; goto restore_flag; } @@ -1508,7 +1525,7 @@ static int f2fs_disable_checkpoint(struct f2fs_sb_info *sbi) goto out_unlock; spin_lock(&sbi->stat_lock); - sbi->unusable_block_count = 0; + sbi->unusable_block_count = unusable; spin_unlock(&sbi->stat_lock); out_unlock: diff --git a/fs/f2fs/sysfs.c b/fs/f2fs/sysfs.c index 729f46a3c9ee0..fa184880cff34 100644 --- a/fs/f2fs/sysfs.c +++ b/fs/f2fs/sysfs.c @@ -68,6 +68,20 @@ static ssize_t dirty_segments_show(struct f2fs_attr *a, (unsigned long long)(dirty_segments(sbi))); } +static ssize_t unusable_show(struct f2fs_attr *a, + struct f2fs_sb_info *sbi, char *buf) +{ + block_t unusable; + + if (test_opt(sbi, DISABLE_CHECKPOINT)) + unusable = sbi->unusable_block_count; + else + unusable = f2fs_get_unusable_blocks(sbi); + return snprintf(buf, PAGE_SIZE, "%llu\n", + (unsigned long long)unusable); +} + + static ssize_t lifetime_write_kbytes_show(struct f2fs_attr *a, struct f2fs_sb_info *sbi, char *buf) { @@ -440,6 +454,7 @@ F2FS_GENERAL_RO_ATTR(dirty_segments); F2FS_GENERAL_RO_ATTR(lifetime_write_kbytes); F2FS_GENERAL_RO_ATTR(features); F2FS_GENERAL_RO_ATTR(current_reserved_blocks); +F2FS_GENERAL_RO_ATTR(unusable); #ifdef CONFIG_FS_ENCRYPTION F2FS_FEATURE_RO_ATTR(encryption, FEAT_CRYPTO); @@ -495,6 +510,7 @@ static struct attribute *f2fs_attrs[] = { ATTR_LIST(inject_type), #endif ATTR_LIST(dirty_segments), + ATTR_LIST(unusable), ATTR_LIST(lifetime_write_kbytes), ATTR_LIST(features), ATTR_LIST(reserved_blocks),