From patchwork Mon Apr 20 17:07:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11499473 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A0BCD14B4 for ; Mon, 20 Apr 2020 17:09:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 80453208E4 for ; Mon, 20 Apr 2020 17:09:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402540; bh=tPDZwM/4wQHICe/qYNofhfEPheexviXoxm4aIF+HkMM=; h=From:To:Cc:Subject:Date:List-ID:From; b=z6PaSM/OzhQgVmJ0/GxH5y8f1eSdR1pzKEOh/e6cw1fZiT1tK39f69WksnM3h6XzM Q54VyGpEUBB+KHVnGzzENiSUxeB/We0a/Ozj3S3nsvUVWafg75PZwDIwBcDPlpO/QJ NgFf1jFDSC/MDRQdHuhZGBxcMuA6nSvGaubIC/gc= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726501AbgDTRJA (ORCPT ); Mon, 20 Apr 2020 13:09:00 -0400 Received: from mail.kernel.org ([198.145.29.99]:42654 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725784AbgDTRI7 (ORCPT ); Mon, 20 Apr 2020 13:08:59 -0400 Received: from debian7.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DA258206D9; Mon, 20 Apr 2020 17:08:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402539; bh=tPDZwM/4wQHICe/qYNofhfEPheexviXoxm4aIF+HkMM=; h=From:To:Cc:Subject:Date:From; b=ZodWwYFXm/PO6kL7uIaDEoy5WDPQfGh/BuqL/cXnoHzPt7N+Sb3IGKy9HHogrOu6P glYU41MYxa7VfXOvbFskposhG9HEtLvgw+fT0+yRGPtXq2OHiw8kCPNjVqcaN//svc AmREQUqe4TU5Ri1ZvaRbttxMWA+fMyqkMeJ/pFp8= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 1/4] fsx: allow zero range operations to cross eof Date: Mon, 20 Apr 2020 18:07:38 +0100 Message-Id: <20200420170738.9879-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Filipe Manana Currently we are limiting the range for zero range operations to stay within the i_size boundary. This is not optimal because like this we lose coverage of the filesystem's zero range implementation, since zero range operations are allowed to cross the i_size. Fix this by limiting the range to 'maxfilelen' and not 'file_size', and update the 'file_size' after each zero range operation if needed. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 9d598a4f..56479eda 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -1244,6 +1244,17 @@ do_zero_range(unsigned offset, unsigned length, int keep_size) } memset(good_buf + offset, '\0', length); + + if (!keep_size && end_offset > file_size) { + /* + * If there's a gap between the old file size and the offset of + * the zero range operation, fill the gap with zeroes. + */ + if (offset > file_size) + memset(good_buf + file_size, '\0', offset - file_size); + + file_size = end_offset; + } } #else @@ -2141,7 +2152,7 @@ have_op: do_punch_hole(offset, size); break; case OP_ZERO_RANGE: - TRIM_OFF_LEN(offset, size, file_size); + TRIM_OFF_LEN(offset, size, maxfilelen); do_zero_range(offset, size, keep_size); break; case OP_COLLAPSE_RANGE: From patchwork Mon Apr 20 17:09:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11499477 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AEF9F14B4 for ; Mon, 20 Apr 2020 17:09:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8DB3722209 for ; Mon, 20 Apr 2020 17:09:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402550; bh=/09lFz86nyjgOxhAxgGqNyvCbXjNDXalZtzZZTnqodE=; h=From:To:Cc:Subject:Date:List-ID:From; b=ltudXk429BSv6xQ2DbRHPNQZCGPswAPNRlW3/gDfnZbLn+qXjAF4AKAldflOdws3q g+S3Cpv7WntYmI7IuxIgW+v7b2TTp9qpGrMKTMWmhx2pQevU9ltA5fTR9LiU/srDtt scplb5lBtPowCv2LuI0q25t8uKTZ0myom8zznnWM= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726482AbgDTRJK (ORCPT ); Mon, 20 Apr 2020 13:09:10 -0400 Received: from mail.kernel.org ([198.145.29.99]:42710 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725784AbgDTRJK (ORCPT ); Mon, 20 Apr 2020 13:09:10 -0400 Received: from debian7.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BBBF4206D9; Mon, 20 Apr 2020 17:09:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402549; bh=/09lFz86nyjgOxhAxgGqNyvCbXjNDXalZtzZZTnqodE=; h=From:To:Cc:Subject:Date:From; b=oIvhkGvh8dNq1KRn0wMOb5dpf7Heg+flEnRxEb+xiCTGqgBib7EsCIdC/D7hY14WW FB0tIf5KtSYrlkU6eK6r2EumZ6YwynScbH+SkU9GThHXcaqlEliDKqNpZCTzRnT0KC uusGL6hlPGtrMCjLEZGygRZwJORLaMIR0gqK8mwI= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 2/4] fsx: fix infinite/too long loops when generating ranges for clone operations Date: Mon, 20 Apr 2020 18:09:05 +0100 Message-Id: <20200420170905.9944-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Filipe Manana While running generic/457 I've had fsx taking a lot of CPU time and not making any progress for over an hour. Attaching gdb to the fsx process revealed that fsx was in the loop that generates the ranges for a clone operation, in particular the loop seemed to never end because the range defined by 'offset2' kept overlapping with the range defined by 'offset'. So far this happened two times in one of my test VMs with generic/457. Fix this by breaking out of the loop after trying 30 times, like we currently do for dedupe operations, which results in logging the operation as skipped. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 56479eda..ab64b50a 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -2013,16 +2013,24 @@ test(void) keep_size = random() % 2; break; case OP_CLONE_RANGE: - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 = offset2 & ~(block_size - 1); + } while (range_overlaps(offset, offset2, size) || + offset2 + size > maxfilelen); + break; + } case OP_DEDUPE_RANGE: { int tries = 0; From patchwork Mon Apr 20 17:09:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11499481 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90E5A14B4 for ; Mon, 20 Apr 2020 17:09:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6E633208E4 for ; Mon, 20 Apr 2020 17:09:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402562; bh=1NURecO0mrbzqRJJhqwcnUklSVWjq4mXylGXv77qwzU=; h=From:To:Cc:Subject:Date:List-ID:From; b=NeaIxuGgtNKuVVgqaV6mlMbrZofan/AT7ZeFRQXArXOMll6DwRSLoLYIV7WH1y64J rBCiW3eJKZF3g5ZdIwkXjWoXFmbxk+0E5Ch+WmLRfPbM06fGfuHbRbbAEvICYtzNqU UMhsJYwg0d/b9S81rTAfWUf13g5RYLRdyf6gs/tU= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726532AbgDTRJW (ORCPT ); Mon, 20 Apr 2020 13:09:22 -0400 Received: from mail.kernel.org ([198.145.29.99]:42760 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726296AbgDTRJV (ORCPT ); Mon, 20 Apr 2020 13:09:21 -0400 Received: from debian7.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C0D49206D9; Mon, 20 Apr 2020 17:09:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402561; bh=1NURecO0mrbzqRJJhqwcnUklSVWjq4mXylGXv77qwzU=; h=From:To:Cc:Subject:Date:From; b=peRQsf5jD8aed7u1ZY4wNJK+QO4AgM4RRvq4VZgOP6ZyOV+jbl1zygQBSGAAqmxQ2 fuSl2M9t7MkNFHsUIW7tQYhU5oInyVJA26MDEHAf07q7W//7rjzCFUgfbyO4Ar4wtf xV5RRCht31X+atU411k4d7RV4cglJfDMA7nQLGMA= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 3/4] fsx: fix infinite/too long loops when generating ranges for copy_file_range Date: Mon, 20 Apr 2020 18:09:17 +0100 Message-Id: <20200420170917.9994-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Filipe Manana While running generic/521 I've had fsx taking a lot of CPU time and not making any progress for several hours. Attaching gdb to the fsx process revealed that fsx was in the loop that generates the ranges for a copy_file_range operation, in particular the loop seemed to never end because the range defined by 'offset2' kept overlapping with the range defined by 'offset'. So far this happened one time only in one of my test VMs with generic/521. Fix this by breaking out of the loop after trying 30 times, like we currently do for dedupe operations, which results in logging the operation as skipped. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index ab64b50a..40cbd401 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -2051,17 +2051,25 @@ test(void) break; } case OP_COPY_RANGE: - TRIM_OFF_LEN(offset, size, file_size); - offset -= offset % readbdy; - if (o_direct) - size -= size % readbdy; - do { - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 -= offset2 % writebdy; - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset -= offset % readbdy; + if (o_direct) + size -= size % readbdy; + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 -= offset2 % writebdy; + } while (range_overlaps(offset, offset2, size) || + offset2 + size > maxfilelen); + break; + } } have_op: From patchwork Mon Apr 20 17:09:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11499485 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF1781392 for ; Mon, 20 Apr 2020 17:09:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B521722209 for ; Mon, 20 Apr 2020 17:09:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402576; bh=WcE77tt/bEU+P5Fom4lh616Ry8OSojCe1UWaZIb3OsQ=; h=From:To:Cc:Subject:Date:List-ID:From; b=KWOYzi5bJR/Jg/W71eOHp0boPqiDUxhc9AMcQHL/b4U2COaTkZK3afmtzyfJ7wPDo mKUqFrBzcr0ZYr3tqm8uf3dqNdW2r6eziJEMvekov99H5JShUT07OgauLJF4+kB/39 iJtqEbzHCryyawednnarmZl5otk+h7Z/3l3hUvnU= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726596AbgDTRJg (ORCPT ); Mon, 20 Apr 2020 13:09:36 -0400 Received: from mail.kernel.org ([198.145.29.99]:42844 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726534AbgDTRJf (ORCPT ); Mon, 20 Apr 2020 13:09:35 -0400 Received: from debian7.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 67F9A206D9; Mon, 20 Apr 2020 17:09:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402575; bh=WcE77tt/bEU+P5Fom4lh616Ry8OSojCe1UWaZIb3OsQ=; h=From:To:Cc:Subject:Date:From; b=GjEBAfi4jMYQJJO6Dh2OGAVT7U4Iv7XsGCyjUF/VJg3UgkouxgxjPIzg3s9c/iXSr dm8x3vIZ7frGmHRQkp18iBNTF3p9vqQoUjnTINJ/vTG1dR/RKVo+BclEBnD9am9gdA FByEflh3OK3QA3+4i3aNPCxGsBwRIvY87DrNs094= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 4/4] fsx: move range generation logic into a common helper Date: Mon, 20 Apr 2020 18:09:31 +0100 Message-Id: <20200420170931.10047-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Filipe Manana We have very similar code that generates the destination range for clone, dedupe and copy_file_range operations, so avoid duplicating the code three times and move it into a helper function. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 94 ++++++++++++++++++++++++++------------------------------------- 1 file changed, 39 insertions(+), 55 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 40cbd401..7c76655a 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -1939,6 +1939,39 @@ range_overlaps( return llabs((unsigned long long)off1 - off0) < size; } +static void generate_dest_range(bool bdy_align, + unsigned long max_range_end, + unsigned long *src_offset, + unsigned long *size, + unsigned long *dst_offset) +{ + int tries = 0; + + TRIM_OFF_LEN(*src_offset, *size, file_size); + if (bdy_align) { + *src_offset -= *src_offset % readbdy; + if (o_direct) + *size -= *size % readbdy; + } else { + *src_offset = *src_offset & ~(block_size - 1); + *size = *size & ~(block_size - 1); + } + + do { + if (tries++ >= 30) { + *size = 0; + break; + } + *dst_offset = random(); + TRIM_OFF(*dst_offset, max_range_end); + if (bdy_align) + *dst_offset -= *dst_offset % writebdy; + else + *dst_offset = *dst_offset & ~(block_size - 1); + } while (range_overlaps(*src_offset, *dst_offset, *size) || + *dst_offset + *size > max_range_end); +} + int test(void) { @@ -2013,63 +2046,14 @@ test(void) keep_size = random() % 2; break; case OP_CLONE_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; - } + generate_dest_range(false, maxfilelen, &offset, &size, &offset2); + break; case OP_DEDUPE_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, file_size); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > file_size); - break; - } + generate_dest_range(false, file_size, &offset, &size, &offset2); + break; case OP_COPY_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset -= offset % readbdy; - if (o_direct) - size -= size % readbdy; - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 -= offset2 % writebdy; - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; - } + generate_dest_range(true, maxfilelen, &offset, &size, &offset2); + break; } have_op: