From patchwork Wed Apr 8 10:35:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11479735 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB703112C for ; Wed, 8 Apr 2020 10:35:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BC32B20771 for ; Wed, 8 Apr 2020 10:35:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586342158; bh=8QVO30BHDBcabGdLbwR5eUB3BhOHkhInXscyDaLbJgA=; h=From:To:Cc:Subject:Date:List-ID:From; b=lWvSs+bIdCLDDlL3L2Ce7e3lg1/OlbNYw0Fr4EymxBfHkGN8ZPSZexNqR6m3b46AX rW0EMin9yBnyMjnX74TSuz4KggYyh24ZbhMq0bv1g/XjC15YB+pwX9CRfbOxWs2OLi YdfFK8AqVAETlI8rYOtXw6u/UbKubFg8vlev1ZTY= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728200AbgDHKf6 (ORCPT ); Wed, 8 Apr 2020 06:35:58 -0400 Received: from mail.kernel.org ([198.145.29.99]:40280 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726436AbgDHKf6 (ORCPT ); Wed, 8 Apr 2020 06:35:58 -0400 Received: from debian6.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D135020753; Wed, 8 Apr 2020 10:35:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586342157; bh=8QVO30BHDBcabGdLbwR5eUB3BhOHkhInXscyDaLbJgA=; h=From:To:Cc:Subject:Date:From; b=Z2na5MJXqpI4DKs9VTBs1gY9P8tIJ35TdPmku/9D0zW8RIi/xuIh5+nuIpmQSQtTB 4xDYScXoZSRChS6N1NP+NveywC0MKxpo8s7mw/IGwG3TANRJDknxwVz22uzrNlRIVp 2pY0VoRRb91aNkWUxzraDkgNjzBI8uoAs4cr1ugY= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 1/4] fsx: add missing file size update on zero range operations Date: Wed, 8 Apr 2020 11:35:52 +0100 Message-Id: <20200408103552.11339-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana When a zero range operation increases the size of the test file we were not updating the global variable 'file_size' which tracks the current size of the test file. This variable is used to for example compute the offset for a source range of clone, dedupe and copy file range operations. So just fix it by updating the 'file_size' global variable whenever a zero range operation does not use the keep size flag and its range goes beyond the current file size. Signed-off-by: Filipe Manana --- ltp/fsx.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ltp/fsx.c b/ltp/fsx.c index 9d598a4f..fa383c94 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -1212,6 +1212,8 @@ do_zero_range(unsigned offset, unsigned length, int keep_size) } end_offset = keep_size ? 0 : offset + length; + if (!keep_size && end_offset > file_size) + file_size = end_offset; if (end_offset > biggest) { biggest = end_offset; From patchwork Wed Apr 8 10:36:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11479741 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF368112C for ; Wed, 8 Apr 2020 10:36:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C55802082F for ; Wed, 8 Apr 2020 10:36:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586342169; bh=lqnX6aqfXprmHrTHkjUwHROnv21mqZCBRr4TcuCabGc=; h=From:To:Cc:Subject:Date:List-ID:From; b=TqkkV5bo/8iu25FKHLP71EntklAKpYy1TmPATaSJp80lFV7bZp2pwO/k+7668mZ59 BjXHbEn5J6HOaLqz3YEGMq09g7ML2mEWh7ZPwInpDIage+99ZNDJeg9m2zCxMnKqVh yp+dvQ9QsW9EgapZgstJPFISXV0lZUiH+khNwQ3Y= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728080AbgDHKgJ (ORCPT ); Wed, 8 Apr 2020 06:36:09 -0400 Received: from mail.kernel.org ([198.145.29.99]:40506 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726436AbgDHKgJ (ORCPT ); Wed, 8 Apr 2020 06:36:09 -0400 Received: from debian6.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E2CA620753; Wed, 8 Apr 2020 10:36:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586342168; bh=lqnX6aqfXprmHrTHkjUwHROnv21mqZCBRr4TcuCabGc=; h=From:To:Cc:Subject:Date:From; b=dRbedabZEWbIMUldmdM4rbjA7f8mXcHAXQfKDQapr83gXDgwtlMNXyMUBu2nkZbyj NbdzA9IvMgFZvRilDt8ZJFZu07O2rv43blU3LrX0+h9Yton5x7mPYKGCTL9D2Jnxjm P8IV7WzrvtPsQwaHFCZXP6O1JvBiaVSsJES7pB/I= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 2/4] fsx: fix infinite/too long loops when generating ranges for clone operations Date: Wed, 8 Apr 2020 11:36:04 +0100 Message-Id: <20200408103604.11395-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana While running generic/457 I've had fsx taking a lot of CPU time and not making any progress for over an hour. Attaching gdb to the fsx process revealed that fsx was in the loop that generates the ranges for a clone operation, in particular the loop seemed to never end because the range defined by 'offset2' kept overlapping with the range defined by 'offset'. So far this happened two times in one of my test VMs with generic/457. Fix this by breaking out of the loop after trying 30 times, like we currently do for dedupe operations, which results in logging the operation as skipped. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index fa383c94..5949ebf0 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -2004,16 +2004,24 @@ test(void) keep_size = random() % 2; break; case OP_CLONE_RANGE: - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 = offset2 & ~(block_size - 1); + } while (range_overlaps(offset, offset2, size) || + offset2 + size > maxfilelen); + break; + } case OP_DEDUPE_RANGE: { int tries = 0; From patchwork Wed Apr 8 10:36:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11479745 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B1D2112C for ; Wed, 8 Apr 2020 10:36:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0218C20787 for ; Wed, 8 Apr 2020 10:36:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586342181; bh=6FSXzXkB2zdDHEGg/5jNKvB21ddsGbVwCdko3LS1A34=; h=From:To:Cc:Subject:Date:List-ID:From; b=Utcb0RBnWBNcwWmiJP2zSzPiDbF5QF7nBwNJIuU6xRL/uHljH/m2dGm3isJnCzxjX 21mLo+gbTVSHnrJCO/7MTJm6BWgOZTXSk6YuBLkyYICqSkaxc0T2FKAryGEdpKQa2e unf5/n7wjVVSZQE6RfDHan3Wox8KYhGkGC2u6jCM= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728211AbgDHKgU (ORCPT ); Wed, 8 Apr 2020 06:36:20 -0400 Received: from mail.kernel.org ([198.145.29.99]:40576 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728209AbgDHKgU (ORCPT ); Wed, 8 Apr 2020 06:36:20 -0400 Received: from debian6.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5716F20753; Wed, 8 Apr 2020 10:36:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586342180; bh=6FSXzXkB2zdDHEGg/5jNKvB21ddsGbVwCdko3LS1A34=; h=From:To:Cc:Subject:Date:From; b=vZgWRG2C38tZL9zRzxUKZ46RP6fzz8YnSDMAAQ9qwh9gQwUlTQT35s4ZrMAYdD0Jw THvzsojLY1Brrl31fVNBAVD6RryAJ9jAhm70NK2iYNpdy41ZzO2FysiQy6A0zg7jRf m847j5ItpWOUMin9HY/syad7GPwiS24mjJt0gTZE= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 3/4] fsx: fix infinite/too long loops when generating ranges for copy_file_range Date: Wed, 8 Apr 2020 11:36:16 +0100 Message-Id: <20200408103616.11458-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana While running generic/521 I've had fsx taking a lot of CPU time and not making any progress for several hours. Attaching gdb to the fsx process revealed that fsx was in the loop that generates the ranges for a copy_file_range operation, in particular the loop seemed to never end because the range defined by 'offset2' kept overlapping with the range defined by 'offset'. So far this happened one time only in one of my test VMs with generic/521. Fix this by breaking out of the loop after trying 30 times, like we currently do for dedupe operations, which results in logging the operation as skipped. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 5949ebf0..89a5f60e 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -2042,17 +2042,25 @@ test(void) break; } case OP_COPY_RANGE: - TRIM_OFF_LEN(offset, size, file_size); - offset -= offset % readbdy; - if (o_direct) - size -= size % readbdy; - do { - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 -= offset2 % writebdy; - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset -= offset % readbdy; + if (o_direct) + size -= size % readbdy; + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 -= offset2 % writebdy; + } while (range_overlaps(offset, offset2, size) || + offset2 + size > maxfilelen); + break; + } } have_op: From patchwork Wed Apr 8 10:36:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11479749 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24C4114B4 for ; Wed, 8 Apr 2020 10:36:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 05E8720787 for ; Wed, 8 Apr 2020 10:36:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586342192; bh=kJQLcrgzAwge5U1k+lUnVNgNoNltpoecal7YgmW1t+M=; h=From:To:Cc:Subject:Date:List-ID:From; b=G5n1Ard+LRU4DpWAxSHItMHHdcYBGDR4emQfJpuRbKCRWjTJ+ioSzAB9g7bpsSesO 9y3gb02i835wub6ob0LZRe/nPLo1Pa1p37eFyh4kWW8FbQ1TdVKqZ6mMHX/R2qdn7s eGEkWo9Jftd6QboDDdXvZBXA/S+n3MIc6mYUhC+I= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728215AbgDHKgb (ORCPT ); Wed, 8 Apr 2020 06:36:31 -0400 Received: from mail.kernel.org ([198.145.29.99]:40630 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728209AbgDHKgb (ORCPT ); Wed, 8 Apr 2020 06:36:31 -0400 Received: from debian6.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1C19A20753; Wed, 8 Apr 2020 10:36:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586342190; bh=kJQLcrgzAwge5U1k+lUnVNgNoNltpoecal7YgmW1t+M=; h=From:To:Cc:Subject:Date:From; b=Qr2GoGw1WcBdtGsrgGMCc7ru5Vl/Ss43LZftEw/3ruVEDoQIeN1KSF2tqelyi05lc SKJJrR6HQgSlg9BifvZYgePNDPo9ug5dCfHQKBMgPYqIaqtPsrVIDOzWlRYkvi+CTg GRpajTjTCs+obRkiH8ebrE5v7RPxbd+HFCRVgfK0= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 4/4] fsx: move range generation logic into a common helper Date: Wed, 8 Apr 2020 11:36:27 +0100 Message-Id: <20200408103627.11514-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana We have very similar code that generates the destination range for clone, dedupe and copy_file_range operations, so avoid duplicating the code three times and move it into a helper function. Signed-off-by: Filipe Manana --- ltp/fsx.c | 91 +++++++++++++++++++++++++-------------------------------------- 1 file changed, 36 insertions(+), 55 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 89a5f60e..8873cd01 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -1930,6 +1930,36 @@ range_overlaps( return llabs((unsigned long long)off1 - off0) < size; } +static void generate_dest_range(unsigned long op, + unsigned long max_range_end, + unsigned long *src_offset, + unsigned long *size, + unsigned long *dst_offset) +{ + int tries = 0; + + TRIM_OFF_LEN(*src_offset, *size, file_size); + if (op == OP_COPY_RANGE) { + *src_offset -= *src_offset % readbdy; + if (o_direct) + *size -= *size % readbdy; + } else { + *src_offset = *src_offset & ~(block_size - 1); + *size = *size & ~(block_size - 1); + } + + do { + if (tries++ >= 30) { + *size = 0; + break; + } + *dst_offset = random(); + TRIM_OFF(*dst_offset, max_range_end); + *dst_offset = *dst_offset & ~(block_size - 1); + } while (range_overlaps(*src_offset, *dst_offset, *size) || + *dst_offset + *size > max_range_end); +} + int test(void) { @@ -2004,63 +2034,14 @@ test(void) keep_size = random() % 2; break; case OP_CLONE_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; - } + generate_dest_range(op, maxfilelen, &offset, &size, &offset2); + break; case OP_DEDUPE_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, file_size); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > file_size); - break; - } + generate_dest_range(op, file_size, &offset, &size, &offset2); + break; case OP_COPY_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset -= offset % readbdy; - if (o_direct) - size -= size % readbdy; - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 -= offset2 % writebdy; - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; - } + generate_dest_range(op, maxfilelen, &offset, &size, &offset2); + break; } have_op: