From patchwork Fri Feb 1 16:21:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 10793203 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C20B6C2 for ; Fri, 1 Feb 2019 16:22:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3696E3254F for ; Fri, 1 Feb 2019 16:22:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2A58F325D4; Fri, 1 Feb 2019 16:22:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E697D325C8 for ; Fri, 1 Feb 2019 16:22:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728166AbfBAQWX (ORCPT ); Fri, 1 Feb 2019 11:22:23 -0500 Received: from mail-pl1-f176.google.com ([209.85.214.176]:36655 "EHLO mail-pl1-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726460AbfBAQWX (ORCPT ); Fri, 1 Feb 2019 11:22:23 -0500 Received: by mail-pl1-f176.google.com with SMTP id g9so3459042plo.3 for ; Fri, 01 Feb 2019 08:22:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zaGzBv+5VAU+O2Rk6mfJvWCTkVs6XhzmtPTIrumhYms=; b=eDhAPXQBRMVqOYNISwNPLTxuFp+hevvtztwtuXF9k7SqjjZ6OjObQCLASs9mngoTTb 9Ab+er6TwnIbt//0l9AUMphr9kR2TTSHXpIqz/23mlAnAGlIEm88P6tgYU8VNBWEQGsa 8sNETP+rPg64X8U7wSPC5ztP0xhpggMW3A5/Dr3M9XWEu04XBahUNxskxrcNhuv70XMx 4fqjq4ImDS3MW2vIMVnJlp3aPdXuT4Hjlp4YnL1+ZMf1FpeynqvKze8QwnFMDnL7cxkb WBU3TsoVd8qdb0ZkwvTA9DqVsHv9VziUyhLBMW8PoX3SyLQXTHuSHw/I1hbdVboUqWti jgig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zaGzBv+5VAU+O2Rk6mfJvWCTkVs6XhzmtPTIrumhYms=; b=mKQzTAJSmxMePYKKecgRqEgSOQk9cnyRGThHBrNMA0H/9uoPSeMB2dwHt0SF4atFT4 Ws5pLU1hCquIw7jF3uCTcc/h/SzWyBLYUwt/pgxykZ2zrKFRddPKurE/M88TrHBGUTUE z2MQdNgLizExk585l1MtBrYtadCfvd9FRycDAmNH4KBYo5uRbpRiryiSIchFja5aE6cO VRCr6+/oIzlMCe452Q0jdVFQwy9QGjEMWz7vP/09Y+6vOlJZAeTn0hIw/WRoDrDXNs+o WrHmmsSBnqpAoIEVXaofJAwZyAkEKOb6LVSyYv2woMbIPsIpKAqyM1MYPBtRz2TSZrX5 Q1bw== X-Gm-Message-State: AJcUukePrqZhPiZRiYWXvuqNSyatb1OsC0H13owuZqTo+O2EDxzdmL/T RuDBZaMmzO0xHsPXFLLXL7c= X-Google-Smtp-Source: ALg8bN4+ir5+YcOPNjl42EG5hW+aXPdtkivuwnE/5RZF57+ph1gYXwy3TNRwjN6vn6WRwvZ5aUmxdw== X-Received: by 2002:a17:902:bc3:: with SMTP id 61mr40453982plr.15.1549038142562; Fri, 01 Feb 2019 08:22:22 -0800 (PST) Received: from localhost.localdomain ([2408:84e1:1:ce70:7c5c:7512:51d2:8f91]) by smtp.gmail.com with ESMTPSA id x2sm11872694pfx.78.2019.02.01.08.22.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Feb 2019 08:22:21 -0800 (PST) From: Jiang Xin To: Junio C Hamano , Git List , Sun Chao Cc: Jiang Xin , Jiang Xin , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine Subject: [PATCH v9 1/6] t5323: test cases for git-pack-redundant Date: Sat, 2 Feb 2019 00:21:47 +0800 Message-Id: <20190201162152.31136-2-worldhello.net@gmail.com> X-Mailer: git-send-email 2.20.1.103.ged0fc2ca7b In-Reply-To: <20190130114736.30357-1-worldhello.net@gmail.com> References: <20190130114736.30357-1-worldhello.net@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jiang Xin Add test cases for git pack-redundant to validate new algorithm for git pack-redundant. Signed-off-by: Jiang Xin Reviewed-by: SZEDER Gábor Helped-by: Eric Sunshine Reviewed-by: Sun Chao Signed-off-by: Junio C Hamano --- t/t5323-pack-redundant.sh | 510 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 510 insertions(+) create mode 100755 t/t5323-pack-redundant.sh diff --git a/t/t5323-pack-redundant.sh b/t/t5323-pack-redundant.sh new file mode 100755 index 0000000000..d224ff3c50 --- /dev/null +++ b/t/t5323-pack-redundant.sh @@ -0,0 +1,510 @@ +#!/bin/sh +# +# Copyright (c) 2018 Jiang Xin +# + +test_description='Test git pack-redundant + +In order to test git-pack-redundant, we will create a number of objects and +packs in the repository `master.git`. The relationship between packs (P1-P8) +and objects (T, A-R) is showed in the following chart. Objects of a pack will +be marked with letter x, while objects of redundant packs will be marked with +exclamation point, and redundant pack itself will be marked with asterisk. + + | T A B C D E F G H I J K L M N O P Q R + ----+-------------------------------------- + P1 | x x x x x x x x + P2* | ! ! ! ! ! ! ! + P3 | x x x x x x + P4* | ! ! ! ! ! + P5 | x x x x + P6* | ! ! ! + P7 | x x + P8* | ! + ----+-------------------------------------- + ALL | x x x x x x x x x x x x x x x x x x x + +Another repository `shared.git` has unique objects (X-Z), while other objects +(marked with letter s) are shared through alt-odb (of `master.git`). The +relationship between packs and objects is as follows: + + | T A B C D E F G H I J K L M N O P Q R X Y Z + ----+---------------------------------------------- + Px1 | s s s x x x + Px2 | s s s x x x +' + +. ./test-lib.sh + +master_repo=master.git +shared_repo=shared.git + +# Note: DO NOT run it in a subshell, otherwise the variables will not be set +# Usage: create_commits_in A B C ... +create_commits_in () { + repo="$1" && + parent=$(git -C "$repo" rev-parse HEAD^{} 2>/dev/null) || parent= + T=$(git -C "$repo" write-tree) && + shift && + while test $# -gt 0 + do + name=$1 && + test_tick && + if test -z "$parent" + then + oid=$(echo $name | git -C "$repo" commit-tree $T) + else + oid=$(echo $name | git -C "$repo" commit-tree -p $parent $T) + fi && + eval $name=$oid && + parent=$oid && + shift || + return 1 + done + git -C "$repo" update-ref refs/heads/master $oid +} + +# Note: DO NOT run it in a subshell, otherwise the variables will not be set +create_pack_1 () { + P1=$(git -C "$master_repo/objects/pack" pack-objects -q pack <<-EOF + $T + $A + $B + $C + $D + $E + $F + $R + EOF + ) && + eval P$P1=P1:$P1 +} + +create_pack_2 () { + P2=$(git -C "$master_repo/objects/pack" pack-objects -q pack <<-EOF + $B + $C + $D + $E + $G + $H + $I + EOF + ) && + eval P$P2=P2:$P2 +} + +create_pack_3 () { + P3=$(git -C "$master_repo/objects/pack" pack-objects -q pack <<-EOF + $F + $I + $J + $K + $L + $M + EOF + ) && + eval P$P3=P3:$P3 +} + +create_pack_4 () { + P4=$(git -C "$master_repo/objects/pack" pack-objects -q pack <<-EOF + $J + $K + $L + $M + $P + EOF + ) && + eval P$P4=P4:$P4 +} + +create_pack_5 () { + P5=$(git -C "$master_repo/objects/pack" pack-objects -q pack <<-EOF + $G + $H + $N + $O + EOF + ) && + eval P$P5=P5:$P5 +} + +create_pack_6 () { + P6=$(git -C "$master_repo/objects/pack" pack-objects -q pack <<-EOF + $N + $O + $Q + EOF + ) && + eval P$P6=P6:$P6 +} + +create_pack_7 () { + P7=$(git -C "$master_repo/objects/pack" pack-objects -q pack <<-EOF + $P + $Q + EOF + ) && + eval P$P7=P7:$P7 +} + +create_pack_8 () { + P8=$(git -C "$master_repo/objects/pack" pack-objects -q pack <<-EOF + $A + EOF + ) && + eval P$P8=P8:$P8 +} + +format_packfiles () { + sed \ + -e "s#.*/pack-\(.*\)\.idx#\1#" \ + -e "s#.*/pack-\(.*\)\.pack#\1#" | + sort -u | + while read p + do + if test -z "$(eval echo \${P$p})" + then + echo $p + else + eval echo "\${P$p}" + fi + done | + sort +} + +test_expect_success 'setup master repo' ' + git init --bare "$master_repo" && + create_commits_in "$master_repo" A B C D E F G H I J K L M N O P Q R +' + +############################################################################# +# Chart of packs and objects for this test case +# +# | T A B C D E F G H I J K L M N O P Q R +# ----+-------------------------------------- +# P1 | x x x x x x x x +# P2 | x x x x x x x +# P3 | x x x x x x +# ----+-------------------------------------- +# ALL | x x x x x x x x x x x x x x x +# +############################################################################# +test_expect_success 'no redundant for pack 1, 2, 3' ' + create_pack_1 && create_pack_2 && create_pack_3 && + ( + cd "$master_repo" && + git pack-redundant --all >out && + test_must_be_empty out + ) +' + +test_expect_success 'create pack 4, 5' ' + create_pack_4 && create_pack_5 +' + +############################################################################# +# Chart of packs and objects for this test case +# +# | T A B C D E F G H I J K L M N O P Q R +# ----+-------------------------------------- +# P1 | x x x x x x x x +# P2* | ! ! ! ! ! ! ! +# P3 | x x x x x x +# P4 | x x x x x +# P5 | x x x x +# ----+-------------------------------------- +# ALL | x x x x x x x x x x x x x x x x x x +# +############################################################################# +test_expect_success 'one of pack-2/pack-3 is redundant' ' + ( + cd "$master_repo" && + cat >expect <<-EOF && + P2:$P2 + EOF + git pack-redundant --all >out && + format_packfiles actual && + test_cmp expect actual + ) +' + +test_expect_success 'create pack 6, 7' ' + create_pack_6 && create_pack_7 +' + +############################################################################# +# Chart of packs and objects for this test case +# +# | T A B C D E F G H I J K L M N O P Q R +# ----+-------------------------------------- +# P1 | x x x x x x x x +# P2* | ! ! ! ! ! ! ! +# P3 | x x x x x x +# P4* | ! ! ! ! ! +# P5 | x x x x +# P6* | ! ! ! +# P7 | x x +# ----+-------------------------------------- +# ALL | x x x x x x x x x x x x x x x x x x x +# +############################################################################# +test_expect_success 'pack 2, 4, and 6 are redundant' ' + ( + cd "$master_repo" && + cat >expect <<-EOF && + P2:$P2 + P4:$P4 + P6:$P6 + EOF + git pack-redundant --all >out && + format_packfiles actual && + test_cmp expect actual + ) +' + +test_expect_success 'create pack 8' ' + create_pack_8 +' + +############################################################################# +# Chart of packs and objects for this test case +# +# | T A B C D E F G H I J K L M N O P Q R +# ----+-------------------------------------- +# P1 | x x x x x x x x +# P2* | ! ! ! ! ! ! ! +# P3 | x x x x x x +# P4* | ! ! ! ! ! +# P5 | x x x x +# P6* | ! ! ! +# P7 | x x +# P8* | ! +# ----+-------------------------------------- +# ALL | x x x x x x x x x x x x x x x x x x x +# +############################################################################# +test_expect_success 'pack-8 (subset of pack-1) is also redundant' ' + ( + cd "$master_repo" && + cat >expect <<-EOF && + P2:$P2 + P4:$P4 + P6:$P6 + P8:$P8 + EOF + git pack-redundant --all >out && + format_packfiles actual && + test_cmp expect actual + ) +' + +test_expect_success 'clean loose objects' ' + ( + cd "$master_repo" && + git prune-packed && + find objects -type f | sed -e "/objects\/pack\//d" >out && + test_must_be_empty out + ) +' + +test_expect_success 'remove redundant packs and pass fsck' ' + ( + cd "$master_repo" && + git pack-redundant --all | xargs rm && + git fsck && + git pack-redundant --all >out && + test_must_be_empty out + ) +' + +# The following test cases will execute inside `shared.git`, instead of +# inside `master.git`. +test_expect_success 'setup shared.git' ' + git clone --mirror "$master_repo" "$shared_repo" && + ( + cd "$shared_repo" && + printf "../../$master_repo/objects\n" >objects/info/alternates + ) +' + +test_expect_success 'no redundant packs without --alt-odb' ' + ( + cd "$shared_repo" && + git pack-redundant --all >out && + test_must_be_empty out + ) +' + +############################################################################# +# Chart of packs and objects for this test case +# +# ================ master.git =============== +# | T A B C D E F G H I J K L M N O P Q R <----------+ +# ----+-------------------------------------- | +# P1 | x x x x x x x x | +# P3 | x x x x x x | +# P5 | x x x x | +# P7 | x x | +# ----+-------------------------------------- | +# ALL | x x x x x x x x x x x x x x x x x x x | +# | +# | +# ================ shared.git =============== | +# | T A B C D E F G H I J K L M N O P Q R +# ----+-------------------------------------- +# P1* | s s s s s s s s +# P3* | s s s s s s +# P5* | s s s s +# P7* | s s +# ----+-------------------------------------- +# ALL | x x x x x x x x x x x x x x x x x x x +# +############################################################################# +test_expect_success 'pack-redundant --verbose: show duplicate packs in stderr' ' + ( + cd "$shared_repo" && + cat >expect <<-EOF && + P1:$P1 + P3:$P3 + P5:$P5 + P7:$P7 + EOF + git pack-redundant --all --verbose >out 2>out.err && + test_must_be_empty out && + grep "pack$" out.err | format_packfiles >actual && + test_cmp expect actual + ) +' + +test_expect_success 'remove redundant packs by alt-odb, no packs left' ' + ( + cd "$shared_repo" && + cat >expect <<-EOF && + fatal: Zero packs found! + EOF + git pack-redundant --all --alt-odb | xargs rm && + git fsck && + test_must_fail git pack-redundant --all --alt-odb >actual 2>&1 && + test_cmp expect actual + ) +' + +# Note: DO NOT run function `create_pack_*` in sub shell, or variables are not set +create_pack_x1_in () { + repo="$1" && + Px1=$(git -C "$repo/objects/pack" pack-objects -q pack <<-EOF + $X + $Y + $Z + $A + $B + $C + EOF + ) && + eval P${Px1}=Px1:${Px1} +} + +create_pack_x2_in () { + repo="$1" && + Px2=$(git -C "$repo/objects/pack" pack-objects -q pack <<-EOF + $X + $Y + $Z + $D + $E + $F + EOF + ) && + eval P${Px2}=Px2:${Px2} +} + +test_expect_success 'create new objects and packs in shared.git' ' + create_commits_in "$shared_repo" X Y Z && + create_pack_x1_in "$shared_repo" && + create_pack_x2_in "$shared_repo" +' + +test_expect_success 'no redundant without --alt-odb' ' + ( + cd "$shared_repo" && + git pack-redundant --all >out && + test_must_be_empty out + ) +' + +############################################################################# +# Chart of packs and objects for this test case +# +# ================ master.git =============== +# | T A B C D E F G H I J K L M N O P Q R <----------------+ +# ----+-------------------------------------- | +# P1 | x x x x x x x x | +# P3 | x x x x x x | +# P5 | x x x x | +# P7 | x x | +# ----+-------------------------------------- | +# ALL | x x x x x x x x x x x x x x x x x x x | +# | +# | +# ================ shared.git ======================= | +# | T A B C D E F G H I J K L M N O P Q R X Y Z +# ----+---------------------------------------------- +# Px1 | s s s x x x +# Px2*| s s s ! ! ! +# ----+---------------------------------------------- +# ALL | s s s s s s s s s s s s s s s s s s s x x x +# +############################################################################# +test_expect_success 'one pack is redundant' ' + ( + cd "$shared_repo" && + git pack-redundant --all --alt-odb >out && + format_packfiles actual && + test_line_count = 1 actual + ) +' + +############################################################################# +# Chart of packs and objects for this test case +# +# ================ master.git =============== +# | T A B C D E F G H I J K L M N O P Q R <----------------+ +# ----+-------------------------------------- | +# P1 | x x x x x x x x | +# P3 | x x x x x x | +# P5 | x x x x | +# P7 | x x | +# ----+-------------------------------------- | +# ALL | x x x x x x x x x x x x x x x x x x x | +# | +# | +# ================ shared.git ======================= | +# | T A B C D E F G H I J K L M N O P Q R X Y Z +# ----+---------------------------------------------- +# Px1*| s s s i i i +# Px2*| s s s i i i +# ----+---------------------------------------------- +# ALL | s s s s s s s s s s s s s s s s s s s i i i +# (ignored objects, marked with i) +# +############################################################################# +test_expect_success 'set ignore objects and all two packs are redundant' ' + ( + cd "$shared_repo" && + cat >expect <<-EOF && + Px1:$Px1 + Px2:$Px2 + EOF + git pack-redundant --all --alt-odb >out <<-EOF && + $X + $Y + $Z + EOF + format_packfiles actual && + test_cmp expect actual + ) +' + +test_done From patchwork Fri Feb 1 16:21:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 10793205 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 591616C2 for ; Fri, 1 Feb 2019 16:22:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 48C67325D0 for ; Fri, 1 Feb 2019 16:22:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3D628325D7; Fri, 1 Feb 2019 16:22:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B101F325D0 for ; Fri, 1 Feb 2019 16:22:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728824AbfBAQW3 (ORCPT ); Fri, 1 Feb 2019 11:22:29 -0500 Received: from mail-pf1-f193.google.com ([209.85.210.193]:45652 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728357AbfBAQW3 (ORCPT ); Fri, 1 Feb 2019 11:22:29 -0500 Received: by mail-pf1-f193.google.com with SMTP id g62so3422352pfd.12 for ; Fri, 01 Feb 2019 08:22:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=clTtnisEtJp6hRAy0x/TLhfQUCf9WYBfVLJ0GExX3ws=; b=I6kOUmDxMMh0AdfHrn6C+3ta9V4FXdxEw42D9/yd8uBgSK2vKLAJZ/bqhURjeAU3Mj XiQmhvkNDBZXoixsOrXXMx0Uatm2yV9MC1PkALOPEJV2ifa3+b1Y3pILtVi0lHagGFj7 VZQZ3p97wGJRxuxQR7i4Sg+IrgJmHdqJFTgFF/RqkShGRl4KpM01r7n8zo6U2bDOiLVd nN72v4iFeULn+vflMFJHY8YDsUEGZPLfvzqdasvQtXzv/eGLzuFramemeGaKpg50LDuE 8iIBJodyV0P5Ed30PdzSKZBerBm7jPxvOqxaBJYuha67vvCpQ0rFeWHJgop/2l4FKOnE 7j1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=clTtnisEtJp6hRAy0x/TLhfQUCf9WYBfVLJ0GExX3ws=; b=LMkZ22itv+A21m1y4a6KMhr6p3vxjhXyRC5Yy/fJEMCnFfF/63knRDFqiUdXhlv3jg akvjTie8q0w0IxziaFyXLCUoab46Td99so8sNCs3ELrzvqKeyW7PVcjc8S5xIiQvb18k 8gPXQSpBagatDvP9+r7vR/w5l/6x68eriBVAxcIfid1zgGhwSJeKq7LupF83+LpJlXvN fZLlXUWRahDfCuQ1c6c2L4xk47P8S+PfNrOY2dpR2y6bqt8vW9RUG18KKC9QmwCR8ynn mVtuh49jq28Nmaq0vOZlFYqQ8qC4QvFNPwxkBYa2G1pUwORmmqrK/2jTO1YC2U/RApLb y8ng== X-Gm-Message-State: AJcUukcv39sy/Z9ptleJd/RDcZPYfv7M3UTpgm1ZM8KrpukjVqFqFzBz A+mQNBp0yh7nhgq/5sv3/nc= X-Google-Smtp-Source: ALg8bN5vAkyMTSi/El9sZKWC47C3M0qmcu+aArZwPyiQkzMSWXFiWPcZHckVrcy/DhlZ1BDIlkfb/Q== X-Received: by 2002:aa7:8045:: with SMTP id y5mr39634853pfm.62.1549038148707; Fri, 01 Feb 2019 08:22:28 -0800 (PST) Received: from localhost.localdomain ([2408:84e1:1:ce70:7c5c:7512:51d2:8f91]) by smtp.gmail.com with ESMTPSA id x2sm11872694pfx.78.2019.02.01.08.22.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Feb 2019 08:22:27 -0800 (PST) From: Jiang Xin To: Junio C Hamano , Git List , Sun Chao Cc: Jiang Xin , Jiang Xin Subject: [PATCH v9 2/6] pack-redundant: delay creation of unique_objects Date: Sat, 2 Feb 2019 00:21:48 +0800 Message-Id: <20190201162152.31136-3-worldhello.net@gmail.com> X-Mailer: git-send-email 2.20.1.103.ged0fc2ca7b In-Reply-To: <20190130114736.30357-1-worldhello.net@gmail.com> References: <20190130114736.30357-1-worldhello.net@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jiang Xin Instead of initializing unique_objects in `add_pack()`, copy from all_objects in `cmp_two_packs()`, when unwanted objects are removed from all_objects. This will save memory (no allocate memory for alt-odb packs), and run `llist_sorted_difference_inplace()` only once when removing ignored objects and removing objects in alt-odb in `scan_alt_odb_packs()`. Signed-off-by: Jiang Xin --- builtin/pack-redundant.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c index cf9a9aabd4..f7dab0ec60 100644 --- a/builtin/pack-redundant.c +++ b/builtin/pack-redundant.c @@ -254,6 +254,11 @@ static void cmp_two_packs(struct pack_list *p1, struct pack_list *p2) struct llist_item *p1_hint = NULL, *p2_hint = NULL; const unsigned int hashsz = the_hash_algo->rawsz; + if (!p1->unique_objects) + p1->unique_objects = llist_copy(p1->all_objects); + if (!p2->unique_objects) + p2->unique_objects = llist_copy(p2->all_objects); + p1_base = p1->pack->index_data; p2_base = p2->pack->index_data; p1_base += 256 * 4 + ((p1->pack->index_version < 2) ? 4 : 8); @@ -536,7 +541,7 @@ static void scan_alt_odb_packs(void) while (alt) { local = local_packs; while (local) { - llist_sorted_difference_inplace(local->unique_objects, + llist_sorted_difference_inplace(local->all_objects, alt->all_objects); local = local->next; } @@ -567,8 +572,7 @@ static struct pack_list * add_pack(struct packed_git *p) llist_insert_back(l.all_objects, (const struct object_id *)(base + off)); off += step; } - /* this list will be pruned in cmp_two_packs later */ - l.unique_objects = llist_copy(l.all_objects); + l.unique_objects = NULL; if (p->pack_local) return pack_list_insert(&local_packs, &l); else @@ -646,7 +650,6 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix) load_all_objects(); - cmp_local_packs(); if (alt_odb) scan_alt_odb_packs(); @@ -663,10 +666,12 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix) llist_sorted_difference_inplace(all_objects, ignore); pl = local_packs; while (pl) { - llist_sorted_difference_inplace(pl->unique_objects, ignore); + llist_sorted_difference_inplace(pl->all_objects, ignore); pl = pl->next; } + cmp_local_packs(); + minimize(&min); if (verbose) { From patchwork Fri Feb 1 16:21:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 10793209 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1844014E1 for ; Fri, 1 Feb 2019 16:22:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0478B321CB for ; Fri, 1 Feb 2019 16:22:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EC53432511; Fri, 1 Feb 2019 16:22:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96962321CB for ; Fri, 1 Feb 2019 16:22:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729542AbfBAQWg (ORCPT ); Fri, 1 Feb 2019 11:22:36 -0500 Received: from mail-pf1-f193.google.com ([209.85.210.193]:35849 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728357AbfBAQWg (ORCPT ); Fri, 1 Feb 2019 11:22:36 -0500 Received: by mail-pf1-f193.google.com with SMTP id b85so3447178pfc.3 for ; Fri, 01 Feb 2019 08:22:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jk4bKCwkogKKQ34axZb1eDG5+hZ7gRU3tZhU8knyD2w=; b=Sn5oNKPsayVzYjYy27aaxtyKC2KyZyvmkLc0tO6hUX6J28c2fkVGaI2xUqCzJApcCq 2bwky9a6XSXAekN8yBWi5hSc8YZSasS0OjzZYtFKhjXpYl7YmaQvFvK7dvxlVBU1DXzq LvGfT5M1tjasK6FnKJ9j8ulaCh8NHpuel4WfNFxpIgnCia4i9xsW8hCu5JOemuoSP7Mb fBr5oS5MUcqk9Te4DziUZX98T50A4gHM3IvLQTLFSDA5KgDxBmsI1PcLC+wyNEzVKjqL 6tCpmz+4i9ibBSW6xaIr/0ptqMp630S4xA8PYgsD4/mjQwtvTAdf5VKzi7ZVt6btf875 Zpzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jk4bKCwkogKKQ34axZb1eDG5+hZ7gRU3tZhU8knyD2w=; b=KOHD9tJZV7ZTLpcUH2CS51HIxAl30fZo6pvIvs8qw+5IjGHo0WrgKFDx/QQ7CSHOZD 0nfJRZXiBRKTSzS1vn1jVdwggN4dyhZnAZ9PhRMhjxq9IhE8ZtyCPmqbN0oTUZg64NuP fqYN4qypi5DXj2A6RCmMzPqFDgwE7dizNC1fu5XG588/LMcprlJmtIUJNyTFO7/g14k2 NNaCeOC5vbpvcfIlm7o+3EhhJXSmXk8+KcZsiRqtwcKlWo0WXSNmYDjwJcXupXatyHzG tP5l97rf3Nr0TnHltxe9WQyvRmkuGuIRbY1STRjK7qL6BXqSP25+H5ZRQkL19PoGGpKA H+rg== X-Gm-Message-State: AHQUAuZz1RninzM4KJPoJZUUHdZSBJ8x8PdrQoo5kvFmIJIjVkNvaU0R 19ltZhVTyYY1vEkksIOyX7U= X-Google-Smtp-Source: AHgI3IZkAyyOWg4/qSaVSWcrHV7oEMccIRg5PN9FHFkkc27P6ZTHl9uPy2u6r716UAFMu3jPSdlpng== X-Received: by 2002:a63:1d1d:: with SMTP id d29mr2826500pgd.49.1549038155640; Fri, 01 Feb 2019 08:22:35 -0800 (PST) Received: from localhost.localdomain ([2408:84e1:1:ce70:7c5c:7512:51d2:8f91]) by smtp.gmail.com with ESMTPSA id x2sm11872694pfx.78.2019.02.01.08.22.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Feb 2019 08:22:34 -0800 (PST) From: Jiang Xin To: Junio C Hamano , Git List , Sun Chao Cc: Jiang Xin , Jiang Xin Subject: [PATCH v9 3/6] pack-redundant: delete redundant code Date: Sat, 2 Feb 2019 00:21:49 +0800 Message-Id: <20190201162152.31136-4-worldhello.net@gmail.com> X-Mailer: git-send-email 2.20.1.103.ged0fc2ca7b In-Reply-To: <20190130114736.30357-1-worldhello.net@gmail.com> References: <20190130114736.30357-1-worldhello.net@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sun Chao The objects in alt-odb are removed from `all_objects` twice in `load_all_objects` and `scan_alt_odb_packs`, remove it from the later function. Signed-off-by: Sun Chao Signed-off-by: Jiang Xin --- builtin/pack-redundant.c | 1 - 1 file changed, 1 deletion(-) diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c index f7dab0ec60..4a06f057dd 100644 --- a/builtin/pack-redundant.c +++ b/builtin/pack-redundant.c @@ -545,7 +545,6 @@ static void scan_alt_odb_packs(void) alt->all_objects); local = local->next; } - llist_sorted_difference_inplace(all_objects, alt->all_objects); alt = alt->next; } } From patchwork Fri Feb 1 16:21:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 10793211 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C9A56C2 for ; Fri, 1 Feb 2019 16:22:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B48E321CB for ; Fri, 1 Feb 2019 16:22:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6EF9232572; Fri, 1 Feb 2019 16:22:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ABCC0321CB for ; Fri, 1 Feb 2019 16:22:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729961AbfBAQWq (ORCPT ); Fri, 1 Feb 2019 11:22:46 -0500 Received: from mail-pg1-f170.google.com ([209.85.215.170]:44346 "EHLO mail-pg1-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728357AbfBAQWq (ORCPT ); Fri, 1 Feb 2019 11:22:46 -0500 Received: by mail-pg1-f170.google.com with SMTP id t13so3139864pgr.11 for ; Fri, 01 Feb 2019 08:22:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mzRw9njaBFhQ7ASe/gkWGtPh8xKbSKnOUsyhvE6Au40=; b=Uy2U+o9vV4UfnTydC6Jsi3xZxM13RbuHjF1BLJBE7ldp6OA0Hete+OxeeHSAucXe+C BeKDwuZYo81jU8uoxUSmBWXVh9/Be8ttuEMhXH4zRSmO+wOdmlrJWQpB9bOReIzQ/Oa3 a6s4C9RehDalQVj1vFsFNpDaJNgC83AlkdKtCLEacTlVDYAk7w33wK0pmBvcuKgy6qCr bsO75vjxnK1WVq++kgGd98pDg8d54YanUJ4u/mxnXavYou6pcs2DXTAz4zA/ynMs6T79 GBv4+A8QgK5d/bt67UNs24ur5vSuvDHoB+8rJtmhFB1TawcgZd/fR8GfWoHUWXfuqVfd rnpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mzRw9njaBFhQ7ASe/gkWGtPh8xKbSKnOUsyhvE6Au40=; b=fqpVMGQ0FCLXMWCt1YyKRnxx/fH24OnNv51mJyPmVrXj2XWuZ2jLULMf8l46BlJ/xh 3kjyTGlg1KA4ATgW+gCuc0bpf5Quzk5SKfJkk9w51SJoxIHVftXC4gyDz3t7Yq2QofgB qUDb7gRM/kH0qEgQLSH1zNSSOp+Ow27CGMkP/diyS11QJzPpCLlqVX9/LQnUzACvMlrN d+t1xfg1ieulf65BjSQga33a7Tz7347I2ZVwWGedSBjFETf1ZSYS3qQwIppNkhaYx5kW OgTiBQnD6j0DsVne4ORta0eDXKISM40L37OaTVYTsef34Hj2M5xc3HtPM0FOp32fNvKy RMGA== X-Gm-Message-State: AHQUAubkKxOEUhkcJglGM3xuFy9/EDEe+6e5RNLYl3VgM0vtrV5W8TIO QvDma0MZI5avpqH13rBj0P4= X-Google-Smtp-Source: AHgI3IaC4f9AGXLO7tikqV6j+cbcUcOzhQlb53NkBOXMmLjRiXL0aPtTrmZUo1drXQD0RubqfULkvg== X-Received: by 2002:a63:7044:: with SMTP id a4mr2771026pgn.359.1549038165093; Fri, 01 Feb 2019 08:22:45 -0800 (PST) Received: from localhost.localdomain ([2408:84e1:1:ce70:7c5c:7512:51d2:8f91]) by smtp.gmail.com with ESMTPSA id x2sm11872694pfx.78.2019.02.01.08.22.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Feb 2019 08:22:44 -0800 (PST) From: Jiang Xin To: Junio C Hamano , Git List , Sun Chao Cc: Jiang Xin , Jiang Xin Subject: [PATCH v9 4/6] pack-redundant: new algorithm to find min packs Date: Sat, 2 Feb 2019 00:21:50 +0800 Message-Id: <20190201162152.31136-5-worldhello.net@gmail.com> X-Mailer: git-send-email 2.20.1.103.ged0fc2ca7b In-Reply-To: <20190130114736.30357-1-worldhello.net@gmail.com> References: <20190130114736.30357-1-worldhello.net@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sun Chao When calling `git pack-redundant --all`, if there are too many local packs and too many redundant objects within them, the too deep iteration of `get_permutations` will exhaust all the resources, and the process of `git pack-redundant` will be killed. The following script could create a repository with too many redundant packs, and running `git pack-redundant --all` in the `test.git` repo will die soon. #!/bin/sh repo="$(pwd)/test.git" work="$(pwd)/test" i=1 max=199 if test -d "$repo" || test -d "$work"; then echo >&2 "ERROR: '$repo' or '$work' already exist" exit 1 fi git init -q --bare "$repo" git --git-dir="$repo" config gc.auto 0 git --git-dir="$repo" config transfer.unpackLimit 0 git clone -q "$repo" "$work" 2>/dev/null while :; do cd "$work" echo "loop $i: $(date +%s)" >$i git add $i git commit -q -sm "loop $i" git push -q origin HEAD:master printf "\rCreate pack %4d/%d\t" $i $max if test $i -ge $max; then break; fi cd "$repo" git repack -q if test $(($i % 2)) -eq 0; then git repack -aq pack=$(ls -t $repo/objects/pack/*.pack | head -1) touch "${pack%.pack}.keep" fi i=$((i+1)) done printf "\ndone\n" To get the `min` unique pack list, we can replace the iteration in `minimize` function with a new algorithm, and this could solve this issue: 1. Get the unique and non_uniqe packs, add the unique packs to the `min` list. 2. Remove the objects of unique packs from non_unique packs, then each object left in the non_unique packs will have at least two copies. 3. Sort the non_unique packs by the objects' size, more objects first, and add the first non_unique pack to `min` list. 4. Drop the duplicated objects from other packs in the ordered non_unique pack list, and repeat step 3. Some test cases will fail on Mac OS X. Mark them and will resolve in later commit. Original PR and discussions: https://github.com/jiangxin/git/pull/25 Signed-off-by: Sun Chao Signed-off-by: Jiang Xin Signed-off-by: Junio C Hamano --- builtin/pack-redundant.c | 194 +++++++++++++------------------------- t/t5323-pack-redundant.sh | 12 +-- 2 files changed, 73 insertions(+), 133 deletions(-) diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c index 4a06f057dd..d6d9a66e46 100644 --- a/builtin/pack-redundant.c +++ b/builtin/pack-redundant.c @@ -35,11 +35,6 @@ static struct pack_list { struct llist *all_objects; } *local_packs = NULL, *altodb_packs = NULL; -struct pll { - struct pll *next; - struct pack_list *pl; -}; - static struct llist_item *free_nodes; static inline void llist_item_put(struct llist_item *item) @@ -63,15 +58,6 @@ static inline struct llist_item *llist_item_get(void) return new_item; } -static void llist_free(struct llist *list) -{ - while ((list->back = list->front)) { - list->front = list->front->next; - llist_item_put(list->back); - } - free(list); -} - static inline void llist_init(struct llist **list) { *list = xmalloc(sizeof(struct llist)); @@ -290,78 +276,6 @@ static void cmp_two_packs(struct pack_list *p1, struct pack_list *p2) } } -static void pll_free(struct pll *l) -{ - struct pll *old; - struct pack_list *opl; - - while (l) { - old = l; - while (l->pl) { - opl = l->pl; - l->pl = opl->next; - free(opl); - } - l = l->next; - free(old); - } -} - -/* all the permutations have to be free()d at the same time, - * since they refer to each other - */ -static struct pll * get_permutations(struct pack_list *list, int n) -{ - struct pll *subset, *ret = NULL, *new_pll = NULL; - - if (list == NULL || pack_list_size(list) < n || n == 0) - return NULL; - - if (n == 1) { - while (list) { - new_pll = xmalloc(sizeof(*new_pll)); - new_pll->pl = NULL; - pack_list_insert(&new_pll->pl, list); - new_pll->next = ret; - ret = new_pll; - list = list->next; - } - return ret; - } - - while (list->next) { - subset = get_permutations(list->next, n - 1); - while (subset) { - new_pll = xmalloc(sizeof(*new_pll)); - new_pll->pl = subset->pl; - pack_list_insert(&new_pll->pl, list); - new_pll->next = ret; - ret = new_pll; - subset = subset->next; - } - list = list->next; - } - return ret; -} - -static int is_superset(struct pack_list *pl, struct llist *list) -{ - struct llist *diff; - - diff = llist_copy(list); - - while (pl) { - llist_sorted_difference_inplace(diff, pl->all_objects); - if (diff->size == 0) { /* we're done */ - llist_free(diff); - return 1; - } - pl = pl->next; - } - llist_free(diff); - return 0; -} - static size_t sizeof_union(struct packed_git *p1, struct packed_git *p2) { size_t ret = 0; @@ -426,14 +340,52 @@ static inline off_t pack_set_bytecount(struct pack_list *pl) return ret; } +static int cmp_pack_list_reverse(const void *a, const void *b) +{ + struct pack_list *pl_a = *((struct pack_list **)a); + struct pack_list *pl_b = *((struct pack_list **)b); + size_t sz_a = pl_a->all_objects->size; + size_t sz_b = pl_b->all_objects->size; + + if (sz_a == sz_b) + return 0; + else if (sz_a < sz_b) + return 1; + else + return -1; +} + +/* Sort pack_list, greater size of all_objects first */ +static void sort_pack_list(struct pack_list **pl) +{ + struct pack_list **ary, *p; + int i; + size_t n = pack_list_size(*pl); + + if (n < 2) + return; + + /* prepare an array of packed_list for easier sorting */ + ary = xcalloc(n, sizeof(struct pack_list *)); + for (n = 0, p = *pl; p; p = p->next) + ary[n++] = p; + + QSORT(ary, n, cmp_pack_list_reverse); + + /* link them back again */ + for (i = 0; i < n - 1; i++) + ary[i]->next = ary[i + 1]; + ary[n - 1]->next = NULL; + *pl = ary[0]; + + free(ary); +} + + static void minimize(struct pack_list **min) { - struct pack_list *pl, *unique = NULL, - *non_unique = NULL, *min_perm = NULL; - struct pll *perm, *perm_all, *perm_ok = NULL, *new_perm; - struct llist *missing; - off_t min_perm_size = 0, perm_size; - int n; + struct pack_list *pl, *unique = NULL, *non_unique = NULL; + struct llist *missing, *unique_pack_objects; pl = local_packs; while (pl) { @@ -451,49 +403,37 @@ static void minimize(struct pack_list **min) pl = pl->next; } + *min = unique; + /* return if there are no objects missing from the unique set */ if (missing->size == 0) { - *min = unique; free(missing); return; } - /* find the permutations which contain all missing objects */ - for (n = 1; n <= pack_list_size(non_unique) && !perm_ok; n++) { - perm_all = perm = get_permutations(non_unique, n); - while (perm) { - if (is_superset(perm->pl, missing)) { - new_perm = xmalloc(sizeof(struct pll)); - memcpy(new_perm, perm, sizeof(struct pll)); - new_perm->next = perm_ok; - perm_ok = new_perm; - } - perm = perm->next; - } - if (perm_ok) - break; - pll_free(perm_all); - } - if (perm_ok == NULL) - die("Internal error: No complete sets found!"); - - /* find the permutation with the smallest size */ - perm = perm_ok; - while (perm) { - perm_size = pack_set_bytecount(perm->pl); - if (!min_perm_size || min_perm_size > perm_size) { - min_perm_size = perm_size; - min_perm = perm->pl; - } - perm = perm->next; - } - *min = min_perm; - /* add the unique packs to the list */ - pl = unique; + unique_pack_objects = llist_copy(all_objects); + llist_sorted_difference_inplace(unique_pack_objects, missing); + + /* remove unique pack objects from the non_unique packs */ + pl = non_unique; while (pl) { - pack_list_insert(min, pl); + llist_sorted_difference_inplace(pl->all_objects, unique_pack_objects); pl = pl->next; } + + while (non_unique) { + /* sort the non_unique packs, greater size of all_objects first */ + sort_pack_list(&non_unique); + if (non_unique->all_objects->size == 0) + break; + + pack_list_insert(min, non_unique); + + for (pl = non_unique->next; pl && pl->all_objects->size > 0; pl = pl->next) + llist_sorted_difference_inplace(pl->all_objects, non_unique->all_objects); + + non_unique = non_unique->next; + } } static void load_all_objects(void) @@ -606,7 +546,7 @@ static void load_all(void) int cmd_pack_redundant(int argc, const char **argv, const char *prefix) { int i; - struct pack_list *min, *red, *pl; + struct pack_list *min = NULL, *red, *pl; struct llist *ignore; struct object_id *oid; char buf[GIT_MAX_HEXSZ + 2]; /* hex hash + \n + \0 */ diff --git a/t/t5323-pack-redundant.sh b/t/t5323-pack-redundant.sh index d224ff3c50..897acaa365 100755 --- a/t/t5323-pack-redundant.sh +++ b/t/t5323-pack-redundant.sh @@ -218,7 +218,7 @@ test_expect_success 'create pack 4, 5' ' # ALL | x x x x x x x x x x x x x x x x x x # ############################################################################# -test_expect_success 'one of pack-2/pack-3 is redundant' ' +test_expect_failure 'one of pack-2/pack-3 is redundant (failed on Mac)' ' ( cd "$master_repo" && cat >expect <<-EOF && @@ -250,7 +250,7 @@ test_expect_success 'create pack 6, 7' ' # ALL | x x x x x x x x x x x x x x x x x x x # ############################################################################# -test_expect_success 'pack 2, 4, and 6 are redundant' ' +test_expect_failure 'pack 2, 4, and 6 are redundant (failed on Mac)' ' ( cd "$master_repo" && cat >expect <<-EOF && @@ -285,7 +285,7 @@ test_expect_success 'create pack 8' ' # ALL | x x x x x x x x x x x x x x x x x x x # ############################################################################# -test_expect_success 'pack-8 (subset of pack-1) is also redundant' ' +test_expect_failure 'pack-8 (subset of pack-1) is also redundant (failed on Mac)' ' ( cd "$master_repo" && cat >expect <<-EOF && @@ -309,7 +309,7 @@ test_expect_success 'clean loose objects' ' ) ' -test_expect_success 'remove redundant packs and pass fsck' ' +test_expect_failure 'remove redundant packs and pass fsck (failed on Mac)' ' ( cd "$master_repo" && git pack-redundant --all | xargs rm && @@ -329,7 +329,7 @@ test_expect_success 'setup shared.git' ' ) ' -test_expect_success 'no redundant packs without --alt-odb' ' +test_expect_failure 'no redundant packs without --alt-odb (failed on Mac)' ' ( cd "$shared_repo" && git pack-redundant --all >out && @@ -362,7 +362,7 @@ test_expect_success 'no redundant packs without --alt-odb' ' # ALL | x x x x x x x x x x x x x x x x x x x # ############################################################################# -test_expect_success 'pack-redundant --verbose: show duplicate packs in stderr' ' +test_expect_failure 'pack-redundant --verbose: show duplicate packs in stderr (failed on Mac)' ' ( cd "$shared_repo" && cat >expect <<-EOF && From patchwork Fri Feb 1 16:21:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 10793213 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 035B214E1 for ; Fri, 1 Feb 2019 16:22:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E6B5732517 for ; Fri, 1 Feb 2019 16:22:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DAD4D3254F; Fri, 1 Feb 2019 16:22:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6011A32511 for ; Fri, 1 Feb 2019 16:22:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730138AbfBAQWx (ORCPT ); Fri, 1 Feb 2019 11:22:53 -0500 Received: from mail-pg1-f195.google.com ([209.85.215.195]:43216 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728357AbfBAQWx (ORCPT ); Fri, 1 Feb 2019 11:22:53 -0500 Received: by mail-pg1-f195.google.com with SMTP id v28so3147121pgk.10 for ; Fri, 01 Feb 2019 08:22:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fdByC06wKXJ0nXG+BjICNEdsDzlfp3IIPm/Z0CdG+V0=; b=RNbYL127Z4nSGQ1p85z6+5aagp6bZaly8LuO5I3yT0x6oVZ+PShipM+10fXtO4obWo m/82zr8s0h4mAnbf4TvbGjHwGh6lswWEQbI09VYfPHLzJU5YygjVTccc3xbhHB+iuSvN u2gTNkOLmdVF8JBPFqFl5kSvOU6IgdFNJDgYjw7/k0ipBwcCgwOerDD1mwy4y0kHog/p MAAL5AKAZuKdQ6qtKFB7SxG6NMjXx6ej1VL5Myt8TZTcsECIU9zel0iumB2LR7awJ8wJ eG9sYNuKcSJdW88CWFCEe35pyYwygD0e0VnaXK+Od7qi51WG8ai0ktCKzGDvTmgFU632 lV6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fdByC06wKXJ0nXG+BjICNEdsDzlfp3IIPm/Z0CdG+V0=; b=r+DWJf/pjO3i29wj61cyuJzT43M6kNJgXZ9KHfu4qTe/SyBY69r6hHK8Z7L0gh/Ylu YmePIUL7qcTiOcmSa7hEdzCJ0pY3w5hQ+gIPHhnH/C2/mj62SLW35/3BqQLRm6Y94Uq9 Z1rLEgQfS8eiFkzkoGqcrRwQRnEkDsfsgvovttikU0PGCYCpv/KyyZ+WDVszzn5IHUk1 eZ2IPXyUBr3X4U3q7zSZgwFnKEMDFuTeFHiK0v4FT4nfqU9j8qrOOe85ODaPmJsK5CRA w/3hhJwPISrEzjWhYex5nv49sJ4y5kieaq/LpL9SlrwJNLs36/HMdDqk453kxrN7sOPV DXdg== X-Gm-Message-State: AHQUAuYIHnEK9GQEmJU5DuhndpfPYMD7Tyw52qOdZ9tjtWDWgrM4RDRV ugfOCDYPJeFRAoLBqipjU5M= X-Google-Smtp-Source: AHgI3IYlkyse1rv4RBtOQg6AEURLugZxCKwq3BPu6R+mt/6sZvam/4l3GaENsp6HcA3RMl/zCG/pVQ== X-Received: by 2002:a63:151f:: with SMTP id v31mr2828404pgl.34.1549038171987; Fri, 01 Feb 2019 08:22:51 -0800 (PST) Received: from localhost.localdomain ([2408:84e1:1:ce70:7c5c:7512:51d2:8f91]) by smtp.gmail.com with ESMTPSA id x2sm11872694pfx.78.2019.02.01.08.22.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Feb 2019 08:22:51 -0800 (PST) From: Jiang Xin To: Junio C Hamano , Git List , Sun Chao Cc: Jiang Xin , Jiang Xin Subject: [PATCH v9 5/6] pack-redundant: rename pack_list.all_objects Date: Sat, 2 Feb 2019 00:21:51 +0800 Message-Id: <20190201162152.31136-6-worldhello.net@gmail.com> X-Mailer: git-send-email 2.20.1.103.ged0fc2ca7b In-Reply-To: <20190130114736.30357-1-worldhello.net@gmail.com> References: <20190130114736.30357-1-worldhello.net@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jiang Xin New algorithm uses `pack_list.all_objects` to track remaining objects, so rename it to `pack_list.remaining_objects`. Signed-off-by: Jiang Xin Signed-off-by: Junio C Hamano --- builtin/pack-redundant.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c index d6d9a66e46..15cdf233c4 100644 --- a/builtin/pack-redundant.c +++ b/builtin/pack-redundant.c @@ -32,7 +32,7 @@ static struct pack_list { struct pack_list *next; struct packed_git *pack; struct llist *unique_objects; - struct llist *all_objects; + struct llist *remaining_objects; } *local_packs = NULL, *altodb_packs = NULL; static struct llist_item *free_nodes; @@ -241,9 +241,9 @@ static void cmp_two_packs(struct pack_list *p1, struct pack_list *p2) const unsigned int hashsz = the_hash_algo->rawsz; if (!p1->unique_objects) - p1->unique_objects = llist_copy(p1->all_objects); + p1->unique_objects = llist_copy(p1->remaining_objects); if (!p2->unique_objects) - p2->unique_objects = llist_copy(p2->all_objects); + p2->unique_objects = llist_copy(p2->remaining_objects); p1_base = p1->pack->index_data; p2_base = p2->pack->index_data; @@ -344,8 +344,8 @@ static int cmp_pack_list_reverse(const void *a, const void *b) { struct pack_list *pl_a = *((struct pack_list **)a); struct pack_list *pl_b = *((struct pack_list **)b); - size_t sz_a = pl_a->all_objects->size; - size_t sz_b = pl_b->all_objects->size; + size_t sz_a = pl_a->remaining_objects->size; + size_t sz_b = pl_b->remaining_objects->size; if (sz_a == sz_b) return 0; @@ -355,7 +355,7 @@ static int cmp_pack_list_reverse(const void *a, const void *b) return -1; } -/* Sort pack_list, greater size of all_objects first */ +/* Sort pack_list, greater size of remaining_objects first */ static void sort_pack_list(struct pack_list **pl) { struct pack_list **ary, *p; @@ -399,7 +399,7 @@ static void minimize(struct pack_list **min) missing = llist_copy(all_objects); pl = unique; while (pl) { - llist_sorted_difference_inplace(missing, pl->all_objects); + llist_sorted_difference_inplace(missing, pl->remaining_objects); pl = pl->next; } @@ -417,20 +417,20 @@ static void minimize(struct pack_list **min) /* remove unique pack objects from the non_unique packs */ pl = non_unique; while (pl) { - llist_sorted_difference_inplace(pl->all_objects, unique_pack_objects); + llist_sorted_difference_inplace(pl->remaining_objects, unique_pack_objects); pl = pl->next; } while (non_unique) { - /* sort the non_unique packs, greater size of all_objects first */ + /* sort the non_unique packs, greater size of remaining_objects first */ sort_pack_list(&non_unique); - if (non_unique->all_objects->size == 0) + if (non_unique->remaining_objects->size == 0) break; pack_list_insert(min, non_unique); - for (pl = non_unique->next; pl && pl->all_objects->size > 0; pl = pl->next) - llist_sorted_difference_inplace(pl->all_objects, non_unique->all_objects); + for (pl = non_unique->next; pl && pl->remaining_objects->size > 0; pl = pl->next) + llist_sorted_difference_inplace(pl->remaining_objects, non_unique->remaining_objects); non_unique = non_unique->next; } @@ -445,7 +445,7 @@ static void load_all_objects(void) while (pl) { hint = NULL; - l = pl->all_objects->front; + l = pl->remaining_objects->front; while (l) { hint = llist_insert_sorted_unique(all_objects, l->oid, hint); @@ -456,7 +456,7 @@ static void load_all_objects(void) /* remove objects present in remote packs */ pl = altodb_packs; while (pl) { - llist_sorted_difference_inplace(all_objects, pl->all_objects); + llist_sorted_difference_inplace(all_objects, pl->remaining_objects); pl = pl->next; } } @@ -481,8 +481,8 @@ static void scan_alt_odb_packs(void) while (alt) { local = local_packs; while (local) { - llist_sorted_difference_inplace(local->all_objects, - alt->all_objects); + llist_sorted_difference_inplace(local->remaining_objects, + alt->remaining_objects); local = local->next; } alt = alt->next; @@ -499,7 +499,7 @@ static struct pack_list * add_pack(struct packed_git *p) return NULL; l.pack = p; - llist_init(&l.all_objects); + llist_init(&l.remaining_objects); if (open_pack_index(p)) return NULL; @@ -508,7 +508,7 @@ static struct pack_list * add_pack(struct packed_git *p) base += 256 * 4 + ((p->index_version < 2) ? 4 : 8); step = the_hash_algo->rawsz + ((p->index_version < 2) ? 4 : 0); while (off < p->num_objects * step) { - llist_insert_back(l.all_objects, (const struct object_id *)(base + off)); + llist_insert_back(l.remaining_objects, (const struct object_id *)(base + off)); off += step; } l.unique_objects = NULL; @@ -605,7 +605,7 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix) llist_sorted_difference_inplace(all_objects, ignore); pl = local_packs; while (pl) { - llist_sorted_difference_inplace(pl->all_objects, ignore); + llist_sorted_difference_inplace(pl->remaining_objects, ignore); pl = pl->next; } From patchwork Fri Feb 1 16:21:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 10793215 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD4C914E1 for ; Fri, 1 Feb 2019 16:23:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B727B321CB for ; Fri, 1 Feb 2019 16:23:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A8C4232511; Fri, 1 Feb 2019 16:23:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2C0A321CB for ; Fri, 1 Feb 2019 16:23:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730336AbfBAQXD (ORCPT ); Fri, 1 Feb 2019 11:23:03 -0500 Received: from mail-pl1-f196.google.com ([209.85.214.196]:35822 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728357AbfBAQXC (ORCPT ); Fri, 1 Feb 2019 11:23:02 -0500 Received: by mail-pl1-f196.google.com with SMTP id p8so3457638plo.2 for ; Fri, 01 Feb 2019 08:23:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UsYzPFmfEv7A5r/dnG3QtX4UaoMx0fruIY07fmLeWiQ=; b=i6yMrpz9BMBue0VUw1JCbB8yOtzyvWgsBsUVdNm1S+RZiNEccTRmWaPYNbVQq6hZx3 wiinHEW0x9uxOGolkZuYFAYUyw6cV1nwB4wpxU/iaYVhM6FtrbKa9vHKGlVCUg3dfkyM jeAENz1WnuUbrAmO7phxMuF183y+FwrtDUmxFco3bf8w3pI9TzKBVNXan+uaBX1Z6FE/ bUTVOxPKpheDPvI91Zfixj/v48pTGK3YlGfmYWqW0hFnIfeQeq/xzicMxyDyElUdkWXE Ba6XLu1nMqr3bYhy0oxJIrSzu+UEt6jPkNUowBsNTDmvAG5DgspRMqUbhqsfrQKN3fha 43cA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UsYzPFmfEv7A5r/dnG3QtX4UaoMx0fruIY07fmLeWiQ=; b=rQdkPk+CfYTu4FrBlg+oP0xdF2w1IL+7vX3hTgoehnaAxnip7z1w59Bg6x1Jr8dSPK f1YMRF33m5bZlzMQ8IZ98zMjEIHC38YUbDyWXJ/5m2XRsvhu+RmOJB9tGIdLkZ5lchwo yKkWiHjLHINmUzLyVzCsjvaUOLy/q+nsZYiQNKgCntAksQd9X2w9ofB2ojGNLYaGyV9i yTV9muRiqhAv3H07qnLdWOnkVumLIxpt94MWI1GXqYZlvd8c2C6/ZWTJUdDqABZMsxwl pKyyDcQIKRL5dH2GdfRD6ITFWiiVC0XO4yzsYlPWErrmiH22Ff3fEy62LAgCWS9OVE6O 8dYA== X-Gm-Message-State: AJcUuke5WgyaRQ6WYlHfBpvUEyUc0UBIYis9jl41wqU4Aoxaebbsv9cS dFgywP3TKIhiWeMGUokk9Wis1MUz/NbiMQ== X-Google-Smtp-Source: ALg8bN4vMPY4a9rMRcUBn8GpFqoLJ7aB03iEtI2qs6353bVM89SbmXUwzO/VJg2LtQdwv8TQgRA/mA== X-Received: by 2002:a17:902:7882:: with SMTP id q2mr41297581pll.305.1549038182006; Fri, 01 Feb 2019 08:23:02 -0800 (PST) Received: from localhost.localdomain ([2408:84e1:1:ce70:7c5c:7512:51d2:8f91]) by smtp.gmail.com with ESMTPSA id x2sm11872694pfx.78.2019.02.01.08.22.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Feb 2019 08:23:01 -0800 (PST) From: Jiang Xin To: Junio C Hamano , Git List , Sun Chao Cc: Jiang Xin , Jiang Xin , =?utf-8?q?SZEDER_G=C3=A1bor?= Subject: [PATCH v9 6/6] pack-redundant: consistent sort method Date: Sat, 2 Feb 2019 00:21:52 +0800 Message-Id: <20190201162152.31136-7-worldhello.net@gmail.com> X-Mailer: git-send-email 2.20.1.103.ged0fc2ca7b In-Reply-To: <20190130114736.30357-1-worldhello.net@gmail.com> References: <20190130114736.30357-1-worldhello.net@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jiang Xin SZEDER reported that test case t5323 has different test result on MacOS. This is because `cmp_pack_list_reverse` cannot give identical result when two pack being sorted has the same size of remaining_objects. Changes to the sorting function will make consistent test result for t5323. The new algorithm to find redundant packs is a trade-off to save memory resources, and the result of it may be different with old one, and may be not the best result sometimes. Update t5323 for the new algorithm. Reported-by: SZEDER Gábor Signed-off-by: Jiang Xin Signed-off-by: Junio C Hamano --- builtin/pack-redundant.c | 24 ++++++++++++++++-------- t/t5323-pack-redundant.sh | 18 +++++++++--------- 2 files changed, 25 insertions(+), 17 deletions(-) diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c index 15cdf233c4..29ff5e99cb 100644 --- a/builtin/pack-redundant.c +++ b/builtin/pack-redundant.c @@ -33,6 +33,7 @@ static struct pack_list { struct packed_git *pack; struct llist *unique_objects; struct llist *remaining_objects; + size_t all_objects_size; } *local_packs = NULL, *altodb_packs = NULL; static struct llist_item *free_nodes; @@ -340,19 +341,25 @@ static inline off_t pack_set_bytecount(struct pack_list *pl) return ret; } -static int cmp_pack_list_reverse(const void *a, const void *b) +static int cmp_remaining_objects(const void *a, const void *b) { struct pack_list *pl_a = *((struct pack_list **)a); struct pack_list *pl_b = *((struct pack_list **)b); - size_t sz_a = pl_a->remaining_objects->size; - size_t sz_b = pl_b->remaining_objects->size; - if (sz_a == sz_b) - return 0; - else if (sz_a < sz_b) + if (pl_a->remaining_objects->size == pl_b->remaining_objects->size) { + /* have the same remaining_objects, big pack first */ + if (pl_a->all_objects_size == pl_b->all_objects_size) + return 0; + else if (pl_a->all_objects_size < pl_b->all_objects_size) + return 1; + else + return -1; + } else if (pl_a->remaining_objects->size < pl_b->remaining_objects->size) { + /* sort by remaining objects, more objects first */ return 1; - else + } else { return -1; + } } /* Sort pack_list, greater size of remaining_objects first */ @@ -370,7 +377,7 @@ static void sort_pack_list(struct pack_list **pl) for (n = 0, p = *pl; p; p = p->next) ary[n++] = p; - QSORT(ary, n, cmp_pack_list_reverse); + QSORT(ary, n, cmp_remaining_objects); /* link them back again */ for (i = 0; i < n - 1; i++) @@ -511,6 +518,7 @@ static struct pack_list * add_pack(struct packed_git *p) llist_insert_back(l.remaining_objects, (const struct object_id *)(base + off)); off += step; } + l.all_objects_size = l.remaining_objects->size; l.unique_objects = NULL; if (p->pack_local) return pack_list_insert(&local_packs, &l); diff --git a/t/t5323-pack-redundant.sh b/t/t5323-pack-redundant.sh index 897acaa365..e642240d71 100755 --- a/t/t5323-pack-redundant.sh +++ b/t/t5323-pack-redundant.sh @@ -210,19 +210,19 @@ test_expect_success 'create pack 4, 5' ' # | T A B C D E F G H I J K L M N O P Q R # ----+-------------------------------------- # P1 | x x x x x x x x -# P2* | ! ! ! ! ! ! ! -# P3 | x x x x x x +# P2 | x x x x x x x +# P3* | ! ! ! ! ! ! # P4 | x x x x x # P5 | x x x x # ----+-------------------------------------- # ALL | x x x x x x x x x x x x x x x x x x # ############################################################################# -test_expect_failure 'one of pack-2/pack-3 is redundant (failed on Mac)' ' +test_expect_success 'one of pack-2/pack-3 is redundant' ' ( cd "$master_repo" && cat >expect <<-EOF && - P2:$P2 + P3:$P3 EOF git pack-redundant --all >out && format_packfiles actual && @@ -250,7 +250,7 @@ test_expect_success 'create pack 6, 7' ' # ALL | x x x x x x x x x x x x x x x x x x x # ############################################################################# -test_expect_failure 'pack 2, 4, and 6 are redundant (failed on Mac)' ' +test_expect_success 'pack 2, 4, and 6 are redundant' ' ( cd "$master_repo" && cat >expect <<-EOF && @@ -285,7 +285,7 @@ test_expect_success 'create pack 8' ' # ALL | x x x x x x x x x x x x x x x x x x x # ############################################################################# -test_expect_failure 'pack-8 (subset of pack-1) is also redundant (failed on Mac)' ' +test_expect_success 'pack-8 (subset of pack-1) is also redundant' ' ( cd "$master_repo" && cat >expect <<-EOF && @@ -309,7 +309,7 @@ test_expect_success 'clean loose objects' ' ) ' -test_expect_failure 'remove redundant packs and pass fsck (failed on Mac)' ' +test_expect_success 'remove redundant packs and pass fsck' ' ( cd "$master_repo" && git pack-redundant --all | xargs rm && @@ -329,7 +329,7 @@ test_expect_success 'setup shared.git' ' ) ' -test_expect_failure 'no redundant packs without --alt-odb (failed on Mac)' ' +test_expect_success 'no redundant packs without --alt-odb' ' ( cd "$shared_repo" && git pack-redundant --all >out && @@ -362,7 +362,7 @@ test_expect_failure 'no redundant packs without --alt-odb (failed on Mac)' ' # ALL | x x x x x x x x x x x x x x x x x x x # ############################################################################# -test_expect_failure 'pack-redundant --verbose: show duplicate packs in stderr (failed on Mac)' ' +test_expect_success 'pack-redundant --verbose: show duplicate packs in stderr' ' ( cd "$shared_repo" && cat >expect <<-EOF &&