From patchwork Mon Dec 10 16:42:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 10721743 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66A4915A6 for ; Mon, 10 Dec 2018 16:42:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 52B5E2AA6F for ; Mon, 10 Dec 2018 16:42:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 43D372AA82; Mon, 10 Dec 2018 16:42:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98B2A2AA36 for ; Mon, 10 Dec 2018 16:42:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727126AbeLJQmr (ORCPT ); Mon, 10 Dec 2018 11:42:47 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:40884 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726324AbeLJQmq (ORCPT ); Mon, 10 Dec 2018 11:42:46 -0500 Received: by mail-pl1-f194.google.com with SMTP id u18so5487979plq.7 for ; Mon, 10 Dec 2018 08:42:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=g4hO2A6WhnkK+0Xu4O9k7rPrftRPwAkIN8AY9QiikNk=; b=G0m7bSEkWJoIoqRySKzN0ZeRHWGmSwsn+kcrmHshhl9poWcGKr375aYJuSYXDAbEIL AjDlldA89m0P+7o8CeeeXVZf4qtQj53IVUc4rJrasFV/8enyzSGVJxLTXufE8JO4GEOO 6zBBc5Lecp/zpwcg5DTQ8eTuxVOAujFrEFa/5Nuhd1fDHqrzVEtebpbI7/6s79iZwzBA xSFAZXq8YI43iFsaSMofNUmHtMNBITHC7TVa+DVomt4pEN5inUY6fkOF/i2TqcCkrKnI xM92sYWFk+AvM/How/MifciCcBbQe9GnHY/FuiZsWr3v/C7qtd9evhjXY6h8hdcbvoLF 8fHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=g4hO2A6WhnkK+0Xu4O9k7rPrftRPwAkIN8AY9QiikNk=; b=X4v35688+JevS600UMv9W4bRnyC6Ari6lONCRiQTmzL3V6Zz54ovPLLotcOdFVvLUm +kzdV7HHlIWh0wURFfpDik4jr9Zp/ku7d89EuPqnLBWYxPnn+QX0vuK/qYJP7OpplGaU PApKGMt8kmRgmG5lV4n6vcj3iCwamMMNIOWsLK1Lq1Ql8eVuHcwsVJtG0FYL97PoDnWG qxjcgAB/6VU79qQC751gpamUeyHqzWS/SGvRX1CPXyMpOw2zo32DdvxkOmdl5AoWLw1E FIZtUV3ljmGaviJptro4EGWAnoR5CtX//m/xUmI2hBAQKZOh9ClQh9NGNIiXbnulJ5dj +bAQ== X-Gm-Message-State: AA+aEWaPm/SofiGJh2uGgGJirghsDDDeF/Sc45DmckeQYXF2JaF1AJSB M6TkKrEBNTFzzs06lMhtzkxqgfGk X-Google-Smtp-Source: AFSGD/XjXJ4tZl8bv82CGtzXmuZbhyyZnwkf7s+4GswKx3IXC1Ggww7NZcMVRRwaUWIFZMvgKqJXFg== X-Received: by 2002:a17:902:6f09:: with SMTP id w9mr13040444plk.309.1544460164986; Mon, 10 Dec 2018 08:42:44 -0800 (PST) Received: from [127.0.0.1] ([40.112.139.188]) by smtp.gmail.com with ESMTPSA id e23sm17973566pfh.68.2018.12.10.08.42.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Dec 2018 08:42:44 -0800 (PST) Date: Mon, 10 Dec 2018 08:42:44 -0800 (PST) X-Google-Original-Date: Mon, 10 Dec 2018 16:42:35 GMT Message-Id: <4ef318bdb2f1f7c55130f875280604e816cc183f.1544460159.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v3 3/6] pack-objects: add --sparse option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: peff@peff.net, avarab@gmail.com, jrnieder@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee Add a '--sparse' option flag to the pack-objects builtin. This allows the user to specify that they want to use the new logic for walking trees. This logic currently does not differ from the existing output, but will in a later change. Create a new test script, t5322-pack-objects-sparse.sh, to ensure the object list that is selected matches what we expect. When we update the logic to walk in a sparse fashion, the final test will be updated to show the extra objects that are added. Signed-off-by: Derrick Stolee --- Documentation/git-pack-objects.txt | 11 ++- builtin/pack-objects.c | 5 +- t/t5322-pack-objects-sparse.sh | 115 +++++++++++++++++++++++++++++ 3 files changed, 129 insertions(+), 2 deletions(-) create mode 100755 t/t5322-pack-objects-sparse.sh diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt index 40c825c381..e45f3e680d 100644 --- a/Documentation/git-pack-objects.txt +++ b/Documentation/git-pack-objects.txt @@ -14,7 +14,7 @@ SYNOPSIS [--local] [--incremental] [--window=] [--depth=] [--revs [--unpacked | --all]] [--keep-pack=] [--stdout [--filter=] | base-name] - [--shallow] [--keep-true-parents] < object-list + [--shallow] [--keep-true-parents] [--sparse] < object-list DESCRIPTION @@ -196,6 +196,15 @@ depth is 4095. Add --no-reuse-object if you want to force a uniform compression level on all data no matter the source. +--sparse:: + Use the "sparse" algorithm to determine which objects to include in + the pack, when combined with the "--revs" option. This algorithm + only walks trees that appear in paths that introduce new objects. + This can have significant performance benefits when computing + a pack to send a small change. However, it is possible that extra + objects are added to the pack-file if the included commits contain + certain types of direct renames. + --thin:: Create a "thin" pack by omitting the common objects between a sender and a receiver in order to reduce network transfer. This diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 5f70d840a7..7d5b0735e3 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -84,6 +84,7 @@ static unsigned long pack_size_limit; static int depth = 50; static int delta_search_threads; static int pack_to_stdout; +static int sparse; static int thin; static int num_preferred_base; static struct progress *progress_state; @@ -3135,7 +3136,7 @@ static void get_object_list(int ac, const char **av) if (prepare_revision_walk(&revs)) die(_("revision walk setup failed")); - mark_edges_uninteresting(&revs, show_edge, 0); + mark_edges_uninteresting(&revs, show_edge, sparse); if (!fn_show_object) fn_show_object = show_object; @@ -3292,6 +3293,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) { OPTION_CALLBACK, 0, "unpack-unreachable", NULL, N_("time"), N_("unpack unreachable objects newer than