From patchwork Fri Sep 21 22:35:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10611121 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0CA3514DA for ; Fri, 21 Sep 2018 22:36:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F03FC2D796 for ; Fri, 21 Sep 2018 22:36:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E2EBD2D7AD; Fri, 21 Sep 2018 22:36:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 933592D796 for ; Fri, 21 Sep 2018 22:36:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726376AbeIVE1O (ORCPT ); Sat, 22 Sep 2018 00:27:14 -0400 Received: from mail-yw1-f73.google.com ([209.85.161.73]:44891 "EHLO mail-yw1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725837AbeIVE1N (ORCPT ); Sat, 22 Sep 2018 00:27:13 -0400 Received: by mail-yw1-f73.google.com with SMTP id w23-v6so6555846ywg.11 for ; Fri, 21 Sep 2018 15:36:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=cE3W+XwBM7hWw/AFGsDXDmScj4IvN7B5gFDXK16U1g8=; b=O5qqJC4vF39UAyoAEK/2ssdpXh0ZKbAwTy7BG7jixq9bcRj7L38qjpG4vM80vWwa7W h+m5/771LMDko5wIU9HSfEuez7d1nR4oqEPXx63RdOQLOlE6rBulAFqAMo4t1N1xXVTV Qmua85B0smhlw9pmzVr9dr+ZdHqlYfFjq797r50Dy26v/CzmPz0+PGZhj5gc2dOSHNTP uo0woA7Bqsl0o7gNPQZj7ig3fXLKErDQHL9TVPQarNENWDUa5N3jM5H/wJ34o5RBuBGU G1tlxySbDo4VSJFBOhOxrc1DItFBKJoYmcD5ClIpLGTB8tvNsX6dPmbNKcRp46ICrpK5 1+3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=cE3W+XwBM7hWw/AFGsDXDmScj4IvN7B5gFDXK16U1g8=; b=S7Zx4UsMFHqZ7qopq0ZdtiW24whz3cTXbKFk+KRTav7rSJSopbubkiR0xAkbBWB+jP Ox3u4vKFtJDXkjYP0b+zBs5FBgCS0pNzVSINxG/ygsC/OitRnwKPZ9AvbWxPDsH6SRUd 085bqn8c39SCrzcnf6AEBSLKuu4MDvoqWWSQFI6joMu+6St0bBcwNa9blPF31RNZaRv2 sX7bnQs+8quGNZaQK7k1C6RE+P0IHwAH2oI9F03wvYW6mDp34HQSuONdNT+u2IdxR6Rx /qwZ0pKcYbzz265ah2mDGrorCcPpXR6zUzSxemYgDTucnZORI3j9LT28q5DXToz/pGEs 7KGQ== X-Gm-Message-State: APzg51Bnm+IGdbnS6KnrMYlOe953zyZ5ykkxxjNa4oOuaxT0RCGMDi1U srJ6Y0kc2PPPVocStRsikQ+k1uSR3E/9DV3wW1p4b49vY+gwOThhfjowXxJUGo9gqnXMGM11L8U MaMsntk3fNat+r5RICEaDK7Jfb0ZTEgCPXWwZnTy60kRPD7Pijs3yyEQUYXOe X-Google-Smtp-Source: ANB0VdbBbc/FxwkwBXNfAfrrfj86Qcb4Zu/MNZHlVSU9gi2UtgJ7QQ0Zifi0n2xQ6nriVBf8EfVXtH0eHwzj X-Received: by 2002:a81:8185:: with SMTP id r127-v6mr5543502ywf.147.1537569377580; Fri, 21 Sep 2018 15:36:17 -0700 (PDT) Date: Fri, 21 Sep 2018 15:35:51 -0700 In-Reply-To: <20180921223558.65055-1-sbeller@google.com> Message-Id: <20180921223558.65055-2-sbeller@google.com> Mime-Version: 1.0 References: <20180921223558.65055-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 1/8] sha1-array: provide oid_array_filter From: Stefan Beller To: git@vger.kernel.org Cc: Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Helped-by: Junio C Hamano Signed-off-by: Stefan Beller --- sha1-array.c | 17 +++++++++++++++++ sha1-array.h | 9 +++++++++ 2 files changed, 26 insertions(+) diff --git a/sha1-array.c b/sha1-array.c index b94e0ec0f5e..d922e94e3fc 100644 --- a/sha1-array.c +++ b/sha1-array.c @@ -77,3 +77,20 @@ int oid_array_for_each_unique(struct oid_array *array, } return 0; } + +void oid_array_filter(struct oid_array *array, + for_each_oid_fn want, + void *cb_data) +{ + unsigned nr = array->nr, src, dst; + struct object_id *oids = array->oid; + + for (src = dst = 0; src < nr; src++) { + if (want(&oids[src], cb_data)) { + if (src != dst) + oidcpy(&oids[dst], &oids[src]); + dst++; + } + } + array->nr = dst; +} diff --git a/sha1-array.h b/sha1-array.h index 232bf950172..275e5b02f5e 100644 --- a/sha1-array.h +++ b/sha1-array.h @@ -23,4 +23,13 @@ int oid_array_for_each_unique(struct oid_array *array, for_each_oid_fn fn, void *data); +/* + * Apply want to each entry in array, retaining only the entries for + * which the function returns true. Preserve the order of the entries + * that are retained. + */ +void oid_array_filter(struct oid_array *array, + for_each_oid_fn want, + void *cbdata); + #endif /* SHA1_ARRAY_H */ From patchwork Fri Sep 21 22:35:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10611123 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35F14112B for ; Fri, 21 Sep 2018 22:36:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 284D42D796 for ; Fri, 21 Sep 2018 22:36:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1B7FD2D7AD; Fri, 21 Sep 2018 22:36:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 894132D796 for ; Fri, 21 Sep 2018 22:36:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727061AbeIVE1W (ORCPT ); Sat, 22 Sep 2018 00:27:22 -0400 Received: from mail-qk1-f201.google.com ([209.85.222.201]:43128 "EHLO mail-qk1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725837AbeIVE1V (ORCPT ); Sat, 22 Sep 2018 00:27:21 -0400 Received: by mail-qk1-f201.google.com with SMTP id u22-v6so14332275qkk.10 for ; Fri, 21 Sep 2018 15:36:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=T2eIYO+W3QXTcdtue+CtselyEgFKQY1rp/XH4muMrNI=; b=lKHfUJuhpyZULvbC90NihFiv1/lrg/N4wJ7cuRrdAYWaK8rF1U72A10ZQXB0tXHz/m IRw+rKOavMMDEf8AQ6+ZHFPB9JZ6Voygsp81HzxI8L+h06wmESni48DY/Ly0D5vqNQHh TIOILJhrQ920WIQettsxpNmDD/IC7YNfnBOYivJnl5Q+tJ/E8tVx5kaZcBY6M+yOvHY7 XANcwkok173zmTAneNEfXbc7xBiLuQ7RHS/C0h+7eP0T4dsT2gh4tGgiEEhDNvXtdWL/ gAdIdcBn+fgtrYS/aG/vvxU7DHLWY21yLYB4d1Ajq/HIpTLyI4B9e1r7xk0scXurMTkK MLCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=T2eIYO+W3QXTcdtue+CtselyEgFKQY1rp/XH4muMrNI=; b=iCp01D8GhHCxGt7rGTfr0e+oPtrMrdZJq74e7l2RqisUwOq5u9nQZndPVggcFFTqYB 62zTS7EOPtcSgGKP09mSKSCGPA1u3yYJ4wNo21mzjZAvr61LEL1W5zGkkXx1iD4VutT8 weCSrZEaaNAxQP/PHkB5nomfGiI9D5GIaFD5oSQfTRA7Uj9EBG4/T1hrFuaIwRqf6sgJ diPAulEzD6UA3pcLORbmhgcNdOOexSSsKTZ7uHBmmPUg8b7vEiguG24LltGFisnVXRes 5EZi1PUrK2O9mAIF+3JCRkdxxI6yRc8bF8HiG+pyYb9+N6msl3bLCaCTNHd3NjbBDlE1 ezHA== X-Gm-Message-State: APzg51CvGsEE9i1JV1EClwC5ZLziYwKeSJEHm5BttWPtmkTcgVZTZwjQ dfYjbjT+89/dZr8h4kVU4kiFB/KeSUZZEimRkVPNDYW8GTlU+JvZkwULP2Lh/WFipY8IHtw9Ohz w6kpuTV9PsSzEETo7drj6dV3C2BpnbC0dw2lcbzTzCwJSyeq9xN+eY5pbEgIE X-Google-Smtp-Source: ANB0VdbmKp9NnzIVV9GHNTIr1oRPvTXJ/zF5VBJz2hUtib+RdcCdrWvD1daQ5j3dIZwxA7IqAjZWk7xv9ktd X-Received: by 2002:aed:24ab:: with SMTP id t40-v6mr14844245qtc.25.1537569385598; Fri, 21 Sep 2018 15:36:25 -0700 (PDT) Date: Fri, 21 Sep 2018 15:35:52 -0700 In-Reply-To: <20180921223558.65055-1-sbeller@google.com> Message-Id: <20180921223558.65055-3-sbeller@google.com> Mime-Version: 1.0 References: <20180921223558.65055-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 2/8] submodule.c: fix indentation From: Stefan Beller To: git@vger.kernel.org Cc: Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The submodule subsystem is really bad at staying within 80 characters. Fix it while we are here. Signed-off-by: Stefan Beller --- submodule.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/submodule.c b/submodule.c index ed05339b588..67469a8f513 100644 --- a/submodule.c +++ b/submodule.c @@ -1245,7 +1245,8 @@ static int get_next_submodule(struct child_process *cp, if (!submodule) { const char *name = default_name_or_path(ce->name); if (name) { - default_submodule.path = default_submodule.name = name; + default_submodule.path = name; + default_submodule.name = name; submodule = &default_submodule; } } @@ -1255,8 +1256,10 @@ static int get_next_submodule(struct child_process *cp, default: case RECURSE_SUBMODULES_DEFAULT: case RECURSE_SUBMODULES_ON_DEMAND: - if (!submodule || !unsorted_string_list_lookup(&changed_submodule_names, - submodule->name)) + if (!submodule || + !unsorted_string_list_lookup( + &changed_submodule_names, + submodule->name)) continue; default_argv = "on-demand"; break; From patchwork Fri Sep 21 22:35:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10611127 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1607414DA for ; Fri, 21 Sep 2018 22:36:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0891F2D796 for ; Fri, 21 Sep 2018 22:36:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F13B72D7AD; Fri, 21 Sep 2018 22:36:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A2F972D796 for ; Fri, 21 Sep 2018 22:36:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727169AbeIVE1a (ORCPT ); Sat, 22 Sep 2018 00:27:30 -0400 Received: from mail-io1-f73.google.com ([209.85.166.73]:54335 "EHLO mail-io1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725837AbeIVE1a (ORCPT ); Sat, 22 Sep 2018 00:27:30 -0400 Received: by mail-io1-f73.google.com with SMTP id l24-v6so25715487iok.21 for ; Fri, 21 Sep 2018 15:36:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=1V9VDzgdE3qRBsghEtQGr9pDi3zbWh6RGSxhIhwaeWg=; b=TkWtgd6dbUzglv5Ud7UQ588UGdV0OCAsPihSPJ6iNoem9NBhv9aWzyF8EGaauXNYcV FUfCzcpV9i4zu0OTSLr13ks6WLRWfxazakMGUD9p6bNvBf4fMZuqPOXIVayyjJjBZ1FY Za/fUmnnFiGhBU/pCNp+yC607nuwOdf4M4Wy8Z3LXV4wlAq/TGV23J1/KQOrs905fY3G gUHB+EDd8Pe2uNUuQqXO2gVgV1Eu6qLpiC0FedOolAYE9DgtVfpzvqVWs8xFjJJZJdJi dOfaO3rioDnRQgVTvnDwQgcU9AvlgLCmI6xL8BY43JcyW10GJKDsIQjCqXR3K+gJMkB+ Onig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=1V9VDzgdE3qRBsghEtQGr9pDi3zbWh6RGSxhIhwaeWg=; b=n02loM2yqayf1zD188tCEKylxDujrz45gMohlocnOXTu7/klvcca8Cpn7DCk6LIg2f BmNmkgzn51hD9LVnUgahni8Xw8TdtcrW4cLHl+EqXsi9Bt/A0bBm7dZSavFqPswyeXjf xOuJrr3FN8vki/eFq/iIRNgx5OypLDjW7em4jqK9Hk+Q80RKQGeOMwlD5FE1TJt0moev ZVXVPwFNnEzOmpP+BRFTF3VRvR0UKxaj5CtAtDeE4t3IYanXHZ8gUMrNE9UhXVHqRsLL 3ATTyO45bVqVvBKkXUQZ44fnRruhcINyqgQ54RyCmDZhXHVuXcKit/DIKeqcMhBfi5ZK gy0w== X-Gm-Message-State: APzg51D+EQZP/X6gNji1Ws8Y8dtyLbEx2wONQzJN29rHT7gJTjk/hvXF be8i2+74cXwXpxKOF//MeNSDUSDwvR9eAcCzg8wseSyTOih5NEGlmVkM5cMn1Krypn9XNN/bLWY ywUq9ntYv76zlEJb+6WI63p0ksQ+aY8imwOYvrRMRTJsMoiRo73qNTyUlE7Si X-Google-Smtp-Source: ANB0VdYiPmpol44zIuOxy7Y7HbsUyPKMDqWixr3CMkgDFDBrGtOYrFGiwroxFKRWrsrWEkHm4Qfn/h462g6x X-Received: by 2002:a6b:da12:: with SMTP id x18-v6mr36049221iob.28.1537569393653; Fri, 21 Sep 2018 15:36:33 -0700 (PDT) Date: Fri, 21 Sep 2018 15:35:53 -0700 In-Reply-To: <20180921223558.65055-1-sbeller@google.com> Message-Id: <20180921223558.65055-4-sbeller@google.com> Mime-Version: 1.0 References: <20180921223558.65055-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 3/8] submodule.c: sort changed_submodule_names before searching it From: Stefan Beller To: git@vger.kernel.org Cc: Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We can string_list_insert() to maintain sorted-ness of the list as we find new items, or we can string_list_append() to build an unsorted list and sort it at the end just once. To pick which one is more appropriate, we notice the fact that we discover new items more or less in the already sorted order. That makes "append then sort" more appropriate. Signed-off-by: Stefan Beller --- submodule.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/submodule.c b/submodule.c index 67469a8f513..2f5c19d0aac 100644 --- a/submodule.c +++ b/submodule.c @@ -1257,7 +1257,7 @@ static int get_next_submodule(struct child_process *cp, case RECURSE_SUBMODULES_DEFAULT: case RECURSE_SUBMODULES_ON_DEMAND: if (!submodule || - !unsorted_string_list_lookup( + !string_list_lookup( &changed_submodule_names, submodule->name)) continue; @@ -1351,6 +1351,7 @@ int fetch_populated_submodules(struct repository *r, /* default value, "--submodule-prefix" and its value are added later */ calculate_changed_submodule_paths(); + string_list_sort(&changed_submodule_names); run_processes_parallel(max_parallel_jobs, get_next_submodule, fetch_start_failure, From patchwork Fri Sep 21 22:35:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10611129 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0583814DA for ; Fri, 21 Sep 2018 22:36:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E9EFF2D796 for ; Fri, 21 Sep 2018 22:36:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC6B12D7AD; Fri, 21 Sep 2018 22:36:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78F0C2D796 for ; Fri, 21 Sep 2018 22:36:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727273AbeIVE1j (ORCPT ); Sat, 22 Sep 2018 00:27:39 -0400 Received: from mail-yw1-f73.google.com ([209.85.161.73]:57301 "EHLO mail-yw1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725837AbeIVE1i (ORCPT ); Sat, 22 Sep 2018 00:27:38 -0400 Received: by mail-yw1-f73.google.com with SMTP id v144-v6so6503221ywa.23 for ; Fri, 21 Sep 2018 15:36:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=wHEd51gby15KfcydoxWBo+N4NGlmZgwD+vuLVkHalLM=; b=S9LICF+5jxVRszt/Hqh8JrBc50jXrXAW32k5k0oTLiT2+9Q0AGs3Yi32zulbFfXbJw XCmAT23hRz56/GUmv7mXEfq5/j5bVTb96+K1nnRwI9HuzRlX0CkwdevAXir8cBhRFM+M JW4a8+Zu9imsBDG5VVOY754HjuOKd6fRLg188mJVlC0jY63i8NWoRtzxKVpGdmcgXXMd MgHPlyOFlHfKbX8n+U8goTAgVXmHa9WkDC+AXv5VmylX6Egzh2UdNcLYnF6+XSfLBIXd z3+uMs0z+AdaAh5tdhhxf6Br+IQ6QXFi0ipwVZ32Vv+UwNxNC+O6Z3TiY/gi5L6NuU4J 5ApA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=wHEd51gby15KfcydoxWBo+N4NGlmZgwD+vuLVkHalLM=; b=MTWGeSAlzx4N9vR9K6Njv+brBEYnZz96fNoNVoyFsWUOp1rOa3qoC/NkT+gn3HVWZH p4sFbVYGZmNKvCEy0bDzjmT1lxcxAEFaOYeR+y3BX749N6eXmCnhXnfW0GQN4TEExzt6 vhAAOX6AFWG2rvImil2tHY8ClVT3ZfZHKV47twKwcEe8hGwG7wJRFXeM+aGYhBDsYd/D 358MH961KbE4xbW7OOPpE36yCeYnq6KBSyJNlB/00Pdh3XV7M4pgqNauB49EECHxiJjQ Y4mzRUNrox4pQUzkpG1JuykxqwdgqkSlpGcghenflvUgzJqycGEBNDs78AsIXzGv1ylZ aNVA== X-Gm-Message-State: APzg51CwiFK0Iu0ho6nBQq8DQh3GqlA+Wcls3p8J5zk6iw5BWYyFHT5R 1sOz7X+EKIRo+2wCFxAxkTukHf94JsHXK2vysYbYDekjhziSmEHlVZFkoY8zaszD6m0+mWLw0MI reNu2jiMJvzaaPOnsyB4vTNg4N820Agvn+SsfBy/ygf+vxesx2Fw5vcf0avdl X-Google-Smtp-Source: ANB0VdZgfhVmK2B399khe5l5RXTtbv5hjXsO61Nq70IO0m1tFFmaiEfP2l5lR2knxiIn51NlWBl388ffaM+T X-Received: by 2002:a81:2988:: with SMTP id p130-v6mr4107697ywp.25.1537569402073; Fri, 21 Sep 2018 15:36:42 -0700 (PDT) Date: Fri, 21 Sep 2018 15:35:54 -0700 In-Reply-To: <20180921223558.65055-1-sbeller@google.com> Message-Id: <20180921223558.65055-5-sbeller@google.com> Mime-Version: 1.0 References: <20180921223558.65055-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 4/8] submodule: move global changed_submodule_names into fetch submodule struct From: Stefan Beller To: git@vger.kernel.org Cc: Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The `changed_submodule_names` are only used for fetching, so let's make it part of the struct that is passed around for fetching submodules. Signed-off-by: Stefan Beller --- submodule.c | 42 +++++++++++++++++++++++------------------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/submodule.c b/submodule.c index 2f5c19d0aac..7b4d136849b 100644 --- a/submodule.c +++ b/submodule.c @@ -25,7 +25,7 @@ #include "commit-reach.h" static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF; -static struct string_list changed_submodule_names = STRING_LIST_INIT_DUP; + static int initialized_fetch_ref_tips; static struct oid_array ref_tips_before_fetch; static struct oid_array ref_tips_after_fetch; @@ -1111,7 +1111,22 @@ void check_for_new_submodule_commits(struct object_id *oid) oid_array_append(&ref_tips_after_fetch, oid); } -static void calculate_changed_submodule_paths(void) +struct submodule_parallel_fetch { + int count; + struct argv_array args; + struct repository *r; + const char *prefix; + int command_line_option; + int default_option; + int quiet; + int result; + + struct string_list changed_submodule_names; +}; +#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, STRING_LIST_INIT_DUP } + +static void calculate_changed_submodule_paths( + struct submodule_parallel_fetch *spf) { struct argv_array argv = ARGV_ARRAY_INIT; struct string_list changed_submodules = STRING_LIST_INIT_DUP; @@ -1149,7 +1164,8 @@ static void calculate_changed_submodule_paths(void) continue; if (!submodule_has_commits(path, commits)) - string_list_append(&changed_submodule_names, name->string); + string_list_append(&spf->changed_submodule_names, + name->string); } free_submodules_oids(&changed_submodules); @@ -1186,18 +1202,6 @@ int submodule_touches_in_range(struct object_id *excl_oid, return ret; } -struct submodule_parallel_fetch { - int count; - struct argv_array args; - struct repository *r; - const char *prefix; - int command_line_option; - int default_option; - int quiet; - int result; -}; -#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0} - static int get_fetch_recurse_config(const struct submodule *submodule, struct submodule_parallel_fetch *spf) { @@ -1258,7 +1262,7 @@ static int get_next_submodule(struct child_process *cp, case RECURSE_SUBMODULES_ON_DEMAND: if (!submodule || !string_list_lookup( - &changed_submodule_names, + &spf->changed_submodule_names, submodule->name)) continue; default_argv = "on-demand"; @@ -1350,8 +1354,8 @@ int fetch_populated_submodules(struct repository *r, argv_array_push(&spf.args, "--recurse-submodules-default"); /* default value, "--submodule-prefix" and its value are added later */ - calculate_changed_submodule_paths(); - string_list_sort(&changed_submodule_names); + calculate_changed_submodule_paths(&spf); + string_list_sort(&spf.changed_submodule_names); run_processes_parallel(max_parallel_jobs, get_next_submodule, fetch_start_failure, @@ -1360,7 +1364,7 @@ int fetch_populated_submodules(struct repository *r, argv_array_clear(&spf.args); out: - string_list_clear(&changed_submodule_names, 1); + string_list_clear(&spf.changed_submodule_names, 1); return spf.result; } From patchwork Fri Sep 21 22:35:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10611131 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 707A9112B for ; Fri, 21 Sep 2018 22:36:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F4202D796 for ; Fri, 21 Sep 2018 22:36:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 537CA2D7AD; Fri, 21 Sep 2018 22:36:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB6392D796 for ; Fri, 21 Sep 2018 22:36:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726218AbeIVE1o (ORCPT ); Sat, 22 Sep 2018 00:27:44 -0400 Received: from mail-yb1-f201.google.com ([209.85.219.201]:39011 "EHLO mail-yb1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725837AbeIVE1o (ORCPT ); Sat, 22 Sep 2018 00:27:44 -0400 Received: by mail-yb1-f201.google.com with SMTP id g139-v6so6417756ybf.6 for ; Fri, 21 Sep 2018 15:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PasKoMTvPQ5hM57cqvFIxQWPIYaVNQFbxKIOgHG+mYk=; b=AFxI6sRvPMhbGez8+d38vAgqQOFUy5qy3u2rhBCTH0lFG+cY+ovHbAXN5bA9PRZZ4A OqwsXkLsbpM5CEt69Z64enfWN8I/lEhQLn07M73Mwl2XsaGKr/qA0ZFor1/f0WKzUPWe HkrniJOj3vahY8C6Sq3ukTtbqtS9MWo3BbnNj+WpyXtWtcumClv3HxfPP/JSdTEX9088 KyLj1JqWbtVSYw1jJfenqbmBbE1kZuvFfxaAMAVea2bfOMmEjVy5arYr66RtIi2PyFnd ZjrBtn/EBuzrcSYAudTNXOstqYLrgneQZsWLbCaHKEz4eMMtKLh0OChXd0EFWB+ZWAmB /G1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PasKoMTvPQ5hM57cqvFIxQWPIYaVNQFbxKIOgHG+mYk=; b=uSSYml9GGs978lJRgWQYK8GxdWe+ieeWkIo36NRpRoVAhqQ8d4eowgPsI6ZbivdKVx jwaF2kEiHZSFVJ3fGiCQlKfs1ClZvt0EmxcElpasb594ca6vknu9Ob0Nvi/2yTtYsXjO vJHDEpGEtKNGWGZvF4rqPyc8psDVtu5veA4J61d7NAAfV3CnMPKrJxG2MPTxligbViEp uC/DVINnEM3jFyNQ0br4ksHUxo4hfLvhnnZMx7vDno4wodeMP5pPWBhF2Jqg7oHWQQ66 wtpYjWozQOeDYeuFae8wcirbQgXlo2Lg7ffc0MjkIDfC8+qqojSdpKdRosBuqtk/0khq vQsg== X-Gm-Message-State: APzg51D/W9TDiA8WWgTjMA+2y17aOwuHVq4vW2nYVc7G01Vw1Rxzi7oJ 6ajNpZ+wJNe3NjS3YfkErZl20g8I92w5vGopEL+527l/0ZCOreK+2w7m/WwqsDLAKLStPKmifxJ syDqBaId5Ac0FZPPVX4tgzUSyBT5nmS8k86QfOYpENubxAoV4Qk5y1S32WdLR X-Google-Smtp-Source: ANB0VdZ7y6N3kLHH7OWMkuj1Mz4+Htb088YVHMxK6eckCdbEmIvYuFOQzADdMCC8Kwzi7HWnTlQep/eh+Abp X-Received: by 2002:a25:8812:: with SMTP id c18-v6mr3166925ybl.71.1537569407753; Fri, 21 Sep 2018 15:36:47 -0700 (PDT) Date: Fri, 21 Sep 2018 15:35:55 -0700 In-Reply-To: <20180921223558.65055-1-sbeller@google.com> Message-Id: <20180921223558.65055-6-sbeller@google.com> Mime-Version: 1.0 References: <20180921223558.65055-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 5/8] submodule.c: do not copy around submodule list From: Stefan Beller To: git@vger.kernel.org Cc: Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP 'calculate_changed_submodule_paths' uses a local list to compute the changed submodules, and then produces the result by copying appropriate items into the result list. Instead use the result list directly and prune items afterwards using string_list_remove_empty_items. By doing so we'll have access to the util pointer for longer that contains the commits that we need to fetch, which will be useful in a later patch. Signed-off-by: Stefan Beller --- submodule.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/submodule.c b/submodule.c index 7b4d136849b..209680ec076 100644 --- a/submodule.c +++ b/submodule.c @@ -1129,8 +1129,7 @@ static void calculate_changed_submodule_paths( struct submodule_parallel_fetch *spf) { struct argv_array argv = ARGV_ARRAY_INIT; - struct string_list changed_submodules = STRING_LIST_INIT_DUP; - const struct string_list_item *name; + struct string_list_item *name; /* No need to check if there are no submodules configured */ if (!submodule_from_path(the_repository, NULL, NULL)) @@ -1147,9 +1146,9 @@ static void calculate_changed_submodule_paths( * Collect all submodules (whether checked out or not) for which new * commits have been recorded upstream in "changed_submodule_names". */ - collect_changed_submodules(&changed_submodules, &argv); + collect_changed_submodules(&spf->changed_submodule_names, &argv); - for_each_string_list_item(name, &changed_submodules) { + for_each_string_list_item(name, &spf->changed_submodule_names) { struct oid_array *commits = name->util; const struct submodule *submodule; const char *path = NULL; @@ -1163,12 +1162,14 @@ static void calculate_changed_submodule_paths( if (!path) continue; - if (!submodule_has_commits(path, commits)) - string_list_append(&spf->changed_submodule_names, - name->string); + if (submodule_has_commits(path, commits)) { + oid_array_clear(commits); + *name->string = '\0'; + } } - free_submodules_oids(&changed_submodules); + string_list_remove_empty_items(&spf->changed_submodule_names, 1); + argv_array_clear(&argv); oid_array_clear(&ref_tips_before_fetch); oid_array_clear(&ref_tips_after_fetch); @@ -1364,7 +1365,7 @@ int fetch_populated_submodules(struct repository *r, argv_array_clear(&spf.args); out: - string_list_clear(&spf.changed_submodule_names, 1); + free_submodules_oids(&spf.changed_submodule_names); return spf.result; } From patchwork Fri Sep 21 22:35:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10611133 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42DCA14DA for ; Fri, 21 Sep 2018 22:36:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 358192D796 for ; Fri, 21 Sep 2018 22:36:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2A0752D7AD; Fri, 21 Sep 2018 22:36:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A452F2D796 for ; Fri, 21 Sep 2018 22:36:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727624AbeIVE1r (ORCPT ); Sat, 22 Sep 2018 00:27:47 -0400 Received: from mail-qk1-f201.google.com ([209.85.222.201]:45940 "EHLO mail-qk1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725811AbeIVE1r (ORCPT ); Sat, 22 Sep 2018 00:27:47 -0400 Received: by mail-qk1-f201.google.com with SMTP id d194-v6so14237331qkb.12 for ; Fri, 21 Sep 2018 15:36:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=foxYFSc8HbnA0pX8j2OReCVPNgnCu5bvkqDoawxm18s=; b=T5WAELPlxu1KptWdkDjxP0g8ZY/UVurfTIAMdsl4PR8zu3IYm44nnc1HmooDLRWd1j ruNiqNCn3jzz+Rw6Q6GpF90cYdACk4J3I68jnnwLhdFvndCnAXNaKosdbdpJ5/K9EElu BW+8bqXE7CFRgEBedFWmOcsrzh0ISM/U0+EWJ6Hlv6YH9tY9YNQvvp9XUKi0rMtmRlEO rc/SECI/63h/XQkEh5c49Vtq+ChujR8x0tEhQQ0vGnIDykW5IN44z5HLJZ0R30ufVLWX jX3PqSSWlxuH/nuw5XIGvsTTGAwqHRGeVh2Pf4gPJ8nb1Lmu71N5Gn5VnZwDGKUgA94n 6nEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=foxYFSc8HbnA0pX8j2OReCVPNgnCu5bvkqDoawxm18s=; b=dCmGVgyrenSAvgWpFloET0eDk/xClua+WyY0l7cMOk4NDUOvvIoSr1ESwnElKYcwOm 3vrtbJcM/ZjwQj8syPuxSTkLYF4gXVcSxiWRx1Q/4k8ZVptBiSRK5GpSssNJRKCs6dWY V7AKhGi6Vy6DAf/1BPTiJ0NYCTkWHVoFbi317bBJN3VY+lSIzn2KaF+pKSCk1tAwK4yd tytyFZiO45HLXSIB5EDCMFS15OpkiuvohRH4NPWU88GX0+l1afJNXYCcy0CfwFQCYkSC +G4egw31SHocvOgtprwc7RKQAtZUDLBFgEw5ERb2VLvKoeYR/91yDe12yR8R6adWyA3I X6Vw== X-Gm-Message-State: ABuFfoitnaZ1EgpseraceZtWS7ZvfOPBPEK0IUeQjtWdiO6ajsQurSiH MtFJFxZMvBYLzD7+v4cn4zJ4qX2SJWYFzx3hYrEj1dW6KPfqH6LLSFFzM5UxPQQ02tHzzNu5WsU GHUsoG28Tu4dUfTd3SN4bibSutAW3jwdKPgnGpgHSlDD9NIUlGurSnWG70iyY X-Google-Smtp-Source: ACcGV63PzMkdpyzO7jilf6GaAMNMExGnXmS20gOI4Ebfk8jNzLGRVvWpPcZMk1k77+m93qP0DK4WCCI+U8vf X-Received: by 2002:a0c:d892:: with SMTP id q18-v6mr2022706qvj.43.1537569410146; Fri, 21 Sep 2018 15:36:50 -0700 (PDT) Date: Fri, 21 Sep 2018 15:35:56 -0700 In-Reply-To: <20180921223558.65055-1-sbeller@google.com> Message-Id: <20180921223558.65055-7-sbeller@google.com> Mime-Version: 1.0 References: <20180921223558.65055-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 6/8] submodule: fetch in submodules git directory instead of in worktree From: Stefan Beller To: git@vger.kernel.org Cc: Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch started as a refactoring to make 'get_next_submodule' more readable, but upon doing so, I realized that "git fetch" of the submodule actually doesn't need to be run in the submodules worktree. So let's run it in its git dir instead. That should pave the way towards fetching submodules that are currently not checked out. This patch leaks the cp->dir in get_next_submodule, as any further callback in run_processes_parallel doesn't have access to the child process any more. In an early iteration of this patch, the function get_submodule_repo_for directly returned the string containing the git directory, which would be a better design choice for this patch. However the next patch both fixes the memory leak of cp->dir and also has a use case for using the full repository handle of the submodule, so it makes sense to introduce the get_submodule_repo_for here already. Signed-off-by: Stefan Beller --- submodule.c | 47 +++++++++++++++++++++++++++---------- t/t5526-fetch-submodules.sh | 7 +++++- 2 files changed, 40 insertions(+), 14 deletions(-) diff --git a/submodule.c b/submodule.c index 209680ec076..8829e8d4eff 100644 --- a/submodule.c +++ b/submodule.c @@ -482,6 +482,12 @@ void prepare_submodule_repo_env(struct argv_array *out) DEFAULT_GIT_DIR_ENVIRONMENT); } +static void prepare_submodule_repo_env_in_gitdir(struct argv_array *out) +{ + prepare_submodule_repo_env_no_git_dir(out); + argv_array_pushf(out, "%s=.", GIT_DIR_ENVIRONMENT); +} + /* Helper function to display the submodule header line prior to the full * summary output. If it can locate the submodule objects directory it will * attempt to lookup both the left and right commits and put them into the @@ -1228,6 +1234,25 @@ static int get_fetch_recurse_config(const struct submodule *submodule, return spf->default_option; } +static struct repository *get_submodule_repo_for(struct repository *r, + const char *path) +{ + struct repository *ret = xmalloc(sizeof(*ret)); + + if (repo_submodule_init(ret, r, path)) { + /* no entry in .gitmodules? */ + struct strbuf gitdir = STRBUF_INIT; + strbuf_repo_worktree_path(&gitdir, r, "%s/.git", path); + if (repo_init(ret, gitdir.buf, NULL)) { + strbuf_release(&gitdir); + return NULL; + } + strbuf_release(&gitdir); + } + + return ret; +} + static int get_next_submodule(struct child_process *cp, struct strbuf *err, void *data, void **task_cb) { @@ -1235,12 +1260,11 @@ static int get_next_submodule(struct child_process *cp, struct submodule_parallel_fetch *spf = data; for (; spf->count < spf->r->index->cache_nr; spf->count++) { - struct strbuf submodule_path = STRBUF_INIT; - struct strbuf submodule_git_dir = STRBUF_INIT; struct strbuf submodule_prefix = STRBUF_INIT; const struct cache_entry *ce = spf->r->index->cache[spf->count]; - const char *git_dir, *default_argv; + const char *default_argv; const struct submodule *submodule; + struct repository *repo; struct submodule default_submodule = SUBMODULE_INIT; if (!S_ISGITLINK(ce->ce_mode)) @@ -1275,16 +1299,12 @@ static int get_next_submodule(struct child_process *cp, continue; } - strbuf_repo_worktree_path(&submodule_path, spf->r, "%s", ce->name); - strbuf_addf(&submodule_git_dir, "%s/.git", submodule_path.buf); strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, ce->name); - git_dir = read_gitfile(submodule_git_dir.buf); - if (!git_dir) - git_dir = submodule_git_dir.buf; - if (is_directory(git_dir)) { + repo = get_submodule_repo_for(spf->r, ce->name); + if (repo) { child_process_init(cp); - cp->dir = strbuf_detach(&submodule_path, NULL); - prepare_submodule_repo_env(&cp->env_array); + prepare_submodule_repo_env_in_gitdir(&cp->env_array); + cp->dir = xstrdup(repo->gitdir); cp->git_cmd = 1; if (!spf->quiet) strbuf_addf(err, "Fetching submodule %s%s\n", @@ -1294,10 +1314,11 @@ static int get_next_submodule(struct child_process *cp, argv_array_push(&cp->args, default_argv); argv_array_push(&cp->args, "--submodule-prefix"); argv_array_push(&cp->args, submodule_prefix.buf); + + repo_clear(repo); + free(repo); ret = 1; } - strbuf_release(&submodule_path); - strbuf_release(&submodule_git_dir); strbuf_release(&submodule_prefix); if (ret) { spf->count++; diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 6c2f9b2ba26..42692219a1a 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -566,7 +566,12 @@ test_expect_success 'fetching submodule into a broken repository' ' test_must_fail git -C dst status && test_must_fail git -C dst diff && - test_must_fail git -C dst fetch --recurse-submodules + + # git-fetch cannot find the git directory of the submodule, + # so it will do nothing, successfully, as it cannot distinguish between + # this broken submodule and a submodule that was just set active but + # not cloned yet + git -C dst fetch --recurse-submodules ' test_expect_success "fetch new commits when submodule got renamed" ' From patchwork Fri Sep 21 22:35:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10611135 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43A1A14DA for ; Fri, 21 Sep 2018 22:36:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 348882D7A3 for ; Fri, 21 Sep 2018 22:36:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2895A2D7B8; Fri, 21 Sep 2018 22:36:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 27F332D7A3 for ; Fri, 21 Sep 2018 22:36:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391492AbeIVE1t (ORCPT ); Sat, 22 Sep 2018 00:27:49 -0400 Received: from mail-pg1-f202.google.com ([209.85.215.202]:55259 "EHLO mail-pg1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725811AbeIVE1t (ORCPT ); Sat, 22 Sep 2018 00:27:49 -0400 Received: by mail-pg1-f202.google.com with SMTP id o16-v6so6116017pgv.21 for ; Fri, 21 Sep 2018 15:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=EeftGsmioQiDoHwhdBwoxPT1IokguOXRXR0jEzMssgA=; b=pQ/JcjXZBnx8cpsXGPv6LEO+U7Si4AOZMh6Mwk4WQpkZ3qFBH+FZPvFROgD6hPz8GF 4/mB3ro1HmCSwlI/h3nWOCQBw2o0ABh9XzVlxEmy5NwkfknR8tUWusXQ8P1a8+z+ejQs Bn3N4EDm3T5ABeL8Agk7S2MQwL5BjN/J5eLZJ7pbZcgZg40wXVwb3TUINwLhChPFzZA7 h/GFQGLJp5muveJfLvgC4eGafRTLPdaNfFIa8UiI+8nMaKwgTtl/Otif2MrylnS/PNal 4pXGMz7E7sLq0l1ejlEt+OhZEeoIdVcFJrL5zXcB68wn8g3UmzglkW8in0Ur+GYo5HcU J2CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=EeftGsmioQiDoHwhdBwoxPT1IokguOXRXR0jEzMssgA=; b=NBKgyFuSqHGpviH9LEwpJQtOd4NFYrZQKLoC/H2IFANCU/+CBJ9rdopoCatf7XVIM1 nK47OThM5oKPRIIjN14WDRWYaJuZS0gpHXPUlvng3a/vkSRzKM2s1fdkQg9ZyoqFXL9W k9KffXtPyhzLqtweSvdwbSdepnkeUPpCKhyYizQTCvLhRkXDHVsdeL/OwolA8LUKaBzT XMzXccP7/+SffEV9YFJAdrwJcDa33kaZUf0U5zXidcSqvoj2MGs8o99rGUaGxzaoCYuN 7Qxfgg3lorSP6cwkQ1RiKadfUnRhrt0iKgbseNTKx7s04YipfMI3poNrvHC6kIVtn25e i5DA== X-Gm-Message-State: APzg51Af/WOKh6zLdLTjhAVPP6HFntWZ0Ey/ojqKM4a5W5iGnjhiun5i j7NwSxYOSo7tNHbUA4o0x8QDZnpTCi/gn9fDGghbpDmuQElMnC+ZGtgOQa2mlr1dqpka1g9G4Dm 7IH+8s4kDkwFFlCBesFRCBa5yiBTiqKkmn16DuGfv2OVGjvZssr3xTsd129lx X-Google-Smtp-Source: ANB0VdYI2JN+NrrXcinZ9FBswXEreNe6TlTLJeN3G19hm5yzZh+rTuGtKFY2OjmwUVk68MYJk1I9GJFOz8wt X-Received: by 2002:a63:5504:: with SMTP id j4-v6mr2152715pgb.106.1537569412471; Fri, 21 Sep 2018 15:36:52 -0700 (PDT) Date: Fri, 21 Sep 2018 15:35:57 -0700 In-Reply-To: <20180921223558.65055-1-sbeller@google.com> Message-Id: <20180921223558.65055-8-sbeller@google.com> Mime-Version: 1.0 References: <20180921223558.65055-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 7/8] fetch: retry fetching submodules if needed objects were not fetched From: Stefan Beller To: git@vger.kernel.org Cc: Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently when git-fetch is asked to recurse into submodules, it dispatches a plain "git-fetch -C " (with some submodule related options such as prefix and recusing strategy, but) without any information of the remote or the tip that should be fetched. This works surprisingly well in some workflows (such as using submodules as a third party library), while not so well in other scenarios, such as in a Gerrit topic-based workflow, that can tie together changes (potentially across repositories) on the server side. One of the parts of such a Gerrit workflow is to download a change when wanting to examine it, and you'd want to have its submodule changes that are in the same topic downloaded as well. However these submodule changes reside in their own repository in their own ref (refs/changes/). Retry fetching a submodule by object name if the object id that the superproject points to, cannot be found. This retrying does not happen when the "git fetch" done at the superproject is not storing the fetched results in remote tracking branches (i.e. instead just recording them to FETCH_HEAD) in this step. A later patch will fix this. builtin/fetch used to only inspect submodules when they were fetched "on-demand", as in either on/off case it was clear whether the submodule needs to be fetched. However to know whether we need to try fetching the object ids, we need to identify the object names, which is done in this function check_for_new_submodule_commits(), so we'll also run that code in case the submodule recursion is set to "on". Signed-off-by: Stefan Beller --- builtin/fetch.c | 9 +- submodule.c | 182 ++++++++++++++++++++++++++++++------ t/t5526-fetch-submodules.sh | 16 ++++ 3 files changed, 174 insertions(+), 33 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 0696abfc2a1..e3b03ad3bd3 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -707,8 +707,7 @@ static int update_local_ref(struct ref *ref, what = _("[new ref]"); } - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref(msg, ref, 0); format_display(display, r ? '!' : '*', what, @@ -723,8 +722,7 @@ static int update_local_ref(struct ref *ref, strbuf_add_unique_abbrev(&quickref, ¤t->object.oid, DEFAULT_ABBREV); strbuf_addstr(&quickref, ".."); strbuf_add_unique_abbrev(&quickref, &ref->new_oid, DEFAULT_ABBREV); - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref("fast-forward", ref, 1); format_display(display, r ? '!' : ' ', quickref.buf, @@ -738,8 +736,7 @@ static int update_local_ref(struct ref *ref, strbuf_add_unique_abbrev(&quickref, ¤t->object.oid, DEFAULT_ABBREV); strbuf_addstr(&quickref, "..."); strbuf_add_unique_abbrev(&quickref, &ref->new_oid, DEFAULT_ABBREV); - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref("forced-update", ref, 1); format_display(display, r ? '!' : '+', quickref.buf, diff --git a/submodule.c b/submodule.c index 8829e8d4eff..ba98a94d4db 100644 --- a/submodule.c +++ b/submodule.c @@ -1128,8 +1128,12 @@ struct submodule_parallel_fetch { int result; struct string_list changed_submodule_names; + struct get_next_submodule_task **retry; + int retry_nr, retry_alloc; }; -#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, STRING_LIST_INIT_DUP } +#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, \ + STRING_LIST_INIT_DUP, \ + NULL, 0, 0} static void calculate_changed_submodule_paths( struct submodule_parallel_fetch *spf) @@ -1234,6 +1238,56 @@ static int get_fetch_recurse_config(const struct submodule *submodule, return spf->default_option; } +struct get_next_submodule_task { + struct repository *repo; + const struct submodule *sub; + unsigned free_sub : 1; /* Do we need to free the submodule? */ + struct oid_array *commits; +}; + +static const struct submodule *get_default_submodule(const char *path) +{ + struct submodule *ret = NULL; + const char *name = default_name_or_path(path); + + if (!name) + return NULL; + + ret = xmalloc(sizeof(*ret)); + memset(ret, 0, sizeof(*ret)); + ret->path = name; + ret->name = name; + + return (const struct submodule *) ret; +} + +static struct get_next_submodule_task *get_next_submodule_task_create( + struct repository *r, const char *path) +{ + struct get_next_submodule_task *task = xmalloc(sizeof(*task)); + memset(task, 0, sizeof(*task)); + + task->sub = submodule_from_path(r, &null_oid, path); + if (!task->sub) { + task->sub = get_default_submodule(path); + task->free_sub = 1; + } + + return task; +} + +static void get_next_submodule_task_release(struct get_next_submodule_task *p) +{ + if (p->free_sub) + free((void*)p->sub); + p->free_sub = 0; + p->sub = NULL; + + if (p->repo) + repo_clear(p->repo); + FREE_AND_NULL(p->repo); +} + static struct repository *get_submodule_repo_for(struct repository *r, const char *path) { @@ -1256,39 +1310,35 @@ static struct repository *get_submodule_repo_for(struct repository *r, static int get_next_submodule(struct child_process *cp, struct strbuf *err, void *data, void **task_cb) { - int ret = 0; struct submodule_parallel_fetch *spf = data; for (; spf->count < spf->r->index->cache_nr; spf->count++) { - struct strbuf submodule_prefix = STRBUF_INIT; + int recurse_config; const struct cache_entry *ce = spf->r->index->cache[spf->count]; const char *default_argv; - const struct submodule *submodule; - struct repository *repo; - struct submodule default_submodule = SUBMODULE_INIT; + struct get_next_submodule_task *task; if (!S_ISGITLINK(ce->ce_mode)) continue; - submodule = submodule_from_path(spf->r, &null_oid, ce->name); - if (!submodule) { - const char *name = default_name_or_path(ce->name); - if (name) { - default_submodule.path = name; - default_submodule.name = name; - submodule = &default_submodule; - } + task = get_next_submodule_task_create(spf->r, ce->name); + + if (!task->sub) { + free(task); + continue; } - switch (get_fetch_recurse_config(submodule, spf)) + recurse_config = get_fetch_recurse_config(task->sub, spf); + + switch (recurse_config) { default: case RECURSE_SUBMODULES_DEFAULT: case RECURSE_SUBMODULES_ON_DEMAND: - if (!submodule || + if (!task->sub || !string_list_lookup( &spf->changed_submodule_names, - submodule->name)) + task->sub->name)) continue; default_argv = "on-demand"; break; @@ -1299,12 +1349,13 @@ static int get_next_submodule(struct child_process *cp, continue; } - strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, ce->name); - repo = get_submodule_repo_for(spf->r, ce->name); - if (repo) { + task->repo = get_submodule_repo_for(spf->r, + task->sub->path); + if (task->repo) { + struct strbuf submodule_prefix = STRBUF_INIT; child_process_init(cp); prepare_submodule_repo_env_in_gitdir(&cp->env_array); - cp->dir = xstrdup(repo->gitdir); + cp->dir = task->repo->gitdir; cp->git_cmd = 1; if (!spf->quiet) strbuf_addf(err, "Fetching submodule %s%s\n", @@ -1313,18 +1364,51 @@ static int get_next_submodule(struct child_process *cp, argv_array_pushv(&cp->args, spf->args.argv); argv_array_push(&cp->args, default_argv); argv_array_push(&cp->args, "--submodule-prefix"); + + strbuf_addf(&submodule_prefix, "%s%s/", + spf->prefix, + task->sub->path); argv_array_push(&cp->args, submodule_prefix.buf); - repo_clear(repo); - free(repo); - ret = 1; - } - strbuf_release(&submodule_prefix); - if (ret) { spf->count++; + *task_cb = task; + + strbuf_release(&submodule_prefix); return 1; + } else { + get_next_submodule_task_release(task); + free(task); } } + + if (spf->retry_nr) { + struct get_next_submodule_task *task = spf->retry[spf->retry_nr - 1]; + struct strbuf submodule_prefix = STRBUF_INIT; + spf->retry_nr--; + + strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, task->sub->path); + + child_process_init(cp); + prepare_submodule_repo_env_in_gitdir(&cp->env_array); + cp->git_cmd = 1; + cp->dir = task->repo->gitdir; + + argv_array_init(&cp->args); + argv_array_pushv(&cp->args, spf->args.argv); + argv_array_push(&cp->args, "on-demand"); + argv_array_push(&cp->args, "--submodule-prefix"); + argv_array_push(&cp->args, submodule_prefix.buf); + + /* NEEDSWORK: have get_default_remote from s--h */ + argv_array_push(&cp->args, "origin"); + oid_array_for_each_unique(task->commits, + append_oid_to_argv, &cp->args); + + *task_cb = task; + strbuf_release(&submodule_prefix); + return 1; + } + return 0; } @@ -1332,20 +1416,64 @@ static int fetch_start_failure(struct strbuf *err, void *cb, void *task_cb) { struct submodule_parallel_fetch *spf = cb; + struct get_next_submodule_task *task = task_cb; spf->result = 1; + get_next_submodule_task_release(task); return 0; } +static int commit_exists_in_sub(const struct object_id *oid, void *data) +{ + struct repository *subrepo = data; + + enum object_type type = oid_object_info(subrepo, oid, NULL); + + return type != OBJ_COMMIT; +} + static int fetch_finish(int retvalue, struct strbuf *err, void *cb, void *task_cb) { struct submodule_parallel_fetch *spf = cb; + struct get_next_submodule_task *task = task_cb; + const struct submodule *sub; + + struct string_list_item *it; + struct oid_array *commits; if (retvalue) spf->result = 1; + if (!task) + return 0; + + sub = task->sub; + if (!sub) + goto out; + + it = string_list_lookup(&spf->changed_submodule_names, sub->name); + if (!it) + goto out; + + commits = it->util; + oid_array_filter(commits, + commit_exists_in_sub, + task->repo); + + trace_printf("checking for submodule: needs %d more commits", commits->nr); + if (commits->nr) { + task->commits = commits; + ALLOC_GROW(spf->retry, spf->retry_nr + 1, spf->retry_alloc); + spf->retry[spf->retry_nr] = task; + spf->retry_nr++; + return 0; + } + +out: + get_next_submodule_task_release(task); + return 0; } diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 42692219a1a..af12c50e7dd 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -605,4 +605,20 @@ test_expect_success "fetch new commits when submodule got renamed" ' test_cmp expect actual ' +test_expect_success "fetch new commits on-demand when they are not reachable" ' + git checkout --detach && + C=$(git -C submodule commit-tree -m "new change outside refs/heads" HEAD^{tree}) && + git -C submodule update-ref refs/changes/1 $C && + git update-index --cacheinfo 160000 $C submodule && + git commit -m "updated submodule outside of refs/heads" && + D=$(git rev-parse HEAD) && + git update-ref refs/changes/2 $D && + ( + cd downstream && + git fetch --recurse-submodules --recurse-submodules-default on-demand origin refs/changes/2:refs/heads/my_branch && + git -C submodule cat-file -t $C && + git checkout --recurse-submodules FETCH_HEAD + ) +' + test_done From patchwork Fri Sep 21 22:35:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10611137 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CAE18112B for ; Fri, 21 Sep 2018 22:36:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BCF6E2D7A3 for ; Fri, 21 Sep 2018 22:36:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B12F02D7B8; Fri, 21 Sep 2018 22:36:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4F9AF2D7A3 for ; Fri, 21 Sep 2018 22:36:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391563AbeIVE1v (ORCPT ); Sat, 22 Sep 2018 00:27:51 -0400 Received: from mail-io1-f74.google.com ([209.85.166.74]:43948 "EHLO mail-io1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725811AbeIVE1v (ORCPT ); Sat, 22 Sep 2018 00:27:51 -0400 Received: by mail-io1-f74.google.com with SMTP id w19-v6so25729456ioa.10 for ; Fri, 21 Sep 2018 15:36:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=3aVAT3rqLUyaHyBUNyJ+QSsf5/Fz+G3E3OGDfYKpE64=; b=mq/IW6FIOl9M0yjmtTzqyzdsTv0DTE2LE6Uy0rDLKAMzar0dB2myzLDSRRYXD54+kB 1Hgq/DKaWa5N7iYbyAgbQs3PxlFYCSicjrf32gybxtyh2S2XHHOcMD+yQ7uIISI6KPqH A+Z3Ob8bISnpcvM10eQGcgmV2JmZpvOuCfvYwMgznyRXBL+gXe4fQhBX2JYxuv+LTTOE 3FjUJw3l6qMKNaCiIwZm8Eid3yiN0JHUT4Rj4sWaYgnGbu3xsnFq9OPuaWx+7yhONXGF P2K+aefUJnoqlNDDSnUCdus9m1RDpnCm4wV1tlVIdH1ENPpL5D1o4QJhn0fcjY3vajUE 0xtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=3aVAT3rqLUyaHyBUNyJ+QSsf5/Fz+G3E3OGDfYKpE64=; b=jgA0T95tgtGbYNzb2iA8pZxqqNxNLgbQFG1MBC7jxHVMeBj2bPeP1p+KULUFDmKhgL 9BFjqh3GACjQMpBsp2BdVt3+eWEYPcGWyk3q+o1QW7Ky412MqKFv6zb+NV0UGAqtdqvE PLLDbkq7Y9ecNWsrehN05+3qLSO/wzS9b9k+2iL4Dt9YbEVJzCfCK16vDSWeAjLoNZ61 W4vAKF+RNb1QnlyjwF0bPAopAeAuB7ie+S6FHbZuuE/0jOIssbEq7kyPPzGinXkk+uXE bqskYbIyhWB0gjG9ksMoPjUbC6UaYT20O9Ai17/QfE+Ly6TTxwa908gTMjMcSw335vZB qWPg== X-Gm-Message-State: ABuFfogn3TqabPy005Avie8BkPywk11UbpdE1w3Ews+qXnFdoGuR8fUo mFu0QBlMWUI3E9oSsNxfF34FP1aoYTdfCVBKIjEOMv7uk+ShEg6BRbJfeLO3eSxii/zIxwbYDIm bV4yv4Fyh6SyLwlxisB0pJ6i3FMYiUk79DoX5YUlN3pHVs6k/k0Pij882uE/d X-Google-Smtp-Source: ACcGV63lO9esXcmeCjB3aso9YP9j2CurXW5kr0rB0P7l+ldtuO+Rk/wF9bchcgcXziSI/yZfCsI4mwyrgz+C X-Received: by 2002:a6b:8fd1:: with SMTP id r200-v6mr7371388iod.72.1537569414711; Fri, 21 Sep 2018 15:36:54 -0700 (PDT) Date: Fri, 21 Sep 2018 15:35:58 -0700 In-Reply-To: <20180921223558.65055-1-sbeller@google.com> Message-Id: <20180921223558.65055-9-sbeller@google.com> Mime-Version: 1.0 References: <20180921223558.65055-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 8/8] builtin/fetch: check for submodule updates for non branch fetches From: Stefan Beller To: git@vger.kernel.org Cc: Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Gerrit, the code review tool, has a different workflow than our mailing list based approach. Usually users upload changes to a Gerrit server and continuous integration and testing happens by bots. Sometimes however a user wants to checkout a change locally and look at it locally. For this use case, Gerrit offers a command line snippet to copy and paste to your terminal, which looks like git fetch https:///gerrit refs/changes/ && git checkout FETCH_HEAD For Gerrit changes that contain changing submodule gitlinks, it would be easy to extend both the fetch and checkout with the '--recurse-submodules' flag, such that this command line snippet would produce the state of a change locally. However the functionality added in the previous patch, which would ensure that we fetch the objects in the submodule that the gitlink pointed at, only works for remote tracking branches so far, not for FETCH_HEAD. Make sure that fetching a superproject to its FETCH_HEAD, also respects the existence checks for objects in the submodule recursion. Signed-off-by: Stefan Beller --- builtin/fetch.c | 5 ++++- t/t5526-fetch-submodules.sh | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index e3b03ad3bd3..f2d9e548bf0 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -894,11 +894,14 @@ static int store_updated_refs(const char *raw_url, const char *remote_name, rc |= update_local_ref(ref, what, rm, ¬e, summary_width); free(ref); - } else + } else { + check_for_new_submodule_commits(&rm->old_oid); format_display(¬e, '*', *kind ? kind : "branch", NULL, *what ? what : "HEAD", "FETCH_HEAD", summary_width); + } + if (note.len) { if (verbosity >= 0 && !shown_url) { fprintf(stderr, _("From %.*s\n"), diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index af12c50e7dd..a509eabb044 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -615,7 +615,7 @@ test_expect_success "fetch new commits on-demand when they are not reachable" ' git update-ref refs/changes/2 $D && ( cd downstream && - git fetch --recurse-submodules --recurse-submodules-default on-demand origin refs/changes/2:refs/heads/my_branch && + git fetch --recurse-submodules origin refs/changes/2 && git -C submodule cat-file -t $C && git checkout --recurse-submodules FETCH_HEAD )