From patchwork Mon Sep 17 21:35:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603455 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E76306CB for ; Mon, 17 Sep 2018 21:36:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB84F2996D for ; Mon, 17 Sep 2018 21:36:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CFB1C29DA9; Mon, 17 Sep 2018 21:36:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 67B822996D for ; Mon, 17 Sep 2018 21:36:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728575AbeIRDFP (ORCPT ); Mon, 17 Sep 2018 23:05:15 -0400 Received: from mail-it0-f73.google.com ([209.85.214.73]:57439 "EHLO mail-it0-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727672AbeIRDFP (ORCPT ); Mon, 17 Sep 2018 23:05:15 -0400 Received: by mail-it0-f73.google.com with SMTP id e6-v6so314531itc.7 for ; Mon, 17 Sep 2018 14:36:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=q/YHaaUEkXqeiP8cs5krMvpohGk9wgEzGvicQ92UvPE=; b=WLm9WTZpgIf1OQ/KvkFHnL4jCdZYelWfsiKkQ06zOv0RMRPg0Hk+04bB8r3S655KXi cizaFDj7bBBeyc7ZL94J+1cN1m3XHwOz2Rp6tUv4fsNUhDlEe+TN+bqdSjCEyPTftMVg qJ5ZRRHxQjNNR9rvzLRzH2wsSsOsMAabLwX6hS/g0NgCHX2NjgHXVxuCui3PN76+FFZz NN6jxDUdwE53aosrzUcjD/GAQNwOK2xOP7qDu2k7k3sZGcAP8QHxlSRpa7k03WE1KcPs SAhLUxWYFaBwq+dLvhdTDrto6wYXluR5TcVH0HR4Ma4+/6TUcRYHP0chZjJX6EjDz7Vr UBWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=q/YHaaUEkXqeiP8cs5krMvpohGk9wgEzGvicQ92UvPE=; b=OM4Zn3MhOfquMmZSV1VUCdXEDSSkadHeaqw0VqBF+IGGeDAXoOdTU6ckPkdAqY93Mk /cA0Bq0lXxtq6xUmbTLrl2WvuRpe+qusbteX4p+nHLMO7yHCj8AS/noPmW37bFlf6Mlq fb3PPKKLYIbniA2r3T9SXc9xhAn/196DuY4sCRvQvnQj5gMKrgWOGcmSTnCpXpIOkyqB cvpvKGp79a/Al75Q4og0lnSoxYelBsFXHuYkmyQcIRFMPuTRYBBiyyHyhA/LiUytdkqO Zw7166IqXaAcwWVnvqAYaUQsevFqM1u6kCvaAirUSp2A345dFyLxcqPzxMsrUbmze4qM 8etw== X-Gm-Message-State: APzg51A7wa9UNZ38WrqyUXOyk5PmN5pSAC4Bn+lOiNPlwcR0Rkxj7fZw hyASTBQd7s9kqMN3xYQv9cp2bXBl5ZYE X-Google-Smtp-Source: ANB0Vdb4rclKxFo+XayGfxOhG1jvk/kClFQ0MqkOX9aIGV4tDS/XbpGd/eNAbJPz7O0nDWZ8ePWn4oNaFfHP X-Received: by 2002:a24:eec8:: with SMTP id b191-v6mr12167529iti.42.1537220165520; Mon, 17 Sep 2018 14:36:05 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:51 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-2-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 1/9] string-list: add string_list_{pop, last} functions From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a few functions to allow a string-list to be used as a stack: - string_list_last() lets a caller peek the string_list_item at the end of the string list. The caller needs to be aware that it is borrowing a pointer, which can become invalid if/when the string_list is resized. - string_list_pop() removes the string_list_item at the end of the string list. - there is no string_list_push(); string_list_append() can be used in its place. You can use them in this pattern: while (list.nr) { struct string_list_item *item = string_list_last(&list); work_on(item); string_list_pop(&list, free_util); } Helped-by: Junio C Hamano Signed-off-by: Stefan Beller --- string-list.c | 14 ++++++++++++++ string-list.h | 15 +++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/string-list.c b/string-list.c index 771c4550980..04db2b537c0 100644 --- a/string-list.c +++ b/string-list.c @@ -80,6 +80,20 @@ void string_list_remove(struct string_list *list, const char *string, } } +void string_list_pop(struct string_list *list, int free_util) +{ + if (list->nr == 0) + BUG("tried to remove an item from empty string list"); + + if (list->strdup_strings) + free(list->items[list->nr - 1].string); + + if (free_util) + free(list->items[list->nr - 1].util); + + list->nr--; +} + int string_list_has_string(const struct string_list *list, const char *string) { int exact_match; diff --git a/string-list.h b/string-list.h index ff8f6094a33..15f2936e337 100644 --- a/string-list.h +++ b/string-list.h @@ -191,6 +191,21 @@ extern void string_list_remove(struct string_list *list, const char *string, */ struct string_list_item *string_list_lookup(struct string_list *list, const char *string); +/** + * Removes the last item from the list. + * The caller must ensure that the list is not empty. + */ +void string_list_pop(struct string_list *list, int free_util); + +/* + * Returns the last item of the list. As it returns the raw access, do not + * modify the list while holding onto the returned pointer. + */ +static inline struct string_list_item *string_list_last(struct string_list *list) +{ + return &list->items[list->nr - 1]; +} + /* * Remove all but the first of consecutive entries with the same * string value. If free_util is true, call free() on the util From patchwork Mon Sep 17 21:35:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603457 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16C8C13AD for ; Mon, 17 Sep 2018 21:36:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C3BE2996D for ; Mon, 17 Sep 2018 21:36:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 00DFD29DA9; Mon, 17 Sep 2018 21:36:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A82582996D for ; Mon, 17 Sep 2018 21:36:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728605AbeIRDFT (ORCPT ); Mon, 17 Sep 2018 23:05:19 -0400 Received: from mail-pg1-f202.google.com ([209.85.215.202]:52651 "EHLO mail-pg1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727672AbeIRDFQ (ORCPT ); Mon, 17 Sep 2018 23:05:16 -0400 Received: by mail-pg1-f202.google.com with SMTP id m4-v6so6798504pgq.19 for ; Mon, 17 Sep 2018 14:36:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=1tJhZiYTYExsGZ+1the7R9+lAxiqWj13Klz5W2m723c=; b=BlslopDKfZUYfk3qIPSr+q/SCzn/9KE/in5u2pP1q7Tgy+qTZ1A/gmxLitsrak4MAv FVvKHqyp7SwT2GBLVAtgNQYcqPgvM6bVIHsj2teIZQR2kcv0op1dPC60S2cvmmGOcuOM hwINxxG1TSyx2aW00NoH//YkSSSpN2C91DY9Q1MzYY5q2yBWHfaFvPrXGB7asj1rLqZM 1cRpUwl+zJj7MOmjxcrgAy3K4vGv3JHlU576WWFaFeW2Ra6D7z1u/322BvjkkQnVN66j D/3YpZE7a/4goO6fzPr+Kn/dSDuRJO8nEnv3O/R63FTJoGvV6G6U7gWPL2YvPdieqX+B qooQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=1tJhZiYTYExsGZ+1the7R9+lAxiqWj13Klz5W2m723c=; b=mMMzCxOZVtbSBlVe4OZ6zqX+AiKHoc5RXhrw5Xbt6V5cqPNRAWzLfc6YLEf735/3Ee ECY8C4kKjSfkiW6Py+TrpBGyQ9vmVyR0YVTNKUc2vaADELt2kdm9WennwiHeonwHM6a5 ylNVxd/W7p2hIPk3tW8Bf8bh2TYDOa9HDeNVlgOPGIiIsX6vkyvSU9IiUcTI9NS42uV3 CXCnURrFvs3eupY6lk25lFErDnvybYQiUlj5tNabkKjp7A2LnYs0BBiclMZCcjdru3rd cBKWqOV2ruZMOgU4QiWj5VO7LVfKBfOz7TpsYQStD6J9mEIIbbSJf7xuV5SnoI6IG4Oh YY8Q== X-Gm-Message-State: APzg51Bn5terb4FcHJ9789CfeMKrxVJrBoZehXBZybQYCw2lunFu3QaP fyazc1XQQH+SWXA2eiK+rcTLbzkq9dpI X-Google-Smtp-Source: ANB0VdbJiv/6+cUQJlVaeWT9HCwWRWPmr1QFsLQNp1JuJ0+PXkHO4GbLT/TORxCD/YBfo/zGZ6xaWmUU6OZR X-Received: by 2002:a63:7218:: with SMTP id n24-v6mr1082547pgc.157.1537220167603; Mon, 17 Sep 2018 14:36:07 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:52 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-3-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 2/9] sha1-array: provide oid_array_filter From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Helped-by: Junio C Hamano Signed-off-by: Stefan Beller --- sha1-array.c | 17 +++++++++++++++++ sha1-array.h | 9 +++++++++ 2 files changed, 26 insertions(+) diff --git a/sha1-array.c b/sha1-array.c index 265941fbf40..67db5eeec9a 100644 --- a/sha1-array.c +++ b/sha1-array.c @@ -77,3 +77,20 @@ int oid_array_for_each_unique(struct oid_array *array, } return 0; } + +void oid_array_filter(struct oid_array *array, + for_each_oid_fn want, + void *cb_data) +{ + unsigned nr = array->nr, src, dst; + struct object_id *oids = array->oids; + + for (src = dst = 0; src < nr; src++) { + if (want(&oids[src], cb_data)) { + if (src != dst) + oidcpy(oids[dst], &oids[src]); + dst++; + } + } + array->nr = dst; +} diff --git a/sha1-array.h b/sha1-array.h index 232bf950172..ae059ca0431 100644 --- a/sha1-array.h +++ b/sha1-array.h @@ -23,4 +23,13 @@ int oid_array_for_each_unique(struct oid_array *array, for_each_oid_fn fn, void *data); +/* + * Apply want to each entry in array, retaining only the entries for + * which the function returns true. Preserve the order of the entries + * that are retained. + */ +int oid_array_filter(struct oid_array *array, + for_each_oid_fn want, + void *cbdata); + #endif /* SHA1_ARRAY_H */ From patchwork Mon Sep 17 21:35:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603459 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 00EDA13AD for ; Mon, 17 Sep 2018 21:36:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EAE1A2996D for ; Mon, 17 Sep 2018 21:36:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DF68829DA9; Mon, 17 Sep 2018 21:36:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 901382996D for ; Mon, 17 Sep 2018 21:36:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728607AbeIRDFT (ORCPT ); Mon, 17 Sep 2018 23:05:19 -0400 Received: from mail-pl1-f202.google.com ([209.85.214.202]:56900 "EHLO mail-pl1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728564AbeIRDFT (ORCPT ); Mon, 17 Sep 2018 23:05:19 -0400 Received: by mail-pl1-f202.google.com with SMTP id a10-v6so8208081pls.23 for ; Mon, 17 Sep 2018 14:36:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=UY9P6GTLun6sLU3lqilisBa+iFBbQb4RZUF6QIpwrXk=; b=VakmKE+cqAlQQNN9wlc1Y0P+bOV5pxJzTH6btE+8AbNg218CEdH2PmJCO9IdiGph1V JrCZNvw/XPIghPR9ZaBbO1a9TVyYIIhs+Mn95A79BckWJM58laoLlb8TLm/BKXO52/tk UwJqfNikj42hUjr/8QbZpkyXC/2/qtJfbKtwNpDYfIrgIXjqgykQ0zMSEQWyxHUAOH1V XigSl8RSVzfQA+AcYmSidRc+Bi1bDYiOuSozmr4en2VWMeMHwkJRyS/+vh8UCf5QQe2m 9ieqLjQv2kRfP4SSSeN6Y49s8wVyOXWkWPbmIjnD5VwT8pTF39hmzGyQQP3VqWtm/Vuo sQUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=UY9P6GTLun6sLU3lqilisBa+iFBbQb4RZUF6QIpwrXk=; b=UB8LtJBtH7SH5D14RrlnWe15V9ZF97FsLIAW6rh7iPrKUopRW/CHLZR1uprsO8Qysl nxHjThZsueM6ZAAGkmhmZ7onlFP+4+SxMwOG9YjeTTWjjJMbY3DGUdUWRvj87mkYgfCa tInYPM1/+4J5lvcDsHGUnDrm3xyNy38vS0a2mUZTItccarMH/vbfnBPw1vLlJUgNwqIK dbJjBOpF8kAaWQZH0fOCgRBMe81Wq35jx2OWn0dNGzTQVUq1FXD5bG41EYj/9TgxEdsT nwRcukepaWVK5MDSGrMGgjJ9leDlZMqzLjsKpl2GpLZFU0oqkcppvL2grh6HY8XbRE1C cxNw== X-Gm-Message-State: APzg51A6nh4qtAK3u7OUDojXelyDEG2mildEbjAut6BUbDs15JU7kKGA 0BiFBCrWsYbyEMS9RU+RJVY2nXgUO5wy X-Google-Smtp-Source: ANB0Vdal+PWpGL0h9cavv1EsPwoqTsdb3yOoO+bwm5ClKSsM/u6i75rbyM4JSR+SR+4Ptac2rbYWbPO0aIGN X-Received: by 2002:a63:7945:: with SMTP id u66-v6mr1186132pgc.9.1537220169791; Mon, 17 Sep 2018 14:36:09 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:53 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-4-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 3/9] submodule.c: fix indentation From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The submodule subsystem is really bad at staying within 80 characters. Fix it while we are here. Signed-off-by: Stefan Beller --- submodule.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/submodule.c b/submodule.c index a2b266fbfae..d29dfa3d1f5 100644 --- a/submodule.c +++ b/submodule.c @@ -1244,7 +1244,8 @@ static int get_next_submodule(struct child_process *cp, if (!submodule) { const char *name = default_name_or_path(ce->name); if (name) { - default_submodule.path = default_submodule.name = name; + default_submodule.path = name; + default_submodule.name = name; submodule = &default_submodule; } } @@ -1254,8 +1255,10 @@ static int get_next_submodule(struct child_process *cp, default: case RECURSE_SUBMODULES_DEFAULT: case RECURSE_SUBMODULES_ON_DEMAND: - if (!submodule || !unsorted_string_list_lookup(&changed_submodule_names, - submodule->name)) + if (!submodule || + !unsorted_string_list_lookup( + &changed_submodule_names, + submodule->name)) continue; default_argv = "on-demand"; break; From patchwork Mon Sep 17 21:35:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603461 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC8126CB for ; Mon, 17 Sep 2018 21:36:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CEFD92996D for ; Mon, 17 Sep 2018 21:36:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C344F29DA9; Mon, 17 Sep 2018 21:36:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 699F52996D for ; Mon, 17 Sep 2018 21:36:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728616AbeIRDFW (ORCPT ); Mon, 17 Sep 2018 23:05:22 -0400 Received: from mail-ua1-f73.google.com ([209.85.222.73]:44893 "EHLO mail-ua1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728564AbeIRDFV (ORCPT ); Mon, 17 Sep 2018 23:05:21 -0400 Received: by mail-ua1-f73.google.com with SMTP id d22-v6so5757698uaq.11 for ; Mon, 17 Sep 2018 14:36:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=aePnPx2YqOtLYlyVK5CDLEc/HmVD9vFEee9X8pOHmKg=; b=N49N58/fPUa0Q1zFZuOt9Ve8QEWfOhI1c3hU9urAwHH2LVBW4sXc0k2MqMCkr3cBt1 z7AVb2SuePehDhJQqbH9nbZHfKJkTYV3v5etIAQPXJUEHD9IE+VX//SdfKY+dVP+kjB0 te+8RK35/kfwF0kqCEM4TDpWoMuJd7vFLvdVnVKlVele57WRkAnM9k9bRr7hy4KfvlTr c+KfpJyhdMGnVfve0qIEqkVy4U8nOJvKB2SfUbsNqfTE3R7Bl2snfBGG9cMLDJoW8Ffc FUzWnwvMOdK+RpQrVVfoNBhp4Hr6QfgHDcIGeHPxQl/u5VrOSzdG49C+3v1kWSeF8Nj/ xdVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=aePnPx2YqOtLYlyVK5CDLEc/HmVD9vFEee9X8pOHmKg=; b=lMc8uwze+uCR+4AlFtMQ4aAv/8dJVPChO4zMLsyuTCKG5CpoVXd7mSQ6VsbZ79Rl7l YixY8QCi4DN1rQI0sTEhELIDzMdfv1xbiFMgJj85Fd88b7s53jtJRdDZEFYqu/TGIFHl Ec0PCXaDRnBP82ZHc5L+DQls0jZFcvS/azMauq0t1f0Mp8f7jl+m6sgtJsVK3ZAj4iJy e45YxF4UJoRTimj2rgw8ImeUNZKOxyQ5sbZv43oMtBOXwPjqsiQFLuWUO0upZ6BGY0LE TV5b1PWaiguWLsy0T30sYvDPyjNDSHayIIidErEdLKnJZhrmgJgV1p+ZXm589eBGSVGZ gomA== X-Gm-Message-State: APzg51BuwoBfa+T0TlRY4xqyMnCaqMIokVEHdt85PSrRULBw3Jwe39x9 UsjcwoD6LGZ8ngUd1z6ezoMxPModzVRS X-Google-Smtp-Source: ANB0VdZW3oTrBv9haNtgzpEZtwNIfB6eOAru7jVK1K1XjySNJ739zL3s96VXaInntE/aQPecQ5BDO/ZRuUke X-Received: by 2002:a67:a8ce:: with SMTP id h75-v6mr5607253vsi.48.1537220172406; Mon, 17 Sep 2018 14:36:12 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:54 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-5-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 4/9] submodule.c: sort changed_submodule_names before searching it From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We can string_list_insert() to maintain sorted-ness of the list as we find new items, or we can string_list_append() to build an unsorted list and sort it at the end just once. To pick which one is more appropriate, we notice the fact that we discover new items more or less in the already sorted order. That makes "append then sort" more appropriate. Signed-off-by: Stefan Beller --- submodule.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/submodule.c b/submodule.c index d29dfa3d1f5..c6eff7699f3 100644 --- a/submodule.c +++ b/submodule.c @@ -1256,7 +1256,7 @@ static int get_next_submodule(struct child_process *cp, case RECURSE_SUBMODULES_DEFAULT: case RECURSE_SUBMODULES_ON_DEMAND: if (!submodule || - !unsorted_string_list_lookup( + !string_list_lookup( &changed_submodule_names, submodule->name)) continue; @@ -1350,6 +1350,7 @@ int fetch_populated_submodules(struct repository *r, /* default value, "--submodule-prefix" and its value are added later */ calculate_changed_submodule_paths(); + string_list_sort(&changed_submodule_names); run_processes_parallel(max_parallel_jobs, get_next_submodule, fetch_start_failure, From patchwork Mon Sep 17 21:35:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603463 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 956BA6CB for ; Mon, 17 Sep 2018 21:36:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 87BC42996D for ; Mon, 17 Sep 2018 21:36:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7C56A29DA9; Mon, 17 Sep 2018 21:36:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 18ED32996D for ; Mon, 17 Sep 2018 21:36:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728641AbeIRDFY (ORCPT ); Mon, 17 Sep 2018 23:05:24 -0400 Received: from mail-pf1-f201.google.com ([209.85.210.201]:36083 "EHLO mail-pf1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728564AbeIRDFY (ORCPT ); Mon, 17 Sep 2018 23:05:24 -0400 Received: by mail-pf1-f201.google.com with SMTP id d22-v6so9054085pfn.3 for ; Mon, 17 Sep 2018 14:36:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=N+C9LG44r+za892wkBrqwOOomAK6Va9G9n8tpBcil7o=; b=aHXy1WUu05rxg2nTRvtQkR7bBtIHL6cveXEN5eWKfRCQeTCGSr2GCc50I59kTI8ZWl 6WKpXjpGUVDjPC9c+sKHOLFMc0WVpf77hJSpplrjWfyRaPDZCpRZuov5gtFM2HDaQo48 FmGbfMdSfymaN/dc29Yf6jxtFhgFBBB1Wv0DroIEKQfIKBo1385F2cO3+Or0DCx213Hb mWhUugNC5NXoC3QGMfz7SZqgefnVGO/eFt1k+DGe1T24gavEoxy+5DQ4soIMYBZMJAMG 4+UfLvCAohlO+vc6mqG+auBRC8cfTdz1JiMkVxSxiFpUFOOx2fceTK4TS4dqXfRAzB6N Rxwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=N+C9LG44r+za892wkBrqwOOomAK6Va9G9n8tpBcil7o=; b=EE+a+EdeuxnRBpKG3bqnaHZOewO+oa/fqs30BvzdErn9vXE09aZTyElT5Oh85raGwX yNg8GbMLdQ/mxpNyRvFE4cf5Ikj9m4yWtua2wQM62J6ucH9fBXwPufAdKPybs/1TUxVi uTHvfZUUsodtcCsaBGoGqZQ5RQ5uSWjgyPKzMni4RuLStaqTaJc8XA2vFanINmkiyEVo FHWpsZ9Ml1cTvI+H/aRqobVtqyWppXiBV1QQJ2syBhIObRkX/CdzonGANEjy3cfDPP6V OFkbF6Qs9PyzNIxzzLVYNXMbQ3npDIndp5EDyDmTahF9UjC1esLHW0ctlTvpQ3PBZwVs e69A== X-Gm-Message-State: APzg51A1aq+sWi862xD6opamRs1lPEfEMuWT4z6cn52AVwRAoyJbRw51 xMXNIwPQayWpaWbJb5W2vENinRXwlAYi X-Google-Smtp-Source: ANB0VdZeLWiGHjWMScVmHTpKsJaLw/yM+eYZwUEsKsZx9HlkWDWL0OSjIsqgRc0FAp1J5hTeiIFQN5QDlebS X-Received: by 2002:a63:2bd2:: with SMTP id r201-v6mr1184707pgr.116.1537220174764; Mon, 17 Sep 2018 14:36:14 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:55 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-6-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 5/9] submodule: move global changed_submodule_names into fetch submodule struct From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The `changed_submodule_names` are only used for fetching, so let's make it part of the struct that is passed around for fetching submodules. Signed-off-by: Stefan Beller --- submodule.c | 42 +++++++++++++++++++++++------------------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/submodule.c b/submodule.c index c6eff7699f3..3520dd76bdf 100644 --- a/submodule.c +++ b/submodule.c @@ -24,7 +24,7 @@ #include "object-store.h" static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF; -static struct string_list changed_submodule_names = STRING_LIST_INIT_DUP; + static int initialized_fetch_ref_tips; static struct oid_array ref_tips_before_fetch; static struct oid_array ref_tips_after_fetch; @@ -1110,7 +1110,22 @@ void check_for_new_submodule_commits(struct object_id *oid) oid_array_append(&ref_tips_after_fetch, oid); } -static void calculate_changed_submodule_paths(void) +struct submodule_parallel_fetch { + int count; + struct argv_array args; + struct repository *r; + const char *prefix; + int command_line_option; + int default_option; + int quiet; + int result; + + struct string_list changed_submodule_names; +}; +#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, STRING_LIST_INIT_DUP } + +static void calculate_changed_submodule_paths( + struct submodule_parallel_fetch *spf) { struct argv_array argv = ARGV_ARRAY_INIT; struct string_list changed_submodules = STRING_LIST_INIT_DUP; @@ -1148,7 +1163,8 @@ static void calculate_changed_submodule_paths(void) continue; if (!submodule_has_commits(path, commits)) - string_list_append(&changed_submodule_names, name->string); + string_list_append(&spf->changed_submodule_names, + name->string); } free_submodules_oids(&changed_submodules); @@ -1185,18 +1201,6 @@ int submodule_touches_in_range(struct object_id *excl_oid, return ret; } -struct submodule_parallel_fetch { - int count; - struct argv_array args; - struct repository *r; - const char *prefix; - int command_line_option; - int default_option; - int quiet; - int result; -}; -#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0} - static int get_fetch_recurse_config(const struct submodule *submodule, struct submodule_parallel_fetch *spf) { @@ -1257,7 +1261,7 @@ static int get_next_submodule(struct child_process *cp, case RECURSE_SUBMODULES_ON_DEMAND: if (!submodule || !string_list_lookup( - &changed_submodule_names, + &spf->changed_submodule_names, submodule->name)) continue; default_argv = "on-demand"; @@ -1349,8 +1353,8 @@ int fetch_populated_submodules(struct repository *r, argv_array_push(&spf.args, "--recurse-submodules-default"); /* default value, "--submodule-prefix" and its value are added later */ - calculate_changed_submodule_paths(); - string_list_sort(&changed_submodule_names); + calculate_changed_submodule_paths(&spf); + string_list_sort(&spf.changed_submodule_names); run_processes_parallel(max_parallel_jobs, get_next_submodule, fetch_start_failure, @@ -1359,7 +1363,7 @@ int fetch_populated_submodules(struct repository *r, argv_array_clear(&spf.args); out: - string_list_clear(&changed_submodule_names, 1); + string_list_clear(&spf.changed_submodule_names, 1); return spf.result; } From patchwork Mon Sep 17 21:35:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603465 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3FA7613AD for ; Mon, 17 Sep 2018 21:36:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3536E2996D for ; Mon, 17 Sep 2018 21:36:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 294EF29DA9; Mon, 17 Sep 2018 21:36:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C8BA72996D for ; Mon, 17 Sep 2018 21:36:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728668AbeIRDF1 (ORCPT ); Mon, 17 Sep 2018 23:05:27 -0400 Received: from mail-io1-f73.google.com ([209.85.166.73]:43585 "EHLO mail-io1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728564AbeIRDF1 (ORCPT ); Mon, 17 Sep 2018 23:05:27 -0400 Received: by mail-io1-f73.google.com with SMTP id w19-v6so19928251ioa.10 for ; Mon, 17 Sep 2018 14:36:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=TniJD7JODSLZpF0IMCAmfYzJECJnfUzycru4/HQVYAQ=; b=H4U0at3Y064rW16YPMhZRCFu5x75Zd0W4Nnx6xrO3auAsIaOuvIyYiJmLBFAlfRB77 KrM9cj0lyZs9lDM1+ef2Ne/BTGNwthuwibzigMEbpTt16EuBaroVJhkK8oPoMyHRMXan MLGqWRunEYd9zalSQK2Ob//IHnmm3pO566OGBxBLz7g8Aqjx7aKfRVTta1VblbZUsd6h HqbTk5qxhcjvd9lSG15R0stRSGzXUNhhHj0PSjJR05J6H/4zNA+HgHrb/DdNulUs7kR9 VkUDbc6J3zC3embULpt1ZG7q9dw4zIqhjUOMBx8vOx7yOKpC/2rm9VtR1jF87CPNpQ2T MOcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=TniJD7JODSLZpF0IMCAmfYzJECJnfUzycru4/HQVYAQ=; b=WJVkpiYSHi2dvVhFlQLc3clkwNsg+vO6L4XG9vLJpMhQMQ55GwgIaCrtb7ZFCLSXQO 4msQjMZB3IsvzdqiWmi/wuLIYE+DoNwK0KLVu/2ZtPf9OZZb7qdBtxPfCfFOL45wr+JU byHKW8C1iN/g7JmxZwqbyjY5o18knzttZxB/IaPPo0aQxHQrNqskNxJgEgunHuCMYD+a vCKyoSrIJu0Y63O8IABG5D9zz1uQmz5FoasDbM4aFjRZIuIxKsawgnEUA6cK+wsTDZxx JkdbzdJSGQM7FImitsb0Xi+6eLCxZ3Spx5r469k67JhoMJyUH14tQTazY1F9mNpBfJcv xEtQ== X-Gm-Message-State: APzg51CUMptBzZmDdBzIcDYucCGDXlw6l7Vj6phN6yJTKTi3tDjbfWQc ZxSICkpEeqkn3SILjEl7hge+E56Zf8kV X-Google-Smtp-Source: ANB0VdbljlM3hTCsut+YlMmVorO72Ckq8TXK0zvBF4lf0YCk2uxafLs4U6XANJvQkNI2UM4ZazdUrl5L2V+5 X-Received: by 2002:a24:9f84:: with SMTP id c126-v6mr12510847ite.29.1537220176975; Mon, 17 Sep 2018 14:36:16 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:56 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-7-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 6/9] submodule.c: do not copy around submodule list From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP 'calculate_changed_submodule_paths' uses a local list to compute the changed submodules, and then produces the result by copying appropriate items into the result list. Instead use the result list directly and prune items afterwards using string_list_remove_empty_items. By doing so we'll have access to the util pointer for longer that contains the commits that we need to fetch, which will be useful in a later patch. Signed-off-by: Stefan Beller --- submodule.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/submodule.c b/submodule.c index 3520dd76bdf..00a9a3c6b12 100644 --- a/submodule.c +++ b/submodule.c @@ -1128,8 +1128,7 @@ static void calculate_changed_submodule_paths( struct submodule_parallel_fetch *spf) { struct argv_array argv = ARGV_ARRAY_INIT; - struct string_list changed_submodules = STRING_LIST_INIT_DUP; - const struct string_list_item *name; + struct string_list_item *name; /* No need to check if there are no submodules configured */ if (!submodule_from_path(the_repository, NULL, NULL)) @@ -1146,9 +1145,9 @@ static void calculate_changed_submodule_paths( * Collect all submodules (whether checked out or not) for which new * commits have been recorded upstream in "changed_submodule_names". */ - collect_changed_submodules(&changed_submodules, &argv); + collect_changed_submodules(&spf->changed_submodule_names, &argv); - for_each_string_list_item(name, &changed_submodules) { + for_each_string_list_item(name, &spf->changed_submodule_names) { struct oid_array *commits = name->util; const struct submodule *submodule; const char *path = NULL; @@ -1162,12 +1161,14 @@ static void calculate_changed_submodule_paths( if (!path) continue; - if (!submodule_has_commits(path, commits)) - string_list_append(&spf->changed_submodule_names, - name->string); + if (submodule_has_commits(path, commits)) { + oid_array_clear(commits); + *name->string = '\0'; + } } - free_submodules_oids(&changed_submodules); + string_list_remove_empty_items(&spf->changed_submodule_names, 1); + argv_array_clear(&argv); oid_array_clear(&ref_tips_before_fetch); oid_array_clear(&ref_tips_after_fetch); @@ -1363,7 +1364,7 @@ int fetch_populated_submodules(struct repository *r, argv_array_clear(&spf.args); out: - string_list_clear(&spf.changed_submodule_names, 1); + free_submodules_oids(&spf.changed_submodule_names); return spf.result; } From patchwork Mon Sep 17 21:35:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603467 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0EFB313AD for ; Mon, 17 Sep 2018 21:36:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 03EEA2996D for ; Mon, 17 Sep 2018 21:36:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EC6AE2A1D5; Mon, 17 Sep 2018 21:36:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 689C92996D for ; Mon, 17 Sep 2018 21:36:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728674AbeIRDF3 (ORCPT ); Mon, 17 Sep 2018 23:05:29 -0400 Received: from mail-pf1-f201.google.com ([209.85.210.201]:49802 "EHLO mail-pf1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728564AbeIRDF2 (ORCPT ); Mon, 17 Sep 2018 23:05:28 -0400 Received: by mail-pf1-f201.google.com with SMTP id d1-v6so9024671pfo.16 for ; Mon, 17 Sep 2018 14:36:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dGBUGTNog18hlMse2uaY+4/W+OqsYZ7k0qYZST7kDUI=; b=RAPofrgmqb1Zn/UIjSFgg/+pIDqhNWZpW3PkcOAEXnqciilU5yPOqKOcI9VFw97r3b cZ+EZGLQqbIfFBHiwR1odP3N7agx+kG3M8vXUghT4NiHfLGfkUkVPRNSr7l4ySoA7DRt rqQXLWCmudd1DI9n604GaVq1B9/03L8gkwCOIdwMiNMKTWKmu0fsLh7YbFNeCptX7pOP Q1LL2x1Ijjlc2e2b72EfiyQjg1uZkKfWTOOGUGzyMYs41ZbrKqm6ENg8orQKN4TzUht1 xI2bygy14UVwffqWbZrqgCpIMzOL2RqbCTC8/7TQFf2BFrCnumf+b9X4O/XgevO6PEvE wKIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dGBUGTNog18hlMse2uaY+4/W+OqsYZ7k0qYZST7kDUI=; b=O/sbl1JL1aWRPg3VBoJi3PDKGAMp7VONxkDuIxd+8egxLbIlHpHAzv2yXv85hKySDd Q2VHWHQaVp803ABZYX7aXBsKK5tzffQ+ls079Iy+hVSlnSr4/ZuosSzBqid5CvaZ5c6N NVqWWfVSpB3t8DNAESNkBvoczL5iS61EA0P2TPWL/cEU1gFNF9HRUj4o6cgjJe1etOzD k6Y+tn2aeXOo9uo4L16N1e2g19019V+ubCvKaB9qbByfSQKE/kAB11Hjh6xhodvZ3zmD MaVxcKvNMBOS5bZGp2L3ISQLMWESj+xbr21grOd5xKuOPHRSFZ8Q9q4oUuYMFsEF/wJE 5exQ== X-Gm-Message-State: APzg51AHtUWbQdYWk5Io2I1K2M4cmtClF0NcAMvrKs2CVfoLuYCNrPyh scPwonBkCpBLhlOxAVepGSk7EXGJeCcf X-Google-Smtp-Source: ANB0VdbgkO4JzzxuPsLVZEpP5VyNLK2Pyhohw98+3DWhney9TCR5+UbJYszqfUUM/ZlsDqHx+M2hgy1f7QWt X-Received: by 2002:a63:f90a:: with SMTP id h10-v6mr1203296pgi.130.1537220178966; Mon, 17 Sep 2018 14:36:18 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:57 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-8-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 7/9] submodule: fetch in submodules git directory instead of in worktree From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch started as a refactoring to make 'get_next_submodule' more readable, but upon doing so, I realized that "git fetch" of the submodule actually doesn't need to be run in the submodules worktree. So let's run it in its git dir instead. That should pave the way towards fetching submodules that are currently not checked out. Signed-off-by: Stefan Beller --- submodule.c | 45 +++++++++++++++++++++++++++---------- t/t5526-fetch-submodules.sh | 7 +++++- 2 files changed, 39 insertions(+), 13 deletions(-) diff --git a/submodule.c b/submodule.c index 00a9a3c6b12..88bce534268 100644 --- a/submodule.c +++ b/submodule.c @@ -481,6 +481,12 @@ void prepare_submodule_repo_env(struct argv_array *out) DEFAULT_GIT_DIR_ENVIRONMENT); } +static void prepare_submodule_repo_env_in_gitdir(struct argv_array *out) +{ + prepare_submodule_repo_env_no_git_dir(out); + argv_array_pushf(out, "%s=.", GIT_DIR_ENVIRONMENT); +} + /* Helper function to display the submodule header line prior to the full * summary output. If it can locate the submodule objects directory it will * attempt to lookup both the left and right commits and put them into the @@ -1227,6 +1233,27 @@ static int get_fetch_recurse_config(const struct submodule *submodule, return spf->default_option; } +static char *get_submodule_git_dir(struct repository *r, const char *path) +{ + struct repository subrepo; + const char *ret; + + if (repo_submodule_init(&subrepo, r, path)) { + /* no entry in .gitmodules? */ + struct strbuf gitdir = STRBUF_INIT; + strbuf_repo_worktree_path(&gitdir, r, "%s/.git", path); + if (repo_init(&subrepo, gitdir.buf, NULL)) { + strbuf_release(&gitdir); + return NULL; + } + } + + ret = xstrdup(subrepo.gitdir); + repo_clear(&subrepo); + + return ret; +} + static int get_next_submodule(struct child_process *cp, struct strbuf *err, void *data, void **task_cb) { @@ -1234,8 +1261,6 @@ static int get_next_submodule(struct child_process *cp, struct submodule_parallel_fetch *spf = data; for (; spf->count < spf->r->index->cache_nr; spf->count++) { - struct strbuf submodule_path = STRBUF_INIT; - struct strbuf submodule_git_dir = STRBUF_INIT; struct strbuf submodule_prefix = STRBUF_INIT; const struct cache_entry *ce = spf->r->index->cache[spf->count]; const char *git_dir, *default_argv; @@ -1274,16 +1299,12 @@ static int get_next_submodule(struct child_process *cp, continue; } - strbuf_repo_worktree_path(&submodule_path, spf->r, "%s", ce->name); - strbuf_addf(&submodule_git_dir, "%s/.git", submodule_path.buf); strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, ce->name); - git_dir = read_gitfile(submodule_git_dir.buf); - if (!git_dir) - git_dir = submodule_git_dir.buf; - if (is_directory(git_dir)) { + git_dir = get_submodule_git_dir(spf->r, ce->name); + if (git_dir) { child_process_init(cp); - cp->dir = strbuf_detach(&submodule_path, NULL); - prepare_submodule_repo_env(&cp->env_array); + prepare_submodule_repo_env_in_gitdir(&cp->env_array); + cp->dir = git_dir; cp->git_cmd = 1; if (!spf->quiet) strbuf_addf(err, "Fetching submodule %s%s\n", @@ -1293,10 +1314,10 @@ static int get_next_submodule(struct child_process *cp, argv_array_push(&cp->args, default_argv); argv_array_push(&cp->args, "--submodule-prefix"); argv_array_push(&cp->args, submodule_prefix.buf); + + free(git_dir); ret = 1; } - strbuf_release(&submodule_path); - strbuf_release(&submodule_git_dir); strbuf_release(&submodule_prefix); if (ret) { spf->count++; diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 6c2f9b2ba26..42692219a1a 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -566,7 +566,12 @@ test_expect_success 'fetching submodule into a broken repository' ' test_must_fail git -C dst status && test_must_fail git -C dst diff && - test_must_fail git -C dst fetch --recurse-submodules + + # git-fetch cannot find the git directory of the submodule, + # so it will do nothing, successfully, as it cannot distinguish between + # this broken submodule and a submodule that was just set active but + # not cloned yet + git -C dst fetch --recurse-submodules ' test_expect_success "fetch new commits when submodule got renamed" ' From patchwork Mon Sep 17 21:35:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603469 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 18F3F6CB for ; Mon, 17 Sep 2018 21:36:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C4082996D for ; Mon, 17 Sep 2018 21:36:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F3ADB2A1D5; Mon, 17 Sep 2018 21:36:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4F9FA2996D for ; Mon, 17 Sep 2018 21:36:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728680AbeIRDFb (ORCPT ); Mon, 17 Sep 2018 23:05:31 -0400 Received: from mail-ua1-f74.google.com ([209.85.222.74]:52549 "EHLO mail-ua1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728564AbeIRDFb (ORCPT ); Mon, 17 Sep 2018 23:05:31 -0400 Received: by mail-ua1-f74.google.com with SMTP id w10-v6so5867765uam.19 for ; Mon, 17 Sep 2018 14:36:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=sGlQMWR/00T+OpNMQAada+MpF2Y0Rc76PclyfAqDQTQ=; b=jNPKAHIsc8Jqqj8/jCACy2GRKFO1SXclCey1qfieySgWzr0vhweacqoLzqeqV9bP/q ddAW6EoJHF4DRSHyWekEK2bI46nhR2tT92SRgTAqg8a0CyGAcMa0QT15r3ABx3et+8w4 3DL9SQfXmE/7+d4G8Zo4Vy0iTjPZGr8jNvAJlO9QUEnOCBnRGcalcEPQrQhAlxH/iDyz iAs9YcACwwIwouDjcqa5L96y+69TfnDh7azNp5XQNaeP40jyAU19UxWraC5WQt+mDSGo Ws7xxjLvJl1a14NFkNtBWH0w0iZk5mqiP39smTIZ8sgnP1Am1boSaoxyhIpGKl4zKdvk pehA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=sGlQMWR/00T+OpNMQAada+MpF2Y0Rc76PclyfAqDQTQ=; b=RCN9vzzrFAYxqp+LWN//WOpM0jfTFY+6m6eNvDHUYKzlaOuDFdnMfrLidjqc9i3waE NlA7rrZ6YqIdHK1Y+eTlzaGUtDypiXDo2q44QsDgRUQmv1uHyxyRnxG98vWuLm6Okhcw Lc/yKIUuAZokN4s4NAj/WEMbx17rxy2BhI4G+/jr2p3nnNJD78Ya8PJ9pI59xBKCFX7g MUKLZqhx8i4q0liF4v4t3adikeBDf7CxLOIItF8axpSdWx2mlGJWWMav9X+BrkbvLWEk AB54tUj2tcNfPe3MlZp1yzYIktN7MEMBkyg/2qCvuswr7YNp8imcvDmAx9p2xLzY1qoI 161w== X-Gm-Message-State: APzg51AeHXWjNsrj7SqurvpetGYEf8scCAPufv+4aJZr/mHCV/5pShR2 JL4VTgYv+9vqAf/oTU8VB/ByZ1rquWLO X-Google-Smtp-Source: ANB0VdaSzL796oNwmoXvXDe7nlmBdHS0pUKwAPeBrzqHn1R5wYeNRlrq5xMGO5GotiamwXN9eY+KZQJvxDnK X-Received: by 2002:a9f:3aca:: with SMTP id q10-v6mr4815597uag.31.1537220181271; Mon, 17 Sep 2018 14:36:21 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:58 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-9-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 8/9] fetch: retry fetching submodules if needed objects were not fetched From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently when git-fetch is asked to recurse into submodules, it dispatches a plain "git-fetch -C " (with some submodule related options such as prefix and recusing strategy, but) without any information of the remote or the tip that should be fetched. This works surprisingly well in some workflows (such as using submodules as a third party library), while not so well in other scenarios, such as in a Gerrit topic-based workflow, that can tie together changes (potentially across repositories) on the server side. One of the parts of such a Gerrit workflow is to download a change when wanting to examine it, and you'd want to have its submodule changes that are in the same topic downloaded as well. However these submodule changes reside in their own repository in their own ref (refs/changes/). Retry fetching a submodule by object name if the object id that the superproject points to, cannot be found. This retrying does not happen when the "git fetch" done at the superproject is not storing the fetched results in remote tracking branches (i.e. instead just recording them to FETCH_HEAD) in this step. A later patch will fix this. builtin/fetch used to only inspect submodules when they were fetched "on-demand", as in either on/off case it was clear whether the submodule needs to be fetched. However to know whether we need to try fetching the object ids, we need to identify the object names, which is done in this function check_for_new_submodule_commits(), so we'll also run that code in case the submodule recursion is set to "on". Signed-off-by: Stefan Beller --- builtin/fetch.c | 9 ++-- submodule.c | 87 ++++++++++++++++++++++++++++++++++++- t/t5526-fetch-submodules.sh | 16 +++++++ 3 files changed, 104 insertions(+), 8 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 61bec5d213d..95c44bf6ffa 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -700,8 +700,7 @@ static int update_local_ref(struct ref *ref, what = _("[new ref]"); } - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref(msg, ref, 0); format_display(display, r ? '!' : '*', what, @@ -716,8 +715,7 @@ static int update_local_ref(struct ref *ref, strbuf_add_unique_abbrev(&quickref, ¤t->object.oid, DEFAULT_ABBREV); strbuf_addstr(&quickref, ".."); strbuf_add_unique_abbrev(&quickref, &ref->new_oid, DEFAULT_ABBREV); - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref("fast-forward", ref, 1); format_display(display, r ? '!' : ' ', quickref.buf, @@ -731,8 +729,7 @@ static int update_local_ref(struct ref *ref, strbuf_add_unique_abbrev(&quickref, ¤t->object.oid, DEFAULT_ABBREV); strbuf_addstr(&quickref, "..."); strbuf_add_unique_abbrev(&quickref, &ref->new_oid, DEFAULT_ABBREV); - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref("forced-update", ref, 1); format_display(display, r ? '!' : '+', quickref.buf, diff --git a/submodule.c b/submodule.c index 88bce534268..7d59e56171f 100644 --- a/submodule.c +++ b/submodule.c @@ -1127,8 +1127,11 @@ struct submodule_parallel_fetch { int result; struct string_list changed_submodule_names; + struct string_list retry; }; -#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, STRING_LIST_INIT_DUP } +#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, \ + STRING_LIST_INIT_DUP, \ + STRING_LIST_INIT_NODUP} static void calculate_changed_submodule_paths( struct submodule_parallel_fetch *spf) @@ -1259,8 +1262,10 @@ static int get_next_submodule(struct child_process *cp, { int ret = 0; struct submodule_parallel_fetch *spf = data; + struct string_list_item *it; for (; spf->count < spf->r->index->cache_nr; spf->count++) { + int recurse_config; struct strbuf submodule_prefix = STRBUF_INIT; const struct cache_entry *ce = spf->r->index->cache[spf->count]; const char *git_dir, *default_argv; @@ -1280,7 +1285,9 @@ static int get_next_submodule(struct child_process *cp, } } - switch (get_fetch_recurse_config(submodule, spf)) + recurse_config = get_fetch_recurse_config(submodule, spf); + + switch (recurse_config) { default: case RECURSE_SUBMODULES_DEFAULT: @@ -1321,9 +1328,50 @@ static int get_next_submodule(struct child_process *cp, strbuf_release(&submodule_prefix); if (ret) { spf->count++; + if (submodule != &default_submodule) + /* discard const-ness: */ + *task_cb = (void*)submodule; return 1; } } + +retry_next: + + if (spf->retry.nr) { + struct strbuf submodule_prefix = STRBUF_INIT; + const struct submodule *sub; + + it = string_list_last(&spf->retry); + sub = submodule_from_name(spf->r, &null_oid, + it->string); + + child_process_init(cp); + cp->dir = get_submodule_git_dir(spf->r, sub->path); + if (!cp->dir) { + string_list_pop(&spf->retry, 0); + goto retry_next; + } + prepare_submodule_repo_env_in_gitdir(&cp->env_array); + cp->git_cmd = 1; + + strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, sub->path); + argv_array_init(&cp->args); + argv_array_pushv(&cp->args, spf->args.argv); + argv_array_push(&cp->args, "on-demand"); + argv_array_push(&cp->args, "--submodule-prefix"); + argv_array_push(&cp->args, submodule_prefix.buf); + + /* NEEDSWORK: have get_default_remote from s--h */ + argv_array_push(&cp->args, "origin"); + oid_array_for_each_unique(it->util, + append_oid_to_argv, &cp->args); + + *task_cb = NULL; /* make sure we do not recurse forever */ + strbuf_release(&submodule_prefix); + string_list_pop(&spf->retry, 0); + return 1; + } + return 0; } @@ -1337,14 +1385,49 @@ static int fetch_start_failure(struct strbuf *err, return 0; } +static int commit_exists_in_sub(const struct object_id *oid, void *data) +{ + struct repository *subrepo = data; + + enum object_type type = oid_object_info(subrepo, oid, NULL); + + return type != OBJ_COMMIT; +} + static int fetch_finish(int retvalue, struct strbuf *err, void *cb, void *task_cb) { struct submodule_parallel_fetch *spf = cb; + struct submodule *sub = task_cb; + struct repository subrepo; if (retvalue) spf->result = 1; + if (!sub) + return 0; + + if (repo_submodule_init(&subrepo, spf->r, sub->path) < 0) + warning(_("Could not get submodule repository for submodule '%s' in repository '%s'"), + sub->path, spf->r->worktree); + else { + struct string_list_item *it; + struct oid_array *commits; + + it = string_list_lookup(&spf->changed_submodule_names, sub->name); + if (!it) + return 0; + + commits = it->util; + oid_array_filter(commits, + commit_exists_in_sub, + &subrepo); + + if (commits->nr) + string_list_append(&spf->retry, sub->name) + ->util = commits; + } + return 0; } diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 42692219a1a..af12c50e7dd 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -605,4 +605,20 @@ test_expect_success "fetch new commits when submodule got renamed" ' test_cmp expect actual ' +test_expect_success "fetch new commits on-demand when they are not reachable" ' + git checkout --detach && + C=$(git -C submodule commit-tree -m "new change outside refs/heads" HEAD^{tree}) && + git -C submodule update-ref refs/changes/1 $C && + git update-index --cacheinfo 160000 $C submodule && + git commit -m "updated submodule outside of refs/heads" && + D=$(git rev-parse HEAD) && + git update-ref refs/changes/2 $D && + ( + cd downstream && + git fetch --recurse-submodules --recurse-submodules-default on-demand origin refs/changes/2:refs/heads/my_branch && + git -C submodule cat-file -t $C && + git checkout --recurse-submodules FETCH_HEAD + ) +' + test_done From patchwork Mon Sep 17 21:35:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10603471 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4CB913AD for ; Mon, 17 Sep 2018 21:36:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9EDE2996D for ; Mon, 17 Sep 2018 21:36:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9E42D2A1D5; Mon, 17 Sep 2018 21:36:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 453B82996D for ; Mon, 17 Sep 2018 21:36:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728681AbeIRDFd (ORCPT ); Mon, 17 Sep 2018 23:05:33 -0400 Received: from mail-it0-f74.google.com ([209.85.214.74]:52160 "EHLO mail-it0-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728564AbeIRDFd (ORCPT ); Mon, 17 Sep 2018 23:05:33 -0400 Received: by mail-it0-f74.google.com with SMTP id q5-v6so368319ith.1 for ; Mon, 17 Sep 2018 14:36:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=WuOjcVomLEraU5DuyDhDDDKiuQEUBzcIXg3M6UrkuCU=; b=OWHnkdDv8LmE7ZYXLHsACl+2jUCXgsY/gSHlK9xqwSyeaRAdXZ4FS+nyS2Ag57yucN Nik56xgpqcMl5K8bAXfDr+7937rauUJoWg+lo8T4Ln4vmmPiolXTZb8Ug6Lv+JwtcDGu UWMQ9IBeDNaYTy+CQGQXu4jFIHov22HAJ1c5xP79hv43tsgDdRsYMXPpEY2QzM6eqNxF JFfzHJleo8s4VN23fq9xIBacXommutjx3MKwdl7ff59EgFOUC1DM3ITbWBLUVUR8Icno 8lLHfLIDLZ+bTh0U0uMbnjZ+4ZjPosAkar9QjJNR5q6wYr9dYAG+7NbMZD3tJrL462kP sSZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=WuOjcVomLEraU5DuyDhDDDKiuQEUBzcIXg3M6UrkuCU=; b=k/m+37bINO1B8YWKjDMBvQkaZFriND9RJIagGfO9zV62YsndrNpRoEDs1ZdBrNFVwG gOssH2h33LWhCVqUlnQedxgI6Gbv850VURNagJWSBCKabtDXlvmv2JCHSdHwFwhV341F vZ0c38pCyYz2jEwoqCxHjVByo22dF1a9OqXmk+GFhdL9tKeumh9Le7+9lhJG0DlwHcU3 pZf8yk8XMUyFi+rm+sOoSezCQqVzqePTiovclDd8aEOuRk2KcLn8vTHY6SYAdDLlgvEB kJxDHC2z1B5X3UqyCgUZqGaEL0H6R84ifsZP5ovUlVyoIbvgBZ/3lgMCfdkanj5Tvmeh 8xJA== X-Gm-Message-State: APzg51DHryaXtn5xeNkSXK4MMQ4gshPAqXNzKd8jKOphjnd4Md3dlN/M 5RCNIEQt22SUwFiuxk88xkUcG88+xNHp X-Google-Smtp-Source: ANB0VdZScbn/xBW1dMBkgDxrEi7IyCduuvn4Zxp1KzeXD4gLoqiXX8SuyhPrBKrKvetoVKZc3Wzmtlk/XaRJ X-Received: by 2002:a24:5953:: with SMTP id p80-v6mr13211081itb.1.1537220183494; Mon, 17 Sep 2018 14:36:23 -0700 (PDT) Date: Mon, 17 Sep 2018 14:35:59 -0700 In-Reply-To: <20180917213559.126404-1-sbeller@google.com> Message-Id: <20180917213559.126404-10-sbeller@google.com> Mime-Version: 1.0 References: <20180911234951.14129-1-sbeller@google.com> <20180917213559.126404-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0.444.g18242da7ef-goog Subject: [PATCH 9/9] builtin/fetch: check for submodule updates for non branch fetches From: Stefan Beller To: sbeller@google.com Cc: git@vger.kernel.org Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Gerrit, the code review tool, has a different workflow than our mailing list based approach. Usually users upload changes to a Gerrit server and continuous integration and testing happens by bots. Sometimes however a user wants to checkout a change locally and look at it locally. For this use case, Gerrit offers a command line snippet to copy and paste to your terminal, which looks like git fetch https:///gerrit refs/changes/ && git checkout FETCH_HEAD For Gerrit changes that contain changing submodule gitlinks, it would be easy to extend both the fetch and checkout with the '--recurse-submodules' flag, such that this command line snippet would produce the state of a change locally. However the functionality added in the previous patch, which would ensure that we fetch the objects in the submodule that the gitlink pointed at, only works for remote tracking branches so far, not for FETCH_HEAD. Make sure that fetching a superproject to its FETCH_HEAD, also respects the existence checks for objects in the submodule recursion. Signed-off-by: Stefan Beller --- builtin/fetch.c | 5 ++++- t/t5526-fetch-submodules.sh | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 95c44bf6ffa..ea6ecd123e7 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -887,11 +887,14 @@ static int store_updated_refs(const char *raw_url, const char *remote_name, rc |= update_local_ref(ref, what, rm, ¬e, summary_width); free(ref); - } else + } else { + check_for_new_submodule_commits(&rm->old_oid); format_display(¬e, '*', *kind ? kind : "branch", NULL, *what ? what : "HEAD", "FETCH_HEAD", summary_width); + } + if (note.len) { if (verbosity >= 0 && !shown_url) { fprintf(stderr, _("From %.*s\n"), diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index af12c50e7dd..a509eabb044 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -615,7 +615,7 @@ test_expect_success "fetch new commits on-demand when they are not reachable" ' git update-ref refs/changes/2 $D && ( cd downstream && - git fetch --recurse-submodules --recurse-submodules-default on-demand origin refs/changes/2:refs/heads/my_branch && + git fetch --recurse-submodules origin refs/changes/2 && git -C submodule cat-file -t $C && git checkout --recurse-submodules FETCH_HEAD )