From patchwork Tue Jun 25 13:40:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015577 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 502871575 for ; Tue, 25 Jun 2019 13:41:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 42E7C284C9 for ; Tue, 25 Jun 2019 13:41:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 369AD286E5; Tue, 25 Jun 2019 13:41:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D19B928047 for ; Tue, 25 Jun 2019 13:41:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730029AbfFYNlJ (ORCPT ); Tue, 25 Jun 2019 09:41:09 -0400 Received: from mail-ed1-f68.google.com ([209.85.208.68]:46257 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727975AbfFYNlJ (ORCPT ); Tue, 25 Jun 2019 09:41:09 -0400 Received: by mail-ed1-f68.google.com with SMTP id d4so27161751edr.13 for ; Tue, 25 Jun 2019 06:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vYCstU0Bggt562mLgs8/RcIkj1w9HUhQa5RxlH4Ebtw=; b=R6lY8vnWMXXJYVP31VdfOvb/2lEQGMrbdwpQ0Nis9gP8zaZOSmlQ2DaQYiMalqT0dF 1I/e1+tlOlyLibQLrMGBhAMvdBYmk5O5ouQC2KCrf12EKCagpFffQRRpLnN0nyUaXBhn hWX+O4y7VUBcx4ZknEzf4N3TjBFNk4gDufCb82rMXoRKAMsx0XBMXlAXuzd5YSv6sJRN XocddED1XfSs9m/I6pGpa4QGNVJMgJiNvNsfCB8S113sCqeQx1eAeU4ar5WvBVJE6YCr QMMsW1R/chc2WLcxjqvlLaoM+zJ26EUodcxXblAwPFFzBy4NKlVw98dkm9YrXg+aJ++H NTLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vYCstU0Bggt562mLgs8/RcIkj1w9HUhQa5RxlH4Ebtw=; b=d3E2EEJLk4l97IZz/iZD4I0ZYQRU0AYibonahvNPrO7S6gmaAYxJ7GMcDC5phP5O69 Giv0TlhuSG9CympoKVc/ALkTvDIO2zi1LslQUiNN1XHhnqIuxvxoTscA6c4pgEOIa05n lSWiRii+Suy8qdGnPkbUJKNuOFq6+hQBJv6zenWLSRU3Lv+cVvMtVuzYfS1DuxqPCbRH aw83/BZGuc2nzhaVpUU5aulf5yA1zr4zp54UsgrXhJ+/eCnbU7E3Scvv2ZTD3GGUd2ZR ZDU1T5mYU0F85N6FnwFGoPorf3Mr//8FlYY4mNzU7z1Jprr3LeFZnvIcGUUB8kuinlQu J/ow== X-Gm-Message-State: APjAAAWNjjtFVJ3JFdBjnAtPuaMtv9aaP52ezvLaTz7j6rusBaxb5J+r XoIYvYPc7jW1Ho9ac5JU0C9bomYgqEs= X-Google-Smtp-Source: APXvYqw8pPRBw51BZPnkJjCNKYaDLFAswaWF4BmLQyxi43oWhww8u3Y8nLLnsfXfB1jKHacactTq6g== X-Received: by 2002:a50:90af:: with SMTP id c44mr143020229eda.126.1561470066668; Tue, 25 Jun 2019 06:41:06 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.05 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:05 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 01/15] t0410: remove pipes after git commands Date: Tue, 25 Jun 2019 15:40:25 +0200 Message-Id: <20190625134039.21707-2-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Let's not run a git command, especially one with "verify" in its name, upstream of a pipe, because the pipe will hide the git command's exit code. While at it, let's also avoid a useless `cat` command piping into `sed`. Helped-by: SZEDER Gábor Signed-off-by: Christian Couder --- t/t0410-partial-clone.sh | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh index 5bd892f2f7..3559313bd0 100755 --- a/t/t0410-partial-clone.sh +++ b/t/t0410-partial-clone.sh @@ -166,8 +166,9 @@ test_expect_success 'fetching of missing objects' ' # associated packfile contains the object ls repo/.git/objects/pack/pack-*.promisor >promisorlist && test_line_count = 1 promisorlist && - IDX=$(cat promisorlist | sed "s/promisor$/idx/") && - git verify-pack --verbose "$IDX" | grep "$HASH" + IDX=$(sed "s/promisor$/idx/" promisorlist) && + git verify-pack --verbose "$IDX" >out && + grep "$HASH" out ' test_expect_success 'fetching of missing objects works with ref-in-want enabled' ' @@ -514,8 +515,9 @@ test_expect_success 'fetching of missing objects from an HTTP server' ' # associated packfile contains the object ls repo/.git/objects/pack/pack-*.promisor >promisorlist && test_line_count = 1 promisorlist && - IDX=$(cat promisorlist | sed "s/promisor$/idx/") && - git verify-pack --verbose "$IDX" | grep "$HASH" + IDX=$(sed "s/promisor$/idx/" promisorlist) && + git verify-pack --verbose "$IDX" >out && + grep "$HASH" out ' test_done From patchwork Tue Jun 25 13:40:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015579 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5415112C for ; Tue, 25 Jun 2019 13:41:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 97AF628047 for ; Tue, 25 Jun 2019 13:41:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8BCED284C9; Tue, 25 Jun 2019 13:41:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4A4D2852A for ; Tue, 25 Jun 2019 13:41:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730145AbfFYNlL (ORCPT ); Tue, 25 Jun 2019 09:41:11 -0400 Received: from mail-ed1-f65.google.com ([209.85.208.65]:37158 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729846AbfFYNlJ (ORCPT ); Tue, 25 Jun 2019 09:41:09 -0400 Received: by mail-ed1-f65.google.com with SMTP id w13so27250282eds.4 for ; Tue, 25 Jun 2019 06:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hKffiI0+5tfnysJsQyQSk/CillMAkAJj8w4TB7sLADQ=; b=tjk546hVBf3QSc5Yc8HR05oBeCkuAzohDgFhooZc4dHhIefD2frG/eMjeQMjxlp+wB ZsPjXRZBE2535hw2IpdF8l98eXC6k8u2vdAK049/TayZEo3qHKDBydYMcDu/pLK78lzR fLJ/ayGEvpWPbK8Sv/l1rVLNXB38Z+uSfoGPq2qSUh2ZkIfEMqrjRLbp/xvd2ZisZiPi YkKyqkelAcX2yH6thfd8mMrcBYxCH8fW+qnhXzJ6dXwIVu4FuYkqBCqUKc36m+WG2g2o f/AY3zRYCfewQaWIfTk2OX5cex4bL+JBAebvCNa3xZg7ChYPBYe0w/JuSNKCK0SSIiU9 v0pA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hKffiI0+5tfnysJsQyQSk/CillMAkAJj8w4TB7sLADQ=; b=pAt8mfw7k55nU4799wwEyFc0Secft4aHwxdtmfXDjnpbmKiHamYMY5QsL3bi/Zrd/r YEXXn3bstvby6kiEFSvGzVBbsn4HzVtptKTdIgx6Lp3yx03IIK47gb53QVArNAwrsZEl TeaB979S1Fn5mcaSSAs5TRmSp/AnjxGSQ0mTStjYLxVbA4Bedwbdc7BSUfAYTsl78JW5 pYThmBjGj0EFpKaiZR/V8FgHi/XkLnh8wdoFYJpCGMm4LPIn91wcmh7GMRcH31kTlwhy q/1CcvEpGKBVFogYcjydn4G1e0OVkNJRidB+/74ji4L1aYKBpJAQzXl+zaPOAXj/Hu06 4xPw== X-Gm-Message-State: APjAAAXFfoAMWIy05opPGxMdv6wqjZbCEbGXpcauUVDioMjYwTT4XiiU 4h+AUVYDmubK2H9iK2PVrFBTJaMxaio= X-Google-Smtp-Source: APXvYqyxZr/cfvGghIg5gDOm4KiAcOy511p5W0v1y0d/RfOO5J0AQpVbbnz4FwanT+0AW540bsww7Q== X-Received: by 2002:a50:ad45:: with SMTP id z5mr148602841edc.21.1561470067829; Tue, 25 Jun 2019 06:41:07 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.06 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:07 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee , Christian Couder Subject: [PATCH v6 02/15] fetch-object: make functions return an error code Date: Tue, 25 Jun 2019 15:40:26 +0200 Message-Id: <20190625134039.21707-3-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christian Couder The callers of the fetch_object() and fetch_objects() might be interested in knowing if these functions succeeded or not. Signed-off-by: Christian Couder Signed-off-by: Junio C Hamano --- fetch-object.c | 13 ++++++++----- fetch-object.h | 4 ++-- sha1-file.c | 4 ++-- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/fetch-object.c b/fetch-object.c index 4266548800..eac4d448ef 100644 --- a/fetch-object.c +++ b/fetch-object.c @@ -5,11 +5,12 @@ #include "transport.h" #include "fetch-object.h" -static void fetch_refs(const char *remote_name, struct ref *ref) +static int fetch_refs(const char *remote_name, struct ref *ref) { struct remote *remote; struct transport *transport; int original_fetch_if_missing = fetch_if_missing; + int res; fetch_if_missing = 0; remote = remote_get(remote_name); @@ -19,12 +20,14 @@ static void fetch_refs(const char *remote_name, struct ref *ref) transport_set_option(transport, TRANS_OPT_FROM_PROMISOR, "1"); transport_set_option(transport, TRANS_OPT_NO_DEPENDENTS, "1"); - transport_fetch_refs(transport, ref); + res = transport_fetch_refs(transport, ref); fetch_if_missing = original_fetch_if_missing; + + return res; } -void fetch_objects(const char *remote_name, const struct object_id *oids, - int oid_nr) +int fetch_objects(const char *remote_name, const struct object_id *oids, + int oid_nr) { struct ref *ref = NULL; int i; @@ -36,5 +39,5 @@ void fetch_objects(const char *remote_name, const struct object_id *oids, new_ref->next = ref; ref = new_ref; } - fetch_refs(remote_name, ref); + return fetch_refs(remote_name, ref); } diff --git a/fetch-object.h b/fetch-object.h index d6444caa5a..7bcc7cadb0 100644 --- a/fetch-object.h +++ b/fetch-object.h @@ -3,7 +3,7 @@ struct object_id; -void fetch_objects(const char *remote_name, const struct object_id *oids, - int oid_nr); +int fetch_objects(const char *remote_name, const struct object_id *oids, + int oid_nr); #endif diff --git a/sha1-file.c b/sha1-file.c index 888b6024d5..819d32cdb8 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -1381,8 +1381,8 @@ int oid_object_info_extended(struct repository *r, const struct object_id *oid, !already_retried && r == the_repository && !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) { /* - * TODO Investigate having fetch_object() return - * TODO error/success and stopping the music here. + * TODO Investigate checking fetch_object() return + * TODO value and stopping on error here. * TODO Pass a repository struct through fetch_object, * such that arbitrary repositories work. */ From patchwork Tue Jun 25 13:40:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015581 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1EF2C1575 for ; Tue, 25 Jun 2019 13:41:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0F45E2848F for ; Tue, 25 Jun 2019 13:41:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 03B882852A; Tue, 25 Jun 2019 13:41:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88028284C9 for ; Tue, 25 Jun 2019 13:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727177AbfFYNlM (ORCPT ); Tue, 25 Jun 2019 09:41:12 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:43626 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727975AbfFYNlL (ORCPT ); Tue, 25 Jun 2019 09:41:11 -0400 Received: by mail-ed1-f67.google.com with SMTP id e3so27164556edr.10 for ; Tue, 25 Jun 2019 06:41:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Nwn5LqtJPnLfB1W/n67QhtQIQYjR44RipOuBBt93sys=; b=MR9rmF86F1Siawr6j85Fa/BypsaOvlKUqhksEPOE3+hnOOZYwGhrU1ATAvz62mEV+i s/jW/WC6vntPe9jaK13zNrEs/YXxkWm1POSPY84iwTPX4Mss3QE3aSPHNVYgJXw7szvT 4BzZ8hHO0Gl30aeOvqPgSXOdu4306uy2xi/ZTJTpbpLEnNx4IPc+ZpZoqSYOmJnmnyrV rZlOkM0AU0Tbz37z+BuOtDGWnHcCIS1PUYUeNdH7Y+DOuzCrYgVf0b5rmb5bkeIE1Tkp /v2e0XDNWjkzi8niOCQsntBCntyfTW8bcrJIkTfXGY9VHV4sJqCxPW3P5a9y9BJRB31i vxRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Nwn5LqtJPnLfB1W/n67QhtQIQYjR44RipOuBBt93sys=; b=R2en0wNxq7nglfZrRwtEMj3R4sXRN64oYJ/HEmO4QVl7sGvb2PB077860KxHZsRrQP ZtRqWRkBoqIbEIoOC9wYzkEkUfLfr8bkdMjfC/qGQN0bbqNpSsS/Oy+wCrU+jexIlzFH C6qObikneQX1BqIRrKv2CV+7BxNyd+UYWneB1afGpCppiSoWpGYwgqhQapjJobWI9vpM hAOC8BQt3oR/NV+z58fg5YfAjcYZoy8os9KWwdjn6GPhQfWN/eJIiotNB3XBQyl/NmK5 qJ9i5siQlOkoWPESDoOF9tSY6JqRaCFj/UVcIAIzo/RQO1CpW/HTG4WafxeRzvTZtsmX 4pcg== X-Gm-Message-State: APjAAAVZGoMkfSnzSACEm3brOgitHS0tANDvK0QIdW8Hd4TU/y8QC9tq DsOtZModU1lnR4LM8vcTnF99SfuLMZQ= X-Google-Smtp-Source: APXvYqyyLi2TaERZlxM4KtS3cnYG6mpemRoWnWzTM6YfO6dtnnIT8ZzpFRLdzOJCc/xu/I42D0rsOw== X-Received: by 2002:a50:940b:: with SMTP id p11mr61361756eda.194.1561470068946; Tue, 25 Jun 2019 06:41:08 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.07 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:08 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee , Christian Couder Subject: [PATCH v6 03/15] Add initial support for many promisor remotes Date: Tue, 25 Jun 2019 15:40:27 +0200 Message-Id: <20190625134039.21707-4-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christian Couder The promisor-remote.{c,h} files will contain functions to manage many promisor remotes. We expect that there will not be a lot of promisor remotes, so it is ok to use a simple linked list to manage them. Helped-by: Jeff King Helped-by: SZEDER Gábor Signed-off-by: Christian Couder --- Makefile | 1 + promisor-remote.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++ promisor-remote.h | 16 +++++++++ 3 files changed, 109 insertions(+) create mode 100644 promisor-remote.c create mode 100644 promisor-remote.h diff --git a/Makefile b/Makefile index f58bf14c7b..049bc8cfd4 100644 --- a/Makefile +++ b/Makefile @@ -944,6 +944,7 @@ LIB_OBJS += preload-index.o LIB_OBJS += pretty.o LIB_OBJS += prio-queue.o LIB_OBJS += progress.o +LIB_OBJS += promisor-remote.o LIB_OBJS += prompt.o LIB_OBJS += protocol.o LIB_OBJS += quote.o diff --git a/promisor-remote.c b/promisor-remote.c new file mode 100644 index 0000000000..c249b80e02 --- /dev/null +++ b/promisor-remote.c @@ -0,0 +1,92 @@ +#include "cache.h" +#include "promisor-remote.h" +#include "config.h" + +static struct promisor_remote *promisors; +static struct promisor_remote **promisors_tail = &promisors; + +static struct promisor_remote *promisor_remote_new(const char *remote_name) +{ + struct promisor_remote *r; + + if (*remote_name == '/') { + warning(_("promisor remote name cannot begin with '/': %s"), + remote_name); + return NULL; + } + + FLEX_ALLOC_STR(r, name, remote_name); + + *promisors_tail = r; + promisors_tail = &r->next; + + return r; +} + +static struct promisor_remote *promisor_remote_lookup(const char *remote_name, + struct promisor_remote **previous) +{ + struct promisor_remote *r, *p; + + for (p = NULL, r = promisors; r; p = r, r = r->next) + if (!strcmp(r->name, remote_name)) { + if (previous) + *previous = p; + return r; + } + + return NULL; +} + +static int promisor_remote_config(const char *var, const char *value, void *data) +{ + const char *name; + int namelen; + const char *subkey; + + if (parse_config_key(var, "remote", &name, &namelen, &subkey) < 0) + return 0; + + if (!strcmp(subkey, "promisor")) { + char *remote_name; + + if (!git_config_bool(var, value)) + return 0; + + remote_name = xmemdupz(name, namelen); + + if (!promisor_remote_lookup(remote_name, NULL)) + promisor_remote_new(remote_name); + + free(remote_name); + return 0; + } + + return 0; +} + +static void promisor_remote_init(void) +{ + static int initialized; + + if (initialized) + return; + initialized = 1; + + git_config(promisor_remote_config, NULL); +} + +struct promisor_remote *promisor_remote_find(const char *remote_name) +{ + promisor_remote_init(); + + if (!remote_name) + return promisors; + + return promisor_remote_lookup(remote_name, NULL); +} + +int has_promisor_remote(void) +{ + return !!promisor_remote_find(NULL); +} diff --git a/promisor-remote.h b/promisor-remote.h new file mode 100644 index 0000000000..01dcdf4dc7 --- /dev/null +++ b/promisor-remote.h @@ -0,0 +1,16 @@ +#ifndef PROMISOR_REMOTE_H +#define PROMISOR_REMOTE_H + +/* + * Promisor remote linked list + * Its information come from remote.XXX config entries. + */ +struct promisor_remote { + struct promisor_remote *next; + const char name[FLEX_ARRAY]; +}; + +extern struct promisor_remote *promisor_remote_find(const char *remote_name); +extern int has_promisor_remote(void); + +#endif /* PROMISOR_REMOTE_H */ From patchwork Tue Jun 25 13:40:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015587 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22EC7112C for ; Tue, 25 Jun 2019 13:41:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 151F6205F7 for ; Tue, 25 Jun 2019 13:41:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 07E48286E5; Tue, 25 Jun 2019 13:41:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6FFCA205F7 for ; Tue, 25 Jun 2019 13:41:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730018AbfFYNlP (ORCPT ); Tue, 25 Jun 2019 09:41:15 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:38422 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727770AbfFYNlM (ORCPT ); Tue, 25 Jun 2019 09:41:12 -0400 Received: by mail-ed1-f66.google.com with SMTP id r12so27259620edo.5 for ; Tue, 25 Jun 2019 06:41:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v7xYUgFCv1yB8odZItBcl+uRJGhBZUO/PWSuRmjzAqI=; b=uMlu8z7AR8bhTad0frCdbijRenOyV+Eyq9lEjt5lt6ov64VBCTVfzcHdFgh0TSCYVt l4MBcJfQycICgFv8kIJe/sdQU4QT5aOnw1Y3XbecCuBzPOivf+OKHR1Q5YeTFyVrIy7y C1pCgP4OHe5pfogrFzqOxLbCFUpG0RauETzmoPFXoL4YkNM1OMiYY4pHSEEl/ovTiHhn IOhccxsIeIpI6S8IosEcJUIsYMBFwbM/DJ+5dErV5AGiSuknrXSSE7+47ZCfQjFddyh+ 6v0EsxAfoFsf0+eeXJrA+mJMPj43M86bqY26PPnkuQ4QQ2BW+87j1Lm2BXAiqG07boAz H0cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v7xYUgFCv1yB8odZItBcl+uRJGhBZUO/PWSuRmjzAqI=; b=TBoe3374W5R8m7GDpNJC22azBmQhPnnAadFc1nsS5lhKr2rgy3PcQhjGrHVeMSPKoc aSN2hvnXao52dDjV9MgLbq+jjXlR3AoQnEa2lP0i7ZhuwHfYpEfH1FIUi+DnxNf8SUhs lMyYX8hdID3D5A3ccbB92n07nnVfVkujb2P2rvCQufO8cmPhz94Jz3n5cY+UW+D/VK+R sthNK/f9733M+3YcMOTJwTvUP3WebmGFJSM6oV0sOv6yXjXCtCtIWbSWWzReQ7wauTMd 6X9BqBv9c84Z+PWLLzSUjjYFRPl1ZT8CN78eBMjSDAcoSypSXHLXKFUWqYnuSvP8RiQ2 gfnQ== X-Gm-Message-State: APjAAAV4DhJEZiLt1raAWoRVMiuYN1Ez5iOK/fRlI6wtxD0nsXf53xyK J9Os2Dc4vemM3D5T7nz3Y/Z31Ac9k7k= X-Google-Smtp-Source: APXvYqzWKrg4/dhe8G0L3R7sWX4+ttwyAed8KASazyLg7IaCJv7RA38PADNaHve9Ac5VYaxzCnnENQ== X-Received: by 2002:a50:fd0c:: with SMTP id i12mr82974221eds.55.1561470070138; Tue, 25 Jun 2019 06:41:10 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.08 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:09 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee , Christian Couder Subject: [PATCH v6 04/15] promisor-remote: implement promisor_remote_get_direct() Date: Tue, 25 Jun 2019 15:40:28 +0200 Message-Id: <20190625134039.21707-5-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christian Couder This is implemented for now by calling fetch_objects(). It fetches from all the promisor remotes. Helped-by: Ramsay Jones Helped-by: Derrick Stolee Signed-off-by: Christian Couder --- promisor-remote.c | 67 +++++++++++++++++++++++++++++++++++++++++++++++ promisor-remote.h | 5 ++++ 2 files changed, 72 insertions(+) diff --git a/promisor-remote.c b/promisor-remote.c index c249b80e02..b79a84ce3a 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -1,6 +1,8 @@ #include "cache.h" +#include "object-store.h" #include "promisor-remote.h" #include "config.h" +#include "fetch-object.h" static struct promisor_remote *promisors; static struct promisor_remote **promisors_tail = &promisors; @@ -90,3 +92,68 @@ int has_promisor_remote(void) { return !!promisor_remote_find(NULL); } + +static int remove_fetched_oids(struct repository *repo, + struct object_id **oids, + int oid_nr, int to_free) +{ + int i, remaining_nr = 0; + int *remaining = xcalloc(oid_nr, sizeof(*remaining)); + struct object_id *old_oids = *oids; + struct object_id *new_oids; + + for (i = 0; i < oid_nr; i++) + if (oid_object_info_extended(repo, &old_oids[i], NULL, + OBJECT_INFO_SKIP_FETCH_OBJECT)) { + remaining[i] = 1; + remaining_nr++; + } + + if (remaining_nr) { + int j = 0; + new_oids = xcalloc(remaining_nr, sizeof(*new_oids)); + for (i = 0; i < oid_nr; i++) + if (remaining[i]) + oidcpy(&new_oids[j++], &old_oids[i]); + *oids = new_oids; + if (to_free) + free(old_oids); + } + + free(remaining); + + return remaining_nr; +} + +int promisor_remote_get_direct(struct repository *repo, + const struct object_id *oids, + int oid_nr) +{ + struct promisor_remote *r; + struct object_id *remaining_oids = (struct object_id *)oids; + int remaining_nr = oid_nr; + int to_free = 0; + int res = -1; + + promisor_remote_init(); + + for (r = promisors; r; r = r->next) { + if (fetch_objects(r->name, remaining_oids, remaining_nr) < 0) { + if (remaining_nr == 1) + continue; + remaining_nr = remove_fetched_oids(repo, &remaining_oids, + remaining_nr, to_free); + if (remaining_nr) { + to_free = 1; + continue; + } + } + res = 0; + break; + } + + if (to_free) + free(remaining_oids); + + return res; +} diff --git a/promisor-remote.h b/promisor-remote.h index 01dcdf4dc7..ed4ecead36 100644 --- a/promisor-remote.h +++ b/promisor-remote.h @@ -1,6 +1,8 @@ #ifndef PROMISOR_REMOTE_H #define PROMISOR_REMOTE_H +struct object_id; + /* * Promisor remote linked list * Its information come from remote.XXX config entries. @@ -12,5 +14,8 @@ struct promisor_remote { extern struct promisor_remote *promisor_remote_find(const char *remote_name); extern int has_promisor_remote(void); +extern int promisor_remote_get_direct(struct repository *repo, + const struct object_id *oids, + int oid_nr); #endif /* PROMISOR_REMOTE_H */ From patchwork Tue Jun 25 13:40:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015583 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9AABC1575 for ; Tue, 25 Jun 2019 13:41:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D456205F7 for ; Tue, 25 Jun 2019 13:41:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 81F9F2848F; Tue, 25 Jun 2019 13:41:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 303EE285A3 for ; Tue, 25 Jun 2019 13:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730175AbfFYNlP (ORCPT ); Tue, 25 Jun 2019 09:41:15 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:36140 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729846AbfFYNlN (ORCPT ); Tue, 25 Jun 2019 09:41:13 -0400 Received: by mail-ed1-f66.google.com with SMTP id k21so27254720edq.3 for ; Tue, 25 Jun 2019 06:41:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r+faeZ4bLagCledobRR4n/1KI7yqRn5WZuiC2MgIos4=; b=XN4sZSA70mCfAqzH2vHW9evsYRUDW3lfZAUigXixcaGY9zaB1Ktq+K/o3bx+dXStlm ZZ1zbQHW/fVHRzSNVB0c+wrwwC15xeVATL8Q5yvunld0Mdf6bYEEYeGKIG8/KBjgu7pd 3Zrxn/ThCNedzSzlCsOygAp8G6JY1Qa6RtEQQMOj+k/gQYgh43pmsa9UryNfk/zRhAXV dMUbLUEGWdHy7TuUwWiK1VqgSM167FxyvgLsEC2Gol2u6tSGgFX+EQNjI6g9uNN2b+V5 MXfTjMKQsq4lOUXmo6VR6diXYB0glNX/nAF2yw36XW6yivHHcUGLBDxnwIGV3cucttAs ZMYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r+faeZ4bLagCledobRR4n/1KI7yqRn5WZuiC2MgIos4=; b=p8dcME/cqyxEoJefdmGM/EvpRCB8it/uYXNU4fof848soG1a3F0JYa/wpH3C43C/CH 6yjyeUn7U+rngvNyFnqO2UsFselbF2dcvd+v7MFoLcpcerG1NPzUGRWpmSQr330pUp/L 0sGpzxj6URKFPBv9A2NetPzNXnooB/dhhlpIGDwH6NvjOwQ86O0fyQw89Ehew7vh1Iu4 dozoHYgn9+HQtzaAD6j0oJv6yYc2cKsdOorVigOu7PI5NjMHJIJFrKGC69FibjeEQpaZ YHZ8VKnvaRWPOEhaBoEQkVrVuD3R6RfVG0OGV1IwCuR5qlQwZ4m3jJ2IZYjA1JsYd8UJ Ijbw== X-Gm-Message-State: APjAAAV63wir6nRLD+K+phuu84B4+DMlmVFUz20vynLqNOmiUYps5rUI xOaZwa55c6QhXOAnqByuT5oa6I1Xp7k= X-Google-Smtp-Source: APXvYqxomcIAedpLgbG2w+uONNA8RzO2zO7Tkgw8HaZAqtZlSQPFZaeErWNcPJaPTM5Q/G7ZD2tW5A== X-Received: by 2002:a17:906:9147:: with SMTP id y7mr55265368ejw.66.1561470071255; Tue, 25 Jun 2019 06:41:11 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.10 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:10 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee , Christian Couder Subject: [PATCH v6 05/15] promisor-remote: add promisor_remote_reinit() Date: Tue, 25 Jun 2019 15:40:29 +0200 Message-Id: <20190625134039.21707-6-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christian Couder We will need to reinitialize the promisor remote configuration as we will make some changes to it in a later commit. Signed-off-by: Christian Couder --- promisor-remote.c | 22 ++++++++++++++++++++-- promisor-remote.h | 1 + 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/promisor-remote.c b/promisor-remote.c index b79a84ce3a..763d98aedd 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -67,10 +67,10 @@ static int promisor_remote_config(const char *var, const char *value, void *data return 0; } +static int initialized; + static void promisor_remote_init(void) { - static int initialized; - if (initialized) return; initialized = 1; @@ -78,6 +78,24 @@ static void promisor_remote_init(void) git_config(promisor_remote_config, NULL); } +static void promisor_remote_clear(void) +{ + while (promisors) { + struct promisor_remote *r = promisors; + promisors = promisors->next; + free(r); + } + + promisors_tail = &promisors; +} + +void promisor_remote_reinit(void) +{ + initialized = 0; + promisor_remote_clear(); + promisor_remote_init(); +} + struct promisor_remote *promisor_remote_find(const char *remote_name) { promisor_remote_init(); diff --git a/promisor-remote.h b/promisor-remote.h index ed4ecead36..dddd4048e0 100644 --- a/promisor-remote.h +++ b/promisor-remote.h @@ -12,6 +12,7 @@ struct promisor_remote { const char name[FLEX_ARRAY]; }; +extern void promisor_remote_reinit(void); extern struct promisor_remote *promisor_remote_find(const char *remote_name); extern int has_promisor_remote(void); extern int promisor_remote_get_direct(struct repository *repo, From patchwork Tue Jun 25 13:40:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015585 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19B5A112C for ; Tue, 25 Jun 2019 13:41:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0CB3E285A3 for ; Tue, 25 Jun 2019 13:41:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 00F0A286E5; Tue, 25 Jun 2019 13:41:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99AC9285A3 for ; Tue, 25 Jun 2019 13:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730180AbfFYNlP (ORCPT ); Tue, 25 Jun 2019 09:41:15 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:38427 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727975AbfFYNlO (ORCPT ); Tue, 25 Jun 2019 09:41:14 -0400 Received: by mail-ed1-f66.google.com with SMTP id r12so27259768edo.5 for ; Tue, 25 Jun 2019 06:41:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VPLkInReMhBYY7H2msBeY0ml8DrmkLjlaT9/hjyg7ZY=; b=F/aesQk4tfwKEu5J3T55shp30WHx2B9Cjjcod5ic/vzUvPAXdBwy+5O9Przbop3QiS qBN+fjaf42f6r+6UbIWMmD8Y1UwGuBzcrU2fent9Zd1hQSET5u/FRe/cgjmtqw3SuvkH vcnRsQOeAfihm1q2t0GZSnD+2G7UwrJjvAVvtAPn7MXNegrJWTjsScxRhe5/aHvJq/ko zqp9R3v3v/uq4K9/g8Bvo1xvsRpKoNhmRgToD3V8cs24HQPXZEqImAs4N2neqsxj1c8n tNZ8R+LxGnq53VG1IeyjBlYH75TLSEjFq1wTpqo2RtyIANbLXZLtF11iA/oMShfJmlbu jDDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VPLkInReMhBYY7H2msBeY0ml8DrmkLjlaT9/hjyg7ZY=; b=a3REhlpskmvBCPIqRXaglADAlqtAqqYtBhMHpq53LnSL7XcN/ms6RiGFk+ZcjKwPfx +C0EwnFStFlUsasQ3cHmACm9pqcJABgECYZXZx4mhLKXVYGwIoH8DG4C7uq8s/gWTewm 3F/sLKeu4V1ZCcNQsMwizSOcVgeQTfn4L2FajocY2qUt1ojGBIzcgMw8huNlS661wG0T Svvt9R0MZGepTp+zcK1EBfAjItpDrFz9TMPM5uuDnBBDMMWw5rLqMR3oXEIgqEX33r1I UCY0tDDq51SdUox9xMrdoLWm6mtnF1KwlDDrCylgdHsyyx4ZnujYtL2hopsW/J1J9okT VhTw== X-Gm-Message-State: APjAAAWLjtiQ1lZIQLXifU9Q1VVsFF9KxAZkqXVctZtjZgeE3bWz5r/Y YAnsFX+2dcmqpAcmFZKO3L8WZg3sC/U= X-Google-Smtp-Source: APXvYqw/jsr9bx78/LHTEeDPb3Wa1nqJhg2eB7vUn33xtlrcNeEYZTFUc3zTSd/pfjBpsShY2dysBw== X-Received: by 2002:a17:906:5008:: with SMTP id s8mr99228176ejj.308.1561470072228; Tue, 25 Jun 2019 06:41:12 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.11 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:11 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 06/15] promisor-remote: use repository_format_partial_clone Date: Tue, 25 Jun 2019 15:40:30 +0200 Message-Id: <20190625134039.21707-7-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A remote specified using the extensions.partialClone config option should be considered a promisor remote too. For simplicity and to make things predictable, this promisor remote should be either always the last one we try to get objects from, or the first one. So it should always be either at the end of the promisor remote list, or at its start. We decided to make it the last one we try, because it is likely that someone using many promisor remotes is doing so because the other promisor remotes are better for some reason (maybe they are closer or faster for some kind of objects) than the origin, and the origin is likely to be the remote specified by extensions.partialClone. This justification is not very strong, but one choice had to be made, and anyway the long term plan should be to make the order somehow fully configurable. Signed-off-by: Christian Couder --- promisor-remote.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/promisor-remote.c b/promisor-remote.c index 763d98aedd..6a8856f475 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -40,6 +40,18 @@ static struct promisor_remote *promisor_remote_lookup(const char *remote_name, return NULL; } +static void promisor_remote_move_to_tail(struct promisor_remote *r, + struct promisor_remote *previous) +{ + if (previous) + previous->next = r->next; + else + promisors = r->next ? r->next : r; + r->next = NULL; + *promisors_tail = r; + promisors_tail = &r->next; +} + static int promisor_remote_config(const char *var, const char *value, void *data) { const char *name; @@ -76,6 +88,17 @@ static void promisor_remote_init(void) initialized = 1; git_config(promisor_remote_config, NULL); + + if (repository_format_partial_clone) { + struct promisor_remote *o, *previous; + + o = promisor_remote_lookup(repository_format_partial_clone, + &previous); + if (o) + promisor_remote_move_to_tail(o, previous); + else + promisor_remote_new(repository_format_partial_clone); + } } static void promisor_remote_clear(void) From patchwork Tue Jun 25 13:40:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015609 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE3A6112C for ; Tue, 25 Jun 2019 13:42:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFC61285A3 for ; Tue, 25 Jun 2019 13:42:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D43052873A; Tue, 25 Jun 2019 13:42:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B83D1286C6 for ; Tue, 25 Jun 2019 13:42:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730264AbfFYNlR (ORCPT ); Tue, 25 Jun 2019 09:41:17 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:44667 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728703AbfFYNlQ (ORCPT ); Tue, 25 Jun 2019 09:41:16 -0400 Received: by mail-ed1-f67.google.com with SMTP id k8so27166434edr.11 for ; Tue, 25 Jun 2019 06:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aUjbfaX8x/Hkcz066532dyQDPSZwjfDnEtWBELfOa/c=; b=lLocThYbPXloHgK5vJVHeFpoOja8CXzsSM5uzH1Cwis1Xk700uPR7E3m1Hf/wCbRPo 0oivaMKoOK/Hqf5wIf0IV6qGUQJuZQOcGuCw/XyV8y5nOrq/i3UeCVmAHNZZu4QTsIlf K0KNPZ0MCMHNaa7JYxH9xSSmJ84WqiJHO5Woie17oPy5KZTv2kIq4hd+w5yv0P5Bmoci 9pj58HVpbdaZmxEF/PMEH1OLVrTKWtYETQuIpoDN/2TVZIQ8qUKQgYm4kwWmsPqO6yLe MP/eSpKBGKntTKe9jDlXAWuBSpfXOo9EHebaIWRsbuE39ftfJMtFs2fvwACbcLNZp5rO WDrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aUjbfaX8x/Hkcz066532dyQDPSZwjfDnEtWBELfOa/c=; b=rvBNZiXVsJVHfxD0gb4YeVhc79Pr/VE2HHkg0N52z7v9x0hFfT9Ez8TUFPB0bX9dAI W7zOEwKQbysgyLo4D5MSh52U4F5t8U0cA3xozN0L6/xk/PPISpR+v0Z0VT6/OQFI1qBo aeOxtBkko7Ev9F8rY2M7HrlaTf2VK/BitznTeuLDDn7V7mqF97BsJmMoTR7a4CxXcJ7Q Y4SnG0Mn60zZWd5b8N+CfHtSAI86tGVQP5ub/s7qSybFF9wolfCoJo3v+es9QxrOn2mO pKKAoRbuwlT2tp1pX6VcMGz7w8/utLH680JQCXZNo/Zlo7Lh2nuOVQxwD7p/4Gi5eXVF ITzQ== X-Gm-Message-State: APjAAAUIB3/mNy6AVfpljkcSHHKVOxWRXLo4ialwjsTy35Dy7PXtVp5p MBwqU2v0lqlX/o/TL12UeE69EumbO9c= X-Google-Smtp-Source: APXvYqyyHLlu3OmJ61SbFC4OPeoxx7hhzlnDSd1Y3fJF8RvsdU7k44aTfSzY7ku837uKL0yLZZZZ/Q== X-Received: by 2002:a17:906:c404:: with SMTP id u4mr6731999ejz.123.1561470073435; Tue, 25 Jun 2019 06:41:13 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.12 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:12 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 07/15] Use promisor_remote_get_direct() and has_promisor_remote() Date: Tue, 25 Jun 2019 15:40:31 +0200 Message-Id: <20190625134039.21707-8-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of using the repository_format_partial_clone global and fetch_objects() directly, let's use has_promisor_remote() and promisor_remote_get_direct(). This way all the configured promisor remotes will be taken into account, not only the one specified by extensions.partialClone. Also when cloning or fetching using a partial clone filter, remote.origin.promisor will be set to "true" instead of setting extensions.partialClone to "origin". This makes it possible to use many promisor remote just by fetching from them. Signed-off-by: Christian Couder --- builtin/cat-file.c | 5 +++-- builtin/fetch.c | 11 ++++++----- builtin/gc.c | 3 ++- builtin/index-pack.c | 8 ++++---- builtin/repack.c | 3 ++- cache-tree.c | 3 ++- connected.c | 3 ++- diff.c | 9 ++++----- list-objects-filter-options.c | 28 +++++++++++++++------------- packfile.c | 3 ++- sha1-file.c | 15 ++++++++------- t/t5601-clone.sh | 2 +- t/t5616-partial-clone.sh | 2 +- unpack-trees.c | 8 ++++---- 14 files changed, 56 insertions(+), 47 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 0f092382e1..85ae10bf0b 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -15,6 +15,7 @@ #include "sha1-array.h" #include "packfile.h" #include "object-store.h" +#include "promisor-remote.h" struct batch_options { int enabled; @@ -523,8 +524,8 @@ static int batch_objects(struct batch_options *opt) if (opt->all_objects) { struct object_cb_data cb; - if (repository_format_partial_clone) - warning("This repository has extensions.partialClone set. Some objects may not be loaded."); + if (has_promisor_remote()) + warning("This repository uses promisor remotes. Some objects may not be loaded."); cb.opt = opt; cb.expand = &data; diff --git a/builtin/fetch.c b/builtin/fetch.c index 4ba63d5ac6..f74bd78144 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -23,6 +23,7 @@ #include "packfile.h" #include "list-objects-filter-options.h" #include "commit-reach.h" +#include "promisor-remote.h" static const char * const builtin_fetch_usage[] = { N_("git fetch [] [ [...]]"), @@ -1460,7 +1461,7 @@ static inline void fetch_one_setup_partial(struct remote *remote) * If no prior partial clone/fetch and the current fetch DID NOT * request a partial-fetch, do a normal fetch. */ - if (!repository_format_partial_clone && !filter_options.choice) + if (!has_promisor_remote() && !filter_options.choice) return; /* @@ -1468,7 +1469,7 @@ static inline void fetch_one_setup_partial(struct remote *remote) * on this repo and remember the given filter-spec as the default * for subsequent fetches to this remote. */ - if (!repository_format_partial_clone && filter_options.choice) { + if (!has_promisor_remote() && filter_options.choice) { partial_clone_register(remote->name, &filter_options); return; } @@ -1477,7 +1478,7 @@ static inline void fetch_one_setup_partial(struct remote *remote) * We are currently limited to only ONE promisor remote and only * allow partial-fetches from the promisor remote. */ - if (strcmp(remote->name, repository_format_partial_clone)) { + if (!promisor_remote_find(remote->name)) { if (filter_options.choice) die(_("--filter can only be used with the remote " "configured in extensions.partialClone")); @@ -1611,7 +1612,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) if (depth || deepen_since || deepen_not.nr) deepen = 1; - if (filter_options.choice && !repository_format_partial_clone) + if (filter_options.choice && !has_promisor_remote()) die("--filter can only be used when extensions.partialClone is set"); if (all) { @@ -1645,7 +1646,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) } if (remote) { - if (filter_options.choice || repository_format_partial_clone) + if (filter_options.choice || has_promisor_remote()) fetch_one_setup_partial(remote); result = fetch_one(remote, argc, argv, prune_tags_ok); } else { diff --git a/builtin/gc.c b/builtin/gc.c index 8943bcc300..824a8832b5 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -27,6 +27,7 @@ #include "pack-objects.h" #include "blob.h" #include "tree.h" +#include "promisor-remote.h" #define FAILED_RUN "failed to run %s" @@ -661,7 +662,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix) argv_array_push(&prune, prune_expire); if (quiet) argv_array_push(&prune, "--no-progress"); - if (repository_format_partial_clone) + if (has_promisor_remote()) argv_array_push(&prune, "--exclude-promisor-objects"); if (run_command_v_opt(prune.argv, RUN_GIT_CMD)) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 0d55f73b0b..a23454da6e 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -14,7 +14,7 @@ #include "thread-utils.h" #include "packfile.h" #include "object-store.h" -#include "fetch-object.h" +#include "promisor-remote.h" static const char index_pack_usage[] = "git index-pack [-v] [-o ] [--keep | --keep=] [--verify] [--strict] ( | --stdin [--fix-thin] [])"; @@ -1352,7 +1352,7 @@ static void fix_unresolved_deltas(struct hashfile *f) sorted_by_pos[i] = &ref_deltas[i]; QSORT(sorted_by_pos, nr_ref_deltas, delta_pos_compare); - if (repository_format_partial_clone) { + if (has_promisor_remote()) { /* * Prefetch the delta bases. */ @@ -1366,8 +1366,8 @@ static void fix_unresolved_deltas(struct hashfile *f) oid_array_append(&to_fetch, &d->oid); } if (to_fetch.nr) - fetch_objects(repository_format_partial_clone, - to_fetch.oid, to_fetch.nr); + promisor_remote_get_direct(the_repository, + to_fetch.oid, to_fetch.nr); oid_array_clear(&to_fetch); } diff --git a/builtin/repack.c b/builtin/repack.c index caca113927..df9a32c906 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -11,6 +11,7 @@ #include "midx.h" #include "packfile.h" #include "object-store.h" +#include "promisor-remote.h" static int delta_base_offset = 1; static int pack_kept_objects = -1; @@ -369,7 +370,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) argv_array_push(&cmd.args, "--all"); argv_array_push(&cmd.args, "--reflog"); argv_array_push(&cmd.args, "--indexed-objects"); - if (repository_format_partial_clone) + if (has_promisor_remote()) argv_array_push(&cmd.args, "--exclude-promisor-objects"); if (write_bitmaps) argv_array_push(&cmd.args, "--write-bitmap-index"); diff --git a/cache-tree.c b/cache-tree.c index b13bfaf71e..64c285a746 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -5,6 +5,7 @@ #include "cache-tree.h" #include "object-store.h" #include "replace-object.h" +#include "promisor-remote.h" #ifndef DEBUG #define DEBUG 0 @@ -357,7 +358,7 @@ static int update_one(struct cache_tree *it, } ce_missing_ok = mode == S_IFGITLINK || missing_ok || - (repository_format_partial_clone && + (has_promisor_remote() && ce_skip_worktree(ce)); if (is_null_oid(oid) || (!ce_missing_ok && !has_object_file(oid))) { diff --git a/connected.c b/connected.c index 1ab481fed6..b0e4968fbd 100644 --- a/connected.c +++ b/connected.c @@ -5,6 +5,7 @@ #include "connected.h" #include "transport.h" #include "packfile.h" +#include "promisor-remote.h" /* * If we feed all the commits we want to verify to this command @@ -73,7 +74,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data, argv_array_push(&rev_list.args,"rev-list"); argv_array_push(&rev_list.args, "--objects"); argv_array_push(&rev_list.args, "--stdin"); - if (repository_format_partial_clone) + if (has_promisor_remote()) argv_array_push(&rev_list.args, "--exclude-promisor-objects"); if (!opt->is_deepening_fetch) { argv_array_push(&rev_list.args, "--not"); diff --git a/diff.c b/diff.c index 1ee04e321b..249cc6eace 100644 --- a/diff.c +++ b/diff.c @@ -25,7 +25,7 @@ #include "packfile.h" #include "parse-options.h" #include "help.h" -#include "fetch-object.h" +#include "promisor-remote.h" #ifdef NO_FAST_WORKING_DIRECTORY #define FAST_WORKING_DIRECTORY 0 @@ -6514,8 +6514,7 @@ static void add_if_missing(struct repository *r, void diffcore_std(struct diff_options *options) { - if (options->repo == the_repository && - repository_format_partial_clone) { + if (options->repo == the_repository && has_promisor_remote()) { /* * Prefetch the diff pairs that are about to be flushed. */ @@ -6532,8 +6531,8 @@ void diffcore_std(struct diff_options *options) /* * NEEDSWORK: Consider deduplicating the OIDs sent. */ - fetch_objects(repository_format_partial_clone, - to_fetch.oid, to_fetch.nr); + promisor_remote_get_direct(options->repo, + to_fetch.oid, to_fetch.nr); oid_array_clear(&to_fetch); } diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index 1cb20c659c..b0de7d3c17 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -6,6 +6,7 @@ #include "list-objects.h" #include "list-objects-filter.h" #include "list-objects-filter-options.h" +#include "promisor-remote.h" /* * Parse value of the argument to the "filter" keyword. @@ -146,30 +147,31 @@ void partial_clone_register( const char *remote, const struct list_objects_filter_options *filter_options) { - /* - * Record the name of the partial clone remote in the - * config and in the global variable -- the latter is - * used throughout to indicate that partial clone is - * enabled and to expect missing objects. - */ - if (repository_format_partial_clone && - *repository_format_partial_clone && - strcmp(remote, repository_format_partial_clone)) - die(_("cannot change partial clone promisor remote")); + char *cfg_name; - git_config_set("core.repositoryformatversion", "1"); - git_config_set("extensions.partialclone", remote); + /* Check if it is already registered */ + if (!promisor_remote_find(remote)) { + git_config_set("core.repositoryformatversion", "1"); - repository_format_partial_clone = xstrdup(remote); + /* Add promisor config for the remote */ + cfg_name = xstrfmt("remote.%s.promisor", remote); + git_config_set(cfg_name, "true"); + free(cfg_name); + } /* * Record the initial filter-spec in the config as * the default for subsequent fetches from this remote. + * + * TODO: record it into remote..partialclonefilter */ core_partial_clone_filter_default = xstrdup(filter_options->filter_spec); git_config_set("core.partialclonefilter", core_partial_clone_filter_default); + + /* Make sure the config info are reset */ + promisor_remote_reinit(); } void partial_clone_get_default_filter_spec( diff --git a/packfile.c b/packfile.c index d786ec7312..50aaf93bc3 100644 --- a/packfile.c +++ b/packfile.c @@ -16,6 +16,7 @@ #include "tree.h" #include "object-store.h" #include "midx.h" +#include "promisor-remote.h" char *odb_pack_name(struct strbuf *buf, const unsigned char *sha1, @@ -2119,7 +2120,7 @@ int is_promisor_object(const struct object_id *oid) static int promisor_objects_prepared; if (!promisor_objects_prepared) { - if (repository_format_partial_clone) { + if (has_promisor_remote()) { for_each_packed_object(add_promisor_object, &promisor_objects, FOR_EACH_OBJECT_PROMISOR_ONLY); diff --git a/sha1-file.c b/sha1-file.c index 819d32cdb8..fe250c4b6e 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -30,8 +30,8 @@ #include "mergesort.h" #include "quote.h" #include "packfile.h" -#include "fetch-object.h" #include "object-store.h" +#include "promisor-remote.h" /* The maximum size for an object header. */ #define MAX_HEADER_LEN 32 @@ -1377,16 +1377,17 @@ int oid_object_info_extended(struct repository *r, const struct object_id *oid, } /* Check if it is a missing object */ - if (fetch_if_missing && repository_format_partial_clone && + if (fetch_if_missing && has_promisor_remote() && !already_retried && r == the_repository && !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) { /* - * TODO Investigate checking fetch_object() return - * TODO value and stopping on error here. - * TODO Pass a repository struct through fetch_object, - * such that arbitrary repositories work. + * TODO Investigate checking promisor_remote_get_direct() + * TODO return value and stopping on error here. + * TODO Pass a repository struct through + * promisor_remote_get_direct(), such that arbitrary + * repositories work. */ - fetch_objects(repository_format_partial_clone, real, 1); + promisor_remote_get_direct(r, real, 1); already_retried = 1; continue; } diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh index 37d76808d4..534d03a4d7 100755 --- a/t/t5601-clone.sh +++ b/t/t5601-clone.sh @@ -654,7 +654,7 @@ partial_clone () { git -C client fsck && # Ensure that unneeded blobs are not inadvertently fetched. - test_config -C client extensions.partialclone "not a remote" && + test_config -C client remote.origin.promisor "false" && test_must_fail git -C client cat-file -e "$HASH1" && # But this blob was fetched, because clone performs an initial checkout diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index b91ef548f8..8f9a62aac0 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -42,7 +42,7 @@ test_expect_success 'do partial clone 1' ' test_cmp expect_1.oids observed.oids && test "$(git -C pc1 config --local core.repositoryformatversion)" = "1" && - test "$(git -C pc1 config --local extensions.partialclone)" = "origin" && + test "$(git -C pc1 config --local remote.origin.promisor)" = "true" && test "$(git -C pc1 config --local core.partialclonefilter)" = "blob:none" ' diff --git a/unpack-trees.c b/unpack-trees.c index 50189909b8..aebd865ef6 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -16,7 +16,7 @@ #include "submodule-config.h" #include "fsmonitor.h" #include "object-store.h" -#include "fetch-object.h" +#include "promisor-remote.h" /* * Error messages expected by scripts out of plumbing commands such as @@ -400,7 +400,7 @@ static int check_updates(struct unpack_trees_options *o) load_gitmodules_file(index, &state); enable_delayed_checkout(&state); - if (repository_format_partial_clone && o->update && !o->dry_run) { + if (has_promisor_remote() && o->update && !o->dry_run) { /* * Prefetch the objects that are to be checked out in the loop * below. @@ -419,8 +419,8 @@ static int check_updates(struct unpack_trees_options *o) oid_array_append(&to_fetch, &ce->oid); } if (to_fetch.nr) - fetch_objects(repository_format_partial_clone, - to_fetch.oid, to_fetch.nr); + promisor_remote_get_direct(the_repository, + to_fetch.oid, to_fetch.nr); oid_array_clear(&to_fetch); } for (i = 0; i < index->cache_nr; i++) { From patchwork Tue Jun 25 13:40:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015591 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D31A112C for ; Tue, 25 Jun 2019 13:41:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F166A286C6 for ; Tue, 25 Jun 2019 13:41:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E5D792873A; Tue, 25 Jun 2019 13:41:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49EEA1FF73 for ; Tue, 25 Jun 2019 13:41:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730838AbfFYNlU (ORCPT ); Tue, 25 Jun 2019 09:41:20 -0400 Received: from mail-ed1-f68.google.com ([209.85.208.68]:44675 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727975AbfFYNlR (ORCPT ); Tue, 25 Jun 2019 09:41:17 -0400 Received: by mail-ed1-f68.google.com with SMTP id k8so27166570edr.11 for ; Tue, 25 Jun 2019 06:41:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OJddKki4i5LM7JqqBW9SQzb4BiJmxM38qI9ZRfQYRbY=; b=FD8dmn8uEFpVJobXchp+8zElWr2H7TveyRleliHLviZsoha2JUQ5p0i8+deYY3r7cI s7i8FcC6XB9dWlMR3jp18rH/nDubhA4a5tiBnv3Qyv2QMRv6uKC+YAQOsLW5S6KY4XDo NTbUEaLTXwEd7KvT3Lbb8pPhWMwdMhfZpzUBy0QcbHPgbW+7q4dTK+KMItYq8hUv/QDO jZMcvS83y0y3bkIozXAiJDm+RIOUnmwn3g8VH3Whdt9ZHFKJ6zIsGwp0EBzYJT1eGAug PYaeZVblngimHtXeZsfKLmrpqUl4NQ0rtLxkCHwMUjmazpSqTfjhmBW51S/jEOzwCa79 I07w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OJddKki4i5LM7JqqBW9SQzb4BiJmxM38qI9ZRfQYRbY=; b=s4xWR97wuEfsi4WTP1ESaCDhuio5XgaHJwPfGySUmtVGlNqrZEZHkUqekOR6+setd9 FfLYoL6vslnrLRfzBvAyaQ39oMHNfWh0CEpei10TeWbFK7XZJB89qiLpGW3oCI7fEe27 PL3OMAQ2YFjdCfHALL0qlTJYg9J6SOSkRn7wFp3AtHbsG3h1RbNtZ3UIn3LQ/UR2KlGg H74vIDJh4eMMmUr8PAmS6+1Bch8SQo3Bmt7FsdBxwJobRFJH3cTZKuiwrKWVpx3gtFKq FB62yqQe60b/96MQ7L6mWUPDJ1TfR1+mGTmU+HiMAP1NWK3nmtSugKBmHYALNTPDkwIv c3IA== X-Gm-Message-State: APjAAAW7dHQCm7iqxiazirql75U8M8yiFfwbLnvQaayOngfnFLTPExD5 XwwLP7mFyx7w3EdACPsOYwnoOVrNZCU= X-Google-Smtp-Source: APXvYqwV2FcoXNRHoYga8vWgUfykka9/PaGMMlZHtk7sJIiophkb1yr9r0kw0ilQuP8N39bpOmfPzw== X-Received: by 2002:a17:906:914:: with SMTP id i20mr40607945ejd.213.1561470074961; Tue, 25 Jun 2019 06:41:14 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.13 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:13 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 08/15] promisor-remote: parse remote.*.partialclonefilter Date: Tue, 25 Jun 2019 15:40:32 +0200 Message-Id: <20190625134039.21707-9-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This makes it possible to specify a different partial clone filter for each promisor remote. Signed-off-by: Christian Couder --- builtin/fetch.c | 2 +- list-objects-filter-options.c | 27 +++++++++++++++------------ list-objects-filter-options.h | 3 ++- promisor-remote.c | 15 +++++++++++++++ promisor-remote.h | 5 ++++- t/t0410-partial-clone.sh | 2 +- t/t5601-clone.sh | 1 + t/t5616-partial-clone.sh | 2 +- 8 files changed, 40 insertions(+), 17 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index f74bd78144..13d8133130 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1491,7 +1491,7 @@ static inline void fetch_one_setup_partial(struct remote *remote) * the config. */ if (!filter_options.choice) - partial_clone_get_default_filter_spec(&filter_options); + partial_clone_get_default_filter_spec(&filter_options, remote->name); return; } diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index b0de7d3c17..28c571f922 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -30,6 +30,9 @@ static int gently_parse_list_objects_filter( { const char *v0; + if (!arg) + return 0; + if (filter_options->choice) { if (errbuf) { strbuf_addstr( @@ -148,6 +151,7 @@ void partial_clone_register( const struct list_objects_filter_options *filter_options) { char *cfg_name; + char *filter_name; /* Check if it is already registered */ if (!promisor_remote_find(remote)) { @@ -162,27 +166,26 @@ void partial_clone_register( /* * Record the initial filter-spec in the config as * the default for subsequent fetches from this remote. - * - * TODO: record it into remote..partialclonefilter */ - core_partial_clone_filter_default = - xstrdup(filter_options->filter_spec); - git_config_set("core.partialclonefilter", - core_partial_clone_filter_default); + filter_name = xstrfmt("remote.%s.partialclonefilter", remote); + git_config_set(filter_name, filter_options->filter_spec); + free(filter_name); /* Make sure the config info are reset */ promisor_remote_reinit(); } void partial_clone_get_default_filter_spec( - struct list_objects_filter_options *filter_options) + struct list_objects_filter_options *filter_options, + const char *remote) { + struct promisor_remote *promisor = promisor_remote_find(remote); + /* * Parse default value, but silently ignore it if it is invalid. */ - if (!core_partial_clone_filter_default) - return; - gently_parse_list_objects_filter(filter_options, - core_partial_clone_filter_default, - NULL); + if (promisor) + gently_parse_list_objects_filter(filter_options, + promisor->partial_clone_filter, + NULL); } diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index c54f0000fb..8deaa287b5 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -87,6 +87,7 @@ void partial_clone_register( const char *remote, const struct list_objects_filter_options *filter_options); void partial_clone_get_default_filter_spec( - struct list_objects_filter_options *filter_options); + struct list_objects_filter_options *filter_options, + const char *remote); #endif /* LIST_OBJECTS_FILTER_OPTIONS_H */ diff --git a/promisor-remote.c b/promisor-remote.c index 6a8856f475..826890f7b8 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -75,6 +75,21 @@ static int promisor_remote_config(const char *var, const char *value, void *data free(remote_name); return 0; } + if (!strcmp(subkey, "partialclonefilter")) { + struct promisor_remote *r; + char *remote_name = xmemdupz(name, namelen); + + r = promisor_remote_lookup(remote_name, NULL); + if (!r) + r = promisor_remote_new(remote_name); + + free(remote_name); + + if (!r) + return 0; + + return git_config_string(&r->partial_clone_filter, var, value); + } return 0; } diff --git a/promisor-remote.h b/promisor-remote.h index dddd4048e0..838cb092f3 100644 --- a/promisor-remote.h +++ b/promisor-remote.h @@ -5,10 +5,13 @@ struct object_id; /* * Promisor remote linked list - * Its information come from remote.XXX config entries. + * + * Information in its fields come from remote.XXX config entries or + * from extensions.partialclone or core.partialclonefilter. */ struct promisor_remote { struct promisor_remote *next; + const char *partial_clone_filter; const char name[FLEX_ARRAY]; }; diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh index 3559313bd0..3082eff2bf 100755 --- a/t/t0410-partial-clone.sh +++ b/t/t0410-partial-clone.sh @@ -26,7 +26,7 @@ promise_and_delete () { test_expect_success 'extensions.partialclone without filter' ' test_create_repo server && git clone --filter="blob:none" "file://$(pwd)/server" client && - git -C client config --unset core.partialclonefilter && + git -C client config --unset remote.origin.partialclonefilter && git -C client fetch origin ' diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh index 534d03a4d7..078cf48dd6 100755 --- a/t/t5601-clone.sh +++ b/t/t5601-clone.sh @@ -655,6 +655,7 @@ partial_clone () { # Ensure that unneeded blobs are not inadvertently fetched. test_config -C client remote.origin.promisor "false" && + git -C client config --unset remote.origin.partialclonefilter && test_must_fail git -C client cat-file -e "$HASH1" && # But this blob was fetched, because clone performs an initial checkout diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index 8f9a62aac0..8ae7ba9c95 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -43,7 +43,7 @@ test_expect_success 'do partial clone 1' ' test_cmp expect_1.oids observed.oids && test "$(git -C pc1 config --local core.repositoryformatversion)" = "1" && test "$(git -C pc1 config --local remote.origin.promisor)" = "true" && - test "$(git -C pc1 config --local core.partialclonefilter)" = "blob:none" + test "$(git -C pc1 config --local remote.origin.partialclonefilter)" = "blob:none" ' # checkout master to force dynamic object fetch of blobs at HEAD. From patchwork Tue Jun 25 13:40:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015589 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE4CD1580 for ; Tue, 25 Jun 2019 13:41:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A1C68285A3 for ; Tue, 25 Jun 2019 13:41:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 95F2D286F2; Tue, 25 Jun 2019 13:41:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3FA07286C6 for ; Tue, 25 Jun 2019 13:41:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727338AbfFYNlT (ORCPT ); Tue, 25 Jun 2019 09:41:19 -0400 Received: from mail-ed1-f68.google.com ([209.85.208.68]:45416 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730220AbfFYNlS (ORCPT ); Tue, 25 Jun 2019 09:41:18 -0400 Received: by mail-ed1-f68.google.com with SMTP id a14so27178719edv.12 for ; Tue, 25 Jun 2019 06:41:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hQ6bGiqz4a2cNG6935V15lPG6hJOhxhpuDE2K4gYTHc=; b=N3/uKG0/H6psUt7bC0JR5irI5meP1JovrG+8wu+lmdw5l/hfnTFWD97gj0nHntWXZR X54i+9Bxf+s7+F0rUQqUqn0WFLwQw5D1m3TagrjAUkQ+xhrxDtKoc+xO6Dar8sq1E2/4 bqnBMmgLGvuxCv+Ld52glFSXO6nfk53e6Cb0LDKhkoijWrhKiwVv2mx89+gb9I63bMlx 0vXftLfkmGtgk5IcmZgC4sWCYxv+j1XHJtstu66JJ9tFnUsWloW4xv/EkRVZYZd9QskX PhLt3I9rOPjRlkP66FfZlVGvVRI3Z3e03hxWsMoiX0ojm1DIbXzfB9FZCaywWVlSWGee pmcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hQ6bGiqz4a2cNG6935V15lPG6hJOhxhpuDE2K4gYTHc=; b=rJCIQcWi8wm7v9u8tNosBZYpn/GX85kqsc0YXkwRUulLj2fBM/9/3D//FbzGQuq67S pcNDhqbVaxtosznNcSkG3E5s8R0unFNr9BHkkP6NNWqzZAJB6PlJYH9v09kcqY960hF5 IvIHadD8cp6pERLz/KafTZoZFIQAy1CP7P9kHJxtwqFGazEZXMmEe5uBdZGgLvqWhfpk S2PJNHtpV0zXDnHWeaXfoA6RhuENkt9tlcidKN51aFqI+VR1AC0pGvjq8Gc+OTgyKgby g/VugWvBnUgdlE9UVDtWkdQwyP05/v6QBjqdsjsQ7GadmmnrDmXN/+qYG3YEhS0M+2aL vPaQ== X-Gm-Message-State: APjAAAXYBPE2jzFqwmn3+QtRjhK/v9IhihXMXmEn3nS5gxxA0V+ikxfp ovUghQGL4WZfTZCtv8JuwEEeV2yjE18= X-Google-Smtp-Source: APXvYqyEN0zFPXFgRoW7vvV+m12kV4cArMJ62Wn2g0w1D+jQvxDpbFauGmKTQEXZjFUz/Eelbp/XGw== X-Received: by 2002:a17:906:158c:: with SMTP id k12mr25643172ejd.83.1561470076012; Tue, 25 Jun 2019 06:41:16 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.15 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:15 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 09/15] builtin/fetch: remove unique promisor remote limitation Date: Tue, 25 Jun 2019 15:40:33 +0200 Message-Id: <20190625134039.21707-10-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP As the infrastructure for more than one promisor remote has been introduced in previous patches, we can remove code that forbids the registration of more than one promisor remote. Signed-off-by: Christian Couder --- builtin/fetch.c | 20 +++++--------------- 1 file changed, 5 insertions(+), 15 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 13d8133130..5657d054ec 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1465,26 +1465,16 @@ static inline void fetch_one_setup_partial(struct remote *remote) return; /* - * If this is the FIRST partial-fetch request, we enable partial - * on this repo and remember the given filter-spec as the default - * for subsequent fetches to this remote. + * If this is a partial-fetch request, we enable partial on + * this repo if not already enabled and remember the given + * filter-spec as the default for subsequent fetches to this + * remote. */ - if (!has_promisor_remote() && filter_options.choice) { + if (filter_options.choice) { partial_clone_register(remote->name, &filter_options); return; } - /* - * We are currently limited to only ONE promisor remote and only - * allow partial-fetches from the promisor remote. - */ - if (!promisor_remote_find(remote->name)) { - if (filter_options.choice) - die(_("--filter can only be used with the remote " - "configured in extensions.partialClone")); - return; - } - /* * Do a partial-fetch from the promisor remote using either the * explicitly given filter-spec or inherit the filter-spec from From patchwork Tue Jun 25 13:40:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015593 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E55F8112C for ; Tue, 25 Jun 2019 13:41:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D5E981FF73 for ; Tue, 25 Jun 2019 13:41:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C9E93286F2; Tue, 25 Jun 2019 13:41:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 622C11FF73 for ; Tue, 25 Jun 2019 13:41:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726931AbfFYNlV (ORCPT ); Tue, 25 Jun 2019 09:41:21 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:46290 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730591AbfFYNlT (ORCPT ); Tue, 25 Jun 2019 09:41:19 -0400 Received: by mail-ed1-f67.google.com with SMTP id d4so27162663edr.13 for ; Tue, 25 Jun 2019 06:41:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w32hNmpnCQv0SjxcKVptiQUzm2FTMKi1ZOMuwrEJ9WE=; b=ayTeTGVr7L4UekjGXs1bkBfxfg3QPeeGCr8IdEKLZ7JYBnfI+fz3NSeC8Oq9GaZQbX aq1F3p95SYMK1dVNJt6YHq2c1MppPra7ytcP/NyNqc8lxMQYEvnEPQR1CKpDL9rIHqQZ Rfp0Qg/t9lGU0oXjQZX2usz0DWOv/DtD2qNmSQPNxuSIJ/HANUKr4q8NRhHNo9WJYjlv M+TIkI3bMcT7pxHleoKW9CXdcG9S77ryL0mCZ8Fk7RETZ8YWYeEu3R8pMcYwaWBAJlYl vZu4E7vp36pDySjB2pwnjKREWuKu92GyPVYxecWkEVdVKraIJku5Tq/fKkh8xbum6t7S Lztw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w32hNmpnCQv0SjxcKVptiQUzm2FTMKi1ZOMuwrEJ9WE=; b=SZr83WRdDrxiGl+34EQb3H/NUn7oIPMyIlO9Tq7FsfZK0pkNDL+lgKnrztCoHWqTex RLdfxzCSj6IpwStwhvG75JXP671etMguQZ2qwxluY8eHiyG7rjtsRYWHlQhO/DeO933i gSJ5WjpnOpD9NXVP/ayJEmbcrLJ9ZwhmSSds0qv042952L8jkCRDdXDV3SJw432yrHCy 2bJ9QBNcfnRL4WP1xRIKLGdbIopxN7/zaMHoSHRuu+CDsROP0NFuycRQXdhfQKXK3TGQ PQjE1QbwyQLO9bhqFNlT/k3+8l0woSLIeJSdo9rdqI0JB5VWbRJZx3jeLyq1GT6B+eUP C17g== X-Gm-Message-State: APjAAAVkPrZ7XhBaHP7XMELVI+g5vvlpTnqCasQBqgdWTwLp0ogG7Xlu cQwHo36lFTNMrmMq581vqlG6O8EvXTg= X-Google-Smtp-Source: APXvYqyaa1Yks33swrhCbSfRnG5lFf0UYhEeU0kR0GzQWy4ve2KJucLIWB+Bl3q+xGNVPDSLlFWoYg== X-Received: by 2002:a17:906:2191:: with SMTP id 17mr29047317eju.280.1561470077195; Tue, 25 Jun 2019 06:41:17 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.16 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:16 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee , Christian Couder Subject: [PATCH v6 10/15] t0410: test fetching from many promisor remotes Date: Tue, 25 Jun 2019 15:40:34 +0200 Message-Id: <20190625134039.21707-11-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christian Couder This shows that it is now possible to fetch objects from more than one promisor remote, and that fetching from a new promisor remote can configure it as one. Helped-by: SZEDER Gábor Signed-off-by: Christian Couder --- t/t0410-partial-clone.sh | 49 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh index 3082eff2bf..2498e72a34 100755 --- a/t/t0410-partial-clone.sh +++ b/t/t0410-partial-clone.sh @@ -183,8 +183,55 @@ test_expect_success 'fetching of missing objects works with ref-in-want enabled' grep "git< fetch=.*ref-in-want" trace ' +test_expect_success 'fetching of missing objects from another promisor remote' ' + git clone "file://$(pwd)/server" server2 && + test_commit -C server2 bar && + git -C server2 repack -a -d --write-bitmap-index && + HASH2=$(git -C server2 rev-parse bar) && + + git -C repo remote add server2 "file://$(pwd)/server2" && + git -C repo config remote.server2.promisor true && + git -C repo cat-file -p "$HASH2" && + + git -C repo fetch server2 && + rm -rf repo/.git/objects/* && + git -C repo cat-file -p "$HASH2" && + + # Ensure that the .promisor file is written, and check that its + # associated packfile contains the object + ls repo/.git/objects/pack/pack-*.promisor >promisorlist && + test_line_count = 1 promisorlist && + IDX=$(sed "s/promisor$/idx/" promisorlist) && + git verify-pack --verbose "$IDX" >out && + grep "$HASH2" out +' + +test_expect_success 'fetching of missing objects configures a promisor remote' ' + git clone "file://$(pwd)/server" server3 && + test_commit -C server3 baz && + git -C server3 repack -a -d --write-bitmap-index && + HASH3=$(git -C server3 rev-parse baz) && + git -C server3 config uploadpack.allowfilter 1 && + + rm repo/.git/objects/pack/pack-*.promisor && + + git -C repo remote add server3 "file://$(pwd)/server3" && + git -C repo fetch --filter="blob:none" server3 $HASH3 && + + test_cmp_config -C repo true remote.server3.promisor && + + # Ensure that the .promisor file is written, and check that its + # associated packfile contains the object + ls repo/.git/objects/pack/pack-*.promisor >promisorlist && + test_line_count = 1 promisorlist && + IDX=$(sed "s/promisor$/idx/" promisorlist) && + git verify-pack --verbose "$IDX" >out && + grep "$HASH3" out +' + test_expect_success 'fetching of missing blobs works' ' - rm -rf server repo && + rm -rf server server2 repo && + rm -rf server server3 repo && test_create_repo server && test_commit -C server foo && git -C server repack -a -d --write-bitmap-index && From patchwork Tue Jun 25 13:40:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015595 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BCEDF1575 for ; Tue, 25 Jun 2019 13:41:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF0CE1FF73 for ; Tue, 25 Jun 2019 13:41:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A287B28A00; Tue, 25 Jun 2019 13:41:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CDC711FF73 for ; Tue, 25 Jun 2019 13:41:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730837AbfFYNlX (ORCPT ); Tue, 25 Jun 2019 09:41:23 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:42050 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726871AbfFYNlV (ORCPT ); Tue, 25 Jun 2019 09:41:21 -0400 Received: by mail-ed1-f67.google.com with SMTP id z25so27210009edq.9 for ; Tue, 25 Jun 2019 06:41:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vfu6BLCHfXpS0QbQy6I9jG+42bXaSELjF5aTG+mbQmU=; b=IVMYiLKjFQ7uGyWiDXpq2BKyPB8l45tOT3gTp5PKUdAnBTftrvU+R159fPADxjy4h3 FQlaBxNN6Gr2q7Wp7mzIHoj+jd7jsMf+3wiGq912gTEboAOUmEPDMCnZbX5kCMi/FgBl hXbwtd6Zj05TAg014po1/zd3KcKzoArtxuvUdKJgivKSI46nC9AKqRA8AQWgxloJZpJ3 Al03FzBGvG3f/UZRoYrIycGmT6mTIbUBpVhTaoqk33KirarVpSG6xDfasZ99qtOiHoBn /Rh5jgOn3EYPK/9n64qJlX/UyGmB6LdJ51FYxe2X7vgef7qCErC9jjfECodw/XkCimnY Ebpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vfu6BLCHfXpS0QbQy6I9jG+42bXaSELjF5aTG+mbQmU=; b=e9L45YhyIoTlUqHe9HavrHWmgux9B4h0y/3jwXHOJj5yTaSWkwiToGphKFAxN69i2e GwIrVCdeuRvmJUqZGrlmxHj/saRx04l9GDn3qpd2SA8HlIUrrWM7XF9QTsJnGkpw2RHq gsqyzBo6dc6Y2f9N1jBEH491CbGnIhdJYrzgfPSnwm7NwuoaYizukfps3Ngi5wjEBF5F 0smRAe4kOVlVQK4s2dIoq3TE8zn3uxYy1z94TDnqVtIEgqxxypQfuMgYUnxMf2CNWZCm 3JLSKAVGLaaTc9qEJpWSBLGRQczMBDAnxQ3W3TsaCQUj4nAwrxR5G4gbuulKI/hp5XK3 2LLw== X-Gm-Message-State: APjAAAVzOCshdfz6upBuS/0YSFGIjctAy6blc+orwGrImp3UfL+KmIPY 6qyVJxAEkQa0ImgmVah0jF2FdtFY0LY= X-Google-Smtp-Source: APXvYqw4zJ0chRr87JRo3nOY06P3aT8h4RYllwdFOHNc3framKVusZE4VckFt5UoPbiSIJuvwqQJCQ== X-Received: by 2002:a17:906:1252:: with SMTP id u18mr5754272eja.13.1561470078370; Tue, 25 Jun 2019 06:41:18 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.17 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:17 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 11/15] partial-clone: add multiple remotes in the doc Date: Tue, 25 Jun 2019 15:40:35 +0200 Message-Id: <20190625134039.21707-12-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP While at it, let's remove a reference to ODB effort as the ODB effort has been replaced by directly enhancing partial clone and promisor remote features. Signed-off-by: Christian Couder --- Documentation/technical/partial-clone.txt | 117 ++++++++++++++++------ 1 file changed, 84 insertions(+), 33 deletions(-) diff --git a/Documentation/technical/partial-clone.txt b/Documentation/technical/partial-clone.txt index 896c7b3878..210373e258 100644 --- a/Documentation/technical/partial-clone.txt +++ b/Documentation/technical/partial-clone.txt @@ -30,12 +30,20 @@ advance* during clone and fetch operations and thereby reduce download times and disk usage. Missing objects can later be "demand fetched" if/when needed. +A remote that can later provide the missing objects is called a +promisor remote, as it promises to send the objects when +requested. Initialy Git supported only one promisor remote, the origin +remote from which the user cloned and that was configured in the +"extensions.partialClone" config option. Later support for more than +one promisor remote has been implemented. + Use of partial clone requires that the user be online and the origin -remote be available for on-demand fetching of missing objects. This may -or may not be problematic for the user. For example, if the user can -stay within the pre-selected subset of the source tree, they may not -encounter any missing objects. Alternatively, the user could try to -pre-fetch various objects if they know that they are going offline. +remote or other promisor remotes be available for on-demand fetching +of missing objects. This may or may not be problematic for the user. +For example, if the user can stay within the pre-selected subset of +the source tree, they may not encounter any missing objects. +Alternatively, the user could try to pre-fetch various objects if they +know that they are going offline. Non-Goals @@ -100,18 +108,18 @@ or commits that reference missing trees. Handling Missing Objects ------------------------ -- An object may be missing due to a partial clone or fetch, or missing due - to repository corruption. To differentiate these cases, the local - repository specially indicates such filtered packfiles obtained from the - promisor remote as "promisor packfiles". +- An object may be missing due to a partial clone or fetch, or missing + due to repository corruption. To differentiate these cases, the + local repository specially indicates such filtered packfiles + obtained from promisor remotes as "promisor packfiles". + These promisor packfiles consist of a ".promisor" file with arbitrary contents (like the ".keep" files), in addition to their ".pack" and ".idx" files. - The local repository considers a "promisor object" to be an object that - it knows (to the best of its ability) that the promisor remote has promised - that it has, either because the local repository has that object in one of + it knows (to the best of its ability) that promisor remotes have promised + that they have, either because the local repository has that object in one of its promisor packfiles, or because another promisor object refers to it. + When Git encounters a missing object, Git can see if it is a promisor object @@ -123,12 +131,12 @@ expensive-to-modify list of missing objects.[a] - Since almost all Git code currently expects any referenced object to be present locally and because we do not want to force every command to do a dry-run first, a fallback mechanism is added to allow Git to attempt - to dynamically fetch missing objects from the promisor remote. + to dynamically fetch missing objects from promisor remotes. + When the normal object lookup fails to find an object, Git invokes -fetch-object to try to get the object from the server and then retry -the object lookup. This allows objects to be "faulted in" without -complicated prediction algorithms. +promisor_remote_get_direct() to try to get the object from a promisor +remote and then retry the object lookup. This allows objects to be +"faulted in" without complicated prediction algorithms. + For efficiency reasons, no check as to whether the missing object is actually a promisor object is performed. @@ -157,8 +165,7 @@ and prefetch those objects in bulk. + We are not happy with this global variable and would like to remove it, but that requires significant refactoring of the object code to pass an -additional flag. We hope that concurrent efforts to add an ODB API can -encompass this. +additional flag. Fetching Missing Objects @@ -182,21 +189,63 @@ has been updated to not use any object flags when the corresponding argument though they are not necessary. +Using many promisor remotes +--------------------------- + +Many promisor remotes can be configured and used. + +This allows for example a user to have multiple geographically-close +cache servers for fetching missing blobs while continuing to do +filtered `git-fetch` commands from the central server. + +When fetching objects, promisor remotes are tried one after the other +until all the objects have been fetched. + +Remotes that are considered "promisor" remotes are those specified by +the following configuration variables: + +- `extensions.partialClone = ` + +- `remote..promisor = true` + +- `remote..partialCloneFilter = ...` + +Only one promisor remote can be configured using the +`extensions.partialClone` config variable. This promisor remote will +be the last one tried when fetching objects. + +We decided to make it the last one we try, because it is likely that +someone using many promisor remotes is doing so because the other +promisor remotes are better for some reason (maybe they are closer or +faster for some kind of objects) than the origin, and the origin is +likely to be the remote specified by extensions.partialClone. + +This justification is not very strong, but one choice had to be made, +and anyway the long term plan should be to make the order somehow +fully configurable. + +For now though the other promisor remotes will be tried in the order +they appear in the config file. + Current Limitations ------------------- -- The remote used for a partial clone (or the first partial fetch - following a regular clone) is marked as the "promisor remote". +- It is not possible to specify the order in which the promisor + remotes are tried in other ways than the order in which they appear + in the config file. + -We are currently limited to a single promisor remote and only that -remote may be used for subsequent partial fetches. +It is also not possible to specify an order to be used when fetching +from one remote and a different order when fetching from another +remote. + +- It is not possible to push only specific objects to a promisor + remote. + -We accept this limitation because we believe initial users of this -feature will be using it on repositories with a strong single central -server. +It is not possible to push at the same time to multiple promisor +remote in a specific order. -- Dynamic object fetching will only ask the promisor remote for missing - objects. We assume that the promisor remote has a complete view of the +- Dynamic object fetching will only ask promisor remotes for missing + objects. We assume that promisor remotes have a complete view of the repository and can satisfy all such requests. - Repack essentially treats promisor and non-promisor packfiles as 2 @@ -218,15 +267,17 @@ server. Future Work ----------- -- Allow more than one promisor remote and define a strategy for fetching - missing objects from specific promisor remotes or of iterating over the - set of promisor remotes until a missing object is found. +- Improve the way to specify the order in which promisor remotes are + tried. + -A user might want to have multiple geographically-close cache servers -for fetching missing blobs while continuing to do filtered `git-fetch` -commands from the central server, for example. +For example this could allow to specify explicitly something like: +"When fetching from this remote, I want to use these promisor remotes +in this order, though, when pushing or fetching to that remote, I want +to use those promisor remotes in that order." + +- Allow pushing to promisor remotes. + -Or the user might want to work in a triangular work flow with multiple +The user might want to work in a triangular work flow with multiple promisor remotes that each have an incomplete view of the repository. - Allow repack to work on promisor packfiles (while keeping them distinct From patchwork Tue Jun 25 13:40:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015601 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E9D3112C for ; Tue, 25 Jun 2019 13:41:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 401FB2848F for ; Tue, 25 Jun 2019 13:41:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 33A392873A; Tue, 25 Jun 2019 13:41:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DAB022848F for ; Tue, 25 Jun 2019 13:41:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730870AbfFYNlW (ORCPT ); Tue, 25 Jun 2019 09:41:22 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:32994 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730304AbfFYNlV (ORCPT ); Tue, 25 Jun 2019 09:41:21 -0400 Received: by mail-ed1-f67.google.com with SMTP id i11so27271953edq.0 for ; Tue, 25 Jun 2019 06:41:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZfzpubbHAcLJUt4CwStdJgNBdC3xUrbjHUmQ2Q/WJU4=; b=R1dc1z0uAKpyTvKAXSDf+y0ztA0gpACfxnUbJqbeC1CHEJdDFxgFzlrc/x6WReOs2W ku+rjiGiy2VQj0/9kBXWwSkjSdSkCSxjfAlXN5gFmqwCIVOU/slFFBrsybucaGT7l7Q+ QqRgWs0EUTHnvx/1U8WkMubFxz4uzfATbmhNuF4GO+Ux4ph3jltpjMRoS/YPNeECdtIv B0sDEcNxZhLOy/JlcmHaeR9s2a8SFuGj1+T82XXhr907Y200i4W8hSK6VqO+fJepH4ci SLKDPj0w/c3I6p5fzzmeVNoFDb6xsp7KVM2549CQf7zpujmoDB7H2+IKL9k0iDEEyihy YH5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZfzpubbHAcLJUt4CwStdJgNBdC3xUrbjHUmQ2Q/WJU4=; b=NdOd2RmfZ/6kMMzPlyULZnPS4NT8/MHev6kU773ufdcXHgx15QkXo7CnMHFX8a1oef OcSS/cIEQLMW/yOdaksl53dvcfWy+yIFiB92w9WZW3VpIq1P7I4FtP/mcfIqlypIcy2D tbKF/SNvkxRCLKBnzh9jxZoSERiJDJR1Lsnm2+6XlfnBeXcpEJhb3yoOLraJrxuxWeY8 YItrKgsdcYV6YAHcCYbkGdgA7DJwXd4deGU7mZjIDmJGW2p/v/vg4Z7MnN6zOVfZX1W2 EOY7dh092EI1ZJ4Mp2lZk2SlfCboGqtfBDFzl/8v3jyCaKlmsmFDO0I+MUEaqlGvrDTj NQbw== X-Gm-Message-State: APjAAAXUZIfsNt2LfZKk3K8JvAKC9JdRfbulyg568Q+pGzJhQ3LbbBZO 9MahefpsUUqTUD4vkDaUiaUKukfoAnI= X-Google-Smtp-Source: APXvYqxK+WVO5NJySSc1C6d0Ap3Zv1Q55PE10CcooT3BLJ2j+YM8+IB8dFJKbZ/qr9dn0j2aW1441A== X-Received: by 2002:a17:906:9711:: with SMTP id k17mr25203499ejx.298.1561470079426; Tue, 25 Jun 2019 06:41:19 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.18 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:18 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 12/15] remote: add promisor and partial clone config to the doc Date: Tue, 25 Jun 2019 15:40:36 +0200 Message-Id: <20190625134039.21707-13-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Christian Couder --- Documentation/config/remote.txt | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/Documentation/config/remote.txt b/Documentation/config/remote.txt index 6c4cad83a2..a8e6437a90 100644 --- a/Documentation/config/remote.txt +++ b/Documentation/config/remote.txt @@ -76,3 +76,11 @@ remote..pruneTags:: + See also `remote..prune` and the PRUNING section of linkgit:git-fetch[1]. + +remote..promisor:: + When set to true, this remote will be used to fetch promisor + objects. + +remote..partialclonefilter:: + The filter that will be applied when fetching from this + promisor remote. From patchwork Tue Jun 25 13:40:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 132C9112C for ; Tue, 25 Jun 2019 13:41:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 05DBF205F7 for ; Tue, 25 Jun 2019 13:41:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EE2D8286C6; Tue, 25 Jun 2019 13:41:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 58F2A205F7 for ; Tue, 25 Jun 2019 13:41:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730279AbfFYNlc (ORCPT ); Tue, 25 Jun 2019 09:41:32 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:39830 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730847AbfFYNlW (ORCPT ); Tue, 25 Jun 2019 09:41:22 -0400 Received: by mail-ed1-f67.google.com with SMTP id m10so27236045edv.6 for ; Tue, 25 Jun 2019 06:41:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=flz8AQf2YyWfeUF6PkCk1aqxzsb+TL0CnOz7yPbe5Ws=; b=Lw7lXDSme24srlg/9hwV+bp2HI0dlQF19XE1h6WMLN81e2nx8XoHPniIURXv8QP1ih o0oEx7UArWVPR6coyqBUs/LTv/uV5yKxFGLqyEV73dp/JFpHjBzCOZUkeEmnDNozxu0C kR4fTPvQY8/xNnPwpuI1390dgGOqgHWiTTrKnQmUVkdajXrwSHCqKJlcddd0oKaR583I SyeKdxRSW58lZh6q+s16/DoLDDm2pw+ZUdGzbJZfctu5UF3cfUCKzvibCi2TcFgXs7q0 sQJM6nIOdVC4VmSngVgCUXxArvcxcnjTF7K1RvFcopcJq7P88UKwXdI2jVpofvXGvRgf EsIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=flz8AQf2YyWfeUF6PkCk1aqxzsb+TL0CnOz7yPbe5Ws=; b=hZPyOqev9xF2Kzff5icftETlP+66G8S+0B5EYP0OJ3mvNyyBIIV7AwbMeFw1EUvmeW cn4OhsrOlhU2Bapwrm9o4zPZLjzwaE/hL9DLm9Ovwb923wWv6hJVhVmlRC5+7acgKGRo qdOJXlXJL/jj4nGYbVl6e2MA3nrM3lR5HY8LlzDuVAX65QK9LEMezeWvnfR7C4W3dr3O aujTO1vkEnrb02cP2EWKhoXD68qhnWVxG7vlM8VwmZKAfTIiubQ7afhXrd9/bF27XiuT urtqY+tbPX8dwEP97ztHsCdXI8sQs+0YPGPCAB5QKwU3UwDJh7hmJgCSi7l2YFVvRbQO Uysw== X-Gm-Message-State: APjAAAUMED90VEE+uRHE5v88lgfJpI97KSUMWRX48ddEZaUrbkxN4+jb mi0aY35hYQzGawsM7W01jdfY8dT5glM= X-Google-Smtp-Source: APXvYqynPCGhU+Ab31KsSG9WgwfHit2nb2FtzThfzIMnrM9rQjmCf+YRBVOZaMW0nulw8+Mm15dvVg== X-Received: by 2002:a17:906:944f:: with SMTP id z15mr25055248ejx.137.1561470080531; Tue, 25 Jun 2019 06:41:20 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.19 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:19 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 13/15] Remove fetch-object.{c,h} in favor of promisor-remote.{c,h} Date: Tue, 25 Jun 2019 15:40:37 +0200 Message-Id: <20190625134039.21707-14-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP As fetch_objects() is now used only in promisor-remote.c and should't be used outside it, let's move it into promisor-remote.c, make it static there, and remove fetch-object.{c,h}. Signed-off-by: Christian Couder --- Makefile | 1 - fetch-object.c | 43 ------------------------------------------- fetch-object.h | 9 --------- promisor-remote.c | 40 +++++++++++++++++++++++++++++++++++++++- 4 files changed, 39 insertions(+), 54 deletions(-) delete mode 100644 fetch-object.c delete mode 100644 fetch-object.h diff --git a/Makefile b/Makefile index 049bc8cfd4..9b0baa7239 100644 --- a/Makefile +++ b/Makefile @@ -880,7 +880,6 @@ LIB_OBJS += ewah/ewah_io.o LIB_OBJS += ewah/ewah_rlw.o LIB_OBJS += exec-cmd.o LIB_OBJS += fetch-negotiator.o -LIB_OBJS += fetch-object.o LIB_OBJS += fetch-pack.o LIB_OBJS += fsck.o LIB_OBJS += fsmonitor.o diff --git a/fetch-object.c b/fetch-object.c deleted file mode 100644 index eac4d448ef..0000000000 --- a/fetch-object.c +++ /dev/null @@ -1,43 +0,0 @@ -#include "cache.h" -#include "packfile.h" -#include "pkt-line.h" -#include "strbuf.h" -#include "transport.h" -#include "fetch-object.h" - -static int fetch_refs(const char *remote_name, struct ref *ref) -{ - struct remote *remote; - struct transport *transport; - int original_fetch_if_missing = fetch_if_missing; - int res; - - fetch_if_missing = 0; - remote = remote_get(remote_name); - if (!remote->url[0]) - die(_("Remote with no URL")); - transport = transport_get(remote, remote->url[0]); - - transport_set_option(transport, TRANS_OPT_FROM_PROMISOR, "1"); - transport_set_option(transport, TRANS_OPT_NO_DEPENDENTS, "1"); - res = transport_fetch_refs(transport, ref); - fetch_if_missing = original_fetch_if_missing; - - return res; -} - -int fetch_objects(const char *remote_name, const struct object_id *oids, - int oid_nr) -{ - struct ref *ref = NULL; - int i; - - for (i = 0; i < oid_nr; i++) { - struct ref *new_ref = alloc_ref(oid_to_hex(&oids[i])); - oidcpy(&new_ref->old_oid, &oids[i]); - new_ref->exact_oid = 1; - new_ref->next = ref; - ref = new_ref; - } - return fetch_refs(remote_name, ref); -} diff --git a/fetch-object.h b/fetch-object.h deleted file mode 100644 index 7bcc7cadb0..0000000000 --- a/fetch-object.h +++ /dev/null @@ -1,9 +0,0 @@ -#ifndef FETCH_OBJECT_H -#define FETCH_OBJECT_H - -struct object_id; - -int fetch_objects(const char *remote_name, const struct object_id *oids, - int oid_nr); - -#endif diff --git a/promisor-remote.c b/promisor-remote.c index 826890f7b8..92c4c12c1c 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -2,7 +2,45 @@ #include "object-store.h" #include "promisor-remote.h" #include "config.h" -#include "fetch-object.h" +#include "transport.h" + +static int fetch_refs(const char *remote_name, struct ref *ref) +{ + struct remote *remote; + struct transport *transport; + int original_fetch_if_missing = fetch_if_missing; + int res; + + fetch_if_missing = 0; + remote = remote_get(remote_name); + if (!remote->url[0]) + die(_("Remote with no URL")); + transport = transport_get(remote, remote->url[0]); + + transport_set_option(transport, TRANS_OPT_FROM_PROMISOR, "1"); + transport_set_option(transport, TRANS_OPT_NO_DEPENDENTS, "1"); + res = transport_fetch_refs(transport, ref); + fetch_if_missing = original_fetch_if_missing; + + return res; +} + +static int fetch_objects(const char *remote_name, + const struct object_id *oids, + int oid_nr) +{ + struct ref *ref = NULL; + int i; + + for (i = 0; i < oid_nr; i++) { + struct ref *new_ref = alloc_ref(oid_to_hex(&oids[i])); + oidcpy(&new_ref->old_oid, &oids[i]); + new_ref->exact_oid = 1; + new_ref->next = ref; + ref = new_ref; + } + return fetch_refs(remote_name, ref); +} static struct promisor_remote *promisors; static struct promisor_remote **promisors_tail = &promisors; From patchwork Tue Jun 25 13:40:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015597 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 984A3112C for ; Tue, 25 Jun 2019 13:41:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89B222852A for ; Tue, 25 Jun 2019 13:41:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7D9C6285A3; Tue, 25 Jun 2019 13:41:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 302DC27FB3 for ; Tue, 25 Jun 2019 13:41:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730274AbfFYNlY (ORCPT ); Tue, 25 Jun 2019 09:41:24 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:35003 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730850AbfFYNlX (ORCPT ); Tue, 25 Jun 2019 09:41:23 -0400 Received: by mail-ed1-f66.google.com with SMTP id w20so19618957edd.2 for ; Tue, 25 Jun 2019 06:41:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L10ZnmDZR3c3XSdNMQHUwIL5QsCHGJ48wMArTxZpXo8=; b=DyINuUEAhqFgpru32hBLBOW5AzRTerfOoaLPocxkDkQ9bhdDKvM1+2ZcyH+sitJxMp ySEKz89Y5+Wp76YhIeiyhkto5Kt2g7obEOzd12s1gNg6rGLK8Kd+yL0CrQk5VbIIZ4Ap P9SRKS4Wjwiabh8MXG4Fw70TtElZ6IjeUd8FxwCyscAK/11AiidIAH6WN4UJluSvq1W9 bpc+GP8HA6DlWAibmMc/+g7a2ezVoeq0tL6Io2TYrn/0gbq6uQJtbAHhnKxC+/ClBH0y OKHPLcTN4L+5HUGCWmjtspb1ynT0HGyLUDnouXBbqJjB52bpSTJ73BbA2Zm1ZLfGHM0g 7uxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L10ZnmDZR3c3XSdNMQHUwIL5QsCHGJ48wMArTxZpXo8=; b=FbSE0bEbvdHmK9O0l9lMxgVNN29tI+gXPT6ECnYfpPoanJV/phAuJesDTntGtqJN8K q/f+e9etQ+M1IpZKJfBEl4e2wlhrwkIrfXUgHcOGP7PAPyrcUI3IzNQcS9ESdzb8xkPS XgiYBGOAP0FUS5I2oaEYgaUhE98nyirLFnlaw5eVVV6gwgbAC5YWcM9p5NZHbwWZODoX j8p5j1DJKmrXbu688xfPt9wkqegexrVwA+m81kOfNRRnw/7sJpcU+qTrnRUYb2/dZOMN aZ4fs/FYiHxi1xYVf4G5wdheLBQb5sHImhpyjQWgNyVrV8qMACRsV9DGeqie1Xz9YjcK 1nVw== X-Gm-Message-State: APjAAAXQj2EPKyyX7SMSeJO7p8rpk26/GKkUIBAMERJFEZKH8MWVghxW eFHjZXXgToxp6wKWR/+rYbT5dmqcu9w= X-Google-Smtp-Source: APXvYqymoPtQvUR19rNVEbHsPbPriKvgHB0VoDcV7LG9PDiyoQ/rw3R2zEsftehJ1F8/vCtF/AYHWQ== X-Received: by 2002:a17:906:2658:: with SMTP id i24mr35357135ejc.18.1561470081645; Tue, 25 Jun 2019 06:41:21 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.20 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:20 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 14/15] Move repository_format_partial_clone to promisor-remote.c Date: Tue, 25 Jun 2019 15:40:38 +0200 Message-Id: <20190625134039.21707-15-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Now that we have has_promisor_remote() and can use many promisor remotes, let's hide repository_format_partial_clone as a static in promisor-remote.c to avoid it being use for anything other than managing backward compatibility. Signed-off-by: Christian Couder --- cache.h | 1 - environment.c | 1 - promisor-remote.c | 7 +++++++ promisor-remote.h | 6 ++++++ setup.c | 3 ++- 5 files changed, 15 insertions(+), 3 deletions(-) diff --git a/cache.h b/cache.h index bf20337ef4..e34b9e66d2 100644 --- a/cache.h +++ b/cache.h @@ -961,7 +961,6 @@ extern int grafts_replace_parents; #define GIT_REPO_VERSION 0 #define GIT_REPO_VERSION_READ 1 extern int repository_format_precious_objects; -extern char *repository_format_partial_clone; extern const char *core_partial_clone_filter_default; extern int repository_format_worktree_config; diff --git a/environment.c b/environment.c index 89af47cb85..8855d2fc11 100644 --- a/environment.c +++ b/environment.c @@ -31,7 +31,6 @@ int warn_ambiguous_refs = 1; int warn_on_object_refname_ambiguity = 1; int ref_paranoia = -1; int repository_format_precious_objects; -char *repository_format_partial_clone; const char *core_partial_clone_filter_default; int repository_format_worktree_config; const char *git_commit_encoding; diff --git a/promisor-remote.c b/promisor-remote.c index 92c4c12c1c..31d51bb50e 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -4,6 +4,13 @@ #include "config.h" #include "transport.h" +static char *repository_format_partial_clone; + +void set_repository_format_partial_clone(char *partial_clone) +{ + repository_format_partial_clone = xstrdup_or_null(partial_clone); +} + static int fetch_refs(const char *remote_name, struct ref *ref) { struct remote *remote; diff --git a/promisor-remote.h b/promisor-remote.h index 838cb092f3..8200dfc940 100644 --- a/promisor-remote.h +++ b/promisor-remote.h @@ -22,4 +22,10 @@ extern int promisor_remote_get_direct(struct repository *repo, const struct object_id *oids, int oid_nr); +/* + * This should be used only once from setup.c to set the value we got + * from the extensions.partialclone config option. + */ +extern void set_repository_format_partial_clone(char *partial_clone); + #endif /* PROMISOR_REMOTE_H */ diff --git a/setup.c b/setup.c index 8dcb4631f7..25a3038277 100644 --- a/setup.c +++ b/setup.c @@ -4,6 +4,7 @@ #include "dir.h" #include "string-list.h" #include "chdir-notify.h" +#include "promisor-remote.h" static int inside_git_dir = -1; static int inside_work_tree = -1; @@ -478,7 +479,7 @@ static int check_repository_format_gently(const char *gitdir, struct repository_ } repository_format_precious_objects = candidate->precious_objects; - repository_format_partial_clone = xstrdup_or_null(candidate->partial_clone); + set_repository_format_partial_clone(candidate->partial_clone); repository_format_worktree_config = candidate->worktree_config; string_list_clear(&candidate->unknown_extensions, 0); From patchwork Tue Jun 25 13:40:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 11015599 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A5641575 for ; Tue, 25 Jun 2019 13:41:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D840285A3 for ; Tue, 25 Jun 2019 13:41:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 21FCE286E5; Tue, 25 Jun 2019 13:41:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8B4C286F2 for ; Tue, 25 Jun 2019 13:41:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730434AbfFYNla (ORCPT ); Tue, 25 Jun 2019 09:41:30 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:35007 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726871AbfFYNlY (ORCPT ); Tue, 25 Jun 2019 09:41:24 -0400 Received: by mail-ed1-f67.google.com with SMTP id w20so19619057edd.2 for ; Tue, 25 Jun 2019 06:41:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=P+Uh3OM2UyPjAqXRs0MwDF+mqy/zbc7X8AYuR/mZiNU=; b=MGbqvh5pI9gn2olY8opW1WM1MDH9FoX/KbW/1De/L+SlRGcAAotYBlftbHV6MUw6Rs 6UMLAHZWr3TE6tKf3B9oOS6QP7qc/ag87v+Omq1/mm42oMvk8GWa04E525tyUN569DMY xcn/TT56KUWn+UJ+Z501TKNLIBhJjVLqddQPLludkpLfLou6XsK2jM3r7VWKnMMnnedD JmTBP+QpDl793xDZWepcUO7D+0dV7Ye7ydHA44XIEXYz/M0amjBKpzpLYgA2Qshz579F pT8x0Ar5YF7Hmg5t8xhHeUKxGdvrbQ9M8skTlJX3uBroj7mKzRjpDtXxa/YM0Av7V2bT h2nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P+Uh3OM2UyPjAqXRs0MwDF+mqy/zbc7X8AYuR/mZiNU=; b=idR03oAPFqhiHW0aFtu2zdJehvqi6e/Yh1dFSoAwv8bVKP8r1AHRh2wCcXqIvUU2Wh kU4vhAZNO2VZXVCuFdhTV+HjR3UXluSkXHcExa7vfOTxiYs7b55MpwiAXmGjb53pZTFO 9C0tTJoCKuyeUyRjp/ojvFjBpXIG4cF9VetNtm9AB3kpBPGliGsL+z65039Z6B7rVY7i 6j51AuHnVdas3Hk3N0tYOdWqcwhlOIFzw/jkWqAsKdaLqSB1JhOy8EJnEBaiw+i8KhKK qFpb5FK48pTpmzVaXprTxhJZYGyJjuafbFJB6ARknXtJymkRGFWK9akUHOlGvOKHfrQK btVw== X-Gm-Message-State: APjAAAX1q2nnOuvSUkn2dEmILxIgmnKkadhBzgdixyg54NUATdtAJO0T FSrpohT0oQ163RHr/noG3IQltgz7MyY= X-Google-Smtp-Source: APXvYqzrM4bNSqSqu/oN3jG7vx9wldWk206mZGN4GH2VO+wsnvnBcq5X3XuK89afpwtKFankBN+f3g== X-Received: by 2002:a50:86b7:: with SMTP id r52mr145425023eda.100.1561470082679; Tue, 25 Jun 2019 06:41:22 -0700 (PDT) Received: from ubuntu-N150ZU.booking.pcln.com (access-114.38.rev.fr.colt.net. [213.41.38.114]) by smtp.gmail.com with ESMTPSA id v11sm2448445ejx.24.2019.06.25.06.41.21 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 25 Jun 2019 06:41:22 -0700 (PDT) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Jonathan Nieder , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler , Eric Sunshine , Beat Bolli , =?utf-8?q?SZEDER_G=C3=A1bor?= , Ramsay Jones , Derrick Stolee Subject: [PATCH v6 15/15] Move core_partial_clone_filter_default to promisor-remote.c Date: Tue, 25 Jun 2019 15:40:39 +0200 Message-Id: <20190625134039.21707-16-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.22.0.229.ga13d9ffdf7.dirty In-Reply-To: <20190625134039.21707-1-chriscool@tuxfamily.org> References: <20190625134039.21707-1-chriscool@tuxfamily.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Now that we can have a different default partial clone filter for each promisor remote, let's hide core_partial_clone_filter_default as a static in promisor-remote.c to avoid it being use for anything other than managing backward compatibility. Signed-off-by: Christian Couder --- cache.h | 1 - config.c | 5 ----- environment.c | 1 - promisor-remote.c | 5 +++++ 4 files changed, 5 insertions(+), 7 deletions(-) diff --git a/cache.h b/cache.h index e34b9e66d2..a4d7f84eeb 100644 --- a/cache.h +++ b/cache.h @@ -961,7 +961,6 @@ extern int grafts_replace_parents; #define GIT_REPO_VERSION 0 #define GIT_REPO_VERSION_READ 1 extern int repository_format_precious_objects; -extern const char *core_partial_clone_filter_default; extern int repository_format_worktree_config; /* diff --git a/config.c b/config.c index 296a6d9cc4..317b226bc8 100644 --- a/config.c +++ b/config.c @@ -1344,11 +1344,6 @@ static int git_default_core_config(const char *var, const char *value, void *cb) return 0; } - if (!strcmp(var, "core.partialclonefilter")) { - return git_config_string(&core_partial_clone_filter_default, - var, value); - } - if (!strcmp(var, "core.usereplacerefs")) { read_replace_refs = git_config_bool(var, value); return 0; diff --git a/environment.c b/environment.c index 8855d2fc11..efa072680a 100644 --- a/environment.c +++ b/environment.c @@ -31,7 +31,6 @@ int warn_ambiguous_refs = 1; int warn_on_object_refname_ambiguity = 1; int ref_paranoia = -1; int repository_format_precious_objects; -const char *core_partial_clone_filter_default; int repository_format_worktree_config; const char *git_commit_encoding; const char *git_log_output_encoding; diff --git a/promisor-remote.c b/promisor-remote.c index 31d51bb50e..9bc296cdde 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -5,6 +5,7 @@ #include "transport.h" static char *repository_format_partial_clone; +static const char *core_partial_clone_filter_default; void set_repository_format_partial_clone(char *partial_clone) { @@ -103,6 +104,10 @@ static int promisor_remote_config(const char *var, const char *value, void *data int namelen; const char *subkey; + if (!strcmp(var, "core.partialclonefilter")) + return git_config_string(&core_partial_clone_filter_default, + var, value); + if (parse_config_key(var, "remote", &name, &namelen, &subkey) < 0) return 0;