From patchwork Mon Aug 3 18:00:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11698533 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F2F3913B6 for ; Mon, 3 Aug 2020 18:00:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0AB3C22B45 for ; Mon, 3 Aug 2020 18:00:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="eZxj2FHh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728689AbgHCSAT (ORCPT ); Mon, 3 Aug 2020 14:00:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727050AbgHCSAQ (ORCPT ); Mon, 3 Aug 2020 14:00:16 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F706C06174A for ; Mon, 3 Aug 2020 11:00:16 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id x12so20386600qtp.1 for ; Mon, 03 Aug 2020 11:00:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=PJrFU+75QPJCoYigEyOKR4oxa0fQ98NHJ7TU05rAv8E=; b=eZxj2FHhNss1xlLhLQeM6KuCDRr/VW+0tkoxCz6DqlV52dMXCM4ZPxtEPv09wS8iRx BoI0pHnnJfe5iu3jXaP40fI4OGBNSP0ls0FfG6I/cCBeZHEXMIm+p8TezRs/Wxcu71pv xS0qj3uZF/ubC7MlaOXG2+14B/Z/iF7ukc0RiZMjbPDZOJKjcDS2mU/1sX6eAVDvl/RU 2wOl9NmpXPwVzaMMSJZZYSyAbrbn+11Uf4pDbz2QCM9X0cbv2i2CmLM/OlpwkCBGAmjw E9BQVz8XkTYmevIXIkl5898fa/2/A+zQcE5Jbd+HCKgbXcUqXSi8IyqeAogOFGLsV+nN kPgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=PJrFU+75QPJCoYigEyOKR4oxa0fQ98NHJ7TU05rAv8E=; b=H2JZBaTEj27Aido//wgD30iU77eas3vdj5lWeoUBmvzjlrRsSk+YxxOrb10mAKqDBU FiUUgNr/PJ3aTZk9pK4jpt2hnZbyeIX+ZcAB3rp7RIxg3hjRfwZG2IEhV9Ru0BCSXUAk D2pyeyVSaK1nCX0zjmTwObxFEKYDK45EsyTy4BiRopCSh8v45QGYgehTWILmiGSBUdmo fTVC66GJUIlrjMyDCRTHK8Jk2l2V3DGToAgAvD3qBNfeKcvyWrFRmwxibM2lzorRIqmq xtzmPRaG4favNqCmfjbtsLewuiKf/o8SVRH6S+ZXOk71ORHOkpWytVecnjLYA2CaqVOT 5y1w== X-Gm-Message-State: AOAM531rcDfWGFwkjX/0TmSFHJw4ZaiMfXIJIKjyY1Q+OdUL0R5r7pmf NCuPMrRTBOgYZnmBww1c//NSwa9eK0Uysg== X-Google-Smtp-Source: ABdhPJxvyeaoN4GJEA2Yd6IhvYm8DTf5kkYlnhkd3+L0IJdzZ5m1prq34cxi7myOe0zgvDYhh7L8BA== X-Received: by 2002:ac8:71cf:: with SMTP id i15mr17823412qtp.119.1596477615511; Mon, 03 Aug 2020 11:00:15 -0700 (PDT) Received: from localhost ([2605:9480:22e:ff10:3475:b417:c07c:c811]) by smtp.gmail.com with ESMTPSA id f4sm21752519qtp.38.2020.08.03.11.00.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2020 11:00:14 -0700 (PDT) Date: Mon, 3 Aug 2020 14:00:14 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, chriscool@tuxfamily.org, gitster@pobox.com, szeder.dev@gmail.com Subject: [PATCH v4 1/3] list_objects_filter_options: introduce 'list_object_filter_config_name' Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a subsequent commit, we will add configuration options that are specific to each kind of object filter, in which case it is handy to have a function that translates between 'enum list_objects_filter_choice' and an appropriate configuration-friendly string. Signed-off-by: Taylor Blau --- list-objects-filter-options.c | 23 +++++++++++++++++++++++ list-objects-filter-options.h | 6 ++++++ 2 files changed, 29 insertions(+) diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index 3553ad7b0a..92b408c0c8 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -15,6 +15,29 @@ static int parse_combine_filter( const char *arg, struct strbuf *errbuf); +const char *list_object_filter_config_name(enum list_objects_filter_choice c) +{ + switch (c) { + case LOFC_DISABLED: + /* we have no name for "no filter at all" */ + break; + case LOFC_BLOB_NONE: + return "blob:none"; + case LOFC_BLOB_LIMIT: + return "blob:limit"; + case LOFC_TREE_DEPTH: + return "tree"; + case LOFC_SPARSE_OID: + return "sparse:oid"; + case LOFC_COMBINE: + return "combine"; + case LOFC__COUNT: + /* not a real filter type; just the count of all filters */ + break; + } + BUG("list_object_filter_choice_name: invalid argument '%d'", c); +} + /* * Parse value of the argument to the "filter" keyword. * On the command line this looks like: diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 73fffa4ad7..01767c3c96 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -17,6 +17,12 @@ enum list_objects_filter_choice { LOFC__COUNT /* must be last */ }; +/* + * Returns a configuration key suitable for describing the given object filter, + * e.g.: "blob:none", "combine", etc. + */ +const char *list_object_filter_config_name(enum list_objects_filter_choice c); + struct list_objects_filter_options { /* * 'filter_spec' is the raw argument value given on the command line From patchwork Mon Aug 3 18:00:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11698529 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0ED191575 for ; Mon, 3 Aug 2020 18:00:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1713722B4E for ; Mon, 3 Aug 2020 18:00:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="xUoot0Gt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727070AbgHCSAO (ORCPT ); Mon, 3 Aug 2020 14:00:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726239AbgHCSAN (ORCPT ); Mon, 3 Aug 2020 14:00:13 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F0F9C06174A for ; Mon, 3 Aug 2020 11:00:13 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id g26so36008107qka.3 for ; Mon, 03 Aug 2020 11:00:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=+rqJWSzjO6AwAzEo8mDVubZovA8EKMN9Owj/xcxOO+8=; b=xUoot0GtxoCrKKzLkIOq9yFfp7sKBcCwd//4tjF2XoR1+/LsSHLmeKZ+2gzOwAjELm 7pXuTzWegKGQvcu/WZV19MJ8rPM3fQMZVLIHtKnnrpFqpJGJFJTY5he7L1AtOPCvkf87 rDx5CxR13J+FY5mphKGRBof8S0zQFvYO5MADJuVMNkZ/1zjQiZSMuk6ln7cuz3Es0q+L 1Zmw46W3c+Jl+xzi32mRGnt1kRkybuzkkLStBtxRW3nauoU1hNJyQ9B2dzdORjI1MmxZ khkM+K08BHDiLeps/SQyqnWiUeHUTJRrTAfqhl1a7NQ71+gPNdzeRZ0T2ZwjDAM1jNPl M47Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=+rqJWSzjO6AwAzEo8mDVubZovA8EKMN9Owj/xcxOO+8=; b=cjqFwgG0xjQj7hiqWPFFeHjlic15ZWhJYMuL2UArDd3Cs06z0HBhHfpyZ0zh3W3aXA EZWEbWcOI3A+1N4/+jjjB+IWqStBTZa64D0KVE0x7/FMoViuN3xHbVf1hx63dMkODoOa Wq33YGNhf6WjZNYumq0LFiZix5HNk4ryJeUlCNdJWyNlJ+WfGklA8x7kogfm2ucGf3FG vCmYEtveZ53NKUbsNnMJ9ZdHuWg3X9y3tZR5ssAqX39aFvZOkJNyU0KW77f3Ng7m7WCk v3vh/pg2uyQm0fRqkhN6bOy/3SqOtkovXANNqEU10zH/R9A97eVCg59Se4ppMoGfwzoo svmQ== X-Gm-Message-State: AOAM53242O+M8KiJ/dESkHpfFkCXl5Idt6eBbMOUuewu8j6v+QxjyoBv nx7a68GnXdrRKxLAY+VdZ2xz5Q2a9a0GzA== X-Google-Smtp-Source: ABdhPJxdbfO/5s+78OCXuBI0r8m7Mp3sTbdy+BSSz1QFI6UzqHHwCuyZBM0Agu/oIFt8wGmX9bucVA== X-Received: by 2002:a37:6594:: with SMTP id z142mr16972762qkb.10.1596477611968; Mon, 03 Aug 2020 11:00:11 -0700 (PDT) Received: from localhost ([2605:9480:22e:ff10:3475:b417:c07c:c811]) by smtp.gmail.com with ESMTPSA id 3sm21155590qkm.117.2020.08.03.11.00.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2020 11:00:11 -0700 (PDT) Date: Mon, 3 Aug 2020 14:00:10 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, chriscool@tuxfamily.org, gitster@pobox.com, szeder.dev@gmail.com Subject: [PATCH v4 2/3] upload-pack.c: allow banning certain object filter(s) Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Git clients may ask the server for a partial set of objects, where the set of objects being requested is refined by one or more object filters. Server administrators can configure 'git upload-pack' to allow or ban these filters by setting the 'uploadpack.allowFilter' variable to 'true' or 'false', respectively. However, administrators using bitmaps may wish to allow certain kinds of object filters, but ban others. Specifically, they may wish to allow object filters that can be optimized by the use of bitmaps, while rejecting other object filters which aren't and represent a perceived performance degradation (as well as an increased load factor on the server). Allow configuring 'git upload-pack' to support object filters on a case-by-case basis by introducing two new configuration variables: - 'uploadpackfilter.allow' - 'uploadpackfilter..allow' where '' may be one of 'blobNone', 'blobLimit', 'tree', and so on. Setting the second configuration variable for any valid value of '' explicitly allows or disallows restricting that kind of object filter. If a client requests the object filter and the respective configuration value is not set, 'git upload-pack' will default to the value of 'uploadpackfilter.allow', which itself defaults to 'true' to maintain backwards compatibility. Note that this differs from 'uploadpack.allowfilter', which controls whether or not the 'filter' capability is advertised. Helped-by: Jeff King Signed-off-by: Taylor Blau --- Documentation/config/uploadpack.txt | 12 ++++ t/t5616-partial-clone.sh | 24 ++++++++ upload-pack.c | 86 +++++++++++++++++++++++++++++ 3 files changed, 122 insertions(+) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index ed1c835695..fffe8ac648 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -57,6 +57,18 @@ uploadpack.allowFilter:: If this option is set, `upload-pack` will support partial clone and partial fetch object filtering. +uploadpackfilter.allow:: + Provides a default value for unspecified object filters (see: the + below configuration variable). + Defaults to `true`. + +uploadpackfilter..allow:: + Explicitly allow or ban the object filter corresponding to + ``, where `` may be one of: `blob:none`, + `blob:limit`, `tree`, `sparse:oid`, or `combine`. If using + combined filters, both `combine` and all of the nested filter + kinds must be allowed. Defaults to `uploadpackfilter.allow`. + uploadpack.allowRefInWant:: If this option is set, `upload-pack` will support the `ref-in-want` feature of the protocol version 2 `fetch` command. This feature diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index 37de0afb02..aa68782587 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -235,6 +235,30 @@ test_expect_success 'implicitly construct combine: filter with repeated flags' ' test_cmp unique_types.expected unique_types.actual ' +test_expect_success 'upload-pack fails banned object filters' ' + test_config -C srv.bare uploadpackfilter.blob:none.allow false && + test_must_fail ok=sigpipe git clone --no-checkout --filter=blob:none \ + "file://$(pwd)/srv.bare" pc3 2>err && + grep "filter '\''blob:none'\'' not supported" err +' + +test_expect_success 'upload-pack fails banned combine object filters' ' + test_config -C srv.bare uploadpackfilter.allow false && + test_config -C srv.bare uploadpackfilter.combine.allow true && + test_config -C srv.bare uploadpackfilter.tree.allow true && + test_config -C srv.bare uploadpackfilter.blob:none.allow false && + test_must_fail ok=sigpipe git clone --no-checkout --filter=tree:1 \ + --filter=blob:none "file://$(pwd)/srv.bare" pc3 2>err && + grep "filter '\''blob:none'\'' not supported" err +' + +test_expect_success 'upload-pack fails banned object filters with fallback' ' + test_config -C srv.bare uploadpackfilter.allow false && + test_must_fail ok=sigpipe git clone --no-checkout --filter=blob:none \ + "file://$(pwd)/srv.bare" pc3 2>err && + grep "filter '\''blob:none'\'' not supported" err +' + test_expect_success 'partial clone fetches blobs pointed to by refs even if normally filtered out' ' rm -rf src dst && git init src && diff --git a/upload-pack.c b/upload-pack.c index 8673741070..2098af8156 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -88,6 +88,7 @@ struct upload_pack_data { enum allow_uor allow_uor; struct list_objects_filter_options filter_options; + struct string_list allowed_filters; struct packet_writer writer; @@ -103,6 +104,7 @@ struct upload_pack_data { unsigned no_progress : 1; unsigned use_include_tag : 1; unsigned allow_filter : 1; + unsigned allow_filter_fallback : 1; unsigned done : 1; /* v2 only */ unsigned allow_ref_in_want : 1; /* v2 only */ @@ -120,6 +122,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) struct string_list deepen_not = STRING_LIST_INIT_DUP; struct string_list uri_protocols = STRING_LIST_INIT_DUP; struct object_array extra_edge_obj = OBJECT_ARRAY_INIT; + struct string_list allowed_filters = STRING_LIST_INIT_DUP; memset(data, 0, sizeof(*data)); data->symref = symref; @@ -131,6 +134,8 @@ static void upload_pack_data_init(struct upload_pack_data *data) data->deepen_not = deepen_not; data->uri_protocols = uri_protocols; data->extra_edge_obj = extra_edge_obj; + data->allowed_filters = allowed_filters; + data->allow_filter_fallback = 1; packet_writer_init(&data->writer, 1); data->keepalive = 5; @@ -147,6 +152,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data) string_list_clear(&data->deepen_not, 0); object_array_clear(&data->extra_edge_obj); list_objects_filter_release(&data->filter_options); + string_list_clear(&data->allowed_filters, 1); free((char *)data->pack_objects_hook); } @@ -984,6 +990,56 @@ static int process_deepen_not(const char *line, struct string_list *deepen_not, return 0; } +NORETURN __attribute__((format(printf,2,3))) +static void send_err_and_die(struct upload_pack_data *data, + const char *fmt, ...) +{ + struct strbuf buf = STRBUF_INIT; + va_list ap; + + va_start(ap, fmt); + strbuf_vaddf(&buf, fmt, ap); + va_end(ap); + + packet_writer_error(&data->writer, "%s", buf.buf); + die("%s", buf.buf); +} + +static void check_one_filter(struct upload_pack_data *data, + struct list_objects_filter_options *opts) +{ + const char *key = list_object_filter_config_name(opts->choice); + struct string_list_item *item = string_list_lookup(&data->allowed_filters, + key); + int allowed; + + if (item) + allowed = (intptr_t)item->util; + else + allowed = data->allow_filter_fallback; + + if (!allowed) + send_err_and_die(data, "filter '%s' not supported", key); +} + +static void check_filter_recurse(struct upload_pack_data *data, + struct list_objects_filter_options *opts) +{ + size_t i; + + check_one_filter(data, opts); + if (opts->choice != LOFC_COMBINE) + return; + + for (i = 0; i < opts->sub_nr; i++) + check_filter_recurse(data, &opts->sub[i]); +} + +static void die_if_using_banned_filter(struct upload_pack_data *data) +{ + check_filter_recurse(data, &data->filter_options); +} + static void receive_needs(struct upload_pack_data *data, struct packet_reader *reader) { @@ -1014,6 +1070,7 @@ static void receive_needs(struct upload_pack_data *data, die("git upload-pack: filtering capability not negotiated"); list_objects_filter_die_if_populated(&data->filter_options); parse_list_objects_filter(&data->filter_options, arg); + die_if_using_banned_filter(data); continue; } @@ -1171,6 +1228,32 @@ static int find_symref(const char *refname, const struct object_id *oid, return 0; } +static int parse_object_filter_config(const char *var, const char *value, + struct upload_pack_data *data) +{ + struct strbuf buf = STRBUF_INIT; + const char *sub, *key; + size_t sub_len; + + if (parse_config_key(var, "uploadpackfilter", &sub, &sub_len, &key)) + return 0; + + if (!sub) { + if (!strcmp(key, "allow")) + data->allow_filter_fallback = git_config_bool(var, value); + return 0; + } + + strbuf_add(&buf, sub, sub_len); + + if (!strcmp(key, "allow")) + string_list_insert(&data->allowed_filters, buf.buf)->util = + (void *)(intptr_t)git_config_bool(var, value); + + strbuf_release(&buf); + return 0; +} + static int upload_pack_config(const char *var, const char *value, void *cb_data) { struct upload_pack_data *data = cb_data; @@ -1210,6 +1293,8 @@ static int upload_pack_config(const char *var, const char *value, void *cb_data) return git_config_string(&data->pack_objects_hook, var, value); } + parse_object_filter_config(var, value, data); + return parse_hide_refs_config(var, value, "uploadpack"); } @@ -1390,6 +1475,7 @@ static void process_args(struct packet_reader *request, if (data->allow_filter && skip_prefix(arg, "filter ", &p)) { list_objects_filter_die_if_populated(&data->filter_options); parse_list_objects_filter(&data->filter_options, p); + die_if_using_banned_filter(data); continue; } From patchwork Mon Aug 3 18:00:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11698535 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1167F138A for ; Mon, 3 Aug 2020 18:00:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E9C022B45 for ; Mon, 3 Aug 2020 18:00:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="Vcc75xj5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727783AbgHCSAZ (ORCPT ); Mon, 3 Aug 2020 14:00:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727050AbgHCSAU (ORCPT ); Mon, 3 Aug 2020 14:00:20 -0400 Received: from mail-qv1-xf41.google.com (mail-qv1-xf41.google.com [IPv6:2607:f8b0:4864:20::f41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93057C06174A for ; Mon, 3 Aug 2020 11:00:20 -0700 (PDT) Received: by mail-qv1-xf41.google.com with SMTP id b2so7178505qvp.9 for ; Mon, 03 Aug 2020 11:00:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=Wwjj3nW+/A4IB9Jolhj7VMy6tdooPj+JpfFiwvAHNkY=; b=Vcc75xj5d8ySfJA187zmQ03aL2HkmOwZ4wszfymmbGbH5MsPH55SaLKktd82OX7PSB IjFxkYd18zauMJn/WheEhJAsOsvF0/xi3tKLLg1AbxzuUBAgdODXhaU/6K/WT5nj6sYE 6Y3mVxY0Sl02ZPDOnefbdI5Y8/3YyHH05MDP+bdZxd+ApzxVYK81ZIBcEVlwbblaWq9s PmYSitQAFwr6CU+Ov+8P00IpIuMw27fj6oLHSQoonwtf5exDVDA4ZkqQYC2ipLmv5Al5 qye8QAXP2A/sZMiTFCMbdlQmqb/AJNV2FZuQQZ2UtPCNwqGQ4d5KT9aXOrY07APRrWCF UnNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Wwjj3nW+/A4IB9Jolhj7VMy6tdooPj+JpfFiwvAHNkY=; b=h3sNCnz/7HSV9k1CDEQPtL3atjLrTx5zyFgROhhlLVdwSmgiG1kDQpBrTVdCrDWXN7 BifUWrlv67Dtd3LpnLFyLVXAgdhYEBzLg/reH9vFfeYbzxNBsaSaYs8hvYWjUnB32VpM 3fKvLtn1QQbJDShuC2ePAqePhetBRAjAinxygqPcYUPdCzNc0QQi2szllY4svWmqS6xi biQt/BrNbOr6AyHAjrMteXmGo7Um3JuoKOOKhAQwFX5ucDLsi6G4NxG4GH+ki7y9AfJ8 xslPT4yXMW/n1qRAIuOl8OPIS9l88HP/vSviE3LlpTti9/ihscAoaBZAM48a/3IcbTA2 a16A== X-Gm-Message-State: AOAM533cg7lNBsFzaSLixdruLJC4GaXDCho5J8nx8pYqRX4zAbHjtPX5 WSXvprKeZTNaAcSBbQK/4xYhwLaDYFALqA== X-Google-Smtp-Source: ABdhPJwiRs2D+yoywHEAV5t24Vy1mS1qsiz53pFpklipNn09LJtlo+4K2DKtdhl8+T+Yc89wrF3TWQ== X-Received: by 2002:a05:6214:1143:: with SMTP id b3mr18513307qvt.84.1596477618980; Mon, 03 Aug 2020 11:00:18 -0700 (PDT) Received: from localhost ([2605:9480:22e:ff10:3475:b417:c07c:c811]) by smtp.gmail.com with ESMTPSA id j15sm18695799qkl.63.2020.08.03.11.00.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2020 11:00:18 -0700 (PDT) Date: Mon, 3 Aug 2020 14:00:17 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, chriscool@tuxfamily.org, gitster@pobox.com, szeder.dev@gmail.com Subject: [PATCH v4 3/3] upload-pack.c: introduce 'uploadpackfilter.tree.maxDepth' Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In b79cf959b2 (upload-pack.c: allow banning certain object filter(s), 2020-02-26), we introduced functionality to disallow certain object filters from being chosen from within 'git upload-pack'. Traditionally, administrators use this functionality to disallow filters that are known to perform slowly, for e.g., those that do not have bitmap-level filtering. In the past, the '--filter=tree:' was one such filter that does not have bitmap-level filtering support, and so was likely to be banned by administrators. However, in the previous couple of commits, we introduced bitmap-level filtering for the case when 'n' is equal to '0', i.e., as if we had a '--filter=tree:none' choice. While it would be sufficient to simply write $ git config uploadpackfilter.tree.allow true (since it would allow all values of 'n'), we would like to be able to allow this filter for certain values of 'n', i.e., those no greater than some pre-specified maximum. In order to do this, introduce a new configuration key, as follows: $ git config uploadpackfilter.tree.maxDepth where '' specifies the maximum allowed value of 'n' in the filter 'tree:n'. Administrators who wish to allow for only the value '0' can write: $ git config uploadpackfilter.tree.allow true $ git config uploadpackfilter.tree.maxDepth 0 which allows '--filter=tree:0', but no other values. Signed-off-by: Taylor Blau --- Documentation/config/uploadpack.txt | 6 ++++++ t/t5616-partial-clone.sh | 9 +++++++++ upload-pack.c | 18 ++++++++++++++++++ 3 files changed, 33 insertions(+) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index fffe8ac648..ee7b3ac94f 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -69,6 +69,12 @@ uploadpackfilter..allow:: combined filters, both `combine` and all of the nested filter kinds must be allowed. Defaults to `uploadpackfilter.allow`. +uploadpackfilter.tree.maxDepth:: + Only allow `--filter=tree=` when `n` is no more than the value of + `uploadpackfilter.tree.maxDepth`. If set, this also implies + `uploadpackfilter.tree.allow=true`, unless this configuration + variable had already been set. Has no effect if unset. + uploadpack.allowRefInWant:: If this option is set, `upload-pack` will support the `ref-in-want` feature of the protocol version 2 `fetch` command. This feature diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index aa68782587..96ab45feab 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -259,6 +259,15 @@ test_expect_success 'upload-pack fails banned object filters with fallback' ' grep "filter '\''blob:none'\'' not supported" err ' +test_expect_success 'upload-pack limits tree depth filters' ' + test_config -C srv.bare uploadpackfilter.allow false && + test_config -C srv.bare uploadpackfilter.tree.allow true && + test_config -C srv.bare uploadpackfilter.tree.maxDepth 0 && + test_must_fail ok=sigpipe git clone --no-checkout --filter=tree:1 \ + "file://$(pwd)/srv.bare" pc3 2>err && + grep "tree filter allows max depth 0, but got 1" err +' + test_expect_success 'partial clone fetches blobs pointed to by refs even if normally filtered out' ' rm -rf src dst && git init src && diff --git a/upload-pack.c b/upload-pack.c index 2098af8156..fbd2db04cd 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -105,6 +105,7 @@ struct upload_pack_data { unsigned use_include_tag : 1; unsigned allow_filter : 1; unsigned allow_filter_fallback : 1; + unsigned long tree_filter_max_depth; unsigned done : 1; /* v2 only */ unsigned allow_ref_in_want : 1; /* v2 only */ @@ -136,6 +137,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) data->extra_edge_obj = extra_edge_obj; data->allowed_filters = allowed_filters; data->allow_filter_fallback = 1; + data->tree_filter_max_depth = ULONG_MAX; packet_writer_init(&data->writer, 1); data->keepalive = 5; @@ -1020,6 +1022,13 @@ static void check_one_filter(struct upload_pack_data *data, if (!allowed) send_err_and_die(data, "filter '%s' not supported", key); + + if (opts->choice == LOFC_TREE_DEPTH && + opts->tree_exclude_depth > data->tree_filter_max_depth) + send_err_and_die(data, + "tree filter allows max depth %lu, but got %lu", + data->tree_filter_max_depth, + opts->tree_exclude_depth); } static void check_filter_recurse(struct upload_pack_data *data, @@ -1249,6 +1258,15 @@ static int parse_object_filter_config(const char *var, const char *value, if (!strcmp(key, "allow")) string_list_insert(&data->allowed_filters, buf.buf)->util = (void *)(intptr_t)git_config_bool(var, value); + else if (!strcmp(buf.buf, "tree") && !strcmp(key, "maxdepth")) { + if (!value) { + strbuf_release(&buf); + return config_error_nonbool(var); + } + string_list_insert(&data->allowed_filters, buf.buf)->util = + (void *)(intptr_t)1; + data->tree_filter_max_depth = git_config_ulong(var, value); + } strbuf_release(&buf); return 0;