From patchwork Wed May 22 00:21:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10954597 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8421924 for ; Wed, 22 May 2019 00:22:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C39A628ADE for ; Wed, 22 May 2019 00:22:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B493728AE6; Wed, 22 May 2019 00:22:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 66E8528ADE for ; Wed, 22 May 2019 00:22:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727244AbfEVAWT (ORCPT ); Tue, 21 May 2019 20:22:19 -0400 Received: from mail-oi1-f202.google.com ([209.85.167.202]:47217 "EHLO mail-oi1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725797AbfEVAWT (ORCPT ); Tue, 21 May 2019 20:22:19 -0400 Received: by mail-oi1-f202.google.com with SMTP id v3so308803oia.14 for ; Tue, 21 May 2019 17:22:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Fu+KxqorTXsSSSbU7rsIFkxijPGpB0m1O5O/e2ApYPA=; b=FPrbLs2txqvnb7IVnKUuUMTTuqjRaxVcLdK5gV2Et8+V8DwBAz+dwimNQRcPJjoA5n p/GS+9auqBZB/xw/1jHKNgnk46awGCL8WV2+fCCdKeo++GCAK9tTxTw+M92MDpINwwdj ugct2oYui0pQDsE1gGm72+wVp6+NSDL8vNmbOf95CiTfbHB4KvdnOH2VRlnbFLxSVl7H MSrjkHqR4iHXGGHt6kPmQasV1AAnktBs6dr7NXuhxA77yxe7TQimpDkL9zThYDJKqR6Y +FluU7ae/efyEfDOVgl/LwUKdeG5S2Tn4hZMJjN/bQAN80aw/xB+FVexnhe3bHOr2gQ8 kipA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Fu+KxqorTXsSSSbU7rsIFkxijPGpB0m1O5O/e2ApYPA=; b=DykdcCSsO3DB4HQLgIZJUlcUGTYsiVewR3N/F2fDxyocUc/fc1Y3ifyxwBU/vyniXF L34Rtmtpzl17BeQ2d+95ozsVq/qObeBn0EK/DODEWwEQL+UDLiBUoNfRxlKolwByLpiX LUrxBSZVg07MQmn851HjUagmlQJQYbYu3/Bi4ivJ281ylk8gio/9WgzZTCzX/69ZrWWI fnAEIa8jBHa3U0HDr7sCx5qvnKsNhobWYHQMokIXzSNzoSeT4XEEsE6EweTIVcwXQWYS RXa50rFUzj76J0bt/Tf/HqaL+zol5EpY/r68vcGzzMdJIplXiJb2YOedp4yw3h1kkt57 ytvQ== X-Gm-Message-State: APjAAAVIP9KQTffRVC8qHtOfXXmGMsKM8HxpLSAgztJyXpIqTDvxscKp Gdn9Sd748ZgmWe4j6Xkw47LFian7P9I= X-Google-Smtp-Source: APXvYqzsrw4OZ20nyWtkfrse8+F6CoSvUquACInFHA32cdupvF7ksNv1muYAh3xWYx14FowyDHk5ndFBf9dj X-Received: by 2002:aca:240d:: with SMTP id n13mr2478071oic.145.1558484538236; Tue, 21 May 2019 17:22:18 -0700 (PDT) Date: Tue, 21 May 2019 17:21:50 -0700 In-Reply-To: Message-Id: <341bc55d4a3f5438b1523525cf683f96d75e8c3e.1558484115.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH v1 1/5] list-objects-filter: refactor into a context struct From: Matthew DeVore To: jonathantanmy@google.com, jrn@google.com, git@vger.kernel.org, dstolee@microsoft.com, jeffhost@microsoft.com, jrnieder@gmail.com, pclouds@gmail.com Cc: Matthew DeVore , matvore@comcast.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The next patch will create and manage filters in a new way, which means that this bundle of data will have to be managed at a new callsite. Make this bundle of data more manageable by putting it in a struct and making it part of the list-objects-filter module's API. Signed-off-by: Matthew DeVore --- list-objects-filter.c | 156 +++++++++++++++--------------------------- list-objects-filter.h | 31 ++++++--- list-objects.c | 45 ++++++------ 3 files changed, 96 insertions(+), 136 deletions(-) diff --git a/list-objects-filter.c b/list-objects-filter.c index ee449de3f7..8e8616b9b8 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -19,82 +19,60 @@ * FILTER_SHOWN_BUT_REVISIT -- we set this bit on tree objects * that have been shown, but should be revisited if they appear * in the traversal (until we mark it SEEN). This is a way to * let us silently de-dup calls to show() in the caller. This * is subtly different from the "revision.h:SHOWN" and the * "sha1-name.c:ONELINE_SEEN" bits. And also different from * the non-de-dup usage in pack-bitmap.c */ #define FILTER_SHOWN_BUT_REVISIT (1<<21) -/* - * A filter for list-objects to omit ALL blobs from the traversal. - * And to OPTIONALLY collect a list of the omitted OIDs. - */ -struct filter_blobs_none_data { - struct oidset *omits; -}; - static enum list_objects_filter_result filter_blobs_none( struct repository *r, enum list_objects_filter_situation filter_situation, struct object *obj, const char *pathname, const char *filename, - void *filter_data_) + struct filter_context *ctx) { - struct filter_blobs_none_data *filter_data = filter_data_; - switch (filter_situation) { default: BUG("unknown filter_situation: %d", filter_situation); case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ return LOFR_MARK_SEEN | LOFR_DO_SHOW; case LOFS_END_TREE: assert(obj->type == OBJ_TREE); return LOFR_ZERO; case LOFS_BLOB: assert(obj->type == OBJ_BLOB); assert((obj->flags & SEEN) == 0); - if (filter_data->omits) - oidset_insert(filter_data->omits, &obj->oid); + if (ctx->omits) + oidset_insert(ctx->omits, &obj->oid); return LOFR_MARK_SEEN; /* but not LOFR_DO_SHOW (hard omit) */ } } -static void *filter_blobs_none__init( - struct oidset *omitted, +static void filter_blobs_none__init( struct list_objects_filter_options *filter_options, - filter_object_fn *filter_fn, - filter_free_fn *filter_free_fn) + struct filter_context *ctx) { - struct filter_blobs_none_data *d = xcalloc(1, sizeof(*d)); - d->omits = omitted; - - *filter_fn = filter_blobs_none; - *filter_free_fn = free; - return d; + ctx->filter_fn = filter_blobs_none; } -/* - * A filter for list-objects to omit ALL trees and blobs from the traversal. - * Can OPTIONALLY collect a list of the omitted OIDs. - */ +/* A filter for list-objects to omit ALL trees and blobs from the traversal. */ struct filter_trees_depth_data { - struct oidset *omits; - /* * Maps trees to the minimum depth at which they were seen. It is not * necessary to re-traverse a tree at deeper or equal depths than it has * already been traversed. * * We can't use LOFR_MARK_SEEN for tree objects since this will prevent * it from being traversed at shallower depths. */ struct oidmap seen_at_depth; @@ -103,41 +81,41 @@ struct filter_trees_depth_data { }; struct seen_map_entry { struct oidmap_entry base; size_t depth; }; /* Returns 1 if the oid was in the omits set before it was invoked. */ static int filter_trees_update_omits( struct object *obj, - struct filter_trees_depth_data *filter_data, + struct filter_context *ctx, int include_it) { - if (!filter_data->omits) + if (!ctx->omits) return 0; if (include_it) - return oidset_remove(filter_data->omits, &obj->oid); + return oidset_remove(ctx->omits, &obj->oid); else - return oidset_insert(filter_data->omits, &obj->oid); + return oidset_insert(ctx->omits, &obj->oid); } static enum list_objects_filter_result filter_trees_depth( struct repository *r, enum list_objects_filter_situation filter_situation, struct object *obj, const char *pathname, const char *filename, - void *filter_data_) + struct filter_context *ctx) { - struct filter_trees_depth_data *filter_data = filter_data_; + struct filter_trees_depth_data *filter_data = ctx->data; struct seen_map_entry *seen_info; int include_it = filter_data->current_depth < filter_data->exclude_depth; int filter_res; int already_seen; /* * Note that we do not use _MARK_SEEN in order to allow re-traversal in * case we encounter a tree or blob again at a shallower depth. */ @@ -145,47 +123,47 @@ static enum list_objects_filter_result filter_trees_depth( switch (filter_situation) { default: BUG("unknown filter_situation: %d", filter_situation); case LOFS_END_TREE: assert(obj->type == OBJ_TREE); filter_data->current_depth--; return LOFR_ZERO; case LOFS_BLOB: - filter_trees_update_omits(obj, filter_data, include_it); + filter_trees_update_omits(obj, ctx, include_it); return include_it ? LOFR_MARK_SEEN | LOFR_DO_SHOW : LOFR_ZERO; case LOFS_BEGIN_TREE: seen_info = oidmap_get( &filter_data->seen_at_depth, &obj->oid); if (!seen_info) { seen_info = xcalloc(1, sizeof(*seen_info)); oidcpy(&seen_info->base.oid, &obj->oid); seen_info->depth = filter_data->current_depth; oidmap_put(&filter_data->seen_at_depth, seen_info); already_seen = 0; } else { already_seen = filter_data->current_depth >= seen_info->depth; } if (already_seen) { filter_res = LOFR_SKIP_TREE; } else { int been_omitted = filter_trees_update_omits( - obj, filter_data, include_it); + obj, ctx, include_it); seen_info->depth = filter_data->current_depth; if (include_it) filter_res = LOFR_DO_SHOW; - else if (filter_data->omits && !been_omitted) + else if (ctx->omits && !been_omitted) /* * Must update omit information of children * recursively; they have not been omitted yet. */ filter_res = LOFR_ZERO; else filter_res = LOFR_SKIP_TREE; } filter_data->current_depth++; @@ -194,55 +172,48 @@ static enum list_objects_filter_result filter_trees_depth( } static void filter_trees_free(void *filter_data) { struct filter_trees_depth_data *d = filter_data; if (!d) return; oidmap_free(&d->seen_at_depth, 1); free(d); } -static void *filter_trees_depth__init( - struct oidset *omitted, +static void filter_trees_depth__init( struct list_objects_filter_options *filter_options, - filter_object_fn *filter_fn, - filter_free_fn *filter_free_fn) + struct filter_context *ctx) { struct filter_trees_depth_data *d = xcalloc(1, sizeof(*d)); - d->omits = omitted; oidmap_init(&d->seen_at_depth, 0); d->exclude_depth = filter_options->tree_exclude_depth; d->current_depth = 0; - *filter_fn = filter_trees_depth; - *filter_free_fn = filter_trees_free; - return d; + ctx->filter_fn = filter_trees_depth; + ctx->free_fn = filter_trees_free; + ctx->data = d; } -/* - * A filter for list-objects to omit large blobs. - * And to OPTIONALLY collect a list of the omitted OIDs. - */ +/* A filter for list-objects to omit large blobs. */ struct filter_blobs_limit_data { - struct oidset *omits; unsigned long max_bytes; }; static enum list_objects_filter_result filter_blobs_limit( struct repository *r, enum list_objects_filter_situation filter_situation, struct object *obj, const char *pathname, const char *filename, - void *filter_data_) + struct filter_context *ctx) { - struct filter_blobs_limit_data *filter_data = filter_data_; + struct filter_blobs_limit_data *filter_data = ctx->data; unsigned long object_length; enum object_type t; switch (filter_situation) { default: BUG("unknown filter_situation: %d", filter_situation); case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -263,44 +234,41 @@ static enum list_objects_filter_result filter_blobs_limit( * apply the size filter criteria. Be conservative * and force show it (and let the caller deal with * the ambiguity). */ goto include_it; } if (object_length < filter_data->max_bytes) goto include_it; - if (filter_data->omits) - oidset_insert(filter_data->omits, &obj->oid); + if (ctx->omits) + oidset_insert(ctx->omits, &obj->oid); return LOFR_MARK_SEEN; /* but not LOFR_DO_SHOW (hard omit) */ } include_it: - if (filter_data->omits) - oidset_remove(filter_data->omits, &obj->oid); + if (ctx->omits) + oidset_remove(ctx->omits, &obj->oid); return LOFR_MARK_SEEN | LOFR_DO_SHOW; } -static void *filter_blobs_limit__init( - struct oidset *omitted, +static void filter_blobs_limit__init( struct list_objects_filter_options *filter_options, - filter_object_fn *filter_fn, - filter_free_fn *filter_free_fn) + struct filter_context *ctx) { struct filter_blobs_limit_data *d = xcalloc(1, sizeof(*d)); - d->omits = omitted; d->max_bytes = filter_options->blob_limit_value; - *filter_fn = filter_blobs_limit; - *filter_free_fn = free; - return d; + ctx->filter_fn = filter_blobs_limit; + ctx->free_fn = free; + ctx->data = d; } /* * A filter driven by a sparse-checkout specification to only * include blobs that a sparse checkout would populate. * * The sparse-checkout spec can be loaded from a blob with the * given OID or from a local pathname. We allow an OID because * the repo may be bare or we may be doing the filtering on the * server. @@ -319,36 +287,35 @@ struct frame { * omitted objects. * * 0 if everything (recursively) contained in this directory * has been explicitly included (SHOWN) in the result and * the directory may be short-cut later in the traversal. */ unsigned child_prov_omit : 1; }; struct filter_sparse_data { - struct oidset *omits; struct exclude_list el; size_t nr, alloc; struct frame *array_frame; }; static enum list_objects_filter_result filter_sparse( struct repository *r, enum list_objects_filter_situation filter_situation, struct object *obj, const char *pathname, const char *filename, - void *filter_data_) + struct filter_context *ctx) { - struct filter_sparse_data *filter_data = filter_data_; + struct filter_sparse_data *filter_data = ctx->data; int val, dtype; struct frame *frame; switch (filter_situation) { default: BUG("unknown filter_situation: %d", filter_situation); case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); dtype = DT_DIR; @@ -414,128 +381,117 @@ static enum list_objects_filter_result filter_sparse( frame = &filter_data->array_frame[filter_data->nr]; dtype = DT_REG; val = is_excluded_from_list(pathname, strlen(pathname), filename, &dtype, &filter_data->el, r->index); if (val < 0) val = frame->defval; if (val > 0) { - if (filter_data->omits) - oidset_remove(filter_data->omits, &obj->oid); + if (ctx->omits) + oidset_remove(ctx->omits, &obj->oid); return LOFR_MARK_SEEN | LOFR_DO_SHOW; } /* * Provisionally omit it. We've already established that * this pathname is not in the sparse-checkout specification * with the CURRENT pathname, so we *WANT* to omit this blob. * * However, a pathname elsewhere in the tree may also * reference this same blob, so we cannot reject it yet. * Leave the LOFR_ bits unset so that if the blob appears * again in the traversal, we will be asked again. */ - if (filter_data->omits) - oidset_insert(filter_data->omits, &obj->oid); + if (ctx->omits) + oidset_insert(ctx->omits, &obj->oid); /* * Remember that at least 1 blob in this tree was * provisionally omitted. This prevents us from short * cutting the tree in future iterations. */ frame->child_prov_omit = 1; return LOFR_ZERO; } } static void filter_sparse_free(void *filter_data) { struct filter_sparse_data *d = filter_data; /* TODO free contents of 'd' */ free(d); } -static void *filter_sparse_oid__init( - struct oidset *omitted, +static void filter_sparse_oid__init( struct list_objects_filter_options *filter_options, - filter_object_fn *filter_fn, - filter_free_fn *filter_free_fn) + struct filter_context *ctx) { struct filter_sparse_data *d = xcalloc(1, sizeof(*d)); - d->omits = omitted; if (add_excludes_from_blob_to_list(filter_options->sparse_oid_value, NULL, 0, &d->el) < 0) die("could not load filter specification"); ALLOC_GROW(d->array_frame, d->nr + 1, d->alloc); d->array_frame[d->nr].defval = 0; /* default to include */ d->array_frame[d->nr].child_prov_omit = 0; - *filter_fn = filter_sparse; - *filter_free_fn = filter_sparse_free; - return d; + ctx->filter_fn = filter_sparse; + ctx->free_fn = filter_sparse_free; + ctx->data = d; } -static void *filter_sparse_path__init( - struct oidset *omitted, +static void filter_sparse_path__init( struct list_objects_filter_options *filter_options, - filter_object_fn *filter_fn, - filter_free_fn *filter_free_fn) + struct filter_context *ctx) { struct filter_sparse_data *d = xcalloc(1, sizeof(*d)); - d->omits = omitted; if (add_excludes_from_file_to_list(filter_options->sparse_path_value, NULL, 0, &d->el, NULL) < 0) die("could not load filter specification"); ALLOC_GROW(d->array_frame, d->nr + 1, d->alloc); d->array_frame[d->nr].defval = 0; /* default to include */ d->array_frame[d->nr].child_prov_omit = 0; - *filter_fn = filter_sparse; - *filter_free_fn = filter_sparse_free; - return d; + ctx->filter_fn = filter_sparse; + ctx->free_fn = filter_sparse_free; + ctx->data = d; } -typedef void *(*filter_init_fn)( - struct oidset *omitted, +typedef void (*filter_init_fn)( struct list_objects_filter_options *filter_options, - filter_object_fn *filter_fn, - filter_free_fn *filter_free_fn); + struct filter_context *ctx); /* * Must match "enum list_objects_filter_choice". */ static filter_init_fn s_filters[] = { NULL, filter_blobs_none__init, filter_blobs_limit__init, filter_trees_depth__init, filter_sparse_oid__init, filter_sparse_path__init, }; -void *list_objects_filter__init( +void list_objects_filter__init( struct oidset *omitted, struct list_objects_filter_options *filter_options, - filter_object_fn *filter_fn, - filter_free_fn *filter_free_fn) + struct filter_context *ctx) { filter_init_fn init_fn; assert((sizeof(s_filters) / sizeof(s_filters[0])) == LOFC__COUNT); if (filter_options->choice >= LOFC__COUNT) BUG("invalid list-objects filter choice: %d", filter_options->choice); + memset(ctx, 0, sizeof(*ctx)); + ctx->omits = omitted; init_fn = s_filters[filter_options->choice]; if (init_fn) - return init_fn(omitted, filter_options, - filter_fn, filter_free_fn); - *filter_fn = NULL; - *filter_free_fn = NULL; - return NULL; + init_fn(filter_options, ctx); } diff --git a/list-objects-filter.h b/list-objects-filter.h index 1d45a4ad57..ee807f5d9b 100644 --- a/list-objects-filter.h +++ b/list-objects-filter.h @@ -53,37 +53,46 @@ enum list_objects_filter_result { LOFR_DO_SHOW = 1<<1, LOFR_SKIP_TREE = 1<<2, }; enum list_objects_filter_situation { LOFS_BEGIN_TREE, LOFS_END_TREE, LOFS_BLOB }; -typedef enum list_objects_filter_result (*filter_object_fn)( - struct repository *r, - enum list_objects_filter_situation filter_situation, - struct object *obj, - const char *pathname, - const char *filename, - void *filter_data); +struct filter_context { + enum list_objects_filter_result (*filter_fn)( + struct repository *r, + enum list_objects_filter_situation filter_situation, + struct object *obj, + const char *pathname, + const char *filename, + struct filter_context *ctx); + void (*free_fn)(void *filter_data); -typedef void (*filter_free_fn)(void *filter_data); + struct oidset *omits; + void *data; +}; /* * Constructor for the set of defined list-objects filters. * Returns a generic "void *filter_data". * * The returned "filter_fn" will be used by traverse_commit_list() * to filter the results. * * The returned "filter_free_fn" is a destructor for the * filter_data. */ -void *list_objects_filter__init( +void list_objects_filter__init( struct oidset *omitted, struct list_objects_filter_options *filter_options, - filter_object_fn *filter_fn, - filter_free_fn *filter_free_fn); + struct filter_context *ctx); + +static inline void list_objects_filter__release(struct filter_context *ctx) { + if (ctx->data && ctx->free_fn) + ctx->free_fn(ctx->data); + memset(ctx, 0, sizeof(*ctx)); +} #endif /* LIST_OBJECTS_FILTER_H */ diff --git a/list-objects.c b/list-objects.c index b5651ddd5b..7a73f7deee 100644 --- a/list-objects.c +++ b/list-objects.c @@ -11,22 +11,21 @@ #include "list-objects-filter-options.h" #include "packfile.h" #include "object-store.h" #include "trace.h" struct traversal_context { struct rev_info *revs; show_object_fn show_object; show_commit_fn show_commit; void *show_data; - filter_object_fn filter_fn; - void *filter_data; + struct filter_context filter_ctx; }; static void process_blob(struct traversal_context *ctx, struct blob *blob, struct strbuf *path, const char *name) { struct object *obj = &blob->object; size_t pathlen; enum list_objects_filter_result r = LOFR_MARK_SEEN | LOFR_DO_SHOW; @@ -47,25 +46,25 @@ static void process_blob(struct traversal_context *ctx, * may cause the actual filter to report an incomplete list * of missing objects. */ if (ctx->revs->exclude_promisor_objects && !has_object_file(&obj->oid) && is_promisor_object(&obj->oid)) return; pathlen = path->len; strbuf_addstr(path, name); - if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_fn) - r = ctx->filter_fn(ctx->revs->repo, - LOFS_BLOB, obj, - path->buf, &path->buf[pathlen], - ctx->filter_data); + if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_ctx.filter_fn) + r = ctx->filter_ctx.filter_fn(ctx->revs->repo, + LOFS_BLOB, obj, + path->buf, &path->buf[pathlen], + &ctx->filter_ctx); if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) ctx->show_object(obj, path->buf, ctx->show_data); strbuf_setlen(path, pathlen); } /* * Processing a gitlink entry currently does nothing, since * we do not recurse into the subproject. @@ -179,42 +178,42 @@ static void process_tree(struct traversal_context *ctx, */ if (revs->exclude_promisor_objects && is_promisor_object(&obj->oid)) return; if (!revs->do_not_die_on_missing_tree) die("bad tree object %s", oid_to_hex(&obj->oid)); } strbuf_addstr(base, name); - if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_fn) - r = ctx->filter_fn(ctx->revs->repo, - LOFS_BEGIN_TREE, obj, - base->buf, &base->buf[baselen], - ctx->filter_data); + if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_ctx.filter_fn) + r = ctx->filter_ctx.filter_fn(ctx->revs->repo, + LOFS_BEGIN_TREE, obj, + base->buf, &base->buf[baselen], + &ctx->filter_ctx); if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) ctx->show_object(obj, base->buf, ctx->show_data); if (base->len) strbuf_addch(base, '/'); if (r & LOFR_SKIP_TREE) trace_printf("Skipping contents of tree %s...\n", base->buf); else if (!failed_parse) process_tree_contents(ctx, tree, base); - if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_fn) { - r = ctx->filter_fn(ctx->revs->repo, - LOFS_END_TREE, obj, - base->buf, &base->buf[baselen], - ctx->filter_data); + if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_ctx.filter_fn) { + r = ctx->filter_ctx.filter_fn(ctx->revs->repo, + LOFS_END_TREE, obj, + base->buf, &base->buf[baselen], + &ctx->filter_ctx); if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) ctx->show_object(obj, base->buf, ctx->show_data); } strbuf_setlen(base, baselen); free_tree_buffer(tree); } @@ -395,38 +394,34 @@ static void do_traverse(struct traversal_context *ctx) void traverse_commit_list(struct rev_info *revs, show_commit_fn show_commit, show_object_fn show_object, void *show_data) { struct traversal_context ctx; ctx.revs = revs; ctx.show_commit = show_commit; ctx.show_object = show_object; ctx.show_data = show_data; - ctx.filter_fn = NULL; - ctx.filter_data = NULL; + memset(&ctx.filter_ctx, 0, sizeof(ctx.filter_ctx)); do_traverse(&ctx); } void traverse_commit_list_filtered( struct list_objects_filter_options *filter_options, struct rev_info *revs, show_commit_fn show_commit, show_object_fn show_object, void *show_data, struct oidset *omitted) { struct traversal_context ctx; - filter_free_fn filter_free_fn = NULL; + memset(&ctx, 0, sizeof(ctx)); ctx.revs = revs; ctx.show_object = show_object; ctx.show_commit = show_commit; ctx.show_data = show_data; - ctx.filter_fn = NULL; - ctx.filter_data = list_objects_filter__init(omitted, filter_options, - &ctx.filter_fn, &filter_free_fn); + list_objects_filter__init(omitted, filter_options, &ctx.filter_ctx); do_traverse(&ctx); - if (ctx.filter_data && filter_free_fn) - filter_free_fn(ctx.filter_data); + list_objects_filter__release(&ctx.filter_ctx); } From patchwork Wed May 22 00:21:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10954599 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 34E1F924 for ; Wed, 22 May 2019 00:22:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25A4D28ADE for ; Wed, 22 May 2019 00:22:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19AFB28AE6; Wed, 22 May 2019 00:22:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C797628ADE for ; Wed, 22 May 2019 00:22:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727424AbfEVAWW (ORCPT ); Tue, 21 May 2019 20:22:22 -0400 Received: from mail-qt1-f201.google.com ([209.85.160.201]:44410 "EHLO mail-qt1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725797AbfEVAWV (ORCPT ); Tue, 21 May 2019 20:22:21 -0400 Received: by mail-qt1-f201.google.com with SMTP id t51so398013qtb.11 for ; Tue, 21 May 2019 17:22:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Le9FMCTJI6pXCdFlqVPhGri1nKiAp54cd08micw8HzI=; b=Pr2GWo8zDhW1iw5Xoj9Eis6e/qX8dUf0PMQct4nVcZPPh80X1J4hnfJWTd7e8V5XL5 WhabebwvZyFoRYXRsVReE06fGlKPrIMfPEe+6qt76jsHXUog8zxBcwway6Jj/IfTwauZ OFwX0vtc1H270Ka/98G1+N0wje9uz+mHvrzMrsQhangCto4dImuABlgcE7PBZRSwXVZc 9gmVEktS1PWYLmithKriZC2whyEjor9I8Bp0dzix4ubhES+7dqKhGOtNDEXCchIeXYbT ZFeENBSI5/FENtd1xPc5tOVgbECOZeRvaD8UbZffdIqeBRiWk5hIvquyy9lX2oTNXpyR cGrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Le9FMCTJI6pXCdFlqVPhGri1nKiAp54cd08micw8HzI=; b=ZSLFi76rjpA44Xaj7WUNRBXK5Yx1fM3SSzFRecdUhbJpnupOtqRNtLYOqr+ODRqnHZ Ov45/lYZmzPT7kKH0+ArK9O+Rv9HZy6l1DyDgHPXtuhQW8pdKdJ9ii3IjRNMdWOl5vU4 Ab5aEYlYMqJ/9QSPdN45qJUk+z+Nai9rnGxSHKiihEWKMpXjfGWfOWQ5qEhENJ9VIB+K QImIGeevwixqlRcuXTwQUCCZi87vAaczEZSHIK63RG+KUayBHOxUTMzrltWiEFOhfMJk e+V6hipqEVCe0Vqqg4Cz86lQWX+ZEk875smkmJ2gwwuH6Jkw811SX+y5ZPAu2ZCdKL5L R65w== X-Gm-Message-State: APjAAAVXIUV1ZOljsHqz/hNrNByYCpaZxVhg8uGLhdyhHosN4EZx8lqd cMzAwUjJm3ZeI+lhEg2WdKK0A2KgVio= X-Google-Smtp-Source: APXvYqxIhYykKB4R2pEw306eTKPrbdlc9cjkJ8pFjIGkQw1twOhru6BdkRDQJQ49OtGC2sFactVElV6whphH X-Received: by 2002:ac8:7585:: with SMTP id s5mr43854032qtq.38.1558484540879; Tue, 21 May 2019 17:22:20 -0700 (PDT) Date: Tue, 21 May 2019 17:21:51 -0700 In-Reply-To: Message-Id: <6f4da02d494323e3ca946b4b20bf78d9dee419e4.1558484115.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH v1 2/5] list-objects-filter-options: error is localizeable From: Matthew DeVore To: jonathantanmy@google.com, jrn@google.com, git@vger.kernel.org, dstolee@microsoft.com, jeffhost@microsoft.com, jrnieder@gmail.com, pclouds@gmail.com Cc: Matthew DeVore , matvore@comcast.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The "invalid filter-spec" message is user-facing and not a BUG, so make it localizeable. Signed-off-by: Matthew DeVore --- list-objects-filter-options.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index c0036f7378..e46ea467bc 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -81,21 +81,21 @@ static int gently_parse_list_objects_filter( filter_options->choice = LOFC_SPARSE_PATH; filter_options->sparse_path_value = strdup(v0); return 0; } /* * Please update _git_fetch() in git-completion.bash when you * add new filters */ if (errbuf) - strbuf_addf(errbuf, "invalid filter-spec '%s'", arg); + strbuf_addf(errbuf, _("invalid filter-spec '%s'"), arg); memset(filter_options, 0, sizeof(*filter_options)); return 1; } int parse_list_objects_filter(struct list_objects_filter_options *filter_options, const char *arg) { struct strbuf buf = STRBUF_INIT; if (gently_parse_list_objects_filter(filter_options, arg, &buf)) From patchwork Wed May 22 00:21:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10954601 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 45C2F924 for ; Wed, 22 May 2019 00:22:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3537A28AE3 for ; Wed, 22 May 2019 00:22:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 295B228AE8; Wed, 22 May 2019 00:22:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 92C7928AE3 for ; Wed, 22 May 2019 00:22:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727733AbfEVAWZ (ORCPT ); Tue, 21 May 2019 20:22:25 -0400 Received: from mail-ua1-f73.google.com ([209.85.222.73]:36938 "EHLO mail-ua1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725797AbfEVAWZ (ORCPT ); Tue, 21 May 2019 20:22:25 -0400 Received: by mail-ua1-f73.google.com with SMTP id u3so208023uao.4 for ; Tue, 21 May 2019 17:22:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=EsO5XkfhzoPUcBOTkKy74yxpewfeto2+jgocSyR5Ni8=; b=V/CRaYp7dCGfLlJN7qD1A3J1LAND3S15bTis2XFsMqfaZkLNe+ME/qFcW20ipynZDi RYKiBzsvBOtwWq7m0GLyK5kyegHVEzQyu8DQX2OtkxJe7LadfIyzaOU5QxTFip5tA1KI 0JvogqAiyCOPcQtHo2HoBkzIpeS8V+6y8uMsu1gAYVsv7FHLQD9iQEvkdB0zYZlVBzwR oBPZ9niyUGd7fNnNLBHmcQ638E6q0u1wANMcsesYz9JeLQc0Z+PepUS3rdxMizq5O0Ln EMN4GcFR9NcJw4H8EqaOzVLSQNrFwuRJUHIHGUvrPa2t6GYECNi0TTDmvSxIAG0Y/smG vO2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=EsO5XkfhzoPUcBOTkKy74yxpewfeto2+jgocSyR5Ni8=; b=WCAuTXcJ/chciLc/22apcPb/oAAYZZz229i/YqCDynUv4ug5fufkmdXCfymSYy5RtB 96RtET/Sp9hd9IWtvlBdxlkcXdm5M1JNGQyY6KYTn9+qWYu0NYGIUhyuzqdYDWbRiCkB 8YsSJCSBqOti7/PfHzj2+UDb0otheo5D3CC6212hyk2/I4NMzOAzS44f9VS2XvBGS81j CpWT1RAm1l3rVxwWEMnNePOjvCWAvYlDeWk+bl3f/mYZZNE0yTmic4aZIX1BV3LfiKFY sSJmFcc5WTnKPN0ZG15dJhXcat2KfdI1nwK7IcnOWfBJipABVNW4kKQY8raAgq5jlUOY uf8A== X-Gm-Message-State: APjAAAWcEtGoP9hA4QkYKDmPsj0st7nw/hIvwWsxa/yPlyb4Ohh6ZazX 4TbccGCmiZCG8UGbXOZ2ZKVTVx6rj7U= X-Google-Smtp-Source: APXvYqx49S52FkWGFm1jRulZ0+zkYEYtketvlp5QxZzGXLDb833AaQhlonTnysuAQIxNZIH1g6+PSWy9WpTV X-Received: by 2002:a67:e918:: with SMTP id c24mr42362vso.138.1558484543550; Tue, 21 May 2019 17:22:23 -0700 (PDT) Date: Tue, 21 May 2019 17:21:52 -0700 In-Reply-To: Message-Id: <1f95597eedc4c651868601c0ff7c4a4d97ca4457.1558484115.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH v1 3/5] list-objects-filter: implement composite filters From: Matthew DeVore To: jonathantanmy@google.com, jrn@google.com, git@vger.kernel.org, dstolee@microsoft.com, jeffhost@microsoft.com, jrnieder@gmail.com, pclouds@gmail.com Cc: Matthew DeVore , matvore@comcast.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow combining filters such that only objects accepted by all filters are shown. The motivation for this is to allow getting directory listings without also fetching blobs. This can be done by combining blob:none with tree:. There are massive repositories that have larger-than-expected trees - even if you include only a single commit. The current usage requires passing the filter to rev-list, or sending it over the wire, as: combine:+ (i.e.: git rev-list --filter=combine:tree:2+blob:limit=32k). This is potentially awkward because individual filters must be URL-encoded if they contain + or %. This can potentially be improved by supporting a repeated flag syntax, e.g.: $ git rev-list --filter=tree:2 --filter=blob:limit=32k Such usage is currently an error, so giving it a meaning is backwards- compatible. Signed-off-by: Matthew DeVore --- Documentation/rev-list-options.txt | 12 ++ contrib/completion/git-completion.bash | 2 +- list-objects-filter-options.c | 161 ++++++++++++++++++++++++- list-objects-filter-options.h | 14 ++- list-objects-filter.c | 114 +++++++++++++++++ t/t6112-rev-list-filters-objects.sh | 159 +++++++++++++++++++++++- 6 files changed, 455 insertions(+), 7 deletions(-) diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index ddbc1de43f..4fb0c4fbb0 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -730,20 +730,32 @@ specification contained in . + The form '--filter=tree:' omits all blobs and trees whose depth from the root tree is >= (minimum depth if an object is located at multiple depths in the commits traversed). =0 will not include any trees or blobs unless included explicitly in the command-line (or standard input when --stdin is used). =1 will include only the tree and blobs which are referenced directly by a commit reachable from or an explicitly-given object. =2 is like =1 while also including trees and blobs one more level removed from an explicitly-given commit or tree. ++ +The form '--filter=combine:++...' combines +several filters. Only objects which are accepted by every filter are +included. Filters are joined by '{plus}' and individual filters are %-encoded +(i.e. URL-encoded). Besides the '{plus}' and '%' characters, the following +characters are reserved and also must be encoded: +`~!@#$^&*()[]{}\;",<>?`+'`+ as well as all characters with ASCII code +<= `0x20`, which includes space and newline. ++ +Other arbitrary characters can also be encoded. For instance, +'combine:tree:3+blob:none' and 'combine:tree%3A2+blob%3Anone' are +equivalent. --no-filter:: Turn off any previous `--filter=` argument. --filter-print-omitted:: Only useful with `--filter=`; prints a list of the objects omitted by the filter. Object IDs are prefixed with a ``~'' character. --missing=:: A debug option to help with future "partial clone" development. diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash index 3eefbabdb1..0fd0a10d0c 100644 --- a/contrib/completion/git-completion.bash +++ b/contrib/completion/git-completion.bash @@ -1529,21 +1529,21 @@ _git_difftool () __git_fetch_recurse_submodules="yes on-demand no" _git_fetch () { case "$cur" in --recurse-submodules=*) __gitcomp "$__git_fetch_recurse_submodules" "" "${cur##--recurse-submodules=}" return ;; --filter=*) - __gitcomp "blob:none blob:limit= sparse:oid= sparse:path=" "" "${cur##--filter=}" + __gitcomp "blob:none blob:limit= sparse:oid= sparse:path= combine: tree:" "" "${cur##--filter=}" return ;; --*) __gitcomp_builtin fetch return ;; esac __git_complete_remote_or_refspec } diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index e46ea467bc..d7a1516188 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -1,19 +1,24 @@ #include "cache.h" #include "commit.h" #include "config.h" #include "revision.h" #include "argv-array.h" #include "list-objects.h" #include "list-objects-filter.h" #include "list-objects-filter-options.h" +static int parse_combine_filter( + struct list_objects_filter_options *filter_options, + const char *arg, + struct strbuf *errbuf); + /* * Parse value of the argument to the "filter" keyword. * On the command line this looks like: * --filter= * and in the pack protocol as: * "filter" SP * * The filter keyword will be used by many commands. * See Documentation/rev-list-options.txt for allowed values for . * @@ -31,22 +36,20 @@ static int gently_parse_list_objects_filter( if (filter_options->choice) { if (errbuf) { strbuf_addstr( errbuf, _("multiple filter-specs cannot be combined")); } return 1; } - filter_options->filter_spec = strdup(arg); - if (!strcmp(arg, "blob:none")) { filter_options->choice = LOFC_BLOB_NONE; return 0; } else if (skip_prefix(arg, "blob:limit=", &v0)) { if (git_parse_ulong(v0, &filter_options->blob_limit_value)) { filter_options->choice = LOFC_BLOB_LIMIT; return 0; } @@ -74,37 +77,183 @@ static int gently_parse_list_objects_filter( if (!get_oid_with_context(the_repository, v0, GET_OID_BLOB, &sparse_oid, &oc)) filter_options->sparse_oid_value = oiddup(&sparse_oid); filter_options->choice = LOFC_SPARSE_OID; return 0; } else if (skip_prefix(arg, "sparse:path=", &v0)) { filter_options->choice = LOFC_SPARSE_PATH; filter_options->sparse_path_value = strdup(v0); return 0; + + } else if (skip_prefix(arg, "combine:", &v0)) { + int sub_parse_res = parse_combine_filter( + filter_options, v0, errbuf); + if (sub_parse_res) + return sub_parse_res; + return 0; + } /* * Please update _git_fetch() in git-completion.bash when you * add new filters */ if (errbuf) strbuf_addf(errbuf, _("invalid filter-spec '%s'"), arg); memset(filter_options, 0, sizeof(*filter_options)); return 1; } +static int digit_value(int c, struct strbuf *errbuf) { + if (c >= '0' && c <= '9') + return c - '0'; + if (c >= 'a' && c <= 'f') + return c - 'a' + 10; + if (c >= 'A' && c <= 'F') + return c - 'A' + 10; + + if (!errbuf) + return -1; + + strbuf_addf(errbuf, _("error in filter-spec - ")); + if (c) + strbuf_addf( + errbuf, + _("expect two hex digits after %%, but got: '%c'"), + c); + else + strbuf_addf( + errbuf, + _("not enough hex digits after %%; expected two")); + + return -1; +} + +static int url_decode(struct strbuf *s, struct strbuf *errbuf) { + char *dest = s->buf; + char *src = s->buf; + size_t new_len; + + while (*src) { + int digit_value_0, digit_value_1; + + if (src[0] != '%') { + *dest++ = *src++; + continue; + } + src++; + + digit_value_0 = digit_value(*src++, errbuf); + if (digit_value_0 < 0) + return 1; + digit_value_1 = digit_value(*src++, errbuf); + if (digit_value_1 < 0) + return 1; + *dest++ = digit_value_0 * 16 + digit_value_1; + } + new_len = dest - s->buf; + strbuf_remove(s, new_len, s->len - new_len); + + return 0; +} + +static const char *RESERVED_NON_WS = "~`!@#$^&*()[]{}\\;'\",<>?"; + +static int has_reserved_character( + struct strbuf *sub_spec, struct strbuf *errbuf) +{ + const char *c = sub_spec->buf; + while (*c) { + if (*c <= ' ' || strchr(RESERVED_NON_WS, *c)) + goto found_reserved; + c++; + } + + return 0; + +found_reserved: + if (errbuf) + strbuf_addf(errbuf, + "must escape char in sub-filter-spec: '%c'", + *c); + return 1; +} + +static int parse_combine_filter( + struct list_objects_filter_options *filter_options, + const char *arg, + struct strbuf *errbuf) +{ + struct strbuf **sub_specs = strbuf_split_str(arg, '+', 2); + int result; + + if (!sub_specs[0]) { + if (errbuf) + strbuf_addf(errbuf, + _("expected something after combine:")); + result = 1; + goto cleanup; + } + + result = has_reserved_character(sub_specs[0], errbuf); + if (result) + goto cleanup; + + /* + * Only decode the first sub-filter, since the rest will be decoded on + * the recursive call. + */ + result = url_decode(sub_specs[0], errbuf); + if (result) + goto cleanup; + + if (!sub_specs[1]) { + /* + * There is only one sub-filter, so we don't need the + * combine: - just parse it as a non-composite filter. + */ + result = gently_parse_list_objects_filter( + filter_options, sub_specs[0]->buf, errbuf); + goto cleanup; + } + + /* Remove trailing "+" so we can parse it. */ + assert(sub_specs[0]->buf[sub_specs[0]->len - 1] == '+'); + strbuf_remove(sub_specs[0], sub_specs[0]->len - 1, 1); + + filter_options->choice = LOFC_COMBINE; + filter_options->lhs = xcalloc(1, sizeof(*filter_options->lhs)); + filter_options->rhs = xcalloc(1, sizeof(*filter_options->rhs)); + + result = gently_parse_list_objects_filter(filter_options->lhs, + sub_specs[0]->buf, + errbuf) || + parse_combine_filter(filter_options->rhs, + sub_specs[1]->buf, + errbuf); + +cleanup: + strbuf_list_free(sub_specs); + if (result) { + list_objects_filter_release(filter_options); + memset(filter_options, 0, sizeof(*filter_options)); + } + return result; +} + int parse_list_objects_filter(struct list_objects_filter_options *filter_options, const char *arg) { struct strbuf buf = STRBUF_INIT; + filter_options->filter_spec = strdup(arg); if (gently_parse_list_objects_filter(filter_options, arg, &buf)) die("%s", buf.buf); return 0; } int opt_parse_list_objects_filter(const struct option *opt, const char *arg, int unset) { struct list_objects_filter_options *filter_options = opt->value; @@ -127,23 +276,29 @@ void expand_list_objects_filter_spec( else if (filter->choice == LOFC_TREE_DEPTH) strbuf_addf(expanded_spec, "tree:%lu", filter->tree_exclude_depth); else strbuf_addstr(expanded_spec, filter->filter_spec); } void list_objects_filter_release( struct list_objects_filter_options *filter_options) { + if (!filter_options) + return; free(filter_options->filter_spec); free(filter_options->sparse_oid_value); free(filter_options->sparse_path_value); + list_objects_filter_release(filter_options->lhs); + free(filter_options->lhs); + list_objects_filter_release(filter_options->rhs); + free(filter_options->rhs); memset(filter_options, 0, sizeof(*filter_options)); } void partial_clone_register( const char *remote, const struct list_objects_filter_options *filter_options) { /* * Record the name of the partial clone remote in the * config and in the global variable -- the latter is @@ -171,14 +326,16 @@ void partial_clone_register( } void partial_clone_get_default_filter_spec( struct list_objects_filter_options *filter_options) { /* * Parse default value, but silently ignore it if it is invalid. */ if (!core_partial_clone_filter_default) return; + + filter_options->filter_spec = strdup(core_partial_clone_filter_default); gently_parse_list_objects_filter(filter_options, core_partial_clone_filter_default, NULL); } diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index e3adc78ebf..6c0f0ecd08 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -7,20 +7,21 @@ /* * The list of defined filters for list-objects. */ enum list_objects_filter_choice { LOFC_DISABLED = 0, LOFC_BLOB_NONE, LOFC_BLOB_LIMIT, LOFC_TREE_DEPTH, LOFC_SPARSE_OID, LOFC_SPARSE_PATH, + LOFC_COMBINE, LOFC__COUNT /* must be last */ }; struct list_objects_filter_options { /* * 'filter_spec' is the raw argument value given on the command line * or protocol request. (The part after the "--keyword=".) For * commands that launch filtering sub-processes, or for communication * over the network, don't use this value; use the result of * expand_list_objects_filter_spec() instead. @@ -32,28 +33,35 @@ struct list_objects_filter_options { * the filtering algorithm to use. */ enum list_objects_filter_choice choice; /* * Choice is LOFC_DISABLED because "--no-filter" was requested. */ unsigned int no_filter : 1; /* - * Parsed values (fields) from within the filter-spec. These are - * choice-specific; not all values will be defined for any given - * choice. + * BEGIN choice-specific parsed values from within the filter-spec. Only + * some values will be defined for any given choice. */ + struct object_id *sparse_oid_value; char *sparse_path_value; unsigned long blob_limit_value; unsigned long tree_exclude_depth; + + /* LOFC_COMBINE values */ + struct list_objects_filter_options *lhs, *rhs; + + /* + * END choice-specific parsed values. + */ }; /* Normalized command line arguments */ #define CL_ARG__FILTER "filter" int parse_list_objects_filter( struct list_objects_filter_options *filter_options, const char *arg); int opt_parse_list_objects_filter(const struct option *opt, diff --git a/list-objects-filter.c b/list-objects-filter.c index 8e8616b9b8..b97277a46f 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -453,34 +453,148 @@ static void filter_sparse_path__init( ALLOC_GROW(d->array_frame, d->nr + 1, d->alloc); d->array_frame[d->nr].defval = 0; /* default to include */ d->array_frame[d->nr].child_prov_omit = 0; ctx->filter_fn = filter_sparse; ctx->free_fn = filter_sparse_free; ctx->data = d; } +struct filter_combine_data { + /* sub[0] corresponds to lhs, sub[1] to rhs. */ + struct { + struct filter_context ctx; + struct oidset seen; + struct object_id skip_tree; + unsigned is_skipping_tree : 1; + } sub[2]; + + struct oidset rhs_omits; +}; + +static void add_all(struct oidset *dest, struct oidset *src) { + struct oidset_iter iter; + struct object_id *src_oid; + + oidset_iter_init(src, &iter); + while ((src_oid = oidset_iter_next(&iter)) != NULL) + oidset_insert(dest, src_oid); +} + +static void filter_combine_free(void *filter_data) +{ + struct filter_combine_data *d = filter_data; + int i; + + /* Anything omitted by rhs should be added to the overall omits set. */ + if (d->sub[0].ctx.omits) + add_all(d->sub[0].ctx.omits, d->sub[1].ctx.omits); + + for (i = 0; i < 2; i++) { + list_objects_filter__release(&d->sub[i].ctx); + oidset_clear(&d->sub[i].seen); + } + oidset_clear(&d->rhs_omits); + free(d); +} + +static int should_delegate(enum list_objects_filter_situation filter_situation, + struct object *obj, + struct filter_combine_data *d, + int side) +{ + if (!d->sub[side].is_skipping_tree) + return 1; + if (filter_situation == LOFS_END_TREE && + oideq(&obj->oid, &d->sub[side].skip_tree)) { + d->sub[side].is_skipping_tree = 0; + return 1; + } + return 0; +} + +static enum list_objects_filter_result filter_combine( + struct repository *r, + enum list_objects_filter_situation filter_situation, + struct object *obj, + const char *pathname, + const char *filename, + struct filter_context *ctx) +{ + struct filter_combine_data *d = ctx->data; + enum list_objects_filter_result result[2]; + enum list_objects_filter_result combined_result = LOFR_ZERO; + int i; + + for (i = 0; i < 2; i++) { + if (oidset_contains(&d->sub[i].seen, &obj->oid) || + !should_delegate(filter_situation, obj, d, i)) { + result[i] = LOFR_ZERO; + continue; + } + + result[i] = d->sub[i].ctx.filter_fn( + r, filter_situation, obj, pathname, filename, + &d->sub[i].ctx); + + if (result[i] & LOFR_MARK_SEEN) + oidset_insert(&d->sub[i].seen, &obj->oid); + + if (result[i] & LOFR_SKIP_TREE) { + d->sub[i].is_skipping_tree = 1; + d->sub[i].skip_tree = obj->oid; + } + } + + if ((result[0] & LOFR_DO_SHOW) && (result[1] & LOFR_DO_SHOW)) + combined_result |= LOFR_DO_SHOW; + if (d->sub[0].is_skipping_tree && d->sub[1].is_skipping_tree) + combined_result |= LOFR_SKIP_TREE; + + return combined_result; +} + +static void filter_combine__init( + struct list_objects_filter_options *filter_options, + struct filter_context *ctx) +{ + struct filter_combine_data *d = xcalloc(1, sizeof(*d)); + + if (ctx->omits) + oidset_init(&d->rhs_omits, 16); + + list_objects_filter__init(ctx->omits, filter_options->lhs, + &d->sub[0].ctx); + list_objects_filter__init(&d->rhs_omits, filter_options->rhs, + &d->sub[1].ctx); + + ctx->filter_fn = filter_combine; + ctx->free_fn = filter_combine_free; + ctx->data = d; +} + typedef void (*filter_init_fn)( struct list_objects_filter_options *filter_options, struct filter_context *ctx); /* * Must match "enum list_objects_filter_choice". */ static filter_init_fn s_filters[] = { NULL, filter_blobs_none__init, filter_blobs_limit__init, filter_trees_depth__init, filter_sparse_oid__init, filter_sparse_path__init, + filter_combine__init, }; void list_objects_filter__init( struct oidset *omitted, struct list_objects_filter_options *filter_options, struct filter_context *ctx) { filter_init_fn init_fn; assert((sizeof(s_filters) / sizeof(s_filters[0])) == LOFC__COUNT); diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index 9c11427719..ddfacb1a1a 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -284,21 +284,33 @@ test_expect_success 'verify tree:0 includes trees in "filtered" output' ' # Make sure tree:0 does not iterate through any trees. test_expect_success 'verify skipping tree iteration when not collecting omits' ' GIT_TRACE=1 git -C r3 rev-list \ --objects --filter=tree:0 HEAD 2>filter_trace && grep "Skipping contents of tree [.][.][.]" filter_trace >actual && # One line for each commit traversed. test_line_count = 2 actual && # Make sure no other trees were considered besides the root. - ! grep "Skipping contents of tree [^.]" filter_trace + ! grep "Skipping contents of tree [^.]" filter_trace && + + # Try this again with "combine:". If both sub-filters are skipping + # trees, the composite filter should also skip trees. This is not + # important unless the user does combine:tree:X+tree:Y or another filter + # besides "tree:" is implemented in the future which can skip trees. + GIT_TRACE=1 git -C r3 rev-list \ + --objects --filter=combine:tree:1+tree:3 HEAD 2>filter_trace && + + # Only skip the dir1/ tree, which is shared between the two commits. + grep "Skipping contents of tree " filter_trace >actual && + test_write_lines "Skipping contents of tree dir1/..." >expected && + test_cmp expected actual ' # Test tree:# filters. expect_has () { commit=$1 && name=$2 && hash=$(git -C r3 rev-parse $commit:$name) && grep "^$hash $name$" actual @@ -336,20 +348,134 @@ test_expect_success 'verify tree:3 includes everything expected' ' expect_has HEAD dir1/sparse1 && expect_has HEAD dir1/sparse2 && expect_has HEAD pattern && expect_has HEAD sparse1 && expect_has HEAD sparse2 && # There are also 2 commit objects test_line_count = 10 actual ' +test_expect_success 'combine:... for a simple combination' ' + git -C r3 rev-list --objects --filter=combine:tree:2+blob:none HEAD \ + >actual && + + expect_has HEAD "" && + expect_has HEAD~1 "" && + expect_has HEAD dir1 && + + # There are also 2 commit objects + test_line_count = 5 actual +' + +test_expect_success 'combine:... with URL encoding' ' + git -C r3 rev-list --objects \ + --filter=combine:tree%3a2+blob:%6Eon%65 HEAD >actual && + + expect_has HEAD "" && + expect_has HEAD~1 "" && + expect_has HEAD dir1 && + + # There are also 2 commit objects + test_line_count = 5 actual +' + +expect_invalid_filter_spec () { + spec="$1" && + err="$2" && + + test_must_fail git -C r3 rev-list --objects --filter="$spec" HEAD \ + >actual 2>actual_stderr && + test_must_be_empty actual && + test_i18ngrep "$err" actual_stderr +} + +test_expect_success 'combine:... while URL-encoding things that should not be' ' + expect_invalid_filter_spec combine%3Atree:2+blob:none \ + "invalid filter-spec" +' + +test_expect_success 'combine: with nothing after the :' ' + expect_invalid_filter_spec combine: "expected something after combine:" +' + +test_expect_success 'parse error in first sub-filter in combine:' ' + expect_invalid_filter_spec combine:tree:asdf+blob:none \ + "expected .tree:." +' + +test_expect_success 'combine:... with invalid URL-encoded sequences' ' + expect_invalid_filter_spec combine:tree:2+blob:non%a \ + "error in filter-spec - not enough hex digits after %" && + # Edge cases for non-hex chars: "Gg/:" + expect_invalid_filter_spec combine:tree:2+blob%G5none \ + "error in filter-spec - expect two hex digits .*: .G." && + expect_invalid_filter_spec combine:tree:2+blob%g5none \ + "error in filter-spec - expect two hex digits .*: .g." && + expect_invalid_filter_spec combine:tree:2+blob%5/none \ + "error in filter-spec - expect two hex digits .*: ./." && + expect_invalid_filter_spec combine:%:5tree:2+blob:none \ + "error in filter-spec - expect two hex digits .*: .:." +' + +test_expect_success 'combine:... with non-encoded reserved chars' ' + expect_invalid_filter_spec combine:tree:2+sparse:@xyz \ + "must escape char in sub-filter-spec: .@." && + expect_invalid_filter_spec combine:tree:2+sparse:\` \ + "must escape char in sub-filter-spec: .\`." && + expect_invalid_filter_spec combine:tree:2+sparse:~abc \ + "must escape char in sub-filter-spec: .\~." +' + +test_expect_success 'validate err msg for "combine:+"' ' + expect_invalid_filter_spec combine:tree:2+ "expected .tree:." +' + +test_expect_success 'combine:... with edge-case hex digits: Ff Aa 0 9' ' + git -C r3 rev-list --objects --filter="combine:tree:2+bl%6Fb:n%6fne" \ + HEAD >actual && + test_line_count = 5 actual && + git -C r3 rev-list --objects --filter="combine:tree%3A2+blob%3anone" \ + HEAD >actual && + test_line_count = 5 actual && + git -C r3 rev-list --objects --filter="combine:tree:%30" HEAD >actual && + test_line_count = 2 actual && + git -C r3 rev-list --objects --filter="combine:tree:%39+blob:none" \ + HEAD >actual && + test_line_count = 5 actual +' + +test_expect_success 'combine:... with more than two sub-filters' ' + git -C r3 rev-list --objects \ + --filter=combine:tree:3+blob:limit=40+sparse:path=../pattern1 \ + HEAD >actual && + + expect_has HEAD "" && + expect_has HEAD~1 "" && + expect_has HEAD dir1 && + expect_has HEAD dir1/sparse1 && + expect_has HEAD dir1/sparse2 && + + # Should also have 2 commits + test_line_count = 7 actual && + + # Try again, this time making sure the last sub-filter is only + # URL-decoded once. + cp pattern1 pattern1+renamed% && + cp actual expect && + + git -C r3 rev-list --objects \ + --filter=combine:tree:3+blob:limit=40+sparse:path=../pattern1%2brenamed%25 \ + HEAD >actual && + test_cmp expect actual +' + # Test provisional omit collection logic with a repo that has objects appearing # at multiple depths - first deeper than the filter's threshold, then shallow. test_expect_success 'setup r4' ' git init r4 && echo foo > r4/foo && mkdir r4/subdir && echo bar > r4/subdir/bar && @@ -379,20 +505,51 @@ test_expect_success 'test tree:# filter provisional omit for blob and tree' ' test_expect_success 'verify skipping tree iteration when collecting omits' ' GIT_TRACE=1 git -C r4 rev-list --filter-print-omitted \ --objects --filter=tree:0 HEAD 2>filter_trace && grep "^Skipping contents of tree " filter_trace >actual && echo "Skipping contents of tree subdir/..." >expect && test_cmp expect actual ' +test_expect_success 'setup r5' ' + git init r5 && + mkdir -p r5/subdir && + + echo 1 >r5/short-root && + echo 12345 >r5/long-root && + echo a >r5/subdir/short-subdir && + echo abcde >r5/subdir/long-subdir && + + git -C r5 add short-root long-root subdir && + git -C r5 commit -m "commit msg" +' + +test_expect_success 'verify collecting omits in combined: filter' ' + # Note that this test guards against the naive implementation of simply + # giving both filters the same "omits" set and expecting it to + # automatically merge them. + git -C r5 rev-list --objects --quiet --filter-print-omitted \ + --filter=combine:tree:2+blob:limit=3 HEAD >actual && + + # Expect 0 trees/commits, 3 blobs omitted (all blobs except short-root) + omitted_1=$(echo 12345 | git hash-object --stdin) && + omitted_2=$(echo a | git hash-object --stdin) && + omitted_3=$(echo abcde | git hash-object --stdin) && + + grep ~$omitted_1 actual && + grep ~$omitted_2 actual && + grep ~$omitted_3 actual && + test_line_count = 3 actual +' + # Test tree: where a tree is iterated to twice - once where a subentry is # too deep to be included, and again where the blob inside it is shallow enough # to be included. This makes sure we don't use LOFR_MARK_SEEN incorrectly (we # can't use it because a tree can be iterated over again at a lower depth). test_expect_success 'tree: where we iterate over tree at two levels' ' git init r5 && mkdir -p r5/a/subdir/b && echo foo > r5/a/subdir/b/foo && From patchwork Wed May 22 00:21:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10954603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B112A13AD for ; Wed, 22 May 2019 00:22:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FA6A28AE3 for ; Wed, 22 May 2019 00:22:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9442A28AE6; Wed, 22 May 2019 00:22:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3C74728AEB for ; Wed, 22 May 2019 00:22:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727899AbfEVAW1 (ORCPT ); Tue, 21 May 2019 20:22:27 -0400 Received: from mail-yb1-f202.google.com ([209.85.219.202]:46130 "EHLO mail-yb1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725797AbfEVAW1 (ORCPT ); Tue, 21 May 2019 20:22:27 -0400 Received: by mail-yb1-f202.google.com with SMTP id v15so577544ybe.13 for ; Tue, 21 May 2019 17:22:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=GdyG56vSCo0x4ulaUpduxV8qpRdNpP8CGYxlpDpGSAw=; b=Rtw67QFR2tXnqIV5l16mseBZhi/lTxfM6G08vJg8j2+kdOIjl46WcXVHedMPN3n6+h YEWWUSAmPDwswdbvI30cEUdQ0vgtgENvEzc+QU/UKR+KqtSX7hwwxyEpotP3um0Xa/Zf AZaY6MVhQBPhLPbp80/h2N0EMsGIOgGiZERg79r0Dv9m42g9eL1L7znYXgzhxKAqroEX 2vtWJjfKnRKqi/HslfnuZR62s0Jp8S4QLbD0rbyKoXj6X7Ec54iHXdoQ1EM2llPMDn46 j1q1rPviCWiL8g4j1p7AeRNltS3NxDmFz9O6/xRD0WkSaJXE+pKbm5nLnPLAANg+3z9w QOXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=GdyG56vSCo0x4ulaUpduxV8qpRdNpP8CGYxlpDpGSAw=; b=AbgX7Buv/apNkKaBcfj/9Yl6Z8n/NKlf4yvKPuSdS923Y6VpXvcFfyp7CEbtnII+y0 ljcdHn6hGuaueEyegxEMfoF79UAa8ucPfJfqoT8mXHeYNlsPiVXTbdMsCaenol8ls5TF J/Yfc9T1jmeaQ2rmR3wuv4Do00akZR3M7MVV0P0GMpSj+gVwaOT8O5dEg2B2GrF9Xb87 dKWUIHkYOdSd2q1xVuI7LB3GmWQsDWIZW1zBIF0j3sp/vnzdz+oTcXRYOIdmEcVQmCE9 pU7K3H9K8I8HcmA3peuMcDW6A40mkkWu0GWs0HMAGO3csXdPociB/Da6oJ9IBUqZfZcp l9zQ== X-Gm-Message-State: APjAAAXc/mSgmxX6ChX15F5bIS1iBmIMVgVYqywxfiuh1zF2t1uPKS8X ODQUgvG9Ej8qE/Y7If7lHwddQK/+IS8= X-Google-Smtp-Source: APXvYqyV5JIr3v61NOdGVx3Oe8dKgkTk9zq5xSSjUzk55ZCj49QLmh3adP/t3Toa24cKZtSY96A77OV4v1Kd X-Received: by 2002:a81:2d07:: with SMTP id t7mr29465567ywt.495.1558484546139; Tue, 21 May 2019 17:22:26 -0700 (PDT) Date: Tue, 21 May 2019 17:21:53 -0700 In-Reply-To: Message-Id: <4a8e92ad97e65f6cda65b5cb120182ae8612b436.1558484115.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH v1 4/5] list-objects-filter-options: move error check up From: Matthew DeVore To: jonathantanmy@google.com, jrn@google.com, git@vger.kernel.org, dstolee@microsoft.com, jeffhost@microsoft.com, jrnieder@gmail.com, pclouds@gmail.com Cc: Matthew DeVore , matvore@comcast.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Move the check that filter_options->choice is set to higher in the call stack. This can only be set when the gentle parse function is called from one of the two call sites. This is important because in an upcoming patch this may or may not be an error, and whether it is an error is only known to the parse_list_objects_filter function. Signed-off-by: Matthew DeVore --- list-objects-filter-options.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index d7a1516188..647b2b220e 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -27,28 +27,22 @@ static int parse_combine_filter( * expand_list_objects_filter_spec() first). We also "intern" the arg for the * convenience of the current command. */ static int gently_parse_list_objects_filter( struct list_objects_filter_options *filter_options, const char *arg, struct strbuf *errbuf) { const char *v0; - if (filter_options->choice) { - if (errbuf) { - strbuf_addstr( - errbuf, - _("multiple filter-specs cannot be combined")); - } - return 1; - } + if (filter_options->choice) + BUG("filter_options already populated"); if (!strcmp(arg, "blob:none")) { filter_options->choice = LOFC_BLOB_NONE; return 0; } else if (skip_prefix(arg, "blob:limit=", &v0)) { if (git_parse_ulong(v0, &filter_options->blob_limit_value)) { filter_options->choice = LOFC_BLOB_LIMIT; return 0; } @@ -239,20 +233,22 @@ cleanup: list_objects_filter_release(filter_options); memset(filter_options, 0, sizeof(*filter_options)); } return result; } int parse_list_objects_filter(struct list_objects_filter_options *filter_options, const char *arg) { struct strbuf buf = STRBUF_INIT; + if (filter_options->choice) + die(_("multiple filter-specs cannot be combined")); filter_options->filter_spec = strdup(arg); if (gently_parse_list_objects_filter(filter_options, arg, &buf)) die("%s", buf.buf); return 0; } int opt_parse_list_objects_filter(const struct option *opt, const char *arg, int unset) { struct list_objects_filter_options *filter_options = opt->value; From patchwork Wed May 22 00:21:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10954605 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B44D924 for ; Wed, 22 May 2019 00:22:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5959A28AE3 for ; Wed, 22 May 2019 00:22:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4DAB928AE8; Wed, 22 May 2019 00:22:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17D2228AE3 for ; Wed, 22 May 2019 00:22:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727946AbfEVAWa (ORCPT ); Tue, 21 May 2019 20:22:30 -0400 Received: from mail-qk1-f202.google.com ([209.85.222.202]:43630 "EHLO mail-qk1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725797AbfEVAWa (ORCPT ); Tue, 21 May 2019 20:22:30 -0400 Received: by mail-qk1-f202.google.com with SMTP id p190so716216qke.10 for ; Tue, 21 May 2019 17:22:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=sQaME9Q86Ia4lRkeUz2RWTcxX6g9FO7z0lOHX9WVDFU=; b=T/QsuouiTHHydGaxi5t2SJ0B/3tahpFmeQKeqFkT4WMKjxmfMwSSWC4sA9BrOEhiRy mY8nHb1xHY24WICAyZpDHj3hM41Ms1vVaZVSr9jbxpCHSQ+bYt0CigjNncSv2SfCLvgc PcH7AZCphbYz/J77p8QvwM6bZ5SQp5My2u3EHzJW1Zh2WBp5ferIYT9NePiJYiIfbGHB oAgmizdouxGrzdiHoew7Sikck/rszioW3U/mjHe+JWIOk5s6YCNDxffm+XAbNHSgt91/ +GGRcy/0tzPoUOiPLDc/uKNljwocDiXzbGZoPmyYzrF9mSxxj65ffirVjsKzt3MCeurL 9rdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=sQaME9Q86Ia4lRkeUz2RWTcxX6g9FO7z0lOHX9WVDFU=; b=GudfhOVMb5pN1PZqVOhoWxqU/SvMtpdcuCm8NqbDLF+LcP7e0Zg2+dzTf2bpBrZ3Rd LKv1PqhekQduHjpQn/F+vpPzG+8LTFBLVJR6/6Np3G2UeTamYY3wSpogj759mmjaXSZi XiEtwLANthQFUrZHoPL48wpmDSyIrG34jrxg4H/LmDMsIOt1mK0FjvGbU5DhXd9CXrIH dribxjgaMpoG3qf3iyMYoeMLmDLfTxB1F6c72VFfVuoxpewIU5JWEWPOlKS4Q6QsE6rH Hl48nQV/r0SzruaQH+rbxCQUCo8fFumj74iP8hPpTI4R/0uA+ie3cmiEZLSxeeC+Lpxu hhkQ== X-Gm-Message-State: APjAAAUart/hUEbb6tkRICqjj8LtQLOjhvmORFa3wpEUv+yb3wk9E+sg 0lj6tw3Wnj6ByzMWXZ6lsKamRlt7QSU= X-Google-Smtp-Source: APXvYqxqpv0yRVZGOMLsyG0vKIO1X83QJ1cUqoH31Ry1HJfbc5VFblU17fLhqWzMZvMyk6Cg+IsOSW2vdxPt X-Received: by 2002:ac8:3128:: with SMTP id g37mr73184376qtb.65.1558484548748; Tue, 21 May 2019 17:22:28 -0700 (PDT) Date: Tue, 21 May 2019 17:21:54 -0700 In-Reply-To: Message-Id: <490519da8013a49b27040804c6ef50e42fd8754d.1558484115.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH v1 5/5] list-objects-filter-options: allow mult. --filter From: Matthew DeVore To: jonathantanmy@google.com, jrn@google.com, git@vger.kernel.org, dstolee@microsoft.com, jeffhost@microsoft.com, jrnieder@gmail.com, pclouds@gmail.com Cc: Matthew DeVore , matvore@comcast.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow combining of multiple filters by simply repeating the --filter flag. Before this patch, the user had to combine them in a single flag somewhat awkwardly (e.g. --filter=combine:FOO+BAR), including URL-encoding the individual filters. To make this work, in the --filter flag parsing callback, rather than error out when we detect that the filter_options struct is already populated, we modify it in-place to contain the added sub-filter. The existing sub-filter becomes the lhs of the combined filter, and the next sub-filter becomes the rhs. We also have to URL-encode the LHS and RHS sub-filters. We can simplify the operation if the LHS is already a combine: filter. In that case, we just append the URL-encoded RHS sub-filter to the LHS spec to get the new spec. Signed-off-by: Matthew DeVore --- Documentation/rev-list-options.txt | 16 +++--- builtin/fetch-pack.c | 5 +- builtin/rev-list.c | 5 +- fetch-pack.c | 5 +- list-objects-filter-options.c | 83 +++++++++++++++++++++++++---- list-objects-filter-options.h | 3 +- t/t5616-partial-clone.sh | 19 +++++++ t/t6112-rev-list-filters-objects.sh | 38 ++++++++++++- transport.c | 5 +- upload-pack.c | 10 +++- 10 files changed, 164 insertions(+), 25 deletions(-) diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 4fb0c4fbb0..2be5f3a6d1 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -731,27 +731,29 @@ specification contained in . The form '--filter=tree:' omits all blobs and trees whose depth from the root tree is >= (minimum depth if an object is located at multiple depths in the commits traversed). =0 will not include any trees or blobs unless included explicitly in the command-line (or standard input when --stdin is used). =1 will include only the tree and blobs which are referenced directly by a commit reachable from or an explicitly-given object. =2 is like =1 while also including trees and blobs one more level removed from an explicitly-given commit or tree. + -The form '--filter=combine:++...' combines -several filters. Only objects which are accepted by every filter are -included. Filters are joined by '{plus}' and individual filters are %-encoded -(i.e. URL-encoded). Besides the '{plus}' and '%' characters, the following -characters are reserved and also must be encoded: -`~!@#$^&*()[]{}\;",<>?`+'`+ as well as all characters with ASCII code -<= `0x20`, which includes space and newline. +Multiple '--filter=' flags can be specified to combine filters. Only +objects which are accepted by every filter are included. ++ +The form '--filter=combine:++...' can also be +used to combine filters. Filters are joined by '{plus}' and individual +filters are %-encoded (i.e. URL-encoded). Besides the '{plus}' and '%' +characters, the following characters are reserved and also must be +encoded: `~!@#$^&*()[]{}\;",<>?`+'`+ as well as all characters +with ASCII code <= `0x20`, which includes space and newline. + Other arbitrary characters can also be encoded. For instance, 'combine:tree:3+blob:none' and 'combine:tree%3A2+blob%3Anone' are equivalent. --no-filter:: Turn off any previous `--filter=` argument. --filter-print-omitted:: Only useful with `--filter=`; prints a list of the objects omitted diff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c index dc1485c8aa..cadcb2b915 100644 --- a/builtin/fetch-pack.c +++ b/builtin/fetch-pack.c @@ -151,21 +151,24 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix) } if (!strcmp("--from-promisor", arg)) { args.from_promisor = 1; continue; } if (!strcmp("--no-dependents", arg)) { args.no_dependents = 1; continue; } if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { - parse_list_objects_filter(&args.filter_options, arg); + parse_list_objects_filter( + &args.filter_options, + arg, + /*allow_implicit_combine=*/1); continue; } if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { list_objects_filter_set_no_filter(&args.filter_options); continue; } usage(fetch_pack_usage); } if (deepen_not.nr) args.deepen_not = &deepen_not; diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 9f31837d30..e584e7d1ac 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -454,21 +454,24 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) if (!strcmp(arg, "--test-bitmap")) { test_bitmap_walk(&revs); return 0; } if (skip_prefix(arg, "--progress=", &arg)) { show_progress = arg; continue; } if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { - parse_list_objects_filter(&filter_options, arg); + parse_list_objects_filter( + &filter_options, + arg, + /*allow_implicit_combine=*/1); if (filter_options.choice && !revs.blob_objects) die(_("object filtering requires --objects")); if (filter_options.choice == LOFC_SPARSE_OID && !filter_options.sparse_oid_value) die(_("invalid sparse value '%s'"), filter_options.filter_spec); continue; } if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { list_objects_filter_set_no_filter(&filter_options); diff --git a/fetch-pack.c b/fetch-pack.c index 3f24d0c8a6..c58fd9148a 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1661,21 +1661,24 @@ struct ref *fetch_pack(struct fetch_pack_args *args, /* * The protocol does not support requesting that only the * wanted objects be sent, so approximate this by setting a * "blob:none" filter if no filter is already set. This works * for all object types: note that wanted blobs will still be * sent because they are directly specified as a "want". * * NEEDSWORK: Add an option in the protocol to request that * only the wanted objects be sent, and implement it. */ - parse_list_objects_filter(&args->filter_options, "blob:none"); + parse_list_objects_filter( + &args->filter_options, + "blob:none", + /*allow_implicit_combine=*/0); } if (version != protocol_v2 && !ref) { packet_flush(fd[1]); die(_("no matching remote head")); } if (version == protocol_v2) { if (shallow->nr) BUG("Protocol V2 does not provide shallows at this point in the fetch"); memset(&si, 0, sizeof(si)); diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index 647b2b220e..a0cc87c62b 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -1,18 +1,19 @@ #include "cache.h" #include "commit.h" #include "config.h" #include "revision.h" #include "argv-array.h" #include "list-objects.h" #include "list-objects-filter.h" #include "list-objects-filter-options.h" +#include "trace.h" static int parse_combine_filter( struct list_objects_filter_options *filter_options, const char *arg, struct strbuf *errbuf); /* * Parse value of the argument to the "filter" keyword. * On the command line this looks like: * --filter= @@ -229,43 +230,107 @@ static int parse_combine_filter( cleanup: strbuf_list_free(sub_specs); if (result) { list_objects_filter_release(filter_options); memset(filter_options, 0, sizeof(*filter_options)); } return result; } -int parse_list_objects_filter(struct list_objects_filter_options *filter_options, - const char *arg) +static void add_url_encoded(struct strbuf *dest, const char *s) { - struct strbuf buf = STRBUF_INIT; - if (filter_options->choice) - die(_("multiple filter-specs cannot be combined")); - filter_options->filter_spec = strdup(arg); - if (gently_parse_list_objects_filter(filter_options, arg, &buf)) - die("%s", buf.buf); + while (*s) { + if (*s <= ' ' || strchr(RESERVED_NON_WS, *s) || + *s == '%' || *s == '+') + strbuf_addf(dest, "%%%02X", (int)*s); + else + strbuf_addf(dest, "%c", *s); + s++; + } +} + +/* + * Returns a new filter-spec string by combining (with combine:) the two + * sub-specs. The caller gains ownership of a new string, and lhs and rhs are + * not freed. + */ +static char *combine_specs(const char *lhs, const char *rhs) +{ + struct strbuf combined = STRBUF_INIT; + if (starts_with(lhs, "combine:")) { + strbuf_addf(&combined, "%s", lhs); + } else { + strbuf_addf(&combined, "combine:"); + add_url_encoded(&combined, lhs); + } + strbuf_addf(&combined, "+"); + + add_url_encoded(&combined, rhs); + trace_printf("Generated composite filter-spec: %s\n", combined.buf); + return strbuf_detach(&combined, NULL); +} + +int parse_list_objects_filter( + struct list_objects_filter_options *filter_options, + const char *arg, + int allow_implicit_combine) +{ + struct strbuf errbuf = STRBUF_INIT; + if (filter_options->choice) { + struct list_objects_filter_options *lhs; + + if (!allow_implicit_combine) + die(_("multiple filter-specs cannot be combined")); + + lhs = xcalloc(1, sizeof(*lhs)); + *lhs = *filter_options; + memset(filter_options, 0, sizeof(*filter_options)); + + filter_options->lhs = lhs; + filter_options->rhs = xcalloc(1, sizeof(*filter_options->rhs)); + filter_options->choice = LOFC_COMBINE; + + /* + * Build up the filter-spec string using the already-parsed + * portion (the lhs) and the to-be-parsed portion (the rhs). + */ + filter_options->filter_spec = combine_specs( + lhs->filter_spec, arg); + FREE_AND_NULL(lhs->filter_spec); + + /* + * The gentle parse function below will populate the rhs of the + * combined filter. But the caller of *this* function sees + * filter_options as the combined filter. + */ + filter_options = filter_options->rhs; + } else { + filter_options->filter_spec = strdup(arg); + } + if (gently_parse_list_objects_filter(filter_options, arg, &errbuf)) + die("%s", errbuf.buf); return 0; } int opt_parse_list_objects_filter(const struct option *opt, const char *arg, int unset) { struct list_objects_filter_options *filter_options = opt->value; if (unset || !arg) { list_objects_filter_set_no_filter(filter_options); return 0; } - return parse_list_objects_filter(filter_options, arg); + return parse_list_objects_filter( + filter_options, arg, /*allow_implicit_combine=*/1); } void expand_list_objects_filter_spec( const struct list_objects_filter_options *filter, struct strbuf *expanded_spec) { strbuf_init(expanded_spec, strlen(filter->filter_spec)); if (filter->choice == LOFC_BLOB_LIMIT) strbuf_addf(expanded_spec, "blob:limit=%lu", filter->blob_limit_value); diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 6c0f0ecd08..a3ad00fde2 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -55,21 +55,22 @@ struct list_objects_filter_options { /* * END choice-specific parsed values. */ }; /* Normalized command line arguments */ #define CL_ARG__FILTER "filter" int parse_list_objects_filter( struct list_objects_filter_options *filter_options, - const char *arg); + const char *arg, + int allow_implicit_combine); int opt_parse_list_objects_filter(const struct option *opt, const char *arg, int unset); #define OPT_PARSE_LIST_OBJECTS_FILTER(fo) \ { OPTION_CALLBACK, 0, CL_ARG__FILTER, fo, N_("args"), \ N_("object filtering"), 0, \ opt_parse_list_objects_filter } /* diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index 9a8f9886b3..11536f4028 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -201,20 +201,39 @@ test_expect_success 'use fsck before and after manually fetching a missing subtr test_line_count = 70 fetched_objects && awk -f print_1.awk fetched_objects | xargs -n1 git -C dst cat-file -t >fetched_types && sort -u fetched_types >unique_types.observed && test_write_lines blob commit tree >unique_types.expected && test_cmp unique_types.expected unique_types.observed ' +test_expect_success 'implicitly construct combine: filter with repeated flags' ' + GIT_TRACE=$(pwd)/trace git clone --bare \ + --filter=blob:none --filter=tree:1 \ + "file://$(pwd)/srv.bare" pc2 && + grep "trace:.* git pack-objects .*--filter=combine:blob:none+tree:1" \ + trace && + git -C pc2 rev-list --objects --missing=allow-any HEAD >objects && + + # We should have gotten some root trees. + grep " $" objects && + # Should not have gotten any non-root trees or blobs. + ! grep " ." objects && + + xargs -n 1 git -C pc2 cat-file -t types && + sort -u types >unique_types.actual && + test_write_lines commit tree >unique_types.expected && + test_cmp unique_types.expected unique_types.actual +' + test_expect_success 'partial clone fetches blobs pointed to by refs even if normally filtered out' ' rm -rf src dst && git init src && test_commit -C src x && test_config -C src uploadpack.allowfilter 1 && test_config -C src uploadpack.allowanysha1inwant 1 && # Create a tag pointing to a blob. BLOB=$(echo blob-contents | git -C src hash-object --stdin -w) && git -C src tag myblob "$BLOB" && diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index ddfacb1a1a..104248c73d 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -357,21 +357,30 @@ test_expect_success 'verify tree:3 includes everything expected' ' test_expect_success 'combine:... for a simple combination' ' git -C r3 rev-list --objects --filter=combine:tree:2+blob:none HEAD \ >actual && expect_has HEAD "" && expect_has HEAD~1 "" && expect_has HEAD dir1 && # There are also 2 commit objects - test_line_count = 5 actual + test_line_count = 5 actual && + + cp actual expected && + + # Try again using repeated --filter - this is equivalent to a manual + # combine with "combine:...+..." + git -C r3 rev-list --objects --filter=combine:tree:2 \ + --filter=blob:none HEAD >actual && + + test_cmp expected actual ' test_expect_success 'combine:... with URL encoding' ' git -C r3 rev-list --objects \ --filter=combine:tree%3a2+blob:%6Eon%65 HEAD >actual && expect_has HEAD "" && expect_has HEAD~1 "" && expect_has HEAD dir1 && @@ -459,21 +468,46 @@ test_expect_success 'combine:... with more than two sub-filters' ' test_line_count = 7 actual && # Try again, this time making sure the last sub-filter is only # URL-decoded once. cp pattern1 pattern1+renamed% && cp actual expect && git -C r3 rev-list --objects \ --filter=combine:tree:3+blob:limit=40+sparse:path=../pattern1%2brenamed%25 \ HEAD >actual && - test_cmp expect actual + test_cmp expect actual && + + # Use the same composite filter again, but with a pattern file name that + # requires encoding multiple characters, and use implicit filter + # combining. + cp pattern1 "p;at%ter+n" && + GIT_TRACE=$(pwd)/trace git -C r3 rev-list --objects \ + --filter=tree:3 --filter=blob:limit=40 \ + --filter=sparse:path="../p;at%ter+n" \ + HEAD >actual && + + test_cmp expect actual && + grep "Generated composite filter-spec: combine:tree:3+blob:limit=40+sparse:path=../p%3Bat%25ter%2B" \ + trace && + + # Repeat the above test, but this time, the characters to encode are in + # the LHS of the combined filter. + cp pattern1 "^~pattern" && + GIT_TRACE=$(pwd)/trace git -C r3 rev-list --objects \ + --filter=sparse:path="../^~pattern" \ + --filter=tree:3 --filter=blob:limit=40 \ + HEAD >actual && + + test_cmp expect actual && + grep "Generated composite filter-spec: combine:sparse:path=../%5E%7Epattern+tree:3+blob:limit=40" \ + trace ' # Test provisional omit collection logic with a repo that has objects appearing # at multiple depths - first deeper than the filter's threshold, then shallow. test_expect_success 'setup r4' ' git init r4 && echo foo > r4/foo && mkdir r4/subdir && diff --git a/transport.c b/transport.c index f1fcd2c4b0..63100da143 100644 --- a/transport.c +++ b/transport.c @@ -217,21 +217,24 @@ static int set_git_option(struct git_transport_options *opts, } else if (!strcmp(name, TRANS_OPT_DEEPEN_RELATIVE)) { opts->deepen_relative = !!value; return 0; } else if (!strcmp(name, TRANS_OPT_FROM_PROMISOR)) { opts->from_promisor = !!value; return 0; } else if (!strcmp(name, TRANS_OPT_NO_DEPENDENTS)) { opts->no_dependents = !!value; return 0; } else if (!strcmp(name, TRANS_OPT_LIST_OBJECTS_FILTER)) { - parse_list_objects_filter(&opts->filter_options, value); + parse_list_objects_filter( + &opts->filter_options, + value, + /*allow_implicit_combine=*/0); return 0; } return 1; } static int connect_setup(struct transport *transport, int for_push) { struct git_transport_data *data = transport->data; int flags = transport->verbose > 0 ? CONNECT_VERBOSE : 0; diff --git a/upload-pack.c b/upload-pack.c index d2ea5eb20d..e3f2618600 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -877,21 +877,24 @@ static void receive_needs(struct packet_reader *reader, struct object_array *wan if (process_deepen(reader->line, &depth)) continue; if (process_deepen_since(reader->line, &deepen_since, &deepen_rev_list)) continue; if (process_deepen_not(reader->line, &deepen_not, &deepen_rev_list)) continue; if (skip_prefix(reader->line, "filter ", &arg)) { if (!filter_capability_requested) die("git upload-pack: filtering capability not negotiated"); - parse_list_objects_filter(&filter_options, arg); + parse_list_objects_filter( + &filter_options, + arg, + /*allow_implicit_combine=*/0); continue; } if (!skip_prefix(reader->line, "want ", &arg) || parse_oid_hex(arg, &oid_buf, &features)) die("git upload-pack: protocol error, " "expected to get object ID, not '%s'", reader->line); if (parse_feature_request(features, "deepen-relative")) deepen_relative = 1; @@ -1296,21 +1299,24 @@ static void process_args(struct packet_reader *request, continue; if (process_deepen_not(arg, &data->deepen_not, &data->deepen_rev_list)) continue; if (!strcmp(arg, "deepen-relative")) { data->deepen_relative = 1; continue; } if (allow_filter && skip_prefix(arg, "filter ", &p)) { - parse_list_objects_filter(&filter_options, p); + parse_list_objects_filter( + &filter_options, + p, + /*allow_implicit_combine=*/0); continue; } if ((git_env_bool("GIT_TEST_SIDEBAND_ALL", 0) || allow_sideband_all) && !strcmp(arg, "sideband-all")) { data->writer.use_sideband = 1; continue; }