From patchwork Thu Jul 15 15:40:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99C76C47E48 for ; Thu, 15 Jul 2021 15:40:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F7D0613E0 for ; Thu, 15 Jul 2021 15:40:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239097AbhGOPnY (ORCPT ); Thu, 15 Jul 2021 11:43:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238963AbhGOPnX (ORCPT ); Thu, 15 Jul 2021 11:43:23 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BD08C061760 for ; Thu, 15 Jul 2021 08:40:30 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id i94so8377568wri.4 for ; Thu, 15 Jul 2021 08:40:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=/+pHTUCgsn/eJhBt7TH6g2FT5CeygqVH9py2mRcVtgU=; b=BLNx5J7yMWNgRGAuvBSQiV3gzbdFnRrB5Ed6CWKfeAjErTX6+MiDaP/R0GVooAGEUL wSlWeSF1if8/slb2xb4C0whTut5vT0foIWnNEFOemBBwioU20ZcAGNmWD1hqMgLojWud Zh5lgMsN+WX5Pjb0+vdH74Zgc5Qn972MSV6wZjDsMN37WK1mCxxdSgz9BXnaLdn45LbJ k0xa5g+vdE0EsmeV1Lox0q5lUvQpSoxgjBD7texobPO3JuB/ch1s5S1P7C5hkFL1h+Nb uMDQprnVkL/3qHBMM1d+bwfd/lyE/NbSpC10DIs/9oRp4pTx6UUHG3hg3/ybXjXcz/J3 S2Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=/+pHTUCgsn/eJhBt7TH6g2FT5CeygqVH9py2mRcVtgU=; b=Xxho878vMwTtnF+r/fY/CoBBrhgoLu8Ncy4fSaC+pGU882+NlcTYBoOHKofb3PzyVA 9RaVZy/d126/j3sP7LOO0anZyc1fV/DXyB1uqyQh5U90dUkCRBjT/mHrYUI7yKT+fwgb WH6UccHaPtjGnd11EiYVtfx2ib6KhlRgLBKCj9MzdIATpR5uem93zX+JN4c+CqGwMAmu +u7k2RPW+xgm9b/9atMI+jon7FaJI7er26SmzSllMmAnRVYsxE8HFHBMNxUQnLzTJW/T /L0iDjt+0SVXLIOhLTRO/9lNlZ5Q/GA/cC3aQwZ1cR4rQpOe72cKTy8shHed8tPiEHHz nW8A== X-Gm-Message-State: AOAM530C8iGnyvN4nFKG/e9s8nGiEXY0jGjii8aXx0F9CNUCtloQ5PJ4 /ugyUuQ/FVNSS5In15tLQqC2xSzk8Rg= X-Google-Smtp-Source: ABdhPJwgH0XCE05xp40zjc/xhxlQJbkUfU9PANlGkSkJu2z1azFcs1+Gb/NyUj35uAllEoY9YBfTxw== X-Received: by 2002:a5d:6148:: with SMTP id y8mr6400053wrt.20.1626363628730; Thu, 15 Jul 2021 08:40:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d15sm6870322wri.39.2021.07.15.08.40.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:28 -0700 (PDT) Message-Id: <45c0cbe44d56b7f6dbe08db5fc1d06da529f6bd5.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:09 +0000 Subject: [PATCH v2 01/17] [GSOC] ref-filter: add obj-type check in grab contents Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Only tag and commit objects use `grab_sub_body_contents()` to grab object contents in the current codebase. We want to teach the function to also handle blobs and trees to get their raw data, without parsing a blob (whose contents looks like a commit or a tag) incorrectly as a commit or a tag. Skip the block of code that is specific to handling commits and tags early when the given object is of a wrong type to help later addition to handle other types of objects in this function. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Junio C Hamano Signed-off-by: ZheNing Hu --- ref-filter.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 4db0e40ff4c..5cee6512fba 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1356,11 +1356,12 @@ static void append_lines(struct strbuf *out, const char *buf, unsigned long size } /* See grab_values */ -static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) +static void grab_sub_body_contents(struct atom_value *val, int deref, struct expand_data *data) { int i; const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; + void *buf = data->content; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; @@ -1371,10 +1372,13 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) continue; if (deref) name++; - if (strcmp(name, "body") && - !starts_with(name, "subject") && - !starts_with(name, "trailers") && - !starts_with(name, "contents")) + + if ((data->type != OBJ_TAG && + data->type != OBJ_COMMIT) || + (strcmp(name, "body") && + !starts_with(name, "subject") && + !starts_with(name, "trailers") && + !starts_with(name, "contents"))) continue; if (!subpos) find_subpos(buf, @@ -1438,17 +1442,19 @@ static void fill_missing_values(struct atom_value *val) * pointed at by the ref itself; otherwise it is the object the * ref (which is a tag) refers to. */ -static void grab_values(struct atom_value *val, int deref, struct object *obj, void *buf) +static void grab_values(struct atom_value *val, int deref, struct object *obj, struct expand_data *data) { + void *buf = data->content; + switch (obj->type) { case OBJ_TAG: grab_tag_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("tagger", val, deref, buf); break; case OBJ_COMMIT: grab_commit_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("author", val, deref, buf); grab_person("committer", val, deref, buf); break; @@ -1678,7 +1684,7 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi->content); + grab_values(ref->value, deref, *obj, oi); } grab_common_values(ref->value, deref, oi); From patchwork Thu Jul 15 15:40:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7474C47E4D for ; Thu, 15 Jul 2021 15:40:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8F8A2613E7 for ; Thu, 15 Jul 2021 15:40:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239129AbhGOPnZ (ORCPT ); Thu, 15 Jul 2021 11:43:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239103AbhGOPnY (ORCPT ); Thu, 15 Jul 2021 11:43:24 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E0F9C061760 for ; Thu, 15 Jul 2021 08:40:31 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id g12so3999683wme.2 for ; Thu, 15 Jul 2021 08:40:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=SUDgA8t1JEBztczygiAw3+rz9y2lUTOzWXg3n4xUxXY=; b=nKDupV2U/ow3phq98bq05nYSJmICZVLlXYRKmJqUiz2Fa5UP4vyWKE1nvrevtEUccg S49wbrCxtJ6ArQtWob9FwH8iuo60/nlaWElyeXJOtzMYPF/gr2ielw/asU9sL2R5KywN dGKP/Q2khxbAA98Hr3sZXVjtGrXpyog+hFOuJt2mm3XkLUERt2XK/rxpNIeqVNDKgcRT tZpX/CIWnjhcjt1BN+2MuONi+f5Yy0Mu5upsms0FD2HyN6LWkYbPZPer8XgWaEmFpAjz hptkkRpqsYkbDnoG8i5j+1qDJ/uzELtlSyNttPNS21Wq2fRn/+iJ3/4hS23eRX8/pORl kaXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=SUDgA8t1JEBztczygiAw3+rz9y2lUTOzWXg3n4xUxXY=; b=i+mHonjPUp3zOFHz44JkwWRQBaUHfgNASsbgy/NXv7kgfoxkg5hWJduWOX71rHYVsC xyTIKHAOIv7kQFZB9LgZllsDJ7PWqiCI5DqnlkPI9Rg/olpA3tUO7Zd7tdpJVznHl9TY cFaKSww8T3kVf+3f0/V1FoteK4L54jo33dY7E7ZKCSw0eI2onwt1JpiVWIOwHAWkuVBT 6tgK2E0NsWxS749BtdypxWfYjkR96+i+aZHq+61G6nK3FjeZQH8xcWizntvu6Z7GLeTD wtCeBLkWPpg85OQqljHwAerK5lecNCp8vRfRy4VHq8EAP8hIbu/fYpLRXOL+w4sktaXB P2jQ== X-Gm-Message-State: AOAM530qXy9RYZHl+4hCV4a+GpRDrzn0ASQcDVxc5jwk+/NiT8/aBjV1 J0wWrKyjCT320aL1b4irSw+ztbYPORU= X-Google-Smtp-Source: ABdhPJwV4VytsSXTk0PtK7lGmszsnXR5FrDTS7Ebs3LZtOphrfYkbFpNHoQssu8eGRBU4piJIUOqOw== X-Received: by 2002:a7b:c762:: with SMTP id x2mr2738815wmk.21.1626363629523; Thu, 15 Jul 2021 08:40:29 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c2sm6944356wrn.28.2021.07.15.08.40.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:29 -0700 (PDT) Message-Id: <554d7653ee7a9c4f57136eb5bdcd924338268532.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:10 +0000 Subject: [PATCH v2 02/17] [GSOC] ref-filter: add %(raw) atom MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add new formatting option `%(raw)`, which will print the raw object data without any changes. It will help further to migrate all cat-file formatting logic from cat-file to ref-filter. The raw data of blob, tree objects may contain '\0', but most of the logic in `ref-filter` depends on the output of the atom being text (specifically, no embedded NULs in it). E.g. `quote_formatting()` use `strbuf_addstr()` or `*._quote_buf()` add the data to the buffer. The raw data of a tree object is `100644 one\0...`, only the `100644 one` will be added to the buffer, which is incorrect. Therefore, we need to find a way to record the length of the atom_value's member `s`. Although strbuf can already record the string and its length, if we want to replace the type of atom_value's member `s` with strbuf, many places in ref-filter that are filled with dynamically allocated mermory in `v->s` are not easy to replace. At the same time, we need to check if `v->s == NULL` in populate_value(), and strbuf cannot easily distinguish NULL and empty strings, but c-style "const char *" can do it. So add a new member in `struct atom_value`: `s_size`, which can record raw object size, it can help us add raw object data to the buffer or compare two buffers which contain raw object data. Note that `--format=%(raw)` cannot be used with `--python`, `--shell`, `--tcl`, and `--perl` because if the binary raw data is passed to a variable in such languages, these may not support arbitrary binary data in their string variable type. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Bagas Sanjaya Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Felipe Contreras Helped-by: Phillip Wood Helped-by: Junio C Hamano Based-on-patch-by: Olga Telezhnaya Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 9 ++ ref-filter.c | 140 +++++++++++++++---- t/t6300-for-each-ref.sh | 216 +++++++++++++++++++++++++++++ 3 files changed, 338 insertions(+), 27 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 2ae2478de70..cbb6f87d13f 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -235,6 +235,15 @@ and `date` to extract the named component. For email fields (`authoremail`, without angle brackets, and `:localpart` to get the part before the `@` symbol out of the trimmed email. +The raw data in an object is `raw`. + +raw:size:: + The raw data size of the object. + +Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, +`--perl` because such language may not support arbitrary binary data in their +string variable type. + The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/ref-filter.c b/ref-filter.c index 5cee6512fba..506fbc3d691 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -144,6 +144,7 @@ enum atom_type { ATOM_BODY, ATOM_TRAILERS, ATOM_CONTENTS, + ATOM_RAW, ATOM_UPSTREAM, ATOM_PUSH, ATOM_SYMREF, @@ -189,6 +190,9 @@ static struct used_atom { struct process_trailer_options trailer_opts; unsigned int nlines; } contents; + struct { + enum { RAW_BARE, RAW_LENGTH } option; + } raw_data; struct { cmp_status cmp_status; const char *str; @@ -426,6 +430,18 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } +static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (!arg) + atom->u.raw_data.option = RAW_BARE; + else if (!strcmp(arg, "size")) + atom->u.raw_data.option = RAW_LENGTH; + else + return strbuf_addf_ret(err, -1, _("unrecognized %%(raw) argument: %s"), arg); + return 0; +} + static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { @@ -586,6 +602,7 @@ static struct { [ATOM_BODY] = { "body", SOURCE_OBJ, FIELD_STR, body_atom_parser }, [ATOM_TRAILERS] = { "trailers", SOURCE_OBJ, FIELD_STR, trailers_atom_parser }, [ATOM_CONTENTS] = { "contents", SOURCE_OBJ, FIELD_STR, contents_atom_parser }, + [ATOM_RAW] = { "raw", SOURCE_OBJ, FIELD_STR, raw_atom_parser }, [ATOM_UPSTREAM] = { "upstream", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_PUSH] = { "push", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_SYMREF] = { "symref", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -620,12 +637,15 @@ struct ref_formatting_state { struct atom_value { const char *s; + size_t s_size; int (*handler)(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err); uintmax_t value; /* used for sorting when not FIELD_STR */ struct used_atom *atom; }; +#define ATOM_VALUE_S_SIZE_INIT (-1) + /* * Used to parse format string and sort specifiers */ @@ -644,13 +664,6 @@ static int parse_ref_filter_atom(const struct ref_format *format, return strbuf_addf_ret(err, -1, _("malformed field name: %.*s"), (int)(ep-atom), atom); - /* Do we have the atom already used elsewhere? */ - for (i = 0; i < used_atom_cnt; i++) { - int len = strlen(used_atom[i].name); - if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) - return i; - } - /* * If the atom name has a colon, strip it and everything after * it off - it specifies the format for this entry, and @@ -660,6 +673,13 @@ static int parse_ref_filter_atom(const struct ref_format *format, arg = memchr(sp, ':', ep - sp); atom_len = (arg ? arg : ep) - sp; + /* Do we have the atom already used elsewhere? */ + for (i = 0; i < used_atom_cnt; i++) { + int len = strlen(used_atom[i].name); + if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) + return i; + } + /* Is the atom a valid one? */ for (i = 0; i < ARRAY_SIZE(valid_atom); i++) { int len = strlen(valid_atom[i].name); @@ -709,11 +729,14 @@ static int parse_ref_filter_atom(const struct ref_format *format, return at; } -static void quote_formatting(struct strbuf *s, const char *str, int quote_style) +static void quote_formatting(struct strbuf *s, const char *str, size_t len, int quote_style) { switch (quote_style) { case QUOTE_NONE: - strbuf_addstr(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(s, str, len); + else + strbuf_addstr(s, str); break; case QUOTE_SHELL: sq_quote_buf(s, str); @@ -740,9 +763,12 @@ static int append_atom(struct atom_value *v, struct ref_formatting_state *state, * encountered. */ if (!state->stack->prev) - quote_formatting(&state->stack->output, v->s, state->quote_style); + quote_formatting(&state->stack->output, v->s, v->s_size, state->quote_style); else - strbuf_addstr(&state->stack->output, v->s); + if (v->s_size != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(&state->stack->output, v->s, v->s_size); + else + strbuf_addstr(&state->stack->output, v->s); return 0; } @@ -842,21 +868,23 @@ static int if_atom_handler(struct atom_value *atomv, struct ref_formatting_state return 0; } -static int is_empty(const char *s) +static int is_empty(struct strbuf *buf) { - while (*s != '\0') { - if (!isspace(*s)) - return 0; - s++; - } - return 1; -} + const char *cur = buf->buf; + const char *end = buf->buf + buf->len; + + while (cur != end && (isspace(*cur))) + cur++; + + return cur == end; + } static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err) { struct ref_formatting_stack *cur = state->stack; struct if_then_else *if_then_else = NULL; + size_t str_len = 0; if (cur->at_end == if_then_else_handler) if_then_else = (struct if_then_else *)cur->at_end_data; @@ -867,18 +895,22 @@ static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_sta if (if_then_else->else_atom_seen) return strbuf_addf_ret(err, -1, _("format: %%(then) atom used after %%(else)")); if_then_else->then_atom_seen = 1; + if (if_then_else->str) + str_len = strlen(if_then_else->str); /* * If the 'equals' or 'notequals' attribute is used then * perform the required comparison. If not, only non-empty * strings satisfy the 'if' condition. */ if (if_then_else->cmp_status == COMPARE_EQUAL) { - if (!strcmp(if_then_else->str, cur->output.buf)) + if (str_len == cur->output.len && + !memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; } else if (if_then_else->cmp_status == COMPARE_UNEQUAL) { - if (strcmp(if_then_else->str, cur->output.buf)) + if (str_len != cur->output.len || + memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; - } else if (cur->output.len && !is_empty(cur->output.buf)) + } else if (cur->output.len && !is_empty(&cur->output)) if_then_else->condition_satisfied = 1; strbuf_reset(&cur->output); return 0; @@ -924,7 +956,7 @@ static int end_atom_handler(struct atom_value *atomv, struct ref_formatting_stat * only on the topmost supporting atom. */ if (!current->prev->prev) { - quote_formatting(&s, current->output.buf, state->quote_style); + quote_formatting(&s, current->output.buf, current->output.len, state->quote_style); strbuf_swap(¤t->output, &s); } strbuf_release(&s); @@ -974,6 +1006,10 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) + die(_("--format=%.*s cannot be used with" + "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) @@ -1367,12 +1403,25 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp struct used_atom *atom = &used_atom[i]; const char *name = atom->name; struct atom_value *v = &val[i]; + enum atom_type atom_type = atom->atom_type; if (!!deref != (*name == '*')) continue; if (deref) name++; + if (atom_type == ATOM_RAW) { + unsigned long buf_size = data->size; + + if (atom->u.raw_data.option == RAW_BARE) { + v->s = xmemdupz(buf, buf_size); + v->s_size = buf_size; + } else if (atom->u.raw_data.option == RAW_LENGTH) { + v->s = xstrfmt("%"PRIuMAX, (uintmax_t)buf_size); + } + continue; + } + if ((data->type != OBJ_TAG && data->type != OBJ_COMMIT) || (strcmp(name, "body") && @@ -1460,9 +1509,11 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s break; case OBJ_TREE: /* grab_tree_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; case OBJ_BLOB: /* grab_blob_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; default: die("Eh? Object of type %d?", obj->type); @@ -1766,6 +1817,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) const char *refname; struct branch *branch = NULL; + v->s_size = ATOM_VALUE_S_SIZE_INIT; v->handler = append_atom; v->atom = atom; @@ -2369,6 +2421,19 @@ static int compare_detached_head(struct ref_array_item *a, struct ref_array_item return 0; } +static int memcasecmp(const void *vs1, const void *vs2, size_t n) +{ + const char *s1 = vs1, *s2 = vs2; + const char *end = s1 + n; + + for (; s1 < end; s1++, s2++) { + int diff = tolower(*s1) - tolower(*s2); + if (diff) + return diff; + } + return 0; +} + static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, struct ref_array_item *b) { struct atom_value *va, *vb; @@ -2389,10 +2454,30 @@ static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, stru } else if (s->sort_flags & REF_SORTING_VERSION) { cmp = versioncmp(va->s, vb->s); } else if (cmp_type == FIELD_STR) { - int (*cmp_fn)(const char *, const char *); - cmp_fn = s->sort_flags & REF_SORTING_ICASE - ? strcasecmp : strcmp; - cmp = cmp_fn(va->s, vb->s); + if (va->s_size == ATOM_VALUE_S_SIZE_INIT && + vb->s_size == ATOM_VALUE_S_SIZE_INIT) { + int (*cmp_fn)(const char *, const char *); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? strcasecmp : strcmp; + cmp = cmp_fn(va->s, vb->s); + } else { + size_t a_size = va->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(va->s) : va->s_size; + size_t b_size = vb->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(vb->s) : vb->s_size; + int (*cmp_fn)(const void *, const void *, size_t); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? memcasecmp : memcmp; + + cmp = cmp_fn(va->s, vb->s, b_size > a_size ? + a_size : b_size); + if (!cmp) { + if (a_size > b_size) + cmp = 1; + else if (a_size < b_size) + cmp = -1; + } + } } else { if (va->value < vb->value) cmp = -1; @@ -2492,6 +2577,7 @@ int format_ref_array_item(struct ref_array_item *info, } if (format->need_color_reset_at_eol) { struct atom_value resetv; + resetv.s_size = ATOM_VALUE_S_SIZE_INIT; resetv.s = GIT_COLOR_RESET; if (append_atom(&resetv, &state, error_buf)) { pop_stack_element(&state.stack); diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 9e0214076b4..18554f62d94 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -130,6 +130,8 @@ test_atom head parent:short=10 '' test_atom head numparent 0 test_atom head object '' test_atom head type '' +test_atom head raw "$(git cat-file commit refs/heads/main) +" test_atom head '*objectname' '' test_atom head '*objecttype' '' test_atom head author 'A U Thor 1151968724 +0200' @@ -221,6 +223,15 @@ test_atom tag contents 'Tagging at 1151968727 ' test_atom tag HEAD ' ' +test_expect_success 'basic atom: refs/tags/testtag *raw' ' + git cat-file commit refs/tags/testtag^{} >expected && + git for-each-ref --format="%(*raw)" refs/tags/testtag >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'Check invalid atoms names are errors' ' test_must_fail git for-each-ref --format="%(INVALID)" refs/heads ' @@ -686,6 +697,15 @@ test_atom refs/tags/signed-empty contents:body '' test_atom refs/tags/signed-empty contents:signature "$sig" test_atom refs/tags/signed-empty contents "$sig" +test_expect_success GPG 'basic atom: refs/tags/signed-empty raw' ' + git cat-file tag refs/tags/signed-empty >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-empty >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-short subject 'subject line' test_atom refs/tags/signed-short subject:sanitize 'subject-line' test_atom refs/tags/signed-short contents:subject 'subject line' @@ -695,6 +715,15 @@ test_atom refs/tags/signed-short contents:signature "$sig" test_atom refs/tags/signed-short contents "subject line $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-short raw' ' + git cat-file tag refs/tags/signed-short >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-short >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-long subject 'subject line' test_atom refs/tags/signed-long subject:sanitize 'subject-line' test_atom refs/tags/signed-long contents:subject 'subject line' @@ -708,6 +737,15 @@ test_atom refs/tags/signed-long contents "subject line body contents $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-long raw' ' + git cat-file tag refs/tags/signed-long >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-long >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'set up refs pointing to tree and blob' ' git update-ref refs/mytrees/first refs/heads/main^{tree} && git update-ref refs/myblobs/first refs/heads/main:one @@ -720,6 +758,16 @@ test_atom refs/mytrees/first contents:body "" test_atom refs/mytrees/first contents:signature "" test_atom refs/mytrees/first contents "" +test_expect_success 'basic atom: refs/mytrees/first raw' ' + git cat-file tree refs/mytrees/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/mytrees/first >actual && + test_cmp expected actual && + git cat-file -s refs/mytrees/first >expected && + git for-each-ref --format="%(raw:size)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_atom refs/myblobs/first subject "" test_atom refs/myblobs/first contents:subject "" test_atom refs/myblobs/first body "" @@ -727,6 +775,174 @@ test_atom refs/myblobs/first contents:body "" test_atom refs/myblobs/first contents:signature "" test_atom refs/myblobs/first contents "" +test_expect_success 'basic atom: refs/myblobs/first raw' ' + git cat-file blob refs/myblobs/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/myblobs/first >actual && + test_cmp expected actual && + git cat-file -s refs/myblobs/first >expected && + git for-each-ref --format="%(raw:size)" refs/myblobs/first >actual && + test_cmp expected actual +' + +test_expect_success 'set up refs pointing to binary blob' ' + printf "a\0b\0c" >blob1 && + printf "a\0c\0b" >blob2 && + printf "\0a\0b\0c" >blob3 && + printf "abc" >blob4 && + printf "\0 \0 \0 " >blob5 && + printf "\0 \0a\0 " >blob6 && + printf " " >blob7 && + >blob8 && + obj=$(git hash-object -w blob1) && + git update-ref refs/myblobs/blob1 "$obj" && + obj=$(git hash-object -w blob2) && + git update-ref refs/myblobs/blob2 "$obj" && + obj=$(git hash-object -w blob3) && + git update-ref refs/myblobs/blob3 "$obj" && + obj=$(git hash-object -w blob4) && + git update-ref refs/myblobs/blob4 "$obj" && + obj=$(git hash-object -w blob5) && + git update-ref refs/myblobs/blob5 "$obj" && + obj=$(git hash-object -w blob6) && + git update-ref refs/myblobs/blob6 "$obj" && + obj=$(git hash-object -w blob7) && + git update-ref refs/myblobs/blob7 "$obj" && + obj=$(git hash-object -w blob8) && + git update-ref refs/myblobs/blob8 "$obj" +' + +test_expect_success 'Verify sorts with raw' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob3 + refs/myblobs/blob7 + refs/mytrees/first + refs/myblobs/first + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob4 + refs/heads/main + EOF + git for-each-ref --format="%(refname)" --sort=raw \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'Verify sorts with raw:size' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/first + refs/myblobs/blob7 + refs/heads/main + refs/myblobs/blob4 + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/mytrees/first + EOF + git for-each-ref --format="%(refname)" --sort=raw:size \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:equals)' ' + cat >expected <<-EOF && + not equals + not equals + not equals + not equals + not equals + not equals + refs/myblobs/blob4 + not equals + not equals + not equals + not equals + not equals + EOF + git for-each-ref --format="%(if:equals=abc)%(raw)%(then)%(refname)%(else)not equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:notequals)' ' + cat >expected <<-EOF && + refs/heads/ambiguous + refs/heads/main + refs/heads/newtag + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + equals + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob7 + refs/myblobs/blob8 + refs/myblobs/first + EOF + git for-each-ref --format="%(if:notequals=abc)%(raw)%(then)%(refname)%(else)equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'empty raw refs with %(if)' ' + cat >expected <<-EOF && + refs/myblobs/blob1 not empty + refs/myblobs/blob2 not empty + refs/myblobs/blob3 not empty + refs/myblobs/blob4 not empty + refs/myblobs/blob5 not empty + refs/myblobs/blob6 not empty + refs/myblobs/blob7 empty + refs/myblobs/blob8 empty + refs/myblobs/first not empty + EOF + git for-each-ref --format="%(refname) %(if)%(raw)%(then)not empty%(else)empty%(end)" \ + refs/myblobs/ >actual && + test_cmp expected actual +' + +test_expect_success '%(raw) with --python must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --python +' + +test_expect_success '%(raw) with --tcl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --tcl +' + +test_expect_success '%(raw) with --perl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --perl +' + +test_expect_success '%(raw) with --shell must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --shell +' + +test_expect_success '%(raw) with --shell and --sort=raw must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --sort=raw --shell +' + +test_expect_success '%(raw:size) with --shell' ' + git for-each-ref --format="%(raw:size)" | while read line + do + echo "'\''$line'\''" >>expect + done && + git for-each-ref --format="%(raw:size)" --shell >actual && + test_cmp expect actual +' + +test_expect_success 'for-each-ref --format compare with cat-file --batch' ' + git rev-parse refs/mytrees/first | git cat-file --batch >expected && + git for-each-ref --format="%(objectname) %(objecttype) %(objectsize) +%(raw)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_expect_success 'set up multiple-sort tags' ' for when in 100000 200000 do From patchwork Thu Jul 15 15:40:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8FF9C07E96 for ; Thu, 15 Jul 2021 15:40:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BDE67613E0 for ; Thu, 15 Jul 2021 15:40:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239160AbhGOPn2 (ORCPT ); Thu, 15 Jul 2021 11:43:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239125AbhGOPnZ (ORCPT ); Thu, 15 Jul 2021 11:43:25 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90758C06175F for ; Thu, 15 Jul 2021 08:40:31 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id l6so4032759wmq.0 for ; Thu, 15 Jul 2021 08:40:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=5lP/QekKLM+IAZt7gPZOgld34VmFhgG0WJwvzYiMv3c=; b=o/5zAepK2ELz/xLUy8zCxKRYystAFmUWZDIhAZEaT7MML5HBy9whvFZAe7qAk9+Ux0 Ql4wdeA7GLeztiyHDQgy6Hy9eO6J3glu2vjIcEwcXtGNVGjdQxRkmi25IDQEn4Hb4SZS Yuf488RIe+WNVmRC+YAh3rrzu/bW7bFJfe3RcQS5y4wKY3YRXb+X9VtlBWK0tRDhamjN pqJLrO8t+YIkqvG6b43pk2MCh5M+KmZAgR0ABRT13T7dV6M9sTunOq1ENSYyoJ1WLn3+ jDlAkzWQHMd9R4CbofeI7wwMQGzkRaJiX2tezYt9MWKmJt/cLCTaQ5bdo7HkauuO/86S F2HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=5lP/QekKLM+IAZt7gPZOgld34VmFhgG0WJwvzYiMv3c=; b=KJfQ7kTyF1mvaeiCWVozjQB9H8bkeNoE4KARupY9C2ikcyTHgwFuWUH7GchPTDgus2 FrmV41yR8/pMEAk3F8KP/4bSQirrXKwa318RHCRcXhPZsezRDgReTcYDjo3TFr9sta6a xULiICmJ1OHCLdZpXu6yBAvOuYHp1ifBi1ybki46HJ0j+2oncmFGR4avnEDm307sOzim XUAwHgpTNfBvgFeM/hBDidb6O4P7lP9FKx3ZkD8G/wJYGOw3KaiuSm13/xUlsMGSFgYw JzBbNWrVStHMtLa9p1STpoMx9g2+e6MW3OX1iJ1gsyDwPLHLKTkaq9e7XQeEbIbca2Wn pVXg== X-Gm-Message-State: AOAM5308Dw94rK6iJMyFjIDV9GcZJdj0gmxBPZDfLb3mO6Cg6mhHbKrI VaqyNZVXRTJobuyT/7fP8fNGifDAeks= X-Google-Smtp-Source: ABdhPJzmjaHOe5XwhTsaQOqBBjLtSFMrvw0QIuL0JWVCkqB1BKu7ybFAbUktVd0uFOZchdFLG7WOsg== X-Received: by 2002:a7b:c1c2:: with SMTP id a2mr5111356wmj.15.1626363630179; Thu, 15 Jul 2021 08:40:30 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n20sm5385198wmk.12.2021.07.15.08.40.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:29 -0700 (PDT) Message-Id: <94addd4676af68352d53d1b2bbb0ec911d8e70ce.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:11 +0000 Subject: [PATCH v2 03/17] [GSOC] ref-filter: --format=%(raw) re-support --perl MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because the perl language can handle binary data correctly, add the function perl_quote_buf_with_len(), which can specify the length of the data and prevent the data from being truncated at '\0' to help `--format="%(raw)"` re-support `--perl`. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 4 ++-- quote.c | 17 +++++++++++++++++ quote.h | 1 + ref-filter.c | 15 +++++++++++---- t/t6300-for-each-ref.sh | 19 +++++++++++++++++-- 5 files changed, 48 insertions(+), 8 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index cbb6f87d13f..6da899c6296 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -241,8 +241,8 @@ raw:size:: The raw data size of the object. Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, -`--perl` because such language may not support arbitrary binary data in their -string variable type. +because such language may not support arbitrary binary data in their string +variable type. The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/quote.c b/quote.c index 8a3a5e39eb1..26719d21d1e 100644 --- a/quote.c +++ b/quote.c @@ -471,6 +471,23 @@ void perl_quote_buf(struct strbuf *sb, const char *src) strbuf_addch(sb, sq); } +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len) +{ + const char sq = '\''; + const char bq = '\\'; + const char *c = src; + const char *end = src + len; + + strbuf_addch(sb, sq); + while (c != end) { + if (*c == sq || *c == bq) + strbuf_addch(sb, bq); + strbuf_addch(sb, *c); + c++; + } + strbuf_addch(sb, sq); +} + void python_quote_buf(struct strbuf *sb, const char *src) { const char sq = '\''; diff --git a/quote.h b/quote.h index 768cc6338e2..0fe69e264b0 100644 --- a/quote.h +++ b/quote.h @@ -94,6 +94,7 @@ char *quote_path(const char *in, const char *prefix, struct strbuf *out, unsigne /* quoting as a string literal for other languages */ void perl_quote_buf(struct strbuf *sb, const char *src); +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len); void python_quote_buf(struct strbuf *sb, const char *src); void tcl_quote_buf(struct strbuf *sb, const char *src); void basic_regex_quote_buf(struct strbuf *sb, const char *src); diff --git a/ref-filter.c b/ref-filter.c index 506fbc3d691..ba9ab35d7ec 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -742,7 +742,10 @@ static void quote_formatting(struct strbuf *s, const char *str, size_t len, int sq_quote_buf(s, str); break; case QUOTE_PERL: - perl_quote_buf(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + perl_quote_buf_with_len(s, str, len); + else + perl_quote_buf(s, str); break; case QUOTE_PYTHON: python_quote_buf(s, str); @@ -1006,10 +1009,14 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && - used_atom[at].u.raw_data.option == RAW_BARE) + + if ((format->quote_style == QUOTE_PYTHON || + format->quote_style == QUOTE_SHELL || + format->quote_style == QUOTE_TCL) && + used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) die(_("--format=%.*s cannot be used with" - "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); + "--python, --shell, --tcl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 18554f62d94..3d15d0a5360 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -915,8 +915,23 @@ test_expect_success '%(raw) with --tcl must fail' ' test_must_fail git for-each-ref --format="%(raw)" --tcl ' -test_expect_success '%(raw) with --perl must fail' ' - test_must_fail git for-each-ref --format="%(raw)" --perl +test_expect_success '%(raw) with --perl' ' + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob1 --perl | perl >actual && + cmp blob1 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob3 --perl | perl >actual && + cmp blob3 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob8 --perl | perl >actual && + cmp blob8 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/first --perl | perl >actual && + cmp one actual && + git cat-file tree refs/mytrees/first > expected && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/mytrees/first --perl | perl >actual && + cmp expected actual ' test_expect_success '%(raw) with --shell must fail' ' From patchwork Thu Jul 15 15:40:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7835C1B08C for ; Thu, 15 Jul 2021 15:40:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B16BE613DF for ; Thu, 15 Jul 2021 15:40:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239161AbhGOPn2 (ORCPT ); Thu, 15 Jul 2021 11:43:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239126AbhGOPnZ (ORCPT ); Thu, 15 Jul 2021 11:43:25 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F2E6C061760 for ; Thu, 15 Jul 2021 08:40:32 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id l7so8359928wrv.7 for ; Thu, 15 Jul 2021 08:40:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=oz3JgcuWDaxi4uV/irOCjt4bpOTOyAMAqZMr2CVa+Ms=; b=HlEIZx99Hp/XOtmprhLsXjp+z0UKyO/OKTFPB9O7TuvvSGUG+B6E2ukzKCBKiAHb2w SJ4aySVqNwkPGwWAxdX1XrORS+WoCgAJoGZbEMP/iZu7DoqoGLUwFLDWVFrHPkc8hjkX gzuIPL7+XZrK6vr5K3wTvc64s4/GPHRV4pSbBsA7l/k1CurAD7x0Neo8FGbsN+2wm0ie pCjebN/SIzT/lXA8m/oAt00oXBtIlIinydIuzUIfYg1ca1iueP/zRtZTbNGsseh+zYSr jDQmBw6PBdCMn1l2Ac+vNyPPFcVFRSbfwyXYWHfr5hR1KNRZU06MBuJNd5q+PqKtowZU Y1xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=oz3JgcuWDaxi4uV/irOCjt4bpOTOyAMAqZMr2CVa+Ms=; b=mp21V7YKqQVo/DoXWgQyKGt/IBFL9Mzhw8rLmWoGWd1SNWOqanhk6nBjE7OLnyZVxZ Pjz78BYMolSB6QwJQTUQ2yqyaaZZJK5iIQ2SfBBiOB714ZZJox1c0DmoWKipyVuja2WV fMuqv1ikrmBoccDbIMOMbmZAK1vi+W6STDPK9fpE7nnaYCfFrHj3DVmi3TU2/LbL4oJw w69BNpkkuPD+BMQPcz+rmrmx9tCYhegEjk9c66OOZ9bH47NUifrFNdzUNaTUYSSsAyyC VKjk3T4loJ7jsEJu7QSgjQNSh9iEijbznLdk8gcI/iNUQLORAj09rnTmbH81jnJOs4eb wQNQ== X-Gm-Message-State: AOAM532ipnm/DXjp7wShU5crzhoNvAN1NxCxh8Te0ygE8O6rRNSdqpmp 6lMi11mTtrjwYcAjgOnpxNtnC9T0Mfo= X-Google-Smtp-Source: ABdhPJz6BO2BszEgIY9V10o4pa00byy3wIETcQgMfZqHo7iBixLAjNzK1hO/aUoK5e1q24T/4EUoxg== X-Received: by 2002:a05:6000:144a:: with SMTP id v10mr6318908wrx.339.1626363630778; Thu, 15 Jul 2021 08:40:30 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v21sm8552886wml.5.2021.07.15.08.40.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:30 -0700 (PDT) Message-Id: <45984f94bf3514d244f4541390a0ec4f8d3b8c51.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:12 +0000 Subject: [PATCH v2 04/17] [GSOC] ref-filter: use non-const ref_format in *_atom_parser() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use non-const ref_format in *_atom_parser(), which can help us modify the members of ref_format in *_atom_parser(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/tag.c | 2 +- ref-filter.c | 44 ++++++++++++++++++++++---------------------- ref-filter.h | 4 ++-- 3 files changed, 25 insertions(+), 25 deletions(-) diff --git a/builtin/tag.c b/builtin/tag.c index 82fcfc09824..452558ec957 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -146,7 +146,7 @@ static int verify_tag(const char *name, const char *ref, const struct object_id *oid, void *cb_data) { int flags; - const struct ref_format *format = cb_data; + struct ref_format *format = cb_data; flags = GPG_VERIFY_VERBOSE; if (format->format) diff --git a/ref-filter.c b/ref-filter.c index ba9ab35d7ec..c8e561a3687 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -226,7 +226,7 @@ static int strbuf_addf_ret(struct strbuf *sb, int ret, const char *fmt, ...) return ret; } -static int color_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int color_atom_parser(struct ref_format *format, struct used_atom *atom, const char *color_value, struct strbuf *err) { if (!color_value) @@ -264,7 +264,7 @@ static int refname_atom_parser_internal(struct refname_atom *atom, const char *a return 0; } -static int remote_ref_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int remote_ref_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct string_list params = STRING_LIST_INIT_DUP; @@ -311,7 +311,7 @@ static int remote_ref_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objecttype_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objecttype_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -323,7 +323,7 @@ static int objecttype_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objectsize_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objectsize_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -343,7 +343,7 @@ static int objectsize_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int deltabase_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int deltabase_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -355,7 +355,7 @@ static int deltabase_atom_parser(const struct ref_format *format, struct used_at return 0; } -static int body_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int body_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -364,7 +364,7 @@ static int body_atom_parser(const struct ref_format *format, struct used_atom *a return 0; } -static int subject_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int subject_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -376,7 +376,7 @@ static int subject_atom_parser(const struct ref_format *format, struct used_atom return 0; } -static int trailers_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int trailers_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { atom->u.contents.trailer_opts.no_divider = 1; @@ -402,7 +402,7 @@ static int trailers_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int contents_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int contents_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -430,7 +430,7 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int raw_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -442,7 +442,7 @@ static int raw_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int oid_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -461,7 +461,7 @@ static int oid_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int person_email_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int person_email_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -475,7 +475,7 @@ static int person_email_atom_parser(const struct ref_format *format, struct used return 0; } -static int refname_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int refname_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { return refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err); @@ -492,7 +492,7 @@ static align_type parse_align_position(const char *s) return -1; } -static int align_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int align_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct align *align = &atom->u.align; @@ -544,7 +544,7 @@ static int align_atom_parser(const struct ref_format *format, struct used_atom * return 0; } -static int if_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int if_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -559,7 +559,7 @@ static int if_atom_parser(const struct ref_format *format, struct used_atom *ato return 0; } -static int head_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { atom->u.head = resolve_refdup("HEAD", RESOLVE_REF_READING, NULL, NULL); @@ -570,7 +570,7 @@ static struct { const char *name; info_source source; cmp_type cmp_type; - int (*parser)(const struct ref_format *format, struct used_atom *atom, + int (*parser)(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err); } valid_atom[] = { [ATOM_REFNAME] = { "refname", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -649,7 +649,7 @@ struct atom_value { /* * Used to parse format string and sort specifiers */ -static int parse_ref_filter_atom(const struct ref_format *format, +static int parse_ref_filter_atom(struct ref_format *format, const char *atom, const char *ep, struct strbuf *err) { @@ -2554,9 +2554,9 @@ static void append_literal(const char *cp, const char *ep, struct ref_formatting } int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, - struct strbuf *final_buf, - struct strbuf *error_buf) + struct ref_format *format, + struct strbuf *final_buf, + struct strbuf *error_buf) { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; @@ -2601,7 +2601,7 @@ int format_ref_array_item(struct ref_array_item *info, } void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format) + struct ref_format *format) { struct ref_array_item *ref_item; struct strbuf output = STRBUF_INIT; diff --git a/ref-filter.h b/ref-filter.h index baf72a71896..74fb423fc89 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -116,7 +116,7 @@ void ref_array_sort(struct ref_sorting *sort, struct ref_array *array); void ref_sorting_set_sort_flags_all(struct ref_sorting *sorting, unsigned int mask, int on); /* Based on the given format and quote_style, fill the strbuf */ int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, + struct ref_format *format, struct strbuf *final_buf, struct strbuf *error_buf); /* Parse a single sort specifier and add it to the list */ @@ -137,7 +137,7 @@ void setup_ref_filter_porcelain_msg(void); * name must be a fully qualified refname. */ void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format); + struct ref_format *format); /* * Push a single ref onto the array; this can be used to construct your own From patchwork Thu Jul 15 15:40:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07342C47E4B for ; Thu, 15 Jul 2021 15:40:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E6B60613E2 for ; Thu, 15 Jul 2021 15:40:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239230AbhGOPnb (ORCPT ); Thu, 15 Jul 2021 11:43:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239133AbhGOPn0 (ORCPT ); Thu, 15 Jul 2021 11:43:26 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B75F4C061762 for ; Thu, 15 Jul 2021 08:40:32 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id i94so8377779wri.4 for ; Thu, 15 Jul 2021 08:40:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Vxb3zASL0DKk506JyjXBLgEAoDw46+tSVzIvEoLLdas=; b=T3nV0v6tu4O9SEIl8rhq+jOaNLksH0vyylyHN4zd398cXaxNOeAKiHw1TZBzo7EiNf HYHBGxqRZ9ioIB8qMnhBed0TPjIBAmonW02SMyNKBwgSP3H/HQp5GGsOrqvM+FMm6r92 o9xXw90IE4KbnZPEfgheki/VcHB//POjCEwGCnB2GGeOirdf37Ll+dzMta7bIDuXtvP1 /R0kXhLW60I+A03M/I2mcfc2m8NqOtSoOsVSpjdhwBGTw0yccWd6zeNmseeLaE1321s1 09NSpPay8E4rpPTWejiMx0bc0Nic5TAAm08VeL8GBEobzyT0nAQDCYxb0oCUZzBvTKTh e14Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Vxb3zASL0DKk506JyjXBLgEAoDw46+tSVzIvEoLLdas=; b=n3QhpROFgvqKK90pXskoj3elmJUcFufM4uTqCFxK2rvEjaM3T361qj9qyq68L8cSFX icBq+OWyPcdKDMpv5TzxbmNUm7tT9MOpTe5ahogNbKzPbx2ydVoIastVFpMlGE+z2KuA hLmVXDzgmL/XN63Rz9A9HRhly1dvg1WPJABuuGcsX6+oyq2/xqqKNoaq2Ldr+WNx9tFf QnXDaVmoHAUTzq/pPaogxE4xLe0dxGglSajxF0CvIpGPlv4JIfvD+gkpZ2f47Qr5QOAm z32stMbrwgCGJ7B7roAa15a7u6fcTS7Hvpuq0LlBBS4tJPusZaS41poNzcstk4USVsmh xH8A== X-Gm-Message-State: AOAM531N1tehZzG6/yWjO2zMMVgLem4pB/mRHUlFhZXtCItvnpu58t1D HOfDmBtPP1d5U3A356RbnjgNTCChy8k= X-Google-Smtp-Source: ABdhPJwSuNHb6ZtUoYrRy1AIhqEs4EUWQUZwfuvhHNcFgFrxGB6nuW51qP/BUZci3Ch5pQOxDgaGkQ== X-Received: by 2002:adf:b307:: with SMTP id j7mr6445698wrd.243.1626363631413; Thu, 15 Jul 2021 08:40:31 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q19sm8297486wmq.38.2021.07.15.08.40.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:31 -0700 (PDT) Message-Id: <6fb9cfdeab1910f7fc2faf2ba6b34efea4399a85.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:13 +0000 Subject: [PATCH v2 05/17] [GSOC] ref-filter: add %(rest) atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let "cat-file --batch=%(rest)" use the ref-filter interface, add %(rest) atom for ref-filter. Introduce the reject_atom() to reject the atom %(rest) for "git for-each-ref", "git branch", "git tag" and "git verify-tag". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 25 +++++++++++++++++++++++++ ref-filter.h | 5 ++++- t/t3203-branch-output.sh | 4 ++++ t/t6300-for-each-ref.sh | 4 ++++ t/t7004-tag.sh | 4 ++++ t/t7030-verify-tag.sh | 4 ++++ 6 files changed, 45 insertions(+), 1 deletion(-) diff --git a/ref-filter.c b/ref-filter.c index c8e561a3687..af8216dcd5b 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -157,6 +157,7 @@ enum atom_type { ATOM_IF, ATOM_THEN, ATOM_ELSE, + ATOM_REST, }; /* @@ -559,6 +560,15 @@ static int if_atom_parser(struct ref_format *format, struct used_atom *atom, return 0; } +static int rest_atom_parser(struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (arg) + return strbuf_addf_ret(err, -1, _("%%(rest) does not take arguments")); + format->use_rest = 1; + return 0; +} + static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { @@ -615,6 +625,7 @@ static struct { [ATOM_IF] = { "if", SOURCE_NONE, FIELD_STR, if_atom_parser }, [ATOM_THEN] = { "then", SOURCE_NONE }, [ATOM_ELSE] = { "else", SOURCE_NONE }, + [ATOM_REST] = { "rest", SOURCE_NONE, FIELD_STR, rest_atom_parser }, /* * Please update $__git_ref_fieldlist in git-completion.bash * when you add new atoms @@ -989,6 +1000,11 @@ static const char *find_next(const char *cp) return NULL; } +static int reject_atom(enum atom_type atom_type) +{ + return atom_type == ATOM_REST; +} + /* * Make sure the format string is well formed, and parse out * the used atoms. @@ -1009,6 +1025,8 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (reject_atom(used_atom[at].atom_type)) + die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || @@ -1928,6 +1946,12 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->handler = else_atom_handler; v->s = xstrdup(""); continue; + } else if (atom_type == ATOM_REST) { + if (ref->rest) + v->s = xstrdup(ref->rest); + else + v->s = xstrdup(""); + continue; } else continue; @@ -2145,6 +2169,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); + ref->rest = NULL; return ref; } diff --git a/ref-filter.h b/ref-filter.h index 74fb423fc89..c15dee8d6b9 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -38,6 +38,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; + const char *rest; int flag; unsigned int kind; const char *symref; @@ -76,14 +77,16 @@ struct ref_format { * verify_ref_format() afterwards to finalize. */ const char *format; + const char *rest; int quote_style; + int use_rest; int use_color; /* Internal state to ref-filter */ int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { NULL, 0, -1 } +#define REF_FORMAT_INIT { .use_color = -1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ diff --git a/t/t3203-branch-output.sh b/t/t3203-branch-output.sh index 5325b9f67a0..6e94c6db7b5 100755 --- a/t/t3203-branch-output.sh +++ b/t/t3203-branch-output.sh @@ -340,6 +340,10 @@ test_expect_success 'git branch --format option' ' test_cmp expect actual ' +test_expect_success 'git branch with --format=%(rest) must fail' ' + test_must_fail git branch --format="%(rest)" >actual +' + test_expect_success 'worktree colors correct' ' cat >expect <<-EOF && * (HEAD detached from fromtag) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 3d15d0a5360..0d2e062f791 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -1211,6 +1211,10 @@ test_expect_success 'basic atom: head contents:trailers' ' test_cmp expect actual.clean ' +test_expect_success 'basic atom: rest must fail' ' + test_must_fail git for-each-ref --format="%(rest)" refs/heads/main +' + test_expect_success 'trailer parsing not fooled by --- line' ' git commit --allow-empty -F - <<-\EOF && this is the subject diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh index 2f72c5c6883..082be85dffc 100755 --- a/t/t7004-tag.sh +++ b/t/t7004-tag.sh @@ -1998,6 +1998,10 @@ test_expect_success '--format should list tags as per format given' ' test_cmp expect actual ' +test_expect_success 'git tag -l with --format="%(rest)" must fail' ' + test_must_fail git tag -l --format="%(rest)" "v1*" +' + test_expect_success "set up color tests" ' echo "v1.0" >expect.color && echo "v1.0" >expect.bare && diff --git a/t/t7030-verify-tag.sh b/t/t7030-verify-tag.sh index 3cefde9602b..10faa645157 100755 --- a/t/t7030-verify-tag.sh +++ b/t/t7030-verify-tag.sh @@ -194,6 +194,10 @@ test_expect_success GPG 'verifying tag with --format' ' test_cmp expect actual ' +test_expect_success GPG 'verifying tag with --format="%(rest)" must fail' ' + test_must_fail git verify-tag --format="%(rest)" "fourth-signed" +' + test_expect_success GPG 'verifying a forged tag with --format should fail silently' ' test_must_fail git verify-tag --format="tagname : %(tag)" $(cat forged1.tag) >actual-forged && test_must_be_empty actual-forged From patchwork Thu Jul 15 15:40:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34D39C47E48 for ; Thu, 15 Jul 2021 15:40:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 205EC613E0 for ; Thu, 15 Jul 2021 15:40:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239206AbhGOPna (ORCPT ); Thu, 15 Jul 2021 11:43:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239131AbhGOPn0 (ORCPT ); Thu, 15 Jul 2021 11:43:26 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5427AC061764 for ; Thu, 15 Jul 2021 08:40:33 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id f9so8339355wrq.11 for ; Thu, 15 Jul 2021 08:40:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=W8ooZCt6Qfgnwu0vRHhx5yRmsci71JFTX9Sq2mPY6j0=; b=PZBtrSTGUpQ+RW4ex3S6q7CyP7kvt8x9dAYau80gARBGuskHptz4HYOkWAEdNjmpRp oxtCYVlFqNDKGuF+eweVX25uUlP3zGZX/mhqfO4GGZ3rIpTHHr8TVY/BreE3IxMU73GI vIqLNrJ4XOeAwJpslAeNZT+6tVLz+Wh1dQYzpF+TXzT8uRc1SjEAgVODUTd8qNlEL+AF MYsIS2bXg5rODSDWOZGX5vaG1etTxlsuocvBE6r4WYQIweeD+xYEmb+q9Fs76jdVfC2c kNpSJasYZ9cq3iVQ50gs6m3NkU5ZkReV0vRGH3EUpO6ysFjcpIrvp5Gw1/IwaNXMYrf7 lyiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=W8ooZCt6Qfgnwu0vRHhx5yRmsci71JFTX9Sq2mPY6j0=; b=EjyO2sAx4lihuSAa1TaYsXGjeJLLBGGUEcd+zFGO9jfZ5nZLd79Ol4B7GKhQls/A4Q 6OOxgncK4UmcWbP1XmOp/AE03fAUxdejuqiTrW3rs0RhKSEF1U55MVh+8Z72feTdF/6w zJ6daAGM3EXSLkWh3s2/O1YhNTAtlFZAQ0/nJkrVzYAXMXMiuh0VNM3Pb7S2Efs+AqmO HPvHVj7ADkeEwSCcgLOf+/ipSa+VUG90//ib4p/nt438HzR8n5EHLyOL8p0lt4kRsJdw 92uaiCx3DPbzeCmxhUdgNlorpIUp6Q/MbPvttX3byr/BxwU0C/rQd/mGvKbDsTkmWrRC wfzA== X-Gm-Message-State: AOAM530/k1Ax8JayWZovKnNZQ6RolqxpkWvivQ//EEbp/zyBTqUD9HGC FA82wONXTdRYQo0Czwfr579wKJPzJLg= X-Google-Smtp-Source: ABdhPJxZH5pn0mZvnfDMffNoQ1ddpTIOXaHb4M7vQVAOkGL7SeEmKhJKScqOU3JDR8ogEjxELXsphA== X-Received: by 2002:a5d:53c5:: with SMTP id a5mr6508228wrw.15.1626363631983; Thu, 15 Jul 2021 08:40:31 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s13sm6777792wrm.13.2021.07.15.08.40.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:31 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:14 +0000 Subject: [PATCH v2 06/17] [GSOC] ref-filter: pass get_object() return value to their callers MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because in the refactor of `git cat-file --batch` later, oid_object_info_extended() in get_object() will be used to obtain the info of an object with it's oid. When the object cannot be obtained in the git repository, `cat-file --batch` expects to output " missing" and continue the next oid query instead of letting Git exit. In other error conditions, Git should exit normally. So we can achieve this function by passing the return value of get_object(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- ref-filter.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index af8216dcd5b..cfcea0e507e 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1821,6 +1821,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) { struct object *obj; int i; + int ret; struct object_info empty = OBJECT_INFO_INIT; CALLOC_ARRAY(ref->value, used_atom_cnt); @@ -1977,8 +1978,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) oi.oid = ref->objectname; - if (get_object(ref, 0, &obj, &oi, err)) - return -1; + ret = get_object(ref, 0, &obj, &oi, err); + if (ret) + return ret; /* * If there is no atom that wants to know about tagged @@ -2010,8 +2012,10 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom, struct atom_value **v, struct strbuf *err) { if (!ref->value) { - if (populate_value(ref, err)) - return -1; + int ret = populate_value(ref, err); + + if (ret) + return ret; fill_missing_values(ref->value); } *v = &ref->value[atom]; @@ -2585,6 +2589,7 @@ int format_ref_array_item(struct ref_array_item *info, { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; + int ret; state.quote_style = format->quote_style; push_stack_element(&state.stack); @@ -2597,10 +2602,10 @@ int format_ref_array_item(struct ref_array_item *info, if (cp < sp) append_literal(cp, sp, &state); pos = parse_ref_filter_atom(format, sp + 2, ep, error_buf); - if (pos < 0 || get_ref_atom_value(info, pos, &atomv, error_buf) || + if (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) || atomv->handler(atomv, &state, error_buf)) { pop_stack_element(&state.stack); - return -1; + return ret ? ret : -1; } } if (*cp) { From patchwork Thu Jul 15 15:40:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3164C47E4E for ; Thu, 15 Jul 2021 15:40:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A6D0D61396 for ; Thu, 15 Jul 2021 15:40:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239136AbhGOPnb (ORCPT ); Thu, 15 Jul 2021 11:43:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239150AbhGOPn2 (ORCPT ); Thu, 15 Jul 2021 11:43:28 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3662C061765 for ; Thu, 15 Jul 2021 08:40:33 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id l17-20020a05600c1d11b029021f84fcaf75so6337049wms.1 for ; Thu, 15 Jul 2021 08:40:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=e0jaRxoSDbvjSbYc4km+v8HozV2Lm56nE1KTtjAQGpk=; b=MujnGlcX2peD9/ZzdGFGh5em+YdMgvI856ww8BKpYW4AziLLvFaVHj+cFxzmllnnpP vB8cilujQU8F2CLcX/D/YiZFvt868REY34+efV3x0XSRC9SggS+eVj7qv4trJdkh8Vh2 bzxBRakzuX4uuQphQYAxz5jc7vvPo7yYTR7HTsOEKNHOiyFntjJLW7fo9RPrNhoxdFW9 Y7EfP23pchTeOngKy1UKKLwojFxDk9o0NQ13KnkIJ8lZfC7+/K2CdW6qedJESF9RlvdA T48tNnME5jWPBGczcpZuhvSuahBr5GUJyAUAGm9o92s459HzrK2AEyJcCiiSW0rqUMZU VYfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=e0jaRxoSDbvjSbYc4km+v8HozV2Lm56nE1KTtjAQGpk=; b=d91Lyu4MmvagwXIncqXmUjQ1UZu3knuLFn35/gOc0BI1TIWaGX7JgcNkeNPpEXsyy7 DDOJ+/24n1ktv4LhNYLxuEWb8jVohgCg9BOqXOoUCH2kV9J1q0Roic2hfxTanRy1+Vev lW2zcI0FVc1XodtZU1r7ybsF/6hfDGS+NSkqZyxYEBAeo5QxUV8LtblEjBEb3Cg0BtG0 l64Vht9tGI031Gui1w+IsFSrp/pBuyxkU9lysr+A2Pnbjr0nufFMV7d3JYc7AQ5be2ZM 4SY508dfJcBbqqTcf050aanK0S72Tiss0/IVYMMPSsiF3whiT8hliiG/UYRTjCySzJO7 HJsg== X-Gm-Message-State: AOAM533JFTrvaX8rbYsOeo/o4dLjuOHTx6Wd+r+XZp5tL/I9m/XLGl9n 8WUAjr/mL8VLh+mIN1R/9eU39zGeJ5w= X-Google-Smtp-Source: ABdhPJyrPlScV4qZVxwi9pWo2rf3afIRAdhrlx1/LN34bl+oHDP2f45PKiYdK3FjROUe/Nzwv5SrOw== X-Received: by 2002:a05:600c:4304:: with SMTP id p4mr491732wme.93.1626363632583; Thu, 15 Jul 2021 08:40:32 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k13sm7359760wrp.34.2021.07.15.08.40.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:32 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:15 +0000 Subject: [PATCH v2 07/17] [GSOC] ref-filter: introduce free_ref_array_item_value() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu When we use ref_array_item which is not dynamically allocated and want to free the space of its member "value" after the end of use, free_array_item() does not meet our needs, because it tries to free ref_array_item itself and its member "symref". Introduce free_ref_array_item_value() for freeing ref_array_item value. It will be called internally by free_array_item(), and it will help `cat-file --batch` free ref_array_item's value memory later. Helped-by: Junio C Hamano Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 ++++++++--- ref-filter.h | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index cfcea0e507e..d70b295672f 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2296,16 +2296,21 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, return 0; } -/* Free memory allocated for a ref_array_item */ -static void free_array_item(struct ref_array_item *item) +void free_ref_array_item_value(struct ref_array_item *item) { - free((char *)item->symref); if (item->value) { int i; for (i = 0; i < used_atom_cnt; i++) free((char *)item->value[i].s); free(item->value); } +} + +/* Free memory allocated for a ref_array_item */ +static void free_array_item(struct ref_array_item *item) +{ + free((char *)item->symref); + free_ref_array_item_value(item); free(item); } diff --git a/ref-filter.h b/ref-filter.h index c15dee8d6b9..44e6dc05ac2 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -111,6 +111,8 @@ struct ref_format { int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type); /* Clear all memory allocated to ref_array */ void ref_array_clear(struct ref_array *array); +/* Free ref_array_item's value */ +void free_ref_array_item_value(struct ref_array_item *item); /* Used to verify if the given format is correct and to parse out the used atoms */ int verify_ref_format(struct ref_format *format); /* Sort the given ref_array as per the ref_sorting provided */ From patchwork Thu Jul 15 15:40:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 547AEC07E96 for ; Thu, 15 Jul 2021 15:40:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3D29961374 for ; Thu, 15 Jul 2021 15:40:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239361AbhGOPng (ORCPT ); Thu, 15 Jul 2021 11:43:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239169AbhGOPn3 (ORCPT ); Thu, 15 Jul 2021 11:43:29 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2258C061767 for ; Thu, 15 Jul 2021 08:40:34 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id a5-20020a7bc1c50000b02901e3bbe0939bso3965660wmj.0 for ; Thu, 15 Jul 2021 08:40:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=/eivstvOXJvKVWdkJhnzJmSkxFiMa1WMQVdYhPN5R2Y=; b=s455+Cvg6pXQ2yTGO4z8F1zLfxn1sO2l1KoKbv3fn2hl1WVFM0pNZvbo4KCmWLIVbV BjytePBBeF9YH+L2fdbnAFvCG4xFD5wW2+Q1NgYZzpsyjKRjUQfvf7uBWS5KJsgyVSm6 bT/p9XgPxTrO1QdQG7a+MdZiRULdh+R3NDD3j91gBAAVvQK6Du33hvSv+195+U0QnAFW osqoURK3RUgf0dqEWgUAkEvBH0WpBSC89msrD7c5NqBtGhHhH9gJjjtGkGKcDNU/gm3u 6GmcLZlr7qEQ/kbE1lN23azH7V/mjOVl0LTnamWWrm0TI8hwRVwmXieFhth7N9fitJvS L1Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=/eivstvOXJvKVWdkJhnzJmSkxFiMa1WMQVdYhPN5R2Y=; b=E9cYexBpOoyS0m/I6rL8lNL6uByvjcj1DBVj2PY7I9TOOUGaQpCKBWhDPzEFfQ2ZrW LfzJZkDIgqXFrU4uMyWscz/73cu/ugxaL/FVnJlHRpUxWEwtVlpRBBPVCmwc/PdxumyP Cv/cjgMbwMk2GsGQ7FuspWIYWHki5PWavlaTYmZy7a22d+/L+YviXkgXF0bAxLAiaMk9 O0vSH5OjbLluo10TSrdXZXrA64WBBCH1Gazld5YeaaXU7TjkU5OoGlQIEQfL+KS01nAU Yz/hSqfxY8LfzASjDjwXTt1nuKnlFGgNW3RBU3QIDmzOo3njrgrQbPNGPuSxNyOP/0XJ /jSA== X-Gm-Message-State: AOAM533an6mqc0vlVaV7sMgkx8xStSA98Voe981KLO3KLRo6thFGxUWq R+eh08ttua94mMfeopJvAydXhPGJOu8= X-Google-Smtp-Source: ABdhPJzOVfOZXu1KTNbyFjFjqZzoQ69hJ5szfMRyBQjHP0x3eK3POJWd31mUOLiABQ8Gc/19Wr/r7g== X-Received: by 2002:a1c:48a:: with SMTP id 132mr11296857wme.157.1626363633192; Thu, 15 Jul 2021 08:40:33 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l14sm6709770wrs.22.2021.07.15.08.40.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:32 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:16 +0000 Subject: [PATCH v2 08/17] [GSOC] ref-filter: add cat_file_mode to ref_format MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add `cat_file_mode` member to struct `ref_format`. When `cat-file --batch` use ref-filter logic later, it can help us reject atoms in verify_ref_format() which cat-file cannot use, e.g. `%(refname)`, `%(push)`, `%(upstream)"... Add batch_test_atom() to t/t1006-cat-file.sh and add check for cat-file --batch, this can help us clearly show which atoms cat-file accepts and which atoms it rejects. Helped-by: Eric Sunshine Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Christian Couder Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 23 ++++- ref-filter.h | 1 + t/t1006-cat-file.sh | 226 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 246 insertions(+), 4 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index d70b295672f..27199ba40f5 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1000,9 +1000,24 @@ static const char *find_next(const char *cp) return NULL; } -static int reject_atom(enum atom_type atom_type) -{ - return atom_type == ATOM_REST; +static int reject_atom(int cat_file_mode, enum atom_type atom_type) +{ + if (!cat_file_mode) + return atom_type == ATOM_REST; + + /* cat_file_mode */ + switch (atom_type) { + case ATOM_FLAG: + case ATOM_HEAD: + case ATOM_PUSH: + case ATOM_REFNAME: + case ATOM_SYMREF: + case ATOM_UPSTREAM: + case ATOM_WORKTREEPATH: + return 1; + default: + return 0; + } } /* @@ -1025,7 +1040,7 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - if (reject_atom(used_atom[at].atom_type)) + if (reject_atom(format->cat_file_mode, used_atom[at].atom_type)) die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); if ((format->quote_style == QUOTE_PYTHON || diff --git a/ref-filter.h b/ref-filter.h index 44e6dc05ac2..053980a6a42 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -78,6 +78,7 @@ struct ref_format { */ const char *format; const char *rest; + int cat_file_mode; int quote_style; int use_rest; int use_color; diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 18b3779ccb6..95d760652eb 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -608,4 +608,230 @@ test_expect_success 'cat-file --batch="batman" with --batch-all-objects will wor cmp expect actual ' +. "$TEST_DIRECTORY"/lib-gpg.sh +. "$TEST_DIRECTORY"/lib-terminal.sh + +test_expect_success 'cat-file --batch|--batch-check setup' ' + echo 1>blob1 && + printf "a\0b\0\c" >blob2 && + git add blob1 blob2 && + git commit -m "Commit Message" && + git branch -M main && + git tag -a -m "v0.0.0" testtag && + git update-ref refs/myblobs/blob1 HEAD:blob1 && + git update-ref refs/myblobs/blob2 HEAD:blob2 && + git update-ref refs/mytrees/tree1 HEAD^{tree} +' + +batch_test_atom() { + if test "$3" = "fail" + then + test_expect_${4:-success} $PREREQ "basic atom: $1 $2 must fail" " + test_must_fail git cat-file --batch-check='$2' >bad <<-EOF + $1 + EOF + " + else + test_expect_${4:-success} $PREREQ "basic atom: $1 $2" " + git for-each-ref --format='$2' $1 >expected && + git cat-file --batch-check='$2' >actual <<-EOF && + $1 + EOF + sanitize_pgp actual.clean && + cmp expected actual.clean + " + fi +} + +batch_test_atom refs/heads/main '%(refname)' fail +batch_test_atom refs/heads/main '%(refname:)' fail +batch_test_atom refs/heads/main '%(refname:short)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:strip=1)' fail +batch_test_atom refs/heads/main '%(refname:strip=2)' fail +batch_test_atom refs/heads/main '%(refname:strip=-1)' fail +batch_test_atom refs/heads/main '%(refname:strip=-2)' fail +batch_test_atom refs/heads/main '%(upstream)' fail +batch_test_atom refs/heads/main '%(upstream:short)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=-2)' fail +batch_test_atom refs/heads/main '%(push)' fail +batch_test_atom refs/heads/main '%(push:short)' fail +batch_test_atom refs/heads/main '%(push:lstrip=1)' fail +batch_test_atom refs/heads/main '%(push:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:strip=1)' fail +batch_test_atom refs/heads/main '%(push:strip=-1)' fail +batch_test_atom refs/heads/main '%(objecttype)' +batch_test_atom refs/heads/main '%(objectsize)' +batch_test_atom refs/heads/main '%(objectsize:disk)' +batch_test_atom refs/heads/main '%(deltabase)' +batch_test_atom refs/heads/main '%(objectname)' +batch_test_atom refs/heads/main '%(objectname:short)' fail +batch_test_atom refs/heads/main '%(objectname:short=1)' fail +batch_test_atom refs/heads/main '%(objectname:short=10)' fail +batch_test_atom refs/heads/main '%(tree)' fail +batch_test_atom refs/heads/main '%(tree:short)' fail +batch_test_atom refs/heads/main '%(tree:short=1)' fail +batch_test_atom refs/heads/main '%(tree:short=10)' fail +batch_test_atom refs/heads/main '%(parent)' fail +batch_test_atom refs/heads/main '%(parent:short)' fail +batch_test_atom refs/heads/main '%(parent:short=1)' fail +batch_test_atom refs/heads/main '%(parent:short=10)' fail +batch_test_atom refs/heads/main '%(numparent)' fail +batch_test_atom refs/heads/main '%(object)' fail +batch_test_atom refs/heads/main '%(type)' fail +batch_test_atom refs/heads/main '%(raw)' fail +batch_test_atom refs/heads/main '%(*objectname)' fail +batch_test_atom refs/heads/main '%(*objecttype)' fail +batch_test_atom refs/heads/main '%(author)' fail +batch_test_atom refs/heads/main '%(authorname)' fail +batch_test_atom refs/heads/main '%(authoremail)' fail +batch_test_atom refs/heads/main '%(authoremail:trim)' fail +batch_test_atom refs/heads/main '%(authoremail:localpart)' fail +batch_test_atom refs/heads/main '%(authordate)' fail +batch_test_atom refs/heads/main '%(committer)' fail +batch_test_atom refs/heads/main '%(committername)' fail +batch_test_atom refs/heads/main '%(committeremail)' fail +batch_test_atom refs/heads/main '%(committeremail:trim)' fail +batch_test_atom refs/heads/main '%(committeremail:localpart)' fail +batch_test_atom refs/heads/main '%(committerdate)' fail +batch_test_atom refs/heads/main '%(tag)' fail +batch_test_atom refs/heads/main '%(tagger)' fail +batch_test_atom refs/heads/main '%(taggername)' fail +batch_test_atom refs/heads/main '%(taggeremail)' fail +batch_test_atom refs/heads/main '%(taggeremail:trim)' fail +batch_test_atom refs/heads/main '%(taggeremail:localpart)' fail +batch_test_atom refs/heads/main '%(taggerdate)' fail +batch_test_atom refs/heads/main '%(creator)' fail +batch_test_atom refs/heads/main '%(creatordate)' fail +batch_test_atom refs/heads/main '%(subject)' fail +batch_test_atom refs/heads/main '%(subject:sanitize)' fail +batch_test_atom refs/heads/main '%(contents:subject)' fail +batch_test_atom refs/heads/main '%(body)' fail +batch_test_atom refs/heads/main '%(contents:body)' fail +batch_test_atom refs/heads/main '%(contents:signature)' fail +batch_test_atom refs/heads/main '%(contents)' fail +batch_test_atom refs/heads/main '%(HEAD)' fail +batch_test_atom refs/heads/main '%(upstream:track)' fail +batch_test_atom refs/heads/main '%(upstream:trackshort)' fail +batch_test_atom refs/heads/main '%(upstream:track,nobracket)' fail +batch_test_atom refs/heads/main '%(upstream:nobracket,track)' fail +batch_test_atom refs/heads/main '%(push:track)' fail +batch_test_atom refs/heads/main '%(push:trackshort)' fail +batch_test_atom refs/heads/main '%(worktreepath)' fail +batch_test_atom refs/heads/main '%(symref)' fail +batch_test_atom refs/heads/main '%(flag)' fail + +batch_test_atom refs/tags/testtag '%(refname)' fail +batch_test_atom refs/tags/testtag '%(refname:short)' fail +batch_test_atom refs/tags/testtag '%(upstream)' fail +batch_test_atom refs/tags/testtag '%(push)' fail +batch_test_atom refs/tags/testtag '%(objecttype)' +batch_test_atom refs/tags/testtag '%(objectsize)' +batch_test_atom refs/tags/testtag '%(objectsize:disk)' +batch_test_atom refs/tags/testtag '%(*objectsize:disk)' fail +batch_test_atom refs/tags/testtag '%(deltabase)' +batch_test_atom refs/tags/testtag '%(*deltabase)' fail +batch_test_atom refs/tags/testtag '%(objectname)' +batch_test_atom refs/tags/testtag '%(objectname:short)' fail +batch_test_atom refs/tags/testtag '%(tree)' fail +batch_test_atom refs/tags/testtag '%(tree:short)' fail +batch_test_atom refs/tags/testtag '%(tree:short=1)' fail +batch_test_atom refs/tags/testtag '%(tree:short=10)' fail +batch_test_atom refs/tags/testtag '%(parent)' fail +batch_test_atom refs/tags/testtag '%(parent:short)' fail +batch_test_atom refs/tags/testtag '%(parent:short=1)' fail +batch_test_atom refs/tags/testtag '%(parent:short=10)' fail +batch_test_atom refs/tags/testtag '%(numparent)' fail +batch_test_atom refs/tags/testtag '%(object)' fail +batch_test_atom refs/tags/testtag '%(type)' fail +batch_test_atom refs/tags/testtag '%(*objectname)' fail +batch_test_atom refs/tags/testtag '%(*objecttype)' fail +batch_test_atom refs/tags/testtag '%(author)' fail +batch_test_atom refs/tags/testtag '%(authorname)' fail +batch_test_atom refs/tags/testtag '%(authoremail)' fail +batch_test_atom refs/tags/testtag '%(authoremail:trim)' fail +batch_test_atom refs/tags/testtag '%(authoremail:localpart)' fail +batch_test_atom refs/tags/testtag '%(authordate)' fail +batch_test_atom refs/tags/testtag '%(committer)' fail +batch_test_atom refs/tags/testtag '%(committername)' fail +batch_test_atom refs/tags/testtag '%(committeremail)' fail +batch_test_atom refs/tags/testtag '%(committeremail:trim)' fail +batch_test_atom refs/tags/testtag '%(committeremail:localpart)' fail +batch_test_atom refs/tags/testtag '%(committerdate)' fail +batch_test_atom refs/tags/testtag '%(tag)' fail +batch_test_atom refs/tags/testtag '%(tagger)' fail +batch_test_atom refs/tags/testtag '%(taggername)' fail +batch_test_atom refs/tags/testtag '%(taggeremail)' fail +batch_test_atom refs/tags/testtag '%(taggeremail:trim)' fail +batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' fail +batch_test_atom refs/tags/testtag '%(taggerdate)' fail +batch_test_atom refs/tags/testtag '%(creator)' fail +batch_test_atom refs/tags/testtag '%(creatordate)' fail +batch_test_atom refs/tags/testtag '%(subject)' fail +batch_test_atom refs/tags/testtag '%(subject:sanitize)' fail +batch_test_atom refs/tags/testtag '%(contents:subject)' fail +batch_test_atom refs/tags/testtag '%(body)' fail +batch_test_atom refs/tags/testtag '%(contents:body)' fail +batch_test_atom refs/tags/testtag '%(contents:signature)' fail +batch_test_atom refs/tags/testtag '%(contents)' fail +batch_test_atom refs/tags/testtag '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(refname)' fail +batch_test_atom refs/myblobs/blob1 '%(upstream)' fail +batch_test_atom refs/myblobs/blob1 '%(push)' fail +batch_test_atom refs/myblobs/blob1 '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(objectname)' +batch_test_atom refs/myblobs/blob1 '%(objecttype)' +batch_test_atom refs/myblobs/blob1 '%(objectsize)' +batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)' +batch_test_atom refs/myblobs/blob1 '%(deltabase)' + +batch_test_atom refs/myblobs/blob1 '%(contents)' fail +batch_test_atom refs/myblobs/blob2 '%(contents)' fail + +batch_test_atom refs/myblobs/blob1 '%(raw)' fail +batch_test_atom refs/myblobs/blob2 '%(raw)' fail +batch_test_atom refs/mytrees/tree1 '%(raw)' fail + +batch_test_atom refs/myblobs/blob1 '%(raw:size)' fail +batch_test_atom refs/myblobs/blob2 '%(raw:size)' fail +batch_test_atom refs/mytrees/tree1 '%(raw:size)' fail + +batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail +batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail +batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' fail + +batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' fail +batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' fail +batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' fail +batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' fail + +batch_test_atom refs/heads/main 'VALID' +batch_test_atom refs/heads/main '%(INVALID)' fail +batch_test_atom refs/heads/main '%(authordate:INVALID)' fail + +batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize) +%(raw)' fail +batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize) +%(raw)' fail +batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize) +%(raw)' fail +batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize) +%(raw)' fail + test_done From patchwork Thu Jul 15 15:40:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAA3EC47E4D for ; Thu, 15 Jul 2021 15:40:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91040613DF for ; Thu, 15 Jul 2021 15:40:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239199AbhGOPnc (ORCPT ); Thu, 15 Jul 2021 11:43:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239179AbhGOPn3 (ORCPT ); Thu, 15 Jul 2021 11:43:29 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 570FEC06175F for ; Thu, 15 Jul 2021 08:40:35 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id l6so4032868wmq.0 for ; Thu, 15 Jul 2021 08:40:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=6x+Lq6UZlHCJIcz9CvulEshnfXho4geSxyabj2Ck1uw=; b=MVgReOLN9bG1yRdWWDm9s6P/9DhrDnSOSmCRmp44ZqeBXnp+nqtCBLSo3tD6U3B2bD NQsMw0Hnbbb+YykjBxnFkgxkBIaL2GKCVRMBhk81V7XUlNdVQDGsuR/aPRlFSEE99aAg qT4+MrrW63tkFnXES+h4+7B7yk2mTKZYrGwgKu9+AXvnN/1dU0MpnfMT8lO2sTDliDIN tyxJlfpkyBhRMwyzXL0AVV7fd4wdhVI6+IHb7ypDWXcvSIkINJL+j9z+hfPqf51PcvjA EJG3nwAcR2VE85TdYWY+5a976glwkkBZUugCpchcm9x/VgDPZJviC7K2B/6YTM0LxS5O g1SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=6x+Lq6UZlHCJIcz9CvulEshnfXho4geSxyabj2Ck1uw=; b=NTzgKuiVyfc+VwJxIgbENffssNFzt5+x5Au7qIxLpA+Ot2svKlYXbJFqzdPOvoWCC0 w8CJ003YV4vJ68/wT3+5BT+lR8237H//IPxcYSmVEVyD0SIYH0ee1+A3pZnze5eVZQMd Y79/SmfgS2Or+MAxvYJuIZ1niH2rYIoKCuUW5o+jHjpnMbRvoJHc2T2Uyncv2TRHs6ey xJ1ZbQjq3M+AFm92P6S8wcPgYVvNvIsBsHp1lHOrzIxqVlAnZPC6QXu4e7KKAzuZrvjE 2u0qoHKqwyipHBou6QJtHPXUknTOjJa/bwSbcnu2xJi6THkCNv6yQ5rbY1b9gPCr48U7 t+VA== X-Gm-Message-State: AOAM533HDriLe82vSaXjKsRKqyipktUD76cJYgKD2KPXr2IfUecdW5uh gGtw1By4nkw7GHYzBIaHcwGZsQANmec= X-Google-Smtp-Source: ABdhPJyKUcQ6ms/lo58JosZaiXK17wetOZQ0G/vTzNkPEDtK19Dphb300nmUEecEndDVKgfDhRE71A== X-Received: by 2002:a05:600c:25a:: with SMTP id 26mr11313182wmj.87.1626363633901; Thu, 15 Jul 2021 08:40:33 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l24sm8496067wmi.30.2021.07.15.08.40.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:33 -0700 (PDT) Message-Id: <7058f47d41d74ad874b9e74f469dc473fd0d3413.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:17 +0000 Subject: [PATCH v2 09/17] [GSOC] ref-filter: modify the error message and value in get_object Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Let get_object() return 1 and print " missing" instead of returning -1 and printing "missing object for " if oid_object_info_extended() unable to find the data corresponding to oid. When `cat-file --batch` use ref-filter logic later it can help `format_ref_array_item()` just report that the object is missing without letting Git exit. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 4 ++-- t/t6301-for-each-ref-errors.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 27199ba40f5..b4f41fec871 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1762,8 +1762,8 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) - return strbuf_addf_ret(err, -1, _("missing object %s for %s"), - oid_to_hex(&oi->oid), ref->refname); + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); if (oi->info.disk_sizep && oi->disk_size < 0) BUG("Object size is less than zero."); diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh index 40edf9dab53..3553f84a00c 100755 --- a/t/t6301-for-each-ref-errors.sh +++ b/t/t6301-for-each-ref-errors.sh @@ -41,7 +41,7 @@ test_expect_success 'Missing objects are reported correctly' ' r=refs/heads/missing && echo $MISSING >.git/$r && test_when_finished "rm -f .git/$r" && - echo "fatal: missing object $MISSING for $r" >missing-err && + echo "fatal: $MISSING missing" >missing-err && test_must_fail git for-each-ref 2>err && test_cmp missing-err err && ( From patchwork Thu Jul 15 15:40:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67E3EC47E48 for ; Thu, 15 Jul 2021 15:40:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 528DC613E9 for ; Thu, 15 Jul 2021 15:40:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239285AbhGOPnd (ORCPT ); Thu, 15 Jul 2021 11:43:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239180AbhGOPn3 (ORCPT ); Thu, 15 Jul 2021 11:43:29 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4C91C061762 for ; Thu, 15 Jul 2021 08:40:35 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id r11so8352522wro.9 for ; Thu, 15 Jul 2021 08:40:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QNasbiGAZli473fXu4QgD2u/UTIEGvkgntmX6viQd9U=; b=puplOe3BsQJrK0XKegkTE4nSWRPQ9CdqUIhJQsmctXQWJrCY8P5yVvDrMK2kYvTN1M tX4nxy+hcsGdnQIF1G++BJLDFZfH/TmZTcC9YG212UgXTb6/ZOs0o54I0IONydtM/Ha2 xUCAdKNb6r240bVBoTqF0q2Sz3U+eRsl0LXEi3Oan3cjskBIOE+mRiALUjJVuaXivnDd E97ngR859W7n7r5n4JKKqMVYvqn69RQkyB9Ykm1CxHTI11XA/uGF4YKZmnr6YV93WvJr PHC3Zy/GoLj9tiremKJOWKnTW8OqVDZc27TnihwX9OS9On093EJCEzpknAkVePHqfXMf Jj3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QNasbiGAZli473fXu4QgD2u/UTIEGvkgntmX6viQd9U=; b=cyC3M+x+L/Q+O32NDkjPaj+xxtMK9BqxNoVlC7HVXoijfiVFtzt8l61d4cH5qMp88R WO1Wa6Nd3bwZBie0+qDzgfvDQOmpQuAJGmHGLO9cO3HLHJG+U7hiReshtR7f2pI3wAOl Vk212hTj99o20Xq4qbClAne/KL9bmGFR+TmFZRLgcTtieswfI5DrE9bfQXnbbFXGLjaS 4i749sztSgf1GSL9hlhkcAa+ne+jIV6p04qA/bOSisMlzkaH/i63enQvEUGPcldFVosN vOefZ+xShoIVzxHZeseWbHVhn4vtW79aJ1Yrpf6v5g2a2KP7c+ZkHBJNKJG+DdH+FZB/ wPwQ== X-Gm-Message-State: AOAM531BxRhrvee70I3drRWdNpymC4Kr/hmQzEdLx6eDXJCb+7+a7fr2 xCtxwuPHR0ieloJpU2fchs/mt0/ZNMU= X-Google-Smtp-Source: ABdhPJxSBfFcaSyo9Tdu6Lwz2nIRFyBFcdXJo4OGjHmp68GSkYvQNTfLTzfoJvyZKNCp6Psj9nkx6g== X-Received: by 2002:adf:e749:: with SMTP id c9mr6492567wrn.407.1626363634494; Thu, 15 Jul 2021 08:40:34 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j12sm7168189wrq.83.2021.07.15.08.40.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:34 -0700 (PDT) Message-Id: <3af0def894c44cecd9f621daf8708a232bc0a143.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:18 +0000 Subject: [PATCH v2 10/17] [GSOC] cat-file: add has_object_file() check Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use `has_object_file()` in `batch_one_object()` to check whether the input object exists. This can help us reject the missing oid when we let `cat-file --batch` use ref-filter logic later. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 243fe6844bc..59a86412fd0 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -428,6 +428,13 @@ static void batch_one_object(const char *obj_name, return; } + if (!has_object_file(&data->oid)) { + printf("%s missing\n", + obj_name ? obj_name : oid_to_hex(&data->oid)); + fflush(stdout); + return; + } + batch_object_write(obj_name, scratch, opt, data); } From patchwork Thu Jul 15 15:40:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43CE3C1B08C for ; Thu, 15 Jul 2021 15:40:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3134061374 for ; Thu, 15 Jul 2021 15:40:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239289AbhGOPne (ORCPT ); Thu, 15 Jul 2021 11:43:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239190AbhGOPna (ORCPT ); Thu, 15 Jul 2021 11:43:30 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7715CC061765 for ; Thu, 15 Jul 2021 08:40:36 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id b14-20020a1c1b0e0000b02901fc3a62af78so6325860wmb.3 for ; Thu, 15 Jul 2021 08:40:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=wxw28PT0y1HD093q1CN4MSHFbLeN7akEybJmased0TU=; b=n/S9bfuPkifb98E/49SW6byugHahOxjzfXEzebfN6ZFFtHkbPhO7AzCqtdvOZ4xi8W NMY2HZgC8bbmcTJ4ttbveUaKJVvtrW2LwP6iZDsplTFmBLmF1yG7P4w09PWUDb+TBylR t4jdobny1zVVYw/mkQDhkACwcDtCKB8r56frdK2yKxDcBqGJpEDZmqEmGtyVm8Ety9B6 cskciW1mG/78qgt+K06zD5yDVqEMK6tlAgNJychIxwkn/a2l7OYlJiOX/aTxPLKS3TKL qjC2Fdj8EPOvYJIrFhaLSc2aErkRmvjQUtPZd5+x4Wkeyb/KCJIqCNLuDSAjaqdA3kjH cYMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=wxw28PT0y1HD093q1CN4MSHFbLeN7akEybJmased0TU=; b=OSBdj4KVHyKjVgHDclH+SNmiWcl3RpMQPUoGTZHO1nRHx0lPDxB1XxmW7yThNH75BG +iVQY7IAW1/jww0KiQsnWIBjVqyjZXlsTDZWBPmRRIqFm4MDYjlpDjyUjbwvWJ4ra5AX yck0yg/TDINxWkJLuMsxB89rFzpeii7PBOPiix/5H5yoBrTRPc5YzQ5TnRezxVOnPXKh LcjLpnYhidVx3G9fcgNnscHNHSxhC3f30azLEb1+TKwRQmPqDuM62qbWuWdTCqkcYj6P njoP1O+M9Y2IorZ1ACiGUbP4UaQ9cYPceWwckQDJUzeUE6XSL9+AGNv1USC4lqMcSL4/ Mluw== X-Gm-Message-State: AOAM53175CCr9zN5P3CE7Z5MLrkM+yeJzYfTufaxdC30BVuKwU3Xfamd OqnUHq15LtkJLzOW+O1sXFngMvMPd4M= X-Google-Smtp-Source: ABdhPJzVRE1VRd7yWzboNpOdhzbf2pNHWmCTs84C2/abmFj+9H5133IMbkXbrbmlUHGqJpfoxlhA+A== X-Received: by 2002:a05:600c:4e92:: with SMTP id f18mr11246062wmq.62.1626363635032; Thu, 15 Jul 2021 08:40:35 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p16sm7108171wrs.52.2021.07.15.08.40.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:34 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:19 +0000 Subject: [PATCH v2 11/17] [GSOC] cat-file: change batch_objects parameter name Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because later cat-file reuses ref-filter logic that will add parameter "const struct option *options" to batch_objects(), the two synonymous parameters of "opt" and "options" may confuse readers, so change batch_options parameter of batch_objects() from "opt" to "batch". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 59a86412fd0..41d407638d5 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -495,7 +495,7 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *opt) +static int batch_objects(struct batch_options *batch) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; @@ -503,8 +503,8 @@ static int batch_objects(struct batch_options *opt) int save_warning; int retval = 0; - if (!opt->format) - opt->format = "%(objectname) %(objecttype) %(objectsize)"; + if (!batch->format) + batch->format = "%(objectname) %(objecttype) %(objectsize)"; /* * Expand once with our special mark_query flag, which will prime the @@ -513,20 +513,20 @@ static int batch_objects(struct batch_options *opt) */ memset(&data, 0, sizeof(data)); data.mark_query = 1; - strbuf_expand(&output, opt->format, expand_format, &data); + strbuf_expand(&output, batch->format, expand_format, &data); data.mark_query = 0; strbuf_release(&output); - if (opt->cmdmode) + if (batch->cmdmode) data.split_on_whitespace = 1; /* * If we are printing out the object, then always fill in the type, * since we will want to decide whether or not to stream. */ - if (opt->print_contents) + if (batch->print_contents) data.info.typep = &data.type; - if (opt->all_objects) { + if (batch->all_objects) { struct object_cb_data cb; struct object_info empty = OBJECT_INFO_INIT; @@ -536,11 +536,11 @@ static int batch_objects(struct batch_options *opt) if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); - cb.opt = opt; + cb.opt = batch; cb.expand = &data; cb.scratch = &output; - if (opt->unordered) { + if (batch->unordered) { struct oidset seen = OIDSET_INIT; cb.seen = &seen; @@ -590,7 +590,7 @@ static int batch_objects(struct batch_options *opt) data.rest = p; } - batch_one_object(input.buf, &output, opt, &data); + batch_one_object(input.buf, &output, batch, &data); } strbuf_release(&input); From patchwork Thu Jul 15 15:40:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 955F4C47E4E for ; Thu, 15 Jul 2021 15:40:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 82FFB61374 for ; Thu, 15 Jul 2021 15:40:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239311AbhGOPnf (ORCPT ); Thu, 15 Jul 2021 11:43:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239236AbhGOPnb (ORCPT ); Thu, 15 Jul 2021 11:43:31 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30F37C0613DD for ; Thu, 15 Jul 2021 08:40:36 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id w13so3997066wmc.3 for ; Thu, 15 Jul 2021 08:40:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=9N06sBMGSwslqzktoVL9OhM7KzXq4rCuo4jo7obxQWo=; b=vDUW5T6QowNzFcHGDbDhPT33dJUP21r5QqIEWNcSBJ8KBNj9KxYM8P/6jDTeqeUgwq hbNq04lJx5DoX/GTwgc+mdo704nfq2Ovan1wUa3FDxQ7SiqnmeYlQlZP40wvlEyF4D0X VWOURjbQv6/NwIuUHKWJOLLZ9a9Y68YbPaQStvrPiO/DNcJ4E9gZLKQaPVsqEkRQ7XwF TprZxFpChWnM4UPryNo3PR0E5aU8jfYipNuPuwSGV4141pfxmI5ZtHHi9zdbdnUyQXzK ZmRpBNEsHvAgbZOOh8QHOJ6sPwbPQKvQtjxk4Pqv4kZT7hEEhgva0J5iy62f937Q9k4L TyPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=9N06sBMGSwslqzktoVL9OhM7KzXq4rCuo4jo7obxQWo=; b=nNMdP1OYMZbMH+TD+oyZXmvRfwUaOymC5XCZtCNFe9UJB4JCN2mjmblThUyp0TkzE2 BH1LjMgv7E/i2m5RirtzGDtdDta/yrZJX0a02U165AVJmN9tE2uHulg3a/TP4YOjeAj0 +wE/JYaFSTfMlviqTI5qBTnFsTb9kcjFBn842buOEtEPGagcj54K5IwTnjjxoC9aUoWR dhsNgILss2Ar8c1lHhuN3cS9AeA+5kADQ8hcuxaX38RXoPXWh6LOSDAeVC3GlPAETjQA Q5iFqOMrxZjXf8GtjcTiKo4Q+PMaH5WElj2bjXE+AWgWSg0xzrlNfKmYCypNHWDVGQNu jXmg== X-Gm-Message-State: AOAM533995pTRN0whXPzwB40k8Vglb99ArrCKI3gRF75mlpHeNhGuuuR CRbYVLVyAkmatIdJmvsd1AqfxTXbxn4= X-Google-Smtp-Source: ABdhPJyA/pAKfh/ooDH7FbyP+6QY//BV5JDsYdFB8eNf6W7jQsIRiPlXHkg4Lsb0vgjY6XK332UIjw== X-Received: by 2002:a05:600c:3783:: with SMTP id o3mr5454494wmr.23.1626363635619; Thu, 15 Jul 2021 08:40:35 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f130sm6649377wmf.23.2021.07.15.08.40.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:35 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:20 +0000 Subject: [PATCH v2 12/17] [GSOC] cat-file: create p1006-cat-file.sh MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Create p1006-cat-file.sh to provide performance testing for `git cat-file --batch` and `git cat-file --batch-check`. This will help us compare the performance changes after we let cat-file reuse the ref-filter logic. Helped-by: Ævar Arnfjörð Bjarmason Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- t/perf/p1006-cat-file.sh | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 t/perf/p1006-cat-file.sh diff --git a/t/perf/p1006-cat-file.sh b/t/perf/p1006-cat-file.sh new file mode 100755 index 00000000000..b84ac31f9cc --- /dev/null +++ b/t/perf/p1006-cat-file.sh @@ -0,0 +1,28 @@ +#!/bin/sh + +test_description='Basic sort performance tests' +. ./perf-lib.sh + +test_perf_default_repo + +test_expect_success 'setup' ' + git rev-list --all >rla +' + +test_perf 'cat-file --batch-check' ' + git cat-file --batch-check X-Patchwork-Id: 12380343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FC8AC47E4B for ; Thu, 15 Jul 2021 15:40:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D666613E0 for ; Thu, 15 Jul 2021 15:40:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239370AbhGOPnh (ORCPT ); Thu, 15 Jul 2021 11:43:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239150AbhGOPnc (ORCPT ); Thu, 15 Jul 2021 11:43:32 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B630CC0613DF for ; Thu, 15 Jul 2021 08:40:37 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id l7so8360367wrv.7 for ; Thu, 15 Jul 2021 08:40:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=g7oLqekIzUHOLeKLpyNBnQPDFvutHgQ1qGR0c0eP25U=; b=q5ewpeU16Lqe4pcEm2GWScp3ThvcEb0Zy3q/o/Ym40Q/0SxNQQZ7bG5k8QHiq0u3af tPZ6zyGNEXOD4c72IVmisdFLZIVoCAFd6tL/35815dUEjq6sRDzXzcyRpnkvxrAtdZnw YPMzk3Monq+gL91eJrQa/j9EnH1Ngc3Toqu2G0jw0snKSjbyz892s4fMKaTLrTOtHKeT pTEGvgZk8I9ySFzbi1RM15tVZaj7B6ZMCUQ6KPguYxPniltriVqheNG/TtVJWs6FArCz Zx46TeY1yrfGqmC2xGVSckWVxK0Wc1/mZECmubaqgMrLQfv+EztJmQl0+VSaQB+XkRwF xLzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=g7oLqekIzUHOLeKLpyNBnQPDFvutHgQ1qGR0c0eP25U=; b=r+s4/rfK7f7sJeUXCmF7dpzzpACtPywEsaghnSMgOCdXg2teQrZI7aU2DBlN5Hi36v Jrqf+bRwPVSug1i/vCG8kRUdW+NRGyFVGUIsgcQSPXUeVkxDWCAVWJ+Xdb4uZmLNzAdi F+E0aje7shu00wdVz0V8l8QWi0Uh5XcfCjPoCpETR2ujQbfm42oMNQnf6eAo0iCCRRR/ jr5MJ9iDeEXU9je2YKvpUHdhK3bjawBEe+AZ2wTX4oQzzA0qN/H68ySlu9OcI9+H/oJu 3gX5k/IhgIjY4D6krlZ0U249sKvRmeMVXfMPku/yTwFrgUrTxFDsoMDZ7mzWUyOwdcdv mp/g== X-Gm-Message-State: AOAM531mVxtWN16KqKpsBgftGNoXpAqtYU14jWyYyOwhbzL2plNytNXQ lsgKs5F2MZWRCfvktg5F4Lhlt+63FVo= X-Google-Smtp-Source: ABdhPJwcu4WupVI8NQljxGgqKns35kmT6CBMfODJnNz/2zVxA0eXKB1ZOvVfp9xkxXbqfIpWiCbUYw== X-Received: by 2002:adf:ba07:: with SMTP id o7mr6343564wrg.160.1626363636291; Thu, 15 Jul 2021 08:40:36 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o29sm5065375wms.13.2021.07.15.08.40.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:35 -0700 (PDT) Message-Id: <8b26c9e745523ab7b1ca07065d0c338fef9c4293.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:21 +0000 Subject: [PATCH v2 13/17] [GSOC] cat-file: reuse ref-filter logic MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let cat-file use ref-filter logic, let's do the following: 1. Change the type of member `format` in struct `batch_options` to `ref_format`, we will pass it to ref-filter later. 2. Let `batch_objects()` add atoms to format, and use `verify_ref_format()` to check atoms. 3. Use `format_ref_array_item()` in `batch_object_write()` to get the formatted data corresponding to the object. If the return value of `format_ref_array_item()` is equals to zero, use `batch_write()` to print object data; else if the return value is less than zero, use `die()` to print the error message and exit; else if return value is greater than zero, only print the error message, but don't exit. 4. Use free_ref_array_item_value() to free ref_array_item's value. Most of the atoms in `for-each-ref --format` are now supported, such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, `%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: `%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, `%(flag)`, `%(HEAD)`, because these atoms are unique to those objects that pointed to by a ref, "for-each-ref"'s family can naturally use these atoms, but not all objects are pointed to be a ref, so "cat-file" will not be able to use them. The performance for `git cat-file --batch-all-objects --batch-check` on the Git repository itself with performance testing tool `hyperfine` changes from 616.7 ms ± 8.9 ms to 758.7 ms ± 16.4 ms. The performance for `git cat-file --batch-all-objects --batch >/dev/null` on the Git repository itself with performance testing tool `time` change from "25.26s user 0.30s system 98% cpu 25.840 total" to "28.79s user 0.83s system 99% cpu 29.829 total". The reasons for the performance degradation are as follows: 1. There are a lot of data copies in the logic of ref-filter. 2, In order to be able to support more useful formats, complex data structure and parsing process are used in ref-filter. A later patch will add a fast path which will mitigate the performance regression introduced by this patch. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- Documentation/git-cat-file.txt | 6 + builtin/cat-file.c | 242 +++++++-------------------------- t/t1006-cat-file.sh | 229 ++++++++++++++++--------------- 3 files changed, 174 insertions(+), 303 deletions(-) diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index 4eb0421b3fd..ef8ab952b2f 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -226,6 +226,12 @@ newline. The available atoms are: after that first run of whitespace (i.e., the "rest" of the line) are output in place of the `%(rest)` atom. +Note that most of the atoms in `for-each-ref --format` are now supported, +such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, +`%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: +`%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, +`%(flag)`, `%(HEAD)`. See linkgit:git-for-each-ref[1]. + If no format is specified, the default format is `%(objectname) %(objecttype) %(objectsize)`. diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 41d407638d5..5b163551fc6 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -16,6 +16,7 @@ #include "packfile.h" #include "object-store.h" #include "promisor-remote.h" +#include "ref-filter.h" struct batch_options { int enabled; @@ -25,7 +26,7 @@ struct batch_options { int all_objects; int unordered; int cmdmode; /* may be 'w' or 'c' for --filters or --textconv */ - const char *format; + struct ref_format format; }; static const char *force_path; @@ -195,99 +196,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct expand_data { struct object_id oid; - enum object_type type; - unsigned long size; - off_t disk_size; const char *rest; - struct object_id delta_base_oid; - - /* - * If mark_query is true, we do not expand anything, but rather - * just mark the object_info with items we wish to query. - */ - int mark_query; - - /* - * Whether to split the input on whitespace before feeding it to - * get_sha1; this is decided during the mark_query phase based on - * whether we have a %(rest) token in our format. - */ int split_on_whitespace; - - /* - * After a mark_query run, this object_info is set up to be - * passed to oid_object_info_extended. It will point to the data - * elements above, so you can retrieve the response from there. - */ - struct object_info info; - - /* - * This flag will be true if the requested batch format and options - * don't require us to call oid_object_info, which can then be - * optimized out. - */ - unsigned skip_object_info : 1; }; -static int is_atom(const char *atom, const char *s, int slen) -{ - int alen = strlen(atom); - return alen == slen && !memcmp(atom, s, alen); -} - -static void expand_atom(struct strbuf *sb, const char *atom, int len, - void *vdata) -{ - struct expand_data *data = vdata; - - if (is_atom("objectname", atom, len)) { - if (!data->mark_query) - strbuf_addstr(sb, oid_to_hex(&data->oid)); - } else if (is_atom("objecttype", atom, len)) { - if (data->mark_query) - data->info.typep = &data->type; - else - strbuf_addstr(sb, type_name(data->type)); - } else if (is_atom("objectsize", atom, len)) { - if (data->mark_query) - data->info.sizep = &data->size; - else - strbuf_addf(sb, "%"PRIuMAX , (uintmax_t)data->size); - } else if (is_atom("objectsize:disk", atom, len)) { - if (data->mark_query) - data->info.disk_sizep = &data->disk_size; - else - strbuf_addf(sb, "%"PRIuMAX, (uintmax_t)data->disk_size); - } else if (is_atom("rest", atom, len)) { - if (data->mark_query) - data->split_on_whitespace = 1; - else if (data->rest) - strbuf_addstr(sb, data->rest); - } else if (is_atom("deltabase", atom, len)) { - if (data->mark_query) - data->info.delta_base_oid = &data->delta_base_oid; - else - strbuf_addstr(sb, - oid_to_hex(&data->delta_base_oid)); - } else - die("unknown format element: %.*s", len, atom); -} - -static size_t expand_format(struct strbuf *sb, const char *start, void *data) -{ - const char *end; - - if (*start != '(') - return 0; - end = strchr(start + 1, ')'); - if (!end) - die("format element '%s' does not end in ')'", start); - - expand_atom(sb, start + 1, end - start - 1, data); - - return end - start + 1; -} - static void batch_write(struct batch_options *opt, const void *data, int len) { if (opt->buffer_output) { @@ -297,87 +209,34 @@ static void batch_write(struct batch_options *opt, const void *data, int len) write_or_die(1, data, len); } -static void print_object_or_die(struct batch_options *opt, struct expand_data *data) -{ - const struct object_id *oid = &data->oid; - - assert(data->info.typep); - - if (data->type == OBJ_BLOB) { - if (opt->buffer_output) - fflush(stdout); - if (opt->cmdmode) { - char *contents; - unsigned long size; - - if (!data->rest) - die("missing path for '%s'", oid_to_hex(oid)); - - if (opt->cmdmode == 'w') { - if (filter_object(data->rest, 0100644, oid, - &contents, &size)) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else if (opt->cmdmode == 'c') { - enum object_type type; - if (!textconv_object(the_repository, - data->rest, 0100644, oid, - 1, &contents, &size)) - contents = read_object_file(oid, - &type, - &size); - if (!contents) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else - BUG("invalid cmdmode: %c", opt->cmdmode); - batch_write(opt, contents, size); - free(contents); - } else { - stream_blob(oid); - } - } - else { - enum object_type type; - unsigned long size; - void *contents; - - contents = read_object_file(oid, &type, &size); - if (!contents) - die("object %s disappeared", oid_to_hex(oid)); - if (type != data->type) - die("object %s changed type!?", oid_to_hex(oid)); - if (data->info.sizep && size != data->size) - die("object %s changed size!?", oid_to_hex(oid)); - - batch_write(opt, contents, size); - free(contents); - } -} static void batch_object_write(const char *obj_name, struct strbuf *scratch, struct batch_options *opt, struct expand_data *data) { - if (!data->skip_object_info && - oid_object_info_extended(the_repository, &data->oid, &data->info, - OBJECT_INFO_LOOKUP_REPLACE) < 0) { - printf("%s missing\n", - obj_name ? obj_name : oid_to_hex(&data->oid)); - fflush(stdout); - return; - } + int ret; + struct strbuf err = STRBUF_INIT; + struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); - strbuf_expand(scratch, opt->format, expand_format, data); - strbuf_addch(scratch, '\n'); - batch_write(opt, scratch->buf, scratch->len); - if (opt->print_contents) { - print_object_or_die(opt, data); - batch_write(opt, "\n", 1); + ret = format_ref_array_item(&item, &opt->format, scratch, &err); + if (ret < 0) + die("%s\n", err.buf); + if (ret) { + /* ret > 0 means when the object corresponding to oid + * cannot be found in format_ref_array_item(), we only print + * the error message. + */ + printf("%s\n", err.buf); + fflush(stdout); + } else { + strbuf_addch(scratch, '\n'); + batch_write(opt, scratch->buf, scratch->len); } + free_ref_array_item_value(&item); + strbuf_release(&err); } static void batch_one_object(const char *obj_name, @@ -495,43 +354,37 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *batch) +static const char * const cat_file_usage[] = { + N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), + N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), + NULL +}; + +static int batch_objects(struct batch_options *batch, const struct option *options) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; int retval = 0; - if (!batch->format) - batch->format = "%(objectname) %(objecttype) %(objectsize)"; - - /* - * Expand once with our special mark_query flag, which will prime the - * object_info to be handed to oid_object_info_extended for each - * object. - */ memset(&data, 0, sizeof(data)); - data.mark_query = 1; - strbuf_expand(&output, batch->format, expand_format, &data); - data.mark_query = 0; - strbuf_release(&output); - if (batch->cmdmode) - data.split_on_whitespace = 1; - - /* - * If we are printing out the object, then always fill in the type, - * since we will want to decide whether or not to stream. - */ + if (batch->format.format) + strbuf_addstr(&format, batch->format.format); + else + strbuf_addstr(&format, "%(objectname) %(objecttype) %(objectsize)"); if (batch->print_contents) - data.info.typep = &data.type; + strbuf_addstr(&format, "\n%(raw)"); + batch->format.format = format.buf; + if (verify_ref_format(&batch->format)) + usage_with_options(cat_file_usage, options); + + if (batch->cmdmode || batch->format.use_rest) + data.split_on_whitespace = 1; if (batch->all_objects) { struct object_cb_data cb; - struct object_info empty = OBJECT_INFO_INIT; - - if (!memcmp(&data.info, &empty, sizeof(empty))) - data.skip_object_info = 1; if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); @@ -561,6 +414,7 @@ static int batch_objects(struct batch_options *batch) oid_array_clear(&sa); } + strbuf_release(&format); strbuf_release(&output); return 0; } @@ -593,18 +447,13 @@ static int batch_objects(struct batch_options *batch) batch_one_object(input.buf, &output, batch, &data); } + strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); warn_on_object_refname_ambiguity = save_warning; return retval; } -static const char * const cat_file_usage[] = { - N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), - N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), - NULL -}; - static int git_cat_file_config(const char *var, const char *value, void *cb) { if (userdiff_config(var, value) < 0) @@ -627,7 +476,7 @@ static int batch_option_callback(const struct option *opt, bo->enabled = 1; bo->print_contents = !strcmp(opt->long_name, "batch"); - bo->format = arg; + bo->format.format = arg; return 0; } @@ -636,7 +485,9 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) { int opt = 0; const char *exp_type = NULL, *obj_name = NULL; - struct batch_options batch = {0}; + struct batch_options batch = { + .format = REF_FORMAT_INIT + }; int unknown_type = 0; const struct option options[] = { @@ -675,6 +526,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) git_config(git_cat_file_config, NULL); batch.buffer_output = -1; + batch.format.cat_file_mode = 1; argc = parse_options(argc, argv, prefix, options, cat_file_usage, 0); if (opt) { @@ -718,7 +570,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) batch.buffer_output = batch.all_objects; if (batch.enabled) - return batch_objects(&batch); + return batch_objects(&batch, options); if (unknown_type && opt != 't' && opt != 's') die("git cat-file --allow-unknown-type: use with -s or -t"); diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 95d760652eb..8c1943011fb 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -679,51 +679,51 @@ batch_test_atom refs/heads/main '%(objectsize)' batch_test_atom refs/heads/main '%(objectsize:disk)' batch_test_atom refs/heads/main '%(deltabase)' batch_test_atom refs/heads/main '%(objectname)' -batch_test_atom refs/heads/main '%(objectname:short)' fail -batch_test_atom refs/heads/main '%(objectname:short=1)' fail -batch_test_atom refs/heads/main '%(objectname:short=10)' fail -batch_test_atom refs/heads/main '%(tree)' fail -batch_test_atom refs/heads/main '%(tree:short)' fail -batch_test_atom refs/heads/main '%(tree:short=1)' fail -batch_test_atom refs/heads/main '%(tree:short=10)' fail -batch_test_atom refs/heads/main '%(parent)' fail -batch_test_atom refs/heads/main '%(parent:short)' fail -batch_test_atom refs/heads/main '%(parent:short=1)' fail -batch_test_atom refs/heads/main '%(parent:short=10)' fail -batch_test_atom refs/heads/main '%(numparent)' fail -batch_test_atom refs/heads/main '%(object)' fail -batch_test_atom refs/heads/main '%(type)' fail -batch_test_atom refs/heads/main '%(raw)' fail -batch_test_atom refs/heads/main '%(*objectname)' fail -batch_test_atom refs/heads/main '%(*objecttype)' fail -batch_test_atom refs/heads/main '%(author)' fail -batch_test_atom refs/heads/main '%(authorname)' fail -batch_test_atom refs/heads/main '%(authoremail)' fail -batch_test_atom refs/heads/main '%(authoremail:trim)' fail -batch_test_atom refs/heads/main '%(authoremail:localpart)' fail -batch_test_atom refs/heads/main '%(authordate)' fail -batch_test_atom refs/heads/main '%(committer)' fail -batch_test_atom refs/heads/main '%(committername)' fail -batch_test_atom refs/heads/main '%(committeremail)' fail -batch_test_atom refs/heads/main '%(committeremail:trim)' fail -batch_test_atom refs/heads/main '%(committeremail:localpart)' fail -batch_test_atom refs/heads/main '%(committerdate)' fail -batch_test_atom refs/heads/main '%(tag)' fail -batch_test_atom refs/heads/main '%(tagger)' fail -batch_test_atom refs/heads/main '%(taggername)' fail -batch_test_atom refs/heads/main '%(taggeremail)' fail -batch_test_atom refs/heads/main '%(taggeremail:trim)' fail -batch_test_atom refs/heads/main '%(taggeremail:localpart)' fail -batch_test_atom refs/heads/main '%(taggerdate)' fail -batch_test_atom refs/heads/main '%(creator)' fail -batch_test_atom refs/heads/main '%(creatordate)' fail -batch_test_atom refs/heads/main '%(subject)' fail -batch_test_atom refs/heads/main '%(subject:sanitize)' fail -batch_test_atom refs/heads/main '%(contents:subject)' fail -batch_test_atom refs/heads/main '%(body)' fail -batch_test_atom refs/heads/main '%(contents:body)' fail -batch_test_atom refs/heads/main '%(contents:signature)' fail -batch_test_atom refs/heads/main '%(contents)' fail +batch_test_atom refs/heads/main '%(objectname:short)' +batch_test_atom refs/heads/main '%(objectname:short=1)' +batch_test_atom refs/heads/main '%(objectname:short=10)' +batch_test_atom refs/heads/main '%(tree)' +batch_test_atom refs/heads/main '%(tree:short)' +batch_test_atom refs/heads/main '%(tree:short=1)' +batch_test_atom refs/heads/main '%(tree:short=10)' +batch_test_atom refs/heads/main '%(parent)' +batch_test_atom refs/heads/main '%(parent:short)' +batch_test_atom refs/heads/main '%(parent:short=1)' +batch_test_atom refs/heads/main '%(parent:short=10)' +batch_test_atom refs/heads/main '%(numparent)' +batch_test_atom refs/heads/main '%(object)' +batch_test_atom refs/heads/main '%(type)' +batch_test_atom refs/heads/main '%(raw)' +batch_test_atom refs/heads/main '%(*objectname)' +batch_test_atom refs/heads/main '%(*objecttype)' +batch_test_atom refs/heads/main '%(author)' +batch_test_atom refs/heads/main '%(authorname)' +batch_test_atom refs/heads/main '%(authoremail)' +batch_test_atom refs/heads/main '%(authoremail:trim)' +batch_test_atom refs/heads/main '%(authoremail:localpart)' +batch_test_atom refs/heads/main '%(authordate)' +batch_test_atom refs/heads/main '%(committer)' +batch_test_atom refs/heads/main '%(committername)' +batch_test_atom refs/heads/main '%(committeremail)' +batch_test_atom refs/heads/main '%(committeremail:trim)' +batch_test_atom refs/heads/main '%(committeremail:localpart)' +batch_test_atom refs/heads/main '%(committerdate)' +batch_test_atom refs/heads/main '%(tag)' +batch_test_atom refs/heads/main '%(tagger)' +batch_test_atom refs/heads/main '%(taggername)' +batch_test_atom refs/heads/main '%(taggeremail)' +batch_test_atom refs/heads/main '%(taggeremail:trim)' +batch_test_atom refs/heads/main '%(taggeremail:localpart)' +batch_test_atom refs/heads/main '%(taggerdate)' +batch_test_atom refs/heads/main '%(creator)' +batch_test_atom refs/heads/main '%(creatordate)' +batch_test_atom refs/heads/main '%(subject)' +batch_test_atom refs/heads/main '%(subject:sanitize)' +batch_test_atom refs/heads/main '%(contents:subject)' +batch_test_atom refs/heads/main '%(body)' +batch_test_atom refs/heads/main '%(contents:body)' +batch_test_atom refs/heads/main '%(contents:signature)' +batch_test_atom refs/heads/main '%(contents)' batch_test_atom refs/heads/main '%(HEAD)' fail batch_test_atom refs/heads/main '%(upstream:track)' fail batch_test_atom refs/heads/main '%(upstream:trackshort)' fail @@ -742,52 +742,52 @@ batch_test_atom refs/tags/testtag '%(push)' fail batch_test_atom refs/tags/testtag '%(objecttype)' batch_test_atom refs/tags/testtag '%(objectsize)' batch_test_atom refs/tags/testtag '%(objectsize:disk)' -batch_test_atom refs/tags/testtag '%(*objectsize:disk)' fail +batch_test_atom refs/tags/testtag '%(*objectsize:disk)' batch_test_atom refs/tags/testtag '%(deltabase)' -batch_test_atom refs/tags/testtag '%(*deltabase)' fail +batch_test_atom refs/tags/testtag '%(*deltabase)' batch_test_atom refs/tags/testtag '%(objectname)' -batch_test_atom refs/tags/testtag '%(objectname:short)' fail -batch_test_atom refs/tags/testtag '%(tree)' fail -batch_test_atom refs/tags/testtag '%(tree:short)' fail -batch_test_atom refs/tags/testtag '%(tree:short=1)' fail -batch_test_atom refs/tags/testtag '%(tree:short=10)' fail -batch_test_atom refs/tags/testtag '%(parent)' fail -batch_test_atom refs/tags/testtag '%(parent:short)' fail -batch_test_atom refs/tags/testtag '%(parent:short=1)' fail -batch_test_atom refs/tags/testtag '%(parent:short=10)' fail -batch_test_atom refs/tags/testtag '%(numparent)' fail -batch_test_atom refs/tags/testtag '%(object)' fail -batch_test_atom refs/tags/testtag '%(type)' fail -batch_test_atom refs/tags/testtag '%(*objectname)' fail -batch_test_atom refs/tags/testtag '%(*objecttype)' fail -batch_test_atom refs/tags/testtag '%(author)' fail -batch_test_atom refs/tags/testtag '%(authorname)' fail -batch_test_atom refs/tags/testtag '%(authoremail)' fail -batch_test_atom refs/tags/testtag '%(authoremail:trim)' fail -batch_test_atom refs/tags/testtag '%(authoremail:localpart)' fail -batch_test_atom refs/tags/testtag '%(authordate)' fail -batch_test_atom refs/tags/testtag '%(committer)' fail -batch_test_atom refs/tags/testtag '%(committername)' fail -batch_test_atom refs/tags/testtag '%(committeremail)' fail -batch_test_atom refs/tags/testtag '%(committeremail:trim)' fail -batch_test_atom refs/tags/testtag '%(committeremail:localpart)' fail -batch_test_atom refs/tags/testtag '%(committerdate)' fail -batch_test_atom refs/tags/testtag '%(tag)' fail -batch_test_atom refs/tags/testtag '%(tagger)' fail -batch_test_atom refs/tags/testtag '%(taggername)' fail -batch_test_atom refs/tags/testtag '%(taggeremail)' fail -batch_test_atom refs/tags/testtag '%(taggeremail:trim)' fail -batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' fail -batch_test_atom refs/tags/testtag '%(taggerdate)' fail -batch_test_atom refs/tags/testtag '%(creator)' fail -batch_test_atom refs/tags/testtag '%(creatordate)' fail -batch_test_atom refs/tags/testtag '%(subject)' fail -batch_test_atom refs/tags/testtag '%(subject:sanitize)' fail -batch_test_atom refs/tags/testtag '%(contents:subject)' fail -batch_test_atom refs/tags/testtag '%(body)' fail -batch_test_atom refs/tags/testtag '%(contents:body)' fail -batch_test_atom refs/tags/testtag '%(contents:signature)' fail -batch_test_atom refs/tags/testtag '%(contents)' fail +batch_test_atom refs/tags/testtag '%(objectname:short)' +batch_test_atom refs/tags/testtag '%(tree)' +batch_test_atom refs/tags/testtag '%(tree:short)' +batch_test_atom refs/tags/testtag '%(tree:short=1)' +batch_test_atom refs/tags/testtag '%(tree:short=10)' +batch_test_atom refs/tags/testtag '%(parent)' +batch_test_atom refs/tags/testtag '%(parent:short)' +batch_test_atom refs/tags/testtag '%(parent:short=1)' +batch_test_atom refs/tags/testtag '%(parent:short=10)' +batch_test_atom refs/tags/testtag '%(numparent)' +batch_test_atom refs/tags/testtag '%(object)' +batch_test_atom refs/tags/testtag '%(type)' +batch_test_atom refs/tags/testtag '%(*objectname)' +batch_test_atom refs/tags/testtag '%(*objecttype)' +batch_test_atom refs/tags/testtag '%(author)' +batch_test_atom refs/tags/testtag '%(authorname)' +batch_test_atom refs/tags/testtag '%(authoremail)' +batch_test_atom refs/tags/testtag '%(authoremail:trim)' +batch_test_atom refs/tags/testtag '%(authoremail:localpart)' +batch_test_atom refs/tags/testtag '%(authordate)' +batch_test_atom refs/tags/testtag '%(committer)' +batch_test_atom refs/tags/testtag '%(committername)' +batch_test_atom refs/tags/testtag '%(committeremail)' +batch_test_atom refs/tags/testtag '%(committeremail:trim)' +batch_test_atom refs/tags/testtag '%(committeremail:localpart)' +batch_test_atom refs/tags/testtag '%(committerdate)' +batch_test_atom refs/tags/testtag '%(tag)' +batch_test_atom refs/tags/testtag '%(tagger)' +batch_test_atom refs/tags/testtag '%(taggername)' +batch_test_atom refs/tags/testtag '%(taggeremail)' +batch_test_atom refs/tags/testtag '%(taggeremail:trim)' +batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' +batch_test_atom refs/tags/testtag '%(taggerdate)' +batch_test_atom refs/tags/testtag '%(creator)' +batch_test_atom refs/tags/testtag '%(creatordate)' +batch_test_atom refs/tags/testtag '%(subject)' +batch_test_atom refs/tags/testtag '%(subject:sanitize)' +batch_test_atom refs/tags/testtag '%(contents:subject)' +batch_test_atom refs/tags/testtag '%(body)' +batch_test_atom refs/tags/testtag '%(contents:body)' +batch_test_atom refs/tags/testtag '%(contents:signature)' +batch_test_atom refs/tags/testtag '%(contents)' batch_test_atom refs/tags/testtag '%(HEAD)' fail batch_test_atom refs/myblobs/blob1 '%(refname)' fail @@ -801,37 +801,50 @@ batch_test_atom refs/myblobs/blob1 '%(objectsize)' batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)' batch_test_atom refs/myblobs/blob1 '%(deltabase)' -batch_test_atom refs/myblobs/blob1 '%(contents)' fail -batch_test_atom refs/myblobs/blob2 '%(contents)' fail +batch_test_atom refs/myblobs/blob1 '%(contents)' +batch_test_atom refs/myblobs/blob2 '%(contents)' -batch_test_atom refs/myblobs/blob1 '%(raw)' fail -batch_test_atom refs/myblobs/blob2 '%(raw)' fail -batch_test_atom refs/mytrees/tree1 '%(raw)' fail +batch_test_atom refs/myblobs/blob1 '%(raw)' +batch_test_atom refs/myblobs/blob2 '%(raw)' +batch_test_atom refs/mytrees/tree1 '%(raw)' -batch_test_atom refs/myblobs/blob1 '%(raw:size)' fail -batch_test_atom refs/myblobs/blob2 '%(raw:size)' fail -batch_test_atom refs/mytrees/tree1 '%(raw:size)' fail +batch_test_atom refs/myblobs/blob1 '%(raw:size)' +batch_test_atom refs/myblobs/blob2 '%(raw:size)' +batch_test_atom refs/mytrees/tree1 '%(raw:size)' -batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail -batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail -batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' fail +batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' -batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' fail -batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' fail -batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' fail -batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' fail +batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' batch_test_atom refs/heads/main 'VALID' batch_test_atom refs/heads/main '%(INVALID)' fail batch_test_atom refs/heads/main '%(authordate:INVALID)' fail batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize) -%(raw)' fail +%(raw)' batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize) -%(raw)' fail +%(raw)' batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize) -%(raw)' fail +%(raw)' batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize) -%(raw)' fail +%(raw)' + +test_expect_success 'cat-file --batch equals to --batch-check with atoms' ' + git cat-file --batch-check="%(objectname) %(objecttype) %(objectsize) +%(raw)" >expected <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + git cat-file --batch >actual <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + cmp expected actual +' test_done From patchwork Thu Jul 15 15:40:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380341 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A98D8C47E4D for ; Thu, 15 Jul 2021 15:40:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91A6461396 for ; Thu, 15 Jul 2021 15:40:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239415AbhGOPnj (ORCPT ); Thu, 15 Jul 2021 11:43:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239191AbhGOPnc (ORCPT ); Thu, 15 Jul 2021 11:43:32 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50614C061766 for ; Thu, 15 Jul 2021 08:40:38 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id t5so8347078wrw.12 for ; Thu, 15 Jul 2021 08:40:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=qAwXHcrowoAV1ZvkfF6sVHaTDEdBOu0rUNJ1Ia1vdCk=; b=exgEU1v3Lm82T2w4+e1YjUmHDTOUxFPxuEK6G2lmveTeryrKDvbrRRhxJZFf6QFtAJ uk+i52lVvuAhZXA5Cn7jGZ+wS6A1+FXfM+j/rM3XHw6/BCgNhaWf33WAMgQOiu4RHCAo +5Z/0+ksmNgMcYHDcRtUCzh/f3VGg8bNF2x1n6kEMnS69WCV0kLeuS1LeVn1fzlYZ9fs Hu9jUKiHcnTl9sgXM1x6GsJvk17L48nSCMbjCUl4RiRVAT9+2PY4JFhFxM3G66fPjZVB xdoR+4vb+m/O+MBqnnDBWSiUIisbp9iBkuM9bVeycszq7x+NCzw2ELTX7XAc6sj2Dvb3 6Fzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=qAwXHcrowoAV1ZvkfF6sVHaTDEdBOu0rUNJ1Ia1vdCk=; b=tpX+d2+VcAXNqMyZraedhqQ28U4pTh+ayxZS8BfsZsaNL7WRSwqrlSmYK++rEZFYx2 9h9OsFZW5BlUn6j8bg71ncGZYSaUsVEn51gMVoB2qp2D+dG3tnSvf2AlpEsvgzlFKCsT wI4ezgRSY+0bYzD/9/VbdmIibbF0LSVdARy1c7XWqDpqk1vrHjWHKdIm8/eZng4l42i8 vpnqLn6tRWeT88HaDnHAc6m74B0bVQPSRHqMKOk063Dm/Zd35yhpN1XZAiiqg77wrVqA Z5aNqVfrlD0itG15tCSjl9esLNtQgP4dsxbecvV055FpzGdofFQpvMKDv3HRmysSmXh8 9UCw== X-Gm-Message-State: AOAM532ez6DT9hiCybSAt3fZMCYmPYXc7DtjLowT6N8fMPPOdiPwhgkz ZI/vmLw+fmsvQfM2XF/X6+7jHbkHL4Y= X-Google-Smtp-Source: ABdhPJyK4wVW2N7svnSpeqp9JLkNq8hwUVTbrGjIgKQ5Y+4z6cjlRzlDIPUf/xdqOK+5ODTm+WTa2A== X-Received: by 2002:adf:f60a:: with SMTP id t10mr6213920wrp.127.1626363636987; Thu, 15 Jul 2021 08:40:36 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f11sm387615wmb.14.2021.07.15.08.40.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:36 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:22 +0000 Subject: [PATCH v2 14/17] [GSOC] cat-file: reuse err buf in batch_object_write() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Reuse the `err` buffer in batch_object_write(), as the buffer `scratch` does. This will reduce the overhead of multiple allocations of memory of the err buffer. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 5b163551fc6..dc604a9879d 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -212,35 +212,36 @@ static void batch_write(struct batch_options *opt, const void *data, int len) static void batch_object_write(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { int ret; - struct strbuf err = STRBUF_INIT; struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); + strbuf_reset(err); - ret = format_ref_array_item(&item, &opt->format, scratch, &err); + ret = format_ref_array_item(&item, &opt->format, scratch, err); if (ret < 0) - die("%s\n", err.buf); + die("%s\n", err->buf); if (ret) { /* ret > 0 means when the object corresponding to oid * cannot be found in format_ref_array_item(), we only print * the error message. */ - printf("%s\n", err.buf); + printf("%s\n", err->buf); fflush(stdout); } else { strbuf_addch(scratch, '\n'); batch_write(opt, scratch->buf, scratch->len); } free_ref_array_item_value(&item); - strbuf_release(&err); } static void batch_one_object(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { @@ -294,7 +295,7 @@ static void batch_one_object(const char *obj_name, return; } - batch_object_write(obj_name, scratch, opt, data); + batch_object_write(obj_name, scratch, err, opt, data); } struct object_cb_data { @@ -302,13 +303,14 @@ struct object_cb_data { struct expand_data *expand; struct oidset *seen; struct strbuf *scratch; + struct strbuf *err; }; static int batch_object_cb(const struct object_id *oid, void *vdata) { struct object_cb_data *data = vdata; oidcpy(&data->expand->oid, oid); - batch_object_write(NULL, data->scratch, data->opt, data->expand); + batch_object_write(NULL, data->scratch, data->err, data->opt, data->expand); return 0; } @@ -364,6 +366,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; @@ -392,6 +395,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio cb.opt = batch; cb.expand = &data; cb.scratch = &output; + cb.err = &err; if (batch->unordered) { struct oidset seen = OIDSET_INIT; @@ -416,6 +420,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio strbuf_release(&format); strbuf_release(&output); + strbuf_release(&err); return 0; } @@ -444,12 +449,13 @@ static int batch_objects(struct batch_options *batch, const struct option *optio data.rest = p; } - batch_one_object(input.buf, &output, batch, &data); + batch_one_object(input.buf, &output, &err, batch, &data); } strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); + strbuf_release(&err); warn_on_object_refname_ambiguity = save_warning; return retval; } From patchwork Thu Jul 15 15:40:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380339 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F41A9C47E48 for ; Thu, 15 Jul 2021 15:40:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D638B613E0 for ; Thu, 15 Jul 2021 15:40:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239367AbhGOPnh (ORCPT ); Thu, 15 Jul 2021 11:43:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239183AbhGOPnc (ORCPT ); Thu, 15 Jul 2021 11:43:32 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EAA8DC061764 for ; Thu, 15 Jul 2021 08:40:38 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id u8-20020a7bcb080000b02901e44e9caa2aso3929120wmj.4 for ; Thu, 15 Jul 2021 08:40:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=2wAhzEz1dqFEFD7ZYTaOKtiC0HQ3bg2FSnbAhq07rRM=; b=AFDrZaezJhEldE+ZuVxT/XEERZo8RX5DzU5u2vVO6qtxcmBsXPXNUejseR8b7Jr5VZ kNQAhr9XA3ow7cAK8kftbsc0/bfmz5NIf5c/HxUlZRfGqvR2yuoUmELxcTu6kTQjR+or uUTjNhJ3KrjHYiuTkJ+QGEmOt0TNxEklf0IxSwTxIF5iNp1JXPelGowrq/95mh8RdUVB 9MdjCo1p+myw4fUg1mSr6r8Y+LrKi2aRmJQvkxCBQYxXVlE5nJe0z9beuPOrEFfr/o3K 3mBDsh6fZxDnwPzxKWS33I81oZYOBp4baIR3FPF8buGB9na2cnjDGpvUyqWHklm4hphI D1fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=2wAhzEz1dqFEFD7ZYTaOKtiC0HQ3bg2FSnbAhq07rRM=; b=RaxFDw3XX7try/Z8ExXM18SM9umthaTawGPBZe6NVAn1OhorFfpC0bJt515xOSQEi2 Bs5BEAcURFHE71Ejqw4I+Rv7vQdQObVQ29F61o1Fl+3AY2LuqN8/CLGiZU2AEkwQfpgB uwTCejHTJ9RKLSc9PWvTPqxBUxoiUzxU1uWSXw6Pl5V+ErUqzYV/BEb1J4vIJ4mJjl7E AMCw30dB+Cu107YYrgb6Zr8hE3utHN9gEnhY0Fzw+QAL1HCrvmI0DJ3SClZhRlwuiM5W bJintQ16nfcEEeziMUlfMSJfeIIHYonk96lFEIN1lJDv0Wa5XTYhJPLc/foxBvwIb3ns R/yw== X-Gm-Message-State: AOAM533y+1qJvFtnY3UKTyjPsfrGfD6H1R/7XaKuoP7Q81eZvQ7ZH9Al n1YIYtNpOkSN1+HnqA8gM1YNtVutScI= X-Google-Smtp-Source: ABdhPJxrB4muB7Ad75UR/FUoOKLLx9gUktL5bgwfUCFqoAaFzmqpxhAI8eAehl01XF9gduXNPYvYgw== X-Received: by 2002:a05:600c:4c96:: with SMTP id g22mr11018447wmp.70.1626363637543; Thu, 15 Jul 2021 08:40:37 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a10sm6688328wrm.12.2021.07.15.08.40.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:37 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:23 +0000 Subject: [PATCH v2 15/17] [GSOC] cat-file: re-implement --textconv, --filters options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu After cat-file reuses the ref-filter logic, we re-implement the functions of --textconv and --filters options. Add members `cat_file_cmdmode` to struct `ref_array_item`, so that struct `batch_option` member `cmdmode` will be passed to ref-filter, and then ref-filter will take use of it to filter the content of the object in get_object(). Use `actual_oi` to record the real expand_data: it may point to the original `oi` or the `act_oi` processed by `textconv_object()` or `convert_to_working_tree()`. `grab_values()` will grab the contents of `actual_oi` and `grab_common_values()` to grab the contents of origin `oi`, this ensures that `%(objectsize)` still uses the size of the unfiltered data. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 2 +- ref-filter.c | 35 +++++++++++++++++++++++++++++++++-- ref-filter.h | 1 + 3 files changed, 35 insertions(+), 3 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index dc604a9879d..3a6153e778f 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -217,7 +217,7 @@ static void batch_object_write(const char *obj_name, struct expand_data *data) { int ret; - struct ref_array_item item = { data->oid, data->rest }; + struct ref_array_item item = { data->oid, data->rest, opt->cmdmode }; strbuf_reset(scratch); strbuf_reset(err); diff --git a/ref-filter.c b/ref-filter.c index b4f41fec871..91e26c9aba3 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1,3 +1,4 @@ +#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "cache.h" #include "parse-options.h" @@ -1755,6 +1756,9 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj { /* parse_object_buffer() will set eaten to 0 if free() will be needed */ int eaten = 1; + struct expand_data *actual_oi = oi; + struct expand_data act_oi = {0}; + if (oi->info.contentp) { /* We need to know that to use parse_object_buffer properly */ oi->info.sizep = &oi->size; @@ -1768,19 +1772,45 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj BUG("Object size is less than zero."); if (oi->info.contentp) { - *obj = parse_object_buffer(the_repository, &oi->oid, oi->type, oi->size, oi->content, &eaten); + if ((ref->cat_file_cmdmode == 'c' || ref->cat_file_cmdmode == 'w') && !ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&act_oi.oid)); + if (oi->type == OBJ_BLOB) { + if (ref->cat_file_cmdmode == 'c') { + act_oi = *oi; + if (textconv_object(the_repository, + ref->rest, 0100644, &act_oi.oid, + 1, (char **)(&act_oi.content), &act_oi.size)) + actual_oi = &act_oi; + } else if (ref->cat_file_cmdmode == 'w') { + struct strbuf strbuf = STRBUF_INIT; + struct checkout_metadata meta; + act_oi = *oi; + + init_checkout_metadata(&meta, NULL, NULL, &act_oi.oid); + if (!convert_to_working_tree(&the_index, ref->rest, act_oi.content, act_oi.size, &strbuf, &meta)) + die("could not convert '%s' %s", + oid_to_hex(&oi->oid), ref->rest); + act_oi.size = strbuf.len; + act_oi.content = strbuf_detach(&strbuf, NULL); + actual_oi = &act_oi; + } + } + *obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten); if (!*obj) { if (!eaten) free(oi->content); return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi); + grab_values(ref->value, deref, *obj, actual_oi); } grab_common_values(ref->value, deref, oi); if (!eaten) free(oi->content); + if (actual_oi != oi) + free(actual_oi->content); return 0; } @@ -2189,6 +2219,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); ref->rest = NULL; + ref->cat_file_cmdmode = 0; return ref; } diff --git a/ref-filter.h b/ref-filter.h index 053980a6a42..a93d5e4dd61 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -39,6 +39,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; const char *rest; + int cat_file_cmdmode; int flag; unsigned int kind; const char *symref; From patchwork Thu Jul 15 15:40:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DB66C07E96 for ; Thu, 15 Jul 2021 15:40:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 19828613DB for ; Thu, 15 Jul 2021 15:40:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239426AbhGOPnk (ORCPT ); Thu, 15 Jul 2021 11:43:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239179AbhGOPnc (ORCPT ); Thu, 15 Jul 2021 11:43:32 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DB6EC06175F for ; Thu, 15 Jul 2021 08:40:39 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id a5-20020a7bc1c50000b02901e3bbe0939bso3965855wmj.0 for ; Thu, 15 Jul 2021 08:40:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=4Ib0SWYpLaPximun7MUajcXT3GjLWT38oNkCGu16y08=; b=kkXc34dA2A4B4rvRIldrPJLWemUemgJzWlgV1nehTuA0MKeT8nyQ/FEUpcZ1lD/ho4 7UTkbsZW5+hsXPQD9INj8+s1GgVsyiiQhmuJ6vKvpkHfUsl43Z7w8S69/wsIliuf5eob 9Px9dwAXb4NQXh96o2ETRHFv/1CoJ+KROeLZz69PUJ7tqXjG5hQl7VQjvbCOHiDNZIDi BEo1HbMh2MiljCMVmxIsgZSuiXeD528SBP3cW+qwk/Y8FdOcmOHVFXhYNndezdB2+b0v sRGOiiIR4o+RAOTQMGj46AOXkWccvepTYfApleDUz5aFRjvuetP6xzb3UibvwgYXhNI7 kVxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=4Ib0SWYpLaPximun7MUajcXT3GjLWT38oNkCGu16y08=; b=jMMay2hIpgB+5jqcurOQ6B8Alirf8QmnxlStK7AKjxhrhsMcTg5Opx8yw2NQd/Wvnh PxilGfaj18Kp42kgUBAatGwqIbliRWniDzFhu9wyqllEM7XNlqqlKAmj7agdX9ifeI8x BFmAXdlfitHiEblV2Sejd3WdPtu4S/0DaKYNiD4EknWSLx5qDvm0v/UPjyJNtCxPlvSO AlNyH/BgwoZFwc4P13/pAdh/XET0DhGgnUU+UmNUR0BVx9C8WOGmfdMmjj8KHKkJPmr4 oyec76NVJ1pbNiGDdQM3hHrghvItUZLiBOkygZwG5S/i96GO9SgaGveih4yJMO7EytNV CI+g== X-Gm-Message-State: AOAM533MTqpObPwPZ/BvPZ9fICZw9wKBLq+EcbaHesnc+rzA9F+x8Ggt wfJ81uKBZ9AQXpkL24UuPoOThPQiXG4= X-Google-Smtp-Source: ABdhPJw9OHMl4+mKCyXabEotdiYgO1DY3qQvWNR2G+ic+NwJJDk4WM1lNzahht/7yuTgbNWvOfOvWw== X-Received: by 2002:a7b:c30f:: with SMTP id k15mr11347905wmj.128.1626363638144; Thu, 15 Jul 2021 08:40:38 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g3sm6953548wru.95.2021.07.15.08.40.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:37 -0700 (PDT) Message-Id: <96ef6431a2b64bbf15a9fbe7755dc3e882db072e.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:24 +0000 Subject: [PATCH v2 16/17] [GSOC] ref-filter: remove grab_oid() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because "atom_type == ATOM_OBJECTNAME" implies the condition of `starts_with(name, "objectname")`, "atom_type == ATOM_TREE" implies the condition of `starts_with(name, "tree")`, so the check for `starts_with(name, field)` in grab_oid() is redundant. So Remove the grab_oid() from ref-filter, to reduce repeated check. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 91e26c9aba3..1c7287f1061 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1077,16 +1077,6 @@ static const char *do_grab_oid(const char *field, const struct object_id *oid, } } -static int grab_oid(const char *name, const char *field, const struct object_id *oid, - struct atom_value *v, struct used_atom *atom) -{ - if (starts_with(name, field)) { - v->s = xstrdup(do_grab_oid(field, oid, atom)); - return 1; - } - return 0; -} - /* See grab_values */ static void grab_common_values(struct atom_value *val, int deref, struct expand_data *oi) { @@ -1112,8 +1102,9 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ } } else if (atom_type == ATOM_DELTABASE) v->s = xstrdup(oid_to_hex(&oi->delta_base_oid)); - else if (atom_type == ATOM_OBJECTNAME && deref) - grab_oid(name, "objectname", &oi->oid, v, &used_atom[i]); + else if (atom_type == ATOM_OBJECTNAME && deref) { + v->s = xstrdup(do_grab_oid("objectname", &oi->oid, &used_atom[i])); + } } } @@ -1154,9 +1145,10 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object continue; if (deref) name++; - if (atom_type == ATOM_TREE && - grab_oid(name, "tree", get_commit_tree_oid(commit), v, &used_atom[i])) + if (atom_type == ATOM_TREE) { + v->s = xstrdup(do_grab_oid("tree", get_commit_tree_oid(commit), &used_atom[i])); continue; + } if (atom_type == ATOM_NUMPARENT) { v->value = commit_list_count(commit->parents); v->s = xstrfmt("%lu", (unsigned long)v->value); @@ -1959,9 +1951,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->s = xstrdup(buf + 1); } continue; - } else if (!deref && atom_type == ATOM_OBJECTNAME && - grab_oid(name, "objectname", &ref->objectname, v, atom)) { - continue; + } else if (!deref && atom_type == ATOM_OBJECTNAME) { + v->s = xstrdup(do_grab_oid("objectname", &ref->objectname, atom)); + continue; } else if (atom_type == ATOM_HEAD) { if (atom->u.head && !strcmp(ref->refname, atom->u.head)) v->s = xstrdup("*"); From patchwork Thu Jul 15 15:40:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12380347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94AFDC1B08C for ; Thu, 15 Jul 2021 15:40:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7EA5961374 for ; Thu, 15 Jul 2021 15:40:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239465AbhGOPnm (ORCPT ); Thu, 15 Jul 2021 11:43:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239286AbhGOPnd (ORCPT ); Thu, 15 Jul 2021 11:43:33 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 216A7C061768 for ; Thu, 15 Jul 2021 08:40:40 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id g16so8375688wrw.5 for ; Thu, 15 Jul 2021 08:40:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=IzJopMreD03mb0H5DzVSKa/vpC4rGACWQqE7utDtPrg=; b=nmmqPKfT6h8bOyszT0nDHGHoU78EbZGvJbaprQveR1BLqkJFCMPjwPHP8Gc8WlwwgZ laWBctZKGlgcux8tPce5+mc0QUZEizXdkotoalZPWG0EZqx+YB+7n2pUNW4KDlu3dhUG rZWLXBjEPHFiZ3T004ujq8QzHw77MZL1+UaSeibnHeygzW3/jQeaDfdz5MZ2E0NnT2zk PEE2RAIFyKMrY5zA32MqB7E6e0d4tMXYNi+1BQnZx7KwMRgRIqFvwxFgXZWNAPDpuB9p WpxTpjRoyeHvZ3eZcFf4WkdF0d5D+Edm7YQwNe3LBFMSrAsxgFTl/aSDPna44rF4wSCx 0+qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=IzJopMreD03mb0H5DzVSKa/vpC4rGACWQqE7utDtPrg=; b=UAwC/3ercuLl6GXldoCuQYKmtHOcdN/qcPihFNSoINBOWGFOF07TzUcjNGhnvxLHDm 8qSLGEmeYommKrrId778I6Nh2zLE2KeLBiLuoUl37PbCbcaDAX8bmhiaoSzKO1wr+BKZ XVI/GXzMSjFajA3jSyq5HCTOM/Xka8Hu+N/gaVvFUebwjsy6yMpdJlkh6xSOJKha40d/ bWrisySXzrziCK2CAP0GW3hSEry0Y3TL9W+IxBH5f24nhmc71ePRUTxy0KGBtIbaLNcU 4iJg+EdG9Veg/DiiR4DCWOeDU8nJhhYRx/ujBUUXBZ8We4XRB2Zk1MmHffLZFLPxzxGA bFbQ== X-Gm-Message-State: AOAM531Xg7Xs8dLNKykNwJVVgeKfXGrdCzIzpngg3qZbh0VLeIoaoWFg dj/UCOr6/DXweb5SOva6Idf1u1bWmeI= X-Google-Smtp-Source: ABdhPJwrnWaG2dlOaXoPs46zPnrVdRg0Ksn6ewr3r1ovbynbuYvr79SXjZCFelZk0zsv2dVym3zgbQ== X-Received: by 2002:adf:f74f:: with SMTP id z15mr6485222wrp.54.1626363638748; Thu, 15 Jul 2021 08:40:38 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z25sm5530128wmf.9.2021.07.15.08.40.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 08:40:38 -0700 (PDT) Message-Id: <5903d02324f3275b3aa442bb3ca2602564c543dc.1626363626.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 15 Jul 2021 15:40:25 +0000 Subject: [PATCH v2 17/17] [GSOC] cat-file: use fast path when using default_format MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add the member `default_format` to struct `batch_options`, when we are using the default format on `git cat-file --batch`, or `git cat-file --batch-check`, `default_format` will be set, if we don't use `--textconv` or `--filter`, then we will not call verify_ref_format(), has_object_file() and format_ref_array_item(). Instead, we get the object data directly through oid_object_info_extended() and then output the data directly. By using this fast path, we can reduce some of the extra overhead when `cat-file --batch` using ref-filter. The running time of `git cat-file --batch-check` will be similar to before, and the running time of `git cat-file --batch` will be 9.1% less than before. Helped-by: Ævar Arnfjörð Bjarmason Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 79 +++++++++++++++++++++++++++++++++------------- 1 file changed, 57 insertions(+), 22 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 3a6153e778f..8edc19f2d5a 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -26,6 +26,7 @@ struct batch_options { int all_objects; int unordered; int cmdmode; /* may be 'w' or 'c' for --filters or --textconv */ + int default_format; struct ref_format format; }; @@ -196,6 +197,7 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct expand_data { struct object_id oid; + struct object_info info; const char *rest; int split_on_whitespace; }; @@ -216,27 +218,58 @@ static void batch_object_write(const char *obj_name, struct batch_options *opt, struct expand_data *data) { - int ret; - struct ref_array_item item = { data->oid, data->rest, opt->cmdmode }; - - strbuf_reset(scratch); - strbuf_reset(err); - - ret = format_ref_array_item(&item, &opt->format, scratch, err); - if (ret < 0) - die("%s\n", err->buf); - if (ret) { - /* ret > 0 means when the object corresponding to oid - * cannot be found in format_ref_array_item(), we only print - * the error message. - */ - printf("%s\n", err->buf); + if (opt->default_format && !opt->cmdmode) { + struct strbuf type_name = STRBUF_INIT; + unsigned long size; + void *content; + + if (opt->print_contents) + data->info.contentp = &content; + + data->info.type_name = &type_name; + data->info.sizep = &size; + + if (oid_object_info_extended(the_repository, &data->oid, &data->info, + OBJECT_INFO_LOOKUP_REPLACE) < 0) { + printf("%s missing\n", + obj_name ? obj_name : oid_to_hex(&data->oid)); + fflush(stdout); + return; + } + + fprintf(stdout, "%s %s %"PRIuMAX"\n", oid_to_hex(&data->oid), + data->info.type_name->buf, + (uintmax_t)*data->info.sizep); fflush(stdout); + strbuf_release(&type_name); + if (opt->print_contents) { + batch_write(opt, content, *data->info.sizep); + batch_write(opt, "\n", 1); + free(content); + } } else { - strbuf_addch(scratch, '\n'); - batch_write(opt, scratch->buf, scratch->len); + int ret; + struct ref_array_item item = { data->oid, data->rest, opt->cmdmode }; + + strbuf_reset(scratch); + strbuf_reset(err); + + ret = format_ref_array_item(&item, &opt->format, scratch, err); + if (ret < 0) + die("%s\n", err->buf); + if (ret) { + /* ret > 0 means when the object corresponding to oid + * cannot be found in format_ref_array_item(), we only print + * the error message. + */ + printf("%s\n", err->buf); + fflush(stdout); + } else { + strbuf_addch(scratch, '\n'); + batch_write(opt, scratch->buf, scratch->len); + } + free_ref_array_item_value(&item); } - free_ref_array_item_value(&item); } static void batch_one_object(const char *obj_name, @@ -288,7 +321,7 @@ static void batch_one_object(const char *obj_name, return; } - if (!has_object_file(&data->oid)) { + if ((!opt->default_format || opt->cmdmode) && !has_object_file(&data->oid)) { printf("%s missing\n", obj_name ? obj_name : oid_to_hex(&data->oid)); fflush(stdout); @@ -380,7 +413,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio if (batch->print_contents) strbuf_addstr(&format, "\n%(raw)"); batch->format.format = format.buf; - if (verify_ref_format(&batch->format)) + if ((!batch->default_format || batch->cmdmode) && verify_ref_format(&batch->format)) usage_with_options(cat_file_usage, options); if (batch->cmdmode || batch->format.use_rest) @@ -483,7 +516,8 @@ static int batch_option_callback(const struct option *opt, bo->enabled = 1; bo->print_contents = !strcmp(opt->long_name, "batch"); bo->format.format = arg; - + if (arg) + bo->default_format = 0; return 0; } @@ -492,7 +526,8 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) int opt = 0; const char *exp_type = NULL, *obj_name = NULL; struct batch_options batch = { - .format = REF_FORMAT_INIT + .format = REF_FORMAT_INIT, + .default_format = 1 }; int unknown_type = 0;