From patchwork Fri Jun 25 16:02:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 109ABC48BC2 for ; Fri, 25 Jun 2021 16:02:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EA0A06196E for ; Fri, 25 Jun 2021 16:02:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229915AbhFYQE5 (ORCPT ); Fri, 25 Jun 2021 12:04:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229926AbhFYQEv (ORCPT ); Fri, 25 Jun 2021 12:04:51 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39BC1C061766 for ; Fri, 25 Jun 2021 09:02:29 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id a13so11113606wrf.10 for ; Fri, 25 Jun 2021 09:02:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=/+pHTUCgsn/eJhBt7TH6g2FT5CeygqVH9py2mRcVtgU=; b=Kus69Mx+Vi+MgbdHY15NQFME4jhAQQ6kbXUA4FZM0wsPc9hEvVYdrCf7lcJDSWlyLE 8Nirh1wnkTgQD965nlf49gsh4bx34Ob0vLtWZeoHcWVbNi+aZ1Pm6pXDiSccCxsPHam3 qZawr8kJmyz5CVl9kB5SRfQyYHVUanXGqY4TW9uFREYKtDogH5iuXlxbZZfjpoBaKBX1 czL6S8EyAlf4UyGdfxiR0RnUzQhyXhIzyZy2iazzuxWY3IIru0XST7I1KSQ16UUDvp3E k86u9phj4ESZgxTM0MrOcdlPY7i2sEc191eBWnehlENEf/WDka1ydekZ4k3IyOAz6MsB pp/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=/+pHTUCgsn/eJhBt7TH6g2FT5CeygqVH9py2mRcVtgU=; b=Xks1P9CVj5hM0CBzlfKcR0fWv8npae+o6xMeOi0XUz9RXOesLzas9i6/oUKtKHULiw wPuFfOKywwNbVk7YGewK0bkD5/TKwsFu0Jj30pFHujppbsxG5xTaL4JmbUvkDhSb9NHT M/vmHH2fGHa2laQ6yUkP+ET/9fjlhVNkoABiomWii3SQgu8ZKEVhLmVpTkKD+aJwg7N1 w5puIM14sJuhEExWo/wMad8yV8xi7Xmn+egLoi+jYDajdzVyIAEpg2q0KDMB/YAdkijd BH2JC4/yGFNtU6rktVgZu81gJfAqhFLZxZlXvj09DXupWkigN3lIht/ZEeRRccEK59W1 HnMQ== X-Gm-Message-State: AOAM53370h0BmHCexAhxW6nKqsQV3RfN8p5aLroNXlVh5T0kYRgNNKZB CLuGtEfZb+PAdGQs3JQRHTSmqB4TpQI= X-Google-Smtp-Source: ABdhPJz4Wk8jDdPwz+xT9r3kLJj/HYht2JRUqCvWd5Xfe2ynaLFGOYyCcZdOw6sgmfRRyhqjsGmQmw== X-Received: by 2002:a5d:440a:: with SMTP id z10mr12082897wrq.269.1624636947865; Fri, 25 Jun 2021 09:02:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o203sm6785960wmo.36.2021.06.25.09.02.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:27 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:11 +0000 Subject: [PATCH v5 01/15] [GSOC] ref-filter: add obj-type check in grab contents Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Only tag and commit objects use `grab_sub_body_contents()` to grab object contents in the current codebase. We want to teach the function to also handle blobs and trees to get their raw data, without parsing a blob (whose contents looks like a commit or a tag) incorrectly as a commit or a tag. Skip the block of code that is specific to handling commits and tags early when the given object is of a wrong type to help later addition to handle other types of objects in this function. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Junio C Hamano Signed-off-by: ZheNing Hu --- ref-filter.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 4db0e40ff4c..5cee6512fba 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1356,11 +1356,12 @@ static void append_lines(struct strbuf *out, const char *buf, unsigned long size } /* See grab_values */ -static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) +static void grab_sub_body_contents(struct atom_value *val, int deref, struct expand_data *data) { int i; const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; + void *buf = data->content; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; @@ -1371,10 +1372,13 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) continue; if (deref) name++; - if (strcmp(name, "body") && - !starts_with(name, "subject") && - !starts_with(name, "trailers") && - !starts_with(name, "contents")) + + if ((data->type != OBJ_TAG && + data->type != OBJ_COMMIT) || + (strcmp(name, "body") && + !starts_with(name, "subject") && + !starts_with(name, "trailers") && + !starts_with(name, "contents"))) continue; if (!subpos) find_subpos(buf, @@ -1438,17 +1442,19 @@ static void fill_missing_values(struct atom_value *val) * pointed at by the ref itself; otherwise it is the object the * ref (which is a tag) refers to. */ -static void grab_values(struct atom_value *val, int deref, struct object *obj, void *buf) +static void grab_values(struct atom_value *val, int deref, struct object *obj, struct expand_data *data) { + void *buf = data->content; + switch (obj->type) { case OBJ_TAG: grab_tag_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("tagger", val, deref, buf); break; case OBJ_COMMIT: grab_commit_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("author", val, deref, buf); grab_person("committer", val, deref, buf); break; @@ -1678,7 +1684,7 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi->content); + grab_values(ref->value, deref, *obj, oi); } grab_common_values(ref->value, deref, oi); From patchwork Fri Jun 25 16:02:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAA74C49EA7 for ; Fri, 25 Jun 2021 16:02:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B76406197B for ; Fri, 25 Jun 2021 16:02:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230019AbhFYQE4 (ORCPT ); Fri, 25 Jun 2021 12:04:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229929AbhFYQEv (ORCPT ); Fri, 25 Jun 2021 12:04:51 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 330B2C061574 for ; Fri, 25 Jun 2021 09:02:30 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id k30-20020a05600c1c9eb02901d4d33c5ca0so734799wms.3 for ; Fri, 25 Jun 2021 09:02:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=pw89SO79RGppVLXRuv0VvM0SQU6sUz31Mu8ZDayml4k=; b=SDJ3M0F2Khd/G2afhrd6hHksp4ocx7awie+0dc0+DPRVoABL0QV6kYrcYWHO7zXvhY s+YZONj22ldZ4M13KdM3JZLPb3I6Mc2r9JdYcpys6MtfAhNqqbPKzOTaYdd/t7yT3gP4 5aO2zSCejLtLuBvYyquEWBtShvKkz4sUBdgQrors94ACNeX7C+Xq03J/aA8wwf49NQ97 RND2zpi5tw/3Z/IsjGEFmdcrvvy5QCZDNpwscstApn13qG6GlHJ8F94d6oYfNh/h7Pv6 LJ1B3BPSEypmmSVDdkLYszVpni6TkKTb5O6UBanPMxAuOyTNBuhudf1BztZUGoaT3376 5Yig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=pw89SO79RGppVLXRuv0VvM0SQU6sUz31Mu8ZDayml4k=; b=cwYrFAce16bYxoa8/wT3lHbPNC7u3ssEZ1rtHaVQ22GOQSamM6KK+nFieEhB9FNpZ7 ZyMYdN+000neInuRNB7ptZSooQzYIRHKSppB19ZJc/V4dnP6r0hMzhpIqOzHneNMgOXK bQM21+iWcs2MZbl1opyeV5ohWJa9K14tGqnjOJ3mIzii7qwNx2JM9TOZ2R40eGhG8k/d d44tW3uG7wHY7IJRl8S08sS77EpXVk1UnyFzfIuJS1D8AXrLTcsbK2pVAQUiwcdMIIbI I3by8WzHjmyPzF/f8BvPuTfsG07JXo0M98bO0a3TmWOXIZFUWYPrC3QxnNJcumonfoK6 b9og== X-Gm-Message-State: AOAM532bl1l3GUes3cuHgJo4UsWjpT64NR2BmD6I8JIXilsd32uBwhhI AWLZMDN6YLsvDXEdFL67g5svmhObgX4= X-Google-Smtp-Source: ABdhPJxULuDKEwg9tu+SrHu+UMFcg+0gNG4JfVJv2CPq8L64F03FNz5odnkQHyPLoDmXcIcSilfHNg== X-Received: by 2002:a1c:5413:: with SMTP id i19mr11883038wmb.12.1624636948565; Fri, 25 Jun 2021 09:02:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y16sm6513354wrp.51.2021.06.25.09.02.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:28 -0700 (PDT) Message-Id: <4e473838b9d2651a8e4be27332697c2ba354db5a.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:12 +0000 Subject: [PATCH v5 02/15] [GSOC] ref-filter: add %(raw) atom MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add new formatting option `%(raw)`, which will print the raw object data without any changes. It will help further to migrate all cat-file formatting logic from cat-file to ref-filter. The raw data of blob, tree objects may contain '\0', but most of the logic in `ref-filter` depends on the output of the atom being text (specifically, no embedded NULs in it). E.g. `quote_formatting()` use `strbuf_addstr()` or `*._quote_buf()` add the data to the buffer. The raw data of a tree object is `100644 one\0...`, only the `100644 one` will be added to the buffer, which is incorrect. Therefore, we need to find a way to record the length of the atom_value's member `s`. Although strbuf can already record the string and its length, if we want to replace the type of atom_value's member `s` with strbuf, many places in ref-filter that are filled with dynamically allocated mermory in `v->s` are not easy to replace. At the same time, we need to check if `v->s == NULL` in populate_value(), and strbuf cannot easily distinguish NULL and empty strings, but c-style "const char *" can do it. So add a new member in `struct atom_value`: `s_size`, which can record raw object size, it can help us add raw object data to the buffer or compare two buffers which contain raw object data. Note that `--format=%(raw)` cannot be used with `--python`, `--shell`, `--tcl`, and `--perl` because if the binary raw data is passed to a variable in such languages, these may not support arbitrary binary data in their string variable type. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Felipe Contreras Helped-by: Phillip Wood Helped-by: Junio C Hamano Based-on-patch-by: Olga Telezhnaya Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 9 ++ ref-filter.c | 139 +++++++++++++++---- t/t6300-for-each-ref.sh | 216 +++++++++++++++++++++++++++++ 3 files changed, 337 insertions(+), 27 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 2ae2478de70..7f1f0a1ca3b 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -235,6 +235,15 @@ and `date` to extract the named component. For email fields (`authoremail`, without angle brackets, and `:localpart` to get the part before the `@` symbol out of the trimmed email. +The raw data in an object is `raw`. + +raw:size:: + The raw data size of the object. + +Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, +`--perl` because the host language may not support arbitrary binary data in the +variables of its string type. + The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/ref-filter.c b/ref-filter.c index 5cee6512fba..7822be90307 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -144,6 +144,7 @@ enum atom_type { ATOM_BODY, ATOM_TRAILERS, ATOM_CONTENTS, + ATOM_RAW, ATOM_UPSTREAM, ATOM_PUSH, ATOM_SYMREF, @@ -189,6 +190,9 @@ static struct used_atom { struct process_trailer_options trailer_opts; unsigned int nlines; } contents; + struct { + enum { RAW_BARE, RAW_LENGTH } option; + } raw_data; struct { cmp_status cmp_status; const char *str; @@ -426,6 +430,18 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } +static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (!arg) + atom->u.raw_data.option = RAW_BARE; + else if (!strcmp(arg, "size")) + atom->u.raw_data.option = RAW_LENGTH; + else + return strbuf_addf_ret(err, -1, _("unrecognized %%(raw) argument: %s"), arg); + return 0; +} + static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { @@ -586,6 +602,7 @@ static struct { [ATOM_BODY] = { "body", SOURCE_OBJ, FIELD_STR, body_atom_parser }, [ATOM_TRAILERS] = { "trailers", SOURCE_OBJ, FIELD_STR, trailers_atom_parser }, [ATOM_CONTENTS] = { "contents", SOURCE_OBJ, FIELD_STR, contents_atom_parser }, + [ATOM_RAW] = { "raw", SOURCE_OBJ, FIELD_STR, raw_atom_parser }, [ATOM_UPSTREAM] = { "upstream", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_PUSH] = { "push", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_SYMREF] = { "symref", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -620,12 +637,15 @@ struct ref_formatting_state { struct atom_value { const char *s; + size_t s_size; int (*handler)(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err); uintmax_t value; /* used for sorting when not FIELD_STR */ struct used_atom *atom; }; +#define ATOM_VALUE_S_SIZE_INIT (-1) + /* * Used to parse format string and sort specifiers */ @@ -644,13 +664,6 @@ static int parse_ref_filter_atom(const struct ref_format *format, return strbuf_addf_ret(err, -1, _("malformed field name: %.*s"), (int)(ep-atom), atom); - /* Do we have the atom already used elsewhere? */ - for (i = 0; i < used_atom_cnt; i++) { - int len = strlen(used_atom[i].name); - if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) - return i; - } - /* * If the atom name has a colon, strip it and everything after * it off - it specifies the format for this entry, and @@ -660,6 +673,13 @@ static int parse_ref_filter_atom(const struct ref_format *format, arg = memchr(sp, ':', ep - sp); atom_len = (arg ? arg : ep) - sp; + /* Do we have the atom already used elsewhere? */ + for (i = 0; i < used_atom_cnt; i++) { + int len = strlen(used_atom[i].name); + if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) + return i; + } + /* Is the atom a valid one? */ for (i = 0; i < ARRAY_SIZE(valid_atom); i++) { int len = strlen(valid_atom[i].name); @@ -709,11 +729,14 @@ static int parse_ref_filter_atom(const struct ref_format *format, return at; } -static void quote_formatting(struct strbuf *s, const char *str, int quote_style) +static void quote_formatting(struct strbuf *s, const char *str, size_t len, int quote_style) { switch (quote_style) { case QUOTE_NONE: - strbuf_addstr(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(s, str, len); + else + strbuf_addstr(s, str); break; case QUOTE_SHELL: sq_quote_buf(s, str); @@ -740,9 +763,12 @@ static int append_atom(struct atom_value *v, struct ref_formatting_state *state, * encountered. */ if (!state->stack->prev) - quote_formatting(&state->stack->output, v->s, state->quote_style); + quote_formatting(&state->stack->output, v->s, v->s_size, state->quote_style); else - strbuf_addstr(&state->stack->output, v->s); + if (v->s_size != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(&state->stack->output, v->s, v->s_size); + else + strbuf_addstr(&state->stack->output, v->s); return 0; } @@ -842,21 +868,23 @@ static int if_atom_handler(struct atom_value *atomv, struct ref_formatting_state return 0; } -static int is_empty(const char *s) +static int is_empty(struct strbuf *buf) { - while (*s != '\0') { - if (!isspace(*s)) - return 0; - s++; - } - return 1; -} + const char *cur = buf->buf; + const char *end = buf->buf + buf->len; + + while (cur != end && (isspace(*cur))) + cur++; + + return cur == end; + } static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err) { struct ref_formatting_stack *cur = state->stack; struct if_then_else *if_then_else = NULL; + size_t str_len = 0; if (cur->at_end == if_then_else_handler) if_then_else = (struct if_then_else *)cur->at_end_data; @@ -867,18 +895,22 @@ static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_sta if (if_then_else->else_atom_seen) return strbuf_addf_ret(err, -1, _("format: %%(then) atom used after %%(else)")); if_then_else->then_atom_seen = 1; + if (if_then_else->str) + str_len = strlen(if_then_else->str); /* * If the 'equals' or 'notequals' attribute is used then * perform the required comparison. If not, only non-empty * strings satisfy the 'if' condition. */ if (if_then_else->cmp_status == COMPARE_EQUAL) { - if (!strcmp(if_then_else->str, cur->output.buf)) + if (str_len == cur->output.len && + !memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; } else if (if_then_else->cmp_status == COMPARE_UNEQUAL) { - if (strcmp(if_then_else->str, cur->output.buf)) + if (str_len != cur->output.len || + memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; - } else if (cur->output.len && !is_empty(cur->output.buf)) + } else if (cur->output.len && !is_empty(&cur->output)) if_then_else->condition_satisfied = 1; strbuf_reset(&cur->output); return 0; @@ -924,7 +956,7 @@ static int end_atom_handler(struct atom_value *atomv, struct ref_formatting_stat * only on the topmost supporting atom. */ if (!current->prev->prev) { - quote_formatting(&s, current->output.buf, state->quote_style); + quote_formatting(&s, current->output.buf, current->output.len, state->quote_style); strbuf_swap(¤t->output, &s); } strbuf_release(&s); @@ -974,6 +1006,10 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) + die(_("--format=%.*s cannot be used with" + "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) @@ -1362,17 +1398,29 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; void *buf = data->content; + unsigned long buf_size = data->size; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; const char *name = atom->name; struct atom_value *v = &val[i]; + enum atom_type atom_type = atom->atom_type; if (!!deref != (*name == '*')) continue; if (deref) name++; + if (atom_type == ATOM_RAW) { + if (atom->u.raw_data.option == RAW_BARE) { + v->s = xmemdupz(buf, buf_size); + v->s_size = buf_size; + } else if (atom->u.raw_data.option == RAW_LENGTH) { + v->s = xstrfmt("%"PRIuMAX, (uintmax_t)buf_size); + } + continue; + } + if ((data->type != OBJ_TAG && data->type != OBJ_COMMIT) || (strcmp(name, "body") && @@ -1460,9 +1508,11 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s break; case OBJ_TREE: /* grab_tree_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; case OBJ_BLOB: /* grab_blob_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; default: die("Eh? Object of type %d?", obj->type); @@ -1766,6 +1816,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) const char *refname; struct branch *branch = NULL; + v->s_size = ATOM_VALUE_S_SIZE_INIT; v->handler = append_atom; v->atom = atom; @@ -2369,6 +2420,19 @@ static int compare_detached_head(struct ref_array_item *a, struct ref_array_item return 0; } +static int memcasecmp(const void *vs1, const void *vs2, size_t n) +{ + const char *s1 = vs1, *s2 = vs2; + const char *end = s1 + n; + + for (; s1 < end; s1++, s2++) { + int diff = tolower(*s1) - tolower(*s2); + if (diff) + return diff; + } + return 0; +} + static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, struct ref_array_item *b) { struct atom_value *va, *vb; @@ -2389,10 +2453,30 @@ static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, stru } else if (s->sort_flags & REF_SORTING_VERSION) { cmp = versioncmp(va->s, vb->s); } else if (cmp_type == FIELD_STR) { - int (*cmp_fn)(const char *, const char *); - cmp_fn = s->sort_flags & REF_SORTING_ICASE - ? strcasecmp : strcmp; - cmp = cmp_fn(va->s, vb->s); + if (va->s_size == ATOM_VALUE_S_SIZE_INIT && + vb->s_size == ATOM_VALUE_S_SIZE_INIT) { + int (*cmp_fn)(const char *, const char *); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? strcasecmp : strcmp; + cmp = cmp_fn(va->s, vb->s); + } else { + size_t a_size = va->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(va->s) : va->s_size; + size_t b_size = vb->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(vb->s) : vb->s_size; + int (*cmp_fn)(const void *, const void *, size_t); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? memcasecmp : memcmp; + + cmp = cmp_fn(va->s, vb->s, b_size > a_size ? + a_size : b_size); + if (!cmp) { + if (a_size > b_size) + cmp = 1; + else if (a_size < b_size) + cmp = -1; + } + } } else { if (va->value < vb->value) cmp = -1; @@ -2492,6 +2576,7 @@ int format_ref_array_item(struct ref_array_item *info, } if (format->need_color_reset_at_eol) { struct atom_value resetv; + resetv.s_size = ATOM_VALUE_S_SIZE_INIT; resetv.s = GIT_COLOR_RESET; if (append_atom(&resetv, &state, error_buf)) { pop_stack_element(&state.stack); diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 9e0214076b4..9c5379e2f56 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -130,6 +130,8 @@ test_atom head parent:short=10 '' test_atom head numparent 0 test_atom head object '' test_atom head type '' +test_atom head raw "$(git cat-file commit refs/heads/main) +" test_atom head '*objectname' '' test_atom head '*objecttype' '' test_atom head author 'A U Thor 1151968724 +0200' @@ -221,6 +223,15 @@ test_atom tag contents 'Tagging at 1151968727 ' test_atom tag HEAD ' ' +test_expect_success 'basic atom: refs/tags/testtag *raw' ' + git cat-file commit refs/tags/testtag^{} >expected && + git for-each-ref --format="%(*raw)" refs/tags/testtag >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'Check invalid atoms names are errors' ' test_must_fail git for-each-ref --format="%(INVALID)" refs/heads ' @@ -686,6 +697,15 @@ test_atom refs/tags/signed-empty contents:body '' test_atom refs/tags/signed-empty contents:signature "$sig" test_atom refs/tags/signed-empty contents "$sig" +test_expect_success GPG 'basic atom: refs/tags/signed-empty raw' ' + git cat-file tag refs/tags/signed-empty >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-empty >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-short subject 'subject line' test_atom refs/tags/signed-short subject:sanitize 'subject-line' test_atom refs/tags/signed-short contents:subject 'subject line' @@ -695,6 +715,15 @@ test_atom refs/tags/signed-short contents:signature "$sig" test_atom refs/tags/signed-short contents "subject line $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-short raw' ' + git cat-file tag refs/tags/signed-short >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-short >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-long subject 'subject line' test_atom refs/tags/signed-long subject:sanitize 'subject-line' test_atom refs/tags/signed-long contents:subject 'subject line' @@ -708,6 +737,15 @@ test_atom refs/tags/signed-long contents "subject line body contents $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-long raw' ' + git cat-file tag refs/tags/signed-long >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-long >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'set up refs pointing to tree and blob' ' git update-ref refs/mytrees/first refs/heads/main^{tree} && git update-ref refs/myblobs/first refs/heads/main:one @@ -720,6 +758,16 @@ test_atom refs/mytrees/first contents:body "" test_atom refs/mytrees/first contents:signature "" test_atom refs/mytrees/first contents "" +test_expect_success 'basic atom: refs/mytrees/first raw' ' + git cat-file tree refs/mytrees/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/mytrees/first >actual && + test_cmp expected actual && + git cat-file -s refs/mytrees/first >expected && + git for-each-ref --format="%(raw:size)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_atom refs/myblobs/first subject "" test_atom refs/myblobs/first contents:subject "" test_atom refs/myblobs/first body "" @@ -727,6 +775,174 @@ test_atom refs/myblobs/first contents:body "" test_atom refs/myblobs/first contents:signature "" test_atom refs/myblobs/first contents "" +test_expect_success 'basic atom: refs/myblobs/first raw' ' + git cat-file blob refs/myblobs/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/myblobs/first >actual && + test_cmp expected actual && + git cat-file -s refs/myblobs/first >expected && + git for-each-ref --format="%(raw:size)" refs/myblobs/first >actual && + test_cmp expected actual +' + +test_expect_success 'set up refs pointing to binary blob' ' + printf "a\0b\0c" >blob1 && + printf "a\0c\0b" >blob2 && + printf "\0a\0b\0c" >blob3 && + printf "abc" >blob4 && + printf "\0 \0 \0 " >blob5 && + printf "\0 \0a\0 " >blob6 && + printf " " >blob7 && + >blob8 && + obj=$(git hash-object -w blob1) && + git update-ref refs/myblobs/blob1 "$obj" && + obj=$(git hash-object -w blob2) && + git update-ref refs/myblobs/blob2 "$obj" && + obj=$(git hash-object -w blob3) && + git update-ref refs/myblobs/blob3 "$obj" && + obj=$(git hash-object -w blob4) && + git update-ref refs/myblobs/blob4 "$obj" && + obj=$(git hash-object -w blob5) && + git update-ref refs/myblobs/blob5 "$obj" && + obj=$(git hash-object -w blob6) && + git update-ref refs/myblobs/blob6 "$obj" && + obj=$(git hash-object -w blob7) && + git update-ref refs/myblobs/blob7 "$obj" && + obj=$(git hash-object -w blob8) && + git update-ref refs/myblobs/blob8 "$obj" +' + +test_expect_success 'Verify sorts with raw' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob3 + refs/myblobs/blob7 + refs/mytrees/first + refs/myblobs/first + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob4 + refs/heads/main + EOF + git for-each-ref --format="%(refname)" --sort=raw \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'Verify sorts with raw:size' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/first + refs/myblobs/blob7 + refs/heads/main + refs/myblobs/blob4 + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/mytrees/first + EOF + git for-each-ref --format="%(refname)" --sort=raw:size \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:equals)' ' + cat >expected <<-EOF && + not equals + not equals + not equals + not equals + not equals + not equals + refs/myblobs/blob4 + not equals + not equals + not equals + not equals + not equals + EOF + git for-each-ref --format="%(if:equals=abc)%(raw)%(then)%(refname)%(else)not equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:notequals)' ' + cat >expected <<-EOF && + refs/heads/ambiguous + refs/heads/main + refs/heads/newtag + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + equals + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob7 + refs/myblobs/blob8 + refs/myblobs/first + EOF + git for-each-ref --format="%(if:notequals=abc)%(raw)%(then)%(refname)%(else)equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'empty raw refs with %(if)' ' + cat >expected <<-EOF && + refs/myblobs/blob1 not empty + refs/myblobs/blob2 not empty + refs/myblobs/blob3 not empty + refs/myblobs/blob4 not empty + refs/myblobs/blob5 not empty + refs/myblobs/blob6 not empty + refs/myblobs/blob7 empty + refs/myblobs/blob8 empty + refs/myblobs/first not empty + EOF + git for-each-ref --format="%(refname) %(if)%(raw)%(then)not empty%(else)empty%(end)" \ + refs/myblobs/ >actual && + test_cmp expected actual +' + +test_expect_success '%(raw) with --python must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --python +' + +test_expect_success '%(raw) with --tcl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --tcl +' + +test_expect_success '%(raw) with --perl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --perl +' + +test_expect_success '%(raw) with --shell must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --shell +' + +test_expect_success '%(raw) with --shell and --sort=raw must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --sort=raw --shell +' + +test_expect_success '%(raw:size) with --shell' ' + git for-each-ref --format="%(raw:size)" | while read line + do + echo "'\''$line'\''" >>expect + done && + git for-each-ref --format="%(raw:size)" --shell >actual && + test_cmp expect actual +' + +test_expect_success 'for-each-ref --format compare with cat-file --batch' ' + git rev-parse refs/mytrees/first | git cat-file --batch >expected && + git for-each-ref --format="%(objectname) %(objecttype) %(objectsize) +%(raw)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_expect_success 'set up multiple-sort tags' ' for when in 100000 200000 do From patchwork Fri Jun 25 16:02:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 915FBC2B9F4 for ; Fri, 25 Jun 2021 16:02:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 789C661963 for ; Fri, 25 Jun 2021 16:02:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229630AbhFYQE6 (ORCPT ); Fri, 25 Jun 2021 12:04:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229944AbhFYQEw (ORCPT ); Fri, 25 Jun 2021 12:04:52 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF98AC061767 for ; Fri, 25 Jun 2021 09:02:30 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id h11so11122755wrx.5 for ; Fri, 25 Jun 2021 09:02:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=dI5PvriHjBHKZjIH7QTxTCcyv+x4/TrtTfZKy7Gxc5U=; b=V1gITNVAAWHDmTTPxXqwkOdUAvv5QKE7ByBHwjqzNOUXIud2UuepEGImEewSLXCaux 8a6VT9Jf29NP/RoJ6gb2CZI28IYSrtFS6g1N2dvNDel5hAHj/SGu/cqo9PFTzndLkb1o aG6ClMltD08XrOEejMMpN2dNZa65iYzpf3Z85MQzKmT5LNEXE2Rm6kNp6U1C0gTHsY+0 +tKL+twbRDJbAw/N5sUglTxS0TeHQxiNJXXym5CiaWv55s0GTNSWE1qi3WevTuOMG+PY z7zrQ1fHZA/UWXpzfiKUdmxQjYCYjC9xD5sw/9R2ZgAMdE8tnjAViBqw6FVb+Q7bTGHy 7Tfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=dI5PvriHjBHKZjIH7QTxTCcyv+x4/TrtTfZKy7Gxc5U=; b=sCAT+ba4cT2IjQC94LhVB3dvhma7hKtHYBC3mu0jV16vvHWbw1n1YnbevwQRLkFGDH m/Sdq+daq6d4TIMaKZJmA6h1XZszlKv7irrExDUHLmD4+9aGYbH4cdBWojicUQkGYWxi zslVqQ4bsnxSRyQQM/PIAuhpcIagZypv6dihsyVHpcB86JdU4QGXvjPRjI16sMqf0ky5 Mu0Iak+crkGj8qbPtLR3Fl9j+TVHdSXKg2fu18es7ORFbvuj0/Ov+zBgCfwQfT6xeesT xgqbxCgWaq/wvLTRE4TidSUsJ2OKGGlEAwPZXeMIf4lo1qgBp8v77bqjAypiZdWqceQh TV8w== X-Gm-Message-State: AOAM5332MDg+djWbVRZYEaAv9fGORpZrTE5OG4u62bCaetce8HlbpCH0 9HHiTdmt1hk3Mwa79vUVcsIiAgybCGo= X-Google-Smtp-Source: ABdhPJxRTO5bHP0ukIyoTQ8uC8Dd6LDW5OFJMCPGdmU9u0tbv3Y5HDaorYLU8Klf6R9yPtYBfaIVlw== X-Received: by 2002:a5d:6d8d:: with SMTP id l13mr10299097wrs.358.1624636949300; Fri, 25 Jun 2021 09:02:29 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r4sm6519003wre.84.2021.06.25.09.02.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:28 -0700 (PDT) Message-Id: <765cf08a108ffca4b096ac46f6be6fb26601f718.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:13 +0000 Subject: [PATCH v5 03/15] [GSOC] ref-filter: --format=%(raw) re-support --perl MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because the perl language can handle binary data correctly, add the function perl_quote_buf_with_len(), which can specify the length of the data and prevent the data from being truncated at '\0' to help `--format="%(raw)"` re-support `--perl`. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 2 +- quote.c | 17 +++++++++++++++++ quote.h | 1 + ref-filter.c | 15 +++++++++++---- t/t6300-for-each-ref.sh | 19 +++++++++++++++++-- 5 files changed, 47 insertions(+), 7 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 7f1f0a1ca3b..ea9b438c16f 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -241,7 +241,7 @@ raw:size:: The raw data size of the object. Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, -`--perl` because the host language may not support arbitrary binary data in the +because the host language may not support arbitrary binary data in the variables of its string type. The message in a commit or a tag object is `contents`, from which diff --git a/quote.c b/quote.c index 8a3a5e39eb1..26719d21d1e 100644 --- a/quote.c +++ b/quote.c @@ -471,6 +471,23 @@ void perl_quote_buf(struct strbuf *sb, const char *src) strbuf_addch(sb, sq); } +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len) +{ + const char sq = '\''; + const char bq = '\\'; + const char *c = src; + const char *end = src + len; + + strbuf_addch(sb, sq); + while (c != end) { + if (*c == sq || *c == bq) + strbuf_addch(sb, bq); + strbuf_addch(sb, *c); + c++; + } + strbuf_addch(sb, sq); +} + void python_quote_buf(struct strbuf *sb, const char *src) { const char sq = '\''; diff --git a/quote.h b/quote.h index 768cc6338e2..0fe69e264b0 100644 --- a/quote.h +++ b/quote.h @@ -94,6 +94,7 @@ char *quote_path(const char *in, const char *prefix, struct strbuf *out, unsigne /* quoting as a string literal for other languages */ void perl_quote_buf(struct strbuf *sb, const char *src); +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len); void python_quote_buf(struct strbuf *sb, const char *src); void tcl_quote_buf(struct strbuf *sb, const char *src); void basic_regex_quote_buf(struct strbuf *sb, const char *src); diff --git a/ref-filter.c b/ref-filter.c index 7822be90307..797b20ffa61 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -742,7 +742,10 @@ static void quote_formatting(struct strbuf *s, const char *str, size_t len, int sq_quote_buf(s, str); break; case QUOTE_PERL: - perl_quote_buf(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + perl_quote_buf_with_len(s, str, len); + else + perl_quote_buf(s, str); break; case QUOTE_PYTHON: python_quote_buf(s, str); @@ -1006,10 +1009,14 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && - used_atom[at].u.raw_data.option == RAW_BARE) + + if ((format->quote_style == QUOTE_PYTHON || + format->quote_style == QUOTE_SHELL || + format->quote_style == QUOTE_TCL) && + used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) die(_("--format=%.*s cannot be used with" - "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); + "--python, --shell, --tcl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 9c5379e2f56..5556063c347 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -915,8 +915,23 @@ test_expect_success '%(raw) with --tcl must fail' ' test_must_fail git for-each-ref --format="%(raw)" --tcl ' -test_expect_success '%(raw) with --perl must fail' ' - test_must_fail git for-each-ref --format="%(raw)" --perl +test_expect_success '%(raw) with --perl' ' + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob1 --perl | perl > actual && + cmp blob1 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob3 --perl | perl > actual && + cmp blob3 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob8 --perl | perl > actual && + cmp blob8 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/first --perl | perl > actual && + cmp one actual && + git cat-file tree refs/mytrees/first > expected && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/mytrees/first --perl | perl > actual && + cmp expected actual ' test_expect_success '%(raw) with --shell must fail' ' From patchwork Fri Jun 25 16:02:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 236EBC49EA7 for ; Fri, 25 Jun 2021 16:02:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0AA136196E for ; Fri, 25 Jun 2021 16:02:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230112AbhFYQFE (ORCPT ); Fri, 25 Jun 2021 12:05:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229962AbhFYQEy (ORCPT ); Fri, 25 Jun 2021 12:04:54 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6779BC061768 for ; Fri, 25 Jun 2021 09:02:31 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id p10-20020a05600c430ab02901df57d735f7so8778669wme.3 for ; Fri, 25 Jun 2021 09:02:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=KO8wLmUvbUfkVrtYPM8dGAeXgDDnPeti6Zejn/XfVtY=; b=HqNtGkzuaIFo8Ihg27fEl62YqtNWnzMAZfvhblBqFiA0OtcWWTq0ekFYfdCtvnNI// fMKxERaXcp2SKQthE2jAG2cigBJj9z01WTxBZBBMCyBlstbDTsiUI2Tib41hhvLcD5bh 2e5Kp3QsslaQkGqKKSkqbr9nIo09FYfnwOydRkOBDW9E5P68y60HDSHrk5TR2vv1w9ga M4kMTqqjqH+XYGS+A/Q6jqwOtMubdxv4Cgi5jQEAsn/2a8jtB3vulyM8mZGICwt+18RQ m21ntG64UUkyhQgdRNB1UMF1LA68ubnBiyv0dxHaE9TTAWvFHfgidtNF5eYnYN5v6c3R Bp/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=KO8wLmUvbUfkVrtYPM8dGAeXgDDnPeti6Zejn/XfVtY=; b=aoAGIPdkAcNjNeeeIlPu0jBxO1iKt/Yd897HJ/3ewYyxRf3sakY9MafCQfDRmmIygP F15TY9UBOj7+rBrmNPiXK59GosUBES8852T+vrS1art8eB1Walx1Evi28+8cG8sH9RRv 816hb2JGq4sGr1R4VDCLFPiGc6ff3LRW9VpET7GtdBlHqhCVgg4pRalXcAnZk8F1j7bl lhVb2HTG4Fgq4RVJPPjJB4K15+9nqGiDuZqFROtWU3q/OBOicCJUKE30HG/CtMz5g5dJ LpF4fBd4AKXmo+wf2iI9rovgCDaM5H/9qJTU8d3wUF4jNwnzsyMl4Qc6YXO/Dp262O0J vIJQ== X-Gm-Message-State: AOAM531GRtb2kEJNtFIcff8W3JAOw/5qqr9I4F/0ROLW8e46h9tpLGgJ I3N8e+uJci/+wksye+5mUXJUEs0NHLc= X-Google-Smtp-Source: ABdhPJwb+Ze2kCQ6r1hJfOS21+BgHXeTAUHqNyPMOqNU84ARqQBxxpYHl1mq6Eudyu9YoBwZ5wTzcQ== X-Received: by 2002:a05:600c:1c2a:: with SMTP id j42mr11169335wms.173.1624636950009; Fri, 25 Jun 2021 09:02:30 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r12sm7189371wrx.63.2021.06.25.09.02.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:29 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:14 +0000 Subject: [PATCH v5 04/15] [GSOC] ref-filter: use non-const ref_format in *_atom_parser() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use non-const ref_format in *_atom_parser(), which can help us modify the members of ref_format in *_atom_parser(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/tag.c | 2 +- ref-filter.c | 44 ++++++++++++++++++++++---------------------- ref-filter.h | 4 ++-- 3 files changed, 25 insertions(+), 25 deletions(-) diff --git a/builtin/tag.c b/builtin/tag.c index 82fcfc09824..452558ec957 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -146,7 +146,7 @@ static int verify_tag(const char *name, const char *ref, const struct object_id *oid, void *cb_data) { int flags; - const struct ref_format *format = cb_data; + struct ref_format *format = cb_data; flags = GPG_VERIFY_VERBOSE; if (format->format) diff --git a/ref-filter.c b/ref-filter.c index 797b20ffa61..d01a0266fb8 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -226,7 +226,7 @@ static int strbuf_addf_ret(struct strbuf *sb, int ret, const char *fmt, ...) return ret; } -static int color_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int color_atom_parser(struct ref_format *format, struct used_atom *atom, const char *color_value, struct strbuf *err) { if (!color_value) @@ -264,7 +264,7 @@ static int refname_atom_parser_internal(struct refname_atom *atom, const char *a return 0; } -static int remote_ref_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int remote_ref_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct string_list params = STRING_LIST_INIT_DUP; @@ -311,7 +311,7 @@ static int remote_ref_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objecttype_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objecttype_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -323,7 +323,7 @@ static int objecttype_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objectsize_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objectsize_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -343,7 +343,7 @@ static int objectsize_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int deltabase_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int deltabase_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -355,7 +355,7 @@ static int deltabase_atom_parser(const struct ref_format *format, struct used_at return 0; } -static int body_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int body_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -364,7 +364,7 @@ static int body_atom_parser(const struct ref_format *format, struct used_atom *a return 0; } -static int subject_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int subject_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -376,7 +376,7 @@ static int subject_atom_parser(const struct ref_format *format, struct used_atom return 0; } -static int trailers_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int trailers_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { atom->u.contents.trailer_opts.no_divider = 1; @@ -402,7 +402,7 @@ static int trailers_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int contents_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int contents_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -430,7 +430,7 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int raw_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -442,7 +442,7 @@ static int raw_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int oid_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -461,7 +461,7 @@ static int oid_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int person_email_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int person_email_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -475,7 +475,7 @@ static int person_email_atom_parser(const struct ref_format *format, struct used return 0; } -static int refname_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int refname_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { return refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err); @@ -492,7 +492,7 @@ static align_type parse_align_position(const char *s) return -1; } -static int align_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int align_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct align *align = &atom->u.align; @@ -544,7 +544,7 @@ static int align_atom_parser(const struct ref_format *format, struct used_atom * return 0; } -static int if_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int if_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -559,7 +559,7 @@ static int if_atom_parser(const struct ref_format *format, struct used_atom *ato return 0; } -static int head_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { atom->u.head = resolve_refdup("HEAD", RESOLVE_REF_READING, NULL, NULL); @@ -570,7 +570,7 @@ static struct { const char *name; info_source source; cmp_type cmp_type; - int (*parser)(const struct ref_format *format, struct used_atom *atom, + int (*parser)(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err); } valid_atom[] = { [ATOM_REFNAME] = { "refname", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -649,7 +649,7 @@ struct atom_value { /* * Used to parse format string and sort specifiers */ -static int parse_ref_filter_atom(const struct ref_format *format, +static int parse_ref_filter_atom(struct ref_format *format, const char *atom, const char *ep, struct strbuf *err) { @@ -2553,9 +2553,9 @@ static void append_literal(const char *cp, const char *ep, struct ref_formatting } int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, - struct strbuf *final_buf, - struct strbuf *error_buf) + struct ref_format *format, + struct strbuf *final_buf, + struct strbuf *error_buf) { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; @@ -2600,7 +2600,7 @@ int format_ref_array_item(struct ref_array_item *info, } void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format) + struct ref_format *format) { struct ref_array_item *ref_item; struct strbuf output = STRBUF_INIT; diff --git a/ref-filter.h b/ref-filter.h index baf72a71896..74fb423fc89 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -116,7 +116,7 @@ void ref_array_sort(struct ref_sorting *sort, struct ref_array *array); void ref_sorting_set_sort_flags_all(struct ref_sorting *sorting, unsigned int mask, int on); /* Based on the given format and quote_style, fill the strbuf */ int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, + struct ref_format *format, struct strbuf *final_buf, struct strbuf *error_buf); /* Parse a single sort specifier and add it to the list */ @@ -137,7 +137,7 @@ void setup_ref_filter_porcelain_msg(void); * name must be a fully qualified refname. */ void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format); + struct ref_format *format); /* * Push a single ref onto the array; this can be used to construct your own From patchwork Fri Jun 25 16:02:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0A45C2B9F4 for ; Fri, 25 Jun 2021 16:02:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9668661976 for ; Fri, 25 Jun 2021 16:02:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230104AbhFYQFA (ORCPT ); Fri, 25 Jun 2021 12:05:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229776AbhFYQEy (ORCPT ); Fri, 25 Jun 2021 12:04:54 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 115F7C061787 for ; Fri, 25 Jun 2021 09:02:32 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id a13so11113765wrf.10 for ; Fri, 25 Jun 2021 09:02:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=wiAr+9TcoDA+DITJvCWR545Yc3F75tfbyoCTpU6MrNM=; b=chMo0o5RYtrhnc5hiS5jYJKh0U2LFuIy2oQ2JXcelmd0+ZtK2thTwx4XZKDVR07YJj knepJ/Zp13a35fouLdLVgfvgZF9BIlziCKbzmVczPPUJyZkQwoSzwVMyodLZbnDg/TBy 8phfMV7Bxiqh3KEkYpNJmlZlHbHPwDR66f+RNf66WZPPxx3B5s+xDhfTHt2gPp6az1yr zTEUHKv1AiF0si91UfVPfEWWETzK8SmgBMSxqJpLFzrgdd7qAOdMfjrPHl/hOyAfWf3t k864Q/XXmFGjbJLQMyKZ8zWqU31EQegvR30jBNPI1sgU9kP4kP3StnUkzz/2T8rNkXs3 LUMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=wiAr+9TcoDA+DITJvCWR545Yc3F75tfbyoCTpU6MrNM=; b=BKmTqyhnqD7OEiXX/iT0gSXchKTyAlykjHhhjYAWdWSwp7kag6t394wXeal4j5zOHv 71NQsKPWoAml2PcEJVA94jyfZS/DzkoQIUmP136Sgn5nFXqovYwT6X3BtwfmRQeRGfSi YjSPVw6NlkhuNiVWVrcp9bKoicrxrQWj7zqZ9fAvgfAuNJ5uzcszreY/PqDHc+n3s14m wnEhaY27L5tWRbqrrqr6t7GiUaRu8Yk6e6iCGFbuZzoS136cvoRmH6H0a65LWMDkRGoh 29ZLDBmdLBLj3vlPekvxsHzXv3P8zILVCoQkxUv3aUNZIrCPxyUS7jyObWqT1XJY22J5 NwMg== X-Gm-Message-State: AOAM530EWlZuq5dwLAyYdRM3aY/T02SCZ7PkNAN9G+fdQaNIoJzlkPm/ tFwqr7ZR4Atl+Jp/e9+lmQQhmA826Y8= X-Google-Smtp-Source: ABdhPJx+CXXyKE6Aj6iyxxIDxFL3lzZiEB3Vh21rDagbBOKPDdm3N4R6DFlTpClyPjMPnOJKbv0AJQ== X-Received: by 2002:adf:e706:: with SMTP id c6mr1861062wrm.13.1624636950675; Fri, 25 Jun 2021 09:02:30 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z3sm11455929wmi.29.2021.06.25.09.02.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:30 -0700 (PDT) Message-Id: <1ca3a42f041087e3640ec7e3a8980798edaf9857.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:15 +0000 Subject: [PATCH v5 05/15] [GSOC] ref-filter: add %(rest) atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let "cat-file --batch=%(rest)" use the ref-filter interface, add %(rest) atom for ref-filter. "git for-each-ref", "git branch", "git tag" and "git verify-tag" will reject %(rest) by default. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 21 +++++++++++++++++++++ ref-filter.h | 5 ++++- t/t3203-branch-output.sh | 4 ++++ t/t6300-for-each-ref.sh | 4 ++++ t/t7004-tag.sh | 4 ++++ t/t7030-verify-tag.sh | 4 ++++ 6 files changed, 41 insertions(+), 1 deletion(-) diff --git a/ref-filter.c b/ref-filter.c index d01a0266fb8..10c78de9cfa 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -157,6 +157,7 @@ enum atom_type { ATOM_IF, ATOM_THEN, ATOM_ELSE, + ATOM_REST, }; /* @@ -559,6 +560,15 @@ static int if_atom_parser(struct ref_format *format, struct used_atom *atom, return 0; } +static int rest_atom_parser(struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (arg) + return strbuf_addf_ret(err, -1, _("%%(rest) does not take arguments")); + format->use_rest = 1; + return 0; +} + static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { @@ -615,6 +625,7 @@ static struct { [ATOM_IF] = { "if", SOURCE_NONE, FIELD_STR, if_atom_parser }, [ATOM_THEN] = { "then", SOURCE_NONE }, [ATOM_ELSE] = { "else", SOURCE_NONE }, + [ATOM_REST] = { "rest", SOURCE_NONE, FIELD_STR, rest_atom_parser }, /* * Please update $__git_ref_fieldlist in git-completion.bash * when you add new atoms @@ -1010,6 +1021,9 @@ int verify_ref_format(struct ref_format *format) if (at < 0) die("%s", err.buf); + if (used_atom[at].atom_type == ATOM_REST) + die("this command reject atom %%(%.*s)", (int)(ep - sp - 2), sp + 2); + if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || format->quote_style == QUOTE_TCL) && @@ -1927,6 +1941,12 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->handler = else_atom_handler; v->s = xstrdup(""); continue; + } else if (atom_type == ATOM_REST) { + if (ref->rest) + v->s = xstrdup(ref->rest); + else + v->s = xstrdup(""); + continue; } else continue; @@ -2144,6 +2164,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); + ref->rest = NULL; return ref; } diff --git a/ref-filter.h b/ref-filter.h index 74fb423fc89..c15dee8d6b9 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -38,6 +38,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; + const char *rest; int flag; unsigned int kind; const char *symref; @@ -76,14 +77,16 @@ struct ref_format { * verify_ref_format() afterwards to finalize. */ const char *format; + const char *rest; int quote_style; + int use_rest; int use_color; /* Internal state to ref-filter */ int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { NULL, 0, -1 } +#define REF_FORMAT_INIT { .use_color = -1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ diff --git a/t/t3203-branch-output.sh b/t/t3203-branch-output.sh index 5325b9f67a0..6e94c6db7b5 100755 --- a/t/t3203-branch-output.sh +++ b/t/t3203-branch-output.sh @@ -340,6 +340,10 @@ test_expect_success 'git branch --format option' ' test_cmp expect actual ' +test_expect_success 'git branch with --format=%(rest) must fail' ' + test_must_fail git branch --format="%(rest)" >actual +' + test_expect_success 'worktree colors correct' ' cat >expect <<-EOF && * (HEAD detached from fromtag) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 5556063c347..82c0ad2cb11 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -1211,6 +1211,10 @@ test_expect_success 'basic atom: head contents:trailers' ' test_cmp expect actual.clean ' +test_expect_success 'basic atom: rest must fail' ' + test_must_fail git for-each-ref --format="%(rest)" refs/heads/main +' + test_expect_success 'trailer parsing not fooled by --- line' ' git commit --allow-empty -F - <<-\EOF && this is the subject diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh index 2f72c5c6883..082be85dffc 100755 --- a/t/t7004-tag.sh +++ b/t/t7004-tag.sh @@ -1998,6 +1998,10 @@ test_expect_success '--format should list tags as per format given' ' test_cmp expect actual ' +test_expect_success 'git tag -l with --format="%(rest)" must fail' ' + test_must_fail git tag -l --format="%(rest)" "v1*" +' + test_expect_success "set up color tests" ' echo "v1.0" >expect.color && echo "v1.0" >expect.bare && diff --git a/t/t7030-verify-tag.sh b/t/t7030-verify-tag.sh index 3cefde9602b..10faa645157 100755 --- a/t/t7030-verify-tag.sh +++ b/t/t7030-verify-tag.sh @@ -194,6 +194,10 @@ test_expect_success GPG 'verifying tag with --format' ' test_cmp expect actual ' +test_expect_success GPG 'verifying tag with --format="%(rest)" must fail' ' + test_must_fail git verify-tag --format="%(rest)" "fourth-signed" +' + test_expect_success GPG 'verifying a forged tag with --format should fail silently' ' test_must_fail git verify-tag --format="tagname : %(tag)" $(cat forged1.tag) >actual-forged && test_must_be_empty actual-forged From patchwork Fri Jun 25 16:02:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47CE8C2B9F4 for ; Fri, 25 Jun 2021 16:02:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2FBCA61969 for ; Fri, 25 Jun 2021 16:02:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230127AbhFYQFG (ORCPT ); Fri, 25 Jun 2021 12:05:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229999AbhFYQEy (ORCPT ); Fri, 25 Jun 2021 12:04:54 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBBF4C061766 for ; Fri, 25 Jun 2021 09:02:32 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id p8-20020a7bcc880000b02901dbb595a9f1so6170393wma.2 for ; Fri, 25 Jun 2021 09:02:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=lI8pf131+MruoryekmU3fntYx6Qx5psRY7MDQCJZF8c=; b=WFazJh5YJvq6k0cWChurbEH2tJiU7IlyGPkHdHEa7ofmv8PoHDHrGHuQp23vPm2JDg OEby6YWoSg7ryVidKq+2mDGF1D+wJ85T5Yrh0tOijHLuO0Y/MUy9YPJYBChpNrF/7Dv/ CXrRDTKR/LHvK17nDnWHq4FvOHlkulrWWmzaftYrO4U6MlIiAAMIlBYGGJNJuCPp5upS QJJNu4ApwxmVnUKLeMoBtWadJ4SkImokAuI26rRXwSfyN8tEyIFXX7icmFr2zqDAZjBa qHKPRxc5VWHbX09d4PbT26fs3fUfdWbDKua027pPccolW+mQvTv6HY4LyWXEwNiIQW0m Gc3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=lI8pf131+MruoryekmU3fntYx6Qx5psRY7MDQCJZF8c=; b=U7mKgMrEzd7F4LNVhEWZpu1MLZcIbO4Vu3MaH5uKE+YN44vMo74zlI4f0q6AjbdAvj nCkU0wNo6FZmUElSN5XHSQFMD9+uIREniXOxOVBByL7B42Cv7bqzNN6oZYEwwYMriQ1Y GfVoUMMTien1nt0050O/VKO+YeDbU4w8NRa/ClsW7GgkVU6CF/3TFZeQeU2Ar6oSFP5H MtqWx9W68RzDifucUtYOE+b1O29UasjzAO3GtApRoxVBFD/lwLuWZ9sWK99sA+S5Y5ai /6XTXW5U2RpB+30a5kPHqOX1oGK0Ckdgj8UUzvHTqXSl3p6VUgwS3wA5GJ+xZjAVXYA7 zhgA== X-Gm-Message-State: AOAM530luMkxzAB8VpSTFpbxIyqxA+bEgGjj20UvTPDXYbN6yio8l/qq T3GX/sf38lmUvYCbeIjKqjW3iVp66Bc= X-Google-Smtp-Source: ABdhPJypX2khtNsSBjYv1ntay1AzJm+XSi/r3gUnNQRZmauPc0Z+nyqauQAGA5QSpRfS3hhpl96qaQ== X-Received: by 2002:a05:600c:2318:: with SMTP id 24mr11884817wmo.36.1624636951471; Fri, 25 Jun 2021 09:02:31 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x81sm13639544wmg.36.2021.06.25.09.02.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:31 -0700 (PDT) Message-Id: <67f1a3cca9a85fa94cb0158de71b36ce3dd6b444.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:16 +0000 Subject: [PATCH v5 06/15] [GSOC] ref-filter: pass get_object() return value to their callers MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because in the refactor of `git cat-file --batch` later, oid_object_info_extended() in get_object() will be used to obtain the info of an object with it's oid. When the object cannot be obtained in the git repository, `cat-file --batch` expects to output " missing" and continue the next oid query instead of letting Git exit. In other error conditions, Git should exit normally. So we can achieve this function by passing the return value of get_object(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- ref-filter.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 10c78de9cfa..58def6ccd33 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1816,6 +1816,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) { struct object *obj; int i; + int ret = 0; struct object_info empty = OBJECT_INFO_INIT; CALLOC_ARRAY(ref->value, used_atom_cnt); @@ -1972,8 +1973,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) oi.oid = ref->objectname; - if (get_object(ref, 0, &obj, &oi, err)) - return -1; + ret = get_object(ref, 0, &obj, &oi, err); + if (ret) + return ret; /* * If there is no atom that wants to know about tagged @@ -2005,8 +2007,10 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom, struct atom_value **v, struct strbuf *err) { if (!ref->value) { - if (populate_value(ref, err)) - return -1; + int ret = populate_value(ref, err); + + if (ret) + return ret; fill_missing_values(ref->value); } *v = &ref->value[atom]; @@ -2580,6 +2584,7 @@ int format_ref_array_item(struct ref_array_item *info, { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; + int ret = 0; state.quote_style = format->quote_style; push_stack_element(&state.stack); @@ -2592,10 +2597,10 @@ int format_ref_array_item(struct ref_array_item *info, if (cp < sp) append_literal(cp, sp, &state); pos = parse_ref_filter_atom(format, sp + 2, ep, error_buf); - if (pos < 0 || get_ref_atom_value(info, pos, &atomv, error_buf) || + if (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) || atomv->handler(atomv, &state, error_buf)) { pop_stack_element(&state.stack); - return -1; + return ret ? ret : -1; } } if (*cp) { From patchwork Fri Jun 25 16:02:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65911C48BC2 for ; Fri, 25 Jun 2021 16:02:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4E3EB6196E for ; Fri, 25 Jun 2021 16:02:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230138AbhFYQFL (ORCPT ); Fri, 25 Jun 2021 12:05:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbhFYQEy (ORCPT ); Fri, 25 Jun 2021 12:04:54 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89D19C0617A6 for ; Fri, 25 Jun 2021 09:02:33 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id r3so2061678wmq.1 for ; Fri, 25 Jun 2021 09:02:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=HpqslNZoUVwD073Xtsuq4EgEbeUegw9CooHr/ZNAVwc=; b=HokEFIa1LQUXdcEipaS9oWJR6hvR9H6o32PyIMFsIK13KlVB+rsAES/rhvixwxPSCG 4wXpHm0wVr2+GxnDbKFX0t/WmT7U20yiVnoTIQsF368Yrbbm5yP60nYQXpRajwVd3aOQ YeL5zDxEXUmSLGJiNp5Dz2ODEp+qzimojiXgvmFXgqmH/MKePU90f9eOxA5d/a82X88P EV85EosskHc+2SHeMI/j2SvTEMGWQDDbbVFbkoYC40z1Fa3JMdk8FozQoNkN+2TMUYhQ AN4CYmuJy3iFBL2VlVJifJbTQLZAmJPr2n/ao80qBW9MnkQ/3r7cO0Ggzkz3LlmtG6dI B+YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=HpqslNZoUVwD073Xtsuq4EgEbeUegw9CooHr/ZNAVwc=; b=ods6A3Bs8ykFrFs6IwHsC897TBsF9fU/jJXfYvmU6AssWn3b27Wp7nNDaDZbGcUYai hPNnDMCuV/WalNdJx1hgCfj54WXNcJ/K52/BAvBDammaQ7vHy7OieSQP5tKD87OYrIDy +479NL20JWkCzhx2C5nmVutQGjxvg0QMd74W61sPoWPVPUU8G1y725YmcnJTTdhukVzR tDX7Pw1P9cNdVk3F3Tg44Xa5nFFDiDcpd1MpHDA774kLgTUcuInKfHQXInWtXsDsOEP5 e+t8Xq+WmYDeUV6QadpQSDBRa6e3hBHV11PVli30JaR61dEJ6vFmnQWAszSXf+nmlnmg BoLA== X-Gm-Message-State: AOAM5308ZzponCC15RT1k0nB0hAAAHWKf/UpuQYURoRqcM0YP5NITmvA nb6oT0oPDwgcNKoHRDix2nI/AnbWgQs= X-Google-Smtp-Source: ABdhPJxSMU5l5OLubDOHEhJMuFgMvfND94cFD8W2EH0jZom3EMsSepQtHMOXJYdt5PcrLTazDDkljA== X-Received: by 2002:a05:600c:3ba6:: with SMTP id n38mr10467300wms.9.1624636952135; Fri, 25 Jun 2021 09:02:32 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u18sm2371700wrt.76.2021.06.25.09.02.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:31 -0700 (PDT) Message-Id: <2a48a48e81c6389c1eb8dd943bb8e323ed574bd2.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:17 +0000 Subject: [PATCH v5 07/15] [GSOC] ref-filter: introduce free_ref_array_item_value() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu When we use ref_array_item which is not dynamically allocated and want to free the space of its member "value" after the end of use, free_array_item() does not meet our needs, because it tries to free ref_array_item itself and its member "symref". Introduce free_ref_array_item_value() for freeing ref_array_item value. It will be called internally by free_array_item(), and it will help `cat-file --batch` free ref_array_item's value memory later. Helped-by: Junio C Hamano Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 ++++++++--- ref-filter.h | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 58def6ccd33..22315d4809d 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2291,16 +2291,21 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, return 0; } -/* Free memory allocated for a ref_array_item */ -static void free_array_item(struct ref_array_item *item) +void free_ref_array_item_value(struct ref_array_item *item) { - free((char *)item->symref); if (item->value) { int i; for (i = 0; i < used_atom_cnt; i++) free((char *)item->value[i].s); free(item->value); } +} + +/* Free memory allocated for a ref_array_item */ +static void free_array_item(struct ref_array_item *item) +{ + free((char *)item->symref); + free_ref_array_item_value(item); free(item); } diff --git a/ref-filter.h b/ref-filter.h index c15dee8d6b9..44e6dc05ac2 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -111,6 +111,8 @@ struct ref_format { int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type); /* Clear all memory allocated to ref_array */ void ref_array_clear(struct ref_array *array); +/* Free ref_array_item's value */ +void free_ref_array_item_value(struct ref_array_item *item); /* Used to verify if the given format is correct and to parse out the used atoms */ int verify_ref_format(struct ref_format *format); /* Sort the given ref_array as per the ref_sorting provided */ From patchwork Fri Jun 25 16:02:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB9E2C49EA7 for ; Fri, 25 Jun 2021 16:02:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D6B3861969 for ; Fri, 25 Jun 2021 16:02:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229741AbhFYQFJ (ORCPT ); Fri, 25 Jun 2021 12:05:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230011AbhFYQE4 (ORCPT ); Fri, 25 Jun 2021 12:04:56 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12DFBC061574 for ; Fri, 25 Jun 2021 09:02:34 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id a13so11113866wrf.10 for ; Fri, 25 Jun 2021 09:02:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=LRrzoPDrZTa1LnGfka0ELt/RiqwZjnAMrZ6gEUn+wnY=; b=Ozhwy8eFcHcqatNqaTCziH/YffcJRJMo2+YtqaX2ykUsv9viT7BiouVpOs1JrCE/UL tYlCjqMYwfTsyo8FsNlwSVgFeriM5A0i+Nbv0jKdJav4w49brOVfu7bcZAUSaPR3nTyf Gekzr5hFqkJdyGUwVpchimtzlxS08p33C/Rl1QYw4Y+OQgQIZNB52WhtgQmJbI+mWAPm URZznwQFAbcEWkGYr8YH25rc+xLRYc5udlZYyljZmwEcdV7sX7xSFexXnKI5jHq7lb2S 7nbrAb3u2PauAMS0JcyvUUgCKmqEtrMcVrPAoZTfdOW6XSRmU8+5ytzBTVVet10YHPEs rTcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=LRrzoPDrZTa1LnGfka0ELt/RiqwZjnAMrZ6gEUn+wnY=; b=C+QXDcDSslivx/i4qONNKYUaxK8uSNDhguoUuk/f+S2aEOF04GYfd5ndOldpVD0cm8 7VGXlNx3/XXdEWNcCO1uXgAkjsQ+2WmOMq5kayezYYJoBY2F8wSkXVuvljJ/6Ud4IoQf rK662keaTBMODIQQdzAD+U7U/8lWzb6IgF2qHCEfXDa4Skh44Ey3A4ywOYhs70xkRAOZ /pk6gpv+uwu5pY5uVcQklaVG6RgA+m1Ikq4YgFHIzqAnhP6zXr+pkgst7uHXbMO+YZ+U OUNaG9OhBh5Nc8WWcGNn5bz4yU8aB8lWvbrfYJFJ2pIpIbx9hge27+fV53FzTTA2l37k FgFg== X-Gm-Message-State: AOAM531gV5HEccUFBUqu6nemCC2vn/1T9kinksVJWYF8C+3wKBHBNlww +bVsv1UfQr1tiZlawBTEODqT5k4HDo8= X-Google-Smtp-Source: ABdhPJwNZiHDwf8fmN3cB61Jqajgoz4ccXmPu6CELtSVbXyE1XGaWruA5WtwIznVp5uDY/B9iXdD6Q== X-Received: by 2002:a5d:6a91:: with SMTP id s17mr12137299wru.364.1624636952715; Fri, 25 Jun 2021 09:02:32 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 4sm6330493wry.74.2021.06.25.09.02.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:32 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:18 +0000 Subject: [PATCH v5 08/15] [GSOC] ref-filter: add cat_file_mode in struct ref_format Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add `cat_file_mode` member in struct `ref_format`, when `cat-file --batch` use ref-filter logic later, it can help us reject atoms in verify_ref_format() which cat-file cannot use, e.g. `%(refname)`, `%(push)`, `%(upstream)`... Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 +++++++++-- ref-filter.h | 1 + 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 22315d4809d..f21f41df0d8 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1021,8 +1021,15 @@ int verify_ref_format(struct ref_format *format) if (at < 0) die("%s", err.buf); - if (used_atom[at].atom_type == ATOM_REST) - die("this command reject atom %%(%.*s)", (int)(ep - sp - 2), sp + 2); + if ((!format->cat_file_mode && used_atom[at].atom_type == ATOM_REST) || + (format->cat_file_mode && (used_atom[at].atom_type == ATOM_FLAG || + used_atom[at].atom_type == ATOM_HEAD || + used_atom[at].atom_type == ATOM_PUSH || + used_atom[at].atom_type == ATOM_REFNAME || + used_atom[at].atom_type == ATOM_SYMREF || + used_atom[at].atom_type == ATOM_UPSTREAM || + used_atom[at].atom_type == ATOM_WORKTREEPATH))) + die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || diff --git a/ref-filter.h b/ref-filter.h index 44e6dc05ac2..053980a6a42 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -78,6 +78,7 @@ struct ref_format { */ const char *format; const char *rest; + int cat_file_mode; int quote_style; int use_rest; int use_color; From patchwork Fri Jun 25 16:02:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35622C2B9F4 for ; Fri, 25 Jun 2021 16:02:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 236276196E for ; Fri, 25 Jun 2021 16:02:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229774AbhFYQFM (ORCPT ); Fri, 25 Jun 2021 12:05:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50462 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229712AbhFYQE5 (ORCPT ); Fri, 25 Jun 2021 12:04:57 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DABC2C061767 for ; Fri, 25 Jun 2021 09:02:34 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id k30-20020a05600c1c9eb02901d4d33c5ca0so734942wms.3 for ; Fri, 25 Jun 2021 09:02:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=1e0iKytryCbsbJLjnyrovo3yeJs6kZmNWN65BNPEnEY=; b=CCZundseRNaaJZVRagP5QdQ5/T3DAXhAn0NlbKPe/+FxvUpGE6QlQmnFgBEdmBakcU LqTRGAjutxkhNDR4sisR6ndB9OrrKG1r/PTd4SoJrPs438K3GbTMDyWP+gxXAlrA101/ KXqKUM48h437WRjig3mYVjdDI0ZH49uP4mKfXUM04PQZpBaLMfMFj7UgKzrbVxJewy9Q qJ5i/ff/Gjqb931Sr9SOdionOuvf8FvFxq9aJw+MIxXzzxeaFuud0qzfEkEKgsrLbD6O roJOZg3i4GKC0VKx+kEjFRVOZfiISvK2yM02luqe8NqyIMeAoDSxCAs64SLDH9d7WSu7 LrfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=1e0iKytryCbsbJLjnyrovo3yeJs6kZmNWN65BNPEnEY=; b=eCTXOYaWBKpvLa+vhvA0Aqdq1omHqiT1lvuT63MaPAcGR1NrzqmB8YFJUDOS2BhCBv 4vd9i+BtFQCLyhuyW//TQROn7JaS02CmSQW8SRxcNogxNub5t9RW9QkTj8QXieSWi0nn wulBzWW+Z0ny0YPSdOaF7pU51pYHuy8GVpGcmj0cQuPRvIO7mcO9KVe406OhQo7uddqI xhTdppZUmY+2RNpKNEGMUqy/9civnodAE/U4y4KDHWYwkW6t60ZHF+FhaedD4YeGI4rc sRTWiN6Pn6cmvY+jv1LrLl62anFH53keLFtA9hiAtfI/Kxky1vsW+xonCISYxlJGHPuB 6+tA== X-Gm-Message-State: AOAM532ez8dg2LqwWPMxeeQlDnpBlIWySrMrJKoaaN7Jg26PxkbV4g3n tbOe/TB0AyX/gDB+C6Z/L+/6LuefWfw= X-Google-Smtp-Source: ABdhPJzhPJ/N3sqzFoLRxFT3AJSSF3Tk1m4NC50wL7Y5bqVMpdncaKvfNg8Gq5JtS9AXEBYDjuNK3w== X-Received: by 2002:a05:600c:218c:: with SMTP id e12mr11526948wme.92.1624636953471; Fri, 25 Jun 2021 09:02:33 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y15sm5965542wmq.17.2021.06.25.09.02.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:33 -0700 (PDT) Message-Id: <937f88b78371d1a4497c8dc389a499ab51446075.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:19 +0000 Subject: [PATCH v5 09/15] [GSOC] ref-filter: modify the error message and value in get_object Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Let get_object() return 1 and print " missing" instead of returning -1 and printing "missing object for " if oid_object_info_extended() unable to find the data corresponding to oid. When `cat-file --batch` use ref-filter logic later it can help `format_ref_array_item()` just report that the object is missing without letting Git exit. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 4 ++-- t/t6301-for-each-ref-errors.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index f21f41df0d8..181d99c9273 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1749,8 +1749,8 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) - return strbuf_addf_ret(err, -1, _("missing object %s for %s"), - oid_to_hex(&oi->oid), ref->refname); + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); if (oi->info.disk_sizep && oi->disk_size < 0) BUG("Object size is less than zero."); diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh index 40edf9dab53..3553f84a00c 100755 --- a/t/t6301-for-each-ref-errors.sh +++ b/t/t6301-for-each-ref-errors.sh @@ -41,7 +41,7 @@ test_expect_success 'Missing objects are reported correctly' ' r=refs/heads/missing && echo $MISSING >.git/$r && test_when_finished "rm -f .git/$r" && - echo "fatal: missing object $MISSING for $r" >missing-err && + echo "fatal: $MISSING missing" >missing-err && test_must_fail git for-each-ref 2>err && test_cmp missing-err err && ( From patchwork Fri Jun 25 16:02:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F08ADC49EA7 for ; Fri, 25 Jun 2021 16:02:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DA2146196E for ; Fri, 25 Jun 2021 16:02:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229586AbhFYQFM (ORCPT ); Fri, 25 Jun 2021 12:05:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230032AbhFYQE4 (ORCPT ); Fri, 25 Jun 2021 12:04:56 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64DFEC061768 for ; Fri, 25 Jun 2021 09:02:35 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id t11-20020a1cc30b0000b02901cec841b6a0so6677268wmf.0 for ; Fri, 25 Jun 2021 09:02:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=r7Z6d5ifsh1v6IjHQmKUm8lmxTGmT95I9U09SdFQEqM=; b=pqxrmShRJiw8bVLgOVsVs9c4xUb2vMbRzhsja+pr5O+D7pfIgeiTCjXzQvHmBwpGbe yW4NIetOy3WLn84C9T13tBq4pwoPCgxrqBHZNiB6fHicnM7IJBMXvYeYzraj2WfqhGaK XsBuOb0kyQPInOiAezBmlshl2aTCHUSTTlCFc28JLIO/uEh9YgFAjX2oqj+Sd+uaxaji YGLvnJ6mG9ab0dcjzrJDSzgmaVm3ij2mXSRidhKK4mEph7OilH24aXTp6ierVxh6LtXJ EYi7+nZGaOIohEejkfbwkUq7D+RttMezFPRZYzNvyziJ/ad6KeMQADg+oHPboTwpvrGC a++A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=r7Z6d5ifsh1v6IjHQmKUm8lmxTGmT95I9U09SdFQEqM=; b=AnT46PVXM+35ktRnJyvhu5f/iF7VuNK0jwc5u4g7BPPOSVYNqMEPLmMXLsaDALwCiN eJrZ4TKlq3DTythWSWM/phYsaX55PxAZN7M5Om+ZWAdkjBfrYPVDmuSWHRMrbEZnvtV2 rVEG/+wSphptNOohnoN4sxbxQs/TyFmO4LKD2NJ6LCfxfK43pHmHkdfV4ob9zkqOE2Au dnRQh4yob2Q8BsKFOaGU/j+Kz5Dh0s0fEelv3PFT7vyDcXvaheudOiFp7+vMGlZKQ64S IcH9wlEbIry7BiFxSkqtMcycW2tufARQ7aQ+Ng3UnVTmGV1TCciWbhuSI5ICcJ+JS4bZ YS4w== X-Gm-Message-State: AOAM530/RlPISA/l9h4KvyMzLodIlWkXwZin7AMAMbJpti8kSHkQ+J9t kbtzYu0U/0fPljWIJCytJu2Ea9nJ9kI= X-Google-Smtp-Source: ABdhPJwN7EVRHnbyDDGPHBw1V+IdCpG4xyR8Ixqaij8sychZ40PEKy/SeVK/wX7uqiuA44kauB9gZg== X-Received: by 2002:a1c:f016:: with SMTP id a22mr11601841wmb.65.1624636954118; Fri, 25 Jun 2021 09:02:34 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i3sm12132375wmb.15.2021.06.25.09.02.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:33 -0700 (PDT) Message-Id: <45657499c55ed91eb78498be4355459b00f76404.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:20 +0000 Subject: [PATCH v5 10/15] [GSOC] cat-file: add has_object_file() check Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use `has_object_file()` in `batch_one_object()` to check whether the input object exists. This can help us reject the missing oid when we let `cat-file --batch` use ref-filter logic later. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 5ebf13359e8..9fd3c04ff20 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -428,6 +428,13 @@ static void batch_one_object(const char *obj_name, return; } + if (!has_object_file(&data->oid)) { + printf("%s missing\n", + obj_name ? obj_name : oid_to_hex(&data->oid)); + fflush(stdout); + return; + } + batch_object_write(obj_name, scratch, opt, data); } From patchwork Fri Jun 25 16:02:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01E0BC48BC2 for ; Fri, 25 Jun 2021 16:02:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD0776196E for ; Fri, 25 Jun 2021 16:02:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230159AbhFYQFO (ORCPT ); Fri, 25 Jun 2021 12:05:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230056AbhFYQE5 (ORCPT ); Fri, 25 Jun 2021 12:04:57 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 402B5C0617A8 for ; Fri, 25 Jun 2021 09:02:36 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id a13so11113984wrf.10 for ; Fri, 25 Jun 2021 09:02:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=HluKIvI8PnxXFef46ukO9OSyoyTW6LZ2nDr1+/IAdRM=; b=Em1zyPS711Qc3kizOFYQTEUbvN3BQrXnD/02o4FVTL1cPF0W5ogvgQeNsYyCiSpQCT yB1OdGl/pVwoSD0JwLiUVFRFBPV/uheIeoP3lkM61MHJar1rLyRYtMjaQFLcj3GRt6zD hEtwCcexLuc5nkh1F8OwD8tXEPnlNrlk9LGWVKhlCqBwo3P4voBjp9K0f9aoGwnFV0Iz uz5+1+YX0LihGiTKtKOrXPnNy+svaVK4de8eZ5g0DX/IZBmlMayQQ94tEGhJD/SuyjoB onxzyxtWXQ43YSjrNjUk6ab8IOAP/tNNa7zgcJ+thwqNK5OXOOGDwSDdNmUwxpUN/8rh gWnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=HluKIvI8PnxXFef46ukO9OSyoyTW6LZ2nDr1+/IAdRM=; b=uNqzwsPnIvMY6R8Qf/FcuOrnpiTX5buVQPubDUT54gmPphu23uQD5u/3MfgudPGQcQ BPWRviy4OPCjD2D1ENlCfc1VOyBnCfkGbKk4e2UfcBIMaUjDQr94O3GDfRGL6QyJAsox DgbJ0/qTDT7yquZfmilkyr3JC8geOE+bT0Ptee8VoxJjCIFq7nmr91jDxfwJlIyEEgDY J8Ifhx5N/VHUPb2oFC4Pwr6uP2TPDlEIgInWy+QOGYTIO+H9R1gsBC9NrwJdS5t3CqeO VK3tasEhu2sY/Ztwf5jcYgQ+cHac1jJSkq7zbqBgvqsmyzzUx2J5pq4OeyokrUx/bYfg 6bIw== X-Gm-Message-State: AOAM530HCzRZ6sX9NVhRhoNnre3geNVNRSp8LcpbMtshfKUPz9B2QLMX CP3rwHXL9CFxb7bEhRkm/qPdMpPPFOI= X-Google-Smtp-Source: ABdhPJzsPMZGacx/ocsSWVJ/ZD42urhs6VG5oXbvB0a+xpLi/37k9PIYmBOF7mX+HDPbtWC0HNtcKw== X-Received: by 2002:adf:bd84:: with SMTP id l4mr12035279wrh.346.1624636954693; Fri, 25 Jun 2021 09:02:34 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r1sm11720842wmn.10.2021.06.25.09.02.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:34 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:21 +0000 Subject: [PATCH v5 11/15] [GSOC] cat-file: change batch_objects parameter name Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because later cat-file reuses ref-filter logic that will add parameter "const struct option *options" to batch_objects(), the two synonymous parameters of "opt" and "options" may confuse readers, so change batch_options parameter of batch_objects() from "opt" to "batch". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 9fd3c04ff20..cd84c39df96 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -495,7 +495,7 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *opt) +static int batch_objects(struct batch_options *batch) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; @@ -503,8 +503,8 @@ static int batch_objects(struct batch_options *opt) int save_warning; int retval = 0; - if (!opt->format) - opt->format = "%(objectname) %(objecttype) %(objectsize)"; + if (!batch->format) + batch->format = "%(objectname) %(objecttype) %(objectsize)"; /* * Expand once with our special mark_query flag, which will prime the @@ -513,13 +513,13 @@ static int batch_objects(struct batch_options *opt) */ memset(&data, 0, sizeof(data)); data.mark_query = 1; - strbuf_expand(&output, opt->format, expand_format, &data); + strbuf_expand(&output, batch->format, expand_format, &data); data.mark_query = 0; strbuf_release(&output); - if (opt->cmdmode) + if (batch->cmdmode) data.split_on_whitespace = 1; - if (opt->all_objects) { + if (batch->all_objects) { struct object_info empty = OBJECT_INFO_INIT; if (!memcmp(&data.info, &empty, sizeof(empty))) data.skip_object_info = 1; @@ -529,20 +529,20 @@ static int batch_objects(struct batch_options *opt) * If we are printing out the object, then always fill in the type, * since we will want to decide whether or not to stream. */ - if (opt->print_contents) + if (batch->print_contents) data.info.typep = &data.type; - if (opt->all_objects) { + if (batch->all_objects) { struct object_cb_data cb; if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); - cb.opt = opt; + cb.opt = batch; cb.expand = &data; cb.scratch = &output; - if (opt->unordered) { + if (batch->unordered) { struct oidset seen = OIDSET_INIT; cb.seen = &seen; @@ -592,7 +592,7 @@ static int batch_objects(struct batch_options *opt) data.rest = p; } - batch_one_object(input.buf, &output, opt, &data); + batch_one_object(input.buf, &output, batch, &data); } strbuf_release(&input); From patchwork Fri Jun 25 16:02:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8ED12C49EAB for ; Fri, 25 Jun 2021 16:02:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 76EF861969 for ; Fri, 25 Jun 2021 16:02:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230170AbhFYQFP (ORCPT ); Fri, 25 Jun 2021 12:05:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230044AbhFYQE6 (ORCPT ); Fri, 25 Jun 2021 12:04:58 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1A6EC061574 for ; Fri, 25 Jun 2021 09:02:36 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id j2so11116987wrs.12 for ; Fri, 25 Jun 2021 09:02:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=SeIwwJ8uyCaCbfbqkhVGRD/UMr4GVeH7ztA2WD87zpM=; b=FvDeLjx1HtH8KnvEDDzTMQ9TNIfdENOTQYleFS7ucxxsi0CrZCFALZFVwiKKsej8Xs R2OafXrRU6Fx3WHZYGLu51RvYdaV+JAhQBFUUxqzj1q/hyGmb7kHLb7PKv/LTVEoVmRO U+VPHhjR0nvFjvHrTnTB1/2T1yxpBIp0L126U1Awzwi3OcTYZBREwzF/ppTYpqDoRGw5 2zAvyUNqQJjyhQLEZC6XnH11/hIi64BYVKYKBe7DbfLo9PZhzsjtRxjEStOyk74WpzCh rU8eGAScGTrNj3mNQvHxYiIA/TMUb1e2VO5UEmAbFLtktlp5dsXti2xCHBoCAH7OKA+J bIaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=SeIwwJ8uyCaCbfbqkhVGRD/UMr4GVeH7ztA2WD87zpM=; b=Xf4PMHXRSiTrXJfIy2njYE7davRrNI6vbfJ7RlokPbm06ogeA3NJ+GRXXNdHf8k9lT fzXMN41IKiXW3XI8IqBROGTErZrbKd+tCwljTuIv7dXph+viv48uQQGdx9IBtSvQxNT7 z5PInu8nu5CiBTjH3ijFqi8sDO4fXj3POrxDnqdjN+HGHDNwPpqLPIAsvF0kuI9XtGBz iOf7DiBGmOLoXHqVUTWYC8Q3XAiJDTXS0XjiX8p7UKCHx+RgJcGTdricTvN6cNNE4P9N 7YS2uyLQRkR4fdUP9DmozdalUjmWhUlbmW+wOEt4sEGoGpnyKU84hmSGq7rmdNrW9QQe kNgQ== X-Gm-Message-State: AOAM5301lZEu/wA2i4EZSP9Hau4yh5yOdcP6Q1bfHmoyroA59kMPyrC+ RM0cuONnLZWsAKYDcFFuGSXtsSfhFvo= X-Google-Smtp-Source: ABdhPJxPU1cw8k4AUTkK0yVebBWuQDS9/QXP8oP6XSg56B/+eRKJPngINMX/sbSbbjlybtlWksFb7w== X-Received: by 2002:a5d:65cc:: with SMTP id e12mr11693271wrw.354.1624636955515; Fri, 25 Jun 2021 09:02:35 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n29sm932360wms.34.2021.06.25.09.02.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:35 -0700 (PDT) Message-Id: <370101ba65f0989487360366f8b83144a6641a04.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:22 +0000 Subject: [PATCH v5 12/15] [GSOC] cat-file: reuse ref-filter logic MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let cat-file use ref-filter logic, let's do the following: 1. Change the type of member `format` in struct `batch_options` to `ref_format`, we will pass it to ref-filter later. 2. Let `batch_objects()` add atoms to format, and use `verify_ref_format()` to check atoms. 3. Use `format_ref_array_item()` in `batch_object_write()` to get the formatted data corresponding to the object. If the return value of `format_ref_array_item()` is equals to zero, use `batch_write()` to print object data; else if the return value is less than zero, use `die()` to print the error message and exit; else if return value is greater than zero, only print the error message, but don't exit. 4. Use free_ref_array_item_value() to free ref_array_item's value. Most of the atoms in `for-each-ref --format` are now supported, such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, `%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: `%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, `%(flag)`, `%(HEAD)`, because our objects don't have a refname. The performance for `git cat-file --batch-all-objects --batch-check` on the Git repository itself with performance testing tool `hyperfine` changes from 669.4 ms ± 31.1 ms to 1.134 s ± 0.063 s. The performance for `git cat-file --batch-all-objects --batch >/dev/null` on the Git repository itself with performance testing tool `time` change from "27.37s user 0.29s system 98% cpu 28.089 total" to "33.69s user 1.54s system 87% cpu 40.258 total". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- Documentation/git-cat-file.txt | 6 + builtin/cat-file.c | 244 ++++++------------------------- t/t1006-cat-file.sh | 252 +++++++++++++++++++++++++++++++++ 3 files changed, 305 insertions(+), 197 deletions(-) diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index 4eb0421b3fd..ef8ab952b2f 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -226,6 +226,12 @@ newline. The available atoms are: after that first run of whitespace (i.e., the "rest" of the line) are output in place of the `%(rest)` atom. +Note that most of the atoms in `for-each-ref --format` are now supported, +such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, +`%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: +`%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, +`%(flag)`, `%(HEAD)`. See linkgit:git-for-each-ref[1]. + If no format is specified, the default format is `%(objectname) %(objecttype) %(objectsize)`. diff --git a/builtin/cat-file.c b/builtin/cat-file.c index cd84c39df96..0e7ad038e5f 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -16,6 +16,7 @@ #include "packfile.h" #include "object-store.h" #include "promisor-remote.h" +#include "ref-filter.h" struct batch_options { int enabled; @@ -25,7 +26,7 @@ struct batch_options { int all_objects; int unordered; int cmdmode; /* may be 'w' or 'c' for --filters or --textconv */ - const char *format; + struct ref_format format; }; static const char *force_path; @@ -195,99 +196,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct expand_data { struct object_id oid; - enum object_type type; - unsigned long size; - off_t disk_size; const char *rest; - struct object_id delta_base_oid; - - /* - * If mark_query is true, we do not expand anything, but rather - * just mark the object_info with items we wish to query. - */ - int mark_query; - - /* - * Whether to split the input on whitespace before feeding it to - * get_sha1; this is decided during the mark_query phase based on - * whether we have a %(rest) token in our format. - */ int split_on_whitespace; - - /* - * After a mark_query run, this object_info is set up to be - * passed to oid_object_info_extended. It will point to the data - * elements above, so you can retrieve the response from there. - */ - struct object_info info; - - /* - * This flag will be true if the requested batch format and options - * don't require us to call oid_object_info, which can then be - * optimized out. - */ - unsigned skip_object_info : 1; }; -static int is_atom(const char *atom, const char *s, int slen) -{ - int alen = strlen(atom); - return alen == slen && !memcmp(atom, s, alen); -} - -static void expand_atom(struct strbuf *sb, const char *atom, int len, - void *vdata) -{ - struct expand_data *data = vdata; - - if (is_atom("objectname", atom, len)) { - if (!data->mark_query) - strbuf_addstr(sb, oid_to_hex(&data->oid)); - } else if (is_atom("objecttype", atom, len)) { - if (data->mark_query) - data->info.typep = &data->type; - else - strbuf_addstr(sb, type_name(data->type)); - } else if (is_atom("objectsize", atom, len)) { - if (data->mark_query) - data->info.sizep = &data->size; - else - strbuf_addf(sb, "%"PRIuMAX , (uintmax_t)data->size); - } else if (is_atom("objectsize:disk", atom, len)) { - if (data->mark_query) - data->info.disk_sizep = &data->disk_size; - else - strbuf_addf(sb, "%"PRIuMAX, (uintmax_t)data->disk_size); - } else if (is_atom("rest", atom, len)) { - if (data->mark_query) - data->split_on_whitespace = 1; - else if (data->rest) - strbuf_addstr(sb, data->rest); - } else if (is_atom("deltabase", atom, len)) { - if (data->mark_query) - data->info.delta_base_oid = &data->delta_base_oid; - else - strbuf_addstr(sb, - oid_to_hex(&data->delta_base_oid)); - } else - die("unknown format element: %.*s", len, atom); -} - -static size_t expand_format(struct strbuf *sb, const char *start, void *data) -{ - const char *end; - - if (*start != '(') - return 0; - end = strchr(start + 1, ')'); - if (!end) - die("format element '%s' does not end in ')'", start); - - expand_atom(sb, start + 1, end - start - 1, data); - - return end - start + 1; -} - static void batch_write(struct batch_options *opt, const void *data, int len) { if (opt->buffer_output) { @@ -297,87 +209,34 @@ static void batch_write(struct batch_options *opt, const void *data, int len) write_or_die(1, data, len); } -static void print_object_or_die(struct batch_options *opt, struct expand_data *data) -{ - const struct object_id *oid = &data->oid; - - assert(data->info.typep); - - if (data->type == OBJ_BLOB) { - if (opt->buffer_output) - fflush(stdout); - if (opt->cmdmode) { - char *contents; - unsigned long size; - - if (!data->rest) - die("missing path for '%s'", oid_to_hex(oid)); - - if (opt->cmdmode == 'w') { - if (filter_object(data->rest, 0100644, oid, - &contents, &size)) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else if (opt->cmdmode == 'c') { - enum object_type type; - if (!textconv_object(the_repository, - data->rest, 0100644, oid, - 1, &contents, &size)) - contents = read_object_file(oid, - &type, - &size); - if (!contents) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else - BUG("invalid cmdmode: %c", opt->cmdmode); - batch_write(opt, contents, size); - free(contents); - } else { - stream_blob(oid); - } - } - else { - enum object_type type; - unsigned long size; - void *contents; - - contents = read_object_file(oid, &type, &size); - if (!contents) - die("object %s disappeared", oid_to_hex(oid)); - if (type != data->type) - die("object %s changed type!?", oid_to_hex(oid)); - if (data->info.sizep && size != data->size) - die("object %s changed size!?", oid_to_hex(oid)); - - batch_write(opt, contents, size); - free(contents); - } -} static void batch_object_write(const char *obj_name, struct strbuf *scratch, struct batch_options *opt, struct expand_data *data) { - if (!data->skip_object_info && - oid_object_info_extended(the_repository, &data->oid, &data->info, - OBJECT_INFO_LOOKUP_REPLACE) < 0) { - printf("%s missing\n", - obj_name ? obj_name : oid_to_hex(&data->oid)); - fflush(stdout); - return; - } + int ret = 0; + struct strbuf err = STRBUF_INIT; + struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); - strbuf_expand(scratch, opt->format, expand_format, data); - strbuf_addch(scratch, '\n'); - batch_write(opt, scratch->buf, scratch->len); - if (opt->print_contents) { - print_object_or_die(opt, data); - batch_write(opt, "\n", 1); + ret = format_ref_array_item(&item, &opt->format, scratch, &err); + if (ret < 0) { + die("%s\n", err.buf); + } if (ret) { + /* ret > 0 means when the object corresponding to oid + * cannot be found in format_ref_array_item(), we only print + * the error message. + */ + printf("%s\n", err.buf); + fflush(stdout); + } else { + strbuf_addch(scratch, '\n'); + batch_write(opt, scratch->buf, scratch->len); } + free_ref_array_item_value(&item); + strbuf_release(&err); } static void batch_one_object(const char *obj_name, @@ -495,42 +354,34 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *batch) +static const char * const cat_file_usage[] = { + N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), + N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), + NULL +}; + +static int batch_objects(struct batch_options *batch, const struct option *options) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; int retval = 0; - if (!batch->format) - batch->format = "%(objectname) %(objecttype) %(objectsize)"; - - /* - * Expand once with our special mark_query flag, which will prime the - * object_info to be handed to oid_object_info_extended for each - * object. - */ memset(&data, 0, sizeof(data)); - data.mark_query = 1; - strbuf_expand(&output, batch->format, expand_format, &data); - data.mark_query = 0; - strbuf_release(&output); - if (batch->cmdmode) - data.split_on_whitespace = 1; - - if (batch->all_objects) { - struct object_info empty = OBJECT_INFO_INIT; - if (!memcmp(&data.info, &empty, sizeof(empty))) - data.skip_object_info = 1; - } - - /* - * If we are printing out the object, then always fill in the type, - * since we will want to decide whether or not to stream. - */ + if (batch->format.format) + strbuf_addstr(&format, batch->format.format); + else + strbuf_addstr(&format, "%(objectname) %(objecttype) %(objectsize)"); if (batch->print_contents) - data.info.typep = &data.type; + strbuf_addstr(&format, "\n%(raw)"); + batch->format.format = format.buf; + if (verify_ref_format(&batch->format)) + usage_with_options(cat_file_usage, options); + + if (batch->cmdmode || batch->format.use_rest) + data.split_on_whitespace = 1; if (batch->all_objects) { struct object_cb_data cb; @@ -563,6 +414,7 @@ static int batch_objects(struct batch_options *batch) oid_array_clear(&sa); } + strbuf_release(&format); strbuf_release(&output); return 0; } @@ -595,18 +447,13 @@ static int batch_objects(struct batch_options *batch) batch_one_object(input.buf, &output, batch, &data); } + strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); warn_on_object_refname_ambiguity = save_warning; return retval; } -static const char * const cat_file_usage[] = { - N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), - N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), - NULL -}; - static int git_cat_file_config(const char *var, const char *value, void *cb) { if (userdiff_config(var, value) < 0) @@ -629,7 +476,7 @@ static int batch_option_callback(const struct option *opt, bo->enabled = 1; bo->print_contents = !strcmp(opt->long_name, "batch"); - bo->format = arg; + bo->format.format = arg; return 0; } @@ -638,7 +485,9 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) { int opt = 0; const char *exp_type = NULL, *obj_name = NULL; - struct batch_options batch = {0}; + struct batch_options batch = { + .format = REF_FORMAT_INIT + }; int unknown_type = 0; const struct option options[] = { @@ -677,6 +526,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) git_config(git_cat_file_config, NULL); batch.buffer_output = -1; + batch.format.cat_file_mode = 1; argc = parse_options(argc, argv, prefix, options, cat_file_usage, 0); if (opt) { @@ -720,7 +570,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) batch.buffer_output = batch.all_objects; if (batch.enabled) - return batch_objects(&batch); + return batch_objects(&batch, options); if (unknown_type && opt != 't' && opt != 's') die("git cat-file --allow-unknown-type: use with -s or -t"); diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 5d2dc99b74a..69eb627774d 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -586,4 +586,256 @@ test_expect_success 'cat-file --unordered works' ' test_cmp expect actual ' +. "$TEST_DIRECTORY"/lib-gpg.sh +. "$TEST_DIRECTORY"/lib-terminal.sh + +test_expect_success 'cat-file --batch|--batch-check setup' ' + echo 1>blob1 && + printf "a\0b\0\c" >blob2 && + git add blob1 blob2 && + git commit -m "Commit Message" && + git branch -M main && + git tag -a -m "v0.0.0" testtag && + git update-ref refs/myblobs/blob1 HEAD:blob1 && + git update-ref refs/myblobs/blob2 HEAD:blob2 && + git update-ref refs/mytrees/tree1 HEAD^{tree} +' + +batch_test_atom() { + if test "$3" = "fail" + then + test_expect_${4:-success} $PREREQ "basic atom: $1 $2 must fail" " + test_must_fail git cat-file --batch-check='$2' >bad <<-EOF + $1 + EOF + " + else + test_expect_${4:-success} $PREREQ "basic atom: $1 $2" " + git for-each-ref --format='$2' $1 >expected && + git cat-file --batch-check='$2' >actual <<-EOF && + $1 + EOF + sanitize_pgp actual.clean && + cmp expected actual.clean + " + fi +} + +batch_test_atom refs/heads/main '%(refname)' fail +batch_test_atom refs/heads/main '%(refname:)' fail +batch_test_atom refs/heads/main '%(refname:short)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:strip=1)' fail +batch_test_atom refs/heads/main '%(refname:strip=2)' fail +batch_test_atom refs/heads/main '%(refname:strip=-1)' fail +batch_test_atom refs/heads/main '%(refname:strip=-2)' fail +batch_test_atom refs/heads/main '%(upstream)' fail +batch_test_atom refs/heads/main '%(upstream:short)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=-2)' fail +batch_test_atom refs/heads/main '%(push)' fail +batch_test_atom refs/heads/main '%(push:short)' fail +batch_test_atom refs/heads/main '%(push:lstrip=1)' fail +batch_test_atom refs/heads/main '%(push:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:strip=1)' fail +batch_test_atom refs/heads/main '%(push:strip=-1)' fail +batch_test_atom refs/heads/main '%(objecttype)' +batch_test_atom refs/heads/main '%(objectsize)' +batch_test_atom refs/heads/main '%(objectsize:disk)' +batch_test_atom refs/heads/main '%(deltabase)' +batch_test_atom refs/heads/main '%(objectname)' +batch_test_atom refs/heads/main '%(objectname:short)' +batch_test_atom refs/heads/main '%(objectname:short=1)' +batch_test_atom refs/heads/main '%(objectname:short=10)' +batch_test_atom refs/heads/main '%(tree)' +batch_test_atom refs/heads/main '%(tree:short)' +batch_test_atom refs/heads/main '%(tree:short=1)' +batch_test_atom refs/heads/main '%(tree:short=10)' +batch_test_atom refs/heads/main '%(parent)' +batch_test_atom refs/heads/main '%(parent:short)' +batch_test_atom refs/heads/main '%(parent:short=1)' +batch_test_atom refs/heads/main '%(parent:short=10)' +batch_test_atom refs/heads/main '%(numparent)' +batch_test_atom refs/heads/main '%(object)' +batch_test_atom refs/heads/main '%(type)' +batch_test_atom refs/heads/main '%(raw)' +batch_test_atom refs/heads/main '%(*objectname)' +batch_test_atom refs/heads/main '%(*objecttype)' +batch_test_atom refs/heads/main '%(author)' +batch_test_atom refs/heads/main '%(authorname)' +batch_test_atom refs/heads/main '%(authoremail)' +batch_test_atom refs/heads/main '%(authoremail:trim)' +batch_test_atom refs/heads/main '%(authoremail:localpart)' +batch_test_atom refs/heads/main '%(authordate)' +batch_test_atom refs/heads/main '%(committer)' +batch_test_atom refs/heads/main '%(committername)' +batch_test_atom refs/heads/main '%(committeremail)' +batch_test_atom refs/heads/main '%(committeremail:trim)' +batch_test_atom refs/heads/main '%(committeremail:localpart)' +batch_test_atom refs/heads/main '%(committerdate)' +batch_test_atom refs/heads/main '%(tag)' +batch_test_atom refs/heads/main '%(tagger)' +batch_test_atom refs/heads/main '%(taggername)' +batch_test_atom refs/heads/main '%(taggeremail)' +batch_test_atom refs/heads/main '%(taggeremail:trim)' +batch_test_atom refs/heads/main '%(taggeremail:localpart)' +batch_test_atom refs/heads/main '%(taggerdate)' +batch_test_atom refs/heads/main '%(creator)' +batch_test_atom refs/heads/main '%(creatordate)' +batch_test_atom refs/heads/main '%(subject)' +batch_test_atom refs/heads/main '%(subject:sanitize)' +batch_test_atom refs/heads/main '%(contents:subject)' +batch_test_atom refs/heads/main '%(body)' +batch_test_atom refs/heads/main '%(contents:body)' +batch_test_atom refs/heads/main '%(contents:signature)' +batch_test_atom refs/heads/main '%(contents)' +batch_test_atom refs/heads/main '%(HEAD)' fail +batch_test_atom refs/heads/main '%(upstream:track)' fail +batch_test_atom refs/heads/main '%(upstream:trackshort)' fail +batch_test_atom refs/heads/main '%(upstream:track,nobracket)' fail +batch_test_atom refs/heads/main '%(upstream:nobracket,track)' fail +batch_test_atom refs/heads/main '%(push:track)' fail +batch_test_atom refs/heads/main '%(push:trackshort)' fail +batch_test_atom refs/heads/main '%(worktreepath)' fail +batch_test_atom refs/heads/main '%(symref)' fail +batch_test_atom refs/heads/main '%(flag)' fail + +batch_test_atom refs/tags/testtag '%(refname)' fail +batch_test_atom refs/tags/testtag '%(refname:short)' fail +batch_test_atom refs/tags/testtag '%(upstream)' fail +batch_test_atom refs/tags/testtag '%(push)' fail +batch_test_atom refs/tags/testtag '%(objecttype)' +batch_test_atom refs/tags/testtag '%(objectsize)' +batch_test_atom refs/tags/testtag '%(objectsize:disk)' +batch_test_atom refs/tags/testtag '%(*objectsize:disk)' +batch_test_atom refs/tags/testtag '%(deltabase)' +batch_test_atom refs/tags/testtag '%(*deltabase)' +batch_test_atom refs/tags/testtag '%(objectname)' +batch_test_atom refs/tags/testtag '%(objectname:short)' +batch_test_atom refs/tags/testtag '%(tree)' +batch_test_atom refs/tags/testtag '%(tree:short)' +batch_test_atom refs/tags/testtag '%(tree:short=1)' +batch_test_atom refs/tags/testtag '%(tree:short=10)' +batch_test_atom refs/tags/testtag '%(parent)' +batch_test_atom refs/tags/testtag '%(parent:short)' +batch_test_atom refs/tags/testtag '%(parent:short=1)' +batch_test_atom refs/tags/testtag '%(parent:short=10)' +batch_test_atom refs/tags/testtag '%(numparent)' +batch_test_atom refs/tags/testtag '%(object)' +batch_test_atom refs/tags/testtag '%(type)' +batch_test_atom refs/tags/testtag '%(*objectname)' +batch_test_atom refs/tags/testtag '%(*objecttype)' +batch_test_atom refs/tags/testtag '%(author)' +batch_test_atom refs/tags/testtag '%(authorname)' +batch_test_atom refs/tags/testtag '%(authoremail)' +batch_test_atom refs/tags/testtag '%(authoremail:trim)' +batch_test_atom refs/tags/testtag '%(authoremail:localpart)' +batch_test_atom refs/tags/testtag '%(authordate)' +batch_test_atom refs/tags/testtag '%(committer)' +batch_test_atom refs/tags/testtag '%(committername)' +batch_test_atom refs/tags/testtag '%(committeremail)' +batch_test_atom refs/tags/testtag '%(committeremail:trim)' +batch_test_atom refs/tags/testtag '%(committeremail:localpart)' +batch_test_atom refs/tags/testtag '%(committerdate)' +batch_test_atom refs/tags/testtag '%(tag)' +batch_test_atom refs/tags/testtag '%(tagger)' +batch_test_atom refs/tags/testtag '%(taggername)' +batch_test_atom refs/tags/testtag '%(taggeremail)' +batch_test_atom refs/tags/testtag '%(taggeremail:trim)' +batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' +batch_test_atom refs/tags/testtag '%(taggerdate)' +batch_test_atom refs/tags/testtag '%(creator)' +batch_test_atom refs/tags/testtag '%(creatordate)' +batch_test_atom refs/tags/testtag '%(subject)' +batch_test_atom refs/tags/testtag '%(subject:sanitize)' +batch_test_atom refs/tags/testtag '%(contents:subject)' +batch_test_atom refs/tags/testtag '%(body)' +batch_test_atom refs/tags/testtag '%(contents:body)' +batch_test_atom refs/tags/testtag '%(contents:signature)' +batch_test_atom refs/tags/testtag '%(contents)' +batch_test_atom refs/tags/testtag '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(refname)' fail +batch_test_atom refs/myblobs/blob1 '%(upstream)' fail +batch_test_atom refs/myblobs/blob1 '%(push)' fail +batch_test_atom refs/myblobs/blob1 '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(objectname)' +batch_test_atom refs/myblobs/blob1 '%(objecttype)' +batch_test_atom refs/myblobs/blob1 '%(objectsize)' +batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)' +batch_test_atom refs/myblobs/blob1 '%(deltabase)' + +batch_test_atom refs/myblobs/blob1 '%(contents)' +batch_test_atom refs/myblobs/blob2 '%(contents)' + +batch_test_atom refs/myblobs/blob1 '%(raw)' +batch_test_atom refs/myblobs/blob2 '%(raw)' +batch_test_atom refs/mytrees/tree1 '%(raw)' + +batch_test_atom refs/myblobs/blob1 '%(raw:size)' +batch_test_atom refs/myblobs/blob2 '%(raw:size)' +batch_test_atom refs/mytrees/tree1 '%(raw:size)' + +batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' + +batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' + +batch_test_atom refs/heads/main 'VALID' +batch_test_atom refs/heads/main '%(INVALID)' fail +batch_test_atom refs/heads/main '%(authordate:INVALID)' fail + +test_expect_success '%(rest) works with both a branch and a tag' ' + cat >expected <<-EOF && + 123 commit 123 + 456 tag 456 + EOF + git cat-file --batch-check="%(rest) %(objecttype) %(rest)" >actual <<-EOF && + refs/heads/main 123 + refs/tags/testtag 456 + EOF + test_cmp expected actual +' + +batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize) +%(raw)' + + +test_expect_success 'cat-file --batch equals to --batch-check with atoms' ' + git cat-file --batch-check="%(objectname) %(objecttype) %(objectsize) +%(raw)" >expected <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + git cat-file --batch >actual <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + cmp expected actual +' + test_done From patchwork Fri Jun 25 16:02:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2181C49EA7 for ; Fri, 25 Jun 2021 16:02:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B4D6961976 for ; Fri, 25 Jun 2021 16:02:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230177AbhFYQFQ (ORCPT ); Fri, 25 Jun 2021 12:05:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230091AbhFYQE6 (ORCPT ); Fri, 25 Jun 2021 12:04:58 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53884C061766 for ; Fri, 25 Jun 2021 09:02:37 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id b3so11126520wrm.6 for ; Fri, 25 Jun 2021 09:02:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+bsFc6e3DqMp3rf/hPSj63ldu2+jiHin07ZYcR5fSM4=; b=GcazcYlxL8Z7B/kN73dn3MW0kgOkxj+sl6DKeOH6UAWS2mFkdmSTg1D/Ba6vVAyVMa ipy8MZ5hKUAs3R6MHsOyqIL1wz83Y7p4Mug0ki67H8gFx/lVzzA2dETzeJwngDztIxvC tvl2Mg7I3O+sNeSOfn/OKmg9QOy0zMaXeZSVFA5nxIRZ8U2Nixj5tHmAtn2U9BIspGLB Xm4xAllfTRucPvH2ezbgVQeACOD5AZt+QDNrOlaCiRB1xvU+bPyMzYUcRrj7/e9Jpprp 5DW/lB7CqFNqjmDxkw3IDtAjdG32avSpShAT1X+0YKODKluvKRpxFKwFpzSMmib6MHiQ h7Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+bsFc6e3DqMp3rf/hPSj63ldu2+jiHin07ZYcR5fSM4=; b=dZLsr65KC1viBnABNRYR/252A/aACfTC7t6fJFTyK/Fon8UXdqjC9M44zRyBQXTUJ3 xcmFlm52uR3mQkGXiA/GQF7Rtzqlo6/zHWZ4eWbrh8foK619Ct8FNpMfwhY50Sj82kfV FjF8hFft10Yth6VF2Xv0dyY7DsnjtJg6gEGTFBkA48pM3TWujLX+oxSGYIrYZMGTNxm3 vYWtvIKtHfNv8csdPDnl0FJnKpXs8Wg6PkzlQ2YDkn6q7Iym9mQ7w5UwkRr6Snc3fcTl 7dEW1fuMngF5qub5Dd+TEurorbJIy2pqUTMbFdKUWwQycVCSf8J8r7t2hxZOKdP6srn4 /WwA== X-Gm-Message-State: AOAM533YQi4LRChk6zUgTT+RewNTw/ZzZVTurWLL5nzqpgAc93K4R5d6 VcV8eWoWJ0oeHdrx9oc9RagjpdsoHcA= X-Google-Smtp-Source: ABdhPJy42mqtiQ60MY/eTl5jpqhnVLLcPwZvf0Uyxb+QT0le2avfEpSnhZBjbTvh71KLIdmhTc+XzQ== X-Received: by 2002:a5d:6d8b:: with SMTP id l11mr11786633wrs.21.1624636956033; Fri, 25 Jun 2021 09:02:36 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o203sm6786252wmo.36.2021.06.25.09.02.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:35 -0700 (PDT) Message-Id: <69eef47065d27cc997a26c853691534c5d84df6d.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:23 +0000 Subject: [PATCH v5 13/15] [GSOC] cat-file: reuse err buf in batch_object_write() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Reuse the `err` buffer in batch_object_write(), as the buffer `scratch` does. This will reduce the overhead of multiple allocations of memory of the err buffer. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 0e7ad038e5f..27403326e7a 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -212,35 +212,36 @@ static void batch_write(struct batch_options *opt, const void *data, int len) static void batch_object_write(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { int ret = 0; - struct strbuf err = STRBUF_INIT; struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); + strbuf_reset(err); - ret = format_ref_array_item(&item, &opt->format, scratch, &err); + ret = format_ref_array_item(&item, &opt->format, scratch, err); if (ret < 0) { - die("%s\n", err.buf); + die("%s\n", err->buf); } if (ret) { /* ret > 0 means when the object corresponding to oid * cannot be found in format_ref_array_item(), we only print * the error message. */ - printf("%s\n", err.buf); + printf("%s\n", err->buf); fflush(stdout); } else { strbuf_addch(scratch, '\n'); batch_write(opt, scratch->buf, scratch->len); } free_ref_array_item_value(&item); - strbuf_release(&err); } static void batch_one_object(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { @@ -294,7 +295,7 @@ static void batch_one_object(const char *obj_name, return; } - batch_object_write(obj_name, scratch, opt, data); + batch_object_write(obj_name, scratch, err, opt, data); } struct object_cb_data { @@ -302,13 +303,14 @@ struct object_cb_data { struct expand_data *expand; struct oidset *seen; struct strbuf *scratch; + struct strbuf *err; }; static int batch_object_cb(const struct object_id *oid, void *vdata) { struct object_cb_data *data = vdata; oidcpy(&data->expand->oid, oid); - batch_object_write(NULL, data->scratch, data->opt, data->expand); + batch_object_write(NULL, data->scratch, data->err, data->opt, data->expand); return 0; } @@ -364,6 +366,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; @@ -392,6 +395,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio cb.opt = batch; cb.expand = &data; cb.scratch = &output; + cb.err = &err; if (batch->unordered) { struct oidset seen = OIDSET_INIT; @@ -416,6 +420,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio strbuf_release(&format); strbuf_release(&output); + strbuf_release(&err); return 0; } @@ -444,12 +449,13 @@ static int batch_objects(struct batch_options *batch, const struct option *optio data.rest = p; } - batch_one_object(input.buf, &output, batch, &data); + batch_one_object(input.buf, &output, &err, batch, &data); } strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); + strbuf_release(&err); warn_on_object_refname_ambiguity = save_warning; return retval; } From patchwork Fri Jun 25 16:02:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27614C2B9F4 for ; Fri, 25 Jun 2021 16:02:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 112B361976 for ; Fri, 25 Jun 2021 16:02:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230188AbhFYQFS (ORCPT ); Fri, 25 Jun 2021 12:05:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229997AbhFYQFA (ORCPT ); Fri, 25 Jun 2021 12:05:00 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28A15C061574 for ; Fri, 25 Jun 2021 09:02:38 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id j2so11117064wrs.12 for ; Fri, 25 Jun 2021 09:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=I1siE7gIfw4HR3UIZz1G6rr97pPh2jQSzQNQqQn4Rpc=; b=Z+zpLqYfE7q9hCl5b7ZRP2czHa+tn+dgSdH5hL5nLxncNNcOpUEfoGnVx2ZXVV6in9 MJ4wPdX6Hbk3mfZcbtc9m/qV0P6NUmX2VyVpUsp8M1OT8Dyb5GDQp1ALhOSGvi4/1vVh 3jeyyGwYuwW2MOgB/z59rRYN7GpPXY82MDxKKu7oLIpKwBGCszbtigAVsldldJHVG1zP 104RbCI40fSiWn83tcbmjto1tmj2u/Pq7u5CgpfnXRai6nYpsP47q2rb6+JsGpAONNYx ApGYLsOsA4BIoj3SkghZDERuwPrDmDrwQ0mPIkqQkmN2iOiPE3VvJRzC6fQMphePmxIO M3jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=I1siE7gIfw4HR3UIZz1G6rr97pPh2jQSzQNQqQn4Rpc=; b=PDA/2piI76UaKAF1UoisCX3XFEoVyYsnT6tI8Q9yJ/IoF1XHsEGOI8IJQmDcDkkgfL kIVb7dyU7E/Xvupjeh8vcNhfVrJS0r7bhmBGigp/gQKxWXCDMhURFafXoaNf050DNoL8 uXN9gjn/1ubVUR+Gg6dP7ZlUUVH8aDWFpT3GEMzWpnbNzd5TAraSnm6q1BBuKQkavan0 P/3qRowXYd/OaT272lCIf4gomYtzuqT8r/6/3gEhviR4IG06jyo86XJksjVL9+4JVjdd VMU9GvHkkdo8GnDVShp5hNE8PItsHfzUM8SKM4f2jj92p9TnA/oza/G5P48oTJ4OLGhL gcmw== X-Gm-Message-State: AOAM532chuVTlGwE1HmYn4PV/Sx0bgQ8+guA1kArLcWguHvbqaQ//lN/ yfVOh9pkQEg5VYjpWirSZHiPP5lxGZk= X-Google-Smtp-Source: ABdhPJwCpbGiq8teTsaypsXxVxV9CyEIFrw9NENcZwKxuh5JdOqkECKxqMTceUvUVlrCpN93Pi309A== X-Received: by 2002:a5d:4a91:: with SMTP id o17mr12001236wrq.335.1624636956678; Fri, 25 Jun 2021 09:02:36 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v22sm5585718wmh.40.2021.06.25.09.02.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:36 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:24 +0000 Subject: [PATCH v5 14/15] [GSOC] cat-file: re-implement --textconv, --filters options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu After cat-file reuses the ref-filter logic, we re-implement the functions of --textconv and --filters options. Add members `use_textconv` and `use_filters` in struct `ref_format`, and use global variables `use_filters` and `use_textconv` in `ref-filter.c`, so that we can filter the content of the object in get_object(). Use `actual_oi` to record the real expand_data: it may point to the original `oi` or the `act_oi` processed by `textconv_object()` or `convert_to_working_tree()`. `grab_values()` will grab the contents of `actual_oi` and `grab_common_values()` to grab the contents of origin `oi`, this ensures that `%(objectsize)` still uses the size of the unfiltered data. In `get_object()`, we made an optimization: Firstly, get the size and type of the object instead of directly getting the object data. If using --textconv, after successfully obtaining the filtered object data, an extra oid_object_info_extended() will be skipped, which can reduce the cost of object data copy; If using --filter, the data of the object first will be getted first, and then convert_to_working_tree() will be used to get the filtered object data. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 6 +++++ ref-filter.c | 59 ++++++++++++++++++++++++++++++++++++++++++++-- ref-filter.h | 2 ++ 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 27403326e7a..f3140d927f7 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -380,6 +380,12 @@ static int batch_objects(struct batch_options *batch, const struct option *optio if (batch->print_contents) strbuf_addstr(&format, "\n%(raw)"); batch->format.format = format.buf; + + if (batch->cmdmode == 'c') + batch->format.use_textconv = 1; + else if (batch->cmdmode == 'w') + batch->format.use_filters = 1; + if (verify_ref_format(&batch->format)) usage_with_options(cat_file_usage, options); diff --git a/ref-filter.c b/ref-filter.c index 181d99c9273..99b87742b0f 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1,3 +1,4 @@ +#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "cache.h" #include "parse-options.h" @@ -84,6 +85,9 @@ static struct expand_data { struct object_info info; } oi, oi_deref; +int use_filters; +int use_textconv; + struct ref_to_worktree_entry { struct hashmap_entry ent; struct worktree *wt; /* key is wt->head_ref */ @@ -1031,6 +1035,9 @@ int verify_ref_format(struct ref_format *format) used_atom[at].atom_type == ATOM_WORKTREEPATH))) die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); + use_filters = format->use_filters; + use_textconv = format->use_textconv; + if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || format->quote_style == QUOTE_TCL) && @@ -1742,10 +1749,38 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj { /* parse_object_buffer() will set eaten to 0 if free() will be needed */ int eaten = 1; + struct expand_data *actual_oi = oi; + struct expand_data act_oi = {0}; + if (oi->info.contentp) { /* We need to know that to use parse_object_buffer properly */ + void **temp_contentp = oi->info.contentp; + oi->info.contentp = NULL; oi->info.sizep = &oi->size; oi->info.typep = &oi->type; + + /* get the type and size */ + if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, + OBJECT_INFO_LOOKUP_REPLACE)) + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); + + oi->info.sizep = NULL; + oi->info.typep = NULL; + oi->info.contentp = temp_contentp; + + if (use_textconv && !ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&act_oi.oid)); + if (use_textconv && oi->type == OBJ_BLOB) { + act_oi = *oi; + if (textconv_object(the_repository, + ref->rest, 0100644, &act_oi.oid, + 1, (char **)(&act_oi.content), &act_oi.size)) { + actual_oi = &act_oi; + goto success; + } + } } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) @@ -1755,19 +1790,39 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj BUG("Object size is less than zero."); if (oi->info.contentp) { - *obj = parse_object_buffer(the_repository, &oi->oid, oi->type, oi->size, oi->content, &eaten); + if (use_filters && !ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&oi->oid)); + if (use_filters && oi->type == OBJ_BLOB) { + struct strbuf strbuf = STRBUF_INIT; + struct checkout_metadata meta; + act_oi = *oi; + + init_checkout_metadata(&meta, NULL, NULL, &act_oi.oid); + if (!convert_to_working_tree(&the_index, ref->rest, act_oi.content, act_oi.size, &strbuf, &meta)) + die("could not convert '%s' %s", + oid_to_hex(&oi->oid), ref->rest); + act_oi.size = strbuf.len; + act_oi.content = strbuf_detach(&strbuf, NULL); + actual_oi = &act_oi; + } + +success: + *obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten); if (!*obj) { if (!eaten) free(oi->content); return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi); + grab_values(ref->value, deref, *obj, actual_oi); } grab_common_values(ref->value, deref, oi); if (!eaten) free(oi->content); + if (actual_oi != oi) + free(actual_oi->content); return 0; } diff --git a/ref-filter.h b/ref-filter.h index 053980a6a42..497e3e93632 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -80,6 +80,8 @@ struct ref_format { const char *rest; int cat_file_mode; int quote_style; + int use_textconv; + int use_filters; int use_rest; int use_color; From patchwork Fri Jun 25 16:02:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12345579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01F7FC49EAF for ; Fri, 25 Jun 2021 16:02:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DF41F61969 for ; Fri, 25 Jun 2021 16:02:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230205AbhFYQFS (ORCPT ); Fri, 25 Jun 2021 12:05:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230018AbhFYQFA (ORCPT ); Fri, 25 Jun 2021 12:05:00 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC424C061787 for ; Fri, 25 Jun 2021 09:02:38 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id a5-20020a7bc1c50000b02901e3bbe0939bso6202002wmj.0 for ; Fri, 25 Jun 2021 09:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=wzxPQDRwRGk2/Nmy4hWXIToRWnXsdBb3MWMALKink7A=; b=rfRoK/gGI/qlNpf+aj2lSGsjxIPAmfvNQWQ/0DqH1Boz1poXbPLGzPlbNTChMYZzOZ q1HwntlENYQ1OcKSoFiDmWOVua7raBBP8a+sS0JOVlx172QyBDh1JA+bdb8CQradyr+s UvsE3i3W6/n8PlnVWY3k7Tn/VLvz4RiD0mg5QNRLMuVPiqi3OQL+qOOp6pZJC6iBEgBF bM8XJAqgcXdlylWxt88C9q6SWjz/jEwQEMr8SeoVy+VgTqsM0AP9Sdy9fig2Gs+FDsxP H7eo8XPDW6tdJYqqCwW0V+oDwSwpusLX2oeGr4m3mcNcDv3CF6gPCH+0SzjQzdP+AO9d asZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=wzxPQDRwRGk2/Nmy4hWXIToRWnXsdBb3MWMALKink7A=; b=BT862XEUitm4Sq/qiFsVvV1bOzEcoQJUfb3z9SHOA1OuH7BA8GGZqzhKBMmdnr02Wt h1jDhqPWMNfsEDccpOISAFbmppACTTAeSIpuSENY4aFmWo4UY7V8ggNJDV1vJXfW9h/d wYPk2G8iI2nb3cXjUeMr5L0WyVoc/6lt+tXQEUUV6aS6FDmXcI2bn9gIE1CgqDd88qG2 QRIaORaCKAw7ZkU7WOhUfEwG3TCvgc45t3KK6joVF7dVcIvcWczlu3NJmJ2HAfUcLRKF MoDNXZ/5rvnIBeZO8QaZ1sYIOIz3HeHlW+pveU+IPiebK6rzxqkza795eqwv6xBUwAzh H3nA== X-Gm-Message-State: AOAM533eoi+wHKANNL86n6jmopzCZ2ruUNzkwiOSK6qZ0idT7qu+b6vz YLIT2tPku7aBK1Yyh7cyksv7Z6wUPwo= X-Google-Smtp-Source: ABdhPJw4J1cVvtmBoEhMxX+/8/jwzmri3mWv8PbzbZ2VUnlQ6XvQCHs3NZNKqfzWZTeFJNRXkW+jAg== X-Received: by 2002:a1c:730d:: with SMTP id d13mr11786308wmb.129.1624636957237; Fri, 25 Jun 2021 09:02:37 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l2sm4357640wms.20.2021.06.25.09.02.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:02:36 -0700 (PDT) Message-Id: <843de8864a9fa0b89b5dede0d24a982959b0ad1a.1624636945.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 25 Jun 2021 16:02:25 +0000 Subject: [PATCH v5 15/15] [GSOC] ref-filter: remove grab_oid() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because "atom_type == ATOM_OBJECTNAME" implies the condition of `starts_with(name, "objectname")`, "atom_type == ATOM_TREE" implies the condition of `starts_with(name, "tree")`, so the check for `starts_with(name, field)` in grab_oid() is redundant. So Remove the grab_oid() from ref-filter, to reduce repeated check. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 99b87742b0f..ab53c1fd22f 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1071,16 +1071,6 @@ static const char *do_grab_oid(const char *field, const struct object_id *oid, } } -static int grab_oid(const char *name, const char *field, const struct object_id *oid, - struct atom_value *v, struct used_atom *atom) -{ - if (starts_with(name, field)) { - v->s = xstrdup(do_grab_oid(field, oid, atom)); - return 1; - } - return 0; -} - /* See grab_values */ static void grab_common_values(struct atom_value *val, int deref, struct expand_data *oi) { @@ -1106,8 +1096,9 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ } } else if (atom_type == ATOM_DELTABASE) v->s = xstrdup(oid_to_hex(&oi->delta_base_oid)); - else if (atom_type == ATOM_OBJECTNAME && deref) - grab_oid(name, "objectname", &oi->oid, v, &used_atom[i]); + else if (atom_type == ATOM_OBJECTNAME && deref) { + v->s = xstrdup(do_grab_oid("objectname", &oi->oid, &used_atom[i])); + } } } @@ -1148,9 +1139,10 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object continue; if (deref) name++; - if (atom_type == ATOM_TREE && - grab_oid(name, "tree", get_commit_tree_oid(commit), v, &used_atom[i])) + if (atom_type == ATOM_TREE) { + v->s = xstrdup(do_grab_oid("tree", get_commit_tree_oid(commit), &used_atom[i])); continue; + } if (atom_type == ATOM_NUMPARENT) { v->value = commit_list_count(commit->parents); v->s = xstrfmt("%lu", (unsigned long)v->value); @@ -1971,9 +1963,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->s = xstrdup(buf + 1); } continue; - } else if (!deref && atom_type == ATOM_OBJECTNAME && - grab_oid(name, "objectname", &ref->objectname, v, atom)) { - continue; + } else if (!deref && atom_type == ATOM_OBJECTNAME) { + v->s = xstrdup(do_grab_oid("objectname", &ref->objectname, atom)); + continue; } else if (atom_type == ATOM_HEAD) { if (atom->u.head && !strcmp(ref->refname, atom->u.head)) v->s = xstrdup("*");