From patchwork Fri Aug 13 08:22:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC9C5C432BE for ; Fri, 13 Aug 2021 08:23:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D23FD6109E for ; Fri, 13 Aug 2021 08:23:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238582AbhHMIXq (ORCPT ); Fri, 13 Aug 2021 04:23:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234152AbhHMIXk (ORCPT ); Fri, 13 Aug 2021 04:23:40 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC8BDC0617AD for ; Fri, 13 Aug 2021 01:23:13 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id f10so3211382wml.2 for ; Fri, 13 Aug 2021 01:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=MMY3jZF25WfsbdhxZPrT7X1HRSO8COXkupSiBa9x7T0=; b=W1A6lbpTKRYRVEn+zWS/wsEBuv89ZOGvFiBUDpL8RJzqE3/DewcNBsfxpqsK17Fn81 1iiBul8BHbm9O+KrdWVNbz3XVWvqrzEXActQOtklF2l6s356GyENBaPTiJSk7mF3eCj6 tZPNcsxE8G9ZG9gy43lStQ1slW2+8nbFP2Vp2Ys4C08LNVdlKDY0c8wbsPiv26qXzEox n3QUhF/Z3nbFdLc5NfKIeRBI2g20Kc3qDvq/T1y5GzkPMJVjVVBhAhQ9drqhHoU5hQoE vgvIRs1gcwhBlVvN7T/d8HQHhgebnakPmvS1JVa0EatPUbJ1AbU7HiDw9oeWpBDupILl lixA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=MMY3jZF25WfsbdhxZPrT7X1HRSO8COXkupSiBa9x7T0=; b=uICDF4Jj7UwKyR2PaikkqhwnxLj5vt6EC53H+jIscAU8cq4hE0hlhZI+k3iYCwPQ1y +PwPurR2nBLaWDDyqpMDc/f5OX43dvecHfWXc6+iMlXjCFmjS+Ml5dP1oyOjBeg6FOLw BrGmje/iRkGSLXhd/84HpKErLSIiwe0AKVPtB45K2T0EVhNo1yXGldRcfh37pjyzx0Co KUlsk3+3+8FNoZ4oRFQ8aiAfykle68p8GBl/TNC2juDukuvIb5UWZg8BOwArjMeUPzzZ UNSLX6xTqkvnAGiH0KJYkmWo+k8cZOT8fRuOhFnfQyopANMpC/oLeIbLu7NlSLVeh24V M7aw== X-Gm-Message-State: AOAM531D/XyvqWNWFMgJPJRKjSbiGkoDKkbC8GZqA4mrg2wduUFM9kaY wmjj6wEnW+nAvkeoAeYrErO1NS9+arY= X-Google-Smtp-Source: ABdhPJw8QjIKPtn+Uev2KO6oFB0w20O7Lmg+wjmknSAnMXF/lAHgtk7jiTkGbAEvPcRI+T5ruAiH6A== X-Received: by 2002:a7b:c416:: with SMTP id k22mr1465035wmi.177.1628842992569; Fri, 13 Aug 2021 01:23:12 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n5sm645667wme.47.2021.08.13.01.23.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:12 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:44 +0000 Subject: [PATCH 01/27] [GSOC] ref-filter: add obj-type check in grab contents Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Only tag and commit objects use `grab_sub_body_contents()` to grab object contents in the current codebase. We want to teach the function to also handle blobs and trees to get their raw data, without parsing a blob (whose contents looks like a commit or a tag) incorrectly as a commit or a tag. So it's needed to pass a `struct expand_data *data` instread of only `void *buf` to both `grab_sub_body_contents()` and `grab_values()` to be able to check the object type. Skip the block of code that is specific to handling commits and tags early when the given object is of a wrong type to help later addition to handle other types of objects in this function. Reviewed-by: Jacob Keller Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Junio C Hamano Signed-off-by: ZheNing Hu --- ref-filter.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 0cfef7b719b..e3715210e5c 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1357,11 +1357,12 @@ static void append_lines(struct strbuf *out, const char *buf, unsigned long size } /* See grab_values */ -static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) +static void grab_sub_body_contents(struct atom_value *val, int deref, struct expand_data *data) { int i; const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; + void *buf = data->content; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; @@ -1372,10 +1373,13 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) continue; if (deref) name++; - if (strcmp(name, "body") && - !starts_with(name, "subject") && - !starts_with(name, "trailers") && - !starts_with(name, "contents")) + + if ((data->type != OBJ_TAG && + data->type != OBJ_COMMIT) || + (strcmp(name, "body") && + !starts_with(name, "subject") && + !starts_with(name, "trailers") && + !starts_with(name, "contents"))) continue; if (!subpos) find_subpos(buf, @@ -1439,17 +1443,19 @@ static void fill_missing_values(struct atom_value *val) * pointed at by the ref itself; otherwise it is the object the * ref (which is a tag) refers to. */ -static void grab_values(struct atom_value *val, int deref, struct object *obj, void *buf) +static void grab_values(struct atom_value *val, int deref, struct object *obj, struct expand_data *data) { + void *buf = data->content; + switch (obj->type) { case OBJ_TAG: grab_tag_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("tagger", val, deref, buf); break; case OBJ_COMMIT: grab_commit_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("author", val, deref, buf); grab_person("committer", val, deref, buf); break; @@ -1679,7 +1685,7 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi->content); + grab_values(ref->value, deref, *obj, oi); } grab_common_values(ref->value, deref, oi); From patchwork Fri Aug 13 08:22:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1418C4320A for ; Fri, 13 Aug 2021 08:23:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 815B3610E9 for ; Fri, 13 Aug 2021 08:23:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238533AbhHMIXq (ORCPT ); Fri, 13 Aug 2021 04:23:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235413AbhHMIXl (ORCPT ); Fri, 13 Aug 2021 04:23:41 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BAD74C0617AE for ; Fri, 13 Aug 2021 01:23:14 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id h13so12173898wrp.1 for ; Fri, 13 Aug 2021 01:23:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=mLifnQKlorZ0J4UXJ/fKprkfZarPmuaD56/Dgkl6g28=; b=e5T9ES2u0lc02oJZhstmslSNMtFtmWYAf3xomRN4kOqGsxzmbZl7R9xhh9HldHlQDo 0jIDIJ10UA9t0P9VZyirUKcKmXfDtXgQhdqtusJCjhJNiIEC7EMMsjT9ucV5x8m4d988 BQ+94Lr2XwLb1AgB34ZCroftVftFNhZkCjm8rrkEFHbpCeCGyvwZy6W5Vy98xLL5mfWi d8uf1/j4V/EoRv2Jhxlr/3thfBRQ+5HRT0cjgk9xIGZ49wacfwm6jzHzUsGelZVVWykB fS4lYqDKitU955kkjWf0wy5UNMzrOv6V5+ygFFBgBTTYJDqlrne3rW9LPOVq8CvIpb4l c8Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=mLifnQKlorZ0J4UXJ/fKprkfZarPmuaD56/Dgkl6g28=; b=IXv9YqWN7hFbz7I4Pt3D1d7/QJOUh8VR87ZDRaFfq2aw9a7aE5goxm0mPOeQW81vfp h8+7LXcwgX8+KiTlc4VlquGvQXsKqFeLJ2i+NC/+8xI/w3fl0ES91mqNU5aWO0Ax2mq3 OFRRyEeusGu8sszuXZmZUk4KV4L8JxNXE23Lt1+bDZEV6G+uTZ1DtPeziSSvVdAZFbvw KMr28LZzeX4rWs/1pBCTF9e92VDAmqNX3BIDs1Eag66tzZaqfb8QxE6jwGb7laIdd1TE Rg9MgB1kvfJ55h0HYIahA4zQIEG/AxlY5qLBdcq6zYFzjJZrfYt0JtL+8mjt8C0pqQvo uFnA== X-Gm-Message-State: AOAM5320FOLy4NPbkL0xGwcKqvV56t7WRjDrmi7fFwGocYOiq+WPs+/F YzNEQf05PBqdXJgWUCK75h+jaVlX4dU= X-Google-Smtp-Source: ABdhPJwyriOl6tzt4dyUwZnfmrnuyYQt8fBTLiXD/Su3EsG9NpbJnA0GhsENmQ0NqCkycKpVqrpliA== X-Received: by 2002:a5d:4490:: with SMTP id j16mr1636852wrq.272.1628842993193; Fri, 13 Aug 2021 01:23:13 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o24sm706368wmm.37.2021.08.13.01.23.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:12 -0700 (PDT) Message-Id: <59577ccf140900bd04177ffc7ef0afb92c2d45e2.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:45 +0000 Subject: [PATCH 02/27] [GSOC] ref-filter: add %(raw) atom MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add new formatting option `%(raw)`, which will print the raw object data without any changes. It will help further to migrate all cat-file formatting logic from cat-file to ref-filter. The raw data of blob, tree objects may contain '\0', but most of the logic in `ref-filter` depends on the output of the atom being text (specifically, no embedded NULs in it). E.g. `quote_formatting()` use `strbuf_addstr()` or `*._quote_buf()` add the data to the buffer. The raw data of a tree object is `100644 one\0...`, only the `100644 one` will be added to the buffer, which is incorrect. Therefore, we need to find a way to record the length of the atom_value's member `s`. Although strbuf can already record the string and its length, if we want to replace the type of atom_value's member `s` with strbuf, many places in ref-filter that are filled with dynamically allocated mermory in `v->s` are not easy to replace. At the same time, we need to check if `v->s == NULL` in populate_value(), and strbuf cannot easily distinguish NULL and empty strings, but c-style "const char *" can do it. So add a new member in `struct atom_value`: `s_size`, which can record raw object size, it can help us add raw object data to the buffer or compare two buffers which contain raw object data. Note that `--format=%(raw)` cannot be used with `--python`, `--shell`, `--tcl`, and `--perl` because if the binary raw data is passed to a variable in such languages, these may not support arbitrary binary data in their string variable type. Reviewed-by: Jacob Keller Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Bagas Sanjaya Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Felipe Contreras Helped-by: Phillip Wood Helped-by: Junio C Hamano Based-on-patch-by: Olga Telezhnaya Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 9 ++ ref-filter.c | 143 +++++++++++++++---- t/t6300-for-each-ref.sh | 216 +++++++++++++++++++++++++++++ 3 files changed, 340 insertions(+), 28 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 2ae2478de70..cbb6f87d13f 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -235,6 +235,15 @@ and `date` to extract the named component. For email fields (`authoremail`, without angle brackets, and `:localpart` to get the part before the `@` symbol out of the trimmed email. +The raw data in an object is `raw`. + +raw:size:: + The raw data size of the object. + +Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, +`--perl` because such language may not support arbitrary binary data in their +string variable type. + The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/ref-filter.c b/ref-filter.c index e3715210e5c..1e660f60c4b 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -144,6 +144,7 @@ enum atom_type { ATOM_BODY, ATOM_TRAILERS, ATOM_CONTENTS, + ATOM_RAW, ATOM_UPSTREAM, ATOM_PUSH, ATOM_SYMREF, @@ -189,6 +190,9 @@ static struct used_atom { struct process_trailer_options trailer_opts; unsigned int nlines; } contents; + struct { + enum { RAW_BARE, RAW_LENGTH } option; + } raw_data; struct { cmp_status cmp_status; const char *str; @@ -427,6 +431,18 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } +static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (!arg) + atom->u.raw_data.option = RAW_BARE; + else if (!strcmp(arg, "size")) + atom->u.raw_data.option = RAW_LENGTH; + else + return strbuf_addf_ret(err, -1, _("unrecognized %%(raw) argument: %s"), arg); + return 0; +} + static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { @@ -587,6 +603,7 @@ static struct { [ATOM_BODY] = { "body", SOURCE_OBJ, FIELD_STR, body_atom_parser }, [ATOM_TRAILERS] = { "trailers", SOURCE_OBJ, FIELD_STR, trailers_atom_parser }, [ATOM_CONTENTS] = { "contents", SOURCE_OBJ, FIELD_STR, contents_atom_parser }, + [ATOM_RAW] = { "raw", SOURCE_OBJ, FIELD_STR, raw_atom_parser }, [ATOM_UPSTREAM] = { "upstream", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_PUSH] = { "push", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_SYMREF] = { "symref", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -621,12 +638,19 @@ struct ref_formatting_state { struct atom_value { const char *s; + ssize_t s_size; int (*handler)(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err); uintmax_t value; /* used for sorting when not FIELD_STR */ struct used_atom *atom; }; +#define ATOM_SIZE_UNSPECIFIED (-1) + +#define ATOM_VALUE_INIT { \ + .s_size = ATOM_SIZE_UNSPECIFIED \ +} + /* * Used to parse format string and sort specifiers */ @@ -645,13 +669,6 @@ static int parse_ref_filter_atom(const struct ref_format *format, return strbuf_addf_ret(err, -1, _("malformed field name: %.*s"), (int)(ep-atom), atom); - /* Do we have the atom already used elsewhere? */ - for (i = 0; i < used_atom_cnt; i++) { - int len = strlen(used_atom[i].name); - if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) - return i; - } - /* * If the atom name has a colon, strip it and everything after * it off - it specifies the format for this entry, and @@ -661,6 +678,13 @@ static int parse_ref_filter_atom(const struct ref_format *format, arg = memchr(sp, ':', ep - sp); atom_len = (arg ? arg : ep) - sp; + /* Do we have the atom already used elsewhere? */ + for (i = 0; i < used_atom_cnt; i++) { + int len = strlen(used_atom[i].name); + if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) + return i; + } + /* Is the atom a valid one? */ for (i = 0; i < ARRAY_SIZE(valid_atom); i++) { int len = strlen(valid_atom[i].name); @@ -710,11 +734,14 @@ static int parse_ref_filter_atom(const struct ref_format *format, return at; } -static void quote_formatting(struct strbuf *s, const char *str, int quote_style) +static void quote_formatting(struct strbuf *s, const char *str, ssize_t len, int quote_style) { switch (quote_style) { case QUOTE_NONE: - strbuf_addstr(s, str); + if (len < 0) + strbuf_addstr(s, str); + else + strbuf_add(s, str, len); break; case QUOTE_SHELL: sq_quote_buf(s, str); @@ -741,9 +768,11 @@ static int append_atom(struct atom_value *v, struct ref_formatting_state *state, * encountered. */ if (!state->stack->prev) - quote_formatting(&state->stack->output, v->s, state->quote_style); - else + quote_formatting(&state->stack->output, v->s, v->s_size, state->quote_style); + else if (v->s_size < 0) strbuf_addstr(&state->stack->output, v->s); + else + strbuf_add(&state->stack->output, v->s, v->s_size); return 0; } @@ -843,21 +872,23 @@ static int if_atom_handler(struct atom_value *atomv, struct ref_formatting_state return 0; } -static int is_empty(const char *s) +static int is_empty(struct strbuf *buf) { - while (*s != '\0') { - if (!isspace(*s)) - return 0; - s++; - } - return 1; -} + const char *cur = buf->buf; + const char *end = buf->buf + buf->len; + + while (cur != end && (isspace(*cur))) + cur++; + + return cur == end; + } static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err) { struct ref_formatting_stack *cur = state->stack; struct if_then_else *if_then_else = NULL; + size_t str_len = 0; if (cur->at_end == if_then_else_handler) if_then_else = (struct if_then_else *)cur->at_end_data; @@ -868,18 +899,22 @@ static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_sta if (if_then_else->else_atom_seen) return strbuf_addf_ret(err, -1, _("format: %%(then) atom used after %%(else)")); if_then_else->then_atom_seen = 1; + if (if_then_else->str) + str_len = strlen(if_then_else->str); /* * If the 'equals' or 'notequals' attribute is used then * perform the required comparison. If not, only non-empty * strings satisfy the 'if' condition. */ if (if_then_else->cmp_status == COMPARE_EQUAL) { - if (!strcmp(if_then_else->str, cur->output.buf)) + if (str_len == cur->output.len && + !memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; } else if (if_then_else->cmp_status == COMPARE_UNEQUAL) { - if (strcmp(if_then_else->str, cur->output.buf)) + if (str_len != cur->output.len || + memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; - } else if (cur->output.len && !is_empty(cur->output.buf)) + } else if (cur->output.len && !is_empty(&cur->output)) if_then_else->condition_satisfied = 1; strbuf_reset(&cur->output); return 0; @@ -925,7 +960,7 @@ static int end_atom_handler(struct atom_value *atomv, struct ref_formatting_stat * only on the topmost supporting atom. */ if (!current->prev->prev) { - quote_formatting(&s, current->output.buf, state->quote_style); + quote_formatting(&s, current->output.buf, current->output.len, state->quote_style); strbuf_swap(¤t->output, &s); } strbuf_release(&s); @@ -975,6 +1010,10 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) + die(_("--format=%.*s cannot be used with" + "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) @@ -1368,12 +1407,25 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp struct used_atom *atom = &used_atom[i]; const char *name = atom->name; struct atom_value *v = &val[i]; + enum atom_type atom_type = atom->atom_type; if (!!deref != (*name == '*')) continue; if (deref) name++; + if (atom_type == ATOM_RAW) { + unsigned long buf_size = data->size; + + if (atom->u.raw_data.option == RAW_BARE) { + v->s = xmemdupz(buf, buf_size); + v->s_size = buf_size; + } else if (atom->u.raw_data.option == RAW_LENGTH) { + v->s = xstrfmt("%"PRIuMAX, (uintmax_t)buf_size); + } + continue; + } + if ((data->type != OBJ_TAG && data->type != OBJ_COMMIT) || (strcmp(name, "body") && @@ -1461,9 +1513,11 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s break; case OBJ_TREE: /* grab_tree_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; case OBJ_BLOB: /* grab_blob_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; default: die("Eh? Object of type %d?", obj->type); @@ -1767,6 +1821,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) const char *refname; struct branch *branch = NULL; + v->s_size = ATOM_SIZE_UNSPECIFIED; v->handler = append_atom; v->atom = atom; @@ -2374,6 +2429,19 @@ static int compare_detached_head(struct ref_array_item *a, struct ref_array_item return 0; } +static int memcasecmp(const void *vs1, const void *vs2, size_t n) +{ + const char *s1 = vs1, *s2 = vs2; + const char *end = s1 + n; + + for (; s1 < end; s1++, s2++) { + int diff = tolower(*s1) - tolower(*s2); + if (diff) + return diff; + } + return 0; +} + static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, struct ref_array_item *b) { struct atom_value *va, *vb; @@ -2394,10 +2462,29 @@ static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, stru } else if (s->sort_flags & REF_SORTING_VERSION) { cmp = versioncmp(va->s, vb->s); } else if (cmp_type == FIELD_STR) { - int (*cmp_fn)(const char *, const char *); - cmp_fn = s->sort_flags & REF_SORTING_ICASE - ? strcasecmp : strcmp; - cmp = cmp_fn(va->s, vb->s); + if (va->s_size < 0 && vb->s_size < 0) { + int (*cmp_fn)(const char *, const char *); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? strcasecmp : strcmp; + cmp = cmp_fn(va->s, vb->s); + } else { + size_t a_size = va->s_size < 0 ? + strlen(va->s) : va->s_size; + size_t b_size = vb->s_size < 0 ? + strlen(vb->s) : vb->s_size; + int (*cmp_fn)(const void *, const void *, size_t); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? memcasecmp : memcmp; + + cmp = cmp_fn(va->s, vb->s, b_size > a_size ? + a_size : b_size); + if (!cmp) { + if (a_size > b_size) + cmp = 1; + else if (a_size < b_size) + cmp = -1; + } + } } else { if (va->value < vb->value) cmp = -1; @@ -2496,7 +2583,7 @@ int format_ref_array_item(struct ref_array_item *info, append_literal(cp, sp, &state); } if (format->need_color_reset_at_eol) { - struct atom_value resetv; + struct atom_value resetv = ATOM_VALUE_INIT; resetv.s = GIT_COLOR_RESET; if (append_atom(&resetv, &state, error_buf)) { pop_stack_element(&state.stack); diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 9e0214076b4..18554f62d94 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -130,6 +130,8 @@ test_atom head parent:short=10 '' test_atom head numparent 0 test_atom head object '' test_atom head type '' +test_atom head raw "$(git cat-file commit refs/heads/main) +" test_atom head '*objectname' '' test_atom head '*objecttype' '' test_atom head author 'A U Thor 1151968724 +0200' @@ -221,6 +223,15 @@ test_atom tag contents 'Tagging at 1151968727 ' test_atom tag HEAD ' ' +test_expect_success 'basic atom: refs/tags/testtag *raw' ' + git cat-file commit refs/tags/testtag^{} >expected && + git for-each-ref --format="%(*raw)" refs/tags/testtag >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'Check invalid atoms names are errors' ' test_must_fail git for-each-ref --format="%(INVALID)" refs/heads ' @@ -686,6 +697,15 @@ test_atom refs/tags/signed-empty contents:body '' test_atom refs/tags/signed-empty contents:signature "$sig" test_atom refs/tags/signed-empty contents "$sig" +test_expect_success GPG 'basic atom: refs/tags/signed-empty raw' ' + git cat-file tag refs/tags/signed-empty >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-empty >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-short subject 'subject line' test_atom refs/tags/signed-short subject:sanitize 'subject-line' test_atom refs/tags/signed-short contents:subject 'subject line' @@ -695,6 +715,15 @@ test_atom refs/tags/signed-short contents:signature "$sig" test_atom refs/tags/signed-short contents "subject line $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-short raw' ' + git cat-file tag refs/tags/signed-short >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-short >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-long subject 'subject line' test_atom refs/tags/signed-long subject:sanitize 'subject-line' test_atom refs/tags/signed-long contents:subject 'subject line' @@ -708,6 +737,15 @@ test_atom refs/tags/signed-long contents "subject line body contents $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-long raw' ' + git cat-file tag refs/tags/signed-long >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-long >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'set up refs pointing to tree and blob' ' git update-ref refs/mytrees/first refs/heads/main^{tree} && git update-ref refs/myblobs/first refs/heads/main:one @@ -720,6 +758,16 @@ test_atom refs/mytrees/first contents:body "" test_atom refs/mytrees/first contents:signature "" test_atom refs/mytrees/first contents "" +test_expect_success 'basic atom: refs/mytrees/first raw' ' + git cat-file tree refs/mytrees/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/mytrees/first >actual && + test_cmp expected actual && + git cat-file -s refs/mytrees/first >expected && + git for-each-ref --format="%(raw:size)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_atom refs/myblobs/first subject "" test_atom refs/myblobs/first contents:subject "" test_atom refs/myblobs/first body "" @@ -727,6 +775,174 @@ test_atom refs/myblobs/first contents:body "" test_atom refs/myblobs/first contents:signature "" test_atom refs/myblobs/first contents "" +test_expect_success 'basic atom: refs/myblobs/first raw' ' + git cat-file blob refs/myblobs/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/myblobs/first >actual && + test_cmp expected actual && + git cat-file -s refs/myblobs/first >expected && + git for-each-ref --format="%(raw:size)" refs/myblobs/first >actual && + test_cmp expected actual +' + +test_expect_success 'set up refs pointing to binary blob' ' + printf "a\0b\0c" >blob1 && + printf "a\0c\0b" >blob2 && + printf "\0a\0b\0c" >blob3 && + printf "abc" >blob4 && + printf "\0 \0 \0 " >blob5 && + printf "\0 \0a\0 " >blob6 && + printf " " >blob7 && + >blob8 && + obj=$(git hash-object -w blob1) && + git update-ref refs/myblobs/blob1 "$obj" && + obj=$(git hash-object -w blob2) && + git update-ref refs/myblobs/blob2 "$obj" && + obj=$(git hash-object -w blob3) && + git update-ref refs/myblobs/blob3 "$obj" && + obj=$(git hash-object -w blob4) && + git update-ref refs/myblobs/blob4 "$obj" && + obj=$(git hash-object -w blob5) && + git update-ref refs/myblobs/blob5 "$obj" && + obj=$(git hash-object -w blob6) && + git update-ref refs/myblobs/blob6 "$obj" && + obj=$(git hash-object -w blob7) && + git update-ref refs/myblobs/blob7 "$obj" && + obj=$(git hash-object -w blob8) && + git update-ref refs/myblobs/blob8 "$obj" +' + +test_expect_success 'Verify sorts with raw' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob3 + refs/myblobs/blob7 + refs/mytrees/first + refs/myblobs/first + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob4 + refs/heads/main + EOF + git for-each-ref --format="%(refname)" --sort=raw \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'Verify sorts with raw:size' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/first + refs/myblobs/blob7 + refs/heads/main + refs/myblobs/blob4 + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/mytrees/first + EOF + git for-each-ref --format="%(refname)" --sort=raw:size \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:equals)' ' + cat >expected <<-EOF && + not equals + not equals + not equals + not equals + not equals + not equals + refs/myblobs/blob4 + not equals + not equals + not equals + not equals + not equals + EOF + git for-each-ref --format="%(if:equals=abc)%(raw)%(then)%(refname)%(else)not equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:notequals)' ' + cat >expected <<-EOF && + refs/heads/ambiguous + refs/heads/main + refs/heads/newtag + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + equals + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob7 + refs/myblobs/blob8 + refs/myblobs/first + EOF + git for-each-ref --format="%(if:notequals=abc)%(raw)%(then)%(refname)%(else)equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'empty raw refs with %(if)' ' + cat >expected <<-EOF && + refs/myblobs/blob1 not empty + refs/myblobs/blob2 not empty + refs/myblobs/blob3 not empty + refs/myblobs/blob4 not empty + refs/myblobs/blob5 not empty + refs/myblobs/blob6 not empty + refs/myblobs/blob7 empty + refs/myblobs/blob8 empty + refs/myblobs/first not empty + EOF + git for-each-ref --format="%(refname) %(if)%(raw)%(then)not empty%(else)empty%(end)" \ + refs/myblobs/ >actual && + test_cmp expected actual +' + +test_expect_success '%(raw) with --python must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --python +' + +test_expect_success '%(raw) with --tcl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --tcl +' + +test_expect_success '%(raw) with --perl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --perl +' + +test_expect_success '%(raw) with --shell must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --shell +' + +test_expect_success '%(raw) with --shell and --sort=raw must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --sort=raw --shell +' + +test_expect_success '%(raw:size) with --shell' ' + git for-each-ref --format="%(raw:size)" | while read line + do + echo "'\''$line'\''" >>expect + done && + git for-each-ref --format="%(raw:size)" --shell >actual && + test_cmp expect actual +' + +test_expect_success 'for-each-ref --format compare with cat-file --batch' ' + git rev-parse refs/mytrees/first | git cat-file --batch >expected && + git for-each-ref --format="%(objectname) %(objecttype) %(objectsize) +%(raw)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_expect_success 'set up multiple-sort tags' ' for when in 100000 200000 do From patchwork Fri Aug 13 08:22:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BCFEC4320E for ; Fri, 13 Aug 2021 08:23:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 85DFF6056B for ; Fri, 13 Aug 2021 08:23:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238637AbhHMIXt (ORCPT ); Fri, 13 Aug 2021 04:23:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237439AbhHMIXl (ORCPT ); Fri, 13 Aug 2021 04:23:41 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3472EC0617AF for ; Fri, 13 Aug 2021 01:23:15 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id q6so618021wrv.6 for ; Fri, 13 Aug 2021 01:23:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=AOOm8J191P4j4KlKZ3cqk8UGe++6yL/hpw1DpiVJqyA=; b=I0asmJR32+EDkDdPkVARXhjk3Uf+sY2h7oXdQCEtuK5gRM/xvxHsrriNPmNEnAPsQO r3MR3reErJH2ZX9uGVZbJVkogjnITQ8/WFT1RMVA+bqD1DiPfNT7Dk/nnrQQVPo3e9cA x0Wxyr4xDlD2vlI2m6mHG2rAHO8oO1DublxQXUNS+r39OoW1B2Uoso59aYXqgANltoKZ hx0N5mg/2/8YMttdre0ZmR2XWmZPnaIixmS3dK7PqyzpJxlO5g1EvbXkNkeDVrmbXIEN rUL1waZaXGniqlIvomhoJ9EGVXe83KtMx6fV7cqKn6D3vmji4R5aOmNMkkUy3gmHbmYu i23Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=AOOm8J191P4j4KlKZ3cqk8UGe++6yL/hpw1DpiVJqyA=; b=dbBXQwbg3jAiGuItxkrBqg0HVABnxFt/qTuMTzrNLWyGS+hH7ptuk0bNNwLN0Y+DJu 89JiocxFTzJTrA4HMgG7VWo6hWMJkx45gZdEFkG3y5ZTA3209K1murEN2lYdhiVWEinU OzCN2fOgA/qe5Vm+uO/obwmvpBaGQE/dmIyV6QlQwjxvsynsYbzkJkghYyho3Ytkrw5x glF7c8HS1JE+RBmnAaaqFT8GzaRRI5HnAx506OYK1yD3U2+5QwSTzehl1j6xuVE//gYt eKg1XhDyoftT2tw0IfxL+uR4yCKK9X0OKC8mdXE7HU5HojvpItY/01TBIJ0hUdqcRHmC Av5Q== X-Gm-Message-State: AOAM531dBgoXavBVLSviF+X3TOMk6jxdkztX31a3M7p4yrGWYpPaYpPL GSc5b4t9ZdFVoujGrq2Ymar7SmX/D/o= X-Google-Smtp-Source: ABdhPJzm0LYx9tSFKTY82aY1EIvxtSXox1GuFGBY3DQHY219+7Ui9rxH7vrmGoz5gW6eedUvgQw+Xw== X-Received: by 2002:adf:a452:: with SMTP id e18mr1767104wra.244.1628842993870; Fri, 13 Aug 2021 01:23:13 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o24sm706406wmm.37.2021.08.13.01.23.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:13 -0700 (PDT) Message-Id: <452ea887e6123bc74b1a2f563ab0ea7c650867b9.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:46 +0000 Subject: [PATCH 03/27] [GSOC] ref-filter: --format=%(raw) support --perl MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because the perl language can handle binary data correctly, add the function perl_quote_buf_with_len(), which can specify the length of the data and prevent the data from being truncated at '\0' to help `--format="%(raw)"` support `--perl`. Reviewed-by: Jacob Keller Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 4 ++-- quote.c | 17 +++++++++++++++++ quote.h | 1 + ref-filter.c | 15 +++++++++++---- t/t6300-for-each-ref.sh | 19 +++++++++++++++++-- 5 files changed, 48 insertions(+), 8 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index cbb6f87d13f..6da899c6296 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -241,8 +241,8 @@ raw:size:: The raw data size of the object. Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, -`--perl` because such language may not support arbitrary binary data in their -string variable type. +because such language may not support arbitrary binary data in their string +variable type. The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/quote.c b/quote.c index 8a3a5e39eb1..26719d21d1e 100644 --- a/quote.c +++ b/quote.c @@ -471,6 +471,23 @@ void perl_quote_buf(struct strbuf *sb, const char *src) strbuf_addch(sb, sq); } +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len) +{ + const char sq = '\''; + const char bq = '\\'; + const char *c = src; + const char *end = src + len; + + strbuf_addch(sb, sq); + while (c != end) { + if (*c == sq || *c == bq) + strbuf_addch(sb, bq); + strbuf_addch(sb, *c); + c++; + } + strbuf_addch(sb, sq); +} + void python_quote_buf(struct strbuf *sb, const char *src) { const char sq = '\''; diff --git a/quote.h b/quote.h index 049d8dd0b3d..87ff458b06d 100644 --- a/quote.h +++ b/quote.h @@ -95,6 +95,7 @@ char *quote_path(const char *in, const char *prefix, struct strbuf *out, unsigne /* quoting as a string literal for other languages */ void perl_quote_buf(struct strbuf *sb, const char *src); +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len); void python_quote_buf(struct strbuf *sb, const char *src); void tcl_quote_buf(struct strbuf *sb, const char *src); void basic_regex_quote_buf(struct strbuf *sb, const char *src); diff --git a/ref-filter.c b/ref-filter.c index 1e660f60c4b..f2454d34ce8 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -747,7 +747,10 @@ static void quote_formatting(struct strbuf *s, const char *str, ssize_t len, int sq_quote_buf(s, str); break; case QUOTE_PERL: - perl_quote_buf(s, str); + if (len < 0) + perl_quote_buf(s, str); + else + perl_quote_buf_with_len(s, str, len); break; case QUOTE_PYTHON: python_quote_buf(s, str); @@ -1010,10 +1013,14 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && - used_atom[at].u.raw_data.option == RAW_BARE) + + if ((format->quote_style == QUOTE_PYTHON || + format->quote_style == QUOTE_SHELL || + format->quote_style == QUOTE_TCL) && + used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) die(_("--format=%.*s cannot be used with" - "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); + "--python, --shell, --tcl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 18554f62d94..3d15d0a5360 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -915,8 +915,23 @@ test_expect_success '%(raw) with --tcl must fail' ' test_must_fail git for-each-ref --format="%(raw)" --tcl ' -test_expect_success '%(raw) with --perl must fail' ' - test_must_fail git for-each-ref --format="%(raw)" --perl +test_expect_success '%(raw) with --perl' ' + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob1 --perl | perl >actual && + cmp blob1 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob3 --perl | perl >actual && + cmp blob3 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob8 --perl | perl >actual && + cmp blob8 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/first --perl | perl >actual && + cmp one actual && + git cat-file tree refs/mytrees/first > expected && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/mytrees/first --perl | perl >actual && + cmp expected actual ' test_expect_success '%(raw) with --shell must fail' ' From patchwork Fri Aug 13 08:22:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3C8EC432BE for ; Fri, 13 Aug 2021 08:23:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4A1E610E9 for ; Fri, 13 Aug 2021 08:23:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238656AbhHMIXu (ORCPT ); Fri, 13 Aug 2021 04:23:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238157AbhHMIXo (ORCPT ); Fri, 13 Aug 2021 04:23:44 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75206C0613A3 for ; Fri, 13 Aug 2021 01:23:16 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id x10so5839057wrt.8 for ; Fri, 13 Aug 2021 01:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=HS23BySheNzBE0Hf5RGnC8+XSlLScrPOju3gWc2sYqo=; b=GQpTOQleeZjLjt4ZHj2Er4l8EnvIgj+r1f4D07n6SJfQ5Kzo0kheXinpvzYdozDVaH T5qoeTFlrVZdvCAX5RfCii5YQGui5Zaq0FvqIYFaYo8/2f/cMGONBVtMRxy3SSqoBzId yAs9K3YuUrQgkBi+/kGlNHnJdrhF+X2g9yeHpSQccq/jlnu6ZOINpeNoQjJ6a2u5H4gS 2vWtTaHM56pfqzWzEqXx1tXZNl2YinpbBCloiqOJf30amtR/XqtjQUNq6eSlvGstsLK8 YR+uqYFGTBklBvwo9zv+4oo/0jC6sZ+STkaNNhP3YrWkPD594ZIIyJRFS8mKpa05P9sF KbqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=HS23BySheNzBE0Hf5RGnC8+XSlLScrPOju3gWc2sYqo=; b=SjNn5UDPwB/bOS1YfAA+UGkeUBFkxFy5AlR2dyVDqwqlKFVvIRasgyS6GUH0EMcFFh BeCXP/NlbRCWJyTL4KZJiyWNx423J3+qLYc04p2XUmODEwof8wcF/LY36qoeVD5ZNnnT O6anHclFKrdI1sRp+gFQXMD1MC7VgI/FQEteZ0ZHkc8sfrydUZg0Cu/UvK25yHrVPbkI az9r+iDO3sgTLYJqpwHa0zBvut8MxBFmvh6hI4UnbGL1kX3vmbYWbOR1NFwRmEq7djp+ Z/XcVU+THqXSBIzdtVqD5aMnmIGCP3TMOPFi543LbLWSK2Wea74urgpZngJHi17Nu/aJ IT+A== X-Gm-Message-State: AOAM532hpCiZq8CGSjV1z43kCV90aleJKr3mj6ynBhtnTJ/ZhKiQioUy mk2RM3IbesVdS0oDKpt3AogHvG38gMA= X-Google-Smtp-Source: ABdhPJyGvU9gzM8O80mPPXaNa634Bzd14vrdaL+tD+34GhKR7tCYV1NQcVeSbLyGgH2WDlPcHEAwHw== X-Received: by 2002:adf:e10c:: with SMTP id t12mr1661052wrz.36.1628842994479; Fri, 13 Aug 2021 01:23:14 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q11sm714554wmc.41.2021.08.13.01.23.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:14 -0700 (PDT) Message-Id: <0dc3cf095934e81071189f9c116a98b457f26335.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:47 +0000 Subject: [PATCH 04/27] [GSOC] ref-filter: use non-const ref_format in *_atom_parser() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use non-const ref_format in *_atom_parser(), which can help us modify the members of ref_format in *_atom_parser(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/tag.c | 2 +- ref-filter.c | 44 ++++++++++++++++++++++---------------------- ref-filter.h | 4 ++-- 3 files changed, 25 insertions(+), 25 deletions(-) diff --git a/builtin/tag.c b/builtin/tag.c index 82fcfc09824..452558ec957 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -146,7 +146,7 @@ static int verify_tag(const char *name, const char *ref, const struct object_id *oid, void *cb_data) { int flags; - const struct ref_format *format = cb_data; + struct ref_format *format = cb_data; flags = GPG_VERIFY_VERBOSE; if (format->format) diff --git a/ref-filter.c b/ref-filter.c index f2454d34ce8..3c101500ed1 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -227,7 +227,7 @@ static int strbuf_addf_ret(struct strbuf *sb, int ret, const char *fmt, ...) return ret; } -static int color_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int color_atom_parser(struct ref_format *format, struct used_atom *atom, const char *color_value, struct strbuf *err) { if (!color_value) @@ -265,7 +265,7 @@ static int refname_atom_parser_internal(struct refname_atom *atom, const char *a return 0; } -static int remote_ref_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int remote_ref_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct string_list params = STRING_LIST_INIT_DUP; @@ -312,7 +312,7 @@ static int remote_ref_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objecttype_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objecttype_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -324,7 +324,7 @@ static int objecttype_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objectsize_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objectsize_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -344,7 +344,7 @@ static int objectsize_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int deltabase_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int deltabase_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -356,7 +356,7 @@ static int deltabase_atom_parser(const struct ref_format *format, struct used_at return 0; } -static int body_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int body_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -365,7 +365,7 @@ static int body_atom_parser(const struct ref_format *format, struct used_atom *a return 0; } -static int subject_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int subject_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -377,7 +377,7 @@ static int subject_atom_parser(const struct ref_format *format, struct used_atom return 0; } -static int trailers_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int trailers_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { atom->u.contents.trailer_opts.no_divider = 1; @@ -403,7 +403,7 @@ static int trailers_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int contents_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int contents_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -431,7 +431,7 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int raw_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -443,7 +443,7 @@ static int raw_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int oid_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -462,7 +462,7 @@ static int oid_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int person_email_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int person_email_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -476,7 +476,7 @@ static int person_email_atom_parser(const struct ref_format *format, struct used return 0; } -static int refname_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int refname_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { return refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err); @@ -493,7 +493,7 @@ static align_type parse_align_position(const char *s) return -1; } -static int align_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int align_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct align *align = &atom->u.align; @@ -545,7 +545,7 @@ static int align_atom_parser(const struct ref_format *format, struct used_atom * return 0; } -static int if_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int if_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -560,7 +560,7 @@ static int if_atom_parser(const struct ref_format *format, struct used_atom *ato return 0; } -static int head_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { atom->u.head = resolve_refdup("HEAD", RESOLVE_REF_READING, NULL, NULL); @@ -571,7 +571,7 @@ static struct { const char *name; info_source source; cmp_type cmp_type; - int (*parser)(const struct ref_format *format, struct used_atom *atom, + int (*parser)(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err); } valid_atom[] = { [ATOM_REFNAME] = { "refname", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -654,7 +654,7 @@ struct atom_value { /* * Used to parse format string and sort specifiers */ -static int parse_ref_filter_atom(const struct ref_format *format, +static int parse_ref_filter_atom(struct ref_format *format, const char *atom, const char *ep, struct strbuf *err) { @@ -2561,9 +2561,9 @@ static void append_literal(const char *cp, const char *ep, struct ref_formatting } int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, - struct strbuf *final_buf, - struct strbuf *error_buf) + struct ref_format *format, + struct strbuf *final_buf, + struct strbuf *error_buf) { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; @@ -2607,7 +2607,7 @@ int format_ref_array_item(struct ref_array_item *info, } void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format) + struct ref_format *format) { struct ref_array_item *ref_item; struct strbuf output = STRBUF_INIT; diff --git a/ref-filter.h b/ref-filter.h index baf72a71896..74fb423fc89 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -116,7 +116,7 @@ void ref_array_sort(struct ref_sorting *sort, struct ref_array *array); void ref_sorting_set_sort_flags_all(struct ref_sorting *sorting, unsigned int mask, int on); /* Based on the given format and quote_style, fill the strbuf */ int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, + struct ref_format *format, struct strbuf *final_buf, struct strbuf *error_buf); /* Parse a single sort specifier and add it to the list */ @@ -137,7 +137,7 @@ void setup_ref_filter_porcelain_msg(void); * name must be a fully qualified refname. */ void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format); + struct ref_format *format); /* * Push a single ref onto the array; this can be used to construct your own From patchwork Fri Aug 13 08:22:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5B38C4320E for ; Fri, 13 Aug 2021 08:23:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A77456056B for ; Fri, 13 Aug 2021 08:23:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238733AbhHMIXw (ORCPT ); Fri, 13 Aug 2021 04:23:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238278AbhHMIXo (ORCPT ); Fri, 13 Aug 2021 04:23:44 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 836CAC0613A4 for ; Fri, 13 Aug 2021 01:23:16 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id l18so12141043wrv.5 for ; Fri, 13 Aug 2021 01:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=kwde3c2810kTFxRkLVk+Yp/yaU2qGCkNzG0TW5ylhLI=; b=TuNNkRQTXN2bm4XLhd21D+s4RZuzLVL/p1Y1XHafLWdq4yztjfEE195le5+NWIckYo aMkNtRS5IMwwlHpcIRsZq4lAN0nHKC7EypcrBbsac3dJ2CJWtEwx8v7wcem6YDTLhrhP CfUGBXK6DGydyIdpqs8Bo25nCHIjBVIH1ZTA6GBdleLm64Gkg0/PgT4Cs7XHyxJW+rO2 NwMC4fbQ1slVmKP7SqiVcVedq3YroVP7lY9gR4JLEV7Q2rPv/e4xwAxdoyLx3ULK+3mL 30YpD4dLq05TXIfmyEWh1yapTfJDpnnhqu/tY0g7DHanGAnyCueaaxK+/dxATRyYp4t4 HkDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=kwde3c2810kTFxRkLVk+Yp/yaU2qGCkNzG0TW5ylhLI=; b=OAnmf8TlsCqk6FbvSPTGO7BwVio1cNtXs4d5yrIIDgz1Ta+sixnPT+NxzWaZ77waVz U6h1dtJAXGvbvn8pBSkMlI0j6mC1BN0owI1leRg4c4th3VepsDuUGfiib/psmU1DZlgj R1N+NUh0lr4SXGROBu0aTd/+xMR/nnbPsWjiTu0tPxqNKC5kYqpq6/cFA3OVPDMvUNI1 9fkIFXYDOVId/crcWq6LqQGOT3j3fpGOkK0bEequ03XzbfHe6AabMksxVTTluocULO/7 +yNANZNcklOfGLgSh13v4wMDBkBJ8o1G3PtefQEXq4NfyYl1vALXOlbgU0OVfhy7pUVw TE+w== X-Gm-Message-State: AOAM533Ld0z6a5mSJPx6V5pEO5Xzh4UZPchWfP6HlkaU8mJbUMoAMwi2 3a/7exRT1h9r+otClw7O85QszStsZps= X-Google-Smtp-Source: ABdhPJxYSIdgHor+Xns/yl2B+3uK+XEEb6pzLyqahso6vVtpiY4aCHrbciAyMmKRz+pMMAt6Eygtvg== X-Received: by 2002:a5d:4846:: with SMTP id n6mr1637779wrs.91.1628842995193; Fri, 13 Aug 2021 01:23:15 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y13sm691575wmj.27.2021.08.13.01.23.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:14 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:48 +0000 Subject: [PATCH 05/27] [GSOC] ref-filter: add %(rest) atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu %(rest) is a atom used for cat-file batch mode, which can split the input lines at the first whitespace boundary, all characters before that whitespace are considered to be the object name; characters after that first run of whitespace (i.e., the "rest" of the line) are output in place of the %(rest) atom. In order to let "cat-file --batch=%(rest)" use the ref-filter interface, add %(rest) atom for ref-filter. Introduce the reject_atom() to reject the atom %(rest) for "git for-each-ref", "git branch", "git tag" and "git verify-tag". Reviewed-by: Jacob Keller Suggected-by: Jacob Keller Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 25 +++++++++++++++++++++++++ ref-filter.h | 5 ++++- t/t3203-branch-output.sh | 4 ++++ t/t6300-for-each-ref.sh | 4 ++++ t/t7004-tag.sh | 4 ++++ t/t7030-verify-tag.sh | 4 ++++ 6 files changed, 45 insertions(+), 1 deletion(-) diff --git a/ref-filter.c b/ref-filter.c index 3c101500ed1..93ce2a6ef2e 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -157,6 +157,7 @@ enum atom_type { ATOM_IF, ATOM_THEN, ATOM_ELSE, + ATOM_REST, }; /* @@ -560,6 +561,15 @@ static int if_atom_parser(struct ref_format *format, struct used_atom *atom, return 0; } +static int rest_atom_parser(struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (arg) + return strbuf_addf_ret(err, -1, _("%%(rest) does not take arguments")); + format->use_rest = 1; + return 0; +} + static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { @@ -616,6 +626,7 @@ static struct { [ATOM_IF] = { "if", SOURCE_NONE, FIELD_STR, if_atom_parser }, [ATOM_THEN] = { "then", SOURCE_NONE }, [ATOM_ELSE] = { "else", SOURCE_NONE }, + [ATOM_REST] = { "rest", SOURCE_NONE, FIELD_STR, rest_atom_parser }, /* * Please update $__git_ref_fieldlist in git-completion.bash * when you add new atoms @@ -993,6 +1004,11 @@ static const char *find_next(const char *cp) return NULL; } +static int reject_atom(enum atom_type atom_type) +{ + return atom_type == ATOM_REST; +} + /* * Make sure the format string is well formed, and parse out * the used atoms. @@ -1013,6 +1029,8 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (reject_atom(used_atom[at].atom_type)) + die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || @@ -1932,6 +1950,12 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->handler = else_atom_handler; v->s = xstrdup(""); continue; + } else if (atom_type == ATOM_REST) { + if (ref->rest) + v->s = xstrdup(ref->rest); + else + v->s = xstrdup(""); + continue; } else continue; @@ -2149,6 +2173,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); + ref->rest = NULL; return ref; } diff --git a/ref-filter.h b/ref-filter.h index 74fb423fc89..c15dee8d6b9 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -38,6 +38,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; + const char *rest; int flag; unsigned int kind; const char *symref; @@ -76,14 +77,16 @@ struct ref_format { * verify_ref_format() afterwards to finalize. */ const char *format; + const char *rest; int quote_style; + int use_rest; int use_color; /* Internal state to ref-filter */ int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { NULL, 0, -1 } +#define REF_FORMAT_INIT { .use_color = -1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ diff --git a/t/t3203-branch-output.sh b/t/t3203-branch-output.sh index 5325b9f67a0..6e94c6db7b5 100755 --- a/t/t3203-branch-output.sh +++ b/t/t3203-branch-output.sh @@ -340,6 +340,10 @@ test_expect_success 'git branch --format option' ' test_cmp expect actual ' +test_expect_success 'git branch with --format=%(rest) must fail' ' + test_must_fail git branch --format="%(rest)" >actual +' + test_expect_success 'worktree colors correct' ' cat >expect <<-EOF && * (HEAD detached from fromtag) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 3d15d0a5360..0d2e062f791 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -1211,6 +1211,10 @@ test_expect_success 'basic atom: head contents:trailers' ' test_cmp expect actual.clean ' +test_expect_success 'basic atom: rest must fail' ' + test_must_fail git for-each-ref --format="%(rest)" refs/heads/main +' + test_expect_success 'trailer parsing not fooled by --- line' ' git commit --allow-empty -F - <<-\EOF && this is the subject diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh index 2f72c5c6883..082be85dffc 100755 --- a/t/t7004-tag.sh +++ b/t/t7004-tag.sh @@ -1998,6 +1998,10 @@ test_expect_success '--format should list tags as per format given' ' test_cmp expect actual ' +test_expect_success 'git tag -l with --format="%(rest)" must fail' ' + test_must_fail git tag -l --format="%(rest)" "v1*" +' + test_expect_success "set up color tests" ' echo "v1.0" >expect.color && echo "v1.0" >expect.bare && diff --git a/t/t7030-verify-tag.sh b/t/t7030-verify-tag.sh index 3cefde9602b..10faa645157 100755 --- a/t/t7030-verify-tag.sh +++ b/t/t7030-verify-tag.sh @@ -194,6 +194,10 @@ test_expect_success GPG 'verifying tag with --format' ' test_cmp expect actual ' +test_expect_success GPG 'verifying tag with --format="%(rest)" must fail' ' + test_must_fail git verify-tag --format="%(rest)" "fourth-signed" +' + test_expect_success GPG 'verifying a forged tag with --format should fail silently' ' test_must_fail git verify-tag --format="tagname : %(tag)" $(cat forged1.tag) >actual-forged && test_must_be_empty actual-forged From patchwork Fri Aug 13 08:22:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4058AC432BE for ; Fri, 13 Aug 2021 08:23:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 29646610CC for ; Fri, 13 Aug 2021 08:23:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238748AbhHMIXx (ORCPT ); Fri, 13 Aug 2021 04:23:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238302AbhHMIXo (ORCPT ); Fri, 13 Aug 2021 04:23:44 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EE39C0612E7 for ; Fri, 13 Aug 2021 01:23:17 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id z9so12049552wrh.10 for ; Fri, 13 Aug 2021 01:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=bRGi4pz2Xh9luD1c5Eb25LfrkRp+oBcvgyF4Y4GisV8=; b=cFx+Df30eL1G4j5AZECJQAW3VfVBDx6PSQi97YjhbykMx3h41qcrVABG0MNMUUQvWq BfmEpGMnbULrmvUX+mY2ifmMaeVdmMrwFqT/oMpOyjZrv23e25moYFYIj61JZE7OZWSv N4AqZXh8yPV0uCrS9nfEUm8chso7GL0lYCsp+mXpmy6m4qEA74ci8wE6lE54Pgi4OBdA Y0dBY56FMbZcfgB2Efbxk4KCNp9ApYDYw9V+bzeAATsULXE6pTCgCOowDNyc98EDcepu qkLwZCbp+csiM/Ch8PEkFbQHI58QyqGIvfnlROw3wm9P04UfkV1MfqXitjiIor7ixGmS gVFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=bRGi4pz2Xh9luD1c5Eb25LfrkRp+oBcvgyF4Y4GisV8=; b=mUzTgzP7bcb7OHvHBwI9+PzD/VCBTda7CO0zKeVSYasX1FlgKu+r3wf7LejHTqZJyg IhAsaIKz+TgPDWhJAU/t9gXc2oPrLU9DfXQC4BmOKJCGJyAz+ZWtY1o1XnTaIiTL62Xr hH1CPfHZBXAIxFU4SSTr1Xy0/U8wN9fn1n/wTvp5jqLer+EPWzIQQKQmoAQJlYY/tSyy WeXh+opG7L30/3ifceTF30Sh0dpwMigXKxvM8cMom2Nr1Ydd2l0KWCaJS56IdZYv2sBR a0s2mImiZAKz1ZDmMFIJeEZV4G7RkyW0kLq94XThe9xIHOiKM9qey8dChtJWM6Es8vrO M/CQ== X-Gm-Message-State: AOAM5337SeRWDyHO6YfFn2bsCsS4qkVSwzJSpJooBQqFcQ31pxF9Qaca dp5vKGD4AhVN/i8O6tBvp4Pk34kYbKs= X-Google-Smtp-Source: ABdhPJyriD1eBFOcIES5vvub87ICTXuFWInocrgGOL3QHHnWawj8nGSZBZQUfU+lx9WvimTIG2aQXQ== X-Received: by 2002:adf:ea09:: with SMTP id q9mr1667930wrm.64.1628842995835; Fri, 13 Aug 2021 01:23:15 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u23sm702808wmc.24.2021.08.13.01.23.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:15 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:49 +0000 Subject: [PATCH 06/27] [GSOC] ref-filter: pass get_object() return value to their callers MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because in the refactor of `git cat-file --batch` later, oid_object_info_extended() in get_object() will be used to obtain the info of an object with it's oid. When the object cannot be obtained in the git repository, `cat-file --batch` expects to output " missing" and continue the next oid query instead of letting Git exit. In other error conditions, Git should exit normally. So we can achieve this function by passing the return value of get_object(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- ref-filter.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 93ce2a6ef2e..2fd3b69f6d6 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1825,6 +1825,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) { struct object *obj; int i; + int ret; struct object_info empty = OBJECT_INFO_INIT; CALLOC_ARRAY(ref->value, used_atom_cnt); @@ -1981,8 +1982,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) oi.oid = ref->objectname; - if (get_object(ref, 0, &obj, &oi, err)) - return -1; + ret = get_object(ref, 0, &obj, &oi, err); + if (ret) + return ret; /* * If there is no atom that wants to know about tagged @@ -2014,8 +2016,10 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom, struct atom_value **v, struct strbuf *err) { if (!ref->value) { - if (populate_value(ref, err)) - return -1; + int ret = populate_value(ref, err); + + if (ret) + return ret; fill_missing_values(ref->value); } *v = &ref->value[atom]; @@ -2592,6 +2596,7 @@ int format_ref_array_item(struct ref_array_item *info, { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; + int ret; state.quote_style = format->quote_style; push_stack_element(&state.stack); @@ -2604,10 +2609,10 @@ int format_ref_array_item(struct ref_array_item *info, if (cp < sp) append_literal(cp, sp, &state); pos = parse_ref_filter_atom(format, sp + 2, ep, error_buf); - if (pos < 0 || get_ref_atom_value(info, pos, &atomv, error_buf) || + if (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) || atomv->handler(atomv, &state, error_buf)) { pop_stack_element(&state.stack); - return -1; + return ret ? ret : -1; } } if (*cp) { From patchwork Fri Aug 13 08:22:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 469A9C4338F for ; Fri, 13 Aug 2021 08:23:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2F3BE610CC for ; Fri, 13 Aug 2021 08:23:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238717AbhHMIXv (ORCPT ); Fri, 13 Aug 2021 04:23:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238273AbhHMIXo (ORCPT ); Fri, 13 Aug 2021 04:23:44 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4AFDC061292 for ; Fri, 13 Aug 2021 01:23:17 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id q6so618174wrv.6 for ; Fri, 13 Aug 2021 01:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Y20aWlRrxOJLaEfYStQAZHABxNh2sm7uigcSyzncrgA=; b=P8bTZ4Tnk1853PbV/I1MZrlpRbZWki74fkyV51udp12dKnu1HIXsC8WxROusTKawLj QguPp5wLGZZsuK0Cr1nqai324bQQfj+Yqqu9bXHjlA+8jeh+rNvBIpjUTxDADxJZ0iYl JdrBVBW7m1trS4vGAnzyEolmzfDLzZwKXtDDSf+o65QY3/k5jquD0thjRDWD94CPn2Du Frgcia5QQ5ujCKuTnNG7f38BAy2DtIutrnHszLaI5hqqWL+6/4avngl10QOFs4sjWcsQ TczWmsYevuuf+azAnMtk/xjog0u1jgmuybeXkmOdvwNf3Dd5gPOVXMWAaZg03RFY5ZRW h57A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Y20aWlRrxOJLaEfYStQAZHABxNh2sm7uigcSyzncrgA=; b=X1SvGA4fDMD5Gw4l25HsrZqW5DM2QIXx+jW+Kwe6BSwMCb14G3qhvHYQwKQLweUThX antmRJK/gTVODqidV7ltxV7i4FGhY1H6SLesdKPdI09wfyY2CUdlhyfLTJHjG+kj9Gza eBq7pCwhRCVRMFz21Ray3ev+4U6Z1sEoT7R/JNEWl2S8ka1LKUbpWCBBsSI5JKyEcZOw lkFLLDoE6pHiZ6MwDJ/JAFFqcmM574gPk6DLIapjV+013nE6jslIZhn9SxX3VOyabRsg nUjsnA8CnuD7ULemnvMmG9mGcib6Hd9ZmP4QaqJ7ojZXau9QKZ5JDVeIV9FbWwJUUTPr RX4w== X-Gm-Message-State: AOAM532fcm7B3+nBl97hIco200l3NFu8Lju2xvBPiSQZFkegIJ0ppUvi uZBflccr4h4WRLZSF4iy+XocMVaOLi4= X-Google-Smtp-Source: ABdhPJwIUj4+o9SxO8I/48Z8TBG6KXRyYcGLKglKP97rQ5ZXQJ3gvdvlUoLXE6lqReaDRWpUzkaY2g== X-Received: by 2002:a5d:4ed2:: with SMTP id s18mr1716896wrv.72.1628842996408; Fri, 13 Aug 2021 01:23:16 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z7sm711927wmi.4.2021.08.13.01.23.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:16 -0700 (PDT) Message-Id: <9a6e39c4c9994362212551ec99bb2291b27b78d9.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:50 +0000 Subject: [PATCH 07/27] [GSOC] ref-filter: introduce free_ref_array_item_value() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu When we use ref_array_item which is not dynamically allocated and want to free the space of its member "value" after the end of use, free_array_item() does not meet our needs, because it tries to free ref_array_item itself and its member "symref". Introduce free_ref_array_item_value() for freeing ref_array_item value. It will be called internally by free_array_item(), and it will help `cat-file --batch` free ref_array_item's value memory later. Helped-by: Junio C Hamano Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 ++++++++--- ref-filter.h | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 2fd3b69f6d6..7df7f06002b 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2300,16 +2300,21 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, return 0; } -/* Free memory allocated for a ref_array_item */ -static void free_array_item(struct ref_array_item *item) +void free_ref_array_item_value(struct ref_array_item *item) { - free((char *)item->symref); if (item->value) { int i; for (i = 0; i < used_atom_cnt; i++) free((char *)item->value[i].s); free(item->value); } +} + +/* Free memory allocated for a ref_array_item */ +static void free_array_item(struct ref_array_item *item) +{ + free((char *)item->symref); + free_ref_array_item_value(item); free(item); } diff --git a/ref-filter.h b/ref-filter.h index c15dee8d6b9..44e6dc05ac2 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -111,6 +111,8 @@ struct ref_format { int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type); /* Clear all memory allocated to ref_array */ void ref_array_clear(struct ref_array *array); +/* Free ref_array_item's value */ +void free_ref_array_item_value(struct ref_array_item *item); /* Used to verify if the given format is correct and to parse out the used atoms */ int verify_ref_format(struct ref_format *format); /* Sort the given ref_array as per the ref_sorting provided */ From patchwork Fri Aug 13 08:22:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FA0FC43214 for ; Fri, 13 Aug 2021 08:23:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B883610CC for ; Fri, 13 Aug 2021 08:23:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238812AbhHMIXy (ORCPT ); Fri, 13 Aug 2021 04:23:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238433AbhHMIXp (ORCPT ); Fri, 13 Aug 2021 04:23:45 -0400 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC813C061756 for ; Fri, 13 Aug 2021 01:23:18 -0700 (PDT) Received: by mail-wm1-x332.google.com with SMTP id c129-20020a1c35870000b02902e6b6135279so4912690wma.0 for ; Fri, 13 Aug 2021 01:23:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=K90EWgVORZ1FqckqjiYOLU/YsFumUbUlqpFTHZKv7/8=; b=A042kkoz+KfiLqHyIfJQYrkd90P6Rxn3ApSYz6IGvNXncZWm3EzgRIBbJmzEvs6nnp 9csqdxI/T38Ue6QozmTV7Qh45enxKOyp/zfcpYZVI+h8s+rv8w8/wNCxZ+4DJORsrCM6 NB2lu2pKGlgHARkBu91fnAPw+S8CaoHmJOo9EyrLUniOrtwvf03OCgn2K0uqQ55l/O2i g8SPkLNtA03zFS4VHb/jRAfBqYmHjLUQut/7ARmGdXP6Otby7GdzD1Yj7tLVApIXVF4O yeX0KH8oFBC9z8xS30df0h9xCUfNvn4hjwrtlE3lK1bf+TPglbzBQgYSTSv53qcFzI8C HtFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=K90EWgVORZ1FqckqjiYOLU/YsFumUbUlqpFTHZKv7/8=; b=rHBMHUd45ZJ/eUFdcDAzxhFF+IwQWZ/9Q11w5fyYUBLNaLWry2J9yc+QDzSDJDEa5i n7ZZH+ufMR5MijZpjwGsz51sbTNU7IeN7NcN5eEQVWu3SSZ3Ui5gGF5F2shzzSNYAszH dlXuOspBZx+VH0WQsEhIiouTQUkYOcL8+ZIxuRbTnpBSBP0E02V6tX+o7Ym3WWuUKN31 Cw56V9mOyikIifScL/FnoZrbxlAqbDBs+E8dfdAClJiPgtGtT8uYv3XAkWJCdz5mY1mA zuBOFKwf1dowWeodbvQCIqGIMZekar8aUQ1Ru5nNWqsgFjIiH0t1TTfSMtEbD5nTN6Jb ZIuw== X-Gm-Message-State: AOAM532RM4Z29YsC/aTT7NN1z+z8cudDv9L6hUKu+eb/nWcq7SXhxq3x NTeVKA9Gl3nk6KzN1glPOq3+3dXONOc= X-Google-Smtp-Source: ABdhPJz4mNQFCU382h0ZF1VbcZunDfAeBDXcY6e9uInxRZApG/J5XtoUoVdWWeaUxrSpAjUXGkU2JQ== X-Received: by 2002:a05:600c:4a29:: with SMTP id c41mr1422225wmp.86.1628842996985; Fri, 13 Aug 2021 01:23:16 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b18sm822136wrr.89.2021.08.13.01.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:16 -0700 (PDT) Message-Id: <305ef165064a9c39a53b8ad464bed27db5def355.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:51 +0000 Subject: [PATCH 08/27] [GSOC] ref-filter: add cat_file_mode to ref_format MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add `cat_file_mode` member to struct `ref_format`. When `cat-file --batch` use ref-filter logic later, it can help us reject atoms in verify_ref_format() which cat-file cannot use, e.g. `%(refname)`, `%(push)`, `%(upstream)"... Add batch_test_atom() to t/t1006-cat-file.sh and add check for cat-file --batch, this can help us clearly show which atoms cat-file accepts and which atoms it rejects. Helped-by: Eric Sunshine Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Christian Couder Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 23 ++++- ref-filter.h | 1 + t/t1006-cat-file.sh | 226 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 246 insertions(+), 4 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 7df7f06002b..e50acfc7378 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1004,9 +1004,24 @@ static const char *find_next(const char *cp) return NULL; } -static int reject_atom(enum atom_type atom_type) -{ - return atom_type == ATOM_REST; +static int reject_atom(int cat_file_mode, enum atom_type atom_type) +{ + if (!cat_file_mode) + return atom_type == ATOM_REST; + + /* cat_file_mode */ + switch (atom_type) { + case ATOM_FLAG: + case ATOM_HEAD: + case ATOM_PUSH: + case ATOM_REFNAME: + case ATOM_SYMREF: + case ATOM_UPSTREAM: + case ATOM_WORKTREEPATH: + return 1; + default: + return 0; + } } /* @@ -1029,7 +1044,7 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - if (reject_atom(used_atom[at].atom_type)) + if (reject_atom(format->cat_file_mode, used_atom[at].atom_type)) die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); if ((format->quote_style == QUOTE_PYTHON || diff --git a/ref-filter.h b/ref-filter.h index 44e6dc05ac2..053980a6a42 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -78,6 +78,7 @@ struct ref_format { */ const char *format; const char *rest; + int cat_file_mode; int quote_style; int use_rest; int use_color; diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 18b3779ccb6..95d760652eb 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -608,4 +608,230 @@ test_expect_success 'cat-file --batch="batman" with --batch-all-objects will wor cmp expect actual ' +. "$TEST_DIRECTORY"/lib-gpg.sh +. "$TEST_DIRECTORY"/lib-terminal.sh + +test_expect_success 'cat-file --batch|--batch-check setup' ' + echo 1>blob1 && + printf "a\0b\0\c" >blob2 && + git add blob1 blob2 && + git commit -m "Commit Message" && + git branch -M main && + git tag -a -m "v0.0.0" testtag && + git update-ref refs/myblobs/blob1 HEAD:blob1 && + git update-ref refs/myblobs/blob2 HEAD:blob2 && + git update-ref refs/mytrees/tree1 HEAD^{tree} +' + +batch_test_atom() { + if test "$3" = "fail" + then + test_expect_${4:-success} $PREREQ "basic atom: $1 $2 must fail" " + test_must_fail git cat-file --batch-check='$2' >bad <<-EOF + $1 + EOF + " + else + test_expect_${4:-success} $PREREQ "basic atom: $1 $2" " + git for-each-ref --format='$2' $1 >expected && + git cat-file --batch-check='$2' >actual <<-EOF && + $1 + EOF + sanitize_pgp actual.clean && + cmp expected actual.clean + " + fi +} + +batch_test_atom refs/heads/main '%(refname)' fail +batch_test_atom refs/heads/main '%(refname:)' fail +batch_test_atom refs/heads/main '%(refname:short)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:strip=1)' fail +batch_test_atom refs/heads/main '%(refname:strip=2)' fail +batch_test_atom refs/heads/main '%(refname:strip=-1)' fail +batch_test_atom refs/heads/main '%(refname:strip=-2)' fail +batch_test_atom refs/heads/main '%(upstream)' fail +batch_test_atom refs/heads/main '%(upstream:short)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=-2)' fail +batch_test_atom refs/heads/main '%(push)' fail +batch_test_atom refs/heads/main '%(push:short)' fail +batch_test_atom refs/heads/main '%(push:lstrip=1)' fail +batch_test_atom refs/heads/main '%(push:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:strip=1)' fail +batch_test_atom refs/heads/main '%(push:strip=-1)' fail +batch_test_atom refs/heads/main '%(objecttype)' +batch_test_atom refs/heads/main '%(objectsize)' +batch_test_atom refs/heads/main '%(objectsize:disk)' +batch_test_atom refs/heads/main '%(deltabase)' +batch_test_atom refs/heads/main '%(objectname)' +batch_test_atom refs/heads/main '%(objectname:short)' fail +batch_test_atom refs/heads/main '%(objectname:short=1)' fail +batch_test_atom refs/heads/main '%(objectname:short=10)' fail +batch_test_atom refs/heads/main '%(tree)' fail +batch_test_atom refs/heads/main '%(tree:short)' fail +batch_test_atom refs/heads/main '%(tree:short=1)' fail +batch_test_atom refs/heads/main '%(tree:short=10)' fail +batch_test_atom refs/heads/main '%(parent)' fail +batch_test_atom refs/heads/main '%(parent:short)' fail +batch_test_atom refs/heads/main '%(parent:short=1)' fail +batch_test_atom refs/heads/main '%(parent:short=10)' fail +batch_test_atom refs/heads/main '%(numparent)' fail +batch_test_atom refs/heads/main '%(object)' fail +batch_test_atom refs/heads/main '%(type)' fail +batch_test_atom refs/heads/main '%(raw)' fail +batch_test_atom refs/heads/main '%(*objectname)' fail +batch_test_atom refs/heads/main '%(*objecttype)' fail +batch_test_atom refs/heads/main '%(author)' fail +batch_test_atom refs/heads/main '%(authorname)' fail +batch_test_atom refs/heads/main '%(authoremail)' fail +batch_test_atom refs/heads/main '%(authoremail:trim)' fail +batch_test_atom refs/heads/main '%(authoremail:localpart)' fail +batch_test_atom refs/heads/main '%(authordate)' fail +batch_test_atom refs/heads/main '%(committer)' fail +batch_test_atom refs/heads/main '%(committername)' fail +batch_test_atom refs/heads/main '%(committeremail)' fail +batch_test_atom refs/heads/main '%(committeremail:trim)' fail +batch_test_atom refs/heads/main '%(committeremail:localpart)' fail +batch_test_atom refs/heads/main '%(committerdate)' fail +batch_test_atom refs/heads/main '%(tag)' fail +batch_test_atom refs/heads/main '%(tagger)' fail +batch_test_atom refs/heads/main '%(taggername)' fail +batch_test_atom refs/heads/main '%(taggeremail)' fail +batch_test_atom refs/heads/main '%(taggeremail:trim)' fail +batch_test_atom refs/heads/main '%(taggeremail:localpart)' fail +batch_test_atom refs/heads/main '%(taggerdate)' fail +batch_test_atom refs/heads/main '%(creator)' fail +batch_test_atom refs/heads/main '%(creatordate)' fail +batch_test_atom refs/heads/main '%(subject)' fail +batch_test_atom refs/heads/main '%(subject:sanitize)' fail +batch_test_atom refs/heads/main '%(contents:subject)' fail +batch_test_atom refs/heads/main '%(body)' fail +batch_test_atom refs/heads/main '%(contents:body)' fail +batch_test_atom refs/heads/main '%(contents:signature)' fail +batch_test_atom refs/heads/main '%(contents)' fail +batch_test_atom refs/heads/main '%(HEAD)' fail +batch_test_atom refs/heads/main '%(upstream:track)' fail +batch_test_atom refs/heads/main '%(upstream:trackshort)' fail +batch_test_atom refs/heads/main '%(upstream:track,nobracket)' fail +batch_test_atom refs/heads/main '%(upstream:nobracket,track)' fail +batch_test_atom refs/heads/main '%(push:track)' fail +batch_test_atom refs/heads/main '%(push:trackshort)' fail +batch_test_atom refs/heads/main '%(worktreepath)' fail +batch_test_atom refs/heads/main '%(symref)' fail +batch_test_atom refs/heads/main '%(flag)' fail + +batch_test_atom refs/tags/testtag '%(refname)' fail +batch_test_atom refs/tags/testtag '%(refname:short)' fail +batch_test_atom refs/tags/testtag '%(upstream)' fail +batch_test_atom refs/tags/testtag '%(push)' fail +batch_test_atom refs/tags/testtag '%(objecttype)' +batch_test_atom refs/tags/testtag '%(objectsize)' +batch_test_atom refs/tags/testtag '%(objectsize:disk)' +batch_test_atom refs/tags/testtag '%(*objectsize:disk)' fail +batch_test_atom refs/tags/testtag '%(deltabase)' +batch_test_atom refs/tags/testtag '%(*deltabase)' fail +batch_test_atom refs/tags/testtag '%(objectname)' +batch_test_atom refs/tags/testtag '%(objectname:short)' fail +batch_test_atom refs/tags/testtag '%(tree)' fail +batch_test_atom refs/tags/testtag '%(tree:short)' fail +batch_test_atom refs/tags/testtag '%(tree:short=1)' fail +batch_test_atom refs/tags/testtag '%(tree:short=10)' fail +batch_test_atom refs/tags/testtag '%(parent)' fail +batch_test_atom refs/tags/testtag '%(parent:short)' fail +batch_test_atom refs/tags/testtag '%(parent:short=1)' fail +batch_test_atom refs/tags/testtag '%(parent:short=10)' fail +batch_test_atom refs/tags/testtag '%(numparent)' fail +batch_test_atom refs/tags/testtag '%(object)' fail +batch_test_atom refs/tags/testtag '%(type)' fail +batch_test_atom refs/tags/testtag '%(*objectname)' fail +batch_test_atom refs/tags/testtag '%(*objecttype)' fail +batch_test_atom refs/tags/testtag '%(author)' fail +batch_test_atom refs/tags/testtag '%(authorname)' fail +batch_test_atom refs/tags/testtag '%(authoremail)' fail +batch_test_atom refs/tags/testtag '%(authoremail:trim)' fail +batch_test_atom refs/tags/testtag '%(authoremail:localpart)' fail +batch_test_atom refs/tags/testtag '%(authordate)' fail +batch_test_atom refs/tags/testtag '%(committer)' fail +batch_test_atom refs/tags/testtag '%(committername)' fail +batch_test_atom refs/tags/testtag '%(committeremail)' fail +batch_test_atom refs/tags/testtag '%(committeremail:trim)' fail +batch_test_atom refs/tags/testtag '%(committeremail:localpart)' fail +batch_test_atom refs/tags/testtag '%(committerdate)' fail +batch_test_atom refs/tags/testtag '%(tag)' fail +batch_test_atom refs/tags/testtag '%(tagger)' fail +batch_test_atom refs/tags/testtag '%(taggername)' fail +batch_test_atom refs/tags/testtag '%(taggeremail)' fail +batch_test_atom refs/tags/testtag '%(taggeremail:trim)' fail +batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' fail +batch_test_atom refs/tags/testtag '%(taggerdate)' fail +batch_test_atom refs/tags/testtag '%(creator)' fail +batch_test_atom refs/tags/testtag '%(creatordate)' fail +batch_test_atom refs/tags/testtag '%(subject)' fail +batch_test_atom refs/tags/testtag '%(subject:sanitize)' fail +batch_test_atom refs/tags/testtag '%(contents:subject)' fail +batch_test_atom refs/tags/testtag '%(body)' fail +batch_test_atom refs/tags/testtag '%(contents:body)' fail +batch_test_atom refs/tags/testtag '%(contents:signature)' fail +batch_test_atom refs/tags/testtag '%(contents)' fail +batch_test_atom refs/tags/testtag '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(refname)' fail +batch_test_atom refs/myblobs/blob1 '%(upstream)' fail +batch_test_atom refs/myblobs/blob1 '%(push)' fail +batch_test_atom refs/myblobs/blob1 '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(objectname)' +batch_test_atom refs/myblobs/blob1 '%(objecttype)' +batch_test_atom refs/myblobs/blob1 '%(objectsize)' +batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)' +batch_test_atom refs/myblobs/blob1 '%(deltabase)' + +batch_test_atom refs/myblobs/blob1 '%(contents)' fail +batch_test_atom refs/myblobs/blob2 '%(contents)' fail + +batch_test_atom refs/myblobs/blob1 '%(raw)' fail +batch_test_atom refs/myblobs/blob2 '%(raw)' fail +batch_test_atom refs/mytrees/tree1 '%(raw)' fail + +batch_test_atom refs/myblobs/blob1 '%(raw:size)' fail +batch_test_atom refs/myblobs/blob2 '%(raw:size)' fail +batch_test_atom refs/mytrees/tree1 '%(raw:size)' fail + +batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail +batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail +batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' fail + +batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' fail +batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' fail +batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' fail +batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' fail + +batch_test_atom refs/heads/main 'VALID' +batch_test_atom refs/heads/main '%(INVALID)' fail +batch_test_atom refs/heads/main '%(authordate:INVALID)' fail + +batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize) +%(raw)' fail +batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize) +%(raw)' fail +batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize) +%(raw)' fail +batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize) +%(raw)' fail + test_done From patchwork Fri Aug 13 08:22:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BBE4C4320A for ; Fri, 13 Aug 2021 08:23:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 273EE610CC for ; Fri, 13 Aug 2021 08:23:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238896AbhHMIXz (ORCPT ); Fri, 13 Aug 2021 04:23:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238494AbhHMIXp (ORCPT ); Fri, 13 Aug 2021 04:23:45 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0FC6C0617AD for ; Fri, 13 Aug 2021 01:23:18 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id c129-20020a1c35870000b02902e6b6135279so4912706wma.0 for ; Fri, 13 Aug 2021 01:23:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=FT/cG9QZkDBiuJQX/fvgtVt3Iicb5lgNvBtJ43lZGPI=; b=COAsgrqaGYkB50J9zF3o2jyVnAJX+IqsSrLxfmOlrrqYuhikS4KaxH3uWdf1sp3YN2 wIqhgqau/Us3c9VqObnqrnm4w7YZ3/puuELxwZ8yv0JmRV8TFUEc7+4HXCoQxt7A2oKz YCtOlq1EoTpOxKkG7oYGyw8t8wITxxD9irIWB8YCaGcCHrMnr0UjnosOOu3MitWH1ty3 HzMNUvQhBYqZlS+V+6XsouveSH2Lpa+VK2BEKWO3YhWJUjuI4ZyU5qfnbVg69Xt3IVCP IKrQTgMZJBLHegXGpu2N33vj3qsZmXIZ5KrhFiK2VFvAIdEh6ptA7NiQOtDD7e/r8tGz O6Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=FT/cG9QZkDBiuJQX/fvgtVt3Iicb5lgNvBtJ43lZGPI=; b=Kmd2QbgELUCS0p7gRcwnXNyZKt3yIRhWHUxCO8Tt/1hQMsA8WtM12vrjXYqHS/pbG1 O6QvDvnbnWqKshcglwZLiUsdZwC5CaPsMtQrp9LFuoju2UMOXCO+cqOzBKCb2uAodPLt A2dOOW4dK3IVklQzWi/0Sjj43ktIY7neWDNynFpivOtbOgG9nKE4K4xwadWHVTLf4fvu yJfGlmkGlIVuD71zqdyJHz3tupzSS+fZbKCcB7/qMWag26enGFV+7/TJ9TtyeaiR5I+O 22r3TyFHnraJMy9/6kzy0oOb95/Xu7DIM+Cvgjrsi/btvJjW6xu/Wko3JlV7+ptAhQif NtrA== X-Gm-Message-State: AOAM531ENf8fb1FbP8WrzekfAoex1tN84twu7OXgFOs4i43kX4xiJaGF sCFIkOyFCmk2+oQJgD9uiPjCvRuYR8U= X-Google-Smtp-Source: ABdhPJz/GZRjKuwhArEdd0AW8lu1C9P9SVczTz24b58HaDglKBmbN3mE5Sy/RCZSGJ9rCls9gZZxEQ== X-Received: by 2002:a05:600c:2f8f:: with SMTP id t15mr1523413wmn.114.1628842997618; Fri, 13 Aug 2021 01:23:17 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o6sm825286wru.92.2021.08.13.01.23.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:17 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:52 +0000 Subject: [PATCH 09/27] [GSOC] ref-filter: modify the error message and value in get_object Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Let get_object() return 1 and print " missing" instead of returning -1 and printing "missing object for " if oid_object_info_extended() unable to find the data corresponding to oid. When `cat-file --batch` use ref-filter logic later it can help `format_ref_array_item()` just report that the object is missing without letting Git exit. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 4 ++-- t/t6301-for-each-ref-errors.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index e50acfc7378..fd482e9ff20 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1766,8 +1766,8 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) - return strbuf_addf_ret(err, -1, _("missing object %s for %s"), - oid_to_hex(&oi->oid), ref->refname); + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); if (oi->info.disk_sizep && oi->disk_size < 0) BUG("Object size is less than zero."); diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh index 40edf9dab53..3553f84a00c 100755 --- a/t/t6301-for-each-ref-errors.sh +++ b/t/t6301-for-each-ref-errors.sh @@ -41,7 +41,7 @@ test_expect_success 'Missing objects are reported correctly' ' r=refs/heads/missing && echo $MISSING >.git/$r && test_when_finished "rm -f .git/$r" && - echo "fatal: missing object $MISSING for $r" >missing-err && + echo "fatal: $MISSING missing" >missing-err && test_must_fail git for-each-ref 2>err && test_cmp missing-err err && ( From patchwork Fri Aug 13 08:22:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5D96C432BE for ; Fri, 13 Aug 2021 08:23:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BBDC9610CC for ; Fri, 13 Aug 2021 08:23:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238769AbhHMIX4 (ORCPT ); Fri, 13 Aug 2021 04:23:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238500AbhHMIXq (ORCPT ); Fri, 13 Aug 2021 04:23:46 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E202C0617AE for ; Fri, 13 Aug 2021 01:23:19 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id h13so12174215wrp.1 for ; Fri, 13 Aug 2021 01:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QNasbiGAZli473fXu4QgD2u/UTIEGvkgntmX6viQd9U=; b=tyCkZOh/WlA3f5X2DB7VazRVlTVZ++gtD7r0rmM9rcsH1iJh2ooj1XO/pAfDDrVyt0 qr5cAtfwZ0Rd2N9yv2GA8ozzNxlvQuSpUt9IjdYLGQMszWsErbM57YxfEemgs0lax4Ba Pm7uEm5g6cINiEcezExiz9yOXTlWhIOWswwg8FiQtvBaKHtVucknhQfu5hsBdyqBQbfo rAoNZMLlkcWYaM1BQnFmA0Qm/X5mwpWa1c/9DcAPyRVdE+JY0Bke6GwDVzhCDAXKVJxW pOadf3PznGcW73NFLcn1dultwBaynp5Fx8lI4qbBBrqI9NTPHnn84PUjBEb+t6Jy3Y/u V+Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QNasbiGAZli473fXu4QgD2u/UTIEGvkgntmX6viQd9U=; b=VTOKY6vX9G8STrDpYdtl52TfGS0wNLi//I0vG/fM5wB7h6KxGSiiOy234TrgHVdsQZ yDbjeBchIBYRQqsNeqeM4e1CgHRyyVfaWZMHzjGqtFwB3wcTQ2EyxDbJD3m/k+tX/2f+ fLHRQzfuMdhqWIzF8ttUqxmmtz7trHEk+tLJHmqLEEBVM4XcyDQZ7P2Vag62zcK2oWz5 yicrqWT2s+MZUgDcuLV+o3ONeTCtsAN68lDuEd4UVlVdan5RbJu4KmZQrSenULWE/zIC QB4VDPrVWfc1+daYldp33hV7jwoDipmmF3pL/Tx/ftIHyc+vUcogEdPpHxYIODRY8Zws Xo0Q== X-Gm-Message-State: AOAM530SmBhesbCzKGF/1GFlEoYifBkNbtC3gSsetQHdGzxCjcCVH3/T kciKMAHX+TxtZxcW5P3j1tquVOGLBXI= X-Google-Smtp-Source: ABdhPJydTdux6OtQ6J01xfn0qU55dROW/0I5pAhWQ6Ah195MI2HzaBIw2wmKfJWaMqr3uMm7paKlIQ== X-Received: by 2002:adf:f7c5:: with SMTP id a5mr1618637wrq.355.1628842998170; Fri, 13 Aug 2021 01:23:18 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y4sm680986wmi.22.2021.08.13.01.23.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:17 -0700 (PDT) Message-Id: <39d930361dbd87d1ddd2608e3346c10861653658.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:53 +0000 Subject: [PATCH 10/27] [GSOC] cat-file: add has_object_file() check Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use `has_object_file()` in `batch_one_object()` to check whether the input object exists. This can help us reject the missing oid when we let `cat-file --batch` use ref-filter logic later. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 243fe6844bc..59a86412fd0 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -428,6 +428,13 @@ static void batch_one_object(const char *obj_name, return; } + if (!has_object_file(&data->oid)) { + printf("%s missing\n", + obj_name ? obj_name : oid_to_hex(&data->oid)); + fflush(stdout); + return; + } + batch_object_write(obj_name, scratch, opt, data); } From patchwork Fri Aug 13 08:22:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF02BC4320E for ; Fri, 13 Aug 2021 08:23:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CCB5C610CC for ; Fri, 13 Aug 2021 08:23:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238796AbhHMIX5 (ORCPT ); Fri, 13 Aug 2021 04:23:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238535AbhHMIXq (ORCPT ); Fri, 13 Aug 2021 04:23:46 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AB9CC061756 for ; Fri, 13 Aug 2021 01:23:20 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id w21-20020a7bc1150000b02902e69ba66ce6so6380186wmi.1 for ; Fri, 13 Aug 2021 01:23:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=wxw28PT0y1HD093q1CN4MSHFbLeN7akEybJmased0TU=; b=EKaLUUdqnVXOsrl6190CvEWcqcGBsvlmRGaBvFR4OqRZJi8V97Oo+H25C67I6DLoQ9 iMciaW/YWZwFFiCK3N5Hj+go7R80sMeLhoN+C5H6QupyHyiShUyId3uDIK7rAQLwzJaB iN4pyjvSLeQPfbKrpkAauxjQR4yNfD1mibcY4JNRI+grFS8f3KWOH9GZpny8XdSa7kQf TZoBSUiJ+z7k+S2x6HsLgbsDA3+wDqd9pwsU2XI3yA9JsES9GpYlOXjRvw5oxvaPJsN/ ozo1zeMFzSDiMQfdXZwDh8VfBgMLpLxM2fLS5nnaTI6gxCgRfvFJ5WpbjuESwZhhd9Bn /tIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=wxw28PT0y1HD093q1CN4MSHFbLeN7akEybJmased0TU=; b=ZZTWYFlDaDPCUhw4fFhmevX85fGc8Fs6J7uZOYTP5iFb/AEj71cwb2yRJ7WkT7Bbgr FDcVWFB9zH9VyHzbN7gjYH9LPU28nYKPMBqwVqcE6CQgfew5Kd3tVLf1GqPPFbPp86zN qQzyB8syK06RefdPKxBRg5e0ySXmY6NivtztaHXWP28yTbqm3KXAJ3Dr5jqvf/fD+dvi hl28Znfqm4us5OoOO1kWp14rNLHFOanR++pgRTAmx40rDMctpb3PFQaBVVfk5U4UPl83 sJ5/p1YtRanEIiU2pUoDm7cmXxIwZPXB4RXJ+BFdcHFtOO4X/U8CrA2JjcQwS5PidpUP 0sTQ== X-Gm-Message-State: AOAM530TCP64t1ibsPZ5/jNe73NYgsro35DgZy2S/uZCCDaD08zgTTN7 o/9P8rAZMWWBf0n0VLd8R4UrAdbFlPM= X-Google-Smtp-Source: ABdhPJxD0woBmo7VVJuVNtK4XSfbFInjArh6UfyuX4f6aCwFOzVZy2taf4oh5epkmHFcFkAr91HBrg== X-Received: by 2002:a05:600c:2290:: with SMTP id 16mr1459893wmf.26.1628842998795; Fri, 13 Aug 2021 01:23:18 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y21sm716579wma.38.2021.08.13.01.23.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:18 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:54 +0000 Subject: [PATCH 11/27] [GSOC] cat-file: change batch_objects parameter name Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because later cat-file reuses ref-filter logic that will add parameter "const struct option *options" to batch_objects(), the two synonymous parameters of "opt" and "options" may confuse readers, so change batch_options parameter of batch_objects() from "opt" to "batch". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 59a86412fd0..41d407638d5 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -495,7 +495,7 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *opt) +static int batch_objects(struct batch_options *batch) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; @@ -503,8 +503,8 @@ static int batch_objects(struct batch_options *opt) int save_warning; int retval = 0; - if (!opt->format) - opt->format = "%(objectname) %(objecttype) %(objectsize)"; + if (!batch->format) + batch->format = "%(objectname) %(objecttype) %(objectsize)"; /* * Expand once with our special mark_query flag, which will prime the @@ -513,20 +513,20 @@ static int batch_objects(struct batch_options *opt) */ memset(&data, 0, sizeof(data)); data.mark_query = 1; - strbuf_expand(&output, opt->format, expand_format, &data); + strbuf_expand(&output, batch->format, expand_format, &data); data.mark_query = 0; strbuf_release(&output); - if (opt->cmdmode) + if (batch->cmdmode) data.split_on_whitespace = 1; /* * If we are printing out the object, then always fill in the type, * since we will want to decide whether or not to stream. */ - if (opt->print_contents) + if (batch->print_contents) data.info.typep = &data.type; - if (opt->all_objects) { + if (batch->all_objects) { struct object_cb_data cb; struct object_info empty = OBJECT_INFO_INIT; @@ -536,11 +536,11 @@ static int batch_objects(struct batch_options *opt) if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); - cb.opt = opt; + cb.opt = batch; cb.expand = &data; cb.scratch = &output; - if (opt->unordered) { + if (batch->unordered) { struct oidset seen = OIDSET_INIT; cb.seen = &seen; @@ -590,7 +590,7 @@ static int batch_objects(struct batch_options *opt) data.rest = p; } - batch_one_object(input.buf, &output, opt, &data); + batch_one_object(input.buf, &output, batch, &data); } strbuf_release(&input); From patchwork Fri Aug 13 08:22:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36747C432BE for ; Fri, 13 Aug 2021 08:23:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D5666109E for ; Fri, 13 Aug 2021 08:23:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238857AbhHMIX6 (ORCPT ); Fri, 13 Aug 2021 04:23:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238598AbhHMIXr (ORCPT ); Fri, 13 Aug 2021 04:23:47 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9208C0617AD for ; Fri, 13 Aug 2021 01:23:20 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id k4so6336405wms.3 for ; Fri, 13 Aug 2021 01:23:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=9N06sBMGSwslqzktoVL9OhM7KzXq4rCuo4jo7obxQWo=; b=mvK+HeAjgr1n8XD6NpALfu+NoZmsas8y4ToSjk5Cif8Y/eJiocBVcHxxIcdIUh/L6i 6XceF3ygfEr9GxYnguh52ISBhHrFHuQ1XYJy4BeLegBYfX4G1Y/jNz9qGXJz+6h7jnyf zy4hMrgzJgqro6bAoLWiiV16UYkzpaGqGylQgAT4r3Kp1iWPMVUKS0aONqfAjgCRf3xr Hb3A2Q+Yz4Pnu62pVm+qst1UCdnic7sIm15qbvBTVElpMaNOiO1LIaePs9hscH7wE5eq fl77dtEY0d7WtkGIvNYE8JwaFwBJvk0YrprqrHoY2FMSHmiq479ZdyPBot8/ZziC8RBR lxDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=9N06sBMGSwslqzktoVL9OhM7KzXq4rCuo4jo7obxQWo=; b=TnFH4IiULXRgPykYApD2SOwXJ3q12x2Bs5lHBkh/ZLMU3mRRYLR2zIi4GMA0qx4la4 0Uc11CcUjFnEAdJGsW/h1Dgx/JR3iaJV+pLg+dkkc/AIfM3IMoyJFPXBeSbCIArNR8Y+ SKZOZb6dc2GtztHiwYCCowKCcKdzEE+3Obcalvo1KBzIQeOesfYAyk9QwHrEtTNVPV2B v8bxaepX2TJgd9HnBqPPVFU94JnXzEbzNs8QR0tpHR8sJJ+kw9HBYTYEUbCmGaeLK3p9 o1NlfGTkCJcdqTzHbd6Pc6NeW9DQL7ILf/UGkx+jzOQPHMTvZAy9THMs62Xtge0AFdf5 /8ow== X-Gm-Message-State: AOAM533mHE68lxrvFF+wi/QEE+ws1nY13EHdFx8iaW0cISMnMy6fNzM1 luQF2UWieC2yg/5+rFAQ993pSdpd7II= X-Google-Smtp-Source: ABdhPJw2b+dbu6DAc5JVy1PaKqW8CxhuveBfHYU4I7fGXdxQ60pD5sSA8nMt9bkv1Kld99QmVpIQjw== X-Received: by 2002:a05:600c:198e:: with SMTP id t14mr1381423wmq.167.1628842999421; Fri, 13 Aug 2021 01:23:19 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p4sm928696wrq.81.2021.08.13.01.23.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:19 -0700 (PDT) Message-Id: <1f72eb6e4e0f0e5eaeef2b45deebc81f2b611357.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:55 +0000 Subject: [PATCH 12/27] [GSOC] cat-file: create p1006-cat-file.sh MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Create p1006-cat-file.sh to provide performance testing for `git cat-file --batch` and `git cat-file --batch-check`. This will help us compare the performance changes after we let cat-file reuse the ref-filter logic. Helped-by: Ævar Arnfjörð Bjarmason Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- t/perf/p1006-cat-file.sh | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 t/perf/p1006-cat-file.sh diff --git a/t/perf/p1006-cat-file.sh b/t/perf/p1006-cat-file.sh new file mode 100755 index 00000000000..b84ac31f9cc --- /dev/null +++ b/t/perf/p1006-cat-file.sh @@ -0,0 +1,28 @@ +#!/bin/sh + +test_description='Basic sort performance tests' +. ./perf-lib.sh + +test_perf_default_repo + +test_expect_success 'setup' ' + git rev-list --all >rla +' + +test_perf 'cat-file --batch-check' ' + git cat-file --batch-check X-Patchwork-Id: 12435107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A9C2C4338F for ; Fri, 13 Aug 2021 08:23:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31FBB6056B for ; Fri, 13 Aug 2021 08:23:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238858AbhHMIX7 (ORCPT ); Fri, 13 Aug 2021 04:23:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238615AbhHMIXs (ORCPT ); Fri, 13 Aug 2021 04:23:48 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E17EC0617AE for ; Fri, 13 Aug 2021 01:23:21 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id f5so12088218wrm.13 for ; Fri, 13 Aug 2021 01:23:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=ToHDwQjxG11gRqhYcRQilRtbLuv3mILTyrTDp4s4nuc=; b=M7GKN1ZiQyNxUSE/zTYcCLsVngZHnr0jfuwjcEGLQ2C4w0yMHCozYzJF5prS0kWIkQ 2CIFyley0LM05XQ6sX9gWfyzN/km2lTldWanE/fl9r5XN2buRVyJlqdxyp8p9oSuVXoc KucwHlRJt/4MHMuMmMrsaV5yAb53xf0BmNdzA7vNirqX57xh0VVtrveueLoM7DNTz6E8 QP0rcsLzujGKkX7CvFQ37JZtiMkyEzuV1mC+KVe8H6tznKTYBHNVarywTTCcbV6JKQBI h/XJ0wIHeg1bhHSuLbeF90PJ5nU7bWvelKXSygFHHpawa3p8qewa/g0U1UnCwAhr1t1d x71A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=ToHDwQjxG11gRqhYcRQilRtbLuv3mILTyrTDp4s4nuc=; b=V9QkjuFhtUMwXc51u8Siz5lkHl2LYM9VZYtSDoX5ao5vSz9XPW5SKVWNIwALXt8yeR DZtx+oeEWfYtt3gE5wdO2fmimMB+Mse97CE9dR1uRBbhGzpphNVOXsG/wUH0Ntrcp+dY JzYg5mmXpEH5kYmvHds9F3LADFa7Jz5v+UWzO6WX2AgdvxWp1sXjhL+x802+2bwnSW42 rjNnIqHjIMlNW+FY0mH1+1EKfw5f2LeaDWNeu46nDCjyQUmyC4e6g8snOJsaFZdtZtVL ItXqrb9LnGfzi0fZZenlM02sFWcaq3MA801s8zYozW2mXyMB0iqor6wba5yuvJLAdVXM rrKA== X-Gm-Message-State: AOAM531v9WBLl1L5FeuNC8GfPYQaIGqUw+oyIRRuwM+0aWTig9Ed4UcI P3RfW+Nh2d/5IrtNj3wLCvzWiboBJck= X-Google-Smtp-Source: ABdhPJxxV02WPmZUM8RjxHx47pdFsISOjL1JGLMaFCEIsu3jYgKOWdgrvBsQPbI5Uuex1/p9eQMTXQ== X-Received: by 2002:a5d:464a:: with SMTP id j10mr1641988wrs.190.1628843000175; Fri, 13 Aug 2021 01:23:20 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z13sm827079wrs.71.2021.08.13.01.23.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:19 -0700 (PDT) Message-Id: <9c5fddf6885875ccd3ce3f047bb938c77d9bbca2.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:56 +0000 Subject: [PATCH 13/27] [GSOC] cat-file: reuse ref-filter logic MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let cat-file use ref-filter logic, let's do the following: 1. Change the type of member `format` in struct `batch_options` to `ref_format`, we will pass it to ref-filter later. 2. Let `batch_objects()` add atoms to format, and use `verify_ref_format()` to check atoms. 3. Use `format_ref_array_item()` in `batch_object_write()` to get the formatted data corresponding to the object. If the return value of `format_ref_array_item()` is equals to zero, use `batch_write()` to print object data; else if the return value is less than zero, use `die()` to print the error message and exit; else if return value is greater than zero, only print the error message, but don't exit. 4. Use free_ref_array_item_value() to free ref_array_item's value, and use free_global_resource() to free global memory allocated resource. Most of the atoms in `for-each-ref --format` are now supported, such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, `%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: `%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, `%(flag)`, `%(HEAD)`, because these atoms are unique to those objects that pointed to by a ref, "for-each-ref"'s family can naturally use these atoms, but not all objects are pointed to be a ref, so "cat-file" will not be able to use them. The performance for `git cat-file --batch-all-objects --batch-check` on the Git repository itself with performance testing tool `hyperfine` changes from 616.7 ms ± 8.9 ms to 758.7 ms ± 16.4 ms. The performance for `git cat-file --batch-all-objects --batch >/dev/null` on the Git repository itself with performance testing tool `time` change from "25.26s user 0.30s system 98% cpu 25.840 total" to "28.79s user 0.83s system 99% cpu 29.829 total". The reasons for the performance degradation are as follows: 1. There are a lot of data copies in the logic of ref-filter. 2, In order to be able to support more useful formats, complex data structure and parsing process are used in ref-filter. Later patches will gradually deal with the problem of performance degradation. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- Documentation/git-cat-file.txt | 6 + builtin/cat-file.c | 245 +++++++-------------------------- ref-filter.c | 20 ++- ref-filter.h | 2 + t/t1006-cat-file.sh | 229 +++++++++++++++--------------- 5 files changed, 191 insertions(+), 311 deletions(-) diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index 4eb0421b3fd..ef8ab952b2f 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -226,6 +226,12 @@ newline. The available atoms are: after that first run of whitespace (i.e., the "rest" of the line) are output in place of the `%(rest)` atom. +Note that most of the atoms in `for-each-ref --format` are now supported, +such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, +`%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: +`%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, +`%(flag)`, `%(HEAD)`. See linkgit:git-for-each-ref[1]. + If no format is specified, the default format is `%(objectname) %(objecttype) %(objectsize)`. diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 41d407638d5..306b4d49493 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -16,6 +16,7 @@ #include "packfile.h" #include "object-store.h" #include "promisor-remote.h" +#include "ref-filter.h" struct batch_options { int enabled; @@ -25,7 +26,7 @@ struct batch_options { int all_objects; int unordered; int cmdmode; /* may be 'w' or 'c' for --filters or --textconv */ - const char *format; + struct ref_format format; }; static const char *force_path; @@ -195,99 +196,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct expand_data { struct object_id oid; - enum object_type type; - unsigned long size; - off_t disk_size; const char *rest; - struct object_id delta_base_oid; - - /* - * If mark_query is true, we do not expand anything, but rather - * just mark the object_info with items we wish to query. - */ - int mark_query; - - /* - * Whether to split the input on whitespace before feeding it to - * get_sha1; this is decided during the mark_query phase based on - * whether we have a %(rest) token in our format. - */ int split_on_whitespace; - - /* - * After a mark_query run, this object_info is set up to be - * passed to oid_object_info_extended. It will point to the data - * elements above, so you can retrieve the response from there. - */ - struct object_info info; - - /* - * This flag will be true if the requested batch format and options - * don't require us to call oid_object_info, which can then be - * optimized out. - */ - unsigned skip_object_info : 1; }; -static int is_atom(const char *atom, const char *s, int slen) -{ - int alen = strlen(atom); - return alen == slen && !memcmp(atom, s, alen); -} - -static void expand_atom(struct strbuf *sb, const char *atom, int len, - void *vdata) -{ - struct expand_data *data = vdata; - - if (is_atom("objectname", atom, len)) { - if (!data->mark_query) - strbuf_addstr(sb, oid_to_hex(&data->oid)); - } else if (is_atom("objecttype", atom, len)) { - if (data->mark_query) - data->info.typep = &data->type; - else - strbuf_addstr(sb, type_name(data->type)); - } else if (is_atom("objectsize", atom, len)) { - if (data->mark_query) - data->info.sizep = &data->size; - else - strbuf_addf(sb, "%"PRIuMAX , (uintmax_t)data->size); - } else if (is_atom("objectsize:disk", atom, len)) { - if (data->mark_query) - data->info.disk_sizep = &data->disk_size; - else - strbuf_addf(sb, "%"PRIuMAX, (uintmax_t)data->disk_size); - } else if (is_atom("rest", atom, len)) { - if (data->mark_query) - data->split_on_whitespace = 1; - else if (data->rest) - strbuf_addstr(sb, data->rest); - } else if (is_atom("deltabase", atom, len)) { - if (data->mark_query) - data->info.delta_base_oid = &data->delta_base_oid; - else - strbuf_addstr(sb, - oid_to_hex(&data->delta_base_oid)); - } else - die("unknown format element: %.*s", len, atom); -} - -static size_t expand_format(struct strbuf *sb, const char *start, void *data) -{ - const char *end; - - if (*start != '(') - return 0; - end = strchr(start + 1, ')'); - if (!end) - die("format element '%s' does not end in ')'", start); - - expand_atom(sb, start + 1, end - start - 1, data); - - return end - start + 1; -} - static void batch_write(struct batch_options *opt, const void *data, int len) { if (opt->buffer_output) { @@ -297,87 +209,34 @@ static void batch_write(struct batch_options *opt, const void *data, int len) write_or_die(1, data, len); } -static void print_object_or_die(struct batch_options *opt, struct expand_data *data) -{ - const struct object_id *oid = &data->oid; - - assert(data->info.typep); - - if (data->type == OBJ_BLOB) { - if (opt->buffer_output) - fflush(stdout); - if (opt->cmdmode) { - char *contents; - unsigned long size; - - if (!data->rest) - die("missing path for '%s'", oid_to_hex(oid)); - - if (opt->cmdmode == 'w') { - if (filter_object(data->rest, 0100644, oid, - &contents, &size)) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else if (opt->cmdmode == 'c') { - enum object_type type; - if (!textconv_object(the_repository, - data->rest, 0100644, oid, - 1, &contents, &size)) - contents = read_object_file(oid, - &type, - &size); - if (!contents) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else - BUG("invalid cmdmode: %c", opt->cmdmode); - batch_write(opt, contents, size); - free(contents); - } else { - stream_blob(oid); - } - } - else { - enum object_type type; - unsigned long size; - void *contents; - - contents = read_object_file(oid, &type, &size); - if (!contents) - die("object %s disappeared", oid_to_hex(oid)); - if (type != data->type) - die("object %s changed type!?", oid_to_hex(oid)); - if (data->info.sizep && size != data->size) - die("object %s changed size!?", oid_to_hex(oid)); - - batch_write(opt, contents, size); - free(contents); - } -} static void batch_object_write(const char *obj_name, struct strbuf *scratch, struct batch_options *opt, struct expand_data *data) { - if (!data->skip_object_info && - oid_object_info_extended(the_repository, &data->oid, &data->info, - OBJECT_INFO_LOOKUP_REPLACE) < 0) { - printf("%s missing\n", - obj_name ? obj_name : oid_to_hex(&data->oid)); - fflush(stdout); - return; - } + int ret; + struct strbuf err = STRBUF_INIT; + struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); - strbuf_expand(scratch, opt->format, expand_format, data); - strbuf_addch(scratch, '\n'); - batch_write(opt, scratch->buf, scratch->len); - if (opt->print_contents) { - print_object_or_die(opt, data); - batch_write(opt, "\n", 1); + ret = format_ref_array_item(&item, &opt->format, scratch, &err); + if (ret < 0) + die("%s\n", err.buf); + if (ret) { + /* ret > 0 means when the object corresponding to oid + * cannot be found in format_ref_array_item(), we only print + * the error message. + */ + printf("%s\n", err.buf); + fflush(stdout); + } else { + strbuf_addch(scratch, '\n'); + batch_write(opt, scratch->buf, scratch->len); } + free_ref_array_item_value(&item); + strbuf_release(&err); } static void batch_one_object(const char *obj_name, @@ -495,43 +354,37 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *batch) +static const char * const cat_file_usage[] = { + N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), + N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), + NULL +}; + +static int batch_objects(struct batch_options *batch, const struct option *options) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; int retval = 0; - if (!batch->format) - batch->format = "%(objectname) %(objecttype) %(objectsize)"; - - /* - * Expand once with our special mark_query flag, which will prime the - * object_info to be handed to oid_object_info_extended for each - * object. - */ memset(&data, 0, sizeof(data)); - data.mark_query = 1; - strbuf_expand(&output, batch->format, expand_format, &data); - data.mark_query = 0; - strbuf_release(&output); - if (batch->cmdmode) - data.split_on_whitespace = 1; - - /* - * If we are printing out the object, then always fill in the type, - * since we will want to decide whether or not to stream. - */ + if (batch->format.format) + strbuf_addstr(&format, batch->format.format); + else + strbuf_addstr(&format, "%(objectname) %(objecttype) %(objectsize)"); if (batch->print_contents) - data.info.typep = &data.type; + strbuf_addstr(&format, "\n%(raw)"); + batch->format.format = format.buf; + if (verify_ref_format(&batch->format)) + usage_with_options(cat_file_usage, options); + + if (batch->cmdmode || batch->format.use_rest) + data.split_on_whitespace = 1; if (batch->all_objects) { struct object_cb_data cb; - struct object_info empty = OBJECT_INFO_INIT; - - if (!memcmp(&data.info, &empty, sizeof(empty))) - data.skip_object_info = 1; if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); @@ -560,8 +413,9 @@ static int batch_objects(struct batch_options *batch) oid_array_clear(&sa); } - + strbuf_release(&format); strbuf_release(&output); + free_global_resource(); return 0; } @@ -593,18 +447,14 @@ static int batch_objects(struct batch_options *batch) batch_one_object(input.buf, &output, batch, &data); } + strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); + free_global_resource(); warn_on_object_refname_ambiguity = save_warning; return retval; } -static const char * const cat_file_usage[] = { - N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), - N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), - NULL -}; - static int git_cat_file_config(const char *var, const char *value, void *cb) { if (userdiff_config(var, value) < 0) @@ -627,7 +477,7 @@ static int batch_option_callback(const struct option *opt, bo->enabled = 1; bo->print_contents = !strcmp(opt->long_name, "batch"); - bo->format = arg; + bo->format.format = arg; return 0; } @@ -636,7 +486,9 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) { int opt = 0; const char *exp_type = NULL, *obj_name = NULL; - struct batch_options batch = {0}; + struct batch_options batch = { + .format = REF_FORMAT_INIT + }; int unknown_type = 0; const struct option options[] = { @@ -675,6 +527,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) git_config(git_cat_file_config, NULL); batch.buffer_output = -1; + batch.format.cat_file_mode = 1; argc = parse_options(argc, argv, prefix, options, cat_file_usage, 0); if (opt) { @@ -718,7 +571,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) batch.buffer_output = batch.all_objects; if (batch.enabled) - return batch_objects(&batch); + return batch_objects(&batch, options); if (unknown_type && opt != 't' && opt != 's') die("git cat-file --allow-unknown-type: use with -s or -t"); diff --git a/ref-filter.c b/ref-filter.c index fd482e9ff20..f83b6f83494 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2333,16 +2333,11 @@ static void free_array_item(struct ref_array_item *item) free(item); } -/* Free all memory allocated for ref_array */ -void ref_array_clear(struct ref_array *array) +/* Free global memory allocated resource */ +void free_global_resource(void) { int i; - for (i = 0; i < array->nr; i++) - free_array_item(array->items[i]); - FREE_AND_NULL(array->items); - array->nr = array->alloc = 0; - for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; if (atom->atom_type == ATOM_HEAD) @@ -2359,6 +2354,17 @@ void ref_array_clear(struct ref_array *array) ref_to_worktree_map.worktrees = NULL; } } +/* Free all memory allocated for ref_array */ +void ref_array_clear(struct ref_array *array) +{ + int i; + + for (i = 0; i < array->nr; i++) + free_array_item(array->items[i]); + FREE_AND_NULL(array->items); + array->nr = array->alloc = 0; + free_global_resource(); +} #define EXCLUDE_REACHED 0 #define INCLUDE_REACHED 1 diff --git a/ref-filter.h b/ref-filter.h index 053980a6a42..ef3d89d73d4 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -112,6 +112,8 @@ struct ref_format { int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type); /* Clear all memory allocated to ref_array */ void ref_array_clear(struct ref_array *array); +/* Free global memory allocated resource */ +void free_global_resource(void); /* Free ref_array_item's value */ void free_ref_array_item_value(struct ref_array_item *item); /* Used to verify if the given format is correct and to parse out the used atoms */ diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 95d760652eb..8c1943011fb 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -679,51 +679,51 @@ batch_test_atom refs/heads/main '%(objectsize)' batch_test_atom refs/heads/main '%(objectsize:disk)' batch_test_atom refs/heads/main '%(deltabase)' batch_test_atom refs/heads/main '%(objectname)' -batch_test_atom refs/heads/main '%(objectname:short)' fail -batch_test_atom refs/heads/main '%(objectname:short=1)' fail -batch_test_atom refs/heads/main '%(objectname:short=10)' fail -batch_test_atom refs/heads/main '%(tree)' fail -batch_test_atom refs/heads/main '%(tree:short)' fail -batch_test_atom refs/heads/main '%(tree:short=1)' fail -batch_test_atom refs/heads/main '%(tree:short=10)' fail -batch_test_atom refs/heads/main '%(parent)' fail -batch_test_atom refs/heads/main '%(parent:short)' fail -batch_test_atom refs/heads/main '%(parent:short=1)' fail -batch_test_atom refs/heads/main '%(parent:short=10)' fail -batch_test_atom refs/heads/main '%(numparent)' fail -batch_test_atom refs/heads/main '%(object)' fail -batch_test_atom refs/heads/main '%(type)' fail -batch_test_atom refs/heads/main '%(raw)' fail -batch_test_atom refs/heads/main '%(*objectname)' fail -batch_test_atom refs/heads/main '%(*objecttype)' fail -batch_test_atom refs/heads/main '%(author)' fail -batch_test_atom refs/heads/main '%(authorname)' fail -batch_test_atom refs/heads/main '%(authoremail)' fail -batch_test_atom refs/heads/main '%(authoremail:trim)' fail -batch_test_atom refs/heads/main '%(authoremail:localpart)' fail -batch_test_atom refs/heads/main '%(authordate)' fail -batch_test_atom refs/heads/main '%(committer)' fail -batch_test_atom refs/heads/main '%(committername)' fail -batch_test_atom refs/heads/main '%(committeremail)' fail -batch_test_atom refs/heads/main '%(committeremail:trim)' fail -batch_test_atom refs/heads/main '%(committeremail:localpart)' fail -batch_test_atom refs/heads/main '%(committerdate)' fail -batch_test_atom refs/heads/main '%(tag)' fail -batch_test_atom refs/heads/main '%(tagger)' fail -batch_test_atom refs/heads/main '%(taggername)' fail -batch_test_atom refs/heads/main '%(taggeremail)' fail -batch_test_atom refs/heads/main '%(taggeremail:trim)' fail -batch_test_atom refs/heads/main '%(taggeremail:localpart)' fail -batch_test_atom refs/heads/main '%(taggerdate)' fail -batch_test_atom refs/heads/main '%(creator)' fail -batch_test_atom refs/heads/main '%(creatordate)' fail -batch_test_atom refs/heads/main '%(subject)' fail -batch_test_atom refs/heads/main '%(subject:sanitize)' fail -batch_test_atom refs/heads/main '%(contents:subject)' fail -batch_test_atom refs/heads/main '%(body)' fail -batch_test_atom refs/heads/main '%(contents:body)' fail -batch_test_atom refs/heads/main '%(contents:signature)' fail -batch_test_atom refs/heads/main '%(contents)' fail +batch_test_atom refs/heads/main '%(objectname:short)' +batch_test_atom refs/heads/main '%(objectname:short=1)' +batch_test_atom refs/heads/main '%(objectname:short=10)' +batch_test_atom refs/heads/main '%(tree)' +batch_test_atom refs/heads/main '%(tree:short)' +batch_test_atom refs/heads/main '%(tree:short=1)' +batch_test_atom refs/heads/main '%(tree:short=10)' +batch_test_atom refs/heads/main '%(parent)' +batch_test_atom refs/heads/main '%(parent:short)' +batch_test_atom refs/heads/main '%(parent:short=1)' +batch_test_atom refs/heads/main '%(parent:short=10)' +batch_test_atom refs/heads/main '%(numparent)' +batch_test_atom refs/heads/main '%(object)' +batch_test_atom refs/heads/main '%(type)' +batch_test_atom refs/heads/main '%(raw)' +batch_test_atom refs/heads/main '%(*objectname)' +batch_test_atom refs/heads/main '%(*objecttype)' +batch_test_atom refs/heads/main '%(author)' +batch_test_atom refs/heads/main '%(authorname)' +batch_test_atom refs/heads/main '%(authoremail)' +batch_test_atom refs/heads/main '%(authoremail:trim)' +batch_test_atom refs/heads/main '%(authoremail:localpart)' +batch_test_atom refs/heads/main '%(authordate)' +batch_test_atom refs/heads/main '%(committer)' +batch_test_atom refs/heads/main '%(committername)' +batch_test_atom refs/heads/main '%(committeremail)' +batch_test_atom refs/heads/main '%(committeremail:trim)' +batch_test_atom refs/heads/main '%(committeremail:localpart)' +batch_test_atom refs/heads/main '%(committerdate)' +batch_test_atom refs/heads/main '%(tag)' +batch_test_atom refs/heads/main '%(tagger)' +batch_test_atom refs/heads/main '%(taggername)' +batch_test_atom refs/heads/main '%(taggeremail)' +batch_test_atom refs/heads/main '%(taggeremail:trim)' +batch_test_atom refs/heads/main '%(taggeremail:localpart)' +batch_test_atom refs/heads/main '%(taggerdate)' +batch_test_atom refs/heads/main '%(creator)' +batch_test_atom refs/heads/main '%(creatordate)' +batch_test_atom refs/heads/main '%(subject)' +batch_test_atom refs/heads/main '%(subject:sanitize)' +batch_test_atom refs/heads/main '%(contents:subject)' +batch_test_atom refs/heads/main '%(body)' +batch_test_atom refs/heads/main '%(contents:body)' +batch_test_atom refs/heads/main '%(contents:signature)' +batch_test_atom refs/heads/main '%(contents)' batch_test_atom refs/heads/main '%(HEAD)' fail batch_test_atom refs/heads/main '%(upstream:track)' fail batch_test_atom refs/heads/main '%(upstream:trackshort)' fail @@ -742,52 +742,52 @@ batch_test_atom refs/tags/testtag '%(push)' fail batch_test_atom refs/tags/testtag '%(objecttype)' batch_test_atom refs/tags/testtag '%(objectsize)' batch_test_atom refs/tags/testtag '%(objectsize:disk)' -batch_test_atom refs/tags/testtag '%(*objectsize:disk)' fail +batch_test_atom refs/tags/testtag '%(*objectsize:disk)' batch_test_atom refs/tags/testtag '%(deltabase)' -batch_test_atom refs/tags/testtag '%(*deltabase)' fail +batch_test_atom refs/tags/testtag '%(*deltabase)' batch_test_atom refs/tags/testtag '%(objectname)' -batch_test_atom refs/tags/testtag '%(objectname:short)' fail -batch_test_atom refs/tags/testtag '%(tree)' fail -batch_test_atom refs/tags/testtag '%(tree:short)' fail -batch_test_atom refs/tags/testtag '%(tree:short=1)' fail -batch_test_atom refs/tags/testtag '%(tree:short=10)' fail -batch_test_atom refs/tags/testtag '%(parent)' fail -batch_test_atom refs/tags/testtag '%(parent:short)' fail -batch_test_atom refs/tags/testtag '%(parent:short=1)' fail -batch_test_atom refs/tags/testtag '%(parent:short=10)' fail -batch_test_atom refs/tags/testtag '%(numparent)' fail -batch_test_atom refs/tags/testtag '%(object)' fail -batch_test_atom refs/tags/testtag '%(type)' fail -batch_test_atom refs/tags/testtag '%(*objectname)' fail -batch_test_atom refs/tags/testtag '%(*objecttype)' fail -batch_test_atom refs/tags/testtag '%(author)' fail -batch_test_atom refs/tags/testtag '%(authorname)' fail -batch_test_atom refs/tags/testtag '%(authoremail)' fail -batch_test_atom refs/tags/testtag '%(authoremail:trim)' fail -batch_test_atom refs/tags/testtag '%(authoremail:localpart)' fail -batch_test_atom refs/tags/testtag '%(authordate)' fail -batch_test_atom refs/tags/testtag '%(committer)' fail -batch_test_atom refs/tags/testtag '%(committername)' fail -batch_test_atom refs/tags/testtag '%(committeremail)' fail -batch_test_atom refs/tags/testtag '%(committeremail:trim)' fail -batch_test_atom refs/tags/testtag '%(committeremail:localpart)' fail -batch_test_atom refs/tags/testtag '%(committerdate)' fail -batch_test_atom refs/tags/testtag '%(tag)' fail -batch_test_atom refs/tags/testtag '%(tagger)' fail -batch_test_atom refs/tags/testtag '%(taggername)' fail -batch_test_atom refs/tags/testtag '%(taggeremail)' fail -batch_test_atom refs/tags/testtag '%(taggeremail:trim)' fail -batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' fail -batch_test_atom refs/tags/testtag '%(taggerdate)' fail -batch_test_atom refs/tags/testtag '%(creator)' fail -batch_test_atom refs/tags/testtag '%(creatordate)' fail -batch_test_atom refs/tags/testtag '%(subject)' fail -batch_test_atom refs/tags/testtag '%(subject:sanitize)' fail -batch_test_atom refs/tags/testtag '%(contents:subject)' fail -batch_test_atom refs/tags/testtag '%(body)' fail -batch_test_atom refs/tags/testtag '%(contents:body)' fail -batch_test_atom refs/tags/testtag '%(contents:signature)' fail -batch_test_atom refs/tags/testtag '%(contents)' fail +batch_test_atom refs/tags/testtag '%(objectname:short)' +batch_test_atom refs/tags/testtag '%(tree)' +batch_test_atom refs/tags/testtag '%(tree:short)' +batch_test_atom refs/tags/testtag '%(tree:short=1)' +batch_test_atom refs/tags/testtag '%(tree:short=10)' +batch_test_atom refs/tags/testtag '%(parent)' +batch_test_atom refs/tags/testtag '%(parent:short)' +batch_test_atom refs/tags/testtag '%(parent:short=1)' +batch_test_atom refs/tags/testtag '%(parent:short=10)' +batch_test_atom refs/tags/testtag '%(numparent)' +batch_test_atom refs/tags/testtag '%(object)' +batch_test_atom refs/tags/testtag '%(type)' +batch_test_atom refs/tags/testtag '%(*objectname)' +batch_test_atom refs/tags/testtag '%(*objecttype)' +batch_test_atom refs/tags/testtag '%(author)' +batch_test_atom refs/tags/testtag '%(authorname)' +batch_test_atom refs/tags/testtag '%(authoremail)' +batch_test_atom refs/tags/testtag '%(authoremail:trim)' +batch_test_atom refs/tags/testtag '%(authoremail:localpart)' +batch_test_atom refs/tags/testtag '%(authordate)' +batch_test_atom refs/tags/testtag '%(committer)' +batch_test_atom refs/tags/testtag '%(committername)' +batch_test_atom refs/tags/testtag '%(committeremail)' +batch_test_atom refs/tags/testtag '%(committeremail:trim)' +batch_test_atom refs/tags/testtag '%(committeremail:localpart)' +batch_test_atom refs/tags/testtag '%(committerdate)' +batch_test_atom refs/tags/testtag '%(tag)' +batch_test_atom refs/tags/testtag '%(tagger)' +batch_test_atom refs/tags/testtag '%(taggername)' +batch_test_atom refs/tags/testtag '%(taggeremail)' +batch_test_atom refs/tags/testtag '%(taggeremail:trim)' +batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' +batch_test_atom refs/tags/testtag '%(taggerdate)' +batch_test_atom refs/tags/testtag '%(creator)' +batch_test_atom refs/tags/testtag '%(creatordate)' +batch_test_atom refs/tags/testtag '%(subject)' +batch_test_atom refs/tags/testtag '%(subject:sanitize)' +batch_test_atom refs/tags/testtag '%(contents:subject)' +batch_test_atom refs/tags/testtag '%(body)' +batch_test_atom refs/tags/testtag '%(contents:body)' +batch_test_atom refs/tags/testtag '%(contents:signature)' +batch_test_atom refs/tags/testtag '%(contents)' batch_test_atom refs/tags/testtag '%(HEAD)' fail batch_test_atom refs/myblobs/blob1 '%(refname)' fail @@ -801,37 +801,50 @@ batch_test_atom refs/myblobs/blob1 '%(objectsize)' batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)' batch_test_atom refs/myblobs/blob1 '%(deltabase)' -batch_test_atom refs/myblobs/blob1 '%(contents)' fail -batch_test_atom refs/myblobs/blob2 '%(contents)' fail +batch_test_atom refs/myblobs/blob1 '%(contents)' +batch_test_atom refs/myblobs/blob2 '%(contents)' -batch_test_atom refs/myblobs/blob1 '%(raw)' fail -batch_test_atom refs/myblobs/blob2 '%(raw)' fail -batch_test_atom refs/mytrees/tree1 '%(raw)' fail +batch_test_atom refs/myblobs/blob1 '%(raw)' +batch_test_atom refs/myblobs/blob2 '%(raw)' +batch_test_atom refs/mytrees/tree1 '%(raw)' -batch_test_atom refs/myblobs/blob1 '%(raw:size)' fail -batch_test_atom refs/myblobs/blob2 '%(raw:size)' fail -batch_test_atom refs/mytrees/tree1 '%(raw:size)' fail +batch_test_atom refs/myblobs/blob1 '%(raw:size)' +batch_test_atom refs/myblobs/blob2 '%(raw:size)' +batch_test_atom refs/mytrees/tree1 '%(raw:size)' -batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail -batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail -batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' fail +batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' -batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' fail -batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' fail -batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' fail -batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' fail +batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' batch_test_atom refs/heads/main 'VALID' batch_test_atom refs/heads/main '%(INVALID)' fail batch_test_atom refs/heads/main '%(authordate:INVALID)' fail batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize) -%(raw)' fail +%(raw)' batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize) -%(raw)' fail +%(raw)' batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize) -%(raw)' fail +%(raw)' batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize) -%(raw)' fail +%(raw)' + +test_expect_success 'cat-file --batch equals to --batch-check with atoms' ' + git cat-file --batch-check="%(objectname) %(objecttype) %(objectsize) +%(raw)" >expected <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + git cat-file --batch >actual <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + cmp expected actual +' test_done From patchwork Fri Aug 13 08:22:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435109 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C0D7C432BE for ; Fri, 13 Aug 2021 08:23:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 66EA06056B for ; Fri, 13 Aug 2021 08:23:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238840AbhHMIYC (ORCPT ); Fri, 13 Aug 2021 04:24:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238621AbhHMIXs (ORCPT ); Fri, 13 Aug 2021 04:23:48 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DDC2C0617AF for ; Fri, 13 Aug 2021 01:23:22 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id l34-20020a05600c1d22b02902573c214807so9080134wms.2 for ; Fri, 13 Aug 2021 01:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=GGTmZhBe5osAznIE5fPt/loNnyl3SsJoiQDMpWQZxU0=; b=vHiHahJYDljiS0V5PNpxPU0zQePkrP4ISBIRTtxFB4CF7bf/6DyH25jr1PbEGoH1Ft ch2PYDFcqBza9gFkwanhYUO3Uj5kvNsIKLriZlA/JbzJXHRHt+GCEfy1y/l6B8uZ9zgl VJMu6tiqLm0ak4yUSzZklOXk5013fE6nsQwvLf01+x2+O/nS4Fg3ApZLVNz+Nrder8tb IPIPODceXowZeET2RPBO2jLjTQF6pqbkxKosYlsgdcHNY5Q7o7qskEosTeu9hfzPFNbW YZlY+XgTPBebECWjSUoF0ujd7aIIfqPQFTFl8XFMrv/4wytZPWEQI0XZ29hGJueLfJ6u iEVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=GGTmZhBe5osAznIE5fPt/loNnyl3SsJoiQDMpWQZxU0=; b=tlzyas0aqewtUKqIJJDM4yVEiMOSsCyGDaABUGnz4wDfXxK1MpJwf7dRmkZI0U5Pw0 kxdyE/OBIZRrml7vSuqj0zdHXiKhbEf4qC43Yn7hHy+3pAsiQwCbF2rxfYWIe/QBBRsW EqiBgGwPWB88jFZJfWxfavgrLOpLK7eBheZIKX/onMuthKCtFMjUhX1E9T81+HUnuPvn tBNJm72uPWgvuW6lXUYEHCiM5Mi2y67sGcG3iDaR8yI0Tbpb6OOL0lreuy/Osx+edImf nj2X3UqdHd3ohX3UfeAbvs8UxymDPnENeR8/oJF+fNpbWNexNxxcGmmOVqpu87Q1ov9B S4LA== X-Gm-Message-State: AOAM5305nOvrjIRfLge/kn3Ts9a3AWCjYAfXvR9ReOPxYZaUN78D1m4E nMB/xx829WvsWdaSkgXC2p8iShJpQqU= X-Google-Smtp-Source: ABdhPJznrAm2KoZW4MMog/F4EdxJiTYE+LmaHj6TsgPh5fAYEXuAz0KJaiYl7OCmn0GxGO/8AQsx3g== X-Received: by 2002:a05:600c:2147:: with SMTP id v7mr1406891wml.124.1628843000767; Fri, 13 Aug 2021 01:23:20 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z5sm678596wmp.26.2021.08.13.01.23.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:20 -0700 (PDT) Message-Id: <69493864d535941d31a41c145eb594fe38ea361d.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:57 +0000 Subject: [PATCH 14/27] [GSOC] cat-file: reuse err buf in batch_object_write() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Reuse the `err` buffer in batch_object_write(), as the buffer `scratch` does. This will reduce the overhead of multiple allocations of memory of the err buffer. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 306b4d49493..034094dc640 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -212,35 +212,36 @@ static void batch_write(struct batch_options *opt, const void *data, int len) static void batch_object_write(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { int ret; - struct strbuf err = STRBUF_INIT; struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); + strbuf_reset(err); - ret = format_ref_array_item(&item, &opt->format, scratch, &err); + ret = format_ref_array_item(&item, &opt->format, scratch, err); if (ret < 0) - die("%s\n", err.buf); + die("%s\n", err->buf); if (ret) { /* ret > 0 means when the object corresponding to oid * cannot be found in format_ref_array_item(), we only print * the error message. */ - printf("%s\n", err.buf); + printf("%s\n", err->buf); fflush(stdout); } else { strbuf_addch(scratch, '\n'); batch_write(opt, scratch->buf, scratch->len); } free_ref_array_item_value(&item); - strbuf_release(&err); } static void batch_one_object(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { @@ -294,7 +295,7 @@ static void batch_one_object(const char *obj_name, return; } - batch_object_write(obj_name, scratch, opt, data); + batch_object_write(obj_name, scratch, err, opt, data); } struct object_cb_data { @@ -302,13 +303,14 @@ struct object_cb_data { struct expand_data *expand; struct oidset *seen; struct strbuf *scratch; + struct strbuf *err; }; static int batch_object_cb(const struct object_id *oid, void *vdata) { struct object_cb_data *data = vdata; oidcpy(&data->expand->oid, oid); - batch_object_write(NULL, data->scratch, data->opt, data->expand); + batch_object_write(NULL, data->scratch, data->err, data->opt, data->expand); return 0; } @@ -364,6 +366,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; @@ -392,6 +395,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio cb.opt = batch; cb.expand = &data; cb.scratch = &output; + cb.err = &err; if (batch->unordered) { struct oidset seen = OIDSET_INIT; @@ -416,6 +420,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio strbuf_release(&format); strbuf_release(&output); free_global_resource(); + strbuf_release(&err); return 0; } @@ -444,13 +449,14 @@ static int batch_objects(struct batch_options *batch, const struct option *optio data.rest = p; } - batch_one_object(input.buf, &output, batch, &data); + batch_one_object(input.buf, &output, &err, batch, &data); } strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); free_global_resource(); + strbuf_release(&err); warn_on_object_refname_ambiguity = save_warning; return retval; } From patchwork Fri Aug 13 08:22:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB675C4320E for ; Fri, 13 Aug 2021 08:23:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C334A610CC for ; Fri, 13 Aug 2021 08:23:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238995AbhHMIYE (ORCPT ); Fri, 13 Aug 2021 04:24:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238638AbhHMIXt (ORCPT ); Fri, 13 Aug 2021 04:23:49 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4800C0613A3 for ; Fri, 13 Aug 2021 01:23:22 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id g138so6324086wmg.4 for ; Fri, 13 Aug 2021 01:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=fOADI/trlN6mNVmwFa7qg+nRyb7bve53nFF5H0UTQbo=; b=LpUEekryTfxCTWfVNEPDJoEaXm5LQRvM0q1K75Ko28ewH5iaU2OKXjD43GwStWmzKT UVTlkZW8309P/YotIeov4waE1PVRYHFhd6fYQNRyHWw7XKzcd7nioKtCLlP+O7qwYESH x/Noe1pK733vpVYBJys134RGAxrJA+BPpFs+WG1IXXws5y75Jy84tIJw6NbgZw1WGaER HvjK03As49gfeYUOqIOlHRepYCnNmlf89CNlEg8uX11Nh9w6/QN2vuxvxShK6iUPaHHr ikuxoelb3NWfGoktnFowIIfBtueAvNoTLaGl7DEWNfyYiCwZDlZhdu1mQrs4EhDmsCle ta4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=fOADI/trlN6mNVmwFa7qg+nRyb7bve53nFF5H0UTQbo=; b=qzU+M5gHZ/hxns18djIvYus09q1F1rBi2mVL4VNiuvuPvZpA7DEGku96Adh41EPFfV zwpSliokNOJKzc4OkiTGmkTmeWQCV/6KBS96d4l6fFYz0hKNnNKEYgZEzyRnq6hAwjTI LgEpPQyiGskZEv1paFf1Ankh4JRn9CZmuRuoki29wK3vnZccWVYkapo2H4jaUYMrKku/ R1D2J+mZouE7jJ/KiAUS1AQMUlbF1Tge5x1So10tO8QscVfM3hxtcIOUR10QWKZgG6LV /N9ExSWoQ5mzFi22mVse4/LkmAivvYRwXUqj8UBMne2SJnhHotnWIbEM+2IWZvoD4M2W anoQ== X-Gm-Message-State: AOAM531i26MbSDz6Kjpuc1txV8Kn2FXASDmbchjX0sLzFdwrdEpCOM/P BNt4dfTAYCbG9FvUKFnNPW2J0bUUcEQ= X-Google-Smtp-Source: ABdhPJxd7hZZi3YTUfZJUX6qWKwrfIcPdvEJ9WATivmL0SjrTiNdz1/zVCS4G7VPreGTjPjcJYmeUw== X-Received: by 2002:a1c:80c9:: with SMTP id b192mr1468430wmd.73.1628843001393; Fri, 13 Aug 2021 01:23:21 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p14sm705306wmi.42.2021.08.13.01.23.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:21 -0700 (PDT) Message-Id: <898e36a92b778a7ec9dfb4f7780dc2dcf95057c1.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:58 +0000 Subject: [PATCH 15/27] [GSOC] cat-file: re-implement --textconv, --filters options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu After cat-file reuses the ref-filter logic, we re-implement the functions of --textconv and --filters options. Add members `cat_file_cmdmode` to struct `ref_array_item`, so that struct `batch_option` member `cmdmode` will be passed to ref-filter, and then ref-filter will take use of it to filter the content of the object in get_object(). Use `actual_oi` to record the real expand_data: it may point to the original `oi` or the `act_oi` processed by `textconv_object()` or `convert_to_working_tree()`. `grab_values()` will grab the contents of `actual_oi` and `grab_common_values()` to grab the contents of origin `oi`, this ensures that `%(objectsize)` still uses the size of the unfiltered data. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 2 +- ref-filter.c | 37 +++++++++++++++++++++++++++++++++++-- ref-filter.h | 1 + 3 files changed, 37 insertions(+), 3 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 034094dc640..77b470687e1 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -217,7 +217,7 @@ static void batch_object_write(const char *obj_name, struct expand_data *data) { int ret; - struct ref_array_item item = { data->oid, data->rest }; + struct ref_array_item item = { data->oid, data->rest, opt->cmdmode }; strbuf_reset(scratch); strbuf_reset(err); diff --git a/ref-filter.c b/ref-filter.c index f83b6f83494..f7c448f6fa1 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1,3 +1,4 @@ +#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "cache.h" #include "parse-options.h" @@ -1759,6 +1760,9 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj { /* parse_object_buffer() will set eaten to 0 if free() will be needed */ int eaten = 1; + struct expand_data *actual_oi = oi; + struct expand_data act_oi = {0}; + if (oi->info.contentp) { /* We need to know that to use parse_object_buffer properly */ oi->info.sizep = &oi->size; @@ -1772,19 +1776,47 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj BUG("Object size is less than zero."); if (oi->info.contentp) { - *obj = parse_object_buffer(the_repository, &oi->oid, oi->type, oi->size, oi->content, &eaten); + if ((ref->cat_file_cmdmode == 'c' || ref->cat_file_cmdmode == 'w') && !ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&act_oi.oid)); + if (oi->type == OBJ_BLOB) { + if (ref->cat_file_cmdmode == 'c') { + act_oi = *oi; + if (textconv_object(the_repository, + ref->rest, 0100644, &act_oi.oid, + 1, (char **)(&act_oi.content), &act_oi.size)) + actual_oi = &act_oi; + } else if (ref->cat_file_cmdmode == 'w') { + struct strbuf strbuf = STRBUF_INIT; + struct checkout_metadata meta; + act_oi = *oi; + + init_checkout_metadata(&meta, NULL, NULL, &act_oi.oid); + if (!convert_to_working_tree(&the_index, ref->rest, act_oi.content, act_oi.size, &strbuf, &meta)) + die("could not convert '%s' %s", + oid_to_hex(&oi->oid), ref->rest); + act_oi.size = strbuf.len; + act_oi.content = strbuf_detach(&strbuf, NULL); + actual_oi = &act_oi; + } + } + *obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten); if (!*obj) { if (!eaten) + free(actual_oi->content); + if (actual_oi != oi) free(oi->content); return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi); + grab_values(ref->value, deref, *obj, actual_oi); } grab_common_values(ref->value, deref, oi); if (!eaten) free(oi->content); + if (actual_oi != oi) + free(actual_oi->content); return 0; } @@ -2193,6 +2225,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); ref->rest = NULL; + ref->cat_file_cmdmode = 0; return ref; } diff --git a/ref-filter.h b/ref-filter.h index ef3d89d73d4..853eb554a5b 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -39,6 +39,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; const char *rest; + int cat_file_cmdmode; int flag; unsigned int kind; const char *symref; From patchwork Fri Aug 13 08:22:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435113 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A92EC4338F for ; Fri, 13 Aug 2021 08:23:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 45516610A5 for ; Fri, 13 Aug 2021 08:23:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238886AbhHMIYF (ORCPT ); Fri, 13 Aug 2021 04:24:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238650AbhHMIXu (ORCPT ); Fri, 13 Aug 2021 04:23:50 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 420E7C0612E7 for ; Fri, 13 Aug 2021 01:23:23 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id o7-20020a05600c5107b0290257f956e02dso9083009wms.1 for ; Fri, 13 Aug 2021 01:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3drzlXPZkamKQqKZG3sXGWeLRwxrKrGClhbtE40WJZI=; b=m48vTzG/2ZAK737JlrjcrNchv7e+DvzdofjWP2v095mmFCZujdsQK85T+LOSa3Hz2K eoTRgqhFBrwp5LDfEI2rr0EHjTsfOmvuP7ZXz17UsdT5OvCzi/v5962noPtvkSJhGeTb 2330YRNGFVLByh1eZQq4zIILF/srflhFn/stAnQNdS6nY4ZvIJ+D5m8fEcqELrvnAkgD nOrZXWzNdXdoenQGnu0d2XceQ/ACRXmzJdQ3W9xjhZFdUu+90ZBTphfUJYhY39TSlPcc 52YUzVvIdCkKWr9DF5joT3t3RLL4dl43lImbSwxFmSzLKfD4FvPVMEKq1HJpUKEfXLFi OOcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3drzlXPZkamKQqKZG3sXGWeLRwxrKrGClhbtE40WJZI=; b=l5FvW3f6CLs1CmGL17zN0CidcJme8uztf6qlqhlrksiCG5ElXO31InO0YYqAQtmjJk IkbAgxcLJRmeAfuPZoEKaTwWQ+mo6MDJHhe6Ns7AHD5AhKt2I38xRb3Kt9aI3BWwwskp ZD0N4BYk/Dy+jjC7Wh5YszRBZlizNscaVxk8abJN2MDHzuLGy9b/P9s+/Xrt2wCR4Ikh WKFdCNimVwurWwxNaLetfxu8X2osCRj/vQIjo3C/LwoJR2iO2x+gNlCUJY2t0mCJkzG8 tJcoPOfXibz6D/t2skr3Ll6xgllXXvkB3ZI23xG7K14b7kqotCklHcOc0hJJLahFlejc TD3A== X-Gm-Message-State: AOAM531FoiwWqV+dOPOJu1hL2JkZgWxXA2KGTE1+HYFo7wtJMlPXVTgh 3zt7NuW5xxxLDVw5uSdDnQnoPSbKXTI= X-Google-Smtp-Source: ABdhPJzdJjhrKsW7wWSDhXV7Vd5lIJIthZqRLeVo5xAdaMCw68clVJBlse5lJqoNMnUz1Eq4nWfqxg== X-Received: by 2002:a1c:4b0a:: with SMTP id y10mr1374866wma.111.1628843001932; Fri, 13 Aug 2021 01:23:21 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u25sm682828wml.8.2021.08.13.01.23.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:21 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:22:59 +0000 Subject: [PATCH 16/27] [GSOC] ref-filter: remove grab_oid() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because "atom_type == ATOM_OBJECTNAME" implies the condition of `starts_with(name, "objectname")`, "atom_type == ATOM_TREE" implies the condition of `starts_with(name, "tree")`, so the check for `starts_with(name, field)` in grab_oid() is redundant. So Remove the grab_oid() from ref-filter, to reduce repeated check. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index f7c448f6fa1..8664d9a6f4e 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1081,16 +1081,6 @@ static const char *do_grab_oid(const char *field, const struct object_id *oid, } } -static int grab_oid(const char *name, const char *field, const struct object_id *oid, - struct atom_value *v, struct used_atom *atom) -{ - if (starts_with(name, field)) { - v->s = xstrdup(do_grab_oid(field, oid, atom)); - return 1; - } - return 0; -} - /* See grab_values */ static void grab_common_values(struct atom_value *val, int deref, struct expand_data *oi) { @@ -1116,8 +1106,9 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ } } else if (atom_type == ATOM_DELTABASE) v->s = xstrdup(oid_to_hex(&oi->delta_base_oid)); - else if (atom_type == ATOM_OBJECTNAME && deref) - grab_oid(name, "objectname", &oi->oid, v, &used_atom[i]); + else if (atom_type == ATOM_OBJECTNAME && deref) { + v->s = xstrdup(do_grab_oid("objectname", &oi->oid, &used_atom[i])); + } } } @@ -1158,9 +1149,10 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object continue; if (deref) name++; - if (atom_type == ATOM_TREE && - grab_oid(name, "tree", get_commit_tree_oid(commit), v, &used_atom[i])) + if (atom_type == ATOM_TREE) { + v->s = xstrdup(do_grab_oid("tree", get_commit_tree_oid(commit), &used_atom[i])); continue; + } if (atom_type == ATOM_NUMPARENT) { v->value = commit_list_count(commit->parents); v->s = xstrfmt("%lu", (unsigned long)v->value); @@ -1965,9 +1957,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->s = xstrdup(buf + 1); } continue; - } else if (!deref && atom_type == ATOM_OBJECTNAME && - grab_oid(name, "objectname", &ref->objectname, v, atom)) { - continue; + } else if (!deref && atom_type == ATOM_OBJECTNAME) { + v->s = xstrdup(do_grab_oid("objectname", &ref->objectname, atom)); + continue; } else if (atom_type == ATOM_HEAD) { if (atom->u.head && !strcmp(ref->refname, atom->u.head)) v->s = xstrdup("*"); From patchwork Fri Aug 13 08:23:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435115 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4E43C4320A for ; Fri, 13 Aug 2021 08:23:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C0B1A6056B for ; Fri, 13 Aug 2021 08:23:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239075AbhHMIYH (ORCPT ); Fri, 13 Aug 2021 04:24:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238671AbhHMIXu (ORCPT ); Fri, 13 Aug 2021 04:23:50 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DABB6C061292 for ; Fri, 13 Aug 2021 01:23:23 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id h13so12174501wrp.1 for ; Fri, 13 Aug 2021 01:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ejOQzbNvR0GgKnTzUw0qecz84LDsr6+RdcltC+eMp0A=; b=Gx1aYjHK4nm84V3XsCca9qmIsz0MoUNbTN6j0TIr+TbLyhkhi4ZoI2B1LcqgsPEDuu THv66+6YtF1AUtMjxxDzKlSY5cj35f/Hbknp+wOLSXj3hPDBLrttu8byFPnGlv1WXptI +AS2Rby+QUGacImOo/gCZIvDM0KRxpW93Pi0ZpxS1LaAEYq0Qinu8ZUo736Tsvupc910 4ZTqR1w4Wyr5yve/m+DQ/ei1vD+Yik4dkPrvHSD+Ka3CCK0BRKbj1t3KGChqsbSXMMRn JSRC3sSznsquw20lLvn/wWXO4xswlfTdYlmBIw2XCxhSeQr2YvTcMgIqZ0+tm/1AvVf5 2fVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ejOQzbNvR0GgKnTzUw0qecz84LDsr6+RdcltC+eMp0A=; b=eSaMlwa6MlLZyIuhf51Rykpl+z9lu1199Ivtpb5RzGdljf0Tx1pqTf9mYcixTkkFCo zA3sKfGwUp0ycxLTM0Z4Nrtk9PTDeZJrXZSrag1z7wzpYsyyWOedvR1KtOLTyKZfUl6N 46r/dQgB1We/pIKSa4S/m7wopB1Lwvf+S1cHmt4Dq9vVoi7QSIOD0mRUdc4lVlxSATFQ H9hSpi+DMLhFbYohYHSqer6UvAp+gEhgBN4sMCaZ/UGtGUGNsRIYjKMVG4ZFLazWL5QE 8DIWWUvrzpTn0BgZYByJKjJiwB4xuUoVVnx6KvE4IbpsbJfWPYZDiRg/3VYJAot9OVhK pCiA== X-Gm-Message-State: AOAM530th+yvQDe4rJIdvtZkS9OtyMWfRR9U7RPNOt1Ne5v6cZnGAuW2 FshBWIjE/Oa1HcvPkjHxzIkUmpj3EFc= X-Google-Smtp-Source: ABdhPJwQj+EchRBRbUYAhX89NKFsfd4p28XJh53a3hPezSdfxnsUjCwUlC5DNemXkn0QylB5Z7RcVQ== X-Received: by 2002:a5d:668f:: with SMTP id l15mr1694858wru.390.1628843002518; Fri, 13 Aug 2021 01:23:22 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t8sm762525wmj.5.2021.08.13.01.23.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:22 -0700 (PDT) Message-Id: <2c6ce95c8278710b43bf9b7a2fe09732a3172b09.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:00 +0000 Subject: [PATCH 17/27] [GSOC] ref-filter: performance optimization by skip parse_object_buffer Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu When we are using some atoms such as %(raw), %(objectname), %(objecttype), we don't need to parse the content of the object, so we can skip parse_object_buffer() for performance optimization. It is worth noting that in these cases, we still need to call parse_object_buffer() for parsing: 1. The atom type is one of %(tag), %(type), %(object), %(tree), %(numparent) or %(parent). 2. The type of the object is tag and the atom need to be dereferenced e.g. %(*objecttype). With this patch, `git cat-file --batch` has a 12.5% performance improvement. But there is still a certain gap with `git cat-file --batch` before using the logic of ref-filter. `git cat-file --batch` is 10% worse, `git cat-file --batch-check` is 30% worse. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 53 ++++++++++++++++++++++++++++++++++++++-------------- ref-filter.h | 5 +++-- 2 files changed, 42 insertions(+), 16 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 8664d9a6f4e..1251e062ff8 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1025,6 +1025,20 @@ static int reject_atom(int cat_file_mode, enum atom_type atom_type) } } +static int need_parse_buffer(enum atom_type atom_type) { + switch (atom_type) { + case ATOM_TAG: + case ATOM_TYPE: + case ATOM_OBJECT: + case ATOM_TREE: + case ATOM_NUMPARENT: + case ATOM_PARENT: + return 1; + default: + return 0; + } +} + /* * Make sure the format string is well formed, and parse out * the used atoms. @@ -1045,6 +1059,8 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (need_parse_buffer(used_atom[at].atom_type)) + format->can_skip_parse_buffer = 0; if (reject_atom(format->cat_file_mode, used_atom[at].atom_type)) die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); @@ -1532,14 +1548,16 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s { void *buf = data->content; - switch (obj->type) { + switch (data->type) { case OBJ_TAG: - grab_tag_values(val, deref, obj); + if (obj) + grab_tag_values(val, deref, obj); grab_sub_body_contents(val, deref, data); grab_person("tagger", val, deref, buf); break; case OBJ_COMMIT: - grab_commit_values(val, deref, obj); + if (obj) + grab_commit_values(val, deref, obj); grab_sub_body_contents(val, deref, data); grab_person("author", val, deref, buf); grab_person("committer", val, deref, buf); @@ -1792,16 +1810,23 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj actual_oi = &act_oi; } } - *obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten); - if (!*obj) { - if (!eaten) - free(actual_oi->content); - if (actual_oi != oi) - free(oi->content); - return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), - oid_to_hex(&oi->oid), ref->refname); + if (ref->can_skip_parse_buffer && + ((!deref && + (!need_tagged || oi->type != OBJ_TAG)) || + deref)) { + grab_values(ref->value, deref, NULL, actual_oi); + } else { + *obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten); + if (!*obj) { + if (!eaten) + free(actual_oi->content); + if (actual_oi != oi) + free(oi->content); + return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), + oid_to_hex(&oi->oid), ref->refname); + } + grab_values(ref->value, deref, *obj, actual_oi); } - grab_values(ref->value, deref, *obj, actual_oi); } grab_common_values(ref->value, deref, oi); @@ -2029,7 +2054,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) * If there is no atom that wants to know about tagged * object, we are done. */ - if (!need_tagged || (obj->type != OBJ_TAG)) + if (!need_tagged || (oi.type != OBJ_TAG)) return 0; /* @@ -2651,7 +2676,7 @@ int format_ref_array_item(struct ref_array_item *info, state.quote_style = format->quote_style; push_stack_element(&state.stack); - + info->can_skip_parse_buffer = format->can_skip_parse_buffer; for (cp = format->format; *cp && (sp = find_next(cp)); cp = ep + 1) { struct atom_value *atomv; int pos; diff --git a/ref-filter.h b/ref-filter.h index 853eb554a5b..8cae9f576d5 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -41,6 +41,7 @@ struct ref_array_item { const char *rest; int cat_file_cmdmode; int flag; + int can_skip_parse_buffer; unsigned int kind; const char *symref; struct commit *commit; @@ -83,12 +84,12 @@ struct ref_format { int quote_style; int use_rest; int use_color; - + int can_skip_parse_buffer; /* Internal state to ref-filter */ int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { .use_color = -1 } +#define REF_FORMAT_INIT { .use_color = -1, .can_skip_parse_buffer = 1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ From patchwork Fri Aug 13 08:23:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5ECFC432BE for ; Fri, 13 Aug 2021 08:23:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CE6DF6109E for ; Fri, 13 Aug 2021 08:23:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239113AbhHMIYI (ORCPT ); Fri, 13 Aug 2021 04:24:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238157AbhHMIXv (ORCPT ); Fri, 13 Aug 2021 04:23:51 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E5A7C0612A3 for ; Fri, 13 Aug 2021 01:23:24 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id l7-20020a1c2507000000b002e6be5d86b3so2933132wml.3 for ; Fri, 13 Aug 2021 01:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sEKGboegygc9+Zf9rgcLf6+DcBSR9uC6Y122z9iM3lY=; b=UGnEoSCMdwOwwcsyZKvS0u8BWX7Bis93kZ1DYodywHBX8axWZLps1Hq403/cZNJSfD rZ5v0Khf2ht1JlYQVfiYJpXY4ONmGHTo88ZsxBXTghLil6EKnUcvyKA+3qHllGMLHGOg us17tR8bcSY4IT13TZFn8fBoOgbq7nyr9caHXDpXofNZpzdDrpuewdsItAxdBQrusxoy KdBzN6pCn3m1zOMLSp4D3nXNDFXR4bdlhDHPUIsCia8V8EN+ifkoREX3w+/BHqgB9adf yTcm/2oQ13qRo9FMofRE761oODr5xSBKsTi+r8kZRovECh7Q0LSybto0B6TW4fqi2Dtj 9FLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sEKGboegygc9+Zf9rgcLf6+DcBSR9uC6Y122z9iM3lY=; b=OGFOWvfG0mRdfZbun4wEizjWZmct0sWirHnJb/+EKM+y+sBXITPbYRw1l4HejxlMBt zRGMdXi3AWBF0DpsklnVenNQlflFVZH77p/qalTo87s0JVC8pB43BLG+oDrTYUbwA04y rPlsGrlejtQD1aWBojq/qC4kHbPVXWAQBi26EfOMCAvu33AUmYIuyIWLQ+V1Be2Y+Sx7 WSHeARoTAqcX94L59eXz8BaOVyueXH5X6O016ARp9M9kqyErjGMnqxCPH9nWpqA5xalT w2I3Wb1Lig9njktUpIVdLQLb/StwP6xN9X7ixZ1/bq114kMqF+d8o8u/m63zaqc3lOf+ OkoQ== X-Gm-Message-State: AOAM5317RHN6z3kNqo/qLVzdLruFDDFiwmKVil2e942DfhWAUUZ6Yr5V X+I1Zkk+T8R4tKP7nFArZR1XQ7rrvcs= X-Google-Smtp-Source: ABdhPJzURhI3hy6hZqLQSjaK9kUVUrC5B1GD4oxSKDEDxxf5rrkSbWxNdv5Ry+StJhb0J9jR3+BIaw== X-Received: by 2002:a05:600c:1c9c:: with SMTP id k28mr1512169wms.148.1628843003106; Fri, 13 Aug 2021 01:23:23 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g6sm800757wrm.73.2021.08.13.01.23.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:22 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:01 +0000 Subject: [PATCH 18/27] [GSOC] ref-filter: use atom_type and merge two for loop in grab_person Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu grab_person() uses string for atom type comparison, this is very inefficient, we can use atom_type to replace it. At the same time, we can merge the two for loops in grab_person() into one, this can reduce unnecessary traversal of the used_atom list. With this patch, `git cat-file --batch` has a 2% performance improvement. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 57 +++++++++++++++++----------------------------------- 1 file changed, 18 insertions(+), 39 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 1251e062ff8..3ccb531e073 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1307,63 +1307,42 @@ static void grab_date(const char *buf, struct atom_value *v, const char *atomnam } /* See grab_values */ -static void grab_person(const char *who, struct atom_value *val, int deref, void *buf) +static void grab_person(enum atom_type type, struct atom_value *val, int deref, void *buf) { int i; + const char *who = valid_atom[type].name; int wholen = strlen(who); const char *wholine = NULL; for (i = 0; i < used_atom_cnt; i++) { const char *name = used_atom[i].name; + enum atom_type atom_type = used_atom[i].atom_type; struct atom_value *v = &val[i]; if (!!deref != (*name == '*')) continue; if (deref) name++; - if (strncmp(who, name, wholen)) - continue; - if (name[wholen] != 0 && - strcmp(name + wholen, "name") && - !starts_with(name + wholen, "email") && - !starts_with(name + wholen, "date")) + if ((atom_type < type || atom_type > type + 3) && + /* + * For a tag or a commit object, if "creator" or "creatordate" is + * requested, do something special. + */ + ((atom_type != ATOM_CREATOR && atom_type != ATOM_CREATORDATE) || + ((atom_type == ATOM_CREATOR || atom_type == ATOM_CREATORDATE) && + type != ATOM_TAGGER && type != ATOM_COMMITTER))) continue; if (!wholine) wholine = find_wholine(who, wholen, buf); if (!wholine) return; /* no point looking for it */ - if (name[wholen] == 0) + if (atom_type == type || atom_type == ATOM_CREATOR) v->s = copy_line(wholine); - else if (!strcmp(name + wholen, "name")) + else if (atom_type == type + 1) v->s = copy_name(wholine); - else if (starts_with(name + wholen, "email")) + else if (atom_type == type + 2) v->s = copy_email(wholine, &used_atom[i]); - else if (starts_with(name + wholen, "date")) - grab_date(wholine, v, name); - } - - /* - * For a tag or a commit object, if "creator" or "creatordate" is - * requested, do something special. - */ - if (strcmp(who, "tagger") && strcmp(who, "committer")) - return; /* "author" for commit object is not wanted */ - if (!wholine) - wholine = find_wholine(who, wholen, buf); - if (!wholine) - return; - for (i = 0; i < used_atom_cnt; i++) { - const char *name = used_atom[i].name; - enum atom_type atom_type = used_atom[i].atom_type; - struct atom_value *v = &val[i]; - if (!!deref != (*name == '*')) - continue; - if (deref) - name++; - - if (atom_type == ATOM_CREATORDATE) + else if (atom_type == type + 3 || atom_type == ATOM_CREATORDATE) grab_date(wholine, v, name); - else if (atom_type == ATOM_CREATOR) - v->s = copy_line(wholine); } } @@ -1553,14 +1532,14 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s if (obj) grab_tag_values(val, deref, obj); grab_sub_body_contents(val, deref, data); - grab_person("tagger", val, deref, buf); + grab_person(ATOM_TAGGER, val, deref, buf); break; case OBJ_COMMIT: if (obj) grab_commit_values(val, deref, obj); grab_sub_body_contents(val, deref, data); - grab_person("author", val, deref, buf); - grab_person("committer", val, deref, buf); + grab_person(ATOM_AUTHOR, val, deref, buf); + grab_person(ATOM_COMMITTER, val, deref, buf); break; case OBJ_TREE: /* grab_tree_values(val, deref, obj, buf, sz); */ From patchwork Fri Aug 13 08:23:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E6A1C4320E for ; Fri, 13 Aug 2021 08:23:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 22CAF6109E for ; Fri, 13 Aug 2021 08:23:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239202AbhHMIYJ (ORCPT ); Fri, 13 Aug 2021 04:24:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238624AbhHMIXy (ORCPT ); Fri, 13 Aug 2021 04:23:54 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D859C061756 for ; Fri, 13 Aug 2021 01:23:25 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id h13so12174563wrp.1 for ; Fri, 13 Aug 2021 01:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=R5l2xbFARzrNIJu+PtNYP7lBcR3O4QNQ1MhipoiwXSg=; b=QGAf29UYRrMcYQSgryi+mdmFB7OncLLAwQRIkSGA35kMvBMoLpm9gATCso06I3VF6n nCN82KTNUOYWnkHU1k0zGZfLInkiXVChPQavcvScCVYnOFOf7+UPLClMDKnwEC7GAtrx x/HpLkPIuXG2GQ+zR3c7iVY/X9n/wp4L+/EicZJtZrDYsoRoiLKbsYi69q0Grb2Dlyk/ MY9fiM1EFag0xL9QHLyIKht52Ayl21PMCRhdBLe4K7ontiIGSzIjxhkJsmdev8yPQttL NpEYkjJXw3ElKVOAlxaLD8pIqzQJ+AuUaqV+UNJWjXauFBj0U/j6cU369VCSR5LmgZ2j 4YOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=R5l2xbFARzrNIJu+PtNYP7lBcR3O4QNQ1MhipoiwXSg=; b=ccnt++69nqzW/YZvJV3Sz50UISsXt7NXVYUxTQAqljs/xU/HP9xo4KLBAvkt/4x2ZL /piNNBiDrRtob4MxsBLRFbEYUzfWm387A3j5D0gf7hPNpKr0oMOxFQ7HVQYs9Nh8BIiv w2xF4AQ/tpMRHl8H1jHAf6NZ2FGcEdvRKQJBc13X7JJsmliXkro0Fq0nuEiKCm8OUJDM xmgscBgUEkBXMhjKOAWAnIZmfwq13Z5ByrISBok1uEItLd0ipiTyGBpDJfWUpkcrbhJy dvrPI1EMxKD07gbxSxj32tqGH+lhnTmzarCRliAQuAn8o8+W2Qr63n87M+mL+Fwe7Ti/ a91Q== X-Gm-Message-State: AOAM531RhyBxzQodgJ6RHw/nXlNAJtxFcLZQW2+t9wm9Qkh0xoXIQIbO hySFnZaVq/KKSHYIokEI/lQBaLCKEwI= X-Google-Smtp-Source: ABdhPJz9cMfvCyeV3LhtTY3zmir+nXZVbOTAjDlUvkBzpwHyF06qEwN+v0HzE5cqxHFcYjnzitZfkQ== X-Received: by 2002:a5d:6090:: with SMTP id w16mr1694429wrt.38.1628843003742; Fri, 13 Aug 2021 01:23:23 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t1sm741939wma.25.2021.08.13.01.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:23 -0700 (PDT) Message-Id: <20cbc3645f14a2090195abeb880cca1541aa1206.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:02 +0000 Subject: [PATCH 19/27] [GSOC] ref-filter: remove strlen from find_subpos Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu find_subpos() needs to use strlen() to get the length of the object content, but in the caller of find_subpos(): grab_sub_body_contents(), we can already get the length of the object content through data->size. So add a `size_t buf_size` in find_subpos(), and pass data->size to it. This can lead to weak performance optimizations. Signed-off-by: ZheNing Hu Mentored-by: Christian Couder Mentored-by: Hariom Verma --- ref-filter.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 3ccb531e073..be4c4405b12 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1346,7 +1346,7 @@ static void grab_person(enum atom_type type, struct atom_value *val, int deref, } } -static void find_subpos(const char *buf, +static void find_subpos(const char *buf, size_t buf_size, const char **sub, size_t *sublen, const char **body, size_t *bodylen, size_t *nonsiglen, @@ -1354,12 +1354,12 @@ static void find_subpos(const char *buf, { struct strbuf payload = STRBUF_INIT; struct strbuf signature = STRBUF_INIT; + const char *begin = buf; const char *eol; - const char *end = buf + strlen(buf); const char *sigstart; /* parse signature first; we might not even have a subject line */ - parse_signature(buf, end - buf, &payload, &signature); + parse_signature(buf, buf_size, &payload, &signature); /* skip past header until we hit empty line */ while (*buf && *buf != '\n') { @@ -1372,7 +1372,7 @@ static void find_subpos(const char *buf, while (*buf == '\n') buf++; *sig = strbuf_detach(&signature, siglen); - sigstart = buf + parse_signed_buffer(buf, strlen(buf)); + sigstart = buf + parse_signed_buffer(buf, buf_size - (buf - begin)); /* subject is first non-empty line */ *sub = buf; @@ -1395,7 +1395,7 @@ static void find_subpos(const char *buf, while (*buf == '\n' || *buf == '\r') buf++; *body = buf; - *bodylen = strlen(buf); + *bodylen = buf_size - (buf - begin); *nonsiglen = sigstart - buf; } @@ -1462,7 +1462,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp !starts_with(name, "contents"))) continue; if (!subpos) - find_subpos(buf, + find_subpos(buf, data->size, &subpos, &sublen, &bodypos, &bodylen, &nonsiglen, &sigpos, &siglen); From patchwork Fri Aug 13 08:23:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 472C5C43216 for ; Fri, 13 Aug 2021 08:23:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 32452610CC for ; Fri, 13 Aug 2021 08:23:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238865AbhHMIYK (ORCPT ); Fri, 13 Aug 2021 04:24:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238790AbhHMIXy (ORCPT ); Fri, 13 Aug 2021 04:23:54 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1E74C0617AF for ; Fri, 13 Aug 2021 01:23:25 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id r6so12168808wrt.4 for ; Fri, 13 Aug 2021 01:23:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RVBdyjgdTjkcwxA3c9OBtKd0HvGOq5DiNb8Kn1yXPqc=; b=LqombtknxBINtDR6PAGYeTbfg2lpTTo+nEOCyWFOJZ3f9BlgBLeXqxkClN2niwv4qN vwyEsUzloWfjPsVdFV3DFV17lBuXYdZtdb06Q915nybNDVWhbZDUZ3AtAWudvoKK94Jg xHP2WZr2yEVkMCeQ6bNtM/2Td1rCxWIOrdDcvaVnq2PTRhKQBH5/WoqYj8HsAEwFImcF IzcdhqfqWv+AJOtCNmQNA8HiPxdx69ZaLqoQCFp7jQC+aDUQNm6hpvYURoKkH/knEYuD XI41+CrUDL7fdbxO5rLBopNudbLzZDtPMB14FQSl38XM+AXO+2Xay4JbO4PsmkkzfU82 0WdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RVBdyjgdTjkcwxA3c9OBtKd0HvGOq5DiNb8Kn1yXPqc=; b=b6bFhfpvZrBzCWKK/M/KED0A7jGUG2rwgKrmtFonwG/InlpceLfFmjvTC29xi4O5u5 qaUm+noUlVbbquT34CCGdKQBixIOjiEAZD9s00jfbKp1VWxDtNjD4IqSDpEd0RoxJd9u 2DB/CXG4Wcw1ciRjcwSOdgPsBuU41KNtqiytZXwuPiqJvBKceJDlFYc0ThpsYsIRyfsX EhCqzreNHq1oA8KqLGxHC5jeCxt0sIIEDmbmmkdz+1Wf80u/1NE7DMf7taFQHfDzrRo4 EmmQBRixCWa2+R48oFOH9+zNApqFWrvoxxTlK8BlA3P4BUGuEIkzjAbZ9MNiDZo8rhxF XI2g== X-Gm-Message-State: AOAM533RWxqXitcFDjLQzbKrw/X3PeIAzckUvTdK7as+BES6LF77YCJu ctysy7w7ZRcoUmyk6h3EGjSv3uvaXyI= X-Google-Smtp-Source: ABdhPJyFMM/jyt+XpLxZLCsSl9jRA8tzc5roLHw42ilCPWFValrHaXvvMJYcYqGcY4CQcCk1H+gphg== X-Received: by 2002:a5d:5906:: with SMTP id v6mr1674992wrd.194.1628843004264; Fri, 13 Aug 2021 01:23:24 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e10sm811667wrt.82.2021.08.13.01.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:24 -0700 (PDT) Message-Id: <5afac0d412869d68f8e5a231e3967ab7d62e8707.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:03 +0000 Subject: [PATCH 20/27] [GSOC] ref-filter: introducing xstrvfmt_len() and xstrfmt_len() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu xstrfmt(), xstrvfmt(), they are functions used to add formatt strings to a dynamic memory. Because their implementation uses strbuf to store its buffer and its length, so we can easily pass its length to the caller, so create xstrvfmt_len() and xstrfmt_len() which can fetch the length of the buffer through the parameter `len`. This can reduce the cost of calculating the length of the string later through strlen(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 50 +++++++++++++++++++++++++------------------------- strbuf.c | 21 +++++++++++++++++++++ strbuf.h | 6 ++++++ 3 files changed, 52 insertions(+), 25 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index be4c4405b12..6e904389d6b 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1115,10 +1115,10 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ else if (atom_type == ATOM_OBJECTSIZE) { if (used_atom[i].u.objectsize.option == O_SIZE_DISK) { v->value = oi->disk_size; - v->s = xstrfmt("%"PRIuMAX, (uintmax_t)oi->disk_size); + v->s = xstrfmt_len(&v->s_size, "%"PRIuMAX, (uintmax_t)oi->disk_size); } else if (used_atom[i].u.objectsize.option == O_SIZE) { v->value = oi->size; - v->s = xstrfmt("%"PRIuMAX , (uintmax_t)oi->size); + v->s = xstrfmt_len(&v->s_size, "%"PRIuMAX , (uintmax_t)oi->size); } } else if (atom_type == ATOM_DELTABASE) v->s = xstrdup(oid_to_hex(&oi->delta_base_oid)); @@ -1171,7 +1171,7 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object } if (atom_type == ATOM_NUMPARENT) { v->value = commit_list_count(commit->parents); - v->s = xstrfmt("%lu", (unsigned long)v->value); + v->s = xstrfmt_len(&v->s_size, "%lu", (unsigned long)v->value); } else if (atom_type == ATOM_PARENT) { struct commit_list *parents; @@ -1449,7 +1449,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp v->s = xmemdupz(buf, buf_size); v->s_size = buf_size; } else if (atom->u.raw_data.option == RAW_LENGTH) { - v->s = xstrfmt("%"PRIuMAX, (uintmax_t)buf_size); + v->s = xstrfmt_len(&v->s_size, "%"PRIuMAX, (uintmax_t)buf_size); } continue; } @@ -1476,7 +1476,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp } else if (atom->u.contents.option == C_BODY_DEP) v->s = xmemdupz(bodypos, bodylen); else if (atom->u.contents.option == C_LENGTH) - v->s = xstrfmt("%"PRIuMAX, (uintmax_t)strlen(subpos)); + v->s = xstrfmt_len(&v->s_size, "%"PRIuMAX, (uintmax_t)strlen(subpos)); else if (atom->u.contents.option == C_BODY) v->s = xmemdupz(bodypos, nonsiglen); else if (atom->u.contents.option == C_SIG) @@ -1645,56 +1645,56 @@ static const char *show_ref(struct refname_atom *atom, const char *refname) } static void fill_remote_ref_details(struct used_atom *atom, const char *refname, - struct branch *branch, const char **s) + struct branch *branch, struct atom_value *v) { int num_ours, num_theirs; if (atom->u.remote_ref.option == RR_REF) - *s = show_ref(&atom->u.remote_ref.refname, refname); + v->s = show_ref(&atom->u.remote_ref.refname, refname); else if (atom->u.remote_ref.option == RR_TRACK) { if (stat_tracking_info(branch, &num_ours, &num_theirs, NULL, atom->u.remote_ref.push, AHEAD_BEHIND_FULL) < 0) { - *s = xstrdup(msgs.gone); + v->s = xstrdup(msgs.gone); } else if (!num_ours && !num_theirs) - *s = xstrdup(""); + v->s = xstrdup(""); else if (!num_ours) - *s = xstrfmt(msgs.behind, num_theirs); + v->s = xstrfmt_len(&v->s_size, msgs.behind, num_theirs); else if (!num_theirs) - *s = xstrfmt(msgs.ahead, num_ours); + v->s = xstrfmt_len(&v->s_size, msgs.ahead, num_ours); else - *s = xstrfmt(msgs.ahead_behind, + v->s= xstrfmt_len(&v->s_size, msgs.ahead_behind, num_ours, num_theirs); - if (!atom->u.remote_ref.nobracket && *s[0]) { - const char *to_free = *s; - *s = xstrfmt("[%s]", *s); + if (!atom->u.remote_ref.nobracket && v->s[0]) { + const char *to_free = v->s; + v->s = xstrfmt_len(&v->s_size, "[%s]", v->s); free((void *)to_free); } } else if (atom->u.remote_ref.option == RR_TRACKSHORT) { if (stat_tracking_info(branch, &num_ours, &num_theirs, NULL, atom->u.remote_ref.push, AHEAD_BEHIND_FULL) < 0) { - *s = xstrdup(""); + v->s = xstrdup(""); return; } if (!num_ours && !num_theirs) - *s = xstrdup("="); + v->s = xstrdup("="); else if (!num_ours) - *s = xstrdup("<"); + v->s = xstrdup("<"); else if (!num_theirs) - *s = xstrdup(">"); + v->s = xstrdup(">"); else - *s = xstrdup("<>"); + v->s = xstrdup("<>"); } else if (atom->u.remote_ref.option == RR_REMOTE_NAME) { int explicit; const char *remote = atom->u.remote_ref.push ? pushremote_for_branch(branch, &explicit) : remote_for_branch(branch, &explicit); - *s = xstrdup(explicit ? remote : ""); + v->s = xstrdup(explicit ? remote : ""); } else if (atom->u.remote_ref.option == RR_REMOTE_REF) { const char *merge; merge = remote_ref_for_branch(branch, atom->u.remote_ref.push); - *s = xstrdup(merge ? merge : ""); + v->s = xstrdup(merge ? merge : ""); } else BUG("unhandled RR_* enum"); } @@ -1922,7 +1922,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) refname = branch_get_upstream(branch, NULL); if (refname) - fill_remote_ref_details(atom, refname, branch, &v->s); + fill_remote_ref_details(atom, refname, branch, v); else v->s = xstrdup(""); continue; @@ -1943,7 +1943,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) } /* We will definitely re-init v->s on the next line. */ free((char *)v->s); - fill_remote_ref_details(atom, refname, branch, &v->s); + fill_remote_ref_details(atom, refname, branch, v); continue; } else if (atom_type == ATOM_COLOR) { v->s = xstrdup(atom->u.color); @@ -2006,7 +2006,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) if (!deref) v->s = xstrdup(refname); else - v->s = xstrfmt("%s^{}", refname); + v->s = xstrfmt_len(&v->s_size, "%s^{}", refname); free((char *)refname); } diff --git a/strbuf.c b/strbuf.c index c8a5789694c..8358d9c3f86 100644 --- a/strbuf.c +++ b/strbuf.c @@ -982,6 +982,15 @@ char *xstrvfmt(const char *fmt, va_list ap) return strbuf_detach(&buf, NULL); } +char *xstrvfmt_len(ssize_t *len, const char *fmt, va_list ap) +{ + struct strbuf buf = STRBUF_INIT; + strbuf_vaddf(&buf, fmt, ap); + if (len) + *len = buf.len; + return strbuf_detach(&buf, NULL); +} + char *xstrfmt(const char *fmt, ...) { va_list ap; @@ -994,6 +1003,18 @@ char *xstrfmt(const char *fmt, ...) return ret; } +char *xstrfmt_len(ssize_t *len, const char *fmt, ...) +{ + va_list ap; + char *ret; + + va_start(ap, fmt); + ret = xstrvfmt_len(len ,fmt, ap); + va_end(ap); + + return ret; +} + void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm, int tz_offset, int suppress_tz_name) { diff --git a/strbuf.h b/strbuf.h index 5b1113abf8f..5211e0b714f 100644 --- a/strbuf.h +++ b/strbuf.h @@ -738,4 +738,10 @@ char *xstrvfmt(const char *fmt, va_list ap); __attribute__((format (printf, 1, 2))) char *xstrfmt(const char *fmt, ...); + +__attribute__((format (printf, 2, 0))) +char *xstrvfmt_len(ssize_t *len, const char *fmt, va_list ap); +__attribute__((format (printf, 2, 3))) +char *xstrfmt_len(ssize_t *len, const char *fmt, ...); + #endif /* STRBUF_H */ From patchwork Fri Aug 13 08:23:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05B60C4338F for ; Fri, 13 Aug 2021 08:23:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E6AF4610CC for ; Fri, 13 Aug 2021 08:23:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238535AbhHMIYL (ORCPT ); Fri, 13 Aug 2021 04:24:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238817AbhHMIXy (ORCPT ); Fri, 13 Aug 2021 04:23:54 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F77BC0612A9 for ; Fri, 13 Aug 2021 01:23:26 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id h24-20020a1ccc180000b029022e0571d1a0so6365080wmb.5 for ; Fri, 13 Aug 2021 01:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QSepH9LUA1lxdb6GQVSFkbpmiFYQKBWi3we7F+hr07Q=; b=qgENPGWN/TZIq4Y3YDQnmmsk4TUdquOZSNaCOCE0STaSwxQBX6XlsFcqLxwwpkY9Nq orrHCfNz5Y9JidReBMnGBZ9m1jPD3l77h6T8KkgNcWo2qJcw/HDvSdLF0jhRa3/K6Yco RBPgs7uTsU+7QhwA21q1UcOcuK3wy+nabMmc/acWQsj4yJCyFaGomHEUE4Z/qSYgdh/K SVrhDgOabuldnQeq4ywetJ4ClAGSjZpLHrOMiFbiHo2egWWCGAmpFjgE8GNEEaiGsIOf xXtfuDhq5KHFAhZMzhfRGskX0oyb907fzJ9LPF4hJrQfIK4iqxGcb0LD07Yq63xdPLtR TDlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QSepH9LUA1lxdb6GQVSFkbpmiFYQKBWi3we7F+hr07Q=; b=dTpCZqwZdQW03DwpiFOB6rsCScAl8LUo32bGJU1La6SoUbwpDAXjJgetlKXCfegtFA OfdCBJo6ENtQ0s7jF9wQWa4jgrBqQ9ozM35pXFLlhk6h3Sipcm8YhQhmso3RKw9E6l8n r1DkALapf+ovx2rdcbSv2KmgpMLMD1Nlz70hvwr8pDMZoahJB6JqxGbNA2PaBzELc8X0 tGeFHeJ1ChG/PWQXJ4aobiYzHrCvQ6CHe6Sj6ZVbPqS57DYhL29CcjIFNUhpf8Ho8CvQ xbtv0vkba/RptScmhmxBjPQk5Ic4GrmCj0ylcf27CQrVb7jv2M5BemKnhsjxmh6adof1 NZJw== X-Gm-Message-State: AOAM533VawnO3bZCKBoGldTGQjPC1zXu+vW+qLGH80uv2zRd1uAR+FFV +U+yp654j1zkgPufzSyq3aczZwIEiwY= X-Google-Smtp-Source: ABdhPJyIrv45TnZoaKKot21QdCOsNgR3IVSY93ZvWe6qY8k1cgMCqtjgMOKPL67wVBI6bJQqVjyytA== X-Received: by 2002:a05:600c:225a:: with SMTP id a26mr1446739wmm.189.1628843004917; Fri, 13 Aug 2021 01:23:24 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c10sm738301wml.44.2021.08.13.01.23.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:24 -0700 (PDT) Message-Id: <31acac9fde7d7239303255ccee2a3f066b2b5f6b.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:04 +0000 Subject: [PATCH 21/27] [GSOC] ref-filter: remove second parsing in format_ref_array_item Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu We parsed format string in verify_ref_format() and stored the parsed atom in used_atom array. But in format_ref_array_item() we have another round of parsing format string. This affects performance. Introducing the struct parsed_atom_list which can save the current atom's start and end address in format string and its index in used_atom. All parsed_atom_list entry are linked together in the form of linked list, and the head node of the linked list is stored in struct ref_format. We can use clear_parsed_atom_list() to clear the nodes on the linked list. This can bring 1.9% performance improvement for git cat-file --batch. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/branch.c | 2 ++ builtin/cat-file.c | 3 +++ builtin/for-each-ref.c | 3 ++- builtin/tag.c | 2 ++ builtin/verify-tag.c | 2 ++ ref-filter.c | 45 +++++++++++++++++++++++++++++++++--------- ref-filter.h | 11 +++++++++++ 7 files changed, 58 insertions(+), 10 deletions(-) diff --git a/builtin/branch.c b/builtin/branch.c index b23b1d1752a..e361f8cc661 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -459,6 +459,7 @@ static void print_ref_list(struct ref_filter *filter, struct ref_sorting *sortin strbuf_release(&err); strbuf_release(&out); ref_array_clear(&array); + clear_parsed_atom_list(&format->parsed_atom_head); free(to_free); } @@ -678,6 +679,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix) memset(&filter, 0, sizeof(filter)); filter.kind = FILTER_REFS_BRANCHES; filter.abbrev = -1; + INIT_LIST_HEAD(&format.parsed_atom_head); if (argc == 2 && !strcmp(argv[1], "-h")) usage_with_options(builtin_branch_usage, options); diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 77b470687e1..7205b21fb29 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -421,6 +421,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio strbuf_release(&output); free_global_resource(); strbuf_release(&err); + clear_parsed_atom_list(&batch->format.parsed_atom_head); return 0; } @@ -457,6 +458,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio strbuf_release(&output); free_global_resource(); strbuf_release(&err); + clear_parsed_atom_list(&batch->format.parsed_atom_head); warn_on_object_refname_ambiguity = save_warning; return retval; } @@ -534,6 +536,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) batch.buffer_output = -1; batch.format.cat_file_mode = 1; + INIT_LIST_HEAD(&batch.format.parsed_atom_head); argc = parse_options(argc, argv, prefix, options, cat_file_usage, 0); if (opt) { diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 89cb6307d46..6e22d80d5b5 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -53,8 +53,8 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) memset(&array, 0, sizeof(array)); memset(&filter, 0, sizeof(filter)); - format.format = "%(objectname) %(objecttype)\t%(refname)"; + INIT_LIST_HEAD(&format.parsed_atom_head); git_config(git_default_config, NULL); @@ -96,6 +96,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) ref_array_clear(&array); free_commit_list(filter.with_commit); free_commit_list(filter.no_commit); + clear_parsed_atom_list(&format.parsed_atom_head); UNLEAK(sorting); return 0; } diff --git a/builtin/tag.c b/builtin/tag.c index 452558ec957..549339cbbe4 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -78,6 +78,7 @@ static int list_tags(struct ref_filter *filter, struct ref_sorting *sorting, strbuf_release(&output); ref_array_clear(&array); free(to_free); + clear_parsed_atom_list(&format->parsed_atom_head); return 0; } @@ -493,6 +494,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix) memset(&filter, 0, sizeof(filter)); filter.lines = -1; opt.sign = -1; + INIT_LIST_HEAD(&format.parsed_atom_head); argc = parse_options(argc, argv, prefix, options, git_tag_usage, 0); diff --git a/builtin/verify-tag.c b/builtin/verify-tag.c index f45136a06ba..8b0a2b2587c 100644 --- a/builtin/verify-tag.c +++ b/builtin/verify-tag.c @@ -39,6 +39,7 @@ int cmd_verify_tag(int argc, const char **argv, const char *prefix) OPT_END() }; + INIT_LIST_HEAD(&format.parsed_atom_head); git_config(git_verify_tag_config, NULL); argc = parse_options(argc, argv, prefix, verify_tag_options, @@ -73,5 +74,6 @@ int cmd_verify_tag(int argc, const char **argv, const char *prefix) if (format.format) pretty_print_ref(name, &oid, &format); } + clear_parsed_atom_list(&format.parsed_atom_head); return had_error; } diff --git a/ref-filter.c b/ref-filter.c index 6e904389d6b..771804c06f0 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1051,6 +1051,7 @@ int verify_ref_format(struct ref_format *format) for (cp = format->format; *cp && (sp = find_next(cp)); ) { struct strbuf err = STRBUF_INIT; const char *color, *ep = strchr(sp, ')'); + struct parsed_atom_list *e; int at; if (!ep) @@ -1059,6 +1060,12 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + e = xmalloc(sizeof(*e)); + e->beg = sp + 2; + e->end = ep; + e->at = at; + list_add_tail(&e->list, &format->parsed_atom_head); + if (need_parse_buffer(used_atom[at].atom_type)) format->can_skip_parse_buffer = 0; if (reject_atom(format->cat_file_mode, used_atom[at].atom_type)) @@ -2649,26 +2656,32 @@ int format_ref_array_item(struct ref_array_item *info, struct strbuf *final_buf, struct strbuf *error_buf) { - const char *cp, *sp, *ep; + const char *cp, *sp; + struct list_head *item; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; int ret; state.quote_style = format->quote_style; push_stack_element(&state.stack); info->can_skip_parse_buffer = format->can_skip_parse_buffer; - for (cp = format->format; *cp && (sp = find_next(cp)); cp = ep + 1) { + + cp = format->format; + + list_for_each(item, &format->parsed_atom_head) { struct atom_value *atomv; - int pos; + struct parsed_atom_list *e = + list_entry(item, struct parsed_atom_list, list); - ep = strchr(sp, ')'); - if (cp < sp) - append_literal(cp, sp, &state); - pos = parse_ref_filter_atom(format, sp + 2, ep, error_buf); - if (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) || + if (cp < e->beg - 2) + append_literal(cp, e->beg - 2, &state); + if ((ret = get_ref_atom_value(info, e->at, &atomv, error_buf)) || atomv->handler(atomv, &state, error_buf)) { pop_stack_element(&state.stack); return ret ? ret : -1; } + cp = e->end + 1; + if (!*cp) + break; } if (*cp) { sp = cp + strlen(cp); @@ -2716,10 +2729,12 @@ static int parse_sorting_atom(const char *atom) * This parses an atom using a dummy ref_format, since we don't * actually care about the formatting details. */ + int res; struct ref_format dummy = REF_FORMAT_INIT; const char *end = atom + strlen(atom); struct strbuf err = STRBUF_INIT; - int res = parse_ref_filter_atom(&dummy, atom, end, &err); + + res = parse_ref_filter_atom(&dummy, atom, end, &err); if (res < 0) die("%s", err.buf); strbuf_release(&err); @@ -2792,3 +2807,15 @@ int parse_opt_merge_filter(const struct option *opt, const char *arg, int unset) return 0; } + +void clear_parsed_atom_list(struct list_head *parsed_atom_head) +{ + struct list_head *pos, *tmp; + struct parsed_atom_list *item; + + list_for_each_safe(pos, tmp, parsed_atom_head) { + item = list_entry(pos, struct parsed_atom_list, list); + list_del(pos); + free(item); + } +} diff --git a/ref-filter.h b/ref-filter.h index 8cae9f576d5..2b871f9ed4f 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -73,6 +73,13 @@ struct ref_filter { verbose; }; +struct parsed_atom_list { + const char *beg; + const char *end; + int at; + struct list_head list; +}; + struct ref_format { /* * Set these to define the format; make sure you call @@ -87,6 +94,7 @@ struct ref_format { int can_skip_parse_buffer; /* Internal state to ref-filter */ int need_color_reset_at_eol; + struct list_head parsed_atom_head; }; #define REF_FORMAT_INIT { .use_color = -1, .can_skip_parse_buffer = 1 } @@ -118,6 +126,9 @@ void ref_array_clear(struct ref_array *array); void free_global_resource(void); /* Free ref_array_item's value */ void free_ref_array_item_value(struct ref_array_item *item); +/* Clear the parsed_atom_list in ref_format*/ +void clear_parsed_atom_list(struct list_head *parsed_atom_head); + /* Used to verify if the given format is correct and to parse out the used atoms */ int verify_ref_format(struct ref_format *format); /* Sort the given ref_array as per the ref_sorting provided */ From patchwork Fri Aug 13 08:23:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE420C432BE for ; Fri, 13 Aug 2021 08:23:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 972B6610A5 for ; Fri, 13 Aug 2021 08:23:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238947AbhHMIYM (ORCPT ); Fri, 13 Aug 2021 04:24:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238821AbhHMIXy (ORCPT ); Fri, 13 Aug 2021 04:23:54 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D321EC0612AB for ; Fri, 13 Aug 2021 01:23:26 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id w21-20020a7bc1150000b02902e69ba66ce6so6380352wmi.1 for ; Fri, 13 Aug 2021 01:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=LO+D6cv0mztmNnBkCrNz6+BCjUAnRRy35krJ9OQuw/0=; b=KQKXJRw32eN8X+OF6+OO8+5BST9m8h5j0qtgIKjJWaPjQR1zk8xkdVKGIgOIkvBiA2 /POPj3iFBLucpi6V2osC8RyeVIg+glfSZjobUeKUzx1nfNvw7DRH2S/LSX7c+sf76sU3 1HoqOyEff17m7KDtmDKCnzd+OFuD7dB2U9YiKd7+aRmHRTRZDQj1dfpO2ekTUxJf+xKr QBV/O1LwOpZMqmMA9V2ravpRE8lDGMWXa/DyYlOytyTF+VuppT4HRwI4hnNO+BbbqwAj 4cgJ1UvUi08SAPPmHUz3QSNJXIN+QTcKLKMnqyF5CLYMGCU0VKPRg6JM0vAYNrIwwKHK 4Urg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=LO+D6cv0mztmNnBkCrNz6+BCjUAnRRy35krJ9OQuw/0=; b=fxVWapGv23zz9tnCfO6HaAbMf5ijsaVQsMdhE2AC1tSwfKU2ozwLbqd2XvdfDeYEkB 8HfTOTRR3vhPwtdQmTfJSSJ3jL4AixC8y8EDAfw/swsIx3MjX0T8YWwPzgjfu+Yoidq7 fTOQmF9yRAvHCc5CV2AqgP/GW4i6itEmt2H5JUI1g1cJD19EfZcN0vgdllmz7htsCRTI VHclmLrM3kHJO+rV7jj2kQ2jzclSR3nn9SDkPhgF8Q25jKNMW21IEI0XVgaRj9Pxy+Qf dBVct2PCY0tNzckGK3oakpXLJK+BfqH8yUfI7PV5o6CtkVf0pkV/xnHddsqyqcVG2F08 unvQ== X-Gm-Message-State: AOAM533u8ERNKBd2EHCbNGPJEQ6RafU3YPUSmRFCzzaGoO4Zwa2SGGo8 5CRSxZII1lzdDrNTIIpjuo2vTPqR0uM= X-Google-Smtp-Source: ABdhPJxFBpy1JlK55x8gTPAdGX0jOh9i2hGlIoJzgjBE2VrGfTjfM76yL/kpMYGaDIBOugS17MWDFw== X-Received: by 2002:a7b:c1cf:: with SMTP id a15mr1419155wmj.72.1628843005504; Fri, 13 Aug 2021 01:23:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i8sm728098wma.7.2021.08.13.01.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:25 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:05 +0000 Subject: [PATCH 22/27] [GSOC] ref-filter: introduction ref_filter_slopbuf Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu xstrdup("") calls strdup(""), which in glibc will call malloc(1), this means that there will have unwanted memory allocations. Introduce the global one-byte array ref_filter_slopbuf, which learn from strbuf_slopbuf. let those places which need fill strdup("") now fill ref_filter_slopbuf. This can reduce some unnecessary memory allocation. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 53 ++++++++++++++++++++++++++-------------------------- 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 771804c06f0..5e00dbd993a 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -25,6 +25,8 @@ #include "hashmap.h" #include "strvec.h" +char ref_filter_slopbuf[1]; + static struct ref_msg { const char *gone; const char *ahead; @@ -1225,7 +1227,7 @@ static const char *copy_name(const char *buf) if (!strncmp(cp, " <", 2)) return xmemdupz(buf, cp - buf); } - return xstrdup(""); + return ref_filter_slopbuf; } static const char *copy_email(const char *buf, struct used_atom *atom) @@ -1233,7 +1235,7 @@ static const char *copy_email(const char *buf, struct used_atom *atom) const char *email = strchr(buf, '<'); const char *eoemail; if (!email) - return xstrdup(""); + return ref_filter_slopbuf; switch (atom->u.email_option.option) { case EO_RAW: eoemail = strchr(email, '>'); @@ -1255,7 +1257,7 @@ static const char *copy_email(const char *buf, struct used_atom *atom) } if (!eoemail) - return xstrdup(""); + return ref_filter_slopbuf; return xmemdupz(email, eoemail - email); } @@ -1309,7 +1311,7 @@ static void grab_date(const char *buf, struct atom_value *v, const char *atomnam v->value = timestamp; return; bad: - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; v->value = 0; } @@ -1519,7 +1521,7 @@ static void fill_missing_values(struct atom_value *val) for (i = 0; i < used_atom_cnt; i++) { struct atom_value *v = &val[i]; if (v->s == NULL) - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; } } @@ -1594,7 +1596,7 @@ static const char *lstrip_ref_components(const char *refname, int len) switch (*start++) { case '\0': free((char *)to_free); - return xstrdup(""); + return ref_filter_slopbuf; case '/': remaining--; break; @@ -1632,7 +1634,7 @@ static const char *rstrip_ref_components(const char *refname, int len) char *p = strrchr(start, '/'); if (p == NULL) { free((char *)to_free); - return xstrdup(""); + return ref_filter_slopbuf; } else p[0] = '\0'; } @@ -1663,7 +1665,7 @@ static void fill_remote_ref_details(struct used_atom *atom, const char *refname, AHEAD_BEHIND_FULL) < 0) { v->s = xstrdup(msgs.gone); } else if (!num_ours && !num_theirs) - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; else if (!num_ours) v->s = xstrfmt_len(&v->s_size, msgs.behind, num_theirs); else if (!num_theirs) @@ -1680,7 +1682,7 @@ static void fill_remote_ref_details(struct used_atom *atom, const char *refname, if (stat_tracking_info(branch, &num_ours, &num_theirs, NULL, atom->u.remote_ref.push, AHEAD_BEHIND_FULL) < 0) { - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; return; } if (!num_ours && !num_theirs) @@ -1739,7 +1741,7 @@ char *get_head_description(void) static const char *get_symref(struct used_atom *atom, struct ref_array_item *ref) { if (!ref->symref) - return xstrdup(""); + return ref_filter_slopbuf; else return show_ref(&atom->u.refname, ref->symref); } @@ -1861,7 +1863,7 @@ static char *get_worktree_path(const struct used_atom *atom, const struct ref_ar e = hashmap_get(&(ref_to_worktree_map.map), &entry, ref->refname); if (!e) - return xstrdup(""); + return ref_filter_slopbuf; lookup_result = container_of(e, struct ref_to_worktree_entry, ent); @@ -1884,7 +1886,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) ref->symref = resolve_refdup(ref->refname, RESOLVE_REF_READING, NULL, NULL); if (!ref->symref) - ref->symref = xstrdup(""); + ref->symref = ref_filter_slopbuf; } /* Fill in specials first */ @@ -1912,7 +1914,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) if (ref->kind == FILTER_REFS_BRANCHES) v->s = get_worktree_path(atom, ref); else - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; continue; } else if (atom_type == ATOM_SYMREF) @@ -1922,7 +1924,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) /* only local branches may have an upstream */ if (!skip_prefix(ref->refname, "refs/heads/", &branch_name)) { - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; continue; } branch = branch_get(branch_name); @@ -1931,11 +1933,11 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) if (refname) fill_remote_ref_details(atom, refname, branch, v); else - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; continue; } else if (atom_type == ATOM_PUSH && atom->u.remote_ref.push) { const char *branch_name; - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; if (!skip_prefix(ref->refname, "refs/heads/", &branch_name)) continue; @@ -1948,8 +1950,6 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) if (!refname) continue; } - /* We will definitely re-init v->s on the next line. */ - free((char *)v->s); fill_remote_ref_details(atom, refname, branch, v); continue; } else if (atom_type == ATOM_COLOR) { @@ -1962,7 +1962,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) if (ref->flag & REF_ISPACKED) cp = copy_advance(cp, ",packed"); if (cp == buf) - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; else { *cp = '\0'; v->s = xstrdup(buf + 1); @@ -1979,33 +1979,33 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) continue; } else if (atom_type == ATOM_ALIGN) { v->handler = align_atom_handler; - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; continue; } else if (atom_type == ATOM_END) { v->handler = end_atom_handler; - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; continue; } else if (atom_type == ATOM_IF) { const char *s; if (skip_prefix(name, "if:", &s)) v->s = xstrdup(s); else - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; v->handler = if_atom_handler; continue; } else if (atom_type == ATOM_THEN) { v->handler = then_atom_handler; - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; continue; } else if (atom_type == ATOM_ELSE) { v->handler = else_atom_handler; - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; continue; } else if (atom_type == ATOM_REST) { if (ref->rest) v->s = xstrdup(ref->rest); else - v->s = xstrdup(""); + v->s = ref_filter_slopbuf; continue; } else continue; @@ -2356,7 +2356,8 @@ void free_ref_array_item_value(struct ref_array_item *item) if (item->value) { int i; for (i = 0; i < used_atom_cnt; i++) - free((char *)item->value[i].s); + if (item->value[i].s != ref_filter_slopbuf) + free((char *)item->value[i].s); free(item->value); } } From patchwork Fri Aug 13 08:23:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8C68C4338F for ; Fri, 13 Aug 2021 08:23:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CB9476056B for ; Fri, 13 Aug 2021 08:23:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238821AbhHMIYN (ORCPT ); Fri, 13 Aug 2021 04:24:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238823AbhHMIXy (ORCPT ); Fri, 13 Aug 2021 04:23:54 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7192DC0612AC for ; Fri, 13 Aug 2021 01:23:27 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id f5so12088582wrm.13 for ; Fri, 13 Aug 2021 01:23:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nB54XRvV0PUoeKaFKTkUHVNqT6ILEprINfQkt5qsgLY=; b=o2BffcwZC1NWJ9kDBE4NFNsjXHHpNmULwhv1xM4SapYtPApqog5CNU/WJOQCAqUH/j UrEgyyCgeQIzAcHvZiWEqNj//xS3GXY+Gq0nsc81aI93ArAIFnGoIvTyl8axlcUuMFxV hh209hrZ2/FdKs2UAlceggJmYtsjCZKI4mUbQLUvfxx3aQTc7IPfaInl41KP3UpIsykT FP4mZGC2bdalhZviq/g9O0OX/b6XuvA5bRanr/YR6qnz8JbafFUiDE9LBFLYqNfpBxxm 6fl2VUb17Z/U0QIoD1bY3pT0owqQkrWU+FI+MB0TWFLn7Y6eo7k66uX7VkMuOKkPcZzA SZgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nB54XRvV0PUoeKaFKTkUHVNqT6ILEprINfQkt5qsgLY=; b=X47LB9UVhR3aexpjFu3b1URs3//3jENLANLvSVfkOSYoYoMMIHjepOodH5usqdFPE7 P2a4wdoBdrukqMPyw2Trv7R46Eys0Sjmkm0yyfNXzoiX/+Eo82GX7c5knfhy9dgbpvLu 0o4LwkYYspH4VEwLJaZvPIGpSrttPQAZOTSw9ALsNKOzdGyms9Cf191g0gr3xuoSyFW4 VHZdMo6p8UmprMzHS/lrg5g3Scbfu6E5B6KZU1oZ9yNk47zXuvkKGWoJtgBddmJLKuWA RoXxENTmGg8Qq5RzPvoJe1p+ZVESjZPwshoh+thFYFWcc67SD8Zpjl2ZZfVYZd9LY5sm abyw== X-Gm-Message-State: AOAM5317Mwh8JZsbThCkzQqh3GtuLn4S6hi3tbmZcgnxnQdnh0qbfQlN 4apU1b3KFiGnnnXakb1k6D8duqt+gYU= X-Google-Smtp-Source: ABdhPJxSlGrD/fy3iKY/+T68VfGwCaBOaBPkES6SONunBvdMI99fsAZOqDIKWtR7BpQ1+dp6w7caHQ== X-Received: by 2002:adf:8102:: with SMTP id 2mr1598956wrm.89.1628843006055; Fri, 13 Aug 2021 01:23:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s1sm676847wmh.46.2021.08.13.01.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:25 -0700 (PDT) Message-Id: <5b21b04683429fdf85b9df9723777fb6fad8ea7a.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:06 +0000 Subject: [PATCH 23/27] [GSOC] ref-filter: add deref member to struct used_atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In the original logic, the dereference flag '*' is stored at the beginning of used_atom[i].name. (e.g. '*objectname:short') This leads us to make some check on used_atom[i].name[0] to know if we the atom needs to be dereferenced. So add `deref` flag to struct used_atom used to indicate whether the atom needs to be dereferenced and let used_atom record only the remaining part (i.e. 'objectname:short') This can simplify the logic of the program. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 55 ++++++++++++++++++++-------------------------------- 1 file changed, 21 insertions(+), 34 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 5e00dbd993a..bde1f3de9eb 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -174,6 +174,7 @@ enum atom_type { * array. */ static struct used_atom { + unsigned int deref : 1; enum atom_type atom_type; const char *name; cmp_type type; @@ -321,7 +322,7 @@ static int objecttype_atom_parser(struct ref_format *format, struct used_atom *a { if (arg) return strbuf_addf_ret(err, -1, _("%%(objecttype) does not take arguments")); - if (*atom->name == '*') + if (atom->deref) oi_deref.info.typep = &oi_deref.type; else oi.info.typep = &oi.type; @@ -333,13 +334,13 @@ static int objectsize_atom_parser(struct ref_format *format, struct used_atom *a { if (!arg) { atom->u.objectsize.option = O_SIZE; - if (*atom->name == '*') + if (atom->deref) oi_deref.info.sizep = &oi_deref.size; else oi.info.sizep = &oi.size; } else if (!strcmp(arg, "disk")) { atom->u.objectsize.option = O_SIZE_DISK; - if (*atom->name == '*') + if (atom->deref) oi_deref.info.disk_sizep = &oi_deref.disk_size; else oi.info.disk_sizep = &oi.disk_size; @@ -353,7 +354,7 @@ static int deltabase_atom_parser(struct ref_format *format, struct used_atom *at { if (arg) return strbuf_addf_ret(err, -1, _("%%(deltabase) does not take arguments")); - if (*atom->name == '*') + if (atom->deref) oi_deref.info.delta_base_oid = &oi_deref.delta_base_oid; else oi.info.delta_base_oid = &oi.delta_base_oid; @@ -675,10 +676,13 @@ static int parse_ref_filter_atom(struct ref_format *format, const char *sp; const char *arg; int i, at, atom_len; + int deref = 0; sp = atom; - if (*sp == '*' && sp < ep) + if (*sp == '*' && sp < ep) { sp++; /* deref */ + deref = 1; + } if (ep <= sp) return strbuf_addf_ret(err, -1, _("malformed field name: %.*s"), (int)(ep-atom), atom); @@ -695,7 +699,7 @@ static int parse_ref_filter_atom(struct ref_format *format, /* Do we have the atom already used elsewhere? */ for (i = 0; i < used_atom_cnt; i++) { int len = strlen(used_atom[i].name); - if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) + if (len == ep - sp && !memcmp(used_atom[i].name, sp, len)) return i; } @@ -719,17 +723,18 @@ static int parse_ref_filter_atom(struct ref_format *format, used_atom_cnt++; REALLOC_ARRAY(used_atom, used_atom_cnt); used_atom[at].atom_type = i; - used_atom[at].name = xmemdupz(atom, ep - atom); + used_atom[at].deref = deref; + used_atom[at].name = xmemdupz(sp, ep - sp); used_atom[at].type = valid_atom[i].cmp_type; used_atom[at].source = valid_atom[i].source; if (used_atom[at].source == SOURCE_OBJ) { - if (*atom == '*') + if (deref) oi_deref.info.contentp = &oi_deref.content; else oi.info.contentp = &oi.content; } if (arg) { - arg = used_atom[at].name + (arg - atom) + 1; + arg = used_atom[at].name + (arg - sp) + 1; if (!*arg) { /* * Treat empty sub-arguments list as NULL (i.e., @@ -741,7 +746,7 @@ static int parse_ref_filter_atom(struct ref_format *format, memset(&used_atom[at].u, 0, sizeof(used_atom[at].u)); if (valid_atom[i].parser && valid_atom[i].parser(format, &used_atom[at], arg, err)) return -1; - if (*atom == '*') + if (deref) need_tagged = 1; if (i == ATOM_SYMREF) need_symref = 1; @@ -1112,13 +1117,10 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ int i; for (i = 0; i < used_atom_cnt; i++) { - const char *name = used_atom[i].name; enum atom_type atom_type = used_atom[i].atom_type; struct atom_value *v = &val[i]; - if (!!deref != (*name == '*')) + if (!!deref != used_atom[i].deref) continue; - if (deref) - name++; if (atom_type == ATOM_OBJECTTYPE) v->s = xstrdup(type_name(oi->type)); else if (atom_type == ATOM_OBJECTSIZE) { @@ -1144,13 +1146,10 @@ static void grab_tag_values(struct atom_value *val, int deref, struct object *ob struct tag *tag = (struct tag *) obj; for (i = 0; i < used_atom_cnt; i++) { - const char *name = used_atom[i].name; enum atom_type atom_type = used_atom[i].atom_type; struct atom_value *v = &val[i]; - if (!!deref != (*name == '*')) + if (!!deref != used_atom[i].deref) continue; - if (deref) - name++; if (atom_type == ATOM_TAG) v->s = xstrdup(tag->tag); else if (atom_type == ATOM_TYPE && tag->tagged) @@ -1167,13 +1166,10 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object struct commit *commit = (struct commit *) obj; for (i = 0; i < used_atom_cnt; i++) { - const char *name = used_atom[i].name; enum atom_type atom_type = used_atom[i].atom_type; struct atom_value *v = &val[i]; - if (!!deref != (*name == '*')) + if (!!deref != used_atom[i].deref) continue; - if (deref) - name++; if (atom_type == ATOM_TREE) { v->s = xstrdup(do_grab_oid("tree", get_commit_tree_oid(commit), &used_atom[i])); continue; @@ -1327,10 +1323,8 @@ static void grab_person(enum atom_type type, struct atom_value *val, int deref, const char *name = used_atom[i].name; enum atom_type atom_type = used_atom[i].atom_type; struct atom_value *v = &val[i]; - if (!!deref != (*name == '*')) + if (!!deref != used_atom[i].deref) continue; - if (deref) - name++; if ((atom_type < type || atom_type > type + 3) && /* * For a tag or a commit object, if "creator" or "creatordate" is @@ -1446,10 +1440,8 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp struct atom_value *v = &val[i]; enum atom_type atom_type = atom->atom_type; - if (!!deref != (*name == '*')) + if (!!deref != used_atom[i].deref) continue; - if (deref) - name++; if (atom_type == ATOM_RAW) { unsigned long buf_size = data->size; @@ -1895,7 +1887,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) enum atom_type atom_type = atom->atom_type; const char *name = used_atom[i].name; struct atom_value *v = &ref->value[i]; - int deref = 0; + int deref = atom->deref; const char *refname; struct branch *branch = NULL; @@ -1903,11 +1895,6 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->handler = append_atom; v->atom = atom; - if (*name == '*') { - deref = 1; - name++; - } - if (atom_type == ATOM_REFNAME) refname = get_refname(atom, ref); else if (atom_type == ATOM_WORKTREEPATH) { From patchwork Fri Aug 13 08:23:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50ADFC4338F for ; Fri, 13 Aug 2021 08:23:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 344C46056B for ; Fri, 13 Aug 2021 08:23:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238973AbhHMIYP (ORCPT ); Fri, 13 Aug 2021 04:24:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238673AbhHMIXz (ORCPT ); Fri, 13 Aug 2021 04:23:55 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14A69C06124C for ; Fri, 13 Aug 2021 01:23:28 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id g138so6324228wmg.4 for ; Fri, 13 Aug 2021 01:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=CwEVQU+nR+mX/AL1EZNUNs9TGHKLJB/OiZcOeV7vLyM=; b=OKzYr2ceTfmNY6SY7CXOdu4GG/CCCmHwIyia5cSVtysrjCNsm9FRsPQvhIYcQ9gfdH p6c/d30ChmiJLt0GSuVHV1BRdcsetM2stvCCtTHYelwQEeC7f83LJ8714RZWROGFEMEL dsNMBJidgHvue3yqO0+n4gugEXSUCwF4W9NXasHiC1bIEDMF70ocbm9+HsYIXfAzJ15g Rhc7ClRGgba3fx2Yqv3T7BGuglBp+h4FH9fDa47hQuDPm0c7VeJ1qTPp5QlYbQCd5kKm 71hkeA8ori2SZDOhHOXuhxhkgDBVwHhZHviY/9r/tXk/2ReKToqu3yxAT4GImQ+/WaGE RO+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=CwEVQU+nR+mX/AL1EZNUNs9TGHKLJB/OiZcOeV7vLyM=; b=Ya/XW/Qp3FrJwiYzyQ149QJ6PNmcXWoI1GSZE6KFkxWY37poqgVxv4hx1KKGn8W3I3 IlUt39R5RLLQazmbuyGsTIpZuQ2kshR8T4/Gt3s5QA/v1HSNkZcRc6s5LYvCLGntUBmH BKNWWSXvBR9vtPjVmk/JnnN1V26aS0CemjaKCMkduauZh55ClupGClhtxG/yaMcn5qmg Jls/XgEj/WFurn6vVixfMfKEzRUudcHxeH/VxPYn1SMpm0n6Y2iNLmT68sC0YH5wtUzj PStqn69yOUbGRGRgvFf/RYP7JBZKZA5Pyvk1SjM/FBDZzfp3i3lT1+JJUmJNf1TO27Xm g8kQ== X-Gm-Message-State: AOAM530W/+0xT3/he02tJ1JfAY/4puHEBjPmBBd/lYp0fbf0IdPo62Lk F5xWnAtzBXuX19qNUmp09lrmrTWU5iU= X-Google-Smtp-Source: ABdhPJxl8DlbQOwt6+d4XGTio8UqMKWgC7A0tGk4Ie68DdD6OeEpbpFmlz1WzIGB44TMa+zoofeZ2w== X-Received: by 2002:a05:600c:20f:: with SMTP id 15mr1427846wmi.176.1628843006707; Fri, 13 Aug 2021 01:23:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i5sm827817wrw.13.2021.08.13.01.23.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:26 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:07 +0000 Subject: [PATCH 24/27] [GSOC] ref-filter: introduce symref_atom_parser() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu parse_ref_filter_atom() checks whether the type of the atom is ATOM_SYMREF, this is not necessary for other atoms. So add the symref_atom_parser() specifically to parse the atom, and set need_symref in it. This can make the program logic more concise. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index bde1f3de9eb..a0de5f21ff5 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -481,6 +481,13 @@ static int person_email_atom_parser(struct ref_format *format, struct used_atom return 0; } +static int symref_atom_parser(struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + need_symref = 1; + return refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err); +} + static int refname_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { @@ -620,7 +627,7 @@ static struct { [ATOM_RAW] = { "raw", SOURCE_OBJ, FIELD_STR, raw_atom_parser }, [ATOM_UPSTREAM] = { "upstream", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_PUSH] = { "push", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, - [ATOM_SYMREF] = { "symref", SOURCE_NONE, FIELD_STR, refname_atom_parser }, + [ATOM_SYMREF] = { "symref", SOURCE_NONE, FIELD_STR, symref_atom_parser }, [ATOM_FLAG] = { "flag", SOURCE_NONE }, [ATOM_HEAD] = { "HEAD", SOURCE_NONE, FIELD_STR, head_atom_parser }, [ATOM_COLOR] = { "color", SOURCE_NONE, FIELD_STR, color_atom_parser }, @@ -748,8 +755,6 @@ static int parse_ref_filter_atom(struct ref_format *format, return -1; if (deref) need_tagged = 1; - if (i == ATOM_SYMREF) - need_symref = 1; return at; } From patchwork Fri Aug 13 08:23:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD2C6C4320A for ; Fri, 13 Aug 2021 08:23:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9BB40610CC for ; Fri, 13 Aug 2021 08:23:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238844AbhHMIYR (ORCPT ); Fri, 13 Aug 2021 04:24:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238873AbhHMIXz (ORCPT ); Fri, 13 Aug 2021 04:23:55 -0400 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C39ABC06129D for ; Fri, 13 Aug 2021 01:23:28 -0700 (PDT) Received: by mail-wm1-x332.google.com with SMTP id c129-20020a1c35870000b02902e6b6135279so4912982wma.0 for ; Fri, 13 Aug 2021 01:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=7a6J0DpiisZQGuEWyGaP2+dSaNWMzzdBN3tBLkyi3Y8=; b=Rie+W8JCmCWdlQb5oXeS+LVf0QxVHV2hbmhQX7Hk6XPVYGpxsNhpDDV/uIcZjDqHtO FOVKCkMEOFUbnbZ/wDKLD9JPUkFQch+lAMWnAdKPc/IZHoTRq//t09+/cZRq23e4npaz cIPV6y2zSxnvV8vX88VNCVn2BL+tk25lsUhotF2V65wRKgicnjTLUWAVwmjD1HISqU5D Zcjt08LSKlH5E869RKd8/O0hN0lq9svoA/XfBLUzuvO4SyMeBkHuXUKCk+3NlOxSAKZZ y0z++r5hhGFUw3286meVZsZB9ZE/9uqkpck9bbAn4ldhH7eUKuYYe1Mm/4Lwd42Fko6M 3vkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=7a6J0DpiisZQGuEWyGaP2+dSaNWMzzdBN3tBLkyi3Y8=; b=RMRA9/GRWgomaNU1PrCaiQJNcymJHMevfOoEo2ZTGuWNHESmoSqhbu27DGTomBv61G SK3cM+BWW+1KOwKQzg/XmQIvrqm1blK0veqCk419cyn8bb88M7epQg6Ck6FUOFcN5+Wl opN1L8QbAX/Z299oWlknf5UqIXch64RAsPzdwoZohtRqE9zvOrASTRyrGm5GvzLqqO+A gi2SaxdHK6n6HPl3dLlNThrqJlyc2uOzQl67Pd8ME20cZgF6c4AiMolMbkCFJkF6RUO9 JXMDAhsfWQis02mis/MZZ8DKp4mpUh3nHOmJdMKKXu3pCdZjbw74iRTkRmlTaCTmKvEh DDeA== X-Gm-Message-State: AOAM530XIDm3vTSiuHUmrnU8s7+ADLPZVGP6l9k4oI4onFtE3hNtlolv ZMOj2oABu2mZT00JeJWSVG/DEHEUFU8= X-Google-Smtp-Source: ABdhPJzNgsNZlYZWaRywSebCJh+5sCtZkaJvQG/kkD08LR1QRzZbsM07Z3T4C2BL8unQOPuSy4lYvg== X-Received: by 2002:a1c:638a:: with SMTP id x132mr1483013wmb.146.1628843007418; Fri, 13 Aug 2021 01:23:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id w9sm686013wmc.19.2021.08.13.01.23.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:27 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:08 +0000 Subject: [PATCH 25/27] [GSOC] ref-filter: use switch case instread of if else Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In the original code, the check of the atom type is through if/else conditional check, using switch/case can allow the compiler to reduce unnecessary branch checks and increase the readability of the code. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 163 ++++++++++++++++++++++++++++++++++----------------- 1 file changed, 109 insertions(+), 54 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index a0de5f21ff5..6fbf38f1375 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1126,9 +1126,11 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ struct atom_value *v = &val[i]; if (!!deref != used_atom[i].deref) continue; - if (atom_type == ATOM_OBJECTTYPE) + switch (atom_type) { + case ATOM_OBJECTTYPE : { v->s = xstrdup(type_name(oi->type)); - else if (atom_type == ATOM_OBJECTSIZE) { + break; + } case ATOM_OBJECTSIZE : { if (used_atom[i].u.objectsize.option == O_SIZE_DISK) { v->value = oi->disk_size; v->s = xstrfmt_len(&v->s_size, "%"PRIuMAX, (uintmax_t)oi->disk_size); @@ -1136,10 +1138,20 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ v->value = oi->size; v->s = xstrfmt_len(&v->s_size, "%"PRIuMAX , (uintmax_t)oi->size); } - } else if (atom_type == ATOM_DELTABASE) + break; + } + case ATOM_DELTABASE: { v->s = xstrdup(oid_to_hex(&oi->delta_base_oid)); - else if (atom_type == ATOM_OBJECTNAME && deref) { - v->s = xstrdup(do_grab_oid("objectname", &oi->oid, &used_atom[i])); + break; + } + case ATOM_OBJECTNAME: { + if (deref) + v->s = xstrdup(do_grab_oid("objectname", &oi->oid, &used_atom[i])); + break; + } + default: { + break; + } } } } @@ -1154,13 +1166,22 @@ static void grab_tag_values(struct atom_value *val, int deref, struct object *ob enum atom_type atom_type = used_atom[i].atom_type; struct atom_value *v = &val[i]; if (!!deref != used_atom[i].deref) - continue; - if (atom_type == ATOM_TAG) + break; + switch (atom_type) { + case ATOM_TAG: v->s = xstrdup(tag->tag); - else if (atom_type == ATOM_TYPE && tag->tagged) - v->s = xstrdup(type_name(tag->tagged->type)); - else if (atom_type == ATOM_OBJECT && tag->tagged) - v->s = xstrdup(oid_to_hex(&tag->tagged->oid)); + break; + case ATOM_TYPE: + if (tag->tagged) + v->s = xstrdup(type_name(tag->tagged->type)); + break; + case ATOM_OBJECT: + if (tag->tagged) + v->s = xstrdup(oid_to_hex(&tag->tagged->oid)); + break; + default: + break; + } } } @@ -1175,15 +1196,17 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object struct atom_value *v = &val[i]; if (!!deref != used_atom[i].deref) continue; - if (atom_type == ATOM_TREE) { + switch (atom_type) { + case ATOM_TREE: { v->s = xstrdup(do_grab_oid("tree", get_commit_tree_oid(commit), &used_atom[i])); - continue; + break; } - if (atom_type == ATOM_NUMPARENT) { + case ATOM_NUMPARENT: { v->value = commit_list_count(commit->parents); v->s = xstrfmt_len(&v->s_size, "%lu", (unsigned long)v->value); + break; } - else if (atom_type == ATOM_PARENT) { + case ATOM_PARENT: { struct commit_list *parents; struct strbuf s = STRBUF_INIT; for (parents = commit->parents; parents; parents = parents->next) { @@ -1193,6 +1216,11 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object strbuf_addstr(&s, do_grab_oid("parent", oid, &used_atom[i])); } v->s = strbuf_detach(&s, NULL); + break; + } + default: { + break; + } } } } @@ -1900,24 +1928,40 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->handler = append_atom; v->atom = atom; - if (atom_type == ATOM_REFNAME) + switch (atom_type) { + case ATOM_REFNAME: { refname = get_refname(atom, ref); - else if (atom_type == ATOM_WORKTREEPATH) { + if (!deref) + v->s = xstrdup(refname); + else + v->s = xstrfmt_len(&v->s_size, "%s^{}", refname); + free((char *)refname); + break; + } + case ATOM_WORKTREEPATH: { if (ref->kind == FILTER_REFS_BRANCHES) v->s = get_worktree_path(atom, ref); else v->s = ref_filter_slopbuf; - continue; + break; } - else if (atom_type == ATOM_SYMREF) + case ATOM_SYMREF: { refname = get_symref(atom, ref); - else if (atom_type == ATOM_UPSTREAM) { + if (!deref) + v->s = xstrdup(refname); + else + v->s = xstrfmt_len(&v->s_size, "%s^{}", refname); + if (refname != ref_filter_slopbuf) + free((char *)refname); + break; + } + case ATOM_UPSTREAM: { const char *branch_name; /* only local branches may have an upstream */ if (!skip_prefix(ref->refname, "refs/heads/", &branch_name)) { v->s = ref_filter_slopbuf; - continue; + break; } branch = branch_get(branch_name); @@ -1926,13 +1970,17 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) fill_remote_ref_details(atom, refname, branch, v); else v->s = ref_filter_slopbuf; - continue; - } else if (atom_type == ATOM_PUSH && atom->u.remote_ref.push) { + break; + } + case ATOM_PUSH: { const char *branch_name; + + if (!atom->u.remote_ref.push) + break; v->s = ref_filter_slopbuf; if (!skip_prefix(ref->refname, "refs/heads/", &branch_name)) - continue; + break; branch = branch_get(branch_name); if (atom->u.remote_ref.push_remote) @@ -1940,14 +1988,16 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) else { refname = branch_get_push(branch, NULL); if (!refname) - continue; + break; } fill_remote_ref_details(atom, refname, branch, v); - continue; - } else if (atom_type == ATOM_COLOR) { + break; + } + case ATOM_COLOR: { v->s = xstrdup(atom->u.color); continue; - } else if (atom_type == ATOM_FLAG) { + } + case ATOM_FLAG: { char buf[256], *cp = buf; if (ref->flag & REF_ISSYMREF) cp = copy_advance(cp, ",symref"); @@ -1959,54 +2009,59 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) *cp = '\0'; v->s = xstrdup(buf + 1); } - continue; - } else if (!deref && atom_type == ATOM_OBJECTNAME) { - v->s = xstrdup(do_grab_oid("objectname", &ref->objectname, atom)); - continue; - } else if (atom_type == ATOM_HEAD) { + break; + } + case ATOM_OBJECTNAME: { + if (!deref) + v->s = xstrdup(do_grab_oid("objectname", &ref->objectname, atom)); + break; + } + case ATOM_HEAD: { if (atom->u.head && !strcmp(ref->refname, atom->u.head)) v->s = xstrdup("*"); else v->s = xstrdup(" "); - continue; - } else if (atom_type == ATOM_ALIGN) { + break; + } + case ATOM_ALIGN: { v->handler = align_atom_handler; v->s = ref_filter_slopbuf; - continue; - } else if (atom_type == ATOM_END) { + break; + } + case ATOM_END: { v->handler = end_atom_handler; v->s = ref_filter_slopbuf; - continue; - } else if (atom_type == ATOM_IF) { + break; + } + case ATOM_IF: { const char *s; if (skip_prefix(name, "if:", &s)) v->s = xstrdup(s); else v->s = ref_filter_slopbuf; v->handler = if_atom_handler; - continue; - } else if (atom_type == ATOM_THEN) { + break; + } + case ATOM_THEN: { v->handler = then_atom_handler; v->s = ref_filter_slopbuf; - continue; - } else if (atom_type == ATOM_ELSE) { + break; + } + case ATOM_ELSE: { v->handler = else_atom_handler; v->s = ref_filter_slopbuf; - continue; - } else if (atom_type == ATOM_REST) { + break; + } + case ATOM_REST: { if (ref->rest) v->s = xstrdup(ref->rest); else v->s = ref_filter_slopbuf; - continue; - } else - continue; - - if (!deref) - v->s = xstrdup(refname); - else - v->s = xstrfmt_len(&v->s_size, "%s^{}", refname); - free((char *)refname); + break; + } + default: + break; + } } for (i = 0; i < used_atom_cnt; i++) { From patchwork Fri Aug 13 08:23:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57C90C4338F for ; Fri, 13 Aug 2021 08:23:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B634610CC for ; Fri, 13 Aug 2021 08:23:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238817AbhHMIYS (ORCPT ); Fri, 13 Aug 2021 04:24:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238278AbhHMIXz (ORCPT ); Fri, 13 Aug 2021 04:23:55 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52CF5C061756 for ; Fri, 13 Aug 2021 01:23:29 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id f12-20020a05600c4e8c00b002e6bdd6ffe2so3270331wmq.5 for ; Fri, 13 Aug 2021 01:23:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xl3wEz5h/ZW0dKlQedupx1l6uCaGvRFZ7ih/9kSSfRY=; b=HnzovCXbFOGQDmra17RpyZRI8Oip8YnPtRf8Qn/H5EyDB+/zmtVlVmFaCvLWSKk8sr YOf+O8A7EZRNk9AB/fvN9+MYXB7d2HU+QxXMw5MKcTyhHToNgV6p7S+c0SD1mSy5cbYV Ni2e8gaOts3R4Hoe/mogKmbFE2eB58/y+jSRbgfH4mSNoq+4qMKhwwNeRdPt5QcrVwtX Ezsu+TOllzXUE7olwJQcimsccdHojc1YruEqj4REDtLPit/BW1GqRxyXDY4L1SQpHm/5 QhLEynQs7ECo07gK1Z1dyjGYHN2lGdQbA6LZP0H6MNU4xqc0kC1iV1oYFB0sTBeowwoX ODhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xl3wEz5h/ZW0dKlQedupx1l6uCaGvRFZ7ih/9kSSfRY=; b=jdcMlaPdijQSGbBNjMWRs97JppZuP/7yyWXxwMhocXd+1qwjwnLOVSYQg0bTatQYKZ e0Om45poe3Ppi73MDugtCvJMhAY6wmXc7OirVTz6WHbd3FEe38nckL62msAeTHTY1TCQ DXAlI9bkN1KYB3BRg42xB7EvaNsSthcrrwGEQN2ogrehdLhC9vKlkWcq0zfQj7Z+cOQ4 SX3CVz2bdLNuk2nBO2ddrk0M51ZrHO7Mt3Dp4+MBRTbiAvdHt5z2ZvXY8LlCqdAstHL6 t/GC1F1TY99BPqnPw/9zx6lBRgD+UFTanZwGHFBEeOWaXw83PSmaGxKwi8B9kFoI7n/D baAg== X-Gm-Message-State: AOAM533x3I8IVQ4TgX/wWmqCdK10u86tOTLXGnH2z/YFHz1e9FwvzlxS kEq96e7T1WyiRDr8JUI7GX4e6nsEeeA= X-Google-Smtp-Source: ABdhPJzIt8mVewE45Hi5swYsOgTa6Qk8dvOHFiu2yAC/B7UsJTgu+AZ3DZ/jKdpMABL3ppv+ZMioGg== X-Received: by 2002:a7b:c188:: with SMTP id y8mr1428192wmi.174.1628843007967; Fri, 13 Aug 2021 01:23:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id w14sm803745wrt.23.2021.08.13.01.23.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:27 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:09 +0000 Subject: [PATCH 26/27] [GSOC] ref-filter: reuse finnal buffer if no stack need Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because finnal_buf will be cleared before every time we call format_ref_array_item(). So we actually add content to an empty strbuf, We can add the object's data directly to this finnal_buffer instead of adding objects' data to state.stack->output first, then copying to finnal_buf. So add a can_reuse_final_buffer flag to struct ref_format to check if we are use %(align), %(end), %(if), %(then), %(else). if not, we can reuse the buf of finnal_buf. This can bring 2% performance improvement for git cat-file --batch. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 36 +++++++++++++++++++++++++++++------- ref-filter.h | 3 ++- 2 files changed, 31 insertions(+), 8 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 6fbf38f1375..4d7aa60da0f 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1083,6 +1083,13 @@ int verify_ref_format(struct ref_format *format) if (reject_atom(format->cat_file_mode, used_atom[at].atom_type)) die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); + if (used_atom[at].atom_type == ATOM_ALIGN || + used_atom[at].atom_type == ATOM_END || + used_atom[at].atom_type == ATOM_IF || + used_atom[at].atom_type == ATOM_THEN || + used_atom[at].atom_type == ATOM_ELSE) + format->can_reuse_final_buffer = 0; + if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || format->quote_style == QUOTE_TCL) && @@ -2710,7 +2717,14 @@ int format_ref_array_item(struct ref_array_item *info, int ret; state.quote_style = format->quote_style; - push_stack_element(&state.stack); + if (format->can_reuse_final_buffer) { + struct ref_formatting_stack *s = xmalloc(sizeof(struct ref_formatting_stack)); + s->output = *final_buf; + s->prev = state.stack; + state.stack = s; + } else { + push_stack_element(&state.stack); + } info->can_skip_parse_buffer = format->can_skip_parse_buffer; cp = format->format; @@ -2722,9 +2736,10 @@ int format_ref_array_item(struct ref_array_item *info, if (cp < e->beg - 2) append_literal(cp, e->beg - 2, &state); - if ((ret = get_ref_atom_value(info, e->at, &atomv, error_buf)) || - atomv->handler(atomv, &state, error_buf)) { - pop_stack_element(&state.stack); + if (((ret = get_ref_atom_value(info, e->at, &atomv, error_buf)) || + atomv->handler(atomv, &state, error_buf))) { + if (!format->can_reuse_final_buffer) + pop_stack_element(&state.stack); return ret ? ret : -1; } cp = e->end + 1; @@ -2739,16 +2754,23 @@ int format_ref_array_item(struct ref_array_item *info, struct atom_value resetv = ATOM_VALUE_INIT; resetv.s = GIT_COLOR_RESET; if (append_atom(&resetv, &state, error_buf)) { - pop_stack_element(&state.stack); + if (!format->can_reuse_final_buffer) + pop_stack_element(&state.stack); return -1; } } if (state.stack->prev) { + assert(!format->can_reuse_final_buffer); pop_stack_element(&state.stack); return strbuf_addf_ret(error_buf, -1, _("format: %%(end) atom missing")); } - strbuf_addbuf(final_buf, &state.stack->output); - pop_stack_element(&state.stack); + if(format->can_reuse_final_buffer) { + *final_buf = state.stack->output; + free(state.stack); + } else { + strbuf_addbuf(final_buf, &state.stack->output); + pop_stack_element(&state.stack); + } return 0; } diff --git a/ref-filter.h b/ref-filter.h index 2b871f9ed4f..bf1d2fa18c6 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -94,10 +94,11 @@ struct ref_format { int can_skip_parse_buffer; /* Internal state to ref-filter */ int need_color_reset_at_eol; + int can_reuse_final_buffer; struct list_head parsed_atom_head; }; -#define REF_FORMAT_INIT { .use_color = -1, .can_skip_parse_buffer = 1 } +#define REF_FORMAT_INIT { .use_color = -1, .can_skip_parse_buffer = 1, .can_reuse_final_buffer = 1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ From patchwork Fri Aug 13 08:23:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12435133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EE2BC432BE for ; Fri, 13 Aug 2021 08:23:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 801E2610CC for ; Fri, 13 Aug 2021 08:23:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239333AbhHMIYR (ORCPT ); Fri, 13 Aug 2021 04:24:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238792AbhHMIX4 (ORCPT ); Fri, 13 Aug 2021 04:23:56 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3739C0617AE for ; Fri, 13 Aug 2021 01:23:29 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id k4so6336667wms.3 for ; Fri, 13 Aug 2021 01:23:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=n530dLlV5/doLmnyndxzdNlCvHcz0xSPoKIsuUSlh7E=; b=VVy+vx8wreGHRe3OIX8xxAyWBABjWb385wfOZkHdex9xqzszlZsKfA0NrGIm5PwGYj c+NRzHbjUssX0dl+zLZJ/mWfAiWaktNuPLfA8BFDPzItOtixn5Wsk/HUM2wPgnXX4Dv3 48QzP2D4EXlQZ7O3pqxkovFEbE1ZJRpjo3avC+ZegPcn+FjXsU4GoIvvi7isMtP3LR/B X+eHX3wvF8I94x7fQpf66Qz8amsUwMe/Xg1iqhxXdzyjKNT24poh6gIOJZ36FO+4UWSv Q97DxIAR5XX2Of1J74hb4KZ8o/gCjCACt9Vfkm7V1cGwQrWpt04kD+2by7KjxAIqcYVD ez4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=n530dLlV5/doLmnyndxzdNlCvHcz0xSPoKIsuUSlh7E=; b=mCaWw/BorJP2+k1jrZVxHjCTvSaSDqIUdem/4fTjNVLpmf6fSsYbZYclLPVR/G8JOi w/aXpevGtVaGZtRVjc81FwfFyhN18m0Yyn8wliu3zJzNN4R/wDDSekVAdWplBFcab28g XUuCEoxijWoDu994OYrEaR4esGN08wYrSGawtrCUQZFGhlG/I8QTiX1PKFxs5gTtx493 q/SzKe93n2wKc3SMYkA9Jjw/W9jrm14DP13exqEZZYK0LqTTihBxsbwQfsX1xqAOkqxq QAcP1TZ3Mc3ii/dbL7xu8JLYzLNwZIwt+wvFFpYS05b81ZEyPVd1FIPAxXNzLFJYJq49 5k/A== X-Gm-Message-State: AOAM531H6EY+GQ0cnyIW+Ddg6QiddCm5Jvu/KsTVYDN2pSBYGTfxpmUN XIaVUdgzhpTtSlCysORLm3Gohk4pnF0= X-Google-Smtp-Source: ABdhPJzG4K7Swa0Vya1qRD2yWU//lK1WCJKs3IqzPbzCtMJUhKSeKKW2EQ0UEsoVf/LGeRDWJnnmGQ== X-Received: by 2002:a05:600c:190f:: with SMTP id j15mr1407291wmq.11.1628843008565; Fri, 13 Aug 2021 01:23:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o17sm845716wrw.17.2021.08.13.01.23.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 01:23:28 -0700 (PDT) Message-Id: <8591897fbce05e9987a6ee9af157471a3fc9359f.1628842990.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Aug 2021 08:23:10 +0000 Subject: [PATCH 27/27] [GSOC] ref-filter: add need_get_object_info flag to struct expand_data Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , Philip Oakley , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In populate_value() we use an empty struct object_info compare with oi and oi_deref, which determine if we can return early, this is actually very inefficient. So add a `need_get_object_info` flag to struct expand_data, which used to indicate whether we need call get_object() or we can return early. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 46 ++++++++++++++++++++++++++++++++-------------- 1 file changed, 32 insertions(+), 14 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 4d7aa60da0f..edcb5d04cf9 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -77,6 +77,7 @@ static struct ref_trailer_buf { } ref_trailer_buf = {STRING_LIST_INIT_NODUP, STRBUF_INIT, STRBUF_INIT}; static struct expand_data { + unsigned int need_get_object_info : 1; struct object_id oid; enum object_type type; unsigned long size; @@ -322,10 +323,13 @@ static int objecttype_atom_parser(struct ref_format *format, struct used_atom *a { if (arg) return strbuf_addf_ret(err, -1, _("%%(objecttype) does not take arguments")); - if (atom->deref) + if (atom->deref) { oi_deref.info.typep = &oi_deref.type; - else + oi_deref.need_get_object_info = 1; + } else { oi.info.typep = &oi.type; + oi.need_get_object_info = 1; + } return 0; } @@ -334,16 +338,23 @@ static int objectsize_atom_parser(struct ref_format *format, struct used_atom *a { if (!arg) { atom->u.objectsize.option = O_SIZE; - if (atom->deref) + if (atom->deref) { oi_deref.info.sizep = &oi_deref.size; - else + oi_deref.need_get_object_info = 1; + } else { oi.info.sizep = &oi.size; + oi.need_get_object_info = 1; + } } else if (!strcmp(arg, "disk")) { atom->u.objectsize.option = O_SIZE_DISK; - if (atom->deref) + if (atom->deref) { oi_deref.info.disk_sizep = &oi_deref.disk_size; - else + oi_deref.need_get_object_info = 1; + } + else { oi.info.disk_sizep = &oi.disk_size; + oi.need_get_object_info = 1; + } } else return strbuf_addf_ret(err, -1, _("unrecognized %%(objectsize) argument: %s"), arg); return 0; @@ -354,10 +365,14 @@ static int deltabase_atom_parser(struct ref_format *format, struct used_atom *at { if (arg) return strbuf_addf_ret(err, -1, _("%%(deltabase) does not take arguments")); - if (atom->deref) + if (atom->deref) { oi_deref.info.delta_base_oid = &oi_deref.delta_base_oid; - else + oi_deref.need_get_object_info = 1; + + } else { oi.info.delta_base_oid = &oi.delta_base_oid; + oi.need_get_object_info = 1; + } return 0; } @@ -735,10 +750,13 @@ static int parse_ref_filter_atom(struct ref_format *format, used_atom[at].type = valid_atom[i].cmp_type; used_atom[at].source = valid_atom[i].source; if (used_atom[at].source == SOURCE_OBJ) { - if (deref) + if (deref) { oi_deref.info.contentp = &oi_deref.content; - else + oi_deref.need_get_object_info = 1; + } else { oi.info.contentp = &oi.content; + oi.need_get_object_info = 1; + } } if (arg) { arg = used_atom[at].name + (arg - sp) + 1; @@ -1910,7 +1928,6 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) struct object *obj; int i; int ret; - struct object_info empty = OBJECT_INFO_INIT; CALLOC_ARRAY(ref->value, used_atom_cnt); @@ -2078,10 +2095,11 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) oid_to_hex(&ref->objectname), ref->refname); } - if (need_tagged) + if (need_tagged) { oi.info.contentp = &oi.content; - if (!memcmp(&oi.info, &empty, sizeof(empty)) && - !memcmp(&oi_deref.info, &empty, sizeof(empty))) + oi.need_get_object_info = 1; + } + if (!oi.need_get_object_info && !oi_deref.need_get_object_info) return 0;