From patchwork Sun Jun 27 12:35:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE1ACC49EAB for ; Sun, 27 Jun 2021 12:36:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A00A461AC0 for ; Sun, 27 Jun 2021 12:36:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230282AbhF0MiX (ORCPT ); Sun, 27 Jun 2021 08:38:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230098AbhF0MiT (ORCPT ); Sun, 27 Jun 2021 08:38:19 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74567C061766 for ; Sun, 27 Jun 2021 05:35:54 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id b3so17012824wrm.6 for ; Sun, 27 Jun 2021 05:35:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=/+pHTUCgsn/eJhBt7TH6g2FT5CeygqVH9py2mRcVtgU=; b=SaA4xYzYKRjMaPP+DX4wgVzVbvX14FVZ+hQIgo9hHX1fKeNP3KapG3Z4TmoYxqovHb da/a0jDvWMg7CPXfns0iXf5Shz9f5tQf0OW0KD9kH5r+FKBssrs/Vd5qF1dJe8Zt6PDH Ym49PNFHP8ajtfpM2DhVxipvnqt6x6FhUvZEoc69EEFcRG/ok7GA51ZmrYLFvzhnMtA+ mUdmFI197ujDlZvayfONVKDmslQui3LxrCCr8mpsoJFJw6quMOPglel9NIU61YufIVjC mRyvucpnLqihdfZhHepsM8BXl+wdZkONPQl+B7JBcumpjVjaJStreav077Y1WPCBIzWZ KFHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=/+pHTUCgsn/eJhBt7TH6g2FT5CeygqVH9py2mRcVtgU=; b=deIh6bCo2HuZJKza6BrDB4wB60nUrzw/UFWwq2f92zFvDjphKtRBT1tWe4F+pjmsHC PGhmzjFZ90VmL7b3IlTypKFT1x/awEEFuxNySzkBZJpAT+u09E5W7OA6ogYjYvtT/KVC yJwHkM5z9svD8OKDv+Ql56QfhriWQmtjNbYB9naVU7Aw9rszdYTJM0aKPHGtQi/FAvQl k6SibseVORjNqfcUjG5TuUPviPFsc/oFsZCVyc0xDtXa25H3fV8iCGvFBzAacJthdzs5 0cTmSQD6nlfgYXmzhOyWKoSl7d0UGEeJkMGyXuHcYs5maQDowyZ3LHIf7gpHmYzfqULi 1m6g== X-Gm-Message-State: AOAM530Ei5Pfs83sQCwM5t2W8fjRXR1SehMFw0HJOGxm0Tzocq0oVd3j RzsrBBykPlq1KEuZDgsfflASNE5L5gw= X-Google-Smtp-Source: ABdhPJysa4Rb4UvhDasUNog1pHGsPQjNMkeH7oJSAXblWdVh39H8XIj2wo64eN56np2EL/IQ4e+Y2Q== X-Received: by 2002:a5d:6d83:: with SMTP id l3mr14342555wrs.91.1624797353104; Sun, 27 Jun 2021 05:35:53 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e8sm2425416wrx.26.2021.06.27.05.35.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:52 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:36 +0000 Subject: [PATCH v6 01/15] [GSOC] ref-filter: add obj-type check in grab contents Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Only tag and commit objects use `grab_sub_body_contents()` to grab object contents in the current codebase. We want to teach the function to also handle blobs and trees to get their raw data, without parsing a blob (whose contents looks like a commit or a tag) incorrectly as a commit or a tag. Skip the block of code that is specific to handling commits and tags early when the given object is of a wrong type to help later addition to handle other types of objects in this function. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Junio C Hamano Signed-off-by: ZheNing Hu --- ref-filter.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 4db0e40ff4c..5cee6512fba 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1356,11 +1356,12 @@ static void append_lines(struct strbuf *out, const char *buf, unsigned long size } /* See grab_values */ -static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) +static void grab_sub_body_contents(struct atom_value *val, int deref, struct expand_data *data) { int i; const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; + void *buf = data->content; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; @@ -1371,10 +1372,13 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) continue; if (deref) name++; - if (strcmp(name, "body") && - !starts_with(name, "subject") && - !starts_with(name, "trailers") && - !starts_with(name, "contents")) + + if ((data->type != OBJ_TAG && + data->type != OBJ_COMMIT) || + (strcmp(name, "body") && + !starts_with(name, "subject") && + !starts_with(name, "trailers") && + !starts_with(name, "contents"))) continue; if (!subpos) find_subpos(buf, @@ -1438,17 +1442,19 @@ static void fill_missing_values(struct atom_value *val) * pointed at by the ref itself; otherwise it is the object the * ref (which is a tag) refers to. */ -static void grab_values(struct atom_value *val, int deref, struct object *obj, void *buf) +static void grab_values(struct atom_value *val, int deref, struct object *obj, struct expand_data *data) { + void *buf = data->content; + switch (obj->type) { case OBJ_TAG: grab_tag_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("tagger", val, deref, buf); break; case OBJ_COMMIT: grab_commit_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("author", val, deref, buf); grab_person("committer", val, deref, buf); break; @@ -1678,7 +1684,7 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi->content); + grab_values(ref->value, deref, *obj, oi); } grab_common_values(ref->value, deref, oi); From patchwork Sun Jun 27 12:35:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3B27C48BC2 for ; Sun, 27 Jun 2021 12:35:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D9AAF61AC0 for ; Sun, 27 Jun 2021 12:35:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230236AbhF0MiU (ORCPT ); Sun, 27 Jun 2021 08:38:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230198AbhF0MiT (ORCPT ); Sun, 27 Jun 2021 08:38:19 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BE08C061574 for ; Sun, 27 Jun 2021 05:35:55 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id n23so9347029wms.2 for ; Sun, 27 Jun 2021 05:35:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=y3zDDhW/5m0r5Hf/3+JaFL8u0O65ubf+k2+MDP7k86o=; b=X/lqaBjvSqxsCkOnWqhadP77FqLmd+B1pJOJUz8ZfBUvJmTSjFXIscnMA74Kpq4Emg +r+glGs62CENKYc8W5oi0dycGmbDJhnw45lfRJCedcC6mJNYYjY+Tjm3ovdnwLfz2S5h meIHEcEjsRdBTKyYiCCZuz6L0GoW63DW10lWxvYXvaLvJSBXp3GcDrotB7C42ItvbDBr R6G6Sbc3DjGsG/ujQAELWYubfxMMl/i2+9v7INqSpamGydvZpZ/dc63vp/VhL++cVFXk TBgl/zXK1axLLh1yZcm6Q/qCB3JzJJDSg/YoNzPmK4EgTmYtg12vwdNbRr1/oDv5BVgv XNvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=y3zDDhW/5m0r5Hf/3+JaFL8u0O65ubf+k2+MDP7k86o=; b=Y9BzM0drYRzWzQTxzJCPo+Z/5cZjtfxzJpD58WxWiKG024UxxR5GVMF71AKaFgZL9r e/D5sQZRKWNZGaKXSwLGe8YWjyE37aSsxSnPoyzMTOse541er6EzhOSIgJYPohtSpzOd wrQKrB1EwkozTw23ptRUlSLZwXvZn/FpZRdB/IP1AOW7jMT+EcxUSDpB9ZVHxN+M0IL9 wV3yJsqk6VM0gmSWmueZbE/n/ACqUEFxzbCpo/qJWn74w8sNhCEqd/1nQTQYz5ctiI2M w4NA06sKDMM3kQZf2GdVog7kJmfN7RQ6YAzhaFe9E/eNdEQ14zRjyddHo/XIJrjICtm+ knkQ== X-Gm-Message-State: AOAM531QVR+WIKt1FCez0F5vLNjQ+qFsXKyXxnNn2EimKESnVqcG8fNl ayCYsRjoiqwzbovPxaQj3d2Cco3BIDs= X-Google-Smtp-Source: ABdhPJzafeXTfZKyXXxLPO1P38xHsXERJIZEi8drhNi0ZGUoKR6yDo9IQ5xaC5OKX+abPPP2cv0WFw== X-Received: by 2002:a05:600c:2248:: with SMTP id a8mr202751wmm.188.1624797353771; Sun, 27 Jun 2021 05:35:53 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y14sm12106329wrq.66.2021.06.27.05.35.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:53 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:37 +0000 Subject: [PATCH v6 02/15] [GSOC] ref-filter: add %(raw) atom MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add new formatting option `%(raw)`, which will print the raw object data without any changes. It will help further to migrate all cat-file formatting logic from cat-file to ref-filter. The raw data of blob, tree objects may contain '\0', but most of the logic in `ref-filter` depends on the output of the atom being text (specifically, no embedded NULs in it). E.g. `quote_formatting()` use `strbuf_addstr()` or `*._quote_buf()` add the data to the buffer. The raw data of a tree object is `100644 one\0...`, only the `100644 one` will be added to the buffer, which is incorrect. Therefore, we need to find a way to record the length of the atom_value's member `s`. Although strbuf can already record the string and its length, if we want to replace the type of atom_value's member `s` with strbuf, many places in ref-filter that are filled with dynamically allocated mermory in `v->s` are not easy to replace. At the same time, we need to check if `v->s == NULL` in populate_value(), and strbuf cannot easily distinguish NULL and empty strings, but c-style "const char *" can do it. So add a new member in `struct atom_value`: `s_size`, which can record raw object size, it can help us add raw object data to the buffer or compare two buffers which contain raw object data. Note that `--format=%(raw)` cannot be used with `--python`, `--shell`, `--tcl`, and `--perl` because if the binary raw data is passed to a variable in such languages, these may not support arbitrary binary data in their string variable type. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Bagas Sanjaya Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Felipe Contreras Helped-by: Phillip Wood Helped-by: Junio C Hamano Based-on-patch-by: Olga Telezhnaya Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 9 ++ ref-filter.c | 139 +++++++++++++++---- t/t6300-for-each-ref.sh | 216 +++++++++++++++++++++++++++++ 3 files changed, 337 insertions(+), 27 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 2ae2478de70..3727a5ffee7 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -235,6 +235,15 @@ and `date` to extract the named component. For email fields (`authoremail`, without angle brackets, and `:localpart` to get the part before the `@` symbol out of the trimmed email. +The raw data in an object is `raw`. + +raw:size:: + The raw data size of the object. + +Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, +`--perl` because the such language may not support arbitrary binary data in their +string variable type. + The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/ref-filter.c b/ref-filter.c index 5cee6512fba..7822be90307 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -144,6 +144,7 @@ enum atom_type { ATOM_BODY, ATOM_TRAILERS, ATOM_CONTENTS, + ATOM_RAW, ATOM_UPSTREAM, ATOM_PUSH, ATOM_SYMREF, @@ -189,6 +190,9 @@ static struct used_atom { struct process_trailer_options trailer_opts; unsigned int nlines; } contents; + struct { + enum { RAW_BARE, RAW_LENGTH } option; + } raw_data; struct { cmp_status cmp_status; const char *str; @@ -426,6 +430,18 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } +static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (!arg) + atom->u.raw_data.option = RAW_BARE; + else if (!strcmp(arg, "size")) + atom->u.raw_data.option = RAW_LENGTH; + else + return strbuf_addf_ret(err, -1, _("unrecognized %%(raw) argument: %s"), arg); + return 0; +} + static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { @@ -586,6 +602,7 @@ static struct { [ATOM_BODY] = { "body", SOURCE_OBJ, FIELD_STR, body_atom_parser }, [ATOM_TRAILERS] = { "trailers", SOURCE_OBJ, FIELD_STR, trailers_atom_parser }, [ATOM_CONTENTS] = { "contents", SOURCE_OBJ, FIELD_STR, contents_atom_parser }, + [ATOM_RAW] = { "raw", SOURCE_OBJ, FIELD_STR, raw_atom_parser }, [ATOM_UPSTREAM] = { "upstream", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_PUSH] = { "push", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_SYMREF] = { "symref", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -620,12 +637,15 @@ struct ref_formatting_state { struct atom_value { const char *s; + size_t s_size; int (*handler)(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err); uintmax_t value; /* used for sorting when not FIELD_STR */ struct used_atom *atom; }; +#define ATOM_VALUE_S_SIZE_INIT (-1) + /* * Used to parse format string and sort specifiers */ @@ -644,13 +664,6 @@ static int parse_ref_filter_atom(const struct ref_format *format, return strbuf_addf_ret(err, -1, _("malformed field name: %.*s"), (int)(ep-atom), atom); - /* Do we have the atom already used elsewhere? */ - for (i = 0; i < used_atom_cnt; i++) { - int len = strlen(used_atom[i].name); - if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) - return i; - } - /* * If the atom name has a colon, strip it and everything after * it off - it specifies the format for this entry, and @@ -660,6 +673,13 @@ static int parse_ref_filter_atom(const struct ref_format *format, arg = memchr(sp, ':', ep - sp); atom_len = (arg ? arg : ep) - sp; + /* Do we have the atom already used elsewhere? */ + for (i = 0; i < used_atom_cnt; i++) { + int len = strlen(used_atom[i].name); + if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) + return i; + } + /* Is the atom a valid one? */ for (i = 0; i < ARRAY_SIZE(valid_atom); i++) { int len = strlen(valid_atom[i].name); @@ -709,11 +729,14 @@ static int parse_ref_filter_atom(const struct ref_format *format, return at; } -static void quote_formatting(struct strbuf *s, const char *str, int quote_style) +static void quote_formatting(struct strbuf *s, const char *str, size_t len, int quote_style) { switch (quote_style) { case QUOTE_NONE: - strbuf_addstr(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(s, str, len); + else + strbuf_addstr(s, str); break; case QUOTE_SHELL: sq_quote_buf(s, str); @@ -740,9 +763,12 @@ static int append_atom(struct atom_value *v, struct ref_formatting_state *state, * encountered. */ if (!state->stack->prev) - quote_formatting(&state->stack->output, v->s, state->quote_style); + quote_formatting(&state->stack->output, v->s, v->s_size, state->quote_style); else - strbuf_addstr(&state->stack->output, v->s); + if (v->s_size != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(&state->stack->output, v->s, v->s_size); + else + strbuf_addstr(&state->stack->output, v->s); return 0; } @@ -842,21 +868,23 @@ static int if_atom_handler(struct atom_value *atomv, struct ref_formatting_state return 0; } -static int is_empty(const char *s) +static int is_empty(struct strbuf *buf) { - while (*s != '\0') { - if (!isspace(*s)) - return 0; - s++; - } - return 1; -} + const char *cur = buf->buf; + const char *end = buf->buf + buf->len; + + while (cur != end && (isspace(*cur))) + cur++; + + return cur == end; + } static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err) { struct ref_formatting_stack *cur = state->stack; struct if_then_else *if_then_else = NULL; + size_t str_len = 0; if (cur->at_end == if_then_else_handler) if_then_else = (struct if_then_else *)cur->at_end_data; @@ -867,18 +895,22 @@ static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_sta if (if_then_else->else_atom_seen) return strbuf_addf_ret(err, -1, _("format: %%(then) atom used after %%(else)")); if_then_else->then_atom_seen = 1; + if (if_then_else->str) + str_len = strlen(if_then_else->str); /* * If the 'equals' or 'notequals' attribute is used then * perform the required comparison. If not, only non-empty * strings satisfy the 'if' condition. */ if (if_then_else->cmp_status == COMPARE_EQUAL) { - if (!strcmp(if_then_else->str, cur->output.buf)) + if (str_len == cur->output.len && + !memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; } else if (if_then_else->cmp_status == COMPARE_UNEQUAL) { - if (strcmp(if_then_else->str, cur->output.buf)) + if (str_len != cur->output.len || + memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; - } else if (cur->output.len && !is_empty(cur->output.buf)) + } else if (cur->output.len && !is_empty(&cur->output)) if_then_else->condition_satisfied = 1; strbuf_reset(&cur->output); return 0; @@ -924,7 +956,7 @@ static int end_atom_handler(struct atom_value *atomv, struct ref_formatting_stat * only on the topmost supporting atom. */ if (!current->prev->prev) { - quote_formatting(&s, current->output.buf, state->quote_style); + quote_formatting(&s, current->output.buf, current->output.len, state->quote_style); strbuf_swap(¤t->output, &s); } strbuf_release(&s); @@ -974,6 +1006,10 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) + die(_("--format=%.*s cannot be used with" + "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) @@ -1362,17 +1398,29 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; void *buf = data->content; + unsigned long buf_size = data->size; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; const char *name = atom->name; struct atom_value *v = &val[i]; + enum atom_type atom_type = atom->atom_type; if (!!deref != (*name == '*')) continue; if (deref) name++; + if (atom_type == ATOM_RAW) { + if (atom->u.raw_data.option == RAW_BARE) { + v->s = xmemdupz(buf, buf_size); + v->s_size = buf_size; + } else if (atom->u.raw_data.option == RAW_LENGTH) { + v->s = xstrfmt("%"PRIuMAX, (uintmax_t)buf_size); + } + continue; + } + if ((data->type != OBJ_TAG && data->type != OBJ_COMMIT) || (strcmp(name, "body") && @@ -1460,9 +1508,11 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s break; case OBJ_TREE: /* grab_tree_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; case OBJ_BLOB: /* grab_blob_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; default: die("Eh? Object of type %d?", obj->type); @@ -1766,6 +1816,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) const char *refname; struct branch *branch = NULL; + v->s_size = ATOM_VALUE_S_SIZE_INIT; v->handler = append_atom; v->atom = atom; @@ -2369,6 +2420,19 @@ static int compare_detached_head(struct ref_array_item *a, struct ref_array_item return 0; } +static int memcasecmp(const void *vs1, const void *vs2, size_t n) +{ + const char *s1 = vs1, *s2 = vs2; + const char *end = s1 + n; + + for (; s1 < end; s1++, s2++) { + int diff = tolower(*s1) - tolower(*s2); + if (diff) + return diff; + } + return 0; +} + static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, struct ref_array_item *b) { struct atom_value *va, *vb; @@ -2389,10 +2453,30 @@ static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, stru } else if (s->sort_flags & REF_SORTING_VERSION) { cmp = versioncmp(va->s, vb->s); } else if (cmp_type == FIELD_STR) { - int (*cmp_fn)(const char *, const char *); - cmp_fn = s->sort_flags & REF_SORTING_ICASE - ? strcasecmp : strcmp; - cmp = cmp_fn(va->s, vb->s); + if (va->s_size == ATOM_VALUE_S_SIZE_INIT && + vb->s_size == ATOM_VALUE_S_SIZE_INIT) { + int (*cmp_fn)(const char *, const char *); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? strcasecmp : strcmp; + cmp = cmp_fn(va->s, vb->s); + } else { + size_t a_size = va->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(va->s) : va->s_size; + size_t b_size = vb->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(vb->s) : vb->s_size; + int (*cmp_fn)(const void *, const void *, size_t); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? memcasecmp : memcmp; + + cmp = cmp_fn(va->s, vb->s, b_size > a_size ? + a_size : b_size); + if (!cmp) { + if (a_size > b_size) + cmp = 1; + else if (a_size < b_size) + cmp = -1; + } + } } else { if (va->value < vb->value) cmp = -1; @@ -2492,6 +2576,7 @@ int format_ref_array_item(struct ref_array_item *info, } if (format->need_color_reset_at_eol) { struct atom_value resetv; + resetv.s_size = ATOM_VALUE_S_SIZE_INIT; resetv.s = GIT_COLOR_RESET; if (append_atom(&resetv, &state, error_buf)) { pop_stack_element(&state.stack); diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 9e0214076b4..18554f62d94 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -130,6 +130,8 @@ test_atom head parent:short=10 '' test_atom head numparent 0 test_atom head object '' test_atom head type '' +test_atom head raw "$(git cat-file commit refs/heads/main) +" test_atom head '*objectname' '' test_atom head '*objecttype' '' test_atom head author 'A U Thor 1151968724 +0200' @@ -221,6 +223,15 @@ test_atom tag contents 'Tagging at 1151968727 ' test_atom tag HEAD ' ' +test_expect_success 'basic atom: refs/tags/testtag *raw' ' + git cat-file commit refs/tags/testtag^{} >expected && + git for-each-ref --format="%(*raw)" refs/tags/testtag >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'Check invalid atoms names are errors' ' test_must_fail git for-each-ref --format="%(INVALID)" refs/heads ' @@ -686,6 +697,15 @@ test_atom refs/tags/signed-empty contents:body '' test_atom refs/tags/signed-empty contents:signature "$sig" test_atom refs/tags/signed-empty contents "$sig" +test_expect_success GPG 'basic atom: refs/tags/signed-empty raw' ' + git cat-file tag refs/tags/signed-empty >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-empty >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-short subject 'subject line' test_atom refs/tags/signed-short subject:sanitize 'subject-line' test_atom refs/tags/signed-short contents:subject 'subject line' @@ -695,6 +715,15 @@ test_atom refs/tags/signed-short contents:signature "$sig" test_atom refs/tags/signed-short contents "subject line $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-short raw' ' + git cat-file tag refs/tags/signed-short >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-short >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-long subject 'subject line' test_atom refs/tags/signed-long subject:sanitize 'subject-line' test_atom refs/tags/signed-long contents:subject 'subject line' @@ -708,6 +737,15 @@ test_atom refs/tags/signed-long contents "subject line body contents $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-long raw' ' + git cat-file tag refs/tags/signed-long >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-long >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'set up refs pointing to tree and blob' ' git update-ref refs/mytrees/first refs/heads/main^{tree} && git update-ref refs/myblobs/first refs/heads/main:one @@ -720,6 +758,16 @@ test_atom refs/mytrees/first contents:body "" test_atom refs/mytrees/first contents:signature "" test_atom refs/mytrees/first contents "" +test_expect_success 'basic atom: refs/mytrees/first raw' ' + git cat-file tree refs/mytrees/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/mytrees/first >actual && + test_cmp expected actual && + git cat-file -s refs/mytrees/first >expected && + git for-each-ref --format="%(raw:size)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_atom refs/myblobs/first subject "" test_atom refs/myblobs/first contents:subject "" test_atom refs/myblobs/first body "" @@ -727,6 +775,174 @@ test_atom refs/myblobs/first contents:body "" test_atom refs/myblobs/first contents:signature "" test_atom refs/myblobs/first contents "" +test_expect_success 'basic atom: refs/myblobs/first raw' ' + git cat-file blob refs/myblobs/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/myblobs/first >actual && + test_cmp expected actual && + git cat-file -s refs/myblobs/first >expected && + git for-each-ref --format="%(raw:size)" refs/myblobs/first >actual && + test_cmp expected actual +' + +test_expect_success 'set up refs pointing to binary blob' ' + printf "a\0b\0c" >blob1 && + printf "a\0c\0b" >blob2 && + printf "\0a\0b\0c" >blob3 && + printf "abc" >blob4 && + printf "\0 \0 \0 " >blob5 && + printf "\0 \0a\0 " >blob6 && + printf " " >blob7 && + >blob8 && + obj=$(git hash-object -w blob1) && + git update-ref refs/myblobs/blob1 "$obj" && + obj=$(git hash-object -w blob2) && + git update-ref refs/myblobs/blob2 "$obj" && + obj=$(git hash-object -w blob3) && + git update-ref refs/myblobs/blob3 "$obj" && + obj=$(git hash-object -w blob4) && + git update-ref refs/myblobs/blob4 "$obj" && + obj=$(git hash-object -w blob5) && + git update-ref refs/myblobs/blob5 "$obj" && + obj=$(git hash-object -w blob6) && + git update-ref refs/myblobs/blob6 "$obj" && + obj=$(git hash-object -w blob7) && + git update-ref refs/myblobs/blob7 "$obj" && + obj=$(git hash-object -w blob8) && + git update-ref refs/myblobs/blob8 "$obj" +' + +test_expect_success 'Verify sorts with raw' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob3 + refs/myblobs/blob7 + refs/mytrees/first + refs/myblobs/first + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob4 + refs/heads/main + EOF + git for-each-ref --format="%(refname)" --sort=raw \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'Verify sorts with raw:size' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/first + refs/myblobs/blob7 + refs/heads/main + refs/myblobs/blob4 + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/mytrees/first + EOF + git for-each-ref --format="%(refname)" --sort=raw:size \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:equals)' ' + cat >expected <<-EOF && + not equals + not equals + not equals + not equals + not equals + not equals + refs/myblobs/blob4 + not equals + not equals + not equals + not equals + not equals + EOF + git for-each-ref --format="%(if:equals=abc)%(raw)%(then)%(refname)%(else)not equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:notequals)' ' + cat >expected <<-EOF && + refs/heads/ambiguous + refs/heads/main + refs/heads/newtag + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + equals + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob7 + refs/myblobs/blob8 + refs/myblobs/first + EOF + git for-each-ref --format="%(if:notequals=abc)%(raw)%(then)%(refname)%(else)equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'empty raw refs with %(if)' ' + cat >expected <<-EOF && + refs/myblobs/blob1 not empty + refs/myblobs/blob2 not empty + refs/myblobs/blob3 not empty + refs/myblobs/blob4 not empty + refs/myblobs/blob5 not empty + refs/myblobs/blob6 not empty + refs/myblobs/blob7 empty + refs/myblobs/blob8 empty + refs/myblobs/first not empty + EOF + git for-each-ref --format="%(refname) %(if)%(raw)%(then)not empty%(else)empty%(end)" \ + refs/myblobs/ >actual && + test_cmp expected actual +' + +test_expect_success '%(raw) with --python must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --python +' + +test_expect_success '%(raw) with --tcl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --tcl +' + +test_expect_success '%(raw) with --perl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --perl +' + +test_expect_success '%(raw) with --shell must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --shell +' + +test_expect_success '%(raw) with --shell and --sort=raw must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --sort=raw --shell +' + +test_expect_success '%(raw:size) with --shell' ' + git for-each-ref --format="%(raw:size)" | while read line + do + echo "'\''$line'\''" >>expect + done && + git for-each-ref --format="%(raw:size)" --shell >actual && + test_cmp expect actual +' + +test_expect_success 'for-each-ref --format compare with cat-file --batch' ' + git rev-parse refs/mytrees/first | git cat-file --batch >expected && + git for-each-ref --format="%(objectname) %(objecttype) %(objectsize) +%(raw)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_expect_success 'set up multiple-sort tags' ' for when in 100000 200000 do From patchwork Sun Jun 27 12:35:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEFC2C49EAF for ; Sun, 27 Jun 2021 12:36:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE0AC61C17 for ; Sun, 27 Jun 2021 12:36:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230288AbhF0MiY (ORCPT ); Sun, 27 Jun 2021 08:38:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230237AbhF0MiU (ORCPT ); Sun, 27 Jun 2021 08:38:20 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E061BC061766 for ; Sun, 27 Jun 2021 05:35:55 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id i94so17057557wri.4 for ; Sun, 27 Jun 2021 05:35:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=MIyCppMRlmD0nHG20ysXgO9V8sX6FVxpBpdBK4yOAkU=; b=bYrWCOrUWX1Y4bTelbaoR4HGpI4oBA/E30afVqza0RPQyIKVT/RCjZrSnEACFKQx9a 7Afp6vWzZVXILqMzmtG8h2mwE7i7+A9AfZoVQ/W4RZgcRvTVkYcwNLMtJI70X7WRvSeW eFFkk82gp2b/u9WV01cSdGOxJbGayKu4hhTv7sZ3zF8ynxl4PvyCU3BLf0ta3/BK/0ND KLV+NEI9uDbyK4VB4zAAQjPGZS77VwmZNSS+h2nd4ISlbdgBqlweYFZSc3Ou/Zku+SG7 +05uKCej6oliEiff5S7noddG0FQmp7JGAKIrZP+4bYlYqxx54cgNaRHsorCUTx1Sb2sW W5lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=MIyCppMRlmD0nHG20ysXgO9V8sX6FVxpBpdBK4yOAkU=; b=FDD+vP8jMbBamIaf9yE53pd+9TbXTbJpRKKWlJSoi+OPMWO67ZefjnWbP4XgBlb2da eO/SuUaY5CZQIOffBMlrrsY5wV9/vcgpqV/jNX15I82PKxTfbvV7Po7VT3DqWmVMgzW0 u/TPtXsvNO9YwPUOuNNEPZ5WVGVyM7GRlat2M8ESkJMJo59lVk1m7oQzgB/ZCTNg8YX5 xwkue0P18R5VWqyRKd2fnzT7MCrIucd/DDSHjlqfUS5ZjHJ0Lm2qEFMjSNRsjqfETc4b fPCcEwI4mofOJ5682SG720gCa4sQE5zfps0AJb8s6YBRyNZKr9S+9ZWXe+VxOIjOjINB 2coQ== X-Gm-Message-State: AOAM530Ht7svKDef9a9jbul06ElObYt4mil1RTXm3NNSYvdnzE8g2YAr zym/djSZHx6vqyDwyg3qDujoHcJKYto= X-Google-Smtp-Source: ABdhPJx6wa0mNqlbodAL+G8GjBkiubCldubiPuoS17O8XHMTRupanrQuHSKrYFeFN1R3zEEkSBB7kQ== X-Received: by 2002:a05:6000:511:: with SMTP id a17mr21915898wrf.351.1624797354528; Sun, 27 Jun 2021 05:35:54 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m18sm10865572wmq.45.2021.06.27.05.35.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:54 -0700 (PDT) Message-Id: <47f868f63d92c3dee1e62617fdc9b5d95a411d07.1624797351.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:38 +0000 Subject: [PATCH v6 03/15] [GSOC] ref-filter: --format=%(raw) re-support --perl MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because the perl language can handle binary data correctly, add the function perl_quote_buf_with_len(), which can specify the length of the data and prevent the data from being truncated at '\0' to help `--format="%(raw)"` re-support `--perl`. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 2 +- quote.c | 17 +++++++++++++++++ quote.h | 1 + ref-filter.c | 15 +++++++++++---- t/t6300-for-each-ref.sh | 19 +++++++++++++++++-- 5 files changed, 47 insertions(+), 7 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 3727a5ffee7..6f970088f46 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -241,7 +241,7 @@ raw:size:: The raw data size of the object. Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, -`--perl` because the such language may not support arbitrary binary data in their +because the such language may not support arbitrary binary data in their string variable type. The message in a commit or a tag object is `contents`, from which diff --git a/quote.c b/quote.c index 8a3a5e39eb1..26719d21d1e 100644 --- a/quote.c +++ b/quote.c @@ -471,6 +471,23 @@ void perl_quote_buf(struct strbuf *sb, const char *src) strbuf_addch(sb, sq); } +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len) +{ + const char sq = '\''; + const char bq = '\\'; + const char *c = src; + const char *end = src + len; + + strbuf_addch(sb, sq); + while (c != end) { + if (*c == sq || *c == bq) + strbuf_addch(sb, bq); + strbuf_addch(sb, *c); + c++; + } + strbuf_addch(sb, sq); +} + void python_quote_buf(struct strbuf *sb, const char *src) { const char sq = '\''; diff --git a/quote.h b/quote.h index 768cc6338e2..0fe69e264b0 100644 --- a/quote.h +++ b/quote.h @@ -94,6 +94,7 @@ char *quote_path(const char *in, const char *prefix, struct strbuf *out, unsigne /* quoting as a string literal for other languages */ void perl_quote_buf(struct strbuf *sb, const char *src); +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len); void python_quote_buf(struct strbuf *sb, const char *src); void tcl_quote_buf(struct strbuf *sb, const char *src); void basic_regex_quote_buf(struct strbuf *sb, const char *src); diff --git a/ref-filter.c b/ref-filter.c index 7822be90307..797b20ffa61 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -742,7 +742,10 @@ static void quote_formatting(struct strbuf *s, const char *str, size_t len, int sq_quote_buf(s, str); break; case QUOTE_PERL: - perl_quote_buf(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + perl_quote_buf_with_len(s, str, len); + else + perl_quote_buf(s, str); break; case QUOTE_PYTHON: python_quote_buf(s, str); @@ -1006,10 +1009,14 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && - used_atom[at].u.raw_data.option == RAW_BARE) + + if ((format->quote_style == QUOTE_PYTHON || + format->quote_style == QUOTE_SHELL || + format->quote_style == QUOTE_TCL) && + used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) die(_("--format=%.*s cannot be used with" - "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); + "--python, --shell, --tcl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 18554f62d94..0b66e743c58 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -915,8 +915,23 @@ test_expect_success '%(raw) with --tcl must fail' ' test_must_fail git for-each-ref --format="%(raw)" --tcl ' -test_expect_success '%(raw) with --perl must fail' ' - test_must_fail git for-each-ref --format="%(raw)" --perl +test_expect_success '%(raw) with --perl' ' + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob1 --perl | perl > actual && + cmp blob1 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob3 --perl | perl > actual && + cmp blob3 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob8 --perl | perl > actual && + cmp blob8 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/first --perl | perl > actual && + cmp one actual && + git cat-file tree refs/mytrees/first > expected && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/mytrees/first --perl | perl > actual && + cmp expected actual ' test_expect_success '%(raw) with --shell must fail' ' From patchwork Sun Jun 27 12:35:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D2BAC48BC2 for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 83D7661C47 for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230326AbhF0MiZ (ORCPT ); Sun, 27 Jun 2021 08:38:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230232AbhF0MiU (ORCPT ); Sun, 27 Jun 2021 08:38:20 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75052C061574 for ; Sun, 27 Jun 2021 05:35:56 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id u11so16998197wrw.11 for ; Sun, 27 Jun 2021 05:35:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=KO8wLmUvbUfkVrtYPM8dGAeXgDDnPeti6Zejn/XfVtY=; b=QjFMV1ZtNG8z9TmAUk90v38ke/ySR0Na9jUG02PDvqs7+zNRC4pr+yCBLxv8xIJKTG Us4u0OTbXE4SfLx/OcUAVCJFC4pLpwZZLX3rmaqrl2GH8K4MWR9yvK7/tcd6WS34eaqE 95faGK4hw242GRp6YEez7mmKCfrcs1J6WCJ0bJDFNVA6RivWCQZxswPaTW6ppF0GOlsA 2I/WiZa9V9f6+6REtYy+/h1n3THkRAjI7m2yF/oq5n6Q22BxUzDm4y01EodkqHUjFN/K r8LinMAx3rDo5YYqkPJXFpPRhTwL9eCljIWWdIFbSLswrTgQiBtJLFtcb5eJlNUt3+4U 1bGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=KO8wLmUvbUfkVrtYPM8dGAeXgDDnPeti6Zejn/XfVtY=; b=sXGuX8LbMkcVBwhlwoFAqXQNUwSHiSrgVD2Kk+US7uw+jSZ8cXbg2bSFGBdoumDOu6 8CHag7nPdUFW69WGaBSRC6fQZoMTPEDvmPZ/VqcJ17hCK9hWIBCMyZdB2UzwPawoqDvB FLIxEvvrgLi/KWfhb/Yi9I6tL3GXDczJ1q8fPNMdwNQCqwjxBS+hXU5xg1xvFIovbAoA 6vTdcQXn89NGc2laMP5ISuN+fErevTrt3DwC2RzkOs9RGO+ZOwswxFp2Tsb9Xd3ggpe2 qIEN9z9mvVvfJ3NznqIbk6Kkn5L6dyGA5xholIreTU1LirFDWjFUQBzc4wuugnMuhCDe lzsQ== X-Gm-Message-State: AOAM5326Ytqg940xrF4mZ2ncwo96gbu/kkb3p4RcJOEX73Iq2hX7qHJy zdL+5Hjwf5WzNC8a+NAtsZStx5jJDgE= X-Google-Smtp-Source: ABdhPJy+3o4iX4pBRo7qZrEV3U8glwWXvZHRkTiRjLAZ+riEvPmghocqZl73Fw9rd+ns9+yTtTN36A== X-Received: by 2002:adf:df88:: with SMTP id z8mr21573573wrl.330.1624797355061; Sun, 27 Jun 2021 05:35:55 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d4sm10762860wmd.42.2021.06.27.05.35.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:54 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:39 +0000 Subject: [PATCH v6 04/15] [GSOC] ref-filter: use non-const ref_format in *_atom_parser() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use non-const ref_format in *_atom_parser(), which can help us modify the members of ref_format in *_atom_parser(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/tag.c | 2 +- ref-filter.c | 44 ++++++++++++++++++++++---------------------- ref-filter.h | 4 ++-- 3 files changed, 25 insertions(+), 25 deletions(-) diff --git a/builtin/tag.c b/builtin/tag.c index 82fcfc09824..452558ec957 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -146,7 +146,7 @@ static int verify_tag(const char *name, const char *ref, const struct object_id *oid, void *cb_data) { int flags; - const struct ref_format *format = cb_data; + struct ref_format *format = cb_data; flags = GPG_VERIFY_VERBOSE; if (format->format) diff --git a/ref-filter.c b/ref-filter.c index 797b20ffa61..d01a0266fb8 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -226,7 +226,7 @@ static int strbuf_addf_ret(struct strbuf *sb, int ret, const char *fmt, ...) return ret; } -static int color_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int color_atom_parser(struct ref_format *format, struct used_atom *atom, const char *color_value, struct strbuf *err) { if (!color_value) @@ -264,7 +264,7 @@ static int refname_atom_parser_internal(struct refname_atom *atom, const char *a return 0; } -static int remote_ref_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int remote_ref_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct string_list params = STRING_LIST_INIT_DUP; @@ -311,7 +311,7 @@ static int remote_ref_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objecttype_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objecttype_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -323,7 +323,7 @@ static int objecttype_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objectsize_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objectsize_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -343,7 +343,7 @@ static int objectsize_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int deltabase_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int deltabase_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -355,7 +355,7 @@ static int deltabase_atom_parser(const struct ref_format *format, struct used_at return 0; } -static int body_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int body_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -364,7 +364,7 @@ static int body_atom_parser(const struct ref_format *format, struct used_atom *a return 0; } -static int subject_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int subject_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -376,7 +376,7 @@ static int subject_atom_parser(const struct ref_format *format, struct used_atom return 0; } -static int trailers_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int trailers_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { atom->u.contents.trailer_opts.no_divider = 1; @@ -402,7 +402,7 @@ static int trailers_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int contents_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int contents_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -430,7 +430,7 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int raw_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -442,7 +442,7 @@ static int raw_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int oid_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -461,7 +461,7 @@ static int oid_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int person_email_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int person_email_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -475,7 +475,7 @@ static int person_email_atom_parser(const struct ref_format *format, struct used return 0; } -static int refname_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int refname_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { return refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err); @@ -492,7 +492,7 @@ static align_type parse_align_position(const char *s) return -1; } -static int align_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int align_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct align *align = &atom->u.align; @@ -544,7 +544,7 @@ static int align_atom_parser(const struct ref_format *format, struct used_atom * return 0; } -static int if_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int if_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -559,7 +559,7 @@ static int if_atom_parser(const struct ref_format *format, struct used_atom *ato return 0; } -static int head_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { atom->u.head = resolve_refdup("HEAD", RESOLVE_REF_READING, NULL, NULL); @@ -570,7 +570,7 @@ static struct { const char *name; info_source source; cmp_type cmp_type; - int (*parser)(const struct ref_format *format, struct used_atom *atom, + int (*parser)(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err); } valid_atom[] = { [ATOM_REFNAME] = { "refname", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -649,7 +649,7 @@ struct atom_value { /* * Used to parse format string and sort specifiers */ -static int parse_ref_filter_atom(const struct ref_format *format, +static int parse_ref_filter_atom(struct ref_format *format, const char *atom, const char *ep, struct strbuf *err) { @@ -2553,9 +2553,9 @@ static void append_literal(const char *cp, const char *ep, struct ref_formatting } int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, - struct strbuf *final_buf, - struct strbuf *error_buf) + struct ref_format *format, + struct strbuf *final_buf, + struct strbuf *error_buf) { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; @@ -2600,7 +2600,7 @@ int format_ref_array_item(struct ref_array_item *info, } void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format) + struct ref_format *format) { struct ref_array_item *ref_item; struct strbuf output = STRBUF_INIT; diff --git a/ref-filter.h b/ref-filter.h index baf72a71896..74fb423fc89 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -116,7 +116,7 @@ void ref_array_sort(struct ref_sorting *sort, struct ref_array *array); void ref_sorting_set_sort_flags_all(struct ref_sorting *sorting, unsigned int mask, int on); /* Based on the given format and quote_style, fill the strbuf */ int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, + struct ref_format *format, struct strbuf *final_buf, struct strbuf *error_buf); /* Parse a single sort specifier and add it to the list */ @@ -137,7 +137,7 @@ void setup_ref_filter_porcelain_msg(void); * name must be a fully qualified refname. */ void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format); + struct ref_format *format); /* * Push a single ref onto the array; this can be used to construct your own From patchwork Sun Jun 27 12:35:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D93E1C49EA6 for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B9D8461C51 for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230341AbhF0Mi1 (ORCPT ); Sun, 27 Jun 2021 08:38:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230241AbhF0MiV (ORCPT ); Sun, 27 Jun 2021 08:38:21 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3CD9C061767 for ; Sun, 27 Jun 2021 05:35:56 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id b3so17012891wrm.6 for ; Sun, 27 Jun 2021 05:35:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QFBlWXIncwttHrgd04N7RqxYe8ew5peYTqRIybnr11g=; b=A1hKqDIvB4ovhoWyA3yHnwCVnOyy9NRwFNCc7wT2h8lHJHwfo85hVuuIs6dcPGHl+p ax7DsYB1yhj2JfGj+PRsnSNLsPl4IItmQkfZ8UhXAJR0JEzwbvCus0udToAq5L7O8NRu PTCOzFmsaiPC9wmsK06M/Kye2cRTm7V1ZnV6QoHqXuONU2eCQzWQyFvNhR2ZeR3A11rf isBUAymyWf3JRkV1adI4M4AOHmvQOpbGViSs+vXpe5ApPTu276pmsZGrtBA5BIrefGUo Z1KRr0ykLAmUgOWOsOIQFM9CIVm9XfuRZouwVOSw2k/hnR4+d+s/yD6MIcMwm9DtEmdc 7iaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QFBlWXIncwttHrgd04N7RqxYe8ew5peYTqRIybnr11g=; b=tggFA4hOlCMfCmSGcUGBIFi3yp6wQuW+tdkwgeXKtJO8Jr65k4R3Qa8Mu+zRsIbOir Zu6CzQV2lKOl0ZNBEUNv2l81/F8CbIQ2+7V6/JVE+P3voIfvVWqbVSv/+xJXoJKc/4bJ yvRnf23YZd6WCVRJJp99tokudMN2Tht6UFmUmXzOeIO8arwOGHRDO/Xi2s8muRHwI997 ZGKl/SvhrWpS19OtX5n0iNeePrCkZf/BirvxN2KikUNVu3JO34ojqDcrV5VNkaY0kiJU GG6PU2FVnGt+h8gO5VD02w1pConmMjQYwVCDakfoFiKXFIEOZX2sPvnJCtk2KH+S4f1a akvA== X-Gm-Message-State: AOAM533IqoJIbMNEBzhYzUyIYPf6uixOgLLRm3Y/iQURRKjI+yuiaaby Uh6JSPyhuBYYUJ+RPFfrnGb0m1sgH/o= X-Google-Smtp-Source: ABdhPJyYBBU8op2E50UtuA39nsTJxNAq8aXQZ8II2HrL31LEMkdQ49EqWJTGE2n1qnS2w//9+hwQOQ== X-Received: by 2002:a5d:6848:: with SMTP id o8mr22212728wrw.352.1624797355599; Sun, 27 Jun 2021 05:35:55 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h9sm10215291wmb.35.2021.06.27.05.35.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:55 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:40 +0000 Subject: [PATCH v6 05/15] [GSOC] ref-filter: add %(rest) atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let "cat-file --batch=%(rest)" use the ref-filter interface, add %(rest) atom for ref-filter. "git for-each-ref", "git branch", "git tag" and "git verify-tag" will reject %(rest) by default. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 21 +++++++++++++++++++++ ref-filter.h | 5 ++++- t/t3203-branch-output.sh | 4 ++++ t/t6300-for-each-ref.sh | 4 ++++ t/t7004-tag.sh | 4 ++++ t/t7030-verify-tag.sh | 4 ++++ 6 files changed, 41 insertions(+), 1 deletion(-) diff --git a/ref-filter.c b/ref-filter.c index d01a0266fb8..10c78de9cfa 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -157,6 +157,7 @@ enum atom_type { ATOM_IF, ATOM_THEN, ATOM_ELSE, + ATOM_REST, }; /* @@ -559,6 +560,15 @@ static int if_atom_parser(struct ref_format *format, struct used_atom *atom, return 0; } +static int rest_atom_parser(struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (arg) + return strbuf_addf_ret(err, -1, _("%%(rest) does not take arguments")); + format->use_rest = 1; + return 0; +} + static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { @@ -615,6 +625,7 @@ static struct { [ATOM_IF] = { "if", SOURCE_NONE, FIELD_STR, if_atom_parser }, [ATOM_THEN] = { "then", SOURCE_NONE }, [ATOM_ELSE] = { "else", SOURCE_NONE }, + [ATOM_REST] = { "rest", SOURCE_NONE, FIELD_STR, rest_atom_parser }, /* * Please update $__git_ref_fieldlist in git-completion.bash * when you add new atoms @@ -1010,6 +1021,9 @@ int verify_ref_format(struct ref_format *format) if (at < 0) die("%s", err.buf); + if (used_atom[at].atom_type == ATOM_REST) + die("this command reject atom %%(%.*s)", (int)(ep - sp - 2), sp + 2); + if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || format->quote_style == QUOTE_TCL) && @@ -1927,6 +1941,12 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->handler = else_atom_handler; v->s = xstrdup(""); continue; + } else if (atom_type == ATOM_REST) { + if (ref->rest) + v->s = xstrdup(ref->rest); + else + v->s = xstrdup(""); + continue; } else continue; @@ -2144,6 +2164,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); + ref->rest = NULL; return ref; } diff --git a/ref-filter.h b/ref-filter.h index 74fb423fc89..c15dee8d6b9 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -38,6 +38,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; + const char *rest; int flag; unsigned int kind; const char *symref; @@ -76,14 +77,16 @@ struct ref_format { * verify_ref_format() afterwards to finalize. */ const char *format; + const char *rest; int quote_style; + int use_rest; int use_color; /* Internal state to ref-filter */ int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { NULL, 0, -1 } +#define REF_FORMAT_INIT { .use_color = -1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ diff --git a/t/t3203-branch-output.sh b/t/t3203-branch-output.sh index 5325b9f67a0..6e94c6db7b5 100755 --- a/t/t3203-branch-output.sh +++ b/t/t3203-branch-output.sh @@ -340,6 +340,10 @@ test_expect_success 'git branch --format option' ' test_cmp expect actual ' +test_expect_success 'git branch with --format=%(rest) must fail' ' + test_must_fail git branch --format="%(rest)" >actual +' + test_expect_success 'worktree colors correct' ' cat >expect <<-EOF && * (HEAD detached from fromtag) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 0b66e743c58..6ca5c2cc19c 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -1211,6 +1211,10 @@ test_expect_success 'basic atom: head contents:trailers' ' test_cmp expect actual.clean ' +test_expect_success 'basic atom: rest must fail' ' + test_must_fail git for-each-ref --format="%(rest)" refs/heads/main +' + test_expect_success 'trailer parsing not fooled by --- line' ' git commit --allow-empty -F - <<-\EOF && this is the subject diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh index 2f72c5c6883..082be85dffc 100755 --- a/t/t7004-tag.sh +++ b/t/t7004-tag.sh @@ -1998,6 +1998,10 @@ test_expect_success '--format should list tags as per format given' ' test_cmp expect actual ' +test_expect_success 'git tag -l with --format="%(rest)" must fail' ' + test_must_fail git tag -l --format="%(rest)" "v1*" +' + test_expect_success "set up color tests" ' echo "v1.0" >expect.color && echo "v1.0" >expect.bare && diff --git a/t/t7030-verify-tag.sh b/t/t7030-verify-tag.sh index 3cefde9602b..10faa645157 100755 --- a/t/t7030-verify-tag.sh +++ b/t/t7030-verify-tag.sh @@ -194,6 +194,10 @@ test_expect_success GPG 'verifying tag with --format' ' test_cmp expect actual ' +test_expect_success GPG 'verifying tag with --format="%(rest)" must fail' ' + test_must_fail git verify-tag --format="%(rest)" "fourth-signed" +' + test_expect_success GPG 'verifying a forged tag with --format should fail silently' ' test_must_fail git verify-tag --format="tagname : %(tag)" $(cat forged1.tag) >actual-forged && test_must_be_empty actual-forged From patchwork Sun Jun 27 12:35:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3A89C49EAF for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C94F461C5E for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230418AbhF0Mi3 (ORCPT ); Sun, 27 Jun 2021 08:38:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230260AbhF0MiW (ORCPT ); Sun, 27 Jun 2021 08:38:22 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73DA2C061768 for ; Sun, 27 Jun 2021 05:35:57 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id p8-20020a7bcc880000b02901dbb595a9f1so9066074wma.2 for ; Sun, 27 Jun 2021 05:35:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=Ksg2FtXRhnosuihFiEAuxtS7xyeUFAMa47e/tO+Lqq4=; b=EbrALKPuSxMpiQ6HyYPx9Ma6zf20QWka8C6EHblL/D/hvwBTFTqEizBJSjfP2gOv2T p/bj+IfY18SEWQQRJZkENygUALwgQAPbtanwHV2wlFwyEMul07ryrgYBqru5CkAKrx0g OIa5W+DgMVnWjGTolA1OuJtAtcGswLRrxa0/hvDUGjXmt5hUd3OzOLH2E2OFztSTrxe0 fTI1tILMNoGXz8V4qwDcK1ANC3uzcDhr0eYtz1g6unC7tWMksUkuEl5gLswN/MJAtTR/ BoSlgEvfGqlbqLQ0C2AKZZ703CB0qDMNnlwlMz6sPCHlmsqPRP9Q2vQ1ca7+SZ0rl8KV rhSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=Ksg2FtXRhnosuihFiEAuxtS7xyeUFAMa47e/tO+Lqq4=; b=DZHUmx0NnNM8mOKpvCsWCQTcMnlTTBYIhP5oNVJ9y+EEwYcOiBhEFPLAAc5+lm8ykf 3PbdHPxkKen8uRDofNryQB0AIozCn9BDYENMDKYcUvweCRSKxe1tAgEV++JqtRdYfSjd aW78aZHXxjyLrMlvQzAyBgAqACaFjUVaTI1c1d56ETOkA7aLVChISLeTQtI6SkYq8dhX BCUwTZaZdwN7Sa4Hng8DjKBJCHlMosUZeCBCx3Jvf4PXHviBKvW78wcfdWLYRRsCb3mR omYlWBp3N5LKQNWvqtlIYI73UU6qtX9HNWF5GUxoY5GRLWECsFYiuW71CPKGzG6WUsDj YNFQ== X-Gm-Message-State: AOAM5337wgGHd8t2rFfuufAJAPfyHSw2HgsP7bW8RuKplpd120BDYwBA 5rgCYPW6tYw6YAdDQH0GTbrCCH8RPrY= X-Google-Smtp-Source: ABdhPJzTPHrIXfRU8f230m6CJT3/uJoc/x+A/b/zzQYuLbM+dM4Pa/Z8/t+asIos4BP+9SpwvmJlVQ== X-Received: by 2002:a05:600c:4e93:: with SMTP id f19mr15548726wmq.169.1624797356170; Sun, 27 Jun 2021 05:35:56 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p11sm11878364wre.57.2021.06.27.05.35.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:55 -0700 (PDT) Message-Id: <9873354930a51d6480beaf20a8e096bdae247f39.1624797351.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:41 +0000 Subject: [PATCH v6 06/15] [GSOC] ref-filter: pass get_object() return value to their callers MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because in the refactor of `git cat-file --batch` later, oid_object_info_extended() in get_object() will be used to obtain the info of an object with it's oid. When the object cannot be obtained in the git repository, `cat-file --batch` expects to output " missing" and continue the next oid query instead of letting Git exit. In other error conditions, Git should exit normally. So we can achieve this function by passing the return value of get_object(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- ref-filter.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 10c78de9cfa..e4988aa8a24 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1816,6 +1816,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) { struct object *obj; int i; + int ret; struct object_info empty = OBJECT_INFO_INIT; CALLOC_ARRAY(ref->value, used_atom_cnt); @@ -1972,8 +1973,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) oi.oid = ref->objectname; - if (get_object(ref, 0, &obj, &oi, err)) - return -1; + ret = get_object(ref, 0, &obj, &oi, err); + if (ret) + return ret; /* * If there is no atom that wants to know about tagged @@ -2005,8 +2007,10 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom, struct atom_value **v, struct strbuf *err) { if (!ref->value) { - if (populate_value(ref, err)) - return -1; + int ret = populate_value(ref, err); + + if (ret) + return ret; fill_missing_values(ref->value); } *v = &ref->value[atom]; @@ -2580,6 +2584,7 @@ int format_ref_array_item(struct ref_array_item *info, { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; + int ret; state.quote_style = format->quote_style; push_stack_element(&state.stack); @@ -2592,10 +2597,10 @@ int format_ref_array_item(struct ref_array_item *info, if (cp < sp) append_literal(cp, sp, &state); pos = parse_ref_filter_atom(format, sp + 2, ep, error_buf); - if (pos < 0 || get_ref_atom_value(info, pos, &atomv, error_buf) || + if (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) || atomv->handler(atomv, &state, error_buf)) { pop_stack_element(&state.stack); - return -1; + return ret ? ret : -1; } } if (*cp) { From patchwork Sun Jun 27 12:35:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4E2DC49EAB for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 94DF161C51 for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230330AbhF0Mi0 (ORCPT ); Sun, 27 Jun 2021 08:38:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230252AbhF0MiW (ORCPT ); Sun, 27 Jun 2021 08:38:22 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B435C061787 for ; Sun, 27 Jun 2021 05:35:58 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id l8so8264764wry.13 for ; Sun, 27 Jun 2021 05:35:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=g3vjHjNnyLQTmf+en7FqTd/OE8rsXg7l/gd2WpOyhZ0=; b=d7Qm/jUmirjycLyFAm5AlM71xP61/tur4/reY2/4U+Kl9dLlobYKsHl1bz2bs2URbk GEbrRMugcTRA6DOBNYU4WC8ELfx9iJkQtJYWxwNHk52hQyfmBjHPGc0nYNosSWCP1+IP g0j2Nmjlcg537cexgMb5SHzUIS26fyu6IBWIz86Y8labGqAF5DY3bygOPHmX85Lhp1Do uP1SzpooxPXovcBfd9SZRHN144mW8T9/g807MSBwKUTFOBZ6moxGnn0Njm3RnnfiGTw+ lK9Sjqaj+oBuy1CLowYBg8BTXW1x5h2GDTFJCTdYjkKeQ0TFCtb7cwOnKONfEvI1ZD7r IRwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=g3vjHjNnyLQTmf+en7FqTd/OE8rsXg7l/gd2WpOyhZ0=; b=SwMGXitNTF1skpGxHT8g1M8nCiKqgBprfYqEp0kTYFXHz0wKFXFKuJMbS8iEw+a3DQ SVuA2/MAqVSG683noB11y+ShCPwelC3Y51sBbVByZkmlZMjmkr5SBPc/Tb3f+Xb+Nec0 oFmVYdwjkTuAbmH5SR+L6zuQaAidEhu23nGAg/I+krAljxssvRet7hsTZhqCOEj0GEev opz7Yn/EmKeY+7TcJuBfv3cDi6hE/Ht9LzwkUa/iJwVVMSFbO84rCS4V9KkUzbbjV04g NB2sHUtG6/bccLxUIODkWM6pRhoTuUKXFvQD08kXvHHSF7PZsbK/kmxpGDWN5QplTseK bHJw== X-Gm-Message-State: AOAM533R+Ow+2VabJbnH/yClxFNnAberzLWCcqhfb8RjthphJSgQH6dj 2hvsl0JNEpN7F2Vv4Zd8Vi3XAU+Q12A= X-Google-Smtp-Source: ABdhPJw61euiZ6cJT6TT7WnMxwbF1/S0A98CKullQTxqOf05iFLWHZq3tcObZ1zODwJvZs94SB53NA== X-Received: by 2002:a5d:6350:: with SMTP id b16mr22405681wrw.41.1624797356744; Sun, 27 Jun 2021 05:35:56 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 11sm15673411wmf.20.2021.06.27.05.35.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:56 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:42 +0000 Subject: [PATCH v6 07/15] [GSOC] ref-filter: introduce free_ref_array_item_value() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu When we use ref_array_item which is not dynamically allocated and want to free the space of its member "value" after the end of use, free_array_item() does not meet our needs, because it tries to free ref_array_item itself and its member "symref". Introduce free_ref_array_item_value() for freeing ref_array_item value. It will be called internally by free_array_item(), and it will help `cat-file --batch` free ref_array_item's value memory later. Helped-by: Junio C Hamano Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 ++++++++--- ref-filter.h | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index e4988aa8a24..731e596eaa6 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2291,16 +2291,21 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, return 0; } -/* Free memory allocated for a ref_array_item */ -static void free_array_item(struct ref_array_item *item) +void free_ref_array_item_value(struct ref_array_item *item) { - free((char *)item->symref); if (item->value) { int i; for (i = 0; i < used_atom_cnt; i++) free((char *)item->value[i].s); free(item->value); } +} + +/* Free memory allocated for a ref_array_item */ +static void free_array_item(struct ref_array_item *item) +{ + free((char *)item->symref); + free_ref_array_item_value(item); free(item); } diff --git a/ref-filter.h b/ref-filter.h index c15dee8d6b9..44e6dc05ac2 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -111,6 +111,8 @@ struct ref_format { int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type); /* Clear all memory allocated to ref_array */ void ref_array_clear(struct ref_array *array); +/* Free ref_array_item's value */ +void free_ref_array_item_value(struct ref_array_item *item); /* Used to verify if the given format is correct and to parse out the used atoms */ int verify_ref_format(struct ref_format *format); /* Sort the given ref_array as per the ref_sorting provided */ From patchwork Sun Jun 27 12:35:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D99E9C49EB7 for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ABEE361C58 for ; Sun, 27 Jun 2021 12:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230398AbhF0Mi2 (ORCPT ); Sun, 27 Jun 2021 08:38:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230268AbhF0MiW (ORCPT ); Sun, 27 Jun 2021 08:38:22 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 911E6C061574 for ; Sun, 27 Jun 2021 05:35:58 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id u8-20020a7bcb080000b02901e44e9caa2aso8974217wmj.4 for ; Sun, 27 Jun 2021 05:35:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=FVbonolI94qKuU0xrFtaeBSooPbN2a6xB5ve3hV0wB4=; b=AOsbFiq6g0NvrTeu4rHVFJ4SP9IN0EUXUCjzvJudaojqQaovi2G7wVhzhP9q1GvLei DJ69gVpb6jf50e52QDjxu44fYmNHzwKBG4fFup2IQr+CuyO6WECOegHb9PYxS0JsPgx0 4soowxlzgm5UycH+WTFMTJfgTHTng69tBl5mGj08gw0fQtK7zXMC/ZbEBZc16AXGkT6Q 08+58QqsZwNayyLIBOIhMg1rjyYm4+qx6cEBhmo5a/woRpiRxQKhQ7a7UuMsh9rGvVKH DNILREx6PZoxMZD9WFTWk/XfsnsJx5Xvjh44Q0UYkOmD8q9JbYPaumGujZi3REtLw1sB iUCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=FVbonolI94qKuU0xrFtaeBSooPbN2a6xB5ve3hV0wB4=; b=eQVQYw3HOugBWjK/Q3to26cGvOfwK590k1ISUFZswkjJNRiI0SXamQQiNfLuwp4Fbb ZaddwejjedPKYf4iHUyvfIQSs9fZuwZ8nfqvZnz91nkEkyvuAodlXIILAmK4IXErmfU5 Nk2F+aDzqlBtEJyYzPT37mkPPOOZHpDCjlieICqoSbQrVVlwj0ad/R7U8wdIRggVpZHq Q/PlJT0mhIL4ZVdJgIeKbYq1WCg+vLigh/MEukf7oCe8bJqkjO5B3+pJ5YlQsoMd37is Ns2iw/tfgT5VBtd+cc0JqcgtYYriQb9qAgszlX8+6CuAQMQwBXQZI0/gcPCG0CpB1ZBy bw+g== X-Gm-Message-State: AOAM530gV8ygKChLsPwgz5S3V+vg+zEksmmimXaMRkSf+EIp6UZALKiD r9uOq2KOSItGs9xONzFkqCnbOObr88Q= X-Google-Smtp-Source: ABdhPJzRJqMn8UOzPTlUhTJ/CwKONsWyfQDcNyKrTZxxToWeVhvdVvOd5x3Huaa4dKpwpMBxsVx/oA== X-Received: by 2002:a1c:7c0b:: with SMTP id x11mr20709907wmc.183.1624797357263; Sun, 27 Jun 2021 05:35:57 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r2sm11488139wrv.39.2021.06.27.05.35.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:56 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:43 +0000 Subject: [PATCH v6 08/15] [GSOC] ref-filter: add cat_file_mode in struct ref_format Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add `cat_file_mode` member in struct `ref_format`, when `cat-file --batch` use ref-filter logic later, it can help us reject atoms in verify_ref_format() which cat-file cannot use, e.g. `%(refname)`, `%(push)`, `%(upstream)`... Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 +++++++++-- ref-filter.h | 1 + 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 731e596eaa6..45122959eef 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1021,8 +1021,15 @@ int verify_ref_format(struct ref_format *format) if (at < 0) die("%s", err.buf); - if (used_atom[at].atom_type == ATOM_REST) - die("this command reject atom %%(%.*s)", (int)(ep - sp - 2), sp + 2); + if ((!format->cat_file_mode && used_atom[at].atom_type == ATOM_REST) || + (format->cat_file_mode && (used_atom[at].atom_type == ATOM_FLAG || + used_atom[at].atom_type == ATOM_HEAD || + used_atom[at].atom_type == ATOM_PUSH || + used_atom[at].atom_type == ATOM_REFNAME || + used_atom[at].atom_type == ATOM_SYMREF || + used_atom[at].atom_type == ATOM_UPSTREAM || + used_atom[at].atom_type == ATOM_WORKTREEPATH))) + die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || diff --git a/ref-filter.h b/ref-filter.h index 44e6dc05ac2..053980a6a42 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -78,6 +78,7 @@ struct ref_format { */ const char *format; const char *rest; + int cat_file_mode; int quote_style; int use_rest; int use_color; From patchwork Sun Jun 27 12:35:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B49BC48BC2 for ; Sun, 27 Jun 2021 12:36:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 33B4B61C20 for ; Sun, 27 Jun 2021 12:36:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230260AbhF0Mid (ORCPT ); Sun, 27 Jun 2021 08:38:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230198AbhF0MiX (ORCPT ); Sun, 27 Jun 2021 08:38:23 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19AB2C061766 for ; Sun, 27 Jun 2021 05:35:59 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id p8-20020a7bcc880000b02901dbb595a9f1so9066094wma.2 for ; Sun, 27 Jun 2021 05:35:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=lfbYbfzHd+Vf6Fe8m7hXH4Krc1XXpNy92hpUxXnxmLQ=; b=rPVHFk6wi0mjmIiswFUAt0trekAVHSDrZUSUWCalhHBdOJQVPgey9ulqKsUvopyCSt KHvj+k8TgqtXt414qo0qa7zN4YknTpNHjMD41NT1rW/rQLT0tvuNuEDYPvDPwAQRF5m/ g7BW9MkwavsrbOtHV2/NKZRYLgb3ANS18KZ7di8cPXpTYY22lH11xxWrGiwbW6CZFMfk AgppUUr8ijwtFIwlqtMzpC8VPYBMkqzTdRCEQIiKrssGu16oXdgW3Q2clL71r2VK7HDw yDa/Oy8E6lx68gUjvMye4l9BYuSu8EuWAU/234FcWYRBOuvpfFoTtz2IpRGSr0TD9gVm h6og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=lfbYbfzHd+Vf6Fe8m7hXH4Krc1XXpNy92hpUxXnxmLQ=; b=EnGsBdlQRAw6QuecYkkPNF0TPFXoh/hjQSJrRlABQBVMpPNF0S2D42/URDmwJ5oOjm mYJrAYbYhuA7xPx71UaHH7OKKKiL4RT7aftafTFk7cE24W4KU3xtTHiy9dhYz8N4FPyA ZK66FI6qRdaWl73iFFfdRBclfUiSBtz7qO6R8A6sKYpVFsZB4L8WSK1BQLsgPchM20Xc 5xga2WDdO//QE9sF/V0+DrMu1ytekE7gBaQi7zHq2fdQzfoJX88LiWnxtham0CET/DA5 +RbFO2DFJ5Bw9GBYpB6ZhwpNQbdc01jp9EQDVGFV0u1YOZOmZvegv+nG+JH2Ovf/Rgz4 EPeA== X-Gm-Message-State: AOAM530275dK5tygEe8ha6t/d7pTKd9BpOqWv2h/Bltfr7mN9J07n3KS GCgHl2y37DrUWQZQigU98/9l9D5VLek= X-Google-Smtp-Source: ABdhPJyjjxV2kwBEZcKw6tZT01DfqsDdlEt/8hTK0KWenuGcbPf/kXxzp1jdY5bv5PMaRFCXkkidcw== X-Received: by 2002:a05:600c:5112:: with SMTP id o18mr20293647wms.15.1624797357794; Sun, 27 Jun 2021 05:35:57 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g15sm11084876wri.75.2021.06.27.05.35.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:57 -0700 (PDT) Message-Id: <85686187d49500b9ac4afaf526f5031ff5842aff.1624797351.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:44 +0000 Subject: [PATCH v6 09/15] [GSOC] ref-filter: modify the error message and value in get_object Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Let get_object() return 1 and print " missing" instead of returning -1 and printing "missing object for " if oid_object_info_extended() unable to find the data corresponding to oid. When `cat-file --batch` use ref-filter logic later it can help `format_ref_array_item()` just report that the object is missing without letting Git exit. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 4 ++-- t/t6301-for-each-ref-errors.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 45122959eef..9ca3dd5557d 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1749,8 +1749,8 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) - return strbuf_addf_ret(err, -1, _("missing object %s for %s"), - oid_to_hex(&oi->oid), ref->refname); + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); if (oi->info.disk_sizep && oi->disk_size < 0) BUG("Object size is less than zero."); diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh index 40edf9dab53..3553f84a00c 100755 --- a/t/t6301-for-each-ref-errors.sh +++ b/t/t6301-for-each-ref-errors.sh @@ -41,7 +41,7 @@ test_expect_success 'Missing objects are reported correctly' ' r=refs/heads/missing && echo $MISSING >.git/$r && test_when_finished "rm -f .git/$r" && - echo "fatal: missing object $MISSING for $r" >missing-err && + echo "fatal: $MISSING missing" >missing-err && test_must_fail git for-each-ref 2>err && test_cmp missing-err err && ( From patchwork Sun Jun 27 12:35:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C318C49EB7 for ; Sun, 27 Jun 2021 12:36:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1901261AC0 for ; Sun, 27 Jun 2021 12:36:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230334AbhF0Mii (ORCPT ); Sun, 27 Jun 2021 08:38:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230296AbhF0MiY (ORCPT ); Sun, 27 Jun 2021 08:38:24 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E39EC061574 for ; Sun, 27 Jun 2021 05:35:59 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id r8so3269702wrx.5 for ; Sun, 27 Jun 2021 05:35:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=r7Z6d5ifsh1v6IjHQmKUm8lmxTGmT95I9U09SdFQEqM=; b=u+X9qq33M2bM0CkNLgkpPjcquWjaqAnSyKLVrNGjdwfsyj+qC/CpdeyB8nxfK/JdgU 0nmSfpgVu+DUFeM1B5EzJkj8sjlx8+BJvgXICWdP+TCJSfA2lnqKpiOuU8RDvfcKWen8 3dBVxTgInyxIw6qerWefAaZUpkcTqFgDWNTtTqy1UVh0VK1zrs8lx/dIdERLmrolM8Rb BZxxyNk8DnWKGh0TKGJiovwiTsAOIazQkqJbwGWANYkCTlOnLYvFz0Ytc/3B9CcIZCyz Zxz4kAsjSg+m/ZKw5dfMrqd3RKEdXegP041hXmkzdVfVf9LOWg12Tn85OXpOVNI7dI+x yoxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=r7Z6d5ifsh1v6IjHQmKUm8lmxTGmT95I9U09SdFQEqM=; b=m/Nq3saaf3Q7w/22MxYCFHtFfecAnesBhfUtZwJc8X5F8H0qA+8aBzzdOr8FIVcOo3 V4WzJq+DuQnVi7AwFmaaC1f4/PxmvXYhA4Ze6+C9TsyqX+yxiOZEcRBop8VR1lLS3QZI fnLMHnygGNH0n2Aff3QJejDZ1gIG2CNmxEeYZu0r6cR77i3yqyf9Y61sRvXDKWrCrgPv CY+maPPoOsmACJl1997L0IJ6A2EEH01Ud9sXJDXrcJ3Ai7Y8kBJfAvrcFjn6/a2fFEUq Dp+tGYS0Jg3bAauRAQ/WdS7TQCi2fWsWBLJrvL5CPWvA8m6cimm5zEM8JhAcxEi0R2Bc mH7g== X-Gm-Message-State: AOAM5333HiCRgaJITczaXjO1JPRo4O34nHUPhsjzsFJ9eXlX1eGmPIRq LJK6io0yj16SSwECprrNBntIs3B3j1s= X-Google-Smtp-Source: ABdhPJzF4JV/x8tJsLVszijMpSgxEwyYEU5YRIZJfKwVuLpLemSXd5glVcDNSxCHYvb8Nce5VE92GA== X-Received: by 2002:adf:e384:: with SMTP id e4mr5591549wrm.317.1624797358332; Sun, 27 Jun 2021 05:35:58 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t11sm11389625wrz.7.2021.06.27.05.35.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:58 -0700 (PDT) Message-Id: <6037295ee58b9bfb5020f26776728fd12b9fbece.1624797351.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:45 +0000 Subject: [PATCH v6 10/15] [GSOC] cat-file: add has_object_file() check Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use `has_object_file()` in `batch_one_object()` to check whether the input object exists. This can help us reject the missing oid when we let `cat-file --batch` use ref-filter logic later. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 5ebf13359e8..9fd3c04ff20 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -428,6 +428,13 @@ static void batch_one_object(const char *obj_name, return; } + if (!has_object_file(&data->oid)) { + printf("%s missing\n", + obj_name ? obj_name : oid_to_hex(&data->oid)); + fflush(stdout); + return; + } + batch_object_write(obj_name, scratch, opt, data); } From patchwork Sun Jun 27 12:35:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAA2DC49EAF for ; Sun, 27 Jun 2021 12:36:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B62CE61C20 for ; Sun, 27 Jun 2021 12:36:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230498AbhF0Mif (ORCPT ); Sun, 27 Jun 2021 08:38:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbhF0MiZ (ORCPT ); Sun, 27 Jun 2021 08:38:25 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E22BC061767 for ; Sun, 27 Jun 2021 05:36:00 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id l12so16313732wrt.3 for ; Sun, 27 Jun 2021 05:36:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=HluKIvI8PnxXFef46ukO9OSyoyTW6LZ2nDr1+/IAdRM=; b=QFpc0NZ1sV2FuClqcA9GbbR/LJjHpuz4t3mF/R19Pw2dLhWSzJLcJmInNzQOy129qR oKKB4sBNuwsL/sHUgGDEtJDp9yK/wtTp2l5qqcI3PBFRPuIAQdmu1gBoJ+0VatV2Jdlk RtNl99ydNMijRdqlMzZvoHJ5s8wO7Jsg+VV8wFhHbGjTYboIBXaz8c6dUdQUPz7pjyT/ VmFXamg22GTCEzR+aHpTjg7Poo07F0N5PEEdpgaE8I9mcUrEC1xDmtTkcQeMlhK+Vw09 IVAIXtatUfz0wfcA7azByzJEahRok0qbNKIpdNr1VSn1OHpGkym3l5dAXq4RzDKKSIKz VF+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=HluKIvI8PnxXFef46ukO9OSyoyTW6LZ2nDr1+/IAdRM=; b=h1HyPFy4sexjtpmohZM65wuhRr6vTMz95VawlR40ENTPHWGlRyPOMyrQ/GA7G7ZfQo wSENrVYZU6obWSCoNpf+vD6G5yOH1zmSoBt1pxfkAZxYMMpmOY/qh0577yddkWSDtCYg DOUKyhtiBnpxpnA+gSFYdH7xQtP9FOWotmpaPxLvwxNl5ciXgFWMBra2AEFPjuQwUsgo jgtUoCKGeMvAX5BLtGQXs7DqF5xZaEMY4kSe9M15bXneYhG8GxICnMsSu9Q0/QZNbNLw 1kAmNOmZrJo79NjmgKsjBRXJR297EIzl8XogmyfHM6DG3IjPQtWyHvmwZRsZIb9sO7xj Ur3w== X-Gm-Message-State: AOAM5316Dhql2UE6H2C2d9dDJ4ibZDArZGjgH1qvq3iXzWWDtEMMphv9 zOJpU712sRdcIvx6cF7Oxxg+8aBCFng= X-Google-Smtp-Source: ABdhPJxx2w0r1lrCruVYuBW0yyhKYAYrzmAmEAYTxjKPQMC7d932xmfgLfQSKIqPSaGvAPceXZUJLw== X-Received: by 2002:a5d:6d06:: with SMTP id e6mr22874260wrq.28.1624797358980; Sun, 27 Jun 2021 05:35:58 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f2sm1868111wrd.64.2021.06.27.05.35.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:58 -0700 (PDT) Message-Id: <32e1ca5638917eca4855bee5a248dc268168465a.1624797351.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:46 +0000 Subject: [PATCH v6 11/15] [GSOC] cat-file: change batch_objects parameter name Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because later cat-file reuses ref-filter logic that will add parameter "const struct option *options" to batch_objects(), the two synonymous parameters of "opt" and "options" may confuse readers, so change batch_options parameter of batch_objects() from "opt" to "batch". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 9fd3c04ff20..cd84c39df96 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -495,7 +495,7 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *opt) +static int batch_objects(struct batch_options *batch) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; @@ -503,8 +503,8 @@ static int batch_objects(struct batch_options *opt) int save_warning; int retval = 0; - if (!opt->format) - opt->format = "%(objectname) %(objecttype) %(objectsize)"; + if (!batch->format) + batch->format = "%(objectname) %(objecttype) %(objectsize)"; /* * Expand once with our special mark_query flag, which will prime the @@ -513,13 +513,13 @@ static int batch_objects(struct batch_options *opt) */ memset(&data, 0, sizeof(data)); data.mark_query = 1; - strbuf_expand(&output, opt->format, expand_format, &data); + strbuf_expand(&output, batch->format, expand_format, &data); data.mark_query = 0; strbuf_release(&output); - if (opt->cmdmode) + if (batch->cmdmode) data.split_on_whitespace = 1; - if (opt->all_objects) { + if (batch->all_objects) { struct object_info empty = OBJECT_INFO_INIT; if (!memcmp(&data.info, &empty, sizeof(empty))) data.skip_object_info = 1; @@ -529,20 +529,20 @@ static int batch_objects(struct batch_options *opt) * If we are printing out the object, then always fill in the type, * since we will want to decide whether or not to stream. */ - if (opt->print_contents) + if (batch->print_contents) data.info.typep = &data.type; - if (opt->all_objects) { + if (batch->all_objects) { struct object_cb_data cb; if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); - cb.opt = opt; + cb.opt = batch; cb.expand = &data; cb.scratch = &output; - if (opt->unordered) { + if (batch->unordered) { struct oidset seen = OIDSET_INIT; cb.seen = &seen; @@ -592,7 +592,7 @@ static int batch_objects(struct batch_options *opt) data.rest = p; } - batch_one_object(input.buf, &output, opt, &data); + batch_one_object(input.buf, &output, batch, &data); } strbuf_release(&input); From patchwork Sun Jun 27 12:35:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C03DAC49EAB for ; Sun, 27 Jun 2021 12:36:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A418061AC0 for ; Sun, 27 Jun 2021 12:36:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230436AbhF0Mie (ORCPT ); Sun, 27 Jun 2021 08:38:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230334AbhF0Mi0 (ORCPT ); Sun, 27 Jun 2021 08:38:26 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A4EDC061768 for ; Sun, 27 Jun 2021 05:36:01 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id j1so16999496wrn.9 for ; Sun, 27 Jun 2021 05:36:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=JnfHjNVIDgKoljTyj0Qpy0dsQAZkM6NeXPqfWdJup5s=; b=qyygt0gZx6iPJWtLIhedgtsKcG9+z0fUicOZks3GASwZ2B0qSQ4/Hu5cmta2wyxMM6 MaPSkxWvGWFqKPcsUCJQAInO7CH9wOMBknhbAPAvvtZDqvdomtt/EZq9xvSieRbGSTdp LE2Sf7QQCpbtIgaLX/kPusYTEJLIyPzzCALo7xOHziafMmNp+in08fPHMEDImrxUYHvo tKSDZtyrgj1LfMohDKfcDjtdSyRRqkm6ZdJcmRknjdROfuHg3PgIOMHbkYFP+3AkGjwQ l2urag+UOZu6WSbI1p93IbSWce9lt6K1GSiudzoUV7w51xcFG2pbWiOjxLRPbxZK7kJt baoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=JnfHjNVIDgKoljTyj0Qpy0dsQAZkM6NeXPqfWdJup5s=; b=UQXGbL7VrXWlS5PbFlcblwJqrhihErEIIZifq+aCbETBBnQ/1Ac7BU98jL+VVR5xyd KrpMC6CDD6lCGBLDxfABi+2NFqbnx/nUJfH9frhJ2OdnWalVZ76CiwlvbcatXaFfnqrE +IX5q91YU6q6zuonM7fJ8pCR2c5hvyHNSspDJ/AO1FxDo7/+CR4R8d1Iw8FSauLKJrbq apmKjZ73xJKuw2faEfGg2hMqfpdP58trbGKQVXcdeeduBtvnUzSuc979l7ReLV2oiodW yB/POj/acqi50JpNw3CAVbh16po1Dw08DGH5SsMlN9sGdje9bqs4JOV5JCFeABW1m7Dt HV2A== X-Gm-Message-State: AOAM530ybrzdfyjlzG4pCJtlbvV5XJ7IG+z30WbNfFIk25YOtSecvGu8 pIXK+Z0DCbifEXbD+7FaoSqx9/pSKLs= X-Google-Smtp-Source: ABdhPJwAHk1c8ghufMoh6BR9omYw0UJg1BHxu2POfZwqI5Qroe6C/m4gwXdClN56BJjS3zR7NrytUg== X-Received: by 2002:a5d:410f:: with SMTP id l15mr22116244wrp.82.1624797359695; Sun, 27 Jun 2021 05:35:59 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x81sm18580871wmg.36.2021.06.27.05.35.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:35:59 -0700 (PDT) Message-Id: <9a1f07329401434b5960ef8ae002f11b1133506a.1624797351.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:47 +0000 Subject: [PATCH v6 12/15] [GSOC] cat-file: reuse ref-filter logic MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let cat-file use ref-filter logic, let's do the following: 1. Change the type of member `format` in struct `batch_options` to `ref_format`, we will pass it to ref-filter later. 2. Let `batch_objects()` add atoms to format, and use `verify_ref_format()` to check atoms. 3. Use `format_ref_array_item()` in `batch_object_write()` to get the formatted data corresponding to the object. If the return value of `format_ref_array_item()` is equals to zero, use `batch_write()` to print object data; else if the return value is less than zero, use `die()` to print the error message and exit; else if return value is greater than zero, only print the error message, but don't exit. 4. Use free_ref_array_item_value() to free ref_array_item's value. Most of the atoms in `for-each-ref --format` are now supported, such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, `%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: `%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, `%(flag)`, `%(HEAD)`, because our objects don't have a refname. The performance for `git cat-file --batch-all-objects --batch-check` on the Git repository itself with performance testing tool `hyperfine` changes from 669.4 ms ± 31.1 ms to 1.134 s ± 0.063 s. The performance for `git cat-file --batch-all-objects --batch >/dev/null` on the Git repository itself with performance testing tool `time` change from "27.37s user 0.29s system 98% cpu 28.089 total" to "33.69s user 1.54s system 87% cpu 40.258 total". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- Documentation/git-cat-file.txt | 6 + builtin/cat-file.c | 244 ++++++------------------------- t/t1006-cat-file.sh | 252 +++++++++++++++++++++++++++++++++ 3 files changed, 305 insertions(+), 197 deletions(-) diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index 4eb0421b3fd..ef8ab952b2f 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -226,6 +226,12 @@ newline. The available atoms are: after that first run of whitespace (i.e., the "rest" of the line) are output in place of the `%(rest)` atom. +Note that most of the atoms in `for-each-ref --format` are now supported, +such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, +`%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: +`%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, +`%(flag)`, `%(HEAD)`. See linkgit:git-for-each-ref[1]. + If no format is specified, the default format is `%(objectname) %(objecttype) %(objectsize)`. diff --git a/builtin/cat-file.c b/builtin/cat-file.c index cd84c39df96..5b163551fc6 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -16,6 +16,7 @@ #include "packfile.h" #include "object-store.h" #include "promisor-remote.h" +#include "ref-filter.h" struct batch_options { int enabled; @@ -25,7 +26,7 @@ struct batch_options { int all_objects; int unordered; int cmdmode; /* may be 'w' or 'c' for --filters or --textconv */ - const char *format; + struct ref_format format; }; static const char *force_path; @@ -195,99 +196,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct expand_data { struct object_id oid; - enum object_type type; - unsigned long size; - off_t disk_size; const char *rest; - struct object_id delta_base_oid; - - /* - * If mark_query is true, we do not expand anything, but rather - * just mark the object_info with items we wish to query. - */ - int mark_query; - - /* - * Whether to split the input on whitespace before feeding it to - * get_sha1; this is decided during the mark_query phase based on - * whether we have a %(rest) token in our format. - */ int split_on_whitespace; - - /* - * After a mark_query run, this object_info is set up to be - * passed to oid_object_info_extended. It will point to the data - * elements above, so you can retrieve the response from there. - */ - struct object_info info; - - /* - * This flag will be true if the requested batch format and options - * don't require us to call oid_object_info, which can then be - * optimized out. - */ - unsigned skip_object_info : 1; }; -static int is_atom(const char *atom, const char *s, int slen) -{ - int alen = strlen(atom); - return alen == slen && !memcmp(atom, s, alen); -} - -static void expand_atom(struct strbuf *sb, const char *atom, int len, - void *vdata) -{ - struct expand_data *data = vdata; - - if (is_atom("objectname", atom, len)) { - if (!data->mark_query) - strbuf_addstr(sb, oid_to_hex(&data->oid)); - } else if (is_atom("objecttype", atom, len)) { - if (data->mark_query) - data->info.typep = &data->type; - else - strbuf_addstr(sb, type_name(data->type)); - } else if (is_atom("objectsize", atom, len)) { - if (data->mark_query) - data->info.sizep = &data->size; - else - strbuf_addf(sb, "%"PRIuMAX , (uintmax_t)data->size); - } else if (is_atom("objectsize:disk", atom, len)) { - if (data->mark_query) - data->info.disk_sizep = &data->disk_size; - else - strbuf_addf(sb, "%"PRIuMAX, (uintmax_t)data->disk_size); - } else if (is_atom("rest", atom, len)) { - if (data->mark_query) - data->split_on_whitespace = 1; - else if (data->rest) - strbuf_addstr(sb, data->rest); - } else if (is_atom("deltabase", atom, len)) { - if (data->mark_query) - data->info.delta_base_oid = &data->delta_base_oid; - else - strbuf_addstr(sb, - oid_to_hex(&data->delta_base_oid)); - } else - die("unknown format element: %.*s", len, atom); -} - -static size_t expand_format(struct strbuf *sb, const char *start, void *data) -{ - const char *end; - - if (*start != '(') - return 0; - end = strchr(start + 1, ')'); - if (!end) - die("format element '%s' does not end in ')'", start); - - expand_atom(sb, start + 1, end - start - 1, data); - - return end - start + 1; -} - static void batch_write(struct batch_options *opt, const void *data, int len) { if (opt->buffer_output) { @@ -297,87 +209,34 @@ static void batch_write(struct batch_options *opt, const void *data, int len) write_or_die(1, data, len); } -static void print_object_or_die(struct batch_options *opt, struct expand_data *data) -{ - const struct object_id *oid = &data->oid; - - assert(data->info.typep); - - if (data->type == OBJ_BLOB) { - if (opt->buffer_output) - fflush(stdout); - if (opt->cmdmode) { - char *contents; - unsigned long size; - - if (!data->rest) - die("missing path for '%s'", oid_to_hex(oid)); - - if (opt->cmdmode == 'w') { - if (filter_object(data->rest, 0100644, oid, - &contents, &size)) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else if (opt->cmdmode == 'c') { - enum object_type type; - if (!textconv_object(the_repository, - data->rest, 0100644, oid, - 1, &contents, &size)) - contents = read_object_file(oid, - &type, - &size); - if (!contents) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else - BUG("invalid cmdmode: %c", opt->cmdmode); - batch_write(opt, contents, size); - free(contents); - } else { - stream_blob(oid); - } - } - else { - enum object_type type; - unsigned long size; - void *contents; - - contents = read_object_file(oid, &type, &size); - if (!contents) - die("object %s disappeared", oid_to_hex(oid)); - if (type != data->type) - die("object %s changed type!?", oid_to_hex(oid)); - if (data->info.sizep && size != data->size) - die("object %s changed size!?", oid_to_hex(oid)); - - batch_write(opt, contents, size); - free(contents); - } -} static void batch_object_write(const char *obj_name, struct strbuf *scratch, struct batch_options *opt, struct expand_data *data) { - if (!data->skip_object_info && - oid_object_info_extended(the_repository, &data->oid, &data->info, - OBJECT_INFO_LOOKUP_REPLACE) < 0) { - printf("%s missing\n", - obj_name ? obj_name : oid_to_hex(&data->oid)); - fflush(stdout); - return; - } + int ret; + struct strbuf err = STRBUF_INIT; + struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); - strbuf_expand(scratch, opt->format, expand_format, data); - strbuf_addch(scratch, '\n'); - batch_write(opt, scratch->buf, scratch->len); - if (opt->print_contents) { - print_object_or_die(opt, data); - batch_write(opt, "\n", 1); + ret = format_ref_array_item(&item, &opt->format, scratch, &err); + if (ret < 0) + die("%s\n", err.buf); + if (ret) { + /* ret > 0 means when the object corresponding to oid + * cannot be found in format_ref_array_item(), we only print + * the error message. + */ + printf("%s\n", err.buf); + fflush(stdout); + } else { + strbuf_addch(scratch, '\n'); + batch_write(opt, scratch->buf, scratch->len); } + free_ref_array_item_value(&item); + strbuf_release(&err); } static void batch_one_object(const char *obj_name, @@ -495,42 +354,34 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *batch) +static const char * const cat_file_usage[] = { + N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), + N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), + NULL +}; + +static int batch_objects(struct batch_options *batch, const struct option *options) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; int retval = 0; - if (!batch->format) - batch->format = "%(objectname) %(objecttype) %(objectsize)"; - - /* - * Expand once with our special mark_query flag, which will prime the - * object_info to be handed to oid_object_info_extended for each - * object. - */ memset(&data, 0, sizeof(data)); - data.mark_query = 1; - strbuf_expand(&output, batch->format, expand_format, &data); - data.mark_query = 0; - strbuf_release(&output); - if (batch->cmdmode) - data.split_on_whitespace = 1; - - if (batch->all_objects) { - struct object_info empty = OBJECT_INFO_INIT; - if (!memcmp(&data.info, &empty, sizeof(empty))) - data.skip_object_info = 1; - } - - /* - * If we are printing out the object, then always fill in the type, - * since we will want to decide whether or not to stream. - */ + if (batch->format.format) + strbuf_addstr(&format, batch->format.format); + else + strbuf_addstr(&format, "%(objectname) %(objecttype) %(objectsize)"); if (batch->print_contents) - data.info.typep = &data.type; + strbuf_addstr(&format, "\n%(raw)"); + batch->format.format = format.buf; + if (verify_ref_format(&batch->format)) + usage_with_options(cat_file_usage, options); + + if (batch->cmdmode || batch->format.use_rest) + data.split_on_whitespace = 1; if (batch->all_objects) { struct object_cb_data cb; @@ -563,6 +414,7 @@ static int batch_objects(struct batch_options *batch) oid_array_clear(&sa); } + strbuf_release(&format); strbuf_release(&output); return 0; } @@ -595,18 +447,13 @@ static int batch_objects(struct batch_options *batch) batch_one_object(input.buf, &output, batch, &data); } + strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); warn_on_object_refname_ambiguity = save_warning; return retval; } -static const char * const cat_file_usage[] = { - N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), - N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), - NULL -}; - static int git_cat_file_config(const char *var, const char *value, void *cb) { if (userdiff_config(var, value) < 0) @@ -629,7 +476,7 @@ static int batch_option_callback(const struct option *opt, bo->enabled = 1; bo->print_contents = !strcmp(opt->long_name, "batch"); - bo->format = arg; + bo->format.format = arg; return 0; } @@ -638,7 +485,9 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) { int opt = 0; const char *exp_type = NULL, *obj_name = NULL; - struct batch_options batch = {0}; + struct batch_options batch = { + .format = REF_FORMAT_INIT + }; int unknown_type = 0; const struct option options[] = { @@ -677,6 +526,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) git_config(git_cat_file_config, NULL); batch.buffer_output = -1; + batch.format.cat_file_mode = 1; argc = parse_options(argc, argv, prefix, options, cat_file_usage, 0); if (opt) { @@ -720,7 +570,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) batch.buffer_output = batch.all_objects; if (batch.enabled) - return batch_objects(&batch); + return batch_objects(&batch, options); if (unknown_type && opt != 't' && opt != 's') die("git cat-file --allow-unknown-type: use with -s or -t"); diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 5d2dc99b74a..69eb627774d 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -586,4 +586,256 @@ test_expect_success 'cat-file --unordered works' ' test_cmp expect actual ' +. "$TEST_DIRECTORY"/lib-gpg.sh +. "$TEST_DIRECTORY"/lib-terminal.sh + +test_expect_success 'cat-file --batch|--batch-check setup' ' + echo 1>blob1 && + printf "a\0b\0\c" >blob2 && + git add blob1 blob2 && + git commit -m "Commit Message" && + git branch -M main && + git tag -a -m "v0.0.0" testtag && + git update-ref refs/myblobs/blob1 HEAD:blob1 && + git update-ref refs/myblobs/blob2 HEAD:blob2 && + git update-ref refs/mytrees/tree1 HEAD^{tree} +' + +batch_test_atom() { + if test "$3" = "fail" + then + test_expect_${4:-success} $PREREQ "basic atom: $1 $2 must fail" " + test_must_fail git cat-file --batch-check='$2' >bad <<-EOF + $1 + EOF + " + else + test_expect_${4:-success} $PREREQ "basic atom: $1 $2" " + git for-each-ref --format='$2' $1 >expected && + git cat-file --batch-check='$2' >actual <<-EOF && + $1 + EOF + sanitize_pgp actual.clean && + cmp expected actual.clean + " + fi +} + +batch_test_atom refs/heads/main '%(refname)' fail +batch_test_atom refs/heads/main '%(refname:)' fail +batch_test_atom refs/heads/main '%(refname:short)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:strip=1)' fail +batch_test_atom refs/heads/main '%(refname:strip=2)' fail +batch_test_atom refs/heads/main '%(refname:strip=-1)' fail +batch_test_atom refs/heads/main '%(refname:strip=-2)' fail +batch_test_atom refs/heads/main '%(upstream)' fail +batch_test_atom refs/heads/main '%(upstream:short)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=-2)' fail +batch_test_atom refs/heads/main '%(push)' fail +batch_test_atom refs/heads/main '%(push:short)' fail +batch_test_atom refs/heads/main '%(push:lstrip=1)' fail +batch_test_atom refs/heads/main '%(push:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:strip=1)' fail +batch_test_atom refs/heads/main '%(push:strip=-1)' fail +batch_test_atom refs/heads/main '%(objecttype)' +batch_test_atom refs/heads/main '%(objectsize)' +batch_test_atom refs/heads/main '%(objectsize:disk)' +batch_test_atom refs/heads/main '%(deltabase)' +batch_test_atom refs/heads/main '%(objectname)' +batch_test_atom refs/heads/main '%(objectname:short)' +batch_test_atom refs/heads/main '%(objectname:short=1)' +batch_test_atom refs/heads/main '%(objectname:short=10)' +batch_test_atom refs/heads/main '%(tree)' +batch_test_atom refs/heads/main '%(tree:short)' +batch_test_atom refs/heads/main '%(tree:short=1)' +batch_test_atom refs/heads/main '%(tree:short=10)' +batch_test_atom refs/heads/main '%(parent)' +batch_test_atom refs/heads/main '%(parent:short)' +batch_test_atom refs/heads/main '%(parent:short=1)' +batch_test_atom refs/heads/main '%(parent:short=10)' +batch_test_atom refs/heads/main '%(numparent)' +batch_test_atom refs/heads/main '%(object)' +batch_test_atom refs/heads/main '%(type)' +batch_test_atom refs/heads/main '%(raw)' +batch_test_atom refs/heads/main '%(*objectname)' +batch_test_atom refs/heads/main '%(*objecttype)' +batch_test_atom refs/heads/main '%(author)' +batch_test_atom refs/heads/main '%(authorname)' +batch_test_atom refs/heads/main '%(authoremail)' +batch_test_atom refs/heads/main '%(authoremail:trim)' +batch_test_atom refs/heads/main '%(authoremail:localpart)' +batch_test_atom refs/heads/main '%(authordate)' +batch_test_atom refs/heads/main '%(committer)' +batch_test_atom refs/heads/main '%(committername)' +batch_test_atom refs/heads/main '%(committeremail)' +batch_test_atom refs/heads/main '%(committeremail:trim)' +batch_test_atom refs/heads/main '%(committeremail:localpart)' +batch_test_atom refs/heads/main '%(committerdate)' +batch_test_atom refs/heads/main '%(tag)' +batch_test_atom refs/heads/main '%(tagger)' +batch_test_atom refs/heads/main '%(taggername)' +batch_test_atom refs/heads/main '%(taggeremail)' +batch_test_atom refs/heads/main '%(taggeremail:trim)' +batch_test_atom refs/heads/main '%(taggeremail:localpart)' +batch_test_atom refs/heads/main '%(taggerdate)' +batch_test_atom refs/heads/main '%(creator)' +batch_test_atom refs/heads/main '%(creatordate)' +batch_test_atom refs/heads/main '%(subject)' +batch_test_atom refs/heads/main '%(subject:sanitize)' +batch_test_atom refs/heads/main '%(contents:subject)' +batch_test_atom refs/heads/main '%(body)' +batch_test_atom refs/heads/main '%(contents:body)' +batch_test_atom refs/heads/main '%(contents:signature)' +batch_test_atom refs/heads/main '%(contents)' +batch_test_atom refs/heads/main '%(HEAD)' fail +batch_test_atom refs/heads/main '%(upstream:track)' fail +batch_test_atom refs/heads/main '%(upstream:trackshort)' fail +batch_test_atom refs/heads/main '%(upstream:track,nobracket)' fail +batch_test_atom refs/heads/main '%(upstream:nobracket,track)' fail +batch_test_atom refs/heads/main '%(push:track)' fail +batch_test_atom refs/heads/main '%(push:trackshort)' fail +batch_test_atom refs/heads/main '%(worktreepath)' fail +batch_test_atom refs/heads/main '%(symref)' fail +batch_test_atom refs/heads/main '%(flag)' fail + +batch_test_atom refs/tags/testtag '%(refname)' fail +batch_test_atom refs/tags/testtag '%(refname:short)' fail +batch_test_atom refs/tags/testtag '%(upstream)' fail +batch_test_atom refs/tags/testtag '%(push)' fail +batch_test_atom refs/tags/testtag '%(objecttype)' +batch_test_atom refs/tags/testtag '%(objectsize)' +batch_test_atom refs/tags/testtag '%(objectsize:disk)' +batch_test_atom refs/tags/testtag '%(*objectsize:disk)' +batch_test_atom refs/tags/testtag '%(deltabase)' +batch_test_atom refs/tags/testtag '%(*deltabase)' +batch_test_atom refs/tags/testtag '%(objectname)' +batch_test_atom refs/tags/testtag '%(objectname:short)' +batch_test_atom refs/tags/testtag '%(tree)' +batch_test_atom refs/tags/testtag '%(tree:short)' +batch_test_atom refs/tags/testtag '%(tree:short=1)' +batch_test_atom refs/tags/testtag '%(tree:short=10)' +batch_test_atom refs/tags/testtag '%(parent)' +batch_test_atom refs/tags/testtag '%(parent:short)' +batch_test_atom refs/tags/testtag '%(parent:short=1)' +batch_test_atom refs/tags/testtag '%(parent:short=10)' +batch_test_atom refs/tags/testtag '%(numparent)' +batch_test_atom refs/tags/testtag '%(object)' +batch_test_atom refs/tags/testtag '%(type)' +batch_test_atom refs/tags/testtag '%(*objectname)' +batch_test_atom refs/tags/testtag '%(*objecttype)' +batch_test_atom refs/tags/testtag '%(author)' +batch_test_atom refs/tags/testtag '%(authorname)' +batch_test_atom refs/tags/testtag '%(authoremail)' +batch_test_atom refs/tags/testtag '%(authoremail:trim)' +batch_test_atom refs/tags/testtag '%(authoremail:localpart)' +batch_test_atom refs/tags/testtag '%(authordate)' +batch_test_atom refs/tags/testtag '%(committer)' +batch_test_atom refs/tags/testtag '%(committername)' +batch_test_atom refs/tags/testtag '%(committeremail)' +batch_test_atom refs/tags/testtag '%(committeremail:trim)' +batch_test_atom refs/tags/testtag '%(committeremail:localpart)' +batch_test_atom refs/tags/testtag '%(committerdate)' +batch_test_atom refs/tags/testtag '%(tag)' +batch_test_atom refs/tags/testtag '%(tagger)' +batch_test_atom refs/tags/testtag '%(taggername)' +batch_test_atom refs/tags/testtag '%(taggeremail)' +batch_test_atom refs/tags/testtag '%(taggeremail:trim)' +batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' +batch_test_atom refs/tags/testtag '%(taggerdate)' +batch_test_atom refs/tags/testtag '%(creator)' +batch_test_atom refs/tags/testtag '%(creatordate)' +batch_test_atom refs/tags/testtag '%(subject)' +batch_test_atom refs/tags/testtag '%(subject:sanitize)' +batch_test_atom refs/tags/testtag '%(contents:subject)' +batch_test_atom refs/tags/testtag '%(body)' +batch_test_atom refs/tags/testtag '%(contents:body)' +batch_test_atom refs/tags/testtag '%(contents:signature)' +batch_test_atom refs/tags/testtag '%(contents)' +batch_test_atom refs/tags/testtag '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(refname)' fail +batch_test_atom refs/myblobs/blob1 '%(upstream)' fail +batch_test_atom refs/myblobs/blob1 '%(push)' fail +batch_test_atom refs/myblobs/blob1 '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(objectname)' +batch_test_atom refs/myblobs/blob1 '%(objecttype)' +batch_test_atom refs/myblobs/blob1 '%(objectsize)' +batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)' +batch_test_atom refs/myblobs/blob1 '%(deltabase)' + +batch_test_atom refs/myblobs/blob1 '%(contents)' +batch_test_atom refs/myblobs/blob2 '%(contents)' + +batch_test_atom refs/myblobs/blob1 '%(raw)' +batch_test_atom refs/myblobs/blob2 '%(raw)' +batch_test_atom refs/mytrees/tree1 '%(raw)' + +batch_test_atom refs/myblobs/blob1 '%(raw:size)' +batch_test_atom refs/myblobs/blob2 '%(raw:size)' +batch_test_atom refs/mytrees/tree1 '%(raw:size)' + +batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' + +batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' + +batch_test_atom refs/heads/main 'VALID' +batch_test_atom refs/heads/main '%(INVALID)' fail +batch_test_atom refs/heads/main '%(authordate:INVALID)' fail + +test_expect_success '%(rest) works with both a branch and a tag' ' + cat >expected <<-EOF && + 123 commit 123 + 456 tag 456 + EOF + git cat-file --batch-check="%(rest) %(objecttype) %(rest)" >actual <<-EOF && + refs/heads/main 123 + refs/tags/testtag 456 + EOF + test_cmp expected actual +' + +batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize) +%(raw)' + + +test_expect_success 'cat-file --batch equals to --batch-check with atoms' ' + git cat-file --batch-check="%(objectname) %(objecttype) %(objectsize) +%(raw)" >expected <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + git cat-file --batch >actual <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + cmp expected actual +' + test_done From patchwork Sun Jun 27 12:35:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CA49C49EA6 for ; Sun, 27 Jun 2021 12:36:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37F9661AC0 for ; Sun, 27 Jun 2021 12:36:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230522AbhF0Mih (ORCPT ); Sun, 27 Jun 2021 08:38:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230321AbhF0MiZ (ORCPT ); Sun, 27 Jun 2021 08:38:25 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1553C061787 for ; Sun, 27 Jun 2021 05:36:01 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id l8so8264906wry.13 for ; Sun, 27 Jun 2021 05:36:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=qAwXHcrowoAV1ZvkfF6sVHaTDEdBOu0rUNJ1Ia1vdCk=; b=dI9RkV6mSJDCdw0XhOrkE/LHrUbcpcaX28hKfrQ7K02xX+fbhp58WEW8n+aZ6+URIZ ink1IG1/JAgglIC0U/4gBd4bzOcijDDUd/dhlel+IxdDfy+nLx+kdSjQYpdDlrpeQEVY IM3Z+4w922VRzbur5t7LDcvSAmWfr0rJTenIjzJ4hgIB1LCyqKX/vMxBcE8Jl7pcNkBK +HqO8gQN8YC6dnbm6G/3ffXBZN8JQ2KfenDDgIRa8cc8QWdPsazifvfIH2oNFuLn8OmO vH/qsmMD/B8T+BFEOsVMRza0HhlG2tq30Tb52jMZHlrd003/yr6zDGGIKq9Kx9hVdiVH LaCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=qAwXHcrowoAV1ZvkfF6sVHaTDEdBOu0rUNJ1Ia1vdCk=; b=hRJNI6taSwauvgVgDtFoEU0lMOczOgJmQEgxapk0Ud54DC4/0705AeFun0vWUFs+2h Bfc2qKUI7EXS8ExG4GURAaViaxF1rUpF7/0sixWRP6KLfJA8CW9+N0Yfv1ANetJ0/AFT z9whPVWpxmVuevLHRGDVScHExYXcf+UUbGvkp47lTpqZ9HvuGYYeY0x4EL/AFha5O6xU Po6MigWwNeZ6Jx/VFE+oBaVcF8YPC4bZK2PJhBnSDApwYwmBrYr6TUudHQhNZcxk5DjE 6k0VguaxLIjCQFKq5Hf4pq92fZ95MDH6pbMFrfa9hyGIBOhK6VjXXOjxDCrsryvzXBXz vyQg== X-Gm-Message-State: AOAM530v1/mwkEzadM1XAjcc7u7LzTG1dGRU3G5S5t6+NSNcEcn2Oa8N o+OhxDVACFiWXGntUQm7LgftWcy4Hwo= X-Google-Smtp-Source: ABdhPJx41L9fbecSDHgHsm+RXRQMzf/InZE73YhnkaM6hBV1I3wwwz7FQ41MROszkq3xU2rYawaGeg== X-Received: by 2002:a5d:4b88:: with SMTP id b8mr21610129wrt.95.1624797360296; Sun, 27 Jun 2021 05:36:00 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u18sm7289541wrt.76.2021.06.27.05.35.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:36:00 -0700 (PDT) Message-Id: <3fb47584924522e0bb32f667bb215b7d3223015f.1624797351.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:48 +0000 Subject: [PATCH v6 13/15] [GSOC] cat-file: reuse err buf in batch_object_write() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Reuse the `err` buffer in batch_object_write(), as the buffer `scratch` does. This will reduce the overhead of multiple allocations of memory of the err buffer. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 5b163551fc6..dc604a9879d 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -212,35 +212,36 @@ static void batch_write(struct batch_options *opt, const void *data, int len) static void batch_object_write(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { int ret; - struct strbuf err = STRBUF_INIT; struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); + strbuf_reset(err); - ret = format_ref_array_item(&item, &opt->format, scratch, &err); + ret = format_ref_array_item(&item, &opt->format, scratch, err); if (ret < 0) - die("%s\n", err.buf); + die("%s\n", err->buf); if (ret) { /* ret > 0 means when the object corresponding to oid * cannot be found in format_ref_array_item(), we only print * the error message. */ - printf("%s\n", err.buf); + printf("%s\n", err->buf); fflush(stdout); } else { strbuf_addch(scratch, '\n'); batch_write(opt, scratch->buf, scratch->len); } free_ref_array_item_value(&item); - strbuf_release(&err); } static void batch_one_object(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { @@ -294,7 +295,7 @@ static void batch_one_object(const char *obj_name, return; } - batch_object_write(obj_name, scratch, opt, data); + batch_object_write(obj_name, scratch, err, opt, data); } struct object_cb_data { @@ -302,13 +303,14 @@ struct object_cb_data { struct expand_data *expand; struct oidset *seen; struct strbuf *scratch; + struct strbuf *err; }; static int batch_object_cb(const struct object_id *oid, void *vdata) { struct object_cb_data *data = vdata; oidcpy(&data->expand->oid, oid); - batch_object_write(NULL, data->scratch, data->opt, data->expand); + batch_object_write(NULL, data->scratch, data->err, data->opt, data->expand); return 0; } @@ -364,6 +366,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; @@ -392,6 +395,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio cb.opt = batch; cb.expand = &data; cb.scratch = &output; + cb.err = &err; if (batch->unordered) { struct oidset seen = OIDSET_INIT; @@ -416,6 +420,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio strbuf_release(&format); strbuf_release(&output); + strbuf_release(&err); return 0; } @@ -444,12 +449,13 @@ static int batch_objects(struct batch_options *batch, const struct option *optio data.rest = p; } - batch_one_object(input.buf, &output, batch, &data); + batch_one_object(input.buf, &output, &err, batch, &data); } strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); + strbuf_release(&err); warn_on_object_refname_ambiguity = save_warning; return retval; } From patchwork Sun Jun 27 12:35:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A18DC49EAB for ; Sun, 27 Jun 2021 12:36:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E816361C17 for ; Sun, 27 Jun 2021 12:36:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231133AbhF0Mik (ORCPT ); Sun, 27 Jun 2021 08:38:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230379AbhF0Mi2 (ORCPT ); Sun, 27 Jun 2021 08:38:28 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47E18C0617A6 for ; Sun, 27 Jun 2021 05:36:02 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id r8so3269798wrx.5 for ; Sun, 27 Jun 2021 05:36:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=2rDfAXZC1ZPMjY20WsDhQLg7ngrQ22CtojUix0Y0b0Q=; b=eaT3Btb9fCzXEHeGfqsIi3LcCEnjBDFBPu11z01XqDwyaD9fC7VzI8iaxxIL2dMden JlMyk6o3i5x9YGTcerhRD3RkUAKpDm4vtRQPpkWZV/bmxepQWCEDv7o/+i61XpWV5c84 9/M1Iu5Bie2YV9xiYVwN8WzEcpn/Y2wrJV/0OtA63ynDfviUUluf5iUY57WptcOGWagc a6BB2DWfWLV4/m+g8dilrnCGKiEMuNOYoo2t52YChDHKFGlfPrKEsfu0UGZWCtNZVMqF HA/I0P/eKM3Ogcrg0tVJtP9cWjQyHOk1r42ro5CsIZPAYh216XmsO7zQ5yR+3OVRrin9 3lEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=2rDfAXZC1ZPMjY20WsDhQLg7ngrQ22CtojUix0Y0b0Q=; b=mUMAN93PTXe8zrWSCYVnhJXti+la57vFsiKX38wIuHYi1JYYWSkMhxCnjnuJg8bJhi BzQWIR5+xXfpOCnAeb19K/8It4/EPwHHQ9kRm5KSOD2juhxB3gFTG0H7PHHDRQixJII6 pwxyiapxQQzpQozuUNSnLZdK+iXBQcpeNTkwkskZ/qRq4tSr5uR4DyjDcC9bOBUrMHuU 0bZrXSbRTnOIg22BQj+4y6afTln86e2wGz7VAvyOtZpV4guf3WDfcjWGEIPd+HHEHvnn FDg6Xn6k6d9FSLVyjOKa6BqEGsv2nk/znKjKLMDS425Afla0z2fJcK2hyr3r/ND5dkhL ++BQ== X-Gm-Message-State: AOAM533CHNuAVgI7QMhTpZjtQN/A+hKi3gmUiBAWWIY8jB0xmzKg2Hft MQ1Pwi8ESZ/WZmTEUVkWVqhE+Xb7s8Y= X-Google-Smtp-Source: ABdhPJyyrDT9MnEm7i2UyvNXvgaLzCuM3EAzuZ1F2sx/27p+dkCJckOACfCNDHuz89Nm7iZhBeDp9A== X-Received: by 2002:adf:b35a:: with SMTP id k26mr22204497wrd.26.1624797360934; Sun, 27 Jun 2021 05:36:00 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c8sm11037041wri.91.2021.06.27.05.36.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:36:00 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:49 +0000 Subject: [PATCH v6 14/15] [GSOC] cat-file: re-implement --textconv, --filters options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu After cat-file reuses the ref-filter logic, we re-implement the functions of --textconv and --filters options. Add members `use_textconv` and `use_filters` in struct `ref_format`, and use global variables `use_filters` and `use_textconv` in `ref-filter.c`, so that we can filter the content of the object in get_object(). Use `actual_oi` to record the real expand_data: it may point to the original `oi` or the `act_oi` processed by `textconv_object()` or `convert_to_working_tree()`. `grab_values()` will grab the contents of `actual_oi` and `grab_common_values()` to grab the contents of origin `oi`, this ensures that `%(objectsize)` still uses the size of the unfiltered data. In `get_object()`, we made an optimization: Firstly, get the size and type of the object instead of directly getting the object data. If using --textconv, after successfully obtaining the filtered object data, an extra oid_object_info_extended() will be skipped, which can reduce the cost of object data copy; If using --filter, the data of the object first will be getted first, and then convert_to_working_tree() will be used to get the filtered object data. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 6 +++++ ref-filter.c | 59 ++++++++++++++++++++++++++++++++++++++++++++-- ref-filter.h | 2 ++ 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index dc604a9879d..710085b7d72 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -380,6 +380,12 @@ static int batch_objects(struct batch_options *batch, const struct option *optio if (batch->print_contents) strbuf_addstr(&format, "\n%(raw)"); batch->format.format = format.buf; + + if (batch->cmdmode == 'c') + batch->format.use_textconv = 1; + else if (batch->cmdmode == 'w') + batch->format.use_filters = 1; + if (verify_ref_format(&batch->format)) usage_with_options(cat_file_usage, options); diff --git a/ref-filter.c b/ref-filter.c index 9ca3dd5557d..427552b8108 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1,3 +1,4 @@ +#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "cache.h" #include "parse-options.h" @@ -84,6 +85,9 @@ static struct expand_data { struct object_info info; } oi, oi_deref; +int use_filters; +int use_textconv; + struct ref_to_worktree_entry { struct hashmap_entry ent; struct worktree *wt; /* key is wt->head_ref */ @@ -1031,6 +1035,9 @@ int verify_ref_format(struct ref_format *format) used_atom[at].atom_type == ATOM_WORKTREEPATH))) die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); + use_filters = format->use_filters; + use_textconv = format->use_textconv; + if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || format->quote_style == QUOTE_TCL) && @@ -1742,10 +1749,38 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj { /* parse_object_buffer() will set eaten to 0 if free() will be needed */ int eaten = 1; + struct expand_data *actual_oi = oi; + struct expand_data act_oi = {0}; + if (oi->info.contentp) { /* We need to know that to use parse_object_buffer properly */ + void **temp_contentp = oi->info.contentp; + oi->info.contentp = NULL; oi->info.sizep = &oi->size; oi->info.typep = &oi->type; + + /* get the type and size */ + if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, + OBJECT_INFO_LOOKUP_REPLACE)) + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); + + oi->info.sizep = NULL; + oi->info.typep = NULL; + oi->info.contentp = temp_contentp; + + if (use_textconv && !ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&act_oi.oid)); + if (use_textconv && oi->type == OBJ_BLOB) { + act_oi = *oi; + if (textconv_object(the_repository, + ref->rest, 0100644, &act_oi.oid, + 1, (char **)(&act_oi.content), &act_oi.size)) { + actual_oi = &act_oi; + goto success; + } + } } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) @@ -1755,19 +1790,39 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj BUG("Object size is less than zero."); if (oi->info.contentp) { - *obj = parse_object_buffer(the_repository, &oi->oid, oi->type, oi->size, oi->content, &eaten); + if (use_filters && !ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&oi->oid)); + if (use_filters && oi->type == OBJ_BLOB) { + struct strbuf strbuf = STRBUF_INIT; + struct checkout_metadata meta; + act_oi = *oi; + + init_checkout_metadata(&meta, NULL, NULL, &act_oi.oid); + if (!convert_to_working_tree(&the_index, ref->rest, act_oi.content, act_oi.size, &strbuf, &meta)) + die("could not convert '%s' %s", + oid_to_hex(&oi->oid), ref->rest); + act_oi.size = strbuf.len; + act_oi.content = strbuf_detach(&strbuf, NULL); + actual_oi = &act_oi; + } + +success: + *obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten); if (!*obj) { if (!eaten) free(oi->content); return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi); + grab_values(ref->value, deref, *obj, actual_oi); } grab_common_values(ref->value, deref, oi); if (!eaten) free(oi->content); + if (actual_oi != oi) + free(actual_oi->content); return 0; } diff --git a/ref-filter.h b/ref-filter.h index 053980a6a42..497e3e93632 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -80,6 +80,8 @@ struct ref_format { const char *rest; int cat_file_mode; int quote_style; + int use_textconv; + int use_filters; int use_rest; int use_color; From patchwork Sun Jun 27 12:35:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12346789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9717DC48BC2 for ; Sun, 27 Jun 2021 12:36:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 81FDC61C23 for ; Sun, 27 Jun 2021 12:36:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230315AbhF0Mii (ORCPT ); Sun, 27 Jun 2021 08:38:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230386AbhF0Mi2 (ORCPT ); Sun, 27 Jun 2021 08:38:28 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9B77C0617A8 for ; Sun, 27 Jun 2021 05:36:02 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id a5-20020a7bc1c50000b02901e3bbe0939bso9101790wmj.0 for ; Sun, 27 Jun 2021 05:36:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5ITrPb59JYnXOBip2Mfl5gdpEcDZXZOW9cu3U4epO2Y=; b=ijEBjyXQ5aQEDoEGtzUJpFjGnwNQCNE2hyvrAFQ1j6VGSXOeTuMEOE+IXJnwAyfxHK YR9bdU2rU7qjCgAyFKLB9AjAZjJ24TPy1FkvAsiNKzQNotC2U6vW5t9+HjQI1Dq0gK8d gfk77bwt4VFbcZxnxsjZCuQkgijTj8PGH04BFBYcTGdEda/S3RV2Wbi4CdBVBhd93ucy MD8m0X/vpeWqHRTDQYPKYUjghtvN+i3cevLaMNmPPS+cyz5BNrExgITbs4aPgCilwgUM aCI1ZkbaJbDncYO6TJwA62fUqhG5Fa3fGKUzFahD+X/KmG+zoPZ1f/RGwPFJMYWITEH+ uG+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5ITrPb59JYnXOBip2Mfl5gdpEcDZXZOW9cu3U4epO2Y=; b=h14TfRFKi/SS5uGpjiOqf91jjS7cr2Gsgrw2ADIvS2Uw21QxgGZ12QYvfZjxE121MZ MOUIgz00UJYNVIBi7faPc4a47h2ka5WqPB5K+6ybPItHPfCo2gQHV+CTKKgPIC1A7it4 eRNbGxwp1WZmzwL++FDOZ0OKp5RmqlVoBjBt/FURuieDslGChIT6XywsJPn1ySwWMnpx a1IRO4T94wVfS/tYZN03hgVCQWKXxTRmah3j8X6dMtydZCHgVAtsemSMd/HiWigC30Cw ABEBzO9g1exGfXcnVZZII3fqiop8r7tCxVJuXTE4uMl9bgJjbGX7vi5Y/lXkUYsLvD5Z yrpg== X-Gm-Message-State: AOAM5300lvUTcpiXKd+RsV2OJNvsEyDmKYQv+E+8zWP3sTkx/MdVqa2N rM986bOeQ83KsxCCNnOCZtL36rFiptU= X-Google-Smtp-Source: ABdhPJxlkqXt3FBaPcbaLQuh8IpSCloAz5dijExS/HvkBQQhYQMRqQ70WLC1ibb0YCqZHaGleR9YLw== X-Received: by 2002:a05:600c:3b1e:: with SMTP id m30mr21119653wms.162.1624797361478; Sun, 27 Jun 2021 05:36:01 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h9sm10215484wmb.35.2021.06.27.05.36.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Jun 2021 05:36:01 -0700 (PDT) Message-Id: <891d62fd93f795219518d85aaf0cd50c84d7c652.1624797351.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sun, 27 Jun 2021 12:35:50 +0000 Subject: [PATCH v6 15/15] [GSOC] ref-filter: remove grab_oid() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because "atom_type == ATOM_OBJECTNAME" implies the condition of `starts_with(name, "objectname")`, "atom_type == ATOM_TREE" implies the condition of `starts_with(name, "tree")`, so the check for `starts_with(name, field)` in grab_oid() is redundant. So Remove the grab_oid() from ref-filter, to reduce repeated check. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 427552b8108..55bff4ed9e7 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1071,16 +1071,6 @@ static const char *do_grab_oid(const char *field, const struct object_id *oid, } } -static int grab_oid(const char *name, const char *field, const struct object_id *oid, - struct atom_value *v, struct used_atom *atom) -{ - if (starts_with(name, field)) { - v->s = xstrdup(do_grab_oid(field, oid, atom)); - return 1; - } - return 0; -} - /* See grab_values */ static void grab_common_values(struct atom_value *val, int deref, struct expand_data *oi) { @@ -1106,8 +1096,9 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ } } else if (atom_type == ATOM_DELTABASE) v->s = xstrdup(oid_to_hex(&oi->delta_base_oid)); - else if (atom_type == ATOM_OBJECTNAME && deref) - grab_oid(name, "objectname", &oi->oid, v, &used_atom[i]); + else if (atom_type == ATOM_OBJECTNAME && deref) { + v->s = xstrdup(do_grab_oid("objectname", &oi->oid, &used_atom[i])); + } } } @@ -1148,9 +1139,10 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object continue; if (deref) name++; - if (atom_type == ATOM_TREE && - grab_oid(name, "tree", get_commit_tree_oid(commit), v, &used_atom[i])) + if (atom_type == ATOM_TREE) { + v->s = xstrdup(do_grab_oid("tree", get_commit_tree_oid(commit), &used_atom[i])); continue; + } if (atom_type == ATOM_NUMPARENT) { v->value = commit_list_count(commit->parents); v->s = xstrfmt("%lu", (unsigned long)v->value); @@ -1971,9 +1963,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->s = xstrdup(buf + 1); } continue; - } else if (!deref && atom_type == ATOM_OBJECTNAME && - grab_oid(name, "objectname", &ref->objectname, v, atom)) { - continue; + } else if (!deref && atom_type == ATOM_OBJECTNAME) { + v->s = xstrdup(do_grab_oid("objectname", &ref->objectname, atom)); + continue; } else if (atom_type == ATOM_HEAD) { if (atom->u.head && !strcmp(ref->refname, atom->u.head)) v->s = xstrdup("*");