From patchwork Tue Jun 15 13:28:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7235EC49361 for ; Tue, 15 Jun 2021 13:29:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5A5DB61460 for ; Tue, 15 Jun 2021 13:29:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231656AbhFONbh (ORCPT ); Tue, 15 Jun 2021 09:31:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230012AbhFONbQ (ORCPT ); Tue, 15 Jun 2021 09:31:16 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7AB05C0611C2 for ; Tue, 15 Jun 2021 06:29:10 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id k5-20020a05600c1c85b02901affeec3ef8so1813519wms.0 for ; Tue, 15 Jun 2021 06:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sslxU2Zj/H3zu0TbnQu1d/Mc+WxpiqXZj8t9ARAzits=; b=nvdJMJle3NaifwXpaSCM2jjsK7gxqPd/P04CXaqWYsbpVEyHsrohC7aT+51GcQIdmP vQ1T1qYLR+b5XeYjc1aoWt6BtkVdPwVUd/HX/F3W15RlEYYPAlNF+zcKuwX0Jpu7CAy4 hP1d6tW3EHCJkIpJRVjfi164SICQ8GhLIvadHZvHuHfzWCVvgXurcVo0eGoZRIk/4Q8k /Ghsb9euEmOSoXyi6wCvj6AefQxTEcb2wBAO7+3qWGVbGrKTMvHf1msmcfCWhP073s15 wiHhPfvVtzfcPpOPs2tz0li8mRWmjEvMiANXVjPKQCWL/h6J18TCGPWB1t+p4BgFCODj s52Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sslxU2Zj/H3zu0TbnQu1d/Mc+WxpiqXZj8t9ARAzits=; b=RZ4xTZtbgtTRBiNL9uApvJhNzkL0HYoVtBWl3wn9ngK3RLKkTi1XhlkjbRs52C0YMy 6MjRMQrwgMqzn8IE8X2yZB/RlTSReI3Wq9jrfISo7enXe3R5m8LUR4O/nvfA/vwXGxtC 0Z2xiAhkmJcOD5Nt13nu5an4Xb4glicRMFKRUe0Wq9eJ1JuVWYp9Gs9mS+6UX8xvtm5q LAGG9n5fhc15q7X//uk6V1JniNx2tmPPXjGXGxeQM4n+ZwEdkhuj5aib1C6tVeXgk7fd LS5/kKwrgviKyKrFCVb9n7FwkjksSYVn0jcILWxnsqPXSMmNlnUn0cX0wHtYABNOR61g 70NA== X-Gm-Message-State: AOAM530U+37ND3Otwq/AdGpTe9xMsmc3oMqPTFiAEp81V5g3lgD68zRH THDbGC0Ph8N8lfqr9ujrwmVdbaPfYCc= X-Google-Smtp-Source: ABdhPJwzvTPBp5vzuzCRxOf0/oQ1Sq2CN5i8kMyhp26e20akJurrWCXIW33LpDttUg2bgM6xR3PYNA== X-Received: by 2002:a1c:32c6:: with SMTP id y189mr5317392wmy.54.1623763749110; Tue, 15 Jun 2021 06:29:09 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n10sm17777458wri.77.2021.06.15.06.29.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:08 -0700 (PDT) Message-Id: <48d256db5c349c1fa0615bb60d74039c78a831fd.1623763746.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Jun 2021 13:28:57 +0000 Subject: [PATCH v2 1/9] [GSOC] ref-filter: add obj-type check in grab contents Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Only tag and commit objects use `grab_sub_body_contents()` to grab object contents in the current codebase. We want to teach the function to also handle blobs and trees to get their raw data, without parsing a blob (whose contents looks like a commit or a tag) incorrectly as a commit or a tag. Skip the block of code that is specific to handling commits and tags early when the given object is of a wrong type to help later addition to handle other types of objects in this function. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Junio C Hamano Signed-off-by: ZheNing Hu --- ref-filter.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 4db0e40ff4c6..5cee6512fbaf 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1356,11 +1356,12 @@ static void append_lines(struct strbuf *out, const char *buf, unsigned long size } /* See grab_values */ -static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) +static void grab_sub_body_contents(struct atom_value *val, int deref, struct expand_data *data) { int i; const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; + void *buf = data->content; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; @@ -1371,10 +1372,13 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) continue; if (deref) name++; - if (strcmp(name, "body") && - !starts_with(name, "subject") && - !starts_with(name, "trailers") && - !starts_with(name, "contents")) + + if ((data->type != OBJ_TAG && + data->type != OBJ_COMMIT) || + (strcmp(name, "body") && + !starts_with(name, "subject") && + !starts_with(name, "trailers") && + !starts_with(name, "contents"))) continue; if (!subpos) find_subpos(buf, @@ -1438,17 +1442,19 @@ static void fill_missing_values(struct atom_value *val) * pointed at by the ref itself; otherwise it is the object the * ref (which is a tag) refers to. */ -static void grab_values(struct atom_value *val, int deref, struct object *obj, void *buf) +static void grab_values(struct atom_value *val, int deref, struct object *obj, struct expand_data *data) { + void *buf = data->content; + switch (obj->type) { case OBJ_TAG: grab_tag_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("tagger", val, deref, buf); break; case OBJ_COMMIT: grab_commit_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("author", val, deref, buf); grab_person("committer", val, deref, buf); break; @@ -1678,7 +1684,7 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi->content); + grab_values(ref->value, deref, *obj, oi); } grab_common_values(ref->value, deref, oi); From patchwork Tue Jun 15 13:28:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3616BC48BE5 for ; Tue, 15 Jun 2021 13:29:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 189BA61465 for ; Tue, 15 Jun 2021 13:29:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231316AbhFONbl (ORCPT ); Tue, 15 Jun 2021 09:31:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231150AbhFONbR (ORCPT ); Tue, 15 Jun 2021 09:31:17 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44B02C0611C6 for ; Tue, 15 Jun 2021 06:29:11 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id d184so14167656wmd.0 for ; Tue, 15 Jun 2021 06:29:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=HVT0j/JcbPrT2B0Cpe6X0J1UF6zgo7yNBZx0b4JdHKw=; b=dFFWa9EeaLoWlbwx9OA/nYOz49TcS9z5CdylfJaE5d5SlB/WNq0auEgWcWQqK/ApeR AZR9BCPYEDE06Dn5YQEcs413Jpn7639GL6JxZx6e1hV0ws580lBPJHk8LlaCbIX5daKp 1f06pfHHdteW7TOPge+ajdxHYVovb7+t59H1dnP5YCKqK3Dj1AxaKZVEDEw+G9YsSeHd GQ0MXLC9SDAoBktQb5c4J2KRt2WxD/dS6t0uZIVRdnRqV7q5PQ9E2yejS8PzAAoz89/6 BY4yxHTCjSmcOeMVC8p7NHORr5ubyWYJdWbNxiIORCUs0uyQyRUPgIpD3sH7E/g4mn3p gqRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=HVT0j/JcbPrT2B0Cpe6X0J1UF6zgo7yNBZx0b4JdHKw=; b=IUAU4SsAKRWBo2vwb9blrlMoXcbtv6bu6ASF0ZbYQi+pGMIuuy3xqNFFT9Np5vwIcv wMQHtT4qJWi2Auz94aYVO3m5k5VQIGIpDjeKMHYtmR1vOcR2A8Gx6G1mwkaffVQ7oCJl sEsu7fLnXOuwndEgvkRdhU3W2eILvzvgNgGX8lZSUWGH49vC+i08CxREMeJHoKjI4EKy 63EfuWwPjDUvw2okbFI5nvA4m5JUhODHT61mZ8P4PWAVW22FC2fH5MWbvuqR0VP9iagL 8r0zx5c6b68J1UEOUp8a6ij5gTSR2PRYRDWUYFJSOTEocXOW13ea7q53U0L2MRAesK1D sR2w== X-Gm-Message-State: AOAM5302nkM5CKxDJ7ywGo4uAagZZ7/m+UIAHQYWAU//d07OlL0uys4C KGBNzInzda1vKFnJvy6VKhYcmXC3AWc= X-Google-Smtp-Source: ABdhPJwVT4okFMW3WZ9AUkfqkfKFuAC+6lTwMBKTY6zoJrEO8Ui8CINrzwVBZVTt84TFNSZ18z5TCA== X-Received: by 2002:a7b:c7cd:: with SMTP id z13mr22417460wmk.54.1623763749807; Tue, 15 Jun 2021 06:29:09 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y20sm1507397wma.45.2021.06.15.06.29.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:09 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 15 Jun 2021 13:28:58 +0000 Subject: [PATCH v2 2/9] [GSOC] ref-filter: add %(raw) atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add new formatting option `%(raw)`, which will print the raw object data without any changes. It will help further to migrate all cat-file formatting logic from cat-file to ref-filter. The raw data of blob, tree objects may contain '\0', but most of the logic in `ref-filter` depends on the output of the atom being text (specifically, no embedded NULs in it). E.g. `quote_formatting()` use `strbuf_addstr()` or `*._quote_buf()` add the data to the buffer. The raw data of a tree object is `100644 one\0...`, only the `100644 one` will be added to the buffer, which is incorrect. Therefore, add a new member in `struct atom_value`: `s_size`, which can record raw object size, it can help us add raw object data to the buffer or compare two buffers which contain raw object data. Beyond, `--format=%(raw)` cannot be used with `--python`, `--shell`, `--tcl`, `--perl` because if our binary raw data is passed to a variable in the host language, the host language may not support arbitrary binary data in the variables of its string type. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Felipe Contreras Helped-by: Phillip Wood Helped-by: Junio C Hamano Based-on-patch-by: Olga Telezhnaya Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 9 ++ ref-filter.c | 139 +++++++++++++++---- t/t6300-for-each-ref.sh | 207 +++++++++++++++++++++++++++++ 3 files changed, 328 insertions(+), 27 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 2ae2478de706..7f1f0a1ca3b6 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -235,6 +235,15 @@ and `date` to extract the named component. For email fields (`authoremail`, without angle brackets, and `:localpart` to get the part before the `@` symbol out of the trimmed email. +The raw data in an object is `raw`. + +raw:size:: + The raw data size of the object. + +Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, +`--perl` because the host language may not support arbitrary binary data in the +variables of its string type. + The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/ref-filter.c b/ref-filter.c index 5cee6512fbaf..7822be903071 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -144,6 +144,7 @@ enum atom_type { ATOM_BODY, ATOM_TRAILERS, ATOM_CONTENTS, + ATOM_RAW, ATOM_UPSTREAM, ATOM_PUSH, ATOM_SYMREF, @@ -189,6 +190,9 @@ static struct used_atom { struct process_trailer_options trailer_opts; unsigned int nlines; } contents; + struct { + enum { RAW_BARE, RAW_LENGTH } option; + } raw_data; struct { cmp_status cmp_status; const char *str; @@ -426,6 +430,18 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } +static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (!arg) + atom->u.raw_data.option = RAW_BARE; + else if (!strcmp(arg, "size")) + atom->u.raw_data.option = RAW_LENGTH; + else + return strbuf_addf_ret(err, -1, _("unrecognized %%(raw) argument: %s"), arg); + return 0; +} + static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { @@ -586,6 +602,7 @@ static struct { [ATOM_BODY] = { "body", SOURCE_OBJ, FIELD_STR, body_atom_parser }, [ATOM_TRAILERS] = { "trailers", SOURCE_OBJ, FIELD_STR, trailers_atom_parser }, [ATOM_CONTENTS] = { "contents", SOURCE_OBJ, FIELD_STR, contents_atom_parser }, + [ATOM_RAW] = { "raw", SOURCE_OBJ, FIELD_STR, raw_atom_parser }, [ATOM_UPSTREAM] = { "upstream", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_PUSH] = { "push", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_SYMREF] = { "symref", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -620,12 +637,15 @@ struct ref_formatting_state { struct atom_value { const char *s; + size_t s_size; int (*handler)(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err); uintmax_t value; /* used for sorting when not FIELD_STR */ struct used_atom *atom; }; +#define ATOM_VALUE_S_SIZE_INIT (-1) + /* * Used to parse format string and sort specifiers */ @@ -644,13 +664,6 @@ static int parse_ref_filter_atom(const struct ref_format *format, return strbuf_addf_ret(err, -1, _("malformed field name: %.*s"), (int)(ep-atom), atom); - /* Do we have the atom already used elsewhere? */ - for (i = 0; i < used_atom_cnt; i++) { - int len = strlen(used_atom[i].name); - if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) - return i; - } - /* * If the atom name has a colon, strip it and everything after * it off - it specifies the format for this entry, and @@ -660,6 +673,13 @@ static int parse_ref_filter_atom(const struct ref_format *format, arg = memchr(sp, ':', ep - sp); atom_len = (arg ? arg : ep) - sp; + /* Do we have the atom already used elsewhere? */ + for (i = 0; i < used_atom_cnt; i++) { + int len = strlen(used_atom[i].name); + if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) + return i; + } + /* Is the atom a valid one? */ for (i = 0; i < ARRAY_SIZE(valid_atom); i++) { int len = strlen(valid_atom[i].name); @@ -709,11 +729,14 @@ static int parse_ref_filter_atom(const struct ref_format *format, return at; } -static void quote_formatting(struct strbuf *s, const char *str, int quote_style) +static void quote_formatting(struct strbuf *s, const char *str, size_t len, int quote_style) { switch (quote_style) { case QUOTE_NONE: - strbuf_addstr(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(s, str, len); + else + strbuf_addstr(s, str); break; case QUOTE_SHELL: sq_quote_buf(s, str); @@ -740,9 +763,12 @@ static int append_atom(struct atom_value *v, struct ref_formatting_state *state, * encountered. */ if (!state->stack->prev) - quote_formatting(&state->stack->output, v->s, state->quote_style); + quote_formatting(&state->stack->output, v->s, v->s_size, state->quote_style); else - strbuf_addstr(&state->stack->output, v->s); + if (v->s_size != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(&state->stack->output, v->s, v->s_size); + else + strbuf_addstr(&state->stack->output, v->s); return 0; } @@ -842,21 +868,23 @@ static int if_atom_handler(struct atom_value *atomv, struct ref_formatting_state return 0; } -static int is_empty(const char *s) +static int is_empty(struct strbuf *buf) { - while (*s != '\0') { - if (!isspace(*s)) - return 0; - s++; - } - return 1; -} + const char *cur = buf->buf; + const char *end = buf->buf + buf->len; + + while (cur != end && (isspace(*cur))) + cur++; + + return cur == end; + } static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err) { struct ref_formatting_stack *cur = state->stack; struct if_then_else *if_then_else = NULL; + size_t str_len = 0; if (cur->at_end == if_then_else_handler) if_then_else = (struct if_then_else *)cur->at_end_data; @@ -867,18 +895,22 @@ static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_sta if (if_then_else->else_atom_seen) return strbuf_addf_ret(err, -1, _("format: %%(then) atom used after %%(else)")); if_then_else->then_atom_seen = 1; + if (if_then_else->str) + str_len = strlen(if_then_else->str); /* * If the 'equals' or 'notequals' attribute is used then * perform the required comparison. If not, only non-empty * strings satisfy the 'if' condition. */ if (if_then_else->cmp_status == COMPARE_EQUAL) { - if (!strcmp(if_then_else->str, cur->output.buf)) + if (str_len == cur->output.len && + !memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; } else if (if_then_else->cmp_status == COMPARE_UNEQUAL) { - if (strcmp(if_then_else->str, cur->output.buf)) + if (str_len != cur->output.len || + memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; - } else if (cur->output.len && !is_empty(cur->output.buf)) + } else if (cur->output.len && !is_empty(&cur->output)) if_then_else->condition_satisfied = 1; strbuf_reset(&cur->output); return 0; @@ -924,7 +956,7 @@ static int end_atom_handler(struct atom_value *atomv, struct ref_formatting_stat * only on the topmost supporting atom. */ if (!current->prev->prev) { - quote_formatting(&s, current->output.buf, state->quote_style); + quote_formatting(&s, current->output.buf, current->output.len, state->quote_style); strbuf_swap(¤t->output, &s); } strbuf_release(&s); @@ -974,6 +1006,10 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) + die(_("--format=%.*s cannot be used with" + "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) @@ -1362,17 +1398,29 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; void *buf = data->content; + unsigned long buf_size = data->size; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; const char *name = atom->name; struct atom_value *v = &val[i]; + enum atom_type atom_type = atom->atom_type; if (!!deref != (*name == '*')) continue; if (deref) name++; + if (atom_type == ATOM_RAW) { + if (atom->u.raw_data.option == RAW_BARE) { + v->s = xmemdupz(buf, buf_size); + v->s_size = buf_size; + } else if (atom->u.raw_data.option == RAW_LENGTH) { + v->s = xstrfmt("%"PRIuMAX, (uintmax_t)buf_size); + } + continue; + } + if ((data->type != OBJ_TAG && data->type != OBJ_COMMIT) || (strcmp(name, "body") && @@ -1460,9 +1508,11 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s break; case OBJ_TREE: /* grab_tree_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; case OBJ_BLOB: /* grab_blob_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; default: die("Eh? Object of type %d?", obj->type); @@ -1766,6 +1816,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) const char *refname; struct branch *branch = NULL; + v->s_size = ATOM_VALUE_S_SIZE_INIT; v->handler = append_atom; v->atom = atom; @@ -2369,6 +2420,19 @@ static int compare_detached_head(struct ref_array_item *a, struct ref_array_item return 0; } +static int memcasecmp(const void *vs1, const void *vs2, size_t n) +{ + const char *s1 = vs1, *s2 = vs2; + const char *end = s1 + n; + + for (; s1 < end; s1++, s2++) { + int diff = tolower(*s1) - tolower(*s2); + if (diff) + return diff; + } + return 0; +} + static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, struct ref_array_item *b) { struct atom_value *va, *vb; @@ -2389,10 +2453,30 @@ static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, stru } else if (s->sort_flags & REF_SORTING_VERSION) { cmp = versioncmp(va->s, vb->s); } else if (cmp_type == FIELD_STR) { - int (*cmp_fn)(const char *, const char *); - cmp_fn = s->sort_flags & REF_SORTING_ICASE - ? strcasecmp : strcmp; - cmp = cmp_fn(va->s, vb->s); + if (va->s_size == ATOM_VALUE_S_SIZE_INIT && + vb->s_size == ATOM_VALUE_S_SIZE_INIT) { + int (*cmp_fn)(const char *, const char *); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? strcasecmp : strcmp; + cmp = cmp_fn(va->s, vb->s); + } else { + size_t a_size = va->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(va->s) : va->s_size; + size_t b_size = vb->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(vb->s) : vb->s_size; + int (*cmp_fn)(const void *, const void *, size_t); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? memcasecmp : memcmp; + + cmp = cmp_fn(va->s, vb->s, b_size > a_size ? + a_size : b_size); + if (!cmp) { + if (a_size > b_size) + cmp = 1; + else if (a_size < b_size) + cmp = -1; + } + } } else { if (va->value < vb->value) cmp = -1; @@ -2492,6 +2576,7 @@ int format_ref_array_item(struct ref_array_item *info, } if (format->need_color_reset_at_eol) { struct atom_value resetv; + resetv.s_size = ATOM_VALUE_S_SIZE_INIT; resetv.s = GIT_COLOR_RESET; if (append_atom(&resetv, &state, error_buf)) { pop_stack_element(&state.stack); diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 9e0214076b4d..e2867de791e7 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -130,6 +130,8 @@ test_atom head parent:short=10 '' test_atom head numparent 0 test_atom head object '' test_atom head type '' +test_atom head raw "$(git cat-file commit refs/heads/main) +" test_atom head '*objectname' '' test_atom head '*objecttype' '' test_atom head author 'A U Thor 1151968724 +0200' @@ -221,6 +223,15 @@ test_atom tag contents 'Tagging at 1151968727 ' test_atom tag HEAD ' ' +test_expect_success 'basic atom: refs/tags/testtag *raw' ' + git cat-file commit refs/tags/testtag^{} >expected && + git for-each-ref --format="%(*raw)" refs/tags/testtag >actual && + sanitize_pgp expected.clean && + sanitize_pgp actual.clean && + echo "" >>expected.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'Check invalid atoms names are errors' ' test_must_fail git for-each-ref --format="%(INVALID)" refs/heads ' @@ -686,6 +697,15 @@ test_atom refs/tags/signed-empty contents:body '' test_atom refs/tags/signed-empty contents:signature "$sig" test_atom refs/tags/signed-empty contents "$sig" +test_expect_success GPG 'basic atom: refs/tags/signed-empty raw' ' + git cat-file tag refs/tags/signed-empty >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-empty >actual && + sanitize_pgp expected.clean && + sanitize_pgp actual.clean && + echo "" >>expected.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-short subject 'subject line' test_atom refs/tags/signed-short subject:sanitize 'subject-line' test_atom refs/tags/signed-short contents:subject 'subject line' @@ -695,6 +715,15 @@ test_atom refs/tags/signed-short contents:signature "$sig" test_atom refs/tags/signed-short contents "subject line $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-short raw' ' + git cat-file tag refs/tags/signed-short >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-short >actual && + sanitize_pgp expected.clean && + sanitize_pgp actual.clean && + echo "" >>expected.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-long subject 'subject line' test_atom refs/tags/signed-long subject:sanitize 'subject-line' test_atom refs/tags/signed-long contents:subject 'subject line' @@ -708,6 +737,15 @@ test_atom refs/tags/signed-long contents "subject line body contents $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-long raw' ' + git cat-file tag refs/tags/signed-long >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-long >actual && + sanitize_pgp expected.clean && + sanitize_pgp actual.clean && + echo "" >>expected.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'set up refs pointing to tree and blob' ' git update-ref refs/mytrees/first refs/heads/main^{tree} && git update-ref refs/myblobs/first refs/heads/main:one @@ -720,6 +758,16 @@ test_atom refs/mytrees/first contents:body "" test_atom refs/mytrees/first contents:signature "" test_atom refs/mytrees/first contents "" +test_expect_success 'basic atom: refs/mytrees/first raw' ' + git cat-file tree refs/mytrees/first >expected && + echo "" >>expected && + git for-each-ref --format="%(raw)" refs/mytrees/first >actual && + test_cmp expected actual && + git cat-file -s refs/mytrees/first >expected && + git for-each-ref --format="%(raw:size)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_atom refs/myblobs/first subject "" test_atom refs/myblobs/first contents:subject "" test_atom refs/myblobs/first body "" @@ -727,6 +775,165 @@ test_atom refs/myblobs/first contents:body "" test_atom refs/myblobs/first contents:signature "" test_atom refs/myblobs/first contents "" +test_expect_success 'basic atom: refs/myblobs/first raw' ' + git cat-file blob refs/myblobs/first >expected && + echo "" >>expected && + git for-each-ref --format="%(raw)" refs/myblobs/first >actual && + test_cmp expected actual && + git cat-file -s refs/myblobs/first >expected && + git for-each-ref --format="%(raw:size)" refs/myblobs/first >actual && + test_cmp expected actual +' + +test_expect_success 'set up refs pointing to binary blob' ' + printf "a\0b\0c" >blob1 && + printf "a\0c\0b" >blob2 && + printf "\0a\0b\0c" >blob3 && + printf "abc" >blob4 && + printf "\0 \0 \0 " >blob5 && + printf "\0 \0a\0 " >blob6 && + printf " " >blob7 && + >blob8 && + git hash-object blob1 -w | xargs git update-ref refs/myblobs/blob1 && + git hash-object blob2 -w | xargs git update-ref refs/myblobs/blob2 && + git hash-object blob3 -w | xargs git update-ref refs/myblobs/blob3 && + git hash-object blob4 -w | xargs git update-ref refs/myblobs/blob4 && + git hash-object blob5 -w | xargs git update-ref refs/myblobs/blob5 && + git hash-object blob6 -w | xargs git update-ref refs/myblobs/blob6 && + git hash-object blob7 -w | xargs git update-ref refs/myblobs/blob7 && + git hash-object blob8 -w | xargs git update-ref refs/myblobs/blob8 +' + +test_expect_success 'Verify sorts with raw' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob3 + refs/myblobs/blob7 + refs/mytrees/first + refs/myblobs/first + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob4 + refs/heads/main + EOF + git for-each-ref --format="%(refname)" --sort=raw \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'Verify sorts with raw:size' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/first + refs/myblobs/blob7 + refs/heads/main + refs/myblobs/blob4 + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/mytrees/first + EOF + git for-each-ref --format="%(refname)" --sort=raw:size \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:equals)' ' + cat >expected <<-EOF && + not equals + not equals + not equals + not equals + not equals + not equals + refs/myblobs/blob4 + not equals + not equals + not equals + not equals + not equals + EOF + git for-each-ref --format="%(if:equals=abc)%(raw)%(then)%(refname)%(else)not equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' +test_expect_success 'validate raw atom with %(if:notequals)' ' + cat >expected <<-EOF && + refs/heads/ambiguous + refs/heads/main + refs/heads/newtag + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + equals + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob7 + refs/myblobs/blob8 + refs/myblobs/first + EOF + git for-each-ref --format="%(if:notequals=abc)%(raw)%(then)%(refname)%(else)equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'empty raw refs with %(if)' ' + cat >expected <<-EOF && + refs/myblobs/blob1 not empty + refs/myblobs/blob2 not empty + refs/myblobs/blob3 not empty + refs/myblobs/blob4 not empty + refs/myblobs/blob5 not empty + refs/myblobs/blob6 not empty + refs/myblobs/blob7 empty + refs/myblobs/blob8 empty + refs/myblobs/first not empty + EOF + git for-each-ref --format="%(refname) %(if)%(raw)%(then)not empty%(else)empty%(end)" \ + refs/myblobs/ >actual && + test_cmp expected actual +' + +test_expect_success '%(raw) with --python must failed' ' + test_must_fail git for-each-ref --format="%(raw)" --python +' + +test_expect_success '%(raw) with --tcl must failed' ' + test_must_fail git for-each-ref --format="%(raw)" --tcl +' + +test_expect_success '%(raw) with --perl must failed' ' + test_must_fail git for-each-ref --format="%(raw)" --perl +' + +test_expect_success '%(raw) with --shell must failed' ' + test_must_fail git for-each-ref --format="%(raw)" --shell +' + +test_expect_success '%(raw) with --shell and --sort=raw must failed' ' + test_must_fail git for-each-ref --format="%(raw)" --sort=raw --shell +' + +test_expect_success '%(raw:size) with --shell' ' + git for-each-ref --format="%(raw:size)" | while read line + do + echo "'\''$line'\''" >>expect + done && + git for-each-ref --format="%(raw:size)" --shell >actual && + test_cmp expect actual +' + +test_expect_success 'for-each-ref --format compare with cat-file --batch' ' + git rev-parse refs/mytrees/first | git cat-file --batch >expected && + git for-each-ref --format="%(objectname) %(objecttype) %(objectsize) +%(raw)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_expect_success 'set up multiple-sort tags' ' for when in 100000 200000 do From patchwork Tue Jun 15 13:28:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B4EFC49EA2 for ; Tue, 15 Jun 2021 13:29:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6345B61465 for ; Tue, 15 Jun 2021 13:29:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231635AbhFONbo (ORCPT ); Tue, 15 Jun 2021 09:31:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231510AbhFONbS (ORCPT ); Tue, 15 Jun 2021 09:31:18 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B412EC0611F8 for ; Tue, 15 Jun 2021 06:29:11 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id t4-20020a1c77040000b029019d22d84ebdso1767827wmi.3 for ; Tue, 15 Jun 2021 06:29:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=SZGQ943rib04/eVNIKdq6fOG3+lsHpv0pHGs42Vf0XU=; b=kCl5ENcaF5KDdow6xg9OAK8KNAVPyKSMw6Hct4C5RWWwjDlhlcxBkOEiwtNM7NSLec jIQns6IqK6K46dNj6qPOWYaZAPACTMXkr+BxsPq+Yiyc4P8djrFu1Uum1a3YGP4t2S8D b7KiHj72uW1W8O9CaGZdJgMBOVxlceNHu4g4fU4amY9ZtKI8iqXgJm2F2Q8eMEChHdsg Usqsvw5Qu9lT/XnDcFNvFpveKFPHt+LzSl8YM8n8JV5kwsCTXKxz/54CcjTxE7DZttv6 sHx2h6TH9nxx4utTe313KRj4NDQixGEs6+TGpT2wKh4RSPMQscKBwkmpUtrvFvYjL6Ly 6QqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=SZGQ943rib04/eVNIKdq6fOG3+lsHpv0pHGs42Vf0XU=; b=I9pFEpDQtpAMXgxKfZ05nt7weB7q20bfb3FLNLOPTep45Ty8cTZhywjFYve2sXQxeN 9Gl3rgAUmFHpOpGH6w3vXOXeXnss3sct5Vrb85HR7ICpa5iC+tnjQwl4OnaYGW+5+lZb KKhJ2Jb2wi/BcJ3PATTaf0Hhvk6aYtOn04GhZCHChhRLSyadEDtNNDJPtE7fc3fH77q3 oFPDf1/2inq9ljg8vKeknFD+SQxB9RgAeW3G3bDKh4cPr7AsVTSsGXYHru9gcCPMqb9j vTh7Zgbw40mAqbyVWuftVtrpuYXiVhuhBitqi1ynXhgeyjekHbJPHl2ardCaPovfxJ2J pvvA== X-Gm-Message-State: AOAM530AXRrOiiplMxfBXz6p83Db732VyO3u31vg3NHvg8uYLYRRumpX AQkxf9oXtFbEl/yJElA4fbHzfONX0+4= X-Google-Smtp-Source: ABdhPJyxPA8QCE+mt8phC+VbugmQI10lbhkBfhXrGjyL7OKqX8PGG037MXAfi7UCMnxQ21M1EJSliQ== X-Received: by 2002:a1c:6a09:: with SMTP id f9mr5300550wmc.91.1623763750370; Tue, 15 Jun 2021 06:29:10 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m18sm1320177wmq.45.2021.06.15.06.29.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:10 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 15 Jun 2021 13:28:59 +0000 Subject: [PATCH v2 3/9] [GSOC] ref-filter: use non-const ref_format in *_atom_parser() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use non-const ref_format in *_atom_parser(), which can help us modify the members of ref_format in *_atom_parser(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/tag.c | 2 +- ref-filter.c | 44 ++++++++++++++++++++++---------------------- ref-filter.h | 4 ++-- 3 files changed, 25 insertions(+), 25 deletions(-) diff --git a/builtin/tag.c b/builtin/tag.c index 82fcfc098242..452558ec9575 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -146,7 +146,7 @@ static int verify_tag(const char *name, const char *ref, const struct object_id *oid, void *cb_data) { int flags; - const struct ref_format *format = cb_data; + struct ref_format *format = cb_data; flags = GPG_VERIFY_VERBOSE; if (format->format) diff --git a/ref-filter.c b/ref-filter.c index 7822be903071..af8c15aef44d 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -226,7 +226,7 @@ static int strbuf_addf_ret(struct strbuf *sb, int ret, const char *fmt, ...) return ret; } -static int color_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int color_atom_parser(struct ref_format *format, struct used_atom *atom, const char *color_value, struct strbuf *err) { if (!color_value) @@ -264,7 +264,7 @@ static int refname_atom_parser_internal(struct refname_atom *atom, const char *a return 0; } -static int remote_ref_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int remote_ref_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct string_list params = STRING_LIST_INIT_DUP; @@ -311,7 +311,7 @@ static int remote_ref_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objecttype_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objecttype_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -323,7 +323,7 @@ static int objecttype_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objectsize_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objectsize_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -343,7 +343,7 @@ static int objectsize_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int deltabase_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int deltabase_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -355,7 +355,7 @@ static int deltabase_atom_parser(const struct ref_format *format, struct used_at return 0; } -static int body_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int body_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -364,7 +364,7 @@ static int body_atom_parser(const struct ref_format *format, struct used_atom *a return 0; } -static int subject_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int subject_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -376,7 +376,7 @@ static int subject_atom_parser(const struct ref_format *format, struct used_atom return 0; } -static int trailers_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int trailers_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { atom->u.contents.trailer_opts.no_divider = 1; @@ -402,7 +402,7 @@ static int trailers_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int contents_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int contents_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -430,7 +430,7 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int raw_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -442,7 +442,7 @@ static int raw_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int oid_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -461,7 +461,7 @@ static int oid_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int person_email_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int person_email_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -475,7 +475,7 @@ static int person_email_atom_parser(const struct ref_format *format, struct used return 0; } -static int refname_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int refname_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { return refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err); @@ -492,7 +492,7 @@ static align_type parse_align_position(const char *s) return -1; } -static int align_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int align_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct align *align = &atom->u.align; @@ -544,7 +544,7 @@ static int align_atom_parser(const struct ref_format *format, struct used_atom * return 0; } -static int if_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int if_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -559,7 +559,7 @@ static int if_atom_parser(const struct ref_format *format, struct used_atom *ato return 0; } -static int head_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { atom->u.head = resolve_refdup("HEAD", RESOLVE_REF_READING, NULL, NULL); @@ -570,7 +570,7 @@ static struct { const char *name; info_source source; cmp_type cmp_type; - int (*parser)(const struct ref_format *format, struct used_atom *atom, + int (*parser)(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err); } valid_atom[] = { [ATOM_REFNAME] = { "refname", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -649,7 +649,7 @@ struct atom_value { /* * Used to parse format string and sort specifiers */ -static int parse_ref_filter_atom(const struct ref_format *format, +static int parse_ref_filter_atom(struct ref_format *format, const char *atom, const char *ep, struct strbuf *err) { @@ -2546,9 +2546,9 @@ static void append_literal(const char *cp, const char *ep, struct ref_formatting } int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, - struct strbuf *final_buf, - struct strbuf *error_buf) + struct ref_format *format, + struct strbuf *final_buf, + struct strbuf *error_buf) { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; @@ -2593,7 +2593,7 @@ int format_ref_array_item(struct ref_array_item *info, } void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format) + struct ref_format *format) { struct ref_array_item *ref_item; struct strbuf output = STRBUF_INIT; diff --git a/ref-filter.h b/ref-filter.h index baf72a718965..74fb423fc89f 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -116,7 +116,7 @@ void ref_array_sort(struct ref_sorting *sort, struct ref_array *array); void ref_sorting_set_sort_flags_all(struct ref_sorting *sorting, unsigned int mask, int on); /* Based on the given format and quote_style, fill the strbuf */ int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, + struct ref_format *format, struct strbuf *final_buf, struct strbuf *error_buf); /* Parse a single sort specifier and add it to the list */ @@ -137,7 +137,7 @@ void setup_ref_filter_porcelain_msg(void); * name must be a fully qualified refname. */ void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format); + struct ref_format *format); /* * Push a single ref onto the array; this can be used to construct your own From patchwork Tue Jun 15 13:29:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E29DAC49361 for ; Tue, 15 Jun 2021 13:29:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C9F5961474 for ; Tue, 15 Jun 2021 13:29:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231682AbhFONbl (ORCPT ); Tue, 15 Jun 2021 09:31:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231543AbhFONbR (ORCPT ); Tue, 15 Jun 2021 09:31:17 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B295C0611F9 for ; Tue, 15 Jun 2021 06:29:12 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id z8so18355725wrp.12 for ; Tue, 15 Jun 2021 06:29:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=kqWel9z3Hvyqb4Y3h2V+hnS5XlLoK+zT/HLm6vFLVMM=; b=mt9nXDlDP3Z3ocRvxxGFeaHE378SEIqmoz6bSMCiHv+6pO54MwUvEaeDUiY8Fpbet/ sgK+rNAWs+jN38jWml74QLS1uIgDF4CzH+y+S8QRucc9nB1DjDqbeTZvYln0jz2lslei oys9zQW25UJ0WT0Ku5Qs7N0uOFIegV1GDQtHgfz/jurZ6aHmOfWNvlC3iFbpIOw+w1Jh yiWzlPPW/NL/dSPCE9I1rHOeqv5qhUZbOGfgZuz0R4E/6O5WIaHM4rgapbBwVZlTczZ3 Dcv8mssUakJE3lfwL/2tZ14BsqFOGGEOx2F2PzV0T6gxkDqp9QvPnh1UCVkei6yJhdh7 ho2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=kqWel9z3Hvyqb4Y3h2V+hnS5XlLoK+zT/HLm6vFLVMM=; b=BZSr3B0L2xpJpRQ3PYIvOBJfcgXKrgAiwlD4IZupLnDcFAHqkAa+HDP8uVOdnuzoo1 v74g0KG0EhALeMlzsEvCyfQOuI1QBWoz/0hFKnDSmM8/6CbrQafZavjXQmabJdug/fy+ hkf/HcdDw41PLQlVlP1OnYwyGhkwTU9VRqAYeefcyBPS8Dzbsn2UQM2alWxHlKzXl4VJ vzJlrhF+ZRucGMmvphk8UiZEROA7rCSMdu3PV5pR1hmrztKwm7Y2I+t5+LWEY8lF80Dc 0n50Q1OkOviQqvaezGpdUji0H3hOK891KnIBvaNKvePpfsJWMpY0dj2SeOLUCeTPjD0f +TwQ== X-Gm-Message-State: AOAM533wr5PHbVWSoyy3fQ1kYaHIC1UJRpOfEBzvW7K34qW+ep/kySN3 lyP89fIP6I6Mou0MsGXsysn/X1wuYdk= X-Google-Smtp-Source: ABdhPJyggJxVweUeAlbu4iGPVxcQANBmLTADBmesc3zNCpNb46gaY+iwIu1SARIoVnzUuTSrOO3r2Q== X-Received: by 2002:adf:df87:: with SMTP id z7mr25398513wrl.56.1623763750920; Tue, 15 Jun 2021 06:29:10 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r6sm18791441wrz.91.2021.06.15.06.29.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:10 -0700 (PDT) Message-Id: <5a5b5f78aeeac1f541852dc219d617530fbe87ea.1623763746.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Jun 2021 13:29:00 +0000 Subject: [PATCH v2 4/9] [GSOC] ref-filter: add %(rest) atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let "cat-file --batch=%(rest)" use the ref-filter interface, add %(rest) atom for ref-filter. "git for-each-ref", "git branch", "git tag" and "git verify-tag" will reject %(rest) by default. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 21 +++++++++++++++++++++ ref-filter.h | 5 ++++- t/t3203-branch-output.sh | 4 ++++ t/t6300-for-each-ref.sh | 4 ++++ t/t7004-tag.sh | 4 ++++ t/t7030-verify-tag.sh | 4 ++++ 6 files changed, 41 insertions(+), 1 deletion(-) diff --git a/ref-filter.c b/ref-filter.c index af8c15aef44d..8868cf98f090 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -157,6 +157,7 @@ enum atom_type { ATOM_IF, ATOM_THEN, ATOM_ELSE, + ATOM_REST, }; /* @@ -559,6 +560,15 @@ static int if_atom_parser(struct ref_format *format, struct used_atom *atom, return 0; } +static int rest_atom_parser(struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (arg) + return strbuf_addf_ret(err, -1, _("%%(rest) does not take arguments")); + format->use_rest = 1; + return 0; +} + static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { @@ -615,6 +625,7 @@ static struct { [ATOM_IF] = { "if", SOURCE_NONE, FIELD_STR, if_atom_parser }, [ATOM_THEN] = { "then", SOURCE_NONE }, [ATOM_ELSE] = { "else", SOURCE_NONE }, + [ATOM_REST] = { "rest", SOURCE_NONE, FIELD_STR, rest_atom_parser }, /* * Please update $__git_ref_fieldlist in git-completion.bash * when you add new atoms @@ -1006,6 +1017,9 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (used_atom[at].atom_type == ATOM_REST) + die("this command reject atom %%(%.*s)", (int)(ep - sp - 2), sp + 2); + if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && used_atom[at].u.raw_data.option == RAW_BARE) die(_("--format=%.*s cannot be used with" @@ -1920,6 +1934,12 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->handler = else_atom_handler; v->s = xstrdup(""); continue; + } else if (atom_type == ATOM_REST) { + if (ref->rest) + v->s = xstrdup(ref->rest); + else + v->s = xstrdup(""); + continue; } else continue; @@ -2137,6 +2157,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); + ref->rest = NULL; return ref; } diff --git a/ref-filter.h b/ref-filter.h index 74fb423fc89f..9dc07476a584 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -38,6 +38,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; + const char *rest; int flag; unsigned int kind; const char *symref; @@ -76,14 +77,16 @@ struct ref_format { * verify_ref_format() afterwards to finalize. */ const char *format; + const char *rest; int quote_style; + int use_rest; int use_color; /* Internal state to ref-filter */ int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { NULL, 0, -1 } +#define REF_FORMAT_INIT { NULL, NULL, 0, 0, -1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ diff --git a/t/t3203-branch-output.sh b/t/t3203-branch-output.sh index 5325b9f67a00..2780ec8803fd 100755 --- a/t/t3203-branch-output.sh +++ b/t/t3203-branch-output.sh @@ -340,6 +340,10 @@ test_expect_success 'git branch --format option' ' test_cmp expect actual ' +test_expect_success 'git branch with --format=%(rest) must failed' ' + test_must_fail git branch --format="%(rest)" >actual +' + test_expect_success 'worktree colors correct' ' cat >expect <<-EOF && * (HEAD detached from fromtag) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index e2867de791e7..8c97c3b877c6 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -1187,6 +1187,10 @@ test_expect_success 'basic atom: head contents:trailers' ' test_cmp expect actual.clean ' +test_expect_success 'basic atom: rest must failed' ' + test_must_fail git for-each-ref --format="%(rest)" refs/heads/main +' + test_expect_success 'trailer parsing not fooled by --- line' ' git commit --allow-empty -F - <<-\EOF && this is the subject diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh index 2f72c5c6883e..9fc4c4323949 100755 --- a/t/t7004-tag.sh +++ b/t/t7004-tag.sh @@ -1998,6 +1998,10 @@ test_expect_success '--format should list tags as per format given' ' test_cmp expect actual ' +test_expect_success 'git tag -l with --format="%(rest)" must failed' ' + test_must_fail git tag -l --format="%(rest)" "v1*" +' + test_expect_success "set up color tests" ' echo "v1.0" >expect.color && echo "v1.0" >expect.bare && diff --git a/t/t7030-verify-tag.sh b/t/t7030-verify-tag.sh index 3cefde9602bf..785b32eb88f9 100755 --- a/t/t7030-verify-tag.sh +++ b/t/t7030-verify-tag.sh @@ -194,6 +194,10 @@ test_expect_success GPG 'verifying tag with --format' ' test_cmp expect actual ' +test_expect_success GPG 'verifying tag with --format="%(rest)" must failed' ' + test_must_fail git verify-tag --format="%(rest)" "fourth-signed" +' + test_expect_success GPG 'verifying a forged tag with --format should fail silently' ' test_must_fail git verify-tag --format="tagname : %(tag)" $(cat forged1.tag) >actual-forged && test_must_be_empty actual-forged From patchwork Tue Jun 15 13:29:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8362AC48BDF for ; Tue, 15 Jun 2021 13:29:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6D3CF61468 for ; Tue, 15 Jun 2021 13:29:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230351AbhFONbr (ORCPT ); Tue, 15 Jun 2021 09:31:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231519AbhFONbS (ORCPT ); Tue, 15 Jun 2021 09:31:18 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E9EBC06124C for ; Tue, 15 Jun 2021 06:29:13 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id t11-20020a1cc30b0000b02901cec841b6a0so908250wmf.0 for ; Tue, 15 Jun 2021 06:29:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=8t2HQy4s1AMM54eo+tWceP4/zrvrBW6bxYmPB8mQ9/A=; b=kULpzdfTbkkci3tdD0yvQ0yVRtUazmq7eBBE0r7m/sNq9LrZwJHRgygMsl3NAjFBx2 cPoA5yF4I2aUFiUzpJ1owfoEEF3q/hCBNv70bR8TwUWI+l+x6MXVYCsTGuooZhJIFXlJ 8busLLQgowKREfTT5ye+cOB99bALorftmvxkEBeRfFpliwsYWRPzuVOsCxAnSZvdYmqW z7aeLcGZ+AXnd62ilyRZFBTLN2hcB/qKbp9NIzcgo0hwCVPYGA+hrQYgbV/L8vGQKJnz 9ocYVhWDdf+qElLhOxcY9JMF/6wkzHjKZSFdgh2GQE1iUSckt9m3Ugj5Q/V9ypgpF5An jByw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=8t2HQy4s1AMM54eo+tWceP4/zrvrBW6bxYmPB8mQ9/A=; b=fnj7P9D0G+KM5L/A/lC4TKgoglq3fZC+EnJXoA8FP7oHmg4fcKWKqhFzFWyX+ciODV Spnz38/HB1csR7LsGMMCKzq2GrJefQLWI3G5zzSAtu4WJ7IlZ1Eflq9320vftIWw32CH c+7rI7ldEFVU2quBGOlR2j6EtyQKhK8iC5sAIwXnvOvNLEa49j51FWtBsOCaOmaFs7HR 73XAJ+sb1LqSpQ0l2SrwVj6M28C+v9ufIp0GXQh59BlLwlWhuelHhMD8DI26FKMuBE+y pN1edBIdrHp+4CKXIaM9JiNaFrh5sSUxEw4/71217il/hhAG0NYnqSLvDCrLYh2T4Rv1 HiMQ== X-Gm-Message-State: AOAM531ZjKtnNlAK13rz7SxxZykvbSYt/9m2ztcy8mWEq1oDCr0sygK2 Egtyfxw4FRXQZ4q7fBFbmJPpleqny0w= X-Google-Smtp-Source: ABdhPJziwmegzTFWRNBvG3YNJ7ZUamF1oxEn8GFZleqoehMSK4q14NvgEeqmIpkYZwtuBXjBLdukFQ== X-Received: by 2002:a7b:c097:: with SMTP id r23mr22933744wmh.30.1623763751737; Tue, 15 Jun 2021 06:29:11 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t1sm18491462wrx.28.2021.06.15.06.29.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:11 -0700 (PDT) Message-Id: <49063372e0035c5384f834d78854da56f5726d13.1623763747.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Jun 2021 13:29:01 +0000 Subject: [PATCH v2 5/9] [GSOC] ref-filter: teach get_object() return useful value Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Let `populate_value()`, `get_ref_atom_value()` and `format_ref_array_item()` get the return value of `get_object()` correctly. This can help us later let `cat-file --batch` get the correct error message and return value of `get_object()`. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 8868cf98f090..420c0bf9384f 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1808,7 +1808,7 @@ static char *get_worktree_path(const struct used_atom *atom, const struct ref_ar static int populate_value(struct ref_array_item *ref, struct strbuf *err) { struct object *obj; - int i; + int i, ret = 0; struct object_info empty = OBJECT_INFO_INIT; CALLOC_ARRAY(ref->value, used_atom_cnt); @@ -1965,8 +1965,8 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) oi.oid = ref->objectname; - if (get_object(ref, 0, &obj, &oi, err)) - return -1; + if ((ret = get_object(ref, 0, &obj, &oi, err))) + return ret; /* * If there is no atom that wants to know about tagged @@ -1997,9 +1997,11 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) static int get_ref_atom_value(struct ref_array_item *ref, int atom, struct atom_value **v, struct strbuf *err) { + int ret = 0; + if (!ref->value) { - if (populate_value(ref, err)) - return -1; + if ((ret = populate_value(ref, err))) + return ret; fill_missing_values(ref->value); } *v = &ref->value[atom]; @@ -2573,6 +2575,7 @@ int format_ref_array_item(struct ref_array_item *info, { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; + int ret = 0; state.quote_style = format->quote_style; push_stack_element(&state.stack); @@ -2585,10 +2588,10 @@ int format_ref_array_item(struct ref_array_item *info, if (cp < sp) append_literal(cp, sp, &state); pos = parse_ref_filter_atom(format, sp + 2, ep, error_buf); - if (pos < 0 || get_ref_atom_value(info, pos, &atomv, error_buf) || + if (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) || atomv->handler(atomv, &state, error_buf)) { pop_stack_element(&state.stack); - return -1; + return ret ? ret : -1; } } if (*cp) { @@ -2610,7 +2613,7 @@ int format_ref_array_item(struct ref_array_item *info, } strbuf_addbuf(final_buf, &state.stack->output); pop_stack_element(&state.stack); - return 0; + return ret; } void pretty_print_ref(const char *name, const struct object_id *oid, From patchwork Tue Jun 15 13:29:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 258FAC48BE5 for ; Tue, 15 Jun 2021 13:29:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0F23661474 for ; Tue, 15 Jun 2021 13:29:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231184AbhFONbp (ORCPT ); Tue, 15 Jun 2021 09:31:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231518AbhFONbS (ORCPT ); Tue, 15 Jun 2021 09:31:18 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EE12C06121D for ; Tue, 15 Jun 2021 06:29:13 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id m41-20020a05600c3b29b02901b9e5d74f02so2027534wms.3 for ; Tue, 15 Jun 2021 06:29:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=1RyoUSZFJxz7y6PjMiMZAxUO69o5Pe1rFl/cPKTlma8=; b=au1lLZBIMlhBomodZhu3UWYBkJXffiiC9Bkm0l015AFTTpGwhZrCsy75lA60sNAFNS 9cloG63wRPSRTSBEeg24JrI1Z8fdXU/ZAvohNtBy4qZQ0f0KjOZHEjPMDfaOMGyI2Agw K3tomg8TmolshaBxuPIGhXdeUdqvN9ogxKCNRRG3YRzbt1kFQKTtbdAK9DRLtJZ3EPAu sllai4yl/g3zRYBy3WUPh6ar35cneoRLyn0IxK0tll1Lk8zM/rEojQxqqs4nyo6dN7Rj M05/l83LqMZLnJ8Sq5zB5bUWf1IhLc29mMszLwV1QhcSeWN6312+dvnmBEHDCYvIcB72 mXLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=1RyoUSZFJxz7y6PjMiMZAxUO69o5Pe1rFl/cPKTlma8=; b=lngrH4VoCHbc97aCgF4ogyMKSXof1xXTfjP5D7Ms2N+cVUytcWpu53Y8o0hY3URiv4 LHNOBSwzuoqGThzstZlhIX9Dx/1hIIyX/Akkn+TlrLsc6C67Wr9Tl0MjLBpi/hfeamU3 JDxQFnI2rqdXpfmxWc/mXmw2yZFh484wPOZUhx5Do45p/g/VhfqqNukxOksbq2KGa14V N7spwjKuI5nHX6jo/QHK25AHXmeVDl+gGiGJ83viMATU1Il03ebaTr8wjMRd4gz5kGmp 1hyK9Ic+1lHrvPvpD25pJHcujzrhKIaHWH62IaG3ItZcpSm7LXMqO11PBF4mT0ukeER6 NfyQ== X-Gm-Message-State: AOAM531z1cgNEwrmSIr85RKqhdKTZdk6yvSqD80npVlTzbn+TXE01WL+ +/G3bWqm+yI96cZ1rMuWRBezRt+SJdk= X-Google-Smtp-Source: ABdhPJydyE5/Lp3t9vSA+0SSLSsWmkZhUJMyhcFfk+7YhlWkosQU8sXXsg4kNMxWZIT5Mdqs6vkEzg== X-Received: by 2002:a05:600c:4fd0:: with SMTP id o16mr5366741wmq.50.1623763752334; Tue, 15 Jun 2021 06:29:12 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n1sm15771152wms.18.2021.06.15.06.29.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:12 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 15 Jun 2021 13:29:02 +0000 Subject: [PATCH v2 6/9] [GSOC] ref-filter: introduce free_array_item_internal() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Introduce free_array_item_internal() for freeing ref_array_item value. It will be called internally by free_array_item(), and it will help `cat-file --batch` free ref_array_item's memory later. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 ++++++++--- ref-filter.h | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 420c0bf9384f..aa2ce634106b 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2282,16 +2282,21 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, return 0; } -/* Free memory allocated for a ref_array_item */ -static void free_array_item(struct ref_array_item *item) +void free_array_item_internal(struct ref_array_item *item) { - free((char *)item->symref); if (item->value) { int i; for (i = 0; i < used_atom_cnt; i++) free((char *)item->value[i].s); free(item->value); } +} + +/* Free memory allocated for a ref_array_item */ +static void free_array_item(struct ref_array_item *item) +{ + free((char *)item->symref); + free_array_item_internal(item); free(item); } diff --git a/ref-filter.h b/ref-filter.h index 9dc07476a584..d4531fef5f91 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -111,6 +111,8 @@ struct ref_format { int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type); /* Clear all memory allocated to ref_array */ void ref_array_clear(struct ref_array *array); +/* Free array item's value */ +void free_array_item_internal(struct ref_array_item *item); /* Used to verify if the given format is correct and to parse out the used atoms */ int verify_ref_format(struct ref_format *format); /* Sort the given ref_array as per the ref_sorting provided */ From patchwork Tue Jun 15 13:29:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F9F1C48BE5 for ; Tue, 15 Jun 2021 13:29:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6806961468 for ; Tue, 15 Jun 2021 13:29:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231650AbhFONbt (ORCPT ); Tue, 15 Jun 2021 09:31:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231643AbhFONbT (ORCPT ); Tue, 15 Jun 2021 09:31:19 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 800BEC061574 for ; Tue, 15 Jun 2021 06:29:14 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id 3-20020a05600c0243b029019f2f9b2b8aso2029058wmj.2 for ; Tue, 15 Jun 2021 06:29:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=LOl2wWtuFq4OR7A88dXTcqW76V2R/56GLDIcxrEYb+U=; b=KyHjTFh4n8Z3mWelmDGMolWUr/qSTA1haAMypYPVFErmhb2aK7eRs1VL1jF7fx+Ntk gD3uvxOPAvDKaHVtiYpEn98kumo0GbGTelfiNCj9o772ekkH5wA0RU1lVmtrxTVlRPsH xpEEO/GGk3e+4yRpGtzjYAT1fa39Zk443CPcr89MWuw2V4IgS0b3v0ptLz13xwPf0iLg mamrC3X1TZbzz15+00mDhYQqC7t0IWbAAlGEFECnHZr7a+tvyAMrBNbjXKW5djANUn4C WV8P0o4PRXrpAC0TcWZHc1Q50fnVjUUQlBOiLCmis2ed+cQg4Q4nOziyc6gjIi7GVuPc Wadw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=LOl2wWtuFq4OR7A88dXTcqW76V2R/56GLDIcxrEYb+U=; b=dKMDVyXagFe4PQutfKMvXg0ADO9IVuRL+pgW4LjiQa3kx6l2blSPPt2wFroFMp5sxU /QY5VSbEGdQKZlnvhkhdVp94vtuF+hw8gN7as22AyF0IEjKBcekT7Ww+/vCdKRPHKJs5 JWMvgedUNscB9Z5he+K59v6xzN5/hkJ0ThPxVtTuYaLED6DxN92TmujqlYESKy/fgRwm 0HOwLTJQydxni0PGiHM+KcHQ2Oyta4n2B/QtY770uD4MosvFAIQEZZ3KeU6k0LwyP+3W UglGTwiGsEaZlbr6ezBDsyPObS3RikSihl02fuCM42X4gqt1jEadEHEaxerKUbHI/3UM i0HA== X-Gm-Message-State: AOAM532tQDkQiBLtfA7VBAqqZ/i0NRclvNhzSLf9sAIxFSiJVzJoTDrg xADk2Hasu52KLu/t7bCR1fLaLmaDBaY= X-Google-Smtp-Source: ABdhPJyAYTwLOS8L8qBzk4kypuNzMte0DWmZOu8rCYcVGT4J5T6sieZWF4GaMEwgHkNTt3YHuyghrA== X-Received: by 2002:a7b:c750:: with SMTP id w16mr22813770wmk.69.1623763753009; Tue, 15 Jun 2021 06:29:13 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u18sm15527413wmj.15.2021.06.15.06.29.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:12 -0700 (PDT) Message-Id: <765337a46ab0f6c8ad38007a78c850f5dd4cfa38.1623763747.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Jun 2021 13:29:03 +0000 Subject: [PATCH v2 7/9] [GSOC] cat-file: reuse ref-filter logic Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let cat-file use ref-filter logic, the following methods are used: 1. Add `cat_file_mode` member in struct `ref_format`, this can help us reject atoms in verify_ref_format() which cat-file cannot use, e.g. `%(refname)`, `%(push)`, `%(upstream)`... 2. Change the type of member `format` in struct `batch_options` to `ref_format`, We can add format data in it. 3. Let `batch_objects()` add atoms to format, and use `verify_ref_format()` to check atoms. 4. Use `has_object_file()` in `batch_one_object()` to check whether the input object exists. 5. Use `format_ref_array_item()` in `batch_object_write()` to get the formatted data corresponding to the object. If the return value of `format_ref_array_item()` is equals to zero, use `batch_write()` to print object data; else if the return value less than zero, use `die()` to print the error message and exit; else return value greater than zero, only print the error message, but not exit. 6. Let get_object() return 1 and print " missing" instead of returning -1 and printing "missing object for ", this can help `format_ref_array_item()` just report that the object is missing without letting Git exit. Most of the atoms in `for-each-ref --format` are now supported, such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, `%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: `%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, `%(flag)`, `%(HEAD)`, because our objects don't have refname. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- Documentation/git-cat-file.txt | 6 + builtin/cat-file.c | 248 +++++++------------------------- ref-filter.c | 15 +- ref-filter.h | 3 +- t/t1006-cat-file.sh | 252 +++++++++++++++++++++++++++++++++ t/t6301-for-each-ref-errors.sh | 2 +- 6 files changed, 323 insertions(+), 203 deletions(-) diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index 4eb0421b3fd9..ef8ab952b2fa 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -226,6 +226,12 @@ newline. The available atoms are: after that first run of whitespace (i.e., the "rest" of the line) are output in place of the `%(rest)` atom. +Note that most of the atoms in `for-each-ref --format` are now supported, +such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, +`%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: +`%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, +`%(flag)`, `%(HEAD)`. See linkgit:git-for-each-ref[1]. + If no format is specified, the default format is `%(objectname) %(objecttype) %(objectsize)`. diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 5ebf13359e83..026d9405636a 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -16,6 +16,7 @@ #include "packfile.h" #include "object-store.h" #include "promisor-remote.h" +#include "ref-filter.h" struct batch_options { int enabled; @@ -25,7 +26,7 @@ struct batch_options { int all_objects; int unordered; int cmdmode; /* may be 'w' or 'c' for --filters or --textconv */ - const char *format; + struct ref_format format; }; static const char *force_path; @@ -195,99 +196,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct expand_data { struct object_id oid; - enum object_type type; - unsigned long size; - off_t disk_size; const char *rest; - struct object_id delta_base_oid; - - /* - * If mark_query is true, we do not expand anything, but rather - * just mark the object_info with items we wish to query. - */ - int mark_query; - - /* - * Whether to split the input on whitespace before feeding it to - * get_sha1; this is decided during the mark_query phase based on - * whether we have a %(rest) token in our format. - */ int split_on_whitespace; - - /* - * After a mark_query run, this object_info is set up to be - * passed to oid_object_info_extended. It will point to the data - * elements above, so you can retrieve the response from there. - */ - struct object_info info; - - /* - * This flag will be true if the requested batch format and options - * don't require us to call oid_object_info, which can then be - * optimized out. - */ - unsigned skip_object_info : 1; }; -static int is_atom(const char *atom, const char *s, int slen) -{ - int alen = strlen(atom); - return alen == slen && !memcmp(atom, s, alen); -} - -static void expand_atom(struct strbuf *sb, const char *atom, int len, - void *vdata) -{ - struct expand_data *data = vdata; - - if (is_atom("objectname", atom, len)) { - if (!data->mark_query) - strbuf_addstr(sb, oid_to_hex(&data->oid)); - } else if (is_atom("objecttype", atom, len)) { - if (data->mark_query) - data->info.typep = &data->type; - else - strbuf_addstr(sb, type_name(data->type)); - } else if (is_atom("objectsize", atom, len)) { - if (data->mark_query) - data->info.sizep = &data->size; - else - strbuf_addf(sb, "%"PRIuMAX , (uintmax_t)data->size); - } else if (is_atom("objectsize:disk", atom, len)) { - if (data->mark_query) - data->info.disk_sizep = &data->disk_size; - else - strbuf_addf(sb, "%"PRIuMAX, (uintmax_t)data->disk_size); - } else if (is_atom("rest", atom, len)) { - if (data->mark_query) - data->split_on_whitespace = 1; - else if (data->rest) - strbuf_addstr(sb, data->rest); - } else if (is_atom("deltabase", atom, len)) { - if (data->mark_query) - data->info.delta_base_oid = &data->delta_base_oid; - else - strbuf_addstr(sb, - oid_to_hex(&data->delta_base_oid)); - } else - die("unknown format element: %.*s", len, atom); -} - -static size_t expand_format(struct strbuf *sb, const char *start, void *data) -{ - const char *end; - - if (*start != '(') - return 0; - end = strchr(start + 1, ')'); - if (!end) - die("format element '%s' does not end in ')'", start); - - expand_atom(sb, start + 1, end - start - 1, data); - - return end - start + 1; -} - static void batch_write(struct batch_options *opt, const void *data, int len) { if (opt->buffer_output) { @@ -297,87 +209,31 @@ static void batch_write(struct batch_options *opt, const void *data, int len) write_or_die(1, data, len); } -static void print_object_or_die(struct batch_options *opt, struct expand_data *data) -{ - const struct object_id *oid = &data->oid; - - assert(data->info.typep); - - if (data->type == OBJ_BLOB) { - if (opt->buffer_output) - fflush(stdout); - if (opt->cmdmode) { - char *contents; - unsigned long size; - - if (!data->rest) - die("missing path for '%s'", oid_to_hex(oid)); - - if (opt->cmdmode == 'w') { - if (filter_object(data->rest, 0100644, oid, - &contents, &size)) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else if (opt->cmdmode == 'c') { - enum object_type type; - if (!textconv_object(the_repository, - data->rest, 0100644, oid, - 1, &contents, &size)) - contents = read_object_file(oid, - &type, - &size); - if (!contents) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else - BUG("invalid cmdmode: %c", opt->cmdmode); - batch_write(opt, contents, size); - free(contents); - } else { - stream_blob(oid); - } - } - else { - enum object_type type; - unsigned long size; - void *contents; - - contents = read_object_file(oid, &type, &size); - if (!contents) - die("object %s disappeared", oid_to_hex(oid)); - if (type != data->type) - die("object %s changed type!?", oid_to_hex(oid)); - if (data->info.sizep && size != data->size) - die("object %s changed size!?", oid_to_hex(oid)); - - batch_write(opt, contents, size); - free(contents); - } -} static void batch_object_write(const char *obj_name, struct strbuf *scratch, struct batch_options *opt, struct expand_data *data) { - if (!data->skip_object_info && - oid_object_info_extended(the_repository, &data->oid, &data->info, - OBJECT_INFO_LOOKUP_REPLACE) < 0) { - printf("%s missing\n", - obj_name ? obj_name : oid_to_hex(&data->oid)); - fflush(stdout); - return; - } + int ret = 0; + struct strbuf err = STRBUF_INIT; + struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); - strbuf_expand(scratch, opt->format, expand_format, data); - strbuf_addch(scratch, '\n'); - batch_write(opt, scratch->buf, scratch->len); - if (opt->print_contents) { - print_object_or_die(opt, data); - batch_write(opt, "\n", 1); + ret = format_ref_array_item(&item, &opt->format, scratch, &err); + if (!ret) { + strbuf_addch(scratch, '\n'); + batch_write(opt, scratch->buf, scratch->len); + } else if (ret < 0) { + die("%s\n", err.buf); + } else { + /* when ret > 0 , don't call die and print the err to stdout*/ + printf("%s\n", err.buf); + fflush(stdout); } + free_array_item_internal(&item); + strbuf_release(&err); } static void batch_one_object(const char *obj_name, @@ -428,6 +284,13 @@ static void batch_one_object(const char *obj_name, return; } + if (!has_object_file(&data->oid)) { + printf("%s missing\n", + obj_name ? obj_name : oid_to_hex(&data->oid)); + fflush(stdout); + return; + } + batch_object_write(obj_name, scratch, opt, data); } @@ -488,42 +351,34 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *opt) +static const char * const cat_file_usage[] = { + N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), + N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), + NULL +}; + +static int batch_objects(struct batch_options *opt, const struct option *options) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; int retval = 0; - if (!opt->format) - opt->format = "%(objectname) %(objecttype) %(objectsize)"; - - /* - * Expand once with our special mark_query flag, which will prime the - * object_info to be handed to oid_object_info_extended for each - * object. - */ memset(&data, 0, sizeof(data)); - data.mark_query = 1; - strbuf_expand(&output, opt->format, expand_format, &data); - data.mark_query = 0; - strbuf_release(&output); - if (opt->cmdmode) - data.split_on_whitespace = 1; - - if (opt->all_objects) { - struct object_info empty = OBJECT_INFO_INIT; - if (!memcmp(&data.info, &empty, sizeof(empty))) - data.skip_object_info = 1; - } - - /* - * If we are printing out the object, then always fill in the type, - * since we will want to decide whether or not to stream. - */ + if (!opt->format.format) + strbuf_addstr(&format, "%(objectname) %(objecttype) %(objectsize)"); + else + strbuf_addstr(&format, opt->format.format); if (opt->print_contents) - data.info.typep = &data.type; + strbuf_addstr(&format, "\n%(raw)"); + opt->format.format = format.buf; + if (verify_ref_format(&opt->format)) + usage_with_options(cat_file_usage, options); + + if (opt->cmdmode || opt->format.use_rest) + data.split_on_whitespace = 1; if (opt->all_objects) { struct object_cb_data cb; @@ -556,6 +411,7 @@ static int batch_objects(struct batch_options *opt) oid_array_clear(&sa); } + strbuf_release(&format); strbuf_release(&output); return 0; } @@ -588,18 +444,13 @@ static int batch_objects(struct batch_options *opt) batch_one_object(input.buf, &output, opt, &data); } + strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); warn_on_object_refname_ambiguity = save_warning; return retval; } -static const char * const cat_file_usage[] = { - N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), - N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), - NULL -}; - static int git_cat_file_config(const char *var, const char *value, void *cb) { if (userdiff_config(var, value) < 0) @@ -622,7 +473,7 @@ static int batch_option_callback(const struct option *opt, bo->enabled = 1; bo->print_contents = !strcmp(opt->long_name, "batch"); - bo->format = arg; + bo->format.format = arg; return 0; } @@ -631,7 +482,9 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) { int opt = 0; const char *exp_type = NULL, *obj_name = NULL; - struct batch_options batch = {0}; + struct batch_options batch = { + .format = REF_FORMAT_INIT + }; int unknown_type = 0; const struct option options[] = { @@ -670,6 +523,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) git_config(git_cat_file_config, NULL); batch.buffer_output = -1; + batch.format.cat_file_mode = 1; argc = parse_options(argc, argv, prefix, options, cat_file_usage, 0); if (opt) { @@ -713,7 +567,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) batch.buffer_output = batch.all_objects; if (batch.enabled) - return batch_objects(&batch); + return batch_objects(&batch, options); if (unknown_type && opt != 't' && opt != 's') die("git cat-file --allow-unknown-type: use with -s or -t"); diff --git a/ref-filter.c b/ref-filter.c index aa2ce634106b..660e52f97f79 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1017,8 +1017,15 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - if (used_atom[at].atom_type == ATOM_REST) - die("this command reject atom %%(%.*s)", (int)(ep - sp - 2), sp + 2); + if ((!format->cat_file_mode && used_atom[at].atom_type == ATOM_REST) || + (format->cat_file_mode && (used_atom[at].atom_type == ATOM_FLAG || + used_atom[at].atom_type == ATOM_HEAD || + used_atom[at].atom_type == ATOM_PUSH || + used_atom[at].atom_type == ATOM_REFNAME || + used_atom[at].atom_type == ATOM_SYMREF || + used_atom[at].atom_type == ATOM_UPSTREAM || + used_atom[at].atom_type == ATOM_WORKTREEPATH))) + die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && used_atom[at].u.raw_data.option == RAW_BARE) @@ -1735,8 +1742,8 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) - return strbuf_addf_ret(err, -1, _("missing object %s for %s"), - oid_to_hex(&oi->oid), ref->refname); + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); if (oi->info.disk_sizep && oi->disk_size < 0) BUG("Object size is less than zero."); diff --git a/ref-filter.h b/ref-filter.h index d4531fef5f91..3db07729f46d 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -78,6 +78,7 @@ struct ref_format { */ const char *format; const char *rest; + int cat_file_mode; int quote_style; int use_rest; int use_color; @@ -86,7 +87,7 @@ struct ref_format { int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { NULL, NULL, 0, 0, -1 } +#define REF_FORMAT_INIT { NULL, NULL, 0, 0, 0, -1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 5d2dc99b74ad..5efa7397cfbc 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -586,4 +586,256 @@ test_expect_success 'cat-file --unordered works' ' test_cmp expect actual ' +. "$TEST_DIRECTORY"/lib-gpg.sh +. "$TEST_DIRECTORY"/lib-terminal.sh + +test_expect_success 'cat-file --batch|--batch-check setup' ' + echo 1>blob1 && + printf "a\0b\0\c" >blob2 && + git add blob1 blob2 && + git commit -m "Commit Message" && + git branch -M main && + git tag -a -m "v0.0.0" testtag && + git update-ref refs/myblobs/blob1 HEAD:blob1 && + git update-ref refs/myblobs/blob2 HEAD:blob2 && + git update-ref refs/mytrees/tree1 HEAD^{tree} +' + +batch_test_atom() { + if test "$3" = "fail" + then + test_expect_${4:-success} $PREREQ "basic atom: $1 $2 mast failed" " + test_must_fail git cat-file --batch-check='$2' >bad <<-EOF + $1 + EOF + " + else + test_expect_${4:-success} $PREREQ "basic atom: $1 $2" " + git for-each-ref --format='$2' $1 >expected && + git cat-file --batch-check='$2' >actual <<-EOF && + $1 + EOF + sanitize_pgp actual.clean && + cmp expected actual.clean + " + fi +} + +batch_test_atom refs/heads/main '%(refname)' fail +batch_test_atom refs/heads/main '%(refname:)' fail +batch_test_atom refs/heads/main '%(refname:short)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:strip=1)' fail +batch_test_atom refs/heads/main '%(refname:strip=2)' fail +batch_test_atom refs/heads/main '%(refname:strip=-1)' fail +batch_test_atom refs/heads/main '%(refname:strip=-2)' fail +batch_test_atom refs/heads/main '%(upstream)' fail +batch_test_atom refs/heads/main '%(upstream:short)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=-2)' fail +batch_test_atom refs/heads/main '%(push)' fail +batch_test_atom refs/heads/main '%(push:short)' fail +batch_test_atom refs/heads/main '%(push:lstrip=1)' fail +batch_test_atom refs/heads/main '%(push:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:strip=1)' fail +batch_test_atom refs/heads/main '%(push:strip=-1)' fail +batch_test_atom refs/heads/main '%(objecttype)' +batch_test_atom refs/heads/main '%(objectsize)' +batch_test_atom refs/heads/main '%(objectsize:disk)' +batch_test_atom refs/heads/main '%(deltabase)' +batch_test_atom refs/heads/main '%(objectname)' +batch_test_atom refs/heads/main '%(objectname:short)' +batch_test_atom refs/heads/main '%(objectname:short=1)' +batch_test_atom refs/heads/main '%(objectname:short=10)' +batch_test_atom refs/heads/main '%(tree)' +batch_test_atom refs/heads/main '%(tree:short)' +batch_test_atom refs/heads/main '%(tree:short=1)' +batch_test_atom refs/heads/main '%(tree:short=10)' +batch_test_atom refs/heads/main '%(parent)' +batch_test_atom refs/heads/main '%(parent:short)' +batch_test_atom refs/heads/main '%(parent:short=1)' +batch_test_atom refs/heads/main '%(parent:short=10)' +batch_test_atom refs/heads/main '%(numparent)' +batch_test_atom refs/heads/main '%(object)' +batch_test_atom refs/heads/main '%(type)' +batch_test_atom refs/heads/main '%(raw)' +batch_test_atom refs/heads/main '%(*objectname)' +batch_test_atom refs/heads/main '%(*objecttype)' +batch_test_atom refs/heads/main '%(author)' +batch_test_atom refs/heads/main '%(authorname)' +batch_test_atom refs/heads/main '%(authoremail)' +batch_test_atom refs/heads/main '%(authoremail:trim)' +batch_test_atom refs/heads/main '%(authoremail:localpart)' +batch_test_atom refs/heads/main '%(authordate)' +batch_test_atom refs/heads/main '%(committer)' +batch_test_atom refs/heads/main '%(committername)' +batch_test_atom refs/heads/main '%(committeremail)' +batch_test_atom refs/heads/main '%(committeremail:trim)' +batch_test_atom refs/heads/main '%(committeremail:localpart)' +batch_test_atom refs/heads/main '%(committerdate)' +batch_test_atom refs/heads/main '%(tag)' +batch_test_atom refs/heads/main '%(tagger)' +batch_test_atom refs/heads/main '%(taggername)' +batch_test_atom refs/heads/main '%(taggeremail)' +batch_test_atom refs/heads/main '%(taggeremail:trim)' +batch_test_atom refs/heads/main '%(taggeremail:localpart)' +batch_test_atom refs/heads/main '%(taggerdate)' +batch_test_atom refs/heads/main '%(creator)' +batch_test_atom refs/heads/main '%(creatordate)' +batch_test_atom refs/heads/main '%(subject)' +batch_test_atom refs/heads/main '%(subject:sanitize)' +batch_test_atom refs/heads/main '%(contents:subject)' +batch_test_atom refs/heads/main '%(body)' +batch_test_atom refs/heads/main '%(contents:body)' +batch_test_atom refs/heads/main '%(contents:signature)' +batch_test_atom refs/heads/main '%(contents)' +batch_test_atom refs/heads/main '%(HEAD)' fail +batch_test_atom refs/heads/main '%(upstream:track)' fail +batch_test_atom refs/heads/main '%(upstream:trackshort)' fail +batch_test_atom refs/heads/main '%(upstream:track,nobracket)' fail +batch_test_atom refs/heads/main '%(upstream:nobracket,track)' fail +batch_test_atom refs/heads/main '%(push:track)' fail +batch_test_atom refs/heads/main '%(push:trackshort)' fail +batch_test_atom refs/heads/main '%(worktreepath)' fail +batch_test_atom refs/heads/main '%(symref)' fail +batch_test_atom refs/heads/main '%(flag)' fail + +batch_test_atom refs/tags/testtag '%(refname)' fail +batch_test_atom refs/tags/testtag '%(refname:short)' fail +batch_test_atom refs/tags/testtag '%(upstream)' fail +batch_test_atom refs/tags/testtag '%(push)' fail +batch_test_atom refs/tags/testtag '%(objecttype)' +batch_test_atom refs/tags/testtag '%(objectsize)' +batch_test_atom refs/tags/testtag '%(objectsize:disk)' +batch_test_atom refs/tags/testtag '%(*objectsize:disk)' +batch_test_atom refs/tags/testtag '%(deltabase)' +batch_test_atom refs/tags/testtag '%(*deltabase)' +batch_test_atom refs/tags/testtag '%(objectname)' +batch_test_atom refs/tags/testtag '%(objectname:short)' +batch_test_atom refs/tags/testtag '%(tree)' +batch_test_atom refs/tags/testtag '%(tree:short)' +batch_test_atom refs/tags/testtag '%(tree:short=1)' +batch_test_atom refs/tags/testtag '%(tree:short=10)' +batch_test_atom refs/tags/testtag '%(parent)' +batch_test_atom refs/tags/testtag '%(parent:short)' +batch_test_atom refs/tags/testtag '%(parent:short=1)' +batch_test_atom refs/tags/testtag '%(parent:short=10)' +batch_test_atom refs/tags/testtag '%(numparent)' +batch_test_atom refs/tags/testtag '%(object)' +batch_test_atom refs/tags/testtag '%(type)' +batch_test_atom refs/tags/testtag '%(*objectname)' +batch_test_atom refs/tags/testtag '%(*objecttype)' +batch_test_atom refs/tags/testtag '%(author)' +batch_test_atom refs/tags/testtag '%(authorname)' +batch_test_atom refs/tags/testtag '%(authoremail)' +batch_test_atom refs/tags/testtag '%(authoremail:trim)' +batch_test_atom refs/tags/testtag '%(authoremail:localpart)' +batch_test_atom refs/tags/testtag '%(authordate)' +batch_test_atom refs/tags/testtag '%(committer)' +batch_test_atom refs/tags/testtag '%(committername)' +batch_test_atom refs/tags/testtag '%(committeremail)' +batch_test_atom refs/tags/testtag '%(committeremail:trim)' +batch_test_atom refs/tags/testtag '%(committeremail:localpart)' +batch_test_atom refs/tags/testtag '%(committerdate)' +batch_test_atom refs/tags/testtag '%(tag)' +batch_test_atom refs/tags/testtag '%(tagger)' +batch_test_atom refs/tags/testtag '%(taggername)' +batch_test_atom refs/tags/testtag '%(taggeremail)' +batch_test_atom refs/tags/testtag '%(taggeremail:trim)' +batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' +batch_test_atom refs/tags/testtag '%(taggerdate)' +batch_test_atom refs/tags/testtag '%(creator)' +batch_test_atom refs/tags/testtag '%(creatordate)' +batch_test_atom refs/tags/testtag '%(subject)' +batch_test_atom refs/tags/testtag '%(subject:sanitize)' +batch_test_atom refs/tags/testtag '%(contents:subject)' +batch_test_atom refs/tags/testtag '%(body)' +batch_test_atom refs/tags/testtag '%(contents:body)' +batch_test_atom refs/tags/testtag '%(contents:signature)' +batch_test_atom refs/tags/testtag '%(contents)' +batch_test_atom refs/tags/testtag '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(refname)' fail +batch_test_atom refs/myblobs/blob1 '%(upstream)' fail +batch_test_atom refs/myblobs/blob1 '%(push)' fail +batch_test_atom refs/myblobs/blob1 '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(objectname)' +batch_test_atom refs/myblobs/blob1 '%(objecttype)' +batch_test_atom refs/myblobs/blob1 '%(objectsize)' +batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)' +batch_test_atom refs/myblobs/blob1 '%(deltabase)' + +batch_test_atom refs/myblobs/blob1 '%(contents)' +batch_test_atom refs/myblobs/blob2 '%(contents)' + +batch_test_atom refs/myblobs/blob1 '%(raw)' +batch_test_atom refs/myblobs/blob2 '%(raw)' +batch_test_atom refs/mytrees/tree1 '%(raw)' + +batch_test_atom refs/myblobs/blob1 '%(raw:size)' +batch_test_atom refs/myblobs/blob2 '%(raw:size)' +batch_test_atom refs/mytrees/tree1 '%(raw:size)' + +batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' + +batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' + +batch_test_atom refs/heads/main 'VALID' +batch_test_atom refs/heads/main '%(INVALID)' fail +batch_test_atom refs/heads/main '%(authordate:INVALID)' fail + +test_expect_success 'cat-file refs/heads/main refs/tags/testtag %(rest)' ' + cat >expected <<-EOF && + 123 commit 123 + 456 tag 456 + EOF + git cat-file --batch-check="%(rest) %(objecttype) %(rest)" >actual <<-EOF && + refs/heads/main 123 + refs/tags/testtag 456 + EOF + test_cmp expected actual +' + +batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize) +%(raw)' + + +test_expect_success 'cat-file --batch equals to --batch-check with atoms' ' + git cat-file --batch-check="%(objectname) %(objecttype) %(objectsize) +%(raw)" >expected <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + git cat-file --batch >actual <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + cmp expected actual +' + test_done diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh index 40edf9dab534..3553f84a00c1 100755 --- a/t/t6301-for-each-ref-errors.sh +++ b/t/t6301-for-each-ref-errors.sh @@ -41,7 +41,7 @@ test_expect_success 'Missing objects are reported correctly' ' r=refs/heads/missing && echo $MISSING >.git/$r && test_when_finished "rm -f .git/$r" && - echo "fatal: missing object $MISSING for $r" >missing-err && + echo "fatal: $MISSING missing" >missing-err && test_must_fail git for-each-ref 2>err && test_cmp missing-err err && ( From patchwork Tue Jun 15 13:29:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04A6FC48BDF for ; Tue, 15 Jun 2021 13:29:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DE53A61468 for ; Tue, 15 Jun 2021 13:29:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231694AbhFONbv (ORCPT ); Tue, 15 Jun 2021 09:31:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231645AbhFONbT (ORCPT ); Tue, 15 Jun 2021 09:31:19 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC776C061226 for ; Tue, 15 Jun 2021 06:29:14 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id f16-20020a05600c1550b02901b00c1be4abso1778083wmg.2 for ; Tue, 15 Jun 2021 06:29:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Q6K7fgnkRbVYQeT3TbU5ef9TdEvqMaSZB6p/nSr5cyM=; b=NxKkowyRhqB/53yIlQDf8hx37g1VtSxBVjEBNSQdjsBWGg873RFcOzB1WnbHvaLdnJ Bzdz5d+qCNoqci/Fwsj/8lQ4XdBOHYSIgDjZV55fnyTPLUh2r1em1qlbnsWbLQphZMnp uHDVNDtoJV2/sXTcoCBu1r0yvrRcoVFMcpJLeVHFRIbDxkMWnqrMdwk45YaTSsYrNIo5 JcrUttOgm4wcE/IycmgiJ175M7011m2K6tqceKUQz4Tbbu3EPyni7qYDDsJhBLdRF140 S+RnZiuQ/JhcU5Th6xXewSxpDKXtAyAzaxGZbZnWqiVIXSInqHwjoDnA97PN3xgHH8NQ o7Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Q6K7fgnkRbVYQeT3TbU5ef9TdEvqMaSZB6p/nSr5cyM=; b=oEHht42oLCJ7kogQpQz7OFx2c1vqbV5XQt50mHM/s65ezx8vE+dK4HsGZQHT2G8vwN gWXvDfIfX/wW9bpVG7Evfq4u4Zdj19Is0lKsDS8ocmEnyL1nASSkdrV9kkngLDxIJRLZ 7wgkQAz3L/5FNyg2gqDwbYODQ0m3L4HIyRqiKS4Q8sZ+4A4oYokpfjc8RhCGiwLq9OQA 7C0VHOwFiRyplyqpvUjHGYqZO9lNvkBJ6CmvpFbt1xYThcgV9Ifvx56OuZ06Cj3Ff12D l3bZi9BRBYX3JzKUwI5cb/QhfHQTLy5REAv06LcMaZhzf2CO3bz3+/8tS3qkO4TmG/le aDBw== X-Gm-Message-State: AOAM532vbc3yxckCmv46oF8kVIt9q3YybrmOJNYCaSr8iGGA+eFQJUDt hwx7ALPzKIZvpEpxUpkeEr5e8CDHWUw= X-Google-Smtp-Source: ABdhPJxFfQIwk/ymjEMJmNxXjU7ClhpeuQdNOQnwn2KBb2sVZdGw4kW+h+auwDgOQM4oIwAY1DJs8A== X-Received: by 2002:a05:600c:1ca6:: with SMTP id k38mr5466903wms.49.1623763753585; Tue, 15 Jun 2021 06:29:13 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u7sm22584481wrt.18.2021.06.15.06.29.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:13 -0700 (PDT) Message-Id: <058b304686fdd97f1cdd81f4d5eb22f2b08ce987.1623763747.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Jun 2021 13:29:04 +0000 Subject: [PATCH v2 8/9] [GSOC] cat-file: reuse err buf in batch_objet_write() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Reuse the `err` buffer in batch_object_write(), as the buffer `scratch` does. This will reduce the overhead of multiple allocations of memory of the err buffer. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 026d9405636a..6766efb06f41 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -212,32 +212,33 @@ static void batch_write(struct batch_options *opt, const void *data, int len) static void batch_object_write(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { int ret = 0; - struct strbuf err = STRBUF_INIT; struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); + strbuf_reset(err); - ret = format_ref_array_item(&item, &opt->format, scratch, &err); + ret = format_ref_array_item(&item, &opt->format, scratch, err); if (!ret) { strbuf_addch(scratch, '\n'); batch_write(opt, scratch->buf, scratch->len); } else if (ret < 0) { - die("%s\n", err.buf); + die("%s\n", err->buf); } else { /* when ret > 0 , don't call die and print the err to stdout*/ - printf("%s\n", err.buf); + printf("%s\n", err->buf); fflush(stdout); } free_array_item_internal(&item); - strbuf_release(&err); } static void batch_one_object(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { @@ -291,7 +292,7 @@ static void batch_one_object(const char *obj_name, return; } - batch_object_write(obj_name, scratch, opt, data); + batch_object_write(obj_name, scratch, err, opt, data); } struct object_cb_data { @@ -299,13 +300,14 @@ struct object_cb_data { struct expand_data *expand; struct oidset *seen; struct strbuf *scratch; + struct strbuf *err; }; static int batch_object_cb(const struct object_id *oid, void *vdata) { struct object_cb_data *data = vdata; oidcpy(&data->expand->oid, oid); - batch_object_write(NULL, data->scratch, data->opt, data->expand); + batch_object_write(NULL, data->scratch, data->err, data->opt, data->expand); return 0; } @@ -361,6 +363,7 @@ static int batch_objects(struct batch_options *opt, const struct option *options { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; @@ -389,6 +392,7 @@ static int batch_objects(struct batch_options *opt, const struct option *options cb.opt = opt; cb.expand = &data; cb.scratch = &output; + cb.err = &err; if (opt->unordered) { struct oidset seen = OIDSET_INIT; @@ -413,6 +417,7 @@ static int batch_objects(struct batch_options *opt, const struct option *options strbuf_release(&format); strbuf_release(&output); + strbuf_release(&err); return 0; } @@ -441,12 +446,13 @@ static int batch_objects(struct batch_options *opt, const struct option *options data.rest = p; } - batch_one_object(input.buf, &output, opt, &data); + batch_one_object(input.buf, &output, &err, opt, &data); } strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); + strbuf_release(&err); warn_on_object_refname_ambiguity = save_warning; return retval; } From patchwork Tue Jun 15 13:29:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12321395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AFC9C48BE8 for ; Tue, 15 Jun 2021 13:29:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E28E961465 for ; Tue, 15 Jun 2021 13:29:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231654AbhFONbz (ORCPT ); Tue, 15 Jun 2021 09:31:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231652AbhFONbY (ORCPT ); Tue, 15 Jun 2021 09:31:24 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87922C0611C0 for ; Tue, 15 Jun 2021 06:29:15 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id u5-20020a7bc0450000b02901480e40338bso1562886wmc.1 for ; Tue, 15 Jun 2021 06:29:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=UNjMfsSWVy7JXpdnafrhWW6deDtcFzBmaIHb+9KgKtI=; b=ErJmDek6PcvK+kk91Lt6/mJr0h0Q8vtUiicO/mR56+tjuSqlpobidYl3BmRTsIIKaH r8uhGr7U0NDkbHA4NJCMQL0IaIO1JvinZKxgl402MPsF8Buy5JigVjKdpUAJCFENYH0J +t/1yc68JF7oQyXdLrnlDNNb69gicBLhzLhKdd1jyIVitkzXffvVTIAyCIWDo1JDdc7P iGJXkNbL3IqZTDu6Knt+cv4XdWbHnnqJdaZuglSHoJox/9UsJmjwZndf4DtqypcUDBFR GGO/DE/oF9hcr+04uRe1UGv/0B6OMT7ISg0UrboSBvyQU4fI+iBHFkxHCaJ+++ROsp7A UpTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=UNjMfsSWVy7JXpdnafrhWW6deDtcFzBmaIHb+9KgKtI=; b=fIOIIa1B+XhsGNHQgVGZDeUO+RzQPvSc5Pa3Zr7+/VdB+SF4+f/EroD0q5T+81YAnq XwERVwdp+RQHpu9BdEbuotFsGMSsX6ngy3Q46O0JmE83IbxXFytrwt2lTfSVt2ZqDF9F oFFN+/6sDz21yAf6ottuE38EOvX+ES1IXd5CJcYCEfaNg6qwF1Us77Nauv9/SnausZz0 7LVMaELkvN55pstsq244wp8FCyy8/MH9NGml15zk0dOyW4lTm1TCYbS0eM6sWTWBHj/9 7wNVXp04aBDcQXnIblPlXeh0LfJCDQFo1ciiHfzjfhmfPpRteidfgNKn6g+qHAkITF64 43Mg== X-Gm-Message-State: AOAM530i3DaKPL0KEb9xHUZXtA7NBPOQUP+osqY5oqwuLTnKeBbcTBp7 j2Zl1lysQOY+Hm1fqjxJ6bQJyHZJoIs= X-Google-Smtp-Source: ABdhPJzXAXvhnm1W+dcS77XeZGqMu/X7TQe9hGIIXY11Nh71xgtfbrnFfNXPVKjTdCg8QsviHY1Dqw== X-Received: by 2002:a05:600c:2059:: with SMTP id p25mr22197115wmg.56.1623763754119; Tue, 15 Jun 2021 06:29:14 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o9sm18312279wri.68.2021.06.15.06.29.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 06:29:13 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 15 Jun 2021 13:29:05 +0000 Subject: [PATCH v2 9/9] [GSOC] cat-file: re-implement --textconv, --filters options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu After cat-file reuses the ref-filter logic, we re-implement the functions of --textconv and --filters options. Add members `use_textconv` and `use_filters` in struct `ref_format`, and use global variables `use_filters` and `use_textconv` in `ref-filter.c`, so that we can filter the content of the object in get_object(). Use `actual_oi` to record the real expand_data: it may point to the original `oi` or the `act_oi` processed by `textconv_object()` or `convert_to_working_tree()`. `grab_values()` will grab the contents of `actual_oi` and `grab_common_values()` to grab the contents of origin `oi`, this ensures that `%(objectsize)` still uses the size of the unfiltered data. In `get_object()`, we made an optimization: Firstly, get the size and type of the object instead of directly getting the object data. If using --textconv, after successfully obtaining the filtered object data, an extra oid_object_info_extended() will be skipped, which can reduce the cost of object data copy; If using --filter, the data of the object first will be getted first, and then convert_to_working_tree() will be used to get the filtered object data. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 5 ++++ ref-filter.c | 66 ++++++++++++++++++++++++++++++++++++++++++++-- ref-filter.h | 4 ++- 3 files changed, 72 insertions(+), 3 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 6766efb06f41..2978b8990e8d 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -377,6 +377,11 @@ static int batch_objects(struct batch_options *opt, const struct option *options if (opt->print_contents) strbuf_addstr(&format, "\n%(raw)"); opt->format.format = format.buf; + if (opt->cmdmode == 'c') + opt->format.use_textconv = 1; + if (opt->cmdmode == 'w') + opt->format.use_filters = 1; + if (verify_ref_format(&opt->format)) usage_with_options(cat_file_usage, options); diff --git a/ref-filter.c b/ref-filter.c index 660e52f97f79..5e4aaa94db8a 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1,3 +1,4 @@ +#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "cache.h" #include "parse-options.h" @@ -84,6 +85,9 @@ static struct expand_data { struct object_info info; } oi, oi_deref; +int use_filters; +int use_textconv; + struct ref_to_worktree_entry { struct hashmap_entry ent; struct worktree *wt; /* key is wt->head_ref */ @@ -1027,6 +1031,9 @@ int verify_ref_format(struct ref_format *format) used_atom[at].atom_type == ATOM_WORKTREEPATH))) die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); + use_filters = format->use_filters; + use_textconv = format->use_textconv; + if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && used_atom[at].u.raw_data.option == RAW_BARE) die(_("--format=%.*s cannot be used with" @@ -1735,10 +1742,41 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj { /* parse_object_buffer() will set eaten to 0 if free() will be needed */ int eaten = 1; + struct expand_data *actual_oi = oi; + struct expand_data act_oi = {0}; + if (oi->info.contentp) { /* We need to know that to use parse_object_buffer properly */ + void **temp_contentp = oi->info.contentp; + oi->info.contentp = NULL; oi->info.sizep = &oi->size; oi->info.typep = &oi->type; + + /* get the type and size */ + if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, + OBJECT_INFO_LOOKUP_REPLACE)) + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); + + oi->info.sizep = NULL; + oi->info.typep = NULL; + oi->info.contentp = temp_contentp; + + if (use_textconv) { + act_oi = *oi; + + if(!ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&act_oi.oid)); + if (act_oi.type == OBJ_BLOB) { + if (textconv_object(the_repository, + ref->rest, 0100644, &act_oi.oid, + 1, (char **)(&act_oi.content), &act_oi.size)) { + actual_oi = &act_oi; + goto success; + } + } + } } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) @@ -1748,19 +1786,43 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj BUG("Object size is less than zero."); if (oi->info.contentp) { - *obj = parse_object_buffer(the_repository, &oi->oid, oi->type, oi->size, oi->content, &eaten); + if (use_filters) { + if(!ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&oi->oid)); + if (oi->type == OBJ_BLOB) { + struct strbuf strbuf = STRBUF_INIT; + struct checkout_metadata meta; + act_oi = *oi; + + init_checkout_metadata(&meta, NULL, NULL, &act_oi.oid); + if (convert_to_working_tree(&the_index, ref->rest, act_oi.content, act_oi.size, &strbuf, &meta)) { + act_oi.size = strbuf.len; + act_oi.content = strbuf_detach(&strbuf, NULL); + actual_oi = &act_oi; + } else { + die("could not convert '%s' %s", + oid_to_hex(&oi->oid), ref->rest); + } + } + } + +success: + *obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten); if (!*obj) { if (!eaten) free(oi->content); return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi); + grab_values(ref->value, deref, *obj, actual_oi); } grab_common_values(ref->value, deref, oi); if (!eaten) free(oi->content); + if (actual_oi != oi) + free(actual_oi->content); return 0; } diff --git a/ref-filter.h b/ref-filter.h index 3db07729f46d..56a5ec4c9eaa 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -80,6 +80,8 @@ struct ref_format { const char *rest; int cat_file_mode; int quote_style; + int use_textconv; + int use_filters; int use_rest; int use_color; @@ -87,7 +89,7 @@ struct ref_format { int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { NULL, NULL, 0, 0, 0, -1 } +#define REF_FORMAT_INIT { NULL, NULL, 0, 0, 0, 0, 0, -1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \