From patchwork Mon Jul 12 11:46:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46C9BC07E99 for ; Mon, 12 Jul 2021 11:47:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 246EF610E6 for ; Mon, 12 Jul 2021 11:47:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230197AbhGLLtv (ORCPT ); Mon, 12 Jul 2021 07:49:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229910AbhGLLtv (ORCPT ); Mon, 12 Jul 2021 07:49:51 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7EC2C0613E8 for ; Mon, 12 Jul 2021 04:47:02 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id v5so25205052wrt.3 for ; Mon, 12 Jul 2021 04:47:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=gcJvRQAqHD2kOOqwmBikpeCsu0xCI+Gpwxqt7a1E8WM=; b=T33QOOK4igKSOKsbBnUkhnJCCKkEIlSJStH5Uvm4nv4ATUaKDotJnTnJYT/rhUv7uU pdLCBs6EoiNgdevp71vRevwmtBLEpnZRldbDzk8OiMwsjrecYSeo9trA7w2w+bWMWSlV qQPtP3c7SVDteNBYy5pmjPXvijkz0mjfbHmmRn9xuCMHsHM6Fe9OSBrnzTx+1/BhzAUk RkEZnMeQtooBDbnYtQH9AxMD8WZgqidNWgrMRUbmlssBzzBL/TgsHwNygaaDWvy9n3Vt 9QGqqUPTpsSOZTXhPbrcl5uFm8SskWzOINU8m/ZjKY8v/+UtR+Chbl60f2PZnwZVj399 ux6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=gcJvRQAqHD2kOOqwmBikpeCsu0xCI+Gpwxqt7a1E8WM=; b=EmtM9Zg1sCpKg/IsGnMxZ4iHMSYrHFjdCRyeAtHetfxe7wWEl7y4qhpMi1pUvvuwum /A56/pahEe9LGt7z25MskHROTnL64ST3EH8ReuXRzqu+PXiQI8907ee4v0Nu3K0+KwSH xFy0iafr/RMGDIyXUcihbZ3nbQwRdbaC4Dx7HGzucWqDgXgw7VDqDbrJT3GIqWkGV2eq oSeUddnwYUFiiUJlknmx68TgCZhxQcNtxGpsNlgPFYF91xnI5zSaQg6ycwW9ibrBuP46 HcPfZwnqvME6vlrXGiIqMBDHwWybhClECdPRaKZPuTN7wESWssm36ZdxV7CRQTZxoflJ 3hMA== X-Gm-Message-State: AOAM5332A8ZVc4rqQurtbjzJf2rzypgJoJwPA6IQswURkL7N43J6lxim f6S439ehYcJqIZFo9q8yh0vfMS6EZxI= X-Google-Smtp-Source: ABdhPJzZGaXxm8pdJ5Q3gk2c3s8rq3d0AXHr52Yy7flf8/4iGCjeyb3SVdcKoQzSd8Pzp/ozjzWXiw== X-Received: by 2002:a5d:6b8e:: with SMTP id n14mr11376051wrx.96.1626090421472; Mon, 12 Jul 2021 04:47:01 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m6sm17034258wrw.9.2021.07.12.04.47.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:01 -0700 (PDT) Message-Id: <9aef8882bd1e5e9bdd401d92449190f462310e40.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:41 +0000 Subject: [PATCH 01/19] cat-file: handle trivial --batch format with --batch-all-objects Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu The --batch code to print an object assumes we found out the type of the object from calling oid_object_info_extended(). This is true for the default format, but even in a custom format, we manually modify the object_info struct to ask for the type. This assumption was broken by 845de33a5b (cat-file: avoid noop calls to sha1_object_info_extended, 2016-05-18). That commit skips the call to oid_object_info_extended() entirely when --batch-all-objects is in use, and the custom format does not include any placeholders that require calling it. Or when the custom format only include placeholders like %(objectname) or %(rest), oid_object_info_extended() will not get the type of the object. This results in an error when we try to confirm that the type didn't change: $ git cat-file --batch=batman --batch-all-objects batman fatal: object 000023961a0c02d6e21dc51ea3484ff71abf1c74 changed type!? and also has other subtle effects (e.g., we'd fail to stream a blob, since we don't realize it's a blob in the first place). We can fix this by flipping the order of the setup. The check for "do we need to get the object info" must come _after_ we've decided whether we need to look up the type. Helped-by: Jeff King Signed-off-by: ZheNing Hu Acked-by: Jeff King Signed-off-by: Junio C Hamano --- builtin/cat-file.c | 13 +++++++------ t/t1006-cat-file.sh | 22 ++++++++++++++++++++++ 2 files changed, 29 insertions(+), 6 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 5ebf13359e8..02461bb5ea6 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -512,12 +512,6 @@ static int batch_objects(struct batch_options *opt) if (opt->cmdmode) data.split_on_whitespace = 1; - if (opt->all_objects) { - struct object_info empty = OBJECT_INFO_INIT; - if (!memcmp(&data.info, &empty, sizeof(empty))) - data.skip_object_info = 1; - } - /* * If we are printing out the object, then always fill in the type, * since we will want to decide whether or not to stream. @@ -525,6 +519,13 @@ static int batch_objects(struct batch_options *opt) if (opt->print_contents) data.info.typep = &data.type; + if (opt->all_objects) { + struct object_info empty = OBJECT_INFO_INIT; + + if (!memcmp(&data.info, &empty, sizeof(empty))) + data.skip_object_info = 1; + } + if (opt->all_objects) { struct object_cb_data cb; diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 5d2dc99b74a..18b3779ccb6 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -586,4 +586,26 @@ test_expect_success 'cat-file --unordered works' ' test_cmp expect actual ' +test_expect_success 'set up object list for --batch-all-objects tests' ' + git -C all-two cat-file --batch-all-objects --batch-check="%(objectname)" >objects +' + +test_expect_success 'cat-file --batch="%(objectname)" with --batch-all-objects will work' ' + git -C all-two cat-file --batch="%(objectname)" expect && + git -C all-two cat-file --batch-all-objects --batch="%(objectname)" >actual && + cmp expect actual +' + +test_expect_success 'cat-file --batch="%(rest)" with --batch-all-objects will work' ' + git -C all-two cat-file --batch="%(rest)" expect && + git -C all-two cat-file --batch-all-objects --batch="%(rest)" >actual && + cmp expect actual +' + +test_expect_success 'cat-file --batch="batman" with --batch-all-objects will work' ' + git -C all-two cat-file --batch="batman" expect && + git -C all-two cat-file --batch-all-objects --batch="batman" >actual && + cmp expect actual +' + test_done From patchwork Mon Jul 12 11:46:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E69DC07E9C for ; Mon, 12 Jul 2021 11:47:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31D43610CD for ; Mon, 12 Jul 2021 11:47:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230503AbhGLLtx (ORCPT ); Mon, 12 Jul 2021 07:49:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230144AbhGLLtv (ORCPT ); Mon, 12 Jul 2021 07:49:51 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58B59C0613E5 for ; Mon, 12 Jul 2021 04:47:03 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id v5so25205074wrt.3 for ; Mon, 12 Jul 2021 04:47:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=R0c19oCV7JaE9Qk6Yj5Q6fZFwz5yzSxqyBk+M85AK50=; b=LrXCy86/kvZp+5pb/rjkWxAMFkX1vMSQaBjLeLuwnXLUwP6VTIaDhMk9A6vE2qmWfB VFlKloNV95cu8xibtvwGe0uYJXMBo73pwE/vEUr0f5IZRAxW0Nskb7enh8Ph/Vv3z+NH DmvLMX4+GNqpYDKrbW6T6336gt2VENMD3FetpwrwRpPVo0bzA6+ohZQQQ19Jh8hRHEQm GwpZARzjO1BvCvrYyiA+aE65QfX3BI0KcDXWE/d+x07PJusEZmO+BpOpONqoYuFtruYA 8+7ndC43Mcty2VegXkWcwLkXRM+NBPPwFkFE0ulJCk4riyMTKMP2usVV4D925bePuToB 5oJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=R0c19oCV7JaE9Qk6Yj5Q6fZFwz5yzSxqyBk+M85AK50=; b=G5kMdlqlMnyefUT5YW27NYJXeutbarwkku3KurUGWoamCJtRWBV1USMMyEPT5NXmYI TxieId0mZhqrhwOos4AV5iUVxfBUsHysrREwQpO8NPo7OAokL26LaJIv5Jfq/daROrkg jr2R4Ey+syEOlMODCk9Ep6n2+GlKrdocYY8+LdN/Pi28qZS06tKyZhTDjyhsxDqM9HuZ mXmhIR+/2FVL2TnWCrXGZy3LCnz4zzLjtWJyYbgKucL8mBhnuGL34/AeOiS8l3pY6mpr BkHwDHiOC0axKcMdoEvFj05q/qK/Ri1Vc6HFxMmGiwVr/ZpakcwLUBh0p4gfYDJs0nny ULAg== X-Gm-Message-State: AOAM532PA3SEw4A3/4n01bksy0QriM/FwSb6PNWLlTSAvJDpxC2MhLia mvojMOvsU8N2eBSArG5DSdzvqtgAxRA= X-Google-Smtp-Source: ABdhPJykrYoVwBty5PyawEv2L4JQlMiK6WFQOzcmssGFGJkSF+cFuIhWNOS15xNMdJ+iJkE46WmblQ== X-Received: by 2002:a5d:5142:: with SMTP id u2mr31445744wrt.314.1626090422024; Mon, 12 Jul 2021 04:47:02 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k20sm13005163wrd.70.2021.07.12.04.47.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:01 -0700 (PDT) Message-Id: <1332006006f4596122aa05afcb2b35f3ee781e1a.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:42 +0000 Subject: [PATCH 02/19] cat-file: merge two block into one Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu There are two "if (opt->all_objects)" blocks next to each other, merge them into one to provide better readability. Helped-by: Jeff King Signed-off-by: ZheNing Hu Acked-by: Jeff King Signed-off-by: Junio C Hamano --- builtin/cat-file.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 02461bb5ea6..243fe6844bc 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -520,14 +520,11 @@ static int batch_objects(struct batch_options *opt) data.info.typep = &data.type; if (opt->all_objects) { + struct object_cb_data cb; struct object_info empty = OBJECT_INFO_INIT; if (!memcmp(&data.info, &empty, sizeof(empty))) data.skip_object_info = 1; - } - - if (opt->all_objects) { - struct object_cb_data cb; if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); From patchwork Mon Jul 12 11:46:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA68FC11F66 for ; Mon, 12 Jul 2021 11:47:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8952161108 for ; Mon, 12 Jul 2021 11:47:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231373AbhGLLt5 (ORCPT ); Mon, 12 Jul 2021 07:49:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230207AbhGLLtw (ORCPT ); Mon, 12 Jul 2021 07:49:52 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6DA3C0613DD for ; Mon, 12 Jul 2021 04:47:03 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id l18-20020a1ced120000b029014c1adff1edso14231384wmh.4 for ; Mon, 12 Jul 2021 04:47:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=/+pHTUCgsn/eJhBt7TH6g2FT5CeygqVH9py2mRcVtgU=; b=s8lbXqKgk2lpWjeDmZ+m+5sRqq/nDpFZL5bZUH8dM7XrtOTtTIHvXwrhfmO1ubXqIW Bw/eiobMrFuuJL6rPAzCtxRCEapV43nlXvKn2aFgHieXMe9kUCf05oougYE98l809RIx tiiSMliIgHxzdBm2o6MRPpsLL5I5PXG3KwQi3oAu8hHHWwU6tQBkkX263Jzg4ixFmuvk 2NzJ679Y1aVMupWn/9usYFUuNJD+ixC4o8/tBNUVUm7OeLpXq+qwSep5DTw6qtJY88WY JXjB7BBcANCs9mF6cv7U3yX7XRUzeYQjf6qEMB4PQTJ8RiH1a7NmfyNDUmWFqJxlXudj RyLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=/+pHTUCgsn/eJhBt7TH6g2FT5CeygqVH9py2mRcVtgU=; b=Mnkq2rWvX/vwHAQutJMmYEDApceHxwJHgOO2bRCj2DsziKqzHnwuTX3EMjK8MBqDMK Xp2MAjRlMlX4YiIEQxiSp5Ny8uT3nzcGVNGgMN+bunSn6+FtPaEMAMejysSWhOAC3hRX +yHWEH4sV2kOCfTkjxwSq5BkruyGcN+vLRqvrP9Nkdn9RZ8O6hsh13SaQt1/RMxJFm6e n5EVNYdSLYz3Mt+uDMm5vRHS+udM04zYYgOFw3/TD/mJ0NGxwo63dTRfUHdaLMXZgu2A kLZOx/9E3qcGQjnVAI85scURUEbLYpANJwlyDDeSGFK2fIqrLiytQtr3IJ4mq614aE5n 4eQg== X-Gm-Message-State: AOAM5303BCI4BBKwaVEnfCTxUIOkACBOgzcz+QQuv4Nq4ej5T/llw6mF k53aX0sz5LzFykRbJ000byUcE3ljdH0= X-Google-Smtp-Source: ABdhPJzKrST1MCnxzHoo06hkCxk1J1eUbCgAHlr/IW0+E6sIkewu/pr2d3gcv6XRMIBRS5ZCHeS9kQ== X-Received: by 2002:a1c:a903:: with SMTP id s3mr14054827wme.58.1626090422514; Mon, 12 Jul 2021 04:47:02 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p5sm14528891wrd.25.2021.07.12.04.47.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:02 -0700 (PDT) Message-Id: <0dacc60bbcc9ae4376f58c967f43bd0c2396fd7e.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:43 +0000 Subject: [PATCH 03/19] [GSOC] ref-filter: add obj-type check in grab contents Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Only tag and commit objects use `grab_sub_body_contents()` to grab object contents in the current codebase. We want to teach the function to also handle blobs and trees to get their raw data, without parsing a blob (whose contents looks like a commit or a tag) incorrectly as a commit or a tag. Skip the block of code that is specific to handling commits and tags early when the given object is of a wrong type to help later addition to handle other types of objects in this function. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Junio C Hamano Signed-off-by: ZheNing Hu --- ref-filter.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 4db0e40ff4c..5cee6512fba 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1356,11 +1356,12 @@ static void append_lines(struct strbuf *out, const char *buf, unsigned long size } /* See grab_values */ -static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) +static void grab_sub_body_contents(struct atom_value *val, int deref, struct expand_data *data) { int i; const char *subpos = NULL, *bodypos = NULL, *sigpos = NULL; size_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0; + void *buf = data->content; for (i = 0; i < used_atom_cnt; i++) { struct used_atom *atom = &used_atom[i]; @@ -1371,10 +1372,13 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf) continue; if (deref) name++; - if (strcmp(name, "body") && - !starts_with(name, "subject") && - !starts_with(name, "trailers") && - !starts_with(name, "contents")) + + if ((data->type != OBJ_TAG && + data->type != OBJ_COMMIT) || + (strcmp(name, "body") && + !starts_with(name, "subject") && + !starts_with(name, "trailers") && + !starts_with(name, "contents"))) continue; if (!subpos) find_subpos(buf, @@ -1438,17 +1442,19 @@ static void fill_missing_values(struct atom_value *val) * pointed at by the ref itself; otherwise it is the object the * ref (which is a tag) refers to. */ -static void grab_values(struct atom_value *val, int deref, struct object *obj, void *buf) +static void grab_values(struct atom_value *val, int deref, struct object *obj, struct expand_data *data) { + void *buf = data->content; + switch (obj->type) { case OBJ_TAG: grab_tag_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("tagger", val, deref, buf); break; case OBJ_COMMIT: grab_commit_values(val, deref, obj); - grab_sub_body_contents(val, deref, buf); + grab_sub_body_contents(val, deref, data); grab_person("author", val, deref, buf); grab_person("committer", val, deref, buf); break; @@ -1678,7 +1684,7 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi->content); + grab_values(ref->value, deref, *obj, oi); } grab_common_values(ref->value, deref, oi); From patchwork Mon Jul 12 11:46:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EB5CC07E9A for ; Mon, 12 Jul 2021 11:47:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37CE660C3F for ; Mon, 12 Jul 2021 11:47:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232421AbhGLLuA (ORCPT ); Mon, 12 Jul 2021 07:50:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231175AbhGLLty (ORCPT ); Mon, 12 Jul 2021 07:49:54 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAAACC0613DD for ; Mon, 12 Jul 2021 04:47:04 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id m2so14165447wrq.2 for ; Mon, 12 Jul 2021 04:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=SUDgA8t1JEBztczygiAw3+rz9y2lUTOzWXg3n4xUxXY=; b=NJuQDYbkxd/PDKq6c08aeJ8MTi2UjBCJh8fbEeiw2Ob/sGnelVNk3/6ccj2oirPb5I HPOp5EyjEYJY4guMWHl5FKNj9YO+5SSKPVuI46RBDvS8LUhYBv5SlU2/VcS26duol4FD 4vVWqXG3rkUnpNKc49xUP1wJO2TijhuHWAVVFDISM69C1lXtBdNwdRB7qP23hELjnv/q eCQ7ANZwnUVKjKOiXm8nCNV9s++LyvS8bkC/JJd7o7Yt5Q9KI30qYqjOcx/XsQJ0imLy ztICwH/gw/ZeQFL4mVsmZr4O2aYCE0WhSFkJJxotcMcYnBByOpjBW7lcZGAVNc26/wNq LEAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=SUDgA8t1JEBztczygiAw3+rz9y2lUTOzWXg3n4xUxXY=; b=M32WixeWa9Gb2xvDV3BZ03vp/pc0wHMUixZwO85Y85XbVSwcJe7SeuA1e2owwpu/Yh 4wv7NksREDGZ+XTwJn4yRRyslsnoT+O6F+DQk8RF95R+ZFQmc4kPY79Y1dwazRqzxsRL VabeA1d4c7w+G5ZKiV0trbt8ifjFXZh7B1H1W4W1r6DrRWyK0shPTYtD5OUHAmW6Qf2Y qEXNqK4dEMfGf4dHRNFKhK1XhYng7OP9D1mSv06J2UjPddfWFOsogLDKnU8h45FkMJqr Iz1SQqIVZhWdQ9vjISJQNQVvwNTnR++AI9lAECXhNNejcnv9nRvPwDX1ZeJ9sDScQqew VFmA== X-Gm-Message-State: AOAM530fmI3EZs5A5DVoJhsG2CWmvu8ln4T8WF7hn1ACAXt6hJ/FglHI Ikg3ckb4nPYrMBJWi6Wicm3YiuRuQCI= X-Google-Smtp-Source: ABdhPJyGnBRM4/2aS252iXehLBSRjdL6szIrD38qU/mPva9IzpycIdhRWXuoOq3mAQP0avfOYBqVBA== X-Received: by 2002:a5d:5987:: with SMTP id n7mr3027608wri.263.1626090423249; Mon, 12 Jul 2021 04:47:03 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e6sm5112177wrg.18.2021.07.12.04.47.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:02 -0700 (PDT) Message-Id: <5bd715ae9f02a580f3594a3207df45707119aeb1.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:44 +0000 Subject: [PATCH 04/19] [GSOC] ref-filter: add %(raw) atom MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add new formatting option `%(raw)`, which will print the raw object data without any changes. It will help further to migrate all cat-file formatting logic from cat-file to ref-filter. The raw data of blob, tree objects may contain '\0', but most of the logic in `ref-filter` depends on the output of the atom being text (specifically, no embedded NULs in it). E.g. `quote_formatting()` use `strbuf_addstr()` or `*._quote_buf()` add the data to the buffer. The raw data of a tree object is `100644 one\0...`, only the `100644 one` will be added to the buffer, which is incorrect. Therefore, we need to find a way to record the length of the atom_value's member `s`. Although strbuf can already record the string and its length, if we want to replace the type of atom_value's member `s` with strbuf, many places in ref-filter that are filled with dynamically allocated mermory in `v->s` are not easy to replace. At the same time, we need to check if `v->s == NULL` in populate_value(), and strbuf cannot easily distinguish NULL and empty strings, but c-style "const char *" can do it. So add a new member in `struct atom_value`: `s_size`, which can record raw object size, it can help us add raw object data to the buffer or compare two buffers which contain raw object data. Note that `--format=%(raw)` cannot be used with `--python`, `--shell`, `--tcl`, and `--perl` because if the binary raw data is passed to a variable in such languages, these may not support arbitrary binary data in their string variable type. Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Bagas Sanjaya Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Felipe Contreras Helped-by: Phillip Wood Helped-by: Junio C Hamano Based-on-patch-by: Olga Telezhnaya Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 9 ++ ref-filter.c | 140 +++++++++++++++---- t/t6300-for-each-ref.sh | 216 +++++++++++++++++++++++++++++ 3 files changed, 338 insertions(+), 27 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 2ae2478de70..cbb6f87d13f 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -235,6 +235,15 @@ and `date` to extract the named component. For email fields (`authoremail`, without angle brackets, and `:localpart` to get the part before the `@` symbol out of the trimmed email. +The raw data in an object is `raw`. + +raw:size:: + The raw data size of the object. + +Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, +`--perl` because such language may not support arbitrary binary data in their +string variable type. + The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/ref-filter.c b/ref-filter.c index 5cee6512fba..506fbc3d691 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -144,6 +144,7 @@ enum atom_type { ATOM_BODY, ATOM_TRAILERS, ATOM_CONTENTS, + ATOM_RAW, ATOM_UPSTREAM, ATOM_PUSH, ATOM_SYMREF, @@ -189,6 +190,9 @@ static struct used_atom { struct process_trailer_options trailer_opts; unsigned int nlines; } contents; + struct { + enum { RAW_BARE, RAW_LENGTH } option; + } raw_data; struct { cmp_status cmp_status; const char *str; @@ -426,6 +430,18 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } +static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (!arg) + atom->u.raw_data.option = RAW_BARE; + else if (!strcmp(arg, "size")) + atom->u.raw_data.option = RAW_LENGTH; + else + return strbuf_addf_ret(err, -1, _("unrecognized %%(raw) argument: %s"), arg); + return 0; +} + static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { @@ -586,6 +602,7 @@ static struct { [ATOM_BODY] = { "body", SOURCE_OBJ, FIELD_STR, body_atom_parser }, [ATOM_TRAILERS] = { "trailers", SOURCE_OBJ, FIELD_STR, trailers_atom_parser }, [ATOM_CONTENTS] = { "contents", SOURCE_OBJ, FIELD_STR, contents_atom_parser }, + [ATOM_RAW] = { "raw", SOURCE_OBJ, FIELD_STR, raw_atom_parser }, [ATOM_UPSTREAM] = { "upstream", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_PUSH] = { "push", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser }, [ATOM_SYMREF] = { "symref", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -620,12 +637,15 @@ struct ref_formatting_state { struct atom_value { const char *s; + size_t s_size; int (*handler)(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err); uintmax_t value; /* used for sorting when not FIELD_STR */ struct used_atom *atom; }; +#define ATOM_VALUE_S_SIZE_INIT (-1) + /* * Used to parse format string and sort specifiers */ @@ -644,13 +664,6 @@ static int parse_ref_filter_atom(const struct ref_format *format, return strbuf_addf_ret(err, -1, _("malformed field name: %.*s"), (int)(ep-atom), atom); - /* Do we have the atom already used elsewhere? */ - for (i = 0; i < used_atom_cnt; i++) { - int len = strlen(used_atom[i].name); - if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) - return i; - } - /* * If the atom name has a colon, strip it and everything after * it off - it specifies the format for this entry, and @@ -660,6 +673,13 @@ static int parse_ref_filter_atom(const struct ref_format *format, arg = memchr(sp, ':', ep - sp); atom_len = (arg ? arg : ep) - sp; + /* Do we have the atom already used elsewhere? */ + for (i = 0; i < used_atom_cnt; i++) { + int len = strlen(used_atom[i].name); + if (len == ep - atom && !memcmp(used_atom[i].name, atom, len)) + return i; + } + /* Is the atom a valid one? */ for (i = 0; i < ARRAY_SIZE(valid_atom); i++) { int len = strlen(valid_atom[i].name); @@ -709,11 +729,14 @@ static int parse_ref_filter_atom(const struct ref_format *format, return at; } -static void quote_formatting(struct strbuf *s, const char *str, int quote_style) +static void quote_formatting(struct strbuf *s, const char *str, size_t len, int quote_style) { switch (quote_style) { case QUOTE_NONE: - strbuf_addstr(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(s, str, len); + else + strbuf_addstr(s, str); break; case QUOTE_SHELL: sq_quote_buf(s, str); @@ -740,9 +763,12 @@ static int append_atom(struct atom_value *v, struct ref_formatting_state *state, * encountered. */ if (!state->stack->prev) - quote_formatting(&state->stack->output, v->s, state->quote_style); + quote_formatting(&state->stack->output, v->s, v->s_size, state->quote_style); else - strbuf_addstr(&state->stack->output, v->s); + if (v->s_size != ATOM_VALUE_S_SIZE_INIT) + strbuf_add(&state->stack->output, v->s, v->s_size); + else + strbuf_addstr(&state->stack->output, v->s); return 0; } @@ -842,21 +868,23 @@ static int if_atom_handler(struct atom_value *atomv, struct ref_formatting_state return 0; } -static int is_empty(const char *s) +static int is_empty(struct strbuf *buf) { - while (*s != '\0') { - if (!isspace(*s)) - return 0; - s++; - } - return 1; -} + const char *cur = buf->buf; + const char *end = buf->buf + buf->len; + + while (cur != end && (isspace(*cur))) + cur++; + + return cur == end; + } static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_state *state, struct strbuf *err) { struct ref_formatting_stack *cur = state->stack; struct if_then_else *if_then_else = NULL; + size_t str_len = 0; if (cur->at_end == if_then_else_handler) if_then_else = (struct if_then_else *)cur->at_end_data; @@ -867,18 +895,22 @@ static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_sta if (if_then_else->else_atom_seen) return strbuf_addf_ret(err, -1, _("format: %%(then) atom used after %%(else)")); if_then_else->then_atom_seen = 1; + if (if_then_else->str) + str_len = strlen(if_then_else->str); /* * If the 'equals' or 'notequals' attribute is used then * perform the required comparison. If not, only non-empty * strings satisfy the 'if' condition. */ if (if_then_else->cmp_status == COMPARE_EQUAL) { - if (!strcmp(if_then_else->str, cur->output.buf)) + if (str_len == cur->output.len && + !memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; } else if (if_then_else->cmp_status == COMPARE_UNEQUAL) { - if (strcmp(if_then_else->str, cur->output.buf)) + if (str_len != cur->output.len || + memcmp(if_then_else->str, cur->output.buf, cur->output.len)) if_then_else->condition_satisfied = 1; - } else if (cur->output.len && !is_empty(cur->output.buf)) + } else if (cur->output.len && !is_empty(&cur->output)) if_then_else->condition_satisfied = 1; strbuf_reset(&cur->output); return 0; @@ -924,7 +956,7 @@ static int end_atom_handler(struct atom_value *atomv, struct ref_formatting_stat * only on the topmost supporting atom. */ if (!current->prev->prev) { - quote_formatting(&s, current->output.buf, state->quote_style); + quote_formatting(&s, current->output.buf, current->output.len, state->quote_style); strbuf_swap(¤t->output, &s); } strbuf_release(&s); @@ -974,6 +1006,10 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); + if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) + die(_("--format=%.*s cannot be used with" + "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) @@ -1367,12 +1403,25 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp struct used_atom *atom = &used_atom[i]; const char *name = atom->name; struct atom_value *v = &val[i]; + enum atom_type atom_type = atom->atom_type; if (!!deref != (*name == '*')) continue; if (deref) name++; + if (atom_type == ATOM_RAW) { + unsigned long buf_size = data->size; + + if (atom->u.raw_data.option == RAW_BARE) { + v->s = xmemdupz(buf, buf_size); + v->s_size = buf_size; + } else if (atom->u.raw_data.option == RAW_LENGTH) { + v->s = xstrfmt("%"PRIuMAX, (uintmax_t)buf_size); + } + continue; + } + if ((data->type != OBJ_TAG && data->type != OBJ_COMMIT) || (strcmp(name, "body") && @@ -1460,9 +1509,11 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s break; case OBJ_TREE: /* grab_tree_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; case OBJ_BLOB: /* grab_blob_values(val, deref, obj, buf, sz); */ + grab_sub_body_contents(val, deref, data); break; default: die("Eh? Object of type %d?", obj->type); @@ -1766,6 +1817,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) const char *refname; struct branch *branch = NULL; + v->s_size = ATOM_VALUE_S_SIZE_INIT; v->handler = append_atom; v->atom = atom; @@ -2369,6 +2421,19 @@ static int compare_detached_head(struct ref_array_item *a, struct ref_array_item return 0; } +static int memcasecmp(const void *vs1, const void *vs2, size_t n) +{ + const char *s1 = vs1, *s2 = vs2; + const char *end = s1 + n; + + for (; s1 < end; s1++, s2++) { + int diff = tolower(*s1) - tolower(*s2); + if (diff) + return diff; + } + return 0; +} + static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, struct ref_array_item *b) { struct atom_value *va, *vb; @@ -2389,10 +2454,30 @@ static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, stru } else if (s->sort_flags & REF_SORTING_VERSION) { cmp = versioncmp(va->s, vb->s); } else if (cmp_type == FIELD_STR) { - int (*cmp_fn)(const char *, const char *); - cmp_fn = s->sort_flags & REF_SORTING_ICASE - ? strcasecmp : strcmp; - cmp = cmp_fn(va->s, vb->s); + if (va->s_size == ATOM_VALUE_S_SIZE_INIT && + vb->s_size == ATOM_VALUE_S_SIZE_INIT) { + int (*cmp_fn)(const char *, const char *); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? strcasecmp : strcmp; + cmp = cmp_fn(va->s, vb->s); + } else { + size_t a_size = va->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(va->s) : va->s_size; + size_t b_size = vb->s_size == ATOM_VALUE_S_SIZE_INIT ? + strlen(vb->s) : vb->s_size; + int (*cmp_fn)(const void *, const void *, size_t); + cmp_fn = s->sort_flags & REF_SORTING_ICASE + ? memcasecmp : memcmp; + + cmp = cmp_fn(va->s, vb->s, b_size > a_size ? + a_size : b_size); + if (!cmp) { + if (a_size > b_size) + cmp = 1; + else if (a_size < b_size) + cmp = -1; + } + } } else { if (va->value < vb->value) cmp = -1; @@ -2492,6 +2577,7 @@ int format_ref_array_item(struct ref_array_item *info, } if (format->need_color_reset_at_eol) { struct atom_value resetv; + resetv.s_size = ATOM_VALUE_S_SIZE_INIT; resetv.s = GIT_COLOR_RESET; if (append_atom(&resetv, &state, error_buf)) { pop_stack_element(&state.stack); diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 9e0214076b4..18554f62d94 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -130,6 +130,8 @@ test_atom head parent:short=10 '' test_atom head numparent 0 test_atom head object '' test_atom head type '' +test_atom head raw "$(git cat-file commit refs/heads/main) +" test_atom head '*objectname' '' test_atom head '*objecttype' '' test_atom head author 'A U Thor 1151968724 +0200' @@ -221,6 +223,15 @@ test_atom tag contents 'Tagging at 1151968727 ' test_atom tag HEAD ' ' +test_expect_success 'basic atom: refs/tags/testtag *raw' ' + git cat-file commit refs/tags/testtag^{} >expected && + git for-each-ref --format="%(*raw)" refs/tags/testtag >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'Check invalid atoms names are errors' ' test_must_fail git for-each-ref --format="%(INVALID)" refs/heads ' @@ -686,6 +697,15 @@ test_atom refs/tags/signed-empty contents:body '' test_atom refs/tags/signed-empty contents:signature "$sig" test_atom refs/tags/signed-empty contents "$sig" +test_expect_success GPG 'basic atom: refs/tags/signed-empty raw' ' + git cat-file tag refs/tags/signed-empty >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-empty >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-short subject 'subject line' test_atom refs/tags/signed-short subject:sanitize 'subject-line' test_atom refs/tags/signed-short contents:subject 'subject line' @@ -695,6 +715,15 @@ test_atom refs/tags/signed-short contents:signature "$sig" test_atom refs/tags/signed-short contents "subject line $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-short raw' ' + git cat-file tag refs/tags/signed-short >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-short >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_atom refs/tags/signed-long subject 'subject line' test_atom refs/tags/signed-long subject:sanitize 'subject-line' test_atom refs/tags/signed-long contents:subject 'subject line' @@ -708,6 +737,15 @@ test_atom refs/tags/signed-long contents "subject line body contents $sig" +test_expect_success GPG 'basic atom: refs/tags/signed-long raw' ' + git cat-file tag refs/tags/signed-long >expected && + git for-each-ref --format="%(raw)" refs/tags/signed-long >actual && + sanitize_pgp expected.clean && + echo >>expected.clean && + sanitize_pgp actual.clean && + test_cmp expected.clean actual.clean +' + test_expect_success 'set up refs pointing to tree and blob' ' git update-ref refs/mytrees/first refs/heads/main^{tree} && git update-ref refs/myblobs/first refs/heads/main:one @@ -720,6 +758,16 @@ test_atom refs/mytrees/first contents:body "" test_atom refs/mytrees/first contents:signature "" test_atom refs/mytrees/first contents "" +test_expect_success 'basic atom: refs/mytrees/first raw' ' + git cat-file tree refs/mytrees/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/mytrees/first >actual && + test_cmp expected actual && + git cat-file -s refs/mytrees/first >expected && + git for-each-ref --format="%(raw:size)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_atom refs/myblobs/first subject "" test_atom refs/myblobs/first contents:subject "" test_atom refs/myblobs/first body "" @@ -727,6 +775,174 @@ test_atom refs/myblobs/first contents:body "" test_atom refs/myblobs/first contents:signature "" test_atom refs/myblobs/first contents "" +test_expect_success 'basic atom: refs/myblobs/first raw' ' + git cat-file blob refs/myblobs/first >expected && + echo >>expected && + git for-each-ref --format="%(raw)" refs/myblobs/first >actual && + test_cmp expected actual && + git cat-file -s refs/myblobs/first >expected && + git for-each-ref --format="%(raw:size)" refs/myblobs/first >actual && + test_cmp expected actual +' + +test_expect_success 'set up refs pointing to binary blob' ' + printf "a\0b\0c" >blob1 && + printf "a\0c\0b" >blob2 && + printf "\0a\0b\0c" >blob3 && + printf "abc" >blob4 && + printf "\0 \0 \0 " >blob5 && + printf "\0 \0a\0 " >blob6 && + printf " " >blob7 && + >blob8 && + obj=$(git hash-object -w blob1) && + git update-ref refs/myblobs/blob1 "$obj" && + obj=$(git hash-object -w blob2) && + git update-ref refs/myblobs/blob2 "$obj" && + obj=$(git hash-object -w blob3) && + git update-ref refs/myblobs/blob3 "$obj" && + obj=$(git hash-object -w blob4) && + git update-ref refs/myblobs/blob4 "$obj" && + obj=$(git hash-object -w blob5) && + git update-ref refs/myblobs/blob5 "$obj" && + obj=$(git hash-object -w blob6) && + git update-ref refs/myblobs/blob6 "$obj" && + obj=$(git hash-object -w blob7) && + git update-ref refs/myblobs/blob7 "$obj" && + obj=$(git hash-object -w blob8) && + git update-ref refs/myblobs/blob8 "$obj" +' + +test_expect_success 'Verify sorts with raw' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob3 + refs/myblobs/blob7 + refs/mytrees/first + refs/myblobs/first + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob4 + refs/heads/main + EOF + git for-each-ref --format="%(refname)" --sort=raw \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'Verify sorts with raw:size' ' + cat >expected <<-EOF && + refs/myblobs/blob8 + refs/myblobs/first + refs/myblobs/blob7 + refs/heads/main + refs/myblobs/blob4 + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/mytrees/first + EOF + git for-each-ref --format="%(refname)" --sort=raw:size \ + refs/heads/main refs/myblobs/ refs/mytrees/first >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:equals)' ' + cat >expected <<-EOF && + not equals + not equals + not equals + not equals + not equals + not equals + refs/myblobs/blob4 + not equals + not equals + not equals + not equals + not equals + EOF + git for-each-ref --format="%(if:equals=abc)%(raw)%(then)%(refname)%(else)not equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'validate raw atom with %(if:notequals)' ' + cat >expected <<-EOF && + refs/heads/ambiguous + refs/heads/main + refs/heads/newtag + refs/myblobs/blob1 + refs/myblobs/blob2 + refs/myblobs/blob3 + equals + refs/myblobs/blob5 + refs/myblobs/blob6 + refs/myblobs/blob7 + refs/myblobs/blob8 + refs/myblobs/first + EOF + git for-each-ref --format="%(if:notequals=abc)%(raw)%(then)%(refname)%(else)equals%(end)" \ + refs/myblobs/ refs/heads/ >actual && + test_cmp expected actual +' + +test_expect_success 'empty raw refs with %(if)' ' + cat >expected <<-EOF && + refs/myblobs/blob1 not empty + refs/myblobs/blob2 not empty + refs/myblobs/blob3 not empty + refs/myblobs/blob4 not empty + refs/myblobs/blob5 not empty + refs/myblobs/blob6 not empty + refs/myblobs/blob7 empty + refs/myblobs/blob8 empty + refs/myblobs/first not empty + EOF + git for-each-ref --format="%(refname) %(if)%(raw)%(then)not empty%(else)empty%(end)" \ + refs/myblobs/ >actual && + test_cmp expected actual +' + +test_expect_success '%(raw) with --python must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --python +' + +test_expect_success '%(raw) with --tcl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --tcl +' + +test_expect_success '%(raw) with --perl must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --perl +' + +test_expect_success '%(raw) with --shell must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --shell +' + +test_expect_success '%(raw) with --shell and --sort=raw must fail' ' + test_must_fail git for-each-ref --format="%(raw)" --sort=raw --shell +' + +test_expect_success '%(raw:size) with --shell' ' + git for-each-ref --format="%(raw:size)" | while read line + do + echo "'\''$line'\''" >>expect + done && + git for-each-ref --format="%(raw:size)" --shell >actual && + test_cmp expect actual +' + +test_expect_success 'for-each-ref --format compare with cat-file --batch' ' + git rev-parse refs/mytrees/first | git cat-file --batch >expected && + git for-each-ref --format="%(objectname) %(objecttype) %(objectsize) +%(raw)" refs/mytrees/first >actual && + test_cmp expected actual +' + test_expect_success 'set up multiple-sort tags' ' for when in 100000 200000 do From patchwork Mon Jul 12 11:46:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F69BC07E9C for ; Mon, 12 Jul 2021 11:47:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 80F8E610CD for ; Mon, 12 Jul 2021 11:47:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232731AbhGLLuC (ORCPT ); Mon, 12 Jul 2021 07:50:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230517AbhGLLty (ORCPT ); Mon, 12 Jul 2021 07:49:54 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17F86C0613E5 for ; Mon, 12 Jul 2021 04:47:05 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id m2so14165468wrq.2 for ; Mon, 12 Jul 2021 04:47:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=5lP/QekKLM+IAZt7gPZOgld34VmFhgG0WJwvzYiMv3c=; b=fEydNlS73A7/86Q2L+gDsxLxhy3wkEKzbaTYT8lzzFwJu68XsFTQjiueV3pOxp63wy T8Ow9eqJACSaUzPo9sn7r3d/bVqliPMQxldsZSYteOVACOByFsD5nq4P1Ac8y44Rzs7m dBarx3p6K/uF0DoqaA99p4g/eEVjJdXWl04F0/0c/Tu5Tkug0kIyirIpmPpKpO23B704 seXwALPVAP8XAln3r3J5OTN6r07K0DYMjvvumV0b0WeXSp7Osi12OoxTc9waB7In1UfZ SojgPcpTGwfzMiAW3bGJayQp9SJzRGTNI0Es8yG5YUcm3FuCw5xGrIu0c2PjM/p8JYWD G2xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=5lP/QekKLM+IAZt7gPZOgld34VmFhgG0WJwvzYiMv3c=; b=B7f8NMVnq64OFeqrW5hWY2Bw42XuQyfmUrat5Tf1rvWY2NyPX8n7LV8txP6OpTjOfh imHzgSkalF/h42Ndvuqn7cInZD2E4sQKJchpvqsQKr4ehPR0vaL2eYF67WDpsEDAqhng hnzcjkG54gKexnxwkHI+BXPd+GGmTBLQUUpFqbEPZPiMn6IH+RiwHZkncpwBOO6jWaiv AarxN7am3jMexFKFWhHOluwpolFhDnTTQJrfacqKk4xwmf08hyPRokW/kZWmCCsLm0Js 6YUVOQVTPCQCJyHTb5fLVieBEvsRULpngrlPip8aYMVz2cgHV6GgNqJMQI+J4JLm1WDZ xjfA== X-Gm-Message-State: AOAM530j5PP3O71nzgP1QuRuiUzt1LB3+X29f8b0FrRr8qd9itN+p+ts e7pj/RqvV52dQyHxMxl9KOO1yFdHawE= X-Google-Smtp-Source: ABdhPJwAg2YF1p+kIPdWD17KK/xvuyTIklJSrmQekaihewvHd5FruZIlLadEUZTY1n5S2wYLD8qBRw== X-Received: by 2002:adf:f04a:: with SMTP id t10mr57238017wro.318.1626090423764; Mon, 12 Jul 2021 04:47:03 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r4sm14441486wre.84.2021.07.12.04.47.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:03 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:45 +0000 Subject: [PATCH 05/19] [GSOC] ref-filter: --format=%(raw) re-support --perl MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because the perl language can handle binary data correctly, add the function perl_quote_buf_with_len(), which can specify the length of the data and prevent the data from being truncated at '\0' to help `--format="%(raw)"` re-support `--perl`. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- Documentation/git-for-each-ref.txt | 4 ++-- quote.c | 17 +++++++++++++++++ quote.h | 1 + ref-filter.c | 15 +++++++++++---- t/t6300-for-each-ref.sh | 19 +++++++++++++++++-- 5 files changed, 48 insertions(+), 8 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index cbb6f87d13f..6da899c6296 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -241,8 +241,8 @@ raw:size:: The raw data size of the object. Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`, -`--perl` because such language may not support arbitrary binary data in their -string variable type. +because such language may not support arbitrary binary data in their string +variable type. The message in a commit or a tag object is `contents`, from which `contents:` can be used to extract various parts out of: diff --git a/quote.c b/quote.c index 8a3a5e39eb1..26719d21d1e 100644 --- a/quote.c +++ b/quote.c @@ -471,6 +471,23 @@ void perl_quote_buf(struct strbuf *sb, const char *src) strbuf_addch(sb, sq); } +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len) +{ + const char sq = '\''; + const char bq = '\\'; + const char *c = src; + const char *end = src + len; + + strbuf_addch(sb, sq); + while (c != end) { + if (*c == sq || *c == bq) + strbuf_addch(sb, bq); + strbuf_addch(sb, *c); + c++; + } + strbuf_addch(sb, sq); +} + void python_quote_buf(struct strbuf *sb, const char *src) { const char sq = '\''; diff --git a/quote.h b/quote.h index 768cc6338e2..0fe69e264b0 100644 --- a/quote.h +++ b/quote.h @@ -94,6 +94,7 @@ char *quote_path(const char *in, const char *prefix, struct strbuf *out, unsigne /* quoting as a string literal for other languages */ void perl_quote_buf(struct strbuf *sb, const char *src); +void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len); void python_quote_buf(struct strbuf *sb, const char *src); void tcl_quote_buf(struct strbuf *sb, const char *src); void basic_regex_quote_buf(struct strbuf *sb, const char *src); diff --git a/ref-filter.c b/ref-filter.c index 506fbc3d691..ba9ab35d7ec 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -742,7 +742,10 @@ static void quote_formatting(struct strbuf *s, const char *str, size_t len, int sq_quote_buf(s, str); break; case QUOTE_PERL: - perl_quote_buf(s, str); + if (len != ATOM_VALUE_S_SIZE_INIT) + perl_quote_buf_with_len(s, str, len); + else + perl_quote_buf(s, str); break; case QUOTE_PYTHON: python_quote_buf(s, str); @@ -1006,10 +1009,14 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - if (format->quote_style && used_atom[at].atom_type == ATOM_RAW && - used_atom[at].u.raw_data.option == RAW_BARE) + + if ((format->quote_style == QUOTE_PYTHON || + format->quote_style == QUOTE_SHELL || + format->quote_style == QUOTE_TCL) && + used_atom[at].atom_type == ATOM_RAW && + used_atom[at].u.raw_data.option == RAW_BARE) die(_("--format=%.*s cannot be used with" - "--python, --shell, --tcl, --perl"), (int)(ep - sp - 2), sp + 2); + "--python, --shell, --tcl"), (int)(ep - sp - 2), sp + 2); cp = ep + 1; if (skip_prefix(used_atom[at].name, "color:", &color)) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 18554f62d94..3d15d0a5360 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -915,8 +915,23 @@ test_expect_success '%(raw) with --tcl must fail' ' test_must_fail git for-each-ref --format="%(raw)" --tcl ' -test_expect_success '%(raw) with --perl must fail' ' - test_must_fail git for-each-ref --format="%(raw)" --perl +test_expect_success '%(raw) with --perl' ' + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob1 --perl | perl >actual && + cmp blob1 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob3 --perl | perl >actual && + cmp blob3 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/blob8 --perl | perl >actual && + cmp blob8 actual && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/myblobs/first --perl | perl >actual && + cmp one actual && + git cat-file tree refs/mytrees/first > expected && + git for-each-ref --format="\$name= %(raw); +print \"\$name\"" refs/mytrees/first --perl | perl >actual && + cmp expected actual ' test_expect_success '%(raw) with --shell must fail' ' From patchwork Mon Jul 12 11:46:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17D43C11F68 for ; Mon, 12 Jul 2021 11:47:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EFEFC610FA for ; Mon, 12 Jul 2021 11:47:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231967AbhGLLt7 (ORCPT ); Mon, 12 Jul 2021 07:49:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231326AbhGLLtz (ORCPT ); Mon, 12 Jul 2021 07:49:55 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2777C0613E8 for ; Mon, 12 Jul 2021 04:47:05 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id l7so24271834wrv.7 for ; Mon, 12 Jul 2021 04:47:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=oz3JgcuWDaxi4uV/irOCjt4bpOTOyAMAqZMr2CVa+Ms=; b=eBM/fS0y17tYYPSYSj/39+Po11veYhGyEjZLrI32rHlcD+9kqqsFCr41WXh8K6wMa9 rdUXBEW+2pmdBzIfF5ptkNWdWFoFn8owHgmVxQTb3V2FLuRBwjkGxisBndlG6LuZWHTQ lZVNo0Mx9qplhCu/0X2ijSLLrZHRMjK1Nq3k+umer3AmD3Lu2VTZnlTQ1UmGnheyeYyb 4uvpqJdbG682KrsXohDpe4P1LrI3bBhQbHf58Lo9cKSg8oX56N9RWiwGWzpCB7jJMjng kq39uQE1zwq9jEWkoB9+WoqHiM+1a3hDcTC5IIa2UQUJ9gMhuKio/Ec87JmU05yySxxR GjgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=oz3JgcuWDaxi4uV/irOCjt4bpOTOyAMAqZMr2CVa+Ms=; b=oFkpehVWN8Oxpq2BUegvm5Hn4Lweo/mR4D+CYJIwlC6p1J+3uYD/QM2kGlXEWl7QRp IWxWFRVprCwScEepYcZFAeNQ9+uUTyFjDZEdKR8Po/Fiw1S+GqwYu5OPN1b4EbCBSbCY kNnXFwkKq6ked3IvDmoMa0qYggXBm0hP2dJtD7EkQFweFD9nd9Mt4SHD+aIQln9dHA+3 +jJ76+05iGqBHmIESMaEwxY2HzjhN3YX9yhA4lkhPEq0Qyqw3oLBS1pVEddXIYqgmQV3 n+TtjYwsS3dWQyDQ9IFOSBDeS1O6B7uSeo0dcRMuONukDpAcCbt/QmlPRBjfDt20YmER BJqw== X-Gm-Message-State: AOAM532swM9ZodQQ+c9MgqEHkOWzul/SOtgakK7y860wHHrMGj7kIX5A hvZXYzKwfG5bE1EjXLAta1ljxjQeklM= X-Google-Smtp-Source: ABdhPJzpZxfRHottEpZ9d5GwVm9506hXq534rZPjEbGXBa8RDJWKwYDhT67CVFO8o5WFMJV0Y+w0WQ== X-Received: by 2002:adf:f946:: with SMTP id q6mr57368317wrr.283.1626090424310; Mon, 12 Jul 2021 04:47:04 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u16sm16847318wrw.36.2021.07.12.04.47.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:04 -0700 (PDT) Message-Id: <2d34ad3bc77a1a08e5f31545d4a345706541d700.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:46 +0000 Subject: [PATCH 06/19] [GSOC] ref-filter: use non-const ref_format in *_atom_parser() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use non-const ref_format in *_atom_parser(), which can help us modify the members of ref_format in *_atom_parser(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/tag.c | 2 +- ref-filter.c | 44 ++++++++++++++++++++++---------------------- ref-filter.h | 4 ++-- 3 files changed, 25 insertions(+), 25 deletions(-) diff --git a/builtin/tag.c b/builtin/tag.c index 82fcfc09824..452558ec957 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -146,7 +146,7 @@ static int verify_tag(const char *name, const char *ref, const struct object_id *oid, void *cb_data) { int flags; - const struct ref_format *format = cb_data; + struct ref_format *format = cb_data; flags = GPG_VERIFY_VERBOSE; if (format->format) diff --git a/ref-filter.c b/ref-filter.c index ba9ab35d7ec..c8e561a3687 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -226,7 +226,7 @@ static int strbuf_addf_ret(struct strbuf *sb, int ret, const char *fmt, ...) return ret; } -static int color_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int color_atom_parser(struct ref_format *format, struct used_atom *atom, const char *color_value, struct strbuf *err) { if (!color_value) @@ -264,7 +264,7 @@ static int refname_atom_parser_internal(struct refname_atom *atom, const char *a return 0; } -static int remote_ref_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int remote_ref_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct string_list params = STRING_LIST_INIT_DUP; @@ -311,7 +311,7 @@ static int remote_ref_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objecttype_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objecttype_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -323,7 +323,7 @@ static int objecttype_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int objectsize_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int objectsize_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -343,7 +343,7 @@ static int objectsize_atom_parser(const struct ref_format *format, struct used_a return 0; } -static int deltabase_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int deltabase_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -355,7 +355,7 @@ static int deltabase_atom_parser(const struct ref_format *format, struct used_at return 0; } -static int body_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int body_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (arg) @@ -364,7 +364,7 @@ static int body_atom_parser(const struct ref_format *format, struct used_atom *a return 0; } -static int subject_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int subject_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -376,7 +376,7 @@ static int subject_atom_parser(const struct ref_format *format, struct used_atom return 0; } -static int trailers_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int trailers_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { atom->u.contents.trailer_opts.no_divider = 1; @@ -402,7 +402,7 @@ static int trailers_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int contents_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int contents_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -430,7 +430,7 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato return 0; } -static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int raw_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -442,7 +442,7 @@ static int raw_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int oid_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -461,7 +461,7 @@ static int oid_atom_parser(const struct ref_format *format, struct used_atom *at return 0; } -static int person_email_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int person_email_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) @@ -475,7 +475,7 @@ static int person_email_atom_parser(const struct ref_format *format, struct used return 0; } -static int refname_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int refname_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { return refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err); @@ -492,7 +492,7 @@ static align_type parse_align_position(const char *s) return -1; } -static int align_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int align_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { struct align *align = &atom->u.align; @@ -544,7 +544,7 @@ static int align_atom_parser(const struct ref_format *format, struct used_atom * return 0; } -static int if_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int if_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { if (!arg) { @@ -559,7 +559,7 @@ static int if_atom_parser(const struct ref_format *format, struct used_atom *ato return 0; } -static int head_atom_parser(const struct ref_format *format, struct used_atom *atom, +static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { atom->u.head = resolve_refdup("HEAD", RESOLVE_REF_READING, NULL, NULL); @@ -570,7 +570,7 @@ static struct { const char *name; info_source source; cmp_type cmp_type; - int (*parser)(const struct ref_format *format, struct used_atom *atom, + int (*parser)(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err); } valid_atom[] = { [ATOM_REFNAME] = { "refname", SOURCE_NONE, FIELD_STR, refname_atom_parser }, @@ -649,7 +649,7 @@ struct atom_value { /* * Used to parse format string and sort specifiers */ -static int parse_ref_filter_atom(const struct ref_format *format, +static int parse_ref_filter_atom(struct ref_format *format, const char *atom, const char *ep, struct strbuf *err) { @@ -2554,9 +2554,9 @@ static void append_literal(const char *cp, const char *ep, struct ref_formatting } int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, - struct strbuf *final_buf, - struct strbuf *error_buf) + struct ref_format *format, + struct strbuf *final_buf, + struct strbuf *error_buf) { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; @@ -2601,7 +2601,7 @@ int format_ref_array_item(struct ref_array_item *info, } void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format) + struct ref_format *format) { struct ref_array_item *ref_item; struct strbuf output = STRBUF_INIT; diff --git a/ref-filter.h b/ref-filter.h index baf72a71896..74fb423fc89 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -116,7 +116,7 @@ void ref_array_sort(struct ref_sorting *sort, struct ref_array *array); void ref_sorting_set_sort_flags_all(struct ref_sorting *sorting, unsigned int mask, int on); /* Based on the given format and quote_style, fill the strbuf */ int format_ref_array_item(struct ref_array_item *info, - const struct ref_format *format, + struct ref_format *format, struct strbuf *final_buf, struct strbuf *error_buf); /* Parse a single sort specifier and add it to the list */ @@ -137,7 +137,7 @@ void setup_ref_filter_porcelain_msg(void); * name must be a fully qualified refname. */ void pretty_print_ref(const char *name, const struct object_id *oid, - const struct ref_format *format); + struct ref_format *format); /* * Push a single ref onto the array; this can be used to construct your own From patchwork Mon Jul 12 11:46:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0467C07E9C for ; Mon, 12 Jul 2021 11:47:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8D8BD610FA for ; Mon, 12 Jul 2021 11:47:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231499AbhGLLt6 (ORCPT ); Mon, 12 Jul 2021 07:49:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231340AbhGLLtz (ORCPT ); Mon, 12 Jul 2021 07:49:55 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39D73C0613E9 for ; Mon, 12 Jul 2021 04:47:06 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id l4-20020a05600c4f04b0290220f8455631so3687110wmq.1 for ; Mon, 12 Jul 2021 04:47:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Pq8IYJEyiWeBl0MKFag6xI7dzV+BVgtdzOmHeff1v6M=; b=UR+d6wyAlAVvxXi9GIJZLt4T96FeTPxR4oCdSoKzOL3nDcN81SRQx/GHtePetod7Jk J81BCGYfM6+oCoYaMPjw3NQu+a/Iylfe2osMHwPrBU0DHfhhpC1lFRcsK9MBqHo1Y/Rs kS01gPLZ0GAIOHzGLsyp7M8S1lwU+iMEFMjLkvS6o1mJuXqGSgggJ1HiAiRrMJptvFMN 9u6CXvaK3TXVObmc5jz5YOAue0uS3Oa5a3vRCVP3/nBBVVmDdCz01tN8d5WS6NMhYsCq V7Aedkd7L04b6Dtf6xC79/btkFHIyWKDqXDbgDr7diTmrS3VlgsYs9PzIdPD8+93Cz4N xncQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Pq8IYJEyiWeBl0MKFag6xI7dzV+BVgtdzOmHeff1v6M=; b=lYy8vl6/jrZnpOgQ7PHZCZcChaitVFUAu6qvIX+bGLPXCgoSN36ui3ro806eaoG3kB oMfgMTBayh/tlAToPQDugCW/zTQYyhvCVMeCqUt7QpO8WseotS537nvLVl/nCCME4hFi +ADOQPQ+9KieU6AFkKBrJweQNejHGnRpeiPfC5y9E3ZbuKC70QMKi3/IdVDKa8xNK9aH s6hOBhw/hCEg4WUHADGRPCdgXyjjArkFJhnGjI1PtCho+/Hcu3u7Y2ohTXmGCZOZIA6n xdOyx/hFesjr5DFNCh72DXgD1yYCkjKVxkT78kg/TlAL7yA7y4Rm/ZXYlMOm/MZINYgS wj/w== X-Gm-Message-State: AOAM533kA/JnqxoISPVFsZVOW3EM7vS5tcmf//s46wbzJJdVe9zWQbA+ e8LmlXYtt6OqR3XW54DwNfd7IGbcdsA= X-Google-Smtp-Source: ABdhPJx7Jaaw7T5CQZEnC8QEcUlZXS9lw8zLzGwm5yFYWUPU0BJYZQq8QKjapau4w8IfarsxCwOoZQ== X-Received: by 2002:a7b:cd15:: with SMTP id f21mr18554944wmj.148.1626090424852; Mon, 12 Jul 2021 04:47:04 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d24sm19694483wmb.42.2021.07.12.04.47.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:04 -0700 (PDT) Message-Id: <460c807c6ab9010ae166fe77e54d5435960f8e37.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:47 +0000 Subject: [PATCH 07/19] [GSOC] ref-filter: add %(rest) atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let "cat-file --batch=%(rest)" use the ref-filter interface, add %(rest) atom for ref-filter. "git for-each-ref", "git branch", "git tag" and "git verify-tag" will reject %(rest) by default. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 21 +++++++++++++++++++++ ref-filter.h | 5 ++++- t/t3203-branch-output.sh | 4 ++++ t/t6300-for-each-ref.sh | 4 ++++ t/t7004-tag.sh | 4 ++++ t/t7030-verify-tag.sh | 4 ++++ 6 files changed, 41 insertions(+), 1 deletion(-) diff --git a/ref-filter.c b/ref-filter.c index c8e561a3687..ee6c906f22d 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -157,6 +157,7 @@ enum atom_type { ATOM_IF, ATOM_THEN, ATOM_ELSE, + ATOM_REST, }; /* @@ -559,6 +560,15 @@ static int if_atom_parser(struct ref_format *format, struct used_atom *atom, return 0; } +static int rest_atom_parser(struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (arg) + return strbuf_addf_ret(err, -1, _("%%(rest) does not take arguments")); + format->use_rest = 1; + return 0; +} + static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *unused_err) { @@ -615,6 +625,7 @@ static struct { [ATOM_IF] = { "if", SOURCE_NONE, FIELD_STR, if_atom_parser }, [ATOM_THEN] = { "then", SOURCE_NONE }, [ATOM_ELSE] = { "else", SOURCE_NONE }, + [ATOM_REST] = { "rest", SOURCE_NONE, FIELD_STR, rest_atom_parser }, /* * Please update $__git_ref_fieldlist in git-completion.bash * when you add new atoms @@ -1010,6 +1021,9 @@ int verify_ref_format(struct ref_format *format) if (at < 0) die("%s", err.buf); + if (used_atom[at].atom_type == ATOM_REST) + die("this command reject atom %%(%.*s)", (int)(ep - sp - 2), sp + 2); + if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || format->quote_style == QUOTE_TCL) && @@ -1928,6 +1942,12 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->handler = else_atom_handler; v->s = xstrdup(""); continue; + } else if (atom_type == ATOM_REST) { + if (ref->rest) + v->s = xstrdup(ref->rest); + else + v->s = xstrdup(""); + continue; } else continue; @@ -2145,6 +2165,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); + ref->rest = NULL; return ref; } diff --git a/ref-filter.h b/ref-filter.h index 74fb423fc89..c15dee8d6b9 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -38,6 +38,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; + const char *rest; int flag; unsigned int kind; const char *symref; @@ -76,14 +77,16 @@ struct ref_format { * verify_ref_format() afterwards to finalize. */ const char *format; + const char *rest; int quote_style; + int use_rest; int use_color; /* Internal state to ref-filter */ int need_color_reset_at_eol; }; -#define REF_FORMAT_INIT { NULL, 0, -1 } +#define REF_FORMAT_INIT { .use_color = -1 } /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ diff --git a/t/t3203-branch-output.sh b/t/t3203-branch-output.sh index 5325b9f67a0..6e94c6db7b5 100755 --- a/t/t3203-branch-output.sh +++ b/t/t3203-branch-output.sh @@ -340,6 +340,10 @@ test_expect_success 'git branch --format option' ' test_cmp expect actual ' +test_expect_success 'git branch with --format=%(rest) must fail' ' + test_must_fail git branch --format="%(rest)" >actual +' + test_expect_success 'worktree colors correct' ' cat >expect <<-EOF && * (HEAD detached from fromtag) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 3d15d0a5360..0d2e062f791 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -1211,6 +1211,10 @@ test_expect_success 'basic atom: head contents:trailers' ' test_cmp expect actual.clean ' +test_expect_success 'basic atom: rest must fail' ' + test_must_fail git for-each-ref --format="%(rest)" refs/heads/main +' + test_expect_success 'trailer parsing not fooled by --- line' ' git commit --allow-empty -F - <<-\EOF && this is the subject diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh index 2f72c5c6883..082be85dffc 100755 --- a/t/t7004-tag.sh +++ b/t/t7004-tag.sh @@ -1998,6 +1998,10 @@ test_expect_success '--format should list tags as per format given' ' test_cmp expect actual ' +test_expect_success 'git tag -l with --format="%(rest)" must fail' ' + test_must_fail git tag -l --format="%(rest)" "v1*" +' + test_expect_success "set up color tests" ' echo "v1.0" >expect.color && echo "v1.0" >expect.bare && diff --git a/t/t7030-verify-tag.sh b/t/t7030-verify-tag.sh index 3cefde9602b..10faa645157 100755 --- a/t/t7030-verify-tag.sh +++ b/t/t7030-verify-tag.sh @@ -194,6 +194,10 @@ test_expect_success GPG 'verifying tag with --format' ' test_cmp expect actual ' +test_expect_success GPG 'verifying tag with --format="%(rest)" must fail' ' + test_must_fail git verify-tag --format="%(rest)" "fourth-signed" +' + test_expect_success GPG 'verifying a forged tag with --format should fail silently' ' test_must_fail git verify-tag --format="tagname : %(tag)" $(cat forged1.tag) >actual-forged && test_must_be_empty actual-forged From patchwork Mon Jul 12 11:46:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84EBDC07E99 for ; Mon, 12 Jul 2021 11:47:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7281061153 for ; Mon, 12 Jul 2021 11:47:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232020AbhGLLt7 (ORCPT ); Mon, 12 Jul 2021 07:49:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231224AbhGLLty (ORCPT ); Mon, 12 Jul 2021 07:49:54 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B91F7C0613EE for ; Mon, 12 Jul 2021 04:47:06 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id i94so25173386wri.4 for ; Mon, 12 Jul 2021 04:47:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=LqZxe2oaGgcckekUnkv6Q220abzHTuwYCh+rcznSRcU=; b=emm7hWbr0PT4PNq6RqmBJiXvQMLqmcZSXltDO5ZCxnrh6A6fB56KcS7YoGMgYKtprB tE0nmE4+0Jy3CMHGXod5kMyNluDZG9SLNk0dYWOUybqTVwzYYN4V7Anax7G1wYa7gPM5 Y32VWU+lmkMPXZfs+5fs7FoXGx4h1ZxOTTRDDFpYaiuOOTt8o5DFX3zKfzA68TSrFTOP ZNTREamtBaQtWxLN2TvKQEVF1jG9nMz1OPrsCL0rxP4teO01udbeWqs1sFCeBkY+X/Zs upv96SGjJLCcNx3cvjDXcBRzUvXw/qlFvyMk/7MzorwCGBBfJAzQum4RSSf32xJ6o168 eHqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=LqZxe2oaGgcckekUnkv6Q220abzHTuwYCh+rcznSRcU=; b=M7dhYCEfO13j8abebK1km9OQGZF/+v+K+hkGsPPB+G8CzhSDVqsxP14qmlb26v8AWw Ny1SSEpJ0LpcycV5YEaRpLKJfTBYw2LY4/Zf5cSTX5h4q4lQ/RY/V0KyQZiYeod/P4K8 4ATCNud9LBc6YxjzD4kcyDL+vsHj3BMlPGuTqD90ZolgULXI7Qz+c+A6CGu5VQqZHJDk PSJWDuMm+ff2RXjcFm1A7so/0RQD4DHu4RKe7nxminSnx5JGfS0I1a9P/BLgKP9SJxAI yo4G5lBXgzfAXqjgDGRjj88Qh0cTkfv0HHZG/8/0mG5vaUvEtc8O/mq7HN8F7AixWsYJ kZ4g== X-Gm-Message-State: AOAM532/fWjTqP2iymN6b2kEleYyMYWaXYZOWShLadMPPyfkILQAINA+ eOvqQTzqLd28wC9kSAMX0g7RGyk2RMg= X-Google-Smtp-Source: ABdhPJyMnAVMITUf1oaKT+FRlsJRAEyR/APp1iqnnPob5YMjS+vbZgJuXrx18i98AtImZcCXxkz/0g== X-Received: by 2002:a5d:408a:: with SMTP id o10mr19667461wrp.272.1626090425406; Mon, 12 Jul 2021 04:47:05 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 6sm1589747wmi.3.2021.07.12.04.47.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:05 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:48 +0000 Subject: [PATCH 08/19] [GSOC] ref-filter: pass get_object() return value to their callers MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because in the refactor of `git cat-file --batch` later, oid_object_info_extended() in get_object() will be used to obtain the info of an object with it's oid. When the object cannot be obtained in the git repository, `cat-file --batch` expects to output " missing" and continue the next oid query instead of letting Git exit. In other error conditions, Git should exit normally. So we can achieve this function by passing the return value of get_object(). Mentored-by: Christian Couder Mentored-by: Hariom Verma Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: ZheNing Hu --- ref-filter.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index ee6c906f22d..3189872188a 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1817,6 +1817,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) { struct object *obj; int i; + int ret; struct object_info empty = OBJECT_INFO_INIT; CALLOC_ARRAY(ref->value, used_atom_cnt); @@ -1973,8 +1974,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) oi.oid = ref->objectname; - if (get_object(ref, 0, &obj, &oi, err)) - return -1; + ret = get_object(ref, 0, &obj, &oi, err); + if (ret) + return ret; /* * If there is no atom that wants to know about tagged @@ -2006,8 +2008,10 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom, struct atom_value **v, struct strbuf *err) { if (!ref->value) { - if (populate_value(ref, err)) - return -1; + int ret = populate_value(ref, err); + + if (ret) + return ret; fill_missing_values(ref->value); } *v = &ref->value[atom]; @@ -2581,6 +2585,7 @@ int format_ref_array_item(struct ref_array_item *info, { const char *cp, *sp, *ep; struct ref_formatting_state state = REF_FORMATTING_STATE_INIT; + int ret; state.quote_style = format->quote_style; push_stack_element(&state.stack); @@ -2593,10 +2598,10 @@ int format_ref_array_item(struct ref_array_item *info, if (cp < sp) append_literal(cp, sp, &state); pos = parse_ref_filter_atom(format, sp + 2, ep, error_buf); - if (pos < 0 || get_ref_atom_value(info, pos, &atomv, error_buf) || + if (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) || atomv->handler(atomv, &state, error_buf)) { pop_stack_element(&state.stack); - return -1; + return ret ? ret : -1; } } if (*cp) { From patchwork Mon Jul 12 11:46:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95C2AC11F66 for ; Mon, 12 Jul 2021 11:47:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E88360C3F for ; Mon, 12 Jul 2021 11:47:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233028AbhGLLuE (ORCPT ); Mon, 12 Jul 2021 07:50:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231734AbhGLLt4 (ORCPT ); Mon, 12 Jul 2021 07:49:56 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D198C0613EF for ; Mon, 12 Jul 2021 04:47:07 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id j25-20020a05600c1c19b02902269686f585so1007681wms.0 for ; Mon, 12 Jul 2021 04:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=BXvbXQ0ubHB2seBzwtG+86JuqzyN8NpL5FHkziqcdKQ=; b=Ps1LFUwP0DoiZePhHfd+LOGue5Vg0eZQlI69+JsJNcjQU7bEGbc7wKv7mjKhetk0LF Zo9g9yt+dTUnd0hPYGfhakCel3kWlaqEkjp3h3XvBkwMajhescL9cBaamTt/IQAeIKVz 7qnu5EcFLiYiUat+3lKXDWhtSSL4fgACpOprMLv1oDWuMX6/bqvzDehXyhs+Y/Lf6DeA O7jQMFY6KQHJylURi4Aqd3puZFL4xWD2osikinVYdJjwo4DodApyJ6CUEtfH1FoSL9dO b8utYcXVtpkdTzw0KlB7tHBD/eww1+roPZrtuNcFUPrBRQf3jxXuxPhq+bxWXZ0XA8xo Ccdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=BXvbXQ0ubHB2seBzwtG+86JuqzyN8NpL5FHkziqcdKQ=; b=RjN6GZy1T4A7uGEwSDuGoTeVy9X3tPBF60NFob3ZrWsjaMfB0B5Iegeetahm47djxM KMLkFrEJ0MhrAfDC+oKYW2JsCgOYpA81LyaIXDnFYZ9zYz2CYQWiVtOp0761DzEKULQt eWQYol/N+27Tq7FO8i2LESYnvxxAQriSbF2lwxiv9O/o0mzwtlWIvMFtMVCl9/Pe7POW T89LJWpZUqMGlmADk+QzS3qPSB0AlPFyWEm7bGMkhUgF7GqXi/wBoy5a8t/7QLKgJwjP M0HADqATLR9B4J9f6QVspgfFNv3iN9A3Cd+EbJw4DQkzvD4dBS2WZ2becjyMZF5aYLx3 WlZA== X-Gm-Message-State: AOAM5339/dzePjnZn6oYjmVvXVkf86NA1fdAspsOlIPBlOZ2F8UTQVwF FJ1dnEsEa2f/Tu6Ued4dIXoZskkfBgk= X-Google-Smtp-Source: ABdhPJySlvMTQT+OqS/ZAfMi1Zaw/Xi8U6xz4zc16pHSRRuHiLz8jkHpbpY9un03aoQKEvn2rVHP+A== X-Received: by 2002:a05:600c:4f87:: with SMTP id n7mr55867658wmq.9.1626090426050; Mon, 12 Jul 2021 04:47:06 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r18sm7410810wrt.96.2021.07.12.04.47.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:05 -0700 (PDT) Message-Id: <6ad42c964053c9342554d9b43f8cbbb0e69385ad.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:49 +0000 Subject: [PATCH 09/19] [GSOC] ref-filter: introduce free_ref_array_item_value() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu When we use ref_array_item which is not dynamically allocated and want to free the space of its member "value" after the end of use, free_array_item() does not meet our needs, because it tries to free ref_array_item itself and its member "symref". Introduce free_ref_array_item_value() for freeing ref_array_item value. It will be called internally by free_array_item(), and it will help `cat-file --batch` free ref_array_item's value memory later. Helped-by: Junio C Hamano Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 11 ++++++++--- ref-filter.h | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 3189872188a..80b09fce1d5 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2292,16 +2292,21 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, return 0; } -/* Free memory allocated for a ref_array_item */ -static void free_array_item(struct ref_array_item *item) +void free_ref_array_item_value(struct ref_array_item *item) { - free((char *)item->symref); if (item->value) { int i; for (i = 0; i < used_atom_cnt; i++) free((char *)item->value[i].s); free(item->value); } +} + +/* Free memory allocated for a ref_array_item */ +static void free_array_item(struct ref_array_item *item) +{ + free((char *)item->symref); + free_ref_array_item_value(item); free(item); } diff --git a/ref-filter.h b/ref-filter.h index c15dee8d6b9..44e6dc05ac2 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -111,6 +111,8 @@ struct ref_format { int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type); /* Clear all memory allocated to ref_array */ void ref_array_clear(struct ref_array *array); +/* Free ref_array_item's value */ +void free_ref_array_item_value(struct ref_array_item *item); /* Used to verify if the given format is correct and to parse out the used atoms */ int verify_ref_format(struct ref_format *format); /* Sort the given ref_array as per the ref_sorting provided */ From patchwork Mon Jul 12 11:46:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08360C07E99 for ; Mon, 12 Jul 2021 11:47:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E7F3360C3F for ; Mon, 12 Jul 2021 11:47:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233117AbhGLLuI (ORCPT ); Mon, 12 Jul 2021 07:50:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230207AbhGLLt5 (ORCPT ); Mon, 12 Jul 2021 07:49:57 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED2A2C0613DD for ; Mon, 12 Jul 2021 04:47:07 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id f17so25204915wrt.6 for ; Mon, 12 Jul 2021 04:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=O8zaUk8FAh+xl/dCNXsBFsviaNIALe9L5MtO/rYzfcU=; b=Cb0XieVXRRiiURAsNGETXnV/LtwuYz1jfdabtLBstGxMJPy0XDnl1ooA/rP5WrzAkq GObyfER63Sp1PlxRSeIM/7yowSPtGJF2XeQNIh/n4Nj2x/J4rEGYs1kvWXxDvCCI8nbh jP+e6YR3YfZUS7HgJ/CZ7Pycr491TKo37KgM0q271AbO0Ig3+SJinfs1t7ux96siTy2p 3Ku04sW4NtWM/iNyVPX1U6Xj+8crjUsvVeVcPbJhgCwFvla9NIuzSVL0Qf+XBUlypAf3 gG1AHVVsGgNWDxKjWcwU6/dnoREBg5KFckXGGtq/dSWrXafLFe/6QnsPucwtT4HIWHhR QCOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=O8zaUk8FAh+xl/dCNXsBFsviaNIALe9L5MtO/rYzfcU=; b=tYXiyqLQSFJyjS6CRynMLF8euBNYqo6SvJbUD1yKlH1iec8gg2IG14hvI2caI9SZWC DJUvyzeguhvVN7SZ9lqFsQganxuMEUuJ+7EaIgXw93sryz4ni0asxPsf0Or/dlsYH62C 7n+XlA8BdGpzF+llsBshrXWnlu7UTWpmTdewF4ccWYW+eVjX6tKmw+EDLTZWGqgbPUc9 0q12vPFz9GxhoTP1tA/FUfOTbN6bR5eHFzy3BKEKh6wh4PYw+4reT07FXWNa8YA2piaf YuBO6sixkDf/vrtQhdfScn3bwmJXBBuI1IE7Yc3xnCl8t6tNPsy3EBhnkD7mBPf9hr8W i/UA== X-Gm-Message-State: AOAM530I9jgL172TZ0BfeF/3ymkLKgLqCejr3XaDLSaH+fzPA0KFZLGn TYZwbnt1zO1ki64KNcB6MnNO0TOc0AI= X-Google-Smtp-Source: ABdhPJyrKCqNRF9T6Wpzwq3K5eujkMBJVGoDV/9c33KClBHLE96bg39gwFQJKsNILqvv+QBRY8nULw== X-Received: by 2002:adf:d235:: with SMTP id k21mr25935491wrh.222.1626090426645; Mon, 12 Jul 2021 04:47:06 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 19sm2382632wmu.17.2021.07.12.04.47.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:06 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:50 +0000 Subject: [PATCH 10/19] [GSOC] ref-filter: introduce reject_atom() MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add `cat_file_mode` member in struct `ref_format` and introduce the function `reject_atom()`, when `cat-file --batch` use ref-filter logic later, it can help us reject atoms in verify_ref_format() which cat-file cannot use, e.g. `%(refname)`, `%(push)`, `%(upstream)`... or the atom `%(rest)` which for-each-ref family cannot use. Helped-by: Eric Sunshine Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Christian Couder Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 25 ++++++++++++++++++++++--- ref-filter.h | 1 + 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 80b09fce1d5..27199ba40f5 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1000,6 +1000,26 @@ static const char *find_next(const char *cp) return NULL; } +static int reject_atom(int cat_file_mode, enum atom_type atom_type) +{ + if (!cat_file_mode) + return atom_type == ATOM_REST; + + /* cat_file_mode */ + switch (atom_type) { + case ATOM_FLAG: + case ATOM_HEAD: + case ATOM_PUSH: + case ATOM_REFNAME: + case ATOM_SYMREF: + case ATOM_UPSTREAM: + case ATOM_WORKTREEPATH: + return 1; + default: + return 0; + } +} + /* * Make sure the format string is well formed, and parse out * the used atoms. @@ -1020,9 +1040,8 @@ int verify_ref_format(struct ref_format *format) at = parse_ref_filter_atom(format, sp + 2, ep, &err); if (at < 0) die("%s", err.buf); - - if (used_atom[at].atom_type == ATOM_REST) - die("this command reject atom %%(%.*s)", (int)(ep - sp - 2), sp + 2); + if (reject_atom(format->cat_file_mode, used_atom[at].atom_type)) + die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2); if ((format->quote_style == QUOTE_PYTHON || format->quote_style == QUOTE_SHELL || diff --git a/ref-filter.h b/ref-filter.h index 44e6dc05ac2..053980a6a42 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -78,6 +78,7 @@ struct ref_format { */ const char *format; const char *rest; + int cat_file_mode; int quote_style; int use_rest; int use_color; From patchwork Mon Jul 12 11:46:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A230C07E99 for ; Mon, 12 Jul 2021 11:47:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1EB3D610E6 for ; Mon, 12 Jul 2021 11:47:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232813AbhGLLuF (ORCPT ); Mon, 12 Jul 2021 07:50:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231898AbhGLLt4 (ORCPT ); Mon, 12 Jul 2021 07:49:56 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73F9EC0613E5 for ; Mon, 12 Jul 2021 04:47:08 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id j34so11193794wms.5 for ; Mon, 12 Jul 2021 04:47:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=6x+Lq6UZlHCJIcz9CvulEshnfXho4geSxyabj2Ck1uw=; b=lPUDzLaoJmMv8vUGWhsvvPLfWHl1zbuL2hLYcqQBcuYPxiLwiN1Lqd0tR+Qh+KXrQx 0MhqXraXoT7ncv2deOtIxhwGVhvOfj+mMlTfhsKmHwS/57w8XVQ6O1tamirnWoGJDvcK BOqiqvVo6jDSP8DS6AkKfjt1PJ3xr4qHgStCcUvGwFZxGa/WjDl3h6DUIFTu4KjYzoW+ 7EyHetWo+L9hUlf1YwNIQZvgTKix9AkPIceAtzb8lOxLhxWFO33+UBO11LGSH1M1tkzK /f6gXhMPGu+wiwCV9w/9Zb1Sd6e4ZsuA+cPklUqCAELYIHbWKHJkFIIfzISXjPZlzr/S BpKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=6x+Lq6UZlHCJIcz9CvulEshnfXho4geSxyabj2Ck1uw=; b=qkky2RGd9Yq5JiOIslYdvCO4bw1ky3gQUwJr6psR3POQvHdcEi4awDePvC43TExQml fLHRDiNdl+Gmiy51CZcXtterQPznV4Kd23ZvlhickRih1tGPXUSnUvoXsMj92SQlPPwc +tVV+s2qtH46ClqC8UXg0QzR+p2g4PoSeqTWf+cVjcDfNrFU+xsENsgJorrh3jsQKRvc PvAMs1Jc5AUt4utk4y8OhhpHNlYMCkORnIEbNmSZLAgU2vsAGAVpLNcaIjRdSnGecbiD A/pKACIEJit8iuw3WMV7Z/cwXGUiRiV4boAZHcI2+4AzKqFi4lvcqc9zQmuxAvfkPtoR DGSg== X-Gm-Message-State: AOAM531SQEHQQXAoVBH5rhE1IQ+4sQ5Qz7sPiNmc08EFVWsggSNqUb8b NZ8o6k7lX5oHMWB1B64yxWiTAiztkn4= X-Google-Smtp-Source: ABdhPJySmgYUI9nW0rIb3+cal4F2w9BRjnFHgIr+URadDNN/4gL8D1p2M6dRK6tW0DdPpPRAdFIXNg== X-Received: by 2002:a05:600c:198c:: with SMTP id t12mr53825747wmq.140.1626090427147; Mon, 12 Jul 2021 04:47:07 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t16sm14209016wrp.44.2021.07.12.04.47.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:06 -0700 (PDT) Message-Id: <790c558d7cc1f66049c1280e6dddbe194806a0dc.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:51 +0000 Subject: [PATCH 11/19] [GSOC] ref-filter: modify the error message and value in get_object Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Let get_object() return 1 and print " missing" instead of returning -1 and printing "missing object for " if oid_object_info_extended() unable to find the data corresponding to oid. When `cat-file --batch` use ref-filter logic later it can help `format_ref_array_item()` just report that the object is missing without letting Git exit. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 4 ++-- t/t6301-for-each-ref-errors.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 27199ba40f5..b4f41fec871 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1762,8 +1762,8 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj } if (oid_object_info_extended(the_repository, &oi->oid, &oi->info, OBJECT_INFO_LOOKUP_REPLACE)) - return strbuf_addf_ret(err, -1, _("missing object %s for %s"), - oid_to_hex(&oi->oid), ref->refname); + return strbuf_addf_ret(err, 1, _("%s missing"), + oid_to_hex(&oi->oid)); if (oi->info.disk_sizep && oi->disk_size < 0) BUG("Object size is less than zero."); diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh index 40edf9dab53..3553f84a00c 100755 --- a/t/t6301-for-each-ref-errors.sh +++ b/t/t6301-for-each-ref-errors.sh @@ -41,7 +41,7 @@ test_expect_success 'Missing objects are reported correctly' ' r=refs/heads/missing && echo $MISSING >.git/$r && test_when_finished "rm -f .git/$r" && - echo "fatal: missing object $MISSING for $r" >missing-err && + echo "fatal: $MISSING missing" >missing-err && test_must_fail git for-each-ref 2>err && test_cmp missing-err err && ( From patchwork Mon Jul 12 11:46:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 807C5C07E9C for ; Mon, 12 Jul 2021 11:47:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6B453610E6 for ; Mon, 12 Jul 2021 11:47:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232943AbhGLLuI (ORCPT ); Mon, 12 Jul 2021 07:50:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231406AbhGLLt5 (ORCPT ); Mon, 12 Jul 2021 07:49:57 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DC6EC0613E5 for ; Mon, 12 Jul 2021 04:47:09 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id d12so24560878wre.13 for ; Mon, 12 Jul 2021 04:47:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QNasbiGAZli473fXu4QgD2u/UTIEGvkgntmX6viQd9U=; b=IQK3Cl1VH98NOml/JwjA+MDB3BNs+jVcndE1MQ8qmsWmgxwnrIeOt9B9Ll2gVbqd6x PfjJGSfEhqlTlG30suSmx7cJKv1VXhZ32SMTtIJN6RB9/AO618IhbXrwPog+psRIaRxd jWrNsEUGM1wtI/mL9wAAcPkUHkq81vbZAc8Oh/DwQYl9vnYC3soPd31tW1qpek+1Ubx4 feY8VBIUyq85VsCKP3v6laoHQ7SIMhxUTQDtibsKsli3dc8CQR9KIssusYBclZlR6wAw p8fplC6/WlYXRZKs46sjM1tgUhs0zOYW5RHnGLU05gggl6ifXTliuSz4N5tKqiNJUMXj bBjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QNasbiGAZli473fXu4QgD2u/UTIEGvkgntmX6viQd9U=; b=iJZb/QC7u3SUp60AzOi7ohXFxwUptUjQzJSc1b5B8sL5lqGegisu5hIYeLurA7kLF0 E9gXcuk6vgMati5tuI8L8dVqoBt2hrV6MoSrARtVkJUo/yE3EVb+afYi4Y+OBg0aXeDr a+tYs7hktwOEasqdVs1Bo6Q3rLJV9dxh9Y0MRiPudIG0+uXNjjhjbowVk9FxEL7IParU uNjoQydIC1EUW83I01WiC6x62YHyFNNe0HSvYJWNa+7JykcFd/6btXgz9tqZJ9rtP1qs ZsI67yn4EpoCR/iv6Ws/T4GZlYvQ1HNFftY50OPlEA7R//xaJDGa6u2z2140HBgIgWWZ XmOA== X-Gm-Message-State: AOAM532y8ozimiDD5NX9tLM/dDFD6lNzzan7buhu+eemhFEwlyv5ifLd kDW5JvX0qAyJUc9xXkKIDwIuZFKEZ0E= X-Google-Smtp-Source: ABdhPJyC4IF18luIlMp9CoAerTMdHfNTnqmGFBSiq1QqASHjXro02fnzC7x1okFbDS66B4HHwanXGg== X-Received: by 2002:a05:6000:180b:: with SMTP id m11mr40650262wrh.6.1626090427736; Mon, 12 Jul 2021 04:47:07 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m5sm19073321wmq.2.2021.07.12.04.47.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:07 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:52 +0000 Subject: [PATCH 12/19] [GSOC] cat-file: add has_object_file() check Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Use `has_object_file()` in `batch_one_object()` to check whether the input object exists. This can help us reject the missing oid when we let `cat-file --batch` use ref-filter logic later. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 243fe6844bc..59a86412fd0 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -428,6 +428,13 @@ static void batch_one_object(const char *obj_name, return; } + if (!has_object_file(&data->oid)) { + printf("%s missing\n", + obj_name ? obj_name : oid_to_hex(&data->oid)); + fflush(stdout); + return; + } + batch_object_write(obj_name, scratch, opt, data); } From patchwork Mon Jul 12 11:46:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A120C07E99 for ; Mon, 12 Jul 2021 11:47:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E258360C3F for ; Mon, 12 Jul 2021 11:47:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232991AbhGLLuJ (ORCPT ); Mon, 12 Jul 2021 07:50:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231448AbhGLLt6 (ORCPT ); Mon, 12 Jul 2021 07:49:58 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97C72C0613DD for ; Mon, 12 Jul 2021 04:47:09 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id p8so25198441wrr.1 for ; Mon, 12 Jul 2021 04:47:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=wxw28PT0y1HD093q1CN4MSHFbLeN7akEybJmased0TU=; b=bR5t0U8kfFTuQXlj7u60MZnrmY7WA3DrM9E0RgvTHMCKxT55NYx3jvuVOyYguFXFpG WAcvE0YYSRBwVqDH0OoO9vlu+haIQFucgiyrqV4tegKrcz7LBg8imuPJrDVZMLTgoMJV nn0e2FoYdKex6XjqbOXgTJ5etzOWJttYor+c8SBpkAoABogexfAzqHFYDBlTGrRMaKml eEP7BAVL5Y15mNh915ezDsSp18XS4OvRQwZo+ut3rWJ4P4jx3uOFgFD5if8GlK2S6M/G XpnOrcgJ2SQkegfgLVnLbBKVdZgGVh+aRS+LlxVVMV0NVLF015IjyY3P49oBIfFnWk11 WFeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=wxw28PT0y1HD093q1CN4MSHFbLeN7akEybJmased0TU=; b=oheD+RJiIlI3d0lJOS+S23keUXG4Wp9Bmu6jbtgVdBgIxGtxM29E1+pNU5EuU/Wfxp ZcTGf4t6+67DnEGzAoZ7e+UyxeNoCxQzMHXUvQa2JXtTk7i7gKrmW99Gd4a5VsaEDGVL 1HV2SBVraopY9eca7yZy23Yw00Snu3HoQgpAs97i4g/6o1gfyRVZcuSYDhqRuXhoaTHZ QZhHqBOIjrr3n2Ddtr7gPdLQ1a42z4bOPdRWuTWnwfr+VHuDuRpea09/t1fQDnnJvmAZ 6y9MK8qz5T5bKXFZ1v6w+b5seW5dq0DhVZUoZcXzzbm+ofrf/qIxEO9BMMDXD6bHSCsb atMw== X-Gm-Message-State: AOAM532CEQ+B706xdJLcGzjUBRLG2MZswi3OrKh0QBnFDLxdtMucX3zJ wWJYRwkrNV7t2AX6wIwMCdit+6O8vWg= X-Google-Smtp-Source: ABdhPJy/tglofjRj9L5EZZDGFpNWZFAJNhXGCBAOqaHmWXZX0chql1SbxsAlsBI8jMAHCrZ/L5yLVw== X-Received: by 2002:adf:f592:: with SMTP id f18mr19601359wro.179.1626090428253; Mon, 12 Jul 2021 04:47:08 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f82sm18437452wmf.25.2021.07.12.04.47.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:07 -0700 (PDT) Message-Id: <43ce6bf26269dc80f87dabb52cbd2af63d24fd69.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:53 +0000 Subject: [PATCH 13/19] [GSOC] cat-file: change batch_objects parameter name Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because later cat-file reuses ref-filter logic that will add parameter "const struct option *options" to batch_objects(), the two synonymous parameters of "opt" and "options" may confuse readers, so change batch_options parameter of batch_objects() from "opt" to "batch". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 59a86412fd0..41d407638d5 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -495,7 +495,7 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *opt) +static int batch_objects(struct batch_options *batch) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; @@ -503,8 +503,8 @@ static int batch_objects(struct batch_options *opt) int save_warning; int retval = 0; - if (!opt->format) - opt->format = "%(objectname) %(objecttype) %(objectsize)"; + if (!batch->format) + batch->format = "%(objectname) %(objecttype) %(objectsize)"; /* * Expand once with our special mark_query flag, which will prime the @@ -513,20 +513,20 @@ static int batch_objects(struct batch_options *opt) */ memset(&data, 0, sizeof(data)); data.mark_query = 1; - strbuf_expand(&output, opt->format, expand_format, &data); + strbuf_expand(&output, batch->format, expand_format, &data); data.mark_query = 0; strbuf_release(&output); - if (opt->cmdmode) + if (batch->cmdmode) data.split_on_whitespace = 1; /* * If we are printing out the object, then always fill in the type, * since we will want to decide whether or not to stream. */ - if (opt->print_contents) + if (batch->print_contents) data.info.typep = &data.type; - if (opt->all_objects) { + if (batch->all_objects) { struct object_cb_data cb; struct object_info empty = OBJECT_INFO_INIT; @@ -536,11 +536,11 @@ static int batch_objects(struct batch_options *opt) if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); - cb.opt = opt; + cb.opt = batch; cb.expand = &data; cb.scratch = &output; - if (opt->unordered) { + if (batch->unordered) { struct oidset seen = OIDSET_INIT; cb.seen = &seen; @@ -590,7 +590,7 @@ static int batch_objects(struct batch_options *opt) data.rest = p; } - batch_one_object(input.buf, &output, opt, &data); + batch_one_object(input.buf, &output, batch, &data); } strbuf_release(&input); From patchwork Mon Jul 12 11:46:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 859E3C07E9C for ; Mon, 12 Jul 2021 11:47:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6436660C3F for ; Mon, 12 Jul 2021 11:47:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233265AbhGLLuO (ORCPT ); Mon, 12 Jul 2021 07:50:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232274AbhGLLuA (ORCPT ); Mon, 12 Jul 2021 07:50:00 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F51BC0613E5 for ; Mon, 12 Jul 2021 04:47:10 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id l18-20020a1ced120000b029014c1adff1edso14231608wmh.4 for ; Mon, 12 Jul 2021 04:47:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=JdMB+nC1v2FmjNWGvDZwXup1hlHUCHNCdjCOeatBTFA=; b=YoYXAjSLlaq2rYL2h8gDjeF+Q5Qr/i0OvE6Be1FXvkRIoOa1jbkpKMvL40O3jhBrDQ xhbdGRZzVfMlMZfAR0l89Tn5dNrSxTh3boayafgMib1CVWCnbW4RliXIkF/OFFcpj5hl 6m+t2VgGJG8aRGQdIOzSHbPKUUvGU28ugQOda+i4Iz/z2l8ogF4DUhMudHGMcamojN5D buBBD9eVckJpGIZX1Eau0YUijPh8Y3KQ6qSUUkNWelcHO0TId6hBWDms0nW+g+kQLdiP BGIjg+4pA7fxh31bkz6/o4sNZXlN2l+ktcemf1+avu/xnJ/cty+ctHopkECX/BVwrXLY cH7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=JdMB+nC1v2FmjNWGvDZwXup1hlHUCHNCdjCOeatBTFA=; b=eOUDtx1ZPOFI9JIBL646IfRyvSpoouTVsgWvxL6Ht4lK4t9kcpHJkXWnVhUMm2QVZy CvK3OU+IUcIDNmbbD+5PuIFuGJWVCDsDwjfgpuccxWQJqn2F53TEK1zMR0qikfBAvZJA wejJjf+TequBxfx4O4tXy6J67FfQWkWRGSSSSAmuX0hWhJGzuQI26AFP9VUjNux8wDTH E26RmgyopZBBK5qdOEG+ZInQO8SaxmRORiAq7rbefI72LxY9+01cNuxUZ6LMLDz+/iY9 eAS3yKeWm9eZZEIW3Jo0amL6Uvveehclaqahx5K2ffcCEQFtrE8qT9ymB+82OjkGr4vE +ewA== X-Gm-Message-State: AOAM531SRLF2Rw3ZivryGU4r0njqkFgLFReY7iKQCiY3WO+RWgIO7sY2 GM3pJdnjzKWhLn8ytfQtc/7xUhBv0rs= X-Google-Smtp-Source: ABdhPJwgtnolQ01W9KSfVeKTXcI+K7eAyH5U1s7K9AhydCzJgKm7WyNvQrkm5G5zRhm4iWNQW1YYzg== X-Received: by 2002:a05:600c:224a:: with SMTP id a10mr14330891wmm.7.1626090428893; Mon, 12 Jul 2021 04:47:08 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t9sm14019363wmq.14.2021.07.12.04.47.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:08 -0700 (PDT) Message-Id: <70e83e4ba3cc8a55bb8d90cb2c581cd04ac116d4.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:54 +0000 Subject: [PATCH 14/19] [GSOC] cat-file: reuse ref-filter logic MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu In order to let cat-file use ref-filter logic, let's do the following: 1. Change the type of member `format` in struct `batch_options` to `ref_format`, we will pass it to ref-filter later. 2. Let `batch_objects()` add atoms to format, and use `verify_ref_format()` to check atoms. 3. Use `format_ref_array_item()` in `batch_object_write()` to get the formatted data corresponding to the object. If the return value of `format_ref_array_item()` is equals to zero, use `batch_write()` to print object data; else if the return value is less than zero, use `die()` to print the error message and exit; else if return value is greater than zero, only print the error message, but don't exit. 4. Use free_ref_array_item_value() to free ref_array_item's value. Most of the atoms in `for-each-ref --format` are now supported, such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, `%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: `%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, `%(flag)`, `%(HEAD)`, because these atoms are unique to those objects that pointed to by a ref, "for-each-ref"'s family can naturally use these atoms, but not all objects are pointed to be a ref, so "cat-file" will not be able to use them. The performance for `git cat-file --batch-all-objects --batch-check` on the Git repository itself with performance testing tool `hyperfine` changes from 669.4 ms ± 31.1 ms to 1.134 s ± 0.063 s. The performance for `git cat-file --batch-all-objects --batch >/dev/null` on the Git repository itself with performance testing tool `time` change from "27.37s user 0.29s system 98% cpu 28.089 total" to "33.69s user 1.54s system 87% cpu 40.258 total". Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- Documentation/git-cat-file.txt | 6 + builtin/cat-file.c | 242 ++++++------------------------- t/t1006-cat-file.sh | 251 +++++++++++++++++++++++++++++++++ 3 files changed, 304 insertions(+), 195 deletions(-) diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index 4eb0421b3fd..ef8ab952b2f 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -226,6 +226,12 @@ newline. The available atoms are: after that first run of whitespace (i.e., the "rest" of the line) are output in place of the `%(rest)` atom. +Note that most of the atoms in `for-each-ref --format` are now supported, +such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`, +`%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected: +`%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`, +`%(flag)`, `%(HEAD)`. See linkgit:git-for-each-ref[1]. + If no format is specified, the default format is `%(objectname) %(objecttype) %(objectsize)`. diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 41d407638d5..5b163551fc6 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -16,6 +16,7 @@ #include "packfile.h" #include "object-store.h" #include "promisor-remote.h" +#include "ref-filter.h" struct batch_options { int enabled; @@ -25,7 +26,7 @@ struct batch_options { int all_objects; int unordered; int cmdmode; /* may be 'w' or 'c' for --filters or --textconv */ - const char *format; + struct ref_format format; }; static const char *force_path; @@ -195,99 +196,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct expand_data { struct object_id oid; - enum object_type type; - unsigned long size; - off_t disk_size; const char *rest; - struct object_id delta_base_oid; - - /* - * If mark_query is true, we do not expand anything, but rather - * just mark the object_info with items we wish to query. - */ - int mark_query; - - /* - * Whether to split the input on whitespace before feeding it to - * get_sha1; this is decided during the mark_query phase based on - * whether we have a %(rest) token in our format. - */ int split_on_whitespace; - - /* - * After a mark_query run, this object_info is set up to be - * passed to oid_object_info_extended. It will point to the data - * elements above, so you can retrieve the response from there. - */ - struct object_info info; - - /* - * This flag will be true if the requested batch format and options - * don't require us to call oid_object_info, which can then be - * optimized out. - */ - unsigned skip_object_info : 1; }; -static int is_atom(const char *atom, const char *s, int slen) -{ - int alen = strlen(atom); - return alen == slen && !memcmp(atom, s, alen); -} - -static void expand_atom(struct strbuf *sb, const char *atom, int len, - void *vdata) -{ - struct expand_data *data = vdata; - - if (is_atom("objectname", atom, len)) { - if (!data->mark_query) - strbuf_addstr(sb, oid_to_hex(&data->oid)); - } else if (is_atom("objecttype", atom, len)) { - if (data->mark_query) - data->info.typep = &data->type; - else - strbuf_addstr(sb, type_name(data->type)); - } else if (is_atom("objectsize", atom, len)) { - if (data->mark_query) - data->info.sizep = &data->size; - else - strbuf_addf(sb, "%"PRIuMAX , (uintmax_t)data->size); - } else if (is_atom("objectsize:disk", atom, len)) { - if (data->mark_query) - data->info.disk_sizep = &data->disk_size; - else - strbuf_addf(sb, "%"PRIuMAX, (uintmax_t)data->disk_size); - } else if (is_atom("rest", atom, len)) { - if (data->mark_query) - data->split_on_whitespace = 1; - else if (data->rest) - strbuf_addstr(sb, data->rest); - } else if (is_atom("deltabase", atom, len)) { - if (data->mark_query) - data->info.delta_base_oid = &data->delta_base_oid; - else - strbuf_addstr(sb, - oid_to_hex(&data->delta_base_oid)); - } else - die("unknown format element: %.*s", len, atom); -} - -static size_t expand_format(struct strbuf *sb, const char *start, void *data) -{ - const char *end; - - if (*start != '(') - return 0; - end = strchr(start + 1, ')'); - if (!end) - die("format element '%s' does not end in ')'", start); - - expand_atom(sb, start + 1, end - start - 1, data); - - return end - start + 1; -} - static void batch_write(struct batch_options *opt, const void *data, int len) { if (opt->buffer_output) { @@ -297,87 +209,34 @@ static void batch_write(struct batch_options *opt, const void *data, int len) write_or_die(1, data, len); } -static void print_object_or_die(struct batch_options *opt, struct expand_data *data) -{ - const struct object_id *oid = &data->oid; - - assert(data->info.typep); - - if (data->type == OBJ_BLOB) { - if (opt->buffer_output) - fflush(stdout); - if (opt->cmdmode) { - char *contents; - unsigned long size; - - if (!data->rest) - die("missing path for '%s'", oid_to_hex(oid)); - - if (opt->cmdmode == 'w') { - if (filter_object(data->rest, 0100644, oid, - &contents, &size)) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else if (opt->cmdmode == 'c') { - enum object_type type; - if (!textconv_object(the_repository, - data->rest, 0100644, oid, - 1, &contents, &size)) - contents = read_object_file(oid, - &type, - &size); - if (!contents) - die("could not convert '%s' %s", - oid_to_hex(oid), data->rest); - } else - BUG("invalid cmdmode: %c", opt->cmdmode); - batch_write(opt, contents, size); - free(contents); - } else { - stream_blob(oid); - } - } - else { - enum object_type type; - unsigned long size; - void *contents; - - contents = read_object_file(oid, &type, &size); - if (!contents) - die("object %s disappeared", oid_to_hex(oid)); - if (type != data->type) - die("object %s changed type!?", oid_to_hex(oid)); - if (data->info.sizep && size != data->size) - die("object %s changed size!?", oid_to_hex(oid)); - - batch_write(opt, contents, size); - free(contents); - } -} static void batch_object_write(const char *obj_name, struct strbuf *scratch, struct batch_options *opt, struct expand_data *data) { - if (!data->skip_object_info && - oid_object_info_extended(the_repository, &data->oid, &data->info, - OBJECT_INFO_LOOKUP_REPLACE) < 0) { - printf("%s missing\n", - obj_name ? obj_name : oid_to_hex(&data->oid)); - fflush(stdout); - return; - } + int ret; + struct strbuf err = STRBUF_INIT; + struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); - strbuf_expand(scratch, opt->format, expand_format, data); - strbuf_addch(scratch, '\n'); - batch_write(opt, scratch->buf, scratch->len); - if (opt->print_contents) { - print_object_or_die(opt, data); - batch_write(opt, "\n", 1); + ret = format_ref_array_item(&item, &opt->format, scratch, &err); + if (ret < 0) + die("%s\n", err.buf); + if (ret) { + /* ret > 0 means when the object corresponding to oid + * cannot be found in format_ref_array_item(), we only print + * the error message. + */ + printf("%s\n", err.buf); + fflush(stdout); + } else { + strbuf_addch(scratch, '\n'); + batch_write(opt, scratch->buf, scratch->len); } + free_ref_array_item_value(&item); + strbuf_release(&err); } static void batch_one_object(const char *obj_name, @@ -495,43 +354,37 @@ static int batch_unordered_packed(const struct object_id *oid, return batch_unordered_object(oid, data); } -static int batch_objects(struct batch_options *batch) +static const char * const cat_file_usage[] = { + N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), + N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), + NULL +}; + +static int batch_objects(struct batch_options *batch, const struct option *options) { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; int retval = 0; - if (!batch->format) - batch->format = "%(objectname) %(objecttype) %(objectsize)"; - - /* - * Expand once with our special mark_query flag, which will prime the - * object_info to be handed to oid_object_info_extended for each - * object. - */ memset(&data, 0, sizeof(data)); - data.mark_query = 1; - strbuf_expand(&output, batch->format, expand_format, &data); - data.mark_query = 0; - strbuf_release(&output); - if (batch->cmdmode) - data.split_on_whitespace = 1; - - /* - * If we are printing out the object, then always fill in the type, - * since we will want to decide whether or not to stream. - */ + if (batch->format.format) + strbuf_addstr(&format, batch->format.format); + else + strbuf_addstr(&format, "%(objectname) %(objecttype) %(objectsize)"); if (batch->print_contents) - data.info.typep = &data.type; + strbuf_addstr(&format, "\n%(raw)"); + batch->format.format = format.buf; + if (verify_ref_format(&batch->format)) + usage_with_options(cat_file_usage, options); + + if (batch->cmdmode || batch->format.use_rest) + data.split_on_whitespace = 1; if (batch->all_objects) { struct object_cb_data cb; - struct object_info empty = OBJECT_INFO_INIT; - - if (!memcmp(&data.info, &empty, sizeof(empty))) - data.skip_object_info = 1; if (has_promisor_remote()) warning("This repository uses promisor remotes. Some objects may not be loaded."); @@ -561,6 +414,7 @@ static int batch_objects(struct batch_options *batch) oid_array_clear(&sa); } + strbuf_release(&format); strbuf_release(&output); return 0; } @@ -593,18 +447,13 @@ static int batch_objects(struct batch_options *batch) batch_one_object(input.buf, &output, batch, &data); } + strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); warn_on_object_refname_ambiguity = save_warning; return retval; } -static const char * const cat_file_usage[] = { - N_("git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] "), - N_("git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]"), - NULL -}; - static int git_cat_file_config(const char *var, const char *value, void *cb) { if (userdiff_config(var, value) < 0) @@ -627,7 +476,7 @@ static int batch_option_callback(const struct option *opt, bo->enabled = 1; bo->print_contents = !strcmp(opt->long_name, "batch"); - bo->format = arg; + bo->format.format = arg; return 0; } @@ -636,7 +485,9 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) { int opt = 0; const char *exp_type = NULL, *obj_name = NULL; - struct batch_options batch = {0}; + struct batch_options batch = { + .format = REF_FORMAT_INIT + }; int unknown_type = 0; const struct option options[] = { @@ -675,6 +526,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) git_config(git_cat_file_config, NULL); batch.buffer_output = -1; + batch.format.cat_file_mode = 1; argc = parse_options(argc, argv, prefix, options, cat_file_usage, 0); if (opt) { @@ -718,7 +570,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) batch.buffer_output = batch.all_objects; if (batch.enabled) - return batch_objects(&batch); + return batch_objects(&batch, options); if (unknown_type && opt != 't' && opt != 's') die("git cat-file --allow-unknown-type: use with -s or -t"); diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 18b3779ccb6..7452404f24a 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -607,5 +607,256 @@ test_expect_success 'cat-file --batch="batman" with --batch-all-objects will wor git -C all-two cat-file --batch-all-objects --batch="batman" >actual && cmp expect actual ' +. "$TEST_DIRECTORY"/lib-gpg.sh +. "$TEST_DIRECTORY"/lib-terminal.sh + +test_expect_success 'cat-file --batch|--batch-check setup' ' + echo 1>blob1 && + printf "a\0b\0\c" >blob2 && + git add blob1 blob2 && + git commit -m "Commit Message" && + git branch -M main && + git tag -a -m "v0.0.0" testtag && + git update-ref refs/myblobs/blob1 HEAD:blob1 && + git update-ref refs/myblobs/blob2 HEAD:blob2 && + git update-ref refs/mytrees/tree1 HEAD^{tree} +' + +batch_test_atom() { + if test "$3" = "fail" + then + test_expect_${4:-success} $PREREQ "basic atom: $1 $2 must fail" " + test_must_fail git cat-file --batch-check='$2' >bad <<-EOF + $1 + EOF + " + else + test_expect_${4:-success} $PREREQ "basic atom: $1 $2" " + git for-each-ref --format='$2' $1 >expected && + git cat-file --batch-check='$2' >actual <<-EOF && + $1 + EOF + sanitize_pgp actual.clean && + cmp expected actual.clean + " + fi +} + +batch_test_atom refs/heads/main '%(refname)' fail +batch_test_atom refs/heads/main '%(refname:)' fail +batch_test_atom refs/heads/main '%(refname:short)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=2)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(refname:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(refname:strip=1)' fail +batch_test_atom refs/heads/main '%(refname:strip=2)' fail +batch_test_atom refs/heads/main '%(refname:strip=-1)' fail +batch_test_atom refs/heads/main '%(refname:strip=-2)' fail +batch_test_atom refs/heads/main '%(upstream)' fail +batch_test_atom refs/heads/main '%(upstream:short)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:lstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=2)' fail +batch_test_atom refs/heads/main '%(upstream:rstrip=-2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=2)' fail +batch_test_atom refs/heads/main '%(upstream:strip=-2)' fail +batch_test_atom refs/heads/main '%(push)' fail +batch_test_atom refs/heads/main '%(push:short)' fail +batch_test_atom refs/heads/main '%(push:lstrip=1)' fail +batch_test_atom refs/heads/main '%(push:lstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=1)' fail +batch_test_atom refs/heads/main '%(push:rstrip=-1)' fail +batch_test_atom refs/heads/main '%(push:strip=1)' fail +batch_test_atom refs/heads/main '%(push:strip=-1)' fail +batch_test_atom refs/heads/main '%(objecttype)' +batch_test_atom refs/heads/main '%(objectsize)' +batch_test_atom refs/heads/main '%(objectsize:disk)' +batch_test_atom refs/heads/main '%(deltabase)' +batch_test_atom refs/heads/main '%(objectname)' +batch_test_atom refs/heads/main '%(objectname:short)' +batch_test_atom refs/heads/main '%(objectname:short=1)' +batch_test_atom refs/heads/main '%(objectname:short=10)' +batch_test_atom refs/heads/main '%(tree)' +batch_test_atom refs/heads/main '%(tree:short)' +batch_test_atom refs/heads/main '%(tree:short=1)' +batch_test_atom refs/heads/main '%(tree:short=10)' +batch_test_atom refs/heads/main '%(parent)' +batch_test_atom refs/heads/main '%(parent:short)' +batch_test_atom refs/heads/main '%(parent:short=1)' +batch_test_atom refs/heads/main '%(parent:short=10)' +batch_test_atom refs/heads/main '%(numparent)' +batch_test_atom refs/heads/main '%(object)' +batch_test_atom refs/heads/main '%(type)' +batch_test_atom refs/heads/main '%(raw)' +batch_test_atom refs/heads/main '%(*objectname)' +batch_test_atom refs/heads/main '%(*objecttype)' +batch_test_atom refs/heads/main '%(author)' +batch_test_atom refs/heads/main '%(authorname)' +batch_test_atom refs/heads/main '%(authoremail)' +batch_test_atom refs/heads/main '%(authoremail:trim)' +batch_test_atom refs/heads/main '%(authoremail:localpart)' +batch_test_atom refs/heads/main '%(authordate)' +batch_test_atom refs/heads/main '%(committer)' +batch_test_atom refs/heads/main '%(committername)' +batch_test_atom refs/heads/main '%(committeremail)' +batch_test_atom refs/heads/main '%(committeremail:trim)' +batch_test_atom refs/heads/main '%(committeremail:localpart)' +batch_test_atom refs/heads/main '%(committerdate)' +batch_test_atom refs/heads/main '%(tag)' +batch_test_atom refs/heads/main '%(tagger)' +batch_test_atom refs/heads/main '%(taggername)' +batch_test_atom refs/heads/main '%(taggeremail)' +batch_test_atom refs/heads/main '%(taggeremail:trim)' +batch_test_atom refs/heads/main '%(taggeremail:localpart)' +batch_test_atom refs/heads/main '%(taggerdate)' +batch_test_atom refs/heads/main '%(creator)' +batch_test_atom refs/heads/main '%(creatordate)' +batch_test_atom refs/heads/main '%(subject)' +batch_test_atom refs/heads/main '%(subject:sanitize)' +batch_test_atom refs/heads/main '%(contents:subject)' +batch_test_atom refs/heads/main '%(body)' +batch_test_atom refs/heads/main '%(contents:body)' +batch_test_atom refs/heads/main '%(contents:signature)' +batch_test_atom refs/heads/main '%(contents)' +batch_test_atom refs/heads/main '%(HEAD)' fail +batch_test_atom refs/heads/main '%(upstream:track)' fail +batch_test_atom refs/heads/main '%(upstream:trackshort)' fail +batch_test_atom refs/heads/main '%(upstream:track,nobracket)' fail +batch_test_atom refs/heads/main '%(upstream:nobracket,track)' fail +batch_test_atom refs/heads/main '%(push:track)' fail +batch_test_atom refs/heads/main '%(push:trackshort)' fail +batch_test_atom refs/heads/main '%(worktreepath)' fail +batch_test_atom refs/heads/main '%(symref)' fail +batch_test_atom refs/heads/main '%(flag)' fail + +batch_test_atom refs/tags/testtag '%(refname)' fail +batch_test_atom refs/tags/testtag '%(refname:short)' fail +batch_test_atom refs/tags/testtag '%(upstream)' fail +batch_test_atom refs/tags/testtag '%(push)' fail +batch_test_atom refs/tags/testtag '%(objecttype)' +batch_test_atom refs/tags/testtag '%(objectsize)' +batch_test_atom refs/tags/testtag '%(objectsize:disk)' +batch_test_atom refs/tags/testtag '%(*objectsize:disk)' +batch_test_atom refs/tags/testtag '%(deltabase)' +batch_test_atom refs/tags/testtag '%(*deltabase)' +batch_test_atom refs/tags/testtag '%(objectname)' +batch_test_atom refs/tags/testtag '%(objectname:short)' +batch_test_atom refs/tags/testtag '%(tree)' +batch_test_atom refs/tags/testtag '%(tree:short)' +batch_test_atom refs/tags/testtag '%(tree:short=1)' +batch_test_atom refs/tags/testtag '%(tree:short=10)' +batch_test_atom refs/tags/testtag '%(parent)' +batch_test_atom refs/tags/testtag '%(parent:short)' +batch_test_atom refs/tags/testtag '%(parent:short=1)' +batch_test_atom refs/tags/testtag '%(parent:short=10)' +batch_test_atom refs/tags/testtag '%(numparent)' +batch_test_atom refs/tags/testtag '%(object)' +batch_test_atom refs/tags/testtag '%(type)' +batch_test_atom refs/tags/testtag '%(*objectname)' +batch_test_atom refs/tags/testtag '%(*objecttype)' +batch_test_atom refs/tags/testtag '%(author)' +batch_test_atom refs/tags/testtag '%(authorname)' +batch_test_atom refs/tags/testtag '%(authoremail)' +batch_test_atom refs/tags/testtag '%(authoremail:trim)' +batch_test_atom refs/tags/testtag '%(authoremail:localpart)' +batch_test_atom refs/tags/testtag '%(authordate)' +batch_test_atom refs/tags/testtag '%(committer)' +batch_test_atom refs/tags/testtag '%(committername)' +batch_test_atom refs/tags/testtag '%(committeremail)' +batch_test_atom refs/tags/testtag '%(committeremail:trim)' +batch_test_atom refs/tags/testtag '%(committeremail:localpart)' +batch_test_atom refs/tags/testtag '%(committerdate)' +batch_test_atom refs/tags/testtag '%(tag)' +batch_test_atom refs/tags/testtag '%(tagger)' +batch_test_atom refs/tags/testtag '%(taggername)' +batch_test_atom refs/tags/testtag '%(taggeremail)' +batch_test_atom refs/tags/testtag '%(taggeremail:trim)' +batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' +batch_test_atom refs/tags/testtag '%(taggerdate)' +batch_test_atom refs/tags/testtag '%(creator)' +batch_test_atom refs/tags/testtag '%(creatordate)' +batch_test_atom refs/tags/testtag '%(subject)' +batch_test_atom refs/tags/testtag '%(subject:sanitize)' +batch_test_atom refs/tags/testtag '%(contents:subject)' +batch_test_atom refs/tags/testtag '%(body)' +batch_test_atom refs/tags/testtag '%(contents:body)' +batch_test_atom refs/tags/testtag '%(contents:signature)' +batch_test_atom refs/tags/testtag '%(contents)' +batch_test_atom refs/tags/testtag '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(refname)' fail +batch_test_atom refs/myblobs/blob1 '%(upstream)' fail +batch_test_atom refs/myblobs/blob1 '%(push)' fail +batch_test_atom refs/myblobs/blob1 '%(HEAD)' fail + +batch_test_atom refs/myblobs/blob1 '%(objectname)' +batch_test_atom refs/myblobs/blob1 '%(objecttype)' +batch_test_atom refs/myblobs/blob1 '%(objectsize)' +batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)' +batch_test_atom refs/myblobs/blob1 '%(deltabase)' + +batch_test_atom refs/myblobs/blob1 '%(contents)' +batch_test_atom refs/myblobs/blob2 '%(contents)' + +batch_test_atom refs/myblobs/blob1 '%(raw)' +batch_test_atom refs/myblobs/blob2 '%(raw)' +batch_test_atom refs/mytrees/tree1 '%(raw)' + +batch_test_atom refs/myblobs/blob1 '%(raw:size)' +batch_test_atom refs/myblobs/blob2 '%(raw:size)' +batch_test_atom refs/mytrees/tree1 '%(raw:size)' + +batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' +batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' + +batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' +batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' + +batch_test_atom refs/heads/main 'VALID' +batch_test_atom refs/heads/main '%(INVALID)' fail +batch_test_atom refs/heads/main '%(authordate:INVALID)' fail + +test_expect_success '%(rest) works with both a branch and a tag' ' + cat >expected <<-EOF && + 123 commit 123 + 456 tag 456 + EOF + git cat-file --batch-check="%(rest) %(objecttype) %(rest)" >actual <<-EOF && + refs/heads/main 123 + refs/tags/testtag 456 + EOF + test_cmp expected actual +' + +batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize) +%(raw)' +batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize) +%(raw)' + + +test_expect_success 'cat-file --batch equals to --batch-check with atoms' ' + git cat-file --batch-check="%(objectname) %(objecttype) %(objectsize) +%(raw)" >expected <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + git cat-file --batch >actual <<-EOF && + refs/heads/main + refs/tags/testtag + EOF + cmp expected actual +' test_done From patchwork Mon Jul 12 11:46:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F07DC07E9C for ; Mon, 12 Jul 2021 11:47:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3AC78610CD for ; Mon, 12 Jul 2021 11:47:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233214AbhGLLuK (ORCPT ); Mon, 12 Jul 2021 07:50:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231340AbhGLLt7 (ORCPT ); Mon, 12 Jul 2021 07:49:59 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA82BC0613E9 for ; Mon, 12 Jul 2021 04:47:10 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id h18-20020a05600c3512b029020e4ceb9588so14256160wmq.5 for ; Mon, 12 Jul 2021 04:47:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=qAwXHcrowoAV1ZvkfF6sVHaTDEdBOu0rUNJ1Ia1vdCk=; b=D5Gt5+nH2f4o6IBslgQxVSBscszv2g02m4voQuoDgWaotWkyAc4C9vWwt3FFqm4umo bgtpCa/hCsRMEwULbz0EHIdJe0wDEW27jAsVtItj4NLAcUIUWzMCMFCgt2InJiaxmHh0 ITX4ATBuh4fXzgNoVSg0A9uhlwCNR5bWkNqKF+4dJsmGiR+0PZBQ3Lbk4kgZ7iUTyl8J yuHI+Vt18do+F4AgSpjb4zJry5+6QiBEEmchM/xDCI3BDc4lzezLP055b6e6+vAPTyFy HnVxCoaAFLcqICvdp482JmcNm6pYxwvSNK4I4lo0bAouqulRz2CxLqLbG6/CpI3sE8ds Bo4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=qAwXHcrowoAV1ZvkfF6sVHaTDEdBOu0rUNJ1Ia1vdCk=; b=mUoF0Cq5WUz4M3vQlUJBbq+UVzcGSpo28lXmxC/wt6xBC4fU7Z+3b0NogloFLapLYt +H2mjFhwN6mov3FFL6ZHeeg9TLmTVPlGgAE0W2b3KuRi+P9Sh222n+5qMU75pHPLtnCF +lc4HJyFkUwW4AufkWkUXeTU4TgU/nNatM2Ys4syaa/4u9MeINhAAyh7ausXNb244vIn kz4gUD+J1KfRGpVkFJl/m0uDgKShVTUMNtQD55TH5xOkN0QFwx0UE1ebH9PTKBt6GzzK 3hGwnkcuB54kC9sYS7mPdMMQle1nzBqHiIsPM2Nt5qoZ+NLA7sWQaQHJsOple8rQ3R9v rNrA== X-Gm-Message-State: AOAM531xt4cR8IrI7Gano0jdUG3yEu4D3SdVallWRY/CWEQLywQc+zE4 JiXuMpyTaRZVyVKCcnTXZMeu3bezv4M= X-Google-Smtp-Source: ABdhPJyoMD8yQVtvEtm5uPaQMW4t8dXR3L28H0E7uZJAN/6hIPru9V64bQm/PQs1ygZ7pyCCdw0epg== X-Received: by 2002:a1c:143:: with SMTP id 64mr16194902wmb.187.1626090429433; Mon, 12 Jul 2021 04:47:09 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x1sm21143924wmc.0.2021.07.12.04.47.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:09 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:55 +0000 Subject: [PATCH 15/19] [GSOC] cat-file: reuse err buf in batch_object_write() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Reuse the `err` buffer in batch_object_write(), as the buffer `scratch` does. This will reduce the overhead of multiple allocations of memory of the err buffer. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 5b163551fc6..dc604a9879d 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -212,35 +212,36 @@ static void batch_write(struct batch_options *opt, const void *data, int len) static void batch_object_write(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { int ret; - struct strbuf err = STRBUF_INIT; struct ref_array_item item = { data->oid, data->rest }; strbuf_reset(scratch); + strbuf_reset(err); - ret = format_ref_array_item(&item, &opt->format, scratch, &err); + ret = format_ref_array_item(&item, &opt->format, scratch, err); if (ret < 0) - die("%s\n", err.buf); + die("%s\n", err->buf); if (ret) { /* ret > 0 means when the object corresponding to oid * cannot be found in format_ref_array_item(), we only print * the error message. */ - printf("%s\n", err.buf); + printf("%s\n", err->buf); fflush(stdout); } else { strbuf_addch(scratch, '\n'); batch_write(opt, scratch->buf, scratch->len); } free_ref_array_item_value(&item); - strbuf_release(&err); } static void batch_one_object(const char *obj_name, struct strbuf *scratch, + struct strbuf *err, struct batch_options *opt, struct expand_data *data) { @@ -294,7 +295,7 @@ static void batch_one_object(const char *obj_name, return; } - batch_object_write(obj_name, scratch, opt, data); + batch_object_write(obj_name, scratch, err, opt, data); } struct object_cb_data { @@ -302,13 +303,14 @@ struct object_cb_data { struct expand_data *expand; struct oidset *seen; struct strbuf *scratch; + struct strbuf *err; }; static int batch_object_cb(const struct object_id *oid, void *vdata) { struct object_cb_data *data = vdata; oidcpy(&data->expand->oid, oid); - batch_object_write(NULL, data->scratch, data->opt, data->expand); + batch_object_write(NULL, data->scratch, data->err, data->opt, data->expand); return 0; } @@ -364,6 +366,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio { struct strbuf input = STRBUF_INIT; struct strbuf output = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; struct strbuf format = STRBUF_INIT; struct expand_data data; int save_warning; @@ -392,6 +395,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio cb.opt = batch; cb.expand = &data; cb.scratch = &output; + cb.err = &err; if (batch->unordered) { struct oidset seen = OIDSET_INIT; @@ -416,6 +420,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio strbuf_release(&format); strbuf_release(&output); + strbuf_release(&err); return 0; } @@ -444,12 +449,13 @@ static int batch_objects(struct batch_options *batch, const struct option *optio data.rest = p; } - batch_one_object(input.buf, &output, batch, &data); + batch_one_object(input.buf, &output, &err, batch, &data); } strbuf_release(&format); strbuf_release(&input); strbuf_release(&output); + strbuf_release(&err); warn_on_object_refname_ambiguity = save_warning; return retval; } From patchwork Mon Jul 12 11:46:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 644C1C07E99 for ; Mon, 12 Jul 2021 11:47:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4F6C9610CD for ; Mon, 12 Jul 2021 11:47:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233253AbhGLLuM (ORCPT ); Mon, 12 Jul 2021 07:50:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231326AbhGLLt7 (ORCPT ); Mon, 12 Jul 2021 07:49:59 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57108C0613DD for ; Mon, 12 Jul 2021 04:47:11 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id g8-20020a1c9d080000b02901f13dd1672aso9637797wme.0 for ; Mon, 12 Jul 2021 04:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=2wAhzEz1dqFEFD7ZYTaOKtiC0HQ3bg2FSnbAhq07rRM=; b=lZSNedqBeFR+oswlDJGWZwI1XgE64xoC38lrK9ixAfTEDv0BedVCpZXu3OZk+b1WiR DcqZ2PCTQ5B8+eFBqVthkP1gzbC3XY40p+T6qcnprzvIkBWXnXC11dQ4dDbVjkfu1Xz2 8mPaYIUJ/5h0zFQwhoySZ7vfIJsOtPpTFhIXZ4+rfi8ipr6dgjT7CBY36oHmoKQBnJyF 0mwNys31jY8bd7R8tGfKTfroBZbd3YB9wA/BuWe840NxrDp2KDdyf++NHcdr3m/ahblj vbkpcAtNKwuKPDF3STpnEYnpyKG9Asikki21HeEuynaK+nuSeQ7yZgeNDfWbOlAzCEqM 5BaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=2wAhzEz1dqFEFD7ZYTaOKtiC0HQ3bg2FSnbAhq07rRM=; b=HsLxuSF/V7fj6RVRuf6yNUjLjSVF5oGPypAkafJ9QjZqVCNLMshleRVSHo6baIxJRq QG9UFZ9tnl4bXy98E7yy7w6ss9Zuh020O7xGn+FBJOGTk4aTc2KqAVCvOul4+1G3jW4R wUfxaENXQ2ip9thQwVi7SCAQk4K94nWpKCv19dTAbC7u7AM2kZnquSI/eeembvZ2C1QZ OoneolqmarhXxtF8URI8Y5tvgorMkGsSenXyBm9M8XU6xbP+w91NPDDaujD/5ac4V5e5 kJZ4FwysmDsMl+YLURddajEcDAvT1yRw/hK2gXGomTnqEiHwt090jFWnG7D9NFTxmFPQ m6ug== X-Gm-Message-State: AOAM5310UXxfgvKzgQxw35l3/Jp+zznu4v+o75DFC2OUlFrLmvSnjJ9M noH6okPh6QCvCgd2SCbLxcXff/ZFzbk= X-Google-Smtp-Source: ABdhPJxStX9rfqqdrn6xXfXjOkRvVqxlvSBh0dtFbViPBiP+GwkyRsnQ4uctovNCowNnXTEw783TpQ== X-Received: by 2002:a7b:c4d3:: with SMTP id g19mr12796631wmk.78.1626090430018; Mon, 12 Jul 2021 04:47:10 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id w22sm19908909wmc.4.2021.07.12.04.47.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:09 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:56 +0000 Subject: [PATCH 16/19] [GSOC] cat-file: re-implement --textconv, --filters options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu After cat-file reuses the ref-filter logic, we re-implement the functions of --textconv and --filters options. Add members `cat_file_cmdmode` to struct `ref_array_item`, so that struct `batch_option` member `cmdmode` will be passed to ref-filter, and then ref-filter will take use of it to filter the content of the object in get_object(). Use `actual_oi` to record the real expand_data: it may point to the original `oi` or the `act_oi` processed by `textconv_object()` or `convert_to_working_tree()`. `grab_values()` will grab the contents of `actual_oi` and `grab_common_values()` to grab the contents of origin `oi`, this ensures that `%(objectsize)` still uses the size of the unfiltered data. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 2 +- ref-filter.c | 35 +++++++++++++++++++++++++++++++++-- ref-filter.h | 1 + 3 files changed, 35 insertions(+), 3 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index dc604a9879d..3a6153e778f 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -217,7 +217,7 @@ static void batch_object_write(const char *obj_name, struct expand_data *data) { int ret; - struct ref_array_item item = { data->oid, data->rest }; + struct ref_array_item item = { data->oid, data->rest, opt->cmdmode }; strbuf_reset(scratch); strbuf_reset(err); diff --git a/ref-filter.c b/ref-filter.c index b4f41fec871..91e26c9aba3 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1,3 +1,4 @@ +#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "cache.h" #include "parse-options.h" @@ -1755,6 +1756,9 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj { /* parse_object_buffer() will set eaten to 0 if free() will be needed */ int eaten = 1; + struct expand_data *actual_oi = oi; + struct expand_data act_oi = {0}; + if (oi->info.contentp) { /* We need to know that to use parse_object_buffer properly */ oi->info.sizep = &oi->size; @@ -1768,19 +1772,45 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj BUG("Object size is less than zero."); if (oi->info.contentp) { - *obj = parse_object_buffer(the_repository, &oi->oid, oi->type, oi->size, oi->content, &eaten); + if ((ref->cat_file_cmdmode == 'c' || ref->cat_file_cmdmode == 'w') && !ref->rest) + return strbuf_addf_ret(err, -1, _("missing path for '%s'"), + oid_to_hex(&act_oi.oid)); + if (oi->type == OBJ_BLOB) { + if (ref->cat_file_cmdmode == 'c') { + act_oi = *oi; + if (textconv_object(the_repository, + ref->rest, 0100644, &act_oi.oid, + 1, (char **)(&act_oi.content), &act_oi.size)) + actual_oi = &act_oi; + } else if (ref->cat_file_cmdmode == 'w') { + struct strbuf strbuf = STRBUF_INIT; + struct checkout_metadata meta; + act_oi = *oi; + + init_checkout_metadata(&meta, NULL, NULL, &act_oi.oid); + if (!convert_to_working_tree(&the_index, ref->rest, act_oi.content, act_oi.size, &strbuf, &meta)) + die("could not convert '%s' %s", + oid_to_hex(&oi->oid), ref->rest); + act_oi.size = strbuf.len; + act_oi.content = strbuf_detach(&strbuf, NULL); + actual_oi = &act_oi; + } + } + *obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten); if (!*obj) { if (!eaten) free(oi->content); return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"), oid_to_hex(&oi->oid), ref->refname); } - grab_values(ref->value, deref, *obj, oi); + grab_values(ref->value, deref, *obj, actual_oi); } grab_common_values(ref->value, deref, oi); if (!eaten) free(oi->content); + if (actual_oi != oi) + free(actual_oi->content); return 0; } @@ -2189,6 +2219,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname, FLEX_ALLOC_STR(ref, refname, refname); oidcpy(&ref->objectname, oid); ref->rest = NULL; + ref->cat_file_cmdmode = 0; return ref; } diff --git a/ref-filter.h b/ref-filter.h index 053980a6a42..a93d5e4dd61 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -39,6 +39,7 @@ struct ref_sorting { struct ref_array_item { struct object_id objectname; const char *rest; + int cat_file_cmdmode; int flag; unsigned int kind; const char *symref; From patchwork Mon Jul 12 11:46:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04623C11F66 for ; Mon, 12 Jul 2021 11:47:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E07B460C3F for ; Mon, 12 Jul 2021 11:47:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233166AbhGLLuP (ORCPT ); Mon, 12 Jul 2021 07:50:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232342AbhGLLuA (ORCPT ); Mon, 12 Jul 2021 07:50:00 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7608C0613DD for ; Mon, 12 Jul 2021 04:47:11 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id b14-20020a1c1b0e0000b02901fc3a62af78so14216020wmb.3 for ; Mon, 12 Jul 2021 04:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=4Ib0SWYpLaPximun7MUajcXT3GjLWT38oNkCGu16y08=; b=Wp91Q2ZutHEIhigywgA03dN5p+wsX3dljb+6whNxmip751D1hOJrF2Nicle1AdYUpq SFckHELJaVKch6WfwqZ7KB/+jdHdAVIyh+scG1zxGe81dkEXKxYPDGqIGV50zuKyf+sy ZBiCuHJrNvgLSZpNn6SDVb5hgImNuT1lB3PGo1n/okGdhzH88NspwJmChMuw7pS9LYLS wpERxtOnKfM75thswI7KgGby7gpRegeXOunQZ6i4rc7KJYYPlUjwZM4IeFasCBDgjVWE tCJlizb2PznMg4jFSBmXEDrjmhyrk1TYa6QkbBJX6vEGEYImgmWSlD+gk5xquoJxprZW Pacw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=4Ib0SWYpLaPximun7MUajcXT3GjLWT38oNkCGu16y08=; b=d4CSjIwxfLHUylZDm3cceSSVbTW69fcSdI6C3rUy4Q2JGr19MQfUSqRtLLcb6k6FHl I8CEc22lWdLgasAlE2dp0SFt8DsYWg6nlLaycVRlyUFBKHb2GADQUqvzxkUDtCY7GFql iZkZsBUWslws18dM6JbYGReow90Z6wZEm8ZNjyEbsYcC3WOEriF6C6fYxgXRnWPb4xfs 5ORiG5zhYY7I1za758GTjyRgovF5A52qFh/jHdLqLZvUJdRUhCFRotHh6Z7yLXO5Rff0 37LuWYHHExNjP8VNwjJWR4CjcomG/CvFVDepK5JMUomUMaXRSBpLaNFYfFtxnoZrpbA9 Z3+g== X-Gm-Message-State: AOAM5328hrLOW1LJd9aZYh2CR7hr/Ogda0iw/bAomi9SbE/8K8rFlmAJ 1vP8ypP16qtr09zSYy/h3iWs2UF4bGE= X-Google-Smtp-Source: ABdhPJzB6alkGt6JHlaY2TLQ+imo5jwNeawPtVhi0IbG/8ygam9/6WOSPbS4VF0ABdqdpGaZZZFgfg== X-Received: by 2002:a05:600c:3644:: with SMTP id y4mr14323803wmq.85.1626090430561; Mon, 12 Jul 2021 04:47:10 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b11sm14120165wrf.43.2021.07.12.04.47.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:10 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:57 +0000 Subject: [PATCH 17/19] [GSOC] ref-filter: remove grab_oid() function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Because "atom_type == ATOM_OBJECTNAME" implies the condition of `starts_with(name, "objectname")`, "atom_type == ATOM_TREE" implies the condition of `starts_with(name, "tree")`, so the check for `starts_with(name, field)` in grab_oid() is redundant. So Remove the grab_oid() from ref-filter, to reduce repeated check. Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- ref-filter.c | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 91e26c9aba3..1c7287f1061 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1077,16 +1077,6 @@ static const char *do_grab_oid(const char *field, const struct object_id *oid, } } -static int grab_oid(const char *name, const char *field, const struct object_id *oid, - struct atom_value *v, struct used_atom *atom) -{ - if (starts_with(name, field)) { - v->s = xstrdup(do_grab_oid(field, oid, atom)); - return 1; - } - return 0; -} - /* See grab_values */ static void grab_common_values(struct atom_value *val, int deref, struct expand_data *oi) { @@ -1112,8 +1102,9 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_ } } else if (atom_type == ATOM_DELTABASE) v->s = xstrdup(oid_to_hex(&oi->delta_base_oid)); - else if (atom_type == ATOM_OBJECTNAME && deref) - grab_oid(name, "objectname", &oi->oid, v, &used_atom[i]); + else if (atom_type == ATOM_OBJECTNAME && deref) { + v->s = xstrdup(do_grab_oid("objectname", &oi->oid, &used_atom[i])); + } } } @@ -1154,9 +1145,10 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object continue; if (deref) name++; - if (atom_type == ATOM_TREE && - grab_oid(name, "tree", get_commit_tree_oid(commit), v, &used_atom[i])) + if (atom_type == ATOM_TREE) { + v->s = xstrdup(do_grab_oid("tree", get_commit_tree_oid(commit), &used_atom[i])); continue; + } if (atom_type == ATOM_NUMPARENT) { v->value = commit_list_count(commit->parents); v->s = xstrfmt("%lu", (unsigned long)v->value); @@ -1959,9 +1951,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) v->s = xstrdup(buf + 1); } continue; - } else if (!deref && atom_type == ATOM_OBJECTNAME && - grab_oid(name, "objectname", &ref->objectname, v, atom)) { - continue; + } else if (!deref && atom_type == ATOM_OBJECTNAME) { + v->s = xstrdup(do_grab_oid("objectname", &ref->objectname, atom)); + continue; } else if (atom_type == ATOM_HEAD) { if (atom->u.head && !strcmp(ref->refname, atom->u.head)) v->s = xstrdup("*"); From patchwork Mon Jul 12 11:46:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 12370805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86332C07E99 for ; Mon, 12 Jul 2021 11:47:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 730C060C3F for ; Mon, 12 Jul 2021 11:47:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233221AbhGLLuP (ORCPT ); Mon, 12 Jul 2021 07:50:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47584 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232683AbhGLLuB (ORCPT ); Mon, 12 Jul 2021 07:50:01 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73A19C0613DD for ; Mon, 12 Jul 2021 04:47:12 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id j34so11193909wms.5 for ; Mon, 12 Jul 2021 04:47:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=9N06sBMGSwslqzktoVL9OhM7KzXq4rCuo4jo7obxQWo=; b=ePZ922AJEWNWQyfkc3pf+bVIma+DxiTmINgCI/zWOTf4nEX8qTY1XpvBNVbqc4KjYO UprOhcUZdoSlfB4PMAFz5GB0d7XsMEQdwvDlNJEsPdLSGpZclHZ8ZS4TfAog7bmKfPUc pitZouTr0QKRkBcYGalzR54o7ZTbu9TQEP5RRM8BhhZ1BXImuGGhp0Vkj4axTigjOSAc +f6FB3FtT0Kb7DNJwWfsdqeErMmffiKM0CvnvRdDhRORBfFF7jBsguD0KB85EVnn7vQF cP5EvAfzGJjGL1nS0+Zx6gK3t/GAR2wgLI9y+ZefvL7FOSzR/BEXmdKa4GmAVHuRLxuQ pADg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=9N06sBMGSwslqzktoVL9OhM7KzXq4rCuo4jo7obxQWo=; b=b6H8EJAakQV1zWKtDVd0p1bC2h/7oUW2NrImP/XtA6TytirSsryo5rxV7iw4XJn5Ph GXRJEum80c3PUzNzPBSo9Os+i4JKLK6r3D3NuXySXx2cPqKSWFvx4AGvVuts8nP9tPSA ByVuSc4vEr38/ZiAQppPK7Nxo8jGCbs+MtcbTPjtzusWNefXD9pQCSK+D56gpK8qj2Xq gmLu16wznOR1X2NJkhs6qrGIp9n7xEoIQyzdf0Lh3RVpbYuy9gdV09IcW+8dZRzcmKQd P0qrNCLKG6p3TpWdqcRJxDH4GCKHqvTfSNFpO3EDY5RO5GM3NDI1DJxSDGsgPrA2keKn oO5w== X-Gm-Message-State: AOAM531RH1ZPvxUFNc8+y7/ObLPD7NO4YuBf5BfIftu0gp63Ex4gGj/F w8Mn3PyR/vQ5TG4tKYICQUy3OvC+u+A= X-Google-Smtp-Source: ABdhPJwfsvCvLVqO15yHC5HmDSqOoUKtORn0lfCf0XrtSLgVSi7+TpyDBaykKPfacOTicZn/18IIvA== X-Received: by 2002:a7b:c7cb:: with SMTP id z11mr7000913wmk.102.1626090431110; Mon, 12 Jul 2021 04:47:11 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x18sm13678921wrw.19.2021.07.12.04.47.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:10 -0700 (PDT) Message-Id: <2edca9dc46538d85bc32a206e71c3336e384df41.1626090419.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:58 +0000 Subject: [PATCH 18/19] [GSOC] cat-file: create p1006-cat-file.sh MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Create p1006-cat-file.sh to provide performance testing for `git cat-file --batch` and `git cat-file --batch-check`. This will help us compare the performance changes after we let cat-file reuse the ref-filter logic. Helped-by: Ævar Arnfjörð Bjarmason Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- t/perf/p1006-cat-file.sh | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 t/perf/p1006-cat-file.sh diff --git a/t/perf/p1006-cat-file.sh b/t/perf/p1006-cat-file.sh new file mode 100755 index 00000000000..b84ac31f9cc --- /dev/null +++ b/t/perf/p1006-cat-file.sh @@ -0,0 +1,28 @@ +#!/bin/sh + +test_description='Basic sort performance tests' +. ./perf-lib.sh + +test_perf_default_repo + +test_expect_success 'setup' ' + git rev-list --all >rla +' + +test_perf 'cat-file --batch-check' ' + git cat-file --batch-check X-Patchwork-Id: 12370807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E6C7C11F67 for ; Mon, 12 Jul 2021 11:47:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F0F44610CD for ; Mon, 12 Jul 2021 11:47:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233255AbhGLLuQ (ORCPT ); Mon, 12 Jul 2021 07:50:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232548AbhGLLuB (ORCPT ); Mon, 12 Jul 2021 07:50:01 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D42CC0613E5 for ; Mon, 12 Jul 2021 04:47:13 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id r11so19931405wro.9 for ; Mon, 12 Jul 2021 04:47:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=IzJopMreD03mb0H5DzVSKa/vpC4rGACWQqE7utDtPrg=; b=dXqpYFD9eAiBhFGdgve33xrEcPkRH0EaUW4fL5CEdG1cSWRK2tZuHt8taA5uEpScSn +2Z0VpjRauxIIo75hP9lSXTttc97vgOQFTYlp/IIbreeM6zF+LVuRd0qfIXmRYHzOzMI MWyssVM2fN2bWomJ9B9vYMrKVH8efUSzygsyjJYFn+X6U7O30aYogElRqRfYUOYJJv77 kmmxZUHVk1xEshU1gI1TGm0RaHfK6I56/ZEyvYJGvjU/DQXPX+g7jciRD41bhmCmo8Z4 jkFOUCDJu3rFzs7ua6HUBvGJWl/fcsFHZ1Jo+hwEhFYa4iI+ktx8E/icsa4bgd6ks35H VTbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=IzJopMreD03mb0H5DzVSKa/vpC4rGACWQqE7utDtPrg=; b=MeKyw7LUX12nrD/9YfLcSROsmc3/77Nq3NbKj6q5zUO1foCuFSaipcvGvr88DHtDC0 exm+RLQBQN5wvkqsIlhwJIIi04fM+GAJKlmx06mGj+u/xtFtfDCCZRQJjl+mNggvDjkt uNlMC1935b8UUFSa/XlZ/DHaR3CS1OTxlrrCx83v528uPKKniDWj07swsZuF5sIRTiGG 6Q795c/ADBzeFm+68eLWpOboiWPOsF5VaMy24/7g5sP0V2NF5Un8YQ3zQA8fI3qpkW/2 F9RY1dG7D55fUi4e7PcQKi9nqf9rCim5bmujgW340HW85kTb0gRzP8DzxMwiprwQxlTV hnnw== X-Gm-Message-State: AOAM530bU21RJpy1F3wif7YcFBGPw0ELMDTkjjUjEzNhRZJxQcLeJQ86 EVgMgWiocqs5G2XwpuFm7x5cGBO7I88= X-Google-Smtp-Source: ABdhPJyz4dQjLH4tldiRHSID6lQVBN3V+rnf3pwlyRuvTWRmK2s4aq9CnuuhubDSJQbbX4sgNfBeow== X-Received: by 2002:a5d:55c1:: with SMTP id i1mr19476095wrw.77.1626090431725; Mon, 12 Jul 2021 04:47:11 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f7sm9983967wml.35.2021.07.12.04.47.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 04:47:11 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Jul 2021 11:46:59 +0000 Subject: [PATCH 19/19] [GSOC] cat-file: use fast path when using default_format MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: Junio C Hamano , Christian Couder , Hariom Verma , Bagas Sanjaya , Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Eric Sunshine , ZheNing Hu , ZheNing Hu Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: ZheNing Hu From: ZheNing Hu Add the member `default_format` to struct `batch_options`, when we are using the default format on `git cat-file --batch`, or `git cat-file --batch-check`, `default_format` will be set, if we don't use `--textconv` or `--filter`, then we will not call verify_ref_format(), has_object_file() and format_ref_array_item(). Instead, we get the object data directly through oid_object_info_extended() and then output the data directly. By using this fast path, we can reduce some of the extra overhead when `cat-file --batch` using ref-filter. The running time of `git cat-file --batch-check` will be similar to before, and the running time of `git cat-file --batch` will be 9.1% less than before. Helped-by: Ævar Arnfjörð Bjarmason Mentored-by: Christian Couder Mentored-by: Hariom Verma Signed-off-by: ZheNing Hu --- builtin/cat-file.c | 79 +++++++++++++++++++++++++++++++++------------- 1 file changed, 57 insertions(+), 22 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 3a6153e778f..8edc19f2d5a 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -26,6 +26,7 @@ struct batch_options { int all_objects; int unordered; int cmdmode; /* may be 'w' or 'c' for --filters or --textconv */ + int default_format; struct ref_format format; }; @@ -196,6 +197,7 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct expand_data { struct object_id oid; + struct object_info info; const char *rest; int split_on_whitespace; }; @@ -216,27 +218,58 @@ static void batch_object_write(const char *obj_name, struct batch_options *opt, struct expand_data *data) { - int ret; - struct ref_array_item item = { data->oid, data->rest, opt->cmdmode }; - - strbuf_reset(scratch); - strbuf_reset(err); - - ret = format_ref_array_item(&item, &opt->format, scratch, err); - if (ret < 0) - die("%s\n", err->buf); - if (ret) { - /* ret > 0 means when the object corresponding to oid - * cannot be found in format_ref_array_item(), we only print - * the error message. - */ - printf("%s\n", err->buf); + if (opt->default_format && !opt->cmdmode) { + struct strbuf type_name = STRBUF_INIT; + unsigned long size; + void *content; + + if (opt->print_contents) + data->info.contentp = &content; + + data->info.type_name = &type_name; + data->info.sizep = &size; + + if (oid_object_info_extended(the_repository, &data->oid, &data->info, + OBJECT_INFO_LOOKUP_REPLACE) < 0) { + printf("%s missing\n", + obj_name ? obj_name : oid_to_hex(&data->oid)); + fflush(stdout); + return; + } + + fprintf(stdout, "%s %s %"PRIuMAX"\n", oid_to_hex(&data->oid), + data->info.type_name->buf, + (uintmax_t)*data->info.sizep); fflush(stdout); + strbuf_release(&type_name); + if (opt->print_contents) { + batch_write(opt, content, *data->info.sizep); + batch_write(opt, "\n", 1); + free(content); + } } else { - strbuf_addch(scratch, '\n'); - batch_write(opt, scratch->buf, scratch->len); + int ret; + struct ref_array_item item = { data->oid, data->rest, opt->cmdmode }; + + strbuf_reset(scratch); + strbuf_reset(err); + + ret = format_ref_array_item(&item, &opt->format, scratch, err); + if (ret < 0) + die("%s\n", err->buf); + if (ret) { + /* ret > 0 means when the object corresponding to oid + * cannot be found in format_ref_array_item(), we only print + * the error message. + */ + printf("%s\n", err->buf); + fflush(stdout); + } else { + strbuf_addch(scratch, '\n'); + batch_write(opt, scratch->buf, scratch->len); + } + free_ref_array_item_value(&item); } - free_ref_array_item_value(&item); } static void batch_one_object(const char *obj_name, @@ -288,7 +321,7 @@ static void batch_one_object(const char *obj_name, return; } - if (!has_object_file(&data->oid)) { + if ((!opt->default_format || opt->cmdmode) && !has_object_file(&data->oid)) { printf("%s missing\n", obj_name ? obj_name : oid_to_hex(&data->oid)); fflush(stdout); @@ -380,7 +413,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio if (batch->print_contents) strbuf_addstr(&format, "\n%(raw)"); batch->format.format = format.buf; - if (verify_ref_format(&batch->format)) + if ((!batch->default_format || batch->cmdmode) && verify_ref_format(&batch->format)) usage_with_options(cat_file_usage, options); if (batch->cmdmode || batch->format.use_rest) @@ -483,7 +516,8 @@ static int batch_option_callback(const struct option *opt, bo->enabled = 1; bo->print_contents = !strcmp(opt->long_name, "batch"); bo->format.format = arg; - + if (arg) + bo->default_format = 0; return 0; } @@ -492,7 +526,8 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix) int opt = 0; const char *exp_type = NULL, *obj_name = NULL; struct batch_options batch = { - .format = REF_FORMAT_INIT + .format = REF_FORMAT_INIT, + .default_format = 1 }; int unknown_type = 0;