From patchwork Fri Aug 12 20:10:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942496 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 098AEC00140 for ; Fri, 12 Aug 2022 20:10:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234567AbiHLUK0 (ORCPT ); Fri, 12 Aug 2022 16:10:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234491AbiHLUKY (ORCPT ); Fri, 12 Aug 2022 16:10:24 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90EFF6176 for ; Fri, 12 Aug 2022 13:10:23 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id p10so2245232wru.8 for ; Fri, 12 Aug 2022 13:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=UWkmwEMmDQA88Knl6wRhhl/jDXRNKzlTXKO5NvbUwn8=; b=c1LZg0tyIjRaA0tMZh+uMUE9+biJx0sq5GDZ7VEqdlZcm1onsSj0Gx1+BrtX6tZcaT ySHDTlMF0M22HX1fnWMAHf3nzCwI/Zpx/0AwOiwLH/mKPZFZE8xSbkK3wIaYH1YeY+lZ ng+ZnFsvUAiWcPCFIiWWIZ07kQlmW7IsPb7X02JT0MAJYBNoHpAi3C+yQqtdrIlEjCGa E6hlBLOd8SFhjofe1R7DCU8OLBmcfrUCJGedo+JrnFUwm6SnmMKYXGhTcJD0GRAAtt1Z +O12LiE9wMNWTlj6gTqdu8X7Te6YjR6iSiCXX9KNnotK0qlhv5LuMx1ZE3ZCuB8v/MTB xKCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=UWkmwEMmDQA88Knl6wRhhl/jDXRNKzlTXKO5NvbUwn8=; b=rQ0pJbEvInaHnSzMw10DcssEL8dXuibKE77eewskxNBUkZeGyP2crasEOzx0XiG0N5 JqEvAdD7DAbBrG+8w9U3K0SSmSgI77JyXtFFhDwK/l9agv0aAmH2mwtIXz3qYCaft4aH gu4zgICspqNypi5ztabTX5YtYTpLhKbGNFd1u3rLFcuIiGaF9oyz7AOMGqaxzrbAvIrL n7aLhTIueoK+pSIYhMGRmHCRoOmgclz+uEMBF5m5tZEzHoRNgx9AFJlISrph7cj52eoU lKhL7iFMZdO56reOw4qFeJuDuGcHa1Y6k53HqZjKoQ/WMtShh7y9ruMgoptDPUgAW+T2 WpLQ== X-Gm-Message-State: ACgBeo2ZOcOWBxoevz+d7oAKoxpM609aNuLWUrciQUyk6OHA0PXDOVge Ynl+Rzh9zgh0hvczZMO/ldtmq+ixjHI= X-Google-Smtp-Source: AA6agR6vQNPoUhERNJAi8mUfmPohHKJlWvkd6lOxIYhswX4WnmkYjBxTfLaUdp2dnF/TViN0uI3DBg== X-Received: by 2002:a5d:6d0f:0:b0:220:76a3:b31d with SMTP id e15-20020a5d6d0f000000b0022076a3b31dmr3012087wrq.581.1660335021969; Fri, 12 Aug 2022 13:10:21 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id bp15-20020a5d5a8f000000b0021ee65426a2sm289078wrb.65.2022.08.12.13.10.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:21 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:09 +0000 Subject: [PATCH v4 01/11] scalar-diagnose: use "$GIT_UNZIP" in test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Use the "$GIT_UNZIP" test variable rather than verbatim 'unzip' to unzip the 'scalar diagnose' archive. Using "$GIT_UNZIP" is needed to run the Scalar tests on systems where 'unzip' is not in the system path. Signed-off-by: Victoria Dye --- contrib/scalar/t/t9099-scalar.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/contrib/scalar/t/t9099-scalar.sh b/contrib/scalar/t/t9099-scalar.sh index 10b1172a8aa..fac86a57550 100755 --- a/contrib/scalar/t/t9099-scalar.sh +++ b/contrib/scalar/t/t9099-scalar.sh @@ -109,14 +109,14 @@ test_expect_success UNZIP 'scalar diagnose' ' sed -n "s/.*$SQ\\(.*\\.zip\\)$SQ.*/\\1/p" zip_path && zip_path=$(cat zip_path) && test -n "$zip_path" && - unzip -v "$zip_path" && + "$GIT_UNZIP" -v "$zip_path" && folder=${zip_path%.zip} && test_path_is_missing "$folder" && - unzip -p "$zip_path" diagnostics.log >out && + "$GIT_UNZIP" -p "$zip_path" diagnostics.log >out && test_file_not_empty out && - unzip -p "$zip_path" packs-local.txt >out && + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && grep "$(pwd)/.git/objects" out && - unzip -p "$zip_path" objects-local.txt >out && + "$GIT_UNZIP" -p "$zip_path" objects-local.txt >out && grep "^Total: [1-9]" out ' From patchwork Fri Aug 12 20:10:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942498 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28FFCC00140 for ; Fri, 12 Aug 2022 20:10:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235100AbiHLUKa (ORCPT ); Fri, 12 Aug 2022 16:10:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234710AbiHLUK0 (ORCPT ); Fri, 12 Aug 2022 16:10:26 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4749C64CA for ; Fri, 12 Aug 2022 13:10:25 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id p10so2245291wru.8 for ; Fri, 12 Aug 2022 13:10:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=fTQyYbLVeujkIXWBZRa69qVeE19+Lc0IZap+yd+gMZw=; b=F57dzq5xg1Yitx4aa+X5/VAoOGzT3p/dM4+d4mkS5W/HeAoDlH3H9srjhIeArXvuJa KW2Fmf8rDTdmtk885i2zGfxYzxUcxR0EFxcBFclAyJXquVJ3Un6ee2Ip7FyGnXVSnAtp tXFH9czvyHew/ZJgXfDLkCpArAmaDegkMNVLjZIi+oPjKZJgHnsDd/neC1QvC7vgjXCq Vq2dp+kCwKsu9nP6sz3WRu5royR22MN6HtZUPYf5F8beNXY/fxodM1HW/IzaWAHT1iW+ Nzi0k+XsfYdMNWQ4FP3/Kcbe1G47y2nk8qfKr+rLJLsmmgD0jpRLt2LKgGIT+Pdk5T4b YXaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=fTQyYbLVeujkIXWBZRa69qVeE19+Lc0IZap+yd+gMZw=; b=hyWJApzOT1JoYxC7/oA9Ivj/OZMc+o91CQLNLgxfe5HeIPTr0Yfes4PX6z4E8i8fQK cQ5V0BXzt0BbfijYsgqqsL96UN6KEhp/jpyX6Koy9PDLgQr+YR8waRqBdxpl6uTgHO/n nAgXV2MJRX2k8v1ALEGluO1ttfCvKPmPHviJXD8ddTXDqztBfswe56bj34DsptxZ4f9M EOuPi/GNyLb1Mw0SbHdSQ2CRLnaM+hlLhmG7+7IFkMdKnsgj7HYEWXzEnDz61utVh3uL /PWED9dahfTELc8EsX5ERPcOVHelbZ/y5/l3HUEKdMwN5hi6mknDLmWjZBOhfOeSsz6e ibKQ== X-Gm-Message-State: ACgBeo3AW4l4Ulvo1Dtx5+7ztI6/DWRk4oD3A+Qpf7ZLTvsbD5kwmcD7 G108PrJbDnR4k/uxjj3o52KC3ZymGBg= X-Google-Smtp-Source: AA6agR5T/Ei8U/JcjW54MoPUDUzFkCNudHipDs5c2AcRf0TRSoSXGVR2gMB/KMPErm4BjD4AocfGMQ== X-Received: by 2002:a05:6000:3c6:b0:220:5efd:423c with SMTP id b6-20020a05600003c600b002205efd423cmr3039588wrg.214.1660335023426; Fri, 12 Aug 2022 13:10:23 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 3-20020a05600c248300b003a342933727sm574106wms.3.2022.08.12.13.10.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:22 -0700 (PDT) Message-Id: <78a93eb95bb0c43778501f03837bffdcf74e4c99.1660335019.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:10 +0000 Subject: [PATCH v4 02/11] scalar-diagnose: avoid 32-bit overflow of size_t Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Avoid 32-bit size_t overflow when reporting the available disk space in 'get_disk_info' by casting the block size and available block count to 'off_t' before multiplying them. Without this change, 'st_mult' would (correctly) report a size_t overflow on 32-bit systems at or exceeding 2^32 bytes of available space. Note that 'off_t' is a 64-bit integer even on 32-bit systems due to the inclusion of '#define _FILE_OFFSET_BITS 64' in 'git-compat-util.h' (see b97e911643 (Support for large files on 32bit systems., 2007-02-17)). Helped-by: Junio C Hamano Signed-off-by: Victoria Dye --- contrib/scalar/scalar.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index 97e71fe19cd..04046452284 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -348,7 +348,7 @@ static int get_disk_info(struct strbuf *out) } strbuf_addf(out, "Available space on '%s': ", buf.buf); - strbuf_humanise_bytes(out, st_mult(stat.f_bsize, stat.f_bavail)); + strbuf_humanise_bytes(out, (off_t)stat.f_bsize * (off_t)stat.f_bavail); strbuf_addf(out, " (mount flags 0x%lx)\n", stat.f_flag); strbuf_release(&buf); #endif From patchwork Fri Aug 12 20:10:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2266C00140 for ; Fri, 12 Aug 2022 20:10:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235155AbiHLUKd (ORCPT ); Fri, 12 Aug 2022 16:10:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234491AbiHLUK1 (ORCPT ); Fri, 12 Aug 2022 16:10:27 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F18F66170 for ; Fri, 12 Aug 2022 13:10:25 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id h13so2251547wrf.6 for ; Fri, 12 Aug 2022 13:10:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=327gbsyYeccEL0tlq4IQ1Y7CXUns6oBUTnyfqIVz7cI=; b=EDUWwtWwPM2IFRoc5LOML/V1i7NTbGAVl2dcyNuJtjHA7FW0shjnL97x/YKkO1f5r4 JvKgvGbITYNS7c6YcMP+aaBBtXBcR1vRtj1XAF8yv0ksCb/3NmtZ0RUoc8o9IOhJ+dAe YTPWmTsINj7IxwUCKZZRtcpBEoMuO91gHJpMNWvYx141ZneDtFAmBKixnlIZeGhqkttQ uJqLlsUhxAm/81sLI61jNnhd2KEzsnCacNoBhDHU84Ajx5Rgs9R2jICvuKCY5/xQTpyd MOmE1DZvyFQU7+ZHXOaiKpZ+7Cd8jYOyE0aeuFuLEwUly4KXXtYvfgWeEEFro+No4jOC 2Vxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=327gbsyYeccEL0tlq4IQ1Y7CXUns6oBUTnyfqIVz7cI=; b=ynga/nVwzBay+ZJczURob5z0ZZqHe1HPC7NHmueQjFfGAHkuuvmtILSbyKnhT4RyBq SMKGjS2bKHopu2bRaZz2ORjf2imZdblkhme94/n3i0O6uLTA33qpX4+c29irLhWy4WPO 3btSWPma+Z88LqPZ6cjl9RbHf3w519Wqw6BPUoY7A7vKaswm+R1ldAsrqtkbYQ0CI+SU g9y/4WcvWK0snYHEX8q5lzX1Gr/lGnW7BuZKHYo/NW2kL2IhU8uZiVCB0L9gA2si0NwL eYX1CiEEArtIoM5YzR70CIpFjCe83vcGWRimFz0U0SMITmFkx+o6m9khZT3/SLIR3czK L2KA== X-Gm-Message-State: ACgBeo2Bt/lfA77m9apkTghblRsaTU0XkJn2nbOS6PcmurOcu1uo+XiP 0aKu87QfrgKfwtEDiFA337mstSG5qkc= X-Google-Smtp-Source: AA6agR59Y9siVAGb8ojnUkMfU3imTZSo8E2kcqLNpjvpuyT1Nj9nEOgsiTLVE4eOGKDcld+QWzlSFQ== X-Received: by 2002:a05:6000:15c7:b0:222:ce5a:4b01 with SMTP id y7-20020a05600015c700b00222ce5a4b01mr2914648wry.206.1660335024361; Fri, 12 Aug 2022 13:10:24 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l23-20020a05600c2cd700b003a502c23f2asm574585wmc.16.2022.08.12.13.10.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:23 -0700 (PDT) Message-Id: <22ee8ea5a1e04a42ad359be4eb5ebc96bd5e5fa2.1660335019.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:11 +0000 Subject: [PATCH v4 03/11] scalar-diagnose: add directory to archiver more gently Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye If a directory added to the 'scalar diagnose' archiver does not exist, warn and return 0 from 'add_directory_to_archiver()' rather than failing with a fatal error. This handles a failure edge case where the '.git/logs' has not yet been created when running 'scalar diagnose', but extends to any situation where a directory may be missing in the '.git' dir. Now, when a directory is missing a warning is captured in the diagnostic logs. This provides a user with more complete information than if 'scalar diagnose' simply failed with an error. Helped-by: Junio C Hamano Signed-off-by: Victoria Dye --- contrib/scalar/scalar.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index 04046452284..b9092f0b612 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -266,14 +266,20 @@ static int add_directory_to_archiver(struct strvec *archiver_args, const char *path, int recurse) { int at_root = !*path; - DIR *dir = opendir(at_root ? "." : path); + DIR *dir; struct dirent *e; struct strbuf buf = STRBUF_INIT; size_t len; int res = 0; - if (!dir) + dir = opendir(at_root ? "." : path); + if (!dir) { + if (errno == ENOENT) { + warning(_("could not archive missing directory '%s'"), path); + return 0; + } return error_errno(_("could not open directory '%s'"), path); + } if (!at_root) strbuf_addf(&buf, "%s/", path); From patchwork Fri Aug 12 20:10:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942500 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E3F7C25B0F for ; Fri, 12 Aug 2022 20:10:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235672AbiHLUKh (ORCPT ); Fri, 12 Aug 2022 16:10:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234754AbiHLUK2 (ORCPT ); Fri, 12 Aug 2022 16:10:28 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D3CB6592 for ; Fri, 12 Aug 2022 13:10:27 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id 186-20020a1c02c3000000b003a34ac64bdfso4799523wmc.1 for ; Fri, 12 Aug 2022 13:10:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=YKKdcJFhp+40tORJ9u012i+7uSC/4xTtFFym2CPkqAw=; b=lc26Ncc+9ptdvl/RxG8M9J+YCXeQ5bPqBSoRduYeuesgQD1q5bPRB1GbbzIJ+kPDvm po2A6baLh0PVCxU8OPfpGcUrnzQ1o5+x4FNBS84UIgiVob9ZZRk5lwAyR6xiXw4VJKvn tOKuOhtxW5JexsFKidyF5zlb6hBt8RaK6F2ec+pD9e14SOcaHc1xkRa25t5V6+XgCE/h C2Emz5EEi0R2KyGOBBzK+9ChwjfIRouLvv4aqd+rP7bZIgHiZMAkkCR4M+7bP/Cai0MM bULsPUpsHgc6VaCQ9XsH2wkqJhhdrgymWf48fnOEbXuo0qVE8M7lBAshlwoCoOagE15w lVUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=YKKdcJFhp+40tORJ9u012i+7uSC/4xTtFFym2CPkqAw=; b=T7cq9mQIOPzVIaMVI/TCW6iQq1kh8xKenkuD8DnqB5blku4W6A6Yxgx22LlGi+vhJU a1YG1SYC/s9I9mJusQdSrqyj9s4r0kLe+1fvdKz7YpXt0bU/nUtB6L5tFDxzhPUsErSJ s1wwpnfI2AxUmr2abun4OVQgPxOCZ1120O1N7XiXTJql+8RdbUkulKvEPPEMudwTU4uj Ui4p8iCIE065S+8FyLvsmSFrT+6GTfyYMg/oNISUlwfQVBWUzSOPTmcnozCK9yFIj9Zg BgkJMJnatO698AFNX5bWhhe1M38LQ5Cj2psTdloEP3BRZLfvtuRxKcHKFsxFtgHTBXRL 2gjQ== X-Gm-Message-State: ACgBeo1mFW1Vj9DNvrYArzjKOUN1f2WqVgSUtwhRYCNqx/k7VQYU5L1d iUsyeLbIPKZ+Xv9G8pDsrT2TTvxaIMs= X-Google-Smtp-Source: AA6agR4l7r7cU/KTeP6DaUO5QCaTuOSMi03Tt/EFd09HEM4bgjv09yhsnzH3mpbPXANxvJMAqI2piw== X-Received: by 2002:a05:600c:214d:b0:3a5:ce18:bb71 with SMTP id v13-20020a05600c214d00b003a5ce18bb71mr2792622wml.1.1660335025308; Fri, 12 Aug 2022 13:10:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l9-20020a05600c4f0900b003a37d8b864esm552861wmq.30.2022.08.12.13.10.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:24 -0700 (PDT) Message-Id: <18f2ba4e0cdd9aea213dc7d0e908c6c0d8eccfd0.1660335019.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:12 +0000 Subject: [PATCH v4 04/11] scalar-diagnose: move 'get_disk_info()' to 'compat/' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Move 'get_disk_info()' function into 'compat/'. Although Scalar-specific code is generally not part of the main Git tree, 'get_disk_info()' will be used in subsequent patches by additional callers beyond 'scalar diagnose'. This patch prepares for that change, at which point this platform-specific code should be part of 'compat/' as a matter of convention. The function is copied *mostly* verbatim, with two exceptions: * '#ifdef WIN32' is replaced with '#ifdef GIT_WINDOWS_NATIVE' to allow 'statvfs' to be used with Cygwin. * the 'struct strbuf buf' and 'int res' (as well as their corresponding cleanup & return) are moved outside of the '#ifdef' block. Signed-off-by: Victoria Dye --- compat/disk.h | 56 +++++++++++++++++++++++++++++++++++++++++ contrib/scalar/scalar.c | 53 +------------------------------------- git-compat-util.h | 1 + 3 files changed, 58 insertions(+), 52 deletions(-) create mode 100644 compat/disk.h diff --git a/compat/disk.h b/compat/disk.h new file mode 100644 index 00000000000..50a32e3d8a4 --- /dev/null +++ b/compat/disk.h @@ -0,0 +1,56 @@ +#ifndef COMPAT_DISK_H +#define COMPAT_DISK_H + +#include "git-compat-util.h" + +static int get_disk_info(struct strbuf *out) +{ + struct strbuf buf = STRBUF_INIT; + int res = 0; + +#ifdef GIT_WINDOWS_NATIVE + char volume_name[MAX_PATH], fs_name[MAX_PATH]; + DWORD serial_number, component_length, flags; + ULARGE_INTEGER avail2caller, total, avail; + + strbuf_realpath(&buf, ".", 1); + if (!GetDiskFreeSpaceExA(buf.buf, &avail2caller, &total, &avail)) { + error(_("could not determine free disk size for '%s'"), + buf.buf); + res = -1; + goto cleanup; + } + + strbuf_setlen(&buf, offset_1st_component(buf.buf)); + if (!GetVolumeInformationA(buf.buf, volume_name, sizeof(volume_name), + &serial_number, &component_length, &flags, + fs_name, sizeof(fs_name))) { + error(_("could not get info for '%s'"), buf.buf); + res = -1; + goto cleanup; + } + strbuf_addf(out, "Available space on '%s': ", buf.buf); + strbuf_humanise_bytes(out, avail2caller.QuadPart); + strbuf_addch(out, '\n'); +#else + struct statvfs stat; + + strbuf_realpath(&buf, ".", 1); + if (statvfs(buf.buf, &stat) < 0) { + error_errno(_("could not determine free disk size for '%s'"), + buf.buf); + res = -1; + goto cleanup; + } + + strbuf_addf(out, "Available space on '%s': ", buf.buf); + strbuf_humanise_bytes(out, (off_t)stat.f_bsize * (off_t)stat.f_bavail); + strbuf_addf(out, " (mount flags 0x%lx)\n", stat.f_flag); +#endif + +cleanup: + strbuf_release(&buf); + return res; +} + +#endif /* COMPAT_DISK_H */ diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index b9092f0b612..607fedefd82 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -13,6 +13,7 @@ #include "help.h" #include "archive.h" #include "object-store.h" +#include "compat/disk.h" /* * Remove the deepest subdirectory in the provided path string. Path must not @@ -309,58 +310,6 @@ static int add_directory_to_archiver(struct strvec *archiver_args, return res; } -#ifndef WIN32 -#include -#endif - -static int get_disk_info(struct strbuf *out) -{ -#ifdef WIN32 - struct strbuf buf = STRBUF_INIT; - char volume_name[MAX_PATH], fs_name[MAX_PATH]; - DWORD serial_number, component_length, flags; - ULARGE_INTEGER avail2caller, total, avail; - - strbuf_realpath(&buf, ".", 1); - if (!GetDiskFreeSpaceExA(buf.buf, &avail2caller, &total, &avail)) { - error(_("could not determine free disk size for '%s'"), - buf.buf); - strbuf_release(&buf); - return -1; - } - - strbuf_setlen(&buf, offset_1st_component(buf.buf)); - if (!GetVolumeInformationA(buf.buf, volume_name, sizeof(volume_name), - &serial_number, &component_length, &flags, - fs_name, sizeof(fs_name))) { - error(_("could not get info for '%s'"), buf.buf); - strbuf_release(&buf); - return -1; - } - strbuf_addf(out, "Available space on '%s': ", buf.buf); - strbuf_humanise_bytes(out, avail2caller.QuadPart); - strbuf_addch(out, '\n'); - strbuf_release(&buf); -#else - struct strbuf buf = STRBUF_INIT; - struct statvfs stat; - - strbuf_realpath(&buf, ".", 1); - if (statvfs(buf.buf, &stat) < 0) { - error_errno(_("could not determine free disk size for '%s'"), - buf.buf); - strbuf_release(&buf); - return -1; - } - - strbuf_addf(out, "Available space on '%s': ", buf.buf); - strbuf_humanise_bytes(out, (off_t)stat.f_bsize * (off_t)stat.f_bavail); - strbuf_addf(out, " (mount flags 0x%lx)\n", stat.f_flag); - strbuf_release(&buf); -#endif - return 0; -} - /* printf-style interface, expects `=` argument */ static int set_config(const char *fmt, ...) { diff --git a/git-compat-util.h b/git-compat-util.h index 58d7708296b..9a62e3a0d2d 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -258,6 +258,7 @@ static inline int is_xplatform_dir_sep(int c) #include #include #include +#include #include #ifndef NO_SYS_SELECT_H #include From patchwork Fri Aug 12 20:10:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FF16C00140 for ; Fri, 12 Aug 2022 20:10:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235695AbiHLUKj (ORCPT ); Fri, 12 Aug 2022 16:10:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235030AbiHLUK3 (ORCPT ); Fri, 12 Aug 2022 16:10:29 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2036B65DC for ; Fri, 12 Aug 2022 13:10:28 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id j7so2257408wrh.3 for ; Fri, 12 Aug 2022 13:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=SXOhwYqQ9zHvwWq5L0hyRxL2Js95kdaG3yZlchig5OU=; b=HxiYaTgqo0XEy5T8q9uIqecmoEVCdYL2TRRZusg02cIfXhkU318GpjEh2KaFkwfK9v USlOIEVhUzQCpWD7PzTyFp82R1TVtHsz/zypg1U69nJ//y1du+njG4Swwvjigznxx0TE zNl90LntU0BNp1JYzb2nnR6Hr8TWEMy8wJ2ZNTLK9nW4IDLx0JbeelA0keD5TNoYAeKn Bsqgvkgy9JjkNnooMcsu0H/eFn+lXxykoubrtyqi36uO1CV/pNnLRXNE3pEn0dcU8ZWb UrEefAdyAzohQI8L1VwY+CLYfasbxpA/b+6BDSasarzuRO6mu2Ll7S+7aChOOGtcKBJG fcSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=SXOhwYqQ9zHvwWq5L0hyRxL2Js95kdaG3yZlchig5OU=; b=vGQKrCkSgc4RdkSALkTvjp7yxFdCBILU5XmlihjWS3/NMfX+feEuSLW+D+a+XE/20T lWJ0V1gUycFQyXOpR6pAVw5aVKc6ZpY+zt1pK0dKUZ3x/9PLyspt1grlTme+sNHQquwe BwPObG9HkTB/RotaGxPJzTIH+NRKis/CjPp9NdA0ByaSbc78wgmgu2HQcF0s67u93tNl v6bP5Rb0NUkVC92VjBgoo+c+q+Xaslvp2vlsK+vWugxKEaqr6je7FKpU3/HTk95JJxuP jobAEjrI9LYpH+YcF1pwTpIsfTuX/zgsGk011wfB1GFuYzxUf5X4nOvaXVvt9f0sg9yE 5ueg== X-Gm-Message-State: ACgBeo09iGr7S5hh0O5cUBkA9WIA5tBIuPb1DMsTuitArQW6GNEroqzB qBsUXKQzNCnR/8oHckxznNpv/k92+24= X-Google-Smtp-Source: AA6agR4ALeZooxXATXVFp9zJWaU70Ugx7brRsl22fSwbx6xyHg0Qhe8j9tP2oK+EU9oJ9KwQxI+HdQ== X-Received: by 2002:a05:6000:1841:b0:21e:fa04:61f4 with SMTP id c1-20020a056000184100b0021efa0461f4mr2908959wri.491.1660335026209; Fri, 12 Aug 2022 13:10:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r18-20020a05600c35d200b003a2e92edeccsm482611wmq.46.2022.08.12.13.10.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:25 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:13 +0000 Subject: [PATCH v4 05/11] scalar-diagnose: move functionality to common location MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Move the core functionality of 'scalar diagnose' into a new 'diagnose.[c,h]' library to prepare for new callers in the main Git tree generating diagnostic archives. These callers will be introduced in subsequent patches. While this patch appears large, it is mostly made up of moving code out of 'scalar.c' and into 'diagnose.c'. Specifically, the functions - dir_file_stats_objects() - dir_file_stats() - count_files() - loose_objs_stats() - add_directory_to_archiver() are all copied verbatim from 'scalar.c'. The 'create_diagnostics_archive()' function is a mostly identical (partial) copy of 'cmd_diagnose()', with the primary changes being that 'zip_path' is an input and "Enlistment root" is corrected to "Repository root" in the archiver log. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: Victoria Dye --- Makefile | 1 + contrib/scalar/scalar.c | 202 +------------------------------------ diagnose.c | 216 ++++++++++++++++++++++++++++++++++++++++ diagnose.h | 8 ++ 4 files changed, 227 insertions(+), 200 deletions(-) create mode 100644 diagnose.c create mode 100644 diagnose.h diff --git a/Makefile b/Makefile index 2ec9b2dc6bb..ed66cb70e5a 100644 --- a/Makefile +++ b/Makefile @@ -932,6 +932,7 @@ LIB_OBJS += ctype.o LIB_OBJS += date.o LIB_OBJS += decorate.o LIB_OBJS += delta-islands.o +LIB_OBJS += diagnose.o LIB_OBJS += diff-delta.o LIB_OBJS += diff-merges.o LIB_OBJS += diff-lib.o diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index 607fedefd82..3983def760a 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -11,9 +11,7 @@ #include "dir.h" #include "packfile.h" #include "help.h" -#include "archive.h" -#include "object-store.h" -#include "compat/disk.h" +#include "diagnose.h" /* * Remove the deepest subdirectory in the provided path string. Path must not @@ -263,53 +261,6 @@ static int unregister_dir(void) return res; } -static int add_directory_to_archiver(struct strvec *archiver_args, - const char *path, int recurse) -{ - int at_root = !*path; - DIR *dir; - struct dirent *e; - struct strbuf buf = STRBUF_INIT; - size_t len; - int res = 0; - - dir = opendir(at_root ? "." : path); - if (!dir) { - if (errno == ENOENT) { - warning(_("could not archive missing directory '%s'"), path); - return 0; - } - return error_errno(_("could not open directory '%s'"), path); - } - - if (!at_root) - strbuf_addf(&buf, "%s/", path); - len = buf.len; - strvec_pushf(archiver_args, "--prefix=%s", buf.buf); - - while (!res && (e = readdir(dir))) { - if (!strcmp(".", e->d_name) || !strcmp("..", e->d_name)) - continue; - - strbuf_setlen(&buf, len); - strbuf_addstr(&buf, e->d_name); - - if (e->d_type == DT_REG) - strvec_pushf(archiver_args, "--add-file=%s", buf.buf); - else if (e->d_type != DT_DIR) - warning(_("skipping '%s', which is neither file nor " - "directory"), buf.buf); - else if (recurse && - add_directory_to_archiver(archiver_args, - buf.buf, recurse) < 0) - res = -1; - } - - closedir(dir); - strbuf_release(&buf); - return res; -} - /* printf-style interface, expects `=` argument */ static int set_config(const char *fmt, ...) { @@ -550,83 +501,6 @@ cleanup: return res; } -static void dir_file_stats_objects(const char *full_path, size_t full_path_len, - const char *file_name, void *data) -{ - struct strbuf *buf = data; - struct stat st; - - if (!stat(full_path, &st)) - strbuf_addf(buf, "%-70s %16" PRIuMAX "\n", file_name, - (uintmax_t)st.st_size); -} - -static int dir_file_stats(struct object_directory *object_dir, void *data) -{ - struct strbuf *buf = data; - - strbuf_addf(buf, "Contents of %s:\n", object_dir->path); - - for_each_file_in_pack_dir(object_dir->path, dir_file_stats_objects, - data); - - return 0; -} - -static int count_files(char *path) -{ - DIR *dir = opendir(path); - struct dirent *e; - int count = 0; - - if (!dir) - return 0; - - while ((e = readdir(dir)) != NULL) - if (!is_dot_or_dotdot(e->d_name) && e->d_type == DT_REG) - count++; - - closedir(dir); - return count; -} - -static void loose_objs_stats(struct strbuf *buf, const char *path) -{ - DIR *dir = opendir(path); - struct dirent *e; - int count; - int total = 0; - unsigned char c; - struct strbuf count_path = STRBUF_INIT; - size_t base_path_len; - - if (!dir) - return; - - strbuf_addstr(buf, "Object directory stats for "); - strbuf_add_absolute_path(buf, path); - strbuf_addstr(buf, ":\n"); - - strbuf_add_absolute_path(&count_path, path); - strbuf_addch(&count_path, '/'); - base_path_len = count_path.len; - - while ((e = readdir(dir)) != NULL) - if (!is_dot_or_dotdot(e->d_name) && - e->d_type == DT_DIR && strlen(e->d_name) == 2 && - !hex_to_bytes(&c, e->d_name, 1)) { - strbuf_setlen(&count_path, base_path_len); - strbuf_addstr(&count_path, e->d_name); - total += (count = count_files(count_path.buf)); - strbuf_addf(buf, "%s : %7d files\n", e->d_name, count); - } - - strbuf_addf(buf, "Total: %d loose objects", total); - - strbuf_release(&count_path); - closedir(dir); -} - static int cmd_diagnose(int argc, const char **argv) { struct option options[] = { @@ -637,12 +511,8 @@ static int cmd_diagnose(int argc, const char **argv) NULL }; struct strbuf zip_path = STRBUF_INIT; - struct strvec archiver_args = STRVEC_INIT; - char **argv_copy = NULL; - int stdout_fd = -1, archiver_fd = -1; time_t now = time(NULL); struct tm tm; - struct strbuf buf = STRBUF_INIT; int res = 0; argc = parse_options(argc, argv, NULL, options, @@ -663,79 +533,11 @@ static int cmd_diagnose(int argc, const char **argv) zip_path.buf); goto diagnose_cleanup; } - stdout_fd = dup(1); - if (stdout_fd < 0) { - res = error_errno(_("could not duplicate stdout")); - goto diagnose_cleanup; - } - - archiver_fd = xopen(zip_path.buf, O_CREAT | O_WRONLY | O_TRUNC, 0666); - if (archiver_fd < 0 || dup2(archiver_fd, 1) < 0) { - res = error_errno(_("could not redirect output")); - goto diagnose_cleanup; - } - - init_zip_archiver(); - strvec_pushl(&archiver_args, "scalar-diagnose", "--format=zip", NULL); - - strbuf_reset(&buf); - strbuf_addstr(&buf, "Collecting diagnostic info\n\n"); - get_version_info(&buf, 1); - - strbuf_addf(&buf, "Enlistment root: %s\n", the_repository->worktree); - get_disk_info(&buf); - write_or_die(stdout_fd, buf.buf, buf.len); - strvec_pushf(&archiver_args, - "--add-virtual-file=diagnostics.log:%.*s", - (int)buf.len, buf.buf); - - strbuf_reset(&buf); - strbuf_addstr(&buf, "--add-virtual-file=packs-local.txt:"); - dir_file_stats(the_repository->objects->odb, &buf); - foreach_alt_odb(dir_file_stats, &buf); - strvec_push(&archiver_args, buf.buf); - - strbuf_reset(&buf); - strbuf_addstr(&buf, "--add-virtual-file=objects-local.txt:"); - loose_objs_stats(&buf, ".git/objects"); - strvec_push(&archiver_args, buf.buf); - - if ((res = add_directory_to_archiver(&archiver_args, ".git", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/hooks", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/info", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/logs", 1)) || - (res = add_directory_to_archiver(&archiver_args, ".git/objects/info", 0))) - goto diagnose_cleanup; - - strvec_pushl(&archiver_args, "--prefix=", - oid_to_hex(the_hash_algo->empty_tree), "--", NULL); - - /* `write_archive()` modifies the `argv` passed to it. Let it. */ - argv_copy = xmemdupz(archiver_args.v, - sizeof(char *) * archiver_args.nr); - res = write_archive(archiver_args.nr, (const char **)argv_copy, NULL, - the_repository, NULL, 0); - if (res) { - error(_("failed to write archive")); - goto diagnose_cleanup; - } - if (!res) - fprintf(stderr, "\n" - "Diagnostics complete.\n" - "All of the gathered info is captured in '%s'\n", - zip_path.buf); + res = create_diagnostics_archive(&zip_path); diagnose_cleanup: - if (archiver_fd >= 0) { - close(1); - dup2(stdout_fd, 1); - } - free(argv_copy); - strvec_clear(&archiver_args); strbuf_release(&zip_path); - strbuf_release(&buf); - return res; } diff --git a/diagnose.c b/diagnose.c new file mode 100644 index 00000000000..f0dcbfe1a2a --- /dev/null +++ b/diagnose.c @@ -0,0 +1,216 @@ +#include "cache.h" +#include "diagnose.h" +#include "compat/disk.h" +#include "archive.h" +#include "dir.h" +#include "help.h" +#include "strvec.h" +#include "object-store.h" +#include "packfile.h" + +static void dir_file_stats_objects(const char *full_path, size_t full_path_len, + const char *file_name, void *data) +{ + struct strbuf *buf = data; + struct stat st; + + if (!stat(full_path, &st)) + strbuf_addf(buf, "%-70s %16" PRIuMAX "\n", file_name, + (uintmax_t)st.st_size); +} + +static int dir_file_stats(struct object_directory *object_dir, void *data) +{ + struct strbuf *buf = data; + + strbuf_addf(buf, "Contents of %s:\n", object_dir->path); + + for_each_file_in_pack_dir(object_dir->path, dir_file_stats_objects, + data); + + return 0; +} + +static int count_files(char *path) +{ + DIR *dir = opendir(path); + struct dirent *e; + int count = 0; + + if (!dir) + return 0; + + while ((e = readdir(dir)) != NULL) + if (!is_dot_or_dotdot(e->d_name) && e->d_type == DT_REG) + count++; + + closedir(dir); + return count; +} + +static void loose_objs_stats(struct strbuf *buf, const char *path) +{ + DIR *dir = opendir(path); + struct dirent *e; + int count; + int total = 0; + unsigned char c; + struct strbuf count_path = STRBUF_INIT; + size_t base_path_len; + + if (!dir) + return; + + strbuf_addstr(buf, "Object directory stats for "); + strbuf_add_absolute_path(buf, path); + strbuf_addstr(buf, ":\n"); + + strbuf_add_absolute_path(&count_path, path); + strbuf_addch(&count_path, '/'); + base_path_len = count_path.len; + + while ((e = readdir(dir)) != NULL) + if (!is_dot_or_dotdot(e->d_name) && + e->d_type == DT_DIR && strlen(e->d_name) == 2 && + !hex_to_bytes(&c, e->d_name, 1)) { + strbuf_setlen(&count_path, base_path_len); + strbuf_addstr(&count_path, e->d_name); + total += (count = count_files(count_path.buf)); + strbuf_addf(buf, "%s : %7d files\n", e->d_name, count); + } + + strbuf_addf(buf, "Total: %d loose objects", total); + + strbuf_release(&count_path); + closedir(dir); +} + +static int add_directory_to_archiver(struct strvec *archiver_args, + const char *path, int recurse) +{ + int at_root = !*path; + DIR *dir; + struct dirent *e; + struct strbuf buf = STRBUF_INIT; + size_t len; + int res = 0; + + dir = opendir(at_root ? "." : path); + if (!dir) { + if (errno == ENOENT) { + warning(_("could not archive missing directory '%s'"), path); + return 0; + } + return error_errno(_("could not open directory '%s'"), path); + } + + if (!at_root) + strbuf_addf(&buf, "%s/", path); + len = buf.len; + strvec_pushf(archiver_args, "--prefix=%s", buf.buf); + + while (!res && (e = readdir(dir))) { + if (!strcmp(".", e->d_name) || !strcmp("..", e->d_name)) + continue; + + strbuf_setlen(&buf, len); + strbuf_addstr(&buf, e->d_name); + + if (e->d_type == DT_REG) + strvec_pushf(archiver_args, "--add-file=%s", buf.buf); + else if (e->d_type != DT_DIR) + warning(_("skipping '%s', which is neither file nor " + "directory"), buf.buf); + else if (recurse && + add_directory_to_archiver(archiver_args, + buf.buf, recurse) < 0) + res = -1; + } + + closedir(dir); + strbuf_release(&buf); + return res; +} + +int create_diagnostics_archive(struct strbuf *zip_path) +{ + struct strvec archiver_args = STRVEC_INIT; + char **argv_copy = NULL; + int stdout_fd = -1, archiver_fd = -1; + struct strbuf buf = STRBUF_INIT; + int res; + + stdout_fd = dup(STDOUT_FILENO); + if (stdout_fd < 0) { + res = error_errno(_("could not duplicate stdout")); + goto diagnose_cleanup; + } + + archiver_fd = xopen(zip_path->buf, O_CREAT | O_WRONLY | O_TRUNC, 0666); + if (dup2(archiver_fd, STDOUT_FILENO) < 0) { + res = error_errno(_("could not redirect output")); + goto diagnose_cleanup; + } + + init_zip_archiver(); + strvec_pushl(&archiver_args, "git-diagnose", "--format=zip", NULL); + + strbuf_reset(&buf); + strbuf_addstr(&buf, "Collecting diagnostic info\n\n"); + get_version_info(&buf, 1); + + strbuf_addf(&buf, "Repository root: %s\n", the_repository->worktree); + get_disk_info(&buf); + write_or_die(stdout_fd, buf.buf, buf.len); + strvec_pushf(&archiver_args, + "--add-virtual-file=diagnostics.log:%.*s", + (int)buf.len, buf.buf); + + strbuf_reset(&buf); + strbuf_addstr(&buf, "--add-virtual-file=packs-local.txt:"); + dir_file_stats(the_repository->objects->odb, &buf); + foreach_alt_odb(dir_file_stats, &buf); + strvec_push(&archiver_args, buf.buf); + + strbuf_reset(&buf); + strbuf_addstr(&buf, "--add-virtual-file=objects-local.txt:"); + loose_objs_stats(&buf, ".git/objects"); + strvec_push(&archiver_args, buf.buf); + + if ((res = add_directory_to_archiver(&archiver_args, ".git", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/hooks", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/info", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/logs", 1)) || + (res = add_directory_to_archiver(&archiver_args, ".git/objects/info", 0))) + goto diagnose_cleanup; + + strvec_pushl(&archiver_args, "--prefix=", + oid_to_hex(the_hash_algo->empty_tree), "--", NULL); + + /* `write_archive()` modifies the `argv` passed to it. Let it. */ + argv_copy = xmemdupz(archiver_args.v, + sizeof(char *) * archiver_args.nr); + res = write_archive(archiver_args.nr, (const char **)argv_copy, NULL, + the_repository, NULL, 0); + if (res) { + error(_("failed to write archive")); + goto diagnose_cleanup; + } + + fprintf(stderr, "\n" + "Diagnostics complete.\n" + "All of the gathered info is captured in '%s'\n", + zip_path->buf); + +diagnose_cleanup: + if (archiver_fd >= 0) { + dup2(stdout_fd, STDOUT_FILENO); + close(stdout_fd); + close(archiver_fd); + } + free(argv_copy); + strvec_clear(&archiver_args); + strbuf_release(&buf); + + return res; +} diff --git a/diagnose.h b/diagnose.h new file mode 100644 index 00000000000..06dca69bdac --- /dev/null +++ b/diagnose.h @@ -0,0 +1,8 @@ +#ifndef DIAGNOSE_H +#define DIAGNOSE_H + +#include "strbuf.h" + +int create_diagnostics_archive(struct strbuf *zip_path); + +#endif /* DIAGNOSE_H */ From patchwork Fri Aug 12 20:10:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942502 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82D91C25B0F for ; Fri, 12 Aug 2022 20:10:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235295AbiHLUKm (ORCPT ); Fri, 12 Aug 2022 16:10:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235083AbiHLUK3 (ORCPT ); Fri, 12 Aug 2022 16:10:29 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E284D7672 for ; Fri, 12 Aug 2022 13:10:28 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id k17so1046509wmr.2 for ; Fri, 12 Aug 2022 13:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=5XpJGv8eluRhP8y14ld5we3CaQHJVLkyzbbj70llQiA=; b=ZEREHJ6v7txLtL15aYoa/LlDNy299z6YuVHLNDZOGlu5A3BoFgfDbHG90QTyte5vcq kRBMgVGZ0d+Hhq7Xz75DMFPLVr12W74g+jaoMNxU5hJrWFgKXI3EB9r3MHVznwGJ6wOq At89oIo463VLFJBJ8xvUFKh5b3MLB2GeksjuJQ2iW2GwmB8Mbte/ymU4455zxPdMcykA Nls8O0HiuFeuvxJvN5vV8NG0NPacgfVthgc2Qb++02zIJ81kOOaC1ORbaM4Oly4r2vwp K2dk9sQ/tYZRuXy/eDqS3S10ysVCWFXvx7leNiUDTtJ4NL97CpXOyCZgsTi5yy1q/cqG qjOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=5XpJGv8eluRhP8y14ld5we3CaQHJVLkyzbbj70llQiA=; b=lrL491x9Qahd4U/PhJykCcZs7DSGiab84Mgul5L7z8xWDPRIjY527UQmFxVpWN1JMV 8eWOayLwEYSRJ5S2XDQ2Imi8CO3or2b4/Yifm9dImZT/mEC8TpfjOntO24PLL/uMmlds jv+VJfD6YJXUgiL72/WPoiBmTfxPAXQdarZ+/3kbt6eyOe/rhrkA2To1LxT3O+ZFTSHJ 0gIM93oNaz8bTcgc2wFpQzJLgb9G0TWwIOddvczPxTW8b2OghJ+qqVDupu2/5l676EIa lidcyZkYsW4Tjt1iUqqNmPD7vE1dmHI2aIMPja4R257gmYVcQCI4u4U5R/V3hZX86TdK yCKQ== X-Gm-Message-State: ACgBeo30FnbbvGpFSkWVf+lfaLUkeTbZiG8R1ZZtQnd77ot0rn1uNpGe xjWW5SlRBzN/w0FXj5zD9vtywTgsj3Y= X-Google-Smtp-Source: AA6agR6Ouq0TfmtljvAzL1S6zPeVxUMeSuWkTEf1g9Ss7KGcXkCahrCiUyJISyPdn67+HURRQEjHdA== X-Received: by 2002:a7b:c453:0:b0:3a5:b42e:c4fb with SMTP id l19-20020a7bc453000000b003a5b42ec4fbmr3596521wmi.167.1660335027119; Fri, 12 Aug 2022 13:10:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h8-20020a05600c350800b003a325bd8517sm744438wmq.5.2022.08.12.13.10.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:26 -0700 (PDT) Message-Id: <710b67e5776363d199ed5043d019386819d44e7e.1660335019.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:14 +0000 Subject: [PATCH v4 06/11] diagnose.c: add option to configure archive contents Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Update 'create_diagnostics_archive()' to take an argument 'mode'. When archiving diagnostics for a repository, 'mode' is used to selectively include/exclude information based on its value. The initial options for 'mode' are: * DIAGNOSE_NONE: do not collect any diagnostics or create an archive (no-op). * DIAGNOSE_STATS: collect basic repository metadata (Git version, repo path, filesystem available space) as well as sizing and count statistics for the repository's objects and packfiles. * DIAGNOSE_ALL: collect basic repository metadata, sizing/count statistics, and copies of the '.git', '.git/hooks', '.git/info', '.git/logs', and '.git/objects/info' directories. These modes are introduced to provide users the option to collect diagnostics without the sensitive information included in copies of '.git' dir contents. At the moment, only 'scalar diagnose' uses 'create_diagnostics_archive()' (with a hardcoded 'DIAGNOSE_ALL' mode to match existing functionality), but more callers will be introduced in subsequent patches. Finally, refactor from a hardcoded set of 'add_directory_to_archiver()' calls to iterative invocations gated by 'DIAGNOSE_ALL'. This allows for easier future modification of the set of directories to archive and improves error reporting when 'add_directory_to_archiver()' fails. Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- contrib/scalar/scalar.c | 2 +- diagnose.c | 39 +++++++++++++++++++++++++++++++-------- diagnose.h | 8 +++++++- 3 files changed, 39 insertions(+), 10 deletions(-) diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index 3983def760a..d538b8b8f14 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -534,7 +534,7 @@ static int cmd_diagnose(int argc, const char **argv) goto diagnose_cleanup; } - res = create_diagnostics_archive(&zip_path); + res = create_diagnostics_archive(&zip_path, DIAGNOSE_ALL); diagnose_cleanup: strbuf_release(&zip_path); diff --git a/diagnose.c b/diagnose.c index f0dcbfe1a2a..9270056db2f 100644 --- a/diagnose.c +++ b/diagnose.c @@ -8,6 +8,11 @@ #include "object-store.h" #include "packfile.h" +struct archive_dir { + const char *path; + int recursive; +}; + static void dir_file_stats_objects(const char *full_path, size_t full_path_len, const char *file_name, void *data) { @@ -132,13 +137,25 @@ static int add_directory_to_archiver(struct strvec *archiver_args, return res; } -int create_diagnostics_archive(struct strbuf *zip_path) +int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode) { struct strvec archiver_args = STRVEC_INIT; char **argv_copy = NULL; int stdout_fd = -1, archiver_fd = -1; struct strbuf buf = STRBUF_INIT; - int res; + int res, i; + struct archive_dir archive_dirs[] = { + { ".git", 0 }, + { ".git/hooks", 0 }, + { ".git/info", 0 }, + { ".git/logs", 1 }, + { ".git/objects/info", 0 } + }; + + if (mode == DIAGNOSE_NONE) { + res = 0; + goto diagnose_cleanup; + } stdout_fd = dup(STDOUT_FILENO); if (stdout_fd < 0) { @@ -177,12 +194,18 @@ int create_diagnostics_archive(struct strbuf *zip_path) loose_objs_stats(&buf, ".git/objects"); strvec_push(&archiver_args, buf.buf); - if ((res = add_directory_to_archiver(&archiver_args, ".git", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/hooks", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/info", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/logs", 1)) || - (res = add_directory_to_archiver(&archiver_args, ".git/objects/info", 0))) - goto diagnose_cleanup; + /* Only include this if explicitly requested */ + if (mode == DIAGNOSE_ALL) { + for (i = 0; i < ARRAY_SIZE(archive_dirs); i++) { + if (add_directory_to_archiver(&archiver_args, + archive_dirs[i].path, + archive_dirs[i].recursive)) { + res = error_errno(_("could not add directory '%s' to archiver"), + archive_dirs[i].path); + goto diagnose_cleanup; + } + } + } strvec_pushl(&archiver_args, "--prefix=", oid_to_hex(the_hash_algo->empty_tree), "--", NULL); diff --git a/diagnose.h b/diagnose.h index 06dca69bdac..998775857a0 100644 --- a/diagnose.h +++ b/diagnose.h @@ -3,6 +3,12 @@ #include "strbuf.h" -int create_diagnostics_archive(struct strbuf *zip_path); +enum diagnose_mode { + DIAGNOSE_NONE, + DIAGNOSE_STATS, + DIAGNOSE_ALL +}; + +int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode); #endif /* DIAGNOSE_H */ From patchwork Fri Aug 12 20:10:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0470C00140 for ; Fri, 12 Aug 2022 20:10:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235854AbiHLUKs (ORCPT ); Fri, 12 Aug 2022 16:10:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234710AbiHLUKb (ORCPT ); Fri, 12 Aug 2022 16:10:31 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 24A4B9594 for ; Fri, 12 Aug 2022 13:10:28 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id h204-20020a1c21d5000000b003a5b467c3abso4249737wmh.5 for ; Fri, 12 Aug 2022 13:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=3DtPvUJ+3SPdT6od1QbO+EADYby4QM2OS5TX2k4hwJY=; b=aoM7+JcNyutYtcacEjpFDOblJhAkysbiNrc7kIwjlgw37vP5YP3U3JF+/LDoaQoKl0 0my5/nbV1VgtNaJhmuoQWlOtPlhByOMHaBSYWKqkMCDzkoD21AjrwOXbfxWG+4/YWhAs 7W3ev7MzwbvcF2oX7FVCFAtVPDflTQJRDmAcMSVcF0gu0ycVNb1cX5ja9QhUZzH0lXGC mKIq5m5OvBrA7MtGMd0QlZN+GnwD1r6um1GDvosWvLcvW68QR1lNwP5ZDnuHPu/umyEv 0/m+zkSB1VasyQ+6qdcIFWUk/yiigqykhMlR+avgzlnDt6Zh+xSWs97dXtpXZ4Ht/XL6 JhtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=3DtPvUJ+3SPdT6od1QbO+EADYby4QM2OS5TX2k4hwJY=; b=UuHx2aSuwT51O/DpGr+wICi7DoompUPtXn7257eswAEUgixOpxP2+B7oOsXqg/EK00 Rz6Iv3RaYBYPgaaOO75wP9a0X5bwSFHha98DTVQMVZIldN0qmQaNswD3pq70s8Crmzg7 HCqMjGz5ufemqwcJzDmHSq8GR9y+1zH4KpfEmB5HvcNVIBkQR2VEsEL2f7Rn1AldTwvv CdG1PHF4xEy+7I8gCtQ6ebK7yoATjhgTXxIre/ifrYR3kkWiBzWlW6ksFljbVePanyiR Wl/NWAFyu4L9hetWyjiL7wnAveDI9UnK+DDbQaEfHe9gv73PuTBqa5amFhjNqt7Yzqg4 kABg== X-Gm-Message-State: ACgBeo3NTyR6ZXCcFfSh55/k3JaPMn3/ShDYifiga+LGDj3pzcOPzHhE l0cUAI2ESg9UJ85fjYux7qOnFoXmZd0= X-Google-Smtp-Source: AA6agR79EB2GXQ2xD5BEh/+KaHp/H9XF/Egd0dg5L9dasDQqkQqENpKy3dPMA81mK1jGRS2Co3OaZg== X-Received: by 2002:a7b:c354:0:b0:39c:6753:21f8 with SMTP id l20-20020a7bc354000000b0039c675321f8mr3663214wmj.113.1660335028135; Fri, 12 Aug 2022 13:10:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f13-20020a05600c154d00b003a52969e89csm673452wmg.4.2022.08.12.13.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:27 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:15 +0000 Subject: [PATCH v4 07/11] builtin/diagnose.c: create 'git diagnose' builtin MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Create a 'git diagnose' builtin to generate a standalone zip archive of repository diagnostics. The "diagnose" functionality was originally implemented for Scalar in aa5c79a331 (scalar: implement `scalar diagnose`, 2022-05-28). However, the diagnostics gathered are not specific to Scalar-cloned repositories and can be useful when diagnosing issues in any Git repository. Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- .gitignore | 1 + Documentation/git-diagnose.txt | 50 +++++++++++++++++++++++++++++ Makefile | 1 + builtin.h | 1 + builtin/diagnose.c | 57 ++++++++++++++++++++++++++++++++++ git.c | 1 + t/t0092-diagnose.sh | 32 +++++++++++++++++++ 7 files changed, 143 insertions(+) create mode 100644 Documentation/git-diagnose.txt create mode 100644 builtin/diagnose.c create mode 100755 t/t0092-diagnose.sh diff --git a/.gitignore b/.gitignore index 42fd7253b44..80b530bbed2 100644 --- a/.gitignore +++ b/.gitignore @@ -53,6 +53,7 @@ /git-cvsimport /git-cvsserver /git-daemon +/git-diagnose /git-diff /git-diff-files /git-diff-index diff --git a/Documentation/git-diagnose.txt b/Documentation/git-diagnose.txt new file mode 100644 index 00000000000..ce07dd0725d --- /dev/null +++ b/Documentation/git-diagnose.txt @@ -0,0 +1,50 @@ +git-diagnose(1) +================ + +NAME +---- +git-diagnose - Generate a zip archive of diagnostic information + +SYNOPSIS +-------- +[verse] +'git diagnose' [(-o | --output-directory) ] [(-s | --suffix) ] + +DESCRIPTION +----------- +Collects detailed information about the user's machine, Git client, and +repository state and packages that information into a zip archive. The +generated archive can then, for example, be shared with the Git mailing list to +help debug an issue or serve as a reference for independent debugging. + +The following information is captured in the archive: + + * 'git version --build-options' + * The path to the repository root + * The available disk space on the filesystem + * The name and size of each packfile, including those in alternate object + stores + * The total count of loose objects, as well as counts broken down by + `.git/objects` subdirectory + +This tool differs from linkgit:git-bugreport[1] in that it collects much more +detailed information with a greater focus on reporting the size and data shape +of repository contents. + +OPTIONS +------- +-o :: +--output-directory :: + Place the resulting diagnostics archive in `` instead of the + current directory. + +-s :: +--suffix :: + Specify an alternate suffix for the diagnostics archive name, to create + a file named 'git-diagnostics-'. This should take the + form of a strftime(3) format string; the current local time will be + used. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Makefile b/Makefile index ed66cb70e5a..d34f680c065 100644 --- a/Makefile +++ b/Makefile @@ -1153,6 +1153,7 @@ BUILTIN_OBJS += builtin/credential-cache.o BUILTIN_OBJS += builtin/credential-store.o BUILTIN_OBJS += builtin/credential.o BUILTIN_OBJS += builtin/describe.o +BUILTIN_OBJS += builtin/diagnose.o BUILTIN_OBJS += builtin/diff-files.o BUILTIN_OBJS += builtin/diff-index.o BUILTIN_OBJS += builtin/diff-tree.o diff --git a/builtin.h b/builtin.h index 40e9ecc8485..8901a34d6bf 100644 --- a/builtin.h +++ b/builtin.h @@ -144,6 +144,7 @@ int cmd_credential_cache(int argc, const char **argv, const char *prefix); int cmd_credential_cache_daemon(int argc, const char **argv, const char *prefix); int cmd_credential_store(int argc, const char **argv, const char *prefix); int cmd_describe(int argc, const char **argv, const char *prefix); +int cmd_diagnose(int argc, const char **argv, const char *prefix); int cmd_diff_files(int argc, const char **argv, const char *prefix); int cmd_diff_index(int argc, const char **argv, const char *prefix); int cmd_diff(int argc, const char **argv, const char *prefix); diff --git a/builtin/diagnose.c b/builtin/diagnose.c new file mode 100644 index 00000000000..832493bba65 --- /dev/null +++ b/builtin/diagnose.c @@ -0,0 +1,57 @@ +#include "builtin.h" +#include "parse-options.h" +#include "diagnose.h" + +static const char * const diagnose_usage[] = { + N_("git diagnose [-o|--output-directory ] [-s|--suffix ]"), + NULL +}; + +int cmd_diagnose(int argc, const char **argv, const char *prefix) +{ + struct strbuf zip_path = STRBUF_INIT; + time_t now = time(NULL); + struct tm tm; + char *option_output = NULL; + char *option_suffix = "%Y-%m-%d-%H%M"; + char *prefixed_filename; + + const struct option diagnose_options[] = { + OPT_STRING('o', "output-directory", &option_output, N_("path"), + N_("specify a destination for the diagnostics archive")), + OPT_STRING('s', "suffix", &option_suffix, N_("format"), + N_("specify a strftime format suffix for the filename")), + OPT_END() + }; + + argc = parse_options(argc, argv, prefix, diagnose_options, + diagnose_usage, 0); + + /* Prepare the path to put the result */ + prefixed_filename = prefix_filename(prefix, + option_output ? option_output : ""); + strbuf_addstr(&zip_path, prefixed_filename); + strbuf_complete(&zip_path, '/'); + + strbuf_addstr(&zip_path, "git-diagnostics-"); + strbuf_addftime(&zip_path, option_suffix, localtime_r(&now, &tm), 0, 0); + strbuf_addstr(&zip_path, ".zip"); + + switch (safe_create_leading_directories(zip_path.buf)) { + case SCLD_OK: + case SCLD_EXISTS: + break; + default: + die_errno(_("could not create leading directories for '%s'"), + zip_path.buf); + } + + /* Prepare diagnostics */ + if (create_diagnostics_archive(&zip_path, DIAGNOSE_STATS)) + die_errno(_("unable to create diagnostics archive %s"), + zip_path.buf); + + free(prefixed_filename); + strbuf_release(&zip_path); + return 0; +} diff --git a/git.c b/git.c index e5d62fa5a92..0b9d8ef7677 100644 --- a/git.c +++ b/git.c @@ -522,6 +522,7 @@ static struct cmd_struct commands[] = { { "credential-cache--daemon", cmd_credential_cache_daemon }, { "credential-store", cmd_credential_store }, { "describe", cmd_describe, RUN_SETUP }, + { "diagnose", cmd_diagnose, RUN_SETUP_GENTLY }, { "diff", cmd_diff, NO_PARSEOPT }, { "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT }, { "diff-index", cmd_diff_index, RUN_SETUP | NO_PARSEOPT }, diff --git a/t/t0092-diagnose.sh b/t/t0092-diagnose.sh new file mode 100755 index 00000000000..b6923726fd7 --- /dev/null +++ b/t/t0092-diagnose.sh @@ -0,0 +1,32 @@ +#!/bin/sh + +test_description='git diagnose' + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh + +test_expect_success UNZIP 'creates diagnostics zip archive' ' + test_when_finished rm -rf report && + + git diagnose -o report -s test >out && + grep "Available space" out && + + zip_path=report/git-diagnostics-test.zip && + test_path_is_file "$zip_path" && + + # Check zipped archive content + "$GIT_UNZIP" -p "$zip_path" diagnostics.log >out && + test_file_not_empty out && + + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + "$GIT_UNZIP" -p "$zip_path" objects-local.txt >out && + grep "^Total: [0-9][0-9]*" out && + + # Should not include .git directory contents by default + ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" + grep "^Total: [0-9][0-9]*" out +' + +test_done From patchwork Fri Aug 12 20:10:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942506 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B66CC00140 for ; Fri, 12 Aug 2022 20:11:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235083AbiHLULB (ORCPT ); Fri, 12 Aug 2022 16:11:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235476AbiHLUKg (ORCPT ); Fri, 12 Aug 2022 16:10:36 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C472DB1C7 for ; Fri, 12 Aug 2022 13:10:30 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id z16so2233733wrh.12 for ; Fri, 12 Aug 2022 13:10:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=pj2xyvIUYdcDH2UsS/0PcbbfCcss9zcoh3OIoFN9ukY=; b=gG1KVkd3Sdva0m5hvGfJLEFyzSZVAF+ipX/w+fjThLV/6yQOktH258t9WJZ7L3iTzd txPaW30iW6Ld+s7OnpZzVxAG7N/KrZg692Oes76u5ypqvVVNcPVhWLmK4Yrit+tnQn6a JSyAk0SfVyAXLfO+oMl7Tu5zt510zkcjI4/NszjP25dePRkNmU3wiD2HObLf6BiuRd0n xBKA9du8MulEFDAxMkCcP+/nzD1m2gFpsDvnlKU1+vWRsHDxfXv7Qy11NLC7sYB2JCDs rlhle6OkirLC+DP/cH0AenVwN/BPynzEbzZ46gP+o/ngUyzFkdNYnQGjYW399kASUo4O qiRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=pj2xyvIUYdcDH2UsS/0PcbbfCcss9zcoh3OIoFN9ukY=; b=2fVnlHEmRIz2XKOuCyruddTpSezqBOuie19TsjU/cJelzU6HFkngDJWb70Jk4jdaAO BDxEXPeEYwhvqDra+MZYBDDWnAiRhoqJ6ykxxYwNXEJjSeHum7sbJkVAGSQT1wUoQTvi YqUlyM+stu5UN6URj3ewxDyPAgwb4SxSxkroWyalz67nHFMSRXeVBX4KsiISyE1JbCEV xcP8vVtBNcm2Ji/pdzb/Ik55SuuYdPBM3HXHLySnH1cquMg1ob/LsYKE9nneQgu17bit 09Sj30hxvr+7UwT4M8IGXylmLNZV+Bbugp0hhCyeHLgasB154QvAu2X8vbwKpNMpoZBb AlMA== X-Gm-Message-State: ACgBeo1TCEW8KY4uQD4uiYUghfjmZ4ZMKOsAa/hxaP6hHqsk1s8aR1Ww hhSEi5OE5fzbxPfkLXdpjwmiTLt0w7c= X-Google-Smtp-Source: AA6agR7HQI4BL9/mSlTagcio+5Iiral4JVVPMedcneSmgAtJ3whKayEO+iqWV3/4f7tp6rgqdx1d+Q== X-Received: by 2002:a5d:5223:0:b0:21e:fc10:b818 with SMTP id i3-20020a5d5223000000b0021efc10b818mr2976554wra.676.1660335029016; Fri, 12 Aug 2022 13:10:29 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j36-20020a05600c1c2400b003a4c6e67f01sm594523wms.6.2022.08.12.13.10.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:28 -0700 (PDT) Message-Id: <82be069e5e965c7193f15bb38ef523dc4b7b3592.1660335019.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:16 +0000 Subject: [PATCH v4 08/11] builtin/diagnose.c: add '--mode' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Create '--mode=' option in 'git diagnose' to allow users to optionally select non-default diagnostic information to include in the output archive. Additionally, document the currently-available modes, emphasizing the importance of not sharing a '--mode=all' archive publicly due to the presence of sensitive information. Note that the option parsing callback - 'option_parse_diagnose()' - is added to 'diagnose.c' rather than 'builtin/diagnose.c' so that it may be reused in future callers configuring a diagnostics archive. Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- Documentation/git-diagnose.txt | 17 ++++++++++++++++- builtin/diagnose.c | 8 ++++++-- diagnose.c | 30 ++++++++++++++++++++++++++++++ diagnose.h | 3 +++ t/t0092-diagnose.sh | 30 +++++++++++++++++++++++++++++- 5 files changed, 84 insertions(+), 4 deletions(-) diff --git a/Documentation/git-diagnose.txt b/Documentation/git-diagnose.txt index ce07dd0725d..3ec8cc7ad72 100644 --- a/Documentation/git-diagnose.txt +++ b/Documentation/git-diagnose.txt @@ -9,6 +9,7 @@ SYNOPSIS -------- [verse] 'git diagnose' [(-o | --output-directory) ] [(-s | --suffix) ] + [--mode=] DESCRIPTION ----------- @@ -17,7 +18,7 @@ repository state and packages that information into a zip archive. The generated archive can then, for example, be shared with the Git mailing list to help debug an issue or serve as a reference for independent debugging. -The following information is captured in the archive: +By default, the following information is captured in the archive: * 'git version --build-options' * The path to the repository root @@ -27,6 +28,9 @@ The following information is captured in the archive: * The total count of loose objects, as well as counts broken down by `.git/objects` subdirectory +Additional information can be collected by selecting a different diagnostic mode +using the `--mode` option. + This tool differs from linkgit:git-bugreport[1] in that it collects much more detailed information with a greater focus on reporting the size and data shape of repository contents. @@ -45,6 +49,17 @@ OPTIONS form of a strftime(3) format string; the current local time will be used. +--mode=(stats|all):: + Specify the type of diagnostics that should be collected. The default behavior + of 'git diagnose' is equivalent to `--mode=stats`. ++ +The `--mode=all` option collects everything included in `--mode=stats`, as well +as copies of `.git`, `.git/hooks`, `.git/info`, `.git/logs`, and +`.git/objects/info` directories. This additional information may be sensitive, +as it can be used to reconstruct the full contents of the diagnosed repository. +Users should exercise caution when sharing an archive generated with +`--mode=all`. + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/diagnose.c b/builtin/diagnose.c index 832493bba65..cd260c20155 100644 --- a/builtin/diagnose.c +++ b/builtin/diagnose.c @@ -3,7 +3,7 @@ #include "diagnose.h" static const char * const diagnose_usage[] = { - N_("git diagnose [-o|--output-directory ] [-s|--suffix ]"), + N_("git diagnose [-o|--output-directory ] [-s|--suffix ] [--mode=]"), NULL }; @@ -12,6 +12,7 @@ int cmd_diagnose(int argc, const char **argv, const char *prefix) struct strbuf zip_path = STRBUF_INIT; time_t now = time(NULL); struct tm tm; + enum diagnose_mode mode = DIAGNOSE_STATS; char *option_output = NULL; char *option_suffix = "%Y-%m-%d-%H%M"; char *prefixed_filename; @@ -21,6 +22,9 @@ int cmd_diagnose(int argc, const char **argv, const char *prefix) N_("specify a destination for the diagnostics archive")), OPT_STRING('s', "suffix", &option_suffix, N_("format"), N_("specify a strftime format suffix for the filename")), + OPT_CALLBACK_F(0, "mode", &mode, N_("(stats|all)"), + N_("specify the content of the diagnostic archive"), + PARSE_OPT_NONEG, option_parse_diagnose), OPT_END() }; @@ -47,7 +51,7 @@ int cmd_diagnose(int argc, const char **argv, const char *prefix) } /* Prepare diagnostics */ - if (create_diagnostics_archive(&zip_path, DIAGNOSE_STATS)) + if (create_diagnostics_archive(&zip_path, mode)) die_errno(_("unable to create diagnostics archive %s"), zip_path.buf); diff --git a/diagnose.c b/diagnose.c index 9270056db2f..beb0a8741ba 100644 --- a/diagnose.c +++ b/diagnose.c @@ -13,6 +13,36 @@ struct archive_dir { int recursive; }; +struct diagnose_option { + enum diagnose_mode mode; + const char *option_name; +}; + +static struct diagnose_option diagnose_options[] = { + { DIAGNOSE_STATS, "stats" }, + { DIAGNOSE_ALL, "all" }, +}; + +int option_parse_diagnose(const struct option *opt, const char *arg, int unset) +{ + int i; + enum diagnose_mode *diagnose = opt->value; + + if (!arg) { + *diagnose = unset ? DIAGNOSE_NONE : DIAGNOSE_STATS; + return 0; + } + + for (i = 0; i < ARRAY_SIZE(diagnose_options); i++) { + if (!strcmp(arg, diagnose_options[i].option_name)) { + *diagnose = diagnose_options[i].mode; + return 0; + } + } + + return error(_("invalid --%s value '%s'"), opt->long_name, arg); +} + static void dir_file_stats_objects(const char *full_path, size_t full_path_len, const char *file_name, void *data) { diff --git a/diagnose.h b/diagnose.h index 998775857a0..7a4951a7863 100644 --- a/diagnose.h +++ b/diagnose.h @@ -2,6 +2,7 @@ #define DIAGNOSE_H #include "strbuf.h" +#include "parse-options.h" enum diagnose_mode { DIAGNOSE_NONE, @@ -9,6 +10,8 @@ enum diagnose_mode { DIAGNOSE_ALL }; +int option_parse_diagnose(const struct option *opt, const char *arg, int unset); + int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode); #endif /* DIAGNOSE_H */ diff --git a/t/t0092-diagnose.sh b/t/t0092-diagnose.sh index b6923726fd7..fca9b58489c 100755 --- a/t/t0092-diagnose.sh +++ b/t/t0092-diagnose.sh @@ -26,7 +26,35 @@ test_expect_success UNZIP 'creates diagnostics zip archive' ' # Should not include .git directory contents by default ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" - grep "^Total: [0-9][0-9]*" out +' + +test_expect_success UNZIP '--mode=stats excludes .git dir contents' ' + test_when_finished rm -rf report && + + git diagnose -o report -s test --mode=stats >out && + + # Includes pack quantity/size info + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + # Does not include .git directory contents + ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" +' + +test_expect_success UNZIP '--mode=all includes .git dir contents' ' + test_when_finished rm -rf report && + + git diagnose -o report -s test --mode=all >out && + + # Includes pack quantity/size info + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + # Includes .git directory contents + "$GIT_UNZIP" -l "$zip_path" | grep ".git/" && + + "$GIT_UNZIP" -p "$zip_path" .git/HEAD >out && + test_file_not_empty out ' test_done From patchwork Fri Aug 12 20:10:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942504 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AB09C00140 for ; Fri, 12 Aug 2022 20:10:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236627AbiHLUK5 (ORCPT ); Fri, 12 Aug 2022 16:10:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235487AbiHLUKg (ORCPT ); Fri, 12 Aug 2022 16:10:36 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18C6DBF70 for ; Fri, 12 Aug 2022 13:10:31 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id q30so2234058wra.11 for ; Fri, 12 Aug 2022 13:10:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=mARxQC56HROeHfnGhKZI2tZ091Okd3Z5usLlYR6Qcug=; b=E25iyBYPgiMAkT28b3wtZ/e5hPa8+GEMpzXuQGvbkfb+C21Jc3OdMmicUK2+h8e/ZZ gCOaitR9QOJmPpuxF+Kdnk9gijzn3dqkN/ijWx1+pkbSUXEh8cVYo6N6X8ph7oHNIM+V MD4SzwKHOoa7NaREwgR7MGpC9pfsE3wj+RlBUmXAtEroezvglZbz8yr/iTRJsSX0xjs4 G+YqG1ZwBHC2IIVgVpTDfs/A7E2NrZ9w9c2NMx+cLlORucNihYDkvGk4HU58Sosf3oAS njgGynm08SFuvVmpVW8X+q67MogPJhvmD6YzTUByg4Hro9LScpedKw4pxPWLMOe9wOH5 WXcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=mARxQC56HROeHfnGhKZI2tZ091Okd3Z5usLlYR6Qcug=; b=yxi1xIHMQ5WQQ3t+AxwSvIxYRKBdp6txiJYk6b1jsBSDUNdsk1kv8ZqaW7X8KVW9YO ptktBqeUTWDX8pVm0f1xwA/URtsEuF7l95IFL/Kne1a4v8/MHVun0hvClm1Pxt7ndX5j YL8IEXs3hfBvQmnWS3fEZXtvM6k9fwCQYttf5yyAVvMLsn2Bx82SQDLhIz8wJFrxObRQ J5k8fprYVyPbEK5zTxDrxtd+6tst50sqMtQiUxn3FBj9kKJUSfIbR24wBRCxVlrWWbba hgfV9KiOJLtY7XkDV6npFWC2wXYfF18qz8KQnqfJCaj5awf5ojFKmewkyjXlWp7ijLG+ PJ7w== X-Gm-Message-State: ACgBeo2uVL3T7purd9klCCiS3HGtV3Ip3cUsqIlSFseIhyvJzZlp9k97 SNNjy4mpSEEA48ojj0bSftTLKzJS+Qo= X-Google-Smtp-Source: AA6agR4YbplWWnjqOvbin+Zetilc2dlLASgl/o8yPbWZqrmByhchJcgsc4ujGPxatFXBGePEESInsw== X-Received: by 2002:a05:6000:811:b0:220:6262:ac66 with SMTP id bt17-20020a056000081100b002206262ac66mr2809392wrb.529.1660335030055; Fri, 12 Aug 2022 13:10:30 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f4-20020a1c6a04000000b003a54f1d007csm534916wmc.10.2022.08.12.13.10.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:29 -0700 (PDT) Message-Id: <718e3f434841ee4697a505d83f2e4acbeed364fd.1660335019.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:17 +0000 Subject: [PATCH v4 09/11] builtin/bugreport.c: create '--diagnose' option MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Create a '--diagnose' option for 'git bugreport' to collect additional information about the repository and write it to a zipped archive. The '--diagnose' option behaves effectively as an alias for simultaneously running 'git bugreport' and 'git diagnose'. In the documentation, users are explicitly recommended to attach the diagnostics alongside a bug report to provide additional context to readers, ideally reducing some back-and-forth between reporters and those debugging the issue. Note that '--diagnose' may take an optional string arg (either 'stats' or 'all'). If specified without the arg, the behavior corresponds to running 'git diagnose' without '--mode'. As with 'git diagnose', this default is intended to help reduce unintentional leaking of sensitive information). Users can also explicitly specify '--diagnose=(stats|all)' to generate the respective archive created by 'git diagnose --mode=(stats|all)'. Suggested-by: Ævar Arnfjörð Bjarmason Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- Documentation/git-bugreport.txt | 18 +++++++++++++ builtin/bugreport.c | 27 ++++++++++++++++--- t/t0091-bugreport.sh | 48 +++++++++++++++++++++++++++++++++ 3 files changed, 90 insertions(+), 3 deletions(-) diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt index d8817bf3cec..eca726e5791 100644 --- a/Documentation/git-bugreport.txt +++ b/Documentation/git-bugreport.txt @@ -9,6 +9,7 @@ SYNOPSIS -------- [verse] 'git bugreport' [(-o | --output-directory) ] [(-s | --suffix) ] + [--diagnose[=]] DESCRIPTION ----------- @@ -31,6 +32,10 @@ The following information is captured automatically: - A list of enabled hooks - $SHELL +Additional information may be gathered into a separate zip archive using the +`--diagnose` option, and can be attached alongside the bugreport document to +provide additional context to readers. + This tool is invoked via the typical Git setup process, which means that in some cases, it might not be able to launch - for example, if a relevant config file is unreadable. In this kind of scenario, it may be helpful to manually gather @@ -49,6 +54,19 @@ OPTIONS named 'git-bugreport-'. This should take the form of a strftime(3) format string; the current local time will be used. +--no-diagnose:: +--diagnose[=]:: + Create a zip archive of supplemental information about the user's + machine, Git client, and repository state. The archive is written to the + same output directory as the bug report and is named + 'git-diagnostics-'. ++ +Without `mode` specified, the diagnostic archive will contain the default set of +statistics reported by `git diagnose`. An optional `mode` value may be specified +to change which information is included in the archive. See +linkgit:git-diagnose[1] for the list of valid values for `mode` and details +about their usage. + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/bugreport.c b/builtin/bugreport.c index 9de32bc96e7..530895be55f 100644 --- a/builtin/bugreport.c +++ b/builtin/bugreport.c @@ -5,6 +5,7 @@ #include "compat/compiler.h" #include "hook.h" #include "hook-list.h" +#include "diagnose.h" static void get_system_info(struct strbuf *sys_info) @@ -59,7 +60,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit) } static const char * const bugreport_usage[] = { - N_("git bugreport [-o|--output-directory ] [-s|--suffix ]"), + N_("git bugreport [-o|--output-directory ] [-s|--suffix ] [--diagnose[=]"), NULL }; @@ -98,16 +99,21 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix) int report = -1; time_t now = time(NULL); struct tm tm; + enum diagnose_mode diagnose = DIAGNOSE_NONE; char *option_output = NULL; char *option_suffix = "%Y-%m-%d-%H%M"; const char *user_relative_path = NULL; char *prefixed_filename; + size_t output_path_len; const struct option bugreport_options[] = { + OPT_CALLBACK_F(0, "diagnose", &diagnose, N_("mode"), + N_("create an additional zip archive of detailed diagnostics (default 'stats')"), + PARSE_OPT_OPTARG, option_parse_diagnose), OPT_STRING('o', "output-directory", &option_output, N_("path"), - N_("specify a destination for the bugreport file")), + N_("specify a destination for the bugreport file(s)")), OPT_STRING('s', "suffix", &option_suffix, N_("format"), - N_("specify a strftime format suffix for the filename")), + N_("specify a strftime format suffix for the filename(s)")), OPT_END() }; @@ -119,6 +125,7 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix) option_output ? option_output : ""); strbuf_addstr(&report_path, prefixed_filename); strbuf_complete(&report_path, '/'); + output_path_len = report_path.len; strbuf_addstr(&report_path, "git-bugreport-"); strbuf_addftime(&report_path, option_suffix, localtime_r(&now, &tm), 0, 0); @@ -133,6 +140,20 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix) report_path.buf); } + /* Prepare diagnostics, if requested */ + if (diagnose != DIAGNOSE_NONE) { + struct strbuf zip_path = STRBUF_INIT; + strbuf_add(&zip_path, report_path.buf, output_path_len); + strbuf_addstr(&zip_path, "git-diagnostics-"); + strbuf_addftime(&zip_path, option_suffix, localtime_r(&now, &tm), 0, 0); + strbuf_addstr(&zip_path, ".zip"); + + if (create_diagnostics_archive(&zip_path, diagnose)) + die_errno(_("unable to create diagnostics archive %s"), zip_path.buf); + + strbuf_release(&zip_path); + } + /* Prepare the report contents */ get_bug_template(&buffer); diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh index 08f5fe9caef..b6d2f591acd 100755 --- a/t/t0091-bugreport.sh +++ b/t/t0091-bugreport.sh @@ -78,4 +78,52 @@ test_expect_success 'indicates populated hooks' ' test_cmp expect actual ' +test_expect_success UNZIP '--diagnose creates diagnostics zip archive' ' + test_when_finished rm -rf report && + + git bugreport --diagnose -o report -s test >out && + + zip_path=report/git-diagnostics-test.zip && + grep "Available space" out && + test_path_is_file "$zip_path" && + + # Check zipped archive content + "$GIT_UNZIP" -p "$zip_path" diagnostics.log >out && + test_file_not_empty out && + + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + "$GIT_UNZIP" -p "$zip_path" objects-local.txt >out && + grep "^Total: [0-9][0-9]*" out && + + # Should not include .git directory contents by default + ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" +' + +test_expect_success UNZIP '--diagnose=stats excludes .git dir contents' ' + test_when_finished rm -rf report && + + git bugreport --diagnose=stats -o report -s test >out && + + # Includes pack quantity/size info + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + # Does not include .git directory contents + ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" +' + +test_expect_success UNZIP '--diagnose=all includes .git dir contents' ' + test_when_finished rm -rf report && + + git bugreport --diagnose=all -o report -s test >out && + + # Includes .git directory contents + "$GIT_UNZIP" -l "$zip_path" | grep ".git/" && + + "$GIT_UNZIP" -p "$zip_path" .git/HEAD >out && + test_file_not_empty out +' + test_done From patchwork Fri Aug 12 20:10:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC4F3C25B0F for ; Fri, 12 Aug 2022 20:11:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237148AbiHLUK7 (ORCPT ); Fri, 12 Aug 2022 16:10:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235508AbiHLUKg (ORCPT ); Fri, 12 Aug 2022 16:10:36 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2D26EE20 for ; Fri, 12 Aug 2022 13:10:32 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id ay39-20020a05600c1e2700b003a5503a80cfso1002973wmb.2 for ; Fri, 12 Aug 2022 13:10:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=7GCnYaiyaqpfB+KodketEyI+zYP82WFm71eckKhZD9k=; b=ObCWl7jCBp5m/scVOeTaT9pFtjASZjMC47B/S+gO98u8jQsYAy0ZQr5gNDGUBkk1BY D+5qrXu+3iB0wOlS18+iXyK6h7XJCFfV9Oq9NJEA6WWlVu7UoaARYwhewFmtX1P7z9EO av4geDfOzQgQIKmwASHbyi4TPl43N/V1Kwp952d+xJszwp2j3dyam1nwlBv8+5I+DHsH v7AqmxdtP+GXrXIN4KGD5VJDPpSSdhPuHxwxMf4js4ac4YOB519WNm4lBYH063ECj3sE /6RwS1YcOm4fzZKKNQKFv1K3d8Ty6lpyirPzO0VbdY2extF0RSpKwP/MISzWGGn/uxrV Pejw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=7GCnYaiyaqpfB+KodketEyI+zYP82WFm71eckKhZD9k=; b=jUH7pVpJ1dlSNAOtea1x/H82pdXATQiKe09gW+Wf6mrOq2Msw79ElChfqqUT4kQg9t SyLxaHGNJIlzqU5FzjETfr6uZFZ6a4m7abvH3R/4lpMGsw6/sfWrQZl+MY4LTMAg1owJ wN9gKG+0DNgDRg/uunB+l9Iv1sG9DiXrBwU4UI97nkMrTXU6aFmhtqMELiRD0nqEmuaR Tikdsc4eeG20xVm05OcGsqB2jQPhX+wB98tbKiNitOYb1DxNUjvtqa8X0pUPx4adGL7M W2WU7aNTDvxE8yMMM+8e2aYaAWA2547CriLuLUv0mbQCqTV5LLZPXOzWjQIBh7US/4PQ TuLg== X-Gm-Message-State: ACgBeo0AdF/QJPQjgSWHlyUVh+XS1k4MCbRdgNvoxUz+v4JCRFcMHgnX YZnuk0WO2stDqqGfw9WRHv/DG9gSyvs= X-Google-Smtp-Source: AA6agR7xMpn5Wagw0ojJR3h/NHHx2Is5507BAPvMaO+FgFWZ/nFI2Ly3xK90zNXmggxgGaSnkgFrPQ== X-Received: by 2002:a7b:ce10:0:b0:3a5:3f91:e2fb with SMTP id m16-20020a7bce10000000b003a53f91e2fbmr10104137wmc.138.1660335030951; Fri, 12 Aug 2022 13:10:30 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s9-20020a05600c384900b003a53731f273sm530488wmr.31.2022.08.12.13.10.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:30 -0700 (PDT) Message-Id: <94b32eacdd565df665c40bc49929c128b93cc626.1660335019.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:18 +0000 Subject: [PATCH v4 10/11] scalar-diagnose: use 'git diagnose --mode=all' MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Replace implementation of 'scalar diagnose' with an internal invocation of 'git diagnose --mode=all'. This simplifies the implementation of 'cmd_diagnose' by making it a direct alias of 'git diagnose' and removes some code in 'scalar.c' that is duplicated in 'builtin/diagnose.c'. The simplicity of the alias also sets up a clean deprecation path for 'scalar diagnose' (in favor of 'git diagnose'), if that is desired in the future. This introduces one minor change to the output of 'scalar diagnose', which is that the prefix of the created zip archive is changed from 'scalar_' to 'git-diagnostics-'. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: Victoria Dye --- contrib/scalar/scalar.c | 28 ++++++---------------------- 1 file changed, 6 insertions(+), 22 deletions(-) diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index d538b8b8f14..68571ce195f 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -11,7 +11,6 @@ #include "dir.h" #include "packfile.h" #include "help.h" -#include "diagnose.h" /* * Remove the deepest subdirectory in the provided path string. Path must not @@ -510,34 +509,19 @@ static int cmd_diagnose(int argc, const char **argv) N_("scalar diagnose []"), NULL }; - struct strbuf zip_path = STRBUF_INIT; - time_t now = time(NULL); - struct tm tm; + struct strbuf diagnostics_root = STRBUF_INIT; int res = 0; argc = parse_options(argc, argv, NULL, options, usage, 0); - setup_enlistment_directory(argc, argv, usage, options, &zip_path); - - strbuf_addstr(&zip_path, "/.scalarDiagnostics/scalar_"); - strbuf_addftime(&zip_path, - "%Y%m%d_%H%M%S", localtime_r(&now, &tm), 0, 0); - strbuf_addstr(&zip_path, ".zip"); - switch (safe_create_leading_directories(zip_path.buf)) { - case SCLD_EXISTS: - case SCLD_OK: - break; - default: - error_errno(_("could not create directory for '%s'"), - zip_path.buf); - goto diagnose_cleanup; - } + setup_enlistment_directory(argc, argv, usage, options, &diagnostics_root); + strbuf_addstr(&diagnostics_root, "/.scalarDiagnostics"); - res = create_diagnostics_archive(&zip_path, DIAGNOSE_ALL); + res = run_git("diagnose", "--mode=all", "-s", "%Y%m%d_%H%M%S", + "-o", diagnostics_root.buf, NULL); -diagnose_cleanup: - strbuf_release(&zip_path); + strbuf_release(&diagnostics_root); return res; } From patchwork Fri Aug 12 20:10:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12942507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63D21C25B0F for ; Fri, 12 Aug 2022 20:11:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234710AbiHLULD (ORCPT ); Fri, 12 Aug 2022 16:11:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234491AbiHLUKz (ORCPT ); Fri, 12 Aug 2022 16:10:55 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5CC210FDA for ; Fri, 12 Aug 2022 13:10:33 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id s11-20020a1cf20b000000b003a52a0945e8so1006844wmc.1 for ; Fri, 12 Aug 2022 13:10:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=RhPe/2YGJzGbswB35npBDWGmS6OQlK0k/r1JD67kQvU=; b=VZ8GLFKo65Lht6bc4OIAa6XfjjSijGV5BjsLU+uLFFOArGlVdUAHR06dqNR4aeF3LP fVJhgS1mkseQWdNuoic9g3c6LQe5pi6o4PZr7LOO5PAEK6n9iQFAGDKRKO98EE1X9Cmo uKfAVksIK4vXz8E2qMiG5wYsDxn0W2v7JtSxzUhfI6Qh7lQuQa9V1va7wLaUDjqJ3twm 3ujS9vI00GVaBkzJgyKb79SC9nDuyOEqUv7EjzMLVou55nPoHT3Q+BteoJ239SJPIwqK gvhVBQyTOVIOPPd6WPr8HNCkU+DR2yO9+HVCWSw4FOTJXQjM3ZqCLc7YWbs8LFt1mchB UbxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=RhPe/2YGJzGbswB35npBDWGmS6OQlK0k/r1JD67kQvU=; b=MfDuqxvcA2UVY2wWqNBJ+SS2h2MR/+CoOLBqOXJTT5Uk6LaKLi8sIgvLK2dnNdkjHD FVnGA/ALkm8meGfwADaK2V9RpXtoEJKrr3pjr7EnCGFMCe68NmZamQiSq6a10ctWSXUL xda6cCufvPymVgADAYKQKwzTAJ7Tctcrqy0wNRX9r8wFOESlRVSCdVe4KIDGRlzq41Zi iPsqa3OrghRWmwDRRHc27s451mGyuF/V5zYFtwcebClctF7HLXpqKikMz8BkGCd7TAmz YXxIXBAn7ZDSjws4XCagF9ogS2O0eecStywvxIoL3i/JSZWXrVk+oYU0J/8lEDAW1Ey5 nnXg== X-Gm-Message-State: ACgBeo2Qb9+CW98Sz36jtsRGiVKHKecKb2qD3M95R9QBxmSE2pe+/86U 5s32BdhJ1A6P4Nvs7vXwnSe6h11raQM= X-Google-Smtp-Source: AA6agR77Dc5ZBTnlLs7yCzU3pbgJXmOuNEy+7E1jeCh7Ed0fhAlANz3CKDL6gArAnM6YWo5DokDiNg== X-Received: by 2002:a05:600c:34c9:b0:3a3:74a1:2754 with SMTP id d9-20020a05600c34c900b003a374a12754mr9892240wmq.35.1660335031914; Fri, 12 Aug 2022 13:10:31 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s14-20020adfeb0e000000b0021d6d18a9f8sm279650wrn.76.2022.08.12.13.10.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 13:10:31 -0700 (PDT) Message-Id: <728f8b81fd09a3d46f87d98a6eac38c46765d688.1660335019.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 12 Aug 2022 20:10:19 +0000 Subject: [PATCH v4 11/11] scalar: update technical doc roadmap Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Update the Scalar roadmap to reflect the completion of generalizing 'scalar diagnose' into 'git diagnose' and 'git bugreport --diagnose'. Signed-off-by: Victoria Dye --- Documentation/technical/scalar.txt | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/Documentation/technical/scalar.txt b/Documentation/technical/scalar.txt index 08bc09c225a..f6353375f08 100644 --- a/Documentation/technical/scalar.txt +++ b/Documentation/technical/scalar.txt @@ -84,6 +84,9 @@ series have been accepted: - `scalar-diagnose`: The `scalar` command is taught the `diagnose` subcommand. +- `scalar-generalize-diagnose`: Move the functionality of `scalar diagnose` + into `git diagnose` and `git bugreport --diagnose`. + Roughly speaking (and subject to change), the following series are needed to "finish" this initial version of Scalar: @@ -91,12 +94,6 @@ Roughly speaking (and subject to change), the following series are needed to and implement `scalar help`. At the end of this series, Scalar should be feature-complete from the perspective of a user. -- Generalize features not specific to Scalar: In the spirit of making Scalar - configure only what is needed for large repo performance, move common - utilities into other parts of Git. Some of this will be internal-only, but one - major change will be generalizing `scalar diagnose` for use with any Git - repository. - - Move Scalar to toplevel: Move Scalar out of `contrib/` and into the root of `git`, including updates to build and install it with the rest of Git. This change will incorporate Scalar into the Git CI and test framework, as well as