From patchwork Wed Aug 10 23:34:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8517DC00140 for ; Wed, 10 Aug 2022 23:34:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232985AbiHJXek (ORCPT ); Wed, 10 Aug 2022 19:34:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229487AbiHJXej (ORCPT ); Wed, 10 Aug 2022 19:34:39 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C975E84EEA for ; Wed, 10 Aug 2022 16:34:37 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id z17so19479023wrq.4 for ; Wed, 10 Aug 2022 16:34:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=UWkmwEMmDQA88Knl6wRhhl/jDXRNKzlTXKO5NvbUwn8=; b=THM74PETnAsFVlDUnDYv4JVs4222e08VV/KYjF+kmQfw4LbnWsibEpD0h5nunSNSuF OMfP7ktUAz85p+C4IlQzemh3C6TpnSjHzCVO0mLhksORWHMhwI0z5VjnitX9uP8tG37/ 00Ffn6zPlVga6SU2Ra7M9Q4kJQYeX24iBT1K+X3b7jUkapBripMMkGYYEYZhglgirgjD PsPpCrGI3uUd+4b/1yrxykKcJm+L+BOhYKeoU1BCutOwpzIQJ9FAEU1I9Eg/C3kROwVk AqDp6VGW1lPagk8wFhcD4rM7J/Qsbv2+UkafBeAzIauGWuwzUGTkVOmAfm3tFTaeCL9F r8Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=UWkmwEMmDQA88Knl6wRhhl/jDXRNKzlTXKO5NvbUwn8=; b=a+CAfWmV55pBKpUBhE1CZomRR3JKUTXMPw/Oe9339UgFbQRNMi39Bweq1XHAKaY6EP WB85UUzM4XnI1rcGpHsWlaHSwpCtqQ0U4NcSKNANCgnWLlu6/dmW05sDMT4uvgqx03Nc egkcLeDpFrwZ0Y/uMKHKdjMNkqDmz0PDqUZTawrOsWkxCEIvMLKJvJNNOUFaaCV2vxLO 178lpRAQE4dUNqUx1YQRFBnHz2nXh5kA63Bzsn/Hmq2gt7Olpi5NdXJzNjSSPuVI4Ecg eCykyGi/qHh7cBzZjaAh+xZ5Ig0yTAm6TpNhspnaPQ70LUCNMA+Az4NlrodFJYiJwJAk yfZg== X-Gm-Message-State: ACgBeo3Y/2rT1+eptQoqtjcKpHTLhTNuUfTyiELfP/rx35nclLRcKdol LJzVB7HYS4DHhC7kr6XwBwjJYr8fm5U= X-Google-Smtp-Source: AA6agR58YMHllQ7FxFDw0aua5rUqEXPg3qRkD68TDYX9icY44hvQ3s57MVUC2inqzyeBp8vp6ks/vw== X-Received: by 2002:a05:6000:a1a:b0:21f:10a3:924 with SMTP id co26-20020a0560000a1a00b0021f10a30924mr18885148wrb.650.1660174476071; Wed, 10 Aug 2022 16:34:36 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id bu8-20020a056000078800b002217a25ba8dsm14346669wrb.44.2022.08.10.16.34.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:35 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:23 +0000 Subject: [PATCH v3 01/11] scalar-diagnose: use "$GIT_UNZIP" in test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Use the "$GIT_UNZIP" test variable rather than verbatim 'unzip' to unzip the 'scalar diagnose' archive. Using "$GIT_UNZIP" is needed to run the Scalar tests on systems where 'unzip' is not in the system path. Signed-off-by: Victoria Dye --- contrib/scalar/t/t9099-scalar.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/contrib/scalar/t/t9099-scalar.sh b/contrib/scalar/t/t9099-scalar.sh index 10b1172a8aa..fac86a57550 100755 --- a/contrib/scalar/t/t9099-scalar.sh +++ b/contrib/scalar/t/t9099-scalar.sh @@ -109,14 +109,14 @@ test_expect_success UNZIP 'scalar diagnose' ' sed -n "s/.*$SQ\\(.*\\.zip\\)$SQ.*/\\1/p" zip_path && zip_path=$(cat zip_path) && test -n "$zip_path" && - unzip -v "$zip_path" && + "$GIT_UNZIP" -v "$zip_path" && folder=${zip_path%.zip} && test_path_is_missing "$folder" && - unzip -p "$zip_path" diagnostics.log >out && + "$GIT_UNZIP" -p "$zip_path" diagnostics.log >out && test_file_not_empty out && - unzip -p "$zip_path" packs-local.txt >out && + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && grep "$(pwd)/.git/objects" out && - unzip -p "$zip_path" objects-local.txt >out && + "$GIT_UNZIP" -p "$zip_path" objects-local.txt >out && grep "^Total: [1-9]" out ' From patchwork Wed Aug 10 23:34:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35717C00140 for ; Wed, 10 Aug 2022 23:34:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233150AbiHJXep (ORCPT ); Wed, 10 Aug 2022 19:34:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233082AbiHJXem (ORCPT ); Wed, 10 Aug 2022 19:34:42 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8A6486C19 for ; Wed, 10 Aug 2022 16:34:38 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id l22so19494173wrz.7 for ; Wed, 10 Aug 2022 16:34:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=fTQyYbLVeujkIXWBZRa69qVeE19+Lc0IZap+yd+gMZw=; b=VZqy5rsbsaSw8BX7n3+dqC+Ekk/DEziEpJcbwN9v20C6VXHWiw15bUOll8wwaWg6EL hjBdIw8g1aA/lPUZfxblqo42twT+qD+1wdNY0gYP++flaTYNOZf0a+rrlp/egFninTOt j2t+MU8+wF6woPZJgUoV4gjIpQu2yiKB9OOT34S/O2a7KbT73vCfGBdvhAEq0DvbvjeP AGgPhZ87t862x8/zCJBtTwKS9odB0cT6BvRZbw9sdhWQk6Eh+3QNeCL/5V7xK1rgCgUi 5dGhWKEewisHNIUu4SuhvGBG9mYW94vRxorzr41K4IpdJyQzWgavRS7gdLskqbuuiyZ3 q4ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=fTQyYbLVeujkIXWBZRa69qVeE19+Lc0IZap+yd+gMZw=; b=VDN8weCvwUVrmb88ACibFu4Z/j71cV+/QCJhAUbvNblPU/IhbCg6foEvoGYRibd0B1 kyBQtZiiiU0woCe2w7M4oYCHMuJuMAKa29H0L50sEbrT+5N3tNMEmanepCtBhrdtAyd1 Ben24YerNB39+SNQd74tJyVpCz8bMcTA2qaoPQVtKStXoYzCvYD8p2viS3KKyHj/hBFt wcyqjI7szy05BIVvNS3v/We6ogovd13qe/wxk9hu75CDKz+JirKIfBmP5ndjXygOV7va +YZbX/iaB9T2jczIPPe6JeURu3nUO06akDNhz6ih6Qa4XVAJJhsnAQTB5s/fcbQ+21bY 8+TQ== X-Gm-Message-State: ACgBeo1zaPCBe35QvHY6Z1VLM/WhLusEwaBxbzLURqbx719GIVbJvdmn aBtcOMXAt1Kn+76yJEZbPgtkvT2vt9o= X-Google-Smtp-Source: AA6agR42/N65aA1Cc/PIlLC10PBIqPDTt4APaVWvm4rSXa1kDc5dVjrfaaFrKMfcdP0ZzDSc16NT2A== X-Received: by 2002:adf:f5cc:0:b0:221:7e3f:41eb with SMTP id k12-20020adff5cc000000b002217e3f41ebmr13818146wrp.551.1660174476910; Wed, 10 Aug 2022 16:34:36 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 2-20020a05600c274200b003a54f1d007csm4059733wmw.10.2022.08.10.16.34.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:36 -0700 (PDT) Message-Id: <78a93eb95bb0c43778501f03837bffdcf74e4c99.1660174473.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:24 +0000 Subject: [PATCH v3 02/11] scalar-diagnose: avoid 32-bit overflow of size_t Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Avoid 32-bit size_t overflow when reporting the available disk space in 'get_disk_info' by casting the block size and available block count to 'off_t' before multiplying them. Without this change, 'st_mult' would (correctly) report a size_t overflow on 32-bit systems at or exceeding 2^32 bytes of available space. Note that 'off_t' is a 64-bit integer even on 32-bit systems due to the inclusion of '#define _FILE_OFFSET_BITS 64' in 'git-compat-util.h' (see b97e911643 (Support for large files on 32bit systems., 2007-02-17)). Helped-by: Junio C Hamano Signed-off-by: Victoria Dye --- contrib/scalar/scalar.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index 97e71fe19cd..04046452284 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -348,7 +348,7 @@ static int get_disk_info(struct strbuf *out) } strbuf_addf(out, "Available space on '%s': ", buf.buf); - strbuf_humanise_bytes(out, st_mult(stat.f_bsize, stat.f_bavail)); + strbuf_humanise_bytes(out, (off_t)stat.f_bsize * (off_t)stat.f_bavail); strbuf_addf(out, " (mount flags 0x%lx)\n", stat.f_flag); strbuf_release(&buf); #endif From patchwork Wed Aug 10 23:34:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941038 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29074C25B07 for ; Wed, 10 Aug 2022 23:34:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233263AbiHJXet (ORCPT ); Wed, 10 Aug 2022 19:34:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233060AbiHJXem (ORCPT ); Wed, 10 Aug 2022 19:34:42 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 792B08C031 for ; Wed, 10 Aug 2022 16:34:39 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id c187-20020a1c35c4000000b003a30d88fe8eso1857846wma.2 for ; Wed, 10 Aug 2022 16:34:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=327gbsyYeccEL0tlq4IQ1Y7CXUns6oBUTnyfqIVz7cI=; b=i6K21r6tAJXe+l0MG289DJinaeAstu3K4fhie2/6eq5BKVqwUBNnXCZw2wGybS3tn/ d3M+Dy7SmOIAZ4X+FCjXkmjy4TXHglgG1W//DMnrSSFtHVxACgVSLokh4D1uvEkbbCsn KpnqHXkiE0UvXO73FzAAUEKV6sTEN+w5Eu/P3WZeejfhl8ZFcIBH4BzupZUMvDRbNPc2 CeSGaP+KqtbHaCVzwQhv6IOS8H+YxxyEQGSS/pMY+G0gn+xB6r7yph4SQJzrRokwsU6T JdDOpKFDwL9vOVmF7vmeweS0rBFtbwBiVCbCnpaXOldamhyKO95Na1ojCiaUHIrYAxg9 DjIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=327gbsyYeccEL0tlq4IQ1Y7CXUns6oBUTnyfqIVz7cI=; b=8EDghuLaw4JJN5Cyq38yn8A8oTcX/xngyKC4cMxmk3PDcLduW9AC+qHOJMXMGCNh7o 0uu50oM5SbFtp/T06OBjdcE4U+HpOl0xczQE2IHNmKtCFyt3rYW41vpcgCD7j5kPcKC9 B8ajfQY45R7yy7LdnrLthvWTBuBsrgHBY1TaVTfkicAj/o+3sSDMIFJMKySblKKfbuED uLcBF/D0XXsYu6MHh1NvPFLj06loziSmKcLS97ZfIcaLXfMX1wucKrJqj+kz9/sSLjuX ytRQsEy7KJAKYHH9TH+jzoePDzSNxOB4EVnTqiM1mtzTddUQreRTMcMlaaD0CWE5JPJC fzEA== X-Gm-Message-State: ACgBeo3S763dgWeon/UDy3kASHcXY1vz5llqUV+jpdl3M8D5SaN3Hw/i PH7q4+MOXnECDRZMVD/Fq2oYQzjI30g= X-Google-Smtp-Source: AA6agR4jrG2Rsxb8bdytfbnJVpz9lhRLQkDq5odcW/dcibQZuGvFdSeJ+SutWRxv4BbT6ZGImm628Q== X-Received: by 2002:a05:600c:3782:b0:3a5:23f5:6bc9 with SMTP id o2-20020a05600c378200b003a523f56bc9mr3821997wmr.174.1660174477756; Wed, 10 Aug 2022 16:34:37 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ay29-20020a05600c1e1d00b003a3170a7af9sm4123413wmb.4.2022.08.10.16.34.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:37 -0700 (PDT) Message-Id: <22ee8ea5a1e04a42ad359be4eb5ebc96bd5e5fa2.1660174473.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:25 +0000 Subject: [PATCH v3 03/11] scalar-diagnose: add directory to archiver more gently Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye If a directory added to the 'scalar diagnose' archiver does not exist, warn and return 0 from 'add_directory_to_archiver()' rather than failing with a fatal error. This handles a failure edge case where the '.git/logs' has not yet been created when running 'scalar diagnose', but extends to any situation where a directory may be missing in the '.git' dir. Now, when a directory is missing a warning is captured in the diagnostic logs. This provides a user with more complete information than if 'scalar diagnose' simply failed with an error. Helped-by: Junio C Hamano Signed-off-by: Victoria Dye --- contrib/scalar/scalar.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index 04046452284..b9092f0b612 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -266,14 +266,20 @@ static int add_directory_to_archiver(struct strvec *archiver_args, const char *path, int recurse) { int at_root = !*path; - DIR *dir = opendir(at_root ? "." : path); + DIR *dir; struct dirent *e; struct strbuf buf = STRBUF_INIT; size_t len; int res = 0; - if (!dir) + dir = opendir(at_root ? "." : path); + if (!dir) { + if (errno == ENOENT) { + warning(_("could not archive missing directory '%s'"), path); + return 0; + } return error_errno(_("could not open directory '%s'"), path); + } if (!at_root) strbuf_addf(&buf, "%s/", path); From patchwork Wed Aug 10 23:34:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA7E7C00140 for ; Wed, 10 Aug 2022 23:34:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233506AbiHJXe4 (ORCPT ); Wed, 10 Aug 2022 19:34:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232797AbiHJXeo (ORCPT ); Wed, 10 Aug 2022 19:34:44 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BD7B84EEA for ; Wed, 10 Aug 2022 16:34:40 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id p10so19463392wru.8 for ; Wed, 10 Aug 2022 16:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=YKKdcJFhp+40tORJ9u012i+7uSC/4xTtFFym2CPkqAw=; b=TaawuSvTQiidu0fmNLOVRtP/VeZad9E+FMjZQ08eOfj0Eo55ypWgaU1C+XaABpegX9 0D6QVNiOCRCRwkgche1UrI0plMWehGRUG0Odn7Jrw51rv0UgiajtXAn275b8F+3F/l5L BqTKDe4zBCul3WGygeCQ6BR1VBk/5kEnORjdMt0AGQS5NBj/Ij4nDl0YlHc8XAo99TNI tbXvDvWs1f3SOnQrVdh6hP8su/t9Gjgh9JuuGKuE2B6ggxy9f5tJli5x8rQbzlj7yQQ8 Ro5OE27164lkKMHzD873I4gmForuVokVKIJ5oLFSNqVFU2ObDxcgmg+qoN/JDvHSw45O zDew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=YKKdcJFhp+40tORJ9u012i+7uSC/4xTtFFym2CPkqAw=; b=OZqNRuMa4iYRm9mBCDB+LGpTAOE62swYwfi3J2CMt96X7EbjsjWDy2mJTSPld63rQb sRoKBVrHI7vKKxsju3nOk/crFhVyfQpEUX+m+HAjMkX+2chh1fafAlANUuMl8trjJNnE 8XVxhAGvKRbI7sU20ewjO02QHrn5/c5Dpb/sflIFU1CcUT8K5tK1kA+0ApVIwDFa2aMx UMLklWP0shjwQ6/YUirK0bIT/6BKG0pgArE1jFza9foNaWVQbZu92w2rDk1/NM0NalQs n8jLlzFZwf0odJ6oPE4GoLpc4cO22tqLymMNupE4jYK9JqnnZ/4nyT75nJM16HaRYFOw 8mmw== X-Gm-Message-State: ACgBeo0Wr/eK+0riAqTV1hFQuK4aCGCmnTxAUfOvSjM3tLvS18NhU21v 9wCo++h2zGTkcwaKAeixI/bKWnUpENA= X-Google-Smtp-Source: AA6agR7taCakcYh/wFuCtsBcogwKU4Pf1ws6EcJpsVo0JSsuzk9H+JM91HKspUCzd64XLvgM0J8cPw== X-Received: by 2002:adf:fecd:0:b0:21f:d78:743f with SMTP id q13-20020adffecd000000b0021f0d78743fmr18223077wrs.704.1660174478527; Wed, 10 Aug 2022 16:34:38 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n2-20020a5d6b82000000b0021e43b4edf0sm16928943wrx.20.2022.08.10.16.34.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:38 -0700 (PDT) Message-Id: <18f2ba4e0cdd9aea213dc7d0e908c6c0d8eccfd0.1660174473.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:26 +0000 Subject: [PATCH v3 04/11] scalar-diagnose: move 'get_disk_info()' to 'compat/' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Move 'get_disk_info()' function into 'compat/'. Although Scalar-specific code is generally not part of the main Git tree, 'get_disk_info()' will be used in subsequent patches by additional callers beyond 'scalar diagnose'. This patch prepares for that change, at which point this platform-specific code should be part of 'compat/' as a matter of convention. The function is copied *mostly* verbatim, with two exceptions: * '#ifdef WIN32' is replaced with '#ifdef GIT_WINDOWS_NATIVE' to allow 'statvfs' to be used with Cygwin. * the 'struct strbuf buf' and 'int res' (as well as their corresponding cleanup & return) are moved outside of the '#ifdef' block. Signed-off-by: Victoria Dye --- compat/disk.h | 56 +++++++++++++++++++++++++++++++++++++++++ contrib/scalar/scalar.c | 53 +------------------------------------- git-compat-util.h | 1 + 3 files changed, 58 insertions(+), 52 deletions(-) create mode 100644 compat/disk.h diff --git a/compat/disk.h b/compat/disk.h new file mode 100644 index 00000000000..50a32e3d8a4 --- /dev/null +++ b/compat/disk.h @@ -0,0 +1,56 @@ +#ifndef COMPAT_DISK_H +#define COMPAT_DISK_H + +#include "git-compat-util.h" + +static int get_disk_info(struct strbuf *out) +{ + struct strbuf buf = STRBUF_INIT; + int res = 0; + +#ifdef GIT_WINDOWS_NATIVE + char volume_name[MAX_PATH], fs_name[MAX_PATH]; + DWORD serial_number, component_length, flags; + ULARGE_INTEGER avail2caller, total, avail; + + strbuf_realpath(&buf, ".", 1); + if (!GetDiskFreeSpaceExA(buf.buf, &avail2caller, &total, &avail)) { + error(_("could not determine free disk size for '%s'"), + buf.buf); + res = -1; + goto cleanup; + } + + strbuf_setlen(&buf, offset_1st_component(buf.buf)); + if (!GetVolumeInformationA(buf.buf, volume_name, sizeof(volume_name), + &serial_number, &component_length, &flags, + fs_name, sizeof(fs_name))) { + error(_("could not get info for '%s'"), buf.buf); + res = -1; + goto cleanup; + } + strbuf_addf(out, "Available space on '%s': ", buf.buf); + strbuf_humanise_bytes(out, avail2caller.QuadPart); + strbuf_addch(out, '\n'); +#else + struct statvfs stat; + + strbuf_realpath(&buf, ".", 1); + if (statvfs(buf.buf, &stat) < 0) { + error_errno(_("could not determine free disk size for '%s'"), + buf.buf); + res = -1; + goto cleanup; + } + + strbuf_addf(out, "Available space on '%s': ", buf.buf); + strbuf_humanise_bytes(out, (off_t)stat.f_bsize * (off_t)stat.f_bavail); + strbuf_addf(out, " (mount flags 0x%lx)\n", stat.f_flag); +#endif + +cleanup: + strbuf_release(&buf); + return res; +} + +#endif /* COMPAT_DISK_H */ diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index b9092f0b612..607fedefd82 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -13,6 +13,7 @@ #include "help.h" #include "archive.h" #include "object-store.h" +#include "compat/disk.h" /* * Remove the deepest subdirectory in the provided path string. Path must not @@ -309,58 +310,6 @@ static int add_directory_to_archiver(struct strvec *archiver_args, return res; } -#ifndef WIN32 -#include -#endif - -static int get_disk_info(struct strbuf *out) -{ -#ifdef WIN32 - struct strbuf buf = STRBUF_INIT; - char volume_name[MAX_PATH], fs_name[MAX_PATH]; - DWORD serial_number, component_length, flags; - ULARGE_INTEGER avail2caller, total, avail; - - strbuf_realpath(&buf, ".", 1); - if (!GetDiskFreeSpaceExA(buf.buf, &avail2caller, &total, &avail)) { - error(_("could not determine free disk size for '%s'"), - buf.buf); - strbuf_release(&buf); - return -1; - } - - strbuf_setlen(&buf, offset_1st_component(buf.buf)); - if (!GetVolumeInformationA(buf.buf, volume_name, sizeof(volume_name), - &serial_number, &component_length, &flags, - fs_name, sizeof(fs_name))) { - error(_("could not get info for '%s'"), buf.buf); - strbuf_release(&buf); - return -1; - } - strbuf_addf(out, "Available space on '%s': ", buf.buf); - strbuf_humanise_bytes(out, avail2caller.QuadPart); - strbuf_addch(out, '\n'); - strbuf_release(&buf); -#else - struct strbuf buf = STRBUF_INIT; - struct statvfs stat; - - strbuf_realpath(&buf, ".", 1); - if (statvfs(buf.buf, &stat) < 0) { - error_errno(_("could not determine free disk size for '%s'"), - buf.buf); - strbuf_release(&buf); - return -1; - } - - strbuf_addf(out, "Available space on '%s': ", buf.buf); - strbuf_humanise_bytes(out, (off_t)stat.f_bsize * (off_t)stat.f_bavail); - strbuf_addf(out, " (mount flags 0x%lx)\n", stat.f_flag); - strbuf_release(&buf); -#endif - return 0; -} - /* printf-style interface, expects `=` argument */ static int set_config(const char *fmt, ...) { diff --git a/git-compat-util.h b/git-compat-util.h index 58d7708296b..9a62e3a0d2d 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -258,6 +258,7 @@ static inline int is_xplatform_dir_sep(int c) #include #include #include +#include #include #ifndef NO_SYS_SELECT_H #include From patchwork Wed Aug 10 23:34:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941040 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34F6AC25B06 for ; Wed, 10 Aug 2022 23:34:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229722AbiHJXe6 (ORCPT ); Wed, 10 Aug 2022 19:34:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233129AbiHJXep (ORCPT ); Wed, 10 Aug 2022 19:34:45 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 265C881B32 for ; Wed, 10 Aug 2022 16:34:41 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id v5so8647664wmj.0 for ; Wed, 10 Aug 2022 16:34:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=0LEvMAnYM0Z5oCfQYNqyjgrBUeMv0eo6t4oWVAQ3ml4=; b=iFu7Fzdrh+H5obf76GbA9aDlE8BiOw/Djb+LKM21GuHzWROShuxSlWbiXn3FVB/acO nY10f83jp9Wh+oCslcZwcjgfTqtElN0RrtUS0s259T/sZinY2UJIMR4BlPF+4g2sdMAk 14DooDJTnjSSrcAuGrqTV5+PhuTzWSpaeTOewvKRpmmDHGxw3Sbul/QaZX797WGihCrx PFxxGOd8pDlZgr26sLFmJQQcBTbGJuFrPC+MlExNzmqFlACZBJGozJUE1kwTHW29DZ97 NqvAPcTG7U4yw6/cJQkVVqvNqCdx9hpIyHqvvXlNMVKe0EY6wAhI/Vht6sa+Ywk7nFlm O7eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=0LEvMAnYM0Z5oCfQYNqyjgrBUeMv0eo6t4oWVAQ3ml4=; b=cWH+p3qNJio8Aw2F5lhmN0SkwBfEZ9hotnUbYwGa+GRyKBe4SEuLoPNYoUWEuMjzhB XKRqZ4P/l6kGfFIxighx+xmRF9ca38wYSCEZ/thoC5bIHakpObnw29njasQ/NQmWF70s eP5/v1kJjGZS30Tfib/MArHGz/agqxzzURtqH4EHqfwizoz+qK3rpFGtlZHxfiFrAUM+ 9DtdoZ5zFwOjE7F7gn0RPXUtkUIIOg1XQQ3lWmw2Ey2OACO6tvpprtVscV+5GyZhrHiw Sleon+Zou3WnZT01XxXy11jehQlZ0YnqiPzmSeyPRajnZRkXz1bbj0FKL/wHM9xn3AOH 4JpA== X-Gm-Message-State: ACgBeo0F2A9n8k3LnH34YuNQydM5/gGIQGhQwk0n6rHZJRzpgimQX5TO jhR1zlCZNfx85vG++yVy0q60Otiwx48= X-Google-Smtp-Source: AA6agR6iO6tKexXdgGafIXEjrkxKXRUq1/YrZN6K5Qp87F0q7tWi4vsd8oFHK6/VH0Z/PcAob9tEyQ== X-Received: by 2002:a7b:cc94:0:b0:3a5:a965:95ea with SMTP id p20-20020a7bcc94000000b003a5a96595eamr3809273wma.166.1660174479325; Wed, 10 Aug 2022 16:34:39 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n128-20020a1c2786000000b003a302fb9df7sm3970366wmn.21.2022.08.10.16.34.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:39 -0700 (PDT) Message-Id: <7a51fad87a8a21058ac95970d9dd66c731f07cf1.1660174473.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:27 +0000 Subject: [PATCH v3 05/11] scalar-diagnose: move functionality to common location MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Move the core functionality of 'scalar diagnose' into a new 'diagnose.[c,h]' library to prepare for new callers in the main Git tree generating diagnostic archives. These callers will be introduced in subsequent patches. While this patch appears large, it is mostly made up of moving code out of 'scalar.c' and into 'diagnose.c'. Specifically, the functions - dir_file_stats_objects() - dir_file_stats() - count_files() - loose_objs_stats() - add_directory_to_archiver() are all copied verbatim from 'scalar.c'. The 'create_diagnostics_archive()' function is a mostly identical (partial) copy of 'cmd_diagnose()', with the primary changes being that 'zip_path' is an input and "Enlistment root" is corrected to "Repository root" in the archiver log. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: Victoria Dye --- Makefile | 1 + contrib/scalar/scalar.c | 202 +------------------------------------ diagnose.c | 217 ++++++++++++++++++++++++++++++++++++++++ diagnose.h | 8 ++ 4 files changed, 228 insertions(+), 200 deletions(-) create mode 100644 diagnose.c create mode 100644 diagnose.h diff --git a/Makefile b/Makefile index 2ec9b2dc6bb..ed66cb70e5a 100644 --- a/Makefile +++ b/Makefile @@ -932,6 +932,7 @@ LIB_OBJS += ctype.o LIB_OBJS += date.o LIB_OBJS += decorate.o LIB_OBJS += delta-islands.o +LIB_OBJS += diagnose.o LIB_OBJS += diff-delta.o LIB_OBJS += diff-merges.o LIB_OBJS += diff-lib.o diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index 607fedefd82..3983def760a 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -11,9 +11,7 @@ #include "dir.h" #include "packfile.h" #include "help.h" -#include "archive.h" -#include "object-store.h" -#include "compat/disk.h" +#include "diagnose.h" /* * Remove the deepest subdirectory in the provided path string. Path must not @@ -263,53 +261,6 @@ static int unregister_dir(void) return res; } -static int add_directory_to_archiver(struct strvec *archiver_args, - const char *path, int recurse) -{ - int at_root = !*path; - DIR *dir; - struct dirent *e; - struct strbuf buf = STRBUF_INIT; - size_t len; - int res = 0; - - dir = opendir(at_root ? "." : path); - if (!dir) { - if (errno == ENOENT) { - warning(_("could not archive missing directory '%s'"), path); - return 0; - } - return error_errno(_("could not open directory '%s'"), path); - } - - if (!at_root) - strbuf_addf(&buf, "%s/", path); - len = buf.len; - strvec_pushf(archiver_args, "--prefix=%s", buf.buf); - - while (!res && (e = readdir(dir))) { - if (!strcmp(".", e->d_name) || !strcmp("..", e->d_name)) - continue; - - strbuf_setlen(&buf, len); - strbuf_addstr(&buf, e->d_name); - - if (e->d_type == DT_REG) - strvec_pushf(archiver_args, "--add-file=%s", buf.buf); - else if (e->d_type != DT_DIR) - warning(_("skipping '%s', which is neither file nor " - "directory"), buf.buf); - else if (recurse && - add_directory_to_archiver(archiver_args, - buf.buf, recurse) < 0) - res = -1; - } - - closedir(dir); - strbuf_release(&buf); - return res; -} - /* printf-style interface, expects `=` argument */ static int set_config(const char *fmt, ...) { @@ -550,83 +501,6 @@ cleanup: return res; } -static void dir_file_stats_objects(const char *full_path, size_t full_path_len, - const char *file_name, void *data) -{ - struct strbuf *buf = data; - struct stat st; - - if (!stat(full_path, &st)) - strbuf_addf(buf, "%-70s %16" PRIuMAX "\n", file_name, - (uintmax_t)st.st_size); -} - -static int dir_file_stats(struct object_directory *object_dir, void *data) -{ - struct strbuf *buf = data; - - strbuf_addf(buf, "Contents of %s:\n", object_dir->path); - - for_each_file_in_pack_dir(object_dir->path, dir_file_stats_objects, - data); - - return 0; -} - -static int count_files(char *path) -{ - DIR *dir = opendir(path); - struct dirent *e; - int count = 0; - - if (!dir) - return 0; - - while ((e = readdir(dir)) != NULL) - if (!is_dot_or_dotdot(e->d_name) && e->d_type == DT_REG) - count++; - - closedir(dir); - return count; -} - -static void loose_objs_stats(struct strbuf *buf, const char *path) -{ - DIR *dir = opendir(path); - struct dirent *e; - int count; - int total = 0; - unsigned char c; - struct strbuf count_path = STRBUF_INIT; - size_t base_path_len; - - if (!dir) - return; - - strbuf_addstr(buf, "Object directory stats for "); - strbuf_add_absolute_path(buf, path); - strbuf_addstr(buf, ":\n"); - - strbuf_add_absolute_path(&count_path, path); - strbuf_addch(&count_path, '/'); - base_path_len = count_path.len; - - while ((e = readdir(dir)) != NULL) - if (!is_dot_or_dotdot(e->d_name) && - e->d_type == DT_DIR && strlen(e->d_name) == 2 && - !hex_to_bytes(&c, e->d_name, 1)) { - strbuf_setlen(&count_path, base_path_len); - strbuf_addstr(&count_path, e->d_name); - total += (count = count_files(count_path.buf)); - strbuf_addf(buf, "%s : %7d files\n", e->d_name, count); - } - - strbuf_addf(buf, "Total: %d loose objects", total); - - strbuf_release(&count_path); - closedir(dir); -} - static int cmd_diagnose(int argc, const char **argv) { struct option options[] = { @@ -637,12 +511,8 @@ static int cmd_diagnose(int argc, const char **argv) NULL }; struct strbuf zip_path = STRBUF_INIT; - struct strvec archiver_args = STRVEC_INIT; - char **argv_copy = NULL; - int stdout_fd = -1, archiver_fd = -1; time_t now = time(NULL); struct tm tm; - struct strbuf buf = STRBUF_INIT; int res = 0; argc = parse_options(argc, argv, NULL, options, @@ -663,79 +533,11 @@ static int cmd_diagnose(int argc, const char **argv) zip_path.buf); goto diagnose_cleanup; } - stdout_fd = dup(1); - if (stdout_fd < 0) { - res = error_errno(_("could not duplicate stdout")); - goto diagnose_cleanup; - } - - archiver_fd = xopen(zip_path.buf, O_CREAT | O_WRONLY | O_TRUNC, 0666); - if (archiver_fd < 0 || dup2(archiver_fd, 1) < 0) { - res = error_errno(_("could not redirect output")); - goto diagnose_cleanup; - } - - init_zip_archiver(); - strvec_pushl(&archiver_args, "scalar-diagnose", "--format=zip", NULL); - - strbuf_reset(&buf); - strbuf_addstr(&buf, "Collecting diagnostic info\n\n"); - get_version_info(&buf, 1); - - strbuf_addf(&buf, "Enlistment root: %s\n", the_repository->worktree); - get_disk_info(&buf); - write_or_die(stdout_fd, buf.buf, buf.len); - strvec_pushf(&archiver_args, - "--add-virtual-file=diagnostics.log:%.*s", - (int)buf.len, buf.buf); - - strbuf_reset(&buf); - strbuf_addstr(&buf, "--add-virtual-file=packs-local.txt:"); - dir_file_stats(the_repository->objects->odb, &buf); - foreach_alt_odb(dir_file_stats, &buf); - strvec_push(&archiver_args, buf.buf); - - strbuf_reset(&buf); - strbuf_addstr(&buf, "--add-virtual-file=objects-local.txt:"); - loose_objs_stats(&buf, ".git/objects"); - strvec_push(&archiver_args, buf.buf); - - if ((res = add_directory_to_archiver(&archiver_args, ".git", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/hooks", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/info", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/logs", 1)) || - (res = add_directory_to_archiver(&archiver_args, ".git/objects/info", 0))) - goto diagnose_cleanup; - - strvec_pushl(&archiver_args, "--prefix=", - oid_to_hex(the_hash_algo->empty_tree), "--", NULL); - - /* `write_archive()` modifies the `argv` passed to it. Let it. */ - argv_copy = xmemdupz(archiver_args.v, - sizeof(char *) * archiver_args.nr); - res = write_archive(archiver_args.nr, (const char **)argv_copy, NULL, - the_repository, NULL, 0); - if (res) { - error(_("failed to write archive")); - goto diagnose_cleanup; - } - if (!res) - fprintf(stderr, "\n" - "Diagnostics complete.\n" - "All of the gathered info is captured in '%s'\n", - zip_path.buf); + res = create_diagnostics_archive(&zip_path); diagnose_cleanup: - if (archiver_fd >= 0) { - close(1); - dup2(stdout_fd, 1); - } - free(argv_copy); - strvec_clear(&archiver_args); strbuf_release(&zip_path); - strbuf_release(&buf); - return res; } diff --git a/diagnose.c b/diagnose.c new file mode 100644 index 00000000000..509d582f0ea --- /dev/null +++ b/diagnose.c @@ -0,0 +1,217 @@ +#include "cache.h" +#include "diagnose.h" +#include "compat/disk.h" +#include "archive.h" +#include "dir.h" +#include "help.h" +#include "strvec.h" +#include "object-store.h" +#include "packfile.h" + +static void dir_file_stats_objects(const char *full_path, size_t full_path_len, + const char *file_name, void *data) +{ + struct strbuf *buf = data; + struct stat st; + + if (!stat(full_path, &st)) + strbuf_addf(buf, "%-70s %16" PRIuMAX "\n", file_name, + (uintmax_t)st.st_size); +} + +static int dir_file_stats(struct object_directory *object_dir, void *data) +{ + struct strbuf *buf = data; + + strbuf_addf(buf, "Contents of %s:\n", object_dir->path); + + for_each_file_in_pack_dir(object_dir->path, dir_file_stats_objects, + data); + + return 0; +} + +static int count_files(char *path) +{ + DIR *dir = opendir(path); + struct dirent *e; + int count = 0; + + if (!dir) + return 0; + + while ((e = readdir(dir)) != NULL) + if (!is_dot_or_dotdot(e->d_name) && e->d_type == DT_REG) + count++; + + closedir(dir); + return count; +} + +static void loose_objs_stats(struct strbuf *buf, const char *path) +{ + DIR *dir = opendir(path); + struct dirent *e; + int count; + int total = 0; + unsigned char c; + struct strbuf count_path = STRBUF_INIT; + size_t base_path_len; + + if (!dir) + return; + + strbuf_addstr(buf, "Object directory stats for "); + strbuf_add_absolute_path(buf, path); + strbuf_addstr(buf, ":\n"); + + strbuf_add_absolute_path(&count_path, path); + strbuf_addch(&count_path, '/'); + base_path_len = count_path.len; + + while ((e = readdir(dir)) != NULL) + if (!is_dot_or_dotdot(e->d_name) && + e->d_type == DT_DIR && strlen(e->d_name) == 2 && + !hex_to_bytes(&c, e->d_name, 1)) { + strbuf_setlen(&count_path, base_path_len); + strbuf_addstr(&count_path, e->d_name); + total += (count = count_files(count_path.buf)); + strbuf_addf(buf, "%s : %7d files\n", e->d_name, count); + } + + strbuf_addf(buf, "Total: %d loose objects", total); + + strbuf_release(&count_path); + closedir(dir); +} + +static int add_directory_to_archiver(struct strvec *archiver_args, + const char *path, int recurse) +{ + int at_root = !*path; + DIR *dir; + struct dirent *e; + struct strbuf buf = STRBUF_INIT; + size_t len; + int res = 0; + + dir = opendir(at_root ? "." : path); + if (!dir) { + if (errno == ENOENT) { + warning(_("could not archive missing directory '%s'"), path); + return 0; + } + return error_errno(_("could not open directory '%s'"), path); + } + + if (!at_root) + strbuf_addf(&buf, "%s/", path); + len = buf.len; + strvec_pushf(archiver_args, "--prefix=%s", buf.buf); + + while (!res && (e = readdir(dir))) { + if (!strcmp(".", e->d_name) || !strcmp("..", e->d_name)) + continue; + + strbuf_setlen(&buf, len); + strbuf_addstr(&buf, e->d_name); + + if (e->d_type == DT_REG) + strvec_pushf(archiver_args, "--add-file=%s", buf.buf); + else if (e->d_type != DT_DIR) + warning(_("skipping '%s', which is neither file nor " + "directory"), buf.buf); + else if (recurse && + add_directory_to_archiver(archiver_args, + buf.buf, recurse) < 0) + res = -1; + } + + closedir(dir); + strbuf_release(&buf); + return res; +} + +int create_diagnostics_archive(struct strbuf *zip_path) +{ + struct strvec archiver_args = STRVEC_INIT; + char **argv_copy = NULL; + int stdout_fd = -1, archiver_fd = -1; + struct strbuf buf = STRBUF_INIT; + int res; + + stdout_fd = dup(STDOUT_FILENO); + if (stdout_fd < 0) { + res = error_errno(_("could not duplicate stdout")); + goto diagnose_cleanup; + } + + archiver_fd = xopen(zip_path->buf, O_CREAT | O_WRONLY | O_TRUNC, 0666); + if (dup2(archiver_fd, STDOUT_FILENO) < 0) { + res = error_errno(_("could not redirect output")); + goto diagnose_cleanup; + } + + init_zip_archiver(); + strvec_pushl(&archiver_args, "git-diagnose", "--format=zip", NULL); + + strbuf_reset(&buf); + strbuf_addstr(&buf, "Collecting diagnostic info\n\n"); + get_version_info(&buf, 1); + + strbuf_addf(&buf, "Repository root: %s\n", the_repository->worktree); + get_disk_info(&buf); + write_or_die(stdout_fd, buf.buf, buf.len); + strvec_pushf(&archiver_args, + "--add-virtual-file=diagnostics.log:%.*s", + (int)buf.len, buf.buf); + + strbuf_reset(&buf); + strbuf_addstr(&buf, "--add-virtual-file=packs-local.txt:"); + dir_file_stats(the_repository->objects->odb, &buf); + foreach_alt_odb(dir_file_stats, &buf); + strvec_push(&archiver_args, buf.buf); + + strbuf_reset(&buf); + strbuf_addstr(&buf, "--add-virtual-file=objects-local.txt:"); + loose_objs_stats(&buf, ".git/objects"); + strvec_push(&archiver_args, buf.buf); + + if ((res = add_directory_to_archiver(&archiver_args, ".git", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/hooks", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/info", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/logs", 1)) || + (res = add_directory_to_archiver(&archiver_args, ".git/objects/info", 0))) + goto diagnose_cleanup; + + strvec_pushl(&archiver_args, "--prefix=", + oid_to_hex(the_hash_algo->empty_tree), "--", NULL); + + /* `write_archive()` modifies the `argv` passed to it. Let it. */ + argv_copy = xmemdupz(archiver_args.v, + sizeof(char *) * archiver_args.nr); + res = write_archive(archiver_args.nr, (const char **)argv_copy, NULL, + the_repository, NULL, 0); + if (res) { + error(_("failed to write archive")); + goto diagnose_cleanup; + } + + if (!res) + fprintf(stderr, "\n" + "Diagnostics complete.\n" + "All of the gathered info is captured in '%s'\n", + zip_path->buf); + +diagnose_cleanup: + if (archiver_fd >= 0) { + dup2(stdout_fd, STDOUT_FILENO); + close(stdout_fd); + close(archiver_fd); + } + free(argv_copy); + strvec_clear(&archiver_args); + strbuf_release(&buf); + + return res; +} diff --git a/diagnose.h b/diagnose.h new file mode 100644 index 00000000000..06dca69bdac --- /dev/null +++ b/diagnose.h @@ -0,0 +1,8 @@ +#ifndef DIAGNOSE_H +#define DIAGNOSE_H + +#include "strbuf.h" + +int create_diagnostics_archive(struct strbuf *zip_path); + +#endif /* DIAGNOSE_H */ From patchwork Wed Aug 10 23:34:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941042 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47D82C25B06 for ; Wed, 10 Aug 2022 23:35:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233340AbiHJXfC (ORCPT ); Wed, 10 Aug 2022 19:35:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56220 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233160AbiHJXeq (ORCPT ); Wed, 10 Aug 2022 19:34:46 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C06E78C47A for ; Wed, 10 Aug 2022 16:34:41 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id l4so19449132wrm.13 for ; Wed, 10 Aug 2022 16:34:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=mmDpOspHbIoc2UzryC4S2CEZVo1GPtwQ9kMQcuDi3yg=; b=ab0MC1yMWMNXYnrlL5Jm/vglqGQhHjU0pqpgTbJ/0R/g3S2k0piitr37tE/Pqt9FDx BlCeqiIcnVtwHtok7F5FQAdqsJd05+vvMJJ3/fQRi09Tz0ze/DWXFCj1dikVHxc+Gh0I pjd+nPxHaUeUesRbPKtflwk4xAVjUhTRzDadBUht3JvzS+HkkWuDBLUi8VToViqZFI98 qP4uj3Zgu/AhQSWDwUHfwt3H2hvJfdIGfQokKsWMLeR7l+iOGa245r7y1kyS3kqnMtHS k6AL7potrG7U/WrNFHH2CD/MwqEm/yAfLR17adjFLFppRFW1Icfq8ppvwthR92tpH/31 AQkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=mmDpOspHbIoc2UzryC4S2CEZVo1GPtwQ9kMQcuDi3yg=; b=2UkjMVqcQApqhdBtb4VXaxmCSNX3EaQDqC5llWKaZWaQrhmsyK2jwANSjk1Mk0QL7Q nd1Dsijo9lnTnF6F2SZlPFgTZ8Q3QaInuCUmf2S4EsP0KffKT+O0tF3oyWZYVYSbyp5E bd81GDxZFSi/38q0ncIkQX/mcEb2+5jJEDUnpavMTHFqOYKeDOeWFwJZ0h8eUgdxVxb1 s4KMgfifVfzFO1oQOjib3KHySeU8+Y2WNRw5eXDxI/aN2z7cfGHJjIvGGdHTp0+09Jc7 ooIahWKIEZNV4BQ0x35GYZwBYOq8JPhcdQGcg4IlaQ80kw2c8wpeloJpTDSbii0eumgz aFVg== X-Gm-Message-State: ACgBeo1u+6I0N8zOf8y8qz1NWva9fhzgtb6Aa2C98CWdfqalbkrxb/9Q Jwi/aFVx/q7ZvEMjap/MFgJ3jqjN+J0= X-Google-Smtp-Source: AA6agR4nDJr1qAo/5b9u+cDfwzoxfj4FMNN8nnux6kEFoz281tmLDGrVVIaii8WSNYB76f+zmLMKvA== X-Received: by 2002:a05:6000:4084:b0:21e:c694:80db with SMTP id da4-20020a056000408400b0021ec69480dbmr17281811wrb.681.1660174480088; Wed, 10 Aug 2022 16:34:40 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x3-20020a05600c420300b003a54fffa809sm3582371wmh.17.2022.08.10.16.34.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:39 -0700 (PDT) Message-Id: <0a6c55696d88cde666c11cd6b5d723c9e75a3b76.1660174473.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:28 +0000 Subject: [PATCH v3 06/11] diagnose.c: add option to configure archive contents Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Update 'create_diagnostics_archive()' to take an argument 'mode'. When archiving diagnostics for a repository, 'mode' is used to selectively include/exclude information based on its value. The initial options for 'mode' are: * DIAGNOSE_NONE: do not collect any diagnostics or create an archive (no-op). * DIAGNOSE_STATS: collect basic repository metadata (Git version, repo path, filesystem available space) as well as sizing and count statistics for the repository's objects and packfiles. * DIAGNOSE_ALL: collect basic repository metadata, sizing/count statistics, and copies of the '.git', '.git/hooks', '.git/info', '.git/logs', and '.git/objects/info' directories. These modes are introduced to provide users the option to collect diagnostics without the sensitive information included in copies of '.git' dir contents. At the moment, only 'scalar diagnose' uses 'create_diagnostics_archive()' (with a hardcoded 'DIAGNOSE_ALL' mode to match existing functionality), but more callers will be introduced in subsequent patches. Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- contrib/scalar/scalar.c | 2 +- diagnose.c | 19 +++++++++++++------ diagnose.h | 9 ++++++++- 3 files changed, 22 insertions(+), 8 deletions(-) diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index 3983def760a..d538b8b8f14 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -534,7 +534,7 @@ static int cmd_diagnose(int argc, const char **argv) goto diagnose_cleanup; } - res = create_diagnostics_archive(&zip_path); + res = create_diagnostics_archive(&zip_path, DIAGNOSE_ALL); diagnose_cleanup: strbuf_release(&zip_path); diff --git a/diagnose.c b/diagnose.c index 509d582f0ea..aadc3d4b21f 100644 --- a/diagnose.c +++ b/diagnose.c @@ -132,7 +132,7 @@ static int add_directory_to_archiver(struct strvec *archiver_args, return res; } -int create_diagnostics_archive(struct strbuf *zip_path) +int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode) { struct strvec archiver_args = STRVEC_INIT; char **argv_copy = NULL; @@ -140,6 +140,11 @@ int create_diagnostics_archive(struct strbuf *zip_path) struct strbuf buf = STRBUF_INIT; int res; + if (mode == DIAGNOSE_NONE) { + res = 0; + goto diagnose_cleanup; + } + stdout_fd = dup(STDOUT_FILENO); if (stdout_fd < 0) { res = error_errno(_("could not duplicate stdout")); @@ -177,11 +182,13 @@ int create_diagnostics_archive(struct strbuf *zip_path) loose_objs_stats(&buf, ".git/objects"); strvec_push(&archiver_args, buf.buf); - if ((res = add_directory_to_archiver(&archiver_args, ".git", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/hooks", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/info", 0)) || - (res = add_directory_to_archiver(&archiver_args, ".git/logs", 1)) || - (res = add_directory_to_archiver(&archiver_args, ".git/objects/info", 0))) + /* Only include this if explicitly requested */ + if (mode == DIAGNOSE_ALL && + ((res = add_directory_to_archiver(&archiver_args, ".git", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/hooks", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/info", 0)) || + (res = add_directory_to_archiver(&archiver_args, ".git/logs", 1)) || + (res = add_directory_to_archiver(&archiver_args, ".git/objects/info", 0)))) goto diagnose_cleanup; strvec_pushl(&archiver_args, "--prefix=", diff --git a/diagnose.h b/diagnose.h index 06dca69bdac..9bb6049bf0c 100644 --- a/diagnose.h +++ b/diagnose.h @@ -2,7 +2,14 @@ #define DIAGNOSE_H #include "strbuf.h" +#include "parse-options.h" -int create_diagnostics_archive(struct strbuf *zip_path); +enum diagnose_mode { + DIAGNOSE_NONE, + DIAGNOSE_STATS, + DIAGNOSE_ALL +}; + +int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode); #endif /* DIAGNOSE_H */ From patchwork Wed Aug 10 23:34:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B596C00140 for ; Wed, 10 Aug 2022 23:35:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230514AbiHJXe7 (ORCPT ); Wed, 10 Aug 2022 19:34:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233201AbiHJXes (ORCPT ); Wed, 10 Aug 2022 19:34:48 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E2698D3F3 for ; Wed, 10 Aug 2022 16:34:42 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id z12so19479206wrs.9 for ; Wed, 10 Aug 2022 16:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=3DtPvUJ+3SPdT6od1QbO+EADYby4QM2OS5TX2k4hwJY=; b=keNW6JgQVme2ZYWBGt77Mc3R/+V1AiCiVloaEzTQmAcf8Uaqz2vDDJ3l9POetuOUCO nxAMRbdaxmpReVgQtVsWzZ+Bt1UmkzfmJpLE2Lhy6r6queEZjgjIVAvQO3kZv+0+WYOB IwKVVNER3a8C0gt05RVpNhiz3ABaH8UdEA0Srp+HsHlRHZuu7miMgvgKR/ehdSSvZct4 aStcRFf8tEkZBGNjDDZ8ysycZFCPyw27P0tSF2OMMIM0syQnth1isi2V3Ud88VycMc02 RUZhTgS+6ruJXVK/0QB8GKNgwecfX4F49d+ejN9mHuZmgox93APsORQK5OnplvDdrJgp huBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=3DtPvUJ+3SPdT6od1QbO+EADYby4QM2OS5TX2k4hwJY=; b=Mg8yIaNAJKmNgWOPoVPEhn8dMjw4FR4kK9WcM8V52r28y9WaNGyXKS5rFHYxg9MUkh Y6WolpCj9JAVjeBo4gBiiVV9xjPg3wxe24khSw5Hxcxg+1j1OyD1az28/rV+T0sVYQE+ QvocExniomuqw6E5e0WycJPolL9hsANfPpdLtiMEcFeChWN4Sc6uKR7JLz+RaGW6B2GQ JCZvZzPz88WCuXowHRZYGlaPzBCeCyEsmLHB2x1bPGKqI52WRVp0L70m4eyXwnDIcMBv W/rPLg5wm46Zm79EPEO1zCRRpo76BmanX5z0UiXUv54kq6q/ZbOsHganRZ4ICe/DB6n3 Wm/Q== X-Gm-Message-State: ACgBeo3C9BEkYGrELj3pSYn5ZoARn0oA8NZ/DQfwoSZIxp1RQxEzvYPa PoBit1EJFiz3ZEuwh/y+jtx7PQxqFNQ= X-Google-Smtp-Source: AA6agR5p1ZHAqtoH8BBGm+SgP/y5vZgPRj5cJ+/hwicQcu/TASelUnqxTISJg3eoSV76kXEiitksVw== X-Received: by 2002:a05:6000:184c:b0:223:2c8b:c43c with SMTP id c12-20020a056000184c00b002232c8bc43cmr8902700wri.16.1660174480806; Wed, 10 Aug 2022 16:34:40 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s6-20020adff806000000b00222cf973e8csm8163406wrp.69.2022.08.10.16.34.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:40 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:29 +0000 Subject: [PATCH v3 07/11] builtin/diagnose.c: create 'git diagnose' builtin MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Create a 'git diagnose' builtin to generate a standalone zip archive of repository diagnostics. The "diagnose" functionality was originally implemented for Scalar in aa5c79a331 (scalar: implement `scalar diagnose`, 2022-05-28). However, the diagnostics gathered are not specific to Scalar-cloned repositories and can be useful when diagnosing issues in any Git repository. Helped-by: Ævar Arnfjörð Bjarmason Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- .gitignore | 1 + Documentation/git-diagnose.txt | 50 +++++++++++++++++++++++++++++ Makefile | 1 + builtin.h | 1 + builtin/diagnose.c | 57 ++++++++++++++++++++++++++++++++++ git.c | 1 + t/t0092-diagnose.sh | 32 +++++++++++++++++++ 7 files changed, 143 insertions(+) create mode 100644 Documentation/git-diagnose.txt create mode 100644 builtin/diagnose.c create mode 100755 t/t0092-diagnose.sh diff --git a/.gitignore b/.gitignore index 42fd7253b44..80b530bbed2 100644 --- a/.gitignore +++ b/.gitignore @@ -53,6 +53,7 @@ /git-cvsimport /git-cvsserver /git-daemon +/git-diagnose /git-diff /git-diff-files /git-diff-index diff --git a/Documentation/git-diagnose.txt b/Documentation/git-diagnose.txt new file mode 100644 index 00000000000..ce07dd0725d --- /dev/null +++ b/Documentation/git-diagnose.txt @@ -0,0 +1,50 @@ +git-diagnose(1) +================ + +NAME +---- +git-diagnose - Generate a zip archive of diagnostic information + +SYNOPSIS +-------- +[verse] +'git diagnose' [(-o | --output-directory) ] [(-s | --suffix) ] + +DESCRIPTION +----------- +Collects detailed information about the user's machine, Git client, and +repository state and packages that information into a zip archive. The +generated archive can then, for example, be shared with the Git mailing list to +help debug an issue or serve as a reference for independent debugging. + +The following information is captured in the archive: + + * 'git version --build-options' + * The path to the repository root + * The available disk space on the filesystem + * The name and size of each packfile, including those in alternate object + stores + * The total count of loose objects, as well as counts broken down by + `.git/objects` subdirectory + +This tool differs from linkgit:git-bugreport[1] in that it collects much more +detailed information with a greater focus on reporting the size and data shape +of repository contents. + +OPTIONS +------- +-o :: +--output-directory :: + Place the resulting diagnostics archive in `` instead of the + current directory. + +-s :: +--suffix :: + Specify an alternate suffix for the diagnostics archive name, to create + a file named 'git-diagnostics-'. This should take the + form of a strftime(3) format string; the current local time will be + used. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Makefile b/Makefile index ed66cb70e5a..d34f680c065 100644 --- a/Makefile +++ b/Makefile @@ -1153,6 +1153,7 @@ BUILTIN_OBJS += builtin/credential-cache.o BUILTIN_OBJS += builtin/credential-store.o BUILTIN_OBJS += builtin/credential.o BUILTIN_OBJS += builtin/describe.o +BUILTIN_OBJS += builtin/diagnose.o BUILTIN_OBJS += builtin/diff-files.o BUILTIN_OBJS += builtin/diff-index.o BUILTIN_OBJS += builtin/diff-tree.o diff --git a/builtin.h b/builtin.h index 40e9ecc8485..8901a34d6bf 100644 --- a/builtin.h +++ b/builtin.h @@ -144,6 +144,7 @@ int cmd_credential_cache(int argc, const char **argv, const char *prefix); int cmd_credential_cache_daemon(int argc, const char **argv, const char *prefix); int cmd_credential_store(int argc, const char **argv, const char *prefix); int cmd_describe(int argc, const char **argv, const char *prefix); +int cmd_diagnose(int argc, const char **argv, const char *prefix); int cmd_diff_files(int argc, const char **argv, const char *prefix); int cmd_diff_index(int argc, const char **argv, const char *prefix); int cmd_diff(int argc, const char **argv, const char *prefix); diff --git a/builtin/diagnose.c b/builtin/diagnose.c new file mode 100644 index 00000000000..832493bba65 --- /dev/null +++ b/builtin/diagnose.c @@ -0,0 +1,57 @@ +#include "builtin.h" +#include "parse-options.h" +#include "diagnose.h" + +static const char * const diagnose_usage[] = { + N_("git diagnose [-o|--output-directory ] [-s|--suffix ]"), + NULL +}; + +int cmd_diagnose(int argc, const char **argv, const char *prefix) +{ + struct strbuf zip_path = STRBUF_INIT; + time_t now = time(NULL); + struct tm tm; + char *option_output = NULL; + char *option_suffix = "%Y-%m-%d-%H%M"; + char *prefixed_filename; + + const struct option diagnose_options[] = { + OPT_STRING('o', "output-directory", &option_output, N_("path"), + N_("specify a destination for the diagnostics archive")), + OPT_STRING('s', "suffix", &option_suffix, N_("format"), + N_("specify a strftime format suffix for the filename")), + OPT_END() + }; + + argc = parse_options(argc, argv, prefix, diagnose_options, + diagnose_usage, 0); + + /* Prepare the path to put the result */ + prefixed_filename = prefix_filename(prefix, + option_output ? option_output : ""); + strbuf_addstr(&zip_path, prefixed_filename); + strbuf_complete(&zip_path, '/'); + + strbuf_addstr(&zip_path, "git-diagnostics-"); + strbuf_addftime(&zip_path, option_suffix, localtime_r(&now, &tm), 0, 0); + strbuf_addstr(&zip_path, ".zip"); + + switch (safe_create_leading_directories(zip_path.buf)) { + case SCLD_OK: + case SCLD_EXISTS: + break; + default: + die_errno(_("could not create leading directories for '%s'"), + zip_path.buf); + } + + /* Prepare diagnostics */ + if (create_diagnostics_archive(&zip_path, DIAGNOSE_STATS)) + die_errno(_("unable to create diagnostics archive %s"), + zip_path.buf); + + free(prefixed_filename); + strbuf_release(&zip_path); + return 0; +} diff --git a/git.c b/git.c index e5d62fa5a92..0b9d8ef7677 100644 --- a/git.c +++ b/git.c @@ -522,6 +522,7 @@ static struct cmd_struct commands[] = { { "credential-cache--daemon", cmd_credential_cache_daemon }, { "credential-store", cmd_credential_store }, { "describe", cmd_describe, RUN_SETUP }, + { "diagnose", cmd_diagnose, RUN_SETUP_GENTLY }, { "diff", cmd_diff, NO_PARSEOPT }, { "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT }, { "diff-index", cmd_diff_index, RUN_SETUP | NO_PARSEOPT }, diff --git a/t/t0092-diagnose.sh b/t/t0092-diagnose.sh new file mode 100755 index 00000000000..b6923726fd7 --- /dev/null +++ b/t/t0092-diagnose.sh @@ -0,0 +1,32 @@ +#!/bin/sh + +test_description='git diagnose' + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh + +test_expect_success UNZIP 'creates diagnostics zip archive' ' + test_when_finished rm -rf report && + + git diagnose -o report -s test >out && + grep "Available space" out && + + zip_path=report/git-diagnostics-test.zip && + test_path_is_file "$zip_path" && + + # Check zipped archive content + "$GIT_UNZIP" -p "$zip_path" diagnostics.log >out && + test_file_not_empty out && + + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + "$GIT_UNZIP" -p "$zip_path" objects-local.txt >out && + grep "^Total: [0-9][0-9]*" out && + + # Should not include .git directory contents by default + ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" + grep "^Total: [0-9][0-9]*" out +' + +test_done From patchwork Wed Aug 10 23:34:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941044 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41A47C25B07 for ; Wed, 10 Aug 2022 23:35:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229488AbiHJXfH (ORCPT ); Wed, 10 Aug 2022 19:35:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233363AbiHJXe4 (ORCPT ); Wed, 10 Aug 2022 19:34:56 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9A3B86C19 for ; Wed, 10 Aug 2022 16:34:43 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id q30so19453126wra.11 for ; Wed, 10 Aug 2022 16:34:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=QCIe/iFEAPCB2qM30bEnNTvDDykjX+xy4qW2EWl808E=; b=XspUrzLNJNB89t3JTa95Q3GW0O9+tnT1Eb69NV43rePTS8REpJc6mQVryTrRqlSJxN qMf0YSOL7AP9l4Wsj+4u+L1Y6lGesPyyt10KLawJ/W07bM1zoTpDTkjUZ8PbWXyZMxM8 8A0yQ3nPjokGcrzAW86W8gidw/ua81B5xYHfrDDaJD7i4/5NY7Tj+9y3jwbJ37vD0QuH cDzzKXqOwGBcYVPqCLW0jctOL4g8fzUs0QTkyGbEHdzHPS21n4fO4bmrDUxl9jdZsEfo 42cHOAs8lYor4kYCKV/Wg/eMZggkZy2c7FMYrWJPJpQ2UwFxCo2jKlLRSy4+u6vqnOSK tHyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=QCIe/iFEAPCB2qM30bEnNTvDDykjX+xy4qW2EWl808E=; b=PKIY6cOE1cG3rI64RMtWtbkyq880omb0Pev4SrGmsw+LWDtuOcQPQLwxZGrmHrwR1p Km1APhGMA5MKi/85VODpOFDDDjZ/cJz7AexMcFLC3sPKUBUIo4KjJgmTCuUZtdF8WTnt s0xRuCUYuT3qu2NoL5jhPnihcIsm2dDJqTtek38X8y25psknu8+88bxOdgIk7IgrypD4 /G92nyQHv5lBDPIx44Vi5NRyZSxlWbsFVeKz8C0z+y9fTuIJikCKyFFUDHb1EmNFehii rNxWqxyxugEFXyVo2WvCHjWm7OsxCIOV18IPEzptzo0AO5/1/aAofTV6EuTvq/7Iybeo bEfw== X-Gm-Message-State: ACgBeo0Y4htJ/SvzTgTNhI5TVrU4hNFQqYoBLHe6nnjplfUZ4pjBUOhc EC6YP8jZt6VG3z/vFZZc8LxPZMOf8sY= X-Google-Smtp-Source: AA6agR5Ufu7LYppXTrsOLcKO09KaP79hAVPTMdvUtB8hC0oqFCy6fn+NjXEzOZ4mz0Vl7WATpzKX6Q== X-Received: by 2002:adf:d1e2:0:b0:223:611b:3f18 with SMTP id g2-20020adfd1e2000000b00223611b3f18mr7523189wrd.236.1660174481587; Wed, 10 Aug 2022 16:34:41 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l25-20020a1ced19000000b003a502c23f2asm3706292wmh.16.2022.08.10.16.34.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:41 -0700 (PDT) Message-Id: <3da0cb725c927d08dd9486286e06bdb76896f5b7.1660174473.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:30 +0000 Subject: [PATCH v3 08/11] builtin/diagnose.c: add '--mode' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Create '--mode=' option in 'git diagnose' to allow users to optionally select non-default diagnostic information to include in the output archive. Additionally, document the currently-available modes, emphasizing the importance of not sharing a '--mode=all' archive publicly due to the presence of sensitive information. Note that the option parsing callback - 'option_parse_diagnose()' - is added to 'diagnose.c' rather than 'builtin/diagnose.c' so that it may be reused in future callers configuring a diagnostics archive. Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- Documentation/git-diagnose.txt | 17 ++++++++++++++++- builtin/diagnose.c | 8 ++++++-- diagnose.c | 30 ++++++++++++++++++++++++++++++ diagnose.h | 2 ++ t/t0092-diagnose.sh | 30 +++++++++++++++++++++++++++++- 5 files changed, 83 insertions(+), 4 deletions(-) diff --git a/Documentation/git-diagnose.txt b/Documentation/git-diagnose.txt index ce07dd0725d..3ec8cc7ad72 100644 --- a/Documentation/git-diagnose.txt +++ b/Documentation/git-diagnose.txt @@ -9,6 +9,7 @@ SYNOPSIS -------- [verse] 'git diagnose' [(-o | --output-directory) ] [(-s | --suffix) ] + [--mode=] DESCRIPTION ----------- @@ -17,7 +18,7 @@ repository state and packages that information into a zip archive. The generated archive can then, for example, be shared with the Git mailing list to help debug an issue or serve as a reference for independent debugging. -The following information is captured in the archive: +By default, the following information is captured in the archive: * 'git version --build-options' * The path to the repository root @@ -27,6 +28,9 @@ The following information is captured in the archive: * The total count of loose objects, as well as counts broken down by `.git/objects` subdirectory +Additional information can be collected by selecting a different diagnostic mode +using the `--mode` option. + This tool differs from linkgit:git-bugreport[1] in that it collects much more detailed information with a greater focus on reporting the size and data shape of repository contents. @@ -45,6 +49,17 @@ OPTIONS form of a strftime(3) format string; the current local time will be used. +--mode=(stats|all):: + Specify the type of diagnostics that should be collected. The default behavior + of 'git diagnose' is equivalent to `--mode=stats`. ++ +The `--mode=all` option collects everything included in `--mode=stats`, as well +as copies of `.git`, `.git/hooks`, `.git/info`, `.git/logs`, and +`.git/objects/info` directories. This additional information may be sensitive, +as it can be used to reconstruct the full contents of the diagnosed repository. +Users should exercise caution when sharing an archive generated with +`--mode=all`. + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/diagnose.c b/builtin/diagnose.c index 832493bba65..cd260c20155 100644 --- a/builtin/diagnose.c +++ b/builtin/diagnose.c @@ -3,7 +3,7 @@ #include "diagnose.h" static const char * const diagnose_usage[] = { - N_("git diagnose [-o|--output-directory ] [-s|--suffix ]"), + N_("git diagnose [-o|--output-directory ] [-s|--suffix ] [--mode=]"), NULL }; @@ -12,6 +12,7 @@ int cmd_diagnose(int argc, const char **argv, const char *prefix) struct strbuf zip_path = STRBUF_INIT; time_t now = time(NULL); struct tm tm; + enum diagnose_mode mode = DIAGNOSE_STATS; char *option_output = NULL; char *option_suffix = "%Y-%m-%d-%H%M"; char *prefixed_filename; @@ -21,6 +22,9 @@ int cmd_diagnose(int argc, const char **argv, const char *prefix) N_("specify a destination for the diagnostics archive")), OPT_STRING('s', "suffix", &option_suffix, N_("format"), N_("specify a strftime format suffix for the filename")), + OPT_CALLBACK_F(0, "mode", &mode, N_("(stats|all)"), + N_("specify the content of the diagnostic archive"), + PARSE_OPT_NONEG, option_parse_diagnose), OPT_END() }; @@ -47,7 +51,7 @@ int cmd_diagnose(int argc, const char **argv, const char *prefix) } /* Prepare diagnostics */ - if (create_diagnostics_archive(&zip_path, DIAGNOSE_STATS)) + if (create_diagnostics_archive(&zip_path, mode)) die_errno(_("unable to create diagnostics archive %s"), zip_path.buf); diff --git a/diagnose.c b/diagnose.c index aadc3d4b21f..00c3e9438e2 100644 --- a/diagnose.c +++ b/diagnose.c @@ -8,6 +8,36 @@ #include "object-store.h" #include "packfile.h" +struct diagnose_option { + enum diagnose_mode mode; + const char *option_name; +}; + +static struct diagnose_option diagnose_options[] = { + { DIAGNOSE_STATS, "stats" }, + { DIAGNOSE_ALL, "all" }, +}; + +int option_parse_diagnose(const struct option *opt, const char *arg, int unset) +{ + int i; + enum diagnose_mode *diagnose = opt->value; + + if (!arg) { + *diagnose = unset ? DIAGNOSE_NONE : DIAGNOSE_STATS; + return 0; + } + + for (i = 0; i < ARRAY_SIZE(diagnose_options); i++) { + if (!strcmp(arg, diagnose_options[i].option_name)) { + *diagnose = diagnose_options[i].mode; + return 0; + } + } + + return error(_("invalid --%s value '%s'"), opt->long_name, arg); +} + static void dir_file_stats_objects(const char *full_path, size_t full_path_len, const char *file_name, void *data) { diff --git a/diagnose.h b/diagnose.h index 9bb6049bf0c..7a4951a7863 100644 --- a/diagnose.h +++ b/diagnose.h @@ -10,6 +10,8 @@ enum diagnose_mode { DIAGNOSE_ALL }; +int option_parse_diagnose(const struct option *opt, const char *arg, int unset); + int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode); #endif /* DIAGNOSE_H */ diff --git a/t/t0092-diagnose.sh b/t/t0092-diagnose.sh index b6923726fd7..fca9b58489c 100755 --- a/t/t0092-diagnose.sh +++ b/t/t0092-diagnose.sh @@ -26,7 +26,35 @@ test_expect_success UNZIP 'creates diagnostics zip archive' ' # Should not include .git directory contents by default ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" - grep "^Total: [0-9][0-9]*" out +' + +test_expect_success UNZIP '--mode=stats excludes .git dir contents' ' + test_when_finished rm -rf report && + + git diagnose -o report -s test --mode=stats >out && + + # Includes pack quantity/size info + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + # Does not include .git directory contents + ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" +' + +test_expect_success UNZIP '--mode=all includes .git dir contents' ' + test_when_finished rm -rf report && + + git diagnose -o report -s test --mode=all >out && + + # Includes pack quantity/size info + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + # Includes .git directory contents + "$GIT_UNZIP" -l "$zip_path" | grep ".git/" && + + "$GIT_UNZIP" -p "$zip_path" .git/HEAD >out && + test_file_not_empty out ' test_done From patchwork Wed Aug 10 23:34:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941043 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7ABE9C00140 for ; Wed, 10 Aug 2022 23:35:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233548AbiHJXfE (ORCPT ); Wed, 10 Aug 2022 19:35:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232987AbiHJXe4 (ORCPT ); Wed, 10 Aug 2022 19:34:56 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 913D98FD5C for ; Wed, 10 Aug 2022 16:34:44 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id i128-20020a1c3b86000000b003a536d58f73so1840037wma.4 for ; Wed, 10 Aug 2022 16:34:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=mARxQC56HROeHfnGhKZI2tZ091Okd3Z5usLlYR6Qcug=; b=dmXSfnrqe4My7C8LsZZ+jd4qsweTkKhTPBY7myC3Gt3OKzPdID28+dW0PKJ6ZyZqqP tDSZdnOZkBwnm0tjrL0fpJqZzHrQJGmTeDO4UhOLW/1OV3ZYsRnXJnHU0xeP+8E0JWzB 8NOMIeqaZiI8gRfVB8uAcEKWbFr4cBrS+S26oI9ODTZvN0TNqDRcVk332GwFg/LPzM8r hCtyLkbMn7+mG9MUF4Nw8oHIt+UwlujkMgnF4PuTx0JoJHUwsuUKJ62/2/h6yre85Su4 At7/zGXS867ohTykBksswlQ4VSiG0I1+P5wDbd8aF9yrn304BHveB//zn01jeZBelOwi R9eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=mARxQC56HROeHfnGhKZI2tZ091Okd3Z5usLlYR6Qcug=; b=wSPU9AJIBwhSaAThWQZBF+86Gyhs9yS1LSOgfS7RvCiBTGrhwpGUizSFFdtH+ZbTPU H/eVqNTyU0HtPgPz62rcusGEbkQIsk9UeFg+FjdmCRW8rBec7EJV1ahgt/iaq86HQXlT 3piSXhU4H35TGFlJrYjcjgeAYuQ8eJ/9q6kOuPc2gM2uutUOhhIl+f1UWNuN75qcPrxM 31zCK42Cq1YKGfxsifPsWtxJ6WFkDiFTFFdSvnTJXMumVY/5XiA8Ot9eFnND5jD2rAgA +UAW4or5lVPJsSbPPO/XlzCqYRgaVhO2LEDC+/B85H1rto9QV8dOsXl6jdmp9ztLtPN7 mjfQ== X-Gm-Message-State: ACgBeo1EoPJLo5ar1CNOb8pYzrQ/8zP7SfGKQDtNqo9HAr5iXLFte74G Dl/FYHa2K3hwn8edD7N8LkISxlzzCw0= X-Google-Smtp-Source: AA6agR5sfVRXgEVA5DyJEfUoxm0soikC2oK0+KH1XyHmOWz4Xl5Z9lW/WVOkEU5LvfuNTvm0/bqKOg== X-Received: by 2002:a1c:f603:0:b0:3a5:23ca:3e7c with SMTP id w3-20020a1cf603000000b003a523ca3e7cmr3827385wmc.38.1660174482382; Wed, 10 Aug 2022 16:34:42 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x17-20020adfec11000000b0021ee0e233d9sm17204866wrn.96.2022.08.10.16.34.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:42 -0700 (PDT) Message-Id: <1a1eb2c980635415c04d5c8d9a62bd972482d7dc.1660174473.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:31 +0000 Subject: [PATCH v3 09/11] builtin/bugreport.c: create '--diagnose' option MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Create a '--diagnose' option for 'git bugreport' to collect additional information about the repository and write it to a zipped archive. The '--diagnose' option behaves effectively as an alias for simultaneously running 'git bugreport' and 'git diagnose'. In the documentation, users are explicitly recommended to attach the diagnostics alongside a bug report to provide additional context to readers, ideally reducing some back-and-forth between reporters and those debugging the issue. Note that '--diagnose' may take an optional string arg (either 'stats' or 'all'). If specified without the arg, the behavior corresponds to running 'git diagnose' without '--mode'. As with 'git diagnose', this default is intended to help reduce unintentional leaking of sensitive information). Users can also explicitly specify '--diagnose=(stats|all)' to generate the respective archive created by 'git diagnose --mode=(stats|all)'. Suggested-by: Ævar Arnfjörð Bjarmason Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- Documentation/git-bugreport.txt | 18 +++++++++++++ builtin/bugreport.c | 27 ++++++++++++++++--- t/t0091-bugreport.sh | 48 +++++++++++++++++++++++++++++++++ 3 files changed, 90 insertions(+), 3 deletions(-) diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt index d8817bf3cec..eca726e5791 100644 --- a/Documentation/git-bugreport.txt +++ b/Documentation/git-bugreport.txt @@ -9,6 +9,7 @@ SYNOPSIS -------- [verse] 'git bugreport' [(-o | --output-directory) ] [(-s | --suffix) ] + [--diagnose[=]] DESCRIPTION ----------- @@ -31,6 +32,10 @@ The following information is captured automatically: - A list of enabled hooks - $SHELL +Additional information may be gathered into a separate zip archive using the +`--diagnose` option, and can be attached alongside the bugreport document to +provide additional context to readers. + This tool is invoked via the typical Git setup process, which means that in some cases, it might not be able to launch - for example, if a relevant config file is unreadable. In this kind of scenario, it may be helpful to manually gather @@ -49,6 +54,19 @@ OPTIONS named 'git-bugreport-'. This should take the form of a strftime(3) format string; the current local time will be used. +--no-diagnose:: +--diagnose[=]:: + Create a zip archive of supplemental information about the user's + machine, Git client, and repository state. The archive is written to the + same output directory as the bug report and is named + 'git-diagnostics-'. ++ +Without `mode` specified, the diagnostic archive will contain the default set of +statistics reported by `git diagnose`. An optional `mode` value may be specified +to change which information is included in the archive. See +linkgit:git-diagnose[1] for the list of valid values for `mode` and details +about their usage. + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/bugreport.c b/builtin/bugreport.c index 9de32bc96e7..530895be55f 100644 --- a/builtin/bugreport.c +++ b/builtin/bugreport.c @@ -5,6 +5,7 @@ #include "compat/compiler.h" #include "hook.h" #include "hook-list.h" +#include "diagnose.h" static void get_system_info(struct strbuf *sys_info) @@ -59,7 +60,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit) } static const char * const bugreport_usage[] = { - N_("git bugreport [-o|--output-directory ] [-s|--suffix ]"), + N_("git bugreport [-o|--output-directory ] [-s|--suffix ] [--diagnose[=]"), NULL }; @@ -98,16 +99,21 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix) int report = -1; time_t now = time(NULL); struct tm tm; + enum diagnose_mode diagnose = DIAGNOSE_NONE; char *option_output = NULL; char *option_suffix = "%Y-%m-%d-%H%M"; const char *user_relative_path = NULL; char *prefixed_filename; + size_t output_path_len; const struct option bugreport_options[] = { + OPT_CALLBACK_F(0, "diagnose", &diagnose, N_("mode"), + N_("create an additional zip archive of detailed diagnostics (default 'stats')"), + PARSE_OPT_OPTARG, option_parse_diagnose), OPT_STRING('o', "output-directory", &option_output, N_("path"), - N_("specify a destination for the bugreport file")), + N_("specify a destination for the bugreport file(s)")), OPT_STRING('s', "suffix", &option_suffix, N_("format"), - N_("specify a strftime format suffix for the filename")), + N_("specify a strftime format suffix for the filename(s)")), OPT_END() }; @@ -119,6 +125,7 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix) option_output ? option_output : ""); strbuf_addstr(&report_path, prefixed_filename); strbuf_complete(&report_path, '/'); + output_path_len = report_path.len; strbuf_addstr(&report_path, "git-bugreport-"); strbuf_addftime(&report_path, option_suffix, localtime_r(&now, &tm), 0, 0); @@ -133,6 +140,20 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix) report_path.buf); } + /* Prepare diagnostics, if requested */ + if (diagnose != DIAGNOSE_NONE) { + struct strbuf zip_path = STRBUF_INIT; + strbuf_add(&zip_path, report_path.buf, output_path_len); + strbuf_addstr(&zip_path, "git-diagnostics-"); + strbuf_addftime(&zip_path, option_suffix, localtime_r(&now, &tm), 0, 0); + strbuf_addstr(&zip_path, ".zip"); + + if (create_diagnostics_archive(&zip_path, diagnose)) + die_errno(_("unable to create diagnostics archive %s"), zip_path.buf); + + strbuf_release(&zip_path); + } + /* Prepare the report contents */ get_bug_template(&buffer); diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh index 08f5fe9caef..b6d2f591acd 100755 --- a/t/t0091-bugreport.sh +++ b/t/t0091-bugreport.sh @@ -78,4 +78,52 @@ test_expect_success 'indicates populated hooks' ' test_cmp expect actual ' +test_expect_success UNZIP '--diagnose creates diagnostics zip archive' ' + test_when_finished rm -rf report && + + git bugreport --diagnose -o report -s test >out && + + zip_path=report/git-diagnostics-test.zip && + grep "Available space" out && + test_path_is_file "$zip_path" && + + # Check zipped archive content + "$GIT_UNZIP" -p "$zip_path" diagnostics.log >out && + test_file_not_empty out && + + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + "$GIT_UNZIP" -p "$zip_path" objects-local.txt >out && + grep "^Total: [0-9][0-9]*" out && + + # Should not include .git directory contents by default + ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" +' + +test_expect_success UNZIP '--diagnose=stats excludes .git dir contents' ' + test_when_finished rm -rf report && + + git bugreport --diagnose=stats -o report -s test >out && + + # Includes pack quantity/size info + "$GIT_UNZIP" -p "$zip_path" packs-local.txt >out && + grep ".git/objects" out && + + # Does not include .git directory contents + ! "$GIT_UNZIP" -l "$zip_path" | grep ".git/" +' + +test_expect_success UNZIP '--diagnose=all includes .git dir contents' ' + test_when_finished rm -rf report && + + git bugreport --diagnose=all -o report -s test >out && + + # Includes .git directory contents + "$GIT_UNZIP" -l "$zip_path" | grep ".git/" && + + "$GIT_UNZIP" -p "$zip_path" .git/HEAD >out && + test_file_not_empty out +' + test_done From patchwork Wed Aug 10 23:34:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57F0AC00140 for ; Wed, 10 Aug 2022 23:35:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233414AbiHJXfJ (ORCPT ); Wed, 10 Aug 2022 19:35:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56462 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233483AbiHJXe4 (ORCPT ); Wed, 10 Aug 2022 19:34:56 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F3338FD5F for ; Wed, 10 Aug 2022 16:34:45 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id z16so19424154wrh.12 for ; Wed, 10 Aug 2022 16:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=7GCnYaiyaqpfB+KodketEyI+zYP82WFm71eckKhZD9k=; b=MuhLGFDFICHCeu9ApDjFqVT5E0QzyZv3S5FxJlf9l66qpMKW0d3HDNncPqLjhubS6C mgo+NdOpEGddZIfFcLQngoamZiG+vAjbLU6EZxuUXX5PTDOI+3zzuYiTe2CKeNsaxy9R 77rT154LuRfMRhgq1+UEz8lBejqiylWE9yqsBjVKRBHdBG2q89e/LIrN9ICx0ucPq3B8 JWyB4Jl1B67COwmgPAL3A3xws3pSxDrzsrSq/dUhCJI/BXAT0gHi0k92xIgqXhTD6+ON CAPbw8Wrk/kEqmz8mpQw0yTfYeGYflChcayhJM/m1RLfqg2KKzn3MV06C76K6tn3k0sR lXDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:fcc:content-transfer-encoding:mime-version:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=7GCnYaiyaqpfB+KodketEyI+zYP82WFm71eckKhZD9k=; b=5KSD0Avkvhw2rXbHEES4bUkgZI1HYMXZG3G3WMG3AwP5EUbwX348M2du9H1a8nCRT+ UEBdAQJs3yBV29Ro0JuOe8Ej6c/q6kF9hTijMQRGX5PfXhEWUXtpVx6LZepznHnT2FeH 0ZI4uzV4i2Z9LzT5YJXs3zxr+e2XLf2dKUvYZUUdUEnKUg4xmCTGW3zfBwzOovG5pMtd 19zgaJTLoPmQmvALnTsDxkp4LXYXwYet5TZbb317Fv7890YANj5jTpBA1h/BJnjzYERF pnBDkwDXa0i9/qjg/L1AjjNHE3l1yUTOaBuhBV2yZiOF6H319eZLkIZ6OVo1hcucji2T NhSg== X-Gm-Message-State: ACgBeo2hpCCgDgOqKKn+l3+E3cl+d/plQ/akdQb+Wl9bwTogQSMXk4eu Ip4colQY7iZCPMHjVOtCQJwKbcbNj9E= X-Google-Smtp-Source: AA6agR72P+1DsK7YzZBRUXY8EBznmptmEgHGPBS9UsaJv24W/nkvqfa0Nu0SNpWRDk1lRemBEDSQuA== X-Received: by 2002:a5d:6687:0:b0:21e:d501:1b03 with SMTP id l7-20020a5d6687000000b0021ed5011b03mr18166019wru.387.1660174483093; Wed, 10 Aug 2022 16:34:43 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p22-20020a1c7416000000b003a541358bdbsm3771481wmc.21.2022.08.10.16.34.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:42 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:32 +0000 Subject: [PATCH v3 10/11] scalar-diagnose: use 'git diagnose --mode=all' MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Replace implementation of 'scalar diagnose' with an internal invocation of 'git diagnose --mode=all'. This simplifies the implementation of 'cmd_diagnose' by making it a direct alias of 'git diagnose' and removes some code in 'scalar.c' that is duplicated in 'builtin/diagnose.c'. The simplicity of the alias also sets up a clean deprecation path for 'scalar diagnose' (in favor of 'git diagnose'), if that is desired in the future. This introduces one minor change to the output of 'scalar diagnose', which is that the prefix of the created zip archive is changed from 'scalar_' to 'git-diagnostics-'. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: Victoria Dye --- contrib/scalar/scalar.c | 28 ++++++---------------------- 1 file changed, 6 insertions(+), 22 deletions(-) diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c index d538b8b8f14..68571ce195f 100644 --- a/contrib/scalar/scalar.c +++ b/contrib/scalar/scalar.c @@ -11,7 +11,6 @@ #include "dir.h" #include "packfile.h" #include "help.h" -#include "diagnose.h" /* * Remove the deepest subdirectory in the provided path string. Path must not @@ -510,34 +509,19 @@ static int cmd_diagnose(int argc, const char **argv) N_("scalar diagnose []"), NULL }; - struct strbuf zip_path = STRBUF_INIT; - time_t now = time(NULL); - struct tm tm; + struct strbuf diagnostics_root = STRBUF_INIT; int res = 0; argc = parse_options(argc, argv, NULL, options, usage, 0); - setup_enlistment_directory(argc, argv, usage, options, &zip_path); - - strbuf_addstr(&zip_path, "/.scalarDiagnostics/scalar_"); - strbuf_addftime(&zip_path, - "%Y%m%d_%H%M%S", localtime_r(&now, &tm), 0, 0); - strbuf_addstr(&zip_path, ".zip"); - switch (safe_create_leading_directories(zip_path.buf)) { - case SCLD_EXISTS: - case SCLD_OK: - break; - default: - error_errno(_("could not create directory for '%s'"), - zip_path.buf); - goto diagnose_cleanup; - } + setup_enlistment_directory(argc, argv, usage, options, &diagnostics_root); + strbuf_addstr(&diagnostics_root, "/.scalarDiagnostics"); - res = create_diagnostics_archive(&zip_path, DIAGNOSE_ALL); + res = run_git("diagnose", "--mode=all", "-s", "%Y%m%d_%H%M%S", + "-o", diagnostics_root.buf, NULL); -diagnose_cleanup: - strbuf_release(&zip_path); + strbuf_release(&diagnostics_root); return res; } From patchwork Wed Aug 10 23:34:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12941046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F773C00140 for ; Wed, 10 Aug 2022 23:35:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233409AbiHJXfM (ORCPT ); Wed, 10 Aug 2022 19:35:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233504AbiHJXe4 (ORCPT ); Wed, 10 Aug 2022 19:34:56 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D65F78FD61 for ; Wed, 10 Aug 2022 16:34:45 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id bs25so1038186wrb.2 for ; Wed, 10 Aug 2022 16:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc; bh=RhPe/2YGJzGbswB35npBDWGmS6OQlK0k/r1JD67kQvU=; b=E1bOPZ3j/+UdimcyxovqB5oD7LnqIJP8ylGT7pfAxcwx9ny4IIAVXNQqT43PpcL5Xb Iv7ud/V9rxpFfPYXah+b7WxtncSE1jClvfawfepmSaQoQfLdUJeZZjgcMJCIWOSO6Fp+ yZ/uz+exnuZfeow2RPi9PPnHUoxbgH8j6eU3NmpjfK/3WOfULUi4e6KHXj6TbKd4sBkQ 5M+i0thXMT22YZpnb1n3FXUeqLhZkJoY0IHi5UxdEPmP2E8bbBTyANh8ZbWP8DVY3HL9 oRt3urBvqHj8yKh94iCZ639t4amEfeXUijDK3SXgurI7FXbEU+0Z7GNCUHGx9e+CuS0U G24g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc; bh=RhPe/2YGJzGbswB35npBDWGmS6OQlK0k/r1JD67kQvU=; b=d78HyzO+vhfT+o1HGrItjjzImkEP3zEagolkwdNXb/cypr8W7h3OHkz+gpSjpmtAzT bLmGsyO3TTNVsX14Et908O6dxtEa9AsdcE+LrZfWy6VsJsut3Um1O9lGpmozNwXTsu+k tZ/Wy8UcigfUcgGvh7sHv/SFBRJ0a1D+x+SWWZ1t28oTsttiz1iAy3opK9UEpC80y/VB GwohYhzkWKejN32XSwtHho0MnUcA83EUf/q1Xc8O3Bk0z6fHfMcYjC60ZHDptsS33Lyp GBf5tcOFau2H4hALWb6ja1wYQYACJtpijN4w7s0JYTZDIZnmcevpSc7oS5ufcCKIgqRR hyCA== X-Gm-Message-State: ACgBeo3xH0bbcJhObMcAXJPtoeD0fn2khuYrlt7eSpSQk4LB9/5uiirL nje437ddb+/UbSdBL8WPv0g2NFXK8jg= X-Google-Smtp-Source: AA6agR5FDv50Ic5KrHR5Uqw0OfDTR4XyerJpzrRYTbC006HR+/nCNIas5RpWNDk2r7p9tdpaUG9HDw== X-Received: by 2002:a5d:5148:0:b0:223:664f:1e7f with SMTP id u8-20020a5d5148000000b00223664f1e7fmr6483596wrt.397.1660174483847; Wed, 10 Aug 2022 16:34:43 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h2-20020a5d4fc2000000b0021e8d205705sm17986136wrw.51.2022.08.10.16.34.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 16:34:43 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 10 Aug 2022 23:34:33 +0000 Subject: [PATCH v3 11/11] scalar: update technical doc roadmap Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: derrickstolee@github.com, johannes.schindelin@gmx.de, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Update the Scalar roadmap to reflect the completion of generalizing 'scalar diagnose' into 'git diagnose' and 'git bugreport --diagnose'. Signed-off-by: Victoria Dye --- Documentation/technical/scalar.txt | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/Documentation/technical/scalar.txt b/Documentation/technical/scalar.txt index 08bc09c225a..f6353375f08 100644 --- a/Documentation/technical/scalar.txt +++ b/Documentation/technical/scalar.txt @@ -84,6 +84,9 @@ series have been accepted: - `scalar-diagnose`: The `scalar` command is taught the `diagnose` subcommand. +- `scalar-generalize-diagnose`: Move the functionality of `scalar diagnose` + into `git diagnose` and `git bugreport --diagnose`. + Roughly speaking (and subject to change), the following series are needed to "finish" this initial version of Scalar: @@ -91,12 +94,6 @@ Roughly speaking (and subject to change), the following series are needed to and implement `scalar help`. At the end of this series, Scalar should be feature-complete from the perspective of a user. -- Generalize features not specific to Scalar: In the spirit of making Scalar - configure only what is needed for large repo performance, move common - utilities into other parts of Git. Some of this will be internal-only, but one - major change will be generalizing `scalar diagnose` for use with any Git - repository. - - Move Scalar to toplevel: Move Scalar out of `contrib/` and into the root of `git`, including updates to build and install it with the rest of Git. This change will incorporate Scalar into the Git CI and test framework, as well as